2024-01-24T23:07:52.528 INFO:root:teuthology version: 0.0.1.dev251+gc856e0f 2024-01-24T23:07:52.528 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi/7530404 --owner scheduled_pdiazbou@teuthology --name pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi --description rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_mon_workunits} -- /home/teuthworker/archive/pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi/7530404/orig.config.yaml 2024-01-24T23:07:52.567 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-24T23:07:52.653 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi/7530404 branch: wip-pereman2-buffer-space description: rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_mon_workunits} email: pdiazbou@redhat.com first_in_suite: false job_id: '7530404' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-pereman2-buffer-space ceph: conf: client: debug monc: 20 debug ms: 1 global: mon client directed command retry: 5 mon election default strategy: 3 ms bind msgr2: false ms inject socket failures: 5000 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 25 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub inject crc mismatch: 0.01 mon scrub inject missing keys: 0.05 mon scrub interval: 300 paxos min: 10 paxos service trim min: 5 paxos trim min: 10 osd: bluestore block size: 96636764160 bluestore compression algorithm: snappy bluestore compression mode: aggressive bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - ScrubResult - scrub mismatch - overall HEALTH_ - \(MGR_DOWN\) - \(PG_AVAILABILITY\) - \(SLOW_OPS\) - slow request - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(MON_DOWN\) - \(POOL_APP_NOT_ENABLED\) - but it is still running - overall HEALTH_ - \(PG_ - \(MON_DOWN\) - \(AUTH_BAD_CAPS\) mon_bind_msgr2: false sha1: 663873555d45def955d1cc09730bfece2847b821 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 663873555d45def955d1cc09730bfece2847b821 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pereman2-buffer-space sha1: 663873555d45def955d1cc09730bfece2847b821 owner: scheduled_pdiazbou@teuthology priority: 200 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mon.d - mon.e - osd.0 - osd.1 - osd.2 - - mon.f - mon.g - mon.h - mon.i - mgr.x - osd.3 - osd.4 - osd.5 - client.0 seed: 5127 sha1: 663873555d45def955d1cc09730bfece2847b821 sleep_before_teardown: 0 subset: 512/20000 suite: rados suite_branch: wip-pereman2-buffer-space suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 663873555d45def955d1cc09730bfece2847b821 targets: smithi115.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCOXMkl/Ynk1TEpOh2tti2OtVJvrzbWqwmf6HW6tcfqp7Hap4hrTg3EZ+YA9lA1+CoFW/ptL+SxGxR4NazkOa4g= smithi132.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFTTYi4YovFS/twV+b1Yr9q7ijDH+XVSKkJ/d6aK6l15sGLggq98xG8YzfvY/MD3pUEpsm2Ic3m9IPm0vVoa0UU= tasks: - install: null - ceph: null - mon_thrash: revive_delay: 90 thrash_delay: 1 - workunit: clients: client.0: - mon/pool_ops.sh - mon/crush_ops.sh - mon/osd.sh - mon/caps.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: c856e0f9244dd29a50697123b480d63f16a4ed81 timestamp: 2024-01-24_11:08:23 tube: smithi user: pdiazbou verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-01-24T23:07:52.654 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa; will attempt to use it 2024-01-24T23:07:52.654 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks 2024-01-24T23:07:52.655 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-01-24T23:07:52.656 INFO:teuthology.task.internal:Checking packages... 2024-01-24T23:07:52.685 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '663873555d45def955d1cc09730bfece2847b821' 2024-01-24T23:07:52.685 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-01-24T23:07:52.685 INFO:teuthology.packaging:ref: None 2024-01-24T23:07:52.685 INFO:teuthology.packaging:tag: None 2024-01-24T23:07:52.686 INFO:teuthology.packaging:branch: wip-pereman2-buffer-space 2024-01-24T23:07:52.686 INFO:teuthology.packaging:sha1: 663873555d45def955d1cc09730bfece2847b821 2024-01-24T23:07:52.686 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-pereman2-buffer-space 2024-01-24T23:07:52.911 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-431-g66387355-1jammy 2024-01-24T23:07:52.916 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-01-24T23:07:52.923 INFO:teuthology.task.internal:no buildpackages task found 2024-01-24T23:07:52.923 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-01-24T23:07:52.930 INFO:teuthology.task.internal:Saving configuration 2024-01-24T23:07:52.946 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-01-24T23:07:52.953 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-24T23:07:52.980 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi/7530404', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-24 23:02:05.266102', 'locked_by': 'scheduled_pdiazbou@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAID+1I9t1GPpJSBCfGv+OVK1IN3FNZ0s8Iq2jxkGNjqZC'} 2024-01-24T23:07:53.002 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi/7530404', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-24 23:02:05.265124', 'locked_by': 'scheduled_pdiazbou@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGvy5dfutrisOuQU/2Ys2TRQYPbhNzlw2VVYNDx1rwBy'} 2024-01-24T23:07:53.002 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-01-24T23:07:53.009 INFO:teuthology.task.internal:roles: ubuntu@smithi115.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mon.d', 'mon.e', 'osd.0', 'osd.1', 'osd.2'] 2024-01-24T23:07:53.009 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['mon.f', 'mon.g', 'mon.h', 'mon.i', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0'] 2024-01-24T23:07:53.009 INFO:teuthology.run_tasks:Running task console_log... 2024-01-24T23:07:53.083 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2dcad13430>, signals=[15]) 2024-01-24T23:07:53.083 INFO:teuthology.run_tasks:Running task internal.connect... 2024-01-24T23:07:53.090 INFO:teuthology.task.internal:Opening connections... 2024-01-24T23:07:53.090 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi115.front.sepia.ceph.com 2024-01-24T23:07:53.092 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T23:07:53.170 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi132.front.sepia.ceph.com 2024-01-24T23:07:53.171 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T23:07:53.253 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-01-24T23:07:53.260 DEBUG:teuthology.orchestra.run.smithi115:> uname -m 2024-01-24T23:07:53.265 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64 2024-01-24T23:07:53.266 DEBUG:teuthology.orchestra.run.smithi115:> cat /etc/os-release 2024-01-24T23:07:53.312 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-01-24T23:07:53.313 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="Ubuntu" 2024-01-24T23:07:53.313 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="22.04" 2024-01-24T23:07:53.313 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-01-24T23:07:53.313 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_CODENAME=jammy 2024-01-24T23:07:53.313 INFO:teuthology.orchestra.run.smithi115.stdout:ID=ubuntu 2024-01-24T23:07:53.313 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE=debian 2024-01-24T23:07:53.313 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://www.ubuntu.com/" 2024-01-24T23:07:53.313 INFO:teuthology.orchestra.run.smithi115.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-01-24T23:07:53.313 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-01-24T23:07:53.313 INFO:teuthology.orchestra.run.smithi115.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-01-24T23:07:53.313 INFO:teuthology.orchestra.run.smithi115.stdout:UBUNTU_CODENAME=jammy 2024-01-24T23:07:53.314 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2024-01-24T23:07:53.340 DEBUG:teuthology.orchestra.run.smithi132:> uname -m 2024-01-24T23:07:53.347 INFO:teuthology.orchestra.run.smithi132.stdout:x86_64 2024-01-24T23:07:53.347 DEBUG:teuthology.orchestra.run.smithi132:> cat /etc/os-release 2024-01-24T23:07:53.396 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-01-24T23:07:53.396 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="Ubuntu" 2024-01-24T23:07:53.396 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="22.04" 2024-01-24T23:07:53.396 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-01-24T23:07:53.396 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_CODENAME=jammy 2024-01-24T23:07:53.396 INFO:teuthology.orchestra.run.smithi132.stdout:ID=ubuntu 2024-01-24T23:07:53.396 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE=debian 2024-01-24T23:07:53.396 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://www.ubuntu.com/" 2024-01-24T23:07:53.396 INFO:teuthology.orchestra.run.smithi132.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-01-24T23:07:53.396 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-01-24T23:07:53.396 INFO:teuthology.orchestra.run.smithi132.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-01-24T23:07:53.397 INFO:teuthology.orchestra.run.smithi132.stdout:UBUNTU_CODENAME=jammy 2024-01-24T23:07:53.397 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2024-01-24T23:07:53.422 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-01-24T23:07:53.432 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-01-24T23:07:53.437 INFO:teuthology.task.internal:Checking for old test directory... 2024-01-24T23:07:53.437 DEBUG:teuthology.orchestra.run.smithi115:> test '!' -e /home/ubuntu/cephtest 2024-01-24T23:07:53.440 DEBUG:teuthology.orchestra.run.smithi132:> test '!' -e /home/ubuntu/cephtest 2024-01-24T23:07:53.445 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-01-24T23:07:53.535 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-01-24T23:07:53.536 DEBUG:teuthology.orchestra.run.smithi115:> test -z $(ls -A /var/lib/ceph) 2024-01-24T23:07:53.540 DEBUG:teuthology.orchestra.run.smithi132:> test -z $(ls -A /var/lib/ceph) 2024-01-24T23:07:53.552 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-01-24T23:07:53.605 INFO:teuthology.run_tasks:Running task kernel... 2024-01-24T23:07:53.619 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-01-24T23:07:53.619 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.f': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-01-24T23:07:53.620 DEBUG:teuthology.orchestra.run.smithi115:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-24T23:07:53.620 DEBUG:teuthology.orchestra.run.smithi132:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-24T23:07:53.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T23:07:53.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T23:07:53.625 DEBUG:teuthology.orchestra.run.smithi132:> uname -r 2024-01-24T23:07:53.625 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-01-24T23:07:53.672 INFO:teuthology.orchestra.run.smithi132.stdout:5.15.0-56-generic 2024-01-24T23:07:53.672 INFO:teuthology.task.kernel:Running kernel on smithi132: 5.15.0-56-generic 2024-01-24T23:07:53.672 DEBUG:teuthology.orchestra.run.smithi132:> sudo apt-get clean 2024-01-24T23:07:53.673 INFO:teuthology.orchestra.run.smithi115.stdout:5.15.0-56-generic 2024-01-24T23:07:53.674 INFO:teuthology.task.kernel:Running kernel on smithi115: 5.15.0-56-generic 2024-01-24T23:07:53.674 DEBUG:teuthology.orchestra.run.smithi115:> sudo apt-get clean 2024-01-24T23:07:53.801 DEBUG:teuthology.orchestra.run.smithi115:> sudo apt-get update 2024-01-24T23:07:53.837 DEBUG:teuthology.orchestra.run.smithi132:> sudo apt-get update 2024-01-24T23:07:53.947 INFO:teuthology.orchestra.run.smithi115.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-01-24T23:07:53.947 INFO:teuthology.orchestra.run.smithi115.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-01-24T23:07:53.947 INFO:teuthology.orchestra.run.smithi115.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-01-24T23:07:53.980 INFO:teuthology.orchestra.run.smithi132.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-01-24T23:07:54.008 INFO:teuthology.orchestra.run.smithi132.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-01-24T23:07:54.035 INFO:teuthology.orchestra.run.smithi115.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-01-24T23:07:54.077 INFO:teuthology.orchestra.run.smithi132.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-01-24T23:07:54.093 INFO:teuthology.orchestra.run.smithi132.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-01-24T23:07:54.181 INFO:teuthology.orchestra.run.smithi115.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [386 kB] 2024-01-24T23:07:54.217 INFO:teuthology.orchestra.run.smithi132.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [555 kB] 2024-01-24T23:07:54.258 INFO:teuthology.orchestra.run.smithi115.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,067 kB] 2024-01-24T23:07:54.258 INFO:teuthology.orchestra.run.smithi115.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,295 kB] 2024-01-24T23:07:54.293 INFO:teuthology.orchestra.run.smithi132.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,295 kB] 2024-01-24T23:07:54.296 INFO:teuthology.orchestra.run.smithi115.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [202 kB] 2024-01-24T23:07:54.301 INFO:teuthology.orchestra.run.smithi115.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-01-24T23:07:54.302 INFO:teuthology.orchestra.run.smithi115.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,260 kB] 2024-01-24T23:07:54.321 INFO:teuthology.orchestra.run.smithi115.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.2 kB] 2024-01-24T23:07:54.321 INFO:teuthology.orchestra.run.smithi115.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [206 kB] 2024-01-24T23:07:54.324 INFO:teuthology.orchestra.run.smithi115.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-01-24T23:07:54.325 INFO:teuthology.orchestra.run.smithi115.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [585 kB] 2024-01-24T23:07:54.333 INFO:teuthology.orchestra.run.smithi115.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [832 kB] 2024-01-24T23:07:54.336 INFO:teuthology.orchestra.run.smithi132.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [264 kB] 2024-01-24T23:07:54.345 INFO:teuthology.orchestra.run.smithi115.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [158 kB] 2024-01-24T23:07:54.347 INFO:teuthology.orchestra.run.smithi115.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-01-24T23:07:54.347 INFO:teuthology.orchestra.run.smithi132.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-01-24T23:07:54.348 INFO:teuthology.orchestra.run.smithi132.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [34.8 kB] 2024-01-24T23:07:54.348 INFO:teuthology.orchestra.run.smithi132.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,301 kB] 2024-01-24T23:07:54.360 INFO:teuthology.orchestra.run.smithi132.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [212 kB] 2024-01-24T23:07:54.362 INFO:teuthology.orchestra.run.smithi132.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-01-24T23:07:54.363 INFO:teuthology.orchestra.run.smithi132.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [682 kB] 2024-01-24T23:07:54.381 INFO:teuthology.orchestra.run.smithi115.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-01-24T23:07:54.381 INFO:teuthology.orchestra.run.smithi115.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [555 kB] 2024-01-24T23:07:54.381 INFO:teuthology.orchestra.run.smithi115.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-01-24T23:07:54.382 INFO:teuthology.orchestra.run.smithi115.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-01-24T23:07:54.382 INFO:teuthology.orchestra.run.smithi115.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-01-24T23:07:54.389 INFO:teuthology.orchestra.run.smithi132.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,033 kB] 2024-01-24T23:07:54.391 INFO:teuthology.orchestra.run.smithi115.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [264 kB] 2024-01-24T23:07:54.397 INFO:teuthology.orchestra.run.smithi115.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-01-24T23:07:54.398 INFO:teuthology.orchestra.run.smithi115.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,301 kB] 2024-01-24T23:07:54.399 INFO:teuthology.orchestra.run.smithi132.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [232 kB] 2024-01-24T23:07:54.401 INFO:teuthology.orchestra.run.smithi132.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-01-24T23:07:54.401 INFO:teuthology.orchestra.run.smithi132.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.1 kB] 2024-01-24T23:07:54.402 INFO:teuthology.orchestra.run.smithi132.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,164 B] 2024-01-24T23:07:54.402 INFO:teuthology.orchestra.run.smithi132.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.1 kB] 2024-01-24T23:07:54.403 INFO:teuthology.orchestra.run.smithi132.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-01-24T23:07:54.409 INFO:teuthology.orchestra.run.smithi132.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [41.7 kB] 2024-01-24T23:07:54.410 INFO:teuthology.orchestra.run.smithi132.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [33.9 kB] 2024-01-24T23:07:54.422 INFO:teuthology.orchestra.run.smithi115.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [34.8 kB] 2024-01-24T23:07:54.423 INFO:teuthology.orchestra.run.smithi115.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [212 kB] 2024-01-24T23:07:54.427 INFO:teuthology.orchestra.run.smithi132.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] 2024-01-24T23:07:54.427 INFO:teuthology.orchestra.run.smithi132.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-01-24T23:07:54.427 INFO:teuthology.orchestra.run.smithi132.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [13.4 kB] 2024-01-24T23:07:54.427 INFO:teuthology.orchestra.run.smithi132.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [24.2 kB] 2024-01-24T23:07:54.427 INFO:teuthology.orchestra.run.smithi132.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-01-24T23:07:54.428 INFO:teuthology.orchestra.run.smithi115.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-01-24T23:07:54.428 INFO:teuthology.orchestra.run.smithi115.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [682 kB] 2024-01-24T23:07:54.434 INFO:teuthology.orchestra.run.smithi132.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-01-24T23:07:54.438 INFO:teuthology.orchestra.run.smithi115.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,033 kB] 2024-01-24T23:07:54.452 INFO:teuthology.orchestra.run.smithi115.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [232 kB] 2024-01-24T23:07:54.455 INFO:teuthology.orchestra.run.smithi115.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-01-24T23:07:54.456 INFO:teuthology.orchestra.run.smithi115.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,164 B] 2024-01-24T23:07:54.456 INFO:teuthology.orchestra.run.smithi115.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.1 kB] 2024-01-24T23:07:54.460 INFO:teuthology.orchestra.run.smithi115.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.1 kB] 2024-01-24T23:07:54.460 INFO:teuthology.orchestra.run.smithi115.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-01-24T23:07:54.461 INFO:teuthology.orchestra.run.smithi115.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [41.7 kB] 2024-01-24T23:07:54.461 INFO:teuthology.orchestra.run.smithi115.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [33.9 kB] 2024-01-24T23:07:54.478 INFO:teuthology.orchestra.run.smithi115.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] 2024-01-24T23:07:54.479 INFO:teuthology.orchestra.run.smithi115.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-01-24T23:07:54.479 INFO:teuthology.orchestra.run.smithi115.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [24.2 kB] 2024-01-24T23:07:54.480 INFO:teuthology.orchestra.run.smithi115.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [13.4 kB] 2024-01-24T23:07:54.480 INFO:teuthology.orchestra.run.smithi115.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-01-24T23:07:54.480 INFO:teuthology.orchestra.run.smithi115.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-01-24T23:07:54.509 INFO:teuthology.orchestra.run.smithi132.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,067 kB] 2024-01-24T23:07:54.890 INFO:teuthology.orchestra.run.smithi132.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [386 kB] 2024-01-24T23:07:54.911 INFO:teuthology.orchestra.run.smithi132.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [202 kB] 2024-01-24T23:07:54.920 INFO:teuthology.orchestra.run.smithi132.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-01-24T23:07:54.921 INFO:teuthology.orchestra.run.smithi132.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,260 kB] 2024-01-24T23:07:55.008 INFO:teuthology.orchestra.run.smithi132.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.2 kB] 2024-01-24T23:07:55.010 INFO:teuthology.orchestra.run.smithi132.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [206 kB] 2024-01-24T23:07:55.021 INFO:teuthology.orchestra.run.smithi132.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-01-24T23:07:55.021 INFO:teuthology.orchestra.run.smithi132.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [585 kB] 2024-01-24T23:07:55.046 INFO:teuthology.orchestra.run.smithi132.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [832 kB] 2024-01-24T23:07:55.234 INFO:teuthology.orchestra.run.smithi132.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [158 kB] 2024-01-24T23:07:55.626 INFO:teuthology.orchestra.run.smithi132.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-01-24T23:07:55.633 INFO:teuthology.orchestra.run.smithi132.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-01-24T23:07:55.646 INFO:teuthology.orchestra.run.smithi132.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-01-24T23:07:55.647 INFO:teuthology.orchestra.run.smithi132.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-01-24T23:07:55.650 INFO:teuthology.orchestra.run.smithi132.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-01-24T23:07:59.757 INFO:teuthology.orchestra.run.smithi115.stdout:Fetched 11.0 MB in 2s (7,062 kB/s) 2024-01-24T23:07:59.815 INFO:teuthology.orchestra.run.smithi132.stdout:Fetched 11.0 MB in 2s (6,318 kB/s) 2024-01-24T23:08:01.095 INFO:teuthology.orchestra.run.smithi132.stdout:Reading package lists... 2024-01-24T23:08:01.112 DEBUG:teuthology.orchestra.run.smithi132:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-01-24T23:08:01.125 INFO:teuthology.orchestra.run.smithi115.stdout:Reading package lists... 2024-01-24T23:08:01.143 DEBUG:teuthology.orchestra.run.smithi115:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-01-24T23:08:01.165 INFO:teuthology.orchestra.run.smithi132.stdout:Reading package lists... 2024-01-24T23:08:01.204 INFO:teuthology.orchestra.run.smithi115.stdout:Reading package lists... 2024-01-24T23:08:01.335 INFO:teuthology.orchestra.run.smithi132.stdout:Building dependency tree... 2024-01-24T23:08:01.336 INFO:teuthology.orchestra.run.smithi132.stdout:Reading state information... 2024-01-24T23:08:01.395 INFO:teuthology.orchestra.run.smithi115.stdout:Building dependency tree... 2024-01-24T23:08:01.396 INFO:teuthology.orchestra.run.smithi115.stdout:Reading state information... 2024-01-24T23:08:01.481 INFO:teuthology.orchestra.run.smithi132.stdout:The following packages were automatically installed and are no longer required: 2024-01-24T23:08:01.482 INFO:teuthology.orchestra.run.smithi132.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-01-24T23:08:01.482 INFO:teuthology.orchestra.run.smithi132.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-01-24T23:08:01.482 INFO:teuthology.orchestra.run.smithi132.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-01-24T23:08:01.482 INFO:teuthology.orchestra.run.smithi132.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-01-24T23:08:01.482 INFO:teuthology.orchestra.run.smithi132.stdout: python2.7-minimal 2024-01-24T23:08:01.482 INFO:teuthology.orchestra.run.smithi132.stdout:Use 'sudo apt autoremove' to remove them. 2024-01-24T23:08:01.483 INFO:teuthology.orchestra.run.smithi132.stdout:The following additional packages will be installed: 2024-01-24T23:08:01.483 INFO:teuthology.orchestra.run.smithi132.stdout: linux-generic linux-headers-5.15.0-91 linux-headers-5.15.0-91-generic 2024-01-24T23:08:01.483 INFO:teuthology.orchestra.run.smithi132.stdout: linux-headers-generic linux-image-5.15.0-91-generic 2024-01-24T23:08:01.484 INFO:teuthology.orchestra.run.smithi132.stdout: linux-modules-5.15.0-91-generic linux-modules-extra-5.15.0-91-generic 2024-01-24T23:08:01.485 INFO:teuthology.orchestra.run.smithi132.stdout:Suggested packages: 2024-01-24T23:08:01.485 INFO:teuthology.orchestra.run.smithi132.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-01-24T23:08:01.485 INFO:teuthology.orchestra.run.smithi132.stdout:Recommended packages: 2024-01-24T23:08:01.485 INFO:teuthology.orchestra.run.smithi132.stdout: thermald 2024-01-24T23:08:01.545 INFO:teuthology.orchestra.run.smithi132.stdout:The following NEW packages will be installed: 2024-01-24T23:08:01.546 INFO:teuthology.orchestra.run.smithi132.stdout: linux-headers-5.15.0-91 linux-headers-5.15.0-91-generic 2024-01-24T23:08:01.546 INFO:teuthology.orchestra.run.smithi132.stdout: linux-image-5.15.0-91-generic linux-modules-5.15.0-91-generic 2024-01-24T23:08:01.546 INFO:teuthology.orchestra.run.smithi132.stdout: linux-modules-extra-5.15.0-91-generic 2024-01-24T23:08:01.547 INFO:teuthology.orchestra.run.smithi132.stdout:The following packages will be upgraded: 2024-01-24T23:08:01.547 INFO:teuthology.orchestra.run.smithi132.stdout: linux-generic linux-headers-generic linux-image-generic 2024-01-24T23:08:01.560 INFO:teuthology.orchestra.run.smithi115.stdout:The following packages were automatically installed and are no longer required: 2024-01-24T23:08:01.560 INFO:teuthology.orchestra.run.smithi115.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-01-24T23:08:01.560 INFO:teuthology.orchestra.run.smithi115.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-01-24T23:08:01.561 INFO:teuthology.orchestra.run.smithi115.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-01-24T23:08:01.561 INFO:teuthology.orchestra.run.smithi115.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-01-24T23:08:01.561 INFO:teuthology.orchestra.run.smithi115.stdout: python2.7-minimal 2024-01-24T23:08:01.561 INFO:teuthology.orchestra.run.smithi115.stdout:Use 'sudo apt autoremove' to remove them. 2024-01-24T23:08:01.562 INFO:teuthology.orchestra.run.smithi115.stdout:The following additional packages will be installed: 2024-01-24T23:08:01.562 INFO:teuthology.orchestra.run.smithi115.stdout: linux-generic linux-headers-5.15.0-91 linux-headers-5.15.0-91-generic 2024-01-24T23:08:01.562 INFO:teuthology.orchestra.run.smithi115.stdout: linux-headers-generic linux-image-5.15.0-91-generic 2024-01-24T23:08:01.562 INFO:teuthology.orchestra.run.smithi115.stdout: linux-modules-5.15.0-91-generic linux-modules-extra-5.15.0-91-generic 2024-01-24T23:08:01.564 INFO:teuthology.orchestra.run.smithi115.stdout:Suggested packages: 2024-01-24T23:08:01.564 INFO:teuthology.orchestra.run.smithi115.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-01-24T23:08:01.564 INFO:teuthology.orchestra.run.smithi115.stdout:Recommended packages: 2024-01-24T23:08:01.564 INFO:teuthology.orchestra.run.smithi115.stdout: thermald 2024-01-24T23:08:01.609 INFO:teuthology.orchestra.run.smithi115.stdout:The following NEW packages will be installed: 2024-01-24T23:08:01.609 INFO:teuthology.orchestra.run.smithi115.stdout: linux-headers-5.15.0-91 linux-headers-5.15.0-91-generic 2024-01-24T23:08:01.610 INFO:teuthology.orchestra.run.smithi115.stdout: linux-image-5.15.0-91-generic linux-modules-5.15.0-91-generic 2024-01-24T23:08:01.610 INFO:teuthology.orchestra.run.smithi115.stdout: linux-modules-extra-5.15.0-91-generic 2024-01-24T23:08:01.611 INFO:teuthology.orchestra.run.smithi115.stdout:The following packages will be upgraded: 2024-01-24T23:08:01.611 INFO:teuthology.orchestra.run.smithi115.stdout: linux-generic linux-headers-generic linux-image-generic 2024-01-24T23:08:01.668 INFO:teuthology.orchestra.run.smithi115.stdout:3 upgraded, 5 newly installed, 0 to remove and 278 not upgraded. 2024-01-24T23:08:01.668 INFO:teuthology.orchestra.run.smithi115.stdout:Need to get 113 MB of archives. 2024-01-24T23:08:01.668 INFO:teuthology.orchestra.run.smithi115.stdout:After this operation, 582 MB of additional disk space will be used. 2024-01-24T23:08:01.668 INFO:teuthology.orchestra.run.smithi115.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-91-generic amd64 5.15.0-91.101 [22.5 MB] 2024-01-24T23:08:01.725 INFO:teuthology.orchestra.run.smithi132.stdout:3 upgraded, 5 newly installed, 0 to remove and 278 not upgraded. 2024-01-24T23:08:01.725 INFO:teuthology.orchestra.run.smithi132.stdout:Need to get 113 MB of archives. 2024-01-24T23:08:01.725 INFO:teuthology.orchestra.run.smithi132.stdout:After this operation, 582 MB of additional disk space will be used. 2024-01-24T23:08:01.725 INFO:teuthology.orchestra.run.smithi132.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-91-generic amd64 5.15.0-91.101 [22.5 MB] 2024-01-24T23:08:02.194 INFO:teuthology.orchestra.run.smithi115.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-91-generic amd64 5.15.0-91.101 [11.5 MB] 2024-01-24T23:08:02.400 INFO:teuthology.orchestra.run.smithi115.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-91-generic amd64 5.15.0-91.101 [63.8 MB] 2024-01-24T23:08:03.142 INFO:teuthology.orchestra.run.smithi132.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-91-generic amd64 5.15.0-91.101 [11.5 MB] 2024-01-24T23:08:03.495 INFO:teuthology.orchestra.run.smithi132.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-91-generic amd64 5.15.0-91.101 [63.8 MB] 2024-01-24T23:08:03.544 INFO:teuthology.orchestra.run.smithi115.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.91.88 [1,700 B] 2024-01-24T23:08:03.544 INFO:teuthology.orchestra.run.smithi115.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.91.88 [2,488 B] 2024-01-24T23:08:03.544 INFO:teuthology.orchestra.run.smithi115.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-91 all 5.15.0-91.101 [12.3 MB] 2024-01-24T23:08:04.001 INFO:teuthology.orchestra.run.smithi115.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-91-generic amd64 5.15.0-91.101 [2,860 kB] 2024-01-24T23:08:04.054 INFO:teuthology.orchestra.run.smithi115.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.91.88 [2,342 B] 2024-01-24T23:08:04.443 INFO:teuthology.orchestra.run.smithi115.stdout:Fetched 113 MB in 2s (46.4 MB/s) 2024-01-24T23:08:04.637 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package linux-modules-5.15.0-91-generic. 2024-01-24T23:08:05.749 INFO:teuthology.orchestra.run.smithi132.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.91.88 [1,700 B] 2024-01-24T23:08:05.749 INFO:teuthology.orchestra.run.smithi132.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.91.88 [2,488 B] 2024-01-24T23:08:05.750 INFO:teuthology.orchestra.run.smithi132.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-91 all 5.15.0-91.101 [12.3 MB] 2024-01-24T23:08:06.707 INFO:teuthology.orchestra.run.smithi132.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-91-generic amd64 5.15.0-91.101 [2,860 kB] 2024-01-24T23:08:06.725 INFO:teuthology.orchestra.run.smithi115.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-01-24T23:08:06.730 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../0-linux-modules-5.15.0-91-generic_5.15.0-91.101_amd64.deb ... 2024-01-24T23:08:06.797 INFO:teuthology.orchestra.run.smithi132.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.91.88 [2,342 B] 2024-01-24T23:08:06.834 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking linux-modules-5.15.0-91-generic (5.15.0-91.101) ... 2024-01-24T23:08:07.219 INFO:teuthology.orchestra.run.smithi132.stdout:Fetched 113 MB in 5s (21.5 MB/s) 2024-01-24T23:08:07.366 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package linux-modules-5.15.0-91-generic. 2024-01-24T23:08:09.018 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package linux-image-5.15.0-91-generic. 2024-01-24T23:08:09.039 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../1-linux-image-5.15.0-91-generic_5.15.0-91.101_amd64.deb ... 2024-01-24T23:08:09.133 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking linux-image-5.15.0-91-generic (5.15.0-91.101) ... 2024-01-24T23:08:09.374 INFO:teuthology.orchestra.run.smithi132.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-01-24T23:08:09.378 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../0-linux-modules-5.15.0-91-generic_5.15.0-91.101_amd64.deb ... 2024-01-24T23:08:09.438 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking linux-modules-5.15.0-91-generic (5.15.0-91.101) ... 2024-01-24T23:08:09.614 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-91-generic. 2024-01-24T23:08:09.634 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-91-generic_5.15.0-91.101_amd64.deb ... 2024-01-24T23:08:09.675 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking linux-modules-extra-5.15.0-91-generic (5.15.0-91.101) ... 2024-01-24T23:08:11.523 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package linux-image-5.15.0-91-generic. 2024-01-24T23:08:11.544 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../1-linux-image-5.15.0-91-generic_5.15.0-91.101_amd64.deb ... 2024-01-24T23:08:11.632 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking linux-image-5.15.0-91-generic (5.15.0-91.101) ... 2024-01-24T23:08:12.418 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-91-generic. 2024-01-24T23:08:12.439 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-91-generic_5.15.0-91.101_amd64.deb ... 2024-01-24T23:08:12.484 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking linux-modules-extra-5.15.0-91-generic (5.15.0-91.101) ... 2024-01-24T23:08:16.737 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../3-linux-generic_5.15.0.91.88_amd64.deb ... 2024-01-24T23:08:16.865 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking linux-generic (5.15.0.91.88) over (5.15.0.56.54) ... 2024-01-24T23:08:17.231 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.91.88_amd64.deb ... 2024-01-24T23:08:17.368 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking linux-image-generic (5.15.0.91.88) over (5.15.0.56.54) ... 2024-01-24T23:08:17.638 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package linux-headers-5.15.0-91. 2024-01-24T23:08:17.649 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../5-linux-headers-5.15.0-91_5.15.0-91.101_all.deb ... 2024-01-24T23:08:17.679 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking linux-headers-5.15.0-91 (5.15.0-91.101) ... 2024-01-24T23:08:18.884 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../3-linux-generic_5.15.0.91.88_amd64.deb ... 2024-01-24T23:08:19.020 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking linux-generic (5.15.0.91.88) over (5.15.0.56.54) ... 2024-01-24T23:08:19.369 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.91.88_amd64.deb ... 2024-01-24T23:08:19.498 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking linux-image-generic (5.15.0.91.88) over (5.15.0.56.54) ... 2024-01-24T23:08:19.801 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package linux-headers-5.15.0-91. 2024-01-24T23:08:19.823 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../5-linux-headers-5.15.0-91_5.15.0-91.101_all.deb ... 2024-01-24T23:08:19.859 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking linux-headers-5.15.0-91 (5.15.0-91.101) ... 2024-01-24T23:08:23.624 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package linux-headers-5.15.0-91-generic. 2024-01-24T23:08:23.660 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../6-linux-headers-5.15.0-91-generic_5.15.0-91.101_amd64.deb ... 2024-01-24T23:08:23.698 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking linux-headers-5.15.0-91-generic (5.15.0-91.101) ... 2024-01-24T23:08:25.214 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package linux-headers-5.15.0-91-generic. 2024-01-24T23:08:25.251 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../6-linux-headers-5.15.0-91-generic_5.15.0-91.101_amd64.deb ... 2024-01-24T23:08:25.288 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking linux-headers-5.15.0-91-generic (5.15.0-91.101) ... 2024-01-24T23:08:26.317 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.91.88_amd64.deb ... 2024-01-24T23:08:26.483 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking linux-headers-generic (5.15.0.91.88) over (5.15.0.56.54) ... 2024-01-24T23:08:26.825 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up linux-headers-5.15.0-91 (5.15.0-91.101) ... 2024-01-24T23:08:26.921 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up linux-headers-5.15.0-91-generic (5.15.0-91.101) ... 2024-01-24T23:08:27.080 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up linux-headers-generic (5.15.0.91.88) ... 2024-01-24T23:08:27.209 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up linux-image-5.15.0-91-generic (5.15.0-91.101) ... 2024-01-24T23:08:28.435 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.91.88_amd64.deb ... 2024-01-24T23:08:28.533 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking linux-headers-generic (5.15.0.91.88) over (5.15.0.56.54) ... 2024-01-24T23:08:28.535 INFO:teuthology.orchestra.run.smithi115.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-01-24T23:08:28.535 INFO:teuthology.orchestra.run.smithi115.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-01-24T23:08:28.535 INFO:teuthology.orchestra.run.smithi115.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-91-generic 2024-01-24T23:08:28.535 INFO:teuthology.orchestra.run.smithi115.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-91-generic 2024-01-24T23:08:28.720 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up linux-modules-extra-5.15.0-91-generic (5.15.0-91.101) ... 2024-01-24T23:08:28.849 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up linux-headers-5.15.0-91 (5.15.0-91.101) ... 2024-01-24T23:08:28.971 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up linux-headers-5.15.0-91-generic (5.15.0-91.101) ... 2024-01-24T23:08:29.098 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up linux-headers-generic (5.15.0.91.88) ... 2024-01-24T23:08:29.210 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up linux-image-5.15.0-91-generic (5.15.0-91.101) ... 2024-01-24T23:08:30.336 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up linux-image-generic (5.15.0.91.88) ... 2024-01-24T23:08:30.463 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up linux-modules-5.15.0-91-generic (5.15.0-91.101) ... 2024-01-24T23:08:30.516 INFO:teuthology.orchestra.run.smithi132.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-01-24T23:08:30.516 INFO:teuthology.orchestra.run.smithi132.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-01-24T23:08:30.516 INFO:teuthology.orchestra.run.smithi132.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-91-generic 2024-01-24T23:08:30.516 INFO:teuthology.orchestra.run.smithi132.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-91-generic 2024-01-24T23:08:30.687 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up linux-modules-extra-5.15.0-91-generic (5.15.0-91.101) ... 2024-01-24T23:08:31.847 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up linux-generic (5.15.0.91.88) ... 2024-01-24T23:08:32.010 INFO:teuthology.orchestra.run.smithi115.stdout:Processing triggers for linux-image-5.15.0-91-generic (5.15.0-91.101) ... 2024-01-24T23:08:32.073 INFO:teuthology.orchestra.run.smithi115.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-01-24T23:08:32.074 INFO:teuthology.orchestra.run.smithi115.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-91-generic 2024-01-24T23:08:32.105 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up linux-image-generic (5.15.0.91.88) ... 2024-01-24T23:08:32.230 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up linux-modules-5.15.0-91-generic (5.15.0-91.101) ... 2024-01-24T23:08:33.632 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up linux-generic (5.15.0.91.88) ... 2024-01-24T23:08:33.733 INFO:teuthology.orchestra.run.smithi132.stdout:Processing triggers for linux-image-5.15.0-91-generic (5.15.0-91.101) ... 2024-01-24T23:08:33.799 INFO:teuthology.orchestra.run.smithi132.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-01-24T23:08:33.799 INFO:teuthology.orchestra.run.smithi132.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-91-generic 2024-01-24T23:08:46.251 INFO:teuthology.orchestra.run.smithi115.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-01-24T23:08:46.251 INFO:teuthology.orchestra.run.smithi115.stdout:Sourcing file `/etc/default/grub' 2024-01-24T23:08:46.268 INFO:teuthology.orchestra.run.smithi115.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-01-24T23:08:46.283 INFO:teuthology.orchestra.run.smithi115.stdout:Generating grub configuration file ... 2024-01-24T23:08:46.578 INFO:teuthology.orchestra.run.smithi115.stdout:Found linux image: /boot/vmlinuz-5.15.0-91-generic 2024-01-24T23:08:46.598 INFO:teuthology.orchestra.run.smithi115.stdout:Found initrd image: /boot/initrd.img-5.15.0-91-generic 2024-01-24T23:08:46.894 INFO:teuthology.orchestra.run.smithi115.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-01-24T23:08:46.899 INFO:teuthology.orchestra.run.smithi115.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-01-24T23:08:46.967 INFO:teuthology.orchestra.run.smithi115.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-01-24T23:08:46.972 INFO:teuthology.orchestra.run.smithi115.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-01-24T23:08:47.161 INFO:teuthology.orchestra.run.smithi115.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-01-24T23:08:47.161 INFO:teuthology.orchestra.run.smithi115.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-01-24T23:08:47.161 INFO:teuthology.orchestra.run.smithi115.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-01-24T23:08:47.198 INFO:teuthology.orchestra.run.smithi115.stdout:done 2024-01-24T23:08:47.710 DEBUG:teuthology.orchestra.run.smithi115:> dpkg -s linux-image-generic 2024-01-24T23:08:47.733 INFO:teuthology.orchestra.run.smithi115.stdout:Package: linux-image-generic 2024-01-24T23:08:47.733 INFO:teuthology.orchestra.run.smithi115.stdout:Status: install ok installed 2024-01-24T23:08:47.733 INFO:teuthology.orchestra.run.smithi115.stdout:Priority: optional 2024-01-24T23:08:47.733 INFO:teuthology.orchestra.run.smithi115.stdout:Section: kernel 2024-01-24T23:08:47.733 INFO:teuthology.orchestra.run.smithi115.stdout:Installed-Size: 20 2024-01-24T23:08:47.733 INFO:teuthology.orchestra.run.smithi115.stdout:Maintainer: Ubuntu Kernel Team 2024-01-24T23:08:47.734 INFO:teuthology.orchestra.run.smithi115.stdout:Architecture: amd64 2024-01-24T23:08:47.734 INFO:teuthology.orchestra.run.smithi115.stdout:Source: linux-meta 2024-01-24T23:08:47.734 INFO:teuthology.orchestra.run.smithi115.stdout:Version: 5.15.0.91.88 2024-01-24T23:08:47.734 INFO:teuthology.orchestra.run.smithi115.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-91), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2024-01-24T23:08:47.734 INFO:teuthology.orchestra.run.smithi115.stdout:Depends: linux-image-5.15.0-91-generic, linux-modules-extra-5.15.0-91-generic, linux-firmware, intel-microcode, amd64-microcode 2024-01-24T23:08:47.734 INFO:teuthology.orchestra.run.smithi115.stdout:Recommends: thermald 2024-01-24T23:08:47.734 INFO:teuthology.orchestra.run.smithi115.stdout:Description: Generic Linux kernel image 2024-01-24T23:08:47.734 INFO:teuthology.orchestra.run.smithi115.stdout: This package will always depend on the latest generic kernel image 2024-01-24T23:08:47.734 INFO:teuthology.orchestra.run.smithi115.stdout: available. 2024-01-24T23:08:47.734 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-91-generic 2024-01-24T23:08:47.734 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-01-24T23:08:47.735 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-01-24T23:08:47.735 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-01-24T23:08:47.735 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi115.front.sepia.ceph.com, path=None, version=distro) 2024-01-24T23:08:47.735 DEBUG:teuthology.orchestra.run.smithi115:> sudo apt-get clean 2024-01-24T23:08:47.886 DEBUG:teuthology.orchestra.run.smithi115:> sudo apt-get update 2024-01-24T23:08:47.908 INFO:teuthology.orchestra.run.smithi132.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-01-24T23:08:47.908 INFO:teuthology.orchestra.run.smithi132.stdout:Sourcing file `/etc/default/grub' 2024-01-24T23:08:47.918 INFO:teuthology.orchestra.run.smithi132.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-01-24T23:08:47.938 INFO:teuthology.orchestra.run.smithi132.stdout:Generating grub configuration file ... 2024-01-24T23:08:48.010 INFO:teuthology.orchestra.run.smithi115.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-01-24T23:08:48.124 INFO:teuthology.orchestra.run.smithi115.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-01-24T23:08:48.176 INFO:teuthology.orchestra.run.smithi132.stdout:Found linux image: /boot/vmlinuz-5.15.0-91-generic 2024-01-24T23:08:48.200 INFO:teuthology.orchestra.run.smithi132.stdout:Found initrd image: /boot/initrd.img-5.15.0-91-generic 2024-01-24T23:08:48.207 INFO:teuthology.orchestra.run.smithi115.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-01-24T23:08:48.302 INFO:teuthology.orchestra.run.smithi115.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-01-24T23:08:48.472 INFO:teuthology.orchestra.run.smithi132.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-01-24T23:08:48.477 INFO:teuthology.orchestra.run.smithi132.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-01-24T23:08:48.551 INFO:teuthology.orchestra.run.smithi132.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-01-24T23:08:48.557 INFO:teuthology.orchestra.run.smithi132.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-01-24T23:08:48.705 INFO:teuthology.orchestra.run.smithi132.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-01-24T23:08:48.705 INFO:teuthology.orchestra.run.smithi132.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-01-24T23:08:48.705 INFO:teuthology.orchestra.run.smithi132.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-01-24T23:08:48.746 INFO:teuthology.orchestra.run.smithi132.stdout:done 2024-01-24T23:08:49.189 DEBUG:teuthology.orchestra.run.smithi132:> dpkg -s linux-image-generic 2024-01-24T23:08:49.215 INFO:teuthology.orchestra.run.smithi132.stdout:Package: linux-image-generic 2024-01-24T23:08:49.216 INFO:teuthology.orchestra.run.smithi132.stdout:Status: install ok installed 2024-01-24T23:08:49.216 INFO:teuthology.orchestra.run.smithi132.stdout:Priority: optional 2024-01-24T23:08:49.216 INFO:teuthology.orchestra.run.smithi132.stdout:Section: kernel 2024-01-24T23:08:49.216 INFO:teuthology.orchestra.run.smithi132.stdout:Installed-Size: 20 2024-01-24T23:08:49.216 INFO:teuthology.orchestra.run.smithi132.stdout:Maintainer: Ubuntu Kernel Team 2024-01-24T23:08:49.216 INFO:teuthology.orchestra.run.smithi132.stdout:Architecture: amd64 2024-01-24T23:08:49.216 INFO:teuthology.orchestra.run.smithi132.stdout:Source: linux-meta 2024-01-24T23:08:49.216 INFO:teuthology.orchestra.run.smithi132.stdout:Version: 5.15.0.91.88 2024-01-24T23:08:49.216 INFO:teuthology.orchestra.run.smithi132.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-91), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2024-01-24T23:08:49.216 INFO:teuthology.orchestra.run.smithi132.stdout:Depends: linux-image-5.15.0-91-generic, linux-modules-extra-5.15.0-91-generic, linux-firmware, intel-microcode, amd64-microcode 2024-01-24T23:08:49.216 INFO:teuthology.orchestra.run.smithi132.stdout:Recommends: thermald 2024-01-24T23:08:49.216 INFO:teuthology.orchestra.run.smithi132.stdout:Description: Generic Linux kernel image 2024-01-24T23:08:49.216 INFO:teuthology.orchestra.run.smithi132.stdout: This package will always depend on the latest generic kernel image 2024-01-24T23:08:49.217 INFO:teuthology.orchestra.run.smithi132.stdout: available. 2024-01-24T23:08:49.217 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-91-generic 2024-01-24T23:08:49.217 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-01-24T23:08:49.217 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-01-24T23:08:49.217 INFO:teuthology.task.kernel:Installing distro kernel on mon.f... 2024-01-24T23:08:49.217 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi132.front.sepia.ceph.com, path=None, version=distro) 2024-01-24T23:08:49.217 DEBUG:teuthology.orchestra.run.smithi132:> sudo apt-get clean 2024-01-24T23:08:49.371 INFO:teuthology.orchestra.run.smithi115.stdout:Reading package lists... 2024-01-24T23:08:49.389 DEBUG:teuthology.orchestra.run.smithi115:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-01-24T23:08:49.421 DEBUG:teuthology.orchestra.run.smithi132:> sudo apt-get update 2024-01-24T23:08:49.447 INFO:teuthology.orchestra.run.smithi115.stdout:Reading package lists... 2024-01-24T23:08:49.638 INFO:teuthology.orchestra.run.smithi115.stdout:Building dependency tree... 2024-01-24T23:08:49.639 INFO:teuthology.orchestra.run.smithi115.stdout:Reading state information... 2024-01-24T23:08:49.663 INFO:teuthology.orchestra.run.smithi132.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-01-24T23:08:49.663 INFO:teuthology.orchestra.run.smithi132.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-01-24T23:08:49.736 INFO:teuthology.orchestra.run.smithi132.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-01-24T23:08:49.800 INFO:teuthology.orchestra.run.smithi115.stdout:linux-image-generic is already the newest version (5.15.0.91.88). 2024-01-24T23:08:49.800 INFO:teuthology.orchestra.run.smithi115.stdout:The following packages were automatically installed and are no longer required: 2024-01-24T23:08:49.800 INFO:teuthology.orchestra.run.smithi115.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-01-24T23:08:49.800 INFO:teuthology.orchestra.run.smithi115.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-01-24T23:08:49.800 INFO:teuthology.orchestra.run.smithi115.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-01-24T23:08:49.800 INFO:teuthology.orchestra.run.smithi115.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-01-24T23:08:49.800 INFO:teuthology.orchestra.run.smithi115.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-01-24T23:08:49.801 INFO:teuthology.orchestra.run.smithi115.stdout:Use 'sudo apt autoremove' to remove them. 2024-01-24T23:08:49.818 INFO:teuthology.orchestra.run.smithi132.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-01-24T23:08:49.837 INFO:teuthology.orchestra.run.smithi115.stdout:0 upgraded, 0 newly installed, 0 to remove and 278 not upgraded. 2024-01-24T23:08:49.839 DEBUG:teuthology.orchestra.run.smithi115:> dpkg -s linux-image-generic 2024-01-24T23:08:49.854 INFO:teuthology.orchestra.run.smithi115.stdout:Package: linux-image-generic 2024-01-24T23:08:49.854 INFO:teuthology.orchestra.run.smithi115.stdout:Status: install ok installed 2024-01-24T23:08:49.854 INFO:teuthology.orchestra.run.smithi115.stdout:Priority: optional 2024-01-24T23:08:49.855 INFO:teuthology.orchestra.run.smithi115.stdout:Section: kernel 2024-01-24T23:08:49.855 INFO:teuthology.orchestra.run.smithi115.stdout:Installed-Size: 20 2024-01-24T23:08:49.855 INFO:teuthology.orchestra.run.smithi115.stdout:Maintainer: Ubuntu Kernel Team 2024-01-24T23:08:49.855 INFO:teuthology.orchestra.run.smithi115.stdout:Architecture: amd64 2024-01-24T23:08:49.855 INFO:teuthology.orchestra.run.smithi115.stdout:Source: linux-meta 2024-01-24T23:08:49.855 INFO:teuthology.orchestra.run.smithi115.stdout:Version: 5.15.0.91.88 2024-01-24T23:08:49.855 INFO:teuthology.orchestra.run.smithi115.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-91), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2024-01-24T23:08:49.855 INFO:teuthology.orchestra.run.smithi115.stdout:Depends: linux-image-5.15.0-91-generic, linux-modules-extra-5.15.0-91-generic, linux-firmware, intel-microcode, amd64-microcode 2024-01-24T23:08:49.855 INFO:teuthology.orchestra.run.smithi115.stdout:Recommends: thermald 2024-01-24T23:08:49.855 INFO:teuthology.orchestra.run.smithi115.stdout:Description: Generic Linux kernel image 2024-01-24T23:08:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: This package will always depend on the latest generic kernel image 2024-01-24T23:08:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: available. 2024-01-24T23:08:49.856 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-01-24T23:08:49.903 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.YqlbCKDveJ 2024-01-24T23:08:49.904 DEBUG:teuthology.orchestra.run.smithi115:> sudo cp /boot/grub/grub.cfg /tmp/tmp.YqlbCKDveJ 2024-01-24T23:08:49.956 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.YqlbCKDveJ 2024-01-24T23:08:50.072 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.YqlbCKDveJ is 10KB 2024-01-24T23:08:50.084 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.YqlbCKDveJ 2024-01-24T23:08:50.089 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-01-24T23:08:50.143 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-01-24T23:08:50.143 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-01-24T23:08:50.143 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-01-24T23:08:50.205 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-91-generic 2024-01-24T23:08:50.205 DEBUG:teuthology.orchestra.run.smithi115:> sudo update-grub 2024-01-24T23:08:50.952 INFO:teuthology.orchestra.run.smithi132.stdout:Reading package lists... 2024-01-24T23:08:50.971 DEBUG:teuthology.orchestra.run.smithi132:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-01-24T23:08:51.038 INFO:teuthology.orchestra.run.smithi132.stdout:Reading package lists... 2024-01-24T23:08:51.229 INFO:teuthology.orchestra.run.smithi132.stdout:Building dependency tree... 2024-01-24T23:08:51.229 INFO:teuthology.orchestra.run.smithi132.stdout:Reading state information... 2024-01-24T23:08:51.400 INFO:teuthology.orchestra.run.smithi132.stdout:linux-image-generic is already the newest version (5.15.0.91.88). 2024-01-24T23:08:51.400 INFO:teuthology.orchestra.run.smithi132.stdout:The following packages were automatically installed and are no longer required: 2024-01-24T23:08:51.400 INFO:teuthology.orchestra.run.smithi132.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-01-24T23:08:51.400 INFO:teuthology.orchestra.run.smithi132.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-01-24T23:08:51.400 INFO:teuthology.orchestra.run.smithi132.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-01-24T23:08:51.400 INFO:teuthology.orchestra.run.smithi132.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-01-24T23:08:51.400 INFO:teuthology.orchestra.run.smithi132.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-01-24T23:08:51.400 INFO:teuthology.orchestra.run.smithi132.stdout:Use 'sudo apt autoremove' to remove them. 2024-01-24T23:08:51.440 INFO:teuthology.orchestra.run.smithi132.stdout:0 upgraded, 0 newly installed, 0 to remove and 278 not upgraded. 2024-01-24T23:08:51.442 DEBUG:teuthology.orchestra.run.smithi132:> dpkg -s linux-image-generic 2024-01-24T23:08:51.453 INFO:teuthology.orchestra.run.smithi115.stderr:Sourcing file `/etc/default/grub' 2024-01-24T23:08:51.455 INFO:teuthology.orchestra.run.smithi115.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-01-24T23:08:51.457 INFO:teuthology.orchestra.run.smithi132.stdout:Package: linux-image-generic 2024-01-24T23:08:51.457 INFO:teuthology.orchestra.run.smithi132.stdout:Status: install ok installed 2024-01-24T23:08:51.457 INFO:teuthology.orchestra.run.smithi132.stdout:Priority: optional 2024-01-24T23:08:51.458 INFO:teuthology.orchestra.run.smithi132.stdout:Section: kernel 2024-01-24T23:08:51.458 INFO:teuthology.orchestra.run.smithi132.stdout:Installed-Size: 20 2024-01-24T23:08:51.458 INFO:teuthology.orchestra.run.smithi132.stdout:Maintainer: Ubuntu Kernel Team 2024-01-24T23:08:51.458 INFO:teuthology.orchestra.run.smithi132.stdout:Architecture: amd64 2024-01-24T23:08:51.458 INFO:teuthology.orchestra.run.smithi132.stdout:Source: linux-meta 2024-01-24T23:08:51.458 INFO:teuthology.orchestra.run.smithi132.stdout:Version: 5.15.0.91.88 2024-01-24T23:08:51.458 INFO:teuthology.orchestra.run.smithi132.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-91), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2024-01-24T23:08:51.458 INFO:teuthology.orchestra.run.smithi132.stdout:Depends: linux-image-5.15.0-91-generic, linux-modules-extra-5.15.0-91-generic, linux-firmware, intel-microcode, amd64-microcode 2024-01-24T23:08:51.458 INFO:teuthology.orchestra.run.smithi132.stdout:Recommends: thermald 2024-01-24T23:08:51.458 INFO:teuthology.orchestra.run.smithi132.stdout:Description: Generic Linux kernel image 2024-01-24T23:08:51.458 INFO:teuthology.orchestra.run.smithi132.stdout: This package will always depend on the latest generic kernel image 2024-01-24T23:08:51.458 INFO:teuthology.orchestra.run.smithi132.stdout: available. 2024-01-24T23:08:51.459 DEBUG:teuthology.orchestra.run.smithi132:> mktemp 2024-01-24T23:08:51.459 INFO:teuthology.orchestra.run.smithi115.stderr:Generating grub configuration file ... 2024-01-24T23:08:51.507 INFO:teuthology.orchestra.run.smithi132.stdout:/tmp/tmp.s8nQcb9UKU 2024-01-24T23:08:51.507 DEBUG:teuthology.orchestra.run.smithi132:> sudo cp /boot/grub/grub.cfg /tmp/tmp.s8nQcb9UKU 2024-01-24T23:08:51.567 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0666 /tmp/tmp.s8nQcb9UKU 2024-01-24T23:08:51.637 INFO:teuthology.orchestra.run.smithi115.stderr:Found linux image: /boot/vmlinuz-5.15.0-91-generic 2024-01-24T23:08:51.643 INFO:teuthology.orchestra.run.smithi115.stderr:Found initrd image: /boot/initrd.img-5.15.0-91-generic 2024-01-24T23:08:51.685 DEBUG:teuthology.orchestra.remote:smithi132:/tmp/tmp.s8nQcb9UKU is 10KB 2024-01-24T23:08:51.697 DEBUG:teuthology.orchestra.run.smithi132:> rm -fr /tmp/tmp.s8nQcb9UKU 2024-01-24T23:08:51.702 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-01-24T23:08:51.759 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-01-24T23:08:51.759 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-01-24T23:08:51.759 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-01-24T23:08:51.833 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-91-generic 2024-01-24T23:08:51.833 DEBUG:teuthology.orchestra.run.smithi132:> sudo update-grub 2024-01-24T23:08:51.920 INFO:teuthology.orchestra.run.smithi115.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-01-24T23:08:51.926 INFO:teuthology.orchestra.run.smithi115.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-01-24T23:08:51.969 INFO:teuthology.orchestra.run.smithi115.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-01-24T23:08:51.974 INFO:teuthology.orchestra.run.smithi115.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-01-24T23:08:52.086 INFO:teuthology.orchestra.run.smithi115.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-01-24T23:08:52.086 INFO:teuthology.orchestra.run.smithi115.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-01-24T23:08:52.086 INFO:teuthology.orchestra.run.smithi115.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-01-24T23:08:52.097 INFO:teuthology.orchestra.run.smithi115.stderr:done 2024-01-24T23:08:52.099 DEBUG:teuthology.orchestra.run.smithi115:> sudo shutdown -r now 2024-01-24T23:08:53.024 INFO:teuthology.orchestra.run.smithi132.stderr:Sourcing file `/etc/default/grub' 2024-01-24T23:08:53.026 INFO:teuthology.orchestra.run.smithi132.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-01-24T23:08:53.028 INFO:teuthology.orchestra.run.smithi132.stderr:Generating grub configuration file ... 2024-01-24T23:08:53.196 INFO:teuthology.orchestra.run.smithi132.stderr:Found linux image: /boot/vmlinuz-5.15.0-91-generic 2024-01-24T23:08:53.202 INFO:teuthology.orchestra.run.smithi132.stderr:Found initrd image: /boot/initrd.img-5.15.0-91-generic 2024-01-24T23:08:53.438 INFO:teuthology.orchestra.run.smithi132.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-01-24T23:08:53.443 INFO:teuthology.orchestra.run.smithi132.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-01-24T23:08:53.483 INFO:teuthology.orchestra.run.smithi132.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-01-24T23:08:53.488 INFO:teuthology.orchestra.run.smithi132.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-01-24T23:08:53.613 INFO:teuthology.orchestra.run.smithi132.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-01-24T23:08:53.613 INFO:teuthology.orchestra.run.smithi132.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-01-24T23:08:53.614 INFO:teuthology.orchestra.run.smithi132.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-01-24T23:08:53.630 INFO:teuthology.orchestra.run.smithi132.stderr:done 2024-01-24T23:08:53.639 DEBUG:teuthology.orchestra.run.smithi132:> sudo shutdown -r now 2024-01-24T23:09:22.128 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-01-24T23:09:22.129 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-01-24T23:09:22.129 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T23:09:23.644 INFO:teuthology.task.kernel:Checking client mon.f for new kernel version... 2024-01-24T23:09:23.644 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-01-24T23:09:23.644 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T23:09:40.540 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-01-24T23:09:42.080 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.132 2024-01-24T23:09:49.549 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-01-24T23:09:49.550 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T23:09:51.082 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-01-24T23:09:51.083 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T23:09:52.604 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-01-24T23:09:54.144 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.132 2024-01-24T23:10:04.614 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-01-24T23:10:04.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T23:10:06.146 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-01-24T23:10:06.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T23:10:09.212 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.132 2024-01-24T23:10:24.227 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-01-24T23:10:24.228 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T23:10:27.292 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.132 2024-01-24T23:10:45.310 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-01-24T23:10:45.312 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T23:10:45.510 DEBUG:teuthology.orchestra.run.smithi132:> true 2024-01-24T23:10:46.114 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-01-24T23:10:46.114 INFO:teuthology.task.kernel:Checking kernel version of mon.f, want "5.15.0-91-generic"... 2024-01-24T23:10:46.114 DEBUG:teuthology.orchestra.run.smithi132:> uname -r 2024-01-24T23:10:46.159 INFO:teuthology.orchestra.run.smithi132.stdout:5.15.0-91-generic 2024-01-24T23:10:46.159 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-91-generic vs 5.15.0-91-generic 2024-01-24T23:10:46.160 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-01-24T23:10:46.160 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-01-24T23:10:47.160 INFO:teuthology.task.kernel:Enabling kdb on mon.f... 2024-01-24T23:10:47.160 DEBUG:teuthology.orchestra.run.smithi132:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-01-24T23:10:47.248 INFO:teuthology.orchestra.run.smithi132.stdout:ttyS1 2024-01-24T23:10:47.269 DEBUG:teuthology.parallel:result is None 2024-01-24T23:11:04.617 DEBUG:teuthology.orchestra.remote:timed out 2024-01-24T23:11:19.618 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-01-24T23:11:19.619 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T23:11:19.884 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-01-24T23:11:20.471 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-01-24T23:11:20.471 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-91-generic"... 2024-01-24T23:11:20.472 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-01-24T23:11:20.520 INFO:teuthology.orchestra.run.smithi115.stdout:5.15.0-91-generic 2024-01-24T23:11:20.520 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-91-generic vs 5.15.0-91-generic 2024-01-24T23:11:20.520 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-01-24T23:11:20.520 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-01-24T23:11:21.521 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-01-24T23:11:21.521 DEBUG:teuthology.orchestra.run.smithi115:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-01-24T23:11:21.874 INFO:teuthology.orchestra.run.smithi115.stdout:ttyS1 2024-01-24T23:11:21.906 DEBUG:teuthology.parallel:result is None 2024-01-24T23:11:21.906 INFO:teuthology.run_tasks:Running task internal.base... 2024-01-24T23:11:21.913 INFO:teuthology.task.internal:Creating test directory... 2024-01-24T23:11:21.914 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-24T23:11:21.916 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-24T23:11:21.923 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-01-24T23:11:21.930 INFO:teuthology.run_tasks:Running task internal.archive... 2024-01-24T23:11:21.938 INFO:teuthology.task.internal:Creating archive directory... 2024-01-24T23:11:21.938 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-24T23:11:21.965 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-24T23:11:21.989 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-01-24T23:11:21.999 INFO:teuthology.task.internal:Enabling coredump saving... 2024-01-24T23:11:21.999 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-24T23:11:22.029 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-24T23:11:22.051 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-24T23:11:22.051 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-24T23:11:22.058 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-24T23:11:22.059 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-24T23:11:22.060 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-01-24T23:11:22.067 INFO:teuthology.task.internal:Configuring sudo... 2024-01-24T23:11:22.067 DEBUG:teuthology.orchestra.run.smithi115:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-24T23:11:22.105 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-24T23:11:22.118 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-01-24T23:11:22.128 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-01-24T23:11:22.128 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-24T23:11:22.161 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-24T23:11:22.167 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-24T23:11:22.213 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-24T23:11:22.260 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-01-24T23:11:22.261 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-24T23:11:22.324 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-24T23:11:22.330 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-24T23:11:22.375 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-01-24T23:11:22.375 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-24T23:11:22.444 DEBUG:teuthology.orchestra.run.smithi115:> sudo service rsyslog restart 2024-01-24T23:11:22.446 DEBUG:teuthology.orchestra.run.smithi132:> sudo service rsyslog restart 2024-01-24T23:11:22.518 INFO:teuthology.run_tasks:Running task internal.timer... 2024-01-24T23:11:22.526 INFO:teuthology.task.internal:Starting timer... 2024-01-24T23:11:22.526 INFO:teuthology.run_tasks:Running task pcp... 2024-01-24T23:11:22.535 INFO:teuthology.run_tasks:Running task selinux... 2024-01-24T23:11:22.543 DEBUG:teuthology.task.selinux:Excluding smithi115: OS 'ubuntu' does not support SELinux 2024-01-24T23:11:22.543 DEBUG:teuthology.task.selinux:Excluding smithi132: OS 'ubuntu' does not support SELinux 2024-01-24T23:11:22.543 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-01-24T23:11:22.544 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-01-24T23:11:22.544 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-01-24T23:11:22.544 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-01-24T23:11:22.552 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-01-24T23:11:22.569 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-01-24T23:11:22.571 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi115.front.sepia.ceph.com,smithi132.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-01-24T23:18:07.712 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi115.front.sepia.ceph.com'), Remote(name='ubuntu@smithi132.front.sepia.ceph.com')] 2024-01-24T23:18:07.713 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-01-24T23:18:07.714 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T23:18:07.796 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-01-24T23:18:07.872 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-01-24T23:18:07.872 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-01-24T23:18:07.873 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T23:18:07.953 DEBUG:teuthology.orchestra.run.smithi132:> true 2024-01-24T23:18:08.028 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-01-24T23:18:08.028 INFO:teuthology.run_tasks:Running task clock... 2024-01-24T23:18:08.038 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-01-24T23:18:08.038 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-24T23:18:08.039 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-24T23:18:08.041 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-24T23:18:08.041 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-24T23:18:08.068 INFO:teuthology.orchestra.run.smithi115.stdout:24 Jan 23:18:08 ntpd[16865]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-01-24T23:18:08.068 INFO:teuthology.orchestra.run.smithi115.stdout:24 Jan 23:18:08 ntpd[16865]: Command line: ntpd -gq 2024-01-24T23:18:08.068 INFO:teuthology.orchestra.run.smithi115.stdout:24 Jan 23:18:08 ntpd[16865]: ---------------------------------------------------- 2024-01-24T23:18:08.068 INFO:teuthology.orchestra.run.smithi115.stdout:24 Jan 23:18:08 ntpd[16865]: ntp-4 is maintained by Network Time Foundation, 2024-01-24T23:18:08.068 INFO:teuthology.orchestra.run.smithi115.stdout:24 Jan 23:18:08 ntpd[16865]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-01-24T23:18:08.068 INFO:teuthology.orchestra.run.smithi115.stdout:24 Jan 23:18:08 ntpd[16865]: corporation. Support and training for ntp-4 are 2024-01-24T23:18:08.069 INFO:teuthology.orchestra.run.smithi115.stdout:24 Jan 23:18:08 ntpd[16865]: available at https://www.nwtime.org/support 2024-01-24T23:18:08.069 INFO:teuthology.orchestra.run.smithi115.stdout:24 Jan 23:18:08 ntpd[16865]: ---------------------------------------------------- 2024-01-24T23:18:08.069 INFO:teuthology.orchestra.run.smithi115.stdout:24 Jan 23:18:08 ntpd[16865]: proto: precision = 0.042 usec (-24) 2024-01-24T23:18:08.069 INFO:teuthology.orchestra.run.smithi115.stdout:24 Jan 23:18:08 ntpd[16865]: basedate set to 2022-02-04 2024-01-24T23:18:08.069 INFO:teuthology.orchestra.run.smithi115.stdout:24 Jan 23:18:08 ntpd[16865]: gps base set to 2022-02-06 (week 2196) 2024-01-24T23:18:08.069 INFO:teuthology.orchestra.run.smithi115.stdout:24 Jan 23:18:08 ntpd[16865]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-01-24T23:18:08.070 INFO:teuthology.orchestra.run.smithi115.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-01-24T23:18:08.070 INFO:teuthology.orchestra.run.smithi115.stderr:restrict ::: KOD does nothing without LIMITED. 2024-01-24T23:18:08.071 INFO:teuthology.orchestra.run.smithi115.stdout:24 Jan 23:18:08 ntpd[16865]: restrict ::: KOD does nothing without LIMITED. 2024-01-24T23:18:08.071 INFO:teuthology.orchestra.run.smithi115.stdout:24 Jan 23:18:08 ntpd[16865]: Listen and drop on 0 v6wildcard [::]:123 2024-01-24T23:18:08.071 INFO:teuthology.orchestra.run.smithi115.stdout:24 Jan 23:18:08 ntpd[16865]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-01-24T23:18:08.071 INFO:teuthology.orchestra.run.smithi115.stdout:24 Jan 23:18:08 ntpd[16865]: Listen normally on 2 lo 127.0.0.1:123 2024-01-24T23:18:08.071 INFO:teuthology.orchestra.run.smithi115.stdout:24 Jan 23:18:08 ntpd[16865]: Listen normally on 3 enp3s0f1 172.21.15.115:123 2024-01-24T23:18:08.071 INFO:teuthology.orchestra.run.smithi115.stdout:24 Jan 23:18:08 ntpd[16865]: Listen normally on 4 lo [::1]:123 2024-01-24T23:18:08.071 INFO:teuthology.orchestra.run.smithi115.stdout:24 Jan 23:18:08 ntpd[16865]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:ce8f%5]:123 2024-01-24T23:18:08.071 INFO:teuthology.orchestra.run.smithi115.stdout:24 Jan 23:18:08 ntpd[16865]: Listening on routing socket on fd #22 for interface updates 2024-01-24T23:18:08.097 INFO:teuthology.orchestra.run.smithi132.stdout:24 Jan 23:18:08 ntpd[16850]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-01-24T23:18:08.097 INFO:teuthology.orchestra.run.smithi132.stdout:24 Jan 23:18:08 ntpd[16850]: Command line: ntpd -gq 2024-01-24T23:18:08.097 INFO:teuthology.orchestra.run.smithi132.stdout:24 Jan 23:18:08 ntpd[16850]: ---------------------------------------------------- 2024-01-24T23:18:08.097 INFO:teuthology.orchestra.run.smithi132.stdout:24 Jan 23:18:08 ntpd[16850]: ntp-4 is maintained by Network Time Foundation, 2024-01-24T23:18:08.097 INFO:teuthology.orchestra.run.smithi132.stdout:24 Jan 23:18:08 ntpd[16850]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-01-24T23:18:08.097 INFO:teuthology.orchestra.run.smithi132.stdout:24 Jan 23:18:08 ntpd[16850]: corporation. Support and training for ntp-4 are 2024-01-24T23:18:08.097 INFO:teuthology.orchestra.run.smithi132.stdout:24 Jan 23:18:08 ntpd[16850]: available at https://www.nwtime.org/support 2024-01-24T23:18:08.097 INFO:teuthology.orchestra.run.smithi132.stdout:24 Jan 23:18:08 ntpd[16850]: ---------------------------------------------------- 2024-01-24T23:18:08.097 INFO:teuthology.orchestra.run.smithi132.stdout:24 Jan 23:18:08 ntpd[16850]: proto: precision = 0.046 usec (-24) 2024-01-24T23:18:08.097 INFO:teuthology.orchestra.run.smithi132.stdout:24 Jan 23:18:08 ntpd[16850]: basedate set to 2022-02-04 2024-01-24T23:18:08.097 INFO:teuthology.orchestra.run.smithi132.stdout:24 Jan 23:18:08 ntpd[16850]: gps base set to 2022-02-06 (week 2196) 2024-01-24T23:18:08.098 INFO:teuthology.orchestra.run.smithi132.stdout:24 Jan 23:18:08 ntpd[16850]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-01-24T23:18:08.098 INFO:teuthology.orchestra.run.smithi132.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-01-24T23:18:08.098 INFO:teuthology.orchestra.run.smithi132.stderr:restrict ::: KOD does nothing without LIMITED. 2024-01-24T23:18:08.099 INFO:teuthology.orchestra.run.smithi132.stdout:24 Jan 23:18:08 ntpd[16850]: restrict ::: KOD does nothing without LIMITED. 2024-01-24T23:18:08.099 INFO:teuthology.orchestra.run.smithi132.stdout:24 Jan 23:18:08 ntpd[16850]: Listen and drop on 0 v6wildcard [::]:123 2024-01-24T23:18:08.099 INFO:teuthology.orchestra.run.smithi132.stdout:24 Jan 23:18:08 ntpd[16850]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-01-24T23:18:08.099 INFO:teuthology.orchestra.run.smithi132.stdout:24 Jan 23:18:08 ntpd[16850]: Listen normally on 2 lo 127.0.0.1:123 2024-01-24T23:18:08.099 INFO:teuthology.orchestra.run.smithi132.stdout:24 Jan 23:18:08 ntpd[16850]: Listen normally on 3 enp3s0f1 172.21.15.132:123 2024-01-24T23:18:08.099 INFO:teuthology.orchestra.run.smithi132.stdout:24 Jan 23:18:08 ntpd[16850]: Listen normally on 4 lo [::1]:123 2024-01-24T23:18:08.099 INFO:teuthology.orchestra.run.smithi132.stdout:24 Jan 23:18:08 ntpd[16850]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:773f%5]:123 2024-01-24T23:18:08.099 INFO:teuthology.orchestra.run.smithi132.stdout:24 Jan 23:18:08 ntpd[16850]: Listening on routing socket on fd #22 for interface updates 2024-01-24T23:18:09.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:09 ntpd[16865]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:09.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:09 ntpd[16865]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:09.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:09 ntpd[16865]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:09.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:09 ntpd[16865]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:09.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:09 ntpd[16865]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:09.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:09 ntpd[16865]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:09.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:09 ntpd[16865]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-24T23:18:09.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:09 ntpd[16865]: can't open /var/log/ntpstats/peerstats.20240124: Permission denied 2024-01-24T23:18:09.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:09 ntpd[16865]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:09.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:09 ntpd[16865]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:09.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:09 ntpd[16865]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-24T23:18:09.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:09 ntpd[16865]: can't open /var/log/ntpstats/peerstats.20240124: Permission denied 2024-01-24T23:18:09.097 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:09 ntpd[16850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:09.097 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:09 ntpd[16850]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:09.097 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:09 ntpd[16850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:09.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:09 ntpd[16850]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:09.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:09 ntpd[16850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:09.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:09 ntpd[16850]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:09.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:09 ntpd[16850]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-24T23:18:09.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:09 ntpd[16850]: can't open /var/log/ntpstats/peerstats.20240124: Permission denied 2024-01-24T23:18:09.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:09 ntpd[16850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:09.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:09 ntpd[16850]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:09.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:09 ntpd[16850]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-24T23:18:09.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:09 ntpd[16850]: can't open /var/log/ntpstats/peerstats.20240124: Permission denied 2024-01-24T23:18:11.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:11 ntpd[16865]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:11.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:11 ntpd[16865]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:11.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:11 ntpd[16865]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:11.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:11 ntpd[16865]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:11.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:11 ntpd[16865]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:11.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:11 ntpd[16865]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:11.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:11 ntpd[16865]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:11.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:11 ntpd[16865]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:11.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:11 ntpd[16865]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-24T23:18:11.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:11 ntpd[16865]: can't open /var/log/ntpstats/peerstats.20240124: Permission denied 2024-01-24T23:18:11.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:11 ntpd[16865]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:11.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:11 ntpd[16865]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:11.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:11 ntpd[16865]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-24T23:18:11.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:11 ntpd[16865]: can't open /var/log/ntpstats/peerstats.20240124: Permission denied 2024-01-24T23:18:11.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:11 ntpd[16850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:11.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:11 ntpd[16850]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:11.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:11 ntpd[16850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:11.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:11 ntpd[16850]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:11.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:11 ntpd[16850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:11.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:11 ntpd[16850]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:11.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:11 ntpd[16850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:11.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:11 ntpd[16850]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:11.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:11 ntpd[16850]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-24T23:18:11.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:11 ntpd[16850]: can't open /var/log/ntpstats/peerstats.20240124: Permission denied 2024-01-24T23:18:11.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:11 ntpd[16850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:11.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:11 ntpd[16850]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:11.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:11 ntpd[16850]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-24T23:18:11.099 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:11 ntpd[16850]: can't open /var/log/ntpstats/peerstats.20240124: Permission denied 2024-01-24T23:18:11.099 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:11 ntpd[16850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:11.099 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:11 ntpd[16850]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:11.099 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:11 ntpd[16850]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-24T23:18:11.099 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:11 ntpd[16850]: can't open /var/log/ntpstats/peerstats.20240124: Permission denied 2024-01-24T23:18:12.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:12 ntpd[16865]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:12.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:12 ntpd[16865]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:12.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:12 ntpd[16865]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:12.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:12 ntpd[16865]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:12.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:12 ntpd[16865]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-24T23:18:12.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:12 ntpd[16865]: can't open /var/log/ntpstats/peerstats.20240124: Permission denied 2024-01-24T23:18:13.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:13 ntpd[16865]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:13.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:13 ntpd[16865]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:13.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:13 ntpd[16865]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:13.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:13 ntpd[16865]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:13.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:13 ntpd[16865]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:13.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:13 ntpd[16865]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:13.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:13 ntpd[16865]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-24T23:18:13.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:13 ntpd[16865]: can't open /var/log/ntpstats/peerstats.20240124: Permission denied 2024-01-24T23:18:13.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:13 ntpd[16865]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:13.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:13 ntpd[16865]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:13.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:13 ntpd[16865]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-24T23:18:13.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:13 ntpd[16865]: can't open /var/log/ntpstats/peerstats.20240124: Permission denied 2024-01-24T23:18:13.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:13 ntpd[16850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:13.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:13 ntpd[16850]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:13.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:13 ntpd[16850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:13.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:13 ntpd[16850]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:13.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:13 ntpd[16850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:13.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:13 ntpd[16850]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:13.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:13 ntpd[16850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:13.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:13 ntpd[16850]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:13.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:13 ntpd[16850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:13.099 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:13 ntpd[16850]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:13.099 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:13 ntpd[16850]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-24T23:18:13.099 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:13 ntpd[16850]: can't open /var/log/ntpstats/peerstats.20240124: Permission denied 2024-01-24T23:18:13.099 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:13 ntpd[16850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:13.099 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:13 ntpd[16850]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:13.099 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:13 ntpd[16850]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-24T23:18:13.099 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:13 ntpd[16850]: can't open /var/log/ntpstats/peerstats.20240124: Permission denied 2024-01-24T23:18:13.099 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:13 ntpd[16850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:13.099 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:13 ntpd[16850]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:13.099 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:13 ntpd[16850]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-24T23:18:13.099 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:13 ntpd[16850]: can't open /var/log/ntpstats/peerstats.20240124: Permission denied 2024-01-24T23:18:14.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:14 ntpd[16865]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:14.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:14 ntpd[16865]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:14.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:14 ntpd[16865]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:14.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:14 ntpd[16865]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:14.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:14 ntpd[16865]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-24T23:18:14.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:14 ntpd[16865]: can't open /var/log/ntpstats/peerstats.20240124: Permission denied 2024-01-24T23:18:15.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:15 ntpd[16865]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:15.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:15 ntpd[16865]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:15.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:15 ntpd[16865]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:15.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:15 ntpd[16865]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:15.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:15 ntpd[16865]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:15.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:15 ntpd[16865]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:15.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:15 ntpd[16865]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-24T23:18:15.069 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:15 ntpd[16865]: can't open /var/log/ntpstats/peerstats.20240124: Permission denied 2024-01-24T23:18:15.070 INFO:teuthology.orchestra.run.smithi115.stdout:24 Jan 23:18:15 ntpd[16865]: ntpd: time slew -0.001214 s 2024-01-24T23:18:15.070 INFO:teuthology.orchestra.run.smithi115.stdout:ntpd: time slew -0.001214s 2024-01-24T23:18:15.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:15 ntpd[16865]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-01-24T23:18:15.070 INFO:teuthology.orchestra.run.smithi115.stderr:24 Jan 23:18:15 ntpd[16865]: can't open /var/log/ntpstats/loopstats.20240124: Permission denied 2024-01-24T23:18:15.097 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:15 ntpd[16850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:15.097 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:15 ntpd[16850]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:15.097 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:15 ntpd[16850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:15.097 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:15 ntpd[16850]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:15.097 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:15 ntpd[16850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:15.097 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:15 ntpd[16850]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:15.097 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:15 ntpd[16850]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-24T23:18:15.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:15 ntpd[16850]: can't open /var/log/ntpstats/rawstats.20240124: Permission denied 2024-01-24T23:18:15.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:15 ntpd[16850]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-24T23:18:15.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:15 ntpd[16850]: can't open /var/log/ntpstats/peerstats.20240124: Permission denied 2024-01-24T23:18:15.098 INFO:teuthology.orchestra.run.smithi132.stdout:24 Jan 23:18:15 ntpd[16850]: ntpd: time slew -0.001848 s 2024-01-24T23:18:15.098 INFO:teuthology.orchestra.run.smithi132.stdout:ntpd: time slew -0.001848s 2024-01-24T23:18:15.098 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:15 ntpd[16850]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-01-24T23:18:15.099 INFO:teuthology.orchestra.run.smithi132.stderr:24 Jan 23:18:15 ntpd[16850]: can't open /var/log/ntpstats/loopstats.20240124: Permission denied 2024-01-24T23:18:15.122 INFO:teuthology.orchestra.run.smithi115.stdout: remote refid st t when poll reach delay offset jitter 2024-01-24T23:18:15.122 INFO:teuthology.orchestra.run.smithi115.stdout:============================================================================== 2024-01-24T23:18:15.122 INFO:teuthology.orchestra.run.smithi115.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-01-24T23:18:15.122 INFO:teuthology.orchestra.run.smithi115.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-01-24T23:18:15.123 INFO:teuthology.orchestra.run.smithi115.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-01-24T23:18:15.123 INFO:teuthology.orchestra.run.smithi115.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-01-24T23:18:15.145 INFO:teuthology.orchestra.run.smithi132.stdout: remote refid st t when poll reach delay offset jitter 2024-01-24T23:18:15.146 INFO:teuthology.orchestra.run.smithi132.stdout:============================================================================== 2024-01-24T23:18:15.146 INFO:teuthology.orchestra.run.smithi132.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-01-24T23:18:15.146 INFO:teuthology.orchestra.run.smithi132.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-01-24T23:18:15.146 INFO:teuthology.orchestra.run.smithi132.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-01-24T23:18:15.146 INFO:teuthology.orchestra.run.smithi132.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-01-24T23:18:15.147 INFO:teuthology.run_tasks:Running task install... 2024-01-24T23:18:15.159 DEBUG:teuthology.task.install:project ceph 2024-01-24T23:18:15.159 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '663873555d45def955d1cc09730bfece2847b821'}} 2024-01-24T23:18:15.159 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '663873555d45def955d1cc09730bfece2847b821'} 2024-01-24T23:18:15.160 INFO:teuthology.task.install:Using flavor: default 2024-01-24T23:18:15.165 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-01-24T23:18:15.166 INFO:teuthology.task.install:extra packages: [] 2024-01-24T23:18:15.166 DEBUG:teuthology.orchestra.run.smithi115:> sudo apt-key list | grep Ceph 2024-01-24T23:18:15.166 DEBUG:teuthology.orchestra.run.smithi132:> sudo apt-key list | grep Ceph 2024-01-24T23:18:15.224 INFO:teuthology.orchestra.run.smithi115.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-01-24T23:18:15.248 INFO:teuthology.orchestra.run.smithi132.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-01-24T23:18:15.250 INFO:teuthology.orchestra.run.smithi115.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-01-24T23:18:15.250 INFO:teuthology.orchestra.run.smithi115.stdout:uid [ unknown] Ceph.com (release key) 2024-01-24T23:18:15.251 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-01-24T23:18:15.252 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=663873555d45def955d1cc09730bfece2847b821 2024-01-24T23:18:15.285 INFO:teuthology.orchestra.run.smithi132.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-01-24T23:18:15.285 INFO:teuthology.orchestra.run.smithi132.stdout:uid [ unknown] Ceph.com (release key) 2024-01-24T23:18:15.285 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-01-24T23:18:15.286 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=663873555d45def955d1cc09730bfece2847b821 2024-01-24T23:18:15.392 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default/ 2024-01-24T23:18:15.392 INFO:teuthology.task.install.deb:Package version is 19.0.0-431-g66387355-1jammy 2024-01-24T23:18:15.425 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default/ 2024-01-24T23:18:15.426 INFO:teuthology.task.install.deb:Package version is 19.0.0-431-g66387355-1jammy 2024-01-24T23:18:15.503 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-01-24T23:18:15.504 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-01-24T23:18:15.518 DEBUG:teuthology.orchestra.run.smithi115:> sudo apt-get update 2024-01-24T23:18:15.533 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-01-24T23:18:15.533 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-01-24T23:18:15.547 DEBUG:teuthology.orchestra.run.smithi132:> sudo apt-get update 2024-01-24T23:18:15.716 INFO:teuthology.orchestra.run.smithi132.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-01-24T23:18:15.717 INFO:teuthology.orchestra.run.smithi132.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-01-24T23:18:15.731 INFO:teuthology.orchestra.run.smithi132.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-01-24T23:18:15.753 INFO:teuthology.orchestra.run.smithi132.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-01-24T23:18:15.813 INFO:teuthology.orchestra.run.smithi115.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-01-24T23:18:15.813 INFO:teuthology.orchestra.run.smithi115.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-01-24T23:18:15.813 INFO:teuthology.orchestra.run.smithi115.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy InRelease 2024-01-24T23:18:15.831 INFO:teuthology.orchestra.run.smithi132.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy InRelease 2024-01-24T23:18:15.849 INFO:teuthology.orchestra.run.smithi115.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy Release [6,978 B] 2024-01-24T23:18:15.877 INFO:teuthology.orchestra.run.smithi132.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy Release [6,978 B] 2024-01-24T23:18:15.885 INFO:teuthology.orchestra.run.smithi115.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-01-24T23:18:15.892 INFO:teuthology.orchestra.run.smithi115.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy Release.gpg 2024-01-24T23:18:15.914 INFO:teuthology.orchestra.run.smithi132.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy Release.gpg 2024-01-24T23:18:15.933 INFO:teuthology.orchestra.run.smithi115.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,394 B] 2024-01-24T23:18:15.953 INFO:teuthology.orchestra.run.smithi132.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,394 B] 2024-01-24T23:18:15.972 INFO:teuthology.orchestra.run.smithi115.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-01-24T23:18:15.977 INFO:teuthology.orchestra.run.smithi115.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.7 kB] 2024-01-24T23:18:15.995 INFO:teuthology.orchestra.run.smithi132.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.7 kB] 2024-01-24T23:18:16.109 INFO:teuthology.orchestra.run.smithi132.stdout:Fetched 28.1 kB in 0s (72.7 kB/s) 2024-01-24T23:18:16.180 INFO:teuthology.orchestra.run.smithi115.stdout:Fetched 28.1 kB in 0s (58.6 kB/s) 2024-01-24T23:18:16.982 INFO:teuthology.orchestra.run.smithi132.stdout:Reading package lists... 2024-01-24T23:18:16.997 DEBUG:teuthology.orchestra.run.smithi132:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-431-g66387355-1jammy cephadm=19.0.0-431-g66387355-1jammy ceph-mds=19.0.0-431-g66387355-1jammy ceph-mgr=19.0.0-431-g66387355-1jammy ceph-common=19.0.0-431-g66387355-1jammy ceph-fuse=19.0.0-431-g66387355-1jammy ceph-test=19.0.0-431-g66387355-1jammy ceph-volume=19.0.0-431-g66387355-1jammy radosgw=19.0.0-431-g66387355-1jammy python3-rados=19.0.0-431-g66387355-1jammy python3-rgw=19.0.0-431-g66387355-1jammy python3-cephfs=19.0.0-431-g66387355-1jammy python3-rbd=19.0.0-431-g66387355-1jammy libcephfs2=19.0.0-431-g66387355-1jammy libcephfs-dev=19.0.0-431-g66387355-1jammy librados2=19.0.0-431-g66387355-1jammy librbd1=19.0.0-431-g66387355-1jammy rbd-fuse=19.0.0-431-g66387355-1jammy 2024-01-24T23:18:17.057 INFO:teuthology.orchestra.run.smithi132.stdout:Reading package lists... 2024-01-24T23:18:17.117 INFO:teuthology.orchestra.run.smithi115.stdout:Reading package lists... 2024-01-24T23:18:17.131 DEBUG:teuthology.orchestra.run.smithi115:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-431-g66387355-1jammy cephadm=19.0.0-431-g66387355-1jammy ceph-mds=19.0.0-431-g66387355-1jammy ceph-mgr=19.0.0-431-g66387355-1jammy ceph-common=19.0.0-431-g66387355-1jammy ceph-fuse=19.0.0-431-g66387355-1jammy ceph-test=19.0.0-431-g66387355-1jammy ceph-volume=19.0.0-431-g66387355-1jammy radosgw=19.0.0-431-g66387355-1jammy python3-rados=19.0.0-431-g66387355-1jammy python3-rgw=19.0.0-431-g66387355-1jammy python3-cephfs=19.0.0-431-g66387355-1jammy python3-rbd=19.0.0-431-g66387355-1jammy libcephfs2=19.0.0-431-g66387355-1jammy libcephfs-dev=19.0.0-431-g66387355-1jammy librados2=19.0.0-431-g66387355-1jammy librbd1=19.0.0-431-g66387355-1jammy rbd-fuse=19.0.0-431-g66387355-1jammy 2024-01-24T23:18:17.181 INFO:teuthology.orchestra.run.smithi115.stdout:Reading package lists... 2024-01-24T23:18:17.246 INFO:teuthology.orchestra.run.smithi132.stdout:Building dependency tree... 2024-01-24T23:18:17.246 INFO:teuthology.orchestra.run.smithi132.stdout:Reading state information... 2024-01-24T23:18:17.348 INFO:teuthology.orchestra.run.smithi115.stdout:Building dependency tree... 2024-01-24T23:18:17.349 INFO:teuthology.orchestra.run.smithi115.stdout:Reading state information... 2024-01-24T23:18:17.421 INFO:teuthology.orchestra.run.smithi132.stdout:The following packages were automatically installed and are no longer required: 2024-01-24T23:18:17.421 INFO:teuthology.orchestra.run.smithi132.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-01-24T23:18:17.421 INFO:teuthology.orchestra.run.smithi132.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-01-24T23:18:17.421 INFO:teuthology.orchestra.run.smithi132.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-01-24T23:18:17.422 INFO:teuthology.orchestra.run.smithi132.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-01-24T23:18:17.422 INFO:teuthology.orchestra.run.smithi132.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-01-24T23:18:17.422 INFO:teuthology.orchestra.run.smithi132.stdout:Use 'sudo apt autoremove' to remove them. 2024-01-24T23:18:17.422 INFO:teuthology.orchestra.run.smithi132.stdout:The following additional packages will be installed: 2024-01-24T23:18:17.423 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-01-24T23:18:17.423 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-01-24T23:18:17.423 INFO:teuthology.orchestra.run.smithi132.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-01-24T23:18:17.423 INFO:teuthology.orchestra.run.smithi132.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-01-24T23:18:17.424 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-01-24T23:18:17.424 INFO:teuthology.orchestra.run.smithi132.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-01-24T23:18:17.424 INFO:teuthology.orchestra.run.smithi132.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-01-24T23:18:17.425 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-01-24T23:18:17.425 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-01-24T23:18:17.425 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-01-24T23:18:17.425 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-01-24T23:18:17.425 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-01-24T23:18:17.425 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-01-24T23:18:17.425 INFO:teuthology.orchestra.run.smithi132.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-01-24T23:18:17.425 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-01-24T23:18:17.425 INFO:teuthology.orchestra.run.smithi132.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-01-24T23:18:17.425 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-01-24T23:18:17.425 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-01-24T23:18:17.425 INFO:teuthology.orchestra.run.smithi132.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-01-24T23:18:17.425 INFO:teuthology.orchestra.run.smithi132.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-01-24T23:18:17.425 INFO:teuthology.orchestra.run.smithi132.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-01-24T23:18:17.425 INFO:teuthology.orchestra.run.smithi132.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-01-24T23:18:17.426 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-01-24T23:18:17.426 INFO:teuthology.orchestra.run.smithi132.stdout: xmlstarlet zip 2024-01-24T23:18:17.426 INFO:teuthology.orchestra.run.smithi132.stdout:Suggested packages: 2024-01-24T23:18:17.426 INFO:teuthology.orchestra.run.smithi132.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-01-24T23:18:17.426 INFO:teuthology.orchestra.run.smithi132.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-01-24T23:18:17.426 INFO:teuthology.orchestra.run.smithi132.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-01-24T23:18:17.426 INFO:teuthology.orchestra.run.smithi132.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-01-24T23:18:17.426 INFO:teuthology.orchestra.run.smithi132.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-01-24T23:18:17.426 INFO:teuthology.orchestra.run.smithi132.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-01-24T23:18:17.426 INFO:teuthology.orchestra.run.smithi132.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-01-24T23:18:17.427 INFO:teuthology.orchestra.run.smithi132.stdout:Recommended packages: 2024-01-24T23:18:17.427 INFO:teuthology.orchestra.run.smithi132.stdout: btrfs-tools 2024-01-24T23:18:17.501 INFO:teuthology.orchestra.run.smithi115.stdout:The following packages were automatically installed and are no longer required: 2024-01-24T23:18:17.502 INFO:teuthology.orchestra.run.smithi115.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-01-24T23:18:17.502 INFO:teuthology.orchestra.run.smithi115.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-01-24T23:18:17.502 INFO:teuthology.orchestra.run.smithi115.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-01-24T23:18:17.502 INFO:teuthology.orchestra.run.smithi115.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-01-24T23:18:17.502 INFO:teuthology.orchestra.run.smithi115.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-01-24T23:18:17.502 INFO:teuthology.orchestra.run.smithi115.stdout:Use 'sudo apt autoremove' to remove them. 2024-01-24T23:18:17.504 INFO:teuthology.orchestra.run.smithi115.stdout:The following additional packages will be installed: 2024-01-24T23:18:17.504 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-01-24T23:18:17.504 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-01-24T23:18:17.504 INFO:teuthology.orchestra.run.smithi115.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-01-24T23:18:17.504 INFO:teuthology.orchestra.run.smithi115.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-01-24T23:18:17.504 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-01-24T23:18:17.505 INFO:teuthology.orchestra.run.smithi115.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-01-24T23:18:17.505 INFO:teuthology.orchestra.run.smithi115.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-01-24T23:18:17.505 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-01-24T23:18:17.505 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-01-24T23:18:17.505 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-01-24T23:18:17.505 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-01-24T23:18:17.505 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-01-24T23:18:17.505 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-01-24T23:18:17.505 INFO:teuthology.orchestra.run.smithi115.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-01-24T23:18:17.505 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-01-24T23:18:17.505 INFO:teuthology.orchestra.run.smithi115.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-01-24T23:18:17.505 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-01-24T23:18:17.505 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-01-24T23:18:17.505 INFO:teuthology.orchestra.run.smithi115.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-01-24T23:18:17.506 INFO:teuthology.orchestra.run.smithi115.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-01-24T23:18:17.506 INFO:teuthology.orchestra.run.smithi115.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-01-24T23:18:17.506 INFO:teuthology.orchestra.run.smithi115.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-01-24T23:18:17.506 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-01-24T23:18:17.506 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet zip 2024-01-24T23:18:17.506 INFO:teuthology.orchestra.run.smithi115.stdout:Suggested packages: 2024-01-24T23:18:17.506 INFO:teuthology.orchestra.run.smithi115.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-01-24T23:18:17.506 INFO:teuthology.orchestra.run.smithi115.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-01-24T23:18:17.506 INFO:teuthology.orchestra.run.smithi115.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-01-24T23:18:17.506 INFO:teuthology.orchestra.run.smithi115.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-01-24T23:18:17.506 INFO:teuthology.orchestra.run.smithi115.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-01-24T23:18:17.506 INFO:teuthology.orchestra.run.smithi115.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-01-24T23:18:17.506 INFO:teuthology.orchestra.run.smithi115.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-01-24T23:18:17.507 INFO:teuthology.orchestra.run.smithi115.stdout:Recommended packages: 2024-01-24T23:18:17.507 INFO:teuthology.orchestra.run.smithi115.stdout: btrfs-tools 2024-01-24T23:18:17.526 INFO:teuthology.orchestra.run.smithi132.stdout:The following NEW packages will be installed: 2024-01-24T23:18:17.527 INFO:teuthology.orchestra.run.smithi132.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-01-24T23:18:17.527 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-01-24T23:18:17.527 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-01-24T23:18:17.527 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-01-24T23:18:17.527 INFO:teuthology.orchestra.run.smithi132.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-01-24T23:18:17.528 INFO:teuthology.orchestra.run.smithi132.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-01-24T23:18:17.529 INFO:teuthology.orchestra.run.smithi132.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-01-24T23:18:17.529 INFO:teuthology.orchestra.run.smithi132.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-01-24T23:18:17.529 INFO:teuthology.orchestra.run.smithi132.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-01-24T23:18:17.529 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-01-24T23:18:17.530 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-01-24T23:18:17.530 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-01-24T23:18:17.530 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-01-24T23:18:17.530 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-01-24T23:18:17.530 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-01-24T23:18:17.530 INFO:teuthology.orchestra.run.smithi132.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-01-24T23:18:17.530 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-01-24T23:18:17.530 INFO:teuthology.orchestra.run.smithi132.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-01-24T23:18:17.531 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-01-24T23:18:17.531 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-01-24T23:18:17.531 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-01-24T23:18:17.531 INFO:teuthology.orchestra.run.smithi132.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-01-24T23:18:17.531 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-01-24T23:18:17.531 INFO:teuthology.orchestra.run.smithi132.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-01-24T23:18:17.532 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-01-24T23:18:17.532 INFO:teuthology.orchestra.run.smithi132.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-01-24T23:18:17.601 INFO:teuthology.orchestra.run.smithi115.stdout:The following NEW packages will be installed: 2024-01-24T23:18:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-01-24T23:18:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-01-24T23:18:17.601 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-01-24T23:18:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-01-24T23:18:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-01-24T23:18:17.602 INFO:teuthology.orchestra.run.smithi115.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-01-24T23:18:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-01-24T23:18:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-01-24T23:18:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-01-24T23:18:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-01-24T23:18:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-01-24T23:18:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-01-24T23:18:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-01-24T23:18:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-01-24T23:18:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-01-24T23:18:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-01-24T23:18:17.603 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-01-24T23:18:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-01-24T23:18:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-01-24T23:18:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-01-24T23:18:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-01-24T23:18:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-01-24T23:18:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-01-24T23:18:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-01-24T23:18:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-01-24T23:18:17.604 INFO:teuthology.orchestra.run.smithi115.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-01-24T23:18:17.706 INFO:teuthology.orchestra.run.smithi115.stdout:0 upgraded, 118 newly installed, 0 to remove and 266 not upgraded. 2024-01-24T23:18:17.706 INFO:teuthology.orchestra.run.smithi115.stdout:Need to get 172 MB of archives. 2024-01-24T23:18:17.706 INFO:teuthology.orchestra.run.smithi115.stdout:After this operation, 712 MB of additional disk space will be used. 2024-01-24T23:18:17.706 INFO:teuthology.orchestra.run.smithi115.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-01-24T23:18:17.714 INFO:teuthology.orchestra.run.smithi132.stdout:0 upgraded, 118 newly installed, 0 to remove and 266 not upgraded. 2024-01-24T23:18:17.714 INFO:teuthology.orchestra.run.smithi132.stdout:Need to get 172 MB of archives. 2024-01-24T23:18:17.714 INFO:teuthology.orchestra.run.smithi132.stdout:After this operation, 712 MB of additional disk space will be used. 2024-01-24T23:18:17.714 INFO:teuthology.orchestra.run.smithi132.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-01-24T23:18:17.741 INFO:teuthology.orchestra.run.smithi115.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-431-g66387355-1jammy [3,514 kB] 2024-01-24T23:18:17.866 INFO:teuthology.orchestra.run.smithi132.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-431-g66387355-1jammy [3,514 kB] 2024-01-24T23:18:17.955 INFO:teuthology.orchestra.run.smithi115.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-431-g66387355-1jammy [3,207 kB] 2024-01-24T23:18:17.955 INFO:teuthology.orchestra.run.smithi115.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-01-24T23:18:17.963 INFO:teuthology.orchestra.run.smithi115.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-01-24T23:18:17.987 INFO:teuthology.orchestra.run.smithi115.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-01-24T23:18:18.058 INFO:teuthology.orchestra.run.smithi115.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-431-g66387355-1jammy [710 kB] 2024-01-24T23:18:18.071 INFO:teuthology.orchestra.run.smithi115.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-01-24T23:18:18.076 INFO:teuthology.orchestra.run.smithi115.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-01-24T23:18:18.080 INFO:teuthology.orchestra.run.smithi115.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-431-g66387355-1jammy [356 kB] 2024-01-24T23:18:18.090 INFO:teuthology.orchestra.run.smithi115.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-01-24T23:18:18.092 INFO:teuthology.orchestra.run.smithi115.stdout:Get:12 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-431-g66387355-1jammy [32.8 kB] 2024-01-24T23:18:18.093 INFO:teuthology.orchestra.run.smithi115.stdout:Get:13 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-431-g66387355-1jammy [180 kB] 2024-01-24T23:18:18.094 INFO:teuthology.orchestra.run.smithi115.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-01-24T23:18:18.095 INFO:teuthology.orchestra.run.smithi115.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-01-24T23:18:18.095 INFO:teuthology.orchestra.run.smithi115.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-01-24T23:18:18.099 INFO:teuthology.orchestra.run.smithi115.stdout:Get:17 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-431-g66387355-1jammy [64.1 kB] 2024-01-24T23:18:18.101 INFO:teuthology.orchestra.run.smithi115.stdout:Get:18 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-431-g66387355-1jammy [330 kB] 2024-01-24T23:18:18.104 INFO:teuthology.orchestra.run.smithi115.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-01-24T23:18:18.107 INFO:teuthology.orchestra.run.smithi115.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-01-24T23:18:18.109 INFO:teuthology.orchestra.run.smithi115.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-01-24T23:18:18.110 INFO:teuthology.orchestra.run.smithi115.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-01-24T23:18:18.112 INFO:teuthology.orchestra.run.smithi115.stdout:Get:23 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-431-g66387355-1jammy [6,220 kB] 2024-01-24T23:18:18.128 INFO:teuthology.orchestra.run.smithi115.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-01-24T23:18:18.131 INFO:teuthology.orchestra.run.smithi115.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.1 [174 kB] 2024-01-24T23:18:18.133 INFO:teuthology.orchestra.run.smithi115.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-01-24T23:18:18.135 INFO:teuthology.orchestra.run.smithi132.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-01-24T23:18:18.135 INFO:teuthology.orchestra.run.smithi115.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-01-24T23:18:18.136 INFO:teuthology.orchestra.run.smithi115.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-01-24T23:18:18.136 INFO:teuthology.orchestra.run.smithi115.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-01-24T23:18:18.147 INFO:teuthology.orchestra.run.smithi132.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-01-24T23:18:18.150 INFO:teuthology.orchestra.run.smithi115.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-01-24T23:18:18.150 INFO:teuthology.orchestra.run.smithi115.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-01-24T23:18:18.150 INFO:teuthology.orchestra.run.smithi115.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-01-24T23:18:18.151 INFO:teuthology.orchestra.run.smithi115.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-01-24T23:18:18.170 INFO:teuthology.orchestra.run.smithi115.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-01-24T23:18:18.170 INFO:teuthology.orchestra.run.smithi115.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-01-24T23:18:18.171 INFO:teuthology.orchestra.run.smithi115.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-01-24T23:18:18.173 INFO:teuthology.orchestra.run.smithi115.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-01-24T23:18:18.174 INFO:teuthology.orchestra.run.smithi115.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-01-24T23:18:18.174 INFO:teuthology.orchestra.run.smithi115.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-01-24T23:18:18.191 INFO:teuthology.orchestra.run.smithi115.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-01-24T23:18:18.191 INFO:teuthology.orchestra.run.smithi115.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-01-24T23:18:18.192 INFO:teuthology.orchestra.run.smithi115.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-01-24T23:18:18.192 INFO:teuthology.orchestra.run.smithi115.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-01-24T23:18:18.211 INFO:teuthology.orchestra.run.smithi115.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-01-24T23:18:18.212 INFO:teuthology.orchestra.run.smithi115.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-01-24T23:18:18.212 INFO:teuthology.orchestra.run.smithi115.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-01-24T23:18:18.218 INFO:teuthology.orchestra.run.smithi115.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-01-24T23:18:18.218 INFO:teuthology.orchestra.run.smithi115.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-01-24T23:18:18.219 INFO:teuthology.orchestra.run.smithi115.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-01-24T23:18:18.228 INFO:teuthology.orchestra.run.smithi132.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-01-24T23:18:18.232 INFO:teuthology.orchestra.run.smithi115.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-01-24T23:18:18.233 INFO:teuthology.orchestra.run.smithi115.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-01-24T23:18:18.235 INFO:teuthology.orchestra.run.smithi115.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-01-24T23:18:18.236 INFO:teuthology.orchestra.run.smithi115.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-01-24T23:18:18.252 INFO:teuthology.orchestra.run.smithi115.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-01-24T23:18:18.253 INFO:teuthology.orchestra.run.smithi115.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-01-24T23:18:18.255 INFO:teuthology.orchestra.run.smithi115.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-01-24T23:18:18.256 INFO:teuthology.orchestra.run.smithi115.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-01-24T23:18:18.257 INFO:teuthology.orchestra.run.smithi115.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-01-24T23:18:18.259 INFO:teuthology.orchestra.run.smithi115.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-01-24T23:18:18.327 INFO:teuthology.orchestra.run.smithi115.stdout:Get:60 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-431-g66387355-1jammy [112 kB] 2024-01-24T23:18:18.330 INFO:teuthology.orchestra.run.smithi115.stdout:Get:61 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-431-g66387355-1jammy [465 kB] 2024-01-24T23:18:18.345 INFO:teuthology.orchestra.run.smithi115.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-01-24T23:18:18.345 INFO:teuthology.orchestra.run.smithi115.stdout:Get:63 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-431-g66387355-1jammy [23.0 MB] 2024-01-24T23:18:18.346 INFO:teuthology.orchestra.run.smithi115.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2024-01-24T23:18:18.347 INFO:teuthology.orchestra.run.smithi115.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-01-24T23:18:18.348 INFO:teuthology.orchestra.run.smithi115.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-01-24T23:18:18.349 INFO:teuthology.orchestra.run.smithi115.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-01-24T23:18:18.373 INFO:teuthology.orchestra.run.smithi115.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-01-24T23:18:18.375 INFO:teuthology.orchestra.run.smithi115.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-01-24T23:18:18.376 INFO:teuthology.orchestra.run.smithi115.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-01-24T23:18:18.407 INFO:teuthology.orchestra.run.smithi115.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-01-24T23:18:18.407 INFO:teuthology.orchestra.run.smithi115.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-01-24T23:18:18.422 INFO:teuthology.orchestra.run.smithi115.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-01-24T23:18:18.423 INFO:teuthology.orchestra.run.smithi115.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-01-24T23:18:18.423 INFO:teuthology.orchestra.run.smithi115.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-01-24T23:18:18.424 INFO:teuthology.orchestra.run.smithi115.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-01-24T23:18:18.424 INFO:teuthology.orchestra.run.smithi115.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-01-24T23:18:18.425 INFO:teuthology.orchestra.run.smithi115.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-01-24T23:18:18.428 INFO:teuthology.orchestra.run.smithi115.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-01-24T23:18:18.447 INFO:teuthology.orchestra.run.smithi115.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-01-24T23:18:18.448 INFO:teuthology.orchestra.run.smithi115.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-01-24T23:18:18.449 INFO:teuthology.orchestra.run.smithi115.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-01-24T23:18:18.458 INFO:teuthology.orchestra.run.smithi132.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-01-24T23:18:18.467 INFO:teuthology.orchestra.run.smithi115.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-01-24T23:18:18.470 INFO:teuthology.orchestra.run.smithi132.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-01-24T23:18:18.471 INFO:teuthology.orchestra.run.smithi115.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-01-24T23:18:18.472 INFO:teuthology.orchestra.run.smithi115.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-01-24T23:18:18.472 INFO:teuthology.orchestra.run.smithi115.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-01-24T23:18:18.478 INFO:teuthology.orchestra.run.smithi115.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-01-24T23:18:18.479 INFO:teuthology.orchestra.run.smithi115.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-01-24T23:18:18.482 INFO:teuthology.orchestra.run.smithi115.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-01-24T23:18:18.493 INFO:teuthology.orchestra.run.smithi115.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-01-24T23:18:18.493 INFO:teuthology.orchestra.run.smithi115.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-01-24T23:18:18.494 INFO:teuthology.orchestra.run.smithi115.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-01-24T23:18:18.500 INFO:teuthology.orchestra.run.smithi132.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-01-24T23:18:18.512 INFO:teuthology.orchestra.run.smithi132.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-01-24T23:18:18.513 INFO:teuthology.orchestra.run.smithi132.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-01-24T23:18:18.514 INFO:teuthology.orchestra.run.smithi132.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-01-24T23:18:18.517 INFO:teuthology.orchestra.run.smithi115.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-01-24T23:18:18.518 INFO:teuthology.orchestra.run.smithi115.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-01-24T23:18:18.527 INFO:teuthology.orchestra.run.smithi115.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-01-24T23:18:18.527 INFO:teuthology.orchestra.run.smithi115.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-01-24T23:18:18.528 INFO:teuthology.orchestra.run.smithi115.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-01-24T23:18:18.530 INFO:teuthology.orchestra.run.smithi132.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-01-24T23:18:18.530 INFO:teuthology.orchestra.run.smithi115.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-01-24T23:18:18.531 INFO:teuthology.orchestra.run.smithi115.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-01-24T23:18:18.699 INFO:teuthology.orchestra.run.smithi132.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-01-24T23:18:19.101 INFO:teuthology.orchestra.run.smithi132.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-01-24T23:18:19.136 INFO:teuthology.orchestra.run.smithi132.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-01-24T23:18:19.137 INFO:teuthology.orchestra.run.smithi132.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-01-24T23:18:19.195 INFO:teuthology.orchestra.run.smithi115.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-431-g66387355-1jammy [5,057 kB] 2024-01-24T23:18:19.217 INFO:teuthology.orchestra.run.smithi132.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.1 [174 kB] 2024-01-24T23:18:19.268 INFO:teuthology.orchestra.run.smithi132.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-01-24T23:18:19.290 INFO:teuthology.orchestra.run.smithi132.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-01-24T23:18:19.297 INFO:teuthology.orchestra.run.smithi132.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-01-24T23:18:19.298 INFO:teuthology.orchestra.run.smithi132.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-01-24T23:18:19.309 INFO:teuthology.orchestra.run.smithi132.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-01-24T23:18:19.310 INFO:teuthology.orchestra.run.smithi132.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-01-24T23:18:19.323 INFO:teuthology.orchestra.run.smithi115.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-431-g66387355-1jammy [238 kB] 2024-01-24T23:18:19.330 INFO:teuthology.orchestra.run.smithi115.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-431-g66387355-1jammy [124 kB] 2024-01-24T23:18:19.335 INFO:teuthology.orchestra.run.smithi115.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-431-g66387355-1jammy [1,517 kB] 2024-01-24T23:18:19.390 INFO:teuthology.orchestra.run.smithi115.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-431-g66387355-1jammy [6,236 kB] 2024-01-24T23:18:19.475 INFO:teuthology.orchestra.run.smithi132.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-01-24T23:18:19.558 INFO:teuthology.orchestra.run.smithi132.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-01-24T23:18:19.587 INFO:teuthology.orchestra.run.smithi132.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-01-24T23:18:19.604 INFO:teuthology.orchestra.run.smithi132.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-01-24T23:18:19.608 INFO:teuthology.orchestra.run.smithi115.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-431-g66387355-1jammy [22.7 MB] 2024-01-24T23:18:19.648 INFO:teuthology.orchestra.run.smithi132.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-01-24T23:18:19.858 INFO:teuthology.orchestra.run.smithi132.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-01-24T23:18:19.874 INFO:teuthology.orchestra.run.smithi132.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-01-24T23:18:19.880 INFO:teuthology.orchestra.run.smithi132.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-01-24T23:18:19.886 INFO:teuthology.orchestra.run.smithi132.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-01-24T23:18:19.907 INFO:teuthology.orchestra.run.smithi132.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-01-24T23:18:19.910 INFO:teuthology.orchestra.run.smithi132.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-01-24T23:18:20.003 INFO:teuthology.orchestra.run.smithi132.stdout:Get:35 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-431-g66387355-1jammy [3,207 kB] 2024-01-24T23:18:20.076 INFO:teuthology.orchestra.run.smithi132.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-01-24T23:18:20.408 INFO:teuthology.orchestra.run.smithi132.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-01-24T23:18:20.439 INFO:teuthology.orchestra.run.smithi132.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-01-24T23:18:20.450 INFO:teuthology.orchestra.run.smithi132.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-01-24T23:18:20.494 INFO:teuthology.orchestra.run.smithi115.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-431-g66387355-1jammy [14.0 kB] 2024-01-24T23:18:20.494 INFO:teuthology.orchestra.run.smithi115.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-431-g66387355-1jammy [865 kB] 2024-01-24T23:18:20.511 INFO:teuthology.orchestra.run.smithi115.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-431-g66387355-1jammy [2,181 kB] 2024-01-24T23:18:20.549 INFO:teuthology.orchestra.run.smithi115.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-431-g66387355-1jammy [837 kB] 2024-01-24T23:18:20.559 INFO:teuthology.orchestra.run.smithi115.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-431-g66387355-1jammy [139 kB] 2024-01-24T23:18:20.561 INFO:teuthology.orchestra.run.smithi115.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-431-g66387355-1jammy [1,905 kB] 2024-01-24T23:18:20.624 INFO:teuthology.orchestra.run.smithi115.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-431-g66387355-1jammy [8,625 kB] 2024-01-24T23:18:20.651 INFO:teuthology.orchestra.run.smithi132.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-01-24T23:18:20.652 INFO:teuthology.orchestra.run.smithi132.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-01-24T23:18:20.659 INFO:teuthology.orchestra.run.smithi132.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-01-24T23:18:20.667 INFO:teuthology.orchestra.run.smithi132.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-01-24T23:18:20.679 INFO:teuthology.orchestra.run.smithi132.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-01-24T23:18:20.701 INFO:teuthology.orchestra.run.smithi132.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-01-24T23:18:20.707 INFO:teuthology.orchestra.run.smithi132.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-01-24T23:18:20.882 INFO:teuthology.orchestra.run.smithi132.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-01-24T23:18:20.927 INFO:teuthology.orchestra.run.smithi115.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-431-g66387355-1jammy [14.2 kB] 2024-01-24T23:18:20.928 INFO:teuthology.orchestra.run.smithi115.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-431-g66387355-1jammy [48.3 MB] 2024-01-24T23:18:21.145 INFO:teuthology.orchestra.run.smithi132.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-01-24T23:18:21.324 INFO:teuthology.orchestra.run.smithi132.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-01-24T23:18:21.338 INFO:teuthology.orchestra.run.smithi132.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-01-24T23:18:21.358 INFO:teuthology.orchestra.run.smithi132.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-01-24T23:18:21.423 INFO:teuthology.orchestra.run.smithi132.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-01-24T23:18:21.514 INFO:teuthology.orchestra.run.smithi132.stdout:Get:53 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-431-g66387355-1jammy [710 kB] 2024-01-24T23:18:21.712 INFO:teuthology.orchestra.run.smithi132.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-01-24T23:18:21.713 INFO:teuthology.orchestra.run.smithi132.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2024-01-24T23:18:21.716 INFO:teuthology.orchestra.run.smithi132.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-01-24T23:18:21.716 INFO:teuthology.orchestra.run.smithi132.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-01-24T23:18:21.719 INFO:teuthology.orchestra.run.smithi132.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-01-24T23:18:21.833 INFO:teuthology.orchestra.run.smithi132.stdout:Get:59 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-431-g66387355-1jammy [356 kB] 2024-01-24T23:18:21.938 INFO:teuthology.orchestra.run.smithi132.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-01-24T23:18:21.996 INFO:teuthology.orchestra.run.smithi132.stdout:Get:61 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-431-g66387355-1jammy [32.8 kB] 2024-01-24T23:18:22.001 INFO:teuthology.orchestra.run.smithi132.stdout:Get:62 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-431-g66387355-1jammy [180 kB] 2024-01-24T23:18:22.081 INFO:teuthology.orchestra.run.smithi132.stdout:Get:63 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-431-g66387355-1jammy [64.1 kB] 2024-01-24T23:18:22.103 INFO:teuthology.orchestra.run.smithi132.stdout:Get:64 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-431-g66387355-1jammy [330 kB] 2024-01-24T23:18:22.251 INFO:teuthology.orchestra.run.smithi132.stdout:Get:65 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-431-g66387355-1jammy [6,220 kB] 2024-01-24T23:18:22.367 INFO:teuthology.orchestra.run.smithi132.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-01-24T23:18:22.373 INFO:teuthology.orchestra.run.smithi132.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-01-24T23:18:22.663 INFO:teuthology.orchestra.run.smithi132.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-01-24T23:18:22.664 INFO:teuthology.orchestra.run.smithi132.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-01-24T23:18:22.666 INFO:teuthology.orchestra.run.smithi132.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-01-24T23:18:22.667 INFO:teuthology.orchestra.run.smithi132.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-01-24T23:18:22.669 INFO:teuthology.orchestra.run.smithi132.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-01-24T23:18:22.671 INFO:teuthology.orchestra.run.smithi132.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-01-24T23:18:22.673 INFO:teuthology.orchestra.run.smithi132.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-01-24T23:18:22.673 INFO:teuthology.orchestra.run.smithi132.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-01-24T23:18:22.856 INFO:teuthology.orchestra.run.smithi132.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-01-24T23:18:22.867 INFO:teuthology.orchestra.run.smithi115.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-431-g66387355-1jammy [121 kB] 2024-01-24T23:18:22.869 INFO:teuthology.orchestra.run.smithi115.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-431-g66387355-1jammy [38.4 kB] 2024-01-24T23:18:22.870 INFO:teuthology.orchestra.run.smithi115.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-431-g66387355-1jammy [12.3 MB] 2024-01-24T23:18:23.182 INFO:teuthology.orchestra.run.smithi115.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-431-g66387355-1jammy [91.7 kB] 2024-01-24T23:18:23.261 INFO:teuthology.orchestra.run.smithi132.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-01-24T23:18:23.308 INFO:teuthology.orchestra.run.smithi132.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-01-24T23:18:23.342 INFO:teuthology.orchestra.run.smithi132.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-01-24T23:18:23.439 INFO:teuthology.orchestra.run.smithi132.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-01-24T23:18:23.479 INFO:teuthology.orchestra.run.smithi132.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-01-24T23:18:23.491 INFO:teuthology.orchestra.run.smithi132.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-01-24T23:18:23.496 INFO:teuthology.orchestra.run.smithi132.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-01-24T23:18:23.543 INFO:teuthology.orchestra.run.smithi132.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-01-24T23:18:23.546 INFO:teuthology.orchestra.run.smithi132.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-01-24T23:18:23.568 INFO:teuthology.orchestra.run.smithi132.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-01-24T23:18:23.732 INFO:teuthology.orchestra.run.smithi132.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-01-24T23:18:23.737 INFO:teuthology.orchestra.run.smithi115.stdout:Fetched 172 MB in 6s (30.9 MB/s) 2024-01-24T23:18:23.891 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-01-24T23:18:23.919 INFO:teuthology.orchestra.run.smithi132.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-431-g66387355-1jammy [112 kB] 2024-01-24T23:18:23.933 INFO:teuthology.orchestra.run.smithi132.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-431-g66387355-1jammy [465 kB] 2024-01-24T23:18:23.991 INFO:teuthology.orchestra.run.smithi115.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165989 files and directories currently installed.) 2024-01-24T23:18:23.997 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-01-24T23:18:23.999 INFO:teuthology.orchestra.run.smithi132.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-01-24T23:18:24.018 INFO:teuthology.orchestra.run.smithi132.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-01-24T23:18:24.065 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-01-24T23:18:24.157 INFO:teuthology.orchestra.run.smithi132.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-431-g66387355-1jammy [23.0 MB] 2024-01-24T23:18:24.171 INFO:teuthology.orchestra.run.smithi132.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-01-24T23:18:24.201 INFO:teuthology.orchestra.run.smithi132.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-01-24T23:18:24.344 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-01-24T23:18:24.359 INFO:teuthology.orchestra.run.smithi132.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-01-24T23:18:24.362 INFO:teuthology.orchestra.run.smithi132.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-01-24T23:18:24.364 INFO:teuthology.orchestra.run.smithi132.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-01-24T23:18:24.370 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-01-24T23:18:24.385 INFO:teuthology.orchestra.run.smithi132.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-01-24T23:18:24.391 INFO:teuthology.orchestra.run.smithi132.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-01-24T23:18:24.401 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-01-24T23:18:24.755 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-01-24T23:18:24.781 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-01-24T23:18:24.821 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-01-24T23:18:25.283 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-01-24T23:18:25.310 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-01-24T23:18:25.358 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-01-24T23:18:25.686 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-01-24T23:18:25.713 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-01-24T23:18:25.744 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-01-24T23:18:26.114 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-01-24T23:18:26.140 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-01-24T23:18:26.181 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-01-24T23:18:26.634 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-01-24T23:18:26.661 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-01-24T23:18:26.700 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-01-24T23:18:26.970 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package librados2. 2024-01-24T23:18:26.997 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../007-librados2_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:27.028 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking librados2 (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:27.491 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package librbd1. 2024-01-24T23:18:27.881 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../008-librbd1_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:27.881 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking librbd1 (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:28.005 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libcephfs2. 2024-01-24T23:18:28.028 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:28.068 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libcephfs2 (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:28.345 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-rados. 2024-01-24T23:18:28.361 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../010-python3-rados_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:28.403 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-rados (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:28.681 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-01-24T23:18:28.696 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-431-g66387355-1jammy_all.deb ... 2024-01-24T23:18:28.731 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-ceph-argparse (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:28.966 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-cephfs. 2024-01-24T23:18:28.981 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:29.025 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-cephfs (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:29.336 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-ceph-common. 2024-01-24T23:18:29.363 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-431-g66387355-1jammy_all.deb ... 2024-01-24T23:18:29.403 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-ceph-common (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:29.756 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-wcwidth. 2024-01-24T23:18:29.781 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-01-24T23:18:29.822 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-01-24T23:18:30.109 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-prettytable. 2024-01-24T23:18:30.124 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-01-24T23:18:30.158 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-01-24T23:18:30.395 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-rbd. 2024-01-24T23:18:30.421 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:30.461 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-rbd (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:30.822 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-01-24T23:18:30.848 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-01-24T23:18:30.891 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-01-24T23:18:31.167 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-01-24T23:18:31.182 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-01-24T23:18:31.225 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-01-24T23:18:31.511 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-01-24T23:18:31.526 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-01-24T23:18:31.561 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-01-24T23:18:31.939 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package lua5.1. 2024-01-24T23:18:31.955 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-01-24T23:18:31.990 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-01-24T23:18:32.309 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package lua-any. 2024-01-24T23:18:32.325 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-01-24T23:18:32.367 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking lua-any (27ubuntu1) ... 2024-01-24T23:18:32.629 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package zip. 2024-01-24T23:18:32.655 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-01-24T23:18:32.695 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking zip (3.0-12build2) ... 2024-01-24T23:18:33.038 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package unzip. 2024-01-24T23:18:33.064 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.1_amd64.deb ... 2024-01-24T23:18:33.107 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking unzip (6.0-26ubuntu3.1) ... 2024-01-24T23:18:33.444 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package luarocks. 2024-01-24T23:18:33.470 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-01-24T23:18:33.502 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-01-24T23:18:33.877 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package librgw2. 2024-01-24T23:18:33.903 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../025-librgw2_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:33.943 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking librgw2 (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:34.471 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-rgw. 2024-01-24T23:18:34.486 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:34.509 INFO:teuthology.orchestra.run.smithi132.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-431-g66387355-1jammy [5,057 kB] 2024-01-24T23:18:34.529 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-rgw (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:34.832 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package liboath0:amd64. 2024-01-24T23:18:34.858 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-01-24T23:18:34.890 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-01-24T23:18:35.185 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libradosstriper1. 2024-01-24T23:18:35.211 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:35.251 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libradosstriper1 (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:35.277 INFO:teuthology.orchestra.run.smithi132.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-431-g66387355-1jammy [238 kB] 2024-01-24T23:18:35.330 INFO:teuthology.orchestra.run.smithi132.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-431-g66387355-1jammy [124 kB] 2024-01-24T23:18:35.346 INFO:teuthology.orchestra.run.smithi132.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-431-g66387355-1jammy [1,517 kB] 2024-01-24T23:18:35.594 INFO:teuthology.orchestra.run.smithi132.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-431-g66387355-1jammy [6,236 kB] 2024-01-24T23:18:35.680 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-common. 2024-01-24T23:18:35.706 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../029-ceph-common_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:35.746 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-common (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:36.425 INFO:teuthology.orchestra.run.smithi132.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-431-g66387355-1jammy [22.7 MB] 2024-01-24T23:18:37.110 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-base. 2024-01-24T23:18:37.136 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../030-ceph-base_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:37.177 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-base (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:37.714 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-01-24T23:18:37.741 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-01-24T23:18:37.781 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-01-24T23:18:38.076 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-cheroot. 2024-01-24T23:18:38.102 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-01-24T23:18:38.142 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-01-24T23:18:38.445 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-01-24T23:18:38.472 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-01-24T23:18:38.503 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-01-24T23:18:38.706 INFO:teuthology.orchestra.run.smithi132.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-431-g66387355-1jammy [14.0 kB] 2024-01-24T23:18:38.707 INFO:teuthology.orchestra.run.smithi132.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-431-g66387355-1jammy [865 kB] 2024-01-24T23:18:38.723 INFO:teuthology.orchestra.run.smithi132.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-431-g66387355-1jammy [2,181 kB] 2024-01-24T23:18:38.790 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-jaraco.text. 2024-01-24T23:18:38.814 INFO:teuthology.orchestra.run.smithi132.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-431-g66387355-1jammy [837 kB] 2024-01-24T23:18:38.817 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-01-24T23:18:38.857 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-01-24T23:18:38.881 INFO:teuthology.orchestra.run.smithi132.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-431-g66387355-1jammy [139 kB] 2024-01-24T23:18:38.883 INFO:teuthology.orchestra.run.smithi132.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-431-g66387355-1jammy [1,905 kB] 2024-01-24T23:18:38.996 INFO:teuthology.orchestra.run.smithi132.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-431-g66387355-1jammy [8,625 kB] 2024-01-24T23:18:39.168 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-01-24T23:18:39.194 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-01-24T23:18:39.234 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-01-24T23:18:39.520 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-tempora. 2024-01-24T23:18:39.522 INFO:teuthology.orchestra.run.smithi132.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-431-g66387355-1jammy [14.2 kB] 2024-01-24T23:18:39.523 INFO:teuthology.orchestra.run.smithi132.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-431-g66387355-1jammy [48.3 MB] 2024-01-24T23:18:39.547 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-01-24T23:18:39.587 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-01-24T23:18:39.848 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-portend. 2024-01-24T23:18:39.875 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-01-24T23:18:39.915 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-01-24T23:18:40.234 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-01-24T23:18:40.261 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-01-24T23:18:40.301 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-01-24T23:18:40.604 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-cherrypy3. 2024-01-24T23:18:40.631 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-01-24T23:18:40.670 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-01-24T23:18:41.033 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-natsort. 2024-01-24T23:18:41.059 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-01-24T23:18:41.099 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-01-24T23:18:41.410 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-logutils. 2024-01-24T23:18:41.437 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-01-24T23:18:41.477 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-01-24T23:18:41.755 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-markupsafe. 2024-01-24T23:18:41.782 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-01-24T23:18:41.821 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-01-24T23:18:42.076 INFO:teuthology.orchestra.run.smithi132.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-431-g66387355-1jammy [121 kB] 2024-01-24T23:18:42.091 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-mako. 2024-01-24T23:18:42.118 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-01-24T23:18:42.158 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-01-24T23:18:42.159 INFO:teuthology.orchestra.run.smithi132.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-431-g66387355-1jammy [38.4 kB] 2024-01-24T23:18:42.160 INFO:teuthology.orchestra.run.smithi132.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-431-g66387355-1jammy [12.3 MB] 2024-01-24T23:18:42.461 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-simplegeneric. 2024-01-24T23:18:42.464 INFO:teuthology.orchestra.run.smithi132.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-pereman2-buffer-space/663873555d45def955d1cc09730bfece2847b821/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-431-g66387355-1jammy [91.7 kB] 2024-01-24T23:18:42.488 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-01-24T23:18:42.527 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-01-24T23:18:42.830 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-singledispatch. 2024-01-24T23:18:42.857 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-01-24T23:18:42.897 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-01-24T23:18:43.022 INFO:teuthology.orchestra.run.smithi132.stdout:Fetched 172 MB in 25s (6,899 kB/s) 2024-01-24T23:18:43.175 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-webob. 2024-01-24T23:18:43.186 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-01-24T23:18:43.202 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-01-24T23:18:43.241 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-01-24T23:18:43.286 INFO:teuthology.orchestra.run.smithi132.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165989 files and directories currently installed.) 2024-01-24T23:18:43.291 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-01-24T23:18:43.376 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-01-24T23:18:43.553 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-waitress. 2024-01-24T23:18:43.579 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-01-24T23:18:43.614 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-01-24T23:18:43.730 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-01-24T23:18:43.756 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-01-24T23:18:43.797 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-01-24T23:18:43.906 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-tempita. 2024-01-24T23:18:43.932 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-01-24T23:18:43.972 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-01-24T23:18:44.100 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-01-24T23:18:44.126 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-01-24T23:18:44.167 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-01-24T23:18:44.275 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-paste. 2024-01-24T23:18:44.302 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-01-24T23:18:44.342 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-01-24T23:18:44.620 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-01-24T23:18:44.628 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-01-24T23:18:44.647 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-01-24T23:18:44.655 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-01-24T23:18:44.695 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-01-24T23:18:44.695 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-01-24T23:18:44.998 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pastedeploy. 2024-01-24T23:18:45.024 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-01-24T23:18:45.064 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-01-24T23:18:45.074 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-01-24T23:18:45.100 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-01-24T23:18:45.132 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-01-24T23:18:45.367 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-webtest. 2024-01-24T23:18:45.394 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-01-24T23:18:45.434 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-01-24T23:18:45.510 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-01-24T23:18:45.537 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-01-24T23:18:45.577 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-01-24T23:18:45.720 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pecan. 2024-01-24T23:18:45.736 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-01-24T23:18:45.770 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-01-24T23:18:45.972 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-01-24T23:18:45.999 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-01-24T23:18:46.038 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-01-24T23:18:46.100 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-certifi. 2024-01-24T23:18:46.126 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-01-24T23:18:46.158 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-01-24T23:18:46.342 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package librados2. 2024-01-24T23:18:46.368 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../007-librados2_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:46.400 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking librados2 (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:46.469 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-idna. 2024-01-24T23:18:46.496 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-01-24T23:18:46.536 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-idna (3.3-1) ... 2024-01-24T23:18:46.803 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package librbd1. 2024-01-24T23:18:46.829 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../008-librbd1_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:46.856 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-urllib3. 2024-01-24T23:18:46.861 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking librbd1 (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:46.883 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-01-24T23:18:46.922 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-01-24T23:18:47.314 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libcephfs2. 2024-01-24T23:18:47.340 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:47.380 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libcephfs2 (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:47.451 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-requests. 2024-01-24T23:18:47.477 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-01-24T23:18:47.517 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-01-24T23:18:47.668 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-rados. 2024-01-24T23:18:47.693 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../010-python3-rados_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:47.733 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-rados (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:47.812 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-werkzeug. 2024-01-24T23:18:47.838 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-01-24T23:18:47.878 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-01-24T23:18:48.181 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-01-24T23:18:48.197 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-431-g66387355-1jammy_all.deb ... 2024-01-24T23:18:48.203 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-01-24T23:18:48.229 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-431-g66387355-1jammy_all.deb ... 2024-01-24T23:18:48.239 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-mgr-modules-core (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:48.261 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-ceph-argparse (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:48.489 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-cephfs. 2024-01-24T23:18:48.505 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:48.548 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-cephfs (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:48.577 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-01-24T23:18:48.605 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:48.636 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:48.833 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-ceph-common. 2024-01-24T23:18:48.860 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-431-g66387355-1jammy_all.deb ... 2024-01-24T23:18:48.892 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-ceph-common (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:48.972 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-bcrypt. 2024-01-24T23:18:48.998 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-01-24T23:18:49.038 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-01-24T23:18:49.161 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-wcwidth. 2024-01-24T23:18:49.177 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-01-24T23:18:49.220 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-01-24T23:18:49.327 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-openssl. 2024-01-24T23:18:49.342 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-01-24T23:18:49.376 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-01-24T23:18:49.556 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-prettytable. 2024-01-24T23:18:49.583 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-01-24T23:18:49.623 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-01-24T23:18:49.646 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-mgr. 2024-01-24T23:18:49.673 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:49.713 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-mgr (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:49.909 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-rbd. 2024-01-24T23:18:49.924 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:49.967 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-rbd (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:50.107 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-mon. 2024-01-24T23:18:50.125 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:50.165 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-mon (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:50.304 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-01-24T23:18:50.330 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-01-24T23:18:50.362 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-01-24T23:18:50.615 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-01-24T23:18:50.641 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-01-24T23:18:50.681 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-01-24T23:18:50.769 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-osd. 2024-01-24T23:18:50.796 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:50.835 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-osd (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:50.985 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-01-24T23:18:51.011 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-01-24T23:18:51.510 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-01-24T23:18:52.005 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package lua5.1. 2024-01-24T23:18:52.031 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-01-24T23:18:52.063 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-01-24T23:18:52.322 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph. 2024-01-24T23:18:52.337 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../066-ceph_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:52.366 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package lua-any. 2024-01-24T23:18:52.371 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:52.381 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-01-24T23:18:52.416 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking lua-any (27ubuntu1) ... 2024-01-24T23:18:52.635 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package zip. 2024-01-24T23:18:52.650 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-01-24T23:18:52.693 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking zip (3.0-12build2) ... 2024-01-24T23:18:52.708 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-fuse. 2024-01-24T23:18:52.735 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:52.774 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-fuse (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:52.981 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package unzip. 2024-01-24T23:18:53.007 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.1_amd64.deb ... 2024-01-24T23:18:53.144 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-mds. 2024-01-24T23:18:53.150 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking unzip (6.0-26ubuntu3.1) ... 2024-01-24T23:18:53.171 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:53.211 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-mds (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:53.559 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package luarocks. 2024-01-24T23:18:53.572 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package cephadm. 2024-01-24T23:18:53.586 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-01-24T23:18:53.598 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../069-cephadm_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:53.626 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-01-24T23:18:53.630 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking cephadm (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:53.924 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-asyncssh. 2024-01-24T23:18:53.951 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-01-24T23:18:53.991 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-01-24T23:18:54.009 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package librgw2. 2024-01-24T23:18:54.036 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../025-librgw2_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:54.101 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking librgw2 (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:54.285 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python-babel-localedata. 2024-01-24T23:18:54.301 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-01-24T23:18:54.335 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-01-24T23:18:54.611 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-rgw. 2024-01-24T23:18:54.627 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:54.670 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-rgw (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:54.990 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package liboath0:amd64. 2024-01-24T23:18:55.016 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-01-24T23:18:55.056 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-01-24T23:18:55.382 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-babel. 2024-01-24T23:18:55.393 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libradosstriper1. 2024-01-24T23:18:55.409 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-01-24T23:18:55.419 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:55.448 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-01-24T23:18:55.459 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libradosstriper1 (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:55.784 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-jinja2. 2024-01-24T23:18:55.811 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1_all.deb ... 2024-01-24T23:18:55.852 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2024-01-24T23:18:55.888 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-common. 2024-01-24T23:18:55.914 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../029-ceph-common_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:55.955 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-common (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:56.187 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-01-24T23:18:56.214 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-431-g66387355-1jammy_all.deb ... 2024-01-24T23:18:56.253 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-mgr-cephadm (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:56.590 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-repoze.lru. 2024-01-24T23:18:56.617 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-01-24T23:18:56.656 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-01-24T23:18:56.951 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-routes. 2024-01-24T23:18:56.978 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-01-24T23:18:57.017 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-01-24T23:18:57.227 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-base. 2024-01-24T23:18:57.253 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../030-ceph-base_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:18:57.303 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-base (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:57.370 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-01-24T23:18:57.398 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-431-g66387355-1jammy_all.deb ... 2024-01-24T23:18:57.437 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-mgr-dashboard (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:18:57.865 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-01-24T23:18:57.880 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-01-24T23:18:57.915 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-01-24T23:18:58.073 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-01-24T23:18:58.089 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-01-24T23:18:58.148 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-01-24T23:18:58.210 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-cheroot. 2024-01-24T23:18:58.236 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-01-24T23:18:58.276 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-01-24T23:18:58.660 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-joblib. 2024-01-24T23:18:58.687 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-01-24T23:18:58.726 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-01-24T23:18:58.813 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-01-24T23:18:58.839 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-01-24T23:18:58.879 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-01-24T23:18:59.096 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-01-24T23:18:59.123 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-01-24T23:18:59.140 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-jaraco.text. 2024-01-24T23:18:59.156 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-01-24T23:18:59.162 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-01-24T23:18:59.190 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-01-24T23:18:59.465 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-sklearn. 2024-01-24T23:18:59.477 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-01-24T23:18:59.492 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-01-24T23:18:59.503 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-01-24T23:18:59.532 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-01-24T23:18:59.543 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-01-24T23:18:59.846 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-tempora. 2024-01-24T23:18:59.873 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-01-24T23:18:59.913 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-01-24T23:19:00.183 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-portend. 2024-01-24T23:19:00.190 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-01-24T23:19:00.206 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-431-g66387355-1jammy_all.deb ... 2024-01-24T23:19:00.209 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-01-24T23:19:00.249 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-01-24T23:19:00.334 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:00.511 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-01-24T23:19:00.537 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-01-24T23:19:00.577 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-01-24T23:19:00.889 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-cherrypy3. 2024-01-24T23:19:00.915 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-01-24T23:19:00.955 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-01-24T23:19:01.275 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-natsort. 2024-01-24T23:19:01.291 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-01-24T23:19:01.304 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-cachetools. 2024-01-24T23:19:01.325 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-01-24T23:19:01.331 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-01-24T23:19:01.370 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-01-24T23:19:01.629 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-logutils. 2024-01-24T23:19:01.648 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pyasn1. 2024-01-24T23:19:01.655 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-01-24T23:19:01.675 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-01-24T23:19:01.695 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-01-24T23:19:01.739 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-01-24T23:19:02.007 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-markupsafe. 2024-01-24T23:19:02.033 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-01-24T23:19:02.043 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-01-24T23:19:02.070 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-01-24T23:19:02.073 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-01-24T23:19:02.109 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-01-24T23:19:02.368 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-mako. 2024-01-24T23:19:02.387 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-rsa. 2024-01-24T23:19:02.395 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-01-24T23:19:02.415 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-01-24T23:19:02.426 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-01-24T23:19:02.454 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-rsa (4.8-1) ... 2024-01-24T23:19:02.696 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-simplegeneric. 2024-01-24T23:19:02.709 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-01-24T23:19:02.746 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-01-24T23:19:02.757 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-google-auth. 2024-01-24T23:19:02.784 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-01-24T23:19:02.823 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-01-24T23:19:03.091 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-singledispatch. 2024-01-24T23:19:03.118 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-01-24T23:19:03.174 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-01-24T23:19:03.185 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-01-24T23:19:03.213 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-01-24T23:19:03.252 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-01-24T23:19:03.469 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-webob. 2024-01-24T23:19:03.496 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-01-24T23:19:03.535 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-01-24T23:19:03.563 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-websocket. 2024-01-24T23:19:03.590 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-01-24T23:19:03.629 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-01-24T23:19:03.889 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-waitress. 2024-01-24T23:19:03.916 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-01-24T23:19:03.949 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-kubernetes. 2024-01-24T23:19:03.967 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-01-24T23:19:03.976 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-01-24T23:19:04.016 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-01-24T23:19:04.250 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-tempita. 2024-01-24T23:19:04.277 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-01-24T23:19:04.308 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-01-24T23:19:04.586 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-paste. 2024-01-24T23:19:04.613 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-01-24T23:19:04.653 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-01-24T23:19:04.730 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-01-24T23:19:04.758 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-431-g66387355-1jammy_all.deb ... 2024-01-24T23:19:04.796 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:04.981 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-01-24T23:19:04.997 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-01-24T23:19:05.031 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-01-24T23:19:05.115 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libonig5:amd64. 2024-01-24T23:19:05.142 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-01-24T23:19:05.182 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-01-24T23:19:05.318 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-pastedeploy. 2024-01-24T23:19:05.344 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-01-24T23:19:05.384 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-01-24T23:19:05.552 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libjq1:amd64. 2024-01-24T23:19:05.580 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-01-24T23:19:05.618 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-01-24T23:19:05.671 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-webtest. 2024-01-24T23:19:05.697 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-01-24T23:19:05.737 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-01-24T23:19:05.845 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package jq. 2024-01-24T23:19:05.862 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-01-24T23:19:05.904 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-01-24T23:19:06.032 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-pecan. 2024-01-24T23:19:06.059 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-01-24T23:19:06.098 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-01-24T23:19:06.156 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package socat. 2024-01-24T23:19:06.172 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-01-24T23:19:06.215 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-01-24T23:19:06.470 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-certifi. 2024-01-24T23:19:06.497 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-01-24T23:19:06.536 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-01-24T23:19:06.543 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package xmlstarlet. 2024-01-24T23:19:06.559 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-01-24T23:19:06.601 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-01-24T23:19:06.840 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-idna. 2024-01-24T23:19:06.855 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-01-24T23:19:06.898 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-idna (3.3-1) ... 2024-01-24T23:19:06.938 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-test. 2024-01-24T23:19:06.954 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../097-ceph-test_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:19:07.021 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-test (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:07.251 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-urllib3. 2024-01-24T23:19:07.278 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-01-24T23:19:07.318 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-01-24T23:19:07.638 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-requests. 2024-01-24T23:19:07.665 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-01-24T23:19:07.704 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-01-24T23:19:08.008 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-werkzeug. 2024-01-24T23:19:08.034 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-01-24T23:19:08.074 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-01-24T23:19:08.344 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-01-24T23:19:08.359 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-431-g66387355-1jammy_all.deb ... 2024-01-24T23:19:08.394 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-mgr-modules-core (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:08.758 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-01-24T23:19:08.786 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:19:08.824 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:09.119 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-bcrypt. 2024-01-24T23:19:09.145 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-01-24T23:19:09.185 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-01-24T23:19:09.217 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-volume. 2024-01-24T23:19:09.244 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-431-g66387355-1jammy_all.deb ... 2024-01-24T23:19:09.283 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-volume (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:09.488 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-openssl. 2024-01-24T23:19:09.514 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-01-24T23:19:09.563 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-01-24T23:19:09.587 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libcephfs-dev. 2024-01-24T23:19:09.615 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:19:09.654 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libcephfs-dev (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:09.925 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-mgr. 2024-01-24T23:19:09.952 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:19:09.973 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package lua-socket:amd64. 2024-01-24T23:19:09.991 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-mgr (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:10.000 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-01-24T23:19:10.039 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-01-24T23:19:10.352 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package lua-sec:amd64. 2024-01-24T23:19:10.353 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-mon. 2024-01-24T23:19:10.380 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:19:10.380 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-01-24T23:19:10.411 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-mon (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:10.418 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-01-24T23:19:10.731 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package nvme-cli. 2024-01-24T23:19:10.758 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-01-24T23:19:10.797 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-01-24T23:19:11.023 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-osd. 2024-01-24T23:19:11.050 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:19:11.090 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-osd (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:11.186 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package pkg-config. 2024-01-24T23:19:11.213 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-01-24T23:19:11.252 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-01-24T23:19:11.513 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-01-24T23:19:11.529 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-01-24T23:19:11.571 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-01-24T23:19:12.007 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-iniconfig. 2024-01-24T23:19:12.035 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-01-24T23:19:12.074 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-01-24T23:19:12.151 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph. 2024-01-24T23:19:12.167 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../066-ceph_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:19:12.210 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:12.368 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pastescript. 2024-01-24T23:19:12.396 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-01-24T23:19:12.435 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-01-24T23:19:12.479 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-fuse. 2024-01-24T23:19:12.495 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:19:12.646 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-fuse (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:12.855 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pluggy. 2024-01-24T23:19:12.882 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-01-24T23:19:12.921 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-01-24T23:19:13.033 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-mds. 2024-01-24T23:19:13.059 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:19:13.099 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-mds (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:13.199 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-psutil. 2024-01-24T23:19:13.226 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-01-24T23:19:13.257 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-01-24T23:19:13.478 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package cephadm. 2024-01-24T23:19:13.504 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../069-cephadm_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:19:13.544 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking cephadm (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:13.602 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-py. 2024-01-24T23:19:13.630 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-01-24T23:19:13.676 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-py (1.10.0-1) ... 2024-01-24T23:19:13.864 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-asyncssh. 2024-01-24T23:19:13.890 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-01-24T23:19:13.930 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-01-24T23:19:13.996 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pygments. 2024-01-24T23:19:14.024 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-01-24T23:19:14.063 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-01-24T23:19:14.483 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python-babel-localedata. 2024-01-24T23:19:14.508 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pyinotify. 2024-01-24T23:19:14.510 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-01-24T23:19:14.536 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-01-24T23:19:14.549 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-01-24T23:19:14.575 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-01-24T23:19:14.844 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-toml. 2024-01-24T23:19:14.871 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-01-24T23:19:14.910 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-01-24T23:19:15.155 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pytest. 2024-01-24T23:19:15.182 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-01-24T23:19:15.221 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-01-24T23:19:15.567 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-simplejson. 2024-01-24T23:19:15.571 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-babel. 2024-01-24T23:19:15.594 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-01-24T23:19:15.598 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-01-24T23:19:15.624 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-01-24T23:19:15.638 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-01-24T23:19:15.885 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-01-24T23:19:15.901 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-01-24T23:19:15.943 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-01-24T23:19:16.000 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-jinja2. 2024-01-24T23:19:16.026 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1_all.deb ... 2024-01-24T23:19:16.066 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2024-01-24T23:19:16.419 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-01-24T23:19:16.446 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-431-g66387355-1jammy_all.deb ... 2024-01-24T23:19:16.486 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-mgr-cephadm (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:16.731 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package radosgw. 2024-01-24T23:19:16.759 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../116-radosgw_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:19:16.797 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking radosgw (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:16.823 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-repoze.lru. 2024-01-24T23:19:16.849 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-01-24T23:19:16.889 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-01-24T23:19:17.167 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-routes. 2024-01-24T23:19:17.183 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-01-24T23:19:17.217 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-01-24T23:19:17.542 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package rbd-fuse. 2024-01-24T23:19:17.570 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:19:17.579 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-01-24T23:19:17.606 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-431-g66387355-1jammy_all.deb ... 2024-01-24T23:19:17.608 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking rbd-fuse (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:17.645 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-mgr-dashboard (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:17.887 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-01-24T23:19:18.041 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-01-24T23:19:18.056 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-01-24T23:19:18.099 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-01-24T23:19:18.138 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-01-24T23:19:18.264 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-01-24T23:19:18.524 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-01-24T23:19:18.602 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-joblib. 2024-01-24T23:19:18.629 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-01-24T23:19:18.669 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-01-24T23:19:18.878 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-01-24T23:19:19.032 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-01-24T23:19:19.058 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-01-24T23:19:19.097 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-01-24T23:19:19.350 INFO:teuthology.orchestra.run.smithi115.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-01-24T23:19:19.409 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-sklearn. 2024-01-24T23:19:19.415 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up cephadm (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:19.436 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-01-24T23:19:19.475 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-01-24T23:19:19.942 INFO:teuthology.orchestra.run.smithi115.stdout:Adding system user cephadm....done 2024-01-24T23:19:20.096 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-01-24T23:19:20.228 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-01-24T23:19:20.257 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-431-g66387355-1jammy_all.deb ... 2024-01-24T23:19:20.364 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-01-24T23:19:20.380 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:20.624 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-01-24T23:19:20.725 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-01-24T23:19:20.959 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-01-24T23:19:21.202 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-01-24T23:19:21.328 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-py (1.10.0-1) ... 2024-01-24T23:19:21.384 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-cachetools. 2024-01-24T23:19:21.411 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-01-24T23:19:21.442 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-01-24T23:19:21.646 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-01-24T23:19:21.703 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-pyasn1. 2024-01-24T23:19:21.730 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-01-24T23:19:21.761 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-01-24T23:19:22.006 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-01-24T23:19:22.073 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-01-24T23:19:22.100 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-01-24T23:19:22.140 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-01-24T23:19:22.266 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-01-24T23:19:22.443 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-rsa. 2024-01-24T23:19:22.470 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-01-24T23:19:22.509 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-rsa (4.8-1) ... 2024-01-24T23:19:22.526 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up unzip (6.0-26ubuntu3.1) ... 2024-01-24T23:19:22.760 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-01-24T23:19:22.838 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-google-auth. 2024-01-24T23:19:22.865 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-01-24T23:19:22.904 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-01-24T23:19:23.011 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-01-24T23:19:23.224 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-01-24T23:19:23.251 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-01-24T23:19:23.270 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-01-24T23:19:23.291 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-01-24T23:19:23.611 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-websocket. 2024-01-24T23:19:23.638 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-01-24T23:19:23.677 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-01-24T23:19:23.840 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-ceph-argparse (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:23.997 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-kubernetes. 2024-01-24T23:19:24.025 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-01-24T23:19:24.063 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-01-24T23:19:24.131 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-01-24T23:19:24.232 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-01-24T23:19:24.333 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-01-24T23:19:24.442 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-01-24T23:19:24.693 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-01-24T23:19:24.753 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-01-24T23:19:24.781 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-431-g66387355-1jammy_all.deb ... 2024-01-24T23:19:24.819 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:25.075 INFO:teuthology.orchestra.run.smithi115.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-01-24T23:19:25.108 INFO:teuthology.orchestra.run.smithi115.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-01-24T23:19:25.139 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libonig5:amd64. 2024-01-24T23:19:25.166 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-01-24T23:19:25.171 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-01-24T23:19:25.205 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-01-24T23:19:25.297 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-01-24T23:19:25.525 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libjq1:amd64. 2024-01-24T23:19:25.552 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-01-24T23:19:25.591 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-01-24T23:19:25.656 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-01-24T23:19:25.757 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-01-24T23:19:25.869 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package jq. 2024-01-24T23:19:25.885 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-01-24T23:19:25.927 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-01-24T23:19:26.009 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-01-24T23:19:26.180 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package socat. 2024-01-24T23:19:26.196 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-01-24T23:19:26.238 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-01-24T23:19:26.276 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-01-24T23:19:26.550 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package xmlstarlet. 2024-01-24T23:19:26.566 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-01-24T23:19:26.577 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up zip (3.0-12build2) ... 2024-01-24T23:19:26.599 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-01-24T23:19:26.703 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-01-24T23:19:26.938 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-test. 2024-01-24T23:19:26.965 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../097-ceph-test_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:19:27.004 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-test (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:27.364 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-01-24T23:19:27.614 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-01-24T23:19:27.716 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-01-24T23:19:27.817 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-01-24T23:19:28.109 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-01-24T23:19:28.494 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-01-24T23:19:28.770 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-01-24T23:19:29.146 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-01-24T23:19:29.158 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-volume. 2024-01-24T23:19:29.185 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-431-g66387355-1jammy_all.deb ... 2024-01-24T23:19:29.224 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-volume (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:29.439 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-01-24T23:19:29.520 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libcephfs-dev. 2024-01-24T23:19:29.548 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:19:29.586 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libcephfs-dev (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:29.774 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-01-24T23:19:30.000 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-01-24T23:19:30.117 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-ceph-common (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:30.206 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package lua-socket:amd64. 2024-01-24T23:19:30.234 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-01-24T23:19:30.273 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-01-24T23:19:30.435 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-idna (3.3-1) ... 2024-01-24T23:19:30.586 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package lua-sec:amd64. 2024-01-24T23:19:30.601 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-01-24T23:19:30.635 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-01-24T23:19:30.769 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-01-24T23:19:31.006 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package nvme-cli. 2024-01-24T23:19:31.034 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-01-24T23:19:31.072 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-01-24T23:19:31.495 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package pkg-config. 2024-01-24T23:19:31.522 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-01-24T23:19:31.561 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-01-24T23:19:31.806 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-01-24T23:19:31.833 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-01-24T23:19:31.872 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-01-24T23:19:31.925 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-01-24T23:19:32.176 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-01-24T23:19:32.275 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-iniconfig. 2024-01-24T23:19:32.303 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-01-24T23:19:32.310 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-toml (0.10.2-1) ... 2024-01-24T23:19:32.342 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-01-24T23:19:32.570 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-01-24T23:19:32.620 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-pastescript. 2024-01-24T23:19:32.647 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-01-24T23:19:32.678 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-01-24T23:19:32.696 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-01-24T23:19:32.813 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-01-24T23:19:32.939 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-pluggy. 2024-01-24T23:19:32.956 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-01-24T23:19:32.999 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-01-24T23:19:33.075 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-01-24T23:19:33.326 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-psutil. 2024-01-24T23:19:33.343 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-01-24T23:19:33.353 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-01-24T23:19:33.401 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-01-24T23:19:33.594 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-01-24T23:19:33.720 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-01-24T23:19:33.745 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-py. 2024-01-24T23:19:33.773 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-01-24T23:19:33.812 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-py (1.10.0-1) ... 2024-01-24T23:19:34.021 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-01-24T23:19:34.140 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-pygments. 2024-01-24T23:19:34.168 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-01-24T23:19:34.206 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-01-24T23:19:34.256 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-01-24T23:19:34.490 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-01-24T23:19:34.678 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-pyinotify. 2024-01-24T23:19:34.706 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-01-24T23:19:34.741 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-01-24T23:19:34.744 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-01-24T23:19:34.992 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-01-24T23:19:35.038 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-toml. 2024-01-24T23:19:35.066 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-01-24T23:19:35.105 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-01-24T23:19:35.260 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-01-24T23:19:35.374 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-pytest. 2024-01-24T23:19:35.386 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-01-24T23:19:35.402 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-01-24T23:19:35.432 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-01-24T23:19:35.670 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-01-24T23:19:35.786 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-simplejson. 2024-01-24T23:19:35.788 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-01-24T23:19:35.814 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-01-24T23:19:35.853 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-01-24T23:19:36.064 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-01-24T23:19:36.113 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-01-24T23:19:36.141 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-01-24T23:19:36.172 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-01-24T23:19:36.365 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-01-24T23:19:36.625 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-01-24T23:19:36.751 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-01-24T23:19:36.877 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-01-24T23:19:36.892 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package radosgw. 2024-01-24T23:19:36.921 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../116-radosgw_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:19:36.959 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking radosgw (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:36.967 INFO:teuthology.orchestra.run.smithi115.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-01-24T23:19:37.228 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-01-24T23:19:37.354 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-01-24T23:19:37.731 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-01-24T23:19:37.754 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package rbd-fuse. 2024-01-24T23:19:37.770 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-431-g66387355-1jammy_amd64.deb ... 2024-01-24T23:19:37.812 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking rbd-fuse (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:37.973 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up lua-any (27ubuntu1) ... 2024-01-24T23:19:38.116 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-portend (3.0.0-1) ... 2024-01-24T23:19:38.124 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-01-24T23:19:38.366 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-01-24T23:19:38.367 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-01-24T23:19:38.493 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2024-01-24T23:19:38.501 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-01-24T23:19:38.762 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-01-24T23:19:38.942 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-01-24T23:19:39.126 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-01-24T23:19:39.362 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-01-24T23:19:39.588 INFO:teuthology.orchestra.run.smithi132.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-01-24T23:19:39.621 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-01-24T23:19:39.731 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-rsa (4.8-1) ... 2024-01-24T23:19:39.829 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up cephadm (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:39.982 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-01-24T23:19:40.250 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-01-24T23:19:40.595 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-01-24T23:19:40.697 INFO:teuthology.orchestra.run.smithi132.stdout:Adding system user cephadm....done 2024-01-24T23:19:40.852 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-01-24T23:19:40.971 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-01-24T23:19:41.119 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-01-24T23:19:41.256 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-01-24T23:19:41.362 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-01-24T23:19:41.505 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-01-24T23:19:41.567 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-01-24T23:19:41.693 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up librados2 (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:41.781 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-01-24T23:19:41.819 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libsqlite3-mod-ceph (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:41.945 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-01-24T23:19:42.049 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-01-24T23:19:42.200 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-py (1.10.0-1) ... 2024-01-24T23:19:42.240 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libcephfs2 (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:42.367 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libradosstriper1 (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:42.492 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-01-24T23:19:42.519 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-01-24T23:19:42.769 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up librbd1 (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:42.879 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-01-24T23:19:42.894 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-mgr-modules-core (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:43.021 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-fuse (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:43.138 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-01-24T23:19:43.230 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-01-24T23:19:43.231 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-01-24T23:19:43.423 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up unzip (6.0-26ubuntu3.1) ... 2024-01-24T23:19:43.716 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-01-24T23:19:43.772 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libcephfs-dev (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:43.873 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-rados (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:43.982 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up librgw2 (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:44.051 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-01-24T23:19:44.116 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-01-24T23:19:44.385 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-01-24T23:19:44.645 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-ceph-argparse (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:44.904 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-01-24T23:19:45.022 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-01-24T23:19:45.140 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-01-24T23:19:45.240 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-01-24T23:19:45.278 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-rbd (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:45.396 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up rbd-fuse (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:45.497 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-rgw (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:45.499 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-01-24T23:19:45.598 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-cephfs (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:45.724 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-common (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:46.177 INFO:teuthology.orchestra.run.smithi132.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-01-24T23:19:46.207 INFO:teuthology.orchestra.run.smithi132.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-01-24T23:19:46.233 INFO:teuthology.orchestra.run.smithi115.stdout:Adding group ceph....done 2024-01-24T23:19:46.270 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-01-24T23:19:46.421 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-01-24T23:19:46.585 INFO:teuthology.orchestra.run.smithi115.stdout:Adding system user ceph....done 2024-01-24T23:19:46.762 INFO:teuthology.orchestra.run.smithi115.stdout:Setting system user ceph properties....done 2024-01-24T23:19:46.779 INFO:teuthology.orchestra.run.smithi115.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-01-24T23:19:46.789 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-01-24T23:19:46.913 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-01-24T23:19:46.915 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-01-24T23:19:47.166 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-01-24T23:19:47.227 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-01-24T23:19:47.425 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-01-24T23:19:47.735 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up zip (3.0-12build2) ... 2024-01-24T23:19:47.742 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-test (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:47.861 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-01-24T23:19:47.868 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up radosgw (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:48.332 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-01-24T23:19:48.332 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-01-24T23:19:48.546 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-01-24T23:19:48.789 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-01-24T23:19:48.845 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-base (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:48.907 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-01-24T23:19:49.008 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-01-24T23:19:49.177 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-01-24T23:19:49.300 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-01-24T23:19:49.671 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-mds (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:49.693 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-01-24T23:19:49.907 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-01-24T23:19:49.907 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-01-24T23:19:49.944 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-01-24T23:19:50.346 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-01-24T23:19:50.364 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-mgr (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:50.579 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-01-24T23:19:50.580 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-01-24T23:19:50.630 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-01-24T23:19:50.974 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-01-24T23:19:51.041 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-osd (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:51.209 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-01-24T23:19:51.325 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-01-24T23:19:51.325 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-01-24T23:19:51.326 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-ceph-common (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:51.619 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-idna (3.3-1) ... 2024-01-24T23:19:51.843 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-mgr-k8sevents (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:51.937 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-01-24T23:19:51.985 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:52.136 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-mon (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:52.337 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-01-24T23:19:52.337 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-01-24T23:19:52.804 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-mgr-cephadm (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:52.930 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:53.056 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-mgr-dashboard (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:53.142 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-01-24T23:19:53.207 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-volume (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:19:53.411 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-01-24T23:19:53.545 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-toml (0.10.2-1) ... 2024-01-24T23:19:53.551 INFO:teuthology.orchestra.run.smithi115.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-01-24T23:19:53.652 INFO:teuthology.orchestra.run.smithi115.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-01-24T23:19:53.805 INFO:teuthology.orchestra.run.smithi115.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-01-24T23:19:53.812 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-01-24T23:19:53.939 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-01-24T23:19:54.066 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-01-24T23:19:54.366 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-01-24T23:19:54.634 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-01-24T23:19:54.877 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-01-24T23:19:54.923 INFO:teuthology.orchestra.run.smithi115.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-01-24T23:19:55.028 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-01-24T23:19:55.321 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-01-24T23:19:55.555 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-01-24T23:19:55.568 INFO:teuthology.orchestra.run.smithi115.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-01-24T23:19:55.574 DEBUG:teuthology.parallel:result is None 2024-01-24T23:19:55.815 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-01-24T23:19:56.066 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-01-24T23:19:56.300 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-01-24T23:19:56.576 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-01-24T23:19:56.702 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-01-24T23:19:56.979 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-01-24T23:19:57.105 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-01-24T23:19:57.397 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-01-24T23:19:57.682 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-01-24T23:19:57.925 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-01-24T23:19:58.059 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-01-24T23:19:58.185 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-01-24T23:19:58.274 INFO:teuthology.orchestra.run.smithi132.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-01-24T23:19:58.511 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-01-24T23:19:58.638 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-01-24T23:19:59.014 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-01-24T23:19:59.257 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up lua-any (27ubuntu1) ... 2024-01-24T23:19:59.400 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-portend (3.0.0-1) ... 2024-01-24T23:19:59.651 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-01-24T23:19:59.777 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2024-01-24T23:20:00.094 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-01-24T23:20:00.387 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-01-24T23:20:00.663 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-01-24T23:20:00.764 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-rsa (4.8-1) ... 2024-01-24T23:20:01.007 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-01-24T23:20:01.267 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-01-24T23:20:01.620 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-01-24T23:20:02.005 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-01-24T23:20:02.265 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-01-24T23:20:02.593 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-01-24T23:20:02.719 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up librados2 (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:02.845 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libsqlite3-mod-ceph (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:02.971 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-01-24T23:20:03.315 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libcephfs2 (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:03.466 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libradosstriper1 (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:03.609 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-01-24T23:20:03.903 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up librbd1 (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:04.029 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-mgr-modules-core (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:04.155 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-fuse (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:04.365 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-01-24T23:20:04.365 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-01-24T23:20:04.856 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libcephfs-dev (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:04.957 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-rados (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:05.058 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up librgw2 (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:05.175 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-01-24T23:20:06.339 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-rbd (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:06.489 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up rbd-fuse (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:06.640 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-rgw (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:06.791 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-cephfs (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:06.942 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-common (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:07.510 INFO:teuthology.orchestra.run.smithi132.stdout:Adding group ceph....done 2024-01-24T23:20:07.903 INFO:teuthology.orchestra.run.smithi132.stdout:Adding system user ceph....done 2024-01-24T23:20:08.133 INFO:teuthology.orchestra.run.smithi132.stdout:Setting system user ceph properties....done 2024-01-24T23:20:08.149 INFO:teuthology.orchestra.run.smithi132.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-01-24T23:20:08.289 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-01-24T23:20:08.588 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-01-24T23:20:09.071 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-test (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:09.222 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up radosgw (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:09.694 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-01-24T23:20:09.694 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-01-24T23:20:10.174 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-base (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:10.561 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-01-24T23:20:11.059 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-mds (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:11.278 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-01-24T23:20:11.279 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-01-24T23:20:11.778 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-mgr (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:12.001 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-01-24T23:20:12.001 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-01-24T23:20:12.488 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-osd (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:12.796 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-01-24T23:20:12.796 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-01-24T23:20:13.273 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-mgr-k8sevents (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:13.390 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:13.566 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-mon (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:13.796 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-01-24T23:20:13.796 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-01-24T23:20:14.276 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-mgr-cephadm (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:14.427 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:14.561 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-mgr-dashboard (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:14.729 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-volume (19.0.0-431-g66387355-1jammy) ... 2024-01-24T23:20:15.107 INFO:teuthology.orchestra.run.smithi132.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-01-24T23:20:15.216 INFO:teuthology.orchestra.run.smithi132.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-01-24T23:20:15.385 INFO:teuthology.orchestra.run.smithi132.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-01-24T23:20:16.954 INFO:teuthology.orchestra.run.smithi132.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-01-24T23:20:17.662 INFO:teuthology.orchestra.run.smithi132.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-01-24T23:20:17.668 DEBUG:teuthology.parallel:result is None 2024-01-24T23:20:17.668 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=663873555d45def955d1cc09730bfece2847b821 2024-01-24T23:20:17.888 DEBUG:teuthology.orchestra.run.smithi115:> dpkg-query -W -f '${Version}' ceph 2024-01-24T23:20:17.908 INFO:teuthology.orchestra.run.smithi115.stdout:19.0.0-431-g66387355-1jammy 2024-01-24T23:20:17.909 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-431-g66387355-1jammy 2024-01-24T23:20:17.909 INFO:teuthology.task.install:The correct ceph version 19.0.0-431-g66387355-1jammy is installed. 2024-01-24T23:20:17.911 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=663873555d45def955d1cc09730bfece2847b821 2024-01-24T23:20:18.129 DEBUG:teuthology.orchestra.run.smithi132:> dpkg-query -W -f '${Version}' ceph 2024-01-24T23:20:18.148 INFO:teuthology.orchestra.run.smithi132.stdout:19.0.0-431-g66387355-1jammy 2024-01-24T23:20:18.148 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-431-g66387355-1jammy 2024-01-24T23:20:18.148 INFO:teuthology.task.install:The correct ceph version 19.0.0-431-g66387355-1jammy is installed. 2024-01-24T23:20:18.150 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-01-24T23:20:18.150 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-01-24T23:20:18.150 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-24T23:20:18.164 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-01-24T23:20:18.164 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-24T23:20:18.207 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-01-24T23:20:18.208 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-01-24T23:20:18.208 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/daemon-helper 2024-01-24T23:20:18.220 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-24T23:20:18.273 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-01-24T23:20:18.274 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/daemon-helper 2024-01-24T23:20:18.286 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-24T23:20:18.342 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-01-24T23:20:18.342 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-01-24T23:20:18.342 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-24T23:20:18.355 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-24T23:20:18.409 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-01-24T23:20:18.409 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-24T23:20:18.422 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-24T23:20:18.479 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-01-24T23:20:18.479 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-01-24T23:20:18.479 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/stdin-killer 2024-01-24T23:20:18.493 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-24T23:20:18.550 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-01-24T23:20:18.550 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/stdin-killer 2024-01-24T23:20:18.563 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-24T23:20:18.618 INFO:teuthology.run_tasks:Running task ceph... 2024-01-24T23:20:18.714 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-01-24T23:20:18.714 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 777 /var/log/ceph 2024-01-24T23:20:18.717 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 777 /var/log/ceph 2024-01-24T23:20:18.730 INFO:tasks.ceph:Disabling ceph logrotate... 2024-01-24T23:20:18.730 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-24T23:20:18.772 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-24T23:20:18.785 INFO:tasks.ceph:Creating extra log directories... 2024-01-24T23:20:18.785 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-24T23:20:18.828 DEBUG:teuthology.orchestra.run.smithi132:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-24T23:20:18.850 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-01-24T23:20:18.850 INFO:tasks.ceph:config {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos min': 10, 'paxos service trim min': 5, 'paxos trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(AUTH_BAD_CAPS\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': False, 'mon_bind_addrvec': True} 2024-01-24T23:20:18.850 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi/7530404', 'branch': 'wip-pereman2-buffer-space', 'description': 'rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_mon_workunits}', 'email': 'pdiazbou@redhat.com', 'first_in_suite': False, 'job_id': '7530404', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-pereman2-buffer-space'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos min': 10, 'paxos service trim min': 5, 'paxos trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(AUTH_BAD_CAPS\\)'], 'mon_bind_msgr2': False, 'sha1': '663873555d45def955d1cc09730bfece2847b821'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '663873555d45def955d1cc09730bfece2847b821'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pereman2-buffer-space', 'sha1': '663873555d45def955d1cc09730bfece2847b821'}}, 'owner': 'scheduled_pdiazbou@teuthology', 'priority': 200, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mon.d', 'mon.e', 'osd.0', 'osd.1', 'osd.2'], ['mon.f', 'mon.g', 'mon.h', 'mon.i', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0']], 'seed': 5127, 'sha1': '663873555d45def955d1cc09730bfece2847b821', 'sleep_before_teardown': 0, 'subset': '512/20000', 'suite': 'rados', 'suite_branch': 'wip-pereman2-buffer-space', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '663873555d45def955d1cc09730bfece2847b821', 'targets': {'smithi115.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCOXMkl/Ynk1TEpOh2tti2OtVJvrzbWqwmf6HW6tcfqp7Hap4hrTg3EZ+YA9lA1+CoFW/ptL+SxGxR4NazkOa4g=', 'smithi132.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFTTYi4YovFS/twV+b1Yr9q7ijDH+XVSKkJ/d6aK6l15sGLggq98xG8YzfvY/MD3pUEpsm2Ic3m9IPm0vVoa0UU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'mon_thrash': {'revive_delay': 90, 'thrash_delay': 1}}, {'workunit': {'clients': {'client.0': ['mon/pool_ops.sh', 'mon/crush_ops.sh', 'mon/osd.sh', 'mon/caps.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'c856e0f9244dd29a50697123b480d63f16a4ed81', 'timestamp': '2024-01-24_11:08:23', 'tube': 'smithi', 'user': 'pdiazbou', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-01-24T23:20:18.851 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-24T23:20:18.883 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-24T23:20:18.890 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m0777 -- /var/run/ceph 2024-01-24T23:20:18.932 DEBUG:teuthology.orchestra.run.smithi132:> sudo install -d -m0777 -- /var/run/ceph 2024-01-24T23:20:18.945 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-01-24T23:20:18.945 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/scratch_devs of=/dev/stdout 2024-01-24T23:20:18.990 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-24T23:20:18.990 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_1 2024-01-24T23:20:19.038 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-24T23:20:19.039 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-24T23:20:19.039 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 884 Links: 1 2024-01-24T23:20:19.039 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-24T23:20:19.039 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-01-24 23:14:51.526898218 +0000 2024-01-24T23:20:19.039 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-01-24 23:14:51.270900505 +0000 2024-01-24T23:20:19.039 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-01-24 23:14:51.270900505 +0000 2024-01-24T23:20:19.039 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2024-01-24T23:20:19.039 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-24T23:20:19.092 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-01-24T23:20:19.093 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-01-24T23:20:19.093 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000446893 s, 1.1 MB/s 2024-01-24T23:20:19.094 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-24T23:20:19.143 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_2 2024-01-24T23:20:19.190 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-24T23:20:19.190 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-24T23:20:19.190 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 895 Links: 1 2024-01-24T23:20:19.190 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-24T23:20:19.190 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-01-24 23:14:52.038893645 +0000 2024-01-24T23:20:19.190 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-01-24 23:14:51.766896075 +0000 2024-01-24T23:20:19.190 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-01-24 23:14:51.766896075 +0000 2024-01-24T23:20:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2024-01-24T23:20:19.191 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-24T23:20:19.244 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-01-24T23:20:19.244 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-01-24T23:20:19.245 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.00042095 s, 1.2 MB/s 2024-01-24T23:20:19.246 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-24T23:20:19.296 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_3 2024-01-24T23:20:19.342 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-24T23:20:19.342 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-24T23:20:19.342 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 918 Links: 1 2024-01-24T23:20:19.343 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-24T23:20:19.343 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-01-24 23:14:52.526889284 +0000 2024-01-24T23:20:19.343 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-01-24 23:14:52.274891537 +0000 2024-01-24T23:20:19.343 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-01-24 23:14:52.274891537 +0000 2024-01-24T23:20:19.343 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2024-01-24T23:20:19.343 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-24T23:20:19.397 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-01-24T23:20:19.397 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-01-24T23:20:19.397 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000381179 s, 1.3 MB/s 2024-01-24T23:20:19.398 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-24T23:20:19.448 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_4 2024-01-24T23:20:19.494 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-24T23:20:19.494 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-24T23:20:19.494 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 934 Links: 1 2024-01-24T23:20:19.495 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-24T23:20:19.495 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-01-24 23:14:53.030884783 +0000 2024-01-24T23:20:19.495 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-01-24 23:14:52.766887141 +0000 2024-01-24T23:20:19.495 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-01-24 23:14:52.766887141 +0000 2024-01-24T23:20:19.495 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2024-01-24T23:20:19.495 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-24T23:20:19.549 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-01-24T23:20:19.549 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-01-24T23:20:19.549 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.00038811 s, 1.3 MB/s 2024-01-24T23:20:19.550 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-24T23:20:19.600 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-01-24T23:20:19.600 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-01-24T23:20:19.600 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/scratch_devs of=/dev/stdout 2024-01-24T23:20:19.606 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-24T23:20:19.606 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_1 2024-01-24T23:20:19.652 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-24T23:20:19.652 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-24T23:20:19.652 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 884 Links: 1 2024-01-24T23:20:19.652 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-24T23:20:19.652 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-01-24 23:14:50.730258615 +0000 2024-01-24T23:20:19.652 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-01-24 23:14:50.498260687 +0000 2024-01-24T23:20:19.652 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-01-24 23:14:50.498260687 +0000 2024-01-24T23:20:19.652 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2024-01-24T23:20:19.653 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-24T23:20:19.706 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-01-24T23:20:19.706 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-01-24T23:20:19.706 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000436208 s, 1.2 MB/s 2024-01-24T23:20:19.707 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-24T23:20:19.759 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_2 2024-01-24T23:20:19.807 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-24T23:20:19.807 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-24T23:20:19.807 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 906 Links: 1 2024-01-24T23:20:19.807 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-24T23:20:19.808 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-01-24 23:14:51.214254292 +0000 2024-01-24T23:20:19.808 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-01-24 23:14:50.954256615 +0000 2024-01-24T23:20:19.808 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-01-24 23:14:50.954256615 +0000 2024-01-24T23:20:19.808 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2024-01-24T23:20:19.808 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-24T23:20:19.862 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-01-24T23:20:19.862 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-01-24T23:20:19.862 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000374753 s, 1.4 MB/s 2024-01-24T23:20:19.863 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-24T23:20:19.912 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_3 2024-01-24T23:20:19.959 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-24T23:20:19.959 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-24T23:20:19.960 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 922 Links: 1 2024-01-24T23:20:19.960 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-24T23:20:19.960 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-01-24 23:14:51.690250041 +0000 2024-01-24T23:20:19.960 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-01-24 23:14:51.430252363 +0000 2024-01-24T23:20:19.960 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-01-24 23:14:51.430252363 +0000 2024-01-24T23:20:19.960 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2024-01-24T23:20:19.960 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-24T23:20:20.017 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-01-24T23:20:20.017 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-01-24T23:20:20.018 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000352283 s, 1.5 MB/s 2024-01-24T23:20:20.019 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-24T23:20:20.068 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_4 2024-01-24T23:20:20.116 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-24T23:20:20.116 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-24T23:20:20.116 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 937 Links: 1 2024-01-24T23:20:20.116 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-24T23:20:20.116 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-01-24 23:14:52.158245861 +0000 2024-01-24T23:20:20.117 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-01-24 23:14:51.906248112 +0000 2024-01-24T23:20:20.117 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-01-24 23:14:51.906248112 +0000 2024-01-24T23:20:20.117 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2024-01-24T23:20:20.117 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-24T23:20:20.170 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-01-24T23:20:20.170 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-01-24T23:20:20.170 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000349695 s, 1.5 MB/s 2024-01-24T23:20:20.171 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-24T23:20:20.220 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-01-24T23:20:20.221 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi115.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi132.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-01-24T23:20:20.221 INFO:tasks.ceph:Generating config... 2024-01-24T23:20:20.222 INFO:tasks.ceph:[client] debug monc = 20 2024-01-24T23:20:20.222 INFO:tasks.ceph:[client] debug ms = 1 2024-01-24T23:20:20.222 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-01-24T23:20:20.222 INFO:tasks.ceph:[global] mon election default strategy = 3 2024-01-24T23:20:20.222 INFO:tasks.ceph:[global] ms bind msgr2 = False 2024-01-24T23:20:20.222 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-01-24T23:20:20.222 INFO:tasks.ceph:[global] ms type = async 2024-01-24T23:20:20.222 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-01-24T23:20:20.222 INFO:tasks.ceph:[mgr] debug ms = 1 2024-01-24T23:20:20.222 INFO:tasks.ceph:[mon] debug mon = 20 2024-01-24T23:20:20.223 INFO:tasks.ceph:[mon] debug ms = 1 2024-01-24T23:20:20.223 INFO:tasks.ceph:[mon] debug paxos = 20 2024-01-24T23:20:20.223 INFO:tasks.ceph:[mon] mon min osdmap epochs = 25 2024-01-24T23:20:20.223 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-01-24T23:20:20.223 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-01-24T23:20:20.223 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-01-24T23:20:20.223 INFO:tasks.ceph:[mon] mon scrub inject crc mismatch = 0.01 2024-01-24T23:20:20.223 INFO:tasks.ceph:[mon] mon scrub inject missing keys = 0.05 2024-01-24T23:20:20.223 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-01-24T23:20:20.223 INFO:tasks.ceph:[mon] paxos min = 10 2024-01-24T23:20:20.223 INFO:tasks.ceph:[mon] paxos service trim min = 5 2024-01-24T23:20:20.223 INFO:tasks.ceph:[mon] paxos trim min = 10 2024-01-24T23:20:20.223 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-01-24T23:20:20.223 INFO:tasks.ceph:[osd] bluestore compression algorithm = snappy 2024-01-24T23:20:20.223 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-01-24T23:20:20.224 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-01-24T23:20:20.224 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-01-24T23:20:20.224 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-01-24T23:20:20.224 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-01-24T23:20:20.224 INFO:tasks.ceph:[osd] debug ms = 1 2024-01-24T23:20:20.224 INFO:tasks.ceph:[osd] debug osd = 20 2024-01-24T23:20:20.224 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-01-24T23:20:20.224 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-01-24T23:20:20.224 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-01-24T23:20:20.224 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-01-24T23:20:20.224 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-01-24T23:20:20.224 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-01-24T23:20:20.224 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-01-24T23:20:20.224 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-01-24T23:20:20.224 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-01-24T23:20:20.225 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-01-24T23:20:20.225 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-01-24T23:20:20.225 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-01-24T23:20:20.225 INFO:tasks.ceph:Setting up mon.a... 2024-01-24T23:20:20.225 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-01-24T23:20:20.260 INFO:teuthology.orchestra.run.smithi115.stdout:creating /etc/ceph/ceph.keyring 2024-01-24T23:20:20.267 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-01-24T23:20:20.344 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-24T23:20:20.403 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '[v1:172.21.15.115:6789]'), ('mon.b', '[v1:172.21.15.115:6790]'), ('mon.c', '[v1:172.21.15.115:6791]'), ('mon.d', '[v1:172.21.15.115:6792]'), ('mon.e', '[v1:172.21.15.115:6793]'), ('mon.f', '[v1:172.21.15.132:6789]'), ('mon.g', '[v1:172.21.15.132:6790]'), ('mon.h', '[v1:172.21.15.132:6791]'), ('mon.i', '[v1:172.21.15.132:6792]')] 2024-01-24T23:20:20.404 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '[v1:172.21.15.115:6789],[v1:172.21.15.115:6790],[v1:172.21.15.115:6791],[v1:172.21.15.115:6792],[v1:172.21.15.115:6793],[v1:172.21.15.132:6789],[v1:172.21.15.132:6790],[v1:172.21.15.132:6791],[v1:172.21.15.132:6792]', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos min': 10, 'paxos service trim min': 5, 'paxos trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug monc': 20, 'debug ms': 1}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mon.d': {}, 'mon.e': {}, 'mon.f': {}, 'mon.g': {}, 'mon.h': {}, 'mon.i': {}} 2024-01-24T23:20:20.405 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-01-24T23:20:20.406 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-01-24T23:20:20.450 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --addv a '[v1:172.21.15.115:6789]' --addv b '[v1:172.21.15.115:6790]' --addv c '[v1:172.21.15.115:6791]' --addv d '[v1:172.21.15.115:6792]' --addv e '[v1:172.21.15.115:6793]' --addv f '[v1:172.21.15.132:6789]' --addv g '[v1:172.21.15.132:6790]' --addv h '[v1:172.21.15.132:6791]' --addv i '[v1:172.21.15.132:6792]' --print /home/ubuntu/cephtest/ceph.monmap 2024-01-24T23:20:20.556 INFO:teuthology.orchestra.run.smithi115.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-01-24T23:20:20.556 INFO:teuthology.orchestra.run.smithi115.stdout:monmaptool: generated fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:20.557 INFO:teuthology.orchestra.run.smithi115.stdout:setting min_mon_release = quincy 2024-01-24T23:20:20.557 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 0 2024-01-24T23:20:20.557 INFO:teuthology.orchestra.run.smithi115.stdout:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:20.557 INFO:teuthology.orchestra.run.smithi115.stdout:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:20.557 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:20.557 INFO:teuthology.orchestra.run.smithi115.stdout:min_mon_release 17 (quincy) 2024-01-24T23:20:20.557 INFO:teuthology.orchestra.run.smithi115.stdout:election_strategy: 3 2024-01-24T23:20:20.557 INFO:teuthology.orchestra.run.smithi115.stdout:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:20.557 INFO:teuthology.orchestra.run.smithi115.stdout:1: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:20.557 INFO:teuthology.orchestra.run.smithi115.stdout:2: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:20.557 INFO:teuthology.orchestra.run.smithi115.stdout:3: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:20.558 INFO:teuthology.orchestra.run.smithi115.stdout:4: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:20.558 INFO:teuthology.orchestra.run.smithi115.stdout:5: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:20.558 INFO:teuthology.orchestra.run.smithi115.stdout:6: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:20.558 INFO:teuthology.orchestra.run.smithi115.stdout:7: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:20.558 INFO:teuthology.orchestra.run.smithi115.stdout:8: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:20.558 INFO:teuthology.orchestra.run.smithi115.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (9 monitors) 2024-01-24T23:20:20.560 DEBUG:teuthology.orchestra.run.smithi115:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-01-24T23:20:20.565 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 4c8039f0-ed94-40df-8309-0a291ebf7f77... 2024-01-24T23:20:20.566 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-24T23:20:20.607 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-24T23:20:20.632 INFO:teuthology.orchestra.run.smithi115.stdout:[global] 2024-01-24T23:20:20.632 INFO:teuthology.orchestra.run.smithi115.stdout: chdir = "" 2024-01-24T23:20:20.633 INFO:teuthology.orchestra.run.smithi115.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-24T23:20:20.633 INFO:teuthology.orchestra.run.smithi115.stdout: auth supported = cephx 2024-01-24T23:20:20.633 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.633 INFO:teuthology.orchestra.run.smithi115.stdout: filestore xattr use omap = true 2024-01-24T23:20:20.633 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.633 INFO:teuthology.orchestra.run.smithi115.stdout: mon clock drift allowed = 1.000 2024-01-24T23:20:20.634 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.634 INFO:teuthology.orchestra.run.smithi115.stdout: osd crush chooseleaf type = 0 2024-01-24T23:20:20.634 INFO:teuthology.orchestra.run.smithi115.stdout: auth debug = true 2024-01-24T23:20:20.634 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.634 INFO:teuthology.orchestra.run.smithi115.stdout: ms die on old message = true 2024-01-24T23:20:20.634 INFO:teuthology.orchestra.run.smithi115.stdout: ms die on bug = true 2024-01-24T23:20:20.635 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.635 INFO:teuthology.orchestra.run.smithi115.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-24T23:20:20.635 INFO:teuthology.orchestra.run.smithi115.stdout: mon pg warn max object skew = 0 2024-01-24T23:20:20.635 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.635 INFO:teuthology.orchestra.run.smithi115.stdout: # disable pg_autoscaler by default for new pools 2024-01-24T23:20:20.636 INFO:teuthology.orchestra.run.smithi115.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-24T23:20:20.636 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.636 INFO:teuthology.orchestra.run.smithi115.stdout: osd pool default size = 2 2024-01-24T23:20:20.636 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.636 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd allow primary affinity = true 2024-01-24T23:20:20.636 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd allow pg remap = true 2024-01-24T23:20:20.637 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on legacy crush tunables = false 2024-01-24T23:20:20.637 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on crush straw calc version zero = false 2024-01-24T23:20:20.637 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on no sortbitwise = false 2024-01-24T23:20:20.637 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on osd down out interval zero = false 2024-01-24T23:20:20.637 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on too few osds = false 2024-01-24T23:20:20.638 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-24T23:20:20.638 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-24T23:20:20.638 INFO:teuthology.orchestra.run.smithi115.stdout: mon_allow_pool_size_one = true 2024-01-24T23:20:20.638 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.638 INFO:teuthology.orchestra.run.smithi115.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-01-24T23:20:20.638 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.639 INFO:teuthology.orchestra.run.smithi115.stdout: osd default data pool replay window = 5 2024-01-24T23:20:20.639 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.639 INFO:teuthology.orchestra.run.smithi115.stdout: mon allow pool delete = true 2024-01-24T23:20:20.640 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.640 INFO:teuthology.orchestra.run.smithi115.stdout: mon cluster log file level = debug 2024-01-24T23:20:20.640 INFO:teuthology.orchestra.run.smithi115.stdout: debug asserts on shutdown = true 2024-01-24T23:20:20.641 INFO:teuthology.orchestra.run.smithi115.stdout: mon health detail to clog = false 2024-01-24T23:20:20.641 INFO:teuthology.orchestra.run.smithi115.stdout: mon host = "[v1:172.21.15.115:6789],[v1:172.21.15.115:6790],[v1:172.21.15.115:6791],[v1:172.21.15.115:6792],[v1:172.21.15.115:6793],[v1:172.21.15.132:6789],[v1:172.21.15.132:6790],[v1:172.21.15.132:6791],[v1:172.21.15.132:6792]" 2024-01-24T23:20:20.641 INFO:teuthology.orchestra.run.smithi115.stdout: mon client directed command retry = 5 2024-01-24T23:20:20.641 INFO:teuthology.orchestra.run.smithi115.stdout: mon election default strategy = 3 2024-01-24T23:20:20.641 INFO:teuthology.orchestra.run.smithi115.stdout: ms bind msgr2 = False 2024-01-24T23:20:20.641 INFO:teuthology.orchestra.run.smithi115.stdout: ms inject socket failures = 5000 2024-01-24T23:20:20.642 INFO:teuthology.orchestra.run.smithi115.stdout: ms type = async 2024-01-24T23:20:20.642 INFO:teuthology.orchestra.run.smithi115.stdout: fsid = 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:20.642 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.642 INFO:teuthology.orchestra.run.smithi115.stdout:[osd] 2024-01-24T23:20:20.642 INFO:teuthology.orchestra.run.smithi115.stdout: osd journal size = 100 2024-01-24T23:20:20.642 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.642 INFO:teuthology.orchestra.run.smithi115.stdout: osd scrub load threshold = 5.0 2024-01-24T23:20:20.642 INFO:teuthology.orchestra.run.smithi115.stdout: osd scrub max interval = 600 2024-01-24T23:20:20.642 INFO:teuthology.orchestra.run.smithi115.stdout: osd mclock profile = high_recovery_ops 2024-01-24T23:20:20.642 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.643 INFO:teuthology.orchestra.run.smithi115.stdout: osd recover clone overlap = true 2024-01-24T23:20:20.643 INFO:teuthology.orchestra.run.smithi115.stdout: osd recovery max chunk = 1048576 2024-01-24T23:20:20.643 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.643 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug shutdown = true 2024-01-24T23:20:20.643 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug op order = true 2024-01-24T23:20:20.643 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug verify stray on activate = true 2024-01-24T23:20:20.643 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug trim objects = true 2024-01-24T23:20:20.643 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.643 INFO:teuthology.orchestra.run.smithi115.stdout: osd open classes on start = true 2024-01-24T23:20:20.644 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug pg log writeout = true 2024-01-24T23:20:20.644 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.644 INFO:teuthology.orchestra.run.smithi115.stdout: osd deep scrub update digest min age = 30 2024-01-24T23:20:20.644 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.644 INFO:teuthology.orchestra.run.smithi115.stdout: osd map max advance = 10 2024-01-24T23:20:20.644 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.644 INFO:teuthology.orchestra.run.smithi115.stdout: journal zero on create = true 2024-01-24T23:20:20.644 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.644 INFO:teuthology.orchestra.run.smithi115.stdout: filestore ondisk finisher threads = 3 2024-01-24T23:20:20.644 INFO:teuthology.orchestra.run.smithi115.stdout: filestore apply finisher threads = 3 2024-01-24T23:20:20.645 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.645 INFO:teuthology.orchestra.run.smithi115.stdout: bdev debug aio = true 2024-01-24T23:20:20.645 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug misdirected ops = true 2024-01-24T23:20:20.645 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore block size = 96636764160 2024-01-24T23:20:20.645 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore compression algorithm = snappy 2024-01-24T23:20:20.645 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore compression mode = aggressive 2024-01-24T23:20:20.645 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore fsck on mount = True 2024-01-24T23:20:20.645 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore zero block detection = True 2024-01-24T23:20:20.645 INFO:teuthology.orchestra.run.smithi115.stdout: debug bluefs = 20 2024-01-24T23:20:20.645 INFO:teuthology.orchestra.run.smithi115.stdout: debug bluestore = 20 2024-01-24T23:20:20.645 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2024-01-24T23:20:20.645 INFO:teuthology.orchestra.run.smithi115.stdout: debug osd = 20 2024-01-24T23:20:20.645 INFO:teuthology.orchestra.run.smithi115.stdout: debug rocksdb = 10 2024-01-24T23:20:20.646 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd backfillfull_ratio = 0.85 2024-01-24T23:20:20.646 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd full ratio = 0.9 2024-01-24T23:20:20.646 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd nearfull ratio = 0.8 2024-01-24T23:20:20.646 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug verify cached snaps = True 2024-01-24T23:20:20.646 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug verify missing on start = True 2024-01-24T23:20:20.646 INFO:teuthology.orchestra.run.smithi115.stdout: osd failsafe full ratio = 0.95 2024-01-24T23:20:20.646 INFO:teuthology.orchestra.run.smithi115.stdout: osd mclock override recovery settings = True 2024-01-24T23:20:20.646 INFO:teuthology.orchestra.run.smithi115.stdout: osd objectstore = bluestore 2024-01-24T23:20:20.646 INFO:teuthology.orchestra.run.smithi115.stdout: osd op queue = debug_random 2024-01-24T23:20:20.646 INFO:teuthology.orchestra.run.smithi115.stdout: osd op queue cut off = debug_random 2024-01-24T23:20:20.646 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.646 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr] 2024-01-24T23:20:20.646 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2024-01-24T23:20:20.646 INFO:teuthology.orchestra.run.smithi115.stdout: debug mgr = 20 2024-01-24T23:20:20.647 INFO:teuthology.orchestra.run.smithi115.stdout: debug mon = 20 2024-01-24T23:20:20.647 INFO:teuthology.orchestra.run.smithi115.stdout: debug auth = 20 2024-01-24T23:20:20.647 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min pgs per osd = 4 2024-01-24T23:20:20.647 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min bytes per osd = 10 2024-01-24T23:20:20.647 INFO:teuthology.orchestra.run.smithi115.stdout: mgr/telemetry/nag = false 2024-01-24T23:20:20.647 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.647 INFO:teuthology.orchestra.run.smithi115.stdout:[mon] 2024-01-24T23:20:20.648 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2024-01-24T23:20:20.648 INFO:teuthology.orchestra.run.smithi115.stdout: debug mon = 20 2024-01-24T23:20:20.648 INFO:teuthology.orchestra.run.smithi115.stdout: debug paxos = 20 2024-01-24T23:20:20.648 INFO:teuthology.orchestra.run.smithi115.stdout: debug auth = 20 2024-01-24T23:20:20.648 INFO:teuthology.orchestra.run.smithi115.stdout: mon data avail warn = 5 2024-01-24T23:20:20.648 INFO:teuthology.orchestra.run.smithi115.stdout: mon mgr mkfs grace = 240 2024-01-24T23:20:20.648 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min pgs per osd = 4 2024-01-24T23:20:20.648 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd reporter subtree level = osd 2024-01-24T23:20:20.648 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd prime pg temp = true 2024-01-24T23:20:20.648 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min bytes per osd = 10 2024-01-24T23:20:20.648 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.648 INFO:teuthology.orchestra.run.smithi115.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-24T23:20:20.648 INFO:teuthology.orchestra.run.smithi115.stdout: auth mon ticket ttl = 660 # 11m 2024-01-24T23:20:20.648 INFO:teuthology.orchestra.run.smithi115.stdout: auth service ticket ttl = 240 # 4m 2024-01-24T23:20:20.649 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.649 INFO:teuthology.orchestra.run.smithi115.stdout: # don't complain about insecure global_id in the test suite 2024-01-24T23:20:20.649 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-24T23:20:20.649 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-24T23:20:20.649 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.649 INFO:teuthology.orchestra.run.smithi115.stdout: # 1m isn't quite enough 2024-01-24T23:20:20.649 INFO:teuthology.orchestra.run.smithi115.stdout: mon_down_mkfs_grace = 2m 2024-01-24T23:20:20.649 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.649 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_filestore_osds = false 2024-01-24T23:20:20.649 INFO:teuthology.orchestra.run.smithi115.stdout: mon min osdmap epochs = 25 2024-01-24T23:20:20.649 INFO:teuthology.orchestra.run.smithi115.stdout: mon osdmap full prune interval = 2 2024-01-24T23:20:20.649 INFO:teuthology.orchestra.run.smithi115.stdout: mon osdmap full prune min = 15 2024-01-24T23:20:20.649 INFO:teuthology.orchestra.run.smithi115.stdout: mon osdmap full prune txsize = 2 2024-01-24T23:20:20.650 INFO:teuthology.orchestra.run.smithi115.stdout: mon scrub inject crc mismatch = 0.01 2024-01-24T23:20:20.650 INFO:teuthology.orchestra.run.smithi115.stdout: mon scrub inject missing keys = 0.05 2024-01-24T23:20:20.650 INFO:teuthology.orchestra.run.smithi115.stdout: mon scrub interval = 300 2024-01-24T23:20:20.650 INFO:teuthology.orchestra.run.smithi115.stdout: paxos min = 10 2024-01-24T23:20:20.650 INFO:teuthology.orchestra.run.smithi115.stdout: paxos service trim min = 5 2024-01-24T23:20:20.650 INFO:teuthology.orchestra.run.smithi115.stdout: paxos trim min = 10 2024-01-24T23:20:20.650 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:20.650 INFO:teuthology.orchestra.run.smithi115.stdout:[client] 2024-01-24T23:20:20.650 INFO:teuthology.orchestra.run.smithi115.stdout: rgw cache enabled = true 2024-01-24T23:20:20.650 INFO:teuthology.orchestra.run.smithi115.stdout: rgw enable ops log = true 2024-01-24T23:20:20.650 INFO:teuthology.orchestra.run.smithi115.stdout: rgw enable usage log = true 2024-01-24T23:20:20.650 INFO:teuthology.orchestra.run.smithi115.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-24T23:20:20.650 INFO:teuthology.orchestra.run.smithi115.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-24T23:20:20.650 INFO:teuthology.orchestra.run.smithi115.stdout: debug monc = 20 2024-01-24T23:20:20.650 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2024-01-24T23:20:20.651 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.a] 2024-01-24T23:20:20.651 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.b] 2024-01-24T23:20:20.651 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.c] 2024-01-24T23:20:20.651 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.d] 2024-01-24T23:20:20.651 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.e] 2024-01-24T23:20:20.651 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.f] 2024-01-24T23:20:20.651 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.g] 2024-01-24T23:20:20.651 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.h] 2024-01-24T23:20:20.651 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.i] 2024-01-24T23:20:20.653 INFO:teuthology.orchestra.run.smithi132.stdout:[global] 2024-01-24T23:20:20.653 INFO:teuthology.orchestra.run.smithi132.stdout: chdir = "" 2024-01-24T23:20:20.653 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-24T23:20:20.654 INFO:teuthology.orchestra.run.smithi132.stdout: auth supported = cephx 2024-01-24T23:20:20.654 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.654 INFO:teuthology.orchestra.run.smithi132.stdout: filestore xattr use omap = true 2024-01-24T23:20:20.654 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.654 INFO:teuthology.orchestra.run.smithi132.stdout: mon clock drift allowed = 1.000 2024-01-24T23:20:20.654 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.654 INFO:teuthology.orchestra.run.smithi132.stdout: osd crush chooseleaf type = 0 2024-01-24T23:20:20.654 INFO:teuthology.orchestra.run.smithi132.stdout: auth debug = true 2024-01-24T23:20:20.654 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.655 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on old message = true 2024-01-24T23:20:20.655 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on bug = true 2024-01-24T23:20:20.655 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.655 INFO:teuthology.orchestra.run.smithi132.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-24T23:20:20.655 INFO:teuthology.orchestra.run.smithi132.stdout: mon pg warn max object skew = 0 2024-01-24T23:20:20.655 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.655 INFO:teuthology.orchestra.run.smithi132.stdout: # disable pg_autoscaler by default for new pools 2024-01-24T23:20:20.655 INFO:teuthology.orchestra.run.smithi132.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-24T23:20:20.655 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.655 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default size = 2 2024-01-24T23:20:20.655 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.655 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow primary affinity = true 2024-01-24T23:20:20.656 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow pg remap = true 2024-01-24T23:20:20.656 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on legacy crush tunables = false 2024-01-24T23:20:20.656 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on crush straw calc version zero = false 2024-01-24T23:20:20.656 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on no sortbitwise = false 2024-01-24T23:20:20.656 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on osd down out interval zero = false 2024-01-24T23:20:20.656 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on too few osds = false 2024-01-24T23:20:20.656 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-24T23:20:20.656 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-24T23:20:20.656 INFO:teuthology.orchestra.run.smithi132.stdout: mon_allow_pool_size_one = true 2024-01-24T23:20:20.656 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.657 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-01-24T23:20:20.657 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.657 INFO:teuthology.orchestra.run.smithi132.stdout: osd default data pool replay window = 5 2024-01-24T23:20:20.657 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.657 INFO:teuthology.orchestra.run.smithi132.stdout: mon allow pool delete = true 2024-01-24T23:20:20.657 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.657 INFO:teuthology.orchestra.run.smithi132.stdout: mon cluster log file level = debug 2024-01-24T23:20:20.657 INFO:teuthology.orchestra.run.smithi132.stdout: debug asserts on shutdown = true 2024-01-24T23:20:20.657 INFO:teuthology.orchestra.run.smithi132.stdout: mon health detail to clog = false 2024-01-24T23:20:20.657 INFO:teuthology.orchestra.run.smithi132.stdout: mon host = "[v1:172.21.15.115:6789],[v1:172.21.15.115:6790],[v1:172.21.15.115:6791],[v1:172.21.15.115:6792],[v1:172.21.15.115:6793],[v1:172.21.15.132:6789],[v1:172.21.15.132:6790],[v1:172.21.15.132:6791],[v1:172.21.15.132:6792]" 2024-01-24T23:20:20.657 INFO:teuthology.orchestra.run.smithi132.stdout: mon client directed command retry = 5 2024-01-24T23:20:20.657 INFO:teuthology.orchestra.run.smithi132.stdout: mon election default strategy = 3 2024-01-24T23:20:20.657 INFO:teuthology.orchestra.run.smithi132.stdout: ms bind msgr2 = False 2024-01-24T23:20:20.658 INFO:teuthology.orchestra.run.smithi132.stdout: ms inject socket failures = 5000 2024-01-24T23:20:20.658 INFO:teuthology.orchestra.run.smithi132.stdout: ms type = async 2024-01-24T23:20:20.658 INFO:teuthology.orchestra.run.smithi132.stdout: fsid = 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:20.658 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.658 INFO:teuthology.orchestra.run.smithi132.stdout:[osd] 2024-01-24T23:20:20.658 INFO:teuthology.orchestra.run.smithi132.stdout: osd journal size = 100 2024-01-24T23:20:20.658 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.658 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub load threshold = 5.0 2024-01-24T23:20:20.658 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub max interval = 600 2024-01-24T23:20:20.658 INFO:teuthology.orchestra.run.smithi132.stdout: osd mclock profile = high_recovery_ops 2024-01-24T23:20:20.658 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.658 INFO:teuthology.orchestra.run.smithi132.stdout: osd recover clone overlap = true 2024-01-24T23:20:20.658 INFO:teuthology.orchestra.run.smithi132.stdout: osd recovery max chunk = 1048576 2024-01-24T23:20:20.658 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.659 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug shutdown = true 2024-01-24T23:20:20.659 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug op order = true 2024-01-24T23:20:20.659 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify stray on activate = true 2024-01-24T23:20:20.659 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug trim objects = true 2024-01-24T23:20:20.659 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.659 INFO:teuthology.orchestra.run.smithi132.stdout: osd open classes on start = true 2024-01-24T23:20:20.659 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug pg log writeout = true 2024-01-24T23:20:20.659 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.659 INFO:teuthology.orchestra.run.smithi132.stdout: osd deep scrub update digest min age = 30 2024-01-24T23:20:20.659 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.659 INFO:teuthology.orchestra.run.smithi132.stdout: osd map max advance = 10 2024-01-24T23:20:20.659 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.659 INFO:teuthology.orchestra.run.smithi132.stdout: journal zero on create = true 2024-01-24T23:20:20.660 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.660 INFO:teuthology.orchestra.run.smithi132.stdout: filestore ondisk finisher threads = 3 2024-01-24T23:20:20.660 INFO:teuthology.orchestra.run.smithi132.stdout: filestore apply finisher threads = 3 2024-01-24T23:20:20.660 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.660 INFO:teuthology.orchestra.run.smithi132.stdout: bdev debug aio = true 2024-01-24T23:20:20.660 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug misdirected ops = true 2024-01-24T23:20:20.660 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore block size = 96636764160 2024-01-24T23:20:20.661 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore compression algorithm = snappy 2024-01-24T23:20:20.661 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore compression mode = aggressive 2024-01-24T23:20:20.661 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore fsck on mount = True 2024-01-24T23:20:20.661 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore zero block detection = True 2024-01-24T23:20:20.661 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluefs = 20 2024-01-24T23:20:20.661 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluestore = 20 2024-01-24T23:20:20.661 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-01-24T23:20:20.661 INFO:teuthology.orchestra.run.smithi132.stdout: debug osd = 20 2024-01-24T23:20:20.661 INFO:teuthology.orchestra.run.smithi132.stdout: debug rocksdb = 10 2024-01-24T23:20:20.661 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd backfillfull_ratio = 0.85 2024-01-24T23:20:20.661 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd full ratio = 0.9 2024-01-24T23:20:20.661 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd nearfull ratio = 0.8 2024-01-24T23:20:20.662 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify cached snaps = True 2024-01-24T23:20:20.662 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify missing on start = True 2024-01-24T23:20:20.662 INFO:teuthology.orchestra.run.smithi132.stdout: osd failsafe full ratio = 0.95 2024-01-24T23:20:20.662 INFO:teuthology.orchestra.run.smithi132.stdout: osd mclock override recovery settings = True 2024-01-24T23:20:20.662 INFO:teuthology.orchestra.run.smithi132.stdout: osd objectstore = bluestore 2024-01-24T23:20:20.662 INFO:teuthology.orchestra.run.smithi132.stdout: osd op queue = debug_random 2024-01-24T23:20:20.662 INFO:teuthology.orchestra.run.smithi132.stdout: osd op queue cut off = debug_random 2024-01-24T23:20:20.662 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.662 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr] 2024-01-24T23:20:20.662 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-01-24T23:20:20.662 INFO:teuthology.orchestra.run.smithi132.stdout: debug mgr = 20 2024-01-24T23:20:20.662 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2024-01-24T23:20:20.662 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2024-01-24T23:20:20.662 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2024-01-24T23:20:20.663 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2024-01-24T23:20:20.663 INFO:teuthology.orchestra.run.smithi132.stdout: mgr/telemetry/nag = false 2024-01-24T23:20:20.663 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.663 INFO:teuthology.orchestra.run.smithi132.stdout:[mon] 2024-01-24T23:20:20.663 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-01-24T23:20:20.663 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2024-01-24T23:20:20.663 INFO:teuthology.orchestra.run.smithi132.stdout: debug paxos = 20 2024-01-24T23:20:20.663 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2024-01-24T23:20:20.663 INFO:teuthology.orchestra.run.smithi132.stdout: mon data avail warn = 5 2024-01-24T23:20:20.663 INFO:teuthology.orchestra.run.smithi132.stdout: mon mgr mkfs grace = 240 2024-01-24T23:20:20.663 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2024-01-24T23:20:20.663 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd reporter subtree level = osd 2024-01-24T23:20:20.663 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd prime pg temp = true 2024-01-24T23:20:20.663 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2024-01-24T23:20:20.664 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.664 INFO:teuthology.orchestra.run.smithi132.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-24T23:20:20.664 INFO:teuthology.orchestra.run.smithi132.stdout: auth mon ticket ttl = 660 # 11m 2024-01-24T23:20:20.664 INFO:teuthology.orchestra.run.smithi132.stdout: auth service ticket ttl = 240 # 4m 2024-01-24T23:20:20.664 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.664 INFO:teuthology.orchestra.run.smithi132.stdout: # don't complain about insecure global_id in the test suite 2024-01-24T23:20:20.664 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-24T23:20:20.664 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-24T23:20:20.664 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.664 INFO:teuthology.orchestra.run.smithi132.stdout: # 1m isn't quite enough 2024-01-24T23:20:20.665 INFO:teuthology.orchestra.run.smithi132.stdout: mon_down_mkfs_grace = 2m 2024-01-24T23:20:20.665 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.665 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_filestore_osds = false 2024-01-24T23:20:20.665 INFO:teuthology.orchestra.run.smithi132.stdout: mon min osdmap epochs = 25 2024-01-24T23:20:20.665 INFO:teuthology.orchestra.run.smithi132.stdout: mon osdmap full prune interval = 2 2024-01-24T23:20:20.665 INFO:teuthology.orchestra.run.smithi132.stdout: mon osdmap full prune min = 15 2024-01-24T23:20:20.665 INFO:teuthology.orchestra.run.smithi132.stdout: mon osdmap full prune txsize = 2 2024-01-24T23:20:20.665 INFO:teuthology.orchestra.run.smithi132.stdout: mon scrub inject crc mismatch = 0.01 2024-01-24T23:20:20.665 INFO:teuthology.orchestra.run.smithi132.stdout: mon scrub inject missing keys = 0.05 2024-01-24T23:20:20.665 INFO:teuthology.orchestra.run.smithi132.stdout: mon scrub interval = 300 2024-01-24T23:20:20.665 INFO:teuthology.orchestra.run.smithi132.stdout: paxos min = 10 2024-01-24T23:20:20.665 INFO:teuthology.orchestra.run.smithi132.stdout: paxos service trim min = 5 2024-01-24T23:20:20.665 INFO:teuthology.orchestra.run.smithi132.stdout: paxos trim min = 10 2024-01-24T23:20:20.665 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-01-24T23:20:20.665 INFO:teuthology.orchestra.run.smithi132.stdout:[client] 2024-01-24T23:20:20.666 INFO:teuthology.orchestra.run.smithi132.stdout: rgw cache enabled = true 2024-01-24T23:20:20.666 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable ops log = true 2024-01-24T23:20:20.666 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable usage log = true 2024-01-24T23:20:20.666 INFO:teuthology.orchestra.run.smithi132.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-24T23:20:20.666 INFO:teuthology.orchestra.run.smithi132.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-24T23:20:20.666 INFO:teuthology.orchestra.run.smithi132.stdout: debug monc = 20 2024-01-24T23:20:20.666 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-01-24T23:20:20.666 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.a] 2024-01-24T23:20:20.666 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.b] 2024-01-24T23:20:20.666 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.c] 2024-01-24T23:20:20.666 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.d] 2024-01-24T23:20:20.666 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.e] 2024-01-24T23:20:20.666 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.f] 2024-01-24T23:20:20.667 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.g] 2024-01-24T23:20:20.667 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.h] 2024-01-24T23:20:20.667 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.i] 2024-01-24T23:20:20.667 INFO:tasks.ceph:Creating admin key on mon.a... 2024-01-24T23:20:20.667 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-01-24T23:20:20.746 INFO:tasks.ceph:Copying monmap to all nodes... 2024-01-24T23:20:20.747 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-01-24T23:20:20.747 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-01-24T23:20:20.752 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-01-24T23:20:20.752 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-01-24T23:20:20.798 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi115.front.sepia.ceph.com 2024-01-24T23:20:20.799 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-01-24T23:20:20.799 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-24T23:20:20.799 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-24T23:20:20.863 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-01-24T23:20:20.863 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-24T23:20:20.912 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi132.front.sepia.ceph.com 2024-01-24T23:20:20.912 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-01-24T23:20:20.912 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-24T23:20:20.912 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-24T23:20:20.933 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-01-24T23:20:20.933 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-24T23:20:20.979 INFO:tasks.ceph:Setting up mon nodes... 2024-01-24T23:20:20.980 INFO:tasks.ceph:Setting up mgr nodes... 2024-01-24T23:20:20.980 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-01-24T23:20:21.065 INFO:teuthology.orchestra.run.smithi132.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-01-24T23:20:21.083 INFO:tasks.ceph:Setting up mds nodes... 2024-01-24T23:20:21.083 INFO:tasks.ceph_client:Setting up client nodes... 2024-01-24T23:20:21.083 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-01-24T23:20:21.160 INFO:teuthology.orchestra.run.smithi132.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-01-24T23:20:21.170 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-01-24T23:20:21.171 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi115.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi132.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-01-24T23:20:21.171 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-01-24T23:20:21.185 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-01-24T23:20:21.185 INFO:tasks.ceph:role: osd.0 2024-01-24T23:20:21.185 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi115.front.sepia.ceph.com 2024-01-24T23:20:21.185 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-24T23:20:21.246 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-24T23:20:21.246 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-24T23:20:21.246 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-24T23:20:21.246 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-24T23:20:21.246 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-24T23:20:21.246 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-01-24T23:20:21.246 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-24T23:20:21.246 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-24T23:20:21.247 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-24T23:20:21.247 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-24T23:20:21.701 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-01-24T23:20:21.703 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-01-24T23:20:21.703 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-01-24T23:20:21.797 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-01-24T23:20:21.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T23:20:21.806 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: /sbin/restorecon: command not found 2024-01-24T23:20:21.806 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-01-24T23:20:21.860 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-01-24T23:20:21.860 INFO:tasks.ceph:role: osd.1 2024-01-24T23:20:21.860 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi115.front.sepia.ceph.com 2024-01-24T23:20:21.860 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-24T23:20:21.922 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-24T23:20:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-24T23:20:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-24T23:20:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-24T23:20:21.923 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-24T23:20:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-01-24T23:20:21.923 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-24T23:20:21.923 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-24T23:20:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-24T23:20:21.923 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-24T23:20:22.393 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-01-24T23:20:22.395 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-01-24T23:20:22.395 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-01-24T23:20:22.537 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-01-24T23:20:22.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T23:20:22.549 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: /sbin/restorecon: command not found 2024-01-24T23:20:22.550 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-01-24T23:20:22.605 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-01-24T23:20:22.605 INFO:tasks.ceph:role: osd.2 2024-01-24T23:20:22.606 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi115.front.sepia.ceph.com 2024-01-24T23:20:22.606 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-01-24T23:20:22.670 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-01-24T23:20:22.670 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-24T23:20:22.670 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-24T23:20:22.670 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-24T23:20:22.670 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-24T23:20:22.670 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-01-24T23:20:22.671 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-24T23:20:22.671 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-24T23:20:22.671 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-24T23:20:22.671 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-24T23:20:23.136 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-01-24T23:20:23.138 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-01-24T23:20:23.138 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-01-24T23:20:23.286 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-01-24T23:20:23.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T23:20:23.298 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: /sbin/restorecon: command not found 2024-01-24T23:20:23.298 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-24T23:20:23.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:23.408+0000 7f8b72357640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-01-24T23:20:23.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:23.412+0000 7f8b72357640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-01-24T23:20:23.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:23.412+0000 7f8b72357640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-24T23:20:23.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:23.412+0000 7f8b72357640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-24T23:20:23.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:23.412+0000 7f8b72357640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-24T23:20:23.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:23.412+0000 7f8b72357640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-01-24T23:20:26.423 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-01-24T23:20:26.437 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-24T23:20:26.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:26.512+0000 7f2abf3d8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-01-24T23:20:26.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:26.512+0000 7f2abf3d8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-01-24T23:20:26.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:26.512+0000 7f2abf3d8640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-24T23:20:26.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:26.512+0000 7f2abf3d8640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-24T23:20:26.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:26.512+0000 7f2abf3d8640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-24T23:20:26.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:26.512+0000 7f2abf3d8640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-01-24T23:20:29.497 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-01-24T23:20:29.555 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-24T23:20:29.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:29.632+0000 7f87f70b9640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-01-24T23:20:29.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:29.632+0000 7f87f70b9640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-01-24T23:20:29.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:29.632+0000 7f87f70b9640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-24T23:20:29.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:29.632+0000 7f87f70b9640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-24T23:20:29.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:29.632+0000 7f87f70b9640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-24T23:20:29.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:29.632+0000 7f87f70b9640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-01-24T23:20:32.610 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-01-24T23:20:32.670 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-01-24T23:20:32.685 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-01-24T23:20:32.685 INFO:tasks.ceph:role: osd.3 2024-01-24T23:20:32.685 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi132.front.sepia.ceph.com 2024-01-24T23:20:32.685 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-24T23:20:32.749 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-24T23:20:32.749 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-24T23:20:32.749 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-24T23:20:32.750 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-24T23:20:32.750 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-24T23:20:32.750 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2024-01-24T23:20:32.750 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-24T23:20:32.750 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-24T23:20:32.750 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-24T23:20:32.750 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-24T23:20:33.205 INFO:teuthology.orchestra.run.smithi132.stdout:Discarding blocks...Done. 2024-01-24T23:20:33.206 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2024-01-24T23:20:33.206 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-01-24T23:20:33.301 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-01-24T23:20:33.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T23:20:33.311 INFO:teuthology.orchestra.run.smithi132.stderr:sudo: /sbin/restorecon: command not found 2024-01-24T23:20:33.311 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-01-24T23:20:33.364 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-01-24T23:20:33.364 INFO:tasks.ceph:role: osd.4 2024-01-24T23:20:33.364 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi132.front.sepia.ceph.com 2024-01-24T23:20:33.364 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-24T23:20:33.431 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-24T23:20:33.431 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-24T23:20:33.431 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-24T23:20:33.431 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-24T23:20:33.431 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-24T23:20:33.432 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2024-01-24T23:20:33.432 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-24T23:20:33.432 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-24T23:20:33.432 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-24T23:20:33.432 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-24T23:20:33.897 INFO:teuthology.orchestra.run.smithi132.stdout:Discarding blocks...Done. 2024-01-24T23:20:33.898 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2024-01-24T23:20:33.898 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-01-24T23:20:34.031 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-01-24T23:20:34.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T23:20:34.043 INFO:teuthology.orchestra.run.smithi132.stderr:sudo: /sbin/restorecon: command not found 2024-01-24T23:20:34.043 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-01-24T23:20:34.099 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-01-24T23:20:34.099 INFO:tasks.ceph:role: osd.5 2024-01-24T23:20:34.099 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi132.front.sepia.ceph.com 2024-01-24T23:20:34.099 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-01-24T23:20:34.163 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-01-24T23:20:34.163 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-24T23:20:34.163 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-24T23:20:34.164 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-24T23:20:34.164 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-24T23:20:34.164 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2024-01-24T23:20:34.164 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-24T23:20:34.164 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-24T23:20:34.164 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-24T23:20:34.164 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-24T23:20:34.658 INFO:teuthology.orchestra.run.smithi132.stdout:Discarding blocks...Done. 2024-01-24T23:20:34.659 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2024-01-24T23:20:34.660 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-01-24T23:20:34.806 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-01-24T23:20:34.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T23:20:34.820 INFO:teuthology.orchestra.run.smithi132.stderr:sudo: /sbin/restorecon: command not found 2024-01-24T23:20:34.820 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-24T23:20:34.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:34.929+0000 7f4dec5bb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-01-24T23:20:34.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:34.929+0000 7f4dec5bb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-01-24T23:20:34.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:34.929+0000 7f4dec5bb640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-24T23:20:34.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:34.929+0000 7f4dec5bb640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-24T23:20:34.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:34.929+0000 7f4dec5bb640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-24T23:20:34.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:34.929+0000 7f4dec5bb640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-01-24T23:20:37.911 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-01-24T23:20:37.925 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-24T23:20:38.004 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:38.001+0000 7f4e5ad75640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-01-24T23:20:38.005 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:38.001+0000 7f4e5ad75640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-01-24T23:20:38.005 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:38.001+0000 7f4e5ad75640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-24T23:20:38.005 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:38.001+0000 7f4e5ad75640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-24T23:20:38.005 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:38.001+0000 7f4e5ad75640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-24T23:20:38.006 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:38.001+0000 7f4e5ad75640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-01-24T23:20:40.970 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-01-24T23:20:41.028 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-24T23:20:41.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:41.105+0000 7f5d0a6c7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-01-24T23:20:41.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:41.105+0000 7f5d0a6c7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-01-24T23:20:41.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:41.105+0000 7f5d0a6c7640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-24T23:20:41.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:41.105+0000 7f5d0a6c7640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-24T23:20:41.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:41.105+0000 7f5d0a6c7640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-24T23:20:41.110 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:41.105+0000 7f5d0a6c7640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-01-24T23:20:44.085 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-01-24T23:20:44.140 INFO:tasks.ceph:Reading keys from all nodes... 2024-01-24T23:20:44.140 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-01-24T23:20:44.140 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-01-24T23:20:44.154 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-01-24T23:20:44.154 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-01-24T23:20:44.209 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-01-24T23:20:44.209 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-01-24T23:20:44.265 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-01-24T23:20:44.265 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-01-24T23:20:44.278 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-01-24T23:20:44.279 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-01-24T23:20:44.334 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-01-24T23:20:44.334 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-01-24T23:20:44.385 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-01-24T23:20:44.385 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-01-24T23:20:44.437 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-01-24T23:20:44.437 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-01-24T23:20:44.483 INFO:tasks.ceph:Adding keys to all mons... 2024-01-24T23:20:44.483 DEBUG:teuthology.orchestra.run.smithi115:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-24T23:20:44.485 DEBUG:teuthology.orchestra.run.smithi132:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-24T23:20:44.529 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.0] 2024-01-24T23:20:44.529 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQA3m7FlzAm2GBAAg1neVm06z5V1oWi2mw81xg== 2024-01-24T23:20:44.529 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.1] 2024-01-24T23:20:44.529 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQA6m7FlwFPCHhAAuhVRiEX4bFoZIN9aiIlVFw== 2024-01-24T23:20:44.529 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.2] 2024-01-24T23:20:44.529 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQA9m7FlMhrzJRAA4GTJHjLBXBJ8N9ca+Y954g== 2024-01-24T23:20:44.529 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr.x] 2024-01-24T23:20:44.530 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQA1m7FldSLyAxAADj3HNUe/wPEyjG14gR2wZw== 2024-01-24T23:20:44.530 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.3] 2024-01-24T23:20:44.530 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBCm7Fl/U++NxAAx90vz9BqaGWQk1yATrInoA== 2024-01-24T23:20:44.530 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.4] 2024-01-24T23:20:44.530 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBGm7Flse1nABAAHDxcCSLxYK0QiuOkv3dklw== 2024-01-24T23:20:44.530 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.5] 2024-01-24T23:20:44.530 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBJm7FlVfSbBhAAm2y4Cfr4j5aPd2o2xVCgUA== 2024-01-24T23:20:44.530 INFO:teuthology.orchestra.run.smithi115.stdout:[client.0] 2024-01-24T23:20:44.530 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQA1m7Fl45SlCRAAy3qu5D81NF8AiBfxklE0Uw== 2024-01-24T23:20:44.536 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.0] 2024-01-24T23:20:44.536 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQA3m7FlzAm2GBAAg1neVm06z5V1oWi2mw81xg== 2024-01-24T23:20:44.537 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.1] 2024-01-24T23:20:44.537 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQA6m7FlwFPCHhAAuhVRiEX4bFoZIN9aiIlVFw== 2024-01-24T23:20:44.537 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.2] 2024-01-24T23:20:44.537 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQA9m7FlMhrzJRAA4GTJHjLBXBJ8N9ca+Y954g== 2024-01-24T23:20:44.537 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr.x] 2024-01-24T23:20:44.537 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQA1m7FldSLyAxAADj3HNUe/wPEyjG14gR2wZw== 2024-01-24T23:20:44.537 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.3] 2024-01-24T23:20:44.537 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQBCm7Fl/U++NxAAx90vz9BqaGWQk1yATrInoA== 2024-01-24T23:20:44.537 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.4] 2024-01-24T23:20:44.537 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQBGm7Flse1nABAAHDxcCSLxYK0QiuOkv3dklw== 2024-01-24T23:20:44.537 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.5] 2024-01-24T23:20:44.537 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQBJm7FlVfSbBhAAm2y4Cfr4j5aPd2o2xVCgUA== 2024-01-24T23:20:44.537 INFO:teuthology.orchestra.run.smithi132.stdout:[client.0] 2024-01-24T23:20:44.538 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQA1m7Fl45SlCRAAy3qu5D81NF8AiBfxklE0Uw== 2024-01-24T23:20:44.538 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T23:20:44.575 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T23:20:44.627 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T23:20:44.629 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T23:20:44.710 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T23:20:44.712 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T23:20:44.799 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-24T23:20:44.801 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-24T23:20:44.844 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T23:20:44.887 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T23:20:44.927 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T23:20:44.971 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T23:20:45.011 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T23:20:45.055 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-24T23:20:45.095 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-24T23:20:45.135 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-24T23:20:45.181 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-01-24T23:20:45.181 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-01-24T23:20:45.192 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-24T23:20:45.285 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-01-24T23:20:45.296 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-01-24T23:20:45.347 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-24T23:20:45.437 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-01-24T23:20:45.493 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-01-24T23:20:45.549 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-24T23:20:45.642 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-01-24T23:20:45.652 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mon/ceph-d 2024-01-24T23:20:45.703 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i d --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-24T23:20:45.788 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-d 2024-01-24T23:20:45.841 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mon/ceph-e 2024-01-24T23:20:45.897 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i e --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-24T23:20:45.985 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-e 2024-01-24T23:20:46.036 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mon/ceph-f 2024-01-24T23:20:46.050 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i f --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-24T23:20:46.144 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-f 2024-01-24T23:20:46.155 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mon/ceph-g 2024-01-24T23:20:46.208 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i g --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-24T23:20:46.296 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-g 2024-01-24T23:20:46.307 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mon/ceph-h 2024-01-24T23:20:46.364 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i h --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-24T23:20:46.454 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-h 2024-01-24T23:20:46.465 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mon/ceph-i 2024-01-24T23:20:46.516 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i i --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-24T23:20:46.602 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-i 2024-01-24T23:20:46.654 DEBUG:teuthology.orchestra.run.smithi115:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-24T23:20:46.657 DEBUG:teuthology.orchestra.run.smithi132:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-24T23:20:46.703 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-01-24T23:20:46.703 INFO:tasks.ceph.mon.a:Restarting daemon 2024-01-24T23:20:46.703 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-01-24T23:20:46.706 INFO:tasks.ceph.mon.a:Started 2024-01-24T23:20:46.706 INFO:tasks.ceph.mon.b:Restarting daemon 2024-01-24T23:20:46.706 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-01-24T23:20:46.708 INFO:tasks.ceph.mon.b:Started 2024-01-24T23:20:46.708 INFO:tasks.ceph.mon.c:Restarting daemon 2024-01-24T23:20:46.708 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-01-24T23:20:46.710 INFO:tasks.ceph.mon.c:Started 2024-01-24T23:20:46.711 INFO:tasks.ceph.mon.d:Restarting daemon 2024-01-24T23:20:46.711 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i d 2024-01-24T23:20:46.713 INFO:tasks.ceph.mon.d:Started 2024-01-24T23:20:46.713 INFO:tasks.ceph.mon.e:Restarting daemon 2024-01-24T23:20:46.713 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2024-01-24T23:20:46.715 INFO:tasks.ceph.mon.e:Started 2024-01-24T23:20:46.716 INFO:tasks.ceph.mon.f:Restarting daemon 2024-01-24T23:20:46.716 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-01-24T23:20:46.748 INFO:tasks.ceph.mon.f:Started 2024-01-24T23:20:46.748 INFO:tasks.ceph.mon.g:Restarting daemon 2024-01-24T23:20:46.748 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-01-24T23:20:46.750 INFO:tasks.ceph.mon.g:Started 2024-01-24T23:20:46.751 INFO:tasks.ceph.mon.h:Restarting daemon 2024-01-24T23:20:46.751 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i h 2024-01-24T23:20:46.753 INFO:tasks.ceph.mon.h:Started 2024-01-24T23:20:46.754 INFO:tasks.ceph.mon.i:Restarting daemon 2024-01-24T23:20:46.754 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-01-24T23:20:46.756 INFO:tasks.ceph.mon.i:Started 2024-01-24T23:20:46.756 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-01-24T23:20:46.756 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-01-24T23:20:46.756 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-01-24T23:20:46.758 INFO:tasks.ceph.mgr.x:Started 2024-01-24T23:20:46.759 INFO:tasks.ceph:Setting crush tunables to default 2024-01-24T23:20:46.759 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd crush tunables default 2024-01-24T23:20:46.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:46.891+0000 7f973e902640 10 monclient: get_monmap_and_config 2024-01-24T23:20:46.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:46.891+0000 7f973e902640 10 monclient: build_initial_monmap 2024-01-24T23:20:46.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:46.891+0000 7f973e902640 10 monclient: monmap: 2024-01-24T23:20:46.895 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:20:46.895 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:46.896 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:46.896069+0000 2024-01-24T23:20:46.896 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:46.896069+0000 2024-01-24T23:20:46.896 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:20:46.897 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:20:46.897 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:20:46.897 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:20:46.897 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:20:46.897 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:20:46.897 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:20:46.897 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:20:46.897 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:20:46.897 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:20:46.897 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:20:46.897 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:20:46.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:46.891+0000 7f973e902640 1 Processor -- start 2024-01-24T23:20:46.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:46.891+0000 7f973e902640 1 -- start start 2024-01-24T23:20:46.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:46.891+0000 7f973e902640 10 monclient: init 2024-01-24T23:20:46.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:46.891+0000 7f973e902640 10 monclient: _reopen_session rank -1 2024-01-24T23:20:46.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:46.891+0000 7f973e902640 10 monclient: _add_conns ranks=[8,0,4,3,6,2,1,5,7] 2024-01-24T23:20:46.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:46.891+0000 7f973e902640 10 monclient(hunting): picked mon.noname-e con 0x7f9738133bf0 addr v1:172.21.15.115:6793/0 2024-01-24T23:20:46.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:46.891+0000 7f973e902640 10 monclient(hunting): picked mon.noname-a con 0x7f9738133fd0 addr v1:172.21.15.115:6789/0 2024-01-24T23:20:46.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:46.891+0000 7f973e902640 10 monclient(hunting): picked mon.noname-c con 0x7f97381343b0 addr v1:172.21.15.115:6791/0 2024-01-24T23:20:46.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:46.891+0000 7f973e902640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973804ed70 con 0x7f9738133fd0 2024-01-24T23:20:46.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:46.891+0000 7f973e902640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97380a4b60 con 0x7f97381343b0 2024-01-24T23:20:46.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:46.891+0000 7f973e902640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973809ae50 con 0x7f9738133bf0 2024-01-24T23:20:46.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:46.891+0000 7f973e902640 10 monclient(hunting): _renew_subs 2024-01-24T23:20:46.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:46.891+0000 7f973e902640 10 monclient(hunting): authenticate will time out at 925.818416s 2024-01-24T23:20:46.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:46.891+0000 7f973ce9f640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f97381343b0 0x7f973813f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:49302/0 (socket says 172.21.15.115:49302) 2024-01-24T23:20:46.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:46.891+0000 7f9737fff640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f9738133bf0 0x7f973809bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:50890/0 (socket says 172.21.15.115:50890) 2024-01-24T23:20:46.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:46.891+0000 7f973ce9f640 1 -- 172.21.15.115:0/1391143716 learned_addr learned my addr 172.21.15.115:0/1391143716 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:20:46.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:46.891+0000 7f97377fe640 1 --1- 172.21.15.115:0/1391143716 >> v1:172.21.15.115:6789/0 conn(0x7f9738133fd0 0x7f973813bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59048/0 (socket says 172.21.15.115:59048) 2024-01-24T23:20:49.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:49.891+0000 7f9735ffb640 10 monclient(hunting): tick 2024-01-24T23:20:49.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:49.891+0000 7f9735ffb640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:20:49.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:49.891+0000 7f9735ffb640 1 monclient(hunting): continuing hunt 2024-01-24T23:20:49.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:49.891+0000 7f9735ffb640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:20:49.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:49.891+0000 7f9735ffb640 1 -- 172.21.15.115:0/1391143716 >> v1:172.21.15.115:6793/0 conn(0x7f9738133bf0 legacy=0x7f973809bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:49.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:49.891+0000 7f9735ffb640 1 -- 172.21.15.115:0/1391143716 >> v1:172.21.15.115:6791/0 conn(0x7f97381343b0 legacy=0x7f973813f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:49.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:49.891+0000 7f9735ffb640 1 -- 172.21.15.115:0/1391143716 >> v1:172.21.15.115:6789/0 conn(0x7f9738133fd0 legacy=0x7f973813bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:49.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:49.891+0000 7f9735ffb640 10 monclient: _add_conns ranks=[6,7,2,1,5,3] 2024-01-24T23:20:49.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:49.891+0000 7f9735ffb640 10 monclient(hunting): picked mon.noname-d con 0x7f97240037f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:20:49.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:49.891+0000 7f9735ffb640 10 monclient(hunting): picked mon.noname-i con 0x7f9724007480 addr v1:172.21.15.132:6792/0 2024-01-24T23:20:49.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:49.891+0000 7f9735ffb640 10 monclient(hunting): picked mon.noname-b con 0x7f972400b210 addr v1:172.21.15.115:6790/0 2024-01-24T23:20:49.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:49.891+0000 7f9735ffb640 1 -- 172.21.15.115:0/1391143716 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973809b130 con 0x7f972400b210 2024-01-24T23:20:49.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:49.891+0000 7f9735ffb640 1 -- 172.21.15.115:0/1391143716 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9724010020 con 0x7f97240037f0 2024-01-24T23:20:49.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:49.891+0000 7f9735ffb640 1 -- 172.21.15.115:0/1391143716 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9724011250 con 0x7f9724007480 2024-01-24T23:20:49.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:49.891+0000 7f9735ffb640 10 monclient(hunting): _renew_subs 2024-01-24T23:20:52.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:52.891+0000 7f9735ffb640 10 monclient(hunting): tick 2024-01-24T23:20:52.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:52.891+0000 7f9735ffb640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:20:52.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:52.891+0000 7f9735ffb640 1 monclient(hunting): continuing hunt 2024-01-24T23:20:52.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:52.891+0000 7f9735ffb640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:20:52.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:52.891+0000 7f9735ffb640 1 -- 172.21.15.115:0/1391143716 >> v1:172.21.15.132:6792/0 conn(0x7f9724007480 legacy=0x7f9724009940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:52.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:52.891+0000 7f9735ffb640 1 -- 172.21.15.115:0/1391143716 >> v1:172.21.15.115:6792/0 conn(0x7f97240037f0 legacy=0x7f9724005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:52.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:52.891+0000 7f9735ffb640 1 -- 172.21.15.115:0/1391143716 >> v1:172.21.15.115:6790/0 conn(0x7f972400b210 legacy=0x7f972400d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:52.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:52.891+0000 7f973ce9f640 1 -- 172.21.15.115:0/1391143716 reap_dead start 2024-01-24T23:20:52.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:52.891+0000 7f9735ffb640 10 monclient: _add_conns ranks=[1,5,3] 2024-01-24T23:20:52.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:52.891+0000 7f973ce9f640 1 -- 172.21.15.115:0/1391143716 reap_dead start 2024-01-24T23:20:52.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:52.891+0000 7f9735ffb640 10 monclient(hunting): picked mon.noname-f con 0x7f9724010430 addr v1:172.21.15.132:6789/0 2024-01-24T23:20:52.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:52.891+0000 7f9735ffb640 10 monclient(hunting): picked mon.noname-h con 0x7f9724010840 addr v1:172.21.15.132:6791/0 2024-01-24T23:20:52.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:52.891+0000 7f9735ffb640 10 monclient(hunting): picked mon.noname-g con 0x7f97240113e0 addr v1:172.21.15.132:6790/0 2024-01-24T23:20:52.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:52.891+0000 7f9735ffb640 1 -- 172.21.15.115:0/1391143716 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9724012480 con 0x7f9724010430 2024-01-24T23:20:52.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:52.891+0000 7f9735ffb640 1 -- 172.21.15.115:0/1391143716 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9724011030 con 0x7f97240113e0 2024-01-24T23:20:52.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:52.891+0000 7f9735ffb640 1 -- 172.21.15.115:0/1391143716 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9724011fe0 con 0x7f9724010840 2024-01-24T23:20:52.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:52.891+0000 7f9735ffb640 10 monclient(hunting): _renew_subs 2024-01-24T23:20:55.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:55.891+0000 7f9735ffb640 10 monclient(hunting): tick 2024-01-24T23:20:55.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:55.891+0000 7f9735ffb640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:20:55.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:55.891+0000 7f9735ffb640 1 monclient(hunting): continuing hunt 2024-01-24T23:20:55.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:55.891+0000 7f9735ffb640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:20:55.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:55.891+0000 7f9735ffb640 1 -- 172.21.15.115:0/1391143716 >> v1:172.21.15.132:6791/0 conn(0x7f9724010840 legacy=0x7f9724005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:55.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:55.891+0000 7f9735ffb640 1 -- 172.21.15.115:0/1391143716 >> v1:172.21.15.132:6790/0 conn(0x7f97240113e0 legacy=0x7f9724011890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:55.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:55.891+0000 7f9735ffb640 1 -- 172.21.15.115:0/1391143716 >> v1:172.21.15.132:6789/0 conn(0x7f9724010430 legacy=0x7f9724009940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:55.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:55.891+0000 7f9735ffb640 10 monclient: _add_conns ranks=[4,6,3,5,1,0,2,8,7] 2024-01-24T23:20:55.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:55.891+0000 7f9735ffb640 10 monclient(hunting): picked mon.noname-c con 0x7f972400e0d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:20:55.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:55.891+0000 7f9735ffb640 10 monclient(hunting): picked mon.noname-d con 0x7f972400c680 addr v1:172.21.15.115:6792/0 2024-01-24T23:20:55.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:55.891+0000 7f9735ffb640 10 monclient(hunting): picked mon.noname-g con 0x7f97240113e0 addr v1:172.21.15.132:6790/0 2024-01-24T23:20:55.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:55.891+0000 7f9735ffb640 1 -- 172.21.15.115:0/1391143716 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9724012220 con 0x7f97240113e0 2024-01-24T23:20:55.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:55.891+0000 7f9735ffb640 1 -- 172.21.15.115:0/1391143716 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f972400d1a0 con 0x7f972400e0d0 2024-01-24T23:20:55.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:55.891+0000 7f9735ffb640 1 -- 172.21.15.115:0/1391143716 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f972400d330 con 0x7f972400c680 2024-01-24T23:20:55.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:55.891+0000 7f9735ffb640 10 monclient(hunting): _renew_subs 2024-01-24T23:20:56.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.799+0000 7f9737fff640 1 -- 172.21.15.115:0/1391143716 >> v1:172.21.15.115:6791/0 conn(0x7f972400e0d0 legacy=0x7f972400e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:20:56.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.799+0000 7f9737fff640 1 -- 172.21.15.115:0/1391143716 >> v1:172.21.15.115:6791/0 conn(0x7f972400e0d0 legacy=0x7f972400e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:20:56.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.799+0000 7f9737fff640 1 --1- 172.21.15.115:0/1391143716 >> v1:172.21.15.115:6791/0 conn(0x7f972400e0d0 0x7f972400e510 :-1 s=OPENED pgs=32 cs=1 l=1).handle_message read tag failed 2024-01-24T23:20:56.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.799+0000 7f9737fff640 1 --1- 172.21.15.115:0/1391143716 >> v1:172.21.15.115:6791/0 conn(0x7f972400e0d0 0x7f972400e510 :-1 s=OPENED pgs=32 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:20:56.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.799+0000 7f9736ffd640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:20:56.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 1 -- 172.21.15.115:0/1391143716 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 590347085 0 0) 0x7f972400d330 con 0x7f972400c680 2024-01-24T23:20:56.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:20:56.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:56.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:56.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 10 monclient(hunting): my global_id is 4110 2024-01-24T23:20:56.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 1 -- 172.21.15.115:0/1391143716 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9720003530 con 0x7f972400c680 2024-01-24T23:20:56.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 1 -- 172.21.15.115:0/1391143716 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 739720406 0 0) 0x7f9724012220 con 0x7f97240113e0 2024-01-24T23:20:56.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:20:56.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:56.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:56.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 10 monclient(hunting): my global_id is 4107 2024-01-24T23:20:56.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 1 -- 172.21.15.115:0/1391143716 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f972400d330 con 0x7f97240113e0 2024-01-24T23:20:56.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 1 -- 172.21.15.115:0/1391143716 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2916375407 0 0) 0x7f9720003530 con 0x7f972400c680 2024-01-24T23:20:56.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 1 -- 172.21.15.115:0/1391143716 >> v1:172.21.15.115:6791/0 conn(0x7f972400e0d0 legacy=0x7f972400e510 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:20:56.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 1 -- 172.21.15.115:0/1391143716 >> v1:172.21.15.132:6790/0 conn(0x7f97240113e0 legacy=0x7f9724011890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:56.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:20:56.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 1 monclient: found mon.noname-d 2024-01-24T23:20:56.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:20:56.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:20:56.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 1 -- 172.21.15.115:0/1391143716 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f972400d510 con 0x7f972400c680 2024-01-24T23:20:56.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 10 monclient: _finish_auth 0 2024-01-24T23:20:56.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:20:56.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:20:56.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 1 -- 172.21.15.115:0/1391143716 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f972c001630 con 0x7f972400c680 2024-01-24T23:20:56.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:56.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:20:56.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 10 monclient: dump: 2024-01-24T23:20:56.971 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:20:56.971 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:56.971 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:56.971 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:56.971 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:56.971 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:20:56.971 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:56.971 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:56.971 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:56.971 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:56.972 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:56.972 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:56.972 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:56.972 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:56.972 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:56.972 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:20:56.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f973e902640 5 monclient: authenticate success, global_id 4110 2024-01-24T23:20:56.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f973e902640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:20:56.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 1 -- 172.21.15.115:0/1391143716 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f972c001880 con 0x7f972400c680 2024-01-24T23:20:56.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:20:56.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 1 -- 172.21.15.115:0/1391143716 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f972c0047f0 con 0x7f972400c680 2024-01-24T23:20:56.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:56.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:20:56.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f9736ffd640 10 monclient: dump: 2024-01-24T23:20:56.973 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:20:56.973 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:56.973 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:56.973 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:56.973 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:56.973 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:20:56.973 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:56.973 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:56.973 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:56.973 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:56.973 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:56.973 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:56.973 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:56.973 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:56.973 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:56.974 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:20:56.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f973e902640 10 monclient: get_monmap_and_config success 2024-01-24T23:20:56.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.963+0000 7f973e902640 4 set_mon_vals no callback set 2024-01-24T23:20:56.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f973e902640 10 monclient: shutdown 2024-01-24T23:20:56.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f973e902640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:20:56.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f973e902640 1 -- 172.21.15.115:0/1391143716 >> v1:172.21.15.115:6792/0 conn(0x7f972400c680 legacy=0x7f972400ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:56.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f973ce9f640 1 -- 172.21.15.115:0/1391143716 reap_dead start 2024-01-24T23:20:56.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f973e902640 1 -- 172.21.15.115:0/1391143716 shutdown_connections 2024-01-24T23:20:56.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f973e902640 1 -- 172.21.15.115:0/1391143716 >> 172.21.15.115:0/1391143716 conn(0x7f97380662f0 msgr2=0x7f9738066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:20:56.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f973e902640 1 -- 172.21.15.115:0/1391143716 shutdown_connections 2024-01-24T23:20:56.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f973e902640 1 -- 172.21.15.115:0/1391143716 wait complete. 2024-01-24T23:20:56.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f973e902640 10 monclient: build_initial_monmap 2024-01-24T23:20:56.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f973e902640 10 monclient: monmap: 2024-01-24T23:20:56.977 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:20:56.977 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:20:56.978 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:20:56.978 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:20:56.978 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:20:56.978 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:20:56.978 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:20:56.978 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:20:56.978 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:20:56.978 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:20:56.978 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:20:56.978 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:20:56.978 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:20:56.978 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:20:56.978 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:20:56.979 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:20:56.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f973e902640 1 Processor -- start 2024-01-24T23:20:56.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f973e902640 1 -- start start 2024-01-24T23:20:56.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f973e902640 10 monclient: init 2024-01-24T23:20:56.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f973e902640 10 monclient: _reopen_session rank -1 2024-01-24T23:20:56.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f973e902640 10 monclient: _add_conns ranks=[1,8,0,7,5,4,6,3,2] 2024-01-24T23:20:56.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f973e902640 10 monclient(hunting): picked mon.noname-f con 0x7f972400c680 addr v1:172.21.15.132:6789/0 2024-01-24T23:20:56.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f973e902640 10 monclient(hunting): picked mon.noname-e con 0x7f97381cb570 addr v1:172.21.15.115:6793/0 2024-01-24T23:20:56.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f973e902640 10 monclient(hunting): picked mon.noname-a con 0x7f97381d1ff0 addr v1:172.21.15.115:6789/0 2024-01-24T23:20:56.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f973e902640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973809c810 con 0x7f97381d1ff0 2024-01-24T23:20:56.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f973e902640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97381cbfd0 con 0x7f972400c680 2024-01-24T23:20:56.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f973e902640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97380a5990 con 0x7f97381cb570 2024-01-24T23:20:56.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f973e902640 10 monclient(hunting): _renew_subs 2024-01-24T23:20:56.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f973e902640 10 monclient(hunting): authenticate will time out at 935.895796s 2024-01-24T23:20:56.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f97377fe640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f97381cb570 0x7f97381d08d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:34614/0 (socket says 172.21.15.115:34614) 2024-01-24T23:20:56.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.967+0000 7f97377fe640 1 -- 172.21.15.115:0/2338495759 learned_addr learned my addr 172.21.15.115:0/2338495759 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:20:56.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 1 -- 172.21.15.115:0/2338495759 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 346279686 0 0) 0x7f973809c810 con 0x7f97381d1ff0 2024-01-24T23:20:56.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:20:56.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:56.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:56.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 10 monclient(hunting): my global_id is 4104 2024-01-24T23:20:56.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 1 -- 172.21.15.115:0/2338495759 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9724010c70 con 0x7f97381d1ff0 2024-01-24T23:20:56.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 1 -- 172.21.15.115:0/2338495759 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3417080606 0 0) 0x7f9724010c70 con 0x7f97381d1ff0 2024-01-24T23:20:56.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 1 -- 172.21.15.115:0/2338495759 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f973809c810 con 0x7f97381d1ff0 2024-01-24T23:20:56.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 1 -- 172.21.15.115:0/2338495759 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9730001b50 con 0x7f97381d1ff0 2024-01-24T23:20:56.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:56.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:20:56.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 10 monclient(hunting): dump: 2024-01-24T23:20:56.981 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:20:56.981 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:56.981 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:56.981 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:56.981 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:56.981 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:20:56.981 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:56.981 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:56.981 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:56.982 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:56.982 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:56.982 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:56.982 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:56.982 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:56.982 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:56.982 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:20:56.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:20:56.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 1 -- 172.21.15.115:0/2338495759 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1767643435 0 0) 0x7f973809c810 con 0x7f97381d1ff0 2024-01-24T23:20:56.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 1 -- 172.21.15.115:0/2338495759 >> v1:172.21.15.115:6793/0 conn(0x7f97381cb570 legacy=0x7f97381d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:56.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 1 -- 172.21.15.115:0/2338495759 >> v1:172.21.15.132:6789/0 conn(0x7f972400c680 legacy=0x7f9738132ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:56.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:20:56.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 1 monclient: found mon.a 2024-01-24T23:20:56.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:20:56.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:20:56.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 1 -- 172.21.15.115:0/2338495759 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97380a2660 con 0x7f97381d1ff0 2024-01-24T23:20:56.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 10 monclient: _finish_auth 0 2024-01-24T23:20:56.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:20:56.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:20:56.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f973e902640 5 monclient: authenticate success, global_id 4104 2024-01-24T23:20:56.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f973e902640 10 monclient: _renew_subs 2024-01-24T23:20:56.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f973e902640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:20:56.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f973e902640 1 -- 172.21.15.115:0/2338495759 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9738120010 con 0x7f97381d1ff0 2024-01-24T23:20:56.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f973e902640 10 monclient: _renew_subs 2024-01-24T23:20:56.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f973e902640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:20:56.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f973e902640 1 -- 172.21.15.115:0/2338495759 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f97380a33e0 con 0x7f97381d1ff0 2024-01-24T23:20:56.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 1 -- 172.21.15.115:0/2338495759 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9730004b70 con 0x7f97381d1ff0 2024-01-24T23:20:56.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:20:56.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 1 -- 172.21.15.115:0/2338495759 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9730005020 con 0x7f97381d1ff0 2024-01-24T23:20:56.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:56.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9735ffb640 4 set_mon_vals no callback set 2024-01-24T23:20:56.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:20:56.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 10 monclient: dump: 2024-01-24T23:20:56.984 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:20:56.984 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:56.984 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:56.984 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:56.984 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:56.984 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:20:56.984 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:56.984 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:56.985 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:56.985 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:56.985 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:56.985 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:56.985 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:56.985 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:56.985 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:56.985 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:20:56.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 1 -- 172.21.15.115:0/2338495759 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 1) v1 ==== 799+0+0 (unknown 314974831 0 0) 0x7f9730005500 con 0x7f97381d1ff0 2024-01-24T23:20:56.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f9734ff9640 1 -- 172.21.15.115:0/2338495759 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 2878621973 0 0) 0x7f9730006b60 con 0x7f97381d1ff0 2024-01-24T23:20:56.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.971+0000 7f973e902640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:56.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.975+0000 7f973e902640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:56.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.975+0000 7f973e902640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:20:56.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.975+0000 7f973e902640 1 -- 172.21.15.115:0/2338495759 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9718005090 con 0x7f97381d1ff0 2024-01-24T23:20:56.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.979+0000 7f9734ff9640 1 -- 172.21.15.115:0/2338495759 <== mon.0 v1:172.21.15.115:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (unknown 1092875540 0 4287968521) 0x7f9730005aa0 con 0x7f97381d1ff0 2024-01-24T23:20:56.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.979+0000 7f9734ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:56.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:56.979+0000 7f9734ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:20:57.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:57.055+0000 7f973e902640 10 start_mon_command cmd=[{"prefix": "osd crush tunables", "profile": "default"}] 2024-01-24T23:20:57.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:57.055+0000 7f973e902640 10 monclient: _send_command 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2024-01-24T23:20:57.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:57.055+0000 7f973e902640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:20:57.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:57.055+0000 7f973e902640 1 -- 172.21.15.115:0/2338495759 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f9718005340 con 0x7f97381d1ff0 2024-01-24T23:20:57.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:57.059+0000 7f9734ff9640 1 -- 172.21.15.115:0/2338495759 <== mon.0 v1:172.21.15.115:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (unknown 2277712955 0 0) 0x7f9730005d20 con 0x7f97381d1ff0 2024-01-24T23:20:57.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:57.059+0000 7f9734ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2024-01-24T23:20:57.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:57.059+0000 7f9734ff9640 10 monclient: _finish_command 2 = system:0 adjusted tunables profile to default 2024-01-24T23:20:57.062 INFO:teuthology.orchestra.run.smithi115.stderr:adjusted tunables profile to default 2024-01-24T23:20:57.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:57.059+0000 7f973e902640 10 monclient: shutdown 2024-01-24T23:20:57.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:57.059+0000 7f973e902640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:20:57.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:57.059+0000 7f973e902640 1 -- 172.21.15.115:0/2338495759 >> v1:172.21.15.115:6789/0 conn(0x7f97381d1ff0 legacy=0x7f97381d43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:57.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:57.059+0000 7f973e902640 1 -- 172.21.15.115:0/2338495759 shutdown_connections 2024-01-24T23:20:57.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:57.059+0000 7f973e902640 1 -- 172.21.15.115:0/2338495759 >> 172.21.15.115:0/2338495759 conn(0x7f97380662f0 msgr2=0x7f973809d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:20:57.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:57.059+0000 7f973e902640 1 -- 172.21.15.115:0/2338495759 shutdown_connections 2024-01-24T23:20:57.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:57.059+0000 7f973e902640 1 -- 172.21.15.115:0/2338495759 wait complete. 2024-01-24T23:20:57.081 INFO:tasks.ceph:check_enable_crimson: False 2024-01-24T23:20:57.081 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-01-24T23:20:57.081 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-01-24T23:20:57.081 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-01-24T23:20:57.093 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-01-24T23:20:57.093 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-01-24T23:20:57.148 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-01-24T23:20:57.149 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-01-24T23:20:57.201 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-01-24T23:20:57.201 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-01-24T23:20:57.215 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-01-24T23:20:57.215 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-01-24T23:20:57.271 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-01-24T23:20:57.271 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-01-24T23:20:57.327 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 902f4a51-85e9-4af8-b11f-e42a51b31723 0 2024-01-24T23:20:57.453 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:20:57.449+0000 7f278372c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-24T23:20:57.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.489+0000 7f81e0d0f640 10 monclient: get_monmap_and_config 2024-01-24T23:20:57.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.489+0000 7f81e0d0f640 10 monclient: build_initial_monmap 2024-01-24T23:20:57.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.489+0000 7f81e0d0f640 10 monclient: monmap: 2024-01-24T23:20:57.492 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-01-24T23:20:57.492 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:57.492 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:57.494389+0000 2024-01-24T23:20:57.492 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:57.494389+0000 2024-01-24T23:20:57.492 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:20:57.493 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:57.493 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:20:57.493 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:20:57.493 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:20:57.493 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:20:57.493 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:20:57.493 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:20:57.493 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:20:57.493 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:20:57.493 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:20:57.493 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:57.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.489+0000 7f81e0d0f640 1 Processor -- start 2024-01-24T23:20:57.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.489+0000 7f81e0d0f640 1 -- start start 2024-01-24T23:20:57.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.489+0000 7f81e0d0f640 10 monclient: init 2024-01-24T23:20:57.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.489+0000 7f81e0d0f640 10 monclient: _reopen_session rank -1 2024-01-24T23:20:57.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.489+0000 7f81e0d0f640 10 monclient: _add_conns ranks=[5,4,0,8,7,6,3,2,1] 2024-01-24T23:20:57.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.489+0000 7f81e0d0f640 10 monclient(hunting): picked mon.noname-h con 0x7f81dc12fc60 addr v1:172.21.15.132:6791/0 2024-01-24T23:20:57.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.489+0000 7f81e0d0f640 10 monclient(hunting): picked mon.noname-c con 0x7f81dc09f450 addr v1:172.21.15.115:6791/0 2024-01-24T23:20:57.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.489+0000 7f81e0d0f640 10 monclient(hunting): picked mon.noname-a con 0x7f81dc1398a0 addr v1:172.21.15.115:6789/0 2024-01-24T23:20:57.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.489+0000 7f81e0d0f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81dc04ed70 con 0x7f81dc1398a0 2024-01-24T23:20:57.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.489+0000 7f81e0d0f640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81dc09e860 con 0x7f81dc09f450 2024-01-24T23:20:57.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.489+0000 7f81e0d0f640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81dc09eb40 con 0x7f81dc12fc60 2024-01-24T23:20:57.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.489+0000 7f81e0d0f640 10 monclient(hunting): _renew_subs 2024-01-24T23:20:57.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.489+0000 7f81e0d0f640 10 monclient(hunting): authenticate will time out at 944.819924s 2024-01-24T23:20:57.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.489+0000 7f81da59c640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f81dc12fc60 0x7f81dc130040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:60218/0 (socket says 172.21.15.132:60218) 2024-01-24T23:20:57.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.489+0000 7f81da59c640 1 -- 172.21.15.132:0/1071184392 learned_addr learned my addr 172.21.15.132:0/1071184392 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:20:57.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 1 -- 172.21.15.132:0/1071184392 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1002398220 0 0) 0x7f81dc09eb40 con 0x7f81dc12fc60 2024-01-24T23:20:57.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:20:57.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:57.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:57.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 10 monclient(hunting): my global_id is 4118 2024-01-24T23:20:57.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 1 -- 172.21.15.132:0/1071184392 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81c8003450 con 0x7f81dc12fc60 2024-01-24T23:20:57.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 1 -- 172.21.15.132:0/1071184392 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 429272676 0 0) 0x7f81c8003450 con 0x7f81dc12fc60 2024-01-24T23:20:57.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 1 -- 172.21.15.132:0/1071184392 >> v1:172.21.15.115:6791/0 conn(0x7f81dc09f450 legacy=0x7f81dc138180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:57.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 1 -- 172.21.15.132:0/1071184392 >> v1:172.21.15.115:6789/0 conn(0x7f81dc1398a0 legacy=0x7f81dc13bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:57.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 10 monclient: _finish_hunting 0 2024-01-24T23:20:57.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 1 monclient: found mon.noname-h 2024-01-24T23:20:57.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:20:57.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:20:57.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 1 -- 172.21.15.132:0/1071184392 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81dc09eff0 con 0x7f81dc12fc60 2024-01-24T23:20:57.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 10 monclient: _finish_auth 0 2024-01-24T23:20:57.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 10 monclient: _check_auth_tickets 2024-01-24T23:20:57.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:20:57.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 1 -- 172.21.15.132:0/1071184392 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f81cc0017c0 con 0x7f81dc12fc60 2024-01-24T23:20:57.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:57.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:20:57.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 10 monclient: dump: 2024-01-24T23:20:57.500 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:57.500 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:57.500 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:57.500 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:57.500 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:57.500 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:57.500 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:57.500 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:57.500 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:57.501 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:57.501 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:57.501 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:57.501 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:57.501 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:57.501 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:57.501 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:57.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 5 monclient: authenticate success, global_id 4118 2024-01-24T23:20:57.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:20:57.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 1 -- 172.21.15.132:0/1071184392 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f81cc001a70 con 0x7f81dc12fc60 2024-01-24T23:20:57.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:20:57.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 10 monclient: get_monmap_and_config success 2024-01-24T23:20:57.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 4 set_mon_vals no callback set 2024-01-24T23:20:57.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 1 -- 172.21.15.132:0/1071184392 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f81cc004bf0 con 0x7f81dc12fc60 2024-01-24T23:20:57.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:57.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:20:57.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d9d9b640 10 monclient: dump: 2024-01-24T23:20:57.502 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:57.502 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:57.502 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:57.502 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:57.502 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:57.502 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:57.502 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:57.502 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:57.503 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:57.503 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:57.503 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:57.503 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:57.503 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:57.503 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:57.503 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:57.503 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:57.503 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 10 monclient: shutdown 2024-01-24T23:20:57.503 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:20:57.503 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 1 -- 172.21.15.132:0/1071184392 >> v1:172.21.15.132:6791/0 conn(0x7f81dc12fc60 legacy=0x7f81dc130040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:57.503 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 1 -- 172.21.15.132:0/1071184392 shutdown_connections 2024-01-24T23:20:57.503 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 1 -- 172.21.15.132:0/1071184392 >> 172.21.15.132:0/1071184392 conn(0x7f81dc0a2170 msgr2=0x7f81dc0a2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:20:57.504 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 1 -- 172.21.15.132:0/1071184392 shutdown_connections 2024-01-24T23:20:57.504 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 1 -- 172.21.15.132:0/1071184392 wait complete. 2024-01-24T23:20:57.504 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 10 monclient: build_initial_monmap 2024-01-24T23:20:57.504 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 10 monclient: monmap: 2024-01-24T23:20:57.504 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-01-24T23:20:57.504 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:20:57.504 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-01-24T23:20:57.504 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-01-24T23:20:57.504 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:20:57.504 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-01-24T23:20:57.504 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:20:57.505 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:20:57.505 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:20:57.505 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:20:57.505 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:20:57.505 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:20:57.505 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:20:57.505 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:20:57.505 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:20:57.505 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:57.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 1 Processor -- start 2024-01-24T23:20:57.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 1 -- start start 2024-01-24T23:20:57.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 10 monclient: init 2024-01-24T23:20:57.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 10 monclient: _reopen_session rank -1 2024-01-24T23:20:57.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 10 monclient: _add_conns ranks=[4,6,1,5,8,2,7,0,3] 2024-01-24T23:20:57.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 10 monclient(hunting): picked mon.noname-c con 0x7f81dc09f450 addr v1:172.21.15.115:6791/0 2024-01-24T23:20:57.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 10 monclient(hunting): picked mon.noname-d con 0x7f81dc12fc60 addr v1:172.21.15.115:6792/0 2024-01-24T23:20:57.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 10 monclient(hunting): picked mon.noname-f con 0x7f81dc1398a0 addr v1:172.21.15.132:6789/0 2024-01-24T23:20:57.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81dc09b8f0 con 0x7f81dc1398a0 2024-01-24T23:20:57.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81dc1c7e40 con 0x7f81dc09f450 2024-01-24T23:20:57.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81dc09da20 con 0x7f81dc12fc60 2024-01-24T23:20:57.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 10 monclient(hunting): _renew_subs 2024-01-24T23:20:57.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81e0d0f640 10 monclient(hunting): authenticate will time out at 944.823472s 2024-01-24T23:20:57.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81dad9d640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f81dc1398a0 0x7f81dc1d3d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:55476/0 (socket says 172.21.15.132:55476) 2024-01-24T23:20:57.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81dad9d640 1 -- 172.21.15.132:0/244424099 learned_addr learned my addr 172.21.15.132:0/244424099 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:20:57.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d37fe640 1 -- 172.21.15.132:0/244424099 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3945639347 0 0) 0x7f81dc09b8f0 con 0x7f81dc1398a0 2024-01-24T23:20:57.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d37fe640 10 monclient(hunting): _negotiate 2024-01-24T23:20:57.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d37fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:57.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d37fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:57.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d37fe640 10 monclient(hunting): my global_id is 4114 2024-01-24T23:20:57.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.493+0000 7f81d37fe640 1 -- 172.21.15.132:0/244424099 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81bc003530 con 0x7f81dc1398a0 2024-01-24T23:20:57.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 1 -- 172.21.15.132:0/244424099 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2850562171 0 0) 0x7f81bc003530 con 0x7f81dc1398a0 2024-01-24T23:20:57.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 1 -- 172.21.15.132:0/244424099 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f81dc09b8f0 con 0x7f81dc1398a0 2024-01-24T23:20:57.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 1 -- 172.21.15.132:0/244424099 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f81d4001d20 con 0x7f81dc1398a0 2024-01-24T23:20:57.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:57.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:20:57.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 10 monclient(hunting): dump: 2024-01-24T23:20:57.510 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:57.510 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:57.510 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:57.510 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:57.510 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:57.511 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:57.511 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:57.511 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:57.511 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:57.511 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:57.511 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:57.511 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:57.511 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:57.511 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:57.511 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:57.511 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:57.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:20:57.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 1 -- 172.21.15.132:0/244424099 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2407189473 0 0) 0x7f81dc09da20 con 0x7f81dc12fc60 2024-01-24T23:20:57.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 10 monclient(hunting): _negotiate 2024-01-24T23:20:57.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:57.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:57.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 10 monclient(hunting): my global_id is 4119 2024-01-24T23:20:57.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 1 -- 172.21.15.132:0/244424099 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81bc003530 con 0x7f81dc12fc60 2024-01-24T23:20:57.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 1 -- 172.21.15.132:0/244424099 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 547110235 0 0) 0x7f81dc09b8f0 con 0x7f81dc1398a0 2024-01-24T23:20:57.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 1 -- 172.21.15.132:0/244424099 >> v1:172.21.15.115:6792/0 conn(0x7f81dc12fc60 legacy=0x7f81dc1d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:57.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 1 -- 172.21.15.132:0/244424099 >> v1:172.21.15.115:6791/0 conn(0x7f81dc09f450 legacy=0x7f81dc1c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:57.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 10 monclient: _finish_hunting 0 2024-01-24T23:20:57.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 1 monclient: found mon.f 2024-01-24T23:20:57.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:20:57.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:20:57.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 1 -- 172.21.15.132:0/244424099 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81dc072280 con 0x7f81dc1398a0 2024-01-24T23:20:57.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 10 monclient: _finish_auth 0 2024-01-24T23:20:57.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 10 monclient: _check_auth_tickets 2024-01-24T23:20:57.512 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:20:57.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81e0d0f640 5 monclient: authenticate success, global_id 4114 2024-01-24T23:20:57.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81e0d0f640 10 monclient: _renew_subs 2024-01-24T23:20:57.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81e0d0f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:20:57.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81e0d0f640 1 -- 172.21.15.132:0/244424099 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81dc09bff0 con 0x7f81dc1398a0 2024-01-24T23:20:57.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81e0d0f640 10 monclient: _renew_subs 2024-01-24T23:20:57.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81e0d0f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:20:57.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81e0d0f640 1 -- 172.21.15.132:0/244424099 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f81dc1c8960 con 0x7f81dc1398a0 2024-01-24T23:20:57.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 1 -- 172.21.15.132:0/244424099 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f81d40020f0 con 0x7f81dc1398a0 2024-01-24T23:20:57.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:20:57.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 1 -- 172.21.15.132:0/244424099 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f81d4004010 con 0x7f81dc1398a0 2024-01-24T23:20:57.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:57.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d959a640 4 set_mon_vals no callback set 2024-01-24T23:20:57.513 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:20:57.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 10 monclient: dump: 2024-01-24T23:20:57.514 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:57.514 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:57.514 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:57.514 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:57.514 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:57.514 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:57.514 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:57.514 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:57.514 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:57.514 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:57.514 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:57.514 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:57.514 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:57.514 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:57.514 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:57.515 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:57.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 1 -- 172.21.15.132:0/244424099 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 1) v1 ==== 799+0+0 (unknown 314974831 0 0) 0x7f81d40044f0 con 0x7f81dc1398a0 2024-01-24T23:20:57.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 1 -- 172.21.15.132:0/244424099 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (unknown 3392147075 0 0) 0x7f81d4006d30 con 0x7f81dc1398a0 2024-01-24T23:20:57.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81e0d0f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:57.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81e0d0f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:57.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81e0d0f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:20:57.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81e0d0f640 1 -- 172.21.15.132:0/244424099 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81c0005090 con 0x7f81dc1398a0 2024-01-24T23:20:57.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 1 -- 172.21.15.132:0/244424099 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (unknown 1092875540 0 4287968521) 0x7f81d4005c60 con 0x7f81dc1398a0 2024-01-24T23:20:57.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:57.515 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.497+0000 7f81d37fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:20:57.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.569+0000 7f81e0d0f640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "902f4a51-85e9-4af8-b11f-e42a51b31723", "id": 0}] 2024-01-24T23:20:57.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.569+0000 7f81e0d0f640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "902f4a51-85e9-4af8-b11f-e42a51b31723", "id": 0}] 2024-01-24T23:20:57.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.569+0000 7f81e0d0f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:20:57.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.569+0000 7f81e0d0f640 1 -- 172.21.15.132:0/244424099 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd new", "uuid": "902f4a51-85e9-4af8-b11f-e42a51b31723", "id": 0} v 0) v1 -- 0x7f81c00058c0 con 0x7f81dc1398a0 2024-01-24T23:20:57.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.577+0000 7f81d37fe640 1 -- 172.21.15.132:0/244424099 <== mon.1 v1:172.21.15.132:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "902f4a51-85e9-4af8-b11f-e42a51b31723", "id": 0}]=0 v3) v1 ==== 112+0+2 (unknown 3371156644 0 732707515) 0x7f81d4005f50 con 0x7f81dc1398a0 2024-01-24T23:20:57.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.577+0000 7f81d37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "902f4a51-85e9-4af8-b11f-e42a51b31723", "id": 0}] 2024-01-24T23:20:57.584 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.577+0000 7f81d37fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:20:57.584 INFO:teuthology.orchestra.run.smithi132.stdout:0 2024-01-24T23:20:57.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.581+0000 7f81e0d0f640 10 monclient: shutdown 2024-01-24T23:20:57.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.581+0000 7f81e0d0f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:20:57.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.581+0000 7f81e0d0f640 1 -- 172.21.15.132:0/244424099 >> v1:172.21.15.132:6789/0 conn(0x7f81dc1398a0 legacy=0x7f81dc1d3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:57.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.581+0000 7f81e0d0f640 1 -- 172.21.15.132:0/244424099 shutdown_connections 2024-01-24T23:20:57.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.581+0000 7f81e0d0f640 1 -- 172.21.15.132:0/244424099 >> 172.21.15.132:0/244424099 conn(0x7f81dc0a2170 msgr2=0x7f81dc09fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:20:57.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.581+0000 7f81e0d0f640 1 -- 172.21.15.132:0/244424099 shutdown_connections 2024-01-24T23:20:57.585 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.581+0000 7f81e0d0f640 1 -- 172.21.15.132:0/244424099 wait complete. 2024-01-24T23:20:57.599 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new e06b92ed-3ab9-4103-8e41-4fa60aea2829 1 2024-01-24T23:20:57.712 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.709+0000 7f5a7d240640 10 monclient: get_monmap_and_config 2024-01-24T23:20:57.713 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.709+0000 7f5a7d240640 10 monclient: build_initial_monmap 2024-01-24T23:20:57.713 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.709+0000 7f5a7d240640 10 monclient: monmap: 2024-01-24T23:20:57.713 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-01-24T23:20:57.713 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:57.713 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:57.715272+0000 2024-01-24T23:20:57.713 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:57.715272+0000 2024-01-24T23:20:57.713 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:20:57.713 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:57.713 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:20:57.713 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:20:57.713 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:20:57.713 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:20:57.713 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:20:57.714 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:20:57.714 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:20:57.714 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:20:57.714 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:20:57.714 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:57.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.709+0000 7f5a7d240640 1 Processor -- start 2024-01-24T23:20:57.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.709+0000 7f5a7d240640 1 -- start start 2024-01-24T23:20:57.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.709+0000 7f5a7d240640 10 monclient: init 2024-01-24T23:20:57.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.709+0000 7f5a7d240640 10 monclient: _reopen_session rank -1 2024-01-24T23:20:57.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.709+0000 7f5a7d240640 10 monclient: _add_conns ranks=[4,2,7,3,5,8,6,1,0] 2024-01-24T23:20:57.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.709+0000 7f5a7d240640 10 monclient(hunting): picked mon.noname-c con 0x7f5a780662f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:20:57.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.709+0000 7f5a7d240640 10 monclient(hunting): picked mon.noname-b con 0x7f5a78134490 addr v1:172.21.15.115:6790/0 2024-01-24T23:20:57.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.709+0000 7f5a7d240640 10 monclient(hunting): picked mon.noname-i con 0x7f5a7809df70 addr v1:172.21.15.132:6792/0 2024-01-24T23:20:57.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.709+0000 7f5a7d240640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a7804ed70 con 0x7f5a78134490 2024-01-24T23:20:57.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a780666d0 con 0x7f5a780662f0 2024-01-24T23:20:57.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a7809f3b0 con 0x7f5a7809df70 2024-01-24T23:20:57.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 10 monclient(hunting): _renew_subs 2024-01-24T23:20:57.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 10 monclient(hunting): authenticate will time out at 945.040065s 2024-01-24T23:20:57.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7759e640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f5a7809df70 0x7f5a7813c220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:37772/0 (socket says 172.21.15.132:37772) 2024-01-24T23:20:57.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7759e640 1 -- 172.21.15.132:0/439856356 learned_addr learned my addr 172.21.15.132:0/439856356 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:20:57.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 1 -- 172.21.15.132:0/439856356 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1674443711 0 0) 0x7f5a7809f3b0 con 0x7f5a7809df70 2024-01-24T23:20:57.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:20:57.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:57.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:57.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 10 monclient(hunting): my global_id is 4120 2024-01-24T23:20:57.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 1 -- 172.21.15.132:0/439856356 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a64003530 con 0x7f5a7809df70 2024-01-24T23:20:57.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 1 -- 172.21.15.132:0/439856356 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3828825975 0 0) 0x7f5a64003530 con 0x7f5a7809df70 2024-01-24T23:20:57.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 1 -- 172.21.15.132:0/439856356 >> v1:172.21.15.115:6791/0 conn(0x7f5a780662f0 legacy=0x7f5a78133d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:57.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 1 -- 172.21.15.132:0/439856356 >> v1:172.21.15.115:6790/0 conn(0x7f5a78134490 legacy=0x7f5a78138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:57.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 10 monclient: _finish_hunting 0 2024-01-24T23:20:57.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 1 monclient: found mon.noname-i 2024-01-24T23:20:57.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:20:57.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:20:57.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 1 -- 172.21.15.132:0/439856356 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a78133950 con 0x7f5a7809df70 2024-01-24T23:20:57.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 10 monclient: _finish_auth 0 2024-01-24T23:20:57.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 10 monclient: _check_auth_tickets 2024-01-24T23:20:57.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:20:57.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 1 -- 172.21.15.132:0/439856356 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5a70001b40 con 0x7f5a7809df70 2024-01-24T23:20:57.722 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:57.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:20:57.723 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 10 monclient: dump: 2024-01-24T23:20:57.723 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:57.723 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:57.723 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:57.723 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:57.723 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:57.723 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:57.723 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:57.723 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:57.723 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:57.723 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:57.723 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:57.723 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:57.723 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:57.724 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:57.724 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:57.724 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:57.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 5 monclient: authenticate success, global_id 4120 2024-01-24T23:20:57.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:20:57.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 1 -- 172.21.15.132:0/439856356 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5a70001d60 con 0x7f5a7809df70 2024-01-24T23:20:57.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:20:57.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 1 -- 172.21.15.132:0/439856356 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5a70004f70 con 0x7f5a7809df70 2024-01-24T23:20:57.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:57.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:20:57.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a75d9b640 10 monclient: dump: 2024-01-24T23:20:57.724 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:57.724 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:57.725 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:57.725 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:57.725 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:57.725 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:57.725 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:57.725 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:57.725 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:57.725 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:57.725 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:57.725 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:57.725 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:57.725 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:57.725 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:57.725 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:57.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 10 monclient: get_monmap_and_config success 2024-01-24T23:20:57.726 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 4 set_mon_vals no callback set 2024-01-24T23:20:57.726 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 10 monclient: shutdown 2024-01-24T23:20:57.726 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:20:57.726 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 1 -- 172.21.15.132:0/439856356 >> v1:172.21.15.132:6792/0 conn(0x7f5a7809df70 legacy=0x7f5a7813c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:57.726 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 1 -- 172.21.15.132:0/439856356 shutdown_connections 2024-01-24T23:20:57.726 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 1 -- 172.21.15.132:0/439856356 >> 172.21.15.132:0/439856356 conn(0x7f5a780a2170 msgr2=0x7f5a780a2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:20:57.726 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 1 -- 172.21.15.132:0/439856356 shutdown_connections 2024-01-24T23:20:57.726 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 1 -- 172.21.15.132:0/439856356 wait complete. 2024-01-24T23:20:57.726 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 10 monclient: build_initial_monmap 2024-01-24T23:20:57.726 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 10 monclient: monmap: 2024-01-24T23:20:57.726 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-01-24T23:20:57.726 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:20:57.726 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-01-24T23:20:57.726 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-01-24T23:20:57.726 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:20:57.727 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-01-24T23:20:57.727 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:20:57.727 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:20:57.727 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:20:57.727 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:20:57.727 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:20:57.727 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:20:57.727 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:20:57.727 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:20:57.727 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:20:57.727 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:57.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 1 Processor -- start 2024-01-24T23:20:57.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 1 -- start start 2024-01-24T23:20:57.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 10 monclient: init 2024-01-24T23:20:57.727 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 10 monclient: _reopen_session rank -1 2024-01-24T23:20:57.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 10 monclient: _add_conns ranks=[0,5,6,7,2,4,3,1,8] 2024-01-24T23:20:57.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 10 monclient(hunting): picked mon.noname-a con 0x7f5a780662f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:20:57.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 10 monclient(hunting): picked mon.noname-h con 0x7f5a7809df70 addr v1:172.21.15.132:6791/0 2024-01-24T23:20:57.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 10 monclient(hunting): picked mon.noname-d con 0x7f5a78134490 addr v1:172.21.15.115:6792/0 2024-01-24T23:20:57.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a7812ea90 con 0x7f5a780662f0 2024-01-24T23:20:57.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a7812e1f0 con 0x7f5a7809df70 2024-01-24T23:20:57.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a7812c3d0 con 0x7f5a78134490 2024-01-24T23:20:57.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 10 monclient(hunting): _renew_subs 2024-01-24T23:20:57.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7d240640 10 monclient(hunting): authenticate will time out at 945.043268s 2024-01-24T23:20:57.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7659c640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f5a7809df70 0x7f5a7812bcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:60220/0 (socket says 172.21.15.132:60220) 2024-01-24T23:20:57.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a7659c640 1 -- 172.21.15.132:0/560135483 learned_addr learned my addr 172.21.15.132:0/560135483 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:20:57.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a6f7fe640 1 -- 172.21.15.132:0/560135483 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1828089391 0 0) 0x7f5a7812e1f0 con 0x7f5a7809df70 2024-01-24T23:20:57.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a6f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:20:57.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a6f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:57.728 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a6f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:57.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a6f7fe640 10 monclient(hunting): my global_id is 4127 2024-01-24T23:20:57.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a6f7fe640 1 -- 172.21.15.132:0/560135483 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a58003530 con 0x7f5a7809df70 2024-01-24T23:20:57.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a6f7fe640 1 -- 172.21.15.132:0/560135483 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2358838208 0 0) 0x7f5a58003530 con 0x7f5a7809df70 2024-01-24T23:20:57.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a6f7fe640 1 -- 172.21.15.132:0/560135483 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5a7812e1f0 con 0x7f5a7809df70 2024-01-24T23:20:57.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a6f7fe640 1 -- 172.21.15.132:0/560135483 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5a68001b20 con 0x7f5a7809df70 2024-01-24T23:20:57.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a6f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:57.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a6f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:20:57.729 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a6f7fe640 10 monclient(hunting): dump: 2024-01-24T23:20:57.729 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:57.729 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:57.729 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:57.729 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:57.729 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:57.729 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:57.729 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:57.730 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:57.730 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:57.730 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:57.731 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:57.731 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:57.731 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:57.731 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:57.731 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:57.731 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:57.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a6f7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:20:57.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a6f7fe640 1 -- 172.21.15.132:0/560135483 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4027171332 0 0) 0x7f5a7812ea90 con 0x7f5a780662f0 2024-01-24T23:20:57.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a6f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:20:57.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a6f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:57.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a6f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:57.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.713+0000 7f5a6f7fe640 10 monclient(hunting): my global_id is 4131 2024-01-24T23:20:57.731 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 1 -- 172.21.15.132:0/560135483 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a58003530 con 0x7f5a780662f0 2024-01-24T23:20:57.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 1 -- 172.21.15.132:0/560135483 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3774865425 0 0) 0x7f5a7812c3d0 con 0x7f5a78134490 2024-01-24T23:20:57.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:20:57.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:57.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:57.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 10 monclient(hunting): my global_id is 4128 2024-01-24T23:20:57.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 1 -- 172.21.15.132:0/560135483 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a7812ea90 con 0x7f5a78134490 2024-01-24T23:20:57.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 1 -- 172.21.15.132:0/560135483 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3137359194 0 0) 0x7f5a7812e1f0 con 0x7f5a7809df70 2024-01-24T23:20:57.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 1 -- 172.21.15.132:0/560135483 >> v1:172.21.15.115:6792/0 conn(0x7f5a78134490 legacy=0x7f5a781cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:57.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 1 -- 172.21.15.132:0/560135483 >> v1:172.21.15.115:6789/0 conn(0x7f5a780662f0 legacy=0x7f5a7812b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:57.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:20:57.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 1 monclient: found mon.h 2024-01-24T23:20:57.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:20:57.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:20:57.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 1 -- 172.21.15.132:0/560135483 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7812e4d0 con 0x7f5a7809df70 2024-01-24T23:20:57.732 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 10 monclient: _finish_auth 0 2024-01-24T23:20:57.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:20:57.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:20:57.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a7d240640 5 monclient: authenticate success, global_id 4127 2024-01-24T23:20:57.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a7d240640 10 monclient: _renew_subs 2024-01-24T23:20:57.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a7d240640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:20:57.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a7d240640 1 -- 172.21.15.132:0/560135483 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a7812e7b0 con 0x7f5a7809df70 2024-01-24T23:20:57.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a7d240640 10 monclient: _renew_subs 2024-01-24T23:20:57.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a7d240640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:20:57.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a7d240640 1 -- 172.21.15.132:0/560135483 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a7812c770 con 0x7f5a7809df70 2024-01-24T23:20:57.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 1 -- 172.21.15.132:0/560135483 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5a68003950 con 0x7f5a7809df70 2024-01-24T23:20:57.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:20:57.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a7559a640 4 set_mon_vals no callback set 2024-01-24T23:20:57.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 1 -- 172.21.15.132:0/560135483 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5a68003e80 con 0x7f5a7809df70 2024-01-24T23:20:57.733 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:57.734 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:20:57.734 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 10 monclient: dump: 2024-01-24T23:20:57.734 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:57.734 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:57.734 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:57.734 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:57.734 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:57.734 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:57.734 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:57.734 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:57.734 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:57.734 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:57.734 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:57.734 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:57.735 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:57.735 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:57.735 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:57.735 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:57.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 1 -- 172.21.15.132:0/560135483 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 1) v1 ==== 799+0+0 (unknown 314974831 0 0) 0x7f5a680057d0 con 0x7f5a7809df70 2024-01-24T23:20:57.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 1 -- 172.21.15.132:0/560135483 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (unknown 1774397980 0 0) 0x7f5a68006be0 con 0x7f5a7809df70 2024-01-24T23:20:57.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a7d240640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:57.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a7d240640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:57.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a7d240640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:20:57.735 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a7d240640 1 -- 172.21.15.132:0/560135483 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a5c005090 con 0x7f5a7809df70 2024-01-24T23:20:57.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 1 -- 172.21.15.132:0/560135483 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (unknown 1092875540 0 4287968521) 0x7f5a68005a80 con 0x7f5a7809df70 2024-01-24T23:20:57.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:57.736 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.717+0000 7f5a6f7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:20:57.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.785+0000 7f5a7d240640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "e06b92ed-3ab9-4103-8e41-4fa60aea2829", "id": 1}] 2024-01-24T23:20:57.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.785+0000 7f5a7d240640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "e06b92ed-3ab9-4103-8e41-4fa60aea2829", "id": 1}] 2024-01-24T23:20:57.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.785+0000 7f5a7d240640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:20:57.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.785+0000 7f5a7d240640 1 -- 172.21.15.132:0/560135483 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd new", "uuid": "e06b92ed-3ab9-4103-8e41-4fa60aea2829", "id": 1} v 0) v1 -- 0x7f5a5c0058c0 con 0x7f5a7809df70 2024-01-24T23:20:57.800 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.797+0000 7f5a6f7fe640 1 -- 172.21.15.132:0/560135483 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e06b92ed-3ab9-4103-8e41-4fa60aea2829", "id": 1}]=0 v4) v1 ==== 112+0+2 (unknown 3498561063 0 940483276) 0x7f5a68005c20 con 0x7f5a7809df70 2024-01-24T23:20:57.800 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.797+0000 7f5a6f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "e06b92ed-3ab9-4103-8e41-4fa60aea2829", "id": 1}] 2024-01-24T23:20:57.800 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.797+0000 7f5a6f7fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:20:57.800 INFO:teuthology.orchestra.run.smithi132.stdout:1 2024-01-24T23:20:57.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.797+0000 7f5a7d240640 10 monclient: shutdown 2024-01-24T23:20:57.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.797+0000 7f5a7d240640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:20:57.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.797+0000 7f5a7d240640 1 -- 172.21.15.132:0/560135483 >> v1:172.21.15.132:6791/0 conn(0x7f5a7809df70 legacy=0x7f5a7812bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:57.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.797+0000 7f5a7d240640 1 -- 172.21.15.132:0/560135483 shutdown_connections 2024-01-24T23:20:57.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.797+0000 7f5a7d240640 1 -- 172.21.15.132:0/560135483 >> 172.21.15.132:0/560135483 conn(0x7f5a780a2170 msgr2=0x7f5a780a17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:20:57.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.797+0000 7f5a7d240640 1 -- 172.21.15.132:0/560135483 shutdown_connections 2024-01-24T23:20:57.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.801+0000 7f5a7d240640 1 -- 172.21.15.132:0/560135483 wait complete. 2024-01-24T23:20:57.824 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 013044c9-553c-4dd9-8438-c9873813e5af 2 2024-01-24T23:20:57.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.945+0000 7fd0471c7640 10 monclient: get_monmap_and_config 2024-01-24T23:20:57.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.945+0000 7fd0471c7640 10 monclient: build_initial_monmap 2024-01-24T23:20:57.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.945+0000 7fd0471c7640 10 monclient: monmap: 2024-01-24T23:20:57.949 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-01-24T23:20:57.949 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:57.949 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:57.951130+0000 2024-01-24T23:20:57.949 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:57.951130+0000 2024-01-24T23:20:57.949 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:20:57.949 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:57.949 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:20:57.949 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:20:57.949 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:20:57.949 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:20:57.949 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:20:57.949 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:20:57.950 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:20:57.950 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:20:57.950 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:20:57.950 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:57.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.945+0000 7fd0471c7640 1 Processor -- start 2024-01-24T23:20:57.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.945+0000 7fd0471c7640 1 -- start start 2024-01-24T23:20:57.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.945+0000 7fd0471c7640 10 monclient: init 2024-01-24T23:20:57.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.945+0000 7fd0471c7640 10 monclient: _reopen_session rank -1 2024-01-24T23:20:57.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.945+0000 7fd0471c7640 10 monclient: _add_conns ranks=[6,8,3,0,7,2,1,4,5] 2024-01-24T23:20:57.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.945+0000 7fd0471c7640 10 monclient(hunting): picked mon.noname-d con 0x7fd040067300 addr v1:172.21.15.115:6792/0 2024-01-24T23:20:57.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.945+0000 7fd0471c7640 10 monclient(hunting): picked mon.noname-e con 0x7fd040133900 addr v1:172.21.15.115:6793/0 2024-01-24T23:20:57.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.945+0000 7fd0471c7640 10 monclient(hunting): picked mon.noname-g con 0x7fd04013a340 addr v1:172.21.15.132:6790/0 2024-01-24T23:20:57.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.945+0000 7fd0471c7640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04004ed70 con 0x7fd04013a340 2024-01-24T23:20:57.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.945+0000 7fd0471c7640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0400676e0 con 0x7fd040067300 2024-01-24T23:20:57.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.945+0000 7fd0471c7640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04009ef10 con 0x7fd040133900 2024-01-24T23:20:57.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.945+0000 7fd0471c7640 10 monclient(hunting): _renew_subs 2024-01-24T23:20:57.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.945+0000 7fd0471c7640 10 monclient(hunting): authenticate will time out at 945.275851s 2024-01-24T23:20:57.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.945+0000 7fd045764640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fd04013a340 0x7fd04013c730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:36178/0 (socket says 172.21.15.132:36178) 2024-01-24T23:20:57.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.945+0000 7fd045764640 1 -- 172.21.15.132:0/2944759117 learned_addr learned my addr 172.21.15.132:0/2944759117 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:20:57.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 1 -- 172.21.15.132:0/2944759117 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487945382 0 0) 0x7fd04004ed70 con 0x7fd04013a340 2024-01-24T23:20:57.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:20:57.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:57.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:57.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 10 monclient(hunting): my global_id is 4116 2024-01-24T23:20:57.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 1 -- 172.21.15.132:0/2944759117 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd030003530 con 0x7fd04013a340 2024-01-24T23:20:57.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 1 -- 172.21.15.132:0/2944759117 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3501019551 0 0) 0x7fd030003530 con 0x7fd04013a340 2024-01-24T23:20:57.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 1 -- 172.21.15.132:0/2944759117 >> v1:172.21.15.115:6793/0 conn(0x7fd040133900 legacy=0x7fd040138c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:57.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 1 -- 172.21.15.132:0/2944759117 >> v1:172.21.15.115:6792/0 conn(0x7fd040067300 legacy=0x7fd04009e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:57.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:20:57.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 1 monclient: found mon.noname-g 2024-01-24T23:20:57.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:20:57.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:20:57.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 1 -- 172.21.15.132:0/2944759117 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd040133f20 con 0x7fd04013a340 2024-01-24T23:20:57.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 10 monclient: _finish_auth 0 2024-01-24T23:20:57.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:20:57.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:20:57.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd0471c7640 5 monclient: authenticate success, global_id 4116 2024-01-24T23:20:57.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd0471c7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:20:57.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 1 -- 172.21.15.132:0/2944759117 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd038001b40 con 0x7fd04013a340 2024-01-24T23:20:57.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:57.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:20:57.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 10 monclient: dump: 2024-01-24T23:20:57.954 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:57.955 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:57.955 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:57.955 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:57.955 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:57.955 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:57.955 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:57.955 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:57.955 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:57.955 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:57.955 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:57.955 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:57.955 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:57.955 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:57.955 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:57.955 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:57.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd0471c7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:20:57.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 1 -- 172.21.15.132:0/2944759117 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd038001df0 con 0x7fd04013a340 2024-01-24T23:20:57.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:20:57.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd0471c7640 10 monclient: get_monmap_and_config success 2024-01-24T23:20:57.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 1 -- 172.21.15.132:0/2944759117 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd038004f70 con 0x7fd04013a340 2024-01-24T23:20:57.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd0471c7640 4 set_mon_vals no callback set 2024-01-24T23:20:57.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:57.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:20:57.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd03f7fe640 10 monclient: dump: 2024-01-24T23:20:57.960 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:57.960 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:57.961 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:57.961 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:57.961 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:57.961 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:57.961 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:57.961 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:57.961 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:57.961 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:57.961 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:57.961 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:57.961 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:57.961 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:57.961 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:57.961 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:57.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd0471c7640 10 monclient: shutdown 2024-01-24T23:20:57.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd0471c7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:20:57.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.949+0000 7fd0471c7640 1 -- 172.21.15.132:0/2944759117 >> v1:172.21.15.132:6790/0 conn(0x7fd04013a340 legacy=0x7fd04013c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:57.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 1 -- 172.21.15.132:0/2944759117 shutdown_connections 2024-01-24T23:20:57.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 1 -- 172.21.15.132:0/2944759117 >> 172.21.15.132:0/2944759117 conn(0x7fd0400a1de0 msgr2=0x7fd0400a21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:20:57.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 1 -- 172.21.15.132:0/2944759117 shutdown_connections 2024-01-24T23:20:57.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 1 -- 172.21.15.132:0/2944759117 wait complete. 2024-01-24T23:20:57.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 10 monclient: build_initial_monmap 2024-01-24T23:20:57.962 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 10 monclient: monmap: 2024-01-24T23:20:57.962 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-01-24T23:20:57.962 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:20:57.962 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-01-24T23:20:57.962 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-01-24T23:20:57.962 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:20:57.962 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-01-24T23:20:57.963 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:20:57.963 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:20:57.963 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:20:57.963 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:20:57.963 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:20:57.963 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:20:57.963 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:20:57.963 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:20:57.963 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:20:57.963 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:57.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 1 Processor -- start 2024-01-24T23:20:57.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 1 -- start start 2024-01-24T23:20:57.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 10 monclient: init 2024-01-24T23:20:57.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 10 monclient: _reopen_session rank -1 2024-01-24T23:20:57.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 10 monclient: _add_conns ranks=[8,3,4,6,2,0,1,7,5] 2024-01-24T23:20:57.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 10 monclient(hunting): picked mon.noname-e con 0x7fd040067300 addr v1:172.21.15.115:6793/0 2024-01-24T23:20:57.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 10 monclient(hunting): picked mon.noname-g con 0x7fd040133900 addr v1:172.21.15.132:6790/0 2024-01-24T23:20:57.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 10 monclient(hunting): picked mon.noname-c con 0x7fd04013a340 addr v1:172.21.15.115:6791/0 2024-01-24T23:20:57.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04006e930 con 0x7fd040133900 2024-01-24T23:20:57.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04009f0e0 con 0x7fd04013a340 2024-01-24T23:20:57.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd040138990 con 0x7fd040067300 2024-01-24T23:20:57.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 10 monclient(hunting): _renew_subs 2024-01-24T23:20:57.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 10 monclient(hunting): authenticate will time out at 945.282012s 2024-01-24T23:20:57.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03ffff640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fd040133900 0x7fd040138280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:36188/0 (socket says 172.21.15.132:36188) 2024-01-24T23:20:57.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03ffff640 1 -- 172.21.15.132:0/2328114942 learned_addr learned my addr 172.21.15.132:0/2328114942 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:20:57.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 1 -- 172.21.15.132:0/2328114942 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1808454274 0 0) 0x7fd04006e930 con 0x7fd040133900 2024-01-24T23:20:57.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:20:57.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:57.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:57.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 10 monclient(hunting): my global_id is 4125 2024-01-24T23:20:57.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 1 -- 172.21.15.132:0/2328114942 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd024003530 con 0x7fd040133900 2024-01-24T23:20:57.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 1 -- 172.21.15.132:0/2328114942 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2686262416 0 0) 0x7fd024003530 con 0x7fd040133900 2024-01-24T23:20:57.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 1 -- 172.21.15.132:0/2328114942 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd04006e930 con 0x7fd040133900 2024-01-24T23:20:57.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 1 -- 172.21.15.132:0/2328114942 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd02c001ae0 con 0x7fd040133900 2024-01-24T23:20:57.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:57.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:20:57.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 10 monclient(hunting): dump: 2024-01-24T23:20:57.965 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:57.965 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:57.965 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:57.965 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:57.966 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:57.966 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:57.966 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:57.966 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:57.966 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:57.966 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:57.966 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:57.966 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:57.966 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:57.966 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:57.966 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:57.966 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:57.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:20:57.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 1 -- 172.21.15.132:0/2328114942 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2374784312 0 0) 0x7fd04006e930 con 0x7fd040133900 2024-01-24T23:20:57.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 1 -- 172.21.15.132:0/2328114942 >> v1:172.21.15.115:6793/0 conn(0x7fd040067300 legacy=0x7fd040137b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:57.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 1 -- 172.21.15.132:0/2328114942 >> v1:172.21.15.115:6791/0 conn(0x7fd04013a340 legacy=0x7fd0401d1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:57.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 10 monclient: _finish_hunting 0 2024-01-24T23:20:57.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 1 monclient: found mon.g 2024-01-24T23:20:57.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:20:57.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:20:57.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 1 -- 172.21.15.132:0/2328114942 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd040139260 con 0x7fd040133900 2024-01-24T23:20:57.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 10 monclient: _finish_auth 0 2024-01-24T23:20:57.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:20:57.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:20:57.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 5 monclient: authenticate success, global_id 4125 2024-01-24T23:20:57.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 10 monclient: _renew_subs 2024-01-24T23:20:57.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:20:57.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 1 -- 172.21.15.132:0/2328114942 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04009f5a0 con 0x7fd040133900 2024-01-24T23:20:57.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 10 monclient: _renew_subs 2024-01-24T23:20:57.967 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:20:57.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd0471c7640 1 -- 172.21.15.132:0/2328114942 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04006e650 con 0x7fd040133900 2024-01-24T23:20:57.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 1 -- 172.21.15.132:0/2328114942 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd02c0038d0 con 0x7fd040133900 2024-01-24T23:20:57.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:20:57.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 1 -- 172.21.15.132:0/2328114942 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd02c003e00 con 0x7fd040133900 2024-01-24T23:20:57.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:57.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:20:57.968 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 10 monclient: dump: 2024-01-24T23:20:57.968 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:57.968 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:57.968 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:57.968 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:57.968 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:57.968 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:57.969 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:57.969 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:57.969 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:57.969 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:57.969 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:57.969 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:57.969 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:57.969 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:57.969 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:57.969 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:57.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03effd640 4 set_mon_vals no callback set 2024-01-24T23:20:57.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 1 -- 172.21.15.132:0/2328114942 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 1) v1 ==== 799+0+0 (unknown 314974831 0 0) 0x7fd02c0042e0 con 0x7fd040133900 2024-01-24T23:20:57.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.953+0000 7fd03d7fa640 1 -- 172.21.15.132:0/2328114942 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (unknown 2139461973 0 0) 0x7fd02c005bf0 con 0x7fd040133900 2024-01-24T23:20:57.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.957+0000 7fd0471c7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:57.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.957+0000 7fd0471c7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:57.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.957+0000 7fd0471c7640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:20:57.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.957+0000 7fd0471c7640 1 -- 172.21.15.132:0/2328114942 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0400008e0 con 0x7fd040133900 2024-01-24T23:20:57.972 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:20:57.969+0000 7f278372c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-24T23:20:57.982 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.981+0000 7fd03d7fa640 1 -- 172.21.15.132:0/2328114942 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (unknown 1092875540 0 4287968521) 0x7fd02c0057d0 con 0x7fd040133900 2024-01-24T23:20:57.982 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.981+0000 7fd03d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:57.982 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:57.981+0000 7fd03d7fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:20:58.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.049+0000 7fd0471c7640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "013044c9-553c-4dd9-8438-c9873813e5af", "id": 2}] 2024-01-24T23:20:58.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.049+0000 7fd0471c7640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "013044c9-553c-4dd9-8438-c9873813e5af", "id": 2}] 2024-01-24T23:20:58.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.049+0000 7fd0471c7640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:20:58.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.049+0000 7fd0471c7640 1 -- 172.21.15.132:0/2328114942 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd new", "uuid": "013044c9-553c-4dd9-8438-c9873813e5af", "id": 2} v 0) v1 -- 0x7fd04013de50 con 0x7fd040133900 2024-01-24T23:20:58.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.057+0000 7fd03d7fa640 1 -- 172.21.15.132:0/2328114942 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "013044c9-553c-4dd9-8438-c9873813e5af", "id": 2}]=0 v5) v1 ==== 112+0+2 (unknown 915505336 0 216599125) 0x7fd02c005970 con 0x7fd040133900 2024-01-24T23:20:58.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.057+0000 7fd03d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "013044c9-553c-4dd9-8438-c9873813e5af", "id": 2}] 2024-01-24T23:20:58.063 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.057+0000 7fd03d7fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:20:58.063 INFO:teuthology.orchestra.run.smithi132.stdout:2 2024-01-24T23:20:58.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.061+0000 7fd0471c7640 10 monclient: shutdown 2024-01-24T23:20:58.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.061+0000 7fd0471c7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:20:58.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.061+0000 7fd0471c7640 1 -- 172.21.15.132:0/2328114942 >> v1:172.21.15.132:6790/0 conn(0x7fd040133900 legacy=0x7fd040138280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.061+0000 7fd0471c7640 1 -- 172.21.15.132:0/2328114942 shutdown_connections 2024-01-24T23:20:58.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.061+0000 7fd0471c7640 1 -- 172.21.15.132:0/2328114942 >> 172.21.15.132:0/2328114942 conn(0x7fd0400a1de0 msgr2=0x7fd040131f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:20:58.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.061+0000 7fd0471c7640 1 -- 172.21.15.132:0/2328114942 shutdown_connections 2024-01-24T23:20:58.064 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.061+0000 7fd0471c7640 1 -- 172.21.15.132:0/2328114942 wait complete. 2024-01-24T23:20:58.078 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 5fcff598-1939-415b-b055-57940893269e 3 2024-01-24T23:20:58.149 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:20:58.145+0000 7f278372c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-24T23:20:58.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.185+0000 7fbf925e6640 10 monclient: get_monmap_and_config 2024-01-24T23:20:58.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.185+0000 7fbf925e6640 10 monclient: build_initial_monmap 2024-01-24T23:20:58.188 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.185+0000 7fbf925e6640 10 monclient: monmap: 2024-01-24T23:20:58.189 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-01-24T23:20:58.189 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:58.189 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:58.190951+0000 2024-01-24T23:20:58.189 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:58.190951+0000 2024-01-24T23:20:58.189 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:20:58.189 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:58.189 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:20:58.189 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:20:58.189 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:20:58.190 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:20:58.190 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:20:58.190 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:20:58.190 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:20:58.190 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:20:58.190 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:20:58.190 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:58.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.185+0000 7fbf925e6640 1 Processor -- start 2024-01-24T23:20:58.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.185+0000 7fbf925e6640 1 -- start start 2024-01-24T23:20:58.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.185+0000 7fbf925e6640 10 monclient: init 2024-01-24T23:20:58.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 10 monclient: _reopen_session rank -1 2024-01-24T23:20:58.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 10 monclient: _add_conns ranks=[6,0,3,4,8,1,2,5,7] 2024-01-24T23:20:58.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 10 monclient(hunting): picked mon.noname-d con 0x7fbf8c09f7b0 addr v1:172.21.15.115:6792/0 2024-01-24T23:20:58.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 10 monclient(hunting): picked mon.noname-a con 0x7fbf8c137b50 addr v1:172.21.15.115:6789/0 2024-01-24T23:20:58.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 10 monclient(hunting): picked mon.noname-g con 0x7fbf8c13b660 addr v1:172.21.15.132:6790/0 2024-01-24T23:20:58.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf8c04ed70 con 0x7fbf8c137b50 2024-01-24T23:20:58.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf8c0a02a0 con 0x7fbf8c13b660 2024-01-24T23:20:58.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf8c135780 con 0x7fbf8c09f7b0 2024-01-24T23:20:58.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 10 monclient(hunting): _renew_subs 2024-01-24T23:20:58.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 10 monclient(hunting): authenticate will time out at 945.516492s 2024-01-24T23:20:58.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf90b83640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fbf8c13b660 0x7fbf8c13da50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:36204/0 (socket says 172.21.15.132:36204) 2024-01-24T23:20:58.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf90b83640 1 -- 172.21.15.132:0/3263922706 learned_addr learned my addr 172.21.15.132:0/3263922706 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:20:58.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 1 -- 172.21.15.132:0/3263922706 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2711735385 0 0) 0x7fbf8c0a02a0 con 0x7fbf8c13b660 2024-01-24T23:20:58.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 10 monclient(hunting): _negotiate 2024-01-24T23:20:58.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:58.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:58.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 10 monclient(hunting): my global_id is 4134 2024-01-24T23:20:58.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 1 -- 172.21.15.132:0/3263922706 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf74003530 con 0x7fbf8c13b660 2024-01-24T23:20:58.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 1 -- 172.21.15.132:0/3263922706 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2446953804 0 0) 0x7fbf74003530 con 0x7fbf8c13b660 2024-01-24T23:20:58.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 1 -- 172.21.15.132:0/3263922706 >> v1:172.21.15.115:6792/0 conn(0x7fbf8c09f7b0 legacy=0x7fbf8c09fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 1 -- 172.21.15.132:0/3263922706 >> v1:172.21.15.115:6789/0 conn(0x7fbf8c137b50 legacy=0x7fbf8c139f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 10 monclient: _finish_hunting 0 2024-01-24T23:20:58.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 1 monclient: found mon.noname-g 2024-01-24T23:20:58.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:20:58.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:20:58.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 1 -- 172.21.15.132:0/3263922706 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8c134d00 con 0x7fbf8c13b660 2024-01-24T23:20:58.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 10 monclient: _finish_auth 0 2024-01-24T23:20:58.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 10 monclient: _check_auth_tickets 2024-01-24T23:20:58.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:20:58.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 1 -- 172.21.15.132:0/3263922706 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbf84001b40 con 0x7fbf8c13b660 2024-01-24T23:20:58.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 5 monclient: authenticate success, global_id 4134 2024-01-24T23:20:58.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:20:58.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:58.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:20:58.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 10 monclient: dump: 2024-01-24T23:20:58.199 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:58.199 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:58.200 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.200 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.200 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:58.200 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:58.200 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:58.200 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:58.200 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:58.200 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:58.200 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:58.200 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:58.200 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:58.200 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:58.200 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:58.200 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:58.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:20:58.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 1 -- 172.21.15.132:0/3263922706 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbf84001d60 con 0x7fbf8c13b660 2024-01-24T23:20:58.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:20:58.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 10 monclient: get_monmap_and_config success 2024-01-24T23:20:58.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 4 set_mon_vals no callback set 2024-01-24T23:20:58.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 1 -- 172.21.15.132:0/3263922706 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbf84004c30 con 0x7fbf8c13b660 2024-01-24T23:20:58.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:58.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:20:58.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf8affd640 10 monclient: dump: 2024-01-24T23:20:58.201 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:58.201 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:58.201 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.201 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.201 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:58.202 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:58.202 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:58.202 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:58.202 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:58.202 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:58.202 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:58.202 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:58.202 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:58.202 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:58.202 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:58.202 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:58.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 10 monclient: shutdown 2024-01-24T23:20:58.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:20:58.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 1 -- 172.21.15.132:0/3263922706 >> v1:172.21.15.132:6790/0 conn(0x7fbf8c13b660 legacy=0x7fbf8c13da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 1 -- 172.21.15.132:0/3263922706 shutdown_connections 2024-01-24T23:20:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 1 -- 172.21.15.132:0/3263922706 >> 172.21.15.132:0/3263922706 conn(0x7fbf8c0a6500 msgr2=0x7fbf8c0a2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:20:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 1 -- 172.21.15.132:0/3263922706 shutdown_connections 2024-01-24T23:20:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 1 -- 172.21.15.132:0/3263922706 wait complete. 2024-01-24T23:20:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 10 monclient: build_initial_monmap 2024-01-24T23:20:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 10 monclient: monmap: 2024-01-24T23:20:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-01-24T23:20:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:20:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-01-24T23:20:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-01-24T23:20:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:20:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-01-24T23:20:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:20:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:20:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:20:58.203 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:20:58.204 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:20:58.204 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:20:58.204 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:20:58.204 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:20:58.204 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:20:58.204 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:58.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 1 Processor -- start 2024-01-24T23:20:58.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 1 -- start start 2024-01-24T23:20:58.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 10 monclient: init 2024-01-24T23:20:58.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 10 monclient: _reopen_session rank -1 2024-01-24T23:20:58.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 10 monclient: _add_conns ranks=[6,1,4,7,3,0,8,2,5] 2024-01-24T23:20:58.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 10 monclient(hunting): picked mon.noname-d con 0x7fbf8c09f7b0 addr v1:172.21.15.115:6792/0 2024-01-24T23:20:58.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 10 monclient(hunting): picked mon.noname-f con 0x7fbf8c137b50 addr v1:172.21.15.132:6789/0 2024-01-24T23:20:58.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 10 monclient(hunting): picked mon.noname-c con 0x7fbf8c13b660 addr v1:172.21.15.115:6791/0 2024-01-24T23:20:58.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf8c06e030 con 0x7fbf8c137b50 2024-01-24T23:20:58.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf8c138d50 con 0x7fbf8c13b660 2024-01-24T23:20:58.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf8c0677b0 con 0x7fbf8c09f7b0 2024-01-24T23:20:58.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 10 monclient(hunting): _renew_subs 2024-01-24T23:20:58.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.189+0000 7fbf925e6640 10 monclient(hunting): authenticate will time out at 945.519999s 2024-01-24T23:20:58.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf8b7fe640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fbf8c137b50 0x7fbf8c138640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:55486/0 (socket says 172.21.15.132:55486) 2024-01-24T23:20:58.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf8b7fe640 1 -- 172.21.15.132:0/1967917905 learned_addr learned my addr 172.21.15.132:0/1967917905 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:20:58.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 1 -- 172.21.15.132:0/1967917905 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2898291784 0 0) 0x7fbf8c06e030 con 0x7fbf8c137b50 2024-01-24T23:20:58.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:20:58.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:58.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:58.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 10 monclient(hunting): my global_id is 4123 2024-01-24T23:20:58.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 1 -- 172.21.15.132:0/1967917905 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf78003530 con 0x7fbf8c137b50 2024-01-24T23:20:58.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 1 -- 172.21.15.132:0/1967917905 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 649355952 0 0) 0x7fbf78003530 con 0x7fbf8c137b50 2024-01-24T23:20:58.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 1 -- 172.21.15.132:0/1967917905 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbf8c06e030 con 0x7fbf8c137b50 2024-01-24T23:20:58.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 1 -- 172.21.15.132:0/1967917905 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbf80001b20 con 0x7fbf8c137b50 2024-01-24T23:20:58.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:58.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:20:58.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 10 monclient(hunting): dump: 2024-01-24T23:20:58.207 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:58.207 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:58.207 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.207 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.207 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:58.207 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:58.207 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:58.207 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:58.207 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:58.207 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:58.207 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:58.207 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:58.207 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:58.207 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:58.207 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:58.208 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:58.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:20:58.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 1 -- 172.21.15.132:0/1967917905 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3475128103 0 0) 0x7fbf8c06e030 con 0x7fbf8c137b50 2024-01-24T23:20:58.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 1 -- 172.21.15.132:0/1967917905 >> v1:172.21.15.115:6792/0 conn(0x7fbf8c09f7b0 legacy=0x7fbf8c137f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 1 -- 172.21.15.132:0/1967917905 >> v1:172.21.15.115:6791/0 conn(0x7fbf8c13b660 legacy=0x7fbf8c1d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:20:58.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 1 monclient: found mon.f 2024-01-24T23:20:58.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:20:58.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:20:58.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 1 -- 172.21.15.132:0/1967917905 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8c13aa40 con 0x7fbf8c137b50 2024-01-24T23:20:58.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 10 monclient: _finish_auth 0 2024-01-24T23:20:58.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:20:58.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:20:58.208 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf925e6640 5 monclient: authenticate success, global_id 4123 2024-01-24T23:20:58.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf925e6640 10 monclient: _renew_subs 2024-01-24T23:20:58.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf925e6640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:20:58.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf925e6640 1 -- 172.21.15.132:0/1967917905 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf8c13ad20 con 0x7fbf8c137b50 2024-01-24T23:20:58.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf925e6640 10 monclient: _renew_subs 2024-01-24T23:20:58.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf925e6640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:20:58.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf925e6640 1 -- 172.21.15.132:0/1967917905 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf8c134030 con 0x7fbf8c137b50 2024-01-24T23:20:58.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 1 -- 172.21.15.132:0/1967917905 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbf800038d0 con 0x7fbf8c137b50 2024-01-24T23:20:58.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:20:58.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 1 -- 172.21.15.132:0/1967917905 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbf80003e00 con 0x7fbf8c137b50 2024-01-24T23:20:58.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:58.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:20:58.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf8a7fc640 4 set_mon_vals no callback set 2024-01-24T23:20:58.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 10 monclient: dump: 2024-01-24T23:20:58.209 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:58.209 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:58.210 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.210 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.210 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:58.210 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:58.210 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:58.210 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:58.210 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:58.210 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:58.210 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:58.210 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:58.210 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:58.210 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:58.210 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:58.210 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:58.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 1 -- 172.21.15.132:0/1967917905 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 1) v1 ==== 799+0+0 (unknown 314974831 0 0) 0x7fbf800042e0 con 0x7fbf8c137b50 2024-01-24T23:20:58.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 1 -- 172.21.15.132:0/1967917905 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (unknown 3308858015 0 0) 0x7fbf80006bb0 con 0x7fbf8c137b50 2024-01-24T23:20:58.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf925e6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:58.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf925e6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:58.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf925e6640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:20:58.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf925e6640 1 -- 172.21.15.132:0/1967917905 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf6c005090 con 0x7fbf8c137b50 2024-01-24T23:20:58.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 1 -- 172.21.15.132:0/1967917905 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (unknown 1092875540 0 4287968521) 0x7fbf80006790 con 0x7fbf8c137b50 2024-01-24T23:20:58.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:58.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.193+0000 7fbf88ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:20:58.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.261+0000 7fbf925e6640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "5fcff598-1939-415b-b055-57940893269e", "id": 3}] 2024-01-24T23:20:58.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.261+0000 7fbf925e6640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "5fcff598-1939-415b-b055-57940893269e", "id": 3}] 2024-01-24T23:20:58.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.261+0000 7fbf925e6640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:20:58.266 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.261+0000 7fbf925e6640 1 -- 172.21.15.132:0/1967917905 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd new", "uuid": "5fcff598-1939-415b-b055-57940893269e", "id": 3} v 0) v1 -- 0x7fbf6c005340 con 0x7fbf8c137b50 2024-01-24T23:20:58.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.269+0000 7fbf88ff9640 1 -- 172.21.15.132:0/1967917905 <== mon.1 v1:172.21.15.132:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5fcff598-1939-415b-b055-57940893269e", "id": 3}]=0 v6) v1 ==== 112+0+2 (unknown 3363970265 0 525046306) 0x7fbf80005960 con 0x7fbf8c137b50 2024-01-24T23:20:58.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.269+0000 7fbf88ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "5fcff598-1939-415b-b055-57940893269e", "id": 3}] 2024-01-24T23:20:58.272 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.269+0000 7fbf88ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:20:58.272 INFO:teuthology.orchestra.run.smithi132.stdout:3 2024-01-24T23:20:58.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.269+0000 7fbf925e6640 10 monclient: shutdown 2024-01-24T23:20:58.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.269+0000 7fbf925e6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:20:58.273 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.269+0000 7fbf925e6640 1 -- 172.21.15.132:0/1967917905 >> v1:172.21.15.132:6789/0 conn(0x7fbf8c137b50 legacy=0x7fbf8c138640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.269+0000 7fbf925e6640 1 -- 172.21.15.132:0/1967917905 shutdown_connections 2024-01-24T23:20:58.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.269+0000 7fbf925e6640 1 -- 172.21.15.132:0/1967917905 >> 172.21.15.132:0/1967917905 conn(0x7fbf8c0a6500 msgr2=0x7fbf8c136bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:20:58.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.269+0000 7fbf925e6640 1 -- 172.21.15.132:0/1967917905 shutdown_connections 2024-01-24T23:20:58.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.269+0000 7fbf925e6640 1 -- 172.21.15.132:0/1967917905 wait complete. 2024-01-24T23:20:58.288 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new ef67ccd9-a21c-4a40-99ff-93f5b10a1fd3 4 2024-01-24T23:20:58.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782c4b3640 10 monclient: get_monmap_and_config 2024-01-24T23:20:58.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782c4b3640 10 monclient: build_initial_monmap 2024-01-24T23:20:58.400 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782c4b3640 10 monclient: monmap: 2024-01-24T23:20:58.400 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-01-24T23:20:58.400 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:58.400 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:58.402347+0000 2024-01-24T23:20:58.400 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:58.402347+0000 2024-01-24T23:20:58.400 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:20:58.400 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:58.400 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:20:58.400 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:20:58.400 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:20:58.401 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:20:58.401 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:20:58.401 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:20:58.401 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:20:58.401 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:20:58.401 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:20:58.401 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:58.404 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782c4b3640 1 Processor -- start 2024-01-24T23:20:58.404 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782c4b3640 1 -- start start 2024-01-24T23:20:58.404 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782c4b3640 10 monclient: init 2024-01-24T23:20:58.404 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782c4b3640 10 monclient: _reopen_session rank -1 2024-01-24T23:20:58.405 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782c4b3640 10 monclient: _add_conns ranks=[5,3,4,2,0,6,7,8,1] 2024-01-24T23:20:58.405 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782c4b3640 10 monclient(hunting): picked mon.noname-h con 0x7f782409b790 addr v1:172.21.15.132:6791/0 2024-01-24T23:20:58.405 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782c4b3640 10 monclient(hunting): picked mon.noname-g con 0x7f7824066550 addr v1:172.21.15.132:6790/0 2024-01-24T23:20:58.405 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782c4b3640 10 monclient(hunting): picked mon.noname-c con 0x7f782413ce90 addr v1:172.21.15.115:6791/0 2024-01-24T23:20:58.405 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782c4b3640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f782404ed70 con 0x7f7824066550 2024-01-24T23:20:58.405 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782c4b3640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f782409c280 con 0x7f782413ce90 2024-01-24T23:20:58.405 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782c4b3640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7824137750 con 0x7f782409b790 2024-01-24T23:20:58.405 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782c4b3640 10 monclient(hunting): _renew_subs 2024-01-24T23:20:58.405 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782c4b3640 10 monclient(hunting): authenticate will time out at 945.727393s 2024-01-24T23:20:58.405 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f7829a4e640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f7824066550 0x7f7824137040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:36206/0 (socket says 172.21.15.132:36206) 2024-01-24T23:20:58.405 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f7829a4e640 1 -- 172.21.15.132:0/734398349 learned_addr learned my addr 172.21.15.132:0/734398349 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:20:58.405 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782924d640 1 -- 172.21.15.132:0/734398349 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1705098853 0 0) 0x7f7824137750 con 0x7f782409b790 2024-01-24T23:20:58.405 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782924d640 10 monclient(hunting): _negotiate 2024-01-24T23:20:58.406 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782924d640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:58.406 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782924d640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:58.406 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782924d640 10 monclient(hunting): my global_id is 4136 2024-01-24T23:20:58.406 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782924d640 1 -- 172.21.15.132:0/734398349 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7810003530 con 0x7f782409b790 2024-01-24T23:20:58.406 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782924d640 1 -- 172.21.15.132:0/734398349 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3844634133 0 0) 0x7f782404ed70 con 0x7f7824066550 2024-01-24T23:20:58.406 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782924d640 10 monclient(hunting): _negotiate 2024-01-24T23:20:58.406 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782924d640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:58.406 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782924d640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:58.406 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782924d640 10 monclient(hunting): my global_id is 4143 2024-01-24T23:20:58.406 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.397+0000 7f782924d640 1 -- 172.21.15.132:0/734398349 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7824137750 con 0x7f7824066550 2024-01-24T23:20:58.406 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782924d640 1 -- 172.21.15.132:0/734398349 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2417245624 0 0) 0x7f7810003530 con 0x7f782409b790 2024-01-24T23:20:58.406 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782924d640 1 -- 172.21.15.132:0/734398349 >> v1:172.21.15.115:6791/0 conn(0x7f782413ce90 legacy=0x7f782413f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.406 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782924d640 1 -- 172.21.15.132:0/734398349 >> v1:172.21.15.132:6790/0 conn(0x7f7824066550 legacy=0x7f7824137040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.406 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782924d640 10 monclient: _finish_hunting 0 2024-01-24T23:20:58.406 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782924d640 1 monclient: found mon.noname-h 2024-01-24T23:20:58.407 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782924d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:20:58.407 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782924d640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:20:58.407 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782924d640 1 -- 172.21.15.132:0/734398349 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7824099800 con 0x7f782409b790 2024-01-24T23:20:58.407 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782924d640 10 monclient: _finish_auth 0 2024-01-24T23:20:58.407 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782924d640 10 monclient: _check_auth_tickets 2024-01-24T23:20:58.407 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782924d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:20:58.407 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 5 monclient: authenticate success, global_id 4136 2024-01-24T23:20:58.407 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782924d640 1 -- 172.21.15.132:0/734398349 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f78200017e0 con 0x7f782409b790 2024-01-24T23:20:58.407 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:20:58.407 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782924d640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:58.407 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782924d640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:20:58.407 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782924d640 10 monclient: dump: 2024-01-24T23:20:58.407 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:58.407 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:58.407 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.408 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.408 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:58.408 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:58.408 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:58.408 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:58.408 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:58.408 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:58.408 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:58.408 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:58.408 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:58.408 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:58.408 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:58.408 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:58.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:20:58.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782924d640 1 -- 172.21.15.132:0/734398349 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7820001a10 con 0x7f782409b790 2024-01-24T23:20:58.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782924d640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:20:58.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 10 monclient: get_monmap_and_config success 2024-01-24T23:20:58.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 4 set_mon_vals no callback set 2024-01-24T23:20:58.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782924d640 1 -- 172.21.15.132:0/734398349 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7820003960 con 0x7f782409b790 2024-01-24T23:20:58.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782924d640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:58.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782924d640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:20:58.409 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782924d640 10 monclient: dump: 2024-01-24T23:20:58.409 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:58.409 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:58.409 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.409 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.409 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:58.409 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:58.409 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:58.410 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:58.410 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:58.410 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:58.410 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:58.410 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:58.410 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:58.410 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:58.410 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:58.410 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:58.410 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 10 monclient: shutdown 2024-01-24T23:20:58.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:20:58.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 1 -- 172.21.15.132:0/734398349 >> v1:172.21.15.132:6791/0 conn(0x7f782409b790 legacy=0x7f782409bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 1 -- 172.21.15.132:0/734398349 shutdown_connections 2024-01-24T23:20:58.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 1 -- 172.21.15.132:0/734398349 >> 172.21.15.132:0/734398349 conn(0x7f78240a1fa0 msgr2=0x7f78240a2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:20:58.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 1 -- 172.21.15.132:0/734398349 shutdown_connections 2024-01-24T23:20:58.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 1 -- 172.21.15.132:0/734398349 wait complete. 2024-01-24T23:20:58.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 10 monclient: build_initial_monmap 2024-01-24T23:20:58.411 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 10 monclient: monmap: 2024-01-24T23:20:58.411 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-01-24T23:20:58.411 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:20:58.411 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-01-24T23:20:58.411 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-01-24T23:20:58.411 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:20:58.411 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-01-24T23:20:58.412 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:20:58.412 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:20:58.412 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:20:58.412 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:20:58.412 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:20:58.412 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:20:58.412 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:20:58.412 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:20:58.412 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:20:58.412 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:58.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 1 Processor -- start 2024-01-24T23:20:58.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 1 -- start start 2024-01-24T23:20:58.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 10 monclient: init 2024-01-24T23:20:58.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 10 monclient: _reopen_session rank -1 2024-01-24T23:20:58.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 10 monclient: _add_conns ranks=[2,6,0,4,5,1,3,7,8] 2024-01-24T23:20:58.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 10 monclient(hunting): picked mon.noname-b con 0x7f7824066550 addr v1:172.21.15.115:6790/0 2024-01-24T23:20:58.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 10 monclient(hunting): picked mon.noname-d con 0x7f782409b790 addr v1:172.21.15.115:6792/0 2024-01-24T23:20:58.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 10 monclient(hunting): picked mon.noname-a con 0x7f782413ce90 addr v1:172.21.15.115:6789/0 2024-01-24T23:20:58.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f782413e570 con 0x7f782413ce90 2024-01-24T23:20:58.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78240729a0 con 0x7f7824066550 2024-01-24T23:20:58.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7824072450 con 0x7f782409b790 2024-01-24T23:20:58.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 10 monclient(hunting): _renew_subs 2024-01-24T23:20:58.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782c4b3640 10 monclient(hunting): authenticate will time out at 945.731035s 2024-01-24T23:20:58.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f7829a4e640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f782409b790 0x7f782413d980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:54586/0 (socket says 172.21.15.132:54586) 2024-01-24T23:20:58.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f782aa50640 1 --1- 172.21.15.132:0/2776215909 >> v1:172.21.15.115:6789/0 conn(0x7f782413ce90 0x7f78241d3a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:38608/0 (socket says 172.21.15.132:38608) 2024-01-24T23:20:58.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f7829a4e640 1 -- 172.21.15.132:0/2776215909 learned_addr learned my addr 172.21.15.132:0/2776215909 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:20:58.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1375909744 0 0) 0x7f7824072450 con 0x7f782409b790 2024-01-24T23:20:58.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f7816ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:20:58.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f7816ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:58.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f7816ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:58.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f7816ffd640 10 monclient(hunting): my global_id is 4155 2024-01-24T23:20:58.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7808003530 con 0x7f782409b790 2024-01-24T23:20:58.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 293963252 0 0) 0x7f782413e570 con 0x7f782413ce90 2024-01-24T23:20:58.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f7816ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:20:58.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f7816ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:58.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f7816ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:58.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f7816ffd640 10 monclient(hunting): my global_id is 4149 2024-01-24T23:20:58.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.401+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7824072450 con 0x7f782413ce90 2024-01-24T23:20:58.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 367843413 0 0) 0x7f78240729a0 con 0x7f7824066550 2024-01-24T23:20:58.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:20:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 10 monclient(hunting): my global_id is 4124 2024-01-24T23:20:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f782413e570 con 0x7f7824066550 2024-01-24T23:20:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3783288723 0 0) 0x7f7824072450 con 0x7f782413ce90 2024-01-24T23:20:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f78240729a0 con 0x7f782413ce90 2024-01-24T23:20:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f781c001c00 con 0x7f782413ce90 2024-01-24T23:20:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:20:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 10 monclient(hunting): dump: 2024-01-24T23:20:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.419 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.420 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:58.420 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:58.420 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:58.420 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:58.420 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:58.420 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:58.420 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:58.420 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:58.420 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:58.420 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:58.420 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:58.420 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:58.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:20:58.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3980505940 0 0) 0x7f7808003530 con 0x7f782409b790 2024-01-24T23:20:58.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7824072450 con 0x7f782409b790 2024-01-24T23:20:58.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7818001510 con 0x7f782409b790 2024-01-24T23:20:58.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:58.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:20:58.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 10 monclient(hunting): dump: 2024-01-24T23:20:58.421 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:58.421 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:58.421 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.421 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.421 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:58.422 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:58.422 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:58.422 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:58.422 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:58.422 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:58.422 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:58.422 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:58.422 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:58.422 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:58.422 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:58.422 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:58.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 115285531 0 0) 0x7f782413e570 con 0x7f7824066550 2024-01-24T23:20:58.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7808003530 con 0x7f7824066550 2024-01-24T23:20:58.422 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7820003940 con 0x7f7824066550 2024-01-24T23:20:58.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:58.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:20:58.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 10 monclient(hunting): dump: 2024-01-24T23:20:58.423 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:58.423 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:58.423 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.423 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.423 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:58.423 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:58.423 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:58.423 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:58.423 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:58.423 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:58.423 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:58.424 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:58.424 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:58.424 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:58.424 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:58.424 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:58.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2772296806 0 0) 0x7f78240729a0 con 0x7f782413ce90 2024-01-24T23:20:58.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 >> v1:172.21.15.115:6792/0 conn(0x7f782409b790 legacy=0x7f782413d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 >> v1:172.21.15.115:6790/0 conn(0x7f7824066550 legacy=0x7f782413d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:20:58.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 1 monclient: found mon.a 2024-01-24T23:20:58.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:20:58.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:20:58.424 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782408c010 con 0x7f782413ce90 2024-01-24T23:20:58.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 10 monclient: _finish_auth 0 2024-01-24T23:20:58.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:20:58.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:20:58.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f782c4b3640 5 monclient: authenticate success, global_id 4149 2024-01-24T23:20:58.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f782c4b3640 10 monclient: _renew_subs 2024-01-24T23:20:58.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f782c4b3640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:20:58.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f782c4b3640 1 -- 172.21.15.132:0/2776215909 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f782413e090 con 0x7f782413ce90 2024-01-24T23:20:58.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f782c4b3640 10 monclient: _renew_subs 2024-01-24T23:20:58.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f782c4b3640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:20:58.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f782c4b3640 1 -- 172.21.15.132:0/2776215909 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f782413e740 con 0x7f782413ce90 2024-01-24T23:20:58.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f781c0039a0 con 0x7f782413ce90 2024-01-24T23:20:58.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:20:58.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7828a4c640 4 set_mon_vals no callback set 2024-01-24T23:20:58.425 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f781c003e50 con 0x7f782413ce90 2024-01-24T23:20:58.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:58.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:20:58.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 10 monclient: dump: 2024-01-24T23:20:58.426 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:58.426 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:58.426 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.426 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.426 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:58.426 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:58.426 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:58.426 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:58.426 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:58.426 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:58.426 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:58.426 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:58.427 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:58.427 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:58.427 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:58.427 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:58.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 1) v1 ==== 799+0+0 (unknown 314974831 0 0) 0x7f781c004470 con 0x7f782413ce90 2024-01-24T23:20:58.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (unknown 1603967552 0 0) 0x7f781c004720 con 0x7f782413ce90 2024-01-24T23:20:58.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f782c4b3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:58.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f782c4b3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:58.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f782c4b3640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:20:58.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.405+0000 7f782c4b3640 1 -- 172.21.15.132:0/2776215909 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f780c005090 con 0x7f782413ce90 2024-01-24T23:20:58.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.409+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 <== mon.0 v1:172.21.15.115:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (unknown 1092875540 0 4287968521) 0x7f781c006810 con 0x7f782413ce90 2024-01-24T23:20:58.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.409+0000 7f7816ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:58.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.409+0000 7f7816ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:20:58.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.477+0000 7f782c4b3640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "ef67ccd9-a21c-4a40-99ff-93f5b10a1fd3", "id": 4}] 2024-01-24T23:20:58.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.477+0000 7f782c4b3640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "ef67ccd9-a21c-4a40-99ff-93f5b10a1fd3", "id": 4}] 2024-01-24T23:20:58.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.477+0000 7f782c4b3640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:20:58.481 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.477+0000 7f782c4b3640 1 -- 172.21.15.132:0/2776215909 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "osd new", "uuid": "ef67ccd9-a21c-4a40-99ff-93f5b10a1fd3", "id": 4} v 0) v1 -- 0x7f780c0058c0 con 0x7f782413ce90 2024-01-24T23:20:58.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.485+0000 7f7816ffd640 1 -- 172.21.15.132:0/2776215909 <== mon.0 v1:172.21.15.115:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ef67ccd9-a21c-4a40-99ff-93f5b10a1fd3", "id": 4}]=0 v7) v1 ==== 112+0+2 (unknown 576886787 0 1697012583) 0x7f781c0069b0 con 0x7f782413ce90 2024-01-24T23:20:58.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.485+0000 7f7816ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "ef67ccd9-a21c-4a40-99ff-93f5b10a1fd3", "id": 4}] 2024-01-24T23:20:58.489 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.485+0000 7f7816ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:20:58.489 INFO:teuthology.orchestra.run.smithi132.stdout:4 2024-01-24T23:20:58.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.489+0000 7f782c4b3640 10 monclient: shutdown 2024-01-24T23:20:58.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.489+0000 7f782c4b3640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:20:58.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.489+0000 7f782c4b3640 1 -- 172.21.15.132:0/2776215909 >> v1:172.21.15.115:6789/0 conn(0x7f782413ce90 legacy=0x7f78241d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.490 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.489+0000 7f782c4b3640 1 -- 172.21.15.132:0/2776215909 shutdown_connections 2024-01-24T23:20:58.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.489+0000 7f782c4b3640 1 -- 172.21.15.132:0/2776215909 >> 172.21.15.132:0/2776215909 conn(0x7f78240a1fa0 msgr2=0x7f78240a10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:20:58.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.489+0000 7f782c4b3640 1 -- 172.21.15.132:0/2776215909 shutdown_connections 2024-01-24T23:20:58.491 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.489+0000 7f782c4b3640 1 -- 172.21.15.132:0/2776215909 wait complete. 2024-01-24T23:20:58.505 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new c2573ce8-6dd2-4272-85c7-2f5aa25eb2f9 5 2024-01-24T23:20:58.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.609+0000 7fa781434640 10 monclient: get_monmap_and_config 2024-01-24T23:20:58.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.609+0000 7fa781434640 10 monclient: build_initial_monmap 2024-01-24T23:20:58.614 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.609+0000 7fa781434640 10 monclient: monmap: 2024-01-24T23:20:58.614 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-01-24T23:20:58.614 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:58.614 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:58.613877+0000 2024-01-24T23:20:58.614 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:58.613877+0000 2024-01-24T23:20:58.614 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:20:58.614 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:58.614 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:20:58.614 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:20:58.615 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:20:58.615 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:20:58.615 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:20:58.615 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:20:58.615 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:20:58.615 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:20:58.616 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:20:58.616 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:58.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.609+0000 7fa781434640 1 Processor -- start 2024-01-24T23:20:58.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.609+0000 7fa781434640 1 -- start start 2024-01-24T23:20:58.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.609+0000 7fa781434640 10 monclient: init 2024-01-24T23:20:58.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.609+0000 7fa781434640 10 monclient: _reopen_session rank -1 2024-01-24T23:20:58.624 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.609+0000 7fa781434640 10 monclient: _add_conns ranks=[6,2,3,8,5,1,0,7,4] 2024-01-24T23:20:58.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.609+0000 7fa781434640 10 monclient(hunting): picked mon.noname-d con 0x7fa77c131810 addr v1:172.21.15.115:6792/0 2024-01-24T23:20:58.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.609+0000 7fa781434640 10 monclient(hunting): picked mon.noname-b con 0x7fa77c1311d0 addr v1:172.21.15.115:6790/0 2024-01-24T23:20:58.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.609+0000 7fa781434640 10 monclient(hunting): picked mon.noname-g con 0x7fa77c06d960 addr v1:172.21.15.132:6790/0 2024-01-24T23:20:58.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.609+0000 7fa781434640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77c04ed70 con 0x7fa77c1311d0 2024-01-24T23:20:58.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.609+0000 7fa781434640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77c06e1f0 con 0x7fa77c06d960 2024-01-24T23:20:58.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.609+0000 7fa781434640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77c09fef0 con 0x7fa77c131810 2024-01-24T23:20:58.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.609+0000 7fa781434640 10 monclient(hunting): _renew_subs 2024-01-24T23:20:58.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.609+0000 7fa781434640 10 monclient(hunting): authenticate will time out at 945.939339s 2024-01-24T23:20:58.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.609+0000 7fa77b7fe640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fa77c06d960 0x7fa77c13da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:36220/0 (socket says 172.21.15.132:36220) 2024-01-24T23:20:58.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.609+0000 7fa77b7fe640 1 -- 172.21.15.132:0/3075870714 learned_addr learned my addr 172.21.15.132:0/3075870714 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:20:58.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.609+0000 7fa77a7fc640 1 -- 172.21.15.132:0/3075870714 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 138129026 0 0) 0x7fa77c06e1f0 con 0x7fa77c06d960 2024-01-24T23:20:58.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.609+0000 7fa77a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:20:58.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.609+0000 7fa77a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:58.625 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.609+0000 7fa77a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:58.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.609+0000 7fa77a7fc640 10 monclient(hunting): my global_id is 4152 2024-01-24T23:20:58.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.609+0000 7fa77a7fc640 1 -- 172.21.15.132:0/3075870714 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa768003530 con 0x7fa77c06d960 2024-01-24T23:20:58.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa77a7fc640 1 -- 172.21.15.132:0/3075870714 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3848959809 0 0) 0x7fa768003530 con 0x7fa77c06d960 2024-01-24T23:20:58.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa77a7fc640 1 -- 172.21.15.132:0/3075870714 >> v1:172.21.15.115:6792/0 conn(0x7fa77c131810 legacy=0x7fa77c0a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa77a7fc640 1 -- 172.21.15.132:0/3075870714 >> v1:172.21.15.115:6790/0 conn(0x7fa77c1311d0 legacy=0x7fa77c0a4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa77a7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:20:58.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa77a7fc640 1 monclient: found mon.noname-g 2024-01-24T23:20:58.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa77a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:20:58.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa77a7fc640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:20:58.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa77a7fc640 1 -- 172.21.15.132:0/3075870714 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c0a54b0 con 0x7fa77c06d960 2024-01-24T23:20:58.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa77a7fc640 10 monclient: _finish_auth 0 2024-01-24T23:20:58.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa77a7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:20:58.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa77a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:20:58.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa77a7fc640 1 -- 172.21.15.132:0/3075870714 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa774001b40 con 0x7fa77c06d960 2024-01-24T23:20:58.626 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa77a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:58.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa77a7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:20:58.627 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa77a7fc640 10 monclient: dump: 2024-01-24T23:20:58.627 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:58.627 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:58.627 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.627 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.627 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:58.627 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:58.627 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:58.627 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:58.627 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:58.627 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:58.627 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:58.627 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:58.627 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:58.628 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:58.628 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:58.628 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:58.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 5 monclient: authenticate success, global_id 4152 2024-01-24T23:20:58.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:20:58.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa77a7fc640 1 -- 172.21.15.132:0/3075870714 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa774001df0 con 0x7fa77c06d960 2024-01-24T23:20:58.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa77a7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:20:58.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 10 monclient: get_monmap_and_config success 2024-01-24T23:20:58.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 4 set_mon_vals no callback set 2024-01-24T23:20:58.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa77a7fc640 1 -- 172.21.15.132:0/3075870714 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa774004f70 con 0x7fa77c06d960 2024-01-24T23:20:58.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa77a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:58.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa77a7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:20:58.628 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa77a7fc640 10 monclient: dump: 2024-01-24T23:20:58.628 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:58.628 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:58.629 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.629 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.629 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:58.629 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:58.629 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:58.629 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:58.629 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:58.629 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:58.629 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:58.629 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:58.629 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:58.629 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:58.629 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:58.629 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:58.629 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 10 monclient: shutdown 2024-01-24T23:20:58.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:20:58.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 1 -- 172.21.15.132:0/3075870714 >> v1:172.21.15.132:6790/0 conn(0x7fa77c06d960 legacy=0x7fa77c13da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 1 -- 172.21.15.132:0/3075870714 shutdown_connections 2024-01-24T23:20:58.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 1 -- 172.21.15.132:0/3075870714 >> 172.21.15.132:0/3075870714 conn(0x7fa77c0a5d40 msgr2=0x7fa77c0a2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:20:58.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 1 -- 172.21.15.132:0/3075870714 shutdown_connections 2024-01-24T23:20:58.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 1 -- 172.21.15.132:0/3075870714 wait complete. 2024-01-24T23:20:58.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 10 monclient: build_initial_monmap 2024-01-24T23:20:58.630 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 10 monclient: monmap: 2024-01-24T23:20:58.630 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 0 2024-01-24T23:20:58.630 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:20:58.630 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 0.000000 2024-01-24T23:20:58.630 INFO:teuthology.orchestra.run.smithi132.stderr:created 0.000000 2024-01-24T23:20:58.630 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:20:58.630 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 1 2024-01-24T23:20:58.630 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:20:58.631 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:20:58.631 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:20:58.631 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:20:58.631 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:20:58.631 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:20:58.631 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:20:58.631 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:20:58.631 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:20:58.631 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:58.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 1 Processor -- start 2024-01-24T23:20:58.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 1 -- start start 2024-01-24T23:20:58.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 10 monclient: init 2024-01-24T23:20:58.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 10 monclient: _reopen_session rank -1 2024-01-24T23:20:58.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 10 monclient: _add_conns ranks=[6,4,5,1,7,8,2,3,0] 2024-01-24T23:20:58.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 10 monclient(hunting): picked mon.noname-d con 0x7fa77c06d960 addr v1:172.21.15.115:6792/0 2024-01-24T23:20:58.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 10 monclient(hunting): picked mon.noname-c con 0x7fa77c1311d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:20:58.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 10 monclient(hunting): picked mon.noname-h con 0x7fa77c131810 addr v1:172.21.15.132:6791/0 2024-01-24T23:20:58.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77c09e4b0 con 0x7fa77c1311d0 2024-01-24T23:20:58.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77c06de30 con 0x7fa77c131810 2024-01-24T23:20:58.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77c09d8e0 con 0x7fa77c06d960 2024-01-24T23:20:58.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 10 monclient(hunting): _renew_subs 2024-01-24T23:20:58.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 10 monclient(hunting): authenticate will time out at 945.942301s 2024-01-24T23:20:58.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa77b7fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fa77c131810 0x7fa77c1d1a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:60242/0 (socket says 172.21.15.132:60242) 2024-01-24T23:20:58.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa77b7fe640 1 -- 172.21.15.132:0/3897624913 learned_addr learned my addr 172.21.15.132:0/3897624913 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:20:58.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 1 -- 172.21.15.132:0/3897624913 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3125877514 0 0) 0x7fa77c06de30 con 0x7fa77c131810 2024-01-24T23:20:58.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 10 monclient(hunting): _negotiate 2024-01-24T23:20:58.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:58.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:58.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 10 monclient(hunting): my global_id is 4145 2024-01-24T23:20:58.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 1 -- 172.21.15.132:0/3897624913 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa75c003530 con 0x7fa77c131810 2024-01-24T23:20:58.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 1 -- 172.21.15.132:0/3897624913 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2920637401 0 0) 0x7fa75c003530 con 0x7fa77c131810 2024-01-24T23:20:58.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 1 -- 172.21.15.132:0/3897624913 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa77c06de30 con 0x7fa77c131810 2024-01-24T23:20:58.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 1 -- 172.21.15.132:0/3897624913 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa774001b00 con 0x7fa77c131810 2024-01-24T23:20:58.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:58.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:20:58.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 10 monclient(hunting): dump: 2024-01-24T23:20:58.633 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:58.633 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:58.633 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.633 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.633 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:58.633 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:58.633 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:58.633 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:58.634 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:58.634 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:58.634 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:58.634 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:58.634 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:58.634 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:58.634 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:58.634 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:58.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:20:58.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 1 -- 172.21.15.132:0/3897624913 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1366079388 0 0) 0x7fa77c09d8e0 con 0x7fa77c06d960 2024-01-24T23:20:58.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 10 monclient(hunting): _negotiate 2024-01-24T23:20:58.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:58.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:58.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 10 monclient(hunting): my global_id is 4173 2024-01-24T23:20:58.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 1 -- 172.21.15.132:0/3897624913 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa75c003530 con 0x7fa77c06d960 2024-01-24T23:20:58.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 1 -- 172.21.15.132:0/3897624913 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3509175042 0 0) 0x7fa77c09e4b0 con 0x7fa77c1311d0 2024-01-24T23:20:58.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 10 monclient(hunting): _negotiate 2024-01-24T23:20:58.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:58.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:58.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 10 monclient(hunting): my global_id is 74128 2024-01-24T23:20:58.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 1 -- 172.21.15.132:0/3897624913 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa77c09d8e0 con 0x7fa77c1311d0 2024-01-24T23:20:58.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 1 -- 172.21.15.132:0/3897624913 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 750133655 0 0) 0x7fa77c06de30 con 0x7fa77c131810 2024-01-24T23:20:58.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 1 -- 172.21.15.132:0/3897624913 >> v1:172.21.15.115:6792/0 conn(0x7fa77c06d960 legacy=0x7fa77c12e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 1 -- 172.21.15.132:0/3897624913 >> v1:172.21.15.115:6791/0 conn(0x7fa77c1311d0 legacy=0x7fa77c1ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 10 monclient: _finish_hunting 0 2024-01-24T23:20:58.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 1 monclient: found mon.h 2024-01-24T23:20:58.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:20:58.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:20:58.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 1 -- 172.21.15.132:0/3897624913 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c09f330 con 0x7fa77c131810 2024-01-24T23:20:58.635 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 10 monclient: _finish_auth 0 2024-01-24T23:20:58.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 10 monclient: _check_auth_tickets 2024-01-24T23:20:58.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:20:58.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 5 monclient: authenticate success, global_id 4145 2024-01-24T23:20:58.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 10 monclient: _renew_subs 2024-01-24T23:20:58.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:20:58.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 1 -- 172.21.15.132:0/3897624913 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa77c09e660 con 0x7fa77c131810 2024-01-24T23:20:58.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 10 monclient: _renew_subs 2024-01-24T23:20:58.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:20:58.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa781434640 1 -- 172.21.15.132:0/3897624913 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa77c12ec60 con 0x7fa77c131810 2024-01-24T23:20:58.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 1 -- 172.21.15.132:0/3897624913 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa774005240 con 0x7fa77c131810 2024-01-24T23:20:58.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:20:58.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 1 -- 172.21.15.132:0/3897624913 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa7740056f0 con 0x7fa77c131810 2024-01-24T23:20:58.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:58.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:20:58.637 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 10 monclient: dump: 2024-01-24T23:20:58.637 INFO:teuthology.orchestra.run.smithi132.stderr:epoch 1 2024-01-24T23:20:58.637 INFO:teuthology.orchestra.run.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:58.637 INFO:teuthology.orchestra.run.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.637 INFO:teuthology.orchestra.run.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.637 INFO:teuthology.orchestra.run.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:58.637 INFO:teuthology.orchestra.run.smithi132.stderr:election_strategy: 3 2024-01-24T23:20:58.637 INFO:teuthology.orchestra.run.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:58.637 INFO:teuthology.orchestra.run.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:58.637 INFO:teuthology.orchestra.run.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:58.637 INFO:teuthology.orchestra.run.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:58.637 INFO:teuthology.orchestra.run.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:58.637 INFO:teuthology.orchestra.run.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:58.637 INFO:teuthology.orchestra.run.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:58.638 INFO:teuthology.orchestra.run.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:58.638 INFO:teuthology.orchestra.run.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:58.638 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:20:58.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa779ffb640 4 set_mon_vals no callback set 2024-01-24T23:20:58.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 1 -- 172.21.15.132:0/3897624913 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 1) v1 ==== 799+0+0 (unknown 314974831 0 0) 0x7fa774005bd0 con 0x7fa77c131810 2024-01-24T23:20:58.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.613+0000 7fa7737fe640 1 -- 172.21.15.132:0/3897624913 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (unknown 1889410806 0 0) 0x7fa7740073d0 con 0x7fa77c131810 2024-01-24T23:20:58.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.617+0000 7fa781434640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:58.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.617+0000 7fa781434640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:58.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.617+0000 7fa781434640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:20:58.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.617+0000 7fa781434640 1 -- 172.21.15.132:0/3897624913 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa77c06a0d0 con 0x7fa77c131810 2024-01-24T23:20:58.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.617+0000 7fa7737fe640 1 -- 172.21.15.132:0/3897624913 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (unknown 1092875540 0 4287968521) 0x7fa774006e50 con 0x7fa77c131810 2024-01-24T23:20:58.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.617+0000 7fa7737fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:58.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.617+0000 7fa7737fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:20:58.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.685+0000 7fa781434640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "c2573ce8-6dd2-4272-85c7-2f5aa25eb2f9", "id": 5}] 2024-01-24T23:20:58.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.685+0000 7fa781434640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "c2573ce8-6dd2-4272-85c7-2f5aa25eb2f9", "id": 5}] 2024-01-24T23:20:58.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.685+0000 7fa781434640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:20:58.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.685+0000 7fa781434640 1 -- 172.21.15.132:0/3897624913 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd new", "uuid": "c2573ce8-6dd2-4272-85c7-2f5aa25eb2f9", "id": 5} v 0) v1 -- 0x7fa77c06b5c0 con 0x7fa77c131810 2024-01-24T23:20:58.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.693+0000 7fa7737fe640 1 -- 172.21.15.132:0/3897624913 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c2573ce8-6dd2-4272-85c7-2f5aa25eb2f9", "id": 5}]=0 v8) v1 ==== 112+0+2 (unknown 348649985 0 1988412176) 0x7fa7740070c0 con 0x7fa77c131810 2024-01-24T23:20:58.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.693+0000 7fa7737fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "c2573ce8-6dd2-4272-85c7-2f5aa25eb2f9", "id": 5}] 2024-01-24T23:20:58.697 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.693+0000 7fa7737fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:20:58.698 INFO:teuthology.orchestra.run.smithi132.stdout:5 2024-01-24T23:20:58.698 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.697+0000 7fa781434640 10 monclient: shutdown 2024-01-24T23:20:58.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.697+0000 7fa781434640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:20:58.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.697+0000 7fa781434640 1 -- 172.21.15.132:0/3897624913 >> v1:172.21.15.132:6791/0 conn(0x7fa77c131810 legacy=0x7fa77c1d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.697+0000 7fa781434640 1 -- 172.21.15.132:0/3897624913 shutdown_connections 2024-01-24T23:20:58.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.697+0000 7fa781434640 1 -- 172.21.15.132:0/3897624913 >> 172.21.15.132:0/3897624913 conn(0x7fa77c0a5d40 msgr2=0x7fa77c074860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:20:58.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.697+0000 7fa781434640 1 -- 172.21.15.132:0/3897624913 shutdown_connections 2024-01-24T23:20:58.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-01-24T23:20:58.697+0000 7fa781434640 1 -- 172.21.15.132:0/3897624913 wait complete. 2024-01-24T23:20:58.702 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:20:58.701+0000 7f278372c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-24T23:20:58.713 INFO:tasks.ceph.osd.0:Restarting daemon 2024-01-24T23:20:58.713 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-01-24T23:20:58.716 INFO:tasks.ceph.osd.0:Started 2024-01-24T23:20:58.716 INFO:tasks.ceph.osd.1:Restarting daemon 2024-01-24T23:20:58.716 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-01-24T23:20:58.718 INFO:tasks.ceph.osd.1:Started 2024-01-24T23:20:58.719 INFO:tasks.ceph.osd.2:Restarting daemon 2024-01-24T23:20:58.719 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-01-24T23:20:58.721 INFO:tasks.ceph.osd.2:Started 2024-01-24T23:20:58.721 INFO:tasks.ceph.osd.3:Restarting daemon 2024-01-24T23:20:58.722 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-01-24T23:20:58.723 INFO:tasks.ceph.osd.3:Started 2024-01-24T23:20:58.723 INFO:tasks.ceph.osd.4:Restarting daemon 2024-01-24T23:20:58.723 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-01-24T23:20:58.725 INFO:tasks.ceph.osd.4:Started 2024-01-24T23:20:58.725 INFO:tasks.ceph.osd.5:Restarting daemon 2024-01-24T23:20:58.725 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-01-24T23:20:58.727 INFO:tasks.ceph.osd.5:Started 2024-01-24T23:20:58.727 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T23:20:58.759 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:20:58.757+0000 7f278372c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-24T23:20:58.816 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:20:58.813+0000 7f278372c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-24T23:20:58.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e7fea640 10 monclient: get_monmap_and_config 2024-01-24T23:20:58.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e7fea640 10 monclient: build_initial_monmap 2024-01-24T23:20:58.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e7fea640 10 monclient: monmap: 2024-01-24T23:20:58.847 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:20:58.847 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:58.848 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:58.848948+0000 2024-01-24T23:20:58.848 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:58.848948+0000 2024-01-24T23:20:58.848 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:20:58.848 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:20:58.848 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:20:58.848 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:20:58.848 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:20:58.848 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:20:58.848 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:20:58.848 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:20:58.848 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:20:58.848 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:20:58.848 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:20:58.848 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:20:58.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e7fea640 1 Processor -- start 2024-01-24T23:20:58.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e7fea640 1 -- start start 2024-01-24T23:20:58.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e7fea640 10 monclient: init 2024-01-24T23:20:58.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e7fea640 10 monclient: _reopen_session rank -1 2024-01-24T23:20:58.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e7fea640 10 monclient: _add_conns ranks=[0,7,4,8,5,2,3,1,6] 2024-01-24T23:20:58.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e7fea640 10 monclient(hunting): picked mon.noname-a con 0x7f57e0133bf0 addr v1:172.21.15.115:6789/0 2024-01-24T23:20:58.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e7fea640 10 monclient(hunting): picked mon.noname-i con 0x7f57e0133fd0 addr v1:172.21.15.132:6792/0 2024-01-24T23:20:58.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e7fea640 10 monclient(hunting): picked mon.noname-c con 0x7f57e01343b0 addr v1:172.21.15.115:6791/0 2024-01-24T23:20:58.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e7fea640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e004ed70 con 0x7f57e0133bf0 2024-01-24T23:20:58.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e7fea640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e00a4b60 con 0x7f57e01343b0 2024-01-24T23:20:58.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e7fea640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e009ae50 con 0x7f57e0133fd0 2024-01-24T23:20:58.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e7fea640 10 monclient(hunting): _renew_subs 2024-01-24T23:20:58.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e7fea640 10 monclient(hunting): authenticate will time out at 937.771311s 2024-01-24T23:20:58.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e5d86640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f57e0133bf0 0x7f57e009bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:55268/0 (socket says 172.21.15.115:55268) 2024-01-24T23:20:58.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e5d86640 1 -- 172.21.15.115:0/1172237416 learned_addr learned my addr 172.21.15.115:0/1172237416 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:20:58.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e4d84640 1 -- 172.21.15.115:0/1172237416 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583400764 0 0) 0x7f57e00a4b60 con 0x7f57e01343b0 2024-01-24T23:20:58.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e4d84640 10 monclient(hunting): _negotiate 2024-01-24T23:20:58.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e4d84640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:58.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e4d84640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:58.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e4d84640 10 monclient(hunting): my global_id is 74146 2024-01-24T23:20:58.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e4d84640 1 -- 172.21.15.115:0/1172237416 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57cc003530 con 0x7f57e01343b0 2024-01-24T23:20:58.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e4d84640 1 -- 172.21.15.115:0/1172237416 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2173766887 0 0) 0x7f57e004ed70 con 0x7f57e0133bf0 2024-01-24T23:20:58.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e4d84640 10 monclient(hunting): _negotiate 2024-01-24T23:20:58.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e4d84640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:58.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e4d84640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:58.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e4d84640 10 monclient(hunting): my global_id is 4185 2024-01-24T23:20:58.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e4d84640 1 -- 172.21.15.115:0/1172237416 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57e00a4b60 con 0x7f57e0133bf0 2024-01-24T23:20:58.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e4d84640 1 -- 172.21.15.115:0/1172237416 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 121266511 0 0) 0x7f57e009ae50 con 0x7f57e0133fd0 2024-01-24T23:20:58.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e4d84640 10 monclient(hunting): _negotiate 2024-01-24T23:20:58.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e4d84640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:58.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e4d84640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:58.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e4d84640 10 monclient(hunting): my global_id is 4147 2024-01-24T23:20:58.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.843+0000 7f57e4d84640 1 -- 172.21.15.115:0/1172237416 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57e004ed70 con 0x7f57e0133fd0 2024-01-24T23:20:58.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e4d84640 1 -- 172.21.15.115:0/1172237416 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 704599126 0 0) 0x7f57cc003530 con 0x7f57e01343b0 2024-01-24T23:20:58.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e4d84640 1 -- 172.21.15.115:0/1172237416 >> v1:172.21.15.132:6792/0 conn(0x7f57e0133fd0 legacy=0x7f57e013bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e4d84640 1 -- 172.21.15.115:0/1172237416 >> v1:172.21.15.115:6789/0 conn(0x7f57e0133bf0 legacy=0x7f57e009bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e4d84640 10 monclient: _finish_hunting 0 2024-01-24T23:20:58.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e4d84640 1 monclient: found mon.noname-c 2024-01-24T23:20:58.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e4d84640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:20:58.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e4d84640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:20:58.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e4d84640 1 -- 172.21.15.115:0/1172237416 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e009b130 con 0x7f57e01343b0 2024-01-24T23:20:58.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e4d84640 10 monclient: _finish_auth 0 2024-01-24T23:20:58.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e4d84640 10 monclient: _check_auth_tickets 2024-01-24T23:20:58.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e4d84640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:20:58.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e4d84640 1 -- 172.21.15.115:0/1172237416 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f57d80017c0 con 0x7f57e01343b0 2024-01-24T23:20:58.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e4d84640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:58.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e4d84640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:20:58.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e4d84640 10 monclient: dump: 2024-01-24T23:20:58.858 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:20:58.858 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:58.858 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.858 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.859 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:58.859 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:20:58.859 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:58.859 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:58.859 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:58.859 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:58.859 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:58.859 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:58.859 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:58.859 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:58.859 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:58.859 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:20:58.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 5 monclient: authenticate success, global_id 74146 2024-01-24T23:20:58.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:20:58.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e4d84640 1 -- 172.21.15.115:0/1172237416 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f57d8001aa0 con 0x7f57e01343b0 2024-01-24T23:20:58.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e4d84640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:20:58.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e4d84640 1 -- 172.21.15.115:0/1172237416 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f57d8003960 con 0x7f57e01343b0 2024-01-24T23:20:58.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e4d84640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:58.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e4d84640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:20:58.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e4d84640 10 monclient: dump: 2024-01-24T23:20:58.860 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:20:58.860 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:58.860 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.860 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.860 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:58.860 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:20:58.860 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:58.860 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:58.860 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:58.861 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:58.861 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:58.861 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:58.861 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:58.861 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:58.861 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:58.861 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:20:58.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 10 monclient: get_monmap_and_config success 2024-01-24T23:20:58.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 4 set_mon_vals no callback set 2024-01-24T23:20:58.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 10 monclient: shutdown 2024-01-24T23:20:58.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:20:58.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 1 -- 172.21.15.115:0/1172237416 >> v1:172.21.15.115:6791/0 conn(0x7f57e01343b0 legacy=0x7f57e013f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 1 -- 172.21.15.115:0/1172237416 shutdown_connections 2024-01-24T23:20:58.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 1 -- 172.21.15.115:0/1172237416 >> 172.21.15.115:0/1172237416 conn(0x7f57e00662f0 msgr2=0x7f57e0066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:20:58.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 1 -- 172.21.15.115:0/1172237416 shutdown_connections 2024-01-24T23:20:58.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 1 -- 172.21.15.115:0/1172237416 wait complete. 2024-01-24T23:20:58.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 10 monclient: build_initial_monmap 2024-01-24T23:20:58.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 10 monclient: monmap: 2024-01-24T23:20:58.862 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:20:58.862 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:20:58.862 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:20:58.862 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:20:58.862 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:20:58.862 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:20:58.862 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:20:58.862 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:20:58.862 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:20:58.862 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:20:58.862 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:20:58.862 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:20:58.862 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:20:58.863 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:20:58.863 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:20:58.863 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:20:58.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 1 Processor -- start 2024-01-24T23:20:58.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 1 -- start start 2024-01-24T23:20:58.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 10 monclient: init 2024-01-24T23:20:58.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 10 monclient: _reopen_session rank -1 2024-01-24T23:20:58.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 10 monclient: _add_conns ranks=[4,8,3,6,5,1,7,0,2] 2024-01-24T23:20:58.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 10 monclient(hunting): picked mon.noname-c con 0x7f57e0133bf0 addr v1:172.21.15.115:6791/0 2024-01-24T23:20:58.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 10 monclient(hunting): picked mon.noname-e con 0x7f57e0133fd0 addr v1:172.21.15.115:6793/0 2024-01-24T23:20:58.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 10 monclient(hunting): picked mon.noname-g con 0x7f57e01343b0 addr v1:172.21.15.132:6790/0 2024-01-24T23:20:58.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e009c880 con 0x7f57e01343b0 2024-01-24T23:20:58.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e01cca70 con 0x7f57e0133bf0 2024-01-24T23:20:58.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e0133720 con 0x7f57e0133fd0 2024-01-24T23:20:58.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 10 monclient(hunting): _renew_subs 2024-01-24T23:20:58.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e5d86640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f57e0133bf0 0x7f57e01cbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:40656/0 (socket says 172.21.15.115:40656) 2024-01-24T23:20:58.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 10 monclient(hunting): authenticate will time out at 937.774571s 2024-01-24T23:20:58.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e5d86640 1 -- 172.21.15.115:0/1251733881 learned_addr learned my addr 172.21.15.115:0/1251733881 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:20:58.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 1 -- 172.21.15.115:0/1251733881 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3730013934 0 0) 0x7f57e01cca70 con 0x7f57e0133bf0 2024-01-24T23:20:58.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:20:58.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:58.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:58.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient(hunting): my global_id is 74155 2024-01-24T23:20:58.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 1 -- 172.21.15.115:0/1251733881 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57c4003530 con 0x7f57e0133bf0 2024-01-24T23:20:58.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 1 -- 172.21.15.115:0/1251733881 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764237624 0 0) 0x7f57e0133720 con 0x7f57e0133fd0 2024-01-24T23:20:58.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:20:58.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:58.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:58.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient(hunting): my global_id is 74123 2024-01-24T23:20:58.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 1 -- 172.21.15.115:0/1251733881 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57e01cca70 con 0x7f57e0133fd0 2024-01-24T23:20:58.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 1 -- 172.21.15.115:0/1251733881 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3121610353 0 0) 0x7f57e009b130 con 0x7f57e01343b0 2024-01-24T23:20:58.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:20:58.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:58.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:58.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient(hunting): my global_id is 4170 2024-01-24T23:20:58.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 1 -- 172.21.15.115:0/1251733881 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57e0133720 con 0x7f57e01343b0 2024-01-24T23:20:58.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 1 -- 172.21.15.115:0/1251733881 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 514152948 0 0) 0x7f57e01cca70 con 0x7f57e0133fd0 2024-01-24T23:20:58.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 1 -- 172.21.15.115:0/1251733881 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f57e009b130 con 0x7f57e0133fd0 2024-01-24T23:20:58.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 1 -- 172.21.15.115:0/1251733881 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f57d0001a20 con 0x7f57e0133fd0 2024-01-24T23:20:58.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:58.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:20:58.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient(hunting): dump: 2024-01-24T23:20:58.866 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:20:58.866 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:58.866 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.866 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.866 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:58.866 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:20:58.866 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:58.866 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:58.866 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:58.866 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:58.866 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:58.866 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:58.866 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:58.866 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:58.866 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:58.867 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:20:58.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:20:58.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 1 -- 172.21.15.115:0/1251733881 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2089378766 0 0) 0x7f57c4003530 con 0x7f57e0133bf0 2024-01-24T23:20:58.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 1 -- 172.21.15.115:0/1251733881 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f57e01cca70 con 0x7f57e0133bf0 2024-01-24T23:20:58.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 1 -- 172.21.15.115:0/1251733881 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f57dc001820 con 0x7f57e0133bf0 2024-01-24T23:20:58.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:58.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:20:58.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient(hunting): dump: 2024-01-24T23:20:58.867 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:20:58.867 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:58.867 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.867 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.867 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:58.867 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:20:58.867 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:58.867 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:58.868 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:58.868 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:58.868 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:58.868 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:58.868 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:58.868 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:58.868 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:58.868 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:20:58.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 1 -- 172.21.15.115:0/1251733881 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 299748678 0 0) 0x7f57e009b130 con 0x7f57e0133fd0 2024-01-24T23:20:58.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 1 -- 172.21.15.115:0/1251733881 >> v1:172.21.15.115:6791/0 conn(0x7f57e0133bf0 legacy=0x7f57e01cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 1 -- 172.21.15.115:0/1251733881 >> v1:172.21.15.132:6790/0 conn(0x7f57e01343b0 legacy=0x7f57e01d3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient: _finish_hunting 0 2024-01-24T23:20:58.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 1 monclient: found mon.e 2024-01-24T23:20:58.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:20:58.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:20:58.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 1 -- 172.21.15.115:0/1251733881 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e00a5990 con 0x7f57e0133fd0 2024-01-24T23:20:58.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient: _finish_auth 0 2024-01-24T23:20:58.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient: _check_auth_tickets 2024-01-24T23:20:58.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:20:58.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 5 monclient: authenticate success, global_id 74123 2024-01-24T23:20:58.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 10 monclient: _renew_subs 2024-01-24T23:20:58.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:20:58.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 1 -- 172.21.15.115:0/1251733881 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e00a3300 con 0x7f57e0133fd0 2024-01-24T23:20:58.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 10 monclient: _renew_subs 2024-01-24T23:20:58.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:20:58.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57e7fea640 1 -- 172.21.15.115:0/1251733881 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e00a2700 con 0x7f57e0133fd0 2024-01-24T23:20:58.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 1 -- 172.21.15.115:0/1251733881 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f57d0001d50 con 0x7f57e0133fd0 2024-01-24T23:20:58.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:20:58.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 1 -- 172.21.15.115:0/1251733881 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f57d0003c10 con 0x7f57e0133fd0 2024-01-24T23:20:58.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:58.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d7fff640 4 set_mon_vals no callback set 2024-01-24T23:20:58.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:20:58.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.847+0000 7f57d67fc640 10 monclient: dump: 2024-01-24T23:20:58.870 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:20:58.870 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:58.870 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.870 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:58.870 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:58.870 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:20:58.870 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:58.870 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:58.870 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:58.870 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:58.870 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:58.871 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:58.871 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:58.871 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:58.871 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:58.871 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:20:58.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.851+0000 7f57d67fc640 1 -- 172.21.15.115:0/1251733881 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 1) v1 ==== 799+0+0 (unknown 314974831 0 0) 0x7f57d00040f0 con 0x7f57e0133fd0 2024-01-24T23:20:58.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.851+0000 7f57d67fc640 1 -- 172.21.15.115:0/1251733881 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (unknown 636836254 0 0) 0x7f57d0005aa0 con 0x7f57e0133fd0 2024-01-24T23:20:58.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.851+0000 7f57e7fea640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:58.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.851+0000 7f57e7fea640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:58.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.851+0000 7f57e7fea640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:20:58.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.851+0000 7f57e7fea640 1 -- 172.21.15.115:0/1251733881 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c8005090 con 0x7f57e0133fd0 2024-01-24T23:20:58.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.851+0000 7f57d67fc640 1 -- 172.21.15.115:0/1251733881 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (unknown 1092875540 0 4287968521) 0x7f57d00054b0 con 0x7f57e0133fd0 2024-01-24T23:20:58.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.851+0000 7f57d67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:58.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.851+0000 7f57d67fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:20:58.895 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:20:58.893+0000 7f278372c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-24T23:20:58.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.911+0000 7f57e7fea640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-01-24T23:20:58.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.911+0000 7f57e7fea640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-01-24T23:20:58.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.911+0000 7f57e7fea640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:20:58.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.911+0000 7f57e7fea640 1 -- 172.21.15.115:0/1251733881 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57c8005340 con 0x7f57e0133fd0 2024-01-24T23:20:58.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.911+0000 7f57d67fc640 1 -- 172.21.15.115:0/1251733881 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) v1 ==== 74+0+5605 (unknown 150273500 0 2292533051) 0x7f57d00057e0 con 0x7f57e0133fd0 2024-01-24T23:20:58.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.911+0000 7f57d67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-01-24T23:20:58.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.911+0000 7f57d67fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:20:58.917 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:20:58.917 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":8,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","created":"2024-01-24T23:20:56.917703+0000","modified":"2024-01-24T23:20:58.690771+0000","last_up_change":"0.000000","last_in_change":"2024-01-24T23:20:58.690771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"902f4a51-85e9-4af8-b11f-e42a51b31723","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e06b92ed-3ab9-4103-8e41-4fa60aea2829","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"013044c9-553c-4dd9-8438-c9873813e5af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5fcff598-1939-415b-b055-57940893269e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ef67ccd9-a21c-4a40-99ff-93f5b10a1fd3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c2573ce8-6dd2-4272-85c7-2f5aa25eb2f9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T23:20:58.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.915+0000 7f57e7fea640 10 monclient: shutdown 2024-01-24T23:20:58.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.915+0000 7f57e7fea640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:20:58.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.915+0000 7f57e7fea640 1 -- 172.21.15.115:0/1251733881 >> v1:172.21.15.115:6793/0 conn(0x7f57e0133fd0 legacy=0x7f57e01cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:58.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.915+0000 7f57e7fea640 1 -- 172.21.15.115:0/1251733881 shutdown_connections 2024-01-24T23:20:58.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.915+0000 7f57e7fea640 1 -- 172.21.15.115:0/1251733881 >> 172.21.15.115:0/1251733881 conn(0x7f57e00662f0 msgr2=0x7f57e01be890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:20:58.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.915+0000 7f57e7fea640 1 -- 172.21.15.115:0/1251733881 shutdown_connections 2024-01-24T23:20:58.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:20:58.915+0000 7f57e7fea640 1 -- 172.21.15.115:0/1251733881 wait complete. 2024-01-24T23:20:58.933 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-01-24T23:20:58.933 INFO:tasks.ceph:Waiting for OSDs to come up 2024-01-24T23:20:58.947 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:20:58.945+0000 7f278372c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-24T23:20:59.027 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:20:59.025+0000 7f278372c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-24T23:20:59.078 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:20:59.077+0000 7f278372c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-24T23:20:59.216 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:20:59.213+0000 7f278372c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-24T23:20:59.275 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:20:59.273+0000 7f278372c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-24T23:20:59.327 INFO:tasks.ceph.osd.3.smithi132.stderr:2024-01-24T23:20:59.325+0000 7fed3f314640 -1 Falling back to public interface 2024-01-24T23:20:59.328 INFO:tasks.ceph.osd.5.smithi132.stderr:2024-01-24T23:20:59.325+0000 7f51da7e3640 -1 Falling back to public interface 2024-01-24T23:20:59.328 INFO:tasks.ceph.osd.4.smithi132.stderr:2024-01-24T23:20:59.325+0000 7f7144909640 -1 Falling back to public interface 2024-01-24T23:20:59.331 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-01-24T23:20:59.327+0000 7f1b56eb6640 -1 Falling back to public interface 2024-01-24T23:20:59.331 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-01-24T23:20:59.327+0000 7f42ed6ce640 -1 Falling back to public interface 2024-01-24T23:20:59.331 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-01-24T23:20:59.327+0000 7f1b55e76640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6805/0: (98) Address already in use 2024-01-24T23:20:59.350 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-01-24T23:20:59.347+0000 7fa653d10640 -1 Falling back to public interface 2024-01-24T23:20:59.538 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-24T23:20:59.656 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.651+0000 7f9ad3c53640 10 monclient: get_monmap_and_config 2024-01-24T23:20:59.657 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.651+0000 7f9ad3c53640 10 monclient: build_initial_monmap 2024-01-24T23:20:59.657 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.651+0000 7f9ad3c53640 10 monclient: monmap: 2024-01-24T23:20:59.657 INFO:teuthology.misc.health.smithi115.stderr:epoch 0 2024-01-24T23:20:59.657 INFO:teuthology.misc.health.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:59.657 INFO:teuthology.misc.health.smithi115.stderr:last_changed 2024-01-24T23:20:59.658024+0000 2024-01-24T23:20:59.657 INFO:teuthology.misc.health.smithi115.stderr:created 2024-01-24T23:20:59.658024+0000 2024-01-24T23:20:59.657 INFO:teuthology.misc.health.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:20:59.657 INFO:teuthology.misc.health.smithi115.stderr:election_strategy: 3 2024-01-24T23:20:59.657 INFO:teuthology.misc.health.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:20:59.657 INFO:teuthology.misc.health.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:20:59.657 INFO:teuthology.misc.health.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:20:59.657 INFO:teuthology.misc.health.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:20:59.657 INFO:teuthology.misc.health.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:20:59.657 INFO:teuthology.misc.health.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:20:59.658 INFO:teuthology.misc.health.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:20:59.658 INFO:teuthology.misc.health.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:20:59.658 INFO:teuthology.misc.health.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:20:59.658 INFO:teuthology.misc.health.smithi115.stderr: 2024-01-24T23:20:59.665 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.651+0000 7f9ad3c53640 1 Processor -- start 2024-01-24T23:20:59.665 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.651+0000 7f9ad3c53640 1 -- start start 2024-01-24T23:20:59.665 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.651+0000 7f9ad3c53640 10 monclient: init 2024-01-24T23:20:59.665 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 10 monclient: _reopen_session rank -1 2024-01-24T23:20:59.665 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 10 monclient: _add_conns ranks=[8,0,1,5,7,2,3,6,4] 2024-01-24T23:20:59.666 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 10 monclient(hunting): picked mon.noname-e con 0x7f9acc0a2c40 addr v1:172.21.15.115:6793/0 2024-01-24T23:20:59.666 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 10 monclient(hunting): picked mon.noname-a con 0x7f9acc137000 addr v1:172.21.15.115:6789/0 2024-01-24T23:20:59.666 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 10 monclient(hunting): picked mon.noname-f con 0x7f9acc0a3020 addr v1:172.21.15.132:6789/0 2024-01-24T23:20:59.666 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9acc04edc0 con 0x7f9acc137000 2024-01-24T23:20:59.666 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9acc0a53b0 con 0x7f9acc0a3020 2024-01-24T23:20:59.666 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9acc137460 con 0x7f9acc0a2c40 2024-01-24T23:20:59.666 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 10 monclient(hunting): _renew_subs 2024-01-24T23:20:59.666 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 10 monclient(hunting): authenticate will time out at 938.580641s 2024-01-24T23:20:59.666 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad19ef640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f9acc0a2c40 0x7f9acc0a3a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:34638/0 (socket says 172.21.15.115:34638) 2024-01-24T23:20:59.666 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad11ee640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f9acc137000 0x7f9acc0a41a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:55274/0 (socket says 172.21.15.115:55274) 2024-01-24T23:20:59.666 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad19ef640 1 -- 172.21.15.115:0/1488599536 learned_addr learned my addr 172.21.15.115:0/1488599536 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:20:59.666 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 1 -- 172.21.15.115:0/1488599536 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2688545478 0 0) 0x7f9acc137460 con 0x7f9acc0a2c40 2024-01-24T23:20:59.667 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 10 monclient(hunting): _negotiate 2024-01-24T23:20:59.667 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:59.667 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:59.667 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 10 monclient(hunting): my global_id is 74132 2024-01-24T23:20:59.667 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 1 -- 172.21.15.115:0/1488599536 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ab8003530 con 0x7f9acc0a2c40 2024-01-24T23:20:59.667 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 1 -- 172.21.15.115:0/1488599536 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1970306527 0 0) 0x7f9acc04edc0 con 0x7f9acc137000 2024-01-24T23:20:59.667 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 10 monclient(hunting): _negotiate 2024-01-24T23:20:59.667 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:59.667 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:59.667 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 10 monclient(hunting): my global_id is 4194 2024-01-24T23:20:59.667 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 1 -- 172.21.15.115:0/1488599536 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9acc137460 con 0x7f9acc137000 2024-01-24T23:20:59.667 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 1 -- 172.21.15.115:0/1488599536 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1971548329 0 0) 0x7f9ac40014a0 con 0x7f9acc0a3020 2024-01-24T23:20:59.667 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 10 monclient(hunting): _negotiate 2024-01-24T23:20:59.667 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:59.668 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:59.668 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 10 monclient(hunting): my global_id is 4150 2024-01-24T23:20:59.668 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 1 -- 172.21.15.115:0/1488599536 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9acc04edc0 con 0x7f9acc0a3020 2024-01-24T23:20:59.668 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 1 -- 172.21.15.115:0/1488599536 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 828391044 0 0) 0x7f9ab8003530 con 0x7f9acc0a2c40 2024-01-24T23:20:59.668 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 1 -- 172.21.15.115:0/1488599536 >> v1:172.21.15.132:6789/0 conn(0x7f9acc0a3020 legacy=0x7f9acc13feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:59.668 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 1 -- 172.21.15.115:0/1488599536 >> v1:172.21.15.115:6789/0 conn(0x7f9acc137000 legacy=0x7f9acc0a41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:59.668 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 10 monclient: _finish_hunting 0 2024-01-24T23:20:59.668 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 1 monclient: found mon.noname-e 2024-01-24T23:20:59.668 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:20:59.668 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:20:59.668 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 1 -- 172.21.15.115:0/1488599536 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc0a48b0 con 0x7f9acc0a2c40 2024-01-24T23:20:59.668 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 10 monclient: _finish_auth 0 2024-01-24T23:20:59.668 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 10 monclient: _check_auth_tickets 2024-01-24T23:20:59.668 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:20:59.668 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 5 monclient: authenticate success, global_id 74132 2024-01-24T23:20:59.669 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:20:59.669 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 1 -- 172.21.15.115:0/1488599536 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9ac80017c0 con 0x7f9acc0a2c40 2024-01-24T23:20:59.669 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:59.669 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:20:59.669 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 10 monclient: dump: 2024-01-24T23:20:59.669 INFO:teuthology.misc.health.smithi115.stderr:epoch 1 2024-01-24T23:20:59.669 INFO:teuthology.misc.health.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:59.669 INFO:teuthology.misc.health.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:59.669 INFO:teuthology.misc.health.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:59.669 INFO:teuthology.misc.health.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:59.669 INFO:teuthology.misc.health.smithi115.stderr:election_strategy: 3 2024-01-24T23:20:59.669 INFO:teuthology.misc.health.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:59.669 INFO:teuthology.misc.health.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:59.669 INFO:teuthology.misc.health.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:59.670 INFO:teuthology.misc.health.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:59.670 INFO:teuthology.misc.health.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:59.670 INFO:teuthology.misc.health.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:59.670 INFO:teuthology.misc.health.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:59.670 INFO:teuthology.misc.health.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:59.670 INFO:teuthology.misc.health.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:59.670 INFO:teuthology.misc.health.smithi115.stderr: 2024-01-24T23:20:59.670 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:20:59.670 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 1 -- 172.21.15.115:0/1488599536 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9ac8001aa0 con 0x7f9acc0a2c40 2024-01-24T23:20:59.670 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:20:59.670 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 1 -- 172.21.15.115:0/1488599536 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9ac8004bf0 con 0x7f9acc0a2c40 2024-01-24T23:20:59.670 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:59.670 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:20:59.670 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad09ed640 10 monclient: dump: 2024-01-24T23:20:59.670 INFO:teuthology.misc.health.smithi115.stderr:epoch 1 2024-01-24T23:20:59.671 INFO:teuthology.misc.health.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:59.671 INFO:teuthology.misc.health.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:59.671 INFO:teuthology.misc.health.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:59.671 INFO:teuthology.misc.health.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:59.671 INFO:teuthology.misc.health.smithi115.stderr:election_strategy: 3 2024-01-24T23:20:59.671 INFO:teuthology.misc.health.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:59.671 INFO:teuthology.misc.health.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:59.671 INFO:teuthology.misc.health.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:59.671 INFO:teuthology.misc.health.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:59.671 INFO:teuthology.misc.health.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:59.671 INFO:teuthology.misc.health.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:59.671 INFO:teuthology.misc.health.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:59.671 INFO:teuthology.misc.health.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:59.671 INFO:teuthology.misc.health.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:59.671 INFO:teuthology.misc.health.smithi115.stderr: 2024-01-24T23:20:59.672 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 10 monclient: get_monmap_and_config success 2024-01-24T23:20:59.672 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 4 set_mon_vals no callback set 2024-01-24T23:20:59.672 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 10 monclient: shutdown 2024-01-24T23:20:59.672 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:20:59.672 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 1 -- 172.21.15.115:0/1488599536 >> v1:172.21.15.115:6793/0 conn(0x7f9acc0a2c40 legacy=0x7f9acc0a3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:59.672 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 1 -- 172.21.15.115:0/1488599536 shutdown_connections 2024-01-24T23:20:59.672 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 1 -- 172.21.15.115:0/1488599536 >> 172.21.15.115:0/1488599536 conn(0x7f9acc066810 msgr2=0x7f9acc066c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:20:59.672 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 1 -- 172.21.15.115:0/1488599536 shutdown_connections 2024-01-24T23:20:59.672 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 1 -- 172.21.15.115:0/1488599536 wait complete. 2024-01-24T23:20:59.672 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 10 monclient: build_initial_monmap 2024-01-24T23:20:59.672 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 10 monclient: monmap: 2024-01-24T23:20:59.672 INFO:teuthology.misc.health.smithi115.stderr:epoch 0 2024-01-24T23:20:59.672 INFO:teuthology.misc.health.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:20:59.673 INFO:teuthology.misc.health.smithi115.stderr:last_changed 0.000000 2024-01-24T23:20:59.673 INFO:teuthology.misc.health.smithi115.stderr:created 0.000000 2024-01-24T23:20:59.673 INFO:teuthology.misc.health.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:20:59.673 INFO:teuthology.misc.health.smithi115.stderr:election_strategy: 1 2024-01-24T23:20:59.673 INFO:teuthology.misc.health.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:20:59.673 INFO:teuthology.misc.health.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:20:59.673 INFO:teuthology.misc.health.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:20:59.673 INFO:teuthology.misc.health.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:20:59.673 INFO:teuthology.misc.health.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:20:59.673 INFO:teuthology.misc.health.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:20:59.673 INFO:teuthology.misc.health.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:20:59.673 INFO:teuthology.misc.health.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:20:59.673 INFO:teuthology.misc.health.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:20:59.673 INFO:teuthology.misc.health.smithi115.stderr: 2024-01-24T23:20:59.674 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 1 Processor -- start 2024-01-24T23:20:59.674 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 1 -- start start 2024-01-24T23:20:59.674 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.655+0000 7f9ad3c53640 10 monclient: init 2024-01-24T23:20:59.674 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9ad3c53640 10 monclient: _reopen_session rank -1 2024-01-24T23:20:59.674 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9ad3c53640 10 monclient: _add_conns ranks=[4,1,2,7,3,8,0,5,6] 2024-01-24T23:20:59.674 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9ad3c53640 10 monclient(hunting): picked mon.noname-c con 0x7f9acc0a2c40 addr v1:172.21.15.115:6791/0 2024-01-24T23:20:59.674 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9ad3c53640 10 monclient(hunting): picked mon.noname-f con 0x7f9acc0a3020 addr v1:172.21.15.132:6789/0 2024-01-24T23:20:59.674 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9ad3c53640 10 monclient(hunting): picked mon.noname-b con 0x7f9acc137000 addr v1:172.21.15.115:6790/0 2024-01-24T23:20:59.674 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9ad3c53640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9acc121540 con 0x7f9acc0a3020 2024-01-24T23:20:59.674 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9ad3c53640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9acc09fc30 con 0x7f9acc137000 2024-01-24T23:20:59.674 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9ad3c53640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9acc0a5630 con 0x7f9acc0a2c40 2024-01-24T23:20:59.674 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9ad19ef640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f9acc0a2c40 0x7f9acc1354e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:40670/0 (socket says 172.21.15.115:40670) 2024-01-24T23:20:59.674 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9ad19ef640 1 -- 172.21.15.115:0/945284634 learned_addr learned my addr 172.21.15.115:0/945284634 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:20:59.674 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9ad3c53640 10 monclient(hunting): _renew_subs 2024-01-24T23:20:59.675 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9ad3c53640 10 monclient(hunting): authenticate will time out at 938.584323s 2024-01-24T23:20:59.675 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3774540338 0 0) 0x7f9acc09fc30 con 0x7f9acc137000 2024-01-24T23:20:59.675 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:20:59.675 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:59.675 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:59.675 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient(hunting): my global_id is 4151 2024-01-24T23:20:59.675 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ab0003530 con 0x7f9acc137000 2024-01-24T23:20:59.675 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3521645237 0 0) 0x7f9acc0a5630 con 0x7f9acc0a2c40 2024-01-24T23:20:59.675 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:20:59.675 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:59.675 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:59.675 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient(hunting): my global_id is 74164 2024-01-24T23:20:59.675 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9acc09fc30 con 0x7f9acc0a2c40 2024-01-24T23:20:59.675 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2771750457 0 0) 0x7f9acc121540 con 0x7f9acc0a3020 2024-01-24T23:20:59.675 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:20:59.676 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:20:59.676 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:20:59.676 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient(hunting): my global_id is 4159 2024-01-24T23:20:59.676 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9acc0a5630 con 0x7f9acc0a3020 2024-01-24T23:20:59.676 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 119742665 0 0) 0x7f9ab0003530 con 0x7f9acc137000 2024-01-24T23:20:59.676 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9acc121540 con 0x7f9acc137000 2024-01-24T23:20:59.676 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9ac4001f70 con 0x7f9acc137000 2024-01-24T23:20:59.676 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:59.676 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:20:59.676 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient(hunting): dump: 2024-01-24T23:20:59.676 INFO:teuthology.misc.health.smithi115.stderr:epoch 1 2024-01-24T23:20:59.676 INFO:teuthology.misc.health.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:59.676 INFO:teuthology.misc.health.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:59.676 INFO:teuthology.misc.health.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:59.677 INFO:teuthology.misc.health.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:59.677 INFO:teuthology.misc.health.smithi115.stderr:election_strategy: 3 2024-01-24T23:20:59.677 INFO:teuthology.misc.health.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:59.677 INFO:teuthology.misc.health.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:59.677 INFO:teuthology.misc.health.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:59.677 INFO:teuthology.misc.health.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:59.677 INFO:teuthology.misc.health.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:59.677 INFO:teuthology.misc.health.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:59.677 INFO:teuthology.misc.health.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:59.677 INFO:teuthology.misc.health.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:59.677 INFO:teuthology.misc.health.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:59.677 INFO:teuthology.misc.health.smithi115.stderr: 2024-01-24T23:20:59.677 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:20:59.677 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1839595895 0 0) 0x7f9acc0a5630 con 0x7f9acc0a3020 2024-01-24T23:20:59.677 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9ab0003530 con 0x7f9acc0a3020 2024-01-24T23:20:59.678 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9ac00018a0 con 0x7f9acc0a3020 2024-01-24T23:20:59.678 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:59.678 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:20:59.678 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient(hunting): dump: 2024-01-24T23:20:59.678 INFO:teuthology.misc.health.smithi115.stderr:epoch 1 2024-01-24T23:20:59.678 INFO:teuthology.misc.health.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:59.678 INFO:teuthology.misc.health.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:59.678 INFO:teuthology.misc.health.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:59.678 INFO:teuthology.misc.health.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:59.678 INFO:teuthology.misc.health.smithi115.stderr:election_strategy: 3 2024-01-24T23:20:59.678 INFO:teuthology.misc.health.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:59.678 INFO:teuthology.misc.health.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:59.678 INFO:teuthology.misc.health.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:59.678 INFO:teuthology.misc.health.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:59.679 INFO:teuthology.misc.health.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:59.679 INFO:teuthology.misc.health.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:59.679 INFO:teuthology.misc.health.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:59.679 INFO:teuthology.misc.health.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:59.679 INFO:teuthology.misc.health.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:59.679 INFO:teuthology.misc.health.smithi115.stderr: 2024-01-24T23:20:59.679 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 544771476 0 0) 0x7f9acc09fc30 con 0x7f9acc0a2c40 2024-01-24T23:20:59.679 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9acc0a5630 con 0x7f9acc0a2c40 2024-01-24T23:20:59.679 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9ac8004b30 con 0x7f9acc0a2c40 2024-01-24T23:20:59.679 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:59.679 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:20:59.679 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient(hunting): dump: 2024-01-24T23:20:59.679 INFO:teuthology.misc.health.smithi115.stderr:epoch 1 2024-01-24T23:20:59.679 INFO:teuthology.misc.health.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:59.679 INFO:teuthology.misc.health.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:59.680 INFO:teuthology.misc.health.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:59.680 INFO:teuthology.misc.health.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:59.680 INFO:teuthology.misc.health.smithi115.stderr:election_strategy: 3 2024-01-24T23:20:59.680 INFO:teuthology.misc.health.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:59.680 INFO:teuthology.misc.health.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:59.680 INFO:teuthology.misc.health.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:59.680 INFO:teuthology.misc.health.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:59.680 INFO:teuthology.misc.health.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:59.680 INFO:teuthology.misc.health.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:59.680 INFO:teuthology.misc.health.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:59.680 INFO:teuthology.misc.health.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:59.680 INFO:teuthology.misc.health.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:59.680 INFO:teuthology.misc.health.smithi115.stderr: 2024-01-24T23:20:59.680 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3749661167 0 0) 0x7f9acc121540 con 0x7f9acc137000 2024-01-24T23:20:59.681 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 >> v1:172.21.15.115:6791/0 conn(0x7f9acc0a2c40 legacy=0x7f9acc1354e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:59.681 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 >> v1:172.21.15.132:6789/0 conn(0x7f9acc0a3020 legacy=0x7f9acc1d0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:59.681 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:20:59.681 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 1 monclient: found mon.b 2024-01-24T23:20:59.681 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:20:59.681 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:20:59.681 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc121710 con 0x7f9acc137000 2024-01-24T23:20:59.681 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient: _finish_auth 0 2024-01-24T23:20:59.681 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:20:59.681 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:20:59.681 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9ad3c53640 5 monclient: authenticate success, global_id 4151 2024-01-24T23:20:59.681 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9ad3c53640 10 monclient: _renew_subs 2024-01-24T23:20:59.681 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9ad3c53640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:20:59.681 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9ad3c53640 1 -- 172.21.15.115:0/945284634 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9acc120bb0 con 0x7f9acc137000 2024-01-24T23:20:59.681 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9ac40014f0 con 0x7f9acc137000 2024-01-24T23:20:59.682 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:20:59.682 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9ad3c53640 10 monclient: _renew_subs 2024-01-24T23:20:59.682 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9ac4003f90 con 0x7f9acc137000 2024-01-24T23:20:59.682 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9ad3c53640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:20:59.682 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9ad3c53640 1 -- 172.21.15.115:0/945284634 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9acc1cb750 con 0x7f9acc137000 2024-01-24T23:20:59.682 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:20:59.682 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:20:59.682 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abffff640 4 set_mon_vals no callback set 2024-01-24T23:20:59.682 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 10 monclient: dump: 2024-01-24T23:20:59.682 INFO:teuthology.misc.health.smithi115.stderr:epoch 1 2024-01-24T23:20:59.682 INFO:teuthology.misc.health.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:20:59.682 INFO:teuthology.misc.health.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:59.682 INFO:teuthology.misc.health.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:20:59.682 INFO:teuthology.misc.health.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:20:59.683 INFO:teuthology.misc.health.smithi115.stderr:election_strategy: 3 2024-01-24T23:20:59.683 INFO:teuthology.misc.health.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:20:59.683 INFO:teuthology.misc.health.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:20:59.683 INFO:teuthology.misc.health.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:20:59.683 INFO:teuthology.misc.health.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:20:59.683 INFO:teuthology.misc.health.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:20:59.683 INFO:teuthology.misc.health.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:20:59.683 INFO:teuthology.misc.health.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:20:59.683 INFO:teuthology.misc.health.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:20:59.683 INFO:teuthology.misc.health.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:20:59.683 INFO:teuthology.misc.health.smithi115.stderr: 2024-01-24T23:20:59.683 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 1) v1 ==== 799+0+0 (unknown 314974831 0 0) 0x7f9ac4004470 con 0x7f9acc137000 2024-01-24T23:20:59.683 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (unknown 636836254 0 0) 0x7f9ac4004d10 con 0x7f9acc137000 2024-01-24T23:20:59.683 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9ad3c53640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:59.683 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9ad3c53640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:59.684 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9ad3c53640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:20:59.684 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.659+0000 7f9ad3c53640 1 -- 172.21.15.115:0/945284634 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9acc067c60 con 0x7f9acc137000 2024-01-24T23:20:59.684 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.663+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 <== mon.2 v1:172.21.15.115:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (unknown 1092875540 0 4287968521) 0x7f9acc0a53b0 con 0x7f9acc137000 2024-01-24T23:20:59.684 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.663+0000 7f9abe7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:20:59.684 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.663+0000 7f9abe7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:20:59.723 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.719+0000 7f9ad3c53640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-01-24T23:20:59.723 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.719+0000 7f9ad3c53640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-01-24T23:20:59.723 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.719+0000 7f9ad3c53640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:20:59.723 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.719+0000 7f9ad3c53640 1 -- 172.21.15.115:0/945284634 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9acc0621c0 con 0x7f9acc137000 2024-01-24T23:20:59.724 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.719+0000 7f9abe7fc640 1 -- 172.21.15.115:0/945284634 <== mon.2 v1:172.21.15.115:6790/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) v1 ==== 74+0+5605 (unknown 150273500 0 2292533051) 0x7f9ac4004fe0 con 0x7f9acc137000 2024-01-24T23:20:59.724 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.719+0000 7f9abe7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-01-24T23:20:59.724 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.719+0000 7f9abe7fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:20:59.724 INFO:teuthology.misc.health.smithi115.stdout: 2024-01-24T23:20:59.724 INFO:teuthology.misc.health.smithi115.stdout:{"epoch":8,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","created":"2024-01-24T23:20:56.917703+0000","modified":"2024-01-24T23:20:58.690771+0000","last_up_change":"0.000000","last_in_change":"2024-01-24T23:20:58.690771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"902f4a51-85e9-4af8-b11f-e42a51b31723","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e06b92ed-3ab9-4103-8e41-4fa60aea2829","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"013044c9-553c-4dd9-8438-c9873813e5af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5fcff598-1939-415b-b055-57940893269e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ef67ccd9-a21c-4a40-99ff-93f5b10a1fd3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c2573ce8-6dd2-4272-85c7-2f5aa25eb2f9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T23:20:59.725 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.719+0000 7f9ad3c53640 10 monclient: shutdown 2024-01-24T23:20:59.725 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.719+0000 7f9ad3c53640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:20:59.725 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.719+0000 7f9ad3c53640 1 -- 172.21.15.115:0/945284634 >> v1:172.21.15.115:6790/0 conn(0x7f9acc137000 legacy=0x7f9acc1d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:20:59.726 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.719+0000 7f9ad3c53640 1 -- 172.21.15.115:0/945284634 shutdown_connections 2024-01-24T23:20:59.726 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.719+0000 7f9ad3c53640 1 -- 172.21.15.115:0/945284634 >> 172.21.15.115:0/945284634 conn(0x7f9acc066810 msgr2=0x7f9acc0a1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:20:59.726 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.719+0000 7f9ad3c53640 1 -- 172.21.15.115:0/945284634 shutdown_connections 2024-01-24T23:20:59.726 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:20:59.723+0000 7f9ad3c53640 1 -- 172.21.15.115:0/945284634 wait complete. 2024-01-24T23:20:59.740 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-01-24T23:20:59.860 INFO:tasks.ceph.mgr.x.smithi132.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-01-24T23:20:59.861 INFO:tasks.ceph.mgr.x.smithi132.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-01-24T23:20:59.861 INFO:tasks.ceph.mgr.x.smithi132.stderr: from numpy import show_config as show_numpy_config 2024-01-24T23:20:59.958 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:20:59.957+0000 7f278372c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-24T23:21:00.130 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:21:00.125+0000 7f278372c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-24T23:21:00.308 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:21:00.305+0000 7f278372c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-24T23:21:00.376 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:21:00.373+0000 7f278372c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-24T23:21:00.511 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:21:00.509+0000 7f278372c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-24T23:21:01.137 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:21:01.133+0000 7f278372c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-24T23:21:01.211 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:21:01.209+0000 7f278372c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-24T23:21:01.379 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:21:01.377+0000 7f278372c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-24T23:21:01.452 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:21:01.449+0000 7f278372c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-24T23:21:01.734 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:21:01.733+0000 7f278372c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-24T23:21:01.793 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:21:01.789+0000 7f278372c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-24T23:21:02.836 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:21:02.833+0000 7f272af79640 -1 mgr.server handle_report got status from non-daemon mon.f 2024-01-24T23:21:02.836 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:21:02.833+0000 7f272af79640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-01-24T23:21:02.837 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:21:02.833+0000 7f272af79640 -1 mgr.server handle_report got status from non-daemon mon.h 2024-01-24T23:21:02.837 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:21:02.833+0000 7f272af79640 -1 mgr.server handle_report got status from non-daemon mon.g 2024-01-24T23:21:02.838 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:21:02.837+0000 7f272af79640 -1 mgr.server handle_report got status from non-daemon mon.d 2024-01-24T23:21:02.839 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:21:02.837+0000 7f272af79640 -1 mgr.server handle_report got status from non-daemon mon.i 2024-01-24T23:21:02.840 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:21:02.837+0000 7f272af79640 -1 mgr.server handle_report got status from non-daemon mon.e 2024-01-24T23:21:02.841 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-01-24T23:21:02.837+0000 7f272af79640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-01-24T23:21:02.938 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-01-24T23:21:02.935+0000 7f42ed6ce640 -1 osd.2 0 log_to_monitors true 2024-01-24T23:21:03.038 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-01-24T23:21:03.035+0000 7f1b56eb6640 -1 osd.1 0 log_to_monitors true 2024-01-24T23:21:03.075 INFO:tasks.ceph.osd.5.smithi132.stderr:2024-01-24T23:21:03.073+0000 7f51da7e3640 -1 osd.5 0 log_to_monitors true 2024-01-24T23:21:03.176 INFO:tasks.ceph.osd.3.smithi132.stderr:2024-01-24T23:21:03.173+0000 7fed3f314640 -1 osd.3 0 log_to_monitors true 2024-01-24T23:21:03.177 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-01-24T23:21:03.171+0000 7fa653d10640 -1 osd.0 0 log_to_monitors true 2024-01-24T23:21:03.219 INFO:tasks.ceph.osd.4.smithi132.stderr:2024-01-24T23:21:03.217+0000 7f7144909640 -1 osd.4 0 log_to_monitors true 2024-01-24T23:21:04.854 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-01-24T23:21:04.851+0000 7f1b52e5f640 -1 osd.1 0 waiting for initial osdmap 2024-01-24T23:21:04.854 INFO:tasks.ceph.osd.4.smithi132.stderr:2024-01-24T23:21:04.849+0000 7f71408b2640 -1 osd.4 0 waiting for initial osdmap 2024-01-24T23:21:04.856 INFO:tasks.ceph.osd.3.smithi132.stderr:2024-01-24T23:21:04.853+0000 7fed3b2bd640 -1 osd.3 0 waiting for initial osdmap 2024-01-24T23:21:04.859 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-01-24T23:21:04.855+0000 7f1b4eca2640 -1 osd.1 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-24T23:21:04.863 INFO:tasks.ceph.osd.4.smithi132.stderr:2024-01-24T23:21:04.861+0000 7f713c6f5640 -1 osd.4 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-24T23:21:04.864 INFO:tasks.ceph.osd.3.smithi132.stderr:2024-01-24T23:21:04.861+0000 7fed37100640 -1 osd.3 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-24T23:21:04.866 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-01-24T23:21:04.863+0000 7fa64fcb9640 -1 osd.0 0 waiting for initial osdmap 2024-01-24T23:21:04.866 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-01-24T23:21:04.863+0000 7f42e9677640 -1 osd.2 0 waiting for initial osdmap 2024-01-24T23:21:04.870 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-01-24T23:21:04.867+0000 7fa64bafc640 -1 osd.0 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-24T23:21:04.871 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-01-24T23:21:04.867+0000 7f42e54ba640 -1 osd.2 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-24T23:21:05.541 INFO:tasks.ceph.osd.5.smithi132.stderr:2024-01-24T23:21:05.537+0000 7f51d678c640 -1 osd.5 0 waiting for initial osdmap 2024-01-24T23:21:05.549 INFO:tasks.ceph.osd.5.smithi132.stderr:2024-01-24T23:21:05.545+0000 7f51ce5c7640 -1 osd.5 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-24T23:21:06.346 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-24T23:21:06.465 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.459+0000 7ff6a8399640 10 monclient: get_monmap_and_config 2024-01-24T23:21:06.465 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.459+0000 7ff6a8399640 10 monclient: build_initial_monmap 2024-01-24T23:21:06.465 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.459+0000 7ff6a8399640 10 monclient: monmap: 2024-01-24T23:21:06.465 INFO:teuthology.misc.health.smithi115.stderr:epoch 0 2024-01-24T23:21:06.465 INFO:teuthology.misc.health.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:06.465 INFO:teuthology.misc.health.smithi115.stderr:last_changed 2024-01-24T23:21:06.466132+0000 2024-01-24T23:21:06.465 INFO:teuthology.misc.health.smithi115.stderr:created 2024-01-24T23:21:06.466132+0000 2024-01-24T23:21:06.465 INFO:teuthology.misc.health.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:06.465 INFO:teuthology.misc.health.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:06.465 INFO:teuthology.misc.health.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:06.465 INFO:teuthology.misc.health.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:06.465 INFO:teuthology.misc.health.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:06.466 INFO:teuthology.misc.health.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:06.466 INFO:teuthology.misc.health.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:06.466 INFO:teuthology.misc.health.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:06.466 INFO:teuthology.misc.health.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:06.466 INFO:teuthology.misc.health.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:06.466 INFO:teuthology.misc.health.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:06.466 INFO:teuthology.misc.health.smithi115.stderr: 2024-01-24T23:21:06.470 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 1 Processor -- start 2024-01-24T23:21:06.470 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 1 -- start start 2024-01-24T23:21:06.470 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 10 monclient: init 2024-01-24T23:21:06.470 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:06.470 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 10 monclient: _add_conns ranks=[8,7,4,0,3,5,6,1,2] 2024-01-24T23:21:06.470 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 10 monclient(hunting): picked mon.noname-e con 0x7ff6a00a3190 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:06.470 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 10 monclient(hunting): picked mon.noname-i con 0x7ff6a00a00c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:06.470 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 10 monclient(hunting): picked mon.noname-c con 0x7ff6a0133d30 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:06.470 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a004edc0 con 0x7ff6a0133d30 2024-01-24T23:21:06.471 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a00a04a0 con 0x7ff6a00a00c0 2024-01-24T23:21:06.471 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a01341c0 con 0x7ff6a00a3190 2024-01-24T23:21:06.471 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:06.471 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 10 monclient(hunting): authenticate will time out at 945.388837s 2024-01-24T23:21:06.471 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a6135640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7ff6a00a3190 0x7ff6a009f9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:48694/0 (socket says 172.21.15.115:48694) 2024-01-24T23:21:06.471 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a6936640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7ff6a0133d30 0x7ff6a013c200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:42396/0 (socket says 172.21.15.115:42396) 2024-01-24T23:21:06.471 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a6135640 1 -- 172.21.15.115:0/1864905114 learned_addr learned my addr 172.21.15.115:0/1864905114 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:06.471 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 1 -- 172.21.15.115:0/1864905114 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 594707279 0 0) 0x7ff6a004edc0 con 0x7ff6a0133d30 2024-01-24T23:21:06.471 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 10 monclient(hunting): _negotiate 2024-01-24T23:21:06.471 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:06.471 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:06.471 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 10 monclient(hunting): my global_id is 74227 2024-01-24T23:21:06.471 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 1 -- 172.21.15.115:0/1864905114 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff68c003530 con 0x7ff6a0133d30 2024-01-24T23:21:06.472 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 1 -- 172.21.15.115:0/1864905114 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318414779 0 0) 0x7ff6a01341c0 con 0x7ff6a00a3190 2024-01-24T23:21:06.472 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 10 monclient(hunting): _negotiate 2024-01-24T23:21:06.472 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:06.472 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:06.472 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 10 monclient(hunting): my global_id is 74159 2024-01-24T23:21:06.472 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 1 -- 172.21.15.115:0/1864905114 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6a004edc0 con 0x7ff6a00a3190 2024-01-24T23:21:06.472 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 1 -- 172.21.15.115:0/1864905114 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1086795165 0 0) 0x7ff6a00a04a0 con 0x7ff6a00a00c0 2024-01-24T23:21:06.472 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 10 monclient(hunting): _negotiate 2024-01-24T23:21:06.472 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:06.472 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:06.472 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 10 monclient(hunting): my global_id is 4219 2024-01-24T23:21:06.472 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 1 -- 172.21.15.115:0/1864905114 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6a01341c0 con 0x7ff6a00a00c0 2024-01-24T23:21:06.472 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 1 -- 172.21.15.115:0/1864905114 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3876665291 0 0) 0x7ff68c003530 con 0x7ff6a0133d30 2024-01-24T23:21:06.472 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 1 -- 172.21.15.115:0/1864905114 >> v1:172.21.15.115:6793/0 conn(0x7ff6a00a3190 legacy=0x7ff6a009f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:06.472 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 1 -- 172.21.15.115:0/1864905114 >> v1:172.21.15.132:6792/0 conn(0x7ff6a00a00c0 legacy=0x7ff6a0138ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:06.473 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 10 monclient: _finish_hunting 0 2024-01-24T23:21:06.473 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 1 monclient: found mon.noname-c 2024-01-24T23:21:06.473 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:06.473 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:21:06.473 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 1 -- 172.21.15.115:0/1864905114 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a00a3570 con 0x7ff6a0133d30 2024-01-24T23:21:06.473 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 10 monclient: _finish_auth 0 2024-01-24T23:21:06.473 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 10 monclient: _check_auth_tickets 2024-01-24T23:21:06.473 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:06.473 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 1 -- 172.21.15.115:0/1864905114 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff69c0017c0 con 0x7ff6a0133d30 2024-01-24T23:21:06.473 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:06.473 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:21:06.473 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 10 monclient: dump: 2024-01-24T23:21:06.473 INFO:teuthology.misc.health.smithi115.stderr:epoch 1 2024-01-24T23:21:06.473 INFO:teuthology.misc.health.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:06.474 INFO:teuthology.misc.health.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:06.474 INFO:teuthology.misc.health.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:06.474 INFO:teuthology.misc.health.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:06.474 INFO:teuthology.misc.health.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:06.474 INFO:teuthology.misc.health.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:06.474 INFO:teuthology.misc.health.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:06.474 INFO:teuthology.misc.health.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:06.474 INFO:teuthology.misc.health.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:06.474 INFO:teuthology.misc.health.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:06.474 INFO:teuthology.misc.health.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:06.474 INFO:teuthology.misc.health.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:06.474 INFO:teuthology.misc.health.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:06.474 INFO:teuthology.misc.health.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:06.474 INFO:teuthology.misc.health.smithi115.stderr: 2024-01-24T23:21:06.474 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 5 monclient: authenticate success, global_id 74227 2024-01-24T23:21:06.474 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:06.474 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 1 -- 172.21.15.115:0/1864905114 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff69c0019e0 con 0x7ff6a0133d30 2024-01-24T23:21:06.475 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:06.475 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:06.475 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 4 set_mon_vals no callback set 2024-01-24T23:21:06.475 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 1 -- 172.21.15.115:0/1864905114 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff69c0047f0 con 0x7ff6a0133d30 2024-01-24T23:21:06.475 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:06.475 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:06.475 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a5133640 10 monclient: dump: 2024-01-24T23:21:06.475 INFO:teuthology.misc.health.smithi115.stderr:epoch 1 2024-01-24T23:21:06.475 INFO:teuthology.misc.health.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:06.475 INFO:teuthology.misc.health.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:06.475 INFO:teuthology.misc.health.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:06.475 INFO:teuthology.misc.health.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:06.475 INFO:teuthology.misc.health.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:06.475 INFO:teuthology.misc.health.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:06.476 INFO:teuthology.misc.health.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:06.476 INFO:teuthology.misc.health.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:06.476 INFO:teuthology.misc.health.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:06.476 INFO:teuthology.misc.health.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:06.476 INFO:teuthology.misc.health.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:06.476 INFO:teuthology.misc.health.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:06.476 INFO:teuthology.misc.health.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:06.476 INFO:teuthology.misc.health.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:06.476 INFO:teuthology.misc.health.smithi115.stderr: 2024-01-24T23:21:06.476 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 10 monclient: shutdown 2024-01-24T23:21:06.476 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:06.476 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 1 -- 172.21.15.115:0/1864905114 >> v1:172.21.15.115:6791/0 conn(0x7ff6a0133d30 legacy=0x7ff6a013c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:06.476 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 1 -- 172.21.15.115:0/1864905114 shutdown_connections 2024-01-24T23:21:06.476 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 1 -- 172.21.15.115:0/1864905114 >> 172.21.15.115:0/1864905114 conn(0x7ff6a00a6580 msgr2=0x7ff6a00a2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:06.477 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 1 -- 172.21.15.115:0/1864905114 shutdown_connections 2024-01-24T23:21:06.477 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 1 -- 172.21.15.115:0/1864905114 wait complete. 2024-01-24T23:21:06.477 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 10 monclient: build_initial_monmap 2024-01-24T23:21:06.477 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 10 monclient: monmap: 2024-01-24T23:21:06.477 INFO:teuthology.misc.health.smithi115.stderr:epoch 0 2024-01-24T23:21:06.477 INFO:teuthology.misc.health.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:06.477 INFO:teuthology.misc.health.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:06.477 INFO:teuthology.misc.health.smithi115.stderr:created 0.000000 2024-01-24T23:21:06.477 INFO:teuthology.misc.health.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:06.477 INFO:teuthology.misc.health.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:06.477 INFO:teuthology.misc.health.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:06.477 INFO:teuthology.misc.health.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:06.477 INFO:teuthology.misc.health.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:06.477 INFO:teuthology.misc.health.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:06.477 INFO:teuthology.misc.health.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:06.478 INFO:teuthology.misc.health.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:06.478 INFO:teuthology.misc.health.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:06.478 INFO:teuthology.misc.health.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:06.478 INFO:teuthology.misc.health.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:06.478 INFO:teuthology.misc.health.smithi115.stderr: 2024-01-24T23:21:06.478 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 1 Processor -- start 2024-01-24T23:21:06.478 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 1 -- start start 2024-01-24T23:21:06.478 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.463+0000 7ff6a8399640 10 monclient: init 2024-01-24T23:21:06.478 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff6a8399640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:06.478 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff6a8399640 10 monclient: _add_conns ranks=[5,7,3,4,6,1,2,8,0] 2024-01-24T23:21:06.478 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff6a8399640 10 monclient(hunting): picked mon.noname-h con 0x7ff6a00a00c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:06.478 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff6a8399640 10 monclient(hunting): picked mon.noname-i con 0x7ff6a00a3190 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:06.478 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff6a8399640 10 monclient(hunting): picked mon.noname-g con 0x7ff6a0133d30 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:06.478 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff6a8399640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a00723e0 con 0x7ff6a0133d30 2024-01-24T23:21:06.478 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff6a8399640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a01223c0 con 0x7ff6a00a00c0 2024-01-24T23:21:06.478 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff6a8399640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a00a2500 con 0x7ff6a00a3190 2024-01-24T23:21:06.478 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff6a8399640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:06.479 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff6a8399640 10 monclient(hunting): authenticate will time out at 945.392596s 2024-01-24T23:21:06.479 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff6a6936640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7ff6a0133d30 0x7ff6a01d3d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.115:53858/0 (socket says 172.21.15.115:53858) 2024-01-24T23:21:06.479 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff6a6936640 1 -- 172.21.15.115:0/414583182 learned_addr learned my addr 172.21.15.115:0/414583182 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:06.482 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1921371915 0 0) 0x7ff6a00a2500 con 0x7ff6a00a3190 2024-01-24T23:21:06.482 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:06.482 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:06.483 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:06.483 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient(hunting): my global_id is 4228 2024-01-24T23:21:06.483 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff684003560 con 0x7ff6a00a3190 2024-01-24T23:21:06.483 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 667862089 0 0) 0x7ff6a01223c0 con 0x7ff6a00a00c0 2024-01-24T23:21:06.483 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:06.483 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:06.483 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:06.483 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient(hunting): my global_id is 4199 2024-01-24T23:21:06.483 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6a00a2500 con 0x7ff6a00a00c0 2024-01-24T23:21:06.483 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3046592239 0 0) 0x7ff6a00a3570 con 0x7ff6a0133d30 2024-01-24T23:21:06.483 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:06.483 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:06.483 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:06.483 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient(hunting): my global_id is 4188 2024-01-24T23:21:06.483 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6a01223c0 con 0x7ff6a0133d30 2024-01-24T23:21:06.484 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3179411679 0 0) 0x7ff6a00a2500 con 0x7ff6a00a00c0 2024-01-24T23:21:06.484 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff6a00a3570 con 0x7ff6a00a00c0 2024-01-24T23:21:06.484 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff690001ab0 con 0x7ff6a00a00c0 2024-01-24T23:21:06.484 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:06.484 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:21:06.484 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient(hunting): dump: 2024-01-24T23:21:06.484 INFO:teuthology.misc.health.smithi115.stderr:epoch 1 2024-01-24T23:21:06.484 INFO:teuthology.misc.health.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:06.484 INFO:teuthology.misc.health.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:06.484 INFO:teuthology.misc.health.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:06.484 INFO:teuthology.misc.health.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:06.484 INFO:teuthology.misc.health.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:06.484 INFO:teuthology.misc.health.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:06.484 INFO:teuthology.misc.health.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:06.484 INFO:teuthology.misc.health.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:06.485 INFO:teuthology.misc.health.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:06.485 INFO:teuthology.misc.health.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:06.485 INFO:teuthology.misc.health.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:06.485 INFO:teuthology.misc.health.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:06.485 INFO:teuthology.misc.health.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:06.485 INFO:teuthology.misc.health.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:06.485 INFO:teuthology.misc.health.smithi115.stderr: 2024-01-24T23:21:06.485 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:06.485 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1117071568 0 0) 0x7ff684003560 con 0x7ff6a00a3190 2024-01-24T23:21:06.485 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff6a00a2500 con 0x7ff6a00a3190 2024-01-24T23:21:06.485 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff698001d40 con 0x7ff6a00a3190 2024-01-24T23:21:06.485 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:06.485 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-01-24T23:21:06.485 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient(hunting): dump: 2024-01-24T23:21:06.485 INFO:teuthology.misc.health.smithi115.stderr:epoch 1 2024-01-24T23:21:06.486 INFO:teuthology.misc.health.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:06.486 INFO:teuthology.misc.health.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:06.486 INFO:teuthology.misc.health.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:06.486 INFO:teuthology.misc.health.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:06.486 INFO:teuthology.misc.health.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:06.486 INFO:teuthology.misc.health.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:06.486 INFO:teuthology.misc.health.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:06.486 INFO:teuthology.misc.health.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:06.486 INFO:teuthology.misc.health.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:06.486 INFO:teuthology.misc.health.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:06.486 INFO:teuthology.misc.health.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:06.486 INFO:teuthology.misc.health.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:06.486 INFO:teuthology.misc.health.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:06.486 INFO:teuthology.misc.health.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:06.486 INFO:teuthology.misc.health.smithi115.stderr: 2024-01-24T23:21:06.487 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1499230549 0 0) 0x7ff6a01223c0 con 0x7ff6a0133d30 2024-01-24T23:21:06.487 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff684003560 con 0x7ff6a0133d30 2024-01-24T23:21:06.487 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff69c0016c0 con 0x7ff6a0133d30 2024-01-24T23:21:06.487 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:06.487 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:21:06.487 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient(hunting): dump: 2024-01-24T23:21:06.487 INFO:teuthology.misc.health.smithi115.stderr:epoch 1 2024-01-24T23:21:06.487 INFO:teuthology.misc.health.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:06.487 INFO:teuthology.misc.health.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:06.487 INFO:teuthology.misc.health.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:06.487 INFO:teuthology.misc.health.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:06.487 INFO:teuthology.misc.health.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:06.487 INFO:teuthology.misc.health.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:06.487 INFO:teuthology.misc.health.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:06.488 INFO:teuthology.misc.health.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:06.488 INFO:teuthology.misc.health.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:06.488 INFO:teuthology.misc.health.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:06.488 INFO:teuthology.misc.health.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:06.488 INFO:teuthology.misc.health.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:06.488 INFO:teuthology.misc.health.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:06.488 INFO:teuthology.misc.health.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:06.488 INFO:teuthology.misc.health.smithi115.stderr: 2024-01-24T23:21:06.488 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 51448706 0 0) 0x7ff6a00a3570 con 0x7ff6a00a00c0 2024-01-24T23:21:06.488 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 >> v1:172.21.15.132:6792/0 conn(0x7ff6a00a3190 legacy=0x7ff6a01d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:06.488 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 >> v1:172.21.15.132:6790/0 conn(0x7ff6a0133d30 legacy=0x7ff6a01d3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:06.488 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:21:06.488 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 1 monclient: found mon.h 2024-01-24T23:21:06.488 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:06.488 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:21:06.489 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a0121e00 con 0x7ff6a00a00c0 2024-01-24T23:21:06.489 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient: _finish_auth 0 2024-01-24T23:21:06.489 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:21:06.489 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:06.489 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff6a8399640 5 monclient: authenticate success, global_id 4199 2024-01-24T23:21:06.489 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff6a8399640 10 monclient: _renew_subs 2024-01-24T23:21:06.489 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff6a8399640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:21:06.489 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff6a8399640 1 -- 172.21.15.115:0/414583182 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a01220e0 con 0x7ff6a00a00c0 2024-01-24T23:21:06.489 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff6a8399640 10 monclient: _renew_subs 2024-01-24T23:21:06.489 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff6a8399640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:21:06.489 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff6a8399640 1 -- 172.21.15.115:0/414583182 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a01c87e0 con 0x7ff6a00a00c0 2024-01-24T23:21:06.489 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff690001c50 con 0x7ff6a00a00c0 2024-01-24T23:21:06.489 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:06.489 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff690004de0 con 0x7ff6a00a00c0 2024-01-24T23:21:06.490 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:06.490 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff6a4932640 4 set_mon_vals no callback set 2024-01-24T23:21:06.490 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:21:06.490 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 10 monclient: dump: 2024-01-24T23:21:06.490 INFO:teuthology.misc.health.smithi115.stderr:epoch 1 2024-01-24T23:21:06.490 INFO:teuthology.misc.health.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:06.490 INFO:teuthology.misc.health.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:06.490 INFO:teuthology.misc.health.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:06.490 INFO:teuthology.misc.health.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:06.490 INFO:teuthology.misc.health.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:06.490 INFO:teuthology.misc.health.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:06.490 INFO:teuthology.misc.health.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:06.490 INFO:teuthology.misc.health.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:06.490 INFO:teuthology.misc.health.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:06.490 INFO:teuthology.misc.health.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:06.490 INFO:teuthology.misc.health.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:06.491 INFO:teuthology.misc.health.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:06.491 INFO:teuthology.misc.health.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:06.491 INFO:teuthology.misc.health.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:06.491 INFO:teuthology.misc.health.smithi115.stderr: 2024-01-24T23:21:06.491 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff6a8399640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:06.491 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff6a8399640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:06.491 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff6a8399640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:21:06.491 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff6a8399640 1 -- 172.21.15.115:0/414583182 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff688005090 con 0x7ff6a00a00c0 2024-01-24T23:21:06.491 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.467+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 4) v1 ==== 43232+0+0 (unknown 913048011 0 0) 0x7ff69000f830 con 0x7ff6a00a00c0 2024-01-24T23:21:06.491 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.471+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(11..11 src has 1..11) v4 ==== 2767+0+0 (unknown 970375975 0 0) 0x7ff6900431d0 con 0x7ff6a00a00c0 2024-01-24T23:21:06.491 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.471+0000 7ff6a5934640 10 monclient: get_auth_request con 0x7ff684037bd0 auth_method 0 2024-01-24T23:21:06.492 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.483+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7ff690014740 con 0x7ff6a00a00c0 2024-01-24T23:21:06.492 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.483+0000 7ff696ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:06.492 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.483+0000 7ff696ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:06.641 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.635+0000 7ff6a8399640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-01-24T23:21:06.641 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.635+0000 7ff6a8399640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-01-24T23:21:06.641 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.635+0000 7ff6a8399640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:21:06.641 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.635+0000 7ff6a8399640 1 -- 172.21.15.115:0/414583182 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff688005340 con 0x7ff6a00a00c0 2024-01-24T23:21:06.642 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.639+0000 7ff696ffd640 1 -- 172.21.15.115:0/414583182 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+7081 (unknown 238102980 0 2850653133) 0x7ff690012de0 con 0x7ff6a00a00c0 2024-01-24T23:21:06.643 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.639+0000 7ff696ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-01-24T23:21:06.643 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.639+0000 7ff696ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:06.643 INFO:teuthology.misc.health.smithi115.stdout: 2024-01-24T23:21:06.643 INFO:teuthology.misc.health.smithi115.stdout:{"epoch":11,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","created":"2024-01-24T23:20:56.917703+0000","modified":"2024-01-24T23:21:05.857369+0000","last_up_change":"2024-01-24T23:21:05.857369+0000","last_in_change":"2024-01-24T23:20:58.690771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"902f4a51-85e9-4af8-b11f-e42a51b31723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6808","nonce":3064328804}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6809","nonce":3064328804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6811","nonce":3064328804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6810","nonce":3064328804}]},"public_addr":"172.21.15.115:6808/3064328804","cluster_addr":"172.21.15.115:6809/3064328804","heartbeat_back_addr":"172.21.15.115:6811/3064328804","heartbeat_front_addr":"172.21.15.115:6810/3064328804","state":["exists","up"]},{"osd":1,"uuid":"e06b92ed-3ab9-4103-8e41-4fa60aea2829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6801","nonce":3052044493}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6803","nonce":3052044493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6807","nonce":3052044493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6806","nonce":3052044493}]},"public_addr":"172.21.15.115:6801/3052044493","cluster_addr":"172.21.15.115:6803/3052044493","heartbeat_back_addr":"172.21.15.115:6807/3052044493","heartbeat_front_addr":"172.21.15.115:6806/3052044493","state":["exists","up"]},{"osd":2,"uuid":"013044c9-553c-4dd9-8438-c9873813e5af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6800","nonce":2776240662}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6802","nonce":2776240662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6805","nonce":2776240662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6804","nonce":2776240662}]},"public_addr":"172.21.15.115:6800/2776240662","cluster_addr":"172.21.15.115:6802/2776240662","heartbeat_back_addr":"172.21.15.115:6805/2776240662","heartbeat_front_addr":"172.21.15.115:6804/2776240662","state":["exists","up"]},{"osd":3,"uuid":"5fcff598-1939-415b-b055-57940893269e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6800","nonce":1048681489}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6801","nonce":1048681489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6804","nonce":1048681489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6802","nonce":1048681489}]},"public_addr":"172.21.15.132:6800/1048681489","cluster_addr":"172.21.15.132:6801/1048681489","heartbeat_back_addr":"172.21.15.132:6804/1048681489","heartbeat_front_addr":"172.21.15.132:6802/1048681489","state":["exists","up"]},{"osd":4,"uuid":"ef67ccd9-a21c-4a40-99ff-93f5b10a1fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6807","nonce":3601895184}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6809","nonce":3601895184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6811","nonce":3601895184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6810","nonce":3601895184}]},"public_addr":"172.21.15.132:6807/3601895184","cluster_addr":"172.21.15.132:6809/3601895184","heartbeat_back_addr":"172.21.15.132:6811/3601895184","heartbeat_front_addr":"172.21.15.132:6810/3601895184","state":["exists","up"]},{"osd":5,"uuid":"c2573ce8-6dd2-4272-85c7-2f5aa25eb2f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6803","nonce":3365641231}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6805","nonce":3365641231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6808","nonce":3365641231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6806","nonce":3365641231}]},"public_addr":"172.21.15.132:6803/3365641231","cluster_addr":"172.21.15.132:6805/3365641231","heartbeat_back_addr":"172.21.15.132:6808/3365641231","heartbeat_front_addr":"172.21.15.132:6806/3365641231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T23:21:06.645 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.639+0000 7ff6a8399640 1 -- 172.21.15.115:0/414583182 >> v1:172.21.15.132:6812/1835401922 conn(0x7ff684037bd0 legacy=0x7ff68403a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:06.645 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.639+0000 7ff6a8399640 10 monclient: shutdown 2024-01-24T23:21:06.645 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.639+0000 7ff6a8399640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:06.645 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.639+0000 7ff6a8399640 1 -- 172.21.15.115:0/414583182 >> v1:172.21.15.132:6791/0 conn(0x7ff6a00a00c0 legacy=0x7ff6a0132cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:06.645 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.639+0000 7ff6a8399640 1 -- 172.21.15.115:0/414583182 shutdown_connections 2024-01-24T23:21:06.645 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.639+0000 7ff6a8399640 1 -- 172.21.15.115:0/414583182 >> 172.21.15.115:0/414583182 conn(0x7ff6a00a6580 msgr2=0x7ff6a006a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:06.645 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.639+0000 7ff6a8399640 1 -- 172.21.15.115:0/414583182 shutdown_connections 2024-01-24T23:21:06.645 INFO:teuthology.misc.health.smithi115.stderr:2024-01-24T23:21:06.639+0000 7ff6a8399640 1 -- 172.21.15.115:0/414583182 wait complete. 2024-01-24T23:21:06.660 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-01-24T23:21:06.660 INFO:tasks.ceph:Creating RBD pool 2024-01-24T23:21:06.660 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd pool create rbd 8 2024-01-24T23:21:06.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586fdc1640 10 monclient: get_monmap_and_config 2024-01-24T23:21:06.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586fdc1640 10 monclient: build_initial_monmap 2024-01-24T23:21:06.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586fdc1640 10 monclient: monmap: 2024-01-24T23:21:06.783 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:06.783 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:06.783 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:06.784231+0000 2024-01-24T23:21:06.783 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:06.784231+0000 2024-01-24T23:21:06.783 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:06.783 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:06.783 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:06.783 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:06.783 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:06.783 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:06.784 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:06.784 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:06.784 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:06.784 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:06.784 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:06.784 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:06.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586fdc1640 1 Processor -- start 2024-01-24T23:21:06.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586fdc1640 1 -- start start 2024-01-24T23:21:06.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586fdc1640 10 monclient: init 2024-01-24T23:21:06.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586fdc1640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:06.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586fdc1640 10 monclient: _add_conns ranks=[2,1,6,0,7,3,5,4,8] 2024-01-24T23:21:06.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586fdc1640 10 monclient(hunting): picked mon.noname-b con 0x7f586809f840 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:06.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586fdc1640 10 monclient(hunting): picked mon.noname-f con 0x7f5868066d10 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:06.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586fdc1640 10 monclient(hunting): picked mon.noname-d con 0x7f586813b260 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:06.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586fdc1640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f586804ed70 con 0x7f5868066d10 2024-01-24T23:21:06.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586fdc1640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5868135760 con 0x7f586809f840 2024-01-24T23:21:06.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586fdc1640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58680a0330 con 0x7f586813b260 2024-01-24T23:21:06.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586fdc1640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:06.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586fdc1640 10 monclient(hunting): authenticate will time out at 945.706940s 2024-01-24T23:21:06.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586db5d640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f586809f840 0x7f586809fc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:54466/0 (socket says 172.21.15.115:54466) 2024-01-24T23:21:06.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586db5d640 1 -- 172.21.15.115:0/3946101287 learned_addr learned my addr 172.21.15.115:0/3946101287 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:06.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 1 -- 172.21.15.115:0/3946101287 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456827055 0 0) 0x7f5868135760 con 0x7f586809f840 2024-01-24T23:21:06.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 10 monclient(hunting): _negotiate 2024-01-24T23:21:06.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:06.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:06.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 10 monclient(hunting): my global_id is 4196 2024-01-24T23:21:06.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 1 -- 172.21.15.115:0/3946101287 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5854003530 con 0x7f586809f840 2024-01-24T23:21:06.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 1 -- 172.21.15.115:0/3946101287 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2920926496 0 0) 0x7f58680a0330 con 0x7f586813b260 2024-01-24T23:21:06.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 10 monclient(hunting): _negotiate 2024-01-24T23:21:06.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:06.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:06.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 10 monclient(hunting): my global_id is 4245 2024-01-24T23:21:06.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 1 -- 172.21.15.115:0/3946101287 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5868135760 con 0x7f586813b260 2024-01-24T23:21:06.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 1 -- 172.21.15.115:0/3946101287 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2901340632 0 0) 0x7f5854003530 con 0x7f586809f840 2024-01-24T23:21:06.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 1 -- 172.21.15.115:0/3946101287 >> v1:172.21.15.115:6792/0 conn(0x7f586813b260 legacy=0x7f586813d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:06.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 1 -- 172.21.15.115:0/3946101287 >> v1:172.21.15.132:6789/0 conn(0x7f5868066d10 legacy=0x7f5868139b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:06.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 10 monclient: _finish_hunting 0 2024-01-24T23:21:06.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 1 monclient: found mon.noname-b 2024-01-24T23:21:06.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:06.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:21:06.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 1 -- 172.21.15.115:0/3946101287 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f586809da70 con 0x7f586809f840 2024-01-24T23:21:06.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 10 monclient: _finish_auth 0 2024-01-24T23:21:06.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 10 monclient: _check_auth_tickets 2024-01-24T23:21:06.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:06.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 1 -- 172.21.15.115:0/3946101287 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f58640017c0 con 0x7f586809f840 2024-01-24T23:21:06.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:06.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:21:06.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 10 monclient: dump: 2024-01-24T23:21:06.793 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:06.793 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:06.793 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:06.793 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:06.793 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:06.793 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:06.793 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:06.793 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:06.793 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:06.793 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:06.793 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:06.793 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:06.794 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:06.794 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:06.794 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:06.794 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:06.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586fdc1640 5 monclient: authenticate success, global_id 4196 2024-01-24T23:21:06.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586fdc1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:06.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 1 -- 172.21.15.115:0/3946101287 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5864001aa0 con 0x7f586809f840 2024-01-24T23:21:06.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:06.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 1 -- 172.21.15.115:0/3946101287 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5864004bf0 con 0x7f586809f840 2024-01-24T23:21:06.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:06.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:06.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586cb5b640 10 monclient: dump: 2024-01-24T23:21:06.794 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:06.794 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:06.794 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:06.794 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:06.795 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:06.795 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:06.795 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:06.795 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:06.795 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:06.795 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:06.795 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:06.795 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:06.795 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:06.795 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:06.795 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:06.795 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:06.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586fdc1640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:06.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.779+0000 7f586fdc1640 4 set_mon_vals no callback set 2024-01-24T23:21:06.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 10 monclient: shutdown 2024-01-24T23:21:06.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:06.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 1 -- 172.21.15.115:0/3946101287 >> v1:172.21.15.115:6790/0 conn(0x7f586809f840 legacy=0x7f586809fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:06.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 1 -- 172.21.15.115:0/3946101287 shutdown_connections 2024-01-24T23:21:06.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 1 -- 172.21.15.115:0/3946101287 >> 172.21.15.115:0/3946101287 conn(0x7f58680a65e0 msgr2=0x7f58680a2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:06.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 1 -- 172.21.15.115:0/3946101287 shutdown_connections 2024-01-24T23:21:06.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 1 -- 172.21.15.115:0/3946101287 wait complete. 2024-01-24T23:21:06.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 10 monclient: build_initial_monmap 2024-01-24T23:21:06.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 10 monclient: monmap: 2024-01-24T23:21:06.796 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:06.796 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:06.796 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:06.796 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:06.796 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:06.797 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:06.797 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:06.797 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:06.797 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:06.797 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:06.797 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:06.797 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:06.797 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:06.797 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:06.797 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:06.797 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:06.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 1 Processor -- start 2024-01-24T23:21:06.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 1 -- start start 2024-01-24T23:21:06.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 10 monclient: init 2024-01-24T23:21:06.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:06.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 10 monclient: _add_conns ranks=[5,3,8,7,0,2,1,6,4] 2024-01-24T23:21:06.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 10 monclient(hunting): picked mon.noname-h con 0x7f5868066d10 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:06.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 10 monclient(hunting): picked mon.noname-g con 0x7f586809f840 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:06.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 10 monclient(hunting): picked mon.noname-e con 0x7f586813b260 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:06.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f586811f480 con 0x7f586809f840 2024-01-24T23:21:06.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58681208c0 con 0x7f5868066d10 2024-01-24T23:21:06.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5868120ba0 con 0x7f586813b260 2024-01-24T23:21:06.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:06.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 10 monclient(hunting): authenticate will time out at 945.710025s 2024-01-24T23:21:06.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586e35e640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f586813b260 0x7f58681d3d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:48702/0 (socket says 172.21.15.115:48702) 2024-01-24T23:21:06.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586e35e640 1 -- 172.21.15.115:0/2241522929 learned_addr learned my addr 172.21.15.115:0/2241522929 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:06.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 1 -- 172.21.15.115:0/2241522929 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826663491 0 0) 0x7f5868120ba0 con 0x7f586813b260 2024-01-24T23:21:06.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:06.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:06.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:06.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 10 monclient(hunting): my global_id is 74168 2024-01-24T23:21:06.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 1 -- 172.21.15.115:0/2241522929 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f584c003530 con 0x7f586813b260 2024-01-24T23:21:06.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 1 -- 172.21.15.115:0/2241522929 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2313033128 0 0) 0x7f584c003530 con 0x7f586813b260 2024-01-24T23:21:06.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 1 -- 172.21.15.115:0/2241522929 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5868120ba0 con 0x7f586813b260 2024-01-24T23:21:06.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 1 -- 172.21.15.115:0/2241522929 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5860001da0 con 0x7f586813b260 2024-01-24T23:21:06.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:06.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:21:06.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 10 monclient(hunting): dump: 2024-01-24T23:21:06.799 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:06.799 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:06.799 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:06.799 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:06.799 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:06.799 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:06.800 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:06.800 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:06.800 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:06.800 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:06.800 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:06.800 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:06.800 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:06.800 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:06.800 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:06.800 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:06.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:06.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 1 -- 172.21.15.115:0/2241522929 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3550747834 0 0) 0x7f5868120ba0 con 0x7f586813b260 2024-01-24T23:21:06.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 1 -- 172.21.15.115:0/2241522929 >> v1:172.21.15.132:6791/0 conn(0x7f5868066d10 legacy=0x7f58681cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:06.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 1 -- 172.21.15.115:0/2241522929 >> v1:172.21.15.132:6790/0 conn(0x7f586809f840 legacy=0x7f58681d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:06.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:21:06.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 1 monclient: found mon.e 2024-01-24T23:21:06.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:06.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:06.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 1 -- 172.21.15.115:0/2241522929 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5868071d20 con 0x7f586813b260 2024-01-24T23:21:06.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 10 monclient: _finish_auth 0 2024-01-24T23:21:06.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:21:06.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:06.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 5 monclient: authenticate success, global_id 74168 2024-01-24T23:21:06.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 10 monclient: _renew_subs 2024-01-24T23:21:06.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:06.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 1 -- 172.21.15.115:0/2241522929 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5868067700 con 0x7f586813b260 2024-01-24T23:21:06.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 10 monclient: _renew_subs 2024-01-24T23:21:06.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:06.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 1 -- 172.21.15.115:0/2241522929 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5868120fc0 con 0x7f586813b260 2024-01-24T23:21:06.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 1 -- 172.21.15.115:0/2241522929 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5860003c30 con 0x7f586813b260 2024-01-24T23:21:06.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:06.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 1 -- 172.21.15.115:0/2241522929 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5860004160 con 0x7f586813b260 2024-01-24T23:21:06.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:06.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:21:06.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585ffff640 4 set_mon_vals no callback set 2024-01-24T23:21:06.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 10 monclient: dump: 2024-01-24T23:21:06.802 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:06.802 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:06.802 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:06.802 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:06.802 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:06.802 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:06.803 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:06.803 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:06.803 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:06.803 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:06.803 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:06.803 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:06.803 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:06.803 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:06.803 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:06.803 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:06.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 1 -- 172.21.15.115:0/2241522929 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 4) v1 ==== 43232+0+0 (unknown 913048011 0 0) 0x7f5860004340 con 0x7f586813b260 2024-01-24T23:21:06.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:06.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:06.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:06.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f585e7fc640 1 -- 172.21.15.115:0/2241522929 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(11..11 src has 1..11) v4 ==== 2767+0+0 (unknown 970375975 0 0) 0x7f5860043710 con 0x7f586813b260 2024-01-24T23:21:06.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.783+0000 7f586fdc1640 1 -- 172.21.15.115:0/2241522929 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5850005090 con 0x7f586813b260 2024-01-24T23:21:06.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.787+0000 7f586db5d640 10 monclient: get_auth_request con 0x7f584c036ea0 auth_method 0 2024-01-24T23:21:06.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.791+0000 7f585e7fc640 1 -- 172.21.15.115:0/2241522929 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f5860004640 con 0x7f586813b260 2024-01-24T23:21:06.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.791+0000 7f585e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:06.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.791+0000 7f585e7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:06.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.963+0000 7f586fdc1640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-01-24T23:21:06.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.963+0000 7f586fdc1640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-01-24T23:21:06.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.963+0000 7f586fdc1640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:06.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:06.963+0000 7f586fdc1640 1 -- 172.21.15.115:0/2241522929 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f58500058c0 con 0x7f586813b260 2024-01-24T23:21:07.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:07.927+0000 7f585e7fc640 1 -- 172.21.15.115:0/2241522929 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v13) v1 ==== 109+0+0 (unknown 2984082091 0 0) 0x7f58600132c0 con 0x7f586813b260 2024-01-24T23:21:07.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:07.927+0000 7f585e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-01-24T23:21:07.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:07.927+0000 7f585e7fc640 10 monclient: _finish_command 2 = system:0 pool 'rbd' created 2024-01-24T23:21:07.931 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'rbd' created 2024-01-24T23:21:07.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:07.931+0000 7f586fdc1640 1 -- 172.21.15.115:0/2241522929 >> v1:172.21.15.132:6812/1835401922 conn(0x7f584c036ea0 legacy=0x7f584c039360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:07.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:07.931+0000 7f586fdc1640 10 monclient: shutdown 2024-01-24T23:21:07.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:07.931+0000 7f586fdc1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:07.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:07.931+0000 7f586fdc1640 1 -- 172.21.15.115:0/2241522929 >> v1:172.21.15.115:6793/0 conn(0x7f586813b260 legacy=0x7f58681d3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:07.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:07.931+0000 7f586fdc1640 1 -- 172.21.15.115:0/2241522929 shutdown_connections 2024-01-24T23:21:07.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:07.931+0000 7f586fdc1640 1 -- 172.21.15.115:0/2241522929 >> 172.21.15.115:0/2241522929 conn(0x7f58680a65e0 msgr2=0x7f58680a4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:07.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:07.931+0000 7f586fdc1640 1 -- 172.21.15.115:0/2241522929 shutdown_connections 2024-01-24T23:21:07.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:07.931+0000 7f586fdc1640 1 -- 172.21.15.115:0/2241522929 wait complete. 2024-01-24T23:21:07.951 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-01-24T23:21:08.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.071+0000 7fbdd6adb640 10 monclient: get_monmap_and_config 2024-01-24T23:21:08.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.071+0000 7fbdd6adb640 10 monclient: build_initial_monmap 2024-01-24T23:21:08.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.071+0000 7fbdd6adb640 10 monclient: monmap: 2024-01-24T23:21:08.076 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:08.076 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:08.076 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:08.077001+0000 2024-01-24T23:21:08.076 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:08.077001+0000 2024-01-24T23:21:08.076 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:08.076 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:08.076 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:08.076 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:08.076 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:08.076 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:08.076 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:08.077 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:08.077 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:08.077 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:08.077 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:08.077 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:08.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.071+0000 7fbdd6adb640 1 Processor -- start 2024-01-24T23:21:08.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.071+0000 7fbdd6adb640 1 -- start start 2024-01-24T23:21:08.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.071+0000 7fbdd6adb640 10 monclient: init 2024-01-24T23:21:08.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.071+0000 7fbdd6adb640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:08.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.071+0000 7fbdd6adb640 10 monclient: _add_conns ranks=[2,7,5,3,1,0,4,6,8] 2024-01-24T23:21:08.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.071+0000 7fbdd6adb640 10 monclient(hunting): picked mon.noname-b con 0x7fbdd0135cf0 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:08.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 10 monclient(hunting): picked mon.noname-i con 0x7fbdd00a54b0 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:08.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 10 monclient(hunting): picked mon.noname-h con 0x7fbdd013d460 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:08.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd004ed70 con 0x7fbdd0135cf0 2024-01-24T23:21:08.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd009b760 con 0x7fbdd013d460 2024-01-24T23:21:08.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd009b1a0 con 0x7fbdd00a54b0 2024-01-24T23:21:08.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:08.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 10 monclient(hunting): authenticate will time out at 947.000104s 2024-01-24T23:21:08.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd4877640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fbdd0135cf0 0x7fbdd01360d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:54482/0 (socket says 172.21.15.115:54482) 2024-01-24T23:21:08.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd4877640 1 -- 172.21.15.115:0/3665561622 learned_addr learned my addr 172.21.15.115:0/3665561622 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:08.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 1 -- 172.21.15.115:0/3665561622 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3551778085 0 0) 0x7fbdd004ed70 con 0x7fbdd0135cf0 2024-01-24T23:21:08.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:08.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:08.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:08.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 10 monclient(hunting): my global_id is 4205 2024-01-24T23:21:08.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 1 -- 172.21.15.115:0/3665561622 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdc0003530 con 0x7fbdd0135cf0 2024-01-24T23:21:08.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 1 -- 172.21.15.115:0/3665561622 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1738107874 0 0) 0x7fbdc0003530 con 0x7fbdd0135cf0 2024-01-24T23:21:08.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 1 -- 172.21.15.115:0/3665561622 >> v1:172.21.15.132:6792/0 conn(0x7fbdd00a54b0 legacy=0x7fbdd013bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:08.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 1 -- 172.21.15.115:0/3665561622 >> v1:172.21.15.132:6791/0 conn(0x7fbdd013d460 legacy=0x7fbdd013f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:08.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:21:08.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 1 monclient: found mon.noname-b 2024-01-24T23:21:08.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:08.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:21:08.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 1 -- 172.21.15.115:0/3665561622 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd01367e0 con 0x7fbdd0135cf0 2024-01-24T23:21:08.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 10 monclient: _finish_auth 0 2024-01-24T23:21:08.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:21:08.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:08.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 1 -- 172.21.15.115:0/3665561622 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbdc40017c0 con 0x7fbdd0135cf0 2024-01-24T23:21:08.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 5 monclient: authenticate success, global_id 4205 2024-01-24T23:21:08.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:08.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:08.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:21:08.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 10 monclient: dump: 2024-01-24T23:21:08.085 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:08.085 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:08.085 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:08.085 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:08.085 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:08.086 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:08.086 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:08.086 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:08.086 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:08.086 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:08.086 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:08.086 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:08.086 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:08.086 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:08.086 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:08.086 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:08.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:08.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 1 -- 172.21.15.115:0/3665561622 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbdc4001a70 con 0x7fbdd0135cf0 2024-01-24T23:21:08.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:08.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:08.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 1 -- 172.21.15.115:0/3665561622 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbdc4004bf0 con 0x7fbdd0135cf0 2024-01-24T23:21:08.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:08.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 4 set_mon_vals no callback set 2024-01-24T23:21:08.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:08.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcf7fe640 10 monclient: dump: 2024-01-24T23:21:08.087 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:08.087 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:08.087 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:08.087 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:08.087 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:08.087 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:08.087 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:08.087 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:08.087 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:08.087 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:08.087 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:08.088 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:08.088 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:08.088 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:08.088 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:08.088 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:08.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 10 monclient: shutdown 2024-01-24T23:21:08.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:08.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 1 -- 172.21.15.115:0/3665561622 >> v1:172.21.15.115:6790/0 conn(0x7fbdd0135cf0 legacy=0x7fbdd01360d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:08.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 1 -- 172.21.15.115:0/3665561622 shutdown_connections 2024-01-24T23:21:08.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 1 -- 172.21.15.115:0/3665561622 >> 172.21.15.115:0/3665561622 conn(0x7fbdd00a98d0 msgr2=0x7fbdd00a5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:08.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 1 -- 172.21.15.115:0/3665561622 shutdown_connections 2024-01-24T23:21:08.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 1 -- 172.21.15.115:0/3665561622 wait complete. 2024-01-24T23:21:08.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 10 monclient: build_initial_monmap 2024-01-24T23:21:08.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 10 monclient: monmap: 2024-01-24T23:21:08.088 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:08.089 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:08.089 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:08.089 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:08.089 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:08.089 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:08.089 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:08.089 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:08.089 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:08.089 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:08.089 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:08.089 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:08.089 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:08.089 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:08.089 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:08.089 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:08.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 1 Processor -- start 2024-01-24T23:21:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 1 -- start start 2024-01-24T23:21:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 10 monclient: init 2024-01-24T23:21:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 10 monclient: _add_conns ranks=[0,5,3,7,6,2,1,4,8] 2024-01-24T23:21:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 10 monclient(hunting): picked mon.noname-a con 0x7fbdd00a54b0 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 10 monclient(hunting): picked mon.noname-h con 0x7fbdd0135cf0 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 10 monclient(hunting): picked mon.noname-g con 0x7fbdd013d460 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd009c3a0 con 0x7fbdd00a54b0 2024-01-24T23:21:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd0134400 con 0x7fbdd013d460 2024-01-24T23:21:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd0134180 con 0x7fbdd0135cf0 2024-01-24T23:21:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd6adb640 10 monclient(hunting): authenticate will time out at 947.003314s 2024-01-24T23:21:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd4877640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fbdd00a54b0 0x7fbdd0133360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:52060/0 (socket says 172.21.15.115:52060) 2024-01-24T23:21:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdd4877640 1 -- 172.21.15.115:0/1406327324 learned_addr learned my addr 172.21.15.115:0/1406327324 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcd7fa640 1 -- 172.21.15.115:0/1406327324 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3540888047 0 0) 0x7fbdd009c3a0 con 0x7fbdd00a54b0 2024-01-24T23:21:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcd7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcd7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcd7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcd7fa640 10 monclient(hunting): my global_id is 4248 2024-01-24T23:21:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcd7fa640 1 -- 172.21.15.115:0/1406327324 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdb4003530 con 0x7fbdd00a54b0 2024-01-24T23:21:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcd7fa640 1 -- 172.21.15.115:0/1406327324 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1033659239 0 0) 0x7fbdb4003530 con 0x7fbdd00a54b0 2024-01-24T23:21:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcd7fa640 1 -- 172.21.15.115:0/1406327324 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbdd009c3a0 con 0x7fbdd00a54b0 2024-01-24T23:21:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcd7fa640 1 -- 172.21.15.115:0/1406327324 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbdc4001780 con 0x7fbdd00a54b0 2024-01-24T23:21:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcd7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcd7fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:21:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcd7fa640 10 monclient(hunting): dump: 2024-01-24T23:21:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:08.092 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:08.092 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:08.092 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:08.092 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:08.092 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:08.092 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:08.092 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:08.092 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:08.092 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:08.092 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:08.092 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:08.092 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:08.094 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:08.094 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:08.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.075+0000 7fbdcd7fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:08.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 1 -- 172.21.15.115:0/1406327324 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1050873654 0 0) 0x7fbdd0134180 con 0x7fbdd0135cf0 2024-01-24T23:21:08.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:08.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:08.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:08.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 10 monclient(hunting): my global_id is 4226 2024-01-24T23:21:08.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 1 -- 172.21.15.115:0/1406327324 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdb4003530 con 0x7fbdd0135cf0 2024-01-24T23:21:08.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 1 -- 172.21.15.115:0/1406327324 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1541835796 0 0) 0x7fbdd009c3a0 con 0x7fbdd00a54b0 2024-01-24T23:21:08.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 1 -- 172.21.15.115:0/1406327324 >> v1:172.21.15.132:6791/0 conn(0x7fbdd0135cf0 legacy=0x7fbdd0133a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 1 -- 172.21.15.115:0/1406327324 >> v1:172.21.15.132:6790/0 conn(0x7fbdd013d460 legacy=0x7fbdd01d3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 10 monclient: _finish_hunting 0 2024-01-24T23:21:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 1 monclient: found mon.a 2024-01-24T23:21:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:21:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 1 -- 172.21.15.115:0/1406327324 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd009c7f0 con 0x7fbdd00a54b0 2024-01-24T23:21:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 10 monclient: _finish_auth 0 2024-01-24T23:21:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:21:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdd6adb640 5 monclient: authenticate success, global_id 4248 2024-01-24T23:21:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdd6adb640 10 monclient: _renew_subs 2024-01-24T23:21:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdd6adb640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:21:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdd6adb640 1 -- 172.21.15.115:0/1406327324 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd00a5900 con 0x7fbdd00a54b0 2024-01-24T23:21:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdd6adb640 10 monclient: _renew_subs 2024-01-24T23:21:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdd6adb640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:21:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdd6adb640 1 -- 172.21.15.115:0/1406327324 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd01356d0 con 0x7fbdd00a54b0 2024-01-24T23:21:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 1 -- 172.21.15.115:0/1406327324 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbdc4001200 con 0x7fbdd00a54b0 2024-01-24T23:21:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 1 -- 172.21.15.115:0/1406327324 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbdc4005280 con 0x7fbdd00a54b0 2024-01-24T23:21:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdceffd640 4 set_mon_vals no callback set 2024-01-24T23:21:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 10 monclient: dump: 2024-01-24T23:21:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:08.097 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdd6adb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdd6adb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdd6adb640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:21:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdd6adb640 1 -- 172.21.15.115:0/1406327324 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb0005090 con 0x7fbdd00a54b0 2024-01-24T23:21:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 1 -- 172.21.15.115:0/1406327324 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 4) v1 ==== 43232+0+0 (unknown 913048011 0 0) 0x7fbdc400fd50 con 0x7fbdd00a54b0 2024-01-24T23:21:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcd7fa640 1 -- 172.21.15.115:0/1406327324 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(13..13 src has 1..13) v4 ==== 3484+0+0 (unknown 1163955648 0 0) 0x7fbdc4043920 con 0x7fbdd00a54b0 2024-01-24T23:21:08.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.079+0000 7fbdcffff640 10 monclient: get_auth_request con 0x7fbdb4036e90 auth_method 0 2024-01-24T23:21:08.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.083+0000 7fbdcd7fa640 1 -- 172.21.15.115:0/1406327324 <== mon.0 v1:172.21.15.115:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fbdc4014cf0 con 0x7fbdd00a54b0 2024-01-24T23:21:08.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.083+0000 7fbdcd7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:08.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.083+0000 7fbdcd7fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:08.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.235+0000 7fbdd6adb640 10 start_mon_command cmd=[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}] 2024-01-24T23:21:08.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.235+0000 7fbdd6adb640 10 monclient: _send_command 2 [{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}] 2024-01-24T23:21:08.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.235+0000 7fbdd6adb640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:21:08.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.235+0000 7fbdd6adb640 1 -- 172.21.15.115:0/1406327324 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbdb00058c0 con 0x7fbdd00a54b0 2024-01-24T23:21:08.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.931+0000 7fbdcd7fa640 1 -- 172.21.15.115:0/1406327324 <== mon.0 v1:172.21.15.115:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v14) v1 ==== 173+0+0 (unknown 3289125425 0 0) 0x7fbdc4013390 con 0x7fbdd00a54b0 2024-01-24T23:21:08.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.931+0000 7fbdcd7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}] 2024-01-24T23:21:08.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.931+0000 7fbdcd7fa640 10 monclient: _finish_command 2 = system:0 enabled application 'rbd' on pool 'rbd' 2024-01-24T23:21:08.937 INFO:teuthology.orchestra.run.smithi115.stderr:enabled application 'rbd' on pool 'rbd' 2024-01-24T23:21:08.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.935+0000 7fbdd6adb640 1 -- 172.21.15.115:0/1406327324 >> v1:172.21.15.132:6812/1835401922 conn(0x7fbdb4036e90 legacy=0x7fbdb4039350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:08.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.935+0000 7fbdd6adb640 10 monclient: shutdown 2024-01-24T23:21:08.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.935+0000 7fbdd6adb640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:08.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.935+0000 7fbdd6adb640 1 -- 172.21.15.115:0/1406327324 >> v1:172.21.15.115:6789/0 conn(0x7fbdd00a54b0 legacy=0x7fbdd0133360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:08.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.935+0000 7fbdd6adb640 1 -- 172.21.15.115:0/1406327324 shutdown_connections 2024-01-24T23:21:08.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.935+0000 7fbdd6adb640 1 -- 172.21.15.115:0/1406327324 >> 172.21.15.115:0/1406327324 conn(0x7fbdd00a98d0 msgr2=0x7fbdd00a7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:08.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.935+0000 7fbdd6adb640 1 -- 172.21.15.115:0/1406327324 shutdown_connections 2024-01-24T23:21:08.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:08.935+0000 7fbdd6adb640 1 -- 172.21.15.115:0/1406327324 wait complete. 2024-01-24T23:21:08.961 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-01-24T23:21:08.961 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-01-24T23:21:08.962 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-01-24T23:21:08.962 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-01-24T23:21:08.962 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-01-24T23:21:09.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.075+0000 7f51faf4b640 10 monclient: get_monmap_and_config 2024-01-24T23:21:09.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.075+0000 7f51faf4b640 10 monclient: build_initial_monmap 2024-01-24T23:21:09.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.075+0000 7f51faf4b640 10 monclient: monmap: 2024-01-24T23:21:09.081 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:09.081 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:09.081 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:09.082207+0000 2024-01-24T23:21:09.081 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:09.082207+0000 2024-01-24T23:21:09.081 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:09.081 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:09.081 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:09.081 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:09.081 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:09.081 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:09.081 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:09.081 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:09.082 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:09.082 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:09.082 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:09.082 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:09.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 1 Processor -- start 2024-01-24T23:21:09.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 1 -- start start 2024-01-24T23:21:09.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 10 monclient: init 2024-01-24T23:21:09.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:09.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 10 monclient: _add_conns ranks=[6,4,5,7,3,1,8,0,2] 2024-01-24T23:21:09.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 10 monclient(hunting): picked mon.noname-d con 0x7f51f4069830 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:09.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 10 monclient(hunting): picked mon.noname-c con 0x7f51f4135320 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:09.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 10 monclient(hunting): picked mon.noname-h con 0x7f51f413b110 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:09.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f404ed70 con 0x7f51f4135320 2024-01-24T23:21:09.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f4069c10 con 0x7f51f413b110 2024-01-24T23:21:09.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f41347e0 con 0x7f51f4069830 2024-01-24T23:21:09.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:09.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 10 monclient(hunting): authenticate will time out at 948.005661s 2024-01-24T23:21:09.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f8ce7640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f51f4069830 0x7f51f4134c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:56834/0 (socket says 172.21.15.115:56834) 2024-01-24T23:21:09.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f8ce7640 1 -- 172.21.15.115:0/355305557 learned_addr learned my addr 172.21.15.115:0/355305557 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:09.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 1 -- 172.21.15.115:0/355305557 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2207425088 0 0) 0x7f51f41347e0 con 0x7f51f4069830 2024-01-24T23:21:09.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:09.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:09.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:09.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 10 monclient(hunting): my global_id is 4254 2024-01-24T23:21:09.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 1 -- 172.21.15.115:0/355305557 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51e4003530 con 0x7f51f4069830 2024-01-24T23:21:09.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 1 -- 172.21.15.115:0/355305557 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3270155089 0 0) 0x7f51f404ed70 con 0x7f51f4135320 2024-01-24T23:21:09.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:09.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:09.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:09.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 10 monclient(hunting): my global_id is 74236 2024-01-24T23:21:09.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 1 -- 172.21.15.115:0/355305557 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51f41347e0 con 0x7f51f4135320 2024-01-24T23:21:09.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 1 -- 172.21.15.115:0/355305557 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1789057829 0 0) 0x7f51e4003530 con 0x7f51f4069830 2024-01-24T23:21:09.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 1 -- 172.21.15.115:0/355305557 >> v1:172.21.15.132:6791/0 conn(0x7f51f413b110 legacy=0x7f51f413d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:09.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 1 -- 172.21.15.115:0/355305557 >> v1:172.21.15.115:6791/0 conn(0x7f51f4135320 legacy=0x7f51f41399f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:09.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 10 monclient: _finish_hunting 0 2024-01-24T23:21:09.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 1 monclient: found mon.noname-d 2024-01-24T23:21:09.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:09.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:21:09.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 1 -- 172.21.15.115:0/355305557 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f4071ef0 con 0x7f51f4069830 2024-01-24T23:21:09.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 10 monclient: _finish_auth 0 2024-01-24T23:21:09.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 10 monclient: _check_auth_tickets 2024-01-24T23:21:09.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:09.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 1 -- 172.21.15.115:0/355305557 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f51e80017e0 con 0x7f51f4069830 2024-01-24T23:21:09.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 5 monclient: authenticate success, global_id 4254 2024-01-24T23:21:09.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:09.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:09.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:21:09.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 10 monclient: dump: 2024-01-24T23:21:09.088 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:09.088 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:09.088 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.088 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.089 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:09.089 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:09.089 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:09.089 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:09.089 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:09.089 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:09.089 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:09.089 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:09.089 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:09.089 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:09.089 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:09.089 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:09.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:09.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 1 -- 172.21.15.115:0/355305557 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f51e8001a80 con 0x7f51f4069830 2024-01-24T23:21:09.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:09.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:09.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 4 set_mon_vals no callback set 2024-01-24T23:21:09.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 1 -- 172.21.15.115:0/355305557 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f51e8004bf0 con 0x7f51f4069830 2024-01-24T23:21:09.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:09.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:09.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51f37fe640 10 monclient: dump: 2024-01-24T23:21:09.090 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:09.090 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:09.090 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.090 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.090 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:09.090 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:09.090 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:09.090 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:09.090 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:09.090 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:09.091 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:09.091 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:09.091 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:09.091 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:09.091 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:09.091 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:09.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 10 monclient: shutdown 2024-01-24T23:21:09.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:09.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 1 -- 172.21.15.115:0/355305557 >> v1:172.21.15.115:6792/0 conn(0x7f51f4069830 legacy=0x7f51f4134c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:09.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 1 -- 172.21.15.115:0/355305557 shutdown_connections 2024-01-24T23:21:09.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 1 -- 172.21.15.115:0/355305557 >> 172.21.15.115:0/355305557 conn(0x7f51f40a2bf0 msgr2=0x7f51f40665f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:09.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 1 -- 172.21.15.115:0/355305557 shutdown_connections 2024-01-24T23:21:09.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 1 -- 172.21.15.115:0/355305557 wait complete. 2024-01-24T23:21:09.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.079+0000 7f51faf4b640 10 monclient: build_initial_monmap 2024-01-24T23:21:09.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 10 monclient: monmap: 2024-01-24T23:21:09.092 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:09.092 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:09.092 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:09.092 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:09.092 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:09.092 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:09.092 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:09.092 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:09.092 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:09.092 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:09.092 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:09.092 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:09.092 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:09.092 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:09.093 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:09.093 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:09.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 1 Processor -- start 2024-01-24T23:21:09.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 1 -- start start 2024-01-24T23:21:09.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 10 monclient: init 2024-01-24T23:21:09.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:09.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 10 monclient: _add_conns ranks=[6,3,0,7,4,2,8,5,1] 2024-01-24T23:21:09.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 10 monclient(hunting): picked mon.noname-d con 0x7f51f4069830 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:09.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 10 monclient(hunting): picked mon.noname-g con 0x7f51f4135320 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:09.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 10 monclient(hunting): picked mon.noname-a con 0x7f51f413b110 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:09.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f4135840 con 0x7f51f413b110 2024-01-24T23:21:09.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f4135b20 con 0x7f51f4135320 2024-01-24T23:21:09.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f4135e00 con 0x7f51f4069830 2024-01-24T23:21:09.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:09.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 10 monclient(hunting): authenticate will time out at 948.009037s 2024-01-24T23:21:09.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f8ce7640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f51f4069830 0x7f51f41271e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:56850/0 (socket says 172.21.15.115:56850) 2024-01-24T23:21:09.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f8ce7640 1 -- 172.21.15.115:0/2306875992 learned_addr learned my addr 172.21.15.115:0/2306875992 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:09.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 1 -- 172.21.15.115:0/2306875992 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 803145918 0 0) 0x7f51f4135e00 con 0x7f51f4069830 2024-01-24T23:21:09.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:09.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:09.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:09.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 10 monclient(hunting): my global_id is 4263 2024-01-24T23:21:09.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 1 -- 172.21.15.115:0/2306875992 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51d8003530 con 0x7f51f4069830 2024-01-24T23:21:09.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 1 -- 172.21.15.115:0/2306875992 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1642784998 0 0) 0x7f51f4135840 con 0x7f51f413b110 2024-01-24T23:21:09.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:09.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:09.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:09.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 10 monclient(hunting): my global_id is 4257 2024-01-24T23:21:09.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 1 -- 172.21.15.115:0/2306875992 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51f4135e00 con 0x7f51f413b110 2024-01-24T23:21:09.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 1 -- 172.21.15.115:0/2306875992 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1352684517 0 0) 0x7f51d8003530 con 0x7f51f4069830 2024-01-24T23:21:09.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 1 -- 172.21.15.115:0/2306875992 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f51f4135840 con 0x7f51f4069830 2024-01-24T23:21:09.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 1 -- 172.21.15.115:0/2306875992 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f51e8004b30 con 0x7f51f4069830 2024-01-24T23:21:09.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:09.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:21:09.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 10 monclient(hunting): dump: 2024-01-24T23:21:09.098 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:09.098 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:09.098 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.099 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.099 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:09.099 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:09.099 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:09.099 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:09.099 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:09.099 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:09.099 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:09.099 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:09.099 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:09.099 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:09.099 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:09.099 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:09.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:09.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 1 -- 172.21.15.115:0/2306875992 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1084738073 0 0) 0x7f51f4135e00 con 0x7f51f413b110 2024-01-24T23:21:09.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 1 -- 172.21.15.115:0/2306875992 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f51d8003530 con 0x7f51f413b110 2024-01-24T23:21:09.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 1 -- 172.21.15.115:0/2306875992 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f51ec001d20 con 0x7f51f413b110 2024-01-24T23:21:09.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:09.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:09.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 10 monclient(hunting): dump: 2024-01-24T23:21:09.100 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:09.100 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:09.100 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.100 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.100 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:09.100 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:09.100 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:09.101 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:09.101 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:09.101 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:09.101 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:09.101 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:09.101 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:09.101 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:09.101 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:09.101 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:09.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 1 -- 172.21.15.115:0/2306875992 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3096844732 0 0) 0x7f51f4135840 con 0x7f51f4069830 2024-01-24T23:21:09.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 1 -- 172.21.15.115:0/2306875992 >> v1:172.21.15.132:6790/0 conn(0x7f51f4135320 legacy=0x7f51f41ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:09.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 1 -- 172.21.15.115:0/2306875992 >> v1:172.21.15.115:6789/0 conn(0x7f51f413b110 legacy=0x7f51f41d1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:09.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 10 monclient: _finish_hunting 0 2024-01-24T23:21:09.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 1 monclient: found mon.d 2024-01-24T23:21:09.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:09.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:09.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 1 -- 172.21.15.115:0/2306875992 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f4131230 con 0x7f51f4069830 2024-01-24T23:21:09.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 10 monclient: _finish_auth 0 2024-01-24T23:21:09.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 10 monclient: _check_auth_tickets 2024-01-24T23:21:09.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:09.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 5 monclient: authenticate success, global_id 4263 2024-01-24T23:21:09.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 10 monclient: _renew_subs 2024-01-24T23:21:09.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:09.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 1 -- 172.21.15.115:0/2306875992 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f4130710 con 0x7f51f4069830 2024-01-24T23:21:09.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 10 monclient: _renew_subs 2024-01-24T23:21:09.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 1 -- 172.21.15.115:0/2306875992 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f51e8004e60 con 0x7f51f4069830 2024-01-24T23:21:09.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:09.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 1 -- 172.21.15.115:0/2306875992 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f4136c20 con 0x7f51f4069830 2024-01-24T23:21:09.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:09.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 1 -- 172.21.15.115:0/2306875992 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f51e8005310 con 0x7f51f4069830 2024-01-24T23:21:09.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:09.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:09.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f2ffd640 4 set_mon_vals no callback set 2024-01-24T23:21:09.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51f17fa640 10 monclient: dump: 2024-01-24T23:21:09.103 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:09.103 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:09.103 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.103 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.103 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:09.103 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:09.103 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:09.103 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:09.104 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:09.104 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:09.104 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:09.104 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:09.104 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:09.104 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:09.104 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:09.104 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:09.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:09.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:09.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:09.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.083+0000 7f51faf4b640 1 -- 172.21.15.115:0/2306875992 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f4130900 con 0x7f51f4069830 2024-01-24T23:21:09.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.087+0000 7f51f17fa640 1 -- 172.21.15.115:0/2306875992 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 4) v1 ==== 43232+0+0 (unknown 913048011 0 0) 0x7f51e800fd60 con 0x7f51f4069830 2024-01-24T23:21:09.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.087+0000 7f51f17fa640 1 -- 172.21.15.115:0/2306875992 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(14..14 src has 1..14) v4 ==== 3506+0+0 (unknown 2934162782 0 0) 0x7f51e80434b0 con 0x7f51f4069830 2024-01-24T23:21:09.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.087+0000 7f51f3fff640 10 monclient: get_auth_request con 0x7f51d80376d0 auth_method 0 2024-01-24T23:21:09.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.159+0000 7f51f17fa640 1 -- 172.21.15.115:0/2306875992 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f51e8014aa0 con 0x7f51f4069830 2024-01-24T23:21:09.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.159+0000 7f51f17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:09.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.159+0000 7f51f17fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:09.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.363+0000 7f51faf4b640 10 start_mon_command cmd=[{"prefix": "mgr dump", "format": "json"}] 2024-01-24T23:21:09.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.363+0000 7f51faf4b640 10 monclient: _send_command 2 [{"prefix": "mgr dump", "format": "json"}] 2024-01-24T23:21:09.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.363+0000 7f51faf4b640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:09.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.363+0000 7f51faf4b640 1 -- 172.21.15.115:0/2306875992 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f51f406a040 con 0x7f51f4069830 2024-01-24T23:21:09.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.367+0000 7f51f17fa640 1 -- 172.21.15.115:0/2306875992 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+82860 (unknown 3797009233 0 615716155) 0x7f51e8014690 con 0x7f51f4069830 2024-01-24T23:21:09.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.367+0000 7f51f17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mgr dump", "format": "json"}] 2024-01-24T23:21:09.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.367+0000 7f51f17fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:21:09.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.371+0000 7f51faf4b640 1 -- 172.21.15.115:0/2306875992 >> v1:172.21.15.132:6812/1835401922 conn(0x7f51d80376d0 legacy=0x7f51d8039b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:09.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.371+0000 7f51faf4b640 10 monclient: shutdown 2024-01-24T23:21:09.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.371+0000 7f51faf4b640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:09.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.371+0000 7f51faf4b640 1 -- 172.21.15.115:0/2306875992 >> v1:172.21.15.115:6792/0 conn(0x7f51f4069830 legacy=0x7f51f41271e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:09.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.371+0000 7f51faf4b640 1 -- 172.21.15.115:0/2306875992 shutdown_connections 2024-01-24T23:21:09.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.371+0000 7f51faf4b640 1 -- 172.21.15.115:0/2306875992 >> 172.21.15.115:0/2306875992 conn(0x7f51f40a2bf0 msgr2=0x7f51f40a1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:09.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.371+0000 7f51faf4b640 1 -- 172.21.15.115:0/2306875992 shutdown_connections 2024-01-24T23:21:09.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.371+0000 7f51faf4b640 1 -- 172.21.15.115:0/2306875992 wait complete. 2024-01-24T23:21:09.390 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":4,"active_gid":4113,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6812","nonce":1835401922}]},"active_addr":"172.21.15.132:6812/1835401922","active_change":"2024-01-24T23:21:01.813132+0000","active_mgr_features":4540701272860131327,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.132:0","nonce":4269658728}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.132:0","nonce":1230418325}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.132:0","nonce":1615890227}]}]} 2024-01-24T23:21:09.395 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-01-24T23:21:09.395 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-01-24T23:21:09.395 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T23:21:09.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa6bfdb640 10 monclient: get_monmap_and_config 2024-01-24T23:21:09.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa6bfdb640 10 monclient: build_initial_monmap 2024-01-24T23:21:09.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa6bfdb640 10 monclient: monmap: 2024-01-24T23:21:09.507 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:09.507 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:09.507 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:09.508536+0000 2024-01-24T23:21:09.508 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:09.508536+0000 2024-01-24T23:21:09.508 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:09.508 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:09.508 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:09.508 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:09.508 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:09.508 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:09.508 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:09.508 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:09.508 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:09.508 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:09.509 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:09.509 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:09.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa6bfdb640 1 Processor -- start 2024-01-24T23:21:09.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa6bfdb640 1 -- start start 2024-01-24T23:21:09.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa6bfdb640 10 monclient: init 2024-01-24T23:21:09.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa6bfdb640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:09.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa6bfdb640 10 monclient: _add_conns ranks=[5,6,0,7,2,8,3,4,1] 2024-01-24T23:21:09.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa6bfdb640 10 monclient(hunting): picked mon.noname-h con 0x7faa64133a80 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:09.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa6bfdb640 10 monclient(hunting): picked mon.noname-d con 0x7faa64133e60 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:09.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa6bfdb640 10 monclient(hunting): picked mon.noname-a con 0x7faa64134270 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:09.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa6bfdb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa6404ed70 con 0x7faa64134270 2024-01-24T23:21:09.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa6bfdb640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa6409c5a0 con 0x7faa64133a80 2024-01-24T23:21:09.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa6bfdb640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa6409ae60 con 0x7faa64133e60 2024-01-24T23:21:09.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa6bfdb640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:09.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa6bfdb640 10 monclient(hunting): authenticate will time out at 948.431043s 2024-01-24T23:21:09.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa6a578640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7faa64134270 0x7faa6413c110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:52082/0 (socket says 172.21.15.115:52082) 2024-01-24T23:21:09.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa6a578640 1 -- 172.21.15.115:0/1455259356 learned_addr learned my addr 172.21.15.115:0/1455259356 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:09.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa69576640 1 --1- 172.21.15.115:0/1455259356 >> v1:172.21.15.115:6792/0 conn(0x7faa64133e60 0x7faa641389e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:56866/0 (socket says 172.21.15.115:56866) 2024-01-24T23:21:09.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 1 -- 172.21.15.115:0/1455259356 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1834825816 0 0) 0x7faa6404ed70 con 0x7faa64134270 2024-01-24T23:21:09.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 10 monclient(hunting): _negotiate 2024-01-24T23:21:09.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:09.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:09.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 10 monclient(hunting): my global_id is 4275 2024-01-24T23:21:09.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 1 -- 172.21.15.115:0/1455259356 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa50005740 con 0x7faa64134270 2024-01-24T23:21:09.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 1 -- 172.21.15.115:0/1455259356 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3601020552 0 0) 0x7faa6409ae60 con 0x7faa64133e60 2024-01-24T23:21:09.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 10 monclient(hunting): _negotiate 2024-01-24T23:21:09.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:09.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:09.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 10 monclient(hunting): my global_id is 4272 2024-01-24T23:21:09.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 1 -- 172.21.15.115:0/1455259356 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa6404ed70 con 0x7faa64133e60 2024-01-24T23:21:09.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 1 -- 172.21.15.115:0/1455259356 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1592951756 0 0) 0x7faa50005740 con 0x7faa64134270 2024-01-24T23:21:09.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 1 -- 172.21.15.115:0/1455259356 >> v1:172.21.15.115:6792/0 conn(0x7faa64133e60 legacy=0x7faa641389e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:09.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 1 -- 172.21.15.115:0/1455259356 >> v1:172.21.15.132:6791/0 conn(0x7faa64133a80 legacy=0x7faa6409db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:09.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 10 monclient: _finish_hunting 0 2024-01-24T23:21:09.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 1 monclient: found mon.noname-a 2024-01-24T23:21:09.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:09.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:21:09.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 1 -- 172.21.15.115:0/1455259356 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa640a33a0 con 0x7faa64134270 2024-01-24T23:21:09.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 10 monclient: _finish_auth 0 2024-01-24T23:21:09.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 10 monclient: _check_auth_tickets 2024-01-24T23:21:09.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:09.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 1 -- 172.21.15.115:0/1455259356 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faa5c001a70 con 0x7faa64134270 2024-01-24T23:21:09.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:09.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:21:09.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa68d75640 10 monclient: dump: 2024-01-24T23:21:09.519 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:09.519 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:09.519 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.519 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.519 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:09.519 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:09.519 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:09.519 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:09.519 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:09.519 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:09.519 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:09.520 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:09.520 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:09.520 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:09.520 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:09.520 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:09.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa6bfdb640 5 monclient: authenticate success, global_id 4275 2024-01-24T23:21:09.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.503+0000 7faa6bfdb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:09.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa68d75640 1 -- 172.21.15.115:0/1455259356 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faa5c001c90 con 0x7faa64134270 2024-01-24T23:21:09.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa68d75640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:09.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:09.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa68d75640 1 -- 172.21.15.115:0/1455259356 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faa5c004c30 con 0x7faa64134270 2024-01-24T23:21:09.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 4 set_mon_vals no callback set 2024-01-24T23:21:09.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa68d75640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:09.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa68d75640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:09.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa68d75640 10 monclient: dump: 2024-01-24T23:21:09.521 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:09.521 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:09.521 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.521 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.521 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:09.521 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:09.521 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:09.521 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:09.521 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:09.521 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:09.521 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:09.521 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:09.521 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:09.521 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:09.521 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:09.522 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:09.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 10 monclient: shutdown 2024-01-24T23:21:09.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:09.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 1 -- 172.21.15.115:0/1455259356 >> v1:172.21.15.115:6789/0 conn(0x7faa64134270 legacy=0x7faa6413c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:09.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 1 -- 172.21.15.115:0/1455259356 shutdown_connections 2024-01-24T23:21:09.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 1 -- 172.21.15.115:0/1455259356 >> 172.21.15.115:0/1455259356 conn(0x7faa640a6500 msgr2=0x7faa640a2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:09.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 1 -- 172.21.15.115:0/1455259356 shutdown_connections 2024-01-24T23:21:09.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 1 -- 172.21.15.115:0/1455259356 wait complete. 2024-01-24T23:21:09.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 10 monclient: build_initial_monmap 2024-01-24T23:21:09.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 10 monclient: monmap: 2024-01-24T23:21:09.522 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:09.522 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:09.522 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:09.522 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:09.523 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:09.523 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:09.523 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:09.523 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:09.523 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:09.523 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:09.523 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:09.523 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:09.523 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:09.523 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:09.523 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:09.523 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:09.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 1 Processor -- start 2024-01-24T23:21:09.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 1 -- start start 2024-01-24T23:21:09.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 10 monclient: init 2024-01-24T23:21:09.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 10 monclient: _add_conns ranks=[3,0,6,1,7,2,5,8,4] 2024-01-24T23:21:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 10 monclient(hunting): picked mon.noname-g con 0x7faa64133a80 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 10 monclient(hunting): picked mon.noname-a con 0x7faa64133e60 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 10 monclient(hunting): picked mon.noname-d con 0x7faa64134270 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa6409b940 con 0x7faa64133e60 2024-01-24T23:21:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa64072140 con 0x7faa64133a80 2024-01-24T23:21:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa6411e1a0 con 0x7faa64134270 2024-01-24T23:21:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 10 monclient(hunting): authenticate will time out at 948.434104s 2024-01-24T23:21:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6a578640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7faa64134270 0x7faa641cf750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:56882/0 (socket says 172.21.15.115:56882) 2024-01-24T23:21:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6a578640 1 -- 172.21.15.115:0/1743995607 learned_addr learned my addr 172.21.15.115:0/1743995607 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 1 -- 172.21.15.115:0/1743995607 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3923578602 0 0) 0x7faa6411e1a0 con 0x7faa64134270 2024-01-24T23:21:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:09.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:09.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient(hunting): my global_id is 4281 2024-01-24T23:21:09.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 1 -- 172.21.15.115:0/1743995607 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa48003530 con 0x7faa64134270 2024-01-24T23:21:09.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 1 -- 172.21.15.115:0/1743995607 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1853022068 0 0) 0x7faa6409b940 con 0x7faa64133e60 2024-01-24T23:21:09.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:09.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:09.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:09.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient(hunting): my global_id is 4284 2024-01-24T23:21:09.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 1 -- 172.21.15.115:0/1743995607 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa6411e1a0 con 0x7faa64133e60 2024-01-24T23:21:09.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 1 -- 172.21.15.115:0/1743995607 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1947213379 0 0) 0x7faa48003530 con 0x7faa64134270 2024-01-24T23:21:09.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 1 -- 172.21.15.115:0/1743995607 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faa6409b940 con 0x7faa64134270 2024-01-24T23:21:09.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 1 -- 172.21.15.115:0/1743995607 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faa5c001970 con 0x7faa64134270 2024-01-24T23:21:09.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:09.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:21:09.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient(hunting): dump: 2024-01-24T23:21:09.526 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:09.526 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:09.526 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.526 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.526 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:09.526 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:09.526 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:09.526 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:09.526 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:09.526 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:09.526 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:09.526 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:09.526 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:09.526 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:09.527 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:09.527 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:09.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:09.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 1 -- 172.21.15.115:0/1743995607 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 843109902 0 0) 0x7faa6411e1a0 con 0x7faa64133e60 2024-01-24T23:21:09.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 1 -- 172.21.15.115:0/1743995607 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faa48003530 con 0x7faa64133e60 2024-01-24T23:21:09.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 1 -- 172.21.15.115:0/1743995607 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888828798 0 0) 0x7faa64072140 con 0x7faa64133a80 2024-01-24T23:21:09.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:09.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:09.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:09.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient(hunting): my global_id is 4224 2024-01-24T23:21:09.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 1 -- 172.21.15.115:0/1743995607 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa6411e1a0 con 0x7faa64133a80 2024-01-24T23:21:09.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 1 -- 172.21.15.115:0/1743995607 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faa54001560 con 0x7faa64133e60 2024-01-24T23:21:09.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:09.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:09.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient(hunting): dump: 2024-01-24T23:21:09.528 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:09.528 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:09.528 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.528 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.528 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:09.528 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:09.528 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:09.528 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:09.528 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:09.528 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:09.528 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:09.528 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:09.529 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:09.529 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:09.529 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:09.529 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:09.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 1 -- 172.21.15.115:0/1743995607 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3771802541 0 0) 0x7faa6409b940 con 0x7faa64134270 2024-01-24T23:21:09.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 1 -- 172.21.15.115:0/1743995607 >> v1:172.21.15.132:6790/0 conn(0x7faa64133a80 legacy=0x7faa64132ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:09.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 1 -- 172.21.15.115:0/1743995607 >> v1:172.21.15.115:6789/0 conn(0x7faa64133e60 legacy=0x7faa641cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:09.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:21:09.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 1 monclient: found mon.d 2024-01-24T23:21:09.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:09.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:09.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 1 -- 172.21.15.115:0/1743995607 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6411fbf0 con 0x7faa64134270 2024-01-24T23:21:09.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient: _finish_auth 0 2024-01-24T23:21:09.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:21:09.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:09.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 5 monclient: authenticate success, global_id 4281 2024-01-24T23:21:09.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 10 monclient: _renew_subs 2024-01-24T23:21:09.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:09.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 1 -- 172.21.15.115:0/1743995607 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa6411ed00 con 0x7faa64134270 2024-01-24T23:21:09.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 10 monclient: _renew_subs 2024-01-24T23:21:09.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:09.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 1 -- 172.21.15.115:0/1743995607 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7faa641333b0 con 0x7faa64134270 2024-01-24T23:21:09.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 1 -- 172.21.15.115:0/1743995607 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faa5c004e20 con 0x7faa64134270 2024-01-24T23:21:09.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:09.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5bfff640 4 set_mon_vals no callback set 2024-01-24T23:21:09.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 1 -- 172.21.15.115:0/1743995607 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faa5c005350 con 0x7faa64134270 2024-01-24T23:21:09.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:09.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:09.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 10 monclient: dump: 2024-01-24T23:21:09.531 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:09.531 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:09.531 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.531 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.531 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:09.531 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:09.531 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:09.531 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:09.531 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:09.531 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:09.531 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:09.531 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:09.532 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:09.532 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:09.532 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:09.532 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:09.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:09.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:09.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:09.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa6bfdb640 1 -- 172.21.15.115:0/1743995607 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa4c005090 con 0x7faa64134270 2024-01-24T23:21:09.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.507+0000 7faa5a7fc640 1 -- 172.21.15.115:0/1743995607 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 4) v1 ==== 43232+0+0 (unknown 913048011 0 0) 0x7faa5c005530 con 0x7faa64134270 2024-01-24T23:21:09.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.511+0000 7faa5a7fc640 1 -- 172.21.15.115:0/1743995607 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(14..14 src has 1..14) v4 ==== 3506+0+0 (unknown 2934162782 0 0) 0x7faa5c043a60 con 0x7faa64134270 2024-01-24T23:21:09.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.515+0000 7faa5a7fc640 1 -- 172.21.15.115:0/1743995607 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7faa5c014d40 con 0x7faa64134270 2024-01-24T23:21:09.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.515+0000 7faa5a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:09.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.515+0000 7faa5a7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:09.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.527+0000 7faa69d77640 10 monclient: get_auth_request con 0x7faa48037740 auth_method 0 2024-01-24T23:21:09.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.679+0000 7faa6bfdb640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-01-24T23:21:09.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.679+0000 7faa6bfdb640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-01-24T23:21:09.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.679+0000 7faa6bfdb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:09.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.679+0000 7faa6bfdb640 1 -- 172.21.15.115:0/1743995607 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa4c005340 con 0x7faa64134270 2024-01-24T23:21:09.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.683+0000 7faa5a7fc640 1 -- 172.21.15.115:0/1743995607 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10790 (unknown 92231148 0 58302584) 0x7faa5c0133e0 con 0x7faa64134270 2024-01-24T23:21:09.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.683+0000 7faa5a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-01-24T23:21:09.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.683+0000 7faa5a7fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:09.687 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:21:09.687 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":14,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","created":"2024-01-24T23:20:56.917703+0000","modified":"2024-01-24T23:21:08.921603+0000","last_up_change":"2024-01-24T23:21:05.857369+0000","last_in_change":"2024-01-24T23:20:58.690771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-01-24T23:21:07.002704+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-01-24T23:21:07.902769+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"902f4a51-85e9-4af8-b11f-e42a51b31723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6808","nonce":3064328804}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6809","nonce":3064328804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6811","nonce":3064328804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6810","nonce":3064328804}]},"public_addr":"172.21.15.115:6808/3064328804","cluster_addr":"172.21.15.115:6809/3064328804","heartbeat_back_addr":"172.21.15.115:6811/3064328804","heartbeat_front_addr":"172.21.15.115:6810/3064328804","state":["exists","up"]},{"osd":1,"uuid":"e06b92ed-3ab9-4103-8e41-4fa60aea2829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6801","nonce":3052044493}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6803","nonce":3052044493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6807","nonce":3052044493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6806","nonce":3052044493}]},"public_addr":"172.21.15.115:6801/3052044493","cluster_addr":"172.21.15.115:6803/3052044493","heartbeat_back_addr":"172.21.15.115:6807/3052044493","heartbeat_front_addr":"172.21.15.115:6806/3052044493","state":["exists","up"]},{"osd":2,"uuid":"013044c9-553c-4dd9-8438-c9873813e5af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6800","nonce":2776240662}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6802","nonce":2776240662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6805","nonce":2776240662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6804","nonce":2776240662}]},"public_addr":"172.21.15.115:6800/2776240662","cluster_addr":"172.21.15.115:6802/2776240662","heartbeat_back_addr":"172.21.15.115:6805/2776240662","heartbeat_front_addr":"172.21.15.115:6804/2776240662","state":["exists","up"]},{"osd":3,"uuid":"5fcff598-1939-415b-b055-57940893269e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6800","nonce":1048681489}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6801","nonce":1048681489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6804","nonce":1048681489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6802","nonce":1048681489}]},"public_addr":"172.21.15.132:6800/1048681489","cluster_addr":"172.21.15.132:6801/1048681489","heartbeat_back_addr":"172.21.15.132:6804/1048681489","heartbeat_front_addr":"172.21.15.132:6802/1048681489","state":["exists","up"]},{"osd":4,"uuid":"ef67ccd9-a21c-4a40-99ff-93f5b10a1fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6807","nonce":3601895184}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6809","nonce":3601895184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6811","nonce":3601895184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6810","nonce":3601895184}]},"public_addr":"172.21.15.132:6807/3601895184","cluster_addr":"172.21.15.132:6809/3601895184","heartbeat_back_addr":"172.21.15.132:6811/3601895184","heartbeat_front_addr":"172.21.15.132:6810/3601895184","state":["exists","up"]},{"osd":5,"uuid":"c2573ce8-6dd2-4272-85c7-2f5aa25eb2f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6803","nonce":3365641231}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6805","nonce":3365641231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6808","nonce":3365641231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6806","nonce":3365641231}]},"public_addr":"172.21.15.132:6803/3365641231","cluster_addr":"172.21.15.132:6805/3365641231","heartbeat_back_addr":"172.21.15.132:6808/3365641231","heartbeat_front_addr":"172.21.15.132:6806/3365641231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T23:21:04.184180+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T23:21:04.023173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T23:21:03.933732+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T23:21:04.204962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T23:21:04.208706+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T23:21:04.085169+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T23:21:09.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.683+0000 7faa6bfdb640 1 -- 172.21.15.115:0/1743995607 >> v1:172.21.15.132:6812/1835401922 conn(0x7faa48037740 legacy=0x7faa48039c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:09.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.683+0000 7faa6bfdb640 10 monclient: shutdown 2024-01-24T23:21:09.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.683+0000 7faa6bfdb640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:09.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.683+0000 7faa6bfdb640 1 -- 172.21.15.115:0/1743995607 >> v1:172.21.15.115:6792/0 conn(0x7faa64134270 legacy=0x7faa641cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:09.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.683+0000 7faa6bfdb640 1 -- 172.21.15.115:0/1743995607 shutdown_connections 2024-01-24T23:21:09.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.683+0000 7faa6bfdb640 1 -- 172.21.15.115:0/1743995607 >> 172.21.15.115:0/1743995607 conn(0x7faa640a6500 msgr2=0x7faa640a41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:09.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.683+0000 7faa6bfdb640 1 -- 172.21.15.115:0/1743995607 shutdown_connections 2024-01-24T23:21:09.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.683+0000 7faa6bfdb640 1 -- 172.21.15.115:0/1743995607 wait complete. 2024-01-24T23:21:09.705 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-01-24T23:21:09.705 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T23:21:09.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.819+0000 7f42a5f08640 10 monclient: get_monmap_and_config 2024-01-24T23:21:09.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.819+0000 7f42a5f08640 10 monclient: build_initial_monmap 2024-01-24T23:21:09.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.819+0000 7f42a5f08640 10 monclient: monmap: 2024-01-24T23:21:09.824 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:09.824 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:09.824 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:09.825107+0000 2024-01-24T23:21:09.824 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:09.825107+0000 2024-01-24T23:21:09.824 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:09.824 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:09.824 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:09.824 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:09.824 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:09.825 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:09.825 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:09.825 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:09.825 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:09.825 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:09.825 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:09.825 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:09.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.819+0000 7f42a5f08640 1 Processor -- start 2024-01-24T23:21:09.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.819+0000 7f42a5f08640 1 -- start start 2024-01-24T23:21:09.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.819+0000 7f42a5f08640 10 monclient: init 2024-01-24T23:21:09.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.819+0000 7f42a5f08640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:09.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.819+0000 7f42a5f08640 10 monclient: _add_conns ranks=[2,3,4,7,0,6,1,8,5] 2024-01-24T23:21:09.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.819+0000 7f42a5f08640 10 monclient(hunting): picked mon.noname-b con 0x7f42a0133bf0 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:09.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 10 monclient(hunting): picked mon.noname-g con 0x7f42a0133fd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:09.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 10 monclient(hunting): picked mon.noname-c con 0x7f42a0134430 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:09.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a004ed70 con 0x7f42a0133bf0 2024-01-24T23:21:09.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a009af30 con 0x7f42a0133fd0 2024-01-24T23:21:09.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a009b210 con 0x7f42a0134430 2024-01-24T23:21:09.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:09.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 10 monclient(hunting): authenticate will time out at 948.748104s 2024-01-24T23:21:09.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429f7fe640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f42a0133bf0 0x7f42a009bd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:54498/0 (socket says 172.21.15.115:54498) 2024-01-24T23:21:09.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429f7fe640 1 -- 172.21.15.115:0/649318154 learned_addr learned my addr 172.21.15.115:0/649318154 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:09.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 1 -- 172.21.15.115:0/649318154 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251461870 0 0) 0x7f42a004ed70 con 0x7f42a0133bf0 2024-01-24T23:21:09.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:09.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:09.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:09.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 10 monclient(hunting): my global_id is 4232 2024-01-24T23:21:09.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 1 -- 172.21.15.115:0/649318154 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4288003530 con 0x7f42a0133bf0 2024-01-24T23:21:09.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 1 -- 172.21.15.115:0/649318154 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181873349 0 0) 0x7f42a009b210 con 0x7f42a0134430 2024-01-24T23:21:09.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:09.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:09.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:09.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 10 monclient(hunting): my global_id is 74254 2024-01-24T23:21:09.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 1 -- 172.21.15.115:0/649318154 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42a004ed70 con 0x7f42a0134430 2024-01-24T23:21:09.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 1 -- 172.21.15.115:0/649318154 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3999299602 0 0) 0x7f4288003530 con 0x7f42a0133bf0 2024-01-24T23:21:09.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 1 -- 172.21.15.115:0/649318154 >> v1:172.21.15.115:6791/0 conn(0x7f42a0134430 legacy=0x7f42a013f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:09.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 1 -- 172.21.15.115:0/649318154 >> v1:172.21.15.132:6790/0 conn(0x7f42a0133fd0 legacy=0x7f42a013bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:09.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:21:09.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 1 monclient: found mon.noname-b 2024-01-24T23:21:09.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:09.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:21:09.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 1 -- 172.21.15.115:0/649318154 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a009b4f0 con 0x7f42a0133bf0 2024-01-24T23:21:09.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 10 monclient: _finish_auth 0 2024-01-24T23:21:09.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:21:09.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:09.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 1 -- 172.21.15.115:0/649318154 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f42940017c0 con 0x7f42a0133bf0 2024-01-24T23:21:09.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:09.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:21:09.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 10 monclient: dump: 2024-01-24T23:21:09.835 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:09.836 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:09.836 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.836 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.836 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:09.836 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:09.836 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:09.836 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:09.836 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:09.836 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:09.836 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:09.836 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:09.836 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:09.836 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:09.836 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:09.837 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:09.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 5 monclient: authenticate success, global_id 4232 2024-01-24T23:21:09.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:09.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 1 -- 172.21.15.115:0/649318154 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4294001a20 con 0x7f42a0133bf0 2024-01-24T23:21:09.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:09.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:09.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 4 set_mon_vals no callback set 2024-01-24T23:21:09.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 1 -- 172.21.15.115:0/649318154 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4294004bf0 con 0x7f42a0133bf0 2024-01-24T23:21:09.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:09.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:09.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429e7fc640 10 monclient: dump: 2024-01-24T23:21:09.837 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:09.837 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:09.837 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.838 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.838 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:09.838 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:09.838 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:09.838 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:09.838 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:09.838 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:09.838 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:09.838 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:09.838 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:09.838 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:09.838 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:09.838 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:09.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 10 monclient: shutdown 2024-01-24T23:21:09.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:09.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 1 -- 172.21.15.115:0/649318154 >> v1:172.21.15.115:6790/0 conn(0x7f42a0133bf0 legacy=0x7f42a009bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:09.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 1 -- 172.21.15.115:0/649318154 shutdown_connections 2024-01-24T23:21:09.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 1 -- 172.21.15.115:0/649318154 >> 172.21.15.115:0/649318154 conn(0x7f42a00a9910 msgr2=0x7f42a00a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:09.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 1 -- 172.21.15.115:0/649318154 shutdown_connections 2024-01-24T23:21:09.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 1 -- 172.21.15.115:0/649318154 wait complete. 2024-01-24T23:21:09.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 10 monclient: build_initial_monmap 2024-01-24T23:21:09.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 10 monclient: monmap: 2024-01-24T23:21:09.839 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:09.839 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:09.839 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:09.839 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:09.839 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:09.839 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:09.839 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:09.839 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:09.839 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:09.840 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:09.840 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:09.840 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:09.840 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:09.840 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:09.840 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:09.840 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:09.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 1 Processor -- start 2024-01-24T23:21:09.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 1 -- start start 2024-01-24T23:21:09.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 10 monclient: init 2024-01-24T23:21:09.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:09.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 10 monclient: _add_conns ranks=[7,6,3,1,0,8,4,2,5] 2024-01-24T23:21:09.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 10 monclient(hunting): picked mon.noname-i con 0x7f42a0133bf0 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:09.840 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 10 monclient(hunting): picked mon.noname-d con 0x7f42a0133fd0 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:09.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 10 monclient(hunting): picked mon.noname-g con 0x7f42a0134430 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:09.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a013ae60 con 0x7f42a0134430 2024-01-24T23:21:09.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a01389c0 con 0x7f42a0133fd0 2024-01-24T23:21:09.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a009c8f0 con 0x7f42a0133bf0 2024-01-24T23:21:09.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:09.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f42a5f08640 10 monclient(hunting): authenticate will time out at 948.751310s 2024-01-24T23:21:09.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429effd640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f42a0133fd0 0x7f42a01382b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:56892/0 (socket says 172.21.15.115:56892) 2024-01-24T23:21:09.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f429effd640 1 -- 172.21.15.115:0/3015498659 learned_addr learned my addr 172.21.15.115:0/3015498659 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:09.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f4287fff640 1 -- 172.21.15.115:0/3015498659 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1480242254 0 0) 0x7f42a01389c0 con 0x7f42a0133fd0 2024-01-24T23:21:09.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f4287fff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:09.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f4287fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:09.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f4287fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:09.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f4287fff640 10 monclient(hunting): my global_id is 4290 2024-01-24T23:21:09.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f4287fff640 1 -- 172.21.15.115:0/3015498659 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f428c003530 con 0x7f42a0133fd0 2024-01-24T23:21:09.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f4287fff640 1 -- 172.21.15.115:0/3015498659 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 314339846 0 0) 0x7f42a009c8f0 con 0x7f42a0133bf0 2024-01-24T23:21:09.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f4287fff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:09.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f4287fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:09.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f4287fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:09.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f4287fff640 10 monclient(hunting): my global_id is 4255 2024-01-24T23:21:09.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f4287fff640 1 -- 172.21.15.115:0/3015498659 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42a01389c0 con 0x7f42a0133bf0 2024-01-24T23:21:09.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f4287fff640 1 -- 172.21.15.115:0/3015498659 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1207459914 0 0) 0x7f42a013ae60 con 0x7f42a0134430 2024-01-24T23:21:09.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f4287fff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:09.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f4287fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:09.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f4287fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:09.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f4287fff640 10 monclient(hunting): my global_id is 4242 2024-01-24T23:21:09.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f4287fff640 1 -- 172.21.15.115:0/3015498659 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42a009c8f0 con 0x7f42a0134430 2024-01-24T23:21:09.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.823+0000 7f4287fff640 1 -- 172.21.15.115:0/3015498659 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2481262211 0 0) 0x7f428c003530 con 0x7f42a0133fd0 2024-01-24T23:21:09.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 1 -- 172.21.15.115:0/3015498659 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f42a013ae60 con 0x7f42a0133fd0 2024-01-24T23:21:09.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 1 -- 172.21.15.115:0/3015498659 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4290001a60 con 0x7f42a0133fd0 2024-01-24T23:21:09.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:09.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:21:09.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 10 monclient(hunting): dump: 2024-01-24T23:21:09.843 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:09.843 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:09.843 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.843 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.843 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:09.843 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:09.843 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:09.843 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:09.843 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:09.843 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:09.844 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:09.844 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:09.844 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:09.844 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:09.844 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:09.844 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:09.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:09.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 1 -- 172.21.15.115:0/3015498659 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3730858469 0 0) 0x7f42a01389c0 con 0x7f42a0133bf0 2024-01-24T23:21:09.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 1 -- 172.21.15.115:0/3015498659 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f428c003530 con 0x7f42a0133bf0 2024-01-24T23:21:09.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 1 -- 172.21.15.115:0/3015498659 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4294001780 con 0x7f42a0133bf0 2024-01-24T23:21:09.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:09.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-01-24T23:21:09.844 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 10 monclient(hunting): dump: 2024-01-24T23:21:09.844 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:09.845 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:09.845 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.845 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.845 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:09.845 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:09.845 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:09.845 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:09.845 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:09.845 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:09.845 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:09.845 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:09.845 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:09.845 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:09.845 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:09.845 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:09.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 1 -- 172.21.15.115:0/3015498659 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1983291422 0 0) 0x7f42a013ae60 con 0x7f42a0133fd0 2024-01-24T23:21:09.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 1 -- 172.21.15.115:0/3015498659 >> v1:172.21.15.132:6792/0 conn(0x7f42a0133bf0 legacy=0x7f42a0137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:09.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 1 -- 172.21.15.115:0/3015498659 >> v1:172.21.15.132:6790/0 conn(0x7f42a0134430 legacy=0x7f42a01d3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:09.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 10 monclient: _finish_hunting 0 2024-01-24T23:21:09.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 1 monclient: found mon.d 2024-01-24T23:21:09.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:09.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:09.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 1 -- 172.21.15.115:0/3015498659 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a00a5960 con 0x7f42a0133fd0 2024-01-24T23:21:09.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 10 monclient: _finish_auth 0 2024-01-24T23:21:09.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 10 monclient: _check_auth_tickets 2024-01-24T23:21:09.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:09.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f42a5f08640 5 monclient: authenticate success, global_id 4290 2024-01-24T23:21:09.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f42a5f08640 10 monclient: _renew_subs 2024-01-24T23:21:09.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f42a5f08640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:09.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f42a5f08640 1 -- 172.21.15.115:0/3015498659 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a0139420 con 0x7f42a0133fd0 2024-01-24T23:21:09.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f42a5f08640 10 monclient: _renew_subs 2024-01-24T23:21:09.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f42a5f08640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:09.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f42a5f08640 1 -- 172.21.15.115:0/3015498659 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a013f740 con 0x7f42a0133fd0 2024-01-24T23:21:09.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 1 -- 172.21.15.115:0/3015498659 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f42900038d0 con 0x7f42a0133fd0 2024-01-24T23:21:09.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:09.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 1 -- 172.21.15.115:0/3015498659 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4290003e00 con 0x7f42a0133fd0 2024-01-24T23:21:09.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:09.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f429dffb640 4 set_mon_vals no callback set 2024-01-24T23:21:09.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:09.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 10 monclient: dump: 2024-01-24T23:21:09.847 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:09.847 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:09.847 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.847 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:09.848 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:09.848 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:09.848 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:09.848 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:09.848 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:09.848 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:09.848 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:09.848 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:09.848 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:09.848 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:09.848 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:09.848 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:09.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 1 -- 172.21.15.115:0/3015498659 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 4) v1 ==== 43232+0+0 (unknown 913048011 0 0) 0x7f4290003f60 con 0x7f42a0133fd0 2024-01-24T23:21:09.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f42a5f08640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:09.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f42a5f08640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:09.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f42a5f08640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:09.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f42a5f08640 1 -- 172.21.15.115:0/3015498659 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a013ac80 con 0x7f42a0133fd0 2024-01-24T23:21:09.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f4287fff640 1 -- 172.21.15.115:0/3015498659 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(14..14 src has 1..14) v4 ==== 3506+0+0 (unknown 2934162782 0 0) 0x7f4290043400 con 0x7f42a0133fd0 2024-01-24T23:21:09.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.827+0000 7f429f7fe640 10 monclient: get_auth_request con 0x7f428c037770 auth_method 0 2024-01-24T23:21:09.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.831+0000 7f4287fff640 1 -- 172.21.15.115:0/3015498659 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f4290014700 con 0x7f42a0133fd0 2024-01-24T23:21:09.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.831+0000 7f4287fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:09.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.831+0000 7f4287fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:09.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.991+0000 7f42a5f08640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-01-24T23:21:09.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.991+0000 7f42a5f08640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-01-24T23:21:09.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.991+0000 7f42a5f08640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:09.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:09.991+0000 7f42a5f08640 1 -- 172.21.15.115:0/3015498659 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42a00a5110 con 0x7f42a0133fd0 2024-01-24T23:21:10.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.007+0000 7f4287fff640 1 -- 172.21.15.115:0/3015498659 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10764 (unknown 4221056075 0 3147136755) 0x7f4290012da0 con 0x7f42a0133fd0 2024-01-24T23:21:10.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.007+0000 7f4287fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-01-24T23:21:10.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.007+0000 7f4287fff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:10.012 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:21:10.013 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":15,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","created":"2024-01-24T23:20:56.917703+0000","modified":"2024-01-24T23:21:09.926628+0000","last_up_change":"2024-01-24T23:21:05.857369+0000","last_in_change":"2024-01-24T23:20:58.690771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-01-24T23:21:07.002704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-01-24T23:21:07.902769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"902f4a51-85e9-4af8-b11f-e42a51b31723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6808","nonce":3064328804}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6809","nonce":3064328804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6811","nonce":3064328804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6810","nonce":3064328804}]},"public_addr":"172.21.15.115:6808/3064328804","cluster_addr":"172.21.15.115:6809/3064328804","heartbeat_back_addr":"172.21.15.115:6811/3064328804","heartbeat_front_addr":"172.21.15.115:6810/3064328804","state":["exists","up"]},{"osd":1,"uuid":"e06b92ed-3ab9-4103-8e41-4fa60aea2829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6801","nonce":3052044493}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6803","nonce":3052044493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6807","nonce":3052044493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6806","nonce":3052044493}]},"public_addr":"172.21.15.115:6801/3052044493","cluster_addr":"172.21.15.115:6803/3052044493","heartbeat_back_addr":"172.21.15.115:6807/3052044493","heartbeat_front_addr":"172.21.15.115:6806/3052044493","state":["exists","up"]},{"osd":2,"uuid":"013044c9-553c-4dd9-8438-c9873813e5af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6800","nonce":2776240662}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6802","nonce":2776240662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6805","nonce":2776240662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6804","nonce":2776240662}]},"public_addr":"172.21.15.115:6800/2776240662","cluster_addr":"172.21.15.115:6802/2776240662","heartbeat_back_addr":"172.21.15.115:6805/2776240662","heartbeat_front_addr":"172.21.15.115:6804/2776240662","state":["exists","up"]},{"osd":3,"uuid":"5fcff598-1939-415b-b055-57940893269e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6800","nonce":1048681489}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6801","nonce":1048681489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6804","nonce":1048681489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6802","nonce":1048681489}]},"public_addr":"172.21.15.132:6800/1048681489","cluster_addr":"172.21.15.132:6801/1048681489","heartbeat_back_addr":"172.21.15.132:6804/1048681489","heartbeat_front_addr":"172.21.15.132:6802/1048681489","state":["exists","up"]},{"osd":4,"uuid":"ef67ccd9-a21c-4a40-99ff-93f5b10a1fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6807","nonce":3601895184}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6809","nonce":3601895184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6811","nonce":3601895184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6810","nonce":3601895184}]},"public_addr":"172.21.15.132:6807/3601895184","cluster_addr":"172.21.15.132:6809/3601895184","heartbeat_back_addr":"172.21.15.132:6811/3601895184","heartbeat_front_addr":"172.21.15.132:6810/3601895184","state":["exists","up"]},{"osd":5,"uuid":"c2573ce8-6dd2-4272-85c7-2f5aa25eb2f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6803","nonce":3365641231}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6805","nonce":3365641231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6808","nonce":3365641231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6806","nonce":3365641231}]},"public_addr":"172.21.15.132:6803/3365641231","cluster_addr":"172.21.15.132:6805/3365641231","heartbeat_back_addr":"172.21.15.132:6808/3365641231","heartbeat_front_addr":"172.21.15.132:6806/3365641231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T23:21:04.184180+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T23:21:04.023173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T23:21:03.933732+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T23:21:04.204962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T23:21:04.208706+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T23:21:04.085169+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T23:21:10.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.011+0000 7f42a5f08640 1 -- 172.21.15.115:0/3015498659 >> v1:172.21.15.132:6812/1835401922 conn(0x7f428c037770 legacy=0x7f428c039c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.011+0000 7f42a5f08640 10 monclient: shutdown 2024-01-24T23:21:10.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.011+0000 7f42a5f08640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.011+0000 7f42a5f08640 1 -- 172.21.15.115:0/3015498659 >> v1:172.21.15.115:6792/0 conn(0x7f42a0133fd0 legacy=0x7f42a01382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.011+0000 7f42a5f08640 1 -- 172.21.15.115:0/3015498659 shutdown_connections 2024-01-24T23:21:10.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.011+0000 7f42a5f08640 1 -- 172.21.15.115:0/3015498659 >> 172.21.15.115:0/3015498659 conn(0x7f42a00a9910 msgr2=0x7f42a013fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.011+0000 7f42a5f08640 1 -- 172.21.15.115:0/3015498659 shutdown_connections 2024-01-24T23:21:10.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.011+0000 7f42a5f08640 1 -- 172.21.15.115:0/3015498659 wait complete. 2024-01-24T23:21:10.033 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-01-24T23:21:10.034 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-01-24T23:21:10.034 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-01-24T23:21:10.034 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-01-24T23:21:10.035 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-01-24T23:21:10.035 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-01-24T23:21:10.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.159+0000 7f88df9bc640 10 monclient: get_monmap_and_config 2024-01-24T23:21:10.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.159+0000 7f88df9bc640 10 monclient: build_initial_monmap 2024-01-24T23:21:10.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.159+0000 7f88df9bc640 10 monclient: monmap: 2024-01-24T23:21:10.166 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:10.166 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.166 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:10.166760+0000 2024-01-24T23:21:10.166 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:10.166760+0000 2024-01-24T23:21:10.166 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:10.166 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.166 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:10.166 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:10.166 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:10.166 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:10.166 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:10.167 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:10.167 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:10.167 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:10.167 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:10.167 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88df9bc640 1 Processor -- start 2024-01-24T23:21:10.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88df9bc640 1 -- start start 2024-01-24T23:21:10.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88df9bc640 10 monclient: init 2024-01-24T23:21:10.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88df9bc640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:10.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88df9bc640 10 monclient: _add_conns ranks=[1,2,5,6,4,3,0,7,8] 2024-01-24T23:21:10.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88df9bc640 10 monclient(hunting): picked mon.noname-f con 0x7f88d8133bf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:10.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88df9bc640 10 monclient(hunting): picked mon.noname-b con 0x7f88d8133fd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:10.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88df9bc640 10 monclient(hunting): picked mon.noname-h con 0x7f88d81343b0 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:10.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88df9bc640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d804ed70 con 0x7f88d8133bf0 2024-01-24T23:21:10.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88df9bc640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d80a4b60 con 0x7f88d8133fd0 2024-01-24T23:21:10.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88df9bc640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d809ae50 con 0x7f88d81343b0 2024-01-24T23:21:10.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88df9bc640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:10.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88df9bc640 10 monclient(hunting): authenticate will time out at 949.089771s 2024-01-24T23:21:10.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88dd758640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f88d8133bf0 0x7f88d809bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.115:35066/0 (socket says 172.21.15.115:35066) 2024-01-24T23:21:10.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88dd758640 1 -- 172.21.15.115:0/2608203819 learned_addr learned my addr 172.21.15.115:0/2608203819 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:10.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 1 -- 172.21.15.115:0/2608203819 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2555857741 0 0) 0x7f88d804ed70 con 0x7f88d8133bf0 2024-01-24T23:21:10.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 10 monclient(hunting): my global_id is 4222 2024-01-24T23:21:10.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 1 -- 172.21.15.115:0/2608203819 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88c4003530 con 0x7f88d8133bf0 2024-01-24T23:21:10.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 1 -- 172.21.15.115:0/2608203819 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3612067462 0 0) 0x7f88d809ae50 con 0x7f88d81343b0 2024-01-24T23:21:10.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 10 monclient(hunting): my global_id is 4253 2024-01-24T23:21:10.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 1 -- 172.21.15.115:0/2608203819 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88d804ed70 con 0x7f88d81343b0 2024-01-24T23:21:10.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 1 -- 172.21.15.115:0/2608203819 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2693113836 0 0) 0x7f88c4003530 con 0x7f88d8133bf0 2024-01-24T23:21:10.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 1 -- 172.21.15.115:0/2608203819 >> v1:172.21.15.132:6791/0 conn(0x7f88d81343b0 legacy=0x7f88d813f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 1 -- 172.21.15.115:0/2608203819 >> v1:172.21.15.115:6790/0 conn(0x7f88d8133fd0 legacy=0x7f88d813bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 10 monclient: _finish_hunting 0 2024-01-24T23:21:10.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 1 monclient: found mon.noname-f 2024-01-24T23:21:10.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:10.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:21:10.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 1 -- 172.21.15.115:0/2608203819 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d809b130 con 0x7f88d8133bf0 2024-01-24T23:21:10.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 10 monclient: _finish_auth 0 2024-01-24T23:21:10.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 10 monclient: _check_auth_tickets 2024-01-24T23:21:10.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:10.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 1 -- 172.21.15.115:0/2608203819 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f88c80017c0 con 0x7f88d8133bf0 2024-01-24T23:21:10.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:21:10.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 10 monclient: dump: 2024-01-24T23:21:10.172 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.172 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.173 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.173 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.173 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.173 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.173 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.173 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.174 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.174 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.174 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.174 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.174 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.174 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.174 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.174 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88df9bc640 5 monclient: authenticate success, global_id 4222 2024-01-24T23:21:10.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88df9bc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:10.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 1 -- 172.21.15.115:0/2608203819 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f88c8001a60 con 0x7f88d8133bf0 2024-01-24T23:21:10.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:10.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 1 -- 172.21.15.115:0/2608203819 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f88c8004bf0 con 0x7f88d8133bf0 2024-01-24T23:21:10.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88df9bc640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:10.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88df9bc640 4 set_mon_vals no callback set 2024-01-24T23:21:10.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:21:10.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.163+0000 7f88cffff640 10 monclient: dump: 2024-01-24T23:21:10.175 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.175 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.175 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.175 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.175 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.176 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.176 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.176 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.176 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.176 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.176 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.176 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.176 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.176 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.176 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.176 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88df9bc640 10 monclient: shutdown 2024-01-24T23:21:10.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88df9bc640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88df9bc640 1 -- 172.21.15.115:0/2608203819 >> v1:172.21.15.132:6789/0 conn(0x7f88d8133bf0 legacy=0x7f88d809bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88df9bc640 1 -- 172.21.15.115:0/2608203819 shutdown_connections 2024-01-24T23:21:10.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88df9bc640 1 -- 172.21.15.115:0/2608203819 >> 172.21.15.115:0/2608203819 conn(0x7f88d80662f0 msgr2=0x7f88d8066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88df9bc640 1 -- 172.21.15.115:0/2608203819 shutdown_connections 2024-01-24T23:21:10.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88df9bc640 1 -- 172.21.15.115:0/2608203819 wait complete. 2024-01-24T23:21:10.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88df9bc640 10 monclient: build_initial_monmap 2024-01-24T23:21:10.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88df9bc640 10 monclient: monmap: 2024-01-24T23:21:10.181 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:10.181 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:10.181 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:10.181 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:10.182 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:10.182 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:10.182 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:10.182 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:10.182 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:10.182 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:10.182 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:10.182 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:10.182 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:10.182 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:10.183 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:10.183 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88df9bc640 1 Processor -- start 2024-01-24T23:21:10.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88df9bc640 1 -- start start 2024-01-24T23:21:10.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88df9bc640 10 monclient: init 2024-01-24T23:21:10.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88df9bc640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:10.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88df9bc640 10 monclient: _add_conns ranks=[8,3,1,4,7,5,6,2,0] 2024-01-24T23:21:10.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88df9bc640 10 monclient(hunting): picked mon.noname-e con 0x7f88d8133bf0 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:10.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88df9bc640 10 monclient(hunting): picked mon.noname-g con 0x7f88d8133fd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:10.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88df9bc640 10 monclient(hunting): picked mon.noname-f con 0x7f88d81343b0 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:10.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88df9bc640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d80a25e0 con 0x7f88d81343b0 2024-01-24T23:21:10.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88df9bc640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d8133720 con 0x7f88d8133fd0 2024-01-24T23:21:10.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88df9bc640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d80a5990 con 0x7f88d8133bf0 2024-01-24T23:21:10.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88df9bc640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:10.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88df9bc640 10 monclient(hunting): authenticate will time out at 949.094824s 2024-01-24T23:21:10.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88dd758640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f88d8133bf0 0x7f88d8133010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:48710/0 (socket says 172.21.15.115:48710) 2024-01-24T23:21:10.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88dd758640 1 -- 172.21.15.115:0/914561935 learned_addr learned my addr 172.21.15.115:0/914561935 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:10.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88cdffb640 1 -- 172.21.15.115:0/914561935 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1122170300 0 0) 0x7f88d80a5990 con 0x7f88d8133bf0 2024-01-24T23:21:10.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88cdffb640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88cdffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88cdffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88cdffb640 10 monclient(hunting): my global_id is 74177 2024-01-24T23:21:10.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88cdffb640 1 -- 172.21.15.115:0/914561935 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88bc003530 con 0x7f88d8133bf0 2024-01-24T23:21:10.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88cdffb640 1 -- 172.21.15.115:0/914561935 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3913776557 0 0) 0x7f88d8133720 con 0x7f88d8133fd0 2024-01-24T23:21:10.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88cdffb640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88cdffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88cdffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88cdffb640 10 monclient(hunting): my global_id is 4251 2024-01-24T23:21:10.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88cdffb640 1 -- 172.21.15.115:0/914561935 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88d80a5990 con 0x7f88d8133fd0 2024-01-24T23:21:10.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88cdffb640 1 -- 172.21.15.115:0/914561935 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2429076215 0 0) 0x7f88d80a25e0 con 0x7f88d81343b0 2024-01-24T23:21:10.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88cdffb640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88cdffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88cdffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88cdffb640 10 monclient(hunting): my global_id is 4231 2024-01-24T23:21:10.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.167+0000 7f88cdffb640 1 -- 172.21.15.115:0/914561935 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88d8133720 con 0x7f88d81343b0 2024-01-24T23:21:10.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 1 -- 172.21.15.115:0/914561935 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 895973153 0 0) 0x7f88d80a5990 con 0x7f88d8133fd0 2024-01-24T23:21:10.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 1 -- 172.21.15.115:0/914561935 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f88d80a25e0 con 0x7f88d8133fd0 2024-01-24T23:21:10.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 1 -- 172.21.15.115:0/914561935 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f88d4001a20 con 0x7f88d8133fd0 2024-01-24T23:21:10.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:21:10.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 10 monclient(hunting): dump: 2024-01-24T23:21:10.187 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.187 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.187 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.187 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.187 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.187 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.187 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.187 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.187 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.187 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.188 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.188 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.188 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.188 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.188 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.188 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:10.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 1 -- 172.21.15.115:0/914561935 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2730605695 0 0) 0x7f88d8133720 con 0x7f88d81343b0 2024-01-24T23:21:10.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 1 -- 172.21.15.115:0/914561935 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f88d80a5990 con 0x7f88d81343b0 2024-01-24T23:21:10.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 1 -- 172.21.15.115:0/914561935 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f88d0001d60 con 0x7f88d81343b0 2024-01-24T23:21:10.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:21:10.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 10 monclient(hunting): dump: 2024-01-24T23:21:10.189 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.189 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.189 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.189 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.189 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.189 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.189 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.190 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.190 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.190 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.190 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.190 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.190 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.190 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.190 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.190 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 1 -- 172.21.15.115:0/914561935 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 809996188 0 0) 0x7f88d80a25e0 con 0x7f88d8133fd0 2024-01-24T23:21:10.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 1 -- 172.21.15.115:0/914561935 >> v1:172.21.15.115:6793/0 conn(0x7f88d8133bf0 legacy=0x7f88d8133010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 1 -- 172.21.15.115:0/914561935 >> v1:172.21.15.132:6789/0 conn(0x7f88d81343b0 legacy=0x7f88d81d4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 10 monclient: _finish_hunting 0 2024-01-24T23:21:10.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 1 monclient: found mon.g 2024-01-24T23:21:10.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:10.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:21:10.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 1 -- 172.21.15.115:0/914561935 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d8120150 con 0x7f88d8133fd0 2024-01-24T23:21:10.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 10 monclient: _finish_auth 0 2024-01-24T23:21:10.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 10 monclient: _check_auth_tickets 2024-01-24T23:21:10.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:10.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88df9bc640 5 monclient: authenticate success, global_id 4251 2024-01-24T23:21:10.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88df9bc640 10 monclient: _renew_subs 2024-01-24T23:21:10.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88df9bc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:21:10.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88df9bc640 1 -- 172.21.15.115:0/914561935 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d8120430 con 0x7f88d8133fd0 2024-01-24T23:21:10.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88df9bc640 10 monclient: _renew_subs 2024-01-24T23:21:10.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88df9bc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:21:10.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88df9bc640 1 -- 172.21.15.115:0/914561935 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d811fd30 con 0x7f88d8133fd0 2024-01-24T23:21:10.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 1 -- 172.21.15.115:0/914561935 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f88d4001ce0 con 0x7f88d8133fd0 2024-01-24T23:21:10.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:10.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 1 -- 172.21.15.115:0/914561935 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f88d4003c10 con 0x7f88d8133fd0 2024-01-24T23:21:10.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:21:10.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cf7fe640 4 set_mon_vals no callback set 2024-01-24T23:21:10.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 10 monclient: dump: 2024-01-24T23:21:10.193 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.193 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.193 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.193 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.193 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.193 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.193 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.194 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.194 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.194 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.194 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.194 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.194 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.194 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.194 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.195 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 1 -- 172.21.15.115:0/914561935 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 4) v1 ==== 43232+0+0 (unknown 913048011 0 0) 0x7f88d4003d70 con 0x7f88d8133fd0 2024-01-24T23:21:10.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88cdffb640 1 -- 172.21.15.115:0/914561935 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f88d4042330 con 0x7f88d8133fd0 2024-01-24T23:21:10.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88c37fe640 1 -- 172.21.15.115:0/914561935 --> v1:172.21.15.115:6808/3064328804 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88d809f380 con 0x7f88d8060fd0 2024-01-24T23:21:10.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88dd758640 10 monclient: get_auth_request con 0x7f88bc037870 auth_method 0 2024-01-24T23:21:10.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.171+0000 7f88ddf59640 10 monclient: get_auth_request con 0x7f88d8060fd0 auth_method 0 2024-01-24T23:21:10.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.175+0000 7f88cdffb640 1 -- 172.21.15.115:0/914561935 <== osd.0 v1:172.21.15.115:6808/3064328804 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f88d809f380 con 0x7f88d8060fd0 2024-01-24T23:21:10.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.175+0000 7f43626ae640 10 monclient: get_monmap_and_config 2024-01-24T23:21:10.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.175+0000 7f43626ae640 10 monclient: build_initial_monmap 2024-01-24T23:21:10.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.175+0000 7f43626ae640 10 monclient: monmap: 2024-01-24T23:21:10.204 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:10.204 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.204 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:10.180596+0000 2024-01-24T23:21:10.204 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:10.180596+0000 2024-01-24T23:21:10.204 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:10.204 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.204 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:10.205 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:10.205 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:10.205 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:10.205 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:10.205 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:10.205 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:10.205 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:10.205 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:10.205 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.175+0000 7f43626ae640 1 Processor -- start 2024-01-24T23:21:10.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.175+0000 7f43626ae640 1 -- start start 2024-01-24T23:21:10.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.175+0000 7f43626ae640 10 monclient: init 2024-01-24T23:21:10.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.175+0000 7f43626ae640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:10.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.175+0000 7f43626ae640 10 monclient: _add_conns ranks=[8,3,6,5,1,0,4,7,2] 2024-01-24T23:21:10.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.175+0000 7f43626ae640 10 monclient(hunting): picked mon.noname-e con 0x7f435c135d10 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:10.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.175+0000 7f43626ae640 10 monclient(hunting): picked mon.noname-g con 0x7f435c139cc0 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:10.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.175+0000 7f43626ae640 10 monclient(hunting): picked mon.noname-d con 0x7f435c13d7d0 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:10.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.175+0000 7f43626ae640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435c073640 con 0x7f435c139cc0 2024-01-24T23:21:10.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.175+0000 7f43626ae640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435c09b7a0 con 0x7f435c13d7d0 2024-01-24T23:21:10.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.175+0000 7f43626ae640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435c09b450 con 0x7f435c135d10 2024-01-24T23:21:10.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.175+0000 7f43626ae640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:10.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.175+0000 7f43626ae640 10 monclient(hunting): authenticate will time out at 949.103532s 2024-01-24T23:21:10.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.175+0000 7f4361ead640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f435c13d7d0 0x7f435c13fbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:56902/0 (socket says 172.21.15.115:56902) 2024-01-24T23:21:10.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.175+0000 7f4361ead640 1 -- 172.21.15.115:0/2982722729 learned_addr learned my addr 172.21.15.115:0/2982722729 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:10.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 1 -- 172.21.15.115:0/2982722729 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819864064 0 0) 0x7f435c09b7a0 con 0x7f435c13d7d0 2024-01-24T23:21:10.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 10 monclient(hunting): my global_id is 4299 2024-01-24T23:21:10.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 1 -- 172.21.15.115:0/2982722729 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4348003530 con 0x7f435c13d7d0 2024-01-24T23:21:10.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 1 -- 172.21.15.115:0/2982722729 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4132221898 0 0) 0x7f4348003530 con 0x7f435c13d7d0 2024-01-24T23:21:10.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 1 -- 172.21.15.115:0/2982722729 >> v1:172.21.15.115:6793/0 conn(0x7f435c135d10 legacy=0x7f435c1360f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 1 -- 172.21.15.115:0/2982722729 >> v1:172.21.15.132:6790/0 conn(0x7f435c139cc0 legacy=0x7f435c13c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T23:21:10.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 10 monclient: _finish_hunting 0 2024-01-24T23:21:10.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 1 monclient: found mon.noname-d 2024-01-24T23:21:10.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:10.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:21:10.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 1 -- 172.21.15.115:0/2982722729 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435c09a9e0 con 0x7f435c13d7d0 2024-01-24T23:21:10.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 10 monclient: _finish_auth 0 2024-01-24T23:21:10.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 10 monclient: _check_auth_tickets 2024-01-24T23:21:10.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:10.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 1 -- 172.21.15.115:0/2982722729 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4354001b40 con 0x7f435c13d7d0 2024-01-24T23:21:10.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:21:10.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 10 monclient: dump: 2024-01-24T23:21:10.209 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.209 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.209 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.209 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.209 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.210 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.210 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.210 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.210 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.210 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.210 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.210 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.210 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.210 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.210 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.210 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f43626ae640 5 monclient: authenticate success, global_id 4299 2024-01-24T23:21:10.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f43626ae640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:10.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 1 -- 172.21.15.115:0/2982722729 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4354001d60 con 0x7f435c13d7d0 2024-01-24T23:21:10.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:10.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 1 -- 172.21.15.115:0/2982722729 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4354004f70 con 0x7f435c13d7d0 2024-01-24T23:21:10.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f43626ae640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:10.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f43626ae640 4 set_mon_vals no callback set 2024-01-24T23:21:10.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:10.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f4353fff640 10 monclient: dump: 2024-01-24T23:21:10.211 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.212 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.212 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.212 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.212 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.212 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.212 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.212 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.212 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.212 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.212 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.212 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.212 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.213 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.213 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.213 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f43626ae640 10 monclient: shutdown 2024-01-24T23:21:10.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f43626ae640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f43626ae640 1 -- 172.21.15.115:0/2982722729 >> v1:172.21.15.115:6792/0 conn(0x7f435c13d7d0 legacy=0x7f435c13fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f43626ae640 1 -- 172.21.15.115:0/2982722729 shutdown_connections 2024-01-24T23:21:10.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f43626ae640 1 -- 172.21.15.115:0/2982722729 >> 172.21.15.115:0/2982722729 conn(0x7f435c0664c0 msgr2=0x7f435c0668d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f43626ae640 1 -- 172.21.15.115:0/2982722729 shutdown_connections 2024-01-24T23:21:10.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f43626ae640 1 -- 172.21.15.115:0/2982722729 wait complete. 2024-01-24T23:21:10.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f43626ae640 10 monclient: build_initial_monmap 2024-01-24T23:21:10.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.179+0000 7f43626ae640 10 monclient: monmap: 2024-01-24T23:21:10.214 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:10.214 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:10.214 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:10.214 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:10.214 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:10.214 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:10.214 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:10.214 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:10.215 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:10.215 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:10.215 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:10.215 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:10.215 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:10.215 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:10.215 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:10.215 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.183+0000 7f43626ae640 1 Processor -- start 2024-01-24T23:21:10.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.183+0000 7f43626ae640 1 -- start start 2024-01-24T23:21:10.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.183+0000 7f43626ae640 10 monclient: init 2024-01-24T23:21:10.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.183+0000 7f43626ae640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:10.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.183+0000 7f43626ae640 10 monclient: _add_conns ranks=[2,5,8,1,0,3,7,6,4] 2024-01-24T23:21:10.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.183+0000 7f43626ae640 10 monclient(hunting): picked mon.noname-b con 0x7f435c135d10 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:10.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.183+0000 7f43626ae640 10 monclient(hunting): picked mon.noname-h con 0x7f435c139cc0 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:10.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.183+0000 7f43626ae640 10 monclient(hunting): picked mon.noname-e con 0x7f435c13d7d0 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:10.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.183+0000 7f43626ae640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435c06bca0 con 0x7f435c135d10 2024-01-24T23:21:10.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.183+0000 7f43626ae640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435c0a5460 con 0x7f435c139cc0 2024-01-24T23:21:10.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.183+0000 7f43626ae640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435c136880 con 0x7f435c13d7d0 2024-01-24T23:21:10.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.183+0000 7f43626ae640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:10.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.183+0000 7f43626ae640 10 monclient(hunting): authenticate will time out at 949.109708s 2024-01-24T23:21:10.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.183+0000 7f43616ac640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f435c135d10 0x7f435c138320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:54512/0 (socket says 172.21.15.115:54512) 2024-01-24T23:21:10.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.183+0000 7f43616ac640 1 -- 172.21.15.115:0/1330690824 learned_addr learned my addr 172.21.15.115:0/1330690824 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:10.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.183+0000 7f4351ffb640 1 -- 172.21.15.115:0/1330690824 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 851798206 0 0) 0x7f435c0a5460 con 0x7f435c139cc0 2024-01-24T23:21:10.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.183+0000 7f4351ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.183+0000 7f4351ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.183+0000 7f4351ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.183+0000 7f4351ffb640 10 monclient(hunting): my global_id is 4262 2024-01-24T23:21:10.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.183+0000 7f4351ffb640 1 -- 172.21.15.115:0/1330690824 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4340003530 con 0x7f435c139cc0 2024-01-24T23:21:10.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.183+0000 7f4351ffb640 1 -- 172.21.15.115:0/1330690824 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1552997319 0 0) 0x7f435c136880 con 0x7f435c13d7d0 2024-01-24T23:21:10.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.183+0000 7f4351ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 10 monclient(hunting): my global_id is 74195 2024-01-24T23:21:10.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 1 -- 172.21.15.115:0/1330690824 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f435c0a5460 con 0x7f435c13d7d0 2024-01-24T23:21:10.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 1 -- 172.21.15.115:0/1330690824 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1419435232 0 0) 0x7f4340003530 con 0x7f435c139cc0 2024-01-24T23:21:10.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 1 -- 172.21.15.115:0/1330690824 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f435c136880 con 0x7f435c139cc0 2024-01-24T23:21:10.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 1 -- 172.21.15.115:0/1330690824 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4358001680 con 0x7f435c139cc0 2024-01-24T23:21:10.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:21:10.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 10 monclient(hunting): dump: 2024-01-24T23:21:10.218 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.218 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.218 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.218 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.218 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.218 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.218 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.219 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.219 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.219 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.219 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.219 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.219 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.219 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.219 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.219 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:10.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 1 -- 172.21.15.115:0/1330690824 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3989632018 0 0) 0x7f435c0a5460 con 0x7f435c13d7d0 2024-01-24T23:21:10.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 1 -- 172.21.15.115:0/1330690824 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4340003530 con 0x7f435c13d7d0 2024-01-24T23:21:10.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 1 -- 172.21.15.115:0/1330690824 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 386922645 0 0) 0x7f435c136880 con 0x7f435c139cc0 2024-01-24T23:21:10.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 1 -- 172.21.15.115:0/1330690824 >> v1:172.21.15.115:6793/0 conn(0x7f435c13d7d0 legacy=0x7f435c1d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 1 -- 172.21.15.115:0/1330690824 >> v1:172.21.15.115:6790/0 conn(0x7f435c135d10 legacy=0x7f435c138320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:21:10.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 1 monclient: found mon.h 2024-01-24T23:21:10.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:10.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:21:10.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 1 -- 172.21.15.115:0/1330690824 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435c0a58b0 con 0x7f435c139cc0 2024-01-24T23:21:10.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 10 monclient: _finish_auth 0 2024-01-24T23:21:10.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:21:10.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:10.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f43626ae640 5 monclient: authenticate success, global_id 4262 2024-01-24T23:21:10.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f43626ae640 10 monclient: _renew_subs 2024-01-24T23:21:10.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f43626ae640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:21:10.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f43626ae640 1 -- 172.21.15.115:0/1330690824 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f435c13cec0 con 0x7f435c139cc0 2024-01-24T23:21:10.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f43626ae640 10 monclient: _renew_subs 2024-01-24T23:21:10.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f43626ae640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:21:10.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f43626ae640 1 -- 172.21.15.115:0/1330690824 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f435c136ab0 con 0x7f435c139cc0 2024-01-24T23:21:10.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 1 -- 172.21.15.115:0/1330690824 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4358001d60 con 0x7f435c139cc0 2024-01-24T23:21:10.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:10.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f43537fe640 4 set_mon_vals no callback set 2024-01-24T23:21:10.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 1 -- 172.21.15.115:0/1330690824 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4358003b50 con 0x7f435c139cc0 2024-01-24T23:21:10.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:21:10.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.187+0000 7f4351ffb640 10 monclient: dump: 2024-01-24T23:21:10.222 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.222 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.222 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.222 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.222 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.222 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.222 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.222 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.223 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.223 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.223 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.223 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.223 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.223 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.223 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.223 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.191+0000 7f43477fe640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:21:10.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.191+0000 7f43477fe640 1 -- 172.21.15.115:0/1330690824 --> v1:172.21.15.132:6791/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f435c137660 con 0x7f435c139cc0 2024-01-24T23:21:10.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.191+0000 7f4351ffb640 1 -- 172.21.15.115:0/1330690824 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 4) v1 ==== 43232+0+0 (unknown 913048011 0 0) 0x7f435800e5a0 con 0x7f435c139cc0 2024-01-24T23:21:10.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.191+0000 7f4351ffb640 1 -- 172.21.15.115:0/1330690824 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f43580421e0 con 0x7f435c139cc0 2024-01-24T23:21:10.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.191+0000 7f43616ac640 10 monclient: get_auth_request con 0x7f43400374a0 auth_method 0 2024-01-24T23:21:10.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.191+0000 7f4351ffb640 1 -- 172.21.15.115:0/1330690824 --> v1:172.21.15.115:6801/3052044493 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4340041c60 con 0x7f434003dff0 2024-01-24T23:21:10.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.191+0000 7f4361ead640 10 monclient: get_auth_request con 0x7f434003dff0 auth_method 0 2024-01-24T23:21:10.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.191+0000 7f4351ffb640 1 -- 172.21.15.115:0/1330690824 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (unknown 2942983318 0 0) 0x7f4358042550 con 0x7f435c139cc0 2024-01-24T23:21:10.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.191+0000 7f4351ffb640 10 monclient: handle_get_version_reply finishing 1 version 15 2024-01-24T23:21:10.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.191+0000 7f4351ffb640 1 -- 172.21.15.115:0/1330690824 <== osd.1 v1:172.21.15.115:6801/3052044493 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f4340041c60 con 0x7f434003dff0 2024-01-24T23:21:10.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.199+0000 7f62c2cd5640 10 monclient: get_monmap_and_config 2024-01-24T23:21:10.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.199+0000 7f62c2cd5640 10 monclient: build_initial_monmap 2024-01-24T23:21:10.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.199+0000 7f62c2cd5640 10 monclient: monmap: 2024-01-24T23:21:10.225 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:10.226 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.226 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:10.206489+0000 2024-01-24T23:21:10.226 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:10.206489+0000 2024-01-24T23:21:10.226 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:10.226 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.226 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:10.226 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:10.226 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:10.226 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:10.226 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:10.226 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:10.226 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:10.226 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:10.227 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:10.227 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f23997ef640 10 monclient: get_monmap_and_config 2024-01-24T23:21:10.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f23997ef640 10 monclient: build_initial_monmap 2024-01-24T23:21:10.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f23997ef640 10 monclient: monmap: 2024-01-24T23:21:10.227 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:10.228 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.228 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:10.207121+0000 2024-01-24T23:21:10.228 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:10.207121+0000 2024-01-24T23:21:10.228 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:10.228 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.228 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:10.228 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:10.228 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:10.228 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:10.228 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:10.228 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:10.228 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:10.228 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:10.229 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:10.229 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62c2cd5640 1 Processor -- start 2024-01-24T23:21:10.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62c2cd5640 1 -- start start 2024-01-24T23:21:10.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62c2cd5640 10 monclient: init 2024-01-24T23:21:10.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f23997ef640 1 Processor -- start 2024-01-24T23:21:10.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f23997ef640 1 -- start start 2024-01-24T23:21:10.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f23997ef640 10 monclient: init 2024-01-24T23:21:10.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62c2cd5640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:10.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62c2cd5640 10 monclient: _add_conns ranks=[0,7,1,5,2,4,8,3,6] 2024-01-24T23:21:10.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62c2cd5640 10 monclient(hunting): picked mon.noname-a con 0x7f62b402a550 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:10.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62c2cd5640 10 monclient(hunting): picked mon.noname-i con 0x7f62b4028270 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:10.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62c0a71640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f62b402a550 0x7f62b4022fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:52088/0 (socket says 172.21.15.115:52088) 2024-01-24T23:21:10.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62c0a71640 1 -- 172.21.15.115:0/1516054822 learned_addr learned my addr 172.21.15.115:0/1516054822 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f23997ef640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f23997ef640 10 monclient: _add_conns ranks=[6,7,2,8,0,5,4,3,1] 2024-01-24T23:21:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f23997ef640 10 monclient(hunting): picked mon.noname-d con 0x7f238c02a550 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f23997ef640 10 monclient(hunting): picked mon.noname-i con 0x7f238c028270 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f23997ef640 10 monclient(hunting): picked mon.noname-b con 0x7f238c0c2bf0 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f23997ef640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f238c004070 con 0x7f238c0c2bf0 2024-01-24T23:21:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f23997ef640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f238c02a930 con 0x7f238c02a550 2024-01-24T23:21:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f23997ef640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f238c025a30 con 0x7f238c028270 2024-01-24T23:21:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f23997ef640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f23997ef640 10 monclient(hunting): authenticate will time out at 949.130800s 2024-01-24T23:21:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f23937fe640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f238c028270 0x7f238c028690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.115:36120/0 (socket says 172.21.15.115:36120) 2024-01-24T23:21:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f23937fe640 1 -- 172.21.15.115:0/3876689653 learned_addr learned my addr 172.21.15.115:0/3876689653 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62c2cd5640 10 monclient(hunting): picked mon.noname-f con 0x7f62b40c2bf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62c2cd5640 1 -- 172.21.15.115:0/1516054822 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b4004070 con 0x7f62b402a550 2024-01-24T23:21:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62c2cd5640 1 -- 172.21.15.115:0/1516054822 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b402a930 con 0x7f62b40c2bf0 2024-01-24T23:21:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62c2cd5640 1 -- 172.21.15.115:0/1516054822 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b4025a30 con 0x7f62b4028270 2024-01-24T23:21:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62c2cd5640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62c2cd5640 10 monclient(hunting): authenticate will time out at 949.131152s 2024-01-24T23:21:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62bb7fe640 1 -- 172.21.15.115:0/1516054822 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2458404833 0 0) 0x7f62b4004070 con 0x7f62b402a550 2024-01-24T23:21:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62bb7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62bb7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62bb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62bb7fe640 10 monclient(hunting): my global_id is 4302 2024-01-24T23:21:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62bb7fe640 1 -- 172.21.15.115:0/1516054822 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62ac003530 con 0x7f62b402a550 2024-01-24T23:21:10.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 1 -- 172.21.15.115:0/3876689653 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2980641911 0 0) 0x7f238c02a930 con 0x7f238c02a550 2024-01-24T23:21:10.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 10 monclient(hunting): my global_id is 4308 2024-01-24T23:21:10.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 1 -- 172.21.15.115:0/3876689653 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2384003530 con 0x7f238c02a550 2024-01-24T23:21:10.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 1 -- 172.21.15.115:0/3876689653 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3907943804 0 0) 0x7f238c025a30 con 0x7f238c028270 2024-01-24T23:21:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 10 monclient(hunting): my global_id is 4264 2024-01-24T23:21:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 1 -- 172.21.15.115:0/3876689653 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f238c02a930 con 0x7f238c028270 2024-01-24T23:21:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62bb7fe640 1 -- 172.21.15.115:0/1516054822 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146406930 0 0) 0x7f62b4025a30 con 0x7f62b4028270 2024-01-24T23:21:10.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62bb7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62bb7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62bb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62bb7fe640 10 monclient(hunting): my global_id is 4273 2024-01-24T23:21:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62bb7fe640 1 -- 172.21.15.115:0/1516054822 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62ac003850 con 0x7f62b4028270 2024-01-24T23:21:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62bb7fe640 1 -- 172.21.15.115:0/1516054822 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3837449238 0 0) 0x7f62b402a930 con 0x7f62b40c2bf0 2024-01-24T23:21:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62bb7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62bb7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62bb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62bb7fe640 10 monclient(hunting): my global_id is 4240 2024-01-24T23:21:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62bb7fe640 1 -- 172.21.15.115:0/1516054822 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62b4025a30 con 0x7f62b40c2bf0 2024-01-24T23:21:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 1 -- 172.21.15.115:0/3876689653 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3349458036 0 0) 0x7f2384003530 con 0x7f238c02a550 2024-01-24T23:21:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 1 -- 172.21.15.115:0/3876689653 >> v1:172.21.15.132:6792/0 conn(0x7f238c028270 legacy=0x7f238c028690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 1 -- 172.21.15.115:0/3876689653 >> v1:172.21.15.115:6790/0 conn(0x7f238c0c2bf0 legacy=0x7f238c0c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:21:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 1 monclient: found mon.noname-d 2024-01-24T23:21:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:21:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 1 -- 172.21.15.115:0/3876689653 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f238c025190 con 0x7f238c02a550 2024-01-24T23:21:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 10 monclient: _finish_auth 0 2024-01-24T23:21:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:21:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 1 -- 172.21.15.115:0/3876689653 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f23880016f0 con 0x7f238c02a550 2024-01-24T23:21:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f23997ef640 5 monclient: authenticate success, global_id 4308 2024-01-24T23:21:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f23997ef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:21:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f2392ffd640 10 monclient: dump: 2024-01-24T23:21:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.241 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.241 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.241 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.241 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.241 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.241 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.241 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.241 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.241 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.241 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.241 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.241 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f23997ef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62bb7fe640 1 -- 172.21.15.115:0/1516054822 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2879631035 0 0) 0x7f62ac003530 con 0x7f62b402a550 2024-01-24T23:21:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62bb7fe640 1 -- 172.21.15.115:0/1516054822 >> v1:172.21.15.132:6792/0 conn(0x7f62b4028270 legacy=0x7f62b4028690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.203+0000 7f62bb7fe640 1 -- 172.21.15.115:0/1516054822 >> v1:172.21.15.132:6789/0 conn(0x7f62b40c2bf0 legacy=0x7f62b40c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62bb7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:21:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62bb7fe640 1 monclient: found mon.noname-a 2024-01-24T23:21:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62bb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62bb7fe640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:21:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62bb7fe640 1 -- 172.21.15.115:0/1516054822 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b4025190 con 0x7f62b402a550 2024-01-24T23:21:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62bb7fe640 10 monclient: _finish_auth 0 2024-01-24T23:21:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62bb7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:21:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62bb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 5 monclient: authenticate success, global_id 4302 2024-01-24T23:21:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62bb7fe640 1 -- 172.21.15.115:0/1516054822 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f62bc0681a0 con 0x7f62b402a550 2024-01-24T23:21:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62bb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62bb7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:21:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62bb7fe640 10 monclient: dump: 2024-01-24T23:21:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.245 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.245 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.245 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.245 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.245 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.245 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.245 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.245 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.245 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.245 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.245 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f2392ffd640 1 -- 172.21.15.115:0/3876689653 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f23880019d0 con 0x7f238c02a550 2024-01-24T23:21:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f2392ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f2392ffd640 1 -- 172.21.15.115:0/3876689653 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2388003940 con 0x7f238c02a550 2024-01-24T23:21:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f2392ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f2392ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f2392ffd640 10 monclient: dump: 2024-01-24T23:21:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.247 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.247 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.247 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.247 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.247 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.247 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.247 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.247 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.247 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.247 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.247 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.247 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.247 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.248 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f23997ef640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:10.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f23997ef640 4 set_mon_vals no callback set 2024-01-24T23:21:10.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62bb7fe640 1 -- 172.21.15.115:0/1516054822 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f62bc04f7e0 con 0x7f62b402a550 2024-01-24T23:21:10.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62bb7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:10.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:10.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 4 set_mon_vals no callback set 2024-01-24T23:21:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62bb7fe640 1 -- 172.21.15.115:0/1516054822 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f62bc0681a0 con 0x7f62b402a550 2024-01-24T23:21:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62bb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62bb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62bb7fe640 10 monclient: dump: 2024-01-24T23:21:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.250 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.250 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.250 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.250 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.250 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.250 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.250 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.250 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.250 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f23997ef640 10 monclient: shutdown 2024-01-24T23:21:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f23997ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f23997ef640 1 -- 172.21.15.115:0/3876689653 >> v1:172.21.15.115:6792/0 conn(0x7f238c02a550 legacy=0x7f238c022fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 10 monclient: shutdown 2024-01-24T23:21:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 1 -- 172.21.15.115:0/1516054822 >> v1:172.21.15.115:6789/0 conn(0x7f62b402a550 legacy=0x7f62b4022fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 1 -- 172.21.15.115:0/1516054822 shutdown_connections 2024-01-24T23:21:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f23997ef640 1 -- 172.21.15.115:0/3876689653 shutdown_connections 2024-01-24T23:21:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f23997ef640 1 -- 172.21.15.115:0/3876689653 >> 172.21.15.115:0/3876689653 conn(0x7f238c007550 msgr2=0x7f238c007960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 1 -- 172.21.15.115:0/1516054822 >> 172.21.15.115:0/1516054822 conn(0x7f62b4007550 msgr2=0x7f62b4007960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.253 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f23997ef640 1 -- 172.21.15.115:0/3876689653 shutdown_connections 2024-01-24T23:21:10.253 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f23997ef640 1 -- 172.21.15.115:0/3876689653 wait complete. 2024-01-24T23:21:10.253 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 1 -- 172.21.15.115:0/1516054822 shutdown_connections 2024-01-24T23:21:10.254 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 1 -- 172.21.15.115:0/1516054822 wait complete. 2024-01-24T23:21:10.254 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f23997ef640 10 monclient: build_initial_monmap 2024-01-24T23:21:10.254 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f23997ef640 10 monclient: monmap: 2024-01-24T23:21:10.254 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:10.254 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:10.254 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:10.254 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:10.254 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:10.254 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:10.255 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:10.255 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:10.255 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:10.255 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:10.255 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:10.255 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:10.255 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:10.255 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:10.255 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:10.255 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 10 monclient: build_initial_monmap 2024-01-24T23:21:10.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 10 monclient: monmap: 2024-01-24T23:21:10.256 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:10.256 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:10.256 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:10.256 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:10.256 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:10.256 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:10.257 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:10.257 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:10.257 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:10.257 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:10.257 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:10.257 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:10.257 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:10.257 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:10.257 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:10.257 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 1 Processor -- start 2024-01-24T23:21:10.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f88c37fe640 1 -- 172.21.15.115:0/914561935 --> v1:172.21.15.115:6808/3064328804 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f88d81cc580 con 0x7f88d8060fd0 2024-01-24T23:21:10.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f88cdffb640 1 -- 172.21.15.115:0/914561935 <== osd.0 v1:172.21.15.115:6808/3064328804 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 654699149) 0x7f88d81cc580 con 0x7f88d8060fd0 2024-01-24T23:21:10.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 1 -- start start 2024-01-24T23:21:10.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 10 monclient: init 2024-01-24T23:21:10.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:10.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 10 monclient: _add_conns ranks=[5,4,1,3,2,7,8,6,0] 2024-01-24T23:21:10.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 10 monclient(hunting): picked mon.noname-h con 0x7f62b4028270 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:10.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 10 monclient(hunting): picked mon.noname-c con 0x7f62b40c2bf0 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:10.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 10 monclient(hunting): picked mon.noname-f con 0x7f62b415bae0 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:10.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b400d570 con 0x7f62b415bae0 2024-01-24T23:21:10.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b4024a80 con 0x7f62b40c2bf0 2024-01-24T23:21:10.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b4024530 con 0x7f62b4028270 2024-01-24T23:21:10.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:10.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c2cd5640 10 monclient(hunting): authenticate will time out at 949.135426s 2024-01-24T23:21:10.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f23997ef640 1 Processor -- start 2024-01-24T23:21:10.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f23997ef640 1 -- start start 2024-01-24T23:21:10.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f23997ef640 10 monclient: init 2024-01-24T23:21:10.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c1272640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f62b415bae0 0x7f62b415ded0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.115:35084/0 (socket says 172.21.15.115:35084) 2024-01-24T23:21:10.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c1272640 1 -- 172.21.15.115:0/3718224289 learned_addr learned my addr 172.21.15.115:0/3718224289 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:10.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.207+0000 7f62c0a71640 1 --1- 172.21.15.115:0/3718224289 >> v1:172.21.15.132:6791/0 conn(0x7f62b4028270 0x7f62b400ab00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.115:47800/0 (socket says 172.21.15.115:47800) 2024-01-24T23:21:10.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f23997ef640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:10.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f23997ef640 10 monclient: _add_conns ranks=[1,6,8,2,0,7,4,3,5] 2024-01-24T23:21:10.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f23997ef640 10 monclient(hunting): picked mon.noname-f con 0x7f238c028270 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:10.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f88c37fe640 1 -- 172.21.15.115:0/914561935 >> v1:172.21.15.115:6808/3064328804 conn(0x7f88d8060fd0 legacy=0x7f88d80613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 956717569 0 0) 0x7f62b4024530 con 0x7f62b4028270 2024-01-24T23:21:10.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient(hunting): my global_id is 4271 2024-01-24T23:21:10.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62a0003530 con 0x7f62b4028270 2024-01-24T23:21:10.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f23997ef640 10 monclient(hunting): picked mon.noname-d con 0x7f238c02a550 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:10.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1645170602 0 0) 0x7f62b400d570 con 0x7f62b415bae0 2024-01-24T23:21:10.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient(hunting): my global_id is 4249 2024-01-24T23:21:10.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62b4024530 con 0x7f62b415bae0 2024-01-24T23:21:10.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f88c37fe640 1 -- 172.21.15.115:0/914561935 >> v1:172.21.15.132:6812/1835401922 conn(0x7f88bc037870 legacy=0x7f88bc039d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f88c37fe640 10 monclient: shutdown 2024-01-24T23:21:10.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f88c37fe640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f88c37fe640 1 -- 172.21.15.115:0/914561935 >> v1:172.21.15.132:6790/0 conn(0x7f88d8133fd0 legacy=0x7f88d81d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f23937fe640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f238c02a550 0x7f238c163080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:56912/0 (socket says 172.21.15.115:56912) 2024-01-24T23:21:10.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f2393fff640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f238c028270 0x7f238c15e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.115:35088/0 (socket says 172.21.15.115:35088) 2024-01-24T23:21:10.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3619685576 0 0) 0x7f62b4024a80 con 0x7f62b40c2bf0 2024-01-24T23:21:10.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient(hunting): my global_id is 74263 2024-01-24T23:21:10.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62b400d570 con 0x7f62b40c2bf0 2024-01-24T23:21:10.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f88ddf59640 1 -- 172.21.15.115:0/914561935 reap_dead start 2024-01-24T23:21:10.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f88c37fe640 1 -- 172.21.15.115:0/914561935 shutdown_connections 2024-01-24T23:21:10.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f88c37fe640 1 -- 172.21.15.115:0/914561935 >> 172.21.15.115:0/914561935 conn(0x7f88d80662f0 msgr2=0x7f88d81bea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f2398fee640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f238c0c2bf0 0x7f238c1667b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:48740/0 (socket says 172.21.15.115:48740) 2024-01-24T23:21:10.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f23997ef640 10 monclient(hunting): picked mon.noname-e con 0x7f238c0c2bf0 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:10.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f23997ef640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f238c0b6af0 con 0x7f238c028270 2024-01-24T23:21:10.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f23997ef640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f238c020b20 con 0x7f238c02a550 2024-01-24T23:21:10.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f23997ef640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f238c024530 con 0x7f238c0c2bf0 2024-01-24T23:21:10.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f23997ef640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:10.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f23997ef640 10 monclient(hunting): authenticate will time out at 949.136726s 2024-01-24T23:21:10.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f23937fe640 1 -- 172.21.15.115:0/462850181 learned_addr learned my addr 172.21.15.115:0/462850181 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:10.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f88c37fe640 1 -- 172.21.15.115:0/914561935 shutdown_connections 2024-01-24T23:21:10.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f88c37fe640 1 -- 172.21.15.115:0/914561935 wait complete. 2024-01-24T23:21:10.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1128165625 0 0) 0x7f62b400d570 con 0x7f62b40c2bf0 2024-01-24T23:21:10.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f62b4024a80 con 0x7f62b40c2bf0 2024-01-24T23:21:10.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2883700109 0 0) 0x7f62a0003530 con 0x7f62b4028270 2024-01-24T23:21:10.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f62b400d570 con 0x7f62b4028270 2024-01-24T23:21:10.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f62a8001aa0 con 0x7f62b40c2bf0 2024-01-24T23:21:10.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:21:10.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient(hunting): dump: 2024-01-24T23:21:10.269 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.269 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.269 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.269 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.269 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.269 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.269 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.269 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.269 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.269 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.269 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.269 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.269 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.270 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.270 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.270 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:10.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f62bc0681a0 con 0x7f62b4028270 2024-01-24T23:21:10.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:21:10.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient(hunting): dump: 2024-01-24T23:21:10.270 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.270 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.270 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.271 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.272 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1086842068 0 0) 0x7f62b4024530 con 0x7f62b415bae0 2024-01-24T23:21:10.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f62a0003530 con 0x7f62b415bae0 2024-01-24T23:21:10.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f62b0001a80 con 0x7f62b415bae0 2024-01-24T23:21:10.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:21:10.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient(hunting): dump: 2024-01-24T23:21:10.272 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.272 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.272 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.272 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.272 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.273 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.273 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.273 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.273 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.273 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.273 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.273 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.273 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.273 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.273 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.273 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3718253090 0 0) 0x7f238c0b6af0 con 0x7f238c028270 2024-01-24T23:21:10.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f2390ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f2390ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f2390ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f2390ff9640 10 monclient(hunting): my global_id is 4258 2024-01-24T23:21:10.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2378003530 con 0x7f238c028270 2024-01-24T23:21:10.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2407120686 0 0) 0x7f62b4024a80 con 0x7f62b40c2bf0 2024-01-24T23:21:10.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 >> v1:172.21.15.132:6791/0 conn(0x7f62b4028270 legacy=0x7f62b400ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 >> v1:172.21.15.132:6789/0 conn(0x7f62b415bae0 legacy=0x7f62b415ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient: _finish_hunting 0 2024-01-24T23:21:10.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 1 monclient: found mon.c 2024-01-24T23:21:10.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:10.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:10.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b400e360 con 0x7f62b40c2bf0 2024-01-24T23:21:10.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient: _finish_auth 0 2024-01-24T23:21:10.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient: _check_auth_tickets 2024-01-24T23:21:10.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:10.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 879524791 0 0) 0x7f238c024530 con 0x7f238c0c2bf0 2024-01-24T23:21:10.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f2390ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f2390ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f2390ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f2390ff9640 10 monclient(hunting): my global_id is 74204 2024-01-24T23:21:10.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f238c0b6af0 con 0x7f238c0c2bf0 2024-01-24T23:21:10.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1478383822 0 0) 0x7f238c020b20 con 0x7f238c02a550 2024-01-24T23:21:10.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f2390ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f62a80038d0 con 0x7f62b40c2bf0 2024-01-24T23:21:10.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62c2cd5640 5 monclient: authenticate success, global_id 74263 2024-01-24T23:21:10.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:10.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f62a8001d90 con 0x7f62b40c2bf0 2024-01-24T23:21:10.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:10.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f2390ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f2390ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f2390ff9640 10 monclient(hunting): my global_id is 4317 2024-01-24T23:21:10.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f238c024530 con 0x7f238c02a550 2024-01-24T23:21:10.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62b97fa640 10 monclient: dump: 2024-01-24T23:21:10.279 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.279 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.279 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.279 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.279 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.279 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.280 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62baffd640 4 set_mon_vals no callback set 2024-01-24T23:21:10.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62c2cd5640 10 monclient: _renew_subs 2024-01-24T23:21:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62c2cd5640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62c2cd5640 1 -- 172.21.15.115:0/3718224289 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b400b920 con 0x7f62b40c2bf0 2024-01-24T23:21:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62c2cd5640 10 monclient: _renew_subs 2024-01-24T23:21:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62c2cd5640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62c2cd5640 1 -- 172.21.15.115:0/3718224289 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b400ce30 con 0x7f62b40c2bf0 2024-01-24T23:21:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62c2cd5640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.211+0000 7f62c2cd5640 1 -- 172.21.15.115:0/3718224289 --> v1:172.21.15.115:6791/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f62b4025750 con 0x7f62b40c2bf0 2024-01-24T23:21:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 4) v1 ==== 43232+0+0 (unknown 913048011 0 0) 0x7f62a800e670 con 0x7f62b40c2bf0 2024-01-24T23:21:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f62a80422b0 con 0x7f62b40c2bf0 2024-01-24T23:21:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 --> v1:172.21.15.115:6800/2776240662 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f62a0042390 con 0x7f62a003e720 2024-01-24T23:21:10.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (unknown 2942983318 0 0) 0x7f62a8042530 con 0x7f62b40c2bf0 2024-01-24T23:21:10.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f62b97fa640 10 monclient: handle_get_version_reply finishing 1 version 15 2024-01-24T23:21:10.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f62c0a71640 10 monclient: get_auth_request con 0x7f62a0037b30 auth_method 0 2024-01-24T23:21:10.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f62c1272640 10 monclient: get_auth_request con 0x7f62a003e720 auth_method 0 2024-01-24T23:21:10.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2365297382 0 0) 0x7f2378003530 con 0x7f238c028270 2024-01-24T23:21:10.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f238c020b20 con 0x7f238c028270 2024-01-24T23:21:10.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 87269648 0 0) 0x7f238c024530 con 0x7f238c02a550 2024-01-24T23:21:10.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2378003530 con 0x7f238c02a550 2024-01-24T23:21:10.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3378022628 0 0) 0x7f238c0b6af0 con 0x7f238c0c2bf0 2024-01-24T23:21:10.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f238c024530 con 0x7f238c0c2bf0 2024-01-24T23:21:10.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2388001370 con 0x7f238c028270 2024-01-24T23:21:10.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:21:10.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 10 monclient(hunting): dump: 2024-01-24T23:21:10.283 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.284 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.284 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.284 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.284 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.284 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.284 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.284 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.284 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.284 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.284 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.284 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.285 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.285 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.285 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.285 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:10.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f23800019a0 con 0x7f238c02a550 2024-01-24T23:21:10.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:10.285 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 10 monclient(hunting): dump: 2024-01-24T23:21:10.285 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.285 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.285 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.286 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.286 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.286 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.286 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.286 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.286 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.286 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.286 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.286 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.286 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.286 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.286 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.286 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2394069b70 con 0x7f238c0c2bf0 2024-01-24T23:21:10.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:21:10.287 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 10 monclient(hunting): dump: 2024-01-24T23:21:10.287 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.287 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.287 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.287 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.287 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.287 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.287 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.287 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.288 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.288 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.288 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.288 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.288 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.288 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.288 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.288 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1816791851 0 0) 0x7f2378003530 con 0x7f238c02a550 2024-01-24T23:21:10.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 >> v1:172.21.15.115:6793/0 conn(0x7f238c0c2bf0 legacy=0x7f238c1667b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 >> v1:172.21.15.132:6789/0 conn(0x7f238c028270 legacy=0x7f238c15e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.288 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:21:10.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 1 monclient: found mon.d 2024-01-24T23:21:10.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:10.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:10.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f238c0b6810 con 0x7f238c02a550 2024-01-24T23:21:10.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 10 monclient: _finish_auth 0 2024-01-24T23:21:10.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:21:10.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:10.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f23997ef640 5 monclient: authenticate success, global_id 4317 2024-01-24T23:21:10.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f23997ef640 10 monclient: _renew_subs 2024-01-24T23:21:10.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f23997ef640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:10.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f23997ef640 1 -- 172.21.15.115:0/462850181 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f238c024a80 con 0x7f238c02a550 2024-01-24T23:21:10.289 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f23997ef640 10 monclient: _renew_subs 2024-01-24T23:21:10.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f23997ef640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:10.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f23997ef640 1 -- 172.21.15.115:0/462850181 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f238c021270 con 0x7f238c02a550 2024-01-24T23:21:10.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2380001c60 con 0x7f238c02a550 2024-01-24T23:21:10.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:10.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2380004de0 con 0x7f238c02a550 2024-01-24T23:21:10.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f23927fc640 4 set_mon_vals no callback set 2024-01-24T23:21:10.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:10.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 10 monclient: dump: 2024-01-24T23:21:10.290 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.290 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.290 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.290 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.291 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.291 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.291 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.291 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.291 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.291 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.291 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.291 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.291 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.291 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.291 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.291 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 4) v1 ==== 43232+0+0 (unknown 913048011 0 0) 0x7f2380004f40 con 0x7f238c02a550 2024-01-24T23:21:10.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f237e7fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:10.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f237e7fc640 1 -- 172.21.15.115:0/462850181 --> v1:172.21.15.115:6792/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2374000f90 con 0x7f238c02a550 2024-01-24T23:21:10.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f2380043320 con 0x7f238c02a550 2024-01-24T23:21:10.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2393fff640 10 monclient: get_auth_request con 0x7f2378037b50 auth_method 0 2024-01-24T23:21:10.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 --> v1:172.21.15.132:6803/3365641231 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f23780423b0 con 0x7f237803e740 2024-01-24T23:21:10.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (unknown 2942983318 0 0) 0x7f2380043700 con 0x7f238c02a550 2024-01-24T23:21:10.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2390ff9640 10 monclient: handle_get_version_reply finishing 1 version 15 2024-01-24T23:21:10.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 <== osd.2 v1:172.21.15.115:6800/2776240662 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f62a0042390 con 0x7f62a003e720 2024-01-24T23:21:10.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f213d42f640 10 monclient: get_monmap_and_config 2024-01-24T23:21:10.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f213d42f640 10 monclient: build_initial_monmap 2024-01-24T23:21:10.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f213d42f640 10 monclient: monmap: 2024-01-24T23:21:10.293 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:10.294 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.294 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:10.221411+0000 2024-01-24T23:21:10.294 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:10.221411+0000 2024-01-24T23:21:10.294 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:10.294 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.294 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:10.294 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:10.294 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:10.294 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:10.294 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:10.294 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:10.294 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:10.295 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:10.295 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:10.295 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2398fee640 10 monclient: get_auth_request con 0x7f237803e740 auth_method 0 2024-01-24T23:21:10.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2b29bce640 10 monclient: get_monmap_and_config 2024-01-24T23:21:10.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2b29bce640 10 monclient: build_initial_monmap 2024-01-24T23:21:10.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.215+0000 7f2b29bce640 10 monclient: monmap: 2024-01-24T23:21:10.295 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:10.296 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.296 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:10.221858+0000 2024-01-24T23:21:10.296 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:10.221858+0000 2024-01-24T23:21:10.296 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:10.296 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.296 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:10.296 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:10.296 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:10.296 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:10.296 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:10.296 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:10.296 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:10.297 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:10.297 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:10.297 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f213d42f640 1 Processor -- start 2024-01-24T23:21:10.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f213d42f640 1 -- start start 2024-01-24T23:21:10.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f213d42f640 10 monclient: init 2024-01-24T23:21:10.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b29bce640 1 Processor -- start 2024-01-24T23:21:10.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b29bce640 1 -- start start 2024-01-24T23:21:10.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b29bce640 10 monclient: init 2024-01-24T23:21:10.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f213d42f640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:10.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f213d42f640 10 monclient: _add_conns ranks=[4,8,7,3,6,0,5,1,2] 2024-01-24T23:21:10.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f213d42f640 10 monclient(hunting): picked mon.noname-c con 0x7f2138136f40 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:10.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f213d42f640 10 monclient(hunting): picked mon.noname-e con 0x7f2138136970 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:10.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f213d42f640 10 monclient(hunting): picked mon.noname-i con 0x7f213813d3b0 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:10.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f213d42f640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2138073640 con 0x7f2138136f40 2024-01-24T23:21:10.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f213d42f640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21380a0700 con 0x7f213813d3b0 2024-01-24T23:21:10.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f213d42f640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2138136d70 con 0x7f2138136970 2024-01-24T23:21:10.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f213d42f640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:10.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f213d42f640 10 monclient(hunting): authenticate will time out at 949.145116s 2024-01-24T23:21:10.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f43477fe640 1 -- 172.21.15.115:0/1330690824 --> v1:172.21.15.115:6801/3052044493 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f435c06bb20 con 0x7f434003dff0 2024-01-24T23:21:10.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2137fff640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f2138136f40 0x7f2138137320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:42438/0 (socket says 172.21.15.115:42438) 2024-01-24T23:21:10.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2137fff640 1 -- 172.21.15.115:0/2979052722 learned_addr learned my addr 172.21.15.115:0/2979052722 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:10.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b29bce640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:10.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b29bce640 10 monclient: _add_conns ranks=[5,4,3,7,1,2,6,0,8] 2024-01-24T23:21:10.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b29bce640 10 monclient(hunting): picked mon.noname-h con 0x7f2b2413b130 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:10.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b29bce640 10 monclient(hunting): picked mon.noname-c con 0x7f2b2413dff0 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:10.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b29bce640 10 monclient(hunting): picked mon.noname-g con 0x7f2b24141b00 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:10.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b29bce640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b24073640 con 0x7f2b24141b00 2024-01-24T23:21:10.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b29bce640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b240a0a80 con 0x7f2b2413dff0 2024-01-24T23:21:10.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b29bce640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b2413bc20 con 0x7f2b2413b130 2024-01-24T23:21:10.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b29bce640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:10.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b29bce640 10 monclient(hunting): authenticate will time out at 949.145249s 2024-01-24T23:21:10.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b28bcc640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f2b2413b130 0x7f2b2413b510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.115:47806/0 (socket says 172.21.15.115:47806) 2024-01-24T23:21:10.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b28bcc640 1 -- 172.21.15.115:0/3340486252 learned_addr learned my addr 172.21.15.115:0/3340486252 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:10.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f4351ffb640 1 -- 172.21.15.115:0/1330690824 <== osd.1 v1:172.21.15.115:6801/3052044493 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1800444376) 0x7f4354007ba0 con 0x7f434003dff0 2024-01-24T23:21:10.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 <== osd.5 v1:172.21.15.132:6803/3365641231 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f23780423b0 con 0x7f237803e740 2024-01-24T23:21:10.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 1 -- 172.21.15.115:0/3340486252 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2263448377 0 0) 0x7f2b240a0a80 con 0x7f2b2413dff0 2024-01-24T23:21:10.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 10 monclient(hunting): my global_id is 74281 2024-01-24T23:21:10.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 1 -- 172.21.15.115:0/3340486252 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b0c003530 con 0x7f2b2413dff0 2024-01-24T23:21:10.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 1 -- 172.21.15.115:0/3340486252 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 905083372 0 0) 0x7f2b2413bc20 con 0x7f2b2413b130 2024-01-24T23:21:10.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 10 monclient(hunting): my global_id is 4280 2024-01-24T23:21:10.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 1 -- 172.21.15.115:0/3340486252 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b240a0a80 con 0x7f2b2413b130 2024-01-24T23:21:10.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 1 -- 172.21.15.115:0/2979052722 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481293313 0 0) 0x7f2138136d70 con 0x7f2138136970 2024-01-24T23:21:10.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 10 monclient(hunting): my global_id is 74213 2024-01-24T23:21:10.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 1 -- 172.21.15.115:0/2979052722 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2120003530 con 0x7f2138136970 2024-01-24T23:21:10.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 1 -- 172.21.15.115:0/2979052722 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2589212691 0 0) 0x7f21380a0700 con 0x7f213813d3b0 2024-01-24T23:21:10.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 10 monclient(hunting): my global_id is 4282 2024-01-24T23:21:10.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 1 -- 172.21.15.115:0/2979052722 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2138136d70 con 0x7f213813d3b0 2024-01-24T23:21:10.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 1 -- 172.21.15.115:0/2979052722 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1017460159 0 0) 0x7f2138073640 con 0x7f2138136f40 2024-01-24T23:21:10.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 10 monclient(hunting): my global_id is 74272 2024-01-24T23:21:10.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 1 -- 172.21.15.115:0/2979052722 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21380a0700 con 0x7f2138136f40 2024-01-24T23:21:10.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f43626ae640 1 -- 172.21.15.115:0/1330690824 >> v1:172.21.15.115:6801/3052044493 conn(0x7f434003dff0 legacy=0x7f4340040410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f43626ae640 1 -- 172.21.15.115:0/1330690824 >> v1:172.21.15.132:6812/1835401922 conn(0x7f43400374a0 legacy=0x7f4340039960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f43626ae640 10 monclient: shutdown 2024-01-24T23:21:10.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f43626ae640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f43626ae640 1 -- 172.21.15.115:0/1330690824 >> v1:172.21.15.132:6791/0 conn(0x7f435c139cc0 legacy=0x7f435c1cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f4361ead640 1 -- 172.21.15.115:0/1330690824 reap_dead start 2024-01-24T23:21:10.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f43626ae640 1 -- 172.21.15.115:0/1330690824 shutdown_connections 2024-01-24T23:21:10.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 1 -- 172.21.15.115:0/2979052722 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4032703051 0 0) 0x7f21380a0700 con 0x7f2138136f40 2024-01-24T23:21:10.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 1 -- 172.21.15.115:0/2979052722 >> v1:172.21.15.115:6793/0 conn(0x7f2138136970 legacy=0x7f213813bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 1 -- 172.21.15.115:0/2979052722 >> v1:172.21.15.132:6792/0 conn(0x7f213813d3b0 legacy=0x7f213813f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f43626ae640 1 -- 172.21.15.115:0/1330690824 >> 172.21.15.115:0/1330690824 conn(0x7f435c0664c0 msgr2=0x7f435c09ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:21:10.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 1 monclient: found mon.noname-c 2024-01-24T23:21:10.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:10.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:21:10.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 1 -- 172.21.15.115:0/2979052722 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21380a09e0 con 0x7f2138136f40 2024-01-24T23:21:10.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 10 monclient: _finish_auth 0 2024-01-24T23:21:10.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:21:10.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:10.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 1 -- 172.21.15.115:0/2979052722 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2128001610 con 0x7f2138136f40 2024-01-24T23:21:10.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:21:10.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 10 monclient: dump: 2024-01-24T23:21:10.310 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.310 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.310 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.310 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.310 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.310 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.310 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.311 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.311 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.311 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.311 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.311 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.311 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.311 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.311 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.311 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f213d42f640 5 monclient: authenticate success, global_id 74272 2024-01-24T23:21:10.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f213d42f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:10.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 1 -- 172.21.15.115:0/3340486252 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1683806975 0 0) 0x7f2b24073640 con 0x7f2b24141b00 2024-01-24T23:21:10.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 10 monclient(hunting): my global_id is 4260 2024-01-24T23:21:10.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 1 -- 172.21.15.115:0/3340486252 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b2413bc20 con 0x7f2b24141b00 2024-01-24T23:21:10.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 1 -- 172.21.15.115:0/2979052722 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2128001830 con 0x7f2138136f40 2024-01-24T23:21:10.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 1 -- 172.21.15.115:0/3340486252 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1848706065 0 0) 0x7f2b240a0a80 con 0x7f2b2413b130 2024-01-24T23:21:10.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 1 -- 172.21.15.115:0/3340486252 >> v1:172.21.15.115:6791/0 conn(0x7f2b2413dff0 legacy=0x7f2b241403e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 1 -- 172.21.15.115:0/3340486252 >> v1:172.21.15.132:6790/0 conn(0x7f2b24141b00 legacy=0x7f2b24143ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f43626ae640 1 -- 172.21.15.115:0/1330690824 shutdown_connections 2024-01-24T23:21:10.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:10.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f43626ae640 1 -- 172.21.15.115:0/1330690824 wait complete. 2024-01-24T23:21:10.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 1 -- 172.21.15.115:0/2979052722 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2128001d60 con 0x7f2138136f40 2024-01-24T23:21:10.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:10.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2136ffd640 10 monclient: dump: 2024-01-24T23:21:10.315 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.315 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.315 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.315 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.315 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.315 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.315 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.315 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.315 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.315 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.315 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.315 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.316 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.316 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.316 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.316 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 10 monclient: _finish_hunting 0 2024-01-24T23:21:10.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 1 monclient: found mon.noname-h 2024-01-24T23:21:10.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:10.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:21:10.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 1 -- 172.21.15.115:0/3340486252 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2409f1d0 con 0x7f2b2413b130 2024-01-24T23:21:10.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 10 monclient: _finish_auth 0 2024-01-24T23:21:10.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 10 monclient: _check_auth_tickets 2024-01-24T23:21:10.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:10.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b29bce640 5 monclient: authenticate success, global_id 4280 2024-01-24T23:21:10.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b29bce640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:10.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 1 -- 172.21.15.115:0/3340486252 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2b1c001720 con 0x7f2b2413b130 2024-01-24T23:21:10.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:21:10.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 10 monclient: dump: 2024-01-24T23:21:10.317 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.317 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.317 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.318 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.318 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.318 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.318 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.318 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.318 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.318 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.318 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.318 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.318 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.318 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.318 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.319 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b29bce640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:10.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f213d42f640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:10.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f213d42f640 4 set_mon_vals no callback set 2024-01-24T23:21:10.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 1 -- 172.21.15.115:0/3340486252 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2b1c001940 con 0x7f2b2413b130 2024-01-24T23:21:10.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b237fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:10.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b29bce640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:10.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.219+0000 7f2b29bce640 4 set_mon_vals no callback set 2024-01-24T23:21:10.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.223+0000 7f213d42f640 10 monclient: shutdown 2024-01-24T23:21:10.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.223+0000 7f213d42f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.223+0000 7f213d42f640 1 -- 172.21.15.115:0/2979052722 >> v1:172.21.15.115:6791/0 conn(0x7f2138136f40 legacy=0x7f2138137320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.223+0000 7f213d42f640 1 -- 172.21.15.115:0/2979052722 shutdown_connections 2024-01-24T23:21:10.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.223+0000 7f213d42f640 1 -- 172.21.15.115:0/2979052722 >> 172.21.15.115:0/2979052722 conn(0x7f21380a5310 msgr2=0x7f21380a1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.223+0000 7f2b29bce640 10 monclient: shutdown 2024-01-24T23:21:10.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.223+0000 7f2b29bce640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.223+0000 7f2b29bce640 1 -- 172.21.15.115:0/3340486252 >> v1:172.21.15.132:6791/0 conn(0x7f2b2413b130 legacy=0x7f2b2413b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.223+0000 7f213d42f640 1 -- 172.21.15.115:0/2979052722 shutdown_connections 2024-01-24T23:21:10.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.223+0000 7f213d42f640 1 -- 172.21.15.115:0/2979052722 wait complete. 2024-01-24T23:21:10.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b29bce640 1 -- 172.21.15.115:0/3340486252 shutdown_connections 2024-01-24T23:21:10.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b29bce640 1 -- 172.21.15.115:0/3340486252 >> 172.21.15.115:0/3340486252 conn(0x7f2b240a5310 msgr2=0x7f2b240a1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b29bce640 1 -- 172.21.15.115:0/3340486252 shutdown_connections 2024-01-24T23:21:10.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b29bce640 1 -- 172.21.15.115:0/3340486252 wait complete. 2024-01-24T23:21:10.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f213d42f640 10 monclient: build_initial_monmap 2024-01-24T23:21:10.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f213d42f640 10 monclient: monmap: 2024-01-24T23:21:10.323 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:10.323 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:10.323 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:10.323 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:10.324 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:10.324 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:10.324 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:10.324 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:10.324 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:10.324 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:10.324 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:10.324 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:10.324 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:10.324 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:10.324 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:10.324 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f213d42f640 1 Processor -- start 2024-01-24T23:21:10.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f213d42f640 1 -- start start 2024-01-24T23:21:10.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f213d42f640 10 monclient: init 2024-01-24T23:21:10.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b29bce640 10 monclient: build_initial_monmap 2024-01-24T23:21:10.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b29bce640 10 monclient: monmap: 2024-01-24T23:21:10.325 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:10.325 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:10.325 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:10.325 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:10.326 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:10.326 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:10.326 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:10.326 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:10.326 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:10.326 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:10.326 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:10.326 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:10.326 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:10.326 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:10.326 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:10.326 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b29bce640 1 Processor -- start 2024-01-24T23:21:10.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f213d42f640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:10.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f213d42f640 10 monclient: _add_conns ranks=[7,0,8,2,4,5,6,1,3] 2024-01-24T23:21:10.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f213d42f640 10 monclient(hunting): picked mon.noname-i con 0x7f2138136970 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:10.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f213d42f640 10 monclient(hunting): picked mon.noname-a con 0x7f2138136f40 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:10.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f213d42f640 10 monclient(hunting): picked mon.noname-e con 0x7f213813d3b0 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:10.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f213d42f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21381377e0 con 0x7f2138136f40 2024-01-24T23:21:10.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b29bce640 1 -- start start 2024-01-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b29bce640 10 monclient: init 2024-01-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b29bce640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b29bce640 10 monclient: _add_conns ranks=[8,2,0,1,5,6,4,3,7] 2024-01-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b29bce640 10 monclient(hunting): picked mon.noname-e con 0x7f2b2413b130 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b29bce640 10 monclient(hunting): picked mon.noname-b con 0x7f2b2413dff0 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b29bce640 10 monclient(hunting): picked mon.noname-a con 0x7f2b24141b00 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b29bce640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b2413f1f0 con 0x7f2b24141b00 2024-01-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b29bce640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b24140070 con 0x7f2b2413dff0 2024-01-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b29bce640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b240a1120 con 0x7f2b2413b130 2024-01-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b29bce640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b29bce640 10 monclient(hunting): authenticate will time out at 949.154465s 2024-01-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b23fff640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f2b2413dff0 0x7f2b2413eae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:54528/0 (socket says 172.21.15.115:54528) 2024-01-24T23:21:10.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b23fff640 1 -- 172.21.15.115:0/4272285455 learned_addr learned my addr 172.21.15.115:0/4272285455 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:10.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f213d42f640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f213809e9c0 con 0x7f2138136970 2024-01-24T23:21:10.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f213d42f640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21380a00e0 con 0x7f213813d3b0 2024-01-24T23:21:10.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f213d42f640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:10.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f213d42f640 10 monclient(hunting): authenticate will time out at 949.154868s 2024-01-24T23:21:10.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f213cc2e640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f213813d3b0 0x7f21381d38e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:48758/0 (socket says 172.21.15.115:48758) 2024-01-24T23:21:10.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f213cc2e640 1 -- 172.21.15.115:0/1665452785 learned_addr learned my addr 172.21.15.115:0/1665452785 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:10.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2134ff9640 1 -- 172.21.15.115:0/1665452785 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2743608108 0 0) 0x7f213809e9c0 con 0x7f2138136970 2024-01-24T23:21:10.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2134ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2134ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2134ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2134ff9640 10 monclient(hunting): my global_id is 4291 2024-01-24T23:21:10.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2134ff9640 1 -- 172.21.15.115:0/1665452785 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2124003530 con 0x7f2138136970 2024-01-24T23:21:10.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2134ff9640 1 -- 172.21.15.115:0/1665452785 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 772873506 0 0) 0x7f21380a00e0 con 0x7f213813d3b0 2024-01-24T23:21:10.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2134ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2134ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2134ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2134ff9640 10 monclient(hunting): my global_id is 74222 2024-01-24T23:21:10.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2134ff9640 1 -- 172.21.15.115:0/1665452785 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f213809e9c0 con 0x7f213813d3b0 2024-01-24T23:21:10.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b217fa640 1 -- 172.21.15.115:0/4272285455 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1690406043 0 0) 0x7f2b240a1120 con 0x7f2b2413b130 2024-01-24T23:21:10.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b217fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b217fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b217fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b217fa640 10 monclient(hunting): my global_id is 74231 2024-01-24T23:21:10.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b217fa640 1 -- 172.21.15.115:0/4272285455 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b08003530 con 0x7f2b2413b130 2024-01-24T23:21:10.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b217fa640 1 -- 172.21.15.115:0/4272285455 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 367105758 0 0) 0x7f2b2413f1f0 con 0x7f2b24141b00 2024-01-24T23:21:10.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b217fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b217fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b217fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b217fa640 10 monclient(hunting): my global_id is 4320 2024-01-24T23:21:10.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2b217fa640 1 -- 172.21.15.115:0/4272285455 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b240a1120 con 0x7f2b24141b00 2024-01-24T23:21:10.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2134ff9640 1 -- 172.21.15.115:0/1665452785 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3454839217 0 0) 0x7f21381377e0 con 0x7f2138136f40 2024-01-24T23:21:10.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2134ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2134ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2134ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2134ff9640 10 monclient(hunting): my global_id is 4311 2024-01-24T23:21:10.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.227+0000 7f2134ff9640 1 -- 172.21.15.115:0/1665452785 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21380a00e0 con 0x7f2138136f40 2024-01-24T23:21:10.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 1 -- 172.21.15.115:0/1665452785 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3425957324 0 0) 0x7f213809e9c0 con 0x7f213813d3b0 2024-01-24T23:21:10.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 1 -- 172.21.15.115:0/1665452785 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f21381377e0 con 0x7f213813d3b0 2024-01-24T23:21:10.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 1 -- 172.21.15.115:0/1665452785 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2249402677 0 0) 0x7f2124003530 con 0x7f2138136970 2024-01-24T23:21:10.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 1 -- 172.21.15.115:0/1665452785 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f213809e9c0 con 0x7f2138136970 2024-01-24T23:21:10.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 1 -- 172.21.15.115:0/1665452785 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2130001800 con 0x7f213813d3b0 2024-01-24T23:21:10.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:21:10.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 10 monclient(hunting): dump: 2024-01-24T23:21:10.335 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.335 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.335 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.335 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.336 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.336 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.336 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.336 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.336 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.336 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.336 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.336 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.336 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.336 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.337 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.337 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:10.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 1 -- 172.21.15.115:0/1665452785 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2128001550 con 0x7f2138136970 2024-01-24T23:21:10.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-01-24T23:21:10.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 10 monclient(hunting): dump: 2024-01-24T23:21:10.337 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.337 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.337 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.337 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.337 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.338 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.338 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.338 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.338 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.338 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.338 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.338 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.338 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.338 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.338 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.338 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 1 -- 172.21.15.115:0/1665452785 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3110243310 0 0) 0x7f213809e9c0 con 0x7f2138136970 2024-01-24T23:21:10.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 1 -- 172.21.15.115:0/4272285455 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4199675259 0 0) 0x7f2b08003530 con 0x7f2b2413b130 2024-01-24T23:21:10.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 1 -- 172.21.15.115:0/1665452785 >> v1:172.21.15.115:6793/0 conn(0x7f213813d3b0 legacy=0x7f21381d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 1 -- 172.21.15.115:0/1665452785 >> v1:172.21.15.115:6789/0 conn(0x7f2138136f40 legacy=0x7f2138138170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:21:10.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 1 monclient: found mon.i 2024-01-24T23:21:10.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:10.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:10.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 1 -- 172.21.15.115:0/1665452785 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2138135e70 con 0x7f2138136970 2024-01-24T23:21:10.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 10 monclient: _finish_auth 0 2024-01-24T23:21:10.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:21:10.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:10.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f213d42f640 5 monclient: authenticate success, global_id 4291 2024-01-24T23:21:10.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f213d42f640 10 monclient: _renew_subs 2024-01-24T23:21:10.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f213d42f640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:10.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f213d42f640 1 -- 172.21.15.115:0/1665452785 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21381393e0 con 0x7f2138136970 2024-01-24T23:21:10.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f213d42f640 10 monclient: _renew_subs 2024-01-24T23:21:10.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f213d42f640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:10.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f213d42f640 1 -- 172.21.15.115:0/1665452785 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f213813cd90 con 0x7f2138136970 2024-01-24T23:21:10.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 1 -- 172.21.15.115:0/4272285455 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2b2413f1f0 con 0x7f2b2413b130 2024-01-24T23:21:10.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 1 -- 172.21.15.115:0/4272285455 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2b1c004eb0 con 0x7f2b2413b130 2024-01-24T23:21:10.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:21:10.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 10 monclient(hunting): dump: 2024-01-24T23:21:10.342 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.342 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.342 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.342 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.342 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.342 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.342 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.343 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.343 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.343 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.343 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.343 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.343 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.343 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.343 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.343 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:10.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 1 -- 172.21.15.115:0/4272285455 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3782332646 0 0) 0x7f2b240a1120 con 0x7f2b24141b00 2024-01-24T23:21:10.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f213d42f640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:10.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f213d42f640 1 -- 172.21.15.115:0/1665452785 --> v1:172.21.15.132:6792/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f21380a0d40 con 0x7f2138136970 2024-01-24T23:21:10.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 1 -- 172.21.15.115:0/4272285455 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2b08003530 con 0x7f2b24141b00 2024-01-24T23:21:10.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 1 -- 172.21.15.115:0/4272285455 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3427427824 0 0) 0x7f2b2413f1f0 con 0x7f2b2413b130 2024-01-24T23:21:10.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 1 -- 172.21.15.115:0/4272285455 >> v1:172.21.15.115:6790/0 conn(0x7f2b2413dff0 legacy=0x7f2b2413eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 1 -- 172.21.15.115:0/4272285455 >> v1:172.21.15.115:6789/0 conn(0x7f2b24141b00 legacy=0x7f2b241d7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 10 monclient: _finish_hunting 0 2024-01-24T23:21:10.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 1 monclient: found mon.e 2024-01-24T23:21:10.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:10.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:10.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 1 -- 172.21.15.115:0/4272285455 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2406ceb0 con 0x7f2b2413b130 2024-01-24T23:21:10.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 10 monclient: _finish_auth 0 2024-01-24T23:21:10.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 10 monclient: _check_auth_tickets 2024-01-24T23:21:10.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:10.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 1 -- 172.21.15.115:0/1665452785 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2128004cd0 con 0x7f2138136970 2024-01-24T23:21:10.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:10.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 1 -- 172.21.15.115:0/1665452785 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2128005180 con 0x7f2138136970 2024-01-24T23:21:10.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:21:10.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 10 monclient: dump: 2024-01-24T23:21:10.347 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.347 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.347 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.348 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.348 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.348 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.348 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.348 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.348 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.348 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.348 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.348 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.348 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.348 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.349 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.349 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f21367fc640 4 set_mon_vals no callback set 2024-01-24T23:21:10.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 1 -- 172.21.15.115:0/1665452785 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 4) v1 ==== 43232+0+0 (unknown 913048011 0 0) 0x7f212800fbd0 con 0x7f2138136970 2024-01-24T23:21:10.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 1 -- 172.21.15.115:0/1665452785 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f2128043520 con 0x7f2138136970 2024-01-24T23:21:10.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 1 -- 172.21.15.115:0/1665452785 --> v1:172.21.15.132:6800/1048681489 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2124042090 con 0x7f212403e420 2024-01-24T23:21:10.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 1 -- 172.21.15.115:0/1665452785 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (unknown 2942983318 0 0) 0x7f2128043860 con 0x7f2138136970 2024-01-24T23:21:10.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2134ff9640 10 monclient: handle_get_version_reply finishing 1 version 15 2024-01-24T23:21:10.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f21377fe640 10 monclient: get_auth_request con 0x7f2124037830 auth_method 0 2024-01-24T23:21:10.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f213cc2e640 10 monclient: get_auth_request con 0x7f212403e420 auth_method 0 2024-01-24T23:21:10.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b29bce640 5 monclient: authenticate success, global_id 74231 2024-01-24T23:21:10.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b29bce640 10 monclient: _renew_subs 2024-01-24T23:21:10.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b29bce640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:10.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b29bce640 1 -- 172.21.15.115:0/4272285455 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2413fdf0 con 0x7f2b2413b130 2024-01-24T23:21:10.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b29bce640 10 monclient: _renew_subs 2024-01-24T23:21:10.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b29bce640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:10.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b29bce640 1 -- 172.21.15.115:0/4272285455 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b24140290 con 0x7f2b2413b130 2024-01-24T23:21:10.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 1 -- 172.21.15.115:0/4272285455 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2b1c0052c0 con 0x7f2b2413b130 2024-01-24T23:21:10.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:10.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 1 -- 172.21.15.115:0/4272285455 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2b1c005770 con 0x7f2b2413b130 2024-01-24T23:21:10.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b22ffd640 4 set_mon_vals no callback set 2024-01-24T23:21:10.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.231+0000 7f2b217fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:21:10.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.235+0000 7f2b217fa640 10 monclient: dump: 2024-01-24T23:21:10.351 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.351 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.351 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.351 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.352 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.352 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.352 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.352 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.352 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.352 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.352 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.352 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.352 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.352 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.352 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.352 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.235+0000 7f2b217fa640 1 -- 172.21.15.115:0/4272285455 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 4) v1 ==== 43232+0+0 (unknown 913048011 0 0) 0x7f2b1c001ab0 con 0x7f2b2413b130 2024-01-24T23:21:10.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.235+0000 7f2b217fa640 1 -- 172.21.15.115:0/4272285455 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f2b1c043600 con 0x7f2b2413b130 2024-01-24T23:21:10.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.235+0000 7f2b23fff640 10 monclient: get_auth_request con 0x7f2b08037350 auth_method 0 2024-01-24T23:21:10.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.235+0000 7f2134ff9640 1 -- 172.21.15.115:0/1665452785 <== osd.3 v1:172.21.15.132:6800/1048681489 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f2124042090 con 0x7f212403e420 2024-01-24T23:21:10.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.235+0000 7f2b29bce640 1 -- 172.21.15.115:0/4272285455 --> v1:172.21.15.132:6807/3601895184 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b000053b0 con 0x7f2b00001640 2024-01-24T23:21:10.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.235+0000 7f2b293cd640 10 monclient: get_auth_request con 0x7f2b00001640 auth_method 0 2024-01-24T23:21:10.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.239+0000 7f62c2cd5640 1 -- 172.21.15.115:0/3718224289 --> v1:172.21.15.115:6800/2776240662 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f62b4003a90 con 0x7f62a003e720 2024-01-24T23:21:10.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.239+0000 7f2b217fa640 1 -- 172.21.15.115:0/4272285455 <== osd.4 v1:172.21.15.132:6807/3601895184 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f2b000053b0 con 0x7f2b00001640 2024-01-24T23:21:10.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.239+0000 7f62b97fa640 1 -- 172.21.15.115:0/3718224289 <== osd.2 v1:172.21.15.115:6800/2776240662 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2570787547) 0x7f62b4003a90 con 0x7f62a003e720 2024-01-24T23:21:10.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.239+0000 7f62c2cd5640 1 -- 172.21.15.115:0/3718224289 >> v1:172.21.15.115:6800/2776240662 conn(0x7f62a003e720 legacy=0x7f62a0040b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.239+0000 7f62c2cd5640 1 -- 172.21.15.115:0/3718224289 >> v1:172.21.15.132:6812/1835401922 conn(0x7f62a0037b30 legacy=0x7f62a0039ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.239+0000 7f62c2cd5640 10 monclient: shutdown 2024-01-24T23:21:10.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.239+0000 7f62c2cd5640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.239+0000 7f62c2cd5640 1 -- 172.21.15.115:0/3718224289 >> v1:172.21.15.115:6791/0 conn(0x7f62b40c2bf0 legacy=0x7f62b400b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.356 INFO:teuthology.orchestra.run.smithi115.stdout:47244640260 2024-01-24T23:21:10.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.239+0000 7f62c1272640 1 -- 172.21.15.115:0/3718224289 reap_dead start 2024-01-24T23:21:10.357 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-24T23:21:10.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.239+0000 7f62c2cd5640 1 -- 172.21.15.115:0/3718224289 shutdown_connections 2024-01-24T23:21:10.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.239+0000 7f62c2cd5640 1 -- 172.21.15.115:0/3718224289 >> 172.21.15.115:0/3718224289 conn(0x7f62b4007550 msgr2=0x7f62b4026250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.239+0000 7f62c2cd5640 1 -- 172.21.15.115:0/3718224289 shutdown_connections 2024-01-24T23:21:10.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.239+0000 7f62c2cd5640 1 -- 172.21.15.115:0/3718224289 wait complete. 2024-01-24T23:21:10.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.247+0000 7f237e7fc640 1 -- 172.21.15.115:0/462850181 --> v1:172.21.15.132:6803/3365641231 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2374002b60 con 0x7f237803e740 2024-01-24T23:21:10.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.247+0000 7f2390ff9640 1 -- 172.21.15.115:0/462850181 <== osd.5 v1:172.21.15.132:6803/3365641231 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2570787547) 0x7f2374002b60 con 0x7f237803e740 2024-01-24T23:21:10.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.247+0000 7f237e7fc640 1 -- 172.21.15.115:0/462850181 >> v1:172.21.15.132:6803/3365641231 conn(0x7f237803e740 legacy=0x7f2378040b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.247+0000 7f237e7fc640 1 -- 172.21.15.115:0/462850181 >> v1:172.21.15.132:6812/1835401922 conn(0x7f2378037b50 legacy=0x7f237803a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.247+0000 7f237e7fc640 10 monclient: shutdown 2024-01-24T23:21:10.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.247+0000 7f237e7fc640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.247+0000 7f237e7fc640 1 -- 172.21.15.115:0/462850181 >> v1:172.21.15.115:6792/0 conn(0x7f238c02a550 legacy=0x7f238c163080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.247+0000 7f2398fee640 1 -- 172.21.15.115:0/462850181 reap_dead start 2024-01-24T23:21:10.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.247+0000 7f237e7fc640 1 -- 172.21.15.115:0/462850181 shutdown_connections 2024-01-24T23:21:10.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.247+0000 7f237e7fc640 1 -- 172.21.15.115:0/462850181 >> 172.21.15.115:0/462850181 conn(0x7f238c007550 msgr2=0x7f238c00e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.247+0000 7f237e7fc640 1 -- 172.21.15.115:0/462850181 shutdown_connections 2024-01-24T23:21:10.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.247+0000 7f237e7fc640 1 -- 172.21.15.115:0/462850181 wait complete. 2024-01-24T23:21:10.362 INFO:teuthology.orchestra.run.smithi115.stdout:47244640259 2024-01-24T23:21:10.362 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-24T23:21:10.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.255+0000 7f2b29bce640 1 -- 172.21.15.115:0/4272285455 --> v1:172.21.15.132:6807/3601895184 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2b00006fa0 con 0x7f2b00001640 2024-01-24T23:21:10.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.259+0000 7f2b217fa640 1 -- 172.21.15.115:0/4272285455 <== osd.4 v1:172.21.15.132:6807/3601895184 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2570787547) 0x7f2b00006fa0 con 0x7f2b00001640 2024-01-24T23:21:10.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.259+0000 7f2b29bce640 1 -- 172.21.15.115:0/4272285455 >> v1:172.21.15.132:6807/3601895184 conn(0x7f2b00001640 legacy=0x7f2b00003b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.259+0000 7f2b29bce640 1 -- 172.21.15.115:0/4272285455 >> v1:172.21.15.132:6812/1835401922 conn(0x7f2b08037350 legacy=0x7f2b08039810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.259+0000 7f2b29bce640 10 monclient: shutdown 2024-01-24T23:21:10.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.259+0000 7f2b29bce640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.259+0000 7f2b29bce640 1 -- 172.21.15.115:0/4272285455 >> v1:172.21.15.115:6793/0 conn(0x7f2b2413b130 legacy=0x7f2b2413e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.259+0000 7f2b293cd640 1 -- 172.21.15.115:0/4272285455 reap_dead start 2024-01-24T23:21:10.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.259+0000 7f2b29bce640 1 -- 172.21.15.115:0/4272285455 shutdown_connections 2024-01-24T23:21:10.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.259+0000 7f2b29bce640 1 -- 172.21.15.115:0/4272285455 >> 172.21.15.115:0/4272285455 conn(0x7f2b240a5310 msgr2=0x7f2b24074200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.259+0000 7f2b29bce640 1 -- 172.21.15.115:0/4272285455 shutdown_connections 2024-01-24T23:21:10.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.259+0000 7f2b29bce640 1 -- 172.21.15.115:0/4272285455 wait complete. 2024-01-24T23:21:10.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.263+0000 7f213d42f640 1 -- 172.21.15.115:0/1665452785 --> v1:172.21.15.132:6800/1048681489 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f213804ed70 con 0x7f212403e420 2024-01-24T23:21:10.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.263+0000 7f2134ff9640 1 -- 172.21.15.115:0/1665452785 <== osd.3 v1:172.21.15.132:6800/1048681489 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 654699149) 0x7f2130003de0 con 0x7f212403e420 2024-01-24T23:21:10.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.263+0000 7f213d42f640 1 -- 172.21.15.115:0/1665452785 >> v1:172.21.15.132:6800/1048681489 conn(0x7f212403e420 legacy=0x7f2124040840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.263+0000 7f213d42f640 1 -- 172.21.15.115:0/1665452785 >> v1:172.21.15.132:6812/1835401922 conn(0x7f2124037830 legacy=0x7f2124039cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.263+0000 7f213d42f640 10 monclient: shutdown 2024-01-24T23:21:10.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.263+0000 7f213d42f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.263+0000 7f213d42f640 1 -- 172.21.15.115:0/1665452785 >> v1:172.21.15.132:6792/0 conn(0x7f2138136970 legacy=0x7f2138137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.263+0000 7f213cc2e640 1 -- 172.21.15.115:0/1665452785 reap_dead start 2024-01-24T23:21:10.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.263+0000 7f213d42f640 1 -- 172.21.15.115:0/1665452785 shutdown_connections 2024-01-24T23:21:10.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.263+0000 7f213d42f640 1 -- 172.21.15.115:0/1665452785 >> 172.21.15.115:0/1665452785 conn(0x7f21380a5310 msgr2=0x7f2138074200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.263+0000 7f213d42f640 1 -- 172.21.15.115:0/1665452785 shutdown_connections 2024-01-24T23:21:10.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.263+0000 7f213d42f640 1 -- 172.21.15.115:0/1665452785 wait complete. 2024-01-24T23:21:10.368 INFO:teuthology.orchestra.run.smithi115.stdout:47244640258 2024-01-24T23:21:10.369 INFO:teuthology.orchestra.run.smithi115.stdout:47244640258 2024-01-24T23:21:10.369 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-01-24T23:21:10.370 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-24T23:21:10.371 INFO:teuthology.orchestra.run.smithi115.stdout:47244640258 2024-01-24T23:21:10.371 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-01-24T23:21:10.372 INFO:teuthology.orchestra.run.smithi115.stdout:47244640260 2024-01-24T23:21:10.373 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-24T23:21:10.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9feea640 10 monclient: get_monmap_and_config 2024-01-24T23:21:10.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9feea640 10 monclient: build_initial_monmap 2024-01-24T23:21:10.527 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9feea640 10 monclient: monmap: 2024-01-24T23:21:10.527 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:10.527 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.528 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:10.528297+0000 2024-01-24T23:21:10.528 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:10.528297+0000 2024-01-24T23:21:10.528 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:10.528 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.528 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:10.528 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:10.528 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:10.528 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:10.528 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:10.528 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:10.528 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:10.528 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:10.529 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:10.529 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9feea640 1 Processor -- start 2024-01-24T23:21:10.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9feea640 1 -- start start 2024-01-24T23:21:10.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9feea640 10 monclient: init 2024-01-24T23:21:10.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9feea640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:10.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9feea640 10 monclient: _add_conns ranks=[3,4,6,5,2,1,7,0,8] 2024-01-24T23:21:10.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9feea640 10 monclient(hunting): picked mon.noname-g con 0x7f9b98133bf0 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:10.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9feea640 10 monclient(hunting): picked mon.noname-c con 0x7f9b98133fd0 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:10.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9feea640 10 monclient(hunting): picked mon.noname-d con 0x7f9b981343b0 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:10.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9feea640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b9804ed70 con 0x7f9b98133bf0 2024-01-24T23:21:10.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9feea640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b980a4b60 con 0x7f9b98133fd0 2024-01-24T23:21:10.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9feea640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b9809ae50 con 0x7f9b981343b0 2024-01-24T23:21:10.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9feea640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:10.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9feea640 10 monclient(hunting): authenticate will time out at 949.450749s 2024-01-24T23:21:10.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9d485640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f9b98133fd0 0x7f9b9813bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:42460/0 (socket says 172.21.15.115:42460) 2024-01-24T23:21:10.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9d485640 1 -- 172.21.15.115:0/1725662659 learned_addr learned my addr 172.21.15.115:0/1725662659 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:10.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9cc84640 1 -- 172.21.15.115:0/1725662659 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844300368 0 0) 0x7f9b980a4b60 con 0x7f9b98133fd0 2024-01-24T23:21:10.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9cc84640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9cc84640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9cc84640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9cc84640 10 monclient(hunting): my global_id is 74299 2024-01-24T23:21:10.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9cc84640 1 -- 172.21.15.115:0/1725662659 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b84003530 con 0x7f9b98133fd0 2024-01-24T23:21:10.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9cc84640 1 -- 172.21.15.115:0/1725662659 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1324930031 0 0) 0x7f9b9809ae50 con 0x7f9b981343b0 2024-01-24T23:21:10.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9cc84640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9cc84640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9cc84640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9cc84640 10 monclient(hunting): my global_id is 4326 2024-01-24T23:21:10.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9cc84640 1 -- 172.21.15.115:0/1725662659 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b980a4b60 con 0x7f9b981343b0 2024-01-24T23:21:10.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9cc84640 1 -- 172.21.15.115:0/1725662659 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2073598928 0 0) 0x7f9b9804ed70 con 0x7f9b98133bf0 2024-01-24T23:21:10.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9cc84640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9cc84640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9cc84640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9cc84640 10 monclient(hunting): my global_id is 4278 2024-01-24T23:21:10.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9cc84640 1 -- 172.21.15.115:0/1725662659 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b9809ae50 con 0x7f9b98133bf0 2024-01-24T23:21:10.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9cc84640 1 -- 172.21.15.115:0/1725662659 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1606424168 0 0) 0x7f9b84003530 con 0x7f9b98133fd0 2024-01-24T23:21:10.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9cc84640 1 -- 172.21.15.115:0/1725662659 >> v1:172.21.15.115:6792/0 conn(0x7f9b981343b0 legacy=0x7f9b9813f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.523+0000 7f9b9cc84640 1 -- 172.21.15.115:0/1725662659 >> v1:172.21.15.132:6790/0 conn(0x7f9b98133bf0 legacy=0x7f9b9809bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9cc84640 10 monclient: _finish_hunting 0 2024-01-24T23:21:10.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9cc84640 1 monclient: found mon.noname-c 2024-01-24T23:21:10.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9cc84640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:10.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9cc84640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:21:10.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9cc84640 1 -- 172.21.15.115:0/1725662659 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9809b130 con 0x7f9b98133fd0 2024-01-24T23:21:10.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9cc84640 10 monclient: _finish_auth 0 2024-01-24T23:21:10.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9cc84640 10 monclient: _check_auth_tickets 2024-01-24T23:21:10.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9cc84640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:10.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9cc84640 1 -- 172.21.15.115:0/1725662659 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9b88001740 con 0x7f9b98133fd0 2024-01-24T23:21:10.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9cc84640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9cc84640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:21:10.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9cc84640 10 monclient: dump: 2024-01-24T23:21:10.537 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.537 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.538 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.538 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.538 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.538 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.538 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.538 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.538 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.538 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.538 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.538 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.538 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.538 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.538 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.538 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 5 monclient: authenticate success, global_id 74299 2024-01-24T23:21:10.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:10.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9cc84640 1 -- 172.21.15.115:0/1725662659 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9b880019d0 con 0x7f9b98133fd0 2024-01-24T23:21:10.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9cc84640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:10.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9cc84640 1 -- 172.21.15.115:0/1725662659 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9b88004bf0 con 0x7f9b98133fd0 2024-01-24T23:21:10.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9cc84640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9cc84640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:10.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9cc84640 10 monclient: dump: 2024-01-24T23:21:10.539 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.539 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.539 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.539 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.539 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.539 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.539 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.539 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.539 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.540 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.540 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.540 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.540 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.540 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.540 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.540 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:10.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 4 set_mon_vals no callback set 2024-01-24T23:21:10.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 10 monclient: shutdown 2024-01-24T23:21:10.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 1 -- 172.21.15.115:0/1725662659 >> v1:172.21.15.115:6791/0 conn(0x7f9b98133fd0 legacy=0x7f9b9813bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 1 -- 172.21.15.115:0/1725662659 shutdown_connections 2024-01-24T23:21:10.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 1 -- 172.21.15.115:0/1725662659 >> 172.21.15.115:0/1725662659 conn(0x7f9b980662f0 msgr2=0x7f9b98066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 1 -- 172.21.15.115:0/1725662659 shutdown_connections 2024-01-24T23:21:10.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 1 -- 172.21.15.115:0/1725662659 wait complete. 2024-01-24T23:21:10.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 10 monclient: build_initial_monmap 2024-01-24T23:21:10.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 10 monclient: monmap: 2024-01-24T23:21:10.541 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:10.541 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:10.541 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:10.541 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:10.541 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:10.541 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:10.541 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:10.541 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:10.541 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:10.541 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:10.541 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:10.542 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:10.542 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:10.542 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:10.542 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:10.542 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 1 Processor -- start 2024-01-24T23:21:10.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 1 -- start start 2024-01-24T23:21:10.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 10 monclient: init 2024-01-24T23:21:10.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:10.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 10 monclient: _add_conns ranks=[1,6,7,5,8,3,2,4,0] 2024-01-24T23:21:10.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 10 monclient(hunting): picked mon.noname-f con 0x7f9b98133bf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:10.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 10 monclient(hunting): picked mon.noname-d con 0x7f9b98133fd0 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:10.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 10 monclient(hunting): picked mon.noname-i con 0x7f9b981343b0 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:10.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b9809c810 con 0x7f9b98133bf0 2024-01-24T23:21:10.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b980a5990 con 0x7f9b98133fd0 2024-01-24T23:21:10.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b981208f0 con 0x7f9b981343b0 2024-01-24T23:21:10.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:10.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9d485640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f9b98133fd0 0x7f9b981d0680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:56936/0 (socket says 172.21.15.115:56936) 2024-01-24T23:21:10.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9d485640 1 -- 172.21.15.115:0/304887310 learned_addr learned my addr 172.21.15.115:0/304887310 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:10.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b9feea640 10 monclient(hunting): authenticate will time out at 949.455345s 2024-01-24T23:21:10.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b8e7fc640 1 -- 172.21.15.115:0/304887310 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1733663707 0 0) 0x7f9b981208f0 con 0x7f9b981343b0 2024-01-24T23:21:10.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b8e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b8e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b8e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b8e7fc640 10 monclient(hunting): my global_id is 4300 2024-01-24T23:21:10.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b8e7fc640 1 -- 172.21.15.115:0/304887310 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b7c003530 con 0x7f9b981343b0 2024-01-24T23:21:10.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b8e7fc640 1 -- 172.21.15.115:0/304887310 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2190124559 0 0) 0x7f9b980a5990 con 0x7f9b98133fd0 2024-01-24T23:21:10.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b8e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b8e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b8e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b8e7fc640 10 monclient(hunting): my global_id is 4335 2024-01-24T23:21:10.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b8e7fc640 1 -- 172.21.15.115:0/304887310 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b981208f0 con 0x7f9b98133fd0 2024-01-24T23:21:10.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b8e7fc640 1 -- 172.21.15.115:0/304887310 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 775508984 0 0) 0x7f9b9809c810 con 0x7f9b98133bf0 2024-01-24T23:21:10.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b8e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b8e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b8e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b8e7fc640 10 monclient(hunting): my global_id is 4267 2024-01-24T23:21:10.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b8e7fc640 1 -- 172.21.15.115:0/304887310 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b980a5990 con 0x7f9b98133bf0 2024-01-24T23:21:10.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.527+0000 7f9b8e7fc640 1 -- 172.21.15.115:0/304887310 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2960894079 0 0) 0x7f9b7c003530 con 0x7f9b981343b0 2024-01-24T23:21:10.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 1 -- 172.21.15.115:0/304887310 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9b9809c810 con 0x7f9b981343b0 2024-01-24T23:21:10.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 1 -- 172.21.15.115:0/304887310 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9b90001d00 con 0x7f9b981343b0 2024-01-24T23:21:10.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:21:10.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 10 monclient(hunting): dump: 2024-01-24T23:21:10.545 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.545 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.545 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.545 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.545 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.545 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.545 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.545 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.545 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.545 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.545 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.545 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.545 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.545 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.545 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.546 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:10.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 1 -- 172.21.15.115:0/304887310 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1372416027 0 0) 0x7f9b981208f0 con 0x7f9b98133fd0 2024-01-24T23:21:10.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 1 -- 172.21.15.115:0/304887310 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9b7c003530 con 0x7f9b98133fd0 2024-01-24T23:21:10.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 1 -- 172.21.15.115:0/304887310 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9b88004b70 con 0x7f9b98133fd0 2024-01-24T23:21:10.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:10.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 10 monclient(hunting): dump: 2024-01-24T23:21:10.546 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.546 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.546 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.546 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.546 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.546 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.546 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.546 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.547 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.547 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.547 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.547 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.547 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.547 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.547 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.547 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 1 -- 172.21.15.115:0/304887310 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3726571747 0 0) 0x7f9b9809c810 con 0x7f9b981343b0 2024-01-24T23:21:10.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 1 -- 172.21.15.115:0/304887310 >> v1:172.21.15.115:6792/0 conn(0x7f9b98133fd0 legacy=0x7f9b981d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 1 -- 172.21.15.115:0/304887310 >> v1:172.21.15.132:6789/0 conn(0x7f9b98133bf0 legacy=0x7f9b98132e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:21:10.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 1 monclient: found mon.i 2024-01-24T23:21:10.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:10.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:10.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 1 -- 172.21.15.115:0/304887310 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b980a2450 con 0x7f9b981343b0 2024-01-24T23:21:10.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 10 monclient: _finish_auth 0 2024-01-24T23:21:10.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:21:10.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:10.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b9feea640 5 monclient: authenticate success, global_id 4300 2024-01-24T23:21:10.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b9feea640 10 monclient: _renew_subs 2024-01-24T23:21:10.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 1 -- 172.21.15.115:0/304887310 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9b90003980 con 0x7f9b981343b0 2024-01-24T23:21:10.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b9feea640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:10.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b9feea640 1 -- 172.21.15.115:0/304887310 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b98120610 con 0x7f9b981343b0 2024-01-24T23:21:10.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:10.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 1 -- 172.21.15.115:0/304887310 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9b90003e30 con 0x7f9b981343b0 2024-01-24T23:21:10.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:21:10.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 10 monclient: dump: 2024-01-24T23:21:10.548 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.549 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.549 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.549 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.549 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.549 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.549 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.549 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.549 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.549 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.549 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.549 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.549 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.549 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.549 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.549 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8ffff640 4 set_mon_vals no callback set 2024-01-24T23:21:10.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b9feea640 10 monclient: _renew_subs 2024-01-24T23:21:10.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b9feea640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:10.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b9feea640 1 -- 172.21.15.115:0/304887310 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b980a31d0 con 0x7f9b981343b0 2024-01-24T23:21:10.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b7bfff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b7bfff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b7bfff640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b7bfff640 1 -- 172.21.15.115:0/304887310 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b74005090 con 0x7f9b981343b0 2024-01-24T23:21:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 1 -- 172.21.15.115:0/304887310 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 4) v1 ==== 43232+0+0 (unknown 913048011 0 0) 0x7f9b90004060 con 0x7f9b981343b0 2024-01-24T23:21:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b8e7fc640 1 -- 172.21.15.115:0/304887310 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f9b90042640 con 0x7f9b981343b0 2024-01-24T23:21:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.531+0000 7f9b9dc86640 10 monclient: get_auth_request con 0x7f9b7c037720 auth_method 0 2024-01-24T23:21:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21759e640 10 monclient: get_monmap_and_config 2024-01-24T23:21:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21759e640 10 monclient: build_initial_monmap 2024-01-24T23:21:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21759e640 10 monclient: monmap: 2024-01-24T23:21:10.552 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:10.552 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.552 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:10.543387+0000 2024-01-24T23:21:10.552 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:10.543387+0000 2024-01-24T23:21:10.552 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:10.552 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.552 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:10.552 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:10.552 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:10.552 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:10.552 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:10.552 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:10.552 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:10.552 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:10.553 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:10.553 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7f9b8e7fc640 1 -- 172.21.15.115:0/304887310 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f9b900136a0 con 0x7f9b981343b0 2024-01-24T23:21:10.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7f9b8e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:10.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7f9b8e7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:10.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21759e640 1 Processor -- start 2024-01-24T23:21:10.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21759e640 1 -- start start 2024-01-24T23:21:10.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21759e640 10 monclient: init 2024-01-24T23:21:10.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21759e640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:10.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21759e640 10 monclient: _add_conns ranks=[0,4,1,6,7,2,5,3,8] 2024-01-24T23:21:10.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21759e640 10 monclient(hunting): picked mon.noname-a con 0x7fd21002a550 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:10.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21759e640 10 monclient(hunting): picked mon.noname-c con 0x7fd210028270 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:10.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21759e640 10 monclient(hunting): picked mon.noname-f con 0x7fd2100c2bf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:10.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21759e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd210004070 con 0x7fd21002a550 2024-01-24T23:21:10.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21759e640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd21002a930 con 0x7fd2100c2bf0 2024-01-24T23:21:10.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21759e640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd210025a30 con 0x7fd210028270 2024-01-24T23:21:10.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21759e640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:10.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21759e640 10 monclient(hunting): authenticate will time out at 949.466327s 2024-01-24T23:21:10.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21659c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fd21002a550 0x7fd210022fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:52112/0 (socket says 172.21.15.115:52112) 2024-01-24T23:21:10.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21659c640 1 -- 172.21.15.115:0/4044430438 learned_addr learned my addr 172.21.15.115:0/4044430438 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:10.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 1 -- 172.21.15.115:0/4044430438 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706545159 0 0) 0x7fd210025a30 con 0x7fd210028270 2024-01-24T23:21:10.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 10 monclient(hunting): my global_id is 74308 2024-01-24T23:21:10.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 1 -- 172.21.15.115:0/4044430438 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd200003530 con 0x7fd210028270 2024-01-24T23:21:10.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 1 -- 172.21.15.115:0/4044430438 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2438991819 0 0) 0x7fd210004070 con 0x7fd21002a550 2024-01-24T23:21:10.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 10 monclient(hunting): my global_id is 4329 2024-01-24T23:21:10.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 1 -- 172.21.15.115:0/4044430438 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd210025a30 con 0x7fd21002a550 2024-01-24T23:21:10.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 1 -- 172.21.15.115:0/4044430438 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224865712 0 0) 0x7fd21002a930 con 0x7fd2100c2bf0 2024-01-24T23:21:10.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 10 monclient(hunting): my global_id is 4276 2024-01-24T23:21:10.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 1 -- 172.21.15.115:0/4044430438 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd210004070 con 0x7fd2100c2bf0 2024-01-24T23:21:10.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 1 -- 172.21.15.115:0/4044430438 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2924008101 0 0) 0x7fd210025a30 con 0x7fd21002a550 2024-01-24T23:21:10.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 1 -- 172.21.15.115:0/4044430438 >> v1:172.21.15.115:6791/0 conn(0x7fd210028270 legacy=0x7fd210028690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 1 -- 172.21.15.115:0/4044430438 >> v1:172.21.15.132:6789/0 conn(0x7fd2100c2bf0 legacy=0x7fd2100c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 10 monclient: _finish_hunting 0 2024-01-24T23:21:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 1 monclient: found mon.noname-a 2024-01-24T23:21:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:21:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 1 -- 172.21.15.115:0/4044430438 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd210025190 con 0x7fd21002a550 2024-01-24T23:21:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 10 monclient: _finish_auth 0 2024-01-24T23:21:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 10 monclient: _check_auth_tickets 2024-01-24T23:21:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 1 -- 172.21.15.115:0/4044430438 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd2180681a0 con 0x7fd21002a550 2024-01-24T23:21:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21759e640 5 monclient: authenticate success, global_id 4329 2024-01-24T23:21:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21759e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:21:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 10 monclient: dump: 2024-01-24T23:21:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.562 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21759e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 1 -- 172.21.15.115:0/4044430438 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd21804f7e0 con 0x7fd21002a550 2024-01-24T23:21:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21759e640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 1 -- 172.21.15.115:0/4044430438 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd218069b70 con 0x7fd21002a550 2024-01-24T23:21:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21759e640 4 set_mon_vals no callback set 2024-01-24T23:21:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.539+0000 7fd21559a640 10 monclient: dump: 2024-01-24T23:21:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.564 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd21759e640 10 monclient: shutdown 2024-01-24T23:21:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd21759e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd21759e640 1 -- 172.21.15.115:0/4044430438 >> v1:172.21.15.115:6789/0 conn(0x7fd21002a550 legacy=0x7fd210022fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd21759e640 1 -- 172.21.15.115:0/4044430438 shutdown_connections 2024-01-24T23:21:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd21759e640 1 -- 172.21.15.115:0/4044430438 >> 172.21.15.115:0/4044430438 conn(0x7fd210007550 msgr2=0x7fd210007960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd21759e640 1 -- 172.21.15.115:0/4044430438 shutdown_connections 2024-01-24T23:21:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd21759e640 1 -- 172.21.15.115:0/4044430438 wait complete. 2024-01-24T23:21:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd21759e640 10 monclient: build_initial_monmap 2024-01-24T23:21:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd21759e640 10 monclient: monmap: 2024-01-24T23:21:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:10.566 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd21759e640 1 Processor -- start 2024-01-24T23:21:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd21759e640 1 -- start start 2024-01-24T23:21:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd21759e640 10 monclient: init 2024-01-24T23:21:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd21759e640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd21759e640 10 monclient: _add_conns ranks=[1,8,3,6,4,5,2,0,7] 2024-01-24T23:21:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd21759e640 10 monclient(hunting): picked mon.noname-f con 0x7fd210028270 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd21759e640 10 monclient(hunting): picked mon.noname-e con 0x7fd21002a550 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd21759e640 10 monclient(hunting): picked mon.noname-g con 0x7fd2100c2bf0 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd21759e640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd210024af0 con 0x7fd210028270 2024-01-24T23:21:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd21759e640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2100be2d0 con 0x7fd2100c2bf0 2024-01-24T23:21:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd21759e640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd210022290 con 0x7fd21002a550 2024-01-24T23:21:10.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd21759e640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:10.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd21759e640 10 monclient(hunting): authenticate will time out at 949.471120s 2024-01-24T23:21:10.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd215d9b640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fd21002a550 0x7fd210020170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:48776/0 (socket says 172.21.15.115:48776) 2024-01-24T23:21:10.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd215d9b640 1 -- 172.21.15.115:0/2672339919 learned_addr learned my addr 172.21.15.115:0/2672339919 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:10.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd21659c640 1 --1- 172.21.15.115:0/2672339919 >> v1:172.21.15.132:6789/0 conn(0x7fd210028270 0x7fd21001fa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.115:35118/0 (socket says 172.21.15.115:35118) 2024-01-24T23:21:10.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828295324 0 0) 0x7fd210022290 con 0x7fd21002a550 2024-01-24T23:21:10.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 10 monclient(hunting): my global_id is 74240 2024-01-24T23:21:10.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1f8003530 con 0x7fd21002a550 2024-01-24T23:21:10.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 786522516 0 0) 0x7fd210025190 con 0x7fd210028270 2024-01-24T23:21:10.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 10 monclient(hunting): my global_id is 4285 2024-01-24T23:21:10.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd210022290 con 0x7fd210028270 2024-01-24T23:21:10.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550425336 0 0) 0x7fd2100be2d0 con 0x7fd2100c2bf0 2024-01-24T23:21:10.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 10 monclient(hunting): my global_id is 4287 2024-01-24T23:21:10.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd210025190 con 0x7fd2100c2bf0 2024-01-24T23:21:10.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2783401172 0 0) 0x7fd1f8003530 con 0x7fd21002a550 2024-01-24T23:21:10.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd2100be2d0 con 0x7fd21002a550 2024-01-24T23:21:10.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd20c001b00 con 0x7fd21002a550 2024-01-24T23:21:10.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:21:10.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 10 monclient(hunting): dump: 2024-01-24T23:21:10.569 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.569 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.569 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.569 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.569 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.569 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.569 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.569 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.569 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.569 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.569 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.569 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.569 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.569 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.569 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.569 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.543+0000 7fd206ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:10.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 445271934 0 0) 0x7fd210022290 con 0x7fd210028270 2024-01-24T23:21:10.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd1f8003530 con 0x7fd210028270 2024-01-24T23:21:10.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd2180681a0 con 0x7fd210028270 2024-01-24T23:21:10.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:21:10.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 10 monclient(hunting): dump: 2024-01-24T23:21:10.570 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.570 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.570 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.570 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.570 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.570 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.570 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.570 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.570 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.571 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.571 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.571 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.571 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.571 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.571 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.571 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2695512997 0 0) 0x7fd210025190 con 0x7fd2100c2bf0 2024-01-24T23:21:10.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd210022290 con 0x7fd2100c2bf0 2024-01-24T23:21:10.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd2080018c0 con 0x7fd2100c2bf0 2024-01-24T23:21:10.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:21:10.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 10 monclient(hunting): dump: 2024-01-24T23:21:10.571 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.571 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.571 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.572 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.572 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.572 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.572 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.572 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.572 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.572 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.572 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.572 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.572 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.572 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.572 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.572 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4211561149 0 0) 0x7fd2100be2d0 con 0x7fd21002a550 2024-01-24T23:21:10.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 >> v1:172.21.15.132:6790/0 conn(0x7fd2100c2bf0 legacy=0x7fd210020880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 >> v1:172.21.15.132:6789/0 conn(0x7fd210028270 legacy=0x7fd21001fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:21:10.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 1 monclient: found mon.e 2024-01-24T23:21:10.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:10.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:10.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd210024530 con 0x7fd21002a550 2024-01-24T23:21:10.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 10 monclient: _finish_auth 0 2024-01-24T23:21:10.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:21:10.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:10.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd21759e640 5 monclient: authenticate success, global_id 74240 2024-01-24T23:21:10.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd21759e640 10 monclient: _renew_subs 2024-01-24T23:21:10.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd20c0038d0 con 0x7fd21002a550 2024-01-24T23:21:10.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd21759e640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:10.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd21759e640 1 -- 172.21.15.115:0/2672339919 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2100be5b0 con 0x7fd21002a550 2024-01-24T23:21:10.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:10.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd20c003d80 con 0x7fd21002a550 2024-01-24T23:21:10.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:21:10.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 10 monclient: dump: 2024-01-24T23:21:10.574 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.574 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.574 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.574 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.574 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.574 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.574 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.574 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.574 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.574 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.574 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.575 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.575 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.575 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.575 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.575 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd214d99640 4 set_mon_vals no callback set 2024-01-24T23:21:10.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd21759e640 10 monclient: _renew_subs 2024-01-24T23:21:10.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd21759e640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:10.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd21759e640 1 -- 172.21.15.115:0/2672339919 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2100ba170 con 0x7fd21002a550 2024-01-24T23:21:10.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 4) v1 ==== 43232+0+0 (unknown 913048011 0 0) 0x7fd20c003ee0 con 0x7fd21002a550 2024-01-24T23:21:10.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7fd20c0434e0 con 0x7fd21002a550 2024-01-24T23:21:10.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd21759e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:10.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd21759e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:10.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd21759e640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:10.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd21759e640 1 -- 172.21.15.115:0/2672339919 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd210003a90 con 0x7fd21002a550 2024-01-24T23:21:10.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.547+0000 7fd21659c640 10 monclient: get_auth_request con 0x7fd1f8037be0 auth_method 0 2024-01-24T23:21:10.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.555+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fd20c0148d0 con 0x7fd21002a550 2024-01-24T23:21:10.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.555+0000 7fd206ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:10.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.555+0000 7fd206ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:10.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.571+0000 7fd0f8781640 10 monclient: get_monmap_and_config 2024-01-24T23:21:10.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.571+0000 7fd0f8781640 10 monclient: build_initial_monmap 2024-01-24T23:21:10.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.571+0000 7fd0f8781640 10 monclient: monmap: 2024-01-24T23:21:10.579 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:10.579 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.579 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:10.577668+0000 2024-01-24T23:21:10.579 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:10.577668+0000 2024-01-24T23:21:10.579 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:10.579 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.579 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:10.579 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:10.579 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:10.579 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:10.579 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:10.579 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:10.579 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:10.580 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:10.580 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:10.580 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.571+0000 7fd0f8781640 1 Processor -- start 2024-01-24T23:21:10.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.571+0000 7fd0f8781640 1 -- start start 2024-01-24T23:21:10.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.571+0000 7fd0f8781640 10 monclient: init 2024-01-24T23:21:10.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f8781640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:10.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f8781640 10 monclient: _add_conns ranks=[4,3,0,7,1,2,6,8,5] 2024-01-24T23:21:10.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f8781640 10 monclient(hunting): picked mon.noname-c con 0x7fd0f0133bf0 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:10.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f8781640 10 monclient(hunting): picked mon.noname-g con 0x7fd0f0133fd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:10.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f651d640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fd0f0133bf0 0x7fd0f009bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:42466/0 (socket says 172.21.15.115:42466) 2024-01-24T23:21:10.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f6d1e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fd0f01343b0 0x7fd0f013f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:52126/0 (socket says 172.21.15.115:52126) 2024-01-24T23:21:10.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f8781640 10 monclient(hunting): picked mon.noname-a con 0x7fd0f01343b0 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:10.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f651d640 1 -- 172.21.15.115:0/2149577543 learned_addr learned my addr 172.21.15.115:0/2149577543 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:10.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f8781640 1 -- 172.21.15.115:0/2149577543 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f004ed70 con 0x7fd0f01343b0 2024-01-24T23:21:10.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f8781640 1 -- 172.21.15.115:0/2149577543 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f00a4b60 con 0x7fd0f0133fd0 2024-01-24T23:21:10.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f8781640 1 -- 172.21.15.115:0/2149577543 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f009ae50 con 0x7fd0f0133bf0 2024-01-24T23:21:10.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 1 -- 172.21.15.115:0/2149577543 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2101089573 0 0) 0x7fd0f004ed70 con 0x7fd0f01343b0 2024-01-24T23:21:10.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f8781640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:10.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f8781640 10 monclient(hunting): authenticate will time out at 949.500995s 2024-01-24T23:21:10.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 10 monclient(hunting): my global_id is 4338 2024-01-24T23:21:10.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 1 -- 172.21.15.115:0/2149577543 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0dc003530 con 0x7fd0f01343b0 2024-01-24T23:21:10.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 1 -- 172.21.15.115:0/2149577543 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825708614 0 0) 0x7fd0f00a4b60 con 0x7fd0f0133fd0 2024-01-24T23:21:10.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 10 monclient(hunting): my global_id is 4296 2024-01-24T23:21:10.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 1 -- 172.21.15.115:0/2149577543 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0f004ed70 con 0x7fd0f0133fd0 2024-01-24T23:21:10.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 1 -- 172.21.15.115:0/2149577543 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 852991937 0 0) 0x7fd0f009ae50 con 0x7fd0f0133bf0 2024-01-24T23:21:10.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 10 monclient(hunting): my global_id is 74317 2024-01-24T23:21:10.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 1 -- 172.21.15.115:0/2149577543 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0f00a4b60 con 0x7fd0f0133bf0 2024-01-24T23:21:10.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 1 -- 172.21.15.115:0/2149577543 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3929969762 0 0) 0x7fd0dc003530 con 0x7fd0f01343b0 2024-01-24T23:21:10.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 1 -- 172.21.15.115:0/2149577543 >> v1:172.21.15.115:6791/0 conn(0x7fd0f0133bf0 legacy=0x7fd0f009bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 1 -- 172.21.15.115:0/2149577543 >> v1:172.21.15.132:6790/0 conn(0x7fd0f0133fd0 legacy=0x7fd0f013bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 10 monclient: _finish_hunting 0 2024-01-24T23:21:10.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 1 monclient: found mon.noname-a 2024-01-24T23:21:10.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:10.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:21:10.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 1 -- 172.21.15.115:0/2149577543 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f009b130 con 0x7fd0f01343b0 2024-01-24T23:21:10.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 10 monclient: _finish_auth 0 2024-01-24T23:21:10.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 10 monclient: _check_auth_tickets 2024-01-24T23:21:10.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:10.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 1 -- 172.21.15.115:0/2149577543 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd0e8001b40 con 0x7fd0f01343b0 2024-01-24T23:21:10.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f8781640 5 monclient: authenticate success, global_id 4338 2024-01-24T23:21:10.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f8781640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:10.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:21:10.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 10 monclient: dump: 2024-01-24T23:21:10.583 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.583 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.583 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.583 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.583 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.584 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f8781640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 1 -- 172.21.15.115:0/2149577543 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd0e8001d60 con 0x7fd0f01343b0 2024-01-24T23:21:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:10.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 1 -- 172.21.15.115:0/2149577543 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd0e8004eb0 con 0x7fd0f01343b0 2024-01-24T23:21:10.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f8781640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:10.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f8781640 4 set_mon_vals no callback set 2024-01-24T23:21:10.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:10.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f551b640 10 monclient: dump: 2024-01-24T23:21:10.585 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.585 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.585 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.585 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.585 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.585 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.585 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.585 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.585 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.585 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.585 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.586 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.586 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.586 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.586 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.586 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f8781640 10 monclient: shutdown 2024-01-24T23:21:10.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f8781640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f8781640 1 -- 172.21.15.115:0/2149577543 >> v1:172.21.15.115:6789/0 conn(0x7fd0f01343b0 legacy=0x7fd0f013f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f8781640 1 -- 172.21.15.115:0/2149577543 shutdown_connections 2024-01-24T23:21:10.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f8781640 1 -- 172.21.15.115:0/2149577543 >> 172.21.15.115:0/2149577543 conn(0x7fd0f00662f0 msgr2=0x7fd0f0066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f8781640 1 -- 172.21.15.115:0/2149577543 shutdown_connections 2024-01-24T23:21:10.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f8781640 1 -- 172.21.15.115:0/2149577543 wait complete. 2024-01-24T23:21:10.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f8781640 10 monclient: build_initial_monmap 2024-01-24T23:21:10.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.575+0000 7fd0f8781640 10 monclient: monmap: 2024-01-24T23:21:10.587 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:10.587 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:10.587 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:10.588 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:10.588 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:10.588 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:10.588 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:10.588 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:10.588 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:10.588 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:10.588 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:10.588 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:10.588 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:10.588 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:10.588 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:10.588 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7fd0f8781640 1 Processor -- start 2024-01-24T23:21:10.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7fd0f8781640 1 -- start start 2024-01-24T23:21:10.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7fb12429e640 10 monclient: get_monmap_and_config 2024-01-24T23:21:10.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7fb12429e640 10 monclient: build_initial_monmap 2024-01-24T23:21:10.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7fb12429e640 10 monclient: monmap: 2024-01-24T23:21:10.589 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:10.589 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.589 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:10.583307+0000 2024-01-24T23:21:10.589 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:10.583307+0000 2024-01-24T23:21:10.589 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:10.589 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.590 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:10.590 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:10.590 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:10.590 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:10.590 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:10.590 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:10.590 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:10.590 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:10.590 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:10.590 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7fd0f8781640 10 monclient: init 2024-01-24T23:21:10.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7fb12429e640 1 Processor -- start 2024-01-24T23:21:10.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7fb12429e640 1 -- start start 2024-01-24T23:21:10.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7fb12429e640 10 monclient: init 2024-01-24T23:21:10.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7fb12429e640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:10.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7fb12429e640 10 monclient: _add_conns ranks=[6,7,2,4,5,3,0,1,8] 2024-01-24T23:21:10.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7fb12429e640 10 monclient(hunting): picked mon.noname-d con 0x7fb11c133bf0 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:10.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7fb12429e640 10 monclient(hunting): picked mon.noname-i con 0x7fb11c133fd0 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:10.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7fb12203a640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fb11c133bf0 0x7fb11c09bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:56940/0 (socket says 172.21.15.115:56940) 2024-01-24T23:21:10.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7faa56c6a640 10 monclient: get_monmap_and_config 2024-01-24T23:21:10.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7faa56c6a640 10 monclient: build_initial_monmap 2024-01-24T23:21:10.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7faa56c6a640 10 monclient: monmap: 2024-01-24T23:21:10.592 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:10.592 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.592 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:10.585501+0000 2024-01-24T23:21:10.592 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:10.585501+0000 2024-01-24T23:21:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:10.593 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:10.593 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7fb121839640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fb11c133fd0 0x7fb11c13bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.115:36160/0 (socket says 172.21.15.115:36160) 2024-01-24T23:21:10.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7fb12283b640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fb11c1343b0 0x7fb11c13f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:54534/0 (socket says 172.21.15.115:54534) 2024-01-24T23:21:10.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7fb12203a640 1 -- 172.21.15.115:0/2793286634 learned_addr learned my addr 172.21.15.115:0/2793286634 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:10.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7fb12429e640 10 monclient(hunting): picked mon.noname-b con 0x7fb11c1343b0 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:10.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7fb12429e640 1 -- 172.21.15.115:0/2793286634 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11c04ed70 con 0x7fb11c1343b0 2024-01-24T23:21:10.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7faa56c6a640 1 Processor -- start 2024-01-24T23:21:10.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7fb12429e640 1 -- 172.21.15.115:0/2793286634 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11c0a4b60 con 0x7fb11c133bf0 2024-01-24T23:21:10.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7fb121038640 1 -- 172.21.15.115:0/2793286634 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1999594004 0 0) 0x7fb11c04ed70 con 0x7fb11c1343b0 2024-01-24T23:21:10.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7faa56c6a640 1 -- start start 2024-01-24T23:21:10.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.579+0000 7faa56c6a640 10 monclient: init 2024-01-24T23:21:10.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb12429e640 1 -- 172.21.15.115:0/2793286634 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11c09ae50 con 0x7fb11c133fd0 2024-01-24T23:21:10.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb12429e640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:10.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb12429e640 10 monclient(hunting): authenticate will time out at 949.508166s 2024-01-24T23:21:10.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 10 monclient(hunting): my global_id is 4250 2024-01-24T23:21:10.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 1 -- 172.21.15.115:0/2793286634 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb108003530 con 0x7fb11c1343b0 2024-01-24T23:21:10.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa56c6a640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:10.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0f8781640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:10.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa56c6a640 10 monclient: _add_conns ranks=[5,4,1,8,7,2,0,6,3] 2024-01-24T23:21:10.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0f8781640 10 monclient: _add_conns ranks=[1,6,4,0,5,2,3,7,8] 2024-01-24T23:21:10.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa56c6a640 10 monclient(hunting): picked mon.noname-h con 0x7faa5009f8c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:10.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa56c6a640 10 monclient(hunting): picked mon.noname-c con 0x7faa5009fca0 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:10.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa56c6a640 10 monclient(hunting): picked mon.noname-f con 0x7faa500a00e0 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:10.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa56c6a640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa5004ed70 con 0x7faa500a00e0 2024-01-24T23:21:10.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 1 -- 172.21.15.115:0/2793286634 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993012088 0 0) 0x7fb11c09ae50 con 0x7fb11c133fd0 2024-01-24T23:21:10.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0f8781640 10 monclient(hunting): picked mon.noname-f con 0x7fd0f0133bf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:10.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 10 monclient(hunting): my global_id is 4309 2024-01-24T23:21:10.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 1 -- 172.21.15.115:0/2793286634 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb11c04ed70 con 0x7fb11c133fd0 2024-01-24T23:21:10.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 1 -- 172.21.15.115:0/2793286634 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2800097263 0 0) 0x7fb108003530 con 0x7fb11c1343b0 2024-01-24T23:21:10.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 1 -- 172.21.15.115:0/2793286634 >> v1:172.21.15.132:6792/0 conn(0x7fb11c133fd0 legacy=0x7fb11c13bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 1 -- 172.21.15.115:0/2793286634 >> v1:172.21.15.115:6792/0 conn(0x7fb11c133bf0 legacy=0x7fb11c09bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 10 monclient: _finish_hunting 0 2024-01-24T23:21:10.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 1 monclient: found mon.noname-b 2024-01-24T23:21:10.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:10.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:21:10.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 1 -- 172.21.15.115:0/2793286634 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11c09b130 con 0x7fb11c1343b0 2024-01-24T23:21:10.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 10 monclient: _finish_auth 0 2024-01-24T23:21:10.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0f8781640 10 monclient(hunting): picked mon.noname-d con 0x7fd0f0133fd0 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:10.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 10 monclient: _check_auth_tickets 2024-01-24T23:21:10.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:10.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 1 -- 172.21.15.115:0/2793286634 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb114001b40 con 0x7fb11c1343b0 2024-01-24T23:21:10.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:21:10.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa56c6a640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa5009ced0 con 0x7faa5009fca0 2024-01-24T23:21:10.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0f651d640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fd0f0133bf0 0x7fd0f00a0d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.115:35128/0 (socket says 172.21.15.115:35128) 2024-01-24T23:21:10.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa56c6a640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa50134450 con 0x7faa5009f8c0 2024-01-24T23:21:10.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa56c6a640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:10.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa56c6a640 10 monclient(hunting): authenticate will time out at 949.508755s 2024-01-24T23:21:10.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0f8781640 10 monclient(hunting): picked mon.noname-c con 0x7fd0f01343b0 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:10.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0f8781640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f00a2740 con 0x7fd0f0133bf0 2024-01-24T23:21:10.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0f8781640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f00a5990 con 0x7fd0f01343b0 2024-01-24T23:21:10.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0f8781640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f0120170 con 0x7fd0f0133fd0 2024-01-24T23:21:10.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0f8781640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:10.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0f8781640 10 monclient(hunting): authenticate will time out at 949.508836s 2024-01-24T23:21:10.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa54a06640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7faa5009f8c0 0x7faa50133d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.115:47808/0 (socket says 172.21.15.115:47808) 2024-01-24T23:21:10.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa54a06640 1 -- 172.21.15.115:0/1438753616 learned_addr learned my addr 172.21.15.115:0/1438753616 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:10.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0f651d640 1 -- 172.21.15.115:0/1778210534 learned_addr learned my addr 172.21.15.115:0/1778210534 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:10.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 10 monclient: dump: 2024-01-24T23:21:10.604 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.604 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.604 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.604 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.604 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.604 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.605 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.605 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.605 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.605 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.605 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.605 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.605 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.605 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.605 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.605 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb12429e640 5 monclient: authenticate success, global_id 4250 2024-01-24T23:21:10.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb12429e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:10.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 1 -- 172.21.15.115:0/2793286634 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb114001d60 con 0x7fb11c1343b0 2024-01-24T23:21:10.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:10.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 1 -- 172.21.15.115:0/2793286634 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb114004eb0 con 0x7fb11c1343b0 2024-01-24T23:21:10.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:10.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb121038640 10 monclient: dump: 2024-01-24T23:21:10.606 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.606 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.606 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.606 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.606 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.606 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.606 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.606 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.606 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.606 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.606 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.606 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.607 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.607 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.607 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.607 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb12429e640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:10.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb12429e640 4 set_mon_vals no callback set 2024-01-24T23:21:10.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 1 -- 172.21.15.115:0/1438753616 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2769448602 0 0) 0x7faa50134450 con 0x7faa5009f8c0 2024-01-24T23:21:10.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 10 monclient(hunting): my global_id is 4298 2024-01-24T23:21:10.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 1 -- 172.21.15.115:0/1438753616 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa40003530 con 0x7faa5009f8c0 2024-01-24T23:21:10.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 1 -- 172.21.15.115:0/1778210534 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2463634029 0 0) 0x7fd0f00a2740 con 0x7fd0f0133bf0 2024-01-24T23:21:10.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 10 monclient(hunting): my global_id is 4294 2024-01-24T23:21:10.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 1 -- 172.21.15.115:0/1778210534 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0d4003530 con 0x7fd0f0133bf0 2024-01-24T23:21:10.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 1 -- 172.21.15.115:0/1778210534 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3425844615 0 0) 0x7fd0d4003530 con 0x7fd0f0133bf0 2024-01-24T23:21:10.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 1 -- 172.21.15.115:0/1778210534 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd0f00a2740 con 0x7fd0f0133bf0 2024-01-24T23:21:10.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 1 -- 172.21.15.115:0/1438753616 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2523293782 0 0) 0x7faa40003530 con 0x7faa5009f8c0 2024-01-24T23:21:10.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 1 -- 172.21.15.115:0/1438753616 >> v1:172.21.15.115:6791/0 conn(0x7faa5009fca0 legacy=0x7faa50138a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T23:21:10.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 1 -- 172.21.15.115:0/1438753616 >> v1:172.21.15.132:6789/0 conn(0x7faa500a00e0 legacy=0x7faa5013c180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-24T23:21:10.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:21:10.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 1 monclient: found mon.noname-h 2024-01-24T23:21:10.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:10.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:21:10.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 1 -- 172.21.15.115:0/1438753616 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5009c5c0 con 0x7faa5009f8c0 2024-01-24T23:21:10.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 10 monclient: _finish_auth 0 2024-01-24T23:21:10.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:21:10.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:10.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 1 -- 172.21.15.115:0/1438753616 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faa440017c0 con 0x7faa5009f8c0 2024-01-24T23:21:10.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:21:10.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 10 monclient: dump: 2024-01-24T23:21:10.610 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.610 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.610 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.610 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.610 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.610 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.610 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.610 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.610 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.610 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.610 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.611 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.611 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.611 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.611 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.611 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 1 -- 172.21.15.115:0/1778210534 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd0ec0019d0 con 0x7fd0f0133bf0 2024-01-24T23:21:10.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:21:10.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 10 monclient(hunting): dump: 2024-01-24T23:21:10.611 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.612 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.612 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.612 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.612 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.612 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.612 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.612 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.612 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.612 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.612 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.612 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.612 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.612 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.612 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.612 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:10.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa56c6a640 5 monclient: authenticate success, global_id 4298 2024-01-24T23:21:10.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa56c6a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:10.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 1 -- 172.21.15.115:0/1438753616 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faa44001ab0 con 0x7faa5009f8c0 2024-01-24T23:21:10.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:10.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa56c6a640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:10.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa56c6a640 4 set_mon_vals no callback set 2024-01-24T23:21:10.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 1 -- 172.21.15.115:0/1778210534 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 691705904 0 0) 0x7fd0f00a2740 con 0x7fd0f0133bf0 2024-01-24T23:21:10.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 1 -- 172.21.15.115:0/1778210534 >> v1:172.21.15.115:6792/0 conn(0x7fd0f0133fd0 legacy=0x7fd0f00a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd308868640 10 monclient: get_monmap_and_config 2024-01-24T23:21:10.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb12429e640 10 monclient: shutdown 2024-01-24T23:21:10.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb12429e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb12429e640 1 -- 172.21.15.115:0/2793286634 >> v1:172.21.15.115:6790/0 conn(0x7fb11c1343b0 legacy=0x7fb11c13f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb12429e640 1 -- 172.21.15.115:0/2793286634 shutdown_connections 2024-01-24T23:21:10.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb12429e640 1 -- 172.21.15.115:0/2793286634 >> 172.21.15.115:0/2793286634 conn(0x7fb11c0662f0 msgr2=0x7fb11c066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 1 -- 172.21.15.115:0/1778210534 >> v1:172.21.15.115:6791/0 conn(0x7fd0f01343b0 legacy=0x7fd0f01d3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:21:10.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 1 monclient: found mon.f 2024-01-24T23:21:10.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:10.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:21:10.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 1 -- 172.21.15.115:0/1778210534 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f0120450 con 0x7fd0f0133bf0 2024-01-24T23:21:10.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 10 monclient: _finish_auth 0 2024-01-24T23:21:10.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:21:10.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:10.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0f8781640 5 monclient: authenticate success, global_id 4294 2024-01-24T23:21:10.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0f8781640 10 monclient: _renew_subs 2024-01-24T23:21:10.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0f8781640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:21:10.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0f8781640 1 -- 172.21.15.115:0/1778210534 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f011f760 con 0x7fd0f0133bf0 2024-01-24T23:21:10.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0f8781640 10 monclient: _renew_subs 2024-01-24T23:21:10.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0f8781640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:21:10.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0f8781640 1 -- 172.21.15.115:0/1778210534 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f00a34a0 con 0x7fd0f0133bf0 2024-01-24T23:21:10.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb12429e640 1 -- 172.21.15.115:0/2793286634 shutdown_connections 2024-01-24T23:21:10.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb12429e640 1 -- 172.21.15.115:0/2793286634 wait complete. 2024-01-24T23:21:10.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 1 -- 172.21.15.115:0/1778210534 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd0ec001d70 con 0x7fd0f0133bf0 2024-01-24T23:21:10.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:10.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 1 -- 172.21.15.115:0/1778210534 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd0ec004de0 con 0x7fd0f0133bf0 2024-01-24T23:21:10.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:21:10.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 10 monclient: dump: 2024-01-24T23:21:10.617 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.617 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.617 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.617 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.617 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.617 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.617 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.618 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.618 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.618 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.618 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.618 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.618 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.618 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.618 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.618 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0f4d1a640 4 set_mon_vals no callback set 2024-01-24T23:21:10.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa56c6a640 10 monclient: shutdown 2024-01-24T23:21:10.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa56c6a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa56c6a640 1 -- 172.21.15.115:0/1438753616 >> v1:172.21.15.132:6791/0 conn(0x7faa5009f8c0 legacy=0x7faa50133d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 1 -- 172.21.15.115:0/1438753616 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faa44004bf0 con 0x7faa5009f8c0 2024-01-24T23:21:10.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa4f7fe640 10 monclient: discarding stray monitor message mon_map magic: 0 v1 2024-01-24T23:21:10.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd308868640 10 monclient: build_initial_monmap 2024-01-24T23:21:10.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd308868640 10 monclient: monmap: 2024-01-24T23:21:10.619 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:10.619 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.619 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:10.588875+0000 2024-01-24T23:21:10.619 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:10.588875+0000 2024-01-24T23:21:10.620 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:10.620 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.620 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:10.620 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:10.620 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:10.620 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:10.620 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:10.620 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:10.620 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:10.620 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:10.620 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:10.620 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa56c6a640 1 -- 172.21.15.115:0/1438753616 shutdown_connections 2024-01-24T23:21:10.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa56c6a640 1 -- 172.21.15.115:0/1438753616 >> 172.21.15.115:0/1438753616 conn(0x7faa500a6490 msgr2=0x7faa500a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa56c6a640 1 -- 172.21.15.115:0/1438753616 shutdown_connections 2024-01-24T23:21:10.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0f8781640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:10.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7faa56c6a640 1 -- 172.21.15.115:0/1438753616 wait complete. 2024-01-24T23:21:10.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb12429e640 10 monclient: build_initial_monmap 2024-01-24T23:21:10.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb12429e640 10 monclient: monmap: 2024-01-24T23:21:10.622 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:10.622 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:10.622 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:10.622 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:10.622 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:10.622 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:10.622 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:10.622 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:10.622 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:10.622 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:10.622 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:10.623 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:10.623 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:10.623 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:10.623 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:10.623 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 1 -- 172.21.15.115:0/1778210534 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 4) v1 ==== 43232+0+0 (unknown 913048011 0 0) 0x7fd0ec004f40 con 0x7fd0f0133bf0 2024-01-24T23:21:10.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0f8781640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:10.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0f8781640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:21:10.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0f8781640 1 -- 172.21.15.115:0/1778210534 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d8005090 con 0x7fd0f0133bf0 2024-01-24T23:21:10.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb12429e640 1 Processor -- start 2024-01-24T23:21:10.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fd0e6ffd640 1 -- 172.21.15.115:0/1778210534 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7fd0ec043710 con 0x7fd0f0133bf0 2024-01-24T23:21:10.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb12429e640 1 -- start start 2024-01-24T23:21:10.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.583+0000 7fb12429e640 10 monclient: init 2024-01-24T23:21:10.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd0f5d1c640 10 monclient: get_auth_request con 0x7fd0d4036fb0 auth_method 0 2024-01-24T23:21:10.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa56c6a640 10 monclient: build_initial_monmap 2024-01-24T23:21:10.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa56c6a640 10 monclient: monmap: 2024-01-24T23:21:10.625 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:10.625 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:10.625 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:10.625 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:10.625 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:10.625 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:10.626 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:10.626 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:10.626 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:10.626 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:10.626 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:10.626 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:10.626 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:10.626 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:10.626 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:10.626 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb12429e640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:10.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa56c6a640 1 Processor -- start 2024-01-24T23:21:10.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb12429e640 10 monclient: _add_conns ranks=[4,6,7,2,8,3,1,5,0] 2024-01-24T23:21:10.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb12429e640 10 monclient(hunting): picked mon.noname-c con 0x7fb11c133bf0 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:10.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb12429e640 10 monclient(hunting): picked mon.noname-d con 0x7fb11c133fd0 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:10.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb12429e640 10 monclient(hunting): picked mon.noname-i con 0x7fb11c1343b0 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:10.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb12429e640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11c0a1e80 con 0x7fb11c133bf0 2024-01-24T23:21:10.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb12429e640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11c133720 con 0x7fb11c133fd0 2024-01-24T23:21:10.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb12429e640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11c0a5990 con 0x7fb11c1343b0 2024-01-24T23:21:10.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb12429e640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:10.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb12429e640 10 monclient(hunting): authenticate will time out at 949.512880s 2024-01-24T23:21:10.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd308868640 1 Processor -- start 2024-01-24T23:21:10.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd308868640 1 -- start start 2024-01-24T23:21:10.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd308868640 10 monclient: init 2024-01-24T23:21:10.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb12283b640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fb11c1343b0 0x7fb11c1d39e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.115:36176/0 (socket says 172.21.15.115:36176) 2024-01-24T23:21:10.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb12283b640 1 -- 172.21.15.115:0/3195832231 learned_addr learned my addr 172.21.15.115:0/3195832231 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:10.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 1 -- 172.21.15.115:0/3195832231 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1852229480 0 0) 0x7fb11c0a5990 con 0x7fb11c1343b0 2024-01-24T23:21:10.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 10 monclient(hunting): my global_id is 4318 2024-01-24T23:21:10.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 1 -- 172.21.15.115:0/3195832231 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb100003530 con 0x7fb11c1343b0 2024-01-24T23:21:10.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd308868640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:10.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd308868640 10 monclient: _add_conns ranks=[5,4,1,6,7,0,8,2,3] 2024-01-24T23:21:10.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd308868640 10 monclient(hunting): picked mon.noname-h con 0x7fd3040a3b10 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:10.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd308868640 10 monclient(hunting): picked mon.noname-c con 0x7fd3040a54f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:10.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd308868640 10 monclient(hunting): picked mon.noname-f con 0x7fd30409c420 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:10.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd308868640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30404ed70 con 0x7fd30409c420 2024-01-24T23:21:10.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd308868640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30409b480 con 0x7fd3040a54f0 2024-01-24T23:21:10.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd308868640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30409b1a0 con 0x7fd3040a3b10 2024-01-24T23:21:10.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd308868640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:10.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd308868640 10 monclient(hunting): authenticate will time out at 949.513558s 2024-01-24T23:21:10.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa56c6a640 1 -- start start 2024-01-24T23:21:10.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa56c6a640 10 monclient: init 2024-01-24T23:21:10.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa56c6a640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:10.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa56c6a640 10 monclient: _add_conns ranks=[7,6,3,5,8,4,1,2,0] 2024-01-24T23:21:10.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa56c6a640 10 monclient(hunting): picked mon.noname-i con 0x7faa5009f8c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:10.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa56c6a640 10 monclient(hunting): picked mon.noname-d con 0x7faa5009fca0 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:10.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa56c6a640 10 monclient(hunting): picked mon.noname-g con 0x7faa500a00e0 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:10.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa56c6a640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa5012a6e0 con 0x7faa500a00e0 2024-01-24T23:21:10.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa56c6a640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa50072270 con 0x7faa5009fca0 2024-01-24T23:21:10.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa56c6a640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa5009ba60 con 0x7faa5009f8c0 2024-01-24T23:21:10.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa56c6a640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:10.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa56c6a640 10 monclient(hunting): authenticate will time out at 949.513018s 2024-01-24T23:21:10.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa54a06640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7faa5009f8c0 0x7faa501cfab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.115:36180/0 (socket says 172.21.15.115:36180) 2024-01-24T23:21:10.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa54a06640 1 -- 172.21.15.115:0/2240506585 learned_addr learned my addr 172.21.15.115:0/2240506585 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:10.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 1 -- 172.21.15.115:0/2240506585 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2652710526 0 0) 0x7faa5009ba60 con 0x7faa5009f8c0 2024-01-24T23:21:10.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 10 monclient(hunting): my global_id is 4327 2024-01-24T23:21:10.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 1 -- 172.21.15.115:0/2240506585 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa34003530 con 0x7faa5009f8c0 2024-01-24T23:21:10.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd302d9d640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fd30409c420 0x7fd304143c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.115:35146/0 (socket says 172.21.15.115:35146) 2024-01-24T23:21:10.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd302d9d640 1 -- 172.21.15.115:0/2681996678 learned_addr learned my addr 172.21.15.115:0/2681996678 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:10.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 1 -- 172.21.15.115:0/2240506585 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1469016303 0 0) 0x7faa5012a6e0 con 0x7faa500a00e0 2024-01-24T23:21:10.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 10 monclient(hunting): my global_id is 4305 2024-01-24T23:21:10.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 1 -- 172.21.15.115:0/2240506585 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa5009ba60 con 0x7faa500a00e0 2024-01-24T23:21:10.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 1 -- 172.21.15.115:0/3195832231 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3066103510 0 0) 0x7fb100003530 con 0x7fb11c1343b0 2024-01-24T23:21:10.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 1 -- 172.21.15.115:0/3195832231 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb11c0a5990 con 0x7fb11c1343b0 2024-01-24T23:21:10.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 1 -- 172.21.15.115:0/3195832231 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb114004f70 con 0x7fb11c1343b0 2024-01-24T23:21:10.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:21:10.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 10 monclient(hunting): dump: 2024-01-24T23:21:10.633 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.634 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.634 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.634 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.634 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.634 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.634 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.634 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.634 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.634 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.634 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.634 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.634 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.634 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.634 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.634 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:10.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 1 -- 172.21.15.115:0/2681996678 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2383282403 0 0) 0x7fd30409b1a0 con 0x7fd3040a3b10 2024-01-24T23:21:10.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 10 monclient(hunting): my global_id is 4307 2024-01-24T23:21:10.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 1 -- 172.21.15.115:0/2681996678 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2ec003530 con 0x7fd3040a3b10 2024-01-24T23:21:10.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 1 -- 172.21.15.115:0/2240506585 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 945948045 0 0) 0x7faa34003530 con 0x7faa5009f8c0 2024-01-24T23:21:10.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 1 -- 172.21.15.115:0/2240506585 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faa5012a6e0 con 0x7faa5009f8c0 2024-01-24T23:21:10.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 1 -- 172.21.15.115:0/2240506585 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faa44004b30 con 0x7faa5009f8c0 2024-01-24T23:21:10.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:21:10.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 10 monclient(hunting): dump: 2024-01-24T23:21:10.636 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.636 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.636 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.636 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.636 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.636 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.636 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.636 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.637 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.637 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.637 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.637 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.637 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.637 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.637 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.637 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:10.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 1 -- 172.21.15.115:0/3195832231 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2489188699 0 0) 0x7fb11c0a5990 con 0x7fb11c1343b0 2024-01-24T23:21:10.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 1 -- 172.21.15.115:0/3195832231 >> v1:172.21.15.115:6792/0 conn(0x7fb11c133fd0 legacy=0x7fb11c1c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 1 -- 172.21.15.115:0/3195832231 >> v1:172.21.15.115:6791/0 conn(0x7fb11c133bf0 legacy=0x7fb11c133010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:21:10.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 1 monclient: found mon.i 2024-01-24T23:21:10.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:10.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:10.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 1 -- 172.21.15.115:0/3195832231 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11c09c810 con 0x7fb11c1343b0 2024-01-24T23:21:10.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 10 monclient: _finish_auth 0 2024-01-24T23:21:10.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:21:10.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:10.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 1 -- 172.21.15.115:0/2681996678 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 446157665 0 0) 0x7fd2ec003530 con 0x7fd3040a3b10 2024-01-24T23:21:10.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 1 -- 172.21.15.115:0/2681996678 >> v1:172.21.15.115:6791/0 conn(0x7fd3040a54f0 legacy=0x7fd304140560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 1 -- 172.21.15.115:0/2240506585 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3504353422 0 0) 0x7faa5009ba60 con 0x7faa500a00e0 2024-01-24T23:21:10.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 1 -- 172.21.15.115:0/2240506585 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faa34003530 con 0x7faa500a00e0 2024-01-24T23:21:10.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 1 -- 172.21.15.115:0/2240506585 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3385451045 0 0) 0x7faa5012a6e0 con 0x7faa5009f8c0 2024-01-24T23:21:10.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 1 -- 172.21.15.115:0/2240506585 >> v1:172.21.15.115:6792/0 conn(0x7faa5009fca0 legacy=0x7faa501d47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 1 -- 172.21.15.115:0/2240506585 >> v1:172.21.15.132:6790/0 conn(0x7faa500a00e0 legacy=0x7faa501d7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 10 monclient: _finish_hunting 0 2024-01-24T23:21:10.639 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 1 monclient: found mon.i 2024-01-24T23:21:10.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:10.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:10.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 1 -- 172.21.15.115:0/2240506585 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5012a400 con 0x7faa5009f8c0 2024-01-24T23:21:10.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 10 monclient: _finish_auth 0 2024-01-24T23:21:10.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:21:10.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:10.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 1 -- 172.21.15.115:0/2681996678 >> v1:172.21.15.132:6789/0 conn(0x7fd30409c420 legacy=0x7fd304143c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 10 monclient: _finish_hunting 0 2024-01-24T23:21:10.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 1 monclient: found mon.noname-h 2024-01-24T23:21:10.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:10.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:21:10.640 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 1 -- 172.21.15.115:0/2681996678 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3040a4e20 con 0x7fd3040a3b10 2024-01-24T23:21:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 10 monclient: _finish_auth 0 2024-01-24T23:21:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 10 monclient: _check_auth_tickets 2024-01-24T23:21:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 1 -- 172.21.15.115:0/2681996678 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd2f4001b40 con 0x7fd3040a3b10 2024-01-24T23:21:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd308868640 5 monclient: authenticate success, global_id 4307 2024-01-24T23:21:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd308868640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:21:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 10 monclient: dump: 2024-01-24T23:21:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.642 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.642 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.642 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.642 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.642 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.642 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.642 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.642 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.642 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.642 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd308868640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:10.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 1 -- 172.21.15.115:0/3195832231 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb114005170 con 0x7fb11c1343b0 2024-01-24T23:21:10.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:10.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 1 -- 172.21.15.115:0/3195832231 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb114005620 con 0x7fb11c1343b0 2024-01-24T23:21:10.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:21:10.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb120837640 4 set_mon_vals no callback set 2024-01-24T23:21:10.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 10 monclient: dump: 2024-01-24T23:21:10.643 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.643 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.643 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.643 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.643 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.643 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.643 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.643 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.643 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.644 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.644 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.644 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.644 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.644 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.644 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.644 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb12429e640 5 monclient: authenticate success, global_id 4318 2024-01-24T23:21:10.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb12429e640 10 monclient: _renew_subs 2024-01-24T23:21:10.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb12429e640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:10.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb12429e640 1 -- 172.21.15.115:0/3195832231 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb11c0a25b0 con 0x7fb11c1343b0 2024-01-24T23:21:10.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 1 -- 172.21.15.115:0/2681996678 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd2f4001d60 con 0x7fd3040a3b10 2024-01-24T23:21:10.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:10.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 1 -- 172.21.15.115:0/2681996678 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd2f4003ce0 con 0x7fd3040a3b10 2024-01-24T23:21:10.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd308868640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:10.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd308868640 4 set_mon_vals no callback set 2024-01-24T23:21:10.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:21:10.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd30159a640 10 monclient: dump: 2024-01-24T23:21:10.645 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.645 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.645 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.645 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.645 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.645 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.645 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.646 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb12429e640 10 monclient: _renew_subs 2024-01-24T23:21:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb12429e640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb12429e640 1 -- 172.21.15.115:0/3195832231 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb11c1209d0 con 0x7fb11c1343b0 2024-01-24T23:21:10.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa56c6a640 5 monclient: authenticate success, global_id 4327 2024-01-24T23:21:10.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa56c6a640 10 monclient: _renew_subs 2024-01-24T23:21:10.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa56c6a640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:10.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa56c6a640 1 -- 172.21.15.115:0/2240506585 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa50128ca0 con 0x7faa5009f8c0 2024-01-24T23:21:10.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa56c6a640 10 monclient: _renew_subs 2024-01-24T23:21:10.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa56c6a640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:10.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa56c6a640 1 -- 172.21.15.115:0/2240506585 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5012b2c0 con 0x7faa5009f8c0 2024-01-24T23:21:10.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd308868640 10 monclient: shutdown 2024-01-24T23:21:10.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd308868640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd308868640 1 -- 172.21.15.115:0/2681996678 >> v1:172.21.15.132:6791/0 conn(0x7fd3040a3b10 legacy=0x7fd3040a3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd308868640 1 -- 172.21.15.115:0/2681996678 shutdown_connections 2024-01-24T23:21:10.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd308868640 1 -- 172.21.15.115:0/2681996678 >> 172.21.15.115:0/2681996678 conn(0x7fd3040662f0 msgr2=0x7fd304066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fd308868640 1 -- 172.21.15.115:0/2681996678 shutdown_connections 2024-01-24T23:21:10.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 1 -- 172.21.15.115:0/2240506585 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faa44004f00 con 0x7faa5009f8c0 2024-01-24T23:21:10.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:10.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 1 -- 172.21.15.115:0/2240506585 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faa440053b0 con 0x7faa5009f8c0 2024-01-24T23:21:10.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:21:10.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4d7fa640 10 monclient: dump: 2024-01-24T23:21:10.649 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.649 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.649 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.649 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.649 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.649 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.649 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.649 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.649 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.649 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.650 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.650 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.650 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.650 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.650 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.650 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa4effd640 4 set_mon_vals no callback set 2024-01-24T23:21:10.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7faa56c6a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:10.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7faa56c6a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:10.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7faa56c6a640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:10.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb112ffd640 1 -- 172.21.15.115:0/3195832231 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 4) v1 ==== 43232+0+0 (unknown 913048011 0 0) 0x7fb114010070 con 0x7fb11c1343b0 2024-01-24T23:21:10.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb110ff9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:10.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb110ff9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:10.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb110ff9640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:10.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.587+0000 7fb110ff9640 1 -- 172.21.15.115:0/3195832231 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb11c06a0d0 con 0x7fb11c1343b0 2024-01-24T23:21:10.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fb112ffd640 1 -- 172.21.15.115:0/3195832231 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7fb114043ca0 con 0x7fb11c1343b0 2024-01-24T23:21:10.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7faa56c6a640 1 -- 172.21.15.115:0/2240506585 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa30005090 con 0x7faa5009f8c0 2024-01-24T23:21:10.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fb12203a640 10 monclient: get_auth_request con 0x7fb100036fb0 auth_method 0 2024-01-24T23:21:10.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd0e6ffd640 1 -- 172.21.15.115:0/1778210534 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fd0ec014930 con 0x7fd0f0133bf0 2024-01-24T23:21:10.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd0e6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:10.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd0e6ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:10.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7faa4d7fa640 1 -- 172.21.15.115:0/2240506585 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 4) v1 ==== 43232+0+0 (unknown 913048011 0 0) 0x7faa4400fe00 con 0x7faa5009f8c0 2024-01-24T23:21:10.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7faa4d7fa640 1 -- 172.21.15.115:0/2240506585 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7faa440436b0 con 0x7faa5009f8c0 2024-01-24T23:21:10.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd308868640 1 -- 172.21.15.115:0/2681996678 wait complete. 2024-01-24T23:21:10.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd308868640 10 monclient: build_initial_monmap 2024-01-24T23:21:10.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd308868640 10 monclient: monmap: 2024-01-24T23:21:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:10.654 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd308868640 1 Processor -- start 2024-01-24T23:21:10.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7faa4ffff640 10 monclient: get_auth_request con 0x7faa34037480 auth_method 0 2024-01-24T23:21:10.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd308868640 1 -- start start 2024-01-24T23:21:10.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd308868640 10 monclient: init 2024-01-24T23:21:10.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd308868640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:10.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd308868640 10 monclient: _add_conns ranks=[8,2,3,6,5,0,7,1,4] 2024-01-24T23:21:10.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd308868640 10 monclient(hunting): picked mon.noname-e con 0x7fd30409c420 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd308868640 10 monclient(hunting): picked mon.noname-b con 0x7fd3040a3b10 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd308868640 10 monclient(hunting): picked mon.noname-g con 0x7fd3040a54f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd308868640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3041af340 con 0x7fd3040a3b10 2024-01-24T23:21:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd308868640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3040707e0 con 0x7fd3040a54f0 2024-01-24T23:21:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd308868640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3040a5930 con 0x7fd30409c420 2024-01-24T23:21:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd308868640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd308868640 10 monclient(hunting): authenticate will time out at 949.518185s 2024-01-24T23:21:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd30259c640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fd30409c420 0x7fd3041c3350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:48778/0 (socket says 172.21.15.115:48778) 2024-01-24T23:21:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd30259c640 1 -- 172.21.15.115:0/1587812515 learned_addr learned my addr 172.21.15.115:0/1587812515 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 1 -- 172.21.15.115:0/1587812515 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3091991052 0 0) 0x7fd3040a5930 con 0x7fd30409c420 2024-01-24T23:21:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 10 monclient(hunting): my global_id is 74249 2024-01-24T23:21:10.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 1 -- 172.21.15.115:0/1587812515 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2e4003530 con 0x7fd30409c420 2024-01-24T23:21:10.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 1 -- 172.21.15.115:0/1587812515 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784131071 0 0) 0x7fd3041af340 con 0x7fd3040a3b10 2024-01-24T23:21:10.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:10.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:10.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:10.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 10 monclient(hunting): my global_id is 4259 2024-01-24T23:21:10.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 1 -- 172.21.15.115:0/1587812515 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3040a5930 con 0x7fd3040a3b10 2024-01-24T23:21:10.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 1 -- 172.21.15.115:0/1587812515 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3419546614 0 0) 0x7fd2e4003530 con 0x7fd30409c420 2024-01-24T23:21:10.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 1 -- 172.21.15.115:0/1587812515 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd3041af340 con 0x7fd30409c420 2024-01-24T23:21:10.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 1 -- 172.21.15.115:0/1587812515 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd2f4003d80 con 0x7fd30409c420 2024-01-24T23:21:10.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:21:10.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 10 monclient(hunting): dump: 2024-01-24T23:21:10.658 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.659 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.659 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.659 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.659 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.659 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.659 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.659 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.659 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.659 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.659 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.659 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.659 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.659 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.659 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.659 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:10.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 1 -- 172.21.15.115:0/1587812515 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3166478385 0 0) 0x7fd3040a5930 con 0x7fd3040a3b10 2024-01-24T23:21:10.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 1 -- 172.21.15.115:0/1587812515 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd2e4003530 con 0x7fd3040a3b10 2024-01-24T23:21:10.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 1 -- 172.21.15.115:0/1587812515 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd2fc001a20 con 0x7fd3040a3b10 2024-01-24T23:21:10.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:10.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 10 monclient(hunting): dump: 2024-01-24T23:21:10.660 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.660 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.660 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.660 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.660 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.660 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.660 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.661 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.661 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.661 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.661 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.661 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.661 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.661 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.661 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.661 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 1 -- 172.21.15.115:0/1587812515 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 396545065 0 0) 0x7fd3041af340 con 0x7fd30409c420 2024-01-24T23:21:10.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 1 -- 172.21.15.115:0/1587812515 >> v1:172.21.15.132:6790/0 conn(0x7fd3040a54f0 legacy=0x7fd3041d81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 1 -- 172.21.15.115:0/1587812515 >> v1:172.21.15.115:6790/0 conn(0x7fd3040a3b10 legacy=0x7fd3041d4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:21:10.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 1 monclient: found mon.e 2024-01-24T23:21:10.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:10.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:10.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 1 -- 172.21.15.115:0/1587812515 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3041b0d10 con 0x7fd30409c420 2024-01-24T23:21:10.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 10 monclient: _finish_auth 0 2024-01-24T23:21:10.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:21:10.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:10.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd308868640 5 monclient: authenticate success, global_id 74249 2024-01-24T23:21:10.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd308868640 10 monclient: _renew_subs 2024-01-24T23:21:10.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd308868640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:10.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd308868640 1 -- 172.21.15.115:0/1587812515 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3041b0470 con 0x7fd30409c420 2024-01-24T23:21:10.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd308868640 10 monclient: _renew_subs 2024-01-24T23:21:10.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd308868640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:10.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd308868640 1 -- 172.21.15.115:0/1587812515 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3041af730 con 0x7fd30409c420 2024-01-24T23:21:10.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 1 -- 172.21.15.115:0/1587812515 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd2f4001560 con 0x7fd30409c420 2024-01-24T23:21:10.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:10.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 1 -- 172.21.15.115:0/1587812515 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd2f40042e0 con 0x7fd30409c420 2024-01-24T23:21:10.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:10.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:21:10.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd2f2ffd640 10 monclient: dump: 2024-01-24T23:21:10.663 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:10.663 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:10.663 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.663 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:10.663 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:10.663 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:10.663 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:10.663 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:10.663 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:10.663 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:10.663 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:10.663 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:10.663 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:10.664 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:10.664 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:10.664 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:10.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.591+0000 7fd300d99640 4 set_mon_vals no callback set 2024-01-24T23:21:10.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.595+0000 7fd308868640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:10.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.595+0000 7fd308868640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:10.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.595+0000 7fd308868640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:10.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.595+0000 7fd308868640 1 -- 172.21.15.115:0/1587812515 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e8005090 con 0x7fd30409c420 2024-01-24T23:21:10.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.595+0000 7fd2f2ffd640 1 -- 172.21.15.115:0/1587812515 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 4) v1 ==== 43232+0+0 (unknown 913048011 0 0) 0x7fd2f4004440 con 0x7fd30409c420 2024-01-24T23:21:10.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.595+0000 7fd2f2ffd640 1 -- 172.21.15.115:0/1587812515 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7fd2f40439a0 con 0x7fd30409c420 2024-01-24T23:21:10.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.595+0000 7fd301d9b640 10 monclient: get_auth_request con 0x7fd2e4037590 auth_method 0 2024-01-24T23:21:10.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.595+0000 7faa4d7fa640 1 -- 172.21.15.115:0/2240506585 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7faa44014990 con 0x7faa5009f8c0 2024-01-24T23:21:10.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.595+0000 7faa4d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:10.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.595+0000 7faa4d7fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:10.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.603+0000 7fb112ffd640 1 -- 172.21.15.115:0/3195832231 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fb1140151b0 con 0x7fb11c1343b0 2024-01-24T23:21:10.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.603+0000 7fb112ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:10.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.603+0000 7fb112ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:10.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.603+0000 7fd2f2ffd640 1 -- 172.21.15.115:0/1587812515 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fd2f4004bc0 con 0x7fd30409c420 2024-01-24T23:21:10.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.603+0000 7fd2f2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:10.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.603+0000 7fd2f2ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:10.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.719+0000 7f9b8e7fc640 1 -- 172.21.15.115:0/304887310 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f9b90040550 con 0x7f9b981343b0 2024-01-24T23:21:10.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.719+0000 7fd0e6ffd640 1 -- 172.21.15.115:0/1778210534 <== mon.1 v1:172.21.15.132:6789/0 10 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fd0ec041620 con 0x7fd0f0133bf0 2024-01-24T23:21:10.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.719+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fd20c004190 con 0x7fd21002a550 2024-01-24T23:21:10.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.719+0000 7fd2f2ffd640 1 -- 172.21.15.115:0/1587812515 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fd2f40418b0 con 0x7fd30409c420 2024-01-24T23:21:10.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.723+0000 7fb112ffd640 1 -- 172.21.15.115:0/3195832231 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fb114011610 con 0x7fb11c1343b0 2024-01-24T23:21:10.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.723+0000 7faa4d7fa640 1 -- 172.21.15.115:0/2240506585 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7faa440415c0 con 0x7faa5009f8c0 2024-01-24T23:21:10.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.775+0000 7fd21759e640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2024-01-24T23:21:10.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.775+0000 7fd21759e640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-01-24T23:21:10.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.775+0000 7fd21759e640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:10.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.775+0000 7fd21759e640 1 -- 172.21.15.115:0/2672339919 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd2100066b0 con 0x7fd21002a550 2024-01-24T23:21:10.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.779+0000 7fd206ffd640 1 -- 172.21.15.115:0/2672339919 <== mon.8 v1:172.21.15.115:6793/0 11 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (unknown 574334944 0 847495621) 0x7fd20c0423e0 con 0x7fd21002a550 2024-01-24T23:21:10.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.779+0000 7fd206ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-01-24T23:21:10.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.779+0000 7fd206ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:10.782 INFO:teuthology.orchestra.run.smithi115.stdout:47244640259 2024-01-24T23:21:10.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.779+0000 7fd21759e640 1 -- 172.21.15.115:0/2672339919 >> v1:172.21.15.132:6812/1835401922 conn(0x7fd1f8037be0 legacy=0x7fd1f803a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.779+0000 7fd21759e640 10 monclient: shutdown 2024-01-24T23:21:10.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.779+0000 7fd21759e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.779+0000 7fd21759e640 1 -- 172.21.15.115:0/2672339919 >> v1:172.21.15.115:6793/0 conn(0x7fd21002a550 legacy=0x7fd210020170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.779+0000 7fd21759e640 1 -- 172.21.15.115:0/2672339919 shutdown_connections 2024-01-24T23:21:10.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.779+0000 7fd21759e640 1 -- 172.21.15.115:0/2672339919 >> 172.21.15.115:0/2672339919 conn(0x7fd210007550 msgr2=0x7fd210026220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.779+0000 7fd21759e640 1 -- 172.21.15.115:0/2672339919 shutdown_connections 2024-01-24T23:21:10.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.779+0000 7fd21759e640 1 -- 172.21.15.115:0/2672339919 wait complete. 2024-01-24T23:21:10.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.783+0000 7f9b7bfff640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2024-01-24T23:21:10.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.783+0000 7f9b7bfff640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-01-24T23:21:10.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.783+0000 7f9b7bfff640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:10.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.783+0000 7f9b7bfff640 1 -- 172.21.15.115:0/304887310 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9b74005340 con 0x7f9b981343b0 2024-01-24T23:21:10.799 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640259 for osd.0 2024-01-24T23:21:10.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.843+0000 7f9b8e7fc640 1 -- 172.21.15.115:0/304887310 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (unknown 832126871 0 555833778) 0x7f9b90013880 con 0x7f9b981343b0 2024-01-24T23:21:10.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.843+0000 7f9b8e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-01-24T23:21:10.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.843+0000 7f9b8e7fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:10.847 INFO:teuthology.orchestra.run.smithi115.stdout:47244640258 2024-01-24T23:21:10.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.847+0000 7f9b7bfff640 1 -- 172.21.15.115:0/304887310 >> v1:172.21.15.132:6812/1835401922 conn(0x7f9b7c037720 legacy=0x7f9b7c039be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.847+0000 7f9b7bfff640 10 monclient: shutdown 2024-01-24T23:21:10.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.847+0000 7f9b7bfff640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.847+0000 7f9b7bfff640 1 -- 172.21.15.115:0/304887310 >> v1:172.21.15.132:6792/0 conn(0x7f9b981343b0 legacy=0x7f9b981d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.847+0000 7f9b7bfff640 1 -- 172.21.15.115:0/304887310 shutdown_connections 2024-01-24T23:21:10.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.847+0000 7f9b7bfff640 1 -- 172.21.15.115:0/304887310 >> 172.21.15.115:0/304887310 conn(0x7f9b980662f0 msgr2=0x7f9b981be7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.847+0000 7f9b7bfff640 1 -- 172.21.15.115:0/304887310 shutdown_connections 2024-01-24T23:21:10.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.847+0000 7f9b7bfff640 1 -- 172.21.15.115:0/304887310 wait complete. 2024-01-24T23:21:10.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.847+0000 7fb110ff9640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2024-01-24T23:21:10.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.847+0000 7fb110ff9640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-01-24T23:21:10.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.847+0000 7fb110ff9640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:10.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.847+0000 7fb110ff9640 1 -- 172.21.15.115:0/3195832231 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb11c09fa60 con 0x7fb11c1343b0 2024-01-24T23:21:10.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.847+0000 7fb112ffd640 1 -- 172.21.15.115:0/3195832231 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (unknown 2131975755 0 3327677263) 0x7fb1140151b0 con 0x7fb11c1343b0 2024-01-24T23:21:10.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.847+0000 7fb112ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-01-24T23:21:10.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.847+0000 7fb112ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:10.852 INFO:teuthology.orchestra.run.smithi115.stdout:47244640257 2024-01-24T23:21:10.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.851+0000 7fb110ff9640 1 -- 172.21.15.115:0/3195832231 >> v1:172.21.15.132:6812/1835401922 conn(0x7fb100036fb0 legacy=0x7fb100039470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.851+0000 7fb110ff9640 10 monclient: shutdown 2024-01-24T23:21:10.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.851+0000 7fb110ff9640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.851+0000 7fb110ff9640 1 -- 172.21.15.115:0/3195832231 >> v1:172.21.15.132:6792/0 conn(0x7fb11c1343b0 legacy=0x7fb11c1d39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.851+0000 7fb110ff9640 1 -- 172.21.15.115:0/3195832231 shutdown_connections 2024-01-24T23:21:10.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.851+0000 7fb110ff9640 1 -- 172.21.15.115:0/3195832231 >> 172.21.15.115:0/3195832231 conn(0x7fb11c0662f0 msgr2=0x7fb11c1bea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.851+0000 7fb110ff9640 1 -- 172.21.15.115:0/3195832231 shutdown_connections 2024-01-24T23:21:10.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.851+0000 7fb110ff9640 1 -- 172.21.15.115:0/3195832231 wait complete. 2024-01-24T23:21:10.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.855+0000 7faa56c6a640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2024-01-24T23:21:10.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.855+0000 7faa56c6a640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-01-24T23:21:10.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.855+0000 7faa56c6a640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:10.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.855+0000 7faa56c6a640 1 -- 172.21.15.115:0/2240506585 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7faa30005340 con 0x7faa5009f8c0 2024-01-24T23:21:10.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.855+0000 7faa4d7fa640 1 -- 172.21.15.115:0/2240506585 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (unknown 383520633 0 847495621) 0x7faa44015220 con 0x7faa5009f8c0 2024-01-24T23:21:10.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.855+0000 7faa4d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-01-24T23:21:10.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.855+0000 7faa4d7fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:10.858 INFO:teuthology.orchestra.run.smithi115.stdout:47244640259 2024-01-24T23:21:10.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.855+0000 7faa56c6a640 1 -- 172.21.15.115:0/2240506585 >> v1:172.21.15.132:6812/1835401922 conn(0x7faa34037480 legacy=0x7faa34039940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.855+0000 7faa56c6a640 10 monclient: shutdown 2024-01-24T23:21:10.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.855+0000 7faa56c6a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.855+0000 7faa56c6a640 1 -- 172.21.15.115:0/2240506585 >> v1:172.21.15.132:6792/0 conn(0x7faa5009f8c0 legacy=0x7faa501cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.855+0000 7faa56c6a640 1 -- 172.21.15.115:0/2240506585 shutdown_connections 2024-01-24T23:21:10.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.855+0000 7faa56c6a640 1 -- 172.21.15.115:0/2240506585 >> 172.21.15.115:0/2240506585 conn(0x7faa500a6490 msgr2=0x7faa5009e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.855+0000 7faa56c6a640 1 -- 172.21.15.115:0/2240506585 shutdown_connections 2024-01-24T23:21:10.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.855+0000 7faa56c6a640 1 -- 172.21.15.115:0/2240506585 wait complete. 2024-01-24T23:21:10.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.863+0000 7fd0f8781640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2024-01-24T23:21:10.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.863+0000 7fd0f8781640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-01-24T23:21:10.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.863+0000 7fd0f8781640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:21:10.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.863+0000 7fd0f8781640 1 -- 172.21.15.115:0/1778210534 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fd0d8005340 con 0x7fd0f0133bf0 2024-01-24T23:21:10.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.863+0000 7fd0e6ffd640 1 -- 172.21.15.115:0/1778210534 <== mon.1 v1:172.21.15.132:6789/0 11 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (unknown 1823589948 0 3327677263) 0x7fd0ec014930 con 0x7fd0f0133bf0 2024-01-24T23:21:10.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.863+0000 7fd0e6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-01-24T23:21:10.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.863+0000 7fd0e6ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:10.868 INFO:teuthology.orchestra.run.smithi115.stdout:47244640257 2024-01-24T23:21:10.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.867+0000 7fd0f8781640 1 -- 172.21.15.115:0/1778210534 >> v1:172.21.15.132:6812/1835401922 conn(0x7fd0d4036fb0 legacy=0x7fd0d4039470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.867+0000 7fd0f8781640 10 monclient: shutdown 2024-01-24T23:21:10.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.867+0000 7fd0f8781640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.867+0000 7fd0f8781640 1 -- 172.21.15.115:0/1778210534 >> v1:172.21.15.132:6789/0 conn(0x7fd0f0133bf0 legacy=0x7fd0f00a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.867+0000 7fd0f8781640 1 -- 172.21.15.115:0/1778210534 shutdown_connections 2024-01-24T23:21:10.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.867+0000 7fd0f8781640 1 -- 172.21.15.115:0/1778210534 >> 172.21.15.115:0/1778210534 conn(0x7fd0f00662f0 msgr2=0x7fd0f009d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.867+0000 7fd0f8781640 1 -- 172.21.15.115:0/1778210534 shutdown_connections 2024-01-24T23:21:10.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.867+0000 7fd0f8781640 1 -- 172.21.15.115:0/1778210534 wait complete. 2024-01-24T23:21:10.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.871+0000 7fd308868640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2024-01-24T23:21:10.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.871+0000 7fd308868640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-01-24T23:21:10.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.871+0000 7fd308868640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:10.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.871+0000 7fd308868640 1 -- 172.21.15.115:0/1587812515 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd2e8005340 con 0x7fd30409c420 2024-01-24T23:21:10.876 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640258 for osd.1 2024-01-24T23:21:10.879 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640259 for osd.3 2024-01-24T23:21:10.880 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640258 got 47244640257 for osd.5 2024-01-24T23:21:10.889 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640258 got 47244640257 for osd.4 2024-01-24T23:21:10.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.931+0000 7fd2f2ffd640 1 -- 172.21.15.115:0/1587812515 <== mon.8 v1:172.21.15.115:6793/0 11 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (unknown 92182286 0 3327677263) 0x7fd2f4014f20 con 0x7fd30409c420 2024-01-24T23:21:10.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.931+0000 7fd2f2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-01-24T23:21:10.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.931+0000 7fd2f2ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:10.934 INFO:teuthology.orchestra.run.smithi115.stdout:47244640257 2024-01-24T23:21:10.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.931+0000 7fd308868640 1 -- 172.21.15.115:0/1587812515 >> v1:172.21.15.132:6812/1835401922 conn(0x7fd2e4037590 legacy=0x7fd2e4039a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.931+0000 7fd308868640 10 monclient: shutdown 2024-01-24T23:21:10.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.931+0000 7fd308868640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:10.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.931+0000 7fd308868640 1 -- 172.21.15.115:0/1587812515 >> v1:172.21.15.115:6793/0 conn(0x7fd30409c420 legacy=0x7fd3041c3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:10.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.931+0000 7fd308868640 1 -- 172.21.15.115:0/1587812515 shutdown_connections 2024-01-24T23:21:10.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.931+0000 7fd308868640 1 -- 172.21.15.115:0/1587812515 >> 172.21.15.115:0/1587812515 conn(0x7fd3040662f0 msgr2=0x7fd30409d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:10.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.931+0000 7fd308868640 1 -- 172.21.15.115:0/1587812515 shutdown_connections 2024-01-24T23:21:10.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:10.931+0000 7fd308868640 1 -- 172.21.15.115:0/1587812515 wait complete. 2024-01-24T23:21:10.951 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640258 got 47244640257 for osd.2 2024-01-24T23:21:11.800 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-24T23:21:11.877 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-24T23:21:11.880 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-24T23:21:11.882 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-01-24T23:21:11.890 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-01-24T23:21:11.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a9dba9640 10 monclient: get_monmap_and_config 2024-01-24T23:21:11.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a9dba9640 10 monclient: build_initial_monmap 2024-01-24T23:21:11.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a9dba9640 10 monclient: monmap: 2024-01-24T23:21:11.926 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:11.926 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:11.926 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:11.924631+0000 2024-01-24T23:21:11.926 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:11.924631+0000 2024-01-24T23:21:11.926 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:11.926 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:11.927 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:11.927 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:11.927 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:11.927 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:11.927 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:11.927 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:11.927 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:11.927 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:11.927 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:11.927 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:11.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a9dba9640 1 Processor -- start 2024-01-24T23:21:11.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a9dba9640 1 -- start start 2024-01-24T23:21:11.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a9dba9640 10 monclient: init 2024-01-24T23:21:11.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a9dba9640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:11.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a9dba9640 10 monclient: _add_conns ranks=[5,2,0,4,8,1,6,3,7] 2024-01-24T23:21:11.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a9dba9640 10 monclient(hunting): picked mon.noname-h con 0x7f7a98134b00 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:11.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a9dba9640 10 monclient(hunting): picked mon.noname-b con 0x7f7a9809e6a0 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:11.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a9dba9640 10 monclient(hunting): picked mon.noname-a con 0x7f7a9813b110 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:11.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a97fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f7a9813b110 0x7f7a9813d500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:52136/0 (socket says 172.21.15.115:52136) 2024-01-24T23:21:11.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a97fff640 1 -- 172.21.15.115:0/60945184 learned_addr learned my addr 172.21.15.115:0/60945184 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:11.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a9dba9640 1 -- 172.21.15.115:0/60945184 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a9804ed70 con 0x7f7a9813b110 2024-01-24T23:21:11.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a9dba9640 1 -- 172.21.15.115:0/60945184 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a98072360 con 0x7f7a9809e6a0 2024-01-24T23:21:11.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a9dba9640 1 -- 172.21.15.115:0/60945184 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a981355f0 con 0x7f7a98134b00 2024-01-24T23:21:11.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a9dba9640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:11.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a9dba9640 10 monclient(hunting): authenticate will time out at 950.847580s 2024-01-24T23:21:11.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a967fc640 1 -- 172.21.15.115:0/60945184 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2885468917 0 0) 0x7f7a9804ed70 con 0x7f7a9813b110 2024-01-24T23:21:11.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a967fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:11.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a967fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:11.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a967fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:11.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a967fc640 10 monclient(hunting): my global_id is 4347 2024-01-24T23:21:11.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a967fc640 1 -- 172.21.15.115:0/60945184 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a80003530 con 0x7f7a9813b110 2024-01-24T23:21:11.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a967fc640 1 -- 172.21.15.115:0/60945184 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3253383311 0 0) 0x7f7a98072360 con 0x7f7a9809e6a0 2024-01-24T23:21:11.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a967fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:11.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a967fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:11.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a967fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:11.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a967fc640 10 monclient(hunting): my global_id is 4268 2024-01-24T23:21:11.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.919+0000 7f7a967fc640 1 -- 172.21.15.115:0/60945184 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a9804ed70 con 0x7f7a9809e6a0 2024-01-24T23:21:11.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 1 -- 172.21.15.115:0/60945184 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2393406470 0 0) 0x7f7a981355f0 con 0x7f7a98134b00 2024-01-24T23:21:11.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:11.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:11.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:11.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 10 monclient(hunting): my global_id is 4316 2024-01-24T23:21:11.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 1 -- 172.21.15.115:0/60945184 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a98072360 con 0x7f7a98134b00 2024-01-24T23:21:11.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 1 -- 172.21.15.115:0/60945184 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1952882153 0 0) 0x7f7a9804ed70 con 0x7f7a9809e6a0 2024-01-24T23:21:11.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 1 -- 172.21.15.115:0/60945184 >> v1:172.21.15.132:6791/0 conn(0x7f7a98134b00 legacy=0x7f7a98134ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:11.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 1 -- 172.21.15.115:0/60945184 >> v1:172.21.15.115:6789/0 conn(0x7f7a9813b110 legacy=0x7f7a9813d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:11.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 10 monclient: _finish_hunting 0 2024-01-24T23:21:11.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 1 monclient: found mon.noname-b 2024-01-24T23:21:11.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:11.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:21:11.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 1 -- 172.21.15.115:0/60945184 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a980677b0 con 0x7f7a9809e6a0 2024-01-24T23:21:11.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 10 monclient: _finish_auth 0 2024-01-24T23:21:11.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 10 monclient: _check_auth_tickets 2024-01-24T23:21:11.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:11.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 1 -- 172.21.15.115:0/60945184 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7a8c0017a0 con 0x7f7a9809e6a0 2024-01-24T23:21:11.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:11.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:21:11.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 10 monclient: dump: 2024-01-24T23:21:11.930 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:11.930 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:11.930 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:11.930 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:11.930 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:11.930 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:11.931 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 5 monclient: authenticate success, global_id 4268 2024-01-24T23:21:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 1 -- 172.21.15.115:0/60945184 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7a8c0019c0 con 0x7f7a9809e6a0 2024-01-24T23:21:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 1 -- 172.21.15.115:0/60945184 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7a8c003960 con 0x7f7a9809e6a0 2024-01-24T23:21:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:11.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 4 set_mon_vals no callback set 2024-01-24T23:21:11.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:11.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:11.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a967fc640 10 monclient: dump: 2024-01-24T23:21:11.932 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:11.932 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:11.932 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:11.932 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:11.932 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:11.932 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:11.932 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:11.932 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:11.932 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:11.932 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:11.932 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:11.933 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:11.933 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:11.933 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:11.933 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:11.933 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:11.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 10 monclient: shutdown 2024-01-24T23:21:11.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:11.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 1 -- 172.21.15.115:0/60945184 >> v1:172.21.15.115:6790/0 conn(0x7f7a9809e6a0 legacy=0x7f7a981399f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:11.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 1 -- 172.21.15.115:0/60945184 shutdown_connections 2024-01-24T23:21:11.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 1 -- 172.21.15.115:0/60945184 >> 172.21.15.115:0/60945184 conn(0x7f7a980a63e0 msgr2=0x7f7a980a67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:11.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 1 -- 172.21.15.115:0/60945184 shutdown_connections 2024-01-24T23:21:11.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 1 -- 172.21.15.115:0/60945184 wait complete. 2024-01-24T23:21:11.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 10 monclient: build_initial_monmap 2024-01-24T23:21:11.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 10 monclient: monmap: 2024-01-24T23:21:11.937 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:11.937 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:11.937 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:11.937 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:11.937 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:11.937 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:11.937 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:11.937 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:11.938 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:11.938 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:11.938 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:11.938 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:11.938 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:11.938 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:11.938 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:11.938 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:11.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 1 Processor -- start 2024-01-24T23:21:11.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 1 -- start start 2024-01-24T23:21:11.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 10 monclient: init 2024-01-24T23:21:11.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:11.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 10 monclient: _add_conns ranks=[7,2,0,3,6,8,1,4,5] 2024-01-24T23:21:11.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 10 monclient(hunting): picked mon.noname-i con 0x7f7a9809e6a0 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:11.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 10 monclient(hunting): picked mon.noname-b con 0x7f7a98134b00 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:11.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 10 monclient(hunting): picked mon.noname-a con 0x7f7a9813b110 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:11.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a98136790 con 0x7f7a9813b110 2024-01-24T23:21:11.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a980a20e0 con 0x7f7a98134b00 2024-01-24T23:21:11.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a98071fc0 con 0x7f7a9809e6a0 2024-01-24T23:21:11.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:11.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a9dba9640 10 monclient(hunting): authenticate will time out at 950.851863s 2024-01-24T23:21:11.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a96ffd640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f7a98134b00 0x7f7a981ce6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:54568/0 (socket says 172.21.15.115:54568) 2024-01-24T23:21:11.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.923+0000 7f7a96ffd640 1 -- 172.21.15.115:0/3467174574 learned_addr learned my addr 172.21.15.115:0/3467174574 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:11.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 1 -- 172.21.15.115:0/3467174574 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3243694825 0 0) 0x7f7a980a20e0 con 0x7f7a98134b00 2024-01-24T23:21:11.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:11.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:11.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:11.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient(hunting): my global_id is 4277 2024-01-24T23:21:11.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 1 -- 172.21.15.115:0/3467174574 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a84003530 con 0x7f7a98134b00 2024-01-24T23:21:11.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 1 -- 172.21.15.115:0/3467174574 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3389049551 0 0) 0x7f7a98136790 con 0x7f7a9813b110 2024-01-24T23:21:11.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:11.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:11.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:11.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient(hunting): my global_id is 4356 2024-01-24T23:21:11.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 1 -- 172.21.15.115:0/3467174574 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a980a20e0 con 0x7f7a9813b110 2024-01-24T23:21:11.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 1 -- 172.21.15.115:0/3467174574 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 314110069 0 0) 0x7f7a98071fc0 con 0x7f7a9809e6a0 2024-01-24T23:21:11.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:11.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:11.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:11.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient(hunting): my global_id is 4345 2024-01-24T23:21:11.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 1 -- 172.21.15.115:0/3467174574 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a98136790 con 0x7f7a9809e6a0 2024-01-24T23:21:11.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 1 -- 172.21.15.115:0/3467174574 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1024787260 0 0) 0x7f7a84003530 con 0x7f7a98134b00 2024-01-24T23:21:11.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 1 -- 172.21.15.115:0/3467174574 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7a98071fc0 con 0x7f7a98134b00 2024-01-24T23:21:11.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 1 -- 172.21.15.115:0/3467174574 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 349458766 0 0) 0x7f7a980a20e0 con 0x7f7a9813b110 2024-01-24T23:21:11.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 1 -- 172.21.15.115:0/3467174574 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7a84003530 con 0x7f7a9813b110 2024-01-24T23:21:11.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 1 -- 172.21.15.115:0/3467174574 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7a8c003a20 con 0x7f7a98134b00 2024-01-24T23:21:11.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:11.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:21:11.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient(hunting): dump: 2024-01-24T23:21:11.941 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:11.941 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:11.941 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:11.941 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:11.941 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:11.941 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:11.941 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:11.941 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:11.941 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:11.941 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:11.941 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:11.941 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:11.941 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:11.942 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:11.942 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:11.942 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:11.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:11.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 1 -- 172.21.15.115:0/3467174574 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7a90001b30 con 0x7f7a9813b110 2024-01-24T23:21:11.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:11.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:11.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient(hunting): dump: 2024-01-24T23:21:11.942 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:11.942 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:11.942 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:11.942 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:11.942 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:11.942 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:11.942 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:11.942 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:11.942 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:11.943 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:11.943 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:11.943 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:11.943 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:11.943 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:11.943 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:11.943 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:11.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 1 -- 172.21.15.115:0/3467174574 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2390048784 0 0) 0x7f7a98071fc0 con 0x7f7a98134b00 2024-01-24T23:21:11.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 1 -- 172.21.15.115:0/3467174574 >> v1:172.21.15.132:6792/0 conn(0x7f7a9809e6a0 legacy=0x7f7a9812c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:11.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 1 -- 172.21.15.115:0/3467174574 >> v1:172.21.15.115:6789/0 conn(0x7f7a9813b110 legacy=0x7f7a981d1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:11.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient: _finish_hunting 0 2024-01-24T23:21:11.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 1 monclient: found mon.b 2024-01-24T23:21:11.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:11.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:11.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 1 -- 172.21.15.115:0/3467174574 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a98071120 con 0x7f7a98134b00 2024-01-24T23:21:11.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient: _finish_auth 0 2024-01-24T23:21:11.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient: _check_auth_tickets 2024-01-24T23:21:11.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:11.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a9dba9640 5 monclient: authenticate success, global_id 4277 2024-01-24T23:21:11.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a9dba9640 10 monclient: _renew_subs 2024-01-24T23:21:11.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a9dba9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:11.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a9dba9640 1 -- 172.21.15.115:0/3467174574 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a98135ef0 con 0x7f7a98134b00 2024-01-24T23:21:11.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a9dba9640 10 monclient: _renew_subs 2024-01-24T23:21:11.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a9dba9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:11.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a9dba9640 1 -- 172.21.15.115:0/3467174574 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a98134f90 con 0x7f7a98134b00 2024-01-24T23:21:11.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 1 -- 172.21.15.115:0/3467174574 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7a8c003c20 con 0x7f7a98134b00 2024-01-24T23:21:11.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:11.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a95ffb640 4 set_mon_vals no callback set 2024-01-24T23:21:11.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 1 -- 172.21.15.115:0/3467174574 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7a8c0040d0 con 0x7f7a98134b00 2024-01-24T23:21:11.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:11.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:11.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 10 monclient: dump: 2024-01-24T23:21:11.945 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:11.945 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:11.945 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:11.945 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:11.945 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:11.945 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:11.945 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:11.945 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:11.945 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:11.945 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:11.945 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:11.945 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:11.945 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:11.945 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:11.946 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:11.946 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:11.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 1 -- 172.21.15.115:0/3467174574 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f7a8c0042b0 con 0x7f7a98134b00 2024-01-24T23:21:11.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a9dba9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:11.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a7ffff640 1 -- 172.21.15.115:0/3467174574 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f7a8c043890 con 0x7f7a98134b00 2024-01-24T23:21:11.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a9dba9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:11.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a9dba9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:11.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a9dba9640 1 -- 172.21.15.115:0/3467174574 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a78005090 con 0x7f7a98134b00 2024-01-24T23:21:11.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.927+0000 7f7a977fe640 10 monclient: get_auth_request con 0x7f7a84037920 auth_method 0 2024-01-24T23:21:11.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.935+0000 7f7a7ffff640 1 -- 172.21.15.115:0/3467174574 <== mon.2 v1:172.21.15.115:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f7a8c014d80 con 0x7f7a98134b00 2024-01-24T23:21:11.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.935+0000 7f7a7ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:11.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.935+0000 7f7a7ffff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:11.952 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-24T23:21:11.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.987+0000 7fbef299b640 10 monclient: get_monmap_and_config 2024-01-24T23:21:11.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.987+0000 7fbef299b640 10 monclient: build_initial_monmap 2024-01-24T23:21:11.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.987+0000 7fbef299b640 10 monclient: monmap: 2024-01-24T23:21:11.993 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:11.993 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:11.993 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:11.994336+0000 2024-01-24T23:21:11.993 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:11.994336+0000 2024-01-24T23:21:11.993 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:11.993 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:11.993 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:11.993 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:11.993 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:11.993 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:11.994 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:11.994 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:11.994 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:11.994 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:11.994 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:11.994 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:11.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 1 Processor -- start 2024-01-24T23:21:11.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 1 -- start start 2024-01-24T23:21:11.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 10 monclient: init 2024-01-24T23:21:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 10 monclient: _add_conns ranks=[0,5,3,8,2,1,4,6,7] 2024-01-24T23:21:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 10 monclient(hunting): picked mon.noname-a con 0x7fbeec133bf0 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 10 monclient(hunting): picked mon.noname-h con 0x7fbeec133fd0 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 10 monclient(hunting): picked mon.noname-g con 0x7fbeec1343b0 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeec04ed70 con 0x7fbeec133bf0 2024-01-24T23:21:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeec0a4b60 con 0x7fbeec1343b0 2024-01-24T23:21:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeec09ae50 con 0x7fbeec133fd0 2024-01-24T23:21:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 10 monclient(hunting): authenticate will time out at 950.916814s 2024-01-24T23:21:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeebfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fbeec133bf0 0x7fbeec09bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:52162/0 (socket says 172.21.15.115:52162) 2024-01-24T23:21:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeebfff640 1 -- 172.21.15.115:0/3675757029 learned_addr learned my addr 172.21.15.115:0/3675757029 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 1 -- 172.21.15.115:0/3675757029 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302224051 0 0) 0x7fbeec04ed70 con 0x7fbeec133bf0 2024-01-24T23:21:11.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:11.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:11.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:11.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 10 monclient(hunting): my global_id is 4365 2024-01-24T23:21:11.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 1 -- 172.21.15.115:0/3675757029 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbed4003530 con 0x7fbeec133bf0 2024-01-24T23:21:11.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 1 -- 172.21.15.115:0/3675757029 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1835177750 0 0) 0x7fbeec09ae50 con 0x7fbeec133fd0 2024-01-24T23:21:11.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:11.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:11.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:11.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 10 monclient(hunting): my global_id is 4325 2024-01-24T23:21:11.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 1 -- 172.21.15.115:0/3675757029 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbeec04ed70 con 0x7fbeec133fd0 2024-01-24T23:21:11.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 1 -- 172.21.15.115:0/3675757029 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1879129401 0 0) 0x7fbed4003530 con 0x7fbeec133bf0 2024-01-24T23:21:11.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 1 -- 172.21.15.115:0/3675757029 >> v1:172.21.15.132:6791/0 conn(0x7fbeec133fd0 legacy=0x7fbeec13bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:11.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 1 -- 172.21.15.115:0/3675757029 >> v1:172.21.15.132:6790/0 conn(0x7fbeec1343b0 legacy=0x7fbeec13f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:11.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 10 monclient: _finish_hunting 0 2024-01-24T23:21:11.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 1 monclient: found mon.noname-a 2024-01-24T23:21:11.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:11.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:21:11.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 1 -- 172.21.15.115:0/3675757029 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeec09b130 con 0x7fbeec133bf0 2024-01-24T23:21:11.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 10 monclient: _finish_auth 0 2024-01-24T23:21:11.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 10 monclient: _check_auth_tickets 2024-01-24T23:21:11.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:11.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 1 -- 172.21.15.115:0/3675757029 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbedc0017c0 con 0x7fbeec133bf0 2024-01-24T23:21:11.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:11.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:21:11.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 10 monclient: dump: 2024-01-24T23:21:11.998 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:11.998 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:11.998 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:11.998 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:11.998 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:11.998 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:11.998 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:11.999 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:11.999 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:11.999 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:11.999 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:11.999 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:11.999 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:11.999 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:11.999 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:11.999 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:11.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 5 monclient: authenticate success, global_id 4365 2024-01-24T23:21:11.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:11.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 1 -- 172.21.15.115:0/3675757029 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbedc0019e0 con 0x7fbeec133bf0 2024-01-24T23:21:11.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:11.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 1 -- 172.21.15.115:0/3675757029 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbedc001c10 con 0x7fbeec133bf0 2024-01-24T23:21:11.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:12.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:12.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbeeaffd640 10 monclient: dump: 2024-01-24T23:21:12.000 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:12.000 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.000 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.000 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.000 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:12.000 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.000 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:12.000 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:12.000 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:12.000 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:12.000 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:12.000 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:12.000 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:12.000 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:12.000 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:12.001 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:12.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 4 set_mon_vals no callback set 2024-01-24T23:21:12.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 10 monclient: shutdown 2024-01-24T23:21:12.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:12.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 1 -- 172.21.15.115:0/3675757029 >> v1:172.21.15.115:6789/0 conn(0x7fbeec133bf0 legacy=0x7fbeec09bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 1 -- 172.21.15.115:0/3675757029 shutdown_connections 2024-01-24T23:21:12.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 1 -- 172.21.15.115:0/3675757029 >> 172.21.15.115:0/3675757029 conn(0x7fbeec0662f0 msgr2=0x7fbeec066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:12.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 1 -- 172.21.15.115:0/3675757029 shutdown_connections 2024-01-24T23:21:12.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 1 -- 172.21.15.115:0/3675757029 wait complete. 2024-01-24T23:21:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 10 monclient: build_initial_monmap 2024-01-24T23:21:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.991+0000 7fbef299b640 10 monclient: monmap: 2024-01-24T23:21:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:12.006 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:12.006 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbef299b640 1 Processor -- start 2024-01-24T23:21:12.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbef299b640 1 -- start start 2024-01-24T23:21:12.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbef299b640 10 monclient: init 2024-01-24T23:21:12.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbef299b640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:12.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbef299b640 10 monclient: _add_conns ranks=[2,7,1,5,8,3,0,4,6] 2024-01-24T23:21:12.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbef299b640 10 monclient(hunting): picked mon.noname-b con 0x7fbeec133bf0 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:12.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbef299b640 10 monclient(hunting): picked mon.noname-i con 0x7fbeec133fd0 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:12.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbef299b640 10 monclient(hunting): picked mon.noname-f con 0x7fbeec1343b0 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:12.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbef299b640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeec0a1e00 con 0x7fbeec1343b0 2024-01-24T23:21:12.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbeebfff640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fbeec133bf0 0x7fbeec132fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:54580/0 (socket says 172.21.15.115:54580) 2024-01-24T23:21:12.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbeebfff640 1 -- 172.21.15.115:0/1925314761 learned_addr learned my addr 172.21.15.115:0/1925314761 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:12.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbef299b640 1 -- 172.21.15.115:0/1925314761 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeec120990 con 0x7fbeec133bf0 2024-01-24T23:21:12.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbef299b640 1 -- 172.21.15.115:0/1925314761 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeec09c810 con 0x7fbeec133fd0 2024-01-24T23:21:12.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbef299b640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:12.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbef299b640 10 monclient(hunting): authenticate will time out at 950.920969s 2024-01-24T23:21:12.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 1 -- 172.21.15.115:0/1925314761 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2150776153 0 0) 0x7fbeec120990 con 0x7fbeec133bf0 2024-01-24T23:21:12.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:12.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:12.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:12.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 10 monclient(hunting): my global_id is 4286 2024-01-24T23:21:12.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 1 -- 172.21.15.115:0/1925314761 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbed8003530 con 0x7fbeec133bf0 2024-01-24T23:21:12.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 1 -- 172.21.15.115:0/1925314761 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3451393746 0 0) 0x7fbeec0a1e00 con 0x7fbeec1343b0 2024-01-24T23:21:12.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:12.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:12.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:12.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 10 monclient(hunting): my global_id is 4303 2024-01-24T23:21:12.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 1 -- 172.21.15.115:0/1925314761 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbeec120990 con 0x7fbeec1343b0 2024-01-24T23:21:12.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 1 -- 172.21.15.115:0/1925314761 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2665920760 0 0) 0x7fbeec09c810 con 0x7fbeec133fd0 2024-01-24T23:21:12.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:12.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:12.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:12.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 10 monclient(hunting): my global_id is 4354 2024-01-24T23:21:12.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 1 -- 172.21.15.115:0/1925314761 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbeec0a1e00 con 0x7fbeec133fd0 2024-01-24T23:21:12.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 1 -- 172.21.15.115:0/1925314761 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 422642439 0 0) 0x7fbed8003530 con 0x7fbeec133bf0 2024-01-24T23:21:12.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 1 -- 172.21.15.115:0/1925314761 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbeec09c810 con 0x7fbeec133bf0 2024-01-24T23:21:12.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 1 -- 172.21.15.115:0/1925314761 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbedc001b50 con 0x7fbeec133bf0 2024-01-24T23:21:12.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:12.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:21:12.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 10 monclient(hunting): dump: 2024-01-24T23:21:12.008 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:12.008 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.008 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.008 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.008 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:12.008 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.008 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:12.008 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:12.009 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:12.009 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:12.009 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:12.009 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:12.009 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:12.009 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:12.009 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:12.009 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:12.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 1 -- 172.21.15.115:0/1925314761 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2878571821 0 0) 0x7fbeec09c810 con 0x7fbeec133bf0 2024-01-24T23:21:12.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 1 -- 172.21.15.115:0/1925314761 >> v1:172.21.15.132:6792/0 conn(0x7fbeec133fd0 legacy=0x7fbeec1d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 1 -- 172.21.15.115:0/1925314761 >> v1:172.21.15.132:6789/0 conn(0x7fbeec1343b0 legacy=0x7fbeec1d3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:21:12.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 1 monclient: found mon.b 2024-01-24T23:21:12.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:12.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:12.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 1 -- 172.21.15.115:0/1925314761 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeec0a5990 con 0x7fbeec133bf0 2024-01-24T23:21:12.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 10 monclient: _finish_auth 0 2024-01-24T23:21:12.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:21:12.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:12.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbef299b640 5 monclient: authenticate success, global_id 4286 2024-01-24T23:21:12.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbef299b640 10 monclient: _renew_subs 2024-01-24T23:21:12.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbef299b640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:12.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbef299b640 1 -- 172.21.15.115:0/1925314761 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeec1336c0 con 0x7fbeec133bf0 2024-01-24T23:21:12.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbef299b640 10 monclient: _renew_subs 2024-01-24T23:21:12.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 1 -- 172.21.15.115:0/1925314761 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbedc0017a0 con 0x7fbeec133bf0 2024-01-24T23:21:12.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbef299b640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:12.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbef299b640 1 -- 172.21.15.115:0/1925314761 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeec0a2d40 con 0x7fbeec133bf0 2024-01-24T23:21:12.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:12.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 1 -- 172.21.15.115:0/1925314761 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbedc005120 con 0x7fbeec133bf0 2024-01-24T23:21:12.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:12.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:12.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 10 monclient: dump: 2024-01-24T23:21:12.011 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:12.011 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.011 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.011 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.011 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:12.011 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.011 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:12.011 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:12.011 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:12.011 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:12.011 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:12.011 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:12.011 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:12.011 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:12.011 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:12.011 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbeea7fc640 4 set_mon_vals no callback set 2024-01-24T23:21:12.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.995+0000 7fbee8ff9640 1 -- 172.21.15.115:0/1925314761 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fbedc005280 con 0x7fbeec133bf0 2024-01-24T23:21:12.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.999+0000 7fbee8ff9640 1 -- 172.21.15.115:0/1925314761 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7fbedc015650 con 0x7fbeec133bf0 2024-01-24T23:21:12.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:11.999+0000 7fbeeb7fe640 10 monclient: get_auth_request con 0x7fbed8037230 auth_method 0 2024-01-24T23:21:12.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.003+0000 7fbed27fc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:12.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.003+0000 7fbed27fc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:12.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.003+0000 7fbed27fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:12.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.003+0000 7fbed27fc640 1 -- 172.21.15.115:0/1925314761 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeec120280 con 0x7fbeec133bf0 2024-01-24T23:21:12.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.011+0000 7fbee8ff9640 1 -- 172.21.15.115:0/1925314761 <== mon.2 v1:172.21.15.115:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fbedc006a70 con 0x7fbeec133bf0 2024-01-24T23:21:12.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.011+0000 7fbee8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:12.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.011+0000 7fbee8ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:12.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f08330be640 10 monclient: get_monmap_and_config 2024-01-24T23:21:12.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f08330be640 10 monclient: build_initial_monmap 2024-01-24T23:21:12.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f08330be640 10 monclient: monmap: 2024-01-24T23:21:12.062 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:12.063 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.063 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:12.064133+0000 2024-01-24T23:21:12.063 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:12.064133+0000 2024-01-24T23:21:12.063 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:12.063 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.063 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:12.063 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:12.063 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:12.063 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:12.063 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:12.063 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:12.063 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:12.063 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:12.063 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:12.063 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f08330be640 1 Processor -- start 2024-01-24T23:21:12.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f08330be640 1 -- start start 2024-01-24T23:21:12.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f08330be640 10 monclient: init 2024-01-24T23:21:12.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f08330be640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:12.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f08330be640 10 monclient: _add_conns ranks=[3,4,6,2,7,0,5,1,8] 2024-01-24T23:21:12.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f08330be640 10 monclient(hunting): picked mon.noname-g con 0x7f082c133bf0 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:12.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f08330be640 10 monclient(hunting): picked mon.noname-c con 0x7f082c133fd0 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:12.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f083165b640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f082c1343b0 0x7f082c13f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:56984/0 (socket says 172.21.15.115:56984) 2024-01-24T23:21:12.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f082bfff640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f082c133fd0 0x7f082c13bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:42504/0 (socket says 172.21.15.115:42504) 2024-01-24T23:21:12.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f083165b640 1 -- 172.21.15.115:0/3410634023 learned_addr learned my addr 172.21.15.115:0/3410634023 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:12.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f08330be640 10 monclient(hunting): picked mon.noname-d con 0x7f082c1343b0 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:12.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f08330be640 1 -- 172.21.15.115:0/3410634023 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082c04ed70 con 0x7f082c133bf0 2024-01-24T23:21:12.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f08330be640 1 -- 172.21.15.115:0/3410634023 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082c0a4b60 con 0x7f082c133fd0 2024-01-24T23:21:12.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f08330be640 1 -- 172.21.15.115:0/3410634023 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082c09ae50 con 0x7f082c1343b0 2024-01-24T23:21:12.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f08330be640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:12.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f08330be640 10 monclient(hunting): authenticate will time out at 950.987273s 2024-01-24T23:21:12.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f082b7fe640 1 -- 172.21.15.115:0/3410634023 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445070725 0 0) 0x7f082c04ed70 con 0x7f082c133bf0 2024-01-24T23:21:12.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f082b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:12.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f082b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:12.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f082b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:12.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f082b7fe640 10 monclient(hunting): my global_id is 4332 2024-01-24T23:21:12.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f082b7fe640 1 -- 172.21.15.115:0/3410634023 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f081c003530 con 0x7f082c133bf0 2024-01-24T23:21:12.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f082b7fe640 1 -- 172.21.15.115:0/3410634023 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371521624 0 0) 0x7f082c0a4b60 con 0x7f082c133fd0 2024-01-24T23:21:12.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f082b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:12.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f082b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:12.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f082b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:12.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f082b7fe640 10 monclient(hunting): my global_id is 74344 2024-01-24T23:21:12.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f082b7fe640 1 -- 172.21.15.115:0/3410634023 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f082c04ed70 con 0x7f082c133fd0 2024-01-24T23:21:12.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f082b7fe640 1 -- 172.21.15.115:0/3410634023 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1513533224 0 0) 0x7f082c09ae50 con 0x7f082c1343b0 2024-01-24T23:21:12.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f082b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:12.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f082b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:12.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f082b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:12.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f082b7fe640 10 monclient(hunting): my global_id is 4353 2024-01-24T23:21:12.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.059+0000 7f082b7fe640 1 -- 172.21.15.115:0/3410634023 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f082c0a4b60 con 0x7f082c1343b0 2024-01-24T23:21:12.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f082b7fe640 1 -- 172.21.15.115:0/3410634023 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3235846330 0 0) 0x7f082c04ed70 con 0x7f082c133fd0 2024-01-24T23:21:12.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f082b7fe640 1 -- 172.21.15.115:0/3410634023 >> v1:172.21.15.115:6792/0 conn(0x7f082c1343b0 legacy=0x7f082c13f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f082b7fe640 1 -- 172.21.15.115:0/3410634023 >> v1:172.21.15.132:6790/0 conn(0x7f082c133bf0 legacy=0x7f082c09bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f082b7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:21:12.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f082b7fe640 1 monclient: found mon.noname-c 2024-01-24T23:21:12.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f082b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:12.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f082b7fe640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:21:12.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f082b7fe640 1 -- 172.21.15.115:0/3410634023 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082c09b130 con 0x7f082c133fd0 2024-01-24T23:21:12.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f082b7fe640 10 monclient: _finish_auth 0 2024-01-24T23:21:12.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f082b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:21:12.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f082b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:12.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 5 monclient: authenticate success, global_id 74344 2024-01-24T23:21:12.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f082b7fe640 1 -- 172.21.15.115:0/3410634023 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0818001720 con 0x7f082c133fd0 2024-01-24T23:21:12.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:12.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f082b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:12.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f082b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:21:12.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f082b7fe640 10 monclient: dump: 2024-01-24T23:21:12.069 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:12.069 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.069 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.069 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.069 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:12.069 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.069 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:12.070 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:12.070 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:12.070 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:12.070 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:12.070 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:12.070 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:12.070 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:12.070 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:12.070 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f082b7fe640 1 -- 172.21.15.115:0/3410634023 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f08180019b0 con 0x7f082c133fd0 2024-01-24T23:21:12.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:12.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f082b7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:12.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:12.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 4 set_mon_vals no callback set 2024-01-24T23:21:12.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f082b7fe640 1 -- 172.21.15.115:0/3410634023 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f08180047f0 con 0x7f082c133fd0 2024-01-24T23:21:12.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f082b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:12.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f082b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:12.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f082b7fe640 10 monclient: dump: 2024-01-24T23:21:12.071 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:12.071 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.071 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.071 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.071 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:12.071 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.071 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:12.071 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:12.071 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:12.071 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:12.071 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:12.071 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:12.071 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:12.071 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:12.071 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:12.071 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 10 monclient: shutdown 2024-01-24T23:21:12.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:12.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 1 -- 172.21.15.115:0/3410634023 >> v1:172.21.15.115:6791/0 conn(0x7f082c133fd0 legacy=0x7f082c13bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 1 -- 172.21.15.115:0/3410634023 shutdown_connections 2024-01-24T23:21:12.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 1 -- 172.21.15.115:0/3410634023 >> 172.21.15.115:0/3410634023 conn(0x7f082c0662f0 msgr2=0x7f082c066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:12.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 1 -- 172.21.15.115:0/3410634023 shutdown_connections 2024-01-24T23:21:12.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 1 -- 172.21.15.115:0/3410634023 wait complete. 2024-01-24T23:21:12.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 10 monclient: build_initial_monmap 2024-01-24T23:21:12.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 10 monclient: monmap: 2024-01-24T23:21:12.076 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:12.077 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:12.077 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:12.077 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:12.077 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:12.077 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:12.077 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:12.077 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:12.077 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:12.077 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:12.077 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:12.077 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:12.077 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:12.077 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:12.077 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:12.078 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 1 Processor -- start 2024-01-24T23:21:12.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 1 -- start start 2024-01-24T23:21:12.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 10 monclient: init 2024-01-24T23:21:12.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:12.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 10 monclient: _add_conns ranks=[6,0,1,5,2,4,8,7,3] 2024-01-24T23:21:12.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 10 monclient(hunting): picked mon.noname-d con 0x7f082c133bf0 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:12.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 10 monclient(hunting): picked mon.noname-a con 0x7f082c133fd0 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:12.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 10 monclient(hunting): picked mon.noname-f con 0x7f082c1343b0 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:12.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082c1200c0 con 0x7f082c133fd0 2024-01-24T23:21:12.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f0830e5a640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f082c133bf0 0x7f082c1cb5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:56986/0 (socket says 172.21.15.115:56986) 2024-01-24T23:21:12.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f0830e5a640 1 -- 172.21.15.115:0/2705363095 learned_addr learned my addr 172.21.15.115:0/2705363095 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:12.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082c1bcd60 con 0x7f082c1343b0 2024-01-24T23:21:12.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 1 -- 172.21.15.115:0/2705363095 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082c0a5990 con 0x7f082c133bf0 2024-01-24T23:21:12.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:12.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08330be640 10 monclient(hunting): authenticate will time out at 950.991653s 2024-01-24T23:21:12.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08297fa640 1 -- 172.21.15.115:0/2705363095 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2157689283 0 0) 0x7f082c0a5990 con 0x7f082c133bf0 2024-01-24T23:21:12.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08297fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:12.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08297fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:12.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08297fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:12.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08297fa640 10 monclient(hunting): my global_id is 4362 2024-01-24T23:21:12.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08297fa640 1 -- 172.21.15.115:0/2705363095 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0810003530 con 0x7f082c133bf0 2024-01-24T23:21:12.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08297fa640 1 -- 172.21.15.115:0/2705363095 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1994943778 0 0) 0x7f082c1200c0 con 0x7f082c133fd0 2024-01-24T23:21:12.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08297fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:12.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08297fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:12.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08297fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:12.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08297fa640 10 monclient(hunting): my global_id is 4374 2024-01-24T23:21:12.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.063+0000 7f08297fa640 1 -- 172.21.15.115:0/2705363095 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f082c0a5990 con 0x7f082c133fd0 2024-01-24T23:21:12.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 1 -- 172.21.15.115:0/2705363095 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1263587161 0 0) 0x7f082c1bcd60 con 0x7f082c1343b0 2024-01-24T23:21:12.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:12.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 10 monclient(hunting): my global_id is 4312 2024-01-24T23:21:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 1 -- 172.21.15.115:0/2705363095 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f082c1200c0 con 0x7f082c1343b0 2024-01-24T23:21:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 1 -- 172.21.15.115:0/2705363095 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 327155034 0 0) 0x7f0810003530 con 0x7f082c133bf0 2024-01-24T23:21:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 1 -- 172.21.15.115:0/2705363095 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f082c1bcd60 con 0x7f082c133bf0 2024-01-24T23:21:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 1 -- 172.21.15.115:0/2705363095 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4095178578 0 0) 0x7f082c0a5990 con 0x7f082c133fd0 2024-01-24T23:21:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 1 -- 172.21.15.115:0/2705363095 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0810003530 con 0x7f082c133fd0 2024-01-24T23:21:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 1 -- 172.21.15.115:0/2705363095 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0820001bd0 con 0x7f082c133bf0 2024-01-24T23:21:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:21:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 10 monclient(hunting): dump: 2024-01-24T23:21:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.080 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:12.081 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.081 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:12.081 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:12.081 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:12.081 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:12.081 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:12.081 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:12.081 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:12.081 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:12.081 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:12.081 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:12.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 1 -- 172.21.15.115:0/2705363095 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0818001620 con 0x7f082c133fd0 2024-01-24T23:21:12.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:12.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:12.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 10 monclient(hunting): dump: 2024-01-24T23:21:12.082 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:12.082 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.082 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.082 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.082 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:12.082 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.082 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:12.082 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:12.082 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:12.082 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:12.082 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:12.082 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:12.082 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:12.082 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:12.083 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:12.083 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 1 -- 172.21.15.115:0/2705363095 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3099834386 0 0) 0x7f082c1bcd60 con 0x7f082c133bf0 2024-01-24T23:21:12.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 1 -- 172.21.15.115:0/2705363095 >> v1:172.21.15.132:6789/0 conn(0x7f082c1343b0 legacy=0x7f082c1d3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 1 -- 172.21.15.115:0/2705363095 >> v1:172.21.15.115:6789/0 conn(0x7f082c133fd0 legacy=0x7f082c132fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 10 monclient: _finish_hunting 0 2024-01-24T23:21:12.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 1 monclient: found mon.d 2024-01-24T23:21:12.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:12.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:12.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 1 -- 172.21.15.115:0/2705363095 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082c1bd510 con 0x7f082c133bf0 2024-01-24T23:21:12.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 10 monclient: _finish_auth 0 2024-01-24T23:21:12.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 10 monclient: _check_auth_tickets 2024-01-24T23:21:12.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:12.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08330be640 5 monclient: authenticate success, global_id 4362 2024-01-24T23:21:12.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08330be640 10 monclient: _renew_subs 2024-01-24T23:21:12.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08330be640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:12.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08330be640 1 -- 172.21.15.115:0/2705363095 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f082c1203a0 con 0x7f082c133bf0 2024-01-24T23:21:12.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 1 -- 172.21.15.115:0/2705363095 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f08200038b0 con 0x7f082c133bf0 2024-01-24T23:21:12.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:12.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f082affd640 4 set_mon_vals no callback set 2024-01-24T23:21:12.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08330be640 10 monclient: _renew_subs 2024-01-24T23:21:12.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 1 -- 172.21.15.115:0/2705363095 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0820003d60 con 0x7f082c133bf0 2024-01-24T23:21:12.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08330be640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:12.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08330be640 1 -- 172.21.15.115:0/2705363095 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f082c1be070 con 0x7f082c133bf0 2024-01-24T23:21:12.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:12.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:12.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 10 monclient: dump: 2024-01-24T23:21:12.084 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:12.084 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.085 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.085 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.085 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:12.085 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.085 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:12.085 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:12.085 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:12.085 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:12.085 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:12.085 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:12.085 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:12.085 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:12.085 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:12.085 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 1 -- 172.21.15.115:0/2705363095 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f0820003f80 con 0x7f082c133bf0 2024-01-24T23:21:12.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08297fa640 1 -- 172.21.15.115:0/2705363095 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f0820043740 con 0x7f082c133bf0 2024-01-24T23:21:12.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08330be640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:12.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08330be640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:12.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08330be640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:12.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f08330be640 1 -- 172.21.15.115:0/2705363095 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f082c06a0d0 con 0x7f082c133bf0 2024-01-24T23:21:12.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.067+0000 7f082bfff640 10 monclient: get_auth_request con 0x7f0810037a70 auth_method 0 2024-01-24T23:21:12.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f08297fa640 1 -- 172.21.15.115:0/2705363095 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f0820004440 con 0x7f082c133bf0 2024-01-24T23:21:12.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f08297fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:12.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f08297fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:12.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f132de12640 10 monclient: get_monmap_and_config 2024-01-24T23:21:12.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f132de12640 10 monclient: build_initial_monmap 2024-01-24T23:21:12.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f132de12640 10 monclient: monmap: 2024-01-24T23:21:12.090 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:12.090 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.090 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:12.084720+0000 2024-01-24T23:21:12.090 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:12.084720+0000 2024-01-24T23:21:12.090 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:12.090 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.090 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:12.090 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:12.090 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:12.090 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:12.090 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:12.090 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:12.090 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:12.090 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:12.090 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:12.090 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.090 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f132de12640 1 Processor -- start 2024-01-24T23:21:12.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f132de12640 1 -- start start 2024-01-24T23:21:12.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f132de12640 10 monclient: init 2024-01-24T23:21:12.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f132de12640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:12.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f132de12640 10 monclient: _add_conns ranks=[6,5,2,7,1,4,8,3,0] 2024-01-24T23:21:12.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f132de12640 10 monclient(hunting): picked mon.noname-d con 0x7f1328133bf0 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:12.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f132de12640 10 monclient(hunting): picked mon.noname-h con 0x7f1328133fd0 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:12.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f132de12640 10 monclient(hunting): picked mon.noname-b con 0x7f13281343b0 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:12.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f132de12640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f132804ed70 con 0x7f13281343b0 2024-01-24T23:21:12.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f132de12640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13280a4b60 con 0x7f1328133fd0 2024-01-24T23:21:12.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f132de12640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f132809ae50 con 0x7f1328133bf0 2024-01-24T23:21:12.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f132de12640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:12.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f132de12640 10 monclient(hunting): authenticate will time out at 951.007083s 2024-01-24T23:21:12.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f13277fe640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f1328133bf0 0x7f132809bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:39598/0 (socket says 172.21.15.115:39598) 2024-01-24T23:21:12.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f13277fe640 1 -- 172.21.15.115:0/577634406 learned_addr learned my addr 172.21.15.115:0/577634406 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:12.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f13267fc640 1 -- 172.21.15.115:0/577634406 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1431822531 0 0) 0x7f132809ae50 con 0x7f1328133bf0 2024-01-24T23:21:12.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f13267fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:12.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f13267fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:12.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f13267fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:12.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f13267fc640 10 monclient(hunting): my global_id is 4371 2024-01-24T23:21:12.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.079+0000 7f13267fc640 1 -- 172.21.15.115:0/577634406 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1310003530 con 0x7f1328133bf0 2024-01-24T23:21:12.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f13267fc640 1 -- 172.21.15.115:0/577634406 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3494710790 0 0) 0x7f1310003530 con 0x7f1328133bf0 2024-01-24T23:21:12.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f13267fc640 1 -- 172.21.15.115:0/577634406 >> v1:172.21.15.132:6791/0 conn(0x7f1328133fd0 legacy=0x7f132813bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f13267fc640 1 -- 172.21.15.115:0/577634406 >> v1:172.21.15.115:6790/0 conn(0x7f13281343b0 legacy=0x7f132813f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f13267fc640 10 monclient: _finish_hunting 0 2024-01-24T23:21:12.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f13267fc640 1 monclient: found mon.noname-d 2024-01-24T23:21:12.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f13267fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:12.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f13267fc640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:21:12.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f13267fc640 1 -- 172.21.15.115:0/577634406 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f132809b130 con 0x7f1328133bf0 2024-01-24T23:21:12.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f13267fc640 10 monclient: _finish_auth 0 2024-01-24T23:21:12.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f13267fc640 10 monclient: _check_auth_tickets 2024-01-24T23:21:12.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f13267fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:12.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f13267fc640 1 -- 172.21.15.115:0/577634406 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f13180016f0 con 0x7f1328133bf0 2024-01-24T23:21:12.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f13267fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:12.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f13267fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:21:12.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f13267fc640 10 monclient: dump: 2024-01-24T23:21:12.093 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:12.093 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.093 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.093 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.093 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:12.093 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.093 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:12.093 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:12.093 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:12.093 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:12.093 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:12.094 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:12.094 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:12.094 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:12.094 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:12.094 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 5 monclient: authenticate success, global_id 4371 2024-01-24T23:21:12.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:12.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f13267fc640 1 -- 172.21.15.115:0/577634406 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1318001a10 con 0x7f1328133bf0 2024-01-24T23:21:12.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f13267fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:12.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f13267fc640 1 -- 172.21.15.115:0/577634406 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1318004bf0 con 0x7f1328133bf0 2024-01-24T23:21:12.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f13267fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:12.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f13267fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:12.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f13267fc640 10 monclient: dump: 2024-01-24T23:21:12.094 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:12.094 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.094 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.094 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.095 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:12.095 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.095 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:12.095 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:12.095 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:12.095 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:12.095 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:12.095 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:12.095 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:12.095 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:12.095 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:12.095 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:12.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 4 set_mon_vals no callback set 2024-01-24T23:21:12.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 10 monclient: shutdown 2024-01-24T23:21:12.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:12.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 1 -- 172.21.15.115:0/577634406 >> v1:172.21.15.115:6792/0 conn(0x7f1328133bf0 legacy=0x7f132809bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 1 -- 172.21.15.115:0/577634406 shutdown_connections 2024-01-24T23:21:12.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 1 -- 172.21.15.115:0/577634406 >> 172.21.15.115:0/577634406 conn(0x7f13280662f0 msgr2=0x7f1328066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:12.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 1 -- 172.21.15.115:0/577634406 shutdown_connections 2024-01-24T23:21:12.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 1 -- 172.21.15.115:0/577634406 wait complete. 2024-01-24T23:21:12.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 10 monclient: build_initial_monmap 2024-01-24T23:21:12.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 10 monclient: monmap: 2024-01-24T23:21:12.096 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:12.096 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:12.096 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:12.096 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:12.096 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:12.096 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:12.096 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:12.096 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:12.096 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:12.097 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:12.097 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:12.097 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:12.097 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:12.097 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:12.097 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:12.097 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 1 Processor -- start 2024-01-24T23:21:12.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 1 -- start start 2024-01-24T23:21:12.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 10 monclient: init 2024-01-24T23:21:12.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:12.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 10 monclient: _add_conns ranks=[4,2,5,1,8,6,3,7,0] 2024-01-24T23:21:12.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 10 monclient(hunting): picked mon.noname-c con 0x7f1328133bf0 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:12.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 10 monclient(hunting): picked mon.noname-b con 0x7f1328133fd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:12.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f1326ffd640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f1328133fd0 0x7f13280a1190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:55358/0 (socket says 172.21.15.115:55358) 2024-01-24T23:21:12.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 10 monclient(hunting): picked mon.noname-h con 0x7f13281343b0 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:12.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f1326ffd640 1 -- 172.21.15.115:0/3161324651 learned_addr learned my addr 172.21.15.115:0/3161324651 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:12.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 1 -- 172.21.15.115:0/3161324651 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1328133620 con 0x7f1328133fd0 2024-01-24T23:21:12.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 1 -- 172.21.15.115:0/3161324651 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13280a5990 con 0x7f1328133bf0 2024-01-24T23:21:12.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 1 -- 172.21.15.115:0/3161324651 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1328120af0 con 0x7f13281343b0 2024-01-24T23:21:12.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:12.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f132de12640 10 monclient(hunting): authenticate will time out at 951.011316s 2024-01-24T23:21:12.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f0c2c587640 10 monclient: get_monmap_and_config 2024-01-24T23:21:12.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f0c2c587640 10 monclient: build_initial_monmap 2024-01-24T23:21:12.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f0c2c587640 10 monclient: monmap: 2024-01-24T23:21:12.098 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:12.098 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.098 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:12.090329+0000 2024-01-24T23:21:12.099 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:12.090329+0000 2024-01-24T23:21:12.099 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:12.099 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.099 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:12.099 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:12.099 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:12.099 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:12.099 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:12.099 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:12.099 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:12.099 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:12.099 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:12.099 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f130ffff640 1 -- 172.21.15.115:0/3161324651 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 129104072 0 0) 0x7f1328133620 con 0x7f1328133fd0 2024-01-24T23:21:12.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f130ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:12.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f130ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:12.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f130ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:12.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f130ffff640 10 monclient(hunting): my global_id is 4304 2024-01-24T23:21:12.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f130ffff640 1 -- 172.21.15.115:0/3161324651 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1314003530 con 0x7f1328133fd0 2024-01-24T23:21:12.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f130ffff640 1 -- 172.21.15.115:0/3161324651 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889491179 0 0) 0x7f13280a5990 con 0x7f1328133bf0 2024-01-24T23:21:12.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f130ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:12.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f130ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:12.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f130ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:12.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.083+0000 7f130ffff640 10 monclient(hunting): my global_id is 74353 2024-01-24T23:21:12.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 1 -- 172.21.15.115:0/3161324651 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1328133620 con 0x7f1328133bf0 2024-01-24T23:21:12.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 1 -- 172.21.15.115:0/3161324651 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3816858037 0 0) 0x7f1328120af0 con 0x7f13281343b0 2024-01-24T23:21:12.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:12.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:12.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:12.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 10 monclient(hunting): my global_id is 4343 2024-01-24T23:21:12.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 1 -- 172.21.15.115:0/3161324651 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13280a5990 con 0x7f13281343b0 2024-01-24T23:21:12.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 1 -- 172.21.15.115:0/3161324651 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 629659079 0 0) 0x7f1314003530 con 0x7f1328133fd0 2024-01-24T23:21:12.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 1 -- 172.21.15.115:0/3161324651 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1328120af0 con 0x7f1328133fd0 2024-01-24T23:21:12.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 1 -- 172.21.15.115:0/3161324651 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f131c001aa0 con 0x7f1328133fd0 2024-01-24T23:21:12.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:12.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:21:12.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 10 monclient(hunting): dump: 2024-01-24T23:21:12.104 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:12.104 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.104 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.104 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.104 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:12.104 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.104 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:12.104 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:12.105 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:12.105 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:12.105 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:12.105 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:12.105 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:12.105 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:12.105 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:12.105 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:12.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 1 -- 172.21.15.115:0/3161324651 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1662195750 0 0) 0x7f1328133620 con 0x7f1328133bf0 2024-01-24T23:21:12.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 1 -- 172.21.15.115:0/3161324651 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1314003530 con 0x7f1328133bf0 2024-01-24T23:21:12.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 1 -- 172.21.15.115:0/3161324651 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f13180016b0 con 0x7f1328133bf0 2024-01-24T23:21:12.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:12.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:12.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 10 monclient(hunting): dump: 2024-01-24T23:21:12.105 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:12.105 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.106 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.106 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.106 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:12.106 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.106 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:12.106 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:12.106 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:12.106 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:12.106 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:12.106 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:12.106 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:12.106 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:12.106 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:12.106 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 1 -- 172.21.15.115:0/3161324651 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2315549274 0 0) 0x7f1314003530 con 0x7f1328133bf0 2024-01-24T23:21:12.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 1 -- 172.21.15.115:0/3161324651 >> v1:172.21.15.132:6791/0 conn(0x7f13281343b0 legacy=0x7f13280a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 1 -- 172.21.15.115:0/3161324651 >> v1:172.21.15.115:6790/0 conn(0x7f1328133fd0 legacy=0x7f13280a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 10 monclient: _finish_hunting 0 2024-01-24T23:21:12.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 1 monclient: found mon.c 2024-01-24T23:21:12.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:12.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:12.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 1 -- 172.21.15.115:0/3161324651 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f132809c810 con 0x7f1328133bf0 2024-01-24T23:21:12.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 10 monclient: _finish_auth 0 2024-01-24T23:21:12.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 10 monclient: _check_auth_tickets 2024-01-24T23:21:12.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:12.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f132de12640 5 monclient: authenticate success, global_id 74353 2024-01-24T23:21:12.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f132de12640 10 monclient: _renew_subs 2024-01-24T23:21:12.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f132de12640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:12.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f132de12640 1 -- 172.21.15.115:0/3161324651 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13280a26d0 con 0x7f1328133bf0 2024-01-24T23:21:12.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f132de12640 10 monclient: _renew_subs 2024-01-24T23:21:12.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f132de12640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:12.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f132de12640 1 -- 172.21.15.115:0/3161324651 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1328120250 con 0x7f1328133bf0 2024-01-24T23:21:12.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 1 -- 172.21.15.115:0/3161324651 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1318001050 con 0x7f1328133bf0 2024-01-24T23:21:12.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:12.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f1325ffb640 4 set_mon_vals no callback set 2024-01-24T23:21:12.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f132de12640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:12.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 1 -- 172.21.15.115:0/3161324651 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1318004eb0 con 0x7f1328133bf0 2024-01-24T23:21:12.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f132de12640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:12.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f132de12640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:12.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f132de12640 1 -- 172.21.15.115:0/3161324651 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1308005090 con 0x7f1328133bf0 2024-01-24T23:21:12.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:12.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:12.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 10 monclient: dump: 2024-01-24T23:21:12.108 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:12.108 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.108 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.108 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.109 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:12.109 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.109 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:12.109 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:12.109 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:12.109 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:12.109 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:12.109 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:12.109 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:12.109 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:12.109 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:12.109 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 1 -- 172.21.15.115:0/3161324651 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f131800f930 con 0x7f1328133bf0 2024-01-24T23:21:12.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f130ffff640 1 -- 172.21.15.115:0/3161324651 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f1318043490 con 0x7f1328133bf0 2024-01-24T23:21:12.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.087+0000 7f1326ffd640 10 monclient: get_auth_request con 0x7f13140378f0 auth_method 0 2024-01-24T23:21:12.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c2c587640 1 Processor -- start 2024-01-24T23:21:12.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f470a488640 10 monclient: get_monmap_and_config 2024-01-24T23:21:12.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f470a488640 10 monclient: build_initial_monmap 2024-01-24T23:21:12.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f470a488640 10 monclient: monmap: 2024-01-24T23:21:12.110 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:12.111 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.111 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:12.095132+0000 2024-01-24T23:21:12.111 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:12.095132+0000 2024-01-24T23:21:12.111 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:12.111 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.111 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:12.111 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:12.111 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:12.111 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:12.111 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:12.111 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:12.111 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:12.111 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:12.111 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:12.111 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f470a488640 1 Processor -- start 2024-01-24T23:21:12.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f470a488640 1 -- start start 2024-01-24T23:21:12.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f470a488640 10 monclient: init 2024-01-24T23:21:12.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c2c587640 1 -- start start 2024-01-24T23:21:12.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c2c587640 10 monclient: init 2024-01-24T23:21:12.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c2c587640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:12.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c2c587640 10 monclient: _add_conns ranks=[8,4,2,0,7,3,1,5,6] 2024-01-24T23:21:12.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c2c587640 10 monclient(hunting): picked mon.noname-e con 0x7f0c240a18a0 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:12.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c2c587640 10 monclient(hunting): picked mon.noname-c con 0x7f0c240a1c80 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:12.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c2c587640 10 monclient(hunting): picked mon.noname-b con 0x7f0c24139b20 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:12.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c2c587640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2404ed70 con 0x7f0c24139b20 2024-01-24T23:21:12.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c2c587640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2409cb70 con 0x7f0c240a1c80 2024-01-24T23:21:12.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c2c587640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2409c800 con 0x7f0c240a18a0 2024-01-24T23:21:12.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c2c587640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:12.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c2c587640 10 monclient(hunting): authenticate will time out at 951.017063s 2024-01-24T23:21:12.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c2ab24640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f0c24139b20 0x7f0c2413bf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:55368/0 (socket says 172.21.15.115:55368) 2024-01-24T23:21:12.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c2ab24640 1 -- 172.21.15.115:0/3380254137 learned_addr learned my addr 172.21.15.115:0/3380254137 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:12.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 1 -- 172.21.15.115:0/3380254137 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1303159529 0 0) 0x7f0c2404ed70 con 0x7f0c24139b20 2024-01-24T23:21:12.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 10 monclient(hunting): _negotiate 2024-01-24T23:21:12.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:12.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:12.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 10 monclient(hunting): my global_id is 4313 2024-01-24T23:21:12.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 1 -- 172.21.15.115:0/3380254137 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c10003530 con 0x7f0c24139b20 2024-01-24T23:21:12.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 1 -- 172.21.15.115:0/3380254137 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2297835763 0 0) 0x7f0c10003530 con 0x7f0c24139b20 2024-01-24T23:21:12.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 1 -- 172.21.15.115:0/3380254137 >> v1:172.21.15.115:6793/0 conn(0x7f0c240a18a0 legacy=0x7f0c2409db30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:21:12.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 1 -- 172.21.15.115:0/3380254137 >> v1:172.21.15.115:6791/0 conn(0x7f0c240a1c80 legacy=0x7f0c240a2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T23:21:12.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 10 monclient: _finish_hunting 0 2024-01-24T23:21:12.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 1 monclient: found mon.noname-b 2024-01-24T23:21:12.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:12.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:21:12.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 1 -- 172.21.15.115:0/3380254137 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2409e240 con 0x7f0c24139b20 2024-01-24T23:21:12.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 10 monclient: _finish_auth 0 2024-01-24T23:21:12.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 10 monclient: _check_auth_tickets 2024-01-24T23:21:12.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:12.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 1 -- 172.21.15.115:0/3380254137 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0c1c001b40 con 0x7f0c24139b20 2024-01-24T23:21:12.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:12.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:21:12.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 10 monclient: dump: 2024-01-24T23:21:12.115 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:12.115 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.115 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.115 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.115 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:12.115 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.115 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:12.115 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:12.115 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:12.116 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:12.116 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:12.116 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:12.116 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:12.116 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:12.116 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:12.116 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c2c587640 5 monclient: authenticate success, global_id 4313 2024-01-24T23:21:12.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c2c587640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:12.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 1 -- 172.21.15.115:0/3380254137 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0c1c001d60 con 0x7f0c24139b20 2024-01-24T23:21:12.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:12.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 1 -- 172.21.15.115:0/3380254137 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0c1c004f70 con 0x7f0c24139b20 2024-01-24T23:21:12.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:12.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:12.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c29321640 10 monclient: dump: 2024-01-24T23:21:12.116 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:12.117 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.117 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.117 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.117 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:12.117 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.117 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:12.117 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:12.117 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:12.117 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:12.117 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:12.117 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:12.117 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:12.117 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:12.117 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:12.117 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c2c587640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:12.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c2c587640 4 set_mon_vals no callback set 2024-01-24T23:21:12.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f470a488640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:12.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f470a488640 10 monclient: _add_conns ranks=[0,1,3,8,7,4,2,6,5] 2024-01-24T23:21:12.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f470a488640 10 monclient(hunting): picked mon.noname-a con 0x7f4704133bf0 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:12.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f470a488640 10 monclient(hunting): picked mon.noname-f con 0x7f4704133fd0 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:12.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f470a488640 10 monclient(hunting): picked mon.noname-g con 0x7f47041343b0 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:12.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4703fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f4704133bf0 0x7f470409bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58092/0 (socket says 172.21.15.115:58092) 2024-01-24T23:21:12.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4703fff640 1 -- 172.21.15.115:0/1366253359 learned_addr learned my addr 172.21.15.115:0/1366253359 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:12.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f470a488640 1 -- 172.21.15.115:0/1366253359 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f470404ed70 con 0x7f4704133bf0 2024-01-24T23:21:12.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f470a488640 1 -- 172.21.15.115:0/1366253359 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47040a4b60 con 0x7f4704133fd0 2024-01-24T23:21:12.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f470a488640 1 -- 172.21.15.115:0/1366253359 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f470409ae50 con 0x7f47041343b0 2024-01-24T23:21:12.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f470a488640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:12.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f470a488640 10 monclient(hunting): authenticate will time out at 951.018901s 2024-01-24T23:21:12.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c2c587640 10 monclient: shutdown 2024-01-24T23:21:12.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c2c587640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:12.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c2c587640 1 -- 172.21.15.115:0/3380254137 >> v1:172.21.15.115:6790/0 conn(0x7f0c24139b20 legacy=0x7f0c2413bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 1 -- 172.21.15.115:0/1366253359 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3806000479 0 0) 0x7f470404ed70 con 0x7f4704133bf0 2024-01-24T23:21:12.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:12.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:12.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:12.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 10 monclient(hunting): my global_id is 4383 2024-01-24T23:21:12.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 1 -- 172.21.15.115:0/1366253359 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46ec003530 con 0x7f4704133bf0 2024-01-24T23:21:12.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c2c587640 1 -- 172.21.15.115:0/3380254137 shutdown_connections 2024-01-24T23:21:12.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f0c2c587640 1 -- 172.21.15.115:0/3380254137 >> 172.21.15.115:0/3380254137 conn(0x7f0c240a6490 msgr2=0x7f0c240a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:12.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 1 -- 172.21.15.115:0/1366253359 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2578497806 0 0) 0x7f470409ae50 con 0x7f47041343b0 2024-01-24T23:21:12.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:12.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:12.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:12.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 10 monclient(hunting): my global_id is 4341 2024-01-24T23:21:12.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 1 -- 172.21.15.115:0/1366253359 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f470404ed70 con 0x7f47041343b0 2024-01-24T23:21:12.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 1 -- 172.21.15.115:0/1366253359 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1957346748 0 0) 0x7f47040a4b60 con 0x7f4704133fd0 2024-01-24T23:21:12.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:12.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:12.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:12.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 10 monclient(hunting): my global_id is 4321 2024-01-24T23:21:12.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 1 -- 172.21.15.115:0/1366253359 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f470409ae50 con 0x7f4704133fd0 2024-01-24T23:21:12.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 1 -- 172.21.15.115:0/1366253359 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2248725518 0 0) 0x7f46ec003530 con 0x7f4704133bf0 2024-01-24T23:21:12.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 1 -- 172.21.15.115:0/1366253359 >> v1:172.21.15.132:6790/0 conn(0x7f47041343b0 legacy=0x7f470413f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 1 -- 172.21.15.115:0/1366253359 >> v1:172.21.15.132:6789/0 conn(0x7f4704133fd0 legacy=0x7f470413bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:21:12.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 1 monclient: found mon.noname-a 2024-01-24T23:21:12.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:12.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:21:12.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 1 -- 172.21.15.115:0/1366253359 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470409b130 con 0x7f4704133bf0 2024-01-24T23:21:12.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 10 monclient: _finish_auth 0 2024-01-24T23:21:12.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:21:12.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:12.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 1 -- 172.21.15.115:0/1366253359 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f46f40017c0 con 0x7f4704133bf0 2024-01-24T23:21:12.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:12.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:21:12.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 10 monclient: dump: 2024-01-24T23:21:12.125 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:12.125 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.126 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.126 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.126 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:12.126 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.126 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:12.126 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:12.126 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:12.126 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:12.126 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:12.126 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:12.126 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:12.126 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:12.126 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:12.126 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f470a488640 5 monclient: authenticate success, global_id 4383 2024-01-24T23:21:12.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f470a488640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:12.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 1 -- 172.21.15.115:0/1366253359 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f46f40019e0 con 0x7f4704133bf0 2024-01-24T23:21:12.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:12.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 1 -- 172.21.15.115:0/1366253359 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f46f4004bf0 con 0x7f4704133bf0 2024-01-24T23:21:12.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:12.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:12.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f4702ffd640 10 monclient: dump: 2024-01-24T23:21:12.127 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:12.127 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.127 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.127 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.127 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:12.127 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.127 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:12.127 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:12.127 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:12.128 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:12.128 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:12.128 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:12.128 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:12.128 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:12.128 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:12.128 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f470a488640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:12.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.091+0000 7f470a488640 4 set_mon_vals no callback set 2024-01-24T23:21:12.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.095+0000 7f470a488640 10 monclient: shutdown 2024-01-24T23:21:12.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.095+0000 7f470a488640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:12.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.095+0000 7f470a488640 1 -- 172.21.15.115:0/1366253359 >> v1:172.21.15.115:6789/0 conn(0x7f4704133bf0 legacy=0x7f470409bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.095+0000 7f470a488640 1 -- 172.21.15.115:0/1366253359 shutdown_connections 2024-01-24T23:21:12.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.095+0000 7f470a488640 1 -- 172.21.15.115:0/1366253359 >> 172.21.15.115:0/1366253359 conn(0x7f47040662f0 msgr2=0x7f4704066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:12.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.095+0000 7f470a488640 1 -- 172.21.15.115:0/1366253359 shutdown_connections 2024-01-24T23:21:12.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.095+0000 7f470a488640 1 -- 172.21.15.115:0/1366253359 wait complete. 2024-01-24T23:21:12.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.095+0000 7f470a488640 10 monclient: build_initial_monmap 2024-01-24T23:21:12.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.095+0000 7f470a488640 10 monclient: monmap: 2024-01-24T23:21:12.129 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:12.129 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:12.129 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:12.129 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:12.129 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:12.129 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:12.129 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:12.129 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:12.129 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:12.129 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:12.129 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:12.130 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:12.130 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:12.130 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:12.130 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:12.130 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.095+0000 7f470a488640 1 Processor -- start 2024-01-24T23:21:12.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.095+0000 7f470a488640 1 -- start start 2024-01-24T23:21:12.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.095+0000 7f470a488640 10 monclient: init 2024-01-24T23:21:12.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.095+0000 7f470a488640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:12.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.095+0000 7f470a488640 10 monclient: _add_conns ranks=[2,8,0,1,6,5,3,7,4] 2024-01-24T23:21:12.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.095+0000 7f470a488640 10 monclient(hunting): picked mon.noname-b con 0x7f4704133bf0 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:12.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.095+0000 7f470a488640 10 monclient(hunting): picked mon.noname-e con 0x7f4704133fd0 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:12.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.095+0000 7f4703fff640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f4704133bf0 0x7f47040a0b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:55384/0 (socket says 172.21.15.115:55384) 2024-01-24T23:21:12.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.095+0000 7f0c2c587640 1 -- 172.21.15.115:0/3380254137 shutdown_connections 2024-01-24T23:21:12.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.095+0000 7f47037fe640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f4704133fd0 0x7f47040a12a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:48396/0 (socket says 172.21.15.115:48396) 2024-01-24T23:21:12.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.095+0000 7f4708a25640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f47041343b0 0x7f47040a19b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58102/0 (socket says 172.21.15.115:58102) 2024-01-24T23:21:12.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.095+0000 7f0c2c587640 1 -- 172.21.15.115:0/3380254137 wait complete. 2024-01-24T23:21:12.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.095+0000 7f470a488640 10 monclient(hunting): picked mon.noname-a con 0x7f47041343b0 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:12.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.095+0000 7f4703fff640 1 -- 172.21.15.115:0/2133760088 learned_addr learned my addr 172.21.15.115:0/2133760088 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:12.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f470a488640 1 -- 172.21.15.115:0/2133760088 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4704133700 con 0x7f47041343b0 2024-01-24T23:21:12.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f470a488640 1 -- 172.21.15.115:0/2133760088 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4704120860 con 0x7f4704133bf0 2024-01-24T23:21:12.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3229085736 0 0) 0x7f4704133700 con 0x7f47041343b0 2024-01-24T23:21:12.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f470a488640 1 -- 172.21.15.115:0/2133760088 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47040a5990 con 0x7f4704133fd0 2024-01-24T23:21:12.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f470a488640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:12.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f470a488640 10 monclient(hunting): authenticate will time out at 951.024429s 2024-01-24T23:21:12.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:12.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:12.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:12.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient(hunting): my global_id is 4392 2024-01-24T23:21:12.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46f0003560 con 0x7f47041343b0 2024-01-24T23:21:12.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881846573 0 0) 0x7f4704120860 con 0x7f4704133bf0 2024-01-24T23:21:12.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:12.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:12.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:12.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient(hunting): my global_id is 4322 2024-01-24T23:21:12.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4704133700 con 0x7f4704133bf0 2024-01-24T23:21:12.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 845693515 0 0) 0x7f47040a5990 con 0x7f4704133fd0 2024-01-24T23:21:12.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:12.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:12.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:12.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient(hunting): my global_id is 74267 2024-01-24T23:21:12.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4704120860 con 0x7f4704133fd0 2024-01-24T23:21:12.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 762000593 0 0) 0x7f46f0003560 con 0x7f47041343b0 2024-01-24T23:21:12.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f47040a5990 con 0x7f47041343b0 2024-01-24T23:21:12.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c2c587640 10 monclient: build_initial_monmap 2024-01-24T23:21:12.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c2c587640 10 monclient: monmap: 2024-01-24T23:21:12.135 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:12.135 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:12.135 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:12.135 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:12.135 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:12.135 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:12.135 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:12.135 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:12.135 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:12.135 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:12.135 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:12.135 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:12.136 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:12.136 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:12.136 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:12.136 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c2c587640 1 Processor -- start 2024-01-24T23:21:12.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1348478725 0 0) 0x7f4704133700 con 0x7f4704133bf0 2024-01-24T23:21:12.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f46f0003560 con 0x7f4704133bf0 2024-01-24T23:21:12.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f46fc001c70 con 0x7f47041343b0 2024-01-24T23:21:12.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:12.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:21:12.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient(hunting): dump: 2024-01-24T23:21:12.137 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:12.137 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.137 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.137 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.137 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:12.137 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.137 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:12.137 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:12.138 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:12.138 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:12.138 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:12.138 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:12.138 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:12.138 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:12.138 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:12.138 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:12.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f46f40016c0 con 0x7f4704133bf0 2024-01-24T23:21:12.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:12.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:12.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient(hunting): dump: 2024-01-24T23:21:12.138 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:12.139 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.139 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.139 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.139 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:12.139 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.139 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:12.139 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:12.139 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:12.139 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:12.139 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:12.139 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:12.139 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:12.139 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:12.139 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:12.139 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2179864620 0 0) 0x7f4704120860 con 0x7f4704133fd0 2024-01-24T23:21:12.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4704133700 con 0x7f4704133fd0 2024-01-24T23:21:12.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f46f8001a50 con 0x7f4704133fd0 2024-01-24T23:21:12.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:12.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:21:12.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient(hunting): dump: 2024-01-24T23:21:12.140 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:12.140 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.140 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.140 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.140 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:12.140 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.140 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:12.140 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:12.141 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 361450607 0 0) 0x7f46f0003560 con 0x7f4704133bf0 2024-01-24T23:21:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 >> v1:172.21.15.115:6793/0 conn(0x7f4704133fd0 legacy=0x7f47040a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 >> v1:172.21.15.115:6789/0 conn(0x7f47041343b0 legacy=0x7f47040a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:21:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 1 monclient: found mon.b 2024-01-24T23:21:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:12.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470409c810 con 0x7f4704133bf0 2024-01-24T23:21:12.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient: _finish_auth 0 2024-01-24T23:21:12.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:21:12.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:12.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c2c587640 1 -- start start 2024-01-24T23:21:12.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c2c587640 10 monclient: init 2024-01-24T23:21:12.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c2c587640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:12.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c2c587640 10 monclient: _add_conns ranks=[7,6,5,8,0,1,4,2,3] 2024-01-24T23:21:12.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c2c587640 10 monclient(hunting): picked mon.noname-i con 0x7f0c240a1c80 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:12.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c2c587640 10 monclient(hunting): picked mon.noname-d con 0x7f0c24139b20 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:12.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c2c587640 10 monclient(hunting): picked mon.noname-h con 0x7f0c2413bd70 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c2c587640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2409b830 con 0x7f0c2413bd70 2024-01-24T23:21:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c2c587640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c24072110 con 0x7f0c24139b20 2024-01-24T23:21:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c2c587640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2413cd30 con 0x7f0c240a1c80 2024-01-24T23:21:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c2c587640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c2c587640 10 monclient(hunting): authenticate will time out at 951.026270s 2024-01-24T23:21:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c29b22640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f0c24139b20 0x7f0c2413b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:39602/0 (socket says 172.21.15.115:39602) 2024-01-24T23:21:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c29b22640 1 -- 172.21.15.115:0/2623613245 learned_addr learned my addr 172.21.15.115:0/2623613245 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 1 -- 172.21.15.115:0/2623613245 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1762727629 0 0) 0x7f0c24072110 con 0x7f0c24139b20 2024-01-24T23:21:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 10 monclient(hunting): my global_id is 4380 2024-01-24T23:21:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 1 -- 172.21.15.115:0/2623613245 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c08003530 con 0x7f0c24139b20 2024-01-24T23:21:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 1 -- 172.21.15.115:0/2623613245 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3493116766 0 0) 0x7f0c08003530 con 0x7f0c24139b20 2024-01-24T23:21:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 1 -- 172.21.15.115:0/2623613245 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0c24072110 con 0x7f0c24139b20 2024-01-24T23:21:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 1 -- 172.21.15.115:0/2623613245 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2020175985 0 0) 0x7f0c2413cd30 con 0x7f0c240a1c80 2024-01-24T23:21:12.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:12.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f470a488640 5 monclient: authenticate success, global_id 4322 2024-01-24T23:21:12.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f470a488640 10 monclient: _renew_subs 2024-01-24T23:21:12.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f470a488640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:12.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f470a488640 1 -- 172.21.15.115:0/2133760088 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4704120580 con 0x7f4704133bf0 2024-01-24T23:21:12.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f470a488640 10 monclient: _renew_subs 2024-01-24T23:21:12.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f470a488640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:12.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f470a488640 1 -- 172.21.15.115:0/2133760088 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f47040a3070 con 0x7f4704133bf0 2024-01-24T23:21:12.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f46f40011c0 con 0x7f4704133bf0 2024-01-24T23:21:12.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:12.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f46f4004bc0 con 0x7f4704133bf0 2024-01-24T23:21:12.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:12.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f47027fc640 4 set_mon_vals no callback set 2024-01-24T23:21:12.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:12.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 10 monclient: dump: 2024-01-24T23:21:12.145 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:12.145 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.145 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.145 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.145 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:12.145 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.145 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:12.145 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:12.145 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:12.145 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:12.146 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:12.146 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:12.146 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:12.146 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:12.146 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:12.146 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f470a488640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:12.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f470a488640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:12.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f470a488640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:12.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f470a488640 1 -- 172.21.15.115:0/2133760088 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e4005090 con 0x7f4704133bf0 2024-01-24T23:21:12.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f46f400fb10 con 0x7f4704133bf0 2024-01-24T23:21:12.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f46f4043610 con 0x7f4704133bf0 2024-01-24T23:21:12.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:12.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:12.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 10 monclient(hunting): my global_id is 4363 2024-01-24T23:21:12.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 1 -- 172.21.15.115:0/2623613245 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c08003530 con 0x7f0c240a1c80 2024-01-24T23:21:12.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 1 -- 172.21.15.115:0/2623613245 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0c14001a90 con 0x7f0c24139b20 2024-01-24T23:21:12.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:12.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:21:12.147 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 10 monclient(hunting): dump: 2024-01-24T23:21:12.147 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:12.147 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.147 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.148 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.148 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:12.148 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.148 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:12.148 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:12.148 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:12.148 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:12.148 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:12.148 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:12.148 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:12.148 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:12.148 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:12.148 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:12.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 1 -- 172.21.15.115:0/2623613245 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2430175364 0 0) 0x7f0c2409b830 con 0x7f0c2413bd70 2024-01-24T23:21:12.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:12.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:12.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:12.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 10 monclient(hunting): my global_id is 4352 2024-01-24T23:21:12.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 1 -- 172.21.15.115:0/2623613245 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c2413cd30 con 0x7f0c2413bd70 2024-01-24T23:21:12.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f47037fe640 10 monclient: get_auth_request con 0x7f46f0037c60 auth_method 0 2024-01-24T23:21:12.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f130ffff640 1 -- 172.21.15.115:0/3161324651 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f1318014710 con 0x7f1328133bf0 2024-01-24T23:21:12.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f130ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:12.149 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f130ffff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:12.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 1 -- 172.21.15.115:0/2623613245 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4272772652 0 0) 0x7f0c24072110 con 0x7f0c24139b20 2024-01-24T23:21:12.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 1 -- 172.21.15.115:0/2623613245 >> v1:172.21.15.132:6792/0 conn(0x7f0c240a1c80 legacy=0x7f0c2413af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 1 -- 172.21.15.115:0/2623613245 >> v1:172.21.15.132:6791/0 conn(0x7f0c2413bd70 legacy=0x7f0c241e07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 10 monclient: _finish_hunting 0 2024-01-24T23:21:12.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 1 monclient: found mon.d 2024-01-24T23:21:12.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:12.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:12.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 1 -- 172.21.15.115:0/2623613245 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c24141090 con 0x7f0c24139b20 2024-01-24T23:21:12.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 10 monclient: _finish_auth 0 2024-01-24T23:21:12.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 10 monclient: _check_auth_tickets 2024-01-24T23:21:12.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c1affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:12.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c2c587640 5 monclient: authenticate success, global_id 4380 2024-01-24T23:21:12.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c2c587640 10 monclient: _renew_subs 2024-01-24T23:21:12.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c2c587640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:12.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.099+0000 7f0c2c587640 1 -- 172.21.15.115:0/2623613245 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2413c690 con 0x7f0c24139b20 2024-01-24T23:21:12.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.103+0000 7f0c2c587640 10 monclient: _renew_subs 2024-01-24T23:21:12.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.103+0000 7f0c2c587640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:12.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.103+0000 7f0c2c587640 1 -- 172.21.15.115:0/2623613245 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c2413c220 con 0x7f0c24139b20 2024-01-24T23:21:12.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.103+0000 7f0c1affd640 1 -- 172.21.15.115:0/2623613245 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0c14003930 con 0x7f0c24139b20 2024-01-24T23:21:12.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.103+0000 7f0c1affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:12.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.103+0000 7f0c1affd640 1 -- 172.21.15.115:0/2623613245 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0c14003de0 con 0x7f0c24139b20 2024-01-24T23:21:12.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.103+0000 7f0c1affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:12.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.103+0000 7f0c28b20640 4 set_mon_vals no callback set 2024-01-24T23:21:12.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.103+0000 7f0c1affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:12.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.103+0000 7f0c1affd640 10 monclient: dump: 2024-01-24T23:21:12.152 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:12.152 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:12.152 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.152 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:12.152 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:12.153 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:12.153 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:12.153 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:12.153 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:12.153 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:12.153 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:12.153 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:12.153 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:12.153 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:12.153 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:12.153 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:12.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.103+0000 7f0c2c587640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:12.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.103+0000 7f0c2c587640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:12.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.103+0000 7f0c2c587640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:12.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.103+0000 7f0c2c587640 1 -- 172.21.15.115:0/2623613245 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c0c005090 con 0x7f0c24139b20 2024-01-24T23:21:12.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.103+0000 7f0c1affd640 1 -- 172.21.15.115:0/2623613245 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f0c14003f40 con 0x7f0c24139b20 2024-01-24T23:21:12.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.103+0000 7f0c1affd640 1 -- 172.21.15.115:0/2623613245 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f0c14043740 con 0x7f0c24139b20 2024-01-24T23:21:12.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.107+0000 7f0c2a323640 10 monclient: get_auth_request con 0x7f0c08037160 auth_method 0 2024-01-24T23:21:12.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.107+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 <== mon.2 v1:172.21.15.115:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f46f4011470 con 0x7f4704133bf0 2024-01-24T23:21:12.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.107+0000 7f4700ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:12.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.107+0000 7f4700ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:12.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.111+0000 7f0c1affd640 1 -- 172.21.15.115:0/2623613245 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f0c140042c0 con 0x7f0c24139b20 2024-01-24T23:21:12.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.111+0000 7f0c1affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:12.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.111+0000 7f0c1affd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:12.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.171+0000 7f7a9dba9640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2024-01-24T23:21:12.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.171+0000 7f7a9dba9640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-01-24T23:21:12.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.171+0000 7f7a9dba9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:12.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.171+0000 7f7a9dba9640 1 -- 172.21.15.115:0/3467174574 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7a78005340 con 0x7f7a98134b00 2024-01-24T23:21:12.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.171+0000 7f7a7ffff640 1 -- 172.21.15.115:0/3467174574 <== mon.2 v1:172.21.15.115:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (unknown 574334944 0 847495621) 0x7f7a8c0131f0 con 0x7f7a98134b00 2024-01-24T23:21:12.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.171+0000 7f7a7ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-01-24T23:21:12.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.171+0000 7f7a7ffff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:12.174 INFO:teuthology.orchestra.run.smithi115.stdout:47244640259 2024-01-24T23:21:12.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.171+0000 7f7a9dba9640 1 -- 172.21.15.115:0/3467174574 >> v1:172.21.15.132:6812/1835401922 conn(0x7f7a84037920 legacy=0x7f7a84039de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.171+0000 7f7a9dba9640 10 monclient: shutdown 2024-01-24T23:21:12.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.171+0000 7f7a9dba9640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:12.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.171+0000 7f7a9dba9640 1 -- 172.21.15.115:0/3467174574 >> v1:172.21.15.115:6790/0 conn(0x7f7a98134b00 legacy=0x7f7a981ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.171+0000 7f7a9dba9640 1 -- 172.21.15.115:0/3467174574 shutdown_connections 2024-01-24T23:21:12.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.171+0000 7f7a9dba9640 1 -- 172.21.15.115:0/3467174574 >> 172.21.15.115:0/3467174574 conn(0x7f7a980a63e0 msgr2=0x7f7a9809eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:12.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.171+0000 7f7a9dba9640 1 -- 172.21.15.115:0/3467174574 shutdown_connections 2024-01-24T23:21:12.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.171+0000 7f7a9dba9640 1 -- 172.21.15.115:0/3467174574 wait complete. 2024-01-24T23:21:12.191 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640259 for osd.0 2024-01-24T23:21:12.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.243+0000 7f08330be640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2024-01-24T23:21:12.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.243+0000 7f08330be640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-01-24T23:21:12.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.243+0000 7f08330be640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:12.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.243+0000 7f08330be640 1 -- 172.21.15.115:0/2705363095 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f082c0a1c20 con 0x7f082c133bf0 2024-01-24T23:21:12.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.243+0000 7f08297fa640 1 -- 172.21.15.115:0/2705363095 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (unknown 92182286 0 3327677263) 0x7f08200146f0 con 0x7f082c133bf0 2024-01-24T23:21:12.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.243+0000 7f08297fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-01-24T23:21:12.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.243+0000 7f08297fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:12.248 INFO:teuthology.orchestra.run.smithi115.stdout:47244640257 2024-01-24T23:21:12.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.247+0000 7f08330be640 1 -- 172.21.15.115:0/2705363095 >> v1:172.21.15.132:6812/1835401922 conn(0x7f0810037a70 legacy=0x7f0810039f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.247+0000 7f08330be640 10 monclient: shutdown 2024-01-24T23:21:12.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.247+0000 7f08330be640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:12.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.247+0000 7f08330be640 1 -- 172.21.15.115:0/2705363095 >> v1:172.21.15.115:6792/0 conn(0x7f082c133bf0 legacy=0x7f082c1cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.247+0000 7f08330be640 1 -- 172.21.15.115:0/2705363095 shutdown_connections 2024-01-24T23:21:12.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.247+0000 7f08330be640 1 -- 172.21.15.115:0/2705363095 >> 172.21.15.115:0/2705363095 conn(0x7f082c0662f0 msgr2=0x7f082c09fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:12.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.247+0000 7f08330be640 1 -- 172.21.15.115:0/2705363095 shutdown_connections 2024-01-24T23:21:12.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.247+0000 7f08330be640 1 -- 172.21.15.115:0/2705363095 wait complete. 2024-01-24T23:21:12.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.255+0000 7fbed27fc640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2024-01-24T23:21:12.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.255+0000 7fbed27fc640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-01-24T23:21:12.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.255+0000 7fbed27fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:12.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.255+0000 7fbed27fc640 1 -- 172.21.15.115:0/1925314761 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbeec06a0d0 con 0x7fbeec133bf0 2024-01-24T23:21:12.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.255+0000 7fbee8ff9640 1 -- 172.21.15.115:0/1925314761 <== mon.2 v1:172.21.15.115:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (unknown 832126871 0 555833778) 0x7fbedc014af0 con 0x7fbeec133bf0 2024-01-24T23:21:12.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.255+0000 7fbee8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-01-24T23:21:12.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.255+0000 7fbee8ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:12.261 INFO:teuthology.orchestra.run.smithi115.stdout:47244640258 2024-01-24T23:21:12.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.259+0000 7fbed27fc640 1 -- 172.21.15.115:0/1925314761 >> v1:172.21.15.132:6812/1835401922 conn(0x7fbed8037230 legacy=0x7fbed80396f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.259+0000 7fbed27fc640 10 monclient: shutdown 2024-01-24T23:21:12.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.259+0000 7fbed27fc640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:12.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.259+0000 7fbed27fc640 1 -- 172.21.15.115:0/1925314761 >> v1:172.21.15.115:6790/0 conn(0x7fbeec133bf0 legacy=0x7fbeec132fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.259+0000 7fbed27fc640 1 -- 172.21.15.115:0/1925314761 shutdown_connections 2024-01-24T23:21:12.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.259+0000 7fbed27fc640 1 -- 172.21.15.115:0/1925314761 >> 172.21.15.115:0/1925314761 conn(0x7fbeec0662f0 msgr2=0x7fbeec1be890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:12.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.259+0000 7fbed27fc640 1 -- 172.21.15.115:0/1925314761 shutdown_connections 2024-01-24T23:21:12.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.259+0000 7fbed27fc640 1 -- 172.21.15.115:0/1925314761 wait complete. 2024-01-24T23:21:12.265 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640258 got 47244640257 for osd.2 2024-01-24T23:21:12.278 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640258 for osd.1 2024-01-24T23:21:12.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.335+0000 7f0c2c587640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2024-01-24T23:21:12.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.335+0000 7f0c2c587640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-01-24T23:21:12.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.335+0000 7f0c2c587640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:12.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.335+0000 7f0c2c587640 1 -- 172.21.15.115:0/2623613245 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f0c0c005340 con 0x7f0c24139b20 2024-01-24T23:21:12.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.335+0000 7f0c1affd640 1 -- 172.21.15.115:0/2623613245 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (unknown 2131975755 0 3327677263) 0x7f0c14012ec0 con 0x7f0c24139b20 2024-01-24T23:21:12.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.335+0000 7f0c1affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-01-24T23:21:12.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.335+0000 7f0c1affd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:12.341 INFO:teuthology.orchestra.run.smithi115.stdout:47244640257 2024-01-24T23:21:12.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.339+0000 7f0c2c587640 1 -- 172.21.15.115:0/2623613245 >> v1:172.21.15.132:6812/1835401922 conn(0x7f0c08037160 legacy=0x7f0c08039620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.339+0000 7f0c2c587640 10 monclient: shutdown 2024-01-24T23:21:12.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.339+0000 7f0c2c587640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:12.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.339+0000 7f0c2c587640 1 -- 172.21.15.115:0/2623613245 >> v1:172.21.15.115:6792/0 conn(0x7f0c24139b20 legacy=0x7f0c2413b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.339+0000 7f0c2c587640 1 -- 172.21.15.115:0/2623613245 shutdown_connections 2024-01-24T23:21:12.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.339+0000 7f0c2c587640 1 -- 172.21.15.115:0/2623613245 >> 172.21.15.115:0/2623613245 conn(0x7f0c240a6490 msgr2=0x7f0c24066c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:12.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.339+0000 7f0c2c587640 1 -- 172.21.15.115:0/2623613245 shutdown_connections 2024-01-24T23:21:12.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.339+0000 7f0c2c587640 1 -- 172.21.15.115:0/2623613245 wait complete. 2024-01-24T23:21:12.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.343+0000 7f132de12640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2024-01-24T23:21:12.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.343+0000 7f132de12640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-01-24T23:21:12.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.343+0000 7f132de12640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:12.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.343+0000 7f132de12640 1 -- 172.21.15.115:0/3161324651 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1308005340 con 0x7f1328133bf0 2024-01-24T23:21:12.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.343+0000 7f130ffff640 1 -- 172.21.15.115:0/3161324651 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (unknown 383520633 0 847495621) 0x7f1318012db0 con 0x7f1328133bf0 2024-01-24T23:21:12.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.343+0000 7f130ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-01-24T23:21:12.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.343+0000 7f130ffff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:12.347 INFO:teuthology.orchestra.run.smithi115.stdout:47244640259 2024-01-24T23:21:12.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.347+0000 7f470a488640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2024-01-24T23:21:12.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.347+0000 7f470a488640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-01-24T23:21:12.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.347+0000 7f470a488640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:12.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.347+0000 7f470a488640 1 -- 172.21.15.115:0/2133760088 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f46e4005340 con 0x7f4704133bf0 2024-01-24T23:21:12.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.347+0000 7f4700ff9640 1 -- 172.21.15.115:0/2133760088 <== mon.2 v1:172.21.15.115:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (unknown 1823589948 0 3327677263) 0x7f46f40146a0 con 0x7f4704133bf0 2024-01-24T23:21:12.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.347+0000 7f4700ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-01-24T23:21:12.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.347+0000 7f4700ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:12.350 INFO:teuthology.orchestra.run.smithi115.stdout:47244640257 2024-01-24T23:21:12.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.347+0000 7f132de12640 1 -- 172.21.15.115:0/3161324651 >> v1:172.21.15.132:6812/1835401922 conn(0x7f13140378f0 legacy=0x7f1314039db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.347+0000 7f132de12640 10 monclient: shutdown 2024-01-24T23:21:12.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.347+0000 7f132de12640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:12.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.347+0000 7f132de12640 1 -- 172.21.15.115:0/3161324651 >> v1:172.21.15.115:6791/0 conn(0x7f1328133bf0 legacy=0x7f13280a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.347+0000 7f132de12640 1 -- 172.21.15.115:0/3161324651 shutdown_connections 2024-01-24T23:21:12.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.347+0000 7f132de12640 1 -- 172.21.15.115:0/3161324651 >> 172.21.15.115:0/3161324651 conn(0x7f13280662f0 msgr2=0x7f13281be760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:12.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.347+0000 7f132de12640 1 -- 172.21.15.115:0/3161324651 shutdown_connections 2024-01-24T23:21:12.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.347+0000 7f132de12640 1 -- 172.21.15.115:0/3161324651 wait complete. 2024-01-24T23:21:12.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.347+0000 7f470a488640 1 -- 172.21.15.115:0/2133760088 >> v1:172.21.15.132:6812/1835401922 conn(0x7f46f0037c60 legacy=0x7f46f003a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.347+0000 7f470a488640 10 monclient: shutdown 2024-01-24T23:21:12.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.347+0000 7f470a488640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:12.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.347+0000 7f470a488640 1 -- 172.21.15.115:0/2133760088 >> v1:172.21.15.115:6790/0 conn(0x7f4704133bf0 legacy=0x7f47040a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:12.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.347+0000 7f470a488640 1 -- 172.21.15.115:0/2133760088 shutdown_connections 2024-01-24T23:21:12.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.347+0000 7f470a488640 1 -- 172.21.15.115:0/2133760088 >> 172.21.15.115:0/2133760088 conn(0x7f47040662f0 msgr2=0x7f470409d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:12.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.347+0000 7f470a488640 1 -- 172.21.15.115:0/2133760088 shutdown_connections 2024-01-24T23:21:12.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:12.347+0000 7f470a488640 1 -- 172.21.15.115:0/2133760088 wait complete. 2024-01-24T23:21:12.368 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640258 got 47244640257 for osd.5 2024-01-24T23:21:12.369 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640258 got 47244640257 for osd.4 2024-01-24T23:21:12.373 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640259 for osd.3 2024-01-24T23:21:13.192 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-24T23:21:13.267 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-24T23:21:13.279 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-24T23:21:13.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee7cfd640 10 monclient: get_monmap_and_config 2024-01-24T23:21:13.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee7cfd640 10 monclient: build_initial_monmap 2024-01-24T23:21:13.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee7cfd640 10 monclient: monmap: 2024-01-24T23:21:13.318 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:13.318 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.318 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:13.319532+0000 2024-01-24T23:21:13.318 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:13.319532+0000 2024-01-24T23:21:13.318 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:13.318 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.318 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:13.318 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:13.318 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:13.318 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:13.319 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:13.319 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:13.319 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:13.319 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:13.319 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:13.319 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee7cfd640 1 Processor -- start 2024-01-24T23:21:13.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee7cfd640 1 -- start start 2024-01-24T23:21:13.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee7cfd640 10 monclient: init 2024-01-24T23:21:13.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee7cfd640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:13.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee7cfd640 10 monclient: _add_conns ranks=[3,5,1,8,4,2,6,0,7] 2024-01-24T23:21:13.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee7cfd640 10 monclient(hunting): picked mon.noname-g con 0x7faee009f970 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:13.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee7cfd640 10 monclient(hunting): picked mon.noname-h con 0x7faee0134c90 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:13.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee7cfd640 10 monclient(hunting): picked mon.noname-f con 0x7faee013b210 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:13.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee7cfd640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee004ed70 con 0x7faee013b210 2024-01-24T23:21:13.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee7cfd640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee009d990 con 0x7faee009f970 2024-01-24T23:21:13.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee7cfd640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee009dc70 con 0x7faee0134c90 2024-01-24T23:21:13.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee7cfd640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:13.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee7cfd640 10 monclient(hunting): authenticate will time out at 952.242107s 2024-01-24T23:21:13.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee5298640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7faee0134c90 0x7faee0139af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.115:53240/0 (socket says 172.21.15.115:53240) 2024-01-24T23:21:13.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee5a99640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7faee009f970 0x7faee009fd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.115:36550/0 (socket says 172.21.15.115:36550) 2024-01-24T23:21:13.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee629a640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7faee013b210 0x7faee013d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.115:54704/0 (socket says 172.21.15.115:54704) 2024-01-24T23:21:13.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee5298640 1 -- 172.21.15.115:0/551107727 learned_addr learned my addr 172.21.15.115:0/551107727 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:13.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 1 -- 172.21.15.115:0/551107727 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2508476207 0 0) 0x7faee009dc70 con 0x7faee0134c90 2024-01-24T23:21:13.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 10 monclient(hunting): my global_id is 4361 2024-01-24T23:21:13.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 1 -- 172.21.15.115:0/551107727 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faec8003530 con 0x7faee0134c90 2024-01-24T23:21:13.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 1 -- 172.21.15.115:0/551107727 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250132945 0 0) 0x7faee009d990 con 0x7faee009f970 2024-01-24T23:21:13.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 10 monclient(hunting): my global_id is 4350 2024-01-24T23:21:13.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 1 -- 172.21.15.115:0/551107727 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faee009dc70 con 0x7faee009f970 2024-01-24T23:21:13.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 1 -- 172.21.15.115:0/551107727 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1122024715 0 0) 0x7faee004ed70 con 0x7faee013b210 2024-01-24T23:21:13.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 10 monclient(hunting): my global_id is 4330 2024-01-24T23:21:13.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 1 -- 172.21.15.115:0/551107727 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faee009d990 con 0x7faee013b210 2024-01-24T23:21:13.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 1 -- 172.21.15.115:0/551107727 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1370003127 0 0) 0x7faec8003530 con 0x7faee0134c90 2024-01-24T23:21:13.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 1 -- 172.21.15.115:0/551107727 >> v1:172.21.15.132:6790/0 conn(0x7faee009f970 legacy=0x7faee009fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 1 -- 172.21.15.115:0/551107727 >> v1:172.21.15.132:6789/0 conn(0x7faee013b210 legacy=0x7faee013d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 10 monclient: _finish_hunting 0 2024-01-24T23:21:13.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 1 monclient: found mon.noname-h 2024-01-24T23:21:13.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:13.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:21:13.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 1 -- 172.21.15.115:0/551107727 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee009df50 con 0x7faee0134c90 2024-01-24T23:21:13.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 10 monclient: _finish_auth 0 2024-01-24T23:21:13.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 10 monclient: _check_auth_tickets 2024-01-24T23:21:13.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:13.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 1 -- 172.21.15.115:0/551107727 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faed40017c0 con 0x7faee0134c90 2024-01-24T23:21:13.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:21:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 10 monclient: dump: 2024-01-24T23:21:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.326 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee7cfd640 5 monclient: authenticate success, global_id 4361 2024-01-24T23:21:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee7cfd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 1 -- 172.21.15.115:0/551107727 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faed4001a30 con 0x7faee0134c90 2024-01-24T23:21:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee7cfd640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee7cfd640 4 set_mon_vals no callback set 2024-01-24T23:21:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 1 -- 172.21.15.115:0/551107727 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faed4004bf0 con 0x7faee0134c90 2024-01-24T23:21:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:21:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.315+0000 7faee4a97640 10 monclient: dump: 2024-01-24T23:21:13.326 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.327 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 10 monclient: shutdown 2024-01-24T23:21:13.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:13.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 1 -- 172.21.15.115:0/551107727 >> v1:172.21.15.132:6791/0 conn(0x7faee0134c90 legacy=0x7faee0139af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 1 -- 172.21.15.115:0/551107727 shutdown_connections 2024-01-24T23:21:13.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 1 -- 172.21.15.115:0/551107727 >> 172.21.15.115:0/551107727 conn(0x7faee00a6490 msgr2=0x7faee00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:13.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 1 -- 172.21.15.115:0/551107727 shutdown_connections 2024-01-24T23:21:13.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 1 -- 172.21.15.115:0/551107727 wait complete. 2024-01-24T23:21:13.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 10 monclient: build_initial_monmap 2024-01-24T23:21:13.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 10 monclient: monmap: 2024-01-24T23:21:13.332 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:13.332 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:13.332 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:13.332 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:13.333 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:13.333 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:13.333 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:13.333 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:13.333 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:13.333 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:13.333 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:13.333 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:13.333 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:13.333 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:13.333 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:13.333 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 1 Processor -- start 2024-01-24T23:21:13.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 1 -- start start 2024-01-24T23:21:13.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 10 monclient: init 2024-01-24T23:21:13.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:13.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 10 monclient: _add_conns ranks=[0,8,5,1,4,2,6,3,7] 2024-01-24T23:21:13.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 10 monclient(hunting): picked mon.noname-a con 0x7faee009f970 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:13.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 10 monclient(hunting): picked mon.noname-e con 0x7faee013b210 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:13.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 10 monclient(hunting): picked mon.noname-h con 0x7faee01d1880 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:13.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee0067590 con 0x7faee009f970 2024-01-24T23:21:13.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee0071450 con 0x7faee01d1880 2024-01-24T23:21:13.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee012b3e0 con 0x7faee013b210 2024-01-24T23:21:13.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:13.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 10 monclient(hunting): authenticate will time out at 952.246683s 2024-01-24T23:21:13.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee5298640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7faee013b210 0x7faee012acd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:48406/0 (socket says 172.21.15.115:48406) 2024-01-24T23:21:13.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee5298640 1 -- 172.21.15.115:0/2293414170 learned_addr learned my addr 172.21.15.115:0/2293414170 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:13.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 1 -- 172.21.15.115:0/2293414170 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 393605017 0 0) 0x7faee012b3e0 con 0x7faee013b210 2024-01-24T23:21:13.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 10 monclient(hunting): my global_id is 74276 2024-01-24T23:21:13.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 1 -- 172.21.15.115:0/2293414170 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faecc003530 con 0x7faee013b210 2024-01-24T23:21:13.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 1 -- 172.21.15.115:0/2293414170 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3106804664 0 0) 0x7faee0067590 con 0x7faee009f970 2024-01-24T23:21:13.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 10 monclient(hunting): my global_id is 4401 2024-01-24T23:21:13.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 1 -- 172.21.15.115:0/2293414170 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faee012b3e0 con 0x7faee009f970 2024-01-24T23:21:13.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 1 -- 172.21.15.115:0/2293414170 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2790359907 0 0) 0x7faee0071450 con 0x7faee01d1880 2024-01-24T23:21:13.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 10 monclient(hunting): my global_id is 4370 2024-01-24T23:21:13.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 1 -- 172.21.15.115:0/2293414170 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faee0067590 con 0x7faee01d1880 2024-01-24T23:21:13.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 1 -- 172.21.15.115:0/2293414170 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 172132692 0 0) 0x7faee012b3e0 con 0x7faee009f970 2024-01-24T23:21:13.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 1 -- 172.21.15.115:0/2293414170 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faee0071450 con 0x7faee009f970 2024-01-24T23:21:13.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 1 -- 172.21.15.115:0/2293414170 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faed0001de0 con 0x7faee009f970 2024-01-24T23:21:13.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:21:13.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 10 monclient(hunting): dump: 2024-01-24T23:21:13.336 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.336 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.336 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.336 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.336 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.336 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.337 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.337 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.337 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.337 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.337 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.337 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.337 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.337 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.337 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.337 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:13.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 1 -- 172.21.15.115:0/2293414170 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2028712969 0 0) 0x7faecc003530 con 0x7faee013b210 2024-01-24T23:21:13.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 1 -- 172.21.15.115:0/2293414170 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faee012b3e0 con 0x7faee013b210 2024-01-24T23:21:13.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 1 -- 172.21.15.115:0/2293414170 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faed4001860 con 0x7faee013b210 2024-01-24T23:21:13.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:21:13.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 10 monclient(hunting): dump: 2024-01-24T23:21:13.338 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.338 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.338 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.338 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.338 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.338 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.338 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.338 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.338 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.338 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.338 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.338 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.338 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.338 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.339 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.339 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 1 -- 172.21.15.115:0/2293414170 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 523001270 0 0) 0x7faee0071450 con 0x7faee009f970 2024-01-24T23:21:13.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 1 -- 172.21.15.115:0/2293414170 >> v1:172.21.15.115:6793/0 conn(0x7faee013b210 legacy=0x7faee012acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 1 -- 172.21.15.115:0/2293414170 >> v1:172.21.15.132:6791/0 conn(0x7faee01d1880 legacy=0x7faee01d3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:21:13.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 1 monclient: found mon.a 2024-01-24T23:21:13.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:13.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:21:13.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 1 -- 172.21.15.115:0/2293414170 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee009c1f0 con 0x7faee009f970 2024-01-24T23:21:13.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 10 monclient: _finish_auth 0 2024-01-24T23:21:13.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:21:13.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faede7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:13.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 5 monclient: authenticate success, global_id 4401 2024-01-24T23:21:13.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 10 monclient: _renew_subs 2024-01-24T23:21:13.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:21:13.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 1 -- 172.21.15.115:0/2293414170 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee01340c0 con 0x7faee009f970 2024-01-24T23:21:13.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 10 monclient: _renew_subs 2024-01-24T23:21:13.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.319+0000 7faee7cfd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:21:13.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.323+0000 7faee7cfd640 1 -- 172.21.15.115:0/2293414170 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7faee0139f60 con 0x7faee009f970 2024-01-24T23:21:13.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.323+0000 7faede7fc640 1 -- 172.21.15.115:0/2293414170 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faed00020b0 con 0x7faee009f970 2024-01-24T23:21:13.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.323+0000 7faede7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:13.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.323+0000 7faede7fc640 1 -- 172.21.15.115:0/2293414170 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faed0005160 con 0x7faee009f970 2024-01-24T23:21:13.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.323+0000 7faede7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.323+0000 7faede7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:13.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.323+0000 7faede7fc640 10 monclient: dump: 2024-01-24T23:21:13.340 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.340 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.340 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.341 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.341 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.341 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.341 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.341 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.341 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.341 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.341 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.341 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.341 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.341 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.341 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.341 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.323+0000 7faedffff640 4 set_mon_vals no callback set 2024-01-24T23:21:13.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.323+0000 7faee7cfd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:13.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.323+0000 7faee7cfd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:13.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.323+0000 7faee7cfd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:21:13.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.323+0000 7faee7cfd640 1 -- 172.21.15.115:0/2293414170 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee006a0d0 con 0x7faee009f970 2024-01-24T23:21:13.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.323+0000 7faede7fc640 1 -- 172.21.15.115:0/2293414170 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7faed000fbe0 con 0x7faee009f970 2024-01-24T23:21:13.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.323+0000 7faede7fc640 1 -- 172.21.15.115:0/2293414170 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7faed0043890 con 0x7faee009f970 2024-01-24T23:21:13.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.323+0000 7faee5298640 10 monclient: get_auth_request con 0x7faecc037790 auth_method 0 2024-01-24T23:21:13.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.331+0000 7faede7fc640 1 -- 172.21.15.115:0/2293414170 <== mon.0 v1:172.21.15.115:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7faed00149d0 con 0x7faee009f970 2024-01-24T23:21:13.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.331+0000 7faede7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:13.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.331+0000 7faede7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:13.369 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-01-24T23:21:13.369 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-01-24T23:21:13.374 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-24T23:21:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c883f2640 10 monclient: get_monmap_and_config 2024-01-24T23:21:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c883f2640 10 monclient: build_initial_monmap 2024-01-24T23:21:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c883f2640 10 monclient: monmap: 2024-01-24T23:21:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:13.384301+0000 2024-01-24T23:21:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:13.384301+0000 2024-01-24T23:21:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:13.383 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:13.384 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:13.384 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:13.384 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:13.384 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:13.384 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:13.384 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:13.384 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:13.384 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c883f2640 1 Processor -- start 2024-01-24T23:21:13.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c883f2640 1 -- start start 2024-01-24T23:21:13.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c883f2640 10 monclient: init 2024-01-24T23:21:13.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c883f2640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:13.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c883f2640 10 monclient: _add_conns ranks=[8,6,0,4,7,2,5,3,1] 2024-01-24T23:21:13.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c883f2640 10 monclient(hunting): picked mon.noname-e con 0x7f8c80133bf0 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:13.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c883f2640 10 monclient(hunting): picked mon.noname-d con 0x7f8c80133fd0 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:13.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c883f2640 10 monclient(hunting): picked mon.noname-a con 0x7f8c801343b0 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:13.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c883f2640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c8004ed70 con 0x7f8c801343b0 2024-01-24T23:21:13.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c883f2640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c800a4b60 con 0x7f8c80133fd0 2024-01-24T23:21:13.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c883f2640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c8009ae50 con 0x7f8c80133bf0 2024-01-24T23:21:13.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c883f2640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:13.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c883f2640 10 monclient(hunting): authenticate will time out at 952.307140s 2024-01-24T23:21:13.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c8698f640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f8c801343b0 0x7f8c8013f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58130/0 (socket says 172.21.15.115:58130) 2024-01-24T23:21:13.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c8698f640 1 -- 172.21.15.115:0/363344618 learned_addr learned my addr 172.21.15.115:0/363344618 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:13.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c8618e640 1 --1- 172.21.15.115:0/363344618 >> v1:172.21.15.115:6793/0 conn(0x7f8c80133bf0 0x7f8c8009bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:48412/0 (socket says 172.21.15.115:48412) 2024-01-24T23:21:13.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c8518c640 1 -- 172.21.15.115:0/363344618 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163516178 0 0) 0x7f8c800a4b60 con 0x7f8c80133fd0 2024-01-24T23:21:13.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c8518c640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c8518c640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c8518c640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c8518c640 10 monclient(hunting): my global_id is 4389 2024-01-24T23:21:13.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c8518c640 1 -- 172.21.15.115:0/363344618 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c6c003530 con 0x7f8c80133fd0 2024-01-24T23:21:13.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c8518c640 1 -- 172.21.15.115:0/363344618 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2912776845 0 0) 0x7f8c8004ed70 con 0x7f8c801343b0 2024-01-24T23:21:13.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c8518c640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c8518c640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c8518c640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c8518c640 10 monclient(hunting): my global_id is 4410 2024-01-24T23:21:13.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.379+0000 7f8c8518c640 1 -- 172.21.15.115:0/363344618 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c800a4b60 con 0x7f8c801343b0 2024-01-24T23:21:13.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c8518c640 1 -- 172.21.15.115:0/363344618 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1570991562 0 0) 0x7f8c6c003530 con 0x7f8c80133fd0 2024-01-24T23:21:13.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c8518c640 1 -- 172.21.15.115:0/363344618 >> v1:172.21.15.115:6793/0 conn(0x7f8c80133bf0 legacy=0x7f8c8009bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T23:21:13.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c8518c640 1 -- 172.21.15.115:0/363344618 >> v1:172.21.15.115:6789/0 conn(0x7f8c801343b0 legacy=0x7f8c8013f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c8518c640 10 monclient: _finish_hunting 0 2024-01-24T23:21:13.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c8518c640 1 monclient: found mon.noname-d 2024-01-24T23:21:13.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c8518c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:13.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c8518c640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:21:13.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c8518c640 1 -- 172.21.15.115:0/363344618 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8009b130 con 0x7f8c80133fd0 2024-01-24T23:21:13.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c8518c640 10 monclient: _finish_auth 0 2024-01-24T23:21:13.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c8518c640 10 monclient: _check_auth_tickets 2024-01-24T23:21:13.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c8518c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:13.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c883f2640 5 monclient: authenticate success, global_id 4389 2024-01-24T23:21:13.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c883f2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:13.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c8518c640 1 -- 172.21.15.115:0/363344618 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8c700017c0 con 0x7f8c80133fd0 2024-01-24T23:21:13.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c8518c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c8518c640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:21:13.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c8518c640 10 monclient: dump: 2024-01-24T23:21:13.390 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.390 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.390 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.390 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.390 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.391 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c883f2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c8518c640 1 -- 172.21.15.115:0/363344618 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8c70001a50 con 0x7f8c80133fd0 2024-01-24T23:21:13.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c8518c640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:13.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c883f2640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:13.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c883f2640 4 set_mon_vals no callback set 2024-01-24T23:21:13.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c883f2640 10 monclient: shutdown 2024-01-24T23:21:13.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c883f2640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:13.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c883f2640 1 -- 172.21.15.115:0/363344618 >> v1:172.21.15.115:6792/0 conn(0x7f8c80133fd0 legacy=0x7f8c8013bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c883f2640 1 -- 172.21.15.115:0/363344618 shutdown_connections 2024-01-24T23:21:13.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.383+0000 7f8c883f2640 1 -- 172.21.15.115:0/363344618 >> 172.21.15.115:0/363344618 conn(0x7f8c800662f0 msgr2=0x7f8c80066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:13.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c883f2640 1 -- 172.21.15.115:0/363344618 shutdown_connections 2024-01-24T23:21:13.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c883f2640 1 -- 172.21.15.115:0/363344618 wait complete. 2024-01-24T23:21:13.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c883f2640 10 monclient: build_initial_monmap 2024-01-24T23:21:13.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c883f2640 10 monclient: monmap: 2024-01-24T23:21:13.394 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:13.394 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:13.394 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:13.394 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:13.394 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:13.394 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:13.394 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:13.394 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:13.394 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:13.395 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:13.395 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:13.395 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:13.395 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:13.395 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:13.395 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:13.395 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c883f2640 1 Processor -- start 2024-01-24T23:21:13.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c883f2640 1 -- start start 2024-01-24T23:21:13.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c883f2640 10 monclient: init 2024-01-24T23:21:13.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c883f2640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:13.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c883f2640 10 monclient: _add_conns ranks=[8,4,1,5,7,2,6,3,0] 2024-01-24T23:21:13.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c883f2640 10 monclient(hunting): picked mon.noname-e con 0x7f8c80133fd0 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:13.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c883f2640 10 monclient(hunting): picked mon.noname-c con 0x7f8c801343b0 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c883f2640 10 monclient(hunting): picked mon.noname-f con 0x7f8c800a1d90 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c883f2640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c8009c810 con 0x7f8c800a1d90 2024-01-24T23:21:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c883f2640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c80120cb0 con 0x7f8c801343b0 2024-01-24T23:21:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c883f2640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c800a5990 con 0x7f8c80133fd0 2024-01-24T23:21:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c883f2640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c883f2640 10 monclient(hunting): authenticate will time out at 952.314540s 2024-01-24T23:21:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c8598d640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f8c801343b0 0x7f8c801d0e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:59832/0 (socket says 172.21.15.115:59832) 2024-01-24T23:21:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c8598d640 1 -- 172.21.15.115:0/2780788313 learned_addr learned my addr 172.21.15.115:0/2780788313 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 1 -- 172.21.15.115:0/2780788313 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836573508 0 0) 0x7f8c80120cb0 con 0x7f8c801343b0 2024-01-24T23:21:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 10 monclient(hunting): my global_id is 74362 2024-01-24T23:21:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 1 -- 172.21.15.115:0/2780788313 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c64003530 con 0x7f8c801343b0 2024-01-24T23:21:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 1 -- 172.21.15.115:0/2780788313 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2929025509 0 0) 0x7f8c800a5990 con 0x7f8c80133fd0 2024-01-24T23:21:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 10 monclient(hunting): my global_id is 74285 2024-01-24T23:21:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 1 -- 172.21.15.115:0/2780788313 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c80120cb0 con 0x7f8c80133fd0 2024-01-24T23:21:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 1 -- 172.21.15.115:0/2780788313 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3457344068 0 0) 0x7f8c64003530 con 0x7f8c801343b0 2024-01-24T23:21:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 1 -- 172.21.15.115:0/2780788313 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8c800a5990 con 0x7f8c801343b0 2024-01-24T23:21:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 1 -- 172.21.15.115:0/2780788313 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8c70003f60 con 0x7f8c801343b0 2024-01-24T23:21:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:21:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 10 monclient(hunting): dump: 2024-01-24T23:21:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.397 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.398 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.398 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.398 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.398 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.398 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.398 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.398 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.398 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.398 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.398 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.398 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.398 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.398 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.398 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:13.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 1 -- 172.21.15.115:0/2780788313 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 956142012 0 0) 0x7f8c800a5990 con 0x7f8c801343b0 2024-01-24T23:21:13.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 1 -- 172.21.15.115:0/2780788313 >> v1:172.21.15.115:6793/0 conn(0x7f8c80133fd0 legacy=0x7f8c801cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 1 -- 172.21.15.115:0/2780788313 >> v1:172.21.15.132:6789/0 conn(0x7f8c800a1d90 legacy=0x7f8c801d45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:21:13.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 1 monclient: found mon.c 2024-01-24T23:21:13.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:13.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:13.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 1 -- 172.21.15.115:0/2780788313 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c80133770 con 0x7f8c801343b0 2024-01-24T23:21:13.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 10 monclient: _finish_auth 0 2024-01-24T23:21:13.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:21:13.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:13.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c883f2640 5 monclient: authenticate success, global_id 74362 2024-01-24T23:21:13.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c76ffd640 1 -- 172.21.15.115:0/2780788313 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8c700040c0 con 0x7f8c801343b0 2024-01-24T23:21:13.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c883f2640 10 monclient: _renew_subs 2024-01-24T23:21:13.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c883f2640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.387+0000 7f8c883f2640 1 -- 172.21.15.115:0/2780788313 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c800a2a10 con 0x7f8c801343b0 2024-01-24T23:21:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.391+0000 7f8c76ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.391+0000 7f8c76ffd640 1 -- 172.21.15.115:0/2780788313 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8c700045f0 con 0x7f8c801343b0 2024-01-24T23:21:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.391+0000 7f8c76ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.391+0000 7f8c76ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.391+0000 7f8c76ffd640 10 monclient: dump: 2024-01-24T23:21:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.400 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.401 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.401 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.401 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.401 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.401 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.401 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.401 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.391+0000 7f8c883f2640 10 monclient: _renew_subs 2024-01-24T23:21:13.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.391+0000 7f8c883f2640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:13.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.391+0000 7f8c883f2640 1 -- 172.21.15.115:0/2780788313 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c800a3570 con 0x7f8c801343b0 2024-01-24T23:21:13.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.391+0000 7f8c8498b640 4 set_mon_vals no callback set 2024-01-24T23:21:13.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.391+0000 7f8c883f2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:13.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.391+0000 7f8c883f2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:13.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.391+0000 7f8c883f2640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:13.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.391+0000 7f8c883f2640 1 -- 172.21.15.115:0/2780788313 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c800a21c0 con 0x7f8c801343b0 2024-01-24T23:21:13.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.391+0000 7f8c76ffd640 1 -- 172.21.15.115:0/2780788313 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f8c70003600 con 0x7f8c801343b0 2024-01-24T23:21:13.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.391+0000 7f8c76ffd640 1 -- 172.21.15.115:0/2780788313 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f8c70043800 con 0x7f8c801343b0 2024-01-24T23:21:13.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.391+0000 7f8c8618e640 10 monclient: get_auth_request con 0x7f8c640371a0 auth_method 0 2024-01-24T23:21:13.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.399+0000 7f8c76ffd640 1 -- 172.21.15.115:0/2780788313 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f8c800a21c0 con 0x7f8c801343b0 2024-01-24T23:21:13.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.399+0000 7f8c76ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:13.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.399+0000 7f8c76ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:13.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.423+0000 7fe2b64e1640 10 monclient: get_monmap_and_config 2024-01-24T23:21:13.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.423+0000 7fe2b64e1640 10 monclient: build_initial_monmap 2024-01-24T23:21:13.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 10 monclient: monmap: 2024-01-24T23:21:13.429 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:13.430 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.430 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:13.430905+0000 2024-01-24T23:21:13.430 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:13.430905+0000 2024-01-24T23:21:13.430 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:13.430 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.430 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:13.430 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:13.430 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:13.430 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:13.430 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:13.430 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:13.430 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:13.430 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:13.430 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:13.430 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 1 Processor -- start 2024-01-24T23:21:13.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 1 -- start start 2024-01-24T23:21:13.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 10 monclient: init 2024-01-24T23:21:13.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:13.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 10 monclient: _add_conns ranks=[7,4,0,6,1,5,2,8,3] 2024-01-24T23:21:13.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 10 monclient(hunting): picked mon.noname-i con 0x7fe2b0133bf0 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:13.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 10 monclient(hunting): picked mon.noname-c con 0x7fe2b0133fd0 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:13.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 10 monclient(hunting): picked mon.noname-a con 0x7fe2b01343b0 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:13.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b004ed70 con 0x7fe2b01343b0 2024-01-24T23:21:13.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b00a4b60 con 0x7fe2b0133fd0 2024-01-24T23:21:13.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b009ae50 con 0x7fe2b0133bf0 2024-01-24T23:21:13.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2af7fe640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fe2b0133fd0 0x7fe2b013bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:59844/0 (socket says 172.21.15.115:59844) 2024-01-24T23:21:13.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2af7fe640 1 -- 172.21.15.115:0/1534923142 learned_addr learned my addr 172.21.15.115:0/1534923142 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:13.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:13.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 10 monclient(hunting): authenticate will time out at 952.353725s 2024-01-24T23:21:13.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 1 -- 172.21.15.115:0/1534923142 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2210570791 0 0) 0x7fe2b00a4b60 con 0x7fe2b0133fd0 2024-01-24T23:21:13.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 10 monclient(hunting): my global_id is 74371 2024-01-24T23:21:13.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 1 -- 172.21.15.115:0/1534923142 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe298003530 con 0x7fe2b0133fd0 2024-01-24T23:21:13.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 1 -- 172.21.15.115:0/1534923142 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2626262815 0 0) 0x7fe2b004ed70 con 0x7fe2b01343b0 2024-01-24T23:21:13.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 10 monclient(hunting): my global_id is 4419 2024-01-24T23:21:13.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 1 -- 172.21.15.115:0/1534923142 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2b00a4b60 con 0x7fe2b01343b0 2024-01-24T23:21:13.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 1 -- 172.21.15.115:0/1534923142 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3323638163 0 0) 0x7fe2b009ae50 con 0x7fe2b0133bf0 2024-01-24T23:21:13.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 10 monclient(hunting): my global_id is 4372 2024-01-24T23:21:13.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 1 -- 172.21.15.115:0/1534923142 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2b004ed70 con 0x7fe2b0133bf0 2024-01-24T23:21:13.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 1 -- 172.21.15.115:0/1534923142 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2247139812 0 0) 0x7fe2b00a4b60 con 0x7fe2b01343b0 2024-01-24T23:21:13.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 1 -- 172.21.15.115:0/1534923142 >> v1:172.21.15.132:6792/0 conn(0x7fe2b0133bf0 legacy=0x7fe2b009bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 1 -- 172.21.15.115:0/1534923142 >> v1:172.21.15.115:6791/0 conn(0x7fe2b0133fd0 legacy=0x7fe2b013bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 10 monclient: _finish_hunting 0 2024-01-24T23:21:13.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 1 monclient: found mon.noname-a 2024-01-24T23:21:13.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:13.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:21:13.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 1 -- 172.21.15.115:0/1534923142 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b009b130 con 0x7fe2b01343b0 2024-01-24T23:21:13.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 10 monclient: _finish_auth 0 2024-01-24T23:21:13.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 10 monclient: _check_auth_tickets 2024-01-24T23:21:13.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:13.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 1 -- 172.21.15.115:0/1534923142 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe2a80019d0 con 0x7fe2b01343b0 2024-01-24T23:21:13.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:21:13.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 10 monclient: dump: 2024-01-24T23:21:13.436 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.436 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.436 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.436 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.436 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.436 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.436 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.437 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.437 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.437 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.437 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.437 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.437 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.437 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.437 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.437 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 5 monclient: authenticate success, global_id 4419 2024-01-24T23:21:13.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:13.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 1 -- 172.21.15.115:0/1534923142 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe2a8001c60 con 0x7fe2b01343b0 2024-01-24T23:21:13.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:13.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 1 -- 172.21.15.115:0/1534923142 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe2a80039a0 con 0x7fe2b01343b0 2024-01-24T23:21:13.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:13.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 4 set_mon_vals no callback set 2024-01-24T23:21:13.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:13.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2aeffd640 10 monclient: dump: 2024-01-24T23:21:13.438 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.438 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.438 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.438 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.438 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.438 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.438 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.438 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.438 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.438 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.438 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.438 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.439 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.439 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.439 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.439 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 10 monclient: shutdown 2024-01-24T23:21:13.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:13.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 1 -- 172.21.15.115:0/1534923142 >> v1:172.21.15.115:6789/0 conn(0x7fe2b01343b0 legacy=0x7fe2b013f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 1 -- 172.21.15.115:0/1534923142 shutdown_connections 2024-01-24T23:21:13.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 1 -- 172.21.15.115:0/1534923142 >> 172.21.15.115:0/1534923142 conn(0x7fe2b00662f0 msgr2=0x7fe2b0066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:13.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 1 -- 172.21.15.115:0/1534923142 shutdown_connections 2024-01-24T23:21:13.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.427+0000 7fe2b64e1640 1 -- 172.21.15.115:0/1534923142 wait complete. 2024-01-24T23:21:13.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2b64e1640 10 monclient: build_initial_monmap 2024-01-24T23:21:13.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2b64e1640 10 monclient: monmap: 2024-01-24T23:21:13.439 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:13.439 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:13.440 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:13.440 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:13.440 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:13.440 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:13.440 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:13.440 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:13.440 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:13.440 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:13.440 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:13.440 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:13.440 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:13.440 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:13.440 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:13.440 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2b64e1640 1 Processor -- start 2024-01-24T23:21:13.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2b64e1640 1 -- start start 2024-01-24T23:21:13.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2b64e1640 10 monclient: init 2024-01-24T23:21:13.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2b64e1640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:13.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2b64e1640 10 monclient: _add_conns ranks=[7,1,3,2,4,5,6,0,8] 2024-01-24T23:21:13.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2b64e1640 10 monclient(hunting): picked mon.noname-i con 0x7fe2b0133bf0 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:13.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2b64e1640 10 monclient(hunting): picked mon.noname-f con 0x7fe2b0133fd0 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:13.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2b64e1640 10 monclient(hunting): picked mon.noname-g con 0x7fe2b01343b0 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:13.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2b64e1640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b009c810 con 0x7fe2b0133fd0 2024-01-24T23:21:13.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2b64e1640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b00a5990 con 0x7fe2b01343b0 2024-01-24T23:21:13.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2b64e1640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b01208f0 con 0x7fe2b0133bf0 2024-01-24T23:21:13.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2b64e1640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:13.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2b64e1640 10 monclient(hunting): authenticate will time out at 952.357484s 2024-01-24T23:21:13.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2affff640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fe2b0133bf0 0x7fe2b0132f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.115:34702/0 (socket says 172.21.15.115:34702) 2024-01-24T23:21:13.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2affff640 1 -- 172.21.15.115:0/4222565922 learned_addr learned my addr 172.21.15.115:0/4222565922 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:13.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 832592216 0 0) 0x7fe2b009c810 con 0x7fe2b0133fd0 2024-01-24T23:21:13.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient(hunting): my global_id is 4348 2024-01-24T23:21:13.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe29c003530 con 0x7fe2b0133fd0 2024-01-24T23:21:13.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 709575874 0 0) 0x7fe2b01208f0 con 0x7fe2b0133bf0 2024-01-24T23:21:13.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient(hunting): my global_id is 4381 2024-01-24T23:21:13.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2b009c810 con 0x7fe2b0133bf0 2024-01-24T23:21:13.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3231709100 0 0) 0x7fe2b00a5990 con 0x7fe2b01343b0 2024-01-24T23:21:13.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient(hunting): my global_id is 4359 2024-01-24T23:21:13.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2b01208f0 con 0x7fe2b01343b0 2024-01-24T23:21:13.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2214593113 0 0) 0x7fe29c003530 con 0x7fe2b0133fd0 2024-01-24T23:21:13.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe2b00a5990 con 0x7fe2b0133fd0 2024-01-24T23:21:13.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe2a4001580 con 0x7fe2b0133fd0 2024-01-24T23:21:13.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:21:13.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient(hunting): dump: 2024-01-24T23:21:13.448 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.449 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.449 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.449 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.449 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.449 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.449 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.449 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.449 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.449 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.449 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.449 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.449 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.449 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.449 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.449 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:13.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 554337077 0 0) 0x7fe2b009c810 con 0x7fe2b0133bf0 2024-01-24T23:21:13.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe29c003530 con 0x7fe2b0133bf0 2024-01-24T23:21:13.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe2a0001b20 con 0x7fe2b0133bf0 2024-01-24T23:21:13.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-01-24T23:21:13.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient(hunting): dump: 2024-01-24T23:21:13.450 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.450 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.450 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.450 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.450 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.450 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.450 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.450 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.451 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.451 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.451 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.451 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.451 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.451 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.451 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.451 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1784137551 0 0) 0x7fe2b01208f0 con 0x7fe2b01343b0 2024-01-24T23:21:13.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe2b009c810 con 0x7fe2b01343b0 2024-01-24T23:21:13.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe2a80018d0 con 0x7fe2b01343b0 2024-01-24T23:21:13.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:21:13.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient(hunting): dump: 2024-01-24T23:21:13.452 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.452 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.452 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.452 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.452 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.452 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.452 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.452 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.452 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.452 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.452 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.452 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.452 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.452 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.452 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.453 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1369103144 0 0) 0x7fe2b00a5990 con 0x7fe2b0133fd0 2024-01-24T23:21:13.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 >> v1:172.21.15.132:6792/0 conn(0x7fe2b0133bf0 legacy=0x7fe2b0132f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 >> v1:172.21.15.132:6790/0 conn(0x7fe2b01343b0 legacy=0x7fe2b01d3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient: _finish_hunting 0 2024-01-24T23:21:13.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 1 monclient: found mon.f 2024-01-24T23:21:13.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:13.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:21:13.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b00a2540 con 0x7fe2b0133fd0 2024-01-24T23:21:13.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient: _finish_auth 0 2024-01-24T23:21:13.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient: _check_auth_tickets 2024-01-24T23:21:13.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:13.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2b64e1640 5 monclient: authenticate success, global_id 4348 2024-01-24T23:21:13.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2b64e1640 10 monclient: _renew_subs 2024-01-24T23:21:13.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2b64e1640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:21:13.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2b64e1640 1 -- 172.21.15.115:0/4222565922 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b0120610 con 0x7fe2b0133fd0 2024-01-24T23:21:13.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2b64e1640 10 monclient: _renew_subs 2024-01-24T23:21:13.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2b64e1640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:21:13.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2b64e1640 1 -- 172.21.15.115:0/4222565922 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b00a32c0 con 0x7fe2b0133fd0 2024-01-24T23:21:13.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe2a40017f0 con 0x7fe2b0133fd0 2024-01-24T23:21:13.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:13.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe2a4004de0 con 0x7fe2b0133fd0 2024-01-24T23:21:13.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:21:13.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2acff9640 10 monclient: dump: 2024-01-24T23:21:13.454 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.454 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.454 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.454 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.455 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.455 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.455 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.455 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.455 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.455 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.455 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.455 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.455 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.455 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.455 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.455 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.431+0000 7fe2ae7fc640 4 set_mon_vals no callback set 2024-01-24T23:21:13.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.435+0000 7fe2967fc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:13.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.435+0000 7fe2967fc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:13.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.435+0000 7fe2967fc640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:21:13.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.435+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fe2a400f8e0 con 0x7fe2b0133fd0 2024-01-24T23:21:13.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.435+0000 7fe2967fc640 1 -- 172.21.15.115:0/4222565922 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe288005090 con 0x7fe2b0133fd0 2024-01-24T23:21:13.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.435+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7fe2a4043610 con 0x7fe2b0133fd0 2024-01-24T23:21:13.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.435+0000 7fe2affff640 10 monclient: get_auth_request con 0x7fe29c037d20 auth_method 0 2024-01-24T23:21:13.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.447+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fe2a4014750 con 0x7fe2b0133fd0 2024-01-24T23:21:13.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.447+0000 7fe2acff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:13.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.447+0000 7fe2acff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:13.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.503+0000 7faee7cfd640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2024-01-24T23:21:13.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.503+0000 7faee7cfd640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-01-24T23:21:13.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.503+0000 7faee7cfd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:21:13.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.503+0000 7faee7cfd640 1 -- 172.21.15.115:0/2293414170 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7faee006bbd0 con 0x7faee009f970 2024-01-24T23:21:13.508 INFO:teuthology.orchestra.run.smithi115.stdout:47244640260 2024-01-24T23:21:13.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.503+0000 7faede7fc640 1 -- 172.21.15.115:0/2293414170 <== mon.0 v1:172.21.15.115:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (unknown 574334944 0 1444633209) 0x7faed00147f0 con 0x7faee009f970 2024-01-24T23:21:13.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.503+0000 7faede7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-01-24T23:21:13.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.503+0000 7faede7fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:13.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.507+0000 7faee7cfd640 1 -- 172.21.15.115:0/2293414170 >> v1:172.21.15.132:6812/1835401922 conn(0x7faecc037790 legacy=0x7faecc039c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.507+0000 7faee7cfd640 10 monclient: shutdown 2024-01-24T23:21:13.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.507+0000 7faee7cfd640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:13.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.507+0000 7faee7cfd640 1 -- 172.21.15.115:0/2293414170 >> v1:172.21.15.115:6789/0 conn(0x7faee009f970 legacy=0x7faee012a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.507+0000 7faee7cfd640 1 -- 172.21.15.115:0/2293414170 shutdown_connections 2024-01-24T23:21:13.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.507+0000 7faee7cfd640 1 -- 172.21.15.115:0/2293414170 >> 172.21.15.115:0/2293414170 conn(0x7faee00a6490 msgr2=0x7faee00a1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:13.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.507+0000 7faee7cfd640 1 -- 172.21.15.115:0/2293414170 shutdown_connections 2024-01-24T23:21:13.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.507+0000 7faee7cfd640 1 -- 172.21.15.115:0/2293414170 wait complete. 2024-01-24T23:21:13.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a8413640 10 monclient: get_monmap_and_config 2024-01-24T23:21:13.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a8413640 10 monclient: build_initial_monmap 2024-01-24T23:21:13.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a8413640 10 monclient: monmap: 2024-01-24T23:21:13.530 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:13.530 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.530 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:13.531393+0000 2024-01-24T23:21:13.530 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:13.531393+0000 2024-01-24T23:21:13.530 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:13.530 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.530 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:13.530 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:13.530 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:13.530 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:13.530 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:13.530 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:13.530 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:13.531 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:13.531 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:13.531 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a8413640 1 Processor -- start 2024-01-24T23:21:13.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a8413640 1 -- start start 2024-01-24T23:21:13.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a8413640 10 monclient: init 2024-01-24T23:21:13.532 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640260 for osd.0 2024-01-24T23:21:13.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7f6929a2e640 10 monclient: get_monmap_and_config 2024-01-24T23:21:13.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7f6929a2e640 10 monclient: build_initial_monmap 2024-01-24T23:21:13.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7f6929a2e640 10 monclient: monmap: 2024-01-24T23:21:13.533 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:13.533 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.533 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:13.532845+0000 2024-01-24T23:21:13.533 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:13.532845+0000 2024-01-24T23:21:13.533 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:13.533 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.533 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:13.533 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:13.534 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:13.534 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:13.534 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:13.534 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:13.534 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:13.534 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:13.534 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:13.534 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.534 DEBUG:teuthology.parallel:result is None 2024-01-24T23:21:13.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a8413640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:13.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a8413640 10 monclient: _add_conns ranks=[0,4,8,5,1,6,3,7,2] 2024-01-24T23:21:13.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a8413640 10 monclient(hunting): picked mon.noname-a con 0x7fe4a00a18a0 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:13.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a8413640 10 monclient(hunting): picked mon.noname-c con 0x7fe4a00a2390 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:13.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a61af640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fe4a00a18a0 0x7fe4a00a1c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58140/0 (socket says 172.21.15.115:58140) 2024-01-24T23:21:13.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a61af640 1 -- 172.21.15.115:0/628184663 learned_addr learned my addr 172.21.15.115:0/628184663 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:13.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a8413640 10 monclient(hunting): picked mon.noname-e con 0x7fe4a009bc60 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:13.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7f6929a2e640 1 Processor -- start 2024-01-24T23:21:13.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a8413640 1 -- 172.21.15.115:0/628184663 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a004ed70 con 0x7fe4a00a18a0 2024-01-24T23:21:13.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a8413640 1 -- 172.21.15.115:0/628184663 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a009b5f0 con 0x7fe4a00a2390 2024-01-24T23:21:13.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a8413640 1 -- 172.21.15.115:0/628184663 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a009afa0 con 0x7fe4a009bc60 2024-01-24T23:21:13.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a8413640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:13.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a8413640 10 monclient(hunting): authenticate will time out at 952.454829s 2024-01-24T23:21:13.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 1 -- 172.21.15.115:0/628184663 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3760082230 0 0) 0x7fe4a004ed70 con 0x7fe4a00a18a0 2024-01-24T23:21:13.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 10 monclient(hunting): my global_id is 4428 2024-01-24T23:21:13.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 1 -- 172.21.15.115:0/628184663 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe48c003530 con 0x7fe4a00a18a0 2024-01-24T23:21:13.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 1 -- 172.21.15.115:0/628184663 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2419601674 0 0) 0x7fe4a009afa0 con 0x7fe4a009bc60 2024-01-24T23:21:13.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 10 monclient(hunting): my global_id is 74294 2024-01-24T23:21:13.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 1 -- 172.21.15.115:0/628184663 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4a004ed70 con 0x7fe4a009bc60 2024-01-24T23:21:13.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7f6929a2e640 1 -- start start 2024-01-24T23:21:13.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7f6929a2e640 10 monclient: init 2024-01-24T23:21:13.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7f6929a2e640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:13.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7f6929a2e640 10 monclient: _add_conns ranks=[8,7,0,4,1,6,3,5,2] 2024-01-24T23:21:13.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7f6929a2e640 10 monclient(hunting): picked mon.noname-e con 0x7f69240a18a0 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:13.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7f6929a2e640 10 monclient(hunting): picked mon.noname-i con 0x7f69240a1c80 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:13.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7f6929a2e640 10 monclient(hunting): picked mon.noname-a con 0x7f6924139b20 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:13.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7f6929a2e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f692404ed70 con 0x7f6924139b20 2024-01-24T23:21:13.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7f6929a2e640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f692409cb70 con 0x7f69240a1c80 2024-01-24T23:21:13.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7f6929a2e640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f692409c800 con 0x7f69240a18a0 2024-01-24T23:21:13.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7f6929a2e640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:13.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7f6929a2e640 10 monclient(hunting): authenticate will time out at 952.455377s 2024-01-24T23:21:13.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7f6922ffd640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f69240a18a0 0x7f692409db30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:48426/0 (socket says 172.21.15.115:48426) 2024-01-24T23:21:13.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7f6922ffd640 1 -- 172.21.15.115:0/2223283744 learned_addr learned my addr 172.21.15.115:0/2223283744 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:13.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 1 -- 172.21.15.115:0/628184663 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 203522188 0 0) 0x7fe4a004ed70 con 0x7fe4a009bc60 2024-01-24T23:21:13.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 1 -- 172.21.15.115:0/628184663 >> v1:172.21.15.115:6791/0 conn(0x7fe4a00a2390 legacy=0x7fe4a013bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 1 -- 172.21.15.115:0/628184663 >> v1:172.21.15.115:6789/0 conn(0x7fe4a00a18a0 legacy=0x7fe4a00a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 10 monclient: _finish_hunting 0 2024-01-24T23:21:13.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 1 monclient: found mon.noname-e 2024-01-24T23:21:13.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:13.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:21:13.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 1 -- 172.21.15.115:0/628184663 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a009b280 con 0x7fe4a009bc60 2024-01-24T23:21:13.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 10 monclient: _finish_auth 0 2024-01-24T23:21:13.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 10 monclient: _check_auth_tickets 2024-01-24T23:21:13.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:13.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 1 -- 172.21.15.115:0/628184663 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe498001b20 con 0x7fe4a009bc60 2024-01-24T23:21:13.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:21:13.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 10 monclient: dump: 2024-01-24T23:21:13.540 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.540 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.540 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.540 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.541 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.541 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.541 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.541 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.541 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.541 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.541 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.541 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.541 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.541 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.541 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.541 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 1 -- 172.21.15.115:0/628184663 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe498001d40 con 0x7fe4a009bc60 2024-01-24T23:21:13.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:13.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 1 -- 172.21.15.115:0/628184663 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe498003ce0 con 0x7fe4a009bc60 2024-01-24T23:21:13.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:21:13.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.527+0000 7fe4a51ad640 10 monclient: dump: 2024-01-24T23:21:13.542 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.542 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.542 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.542 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.542 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.542 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.542 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.542 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.542 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.542 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.542 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.542 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.542 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.542 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.543 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.543 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 1 -- 172.21.15.115:0/2223283744 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122804363 0 0) 0x7f692404ed70 con 0x7f6924139b20 2024-01-24T23:21:13.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 10 monclient(hunting): my global_id is 4437 2024-01-24T23:21:13.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 1 -- 172.21.15.115:0/2223283744 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6910005740 con 0x7f6924139b20 2024-01-24T23:21:13.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 5 monclient: authenticate success, global_id 74294 2024-01-24T23:21:13.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:13.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 4 set_mon_vals no callback set 2024-01-24T23:21:13.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 1 -- 172.21.15.115:0/2223283744 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3852871382 0 0) 0x7f692409cb70 con 0x7f69240a1c80 2024-01-24T23:21:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 10 monclient(hunting): my global_id is 4390 2024-01-24T23:21:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 1 -- 172.21.15.115:0/2223283744 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f692404ed70 con 0x7f69240a1c80 2024-01-24T23:21:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 1 -- 172.21.15.115:0/2223283744 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1806220191 0 0) 0x7f6910005740 con 0x7f6924139b20 2024-01-24T23:21:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 1 -- 172.21.15.115:0/2223283744 >> v1:172.21.15.115:6793/0 conn(0x7f69240a18a0 legacy=0x7f692409db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 1 -- 172.21.15.115:0/2223283744 >> v1:172.21.15.132:6792/0 conn(0x7f69240a1c80 legacy=0x7f69240a2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:21:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 1 monclient: found mon.noname-a 2024-01-24T23:21:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:21:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 1 -- 172.21.15.115:0/2223283744 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692409e240 con 0x7f6924139b20 2024-01-24T23:21:13.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 10 monclient: _finish_auth 0 2024-01-24T23:21:13.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:21:13.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:13.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 1 -- 172.21.15.115:0/2223283744 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f690c0017c0 con 0x7f6924139b20 2024-01-24T23:21:13.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:21:13.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 10 monclient: dump: 2024-01-24T23:21:13.546 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.546 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.546 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.546 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.546 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.546 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.546 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.546 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.546 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.547 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 5 monclient: authenticate success, global_id 4437 2024-01-24T23:21:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 1 -- 172.21.15.115:0/2223283744 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f690c0019e0 con 0x7f6924139b20 2024-01-24T23:21:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 4 set_mon_vals no callback set 2024-01-24T23:21:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 1 -- 172.21.15.115:0/2223283744 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f690c003960 con 0x7f6924139b20 2024-01-24T23:21:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:13.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6921ffb640 10 monclient: dump: 2024-01-24T23:21:13.548 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.548 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.548 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.548 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.548 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.548 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.548 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.548 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.548 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.548 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.548 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.548 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.548 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.549 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.549 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.549 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 10 monclient: shutdown 2024-01-24T23:21:13.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:13.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 1 -- 172.21.15.115:0/628184663 >> v1:172.21.15.115:6793/0 conn(0x7fe4a009bc60 legacy=0x7fe4a013f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 10 monclient: shutdown 2024-01-24T23:21:13.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:13.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 1 -- 172.21.15.115:0/2223283744 >> v1:172.21.15.115:6789/0 conn(0x7f6924139b20 legacy=0x7f692413bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 1 -- 172.21.15.115:0/2223283744 shutdown_connections 2024-01-24T23:21:13.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 1 -- 172.21.15.115:0/2223283744 >> 172.21.15.115:0/2223283744 conn(0x7f69240a6490 msgr2=0x7f69240a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:13.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 1 -- 172.21.15.115:0/628184663 shutdown_connections 2024-01-24T23:21:13.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 1 -- 172.21.15.115:0/628184663 >> 172.21.15.115:0/628184663 conn(0x7fe4a00a6490 msgr2=0x7fe4a00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:13.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 1 -- 172.21.15.115:0/628184663 shutdown_connections 2024-01-24T23:21:13.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 1 -- 172.21.15.115:0/628184663 wait complete. 2024-01-24T23:21:13.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 1 -- 172.21.15.115:0/2223283744 shutdown_connections 2024-01-24T23:21:13.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 1 -- 172.21.15.115:0/2223283744 wait complete. 2024-01-24T23:21:13.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 10 monclient: build_initial_monmap 2024-01-24T23:21:13.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 10 monclient: monmap: 2024-01-24T23:21:13.552 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:13.552 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:13.552 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:13.552 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:13.552 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:13.552 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:13.552 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:13.552 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:13.552 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:13.552 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:13.552 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:13.552 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:13.552 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:13.553 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:13.553 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:13.553 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 10 monclient: build_initial_monmap 2024-01-24T23:21:13.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 10 monclient: monmap: 2024-01-24T23:21:13.553 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:13.553 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:13.553 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:13.554 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:13.554 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:13.554 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:13.554 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:13.554 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:13.554 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:13.554 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:13.554 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:13.554 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:13.554 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:13.554 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:13.554 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:13.554 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 1 Processor -- start 2024-01-24T23:21:13.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 1 -- start start 2024-01-24T23:21:13.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 10 monclient: init 2024-01-24T23:21:13.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 1 Processor -- start 2024-01-24T23:21:13.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 1 -- start start 2024-01-24T23:21:13.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 10 monclient: init 2024-01-24T23:21:13.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:13.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 10 monclient: _add_conns ranks=[7,3,2,4,0,5,6,1,8] 2024-01-24T23:21:13.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 10 monclient(hunting): picked mon.noname-i con 0x7f69240a18a0 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:13.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 10 monclient(hunting): picked mon.noname-g con 0x7f69240a1c80 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:13.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 10 monclient(hunting): picked mon.noname-b con 0x7f6924139b20 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:13.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6924070bb0 con 0x7f6924139b20 2024-01-24T23:21:13.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f692406eac0 con 0x7f69240a1c80 2024-01-24T23:21:13.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f692409b830 con 0x7f69240a18a0 2024-01-24T23:21:13.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:13.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f6929a2e640 10 monclient(hunting): authenticate will time out at 952.458909s 2024-01-24T23:21:13.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f69237fe640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f6924139b20 0x7f69241cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:55394/0 (socket says 172.21.15.115:55394) 2024-01-24T23:21:13.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f69237fe640 1 -- 172.21.15.115:0/3112313285 learned_addr learned my addr 172.21.15.115:0/3112313285 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:13.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 1 -- 172.21.15.115:0/3112313285 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2944016725 0 0) 0x7f6924070bb0 con 0x7f6924139b20 2024-01-24T23:21:13.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 10 monclient(hunting): my global_id is 4331 2024-01-24T23:21:13.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 1 -- 172.21.15.115:0/3112313285 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6904003560 con 0x7f6924139b20 2024-01-24T23:21:13.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:13.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 10 monclient: _add_conns ranks=[5,0,6,3,1,7,4,2,8] 2024-01-24T23:21:13.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 10 monclient(hunting): picked mon.noname-h con 0x7fe4a009bc60 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:13.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 10 monclient(hunting): picked mon.noname-a con 0x7fe4a00a18a0 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:13.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 10 monclient(hunting): picked mon.noname-d con 0x7fe4a00a2390 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:13.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a00a34b0 con 0x7fe4a00a18a0 2024-01-24T23:21:13.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a009c640 con 0x7fe4a009bc60 2024-01-24T23:21:13.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a01214e0 con 0x7fe4a00a2390 2024-01-24T23:21:13.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:13.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a8413640 10 monclient(hunting): authenticate will time out at 952.459666s 2024-01-24T23:21:13.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 1 -- 172.21.15.115:0/3112313285 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 184849634 0 0) 0x7f692409b830 con 0x7f69240a18a0 2024-01-24T23:21:13.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 10 monclient(hunting): my global_id is 4399 2024-01-24T23:21:13.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 1 -- 172.21.15.115:0/3112313285 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6924070bb0 con 0x7f69240a18a0 2024-01-24T23:21:13.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a69b0640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fe4a00a2390 0x7fe4a01d3b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:39620/0 (socket says 172.21.15.115:39620) 2024-01-24T23:21:13.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7fe4a69b0640 1 -- 172.21.15.115:0/3996948492 learned_addr learned my addr 172.21.15.115:0/3996948492 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:13.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 1 -- 172.21.15.115:0/3112313285 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 94348912 0 0) 0x7f692406eac0 con 0x7f69240a1c80 2024-01-24T23:21:13.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 10 monclient(hunting): my global_id is 4368 2024-01-24T23:21:13.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 1 -- 172.21.15.115:0/3112313285 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f692409b830 con 0x7f69240a1c80 2024-01-24T23:21:13.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 1 -- 172.21.15.115:0/3112313285 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4119074735 0 0) 0x7f6904003560 con 0x7f6924139b20 2024-01-24T23:21:13.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 1 -- 172.21.15.115:0/3112313285 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f692406eac0 con 0x7f6924139b20 2024-01-24T23:21:13.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 1 -- 172.21.15.115:0/3112313285 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f690c001380 con 0x7f6924139b20 2024-01-24T23:21:13.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:21:13.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 10 monclient(hunting): dump: 2024-01-24T23:21:13.562 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.562 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.562 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.562 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.562 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.562 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.562 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.562 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.562 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.562 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.562 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.562 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.562 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.563 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.563 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.563 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.531+0000 7f691bfff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:13.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 1 -- 172.21.15.115:0/3996948492 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2292368814 0 0) 0x7fe4a01214e0 con 0x7fe4a00a2390 2024-01-24T23:21:13.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient(hunting): my global_id is 4398 2024-01-24T23:21:13.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 1 -- 172.21.15.115:0/3996948492 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe484003530 con 0x7fe4a00a2390 2024-01-24T23:21:13.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f691bfff640 1 -- 172.21.15.115:0/3112313285 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 766523304 0 0) 0x7f692406eac0 con 0x7f6924139b20 2024-01-24T23:21:13.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f691bfff640 1 -- 172.21.15.115:0/3112313285 >> v1:172.21.15.132:6792/0 conn(0x7f69240a18a0 legacy=0x7f692406dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f691bfff640 1 -- 172.21.15.115:0/3112313285 >> v1:172.21.15.132:6790/0 conn(0x7f69240a1c80 legacy=0x7f692406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f691bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:21:13.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f691bfff640 1 monclient: found mon.b 2024-01-24T23:21:13.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f691bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:13.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f691bfff640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:13.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f691bfff640 1 -- 172.21.15.115:0/3112313285 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6924072110 con 0x7f6924139b20 2024-01-24T23:21:13.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f691bfff640 10 monclient: _finish_auth 0 2024-01-24T23:21:13.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f691bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:21:13.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f691bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:13.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f6929a2e640 5 monclient: authenticate success, global_id 4331 2024-01-24T23:21:13.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f6929a2e640 10 monclient: _renew_subs 2024-01-24T23:21:13.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f6929a2e640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:13.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f6929a2e640 1 -- 172.21.15.115:0/3112313285 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69240708d0 con 0x7f6924139b20 2024-01-24T23:21:13.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f6929a2e640 10 monclient: _renew_subs 2024-01-24T23:21:13.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f6929a2e640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:13.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f6929a2e640 1 -- 172.21.15.115:0/3112313285 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6924138040 con 0x7f6924139b20 2024-01-24T23:21:13.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 1 -- 172.21.15.115:0/3996948492 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 435360277 0 0) 0x7fe4a00a34b0 con 0x7fe4a00a18a0 2024-01-24T23:21:13.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient(hunting): my global_id is 4446 2024-01-24T23:21:13.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 1 -- 172.21.15.115:0/3996948492 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4a01214e0 con 0x7fe4a00a18a0 2024-01-24T23:21:13.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 1 -- 172.21.15.115:0/3996948492 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1410710006 0 0) 0x7fe484003530 con 0x7fe4a00a2390 2024-01-24T23:21:13.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 1 -- 172.21.15.115:0/3996948492 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe4a00a34b0 con 0x7fe4a00a2390 2024-01-24T23:21:13.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 1 -- 172.21.15.115:0/3996948492 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe498003d00 con 0x7fe4a00a2390 2024-01-24T23:21:13.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:21:13.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient(hunting): dump: 2024-01-24T23:21:13.571 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.571 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.571 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.571 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.571 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.571 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.571 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.571 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.571 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.571 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.571 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.571 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.571 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.571 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.572 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.572 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:13.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 1 -- 172.21.15.115:0/3996948492 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2208185156 0 0) 0x7fe4a009c640 con 0x7fe4a009bc60 2024-01-24T23:21:13.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient(hunting): my global_id is 4379 2024-01-24T23:21:13.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 1 -- 172.21.15.115:0/3996948492 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe484003530 con 0x7fe4a009bc60 2024-01-24T23:21:13.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 1 -- 172.21.15.115:0/3996948492 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4975907 0 0) 0x7fe4a01214e0 con 0x7fe4a00a18a0 2024-01-24T23:21:13.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 1 -- 172.21.15.115:0/3996948492 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe4a009c640 con 0x7fe4a00a18a0 2024-01-24T23:21:13.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 1 -- 172.21.15.115:0/3996948492 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe490001920 con 0x7fe4a00a18a0 2024-01-24T23:21:13.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:13.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient(hunting): dump: 2024-01-24T23:21:13.573 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.573 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.573 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.573 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.573 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.573 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.573 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.573 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.573 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.573 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.573 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.573 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.573 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.573 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.573 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.574 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 1 -- 172.21.15.115:0/3996948492 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3685892188 0 0) 0x7fe4a00a34b0 con 0x7fe4a00a2390 2024-01-24T23:21:13.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 1 -- 172.21.15.115:0/3996948492 >> v1:172.21.15.132:6791/0 conn(0x7fe4a009bc60 legacy=0x7fe4a0133b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 1 -- 172.21.15.115:0/3996948492 >> v1:172.21.15.115:6789/0 conn(0x7fe4a00a18a0 legacy=0x7fe4a01d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:21:13.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 1 monclient: found mon.d 2024-01-24T23:21:13.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:13.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:13.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 1 -- 172.21.15.115:0/3996948492 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a009ca90 con 0x7fe4a00a2390 2024-01-24T23:21:13.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient: _finish_auth 0 2024-01-24T23:21:13.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:21:13.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:13.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe4a8413640 5 monclient: authenticate success, global_id 4398 2024-01-24T23:21:13.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe4a8413640 10 monclient: _renew_subs 2024-01-24T23:21:13.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe4a8413640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:13.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe4a8413640 1 -- 172.21.15.115:0/3996948492 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a0121200 con 0x7fe4a00a2390 2024-01-24T23:21:13.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 1 -- 172.21.15.115:0/3996948492 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe498001ac0 con 0x7fe4a00a2390 2024-01-24T23:21:13.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:13.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 1 -- 172.21.15.115:0/3996948492 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe4980042e0 con 0x7fe4a00a2390 2024-01-24T23:21:13.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f691bfff640 1 -- 172.21.15.115:0/3112313285 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f690c003b90 con 0x7f6924139b20 2024-01-24T23:21:13.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f691bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:13.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f691bfff640 1 -- 172.21.15.115:0/3112313285 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f690c0040c0 con 0x7f6924139b20 2024-01-24T23:21:13.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f691bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f69217fa640 4 set_mon_vals no callback set 2024-01-24T23:21:13.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f691bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:13.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f691bfff640 10 monclient: dump: 2024-01-24T23:21:13.576 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.576 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.576 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.576 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.576 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.576 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.576 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.576 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.576 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.576 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.576 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.576 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.577 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.577 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.577 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.577 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f6929a2e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:13.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f6929a2e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:13.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f6929a2e640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:13.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:13.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 10 monclient: dump: 2024-01-24T23:21:13.577 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.577 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.578 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.578 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.578 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.578 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.578 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.578 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.578 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.578 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.578 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.578 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.578 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.578 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.578 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.578 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe4a49ac640 4 set_mon_vals no callback set 2024-01-24T23:21:13.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe4a8413640 10 monclient: _renew_subs 2024-01-24T23:21:13.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe4a8413640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:13.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe4a8413640 1 -- 172.21.15.115:0/3996948492 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a0134230 con 0x7fe4a00a2390 2024-01-24T23:21:13.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f6929a2e640 1 -- 172.21.15.115:0/3112313285 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6908005090 con 0x7f6924139b20 2024-01-24T23:21:13.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 1 -- 172.21.15.115:0/3996948492 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fe498004440 con 0x7fe4a00a2390 2024-01-24T23:21:13.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe4a8413640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:13.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe4a8413640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:13.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe4a8413640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:13.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe4a8413640 1 -- 172.21.15.115:0/3996948492 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a0121040 con 0x7fe4a00a2390 2024-01-24T23:21:13.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f691bfff640 1 -- 172.21.15.115:0/3112313285 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f690c004220 con 0x7f6924139b20 2024-01-24T23:21:13.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f691bfff640 1 -- 172.21.15.115:0/3112313285 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f690c043650 con 0x7f6924139b20 2024-01-24T23:21:13.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f65ed1dd640 10 monclient: get_monmap_and_config 2024-01-24T23:21:13.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f65ed1dd640 10 monclient: build_initial_monmap 2024-01-24T23:21:13.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f65ed1dd640 10 monclient: monmap: 2024-01-24T23:21:13.581 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:13.581 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.581 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:13.542372+0000 2024-01-24T23:21:13.581 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:13.542372+0000 2024-01-24T23:21:13.581 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:13.581 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.581 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:13.581 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:13.581 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:13.581 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:13.581 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:13.582 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:13.582 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:13.582 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:13.582 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:13.582 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7f6922ffd640 10 monclient: get_auth_request con 0x7f6904037290 auth_method 0 2024-01-24T23:21:13.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe496ffd640 1 -- 172.21.15.115:0/3996948492 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7fe498043a30 con 0x7fe4a00a2390 2024-01-24T23:21:13.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.535+0000 7fe4a61af640 10 monclient: get_auth_request con 0x7fe484037940 auth_method 0 2024-01-24T23:21:13.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 1 Processor -- start 2024-01-24T23:21:13.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 1 -- start start 2024-01-24T23:21:13.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 10 monclient: init 2024-01-24T23:21:13.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:13.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 10 monclient: _add_conns ranks=[6,8,1,7,2,0,5,3,4] 2024-01-24T23:21:13.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 10 monclient(hunting): picked mon.noname-d con 0x7f65e8131810 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:13.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 10 monclient(hunting): picked mon.noname-e con 0x7f65e81311d0 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:13.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 10 monclient(hunting): picked mon.noname-f con 0x7f65e806d960 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:13.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e804ed70 con 0x7f65e806d960 2024-01-24T23:21:13.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e806e1f0 con 0x7f65e8131810 2024-01-24T23:21:13.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e809fef0 con 0x7f65e81311d0 2024-01-24T23:21:13.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:13.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 10 monclient(hunting): authenticate will time out at 952.464687s 2024-01-24T23:21:13.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e6d9d640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f65e8131810 0x7f65e80a4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:39630/0 (socket says 172.21.15.115:39630) 2024-01-24T23:21:13.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e6d9d640 1 -- 172.21.15.115:0/767799967 learned_addr learned my addr 172.21.15.115:0/767799967 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:13.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e5d9b640 1 -- 172.21.15.115:0/767799967 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1641803462 0 0) 0x7f65e804ed70 con 0x7f65e806d960 2024-01-24T23:21:13.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e5d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e5d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e5d9b640 10 monclient(hunting): my global_id is 4357 2024-01-24T23:21:13.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e5d9b640 1 -- 172.21.15.115:0/767799967 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65d0003530 con 0x7f65e806d960 2024-01-24T23:21:13.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e5d9b640 1 -- 172.21.15.115:0/767799967 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3786049178 0 0) 0x7f65d0003530 con 0x7f65e806d960 2024-01-24T23:21:13.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e5d9b640 1 -- 172.21.15.115:0/767799967 >> v1:172.21.15.115:6793/0 conn(0x7f65e81311d0 legacy=0x7f65e80a4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T23:21:13.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e5d9b640 1 -- 172.21.15.115:0/767799967 >> v1:172.21.15.115:6792/0 conn(0x7f65e8131810 legacy=0x7f65e80a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e5d9b640 10 monclient: _finish_hunting 0 2024-01-24T23:21:13.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e5d9b640 1 monclient: found mon.noname-f 2024-01-24T23:21:13.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:13.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e5d9b640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:21:13.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e5d9b640 1 -- 172.21.15.115:0/767799967 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e80a54b0 con 0x7f65e806d960 2024-01-24T23:21:13.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e5d9b640 10 monclient: _finish_auth 0 2024-01-24T23:21:13.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e5d9b640 10 monclient: _check_auth_tickets 2024-01-24T23:21:13.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:13.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 5 monclient: authenticate success, global_id 4357 2024-01-24T23:21:13.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:13.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e5d9b640 1 -- 172.21.15.115:0/767799967 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f65e0001b40 con 0x7f65e806d960 2024-01-24T23:21:13.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e5d9b640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:21:13.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e5d9b640 10 monclient: dump: 2024-01-24T23:21:13.588 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.588 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.588 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.588 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.588 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.588 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.588 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.588 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.588 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.588 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.588 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.588 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.588 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.588 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.589 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.589 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:13.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e5d9b640 1 -- 172.21.15.115:0/767799967 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f65e0001ea0 con 0x7f65e806d960 2024-01-24T23:21:13.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65e5d9b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:13.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:13.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 4 set_mon_vals no callback set 2024-01-24T23:21:13.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 10 monclient: shutdown 2024-01-24T23:21:13.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:13.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 1 -- 172.21.15.115:0/767799967 >> v1:172.21.15.132:6789/0 conn(0x7f65e806d960 legacy=0x7f65e813da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 1 -- 172.21.15.115:0/767799967 shutdown_connections 2024-01-24T23:21:13.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 1 -- 172.21.15.115:0/767799967 >> 172.21.15.115:0/767799967 conn(0x7f65e80a5d40 msgr2=0x7f65e80a2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:13.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 1 -- 172.21.15.115:0/767799967 shutdown_connections 2024-01-24T23:21:13.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f65ed1dd640 1 -- 172.21.15.115:0/767799967 wait complete. 2024-01-24T23:21:13.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f691bfff640 1 -- 172.21.15.115:0/3112313285 <== mon.2 v1:172.21.15.115:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f690c014810 con 0x7f6924139b20 2024-01-24T23:21:13.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f691bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:13.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.539+0000 7f691bfff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:13.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65ed1dd640 10 monclient: build_initial_monmap 2024-01-24T23:21:13.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65ed1dd640 10 monclient: monmap: 2024-01-24T23:21:13.592 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:13.592 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:13.592 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:13.592 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:13.592 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:13.592 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:13.592 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:13.592 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:13.592 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:13.592 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:13.592 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:13.592 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:13.592 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:13.593 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:13.593 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:13.593 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65ed1dd640 1 Processor -- start 2024-01-24T23:21:13.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65ed1dd640 1 -- start start 2024-01-24T23:21:13.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65ed1dd640 10 monclient: init 2024-01-24T23:21:13.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65ed1dd640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:13.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65ed1dd640 10 monclient: _add_conns ranks=[8,2,0,5,3,7,4,1,6] 2024-01-24T23:21:13.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65ed1dd640 10 monclient(hunting): picked mon.noname-e con 0x7f65e806d960 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:13.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65ed1dd640 10 monclient(hunting): picked mon.noname-b con 0x7f65e81311d0 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:13.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65ed1dd640 10 monclient(hunting): picked mon.noname-a con 0x7f65e8131810 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:13.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65ed1dd640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e81241a0 con 0x7f65e8131810 2024-01-24T23:21:13.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65ed1dd640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e8124760 con 0x7f65e81311d0 2024-01-24T23:21:13.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65ed1dd640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e806de30 con 0x7f65e806d960 2024-01-24T23:21:13.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65ed1dd640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:13.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65ed1dd640 10 monclient(hunting): authenticate will time out at 952.469557s 2024-01-24T23:21:13.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65e759e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f65e8131810 0x7f65e81d3f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58168/0 (socket says 172.21.15.115:58168) 2024-01-24T23:21:13.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65e759e640 1 -- 172.21.15.115:0/4155724971 learned_addr learned my addr 172.21.15.115:0/4155724971 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:13.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 1 -- 172.21.15.115:0/4155724971 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1918712360 0 0) 0x7f65e81241a0 con 0x7f65e8131810 2024-01-24T23:21:13.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 10 monclient(hunting): my global_id is 4455 2024-01-24T23:21:13.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 1 -- 172.21.15.115:0/4155724971 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65d4003530 con 0x7f65e8131810 2024-01-24T23:21:13.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 1 -- 172.21.15.115:0/4155724971 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 683978256 0 0) 0x7f65e8124760 con 0x7f65e81311d0 2024-01-24T23:21:13.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 10 monclient(hunting): my global_id is 4340 2024-01-24T23:21:13.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 1 -- 172.21.15.115:0/4155724971 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65e81241a0 con 0x7f65e81311d0 2024-01-24T23:21:13.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 1 -- 172.21.15.115:0/4155724971 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4128132717 0 0) 0x7f65d4003530 con 0x7f65e8131810 2024-01-24T23:21:13.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 1 -- 172.21.15.115:0/4155724971 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f65e8124760 con 0x7f65e8131810 2024-01-24T23:21:13.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 1 -- 172.21.15.115:0/4155724971 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f65e00042f0 con 0x7f65e8131810 2024-01-24T23:21:13.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:21:13.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 10 monclient(hunting): dump: 2024-01-24T23:21:13.595 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.595 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.595 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.595 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.595 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.595 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.595 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.595 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.596 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.596 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.596 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.596 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.596 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.596 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.596 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.596 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:13.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 1 -- 172.21.15.115:0/4155724971 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2729227061 0 0) 0x7f65e81241a0 con 0x7f65e81311d0 2024-01-24T23:21:13.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 1 -- 172.21.15.115:0/4155724971 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f65d4003530 con 0x7f65e81311d0 2024-01-24T23:21:13.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 1 -- 172.21.15.115:0/4155724971 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f65dc0019a0 con 0x7f65e81311d0 2024-01-24T23:21:13.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:13.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 10 monclient(hunting): dump: 2024-01-24T23:21:13.597 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.597 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.597 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.597 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.597 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.597 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.597 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.597 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.597 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.597 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.597 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.597 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.597 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.597 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.597 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.598 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 1 -- 172.21.15.115:0/4155724971 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 278870488 0 0) 0x7f65e8124760 con 0x7f65e8131810 2024-01-24T23:21:13.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 1 -- 172.21.15.115:0/4155724971 >> v1:172.21.15.115:6793/0 conn(0x7f65e806d960 legacy=0x7f65e81cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 1 -- 172.21.15.115:0/4155724971 >> v1:172.21.15.115:6790/0 conn(0x7f65e81311d0 legacy=0x7f65e81d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:21:13.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 1 monclient: found mon.a 2024-01-24T23:21:13.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:13.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:21:13.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 1 -- 172.21.15.115:0/4155724971 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e8122470 con 0x7f65e8131810 2024-01-24T23:21:13.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 10 monclient: _finish_auth 0 2024-01-24T23:21:13.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:21:13.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:13.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65ed1dd640 5 monclient: authenticate success, global_id 4455 2024-01-24T23:21:13.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65ed1dd640 10 monclient: _renew_subs 2024-01-24T23:21:13.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65ed1dd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:21:13.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65ed1dd640 1 -- 172.21.15.115:0/4155724971 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e8123ec0 con 0x7f65e8131810 2024-01-24T23:21:13.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65ed1dd640 10 monclient: _renew_subs 2024-01-24T23:21:13.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65ed1dd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:21:13.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65ed1dd640 1 -- 172.21.15.115:0/4155724971 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e8125190 con 0x7f65e8131810 2024-01-24T23:21:13.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 1 -- 172.21.15.115:0/4155724971 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f65e0002010 con 0x7f65e8131810 2024-01-24T23:21:13.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:13.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 1 -- 172.21.15.115:0/4155724971 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f65e0003cc0 con 0x7f65e8131810 2024-01-24T23:21:13.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:13.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65cf7fe640 10 monclient: dump: 2024-01-24T23:21:13.599 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.599 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.599 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.599 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.599 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.600 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.600 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.600 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.600 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.600 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.600 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.600 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.600 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.600 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.600 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.600 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7f65e559a640 4 set_mon_vals no callback set 2024-01-24T23:21:13.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7fe496ffd640 1 -- 172.21.15.115:0/3996948492 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fe498004bf0 con 0x7fe4a00a2390 2024-01-24T23:21:13.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7fe496ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:13.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.543+0000 7fe496ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:13.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.547+0000 7f65ed1dd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:13.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.547+0000 7f65ed1dd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:13.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.547+0000 7f65ed1dd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:21:13.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.547+0000 7f65ed1dd640 1 -- 172.21.15.115:0/4155724971 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c8005090 con 0x7f65e8131810 2024-01-24T23:21:13.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.547+0000 7f65cf7fe640 1 -- 172.21.15.115:0/4155724971 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f65e00047c0 con 0x7f65e8131810 2024-01-24T23:21:13.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.547+0000 7f65e6d9d640 10 monclient: get_auth_request con 0x7f65d4037770 auth_method 0 2024-01-24T23:21:13.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.551+0000 7f65cf7fe640 1 -- 172.21.15.115:0/4155724971 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f65e00159e0 con 0x7f65e8131810 2024-01-24T23:21:13.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.551+0000 7f65cf7fe640 1 -- 172.21.15.115:0/4155724971 <== mon.0 v1:172.21.15.115:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f65e0014ca0 con 0x7f65e8131810 2024-01-24T23:21:13.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.551+0000 7f65cf7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:13.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.551+0000 7f65cf7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:13.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.683+0000 7fe2967fc640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2024-01-24T23:21:13.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.683+0000 7fe2967fc640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-01-24T23:21:13.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.683+0000 7fe2967fc640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:21:13.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.683+0000 7fe2967fc640 1 -- 172.21.15.115:0/4222565922 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe288005340 con 0x7fe2b0133fd0 2024-01-24T23:21:13.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.687+0000 7fe2acff9640 1 -- 172.21.15.115:0/4222565922 <== mon.1 v1:172.21.15.132:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (unknown 832126871 0 847495621) 0x7fe2a4014570 con 0x7fe2b0133fd0 2024-01-24T23:21:13.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.687+0000 7fe2acff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-01-24T23:21:13.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.687+0000 7fe2acff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:13.690 INFO:teuthology.orchestra.run.smithi115.stdout:47244640259 2024-01-24T23:21:13.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.687+0000 7fe2967fc640 1 -- 172.21.15.115:0/4222565922 >> v1:172.21.15.132:6812/1835401922 conn(0x7fe29c037d20 legacy=0x7fe29c03a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.687+0000 7fe2967fc640 10 monclient: shutdown 2024-01-24T23:21:13.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.687+0000 7fe2967fc640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:13.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.687+0000 7fe2967fc640 1 -- 172.21.15.115:0/4222565922 >> v1:172.21.15.132:6789/0 conn(0x7fe2b0133fd0 legacy=0x7fe2b01d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.687+0000 7fe2967fc640 1 -- 172.21.15.115:0/4222565922 shutdown_connections 2024-01-24T23:21:13.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.687+0000 7fe2967fc640 1 -- 172.21.15.115:0/4222565922 >> 172.21.15.115:0/4222565922 conn(0x7fe2b00662f0 msgr2=0x7fe2b01be890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:13.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.687+0000 7fe2967fc640 1 -- 172.21.15.115:0/4222565922 shutdown_connections 2024-01-24T23:21:13.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.687+0000 7fe2967fc640 1 -- 172.21.15.115:0/4222565922 wait complete. 2024-01-24T23:21:13.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.691+0000 7f8c883f2640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2024-01-24T23:21:13.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.691+0000 7f8c883f2640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-01-24T23:21:13.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.691+0000 7f8c883f2640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:13.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.691+0000 7f8c883f2640 1 -- 172.21.15.115:0/2780788313 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8c800a21c0 con 0x7f8c801343b0 2024-01-24T23:21:13.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.691+0000 7f8c76ffd640 1 -- 172.21.15.115:0/2780788313 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (unknown 92182286 0 555833778) 0x7f8c800a21c0 con 0x7f8c801343b0 2024-01-24T23:21:13.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.691+0000 7f8c76ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-01-24T23:21:13.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.691+0000 7f8c76ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:13.694 INFO:teuthology.orchestra.run.smithi115.stdout:47244640258 2024-01-24T23:21:13.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.691+0000 7f8c883f2640 1 -- 172.21.15.115:0/2780788313 >> v1:172.21.15.132:6812/1835401922 conn(0x7f8c640371a0 legacy=0x7f8c64039660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.691+0000 7f8c883f2640 10 monclient: shutdown 2024-01-24T23:21:13.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.691+0000 7f8c883f2640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:13.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.691+0000 7f8c883f2640 1 -- 172.21.15.115:0/2780788313 >> v1:172.21.15.115:6791/0 conn(0x7f8c801343b0 legacy=0x7f8c801d0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.691+0000 7f8c883f2640 1 -- 172.21.15.115:0/2780788313 shutdown_connections 2024-01-24T23:21:13.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.691+0000 7f8c883f2640 1 -- 172.21.15.115:0/2780788313 >> 172.21.15.115:0/2780788313 conn(0x7f8c800662f0 msgr2=0x7f8c801be870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:13.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.691+0000 7f8c883f2640 1 -- 172.21.15.115:0/2780788313 shutdown_connections 2024-01-24T23:21:13.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.691+0000 7f8c883f2640 1 -- 172.21.15.115:0/2780788313 wait complete. 2024-01-24T23:21:13.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.695+0000 7f6929a2e640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2024-01-24T23:21:13.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.695+0000 7f6929a2e640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-01-24T23:21:13.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.695+0000 7f6929a2e640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:13.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.695+0000 7f6929a2e640 1 -- 172.21.15.115:0/3112313285 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6908005340 con 0x7f6924139b20 2024-01-24T23:21:13.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.695+0000 7f691bfff640 1 -- 172.21.15.115:0/3112313285 <== mon.2 v1:172.21.15.115:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (unknown 1823589948 0 555833778) 0x7f690c014810 con 0x7f6924139b20 2024-01-24T23:21:13.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.695+0000 7f691bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-01-24T23:21:13.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.695+0000 7f691bfff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:13.700 INFO:teuthology.orchestra.run.smithi115.stdout:47244640258 2024-01-24T23:21:13.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.699+0000 7f6929a2e640 1 -- 172.21.15.115:0/3112313285 >> v1:172.21.15.132:6812/1835401922 conn(0x7f6904037290 legacy=0x7f6904039750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.699+0000 7f6929a2e640 10 monclient: shutdown 2024-01-24T23:21:13.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.699+0000 7f6929a2e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:13.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.699+0000 7f6929a2e640 1 -- 172.21.15.115:0/3112313285 >> v1:172.21.15.115:6790/0 conn(0x7f6924139b20 legacy=0x7f69241cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.699+0000 7f6929a2e640 1 -- 172.21.15.115:0/3112313285 shutdown_connections 2024-01-24T23:21:13.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.699+0000 7f6929a2e640 1 -- 172.21.15.115:0/3112313285 >> 172.21.15.115:0/3112313285 conn(0x7f69240a6490 msgr2=0x7f6924134120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:13.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.699+0000 7f6929a2e640 1 -- 172.21.15.115:0/3112313285 shutdown_connections 2024-01-24T23:21:13.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.699+0000 7f6929a2e640 1 -- 172.21.15.115:0/3112313285 wait complete. 2024-01-24T23:21:13.715 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640258 got 47244640258 for osd.2 2024-01-24T23:21:13.716 DEBUG:teuthology.parallel:result is None 2024-01-24T23:21:13.720 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640259 for osd.1 2024-01-24T23:21:13.720 DEBUG:teuthology.parallel:result is None 2024-01-24T23:21:13.721 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640258 got 47244640258 for osd.4 2024-01-24T23:21:13.721 DEBUG:teuthology.parallel:result is None 2024-01-24T23:21:13.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.767+0000 7fe4a8413640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2024-01-24T23:21:13.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.767+0000 7fe4a8413640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-01-24T23:21:13.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.767+0000 7fe4a8413640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:13.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.767+0000 7fe4a8413640 1 -- 172.21.15.115:0/3996948492 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe4a00a1c80 con 0x7fe4a00a2390 2024-01-24T23:21:13.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.767+0000 7fe496ffd640 1 -- 172.21.15.115:0/3996948492 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (unknown 2131975755 0 555833778) 0x7fe498014b10 con 0x7fe4a00a2390 2024-01-24T23:21:13.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.767+0000 7fe496ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-01-24T23:21:13.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.767+0000 7fe496ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:13.773 INFO:teuthology.orchestra.run.smithi115.stdout:47244640258 2024-01-24T23:21:13.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.771+0000 7fe4a8413640 1 -- 172.21.15.115:0/3996948492 >> v1:172.21.15.132:6812/1835401922 conn(0x7fe484037940 legacy=0x7fe484039e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.771+0000 7fe4a8413640 10 monclient: shutdown 2024-01-24T23:21:13.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.771+0000 7fe4a8413640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:13.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.771+0000 7fe4a8413640 1 -- 172.21.15.115:0/3996948492 >> v1:172.21.15.115:6792/0 conn(0x7fe4a00a2390 legacy=0x7fe4a01d3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.771+0000 7fe4a8413640 1 -- 172.21.15.115:0/3996948492 shutdown_connections 2024-01-24T23:21:13.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.771+0000 7fe4a8413640 1 -- 172.21.15.115:0/3996948492 >> 172.21.15.115:0/3996948492 conn(0x7fe4a00a6490 msgr2=0x7fe4a0074860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:13.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.771+0000 7fe4a8413640 1 -- 172.21.15.115:0/3996948492 shutdown_connections 2024-01-24T23:21:13.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.771+0000 7fe4a8413640 1 -- 172.21.15.115:0/3996948492 wait complete. 2024-01-24T23:21:13.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.779+0000 7f65ed1dd640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2024-01-24T23:21:13.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.779+0000 7f65ed1dd640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-01-24T23:21:13.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.779+0000 7f65ed1dd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:21:13.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.779+0000 7f65ed1dd640 1 -- 172.21.15.115:0/4155724971 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f65c8005340 con 0x7f65e8131810 2024-01-24T23:21:13.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.779+0000 7f65cf7fe640 1 -- 172.21.15.115:0/4155724971 <== mon.0 v1:172.21.15.115:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (unknown 383520633 0 1444633209) 0x7f65e00152e0 con 0x7f65e8131810 2024-01-24T23:21:13.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.779+0000 7f65cf7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-01-24T23:21:13.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.779+0000 7f65cf7fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:13.785 INFO:teuthology.orchestra.run.smithi115.stdout:47244640260 2024-01-24T23:21:13.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.783+0000 7f65ed1dd640 1 -- 172.21.15.115:0/4155724971 >> v1:172.21.15.132:6812/1835401922 conn(0x7f65d4037770 legacy=0x7f65d4039c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.783+0000 7f65ed1dd640 10 monclient: shutdown 2024-01-24T23:21:13.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.783+0000 7f65ed1dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:13.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.783+0000 7f65ed1dd640 1 -- 172.21.15.115:0/4155724971 >> v1:172.21.15.115:6789/0 conn(0x7f65e8131810 legacy=0x7f65e81d3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.783+0000 7f65ed1dd640 1 -- 172.21.15.115:0/4155724971 shutdown_connections 2024-01-24T23:21:13.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.783+0000 7f65ed1dd640 1 -- 172.21.15.115:0/4155724971 >> 172.21.15.115:0/4155724971 conn(0x7f65e80a5d40 msgr2=0x7f65e8130940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:13.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.783+0000 7f65ed1dd640 1 -- 172.21.15.115:0/4155724971 shutdown_connections 2024-01-24T23:21:13.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.783+0000 7f65ed1dd640 1 -- 172.21.15.115:0/4155724971 wait complete. 2024-01-24T23:21:13.789 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640258 got 47244640258 for osd.5 2024-01-24T23:21:13.789 DEBUG:teuthology.parallel:result is None 2024-01-24T23:21:13.801 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640260 for osd.3 2024-01-24T23:21:13.801 DEBUG:teuthology.parallel:result is None 2024-01-24T23:21:13.801 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-01-24T23:21:13.801 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T23:21:13.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9747409640 10 monclient: get_monmap_and_config 2024-01-24T23:21:13.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9747409640 10 monclient: build_initial_monmap 2024-01-24T23:21:13.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9747409640 10 monclient: monmap: 2024-01-24T23:21:13.951 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:13.951 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.951 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:13.952452+0000 2024-01-24T23:21:13.951 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:13.952452+0000 2024-01-24T23:21:13.951 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:13.951 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.951 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:13.951 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:13.951 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:13.951 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:13.951 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:13.951 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:13.952 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:13.952 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:13.952 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:13.952 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9747409640 1 Processor -- start 2024-01-24T23:21:13.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9747409640 1 -- start start 2024-01-24T23:21:13.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9747409640 10 monclient: init 2024-01-24T23:21:13.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9747409640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:13.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9747409640 10 monclient: _add_conns ranks=[8,2,7,5,3,6,4,0,1] 2024-01-24T23:21:13.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9747409640 10 monclient(hunting): picked mon.noname-e con 0x7f97400a0880 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:13.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9747409640 10 monclient(hunting): picked mon.noname-b con 0x7f974009f1c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:13.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9747409640 10 monclient(hunting): picked mon.noname-i con 0x7f974013d490 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:13.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9747409640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f974004ed70 con 0x7f974009f1c0 2024-01-24T23:21:13.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9747409640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97400a3870 con 0x7f974013d490 2024-01-24T23:21:13.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9747409640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97400a32e0 con 0x7f97400a0880 2024-01-24T23:21:13.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9747409640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:13.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9747409640 10 monclient(hunting): authenticate will time out at 952.874924s 2024-01-24T23:21:13.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f97451a5640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f97400a0880 0x7f97400a0c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:48442/0 (socket says 172.21.15.115:48442) 2024-01-24T23:21:13.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f97451a5640 1 -- 172.21.15.115:0/1191943228 learned_addr learned my addr 172.21.15.115:0/1191943228 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:13.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 1 -- 172.21.15.115:0/1191943228 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3526811239 0 0) 0x7f974004ed70 con 0x7f974009f1c0 2024-01-24T23:21:13.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 10 monclient(hunting): my global_id is 4349 2024-01-24T23:21:13.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 1 -- 172.21.15.115:0/1191943228 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f972c003530 con 0x7f974009f1c0 2024-01-24T23:21:13.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 1 -- 172.21.15.115:0/1191943228 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 484229353 0 0) 0x7f97400a32e0 con 0x7f97400a0880 2024-01-24T23:21:13.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 10 monclient(hunting): my global_id is 74312 2024-01-24T23:21:13.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 1 -- 172.21.15.115:0/1191943228 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f974004ed70 con 0x7f97400a0880 2024-01-24T23:21:13.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 1 -- 172.21.15.115:0/1191943228 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2778349295 0 0) 0x7f972c003530 con 0x7f974009f1c0 2024-01-24T23:21:13.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 1 -- 172.21.15.115:0/1191943228 >> v1:172.21.15.115:6793/0 conn(0x7f97400a0880 legacy=0x7f97400a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 1 -- 172.21.15.115:0/1191943228 >> v1:172.21.15.132:6792/0 conn(0x7f974013d490 legacy=0x7f974013f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 10 monclient: _finish_hunting 0 2024-01-24T23:21:13.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 1 monclient: found mon.noname-b 2024-01-24T23:21:13.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:13.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:21:13.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 1 -- 172.21.15.115:0/1191943228 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974006c220 con 0x7f974009f1c0 2024-01-24T23:21:13.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 10 monclient: _finish_auth 0 2024-01-24T23:21:13.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 10 monclient: _check_auth_tickets 2024-01-24T23:21:13.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:13.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 1 -- 172.21.15.115:0/1191943228 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f97300017c0 con 0x7f974009f1c0 2024-01-24T23:21:13.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9747409640 5 monclient: authenticate success, global_id 4349 2024-01-24T23:21:13.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9747409640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:13.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:21:13.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 10 monclient: dump: 2024-01-24T23:21:13.961 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.961 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.961 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.961 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.962 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.962 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.962 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.962 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.962 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.962 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.962 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.962 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.962 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.962 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.962 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.962 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9747409640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:13.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 1 -- 172.21.15.115:0/1191943228 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9730001a60 con 0x7f974009f1c0 2024-01-24T23:21:13.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:13.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9747409640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:13.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9747409640 4 set_mon_vals no callback set 2024-01-24T23:21:13.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 1 -- 172.21.15.115:0/1191943228 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9730003960 con 0x7f974009f1c0 2024-01-24T23:21:13.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:13.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.947+0000 7f9737fff640 10 monclient: dump: 2024-01-24T23:21:13.963 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.963 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.963 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.963 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.963 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.963 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.963 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.963 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.963 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.963 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.963 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.963 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.963 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 10 monclient: shutdown 2024-01-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 1 -- 172.21.15.115:0/1191943228 >> v1:172.21.15.115:6790/0 conn(0x7f974009f1c0 legacy=0x7f974013bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 1 -- 172.21.15.115:0/1191943228 shutdown_connections 2024-01-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 1 -- 172.21.15.115:0/1191943228 >> 172.21.15.115:0/1191943228 conn(0x7f97400666f0 msgr2=0x7f9740066b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 1 -- 172.21.15.115:0/1191943228 shutdown_connections 2024-01-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 1 -- 172.21.15.115:0/1191943228 wait complete. 2024-01-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 10 monclient: build_initial_monmap 2024-01-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 10 monclient: monmap: 2024-01-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:13.964 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:13.965 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:13.965 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:13.965 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:13.965 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:13.965 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:13.965 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:13.965 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:13.965 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:13.965 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:13.965 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:13.965 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:13.965 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:13.965 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:13.965 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 1 Processor -- start 2024-01-24T23:21:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 1 -- start start 2024-01-24T23:21:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 10 monclient: init 2024-01-24T23:21:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 10 monclient: _add_conns ranks=[2,4,8,5,3,1,0,7,6] 2024-01-24T23:21:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 10 monclient(hunting): picked mon.noname-b con 0x7f974009f1c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 10 monclient(hunting): picked mon.noname-c con 0x7f97400a0880 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 10 monclient(hunting): picked mon.noname-e con 0x7f974013d490 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9740071510 con 0x7f974009f1c0 2024-01-24T23:21:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f974006f1b0 con 0x7f97400a0880 2024-01-24T23:21:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9740071230 con 0x7f974013d490 2024-01-24T23:21:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 10 monclient(hunting): authenticate will time out at 952.877673s 2024-01-24T23:21:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f97451a5640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f974009f1c0 0x7f97401cb8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:55438/0 (socket says 172.21.15.115:55438) 2024-01-24T23:21:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f97451a5640 1 -- 172.21.15.115:0/1211850759 learned_addr learned my addr 172.21.15.115:0/1211850759 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147243591 0 0) 0x7f974006f1b0 con 0x7f97400a0880 2024-01-24T23:21:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient(hunting): my global_id is 74380 2024-01-24T23:21:13.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9724003560 con 0x7f97400a0880 2024-01-24T23:21:13.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2602540453 0 0) 0x7f9740071230 con 0x7f974013d490 2024-01-24T23:21:13.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient(hunting): my global_id is 74321 2024-01-24T23:21:13.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f974006f1b0 con 0x7f974013d490 2024-01-24T23:21:13.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3685867870 0 0) 0x7f9740071510 con 0x7f974009f1c0 2024-01-24T23:21:13.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:21:13.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:13.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:13.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient(hunting): my global_id is 4358 2024-01-24T23:21:13.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9740071230 con 0x7f974009f1c0 2024-01-24T23:21:13.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1247083471 0 0) 0x7f9740071230 con 0x7f974009f1c0 2024-01-24T23:21:13.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9740071510 con 0x7f974009f1c0 2024-01-24T23:21:13.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f973c001a80 con 0x7f974009f1c0 2024-01-24T23:21:13.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:21:13.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient(hunting): dump: 2024-01-24T23:21:13.968 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.968 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.968 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.968 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.968 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.968 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.968 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.968 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.969 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.969 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.969 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.969 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.969 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.969 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.969 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.969 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:13.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1369059683 0 0) 0x7f9724003560 con 0x7f97400a0880 2024-01-24T23:21:13.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9740071230 con 0x7f97400a0880 2024-01-24T23:21:13.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2725858107 0 0) 0x7f974006f1b0 con 0x7f974013d490 2024-01-24T23:21:13.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9724003560 con 0x7f974013d490 2024-01-24T23:21:13.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9730003940 con 0x7f97400a0880 2024-01-24T23:21:13.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:13.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient(hunting): dump: 2024-01-24T23:21:13.970 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.970 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.970 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.970 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.970 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.970 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.970 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.970 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.970 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.970 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.970 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.970 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.970 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.971 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.971 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.971 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9738001c00 con 0x7f974013d490 2024-01-24T23:21:13.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:21:13.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient(hunting): dump: 2024-01-24T23:21:13.971 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.971 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.971 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.971 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.971 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.971 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.971 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.971 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.971 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.972 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.972 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.972 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.972 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.972 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.972 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.972 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 866717629 0 0) 0x7f9740071510 con 0x7f974009f1c0 2024-01-24T23:21:13.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 >> v1:172.21.15.115:6793/0 conn(0x7f974013d490 legacy=0x7f97401d3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 >> v1:172.21.15.115:6791/0 conn(0x7f97400a0880 legacy=0x7f97401d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:13.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:21:13.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 1 monclient: found mon.b 2024-01-24T23:21:13.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:13.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:13.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974006b5c0 con 0x7f974009f1c0 2024-01-24T23:21:13.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient: _finish_auth 0 2024-01-24T23:21:13.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:21:13.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:13.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 5 monclient: authenticate success, global_id 4358 2024-01-24T23:21:13.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 10 monclient: _renew_subs 2024-01-24T23:21:13.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:13.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 1 -- 172.21.15.115:0/1211850759 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9740072030 con 0x7f974009f1c0 2024-01-24T23:21:13.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 10 monclient: _renew_subs 2024-01-24T23:21:13.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:13.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 1 -- 172.21.15.115:0/1211850759 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f974006fa90 con 0x7f974009f1c0 2024-01-24T23:21:13.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f973c001be0 con 0x7f974009f1c0 2024-01-24T23:21:13.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:13.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f973c004de0 con 0x7f974009f1c0 2024-01-24T23:21:13.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:13.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f97377fe640 4 set_mon_vals no callback set 2024-01-24T23:21:13.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:13.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 10 monclient: dump: 2024-01-24T23:21:13.974 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:13.974 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:13.974 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.974 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:13.974 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:13.974 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:13.974 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:13.974 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:13.974 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:13.974 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:13.974 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:13.974 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:13.974 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:13.974 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:13.974 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:13.974 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:13.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:13.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:13.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:13.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9747409640 1 -- 172.21.15.115:0/1211850759 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9720005090 con 0x7f974009f1c0 2024-01-24T23:21:13.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f973c00f860 con 0x7f974009f1c0 2024-01-24T23:21:13.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.951+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f973c043650 con 0x7f974009f1c0 2024-01-24T23:21:13.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.955+0000 7f97449a4640 10 monclient: get_auth_request con 0x7f9724037d40 auth_method 0 2024-01-24T23:21:13.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.955+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 <== mon.2 v1:172.21.15.115:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f973c014730 con 0x7f974009f1c0 2024-01-24T23:21:13.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.955+0000 7f9735ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:13.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:13.955+0000 7f9735ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:14.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.099+0000 7f9747409640 1 -- 172.21.15.115:0/1211850759 --> v1:172.21.15.132:6812/1835401922 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9720000ba0 con 0x7f9724037d40 2024-01-24T23:21:14.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.103+0000 7f9735ffb640 1 -- 172.21.15.115:0/1211850759 <== mgr.4113 v1:172.21.15.132:6812/1835401922 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+40608 (unknown 2965378022 0 824952323) 0x7f9720000ba0 con 0x7f9724037d40 2024-01-24T23:21:14.106 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:21:14.106 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-01-24T23:21:14.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.103+0000 7f9747409640 1 -- 172.21.15.115:0/1211850759 >> v1:172.21.15.132:6812/1835401922 conn(0x7f9724037d40 legacy=0x7f972403a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:14.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.103+0000 7f9747409640 10 monclient: shutdown 2024-01-24T23:21:14.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.103+0000 7f9747409640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:14.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.103+0000 7f9747409640 1 -- 172.21.15.115:0/1211850759 >> v1:172.21.15.115:6790/0 conn(0x7f974009f1c0 legacy=0x7f97401cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:14.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.103+0000 7f9747409640 1 -- 172.21.15.115:0/1211850759 shutdown_connections 2024-01-24T23:21:14.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.103+0000 7f9747409640 1 -- 172.21.15.115:0/1211850759 >> 172.21.15.115:0/1211850759 conn(0x7f97400666f0 msgr2=0x7f974013c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:14.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.103+0000 7f9747409640 1 -- 172.21.15.115:0/1211850759 shutdown_connections 2024-01-24T23:21:14.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.103+0000 7f9747409640 1 -- 172.21.15.115:0/1211850759 wait complete. 2024-01-24T23:21:14.122 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-01-24T23:21:13.831041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163060,"kb_used_data":884,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067980,"statfs":{"total":579820584960,"available":579653611520,"internally_reserved":0,"allocated":905216,"data_stored":1224326,"data_compressed":57630,"data_compressed_allocated":77824,"data_compressed_original":962560,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000558"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.012747+0000","last_change":"2024-01-24T23:21:08.941679+0000","last_active":"2024-01-24T23:21:10.012747+0000","last_peered":"2024-01-24T23:21:10.012747+0000","last_clean":"2024-01-24T23:21:10.012747+0000","last_became_active":"2024-01-24T23:21:08.941408+0000","last_became_peered":"2024-01-24T23:21:08.941408+0000","last_unstale":"2024-01-24T23:21:10.012747+0000","last_undegraded":"2024-01-24T23:21:10.012747+0000","last_fullsized":"2024-01-24T23:21:10.012747+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T10:59:01.658977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.012767+0000","last_change":"2024-01-24T23:21:08.949886+0000","last_active":"2024-01-24T23:21:10.012767+0000","last_peered":"2024-01-24T23:21:10.012767+0000","last_clean":"2024-01-24T23:21:10.012767+0000","last_became_active":"2024-01-24T23:21:08.949572+0000","last_became_peered":"2024-01-24T23:21:08.949572+0000","last_unstale":"2024-01-24T23:21:10.012767+0000","last_undegraded":"2024-01-24T23:21:10.012767+0000","last_fullsized":"2024-01-24T23:21:10.012767+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T10:40:08.936213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.011232+0000","last_change":"2024-01-24T23:21:08.941915+0000","last_active":"2024-01-24T23:21:10.011232+0000","last_peered":"2024-01-24T23:21:10.011232+0000","last_clean":"2024-01-24T23:21:10.011232+0000","last_became_active":"2024-01-24T23:21:08.941537+0000","last_became_peered":"2024-01-24T23:21:08.941537+0000","last_unstale":"2024-01-24T23:21:10.011232+0000","last_undegraded":"2024-01-24T23:21:10.011232+0000","last_fullsized":"2024-01-24T23:21:10.011232+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T03:58:22.010129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.017040+0000","last_change":"2024-01-24T23:21:08.943234+0000","last_active":"2024-01-24T23:21:10.017040+0000","last_peered":"2024-01-24T23:21:10.017040+0000","last_clean":"2024-01-24T23:21:10.017040+0000","last_became_active":"2024-01-24T23:21:08.942798+0000","last_became_peered":"2024-01-24T23:21:08.942798+0000","last_unstale":"2024-01-24T23:21:10.017040+0000","last_undegraded":"2024-01-24T23:21:10.017040+0000","last_fullsized":"2024-01-24T23:21:10.017040+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T01:56:21.645104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":161,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.952984+0000","last_change":"2024-01-24T23:21:08.940580+0000","last_active":"2024-01-24T23:21:10.952984+0000","last_peered":"2024-01-24T23:21:10.952984+0000","last_clean":"2024-01-24T23:21:10.952984+0000","last_became_active":"2024-01-24T23:21:08.940211+0000","last_became_peered":"2024-01-24T23:21:08.940211+0000","last_unstale":"2024-01-24T23:21:10.952984+0000","last_undegraded":"2024-01-24T23:21:10.952984+0000","last_fullsized":"2024-01-24T23:21:10.952984+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T05:41:59.190049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.686443+0000","last_change":"2024-01-24T23:21:08.951610+0000","last_active":"2024-01-24T23:21:10.686443+0000","last_peered":"2024-01-24T23:21:10.686443+0000","last_clean":"2024-01-24T23:21:10.686443+0000","last_became_active":"2024-01-24T23:21:08.950717+0000","last_became_peered":"2024-01-24T23:21:08.950717+0000","last_unstale":"2024-01-24T23:21:10.686443+0000","last_undegraded":"2024-01-24T23:21:10.686443+0000","last_fullsized":"2024-01-24T23:21:10.686443+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T00:26:13.689261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.011863+0000","last_change":"2024-01-24T23:21:08.943904+0000","last_active":"2024-01-24T23:21:10.011863+0000","last_peered":"2024-01-24T23:21:10.011863+0000","last_clean":"2024-01-24T23:21:10.011863+0000","last_became_active":"2024-01-24T23:21:08.943567+0000","last_became_peered":"2024-01-24T23:21:08.943567+0000","last_unstale":"2024-01-24T23:21:10.011863+0000","last_undegraded":"2024-01-24T23:21:10.011863+0000","last_fullsized":"2024-01-24T23:21:10.011863+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T07:45:59.231652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.011220+0000","last_change":"2024-01-24T23:21:08.949886+0000","last_active":"2024-01-24T23:21:10.011220+0000","last_peered":"2024-01-24T23:21:10.011220+0000","last_clean":"2024-01-24T23:21:10.011220+0000","last_became_active":"2024-01-24T23:21:08.949580+0000","last_became_peered":"2024-01-24T23:21:08.949580+0000","last_unstale":"2024-01-24T23:21:10.011220+0000","last_undegraded":"2024-01-24T23:21:10.011220+0000","last_fullsized":"2024-01-24T23:21:10.011220+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T02:18:56.755337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.012731+0000","last_change":"2024-01-24T23:21:08.944072+0000","last_active":"2024-01-24T23:21:10.012731+0000","last_peered":"2024-01-24T23:21:10.012731+0000","last_clean":"2024-01-24T23:21:10.012731+0000","last_became_active":"2024-01-24T23:21:08.943616+0000","last_became_peered":"2024-01-24T23:21:08.943616+0000","last_unstale":"2024-01-24T23:21:10.012731+0000","last_undegraded":"2024-01-24T23:21:10.012731+0000","last_fullsized":"2024-01-24T23:21:10.012731+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T07:20:03.852420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1057856,"data_compressed":60590,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640259,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":192512,"data_stored":566913,"data_compressed":30295,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]}]},{"osd":4,"up_from":11,"seq":47244640259,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":131072,"data_stored":37985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]}]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":131072,"data_stored":37985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]}]},{"osd":2,"up_from":11,"seq":47244640259,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":188416,"data_stored":505473,"data_compressed":27335,"data_compressed_allocated":36864,"data_compressed_original":450560,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]},{"osd":1,"up_from":11,"seq":47244640260,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":131072,"data_stored":37985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]}]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":131072,"data_stored":37985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":30295,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":30295,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T23:21:14.123 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T23:21:14.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.219+0000 7fc1124c6640 10 monclient: get_monmap_and_config 2024-01-24T23:21:14.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.219+0000 7fc1124c6640 10 monclient: build_initial_monmap 2024-01-24T23:21:14.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.219+0000 7fc1124c6640 10 monclient: monmap: 2024-01-24T23:21:14.225 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:14.225 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:14.225 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:14.226208+0000 2024-01-24T23:21:14.225 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:14.226208+0000 2024-01-24T23:21:14.225 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:14.225 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:14.225 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:14.226 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:14.226 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:14.226 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:14.226 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:14.226 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:14.226 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:14.226 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:14.226 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:14.226 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:14.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.219+0000 7fc1124c6640 1 Processor -- start 2024-01-24T23:21:14.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.219+0000 7fc1124c6640 1 -- start start 2024-01-24T23:21:14.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.219+0000 7fc1124c6640 10 monclient: init 2024-01-24T23:21:14.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:14.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 10 monclient: _add_conns ranks=[4,7,8,2,6,5,1,3,0] 2024-01-24T23:21:14.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 10 monclient(hunting): picked mon.noname-c con 0x7fc10c133c10 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:14.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 10 monclient(hunting): picked mon.noname-i con 0x7fc10c136960 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:14.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 10 monclient(hunting): picked mon.noname-e con 0x7fc10c13a470 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:14.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10c04ed70 con 0x7fc10c133c10 2024-01-24T23:21:14.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10c09dee0 con 0x7fc10c136960 2024-01-24T23:21:14.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10c09d400 con 0x7fc10c13a470 2024-01-24T23:21:14.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:14.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 10 monclient(hunting): authenticate will time out at 953.148415s 2024-01-24T23:21:14.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10bfff640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fc10c133c10 0x7fc10c133ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:59866/0 (socket says 172.21.15.115:59866) 2024-01-24T23:21:14.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc110a63640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fc10c13a470 0x7fc10c13c860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:48460/0 (socket says 172.21.15.115:48460) 2024-01-24T23:21:14.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10bfff640 1 -- 172.21.15.115:0/3888136533 learned_addr learned my addr 172.21.15.115:0/3888136533 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:14.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 1 -- 172.21.15.115:0/3888136533 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3237608210 0 0) 0x7fc10c04ed70 con 0x7fc10c133c10 2024-01-24T23:21:14.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:14.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:14.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:14.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 10 monclient(hunting): my global_id is 74389 2024-01-24T23:21:14.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 1 -- 172.21.15.115:0/3888136533 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0f4003530 con 0x7fc10c133c10 2024-01-24T23:21:14.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 1 -- 172.21.15.115:0/3888136533 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807702539 0 0) 0x7fc10c09d400 con 0x7fc10c13a470 2024-01-24T23:21:14.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:14.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:14.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:14.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 10 monclient(hunting): my global_id is 74330 2024-01-24T23:21:14.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 1 -- 172.21.15.115:0/3888136533 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc10c04ed70 con 0x7fc10c13a470 2024-01-24T23:21:14.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 1 -- 172.21.15.115:0/3888136533 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3446438904 0 0) 0x7fc0f4003530 con 0x7fc10c133c10 2024-01-24T23:21:14.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 1 -- 172.21.15.115:0/3888136533 >> v1:172.21.15.115:6793/0 conn(0x7fc10c13a470 legacy=0x7fc10c13c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:14.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 1 -- 172.21.15.115:0/3888136533 >> v1:172.21.15.132:6792/0 conn(0x7fc10c136960 legacy=0x7fc10c138d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:14.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 10 monclient: _finish_hunting 0 2024-01-24T23:21:14.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 1 monclient: found mon.noname-c 2024-01-24T23:21:14.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:14.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:21:14.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 1 -- 172.21.15.115:0/3888136533 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c09f380 con 0x7fc10c133c10 2024-01-24T23:21:14.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 10 monclient: _finish_auth 0 2024-01-24T23:21:14.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 10 monclient: _check_auth_tickets 2024-01-24T23:21:14.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:14.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 1 -- 172.21.15.115:0/3888136533 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc1000017c0 con 0x7fc10c133c10 2024-01-24T23:21:14.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 5 monclient: authenticate success, global_id 74389 2024-01-24T23:21:14.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:14.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:14.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:21:14.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 10 monclient: dump: 2024-01-24T23:21:14.235 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:14.236 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:14.236 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.236 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.236 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:14.236 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:14.236 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:14.236 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:14.236 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:14.236 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:14.236 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:14.236 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:14.236 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:14.236 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:14.237 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:14.237 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:14.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:14.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 1 -- 172.21.15.115:0/3888136533 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc1000019e0 con 0x7fc10c133c10 2024-01-24T23:21:14.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:14.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:14.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 4 set_mon_vals no callback set 2024-01-24T23:21:14.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 1 -- 172.21.15.115:0/3888136533 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc100004bf0 con 0x7fc10c133c10 2024-01-24T23:21:14.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:14.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:14.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10affd640 10 monclient: dump: 2024-01-24T23:21:14.237 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:14.237 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:14.237 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.238 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.238 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:14.238 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:14.238 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:14.238 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:14.238 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:14.238 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:14.238 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:14.238 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:14.238 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:14.238 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:14.238 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:14.238 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:14.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 10 monclient: shutdown 2024-01-24T23:21:14.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:14.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 1 -- 172.21.15.115:0/3888136533 >> v1:172.21.15.115:6791/0 conn(0x7fc10c133c10 legacy=0x7fc10c133ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:14.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 1 -- 172.21.15.115:0/3888136533 shutdown_connections 2024-01-24T23:21:14.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 1 -- 172.21.15.115:0/3888136533 >> 172.21.15.115:0/3888136533 conn(0x7fc10c0a5730 msgr2=0x7fc10c0a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:14.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 1 -- 172.21.15.115:0/3888136533 shutdown_connections 2024-01-24T23:21:14.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 1 -- 172.21.15.115:0/3888136533 wait complete. 2024-01-24T23:21:14.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 10 monclient: build_initial_monmap 2024-01-24T23:21:14.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 10 monclient: monmap: 2024-01-24T23:21:14.239 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:14.239 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:14.239 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:14.239 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:14.239 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:14.239 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:14.239 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:14.239 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:14.240 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:14.240 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:14.240 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:14.240 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:14.240 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:14.240 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:14.240 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:14.240 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:14.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 1 Processor -- start 2024-01-24T23:21:14.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 1 -- start start 2024-01-24T23:21:14.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 10 monclient: init 2024-01-24T23:21:14.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:14.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 10 monclient: _add_conns ranks=[4,6,8,1,0,2,5,7,3] 2024-01-24T23:21:14.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 10 monclient(hunting): picked mon.noname-c con 0x7fc10c133c10 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:14.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 10 monclient(hunting): picked mon.noname-d con 0x7fc10c136960 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:14.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 10 monclient(hunting): picked mon.noname-e con 0x7fc10c13a470 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:14.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10c06da00 con 0x7fc10c133c10 2024-01-24T23:21:14.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10c06f660 con 0x7fc10c136960 2024-01-24T23:21:14.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10c06f940 con 0x7fc10c13a470 2024-01-24T23:21:14.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:14.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc1124c6640 10 monclient(hunting): authenticate will time out at 953.151179s 2024-01-24T23:21:14.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10bfff640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fc10c133c10 0x7fc10c06cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:59882/0 (socket says 172.21.15.115:59882) 2024-01-24T23:21:14.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc10bfff640 1 -- 172.21.15.115:0/3625984434 learned_addr learned my addr 172.21.15.115:0/3625984434 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:14.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3549666157 0 0) 0x7fc10c06da00 con 0x7fc10c133c10 2024-01-24T23:21:14.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:14.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:14.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:14.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 10 monclient(hunting): my global_id is 74398 2024-01-24T23:21:14.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0f8003530 con 0x7fc10c133c10 2024-01-24T23:21:14.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3610300823 0 0) 0x7fc10c06f660 con 0x7fc10c136960 2024-01-24T23:21:14.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:14.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:14.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:14.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 10 monclient(hunting): my global_id is 4407 2024-01-24T23:21:14.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc10c06da00 con 0x7fc10c136960 2024-01-24T23:21:14.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 906122976 0 0) 0x7fc10c06f940 con 0x7fc10c13a470 2024-01-24T23:21:14.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:14.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:14.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:14.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 10 monclient(hunting): my global_id is 74339 2024-01-24T23:21:14.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc10c06f660 con 0x7fc10c13a470 2024-01-24T23:21:14.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 423168262 0 0) 0x7fc0f8003530 con 0x7fc10c133c10 2024-01-24T23:21:14.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc10c06f940 con 0x7fc10c133c10 2024-01-24T23:21:14.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc100001780 con 0x7fc10c133c10 2024-01-24T23:21:14.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:14.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:21:14.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 10 monclient(hunting): dump: 2024-01-24T23:21:14.243 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:14.243 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:14.243 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.243 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.243 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:14.243 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:14.243 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:14.244 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:14.244 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:14.244 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:14.244 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:14.244 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:14.244 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:14.244 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:14.244 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:14.244 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:14.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:14.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2911732045 0 0) 0x7fc10c06da00 con 0x7fc10c136960 2024-01-24T23:21:14.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc0f8003530 con 0x7fc10c136960 2024-01-24T23:21:14.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2267031822 0 0) 0x7fc10c06f660 con 0x7fc10c13a470 2024-01-24T23:21:14.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc10c06da00 con 0x7fc10c13a470 2024-01-24T23:21:14.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc0fc0019e0 con 0x7fc10c136960 2024-01-24T23:21:14.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:14.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:14.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 10 monclient(hunting): dump: 2024-01-24T23:21:14.245 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:14.245 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:14.245 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.245 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.245 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:14.245 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:14.245 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:14.245 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:14.245 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:14.245 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:14.245 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:14.246 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:14.246 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:14.246 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:14.246 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:14.246 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:14.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc104001af0 con 0x7fc10c13a470 2024-01-24T23:21:14.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:14.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:21:14.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 10 monclient(hunting): dump: 2024-01-24T23:21:14.246 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:14.246 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:14.246 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.246 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.246 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:14.246 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:14.246 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:14.247 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:14.247 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:14.247 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:14.247 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:14.247 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:14.247 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:14.247 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:14.247 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:14.247 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:14.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.223+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3092037734 0 0) 0x7fc10c06f940 con 0x7fc10c133c10 2024-01-24T23:21:14.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 >> v1:172.21.15.115:6793/0 conn(0x7fc10c13a470 legacy=0x7fc10c1d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:14.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 >> v1:172.21.15.115:6792/0 conn(0x7fc10c136960 legacy=0x7fc10c06d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:14.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc108ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:21:14.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc108ff9640 1 monclient: found mon.c 2024-01-24T23:21:14.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc108ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:14.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc108ff9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:14.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c06eca0 con 0x7fc10c133c10 2024-01-24T23:21:14.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc108ff9640 10 monclient: _finish_auth 0 2024-01-24T23:21:14.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc108ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:21:14.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc108ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:14.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc1124c6640 5 monclient: authenticate success, global_id 74398 2024-01-24T23:21:14.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc1124c6640 10 monclient: _renew_subs 2024-01-24T23:21:14.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc1124c6640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:14.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc1124c6640 1 -- 172.21.15.115:0/3625984434 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc10c06dc80 con 0x7fc10c133c10 2024-01-24T23:21:14.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc1124c6640 10 monclient: _renew_subs 2024-01-24T23:21:14.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc1124c6640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:14.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc1124c6640 1 -- 172.21.15.115:0/3625984434 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc10c06f230 con 0x7fc10c133c10 2024-01-24T23:21:14.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc100004f00 con 0x7fc10c133c10 2024-01-24T23:21:14.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc108ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:14.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc1000053b0 con 0x7fc10c133c10 2024-01-24T23:21:14.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc108ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:14.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc10a7fc640 4 set_mon_vals no callback set 2024-01-24T23:21:14.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc108ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:14.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc108ff9640 10 monclient: dump: 2024-01-24T23:21:14.249 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:14.249 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:14.249 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.249 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.249 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:14.249 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:14.249 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:14.249 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:14.249 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:14.249 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:14.249 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:14.249 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:14.249 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:14.249 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:14.249 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:14.250 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:14.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc1124c6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:14.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc1124c6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:14.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc1124c6640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:14.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc1124c6640 1 -- 172.21.15.115:0/3625984434 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0ec005090 con 0x7fc10c133c10 2024-01-24T23:21:14.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fc10000fe30 con 0x7fc10c133c10 2024-01-24T23:21:14.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7fc1000438f0 con 0x7fc10c133c10 2024-01-24T23:21:14.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.227+0000 7fc10b7fe640 10 monclient: get_auth_request con 0x7fc0f8037d40 auth_method 0 2024-01-24T23:21:14.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.231+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fc100014a90 con 0x7fc10c133c10 2024-01-24T23:21:14.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.231+0000 7fc108ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:14.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.231+0000 7fc108ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:14.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.371+0000 7fc1124c6640 1 -- 172.21.15.115:0/3625984434 --> v1:172.21.15.132:6812/1835401922 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc0ec000ba0 con 0x7fc0f8037d40 2024-01-24T23:21:14.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.375+0000 7fc108ff9640 1 -- 172.21.15.115:0/3625984434 <== mgr.4113 v1:172.21.15.132:6812/1835401922 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+40608 (unknown 2965378022 0 824952323) 0x7fc0ec000ba0 con 0x7fc0f8037d40 2024-01-24T23:21:14.380 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:21:14.380 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-01-24T23:21:14.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.375+0000 7fc1124c6640 1 -- 172.21.15.115:0/3625984434 >> v1:172.21.15.132:6812/1835401922 conn(0x7fc0f8037d40 legacy=0x7fc0f803a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:14.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.375+0000 7fc1124c6640 10 monclient: shutdown 2024-01-24T23:21:14.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.375+0000 7fc1124c6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:14.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.375+0000 7fc1124c6640 1 -- 172.21.15.115:0/3625984434 >> v1:172.21.15.115:6791/0 conn(0x7fc10c133c10 legacy=0x7fc10c06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:14.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.375+0000 7fc1124c6640 1 -- 172.21.15.115:0/3625984434 shutdown_connections 2024-01-24T23:21:14.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.375+0000 7fc1124c6640 1 -- 172.21.15.115:0/3625984434 >> 172.21.15.115:0/3625984434 conn(0x7fc10c0a5730 msgr2=0x7fc10c0a2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:14.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.379+0000 7fc1124c6640 1 -- 172.21.15.115:0/3625984434 shutdown_connections 2024-01-24T23:21:14.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.379+0000 7fc1124c6640 1 -- 172.21.15.115:0/3625984434 wait complete. 2024-01-24T23:21:14.397 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-01-24T23:21:13.831041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163060,"kb_used_data":884,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067980,"statfs":{"total":579820584960,"available":579653611520,"internally_reserved":0,"allocated":905216,"data_stored":1224326,"data_compressed":57630,"data_compressed_allocated":77824,"data_compressed_original":962560,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000558"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.012747+0000","last_change":"2024-01-24T23:21:08.941679+0000","last_active":"2024-01-24T23:21:10.012747+0000","last_peered":"2024-01-24T23:21:10.012747+0000","last_clean":"2024-01-24T23:21:10.012747+0000","last_became_active":"2024-01-24T23:21:08.941408+0000","last_became_peered":"2024-01-24T23:21:08.941408+0000","last_unstale":"2024-01-24T23:21:10.012747+0000","last_undegraded":"2024-01-24T23:21:10.012747+0000","last_fullsized":"2024-01-24T23:21:10.012747+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T10:59:01.658977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.012767+0000","last_change":"2024-01-24T23:21:08.949886+0000","last_active":"2024-01-24T23:21:10.012767+0000","last_peered":"2024-01-24T23:21:10.012767+0000","last_clean":"2024-01-24T23:21:10.012767+0000","last_became_active":"2024-01-24T23:21:08.949572+0000","last_became_peered":"2024-01-24T23:21:08.949572+0000","last_unstale":"2024-01-24T23:21:10.012767+0000","last_undegraded":"2024-01-24T23:21:10.012767+0000","last_fullsized":"2024-01-24T23:21:10.012767+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T10:40:08.936213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.011232+0000","last_change":"2024-01-24T23:21:08.941915+0000","last_active":"2024-01-24T23:21:10.011232+0000","last_peered":"2024-01-24T23:21:10.011232+0000","last_clean":"2024-01-24T23:21:10.011232+0000","last_became_active":"2024-01-24T23:21:08.941537+0000","last_became_peered":"2024-01-24T23:21:08.941537+0000","last_unstale":"2024-01-24T23:21:10.011232+0000","last_undegraded":"2024-01-24T23:21:10.011232+0000","last_fullsized":"2024-01-24T23:21:10.011232+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T03:58:22.010129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.017040+0000","last_change":"2024-01-24T23:21:08.943234+0000","last_active":"2024-01-24T23:21:10.017040+0000","last_peered":"2024-01-24T23:21:10.017040+0000","last_clean":"2024-01-24T23:21:10.017040+0000","last_became_active":"2024-01-24T23:21:08.942798+0000","last_became_peered":"2024-01-24T23:21:08.942798+0000","last_unstale":"2024-01-24T23:21:10.017040+0000","last_undegraded":"2024-01-24T23:21:10.017040+0000","last_fullsized":"2024-01-24T23:21:10.017040+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T01:56:21.645104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":161,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.952984+0000","last_change":"2024-01-24T23:21:08.940580+0000","last_active":"2024-01-24T23:21:10.952984+0000","last_peered":"2024-01-24T23:21:10.952984+0000","last_clean":"2024-01-24T23:21:10.952984+0000","last_became_active":"2024-01-24T23:21:08.940211+0000","last_became_peered":"2024-01-24T23:21:08.940211+0000","last_unstale":"2024-01-24T23:21:10.952984+0000","last_undegraded":"2024-01-24T23:21:10.952984+0000","last_fullsized":"2024-01-24T23:21:10.952984+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T05:41:59.190049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.686443+0000","last_change":"2024-01-24T23:21:08.951610+0000","last_active":"2024-01-24T23:21:10.686443+0000","last_peered":"2024-01-24T23:21:10.686443+0000","last_clean":"2024-01-24T23:21:10.686443+0000","last_became_active":"2024-01-24T23:21:08.950717+0000","last_became_peered":"2024-01-24T23:21:08.950717+0000","last_unstale":"2024-01-24T23:21:10.686443+0000","last_undegraded":"2024-01-24T23:21:10.686443+0000","last_fullsized":"2024-01-24T23:21:10.686443+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T00:26:13.689261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.011863+0000","last_change":"2024-01-24T23:21:08.943904+0000","last_active":"2024-01-24T23:21:10.011863+0000","last_peered":"2024-01-24T23:21:10.011863+0000","last_clean":"2024-01-24T23:21:10.011863+0000","last_became_active":"2024-01-24T23:21:08.943567+0000","last_became_peered":"2024-01-24T23:21:08.943567+0000","last_unstale":"2024-01-24T23:21:10.011863+0000","last_undegraded":"2024-01-24T23:21:10.011863+0000","last_fullsized":"2024-01-24T23:21:10.011863+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T07:45:59.231652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.011220+0000","last_change":"2024-01-24T23:21:08.949886+0000","last_active":"2024-01-24T23:21:10.011220+0000","last_peered":"2024-01-24T23:21:10.011220+0000","last_clean":"2024-01-24T23:21:10.011220+0000","last_became_active":"2024-01-24T23:21:08.949580+0000","last_became_peered":"2024-01-24T23:21:08.949580+0000","last_unstale":"2024-01-24T23:21:10.011220+0000","last_undegraded":"2024-01-24T23:21:10.011220+0000","last_fullsized":"2024-01-24T23:21:10.011220+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T02:18:56.755337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.012731+0000","last_change":"2024-01-24T23:21:08.944072+0000","last_active":"2024-01-24T23:21:10.012731+0000","last_peered":"2024-01-24T23:21:10.012731+0000","last_clean":"2024-01-24T23:21:10.012731+0000","last_became_active":"2024-01-24T23:21:08.943616+0000","last_became_peered":"2024-01-24T23:21:08.943616+0000","last_unstale":"2024-01-24T23:21:10.012731+0000","last_undegraded":"2024-01-24T23:21:10.012731+0000","last_fullsized":"2024-01-24T23:21:10.012731+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T07:20:03.852420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1057856,"data_compressed":60590,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640259,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":192512,"data_stored":566913,"data_compressed":30295,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]}]},{"osd":4,"up_from":11,"seq":47244640259,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":131072,"data_stored":37985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]}]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":131072,"data_stored":37985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]}]},{"osd":2,"up_from":11,"seq":47244640259,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":188416,"data_stored":505473,"data_compressed":27335,"data_compressed_allocated":36864,"data_compressed_original":450560,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]},{"osd":1,"up_from":11,"seq":47244640260,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":131072,"data_stored":37985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]}]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":131072,"data_stored":37985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":30295,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":30295,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T23:21:14.398 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-01-24T23:21:14.398 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-01-24T23:21:14.398 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-01-24T23:21:14.398 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-01-24T23:21:14.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.499+0000 7fc8dbb84640 10 monclient: get_monmap_and_config 2024-01-24T23:21:14.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.499+0000 7fc8dbb84640 10 monclient: build_initial_monmap 2024-01-24T23:21:14.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.499+0000 7fc8dbb84640 10 monclient: monmap: 2024-01-24T23:21:14.503 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:14.503 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:14.503 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:14.504096+0000 2024-01-24T23:21:14.503 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:14.504096+0000 2024-01-24T23:21:14.503 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:14.503 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:14.503 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:14.503 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:14.503 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:14.503 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:14.503 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:14.503 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:14.503 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:14.503 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:14.503 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:14.504 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:14.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.499+0000 7fc8dbb84640 1 Processor -- start 2024-01-24T23:21:14.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.499+0000 7fc8dbb84640 1 -- start start 2024-01-24T23:21:14.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.499+0000 7fc8dbb84640 10 monclient: init 2024-01-24T23:21:14.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.499+0000 7fc8dbb84640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:14.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.499+0000 7fc8dbb84640 10 monclient: _add_conns ranks=[1,8,7,3,4,2,5,0,6] 2024-01-24T23:21:14.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.499+0000 7fc8dbb84640 10 monclient(hunting): picked mon.noname-f con 0x7fc8d409e740 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:14.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.499+0000 7fc8dbb84640 10 monclient(hunting): picked mon.noname-e con 0x7fc8d409f230 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:14.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.499+0000 7fc8dbb84640 10 monclient(hunting): picked mon.noname-i con 0x7fc8d4071e90 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:14.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.499+0000 7fc8dbb84640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d404ed70 con 0x7fc8d409e740 2024-01-24T23:21:14.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.499+0000 7fc8dbb84640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d40a1400 con 0x7fc8d4071e90 2024-01-24T23:21:14.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.499+0000 7fc8dbb84640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d40a37d0 con 0x7fc8d409f230 2024-01-24T23:21:14.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.499+0000 7fc8dbb84640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:14.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.499+0000 7fc8dbb84640 10 monclient(hunting): authenticate will time out at 953.427400s 2024-01-24T23:21:14.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.499+0000 7fc8d911f640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fc8d409f230 0x7fc8d4139b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:48476/0 (socket says 172.21.15.115:48476) 2024-01-24T23:21:14.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.499+0000 7fc8d911f640 1 -- 172.21.15.115:0/3010325303 learned_addr learned my addr 172.21.15.115:0/3010325303 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:14.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.499+0000 7fc8d891e640 1 -- 172.21.15.115:0/3010325303 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2077816625 0 0) 0x7fc8d40a37d0 con 0x7fc8d409f230 2024-01-24T23:21:14.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.499+0000 7fc8d891e640 10 monclient(hunting): _negotiate 2024-01-24T23:21:14.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.499+0000 7fc8d891e640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:14.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.499+0000 7fc8d891e640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:14.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.499+0000 7fc8d891e640 10 monclient(hunting): my global_id is 74348 2024-01-24T23:21:14.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.499+0000 7fc8d891e640 1 -- 172.21.15.115:0/3010325303 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8c0003530 con 0x7fc8d409f230 2024-01-24T23:21:14.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8d891e640 1 -- 172.21.15.115:0/3010325303 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3349268715 0 0) 0x7fc8c0003530 con 0x7fc8d409f230 2024-01-24T23:21:14.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8d891e640 1 -- 172.21.15.115:0/3010325303 >> v1:172.21.15.132:6792/0 conn(0x7fc8d4071e90 legacy=0x7fc8d413d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:14.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8d891e640 1 -- 172.21.15.115:0/3010325303 >> v1:172.21.15.132:6789/0 conn(0x7fc8d409e740 legacy=0x7fc8d409eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:14.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8d891e640 10 monclient: _finish_hunting 0 2024-01-24T23:21:14.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8d891e640 1 monclient: found mon.noname-e 2024-01-24T23:21:14.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8d891e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:14.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8d891e640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:21:14.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8d891e640 1 -- 172.21.15.115:0/3010325303 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d40a3120 con 0x7fc8d409f230 2024-01-24T23:21:14.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8d891e640 10 monclient: _finish_auth 0 2024-01-24T23:21:14.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8d891e640 10 monclient: _check_auth_tickets 2024-01-24T23:21:14.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8d891e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:14.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8d891e640 1 -- 172.21.15.115:0/3010325303 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc8c40017c0 con 0x7fc8d409f230 2024-01-24T23:21:14.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8d891e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:14.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8d891e640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:21:14.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8d891e640 10 monclient: dump: 2024-01-24T23:21:14.512 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:14.512 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:14.512 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.512 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.512 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:14.512 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:14.512 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:14.512 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:14.512 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:14.512 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:14.512 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:14.512 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:14.512 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:14.512 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:14.513 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:14.513 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:14.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 5 monclient: authenticate success, global_id 74348 2024-01-24T23:21:14.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:14.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8d891e640 1 -- 172.21.15.115:0/3010325303 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc8c4001ae0 con 0x7fc8d409f230 2024-01-24T23:21:14.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8d891e640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:14.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8d891e640 1 -- 172.21.15.115:0/3010325303 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc8c4004bf0 con 0x7fc8d409f230 2024-01-24T23:21:14.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8d891e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:14.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8d891e640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:21:14.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8d891e640 10 monclient: dump: 2024-01-24T23:21:14.513 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:14.513 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:14.513 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.513 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.513 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:14.513 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:14.513 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:14.514 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:14.514 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:14.514 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:14.514 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:14.514 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:14.514 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:14.514 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:14.514 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:14.514 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:14.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:14.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 4 set_mon_vals no callback set 2024-01-24T23:21:14.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 10 monclient: shutdown 2024-01-24T23:21:14.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:14.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 1 -- 172.21.15.115:0/3010325303 >> v1:172.21.15.115:6793/0 conn(0x7fc8d409f230 legacy=0x7fc8d4139b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:14.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 1 -- 172.21.15.115:0/3010325303 shutdown_connections 2024-01-24T23:21:14.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 1 -- 172.21.15.115:0/3010325303 >> 172.21.15.115:0/3010325303 conn(0x7fc8d40666f0 msgr2=0x7fc8d4066b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:14.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 1 -- 172.21.15.115:0/3010325303 shutdown_connections 2024-01-24T23:21:14.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 1 -- 172.21.15.115:0/3010325303 wait complete. 2024-01-24T23:21:14.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 10 monclient: build_initial_monmap 2024-01-24T23:21:14.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 10 monclient: monmap: 2024-01-24T23:21:14.515 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:14.515 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:14.515 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:14.515 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:14.515 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:14.515 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:14.515 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:14.515 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:14.515 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:14.515 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:14.515 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:14.515 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:14.515 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:14.516 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:14.516 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:14.516 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:14.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 1 Processor -- start 2024-01-24T23:21:14.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 1 -- start start 2024-01-24T23:21:14.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 10 monclient: init 2024-01-24T23:21:14.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:14.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 10 monclient: _add_conns ranks=[3,4,5,6,8,2,0,1,7] 2024-01-24T23:21:14.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 10 monclient(hunting): picked mon.noname-g con 0x7fc8d4071e90 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:14.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 10 monclient(hunting): picked mon.noname-c con 0x7fc8d409e740 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:14.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 10 monclient(hunting): picked mon.noname-h con 0x7fc8d406b660 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:14.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d409f640 con 0x7fc8d4071e90 2024-01-24T23:21:14.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d41a9540 con 0x7fc8d409e740 2024-01-24T23:21:14.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d40722e0 con 0x7fc8d406b660 2024-01-24T23:21:14.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:14.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 10 monclient(hunting): authenticate will time out at 953.430233s 2024-01-24T23:21:14.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8d911f640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fc8d409e740 0x7fc8d406af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:59898/0 (socket says 172.21.15.115:59898) 2024-01-24T23:21:14.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8d911f640 1 -- 172.21.15.115:0/897459536 learned_addr learned my addr 172.21.15.115:0/897459536 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:14.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 1 -- 172.21.15.115:0/897459536 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2225616784 0 0) 0x7fc8d41a9540 con 0x7fc8d409e740 2024-01-24T23:21:14.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:14.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:14.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:14.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 10 monclient(hunting): my global_id is 74407 2024-01-24T23:21:14.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 1 -- 172.21.15.115:0/897459536 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8b8003530 con 0x7fc8d409e740 2024-01-24T23:21:14.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 1 -- 172.21.15.115:0/897459536 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3725840772 0 0) 0x7fc8b8003530 con 0x7fc8d409e740 2024-01-24T23:21:14.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 1 -- 172.21.15.115:0/897459536 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc8d41a9540 con 0x7fc8d409e740 2024-01-24T23:21:14.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 1 -- 172.21.15.115:0/897459536 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc8c4001780 con 0x7fc8d409e740 2024-01-24T23:21:14.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:14.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:21:14.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 10 monclient(hunting): dump: 2024-01-24T23:21:14.518 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:14.518 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:14.518 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.518 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.518 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:14.518 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:14.518 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:14.518 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:14.518 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:14.518 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:14.518 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:14.518 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:14.518 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:14.518 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:14.518 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:14.519 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:14.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:14.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 1 -- 172.21.15.115:0/897459536 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4072567896 0 0) 0x7fc8d41a9540 con 0x7fc8d409e740 2024-01-24T23:21:14.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 1 -- 172.21.15.115:0/897459536 >> v1:172.21.15.132:6791/0 conn(0x7fc8d406b660 legacy=0x7fc8d41d3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:14.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 1 -- 172.21.15.115:0/897459536 >> v1:172.21.15.132:6790/0 conn(0x7fc8d4071e90 legacy=0x7fc8d406a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:14.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:21:14.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 1 monclient: found mon.c 2024-01-24T23:21:14.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:14.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:14.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 1 -- 172.21.15.115:0/897459536 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d41a9260 con 0x7fc8d409e740 2024-01-24T23:21:14.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 10 monclient: _finish_auth 0 2024-01-24T23:21:14.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:21:14.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:14.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 5 monclient: authenticate success, global_id 74407 2024-01-24T23:21:14.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 10 monclient: _renew_subs 2024-01-24T23:21:14.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:14.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 1 -- 172.21.15.115:0/897459536 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d41a7900 con 0x7fc8d409e740 2024-01-24T23:21:14.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 10 monclient: _renew_subs 2024-01-24T23:21:14.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:14.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 1 -- 172.21.15.115:0/897459536 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d41ba1b0 con 0x7fc8d409e740 2024-01-24T23:21:14.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 1 -- 172.21.15.115:0/897459536 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc8c4001c50 con 0x7fc8d409e740 2024-01-24T23:21:14.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:14.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 1 -- 172.21.15.115:0/897459536 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc8c4005410 con 0x7fc8d409e740 2024-01-24T23:21:14.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:14.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:14.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8cbfff640 4 set_mon_vals no callback set 2024-01-24T23:21:14.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 10 monclient: dump: 2024-01-24T23:21:14.520 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:14.520 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:14.520 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.521 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.521 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:14.521 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:14.521 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:14.521 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:14.521 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:14.521 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:14.521 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:14.521 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:14.521 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:14.521 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:14.521 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:14.521 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:14.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:14.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:14.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:14.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8dbb84640 1 -- 172.21.15.115:0/897459536 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8bc005090 con 0x7fc8d409e740 2024-01-24T23:21:14.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.503+0000 7fc8ca7fc640 1 -- 172.21.15.115:0/897459536 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fc8c400fe90 con 0x7fc8d409e740 2024-01-24T23:21:14.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.507+0000 7fc8ca7fc640 1 -- 172.21.15.115:0/897459536 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7fc8c40438f0 con 0x7fc8d409e740 2024-01-24T23:21:14.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.507+0000 7fc8d9920640 10 monclient: get_auth_request con 0x7fc8b8036e80 auth_method 0 2024-01-24T23:21:14.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.511+0000 7fc8ca7fc640 1 -- 172.21.15.115:0/897459536 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fc8c4014af0 con 0x7fc8d409e740 2024-01-24T23:21:14.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.511+0000 7fc8ca7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:14.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.511+0000 7fc8ca7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:14.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.679+0000 7fc8dbb84640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2024-01-24T23:21:14.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.679+0000 7fc8dbb84640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2024-01-24T23:21:14.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.683+0000 7fc8dbb84640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:14.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.683+0000 7fc8dbb84640 1 -- 172.21.15.115:0/897459536 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc8bc0058c0 con 0x7fc8d409e740 2024-01-24T23:21:14.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.683+0000 7fc8ca7fc640 1 -- 172.21.15.115:0/897459536 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (unknown 3730786198 0 4185958460) 0x7fc8c4013190 con 0x7fc8d409e740 2024-01-24T23:21:14.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.683+0000 7fc8ca7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2024-01-24T23:21:14.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.683+0000 7fc8ca7fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:14.686 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:21:14.686 INFO:teuthology.orchestra.run.smithi115.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-01-24T23:21:14.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.683+0000 7fc8dbb84640 1 -- 172.21.15.115:0/897459536 >> v1:172.21.15.132:6812/1835401922 conn(0x7fc8b8036e80 legacy=0x7fc8b8039340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:14.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.683+0000 7fc8dbb84640 10 monclient: shutdown 2024-01-24T23:21:14.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.683+0000 7fc8dbb84640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:14.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.683+0000 7fc8dbb84640 1 -- 172.21.15.115:0/897459536 >> v1:172.21.15.115:6791/0 conn(0x7fc8d409e740 legacy=0x7fc8d406af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:14.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.683+0000 7fc8dbb84640 1 -- 172.21.15.115:0/897459536 shutdown_connections 2024-01-24T23:21:14.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.683+0000 7fc8dbb84640 1 -- 172.21.15.115:0/897459536 >> 172.21.15.115:0/897459536 conn(0x7fc8d40666f0 msgr2=0x7fc8d41bc610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:14.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.683+0000 7fc8dbb84640 1 -- 172.21.15.115:0/897459536 shutdown_connections 2024-01-24T23:21:14.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.683+0000 7fc8dbb84640 1 -- 172.21.15.115:0/897459536 wait complete. 2024-01-24T23:21:14.702 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-01-24T23:21:14.702 INFO:teuthology.run_tasks:Running task mon_thrash... 2024-01-24T23:21:14.743 INFO:tasks.mon_thrash:Beginning mon_thrash... 2024-01-24T23:21:14.744 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-24T23:21:14.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.855+0000 7f99c557d640 10 monclient: get_monmap_and_config 2024-01-24T23:21:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.855+0000 7f99c557d640 10 monclient: build_initial_monmap 2024-01-24T23:21:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.855+0000 7f99c557d640 10 monclient: monmap: 2024-01-24T23:21:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:14.861013+0000 2024-01-24T23:21:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:14.861013+0000 2024-01-24T23:21:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:14.861 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:14.861 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:14.861 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:14.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.855+0000 7f99c557d640 1 Processor -- start 2024-01-24T23:21:14.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.855+0000 7f99c557d640 1 -- start start 2024-01-24T23:21:14.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.855+0000 7f99c557d640 10 monclient: init 2024-01-24T23:21:14.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.855+0000 7f99c557d640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:14.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.855+0000 7f99c557d640 10 monclient: _add_conns ranks=[4,1,8,5,6,2,0,7,3] 2024-01-24T23:21:14.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.855+0000 7f99c557d640 10 monclient(hunting): picked mon.noname-c con 0x7f99c009e740 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:14.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.855+0000 7f99c557d640 10 monclient(hunting): picked mon.noname-f con 0x7f99c009f230 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:14.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 10 monclient(hunting): picked mon.noname-e con 0x7f99c0071e90 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:14.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c004ed70 con 0x7f99c009f230 2024-01-24T23:21:14.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c00a1400 con 0x7f99c009e740 2024-01-24T23:21:14.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c00a37d0 con 0x7f99c0071e90 2024-01-24T23:21:14.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:14.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 10 monclient(hunting): authenticate will time out at 953.784061s 2024-01-24T23:21:14.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99beffd640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f99c009e740 0x7f99c009eb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:59914/0 (socket says 172.21.15.115:59914) 2024-01-24T23:21:14.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99beffd640 1 -- 172.21.15.115:0/3651385582 learned_addr learned my addr 172.21.15.115:0/3651385582 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:14.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 1 -- 172.21.15.115:0/3651385582 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1762315484 0 0) 0x7f99c00a1400 con 0x7f99c009e740 2024-01-24T23:21:14.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 10 monclient(hunting): _negotiate 2024-01-24T23:21:14.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:14.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:14.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 10 monclient(hunting): my global_id is 74416 2024-01-24T23:21:14.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 1 -- 172.21.15.115:0/3651385582 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99a8003530 con 0x7f99c009e740 2024-01-24T23:21:14.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 1 -- 172.21.15.115:0/3651385582 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2001194487 0 0) 0x7f99c00a37d0 con 0x7f99c0071e90 2024-01-24T23:21:14.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 10 monclient(hunting): _negotiate 2024-01-24T23:21:14.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:14.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:14.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 10 monclient(hunting): my global_id is 74357 2024-01-24T23:21:14.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 1 -- 172.21.15.115:0/3651385582 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99c00a1400 con 0x7f99c0071e90 2024-01-24T23:21:14.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 1 -- 172.21.15.115:0/3651385582 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1473608834 0 0) 0x7f99a8003530 con 0x7f99c009e740 2024-01-24T23:21:14.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 1 -- 172.21.15.115:0/3651385582 >> v1:172.21.15.115:6793/0 conn(0x7f99c0071e90 legacy=0x7f99c013d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:14.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 1 -- 172.21.15.115:0/3651385582 >> v1:172.21.15.132:6789/0 conn(0x7f99c009f230 legacy=0x7f99c0139b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:14.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 10 monclient: _finish_hunting 0 2024-01-24T23:21:14.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 1 monclient: found mon.noname-c 2024-01-24T23:21:14.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:14.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:21:14.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 1 -- 172.21.15.115:0/3651385582 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c00a3120 con 0x7f99c009e740 2024-01-24T23:21:14.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 10 monclient: _finish_auth 0 2024-01-24T23:21:14.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 10 monclient: _check_auth_tickets 2024-01-24T23:21:14.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:14.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 1 -- 172.21.15.115:0/3651385582 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f99b40017c0 con 0x7f99c009e740 2024-01-24T23:21:14.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:14.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:21:14.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 10 monclient: dump: 2024-01-24T23:21:14.870 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:14.870 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:14.870 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.870 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.870 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:14.870 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:14.870 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:14.870 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:14.870 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:14.870 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:14.870 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:14.870 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:14.870 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:14.870 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:14.871 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:14.871 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:14.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 5 monclient: authenticate success, global_id 74416 2024-01-24T23:21:14.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:14.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 1 -- 172.21.15.115:0/3651385582 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f99b40019e0 con 0x7f99c009e740 2024-01-24T23:21:14.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:14.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 1 -- 172.21.15.115:0/3651385582 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f99b4004bf0 con 0x7f99c009e740 2024-01-24T23:21:14.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:14.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 4 set_mon_vals no callback set 2024-01-24T23:21:14.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:14.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:14.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bdffb640 10 monclient: dump: 2024-01-24T23:21:14.871 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:14.871 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:14.871 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.872 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.872 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:14.872 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:14.872 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:14.872 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:14.872 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:14.872 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:14.872 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:14.872 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:14.872 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:14.872 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:14.872 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:14.872 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:14.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 10 monclient: shutdown 2024-01-24T23:21:14.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:14.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 1 -- 172.21.15.115:0/3651385582 >> v1:172.21.15.115:6791/0 conn(0x7f99c009e740 legacy=0x7f99c009eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:14.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 1 -- 172.21.15.115:0/3651385582 shutdown_connections 2024-01-24T23:21:14.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 1 -- 172.21.15.115:0/3651385582 >> 172.21.15.115:0/3651385582 conn(0x7f99c00666f0 msgr2=0x7f99c0066b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:14.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 1 -- 172.21.15.115:0/3651385582 shutdown_connections 2024-01-24T23:21:14.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 1 -- 172.21.15.115:0/3651385582 wait complete. 2024-01-24T23:21:14.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 10 monclient: build_initial_monmap 2024-01-24T23:21:14.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 10 monclient: monmap: 2024-01-24T23:21:14.873 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:14.873 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:14.873 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:14.873 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:14.873 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:14.873 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:14.874 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:14.874 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:14.874 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:14.874 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:14.874 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:14.874 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:14.874 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:14.874 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:14.874 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:14.874 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:14.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 1 Processor -- start 2024-01-24T23:21:14.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 1 -- start start 2024-01-24T23:21:14.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 10 monclient: init 2024-01-24T23:21:14.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:14.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 10 monclient: _add_conns ranks=[4,5,3,1,0,6,7,2,8] 2024-01-24T23:21:14.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 10 monclient(hunting): picked mon.noname-c con 0x7f99c0071e90 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:14.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 10 monclient(hunting): picked mon.noname-h con 0x7f99c009e740 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:14.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 10 monclient(hunting): picked mon.noname-g con 0x7f99c01cf2c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:14.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c01c9c00 con 0x7f99c01cf2c0 2024-01-24T23:21:14.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c01a7090 con 0x7f99c0071e90 2024-01-24T23:21:14.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c01a8b50 con 0x7f99c009e740 2024-01-24T23:21:14.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:14.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 10 monclient(hunting): authenticate will time out at 953.786795s 2024-01-24T23:21:14.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99beffd640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f99c0071e90 0x7f99c01c94f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:59918/0 (socket says 172.21.15.115:59918) 2024-01-24T23:21:14.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99beffd640 1 -- 172.21.15.115:0/1396860754 learned_addr learned my addr 172.21.15.115:0/1396860754 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:14.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 1 -- 172.21.15.115:0/1396860754 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3756350803 0 0) 0x7f99c01a7090 con 0x7f99c0071e90 2024-01-24T23:21:14.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:14.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:14.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:14.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 10 monclient(hunting): my global_id is 74425 2024-01-24T23:21:14.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 1 -- 172.21.15.115:0/1396860754 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99ac003530 con 0x7f99c0071e90 2024-01-24T23:21:14.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 1 -- 172.21.15.115:0/1396860754 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1346787961 0 0) 0x7f99ac003530 con 0x7f99c0071e90 2024-01-24T23:21:14.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 1 -- 172.21.15.115:0/1396860754 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f99c01a7090 con 0x7f99c0071e90 2024-01-24T23:21:14.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 1 -- 172.21.15.115:0/1396860754 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f99b40016c0 con 0x7f99c0071e90 2024-01-24T23:21:14.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:14.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:21:14.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 10 monclient(hunting): dump: 2024-01-24T23:21:14.876 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:14.876 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:14.876 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.876 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.877 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:14.877 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:14.877 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:14.877 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:14.877 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:14.877 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:14.877 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:14.877 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:14.877 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:14.877 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:14.877 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:14.877 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:14.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:14.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 1 -- 172.21.15.115:0/1396860754 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2140383998 0 0) 0x7f99c01a7090 con 0x7f99c0071e90 2024-01-24T23:21:14.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 1 -- 172.21.15.115:0/1396860754 >> v1:172.21.15.132:6791/0 conn(0x7f99c009e740 legacy=0x7f99c01bba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:14.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 1 -- 172.21.15.115:0/1396860754 >> v1:172.21.15.132:6790/0 conn(0x7f99c01cf2c0 legacy=0x7f99c01d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:14.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 10 monclient: _finish_hunting 0 2024-01-24T23:21:14.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 1 monclient: found mon.c 2024-01-24T23:21:14.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:14.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:14.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 1 -- 172.21.15.115:0/1396860754 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c01a8e30 con 0x7f99c0071e90 2024-01-24T23:21:14.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 10 monclient: _finish_auth 0 2024-01-24T23:21:14.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 10 monclient: _check_auth_tickets 2024-01-24T23:21:14.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:14.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 5 monclient: authenticate success, global_id 74425 2024-01-24T23:21:14.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 10 monclient: _renew_subs 2024-01-24T23:21:14.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:14.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 1 -- 172.21.15.115:0/1396860754 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c01a9110 con 0x7f99c0071e90 2024-01-24T23:21:14.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 10 monclient: _renew_subs 2024-01-24T23:21:14.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:14.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99c557d640 1 -- 172.21.15.115:0/1396860754 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c01bc140 con 0x7f99c0071e90 2024-01-24T23:21:14.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 1 -- 172.21.15.115:0/1396860754 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f99b4004c70 con 0x7f99c0071e90 2024-01-24T23:21:14.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:14.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 1 -- 172.21.15.115:0/1396860754 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f99b40051f0 con 0x7f99c0071e90 2024-01-24T23:21:14.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:14.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99bd7fa640 4 set_mon_vals no callback set 2024-01-24T23:21:14.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:14.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.859+0000 7f99a77fe640 10 monclient: dump: 2024-01-24T23:21:14.879 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:14.879 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:14.879 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.879 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:14.879 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:14.879 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:14.879 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:14.879 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:14.880 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:14.880 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:14.880 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:14.880 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:14.880 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:14.880 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:14.880 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:14.880 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:14.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.863+0000 7f99c557d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:14.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.863+0000 7f99c557d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:14.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.863+0000 7f99c557d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:14.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.863+0000 7f99c557d640 1 -- 172.21.15.115:0/1396860754 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a0005090 con 0x7f99c0071e90 2024-01-24T23:21:14.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.863+0000 7f99a77fe640 1 -- 172.21.15.115:0/1396860754 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f99b400fc70 con 0x7f99c0071e90 2024-01-24T23:21:14.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.863+0000 7f99a77fe640 1 -- 172.21.15.115:0/1396860754 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f99b4043920 con 0x7f99c0071e90 2024-01-24T23:21:14.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.863+0000 7f99be7fc640 10 monclient: get_auth_request con 0x7f99ac036fd0 auth_method 0 2024-01-24T23:21:14.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.867+0000 7f99a77fe640 1 -- 172.21.15.115:0/1396860754 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f99b4011120 con 0x7f99c0071e90 2024-01-24T23:21:14.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.867+0000 7f99a77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:14.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:14.867+0000 7f99a77fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:15.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.015+0000 7f99c557d640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-01-24T23:21:15.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.015+0000 7f99c557d640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-01-24T23:21:15.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.015+0000 7f99c557d640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:15.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.015+0000 7f99c557d640 1 -- 172.21.15.115:0/1396860754 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99a0005340 con 0x7f99c0071e90 2024-01-24T23:21:15.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.015+0000 7f99a77fe640 1 -- 172.21.15.115:0/1396860754 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10764 (unknown 4221056075 0 3147136755) 0x7f99b4013200 con 0x7f99c0071e90 2024-01-24T23:21:15.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.015+0000 7f99a77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-01-24T23:21:15.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.015+0000 7f99a77fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:21:15.020 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":15,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","created":"2024-01-24T23:20:56.917703+0000","modified":"2024-01-24T23:21:09.926628+0000","last_up_change":"2024-01-24T23:21:05.857369+0000","last_in_change":"2024-01-24T23:20:58.690771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-01-24T23:21:07.002704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-01-24T23:21:07.902769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"902f4a51-85e9-4af8-b11f-e42a51b31723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6808","nonce":3064328804}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6809","nonce":3064328804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6811","nonce":3064328804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6810","nonce":3064328804}]},"public_addr":"172.21.15.115:6808/3064328804","cluster_addr":"172.21.15.115:6809/3064328804","heartbeat_back_addr":"172.21.15.115:6811/3064328804","heartbeat_front_addr":"172.21.15.115:6810/3064328804","state":["exists","up"]},{"osd":1,"uuid":"e06b92ed-3ab9-4103-8e41-4fa60aea2829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6801","nonce":3052044493}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6803","nonce":3052044493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6807","nonce":3052044493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6806","nonce":3052044493}]},"public_addr":"172.21.15.115:6801/3052044493","cluster_addr":"172.21.15.115:6803/3052044493","heartbeat_back_addr":"172.21.15.115:6807/3052044493","heartbeat_front_addr":"172.21.15.115:6806/3052044493","state":["exists","up"]},{"osd":2,"uuid":"013044c9-553c-4dd9-8438-c9873813e5af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6800","nonce":2776240662}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6802","nonce":2776240662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6805","nonce":2776240662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6804","nonce":2776240662}]},"public_addr":"172.21.15.115:6800/2776240662","cluster_addr":"172.21.15.115:6802/2776240662","heartbeat_back_addr":"172.21.15.115:6805/2776240662","heartbeat_front_addr":"172.21.15.115:6804/2776240662","state":["exists","up"]},{"osd":3,"uuid":"5fcff598-1939-415b-b055-57940893269e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6800","nonce":1048681489}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6801","nonce":1048681489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6804","nonce":1048681489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6802","nonce":1048681489}]},"public_addr":"172.21.15.132:6800/1048681489","cluster_addr":"172.21.15.132:6801/1048681489","heartbeat_back_addr":"172.21.15.132:6804/1048681489","heartbeat_front_addr":"172.21.15.132:6802/1048681489","state":["exists","up"]},{"osd":4,"uuid":"ef67ccd9-a21c-4a40-99ff-93f5b10a1fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6807","nonce":3601895184}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6809","nonce":3601895184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6811","nonce":3601895184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6810","nonce":3601895184}]},"public_addr":"172.21.15.132:6807/3601895184","cluster_addr":"172.21.15.132:6809/3601895184","heartbeat_back_addr":"172.21.15.132:6811/3601895184","heartbeat_front_addr":"172.21.15.132:6810/3601895184","state":["exists","up"]},{"osd":5,"uuid":"c2573ce8-6dd2-4272-85c7-2f5aa25eb2f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6803","nonce":3365641231}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6805","nonce":3365641231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6808","nonce":3365641231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6806","nonce":3365641231}]},"public_addr":"172.21.15.132:6803/3365641231","cluster_addr":"172.21.15.132:6805/3365641231","heartbeat_back_addr":"172.21.15.132:6808/3365641231","heartbeat_front_addr":"172.21.15.132:6806/3365641231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T23:21:04.184180+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T23:21:04.023173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T23:21:03.933732+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T23:21:04.204962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T23:21:04.208706+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-24T23:21:04.085169+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-24T23:21:15.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.019+0000 7f99c557d640 1 -- 172.21.15.115:0/1396860754 >> v1:172.21.15.132:6812/1835401922 conn(0x7f99ac036fd0 legacy=0x7f99ac039490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:15.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.019+0000 7f99c557d640 10 monclient: shutdown 2024-01-24T23:21:15.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.019+0000 7f99c557d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:15.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.019+0000 7f99c557d640 1 -- 172.21.15.115:0/1396860754 >> v1:172.21.15.115:6791/0 conn(0x7f99c0071e90 legacy=0x7f99c01c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:15.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.019+0000 7f99c557d640 1 -- 172.21.15.115:0/1396860754 shutdown_connections 2024-01-24T23:21:15.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.019+0000 7f99c557d640 1 -- 172.21.15.115:0/1396860754 >> 172.21.15.115:0/1396860754 conn(0x7f99c00666f0 msgr2=0x7f99c00a6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:15.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.019+0000 7f99c557d640 1 -- 172.21.15.115:0/1396860754 shutdown_connections 2024-01-24T23:21:15.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.019+0000 7f99c557d640 1 -- 172.21.15.115:0/1396860754 wait complete. 2024-01-24T23:21:15.037 INFO:tasks.mon_thrash.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-01-24T23:21:07.002704+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-01-24T23:21:07.902769+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-24T23:21:15.037 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-01-24T23:21:15.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc445041640 10 monclient: get_monmap_and_config 2024-01-24T23:21:15.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc445041640 10 monclient: build_initial_monmap 2024-01-24T23:21:15.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc445041640 10 monclient: monmap: 2024-01-24T23:21:15.151 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:15.151 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:15.151 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:15.152400+0000 2024-01-24T23:21:15.151 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:15.152400+0000 2024-01-24T23:21:15.151 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:15.151 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:15.151 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:15.151 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:15.151 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:15.151 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:15.152 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:15.152 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:15.152 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:15.152 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:15.152 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:15.152 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:15.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc445041640 1 Processor -- start 2024-01-24T23:21:15.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc445041640 1 -- start start 2024-01-24T23:21:15.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc445041640 10 monclient: init 2024-01-24T23:21:15.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc445041640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:15.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc445041640 10 monclient: _add_conns ranks=[7,1,5,8,2,4,0,3,6] 2024-01-24T23:21:15.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc445041640 10 monclient(hunting): picked mon.noname-i con 0x7fc4400a2c40 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:15.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc445041640 10 monclient(hunting): picked mon.noname-f con 0x7fc4400a3fd0 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:15.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc445041640 10 monclient(hunting): picked mon.noname-h con 0x7fc4400a06b0 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:15.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc445041640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44004ed70 con 0x7fc4400a3fd0 2024-01-24T23:21:15.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc445041640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4400a0e70 con 0x7fc4400a06b0 2024-01-24T23:21:15.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc445041640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4400a3730 con 0x7fc4400a2c40 2024-01-24T23:21:15.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc445041640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:15.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc445041640 10 monclient(hunting): authenticate will time out at 954.075022s 2024-01-24T23:21:15.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc43e59c640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fc4400a3fd0 0x7fc44013be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.115:54774/0 (socket says 172.21.15.115:54774) 2024-01-24T23:21:15.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc43e59c640 1 -- 172.21.15.115:0/1328614720 learned_addr learned my addr 172.21.15.115:0/1328614720 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:15.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc43dd9b640 1 -- 172.21.15.115:0/1328614720 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2863694784 0 0) 0x7fc4400a3730 con 0x7fc4400a2c40 2024-01-24T23:21:15.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc43dd9b640 10 monclient(hunting): _negotiate 2024-01-24T23:21:15.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc43dd9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:15.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc43dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:15.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc43dd9b640 10 monclient(hunting): my global_id is 4435 2024-01-24T23:21:15.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc43dd9b640 1 -- 172.21.15.115:0/1328614720 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc42c003530 con 0x7fc4400a2c40 2024-01-24T23:21:15.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc43dd9b640 1 -- 172.21.15.115:0/1328614720 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819353999 0 0) 0x7fc4400a0e70 con 0x7fc4400a06b0 2024-01-24T23:21:15.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc43dd9b640 10 monclient(hunting): _negotiate 2024-01-24T23:21:15.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc43dd9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:15.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc43dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:15.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc43dd9b640 10 monclient(hunting): my global_id is 4397 2024-01-24T23:21:15.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc43dd9b640 1 -- 172.21.15.115:0/1328614720 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4400a3730 con 0x7fc4400a06b0 2024-01-24T23:21:15.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc43dd9b640 1 -- 172.21.15.115:0/1328614720 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3413398876 0 0) 0x7fc44004ed70 con 0x7fc4400a3fd0 2024-01-24T23:21:15.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc43dd9b640 10 monclient(hunting): _negotiate 2024-01-24T23:21:15.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc43dd9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:15.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc43dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:15.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc43dd9b640 10 monclient(hunting): my global_id is 4384 2024-01-24T23:21:15.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.147+0000 7fc43dd9b640 1 -- 172.21.15.115:0/1328614720 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4400a0e70 con 0x7fc4400a3fd0 2024-01-24T23:21:15.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43dd9b640 1 -- 172.21.15.115:0/1328614720 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2991552926 0 0) 0x7fc4400a3730 con 0x7fc4400a06b0 2024-01-24T23:21:15.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43dd9b640 1 -- 172.21.15.115:0/1328614720 >> v1:172.21.15.132:6792/0 conn(0x7fc4400a2c40 legacy=0x7fc4400a3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:15.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43dd9b640 1 -- 172.21.15.115:0/1328614720 >> v1:172.21.15.132:6789/0 conn(0x7fc4400a3fd0 legacy=0x7fc44013be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:15.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43dd9b640 10 monclient: _finish_hunting 0 2024-01-24T23:21:15.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43dd9b640 1 monclient: found mon.noname-h 2024-01-24T23:21:15.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:15.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43dd9b640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:21:15.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43dd9b640 1 -- 172.21.15.115:0/1328614720 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4400a1150 con 0x7fc4400a06b0 2024-01-24T23:21:15.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43dd9b640 10 monclient: _finish_auth 0 2024-01-24T23:21:15.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43dd9b640 10 monclient: _check_auth_tickets 2024-01-24T23:21:15.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:15.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43dd9b640 1 -- 172.21.15.115:0/1328614720 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc428001aa0 con 0x7fc4400a06b0 2024-01-24T23:21:15.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:15.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43dd9b640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:21:15.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43dd9b640 10 monclient: dump: 2024-01-24T23:21:15.158 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:15.158 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:15.158 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.158 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.159 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:15.159 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:15.159 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:15.159 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:15.159 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:15.159 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:15.159 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:15.159 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:15.159 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:15.159 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:15.159 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:15.159 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:15.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 5 monclient: authenticate success, global_id 4397 2024-01-24T23:21:15.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:15.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43dd9b640 1 -- 172.21.15.115:0/1328614720 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc428001d50 con 0x7fc4400a06b0 2024-01-24T23:21:15.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43dd9b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:15.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:15.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 4 set_mon_vals no callback set 2024-01-24T23:21:15.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43dd9b640 1 -- 172.21.15.115:0/1328614720 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc428003ce0 con 0x7fc4400a06b0 2024-01-24T23:21:15.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:15.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43dd9b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:21:15.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43dd9b640 10 monclient: dump: 2024-01-24T23:21:15.160 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:15.160 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:15.160 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.160 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.160 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:15.160 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:15.160 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:15.160 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:15.160 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:15.161 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:15.161 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:15.161 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:15.161 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:15.161 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:15.161 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:15.161 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:15.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 10 monclient: shutdown 2024-01-24T23:21:15.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:15.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 1 -- 172.21.15.115:0/1328614720 >> v1:172.21.15.132:6791/0 conn(0x7fc4400a06b0 legacy=0x7fc44013f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:15.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 1 -- 172.21.15.115:0/1328614720 shutdown_connections 2024-01-24T23:21:15.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 1 -- 172.21.15.115:0/1328614720 >> 172.21.15.115:0/1328614720 conn(0x7fc4400a9ba0 msgr2=0x7fc4400a5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:15.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 1 -- 172.21.15.115:0/1328614720 shutdown_connections 2024-01-24T23:21:15.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 1 -- 172.21.15.115:0/1328614720 wait complete. 2024-01-24T23:21:15.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 10 monclient: build_initial_monmap 2024-01-24T23:21:15.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 10 monclient: monmap: 2024-01-24T23:21:15.162 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:15.162 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:15.162 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:15.162 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:15.162 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:15.162 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:15.162 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:15.162 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:15.162 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:15.162 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:15.162 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:15.162 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:15.162 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:15.162 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:15.162 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:15.162 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:15.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 1 Processor -- start 2024-01-24T23:21:15.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 1 -- start start 2024-01-24T23:21:15.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 10 monclient: init 2024-01-24T23:21:15.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:15.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 10 monclient: _add_conns ranks=[8,7,2,3,1,0,6,5,4] 2024-01-24T23:21:15.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 10 monclient(hunting): picked mon.noname-e con 0x7fc4400a06b0 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:15.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 10 monclient(hunting): picked mon.noname-i con 0x7fc4400a2c40 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:15.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 10 monclient(hunting): picked mon.noname-b con 0x7fc4400a3fd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:15.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4401a92d0 con 0x7fc4400a3fd0 2024-01-24T23:21:15.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44009fb50 con 0x7fc4400a2c40 2024-01-24T23:21:15.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4400a58a0 con 0x7fc4400a06b0 2024-01-24T23:21:15.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:15.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 10 monclient(hunting): authenticate will time out at 954.078545s 2024-01-24T23:21:15.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43e59c640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fc4400a2c40 0x7fc4401d0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.115:34772/0 (socket says 172.21.15.115:34772) 2024-01-24T23:21:15.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43e59c640 1 -- 172.21.15.115:0/3280210775 learned_addr learned my addr 172.21.15.115:0/3280210775 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:15.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43ed9d640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fc4400a06b0 0x7fc4401bbb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:48498/0 (socket says 172.21.15.115:48498) 2024-01-24T23:21:15.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43f59e640 1 --1- 172.21.15.115:0/3280210775 >> v1:172.21.15.115:6790/0 conn(0x7fc4400a3fd0 0x7fc4401d3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:55454/0 (socket says 172.21.15.115:55454) 2024-01-24T23:21:15.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4011278614 0 0) 0x7fc4400a58a0 con 0x7fc4400a06b0 2024-01-24T23:21:15.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:15.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:15.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:15.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient(hunting): my global_id is 74366 2024-01-24T23:21:15.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc420003530 con 0x7fc4400a06b0 2024-01-24T23:21:15.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2315067061 0 0) 0x7fc4401a92d0 con 0x7fc4400a3fd0 2024-01-24T23:21:15.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:15.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:15.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:15.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient(hunting): my global_id is 4367 2024-01-24T23:21:15.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4400a58a0 con 0x7fc4400a3fd0 2024-01-24T23:21:15.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 338936693 0 0) 0x7fc44009fb50 con 0x7fc4400a2c40 2024-01-24T23:21:15.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:15.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:15.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:15.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient(hunting): my global_id is 4444 2024-01-24T23:21:15.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4401a92d0 con 0x7fc4400a2c40 2024-01-24T23:21:15.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3711286449 0 0) 0x7fc420003530 con 0x7fc4400a06b0 2024-01-24T23:21:15.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc44009fb50 con 0x7fc4400a06b0 2024-01-24T23:21:15.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc430001cc0 con 0x7fc4400a06b0 2024-01-24T23:21:15.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:15.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:21:15.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient(hunting): dump: 2024-01-24T23:21:15.168 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:15.168 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:15.169 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.169 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.169 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:15.169 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:15.169 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:15.169 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:15.169 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:15.169 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:15.169 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:15.169 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:15.169 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:15.169 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:15.169 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:15.169 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:15.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:15.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2078425055 0 0) 0x7fc4400a58a0 con 0x7fc4400a3fd0 2024-01-24T23:21:15.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc420003530 con 0x7fc4400a3fd0 2024-01-24T23:21:15.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc428003cc0 con 0x7fc4400a3fd0 2024-01-24T23:21:15.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:15.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:15.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient(hunting): dump: 2024-01-24T23:21:15.170 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:15.170 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:15.170 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.170 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.170 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:15.170 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:15.170 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:15.170 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:15.170 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:15.171 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:15.171 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:15.171 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:15.171 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:15.171 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:15.171 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:15.171 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:15.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1377921586 0 0) 0x7fc4401a92d0 con 0x7fc4400a2c40 2024-01-24T23:21:15.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc4400a58a0 con 0x7fc4400a2c40 2024-01-24T23:21:15.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc4380019c0 con 0x7fc4400a2c40 2024-01-24T23:21:15.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:15.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-01-24T23:21:15.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient(hunting): dump: 2024-01-24T23:21:15.171 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:15.171 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:15.171 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.172 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.172 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:15.172 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:15.172 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:15.173 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:15.173 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:15.174 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:15.174 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:15.174 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:15.174 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:15.174 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:15.174 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:15.174 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:15.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3502724735 0 0) 0x7fc44009fb50 con 0x7fc4400a06b0 2024-01-24T23:21:15.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 >> v1:172.21.15.132:6792/0 conn(0x7fc4400a2c40 legacy=0x7fc4401d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:15.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 >> v1:172.21.15.115:6790/0 conn(0x7fc4400a3fd0 legacy=0x7fc4401d3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:15.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient: _finish_hunting 0 2024-01-24T23:21:15.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 1 monclient: found mon.e 2024-01-24T23:21:15.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:15.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:15.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4401a6fe0 con 0x7fc4400a06b0 2024-01-24T23:21:15.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient: _finish_auth 0 2024-01-24T23:21:15.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient: _check_auth_tickets 2024-01-24T23:21:15.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:15.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 5 monclient: authenticate success, global_id 74366 2024-01-24T23:21:15.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 10 monclient: _renew_subs 2024-01-24T23:21:15.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:15.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 1 -- 172.21.15.115:0/3280210775 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4401a8ff0 con 0x7fc4400a06b0 2024-01-24T23:21:15.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 10 monclient: _renew_subs 2024-01-24T23:21:15.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:15.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc445041640 1 -- 172.21.15.115:0/3280210775 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4401bc250 con 0x7fc4400a06b0 2024-01-24T23:21:15.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc4300038b0 con 0x7fc4400a06b0 2024-01-24T23:21:15.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:15.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc430001010 con 0x7fc4400a06b0 2024-01-24T23:21:15.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:15.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc43d59a640 4 set_mon_vals no callback set 2024-01-24T23:21:15.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:21:15.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.151+0000 7fc4377fe640 10 monclient: dump: 2024-01-24T23:21:15.176 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:15.176 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:15.176 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.176 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.176 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:15.176 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:15.176 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:15.176 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:15.176 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:15.177 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:15.177 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:15.177 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:15.177 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:15.177 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:15.177 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:15.177 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:15.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.155+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fc430003de0 con 0x7fc4400a06b0 2024-01-24T23:21:15.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.155+0000 7fc445041640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:15.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.155+0000 7fc445041640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:15.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.155+0000 7fc445041640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:15.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.155+0000 7fc445041640 1 -- 172.21.15.115:0/3280210775 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc424005090 con 0x7fc4400a06b0 2024-01-24T23:21:15.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.155+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7fc4300436e0 con 0x7fc4400a06b0 2024-01-24T23:21:15.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.155+0000 7fc43e59c640 10 monclient: get_auth_request con 0x7fc420037ab0 auth_method 0 2024-01-24T23:21:15.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.159+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fc430014820 con 0x7fc4400a06b0 2024-01-24T23:21:15.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.159+0000 7fc4377fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:15.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.159+0000 7fc4377fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:15.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.303+0000 7fc445041640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-01-24T23:21:15.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.303+0000 7fc445041640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-01-24T23:21:15.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.303+0000 7fc445041640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:15.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.303+0000 7fc445041640 1 -- 172.21.15.115:0/3280210775 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc4240058c0 con 0x7fc4400a06b0 2024-01-24T23:21:15.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.303+0000 7fc4377fe640 1 -- 172.21.15.115:0/3280210775 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v15) v1 ==== 92+0+10 (unknown 946793192 0 265718887) 0x7fc430012ec0 con 0x7fc4400a06b0 2024-01-24T23:21:15.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.303+0000 7fc4377fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-01-24T23:21:15.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.303+0000 7fc4377fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:15.309 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 8 2024-01-24T23:21:15.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.307+0000 7fc445041640 1 -- 172.21.15.115:0/3280210775 >> v1:172.21.15.132:6812/1835401922 conn(0x7fc420037ab0 legacy=0x7fc420039f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:15.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.307+0000 7fc445041640 10 monclient: shutdown 2024-01-24T23:21:15.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.307+0000 7fc445041640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:15.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.307+0000 7fc445041640 1 -- 172.21.15.115:0/3280210775 >> v1:172.21.15.115:6793/0 conn(0x7fc4400a06b0 legacy=0x7fc4401bbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:15.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.307+0000 7fc445041640 1 -- 172.21.15.115:0/3280210775 shutdown_connections 2024-01-24T23:21:15.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.307+0000 7fc445041640 1 -- 172.21.15.115:0/3280210775 >> 172.21.15.115:0/3280210775 conn(0x7fc4400a9ba0 msgr2=0x7fc4400a2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:15.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.307+0000 7fc445041640 1 -- 172.21.15.115:0/3280210775 shutdown_connections 2024-01-24T23:21:15.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.307+0000 7fc445041640 1 -- 172.21.15.115:0/3280210775 wait complete. 2024-01-24T23:21:15.325 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-24T23:21:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.439+0000 7fc2f6fa1640 10 monclient: get_monmap_and_config 2024-01-24T23:21:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.439+0000 7fc2f6fa1640 10 monclient: build_initial_monmap 2024-01-24T23:21:15.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.439+0000 7fc2f6fa1640 10 monclient: monmap: 2024-01-24T23:21:15.445 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:15.445 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:15.445 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:15.446093+0000 2024-01-24T23:21:15.445 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:15.446093+0000 2024-01-24T23:21:15.445 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:15.445 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:15.445 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:15.445 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:15.446 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:15.446 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:15.446 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:15.446 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:15.446 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:15.446 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:15.446 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:15.446 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 1 Processor -- start 2024-01-24T23:21:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 1 -- start start 2024-01-24T23:21:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 10 monclient: init 2024-01-24T23:21:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 10 monclient: _add_conns ranks=[8,5,4,3,2,6,7,0,1] 2024-01-24T23:21:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 10 monclient(hunting): picked mon.noname-e con 0x7fc2f009ff10 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 10 monclient(hunting): picked mon.noname-h con 0x7fc2f006ae80 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 10 monclient(hunting): picked mon.noname-c con 0x7fc2f00a02f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f004ed70 con 0x7fc2f00a02f0 2024-01-24T23:21:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f00a3890 con 0x7fc2f006ae80 2024-01-24T23:21:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f006ac40 con 0x7fc2f009ff10 2024-01-24T23:21:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 10 monclient(hunting): authenticate will time out at 954.368981s 2024-01-24T23:21:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f4d3d640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fc2f009ff10 0x7fc2f00a2a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:48512/0 (socket says 172.21.15.115:48512) 2024-01-24T23:21:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f553e640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fc2f00a02f0 0x7fc2f013fc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:59922/0 (socket says 172.21.15.115:59922) 2024-01-24T23:21:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f4d3d640 1 -- 172.21.15.115:0/3699765934 learned_addr learned my addr 172.21.15.115:0/3699765934 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 1 -- 172.21.15.115:0/3699765934 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2039270337 0 0) 0x7fc2f004ed70 con 0x7fc2f00a02f0 2024-01-24T23:21:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 10 monclient(hunting): my global_id is 74434 2024-01-24T23:21:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 1 -- 172.21.15.115:0/3699765934 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2d8003530 con 0x7fc2f00a02f0 2024-01-24T23:21:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 1 -- 172.21.15.115:0/3699765934 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3389030306 0 0) 0x7fc2f006ac40 con 0x7fc2f009ff10 2024-01-24T23:21:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:15.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:15.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 10 monclient(hunting): my global_id is 74375 2024-01-24T23:21:15.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 1 -- 172.21.15.115:0/3699765934 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2f004ed70 con 0x7fc2f009ff10 2024-01-24T23:21:15.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 1 -- 172.21.15.115:0/3699765934 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 319906246 0 0) 0x7fc2d8003530 con 0x7fc2f00a02f0 2024-01-24T23:21:15.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 1 -- 172.21.15.115:0/3699765934 >> v1:172.21.15.115:6793/0 conn(0x7fc2f009ff10 legacy=0x7fc2f00a2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:15.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 1 -- 172.21.15.115:0/3699765934 >> v1:172.21.15.132:6791/0 conn(0x7fc2f006ae80 legacy=0x7fc2f00a3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:15.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:21:15.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 1 monclient: found mon.noname-c 2024-01-24T23:21:15.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:15.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:21:15.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 1 -- 172.21.15.115:0/3699765934 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f009e9e0 con 0x7fc2f00a02f0 2024-01-24T23:21:15.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 10 monclient: _finish_auth 0 2024-01-24T23:21:15.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:21:15.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:15.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 1 -- 172.21.15.115:0/3699765934 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc2e8001b40 con 0x7fc2f00a02f0 2024-01-24T23:21:15.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 5 monclient: authenticate success, global_id 74434 2024-01-24T23:21:15.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:15.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:15.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:21:15.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 10 monclient: dump: 2024-01-24T23:21:15.456 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:15.456 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:15.456 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.456 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.456 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:15.456 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:15.456 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:15.456 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:15.457 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:15.457 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:15.457 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:15.457 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:15.457 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:15.457 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:15.457 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:15.457 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:15.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:15.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 1 -- 172.21.15.115:0/3699765934 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc2e8001d60 con 0x7fc2f00a02f0 2024-01-24T23:21:15.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:15.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:15.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 1 -- 172.21.15.115:0/3699765934 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc2e8004eb0 con 0x7fc2f00a02f0 2024-01-24T23:21:15.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 4 set_mon_vals no callback set 2024-01-24T23:21:15.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:15.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:15.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2ef7fe640 10 monclient: dump: 2024-01-24T23:21:15.458 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:15.458 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:15.458 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.458 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.458 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:15.458 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:15.458 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:15.458 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:15.458 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:15.458 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:15.459 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:15.459 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:15.459 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:15.459 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:15.459 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:15.459 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:15.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 10 monclient: shutdown 2024-01-24T23:21:15.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:15.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 1 -- 172.21.15.115:0/3699765934 >> v1:172.21.15.115:6791/0 conn(0x7fc2f00a02f0 legacy=0x7fc2f013fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:15.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 1 -- 172.21.15.115:0/3699765934 shutdown_connections 2024-01-24T23:21:15.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 1 -- 172.21.15.115:0/3699765934 >> 172.21.15.115:0/3699765934 conn(0x7fc2f00a7820 msgr2=0x7fc2f00a3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:15.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 1 -- 172.21.15.115:0/3699765934 shutdown_connections 2024-01-24T23:21:15.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 1 -- 172.21.15.115:0/3699765934 wait complete. 2024-01-24T23:21:15.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 10 monclient: build_initial_monmap 2024-01-24T23:21:15.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 10 monclient: monmap: 2024-01-24T23:21:15.460 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:15.460 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:15.460 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:15.460 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:15.460 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:15.460 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:15.460 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:15.460 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:15.460 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:15.461 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 1 Processor -- start 2024-01-24T23:21:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 1 -- start start 2024-01-24T23:21:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.443+0000 7fc2f6fa1640 10 monclient: init 2024-01-24T23:21:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2f6fa1640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2f6fa1640 10 monclient: _add_conns ranks=[7,4,3,0,1,6,2,8,5] 2024-01-24T23:21:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2f6fa1640 10 monclient(hunting): picked mon.noname-i con 0x7fc2f006ae80 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2f6fa1640 10 monclient(hunting): picked mon.noname-c con 0x7fc2f009ff10 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2f6fa1640 10 monclient(hunting): picked mon.noname-g con 0x7fc2f00a02f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2f6fa1640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f01ac200 con 0x7fc2f00a02f0 2024-01-24T23:21:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2f6fa1640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f01aad50 con 0x7fc2f009ff10 2024-01-24T23:21:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2f6fa1640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f006b5b0 con 0x7fc2f006ae80 2024-01-24T23:21:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2f6fa1640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2f6fa1640 10 monclient(hunting): authenticate will time out at 954.372285s 2024-01-24T23:21:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2effff640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fc2f009ff10 0x7fc2f01bf230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:59930/0 (socket says 172.21.15.115:59930) 2024-01-24T23:21:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2effff640 1 -- 172.21.15.115:0/728989256 learned_addr learned my addr 172.21.15.115:0/728989256 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 1 -- 172.21.15.115:0/728989256 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881070944 0 0) 0x7fc2f01aad50 con 0x7fc2f009ff10 2024-01-24T23:21:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 10 monclient(hunting): my global_id is 74443 2024-01-24T23:21:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 1 -- 172.21.15.115:0/728989256 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2dc003530 con 0x7fc2f009ff10 2024-01-24T23:21:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 1 -- 172.21.15.115:0/728989256 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1619857788 0 0) 0x7fc2dc003530 con 0x7fc2f009ff10 2024-01-24T23:21:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 1 -- 172.21.15.115:0/728989256 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc2f01aad50 con 0x7fc2f009ff10 2024-01-24T23:21:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 1 -- 172.21.15.115:0/728989256 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc2e0001b90 con 0x7fc2f009ff10 2024-01-24T23:21:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:21:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 10 monclient(hunting): dump: 2024-01-24T23:21:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:15.464 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 1 -- 172.21.15.115:0/728989256 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2022416900 0 0) 0x7fc2f01ac200 con 0x7fc2f00a02f0 2024-01-24T23:21:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 10 monclient(hunting): my global_id is 4395 2024-01-24T23:21:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 1 -- 172.21.15.115:0/728989256 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2dc003530 con 0x7fc2f00a02f0 2024-01-24T23:21:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 1 -- 172.21.15.115:0/728989256 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3816994110 0 0) 0x7fc2f006b5b0 con 0x7fc2f006ae80 2024-01-24T23:21:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 10 monclient(hunting): my global_id is 4453 2024-01-24T23:21:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 1 -- 172.21.15.115:0/728989256 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2f01ac200 con 0x7fc2f006ae80 2024-01-24T23:21:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 1 -- 172.21.15.115:0/728989256 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 633053535 0 0) 0x7fc2f01aad50 con 0x7fc2f009ff10 2024-01-24T23:21:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 1 -- 172.21.15.115:0/728989256 >> v1:172.21.15.132:6792/0 conn(0x7fc2f006ae80 legacy=0x7fc2f01beb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 1 -- 172.21.15.115:0/728989256 >> v1:172.21.15.132:6790/0 conn(0x7fc2f00a02f0 legacy=0x7fc2f01d32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 10 monclient: _finish_hunting 0 2024-01-24T23:21:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 1 monclient: found mon.c 2024-01-24T23:21:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 1 -- 172.21.15.115:0/728989256 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f01abf20 con 0x7fc2f009ff10 2024-01-24T23:21:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 10 monclient: _finish_auth 0 2024-01-24T23:21:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:21:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2f6fa1640 5 monclient: authenticate success, global_id 74443 2024-01-24T23:21:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2f6fa1640 10 monclient: _renew_subs 2024-01-24T23:21:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2f6fa1640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2f6fa1640 1 -- 172.21.15.115:0/728989256 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f01aaaa0 con 0x7fc2f009ff10 2024-01-24T23:21:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2f6fa1640 10 monclient: _renew_subs 2024-01-24T23:21:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2f6fa1640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2f6fa1640 1 -- 172.21.15.115:0/728989256 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f01cb3d0 con 0x7fc2f009ff10 2024-01-24T23:21:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 1 -- 172.21.15.115:0/728989256 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc2e0003950 con 0x7fc2f009ff10 2024-01-24T23:21:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 1 -- 172.21.15.115:0/728989256 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc2e0003e00 con 0x7fc2f009ff10 2024-01-24T23:21:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2eeffd640 4 set_mon_vals no callback set 2024-01-24T23:21:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 10 monclient: dump: 2024-01-24T23:21:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:15.468 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2f6fa1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2f6fa1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2f6fa1640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 1 -- 172.21.15.115:0/728989256 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fc2e0003f60 con 0x7fc2f009ff10 2024-01-24T23:21:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2f6fa1640 1 -- 172.21.15.115:0/728989256 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d0005090 con 0x7fc2f009ff10 2024-01-24T23:21:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2ed7fa640 1 -- 172.21.15.115:0/728989256 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7fc2e0043830 con 0x7fc2f009ff10 2024-01-24T23:21:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.447+0000 7fc2f4d3d640 10 monclient: get_auth_request con 0x7fc2dc0371e0 auth_method 0 2024-01-24T23:21:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.451+0000 7fc2ed7fa640 1 -- 172.21.15.115:0/728989256 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fc2e0014ba0 con 0x7fc2f009ff10 2024-01-24T23:21:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.451+0000 7fc2ed7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.451+0000 7fc2ed7fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:15.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.603+0000 7fc2f6fa1640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-01-24T23:21:15.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.603+0000 7fc2f6fa1640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-01-24T23:21:15.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.603+0000 7fc2f6fa1640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:15.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.603+0000 7fc2f6fa1640 1 -- 172.21.15.115:0/728989256 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc2d00058c0 con 0x7fc2f009ff10 2024-01-24T23:21:15.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.607+0000 7fc2ed7fa640 1 -- 172.21.15.115:0/728989256 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v15) v1 ==== 93+0+10 (unknown 554344602 0 2170607528) 0x7fc2e0013010 con 0x7fc2f009ff10 2024-01-24T23:21:15.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.607+0000 7fc2ed7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-01-24T23:21:15.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.607+0000 7fc2ed7fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:15.610 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2024-01-24T23:21:15.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.607+0000 7fc2f6fa1640 1 -- 172.21.15.115:0/728989256 >> v1:172.21.15.132:6812/1835401922 conn(0x7fc2dc0371e0 legacy=0x7fc2dc0396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:15.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.607+0000 7fc2f6fa1640 10 monclient: shutdown 2024-01-24T23:21:15.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.607+0000 7fc2f6fa1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:15.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.607+0000 7fc2f6fa1640 1 -- 172.21.15.115:0/728989256 >> v1:172.21.15.115:6791/0 conn(0x7fc2f009ff10 legacy=0x7fc2f01bf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:15.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.607+0000 7fc2f6fa1640 1 -- 172.21.15.115:0/728989256 shutdown_connections 2024-01-24T23:21:15.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.607+0000 7fc2f6fa1640 1 -- 172.21.15.115:0/728989256 >> 172.21.15.115:0/728989256 conn(0x7fc2f00a7820 msgr2=0x7fc2f00a6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:15.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.607+0000 7fc2f6fa1640 1 -- 172.21.15.115:0/728989256 shutdown_connections 2024-01-24T23:21:15.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.607+0000 7fc2f6fa1640 1 -- 172.21.15.115:0/728989256 wait complete. 2024-01-24T23:21:15.628 INFO:tasks.mon_thrash.ceph_manager:waiting for clean 2024-01-24T23:21:15.628 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T23:21:15.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 10 monclient: get_monmap_and_config 2024-01-24T23:21:15.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 10 monclient: build_initial_monmap 2024-01-24T23:21:15.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 10 monclient: monmap: 2024-01-24T23:21:15.734 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:15.734 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:15.735 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:15.735518+0000 2024-01-24T23:21:15.735 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:15.735518+0000 2024-01-24T23:21:15.735 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:15.735 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:15.735 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:15.735 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:15.735 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:15.735 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:15.735 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:15.735 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:15.735 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:15.735 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:15.735 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:15.735 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:15.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 1 Processor -- start 2024-01-24T23:21:15.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 1 -- start start 2024-01-24T23:21:15.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 10 monclient: init 2024-01-24T23:21:15.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:15.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 10 monclient: _add_conns ranks=[6,4,8,7,3,5,2,0,1] 2024-01-24T23:21:15.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 10 monclient(hunting): picked mon.noname-d con 0x7fa7bc133bf0 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:15.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 10 monclient(hunting): picked mon.noname-c con 0x7fa7bc133fd0 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:15.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 10 monclient(hunting): picked mon.noname-e con 0x7fa7bc1343b0 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:15.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7bc04ed70 con 0x7fa7bc133fd0 2024-01-24T23:21:15.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7bc0a4b60 con 0x7fa7bc133bf0 2024-01-24T23:21:15.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7bc09ae50 con 0x7fa7bc1343b0 2024-01-24T23:21:15.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:15.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 10 monclient(hunting): authenticate will time out at 954.658035s 2024-01-24T23:21:15.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7baffd640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fa7bc133fd0 0x7fa7bc13bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:59944/0 (socket says 172.21.15.115:59944) 2024-01-24T23:21:15.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7baffd640 1 -- 172.21.15.115:0/782577783 learned_addr learned my addr 172.21.15.115:0/782577783 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:15.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7bb7fe640 1 --1- 172.21.15.115:0/782577783 >> v1:172.21.15.115:6792/0 conn(0x7fa7bc133bf0 0x7fa7bc09bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:39650/0 (socket says 172.21.15.115:39650) 2024-01-24T23:21:15.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 1 -- 172.21.15.115:0/782577783 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3877934254 0 0) 0x7fa7bc04ed70 con 0x7fa7bc133fd0 2024-01-24T23:21:15.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:15.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:15.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:15.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 10 monclient(hunting): my global_id is 74452 2024-01-24T23:21:15.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 1 -- 172.21.15.115:0/782577783 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7a4003530 con 0x7fa7bc133fd0 2024-01-24T23:21:15.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 1 -- 172.21.15.115:0/782577783 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804662675 0 0) 0x7fa7bc0a4b60 con 0x7fa7bc133bf0 2024-01-24T23:21:15.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:15.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:15.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:15.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 10 monclient(hunting): my global_id is 4416 2024-01-24T23:21:15.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 1 -- 172.21.15.115:0/782577783 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7bc04ed70 con 0x7fa7bc133bf0 2024-01-24T23:21:15.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 1 -- 172.21.15.115:0/782577783 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152097605 0 0) 0x7fa7bc09ae50 con 0x7fa7bc1343b0 2024-01-24T23:21:15.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:15.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:15.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:15.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 10 monclient(hunting): my global_id is 74384 2024-01-24T23:21:15.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 1 -- 172.21.15.115:0/782577783 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7bc0a4b60 con 0x7fa7bc1343b0 2024-01-24T23:21:15.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 1 -- 172.21.15.115:0/782577783 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3584074705 0 0) 0x7fa7a4003530 con 0x7fa7bc133fd0 2024-01-24T23:21:15.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 1 -- 172.21.15.115:0/782577783 >> v1:172.21.15.115:6793/0 conn(0x7fa7bc1343b0 legacy=0x7fa7bc13f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:15.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 1 -- 172.21.15.115:0/782577783 >> v1:172.21.15.115:6792/0 conn(0x7fa7bc133bf0 legacy=0x7fa7bc09bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:15.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:21:15.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 1 monclient: found mon.noname-c 2024-01-24T23:21:15.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:15.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:21:15.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 1 -- 172.21.15.115:0/782577783 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc09b130 con 0x7fa7bc133fd0 2024-01-24T23:21:15.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 10 monclient: _finish_auth 0 2024-01-24T23:21:15.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:21:15.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:15.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 5 monclient: authenticate success, global_id 74452 2024-01-24T23:21:15.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 1 -- 172.21.15.115:0/782577783 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa7b00017c0 con 0x7fa7bc133fd0 2024-01-24T23:21:15.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:15.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:21:15.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 10 monclient: dump: 2024-01-24T23:21:15.745 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:15.745 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:15.745 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.745 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.745 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:15.745 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:15.745 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:15.745 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:15.745 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:15.745 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:15.745 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:15.745 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:15.745 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:15.746 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:15.746 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:15.746 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:15.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:15.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 1 -- 172.21.15.115:0/782577783 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa7b0001a20 con 0x7fa7bc133fd0 2024-01-24T23:21:15.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:15.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 1 -- 172.21.15.115:0/782577783 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa7b0004bf0 con 0x7fa7bc133fd0 2024-01-24T23:21:15.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:15.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:15.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7ba7fc640 10 monclient: dump: 2024-01-24T23:21:15.746 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:15.746 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:15.746 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.746 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.746 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:15.746 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:15.746 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:15.747 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:15.747 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:15.747 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:15.747 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:15.747 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:15.747 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:15.747 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:15.747 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:15.747 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:15.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:15.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 4 set_mon_vals no callback set 2024-01-24T23:21:15.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 10 monclient: shutdown 2024-01-24T23:21:15.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:15.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 1 -- 172.21.15.115:0/782577783 >> v1:172.21.15.115:6791/0 conn(0x7fa7bc133fd0 legacy=0x7fa7bc13bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:15.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 1 -- 172.21.15.115:0/782577783 shutdown_connections 2024-01-24T23:21:15.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 1 -- 172.21.15.115:0/782577783 >> 172.21.15.115:0/782577783 conn(0x7fa7bc0662f0 msgr2=0x7fa7bc066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 1 -- 172.21.15.115:0/782577783 shutdown_connections 2024-01-24T23:21:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.731+0000 7fa7c1f47640 1 -- 172.21.15.115:0/782577783 wait complete. 2024-01-24T23:21:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 10 monclient: build_initial_monmap 2024-01-24T23:21:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 10 monclient: monmap: 2024-01-24T23:21:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:15.749 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:15.749 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:15.749 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:15.749 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:15.749 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:15.749 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:15.749 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:15.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 1 Processor -- start 2024-01-24T23:21:15.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 1 -- start start 2024-01-24T23:21:15.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 10 monclient: init 2024-01-24T23:21:15.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:15.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 10 monclient: _add_conns ranks=[4,2,8,6,7,3,0,5,1] 2024-01-24T23:21:15.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 10 monclient(hunting): picked mon.noname-c con 0x7fa7bc133bf0 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:15.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 10 monclient(hunting): picked mon.noname-b con 0x7fa7bc133fd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:15.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 10 monclient(hunting): picked mon.noname-e con 0x7fa7bc1343b0 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:15.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7bc1335b0 con 0x7fa7bc133fd0 2024-01-24T23:21:15.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7bc0a5990 con 0x7fa7bc133bf0 2024-01-24T23:21:15.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7bc120780 con 0x7fa7bc1343b0 2024-01-24T23:21:15.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:15.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 10 monclient(hunting): authenticate will time out at 954.661060s 2024-01-24T23:21:15.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7bb7fe640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fa7bc133bf0 0x7fa7bc0a0c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:59950/0 (socket says 172.21.15.115:59950) 2024-01-24T23:21:15.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7bb7fe640 1 -- 172.21.15.115:0/2240943452 learned_addr learned my addr 172.21.15.115:0/2240943452 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:15.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1790966913 0 0) 0x7fa7bc0a5990 con 0x7fa7bc133bf0 2024-01-24T23:21:15.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:15.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:15.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:15.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient(hunting): my global_id is 74461 2024-01-24T23:21:15.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7a8003530 con 0x7fa7bc133bf0 2024-01-24T23:21:15.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 379016105 0 0) 0x7fa7bc120780 con 0x7fa7bc1343b0 2024-01-24T23:21:15.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:15.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:15.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:15.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient(hunting): my global_id is 74393 2024-01-24T23:21:15.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7bc0a5990 con 0x7fa7bc1343b0 2024-01-24T23:21:15.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019253551 0 0) 0x7fa7bc1335b0 con 0x7fa7bc133fd0 2024-01-24T23:21:15.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:15.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:15.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:15.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient(hunting): my global_id is 4376 2024-01-24T23:21:15.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7bc120780 con 0x7fa7bc133fd0 2024-01-24T23:21:15.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 318801616 0 0) 0x7fa7bc0a5990 con 0x7fa7bc1343b0 2024-01-24T23:21:15.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa7bc1335b0 con 0x7fa7bc1343b0 2024-01-24T23:21:15.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa7b4001b30 con 0x7fa7bc1343b0 2024-01-24T23:21:15.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:15.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:21:15.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient(hunting): dump: 2024-01-24T23:21:15.752 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:15.752 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:15.752 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.752 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.752 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:15.752 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:15.752 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:15.752 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:15.752 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:15.752 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:15.752 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:15.752 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:15.752 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:15.752 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:15.753 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:15.753 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:15.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:15.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 179911130 0 0) 0x7fa7bc120780 con 0x7fa7bc133fd0 2024-01-24T23:21:15.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa7bc0a5990 con 0x7fa7bc133fd0 2024-01-24T23:21:15.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 18684600 0 0) 0x7fa7a8003530 con 0x7fa7bc133bf0 2024-01-24T23:21:15.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa7bc120780 con 0x7fa7bc133bf0 2024-01-24T23:21:15.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa7b00017c0 con 0x7fa7bc133fd0 2024-01-24T23:21:15.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:15.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:15.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient(hunting): dump: 2024-01-24T23:21:15.753 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:15.753 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:15.753 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.753 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:15.754 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa7ac001330 con 0x7fa7bc133bf0 2024-01-24T23:21:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient(hunting): dump: 2024-01-24T23:21:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:15.756 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:15.756 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:15.756 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:15.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3015011949 0 0) 0x7fa7bc0a5990 con 0x7fa7bc133fd0 2024-01-24T23:21:15.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 >> v1:172.21.15.115:6793/0 conn(0x7fa7bc1343b0 legacy=0x7fa7bc1d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:15.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 >> v1:172.21.15.115:6791/0 conn(0x7fa7bc133bf0 legacy=0x7fa7bc0a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:15.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient: _finish_hunting 0 2024-01-24T23:21:15.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 1 monclient: found mon.b 2024-01-24T23:21:15.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:15.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:15.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc09c810 con 0x7fa7bc133fd0 2024-01-24T23:21:15.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient: _finish_auth 0 2024-01-24T23:21:15.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient: _check_auth_tickets 2024-01-24T23:21:15.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:15.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 5 monclient: authenticate success, global_id 4376 2024-01-24T23:21:15.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 10 monclient: _renew_subs 2024-01-24T23:21:15.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:15.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 1 -- 172.21.15.115:0/2240943452 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7bc0a25c0 con 0x7fa7bc133fd0 2024-01-24T23:21:15.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 10 monclient: _renew_subs 2024-01-24T23:21:15.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:15.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 1 -- 172.21.15.115:0/2240943452 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7bc11f760 con 0x7fa7bc133fd0 2024-01-24T23:21:15.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa7b0004ec0 con 0x7fa7bc133fd0 2024-01-24T23:21:15.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:15.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa7b0005370 con 0x7fa7bc133fd0 2024-01-24T23:21:15.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:15.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7b9ffb640 4 set_mon_vals no callback set 2024-01-24T23:21:15.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:15.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 10 monclient: dump: 2024-01-24T23:21:15.757 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:15.758 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fa7b000fdf0 con 0x7fa7bc133fd0 2024-01-24T23:21:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7c1f47640 1 -- 172.21.15.115:0/2240943452 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa79c005090 con 0x7fa7bc133fd0 2024-01-24T23:21:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7fa7b00438c0 con 0x7fa7bc133fd0 2024-01-24T23:21:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.735+0000 7fa7bb7fe640 10 monclient: get_auth_request con 0x7fa7a8037ca0 auth_method 0 2024-01-24T23:21:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.743+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 <== mon.2 v1:172.21.15.115:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fa7b0014ac0 con 0x7fa7bc133fd0 2024-01-24T23:21:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.743+0000 7fa7a3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.743+0000 7fa7a3fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:15.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.887+0000 7fa7c1f47640 1 -- 172.21.15.115:0/2240943452 --> v1:172.21.15.132:6812/1835401922 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa79c000ba0 con 0x7fa7a8037ca0 2024-01-24T23:21:15.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.891+0000 7fa7a3fff640 1 -- 172.21.15.115:0/2240943452 <== mgr.4113 v1:172.21.15.132:6812/1835401922 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+40608 (unknown 2965378022 0 895350019) 0x7fa79c000ba0 con 0x7fa7a8037ca0 2024-01-24T23:21:15.896 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:21:15.896 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-01-24T23:21:15.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.895+0000 7fa7c1f47640 1 -- 172.21.15.115:0/2240943452 >> v1:172.21.15.132:6812/1835401922 conn(0x7fa7a8037ca0 legacy=0x7fa7a803a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:15.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.895+0000 7fa7c1f47640 10 monclient: shutdown 2024-01-24T23:21:15.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.895+0000 7fa7c1f47640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:15.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.895+0000 7fa7c1f47640 1 -- 172.21.15.115:0/2240943452 >> v1:172.21.15.115:6790/0 conn(0x7fa7bc133fd0 legacy=0x7fa7bc0a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:15.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.895+0000 7fa7c1f47640 1 -- 172.21.15.115:0/2240943452 shutdown_connections 2024-01-24T23:21:15.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.895+0000 7fa7c1f47640 1 -- 172.21.15.115:0/2240943452 >> 172.21.15.115:0/2240943452 conn(0x7fa7bc0662f0 msgr2=0x7fa7bc0a7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:15.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.895+0000 7fa7c1f47640 1 -- 172.21.15.115:0/2240943452 shutdown_connections 2024-01-24T23:21:15.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:15.895+0000 7fa7c1f47640 1 -- 172.21.15.115:0/2240943452 wait complete. 2024-01-24T23:21:15.913 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-01-24T23:21:15.831682+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163060,"kb_used_data":884,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067980,"statfs":{"total":579820584960,"available":579653611520,"internally_reserved":0,"allocated":905216,"data_stored":1224326,"data_compressed":57630,"data_compressed_allocated":77824,"data_compressed_original":962560,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001200"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.012747+0000","last_change":"2024-01-24T23:21:08.941679+0000","last_active":"2024-01-24T23:21:10.012747+0000","last_peered":"2024-01-24T23:21:10.012747+0000","last_clean":"2024-01-24T23:21:10.012747+0000","last_became_active":"2024-01-24T23:21:08.941408+0000","last_became_peered":"2024-01-24T23:21:08.941408+0000","last_unstale":"2024-01-24T23:21:10.012747+0000","last_undegraded":"2024-01-24T23:21:10.012747+0000","last_fullsized":"2024-01-24T23:21:10.012747+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T10:59:01.658977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.012767+0000","last_change":"2024-01-24T23:21:08.949886+0000","last_active":"2024-01-24T23:21:10.012767+0000","last_peered":"2024-01-24T23:21:10.012767+0000","last_clean":"2024-01-24T23:21:10.012767+0000","last_became_active":"2024-01-24T23:21:08.949572+0000","last_became_peered":"2024-01-24T23:21:08.949572+0000","last_unstale":"2024-01-24T23:21:10.012767+0000","last_undegraded":"2024-01-24T23:21:10.012767+0000","last_fullsized":"2024-01-24T23:21:10.012767+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T10:40:08.936213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.011232+0000","last_change":"2024-01-24T23:21:08.941915+0000","last_active":"2024-01-24T23:21:10.011232+0000","last_peered":"2024-01-24T23:21:10.011232+0000","last_clean":"2024-01-24T23:21:10.011232+0000","last_became_active":"2024-01-24T23:21:08.941537+0000","last_became_peered":"2024-01-24T23:21:08.941537+0000","last_unstale":"2024-01-24T23:21:10.011232+0000","last_undegraded":"2024-01-24T23:21:10.011232+0000","last_fullsized":"2024-01-24T23:21:10.011232+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T03:58:22.010129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.017040+0000","last_change":"2024-01-24T23:21:08.943234+0000","last_active":"2024-01-24T23:21:10.017040+0000","last_peered":"2024-01-24T23:21:10.017040+0000","last_clean":"2024-01-24T23:21:10.017040+0000","last_became_active":"2024-01-24T23:21:08.942798+0000","last_became_peered":"2024-01-24T23:21:08.942798+0000","last_unstale":"2024-01-24T23:21:10.017040+0000","last_undegraded":"2024-01-24T23:21:10.017040+0000","last_fullsized":"2024-01-24T23:21:10.017040+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T01:56:21.645104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":161,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.952984+0000","last_change":"2024-01-24T23:21:08.940580+0000","last_active":"2024-01-24T23:21:10.952984+0000","last_peered":"2024-01-24T23:21:10.952984+0000","last_clean":"2024-01-24T23:21:10.952984+0000","last_became_active":"2024-01-24T23:21:08.940211+0000","last_became_peered":"2024-01-24T23:21:08.940211+0000","last_unstale":"2024-01-24T23:21:10.952984+0000","last_undegraded":"2024-01-24T23:21:10.952984+0000","last_fullsized":"2024-01-24T23:21:10.952984+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T05:41:59.190049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.686443+0000","last_change":"2024-01-24T23:21:08.951610+0000","last_active":"2024-01-24T23:21:10.686443+0000","last_peered":"2024-01-24T23:21:10.686443+0000","last_clean":"2024-01-24T23:21:10.686443+0000","last_became_active":"2024-01-24T23:21:08.950717+0000","last_became_peered":"2024-01-24T23:21:08.950717+0000","last_unstale":"2024-01-24T23:21:10.686443+0000","last_undegraded":"2024-01-24T23:21:10.686443+0000","last_fullsized":"2024-01-24T23:21:10.686443+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T00:26:13.689261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.011863+0000","last_change":"2024-01-24T23:21:08.943904+0000","last_active":"2024-01-24T23:21:10.011863+0000","last_peered":"2024-01-24T23:21:10.011863+0000","last_clean":"2024-01-24T23:21:10.011863+0000","last_became_active":"2024-01-24T23:21:08.943567+0000","last_became_peered":"2024-01-24T23:21:08.943567+0000","last_unstale":"2024-01-24T23:21:10.011863+0000","last_undegraded":"2024-01-24T23:21:10.011863+0000","last_fullsized":"2024-01-24T23:21:10.011863+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T07:45:59.231652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.011220+0000","last_change":"2024-01-24T23:21:08.949886+0000","last_active":"2024-01-24T23:21:10.011220+0000","last_peered":"2024-01-24T23:21:10.011220+0000","last_clean":"2024-01-24T23:21:10.011220+0000","last_became_active":"2024-01-24T23:21:08.949580+0000","last_became_peered":"2024-01-24T23:21:08.949580+0000","last_unstale":"2024-01-24T23:21:10.011220+0000","last_undegraded":"2024-01-24T23:21:10.011220+0000","last_fullsized":"2024-01-24T23:21:10.011220+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T02:18:56.755337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.012731+0000","last_change":"2024-01-24T23:21:08.944072+0000","last_active":"2024-01-24T23:21:10.012731+0000","last_peered":"2024-01-24T23:21:10.012731+0000","last_clean":"2024-01-24T23:21:10.012731+0000","last_became_active":"2024-01-24T23:21:08.943616+0000","last_became_peered":"2024-01-24T23:21:08.943616+0000","last_unstale":"2024-01-24T23:21:10.012731+0000","last_undegraded":"2024-01-24T23:21:10.012731+0000","last_fullsized":"2024-01-24T23:21:10.012731+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T07:20:03.852420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1057856,"data_compressed":60590,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640259,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":192512,"data_stored":566913,"data_compressed":30295,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]}]},{"osd":4,"up_from":11,"seq":47244640259,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":131072,"data_stored":37985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]}]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":131072,"data_stored":37985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]}]},{"osd":2,"up_from":11,"seq":47244640259,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":188416,"data_stored":505473,"data_compressed":27335,"data_compressed_allocated":36864,"data_compressed_original":450560,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]},{"osd":1,"up_from":11,"seq":47244640260,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":131072,"data_stored":37985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]}]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":131072,"data_stored":37985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":30295,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":30295,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T23:21:15.914 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T23:21:16.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6c5318640 10 monclient: get_monmap_and_config 2024-01-24T23:21:16.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6c5318640 10 monclient: build_initial_monmap 2024-01-24T23:21:16.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6c5318640 10 monclient: monmap: 2024-01-24T23:21:16.019 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:16.019 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.019 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:16.020441+0000 2024-01-24T23:21:16.019 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:16.020441+0000 2024-01-24T23:21:16.019 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:16.019 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.019 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:16.019 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:16.019 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:16.020 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:16.020 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:16.020 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:16.020 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:16.020 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:16.020 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:16.020 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6c5318640 1 Processor -- start 2024-01-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6c5318640 1 -- start start 2024-01-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6c5318640 10 monclient: init 2024-01-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6c5318640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6c5318640 10 monclient: _add_conns ranks=[2,3,7,1,4,6,0,5,8] 2024-01-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6c5318640 10 monclient(hunting): picked mon.noname-b con 0x7fd6c00720b0 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6c5318640 10 monclient(hunting): picked mon.noname-g con 0x7fd6c009b9f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6c5318640 10 monclient(hunting): picked mon.noname-i con 0x7fd6c0139f10 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6c5318640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c004ed70 con 0x7fd6c00720b0 2024-01-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6c5318640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c00a0eb0 con 0x7fd6c009b9f0 2024-01-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6c5318640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c009d020 con 0x7fd6c0139f10 2024-01-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6c5318640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6c5318640 10 monclient(hunting): authenticate will time out at 954.943042s 2024-01-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6beffd640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fd6c00720b0 0x7fd6c00a2100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:55466/0 (socket says 172.21.15.115:55466) 2024-01-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6beffd640 1 -- 172.21.15.115:0/3809557554 learned_addr learned my addr 172.21.15.115:0/3809557554 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 1 -- 172.21.15.115:0/3809557554 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3501510773 0 0) 0x7fd6c004ed70 con 0x7fd6c00720b0 2024-01-24T23:21:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 10 monclient(hunting): _negotiate 2024-01-24T23:21:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 10 monclient(hunting): my global_id is 4385 2024-01-24T23:21:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 1 -- 172.21.15.115:0/3809557554 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6a8003530 con 0x7fd6c00720b0 2024-01-24T23:21:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 1 -- 172.21.15.115:0/3809557554 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3328774363 0 0) 0x7fd6a8003530 con 0x7fd6c00720b0 2024-01-24T23:21:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 1 -- 172.21.15.115:0/3809557554 >> v1:172.21.15.132:6792/0 conn(0x7fd6c0139f10 legacy=0x7fd6c013c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 1 -- 172.21.15.115:0/3809557554 >> v1:172.21.15.132:6790/0 conn(0x7fd6c009b9f0 legacy=0x7fd6c01387f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 10 monclient: _finish_hunting 0 2024-01-24T23:21:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 1 monclient: found mon.noname-b 2024-01-24T23:21:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:21:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 1 -- 172.21.15.115:0/3809557554 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c009d300 con 0x7fd6c00720b0 2024-01-24T23:21:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 10 monclient: _finish_auth 0 2024-01-24T23:21:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 10 monclient: _check_auth_tickets 2024-01-24T23:21:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6c5318640 5 monclient: authenticate success, global_id 4385 2024-01-24T23:21:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6c5318640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 1 -- 172.21.15.115:0/3809557554 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd6b40017c0 con 0x7fd6c00720b0 2024-01-24T23:21:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:21:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 10 monclient: dump: 2024-01-24T23:21:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.026 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6c5318640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 1 -- 172.21.15.115:0/3809557554 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd6b40019e0 con 0x7fd6c00720b0 2024-01-24T23:21:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 1 -- 172.21.15.115:0/3809557554 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd6b4004bf0 con 0x7fd6c00720b0 2024-01-24T23:21:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6bdffb640 10 monclient: dump: 2024-01-24T23:21:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.027 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6c5318640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.015+0000 7fd6c5318640 4 set_mon_vals no callback set 2024-01-24T23:21:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 10 monclient: shutdown 2024-01-24T23:21:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 1 -- 172.21.15.115:0/3809557554 >> v1:172.21.15.115:6790/0 conn(0x7fd6c00720b0 legacy=0x7fd6c00a2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 1 -- 172.21.15.115:0/3809557554 shutdown_connections 2024-01-24T23:21:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 1 -- 172.21.15.115:0/3809557554 >> 172.21.15.115:0/3809557554 conn(0x7fd6c00a5730 msgr2=0x7fd6c00a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 1 -- 172.21.15.115:0/3809557554 shutdown_connections 2024-01-24T23:21:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 1 -- 172.21.15.115:0/3809557554 wait complete. 2024-01-24T23:21:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 10 monclient: build_initial_monmap 2024-01-24T23:21:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 10 monclient: monmap: 2024-01-24T23:21:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:16.029 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 1 Processor -- start 2024-01-24T23:21:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 1 -- start start 2024-01-24T23:21:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 10 monclient: init 2024-01-24T23:21:16.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:16.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 10 monclient: _add_conns ranks=[2,1,5,3,6,4,8,0,7] 2024-01-24T23:21:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 10 monclient(hunting): picked mon.noname-b con 0x7fd6c00720b0 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 10 monclient(hunting): picked mon.noname-f con 0x7fd6c009b9f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 10 monclient(hunting): picked mon.noname-h con 0x7fd6c0139f10 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c0123520 con 0x7fd6c009b9f0 2024-01-24T23:21:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c0121d20 con 0x7fd6c00720b0 2024-01-24T23:21:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c009c9f0 con 0x7fd6c0139f10 2024-01-24T23:21:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6beffd640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fd6c00720b0 0x7fd6c0134b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:55482/0 (socket says 172.21.15.115:55482) 2024-01-24T23:21:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6beffd640 1 -- 172.21.15.115:0/3740285565 learned_addr learned my addr 172.21.15.115:0/3740285565 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 10 monclient(hunting): authenticate will time out at 954.946310s 2024-01-24T23:21:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 1 -- 172.21.15.115:0/3740285565 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357982492 0 0) 0x7fd6c0121d20 con 0x7fd6c00720b0 2024-01-24T23:21:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 10 monclient(hunting): my global_id is 4394 2024-01-24T23:21:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 1 -- 172.21.15.115:0/3740285565 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6ac003530 con 0x7fd6c00720b0 2024-01-24T23:21:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 1 -- 172.21.15.115:0/3740285565 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2614867415 0 0) 0x7fd6ac003530 con 0x7fd6c00720b0 2024-01-24T23:21:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 1 -- 172.21.15.115:0/3740285565 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd6c0121d20 con 0x7fd6c00720b0 2024-01-24T23:21:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 1 -- 172.21.15.115:0/3740285565 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd6b40016c0 con 0x7fd6c00720b0 2024-01-24T23:21:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:21:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 10 monclient(hunting): dump: 2024-01-24T23:21:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.035 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 1 -- 172.21.15.115:0/3740285565 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3617963215 0 0) 0x7fd6c0121d20 con 0x7fd6c00720b0 2024-01-24T23:21:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 1 -- 172.21.15.115:0/3740285565 >> v1:172.21.15.132:6791/0 conn(0x7fd6c0139f10 legacy=0x7fd6c01d3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 1 -- 172.21.15.115:0/3740285565 >> v1:172.21.15.132:6789/0 conn(0x7fd6c009b9f0 legacy=0x7fd6c01d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 10 monclient: _finish_hunting 0 2024-01-24T23:21:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 1 monclient: found mon.b 2024-01-24T23:21:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 1 -- 172.21.15.115:0/3740285565 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c009c480 con 0x7fd6c00720b0 2024-01-24T23:21:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 10 monclient: _finish_auth 0 2024-01-24T23:21:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 10 monclient: _check_auth_tickets 2024-01-24T23:21:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 5 monclient: authenticate success, global_id 4394 2024-01-24T23:21:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 10 monclient: _renew_subs 2024-01-24T23:21:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 1 -- 172.21.15.115:0/3740285565 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c0123240 con 0x7fd6c00720b0 2024-01-24T23:21:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 10 monclient: _renew_subs 2024-01-24T23:21:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 1 -- 172.21.15.115:0/3740285565 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c0135260 con 0x7fd6c00720b0 2024-01-24T23:21:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 1 -- 172.21.15.115:0/3740285565 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd6b4004c30 con 0x7fd6c00720b0 2024-01-24T23:21:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 1 -- 172.21.15.115:0/3740285565 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd6b4005200 con 0x7fd6c00720b0 2024-01-24T23:21:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6bd7fa640 4 set_mon_vals no callback set 2024-01-24T23:21:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 10 monclient: dump: 2024-01-24T23:21:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.038 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 1 -- 172.21.15.115:0/3740285565 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fd6b400fc80 con 0x7fd6c00720b0 2024-01-24T23:21:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6c5318640 1 -- 172.21.15.115:0/3740285565 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6a0005090 con 0x7fd6c00720b0 2024-01-24T23:21:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.019+0000 7fd6a77fe640 1 -- 172.21.15.115:0/3740285565 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7fd6b4043880 con 0x7fd6c00720b0 2024-01-24T23:21:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.023+0000 7fd6be7fc640 10 monclient: get_auth_request con 0x7fd6ac036f60 auth_method 0 2024-01-24T23:21:16.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.027+0000 7fd6a77fe640 1 -- 172.21.15.115:0/3740285565 <== mon.2 v1:172.21.15.115:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fd6b40115b0 con 0x7fd6c00720b0 2024-01-24T23:21:16.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.027+0000 7fd6a77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:16.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.027+0000 7fd6a77fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:16.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.175+0000 7fd6c5318640 1 -- 172.21.15.115:0/3740285565 --> v1:172.21.15.132:6812/1835401922 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd6a0000ba0 con 0x7fd6ac036f60 2024-01-24T23:21:16.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.179+0000 7fd6a77fe640 1 -- 172.21.15.115:0/3740285565 <== mgr.4113 v1:172.21.15.132:6812/1835401922 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+40608 (unknown 2965378022 0 895350019) 0x7fd6a0000ba0 con 0x7fd6ac036f60 2024-01-24T23:21:16.183 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-01-24T23:21:16.184 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-01-24T23:21:16.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.179+0000 7fd6c5318640 1 -- 172.21.15.115:0/3740285565 >> v1:172.21.15.132:6812/1835401922 conn(0x7fd6ac036f60 legacy=0x7fd6ac039420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.179+0000 7fd6c5318640 10 monclient: shutdown 2024-01-24T23:21:16.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.179+0000 7fd6c5318640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:16.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.179+0000 7fd6c5318640 1 -- 172.21.15.115:0/3740285565 >> v1:172.21.15.115:6790/0 conn(0x7fd6c00720b0 legacy=0x7fd6c0134b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.179+0000 7fd6c5318640 1 -- 172.21.15.115:0/3740285565 shutdown_connections 2024-01-24T23:21:16.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.179+0000 7fd6c5318640 1 -- 172.21.15.115:0/3740285565 >> 172.21.15.115:0/3740285565 conn(0x7fd6c00a5730 msgr2=0x7fd6c009a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:16.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.179+0000 7fd6c5318640 1 -- 172.21.15.115:0/3740285565 shutdown_connections 2024-01-24T23:21:16.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.183+0000 7fd6c5318640 1 -- 172.21.15.115:0/3740285565 wait complete. 2024-01-24T23:21:16.201 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-01-24T23:21:15.831682+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163060,"kb_used_data":884,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067980,"statfs":{"total":579820584960,"available":579653611520,"internally_reserved":0,"allocated":905216,"data_stored":1224326,"data_compressed":57630,"data_compressed_allocated":77824,"data_compressed_original":962560,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001200"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.012747+0000","last_change":"2024-01-24T23:21:08.941679+0000","last_active":"2024-01-24T23:21:10.012747+0000","last_peered":"2024-01-24T23:21:10.012747+0000","last_clean":"2024-01-24T23:21:10.012747+0000","last_became_active":"2024-01-24T23:21:08.941408+0000","last_became_peered":"2024-01-24T23:21:08.941408+0000","last_unstale":"2024-01-24T23:21:10.012747+0000","last_undegraded":"2024-01-24T23:21:10.012747+0000","last_fullsized":"2024-01-24T23:21:10.012747+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T10:59:01.658977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.012767+0000","last_change":"2024-01-24T23:21:08.949886+0000","last_active":"2024-01-24T23:21:10.012767+0000","last_peered":"2024-01-24T23:21:10.012767+0000","last_clean":"2024-01-24T23:21:10.012767+0000","last_became_active":"2024-01-24T23:21:08.949572+0000","last_became_peered":"2024-01-24T23:21:08.949572+0000","last_unstale":"2024-01-24T23:21:10.012767+0000","last_undegraded":"2024-01-24T23:21:10.012767+0000","last_fullsized":"2024-01-24T23:21:10.012767+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T10:40:08.936213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.011232+0000","last_change":"2024-01-24T23:21:08.941915+0000","last_active":"2024-01-24T23:21:10.011232+0000","last_peered":"2024-01-24T23:21:10.011232+0000","last_clean":"2024-01-24T23:21:10.011232+0000","last_became_active":"2024-01-24T23:21:08.941537+0000","last_became_peered":"2024-01-24T23:21:08.941537+0000","last_unstale":"2024-01-24T23:21:10.011232+0000","last_undegraded":"2024-01-24T23:21:10.011232+0000","last_fullsized":"2024-01-24T23:21:10.011232+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T03:58:22.010129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.017040+0000","last_change":"2024-01-24T23:21:08.943234+0000","last_active":"2024-01-24T23:21:10.017040+0000","last_peered":"2024-01-24T23:21:10.017040+0000","last_clean":"2024-01-24T23:21:10.017040+0000","last_became_active":"2024-01-24T23:21:08.942798+0000","last_became_peered":"2024-01-24T23:21:08.942798+0000","last_unstale":"2024-01-24T23:21:10.017040+0000","last_undegraded":"2024-01-24T23:21:10.017040+0000","last_fullsized":"2024-01-24T23:21:10.017040+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T01:56:21.645104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"15'117","reported_seq":161,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.952984+0000","last_change":"2024-01-24T23:21:08.940580+0000","last_active":"2024-01-24T23:21:10.952984+0000","last_peered":"2024-01-24T23:21:10.952984+0000","last_clean":"2024-01-24T23:21:10.952984+0000","last_became_active":"2024-01-24T23:21:08.940211+0000","last_became_peered":"2024-01-24T23:21:08.940211+0000","last_unstale":"2024-01-24T23:21:10.952984+0000","last_undegraded":"2024-01-24T23:21:10.952984+0000","last_fullsized":"2024-01-24T23:21:10.952984+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T05:41:59.190049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.686443+0000","last_change":"2024-01-24T23:21:08.951610+0000","last_active":"2024-01-24T23:21:10.686443+0000","last_peered":"2024-01-24T23:21:10.686443+0000","last_clean":"2024-01-24T23:21:10.686443+0000","last_became_active":"2024-01-24T23:21:08.950717+0000","last_became_peered":"2024-01-24T23:21:08.950717+0000","last_unstale":"2024-01-24T23:21:10.686443+0000","last_undegraded":"2024-01-24T23:21:10.686443+0000","last_fullsized":"2024-01-24T23:21:10.686443+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T00:26:13.689261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.011863+0000","last_change":"2024-01-24T23:21:08.943904+0000","last_active":"2024-01-24T23:21:10.011863+0000","last_peered":"2024-01-24T23:21:10.011863+0000","last_clean":"2024-01-24T23:21:10.011863+0000","last_became_active":"2024-01-24T23:21:08.943567+0000","last_became_peered":"2024-01-24T23:21:08.943567+0000","last_unstale":"2024-01-24T23:21:10.011863+0000","last_undegraded":"2024-01-24T23:21:10.011863+0000","last_fullsized":"2024-01-24T23:21:10.011863+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T07:45:59.231652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.011220+0000","last_change":"2024-01-24T23:21:08.949886+0000","last_active":"2024-01-24T23:21:10.011220+0000","last_peered":"2024-01-24T23:21:10.011220+0000","last_clean":"2024-01-24T23:21:10.011220+0000","last_became_active":"2024-01-24T23:21:08.949580+0000","last_became_peered":"2024-01-24T23:21:08.949580+0000","last_unstale":"2024-01-24T23:21:10.011220+0000","last_undegraded":"2024-01-24T23:21:10.011220+0000","last_fullsized":"2024-01-24T23:21:10.011220+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T02:18:56.755337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-01-24T23:21:10.012731+0000","last_change":"2024-01-24T23:21:08.944072+0000","last_active":"2024-01-24T23:21:10.012731+0000","last_peered":"2024-01-24T23:21:10.012731+0000","last_clean":"2024-01-24T23:21:10.012731+0000","last_became_active":"2024-01-24T23:21:08.943616+0000","last_became_peered":"2024-01-24T23:21:08.943616+0000","last_unstale":"2024-01-24T23:21:10.012731+0000","last_undegraded":"2024-01-24T23:21:10.012731+0000","last_fullsized":"2024-01-24T23:21:10.012731+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-24T23:21:07.910203+0000","last_clean_scrub_stamp":"2024-01-24T23:21:07.910203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T07:20:03.852420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1057856,"data_compressed":60590,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640259,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":192512,"data_stored":566913,"data_compressed":30295,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]}]},{"osd":4,"up_from":11,"seq":47244640259,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":131072,"data_stored":37985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]}]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":131072,"data_stored":37985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]}]},{"osd":2,"up_from":11,"seq":47244640259,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":188416,"data_stored":505473,"data_compressed":27335,"data_compressed_allocated":36864,"data_compressed_original":450560,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]},{"osd":1,"up_from":11,"seq":47244640260,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":131072,"data_stored":37985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]}]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":131072,"data_stored":37985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":30295,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":30295,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-24T23:21:16.202 INFO:tasks.mon_thrash.ceph_manager:clean! 2024-01-24T23:21:16.202 DEBUG:tasks.mon_thrash:Yielding 2024-01-24T23:21:16.202 INFO:teuthology.run_tasks:Running task workunit... 2024-01-24T23:21:16.214 INFO:tasks.workunit:Pulling workunits from ref 663873555d45def955d1cc09730bfece2847b821 2024-01-24T23:21:16.215 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-01-24T23:21:16.215 DEBUG:teuthology.orchestra.run.smithi132:> stat -- /home/ubuntu/cephtest/mnt.0 2024-01-24T23:21:16.215 INFO:tasks.mon_thrash.mon_thrasher:start thrashing 2024-01-24T23:21:16.215 INFO:tasks.mon_thrash.mon_thrasher:seed: 1706138476, revive delay: 90.0, thrash delay: 1.0 thrash many: False, maintain quorum: True store thrash: False, probability: 50 freeze mon: prob 10.0 duration 15.0 2024-01-24T23:21:16.215 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-01-24T23:21:16.216 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-01-24T23:21:16.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T23:21:16.222 INFO:teuthology.orchestra.run.smithi132.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-01-24T23:21:16.222 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-01-24T23:21:16.270 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-01-24T23:21:16.271 DEBUG:teuthology.orchestra.run.smithi132:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-01-24T23:21:16.318 INFO:tasks.workunit:timeout=3h 2024-01-24T23:21:16.318 INFO:tasks.workunit:cleanup=True 2024-01-24T23:21:16.319 DEBUG:teuthology.orchestra.run.smithi132:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 663873555d45def955d1cc09730bfece2847b821 2024-01-24T23:21:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.327+0000 7fa3f5648640 10 monclient: get_monmap_and_config 2024-01-24T23:21:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.327+0000 7fa3f5648640 10 monclient: build_initial_monmap 2024-01-24T23:21:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.327+0000 7fa3f5648640 10 monclient: monmap: 2024-01-24T23:21:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:16.334826+0000 2024-01-24T23:21:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:16.334826+0000 2024-01-24T23:21:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:16.334 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:16.334 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:16.334 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:16.334 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:16.334 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:16.334 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:16.334 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:16.334 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 1 Processor -- start 2024-01-24T23:21:16.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 1 -- start start 2024-01-24T23:21:16.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 10 monclient: init 2024-01-24T23:21:16.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:16.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 10 monclient: _add_conns ranks=[8,4,0,3,1,5,2,7,6] 2024-01-24T23:21:16.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 10 monclient(hunting): picked mon.noname-e con 0x7fa3f0133a10 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:16.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 10 monclient(hunting): picked mon.noname-c con 0x7fa3f0133df0 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 10 monclient(hunting): picked mon.noname-a con 0x7fa3f0134250 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f004ed70 con 0x7fa3f0134250 2024-01-24T23:21:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f009aae0 con 0x7fa3f0133df0 2024-01-24T23:21:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f00678e0 con 0x7fa3f0133a10 2024-01-24T23:21:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 10 monclient(hunting): authenticate will time out at 955.257723s 2024-01-24T23:21:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3eeffd640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fa3f0133a10 0x7fa3f009bc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:48552/0 (socket says 172.21.15.115:48552) 2024-01-24T23:21:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3eeffd640 1 -- 172.21.15.115:0/2173759604 learned_addr learned my addr 172.21.15.115:0/2173759604 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3ee7fc640 1 --1- 172.21.15.115:0/2173759604 >> v1:172.21.15.115:6791/0 conn(0x7fa3f0133df0 0x7fa3f013bc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:59958/0 (socket says 172.21.15.115:59958) 2024-01-24T23:21:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 1 -- 172.21.15.115:0/2173759604 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1687116675 0 0) 0x7fa3f00678e0 con 0x7fa3f0133a10 2024-01-24T23:21:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 10 monclient(hunting): _negotiate 2024-01-24T23:21:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 10 monclient(hunting): my global_id is 74402 2024-01-24T23:21:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 1 -- 172.21.15.115:0/2173759604 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3dc003530 con 0x7fa3f0133a10 2024-01-24T23:21:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 1 -- 172.21.15.115:0/2173759604 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2796569641 0 0) 0x7fa3f004ed70 con 0x7fa3f0134250 2024-01-24T23:21:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 10 monclient(hunting): _negotiate 2024-01-24T23:21:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 10 monclient(hunting): my global_id is 4464 2024-01-24T23:21:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 1 -- 172.21.15.115:0/2173759604 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3f00678e0 con 0x7fa3f0134250 2024-01-24T23:21:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 1 -- 172.21.15.115:0/2173759604 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3134706843 0 0) 0x7fa3d8001120 con 0x7fa3f0133df0 2024-01-24T23:21:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 10 monclient(hunting): _negotiate 2024-01-24T23:21:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 10 monclient(hunting): my global_id is 74470 2024-01-24T23:21:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 1 -- 172.21.15.115:0/2173759604 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3f004ed70 con 0x7fa3f0133df0 2024-01-24T23:21:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 1 -- 172.21.15.115:0/2173759604 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1357605101 0 0) 0x7fa3dc003530 con 0x7fa3f0133a10 2024-01-24T23:21:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 1 -- 172.21.15.115:0/2173759604 >> v1:172.21.15.115:6791/0 conn(0x7fa3f0133df0 legacy=0x7fa3f013bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 1 -- 172.21.15.115:0/2173759604 >> v1:172.21.15.115:6789/0 conn(0x7fa3f0134250 legacy=0x7fa3f013f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 10 monclient: _finish_hunting 0 2024-01-24T23:21:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 1 monclient: found mon.noname-e 2024-01-24T23:21:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:21:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 1 -- 172.21.15.115:0/2173759604 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f0066d10 con 0x7fa3f0133a10 2024-01-24T23:21:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 10 monclient: _finish_auth 0 2024-01-24T23:21:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 10 monclient: _check_auth_tickets 2024-01-24T23:21:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 1 -- 172.21.15.115:0/2173759604 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa3e80017c0 con 0x7fa3f0133a10 2024-01-24T23:21:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:21:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 10 monclient: dump: 2024-01-24T23:21:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.341 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 5 monclient: authenticate success, global_id 74402 2024-01-24T23:21:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 1 -- 172.21.15.115:0/2173759604 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa3e8001a20 con 0x7fa3f0133a10 2024-01-24T23:21:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 4 set_mon_vals no callback set 2024-01-24T23:21:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 1 -- 172.21.15.115:0/2173759604 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa3e8004bf0 con 0x7fa3f0133a10 2024-01-24T23:21:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:21:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3edffb640 10 monclient: dump: 2024-01-24T23:21:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.343 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 10 monclient: shutdown 2024-01-24T23:21:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 1 -- 172.21.15.115:0/2173759604 >> v1:172.21.15.115:6793/0 conn(0x7fa3f0133a10 legacy=0x7fa3f009bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 1 -- 172.21.15.115:0/2173759604 shutdown_connections 2024-01-24T23:21:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 1 -- 172.21.15.115:0/2173759604 >> 172.21.15.115:0/2173759604 conn(0x7fa3f00a6490 msgr2=0x7fa3f00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 1 -- 172.21.15.115:0/2173759604 shutdown_connections 2024-01-24T23:21:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 1 -- 172.21.15.115:0/2173759604 wait complete. 2024-01-24T23:21:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 10 monclient: build_initial_monmap 2024-01-24T23:21:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.331+0000 7fa3f5648640 10 monclient: monmap: 2024-01-24T23:21:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:16.344 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3f5648640 1 Processor -- start 2024-01-24T23:21:16.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3f5648640 1 -- start start 2024-01-24T23:21:16.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3f5648640 10 monclient: init 2024-01-24T23:21:16.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3f5648640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:16.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3f5648640 10 monclient: _add_conns ranks=[4,8,6,1,7,5,3,2,0] 2024-01-24T23:21:16.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3f5648640 10 monclient(hunting): picked mon.noname-c con 0x7fa3f0133a10 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:16.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3f5648640 10 monclient(hunting): picked mon.noname-e con 0x7fa3f0133df0 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:16.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3f5648640 10 monclient(hunting): picked mon.noname-d con 0x7fa3f0134250 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:16.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3f5648640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f0120450 con 0x7fa3f0133a10 2024-01-24T23:21:16.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3f5648640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f009c610 con 0x7fa3f0134250 2024-01-24T23:21:16.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3f5648640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f011e160 con 0x7fa3f0133df0 2024-01-24T23:21:16.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3f5648640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:16.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3f5648640 10 monclient(hunting): authenticate will time out at 955.260791s 2024-01-24T23:21:16.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3eeffd640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fa3f0133a10 0x7fa3f0132a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:59974/0 (socket says 172.21.15.115:59974) 2024-01-24T23:21:16.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3eeffd640 1 -- 172.21.15.115:0/1381496203 learned_addr learned my addr 172.21.15.115:0/1381496203 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:16.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3740134711 0 0) 0x7fa3f009c610 con 0x7fa3f0134250 2024-01-24T23:21:16.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:16.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:16.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:16.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient(hunting): my global_id is 4425 2024-01-24T23:21:16.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3d0003530 con 0x7fa3f0134250 2024-01-24T23:21:16.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 798307225 0 0) 0x7fa3f011e160 con 0x7fa3f0133df0 2024-01-24T23:21:16.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:16.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:16.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:16.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient(hunting): my global_id is 74411 2024-01-24T23:21:16.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3f009c610 con 0x7fa3f0133df0 2024-01-24T23:21:16.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 657973436 0 0) 0x7fa3f0120450 con 0x7fa3f0133a10 2024-01-24T23:21:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient(hunting): my global_id is 74479 2024-01-24T23:21:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3f011e160 con 0x7fa3f0133a10 2024-01-24T23:21:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 384640576 0 0) 0x7fa3d0003530 con 0x7fa3f0134250 2024-01-24T23:21:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa3f0120450 con 0x7fa3f0134250 2024-01-24T23:21:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa3e0001cc0 con 0x7fa3f0134250 2024-01-24T23:21:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:21:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient(hunting): dump: 2024-01-24T23:21:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.351 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2892738363 0 0) 0x7fa3f009c610 con 0x7fa3f0133df0 2024-01-24T23:21:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa3d0003530 con 0x7fa3f0133df0 2024-01-24T23:21:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1289985615 0 0) 0x7fa3f011e160 con 0x7fa3f0133a10 2024-01-24T23:21:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa3f009c610 con 0x7fa3f0133a10 2024-01-24T23:21:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa3d8001600 con 0x7fa3f0133df0 2024-01-24T23:21:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:21:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient(hunting): dump: 2024-01-24T23:21:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.353 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.353 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.353 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.353 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.353 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa3e8004bb0 con 0x7fa3f0133a10 2024-01-24T23:21:16.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:16.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient(hunting): dump: 2024-01-24T23:21:16.353 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.353 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.353 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.353 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.353 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.353 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.354 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.354 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.354 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.354 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.354 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.354 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.354 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.354 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.354 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.354 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3826587590 0 0) 0x7fa3f0120450 con 0x7fa3f0134250 2024-01-24T23:21:16.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 >> v1:172.21.15.115:6793/0 conn(0x7fa3f0133df0 legacy=0x7fa3f01d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 >> v1:172.21.15.115:6791/0 conn(0x7fa3f0133a10 legacy=0x7fa3f0132a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient: _finish_hunting 0 2024-01-24T23:21:16.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 1 monclient: found mon.d 2024-01-24T23:21:16.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:16.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:16.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f011fbb0 con 0x7fa3f0134250 2024-01-24T23:21:16.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient: _finish_auth 0 2024-01-24T23:21:16.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient: _check_auth_tickets 2024-01-24T23:21:16.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:16.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3f5648640 5 monclient: authenticate success, global_id 4425 2024-01-24T23:21:16.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3f5648640 10 monclient: _renew_subs 2024-01-24T23:21:16.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3f5648640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:16.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3f5648640 1 -- 172.21.15.115:0/1381496203 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f011fe90 con 0x7fa3f0134250 2024-01-24T23:21:16.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa3e0002070 con 0x7fa3f0134250 2024-01-24T23:21:16.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:16.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa3e0003f70 con 0x7fa3f0134250 2024-01-24T23:21:16.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:16.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3ed7fa640 4 set_mon_vals no callback set 2024-01-24T23:21:16.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 10 monclient: dump: 2024-01-24T23:21:16.356 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.356 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.356 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.356 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.356 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.356 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.356 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.356 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.356 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.356 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.356 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.357 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.357 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.357 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.357 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.357 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3f5648640 10 monclient: _renew_subs 2024-01-24T23:21:16.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3f5648640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:16.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3f5648640 1 -- 172.21.15.115:0/1381496203 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f0133400 con 0x7fa3f0134250 2024-01-24T23:21:16.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fa3e000e9f0 con 0x7fa3f0134250 2024-01-24T23:21:16.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3f5648640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:16.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3f5648640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:16.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3f5648640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:16.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3f5648640 1 -- 172.21.15.115:0/1381496203 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f009b250 con 0x7fa3f0134250 2024-01-24T23:21:16.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7fa3e0042720 con 0x7fa3f0134250 2024-01-24T23:21:16.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.335+0000 7fa3eeffd640 10 monclient: get_auth_request con 0x7fa3d0037bb0 auth_method 0 2024-01-24T23:21:16.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.339+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fa3f009b250 con 0x7fa3f0134250 2024-01-24T23:21:16.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.339+0000 7fa3e77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:16.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.339+0000 7fa3e77fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:16.368 INFO:tasks.workunit.client.0.smithi132.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-01-24T23:21:16.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.551+0000 7fa3f5648640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-01-24T23:21:16.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.555+0000 7fa3f5648640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-01-24T23:21:16.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.555+0000 7fa3f5648640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:16.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.555+0000 7fa3f5648640 1 -- 172.21.15.115:0/1381496203 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa3f0120080 con 0x7fa3f0134250 2024-01-24T23:21:16.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.555+0000 7fa3e77fe640 1 -- 172.21.15.115:0/1381496203 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2744 (unknown 254746657 0 1482248110) 0x7fa3e00103f0 con 0x7fa3f0134250 2024-01-24T23:21:16.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.555+0000 7fa3e77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-01-24T23:21:16.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.555+0000 7fa3e77fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:16.558 INFO:teuthology.orchestra.run.smithi115.stdout:{"election_epoch":6,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"c","quorum_age":18,"features":{"quorum_con":"4540701272860131327","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","modified":"2024-01-24T23:20:20.557246Z","created":"2024-01-24T23:20:20.557246Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6790","nonce":0}]},"addr":"172.21.15.115:6790/0","public_addr":"172.21.15.115:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6790","nonce":0}]},"addr":"172.21.15.132:6790/0","public_addr":"172.21.15.132:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6791","nonce":0}]},"addr":"172.21.15.115:6791/0","public_addr":"172.21.15.115:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6791","nonce":0}]},"addr":"172.21.15.132:6791/0","public_addr":"172.21.15.132:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6792","nonce":0}]},"addr":"172.21.15.115:6792/0","public_addr":"172.21.15.115:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6792","nonce":0}]},"addr":"172.21.15.132:6792/0","public_addr":"172.21.15.132:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6793","nonce":0}]},"addr":"172.21.15.115:6793/0","public_addr":"172.21.15.115:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-01-24T23:21:16.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.555+0000 7fa3f5648640 1 -- 172.21.15.115:0/1381496203 >> v1:172.21.15.132:6812/1835401922 conn(0x7fa3d0037bb0 legacy=0x7fa3d003a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.555+0000 7fa3f5648640 10 monclient: shutdown 2024-01-24T23:21:16.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.555+0000 7fa3f5648640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:16.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.555+0000 7fa3f5648640 1 -- 172.21.15.115:0/1381496203 >> v1:172.21.15.115:6792/0 conn(0x7fa3f0134250 legacy=0x7fa3f01d3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.555+0000 7fa3f5648640 1 -- 172.21.15.115:0/1381496203 shutdown_connections 2024-01-24T23:21:16.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.555+0000 7fa3f5648640 1 -- 172.21.15.115:0/1381496203 >> 172.21.15.115:0/1381496203 conn(0x7fa3f00a6490 msgr2=0x7fa3f00a4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:16.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.555+0000 7fa3f5648640 1 -- 172.21.15.115:0/1381496203 shutdown_connections 2024-01-24T23:21:16.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.555+0000 7fa3f5648640 1 -- 172.21.15.115:0/1381496203 wait complete. 2024-01-24T23:21:16.576 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-01-24T23:21:16.576 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-01-24T23:21:16.576 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-01-24T23:21:16.576 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-01-24T23:21:16.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.683+0000 7fbf3a312640 10 monclient: get_monmap_and_config 2024-01-24T23:21:16.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.683+0000 7fbf3a312640 10 monclient: build_initial_monmap 2024-01-24T23:21:16.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.683+0000 7fbf3a312640 10 monclient: monmap: 2024-01-24T23:21:16.689 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:16.689 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.689 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:16.690730+0000 2024-01-24T23:21:16.689 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:16.690730+0000 2024-01-24T23:21:16.689 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:16.690 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.690 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:16.690 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:16.690 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:16.690 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:16.690 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:16.690 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:16.690 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:16.690 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:16.690 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:16.690 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 1 Processor -- start 2024-01-24T23:21:16.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 1 -- start start 2024-01-24T23:21:16.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 10 monclient: init 2024-01-24T23:21:16.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:16.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 10 monclient: _add_conns ranks=[1,3,7,6,0,5,4,2,8] 2024-01-24T23:21:16.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 10 monclient(hunting): picked mon.noname-f con 0x7fbf3409b9f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:16.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 10 monclient(hunting): picked mon.noname-g con 0x7fbf340720b0 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:16.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 10 monclient(hunting): picked mon.noname-i con 0x7fbf34139700 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:16.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf3404ed70 con 0x7fbf3409b9f0 2024-01-24T23:21:16.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf340a0eb0 con 0x7fbf340720b0 2024-01-24T23:21:16.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf3409bdd0 con 0x7fbf34139700 2024-01-24T23:21:16.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:16.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 10 monclient(hunting): authenticate will time out at 955.613174s 2024-01-24T23:21:16.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf33fff640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fbf3409b9f0 0x7fbf3409f8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.115:54806/0 (socket says 172.21.15.115:54806) 2024-01-24T23:21:16.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf388af640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fbf34139700 0x7fbf3413baf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.115:34792/0 (socket says 172.21.15.115:34792) 2024-01-24T23:21:16.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf33fff640 1 -- 172.21.15.115:0/2161647483 learned_addr learned my addr 172.21.15.115:0/2161647483 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:16.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 1 -- 172.21.15.115:0/2161647483 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3399187710 0 0) 0x7fbf340a0eb0 con 0x7fbf340720b0 2024-01-24T23:21:16.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:16.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:16.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:16.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 10 monclient(hunting): my global_id is 4413 2024-01-24T23:21:16.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 1 -- 172.21.15.115:0/2161647483 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf1c003530 con 0x7fbf340720b0 2024-01-24T23:21:16.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 1 -- 172.21.15.115:0/2161647483 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4132272029 0 0) 0x7fbf3409bdd0 con 0x7fbf34139700 2024-01-24T23:21:16.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:16.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:16.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:16.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 10 monclient(hunting): my global_id is 4471 2024-01-24T23:21:16.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 1 -- 172.21.15.115:0/2161647483 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf340a0eb0 con 0x7fbf34139700 2024-01-24T23:21:16.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 1 -- 172.21.15.115:0/2161647483 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487011166 0 0) 0x7fbf3404ed70 con 0x7fbf3409b9f0 2024-01-24T23:21:16.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:16.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:16.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:16.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 10 monclient(hunting): my global_id is 4402 2024-01-24T23:21:16.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 1 -- 172.21.15.115:0/2161647483 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf3409bdd0 con 0x7fbf3409b9f0 2024-01-24T23:21:16.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 1 -- 172.21.15.115:0/2161647483 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3406010848 0 0) 0x7fbf1c003530 con 0x7fbf340720b0 2024-01-24T23:21:16.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 1 -- 172.21.15.115:0/2161647483 >> v1:172.21.15.132:6792/0 conn(0x7fbf34139700 legacy=0x7fbf3413baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 1 -- 172.21.15.115:0/2161647483 >> v1:172.21.15.132:6789/0 conn(0x7fbf3409b9f0 legacy=0x7fbf3409f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:21:16.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 1 monclient: found mon.noname-g 2024-01-24T23:21:16.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:16.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:21:16.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 1 -- 172.21.15.115:0/2161647483 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf340a0710 con 0x7fbf340720b0 2024-01-24T23:21:16.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 10 monclient: _finish_auth 0 2024-01-24T23:21:16.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:21:16.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:16.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 1 -- 172.21.15.115:0/2161647483 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbf280017c0 con 0x7fbf340720b0 2024-01-24T23:21:16.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:21:16.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 10 monclient: dump: 2024-01-24T23:21:16.701 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.701 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.701 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.701 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.701 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.701 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.701 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.701 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.701 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.702 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.702 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.702 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.702 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.702 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.702 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.702 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 5 monclient: authenticate success, global_id 4413 2024-01-24T23:21:16.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:16.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 1 -- 172.21.15.115:0/2161647483 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbf280019e0 con 0x7fbf340720b0 2024-01-24T23:21:16.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:16.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:16.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 4 set_mon_vals no callback set 2024-01-24T23:21:16.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 1 -- 172.21.15.115:0/2161647483 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbf28003960 con 0x7fbf340720b0 2024-01-24T23:21:16.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:21:16.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf32ffd640 10 monclient: dump: 2024-01-24T23:21:16.703 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.703 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.703 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.703 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.703 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.703 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.703 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.703 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.703 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.703 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.703 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.703 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.703 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.703 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.704 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.704 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 10 monclient: shutdown 2024-01-24T23:21:16.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:16.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 1 -- 172.21.15.115:0/2161647483 >> v1:172.21.15.132:6790/0 conn(0x7fbf340720b0 legacy=0x7fbf340a0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 1 -- 172.21.15.115:0/2161647483 shutdown_connections 2024-01-24T23:21:16.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 1 -- 172.21.15.115:0/2161647483 >> 172.21.15.115:0/2161647483 conn(0x7fbf340a5730 msgr2=0x7fbf340a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:16.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 1 -- 172.21.15.115:0/2161647483 shutdown_connections 2024-01-24T23:21:16.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 1 -- 172.21.15.115:0/2161647483 wait complete. 2024-01-24T23:21:16.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 10 monclient: build_initial_monmap 2024-01-24T23:21:16.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.687+0000 7fbf3a312640 10 monclient: monmap: 2024-01-24T23:21:16.705 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:16.705 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:16.705 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:16.705 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:16.705 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:16.705 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:16.705 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:16.705 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:16.705 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:16.705 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:16.705 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:16.705 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:16.705 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:16.706 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:16.706 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:16.706 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf3a312640 1 Processor -- start 2024-01-24T23:21:16.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf3a312640 1 -- start start 2024-01-24T23:21:16.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf3a312640 10 monclient: init 2024-01-24T23:21:16.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf3a312640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:16.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf3a312640 10 monclient: _add_conns ranks=[6,0,4,1,7,5,8,2,3] 2024-01-24T23:21:16.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf3a312640 10 monclient(hunting): picked mon.noname-d con 0x7fbf340720b0 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:16.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf3a312640 10 monclient(hunting): picked mon.noname-a con 0x7fbf3409b9f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:16.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf3a312640 10 monclient(hunting): picked mon.noname-c con 0x7fbf34139700 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:16.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf3a312640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf34133b80 con 0x7fbf3409b9f0 2024-01-24T23:21:16.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf3a312640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf34121f50 con 0x7fbf34139700 2024-01-24T23:21:16.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf3a312640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf341cc6e0 con 0x7fbf340720b0 2024-01-24T23:21:16.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf3a312640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:16.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf3a312640 10 monclient(hunting): authenticate will time out at 955.616892s 2024-01-24T23:21:16.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf337fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fbf3409b9f0 0x7fbf341cbfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58188/0 (socket says 172.21.15.115:58188) 2024-01-24T23:21:16.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf388af640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fbf34139700 0x7fbf341d3b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:59980/0 (socket says 172.21.15.115:59980) 2024-01-24T23:21:16.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf337fe640 1 -- 172.21.15.115:0/1138634018 learned_addr learned my addr 172.21.15.115:0/1138634018 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:16.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1054840594 0 0) 0x7fbf34121f50 con 0x7fbf34139700 2024-01-24T23:21:16.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:16.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:16.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:16.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): my global_id is 74488 2024-01-24T23:21:16.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf20003530 con 0x7fbf34139700 2024-01-24T23:21:16.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 747916141 0 0) 0x7fbf34133b80 con 0x7fbf3409b9f0 2024-01-24T23:21:16.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:16.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:16.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:16.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): my global_id is 4473 2024-01-24T23:21:16.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf34121f50 con 0x7fbf3409b9f0 2024-01-24T23:21:16.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1701962782 0 0) 0x7fbf341cc6e0 con 0x7fbf340720b0 2024-01-24T23:21:16.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:16.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:16.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:16.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): my global_id is 4434 2024-01-24T23:21:16.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf34133b80 con 0x7fbf340720b0 2024-01-24T23:21:16.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3840532997 0 0) 0x7fbf34133b80 con 0x7fbf340720b0 2024-01-24T23:21:16.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbf341cc6e0 con 0x7fbf340720b0 2024-01-24T23:21:16.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2752495155 0 0) 0x7fbf20003530 con 0x7fbf34139700 2024-01-24T23:21:16.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbf34133b80 con 0x7fbf34139700 2024-01-24T23:21:16.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbf24001be0 con 0x7fbf340720b0 2024-01-24T23:21:16.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:21:16.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): dump: 2024-01-24T23:21:16.709 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.709 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.709 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.709 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.709 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.709 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.709 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.709 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.709 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.709 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.709 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.709 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.709 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.709 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.709 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.709 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:16.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 313623142 0 0) 0x7fbf34121f50 con 0x7fbf3409b9f0 2024-01-24T23:21:16.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbf20003530 con 0x7fbf3409b9f0 2024-01-24T23:21:16.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbf2c001c80 con 0x7fbf34139700 2024-01-24T23:21:16.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:16.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): dump: 2024-01-24T23:21:16.710 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.710 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.710 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.710 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.710 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.710 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.710 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.710 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.710 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.711 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.711 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.711 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.711 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.711 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.711 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.711 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbf28003940 con 0x7fbf3409b9f0 2024-01-24T23:21:16.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:16.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): dump: 2024-01-24T23:21:16.711 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.712 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1721387789 0 0) 0x7fbf34133b80 con 0x7fbf34139700 2024-01-24T23:21:16.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 >> v1:172.21.15.115:6792/0 conn(0x7fbf340720b0 legacy=0x7fbf341cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 >> v1:172.21.15.115:6789/0 conn(0x7fbf3409b9f0 legacy=0x7fbf341cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:21:16.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 monclient: found mon.c 2024-01-24T23:21:16.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:16.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:16.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf341207c0 con 0x7fbf34139700 2024-01-24T23:21:16.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient: _finish_auth 0 2024-01-24T23:21:16.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:21:16.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:16.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf3a312640 5 monclient: authenticate success, global_id 74488 2024-01-24T23:21:16.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf3a312640 10 monclient: _renew_subs 2024-01-24T23:21:16.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf3a312640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:16.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf3a312640 1 -- 172.21.15.115:0/1138634018 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3409c9f0 con 0x7fbf34139700 2024-01-24T23:21:16.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf3a312640 10 monclient: _renew_subs 2024-01-24T23:21:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf3a312640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf3a312640 1 -- 172.21.15.115:0/1138634018 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf34120510 con 0x7fbf34139700 2024-01-24T23:21:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbf2c004b70 con 0x7fbf34139700 2024-01-24T23:21:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbf2c005020 con 0x7fbf34139700 2024-01-24T23:21:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf327fc640 4 set_mon_vals no callback set 2024-01-24T23:21:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient: dump: 2024-01-24T23:21:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.715 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.715 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.715 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.715 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.715 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.715 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.715 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.715 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fbf2c00faa0 con 0x7fbf34139700 2024-01-24T23:21:16.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf3a312640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:16.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf3a312640 1 -- 172.21.15.115:0/1138634018 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbf140065d0 con 0x7fbf14002870 2024-01-24T23:21:16.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7fbf2c043720 con 0x7fbf34139700 2024-01-24T23:21:16.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3273213291 0 0) 0x7fbf140065d0 con 0x7fbf14002870 2024-01-24T23:21:16.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:16.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:16.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:16.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient(hunting): my global_id is 4482 2024-01-24T23:21:16.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf34133b80 con 0x7fbf14002870 2024-01-24T23:21:16.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3026831587 0 0) 0x7fbf34133b80 con 0x7fbf14002870 2024-01-24T23:21:16.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbf140065d0 con 0x7fbf14002870 2024-01-24T23:21:16.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf337fe640 10 monclient: get_auth_request con 0x7fbf20037c50 auth_method 0 2024-01-24T23:21:16.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbf24004de0 con 0x7fbf14002870 2024-01-24T23:21:16.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:16.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient: dump: 2024-01-24T23:21:16.716 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.716 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.716 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.716 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.716 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.717 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.717 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.717 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.717 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.717 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.717 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.717 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.717 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.717 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.717 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.717 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2434588992 0 0) 0x7fbf140065d0 con 0x7fbf14002870 2024-01-24T23:21:16.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 --> v1:172.21.15.115:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbf1400a120 con 0x7fbf14002870 2024-01-24T23:21:16.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 <== mon.0 v1:172.21.15.115:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fbf240071a0 con 0x7fbf14002870 2024-01-24T23:21:16.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:16.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:16.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 >> v1:172.21.15.115:6789/0 conn(0x7fbf14002870 legacy=0x7fbf14004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.691+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 >> v1:172.21.15.115:6789/0 conn(0x7fbf14002870 legacy=0x7fbf14004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:16.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf3a312640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:21:16.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf3a312640 1 -- 172.21.15.115:0/1138634018 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbf14009df0 con 0x7fbf140090b0 2024-01-24T23:21:16.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3051664362 0 0) 0x7fbf14009df0 con 0x7fbf140090b0 2024-01-24T23:21:16.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf30ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:16.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf30ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:16.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf30ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:16.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf30ff9640 10 monclient(hunting): my global_id is 4491 2024-01-24T23:21:16.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf140065d0 con 0x7fbf140090b0 2024-01-24T23:21:16.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 485741112 0 0) 0x7fbf140065d0 con 0x7fbf140090b0 2024-01-24T23:21:16.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbf14009df0 con 0x7fbf140090b0 2024-01-24T23:21:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbf24005950 con 0x7fbf140090b0 2024-01-24T23:21:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf30ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf30ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf30ff9640 10 monclient: dump: 2024-01-24T23:21:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.721 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.721 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.721 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.721 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.721 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 528291721 0 0) 0x7fbf14009df0 con 0x7fbf140090b0 2024-01-24T23:21:16.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 --> v1:172.21.15.115:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbf1400c310 con 0x7fbf140090b0 2024-01-24T23:21:16.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 <== mon.0 v1:172.21.15.115:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 2704379134) 0x7fbf24006ee0 con 0x7fbf140090b0 2024-01-24T23:21:16.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf30ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:21:16.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf30ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:16.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 >> v1:172.21.15.115:6789/0 conn(0x7fbf140090b0 legacy=0x7fbf14009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf30ff9640 1 -- 172.21.15.115:0/1138634018 >> v1:172.21.15.115:6789/0 conn(0x7fbf140090b0 legacy=0x7fbf14009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:16.722 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:21:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:21:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:21:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:21:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 6, 2024-01-24T23:21:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:21:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:21:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:21:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:21:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:21:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:21:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:21:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:21:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:21:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:21:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 18, 2024-01-24T23:21:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:21:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:21:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:21:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:21:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:21:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:21:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:21:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:21:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:21:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:21:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:21:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:21:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:21:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:21:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:21:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:21:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:21:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:21:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:21:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:21:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:21:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:21:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:21:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:21:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:21:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:21:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:21:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:21:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:21:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:21:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:21:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:21:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:21:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:21:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:21:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:21:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:21:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:21:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:21:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:21:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:21:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:21:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:21:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:21:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:21:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:21:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:21:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:21:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:21:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:21:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:21:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:21:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:21:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:21:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:21:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:21:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:21:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:21:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:21:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:21:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:21:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:21:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:21:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:21:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:21:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:21:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:21:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:21:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:21:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.741 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:21:16.741 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:21:16.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf3a312640 1 -- 172.21.15.115:0/1138634018 >> v1:172.21.15.132:6812/1835401922 conn(0x7fbf20037c50 legacy=0x7fbf2003a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf3a312640 10 monclient: shutdown 2024-01-24T23:21:16.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf3a312640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:16.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf3a312640 1 -- 172.21.15.115:0/1138634018 >> v1:172.21.15.115:6791/0 conn(0x7fbf34139700 legacy=0x7fbf341d3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf388af640 1 -- 172.21.15.115:0/1138634018 reap_dead start 2024-01-24T23:21:16.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf388af640 1 -- 172.21.15.115:0/1138634018 reap_dead start 2024-01-24T23:21:16.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf3a312640 1 -- 172.21.15.115:0/1138634018 shutdown_connections 2024-01-24T23:21:16.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf3a312640 1 -- 172.21.15.115:0/1138634018 >> 172.21.15.115:0/1138634018 conn(0x7fbf340a5730 msgr2=0x7fbf34133f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:16.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf3a312640 1 -- 172.21.15.115:0/1138634018 shutdown_connections 2024-01-24T23:21:16.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.699+0000 7fbf3a312640 1 -- 172.21.15.115:0/1138634018 wait complete. 2024-01-24T23:21:16.743 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:21:16.743 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-01-24T23:21:16.743 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-01-24T23:21:16.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 10 monclient: get_monmap_and_config 2024-01-24T23:21:16.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 10 monclient: build_initial_monmap 2024-01-24T23:21:16.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 10 monclient: monmap: 2024-01-24T23:21:16.858 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:16.858 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.858 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:16.859148+0000 2024-01-24T23:21:16.858 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:16.859148+0000 2024-01-24T23:21:16.858 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:16.858 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.858 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:16.858 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:16.858 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:16.858 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:16.858 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:16.858 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:16.858 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:16.858 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:16.858 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:16.858 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 1 Processor -- start 2024-01-24T23:21:16.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 1 -- start start 2024-01-24T23:21:16.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 10 monclient: init 2024-01-24T23:21:16.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:16.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 10 monclient: _add_conns ranks=[4,8,5,7,0,3,6,1,2] 2024-01-24T23:21:16.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 10 monclient(hunting): picked mon.noname-c con 0x7f7ad40a18a0 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:16.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 10 monclient(hunting): picked mon.noname-e con 0x7f7ad40a1c80 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:16.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 10 monclient(hunting): picked mon.noname-h con 0x7f7ad4139b20 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:16.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad404ed70 con 0x7f7ad40a18a0 2024-01-24T23:21:16.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad409cb70 con 0x7f7ad4139b20 2024-01-24T23:21:16.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad409c800 con 0x7f7ad40a1c80 2024-01-24T23:21:16.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:16.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 10 monclient(hunting): authenticate will time out at 955.781454s 2024-01-24T23:21:16.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad9d22640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f7ad40a18a0 0x7f7ad409db30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:59984/0 (socket says 172.21.15.115:59984) 2024-01-24T23:21:16.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad9d22640 1 -- 172.21.15.115:0/3536902257 learned_addr learned my addr 172.21.15.115:0/3536902257 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:16.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 1 -- 172.21.15.115:0/3536902257 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808094236 0 0) 0x7f7ad404ed70 con 0x7f7ad40a18a0 2024-01-24T23:21:16.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 10 monclient(hunting): _negotiate 2024-01-24T23:21:16.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:16.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:16.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 10 monclient(hunting): my global_id is 74497 2024-01-24T23:21:16.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 1 -- 172.21.15.115:0/3536902257 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ac0003530 con 0x7f7ad40a18a0 2024-01-24T23:21:16.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 1 -- 172.21.15.115:0/3536902257 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1533409938 0 0) 0x7f7ad409c800 con 0x7f7ad40a1c80 2024-01-24T23:21:16.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 10 monclient(hunting): _negotiate 2024-01-24T23:21:16.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:16.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:16.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 10 monclient(hunting): my global_id is 74420 2024-01-24T23:21:16.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 1 -- 172.21.15.115:0/3536902257 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ad404ed70 con 0x7f7ad40a1c80 2024-01-24T23:21:16.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 1 -- 172.21.15.115:0/3536902257 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2548318402 0 0) 0x7f7ac0003530 con 0x7f7ad40a18a0 2024-01-24T23:21:16.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 1 -- 172.21.15.115:0/3536902257 >> v1:172.21.15.115:6793/0 conn(0x7f7ad40a1c80 legacy=0x7f7ad40a2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 1 -- 172.21.15.115:0/3536902257 >> v1:172.21.15.132:6791/0 conn(0x7f7ad4139b20 legacy=0x7f7ad413bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 10 monclient: _finish_hunting 0 2024-01-24T23:21:16.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 1 monclient: found mon.noname-c 2024-01-24T23:21:16.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:16.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:21:16.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 1 -- 172.21.15.115:0/3536902257 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad409e240 con 0x7f7ad40a18a0 2024-01-24T23:21:16.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 10 monclient: _finish_auth 0 2024-01-24T23:21:16.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 10 monclient: _check_auth_tickets 2024-01-24T23:21:16.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:16.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 1 -- 172.21.15.115:0/3536902257 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7acc001b40 con 0x7f7ad40a18a0 2024-01-24T23:21:16.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:21:16.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 10 monclient: dump: 2024-01-24T23:21:16.868 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.868 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.869 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.869 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.869 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.869 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.869 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.869 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.869 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.869 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.869 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.869 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.869 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.869 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.869 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.869 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 5 monclient: authenticate success, global_id 74497 2024-01-24T23:21:16.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:16.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 1 -- 172.21.15.115:0/3536902257 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7acc001e20 con 0x7f7ad40a18a0 2024-01-24T23:21:16.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:16.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 1 -- 172.21.15.115:0/3536902257 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7acc004f70 con 0x7f7ad40a18a0 2024-01-24T23:21:16.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:16.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 4 set_mon_vals no callback set 2024-01-24T23:21:16.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:16.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7ad8d20640 10 monclient: dump: 2024-01-24T23:21:16.870 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.870 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.870 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.870 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.870 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.870 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.870 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.871 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.871 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.871 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.871 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.871 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.871 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.871 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.871 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.871 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 10 monclient: shutdown 2024-01-24T23:21:16.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:16.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 1 -- 172.21.15.115:0/3536902257 >> v1:172.21.15.115:6791/0 conn(0x7f7ad40a18a0 legacy=0x7f7ad409db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 1 -- 172.21.15.115:0/3536902257 shutdown_connections 2024-01-24T23:21:16.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 1 -- 172.21.15.115:0/3536902257 >> 172.21.15.115:0/3536902257 conn(0x7f7ad40a6490 msgr2=0x7f7ad40a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:16.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 1 -- 172.21.15.115:0/3536902257 shutdown_connections 2024-01-24T23:21:16.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 1 -- 172.21.15.115:0/3536902257 wait complete. 2024-01-24T23:21:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 10 monclient: build_initial_monmap 2024-01-24T23:21:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 10 monclient: monmap: 2024-01-24T23:21:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:16.873 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 1 Processor -- start 2024-01-24T23:21:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 1 -- start start 2024-01-24T23:21:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.855+0000 7f7adbf86640 10 monclient: init 2024-01-24T23:21:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7adbf86640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7adbf86640 10 monclient: _add_conns ranks=[0,3,6,4,2,1,7,5,8] 2024-01-24T23:21:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7adbf86640 10 monclient(hunting): picked mon.noname-a con 0x7f7ad40a18a0 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7adbf86640 10 monclient(hunting): picked mon.noname-g con 0x7f7ad40a1c80 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7adbf86640 10 monclient(hunting): picked mon.noname-d con 0x7f7ad4139b20 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7adbf86640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad4072110 con 0x7f7ad40a18a0 2024-01-24T23:21:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7adbf86640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad4071510 con 0x7f7ad40a1c80 2024-01-24T23:21:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7adbf86640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad4070b30 con 0x7f7ad4139b20 2024-01-24T23:21:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7adbf86640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7adbf86640 10 monclient(hunting): authenticate will time out at 955.784109s 2024-01-24T23:21:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7ad9d22640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f7ad40a18a0 0x7f7ad406dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58198/0 (socket says 172.21.15.115:58198) 2024-01-24T23:21:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7ad9d22640 1 -- 172.21.15.115:0/2541735803 learned_addr learned my addr 172.21.15.115:0/2541735803 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176405425 0 0) 0x7f7ad4070b30 con 0x7f7ad4139b20 2024-01-24T23:21:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient(hunting): my global_id is 4443 2024-01-24T23:21:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ab8003530 con 0x7f7ad4139b20 2024-01-24T23:21:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 488964309 0 0) 0x7f7ad4072110 con 0x7f7ad40a18a0 2024-01-24T23:21:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient(hunting): my global_id is 4500 2024-01-24T23:21:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ad4070b30 con 0x7f7ad40a18a0 2024-01-24T23:21:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1645012869 0 0) 0x7f7ab8003530 con 0x7f7ad4139b20 2024-01-24T23:21:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ad4072110 con 0x7f7ad4139b20 2024-01-24T23:21:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7ad00017f0 con 0x7f7ad4139b20 2024-01-24T23:21:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:21:16.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient(hunting): dump: 2024-01-24T23:21:16.875 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.875 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.875 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.875 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.875 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.875 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.875 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.875 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.875 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.875 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.875 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.875 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.875 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.876 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.876 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.876 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:16.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4064505397 0 0) 0x7f7ad4070b30 con 0x7f7ad40a18a0 2024-01-24T23:21:16.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ab8003530 con 0x7f7ad40a18a0 2024-01-24T23:21:16.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7acc004eb0 con 0x7f7ad40a18a0 2024-01-24T23:21:16.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:16.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient(hunting): dump: 2024-01-24T23:21:16.876 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.876 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.876 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.876 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.877 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.877 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.877 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.877 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.877 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.877 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.877 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.877 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.877 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.877 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.877 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.877 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1254450754 0 0) 0x7f7ad4071510 con 0x7f7ad40a1c80 2024-01-24T23:21:16.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:16.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:16.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:16.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient(hunting): my global_id is 4422 2024-01-24T23:21:16.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ad4070b30 con 0x7f7ad40a1c80 2024-01-24T23:21:16.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1406654610 0 0) 0x7f7ad4072110 con 0x7f7ad4139b20 2024-01-24T23:21:16.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 >> v1:172.21.15.132:6790/0 conn(0x7f7ad40a1c80 legacy=0x7f7ad406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 >> v1:172.21.15.115:6789/0 conn(0x7f7ad40a18a0 legacy=0x7f7ad406dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:21:16.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 monclient: found mon.d 2024-01-24T23:21:16.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:16.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:16.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad406eac0 con 0x7f7ad4139b20 2024-01-24T23:21:16.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient: _finish_auth 0 2024-01-24T23:21:16.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:21:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7adbf86640 5 monclient: authenticate success, global_id 4443 2024-01-24T23:21:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7adbf86640 10 monclient: _renew_subs 2024-01-24T23:21:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7adbf86640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7adbf86640 1 -- 172.21.15.115:0/2541735803 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad40a0090 con 0x7f7ad4139b20 2024-01-24T23:21:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7adbf86640 10 monclient: _renew_subs 2024-01-24T23:21:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7adbf86640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7adbf86640 1 -- 172.21.15.115:0/2541735803 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad4070570 con 0x7f7ad4139b20 2024-01-24T23:21:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7ad0003600 con 0x7f7ad4139b20 2024-01-24T23:21:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7ad0003ab0 con 0x7f7ad4139b20 2024-01-24T23:21:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7acbfff640 4 set_mon_vals no callback set 2024-01-24T23:21:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient: dump: 2024-01-24T23:21:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.880 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f7ad0003c10 con 0x7f7ad4139b20 2024-01-24T23:21:16.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f7ad0042140 con 0x7f7ad4139b20 2024-01-24T23:21:16.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7adbf86640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:16.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7adbf86640 1 -- 172.21.15.115:0/2541735803 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7abc0065d0 con 0x7f7abc002870 2024-01-24T23:21:16.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7ad9d22640 10 monclient: get_auth_request con 0x7f7ab8037760 auth_method 0 2024-01-24T23:21:16.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2639326886 0 0) 0x7f7abc0065d0 con 0x7f7abc002870 2024-01-24T23:21:16.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:16.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:16.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:16.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient(hunting): my global_id is 4403 2024-01-24T23:21:16.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ad4072110 con 0x7f7abc002870 2024-01-24T23:21:16.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1305222818 0 0) 0x7f7ad4072110 con 0x7f7abc002870 2024-01-24T23:21:16.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7abc0065d0 con 0x7f7abc002870 2024-01-24T23:21:16.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7ac40018f0 con 0x7f7abc002870 2024-01-24T23:21:16.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient: dump: 2024-01-24T23:21:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.883 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.883 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2177132326 0 0) 0x7f7abc0065d0 con 0x7f7abc002870 2024-01-24T23:21:16.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 --> v1:172.21.15.115:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7abc00a120 con 0x7f7abc002870 2024-01-24T23:21:16.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 <== mon.2 v1:172.21.15.115:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f7ac4003620 con 0x7f7abc002870 2024-01-24T23:21:16.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:16.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:16.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 >> v1:172.21.15.115:6790/0 conn(0x7f7abc002870 legacy=0x7f7abc004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.859+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 >> v1:172.21.15.115:6790/0 conn(0x7f7abc002870 legacy=0x7f7abc004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:16.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7adbf86640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:21:16.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7adbf86640 1 -- 172.21.15.115:0/2541735803 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7abc009e10 con 0x7f7abc0090d0 2024-01-24T23:21:16.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 661016930 0 0) 0x7f7abc009e10 con 0x7f7abc0090d0 2024-01-24T23:21:16.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7aca7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:16.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7aca7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:16.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7aca7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:16.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7aca7fc640 10 monclient(hunting): my global_id is 4412 2024-01-24T23:21:16.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7abc0065d0 con 0x7f7abc0090d0 2024-01-24T23:21:16.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4143158720 0 0) 0x7f7abc0065d0 con 0x7f7abc0090d0 2024-01-24T23:21:16.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7abc009e10 con 0x7f7abc0090d0 2024-01-24T23:21:16.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7ac4003cc0 con 0x7f7abc0090d0 2024-01-24T23:21:16.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7aca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:16.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7aca7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:16.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7aca7fc640 10 monclient: dump: 2024-01-24T23:21:16.885 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:16.885 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:16.885 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.885 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:16.886 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:16.886 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:16.886 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:16.886 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:16.886 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:16.886 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:16.886 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:16.886 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:16.886 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:16.886 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:16.886 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:16.886 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:16.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1982068015 0 0) 0x7f7abc009e10 con 0x7f7abc0090d0 2024-01-24T23:21:16.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 --> v1:172.21.15.115:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7abc00c310 con 0x7f7abc0090d0 2024-01-24T23:21:16.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 <== mon.2 v1:172.21.15.115:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 2698765205) 0x7f7ac4004100 con 0x7f7abc0090d0 2024-01-24T23:21:16.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7aca7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:21:16.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7aca7fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:16.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 >> v1:172.21.15.115:6790/0 conn(0x7f7abc0090d0 legacy=0x7f7abc009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7aca7fc640 1 -- 172.21.15.115:0/2541735803 >> v1:172.21.15.115:6790/0 conn(0x7f7abc0090d0 legacy=0x7f7abc009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:16.887 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:21:16.887 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:21:16.887 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:21:16.887 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:21:16.887 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 6, 2024-01-24T23:21:16.887 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:21:16.887 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:21:16.887 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:21:16.888 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:21:16.888 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:21:16.888 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:21:16.888 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:21:16.888 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:21:16.888 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:21:16.888 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:21:16.888 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:16.888 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 18, 2024-01-24T23:21:16.888 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:16.888 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:21:16.888 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:21:16.888 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:16.888 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:16.888 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:16.889 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:16.889 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:16.889 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:16.889 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:16.889 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:16.889 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:16.889 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:16.889 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:16.889 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:16.889 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:21:16.889 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:21:16.889 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:16.889 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:16.889 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:16.889 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:16.890 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:16.890 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:16.890 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:16.890 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:16.890 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:16.890 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:16.890 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:16.890 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:16.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.890 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:21:16.890 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:21:16.890 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:21:16.890 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:21:16.890 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:21:16.890 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:21:16.891 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:16.891 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:16.891 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:21:16.891 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:21:16.891 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:21:16.891 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:21:16.891 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:21:16.891 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:21:16.891 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:21:16.891 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:16.891 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:21:16.891 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:16.891 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:16.891 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:16.891 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:16.892 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:16.892 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:16.892 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:16.892 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:16.892 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:16.892 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:16.892 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:16.892 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:16.892 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:21:16.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.892 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:21:16.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.892 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:21:16.892 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:21:16.893 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:16.893 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:16.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.893 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:16.893 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:21:16.893 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:16.893 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.893 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:16.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.893 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:21:16.893 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:21:16.893 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:16.893 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:16.893 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:16.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.894 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:21:16.894 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:21:16.894 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:16.894 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:16.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.894 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:16.894 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:21:16.894 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:16.894 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.894 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:16.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.894 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:21:16.894 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:21:16.894 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:16.894 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:16.895 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:16.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.895 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:21:16.895 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:21:16.895 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:16.895 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:16.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.895 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:16.895 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:21:16.895 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:16.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.895 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:16.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.896 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:21:16.896 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:21:16.896 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:16.896 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:16.896 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:16.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.896 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:21:16.896 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:21:16.896 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:16.896 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:16.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.896 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:16.896 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:21:16.897 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:16.897 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.897 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:16.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.897 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:21:16.897 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:21:16.897 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:16.897 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:16.897 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:16.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.897 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:21:16.897 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:21:16.897 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:16.897 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:16.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.898 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:16.898 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:21:16.898 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:16.898 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.898 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:16.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.898 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:21:16.898 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:21:16.898 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:16.898 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:16.898 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:16.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.898 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:21:16.899 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:21:16.899 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:16.899 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:16.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.899 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:16.899 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:21:16.899 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:16.899 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.899 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:16.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.899 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:21:16.899 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:21:16.899 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:16.899 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:16.899 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:16.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.900 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:21:16.900 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:21:16.900 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:16.900 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:16.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.900 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:16.900 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:21:16.900 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:16.900 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.900 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:16.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.900 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:21:16.900 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:21:16.901 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:16.901 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:16.901 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:16.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.901 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:21:16.901 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:21:16.901 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:16.901 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:16.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.901 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:16.901 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:21:16.901 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:16.901 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.902 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:16.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.902 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:21:16.902 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:21:16.902 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:16.902 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:16.902 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:16.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.902 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:21:16.902 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:21:16.902 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:16.902 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:16.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.902 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:16.902 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:21:16.903 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:16.903 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.903 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:16.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.903 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:21:16.903 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:21:16.903 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:16.903 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:16.903 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:16.903 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.903 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:16.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.903 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:21:16.903 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:21:16.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.903 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:16.904 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:16.904 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:16.904 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.904 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:16.904 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:21:16.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.904 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:16.904 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:16.904 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:16.904 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.904 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:16.904 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:21:16.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:16.904 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:16.904 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:16.905 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:21:16.905 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:16.905 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:16.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:16.905 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:21:16.905 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:21:16.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7adbf86640 1 -- 172.21.15.115:0/2541735803 >> v1:172.21.15.132:6812/1835401922 conn(0x7f7ab8037760 legacy=0x7f7ab8039c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7adbf86640 10 monclient: shutdown 2024-01-24T23:21:16.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7adbf86640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:16.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7adbf86640 1 -- 172.21.15.115:0/2541735803 >> v1:172.21.15.115:6792/0 conn(0x7f7ad4139b20 legacy=0x7f7ad41d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:16.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7ada523640 1 -- 172.21.15.115:0/2541735803 reap_dead start 2024-01-24T23:21:16.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7ada523640 1 -- 172.21.15.115:0/2541735803 reap_dead start 2024-01-24T23:21:16.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7adbf86640 1 -- 172.21.15.115:0/2541735803 shutdown_connections 2024-01-24T23:21:16.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7adbf86640 1 -- 172.21.15.115:0/2541735803 >> 172.21.15.115:0/2541735803 conn(0x7f7ad40a6490 msgr2=0x7f7ad40a4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:16.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7adbf86640 1 -- 172.21.15.115:0/2541735803 shutdown_connections 2024-01-24T23:21:16.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:16.867+0000 7f7adbf86640 1 -- 172.21.15.115:0/2541735803 wait complete. 2024-01-24T23:21:16.907 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:21:16.907 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-01-24T23:21:16.907 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-01-24T23:21:17.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.011+0000 7f99aa254640 10 monclient: get_monmap_and_config 2024-01-24T23:21:17.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.011+0000 7f99aa254640 10 monclient: build_initial_monmap 2024-01-24T23:21:17.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.011+0000 7f99aa254640 10 monclient: monmap: 2024-01-24T23:21:17.017 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:17.017 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.017 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:17.018599+0000 2024-01-24T23:21:17.017 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:17.018599+0000 2024-01-24T23:21:17.017 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:17.017 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.017 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:17.017 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:17.017 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:17.018 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:17.018 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:17.018 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:17.018 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:17.018 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:17.018 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:17.018 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99aa254640 1 Processor -- start 2024-01-24T23:21:17.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99aa254640 1 -- start start 2024-01-24T23:21:17.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99aa254640 10 monclient: init 2024-01-24T23:21:17.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99aa254640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:17.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99aa254640 10 monclient: _add_conns ranks=[5,7,1,6,4,2,0,3,8] 2024-01-24T23:21:17.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99aa254640 10 monclient(hunting): picked mon.noname-h con 0x7f99a4133c10 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:17.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99aa254640 10 monclient(hunting): picked mon.noname-i con 0x7f99a4136960 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:17.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99aa254640 10 monclient(hunting): picked mon.noname-f con 0x7f99a413a470 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:17.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99aa254640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a404ed70 con 0x7f99a413a470 2024-01-24T23:21:17.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99aa254640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a409dee0 con 0x7f99a4133c10 2024-01-24T23:21:17.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99aa254640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a409d400 con 0x7f99a4136960 2024-01-24T23:21:17.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99aa254640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:17.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99aa254640 10 monclient(hunting): authenticate will time out at 955.941385s 2024-01-24T23:21:17.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a37fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f99a4133c10 0x7f99a4133ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.115:53330/0 (socket says 172.21.15.115:53330) 2024-01-24T23:21:17.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a3fff640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f99a413a470 0x7f99a413c860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.115:54816/0 (socket says 172.21.15.115:54816) 2024-01-24T23:21:17.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a37fe640 1 -- 172.21.15.115:0/3081394199 learned_addr learned my addr 172.21.15.115:0/3081394199 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:17.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a2ffd640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f99a4136960 0x7f99a4138d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.115:34802/0 (socket says 172.21.15.115:34802) 2024-01-24T23:21:17.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 1 -- 172.21.15.115:0/3081394199 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 874194413 0 0) 0x7f99a409d400 con 0x7f99a4136960 2024-01-24T23:21:17.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 10 monclient(hunting): my global_id is 4480 2024-01-24T23:21:17.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 1 -- 172.21.15.115:0/3081394199 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f998c003530 con 0x7f99a4136960 2024-01-24T23:21:17.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 1 -- 172.21.15.115:0/3081394199 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4067389776 0 0) 0x7f99a409dee0 con 0x7f99a4133c10 2024-01-24T23:21:17.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 10 monclient(hunting): my global_id is 4433 2024-01-24T23:21:17.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 1 -- 172.21.15.115:0/3081394199 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99a409d400 con 0x7f99a4133c10 2024-01-24T23:21:17.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 1 -- 172.21.15.115:0/3081394199 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 207932519 0 0) 0x7f99a404ed70 con 0x7f99a413a470 2024-01-24T23:21:17.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 10 monclient(hunting): my global_id is 4411 2024-01-24T23:21:17.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 1 -- 172.21.15.115:0/3081394199 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99a409dee0 con 0x7f99a413a470 2024-01-24T23:21:17.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 1 -- 172.21.15.115:0/3081394199 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3979417497 0 0) 0x7f998c003530 con 0x7f99a4136960 2024-01-24T23:21:17.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 1 -- 172.21.15.115:0/3081394199 >> v1:172.21.15.132:6791/0 conn(0x7f99a4133c10 legacy=0x7f99a4133ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 1 -- 172.21.15.115:0/3081394199 >> v1:172.21.15.132:6789/0 conn(0x7f99a413a470 legacy=0x7f99a413c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 10 monclient: _finish_hunting 0 2024-01-24T23:21:17.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 1 monclient: found mon.noname-i 2024-01-24T23:21:17.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:17.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:21:17.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 1 -- 172.21.15.115:0/3081394199 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a409f380 con 0x7f99a4136960 2024-01-24T23:21:17.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 10 monclient: _finish_auth 0 2024-01-24T23:21:17.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 10 monclient: _check_auth_tickets 2024-01-24T23:21:17.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:17.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 1 -- 172.21.15.115:0/3081394199 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f99940017c0 con 0x7f99a4136960 2024-01-24T23:21:17.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:21:17.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 10 monclient: dump: 2024-01-24T23:21:17.024 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.024 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.024 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.024 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.025 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.025 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.025 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.025 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.025 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.025 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.025 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.025 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.025 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.025 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.025 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.025 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99aa254640 5 monclient: authenticate success, global_id 4480 2024-01-24T23:21:17.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99aa254640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:17.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 1 -- 172.21.15.115:0/3081394199 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9994001aa0 con 0x7f99a4136960 2024-01-24T23:21:17.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:17.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 1 -- 172.21.15.115:0/3081394199 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9994004b30 con 0x7f99a4136960 2024-01-24T23:21:17.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99aa254640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:17.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99aa254640 4 set_mon_vals no callback set 2024-01-24T23:21:17.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:21:17.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99a27fc640 10 monclient: dump: 2024-01-24T23:21:17.026 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.026 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.026 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.026 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.026 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.026 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.026 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.027 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.027 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.027 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.027 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.027 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.027 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.027 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.027 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.027 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99aa254640 10 monclient: shutdown 2024-01-24T23:21:17.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99aa254640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:17.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99aa254640 1 -- 172.21.15.115:0/3081394199 >> v1:172.21.15.132:6792/0 conn(0x7f99a4136960 legacy=0x7f99a4138d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99aa254640 1 -- 172.21.15.115:0/3081394199 shutdown_connections 2024-01-24T23:21:17.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99aa254640 1 -- 172.21.15.115:0/3081394199 >> 172.21.15.115:0/3081394199 conn(0x7f99a40a5730 msgr2=0x7f99a40a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:17.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99aa254640 1 -- 172.21.15.115:0/3081394199 shutdown_connections 2024-01-24T23:21:17.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.015+0000 7f99aa254640 1 -- 172.21.15.115:0/3081394199 wait complete. 2024-01-24T23:21:17.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99aa254640 10 monclient: build_initial_monmap 2024-01-24T23:21:17.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99aa254640 10 monclient: monmap: 2024-01-24T23:21:17.033 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:17.033 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:17.033 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:17.033 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:17.033 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:17.033 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:17.033 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:17.034 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:17.034 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:17.034 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:17.034 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:17.034 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:17.034 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:17.034 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:17.034 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:17.034 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99aa254640 1 Processor -- start 2024-01-24T23:21:17.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99aa254640 1 -- start start 2024-01-24T23:21:17.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99aa254640 10 monclient: init 2024-01-24T23:21:17.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99aa254640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:17.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99aa254640 10 monclient: _add_conns ranks=[1,2,8,6,4,3,0,7,5] 2024-01-24T23:21:17.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99aa254640 10 monclient(hunting): picked mon.noname-f con 0x7f99a4133c10 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:17.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99aa254640 10 monclient(hunting): picked mon.noname-b con 0x7f99a4136960 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:17.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99aa254640 10 monclient(hunting): picked mon.noname-e con 0x7f99a413a470 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:17.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99aa254640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a4131c50 con 0x7f99a4133c10 2024-01-24T23:21:17.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99aa254640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a41c8400 con 0x7f99a4136960 2024-01-24T23:21:17.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99aa254640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a40a1310 con 0x7f99a413a470 2024-01-24T23:21:17.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99aa254640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:17.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99aa254640 10 monclient(hunting): authenticate will time out at 955.945498s 2024-01-24T23:21:17.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99a2ffd640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f99a4136960 0x7f99a41c7cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:55510/0 (socket says 172.21.15.115:55510) 2024-01-24T23:21:17.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99a2ffd640 1 -- 172.21.15.115:0/2601078374 learned_addr learned my addr 172.21.15.115:0/2601078374 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:17.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99a3fff640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f99a413a470 0x7f99a41cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:48576/0 (socket says 172.21.15.115:48576) 2024-01-24T23:21:17.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2264324024 0 0) 0x7f99a41c8400 con 0x7f99a4136960 2024-01-24T23:21:17.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient(hunting): my global_id is 4421 2024-01-24T23:21:17.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9990003530 con 0x7f99a4136960 2024-01-24T23:21:17.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316268371 0 0) 0x7f99a40a1310 con 0x7f99a413a470 2024-01-24T23:21:17.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient(hunting): my global_id is 74429 2024-01-24T23:21:17.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99a41c8400 con 0x7f99a413a470 2024-01-24T23:21:17.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2668650028 0 0) 0x7f99a4131c50 con 0x7f99a4133c10 2024-01-24T23:21:17.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient(hunting): my global_id is 4420 2024-01-24T23:21:17.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99a40a1310 con 0x7f99a4133c10 2024-01-24T23:21:17.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3431128891 0 0) 0x7f9990003530 con 0x7f99a4136960 2024-01-24T23:21:17.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f99a4131c50 con 0x7f99a4136960 2024-01-24T23:21:17.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9994001780 con 0x7f99a4136960 2024-01-24T23:21:17.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:21:17.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient(hunting): dump: 2024-01-24T23:21:17.037 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.037 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.037 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.037 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.037 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.037 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.037 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.037 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.037 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.037 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.038 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.038 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.038 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.038 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.038 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.038 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:17.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4229260077 0 0) 0x7f99a41c8400 con 0x7f99a413a470 2024-01-24T23:21:17.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9990003530 con 0x7f99a413a470 2024-01-24T23:21:17.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f999c001900 con 0x7f99a413a470 2024-01-24T23:21:17.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:21:17.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient(hunting): dump: 2024-01-24T23:21:17.038 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.039 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.039 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.039 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.039 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.039 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.039 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.039 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.039 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.039 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.039 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.039 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.039 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.039 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.039 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.039 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2913669467 0 0) 0x7f99a4131c50 con 0x7f99a4136960 2024-01-24T23:21:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 >> v1:172.21.15.115:6793/0 conn(0x7f99a413a470 legacy=0x7f99a41cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 >> v1:172.21.15.132:6789/0 conn(0x7f99a4133c10 legacy=0x7f99a41c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:21:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 monclient: found mon.b 2024-01-24T23:21:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a4139730 con 0x7f99a4136960 2024-01-24T23:21:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient: _finish_auth 0 2024-01-24T23:21:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:21:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99aa254640 5 monclient: authenticate success, global_id 4421 2024-01-24T23:21:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99aa254640 10 monclient: _renew_subs 2024-01-24T23:21:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99aa254640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99aa254640 1 -- 172.21.15.115:0/2601078374 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a41313b0 con 0x7f99a4136960 2024-01-24T23:21:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99aa254640 10 monclient: _renew_subs 2024-01-24T23:21:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99aa254640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99aa254640 1 -- 172.21.15.115:0/2601078374 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a41335e0 con 0x7f99a4136960 2024-01-24T23:21:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9994004e00 con 0x7f99a4136960 2024-01-24T23:21:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f99940052b0 con 0x7f99a4136960 2024-01-24T23:21:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99a1ffb640 4 set_mon_vals no callback set 2024-01-24T23:21:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient: dump: 2024-01-24T23:21:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.042 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.042 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.042 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.042 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.042 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.042 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.042 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.042 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.042 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.042 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.042 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.042 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.042 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99aa254640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:17.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99aa254640 1 -- 172.21.15.115:0/2601078374 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f99840065d0 con 0x7f9984002870 2024-01-24T23:21:17.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f999400fd30 con 0x7f99a4136960 2024-01-24T23:21:17.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f99940438b0 con 0x7f99a4136960 2024-01-24T23:21:17.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2377231139 0 0) 0x7f99840065d0 con 0x7f9984002870 2024-01-24T23:21:17.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient(hunting): my global_id is 74506 2024-01-24T23:21:17.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99a4131c50 con 0x7f9984002870 2024-01-24T23:21:17.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1281184703 0 0) 0x7f99a4131c50 con 0x7f9984002870 2024-01-24T23:21:17.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f99840065d0 con 0x7f9984002870 2024-01-24T23:21:17.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9998001c30 con 0x7f9984002870 2024-01-24T23:21:17.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 10 monclient: dump: 2024-01-24T23:21:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.045 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.045 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.045 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.045 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f99a3fff640 10 monclient: get_auth_request con 0x7f99900377c0 auth_method 0 2024-01-24T23:21:17.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4245912866 0 0) 0x7f99840065d0 con 0x7f9984002870 2024-01-24T23:21:17.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.019+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 --> v1:172.21.15.115:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f998400a120 con 0x7f9984002870 2024-01-24T23:21:17.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.023+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 <== mon.4 v1:172.21.15.115:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f9998004b70 con 0x7f9984002870 2024-01-24T23:21:17.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.023+0000 7f998bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:17.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.023+0000 7f998bfff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.023+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 >> v1:172.21.15.115:6791/0 conn(0x7f9984002870 legacy=0x7f9984004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.023+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 >> v1:172.21.15.115:6791/0 conn(0x7f9984002870 legacy=0x7f9984004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f99aa254640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:21:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f99aa254640 1 -- 172.21.15.115:0/2601078374 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9984009e10 con 0x7f99840090d0 2024-01-24T23:21:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 987285166 0 0) 0x7f9984009e10 con 0x7f99840090d0 2024-01-24T23:21:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f998bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f998bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f998bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f998bfff640 10 monclient(hunting): my global_id is 74515 2024-01-24T23:21:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99840065d0 con 0x7f99840090d0 2024-01-24T23:21:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2310824415 0 0) 0x7f99840065d0 con 0x7f99840090d0 2024-01-24T23:21:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9984009e10 con 0x7f99840090d0 2024-01-24T23:21:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9998005260 con 0x7f99840090d0 2024-01-24T23:21:17.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f998bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f998bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:17.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f998bfff640 10 monclient: dump: 2024-01-24T23:21:17.048 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.048 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.048 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.048 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.048 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.048 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.048 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.048 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.048 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.048 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.048 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.048 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.049 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.049 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.049 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.049 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1888420227 0 0) 0x7f9984009e10 con 0x7f99840090d0 2024-01-24T23:21:17.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 --> v1:172.21.15.115:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f998400c310 con 0x7f99840090d0 2024-01-24T23:21:17.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 <== mon.4 v1:172.21.15.115:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7715 (unknown 0 0 2242181401) 0x7f9998007600 con 0x7f99840090d0 2024-01-24T23:21:17.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f998bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:21:17.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f998bfff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:17.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 >> v1:172.21.15.115:6791/0 conn(0x7f99840090d0 legacy=0x7f9984009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f998bfff640 1 -- 172.21.15.115:0/2601078374 >> v1:172.21.15.115:6791/0 conn(0x7f99840090d0 legacy=0x7f9984009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:17.049 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:21:17.050 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:21:17.050 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:21:17.050 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "leader", 2024-01-24T23:21:17.050 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 6, 2024-01-24T23:21:17.050 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:21:17.050 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:21:17.050 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:21:17.050 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:21:17.050 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:21:17.050 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:21:17.050 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:21:17.050 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:21:17.050 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:21:17.050 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:21:17.051 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.051 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 19, 2024-01-24T23:21:17.051 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:17.051 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:21:17.051 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:21:17.051 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:17.051 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:17.051 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:17.051 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:17.051 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:17.051 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:17.051 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:17.051 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:17.051 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:17.052 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:17.052 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:17.052 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.052 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:21:17.052 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:21:17.052 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:17.052 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:17.052 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:17.052 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:17.052 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:17.052 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:17.052 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:17.052 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:17.052 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:17.053 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:17.053 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:17.053 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.053 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:21:17.053 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:21:17.053 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:21:17.053 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:21:17.053 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:21:17.053 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:21:17.053 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:17.053 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:17.053 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:21:17.054 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:21:17.054 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:21:17.054 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:21:17.054 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:21:17.054 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:21:17.054 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:21:17.054 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:17.054 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:21:17.054 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:17.054 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:17.054 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:17.054 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:17.054 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:17.054 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:17.055 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:17.055 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:17.055 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:17.055 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:17.055 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:17.055 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.055 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:21:17.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.055 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:21:17.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.055 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:21:17.055 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:21:17.055 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.055 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.056 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.056 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:21:17.056 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.056 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.056 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.056 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:21:17.056 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:21:17.056 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.056 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.056 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.057 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:21:17.057 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:21:17.057 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.057 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.057 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.057 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:21:17.057 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.057 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.057 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.057 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:21:17.057 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:21:17.058 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.058 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.058 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.058 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:21:17.058 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:21:17.058 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.058 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.058 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.058 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:21:17.058 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.058 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.058 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.059 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:21:17.059 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:21:17.059 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.059 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.059 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.059 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:21:17.059 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:21:17.059 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.059 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.059 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.060 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:21:17.060 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.060 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.060 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.060 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:21:17.060 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:21:17.060 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.060 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.060 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.060 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:21:17.061 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:21:17.061 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.061 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.061 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.061 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:21:17.061 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.061 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.061 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.061 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:21:17.061 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:21:17.062 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.062 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.062 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.062 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:21:17.062 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:21:17.062 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.062 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.062 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.062 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:21:17.062 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.062 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.063 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.063 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:21:17.063 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:21:17.063 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.063 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.063 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.063 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:21:17.063 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:21:17.063 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.063 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:21:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:21:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:21:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:21:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:21:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:21:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:21:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:21:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:21:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:21:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:21:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:21:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:21:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:21:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:21:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:21:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:21:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:17.069 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:17.069 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:17.069 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.069 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.069 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:21:17.069 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:21:17.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f99aa254640 1 -- 172.21.15.115:0/2601078374 >> v1:172.21.15.132:6812/1835401922 conn(0x7f99900377c0 legacy=0x7f9990039c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f99aa254640 10 monclient: shutdown 2024-01-24T23:21:17.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f99aa254640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:17.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f99aa254640 1 -- 172.21.15.115:0/2601078374 >> v1:172.21.15.115:6790/0 conn(0x7f99a4136960 legacy=0x7f99a41c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f99a3fff640 1 -- 172.21.15.115:0/2601078374 reap_dead start 2024-01-24T23:21:17.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f99a3fff640 1 -- 172.21.15.115:0/2601078374 reap_dead start 2024-01-24T23:21:17.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f99aa254640 1 -- 172.21.15.115:0/2601078374 shutdown_connections 2024-01-24T23:21:17.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f99aa254640 1 -- 172.21.15.115:0/2601078374 >> 172.21.15.115:0/2601078374 conn(0x7f99a40a5730 msgr2=0x7f99a412ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:17.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f99aa254640 1 -- 172.21.15.115:0/2601078374 shutdown_connections 2024-01-24T23:21:17.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.027+0000 7f99aa254640 1 -- 172.21.15.115:0/2601078374 wait complete. 2024-01-24T23:21:17.071 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:21:17.071 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-01-24T23:21:17.072 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-01-24T23:21:17.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.171+0000 7f227b24e640 10 monclient: get_monmap_and_config 2024-01-24T23:21:17.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.171+0000 7f227b24e640 10 monclient: build_initial_monmap 2024-01-24T23:21:17.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.171+0000 7f227b24e640 10 monclient: monmap: 2024-01-24T23:21:17.176 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:17.176 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.176 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:17.177265+0000 2024-01-24T23:21:17.176 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:17.177265+0000 2024-01-24T23:21:17.176 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:17.176 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.176 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:17.176 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:17.176 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:17.176 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:17.176 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:17.177 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:17.177 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:17.177 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:17.177 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:17.177 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.171+0000 7f227b24e640 1 Processor -- start 2024-01-24T23:21:17.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.171+0000 7f227b24e640 1 -- start start 2024-01-24T23:21:17.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.171+0000 7f227b24e640 10 monclient: init 2024-01-24T23:21:17.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.171+0000 7f227b24e640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:17.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.171+0000 7f227b24e640 10 monclient: _add_conns ranks=[7,8,2,5,6,3,1,4,0] 2024-01-24T23:21:17.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.171+0000 7f227b24e640 10 monclient(hunting): picked mon.noname-i con 0x7f22740a0a10 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:17.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.171+0000 7f227b24e640 10 monclient(hunting): picked mon.noname-e con 0x7f227409f1e0 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:17.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.171+0000 7f227b24e640 10 monclient(hunting): picked mon.noname-b con 0x7f227413d360 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:17.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.171+0000 7f227b24e640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f227404ed70 con 0x7f227413d360 2024-01-24T23:21:17.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.171+0000 7f227b24e640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22740a1500 con 0x7f22740a0a10 2024-01-24T23:21:17.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.171+0000 7f227b24e640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f227406aaf0 con 0x7f227409f1e0 2024-01-24T23:21:17.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.171+0000 7f227b24e640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:17.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.171+0000 7f227b24e640 10 monclient(hunting): authenticate will time out at 956.099820s 2024-01-24T23:21:17.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.171+0000 7f2273fff640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f227409f1e0 0x7f227413bc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:48590/0 (socket says 172.21.15.115:48590) 2024-01-24T23:21:17.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.171+0000 7f22797eb640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f227413d360 0x7f227413f750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:55526/0 (socket says 172.21.15.115:55526) 2024-01-24T23:21:17.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.171+0000 7f2273fff640 1 -- 172.21.15.115:0/181802267 learned_addr learned my addr 172.21.15.115:0/181802267 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:17.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 1 -- 172.21.15.115:0/181802267 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1887809740 0 0) 0x7f227406aaf0 con 0x7f227409f1e0 2024-01-24T23:21:17.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 10 monclient(hunting): my global_id is 74438 2024-01-24T23:21:17.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 1 -- 172.21.15.115:0/181802267 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f225c003530 con 0x7f227409f1e0 2024-01-24T23:21:17.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 1 -- 172.21.15.115:0/181802267 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129944817 0 0) 0x7f227404ed70 con 0x7f227413d360 2024-01-24T23:21:17.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 10 monclient(hunting): my global_id is 4430 2024-01-24T23:21:17.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 1 -- 172.21.15.115:0/181802267 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f227406aaf0 con 0x7f227413d360 2024-01-24T23:21:17.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 1 -- 172.21.15.115:0/181802267 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1162802138 0 0) 0x7f227406aaf0 con 0x7f227413d360 2024-01-24T23:21:17.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 1 -- 172.21.15.115:0/181802267 >> v1:172.21.15.115:6793/0 conn(0x7f227409f1e0 legacy=0x7f227413bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 1 -- 172.21.15.115:0/181802267 >> v1:172.21.15.132:6792/0 conn(0x7f22740a0a10 legacy=0x7f22740a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 10 monclient: _finish_hunting 0 2024-01-24T23:21:17.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 1 monclient: found mon.noname-b 2024-01-24T23:21:17.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:17.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:21:17.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 1 -- 172.21.15.115:0/181802267 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227409e560 con 0x7f227413d360 2024-01-24T23:21:17.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 10 monclient: _finish_auth 0 2024-01-24T23:21:17.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 10 monclient: _check_auth_tickets 2024-01-24T23:21:17.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:17.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 1 -- 172.21.15.115:0/181802267 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f22640017c0 con 0x7f227413d360 2024-01-24T23:21:17.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:21:17.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 10 monclient: dump: 2024-01-24T23:21:17.188 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.188 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.188 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.188 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.188 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.188 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.188 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.188 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.188 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.189 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.189 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.189 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.189 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.189 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.189 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.189 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 5 monclient: authenticate success, global_id 4430 2024-01-24T23:21:17.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:17.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 1 -- 172.21.15.115:0/181802267 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f22640019e0 con 0x7f227413d360 2024-01-24T23:21:17.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:17.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 1 -- 172.21.15.115:0/181802267 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2264004b30 con 0x7f227413d360 2024-01-24T23:21:17.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:17.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 4 set_mon_vals no callback set 2024-01-24T23:21:17.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:17.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22737fe640 10 monclient: dump: 2024-01-24T23:21:17.190 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.190 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.190 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.190 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.190 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.190 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.190 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.190 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.190 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.190 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.190 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.190 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.190 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.191 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.191 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.191 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 10 monclient: shutdown 2024-01-24T23:21:17.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:17.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 1 -- 172.21.15.115:0/181802267 >> v1:172.21.15.115:6790/0 conn(0x7f227413d360 legacy=0x7f227413f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 1 -- 172.21.15.115:0/181802267 shutdown_connections 2024-01-24T23:21:17.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 1 -- 172.21.15.115:0/181802267 >> 172.21.15.115:0/181802267 conn(0x7f22740a7660 msgr2=0x7f22740a3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:17.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 1 -- 172.21.15.115:0/181802267 shutdown_connections 2024-01-24T23:21:17.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 1 -- 172.21.15.115:0/181802267 wait complete. 2024-01-24T23:21:17.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 10 monclient: build_initial_monmap 2024-01-24T23:21:17.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 10 monclient: monmap: 2024-01-24T23:21:17.191 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:17.191 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:17.192 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:17.192 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:17.192 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:17.192 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:17.192 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:17.192 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:17.192 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:17.192 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:17.192 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:17.192 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:17.192 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:17.192 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:17.192 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:17.193 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 1 Processor -- start 2024-01-24T23:21:17.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 1 -- start start 2024-01-24T23:21:17.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 10 monclient: init 2024-01-24T23:21:17.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:17.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 10 monclient: _add_conns ranks=[0,6,3,4,8,1,7,5,2] 2024-01-24T23:21:17.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 10 monclient(hunting): picked mon.noname-a con 0x7f227409f1e0 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:17.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 10 monclient(hunting): picked mon.noname-d con 0x7f22740a0a10 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:17.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 10 monclient(hunting): picked mon.noname-g con 0x7f227413d360 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:17.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2274072470 con 0x7f227409f1e0 2024-01-24T23:21:17.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2274071eb0 con 0x7f227413d360 2024-01-24T23:21:17.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2274071670 con 0x7f22740a0a10 2024-01-24T23:21:17.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:17.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 10 monclient(hunting): authenticate will time out at 956.102606s 2024-01-24T23:21:17.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f2278fea640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f227409f1e0 0x7f2274131f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58214/0 (socket says 172.21.15.115:58214) 2024-01-24T23:21:17.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f2278fea640 1 -- 172.21.15.115:0/2048372873 learned_addr learned my addr 172.21.15.115:0/2048372873 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:17.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2197517660 0 0) 0x7f22740a1500 con 0x7f227409f1e0 2024-01-24T23:21:17.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient(hunting): my global_id is 4509 2024-01-24T23:21:17.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2260003530 con 0x7f227409f1e0 2024-01-24T23:21:17.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1872220613 0 0) 0x7f2274071670 con 0x7f22740a0a10 2024-01-24T23:21:17.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient(hunting): my global_id is 4452 2024-01-24T23:21:17.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22740a1500 con 0x7f22740a0a10 2024-01-24T23:21:17.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1121990559 0 0) 0x7f2260003530 con 0x7f227409f1e0 2024-01-24T23:21:17.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2274071670 con 0x7f227409f1e0 2024-01-24T23:21:17.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f226c001e60 con 0x7f227409f1e0 2024-01-24T23:21:17.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:21:17.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient(hunting): dump: 2024-01-24T23:21:17.195 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.195 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.195 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.195 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.195 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.195 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.195 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.195 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.196 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.196 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.196 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.196 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.196 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.196 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.196 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.196 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:17.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1233992549 0 0) 0x7f22740a1500 con 0x7f22740a0a10 2024-01-24T23:21:17.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2260003530 con 0x7f22740a0a10 2024-01-24T23:21:17.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2268001580 con 0x7f22740a0a10 2024-01-24T23:21:17.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:17.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient(hunting): dump: 2024-01-24T23:21:17.197 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.197 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.197 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.197 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.197 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.197 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.197 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.197 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.197 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.197 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.197 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.198 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.198 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.198 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.198 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.198 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 194882856 0 0) 0x7f2274071eb0 con 0x7f227413d360 2024-01-24T23:21:17.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient(hunting): my global_id is 4431 2024-01-24T23:21:17.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22740a1500 con 0x7f227413d360 2024-01-24T23:21:17.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2749965101 0 0) 0x7f2274071670 con 0x7f227409f1e0 2024-01-24T23:21:17.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 >> v1:172.21.15.115:6792/0 conn(0x7f22740a0a10 legacy=0x7f22741d0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 >> v1:172.21.15.132:6790/0 conn(0x7f227413d360 legacy=0x7f22741d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient: _finish_hunting 0 2024-01-24T23:21:17.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 1 monclient: found mon.a 2024-01-24T23:21:17.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:17.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:21:17.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227409d530 con 0x7f227409f1e0 2024-01-24T23:21:17.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient: _finish_auth 0 2024-01-24T23:21:17.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient: _check_auth_tickets 2024-01-24T23:21:17.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:17.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 5 monclient: authenticate success, global_id 4509 2024-01-24T23:21:17.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 10 monclient: _renew_subs 2024-01-24T23:21:17.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:21:17.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 1 -- 172.21.15.115:0/2048372873 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227409cc90 con 0x7f227409f1e0 2024-01-24T23:21:17.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 10 monclient: _renew_subs 2024-01-24T23:21:17.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:21:17.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f227b24e640 1 -- 172.21.15.115:0/2048372873 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2274070ca0 con 0x7f227409f1e0 2024-01-24T23:21:17.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f226c003cd0 con 0x7f227409f1e0 2024-01-24T23:21:17.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:17.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f226c004200 con 0x7f227409f1e0 2024-01-24T23:21:17.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:17.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f2272ffd640 4 set_mon_vals no callback set 2024-01-24T23:21:17.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.175+0000 7f22717fa640 10 monclient: dump: 2024-01-24T23:21:17.200 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.200 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.200 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.200 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.200 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.200 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.200 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.201 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.201 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.201 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.201 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.201 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.201 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.201 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.201 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.201 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.179+0000 7f227b24e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:17.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.179+0000 7f227b24e640 1 -- 172.21.15.115:0/2048372873 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f22540065d0 con 0x7f2254002870 2024-01-24T23:21:17.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.179+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f226c004360 con 0x7f227409f1e0 2024-01-24T23:21:17.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.179+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f226c043710 con 0x7f227409f1e0 2024-01-24T23:21:17.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.179+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2426378984 0 0) 0x7f22540065d0 con 0x7f2254002870 2024-01-24T23:21:17.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.179+0000 7f22717fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.179+0000 7f22717fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.179+0000 7f22717fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.179+0000 7f22717fa640 10 monclient(hunting): my global_id is 4461 2024-01-24T23:21:17.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.179+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2274071670 con 0x7f2254002870 2024-01-24T23:21:17.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.179+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4243040040 0 0) 0x7f2274071670 con 0x7f2254002870 2024-01-24T23:21:17.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.179+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f22540065d0 con 0x7f2254002870 2024-01-24T23:21:17.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.179+0000 7f22797eb640 10 monclient: get_auth_request con 0x7f2260037860 auth_method 0 2024-01-24T23:21:17.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.179+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2268001070 con 0x7f2254002870 2024-01-24T23:21:17.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.179+0000 7f22717fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.179+0000 7f22717fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:17.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.179+0000 7f22717fa640 10 monclient: dump: 2024-01-24T23:21:17.202 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.202 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.202 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.202 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.203 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.203 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.203 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.203 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.203 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.203 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.203 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.203 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.203 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.203 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.203 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.203 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.179+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1702818943 0 0) 0x7f22540065d0 con 0x7f2254002870 2024-01-24T23:21:17.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.179+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 --> v1:172.21.15.115:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f225400a120 con 0x7f2254002870 2024-01-24T23:21:17.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.179+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f2268004ef0 con 0x7f2254002870 2024-01-24T23:21:17.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.179+0000 7f22717fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:17.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.179+0000 7f22717fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:17.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.179+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 >> v1:172.21.15.115:6792/0 conn(0x7f2254002870 legacy=0x7f2254004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.179+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 >> v1:172.21.15.115:6792/0 conn(0x7f2254002870 legacy=0x7f2254004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:17.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.183+0000 7f227b24e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:21:17.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.183+0000 7f227b24e640 1 -- 172.21.15.115:0/2048372873 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2254009df0 con 0x7f22540090b0 2024-01-24T23:21:17.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.183+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1465074561 0 0) 0x7f2254009df0 con 0x7f22540090b0 2024-01-24T23:21:17.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.183+0000 7f22717fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.183+0000 7f22717fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.183+0000 7f22717fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.183+0000 7f22717fa640 10 monclient(hunting): my global_id is 4470 2024-01-24T23:21:17.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.183+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22540065d0 con 0x7f22540090b0 2024-01-24T23:21:17.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.183+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3574278819 0 0) 0x7f22540065d0 con 0x7f22540090b0 2024-01-24T23:21:17.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.183+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2254009df0 con 0x7f22540090b0 2024-01-24T23:21:17.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.183+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f22680056e0 con 0x7f22540090b0 2024-01-24T23:21:17.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.183+0000 7f22717fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.183+0000 7f22717fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:17.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.183+0000 7f22717fa640 10 monclient: dump: 2024-01-24T23:21:17.206 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.206 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.206 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.206 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.207 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.207 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.207 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.207 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.207 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.207 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.207 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.207 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.207 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.207 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.207 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.207 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.183+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2628663319 0 0) 0x7f2254009df0 con 0x7f22540090b0 2024-01-24T23:21:17.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.183+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 --> v1:172.21.15.115:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f225400c310 con 0x7f22540090b0 2024-01-24T23:21:17.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.183+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7883 (unknown 0 0 3156786822) 0x7f2268005920 con 0x7f22540090b0 2024-01-24T23:21:17.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.183+0000 7f22717fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:21:17.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.183+0000 7f22717fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:17.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.183+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 >> v1:172.21.15.115:6792/0 conn(0x7f22540090b0 legacy=0x7f2254009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.183+0000 7f22717fa640 1 -- 172.21.15.115:0/2048372873 >> v1:172.21.15.115:6792/0 conn(0x7f22540090b0 legacy=0x7f2254009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:17.208 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:21:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:21:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:21:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:21:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 6, 2024-01-24T23:21:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:21:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:21:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:21:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:21:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:21:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:21:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:21:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:21:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:21:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:21:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 19, 2024-01-24T23:21:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:21:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:21:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:21:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:21:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:21:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:21:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:21:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:21:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:21:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:21:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:21:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:21:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:21:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:21:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:21:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:21:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:21:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:21:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:21:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:21:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:21:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:21:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:21:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:21:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:21:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:21:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:21:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:21:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:21:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:21:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:21:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:21:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:21:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:21:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:21:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:21:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:21:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:21:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:21:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:21:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:21:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:21:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:21:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:21:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:21:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:21:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:21:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:21:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:21:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:21:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:21:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:21:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:21:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:21:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:21:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:21:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:21:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:21:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:21:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:21:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:21:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:21:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:21:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:21:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:21:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:21:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:21:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:21:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:21:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": [ 2024-01-24T23:21:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:21:17.227 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:21:17.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.187+0000 7f227b24e640 1 -- 172.21.15.115:0/2048372873 >> v1:172.21.15.132:6812/1835401922 conn(0x7f2260037860 legacy=0x7f2260039d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.187+0000 7f227b24e640 10 monclient: shutdown 2024-01-24T23:21:17.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.187+0000 7f227b24e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:17.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.187+0000 7f227b24e640 1 -- 172.21.15.115:0/2048372873 >> v1:172.21.15.115:6789/0 conn(0x7f227409f1e0 legacy=0x7f2274131f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.187+0000 7f22797eb640 1 -- 172.21.15.115:0/2048372873 reap_dead start 2024-01-24T23:21:17.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.187+0000 7f22797eb640 1 -- 172.21.15.115:0/2048372873 reap_dead start 2024-01-24T23:21:17.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.187+0000 7f227b24e640 1 -- 172.21.15.115:0/2048372873 shutdown_connections 2024-01-24T23:21:17.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.187+0000 7f227b24e640 1 -- 172.21.15.115:0/2048372873 >> 172.21.15.115:0/2048372873 conn(0x7f22740a7660 msgr2=0x7f227406b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:17.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.187+0000 7f227b24e640 1 -- 172.21.15.115:0/2048372873 shutdown_connections 2024-01-24T23:21:17.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.187+0000 7f227b24e640 1 -- 172.21.15.115:0/2048372873 wait complete. 2024-01-24T23:21:17.230 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:21:17.230 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-01-24T23:21:17.230 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-01-24T23:21:17.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 10 monclient: get_monmap_and_config 2024-01-24T23:21:17.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 10 monclient: build_initial_monmap 2024-01-24T23:21:17.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 10 monclient: monmap: 2024-01-24T23:21:17.330 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:17.331 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.331 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:17.332055+0000 2024-01-24T23:21:17.331 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:17.332055+0000 2024-01-24T23:21:17.331 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:17.331 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.331 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:17.331 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:17.331 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:17.331 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:17.331 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:17.331 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:17.331 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:17.331 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:17.331 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:17.332 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 1 Processor -- start 2024-01-24T23:21:17.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 1 -- start start 2024-01-24T23:21:17.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 10 monclient: init 2024-01-24T23:21:17.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:17.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 10 monclient: _add_conns ranks=[6,2,7,8,3,5,1,4,0] 2024-01-24T23:21:17.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 10 monclient(hunting): picked mon.noname-d con 0x7f8418133bf0 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:17.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 10 monclient(hunting): picked mon.noname-b con 0x7f8418133fd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:17.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 10 monclient(hunting): picked mon.noname-i con 0x7f84181343b0 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:17.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f841804ed70 con 0x7f8418133fd0 2024-01-24T23:21:17.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84180a4b60 con 0x7f8418133bf0 2024-01-24T23:21:17.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f841809ae50 con 0x7f84181343b0 2024-01-24T23:21:17.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:17.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 10 monclient(hunting): authenticate will time out at 956.254511s 2024-01-24T23:21:17.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8416ffd640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f8418133bf0 0x7f841809bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:39696/0 (socket says 172.21.15.115:39696) 2024-01-24T23:21:17.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8416ffd640 1 -- 172.21.15.115:0/828202742 learned_addr learned my addr 172.21.15.115:0/828202742 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:17.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f84167fc640 1 --1- 172.21.15.115:0/828202742 >> v1:172.21.15.115:6790/0 conn(0x7f8418133fd0 0x7f841813bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:55534/0 (socket says 172.21.15.115:55534) 2024-01-24T23:21:17.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 1 -- 172.21.15.115:0/828202742 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1669274527 0 0) 0x7f84180a4b60 con 0x7f8418133bf0 2024-01-24T23:21:17.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 10 monclient(hunting): my global_id is 4479 2024-01-24T23:21:17.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 1 -- 172.21.15.115:0/828202742 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8400003530 con 0x7f8418133bf0 2024-01-24T23:21:17.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 1 -- 172.21.15.115:0/828202742 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220784734 0 0) 0x7f841804ed70 con 0x7f8418133fd0 2024-01-24T23:21:17.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 10 monclient(hunting): my global_id is 4439 2024-01-24T23:21:17.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 1 -- 172.21.15.115:0/828202742 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84180a4b60 con 0x7f8418133fd0 2024-01-24T23:21:17.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 1 -- 172.21.15.115:0/828202742 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 735162967 0 0) 0x7f84180a4b60 con 0x7f8418133fd0 2024-01-24T23:21:17.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 1 -- 172.21.15.115:0/828202742 >> v1:172.21.15.132:6792/0 conn(0x7f84181343b0 legacy=0x7f841813f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 1 -- 172.21.15.115:0/828202742 >> v1:172.21.15.115:6792/0 conn(0x7f8418133bf0 legacy=0x7f841809bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:21:17.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 1 monclient: found mon.noname-b 2024-01-24T23:21:17.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:17.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:21:17.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 1 -- 172.21.15.115:0/828202742 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841809b130 con 0x7f8418133fd0 2024-01-24T23:21:17.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 10 monclient: _finish_auth 0 2024-01-24T23:21:17.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:21:17.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:17.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 1 -- 172.21.15.115:0/828202742 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f840c0017c0 con 0x7f8418133fd0 2024-01-24T23:21:17.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 5 monclient: authenticate success, global_id 4439 2024-01-24T23:21:17.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:17.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:21:17.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 10 monclient: dump: 2024-01-24T23:21:17.342 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.342 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.342 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.342 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.342 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.343 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.343 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.343 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.343 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.343 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.343 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.343 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.343 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.343 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.343 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.343 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:17.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 1 -- 172.21.15.115:0/828202742 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f840c001970 con 0x7f8418133fd0 2024-01-24T23:21:17.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:17.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 1 -- 172.21.15.115:0/828202742 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f840c0047f0 con 0x7f8418133fd0 2024-01-24T23:21:17.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:17.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 4 set_mon_vals no callback set 2024-01-24T23:21:17.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:17.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f8415ffb640 10 monclient: dump: 2024-01-24T23:21:17.344 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.344 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.344 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.344 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.344 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.344 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.344 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.345 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.345 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.345 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.345 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.345 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.345 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.345 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.345 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.345 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 10 monclient: shutdown 2024-01-24T23:21:17.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:17.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 1 -- 172.21.15.115:0/828202742 >> v1:172.21.15.115:6790/0 conn(0x7f8418133fd0 legacy=0x7f841813bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 1 -- 172.21.15.115:0/828202742 shutdown_connections 2024-01-24T23:21:17.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 1 -- 172.21.15.115:0/828202742 >> 172.21.15.115:0/828202742 conn(0x7f84180662f0 msgr2=0x7f8418066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:17.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 1 -- 172.21.15.115:0/828202742 shutdown_connections 2024-01-24T23:21:17.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.327+0000 7f841d944640 1 -- 172.21.15.115:0/828202742 wait complete. 2024-01-24T23:21:17.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f841d944640 10 monclient: build_initial_monmap 2024-01-24T23:21:17.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f841d944640 10 monclient: monmap: 2024-01-24T23:21:17.346 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:17.346 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:17.346 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:17.346 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:17.346 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:17.346 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:17.346 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:17.346 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:17.346 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:17.346 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:17.347 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:17.347 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:17.347 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:17.347 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:17.347 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:17.347 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f841d944640 1 Processor -- start 2024-01-24T23:21:17.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f841d944640 1 -- start start 2024-01-24T23:21:17.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f841d944640 10 monclient: init 2024-01-24T23:21:17.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f841d944640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:17.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f841d944640 10 monclient: _add_conns ranks=[2,1,7,0,5,4,3,8,6] 2024-01-24T23:21:17.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f841d944640 10 monclient(hunting): picked mon.noname-b con 0x7f8418133bf0 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:17.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f841d944640 10 monclient(hunting): picked mon.noname-f con 0x7f8418133fd0 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:17.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f841d944640 10 monclient(hunting): picked mon.noname-i con 0x7f84181343b0 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:17.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f841d944640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f841809c810 con 0x7f8418133fd0 2024-01-24T23:21:17.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f841d944640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84180a5990 con 0x7f8418133bf0 2024-01-24T23:21:17.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f841d944640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8418120630 con 0x7f84181343b0 2024-01-24T23:21:17.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f841d944640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:17.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f841d944640 10 monclient(hunting): authenticate will time out at 956.257328s 2024-01-24T23:21:17.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f8416ffd640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f8418133bf0 0x7f8418132f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:55536/0 (socket says 172.21.15.115:55536) 2024-01-24T23:21:17.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f8416ffd640 1 -- 172.21.15.115:0/2481775462 learned_addr learned my addr 172.21.15.115:0/2481775462 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:17.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3647026230 0 0) 0x7f84180a5990 con 0x7f8418133bf0 2024-01-24T23:21:17.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient(hunting): my global_id is 4448 2024-01-24T23:21:17.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8404003530 con 0x7f8418133bf0 2024-01-24T23:21:17.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 702430078 0 0) 0x7f8404003530 con 0x7f8418133bf0 2024-01-24T23:21:17.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f84180a5990 con 0x7f8418133bf0 2024-01-24T23:21:17.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8408001960 con 0x7f8418133bf0 2024-01-24T23:21:17.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:21:17.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient(hunting): dump: 2024-01-24T23:21:17.349 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.349 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.349 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.349 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.349 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.349 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.349 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.349 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.349 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.349 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.350 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.350 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.350 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.350 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.350 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.350 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:17.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3194839266 0 0) 0x7f84180a5990 con 0x7f8418133bf0 2024-01-24T23:21:17.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 >> v1:172.21.15.132:6792/0 conn(0x7f84181343b0 legacy=0x7f84181d3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 >> v1:172.21.15.132:6789/0 conn(0x7f8418133fd0 legacy=0x7f84181d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient: _finish_hunting 0 2024-01-24T23:21:17.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 1 monclient: found mon.b 2024-01-24T23:21:17.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:17.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:17.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84180a2480 con 0x7f8418133bf0 2024-01-24T23:21:17.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient: _finish_auth 0 2024-01-24T23:21:17.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient: _check_auth_tickets 2024-01-24T23:21:17.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:17.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f841d944640 5 monclient: authenticate success, global_id 4448 2024-01-24T23:21:17.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f841d944640 10 monclient: _renew_subs 2024-01-24T23:21:17.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f841d944640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:17.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f841d944640 1 -- 172.21.15.115:0/2481775462 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84181208f0 con 0x7f8418133bf0 2024-01-24T23:21:17.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f841d944640 10 monclient: _renew_subs 2024-01-24T23:21:17.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f841d944640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:17.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f841d944640 1 -- 172.21.15.115:0/2481775462 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f84180a3200 con 0x7f8418133bf0 2024-01-24T23:21:17.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8408001d50 con 0x7f8418133bf0 2024-01-24T23:21:17.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:17.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8408004de0 con 0x7f8418133bf0 2024-01-24T23:21:17.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f8414ff9640 4 set_mon_vals no callback set 2024-01-24T23:21:17.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:17.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient: dump: 2024-01-24T23:21:17.352 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.352 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.352 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.352 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.352 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.352 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.352 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.352 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.353 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.353 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.353 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.353 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.353 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.353 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.353 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.353 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f841d944640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:17.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f841d944640 1 -- 172.21.15.115:0/2481775462 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f83f80065d0 con 0x7f83f8002870 2024-01-24T23:21:17.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f840800f8e0 con 0x7f8418133bf0 2024-01-24T23:21:17.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f8408043750 con 0x7f8418133bf0 2024-01-24T23:21:17.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1826232913 0 0) 0x7f83f80065d0 con 0x7f83f8002870 2024-01-24T23:21:17.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient(hunting): my global_id is 74447 2024-01-24T23:21:17.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84180a5990 con 0x7f83f8002870 2024-01-24T23:21:17.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f84177fe640 10 monclient: get_auth_request con 0x7f8404036fd0 auth_method 0 2024-01-24T23:21:17.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4084622158 0 0) 0x7f84180a5990 con 0x7f83f8002870 2024-01-24T23:21:17.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f83f80065d0 con 0x7f83f8002870 2024-01-24T23:21:17.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f840c001150 con 0x7f83f8002870 2024-01-24T23:21:17.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:21:17.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient: dump: 2024-01-24T23:21:17.354 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.354 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.354 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.354 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.355 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.355 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.355 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.355 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.355 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.355 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.355 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.355 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.355 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.355 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.355 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.355 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 40840950 0 0) 0x7f83f80065d0 con 0x7f83f8002870 2024-01-24T23:21:17.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 --> v1:172.21.15.115:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f83f800a120 con 0x7f83f8002870 2024-01-24T23:21:17.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f840c001730 con 0x7f83f8002870 2024-01-24T23:21:17.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:17.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:17.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 >> v1:172.21.15.115:6793/0 conn(0x7f83f8002870 legacy=0x7f83f8004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.331+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 >> v1:172.21.15.115:6793/0 conn(0x7f83f8002870 legacy=0x7f83f8004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:17.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f841d944640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:21:17.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f841d944640 1 -- 172.21.15.115:0/2481775462 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f83f8009e10 con 0x7f83f80090d0 2024-01-24T23:21:17.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1264557490 0 0) 0x7f83f8009e10 con 0x7f83f80090d0 2024-01-24T23:21:17.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f83fffff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f83fffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f83fffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f83fffff640 10 monclient(hunting): my global_id is 74456 2024-01-24T23:21:17.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83f80065d0 con 0x7f83f80090d0 2024-01-24T23:21:17.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1913023833 0 0) 0x7f83f80065d0 con 0x7f83f80090d0 2024-01-24T23:21:17.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f83f8009e10 con 0x7f83f80090d0 2024-01-24T23:21:17.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f840c005080 con 0x7f83f80090d0 2024-01-24T23:21:17.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f83fffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f83fffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:21:17.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f83fffff640 10 monclient: dump: 2024-01-24T23:21:17.358 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.358 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.358 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.358 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.358 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.358 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.358 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.359 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.359 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.359 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.359 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.359 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.359 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.359 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.359 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.359 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 775141078 0 0) 0x7f83f8009e10 con 0x7f83f80090d0 2024-01-24T23:21:17.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 --> v1:172.21.15.115:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f83f800c310 con 0x7f83f80090d0 2024-01-24T23:21:17.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 2646679467) 0x7f840c0073d0 con 0x7f83f80090d0 2024-01-24T23:21:17.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f83fffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:21:17.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f83fffff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:17.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 >> v1:172.21.15.115:6793/0 conn(0x7f83f80090d0 legacy=0x7f83f8009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f83fffff640 1 -- 172.21.15.115:0/2481775462 >> v1:172.21.15.115:6793/0 conn(0x7f83f80090d0 legacy=0x7f83f8009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:17.360 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:21:17.360 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:21:17.360 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:21:17.360 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:21:17.360 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 6, 2024-01-24T23:21:17.360 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:21:17.360 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:21:17.360 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:21:17.360 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:21:17.361 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:21:17.361 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:21:17.361 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:21:17.361 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:21:17.361 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:21:17.361 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:21:17.361 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.361 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 19, 2024-01-24T23:21:17.361 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:17.361 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:21:17.361 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:21:17.361 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:17.361 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:17.362 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:17.362 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:17.362 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:17.362 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:17.362 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:17.362 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:17.362 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:17.362 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:17.362 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:17.362 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.362 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:21:17.362 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:21:17.362 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:17.362 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:17.363 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:17.363 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:17.363 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:17.363 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:17.363 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:17.363 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:17.363 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:17.363 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:17.363 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:17.363 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.363 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:21:17.363 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:21:17.364 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:21:17.364 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:21:17.364 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:21:17.364 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:21:17.364 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:17.364 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:17.364 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:21:17.364 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:21:17.364 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:21:17.364 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:21:17.364 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:21:17.364 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:21:17.364 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:21:17.365 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:17.365 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:21:17.365 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:17.365 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:17.365 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:17.365 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:17.365 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:17.365 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:17.365 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:17.365 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:17.365 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:17.365 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:17.365 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:17.365 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.366 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:21:17.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.366 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:21:17.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.366 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:21:17.366 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:21:17.366 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.366 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.366 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.366 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:21:17.366 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.366 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.366 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.367 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:21:17.367 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:21:17.367 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.367 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.367 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.367 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:21:17.367 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:21:17.367 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.367 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.367 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.367 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:21:17.368 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.368 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.368 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.368 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:21:17.368 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:21:17.368 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.368 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.368 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.368 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:21:17.368 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:21:17.369 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.369 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.369 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.369 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:21:17.369 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.369 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.369 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:21:17.369 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:21:17.369 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.369 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.369 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:21:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:21:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:21:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:21:17.370 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:21:17.371 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.371 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.371 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.371 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:21:17.371 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:21:17.371 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.371 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.371 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.371 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:21:17.371 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.371 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.372 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.372 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:21:17.372 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:21:17.372 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.372 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.372 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.372 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:21:17.372 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:21:17.372 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.372 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.373 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.373 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:21:17.373 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.373 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.373 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.373 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:21:17.373 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:21:17.373 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.373 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.373 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.373 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:21:17.374 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:21:17.374 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.374 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.374 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.374 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:21:17.374 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.374 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.374 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.374 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:21:17.374 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:21:17.374 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.374 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.374 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.375 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:21:17.375 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:21:17.375 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.375 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.375 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.375 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:21:17.375 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.375 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.375 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.375 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:21:17.375 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:21:17.376 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.376 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.376 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.376 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:21:17.376 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:21:17.376 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.376 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.376 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.376 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:21:17.376 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.376 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.377 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.377 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:21:17.377 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:21:17.377 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.377 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.377 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.377 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.377 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.377 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:21:17.377 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:21:17.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.377 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:17.378 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:17.378 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:17.378 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.378 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:21:17.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.378 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:17.378 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:17.378 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:17.378 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.378 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.378 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:21:17.378 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:21:17.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f841d944640 1 -- 172.21.15.115:0/2481775462 >> v1:172.21.15.132:6812/1835401922 conn(0x7f8404036fd0 legacy=0x7f8404039490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f841d944640 10 monclient: shutdown 2024-01-24T23:21:17.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f841d944640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:17.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f841d944640 1 -- 172.21.15.115:0/2481775462 >> v1:172.21.15.115:6790/0 conn(0x7f8418133bf0 legacy=0x7f8418132f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f84177fe640 1 -- 172.21.15.115:0/2481775462 reap_dead start 2024-01-24T23:21:17.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f84177fe640 1 -- 172.21.15.115:0/2481775462 reap_dead start 2024-01-24T23:21:17.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f841d944640 1 -- 172.21.15.115:0/2481775462 shutdown_connections 2024-01-24T23:21:17.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f841d944640 1 -- 172.21.15.115:0/2481775462 >> 172.21.15.115:0/2481775462 conn(0x7f84180662f0 msgr2=0x7f84181be890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:17.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f841d944640 1 -- 172.21.15.115:0/2481775462 shutdown_connections 2024-01-24T23:21:17.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.339+0000 7f841d944640 1 -- 172.21.15.115:0/2481775462 wait complete. 2024-01-24T23:21:17.381 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:21:17.381 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-01-24T23:21:17.381 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-01-24T23:21:17.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 10 monclient: get_monmap_and_config 2024-01-24T23:21:17.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 10 monclient: build_initial_monmap 2024-01-24T23:21:17.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 10 monclient: monmap: 2024-01-24T23:21:17.490 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:17.490 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.490 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:17.491703+0000 2024-01-24T23:21:17.490 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:17.491703+0000 2024-01-24T23:21:17.490 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:17.490 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.490 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:17.490 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:17.491 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:17.491 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:17.491 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:17.491 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:17.491 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:17.491 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:17.491 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:17.491 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 1 Processor -- start 2024-01-24T23:21:17.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 1 -- start start 2024-01-24T23:21:17.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 10 monclient: init 2024-01-24T23:21:17.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:17.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 10 monclient: _add_conns ranks=[6,2,5,0,7,1,8,4,3] 2024-01-24T23:21:17.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 10 monclient(hunting): picked mon.noname-d con 0x7f019c133bf0 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:17.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 10 monclient(hunting): picked mon.noname-b con 0x7f019c133fd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:17.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 10 monclient(hunting): picked mon.noname-h con 0x7f019c1343b0 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:17.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019c04ed70 con 0x7f019c133fd0 2024-01-24T23:21:17.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019c0a4b60 con 0x7f019c1343b0 2024-01-24T23:21:17.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019c09ae50 con 0x7f019c133bf0 2024-01-24T23:21:17.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:17.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 10 monclient(hunting): authenticate will time out at 956.414282s 2024-01-24T23:21:17.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019bfff640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f019c133bf0 0x7f019c09bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:39706/0 (socket says 172.21.15.115:39706) 2024-01-24T23:21:17.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019bfff640 1 -- 172.21.15.115:0/4286151840 learned_addr learned my addr 172.21.15.115:0/4286151840 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:17.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 1 -- 172.21.15.115:0/4286151840 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 198871785 0 0) 0x7f019c09ae50 con 0x7f019c133bf0 2024-01-24T23:21:17.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 10 monclient(hunting): my global_id is 4488 2024-01-24T23:21:17.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 1 -- 172.21.15.115:0/4286151840 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f017c003530 con 0x7f019c133bf0 2024-01-24T23:21:17.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 1 -- 172.21.15.115:0/4286151840 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540412385 0 0) 0x7f019c04ed70 con 0x7f019c133fd0 2024-01-24T23:21:17.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 10 monclient(hunting): my global_id is 4457 2024-01-24T23:21:17.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 1 -- 172.21.15.115:0/4286151840 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f019c09ae50 con 0x7f019c133fd0 2024-01-24T23:21:17.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 1 -- 172.21.15.115:0/4286151840 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1661088474 0 0) 0x7f017c003530 con 0x7f019c133bf0 2024-01-24T23:21:17.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 1 -- 172.21.15.115:0/4286151840 >> v1:172.21.15.132:6791/0 conn(0x7f019c1343b0 legacy=0x7f019c13f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 1 -- 172.21.15.115:0/4286151840 >> v1:172.21.15.115:6790/0 conn(0x7f019c133fd0 legacy=0x7f019c13bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 10 monclient: _finish_hunting 0 2024-01-24T23:21:17.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 1 monclient: found mon.noname-d 2024-01-24T23:21:17.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:17.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:21:17.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 1 -- 172.21.15.115:0/4286151840 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019c09b130 con 0x7f019c133bf0 2024-01-24T23:21:17.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 10 monclient: _finish_auth 0 2024-01-24T23:21:17.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 10 monclient: _check_auth_tickets 2024-01-24T23:21:17.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:17.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 5 monclient: authenticate success, global_id 4488 2024-01-24T23:21:17.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 1 -- 172.21.15.115:0/4286151840 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0194001b40 con 0x7f019c133bf0 2024-01-24T23:21:17.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:17.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:21:17.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 10 monclient: dump: 2024-01-24T23:21:17.502 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.502 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.502 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.502 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.502 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.502 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.502 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.502 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.503 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.503 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.503 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.503 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.503 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.503 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.503 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.503 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:17.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 1 -- 172.21.15.115:0/4286151840 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0194001de0 con 0x7f019c133bf0 2024-01-24T23:21:17.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:17.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:17.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 1 -- 172.21.15.115:0/4286151840 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0194004f70 con 0x7f019c133bf0 2024-01-24T23:21:17.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 4 set_mon_vals no callback set 2024-01-24T23:21:17.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:17.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f019affd640 10 monclient: dump: 2024-01-24T23:21:17.504 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.504 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.504 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.504 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.504 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.504 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.504 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.504 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.504 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.504 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.504 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.505 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.505 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.505 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.505 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.505 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 10 monclient: shutdown 2024-01-24T23:21:17.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:17.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 1 -- 172.21.15.115:0/4286151840 >> v1:172.21.15.115:6792/0 conn(0x7f019c133bf0 legacy=0x7f019c09bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 1 -- 172.21.15.115:0/4286151840 shutdown_connections 2024-01-24T23:21:17.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 1 -- 172.21.15.115:0/4286151840 >> 172.21.15.115:0/4286151840 conn(0x7f019c0662f0 msgr2=0x7f019c066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:17.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 1 -- 172.21.15.115:0/4286151840 shutdown_connections 2024-01-24T23:21:17.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.487+0000 7f01a25f4640 1 -- 172.21.15.115:0/4286151840 wait complete. 2024-01-24T23:21:17.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a25f4640 10 monclient: build_initial_monmap 2024-01-24T23:21:17.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a25f4640 10 monclient: monmap: 2024-01-24T23:21:17.506 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:17.506 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:17.506 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:17.506 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:17.506 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:17.506 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:17.506 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:17.506 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:17.506 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:17.506 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:17.506 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:17.506 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:17.506 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:17.506 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:17.507 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:17.507 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a25f4640 1 Processor -- start 2024-01-24T23:21:17.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a25f4640 1 -- start start 2024-01-24T23:21:17.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a25f4640 10 monclient: init 2024-01-24T23:21:17.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a25f4640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:17.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a25f4640 10 monclient: _add_conns ranks=[4,1,8,0,7,5,2,3,6] 2024-01-24T23:21:17.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a25f4640 10 monclient(hunting): picked mon.noname-c con 0x7f019c133bf0 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:17.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a25f4640 10 monclient(hunting): picked mon.noname-f con 0x7f019c133fd0 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:17.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a25f4640 10 monclient(hunting): picked mon.noname-e con 0x7f019c1343b0 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:17.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a25f4640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019c09c810 con 0x7f019c133fd0 2024-01-24T23:21:17.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a25f4640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019c0a5990 con 0x7f019c133bf0 2024-01-24T23:21:17.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a25f4640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019c0a2400 con 0x7f019c1343b0 2024-01-24T23:21:17.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a25f4640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:17.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a25f4640 10 monclient(hunting): authenticate will time out at 956.417332s 2024-01-24T23:21:17.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f019bfff640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f019c133bf0 0x7f019c132f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:60004/0 (socket says 172.21.15.115:60004) 2024-01-24T23:21:17.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f019bfff640 1 -- 172.21.15.115:0/1516026118 learned_addr learned my addr 172.21.15.115:0/1516026118 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:17.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 258213191 0 0) 0x7f019c0a5990 con 0x7f019c133bf0 2024-01-24T23:21:17.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient(hunting): my global_id is 74524 2024-01-24T23:21:17.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0180003530 con 0x7f019c133bf0 2024-01-24T23:21:17.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1009109097 0 0) 0x7f019c0a2400 con 0x7f019c1343b0 2024-01-24T23:21:17.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient(hunting): my global_id is 74465 2024-01-24T23:21:17.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f019c0a5990 con 0x7f019c1343b0 2024-01-24T23:21:17.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1634008654 0 0) 0x7f0180003530 con 0x7f019c133bf0 2024-01-24T23:21:17.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f019c0a2400 con 0x7f019c133bf0 2024-01-24T23:21:17.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0194001b40 con 0x7f019c133bf0 2024-01-24T23:21:17.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:21:17.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient(hunting): dump: 2024-01-24T23:21:17.509 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.509 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.509 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.509 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.509 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.509 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.509 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.509 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.510 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.510 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.510 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.510 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.510 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.510 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.510 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.510 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:17.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3356331594 0 0) 0x7f019c0a5990 con 0x7f019c1343b0 2024-01-24T23:21:17.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0180003530 con 0x7f019c1343b0 2024-01-24T23:21:17.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 914043738 0 0) 0x7f019c09c810 con 0x7f019c133fd0 2024-01-24T23:21:17.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient(hunting): my global_id is 4438 2024-01-24T23:21:17.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f019c0a5990 con 0x7f019c133fd0 2024-01-24T23:21:17.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f018c0019a0 con 0x7f019c1343b0 2024-01-24T23:21:17.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:21:17.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient(hunting): dump: 2024-01-24T23:21:17.511 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.511 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.511 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.511 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.511 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.511 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.511 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.511 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.511 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.512 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.512 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.512 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.512 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.512 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.512 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.512 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 477237928 0 0) 0x7f019c0a2400 con 0x7f019c133bf0 2024-01-24T23:21:17.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 >> v1:172.21.15.115:6793/0 conn(0x7f019c1343b0 legacy=0x7f019c1d3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 >> v1:172.21.15.132:6789/0 conn(0x7f019c133fd0 legacy=0x7f019c1d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:21:17.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 1 monclient: found mon.c 2024-01-24T23:21:17.512 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:17.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:17.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019c1c87b0 con 0x7f019c133bf0 2024-01-24T23:21:17.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient: _finish_auth 0 2024-01-24T23:21:17.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:21:17.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:17.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a25f4640 5 monclient: authenticate success, global_id 74524 2024-01-24T23:21:17.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a25f4640 10 monclient: _renew_subs 2024-01-24T23:21:17.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a25f4640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:17.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a25f4640 1 -- 172.21.15.115:0/1516026118 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f019c1208f0 con 0x7f019c133bf0 2024-01-24T23:21:17.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a25f4640 10 monclient: _renew_subs 2024-01-24T23:21:17.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a25f4640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:17.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a25f4640 1 -- 172.21.15.115:0/1516026118 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f019c133620 con 0x7f019c133bf0 2024-01-24T23:21:17.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0194004f90 con 0x7f019c133bf0 2024-01-24T23:21:17.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:17.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0194005520 con 0x7f019c133bf0 2024-01-24T23:21:17.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:17.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f019a7fc640 4 set_mon_vals no callback set 2024-01-24T23:21:17.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient: dump: 2024-01-24T23:21:17.514 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.514 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.514 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.514 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.514 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.514 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.514 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.514 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.514 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.514 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.514 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.514 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.515 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.515 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.515 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.515 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a25f4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:17.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a25f4640 1 -- 172.21.15.115:0/1516026118 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f01740065d0 con 0x7f0174002870 2024-01-24T23:21:17.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f019400ffa0 con 0x7f019c133bf0 2024-01-24T23:21:17.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f0194043b70 con 0x7f019c133bf0 2024-01-24T23:21:17.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3117742183 0 0) 0x7f01740065d0 con 0x7f0174002870 2024-01-24T23:21:17.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 10 monclient(hunting): my global_id is 4447 2024-01-24T23:21:17.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f019c0a2400 con 0x7f0174002870 2024-01-24T23:21:17.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.491+0000 7f01a0b91640 10 monclient: get_auth_request con 0x7f0180037950 auth_method 0 2024-01-24T23:21:17.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.495+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 362319893 0 0) 0x7f019c0a2400 con 0x7f0174002870 2024-01-24T23:21:17.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.495+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f01740065d0 con 0x7f0174002870 2024-01-24T23:21:17.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.495+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0184001400 con 0x7f0174002870 2024-01-24T23:21:17.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.495+0000 7f0198ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.495+0000 7f0198ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:21:17.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.495+0000 7f0198ff9640 10 monclient: dump: 2024-01-24T23:21:17.516 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.516 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.516 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.516 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.516 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.516 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.516 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.516 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.517 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.517 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.517 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.517 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.517 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.517 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.517 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.517 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.495+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 778380691 0 0) 0x7f01740065d0 con 0x7f0174002870 2024-01-24T23:21:17.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.495+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 --> v1:172.21.15.132:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f017400a120 con 0x7f0174002870 2024-01-24T23:21:17.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.495+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 <== mon.1 v1:172.21.15.132:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f0184003600 con 0x7f0174002870 2024-01-24T23:21:17.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.495+0000 7f0198ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:17.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.495+0000 7f0198ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:17.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.495+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 >> v1:172.21.15.132:6789/0 conn(0x7f0174002870 legacy=0x7f0174004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.495+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 >> v1:172.21.15.132:6789/0 conn(0x7f0174002870 legacy=0x7f0174004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:17.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.499+0000 7f01a25f4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:21:17.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.499+0000 7f01a25f4640 1 -- 172.21.15.115:0/1516026118 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0174009e10 con 0x7f01740090d0 2024-01-24T23:21:17.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.499+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 187533378 0 0) 0x7f0174009e10 con 0x7f01740090d0 2024-01-24T23:21:17.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.499+0000 7f0198ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.499+0000 7f0198ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.499+0000 7f0198ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.499+0000 7f0198ff9640 10 monclient(hunting): my global_id is 4456 2024-01-24T23:21:17.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.499+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01740065d0 con 0x7f01740090d0 2024-01-24T23:21:17.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.503+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3663500969 0 0) 0x7f01740065d0 con 0x7f01740090d0 2024-01-24T23:21:17.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.503+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0174009e10 con 0x7f01740090d0 2024-01-24T23:21:17.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.503+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0184003ca0 con 0x7f01740090d0 2024-01-24T23:21:17.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.503+0000 7f0198ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.503+0000 7f0198ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:21:17.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.503+0000 7f0198ff9640 10 monclient: dump: 2024-01-24T23:21:17.520 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.520 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.520 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.520 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.520 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.520 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.520 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.520 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.520 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.520 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.520 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.520 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.520 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.520 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.521 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.521 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.503+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1080782759 0 0) 0x7f0174009e10 con 0x7f01740090d0 2024-01-24T23:21:17.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.503+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 --> v1:172.21.15.132:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f017400c310 con 0x7f01740090d0 2024-01-24T23:21:17.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.503+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 <== mon.1 v1:172.21.15.132:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 1826705355) 0x7f0184004100 con 0x7f01740090d0 2024-01-24T23:21:17.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.503+0000 7f0198ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:21:17.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.503+0000 7f0198ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:17.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.503+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 >> v1:172.21.15.132:6789/0 conn(0x7f01740090d0 legacy=0x7f0174009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.503+0000 7f0198ff9640 1 -- 172.21.15.115:0/1516026118 >> v1:172.21.15.132:6789/0 conn(0x7f01740090d0 legacy=0x7f0174009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:17.521 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:21:17.521 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:21:17.522 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:21:17.522 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:21:17.522 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 6, 2024-01-24T23:21:17.522 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:21:17.522 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:21:17.522 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:21:17.522 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:21:17.522 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:21:17.522 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:21:17.522 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:21:17.522 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:21:17.522 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:21:17.522 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:21:17.522 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.523 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 19, 2024-01-24T23:21:17.523 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:17.523 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:21:17.523 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:21:17.523 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:17.523 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:17.523 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:17.523 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:17.523 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:17.523 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:17.523 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:17.523 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:17.523 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:17.523 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:17.523 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:17.524 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.524 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:21:17.524 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:21:17.524 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:17.524 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:17.524 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:17.524 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:17.524 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:17.524 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:17.524 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:17.524 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:17.524 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:17.524 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:17.525 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:17.525 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.525 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:21:17.525 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:21:17.525 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:21:17.525 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:21:17.525 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:21:17.525 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:21:17.525 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:17.525 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:17.525 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:21:17.525 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:21:17.525 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:21:17.525 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:21:17.526 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:21:17.526 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:21:17.526 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:21:17.526 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:17.526 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:21:17.526 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:17.526 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:17.526 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:17.526 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:17.526 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:17.526 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:17.526 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:17.526 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:17.526 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:17.526 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:17.527 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:17.527 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.527 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:21:17.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.527 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:21:17.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.527 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:21:17.527 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:21:17.527 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.527 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.527 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.527 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:21:17.527 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.528 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.528 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.528 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:21:17.528 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:21:17.528 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.528 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.528 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.528 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:21:17.528 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:21:17.528 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.528 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.529 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.529 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:21:17.529 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.529 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.529 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.529 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:21:17.529 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:21:17.529 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.529 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.529 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.529 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:21:17.530 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:21:17.530 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.530 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.530 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.530 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:21:17.530 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.530 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.530 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.530 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:21:17.530 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:21:17.530 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.530 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.530 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.531 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:21:17.531 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:21:17.531 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.531 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.531 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.531 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:21:17.531 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.531 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.531 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.531 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:21:17.531 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:21:17.531 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.532 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.532 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.532 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:21:17.532 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:21:17.532 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.532 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.532 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.532 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:21:17.532 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.532 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.532 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.533 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:21:17.533 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:21:17.533 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.533 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.533 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.533 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:21:17.533 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:21:17.533 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.533 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.533 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.533 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:21:17.534 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.534 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.534 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.534 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:21:17.534 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:21:17.534 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.534 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.534 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.534 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:21:17.534 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:21:17.534 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.534 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.535 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.535 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:21:17.535 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.535 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.535 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.535 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:21:17.535 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:21:17.535 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.535 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.535 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.535 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:21:17.535 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:21:17.535 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.536 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.536 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.536 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:21:17.536 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.536 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.536 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.536 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:21:17.536 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:21:17.536 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.536 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.536 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.537 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:21:17.537 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:21:17.537 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.537 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.537 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.537 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:21:17.537 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.537 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.537 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.537 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:21:17.537 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:21:17.537 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.538 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.538 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.538 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.538 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.538 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:21:17.538 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:21:17.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.538 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:17.538 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:17.538 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:17.538 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.538 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.538 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:21:17.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.539 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:17.539 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:17.539 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:21:17.539 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.539 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.539 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:21:17.539 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:21:17.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.503+0000 7f01a25f4640 1 -- 172.21.15.115:0/1516026118 >> v1:172.21.15.132:6812/1835401922 conn(0x7f0180037950 legacy=0x7f0180039e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.503+0000 7f01a25f4640 10 monclient: shutdown 2024-01-24T23:21:17.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.503+0000 7f01a25f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:17.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.503+0000 7f01a25f4640 1 -- 172.21.15.115:0/1516026118 >> v1:172.21.15.115:6791/0 conn(0x7f019c133bf0 legacy=0x7f019c132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.503+0000 7f01a0b91640 1 -- 172.21.15.115:0/1516026118 reap_dead start 2024-01-24T23:21:17.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.503+0000 7f01a0b91640 1 -- 172.21.15.115:0/1516026118 reap_dead start 2024-01-24T23:21:17.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.503+0000 7f01a25f4640 1 -- 172.21.15.115:0/1516026118 shutdown_connections 2024-01-24T23:21:17.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.503+0000 7f01a25f4640 1 -- 172.21.15.115:0/1516026118 >> 172.21.15.115:0/1516026118 conn(0x7f019c0662f0 msgr2=0x7f019c0a6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:17.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.503+0000 7f01a25f4640 1 -- 172.21.15.115:0/1516026118 shutdown_connections 2024-01-24T23:21:17.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.503+0000 7f01a25f4640 1 -- 172.21.15.115:0/1516026118 wait complete. 2024-01-24T23:21:17.541 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:21:17.541 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-01-24T23:21:17.541 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-01-24T23:21:17.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.647+0000 7f1ea2fbb640 10 monclient: get_monmap_and_config 2024-01-24T23:21:17.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.647+0000 7f1ea2fbb640 10 monclient: build_initial_monmap 2024-01-24T23:21:17.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.647+0000 7f1ea2fbb640 10 monclient: monmap: 2024-01-24T23:21:17.653 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:17.653 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.653 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:17.654211+0000 2024-01-24T23:21:17.653 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:17.654211+0000 2024-01-24T23:21:17.653 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:17.653 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.653 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:17.653 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:17.653 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:17.653 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:17.653 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:17.653 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:17.653 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:17.653 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:17.653 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:17.654 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 1 Processor -- start 2024-01-24T23:21:17.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 1 -- start start 2024-01-24T23:21:17.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 10 monclient: init 2024-01-24T23:21:17.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:17.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 10 monclient: _add_conns ranks=[4,2,3,0,1,7,5,8,6] 2024-01-24T23:21:17.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 10 monclient(hunting): picked mon.noname-c con 0x7f1e9c133bf0 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:17.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 10 monclient(hunting): picked mon.noname-b con 0x7f1e9c133fd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:17.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 10 monclient(hunting): picked mon.noname-g con 0x7f1e9c134430 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:17.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e9c04ed70 con 0x7f1e9c133fd0 2024-01-24T23:21:17.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e9c09af30 con 0x7f1e9c134430 2024-01-24T23:21:17.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e9c09b210 con 0x7f1e9c133bf0 2024-01-24T23:21:17.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:17.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 10 monclient(hunting): authenticate will time out at 956.576709s 2024-01-24T23:21:17.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea0d57640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f1e9c133bf0 0x7f1e9c09bd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:60020/0 (socket says 172.21.15.115:60020) 2024-01-24T23:21:17.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea0d57640 1 -- 172.21.15.115:0/1973950482 learned_addr learned my addr 172.21.15.115:0/1973950482 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:17.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 1 -- 172.21.15.115:0/1973950482 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 234332277 0 0) 0x7f1e9c04ed70 con 0x7f1e9c133fd0 2024-01-24T23:21:17.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 10 monclient(hunting): my global_id is 4466 2024-01-24T23:21:17.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 1 -- 172.21.15.115:0/1973950482 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e84003530 con 0x7f1e9c133fd0 2024-01-24T23:21:17.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 1 -- 172.21.15.115:0/1973950482 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 273992548 0 0) 0x7f1e9c09b210 con 0x7f1e9c133bf0 2024-01-24T23:21:17.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 10 monclient(hunting): my global_id is 74533 2024-01-24T23:21:17.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 1 -- 172.21.15.115:0/1973950482 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e9c04ed70 con 0x7f1e9c133bf0 2024-01-24T23:21:17.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 1 -- 172.21.15.115:0/1973950482 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1949856785 0 0) 0x7f1e9c04ed70 con 0x7f1e9c133bf0 2024-01-24T23:21:17.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 1 -- 172.21.15.115:0/1973950482 >> v1:172.21.15.132:6790/0 conn(0x7f1e9c134430 legacy=0x7f1e9c13f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 1 -- 172.21.15.115:0/1973950482 >> v1:172.21.15.115:6790/0 conn(0x7f1e9c133fd0 legacy=0x7f1e9c13bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:21:17.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 1 monclient: found mon.noname-c 2024-01-24T23:21:17.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:17.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:21:17.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 1 -- 172.21.15.115:0/1973950482 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9c09b4f0 con 0x7f1e9c133bf0 2024-01-24T23:21:17.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 10 monclient: _finish_auth 0 2024-01-24T23:21:17.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:21:17.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:17.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 1 -- 172.21.15.115:0/1973950482 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1e940017c0 con 0x7f1e9c133bf0 2024-01-24T23:21:17.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:21:17.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 10 monclient: dump: 2024-01-24T23:21:17.663 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.663 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.663 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.663 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.663 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.663 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.663 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.663 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.663 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.663 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.664 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.664 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.664 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.664 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.664 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.664 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 5 monclient: authenticate success, global_id 74533 2024-01-24T23:21:17.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:17.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 1 -- 172.21.15.115:0/1973950482 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1e94001970 con 0x7f1e9c133bf0 2024-01-24T23:21:17.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:17.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:17.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 4 set_mon_vals no callback set 2024-01-24T23:21:17.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 1 -- 172.21.15.115:0/1973950482 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1e940048b0 con 0x7f1e9c133bf0 2024-01-24T23:21:17.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:17.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e9b7fe640 10 monclient: dump: 2024-01-24T23:21:17.665 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.665 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.665 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.665 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.665 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.665 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.665 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.665 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.665 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.665 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.665 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.665 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.666 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.666 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.666 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.666 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 10 monclient: shutdown 2024-01-24T23:21:17.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:17.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 1 -- 172.21.15.115:0/1973950482 >> v1:172.21.15.115:6791/0 conn(0x7f1e9c133bf0 legacy=0x7f1e9c09bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 1 -- 172.21.15.115:0/1973950482 shutdown_connections 2024-01-24T23:21:17.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 1 -- 172.21.15.115:0/1973950482 >> 172.21.15.115:0/1973950482 conn(0x7f1e9c0a9910 msgr2=0x7f1e9c0a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:17.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 1 -- 172.21.15.115:0/1973950482 shutdown_connections 2024-01-24T23:21:17.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 1 -- 172.21.15.115:0/1973950482 wait complete. 2024-01-24T23:21:17.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 10 monclient: build_initial_monmap 2024-01-24T23:21:17.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 10 monclient: monmap: 2024-01-24T23:21:17.666 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:17.667 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:17.667 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:17.667 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:17.667 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:17.667 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:17.667 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:17.667 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:17.667 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:17.667 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:17.667 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:17.667 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:17.667 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:17.667 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:17.667 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:17.667 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 1 Processor -- start 2024-01-24T23:21:17.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 1 -- start start 2024-01-24T23:21:17.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 10 monclient: init 2024-01-24T23:21:17.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:17.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 10 monclient: _add_conns ranks=[4,7,8,3,5,2,1,6,0] 2024-01-24T23:21:17.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 10 monclient(hunting): picked mon.noname-c con 0x7f1e9c133bf0 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:17.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 10 monclient(hunting): picked mon.noname-i con 0x7f1e9c133fd0 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:17.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 10 monclient(hunting): picked mon.noname-e con 0x7f1e9c134430 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:17.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e9c09c8f0 con 0x7f1e9c133bf0 2024-01-24T23:21:17.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e9c0a5960 con 0x7f1e9c133fd0 2024-01-24T23:21:17.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e9c13a560 con 0x7f1e9c134430 2024-01-24T23:21:17.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:17.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea2fbb640 10 monclient(hunting): authenticate will time out at 956.579637s 2024-01-24T23:21:17.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea0d57640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f1e9c133bf0 0x7f1e9c137ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:60032/0 (socket says 172.21.15.115:60032) 2024-01-24T23:21:17.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1ea0d57640 1 -- 172.21.15.115:0/891246963 learned_addr learned my addr 172.21.15.115:0/891246963 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:17.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4215504322 0 0) 0x7f1e9c09c8f0 con 0x7f1e9c133bf0 2024-01-24T23:21:17.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e997fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e997fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e997fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.651+0000 7f1e997fa640 10 monclient(hunting): my global_id is 74542 2024-01-24T23:21:17.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e88003530 con 0x7f1e9c133bf0 2024-01-24T23:21:17.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245902757 0 0) 0x7f1e9c13a560 con 0x7f1e9c134430 2024-01-24T23:21:17.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient(hunting): my global_id is 74474 2024-01-24T23:21:17.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e9c09c8f0 con 0x7f1e9c134430 2024-01-24T23:21:17.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1695853985 0 0) 0x7f1e9c09c8f0 con 0x7f1e9c134430 2024-01-24T23:21:17.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1e9c13a560 con 0x7f1e9c134430 2024-01-24T23:21:17.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 591329108 0 0) 0x7f1e88003530 con 0x7f1e9c133bf0 2024-01-24T23:21:17.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1e9c09c8f0 con 0x7f1e9c133bf0 2024-01-24T23:21:17.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1e90001900 con 0x7f1e9c134430 2024-01-24T23:21:17.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:21:17.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient(hunting): dump: 2024-01-24T23:21:17.670 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.670 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.670 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.670 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.670 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.670 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.670 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.670 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.671 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.671 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.671 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.671 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.671 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.671 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.671 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.671 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:17.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1e94001780 con 0x7f1e9c133bf0 2024-01-24T23:21:17.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:17.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient(hunting): dump: 2024-01-24T23:21:17.671 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.671 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.672 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.672 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.672 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.672 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.672 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.672 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.672 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.672 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.672 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.672 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.672 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.672 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.672 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.672 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4205652344 0 0) 0x7f1e9c0a5960 con 0x7f1e9c133fd0 2024-01-24T23:21:17.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient(hunting): my global_id is 4516 2024-01-24T23:21:17.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e88003530 con 0x7f1e9c133fd0 2024-01-24T23:21:17.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 732026626 0 0) 0x7f1e9c13a560 con 0x7f1e9c134430 2024-01-24T23:21:17.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 >> v1:172.21.15.132:6792/0 conn(0x7f1e9c133fd0 legacy=0x7f1e9c1382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 >> v1:172.21.15.115:6791/0 conn(0x7f1e9c133bf0 legacy=0x7f1e9c137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient: _finish_hunting 0 2024-01-24T23:21:17.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 monclient: found mon.e 2024-01-24T23:21:17.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:17.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:17.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9c13a840 con 0x7f1e9c134430 2024-01-24T23:21:17.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient: _finish_auth 0 2024-01-24T23:21:17.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient: _check_auth_tickets 2024-01-24T23:21:17.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:17.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1ea2fbb640 5 monclient: authenticate success, global_id 74474 2024-01-24T23:21:17.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1ea2fbb640 10 monclient: _renew_subs 2024-01-24T23:21:17.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1ea2fbb640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:17.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1ea2fbb640 1 -- 172.21.15.115:0/891246963 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e9c13f6b0 con 0x7f1e9c134430 2024-01-24T23:21:17.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1ea2fbb640 10 monclient: _renew_subs 2024-01-24T23:21:17.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1ea2fbb640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:21:17.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1ea2fbb640 1 -- 172.21.15.115:0/891246963 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e9c1389c0 con 0x7f1e9c134430 2024-01-24T23:21:17.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1e90001ab0 con 0x7f1e9c134430 2024-01-24T23:21:17.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:17.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1e90003c10 con 0x7f1e9c134430 2024-01-24T23:21:17.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e9affd640 4 set_mon_vals no callback set 2024-01-24T23:21:17.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:21:17.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient: dump: 2024-01-24T23:21:17.675 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.675 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.675 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.675 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.675 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.675 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.675 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.675 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.675 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.675 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.675 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.675 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.675 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.675 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.675 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.675 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1ea2fbb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:17.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1ea2fbb640 1 -- 172.21.15.115:0/891246963 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1e7c0065d0 con 0x7f1e7c002870 2024-01-24T23:21:17.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f1e9000e750 con 0x7f1e9c134430 2024-01-24T23:21:17.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f1e90042360 con 0x7f1e9c134430 2024-01-24T23:21:17.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e9bfff640 10 monclient: get_auth_request con 0x7f1e88037870 auth_method 0 2024-01-24T23:21:17.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045162760 0 0) 0x7f1e7c0065d0 con 0x7f1e7c002870 2024-01-24T23:21:17.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient(hunting): my global_id is 4449 2024-01-24T23:21:17.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e9c13a560 con 0x7f1e7c002870 2024-01-24T23:21:17.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2336025022 0 0) 0x7f1e9c13a560 con 0x7f1e7c002870 2024-01-24T23:21:17.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1e7c0065d0 con 0x7f1e7c002870 2024-01-24T23:21:17.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1e94001220 con 0x7f1e7c002870 2024-01-24T23:21:17.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:21:17.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient: dump: 2024-01-24T23:21:17.677 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.677 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.677 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.677 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.677 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.677 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.677 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.677 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.677 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.677 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.678 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.678 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.678 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.678 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.678 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.678 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 578640591 0 0) 0x7f1e7c0065d0 con 0x7f1e7c002870 2024-01-24T23:21:17.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 --> v1:172.21.15.132:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e7c00a120 con 0x7f1e7c002870 2024-01-24T23:21:17.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f1e940074f0 con 0x7f1e7c002870 2024-01-24T23:21:17.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:17.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:17.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 >> v1:172.21.15.132:6790/0 conn(0x7f1e7c002870 legacy=0x7f1e7c004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.655+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 >> v1:172.21.15.132:6790/0 conn(0x7f1e7c002870 legacy=0x7f1e7c004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:17.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.663+0000 7f1ea2fbb640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:21:17.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.663+0000 7f1ea2fbb640 1 -- 172.21.15.115:0/891246963 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1e7c009e10 con 0x7f1e7c0090d0 2024-01-24T23:21:17.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.663+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3976178614 0 0) 0x7f1e7c009e10 con 0x7f1e7c0090d0 2024-01-24T23:21:17.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.663+0000 7f1e997fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.663+0000 7f1e997fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.663+0000 7f1e997fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.663+0000 7f1e997fa640 10 monclient(hunting): my global_id is 4458 2024-01-24T23:21:17.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.663+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e7c0065d0 con 0x7f1e7c0090d0 2024-01-24T23:21:17.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.663+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2611986625 0 0) 0x7f1e7c0065d0 con 0x7f1e7c0090d0 2024-01-24T23:21:17.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.663+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1e7c009e10 con 0x7f1e7c0090d0 2024-01-24T23:21:17.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.663+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1e94005cc0 con 0x7f1e7c0090d0 2024-01-24T23:21:17.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.663+0000 7f1e997fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.663+0000 7f1e997fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:21:17.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.663+0000 7f1e997fa640 10 monclient: dump: 2024-01-24T23:21:17.681 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.681 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.681 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.681 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.681 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.681 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.681 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.681 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.681 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.681 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.681 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.681 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.682 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.682 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.682 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.682 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.663+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1354109602 0 0) 0x7f1e7c009e10 con 0x7f1e7c0090d0 2024-01-24T23:21:17.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.663+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 --> v1:172.21.15.132:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1e7c00c310 con 0x7f1e7c0090d0 2024-01-24T23:21:17.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.667+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 1916154894) 0x7f1e94007270 con 0x7f1e7c0090d0 2024-01-24T23:21:17.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.667+0000 7f1e997fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:21:17.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.667+0000 7f1e997fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:17.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.667+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 >> v1:172.21.15.132:6790/0 conn(0x7f1e7c0090d0 legacy=0x7f1e7c009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.667+0000 7f1e997fa640 1 -- 172.21.15.115:0/891246963 >> v1:172.21.15.132:6790/0 conn(0x7f1e7c0090d0 legacy=0x7f1e7c009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:17.682 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:21:17.683 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:21:17.683 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:21:17.683 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:21:17.683 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 6, 2024-01-24T23:21:17.683 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:21:17.683 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:21:17.683 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:21:17.683 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:21:17.683 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:21:17.683 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:21:17.683 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:21:17.683 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:21:17.684 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:21:17.684 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:21:17.684 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.684 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 19, 2024-01-24T23:21:17.684 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:17.684 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:21:17.684 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:21:17.684 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:17.684 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:17.684 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:17.684 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:17.684 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:17.684 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:17.684 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:17.685 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:17.685 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:17.685 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:17.685 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:17.685 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.685 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:21:17.685 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:21:17.685 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:17.685 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:17.685 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:17.685 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:17.685 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:17.685 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:17.685 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:17.686 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:17.686 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:17.686 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:17.686 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:17.686 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.686 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:21:17.686 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:21:17.686 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:21:17.686 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:21:17.686 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:21:17.686 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:21:17.686 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:17.686 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:17.686 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:21:17.687 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:21:17.687 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:21:17.687 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:21:17.687 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:21:17.687 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:21:17.687 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:21:17.687 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:17.687 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:21:17.687 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:17.687 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:17.687 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:17.687 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:17.687 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:17.687 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:17.688 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:17.688 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:17.688 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:17.688 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:17.688 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:17.688 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.688 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:21:17.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.688 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:21:17.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.688 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:21:17.688 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:21:17.688 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.688 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.689 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.689 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:21:17.689 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.689 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.689 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.689 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:21:17.689 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:21:17.689 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.689 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.689 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.690 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:21:17.690 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:21:17.690 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.690 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.690 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.690 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:21:17.690 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.690 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.690 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.690 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:21:17.690 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:21:17.690 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.691 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.691 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.691 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:21:17.691 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:21:17.691 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.691 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.691 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.691 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:21:17.691 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.691 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.691 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.692 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:21:17.692 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:21:17.692 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.692 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.692 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.692 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:21:17.692 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:21:17.692 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.692 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.692 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.693 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:21:17.693 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.693 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.693 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.693 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:21:17.693 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:21:17.693 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.693 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.693 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.693 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:21:17.693 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:21:17.693 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.694 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.694 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.694 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:21:17.694 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.694 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.694 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.694 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:21:17.694 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:21:17.694 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.694 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.694 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.695 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:21:17.695 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:21:17.695 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.695 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.695 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.695 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:21:17.695 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.695 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.695 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.695 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:21:17.695 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:21:17.696 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.696 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.696 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.696 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:21:17.696 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:21:17.696 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.696 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.696 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.696 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:21:17.696 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.696 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.697 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.697 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:21:17.697 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:21:17.697 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.697 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.697 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.697 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:21:17.697 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:21:17.697 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.697 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.698 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.698 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:21:17.698 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.698 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.698 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.698 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:21:17.698 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:21:17.698 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.698 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.698 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.698 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:21:17.698 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:21:17.699 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.699 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.699 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.699 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:21:17.699 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.699 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.699 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.699 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:21:17.699 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:21:17.699 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.699 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.699 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:17.700 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:17.701 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:17.701 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.701 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.701 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:21:17.701 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:21:17.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.667+0000 7f1ea2fbb640 1 -- 172.21.15.115:0/891246963 >> v1:172.21.15.132:6812/1835401922 conn(0x7f1e88037870 legacy=0x7f1e88039d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.667+0000 7f1ea2fbb640 10 monclient: shutdown 2024-01-24T23:21:17.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.667+0000 7f1ea2fbb640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:17.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.667+0000 7f1ea2fbb640 1 -- 172.21.15.115:0/891246963 >> v1:172.21.15.115:6793/0 conn(0x7f1e9c134430 legacy=0x7f1e9c1d3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.667+0000 7f1ea1558640 1 -- 172.21.15.115:0/891246963 reap_dead start 2024-01-24T23:21:17.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.667+0000 7f1ea1558640 1 -- 172.21.15.115:0/891246963 reap_dead start 2024-01-24T23:21:17.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.667+0000 7f1ea2fbb640 1 -- 172.21.15.115:0/891246963 shutdown_connections 2024-01-24T23:21:17.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.667+0000 7f1ea2fbb640 1 -- 172.21.15.115:0/891246963 >> 172.21.15.115:0/891246963 conn(0x7f1e9c0a9910 msgr2=0x7f1e9c13fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:17.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.667+0000 7f1ea2fbb640 1 -- 172.21.15.115:0/891246963 shutdown_connections 2024-01-24T23:21:17.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.667+0000 7f1ea2fbb640 1 -- 172.21.15.115:0/891246963 wait complete. 2024-01-24T23:21:17.703 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:21:17.703 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-01-24T23:21:17.703 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-01-24T23:21:17.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 10 monclient: get_monmap_and_config 2024-01-24T23:21:17.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 10 monclient: build_initial_monmap 2024-01-24T23:21:17.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 10 monclient: monmap: 2024-01-24T23:21:17.814 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:17.814 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.814 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:17.814983+0000 2024-01-24T23:21:17.814 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:17.814983+0000 2024-01-24T23:21:17.814 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:17.815 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.815 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:17.815 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:17.815 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:17.815 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:17.815 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:17.815 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:17.815 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:17.815 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:17.815 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:17.815 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 1 Processor -- start 2024-01-24T23:21:17.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 1 -- start start 2024-01-24T23:21:17.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 10 monclient: init 2024-01-24T23:21:17.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:17.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 10 monclient: _add_conns ranks=[6,7,0,4,3,5,8,1,2] 2024-01-24T23:21:17.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 10 monclient(hunting): picked mon.noname-d con 0x7f9698133bf0 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:17.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 10 monclient(hunting): picked mon.noname-i con 0x7f9698133fd0 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:17.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 10 monclient(hunting): picked mon.noname-a con 0x7f9698134430 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:17.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969804ed70 con 0x7f9698134430 2024-01-24T23:21:17.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969809af30 con 0x7f9698133bf0 2024-01-24T23:21:17.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969809b210 con 0x7f9698133fd0 2024-01-24T23:21:17.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:17.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 10 monclient(hunting): authenticate will time out at 956.737785s 2024-01-24T23:21:17.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969cda0640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f9698133bf0 0x7f969809bd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:39712/0 (socket says 172.21.15.115:39712) 2024-01-24T23:21:17.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969d5a1640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f9698134430 0x7f969813f500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58222/0 (socket says 172.21.15.115:58222) 2024-01-24T23:21:17.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969cda0640 1 -- 172.21.15.115:0/2772897282 learned_addr learned my addr 172.21.15.115:0/2772897282 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:17.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 1 -- 172.21.15.115:0/2772897282 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2395917679 0 0) 0x7f969804ed70 con 0x7f9698134430 2024-01-24T23:21:17.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 10 monclient(hunting): my global_id is 4518 2024-01-24T23:21:17.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 1 -- 172.21.15.115:0/2772897282 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9680003530 con 0x7f9698134430 2024-01-24T23:21:17.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 1 -- 172.21.15.115:0/2772897282 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4016915908 0 0) 0x7f969809af30 con 0x7f9698133bf0 2024-01-24T23:21:17.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 10 monclient(hunting): my global_id is 4497 2024-01-24T23:21:17.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 1 -- 172.21.15.115:0/2772897282 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f969804ed70 con 0x7f9698133bf0 2024-01-24T23:21:17.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 1 -- 172.21.15.115:0/2772897282 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1705044469 0 0) 0x7f9680003530 con 0x7f9698134430 2024-01-24T23:21:17.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 1 -- 172.21.15.115:0/2772897282 >> v1:172.21.15.132:6792/0 conn(0x7f9698133fd0 legacy=0x7f969813bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 1 -- 172.21.15.115:0/2772897282 >> v1:172.21.15.115:6792/0 conn(0x7f9698133bf0 legacy=0x7f969809bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 10 monclient: _finish_hunting 0 2024-01-24T23:21:17.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 1 monclient: found mon.noname-a 2024-01-24T23:21:17.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:17.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:21:17.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 1 -- 172.21.15.115:0/2772897282 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969809b4f0 con 0x7f9698134430 2024-01-24T23:21:17.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 10 monclient: _finish_auth 0 2024-01-24T23:21:17.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 10 monclient: _check_auth_tickets 2024-01-24T23:21:17.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:17.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 1 -- 172.21.15.115:0/2772897282 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9690001b40 con 0x7f9698134430 2024-01-24T23:21:17.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 5 monclient: authenticate success, global_id 4518 2024-01-24T23:21:17.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:21:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 10 monclient: dump: 2024-01-24T23:21:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.827 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.827 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.827 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:17.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 1 -- 172.21.15.115:0/2772897282 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9690001da0 con 0x7f9698134430 2024-01-24T23:21:17.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:17.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 1 -- 172.21.15.115:0/2772897282 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9690004eb0 con 0x7f9698134430 2024-01-24T23:21:17.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:17.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 4 set_mon_vals no callback set 2024-01-24T23:21:17.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:17.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f96977fe640 10 monclient: dump: 2024-01-24T23:21:17.827 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.827 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.828 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 10 monclient: shutdown 2024-01-24T23:21:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 1 -- 172.21.15.115:0/2772897282 >> v1:172.21.15.115:6789/0 conn(0x7f9698134430 legacy=0x7f969813f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 1 -- 172.21.15.115:0/2772897282 shutdown_connections 2024-01-24T23:21:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 1 -- 172.21.15.115:0/2772897282 >> 172.21.15.115:0/2772897282 conn(0x7f96980a9910 msgr2=0x7f96980a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 1 -- 172.21.15.115:0/2772897282 shutdown_connections 2024-01-24T23:21:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 1 -- 172.21.15.115:0/2772897282 wait complete. 2024-01-24T23:21:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 10 monclient: build_initial_monmap 2024-01-24T23:21:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.811+0000 7f969f004640 10 monclient: monmap: 2024-01-24T23:21:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:17.830 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:17.830 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:17.830 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:17.830 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:17.830 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:17.830 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:17.830 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:17.830 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:17.830 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:17.830 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f969f004640 1 Processor -- start 2024-01-24T23:21:17.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f969f004640 1 -- start start 2024-01-24T23:21:17.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f969f004640 10 monclient: init 2024-01-24T23:21:17.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f969f004640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:17.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f969f004640 10 monclient: _add_conns ranks=[3,7,2,6,5,1,4,8,0] 2024-01-24T23:21:17.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f969f004640 10 monclient(hunting): picked mon.noname-g con 0x7f9698133bf0 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:17.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f969f004640 10 monclient(hunting): picked mon.noname-i con 0x7f9698133fd0 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:17.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f969f004640 10 monclient(hunting): picked mon.noname-b con 0x7f9698134430 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:17.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f969f004640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96980a5960 con 0x7f9698134430 2024-01-24T23:21:17.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f969f004640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969813a5d0 con 0x7f9698133bf0 2024-01-24T23:21:17.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f969f004640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969809fd90 con 0x7f9698133fd0 2024-01-24T23:21:17.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f969f004640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:17.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f969f004640 10 monclient(hunting): authenticate will time out at 956.740699s 2024-01-24T23:21:17.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f969d5a1640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f9698134430 0x7f96981d3a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:55566/0 (socket says 172.21.15.115:55566) 2024-01-24T23:21:17.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f969d5a1640 1 -- 172.21.15.115:0/3534135270 learned_addr learned my addr 172.21.15.115:0/3534135270 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:17.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3857541134 0 0) 0x7f96980a5960 con 0x7f9698134430 2024-01-24T23:21:17.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient(hunting): my global_id is 4475 2024-01-24T23:21:17.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9684003530 con 0x7f9698134430 2024-01-24T23:21:17.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2353382273 0 0) 0x7f9684003530 con 0x7f9698134430 2024-01-24T23:21:17.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f96980a5960 con 0x7f9698134430 2024-01-24T23:21:17.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9690001b00 con 0x7f9698134430 2024-01-24T23:21:17.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:21:17.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient(hunting): dump: 2024-01-24T23:21:17.832 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.832 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.832 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.832 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.832 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.833 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 32560774 0 0) 0x7f969809fd90 con 0x7f9698133fd0 2024-01-24T23:21:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient(hunting): my global_id is 4534 2024-01-24T23:21:17.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9684003530 con 0x7f9698133fd0 2024-01-24T23:21:17.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3433831397 0 0) 0x7f969813a5d0 con 0x7f9698133bf0 2024-01-24T23:21:17.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient(hunting): my global_id is 4467 2024-01-24T23:21:17.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f969809fd90 con 0x7f9698133bf0 2024-01-24T23:21:17.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1288571429 0 0) 0x7f96980a5960 con 0x7f9698134430 2024-01-24T23:21:17.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 >> v1:172.21.15.132:6792/0 conn(0x7f9698133fd0 legacy=0x7f96981d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 >> v1:172.21.15.132:6790/0 conn(0x7f9698133bf0 legacy=0x7f969813f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient: _finish_hunting 0 2024-01-24T23:21:17.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 1 monclient: found mon.b 2024-01-24T23:21:17.834 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969813ab90 con 0x7f9698134430 2024-01-24T23:21:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient: _finish_auth 0 2024-01-24T23:21:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient: _check_auth_tickets 2024-01-24T23:21:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f969f004640 5 monclient: authenticate success, global_id 4475 2024-01-24T23:21:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f969f004640 10 monclient: _renew_subs 2024-01-24T23:21:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f969f004640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f969f004640 1 -- 172.21.15.115:0/3534135270 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969813a2f0 con 0x7f9698134430 2024-01-24T23:21:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f969f004640 10 monclient: _renew_subs 2024-01-24T23:21:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f969f004640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:21:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f969f004640 1 -- 172.21.15.115:0/3534135270 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f969809f3e0 con 0x7f9698134430 2024-01-24T23:21:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9690004fe0 con 0x7f9698134430 2024-01-24T23:21:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9690005510 con 0x7f9698134430 2024-01-24T23:21:17.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f9696ffd640 4 set_mon_vals no callback set 2024-01-24T23:21:17.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:21:17.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient: dump: 2024-01-24T23:21:17.836 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.836 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.836 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.836 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.836 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.836 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.836 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.836 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.836 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.836 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.837 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.837 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.837 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.837 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.837 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.837 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f969f004640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:17.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f969f004640 1 -- 172.21.15.115:0/3534135270 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f96780065d0 con 0x7f9678002870 2024-01-24T23:21:17.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f9690005670 con 0x7f9698134430 2024-01-24T23:21:17.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f9690043b20 con 0x7f9698134430 2024-01-24T23:21:17.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f9697fff640 10 monclient: get_auth_request con 0x7f9684037200 auth_method 0 2024-01-24T23:21:17.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1375248910 0 0) 0x7f96780065d0 con 0x7f9678002870 2024-01-24T23:21:17.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient(hunting): my global_id is 4451 2024-01-24T23:21:17.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96980a5960 con 0x7f9678002870 2024-01-24T23:21:17.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4237908962 0 0) 0x7f96980a5960 con 0x7f9678002870 2024-01-24T23:21:17.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f96780065d0 con 0x7f9678002870 2024-01-24T23:21:17.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9688001970 con 0x7f9678002870 2024-01-24T23:21:17.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:21:17.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.815+0000 7f96957fa640 10 monclient: dump: 2024-01-24T23:21:17.838 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.838 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.838 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.838 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.838 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.838 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.838 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.838 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.839 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.839 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.839 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.839 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.839 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.839 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.839 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.839 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.819+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3459899408 0 0) 0x7f96780065d0 con 0x7f9678002870 2024-01-24T23:21:17.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.819+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 --> v1:172.21.15.132:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f967800a120 con 0x7f9678002870 2024-01-24T23:21:17.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.819+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f96880047f0 con 0x7f9678002870 2024-01-24T23:21:17.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.819+0000 7f96957fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:17.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.819+0000 7f96957fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:17.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.819+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 >> v1:172.21.15.132:6791/0 conn(0x7f9678002870 legacy=0x7f9678004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.819+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 >> v1:172.21.15.132:6791/0 conn(0x7f9678002870 legacy=0x7f9678004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:17.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.823+0000 7f969f004640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:21:17.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.823+0000 7f969f004640 1 -- 172.21.15.115:0/3534135270 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9678009df0 con 0x7f96780090b0 2024-01-24T23:21:17.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.823+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1252847605 0 0) 0x7f9678009df0 con 0x7f96780090b0 2024-01-24T23:21:17.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.823+0000 7f96957fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.823+0000 7f96957fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.823+0000 7f96957fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.823+0000 7f96957fa640 10 monclient(hunting): my global_id is 4460 2024-01-24T23:21:17.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.823+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96780065d0 con 0x7f96780090b0 2024-01-24T23:21:17.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.827+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3703711582 0 0) 0x7f96780065d0 con 0x7f96780090b0 2024-01-24T23:21:17.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.827+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9678009df0 con 0x7f96780090b0 2024-01-24T23:21:17.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.827+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9688004e90 con 0x7f96780090b0 2024-01-24T23:21:17.841 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.827+0000 7f96957fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.827+0000 7f96957fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:21:17.842 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.827+0000 7f96957fa640 10 monclient: dump: 2024-01-24T23:21:17.842 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.842 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.842 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.842 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.842 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.842 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.842 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.842 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.842 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.842 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.842 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.842 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.843 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.843 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.843 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.843 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.827+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 117081814 0 0) 0x7f9678009df0 con 0x7f96780090b0 2024-01-24T23:21:17.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.827+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 --> v1:172.21.15.132:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f967800c310 con 0x7f96780090b0 2024-01-24T23:21:17.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.827+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 2957424922) 0x7f9688007170 con 0x7f96780090b0 2024-01-24T23:21:17.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.827+0000 7f96957fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:21:17.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.827+0000 7f96957fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:17.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.827+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 >> v1:172.21.15.132:6791/0 conn(0x7f96780090b0 legacy=0x7f9678009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.843 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.827+0000 7f96957fa640 1 -- 172.21.15.115:0/3534135270 >> v1:172.21.15.132:6791/0 conn(0x7f96780090b0 legacy=0x7f9678009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:17.843 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:21:17.844 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:21:17.844 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:21:17.844 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:21:17.844 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 6, 2024-01-24T23:21:17.844 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:21:17.844 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:21:17.844 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:21:17.844 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:21:17.844 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:21:17.844 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:21:17.844 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:21:17.844 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:21:17.845 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:21:17.845 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:21:17.845 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.845 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 19, 2024-01-24T23:21:17.845 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:17.845 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:21:17.845 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:21:17.845 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:17.845 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:17.845 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:17.845 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:17.845 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:17.845 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:17.846 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:17.846 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:17.846 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:17.846 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:17.846 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:17.846 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.846 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:21:17.846 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:21:17.846 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:17.846 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:17.846 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:17.846 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:17.846 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:17.846 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:17.847 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:17.847 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:17.847 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:17.847 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:17.847 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:17.847 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.847 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:21:17.847 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:21:17.847 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:21:17.847 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:21:17.847 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:21:17.847 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:21:17.847 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:17.848 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:17.848 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:21:17.848 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:21:17.848 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:21:17.848 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:21:17.848 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:21:17.848 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:21:17.848 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:21:17.848 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:17.848 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:21:17.848 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:17.848 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:17.848 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:17.848 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:17.849 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:17.849 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:17.849 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:17.849 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:17.849 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:17.849 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:17.849 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:17.849 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.849 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:21:17.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.849 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:21:17.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.849 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:21:17.850 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:21:17.850 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.850 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.850 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.850 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:21:17.850 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.850 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.850 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.850 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:21:17.850 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:21:17.850 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.850 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.851 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.851 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:21:17.851 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:21:17.851 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.851 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.851 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.851 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:21:17.851 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.851 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.851 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.851 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:21:17.852 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:21:17.852 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.852 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.852 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.852 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:21:17.852 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:21:17.852 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.852 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.852 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.852 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:21:17.853 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.853 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.853 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.853 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:21:17.853 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:21:17.853 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.853 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.853 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.853 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:21:17.853 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:21:17.853 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.854 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.854 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.854 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:21:17.854 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.854 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.854 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.854 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:21:17.854 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:21:17.854 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.854 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.854 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.855 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:21:17.855 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:21:17.855 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.855 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.855 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.855 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:21:17.855 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.855 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.855 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.855 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:21:17.855 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:21:17.855 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.856 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.856 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.856 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:21:17.856 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:21:17.856 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.856 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.856 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.856 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:21:17.856 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.856 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.857 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.857 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:21:17.857 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:21:17.857 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.857 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.857 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.857 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:21:17.857 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:21:17.859 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.859 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.859 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.859 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:21:17.859 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.859 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.859 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.859 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:21:17.859 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:21:17.859 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.859 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.859 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.860 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:21:17.860 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:21:17.860 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.860 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.860 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.860 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:21:17.860 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.860 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.860 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.860 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:21:17.861 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:21:17.861 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.861 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.861 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.861 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:21:17.861 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:21:17.861 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:17.861 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:17.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.861 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:17.861 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:21:17.861 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:17.862 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.862 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.862 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:21:17.862 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:21:17.862 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:17.862 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:17.862 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:17.862 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.862 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.862 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:21:17.862 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:21:17.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.863 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:17.863 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:17.863 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:17.863 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.863 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:17.863 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:21:17.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:17.863 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:17.863 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:17.863 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:17.863 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:17.863 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:17.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:17.863 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:21:17.863 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:21:17.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.827+0000 7f969f004640 1 -- 172.21.15.115:0/3534135270 >> v1:172.21.15.132:6812/1835401922 conn(0x7f9684037200 legacy=0x7f96840396c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.827+0000 7f969f004640 10 monclient: shutdown 2024-01-24T23:21:17.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.827+0000 7f969f004640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:17.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.827+0000 7f969f004640 1 -- 172.21.15.115:0/3534135270 >> v1:172.21.15.115:6790/0 conn(0x7f9698134430 legacy=0x7f96981d3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.827+0000 7f969d5a1640 1 -- 172.21.15.115:0/3534135270 reap_dead start 2024-01-24T23:21:17.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.827+0000 7f969d5a1640 1 -- 172.21.15.115:0/3534135270 reap_dead start 2024-01-24T23:21:17.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.827+0000 7f969f004640 1 -- 172.21.15.115:0/3534135270 shutdown_connections 2024-01-24T23:21:17.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.827+0000 7f969f004640 1 -- 172.21.15.115:0/3534135270 >> 172.21.15.115:0/3534135270 conn(0x7f96980a9910 msgr2=0x7f96980a7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:17.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.827+0000 7f969f004640 1 -- 172.21.15.115:0/3534135270 shutdown_connections 2024-01-24T23:21:17.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.827+0000 7f969f004640 1 -- 172.21.15.115:0/3534135270 wait complete. 2024-01-24T23:21:17.866 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:21:17.866 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-01-24T23:21:17.866 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-01-24T23:21:17.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.971+0000 7fb31c93e640 10 monclient: get_monmap_and_config 2024-01-24T23:21:17.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.971+0000 7fb31c93e640 10 monclient: build_initial_monmap 2024-01-24T23:21:17.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.971+0000 7fb31c93e640 10 monclient: monmap: 2024-01-24T23:21:17.977 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:17.977 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.977 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:17.978681+0000 2024-01-24T23:21:17.977 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:17.978681+0000 2024-01-24T23:21:17.977 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:17.977 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.978 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:17.978 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:17.978 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:17.978 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:17.978 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:17.978 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:17.978 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:17.978 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:17.978 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:17.978 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 1 Processor -- start 2024-01-24T23:21:17.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 1 -- start start 2024-01-24T23:21:17.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 10 monclient: init 2024-01-24T23:21:17.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:17.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 10 monclient: _add_conns ranks=[7,4,1,3,8,5,6,2,0] 2024-01-24T23:21:17.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 10 monclient(hunting): picked mon.noname-i con 0x7fb3180a2c40 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:17.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 10 monclient(hunting): picked mon.noname-c con 0x7fb3180a3fd0 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:17.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 10 monclient(hunting): picked mon.noname-f con 0x7fb3180a06b0 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:17.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb31804ed70 con 0x7fb3180a06b0 2024-01-24T23:21:17.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3180a0e70 con 0x7fb3180a3fd0 2024-01-24T23:21:17.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3180a3730 con 0x7fb3180a2c40 2024-01-24T23:21:17.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:17.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 10 monclient(hunting): authenticate will time out at 956.901166s 2024-01-24T23:21:17.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb315d9b640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fb3180a3fd0 0x7fb31813be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:60046/0 (socket says 172.21.15.115:60046) 2024-01-24T23:21:17.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb315d9b640 1 -- 172.21.15.115:0/2386046336 learned_addr learned my addr 172.21.15.115:0/2386046336 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:17.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 1 -- 172.21.15.115:0/2386046336 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319350644 0 0) 0x7fb3180a0e70 con 0x7fb3180a3fd0 2024-01-24T23:21:17.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 10 monclient(hunting): my global_id is 74551 2024-01-24T23:21:17.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 1 -- 172.21.15.115:0/2386046336 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb300003530 con 0x7fb3180a3fd0 2024-01-24T23:21:17.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 1 -- 172.21.15.115:0/2386046336 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2132835176 0 0) 0x7fb300003530 con 0x7fb3180a3fd0 2024-01-24T23:21:17.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 1 -- 172.21.15.115:0/2386046336 >> v1:172.21.15.132:6792/0 conn(0x7fb3180a2c40 legacy=0x7fb3180a3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 1 -- 172.21.15.115:0/2386046336 >> v1:172.21.15.132:6789/0 conn(0x7fb3180a06b0 legacy=0x7fb31813f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 10 monclient: _finish_hunting 0 2024-01-24T23:21:17.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 1 monclient: found mon.noname-c 2024-01-24T23:21:17.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:17.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:21:17.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 1 -- 172.21.15.115:0/2386046336 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3180a1150 con 0x7fb3180a3fd0 2024-01-24T23:21:17.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 10 monclient: _finish_auth 0 2024-01-24T23:21:17.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 10 monclient: _check_auth_tickets 2024-01-24T23:21:17.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:17.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 5 monclient: authenticate success, global_id 74551 2024-01-24T23:21:17.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:17.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 1 -- 172.21.15.115:0/2386046336 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb3080017c0 con 0x7fb3180a3fd0 2024-01-24T23:21:17.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:21:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 10 monclient: dump: 2024-01-24T23:21:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.989 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.989 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.989 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:17.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 1 -- 172.21.15.115:0/2386046336 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb308001a70 con 0x7fb3180a3fd0 2024-01-24T23:21:17.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:17.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:17.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 1 -- 172.21.15.115:0/2386046336 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb308004bf0 con 0x7fb3180a3fd0 2024-01-24T23:21:17.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 4 set_mon_vals no callback set 2024-01-24T23:21:17.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:17.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31559a640 10 monclient: dump: 2024-01-24T23:21:17.989 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.989 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.989 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.990 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.990 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.990 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.990 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.990 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.990 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.990 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.990 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.990 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.990 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.990 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.990 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.990 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 10 monclient: shutdown 2024-01-24T23:21:17.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:17.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 1 -- 172.21.15.115:0/2386046336 >> v1:172.21.15.115:6791/0 conn(0x7fb3180a3fd0 legacy=0x7fb31813be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 1 -- 172.21.15.115:0/2386046336 shutdown_connections 2024-01-24T23:21:17.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 1 -- 172.21.15.115:0/2386046336 >> 172.21.15.115:0/2386046336 conn(0x7fb3180a9ba0 msgr2=0x7fb3180a5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:17.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 1 -- 172.21.15.115:0/2386046336 shutdown_connections 2024-01-24T23:21:17.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 1 -- 172.21.15.115:0/2386046336 wait complete. 2024-01-24T23:21:17.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 10 monclient: build_initial_monmap 2024-01-24T23:21:17.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 10 monclient: monmap: 2024-01-24T23:21:17.991 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:17.991 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:17.991 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:17.991 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:17.991 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:17.991 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:17.991 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:17.992 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:17.992 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:17.992 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:17.992 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:17.992 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:17.992 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:17.992 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:17.992 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:17.992 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 1 Processor -- start 2024-01-24T23:21:17.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 1 -- start start 2024-01-24T23:21:17.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 10 monclient: init 2024-01-24T23:21:17.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:17.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 10 monclient: _add_conns ranks=[3,1,6,8,0,2,4,5,7] 2024-01-24T23:21:17.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 10 monclient(hunting): picked mon.noname-g con 0x7fb3180a06b0 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:17.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 10 monclient(hunting): picked mon.noname-f con 0x7fb3180a2c40 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:17.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 10 monclient(hunting): picked mon.noname-d con 0x7fb3180a3fd0 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:17.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb31813fff0 con 0x7fb3180a2c40 2024-01-24T23:21:17.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb31813e690 con 0x7fb3180a06b0 2024-01-24T23:21:17.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3180a58a0 con 0x7fb3180a3fd0 2024-01-24T23:21:17.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:17.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.975+0000 7fb31c93e640 10 monclient(hunting): authenticate will time out at 956.903984s 2024-01-24T23:21:17.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb316d9d640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fb3180a3fd0 0x7fb3181d3b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:39718/0 (socket says 172.21.15.115:39718) 2024-01-24T23:21:17.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb316d9d640 1 -- 172.21.15.115:0/664446840 learned_addr learned my addr 172.21.15.115:0/664446840 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:17.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1373173133 0 0) 0x7fb3180a58a0 con 0x7fb3180a3fd0 2024-01-24T23:21:17.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient(hunting): my global_id is 4506 2024-01-24T23:21:17.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2f8003530 con 0x7fb3180a3fd0 2024-01-24T23:21:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1215790481 0 0) 0x7fb2f8003530 con 0x7fb3180a3fd0 2024-01-24T23:21:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb3180a58a0 con 0x7fb3180a3fd0 2024-01-24T23:21:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb310001d20 con 0x7fb3180a3fd0 2024-01-24T23:21:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:21:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient(hunting): dump: 2024-01-24T23:21:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.995 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1669290525 0 0) 0x7fb3180a58a0 con 0x7fb3180a3fd0 2024-01-24T23:21:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 >> v1:172.21.15.132:6790/0 conn(0x7fb3180a06b0 legacy=0x7fb31813c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 >> v1:172.21.15.132:6789/0 conn(0x7fb3180a2c40 legacy=0x7fb31813c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:21:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 1 monclient: found mon.d 2024-01-24T23:21:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:17.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:17.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31813fa30 con 0x7fb3180a3fd0 2024-01-24T23:21:17.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient: _finish_auth 0 2024-01-24T23:21:17.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:21:17.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:17.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb31c93e640 5 monclient: authenticate success, global_id 4506 2024-01-24T23:21:17.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb31c93e640 10 monclient: _renew_subs 2024-01-24T23:21:17.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb31c93e640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:17.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb31c93e640 1 -- 172.21.15.115:0/664446840 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb318137720 con 0x7fb3180a3fd0 2024-01-24T23:21:17.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb31c93e640 10 monclient: _renew_subs 2024-01-24T23:21:17.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb31c93e640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:17.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb31c93e640 1 -- 172.21.15.115:0/664446840 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31813cea0 con 0x7fb3180a3fd0 2024-01-24T23:21:17.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb3100020f0 con 0x7fb3180a3fd0 2024-01-24T23:21:17.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb310003f90 con 0x7fb3180a3fd0 2024-01-24T23:21:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb314d99640 4 set_mon_vals no callback set 2024-01-24T23:21:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient: dump: 2024-01-24T23:21:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:17.998 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:17.998 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:17.998 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:17.998 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:17.998 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:17.998 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:17.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb31c93e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:17.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb31c93e640 1 -- 172.21.15.115:0/664446840 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb2fc0065d0 con 0x7fb2fc002870 2024-01-24T23:21:17.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fb310004170 con 0x7fb3180a3fd0 2024-01-24T23:21:17.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7fb310043a50 con 0x7fb3180a3fd0 2024-01-24T23:21:17.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 832855021 0 0) 0x7fb2fc0065d0 con 0x7fb2fc002870 2024-01-24T23:21:17.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:17.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:17.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:17.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient(hunting): my global_id is 4552 2024-01-24T23:21:17.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3180a58a0 con 0x7fb2fc002870 2024-01-24T23:21:17.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb315d9b640 10 monclient: get_auth_request con 0x7fb2f8036fd0 auth_method 0 2024-01-24T23:21:17.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 835900746 0 0) 0x7fb3180a58a0 con 0x7fb2fc002870 2024-01-24T23:21:17.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb2fc0065d0 con 0x7fb2fc002870 2024-01-24T23:21:17.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb30c0017f0 con 0x7fb2fc002870 2024-01-24T23:21:17.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:17.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:21:17.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient: dump: 2024-01-24T23:21:17.999 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:17.999 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:17.999 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.999 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:17.999 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:17.999 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:17.999 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:18.000 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3833795134 0 0) 0x7fb2fc0065d0 con 0x7fb2fc002870 2024-01-24T23:21:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 --> v1:172.21.15.132:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2fc00a120 con 0x7fb2fc002870 2024-01-24T23:21:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 <== mon.7 v1:172.21.15.132:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fb30c004650 con 0x7fb2fc002870 2024-01-24T23:21:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 >> v1:172.21.15.132:6792/0 conn(0x7fb2fc002870 legacy=0x7fb2fc004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:18.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.979+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 >> v1:172.21.15.132:6792/0 conn(0x7fb2fc002870 legacy=0x7fb2fc004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:18.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.987+0000 7fb31c93e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:21:18.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.987+0000 7fb31c93e640 1 -- 172.21.15.115:0/664446840 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb2fc009e10 con 0x7fb2fc0090d0 2024-01-24T23:21:18.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.987+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336664613 0 0) 0x7fb2fc009e10 con 0x7fb2fc0090d0 2024-01-24T23:21:18.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.987+0000 7fb306ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:18.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.987+0000 7fb306ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:18.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.987+0000 7fb306ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:18.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.987+0000 7fb306ffd640 10 monclient(hunting): my global_id is 4561 2024-01-24T23:21:18.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.987+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2fc0065d0 con 0x7fb2fc0090d0 2024-01-24T23:21:18.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.987+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2803543348 0 0) 0x7fb2fc0065d0 con 0x7fb2fc0090d0 2024-01-24T23:21:18.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.987+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb2fc009e10 con 0x7fb2fc0090d0 2024-01-24T23:21:18.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.987+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb30c003c50 con 0x7fb2fc0090d0 2024-01-24T23:21:18.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.987+0000 7fb306ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:18.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.987+0000 7fb306ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:21:18.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.987+0000 7fb306ffd640 10 monclient: dump: 2024-01-24T23:21:18.003 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:18.003 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:18.003 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:18.003 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:18.003 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:18.003 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:18.003 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:18.003 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:18.003 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:18.003 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:18.003 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:18.004 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:18.004 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:18.004 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:18.004 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:18.004 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:18.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.987+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2461635418 0 0) 0x7fb2fc009e10 con 0x7fb2fc0090d0 2024-01-24T23:21:18.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.987+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 --> v1:172.21.15.132:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb2fc00c310 con 0x7fb2fc0090d0 2024-01-24T23:21:18.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.987+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 <== mon.7 v1:172.21.15.132:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 4004083267) 0x7fb30c003fe0 con 0x7fb2fc0090d0 2024-01-24T23:21:18.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.987+0000 7fb306ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:21:18.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.987+0000 7fb306ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:18.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.987+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 >> v1:172.21.15.132:6792/0 conn(0x7fb2fc0090d0 legacy=0x7fb2fc009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:18.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.987+0000 7fb306ffd640 1 -- 172.21.15.115:0/664446840 >> v1:172.21.15.132:6792/0 conn(0x7fb2fc0090d0 legacy=0x7fb2fc009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:18.005 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:21:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:21:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:21:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:21:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 6, 2024-01-24T23:21:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:21:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:21:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:21:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:21:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:21:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:21:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:21:18.005 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:21:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:21:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:21:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 20, 2024-01-24T23:21:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:21:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:21:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:18.006 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:21:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:21:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:18.007 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:21:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:21:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:21:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:21:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:21:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:21:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:21:18.008 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:21:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:21:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:21:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:21:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:21:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:21:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:21:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:18.009 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:21:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:21:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:21:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:21:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:18.010 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:21:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:21:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:21:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:21:18.011 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:21:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:21:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:21:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:21:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:18.012 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:21:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:21:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:21:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:18.013 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:21:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:21:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:21:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:21:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:21:18.014 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:21:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:21:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:21:18.015 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:21:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:21:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:21:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:21:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:18.016 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:21:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:21:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:21:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:18.017 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:21:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:21:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:21:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:21:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:21:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:18.018 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:21:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:21:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:21:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:21:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:18.019 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:21:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:21:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:21:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:18.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:21:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:21:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:21:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:21:18.021 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:21:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:21:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:21:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:18.022 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:21:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:21:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:21:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 3 2024-01-24T23:21:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:18.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:18.024 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:21:18.024 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:21:18.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.991+0000 7fb31c93e640 1 -- 172.21.15.115:0/664446840 >> v1:172.21.15.132:6812/1835401922 conn(0x7fb2f8036fd0 legacy=0x7fb2f8039490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:18.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.991+0000 7fb31c93e640 10 monclient: shutdown 2024-01-24T23:21:18.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.991+0000 7fb31c93e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:18.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.991+0000 7fb31c93e640 1 -- 172.21.15.115:0/664446840 >> v1:172.21.15.115:6792/0 conn(0x7fb3180a3fd0 legacy=0x7fb3181d3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:18.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.991+0000 7fb316d9d640 1 -- 172.21.15.115:0/664446840 reap_dead start 2024-01-24T23:21:18.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.991+0000 7fb316d9d640 1 -- 172.21.15.115:0/664446840 reap_dead start 2024-01-24T23:21:18.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.991+0000 7fb31c93e640 1 -- 172.21.15.115:0/664446840 shutdown_connections 2024-01-24T23:21:18.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.991+0000 7fb31c93e640 1 -- 172.21.15.115:0/664446840 >> 172.21.15.115:0/664446840 conn(0x7fb3180a9ba0 msgr2=0x7fb3180a7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:18.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.991+0000 7fb31c93e640 1 -- 172.21.15.115:0/664446840 shutdown_connections 2024-01-24T23:21:18.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:17.991+0000 7fb31c93e640 1 -- 172.21.15.115:0/664446840 wait complete. 2024-01-24T23:21:18.026 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:21:18.026 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b', 'e'] 2024-01-24T23:21:18.026 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-01-24T23:21:18.026 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-01-24T23:21:18.026 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-01-24T23:21:18.027 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-01-24T23:21:18.027 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:21:18.037 INFO:tasks.ceph.mon.b:Stopped 2024-01-24T23:21:18.037 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.e 2024-01-24T23:21:18.037 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-01-24T23:21:18.037 DEBUG:tasks.ceph.mon.e:waiting for process to exit 2024-01-24T23:21:18.037 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:21:18.091 INFO:tasks.ceph.mon.e:Stopped 2024-01-24T23:21:18.091 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 7 2024-01-24T23:21:18.091 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-01-24T23:21:18.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.247+0000 7f6a371bf640 10 monclient: get_monmap_and_config 2024-01-24T23:21:18.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.247+0000 7f6a371bf640 10 monclient: build_initial_monmap 2024-01-24T23:21:18.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.247+0000 7f6a371bf640 10 monclient: monmap: 2024-01-24T23:21:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:18.252101+0000 2024-01-24T23:21:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:18.252101+0000 2024-01-24T23:21:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:18.252 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:18.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.247+0000 7f6a371bf640 1 Processor -- start 2024-01-24T23:21:18.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.247+0000 7f6a371bf640 1 -- start start 2024-01-24T23:21:18.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.247+0000 7f6a371bf640 10 monclient: init 2024-01-24T23:21:18.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.247+0000 7f6a371bf640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:18.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.247+0000 7f6a371bf640 10 monclient: _add_conns ranks=[3,0,7,6,8,5,1,4,2] 2024-01-24T23:21:18.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.247+0000 7f6a371bf640 10 monclient(hunting): picked mon.noname-g con 0x7f6a30133d90 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:18.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.247+0000 7f6a371bf640 10 monclient(hunting): picked mon.noname-a con 0x7f6a30134170 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:18.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.247+0000 7f6a371bf640 10 monclient(hunting): picked mon.noname-i con 0x7f6a3013dcf0 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:18.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.247+0000 7f6a371bf640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3004ed70 con 0x7f6a30134170 2024-01-24T23:21:18.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.247+0000 7f6a371bf640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3009b250 con 0x7f6a30133d90 2024-01-24T23:21:18.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.247+0000 7f6a371bf640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a301345d0 con 0x7f6a3013dcf0 2024-01-24T23:21:18.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.247+0000 7f6a371bf640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:18.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.247+0000 7f6a371bf640 10 monclient(hunting): authenticate will time out at 957.175273s 2024-01-24T23:21:18.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.247+0000 7f6a2ffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f6a30134170 0x7f6a300a2100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58232/0 (socket says 172.21.15.115:58232) 2024-01-24T23:21:18.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.247+0000 7f6a2ffff640 1 -- 172.21.15.115:0/1925660046 learned_addr learned my addr 172.21.15.115:0/1925660046 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:18.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.247+0000 7f6a2f7fe640 1 -- 172.21.15.115:0/1925660046 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2285413080 0 0) 0x7f6a3004ed70 con 0x7f6a30134170 2024-01-24T23:21:18.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.247+0000 7f6a2f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:18.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.247+0000 7f6a2f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:18.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.247+0000 7f6a2f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:18.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.247+0000 7f6a2f7fe640 10 monclient(hunting): my global_id is 4527 2024-01-24T23:21:18.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.247+0000 7f6a2f7fe640 1 -- 172.21.15.115:0/1925660046 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a18003530 con 0x7f6a30134170 2024-01-24T23:21:18.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 1 -- 172.21.15.115:0/1925660046 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3740091733 0 0) 0x7f6a3009b250 con 0x7f6a30133d90 2024-01-24T23:21:18.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:18.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:18.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:18.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 10 monclient(hunting): my global_id is 4485 2024-01-24T23:21:18.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 1 -- 172.21.15.115:0/1925660046 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a3004ed70 con 0x7f6a30133d90 2024-01-24T23:21:18.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 1 -- 172.21.15.115:0/1925660046 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2004435991 0 0) 0x7f6a18003530 con 0x7f6a30134170 2024-01-24T23:21:18.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 1 -- 172.21.15.115:0/1925660046 >> v1:172.21.15.132:6792/0 conn(0x7f6a3013dcf0 legacy=0x7f6a301400e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:18.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 1 -- 172.21.15.115:0/1925660046 >> v1:172.21.15.132:6790/0 conn(0x7f6a30133d90 legacy=0x7f6a300a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:18.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:21:18.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 1 monclient: found mon.noname-a 2024-01-24T23:21:18.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:18.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:21:18.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 1 -- 172.21.15.115:0/1925660046 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a300a2810 con 0x7f6a30134170 2024-01-24T23:21:18.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 10 monclient: _finish_auth 0 2024-01-24T23:21:18.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:21:18.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:18.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 5 monclient: authenticate success, global_id 4527 2024-01-24T23:21:18.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 1 -- 172.21.15.115:0/1925660046 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6a24001740 con 0x7f6a30134170 2024-01-24T23:21:18.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:18.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:18.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:21:18.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 10 monclient: dump: 2024-01-24T23:21:18.261 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:18.261 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:18.261 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:18.261 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:18.261 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:18.261 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:18.262 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:18.262 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:18.262 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:18.262 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:18.262 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:18.262 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:18.262 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:18.262 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:18.262 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:18.262 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:18.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:18.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 1 -- 172.21.15.115:0/1925660046 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6a24001960 con 0x7f6a30134170 2024-01-24T23:21:18.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:18.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:18.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 1 -- 172.21.15.115:0/1925660046 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6a240048b0 con 0x7f6a30134170 2024-01-24T23:21:18.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 4 set_mon_vals no callback set 2024-01-24T23:21:18.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:18.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:18.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2f7fe640 10 monclient: dump: 2024-01-24T23:21:18.263 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:18.263 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:18.263 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:18.263 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:18.263 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:18.263 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:18.263 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:18.263 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:18.263 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:18.263 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:18.263 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:18.263 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:18.264 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:18.264 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:18.264 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:18.264 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:18.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 10 monclient: shutdown 2024-01-24T23:21:18.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:18.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 1 -- 172.21.15.115:0/1925660046 >> v1:172.21.15.115:6789/0 conn(0x7f6a30134170 legacy=0x7f6a300a2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:18.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 1 -- 172.21.15.115:0/1925660046 shutdown_connections 2024-01-24T23:21:18.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 1 -- 172.21.15.115:0/1925660046 >> 172.21.15.115:0/1925660046 conn(0x7f6a300a65e0 msgr2=0x7f6a300a2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:18.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 1 -- 172.21.15.115:0/1925660046 shutdown_connections 2024-01-24T23:21:18.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 1 -- 172.21.15.115:0/1925660046 wait complete. 2024-01-24T23:21:18.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 10 monclient: build_initial_monmap 2024-01-24T23:21:18.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 10 monclient: monmap: 2024-01-24T23:21:18.265 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:18.265 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:18.265 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:18.267 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:18.267 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:18.267 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:18.267 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:18.267 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:18.267 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:18.267 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:18.268 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:18.268 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:18.268 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:18.268 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:18.268 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:18.268 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:18.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 1 Processor -- start 2024-01-24T23:21:18.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 1 -- start start 2024-01-24T23:21:18.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 10 monclient: init 2024-01-24T23:21:18.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:18.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 10 monclient: _add_conns ranks=[3,1,4,8,2,7,6,0,5] 2024-01-24T23:21:18.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 10 monclient(hunting): picked mon.noname-g con 0x7f6a30133d90 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:18.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 10 monclient(hunting): picked mon.noname-f con 0x7f6a30134170 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:18.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 10 monclient(hunting): picked mon.noname-c con 0x7f6a3013dcf0 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:18.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3013b7b0 con 0x7f6a30134170 2024-01-24T23:21:18.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3013a300 con 0x7f6a30133d90 2024-01-24T23:21:18.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3009c610 con 0x7f6a3013dcf0 2024-01-24T23:21:18.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:18.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 10 monclient(hunting): authenticate will time out at 957.178450s 2024-01-24T23:21:18.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a3575c640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f6a3013dcf0 0x7f6a301d3d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:60060/0 (socket says 172.21.15.115:60060) 2024-01-24T23:21:18.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a3575c640 1 -- 172.21.15.115:0/443874670 learned_addr learned my addr 172.21.15.115:0/443874670 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:18.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2d7fa640 1 -- 172.21.15.115:0/443874670 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 25383201 0 0) 0x7f6a3009c610 con 0x7f6a3013dcf0 2024-01-24T23:21:18.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2d7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:18.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2d7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:18.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:18.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2d7fa640 10 monclient(hunting): my global_id is 74560 2024-01-24T23:21:18.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2d7fa640 1 -- 172.21.15.115:0/443874670 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a1c003530 con 0x7f6a3013dcf0 2024-01-24T23:21:18.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2d7fa640 1 -- 172.21.15.115:0/443874670 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2372691590 0 0) 0x7f6a1c003530 con 0x7f6a3013dcf0 2024-01-24T23:21:18.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2d7fa640 1 -- 172.21.15.115:0/443874670 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6a3009c610 con 0x7f6a3013dcf0 2024-01-24T23:21:18.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2d7fa640 1 -- 172.21.15.115:0/443874670 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6a28001da0 con 0x7f6a3013dcf0 2024-01-24T23:21:18.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:18.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:21:18.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2d7fa640 10 monclient(hunting): dump: 2024-01-24T23:21:18.270 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:18.270 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:18.270 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:18.270 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:18.270 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:18.270 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:18.270 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:18.270 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:18.270 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:18.270 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:18.270 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:18.271 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:18.271 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:18.271 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:18.271 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:18.271 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:18.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2d7fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:18.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2d7fa640 1 -- 172.21.15.115:0/443874670 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2501175287 0 0) 0x7f6a3009c610 con 0x7f6a3013dcf0 2024-01-24T23:21:18.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2d7fa640 1 -- 172.21.15.115:0/443874670 >> v1:172.21.15.132:6790/0 conn(0x7f6a30133d90 legacy=0x7f6a3013cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:18.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2d7fa640 1 -- 172.21.15.115:0/443874670 >> v1:172.21.15.132:6789/0 conn(0x7f6a30134170 legacy=0x7f6a301d0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:18.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2d7fa640 10 monclient: _finish_hunting 0 2024-01-24T23:21:18.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2d7fa640 1 monclient: found mon.c 2024-01-24T23:21:18.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:18.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2d7fa640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:18.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2d7fa640 1 -- 172.21.15.115:0/443874670 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3013b4d0 con 0x7f6a3013dcf0 2024-01-24T23:21:18.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2d7fa640 10 monclient: _finish_auth 0 2024-01-24T23:21:18.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2d7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:21:18.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a2d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:18.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 5 monclient: authenticate success, global_id 74560 2024-01-24T23:21:18.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 10 monclient: _renew_subs 2024-01-24T23:21:18.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:18.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 1 -- 172.21.15.115:0/443874670 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a30139ff0 con 0x7f6a3013dcf0 2024-01-24T23:21:18.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 10 monclient: _renew_subs 2024-01-24T23:21:18.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:18.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.251+0000 7f6a371bf640 1 -- 172.21.15.115:0/443874670 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a3013d8a0 con 0x7f6a3013dcf0 2024-01-24T23:21:18.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.255+0000 7f6a371bf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:18.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.255+0000 7f6a371bf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:18.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.255+0000 7f6a371bf640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:18.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.255+0000 7f6a371bf640 1 -- 172.21.15.115:0/443874670 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a3009c800 con 0x7f6a3013dcf0 2024-01-24T23:21:18.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.255+0000 7f6a2d7fa640 1 -- 172.21.15.115:0/443874670 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6a28003c50 con 0x7f6a3013dcf0 2024-01-24T23:21:18.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.255+0000 7f6a2d7fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:18.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.255+0000 7f6a2d7fa640 1 -- 172.21.15.115:0/443874670 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6a28002110 con 0x7f6a3013dcf0 2024-01-24T23:21:18.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.255+0000 7f6a2d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:18.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.255+0000 7f6a2d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:18.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.255+0000 7f6a2d7fa640 10 monclient: dump: 2024-01-24T23:21:18.273 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:18.273 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:18.273 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:18.273 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:18.273 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:18.273 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:18.273 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:18.273 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:18.273 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:18.274 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:18.274 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:18.274 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:18.274 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:18.274 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:18.274 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:18.274 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:18.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.255+0000 7f6a2effd640 4 set_mon_vals no callback set 2024-01-24T23:21:18.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.255+0000 7f6a2d7fa640 1 -- 172.21.15.115:0/443874670 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f6a28004180 con 0x7f6a3013dcf0 2024-01-24T23:21:18.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.255+0000 7f6a2d7fa640 1 -- 172.21.15.115:0/443874670 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f6a28043790 con 0x7f6a3013dcf0 2024-01-24T23:21:18.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.255+0000 7f6a34f5b640 10 monclient: get_auth_request con 0x7f6a1c037000 auth_method 0 2024-01-24T23:21:18.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.259+0000 7f6a2d7fa640 1 -- 172.21.15.115:0/443874670 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f6a28014b00 con 0x7f6a3013dcf0 2024-01-24T23:21:18.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.259+0000 7f6a2d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:18.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.259+0000 7f6a2d7fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:18.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.463+0000 7f6a371bf640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-01-24T23:21:18.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.463+0000 7f6a371bf640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-01-24T23:21:18.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.463+0000 7f6a371bf640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:18.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.463+0000 7f6a371bf640 1 -- 172.21.15.115:0/443874670 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f6a3009c7a0 con 0x7f6a3013dcf0 2024-01-24T23:21:18.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.463+0000 7f6a2d7fa640 1 -- 172.21.15.115:0/443874670 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2744 (unknown 254746657 0 755381103) 0x7f6a2801d000 con 0x7f6a3013dcf0 2024-01-24T23:21:18.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.463+0000 7f6a2d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-01-24T23:21:18.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.463+0000 7f6a2d7fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:18.467 INFO:teuthology.orchestra.run.smithi115.stdout:{"election_epoch":6,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"c","quorum_age":20,"features":{"quorum_con":"4540701272860131327","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","modified":"2024-01-24T23:20:20.557246Z","created":"2024-01-24T23:20:20.557246Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6790","nonce":0}]},"addr":"172.21.15.115:6790/0","public_addr":"172.21.15.115:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6790","nonce":0}]},"addr":"172.21.15.132:6790/0","public_addr":"172.21.15.132:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6791","nonce":0}]},"addr":"172.21.15.115:6791/0","public_addr":"172.21.15.115:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6791","nonce":0}]},"addr":"172.21.15.132:6791/0","public_addr":"172.21.15.132:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6792","nonce":0}]},"addr":"172.21.15.115:6792/0","public_addr":"172.21.15.115:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6792","nonce":0}]},"addr":"172.21.15.132:6792/0","public_addr":"172.21.15.132:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6793","nonce":0}]},"addr":"172.21.15.115:6793/0","public_addr":"172.21.15.115:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-01-24T23:21:18.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.463+0000 7f6a371bf640 1 -- 172.21.15.115:0/443874670 >> v1:172.21.15.132:6812/1835401922 conn(0x7f6a1c037000 legacy=0x7f6a1c0394c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:18.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.463+0000 7f6a371bf640 10 monclient: shutdown 2024-01-24T23:21:18.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.463+0000 7f6a371bf640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:18.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.463+0000 7f6a371bf640 1 -- 172.21.15.115:0/443874670 >> v1:172.21.15.115:6791/0 conn(0x7f6a3013dcf0 legacy=0x7f6a301d3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:18.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.463+0000 7f6a371bf640 1 -- 172.21.15.115:0/443874670 shutdown_connections 2024-01-24T23:21:18.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.463+0000 7f6a371bf640 1 -- 172.21.15.115:0/443874670 >> 172.21.15.115:0/443874670 conn(0x7f6a300a65e0 msgr2=0x7f6a300a4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:18.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.463+0000 7f6a371bf640 1 -- 172.21.15.115:0/443874670 shutdown_connections 2024-01-24T23:21:18.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:18.463+0000 7f6a371bf640 1 -- 172.21.15.115:0/443874670 wait complete. 2024-01-24T23:21:21.485 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-01-24T23:21:21.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:21.611+0000 7efe51aa5640 10 monclient: get_monmap_and_config 2024-01-24T23:21:21.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:21.611+0000 7efe51aa5640 10 monclient: build_initial_monmap 2024-01-24T23:21:21.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:21.611+0000 7efe51aa5640 10 monclient: monmap: 2024-01-24T23:21:21.615 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:21.615 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:21.615 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:21.616112+0000 2024-01-24T23:21:21.615 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:21.616112+0000 2024-01-24T23:21:21.615 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:21.615 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:21.615 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:21.616 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:21.616 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:21.616 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:21.616 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:21.616 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:21.616 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:21.616 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:21.616 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:21.616 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:21.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:21.611+0000 7efe51aa5640 1 Processor -- start 2024-01-24T23:21:21.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:21.611+0000 7efe51aa5640 1 -- start start 2024-01-24T23:21:21.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:21.611+0000 7efe51aa5640 10 monclient: init 2024-01-24T23:21:21.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:21.611+0000 7efe51aa5640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:21.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:21.611+0000 7efe51aa5640 10 monclient: _add_conns ranks=[0,3,7,2,4,1,6,8,5] 2024-01-24T23:21:21.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:21.611+0000 7efe51aa5640 10 monclient(hunting): picked mon.noname-a con 0x7efe4c133bf0 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:21.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:21.611+0000 7efe51aa5640 10 monclient(hunting): picked mon.noname-g con 0x7efe4c133fd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:21.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:21.611+0000 7efe51aa5640 10 monclient(hunting): picked mon.noname-i con 0x7efe4c134430 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:21.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:21.611+0000 7efe51aa5640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4c04ed70 con 0x7efe4c133bf0 2024-01-24T23:21:21.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:21.611+0000 7efe51aa5640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4c09af30 con 0x7efe4c133fd0 2024-01-24T23:21:21.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:21.611+0000 7efe51aa5640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4c09b210 con 0x7efe4c134430 2024-01-24T23:21:21.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:21.611+0000 7efe51aa5640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:21.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:21.611+0000 7efe51aa5640 10 monclient(hunting): authenticate will time out at 960.538691s 2024-01-24T23:21:21.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:21.611+0000 7efe4b7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7efe4c133bf0 0x7efe4c09bd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58236/0 (socket says 172.21.15.115:58236) 2024-01-24T23:21:21.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:21.611+0000 7efe4b7fe640 1 -- 172.21.15.115:0/1494496462 learned_addr learned my addr 172.21.15.115:0/1494496462 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:24.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:24.615+0000 7efe497fa640 10 monclient(hunting): tick 2024-01-24T23:21:24.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:24.615+0000 7efe497fa640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:21:24.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:24.615+0000 7efe497fa640 1 monclient(hunting): continuing hunt 2024-01-24T23:21:24.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:24.615+0000 7efe497fa640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:21:24.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:24.615+0000 7efe497fa640 1 -- 172.21.15.115:0/1494496462 >> v1:172.21.15.132:6792/0 conn(0x7efe4c134430 legacy=0x7efe4c13f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:24.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:24.615+0000 7efe497fa640 1 -- 172.21.15.115:0/1494496462 >> v1:172.21.15.132:6790/0 conn(0x7efe4c133fd0 legacy=0x7efe4c13bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:24.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:24.615+0000 7efe497fa640 1 -- 172.21.15.115:0/1494496462 >> v1:172.21.15.115:6789/0 conn(0x7efe4c133bf0 legacy=0x7efe4c09bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:24.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:24.615+0000 7efe497fa640 10 monclient: _add_conns ranks=[8,1,4,6,2,5] 2024-01-24T23:21:24.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:24.615+0000 7efe497fa640 10 monclient(hunting): picked mon.noname-e con 0x7efe380037f0 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:24.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:24.615+0000 7efe497fa640 10 monclient(hunting): picked mon.noname-f con 0x7efe38007480 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:24.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:24.615+0000 7efe497fa640 10 monclient(hunting): picked mon.noname-c con 0x7efe3800b1b0 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:24.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:24.615+0000 7efe497fa640 1 -- 172.21.15.115:0/1494496462 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4c09b4f0 con 0x7efe38007480 2024-01-24T23:21:24.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:24.615+0000 7efe497fa640 1 -- 172.21.15.115:0/1494496462 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe3800ffe0 con 0x7efe3800b1b0 2024-01-24T23:21:24.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:24.615+0000 7efe4b7fe640 1 -- 172.21.15.115:0/1494496462 >> v1:172.21.15.115:6793/0 conn(0x7efe380037f0 legacy=0x7efe38005c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:21:24.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:24.615+0000 7efe497fa640 1 -- 172.21.15.115:0/1494496462 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe38011230 con 0x7efe380037f0 2024-01-24T23:21:24.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:24.615+0000 7efe497fa640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:24.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:24.819+0000 7efe4b7fe640 1 -- 172.21.15.115:0/1494496462 >> v1:172.21.15.115:6793/0 conn(0x7efe380037f0 legacy=0x7efe38005c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:21:25.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:25.223+0000 7efe4b7fe640 1 -- 172.21.15.115:0/1494496462 >> v1:172.21.15.115:6793/0 conn(0x7efe380037f0 legacy=0x7efe38005c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:21:26.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:26.027+0000 7efe4b7fe640 1 -- 172.21.15.115:0/1494496462 >> v1:172.21.15.115:6793/0 conn(0x7efe380037f0 legacy=0x7efe38005c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:21:27.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:27.615+0000 7efe497fa640 10 monclient(hunting): tick 2024-01-24T23:21:27.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:27.615+0000 7efe497fa640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:21:27.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:27.615+0000 7efe497fa640 1 monclient(hunting): continuing hunt 2024-01-24T23:21:27.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:27.615+0000 7efe497fa640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:21:27.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:27.615+0000 7efe497fa640 1 -- 172.21.15.115:0/1494496462 >> v1:172.21.15.115:6793/0 conn(0x7efe380037f0 legacy=0x7efe38005c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:21:27.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:27.615+0000 7efe497fa640 1 -- 172.21.15.115:0/1494496462 >> v1:172.21.15.115:6791/0 conn(0x7efe3800b1b0 legacy=0x7efe3800d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:27.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:27.615+0000 7efe497fa640 1 -- 172.21.15.115:0/1494496462 >> v1:172.21.15.132:6789/0 conn(0x7efe38007480 legacy=0x7efe38009900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:27.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:27.615+0000 7efe497fa640 10 monclient: _add_conns ranks=[2,5,6] 2024-01-24T23:21:27.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:27.615+0000 7efe497fa640 10 monclient(hunting): picked mon.noname-b con 0x7efe3800f100 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:27.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:27.615+0000 7efe4bfff640 1 -- 172.21.15.115:0/1494496462 reap_dead start 2024-01-24T23:21:27.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:27.615+0000 7efe497fa640 10 monclient(hunting): picked mon.noname-h con 0x7efe38010170 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:27.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:27.615+0000 7efe497fa640 10 monclient(hunting): picked mon.noname-d con 0x7efe38010d90 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:27.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:27.615+0000 7efe497fa640 1 -- 172.21.15.115:0/1494496462 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe38012480 con 0x7efe3800f100 2024-01-24T23:21:27.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:27.615+0000 7efe497fa640 1 -- 172.21.15.115:0/1494496462 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe38011230 con 0x7efe38010170 2024-01-24T23:21:27.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:27.615+0000 7efe497fa640 1 -- 172.21.15.115:0/1494496462 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe38006450 con 0x7efe38010d90 2024-01-24T23:21:27.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:27.615+0000 7efe4b7fe640 1 -- 172.21.15.115:0/1494496462 >> v1:172.21.15.115:6790/0 conn(0x7efe3800f100 legacy=0x7efe3800f520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:21:27.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:27.615+0000 7efe497fa640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:27.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:27.615+0000 7efe4bfff640 1 -- 172.21.15.115:0/1494496462 reap_dead start 2024-01-24T23:21:27.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:27.819+0000 7efe4b7fe640 1 -- 172.21.15.115:0/1494496462 >> v1:172.21.15.115:6790/0 conn(0x7efe3800f100 legacy=0x7efe3800f520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:21:28.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:28.223+0000 7efe4b7fe640 1 -- 172.21.15.115:0/1494496462 >> v1:172.21.15.115:6790/0 conn(0x7efe3800f100 legacy=0x7efe3800f520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:21:29.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:29.027+0000 7efe4b7fe640 1 -- 172.21.15.115:0/1494496462 >> v1:172.21.15.115:6790/0 conn(0x7efe3800f100 legacy=0x7efe3800f520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:21:30.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:30.615+0000 7efe497fa640 10 monclient(hunting): tick 2024-01-24T23:21:30.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:30.615+0000 7efe497fa640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:21:30.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:30.615+0000 7efe497fa640 1 monclient(hunting): continuing hunt 2024-01-24T23:21:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:30.615+0000 7efe497fa640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:21:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:30.615+0000 7efe497fa640 1 -- 172.21.15.115:0/1494496462 >> v1:172.21.15.115:6792/0 conn(0x7efe38010d90 legacy=0x7efe380197e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:30.615+0000 7efe497fa640 1 -- 172.21.15.115:0/1494496462 >> v1:172.21.15.132:6791/0 conn(0x7efe38010170 legacy=0x7efe38010620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:30.615+0000 7efe497fa640 1 -- 172.21.15.115:0/1494496462 >> v1:172.21.15.115:6790/0 conn(0x7efe3800f100 legacy=0x7efe3800f520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:21:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:30.615+0000 7efe497fa640 10 monclient: _add_conns ranks=[3,6,5,0,2,1,7,4,8] 2024-01-24T23:21:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:30.615+0000 7efe497fa640 10 monclient(hunting): picked mon.noname-g con 0x7efe38006830 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:30.615+0000 7efe497fa640 10 monclient(hunting): picked mon.noname-d con 0x7efe38006cb0 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:30.615+0000 7efe497fa640 10 monclient(hunting): picked mon.noname-h con 0x7efe380078f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:30.615+0000 7efe497fa640 1 -- 172.21.15.115:0/1494496462 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe38006620 con 0x7efe38006830 2024-01-24T23:21:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:30.615+0000 7efe497fa640 1 -- 172.21.15.115:0/1494496462 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe38012480 con 0x7efe380078f0 2024-01-24T23:21:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:30.615+0000 7efe497fa640 1 -- 172.21.15.115:0/1494496462 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe38009d80 con 0x7efe38006cb0 2024-01-24T23:21:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:30.615+0000 7efe497fa640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:32.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.883+0000 7efe4a7fc640 1 -- 172.21.15.115:0/1494496462 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 126319662 0 0) 0x7efe38006620 con 0x7efe38006830 2024-01-24T23:21:32.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:32.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:32.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:32.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 10 monclient(hunting): my global_id is 4503 2024-01-24T23:21:32.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 1 -- 172.21.15.115:0/1494496462 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe34003530 con 0x7efe38006830 2024-01-24T23:21:32.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 1 -- 172.21.15.115:0/1494496462 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1453307368 0 0) 0x7efe34003530 con 0x7efe38006830 2024-01-24T23:21:32.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 1 -- 172.21.15.115:0/1494496462 >> v1:172.21.15.115:6792/0 conn(0x7efe38006cb0 legacy=0x7efe38005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:32.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 1 -- 172.21.15.115:0/1494496462 >> v1:172.21.15.132:6791/0 conn(0x7efe380078f0 legacy=0x7efe3801b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:32.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:21:32.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 1 monclient: found mon.noname-g 2024-01-24T23:21:32.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:32.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:21:32.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 1 -- 172.21.15.115:0/1494496462 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe380105a0 con 0x7efe38006830 2024-01-24T23:21:32.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 10 monclient: _finish_auth 0 2024-01-24T23:21:32.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:21:32.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:32.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 1 -- 172.21.15.115:0/1494496462 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7efe40001910 con 0x7efe38006830 2024-01-24T23:21:32.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe51aa5640 5 monclient: authenticate success, global_id 4503 2024-01-24T23:21:32.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe51aa5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:32.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:32.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:21:32.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 10 monclient: dump: 2024-01-24T23:21:32.892 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:32.892 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:32.892 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:32.892 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:32.892 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:32.892 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:32.892 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:32.892 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:32.892 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:32.893 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:32.893 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:32.893 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:32.893 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:32.893 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:32.893 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:32.893 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:32.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe51aa5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:32.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 1 -- 172.21.15.115:0/1494496462 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efe40001b30 con 0x7efe38006830 2024-01-24T23:21:32.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:32.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 1 -- 172.21.15.115:0/1494496462 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7efe40004bf0 con 0x7efe38006830 2024-01-24T23:21:32.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:32.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:21:32.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe4a7fc640 10 monclient: dump: 2024-01-24T23:21:32.893 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:32.893 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:32.894 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:32.894 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:32.894 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:32.894 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:32.894 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:32.894 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:32.894 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:32.894 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:32.894 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:32.894 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:32.894 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:32.894 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:32.894 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:32.894 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:32.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe51aa5640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:32.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe51aa5640 4 set_mon_vals no callback set 2024-01-24T23:21:32.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe51aa5640 10 monclient: shutdown 2024-01-24T23:21:32.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe51aa5640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:32.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.887+0000 7efe51aa5640 1 -- 172.21.15.115:0/1494496462 >> v1:172.21.15.132:6790/0 conn(0x7efe38006830 legacy=0x7efe3800d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:32.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe51aa5640 1 -- 172.21.15.115:0/1494496462 shutdown_connections 2024-01-24T23:21:32.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe51aa5640 1 -- 172.21.15.115:0/1494496462 >> 172.21.15.115:0/1494496462 conn(0x7efe4c0a9910 msgr2=0x7efe4c0a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:32.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe51aa5640 1 -- 172.21.15.115:0/1494496462 shutdown_connections 2024-01-24T23:21:32.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe51aa5640 1 -- 172.21.15.115:0/1494496462 wait complete. 2024-01-24T23:21:32.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe51aa5640 10 monclient: build_initial_monmap 2024-01-24T23:21:32.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe51aa5640 10 monclient: monmap: 2024-01-24T23:21:32.899 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:32.899 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:32.899 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:32.899 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:32.899 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:32.899 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:32.899 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:32.899 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:32.899 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:32.899 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:32.899 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:32.899 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:32.899 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:32.899 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:32.900 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:32.900 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:32.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe51aa5640 1 Processor -- start 2024-01-24T23:21:32.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe51aa5640 1 -- start start 2024-01-24T23:21:32.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe51aa5640 10 monclient: init 2024-01-24T23:21:32.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe51aa5640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:32.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe51aa5640 10 monclient: _add_conns ranks=[1,8,7,2,0,3,6,5,4] 2024-01-24T23:21:32.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe51aa5640 10 monclient(hunting): picked mon.noname-f con 0x7efe38006830 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:32.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe51aa5640 10 monclient(hunting): picked mon.noname-e con 0x7efe38006cb0 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:32.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe51aa5640 10 monclient(hunting): picked mon.noname-i con 0x7efe380078f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:32.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe51aa5640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4c09f4f0 con 0x7efe38006830 2024-01-24T23:21:32.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe51aa5640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4c09c8f0 con 0x7efe380078f0 2024-01-24T23:21:32.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe51aa5640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4c0a5960 con 0x7efe38006cb0 2024-01-24T23:21:32.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe51aa5640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:32.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe51aa5640 10 monclient(hunting): authenticate will time out at 971.819075s 2024-01-24T23:21:32.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe4affd640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7efe38006cb0 legacy=0x7efe4c1d0760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:21:32.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe4bfff640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7efe380078f0 0x7efe4c1d3e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.115:46648/0 (socket says 172.21.15.115:46648) 2024-01-24T23:21:32.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe4bfff640 1 -- 172.21.15.115:0/333322325 learned_addr learned my addr 172.21.15.115:0/333322325 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:32.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe4b7fe640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7efe38006830 0x7efe4c132d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.115:52852/0 (socket says 172.21.15.115:52852) 2024-01-24T23:21:32.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe33fff640 1 -- 172.21.15.115:0/333322325 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4142036560 0 0) 0x7efe4c09f4f0 con 0x7efe38006830 2024-01-24T23:21:32.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe33fff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:32.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe33fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:32.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe33fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:32.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe33fff640 10 monclient(hunting): my global_id is 4492 2024-01-24T23:21:32.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.891+0000 7efe33fff640 1 -- 172.21.15.115:0/333322325 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe380072a0 con 0x7efe38006830 2024-01-24T23:21:32.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 1 -- 172.21.15.115:0/333322325 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1165513691 0 0) 0x7efe380072a0 con 0x7efe38006830 2024-01-24T23:21:32.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 1 -- 172.21.15.115:0/333322325 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efe4c09f4f0 con 0x7efe38006830 2024-01-24T23:21:32.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 1 -- 172.21.15.115:0/333322325 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7efe40000f00 con 0x7efe38006830 2024-01-24T23:21:32.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:32.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:21:32.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 10 monclient(hunting): dump: 2024-01-24T23:21:32.902 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:32.902 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:32.902 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:32.902 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:32.902 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:32.902 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:32.902 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:32.902 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:32.902 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:32.903 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:32.903 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:32.903 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:32.903 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:32.903 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:32.903 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:32.903 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:32.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:32.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 1 -- 172.21.15.115:0/333322325 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 814590868 0 0) 0x7efe4c09f4f0 con 0x7efe38006830 2024-01-24T23:21:32.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 1 -- 172.21.15.115:0/333322325 >> v1:172.21.15.115:6793/0 conn(0x7efe38006cb0 legacy=0x7efe4c1d0760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:21:32.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 1 -- 172.21.15.115:0/333322325 >> v1:172.21.15.132:6792/0 conn(0x7efe380078f0 legacy=0x7efe4c1d3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:32.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 10 monclient: _finish_hunting 0 2024-01-24T23:21:32.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 1 monclient: found mon.f 2024-01-24T23:21:32.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:32.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:21:32.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 1 -- 172.21.15.115:0/333322325 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4c1206d0 con 0x7efe38006830 2024-01-24T23:21:32.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 10 monclient: _finish_auth 0 2024-01-24T23:21:32.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 10 monclient: _check_auth_tickets 2024-01-24T23:21:32.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:32.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe51aa5640 5 monclient: authenticate success, global_id 4492 2024-01-24T23:21:32.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe51aa5640 10 monclient: _renew_subs 2024-01-24T23:21:32.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe51aa5640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:21:32.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe51aa5640 1 -- 172.21.15.115:0/333322325 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe4c11fe30 con 0x7efe38006830 2024-01-24T23:21:32.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe51aa5640 10 monclient: _renew_subs 2024-01-24T23:21:32.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe51aa5640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:21:32.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe51aa5640 1 -- 172.21.15.115:0/333322325 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7efe4c09f700 con 0x7efe38006830 2024-01-24T23:21:32.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 1 -- 172.21.15.115:0/333322325 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efe40004de0 con 0x7efe38006830 2024-01-24T23:21:32.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:32.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 1 -- 172.21.15.115:0/333322325 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7efe40005290 con 0x7efe38006830 2024-01-24T23:21:32.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:32.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:21:32.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe49ffb640 4 set_mon_vals no callback set 2024-01-24T23:21:32.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 10 monclient: dump: 2024-01-24T23:21:32.905 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:32.905 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:32.905 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:32.905 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:32.905 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:32.905 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:32.905 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:32.905 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:32.905 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:32.905 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:32.906 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:32.906 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:32.906 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:32.906 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:32.906 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:32.906 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:32.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe51aa5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:32.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe51aa5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:32.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe51aa5640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:21:32.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe51aa5640 1 -- 172.21.15.115:0/333322325 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe2c005090 con 0x7efe38006830 2024-01-24T23:21:32.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 1 -- 172.21.15.115:0/333322325 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7efe4000fd10 con 0x7efe38006830 2024-01-24T23:21:32.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe33fff640 1 -- 172.21.15.115:0/333322325 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7efe40043930 con 0x7efe38006830 2024-01-24T23:21:32.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.895+0000 7efe4affd640 10 monclient: get_auth_request con 0x7efe3803a2b0 auth_method 0 2024-01-24T23:21:32.913 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.911+0000 7efe33fff640 1 -- 172.21.15.115:0/333322325 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7efe40014b30 con 0x7efe38006830 2024-01-24T23:21:32.913 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.911+0000 7efe33fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:32.913 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:32.911+0000 7efe33fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:33.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:33.119+0000 7efe51aa5640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-01-24T23:21:33.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:33.119+0000 7efe51aa5640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-01-24T23:21:33.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:33.119+0000 7efe51aa5640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:21:33.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:33.119+0000 7efe51aa5640 1 -- 172.21.15.115:0/333322325 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7efe2c0058c0 con 0x7efe38006830 2024-01-24T23:21:35.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:35.891+0000 7efe32ffd640 10 monclient: tick 2024-01-24T23:21:35.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:35.891+0000 7efe32ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:21:35.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:35.891+0000 7efe32ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:35.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:35.891+0000 7efe32ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:36.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:36.891+0000 7efe32ffd640 10 monclient: tick 2024-01-24T23:21:36.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:36.891+0000 7efe32ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:21:36.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:36.891+0000 7efe32ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:37.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:37.891+0000 7efe32ffd640 10 monclient: tick 2024-01-24T23:21:37.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:37.891+0000 7efe32ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:21:37.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:37.891+0000 7efe32ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:37.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:37.911+0000 7efe33fff640 1 -- 172.21.15.115:0/333322325 <== mon.1 v1:172.21.15.132:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2732 (unknown 254746657 0 3107291840) 0x7efe400131d0 con 0x7efe38006830 2024-01-24T23:21:37.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:37.911+0000 7efe33fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-01-24T23:21:37.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:37.911+0000 7efe33fff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:37.917 INFO:teuthology.orchestra.run.smithi115.stdout:{"election_epoch":12,"quorum":[0,1,3,4,5,6,7],"quorum_names":["a","f","g","c","h","d","i"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701272860131327","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","modified":"2024-01-24T23:20:20.557246Z","created":"2024-01-24T23:20:20.557246Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6790","nonce":0}]},"addr":"172.21.15.115:6790/0","public_addr":"172.21.15.115:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6790","nonce":0}]},"addr":"172.21.15.132:6790/0","public_addr":"172.21.15.132:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6791","nonce":0}]},"addr":"172.21.15.115:6791/0","public_addr":"172.21.15.115:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6791","nonce":0}]},"addr":"172.21.15.132:6791/0","public_addr":"172.21.15.132:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6792","nonce":0}]},"addr":"172.21.15.115:6792/0","public_addr":"172.21.15.115:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6792","nonce":0}]},"addr":"172.21.15.132:6792/0","public_addr":"172.21.15.132:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6793","nonce":0}]},"addr":"172.21.15.115:6793/0","public_addr":"172.21.15.115:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-01-24T23:21:37.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:37.915+0000 7efe51aa5640 1 -- 172.21.15.115:0/333322325 >> v1:172.21.15.132:6812/1835401922 conn(0x7efe3803a2b0 legacy=0x7efe3803c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:37.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:37.915+0000 7efe51aa5640 10 monclient: shutdown 2024-01-24T23:21:37.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:37.915+0000 7efe51aa5640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:37.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:37.915+0000 7efe51aa5640 1 -- 172.21.15.115:0/333322325 >> v1:172.21.15.132:6789/0 conn(0x7efe38006830 legacy=0x7efe4c132d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:37.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:37.915+0000 7efe51aa5640 1 -- 172.21.15.115:0/333322325 shutdown_connections 2024-01-24T23:21:37.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:37.915+0000 7efe51aa5640 1 -- 172.21.15.115:0/333322325 >> 172.21.15.115:0/333322325 conn(0x7efe4c0a9910 msgr2=0x7efe4c1bea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:37.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:37.915+0000 7efe51aa5640 1 -- 172.21.15.115:0/333322325 shutdown_connections 2024-01-24T23:21:37.921 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:37.915+0000 7efe51aa5640 1 -- 172.21.15.115:0/333322325 wait complete. 2024-01-24T23:21:37.947 INFO:tasks.mon_thrash.ceph_manager:quorum is size 7 2024-01-24T23:21:37.947 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon a 2024-01-24T23:21:37.947 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-01-24T23:21:38.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 10 monclient: get_monmap_and_config 2024-01-24T23:21:38.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 10 monclient: build_initial_monmap 2024-01-24T23:21:38.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 10 monclient: monmap: 2024-01-24T23:21:38.070 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:38.070 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.070 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:38.071192+0000 2024-01-24T23:21:38.070 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:38.071192+0000 2024-01-24T23:21:38.070 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:38.070 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.070 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:38.070 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:38.070 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:38.070 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:38.071 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:38.071 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:38.071 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:38.071 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:38.071 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:38.071 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 1 Processor -- start 2024-01-24T23:21:38.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 1 -- start start 2024-01-24T23:21:38.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 10 monclient: init 2024-01-24T23:21:38.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:38.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 10 monclient: _add_conns ranks=[4,7,8,2,5,3,1,0,6] 2024-01-24T23:21:38.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 10 monclient(hunting): picked mon.noname-c con 0x7fe4a8133bf0 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:38.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 10 monclient(hunting): picked mon.noname-i con 0x7fe4a8133fd0 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:38.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 10 monclient(hunting): picked mon.noname-e con 0x7fe4a8134430 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:38.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a804ed70 con 0x7fe4a8133bf0 2024-01-24T23:21:38.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a809af30 con 0x7fe4a8133fd0 2024-01-24T23:21:38.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a809b210 con 0x7fe4a8134430 2024-01-24T23:21:38.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:38.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 10 monclient(hunting): authenticate will time out at 976.993810s 2024-01-24T23:21:38.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a77fe640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7fe4a8134430 legacy=0x7fe4a813f500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:21:38.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a6ffd640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fe4a8133bf0 0x7fe4a809bd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:51640/0 (socket says 172.21.15.115:51640) 2024-01-24T23:21:38.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a6ffd640 1 -- 172.21.15.115:0/2077614853 learned_addr learned my addr 172.21.15.115:0/2077614853 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:38.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 1 -- 172.21.15.115:0/2077614853 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2040277757 0 0) 0x7fe4a804ed70 con 0x7fe4a8133bf0 2024-01-24T23:21:38.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 10 monclient(hunting): my global_id is 74569 2024-01-24T23:21:38.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 1 -- 172.21.15.115:0/2077614853 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe490003530 con 0x7fe4a8133bf0 2024-01-24T23:21:38.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 1 -- 172.21.15.115:0/2077614853 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4232984396 0 0) 0x7fe490003530 con 0x7fe4a8133bf0 2024-01-24T23:21:38.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 1 -- 172.21.15.115:0/2077614853 >> v1:172.21.15.115:6793/0 conn(0x7fe4a8134430 legacy=0x7fe4a813f500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:21:38.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 1 -- 172.21.15.115:0/2077614853 >> v1:172.21.15.132:6792/0 conn(0x7fe4a8133fd0 legacy=0x7fe4a813bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:21:38.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 1 monclient: found mon.noname-c 2024-01-24T23:21:38.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:38.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:21:38.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 1 -- 172.21.15.115:0/2077614853 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a809b4f0 con 0x7fe4a8133bf0 2024-01-24T23:21:38.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 10 monclient: _finish_auth 0 2024-01-24T23:21:38.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:21:38.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:38.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 5 monclient: authenticate success, global_id 74569 2024-01-24T23:21:38.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 1 -- 172.21.15.115:0/2077614853 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe4a0001b40 con 0x7fe4a8133bf0 2024-01-24T23:21:38.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:38.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:21:38.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 10 monclient: dump: 2024-01-24T23:21:38.083 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.083 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.083 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.083 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.083 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.083 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.083 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.083 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.083 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.083 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.083 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.083 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.083 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.083 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.084 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.084 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:38.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 1 -- 172.21.15.115:0/2077614853 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe4a0001e60 con 0x7fe4a8133bf0 2024-01-24T23:21:38.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:38.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:38.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 4 set_mon_vals no callback set 2024-01-24T23:21:38.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 1 -- 172.21.15.115:0/2077614853 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe4a0004f70 con 0x7fe4a8133bf0 2024-01-24T23:21:38.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:38.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4a5ffb640 10 monclient: dump: 2024-01-24T23:21:38.084 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.084 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.084 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.085 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.085 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.085 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.085 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.085 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.085 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.085 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.085 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.085 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.085 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.085 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.085 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.085 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 10 monclient: shutdown 2024-01-24T23:21:38.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:38.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 1 -- 172.21.15.115:0/2077614853 >> v1:172.21.15.115:6791/0 conn(0x7fe4a8133bf0 legacy=0x7fe4a809bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 1 -- 172.21.15.115:0/2077614853 shutdown_connections 2024-01-24T23:21:38.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 1 -- 172.21.15.115:0/2077614853 >> 172.21.15.115:0/2077614853 conn(0x7fe4a80a9910 msgr2=0x7fe4a80a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:38.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 1 -- 172.21.15.115:0/2077614853 shutdown_connections 2024-01-24T23:21:38.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.067+0000 7fe4ad4f9640 1 -- 172.21.15.115:0/2077614853 wait complete. 2024-01-24T23:21:38.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4ad4f9640 10 monclient: build_initial_monmap 2024-01-24T23:21:38.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4ad4f9640 10 monclient: monmap: 2024-01-24T23:21:38.086 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:38.086 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:38.086 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:38.086 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:38.086 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:38.086 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:38.086 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:38.086 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:38.087 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:38.087 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:38.087 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:38.087 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:38.087 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:38.087 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:38.087 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:38.087 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4ad4f9640 1 Processor -- start 2024-01-24T23:21:38.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4ad4f9640 1 -- start start 2024-01-24T23:21:38.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4ad4f9640 10 monclient: init 2024-01-24T23:21:38.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4ad4f9640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:38.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4ad4f9640 10 monclient: _add_conns ranks=[1,6,4,8,0,5,2,7,3] 2024-01-24T23:21:38.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4ad4f9640 10 monclient(hunting): picked mon.noname-f con 0x7fe4a8133bf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:38.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4ad4f9640 10 monclient(hunting): picked mon.noname-d con 0x7fe4a8133fd0 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:38.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4ad4f9640 10 monclient(hunting): picked mon.noname-c con 0x7fe4a8134430 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:38.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4ad4f9640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a813a610 con 0x7fe4a8133bf0 2024-01-24T23:21:38.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4ad4f9640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a813a8f0 con 0x7fe4a8134430 2024-01-24T23:21:38.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4ad4f9640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a813abd0 con 0x7fe4a8133fd0 2024-01-24T23:21:38.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4ad4f9640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:38.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4ad4f9640 10 monclient(hunting): authenticate will time out at 976.997111s 2024-01-24T23:21:38.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4a77fe640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fe4a8134430 0x7fe4a81d3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:51656/0 (socket says 172.21.15.115:51656) 2024-01-24T23:21:38.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4a77fe640 1 -- 172.21.15.115:0/3956090890 learned_addr learned my addr 172.21.15.115:0/3956090890 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:38.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2785709742 0 0) 0x7fe4a813a8f0 con 0x7fe4a8134430 2024-01-24T23:21:38.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient(hunting): my global_id is 74578 2024-01-24T23:21:38.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe494003530 con 0x7fe4a8134430 2024-01-24T23:21:38.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 721681832 0 0) 0x7fe4a813abd0 con 0x7fe4a8133fd0 2024-01-24T23:21:38.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient(hunting): my global_id is 4524 2024-01-24T23:21:38.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4a813a8f0 con 0x7fe4a8133fd0 2024-01-24T23:21:38.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 371921210 0 0) 0x7fe494003530 con 0x7fe4a8134430 2024-01-24T23:21:38.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe4a813abd0 con 0x7fe4a8134430 2024-01-24T23:21:38.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe4980041e0 con 0x7fe4a8134430 2024-01-24T23:21:38.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:21:38.089 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient(hunting): dump: 2024-01-24T23:21:38.089 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.090 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.090 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.090 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.090 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.090 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.090 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.090 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.090 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.090 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.090 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.090 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.090 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.090 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.090 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.091 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:38.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1837059676 0 0) 0x7fe4a813a8f0 con 0x7fe4a8133fd0 2024-01-24T23:21:38.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe494003530 con 0x7fe4a8133fd0 2024-01-24T23:21:38.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1787293940 0 0) 0x7fe4a813a610 con 0x7fe4a8133bf0 2024-01-24T23:21:38.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient(hunting): my global_id is 4501 2024-01-24T23:21:38.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4a813a8f0 con 0x7fe4a8133bf0 2024-01-24T23:21:38.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe49c001960 con 0x7fe4a8133fd0 2024-01-24T23:21:38.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:38.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient(hunting): dump: 2024-01-24T23:21:38.092 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.092 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.092 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.092 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.092 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.092 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.092 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.092 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.092 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.092 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.092 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.092 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.092 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.092 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.093 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.093 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3539866417 0 0) 0x7fe4a813abd0 con 0x7fe4a8134430 2024-01-24T23:21:38.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 >> v1:172.21.15.115:6792/0 conn(0x7fe4a8133fd0 legacy=0x7fe4a81d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 >> v1:172.21.15.132:6789/0 conn(0x7fe4a8133bf0 legacy=0x7fe4a81c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:21:38.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 monclient: found mon.c 2024-01-24T23:21:38.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:38.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:38.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a8139af0 con 0x7fe4a8134430 2024-01-24T23:21:38.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient: _finish_auth 0 2024-01-24T23:21:38.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:21:38.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:38.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4ad4f9640 5 monclient: authenticate success, global_id 74578 2024-01-24T23:21:38.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4ad4f9640 10 monclient: _renew_subs 2024-01-24T23:21:38.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4ad4f9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:38.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4ad4f9640 1 -- 172.21.15.115:0/3956090890 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a8139d40 con 0x7fe4a8134430 2024-01-24T23:21:38.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4ad4f9640 10 monclient: _renew_subs 2024-01-24T23:21:38.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4ad4f9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:21:38.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4ad4f9640 1 -- 172.21.15.115:0/3956090890 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a8139880 con 0x7fe4a8134430 2024-01-24T23:21:38.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe498004400 con 0x7fe4a8134430 2024-01-24T23:21:38.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:38.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe498004930 con 0x7fe4a8134430 2024-01-24T23:21:38.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4a57fa640 4 set_mon_vals no callback set 2024-01-24T23:21:38.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:38.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient: dump: 2024-01-24T23:21:38.094 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.094 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.095 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.095 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.095 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.095 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.095 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.095 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.095 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.095 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.095 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.095 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.095 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.095 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.095 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.095 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4ad4f9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:38.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4ad4f9640 1 -- 172.21.15.115:0/3956090890 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe4880065d0 con 0x7fe488002870 2024-01-24T23:21:38.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fe498004a90 con 0x7fe4a8134430 2024-01-24T23:21:38.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7fe4980434a0 con 0x7fe4a8134430 2024-01-24T23:21:38.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1446782172 0 0) 0x7fe4880065d0 con 0x7fe488002870 2024-01-24T23:21:38.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient(hunting): my global_id is 4536 2024-01-24T23:21:38.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4a813abd0 con 0x7fe488002870 2024-01-24T23:21:38.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe4a67fc640 10 monclient: get_auth_request con 0x7fe494037850 auth_method 0 2024-01-24T23:21:38.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 187462408 0 0) 0x7fe4a813abd0 con 0x7fe488002870 2024-01-24T23:21:38.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe4880065d0 con 0x7fe488002870 2024-01-24T23:21:38.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe4a0004eb0 con 0x7fe488002870 2024-01-24T23:21:38.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:38.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 10 monclient: dump: 2024-01-24T23:21:38.097 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.097 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.097 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.097 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.097 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.097 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.097 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.097 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.097 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.097 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.097 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.097 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.097 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.098 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.098 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.098 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 780000627 0 0) 0x7fe4880065d0 con 0x7fe488002870 2024-01-24T23:21:38.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.071+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 --> v1:172.21.15.115:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe48800a120 con 0x7fe488002870 2024-01-24T23:21:38.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.075+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 <== mon.0 v1:172.21.15.115:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fe4a0001850 con 0x7fe488002870 2024-01-24T23:21:38.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.075+0000 7fe48f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:38.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.075+0000 7fe48f7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:38.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.075+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 >> v1:172.21.15.115:6789/0 conn(0x7fe488002870 legacy=0x7fe488004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.075+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 >> v1:172.21.15.115:6789/0 conn(0x7fe488002870 legacy=0x7fe488004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:38.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe4ad4f9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:21:38.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe4ad4f9640 1 -- 172.21.15.115:0/3956090890 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe488009e10 con 0x7fe4880090d0 2024-01-24T23:21:38.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714299591 0 0) 0x7fe488009e10 con 0x7fe4880090d0 2024-01-24T23:21:38.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe48f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe48f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe48f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe48f7fe640 10 monclient(hunting): my global_id is 4545 2024-01-24T23:21:38.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4880065d0 con 0x7fe4880090d0 2024-01-24T23:21:38.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3915079495 0 0) 0x7fe4880065d0 con 0x7fe4880090d0 2024-01-24T23:21:38.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe488009e10 con 0x7fe4880090d0 2024-01-24T23:21:38.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe4a0005580 con 0x7fe4880090d0 2024-01-24T23:21:38.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe48f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe48f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:38.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe48f7fe640 10 monclient: dump: 2024-01-24T23:21:38.101 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.101 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.101 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.101 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.101 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.101 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.101 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.101 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.102 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.102 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.102 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.102 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.102 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.102 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.102 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.102 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1919152907 0 0) 0x7fe488009e10 con 0x7fe4880090d0 2024-01-24T23:21:38.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 --> v1:172.21.15.115:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe48800c310 con 0x7fe4880090d0 2024-01-24T23:21:38.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 <== mon.0 v1:172.21.15.115:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7693 (unknown 0 0 1620164272) 0x7fe4a00086f0 con 0x7fe4880090d0 2024-01-24T23:21:38.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe48f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:21:38.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe48f7fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:38.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 >> v1:172.21.15.115:6789/0 conn(0x7fe4880090d0 legacy=0x7fe488009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe48f7fe640 1 -- 172.21.15.115:0/3956090890 >> v1:172.21.15.115:6789/0 conn(0x7fe4880090d0 legacy=0x7fe488009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:38.103 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:21:38.103 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:21:38.103 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:21:38.103 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "leader", 2024-01-24T23:21:38.103 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 12, 2024-01-24T23:21:38.103 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:21:38.103 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:21:38.103 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:21:38.103 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:21:38.104 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:21:38.104 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:21:38.104 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:21:38.104 INFO:teuthology.orchestra.run.smithi115.stdout: 7 2024-01-24T23:21:38.104 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.104 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:21:38.104 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:38.104 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:21:38.104 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:21:38.104 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:38.104 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:38.104 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:38.104 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:38.105 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:38.105 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:38.105 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:38.105 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:38.105 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:38.105 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:38.105 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:38.105 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.105 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:21:38.105 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:21:38.105 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:38.105 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:38.105 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:38.105 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:38.106 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:38.106 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:38.106 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:38.106 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:38.106 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:38.106 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:38.106 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:38.106 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.106 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:21:38.106 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:21:38.106 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:21:38.107 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:21:38.107 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:21:38.107 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:21:38.107 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:38.107 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:38.107 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:21:38.107 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:21:38.107 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:21:38.107 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:21:38.107 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:21:38.107 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:21:38.107 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:21:38.107 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:38.108 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:21:38.108 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:38.108 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:38.108 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:38.108 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:38.108 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:38.108 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:38.108 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:38.108 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:38.108 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:38.108 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:38.108 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:38.108 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.109 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:21:38.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.109 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:21:38.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.109 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:21:38.109 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:21:38.109 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.109 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.109 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.109 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:21:38.109 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.109 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.109 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.110 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:21:38.110 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:21:38.110 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.110 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.110 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.110 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:21:38.110 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:21:38.110 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.110 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.110 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.110 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:21:38.111 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.111 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.111 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.111 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:21:38.111 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:21:38.111 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.111 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.111 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.111 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:21:38.111 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:21:38.111 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.112 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.112 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.112 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:21:38.112 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.112 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.112 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.112 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:21:38.112 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:21:38.112 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.112 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.112 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.113 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:21:38.113 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:21:38.113 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.113 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.113 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.113 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:21:38.113 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.113 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.113 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.113 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:21:38.113 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:21:38.114 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.114 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.114 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.114 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:21:38.114 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:21:38.114 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.114 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.114 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.114 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:21:38.114 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.114 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.115 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.115 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:21:38.115 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:21:38.115 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.115 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.115 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.115 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:21:38.115 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:21:38.115 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.115 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.116 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.116 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:21:38.116 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.116 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.116 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.116 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:21:38.116 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:21:38.116 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.116 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.116 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.117 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:21:38.117 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:21:38.117 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.117 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.117 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.117 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:21:38.117 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.117 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.117 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.117 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:21:38.117 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:21:38.117 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.117 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.118 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.118 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:21:38.118 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:21:38.118 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.118 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.118 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.118 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:21:38.118 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.118 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.118 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.119 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:21:38.119 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:21:38.119 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.119 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.119 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.119 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:21:38.119 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:21:38.119 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.119 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.119 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.119 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:21:38.120 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.120 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.120 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.120 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:21:38.120 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:21:38.120 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.120 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.120 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.120 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.120 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.120 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:21:38.120 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:21:38.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.121 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:38.121 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:38.121 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:38.121 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.121 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.121 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:21:38.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.121 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:38.121 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:38.121 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:38.121 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.121 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.121 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:21:38.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.122 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:38.122 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:38.122 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:21:38.122 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.122 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.122 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:21:38.122 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:21:38.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe4ad4f9640 1 -- 172.21.15.115:0/3956090890 >> v1:172.21.15.132:6812/1835401922 conn(0x7fe494037850 legacy=0x7fe494039d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe4ad4f9640 10 monclient: shutdown 2024-01-24T23:21:38.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe4ad4f9640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:38.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe4ad4f9640 1 -- 172.21.15.115:0/3956090890 >> v1:172.21.15.115:6791/0 conn(0x7fe4a8134430 legacy=0x7fe4a81d3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe4a77fe640 1 -- 172.21.15.115:0/3956090890 reap_dead start 2024-01-24T23:21:38.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe4a77fe640 1 -- 172.21.15.115:0/3956090890 reap_dead start 2024-01-24T23:21:38.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe4ad4f9640 1 -- 172.21.15.115:0/3956090890 shutdown_connections 2024-01-24T23:21:38.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe4ad4f9640 1 -- 172.21.15.115:0/3956090890 >> 172.21.15.115:0/3956090890 conn(0x7fe4a80a9910 msgr2=0x7fe4a8140050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:38.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe4ad4f9640 1 -- 172.21.15.115:0/3956090890 shutdown_connections 2024-01-24T23:21:38.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.079+0000 7fe4ad4f9640 1 -- 172.21.15.115:0/3956090890 wait complete. 2024-01-24T23:21:38.124 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-01-24T23:21:38.124 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon c 2024-01-24T23:21:38.125 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-01-24T23:21:38.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7fdef640 10 monclient: get_monmap_and_config 2024-01-24T23:21:38.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7fdef640 10 monclient: build_initial_monmap 2024-01-24T23:21:38.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7fdef640 10 monclient: monmap: 2024-01-24T23:21:38.238 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:38.239 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.239 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:38.239817+0000 2024-01-24T23:21:38.239 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:38.239817+0000 2024-01-24T23:21:38.239 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:38.239 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.239 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:38.239 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:38.239 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:38.239 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:38.239 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:38.239 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:38.239 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:38.239 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:38.240 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:38.240 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7fdef640 1 Processor -- start 2024-01-24T23:21:38.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7fdef640 1 -- start start 2024-01-24T23:21:38.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7fdef640 10 monclient: init 2024-01-24T23:21:38.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7fdef640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:38.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7fdef640 10 monclient: _add_conns ranks=[2,1,5,3,0,7,8,6,4] 2024-01-24T23:21:38.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7fdef640 10 monclient(hunting): picked mon.noname-b con 0x7f6b78134c70 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:38.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7fdef640 10 monclient(hunting): picked mon.noname-f con 0x7f6b78067360 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:38.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7fdef640 10 monclient(hunting): picked mon.noname-h con 0x7f6b7809ecd0 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:38.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7fdef640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b7804ed70 con 0x7f6b78067360 2024-01-24T23:21:38.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7fdef640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b7809e050 con 0x7f6b78134c70 2024-01-24T23:21:38.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7fdef640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b7809da00 con 0x7f6b7809ecd0 2024-01-24T23:21:38.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7fdef640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:38.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7db8b640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f6b78134c70 legacy=0x7f6b78135050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:21:38.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7fdef640 10 monclient(hunting): authenticate will time out at 977.162461s 2024-01-24T23:21:38.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7d38a640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f6b78067360 0x7f6b78139ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.115:52874/0 (socket says 172.21.15.115:52874) 2024-01-24T23:21:38.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7d38a640 1 -- 172.21.15.115:0/2127438307 learned_addr learned my addr 172.21.15.115:0/2127438307 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:38.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7e38c640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f6b7809ecd0 0x7f6b7813d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.115:37132/0 (socket says 172.21.15.115:37132) 2024-01-24T23:21:38.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 1 -- 172.21.15.115:0/2127438307 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 137049575 0 0) 0x7f6b7809da00 con 0x7f6b7809ecd0 2024-01-24T23:21:38.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 10 monclient(hunting): my global_id is 4469 2024-01-24T23:21:38.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 1 -- 172.21.15.115:0/2127438307 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b64003530 con 0x7f6b7809ecd0 2024-01-24T23:21:38.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 1 -- 172.21.15.115:0/2127438307 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1826487076 0 0) 0x7f6b7804ed70 con 0x7f6b78067360 2024-01-24T23:21:38.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 10 monclient(hunting): my global_id is 4510 2024-01-24T23:21:38.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 1 -- 172.21.15.115:0/2127438307 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b7809da00 con 0x7f6b78067360 2024-01-24T23:21:38.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 1 -- 172.21.15.115:0/2127438307 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 482690706 0 0) 0x7f6b64003530 con 0x7f6b7809ecd0 2024-01-24T23:21:38.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 1 -- 172.21.15.115:0/2127438307 >> v1:172.21.15.115:6790/0 conn(0x7f6b78134c70 legacy=0x7f6b78135050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:21:38.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 1 -- 172.21.15.115:0/2127438307 >> v1:172.21.15.132:6789/0 conn(0x7f6b78067360 legacy=0x7f6b78139ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 10 monclient: _finish_hunting 0 2024-01-24T23:21:38.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 1 monclient: found mon.noname-h 2024-01-24T23:21:38.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:38.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:21:38.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 1 -- 172.21.15.115:0/2127438307 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7809dce0 con 0x7f6b7809ecd0 2024-01-24T23:21:38.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 10 monclient: _finish_auth 0 2024-01-24T23:21:38.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 10 monclient: _check_auth_tickets 2024-01-24T23:21:38.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:38.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 1 -- 172.21.15.115:0/2127438307 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6b700017c0 con 0x7f6b7809ecd0 2024-01-24T23:21:38.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:21:38.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7cb89640 10 monclient: dump: 2024-01-24T23:21:38.246 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.246 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.246 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.246 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.247 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.247 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.247 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.247 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.247 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.247 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.247 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.247 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.247 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.247 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.247 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.247 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7fdef640 5 monclient: authenticate success, global_id 4469 2024-01-24T23:21:38.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.235+0000 7f6b7fdef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:38.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7cb89640 1 -- 172.21.15.115:0/2127438307 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6b70001aa0 con 0x7f6b7809ecd0 2024-01-24T23:21:38.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7cb89640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:38.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7cb89640 1 -- 172.21.15.115:0/2127438307 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6b70004b30 con 0x7f6b7809ecd0 2024-01-24T23:21:38.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:38.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 4 set_mon_vals no callback set 2024-01-24T23:21:38.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7cb89640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7cb89640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:21:38.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7cb89640 10 monclient: dump: 2024-01-24T23:21:38.248 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.248 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.248 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.248 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.248 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.248 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.249 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.249 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.249 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.249 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.249 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.249 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.249 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.249 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.249 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.249 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 10 monclient: shutdown 2024-01-24T23:21:38.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:38.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 1 -- 172.21.15.115:0/2127438307 >> v1:172.21.15.132:6791/0 conn(0x7f6b7809ecd0 legacy=0x7f6b7813d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 1 -- 172.21.15.115:0/2127438307 shutdown_connections 2024-01-24T23:21:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 1 -- 172.21.15.115:0/2127438307 >> 172.21.15.115:0/2127438307 conn(0x7f6b780a65e0 msgr2=0x7f6b780a2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 1 -- 172.21.15.115:0/2127438307 shutdown_connections 2024-01-24T23:21:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 1 -- 172.21.15.115:0/2127438307 wait complete. 2024-01-24T23:21:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 10 monclient: build_initial_monmap 2024-01-24T23:21:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 10 monclient: monmap: 2024-01-24T23:21:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:38.251 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:38.251 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:38.251 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:38.251 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:38.251 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 1 Processor -- start 2024-01-24T23:21:38.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 1 -- start start 2024-01-24T23:21:38.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 10 monclient: init 2024-01-24T23:21:38.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:38.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 10 monclient: _add_conns ranks=[6,7,0,5,4,3,1,2,8] 2024-01-24T23:21:38.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 10 monclient(hunting): picked mon.noname-d con 0x7f6b7809ecd0 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:38.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 10 monclient(hunting): picked mon.noname-i con 0x7f6b78134c70 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:38.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 10 monclient(hunting): picked mon.noname-a con 0x7f6b78130720 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:38.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b78067750 con 0x7f6b78130720 2024-01-24T23:21:38.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b7812fbe0 con 0x7f6b7809ecd0 2024-01-24T23:21:38.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b78071df0 con 0x7f6b78134c70 2024-01-24T23:21:38.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:38.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 10 monclient(hunting): authenticate will time out at 977.166500s 2024-01-24T23:21:38.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7db8b640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f6b7809ecd0 0x7f6b781333e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:38622/0 (socket says 172.21.15.115:38622) 2024-01-24T23:21:38.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7db8b640 1 -- 172.21.15.115:0/3934869731 learned_addr learned my addr 172.21.15.115:0/3934869731 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:38.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7e38c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f6b78130720 0x7f6b781d1800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39614/0 (socket says 172.21.15.115:39614) 2024-01-24T23:21:38.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1208724608 0 0) 0x7f6b78067750 con 0x7f6b78130720 2024-01-24T23:21:38.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient(hunting): my global_id is 4554 2024-01-24T23:21:38.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b5c003530 con 0x7f6b78130720 2024-01-24T23:21:38.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 453908427 0 0) 0x7f6b7812fbe0 con 0x7f6b7809ecd0 2024-01-24T23:21:38.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient(hunting): my global_id is 4533 2024-01-24T23:21:38.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b78067750 con 0x7f6b7809ecd0 2024-01-24T23:21:38.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2624710405 0 0) 0x7f6b5c003530 con 0x7f6b78130720 2024-01-24T23:21:38.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6b7812fbe0 con 0x7f6b78130720 2024-01-24T23:21:38.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2500431204 0 0) 0x7f6b78067750 con 0x7f6b7809ecd0 2024-01-24T23:21:38.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6b5c003530 con 0x7f6b7809ecd0 2024-01-24T23:21:38.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6b70001780 con 0x7f6b78130720 2024-01-24T23:21:38.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:21:38.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient(hunting): dump: 2024-01-24T23:21:38.258 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.258 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.258 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.258 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.259 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.259 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.259 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.259 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.259 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.259 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.259 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.259 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.259 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.259 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.259 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.259 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:38.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6b6c004300 con 0x7f6b7809ecd0 2024-01-24T23:21:38.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:38.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient(hunting): dump: 2024-01-24T23:21:38.260 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.260 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.260 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.260 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.260 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.260 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.260 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.260 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.260 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.260 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.260 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.260 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.260 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.260 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.261 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.261 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2920082711 0 0) 0x7f6b78071df0 con 0x7f6b78134c70 2024-01-24T23:21:38.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient(hunting): my global_id is 4588 2024-01-24T23:21:38.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b78067750 con 0x7f6b78134c70 2024-01-24T23:21:38.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1481076079 0 0) 0x7f6b7812fbe0 con 0x7f6b78130720 2024-01-24T23:21:38.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 >> v1:172.21.15.132:6792/0 conn(0x7f6b78134c70 legacy=0x7f6b781ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 >> v1:172.21.15.115:6792/0 conn(0x7f6b7809ecd0 legacy=0x7f6b781333e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:21:38.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 1 monclient: found mon.a 2024-01-24T23:21:38.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:38.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:21:38.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b78131610 con 0x7f6b78130720 2024-01-24T23:21:38.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient: _finish_auth 0 2024-01-24T23:21:38.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:21:38.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:38.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 5 monclient: authenticate success, global_id 4554 2024-01-24T23:21:38.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 10 monclient: _renew_subs 2024-01-24T23:21:38.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:21:38.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 1 -- 172.21.15.115:0/3934869731 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b781318f0 con 0x7f6b78130720 2024-01-24T23:21:38.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 10 monclient: _renew_subs 2024-01-24T23:21:38.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:21:38.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b7fdef640 1 -- 172.21.15.115:0/3934869731 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b78133af0 con 0x7f6b78130720 2024-01-24T23:21:38.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6b70001150 con 0x7f6b78130720 2024-01-24T23:21:38.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:38.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6b70005140 con 0x7f6b78130720 2024-01-24T23:21:38.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6b7fe640 4 set_mon_vals no callback set 2024-01-24T23:21:38.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:38.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.239+0000 7f6b6a7fc640 10 monclient: dump: 2024-01-24T23:21:38.263 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.263 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.263 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.263 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.263 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.263 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.263 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.263 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.263 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.263 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.263 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.263 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.264 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.264 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.264 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.264 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.243+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f6b7000fbc0 con 0x7f6b78130720 2024-01-24T23:21:38.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.243+0000 7f6b7fdef640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:38.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.243+0000 7f6b7fdef640 1 -- 172.21.15.115:0/3934869731 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6b600065d0 con 0x7f6b60002870 2024-01-24T23:21:38.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.243+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f6b700437d0 con 0x7f6b78130720 2024-01-24T23:21:38.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.243+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089347411 0 0) 0x7f6b600065d0 con 0x7f6b60002870 2024-01-24T23:21:38.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.243+0000 7f6b6a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.243+0000 7f6b6a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.243+0000 7f6b6a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.243+0000 7f6b6a7fc640 10 monclient(hunting): my global_id is 74587 2024-01-24T23:21:38.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.243+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b7812fbe0 con 0x7f6b60002870 2024-01-24T23:21:38.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.243+0000 7f6b7d38a640 10 monclient: get_auth_request con 0x7f6b5c03bc80 auth_method 0 2024-01-24T23:21:38.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.243+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4019724664 0 0) 0x7f6b7812fbe0 con 0x7f6b60002870 2024-01-24T23:21:38.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.243+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6b600065d0 con 0x7f6b60002870 2024-01-24T23:21:38.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.243+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6b6c004de0 con 0x7f6b60002870 2024-01-24T23:21:38.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.243+0000 7f6b6a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.243+0000 7f6b6a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:38.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.243+0000 7f6b6a7fc640 10 monclient: dump: 2024-01-24T23:21:38.265 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.265 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.265 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.265 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.265 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.265 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.266 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.266 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.266 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.266 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.266 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.266 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.266 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.266 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.266 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.266 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.243+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1296658982 0 0) 0x7f6b600065d0 con 0x7f6b60002870 2024-01-24T23:21:38.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.243+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 --> v1:172.21.15.115:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6b6000a120 con 0x7f6b60002870 2024-01-24T23:21:38.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.243+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 <== mon.4 v1:172.21.15.115:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f6b6c0071d0 con 0x7f6b60002870 2024-01-24T23:21:38.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.243+0000 7f6b6a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:38.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.243+0000 7f6b6a7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:38.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.243+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 >> v1:172.21.15.115:6791/0 conn(0x7f6b60002870 legacy=0x7f6b60004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.243+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 >> v1:172.21.15.115:6791/0 conn(0x7f6b60002870 legacy=0x7f6b60004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:38.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.247+0000 7f6b7fdef640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:21:38.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.247+0000 7f6b7fdef640 1 -- 172.21.15.115:0/3934869731 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6b60009e10 con 0x7f6b600090d0 2024-01-24T23:21:38.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.247+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4279738645 0 0) 0x7f6b60009e10 con 0x7f6b600090d0 2024-01-24T23:21:38.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.247+0000 7f6b6a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.247+0000 7f6b6a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.247+0000 7f6b6a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.247+0000 7f6b6a7fc640 10 monclient(hunting): my global_id is 74596 2024-01-24T23:21:38.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.247+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b600065d0 con 0x7f6b600090d0 2024-01-24T23:21:38.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.247+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4011301657 0 0) 0x7f6b600065d0 con 0x7f6b600090d0 2024-01-24T23:21:38.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.247+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6b60009e10 con 0x7f6b600090d0 2024-01-24T23:21:38.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.247+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6b6c0059c0 con 0x7f6b600090d0 2024-01-24T23:21:38.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.247+0000 7f6b6a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.267 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.251+0000 7f6b6a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:21:38.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.251+0000 7f6b6a7fc640 10 monclient: dump: 2024-01-24T23:21:38.268 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.268 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.268 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.268 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.268 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.268 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.268 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.268 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.268 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.268 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.268 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.268 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.268 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.268 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.268 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.269 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.251+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 159685124 0 0) 0x7f6b60009e10 con 0x7f6b600090d0 2024-01-24T23:21:38.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.251+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 --> v1:172.21.15.115:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6b6000c310 con 0x7f6b600090d0 2024-01-24T23:21:38.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.251+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 <== mon.4 v1:172.21.15.115:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7691 (unknown 0 0 3903328686) 0x7f6b6c00a190 con 0x7f6b600090d0 2024-01-24T23:21:38.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.251+0000 7f6b6a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:21:38.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.251+0000 7f6b6a7fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:38.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.251+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 >> v1:172.21.15.115:6791/0 conn(0x7f6b600090d0 legacy=0x7f6b60009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.251+0000 7f6b6a7fc640 1 -- 172.21.15.115:0/3934869731 >> v1:172.21.15.115:6791/0 conn(0x7f6b600090d0 legacy=0x7f6b60009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:38.269 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:21:38.269 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:21:38.269 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:21:38.269 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:21:38.270 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 12, 2024-01-24T23:21:38.270 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:21:38.270 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:21:38.270 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:21:38.270 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:21:38.270 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:21:38.270 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:21:38.270 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:21:38.270 INFO:teuthology.orchestra.run.smithi115.stdout: 7 2024-01-24T23:21:38.270 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.270 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:21:38.270 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:38.270 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:21:38.270 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:21:38.270 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:38.270 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:38.271 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:38.271 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:38.271 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:38.271 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:38.271 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:38.271 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:38.271 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:38.271 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:38.271 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:38.271 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.271 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:21:38.271 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:21:38.271 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:38.271 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:38.271 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:38.272 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:38.272 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:38.272 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:38.272 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:38.272 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:38.272 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:38.272 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:38.272 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:38.272 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.272 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:21:38.272 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:21:38.273 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:21:38.273 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:21:38.273 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:21:38.273 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:21:38.273 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:38.273 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:38.273 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:21:38.273 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:21:38.273 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:21:38.273 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:21:38.273 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:21:38.273 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:21:38.273 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:21:38.273 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:38.274 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:21:38.274 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:38.274 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:38.274 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:38.274 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:38.274 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:38.274 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:38.274 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:38.274 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:38.274 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:38.274 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:38.274 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:38.274 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.275 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:21:38.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.275 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:21:38.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.275 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:21:38.275 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:21:38.275 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.275 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.275 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.275 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:21:38.275 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.275 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.276 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.276 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:21:38.276 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:21:38.276 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.276 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.276 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.276 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:21:38.276 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:21:38.276 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.276 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.277 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.277 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:21:38.277 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.277 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.277 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.277 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:21:38.277 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:21:38.277 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.277 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.277 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.277 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:21:38.277 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:21:38.278 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.278 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.278 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.278 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:21:38.278 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.278 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.278 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.278 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:21:38.278 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:21:38.278 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.278 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.279 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.279 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:21:38.279 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:21:38.279 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.279 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.279 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.279 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:21:38.279 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.279 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.279 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.280 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:21:38.280 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:21:38.280 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.280 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.280 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.280 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:21:38.280 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:21:38.280 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.280 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.280 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.281 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:21:38.281 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.281 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.281 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.281 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:21:38.281 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:21:38.281 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.281 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.281 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.281 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:21:38.281 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:21:38.282 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.282 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.282 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.282 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:21:38.282 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.282 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.282 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.282 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:21:38.282 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:21:38.282 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.282 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.282 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.283 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:21:38.283 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:21:38.283 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.283 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.283 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.283 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:21:38.283 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.283 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.283 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.283 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:21:38.284 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:21:38.284 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.284 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.284 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.284 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:21:38.284 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:21:38.284 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.284 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.284 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.284 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:21:38.285 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.285 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.285 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.285 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:21:38.285 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:21:38.285 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.285 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.285 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.285 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:21:38.285 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:21:38.285 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.285 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.286 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.286 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:21:38.286 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.286 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.286 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.286 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:21:38.286 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:21:38.286 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.286 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.286 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.286 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.287 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.287 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:21:38.287 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:21:38.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.287 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:38.287 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:38.287 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:38.287 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.287 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.287 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:21:38.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.287 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:38.287 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:38.287 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:38.288 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.288 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.288 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:21:38.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.288 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:38.288 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:38.288 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:38.288 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.288 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.288 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:21:38.288 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:21:38.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.251+0000 7f6b7fdef640 1 -- 172.21.15.115:0/3934869731 >> v1:172.21.15.132:6812/1835401922 conn(0x7f6b5c03bc80 legacy=0x7f6b5c03e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.251+0000 7f6b7fdef640 10 monclient: shutdown 2024-01-24T23:21:38.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.251+0000 7f6b7fdef640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:38.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.251+0000 7f6b7fdef640 1 -- 172.21.15.115:0/3934869731 >> v1:172.21.15.115:6789/0 conn(0x7f6b78130720 legacy=0x7f6b781d1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.251+0000 7f6b7e38c640 1 -- 172.21.15.115:0/3934869731 reap_dead start 2024-01-24T23:21:38.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.251+0000 7f6b7e38c640 1 -- 172.21.15.115:0/3934869731 reap_dead start 2024-01-24T23:21:38.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.251+0000 7f6b7fdef640 1 -- 172.21.15.115:0/3934869731 shutdown_connections 2024-01-24T23:21:38.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.251+0000 7f6b7fdef640 1 -- 172.21.15.115:0/3934869731 >> 172.21.15.115:0/3934869731 conn(0x7f6b780a65e0 msgr2=0x7f6b78134130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:38.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.251+0000 7f6b7fdef640 1 -- 172.21.15.115:0/3934869731 shutdown_connections 2024-01-24T23:21:38.290 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.251+0000 7f6b7fdef640 1 -- 172.21.15.115:0/3934869731 wait complete. 2024-01-24T23:21:38.291 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-01-24T23:21:38.291 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon d 2024-01-24T23:21:38.291 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-01-24T23:21:38.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2faa11640 10 monclient: get_monmap_and_config 2024-01-24T23:21:38.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2faa11640 10 monclient: build_initial_monmap 2024-01-24T23:21:38.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2faa11640 10 monclient: monmap: 2024-01-24T23:21:38.406 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:38.406 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.407 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:38.407841+0000 2024-01-24T23:21:38.407 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:38.407841+0000 2024-01-24T23:21:38.407 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:38.407 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.407 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:38.407 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:38.407 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:38.407 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:38.407 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:38.408 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:38.408 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:38.408 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:38.408 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:38.408 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2faa11640 1 Processor -- start 2024-01-24T23:21:38.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2faa11640 1 -- start start 2024-01-24T23:21:38.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2faa11640 10 monclient: init 2024-01-24T23:21:38.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2faa11640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:38.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2faa11640 10 monclient: _add_conns ranks=[2,4,6,3,8,0,1,7,5] 2024-01-24T23:21:38.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2faa11640 10 monclient(hunting): picked mon.noname-b con 0x7fa2f409e740 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:38.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2faa11640 10 monclient(hunting): picked mon.noname-c con 0x7fa2f409f230 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:38.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2faa11640 10 monclient(hunting): picked mon.noname-d con 0x7fa2f4071e90 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:38.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2faa11640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2f404ed70 con 0x7fa2f409e740 2024-01-24T23:21:38.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2faa11640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2f40a1400 con 0x7fa2f409f230 2024-01-24T23:21:38.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2faa11640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2f40a37d0 con 0x7fa2f4071e90 2024-01-24T23:21:38.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2faa11640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:38.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2faa11640 10 monclient(hunting): authenticate will time out at 977.331030s 2024-01-24T23:21:38.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f3fff640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fa2f409e740 legacy=0x7fa2f409eb20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:21:38.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f37fe640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fa2f409f230 0x7fa2f4139b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:51692/0 (socket says 172.21.15.115:51692) 2024-01-24T23:21:38.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f37fe640 1 -- 172.21.15.115:0/1061250938 learned_addr learned my addr 172.21.15.115:0/1061250938 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:38.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 1 -- 172.21.15.115:0/1061250938 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4112883724 0 0) 0x7fa2f40a37d0 con 0x7fa2f4071e90 2024-01-24T23:21:38.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 10 monclient(hunting): my global_id is 4542 2024-01-24T23:21:38.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 1 -- 172.21.15.115:0/1061250938 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2dc003530 con 0x7fa2f4071e90 2024-01-24T23:21:38.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 1 -- 172.21.15.115:0/1061250938 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3759245078 0 0) 0x7fa2f40a1400 con 0x7fa2f409f230 2024-01-24T23:21:38.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 10 monclient(hunting): my global_id is 74605 2024-01-24T23:21:38.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 1 -- 172.21.15.115:0/1061250938 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2f40a37d0 con 0x7fa2f409f230 2024-01-24T23:21:38.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 1 -- 172.21.15.115:0/1061250938 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2215494230 0 0) 0x7fa2dc003530 con 0x7fa2f4071e90 2024-01-24T23:21:38.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 1 -- 172.21.15.115:0/1061250938 >> v1:172.21.15.115:6791/0 conn(0x7fa2f409f230 legacy=0x7fa2f4139b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 1 -- 172.21.15.115:0/1061250938 >> v1:172.21.15.115:6790/0 conn(0x7fa2f409e740 legacy=0x7fa2f409eb20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:21:38.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:21:38.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 1 monclient: found mon.noname-d 2024-01-24T23:21:38.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:38.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:21:38.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 1 -- 172.21.15.115:0/1061250938 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f40a3120 con 0x7fa2f4071e90 2024-01-24T23:21:38.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 10 monclient: _finish_auth 0 2024-01-24T23:21:38.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:21:38.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:38.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 1 -- 172.21.15.115:0/1061250938 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa2ec001b40 con 0x7fa2f4071e90 2024-01-24T23:21:38.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:21:38.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 10 monclient: dump: 2024-01-24T23:21:38.420 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.420 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.420 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.420 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.420 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.420 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.420 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.420 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.420 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.420 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.420 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.420 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.420 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.420 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.421 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.421 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2faa11640 5 monclient: authenticate success, global_id 4542 2024-01-24T23:21:38.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2faa11640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:38.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 1 -- 172.21.15.115:0/1061250938 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa2ec001e20 con 0x7fa2f4071e90 2024-01-24T23:21:38.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:38.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 1 -- 172.21.15.115:0/1061250938 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa2ec003ce0 con 0x7fa2f4071e90 2024-01-24T23:21:38.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:38.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.403+0000 7fa2f2ffd640 10 monclient: dump: 2024-01-24T23:21:38.421 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.421 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.421 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.421 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.422 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.422 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.422 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.422 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.422 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.422 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.422 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.422 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.422 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.422 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.422 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.422 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2faa11640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:38.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2faa11640 4 set_mon_vals no callback set 2024-01-24T23:21:38.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2faa11640 10 monclient: shutdown 2024-01-24T23:21:38.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2faa11640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:38.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2faa11640 1 -- 172.21.15.115:0/1061250938 >> v1:172.21.15.115:6792/0 conn(0x7fa2f4071e90 legacy=0x7fa2f413d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2faa11640 1 -- 172.21.15.115:0/1061250938 shutdown_connections 2024-01-24T23:21:38.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2faa11640 1 -- 172.21.15.115:0/1061250938 >> 172.21.15.115:0/1061250938 conn(0x7fa2f40666f0 msgr2=0x7fa2f4066b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:38.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2faa11640 1 -- 172.21.15.115:0/1061250938 shutdown_connections 2024-01-24T23:21:38.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2faa11640 1 -- 172.21.15.115:0/1061250938 wait complete. 2024-01-24T23:21:38.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2faa11640 10 monclient: build_initial_monmap 2024-01-24T23:21:38.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2faa11640 10 monclient: monmap: 2024-01-24T23:21:38.423 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:38.423 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:38.424 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:38.424 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:38.424 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:38.424 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:38.424 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:38.424 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:38.424 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:38.424 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:38.424 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:38.424 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:38.424 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:38.424 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:38.424 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:38.424 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2faa11640 1 Processor -- start 2024-01-24T23:21:38.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2faa11640 1 -- start start 2024-01-24T23:21:38.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2faa11640 10 monclient: init 2024-01-24T23:21:38.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2faa11640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:38.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2faa11640 10 monclient: _add_conns ranks=[7,5,1,3,6,2,0,4,8] 2024-01-24T23:21:38.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2faa11640 10 monclient(hunting): picked mon.noname-i con 0x7fa2f4071e90 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:38.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2faa11640 10 monclient(hunting): picked mon.noname-h con 0x7fa2f409e740 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:38.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2faa11640 10 monclient(hunting): picked mon.noname-f con 0x7fa2f41a9be0 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:38.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2faa11640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2f406c140 con 0x7fa2f41a9be0 2024-01-24T23:21:38.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2faa11640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2f41aab40 con 0x7fa2f409e740 2024-01-24T23:21:38.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2faa11640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2f41aae20 con 0x7fa2f4071e90 2024-01-24T23:21:38.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2faa11640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:38.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2faa11640 10 monclient(hunting): authenticate will time out at 977.334495s 2024-01-24T23:21:38.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f3fff640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fa2f4071e90 0x7fa2f41c7ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.115:46668/0 (socket says 172.21.15.115:46668) 2024-01-24T23:21:38.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f3fff640 1 -- 172.21.15.115:0/2387273627 learned_addr learned my addr 172.21.15.115:0/2387273627 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:38.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1498187500 0 0) 0x7fa2f41aab40 con 0x7fa2f409e740 2024-01-24T23:21:38.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 10 monclient(hunting): my global_id is 4478 2024-01-24T23:21:38.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2e0003530 con 0x7fa2f409e740 2024-01-24T23:21:38.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 490806662 0 0) 0x7fa2f41aae20 con 0x7fa2f4071e90 2024-01-24T23:21:38.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 10 monclient(hunting): my global_id is 4597 2024-01-24T23:21:38.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2f41aab40 con 0x7fa2f4071e90 2024-01-24T23:21:38.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2579156700 0 0) 0x7fa2f406c140 con 0x7fa2f41a9be0 2024-01-24T23:21:38.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 10 monclient(hunting): my global_id is 4519 2024-01-24T23:21:38.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2f41aae20 con 0x7fa2f41a9be0 2024-01-24T23:21:38.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 783853362 0 0) 0x7fa2f41aab40 con 0x7fa2f4071e90 2024-01-24T23:21:38.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa2f406c140 con 0x7fa2f4071e90 2024-01-24T23:21:38.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa2e4004280 con 0x7fa2f4071e90 2024-01-24T23:21:38.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:21:38.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 10 monclient(hunting): dump: 2024-01-24T23:21:38.427 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.428 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.428 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.428 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.428 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.428 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.428 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.428 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.428 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.428 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.428 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.428 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.428 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.428 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.428 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.428 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:38.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 354822944 0 0) 0x7fa2e0003530 con 0x7fa2f409e740 2024-01-24T23:21:38.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa2f41aab40 con 0x7fa2f409e740 2024-01-24T23:21:38.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa2e8001960 con 0x7fa2f409e740 2024-01-24T23:21:38.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:21:38.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 10 monclient(hunting): dump: 2024-01-24T23:21:38.429 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.429 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.429 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.429 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.429 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.429 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.429 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.429 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.430 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.430 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.430 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.430 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.430 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.430 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.430 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.430 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3475026489 0 0) 0x7fa2f41aae20 con 0x7fa2f41a9be0 2024-01-24T23:21:38.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.407+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa2e0003530 con 0x7fa2f41a9be0 2024-01-24T23:21:38.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa2ec003d60 con 0x7fa2f41a9be0 2024-01-24T23:21:38.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:21:38.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 10 monclient(hunting): dump: 2024-01-24T23:21:38.430 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.431 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.431 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.431 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.431 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.431 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.431 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.431 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.431 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.431 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.431 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.431 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.431 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.431 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.431 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.431 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1130686708 0 0) 0x7fa2f406c140 con 0x7fa2f4071e90 2024-01-24T23:21:38.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 >> v1:172.21.15.132:6791/0 conn(0x7fa2f409e740 legacy=0x7fa2f41bb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 >> v1:172.21.15.132:6789/0 conn(0x7fa2f41a9be0 legacy=0x7fa2f41d3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:21:38.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 1 monclient: found mon.i 2024-01-24T23:21:38.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:38.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:38.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f41ab100 con 0x7fa2f4071e90 2024-01-24T23:21:38.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 10 monclient: _finish_auth 0 2024-01-24T23:21:38.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:21:38.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:38.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2faa11640 5 monclient: authenticate success, global_id 4597 2024-01-24T23:21:38.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2faa11640 10 monclient: _renew_subs 2024-01-24T23:21:38.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2faa11640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:38.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2faa11640 1 -- 172.21.15.115:0/2387273627 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f41bc0f0 con 0x7fa2f4071e90 2024-01-24T23:21:38.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2faa11640 10 monclient: _renew_subs 2024-01-24T23:21:38.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2faa11640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:38.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2faa11640 1 -- 172.21.15.115:0/2387273627 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f41d4bf0 con 0x7fa2f4071e90 2024-01-24T23:21:38.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa2e4004850 con 0x7fa2f4071e90 2024-01-24T23:21:38.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:38.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f27fc640 4 set_mon_vals no callback set 2024-01-24T23:21:38.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa2e4004d80 con 0x7fa2f4071e90 2024-01-24T23:21:38.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:21:38.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 10 monclient: dump: 2024-01-24T23:21:38.433 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.433 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.433 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.433 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.434 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.434 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.434 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.434 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.434 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.434 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.434 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.434 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.434 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.434 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.434 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.434 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2faa11640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:38.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2faa11640 1 -- 172.21.15.115:0/2387273627 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa2d40065d0 con 0x7fa2d4002870 2024-01-24T23:21:38.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331064392 0 0) 0x7fa2d40065d0 con 0x7fa2d4002870 2024-01-24T23:21:38.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 10 monclient(hunting): my global_id is 4551 2024-01-24T23:21:38.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2f406c140 con 0x7fa2d4002870 2024-01-24T23:21:38.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2057389334 0 0) 0x7fa2f406c140 con 0x7fa2d4002870 2024-01-24T23:21:38.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa2d40065d0 con 0x7fa2d4002870 2024-01-24T23:21:38.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa2e8004b30 con 0x7fa2d4002870 2024-01-24T23:21:38.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:38.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 10 monclient: dump: 2024-01-24T23:21:38.436 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.436 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.436 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.436 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.436 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.436 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.436 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.436 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.436 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.436 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.436 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.437 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.437 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.437 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.437 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.437 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fa2e4004f60 con 0x7fa2f4071e90 2024-01-24T23:21:38.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7fa2e4043720 con 0x7fa2f4071e90 2024-01-24T23:21:38.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 414083325 0 0) 0x7fa2d40065d0 con 0x7fa2d4002870 2024-01-24T23:21:38.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 --> v1:172.21.15.115:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa2d400a120 con 0x7fa2d4002870 2024-01-24T23:21:38.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f8fae640 10 monclient: get_auth_request con 0x7fa2e0037fa0 auth_method 0 2024-01-24T23:21:38.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fa2e8007480 con 0x7fa2d4002870 2024-01-24T23:21:38.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:38.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:38.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 >> v1:172.21.15.115:6792/0 conn(0x7fa2d4002870 legacy=0x7fa2d4004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.411+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 >> v1:172.21.15.115:6792/0 conn(0x7fa2d4002870 legacy=0x7fa2d4004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:38.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2faa11640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:21:38.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2faa11640 1 -- 172.21.15.115:0/2387273627 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa2d4009de0 con 0x7fa2d40090a0 2024-01-24T23:21:38.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1482392967 0 0) 0x7fa2d4009de0 con 0x7fa2d40090a0 2024-01-24T23:21:38.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2f0ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2f0ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2f0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2f0ff9640 10 monclient(hunting): my global_id is 4560 2024-01-24T23:21:38.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2d40065d0 con 0x7fa2d40090a0 2024-01-24T23:21:38.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2114424117 0 0) 0x7fa2d40065d0 con 0x7fa2d40090a0 2024-01-24T23:21:38.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa2d4009de0 con 0x7fa2d40090a0 2024-01-24T23:21:38.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa2e8005640 con 0x7fa2d40090a0 2024-01-24T23:21:38.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2f0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2f0ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:38.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2f0ff9640 10 monclient: dump: 2024-01-24T23:21:38.440 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.440 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.440 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.440 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.440 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.440 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.440 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.441 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.441 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.441 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.441 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.441 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.441 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.441 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.441 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.441 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3331713539 0 0) 0x7fa2d4009de0 con 0x7fa2d40090a0 2024-01-24T23:21:38.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 --> v1:172.21.15.115:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa2d400c2d0 con 0x7fa2d40090a0 2024-01-24T23:21:38.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7861 (unknown 0 0 2744823142) 0x7fa2e8006b40 con 0x7fa2d40090a0 2024-01-24T23:21:38.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2f0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:21:38.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2f0ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:38.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 >> v1:172.21.15.115:6792/0 conn(0x7fa2d40090a0 legacy=0x7fa2d4009530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2f0ff9640 1 -- 172.21.15.115:0/2387273627 >> v1:172.21.15.115:6792/0 conn(0x7fa2d40090a0 legacy=0x7fa2d4009530 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:38.442 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:21:38.442 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:21:38.442 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:21:38.442 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:21:38.442 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 12, 2024-01-24T23:21:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:21:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:21:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:21:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:21:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:21:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:21:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:21:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: 7 2024-01-24T23:21:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:21:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:21:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:21:38.443 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:38.444 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:38.444 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:38.444 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:38.444 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:38.444 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:38.444 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:38.444 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:38.444 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:38.444 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:38.444 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:38.444 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.444 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:21:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:21:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:38.445 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:38.446 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.446 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:21:38.446 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:21:38.446 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:21:38.446 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:21:38.446 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:21:38.446 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:21:38.446 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:38.446 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:38.446 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:21:38.446 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:21:38.446 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:21:38.446 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:21:38.446 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:21:38.447 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:21:38.447 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:21:38.447 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:38.447 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:21:38.447 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:38.447 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:38.447 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:38.447 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:38.447 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:38.447 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:38.447 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:38.447 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:38.447 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:38.448 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:38.448 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:38.448 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.448 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:21:38.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.448 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:21:38.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.448 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:21:38.448 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:21:38.448 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.448 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.448 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.448 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:21:38.449 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.449 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.449 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.449 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:21:38.449 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:21:38.449 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.449 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.449 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.450 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:21:38.450 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:21:38.450 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.450 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.450 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.450 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:21:38.450 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.451 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.451 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.451 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:21:38.451 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:21:38.451 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.451 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.451 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.451 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:21:38.451 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:21:38.451 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.452 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.452 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.452 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:21:38.452 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.453 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.453 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.453 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:21:38.454 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:21:38.454 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.454 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.454 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.455 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:21:38.455 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:21:38.455 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.455 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.455 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.455 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:21:38.455 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.456 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.456 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.456 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:21:38.456 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:21:38.456 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.456 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.456 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.457 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:21:38.457 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:21:38.457 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.457 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.457 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.457 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:21:38.457 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.457 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.457 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.458 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:21:38.458 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:21:38.458 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.458 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.458 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.458 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:21:38.458 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:21:38.458 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.459 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.459 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.459 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:21:38.459 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.459 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.459 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.459 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:21:38.459 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:21:38.459 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.459 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.459 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.460 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:21:38.460 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:21:38.460 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.460 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.460 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.460 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:21:38.460 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.460 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.460 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.460 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:21:38.460 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:21:38.461 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.461 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.461 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.461 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:21:38.461 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:21:38.461 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.461 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.461 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.461 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:21:38.462 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.462 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.462 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.462 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:21:38.462 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:21:38.462 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.462 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.462 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.462 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:21:38.462 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:21:38.462 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.462 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.463 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.463 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:21:38.463 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.463 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.463 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.463 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:21:38.463 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:21:38.463 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.463 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.463 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.463 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.463 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.464 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:21:38.464 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:21:38.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.464 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:38.464 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:38.464 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:38.464 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.464 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.464 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:21:38.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.464 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:38.464 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:38.464 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:38.464 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.465 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.465 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:21:38.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.465 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:38.465 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:38.465 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:38.465 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.465 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.465 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": [ 2024-01-24T23:21:38.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.465 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:38.465 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:38.465 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:38.466 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.466 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.466 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:21:38.466 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:21:38.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2faa11640 1 -- 172.21.15.115:0/2387273627 >> v1:172.21.15.132:6812/1835401922 conn(0x7fa2e0037fa0 legacy=0x7fa2e003a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2faa11640 10 monclient: shutdown 2024-01-24T23:21:38.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2faa11640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:38.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2faa11640 1 -- 172.21.15.115:0/2387273627 >> v1:172.21.15.132:6792/0 conn(0x7fa2f4071e90 legacy=0x7fa2f41c7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2f8fae640 1 -- 172.21.15.115:0/2387273627 reap_dead start 2024-01-24T23:21:38.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2f8fae640 1 -- 172.21.15.115:0/2387273627 reap_dead start 2024-01-24T23:21:38.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2faa11640 1 -- 172.21.15.115:0/2387273627 shutdown_connections 2024-01-24T23:21:38.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2faa11640 1 -- 172.21.15.115:0/2387273627 >> 172.21.15.115:0/2387273627 conn(0x7fa2f40666f0 msgr2=0x7fa2f409c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:38.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2faa11640 1 -- 172.21.15.115:0/2387273627 shutdown_connections 2024-01-24T23:21:38.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.419+0000 7fa2faa11640 1 -- 172.21.15.115:0/2387273627 wait complete. 2024-01-24T23:21:38.468 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-01-24T23:21:38.469 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon f 2024-01-24T23:21:38.469 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-01-24T23:21:38.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 10 monclient: get_monmap_and_config 2024-01-24T23:21:38.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 10 monclient: build_initial_monmap 2024-01-24T23:21:38.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 10 monclient: monmap: 2024-01-24T23:21:38.583 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:38.583 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.584 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:38.583553+0000 2024-01-24T23:21:38.584 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:38.583553+0000 2024-01-24T23:21:38.584 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:38.584 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.584 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:38.584 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:38.585 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:38.585 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:38.585 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:38.585 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:38.585 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:38.585 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:38.586 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:38.586 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 1 Processor -- start 2024-01-24T23:21:38.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 1 -- start start 2024-01-24T23:21:38.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 10 monclient: init 2024-01-24T23:21:38.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:38.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 10 monclient: _add_conns ranks=[6,5,4,0,7,2,1,3,8] 2024-01-24T23:21:38.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 10 monclient(hunting): picked mon.noname-d con 0x7f7844135f90 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:38.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 10 monclient(hunting): picked mon.noname-h con 0x7f7844136370 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:38.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 10 monclient(hunting): picked mon.noname-c con 0x7f784413d500 addr v1:172.21.15.115:6791/0 2024-01-24T23:21:38.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f784404ed70 con 0x7f784413d500 2024-01-24T23:21:38.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7844071e30 con 0x7f7844136370 2024-01-24T23:21:38.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f784406b710 con 0x7f7844135f90 2024-01-24T23:21:38.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:38.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 10 monclient(hunting): authenticate will time out at 977.506128s 2024-01-24T23:21:38.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f784259c640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f7844135f90 0x7f78440721e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:38648/0 (socket says 172.21.15.115:38648) 2024-01-24T23:21:38.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f784259c640 1 -- 172.21.15.115:0/2136046310 learned_addr learned my addr 172.21.15.115:0/2136046310 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:38.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7842d9d640 1 --1- 172.21.15.115:0/2136046310 >> v1:172.21.15.115:6791/0 conn(0x7f784413d500 0x7f784413f8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:51694/0 (socket says 172.21.15.115:51694) 2024-01-24T23:21:38.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 1 -- 172.21.15.115:0/2136046310 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 729819114 0 0) 0x7f784406b710 con 0x7f7844135f90 2024-01-24T23:21:38.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 10 monclient(hunting): my global_id is 4569 2024-01-24T23:21:38.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 1 -- 172.21.15.115:0/2136046310 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7830003530 con 0x7f7844135f90 2024-01-24T23:21:38.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 1 -- 172.21.15.115:0/2136046310 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 10587752 0 0) 0x7f784404ed70 con 0x7f784413d500 2024-01-24T23:21:38.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 10 monclient(hunting): my global_id is 74614 2024-01-24T23:21:38.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 1 -- 172.21.15.115:0/2136046310 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f784406b710 con 0x7f784413d500 2024-01-24T23:21:38.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 1 -- 172.21.15.115:0/2136046310 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3447012373 0 0) 0x7f7844071e30 con 0x7f7844136370 2024-01-24T23:21:38.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 10 monclient(hunting): my global_id is 4487 2024-01-24T23:21:38.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 1 -- 172.21.15.115:0/2136046310 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f784404ed70 con 0x7f7844136370 2024-01-24T23:21:38.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 1 -- 172.21.15.115:0/2136046310 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1583193441 0 0) 0x7f7830003530 con 0x7f7844135f90 2024-01-24T23:21:38.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 1 -- 172.21.15.115:0/2136046310 >> v1:172.21.15.132:6791/0 conn(0x7f7844136370 legacy=0x7f784413bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 1 -- 172.21.15.115:0/2136046310 >> v1:172.21.15.115:6791/0 conn(0x7f784413d500 legacy=0x7f784413f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 10 monclient: _finish_hunting 0 2024-01-24T23:21:38.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 1 monclient: found mon.noname-d 2024-01-24T23:21:38.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:38.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:21:38.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 1 -- 172.21.15.115:0/2136046310 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784409ee20 con 0x7f7844135f90 2024-01-24T23:21:38.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 10 monclient: _finish_auth 0 2024-01-24T23:21:38.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 10 monclient: _check_auth_tickets 2024-01-24T23:21:38.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:38.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 5 monclient: authenticate success, global_id 4569 2024-01-24T23:21:38.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:38.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 1 -- 172.21.15.115:0/2136046310 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f782c0017c0 con 0x7f7844135f90 2024-01-24T23:21:38.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:21:38.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 10 monclient: dump: 2024-01-24T23:21:38.609 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.610 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.610 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.610 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.610 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.610 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.610 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.610 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.610 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.610 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.610 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.610 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.610 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.611 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.611 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.611 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:38.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 1 -- 172.21.15.115:0/2136046310 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f782c001aa0 con 0x7f7844135f90 2024-01-24T23:21:38.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:38.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 1 -- 172.21.15.115:0/2136046310 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f782c004bf0 con 0x7f7844135f90 2024-01-24T23:21:38.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:38.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 4 set_mon_vals no callback set 2024-01-24T23:21:38.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:38.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7841d9b640 10 monclient: dump: 2024-01-24T23:21:38.611 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.611 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.612 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.612 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.612 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.612 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.612 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.612 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.612 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.612 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.612 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.612 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.612 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.612 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.612 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.612 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 10 monclient: shutdown 2024-01-24T23:21:38.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:38.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 1 -- 172.21.15.115:0/2136046310 >> v1:172.21.15.115:6792/0 conn(0x7f7844135f90 legacy=0x7f78440721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 1 -- 172.21.15.115:0/2136046310 shutdown_connections 2024-01-24T23:21:38.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 1 -- 172.21.15.115:0/2136046310 >> 172.21.15.115:0/2136046310 conn(0x7f78440a1d50 msgr2=0x7f78440a2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:38.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 1 -- 172.21.15.115:0/2136046310 shutdown_connections 2024-01-24T23:21:38.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.579+0000 7f7848f63640 1 -- 172.21.15.115:0/2136046310 wait complete. 2024-01-24T23:21:38.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7848f63640 10 monclient: build_initial_monmap 2024-01-24T23:21:38.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7848f63640 10 monclient: monmap: 2024-01-24T23:21:38.614 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:38.614 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:38.614 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:38.614 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:38.614 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:38.614 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:38.614 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:38.614 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:38.614 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:38.614 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:38.615 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:38.615 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:38.615 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:38.615 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:38.615 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:38.615 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7848f63640 1 Processor -- start 2024-01-24T23:21:38.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7848f63640 1 -- start start 2024-01-24T23:21:38.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7848f63640 10 monclient: init 2024-01-24T23:21:38.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7848f63640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:38.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7848f63640 10 monclient: _add_conns ranks=[6,8,2,7,0,5,3,4,1] 2024-01-24T23:21:38.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7848f63640 10 monclient(hunting): picked mon.noname-d con 0x7f7844135f90 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:38.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7848f63640 10 monclient(hunting): picked mon.noname-e con 0x7f7844136370 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:38.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7848f63640 10 monclient(hunting): picked mon.noname-b con 0x7f784413d500 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:38.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7848f63640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f784413a8c0 con 0x7f784413d500 2024-01-24T23:21:38.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7848f63640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78441381a0 con 0x7f7844135f90 2024-01-24T23:21:38.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7848f63640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f784409f040 con 0x7f7844136370 2024-01-24T23:21:38.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7848f63640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:38.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7848f63640 10 monclient(hunting): authenticate will time out at 977.509098s 2024-01-24T23:21:38.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7842d9d640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f784413d500 legacy=0x7f78441d3aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:21:38.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783bfff640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f7844136370 legacy=0x7f78441cbee0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:21:38.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f784259c640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f7844135f90 0x7f78441cb7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:38658/0 (socket says 172.21.15.115:38658) 2024-01-24T23:21:38.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f784259c640 1 -- 172.21.15.115:0/2103252541 learned_addr learned my addr 172.21.15.115:0/2103252541 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:38.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1273060241 0 0) 0x7f78441381a0 con 0x7f7844135f90 2024-01-24T23:21:38.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 10 monclient(hunting): my global_id is 4578 2024-01-24T23:21:38.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7824003530 con 0x7f7844135f90 2024-01-24T23:21:38.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1683299875 0 0) 0x7f7824003530 con 0x7f7844135f90 2024-01-24T23:21:38.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f78441381a0 con 0x7f7844135f90 2024-01-24T23:21:38.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f782c004b30 con 0x7f7844135f90 2024-01-24T23:21:38.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:21:38.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 10 monclient(hunting): dump: 2024-01-24T23:21:38.617 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.617 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.617 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.617 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.617 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.617 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.618 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.618 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.618 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.618 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.618 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.618 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.618 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.618 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.618 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.618 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:38.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2182427633 0 0) 0x7f78441381a0 con 0x7f7844135f90 2024-01-24T23:21:38.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 >> v1:172.21.15.115:6793/0 conn(0x7f7844136370 legacy=0x7f78441cbee0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:21:38.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 >> v1:172.21.15.115:6790/0 conn(0x7f784413d500 legacy=0x7f78441d3aa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:21:38.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 10 monclient: _finish_hunting 0 2024-01-24T23:21:38.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 1 monclient: found mon.d 2024-01-24T23:21:38.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:38.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:38.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784413adf0 con 0x7f7844135f90 2024-01-24T23:21:38.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 10 monclient: _finish_auth 0 2024-01-24T23:21:38.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 10 monclient: _check_auth_tickets 2024-01-24T23:21:38.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:38.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7848f63640 5 monclient: authenticate success, global_id 4578 2024-01-24T23:21:38.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7848f63640 10 monclient: _renew_subs 2024-01-24T23:21:38.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7848f63640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:38.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7848f63640 1 -- 172.21.15.115:0/2103252541 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78441cc5f0 con 0x7f7844135f90 2024-01-24T23:21:38.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7848f63640 10 monclient: _renew_subs 2024-01-24T23:21:38.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7848f63640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:21:38.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7848f63640 1 -- 172.21.15.115:0/2103252541 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7844138d70 con 0x7f7844135f90 2024-01-24T23:21:38.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f782c004ec0 con 0x7f7844135f90 2024-01-24T23:21:38.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:38.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f782c005370 con 0x7f7844135f90 2024-01-24T23:21:38.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f784159a640 4 set_mon_vals no callback set 2024-01-24T23:21:38.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:38.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 10 monclient: dump: 2024-01-24T23:21:38.621 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.621 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.621 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.621 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.621 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.621 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.621 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.621 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.621 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.621 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.621 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.621 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.622 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.622 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.622 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.622 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7848f63640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:38.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7848f63640 1 -- 172.21.15.115:0/2103252541 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f78280065d0 con 0x7f7828002870 2024-01-24T23:21:38.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f782c00fdf0 con 0x7f7844135f90 2024-01-24T23:21:38.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f782c043940 con 0x7f7844135f90 2024-01-24T23:21:38.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f7842d9d640 10 monclient: get_auth_request con 0x7f7824036fb0 auth_method 0 2024-01-24T23:21:38.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2951994811 0 0) 0x7f78280065d0 con 0x7f7828002870 2024-01-24T23:21:38.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 10 monclient(hunting): my global_id is 4528 2024-01-24T23:21:38.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78441381a0 con 0x7f7828002870 2024-01-24T23:21:38.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.583+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 797975322 0 0) 0x7f78441381a0 con 0x7f7828002870 2024-01-24T23:21:38.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.587+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f78280065d0 con 0x7f7828002870 2024-01-24T23:21:38.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.587+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7834001870 con 0x7f7828002870 2024-01-24T23:21:38.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.587+0000 7f783affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.587+0000 7f783affd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:21:38.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.587+0000 7f783affd640 10 monclient: dump: 2024-01-24T23:21:38.623 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.623 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.623 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.623 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.624 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.624 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.624 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.624 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.624 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.624 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.624 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.624 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.624 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.624 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.624 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.624 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.587+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 105155693 0 0) 0x7f78280065d0 con 0x7f7828002870 2024-01-24T23:21:38.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.587+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 --> v1:172.21.15.132:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f782800a120 con 0x7f7828002870 2024-01-24T23:21:38.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.587+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 <== mon.1 v1:172.21.15.132:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f78340057d0 con 0x7f7828002870 2024-01-24T23:21:38.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.587+0000 7f783affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:38.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.587+0000 7f783affd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:38.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.587+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 >> v1:172.21.15.132:6789/0 conn(0x7f7828002870 legacy=0x7f7828004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.587+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 >> v1:172.21.15.132:6789/0 conn(0x7f7828002870 legacy=0x7f7828004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:38.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.591+0000 7f7848f63640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:21:38.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.591+0000 7f7848f63640 1 -- 172.21.15.115:0/2103252541 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7828009e10 con 0x7f78280090d0 2024-01-24T23:21:38.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4178724984 0 0) 0x7f7828009e10 con 0x7f78280090d0 2024-01-24T23:21:38.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f783affd640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f783affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f783affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f783affd640 10 monclient(hunting): my global_id is 4537 2024-01-24T23:21:38.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78280065d0 con 0x7f78280090d0 2024-01-24T23:21:38.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 875908533 0 0) 0x7f78280065d0 con 0x7f78280090d0 2024-01-24T23:21:38.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7828009e10 con 0x7f78280090d0 2024-01-24T23:21:38.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7834003bb0 con 0x7f78280090d0 2024-01-24T23:21:38.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f783affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f783affd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:21:38.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f783affd640 10 monclient: dump: 2024-01-24T23:21:38.626 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.626 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.626 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.626 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.626 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.626 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.626 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.626 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.626 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.627 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.627 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.627 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.627 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.627 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.627 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.627 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2946873821 0 0) 0x7f7828009e10 con 0x7f78280090d0 2024-01-24T23:21:38.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 --> v1:172.21.15.132:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f782800c310 con 0x7f78280090d0 2024-01-24T23:21:38.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 <== mon.1 v1:172.21.15.132:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7521 (unknown 0 0 581380960) 0x7f78340059a0 con 0x7f78280090d0 2024-01-24T23:21:38.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f783affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:21:38.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f783affd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:38.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 >> v1:172.21.15.132:6789/0 conn(0x7f78280090d0 legacy=0x7f7828009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f783affd640 1 -- 172.21.15.115:0/2103252541 >> v1:172.21.15.132:6789/0 conn(0x7f78280090d0 legacy=0x7f7828009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:38.628 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:21:38.628 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:21:38.628 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:21:38.628 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:21:38.628 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 12, 2024-01-24T23:21:38.628 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:21:38.628 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:21:38.628 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:21:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:21:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:21:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:21:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:21:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: 7 2024-01-24T23:21:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:21:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:21:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:21:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:21:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:21:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:38.631 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:38.631 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:38.631 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:38.631 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:38.631 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:38.631 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:38.631 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.632 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:21:38.632 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:21:38.632 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:21:38.632 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:21:38.632 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:21:38.632 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:21:38.633 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:38.633 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:38.633 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:21:38.633 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:21:38.633 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:21:38.633 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:21:38.633 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:21:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:21:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:21:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:21:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:38.635 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:38.635 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:38.635 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:38.635 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:38.635 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.635 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:21:38.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.635 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:21:38.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.635 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:21:38.635 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:21:38.635 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.635 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.636 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.636 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:21:38.636 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.636 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.636 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.636 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:21:38.636 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:21:38.636 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.636 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.636 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.636 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:21:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:21:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:21:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:21:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:21:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:21:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:21:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:21:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:21:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:21:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:21:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:21:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:21:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:21:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:21:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:21:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:21:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:21:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:21:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:21:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:21:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:21:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:21:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:21:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:21:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.643 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.643 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.643 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:21:38.643 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:21:38.644 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.644 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.644 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.644 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:21:38.644 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.645 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.645 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.645 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:21:38.645 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:21:38.645 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.646 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.646 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.646 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:21:38.646 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:21:38.647 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.647 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.647 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.647 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:21:38.647 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.648 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.648 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.648 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:21:38.648 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:21:38.648 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.649 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.649 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.649 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:21:38.650 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:21:38.650 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.650 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.650 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.650 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:21:38.650 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.651 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.651 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.651 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:21:38.651 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:21:38.651 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.651 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.652 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.652 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.652 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.652 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:21:38.652 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:21:38.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.653 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:38.653 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:38.653 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:38.653 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.653 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.653 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:21:38.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.654 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:38.654 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:38.654 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:21:38.654 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.654 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.655 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:21:38.655 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:21:38.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f7848f63640 1 -- 172.21.15.115:0/2103252541 >> v1:172.21.15.132:6812/1835401922 conn(0x7f7824036fb0 legacy=0x7f7824039470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f7848f63640 10 monclient: shutdown 2024-01-24T23:21:38.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f7848f63640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:38.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f7848f63640 1 -- 172.21.15.115:0/2103252541 >> v1:172.21.15.115:6792/0 conn(0x7f7844135f90 legacy=0x7f78441cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f7842d9d640 1 -- 172.21.15.115:0/2103252541 reap_dead start 2024-01-24T23:21:38.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f7842d9d640 1 -- 172.21.15.115:0/2103252541 reap_dead start 2024-01-24T23:21:38.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f7848f63640 1 -- 172.21.15.115:0/2103252541 shutdown_connections 2024-01-24T23:21:38.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f7848f63640 1 -- 172.21.15.115:0/2103252541 >> 172.21.15.115:0/2103252541 conn(0x7f78440a1d50 msgr2=0x7f7844131e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:38.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f7848f63640 1 -- 172.21.15.115:0/2103252541 shutdown_connections 2024-01-24T23:21:38.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.595+0000 7f7848f63640 1 -- 172.21.15.115:0/2103252541 wait complete. 2024-01-24T23:21:38.660 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-01-24T23:21:38.660 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon g 2024-01-24T23:21:38.660 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-01-24T23:21:38.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 10 monclient: get_monmap_and_config 2024-01-24T23:21:38.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 10 monclient: build_initial_monmap 2024-01-24T23:21:38.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 10 monclient: monmap: 2024-01-24T23:21:38.782 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:38.782 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.782 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:38.783427+0000 2024-01-24T23:21:38.782 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:38.783427+0000 2024-01-24T23:21:38.782 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:38.782 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.782 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:38.783 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:38.783 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:38.783 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:38.783 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:38.783 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:38.783 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:38.783 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:38.783 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:38.783 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 1 Processor -- start 2024-01-24T23:21:38.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 1 -- start start 2024-01-24T23:21:38.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 10 monclient: init 2024-01-24T23:21:38.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:38.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 10 monclient: _add_conns ranks=[1,8,6,7,3,2,4,5,0] 2024-01-24T23:21:38.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 10 monclient(hunting): picked mon.noname-f con 0x7f88a4133bf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:38.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 10 monclient(hunting): picked mon.noname-e con 0x7f88a4133fd0 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:38.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 10 monclient(hunting): picked mon.noname-d con 0x7f88a41343b0 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:38.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a404ed70 con 0x7f88a4133bf0 2024-01-24T23:21:38.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a40a4b60 con 0x7f88a41343b0 2024-01-24T23:21:38.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a409ae50 con 0x7f88a4133fd0 2024-01-24T23:21:38.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a3fff640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f88a4133fd0 legacy=0x7f88a413bd60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:21:38.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:38.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 10 monclient(hunting): authenticate will time out at 977.706190s 2024-01-24T23:21:38.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a9654640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f88a41343b0 0x7f88a413f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:38670/0 (socket says 172.21.15.115:38670) 2024-01-24T23:21:38.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a9654640 1 -- 172.21.15.115:0/62280774 learned_addr learned my addr 172.21.15.115:0/62280774 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:38.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 1 -- 172.21.15.115:0/62280774 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3391445340 0 0) 0x7f88a40a4b60 con 0x7f88a41343b0 2024-01-24T23:21:38.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 10 monclient(hunting): my global_id is 4587 2024-01-24T23:21:38.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 1 -- 172.21.15.115:0/62280774 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f888c003530 con 0x7f88a41343b0 2024-01-24T23:21:38.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 1 -- 172.21.15.115:0/62280774 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2003043709 0 0) 0x7f888c003530 con 0x7f88a41343b0 2024-01-24T23:21:38.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 1 -- 172.21.15.115:0/62280774 >> v1:172.21.15.115:6793/0 conn(0x7f88a4133fd0 legacy=0x7f88a413bd60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:21:38.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 1 -- 172.21.15.115:0/62280774 >> v1:172.21.15.132:6789/0 conn(0x7f88a4133bf0 legacy=0x7f88a409bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 10 monclient: _finish_hunting 0 2024-01-24T23:21:38.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 1 monclient: found mon.noname-d 2024-01-24T23:21:38.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:38.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:21:38.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 1 -- 172.21.15.115:0/62280774 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a409b130 con 0x7f88a41343b0 2024-01-24T23:21:38.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 10 monclient: _finish_auth 0 2024-01-24T23:21:38.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 10 monclient: _check_auth_tickets 2024-01-24T23:21:38.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:38.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 1 -- 172.21.15.115:0/62280774 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f889c001b40 con 0x7f88a41343b0 2024-01-24T23:21:38.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 5 monclient: authenticate success, global_id 4587 2024-01-24T23:21:38.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:38.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:21:38.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 10 monclient: dump: 2024-01-24T23:21:38.794 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.794 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.794 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.794 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.794 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.794 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.794 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.794 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.795 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.795 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.795 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.795 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.795 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.795 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.795 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.795 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:38.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 1 -- 172.21.15.115:0/62280774 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f889c001de0 con 0x7f88a41343b0 2024-01-24T23:21:38.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:38.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:38.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 4 set_mon_vals no callback set 2024-01-24T23:21:38.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 1 -- 172.21.15.115:0/62280774 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f889c004f70 con 0x7f88a41343b0 2024-01-24T23:21:38.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:38.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88a37fe640 10 monclient: dump: 2024-01-24T23:21:38.796 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.796 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.796 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.796 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.796 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.796 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.796 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.796 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.796 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.796 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.796 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.797 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.797 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.797 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.797 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.797 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 10 monclient: shutdown 2024-01-24T23:21:38.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:38.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 1 -- 172.21.15.115:0/62280774 >> v1:172.21.15.115:6792/0 conn(0x7f88a41343b0 legacy=0x7f88a413f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 1 -- 172.21.15.115:0/62280774 shutdown_connections 2024-01-24T23:21:38.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 1 -- 172.21.15.115:0/62280774 >> 172.21.15.115:0/62280774 conn(0x7f88a40662f0 msgr2=0x7f88a4066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:38.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 1 -- 172.21.15.115:0/62280774 shutdown_connections 2024-01-24T23:21:38.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.779+0000 7f88ab0b7640 1 -- 172.21.15.115:0/62280774 wait complete. 2024-01-24T23:21:38.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88ab0b7640 10 monclient: build_initial_monmap 2024-01-24T23:21:38.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88ab0b7640 10 monclient: monmap: 2024-01-24T23:21:38.798 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:38.798 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:38.798 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:38.798 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:38.798 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:38.798 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:38.798 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:38.798 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:38.798 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:38.798 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:38.798 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:38.798 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:38.798 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:38.798 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:38.799 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:38.799 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88ab0b7640 1 Processor -- start 2024-01-24T23:21:38.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88ab0b7640 1 -- start start 2024-01-24T23:21:38.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88ab0b7640 10 monclient: init 2024-01-24T23:21:38.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88ab0b7640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:38.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88ab0b7640 10 monclient: _add_conns ranks=[5,8,1,4,2,7,0,6,3] 2024-01-24T23:21:38.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88ab0b7640 10 monclient(hunting): picked mon.noname-h con 0x7f88a4133bf0 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:38.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88ab0b7640 10 monclient(hunting): picked mon.noname-e con 0x7f88a4133fd0 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:38.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88ab0b7640 10 monclient(hunting): picked mon.noname-f con 0x7f88a41343b0 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:38.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88ab0b7640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a4120a40 con 0x7f88a41343b0 2024-01-24T23:21:38.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88ab0b7640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a41bdf80 con 0x7f88a4133bf0 2024-01-24T23:21:38.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88ab0b7640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a41207d0 con 0x7f88a4133fd0 2024-01-24T23:21:38.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88ab0b7640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:38.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a3fff640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f88a4133fd0 legacy=0x7f88a41c7ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:21:38.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88ab0b7640 10 monclient(hunting): authenticate will time out at 977.709226s 2024-01-24T23:21:38.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a9654640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f88a41343b0 0x7f88a41d34a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.115:52920/0 (socket says 172.21.15.115:52920) 2024-01-24T23:21:38.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a9654640 1 -- 172.21.15.115:0/3803610402 learned_addr learned my addr 172.21.15.115:0/3803610402 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:38.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336682840 0 0) 0x7f8894001090 con 0x7f88a4133bf0 2024-01-24T23:21:38.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 10 monclient(hunting): my global_id is 4496 2024-01-24T23:21:38.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8890003530 con 0x7f88a4133bf0 2024-01-24T23:21:38.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1057107844 0 0) 0x7f88a4120a40 con 0x7f88a41343b0 2024-01-24T23:21:38.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 10 monclient(hunting): my global_id is 4555 2024-01-24T23:21:38.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8894001090 con 0x7f88a41343b0 2024-01-24T23:21:38.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3486074960 0 0) 0x7f8890003530 con 0x7f88a4133bf0 2024-01-24T23:21:38.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f88a4120a40 con 0x7f88a4133bf0 2024-01-24T23:21:38.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8894001b80 con 0x7f88a4133bf0 2024-01-24T23:21:38.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:21:38.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 10 monclient(hunting): dump: 2024-01-24T23:21:38.801 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.801 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.801 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.801 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.801 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.801 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.801 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.802 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.802 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.802 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.802 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.802 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.802 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.802 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.802 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.802 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:38.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3685313937 0 0) 0x7f8894001090 con 0x7f88a41343b0 2024-01-24T23:21:38.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8890003530 con 0x7f88a41343b0 2024-01-24T23:21:38.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f889c004eb0 con 0x7f88a41343b0 2024-01-24T23:21:38.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:21:38.803 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 10 monclient(hunting): dump: 2024-01-24T23:21:38.803 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.803 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.803 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.803 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.803 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.803 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.804 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.804 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.804 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.804 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.804 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.804 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.804 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.804 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.804 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.804 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3339029637 0 0) 0x7f88a4120a40 con 0x7f88a4133bf0 2024-01-24T23:21:38.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 >> v1:172.21.15.115:6793/0 conn(0x7f88a4133fd0 legacy=0x7f88a41c7ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:21:38.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 >> v1:172.21.15.132:6789/0 conn(0x7f88a41343b0 legacy=0x7f88a41d34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 10 monclient: _finish_hunting 0 2024-01-24T23:21:38.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 1 monclient: found mon.h 2024-01-24T23:21:38.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:38.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:21:38.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a4133790 con 0x7f88a4133bf0 2024-01-24T23:21:38.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 10 monclient: _finish_auth 0 2024-01-24T23:21:38.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 10 monclient: _check_auth_tickets 2024-01-24T23:21:38.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:38.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88ab0b7640 5 monclient: authenticate success, global_id 4496 2024-01-24T23:21:38.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88ab0b7640 10 monclient: _renew_subs 2024-01-24T23:21:38.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88ab0b7640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:21:38.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88ab0b7640 1 -- 172.21.15.115:0/3803610402 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a409c810 con 0x7f88a4133bf0 2024-01-24T23:21:38.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88ab0b7640 10 monclient: _renew_subs 2024-01-24T23:21:38.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88ab0b7640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:21:38.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88ab0b7640 1 -- 172.21.15.115:0/3803610402 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a41be5e0 con 0x7f88a4133bf0 2024-01-24T23:21:38.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f88940038d0 con 0x7f88a4133bf0 2024-01-24T23:21:38.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:38.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8894003e00 con 0x7f88a4133bf0 2024-01-24T23:21:38.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:21:38.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a2ffd640 4 set_mon_vals no callback set 2024-01-24T23:21:38.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88a17fa640 10 monclient: dump: 2024-01-24T23:21:38.806 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.806 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.806 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.807 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.807 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.807 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.807 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.807 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.807 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.807 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.807 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.807 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.807 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.807 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.807 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.807 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88ab0b7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:38.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.783+0000 7f88ab0b7640 1 -- 172.21.15.115:0/3803610402 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f88840065d0 con 0x7f8884002870 2024-01-24T23:21:38.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.787+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f8894003fe0 con 0x7f88a4133bf0 2024-01-24T23:21:38.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.787+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f8894042370 con 0x7f88a4133bf0 2024-01-24T23:21:38.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.787+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181558124 0 0) 0x7f88840065d0 con 0x7f8884002870 2024-01-24T23:21:38.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.787+0000 7f88a17fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.787+0000 7f88a17fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.787+0000 7f88a17fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.787+0000 7f88a17fa640 10 monclient(hunting): my global_id is 4512 2024-01-24T23:21:38.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.787+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88a4120a40 con 0x7f8884002870 2024-01-24T23:21:38.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.787+0000 7f88a9654640 10 monclient: get_auth_request con 0x7f8890037500 auth_method 0 2024-01-24T23:21:38.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.787+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2891462552 0 0) 0x7f88a4120a40 con 0x7f8884002870 2024-01-24T23:21:38.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.787+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f88840065d0 con 0x7f8884002870 2024-01-24T23:21:38.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.787+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8898004090 con 0x7f8884002870 2024-01-24T23:21:38.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.787+0000 7f88a17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.787+0000 7f88a17fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:21:38.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.787+0000 7f88a17fa640 10 monclient: dump: 2024-01-24T23:21:38.809 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.809 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.809 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.809 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.809 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.809 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.809 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.809 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.809 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.809 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.809 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.810 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.810 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.810 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.810 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.810 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.787+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3614395004 0 0) 0x7f88840065d0 con 0x7f8884002870 2024-01-24T23:21:38.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.787+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 --> v1:172.21.15.132:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f888400a120 con 0x7f8884002870 2024-01-24T23:21:38.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.787+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f88980063d0 con 0x7f8884002870 2024-01-24T23:21:38.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.787+0000 7f88a17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:38.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.787+0000 7f88a17fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:38.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.787+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 >> v1:172.21.15.132:6790/0 conn(0x7f8884002870 legacy=0x7f8884004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.787+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 >> v1:172.21.15.132:6790/0 conn(0x7f8884002870 legacy=0x7f8884004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:38.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88ab0b7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:21:38.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88ab0b7640 1 -- 172.21.15.115:0/3803610402 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8884009e10 con 0x7f88840090d0 2024-01-24T23:21:38.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2249024740 0 0) 0x7f8884009e10 con 0x7f88840090d0 2024-01-24T23:21:38.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88a17fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88a17fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88a17fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88a17fa640 10 monclient(hunting): my global_id is 4521 2024-01-24T23:21:38.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88840065d0 con 0x7f88840090d0 2024-01-24T23:21:38.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1383891043 0 0) 0x7f88840065d0 con 0x7f88840090d0 2024-01-24T23:21:38.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8884009e10 con 0x7f88840090d0 2024-01-24T23:21:38.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8898004ca0 con 0x7f88840090d0 2024-01-24T23:21:38.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88a17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88a17fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:21:38.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88a17fa640 10 monclient: dump: 2024-01-24T23:21:38.814 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.814 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.814 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.814 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.814 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.815 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.815 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.815 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.815 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.815 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.815 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.815 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.815 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.815 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.815 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.815 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2942095928 0 0) 0x7f8884009e10 con 0x7f88840090d0 2024-01-24T23:21:38.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 --> v1:172.21.15.132:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f888400c310 con 0x7f88840090d0 2024-01-24T23:21:38.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7521 (unknown 0 0 1499649090) 0x7f8898008d70 con 0x7f88840090d0 2024-01-24T23:21:38.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88a17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:21:38.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88a17fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:38.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 >> v1:172.21.15.132:6790/0 conn(0x7f88840090d0 legacy=0x7f8884009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88a17fa640 1 -- 172.21.15.115:0/3803610402 >> v1:172.21.15.132:6790/0 conn(0x7f88840090d0 legacy=0x7f8884009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:38.816 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:21:38.816 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:21:38.816 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:21:38.816 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:21:38.817 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 12, 2024-01-24T23:21:38.817 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:21:38.817 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:21:38.817 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:21:38.817 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:21:38.817 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:21:38.817 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:21:38.817 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:21:38.817 INFO:teuthology.orchestra.run.smithi115.stdout: 7 2024-01-24T23:21:38.817 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.818 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:21:38.818 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:38.818 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:21:38.818 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:21:38.818 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:38.818 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:38.818 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:38.818 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:38.818 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:21:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:21:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:38.819 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:21:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:21:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:21:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:21:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:21:38.820 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:21:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:21:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:21:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:21:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:21:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:21:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:21:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:21:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:21:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:38.821 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:21:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:21:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:21:38.822 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:21:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:21:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:21:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:21:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:21:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:21:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:21:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.824 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:21:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:21:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:21:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:21:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:21:38.825 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:21:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:21:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:21:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:21:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:21:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:21:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:21:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.827 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:21:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:21:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:21:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:21:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:21:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.828 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:21:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:21:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:21:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:21:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:21:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:21:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:21:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:21:38.830 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:21:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:21:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:21:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:21:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.832 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.832 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:21:38.832 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.832 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.832 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.832 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:21:38.832 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:21:38.832 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.832 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.832 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.832 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:21:38.832 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:21:38.833 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.833 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.833 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.833 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:21:38.833 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.833 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.833 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.833 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:21:38.833 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:21:38.833 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.833 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.833 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.833 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.834 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.834 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:21:38.834 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:21:38.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.834 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:38.834 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:38.834 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:38.834 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.834 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.834 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:21:38.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.834 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:38.834 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:38.834 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:38.835 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.835 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.835 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:21:38.835 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:21:38.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88ab0b7640 1 -- 172.21.15.115:0/3803610402 >> v1:172.21.15.132:6812/1835401922 conn(0x7f8890037500 legacy=0x7f88900399c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88ab0b7640 10 monclient: shutdown 2024-01-24T23:21:38.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88ab0b7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:38.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88ab0b7640 1 -- 172.21.15.115:0/3803610402 >> v1:172.21.15.132:6791/0 conn(0x7f88a4133bf0 legacy=0x7f88a4133080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88a9654640 1 -- 172.21.15.115:0/3803610402 reap_dead start 2024-01-24T23:21:38.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88a9654640 1 -- 172.21.15.115:0/3803610402 reap_dead start 2024-01-24T23:21:38.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88ab0b7640 1 -- 172.21.15.115:0/3803610402 shutdown_connections 2024-01-24T23:21:38.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88ab0b7640 1 -- 172.21.15.115:0/3803610402 >> 172.21.15.115:0/3803610402 conn(0x7f88a40662f0 msgr2=0x7f88a40a0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:38.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88ab0b7640 1 -- 172.21.15.115:0/3803610402 shutdown_connections 2024-01-24T23:21:38.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.795+0000 7f88ab0b7640 1 -- 172.21.15.115:0/3803610402 wait complete. 2024-01-24T23:21:38.837 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-01-24T23:21:38.837 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon h 2024-01-24T23:21:38.837 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-01-24T23:21:38.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.955+0000 7ff72587e640 10 monclient: get_monmap_and_config 2024-01-24T23:21:38.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.955+0000 7ff72587e640 10 monclient: build_initial_monmap 2024-01-24T23:21:38.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.955+0000 7ff72587e640 10 monclient: monmap: 2024-01-24T23:21:38.960 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:38.960 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.960 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:38.961225+0000 2024-01-24T23:21:38.960 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:38.961225+0000 2024-01-24T23:21:38.960 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:38.960 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.960 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:38.960 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:38.960 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:38.960 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:38.961 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:38.961 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:38.961 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:38.962 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:38.962 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:38.962 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.955+0000 7ff72587e640 1 Processor -- start 2024-01-24T23:21:38.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.955+0000 7ff72587e640 1 -- start start 2024-01-24T23:21:38.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.955+0000 7ff72587e640 10 monclient: init 2024-01-24T23:21:38.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.955+0000 7ff72587e640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:38.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.955+0000 7ff72587e640 10 monclient: _add_conns ranks=[0,3,6,2,4,7,1,5,8] 2024-01-24T23:21:38.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.955+0000 7ff72587e640 10 monclient(hunting): picked mon.noname-a con 0x7ff7200a2c40 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:38.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.955+0000 7ff72587e640 10 monclient(hunting): picked mon.noname-g con 0x7ff7200a3fd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:38.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.955+0000 7ff72587e640 10 monclient(hunting): picked mon.noname-d con 0x7ff7200a06b0 addr v1:172.21.15.115:6792/0 2024-01-24T23:21:38.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.955+0000 7ff72587e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff72004ed70 con 0x7ff7200a2c40 2024-01-24T23:21:38.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.955+0000 7ff72587e640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7200a0e70 con 0x7ff7200a3fd0 2024-01-24T23:21:38.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.955+0000 7ff72587e640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7200a3730 con 0x7ff7200a06b0 2024-01-24T23:21:38.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.955+0000 7ff72587e640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:38.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.955+0000 7ff72587e640 10 monclient(hunting): authenticate will time out at 977.883799s 2024-01-24T23:21:38.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.955+0000 7ff71effd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff7200a2c40 0x7ff7200a3020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39630/0 (socket says 172.21.15.115:39630) 2024-01-24T23:21:38.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.955+0000 7ff71effd640 1 -- 172.21.15.115:0/696452573 learned_addr learned my addr 172.21.15.115:0/696452573 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:38.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 1 -- 172.21.15.115:0/696452573 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 732332680 0 0) 0x7ff72004ed70 con 0x7ff7200a2c40 2024-01-24T23:21:38.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 10 monclient(hunting): my global_id is 4563 2024-01-24T23:21:38.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 1 -- 172.21.15.115:0/696452573 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff708003530 con 0x7ff7200a2c40 2024-01-24T23:21:38.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 1 -- 172.21.15.115:0/696452573 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3645829317 0 0) 0x7ff7200a3730 con 0x7ff7200a06b0 2024-01-24T23:21:38.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 10 monclient(hunting): my global_id is 4596 2024-01-24T23:21:38.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 1 -- 172.21.15.115:0/696452573 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff72004ed70 con 0x7ff7200a06b0 2024-01-24T23:21:38.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 1 -- 172.21.15.115:0/696452573 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2096270577 0 0) 0x7ff72004ed70 con 0x7ff7200a06b0 2024-01-24T23:21:38.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 1 -- 172.21.15.115:0/696452573 >> v1:172.21.15.132:6790/0 conn(0x7ff7200a3fd0 legacy=0x7ff72013be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 1 -- 172.21.15.115:0/696452573 >> v1:172.21.15.115:6789/0 conn(0x7ff7200a2c40 legacy=0x7ff7200a3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 10 monclient: _finish_hunting 0 2024-01-24T23:21:38.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 1 monclient: found mon.noname-d 2024-01-24T23:21:38.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:38.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:21:38.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 1 -- 172.21.15.115:0/696452573 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7200a1150 con 0x7ff7200a06b0 2024-01-24T23:21:38.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 10 monclient: _finish_auth 0 2024-01-24T23:21:38.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 10 monclient: _check_auth_tickets 2024-01-24T23:21:38.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:38.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 1 -- 172.21.15.115:0/696452573 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff718001b40 con 0x7ff7200a06b0 2024-01-24T23:21:38.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:21:38.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 10 monclient: dump: 2024-01-24T23:21:38.973 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.973 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.973 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.973 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.973 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.973 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.973 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.973 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.973 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.973 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.974 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.974 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.974 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.974 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.974 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.974 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 5 monclient: authenticate success, global_id 4596 2024-01-24T23:21:38.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:38.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 1 -- 172.21.15.115:0/696452573 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff718001d70 con 0x7ff7200a06b0 2024-01-24T23:21:38.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:38.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:38.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 4 set_mon_vals no callback set 2024-01-24T23:21:38.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 1 -- 172.21.15.115:0/696452573 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff7180039a0 con 0x7ff7200a06b0 2024-01-24T23:21:38.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:21:38.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71dffb640 10 monclient: dump: 2024-01-24T23:21:38.975 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.975 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.975 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.975 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.975 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.975 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.975 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.975 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.975 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.975 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.975 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.975 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.975 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.975 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.975 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.976 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 10 monclient: shutdown 2024-01-24T23:21:38.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:38.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 1 -- 172.21.15.115:0/696452573 >> v1:172.21.15.115:6792/0 conn(0x7ff7200a06b0 legacy=0x7ff72013f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 1 -- 172.21.15.115:0/696452573 shutdown_connections 2024-01-24T23:21:38.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 1 -- 172.21.15.115:0/696452573 >> 172.21.15.115:0/696452573 conn(0x7ff7200a9ba0 msgr2=0x7ff7200a5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:38.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 1 -- 172.21.15.115:0/696452573 shutdown_connections 2024-01-24T23:21:38.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 1 -- 172.21.15.115:0/696452573 wait complete. 2024-01-24T23:21:38.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 10 monclient: build_initial_monmap 2024-01-24T23:21:38.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 10 monclient: monmap: 2024-01-24T23:21:38.976 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:38.976 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:38.976 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:38.976 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:38.977 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:38.977 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:38.977 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:38.977 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:38.977 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:38.977 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:38.977 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:38.977 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:38.977 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:38.977 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:38.977 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:38.977 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 1 Processor -- start 2024-01-24T23:21:38.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 1 -- start start 2024-01-24T23:21:38.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 10 monclient: init 2024-01-24T23:21:38.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:38.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 10 monclient: _add_conns ranks=[0,5,2,7,6,4,8,3,1] 2024-01-24T23:21:38.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 10 monclient(hunting): picked mon.noname-a con 0x7ff7200a06b0 addr v1:172.21.15.115:6789/0 2024-01-24T23:21:38.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 10 monclient(hunting): picked mon.noname-h con 0x7ff7200a2c40 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:38.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 10 monclient(hunting): picked mon.noname-b con 0x7ff7200a3fd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:21:38.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff72013d550 con 0x7ff7200a06b0 2024-01-24T23:21:38.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff72013e5d0 con 0x7ff7200a3fd0 2024-01-24T23:21:38.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff72009fb50 con 0x7ff7200a2c40 2024-01-24T23:21:38.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:38.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 10 monclient(hunting): authenticate will time out at 977.886713s 2024-01-24T23:21:38.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71f7fe640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7ff7200a3fd0 legacy=0x7ff7201d3b10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:21:38.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71effd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff7200a06b0 0x7ff72013c050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39636/0 (socket says 172.21.15.115:39636) 2024-01-24T23:21:38.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71effd640 1 -- 172.21.15.115:0/2131067670 learned_addr learned my addr 172.21.15.115:0/2131067670 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:38.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3590227201 0 0) 0x7ff72013d550 con 0x7ff7200a06b0 2024-01-24T23:21:38.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 10 monclient(hunting): my global_id is 4572 2024-01-24T23:21:38.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff70c003530 con 0x7ff7200a06b0 2024-01-24T23:21:38.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3863108469 0 0) 0x7ff70c003530 con 0x7ff7200a06b0 2024-01-24T23:21:38.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff72013d550 con 0x7ff7200a06b0 2024-01-24T23:21:38.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff7100014a0 con 0x7ff7200a06b0 2024-01-24T23:21:38.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:21:38.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 10 monclient(hunting): dump: 2024-01-24T23:21:38.979 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.979 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.979 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.979 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.979 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.980 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.980 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.980 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.980 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.980 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.980 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.980 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.980 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.980 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.980 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.980 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:38.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 376289932 0 0) 0x7ff72009fb50 con 0x7ff7200a2c40 2024-01-24T23:21:38.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 10 monclient(hunting): my global_id is 4505 2024-01-24T23:21:38.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff70c003530 con 0x7ff7200a2c40 2024-01-24T23:21:38.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3085527592 0 0) 0x7ff72013d550 con 0x7ff7200a06b0 2024-01-24T23:21:38.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 >> v1:172.21.15.132:6791/0 conn(0x7ff7200a2c40 legacy=0x7ff72013c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 >> v1:172.21.15.115:6790/0 conn(0x7ff7200a3fd0 legacy=0x7ff7201d3b10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:21:38.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 10 monclient: _finish_hunting 0 2024-01-24T23:21:38.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 1 monclient: found mon.a 2024-01-24T23:21:38.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:38.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:21:38.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7200a58a0 con 0x7ff7200a06b0 2024-01-24T23:21:38.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 10 monclient: _finish_auth 0 2024-01-24T23:21:38.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 10 monclient: _check_auth_tickets 2024-01-24T23:21:38.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:38.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 5 monclient: authenticate success, global_id 4572 2024-01-24T23:21:38.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 10 monclient: _renew_subs 2024-01-24T23:21:38.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:21:38.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 1 -- 172.21.15.115:0/2131067670 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff72013ce70 con 0x7ff7200a06b0 2024-01-24T23:21:38.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 10 monclient: _renew_subs 2024-01-24T23:21:38.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:21:38.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff72587e640 1 -- 172.21.15.115:0/2131067670 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff72013ff30 con 0x7ff7200a06b0 2024-01-24T23:21:38.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff710004aa0 con 0x7ff7200a06b0 2024-01-24T23:21:38.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:38.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff710004f50 con 0x7ff7200a06b0 2024-01-24T23:21:38.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff71d7fa640 4 set_mon_vals no callback set 2024-01-24T23:21:38.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:21:38.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.959+0000 7ff707fff640 10 monclient: dump: 2024-01-24T23:21:38.983 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.983 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.983 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.983 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.983 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.983 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.983 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.983 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.983 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.983 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.983 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.983 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.983 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.983 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.983 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.984 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.963+0000 7ff72587e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:38.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.963+0000 7ff72587e640 1 -- 172.21.15.115:0/2131067670 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff7000065d0 con 0x7ff700002870 2024-01-24T23:21:38.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.963+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7ff71000f9d0 con 0x7ff7200a06b0 2024-01-24T23:21:38.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.963+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7ff7100436a0 con 0x7ff7200a06b0 2024-01-24T23:21:38.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.963+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828193904 0 0) 0x7ff7000065d0 con 0x7ff700002870 2024-01-24T23:21:38.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.963+0000 7ff707fff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.963+0000 7ff707fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.963+0000 7ff707fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.963+0000 7ff707fff640 10 monclient(hunting): my global_id is 4514 2024-01-24T23:21:38.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.963+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff72013d550 con 0x7ff700002870 2024-01-24T23:21:38.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.963+0000 7ff71f7fe640 10 monclient: get_auth_request con 0x7ff70c0370a0 auth_method 0 2024-01-24T23:21:38.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.963+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 429884694 0 0) 0x7ff72013d550 con 0x7ff700002870 2024-01-24T23:21:38.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.963+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff7000065d0 con 0x7ff700002870 2024-01-24T23:21:38.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.963+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff7140018a0 con 0x7ff700002870 2024-01-24T23:21:38.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.963+0000 7ff707fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.963+0000 7ff707fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:21:38.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.963+0000 7ff707fff640 10 monclient: dump: 2024-01-24T23:21:38.985 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.985 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.985 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.985 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.985 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.985 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.985 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.985 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.985 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.985 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.985 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.986 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.986 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.986 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.986 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.986 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.963+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3431975268 0 0) 0x7ff7000065d0 con 0x7ff700002870 2024-01-24T23:21:38.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.963+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 --> v1:172.21.15.132:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff70000a120 con 0x7ff700002870 2024-01-24T23:21:38.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.963+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7ff714003600 con 0x7ff700002870 2024-01-24T23:21:38.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.963+0000 7ff707fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:38.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.963+0000 7ff707fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:38.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.963+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 >> v1:172.21.15.132:6791/0 conn(0x7ff700002870 legacy=0x7ff700004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.963+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 >> v1:172.21.15.132:6791/0 conn(0x7ff700002870 legacy=0x7ff700004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:38.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff72587e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:21:38.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff72587e640 1 -- 172.21.15.115:0/2131067670 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff700009df0 con 0x7ff7000090b0 2024-01-24T23:21:38.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2524834861 0 0) 0x7ff700009df0 con 0x7ff7000090b0 2024-01-24T23:21:38.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff707fff640 10 monclient(hunting): _negotiate 2024-01-24T23:21:38.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff707fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:38.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff707fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:38.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff707fff640 10 monclient(hunting): my global_id is 4523 2024-01-24T23:21:38.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7000065d0 con 0x7ff7000090b0 2024-01-24T23:21:38.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 984932101 0 0) 0x7ff7000065d0 con 0x7ff7000090b0 2024-01-24T23:21:38.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff700009df0 con 0x7ff7000090b0 2024-01-24T23:21:38.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff714003ca0 con 0x7ff7000090b0 2024-01-24T23:21:38.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff707fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:38.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff707fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:21:38.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff707fff640 10 monclient: dump: 2024-01-24T23:21:38.989 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:38.989 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:38.989 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.989 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:38.989 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:38.989 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:38.989 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:38.989 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:38.989 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:38.989 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:38.989 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:38.989 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:38.989 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:38.990 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:38.990 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:38.990 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:38.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2547350902 0 0) 0x7ff700009df0 con 0x7ff7000090b0 2024-01-24T23:21:38.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 --> v1:172.21.15.132:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff70000c310 con 0x7ff7000090b0 2024-01-24T23:21:38.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7691 (unknown 0 0 58602123) 0x7ff714004180 con 0x7ff7000090b0 2024-01-24T23:21:38.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff707fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:21:38.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff707fff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:38.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 >> v1:172.21.15.132:6791/0 conn(0x7ff7000090b0 legacy=0x7ff700009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:38.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff707fff640 1 -- 172.21.15.115:0/2131067670 >> v1:172.21.15.132:6791/0 conn(0x7ff7000090b0 legacy=0x7ff700009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:38.990 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:21:38.991 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:21:38.991 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:21:38.991 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:21:38.991 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 12, 2024-01-24T23:21:38.991 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:21:38.991 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:21:38.991 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:21:38.991 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:21:38.991 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:21:38.991 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:21:38.991 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:21:38.991 INFO:teuthology.orchestra.run.smithi115.stdout: 7 2024-01-24T23:21:38.991 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.991 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 1, 2024-01-24T23:21:38.991 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:38.992 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:21:38.992 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:21:38.992 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:38.992 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:38.992 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:38.992 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:38.992 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:38.992 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:38.992 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:38.992 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:38.992 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:38.992 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:38.992 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:38.992 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.993 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:21:38.993 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:21:38.993 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:38.993 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:38.993 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:38.993 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:38.993 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:38.993 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:38.993 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:38.993 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:38.993 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:38.993 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:38.993 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:38.993 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.994 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:21:38.994 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:21:38.994 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:21:38.994 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:21:38.994 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:21:38.994 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:21:38.994 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:38.994 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:38.994 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:21:38.994 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:21:38.994 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:21:38.994 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:21:38.994 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:21:38.995 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:21:38.995 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:21:38.995 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:38.995 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:21:38.995 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:38.995 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:38.995 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:38.995 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:38.995 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:38.995 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:38.995 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:38.995 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:38.995 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:38.995 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:38.995 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:38.996 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:38.996 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:21:38.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.996 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:21:38.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.996 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:21:38.996 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:21:38.996 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.996 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.996 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.996 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:21:38.996 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.996 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.997 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.997 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:21:38.997 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:21:38.997 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.997 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.997 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.997 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:21:38.997 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:21:38.997 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.997 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.998 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.998 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:21:38.998 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.998 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.998 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.998 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:21:38.998 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:21:38.998 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.998 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.998 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:38.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.998 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:21:38.998 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:21:38.999 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:38.999 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:38.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:38.999 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:38.999 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:21:38.999 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:38.999 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:38.999 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:38.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:38.999 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:21:38.999 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:21:38.999 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:38.999 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:38.999 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:39.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.000 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:21:39.000 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:21:39.000 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:39.000 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:39.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.000 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:39.000 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:21:39.000 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:39.000 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:39.000 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:39.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.000 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:21:39.000 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:21:39.001 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:39.001 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:39.001 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:39.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.001 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:21:39.001 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:21:39.001 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:39.001 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:39.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.001 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:39.001 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:21:39.001 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:39.001 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:39.002 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:39.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.002 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:21:39.002 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:21:39.002 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:39.002 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:39.002 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:39.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.002 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:21:39.002 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:21:39.002 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:39.002 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:39.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.002 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:39.003 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:21:39.003 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:39.003 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:39.003 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:39.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.003 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:21:39.003 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:21:39.003 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:39.003 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:39.003 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:39.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.003 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:21:39.003 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:21:39.004 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:39.004 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:39.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.004 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:39.004 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:21:39.004 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:39.004 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:39.004 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:39.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.004 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:21:39.004 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:21:39.004 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:39.004 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:39.004 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:39.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.005 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:21:39.005 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:21:39.005 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:39.005 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:39.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.005 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:39.005 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:21:39.005 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:39.005 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:39.005 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:39.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.005 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:21:39.006 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:21:39.006 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:39.006 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:39.006 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:39.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.006 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:21:39.006 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:21:39.006 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:39.006 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:39.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.006 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:39.006 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:21:39.006 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:39.007 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:39.007 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:39.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.007 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:21:39.007 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:21:39.007 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:39.007 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:39.007 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:39.007 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:39.007 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:39.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.007 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:21:39.007 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:21:39.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.007 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:39.008 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:39.008 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:39.008 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:39.008 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:39.008 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:21:39.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.008 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:39.008 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:39.008 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:39.008 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:39.008 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:39.008 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:21:39.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.009 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:39.009 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:39.009 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:39.009 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:39.009 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:39.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.009 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:21:39.009 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:21:39.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff72587e640 1 -- 172.21.15.115:0/2131067670 >> v1:172.21.15.132:6812/1835401922 conn(0x7ff70c0370a0 legacy=0x7ff70c039560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:39.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff72587e640 10 monclient: shutdown 2024-01-24T23:21:39.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff72587e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:39.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff72587e640 1 -- 172.21.15.115:0/2131067670 >> v1:172.21.15.115:6789/0 conn(0x7ff7200a06b0 legacy=0x7ff72013c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:39.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff71f7fe640 1 -- 172.21.15.115:0/2131067670 reap_dead start 2024-01-24T23:21:39.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff71f7fe640 1 -- 172.21.15.115:0/2131067670 reap_dead start 2024-01-24T23:21:39.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff72587e640 1 -- 172.21.15.115:0/2131067670 shutdown_connections 2024-01-24T23:21:39.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff72587e640 1 -- 172.21.15.115:0/2131067670 >> 172.21.15.115:0/2131067670 conn(0x7ff7200a9ba0 msgr2=0x7ff720139cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:39.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff72587e640 1 -- 172.21.15.115:0/2131067670 shutdown_connections 2024-01-24T23:21:39.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:38.971+0000 7ff72587e640 1 -- 172.21.15.115:0/2131067670 wait complete. 2024-01-24T23:21:39.011 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-01-24T23:21:39.011 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon i 2024-01-24T23:21:39.012 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-01-24T23:21:39.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.119+0000 7f7953071640 10 monclient: get_monmap_and_config 2024-01-24T23:21:39.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.119+0000 7f7953071640 10 monclient: build_initial_monmap 2024-01-24T23:21:39.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.119+0000 7f7953071640 10 monclient: monmap: 2024-01-24T23:21:39.126 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:39.126 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:39.126 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:21:39.126502+0000 2024-01-24T23:21:39.126 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:21:39.126502+0000 2024-01-24T23:21:39.126 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:39.126 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:39.126 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:39.126 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:39.126 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:39.126 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:39.126 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:39.127 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:39.127 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:39.127 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:39.127 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:39.127 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:39.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 1 Processor -- start 2024-01-24T23:21:39.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 1 -- start start 2024-01-24T23:21:39.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 10 monclient: init 2024-01-24T23:21:39.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:39.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 10 monclient: _add_conns ranks=[3,8,1,5,4,2,7,6,0] 2024-01-24T23:21:39.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 10 monclient(hunting): picked mon.noname-g con 0x7f794c133c10 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:39.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 10 monclient(hunting): picked mon.noname-e con 0x7f794c136960 addr v1:172.21.15.115:6793/0 2024-01-24T23:21:39.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 10 monclient(hunting): picked mon.noname-f con 0x7f794c13a470 addr v1:172.21.15.132:6789/0 2024-01-24T23:21:39.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f794c04ed70 con 0x7f794c13a470 2024-01-24T23:21:39.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f794c09dee0 con 0x7f794c133c10 2024-01-24T23:21:39.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f794c09d400 con 0x7f794c136960 2024-01-24T23:21:39.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:39.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 10 monclient(hunting): authenticate will time out at 978.048748s 2024-01-24T23:21:39.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794bfff640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f794c136960 legacy=0x7f794c138d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:21:39.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7950e0d640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f794c133c10 0x7f794c133ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.115:49340/0 (socket says 172.21.15.115:49340) 2024-01-24T23:21:39.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7950e0d640 1 -- 172.21.15.115:0/332977803 learned_addr learned my addr 172.21.15.115:0/332977803 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:39.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f795160e640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f794c13a470 0x7f794c13c860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.115:52926/0 (socket says 172.21.15.115:52926) 2024-01-24T23:21:39.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 1 -- 172.21.15.115:0/332977803 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3879688600 0 0) 0x7f794c04ed70 con 0x7f794c13a470 2024-01-24T23:21:39.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:39.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:39.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:39.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 10 monclient(hunting): my global_id is 4564 2024-01-24T23:21:39.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 1 -- 172.21.15.115:0/332977803 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f793c005740 con 0x7f794c13a470 2024-01-24T23:21:39.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 1 -- 172.21.15.115:0/332977803 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1923555749 0 0) 0x7f794c09dee0 con 0x7f794c133c10 2024-01-24T23:21:39.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:21:39.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:39.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:39.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 10 monclient(hunting): my global_id is 4539 2024-01-24T23:21:39.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 1 -- 172.21.15.115:0/332977803 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f794c04ed70 con 0x7f794c133c10 2024-01-24T23:21:39.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 1 -- 172.21.15.115:0/332977803 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 365520352 0 0) 0x7f793c005740 con 0x7f794c13a470 2024-01-24T23:21:39.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 1 -- 172.21.15.115:0/332977803 >> v1:172.21.15.115:6793/0 conn(0x7f794c136960 legacy=0x7f794c138d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:21:39.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 1 -- 172.21.15.115:0/332977803 >> v1:172.21.15.132:6790/0 conn(0x7f794c133c10 legacy=0x7f794c133ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:39.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:21:39.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 1 monclient: found mon.noname-f 2024-01-24T23:21:39.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:39.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:21:39.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 1 -- 172.21.15.115:0/332977803 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794c09f380 con 0x7f794c13a470 2024-01-24T23:21:39.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 10 monclient: _finish_auth 0 2024-01-24T23:21:39.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:21:39.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:39.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 1 -- 172.21.15.115:0/332977803 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7944001b40 con 0x7f794c13a470 2024-01-24T23:21:39.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:39.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:21:39.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 10 monclient: dump: 2024-01-24T23:21:39.138 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:39.138 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:39.138 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:39.138 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:39.138 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:39.139 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:39.139 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:39.139 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:39.139 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:39.139 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:39.139 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:39.139 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:39.139 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:39.139 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:39.139 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:39.139 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:39.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 5 monclient: authenticate success, global_id 4564 2024-01-24T23:21:39.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:21:39.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 1 -- 172.21.15.115:0/332977803 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7944001d60 con 0x7f794c13a470 2024-01-24T23:21:39.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:39.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 10 monclient: get_monmap_and_config success 2024-01-24T23:21:39.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 4 set_mon_vals no callback set 2024-01-24T23:21:39.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 1 -- 172.21.15.115:0/332977803 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7944004eb0 con 0x7f794c13a470 2024-01-24T23:21:39.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:39.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:21:39.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f794b7fe640 10 monclient: dump: 2024-01-24T23:21:39.140 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:39.140 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:39.140 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:39.140 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:39.140 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:39.141 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:39.141 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:39.141 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:39.141 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:39.141 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:39.141 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:39.141 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:39.141 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:39.141 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:39.141 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:39.141 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:39.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 10 monclient: shutdown 2024-01-24T23:21:39.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:39.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 1 -- 172.21.15.115:0/332977803 >> v1:172.21.15.132:6789/0 conn(0x7f794c13a470 legacy=0x7f794c13c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:39.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 1 -- 172.21.15.115:0/332977803 shutdown_connections 2024-01-24T23:21:39.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 1 -- 172.21.15.115:0/332977803 >> 172.21.15.115:0/332977803 conn(0x7f794c0a5730 msgr2=0x7f794c0a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:39.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 1 -- 172.21.15.115:0/332977803 shutdown_connections 2024-01-24T23:21:39.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 1 -- 172.21.15.115:0/332977803 wait complete. 2024-01-24T23:21:39.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 10 monclient: build_initial_monmap 2024-01-24T23:21:39.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 10 monclient: monmap: 2024-01-24T23:21:39.142 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:21:39.142 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:21:39.142 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:21:39.142 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:21:39.142 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:21:39.143 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:21:39.143 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:21:39.143 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:21:39.143 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:21:39.143 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:21:39.143 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:21:39.143 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:21:39.143 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:21:39.143 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:21:39.143 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:21:39.143 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:39.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 1 Processor -- start 2024-01-24T23:21:39.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 1 -- start start 2024-01-24T23:21:39.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.123+0000 7f7953071640 10 monclient: init 2024-01-24T23:21:39.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f7953071640 10 monclient: _reopen_session rank -1 2024-01-24T23:21:39.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f7953071640 10 monclient: _add_conns ranks=[5,7,3,1,8,2,6,4,0] 2024-01-24T23:21:39.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f7953071640 10 monclient(hunting): picked mon.noname-h con 0x7f794c133c10 addr v1:172.21.15.132:6791/0 2024-01-24T23:21:39.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f7953071640 10 monclient(hunting): picked mon.noname-i con 0x7f794c136960 addr v1:172.21.15.132:6792/0 2024-01-24T23:21:39.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f7953071640 10 monclient(hunting): picked mon.noname-g con 0x7f794c13a470 addr v1:172.21.15.132:6790/0 2024-01-24T23:21:39.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f7953071640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f794c132070 con 0x7f794c13a470 2024-01-24T23:21:39.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f7953071640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f794c131c50 con 0x7f794c133c10 2024-01-24T23:21:39.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f7953071640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f794c139780 con 0x7f794c136960 2024-01-24T23:21:39.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f7953071640 10 monclient(hunting): _renew_subs 2024-01-24T23:21:39.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f7953071640 10 monclient(hunting): authenticate will time out at 978.052320s 2024-01-24T23:21:39.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f7950e0d640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f794c133c10 0x7f794c138960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.115:37186/0 (socket says 172.21.15.115:37186) 2024-01-24T23:21:39.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f7950e0d640 1 -- 172.21.15.115:0/733543256 learned_addr learned my addr 172.21.15.115:0/733543256 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:21:39.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 263136916 0 0) 0x7f794c131c50 con 0x7f794c133c10 2024-01-24T23:21:39.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:39.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:39.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:39.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): my global_id is 4532 2024-01-24T23:21:39.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7930003530 con 0x7f794c133c10 2024-01-24T23:21:39.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782302544 0 0) 0x7f794c132070 con 0x7f794c13a470 2024-01-24T23:21:39.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:39.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:39.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:39.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): my global_id is 4548 2024-01-24T23:21:39.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f794c131c50 con 0x7f794c13a470 2024-01-24T23:21:39.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4262830502 0 0) 0x7f794c139780 con 0x7f794c136960 2024-01-24T23:21:39.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:39.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:39.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:39.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): my global_id is 4606 2024-01-24T23:21:39.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f794c132070 con 0x7f794c136960 2024-01-24T23:21:39.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3736572817 0 0) 0x7f794c132070 con 0x7f794c136960 2024-01-24T23:21:39.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f794c139780 con 0x7f794c136960 2024-01-24T23:21:39.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7938004270 con 0x7f794c136960 2024-01-24T23:21:39.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:39.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:21:39.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): dump: 2024-01-24T23:21:39.147 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:39.147 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:39.147 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:39.147 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:39.147 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:39.147 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:39.147 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:39.147 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:39.147 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:39.147 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:39.147 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:39.147 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:39.147 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:39.148 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:39.148 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:39.148 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:39.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:21:39.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 770827942 0 0) 0x7f7930003530 con 0x7f794c133c10 2024-01-24T23:21:39.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f794c132070 con 0x7f794c133c10 2024-01-24T23:21:39.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7940001aa0 con 0x7f794c133c10 2024-01-24T23:21:39.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:39.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:21:39.148 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): dump: 2024-01-24T23:21:39.148 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:39.148 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:39.149 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:39.149 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:39.149 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:39.149 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:39.149 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:39.149 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:39.149 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:39.149 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:39.149 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:39.149 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:39.149 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:39.149 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:39.149 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:39.150 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:39.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1152518810 0 0) 0x7f794c131c50 con 0x7f794c13a470 2024-01-24T23:21:39.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7930003530 con 0x7f794c13a470 2024-01-24T23:21:39.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7944004eb0 con 0x7f794c13a470 2024-01-24T23:21:39.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:39.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:21:39.150 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): dump: 2024-01-24T23:21:39.150 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:39.150 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:39.150 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:39.150 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:39.150 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:39.151 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:39.151 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:39.151 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:39.151 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:39.151 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:39.151 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:39.151 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:39.151 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:39.151 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:39.151 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:39.151 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:39.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3273475095 0 0) 0x7f794c139780 con 0x7f794c136960 2024-01-24T23:21:39.151 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 >> v1:172.21.15.132:6791/0 conn(0x7f794c133c10 legacy=0x7f794c138960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:39.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 >> v1:172.21.15.132:6790/0 conn(0x7f794c13a470 legacy=0x7f794c1d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:39.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient: _finish_hunting 0 2024-01-24T23:21:39.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 monclient: found mon.i 2024-01-24T23:21:39.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:21:39.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:39.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794c0a1310 con 0x7f794c136960 2024-01-24T23:21:39.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient: _finish_auth 0 2024-01-24T23:21:39.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient: _check_auth_tickets 2024-01-24T23:21:39.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:21:39.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f7953071640 5 monclient: authenticate success, global_id 4606 2024-01-24T23:21:39.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f7953071640 10 monclient: _renew_subs 2024-01-24T23:21:39.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f7953071640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:39.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f7953071640 1 -- 172.21.15.115:0/733543256 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f794c133600 con 0x7f794c136960 2024-01-24T23:21:39.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f7953071640 10 monclient: _renew_subs 2024-01-24T23:21:39.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f7953071640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:21:39.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f7953071640 1 -- 172.21.15.115:0/733543256 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f794c130210 con 0x7f794c136960 2024-01-24T23:21:39.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7938004790 con 0x7f794c136960 2024-01-24T23:21:39.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:21:39.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7938004c40 con 0x7f794c136960 2024-01-24T23:21:39.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:39.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:21:39.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f794affd640 4 set_mon_vals no callback set 2024-01-24T23:21:39.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient: dump: 2024-01-24T23:21:39.153 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:39.154 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:39.154 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:39.154 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:39.154 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:39.154 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:39.154 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:39.154 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:39.154 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:39.154 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:39.154 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:39.154 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:39.154 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:39.154 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:39.155 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:39.155 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:39.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f7953071640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:39.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f7953071640 1 -- 172.21.15.115:0/733543256 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f792c0065d0 con 0x7f792c002870 2024-01-24T23:21:39.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f793800f740 con 0x7f794c136960 2024-01-24T23:21:39.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f7938043560 con 0x7f794c136960 2024-01-24T23:21:39.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1300279990 0 0) 0x7f792c0065d0 con 0x7f792c002870 2024-01-24T23:21:39.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:39.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:39.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:39.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 10 monclient(hunting): my global_id is 4615 2024-01-24T23:21:39.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.127+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f794c139780 con 0x7f792c002870 2024-01-24T23:21:39.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.131+0000 7f795160e640 10 monclient: get_auth_request con 0x7f7930037b30 auth_method 0 2024-01-24T23:21:39.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.131+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 765131107 0 0) 0x7f794c139780 con 0x7f792c002870 2024-01-24T23:21:39.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.131+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f792c0065d0 con 0x7f792c002870 2024-01-24T23:21:39.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.131+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7940004d20 con 0x7f792c002870 2024-01-24T23:21:39.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.131+0000 7f79497fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:39.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.131+0000 7f79497fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:21:39.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.131+0000 7f79497fa640 10 monclient: dump: 2024-01-24T23:21:39.156 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:39.156 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:39.156 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:39.156 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:39.156 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:39.157 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:39.157 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:39.157 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:39.157 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:39.157 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:39.157 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:39.157 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:39.157 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:39.157 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:39.157 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:39.157 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:39.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.131+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3709245361 0 0) 0x7f792c0065d0 con 0x7f792c002870 2024-01-24T23:21:39.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.131+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 --> v1:172.21.15.132:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f792c00a120 con 0x7f792c002870 2024-01-24T23:21:39.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.131+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 <== mon.7 v1:172.21.15.132:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f7940005150 con 0x7f792c002870 2024-01-24T23:21:39.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.131+0000 7f79497fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:21:39.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.131+0000 7f79497fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:21:39.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.131+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 >> v1:172.21.15.132:6792/0 conn(0x7f792c002870 legacy=0x7f792c004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:39.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.131+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 >> v1:172.21.15.132:6792/0 conn(0x7f792c002870 legacy=0x7f792c004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:39.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.135+0000 7f7953071640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:21:39.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.135+0000 7f7953071640 1 -- 172.21.15.115:0/733543256 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f792c009e10 con 0x7f792c0090d0 2024-01-24T23:21:39.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2905643105 0 0) 0x7f792c009e10 con 0x7f792c0090d0 2024-01-24T23:21:39.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f79497fa640 10 monclient(hunting): _negotiate 2024-01-24T23:21:39.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f79497fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:21:39.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f79497fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:21:39.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f79497fa640 10 monclient(hunting): my global_id is 4624 2024-01-24T23:21:39.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f792c0065d0 con 0x7f792c0090d0 2024-01-24T23:21:39.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3353397449 0 0) 0x7f792c0065d0 con 0x7f792c0090d0 2024-01-24T23:21:39.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f792c009e10 con 0x7f792c0090d0 2024-01-24T23:21:39.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f79400058b0 con 0x7f792c0090d0 2024-01-24T23:21:39.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f79497fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:21:39.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f79497fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:21:39.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f79497fa640 10 monclient: dump: 2024-01-24T23:21:39.160 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:21:39.160 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:21:39.160 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:39.160 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:21:39.161 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:21:39.161 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:21:39.161 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:21:39.161 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:21:39.161 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:21:39.161 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:21:39.161 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:21:39.161 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:21:39.161 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:21:39.161 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:21:39.161 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:21:39.161 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:21:39.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3786016767 0 0) 0x7f792c009e10 con 0x7f792c0090d0 2024-01-24T23:21:39.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 --> v1:172.21.15.132:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f792c00c310 con 0x7f792c0090d0 2024-01-24T23:21:39.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 <== mon.7 v1:172.21.15.132:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7691 (unknown 0 0 1633739519) 0x7f7940007be0 con 0x7f792c0090d0 2024-01-24T23:21:39.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f79497fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:21:39.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f79497fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:21:39.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 >> v1:172.21.15.132:6792/0 conn(0x7f792c0090d0 legacy=0x7f792c009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:39.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f79497fa640 1 -- 172.21.15.115:0/733543256 >> v1:172.21.15.132:6792/0 conn(0x7f792c0090d0 legacy=0x7f792c009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:21:39.163 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:21:39.163 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:21:39.163 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:21:39.163 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:21:39.163 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 12, 2024-01-24T23:21:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:21:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:21:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:21:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:21:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:21:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:21:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:21:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: 7 2024-01-24T23:21:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 1, 2024-01-24T23:21:39.164 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:21:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:21:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:39.165 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:21:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:21:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:39.166 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:21:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:21:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:21:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:21:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:21:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:21:39.167 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:21:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:21:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:21:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:21:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:21:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:21:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:21:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:21:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:21:39.168 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:21:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:21:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:21:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:21:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:21:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:21:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:21:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:21:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:21:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:21:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:21:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:21:39.169 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:21:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:21:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:21:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:21:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:21:39.170 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:21:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:21:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:21:39.171 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:21:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:21:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:21:39.172 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:21:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:21:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:21:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:39.173 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:21:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:21:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:21:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.174 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:21:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:21:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:21:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:21:39.175 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:21:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:21:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:21:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:39.176 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:21:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:21:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:21:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.177 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:21:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:21:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:21:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:21:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:21:39.178 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:21:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:21:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:39.179 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:21:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:21:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:21:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.180 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:21:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:21:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:21:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:21:39.181 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:21:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:21:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:21:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:21:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:21:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:21:39.182 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:21:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:21:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:21:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:21:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:21:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:21:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:21:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:39.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:21:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:21:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:21:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:21:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:39.184 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:21:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:21:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:21:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:21:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:21:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:21:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 4 2024-01-24T23:21:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:21:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:21:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:21:39.185 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:21:39.185 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:21:39.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f7953071640 1 -- 172.21.15.115:0/733543256 >> v1:172.21.15.132:6812/1835401922 conn(0x7f7930037b30 legacy=0x7f7930039ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:39.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f7953071640 10 monclient: shutdown 2024-01-24T23:21:39.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f7953071640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:21:39.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f7953071640 1 -- 172.21.15.115:0/733543256 >> v1:172.21.15.132:6792/0 conn(0x7f794c136960 legacy=0x7f794c139070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:21:39.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f795160e640 1 -- 172.21.15.115:0/733543256 reap_dead start 2024-01-24T23:21:39.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f795160e640 1 -- 172.21.15.115:0/733543256 reap_dead start 2024-01-24T23:21:39.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f7953071640 1 -- 172.21.15.115:0/733543256 shutdown_connections 2024-01-24T23:21:39.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f7953071640 1 -- 172.21.15.115:0/733543256 >> 172.21.15.115:0/733543256 conn(0x7f794c0a5730 msgr2=0x7f794c09cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:21:39.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f7953071640 1 -- 172.21.15.115:0/733543256 shutdown_connections 2024-01-24T23:21:39.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:21:39.139+0000 7f7953071640 1 -- 172.21.15.115:0/733543256 wait complete. 2024-01-24T23:21:39.188 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-01-24T23:21:39.188 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-01-24T23:22:27.941 INFO:tasks.workunit.client.0.smithi132.stderr:Note: switching to '663873555d45def955d1cc09730bfece2847b821'. 2024-01-24T23:22:27.942 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:27.942 INFO:tasks.workunit.client.0.smithi132.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-01-24T23:22:27.942 INFO:tasks.workunit.client.0.smithi132.stderr:changes and commit them, and you can discard any commits you make in this 2024-01-24T23:22:27.942 INFO:tasks.workunit.client.0.smithi132.stderr:state without impacting any branches by switching back to a branch. 2024-01-24T23:22:27.942 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:27.942 INFO:tasks.workunit.client.0.smithi132.stderr:If you want to create a new branch to retain commits you create, you may 2024-01-24T23:22:27.942 INFO:tasks.workunit.client.0.smithi132.stderr:do so (now or later) by using -c with the switch command. Example: 2024-01-24T23:22:27.942 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:27.943 INFO:tasks.workunit.client.0.smithi132.stderr: git switch -c 2024-01-24T23:22:27.943 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:27.943 INFO:tasks.workunit.client.0.smithi132.stderr:Or undo this operation with: 2024-01-24T23:22:27.943 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:27.943 INFO:tasks.workunit.client.0.smithi132.stderr: git switch - 2024-01-24T23:22:27.943 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:27.943 INFO:tasks.workunit.client.0.smithi132.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-01-24T23:22:27.943 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:27.943 INFO:tasks.workunit.client.0.smithi132.stderr:HEAD is now at 663873555d4 os/bluestore: check collection nullptr on deletion dummy onode 2024-01-24T23:22:27.945 DEBUG:teuthology.orchestra.run.smithi132:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-01-24T23:22:27.992 INFO:tasks.workunit.client.0.smithi132.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-01-24T23:22:27.995 INFO:tasks.workunit.client.0.smithi132.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-01-24T23:22:27.995 INFO:tasks.workunit.client.0.smithi132.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-01-24T23:22:28.746 INFO:tasks.workunit.client.0.smithi132.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-01-24T23:22:28.833 INFO:tasks.workunit.client.0.smithi132.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-01-24T23:22:28.888 INFO:tasks.workunit.client.0.smithi132.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-01-24T23:22:28.889 INFO:tasks.workunit.client.0.smithi132.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-01-24T23:22:28.889 INFO:tasks.workunit.client.0.smithi132.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-01-24T23:22:28.938 INFO:tasks.workunit.client.0.smithi132.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-01-24T23:22:28.941 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-01-24T23:22:28.942 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-01-24T23:22:28.986 INFO:tasks.workunit:Running workunits matching mon/pool_ops.sh on client.0... 2024-01-24T23:22:28.987 INFO:tasks.workunit:Running workunit mon/pool_ops.sh... 2024-01-24T23:22:28.988 DEBUG:teuthology.orchestra.run.smithi132:workunit test mon/pool_ops.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=663873555d45def955d1cc09730bfece2847b821 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/pool_ops.sh 2024-01-24T23:22:29.039 INFO:tasks.workunit.client.0.smithi132.stderr:+ TEST_POOL=testpool1234 2024-01-24T23:22:29.039 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool create testpool1234 8 --autoscale-mode off 2024-01-24T23:22:29.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.158+0000 7f4d67de0640 10 monclient: get_monmap_and_config 2024-01-24T23:22:29.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.158+0000 7f4d67de0640 10 monclient: build_initial_monmap 2024-01-24T23:22:29.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.158+0000 7f4d67de0640 10 monclient: monmap: 2024-01-24T23:22:29.162 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:29.162 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:29.162 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:29.164517+0000 2024-01-24T23:22:29.163 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:29.164517+0000 2024-01-24T23:22:29.163 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:29.163 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:29.163 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:29.163 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:29.163 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:29.163 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:29.163 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:29.163 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:29.163 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:29.163 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:29.163 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:29.163 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:29.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 1 Processor -- start 2024-01-24T23:22:29.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 1 -- start start 2024-01-24T23:22:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 10 monclient: init 2024-01-24T23:22:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 10 monclient: _add_conns ranks=[2,0,4,6,7,5,1,8,3] 2024-01-24T23:22:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 10 monclient(hunting): picked mon.noname-b con 0x7f4d60133ed0 addr v1:172.21.15.115:6790/0 2024-01-24T23:22:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 10 monclient(hunting): picked mon.noname-a con 0x7f4d601342b0 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 10 monclient(hunting): picked mon.noname-c con 0x7f4d60134690 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d60064700 con 0x7f4d601342b0 2024-01-24T23:22:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d6009b3d0 con 0x7f4d60133ed0 2024-01-24T23:22:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d600a58c0 con 0x7f4d60134690 2024-01-24T23:22:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 10 monclient(hunting): authenticate will time out at 1036.489276s 2024-01-24T23:22:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d65b7c640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f4d60133ed0 legacy=0x7f4d6009bdb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:22:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d6637d640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f4d60134690 0x7f4d6013f770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:54214/0 (socket says 172.21.15.132:54214) 2024-01-24T23:22:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d6637d640 1 -- 172.21.15.132:0/3773560924 learned_addr learned my addr 172.21.15.132:0/3773560924 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 1 -- 172.21.15.132:0/3773560924 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 499884843 0 0) 0x7f4d60064700 con 0x7f4d601342b0 2024-01-24T23:22:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 10 monclient(hunting): _negotiate 2024-01-24T23:22:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 10 monclient(hunting): my global_id is 4581 2024-01-24T23:22:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 1 -- 172.21.15.132:0/3773560924 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d4c003530 con 0x7f4d601342b0 2024-01-24T23:22:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 1 -- 172.21.15.132:0/3773560924 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 754877891 0 0) 0x7f4d600a58c0 con 0x7f4d60134690 2024-01-24T23:22:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 10 monclient(hunting): _negotiate 2024-01-24T23:22:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 10 monclient(hunting): my global_id is 74623 2024-01-24T23:22:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 1 -- 172.21.15.132:0/3773560924 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d60064700 con 0x7f4d60134690 2024-01-24T23:22:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 1 -- 172.21.15.132:0/3773560924 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4031791133 0 0) 0x7f4d4c003530 con 0x7f4d601342b0 2024-01-24T23:22:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 1 -- 172.21.15.132:0/3773560924 >> v1:172.21.15.115:6791/0 conn(0x7f4d60134690 legacy=0x7f4d6013f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 1 -- 172.21.15.132:0/3773560924 >> v1:172.21.15.115:6790/0 conn(0x7f4d60133ed0 legacy=0x7f4d6009bdb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 10 monclient: _finish_hunting 0 2024-01-24T23:22:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 1 monclient: found mon.noname-a 2024-01-24T23:22:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:22:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 1 -- 172.21.15.132:0/3773560924 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6009b0f0 con 0x7f4d601342b0 2024-01-24T23:22:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 10 monclient: _finish_auth 0 2024-01-24T23:22:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 10 monclient: _check_auth_tickets 2024-01-24T23:22:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 1 -- 172.21.15.132:0/3773560924 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4d5c0017c0 con 0x7f4d601342b0 2024-01-24T23:22:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:22:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 10 monclient: dump: 2024-01-24T23:22:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:29.175 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 5 monclient: authenticate success, global_id 4581 2024-01-24T23:22:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 1 -- 172.21.15.132:0/3773560924 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4d5c0019e0 con 0x7f4d601342b0 2024-01-24T23:22:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 1 -- 172.21.15.132:0/3773560924 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4d5c003960 con 0x7f4d601342b0 2024-01-24T23:22:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 4 set_mon_vals no callback set 2024-01-24T23:22:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:22:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d64b7a640 10 monclient: dump: 2024-01-24T23:22:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:29.176 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 10 monclient: shutdown 2024-01-24T23:22:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 1 -- 172.21.15.132:0/3773560924 >> v1:172.21.15.115:6789/0 conn(0x7f4d601342b0 legacy=0x7f4d6013c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 1 -- 172.21.15.132:0/3773560924 shutdown_connections 2024-01-24T23:22:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 1 -- 172.21.15.132:0/3773560924 >> 172.21.15.132:0/3773560924 conn(0x7f4d600a5ef0 msgr2=0x7f4d600a6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 1 -- 172.21.15.132:0/3773560924 shutdown_connections 2024-01-24T23:22:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 1 -- 172.21.15.132:0/3773560924 wait complete. 2024-01-24T23:22:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 10 monclient: build_initial_monmap 2024-01-24T23:22:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.162+0000 7f4d67de0640 10 monclient: monmap: 2024-01-24T23:22:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:29.178 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d67de0640 1 Processor -- start 2024-01-24T23:22:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d67de0640 1 -- start start 2024-01-24T23:22:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d67de0640 10 monclient: init 2024-01-24T23:22:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d67de0640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d67de0640 10 monclient: _add_conns ranks=[8,5,6,1,7,0,4,2,3] 2024-01-24T23:22:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d67de0640 10 monclient(hunting): picked mon.noname-e con 0x7f4d60133ed0 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d67de0640 10 monclient(hunting): picked mon.noname-h con 0x7f4d60134690 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d67de0640 10 monclient(hunting): picked mon.noname-d con 0x7f4d60133330 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d67de0640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d600a2cd0 con 0x7f4d60134690 2024-01-24T23:22:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d67de0640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d6009ca90 con 0x7f4d60133330 2024-01-24T23:22:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d67de0640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d601205e0 con 0x7f4d60133ed0 2024-01-24T23:22:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d67de0640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d67de0640 10 monclient(hunting): authenticate will time out at 1036.492849s 2024-01-24T23:22:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d65b7c640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f4d60133ed0 legacy=0x7f4d601cb060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d6537b640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f4d60134690 0x7f4d60132c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:58278/0 (socket says 172.21.15.132:58278) 2024-01-24T23:22:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d6537b640 1 -- 172.21.15.132:0/1528841163 learned_addr learned my addr 172.21.15.132:0/1528841163 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 1 -- 172.21.15.132:0/1528841163 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3222296229 0 0) 0x7f4d600a2cd0 con 0x7f4d60134690 2024-01-24T23:22:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 10 monclient(hunting): my global_id is 4541 2024-01-24T23:22:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 1 -- 172.21.15.132:0/1528841163 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d44003530 con 0x7f4d60134690 2024-01-24T23:22:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 1 -- 172.21.15.132:0/1528841163 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1755988199 0 0) 0x7f4d44003530 con 0x7f4d60134690 2024-01-24T23:22:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 1 -- 172.21.15.132:0/1528841163 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4d600a2cd0 con 0x7f4d60134690 2024-01-24T23:22:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 1 -- 172.21.15.132:0/1528841163 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4d5c001380 con 0x7f4d60134690 2024-01-24T23:22:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:22:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 10 monclient(hunting): dump: 2024-01-24T23:22:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:29.181 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 1 -- 172.21.15.132:0/1528841163 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2146096382 0 0) 0x7f4d6009ca90 con 0x7f4d60133330 2024-01-24T23:22:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 10 monclient(hunting): my global_id is 4605 2024-01-24T23:22:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 1 -- 172.21.15.132:0/1528841163 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d44003530 con 0x7f4d60133330 2024-01-24T23:22:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 1 -- 172.21.15.132:0/1528841163 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2555878852 0 0) 0x7f4d600a2cd0 con 0x7f4d60134690 2024-01-24T23:22:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 1 -- 172.21.15.132:0/1528841163 >> v1:172.21.15.115:6793/0 conn(0x7f4d60133ed0 legacy=0x7f4d601cb060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 1 -- 172.21.15.132:0/1528841163 >> v1:172.21.15.115:6792/0 conn(0x7f4d60133330 legacy=0x7f4d601d33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 10 monclient: _finish_hunting 0 2024-01-24T23:22:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 1 monclient: found mon.h 2024-01-24T23:22:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 1 -- 172.21.15.132:0/1528841163 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d600a5af0 con 0x7f4d60134690 2024-01-24T23:22:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 10 monclient: _finish_auth 0 2024-01-24T23:22:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 10 monclient: _check_auth_tickets 2024-01-24T23:22:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d67de0640 5 monclient: authenticate success, global_id 4541 2024-01-24T23:22:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d67de0640 10 monclient: _renew_subs 2024-01-24T23:22:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d67de0640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d67de0640 1 -- 172.21.15.132:0/1528841163 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d600a2340 con 0x7f4d60134690 2024-01-24T23:22:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d67de0640 10 monclient: _renew_subs 2024-01-24T23:22:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d67de0640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d67de0640 1 -- 172.21.15.132:0/1528841163 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6011fa40 con 0x7f4d60134690 2024-01-24T23:22:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 1 -- 172.21.15.132:0/1528841163 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4d5c003c70 con 0x7f4d60134690 2024-01-24T23:22:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d53fff640 4 set_mon_vals no callback set 2024-01-24T23:22:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 1 -- 172.21.15.132:0/1528841163 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4d5c0041a0 con 0x7f4d60134690 2024-01-24T23:22:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:22:29.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 10 monclient: dump: 2024-01-24T23:22:29.184 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:29.184 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:29.184 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:29.184 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:29.184 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:29.184 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:29.184 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:29.184 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:29.184 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:29.184 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:29.184 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:29.184 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:29.184 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:29.185 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:29.185 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:29.185 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:29.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d67de0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:29.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d67de0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:29.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d67de0640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:29.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d67de0640 1 -- 172.21.15.132:0/1528841163 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d48005090 con 0x7f4d60134690 2024-01-24T23:22:29.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 1 -- 172.21.15.132:0/1528841163 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f4d5c004380 con 0x7f4d60134690 2024-01-24T23:22:29.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d527fc640 1 -- 172.21.15.132:0/1528841163 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 3506+0+0 (unknown 720936730 0 0) 0x7f4d5c0439d0 con 0x7f4d60134690 2024-01-24T23:22:29.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.166+0000 7f4d65b7c640 10 monclient: get_auth_request con 0x7f4d44036ed0 auth_method 0 2024-01-24T23:22:29.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.170+0000 7f4d527fc640 1 -- 172.21.15.132:0/1528841163 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f4d5c014bd0 con 0x7f4d60134690 2024-01-24T23:22:29.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.170+0000 7f4d527fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:29.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.170+0000 7f4d527fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:29.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.330+0000 7f4d67de0640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}] 2024-01-24T23:22:29.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.330+0000 7f4d67de0640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}] 2024-01-24T23:22:29.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.330+0000 7f4d67de0640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:29.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.330+0000 7f4d67de0640 1 -- 172.21.15.132:0/1528841163 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"} v 0) v1 -- 0x7f4d480058c0 con 0x7f4d60134690 2024-01-24T23:22:29.983 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.982+0000 7f4d527fc640 1 -- 172.21.15.132:0/1528841163 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}]=0 pool 'testpool1234' created v16) v1 ==== 152+0+0 (unknown 3280004674 0 0) 0x7f4d5c013270 con 0x7f4d60134690 2024-01-24T23:22:29.984 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.982+0000 7f4d527fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}] 2024-01-24T23:22:29.984 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:29.982+0000 7f4d527fc640 10 monclient: _finish_command 2 = system:0 pool 'testpool1234' created 2024-01-24T23:22:30.079 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.078+0000 7f4d67de0640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}] 2024-01-24T23:22:30.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.078+0000 7f4d67de0640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}] 2024-01-24T23:22:30.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.078+0000 7f4d67de0640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:30.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.078+0000 7f4d67de0640 1 -- 172.21.15.132:0/1528841163 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"} v 0) v1 -- 0x7f4d480010a0 con 0x7f4d60134690 2024-01-24T23:22:30.081 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.078+0000 7f4d527fc640 1 -- 172.21.15.132:0/1528841163 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}]=0 pool 'testpool1234' already exists v16) v1 ==== 159+0+0 (unknown 39399137 0 0) 0x7f4d5c0149f0 con 0x7f4d60134690 2024-01-24T23:22:30.081 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.078+0000 7f4d527fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}] 2024-01-24T23:22:30.081 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.078+0000 7f4d527fc640 10 monclient: _finish_command 3 = system:0 pool 'testpool1234' already exists 2024-01-24T23:22:30.082 INFO:tasks.workunit.client.0.smithi132.stderr:pool 'testpool1234' already exists 2024-01-24T23:22:30.083 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.082+0000 7f4d67de0640 1 -- 172.21.15.132:0/1528841163 >> v1:172.21.15.132:6812/1835401922 conn(0x7f4d44036ed0 legacy=0x7f4d44039390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:30.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.082+0000 7f4d67de0640 10 monclient: shutdown 2024-01-24T23:22:30.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.082+0000 7f4d67de0640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:30.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.082+0000 7f4d67de0640 1 -- 172.21.15.132:0/1528841163 >> v1:172.21.15.132:6791/0 conn(0x7f4d60134690 legacy=0x7f4d60132c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:30.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.082+0000 7f4d67de0640 1 -- 172.21.15.132:0/1528841163 shutdown_connections 2024-01-24T23:22:30.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.082+0000 7f4d67de0640 1 -- 172.21.15.132:0/1528841163 >> 172.21.15.132:0/1528841163 conn(0x7f4d600a5ef0 msgr2=0x7f4d601bea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:30.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.082+0000 7f4d67de0640 1 -- 172.21.15.132:0/1528841163 shutdown_connections 2024-01-24T23:22:30.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.082+0000 7f4d67de0640 1 -- 172.21.15.132:0/1528841163 wait complete. 2024-01-24T23:22:30.099 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool set testpool1234 pg_num_min 2 2024-01-24T23:22:30.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.218+0000 7fe0b40f3640 10 monclient: get_monmap_and_config 2024-01-24T23:22:30.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.218+0000 7fe0b40f3640 10 monclient: build_initial_monmap 2024-01-24T23:22:30.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.218+0000 7fe0b40f3640 10 monclient: monmap: 2024-01-24T23:22:30.220 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:30.220 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:30.220 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:30.222347+0000 2024-01-24T23:22:30.221 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:30.222347+0000 2024-01-24T23:22:30.221 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:30.221 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:30.221 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:30.221 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:30.221 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:30.221 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:30.221 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:30.221 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:30.221 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:30.222 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:30.222 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:30.222 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:30.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.218+0000 7fe0b40f3640 1 Processor -- start 2024-01-24T23:22:30.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.218+0000 7fe0b40f3640 1 -- start start 2024-01-24T23:22:30.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.218+0000 7fe0b40f3640 10 monclient: init 2024-01-24T23:22:30.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.218+0000 7fe0b40f3640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:30.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.218+0000 7fe0b40f3640 10 monclient: _add_conns ranks=[1,6,4,3,8,0,7,5,2] 2024-01-24T23:22:30.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.218+0000 7fe0b40f3640 10 monclient(hunting): picked mon.noname-f con 0x7fe0ac0a3d10 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:30.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.218+0000 7fe0b40f3640 10 monclient(hunting): picked mon.noname-d con 0x7fe0ac0a40f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:30.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.218+0000 7fe0b40f3640 10 monclient(hunting): picked mon.noname-c con 0x7fe0ac13d010 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:30.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.218+0000 7fe0b40f3640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ac064700 con 0x7fe0ac0a3d10 2024-01-24T23:22:30.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.218+0000 7fe0b40f3640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ac09ab40 con 0x7fe0ac13d010 2024-01-24T23:22:30.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.218+0000 7fe0b40f3640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ac09b550 con 0x7fe0ac0a40f0 2024-01-24T23:22:30.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.218+0000 7fe0b40f3640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:30.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.218+0000 7fe0b40f3640 10 monclient(hunting): authenticate will time out at 1037.547551s 2024-01-24T23:22:30.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.218+0000 7fe0b1e8f640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fe0ac0a3d10 0x7fe0ac09bc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:49862/0 (socket says 172.21.15.132:49862) 2024-01-24T23:22:30.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.218+0000 7fe0b1e8f640 1 -- 172.21.15.132:0/2428881752 learned_addr learned my addr 172.21.15.132:0/2428881752 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:30.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.218+0000 7fe0b0e8d640 1 -- 172.21.15.132:0/2428881752 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2074979718 0 0) 0x7fe0ac064700 con 0x7fe0ac0a3d10 2024-01-24T23:22:30.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.218+0000 7fe0b0e8d640 10 monclient(hunting): _negotiate 2024-01-24T23:22:30.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.218+0000 7fe0b0e8d640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:30.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.218+0000 7fe0b0e8d640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:30.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.218+0000 7fe0b0e8d640 10 monclient(hunting): my global_id is 4573 2024-01-24T23:22:30.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.218+0000 7fe0b0e8d640 1 -- 172.21.15.132:0/2428881752 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe098003530 con 0x7fe0ac0a3d10 2024-01-24T23:22:30.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b0e8d640 1 -- 172.21.15.132:0/2428881752 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 868845006 0 0) 0x7fe098003530 con 0x7fe0ac0a3d10 2024-01-24T23:22:30.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b0e8d640 1 -- 172.21.15.132:0/2428881752 >> v1:172.21.15.115:6792/0 conn(0x7fe0ac0a40f0 legacy=0x7fe0ac0a4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:30.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b0e8d640 1 -- 172.21.15.132:0/2428881752 >> v1:172.21.15.115:6791/0 conn(0x7fe0ac13d010 legacy=0x7fe0ac13f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:30.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b0e8d640 10 monclient: _finish_hunting 0 2024-01-24T23:22:30.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b0e8d640 1 monclient: found mon.noname-f 2024-01-24T23:22:30.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b0e8d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:30.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b0e8d640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:22:30.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b0e8d640 1 -- 172.21.15.132:0/2428881752 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac09af90 con 0x7fe0ac0a3d10 2024-01-24T23:22:30.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b0e8d640 10 monclient: _finish_auth 0 2024-01-24T23:22:30.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b0e8d640 10 monclient: _check_auth_tickets 2024-01-24T23:22:30.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b0e8d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:30.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 5 monclient: authenticate success, global_id 4573 2024-01-24T23:22:30.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:30.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b0e8d640 1 -- 172.21.15.132:0/2428881752 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe0a80017c0 con 0x7fe0ac0a3d10 2024-01-24T23:22:30.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b0e8d640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:30.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b0e8d640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:22:30.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b0e8d640 10 monclient: dump: 2024-01-24T23:22:30.232 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:30.232 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:30.232 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:30.232 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:30.232 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:30.232 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:30.232 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:30.232 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:30.232 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:30.232 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:30.232 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:30.233 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:30.233 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:30.233 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:30.233 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:30.233 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:30.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:30.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b0e8d640 1 -- 172.21.15.132:0/2428881752 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe0a8001a70 con 0x7fe0ac0a3d10 2024-01-24T23:22:30.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b0e8d640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:30.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:30.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b0e8d640 1 -- 172.21.15.132:0/2428881752 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe0a8004bf0 con 0x7fe0ac0a3d10 2024-01-24T23:22:30.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 4 set_mon_vals no callback set 2024-01-24T23:22:30.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b0e8d640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:30.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b0e8d640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:22:30.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b0e8d640 10 monclient: dump: 2024-01-24T23:22:30.233 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:30.233 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:30.234 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:30.234 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:30.234 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:30.234 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:30.234 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:30.234 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:30.234 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:30.234 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:30.234 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:30.234 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:30.234 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:30.234 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:30.234 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:30.234 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:30.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 10 monclient: shutdown 2024-01-24T23:22:30.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:30.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 1 -- 172.21.15.132:0/2428881752 >> v1:172.21.15.132:6789/0 conn(0x7fe0ac0a3d10 legacy=0x7fe0ac09bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:30.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 1 -- 172.21.15.132:0/2428881752 shutdown_connections 2024-01-24T23:22:30.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 1 -- 172.21.15.132:0/2428881752 >> 172.21.15.132:0/2428881752 conn(0x7fe0ac0a9b10 msgr2=0x7fe0ac0a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:30.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 1 -- 172.21.15.132:0/2428881752 shutdown_connections 2024-01-24T23:22:30.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 1 -- 172.21.15.132:0/2428881752 wait complete. 2024-01-24T23:22:30.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 10 monclient: build_initial_monmap 2024-01-24T23:22:30.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 10 monclient: monmap: 2024-01-24T23:22:30.235 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:30.235 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:30.235 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:30.235 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:30.235 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:30.235 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:30.236 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:30.236 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:30.236 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:30.236 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:30.236 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:30.236 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:30.236 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:30.236 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:30.236 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:30.236 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:30.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 1 Processor -- start 2024-01-24T23:22:30.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 1 -- start start 2024-01-24T23:22:30.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 10 monclient: init 2024-01-24T23:22:30.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:30.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 10 monclient: _add_conns ranks=[6,7,8,2,1,3,4,5,0] 2024-01-24T23:22:30.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 10 monclient(hunting): picked mon.noname-d con 0x7fe0ac0a3d10 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:30.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 10 monclient(hunting): picked mon.noname-i con 0x7fe0ac0a40f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:30.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 10 monclient(hunting): picked mon.noname-e con 0x7fe0ac13d010 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:30.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ac0a5ad0 con 0x7fe0ac0a3d10 2024-01-24T23:22:30.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ac09d5b0 con 0x7fe0ac0a40f0 2024-01-24T23:22:30.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ac09dae0 con 0x7fe0ac13d010 2024-01-24T23:22:30.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:30.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 10 monclient(hunting): authenticate will time out at 1037.550375s 2024-01-24T23:22:30.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b2690640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7fe0ac13d010 legacy=0x7fe0ac1d3640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:30.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b168e640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fe0ac0a40f0 0x7fe0ac1362c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:54456/0 (socket says 172.21.15.132:54456) 2024-01-24T23:22:30.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b168e640 1 -- 172.21.15.132:0/2195503460 learned_addr learned my addr 172.21.15.132:0/2195503460 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:30.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187311439 0 0) 0x7fe0ac09d5b0 con 0x7fe0ac0a40f0 2024-01-24T23:22:30.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:30.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:30.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:30.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 10 monclient(hunting): my global_id is 4633 2024-01-24T23:22:30.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe090003530 con 0x7fe0ac0a40f0 2024-01-24T23:22:30.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 752904844 0 0) 0x7fe090003530 con 0x7fe0ac0a40f0 2024-01-24T23:22:30.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe0ac09d5b0 con 0x7fe0ac0a40f0 2024-01-24T23:22:30.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe09c001b60 con 0x7fe0ac0a40f0 2024-01-24T23:22:30.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:30.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:22:30.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 10 monclient(hunting): dump: 2024-01-24T23:22:30.238 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:30.239 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:30.239 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:30.239 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:30.239 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:30.239 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:30.239 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:30.239 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:30.239 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:30.239 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:30.239 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:30.239 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:30.240 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:30.240 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:30.240 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:30.240 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:30.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:30.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 941802808 0 0) 0x7fe0ac0a5ad0 con 0x7fe0ac0a3d10 2024-01-24T23:22:30.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:30.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:30.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:30.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 10 monclient(hunting): my global_id is 4623 2024-01-24T23:22:30.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe090003530 con 0x7fe0ac0a3d10 2024-01-24T23:22:30.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1535765869 0 0) 0x7fe0ac09d5b0 con 0x7fe0ac0a40f0 2024-01-24T23:22:30.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 >> v1:172.21.15.115:6793/0 conn(0x7fe0ac13d010 legacy=0x7fe0ac1d3640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:30.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 >> v1:172.21.15.115:6792/0 conn(0x7fe0ac0a3d10 legacy=0x7fe0ac135bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:30.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 10 monclient: _finish_hunting 0 2024-01-24T23:22:30.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 1 monclient: found mon.i 2024-01-24T23:22:30.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:30.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:30.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac0a3400 con 0x7fe0ac0a40f0 2024-01-24T23:22:30.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 10 monclient: _finish_auth 0 2024-01-24T23:22:30.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 10 monclient: _check_auth_tickets 2024-01-24T23:22:30.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:30.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 5 monclient: authenticate success, global_id 4633 2024-01-24T23:22:30.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 10 monclient: _renew_subs 2024-01-24T23:22:30.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:30.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 1 -- 172.21.15.132:0/2195503460 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0ac09d240 con 0x7fe0ac0a40f0 2024-01-24T23:22:30.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 10 monclient: _renew_subs 2024-01-24T23:22:30.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:30.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0b40f3640 1 -- 172.21.15.132:0/2195503460 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0ac09d000 con 0x7fe0ac0a40f0 2024-01-24T23:22:30.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe09c003950 con 0x7fe0ac0a40f0 2024-01-24T23:22:30.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:30.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a3fff640 4 set_mon_vals no callback set 2024-01-24T23:22:30.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe09c003e80 con 0x7fe0ac0a40f0 2024-01-24T23:22:30.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:30.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:22:30.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.222+0000 7fe0a27fc640 10 monclient: dump: 2024-01-24T23:22:30.244 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:30.244 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:30.244 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:30.244 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:30.244 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:30.245 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:30.245 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:30.245 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:30.245 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:30.245 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:30.245 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:30.245 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:30.245 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:30.245 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:30.245 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:30.245 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:30.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.226+0000 7fe0b40f3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:30.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.226+0000 7fe0b40f3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:30.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.226+0000 7fe0b40f3640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:30.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.226+0000 7fe0b40f3640 1 -- 172.21.15.132:0/2195503460 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe094005090 con 0x7fe0ac0a40f0 2024-01-24T23:22:30.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.226+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fe09c004060 con 0x7fe0ac0a40f0 2024-01-24T23:22:30.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.226+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3857+0+0 (unknown 871176060 0 0) 0x7fe09c043740 con 0x7fe0ac0a40f0 2024-01-24T23:22:30.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.226+0000 7fe0b1e8f640 10 monclient: get_auth_request con 0x7fe090036f80 auth_method 0 2024-01-24T23:22:30.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.230+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fe09c014930 con 0x7fe0ac0a40f0 2024-01-24T23:22:30.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.230+0000 7fe0a27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:30.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.230+0000 7fe0a27fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:30.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0b40f3640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-01-24T23:22:30.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0b40f3640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-01-24T23:22:30.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0b40f3640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:30.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0b40f3640 1 -- 172.21.15.132:0/2195503460 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"} v 0) v1 -- 0x7fe094005340 con 0x7fe0ac0a40f0 2024-01-24T23:22:30.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0b168e640 1 -- 172.21.15.132:0/2195503460 >> v1:172.21.15.132:6792/0 conn(0x7fe0ac0a40f0 legacy=0x7fe0ac1362c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-01-24T23:22:30.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0b168e640 1 -- 172.21.15.132:0/2195503460 >> v1:172.21.15.132:6792/0 conn(0x7fe0ac0a40f0 legacy=0x7fe0ac1362c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:22:30.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0b168e640 1 --1- 172.21.15.132:0/2195503460 >> v1:172.21.15.132:6792/0 conn(0x7fe0ac0a40f0 0x7fe0ac1362c0 :-1 s=OPENED pgs=121 cs=1 l=1).handle_message read tag failed 2024-01-24T23:22:30.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0b168e640 1 --1- 172.21.15.132:0/2195503460 >> v1:172.21.15.132:6792/0 conn(0x7fe0ac0a40f0 0x7fe0ac1362c0 :-1 s=OPENED pgs=121 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:22:30.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 10 monclient: ms_handle_reset current mon v1:172.21.15.132:6792/0 2024-01-24T23:22:30.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:30.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 >> v1:172.21.15.132:6792/0 conn(0x7fe0ac0a40f0 legacy=0x7fe0ac1362c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:22:30.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 10 monclient: _add_conns ranks=[4,8,3,0,7,6,2,5,1] 2024-01-24T23:22:30.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 10 monclient(hunting): picked mon.c con 0x7fe09003df30 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:30.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 10 monclient(hunting): picked mon.e con 0x7fe0ac13d010 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:30.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 10 monclient(hunting): picked mon.g con 0x7fe0900459c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:22:30.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe0ac09d5b0 con 0x7fe0900459c0 2024-01-24T23:22:30.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe0ac09dae0 con 0x7fe09003df30 2024-01-24T23:22:30.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe0ac0a5ad0 con 0x7fe0ac13d010 2024-01-24T23:22:30.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:30.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0b2690640 1 -- 172.21.15.132:0/2195503460 >> v1:172.21.15.115:6793/0 conn(0x7fe0ac13d010 legacy=0x7fe090043dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:30.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2574859968 0 0) 0x7fe0ac09d5b0 con 0x7fe0900459c0 2024-01-24T23:22:30.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:30.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:30.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-01-24T23:22:30.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 --> v1:172.21.15.132:6790/0 -- auth(proto 2 132 bytes epoch 0) v1 -- 0x7fe09004ab60 con 0x7fe0900459c0 2024-01-24T23:22:30.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1218482014 0 0) 0x7fe0ac09dae0 con 0x7fe09003df30 2024-01-24T23:22:30.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:30.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:30.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-01-24T23:22:30.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 --> v1:172.21.15.115:6791/0 -- auth(proto 2 132 bytes epoch 0) v1 -- 0x7fe0ac09d5b0 con 0x7fe09003df30 2024-01-24T23:22:30.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 233+0+0 (unknown 2535855850 0 0) 0x7fe09004ab60 con 0x7fe0900459c0 2024-01-24T23:22:30.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 >> v1:172.21.15.115:6793/0 conn(0x7fe0ac13d010 legacy=0x7fe090043dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:30.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 >> v1:172.21.15.115:6791/0 conn(0x7fe09003df30 legacy=0x7fe090040320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:30.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 10 monclient: _finish_hunting 0 2024-01-24T23:22:30.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 1 monclient: found mon.g 2024-01-24T23:22:30.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:30.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:30.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+}) v3 -- 0x7fe09004cbc0 con 0x7fe0900459c0 2024-01-24T23:22:30.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-01-24T23:22:30.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:30.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"} v 0) v1 -- 0x7fe0ac0a5ad0 con 0x7fe0900459c0 2024-01-24T23:22:30.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 10 monclient: _finish_auth 0 2024-01-24T23:22:30.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 10 monclient: _check_auth_tickets 2024-01-24T23:22:30.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:30.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe0a8001780 con 0x7fe0900459c0 2024-01-24T23:22:30.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:30.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:22:30.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 10 monclient: dump: 2024-01-24T23:22:30.394 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:30.394 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:30.394 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:30.394 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:30.394 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:30.394 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:30.394 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:30.394 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:30.394 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:30.394 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:30.394 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:30.395 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:30.395 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:30.395 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:30.395 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:30.395 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:30.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe0a8004b00 con 0x7fe0900459c0 2024-01-24T23:22:30.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:30.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a3fff640 4 set_mon_vals no callback set 2024-01-24T23:22:30.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.386+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fe0a8005ed0 con 0x7fe0900459c0 2024-01-24T23:22:30.987 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.982+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}]=0 set pool 3 pg_num_min to 2 v17) v1 ==== 143+0+0 (unknown 1752662139 0 0) 0x7fe0a80167f0 con 0x7fe0900459c0 2024-01-24T23:22:30.987 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.982+0000 7fe0a27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-01-24T23:22:30.987 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:30.982+0000 7fe0a27fc640 10 monclient: _finish_command 2 = system:0 set pool 3 pg_num_min to 2 2024-01-24T23:22:31.086 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:31.082+0000 7fe0b40f3640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-01-24T23:22:31.086 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:31.082+0000 7fe0b40f3640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-01-24T23:22:31.086 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:31.082+0000 7fe0b40f3640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:31.086 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:31.082+0000 7fe0b40f3640 1 -- 172.21.15.132:0/2195503460 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"} v 0) v1 -- 0x7fe0940023f0 con 0x7fe0900459c0 2024-01-24T23:22:32.000 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:31.998+0000 7fe0a27fc640 1 -- 172.21.15.132:0/2195503460 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}]=0 set pool 3 pg_num_min to 2 v18) v1 ==== 143+0+0 (unknown 706762250 0 0) 0x7fe0a80152d0 con 0x7fe0900459c0 2024-01-24T23:22:32.000 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:31.998+0000 7fe0a27fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-01-24T23:22:32.000 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:31.998+0000 7fe0a27fc640 10 monclient: _finish_command 3 = system:0 set pool 3 pg_num_min to 2 2024-01-24T23:22:32.000 INFO:tasks.workunit.client.0.smithi132.stderr:set pool 3 pg_num_min to 2 2024-01-24T23:22:32.003 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.002+0000 7fe0b40f3640 1 -- 172.21.15.132:0/2195503460 >> v1:172.21.15.132:6812/1835401922 conn(0x7fe090036f80 legacy=0x7fe090039440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:32.003 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.002+0000 7fe0b40f3640 10 monclient: shutdown 2024-01-24T23:22:32.003 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.002+0000 7fe0b40f3640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:32.003 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.002+0000 7fe0b40f3640 1 -- 172.21.15.132:0/2195503460 >> v1:172.21.15.132:6790/0 conn(0x7fe0900459c0 legacy=0x7fe090047e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:32.003 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.002+0000 7fe0b2690640 1 -- 172.21.15.132:0/2195503460 reap_dead start 2024-01-24T23:22:32.003 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.002+0000 7fe0b40f3640 1 -- 172.21.15.132:0/2195503460 shutdown_connections 2024-01-24T23:22:32.003 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.002+0000 7fe0b40f3640 1 -- 172.21.15.132:0/2195503460 >> 172.21.15.132:0/2195503460 conn(0x7fe0ac0a9b10 msgr2=0x7fe0ac066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:32.004 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.002+0000 7fe0b40f3640 1 -- 172.21.15.132:0/2195503460 shutdown_connections 2024-01-24T23:22:32.004 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.002+0000 7fe0b40f3640 1 -- 172.21.15.132:0/2195503460 wait complete. 2024-01-24T23:22:32.022 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool get testpool1234 pg_num_min 2024-01-24T23:22:32.022 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep 2 2024-01-24T23:22:32.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.138+0000 7f6c737b5640 10 monclient: get_monmap_and_config 2024-01-24T23:22:32.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.138+0000 7f6c737b5640 10 monclient: build_initial_monmap 2024-01-24T23:22:32.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.138+0000 7f6c737b5640 10 monclient: monmap: 2024-01-24T23:22:32.141 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:32.141 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:32.141 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:32.143225+0000 2024-01-24T23:22:32.141 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:32.143225+0000 2024-01-24T23:22:32.141 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:32.141 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:32.141 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:32.141 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:32.141 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:32.141 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:32.142 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:32.142 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:32.142 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:32.142 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:32.142 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:32.142 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:32.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.138+0000 7f6c737b5640 1 Processor -- start 2024-01-24T23:22:32.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.138+0000 7f6c737b5640 1 -- start start 2024-01-24T23:22:32.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.138+0000 7f6c737b5640 10 monclient: init 2024-01-24T23:22:32.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.138+0000 7f6c737b5640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:32.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.138+0000 7f6c737b5640 10 monclient: _add_conns ranks=[5,2,4,3,7,8,1,6,0] 2024-01-24T23:22:32.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.138+0000 7f6c737b5640 10 monclient(hunting): picked mon.noname-h con 0x7f6c6c09bc10 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:32.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.138+0000 7f6c737b5640 10 monclient(hunting): picked mon.noname-b con 0x7f6c6c0a00d0 addr v1:172.21.15.115:6790/0 2024-01-24T23:22:32.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 10 monclient(hunting): picked mon.noname-c con 0x7f6c6c0a0560 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:32.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c6c064700 con 0x7f6c6c0a00d0 2024-01-24T23:22:32.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c6c09bff0 con 0x7f6c6c0a0560 2024-01-24T23:22:32.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c6c0a1870 con 0x7f6c6c09bc10 2024-01-24T23:22:32.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:32.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 10 monclient(hunting): authenticate will time out at 1039.468135s 2024-01-24T23:22:32.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c71551640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f6c6c09bc10 0x7f6c6c09f9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:58292/0 (socket says 172.21.15.132:58292) 2024-01-24T23:22:32.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c71551640 1 -- 172.21.15.132:0/344171894 learned_addr learned my addr 172.21.15.132:0/344171894 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:32.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c70d50640 1 -- 172.21.15.132:0/344171894 >> v1:172.21.15.115:6790/0 conn(0x7f6c6c0a00d0 legacy=0x7f6c6c1387c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:22:32.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 1 -- 172.21.15.132:0/344171894 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3905051865 0 0) 0x7f6c6c0a1870 con 0x7f6c6c09bc10 2024-01-24T23:22:32.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:32.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:32.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:32.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 10 monclient(hunting): my global_id is 4550 2024-01-24T23:22:32.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 1 -- 172.21.15.132:0/344171894 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c54003450 con 0x7f6c6c09bc10 2024-01-24T23:22:32.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 1 -- 172.21.15.132:0/344171894 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2809004114 0 0) 0x7f6c54003450 con 0x7f6c6c09bc10 2024-01-24T23:22:32.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 1 -- 172.21.15.132:0/344171894 >> v1:172.21.15.115:6791/0 conn(0x7f6c6c0a0560 legacy=0x7f6c6c13bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:32.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 1 -- 172.21.15.132:0/344171894 >> v1:172.21.15.115:6790/0 conn(0x7f6c6c0a00d0 legacy=0x7f6c6c1387c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:32.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:22:32.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 1 monclient: found mon.noname-h 2024-01-24T23:22:32.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:32.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:22:32.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 1 -- 172.21.15.132:0/344171894 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c0a0fa0 con 0x7f6c6c09bc10 2024-01-24T23:22:32.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 10 monclient: _finish_auth 0 2024-01-24T23:22:32.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:22:32.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:32.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 1 -- 172.21.15.132:0/344171894 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6c640017c0 con 0x7f6c6c09bc10 2024-01-24T23:22:32.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:32.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:22:32.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 10 monclient: dump: 2024-01-24T23:22:32.151 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:32.151 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:32.151 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:32.151 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:32.151 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:32.151 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:32.151 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:32.151 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:32.152 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:32.152 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:32.152 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:32.152 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:32.152 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:32.152 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:32.152 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:32.152 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:32.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 5 monclient: authenticate success, global_id 4550 2024-01-24T23:22:32.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:32.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 1 -- 172.21.15.132:0/344171894 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6c64001a70 con 0x7f6c6c09bc10 2024-01-24T23:22:32.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:32.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 1 -- 172.21.15.132:0/344171894 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6c64004bf0 con 0x7f6c6c09bc10 2024-01-24T23:22:32.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:32.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 4 set_mon_vals no callback set 2024-01-24T23:22:32.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:32.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:22:32.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c6bfff640 10 monclient: dump: 2024-01-24T23:22:32.154 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:32.154 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:32.154 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:32.154 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:32.154 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:32.154 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:32.154 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:32.154 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:32.154 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:32.154 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:32.154 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:32.154 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:32.154 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:32.154 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:32.155 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:32.155 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:32.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 10 monclient: shutdown 2024-01-24T23:22:32.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:32.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 1 -- 172.21.15.132:0/344171894 >> v1:172.21.15.132:6791/0 conn(0x7f6c6c09bc10 legacy=0x7f6c6c09f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:32.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 1 -- 172.21.15.132:0/344171894 shutdown_connections 2024-01-24T23:22:32.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 1 -- 172.21.15.132:0/344171894 >> 172.21.15.132:0/344171894 conn(0x7f6c6c0a5a10 msgr2=0x7f6c6c0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:32.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 1 -- 172.21.15.132:0/344171894 shutdown_connections 2024-01-24T23:22:32.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 1 -- 172.21.15.132:0/344171894 wait complete. 2024-01-24T23:22:32.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 10 monclient: build_initial_monmap 2024-01-24T23:22:32.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 10 monclient: monmap: 2024-01-24T23:22:32.155 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:32.155 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:32.155 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:32.155 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:32.156 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:32.156 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:32.156 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:32.156 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:32.156 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:32.156 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:32.156 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:32.156 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:32.156 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:32.156 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:32.156 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:32.156 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:32.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 1 Processor -- start 2024-01-24T23:22:32.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 1 -- start start 2024-01-24T23:22:32.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 10 monclient: init 2024-01-24T23:22:32.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:32.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 10 monclient: _add_conns ranks=[0,3,8,2,1,5,6,7,4] 2024-01-24T23:22:32.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 10 monclient(hunting): picked mon.noname-a con 0x7f6c6c09bc10 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:32.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 10 monclient(hunting): picked mon.noname-g con 0x7f6c6c0a00d0 addr v1:172.21.15.132:6790/0 2024-01-24T23:22:32.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 10 monclient(hunting): picked mon.noname-e con 0x7f6c6c0a0560 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:32.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c6c09c650 con 0x7f6c6c09bc10 2024-01-24T23:22:32.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c6c06f610 con 0x7f6c6c0a00d0 2024-01-24T23:22:32.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c6c070390 con 0x7f6c6c0a0560 2024-01-24T23:22:32.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:32.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 10 monclient(hunting): authenticate will time out at 1039.471129s 2024-01-24T23:22:32.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c70d50640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f6c6c0a00d0 0x7f6c6c1cc0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:39092/0 (socket says 172.21.15.132:39092) 2024-01-24T23:22:32.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c70d50640 1 -- 172.21.15.132:0/2888653655 learned_addr learned my addr 172.21.15.132:0/2888653655 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:32.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c71d52640 1 -- 172.21.15.132:0/2888653655 >> v1:172.21.15.115:6793/0 conn(0x7f6c6c0a0560 legacy=0x7f6c6c1cf820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:32.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c69ffb640 1 -- 172.21.15.132:0/2888653655 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3702493248 0 0) 0x7f6c6c06f610 con 0x7f6c6c0a00d0 2024-01-24T23:22:32.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c69ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:22:32.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c69ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:32.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c69ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:32.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c69ffb640 10 monclient(hunting): my global_id is 4557 2024-01-24T23:22:32.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c69ffb640 1 -- 172.21.15.132:0/2888653655 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c58003530 con 0x7f6c6c0a00d0 2024-01-24T23:22:32.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c69ffb640 1 -- 172.21.15.132:0/2888653655 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3586435672 0 0) 0x7f6c58003530 con 0x7f6c6c0a00d0 2024-01-24T23:22:32.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c69ffb640 1 -- 172.21.15.132:0/2888653655 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6c6c06f610 con 0x7f6c6c0a00d0 2024-01-24T23:22:32.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c69ffb640 1 -- 172.21.15.132:0/2888653655 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6c600046b0 con 0x7f6c6c0a00d0 2024-01-24T23:22:32.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c69ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:32.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c69ffb640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:22:32.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c69ffb640 10 monclient(hunting): dump: 2024-01-24T23:22:32.158 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:32.158 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:32.158 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:32.159 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:32.159 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:32.159 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:32.159 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:32.159 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:32.159 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:32.159 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:32.159 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:32.159 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:32.159 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:32.159 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:32.159 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:32.159 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:32.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c69ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:32.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c69ffb640 1 -- 172.21.15.132:0/2888653655 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1326090054 0 0) 0x7f6c6c06f610 con 0x7f6c6c0a00d0 2024-01-24T23:22:32.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c69ffb640 1 -- 172.21.15.132:0/2888653655 >> v1:172.21.15.115:6793/0 conn(0x7f6c6c0a0560 legacy=0x7f6c6c1cf820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:32.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c69ffb640 1 -- 172.21.15.132:0/2888653655 >> v1:172.21.15.115:6789/0 conn(0x7f6c6c09bc10 legacy=0x7f6c6c1b0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:32.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c69ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:22:32.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c69ffb640 1 monclient: found mon.g 2024-01-24T23:22:32.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c69ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:32.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c69ffb640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:32.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c69ffb640 1 -- 172.21.15.132:0/2888653655 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c071fe0 con 0x7f6c6c0a00d0 2024-01-24T23:22:32.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c69ffb640 10 monclient: _finish_auth 0 2024-01-24T23:22:32.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c69ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:22:32.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c69ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:32.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 5 monclient: authenticate success, global_id 4557 2024-01-24T23:22:32.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 10 monclient: _renew_subs 2024-01-24T23:22:32.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:32.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.142+0000 7f6c737b5640 1 -- 172.21.15.132:0/2888653655 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c6c071900 con 0x7f6c6c0a00d0 2024-01-24T23:22:32.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.146+0000 7f6c737b5640 10 monclient: _renew_subs 2024-01-24T23:22:32.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.146+0000 7f6c737b5640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:32.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.146+0000 7f6c737b5640 1 -- 172.21.15.132:0/2888653655 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c6c1b0b30 con 0x7f6c6c0a00d0 2024-01-24T23:22:32.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.146+0000 7f6c69ffb640 1 -- 172.21.15.132:0/2888653655 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6c600048d0 con 0x7f6c6c0a00d0 2024-01-24T23:22:32.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.146+0000 7f6c69ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:32.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.146+0000 7f6c69ffb640 1 -- 172.21.15.132:0/2888653655 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6c60004e00 con 0x7f6c6c0a00d0 2024-01-24T23:22:32.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.146+0000 7f6c69ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:32.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.146+0000 7f6c6b7fe640 4 set_mon_vals no callback set 2024-01-24T23:22:32.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.146+0000 7f6c69ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:22:32.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.146+0000 7f6c69ffb640 10 monclient: dump: 2024-01-24T23:22:32.161 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:32.161 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:32.161 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:32.161 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:32.161 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:32.162 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.146+0000 7f6c737b5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.146+0000 7f6c737b5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.146+0000 7f6c737b5640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.146+0000 7f6c737b5640 1 -- 172.21.15.132:0/2888653655 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c4c005090 con 0x7f6c6c0a00d0 2024-01-24T23:22:32.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.146+0000 7f6c69ffb640 1 -- 172.21.15.132:0/2888653655 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f6c60004fe0 con 0x7f6c6c0a00d0 2024-01-24T23:22:32.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.146+0000 7f6c69ffb640 1 -- 172.21.15.132:0/2888653655 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(18..18 src has 1..18) v4 ==== 3873+0+0 (unknown 366335946 0 0) 0x7f6c60043c40 con 0x7f6c6c0a00d0 2024-01-24T23:22:32.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.146+0000 7f6c71551640 10 monclient: get_auth_request con 0x7f6c58036fb0 auth_method 0 2024-01-24T23:22:32.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.150+0000 7f6c69ffb640 1 -- 172.21.15.132:0/2888653655 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f6c600117a0 con 0x7f6c6c0a00d0 2024-01-24T23:22:32.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.150+0000 7f6c69ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:32.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.150+0000 7f6c69ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:32.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.306+0000 7f6c737b5640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-01-24T23:22:32.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.306+0000 7f6c737b5640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-01-24T23:22:32.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.306+0000 7f6c737b5640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:32.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.306+0000 7f6c737b5640 1 -- 172.21.15.132:0/2888653655 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"} v 0) v1 -- 0x7f6c4c0058c0 con 0x7f6c6c0a00d0 2024-01-24T23:22:32.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.306+0000 7f6c69ffb640 1 -- 172.21.15.132:0/2888653655 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}]=0 v18) v1 ==== 105+0+14 (unknown 1992256803 0 3052767636) 0x7f6c600132b0 con 0x7f6c6c0a00d0 2024-01-24T23:22:32.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.306+0000 7f6c69ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-01-24T23:22:32.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.306+0000 7f6c69ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:22:32.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.394+0000 7f6c737b5640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-01-24T23:22:32.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.394+0000 7f6c737b5640 10 monclient: _send_command 3 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-01-24T23:22:32.396 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.394+0000 7f6c737b5640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:32.396 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.394+0000 7f6c737b5640 1 -- 172.21.15.132:0/2888653655 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"} v 0) v1 -- 0x7f6c4c000bc0 con 0x7f6c6c0a00d0 2024-01-24T23:22:32.396 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.394+0000 7f6c69ffb640 1 -- 172.21.15.132:0/2888653655 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}]=0 v18) v1 ==== 105+0+14 (unknown 1992256803 0 3052767636) 0x7f6c60014a30 con 0x7f6c6c0a00d0 2024-01-24T23:22:32.396 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.394+0000 7f6c69ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-01-24T23:22:32.396 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.394+0000 7f6c69ffb640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:22:32.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.394+0000 7f6c737b5640 1 -- 172.21.15.132:0/2888653655 >> v1:172.21.15.132:6812/1835401922 conn(0x7f6c58036fb0 legacy=0x7f6c58039470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:32.398 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.394+0000 7f6c737b5640 10 monclient: shutdown 2024-01-24T23:22:32.398 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.394+0000 7f6c737b5640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:32.398 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.394+0000 7f6c737b5640 1 -- 172.21.15.132:0/2888653655 >> v1:172.21.15.132:6790/0 conn(0x7f6c6c0a00d0 legacy=0x7f6c6c1cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:32.398 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.394+0000 7f6c737b5640 1 -- 172.21.15.132:0/2888653655 shutdown_connections 2024-01-24T23:22:32.398 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.394+0000 7f6c737b5640 1 -- 172.21.15.132:0/2888653655 >> 172.21.15.132:0/2888653655 conn(0x7f6c6c0a5a10 msgr2=0x7f6c6c1b8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:32.398 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.394+0000 7f6c737b5640 1 -- 172.21.15.132:0/2888653655 shutdown_connections 2024-01-24T23:22:32.398 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.394+0000 7f6c737b5640 1 -- 172.21.15.132:0/2888653655 wait complete. 2024-01-24T23:22:32.411 INFO:tasks.workunit.client.0.smithi132.stdout:pg_num_min: 2 2024-01-24T23:22:32.412 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool set testpool1234 pg_num_max 33 2024-01-24T23:22:32.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4fd3e640 10 monclient: get_monmap_and_config 2024-01-24T23:22:32.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4fd3e640 10 monclient: build_initial_monmap 2024-01-24T23:22:32.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4fd3e640 10 monclient: monmap: 2024-01-24T23:22:32.527 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:32.527 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:32.528 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:32.529622+0000 2024-01-24T23:22:32.528 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:32.529622+0000 2024-01-24T23:22:32.528 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:32.528 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:32.528 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:32.528 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:32.528 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:32.528 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:32.528 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:32.528 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:32.528 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:32.528 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:32.528 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:32.528 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:32.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4fd3e640 1 Processor -- start 2024-01-24T23:22:32.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4fd3e640 1 -- start start 2024-01-24T23:22:32.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4fd3e640 10 monclient: init 2024-01-24T23:22:32.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4fd3e640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:32.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4fd3e640 10 monclient: _add_conns ranks=[1,8,3,0,4,7,6,5,2] 2024-01-24T23:22:32.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4fd3e640 10 monclient(hunting): picked mon.noname-f con 0x7faf48134cc0 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:32.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4fd3e640 10 monclient(hunting): picked mon.noname-e con 0x7faf4806de30 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:32.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4fd3e640 10 monclient(hunting): picked mon.noname-g con 0x7faf4813b3a0 addr v1:172.21.15.132:6790/0 2024-01-24T23:22:32.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4fd3e640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf48064700 con 0x7faf48134cc0 2024-01-24T23:22:32.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4fd3e640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf48070e90 con 0x7faf4813b3a0 2024-01-24T23:22:32.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4fd3e640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf48071170 con 0x7faf4806de30 2024-01-24T23:22:32.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4fd3e640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:32.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4fd3e640 10 monclient(hunting): authenticate will time out at 1039.855232s 2024-01-24T23:22:32.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4e2db640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7faf4813b3a0 0x7faf4813d790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:39108/0 (socket says 172.21.15.132:39108) 2024-01-24T23:22:32.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4dada640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7faf48134cc0 0x7faf481350a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:49874/0 (socket says 172.21.15.132:49874) 2024-01-24T23:22:32.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4e2db640 1 -- 172.21.15.132:0/2026109030 learned_addr learned my addr 172.21.15.132:0/2026109030 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:32.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4d2d9640 1 -- 172.21.15.132:0/2026109030 >> v1:172.21.15.115:6793/0 conn(0x7faf4806de30 legacy=0x7faf48139c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:32.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 1 -- 172.21.15.132:0/2026109030 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 696408983 0 0) 0x7faf48070e90 con 0x7faf4813b3a0 2024-01-24T23:22:32.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 10 monclient(hunting): _negotiate 2024-01-24T23:22:32.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:32.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:32.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 10 monclient(hunting): my global_id is 4566 2024-01-24T23:22:32.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 1 -- 172.21.15.132:0/2026109030 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf34003530 con 0x7faf4813b3a0 2024-01-24T23:22:32.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 1 -- 172.21.15.132:0/2026109030 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2857052395 0 0) 0x7faf48064700 con 0x7faf48134cc0 2024-01-24T23:22:32.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 10 monclient(hunting): _negotiate 2024-01-24T23:22:32.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:32.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:32.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 10 monclient(hunting): my global_id is 4582 2024-01-24T23:22:32.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 1 -- 172.21.15.132:0/2026109030 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf48070e90 con 0x7faf48134cc0 2024-01-24T23:22:32.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 1 -- 172.21.15.132:0/2026109030 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4269156426 0 0) 0x7faf48070e90 con 0x7faf48134cc0 2024-01-24T23:22:32.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 1 -- 172.21.15.132:0/2026109030 >> v1:172.21.15.115:6793/0 conn(0x7faf4806de30 legacy=0x7faf48139c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:32.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 1 -- 172.21.15.132:0/2026109030 >> v1:172.21.15.132:6790/0 conn(0x7faf4813b3a0 legacy=0x7faf4813d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:32.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 10 monclient: _finish_hunting 0 2024-01-24T23:22:32.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 1 monclient: found mon.noname-f 2024-01-24T23:22:32.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:32.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:22:32.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 1 -- 172.21.15.132:0/2026109030 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf48071450 con 0x7faf48134cc0 2024-01-24T23:22:32.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 10 monclient: _finish_auth 0 2024-01-24T23:22:32.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 10 monclient: _check_auth_tickets 2024-01-24T23:22:32.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:32.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 1 -- 172.21.15.132:0/2026109030 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faf440017a0 con 0x7faf48134cc0 2024-01-24T23:22:32.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:32.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:22:32.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.526+0000 7faf4cad8640 10 monclient: dump: 2024-01-24T23:22:32.546 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:32.547 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:32.547 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:32.547 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:32.547 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:32.547 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:32.547 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:32.547 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:32.548 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:32.548 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:32.548 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:32.548 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:32.548 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:32.548 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:32.549 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:32.549 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:32.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 5 monclient: authenticate success, global_id 4582 2024-01-24T23:22:32.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:32.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4cad8640 1 -- 172.21.15.132:0/2026109030 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faf44001ac0 con 0x7faf48134cc0 2024-01-24T23:22:32.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4cad8640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:32.550 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4cad8640 1 -- 172.21.15.132:0/2026109030 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faf44004b30 con 0x7faf48134cc0 2024-01-24T23:22:32.550 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:32.550 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 4 set_mon_vals no callback set 2024-01-24T23:22:32.550 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4cad8640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:32.550 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4cad8640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:22:32.550 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4cad8640 10 monclient: dump: 2024-01-24T23:22:32.550 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:32.551 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:32.551 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:32.551 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:32.551 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:32.551 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:32.551 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:32.551 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:32.551 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:32.552 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:32.552 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:32.552 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:32.552 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:32.552 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:32.552 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:32.552 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:32.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 10 monclient: shutdown 2024-01-24T23:22:32.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:32.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 1 -- 172.21.15.132:0/2026109030 >> v1:172.21.15.132:6789/0 conn(0x7faf48134cc0 legacy=0x7faf481350a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:32.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 1 -- 172.21.15.132:0/2026109030 shutdown_connections 2024-01-24T23:22:32.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 1 -- 172.21.15.132:0/2026109030 >> 172.21.15.132:0/2026109030 conn(0x7faf480a2340 msgr2=0x7faf480a2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:32.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 1 -- 172.21.15.132:0/2026109030 shutdown_connections 2024-01-24T23:22:32.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 1 -- 172.21.15.132:0/2026109030 wait complete. 2024-01-24T23:22:32.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 10 monclient: build_initial_monmap 2024-01-24T23:22:32.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 10 monclient: monmap: 2024-01-24T23:22:32.554 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:32.554 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:32.554 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:32.554 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:32.554 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:32.554 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:32.554 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:32.555 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:32.555 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:32.555 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:32.555 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:32.555 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:32.555 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:32.555 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:32.556 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:32.556 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:32.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 1 Processor -- start 2024-01-24T23:22:32.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 1 -- start start 2024-01-24T23:22:32.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 10 monclient: init 2024-01-24T23:22:32.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:32.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 10 monclient: _add_conns ranks=[0,6,8,5,2,4,3,1,7] 2024-01-24T23:22:32.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 10 monclient(hunting): picked mon.noname-a con 0x7faf4806de30 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:32.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 10 monclient(hunting): picked mon.noname-d con 0x7faf48134cc0 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:32.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 10 monclient(hunting): picked mon.noname-e con 0x7faf4813b3a0 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:32.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf481399b0 con 0x7faf4806de30 2024-01-24T23:22:32.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf481c7700 con 0x7faf48134cc0 2024-01-24T23:22:32.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf48137650 con 0x7faf4813b3a0 2024-01-24T23:22:32.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:32.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4fd3e640 10 monclient(hunting): authenticate will time out at 1039.858272s 2024-01-24T23:22:32.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4e2db640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7faf4813b3a0 legacy=0x7faf481d0d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:32.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4d2d9640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7faf48134cc0 0x7faf481c6ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:32774/0 (socket says 172.21.15.132:32774) 2024-01-24T23:22:32.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf4d2d9640 1 -- 172.21.15.132:0/4089250395 learned_addr learned my addr 172.21.15.132:0/4089250395 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:32.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 1 -- 172.21.15.132:0/4089250395 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1782977782 0 0) 0x7faf481c7700 con 0x7faf48134cc0 2024-01-24T23:22:32.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:32.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:32.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:32.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 10 monclient(hunting): my global_id is 4632 2024-01-24T23:22:32.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 1 -- 172.21.15.132:0/4089250395 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf2c003530 con 0x7faf48134cc0 2024-01-24T23:22:32.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 1 -- 172.21.15.132:0/4089250395 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2125621612 0 0) 0x7faf481399b0 con 0x7faf4806de30 2024-01-24T23:22:32.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:32.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:32.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:32.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 10 monclient(hunting): my global_id is 4599 2024-01-24T23:22:32.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 1 -- 172.21.15.132:0/4089250395 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf481c7700 con 0x7faf4806de30 2024-01-24T23:22:32.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 1 -- 172.21.15.132:0/4089250395 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2743957997 0 0) 0x7faf2c003530 con 0x7faf48134cc0 2024-01-24T23:22:32.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 1 -- 172.21.15.132:0/4089250395 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faf481399b0 con 0x7faf48134cc0 2024-01-24T23:22:32.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 1 -- 172.21.15.132:0/4089250395 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faf38004340 con 0x7faf48134cc0 2024-01-24T23:22:32.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:32.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:22:32.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 10 monclient(hunting): dump: 2024-01-24T23:22:32.559 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:32.559 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:32.559 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:32.559 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:32.559 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:32.559 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:32.560 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:32.560 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:32.560 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:32.560 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:32.560 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:32.560 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:32.560 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:32.560 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:32.560 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:32.560 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:32.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:32.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 1 -- 172.21.15.132:0/4089250395 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1472004263 0 0) 0x7faf481c7700 con 0x7faf4806de30 2024-01-24T23:22:32.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 1 -- 172.21.15.132:0/4089250395 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faf2c003530 con 0x7faf4806de30 2024-01-24T23:22:32.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 1 -- 172.21.15.132:0/4089250395 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faf44001760 con 0x7faf4806de30 2024-01-24T23:22:32.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:32.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:22:32.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.530+0000 7faf3e7fc640 10 monclient(hunting): dump: 2024-01-24T23:22:32.561 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:32.561 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:32.561 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:32.561 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:32.561 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:32.561 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:32.561 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:32.561 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:32.562 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:32.562 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:32.562 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:32.562 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:32.562 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:32.562 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:32.562 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:32.562 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:32.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf3e7fc640 1 -- 172.21.15.132:0/4089250395 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4028394955 0 0) 0x7faf481399b0 con 0x7faf48134cc0 2024-01-24T23:22:32.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf3e7fc640 1 -- 172.21.15.132:0/4089250395 >> v1:172.21.15.115:6793/0 conn(0x7faf4813b3a0 legacy=0x7faf481d0d90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:32.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf3e7fc640 1 -- 172.21.15.132:0/4089250395 >> v1:172.21.15.115:6789/0 conn(0x7faf4806de30 legacy=0x7faf481c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:32.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf3e7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:22:32.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf3e7fc640 1 monclient: found mon.d 2024-01-24T23:22:32.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf3e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:32.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf3e7fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:22:32.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf3e7fc640 1 -- 172.21.15.132:0/4089250395 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf481383d0 con 0x7faf48134cc0 2024-01-24T23:22:32.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf3e7fc640 10 monclient: _finish_auth 0 2024-01-24T23:22:32.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf3e7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:22:32.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf3e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:32.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf4fd3e640 5 monclient: authenticate success, global_id 4632 2024-01-24T23:22:32.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf4fd3e640 10 monclient: _renew_subs 2024-01-24T23:22:32.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf4fd3e640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:22:32.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf4fd3e640 1 -- 172.21.15.132:0/4089250395 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf48139110 con 0x7faf48134cc0 2024-01-24T23:22:32.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf4fd3e640 10 monclient: _renew_subs 2024-01-24T23:22:32.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf4fd3e640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:22:32.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf4fd3e640 1 -- 172.21.15.132:0/4089250395 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4812f030 con 0x7faf48134cc0 2024-01-24T23:22:32.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf3e7fc640 1 -- 172.21.15.132:0/4089250395 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faf380045b0 con 0x7faf48134cc0 2024-01-24T23:22:32.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf3e7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:32.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf3e7fc640 1 -- 172.21.15.132:0/4089250395 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faf38004ae0 con 0x7faf48134cc0 2024-01-24T23:22:32.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf3e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:32.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf3ffff640 4 set_mon_vals no callback set 2024-01-24T23:22:32.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf3e7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:22:32.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf3e7fc640 10 monclient: dump: 2024-01-24T23:22:32.564 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:32.564 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:32.564 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:32.564 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:32.564 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:32.564 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:32.564 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:32.564 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:32.564 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:32.564 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:32.565 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:32.565 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:32.565 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:32.565 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:32.565 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:32.565 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:32.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf4fd3e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:32.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf4fd3e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:32.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf4fd3e640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:22:32.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf4fd3e640 1 -- 172.21.15.132:0/4089250395 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf30005090 con 0x7faf48134cc0 2024-01-24T23:22:32.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf3e7fc640 1 -- 172.21.15.132:0/4089250395 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7faf38004cc0 con 0x7faf48134cc0 2024-01-24T23:22:32.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf3e7fc640 1 -- 172.21.15.132:0/4089250395 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(18..18 src has 1..18) v4 ==== 3873+0+0 (unknown 366335946 0 0) 0x7faf38043630 con 0x7faf48134cc0 2024-01-24T23:22:32.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.534+0000 7faf4dada640 10 monclient: get_auth_request con 0x7faf2c037680 auth_method 0 2024-01-24T23:22:32.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.546+0000 7faf3e7fc640 1 -- 172.21.15.132:0/4089250395 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7faf38014970 con 0x7faf48134cc0 2024-01-24T23:22:32.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.546+0000 7faf3e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:32.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.546+0000 7faf3e7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:32.701 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.698+0000 7faf4fd3e640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}] 2024-01-24T23:22:32.702 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.698+0000 7faf4fd3e640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}] 2024-01-24T23:22:32.702 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.698+0000 7faf4fd3e640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:22:32.702 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:32.698+0000 7faf4fd3e640 1 -- 172.21.15.132:0/4089250395 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"} v 0) v1 -- 0x7faf30005340 con 0x7faf48134cc0 2024-01-24T23:22:33.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:33.026+0000 7faf3e7fc640 1 -- 172.21.15.132:0/4089250395 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}]=0 set pool 3 pg_num_max to 33 v19) v1 ==== 145+0+0 (unknown 3029085635 0 0) 0x7faf38014560 con 0x7faf48134cc0 2024-01-24T23:22:33.029 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:33.026+0000 7faf3e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}] 2024-01-24T23:22:33.029 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:33.026+0000 7faf3e7fc640 10 monclient: _finish_command 2 = system:0 set pool 3 pg_num_max to 33 2024-01-24T23:22:33.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:33.118+0000 7faf4fd3e640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}] 2024-01-24T23:22:33.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:33.118+0000 7faf4fd3e640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}] 2024-01-24T23:22:33.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:33.118+0000 7faf4fd3e640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:22:33.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:33.118+0000 7faf4fd3e640 1 -- 172.21.15.132:0/4089250395 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"} v 0) v1 -- 0x7faf300010a0 con 0x7faf48134cc0 2024-01-24T23:22:34.042 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.038+0000 7faf3e7fc640 1 -- 172.21.15.132:0/4089250395 <== mon.6 v1:172.21.15.115:6792/0 11 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}]=0 set pool 3 pg_num_max to 33 v20) v1 ==== 145+0+0 (unknown 3426937336 0 0) 0x7faf38014740 con 0x7faf48134cc0 2024-01-24T23:22:34.042 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.038+0000 7faf3e7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}] 2024-01-24T23:22:34.042 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.038+0000 7faf3e7fc640 10 monclient: _finish_command 3 = system:0 set pool 3 pg_num_max to 33 2024-01-24T23:22:34.042 INFO:tasks.workunit.client.0.smithi132.stderr:set pool 3 pg_num_max to 33 2024-01-24T23:22:34.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.042+0000 7faf4fd3e640 1 -- 172.21.15.132:0/4089250395 >> v1:172.21.15.132:6812/1835401922 conn(0x7faf2c037680 legacy=0x7faf2c039b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:34.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.042+0000 7faf4fd3e640 10 monclient: shutdown 2024-01-24T23:22:34.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.042+0000 7faf4fd3e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:34.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.042+0000 7faf4fd3e640 1 -- 172.21.15.132:0/4089250395 >> v1:172.21.15.115:6792/0 conn(0x7faf48134cc0 legacy=0x7faf481c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:34.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.042+0000 7faf4fd3e640 1 -- 172.21.15.132:0/4089250395 shutdown_connections 2024-01-24T23:22:34.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.042+0000 7faf4fd3e640 1 -- 172.21.15.132:0/4089250395 >> 172.21.15.132:0/4089250395 conn(0x7faf480a2340 msgr2=0x7faf480a5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:34.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.042+0000 7faf4fd3e640 1 -- 172.21.15.132:0/4089250395 shutdown_connections 2024-01-24T23:22:34.046 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.042+0000 7faf4fd3e640 1 -- 172.21.15.132:0/4089250395 wait complete. 2024-01-24T23:22:34.065 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool get testpool1234 pg_num_max 2024-01-24T23:22:34.065 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep 33 2024-01-24T23:22:34.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.174+0000 7fdba2170640 10 monclient: get_monmap_and_config 2024-01-24T23:22:34.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.174+0000 7fdba2170640 10 monclient: build_initial_monmap 2024-01-24T23:22:34.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.174+0000 7fdba2170640 10 monclient: monmap: 2024-01-24T23:22:34.177 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:34.177 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:34.177 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:34.179291+0000 2024-01-24T23:22:34.177 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:34.179291+0000 2024-01-24T23:22:34.177 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:34.177 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:34.177 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:34.177 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:34.177 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:34.178 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:34.178 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:34.178 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:34.178 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:34.178 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:34.178 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:34.178 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:34.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.174+0000 7fdba2170640 1 Processor -- start 2024-01-24T23:22:34.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.174+0000 7fdba2170640 1 -- start start 2024-01-24T23:22:34.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.174+0000 7fdba2170640 10 monclient: init 2024-01-24T23:22:34.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:34.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 10 monclient: _add_conns ranks=[6,1,3,7,8,2,4,5,0] 2024-01-24T23:22:34.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 10 monclient(hunting): picked mon.noname-d con 0x7fdb9c0a1d10 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:34.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 10 monclient(hunting): picked mon.noname-f con 0x7fdb9c0a3630 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:34.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 10 monclient(hunting): picked mon.noname-g con 0x7fdb9c06d240 addr v1:172.21.15.132:6790/0 2024-01-24T23:22:34.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9c064700 con 0x7fdb9c0a3630 2024-01-24T23:22:34.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9c06b600 con 0x7fdb9c06d240 2024-01-24T23:22:34.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9c06c580 con 0x7fdb9c0a1d10 2024-01-24T23:22:34.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:34.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 10 monclient(hunting): authenticate will time out at 1041.504805s 2024-01-24T23:22:34.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb93fff640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fdb9c0a3630 0x7fdb9c13c020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:49086/0 (socket says 172.21.15.132:49086) 2024-01-24T23:22:34.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb93fff640 1 -- 172.21.15.132:0/1000021522 learned_addr learned my addr 172.21.15.132:0/1000021522 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:34.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 1 -- 172.21.15.132:0/1000021522 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1762002197 0 0) 0x7fdb9c06b600 con 0x7fdb9c06d240 2024-01-24T23:22:34.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 10 monclient(hunting): _negotiate 2024-01-24T23:22:34.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:34.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:34.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 10 monclient(hunting): my global_id is 4575 2024-01-24T23:22:34.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 1 -- 172.21.15.132:0/1000021522 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb88003530 con 0x7fdb9c06d240 2024-01-24T23:22:34.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 1 -- 172.21.15.132:0/1000021522 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395339896 0 0) 0x7fdb9c064700 con 0x7fdb9c0a3630 2024-01-24T23:22:34.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 10 monclient(hunting): _negotiate 2024-01-24T23:22:34.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:34.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:34.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 10 monclient(hunting): my global_id is 4591 2024-01-24T23:22:34.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 1 -- 172.21.15.132:0/1000021522 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb9c06b600 con 0x7fdb9c0a3630 2024-01-24T23:22:34.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 1 -- 172.21.15.132:0/1000021522 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2817012869 0 0) 0x7fdb88003530 con 0x7fdb9c06d240 2024-01-24T23:22:34.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 1 -- 172.21.15.132:0/1000021522 >> v1:172.21.15.115:6792/0 conn(0x7fdb9c0a1d10 legacy=0x7fdb9c0a20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:34.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 1 -- 172.21.15.132:0/1000021522 >> v1:172.21.15.132:6789/0 conn(0x7fdb9c0a3630 legacy=0x7fdb9c13c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:34.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 10 monclient: _finish_hunting 0 2024-01-24T23:22:34.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 1 monclient: found mon.noname-g 2024-01-24T23:22:34.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:34.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:22:34.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 1 -- 172.21.15.132:0/1000021522 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9c0a2800 con 0x7fdb9c06d240 2024-01-24T23:22:34.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 10 monclient: _finish_auth 0 2024-01-24T23:22:34.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 10 monclient: _check_auth_tickets 2024-01-24T23:22:34.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:34.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 1 -- 172.21.15.132:0/1000021522 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdb94001b40 con 0x7fdb9c06d240 2024-01-24T23:22:34.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:34.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:22:34.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 10 monclient: dump: 2024-01-24T23:22:34.188 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:34.188 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:34.188 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.188 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.188 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:34.188 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:34.188 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:34.188 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:34.188 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:34.188 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:34.188 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:34.188 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:34.188 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:34.189 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:34.189 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:34.189 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:34.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 5 monclient: authenticate success, global_id 4575 2024-01-24T23:22:34.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:34.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 1 -- 172.21.15.132:0/1000021522 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdb94001de0 con 0x7fdb9c06d240 2024-01-24T23:22:34.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:34.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 1 -- 172.21.15.132:0/1000021522 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdb94003ce0 con 0x7fdb9c06d240 2024-01-24T23:22:34.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:34.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 4 set_mon_vals no callback set 2024-01-24T23:22:34.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:34.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:22:34.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9affd640 10 monclient: dump: 2024-01-24T23:22:34.189 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:34.190 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:34.190 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.190 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.190 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:34.190 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:34.190 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:34.190 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:34.190 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:34.190 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:34.190 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:34.190 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:34.190 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:34.190 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:34.190 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:34.190 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:34.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 10 monclient: shutdown 2024-01-24T23:22:34.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:34.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 1 -- 172.21.15.132:0/1000021522 >> v1:172.21.15.132:6790/0 conn(0x7fdb9c06d240 legacy=0x7fdb9c13f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:34.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 1 -- 172.21.15.132:0/1000021522 shutdown_connections 2024-01-24T23:22:34.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 1 -- 172.21.15.132:0/1000021522 >> 172.21.15.132:0/1000021522 conn(0x7fdb9c0a70e0 msgr2=0x7fdb9c0a74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:34.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 1 -- 172.21.15.132:0/1000021522 shutdown_connections 2024-01-24T23:22:34.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 1 -- 172.21.15.132:0/1000021522 wait complete. 2024-01-24T23:22:34.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 10 monclient: build_initial_monmap 2024-01-24T23:22:34.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 10 monclient: monmap: 2024-01-24T23:22:34.191 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:34.191 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:34.191 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:34.191 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:34.191 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:34.191 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:34.191 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:34.192 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:34.192 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:34.192 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:34.192 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:34.192 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:34.192 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:34.192 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:34.192 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:34.192 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:34.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 1 Processor -- start 2024-01-24T23:22:34.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 1 -- start start 2024-01-24T23:22:34.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 10 monclient: init 2024-01-24T23:22:34.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:34.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 10 monclient: _add_conns ranks=[8,1,7,4,0,5,2,6,3] 2024-01-24T23:22:34.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 10 monclient(hunting): picked mon.noname-e con 0x7fdb9c06d240 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:34.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 10 monclient(hunting): picked mon.noname-f con 0x7fdb9c0a1d10 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:34.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 10 monclient(hunting): picked mon.noname-i con 0x7fdb9c0a3630 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:34.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9c124b80 con 0x7fdb9c0a1d10 2024-01-24T23:22:34.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9c06bed0 con 0x7fdb9c0a3630 2024-01-24T23:22:34.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9c1c86c0 con 0x7fdb9c06d240 2024-01-24T23:22:34.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:34.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdba2170640 10 monclient(hunting): authenticate will time out at 1041.507848s 2024-01-24T23:22:34.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9b7fe640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7fdb9c06d240 legacy=0x7fdb9c1371c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:34.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9bfff640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fdb9c0a3630 0x7fdb9c1d3f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:50308/0 (socket says 172.21.15.132:50308) 2024-01-24T23:22:34.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.178+0000 7fdb9bfff640 1 -- 172.21.15.132:0/2240985636 learned_addr learned my addr 172.21.15.132:0/2240985636 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:34.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 1 -- 172.21.15.132:0/2240985636 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773862045 0 0) 0x7fdb9c06bed0 con 0x7fdb9c0a3630 2024-01-24T23:22:34.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:22:34.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:34.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:34.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 10 monclient(hunting): my global_id is 4642 2024-01-24T23:22:34.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 1 -- 172.21.15.132:0/2240985636 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb7c003530 con 0x7fdb9c0a3630 2024-01-24T23:22:34.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 1 -- 172.21.15.132:0/2240985636 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 103776161 0 0) 0x7fdb9c124b80 con 0x7fdb9c0a1d10 2024-01-24T23:22:34.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:22:34.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:34.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:34.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 10 monclient(hunting): my global_id is 4600 2024-01-24T23:22:34.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 1 -- 172.21.15.132:0/2240985636 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb9c06bed0 con 0x7fdb9c0a1d10 2024-01-24T23:22:34.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 1 -- 172.21.15.132:0/2240985636 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2676288481 0 0) 0x7fdb7c003530 con 0x7fdb9c0a3630 2024-01-24T23:22:34.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 1 -- 172.21.15.132:0/2240985636 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdb9c124b80 con 0x7fdb9c0a3630 2024-01-24T23:22:34.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 1 -- 172.21.15.132:0/2240985636 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdb94003cc0 con 0x7fdb9c0a3630 2024-01-24T23:22:34.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:34.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:22:34.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 10 monclient(hunting): dump: 2024-01-24T23:22:34.194 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:34.194 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:34.195 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.195 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.195 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:34.195 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:34.195 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:34.195 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:34.195 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:34.195 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:34.195 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:34.195 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:34.195 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:34.195 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:34.195 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:34.195 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:34.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:34.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 1 -- 172.21.15.132:0/2240985636 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2377787214 0 0) 0x7fdb9c06bed0 con 0x7fdb9c0a1d10 2024-01-24T23:22:34.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 1 -- 172.21.15.132:0/2240985636 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdb7c003530 con 0x7fdb9c0a1d10 2024-01-24T23:22:34.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 1 -- 172.21.15.132:0/2240985636 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdb8c001940 con 0x7fdb9c0a1d10 2024-01-24T23:22:34.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:34.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:22:34.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 10 monclient(hunting): dump: 2024-01-24T23:22:34.196 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:34.196 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:34.196 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.196 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.196 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:34.196 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:34.196 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:34.196 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:34.197 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:34.197 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:34.197 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:34.197 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:34.197 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:34.197 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:34.197 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:34.197 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:34.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 1 -- 172.21.15.132:0/2240985636 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1461442351 0 0) 0x7fdb9c124b80 con 0x7fdb9c0a3630 2024-01-24T23:22:34.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 1 -- 172.21.15.132:0/2240985636 >> v1:172.21.15.115:6793/0 conn(0x7fdb9c06d240 legacy=0x7fdb9c1371c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:34.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 1 -- 172.21.15.132:0/2240985636 >> v1:172.21.15.132:6789/0 conn(0x7fdb9c0a1d10 legacy=0x7fdb9c1d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:34.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:22:34.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 1 monclient: found mon.i 2024-01-24T23:22:34.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:34.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:34.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 1 -- 172.21.15.132:0/2240985636 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9c09de50 con 0x7fdb9c0a3630 2024-01-24T23:22:34.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 10 monclient: _finish_auth 0 2024-01-24T23:22:34.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:22:34.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:34.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdba2170640 5 monclient: authenticate success, global_id 4642 2024-01-24T23:22:34.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdba2170640 10 monclient: _renew_subs 2024-01-24T23:22:34.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdba2170640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:34.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdba2170640 1 -- 172.21.15.132:0/2240985636 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb9c1242e0 con 0x7fdb9c0a3630 2024-01-24T23:22:34.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdba2170640 10 monclient: _renew_subs 2024-01-24T23:22:34.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdba2170640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:34.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdba2170640 1 -- 172.21.15.132:0/2240985636 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9c1235a0 con 0x7fdb9c0a3630 2024-01-24T23:22:34.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 1 -- 172.21.15.132:0/2240985636 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdb94003ff0 con 0x7fdb9c0a3630 2024-01-24T23:22:34.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:34.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 1 -- 172.21.15.132:0/2240985636 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdb940044a0 con 0x7fdb9c0a3630 2024-01-24T23:22:34.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:34.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb9a7fc640 4 set_mon_vals no callback set 2024-01-24T23:22:34.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:22:34.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 10 monclient: dump: 2024-01-24T23:22:34.199 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:34.199 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:34.199 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.199 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.199 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:34.199 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:34.199 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:34.199 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:34.199 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:34.199 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:34.199 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:34.199 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:34.199 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:34.200 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:34.200 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:34.200 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:34.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdba2170640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:34.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdba2170640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:34.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdba2170640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:34.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdba2170640 1 -- 172.21.15.132:0/2240985636 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb80005090 con 0x7fdb9c0a3630 2024-01-24T23:22:34.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 1 -- 172.21.15.132:0/2240985636 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fdb94004600 con 0x7fdb9c0a3630 2024-01-24T23:22:34.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb98ff9640 1 -- 172.21.15.132:0/2240985636 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(20..20 src has 1..20) v4 ==== 3889+0+0 (unknown 1984694045 0 0) 0x7fdb94043d30 con 0x7fdb9c0a3630 2024-01-24T23:22:34.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.182+0000 7fdb9b7fe640 10 monclient: get_auth_request con 0x7fdb7c0376e0 auth_method 0 2024-01-24T23:22:34.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.186+0000 7fdb98ff9640 1 -- 172.21.15.132:0/2240985636 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fdb94015050 con 0x7fdb9c0a3630 2024-01-24T23:22:34.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.186+0000 7fdb98ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:34.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.186+0000 7fdb98ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:34.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.342+0000 7fdba2170640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-01-24T23:22:34.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.342+0000 7fdba2170640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-01-24T23:22:34.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.342+0000 7fdba2170640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:34.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.342+0000 7fdba2170640 1 -- 172.21.15.132:0/2240985636 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"} v 0) v1 -- 0x7fdb800058c0 con 0x7fdb9c0a3630 2024-01-24T23:22:34.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.342+0000 7fdb98ff9640 1 -- 172.21.15.132:0/2240985636 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}]=0 v20) v1 ==== 105+0+15 (unknown 2967173288 0 2995009362) 0x7fdb940134c0 con 0x7fdb9c0a3630 2024-01-24T23:22:34.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.342+0000 7fdb98ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-01-24T23:22:34.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.342+0000 7fdb98ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:22:34.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.430+0000 7fdba2170640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-01-24T23:22:34.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.430+0000 7fdba2170640 10 monclient: _send_command 3 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-01-24T23:22:34.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.430+0000 7fdba2170640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:34.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.430+0000 7fdba2170640 1 -- 172.21.15.132:0/2240985636 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"} v 0) v1 -- 0x7fdb80000bc0 con 0x7fdb9c0a3630 2024-01-24T23:22:34.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.430+0000 7fdb98ff9640 1 -- 172.21.15.132:0/2240985636 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}]=0 v20) v1 ==== 105+0+15 (unknown 2967173288 0 2995009362) 0x7fdb940134c0 con 0x7fdb9c0a3630 2024-01-24T23:22:34.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.430+0000 7fdb98ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-01-24T23:22:34.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.430+0000 7fdb98ff9640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:22:34.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.430+0000 7fdba2170640 1 -- 172.21.15.132:0/2240985636 >> v1:172.21.15.132:6812/1835401922 conn(0x7fdb7c0376e0 legacy=0x7fdb7c039ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:34.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.430+0000 7fdba2170640 10 monclient: shutdown 2024-01-24T23:22:34.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.430+0000 7fdba2170640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:34.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.430+0000 7fdba2170640 1 -- 172.21.15.132:0/2240985636 >> v1:172.21.15.132:6792/0 conn(0x7fdb9c0a3630 legacy=0x7fdb9c1d3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:34.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.430+0000 7fdba2170640 1 -- 172.21.15.132:0/2240985636 shutdown_connections 2024-01-24T23:22:34.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.430+0000 7fdba2170640 1 -- 172.21.15.132:0/2240985636 >> 172.21.15.132:0/2240985636 conn(0x7fdb9c0a70e0 msgr2=0x7fdb9c0706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:34.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.430+0000 7fdba2170640 1 -- 172.21.15.132:0/2240985636 shutdown_connections 2024-01-24T23:22:34.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.434+0000 7fdba2170640 1 -- 172.21.15.132:0/2240985636 wait complete. 2024-01-24T23:22:34.448 INFO:tasks.workunit.client.0.smithi132.stdout:pg_num_max: 33 2024-01-24T23:22:34.448 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd pool set testpool1234 pg_num_min 9 2024-01-24T23:22:34.448 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:22:34.448 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool set testpool1234 pg_num_min 9 2024-01-24T23:22:34.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 10 monclient: get_monmap_and_config 2024-01-24T23:22:34.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 10 monclient: build_initial_monmap 2024-01-24T23:22:34.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 10 monclient: monmap: 2024-01-24T23:22:34.567 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:34.567 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:34.567 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:34.569406+0000 2024-01-24T23:22:34.567 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:34.569406+0000 2024-01-24T23:22:34.567 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:34.567 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:34.567 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:34.568 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:34.568 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:34.568 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:34.568 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:34.568 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:34.568 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:34.568 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:34.568 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:34.568 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:34.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 1 Processor -- start 2024-01-24T23:22:34.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 1 -- start start 2024-01-24T23:22:34.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 10 monclient: init 2024-01-24T23:22:34.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:34.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 10 monclient: _add_conns ranks=[5,0,1,4,7,6,3,2,8] 2024-01-24T23:22:34.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 10 monclient(hunting): picked mon.noname-h con 0x7f3568072690 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:34.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 10 monclient(hunting): picked mon.noname-a con 0x7f356809ae90 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:34.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 10 monclient(hunting): picked mon.noname-f con 0x7f356809bdc0 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:34.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3568064700 con 0x7f356809ae90 2024-01-24T23:22:34.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f356809a790 con 0x7f356809bdc0 2024-01-24T23:22:34.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35680a18f0 con 0x7f3568072690 2024-01-24T23:22:34.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:34.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 10 monclient(hunting): authenticate will time out at 1041.894034s 2024-01-24T23:22:34.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356659c640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f3568072690 0x7f35680a0ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:33904/0 (socket says 172.21.15.132:33904) 2024-01-24T23:22:34.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356659c640 1 -- 172.21.15.132:0/2429402775 learned_addr learned my addr 172.21.15.132:0/2429402775 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:34.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 1 -- 172.21.15.132:0/2429402775 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 660262509 0 0) 0x7f35680a18f0 con 0x7f3568072690 2024-01-24T23:22:34.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:22:34.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:34.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:34.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 10 monclient(hunting): my global_id is 4559 2024-01-24T23:22:34.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 1 -- 172.21.15.132:0/2429402775 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3554003530 con 0x7f3568072690 2024-01-24T23:22:34.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 1 -- 172.21.15.132:0/2429402775 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3480142656 0 0) 0x7f356809a790 con 0x7f356809bdc0 2024-01-24T23:22:34.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:22:34.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:34.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:34.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 10 monclient(hunting): my global_id is 4609 2024-01-24T23:22:34.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 1 -- 172.21.15.132:0/2429402775 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35680a18f0 con 0x7f356809bdc0 2024-01-24T23:22:34.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 1 -- 172.21.15.132:0/2429402775 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1792935447 0 0) 0x7f3554003530 con 0x7f3568072690 2024-01-24T23:22:34.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 1 -- 172.21.15.132:0/2429402775 >> v1:172.21.15.132:6789/0 conn(0x7f356809bdc0 legacy=0x7f356813d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:34.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 1 -- 172.21.15.132:0/2429402775 >> v1:172.21.15.115:6789/0 conn(0x7f356809ae90 legacy=0x7f35680a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:34.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 10 monclient: _finish_hunting 0 2024-01-24T23:22:34.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 1 monclient: found mon.noname-h 2024-01-24T23:22:34.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:34.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:22:34.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 1 -- 172.21.15.132:0/2429402775 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35680997a0 con 0x7f3568072690 2024-01-24T23:22:34.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 10 monclient: _finish_auth 0 2024-01-24T23:22:34.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 10 monclient: _check_auth_tickets 2024-01-24T23:22:34.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:34.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 5 monclient: authenticate success, global_id 4559 2024-01-24T23:22:34.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:34.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 1 -- 172.21.15.132:0/2429402775 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f35580017c0 con 0x7f3568072690 2024-01-24T23:22:34.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:34.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:22:34.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 10 monclient: dump: 2024-01-24T23:22:34.575 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:34.575 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:34.575 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.575 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.575 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:34.576 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:34.576 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:34.576 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:34.576 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:34.576 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:34.576 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:34.576 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:34.576 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:34.576 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:34.576 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:34.576 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:34.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:34.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 1 -- 172.21.15.132:0/2429402775 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3558001a30 con 0x7f3568072690 2024-01-24T23:22:34.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:34.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 1 -- 172.21.15.132:0/2429402775 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3558004bf0 con 0x7f3568072690 2024-01-24T23:22:34.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:34.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 4 set_mon_vals no callback set 2024-01-24T23:22:34.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:34.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:22:34.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f3565d9b640 10 monclient: dump: 2024-01-24T23:22:34.577 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:34.577 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:34.577 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.577 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.577 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:34.577 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:34.577 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:34.577 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:34.577 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:34.578 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:34.578 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:34.578 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:34.578 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:34.578 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:34.578 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:34.578 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:34.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 10 monclient: shutdown 2024-01-24T23:22:34.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:34.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 1 -- 172.21.15.132:0/2429402775 >> v1:172.21.15.132:6791/0 conn(0x7f3568072690 legacy=0x7f35680a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:34.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 1 -- 172.21.15.132:0/2429402775 shutdown_connections 2024-01-24T23:22:34.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 1 -- 172.21.15.132:0/2429402775 >> 172.21.15.132:0/2429402775 conn(0x7f35680a1bc0 msgr2=0x7f35680a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:34.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 1 -- 172.21.15.132:0/2429402775 shutdown_connections 2024-01-24T23:22:34.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.566+0000 7f356cbc8640 1 -- 172.21.15.132:0/2429402775 wait complete. 2024-01-24T23:22:34.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 10 monclient: build_initial_monmap 2024-01-24T23:22:34.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 10 monclient: monmap: 2024-01-24T23:22:34.579 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:34.579 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:34.579 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:34.579 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:34.579 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:34.579 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:34.579 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:34.579 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:34.579 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:34.579 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:34.579 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:34.579 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:34.579 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:34.579 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:34.579 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:34.580 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:34.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 1 Processor -- start 2024-01-24T23:22:34.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 1 -- start start 2024-01-24T23:22:34.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 10 monclient: init 2024-01-24T23:22:34.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:34.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 10 monclient: _add_conns ranks=[2,8,4,1,7,0,6,3,5] 2024-01-24T23:22:34.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 10 monclient(hunting): picked mon.noname-b con 0x7f3568072690 addr v1:172.21.15.115:6790/0 2024-01-24T23:22:34.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 10 monclient(hunting): picked mon.noname-e con 0x7f356809ae90 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:34.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 10 monclient(hunting): picked mon.noname-c con 0x7f356809bdc0 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:34.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35681ca1d0 con 0x7f3568072690 2024-01-24T23:22:34.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f356809b930 con 0x7f356809bdc0 2024-01-24T23:22:34.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35681b2050 con 0x7f356809ae90 2024-01-24T23:22:34.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:34.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 10 monclient(hunting): authenticate will time out at 1041.897245s 2024-01-24T23:22:34.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355dd9b640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f356809ae90 legacy=0x7f35681c9ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:34.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356659c640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f3568072690 legacy=0x7f35681c93b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:22:34.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f3566d9d640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f356809bdc0 0x7f35681d16f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:50746/0 (socket says 172.21.15.132:50746) 2024-01-24T23:22:34.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f3566d9d640 1 -- 172.21.15.132:0/2745701405 learned_addr learned my addr 172.21.15.132:0/2745701405 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:34.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 1 -- 172.21.15.132:0/2745701405 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2898881822 0 0) 0x7f356809b930 con 0x7f356809bdc0 2024-01-24T23:22:34.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:22:34.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:34.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:34.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 10 monclient(hunting): my global_id is 74650 2024-01-24T23:22:34.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 1 -- 172.21.15.132:0/2745701405 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3548003530 con 0x7f356809bdc0 2024-01-24T23:22:34.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 1 -- 172.21.15.132:0/2745701405 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 197075751 0 0) 0x7f3548003530 con 0x7f356809bdc0 2024-01-24T23:22:34.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 1 -- 172.21.15.132:0/2745701405 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f356809b930 con 0x7f356809bdc0 2024-01-24T23:22:34.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 1 -- 172.21.15.132:0/2745701405 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3560001d20 con 0x7f356809bdc0 2024-01-24T23:22:34.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:34.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:22:34.584 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 10 monclient(hunting): dump: 2024-01-24T23:22:34.584 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:34.584 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:34.584 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.584 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.584 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:34.584 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:34.584 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:34.584 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:34.584 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:34.584 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:34.584 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:34.584 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:34.584 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:34.585 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:34.585 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:34.585 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:34.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:34.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 1 -- 172.21.15.132:0/2745701405 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 220035715 0 0) 0x7f356809b930 con 0x7f356809bdc0 2024-01-24T23:22:34.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 1 -- 172.21.15.132:0/2745701405 >> v1:172.21.15.115:6793/0 conn(0x7f356809ae90 legacy=0x7f35681c9ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:34.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 1 -- 172.21.15.132:0/2745701405 >> v1:172.21.15.115:6790/0 conn(0x7f3568072690 legacy=0x7f35681c93b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:34.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:22:34.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 1 monclient: found mon.c 2024-01-24T23:22:34.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:34.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:22:34.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 1 -- 172.21.15.132:0/2745701405 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35681b2330 con 0x7f356809bdc0 2024-01-24T23:22:34.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 10 monclient: _finish_auth 0 2024-01-24T23:22:34.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:22:34.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:34.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 5 monclient: authenticate success, global_id 74650 2024-01-24T23:22:34.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 10 monclient: _renew_subs 2024-01-24T23:22:34.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:22:34.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 1 -- 172.21.15.132:0/2745701405 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35681b2610 con 0x7f356809bdc0 2024-01-24T23:22:34.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 10 monclient: _renew_subs 2024-01-24T23:22:34.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:22:34.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 1 -- 172.21.15.132:0/2745701405 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f35681b67b0 con 0x7f356809bdc0 2024-01-24T23:22:34.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 1 -- 172.21.15.132:0/2745701405 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3560002020 con 0x7f356809bdc0 2024-01-24T23:22:34.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:34.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 1 -- 172.21.15.132:0/2745701405 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3560003f90 con 0x7f356809bdc0 2024-01-24T23:22:34.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:34.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356559a640 4 set_mon_vals no callback set 2024-01-24T23:22:34.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:22:34.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f355f7fe640 10 monclient: dump: 2024-01-24T23:22:34.587 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:34.587 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:34.587 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.587 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.587 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:34.587 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:34.587 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:34.587 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:34.587 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:34.587 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:34.587 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:34.587 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:34.587 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:34.587 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:34.587 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:34.588 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:34.588 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:34.588 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:34.588 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:22:34.588 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.570+0000 7f356cbc8640 1 -- 172.21.15.132:0/2745701405 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f354c005090 con 0x7f356809bdc0 2024-01-24T23:22:34.588 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.574+0000 7f355f7fe640 1 -- 172.21.15.132:0/2745701405 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f3560004170 con 0x7f356809bdc0 2024-01-24T23:22:34.588 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.574+0000 7f355f7fe640 1 -- 172.21.15.132:0/2745701405 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(20..20 src has 1..20) v4 ==== 3889+0+0 (unknown 1984694045 0 0) 0x7f3560043a80 con 0x7f356809bdc0 2024-01-24T23:22:34.588 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.574+0000 7f356659c640 10 monclient: get_auth_request con 0x7f3548036f90 auth_method 0 2024-01-24T23:22:34.588 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.586+0000 7f355f7fe640 1 -- 172.21.15.132:0/2745701405 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f35600148a0 con 0x7f356809bdc0 2024-01-24T23:22:34.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.586+0000 7f355f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:34.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.586+0000 7f355f7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:34.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.738+0000 7f356cbc8640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "9"}] 2024-01-24T23:22:34.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.738+0000 7f356cbc8640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "9"}] 2024-01-24T23:22:34.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.738+0000 7f356cbc8640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:22:34.742 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.738+0000 7f356cbc8640 1 -- 172.21.15.132:0/2745701405 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "9"} v 0) v1 -- 0x7f354c005340 con 0x7f356809bdc0 2024-01-24T23:22:34.743 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.742+0000 7f355f7fe640 1 -- 172.21.15.132:0/2745701405 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "9"}]=-22 specified pg_num_min 9 > pg_num 8 v20) v1 ==== 150+0+0 (unknown 3399065943 0 0) 0x7f3560014a80 con 0x7f356809bdc0 2024-01-24T23:22:34.743 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.742+0000 7f355f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "9"}] 2024-01-24T23:22:34.743 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.742+0000 7f355f7fe640 10 monclient: _finish_command 2 = mon:22 specified pg_num_min 9 > pg_num 8 2024-01-24T23:22:34.743 INFO:tasks.workunit.client.0.smithi132.stderr:Error EINVAL: specified pg_num_min 9 > pg_num 8 2024-01-24T23:22:34.747 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:22:34.747 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd pool set testpool1234 pg_num_max 7 2024-01-24T23:22:34.747 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:22:34.748 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool set testpool1234 pg_num_max 7 2024-01-24T23:22:34.868 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f464392d640 10 monclient: get_monmap_and_config 2024-01-24T23:22:34.868 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f464392d640 10 monclient: build_initial_monmap 2024-01-24T23:22:34.868 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f464392d640 10 monclient: monmap: 2024-01-24T23:22:34.868 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:34.868 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:34.868 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:34.870624+0000 2024-01-24T23:22:34.869 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:34.870624+0000 2024-01-24T23:22:34.869 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:34.869 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:34.869 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:34.869 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:34.869 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:34.869 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:34.869 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:34.869 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:34.869 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:34.869 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:34.869 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:34.869 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:34.875 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f464392d640 1 Processor -- start 2024-01-24T23:22:34.875 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f464392d640 1 -- start start 2024-01-24T23:22:34.875 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f464392d640 10 monclient: init 2024-01-24T23:22:34.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f464392d640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:34.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f464392d640 10 monclient: _add_conns ranks=[1,8,3,4,2,0,6,5,7] 2024-01-24T23:22:34.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f464392d640 10 monclient(hunting): picked mon.noname-f con 0x7f463c133ed0 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:34.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f464392d640 10 monclient(hunting): picked mon.noname-e con 0x7f463c1342b0 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:34.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f464392d640 10 monclient(hunting): picked mon.noname-g con 0x7f463c134690 addr v1:172.21.15.132:6790/0 2024-01-24T23:22:34.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f464392d640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463c064700 con 0x7f463c133ed0 2024-01-24T23:22:34.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f464392d640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463c09b3d0 con 0x7f463c134690 2024-01-24T23:22:34.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f464392d640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463c0a58c0 con 0x7f463c1342b0 2024-01-24T23:22:34.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f464392d640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:34.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f464392d640 10 monclient(hunting): authenticate will time out at 1042.195268s 2024-01-24T23:22:34.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f46416c9640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f463c133ed0 0x7f463c09bdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:49106/0 (socket says 172.21.15.132:49106) 2024-01-24T23:22:34.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f46416c9640 1 -- 172.21.15.132:0/3051550505 learned_addr learned my addr 172.21.15.132:0/3051550505 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:34.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f4641eca640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f463c134690 0x7f463c13f770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:37042/0 (socket says 172.21.15.132:37042) 2024-01-24T23:22:34.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f4640ec8640 1 -- 172.21.15.132:0/3051550505 >> v1:172.21.15.115:6793/0 conn(0x7f463c1342b0 legacy=0x7f463c13c040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:34.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f4633fff640 1 -- 172.21.15.132:0/3051550505 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1557509712 0 0) 0x7f463c09b3d0 con 0x7f463c134690 2024-01-24T23:22:34.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f4633fff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:34.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f4633fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:34.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f4633fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:34.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f4633fff640 10 monclient(hunting): my global_id is 4584 2024-01-24T23:22:34.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f4633fff640 1 -- 172.21.15.132:0/3051550505 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4628003530 con 0x7f463c134690 2024-01-24T23:22:34.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f4633fff640 1 -- 172.21.15.132:0/3051550505 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969205151 0 0) 0x7f463c064700 con 0x7f463c133ed0 2024-01-24T23:22:34.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f4633fff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:34.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f4633fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:34.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f4633fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:34.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f4633fff640 10 monclient(hunting): my global_id is 4618 2024-01-24T23:22:34.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f4633fff640 1 -- 172.21.15.132:0/3051550505 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f463c09b3d0 con 0x7f463c133ed0 2024-01-24T23:22:34.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f4633fff640 1 -- 172.21.15.132:0/3051550505 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4160958100 0 0) 0x7f463c09b3d0 con 0x7f463c133ed0 2024-01-24T23:22:34.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f4633fff640 1 -- 172.21.15.132:0/3051550505 >> v1:172.21.15.115:6793/0 conn(0x7f463c1342b0 legacy=0x7f463c13c040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:34.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.866+0000 7f4633fff640 1 -- 172.21.15.132:0/3051550505 >> v1:172.21.15.132:6790/0 conn(0x7f463c134690 legacy=0x7f463c13f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:34.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4633fff640 10 monclient: _finish_hunting 0 2024-01-24T23:22:34.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4633fff640 1 monclient: found mon.noname-f 2024-01-24T23:22:34.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4633fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:34.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4633fff640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:22:34.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4633fff640 1 -- 172.21.15.132:0/3051550505 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463c09b0f0 con 0x7f463c133ed0 2024-01-24T23:22:34.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4633fff640 10 monclient: _finish_auth 0 2024-01-24T23:22:34.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4633fff640 10 monclient: _check_auth_tickets 2024-01-24T23:22:34.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4633fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:34.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4633fff640 1 -- 172.21.15.132:0/3051550505 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f46380017a0 con 0x7f463c133ed0 2024-01-24T23:22:34.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4633fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:34.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4633fff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:22:34.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4633fff640 10 monclient: dump: 2024-01-24T23:22:34.879 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:34.879 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:34.879 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.879 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.879 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:34.879 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:34.879 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:34.879 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:34.879 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:34.879 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:34.879 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:34.879 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:34.879 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:34.879 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:34.880 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:34.880 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:34.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 5 monclient: authenticate success, global_id 4618 2024-01-24T23:22:34.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:34.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4633fff640 1 -- 172.21.15.132:0/3051550505 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4638001ac0 con 0x7f463c133ed0 2024-01-24T23:22:34.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4633fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:34.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4633fff640 1 -- 172.21.15.132:0/3051550505 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4638004bf0 con 0x7f463c133ed0 2024-01-24T23:22:34.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4633fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:34.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4633fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:22:34.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4633fff640 10 monclient: dump: 2024-01-24T23:22:34.880 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:34.880 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:34.880 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.880 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.880 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:34.881 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:34.881 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:34.881 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:34.881 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:34.881 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:34.881 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:34.881 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:34.881 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:34.881 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:34.881 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:34.881 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:34.881 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:34.881 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 4 set_mon_vals no callback set 2024-01-24T23:22:34.881 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 10 monclient: shutdown 2024-01-24T23:22:34.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:34.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 1 -- 172.21.15.132:0/3051550505 >> v1:172.21.15.132:6789/0 conn(0x7f463c133ed0 legacy=0x7f463c09bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:34.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 1 -- 172.21.15.132:0/3051550505 shutdown_connections 2024-01-24T23:22:34.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 1 -- 172.21.15.132:0/3051550505 >> 172.21.15.132:0/3051550505 conn(0x7f463c0a5ef0 msgr2=0x7f463c0a6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:34.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 1 -- 172.21.15.132:0/3051550505 shutdown_connections 2024-01-24T23:22:34.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 1 -- 172.21.15.132:0/3051550505 wait complete. 2024-01-24T23:22:34.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 10 monclient: build_initial_monmap 2024-01-24T23:22:34.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 10 monclient: monmap: 2024-01-24T23:22:34.882 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:34.882 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:34.882 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:34.882 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:34.882 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:34.882 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:34.883 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:34.883 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:34.883 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:34.883 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:34.883 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:34.883 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:34.883 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:34.883 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:34.883 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:34.883 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:34.883 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 1 Processor -- start 2024-01-24T23:22:34.883 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 1 -- start start 2024-01-24T23:22:34.883 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 10 monclient: init 2024-01-24T23:22:34.883 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:34.883 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 10 monclient: _add_conns ranks=[4,7,5,8,0,6,1,2,3] 2024-01-24T23:22:34.883 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 10 monclient(hunting): picked mon.noname-c con 0x7f463c133ed0 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:34.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 10 monclient(hunting): picked mon.noname-i con 0x7f463c1342b0 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:34.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 10 monclient(hunting): picked mon.noname-h con 0x7f463c134690 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:34.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463c1205a0 con 0x7f463c133ed0 2024-01-24T23:22:34.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463c0a5af0 con 0x7f463c134690 2024-01-24T23:22:34.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463c11fa40 con 0x7f463c1342b0 2024-01-24T23:22:34.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:34.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 10 monclient(hunting): authenticate will time out at 1042.198579s 2024-01-24T23:22:34.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4641eca640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f463c134690 0x7f463c1d39a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:33908/0 (socket says 172.21.15.132:33908) 2024-01-24T23:22:34.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4641eca640 1 -- 172.21.15.132:0/2557355642 learned_addr learned my addr 172.21.15.132:0/2557355642 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:34.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 1 -- 172.21.15.132:0/2557355642 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4280643411 0 0) 0x7f463c0a5af0 con 0x7f463c134690 2024-01-24T23:22:34.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:22:34.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:34.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:34.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient(hunting): my global_id is 4568 2024-01-24T23:22:34.885 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 1 -- 172.21.15.132:0/2557355642 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4620003530 con 0x7f463c134690 2024-01-24T23:22:34.885 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 1 -- 172.21.15.132:0/2557355642 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2411887055 0 0) 0x7f463c11fa40 con 0x7f463c1342b0 2024-01-24T23:22:34.885 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:22:34.885 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:34.885 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:34.885 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient(hunting): my global_id is 4651 2024-01-24T23:22:34.885 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 1 -- 172.21.15.132:0/2557355642 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f463c0a5af0 con 0x7f463c1342b0 2024-01-24T23:22:34.885 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 1 -- 172.21.15.132:0/2557355642 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 351895441 0 0) 0x7f463c0a5af0 con 0x7f463c1342b0 2024-01-24T23:22:34.885 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 1 -- 172.21.15.132:0/2557355642 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f463c11fa40 con 0x7f463c1342b0 2024-01-24T23:22:34.885 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 1 -- 172.21.15.132:0/2557355642 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f462c0042a0 con 0x7f463c1342b0 2024-01-24T23:22:34.885 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:34.885 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:22:34.885 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient(hunting): dump: 2024-01-24T23:22:34.885 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:34.885 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:34.886 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.886 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.886 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:34.886 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:34.886 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:34.886 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:34.886 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:34.886 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:34.886 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:34.886 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:34.886 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:34.886 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:34.886 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:34.886 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:34.887 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:34.887 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 1 -- 172.21.15.132:0/2557355642 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3458017438 0 0) 0x7f4620003530 con 0x7f463c134690 2024-01-24T23:22:34.887 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 1 -- 172.21.15.132:0/2557355642 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f463c0a5af0 con 0x7f463c134690 2024-01-24T23:22:34.887 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 1 -- 172.21.15.132:0/2557355642 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4634001c70 con 0x7f463c134690 2024-01-24T23:22:34.887 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:34.887 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:22:34.887 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient(hunting): dump: 2024-01-24T23:22:34.887 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:34.887 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:34.887 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.887 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.887 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:34.887 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:34.887 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:34.887 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:34.888 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:34.888 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:34.888 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:34.888 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:34.888 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:34.888 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:34.888 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:34.888 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:34.888 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 1 -- 172.21.15.132:0/2557355642 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347224948 0 0) 0x7f463c1205a0 con 0x7f463c133ed0 2024-01-24T23:22:34.888 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:22:34.888 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:34.888 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:34.888 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient(hunting): my global_id is 74659 2024-01-24T23:22:34.888 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 1 -- 172.21.15.132:0/2557355642 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4620003530 con 0x7f463c133ed0 2024-01-24T23:22:34.889 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 1 -- 172.21.15.132:0/2557355642 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1927001714 0 0) 0x7f463c11fa40 con 0x7f463c1342b0 2024-01-24T23:22:34.889 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 1 -- 172.21.15.132:0/2557355642 >> v1:172.21.15.132:6791/0 conn(0x7f463c134690 legacy=0x7f463c1d39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:34.889 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 1 -- 172.21.15.132:0/2557355642 >> v1:172.21.15.115:6791/0 conn(0x7f463c133ed0 legacy=0x7f463c132be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:34.889 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:22:34.889 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 1 monclient: found mon.i 2024-01-24T23:22:34.889 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:34.889 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:34.889 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 1 -- 172.21.15.132:0/2557355642 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463c11fd00 con 0x7f463c1342b0 2024-01-24T23:22:34.889 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient: _finish_auth 0 2024-01-24T23:22:34.889 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:22:34.889 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:34.889 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 5 monclient: authenticate success, global_id 4651 2024-01-24T23:22:34.889 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 10 monclient: _renew_subs 2024-01-24T23:22:34.889 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:34.890 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 1 -- 172.21.15.132:0/2557355642 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f463c0a2dd0 con 0x7f463c1342b0 2024-01-24T23:22:34.890 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 10 monclient: _renew_subs 2024-01-24T23:22:34.890 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:34.890 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f464392d640 1 -- 172.21.15.132:0/2557355642 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f463c1cb1b0 con 0x7f463c1342b0 2024-01-24T23:22:34.890 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 1 -- 172.21.15.132:0/2557355642 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f462c004540 con 0x7f463c1342b0 2024-01-24T23:22:34.890 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:34.890 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 1 -- 172.21.15.132:0/2557355642 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f462c004a70 con 0x7f463c1342b0 2024-01-24T23:22:34.890 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:34.890 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f46337fe640 4 set_mon_vals no callback set 2024-01-24T23:22:34.890 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:22:34.890 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.870+0000 7f4631ffb640 10 monclient: dump: 2024-01-24T23:22:34.890 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:34.890 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:34.891 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.891 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:34.891 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:34.891 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:34.891 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:34.891 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:34.891 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:34.891 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:34.892 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:34.892 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:34.892 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:34.892 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:34.892 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:34.892 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:34.892 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.874+0000 7f464392d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:34.892 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.874+0000 7f464392d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:34.892 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.874+0000 7f464392d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:34.892 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.874+0000 7f464392d640 1 -- 172.21.15.132:0/2557355642 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461c005090 con 0x7f463c1342b0 2024-01-24T23:22:34.892 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.874+0000 7f4631ffb640 1 -- 172.21.15.132:0/2557355642 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f462c004bd0 con 0x7f463c1342b0 2024-01-24T23:22:34.892 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.874+0000 7f4631ffb640 1 -- 172.21.15.132:0/2557355642 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(20..20 src has 1..20) v4 ==== 3889+0+0 (unknown 1984694045 0 0) 0x7f462c0437b0 con 0x7f463c1342b0 2024-01-24T23:22:34.893 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.874+0000 7f46416c9640 10 monclient: get_auth_request con 0x7f4620037780 auth_method 0 2024-01-24T23:22:34.893 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.878+0000 7f4631ffb640 1 -- 172.21.15.132:0/2557355642 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f462c014a00 con 0x7f463c1342b0 2024-01-24T23:22:34.893 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.878+0000 7f4631ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:34.894 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:34.878+0000 7f4631ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:35.035 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.034+0000 7f464392d640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "7"}] 2024-01-24T23:22:35.035 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.034+0000 7f464392d640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "7"}] 2024-01-24T23:22:35.036 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.034+0000 7f464392d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:35.036 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.034+0000 7f464392d640 1 -- 172.21.15.132:0/2557355642 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "7"} v 0) v1 -- 0x7f461c0058c0 con 0x7f463c1342b0 2024-01-24T23:22:35.042 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.042+0000 7f4631ffb640 1 -- 172.21.15.132:0/2557355642 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "7"}]=-22 specified pg_num_max 7 < pg_num 8 v20) v1 ==== 150+0+0 (unknown 3385876791 0 0) 0x7f462c012e70 con 0x7f463c1342b0 2024-01-24T23:22:35.043 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.042+0000 7f4631ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "7"}] 2024-01-24T23:22:35.043 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.042+0000 7f4631ffb640 10 monclient: _finish_command 2 = mon:22 specified pg_num_max 7 < pg_num 8 2024-01-24T23:22:35.043 INFO:tasks.workunit.client.0.smithi132.stderr:Error EINVAL: specified pg_num_max 7 < pg_num 8 2024-01-24T23:22:35.047 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:22:35.048 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd pool set testpool1234 pg_num 1 2024-01-24T23:22:35.048 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:22:35.048 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool set testpool1234 pg_num 1 2024-01-24T23:22:35.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.162+0000 7ff8feec7640 10 monclient: get_monmap_and_config 2024-01-24T23:22:35.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.162+0000 7ff8feec7640 10 monclient: build_initial_monmap 2024-01-24T23:22:35.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 10 monclient: monmap: 2024-01-24T23:22:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:35.168808+0000 2024-01-24T23:22:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:35.168808+0000 2024-01-24T23:22:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:35.168 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:35.168 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:35.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 1 Processor -- start 2024-01-24T23:22:35.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 1 -- start start 2024-01-24T23:22:35.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 10 monclient: init 2024-01-24T23:22:35.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:35.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 10 monclient: _add_conns ranks=[2,5,6,3,1,0,4,7,8] 2024-01-24T23:22:35.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 10 monclient(hunting): picked mon.noname-b con 0x7ff8f809b810 addr v1:172.21.15.115:6790/0 2024-01-24T23:22:35.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 10 monclient(hunting): picked mon.noname-h con 0x7ff8f809bbf0 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:35.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 10 monclient(hunting): picked mon.noname-d con 0x7ff8f81370a0 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:35.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f8064700 con 0x7ff8f809b810 2024-01-24T23:22:35.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f8099af0 con 0x7ff8f809bbf0 2024-01-24T23:22:35.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f8072110 con 0x7ff8f81370a0 2024-01-24T23:22:35.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:35.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 10 monclient(hunting): authenticate will time out at 1042.493376s 2024-01-24T23:22:35.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8fcc63640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7ff8f809b810 legacy=0x7ff8f80712f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:22:35.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f7fff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7ff8f809bbf0 0x7ff8f8071a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:33920/0 (socket says 172.21.15.132:33920) 2024-01-24T23:22:35.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f7fff640 1 -- 172.21.15.132:0/3041463900 learned_addr learned my addr 172.21.15.132:0/3041463900 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:35.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 1 -- 172.21.15.132:0/3041463900 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2158124379 0 0) 0x7ff8f8099af0 con 0x7ff8f809bbf0 2024-01-24T23:22:35.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:22:35.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:35.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:35.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 10 monclient(hunting): my global_id is 4577 2024-01-24T23:22:35.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 1 -- 172.21.15.132:0/3041463900 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8e4003530 con 0x7ff8f809bbf0 2024-01-24T23:22:35.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 1 -- 172.21.15.132:0/3041463900 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 130908138 0 0) 0x7ff8e4003530 con 0x7ff8f809bbf0 2024-01-24T23:22:35.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 1 -- 172.21.15.132:0/3041463900 >> v1:172.21.15.115:6792/0 conn(0x7ff8f81370a0 legacy=0x7ff8f813ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:35.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 1 -- 172.21.15.132:0/3041463900 >> v1:172.21.15.115:6790/0 conn(0x7ff8f809b810 legacy=0x7ff8f80712f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:35.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 10 monclient: _finish_hunting 0 2024-01-24T23:22:35.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 1 monclient: found mon.noname-h 2024-01-24T23:22:35.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:35.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:22:35.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 1 -- 172.21.15.132:0/3041463900 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f809c050 con 0x7ff8f809bbf0 2024-01-24T23:22:35.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 10 monclient: _finish_auth 0 2024-01-24T23:22:35.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 10 monclient: _check_auth_tickets 2024-01-24T23:22:35.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:35.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 1 -- 172.21.15.132:0/3041463900 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff8e00017c0 con 0x7ff8f809bbf0 2024-01-24T23:22:35.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 5 monclient: authenticate success, global_id 4577 2024-01-24T23:22:35.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:35.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:35.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:22:35.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 10 monclient: dump: 2024-01-24T23:22:35.180 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:35.180 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:35.180 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.180 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.180 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:35.181 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:35.181 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:35.181 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:35.181 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:35.181 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:35.181 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:35.182 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:35.182 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:35.182 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:35.182 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:35.182 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:35.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:35.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 1 -- 172.21.15.132:0/3041463900 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff8e0001a70 con 0x7ff8f809bbf0 2024-01-24T23:22:35.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:35.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:35.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 4 set_mon_vals no callback set 2024-01-24T23:22:35.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 1 -- 172.21.15.132:0/3041463900 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff8e0004bf0 con 0x7ff8f809bbf0 2024-01-24T23:22:35.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:35.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:22:35.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8f77fe640 10 monclient: dump: 2024-01-24T23:22:35.184 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:35.184 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:35.184 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.184 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.184 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:35.184 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:35.185 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:35.185 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:35.185 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:35.185 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:35.185 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:35.185 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:35.186 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:35.186 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:35.186 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:35.186 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:35.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 10 monclient: shutdown 2024-01-24T23:22:35.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:35.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 1 -- 172.21.15.132:0/3041463900 >> v1:172.21.15.132:6791/0 conn(0x7ff8f809bbf0 legacy=0x7ff8f8071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:35.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 1 -- 172.21.15.132:0/3041463900 shutdown_connections 2024-01-24T23:22:35.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 1 -- 172.21.15.132:0/3041463900 >> 172.21.15.132:0/3041463900 conn(0x7ff8f80a1bc0 msgr2=0x7ff8f80a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:35.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 1 -- 172.21.15.132:0/3041463900 shutdown_connections 2024-01-24T23:22:35.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 1 -- 172.21.15.132:0/3041463900 wait complete. 2024-01-24T23:22:35.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 10 monclient: build_initial_monmap 2024-01-24T23:22:35.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 10 monclient: monmap: 2024-01-24T23:22:35.188 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:35.188 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:35.188 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:35.188 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:35.188 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:35.189 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:35.189 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:35.189 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:35.189 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:35.189 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:35.189 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:35.189 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:35.190 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:35.190 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:35.190 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:35.190 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:35.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.166+0000 7ff8feec7640 1 Processor -- start 2024-01-24T23:22:35.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8feec7640 1 -- start start 2024-01-24T23:22:35.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8feec7640 10 monclient: init 2024-01-24T23:22:35.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8feec7640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:35.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8feec7640 10 monclient: _add_conns ranks=[4,5,7,2,8,6,0,1,3] 2024-01-24T23:22:35.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8feec7640 10 monclient(hunting): picked mon.noname-c con 0x7ff8f809b810 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:35.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8feec7640 10 monclient(hunting): picked mon.noname-h con 0x7ff8f809bbf0 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:35.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8feec7640 10 monclient(hunting): picked mon.noname-i con 0x7ff8f81370a0 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:35.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8feec7640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f809b5e0 con 0x7ff8f809b810 2024-01-24T23:22:35.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8feec7640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f81cbfe0 con 0x7ff8f809bbf0 2024-01-24T23:22:35.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8feec7640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f812a970 con 0x7ff8f81370a0 2024-01-24T23:22:35.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8feec7640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:35.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8feec7640 10 monclient(hunting): authenticate will time out at 1042.496528s 2024-01-24T23:22:35.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f7fff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7ff8f809bbf0 0x7ff8f81d0850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:33928/0 (socket says 172.21.15.132:33928) 2024-01-24T23:22:35.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f7fff640 1 -- 172.21.15.132:0/1701674153 learned_addr learned my addr 172.21.15.132:0/1701674153 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:35.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 1 -- 172.21.15.132:0/1701674153 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2393176325 0 0) 0x7ff8f81cbfe0 con 0x7ff8f809bbf0 2024-01-24T23:22:35.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient(hunting): _negotiate 2024-01-24T23:22:35.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:35.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:35.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient(hunting): my global_id is 4586 2024-01-24T23:22:35.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 1 -- 172.21.15.132:0/1701674153 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8d8003530 con 0x7ff8f809bbf0 2024-01-24T23:22:35.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 1 -- 172.21.15.132:0/1701674153 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554261801 0 0) 0x7ff8f812a970 con 0x7ff8f81370a0 2024-01-24T23:22:35.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient(hunting): _negotiate 2024-01-24T23:22:35.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:35.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:35.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient(hunting): my global_id is 4660 2024-01-24T23:22:35.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 1 -- 172.21.15.132:0/1701674153 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8f81cbfe0 con 0x7ff8f81370a0 2024-01-24T23:22:35.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 1 -- 172.21.15.132:0/1701674153 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1747203078 0 0) 0x7ff8d8003530 con 0x7ff8f809bbf0 2024-01-24T23:22:35.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 1 -- 172.21.15.132:0/1701674153 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff8f812a970 con 0x7ff8f809bbf0 2024-01-24T23:22:35.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 1 -- 172.21.15.132:0/1701674153 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff8e0001780 con 0x7ff8f809bbf0 2024-01-24T23:22:35.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:35.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:22:35.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient(hunting): dump: 2024-01-24T23:22:35.195 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:35.195 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:35.195 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.195 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.196 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:35.196 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:35.196 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:35.196 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:35.196 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:35.196 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:35.196 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:35.197 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:35.197 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:35.197 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:35.197 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:35.197 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:35.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:35.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 1 -- 172.21.15.132:0/1701674153 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2977464596 0 0) 0x7ff8f81cbfe0 con 0x7ff8f81370a0 2024-01-24T23:22:35.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 1 -- 172.21.15.132:0/1701674153 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff8d8003530 con 0x7ff8f81370a0 2024-01-24T23:22:35.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 1 -- 172.21.15.132:0/1701674153 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057117266 0 0) 0x7ff8f809b5e0 con 0x7ff8f809b810 2024-01-24T23:22:35.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient(hunting): _negotiate 2024-01-24T23:22:35.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:35.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:35.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient(hunting): my global_id is 74668 2024-01-24T23:22:35.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 1 -- 172.21.15.132:0/1701674153 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8f81cbfe0 con 0x7ff8f809b810 2024-01-24T23:22:35.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 1 -- 172.21.15.132:0/1701674153 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff8f00019a0 con 0x7ff8f81370a0 2024-01-24T23:22:35.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:35.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-01-24T23:22:35.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient(hunting): dump: 2024-01-24T23:22:35.199 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:35.199 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:35.199 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.199 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.199 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:35.199 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:35.199 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:35.200 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:35.200 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:35.200 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:35.200 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:35.200 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:35.200 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:35.200 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:35.200 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:35.201 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:35.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 1 -- 172.21.15.132:0/1701674153 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2763908531 0 0) 0x7ff8f812a970 con 0x7ff8f809bbf0 2024-01-24T23:22:35.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 1 -- 172.21.15.132:0/1701674153 >> v1:172.21.15.132:6792/0 conn(0x7ff8f81370a0 legacy=0x7ff8f81d3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:35.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 1 -- 172.21.15.132:0/1701674153 >> v1:172.21.15.115:6791/0 conn(0x7ff8f809b810 legacy=0x7ff8f8130780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:35.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient: _finish_hunting 0 2024-01-24T23:22:35.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 1 monclient: found mon.h 2024-01-24T23:22:35.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:35.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:35.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 1 -- 172.21.15.132:0/1701674153 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f812ac50 con 0x7ff8f809bbf0 2024-01-24T23:22:35.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient: _finish_auth 0 2024-01-24T23:22:35.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient: _check_auth_tickets 2024-01-24T23:22:35.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:35.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8feec7640 5 monclient: authenticate success, global_id 4586 2024-01-24T23:22:35.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8feec7640 10 monclient: _renew_subs 2024-01-24T23:22:35.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8feec7640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:35.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8feec7640 1 -- 172.21.15.132:0/1701674153 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f812af30 con 0x7ff8f809bbf0 2024-01-24T23:22:35.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8feec7640 10 monclient: _renew_subs 2024-01-24T23:22:35.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 1 -- 172.21.15.132:0/1701674153 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff8e0004ec0 con 0x7ff8f809bbf0 2024-01-24T23:22:35.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8feec7640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:35.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8feec7640 1 -- 172.21.15.132:0/1701674153 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f8135410 con 0x7ff8f809bbf0 2024-01-24T23:22:35.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:35.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 1 -- 172.21.15.132:0/1701674153 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff8e0005370 con 0x7ff8f809bbf0 2024-01-24T23:22:35.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:35.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:22:35.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f6ffd640 4 set_mon_vals no callback set 2024-01-24T23:22:35.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 10 monclient: dump: 2024-01-24T23:22:35.203 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:35.203 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:35.203 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.203 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.203 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:35.203 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:35.203 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:35.203 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:35.203 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:35.203 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:35.203 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:35.203 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:35.204 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:35.204 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:35.204 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:35.204 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:35.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 1 -- 172.21.15.132:0/1701674153 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7ff8e000fdf0 con 0x7ff8f809bbf0 2024-01-24T23:22:35.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8feec7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:35.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8feec7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:35.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8feec7640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:35.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8feec7640 1 -- 172.21.15.132:0/1701674153 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f81374f0 con 0x7ff8f809bbf0 2024-01-24T23:22:35.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8f57fa640 1 -- 172.21.15.132:0/1701674153 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(20..20 src has 1..20) v4 ==== 3889+0+0 (unknown 1984694045 0 0) 0x7ff8e0043af0 con 0x7ff8f809bbf0 2024-01-24T23:22:35.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.170+0000 7ff8fcc63640 10 monclient: get_auth_request con 0x7ff8d8037910 auth_method 0 2024-01-24T23:22:35.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.174+0000 7ff8f57fa640 1 -- 172.21.15.132:0/1701674153 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7ff8f81374f0 con 0x7ff8f809bbf0 2024-01-24T23:22:35.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.174+0000 7ff8f57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:35.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.174+0000 7ff8f57fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:35.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.318+0000 7ff8feec7640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "1"}] 2024-01-24T23:22:35.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.318+0000 7ff8feec7640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "1"}] 2024-01-24T23:22:35.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.318+0000 7ff8feec7640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:35.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.318+0000 7ff8feec7640 1 -- 172.21.15.132:0/1701674153 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "1"} v 0) v1 -- 0x7ff8f81374f0 con 0x7ff8f809bbf0 2024-01-24T23:22:35.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.322+0000 7ff8f57fa640 1 -- 172.21.15.132:0/1701674153 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "1"}]=-22 specified pg_num 1 < pg_num_min 2 v20) v1 ==== 146+0+0 (unknown 4093504435 0 0) 0x7ff8f81374f0 con 0x7ff8f809bbf0 2024-01-24T23:22:35.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.322+0000 7ff8f57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "1"}] 2024-01-24T23:22:35.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.322+0000 7ff8f57fa640 10 monclient: _finish_command 2 = mon:22 specified pg_num 1 < pg_num_min 2 2024-01-24T23:22:35.323 INFO:tasks.workunit.client.0.smithi132.stderr:Error EINVAL: specified pg_num 1 < pg_num_min 2 2024-01-24T23:22:35.328 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:22:35.328 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd pool set testpool1234 pg_num 44 2024-01-24T23:22:35.328 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:22:35.329 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool set testpool1234 pg_num 44 2024-01-24T23:22:35.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 10 monclient: get_monmap_and_config 2024-01-24T23:22:35.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 10 monclient: build_initial_monmap 2024-01-24T23:22:35.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 10 monclient: monmap: 2024-01-24T23:22:35.447 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:35.447 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:35.447 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:35.448867+0000 2024-01-24T23:22:35.447 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:35.448867+0000 2024-01-24T23:22:35.447 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:35.447 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:35.447 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:35.447 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:35.447 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:35.448 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:35.448 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:35.448 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:35.448 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:35.448 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:35.448 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:35.448 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:35.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 1 Processor -- start 2024-01-24T23:22:35.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 1 -- start start 2024-01-24T23:22:35.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 10 monclient: init 2024-01-24T23:22:35.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:35.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 10 monclient: _add_conns ranks=[8,7,2,1,3,6,5,0,4] 2024-01-24T23:22:35.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 10 monclient(hunting): picked mon.noname-e con 0x7f6cf409b810 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:35.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 10 monclient(hunting): picked mon.noname-i con 0x7f6cf409bbf0 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:35.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 10 monclient(hunting): picked mon.noname-b con 0x7f6cf41370a0 addr v1:172.21.15.115:6790/0 2024-01-24T23:22:35.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf4064700 con 0x7f6cf41370a0 2024-01-24T23:22:35.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf4099af0 con 0x7f6cf409bbf0 2024-01-24T23:22:35.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf4072110 con 0x7f6cf409b810 2024-01-24T23:22:35.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:35.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 10 monclient(hunting): authenticate will time out at 1042.773468s 2024-01-24T23:22:35.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf2ffd640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f6cf409bbf0 0x7f6cf4071a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:50344/0 (socket says 172.21.15.132:50344) 2024-01-24T23:22:35.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf37fe640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f6cf409b810 legacy=0x7f6cf40712f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:35.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf2ffd640 1 -- 172.21.15.132:0/207704573 learned_addr learned my addr 172.21.15.132:0/207704573 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:35.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf3fff640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f6cf41370a0 legacy=0x7f6cf413ff30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:22:35.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 1 -- 172.21.15.132:0/207704573 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 696090038 0 0) 0x7f6cf4099af0 con 0x7f6cf409bbf0 2024-01-24T23:22:35.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:35.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:35.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:35.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 10 monclient(hunting): my global_id is 4669 2024-01-24T23:22:35.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 1 -- 172.21.15.132:0/207704573 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cdc003530 con 0x7f6cf409bbf0 2024-01-24T23:22:35.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 1 -- 172.21.15.132:0/207704573 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3953409901 0 0) 0x7f6cdc003530 con 0x7f6cf409bbf0 2024-01-24T23:22:35.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 1 -- 172.21.15.132:0/207704573 >> v1:172.21.15.115:6793/0 conn(0x7f6cf409b810 legacy=0x7f6cf40712f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:35.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 1 -- 172.21.15.132:0/207704573 >> v1:172.21.15.115:6790/0 conn(0x7f6cf41370a0 legacy=0x7f6cf413ff30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:35.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 10 monclient: _finish_hunting 0 2024-01-24T23:22:35.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 1 monclient: found mon.noname-i 2024-01-24T23:22:35.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:35.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:22:35.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 1 -- 172.21.15.132:0/207704573 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf409c050 con 0x7f6cf409bbf0 2024-01-24T23:22:35.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 10 monclient: _finish_auth 0 2024-01-24T23:22:35.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 10 monclient: _check_auth_tickets 2024-01-24T23:22:35.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:35.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 5 monclient: authenticate success, global_id 4669 2024-01-24T23:22:35.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 1 -- 172.21.15.132:0/207704573 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6ce80017c0 con 0x7f6cf409bbf0 2024-01-24T23:22:35.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:35.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:35.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:22:35.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 10 monclient: dump: 2024-01-24T23:22:35.461 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:35.461 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:35.461 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.462 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.462 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:35.462 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:35.462 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:35.462 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:35.462 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:35.462 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:35.463 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:35.463 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:35.463 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:35.463 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:35.463 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:35.463 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:35.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:35.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 1 -- 172.21.15.132:0/207704573 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6ce80019e0 con 0x7f6cf409bbf0 2024-01-24T23:22:35.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:35.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:35.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 4 set_mon_vals no callback set 2024-01-24T23:22:35.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 1 -- 172.21.15.132:0/207704573 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6ce80048b0 con 0x7f6cf409bbf0 2024-01-24T23:22:35.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:35.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:22:35.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cf27fc640 10 monclient: dump: 2024-01-24T23:22:35.465 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:35.465 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:35.465 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.465 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.466 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:35.466 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:35.466 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:35.466 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:35.466 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:35.466 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:35.467 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:35.467 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:35.467 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:35.467 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:35.467 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:35.467 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:35.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 10 monclient: shutdown 2024-01-24T23:22:35.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:35.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 1 -- 172.21.15.132:0/207704573 >> v1:172.21.15.132:6792/0 conn(0x7f6cf409bbf0 legacy=0x7f6cf4071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:35.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 1 -- 172.21.15.132:0/207704573 shutdown_connections 2024-01-24T23:22:35.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 1 -- 172.21.15.132:0/207704573 >> 172.21.15.132:0/207704573 conn(0x7f6cf40a1bc0 msgr2=0x7f6cf40a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:35.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 1 -- 172.21.15.132:0/207704573 shutdown_connections 2024-01-24T23:22:35.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 1 -- 172.21.15.132:0/207704573 wait complete. 2024-01-24T23:22:35.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 10 monclient: build_initial_monmap 2024-01-24T23:22:35.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.446+0000 7f6cfa0a7640 10 monclient: monmap: 2024-01-24T23:22:35.469 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:35.469 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:35.469 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:35.469 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:35.470 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:35.470 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:35.470 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:35.470 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:35.470 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:35.470 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:35.471 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:35.471 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:35.471 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:35.471 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:35.471 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:35.471 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:35.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cfa0a7640 1 Processor -- start 2024-01-24T23:22:35.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cfa0a7640 1 -- start start 2024-01-24T23:22:35.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cfa0a7640 10 monclient: init 2024-01-24T23:22:35.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cfa0a7640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:35.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cfa0a7640 10 monclient: _add_conns ranks=[0,7,6,2,8,4,5,3,1] 2024-01-24T23:22:35.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cfa0a7640 10 monclient(hunting): picked mon.noname-a con 0x7f6cf409b810 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:35.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cfa0a7640 10 monclient(hunting): picked mon.noname-i con 0x7f6cf409bbf0 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:35.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cfa0a7640 10 monclient(hunting): picked mon.noname-d con 0x7f6cf41370a0 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:35.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cfa0a7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf40a1620 con 0x7f6cf409b810 2024-01-24T23:22:35.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cfa0a7640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf409b5e0 con 0x7f6cf41370a0 2024-01-24T23:22:35.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cfa0a7640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf412a8e0 con 0x7f6cf409bbf0 2024-01-24T23:22:35.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cfa0a7640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:35.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cfa0a7640 10 monclient(hunting): authenticate will time out at 1042.776779s 2024-01-24T23:22:35.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cf2ffd640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f6cf409bbf0 0x7f6cf41306f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:50354/0 (socket says 172.21.15.132:50354) 2024-01-24T23:22:35.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cf2ffd640 1 -- 172.21.15.132:0/2774197031 learned_addr learned my addr 172.21.15.132:0/2774197031 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:35.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 1 -- 172.21.15.132:0/2774197031 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 783587064 0 0) 0x7f6cf412a8e0 con 0x7f6cf409bbf0 2024-01-24T23:22:35.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:35.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:35.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:35.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 10 monclient(hunting): my global_id is 4678 2024-01-24T23:22:35.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 1 -- 172.21.15.132:0/2774197031 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ce0003530 con 0x7f6cf409bbf0 2024-01-24T23:22:35.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 1 -- 172.21.15.132:0/2774197031 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2562719261 0 0) 0x7f6ce0003530 con 0x7f6cf409bbf0 2024-01-24T23:22:35.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 1 -- 172.21.15.132:0/2774197031 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6cf412a8e0 con 0x7f6cf409bbf0 2024-01-24T23:22:35.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 1 -- 172.21.15.132:0/2774197031 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6ce80016c0 con 0x7f6cf409bbf0 2024-01-24T23:22:35.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:35.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:22:35.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 10 monclient(hunting): dump: 2024-01-24T23:22:35.475 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:35.475 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:35.475 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.475 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.475 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:35.476 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:35.476 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:35.476 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:35.476 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:35.476 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:35.476 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:35.476 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:35.476 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:35.477 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:35.477 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:35.477 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:35.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:35.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 1 -- 172.21.15.132:0/2774197031 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 410021700 0 0) 0x7f6cf40a1620 con 0x7f6cf409b810 2024-01-24T23:22:35.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:35.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:35.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:35.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 10 monclient(hunting): my global_id is 4617 2024-01-24T23:22:35.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 1 -- 172.21.15.132:0/2774197031 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ce0003530 con 0x7f6cf409b810 2024-01-24T23:22:35.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 1 -- 172.21.15.132:0/2774197031 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2745801328 0 0) 0x7f6cf412a8e0 con 0x7f6cf409bbf0 2024-01-24T23:22:35.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 1 -- 172.21.15.132:0/2774197031 >> v1:172.21.15.115:6792/0 conn(0x7f6cf41370a0 legacy=0x7f6cf41d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:35.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 1 -- 172.21.15.132:0/2774197031 >> v1:172.21.15.115:6789/0 conn(0x7f6cf409b810 legacy=0x7f6cf41cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:35.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 10 monclient: _finish_hunting 0 2024-01-24T23:22:35.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 1 monclient: found mon.i 2024-01-24T23:22:35.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:35.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:35.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 1 -- 172.21.15.132:0/2774197031 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf409e3c0 con 0x7f6cf409bbf0 2024-01-24T23:22:35.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 10 monclient: _finish_auth 0 2024-01-24T23:22:35.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 10 monclient: _check_auth_tickets 2024-01-24T23:22:35.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:35.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cfa0a7640 5 monclient: authenticate success, global_id 4678 2024-01-24T23:22:35.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cfa0a7640 10 monclient: _renew_subs 2024-01-24T23:22:35.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cfa0a7640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:35.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cfa0a7640 1 -- 172.21.15.132:0/2774197031 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf4130e00 con 0x7f6cf409bbf0 2024-01-24T23:22:35.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cfa0a7640 10 monclient: _renew_subs 2024-01-24T23:22:35.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cfa0a7640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:35.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cfa0a7640 1 -- 172.21.15.132:0/2774197031 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf409f040 con 0x7f6cf409bbf0 2024-01-24T23:22:35.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 1 -- 172.21.15.132:0/2774197031 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6ce8004e20 con 0x7f6cf409bbf0 2024-01-24T23:22:35.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:35.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 1 -- 172.21.15.132:0/2774197031 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6ce80052d0 con 0x7f6cf409bbf0 2024-01-24T23:22:35.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:35.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:22:35.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cf1ffb640 4 set_mon_vals no callback set 2024-01-24T23:22:35.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 10 monclient: dump: 2024-01-24T23:22:35.480 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:35.480 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:35.480 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.480 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.480 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:35.480 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:35.481 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:35.481 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:35.481 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:35.481 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:35.481 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:35.481 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:35.481 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:35.481 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:35.481 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:35.481 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:35.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 1 -- 172.21.15.132:0/2774197031 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f6ce800fd50 con 0x7f6cf409bbf0 2024-01-24T23:22:35.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cfa0a7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:35.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cfa0a7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:35.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cfa0a7640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:35.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cfa0a7640 1 -- 172.21.15.132:0/2774197031 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd4005090 con 0x7f6cf409bbf0 2024-01-24T23:22:35.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cdbfff640 1 -- 172.21.15.132:0/2774197031 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(20..20 src has 1..20) v4 ==== 3889+0+0 (unknown 1984694045 0 0) 0x7f6ce8043b20 con 0x7f6cf409bbf0 2024-01-24T23:22:35.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.450+0000 7f6cf37fe640 10 monclient: get_auth_request con 0x7f6ce00370a0 auth_method 0 2024-01-24T23:22:35.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.454+0000 7f6cdbfff640 1 -- 172.21.15.132:0/2774197031 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f6ce8014b40 con 0x7f6cf409bbf0 2024-01-24T23:22:35.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.454+0000 7f6cdbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:35.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.454+0000 7f6cdbfff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:35.607 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.606+0000 7f6cfa0a7640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "44"}] 2024-01-24T23:22:35.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.606+0000 7f6cfa0a7640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "44"}] 2024-01-24T23:22:35.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.606+0000 7f6cfa0a7640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:35.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.606+0000 7f6cfa0a7640 1 -- 172.21.15.132:0/2774197031 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "44"} v 0) v1 -- 0x7f6cd40058c0 con 0x7f6cf409bbf0 2024-01-24T23:22:35.609 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.606+0000 7f6cdbfff640 1 -- 172.21.15.132:0/2774197031 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "44"}]=-22 specified pg_num 44 < pg_num_max 33 v20) v1 ==== 149+0+0 (unknown 199559999 0 0) 0x7f6ce80131e0 con 0x7f6cf409bbf0 2024-01-24T23:22:35.609 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.606+0000 7f6cdbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "44"}] 2024-01-24T23:22:35.609 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.606+0000 7f6cdbfff640 10 monclient: _finish_command 2 = mon:22 specified pg_num 44 < pg_num_max 33 2024-01-24T23:22:35.609 INFO:tasks.workunit.client.0.smithi132.stderr:Error EINVAL: specified pg_num 44 < pg_num_max 33 2024-01-24T23:22:35.615 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:22:35.615 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool set testpool1234 pg_num_min 0 2024-01-24T23:22:35.719 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f68a0b0e640 10 monclient: get_monmap_and_config 2024-01-24T23:22:35.719 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f68a0b0e640 10 monclient: build_initial_monmap 2024-01-24T23:22:35.719 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f68a0b0e640 10 monclient: monmap: 2024-01-24T23:22:35.719 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:35.719 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:35.719 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:35.721583+0000 2024-01-24T23:22:35.719 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:35.721583+0000 2024-01-24T23:22:35.719 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:35.719 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:35.720 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:35.720 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:35.720 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:35.720 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:35.720 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:35.720 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:35.720 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:35.720 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:35.720 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:35.720 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:35.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f68a0b0e640 1 Processor -- start 2024-01-24T23:22:35.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f68a0b0e640 1 -- start start 2024-01-24T23:22:35.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f68a0b0e640 10 monclient: init 2024-01-24T23:22:35.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f68a0b0e640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:35.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f68a0b0e640 10 monclient: _add_conns ranks=[4,6,8,5,2,7,1,0,3] 2024-01-24T23:22:35.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f68a0b0e640 10 monclient(hunting): picked mon.noname-c con 0x7f689c133dc0 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:35.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f68a0b0e640 10 monclient(hunting): picked mon.noname-d con 0x7f689c0a1b50 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:35.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f68a0b0e640 10 monclient(hunting): picked mon.noname-e con 0x7f689c13b430 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:35.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f68a0b0e640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689c064700 con 0x7f689c133dc0 2024-01-24T23:22:35.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f68a0b0e640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689c0a3870 con 0x7f689c0a1b50 2024-01-24T23:22:35.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f68a0b0e640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689c0a35c0 con 0x7f689c13b430 2024-01-24T23:22:35.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f68a0b0e640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:35.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f68a0b0e640 10 monclient(hunting): authenticate will time out at 1043.047127s 2024-01-24T23:22:35.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f689ad9d640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f689c13b430 legacy=0x7f689c13d820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:35.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f689a59c640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f689c133dc0 0x7f689c1341a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:50764/0 (socket says 172.21.15.132:50764) 2024-01-24T23:22:35.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f689a59c640 1 -- 172.21.15.132:0/4289302130 learned_addr learned my addr 172.21.15.132:0/4289302130 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:35.728 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f689959a640 1 -- 172.21.15.132:0/4289302130 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034319880 0 0) 0x7f689c064700 con 0x7f689c133dc0 2024-01-24T23:22:35.728 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f689959a640 10 monclient(hunting): _negotiate 2024-01-24T23:22:35.728 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f689959a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:35.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f689959a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:35.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f689959a640 10 monclient(hunting): my global_id is 74677 2024-01-24T23:22:35.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f689959a640 1 -- 172.21.15.132:0/4289302130 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6884003530 con 0x7f689c133dc0 2024-01-24T23:22:35.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f689959a640 1 -- 172.21.15.132:0/4289302130 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1657635078 0 0) 0x7f689c0a3870 con 0x7f689c0a1b50 2024-01-24T23:22:35.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f689959a640 10 monclient(hunting): _negotiate 2024-01-24T23:22:35.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f689959a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:35.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f689959a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:35.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f689959a640 10 monclient(hunting): my global_id is 4668 2024-01-24T23:22:35.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.718+0000 7f689959a640 1 -- 172.21.15.132:0/4289302130 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f689c064700 con 0x7f689c0a1b50 2024-01-24T23:22:35.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f689959a640 1 -- 172.21.15.132:0/4289302130 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2705642225 0 0) 0x7f6884003530 con 0x7f689c133dc0 2024-01-24T23:22:35.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f689959a640 1 -- 172.21.15.132:0/4289302130 >> v1:172.21.15.115:6793/0 conn(0x7f689c13b430 legacy=0x7f689c13d820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:35.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f689959a640 1 -- 172.21.15.132:0/4289302130 >> v1:172.21.15.115:6792/0 conn(0x7f689c0a1b50 legacy=0x7f689c139d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:35.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f689959a640 10 monclient: _finish_hunting 0 2024-01-24T23:22:35.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f689959a640 1 monclient: found mon.noname-c 2024-01-24T23:22:35.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f689959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:35.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f689959a640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:22:35.730 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f689959a640 1 -- 172.21.15.132:0/4289302130 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689c070770 con 0x7f689c133dc0 2024-01-24T23:22:35.730 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f689959a640 10 monclient: _finish_auth 0 2024-01-24T23:22:35.730 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f689959a640 10 monclient: _check_auth_tickets 2024-01-24T23:22:35.730 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f689959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:35.730 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f689959a640 1 -- 172.21.15.132:0/4289302130 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f688c0017c0 con 0x7f689c133dc0 2024-01-24T23:22:35.730 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 5 monclient: authenticate success, global_id 74677 2024-01-24T23:22:35.730 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:35.730 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f689959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:35.730 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f689959a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:22:35.730 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f689959a640 10 monclient: dump: 2024-01-24T23:22:35.730 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:35.730 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:35.730 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.730 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.730 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:35.730 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:35.731 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:35.731 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:35.731 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:35.731 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:35.731 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:35.731 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:35.731 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:35.731 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:35.731 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:35.731 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:35.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:35.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f689959a640 1 -- 172.21.15.132:0/4289302130 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f688c0019e0 con 0x7f689c133dc0 2024-01-24T23:22:35.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f689959a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:35.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:35.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f689959a640 1 -- 172.21.15.132:0/4289302130 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f688c004bf0 con 0x7f689c133dc0 2024-01-24T23:22:35.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 4 set_mon_vals no callback set 2024-01-24T23:22:35.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f689959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:35.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f689959a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:22:35.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f689959a640 10 monclient: dump: 2024-01-24T23:22:35.732 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:35.732 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:35.732 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.732 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.732 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:35.732 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:35.732 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:35.732 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:35.732 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:35.732 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:35.733 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:35.733 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:35.733 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:35.733 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:35.733 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:35.733 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:35.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 10 monclient: shutdown 2024-01-24T23:22:35.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:35.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 1 -- 172.21.15.132:0/4289302130 >> v1:172.21.15.115:6791/0 conn(0x7f689c133dc0 legacy=0x7f689c1341a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:35.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 1 -- 172.21.15.132:0/4289302130 shutdown_connections 2024-01-24T23:22:35.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 1 -- 172.21.15.132:0/4289302130 >> 172.21.15.132:0/4289302130 conn(0x7f689c06d810 msgr2=0x7f689c0a7150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:35.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 1 -- 172.21.15.132:0/4289302130 shutdown_connections 2024-01-24T23:22:35.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 1 -- 172.21.15.132:0/4289302130 wait complete. 2024-01-24T23:22:35.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 10 monclient: build_initial_monmap 2024-01-24T23:22:35.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 10 monclient: monmap: 2024-01-24T23:22:35.733 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:35.734 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:35.734 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:35.734 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:35.734 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:35.734 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:35.734 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:35.734 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:35.734 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:35.734 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:35.734 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:35.734 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:35.734 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:35.734 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:35.734 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:35.734 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:35.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 1 Processor -- start 2024-01-24T23:22:35.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 1 -- start start 2024-01-24T23:22:35.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 10 monclient: init 2024-01-24T23:22:35.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:35.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 10 monclient: _add_conns ranks=[5,1,0,2,3,8,4,6,7] 2024-01-24T23:22:35.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 10 monclient(hunting): picked mon.noname-h con 0x7f689c0a1b50 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:35.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 10 monclient(hunting): picked mon.noname-f con 0x7f689c133dc0 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:35.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 10 monclient(hunting): picked mon.noname-a con 0x7f689c13b430 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:35.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689c06ffa0 con 0x7f689c13b430 2024-01-24T23:22:35.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689c09bf50 con 0x7f689c133dc0 2024-01-24T23:22:35.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689c09f970 con 0x7f689c0a1b50 2024-01-24T23:22:35.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:35.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 10 monclient(hunting): authenticate will time out at 1043.050589s 2024-01-24T23:22:35.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f689a59c640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f689c0a1b50 0x7f689c09eb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:33940/0 (socket says 172.21.15.132:33940) 2024-01-24T23:22:35.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f689a59c640 1 -- 172.21.15.132:0/4097551298 learned_addr learned my addr 172.21.15.132:0/4097551298 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:35.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 1 -- 172.21.15.132:0/4097551298 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1484548555 0 0) 0x7f689c09f970 con 0x7f689c0a1b50 2024-01-24T23:22:35.736 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient(hunting): _negotiate 2024-01-24T23:22:35.736 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:35.736 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:35.736 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient(hunting): my global_id is 4595 2024-01-24T23:22:35.736 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 1 -- 172.21.15.132:0/4097551298 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f687c003530 con 0x7f689c0a1b50 2024-01-24T23:22:35.736 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 1 -- 172.21.15.132:0/4097551298 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1543083787 0 0) 0x7f689c09bf50 con 0x7f689c133dc0 2024-01-24T23:22:35.736 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient(hunting): _negotiate 2024-01-24T23:22:35.736 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:35.736 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:35.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient(hunting): my global_id is 4627 2024-01-24T23:22:35.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 1 -- 172.21.15.132:0/4097551298 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f689c09f970 con 0x7f689c133dc0 2024-01-24T23:22:35.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 1 -- 172.21.15.132:0/4097551298 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 158975153 0 0) 0x7f687c003530 con 0x7f689c0a1b50 2024-01-24T23:22:35.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 1 -- 172.21.15.132:0/4097551298 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f689c09bf50 con 0x7f689c0a1b50 2024-01-24T23:22:35.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 1 -- 172.21.15.132:0/4097551298 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f688c001740 con 0x7f689c0a1b50 2024-01-24T23:22:35.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:35.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:22:35.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient(hunting): dump: 2024-01-24T23:22:35.737 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:35.737 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:35.737 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.738 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.738 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:35.738 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:35.738 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:35.738 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:35.738 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:35.738 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:35.738 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:35.738 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:35.738 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:35.738 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:35.738 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:35.738 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:35.738 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:35.738 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 1 -- 172.21.15.132:0/4097551298 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 236620716 0 0) 0x7f689c09f970 con 0x7f689c133dc0 2024-01-24T23:22:35.739 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 1 -- 172.21.15.132:0/4097551298 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f687c003530 con 0x7f689c133dc0 2024-01-24T23:22:35.739 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 1 -- 172.21.15.132:0/4097551298 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6890001520 con 0x7f689c133dc0 2024-01-24T23:22:35.739 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:35.739 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:22:35.739 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient(hunting): dump: 2024-01-24T23:22:35.739 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:35.739 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:35.739 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.739 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.739 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:35.739 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:35.739 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:35.739 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:35.739 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:35.739 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:35.739 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:35.740 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:35.740 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:35.740 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:35.740 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:35.740 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:35.740 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 1 -- 172.21.15.132:0/4097551298 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2829391175 0 0) 0x7f689c06ffa0 con 0x7f689c13b430 2024-01-24T23:22:35.740 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient(hunting): _negotiate 2024-01-24T23:22:35.740 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:35.740 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:35.740 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient(hunting): my global_id is 4626 2024-01-24T23:22:35.740 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 1 -- 172.21.15.132:0/4097551298 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f689c09f970 con 0x7f689c13b430 2024-01-24T23:22:35.740 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 1 -- 172.21.15.132:0/4097551298 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2133807510 0 0) 0x7f689c09bf50 con 0x7f689c0a1b50 2024-01-24T23:22:35.740 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 1 -- 172.21.15.132:0/4097551298 >> v1:172.21.15.132:6789/0 conn(0x7f689c133dc0 legacy=0x7f689c09f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:35.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 1 -- 172.21.15.132:0/4097551298 >> v1:172.21.15.115:6789/0 conn(0x7f689c13b430 legacy=0x7f689c1d3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:35.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient: _finish_hunting 0 2024-01-24T23:22:35.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 1 monclient: found mon.h 2024-01-24T23:22:35.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:35.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:35.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 1 -- 172.21.15.132:0/4097551298 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689c070220 con 0x7f689c0a1b50 2024-01-24T23:22:35.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient: _finish_auth 0 2024-01-24T23:22:35.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient: _check_auth_tickets 2024-01-24T23:22:35.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:35.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 5 monclient: authenticate success, global_id 4595 2024-01-24T23:22:35.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 10 monclient: _renew_subs 2024-01-24T23:22:35.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:35.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 1 -- 172.21.15.132:0/4097551298 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f689c09bbe0 con 0x7f689c0a1b50 2024-01-24T23:22:35.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 10 monclient: _renew_subs 2024-01-24T23:22:35.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:35.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f68a0b0e640 1 -- 172.21.15.132:0/4097551298 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f689c09dec0 con 0x7f689c0a1b50 2024-01-24T23:22:35.742 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 1 -- 172.21.15.132:0/4097551298 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f688c001240 con 0x7f689c0a1b50 2024-01-24T23:22:35.742 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:35.742 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 1 -- 172.21.15.132:0/4097551298 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f688c005200 con 0x7f689c0a1b50 2024-01-24T23:22:35.742 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:35.742 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f6898d99640 4 set_mon_vals no callback set 2024-01-24T23:22:35.742 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:22:35.742 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.722+0000 7f688affd640 10 monclient: dump: 2024-01-24T23:22:35.742 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:35.742 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:35.742 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.742 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:35.742 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:35.742 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:35.742 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:35.742 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:35.743 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:35.743 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:35.743 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:35.743 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:35.743 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:35.743 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:35.743 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:35.743 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:35.743 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.726+0000 7f68a0b0e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:35.743 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.726+0000 7f68a0b0e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:35.743 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.726+0000 7f68a0b0e640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:35.743 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.726+0000 7f68a0b0e640 1 -- 172.21.15.132:0/4097551298 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6880005090 con 0x7f689c0a1b50 2024-01-24T23:22:35.743 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.726+0000 7f688affd640 1 -- 172.21.15.132:0/4097551298 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f688c00fc80 con 0x7f689c0a1b50 2024-01-24T23:22:35.743 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.726+0000 7f688affd640 1 -- 172.21.15.132:0/4097551298 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(20..20 src has 1..20) v4 ==== 3889+0+0 (unknown 1984694045 0 0) 0x7f688c043a50 con 0x7f689c0a1b50 2024-01-24T23:22:35.743 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.726+0000 7f6899d9b640 10 monclient: get_auth_request con 0x7f687c037620 auth_method 0 2024-01-24T23:22:35.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.730+0000 7f688affd640 1 -- 172.21.15.132:0/4097551298 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f688c014a70 con 0x7f689c0a1b50 2024-01-24T23:22:35.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.730+0000 7f688affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:35.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.730+0000 7f688affd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:35.874 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.870+0000 7f68a0b0e640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}] 2024-01-24T23:22:35.874 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.870+0000 7f68a0b0e640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}] 2024-01-24T23:22:35.874 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.870+0000 7f68a0b0e640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:35.874 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:35.870+0000 7f68a0b0e640 1 -- 172.21.15.132:0/4097551298 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"} v 0) v1 -- 0x7f68800058c0 con 0x7f689c0a1b50 2024-01-24T23:22:36.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:36.058+0000 7f688affd640 1 -- 172.21.15.132:0/4097551298 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}]=0 set pool 3 pg_num_min to 0 v21) v1 ==== 143+0+0 (unknown 3105387038 0 0) 0x7f688c013110 con 0x7f689c0a1b50 2024-01-24T23:22:36.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:36.058+0000 7f688affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}] 2024-01-24T23:22:36.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:36.058+0000 7f688affd640 10 monclient: _finish_command 2 = system:0 set pool 3 pg_num_min to 0 2024-01-24T23:22:36.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:36.146+0000 7f68a0b0e640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}] 2024-01-24T23:22:36.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:36.146+0000 7f68a0b0e640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}] 2024-01-24T23:22:36.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:36.146+0000 7f68a0b0e640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:36.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:36.146+0000 7f68a0b0e640 1 -- 172.21.15.132:0/4097551298 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"} v 0) v1 -- 0x7f68800010a0 con 0x7f689c0a1b50 2024-01-24T23:22:37.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.054+0000 7f688affd640 1 -- 172.21.15.132:0/4097551298 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}]=0 set pool 3 pg_num_min to 0 v22) v1 ==== 143+0+0 (unknown 4216300655 0 0) 0x7f688c014890 con 0x7f689c0a1b50 2024-01-24T23:22:37.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.054+0000 7f688affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}] 2024-01-24T23:22:37.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.054+0000 7f688affd640 10 monclient: _finish_command 3 = system:0 set pool 3 pg_num_min to 0 2024-01-24T23:22:37.058 INFO:tasks.workunit.client.0.smithi132.stderr:set pool 3 pg_num_min to 0 2024-01-24T23:22:37.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.058+0000 7f68a0b0e640 1 -- 172.21.15.132:0/4097551298 >> v1:172.21.15.132:6812/1835401922 conn(0x7f687c037620 legacy=0x7f687c039ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:37.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.058+0000 7f68a0b0e640 10 monclient: shutdown 2024-01-24T23:22:37.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.058+0000 7f68a0b0e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:37.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.058+0000 7f68a0b0e640 1 -- 172.21.15.132:0/4097551298 >> v1:172.21.15.132:6791/0 conn(0x7f689c0a1b50 legacy=0x7f689c09eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:37.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.058+0000 7f68a0b0e640 1 -- 172.21.15.132:0/4097551298 shutdown_connections 2024-01-24T23:22:37.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.058+0000 7f68a0b0e640 1 -- 172.21.15.132:0/4097551298 >> 172.21.15.132:0/4097551298 conn(0x7f689c06d810 msgr2=0x7f689c0a28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:37.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.062+0000 7f68a0b0e640 1 -- 172.21.15.132:0/4097551298 shutdown_connections 2024-01-24T23:22:37.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.062+0000 7f68a0b0e640 1 -- 172.21.15.132:0/4097551298 wait complete. 2024-01-24T23:22:37.086 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd pool get testpool1234 pg_num_min 2024-01-24T23:22:37.086 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:22:37.086 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool get testpool1234 pg_num_min 2024-01-24T23:22:37.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.198+0000 7feb78b59640 10 monclient: get_monmap_and_config 2024-01-24T23:22:37.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.198+0000 7feb78b59640 10 monclient: build_initial_monmap 2024-01-24T23:22:37.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.198+0000 7feb78b59640 10 monclient: monmap: 2024-01-24T23:22:37.202 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:37.202 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:37.202 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:37.203735+0000 2024-01-24T23:22:37.202 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:37.203735+0000 2024-01-24T23:22:37.202 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:37.202 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:37.202 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:37.202 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:37.202 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:37.202 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:37.202 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:37.202 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:37.202 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:37.202 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:37.203 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:37.203 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:37.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.198+0000 7feb78b59640 1 Processor -- start 2024-01-24T23:22:37.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.198+0000 7feb78b59640 1 -- start start 2024-01-24T23:22:37.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.198+0000 7feb78b59640 10 monclient: init 2024-01-24T23:22:37.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:37.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 10 monclient: _add_conns ranks=[2,7,1,3,5,4,6,8,0] 2024-01-24T23:22:37.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 10 monclient(hunting): picked mon.noname-b con 0x7feb7409ff20 addr v1:172.21.15.115:6790/0 2024-01-24T23:22:37.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 10 monclient(hunting): picked mon.noname-i con 0x7feb7409f9c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:37.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 10 monclient(hunting): picked mon.noname-f con 0x7feb7413b2c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:37.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb74064700 con 0x7feb7413b2c0 2024-01-24T23:22:37.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb7409e760 con 0x7feb7409ff20 2024-01-24T23:22:37.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb740a1160 con 0x7feb7409f9c0 2024-01-24T23:22:37.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:37.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 10 monclient(hunting): authenticate will time out at 1044.528328s 2024-01-24T23:22:37.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7259c640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7feb7409ff20 legacy=0x7feb740a0300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:22:37.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb72d9d640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7feb7413b2c0 0x7feb7413d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:49124/0 (socket says 172.21.15.132:49124) 2024-01-24T23:22:37.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb72d9d640 1 -- 172.21.15.132:0/1104604055 learned_addr learned my addr 172.21.15.132:0/1104604055 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:37.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 1 -- 172.21.15.132:0/1104604055 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3627115840 0 0) 0x7feb74064700 con 0x7feb7413b2c0 2024-01-24T23:22:37.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 10 monclient(hunting): _negotiate 2024-01-24T23:22:37.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:37.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:37.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 10 monclient(hunting): my global_id is 4636 2024-01-24T23:22:37.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 1 -- 172.21.15.132:0/1104604055 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb5c003530 con 0x7feb7413b2c0 2024-01-24T23:22:37.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 1 -- 172.21.15.132:0/1104604055 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1574027228 0 0) 0x7feb740a1160 con 0x7feb7409f9c0 2024-01-24T23:22:37.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 10 monclient(hunting): _negotiate 2024-01-24T23:22:37.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:37.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:37.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 10 monclient(hunting): my global_id is 4687 2024-01-24T23:22:37.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 1 -- 172.21.15.132:0/1104604055 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb74064700 con 0x7feb7409f9c0 2024-01-24T23:22:37.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 1 -- 172.21.15.132:0/1104604055 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1984928202 0 0) 0x7feb74064700 con 0x7feb7409f9c0 2024-01-24T23:22:37.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 1 -- 172.21.15.132:0/1104604055 >> v1:172.21.15.115:6790/0 conn(0x7feb7409ff20 legacy=0x7feb740a0300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:37.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 1 -- 172.21.15.132:0/1104604055 >> v1:172.21.15.132:6789/0 conn(0x7feb7413b2c0 legacy=0x7feb7413d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:37.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 10 monclient: _finish_hunting 0 2024-01-24T23:22:37.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 1 monclient: found mon.noname-i 2024-01-24T23:22:37.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:37.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:22:37.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 1 -- 172.21.15.132:0/1104604055 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb740a0a10 con 0x7feb7409f9c0 2024-01-24T23:22:37.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 10 monclient: _finish_auth 0 2024-01-24T23:22:37.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 10 monclient: _check_auth_tickets 2024-01-24T23:22:37.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:37.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 1 -- 172.21.15.132:0/1104604055 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7feb68001760 con 0x7feb7409f9c0 2024-01-24T23:22:37.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:37.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:22:37.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 10 monclient: dump: 2024-01-24T23:22:37.212 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:37.212 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:37.212 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:37.212 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:37.212 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:37.212 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:37.212 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:37.212 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:37.212 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:37.213 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:37.213 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:37.213 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:37.213 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:37.213 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:37.213 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:37.213 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:37.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 5 monclient: authenticate success, global_id 4687 2024-01-24T23:22:37.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:37.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 1 -- 172.21.15.132:0/1104604055 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7feb68001970 con 0x7feb7409f9c0 2024-01-24T23:22:37.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:37.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:37.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 4 set_mon_vals no callback set 2024-01-24T23:22:37.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 1 -- 172.21.15.132:0/1104604055 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7feb68003620 con 0x7feb7409f9c0 2024-01-24T23:22:37.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:37.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:22:37.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb7159a640 10 monclient: dump: 2024-01-24T23:22:37.214 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:37.214 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:37.214 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:37.214 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:37.214 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:37.214 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:37.214 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:37.214 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:37.214 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:37.214 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:37.214 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:37.215 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:37.215 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:37.215 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:37.215 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:37.215 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:37.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 10 monclient: shutdown 2024-01-24T23:22:37.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:37.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 1 -- 172.21.15.132:0/1104604055 >> v1:172.21.15.132:6792/0 conn(0x7feb7409f9c0 legacy=0x7feb74139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:37.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 1 -- 172.21.15.132:0/1104604055 shutdown_connections 2024-01-24T23:22:37.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 1 -- 172.21.15.132:0/1104604055 >> 172.21.15.132:0/1104604055 conn(0x7feb740a5a10 msgr2=0x7feb740a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:37.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 1 -- 172.21.15.132:0/1104604055 shutdown_connections 2024-01-24T23:22:37.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 1 -- 172.21.15.132:0/1104604055 wait complete. 2024-01-24T23:22:37.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 10 monclient: build_initial_monmap 2024-01-24T23:22:37.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 10 monclient: monmap: 2024-01-24T23:22:37.215 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:37.216 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:37.216 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:37.216 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:37.216 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:37.216 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:37.216 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:37.216 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:37.216 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:37.216 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:37.216 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:37.216 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:37.216 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:37.216 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:37.216 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:37.217 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:37.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 1 Processor -- start 2024-01-24T23:22:37.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 1 -- start start 2024-01-24T23:22:37.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 10 monclient: init 2024-01-24T23:22:37.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:37.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 10 monclient: _add_conns ranks=[6,0,1,8,4,2,5,7,3] 2024-01-24T23:22:37.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 10 monclient(hunting): picked mon.noname-d con 0x7feb7409f9c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:37.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 10 monclient(hunting): picked mon.noname-a con 0x7feb7409ff20 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:37.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 10 monclient(hunting): picked mon.noname-f con 0x7feb7413b2c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:37.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb7411dac0 con 0x7feb7409ff20 2024-01-24T23:22:37.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb74071680 con 0x7feb7413b2c0 2024-01-24T23:22:37.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb74071960 con 0x7feb7409f9c0 2024-01-24T23:22:37.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:37.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb78b59640 10 monclient(hunting): authenticate will time out at 1044.531331s 2024-01-24T23:22:37.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb72d9d640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7feb7413b2c0 0x7feb741d1db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:49136/0 (socket says 172.21.15.132:49136) 2024-01-24T23:22:37.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb72d9d640 1 -- 172.21.15.132:0/735691986 learned_addr learned my addr 172.21.15.132:0/735691986 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:37.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb62ffd640 1 -- 172.21.15.132:0/735691986 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1958195397 0 0) 0x7feb74071680 con 0x7feb7413b2c0 2024-01-24T23:22:37.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb62ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:22:37.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb62ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:37.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb62ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:37.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb62ffd640 10 monclient(hunting): my global_id is 4645 2024-01-24T23:22:37.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb62ffd640 1 -- 172.21.15.132:0/735691986 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb54003530 con 0x7feb7413b2c0 2024-01-24T23:22:37.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb62ffd640 1 -- 172.21.15.132:0/735691986 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3275702596 0 0) 0x7feb54003530 con 0x7feb7413b2c0 2024-01-24T23:22:37.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb62ffd640 1 -- 172.21.15.132:0/735691986 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feb74071680 con 0x7feb7413b2c0 2024-01-24T23:22:37.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb62ffd640 1 -- 172.21.15.132:0/735691986 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7feb6c001d40 con 0x7feb7413b2c0 2024-01-24T23:22:37.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb62ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:37.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb62ffd640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:22:37.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb62ffd640 10 monclient(hunting): dump: 2024-01-24T23:22:37.219 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:37.219 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:37.219 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:37.219 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:37.219 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:37.219 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:37.219 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:37.219 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:37.219 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:37.219 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:37.219 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:37.219 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:37.219 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:37.219 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:37.220 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:37.220 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:37.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.202+0000 7feb62ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:37.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb62ffd640 1 -- 172.21.15.132:0/735691986 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 836928653 0 0) 0x7feb74071680 con 0x7feb7413b2c0 2024-01-24T23:22:37.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb62ffd640 1 -- 172.21.15.132:0/735691986 >> v1:172.21.15.115:6792/0 conn(0x7feb7409f9c0 legacy=0x7feb741c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:37.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb62ffd640 1 -- 172.21.15.132:0/735691986 >> v1:172.21.15.115:6789/0 conn(0x7feb7409ff20 legacy=0x7feb741ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:37.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb62ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:22:37.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb62ffd640 1 monclient: found mon.f 2024-01-24T23:22:37.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb62ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:37.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb62ffd640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:37.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb62ffd640 1 -- 172.21.15.132:0/735691986 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7411b7d0 con 0x7feb7413b2c0 2024-01-24T23:22:37.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb62ffd640 10 monclient: _finish_auth 0 2024-01-24T23:22:37.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb62ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:22:37.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb62ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:37.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb78b59640 5 monclient: authenticate success, global_id 4645 2024-01-24T23:22:37.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb78b59640 10 monclient: _renew_subs 2024-01-24T23:22:37.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb78b59640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:37.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb78b59640 1 -- 172.21.15.132:0/735691986 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb7411d7e0 con 0x7feb7413b2c0 2024-01-24T23:22:37.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb78b59640 10 monclient: _renew_subs 2024-01-24T23:22:37.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb78b59640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:37.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb78b59640 1 -- 172.21.15.132:0/735691986 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7411dee0 con 0x7feb7413b2c0 2024-01-24T23:22:37.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb62ffd640 1 -- 172.21.15.132:0/735691986 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7feb6c004e20 con 0x7feb7413b2c0 2024-01-24T23:22:37.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb62ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:37.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb62ffd640 1 -- 172.21.15.132:0/735691986 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7feb6c0052d0 con 0x7feb7413b2c0 2024-01-24T23:22:37.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb62ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:37.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb70d99640 4 set_mon_vals no callback set 2024-01-24T23:22:37.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb62ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:22:37.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb62ffd640 10 monclient: dump: 2024-01-24T23:22:37.222 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:37.222 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:37.222 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:37.222 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:37.222 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:37.222 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:37.222 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:37.222 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:37.222 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:37.222 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:37.222 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:37.222 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:37.222 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:37.222 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:37.223 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:37.223 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:37.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb78b59640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:37.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb78b59640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:37.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb78b59640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:37.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb78b59640 1 -- 172.21.15.132:0/735691986 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb58005090 con 0x7feb7413b2c0 2024-01-24T23:22:37.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb62ffd640 1 -- 172.21.15.132:0/735691986 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7feb6c00fd50 con 0x7feb7413b2c0 2024-01-24T23:22:37.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb62ffd640 1 -- 172.21.15.132:0/735691986 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(22..22 src has 1..22) v4 ==== 3873+0+0 (unknown 2015331130 0 0) 0x7feb6c043ae0 con 0x7feb7413b2c0 2024-01-24T23:22:37.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.206+0000 7feb7259c640 10 monclient: get_auth_request con 0x7feb54036fc0 auth_method 0 2024-01-24T23:22:37.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.210+0000 7feb62ffd640 1 -- 172.21.15.132:0/735691986 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7feb6c0112a0 con 0x7feb7413b2c0 2024-01-24T23:22:37.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.210+0000 7feb62ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:37.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.210+0000 7feb62ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:37.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.366+0000 7feb78b59640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-01-24T23:22:37.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.366+0000 7feb78b59640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-01-24T23:22:37.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.366+0000 7feb78b59640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:37.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.366+0000 7feb78b59640 1 -- 172.21.15.132:0/735691986 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"} v 0) v1 -- 0x7feb580058c0 con 0x7feb7413b2c0 2024-01-24T23:22:37.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.366+0000 7feb62ffd640 1 -- 172.21.15.132:0/735691986 <== mon.1 v1:172.21.15.132:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}]=-2 option 'pg_num_min' is not set on pool 'testpool1234' v22) v1 ==== 158+0+0 (unknown 4260761247 0 0) 0x7feb6c013300 con 0x7feb7413b2c0 2024-01-24T23:22:37.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.366+0000 7feb62ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-01-24T23:22:37.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.366+0000 7feb62ffd640 10 monclient: _finish_command 2 = mon:2 option 'pg_num_min' is not set on pool 'testpool1234' 2024-01-24T23:22:37.369 INFO:tasks.workunit.client.0.smithi132.stderr:Error ENOENT: option 'pg_num_min' is not set on pool 'testpool1234' 2024-01-24T23:22:37.373 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:22:37.374 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool set testpool1234 pg_num_max 0 2024-01-24T23:22:37.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.482+0000 7f983aab5640 10 monclient: get_monmap_and_config 2024-01-24T23:22:37.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.482+0000 7f983aab5640 10 monclient: build_initial_monmap 2024-01-24T23:22:37.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.482+0000 7f983aab5640 10 monclient: monmap: 2024-01-24T23:22:37.486 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:37.486 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:37.486 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:37.488639+0000 2024-01-24T23:22:37.487 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:37.488639+0000 2024-01-24T23:22:37.487 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:37.487 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:37.487 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:37.487 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:37.487 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:37.487 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:37.487 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:37.487 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:37.487 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:37.487 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:37.487 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:37.487 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:37.494 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 1 Processor -- start 2024-01-24T23:22:37.494 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 1 -- start start 2024-01-24T23:22:37.494 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 10 monclient: init 2024-01-24T23:22:37.494 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:37.494 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 10 monclient: _add_conns ranks=[1,5,0,2,8,4,7,6,3] 2024-01-24T23:22:37.494 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 10 monclient(hunting): picked mon.noname-f con 0x7f9834072690 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:37.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 10 monclient(hunting): picked mon.noname-h con 0x7f983409ae90 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:37.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 10 monclient(hunting): picked mon.noname-a con 0x7f983409bdc0 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:37.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9834064700 con 0x7f983409bdc0 2024-01-24T23:22:37.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f983409a790 con 0x7f9834072690 2024-01-24T23:22:37.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98340a18f0 con 0x7f983409ae90 2024-01-24T23:22:37.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:37.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 10 monclient(hunting): authenticate will time out at 1044.813376s 2024-01-24T23:22:37.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f9838851640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f9834072690 0x7f98340a0ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:49146/0 (socket says 172.21.15.132:49146) 2024-01-24T23:22:37.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f9838851640 1 -- 172.21.15.132:0/2670595939 learned_addr learned my addr 172.21.15.132:0/2670595939 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:37.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 1 -- 172.21.15.132:0/2670595939 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2968946082 0 0) 0x7f983409a790 con 0x7f9834072690 2024-01-24T23:22:37.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 10 monclient(hunting): _negotiate 2024-01-24T23:22:37.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:37.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:37.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 10 monclient(hunting): my global_id is 4654 2024-01-24T23:22:37.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 1 -- 172.21.15.132:0/2670595939 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f981c003530 con 0x7f9834072690 2024-01-24T23:22:37.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 1 -- 172.21.15.132:0/2670595939 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4208016361 0 0) 0x7f98340a18f0 con 0x7f983409ae90 2024-01-24T23:22:37.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 10 monclient(hunting): _negotiate 2024-01-24T23:22:37.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:37.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:37.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 10 monclient(hunting): my global_id is 4604 2024-01-24T23:22:37.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 1 -- 172.21.15.132:0/2670595939 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f983409a790 con 0x7f983409ae90 2024-01-24T23:22:37.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 1 -- 172.21.15.132:0/2670595939 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 80169960 0 0) 0x7f983409a790 con 0x7f983409ae90 2024-01-24T23:22:37.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 1 -- 172.21.15.132:0/2670595939 >> v1:172.21.15.132:6789/0 conn(0x7f9834072690 legacy=0x7f98340a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:37.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 1 -- 172.21.15.132:0/2670595939 >> v1:172.21.15.115:6789/0 conn(0x7f983409bdc0 legacy=0x7f983413d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:37.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 10 monclient: _finish_hunting 0 2024-01-24T23:22:37.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 1 monclient: found mon.noname-h 2024-01-24T23:22:37.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:37.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:22:37.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 1 -- 172.21.15.132:0/2670595939 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98340997a0 con 0x7f983409ae90 2024-01-24T23:22:37.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 10 monclient: _finish_auth 0 2024-01-24T23:22:37.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 10 monclient: _check_auth_tickets 2024-01-24T23:22:37.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:37.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 1 -- 172.21.15.132:0/2670595939 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9828001670 con 0x7f983409ae90 2024-01-24T23:22:37.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 5 monclient: authenticate success, global_id 4604 2024-01-24T23:22:37.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:37.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:37.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:22:37.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 10 monclient: dump: 2024-01-24T23:22:37.497 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:37.497 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:37.497 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:37.497 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:37.497 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:37.498 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:37.498 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:37.498 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:37.498 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:37.498 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:37.498 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:37.498 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:37.498 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:37.498 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:37.498 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:37.498 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:37.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:37.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 1 -- 172.21.15.132:0/2670595939 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f98280018a0 con 0x7f983409ae90 2024-01-24T23:22:37.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:37.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:37.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 4 set_mon_vals no callback set 2024-01-24T23:22:37.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 1 -- 172.21.15.132:0/2670595939 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9828003620 con 0x7f983409ae90 2024-01-24T23:22:37.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:37.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:22:37.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f98337fe640 10 monclient: dump: 2024-01-24T23:22:37.499 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:37.499 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:37.499 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:37.499 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:37.499 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:37.499 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:37.499 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:37.499 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:37.499 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:37.500 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:37.500 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:37.500 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:37.500 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:37.500 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:37.500 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:37.500 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:37.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 10 monclient: shutdown 2024-01-24T23:22:37.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:37.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 1 -- 172.21.15.132:0/2670595939 >> v1:172.21.15.132:6791/0 conn(0x7f983409ae90 legacy=0x7f98340a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:37.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 1 -- 172.21.15.132:0/2670595939 shutdown_connections 2024-01-24T23:22:37.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 1 -- 172.21.15.132:0/2670595939 >> 172.21.15.132:0/2670595939 conn(0x7f98340a1bc0 msgr2=0x7f98340a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:37.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 1 -- 172.21.15.132:0/2670595939 shutdown_connections 2024-01-24T23:22:37.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 1 -- 172.21.15.132:0/2670595939 wait complete. 2024-01-24T23:22:37.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 10 monclient: build_initial_monmap 2024-01-24T23:22:37.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 10 monclient: monmap: 2024-01-24T23:22:37.501 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:37.501 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:37.501 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:37.501 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:37.501 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:37.501 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:37.501 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:37.501 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:37.501 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:37.501 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:37.501 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:37.502 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:37.502 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:37.502 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:37.502 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:37.502 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:37.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 1 Processor -- start 2024-01-24T23:22:37.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 1 -- start start 2024-01-24T23:22:37.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.486+0000 7f983aab5640 10 monclient: init 2024-01-24T23:22:37.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f983aab5640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:37.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f983aab5640 10 monclient: _add_conns ranks=[7,1,0,5,4,3,8,6,2] 2024-01-24T23:22:37.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f983aab5640 10 monclient(hunting): picked mon.noname-i con 0x7f9834072690 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:37.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f983aab5640 10 monclient(hunting): picked mon.noname-f con 0x7f983409ae90 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:37.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f983aab5640 10 monclient(hunting): picked mon.noname-a con 0x7f983409bdc0 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:37.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f983aab5640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f983409d3e0 con 0x7f983409bdc0 2024-01-24T23:22:37.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f983aab5640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98341b3cb0 con 0x7f983409ae90 2024-01-24T23:22:37.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f983aab5640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9834099470 con 0x7f9834072690 2024-01-24T23:22:37.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f983aab5640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:37.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f983aab5640 10 monclient(hunting): authenticate will time out at 1044.816387s 2024-01-24T23:22:37.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f9838851640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f9834072690 0x7f983409c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:50370/0 (socket says 172.21.15.132:50370) 2024-01-24T23:22:37.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f9838851640 1 -- 172.21.15.132:0/1972294111 learned_addr learned my addr 172.21.15.132:0/1972294111 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:37.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 1 -- 172.21.15.132:0/1972294111 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1735828039 0 0) 0x7f98341b3cb0 con 0x7f983409ae90 2024-01-24T23:22:37.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 10 monclient(hunting): _negotiate 2024-01-24T23:22:37.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:37.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:37.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 10 monclient(hunting): my global_id is 4663 2024-01-24T23:22:37.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 1 -- 172.21.15.132:0/1972294111 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9820003530 con 0x7f983409ae90 2024-01-24T23:22:37.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 1 -- 172.21.15.132:0/1972294111 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2036955108 0 0) 0x7f9834099470 con 0x7f9834072690 2024-01-24T23:22:37.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 10 monclient(hunting): _negotiate 2024-01-24T23:22:37.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:37.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:37.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 10 monclient(hunting): my global_id is 4696 2024-01-24T23:22:37.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 1 -- 172.21.15.132:0/1972294111 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98341b3cb0 con 0x7f9834072690 2024-01-24T23:22:37.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 1 -- 172.21.15.132:0/1972294111 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 317896600 0 0) 0x7f98341b3cb0 con 0x7f9834072690 2024-01-24T23:22:37.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 1 -- 172.21.15.132:0/1972294111 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9834099470 con 0x7f9834072690 2024-01-24T23:22:37.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 1 -- 172.21.15.132:0/1972294111 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f98240014a0 con 0x7f9834072690 2024-01-24T23:22:37.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:37.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:22:37.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 10 monclient(hunting): dump: 2024-01-24T23:22:37.504 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:37.504 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:37.504 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:37.505 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:37.505 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:37.505 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:37.505 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:37.505 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:37.505 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:37.505 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:37.505 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:37.505 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:37.505 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:37.505 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:37.505 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:37.505 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:37.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:37.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 1 -- 172.21.15.132:0/1972294111 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3763071204 0 0) 0x7f9820003530 con 0x7f983409ae90 2024-01-24T23:22:37.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 1 -- 172.21.15.132:0/1972294111 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f98341b3cb0 con 0x7f983409ae90 2024-01-24T23:22:37.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 1 -- 172.21.15.132:0/1972294111 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9828001630 con 0x7f983409ae90 2024-01-24T23:22:37.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:37.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:22:37.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 10 monclient(hunting): dump: 2024-01-24T23:22:37.506 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:37.506 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:37.506 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:37.506 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:37.506 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:37.506 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:37.506 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:37.506 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:37.506 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:37.506 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:37.506 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:37.507 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:37.507 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:37.507 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:37.507 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:37.507 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:37.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 1 -- 172.21.15.132:0/1972294111 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2973380455 0 0) 0x7f9834099470 con 0x7f9834072690 2024-01-24T23:22:37.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 1 -- 172.21.15.132:0/1972294111 >> v1:172.21.15.132:6789/0 conn(0x7f983409ae90 legacy=0x7f983409ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:37.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 1 -- 172.21.15.132:0/1972294111 >> v1:172.21.15.115:6789/0 conn(0x7f983409bdc0 legacy=0x7f98341d3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:37.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 10 monclient: _finish_hunting 0 2024-01-24T23:22:37.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 1 monclient: found mon.i 2024-01-24T23:22:37.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:37.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:37.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 1 -- 172.21.15.132:0/1972294111 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983409b930 con 0x7f9834072690 2024-01-24T23:22:37.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 10 monclient: _finish_auth 0 2024-01-24T23:22:37.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 10 monclient: _check_auth_tickets 2024-01-24T23:22:37.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:37.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f983aab5640 5 monclient: authenticate success, global_id 4696 2024-01-24T23:22:37.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f983aab5640 10 monclient: _renew_subs 2024-01-24T23:22:37.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f983aab5640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:37.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f983aab5640 1 -- 172.21.15.132:0/1972294111 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983406f560 con 0x7f9834072690 2024-01-24T23:22:37.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f983aab5640 10 monclient: _renew_subs 2024-01-24T23:22:37.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f983aab5640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:37.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f983aab5640 1 -- 172.21.15.132:0/1972294111 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f983406e9f0 con 0x7f9834072690 2024-01-24T23:22:37.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 1 -- 172.21.15.132:0/1972294111 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f98240017f0 con 0x7f9834072690 2024-01-24T23:22:37.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:37.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 1 -- 172.21.15.132:0/1972294111 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9824004de0 con 0x7f9834072690 2024-01-24T23:22:37.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:37.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f9832ffd640 4 set_mon_vals no callback set 2024-01-24T23:22:37.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:22:37.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 10 monclient: dump: 2024-01-24T23:22:37.509 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:37.509 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:37.509 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:37.509 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:37.509 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:37.509 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:37.509 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:37.509 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:37.509 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:37.509 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:37.509 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:37.509 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:37.510 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:37.510 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:37.510 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:37.510 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:37.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f983aab5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:37.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f983aab5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:37.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f983aab5640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:37.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f983aab5640 1 -- 172.21.15.132:0/1972294111 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9814005090 con 0x7f9834072690 2024-01-24T23:22:37.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 1 -- 172.21.15.132:0/1972294111 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f9824004f40 con 0x7f9834072690 2024-01-24T23:22:37.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f98317fa640 1 -- 172.21.15.132:0/1972294111 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(22..22 src has 1..22) v4 ==== 3873+0+0 (unknown 2015331130 0 0) 0x7f9824043820 con 0x7f9834072690 2024-01-24T23:22:37.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.490+0000 7f9833fff640 10 monclient: get_auth_request con 0x7f9820037860 auth_method 0 2024-01-24T23:22:37.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.494+0000 7f98317fa640 1 -- 172.21.15.132:0/1972294111 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f9824006580 con 0x7f9834072690 2024-01-24T23:22:37.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.494+0000 7f98317fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:37.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.494+0000 7f98317fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:37.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.646+0000 7f983aab5640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}] 2024-01-24T23:22:37.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.646+0000 7f983aab5640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}] 2024-01-24T23:22:37.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.646+0000 7f983aab5640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:37.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:37.646+0000 7f983aab5640 1 -- 172.21.15.132:0/1972294111 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"} v 0) v1 -- 0x7f98140058c0 con 0x7f9834072690 2024-01-24T23:22:38.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:38.082+0000 7f98317fa640 1 -- 172.21.15.132:0/1972294111 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}]=0 set pool 3 pg_num_max to 0 v23) v1 ==== 143+0+0 (unknown 2261640237 0 0) 0x7f9824014680 con 0x7f9834072690 2024-01-24T23:22:38.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:38.082+0000 7f98317fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}] 2024-01-24T23:22:38.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:38.082+0000 7f98317fa640 10 monclient: _finish_command 2 = system:0 set pool 3 pg_num_max to 0 2024-01-24T23:22:38.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:38.170+0000 7f983aab5640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}] 2024-01-24T23:22:38.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:38.170+0000 7f983aab5640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}] 2024-01-24T23:22:38.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:38.170+0000 7f983aab5640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:38.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:38.170+0000 7f983aab5640 1 -- 172.21.15.132:0/1972294111 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"} v 0) v1 -- 0x7f98140010a0 con 0x7f9834072690 2024-01-24T23:22:39.088 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.086+0000 7f98317fa640 1 -- 172.21.15.132:0/1972294111 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}]=0 set pool 3 pg_num_max to 0 v24) v1 ==== 143+0+0 (unknown 3358137705 0 0) 0x7f9824014860 con 0x7f9834072690 2024-01-24T23:22:39.088 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.086+0000 7f98317fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}] 2024-01-24T23:22:39.088 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.086+0000 7f98317fa640 10 monclient: _finish_command 3 = system:0 set pool 3 pg_num_max to 0 2024-01-24T23:22:39.089 INFO:tasks.workunit.client.0.smithi132.stderr:set pool 3 pg_num_max to 0 2024-01-24T23:22:39.091 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.090+0000 7f983aab5640 1 -- 172.21.15.132:0/1972294111 >> v1:172.21.15.132:6812/1835401922 conn(0x7f9820037860 legacy=0x7f9820039d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:39.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.090+0000 7f983aab5640 10 monclient: shutdown 2024-01-24T23:22:39.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.090+0000 7f983aab5640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:39.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.090+0000 7f983aab5640 1 -- 172.21.15.132:0/1972294111 >> v1:172.21.15.132:6792/0 conn(0x7f9834072690 legacy=0x7f983409c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:39.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.090+0000 7f983aab5640 1 -- 172.21.15.132:0/1972294111 shutdown_connections 2024-01-24T23:22:39.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.090+0000 7f983aab5640 1 -- 172.21.15.132:0/1972294111 >> 172.21.15.132:0/1972294111 conn(0x7f98340a1bc0 msgr2=0x7f98340a3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:39.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.090+0000 7f983aab5640 1 -- 172.21.15.132:0/1972294111 shutdown_connections 2024-01-24T23:22:39.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.090+0000 7f983aab5640 1 -- 172.21.15.132:0/1972294111 wait complete. 2024-01-24T23:22:39.111 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd pool get testpool1234 pg_num_max 2024-01-24T23:22:39.111 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:22:39.111 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool get testpool1234 pg_num_max 2024-01-24T23:22:39.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 10 monclient: get_monmap_and_config 2024-01-24T23:22:39.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 10 monclient: build_initial_monmap 2024-01-24T23:22:39.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 10 monclient: monmap: 2024-01-24T23:22:39.227 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:39.227 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:39.227 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:39.229058+0000 2024-01-24T23:22:39.227 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:39.229058+0000 2024-01-24T23:22:39.227 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:39.227 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:39.227 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:39.227 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:39.227 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:39.227 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:39.227 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:39.228 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:39.228 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:39.228 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:39.228 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:39.228 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:39.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 1 Processor -- start 2024-01-24T23:22:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 1 -- start start 2024-01-24T23:22:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 10 monclient: init 2024-01-24T23:22:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 10 monclient: _add_conns ranks=[0,5,1,4,8,2,6,3,7] 2024-01-24T23:22:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 10 monclient(hunting): picked mon.noname-a con 0x7fbec4136cb0 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 10 monclient(hunting): picked mon.noname-h con 0x7fbec41377a0 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 10 monclient(hunting): picked mon.noname-f con 0x7fbec409bcc0 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec4064700 con 0x7fbec4136cb0 2024-01-24T23:22:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec409b650 con 0x7fbec409bcc0 2024-01-24T23:22:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec409c110 con 0x7fbec41377a0 2024-01-24T23:22:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 10 monclient(hunting): authenticate will time out at 1046.553686s 2024-01-24T23:22:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbec9c13640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fbec409bcc0 0x7fbec413f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:49156/0 (socket says 172.21.15.132:49156) 2024-01-24T23:22:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbec9c13640 1 -- 172.21.15.132:0/977613521 learned_addr learned my addr 172.21.15.132:0/977613521 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 1 -- 172.21.15.132:0/977613521 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 496491382 0 0) 0x7fbec409b650 con 0x7fbec409bcc0 2024-01-24T23:22:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 10 monclient(hunting): my global_id is 4672 2024-01-24T23:22:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 1 -- 172.21.15.132:0/977613521 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbeac003530 con 0x7fbec409bcc0 2024-01-24T23:22:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 1 -- 172.21.15.132:0/977613521 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2022614837 0 0) 0x7fbec409c110 con 0x7fbec41377a0 2024-01-24T23:22:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 10 monclient(hunting): my global_id is 4613 2024-01-24T23:22:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 1 -- 172.21.15.132:0/977613521 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbec409b650 con 0x7fbec41377a0 2024-01-24T23:22:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 1 -- 172.21.15.132:0/977613521 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1704772675 0 0) 0x7fbec409b650 con 0x7fbec41377a0 2024-01-24T23:22:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 1 -- 172.21.15.132:0/977613521 >> v1:172.21.15.132:6789/0 conn(0x7fbec409bcc0 legacy=0x7fbec413f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 1 -- 172.21.15.132:0/977613521 >> v1:172.21.15.115:6789/0 conn(0x7fbec4136cb0 legacy=0x7fbec4137090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 10 monclient: _finish_hunting 0 2024-01-24T23:22:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 1 monclient: found mon.noname-h 2024-01-24T23:22:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:22:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 1 -- 172.21.15.132:0/977613521 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec409b000 con 0x7fbec41377a0 2024-01-24T23:22:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 10 monclient: _finish_auth 0 2024-01-24T23:22:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 10 monclient: _check_auth_tickets 2024-01-24T23:22:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 1 -- 172.21.15.132:0/977613521 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbec0001b20 con 0x7fbec41377a0 2024-01-24T23:22:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:22:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 10 monclient: dump: 2024-01-24T23:22:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:39.238 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:39.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 5 monclient: authenticate success, global_id 4613 2024-01-24T23:22:39.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:39.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 1 -- 172.21.15.132:0/977613521 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbec0001dd0 con 0x7fbec41377a0 2024-01-24T23:22:39.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:39.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:39.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 4 set_mon_vals no callback set 2024-01-24T23:22:39.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 1 -- 172.21.15.132:0/977613521 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbec0003ce0 con 0x7fbec41377a0 2024-01-24T23:22:39.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:39.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:22:39.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbebbfff640 10 monclient: dump: 2024-01-24T23:22:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:39.241 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 10 monclient: shutdown 2024-01-24T23:22:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 1 -- 172.21.15.132:0/977613521 >> v1:172.21.15.132:6791/0 conn(0x7fbec41377a0 legacy=0x7fbec413c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 1 -- 172.21.15.132:0/977613521 shutdown_connections 2024-01-24T23:22:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 1 -- 172.21.15.132:0/977613521 >> 172.21.15.132:0/977613521 conn(0x7fbec40a9c10 msgr2=0x7fbec40a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 1 -- 172.21.15.132:0/977613521 shutdown_connections 2024-01-24T23:22:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 1 -- 172.21.15.132:0/977613521 wait complete. 2024-01-24T23:22:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.226+0000 7fbecb676640 10 monclient: build_initial_monmap 2024-01-24T23:22:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 10 monclient: monmap: 2024-01-24T23:22:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:39.242 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 1 Processor -- start 2024-01-24T23:22:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 1 -- start start 2024-01-24T23:22:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 10 monclient: init 2024-01-24T23:22:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 10 monclient: _add_conns ranks=[3,8,0,4,7,2,1,6,5] 2024-01-24T23:22:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 10 monclient(hunting): picked mon.noname-g con 0x7fbec409bcc0 addr v1:172.21.15.132:6790/0 2024-01-24T23:22:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 10 monclient(hunting): picked mon.noname-e con 0x7fbec4136cb0 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 10 monclient(hunting): picked mon.noname-a con 0x7fbec41377a0 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec409c6a0 con 0x7fbec41377a0 2024-01-24T23:22:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec4120050 con 0x7fbec409bcc0 2024-01-24T23:22:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec40a2800 con 0x7fbec4136cb0 2024-01-24T23:22:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 10 monclient(hunting): authenticate will time out at 1046.556931s 2024-01-24T23:22:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbec8c11640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7fbec4136cb0 legacy=0x7fbec41d0400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbec9412640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fbec409bcc0 0x7fbec41cb7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:37054/0 (socket says 172.21.15.132:37054) 2024-01-24T23:22:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbec9412640 1 -- 172.21.15.132:0/2589365651 learned_addr learned my addr 172.21.15.132:0/2589365651 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 1 -- 172.21.15.132:0/2589365651 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1823065072 0 0) 0x7fbec4120050 con 0x7fbec409bcc0 2024-01-24T23:22:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:22:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 10 monclient(hunting): my global_id is 4593 2024-01-24T23:22:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 1 -- 172.21.15.132:0/2589365651 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbeb0003530 con 0x7fbec409bcc0 2024-01-24T23:22:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 1 -- 172.21.15.132:0/2589365651 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3882828221 0 0) 0x7fbeb0003530 con 0x7fbec409bcc0 2024-01-24T23:22:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 1 -- 172.21.15.132:0/2589365651 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbec4120050 con 0x7fbec409bcc0 2024-01-24T23:22:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 1 -- 172.21.15.132:0/2589365651 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbebc001ac0 con 0x7fbec409bcc0 2024-01-24T23:22:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:22:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 10 monclient(hunting): dump: 2024-01-24T23:22:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:39.245 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 1 -- 172.21.15.132:0/2589365651 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2880674455 0 0) 0x7fbec409c6a0 con 0x7fbec41377a0 2024-01-24T23:22:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:22:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:39.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:39.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 10 monclient(hunting): my global_id is 4671 2024-01-24T23:22:39.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 1 -- 172.21.15.132:0/2589365651 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbeb0003530 con 0x7fbec41377a0 2024-01-24T23:22:39.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 1 -- 172.21.15.132:0/2589365651 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 625872309 0 0) 0x7fbec4120050 con 0x7fbec409bcc0 2024-01-24T23:22:39.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 1 -- 172.21.15.132:0/2589365651 >> v1:172.21.15.115:6793/0 conn(0x7fbec4136cb0 legacy=0x7fbec41d0400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:39.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 1 -- 172.21.15.132:0/2589365651 >> v1:172.21.15.115:6789/0 conn(0x7fbec41377a0 legacy=0x7fbec41d3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:39.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:22:39.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 1 monclient: found mon.g 2024-01-24T23:22:39.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:39.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:39.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 1 -- 172.21.15.132:0/2589365651 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec40a2550 con 0x7fbec409bcc0 2024-01-24T23:22:39.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 10 monclient: _finish_auth 0 2024-01-24T23:22:39.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:22:39.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:39.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 5 monclient: authenticate success, global_id 4593 2024-01-24T23:22:39.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 10 monclient: _renew_subs 2024-01-24T23:22:39.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:39.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 1 -- 172.21.15.132:0/2589365651 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec411fd70 con 0x7fbec409bcc0 2024-01-24T23:22:39.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 10 monclient: _renew_subs 2024-01-24T23:22:39.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:39.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 1 -- 172.21.15.132:0/2589365651 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec4130cc0 con 0x7fbec409bcc0 2024-01-24T23:22:39.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 1 -- 172.21.15.132:0/2589365651 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbebc003950 con 0x7fbec409bcc0 2024-01-24T23:22:39.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:39.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 1 -- 172.21.15.132:0/2589365651 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbebc003e80 con 0x7fbec409bcc0 2024-01-24T23:22:39.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:39.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbebb7fe640 4 set_mon_vals no callback set 2024-01-24T23:22:39.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:22:39.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 10 monclient: dump: 2024-01-24T23:22:39.247 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:39.247 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:39.247 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:39.248 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:39.248 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:39.248 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:39.248 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:39.248 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:39.248 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:39.248 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:39.248 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:39.248 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:39.248 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:39.248 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:39.248 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:39.248 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:39.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 1 -- 172.21.15.132:0/2589365651 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fbebc003fe0 con 0x7fbec409bcc0 2024-01-24T23:22:39.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:39.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:39.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:39.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbecb676640 1 -- 172.21.15.132:0/2589365651 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea4005090 con 0x7fbec409bcc0 2024-01-24T23:22:39.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbeb9ffb640 1 -- 172.21.15.132:0/2589365651 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(24..24 src has 1..24) v4 ==== 3857+0+0 (unknown 3668732552 0 0) 0x7fbebc0438c0 con 0x7fbec409bcc0 2024-01-24T23:22:39.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.230+0000 7fbec8c11640 10 monclient: get_auth_request con 0x7fbeb00370a0 auth_method 0 2024-01-24T23:22:39.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.234+0000 7fbeb9ffb640 1 -- 172.21.15.132:0/2589365651 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fbebc014900 con 0x7fbec409bcc0 2024-01-24T23:22:39.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.234+0000 7fbeb9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:39.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.234+0000 7fbeb9ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:39.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.390+0000 7fbecb676640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-01-24T23:22:39.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.390+0000 7fbecb676640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-01-24T23:22:39.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.390+0000 7fbecb676640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:39.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.390+0000 7fbecb676640 1 -- 172.21.15.132:0/2589365651 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"} v 0) v1 -- 0x7fbea40058c0 con 0x7fbec409bcc0 2024-01-24T23:22:39.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.390+0000 7fbeb9ffb640 1 -- 172.21.15.132:0/2589365651 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}]=-2 option 'pg_num_max' is not set on pool 'testpool1234' v24) v1 ==== 158+0+0 (unknown 2544661327 0 0) 0x7fbebc012fa0 con 0x7fbec409bcc0 2024-01-24T23:22:39.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.390+0000 7fbeb9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-01-24T23:22:39.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.390+0000 7fbeb9ffb640 10 monclient: _finish_command 2 = mon:2 option 'pg_num_max' is not set on pool 'testpool1234' 2024-01-24T23:22:39.392 INFO:tasks.workunit.client.0.smithi132.stderr:Error ENOENT: option 'pg_num_max' is not set on pool 'testpool1234' 2024-01-24T23:22:39.396 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:22:39.396 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool delete testpool1234 testpool1234 --yes-i-really-really-mean-it 2024-01-24T23:22:39.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.506+0000 7fc3ef0ef640 10 monclient: get_monmap_and_config 2024-01-24T23:22:39.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.506+0000 7fc3ef0ef640 10 monclient: build_initial_monmap 2024-01-24T23:22:39.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.506+0000 7fc3ef0ef640 10 monclient: monmap: 2024-01-24T23:22:39.510 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:39.511 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:39.511 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:39.512705+0000 2024-01-24T23:22:39.511 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:39.512705+0000 2024-01-24T23:22:39.511 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:39.511 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:39.511 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:39.511 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:39.511 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:39.511 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:39.511 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:39.511 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:39.511 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:39.511 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:39.511 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:39.511 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:39.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 1 Processor -- start 2024-01-24T23:22:39.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 1 -- start start 2024-01-24T23:22:39.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 10 monclient: init 2024-01-24T23:22:39.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:39.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 10 monclient: _add_conns ranks=[2,6,0,8,1,5,4,3,7] 2024-01-24T23:22:39.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 10 monclient(hunting): picked mon.noname-b con 0x7fc3e80a2e10 addr v1:172.21.15.115:6790/0 2024-01-24T23:22:39.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 10 monclient(hunting): picked mon.noname-d con 0x7fc3e8136f70 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:39.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 10 monclient(hunting): picked mon.noname-a con 0x7fc3e813d4f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:39.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e8064700 con 0x7fc3e813d4f0 2024-01-24T23:22:39.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e80a5580 con 0x7fc3e80a2e10 2024-01-24T23:22:39.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e80a0e10 con 0x7fc3e8136f70 2024-01-24T23:22:39.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:39.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 10 monclient(hunting): authenticate will time out at 1046.837340s 2024-01-24T23:22:39.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ece8b640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fc3e80a2e10 legacy=0x7fc3e80a31f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:22:39.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e7fff640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fc3e8136f70 0x7fc3e813bdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:35970/0 (socket says 172.21.15.132:35970) 2024-01-24T23:22:39.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e7fff640 1 -- 172.21.15.132:0/1127726697 learned_addr learned my addr 172.21.15.132:0/1127726697 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:39.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ed68c640 1 --1- 172.21.15.132:0/1127726697 >> v1:172.21.15.115:6789/0 conn(0x7fc3e813d4f0 0x7fc3e813f8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:58746/0 (socket says 172.21.15.132:58746) 2024-01-24T23:22:39.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 1 -- 172.21.15.132:0/1127726697 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2273970174 0 0) 0x7fc3e8064700 con 0x7fc3e813d4f0 2024-01-24T23:22:39.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:22:39.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:39.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:39.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 10 monclient(hunting): my global_id is 4680 2024-01-24T23:22:39.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 1 -- 172.21.15.132:0/1127726697 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3d0003530 con 0x7fc3e813d4f0 2024-01-24T23:22:39.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 1 -- 172.21.15.132:0/1127726697 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1402835271 0 0) 0x7fc3e80a0e10 con 0x7fc3e8136f70 2024-01-24T23:22:39.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:22:39.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:39.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:39.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 10 monclient(hunting): my global_id is 4686 2024-01-24T23:22:39.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 1 -- 172.21.15.132:0/1127726697 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3e8064700 con 0x7fc3e8136f70 2024-01-24T23:22:39.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 1 -- 172.21.15.132:0/1127726697 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2219445782 0 0) 0x7fc3d0003530 con 0x7fc3e813d4f0 2024-01-24T23:22:39.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 1 -- 172.21.15.132:0/1127726697 >> v1:172.21.15.115:6792/0 conn(0x7fc3e8136f70 legacy=0x7fc3e813bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:39.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 1 -- 172.21.15.132:0/1127726697 >> v1:172.21.15.115:6790/0 conn(0x7fc3e80a2e10 legacy=0x7fc3e80a31f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:39.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 10 monclient: _finish_hunting 0 2024-01-24T23:22:39.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 1 monclient: found mon.noname-a 2024-01-24T23:22:39.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:39.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:22:39.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 1 -- 172.21.15.132:0/1127726697 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e80a10f0 con 0x7fc3e813d4f0 2024-01-24T23:22:39.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 10 monclient: _finish_auth 0 2024-01-24T23:22:39.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 10 monclient: _check_auth_tickets 2024-01-24T23:22:39.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:39.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 1 -- 172.21.15.132:0/1127726697 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc3e0001a70 con 0x7fc3e813d4f0 2024-01-24T23:22:39.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 5 monclient: authenticate success, global_id 4680 2024-01-24T23:22:39.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:39.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:39.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:22:39.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 10 monclient: dump: 2024-01-24T23:22:39.521 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:39.521 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:39.521 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:39.521 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:39.521 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:39.521 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:39.521 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:39.521 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:39.521 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:39.521 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:39.521 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:39.522 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:39.522 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:39.522 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:39.522 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:39.522 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:39.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:39.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 1 -- 172.21.15.132:0/1127726697 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc3e0001c90 con 0x7fc3e813d4f0 2024-01-24T23:22:39.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:39.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:39.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 4 set_mon_vals no callback set 2024-01-24T23:22:39.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 1 -- 172.21.15.132:0/1127726697 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc3e0004b70 con 0x7fc3e813d4f0 2024-01-24T23:22:39.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:39.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:22:39.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3e77fe640 10 monclient: dump: 2024-01-24T23:22:39.522 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:39.523 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:39.523 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:39.523 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:39.523 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:39.523 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:39.523 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:39.523 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:39.523 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:39.523 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:39.523 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:39.523 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:39.523 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:39.523 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:39.523 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:39.523 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:39.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 10 monclient: shutdown 2024-01-24T23:22:39.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:39.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 1 -- 172.21.15.132:0/1127726697 >> v1:172.21.15.115:6789/0 conn(0x7fc3e813d4f0 legacy=0x7fc3e813f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:39.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 1 -- 172.21.15.132:0/1127726697 shutdown_connections 2024-01-24T23:22:39.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 1 -- 172.21.15.132:0/1127726697 >> 172.21.15.132:0/1127726697 conn(0x7fc3e80a9b50 msgr2=0x7fc3e80a5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:39.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 1 -- 172.21.15.132:0/1127726697 shutdown_connections 2024-01-24T23:22:39.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.510+0000 7fc3ef0ef640 1 -- 172.21.15.132:0/1127726697 wait complete. 2024-01-24T23:22:39.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 10 monclient: build_initial_monmap 2024-01-24T23:22:39.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 10 monclient: monmap: 2024-01-24T23:22:39.524 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:39.524 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:39.524 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:39.524 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:39.524 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:39.524 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:39.524 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:39.525 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:39.525 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:39.525 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:39.525 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:39.525 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:39.525 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:39.525 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:39.525 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:39.525 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:39.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 1 Processor -- start 2024-01-24T23:22:39.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 1 -- start start 2024-01-24T23:22:39.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 10 monclient: init 2024-01-24T23:22:39.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:39.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 10 monclient: _add_conns ranks=[2,5,6,7,3,1,0,8,4] 2024-01-24T23:22:39.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 10 monclient(hunting): picked mon.noname-b con 0x7fc3e80a2e10 addr v1:172.21.15.115:6790/0 2024-01-24T23:22:39.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 10 monclient(hunting): picked mon.noname-h con 0x7fc3e8136f70 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:39.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 10 monclient(hunting): picked mon.noname-d con 0x7fc3e813d4f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:39.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e8139600 con 0x7fc3e80a2e10 2024-01-24T23:22:39.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e80a2be0 con 0x7fc3e8136f70 2024-01-24T23:22:39.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e8139b30 con 0x7fc3e813d4f0 2024-01-24T23:22:39.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:39.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 10 monclient(hunting): authenticate will time out at 1046.841214s 2024-01-24T23:22:39.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e7fff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fc3e8136f70 0x7fc3e809ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:33968/0 (socket says 172.21.15.132:33968) 2024-01-24T23:22:39.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e7fff640 1 -- 172.21.15.132:0/2078355392 learned_addr learned my addr 172.21.15.132:0/2078355392 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:39.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ece8b640 1 -- 172.21.15.132:0/2078355392 >> v1:172.21.15.115:6790/0 conn(0x7fc3e80a2e10 legacy=0x7fc3e809e5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:22:39.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 1 -- 172.21.15.132:0/2078355392 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2600416357 0 0) 0x7fc3e80a2be0 con 0x7fc3e8136f70 2024-01-24T23:22:39.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 10 monclient(hunting): _negotiate 2024-01-24T23:22:39.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:39.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:39.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 10 monclient(hunting): my global_id is 4622 2024-01-24T23:22:39.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 1 -- 172.21.15.132:0/2078355392 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3d4003530 con 0x7fc3e8136f70 2024-01-24T23:22:39.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 1 -- 172.21.15.132:0/2078355392 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1949245535 0 0) 0x7fc3d4003530 con 0x7fc3e8136f70 2024-01-24T23:22:39.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 1 -- 172.21.15.132:0/2078355392 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc3e80a2be0 con 0x7fc3e8136f70 2024-01-24T23:22:39.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 1 -- 172.21.15.132:0/2078355392 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc3d80014b0 con 0x7fc3e8136f70 2024-01-24T23:22:39.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:39.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:22:39.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 10 monclient(hunting): dump: 2024-01-24T23:22:39.527 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:39.527 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:39.527 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:39.527 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:39.527 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:39.527 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:39.527 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:39.527 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:39.527 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:39.528 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:39.528 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:39.528 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:39.528 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:39.528 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:39.528 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:39.528 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:39.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:39.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 1 -- 172.21.15.132:0/2078355392 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1924224051 0 0) 0x7fc3e8139b30 con 0x7fc3e813d4f0 2024-01-24T23:22:39.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 10 monclient(hunting): _negotiate 2024-01-24T23:22:39.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:39.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:39.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 10 monclient(hunting): my global_id is 4695 2024-01-24T23:22:39.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 1 -- 172.21.15.132:0/2078355392 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3d4003530 con 0x7fc3e813d4f0 2024-01-24T23:22:39.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 1 -- 172.21.15.132:0/2078355392 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2031383669 0 0) 0x7fc3e80a2be0 con 0x7fc3e8136f70 2024-01-24T23:22:39.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 1 -- 172.21.15.132:0/2078355392 >> v1:172.21.15.115:6792/0 conn(0x7fc3e813d4f0 legacy=0x7fc3e81d36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:39.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 1 -- 172.21.15.132:0/2078355392 >> v1:172.21.15.115:6790/0 conn(0x7fc3e80a2e10 legacy=0x7fc3e809e5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:39.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 10 monclient: _finish_hunting 0 2024-01-24T23:22:39.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 1 monclient: found mon.h 2024-01-24T23:22:39.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:39.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:39.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 1 -- 172.21.15.132:0/2078355392 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e8138680 con 0x7fc3e8136f70 2024-01-24T23:22:39.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 10 monclient: _finish_auth 0 2024-01-24T23:22:39.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 10 monclient: _check_auth_tickets 2024-01-24T23:22:39.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:39.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 5 monclient: authenticate success, global_id 4622 2024-01-24T23:22:39.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 10 monclient: _renew_subs 2024-01-24T23:22:39.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:39.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 1 -- 172.21.15.132:0/2078355392 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e80a57b0 con 0x7fc3e8136f70 2024-01-24T23:22:39.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 10 monclient: _renew_subs 2024-01-24T23:22:39.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:39.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 1 -- 172.21.15.132:0/2078355392 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e8139e10 con 0x7fc3e8136f70 2024-01-24T23:22:39.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 1 -- 172.21.15.132:0/2078355392 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc3d8004aa0 con 0x7fc3e8136f70 2024-01-24T23:22:39.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:39.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 1 -- 172.21.15.132:0/2078355392 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc3d8004f50 con 0x7fc3e8136f70 2024-01-24T23:22:39.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:39.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e6ffd640 4 set_mon_vals no callback set 2024-01-24T23:22:39.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:22:39.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 10 monclient: dump: 2024-01-24T23:22:39.530 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:39.530 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:39.530 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:39.530 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:39.530 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:39.530 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:39.530 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:39.531 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:39.531 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:39.531 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:39.531 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:39.531 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:39.531 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:39.531 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:39.531 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:39.531 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:39.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 1 -- 172.21.15.132:0/2078355392 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fc3d800f9d0 con 0x7fc3e8136f70 2024-01-24T23:22:39.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:39.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:39.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:39.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ef0ef640 1 -- 172.21.15.132:0/2078355392 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3c8005090 con 0x7fc3e8136f70 2024-01-24T23:22:39.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3e57fa640 1 -- 172.21.15.132:0/2078355392 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(24..24 src has 1..24) v4 ==== 3857+0+0 (unknown 3668732552 0 0) 0x7fc3d8043970 con 0x7fc3e8136f70 2024-01-24T23:22:39.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.514+0000 7fc3ece8b640 10 monclient: get_auth_request con 0x7fc3d4036fb0 auth_method 0 2024-01-24T23:22:39.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.518+0000 7fc3e57fa640 1 -- 172.21.15.132:0/2078355392 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fc3d8010f20 con 0x7fc3e8136f70 2024-01-24T23:22:39.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.518+0000 7fc3e57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:39.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.518+0000 7fc3e57fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:39.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.678+0000 7fc3ef0ef640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:39.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.678+0000 7fc3ef0ef640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:39.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.678+0000 7fc3ef0ef640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:39.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:39.678+0000 7fc3ef0ef640 1 -- 172.21.15.132:0/2078355392 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc3c8005340 con 0x7fc3e8136f70 2024-01-24T23:22:40.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.110+0000 7fc3e57fa640 1 -- 172.21.15.132:0/2078355392 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}]=0 pool 'testpool1234' removed v25) v1 ==== 176+0+0 (unknown 2731928359 0 0) 0x7fc3d8012fa0 con 0x7fc3e8136f70 2024-01-24T23:22:40.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.110+0000 7fc3e57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:40.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.110+0000 7fc3e57fa640 10 monclient: _finish_command 2 = system:0 pool 'testpool1234' removed 2024-01-24T23:22:40.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.210+0000 7fc3ef0ef640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:40.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.210+0000 7fc3ef0ef640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:40.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.210+0000 7fc3ef0ef640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:40.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.210+0000 7fc3ef0ef640 1 -- 172.21.15.132:0/2078355392 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc3c80010a0 con 0x7fc3e8136f70 2024-01-24T23:22:40.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.210+0000 7fc3e57fa640 1 -- 172.21.15.132:0/2078355392 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}]=0 pool 'testpool1234' does not exist v25) v1 ==== 183+0+0 (unknown 3991880879 0 0) 0x7fc3d8014720 con 0x7fc3e8136f70 2024-01-24T23:22:40.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.210+0000 7fc3e57fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:40.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.210+0000 7fc3e57fa640 10 monclient: _finish_command 3 = system:0 pool 'testpool1234' does not exist 2024-01-24T23:22:40.213 INFO:tasks.workunit.client.0.smithi132.stderr:pool 'testpool1234' does not exist 2024-01-24T23:22:40.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.214+0000 7fc3ef0ef640 1 -- 172.21.15.132:0/2078355392 >> v1:172.21.15.132:6812/1835401922 conn(0x7fc3d4036fb0 legacy=0x7fc3d4039470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:40.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.214+0000 7fc3ef0ef640 10 monclient: shutdown 2024-01-24T23:22:40.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.214+0000 7fc3ef0ef640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:40.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.214+0000 7fc3ef0ef640 1 -- 172.21.15.132:0/2078355392 >> v1:172.21.15.132:6791/0 conn(0x7fc3e8136f70 legacy=0x7fc3e809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:40.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.214+0000 7fc3ef0ef640 1 -- 172.21.15.132:0/2078355392 shutdown_connections 2024-01-24T23:22:40.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.214+0000 7fc3ef0ef640 1 -- 172.21.15.132:0/2078355392 >> 172.21.15.132:0/2078355392 conn(0x7fc3e80a9b50 msgr2=0x7fc3e80a4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:40.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.214+0000 7fc3ef0ef640 1 -- 172.21.15.132:0/2078355392 shutdown_connections 2024-01-24T23:22:40.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.214+0000 7fc3ef0ef640 1 -- 172.21.15.132:0/2078355392 wait complete. 2024-01-24T23:22:40.230 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd pool create foo 123 123 invalid foo-profile foo-rule 2024-01-24T23:22:40.230 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:22:40.230 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool create foo 123 123 invalid foo-profile foo-rule 2024-01-24T23:22:40.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7faa0777f640 10 monclient: get_monmap_and_config 2024-01-24T23:22:40.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7faa0777f640 10 monclient: build_initial_monmap 2024-01-24T23:22:40.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7faa0777f640 10 monclient: monmap: 2024-01-24T23:22:40.352 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:40.352 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:40.352 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:40.354517+0000 2024-01-24T23:22:40.353 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:40.354517+0000 2024-01-24T23:22:40.353 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:40.353 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:40.353 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:40.353 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:40.353 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:40.353 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:40.353 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:40.353 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:40.353 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:40.353 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:40.353 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:40.353 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:40.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7faa0777f640 1 Processor -- start 2024-01-24T23:22:40.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7faa0777f640 1 -- start start 2024-01-24T23:22:40.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7faa0777f640 10 monclient: init 2024-01-24T23:22:40.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7faa0777f640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:40.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7faa0777f640 10 monclient: _add_conns ranks=[8,3,2,5,0,4,7,6,1] 2024-01-24T23:22:40.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7faa0777f640 10 monclient(hunting): picked mon.noname-e con 0x7faa00135f60 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:40.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7faa0777f640 10 monclient(hunting): picked mon.noname-g con 0x7faa0009a9d0 addr v1:172.21.15.132:6790/0 2024-01-24T23:22:40.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7faa0777f640 10 monclient(hunting): picked mon.noname-b con 0x7faa0013d6d0 addr v1:172.21.15.115:6790/0 2024-01-24T23:22:40.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7faa0777f640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa0004f0a0 con 0x7faa0013d6d0 2024-01-24T23:22:40.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7faa0777f640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa0009b200 con 0x7faa0009a9d0 2024-01-24T23:22:40.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7faa0777f640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa00136a50 con 0x7faa00135f60 2024-01-24T23:22:40.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7faa0777f640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:40.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7faa0777f640 10 monclient(hunting): authenticate will time out at 1047.679161s 2024-01-24T23:22:40.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7faa04d1a640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7faa0009a9d0 0x7faa0013bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:37056/0 (socket says 172.21.15.132:37056) 2024-01-24T23:22:40.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7faa04d1a640 1 -- 172.21.15.132:0/2495886048 learned_addr learned my addr 172.21.15.132:0/2495886048 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:40.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7faa05d1c640 1 -- 172.21.15.132:0/2495886048 >> v1:172.21.15.115:6790/0 conn(0x7faa0013d6d0 legacy=0x7faa0013fac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:22:40.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7faa0551b640 1 -- 172.21.15.132:0/2495886048 >> v1:172.21.15.115:6793/0 conn(0x7faa00135f60 legacy=0x7faa00136340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:40.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7fa9f7fff640 1 -- 172.21.15.132:0/2495886048 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1104093491 0 0) 0x7faa0009b200 con 0x7faa0009a9d0 2024-01-24T23:22:40.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7fa9f7fff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:40.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7fa9f7fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:40.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7fa9f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:40.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7fa9f7fff640 10 monclient(hunting): my global_id is 4602 2024-01-24T23:22:40.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7fa9f7fff640 1 -- 172.21.15.132:0/2495886048 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9ec003530 con 0x7faa0009a9d0 2024-01-24T23:22:40.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7fa9f7fff640 1 -- 172.21.15.132:0/2495886048 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2694372856 0 0) 0x7fa9ec003530 con 0x7faa0009a9d0 2024-01-24T23:22:40.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7fa9f7fff640 1 -- 172.21.15.132:0/2495886048 >> v1:172.21.15.115:6793/0 conn(0x7faa00135f60 legacy=0x7faa00136340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:40.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7fa9f7fff640 1 -- 172.21.15.132:0/2495886048 >> v1:172.21.15.115:6790/0 conn(0x7faa0013d6d0 legacy=0x7faa0013fac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:40.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7fa9f7fff640 10 monclient: _finish_hunting 0 2024-01-24T23:22:40.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7fa9f7fff640 1 monclient: found mon.noname-g 2024-01-24T23:22:40.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7fa9f7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:40.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7fa9f7fff640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:22:40.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7fa9f7fff640 1 -- 172.21.15.132:0/2495886048 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0009b470 con 0x7faa0009a9d0 2024-01-24T23:22:40.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7fa9f7fff640 10 monclient: _finish_auth 0 2024-01-24T23:22:40.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7fa9f7fff640 10 monclient: _check_auth_tickets 2024-01-24T23:22:40.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7fa9f7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:40.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7fa9f7fff640 1 -- 172.21.15.132:0/2495886048 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa9fc0017c0 con 0x7faa0009a9d0 2024-01-24T23:22:40.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7fa9f7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:40.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7fa9f7fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:22:40.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7fa9f7fff640 10 monclient: dump: 2024-01-24T23:22:40.362 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:40.362 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:40.362 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:40.363 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:40.363 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:40.363 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:40.363 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:40.363 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:40.363 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:40.363 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:40.363 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:40.363 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:40.363 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:40.363 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:40.363 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:40.363 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:40.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7faa0777f640 5 monclient: authenticate success, global_id 4602 2024-01-24T23:22:40.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7faa0777f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:40.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7fa9f7fff640 1 -- 172.21.15.132:0/2495886048 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa9fc001ae0 con 0x7faa0009a9d0 2024-01-24T23:22:40.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7fa9f7fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:40.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.350+0000 7fa9f7fff640 1 -- 172.21.15.132:0/2495886048 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa9fc004bf0 con 0x7faa0009a9d0 2024-01-24T23:22:40.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:40.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:40.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 4 set_mon_vals no callback set 2024-01-24T23:22:40.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f7fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:22:40.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f7fff640 10 monclient: dump: 2024-01-24T23:22:40.364 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:40.364 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:40.364 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:40.364 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:40.364 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:40.364 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:40.364 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:40.364 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:40.365 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:40.365 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:40.365 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:40.365 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:40.365 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:40.365 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:40.365 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:40.365 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:40.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 10 monclient: shutdown 2024-01-24T23:22:40.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:40.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 1 -- 172.21.15.132:0/2495886048 >> v1:172.21.15.132:6790/0 conn(0x7faa0009a9d0 legacy=0x7faa0013bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:40.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 1 -- 172.21.15.132:0/2495886048 shutdown_connections 2024-01-24T23:22:40.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 1 -- 172.21.15.132:0/2495886048 >> 172.21.15.132:0/2495886048 conn(0x7faa000a9d90 msgr2=0x7faa000a5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:40.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 1 -- 172.21.15.132:0/2495886048 shutdown_connections 2024-01-24T23:22:40.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 1 -- 172.21.15.132:0/2495886048 wait complete. 2024-01-24T23:22:40.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 10 monclient: build_initial_monmap 2024-01-24T23:22:40.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 10 monclient: monmap: 2024-01-24T23:22:40.366 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:40.366 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:40.366 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:40.366 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:40.366 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:40.366 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:40.366 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:40.366 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:40.366 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:40.366 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:40.366 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:40.366 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:40.366 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:40.367 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:40.367 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:40.367 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:40.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 1 Processor -- start 2024-01-24T23:22:40.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 1 -- start start 2024-01-24T23:22:40.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 10 monclient: init 2024-01-24T23:22:40.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:40.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 10 monclient: _add_conns ranks=[1,8,5,6,4,0,2,3,7] 2024-01-24T23:22:40.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 10 monclient(hunting): picked mon.noname-f con 0x7faa0009a9d0 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:40.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 10 monclient(hunting): picked mon.noname-e con 0x7faa00135f60 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:40.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 10 monclient(hunting): picked mon.noname-h con 0x7faa0013d6d0 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:40.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa000a5b50 con 0x7faa0009a9d0 2024-01-24T23:22:40.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa000a3720 con 0x7faa0013d6d0 2024-01-24T23:22:40.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa0009ca30 con 0x7faa00135f60 2024-01-24T23:22:40.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:40.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 10 monclient(hunting): authenticate will time out at 1047.682178s 2024-01-24T23:22:40.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0551b640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7faa0009a9d0 0x7faa0013c1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:49166/0 (socket says 172.21.15.132:49166) 2024-01-24T23:22:40.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0551b640 1 -- 172.21.15.132:0/2468467233 learned_addr learned my addr 172.21.15.132:0/2468467233 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:40.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa04d1a640 1 -- 172.21.15.132:0/2468467233 >> v1:172.21.15.115:6793/0 conn(0x7faa00135f60 legacy=0x7faa001d01e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:40.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 1 -- 172.21.15.132:0/2468467233 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 9672552 0 0) 0x7faa000a5b50 con 0x7faa0009a9d0 2024-01-24T23:22:40.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:22:40.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:40.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:40.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 10 monclient(hunting): my global_id is 4681 2024-01-24T23:22:40.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 1 -- 172.21.15.132:0/2468467233 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9e4003530 con 0x7faa0009a9d0 2024-01-24T23:22:40.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 1 -- 172.21.15.132:0/2468467233 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2046462696 0 0) 0x7faa000a3720 con 0x7faa0013d6d0 2024-01-24T23:22:40.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:22:40.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:40.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:40.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 10 monclient(hunting): my global_id is 4631 2024-01-24T23:22:40.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 1 -- 172.21.15.132:0/2468467233 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa000a5b50 con 0x7faa0013d6d0 2024-01-24T23:22:40.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 1 -- 172.21.15.132:0/2468467233 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3494163232 0 0) 0x7faa000a5b50 con 0x7faa0013d6d0 2024-01-24T23:22:40.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 1 -- 172.21.15.132:0/2468467233 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faa000a3720 con 0x7faa0013d6d0 2024-01-24T23:22:40.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 1 -- 172.21.15.132:0/2468467233 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3054363667 0 0) 0x7fa9e4003530 con 0x7faa0009a9d0 2024-01-24T23:22:40.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 1 -- 172.21.15.132:0/2468467233 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faa000a5b50 con 0x7faa0009a9d0 2024-01-24T23:22:40.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 1 -- 172.21.15.132:0/2468467233 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa9f8004460 con 0x7faa0013d6d0 2024-01-24T23:22:40.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:40.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:22:40.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 10 monclient(hunting): dump: 2024-01-24T23:22:40.369 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:40.369 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:40.369 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:40.369 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:40.369 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:40.369 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:40.369 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:40.369 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:40.369 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:40.369 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:40.370 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:40.370 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:40.370 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:40.370 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:40.370 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:40.370 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:40.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:40.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 1 -- 172.21.15.132:0/2468467233 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa9f00042f0 con 0x7faa0009a9d0 2024-01-24T23:22:40.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:40.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:22:40.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 10 monclient(hunting): dump: 2024-01-24T23:22:40.370 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:40.370 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:40.370 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:40.370 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:40.370 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:40.370 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:40.371 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:40.371 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:40.371 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:40.371 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:40.371 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:40.371 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:40.371 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:40.371 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:40.371 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:40.371 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:40.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 1 -- 172.21.15.132:0/2468467233 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2982480597 0 0) 0x7faa000a5b50 con 0x7faa0009a9d0 2024-01-24T23:22:40.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 1 -- 172.21.15.132:0/2468467233 >> v1:172.21.15.115:6793/0 conn(0x7faa00135f60 legacy=0x7faa001d01e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:40.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 1 -- 172.21.15.132:0/2468467233 >> v1:172.21.15.132:6791/0 conn(0x7faa0013d6d0 legacy=0x7faa001d3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:40.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:22:40.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 1 monclient: found mon.f 2024-01-24T23:22:40.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:40.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:40.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 1 -- 172.21.15.132:0/2468467233 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa00138c70 con 0x7faa0009a9d0 2024-01-24T23:22:40.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 10 monclient: _finish_auth 0 2024-01-24T23:22:40.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:22:40.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:40.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 5 monclient: authenticate success, global_id 4681 2024-01-24T23:22:40.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 10 monclient: _renew_subs 2024-01-24T23:22:40.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:40.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 1 -- 172.21.15.132:0/2468467233 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa001376e0 con 0x7faa0009a9d0 2024-01-24T23:22:40.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 10 monclient: _renew_subs 2024-01-24T23:22:40.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:40.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7faa0777f640 1 -- 172.21.15.132:0/2468467233 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7faa0013c8b0 con 0x7faa0009a9d0 2024-01-24T23:22:40.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 1 -- 172.21.15.132:0/2468467233 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa9f0004490 con 0x7faa0009a9d0 2024-01-24T23:22:40.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:40.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 1 -- 172.21.15.132:0/2468467233 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa9f0004d20 con 0x7faa0009a9d0 2024-01-24T23:22:40.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:40.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f77fe640 4 set_mon_vals no callback set 2024-01-24T23:22:40.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:22:40.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.354+0000 7fa9f5ffb640 10 monclient: dump: 2024-01-24T23:22:40.373 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:40.373 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:40.373 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:40.373 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:40.373 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:40.373 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:40.373 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:40.373 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:40.374 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:40.374 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:40.374 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:40.374 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:40.374 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:40.374 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:40.374 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:40.374 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:40.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.358+0000 7fa9f5ffb640 1 -- 172.21.15.132:0/2468467233 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fa9f000f820 con 0x7faa0009a9d0 2024-01-24T23:22:40.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.358+0000 7faa0777f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:40.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.358+0000 7faa0777f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:40.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.358+0000 7faa0777f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:40.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.358+0000 7fa9f5ffb640 1 -- 172.21.15.132:0/2468467233 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(25..25 src has 1..25) v4 ==== 3506+0+0 (unknown 1380043353 0 0) 0x7fa9f0043670 con 0x7faa0009a9d0 2024-01-24T23:22:40.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.358+0000 7faa0777f640 1 -- 172.21.15.132:0/2468467233 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e0005090 con 0x7faa0009a9d0 2024-01-24T23:22:40.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.358+0000 7faa04d1a640 10 monclient: get_auth_request con 0x7fa9e4037670 auth_method 0 2024-01-24T23:22:40.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.362+0000 7fa9f5ffb640 1 -- 172.21.15.132:0/2468467233 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fa9f00147b0 con 0x7faa0009a9d0 2024-01-24T23:22:40.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.362+0000 7fa9f5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:40.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.362+0000 7fa9f5ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:40.518 INFO:tasks.workunit.client.0.smithi132.stderr:foo-rule not valid: foo-rule not one of 'true', 'false' 2024-01-24T23:22:40.518 INFO:tasks.workunit.client.0.smithi132.stderr:Invalid command: unused arguments: ['foo-rule'] 2024-01-24T23:22:40.518 INFO:tasks.workunit.client.0.smithi132.stderr:osd pool create [] [] [] [] [] [] [] [] [] [] [--bulk] [] [] [--yes-i-really-mean-it] : create pool 2024-01-24T23:22:40.518 INFO:tasks.workunit.client.0.smithi132.stderr:Error EINVAL: invalid command 2024-01-24T23:22:40.520 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:22:40.521 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool create foo 123 123 replicated 2024-01-24T23:22:40.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed7d4c640 10 monclient: get_monmap_and_config 2024-01-24T23:22:40.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed7d4c640 10 monclient: build_initial_monmap 2024-01-24T23:22:40.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed7d4c640 10 monclient: monmap: 2024-01-24T23:22:40.636 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:40.636 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:40.636 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:40.638111+0000 2024-01-24T23:22:40.636 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:40.638111+0000 2024-01-24T23:22:40.636 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:40.636 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:40.636 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:40.636 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:40.636 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:40.636 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:40.636 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:40.637 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:40.637 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:40.637 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:40.637 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:40.637 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:40.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed7d4c640 1 Processor -- start 2024-01-24T23:22:40.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed7d4c640 1 -- start start 2024-01-24T23:22:40.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed7d4c640 10 monclient: init 2024-01-24T23:22:40.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed7d4c640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:40.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed7d4c640 10 monclient: _add_conns ranks=[1,2,4,8,3,5,7,0,6] 2024-01-24T23:22:40.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed7d4c640 10 monclient(hunting): picked mon.noname-f con 0x7f5ed0134af0 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:40.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed7d4c640 10 monclient(hunting): picked mon.noname-b con 0x7f5ed00a1b50 addr v1:172.21.15.115:6790/0 2024-01-24T23:22:40.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed7d4c640 10 monclient(hunting): picked mon.noname-c con 0x7f5ed01355e0 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:40.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed7d4c640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed0064700 con 0x7f5ed0134af0 2024-01-24T23:22:40.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed7d4c640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed006f860 con 0x7f5ed00a1b50 2024-01-24T23:22:40.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed7d4c640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed0071f80 con 0x7f5ed01355e0 2024-01-24T23:22:40.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed7d4c640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:40.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed7d4c640 10 monclient(hunting): authenticate will time out at 1047.963312s 2024-01-24T23:22:40.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed5ae8640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f5ed0134af0 0x7f5ed0134ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:49176/0 (socket says 172.21.15.132:49176) 2024-01-24T23:22:40.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed5ae8640 1 -- 172.21.15.132:0/190228989 learned_addr learned my addr 172.21.15.132:0/190228989 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:40.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed52e7640 1 -- 172.21.15.132:0/190228989 >> v1:172.21.15.115:6790/0 conn(0x7f5ed00a1b50 legacy=0x7f5ed013a090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:22:40.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed4ae6640 1 -- 172.21.15.132:0/190228989 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3371860735 0 0) 0x7f5ed0064700 con 0x7f5ed0134af0 2024-01-24T23:22:40.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed4ae6640 10 monclient(hunting): _negotiate 2024-01-24T23:22:40.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed4ae6640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:40.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed4ae6640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:40.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed4ae6640 10 monclient(hunting): my global_id is 4690 2024-01-24T23:22:40.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed4ae6640 1 -- 172.21.15.132:0/190228989 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ebc003530 con 0x7f5ed0134af0 2024-01-24T23:22:40.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.634+0000 7f5ed4ae6640 1 -- 172.21.15.132:0/190228989 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2691442033 0 0) 0x7f5ebc003530 con 0x7f5ed0134af0 2024-01-24T23:22:40.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed4ae6640 1 -- 172.21.15.132:0/190228989 >> v1:172.21.15.115:6791/0 conn(0x7f5ed01355e0 legacy=0x7f5ed013d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:40.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed4ae6640 1 -- 172.21.15.132:0/190228989 >> v1:172.21.15.115:6790/0 conn(0x7f5ed00a1b50 legacy=0x7f5ed013a090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:40.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed4ae6640 10 monclient: _finish_hunting 0 2024-01-24T23:22:40.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed4ae6640 1 monclient: found mon.noname-f 2024-01-24T23:22:40.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed4ae6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:40.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed4ae6640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:22:40.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed4ae6640 1 -- 172.21.15.132:0/190228989 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed00a36a0 con 0x7f5ed0134af0 2024-01-24T23:22:40.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed4ae6640 10 monclient: _finish_auth 0 2024-01-24T23:22:40.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed4ae6640 10 monclient: _check_auth_tickets 2024-01-24T23:22:40.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed4ae6640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:40.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 5 monclient: authenticate success, global_id 4690 2024-01-24T23:22:40.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:40.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed4ae6640 1 -- 172.21.15.132:0/190228989 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5ecc0017c0 con 0x7f5ed0134af0 2024-01-24T23:22:40.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed4ae6640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:40.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed4ae6640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:22:40.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed4ae6640 10 monclient: dump: 2024-01-24T23:22:40.647 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:40.647 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:40.647 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:40.647 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:40.647 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:40.647 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:40.647 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:40.647 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:40.648 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:40.648 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:40.648 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:40.648 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:40.648 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:40.648 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:40.648 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:40.648 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:40.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:40.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed4ae6640 1 -- 172.21.15.132:0/190228989 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5ecc001ae0 con 0x7f5ed0134af0 2024-01-24T23:22:40.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed4ae6640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:40.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:40.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed4ae6640 1 -- 172.21.15.132:0/190228989 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5ecc004bf0 con 0x7f5ed0134af0 2024-01-24T23:22:40.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 4 set_mon_vals no callback set 2024-01-24T23:22:40.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed4ae6640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:40.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed4ae6640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:22:40.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed4ae6640 10 monclient: dump: 2024-01-24T23:22:40.649 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:40.649 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:40.649 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:40.649 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:40.649 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:40.649 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:40.649 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:40.649 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:40.649 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:40.649 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:40.649 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:40.649 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:40.649 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:40.649 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:40.649 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:40.650 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:40.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 10 monclient: shutdown 2024-01-24T23:22:40.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:40.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 1 -- 172.21.15.132:0/190228989 >> v1:172.21.15.132:6789/0 conn(0x7f5ed0134af0 legacy=0x7f5ed0134ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:40.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 1 -- 172.21.15.132:0/190228989 shutdown_connections 2024-01-24T23:22:40.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 1 -- 172.21.15.132:0/190228989 >> 172.21.15.132:0/190228989 conn(0x7f5ed006d9e0 msgr2=0x7f5ed00a7150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:40.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 1 -- 172.21.15.132:0/190228989 shutdown_connections 2024-01-24T23:22:40.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 1 -- 172.21.15.132:0/190228989 wait complete. 2024-01-24T23:22:40.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 10 monclient: build_initial_monmap 2024-01-24T23:22:40.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 10 monclient: monmap: 2024-01-24T23:22:40.650 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:40.650 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:40.650 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:40.650 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:40.650 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:40.650 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:40.651 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:40.651 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:40.651 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:40.651 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:40.651 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:40.651 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:40.651 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:40.651 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:40.651 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:40.651 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:40.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 1 Processor -- start 2024-01-24T23:22:40.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 1 -- start start 2024-01-24T23:22:40.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 10 monclient: init 2024-01-24T23:22:40.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:40.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 10 monclient: _add_conns ranks=[3,5,4,6,0,8,7,1,2] 2024-01-24T23:22:40.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 10 monclient(hunting): picked mon.noname-g con 0x7f5ed00a1b50 addr v1:172.21.15.132:6790/0 2024-01-24T23:22:40.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 10 monclient(hunting): picked mon.noname-h con 0x7f5ed0134af0 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:40.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 10 monclient(hunting): picked mon.noname-c con 0x7f5ed01355e0 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:40.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed0070300 con 0x7f5ed00a1b50 2024-01-24T23:22:40.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed011e290 con 0x7f5ed01355e0 2024-01-24T23:22:40.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed0135e60 con 0x7f5ed0134af0 2024-01-24T23:22:40.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:40.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 10 monclient(hunting): authenticate will time out at 1047.966337s 2024-01-24T23:22:40.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed5ae8640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f5ed00a1b50 0x7f5ed012e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:37062/0 (socket says 172.21.15.132:37062) 2024-01-24T23:22:40.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed5ae8640 1 -- 172.21.15.132:0/2030888003 learned_addr learned my addr 172.21.15.132:0/2030888003 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:40.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed52e7640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f5ed0134af0 0x7f5ed01ce570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:33986/0 (socket says 172.21.15.132:33986) 2024-01-24T23:22:40.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 1 -- 172.21.15.132:0/2030888003 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2222043676 0 0) 0x7f5ed0070300 con 0x7f5ed00a1b50 2024-01-24T23:22:40.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:40.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:40.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:40.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient(hunting): my global_id is 4611 2024-01-24T23:22:40.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 1 -- 172.21.15.132:0/2030888003 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5eb4003530 con 0x7f5ed00a1b50 2024-01-24T23:22:40.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 1 -- 172.21.15.132:0/2030888003 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136631509 0 0) 0x7f5ed0135e60 con 0x7f5ed0134af0 2024-01-24T23:22:40.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:40.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:40.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:40.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient(hunting): my global_id is 4640 2024-01-24T23:22:40.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 1 -- 172.21.15.132:0/2030888003 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ed0070300 con 0x7f5ed0134af0 2024-01-24T23:22:40.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 1 -- 172.21.15.132:0/2030888003 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2550905912 0 0) 0x7f5eb4003530 con 0x7f5ed00a1b50 2024-01-24T23:22:40.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 1 -- 172.21.15.132:0/2030888003 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5ed0135e60 con 0x7f5ed00a1b50 2024-01-24T23:22:40.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 1 -- 172.21.15.132:0/2030888003 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5ecc001780 con 0x7f5ed00a1b50 2024-01-24T23:22:40.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:40.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:22:40.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient(hunting): dump: 2024-01-24T23:22:40.653 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:40.653 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:40.653 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:40.653 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:40.653 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:40.653 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:40.654 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:40.654 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:40.654 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:40.654 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:40.654 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:40.654 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:40.654 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:40.654 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:40.654 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:40.654 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:40.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:40.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 1 -- 172.21.15.132:0/2030888003 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2444635423 0 0) 0x7f5ed0070300 con 0x7f5ed0134af0 2024-01-24T23:22:40.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 1 -- 172.21.15.132:0/2030888003 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5eb4003530 con 0x7f5ed0134af0 2024-01-24T23:22:40.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 1 -- 172.21.15.132:0/2030888003 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5ec0004220 con 0x7f5ed0134af0 2024-01-24T23:22:40.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:40.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:22:40.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient(hunting): dump: 2024-01-24T23:22:40.655 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:40.655 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:40.655 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:40.655 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:40.655 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:40.655 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:40.655 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:40.655 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:40.655 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:40.655 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:40.655 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:40.655 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:40.655 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:40.655 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:40.655 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:40.655 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:40.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 1 -- 172.21.15.132:0/2030888003 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2805814150 0 0) 0x7f5ed011e290 con 0x7f5ed01355e0 2024-01-24T23:22:40.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:40.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:40.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:40.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient(hunting): my global_id is 74695 2024-01-24T23:22:40.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 1 -- 172.21.15.132:0/2030888003 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ed0070300 con 0x7f5ed01355e0 2024-01-24T23:22:40.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 1 -- 172.21.15.132:0/2030888003 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 479785709 0 0) 0x7f5ed0135e60 con 0x7f5ed00a1b50 2024-01-24T23:22:40.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 1 -- 172.21.15.132:0/2030888003 >> v1:172.21.15.132:6791/0 conn(0x7f5ed0134af0 legacy=0x7f5ed01ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:40.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 1 -- 172.21.15.132:0/2030888003 >> v1:172.21.15.115:6791/0 conn(0x7f5ed01355e0 legacy=0x7f5ed01d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:40.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient: _finish_hunting 0 2024-01-24T23:22:40.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 1 monclient: found mon.g 2024-01-24T23:22:40.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:40.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:40.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 1 -- 172.21.15.132:0/2030888003 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed009fe30 con 0x7f5ed00a1b50 2024-01-24T23:22:40.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient: _finish_auth 0 2024-01-24T23:22:40.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient: _check_auth_tickets 2024-01-24T23:22:40.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:40.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 5 monclient: authenticate success, global_id 4611 2024-01-24T23:22:40.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 10 monclient: _renew_subs 2024-01-24T23:22:40.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:40.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 1 -- 172.21.15.132:0/2030888003 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed011dfb0 con 0x7f5ed00a1b50 2024-01-24T23:22:40.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 10 monclient: _renew_subs 2024-01-24T23:22:40.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:40.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ed7d4c640 1 -- 172.21.15.132:0/2030888003 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed012efc0 con 0x7f5ed00a1b50 2024-01-24T23:22:40.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 1 -- 172.21.15.132:0/2030888003 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5ecc001180 con 0x7f5ed00a1b50 2024-01-24T23:22:40.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:40.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 1 -- 172.21.15.132:0/2030888003 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5ecc001c50 con 0x7f5ed00a1b50 2024-01-24T23:22:40.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:40.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec7fff640 4 set_mon_vals no callback set 2024-01-24T23:22:40.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:22:40.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.638+0000 7f5ec67fc640 10 monclient: dump: 2024-01-24T23:22:40.658 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:40.658 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:40.658 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:40.658 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:40.658 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:40.658 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:40.658 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:40.658 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:40.658 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:40.658 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:40.658 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:40.658 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:40.659 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:40.659 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:40.659 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:40.659 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:40.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.642+0000 7f5ed7d4c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:40.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.642+0000 7f5ed7d4c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:40.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.642+0000 7f5ed7d4c640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:40.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.642+0000 7f5ed7d4c640 1 -- 172.21.15.132:0/2030888003 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb8005090 con 0x7f5ed00a1b50 2024-01-24T23:22:40.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.642+0000 7f5ec67fc640 1 -- 172.21.15.132:0/2030888003 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f5ecc00fdd0 con 0x7f5ed00a1b50 2024-01-24T23:22:40.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.642+0000 7f5ec67fc640 1 -- 172.21.15.132:0/2030888003 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(25..25 src has 1..25) v4 ==== 3506+0+0 (unknown 1380043353 0 0) 0x7f5ecc043880 con 0x7f5ed00a1b50 2024-01-24T23:22:40.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.642+0000 7f5ed52e7640 10 monclient: get_auth_request con 0x7f5eb4037810 auth_method 0 2024-01-24T23:22:40.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.646+0000 7f5ec67fc640 1 -- 172.21.15.132:0/2030888003 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f5ecc014b50 con 0x7f5ed00a1b50 2024-01-24T23:22:40.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.646+0000 7f5ec67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:40.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.646+0000 7f5ec67fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:40.793 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.790+0000 7f5ed7d4c640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}] 2024-01-24T23:22:40.793 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.790+0000 7f5ed7d4c640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}] 2024-01-24T23:22:40.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.790+0000 7f5ed7d4c640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:40.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:40.790+0000 7f5ed7d4c640 1 -- 172.21.15.132:0/2030888003 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"} v 0) v1 -- 0x7f5eb80058c0 con 0x7f5ed00a1b50 2024-01-24T23:22:41.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.118+0000 7f5ec67fc640 1 -- 172.21.15.132:0/2030888003 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}]=0 pool 'foo' created v26) v1 ==== 154+0+0 (unknown 631208373 0 0) 0x7f5ecc0131f0 con 0x7f5ed00a1b50 2024-01-24T23:22:41.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.118+0000 7f5ec67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}] 2024-01-24T23:22:41.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.118+0000 7f5ec67fc640 10 monclient: _finish_command 2 = system:0 pool 'foo' created 2024-01-24T23:22:41.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.230+0000 7f5ed7d4c640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}] 2024-01-24T23:22:41.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.230+0000 7f5ed7d4c640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}] 2024-01-24T23:22:41.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.230+0000 7f5ed7d4c640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:41.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.230+0000 7f5ed7d4c640 1 -- 172.21.15.132:0/2030888003 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"} v 0) v1 -- 0x7f5eb80053f0 con 0x7f5ed00a1b50 2024-01-24T23:22:41.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.230+0000 7f5ec67fc640 1 -- 172.21.15.132:0/2030888003 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}]=0 pool 'foo' already exists v26) v1 ==== 161+0+0 (unknown 220057628 0 0) 0x7f5eb80053f0 con 0x7f5ed00a1b50 2024-01-24T23:22:41.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.230+0000 7f5ec67fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}] 2024-01-24T23:22:41.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.230+0000 7f5ec67fc640 10 monclient: _finish_command 3 = system:0 pool 'foo' already exists 2024-01-24T23:22:41.233 INFO:tasks.workunit.client.0.smithi132.stderr:pool 'foo' already exists 2024-01-24T23:22:41.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.234+0000 7f5ed7d4c640 1 -- 172.21.15.132:0/2030888003 >> v1:172.21.15.132:6812/1835401922 conn(0x7f5eb4037810 legacy=0x7f5eb4039cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:41.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.234+0000 7f5ed7d4c640 10 monclient: shutdown 2024-01-24T23:22:41.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.234+0000 7f5ed7d4c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:41.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.234+0000 7f5ed7d4c640 1 -- 172.21.15.132:0/2030888003 >> v1:172.21.15.132:6790/0 conn(0x7f5ed00a1b50 legacy=0x7f5ed012e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:41.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.234+0000 7f5ed7d4c640 1 -- 172.21.15.132:0/2030888003 shutdown_connections 2024-01-24T23:22:41.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.234+0000 7f5ed7d4c640 1 -- 172.21.15.132:0/2030888003 >> 172.21.15.132:0/2030888003 conn(0x7f5ed006d9e0 msgr2=0x7f5ed01bb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:41.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.234+0000 7f5ed7d4c640 1 -- 172.21.15.132:0/2030888003 shutdown_connections 2024-01-24T23:22:41.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.234+0000 7f5ed7d4c640 1 -- 172.21.15.132:0/2030888003 wait complete. 2024-01-24T23:22:41.250 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool create fooo 123 123 erasure default 2024-01-24T23:22:41.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 10 monclient: get_monmap_and_config 2024-01-24T23:22:41.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 10 monclient: build_initial_monmap 2024-01-24T23:22:41.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 10 monclient: monmap: 2024-01-24T23:22:41.363 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:41.363 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:41.363 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:41.365250+0000 2024-01-24T23:22:41.363 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:41.365250+0000 2024-01-24T23:22:41.363 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:41.363 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:41.363 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:41.363 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:41.364 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:41.364 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:41.364 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:41.364 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:41.364 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:41.364 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:41.364 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:41.364 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:41.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 1 Processor -- start 2024-01-24T23:22:41.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 1 -- start start 2024-01-24T23:22:41.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 10 monclient: init 2024-01-24T23:22:41.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:41.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 10 monclient: _add_conns ranks=[4,6,1,3,0,7,8,2,5] 2024-01-24T23:22:41.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 10 monclient(hunting): picked mon.noname-c con 0x7f50a409bc10 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:41.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 10 monclient(hunting): picked mon.noname-d con 0x7f50a40a00d0 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:41.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 10 monclient(hunting): picked mon.noname-f con 0x7f50a40a0560 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:41.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a4064700 con 0x7f50a40a0560 2024-01-24T23:22:41.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a409bff0 con 0x7f50a409bc10 2024-01-24T23:22:41.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a40a1870 con 0x7f50a40a00d0 2024-01-24T23:22:41.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:41.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 10 monclient(hunting): authenticate will time out at 1048.690403s 2024-01-24T23:22:41.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a37fe640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f50a40a0560 0x7f50a413bef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:49184/0 (socket says 172.21.15.132:49184) 2024-01-24T23:22:41.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a37fe640 1 -- 172.21.15.132:0/3781919628 learned_addr learned my addr 172.21.15.132:0/3781919628 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:41.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 1 -- 172.21.15.132:0/3781919628 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 59034175 0 0) 0x7f50a4064700 con 0x7f50a40a0560 2024-01-24T23:22:41.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:22:41.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:41.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:41.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 10 monclient(hunting): my global_id is 4699 2024-01-24T23:22:41.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 1 -- 172.21.15.132:0/3781919628 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f508c003530 con 0x7f50a40a0560 2024-01-24T23:22:41.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 1 -- 172.21.15.132:0/3781919628 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1811183832 0 0) 0x7f508c003530 con 0x7f50a40a0560 2024-01-24T23:22:41.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 1 -- 172.21.15.132:0/3781919628 >> v1:172.21.15.115:6792/0 conn(0x7f50a40a00d0 legacy=0x7f50a41387c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:41.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 1 -- 172.21.15.132:0/3781919628 >> v1:172.21.15.115:6791/0 conn(0x7f50a409bc10 legacy=0x7f50a409f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:41.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:22:41.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 1 monclient: found mon.noname-f 2024-01-24T23:22:41.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:41.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:22:41.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 1 -- 172.21.15.132:0/3781919628 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a40a0fa0 con 0x7f50a40a0560 2024-01-24T23:22:41.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 10 monclient: _finish_auth 0 2024-01-24T23:22:41.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:22:41.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:41.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 5 monclient: authenticate success, global_id 4699 2024-01-24T23:22:41.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:41.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 1 -- 172.21.15.132:0/3781919628 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f509c001b40 con 0x7f50a40a0560 2024-01-24T23:22:41.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:41.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:22:41.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 10 monclient: dump: 2024-01-24T23:22:41.374 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:41.374 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:41.374 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:41.374 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:41.374 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:41.374 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:41.374 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:41.374 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:41.374 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:41.374 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:41.374 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:41.374 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:41.375 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:41.375 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:41.375 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:41.375 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:41.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:41.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 1 -- 172.21.15.132:0/3781919628 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f509c001df0 con 0x7f50a40a0560 2024-01-24T23:22:41.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:41.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:41.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 1 -- 172.21.15.132:0/3781919628 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f509c004f70 con 0x7f50a40a0560 2024-01-24T23:22:41.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 4 set_mon_vals no callback set 2024-01-24T23:22:41.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:41.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:22:41.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a1ffb640 10 monclient: dump: 2024-01-24T23:22:41.375 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:41.375 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:41.375 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:41.375 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:41.376 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:41.376 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:41.376 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:41.376 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:41.376 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:41.376 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:41.376 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:41.376 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:41.376 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:41.376 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:41.376 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:41.376 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:41.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 10 monclient: shutdown 2024-01-24T23:22:41.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:41.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 1 -- 172.21.15.132:0/3781919628 >> v1:172.21.15.132:6789/0 conn(0x7f50a40a0560 legacy=0x7f50a413bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:41.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 1 -- 172.21.15.132:0/3781919628 shutdown_connections 2024-01-24T23:22:41.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 1 -- 172.21.15.132:0/3781919628 >> 172.21.15.132:0/3781919628 conn(0x7f50a40a5a10 msgr2=0x7f50a40a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:41.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 1 -- 172.21.15.132:0/3781919628 shutdown_connections 2024-01-24T23:22:41.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.362+0000 7f50a94a6640 1 -- 172.21.15.132:0/3781919628 wait complete. 2024-01-24T23:22:41.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 10 monclient: build_initial_monmap 2024-01-24T23:22:41.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 10 monclient: monmap: 2024-01-24T23:22:41.377 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:41.377 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:41.377 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:41.377 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:41.377 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:41.377 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:41.378 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:41.378 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:41.378 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:41.378 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:41.378 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:41.378 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:41.378 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:41.378 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:41.378 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:41.378 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:41.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 1 Processor -- start 2024-01-24T23:22:41.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 1 -- start start 2024-01-24T23:22:41.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 10 monclient: init 2024-01-24T23:22:41.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:41.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 10 monclient: _add_conns ranks=[6,3,5,1,0,7,4,2,8] 2024-01-24T23:22:41.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 10 monclient(hunting): picked mon.noname-d con 0x7f50a409bc10 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:41.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 10 monclient(hunting): picked mon.noname-g con 0x7f50a40a00d0 addr v1:172.21.15.132:6790/0 2024-01-24T23:22:41.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 10 monclient(hunting): picked mon.noname-h con 0x7f50a40a0560 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:41.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a4071fe0 con 0x7f50a40a00d0 2024-01-24T23:22:41.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a411c000 con 0x7f50a40a0560 2024-01-24T23:22:41.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a41cc3b0 con 0x7f50a409bc10 2024-01-24T23:22:41.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:41.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 10 monclient(hunting): authenticate will time out at 1048.693233s 2024-01-24T23:22:41.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a37fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f50a40a0560 0x7f50a41d3ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:33988/0 (socket says 172.21.15.132:33988) 2024-01-24T23:22:41.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a27fc640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f50a40a00d0 0x7f50a41cbca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:37074/0 (socket says 172.21.15.132:37074) 2024-01-24T23:22:41.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a37fe640 1 -- 172.21.15.132:0/4216361535 learned_addr learned my addr 172.21.15.132:0/4216361535 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:41.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 1 -- 172.21.15.132:0/4216361535 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1910845983 0 0) 0x7f50a411c000 con 0x7f50a40a0560 2024-01-24T23:22:41.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:22:41.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:41.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:41.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient(hunting): my global_id is 4649 2024-01-24T23:22:41.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 1 -- 172.21.15.132:0/4216361535 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5090003530 con 0x7f50a40a0560 2024-01-24T23:22:41.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 1 -- 172.21.15.132:0/4216361535 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3599713096 0 0) 0x7f50a4071fe0 con 0x7f50a40a00d0 2024-01-24T23:22:41.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:22:41.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:41.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:41.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient(hunting): my global_id is 4620 2024-01-24T23:22:41.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 1 -- 172.21.15.132:0/4216361535 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50a411c000 con 0x7f50a40a00d0 2024-01-24T23:22:41.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 1 -- 172.21.15.132:0/4216361535 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3779781583 0 0) 0x7f5090003530 con 0x7f50a40a0560 2024-01-24T23:22:41.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 1 -- 172.21.15.132:0/4216361535 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f50a4071fe0 con 0x7f50a40a0560 2024-01-24T23:22:41.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 1 -- 172.21.15.132:0/4216361535 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f509c001ba0 con 0x7f50a40a0560 2024-01-24T23:22:41.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:41.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:22:41.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient(hunting): dump: 2024-01-24T23:22:41.380 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:41.381 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:41.381 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:41.381 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:41.381 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:41.381 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:41.381 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:41.381 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:41.381 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:41.381 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:41.381 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:41.381 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:41.381 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:41.381 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:41.381 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:41.381 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:41.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:41.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 1 -- 172.21.15.132:0/4216361535 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1355077330 0 0) 0x7f50a411c000 con 0x7f50a40a00d0 2024-01-24T23:22:41.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 1 -- 172.21.15.132:0/4216361535 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5090003530 con 0x7f50a40a00d0 2024-01-24T23:22:41.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 1 -- 172.21.15.132:0/4216361535 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f50980019e0 con 0x7f50a40a00d0 2024-01-24T23:22:41.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:41.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:22:41.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient(hunting): dump: 2024-01-24T23:22:41.382 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:41.382 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:41.382 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:41.382 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:41.382 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:41.382 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:41.382 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:41.382 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:41.382 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:41.383 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:41.383 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:41.383 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:41.383 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:41.383 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:41.383 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:41.383 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:41.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 1 -- 172.21.15.132:0/4216361535 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812480983 0 0) 0x7f50a41cc3b0 con 0x7f50a409bc10 2024-01-24T23:22:41.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:22:41.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:41.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:41.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient(hunting): my global_id is 4713 2024-01-24T23:22:41.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 1 -- 172.21.15.132:0/4216361535 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50a411c000 con 0x7f50a409bc10 2024-01-24T23:22:41.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 1 -- 172.21.15.132:0/4216361535 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2986307275 0 0) 0x7f50a4071fe0 con 0x7f50a40a0560 2024-01-24T23:22:41.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 1 -- 172.21.15.132:0/4216361535 >> v1:172.21.15.115:6792/0 conn(0x7f50a409bc10 legacy=0x7f50a41cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:41.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 1 -- 172.21.15.132:0/4216361535 >> v1:172.21.15.132:6790/0 conn(0x7f50a40a00d0 legacy=0x7f50a41cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:41.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:22:41.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 1 monclient: found mon.h 2024-01-24T23:22:41.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:41.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:41.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 1 -- 172.21.15.132:0/4216361535 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a4141020 con 0x7f50a40a0560 2024-01-24T23:22:41.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient: _finish_auth 0 2024-01-24T23:22:41.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:22:41.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:41.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 5 monclient: authenticate success, global_id 4649 2024-01-24T23:22:41.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 10 monclient: _renew_subs 2024-01-24T23:22:41.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:41.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 1 -- 172.21.15.132:0/4216361535 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a411b760 con 0x7f50a40a0560 2024-01-24T23:22:41.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 10 monclient: _renew_subs 2024-01-24T23:22:41.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:41.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 1 -- 172.21.15.132:0/4216361535 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a406e270 con 0x7f50a40a0560 2024-01-24T23:22:41.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 1 -- 172.21.15.132:0/4216361535 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f509c004eb0 con 0x7f50a40a0560 2024-01-24T23:22:41.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:41.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 1 -- 172.21.15.132:0/4216361535 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f509c005360 con 0x7f50a40a0560 2024-01-24T23:22:41.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:41.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a17fa640 4 set_mon_vals no callback set 2024-01-24T23:22:41.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:22:41.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 10 monclient: dump: 2024-01-24T23:22:41.385 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:41.385 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:41.385 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:41.385 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:41.385 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:41.385 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:41.385 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:41.385 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:41.386 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:41.386 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:41.386 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:41.386 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:41.386 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:41.386 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:41.386 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:41.386 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:41.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:41.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:41.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:41.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a94a6640 1 -- 172.21.15.132:0/4216361535 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5084005090 con 0x7f50a40a0560 2024-01-24T23:22:41.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 1 -- 172.21.15.132:0/4216361535 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f509c00fde0 con 0x7f50a40a0560 2024-01-24T23:22:41.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f508b7fe640 1 -- 172.21.15.132:0/4216361535 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(26..26 src has 1..26) v4 ==== 3848+0+0 (unknown 3014898177 0 0) 0x7f509c043ba0 con 0x7f50a40a0560 2024-01-24T23:22:41.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.366+0000 7f50a2ffd640 10 monclient: get_auth_request con 0x7f5090037830 auth_method 0 2024-01-24T23:22:41.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.370+0000 7f508b7fe640 1 -- 172.21.15.132:0/4216361535 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f509c0112e0 con 0x7f50a40a0560 2024-01-24T23:22:41.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.370+0000 7f508b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:41.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.370+0000 7f508b7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:41.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.514+0000 7f50a94a6640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}] 2024-01-24T23:22:41.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.514+0000 7f50a94a6640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}] 2024-01-24T23:22:41.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.514+0000 7f50a94a6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:41.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:41.514+0000 7f50a94a6640 1 -- 172.21.15.132:0/4216361535 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"} v 0) v1 -- 0x7f5084005340 con 0x7f50a40a0560 2024-01-24T23:22:43.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.150+0000 7f508b7fe640 1 -- 172.21.15.132:0/4216361535 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}]=0 pool 'fooo' created v28) v1 ==== 188+0+0 (unknown 2510396809 0 0) 0x7f509c014ae0 con 0x7f50a40a0560 2024-01-24T23:22:43.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.150+0000 7f508b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}] 2024-01-24T23:22:43.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.150+0000 7f508b7fe640 10 monclient: _finish_command 2 = system:0 pool 'fooo' created 2024-01-24T23:22:43.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.269+0000 7f50a94a6640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}] 2024-01-24T23:22:43.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.269+0000 7f50a94a6640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}] 2024-01-24T23:22:43.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.269+0000 7f50a94a6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:43.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.269+0000 7f50a94a6640 1 -- 172.21.15.132:0/4216361535 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"} v 0) v1 -- 0x7f5084005ab0 con 0x7f50a40a0560 2024-01-24T23:22:43.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.273+0000 7f508b7fe640 1 -- 172.21.15.132:0/4216361535 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}]=0 pool 'fooo' already exists v28) v1 ==== 195+0+0 (unknown 1249814627 0 0) 0x7f509c014cc0 con 0x7f50a40a0560 2024-01-24T23:22:43.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.273+0000 7f508b7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}] 2024-01-24T23:22:43.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.273+0000 7f508b7fe640 10 monclient: _finish_command 3 = system:0 pool 'fooo' already exists 2024-01-24T23:22:43.275 INFO:tasks.workunit.client.0.smithi132.stderr:pool 'fooo' already exists 2024-01-24T23:22:43.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.273+0000 7f50a94a6640 1 -- 172.21.15.132:0/4216361535 >> v1:172.21.15.132:6812/1835401922 conn(0x7f5090037830 legacy=0x7f5090039cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:43.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.273+0000 7f50a94a6640 10 monclient: shutdown 2024-01-24T23:22:43.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.273+0000 7f50a94a6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:43.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.273+0000 7f50a94a6640 1 -- 172.21.15.132:0/4216361535 >> v1:172.21.15.132:6791/0 conn(0x7f50a40a0560 legacy=0x7f50a41d3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:43.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.273+0000 7f50a94a6640 1 -- 172.21.15.132:0/4216361535 shutdown_connections 2024-01-24T23:22:43.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.273+0000 7f50a94a6640 1 -- 172.21.15.132:0/4216361535 >> 172.21.15.132:0/4216361535 conn(0x7f50a40a5a10 msgr2=0x7f50a4137b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:43.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.273+0000 7f50a94a6640 1 -- 172.21.15.132:0/4216361535 shutdown_connections 2024-01-24T23:22:43.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.273+0000 7f50a94a6640 1 -- 172.21.15.132:0/4216361535 wait complete. 2024-01-24T23:22:43.291 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool create foooo 123 2024-01-24T23:22:43.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.405+0000 7f2431b24640 10 monclient: get_monmap_and_config 2024-01-24T23:22:43.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.405+0000 7f2431b24640 10 monclient: build_initial_monmap 2024-01-24T23:22:43.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.405+0000 7f2431b24640 10 monclient: monmap: 2024-01-24T23:22:43.409 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:43.409 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:43.409 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:43.411066+0000 2024-01-24T23:22:43.409 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:43.411066+0000 2024-01-24T23:22:43.409 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:43.410 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:43.410 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:43.410 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:43.410 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:43.410 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:43.410 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:43.410 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:43.410 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:43.410 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:43.410 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:43.410 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:43.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.405+0000 7f2431b24640 1 Processor -- start 2024-01-24T23:22:43.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.405+0000 7f2431b24640 1 -- start start 2024-01-24T23:22:43.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.405+0000 7f2431b24640 10 monclient: init 2024-01-24T23:22:43.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:43.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 10 monclient: _add_conns ranks=[8,3,6,5,7,1,2,4,0] 2024-01-24T23:22:43.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 10 monclient(hunting): picked mon.noname-e con 0x7f242c09ff20 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:43.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 10 monclient(hunting): picked mon.noname-g con 0x7f242c09f9c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:22:43.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 10 monclient(hunting): picked mon.noname-d con 0x7f242c13b2c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:43.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f242c064700 con 0x7f242c09f9c0 2024-01-24T23:22:43.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f242c09e760 con 0x7f242c13b2c0 2024-01-24T23:22:43.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f242c0a1160 con 0x7f242c09ff20 2024-01-24T23:22:43.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:43.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 10 monclient(hunting): authenticate will time out at 1050.736294s 2024-01-24T23:22:43.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2423fff640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f242c09f9c0 0x7f242c139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:37076/0 (socket says 172.21.15.132:37076) 2024-01-24T23:22:43.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242b7fe640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f242c09ff20 legacy=0x7f242c0a0300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:43.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2423fff640 1 -- 172.21.15.132:0/3383009023 learned_addr learned my addr 172.21.15.132:0/3383009023 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:43.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 1 -- 172.21.15.132:0/3383009023 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3594624097 0 0) 0x7f242c064700 con 0x7f242c09f9c0 2024-01-24T23:22:43.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 10 monclient(hunting): _negotiate 2024-01-24T23:22:43.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:43.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:43.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 10 monclient(hunting): my global_id is 4629 2024-01-24T23:22:43.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 1 -- 172.21.15.132:0/3383009023 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2418003530 con 0x7f242c09f9c0 2024-01-24T23:22:43.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 1 -- 172.21.15.132:0/3383009023 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3052226895 0 0) 0x7f2418003530 con 0x7f242c09f9c0 2024-01-24T23:22:43.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 1 -- 172.21.15.132:0/3383009023 >> v1:172.21.15.115:6793/0 conn(0x7f242c09ff20 legacy=0x7f242c0a0300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:43.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 1 -- 172.21.15.132:0/3383009023 >> v1:172.21.15.115:6792/0 conn(0x7f242c13b2c0 legacy=0x7f242c13d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:43.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 10 monclient: _finish_hunting 0 2024-01-24T23:22:43.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 1 monclient: found mon.noname-g 2024-01-24T23:22:43.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:43.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:22:43.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 1 -- 172.21.15.132:0/3383009023 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242c0a0a10 con 0x7f242c09f9c0 2024-01-24T23:22:43.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 10 monclient: _finish_auth 0 2024-01-24T23:22:43.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 10 monclient: _check_auth_tickets 2024-01-24T23:22:43.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:43.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 5 monclient: authenticate success, global_id 4629 2024-01-24T23:22:43.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:43.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 1 -- 172.21.15.132:0/3383009023 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f24140017c0 con 0x7f242c09f9c0 2024-01-24T23:22:43.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:43.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:22:43.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 10 monclient: dump: 2024-01-24T23:22:43.421 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:43.421 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:43.421 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:43.421 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:43.421 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:43.421 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:43.421 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:43.421 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:43.421 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:43.422 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:43.422 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:43.422 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:43.422 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:43.422 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:43.422 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:43.422 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:43.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:43.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 1 -- 172.21.15.132:0/3383009023 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2414001ae0 con 0x7f242c09f9c0 2024-01-24T23:22:43.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:43.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:43.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 4 set_mon_vals no callback set 2024-01-24T23:22:43.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 1 -- 172.21.15.132:0/3383009023 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2414004bf0 con 0x7f242c09f9c0 2024-01-24T23:22:43.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:43.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:22:43.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242affd640 10 monclient: dump: 2024-01-24T23:22:43.423 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:43.423 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:43.423 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:43.423 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:43.423 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:43.423 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:43.423 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:43.423 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:43.423 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:43.423 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:43.423 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:43.423 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:43.423 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:43.423 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:43.423 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:43.423 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:43.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 10 monclient: shutdown 2024-01-24T23:22:43.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:43.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 1 -- 172.21.15.132:0/3383009023 >> v1:172.21.15.132:6790/0 conn(0x7f242c09f9c0 legacy=0x7f242c139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:43.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 1 -- 172.21.15.132:0/3383009023 shutdown_connections 2024-01-24T23:22:43.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 1 -- 172.21.15.132:0/3383009023 >> 172.21.15.132:0/3383009023 conn(0x7f242c0a5a10 msgr2=0x7f242c0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:43.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 1 -- 172.21.15.132:0/3383009023 shutdown_connections 2024-01-24T23:22:43.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 1 -- 172.21.15.132:0/3383009023 wait complete. 2024-01-24T23:22:43.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 10 monclient: build_initial_monmap 2024-01-24T23:22:43.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 10 monclient: monmap: 2024-01-24T23:22:43.424 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:43.424 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:43.424 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:43.424 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:43.424 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:43.425 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:43.425 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:43.425 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:43.425 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:43.425 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:43.425 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:43.425 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:43.425 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:43.425 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:43.425 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:43.425 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:43.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 1 Processor -- start 2024-01-24T23:22:43.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 1 -- start start 2024-01-24T23:22:43.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 10 monclient: init 2024-01-24T23:22:43.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:43.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 10 monclient: _add_conns ranks=[6,0,5,2,3,1,7,4,8] 2024-01-24T23:22:43.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 10 monclient(hunting): picked mon.noname-d con 0x7f242c09f9c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:43.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 10 monclient(hunting): picked mon.noname-a con 0x7f242c09ff20 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:43.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 10 monclient(hunting): picked mon.noname-h con 0x7f242c13b2c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:43.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f242c11c430 con 0x7f242c09ff20 2024-01-24T23:22:43.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f242c11d320 con 0x7f242c13b2c0 2024-01-24T23:22:43.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f242c11eab0 con 0x7f242c09f9c0 2024-01-24T23:22:43.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:43.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2431b24640 10 monclient(hunting): authenticate will time out at 1050.739224s 2024-01-24T23:22:43.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242bfff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f242c13b2c0 0x7f242c1d16a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:34010/0 (socket says 172.21.15.132:34010) 2024-01-24T23:22:43.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f242bfff640 1 -- 172.21.15.132:0/2746605053 learned_addr learned my addr 172.21.15.132:0/2746605053 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:43.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2428ff9640 1 -- 172.21.15.132:0/2746605053 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 902817857 0 0) 0x7f242c11d320 con 0x7f242c13b2c0 2024-01-24T23:22:43.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2428ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:22:43.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2428ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:43.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2428ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:43.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2428ff9640 10 monclient(hunting): my global_id is 4658 2024-01-24T23:22:43.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2428ff9640 1 -- 172.21.15.132:0/2746605053 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f240c003560 con 0x7f242c13b2c0 2024-01-24T23:22:43.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2428ff9640 1 -- 172.21.15.132:0/2746605053 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1306112556 0 0) 0x7f240c003560 con 0x7f242c13b2c0 2024-01-24T23:22:43.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2428ff9640 1 -- 172.21.15.132:0/2746605053 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f242c11d320 con 0x7f242c13b2c0 2024-01-24T23:22:43.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2428ff9640 1 -- 172.21.15.132:0/2746605053 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2424001d50 con 0x7f242c13b2c0 2024-01-24T23:22:43.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2428ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:43.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2428ff9640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:22:43.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2428ff9640 10 monclient(hunting): dump: 2024-01-24T23:22:43.427 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:43.427 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:43.427 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:43.427 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:43.427 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:43.428 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:43.428 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:43.428 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:43.428 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:43.428 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:43.428 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:43.428 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:43.428 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:43.428 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:43.428 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:43.428 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:43.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2428ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:43.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2428ff9640 1 -- 172.21.15.132:0/2746605053 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 378120743 0 0) 0x7f242c11eab0 con 0x7f242c09f9c0 2024-01-24T23:22:43.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2428ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:22:43.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2428ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:43.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2428ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:43.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2428ff9640 10 monclient(hunting): my global_id is 4731 2024-01-24T23:22:43.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.409+0000 7f2428ff9640 1 -- 172.21.15.132:0/2746605053 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f240c003560 con 0x7f242c09f9c0 2024-01-24T23:22:43.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 1 -- 172.21.15.132:0/2746605053 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3932423402 0 0) 0x7f242c11c430 con 0x7f242c09ff20 2024-01-24T23:22:43.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:22:43.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:43.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:43.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 10 monclient(hunting): my global_id is 4689 2024-01-24T23:22:43.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 1 -- 172.21.15.132:0/2746605053 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f242c11eab0 con 0x7f242c09ff20 2024-01-24T23:22:43.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 1 -- 172.21.15.132:0/2746605053 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 483478125 0 0) 0x7f242c11d320 con 0x7f242c13b2c0 2024-01-24T23:22:43.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 1 -- 172.21.15.132:0/2746605053 >> v1:172.21.15.115:6792/0 conn(0x7f242c09f9c0 legacy=0x7f242c11b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:43.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 1 -- 172.21.15.132:0/2746605053 >> v1:172.21.15.115:6789/0 conn(0x7f242c09ff20 legacy=0x7f242c11bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:43.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:22:43.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 1 monclient: found mon.h 2024-01-24T23:22:43.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:43.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:43.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 1 -- 172.21.15.132:0/2746605053 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242c071960 con 0x7f242c13b2c0 2024-01-24T23:22:43.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 10 monclient: _finish_auth 0 2024-01-24T23:22:43.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:22:43.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:43.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2431b24640 5 monclient: authenticate success, global_id 4658 2024-01-24T23:22:43.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2431b24640 10 monclient: _renew_subs 2024-01-24T23:22:43.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2431b24640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:43.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2431b24640 1 -- 172.21.15.132:0/2746605053 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f242c071680 con 0x7f242c13b2c0 2024-01-24T23:22:43.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2431b24640 10 monclient: _renew_subs 2024-01-24T23:22:43.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2431b24640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:43.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2431b24640 1 -- 172.21.15.132:0/2746605053 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f242c11e7d0 con 0x7f242c13b2c0 2024-01-24T23:22:43.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 1 -- 172.21.15.132:0/2746605053 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2424003cd0 con 0x7f242c13b2c0 2024-01-24T23:22:43.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:43.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 1 -- 172.21.15.132:0/2746605053 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2424004200 con 0x7f242c13b2c0 2024-01-24T23:22:43.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:43.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f242a7fc640 4 set_mon_vals no callback set 2024-01-24T23:22:43.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:22:43.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 10 monclient: dump: 2024-01-24T23:22:43.431 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:43.431 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:43.431 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:43.431 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:43.431 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:43.431 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:43.431 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:43.431 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:43.431 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:43.431 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:43.431 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:43.431 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:43.432 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:43.432 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:43.432 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:43.432 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:43.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2431b24640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:43.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2431b24640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:43.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2431b24640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:43.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2431b24640 1 -- 172.21.15.132:0/2746605053 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2410005090 con 0x7f242c13b2c0 2024-01-24T23:22:43.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 1 -- 172.21.15.132:0/2746605053 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f24240043e0 con 0x7f242c13b2c0 2024-01-24T23:22:43.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f2428ff9640 1 -- 172.21.15.132:0/2746605053 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(28..28 src has 1..28) v4 ==== 4290+0+0 (unknown 2971113092 0 0) 0x7f2424043c20 con 0x7f242c13b2c0 2024-01-24T23:22:43.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.413+0000 7f242b7fe640 10 monclient: get_auth_request con 0x7f240c0371c0 auth_method 0 2024-01-24T23:22:43.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.417+0000 7f2428ff9640 1 -- 172.21.15.132:0/2746605053 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f2424014e20 con 0x7f242c13b2c0 2024-01-24T23:22:43.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.417+0000 7f2428ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:43.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.417+0000 7f2428ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:43.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.569+0000 7f2431b24640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}] 2024-01-24T23:22:43.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.569+0000 7f2431b24640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}] 2024-01-24T23:22:43.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.569+0000 7f2431b24640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:43.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:43.569+0000 7f2431b24640 1 -- 172.21.15.132:0/2746605053 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd pool create", "pool": "foooo", "pg_num": 123} v 0) v1 -- 0x7f24100058c0 con 0x7f242c13b2c0 2024-01-24T23:22:44.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.157+0000 7f2428ff9640 1 -- 172.21.15.132:0/2746605053 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}]=0 pool 'foooo' created v29) v1 ==== 115+0+0 (unknown 795755057 0 0) 0x7f2424013290 con 0x7f242c13b2c0 2024-01-24T23:22:44.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.157+0000 7f2428ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}] 2024-01-24T23:22:44.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.157+0000 7f2428ff9640 10 monclient: _finish_command 2 = system:0 pool 'foooo' created 2024-01-24T23:22:44.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.313+0000 7f2431b24640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}] 2024-01-24T23:22:44.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.313+0000 7f2431b24640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}] 2024-01-24T23:22:44.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.313+0000 7f2431b24640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:44.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.313+0000 7f2431b24640 1 -- 172.21.15.132:0/2746605053 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd pool create", "pool": "foooo", "pg_num": 123} v 0) v1 -- 0x7f2410000bc0 con 0x7f242c13b2c0 2024-01-24T23:22:44.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.317+0000 7f2428ff9640 1 -- 172.21.15.132:0/2746605053 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}]=0 pool 'foooo' already exists v29) v1 ==== 122+0+0 (unknown 4284502079 0 0) 0x7f2424014a10 con 0x7f242c13b2c0 2024-01-24T23:22:44.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.317+0000 7f2428ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}] 2024-01-24T23:22:44.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.317+0000 7f2428ff9640 10 monclient: _finish_command 3 = system:0 pool 'foooo' already exists 2024-01-24T23:22:44.319 INFO:tasks.workunit.client.0.smithi132.stderr:pool 'foooo' already exists 2024-01-24T23:22:44.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.317+0000 7f2431b24640 1 -- 172.21.15.132:0/2746605053 >> v1:172.21.15.132:6812/1835401922 conn(0x7f240c0371c0 legacy=0x7f240c039680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:44.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.317+0000 7f2431b24640 10 monclient: shutdown 2024-01-24T23:22:44.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.317+0000 7f2431b24640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:44.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.317+0000 7f2431b24640 1 -- 172.21.15.132:0/2746605053 >> v1:172.21.15.132:6791/0 conn(0x7f242c13b2c0 legacy=0x7f242c1d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:44.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.317+0000 7f2431b24640 1 -- 172.21.15.132:0/2746605053 shutdown_connections 2024-01-24T23:22:44.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.317+0000 7f2431b24640 1 -- 172.21.15.132:0/2746605053 >> 172.21.15.132:0/2746605053 conn(0x7f242c0a5a10 msgr2=0x7f242c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:44.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.317+0000 7f2431b24640 1 -- 172.21.15.132:0/2746605053 shutdown_connections 2024-01-24T23:22:44.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.317+0000 7f2431b24640 1 -- 172.21.15.132:0/2746605053 wait complete. 2024-01-24T23:22:44.336 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool create foo 123 2024-01-24T23:22:44.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.445+0000 7f30b1002640 10 monclient: get_monmap_and_config 2024-01-24T23:22:44.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.445+0000 7f30b1002640 10 monclient: build_initial_monmap 2024-01-24T23:22:44.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.445+0000 7f30b1002640 10 monclient: monmap: 2024-01-24T23:22:44.450 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:44.450 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:44.450 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:44.451778+0000 2024-01-24T23:22:44.450 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:44.451778+0000 2024-01-24T23:22:44.450 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:44.450 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:44.450 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:44.450 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:44.450 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:44.450 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:44.450 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:44.451 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:44.451 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:44.451 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:44.451 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:44.451 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:44.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 1 Processor -- start 2024-01-24T23:22:44.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 1 -- start start 2024-01-24T23:22:44.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 10 monclient: init 2024-01-24T23:22:44.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:44.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 10 monclient: _add_conns ranks=[6,4,8,3,1,7,0,2,5] 2024-01-24T23:22:44.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 10 monclient(hunting): picked mon.noname-d con 0x7f30ac134b40 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:44.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 10 monclient(hunting): picked mon.noname-c con 0x7f30ac06dd00 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:44.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 10 monclient(hunting): picked mon.noname-e con 0x7f30ac13b270 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:44.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30ac064700 con 0x7f30ac06dd00 2024-01-24T23:22:44.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30ac071f50 con 0x7f30ac134b40 2024-01-24T23:22:44.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30ac070ba0 con 0x7f30ac13b270 2024-01-24T23:22:44.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:44.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 10 monclient(hunting): authenticate will time out at 1051.777419s 2024-01-24T23:22:44.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30ab59e640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f30ac13b270 legacy=0x7f30ac13d660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:44.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aad9d640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f30ac134b40 0x7f30ac134f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:49470/0 (socket says 172.21.15.132:49470) 2024-01-24T23:22:44.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aad9d640 1 -- 172.21.15.132:0/280579604 learned_addr learned my addr 172.21.15.132:0/280579604 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:44.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30a3fff640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f30ac06dd00 0x7f30ac139b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:60350/0 (socket says 172.21.15.132:60350) 2024-01-24T23:22:44.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 1 -- 172.21.15.132:0/280579604 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1828499634 0 0) 0x7f3094001120 con 0x7f30ac134b40 2024-01-24T23:22:44.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 10 monclient(hunting): _negotiate 2024-01-24T23:22:44.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:44.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:44.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 10 monclient(hunting): my global_id is 4740 2024-01-24T23:22:44.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 1 -- 172.21.15.132:0/280579604 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3098003530 con 0x7f30ac134b40 2024-01-24T23:22:44.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 1 -- 172.21.15.132:0/280579604 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165641287 0 0) 0x7f30ac064700 con 0x7f30ac06dd00 2024-01-24T23:22:44.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 10 monclient(hunting): _negotiate 2024-01-24T23:22:44.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:44.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:44.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 10 monclient(hunting): my global_id is 74713 2024-01-24T23:22:44.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 1 -- 172.21.15.132:0/280579604 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3094001120 con 0x7f30ac06dd00 2024-01-24T23:22:44.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 1 -- 172.21.15.132:0/280579604 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3538638344 0 0) 0x7f3094001120 con 0x7f30ac06dd00 2024-01-24T23:22:44.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 1 -- 172.21.15.132:0/280579604 >> v1:172.21.15.115:6793/0 conn(0x7f30ac13b270 legacy=0x7f30ac13d660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:44.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 1 -- 172.21.15.132:0/280579604 >> v1:172.21.15.115:6792/0 conn(0x7f30ac134b40 legacy=0x7f30ac134f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:44.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 10 monclient: _finish_hunting 0 2024-01-24T23:22:44.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 1 monclient: found mon.noname-c 2024-01-24T23:22:44.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:44.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:22:44.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 1 -- 172.21.15.132:0/280579604 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ac070e80 con 0x7f30ac06dd00 2024-01-24T23:22:44.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 10 monclient: _finish_auth 0 2024-01-24T23:22:44.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 10 monclient: _check_auth_tickets 2024-01-24T23:22:44.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:44.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 1 -- 172.21.15.132:0/280579604 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f309c001650 con 0x7f30ac06dd00 2024-01-24T23:22:44.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 5 monclient: authenticate success, global_id 74713 2024-01-24T23:22:44.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:44.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:44.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:22:44.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 10 monclient: dump: 2024-01-24T23:22:44.461 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:44.461 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:44.461 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:44.461 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:44.461 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:44.461 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:44.461 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:44.461 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:44.461 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:44.461 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:44.461 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:44.461 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:44.461 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:44.461 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:44.461 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:44.461 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:44.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:44.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 1 -- 172.21.15.132:0/280579604 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f309c0018f0 con 0x7f30ac06dd00 2024-01-24T23:22:44.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:44.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:44.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 1 -- 172.21.15.132:0/280579604 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f309c0047f0 con 0x7f30ac06dd00 2024-01-24T23:22:44.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 4 set_mon_vals no callback set 2024-01-24T23:22:44.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:44.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:22:44.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30aa59c640 10 monclient: dump: 2024-01-24T23:22:44.462 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:44.462 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:44.462 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:44.462 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:44.462 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:44.462 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:44.462 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:44.462 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:44.462 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:44.463 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:44.463 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:44.463 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:44.463 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:44.463 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:44.463 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:44.463 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:44.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 10 monclient: shutdown 2024-01-24T23:22:44.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:44.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 1 -- 172.21.15.132:0/280579604 >> v1:172.21.15.115:6791/0 conn(0x7f30ac06dd00 legacy=0x7f30ac139b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:44.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 1 -- 172.21.15.132:0/280579604 shutdown_connections 2024-01-24T23:22:44.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 1 -- 172.21.15.132:0/280579604 >> 172.21.15.132:0/280579604 conn(0x7f30ac0a22a0 msgr2=0x7f30ac0a2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:44.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 1 -- 172.21.15.132:0/280579604 shutdown_connections 2024-01-24T23:22:44.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.449+0000 7f30b1002640 1 -- 172.21.15.132:0/280579604 wait complete. 2024-01-24T23:22:44.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 10 monclient: build_initial_monmap 2024-01-24T23:22:44.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 10 monclient: monmap: 2024-01-24T23:22:44.463 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:44.464 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:44.464 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:44.464 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:44.464 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:44.464 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:44.464 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:44.464 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:44.464 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:44.464 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:44.464 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:44.464 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:44.464 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:44.464 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:44.464 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:44.465 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:44.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 1 Processor -- start 2024-01-24T23:22:44.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 1 -- start start 2024-01-24T23:22:44.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 10 monclient: init 2024-01-24T23:22:44.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:44.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 10 monclient: _add_conns ranks=[4,8,2,1,5,0,3,7,6] 2024-01-24T23:22:44.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 10 monclient(hunting): picked mon.noname-c con 0x7f30ac06dd00 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:44.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 10 monclient(hunting): picked mon.noname-e con 0x7f30ac134b40 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:44.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 10 monclient(hunting): picked mon.noname-b con 0x7f30ac13b270 addr v1:172.21.15.115:6790/0 2024-01-24T23:22:44.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30ac06f500 con 0x7f30ac13b270 2024-01-24T23:22:44.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30ac06faa0 con 0x7f30ac06dd00 2024-01-24T23:22:44.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30ac06e150 con 0x7f30ac134b40 2024-01-24T23:22:44.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:44.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 10 monclient(hunting): authenticate will time out at 1051.781225s 2024-01-24T23:22:44.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30ab59e640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f30ac13b270 legacy=0x7f30ac1d36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:22:44.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a3fff640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f30ac134b40 legacy=0x7f30ac06edf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:44.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30aad9d640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f30ac06dd00 0x7f30ac06e6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:60352/0 (socket says 172.21.15.132:60352) 2024-01-24T23:22:44.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30aad9d640 1 -- 172.21.15.132:0/3228312867 learned_addr learned my addr 172.21.15.132:0/3228312867 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:44.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 1 -- 172.21.15.132:0/3228312867 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2577897605 0 0) 0x7f30ac06faa0 con 0x7f30ac06dd00 2024-01-24T23:22:44.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 10 monclient(hunting): _negotiate 2024-01-24T23:22:44.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:44.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:44.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 10 monclient(hunting): my global_id is 74722 2024-01-24T23:22:44.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 1 -- 172.21.15.132:0/3228312867 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f308c003530 con 0x7f30ac06dd00 2024-01-24T23:22:44.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 1 -- 172.21.15.132:0/3228312867 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2358946352 0 0) 0x7f308c003530 con 0x7f30ac06dd00 2024-01-24T23:22:44.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 1 -- 172.21.15.132:0/3228312867 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f30ac06faa0 con 0x7f30ac06dd00 2024-01-24T23:22:44.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 1 -- 172.21.15.132:0/3228312867 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f30940017b0 con 0x7f30ac06dd00 2024-01-24T23:22:44.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:44.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:22:44.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 10 monclient(hunting): dump: 2024-01-24T23:22:44.467 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:44.467 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:44.467 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:44.467 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:44.467 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:44.467 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:44.467 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:44.467 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:44.467 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:44.467 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:44.467 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:44.468 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:44.468 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:44.468 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:44.468 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:44.468 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:44.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:44.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 1 -- 172.21.15.132:0/3228312867 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2664688039 0 0) 0x7f30ac06faa0 con 0x7f30ac06dd00 2024-01-24T23:22:44.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 1 -- 172.21.15.132:0/3228312867 >> v1:172.21.15.115:6793/0 conn(0x7f30ac134b40 legacy=0x7f30ac06edf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:44.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 1 -- 172.21.15.132:0/3228312867 >> v1:172.21.15.115:6790/0 conn(0x7f30ac13b270 legacy=0x7f30ac1d36e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:44.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 10 monclient: _finish_hunting 0 2024-01-24T23:22:44.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 1 monclient: found mon.c 2024-01-24T23:22:44.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:44.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:22:44.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 1 -- 172.21.15.132:0/3228312867 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ac130290 con 0x7f30ac06dd00 2024-01-24T23:22:44.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 10 monclient: _finish_auth 0 2024-01-24T23:22:44.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 10 monclient: _check_auth_tickets 2024-01-24T23:22:44.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:44.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 5 monclient: authenticate success, global_id 74722 2024-01-24T23:22:44.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 10 monclient: _renew_subs 2024-01-24T23:22:44.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:22:44.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 1 -- 172.21.15.132:0/3228312867 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30ac130570 con 0x7f30ac06dd00 2024-01-24T23:22:44.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 10 monclient: _renew_subs 2024-01-24T23:22:44.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:22:44.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 1 -- 172.21.15.132:0/3228312867 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f30ac12ef20 con 0x7f30ac06dd00 2024-01-24T23:22:44.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 1 -- 172.21.15.132:0/3228312867 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3094004aa0 con 0x7f30ac06dd00 2024-01-24T23:22:44.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:44.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a959a640 4 set_mon_vals no callback set 2024-01-24T23:22:44.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 1 -- 172.21.15.132:0/3228312867 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3094004fd0 con 0x7f30ac06dd00 2024-01-24T23:22:44.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:44.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:22:44.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30a37fe640 10 monclient: dump: 2024-01-24T23:22:44.470 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:44.470 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:44.470 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:44.470 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:44.470 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:44.470 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:44.470 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:44.470 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:44.470 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:44.470 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:44.470 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:44.470 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:44.470 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:44.470 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:44.470 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:44.471 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:44.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:44.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:44.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:22:44.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.453+0000 7f30b1002640 1 -- 172.21.15.132:0/3228312867 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3090005090 con 0x7f30ac06dd00 2024-01-24T23:22:44.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.457+0000 7f30a37fe640 1 -- 172.21.15.132:0/3228312867 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f30940051f0 con 0x7f30ac06dd00 2024-01-24T23:22:44.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.457+0000 7f30a37fe640 1 -- 172.21.15.132:0/3228312867 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(29..29 src has 1..29) v4 ==== 4634+0+0 (unknown 593933149 0 0) 0x7f3094043b50 con 0x7f30ac06dd00 2024-01-24T23:22:44.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.457+0000 7f30a3fff640 10 monclient: get_auth_request con 0x7f308c036ef0 auth_method 0 2024-01-24T23:22:44.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.469+0000 7f30a37fe640 1 -- 172.21.15.132:0/3228312867 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f3094014ac0 con 0x7f30ac06dd00 2024-01-24T23:22:44.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.469+0000 7f30a37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:44.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.469+0000 7f30a37fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:44.626 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.621+0000 7f30b1002640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}] 2024-01-24T23:22:44.626 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.621+0000 7f30b1002640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}] 2024-01-24T23:22:44.626 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.621+0000 7f30b1002640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:22:44.626 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.621+0000 7f30b1002640 1 -- 172.21.15.132:0/3228312867 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd pool create", "pool": "foo", "pg_num": 123} v 0) v1 -- 0x7f3090005b50 con 0x7f30ac06dd00 2024-01-24T23:22:44.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.625+0000 7f30a37fe640 1 -- 172.21.15.132:0/3228312867 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}]=0 pool 'foo' already exists v29) v1 ==== 118+0+0 (unknown 1333109237 0 0) 0x7f30940146d0 con 0x7f30ac06dd00 2024-01-24T23:22:44.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.625+0000 7f30a37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}] 2024-01-24T23:22:44.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.625+0000 7f30a37fe640 10 monclient: _finish_command 2 = system:0 pool 'foo' already exists 2024-01-24T23:22:44.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.717+0000 7f30b1002640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}] 2024-01-24T23:22:44.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.717+0000 7f30b1002640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}] 2024-01-24T23:22:44.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.717+0000 7f30b1002640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:22:44.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.717+0000 7f30b1002640 1 -- 172.21.15.132:0/3228312867 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd pool create", "pool": "foo", "pg_num": 123} v 0) v1 -- 0x7f30900055d0 con 0x7f30ac06dd00 2024-01-24T23:22:44.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.721+0000 7f30a37fe640 1 -- 172.21.15.132:0/3228312867 <== mon.4 v1:172.21.15.115:6791/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}]=0 pool 'foo' already exists v29) v1 ==== 118+0+0 (unknown 1333109237 0 0) 0x7f30940146d0 con 0x7f30ac06dd00 2024-01-24T23:22:44.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.721+0000 7f30a37fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}] 2024-01-24T23:22:44.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.721+0000 7f30a37fe640 10 monclient: _finish_command 3 = system:0 pool 'foo' already exists 2024-01-24T23:22:44.723 INFO:tasks.workunit.client.0.smithi132.stderr:pool 'foo' already exists 2024-01-24T23:22:44.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.721+0000 7f30b1002640 1 -- 172.21.15.132:0/3228312867 >> v1:172.21.15.132:6812/1835401922 conn(0x7f308c036ef0 legacy=0x7f308c0393b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:44.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.721+0000 7f30b1002640 10 monclient: shutdown 2024-01-24T23:22:44.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.721+0000 7f30b1002640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:44.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.721+0000 7f30b1002640 1 -- 172.21.15.132:0/3228312867 >> v1:172.21.15.115:6791/0 conn(0x7f30ac06dd00 legacy=0x7f30ac06e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:44.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.721+0000 7f30b1002640 1 -- 172.21.15.132:0/3228312867 shutdown_connections 2024-01-24T23:22:44.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.721+0000 7f30b1002640 1 -- 172.21.15.132:0/3228312867 >> 172.21.15.132:0/3228312867 conn(0x7f30ac0a22a0 msgr2=0x7f30ac12b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:44.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.721+0000 7f30b1002640 1 -- 172.21.15.132:0/3228312867 shutdown_connections 2024-01-24T23:22:44.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.721+0000 7f30b1002640 1 -- 172.21.15.132:0/3228312867 wait complete. 2024-01-24T23:22:44.740 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool set foo size 1 --yes-i-really-mean-it 2024-01-24T23:22:44.860 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.857+0000 7f337f5d0640 10 monclient: get_monmap_and_config 2024-01-24T23:22:44.860 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.857+0000 7f337f5d0640 10 monclient: build_initial_monmap 2024-01-24T23:22:44.861 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.857+0000 7f337f5d0640 10 monclient: monmap: 2024-01-24T23:22:44.861 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:44.861 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:44.861 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:44.862604+0000 2024-01-24T23:22:44.861 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:44.862604+0000 2024-01-24T23:22:44.861 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:44.861 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:44.861 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:44.861 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:44.861 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:44.861 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:44.861 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:44.861 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:44.862 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:44.862 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:44.862 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:44.862 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:44.868 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.857+0000 7f337f5d0640 1 Processor -- start 2024-01-24T23:22:44.868 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.857+0000 7f337f5d0640 1 -- start start 2024-01-24T23:22:44.868 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.857+0000 7f337f5d0640 10 monclient: init 2024-01-24T23:22:44.868 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.857+0000 7f337f5d0640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:44.868 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.857+0000 7f337f5d0640 10 monclient: _add_conns ranks=[1,4,0,3,8,7,2,5,6] 2024-01-24T23:22:44.868 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.857+0000 7f337f5d0640 10 monclient(hunting): picked mon.noname-f con 0x7f3378133ce0 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:44.868 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.857+0000 7f337f5d0640 10 monclient(hunting): picked mon.noname-c con 0x7f337809ddc0 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:44.869 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.857+0000 7f337f5d0640 10 monclient(hunting): picked mon.noname-a con 0x7f337809e1a0 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:44.869 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3378064700 con 0x7f337809e1a0 2024-01-24T23:22:44.869 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f337809c6d0 con 0x7f3378133ce0 2024-01-24T23:22:44.869 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f337809c9b0 con 0x7f337809ddc0 2024-01-24T23:22:44.869 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:44.869 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 10 monclient(hunting): authenticate will time out at 1052.188058s 2024-01-24T23:22:44.869 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337d36c640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f3378133ce0 0x7f33781340c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:37954/0 (socket says 172.21.15.132:37954) 2024-01-24T23:22:44.869 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337d36c640 1 -- 172.21.15.132:0/1327814184 learned_addr learned my addr 172.21.15.132:0/1327814184 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:44.869 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 1 -- 172.21.15.132:0/1327814184 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266779637 0 0) 0x7f337809c6d0 con 0x7f3378133ce0 2024-01-24T23:22:44.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:44.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:44.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:44.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 10 monclient(hunting): my global_id is 4708 2024-01-24T23:22:44.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 1 -- 172.21.15.132:0/1327814184 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3364003530 con 0x7f3378133ce0 2024-01-24T23:22:44.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 1 -- 172.21.15.132:0/1327814184 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3009636744 0 0) 0x7f3364003530 con 0x7f3378133ce0 2024-01-24T23:22:44.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 1 -- 172.21.15.132:0/1327814184 >> v1:172.21.15.115:6791/0 conn(0x7f337809ddc0 legacy=0x7f3378138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:44.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 1 -- 172.21.15.132:0/1327814184 >> v1:172.21.15.115:6789/0 conn(0x7f337809e1a0 legacy=0x7f337813c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:44.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 10 monclient: _finish_hunting 0 2024-01-24T23:22:44.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 1 monclient: found mon.noname-f 2024-01-24T23:22:44.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:44.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:22:44.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 1 -- 172.21.15.132:0/1327814184 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f337809cc90 con 0x7f3378133ce0 2024-01-24T23:22:44.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 10 monclient: _finish_auth 0 2024-01-24T23:22:44.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 10 monclient: _check_auth_tickets 2024-01-24T23:22:44.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:44.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 1 -- 172.21.15.132:0/1327814184 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f33740017c0 con 0x7f3378133ce0 2024-01-24T23:22:44.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:44.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:22:44.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 10 monclient: dump: 2024-01-24T23:22:44.871 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:44.871 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:44.871 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:44.871 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:44.871 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:44.871 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:44.871 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:44.872 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:44.872 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:44.872 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:44.872 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:44.872 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:44.872 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:44.872 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:44.872 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:44.872 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:44.872 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 5 monclient: authenticate success, global_id 4708 2024-01-24T23:22:44.872 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:44.872 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 1 -- 172.21.15.132:0/1327814184 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3374001a70 con 0x7f3378133ce0 2024-01-24T23:22:44.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:44.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 1 -- 172.21.15.132:0/1327814184 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3374004bf0 con 0x7f3378133ce0 2024-01-24T23:22:44.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:44.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 4 set_mon_vals no callback set 2024-01-24T23:22:44.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:44.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:22:44.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336ffff640 10 monclient: dump: 2024-01-24T23:22:44.873 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:44.873 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:44.873 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:44.873 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:44.873 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:44.873 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:44.873 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:44.874 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:44.874 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:44.874 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:44.874 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:44.874 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:44.874 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:44.874 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:44.874 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:44.874 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:44.874 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 10 monclient: shutdown 2024-01-24T23:22:44.874 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:44.874 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 1 -- 172.21.15.132:0/1327814184 >> v1:172.21.15.132:6789/0 conn(0x7f3378133ce0 legacy=0x7f33781340c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:44.874 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 1 -- 172.21.15.132:0/1327814184 shutdown_connections 2024-01-24T23:22:44.874 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 1 -- 172.21.15.132:0/1327814184 >> 172.21.15.132:0/1327814184 conn(0x7f33780a6840 msgr2=0x7f33780a2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:44.875 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 1 -- 172.21.15.132:0/1327814184 shutdown_connections 2024-01-24T23:22:44.875 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 1 -- 172.21.15.132:0/1327814184 wait complete. 2024-01-24T23:22:44.875 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 10 monclient: build_initial_monmap 2024-01-24T23:22:44.875 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 10 monclient: monmap: 2024-01-24T23:22:44.875 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:44.875 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:44.875 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:44.875 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:44.875 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:44.875 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:44.875 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:44.875 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:44.876 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:44.876 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:44.876 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:44.876 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:44.876 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:44.876 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:44.876 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:44.876 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:44.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 1 Processor -- start 2024-01-24T23:22:44.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 1 -- start start 2024-01-24T23:22:44.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 10 monclient: init 2024-01-24T23:22:44.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:44.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 10 monclient: _add_conns ranks=[3,0,2,7,5,6,4,1,8] 2024-01-24T23:22:44.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 10 monclient(hunting): picked mon.noname-g con 0x7f337809ddc0 addr v1:172.21.15.132:6790/0 2024-01-24T23:22:44.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 10 monclient(hunting): picked mon.noname-a con 0x7f337809e1a0 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:44.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 10 monclient(hunting): picked mon.noname-b con 0x7f3378133ce0 addr v1:172.21.15.115:6790/0 2024-01-24T23:22:44.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33781303d0 con 0x7f337809e1a0 2024-01-24T23:22:44.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33780723d0 con 0x7f3378133ce0 2024-01-24T23:22:44.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33781210b0 con 0x7f337809ddc0 2024-01-24T23:22:44.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:44.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 10 monclient(hunting): authenticate will time out at 1052.191183s 2024-01-24T23:22:44.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337db6d640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f3378133ce0 legacy=0x7f33781d3b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:22:44.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337d36c640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f337809ddc0 0x7f33781cb8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:48736/0 (socket says 172.21.15.132:48736) 2024-01-24T23:22:44.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337d36c640 1 -- 172.21.15.132:0/3467896577 learned_addr learned my addr 172.21.15.132:0/3467896577 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:44.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 1 -- 172.21.15.132:0/3467896577 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3837892312 0 0) 0x7f33781210b0 con 0x7f337809ddc0 2024-01-24T23:22:44.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:22:44.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:44.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:44.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 10 monclient(hunting): my global_id is 4638 2024-01-24T23:22:44.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 1 -- 172.21.15.132:0/3467896577 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f335c003530 con 0x7f337809ddc0 2024-01-24T23:22:44.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 1 -- 172.21.15.132:0/3467896577 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3062076733 0 0) 0x7f335c003530 con 0x7f337809ddc0 2024-01-24T23:22:44.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 1 -- 172.21.15.132:0/3467896577 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f33781210b0 con 0x7f337809ddc0 2024-01-24T23:22:44.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 1 -- 172.21.15.132:0/3467896577 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3374004c30 con 0x7f337809ddc0 2024-01-24T23:22:44.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:44.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:22:44.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 10 monclient(hunting): dump: 2024-01-24T23:22:44.878 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:44.879 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:44.879 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:44.879 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:44.879 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:44.879 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:44.879 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:44.879 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:44.879 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:44.879 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:44.879 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:44.879 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:44.879 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:44.879 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:44.880 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:44.880 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:44.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:44.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 1 -- 172.21.15.132:0/3467896577 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3678003842 0 0) 0x7f33781303d0 con 0x7f337809e1a0 2024-01-24T23:22:44.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:22:44.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:44.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:44.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 10 monclient(hunting): my global_id is 4707 2024-01-24T23:22:44.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 1 -- 172.21.15.132:0/3467896577 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f335c003530 con 0x7f337809e1a0 2024-01-24T23:22:44.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 1 -- 172.21.15.132:0/3467896577 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 148495365 0 0) 0x7f33781210b0 con 0x7f337809ddc0 2024-01-24T23:22:44.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 1 -- 172.21.15.132:0/3467896577 >> v1:172.21.15.115:6790/0 conn(0x7f3378133ce0 legacy=0x7f33781d3b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:44.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 1 -- 172.21.15.132:0/3467896577 >> v1:172.21.15.115:6789/0 conn(0x7f337809e1a0 legacy=0x7f33781cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:44.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 10 monclient: _finish_hunting 0 2024-01-24T23:22:44.881 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 1 monclient: found mon.g 2024-01-24T23:22:44.881 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:44.881 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:44.881 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 1 -- 172.21.15.132:0/3467896577 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3378121670 con 0x7f337809ddc0 2024-01-24T23:22:44.881 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 10 monclient: _finish_auth 0 2024-01-24T23:22:44.881 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 10 monclient: _check_auth_tickets 2024-01-24T23:22:44.881 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f336dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:44.881 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 5 monclient: authenticate success, global_id 4638 2024-01-24T23:22:44.881 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 10 monclient: _renew_subs 2024-01-24T23:22:44.881 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:44.881 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.861+0000 7f337f5d0640 1 -- 172.21.15.132:0/3467896577 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33781219b0 con 0x7f337809ddc0 2024-01-24T23:22:44.881 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.865+0000 7f337f5d0640 10 monclient: _renew_subs 2024-01-24T23:22:44.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.865+0000 7f337f5d0640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:44.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.865+0000 7f337f5d0640 1 -- 172.21.15.132:0/3467896577 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3378071020 con 0x7f337809ddc0 2024-01-24T23:22:44.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.865+0000 7f336dffb640 1 -- 172.21.15.132:0/3467896577 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3374004e20 con 0x7f337809ddc0 2024-01-24T23:22:44.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.865+0000 7f336dffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:44.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.865+0000 7f336dffb640 1 -- 172.21.15.132:0/3467896577 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f33740052d0 con 0x7f337809ddc0 2024-01-24T23:22:44.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.865+0000 7f336dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:44.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.865+0000 7f336dffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:22:44.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.865+0000 7f336f7fe640 4 set_mon_vals no callback set 2024-01-24T23:22:44.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.865+0000 7f336dffb640 10 monclient: dump: 2024-01-24T23:22:44.882 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:44.882 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:44.882 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:44.883 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:44.883 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:44.883 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:44.883 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:44.883 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:44.883 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:44.883 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:44.883 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:44.883 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:44.883 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:44.883 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:44.883 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:44.883 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:44.883 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.865+0000 7f337f5d0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:44.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.865+0000 7f337f5d0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:44.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.865+0000 7f337f5d0640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:44.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.865+0000 7f337f5d0640 1 -- 172.21.15.132:0/3467896577 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3358005090 con 0x7f337809ddc0 2024-01-24T23:22:44.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.865+0000 7f336dffb640 1 -- 172.21.15.132:0/3467896577 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f337400fd50 con 0x7f337809ddc0 2024-01-24T23:22:44.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.865+0000 7f336dffb640 1 -- 172.21.15.132:0/3467896577 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(29..29 src has 1..29) v4 ==== 4634+0+0 (unknown 593933149 0 0) 0x7f3374043d40 con 0x7f337809ddc0 2024-01-24T23:22:44.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.865+0000 7f337cb6b640 10 monclient: get_auth_request con 0x7f335c036f70 auth_method 0 2024-01-24T23:22:44.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.869+0000 7f336dffb640 1 -- 172.21.15.132:0/3467896577 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f33740111b0 con 0x7f337809ddc0 2024-01-24T23:22:44.885 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.869+0000 7f336dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:44.885 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:44.869+0000 7f336dffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:45.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:45.025+0000 7f337f5d0640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}] 2024-01-24T23:22:45.029 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:45.025+0000 7f337f5d0640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}] 2024-01-24T23:22:45.029 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:45.025+0000 7f337f5d0640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:45.029 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:45.025+0000 7f337f5d0640 1 -- 172.21.15.132:0/3467896577 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f33580058c0 con 0x7f337809ddc0 2024-01-24T23:22:45.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:45.189+0000 7f336dffb640 1 -- 172.21.15.132:0/3467896577 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}]=0 set pool 4 size to 1 v30) v1 ==== 152+0+0 (unknown 919328176 0 0) 0x7f33740131f0 con 0x7f337809ddc0 2024-01-24T23:22:45.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:45.189+0000 7f336dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}] 2024-01-24T23:22:45.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:45.189+0000 7f336dffb640 10 monclient: _finish_command 2 = system:0 set pool 4 size to 1 2024-01-24T23:22:45.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:45.329+0000 7f337f5d0640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}] 2024-01-24T23:22:45.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:45.329+0000 7f337f5d0640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}] 2024-01-24T23:22:45.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:45.329+0000 7f337f5d0640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:45.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:45.329+0000 7f337f5d0640 1 -- 172.21.15.132:0/3467896577 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f33580053f0 con 0x7f337809ddc0 2024-01-24T23:22:46.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.225+0000 7f336dffb640 1 -- 172.21.15.132:0/3467896577 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}]=0 set pool 4 size to 1 v31) v1 ==== 152+0+0 (unknown 2973845271 0 0) 0x7f33580053f0 con 0x7f337809ddc0 2024-01-24T23:22:46.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.225+0000 7f336dffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}] 2024-01-24T23:22:46.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.225+0000 7f336dffb640 10 monclient: _finish_command 3 = system:0 set pool 4 size to 1 2024-01-24T23:22:46.234 INFO:tasks.workunit.client.0.smithi132.stderr:set pool 4 size to 1 2024-01-24T23:22:46.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.233+0000 7f337f5d0640 1 -- 172.21.15.132:0/3467896577 >> v1:172.21.15.132:6812/1835401922 conn(0x7f335c036f70 legacy=0x7f335c039430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:46.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.233+0000 7f337f5d0640 10 monclient: shutdown 2024-01-24T23:22:46.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.233+0000 7f337f5d0640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:46.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.233+0000 7f337f5d0640 1 -- 172.21.15.132:0/3467896577 >> v1:172.21.15.132:6790/0 conn(0x7f337809ddc0 legacy=0x7f33781cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:46.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.237+0000 7f337f5d0640 1 -- 172.21.15.132:0/3467896577 shutdown_connections 2024-01-24T23:22:46.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.237+0000 7f337f5d0640 1 -- 172.21.15.132:0/3467896577 >> 172.21.15.132:0/3467896577 conn(0x7f33780a6840 msgr2=0x7f3378130860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:46.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.237+0000 7f337f5d0640 1 -- 172.21.15.132:0/3467896577 shutdown_connections 2024-01-24T23:22:46.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.237+0000 7f337f5d0640 1 -- 172.21.15.132:0/3467896577 wait complete. 2024-01-24T23:22:46.274 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_config_value foo min_size 1 2024-01-24T23:22:46.274 INFO:tasks.workunit.client.0.smithi132.stderr:+ local pool_name config_opt expected_val val 2024-01-24T23:22:46.274 INFO:tasks.workunit.client.0.smithi132.stderr:+ pool_name=foo 2024-01-24T23:22:46.274 INFO:tasks.workunit.client.0.smithi132.stderr:+ config_opt=min_size 2024-01-24T23:22:46.274 INFO:tasks.workunit.client.0.smithi132.stderr:+ expected_val=1 2024-01-24T23:22:46.274 INFO:tasks.workunit.client.0.smithi132.stderr:++ get_config_value_or_die foo min_size 2024-01-24T23:22:46.274 INFO:tasks.workunit.client.0.smithi132.stderr:++ local pool_name config_opt raw val 2024-01-24T23:22:46.274 INFO:tasks.workunit.client.0.smithi132.stderr:++ pool_name=foo 2024-01-24T23:22:46.275 INFO:tasks.workunit.client.0.smithi132.stderr:++ config_opt=min_size 2024-01-24T23:22:46.275 INFO:tasks.workunit.client.0.smithi132.stderr:+++ ceph osd pool get foo min_size 2024-01-24T23:22:46.668 INFO:tasks.workunit.client.0.smithi132.stderr:++ raw='min_size: 1' 2024-01-24T23:22:46.668 INFO:tasks.workunit.client.0.smithi132.stderr:++ [[ 0 -ne 0 ]] 2024-01-24T23:22:46.669 INFO:tasks.workunit.client.0.smithi132.stderr:+++ echo min_size: 1 2024-01-24T23:22:46.669 INFO:tasks.workunit.client.0.smithi132.stderr:+++ sed -e 's/[{} "]//g' 2024-01-24T23:22:46.671 INFO:tasks.workunit.client.0.smithi132.stderr:++ raw=min_size:1 2024-01-24T23:22:46.671 INFO:tasks.workunit.client.0.smithi132.stderr:+++ echo min_size:1 2024-01-24T23:22:46.672 INFO:tasks.workunit.client.0.smithi132.stderr:+++ cut -f2 -d: 2024-01-24T23:22:46.673 INFO:tasks.workunit.client.0.smithi132.stderr:++ val=1 2024-01-24T23:22:46.673 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo 1 2024-01-24T23:22:46.673 INFO:tasks.workunit.client.0.smithi132.stderr:++ return 0 2024-01-24T23:22:46.673 INFO:tasks.workunit.client.0.smithi132.stderr:+ val=1 2024-01-24T23:22:46.673 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ 1 != \1 ]] 2024-01-24T23:22:46.673 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool set foo size 4 2024-01-24T23:22:46.786 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.781+0000 7f75e5d5a640 10 monclient: get_monmap_and_config 2024-01-24T23:22:46.786 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.781+0000 7f75e5d5a640 10 monclient: build_initial_monmap 2024-01-24T23:22:46.786 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.781+0000 7f75e5d5a640 10 monclient: monmap: 2024-01-24T23:22:46.787 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:46.787 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:46.787 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:46.788435+0000 2024-01-24T23:22:46.787 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:46.788435+0000 2024-01-24T23:22:46.787 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:46.787 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:46.787 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:46.787 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:46.787 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:46.787 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:46.787 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:46.787 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:46.787 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:46.787 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:46.787 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:46.787 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:46.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 1 Processor -- start 2024-01-24T23:22:46.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 1 -- start start 2024-01-24T23:22:46.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 10 monclient: init 2024-01-24T23:22:46.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:46.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 10 monclient: _add_conns ranks=[8,6,1,7,4,5,3,2,0] 2024-01-24T23:22:46.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 10 monclient(hunting): picked mon.noname-e con 0x7f75e0135ea0 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:46.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 10 monclient(hunting): picked mon.noname-d con 0x7f75e009bcd0 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:46.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 10 monclient(hunting): picked mon.noname-f con 0x7f75e009c0f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:46.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e0064700 con 0x7f75e009c0f0 2024-01-24T23:22:46.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e0136990 con 0x7f75e009bcd0 2024-01-24T23:22:46.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e009b010 con 0x7f75e0135ea0 2024-01-24T23:22:46.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:46.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 10 monclient(hunting): authenticate will time out at 1054.113375s 2024-01-24T23:22:46.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75dffff640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f75e009c0f0 0x7f75e013f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:37958/0 (socket says 172.21.15.132:37958) 2024-01-24T23:22:46.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75df7fe640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f75e0135ea0 legacy=0x7f75e0136280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:46.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75dffff640 1 -- 172.21.15.132:0/1585063526 learned_addr learned my addr 172.21.15.132:0/1585063526 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:46.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 1 -- 172.21.15.132:0/1585063526 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2226027993 0 0) 0x7f75e0064700 con 0x7f75e009c0f0 2024-01-24T23:22:46.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:46.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:46.792 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:46.792 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 10 monclient(hunting): my global_id is 4717 2024-01-24T23:22:46.792 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 1 -- 172.21.15.132:0/1585063526 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75c8003530 con 0x7f75e009c0f0 2024-01-24T23:22:46.792 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 1 -- 172.21.15.132:0/1585063526 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1808362274 0 0) 0x7f75c8003530 con 0x7f75e009c0f0 2024-01-24T23:22:46.792 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 1 -- 172.21.15.132:0/1585063526 >> v1:172.21.15.115:6793/0 conn(0x7f75e0135ea0 legacy=0x7f75e0136280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:46.792 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 1 -- 172.21.15.132:0/1585063526 >> v1:172.21.15.115:6792/0 conn(0x7f75e009bcd0 legacy=0x7f75e013bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:46.792 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:22:46.792 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 1 monclient: found mon.noname-f 2024-01-24T23:22:46.792 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:46.792 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:22:46.792 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 1 -- 172.21.15.132:0/1585063526 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e00a4f50 con 0x7f75e009c0f0 2024-01-24T23:22:46.792 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 10 monclient: _finish_auth 0 2024-01-24T23:22:46.792 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:22:46.792 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:46.792 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 1 -- 172.21.15.132:0/1585063526 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f75d8001b40 con 0x7f75e009c0f0 2024-01-24T23:22:46.793 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:46.793 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:22:46.793 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 10 monclient: dump: 2024-01-24T23:22:46.793 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:46.793 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:46.793 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:46.793 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:46.793 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:46.793 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:46.793 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:46.793 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:46.793 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:46.793 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:46.793 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:46.793 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:46.793 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:46.794 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:46.794 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:46.794 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:46.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 5 monclient: authenticate success, global_id 4717 2024-01-24T23:22:46.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:46.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 1 -- 172.21.15.132:0/1585063526 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f75d8001e60 con 0x7f75e009c0f0 2024-01-24T23:22:46.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:46.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 1 -- 172.21.15.132:0/1585063526 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f75d8004f70 con 0x7f75e009c0f0 2024-01-24T23:22:46.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:46.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 4 set_mon_vals no callback set 2024-01-24T23:22:46.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:46.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:22:46.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75de7fc640 10 monclient: dump: 2024-01-24T23:22:46.794 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:46.794 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:46.794 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:46.795 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:46.795 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:46.795 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:46.795 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:46.795 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:46.795 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:46.795 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:46.795 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:46.795 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:46.795 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:46.795 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:46.795 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:46.795 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:46.795 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 10 monclient: shutdown 2024-01-24T23:22:46.795 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:46.795 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 1 -- 172.21.15.132:0/1585063526 >> v1:172.21.15.132:6789/0 conn(0x7f75e009c0f0 legacy=0x7f75e013f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:46.796 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 1 -- 172.21.15.132:0/1585063526 shutdown_connections 2024-01-24T23:22:46.796 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 1 -- 172.21.15.132:0/1585063526 >> 172.21.15.132:0/1585063526 conn(0x7f75e00a9bf0 msgr2=0x7f75e00a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:46.796 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 1 -- 172.21.15.132:0/1585063526 shutdown_connections 2024-01-24T23:22:46.796 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 1 -- 172.21.15.132:0/1585063526 wait complete. 2024-01-24T23:22:46.796 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 10 monclient: build_initial_monmap 2024-01-24T23:22:46.796 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 10 monclient: monmap: 2024-01-24T23:22:46.796 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:46.796 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:46.796 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:46.796 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:46.796 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:46.796 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:46.796 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:46.796 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:46.797 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:46.797 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:46.797 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:46.797 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:46.797 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:46.797 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:46.797 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:46.797 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:46.797 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 1 Processor -- start 2024-01-24T23:22:46.797 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 1 -- start start 2024-01-24T23:22:46.797 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.785+0000 7f75e5d5a640 10 monclient: init 2024-01-24T23:22:46.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75e5d5a640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:46.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75e5d5a640 10 monclient: _add_conns ranks=[3,5,1,7,4,6,8,0,2] 2024-01-24T23:22:46.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75e5d5a640 10 monclient(hunting): picked mon.noname-g con 0x7f75e009bcd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:22:46.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75e5d5a640 10 monclient(hunting): picked mon.noname-h con 0x7f75e009c0f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:46.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75e5d5a640 10 monclient(hunting): picked mon.noname-f con 0x7f75e0135ea0 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:46.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75e5d5a640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e0135c70 con 0x7f75e0135ea0 2024-01-24T23:22:46.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75e5d5a640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e01cb920 con 0x7f75e009bcd0 2024-01-24T23:22:46.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75e5d5a640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e009fce0 con 0x7f75e009c0f0 2024-01-24T23:22:46.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75e5d5a640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:46.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75e5d5a640 10 monclient(hunting): authenticate will time out at 1054.116533s 2024-01-24T23:22:46.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75df7fe640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f75e009bcd0 0x7f75e0134f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:48760/0 (socket says 172.21.15.132:48760) 2024-01-24T23:22:46.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75df7fe640 1 -- 172.21.15.132:0/1041097534 learned_addr learned my addr 172.21.15.132:0/1041097534 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:46.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 46160976 0 0) 0x7f75e01cb920 con 0x7f75e009bcd0 2024-01-24T23:22:46.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:46.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:46.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:46.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient(hunting): my global_id is 4665 2024-01-24T23:22:46.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75cc003530 con 0x7f75e009bcd0 2024-01-24T23:22:46.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 809767086 0 0) 0x7f75e0135c70 con 0x7f75e0135ea0 2024-01-24T23:22:46.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:46.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:46.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:46.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient(hunting): my global_id is 4726 2024-01-24T23:22:46.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75e01cb920 con 0x7f75e0135ea0 2024-01-24T23:22:46.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181634255 0 0) 0x7f75e009fce0 con 0x7f75e009c0f0 2024-01-24T23:22:46.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:46.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:46.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:46.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient(hunting): my global_id is 4676 2024-01-24T23:22:46.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75e0135c70 con 0x7f75e009c0f0 2024-01-24T23:22:46.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3820926776 0 0) 0x7f75cc003530 con 0x7f75e009bcd0 2024-01-24T23:22:46.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f75e009fce0 con 0x7f75e009bcd0 2024-01-24T23:22:46.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f75d4004330 con 0x7f75e009bcd0 2024-01-24T23:22:46.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:46.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:22:46.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient(hunting): dump: 2024-01-24T23:22:46.802 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:46.802 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:46.802 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:46.802 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:46.802 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:46.803 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:46.803 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:46.803 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:46.803 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:46.803 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:46.803 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:46.803 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:46.803 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:46.803 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:46.803 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:46.803 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:46.803 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:46.803 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 709525045 0 0) 0x7f75e0135c70 con 0x7f75e009c0f0 2024-01-24T23:22:46.803 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f75cc003530 con 0x7f75e009c0f0 2024-01-24T23:22:46.803 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f75d00019e0 con 0x7f75e009c0f0 2024-01-24T23:22:46.803 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:46.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:22:46.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient(hunting): dump: 2024-01-24T23:22:46.804 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:46.804 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:46.804 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:46.804 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:46.804 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:46.804 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:46.804 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:46.804 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:46.804 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:46.804 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:46.804 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:46.804 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:46.804 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:46.804 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:46.805 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:46.805 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:46.805 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1304449753 0 0) 0x7f75e01cb920 con 0x7f75e0135ea0 2024-01-24T23:22:46.805 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f75e0135c70 con 0x7f75e0135ea0 2024-01-24T23:22:46.805 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f75d8001b00 con 0x7f75e0135ea0 2024-01-24T23:22:46.805 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:46.805 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:22:46.805 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient(hunting): dump: 2024-01-24T23:22:46.805 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:46.805 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:46.805 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:46.805 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:46.805 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:46.805 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:46.805 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:46.806 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:46.806 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:46.806 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:46.806 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:46.806 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:46.806 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:46.806 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:46.806 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:46.806 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:46.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 482415862 0 0) 0x7f75e009fce0 con 0x7f75e009bcd0 2024-01-24T23:22:46.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 >> v1:172.21.15.132:6791/0 conn(0x7f75e009c0f0 legacy=0x7f75e01d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:46.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 >> v1:172.21.15.132:6789/0 conn(0x7f75e0135ea0 legacy=0x7f75e01d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:46.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient: _finish_hunting 0 2024-01-24T23:22:46.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 1 monclient: found mon.g 2024-01-24T23:22:46.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:46.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:46.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e0121e00 con 0x7f75e009bcd0 2024-01-24T23:22:46.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient: _finish_auth 0 2024-01-24T23:22:46.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient: _check_auth_tickets 2024-01-24T23:22:46.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:46.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75e5d5a640 5 monclient: authenticate success, global_id 4665 2024-01-24T23:22:46.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75e5d5a640 10 monclient: _renew_subs 2024-01-24T23:22:46.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75e5d5a640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:46.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75e5d5a640 1 -- 172.21.15.132:0/1041097534 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e01220e0 con 0x7f75e009bcd0 2024-01-24T23:22:46.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75e5d5a640 10 monclient: _renew_subs 2024-01-24T23:22:46.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75e5d5a640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:46.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f75d40045a0 con 0x7f75e009bcd0 2024-01-24T23:22:46.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75e5d5a640 1 -- 172.21.15.132:0/1041097534 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e0120f10 con 0x7f75e009bcd0 2024-01-24T23:22:46.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:46.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f75d4004ad0 con 0x7f75e009bcd0 2024-01-24T23:22:46.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:46.808 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:22:46.808 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 10 monclient: dump: 2024-01-24T23:22:46.808 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:46.808 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:46.808 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:46.808 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:46.808 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:46.809 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:46.809 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:46.809 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:46.809 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:46.809 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:46.809 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:46.809 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:46.810 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:46.810 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:46.810 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:46.810 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:46.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75ddffb640 4 set_mon_vals no callback set 2024-01-24T23:22:46.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f75d4004c30 con 0x7f75e009bcd0 2024-01-24T23:22:46.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75e5d5a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:46.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75e5d5a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:46.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75e5d5a640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:46.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75e5d5a640 1 -- 172.21.15.132:0/1041097534 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e0067580 con 0x7f75e009bcd0 2024-01-24T23:22:46.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(31..31 src has 1..31) v4 ==== 4634+0+0 (unknown 1421202539 0 0) 0x7f75d4043ac0 con 0x7f75e009bcd0 2024-01-24T23:22:46.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.789+0000 7f75deffd640 10 monclient: get_auth_request con 0x7f75cc037bd0 auth_method 0 2024-01-24T23:22:46.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.793+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f75d40149d0 con 0x7f75e009bcd0 2024-01-24T23:22:46.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.793+0000 7f75c7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:46.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.793+0000 7f75c7fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:46.961 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.957+0000 7f75e5d5a640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}] 2024-01-24T23:22:46.961 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.957+0000 7f75e5d5a640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}] 2024-01-24T23:22:46.961 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.957+0000 7f75e5d5a640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:46.961 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:46.957+0000 7f75e5d5a640 1 -- 172.21.15.132:0/1041097534 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"} v 0) v1 -- 0x7f75e0073840 con 0x7f75e009bcd0 2024-01-24T23:22:47.860 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:47.853+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}]=0 set pool 4 size to 4 v32) v1 ==== 122+0+0 (unknown 3836657792 0 0) 0x7f75d40145c0 con 0x7f75e009bcd0 2024-01-24T23:22:47.860 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:47.853+0000 7f75c7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}] 2024-01-24T23:22:47.860 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:47.853+0000 7f75c7fff640 10 monclient: _finish_command 2 = system:0 set pool 4 size to 4 2024-01-24T23:22:47.980 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:47.977+0000 7f75e5d5a640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}] 2024-01-24T23:22:47.980 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:47.977+0000 7f75e5d5a640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}] 2024-01-24T23:22:47.980 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:47.977+0000 7f75e5d5a640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:47.980 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:47.977+0000 7f75e5d5a640 1 -- 172.21.15.132:0/1041097534 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"} v 0) v1 -- 0x7f75e00008e0 con 0x7f75e009bcd0 2024-01-24T23:22:48.857 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:48.853+0000 7f75c7fff640 1 -- 172.21.15.132:0/1041097534 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}]=0 set pool 4 size to 4 v33) v1 ==== 122+0+0 (unknown 1714120015 0 0) 0x7f75d40147a0 con 0x7f75e009bcd0 2024-01-24T23:22:48.857 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:48.853+0000 7f75c7fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}] 2024-01-24T23:22:48.857 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:48.853+0000 7f75c7fff640 10 monclient: _finish_command 3 = system:0 set pool 4 size to 4 2024-01-24T23:22:48.857 INFO:tasks.workunit.client.0.smithi132.stderr:set pool 4 size to 4 2024-01-24T23:22:48.863 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:48.861+0000 7f75e5d5a640 1 -- 172.21.15.132:0/1041097534 >> v1:172.21.15.132:6812/1835401922 conn(0x7f75cc037bd0 legacy=0x7f75cc03a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:48.863 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:48.861+0000 7f75e5d5a640 10 monclient: shutdown 2024-01-24T23:22:48.863 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:48.861+0000 7f75e5d5a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:48.863 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:48.861+0000 7f75e5d5a640 1 -- 172.21.15.132:0/1041097534 >> v1:172.21.15.132:6790/0 conn(0x7f75e009bcd0 legacy=0x7f75e0134f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:48.864 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:48.861+0000 7f75e5d5a640 1 -- 172.21.15.132:0/1041097534 shutdown_connections 2024-01-24T23:22:48.865 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:48.861+0000 7f75e5d5a640 1 -- 172.21.15.132:0/1041097534 >> 172.21.15.132:0/1041097534 conn(0x7f75e00a9bf0 msgr2=0x7f75e00a8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:48.865 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:48.861+0000 7f75e5d5a640 1 -- 172.21.15.132:0/1041097534 shutdown_connections 2024-01-24T23:22:48.865 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:48.861+0000 7f75e5d5a640 1 -- 172.21.15.132:0/1041097534 wait complete. 2024-01-24T23:22:48.898 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_config_value foo min_size 2 2024-01-24T23:22:48.898 INFO:tasks.workunit.client.0.smithi132.stderr:+ local pool_name config_opt expected_val val 2024-01-24T23:22:48.898 INFO:tasks.workunit.client.0.smithi132.stderr:+ pool_name=foo 2024-01-24T23:22:48.899 INFO:tasks.workunit.client.0.smithi132.stderr:+ config_opt=min_size 2024-01-24T23:22:48.899 INFO:tasks.workunit.client.0.smithi132.stderr:+ expected_val=2 2024-01-24T23:22:48.899 INFO:tasks.workunit.client.0.smithi132.stderr:++ get_config_value_or_die foo min_size 2024-01-24T23:22:48.899 INFO:tasks.workunit.client.0.smithi132.stderr:++ local pool_name config_opt raw val 2024-01-24T23:22:48.899 INFO:tasks.workunit.client.0.smithi132.stderr:++ pool_name=foo 2024-01-24T23:22:48.899 INFO:tasks.workunit.client.0.smithi132.stderr:++ config_opt=min_size 2024-01-24T23:22:48.899 INFO:tasks.workunit.client.0.smithi132.stderr:+++ ceph osd pool get foo min_size 2024-01-24T23:22:49.310 INFO:tasks.workunit.client.0.smithi132.stderr:++ raw='min_size: 2' 2024-01-24T23:22:49.310 INFO:tasks.workunit.client.0.smithi132.stderr:++ [[ 0 -ne 0 ]] 2024-01-24T23:22:49.311 INFO:tasks.workunit.client.0.smithi132.stderr:+++ echo min_size: 2 2024-01-24T23:22:49.311 INFO:tasks.workunit.client.0.smithi132.stderr:+++ sed -e 's/[{} "]//g' 2024-01-24T23:22:49.312 INFO:tasks.workunit.client.0.smithi132.stderr:++ raw=min_size:2 2024-01-24T23:22:49.313 INFO:tasks.workunit.client.0.smithi132.stderr:+++ echo min_size:2 2024-01-24T23:22:49.313 INFO:tasks.workunit.client.0.smithi132.stderr:+++ cut -f2 -d: 2024-01-24T23:22:49.314 INFO:tasks.workunit.client.0.smithi132.stderr:++ val=2 2024-01-24T23:22:49.314 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo 2 2024-01-24T23:22:49.314 INFO:tasks.workunit.client.0.smithi132.stderr:++ return 0 2024-01-24T23:22:49.314 INFO:tasks.workunit.client.0.smithi132.stderr:+ val=2 2024-01-24T23:22:49.314 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ 2 != \2 ]] 2024-01-24T23:22:49.315 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool set foo size 10 2024-01-24T23:22:49.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.417+0000 7fc375871640 10 monclient: get_monmap_and_config 2024-01-24T23:22:49.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.417+0000 7fc375871640 10 monclient: build_initial_monmap 2024-01-24T23:22:49.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.417+0000 7fc375871640 10 monclient: monmap: 2024-01-24T23:22:49.423 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:49.423 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:49.423 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:49.424616+0000 2024-01-24T23:22:49.423 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:49.424616+0000 2024-01-24T23:22:49.423 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:49.423 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:49.423 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:49.423 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:49.423 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:49.423 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:49.424 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:49.424 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:49.424 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:49.424 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:49.424 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:49.424 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:49.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 1 Processor -- start 2024-01-24T23:22:49.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 1 -- start start 2024-01-24T23:22:49.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 10 monclient: init 2024-01-24T23:22:49.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:49.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 10 monclient: _add_conns ranks=[3,5,7,8,6,4,1,2,0] 2024-01-24T23:22:49.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 10 monclient(hunting): picked mon.noname-g con 0x7fc37009b810 addr v1:172.21.15.132:6790/0 2024-01-24T23:22:49.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 10 monclient(hunting): picked mon.noname-h con 0x7fc37009bbf0 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:49.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 10 monclient(hunting): picked mon.noname-i con 0x7fc3701370a0 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:49.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc370064700 con 0x7fc37009b810 2024-01-24T23:22:49.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc370099af0 con 0x7fc37009bbf0 2024-01-24T23:22:49.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc370072110 con 0x7fc3701370a0 2024-01-24T23:22:49.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:49.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 10 monclient(hunting): authenticate will time out at 1056.749146s 2024-01-24T23:22:49.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36effd640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fc37009b810 0x7fc3700712f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:48786/0 (socket says 172.21.15.132:48786) 2024-01-24T23:22:49.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36effd640 1 -- 172.21.15.132:0/985398456 learned_addr learned my addr 172.21.15.132:0/985398456 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:49.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 1 -- 172.21.15.132:0/985398456 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2004542237 0 0) 0x7fc370064700 con 0x7fc37009b810 2024-01-24T23:22:49.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:22:49.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:49.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:49.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 10 monclient(hunting): my global_id is 4683 2024-01-24T23:22:49.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 1 -- 172.21.15.132:0/985398456 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc35c003530 con 0x7fc37009b810 2024-01-24T23:22:49.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 1 -- 172.21.15.132:0/985398456 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3103250199 0 0) 0x7fc370072110 con 0x7fc3701370a0 2024-01-24T23:22:49.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:22:49.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:49.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:49.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 10 monclient(hunting): my global_id is 4714 2024-01-24T23:22:49.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 1 -- 172.21.15.132:0/985398456 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc370064700 con 0x7fc3701370a0 2024-01-24T23:22:49.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 1 -- 172.21.15.132:0/985398456 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926133679 0 0) 0x7fc370099af0 con 0x7fc37009bbf0 2024-01-24T23:22:49.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:22:49.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:49.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:49.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 10 monclient(hunting): my global_id is 4694 2024-01-24T23:22:49.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 1 -- 172.21.15.132:0/985398456 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc370072110 con 0x7fc37009bbf0 2024-01-24T23:22:49.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 1 -- 172.21.15.132:0/985398456 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2532349579 0 0) 0x7fc35c003530 con 0x7fc37009b810 2024-01-24T23:22:49.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 1 -- 172.21.15.132:0/985398456 >> v1:172.21.15.132:6792/0 conn(0x7fc3701370a0 legacy=0x7fc37013ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:49.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 1 -- 172.21.15.132:0/985398456 >> v1:172.21.15.132:6791/0 conn(0x7fc37009bbf0 legacy=0x7fc370071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:49.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 10 monclient: _finish_hunting 0 2024-01-24T23:22:49.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 1 monclient: found mon.noname-g 2024-01-24T23:22:49.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:49.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:22:49.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 1 -- 172.21.15.132:0/985398456 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37009c050 con 0x7fc37009b810 2024-01-24T23:22:49.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 10 monclient: _finish_auth 0 2024-01-24T23:22:49.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 10 monclient: _check_auth_tickets 2024-01-24T23:22:49.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:49.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 1 -- 172.21.15.132:0/985398456 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc360001a70 con 0x7fc37009b810 2024-01-24T23:22:49.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:49.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:22:49.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 10 monclient: dump: 2024-01-24T23:22:49.434 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:49.434 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:49.434 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:49.434 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:49.434 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:49.434 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:49.434 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:49.434 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:49.435 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:49.435 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:49.435 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:49.435 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:49.435 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:49.435 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:49.435 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:49.435 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:49.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 5 monclient: authenticate success, global_id 4683 2024-01-24T23:22:49.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:49.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 1 -- 172.21.15.132:0/985398456 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc360001c60 con 0x7fc37009b810 2024-01-24T23:22:49.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:49.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:49.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 1 -- 172.21.15.132:0/985398456 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc360004c30 con 0x7fc37009b810 2024-01-24T23:22:49.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:49.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 4 set_mon_vals no callback set 2024-01-24T23:22:49.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:22:49.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc36dffb640 10 monclient: dump: 2024-01-24T23:22:49.436 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:49.436 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:49.436 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:49.436 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:49.436 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:49.436 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:49.436 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:49.436 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:49.436 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:49.436 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:49.436 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:49.436 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:49.437 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:49.437 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:49.437 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:49.437 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:49.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 10 monclient: shutdown 2024-01-24T23:22:49.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:49.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 1 -- 172.21.15.132:0/985398456 >> v1:172.21.15.132:6790/0 conn(0x7fc37009b810 legacy=0x7fc3700712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:49.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 1 -- 172.21.15.132:0/985398456 shutdown_connections 2024-01-24T23:22:49.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 1 -- 172.21.15.132:0/985398456 >> 172.21.15.132:0/985398456 conn(0x7fc3700a1bc0 msgr2=0x7fc3700a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:49.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 1 -- 172.21.15.132:0/985398456 shutdown_connections 2024-01-24T23:22:49.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 1 -- 172.21.15.132:0/985398456 wait complete. 2024-01-24T23:22:49.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 10 monclient: build_initial_monmap 2024-01-24T23:22:49.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 10 monclient: monmap: 2024-01-24T23:22:49.437 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:49.438 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:49.438 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:49.438 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:49.438 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:49.438 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:49.438 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:49.438 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:49.438 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:49.438 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:49.438 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:49.438 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:49.438 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:49.438 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:49.438 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:49.438 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:49.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 1 Processor -- start 2024-01-24T23:22:49.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 1 -- start start 2024-01-24T23:22:49.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.421+0000 7fc375871640 10 monclient: init 2024-01-24T23:22:49.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc375871640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:49.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc375871640 10 monclient: _add_conns ranks=[5,7,1,4,0,3,8,6,2] 2024-01-24T23:22:49.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc375871640 10 monclient(hunting): picked mon.noname-h con 0x7fc37009b810 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:49.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc375871640 10 monclient(hunting): picked mon.noname-i con 0x7fc37009bbf0 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:49.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc375871640 10 monclient(hunting): picked mon.noname-f con 0x7fc3701370a0 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:49.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc375871640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37009e3c0 con 0x7fc3701370a0 2024-01-24T23:22:49.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc375871640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37009f140 con 0x7fc37009b810 2024-01-24T23:22:49.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc375871640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37009dd40 con 0x7fc37009bbf0 2024-01-24T23:22:49.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc375871640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:49.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc375871640 10 monclient(hunting): authenticate will time out at 1056.752320s 2024-01-24T23:22:49.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc36effd640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fc37009b810 0x7fc3701306b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:39898/0 (socket says 172.21.15.132:39898) 2024-01-24T23:22:49.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc36effd640 1 -- 172.21.15.132:0/553822650 learned_addr learned my addr 172.21.15.132:0/553822650 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:49.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3410494156 0 0) 0x7fc37009f140 con 0x7fc37009b810 2024-01-24T23:22:49.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:49.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:49.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:49.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient(hunting): my global_id is 4703 2024-01-24T23:22:49.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc350003530 con 0x7fc37009b810 2024-01-24T23:22:49.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2887359181 0 0) 0x7fc37009e3c0 con 0x7fc3701370a0 2024-01-24T23:22:49.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:49.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:49.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:49.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient(hunting): my global_id is 4744 2024-01-24T23:22:49.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc37009f140 con 0x7fc3701370a0 2024-01-24T23:22:49.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 93278002 0 0) 0x7fc37009dd40 con 0x7fc37009bbf0 2024-01-24T23:22:49.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:49.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:49.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:49.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient(hunting): my global_id is 4723 2024-01-24T23:22:49.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc37009e3c0 con 0x7fc37009bbf0 2024-01-24T23:22:49.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3041852726 0 0) 0x7fc37009e3c0 con 0x7fc37009bbf0 2024-01-24T23:22:49.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc37009dd40 con 0x7fc37009bbf0 2024-01-24T23:22:49.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3732590663 0 0) 0x7fc37009f140 con 0x7fc3701370a0 2024-01-24T23:22:49.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc37009e3c0 con 0x7fc3701370a0 2024-01-24T23:22:49.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3439782208 0 0) 0x7fc350003530 con 0x7fc37009b810 2024-01-24T23:22:49.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc37009f140 con 0x7fc37009b810 2024-01-24T23:22:49.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc358001960 con 0x7fc37009bbf0 2024-01-24T23:22:49.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:49.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:22:49.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient(hunting): dump: 2024-01-24T23:22:49.441 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:49.441 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:49.441 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:49.442 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:49.442 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:49.442 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:49.442 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:49.442 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:49.442 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:49.442 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:49.442 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:49.442 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:49.442 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:49.442 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:49.442 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:49.442 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:49.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:49.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc368001520 con 0x7fc3701370a0 2024-01-24T23:22:49.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:49.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:22:49.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient(hunting): dump: 2024-01-24T23:22:49.443 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:49.443 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:49.443 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:49.443 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:49.443 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:49.443 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:49.443 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:49.443 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:49.443 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:49.443 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:49.443 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:49.443 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:49.444 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:49.444 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:49.444 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:49.444 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:49.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc360001970 con 0x7fc37009b810 2024-01-24T23:22:49.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:49.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:22:49.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient(hunting): dump: 2024-01-24T23:22:49.444 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:49.444 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:49.444 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:49.444 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:49.444 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:49.444 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:49.444 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:49.444 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:49.445 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:49.445 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:49.445 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:49.445 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:49.445 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:49.445 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:49.445 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:49.445 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:49.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2938357759 0 0) 0x7fc37009dd40 con 0x7fc37009bbf0 2024-01-24T23:22:49.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 >> v1:172.21.15.132:6791/0 conn(0x7fc37009b810 legacy=0x7fc3701306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:49.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 >> v1:172.21.15.132:6789/0 conn(0x7fc3701370a0 legacy=0x7fc3701d3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:49.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient: _finish_hunting 0 2024-01-24T23:22:49.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 1 monclient: found mon.i 2024-01-24T23:22:49.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:49.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:49.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37012b140 con 0x7fc37009bbf0 2024-01-24T23:22:49.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient: _finish_auth 0 2024-01-24T23:22:49.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient: _check_auth_tickets 2024-01-24T23:22:49.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:49.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc375871640 5 monclient: authenticate success, global_id 4723 2024-01-24T23:22:49.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc375871640 10 monclient: _renew_subs 2024-01-24T23:22:49.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc375871640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:49.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc375871640 1 -- 172.21.15.132:0/553822650 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3700a1620 con 0x7fc37009bbf0 2024-01-24T23:22:49.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc375871640 10 monclient: _renew_subs 2024-01-24T23:22:49.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc375871640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:49.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc375871640 1 -- 172.21.15.132:0/553822650 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc37012ae60 con 0x7fc37009bbf0 2024-01-24T23:22:49.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc358003620 con 0x7fc37009bbf0 2024-01-24T23:22:49.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:49.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc358003ad0 con 0x7fc37009bbf0 2024-01-24T23:22:49.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:49.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:22:49.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc36d7fa640 4 set_mon_vals no callback set 2024-01-24T23:22:49.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 10 monclient: dump: 2024-01-24T23:22:49.447 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:49.447 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:49.447 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:49.447 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:49.447 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:49.447 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:49.447 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:49.447 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:49.447 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:49.447 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:49.447 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:49.448 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc375871640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc375871640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc375871640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fc35800e550 con 0x7fc37009bbf0 2024-01-24T23:22:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc375871640 1 -- 172.21.15.132:0/553822650 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc354005090 con 0x7fc37009bbf0 2024-01-24T23:22:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(33..33 src has 1..33) v4 ==== 4634+0+0 (unknown 3262973452 0 0) 0x7fc358042650 con 0x7fc37009bbf0 2024-01-24T23:22:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.425+0000 7fc36effd640 10 monclient: get_auth_request con 0x7fc350037c20 auth_method 0 2024-01-24T23:22:49.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.429+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fc358013370 con 0x7fc37009bbf0 2024-01-24T23:22:49.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.429+0000 7fc367fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:49.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.429+0000 7fc367fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:49.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.581+0000 7fc375871640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}] 2024-01-24T23:22:49.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.581+0000 7fc375871640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}] 2024-01-24T23:22:49.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.581+0000 7fc375871640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:49.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.581+0000 7fc375871640 1 -- 172.21.15.132:0/553822650 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"} v 0) v1 -- 0x7fc3540058c0 con 0x7fc37009bbf0 2024-01-24T23:22:49.917 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.909+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}]=0 set pool 4 size to 10 v34) v1 ==== 124+0+0 (unknown 1976903813 0 0) 0x7fc358013190 con 0x7fc37009bbf0 2024-01-24T23:22:49.917 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.909+0000 7fc367fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}] 2024-01-24T23:22:49.917 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:49.909+0000 7fc367fff640 10 monclient: _finish_command 2 = system:0 set pool 4 size to 10 2024-01-24T23:22:50.093 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:50.089+0000 7fc375871640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}] 2024-01-24T23:22:50.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:50.089+0000 7fc375871640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}] 2024-01-24T23:22:50.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:50.089+0000 7fc375871640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:50.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:50.089+0000 7fc375871640 1 -- 172.21.15.132:0/553822650 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"} v 0) v1 -- 0x7fc354000bc0 con 0x7fc37009bbf0 2024-01-24T23:22:50.901 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:50.897+0000 7fc367fff640 1 -- 172.21.15.132:0/553822650 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}]=0 set pool 4 size to 10 v35) v1 ==== 124+0+0 (unknown 29507002 0 0) 0x7fc358013ed0 con 0x7fc37009bbf0 2024-01-24T23:22:50.902 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:50.897+0000 7fc367fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}] 2024-01-24T23:22:50.902 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:50.897+0000 7fc367fff640 10 monclient: _finish_command 3 = system:0 set pool 4 size to 10 2024-01-24T23:22:50.907 INFO:tasks.workunit.client.0.smithi132.stderr:set pool 4 size to 10 2024-01-24T23:22:50.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:50.917+0000 7fc375871640 1 -- 172.21.15.132:0/553822650 >> v1:172.21.15.132:6812/1835401922 conn(0x7fc350037c20 legacy=0x7fc35003a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:50.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:50.917+0000 7fc375871640 10 monclient: shutdown 2024-01-24T23:22:50.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:50.917+0000 7fc375871640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:50.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:50.917+0000 7fc375871640 1 -- 172.21.15.132:0/553822650 >> v1:172.21.15.132:6792/0 conn(0x7fc37009bbf0 legacy=0x7fc3701d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:50.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:50.917+0000 7fc375871640 1 -- 172.21.15.132:0/553822650 shutdown_connections 2024-01-24T23:22:50.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:50.917+0000 7fc375871640 1 -- 172.21.15.132:0/553822650 >> 172.21.15.132:0/553822650 conn(0x7fc3700a1bc0 msgr2=0x7fc3701beb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:50.920 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:50.917+0000 7fc375871640 1 -- 172.21.15.132:0/553822650 shutdown_connections 2024-01-24T23:22:50.920 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:50.917+0000 7fc375871640 1 -- 172.21.15.132:0/553822650 wait complete. 2024-01-24T23:22:50.959 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_config_value foo min_size 5 2024-01-24T23:22:50.959 INFO:tasks.workunit.client.0.smithi132.stderr:+ local pool_name config_opt expected_val val 2024-01-24T23:22:50.959 INFO:tasks.workunit.client.0.smithi132.stderr:+ pool_name=foo 2024-01-24T23:22:50.959 INFO:tasks.workunit.client.0.smithi132.stderr:+ config_opt=min_size 2024-01-24T23:22:50.959 INFO:tasks.workunit.client.0.smithi132.stderr:+ expected_val=5 2024-01-24T23:22:50.961 INFO:tasks.workunit.client.0.smithi132.stderr:++ get_config_value_or_die foo min_size 2024-01-24T23:22:50.961 INFO:tasks.workunit.client.0.smithi132.stderr:++ local pool_name config_opt raw val 2024-01-24T23:22:50.961 INFO:tasks.workunit.client.0.smithi132.stderr:++ pool_name=foo 2024-01-24T23:22:50.962 INFO:tasks.workunit.client.0.smithi132.stderr:++ config_opt=min_size 2024-01-24T23:22:50.962 INFO:tasks.workunit.client.0.smithi132.stderr:+++ ceph osd pool get foo min_size 2024-01-24T23:22:51.390 INFO:tasks.workunit.client.0.smithi132.stderr:++ raw='min_size: 5' 2024-01-24T23:22:51.390 INFO:tasks.workunit.client.0.smithi132.stderr:++ [[ 0 -ne 0 ]] 2024-01-24T23:22:51.390 INFO:tasks.workunit.client.0.smithi132.stderr:+++ echo min_size: 5 2024-01-24T23:22:51.390 INFO:tasks.workunit.client.0.smithi132.stderr:+++ sed -e 's/[{} "]//g' 2024-01-24T23:22:51.392 INFO:tasks.workunit.client.0.smithi132.stderr:++ raw=min_size:5 2024-01-24T23:22:51.392 INFO:tasks.workunit.client.0.smithi132.stderr:+++ echo min_size:5 2024-01-24T23:22:51.392 INFO:tasks.workunit.client.0.smithi132.stderr:+++ cut -f2 -d: 2024-01-24T23:22:51.394 INFO:tasks.workunit.client.0.smithi132.stderr:++ val=5 2024-01-24T23:22:51.394 INFO:tasks.workunit.client.0.smithi132.stderr:++ echo 5 2024-01-24T23:22:51.394 INFO:tasks.workunit.client.0.smithi132.stderr:++ return 0 2024-01-24T23:22:51.394 INFO:tasks.workunit.client.0.smithi132.stderr:+ val=5 2024-01-24T23:22:51.394 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ 5 != \5 ]] 2024-01-24T23:22:51.394 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd pool set foo size 0 2024-01-24T23:22:51.394 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:22:51.394 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool set foo size 0 2024-01-24T23:22:51.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.509+0000 7f574c1ab640 10 monclient: get_monmap_and_config 2024-01-24T23:22:51.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.509+0000 7f574c1ab640 10 monclient: build_initial_monmap 2024-01-24T23:22:51.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.509+0000 7f574c1ab640 10 monclient: monmap: 2024-01-24T23:22:51.513 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:51.513 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:51.513 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:51.514449+0000 2024-01-24T23:22:51.513 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:51.514449+0000 2024-01-24T23:22:51.513 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:51.513 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:51.513 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:51.513 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:51.513 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:51.513 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:51.514 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:51.514 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:51.514 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:51.514 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:51.514 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:51.514 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:51.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.509+0000 7f574c1ab640 1 Processor -- start 2024-01-24T23:22:51.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.509+0000 7f574c1ab640 1 -- start start 2024-01-24T23:22:51.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.509+0000 7f574c1ab640 10 monclient: init 2024-01-24T23:22:51.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.509+0000 7f574c1ab640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:51.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.509+0000 7f574c1ab640 10 monclient: _add_conns ranks=[0,5,8,1,7,4,2,3,6] 2024-01-24T23:22:51.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.509+0000 7f574c1ab640 10 monclient(hunting): picked mon.noname-a con 0x7f5744099130 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:51.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.509+0000 7f574c1ab640 10 monclient(hunting): picked mon.noname-h con 0x7f574409b020 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:51.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.509+0000 7f574c1ab640 10 monclient(hunting): picked mon.noname-e con 0x7f574409aba0 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:51.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.509+0000 7f574c1ab640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5744064700 con 0x7f5744099130 2024-01-24T23:22:51.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.509+0000 7f574c1ab640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574409bd10 con 0x7f574409b020 2024-01-24T23:22:51.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.509+0000 7f574c1ab640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574409a2d0 con 0x7f574409aba0 2024-01-24T23:22:51.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.509+0000 7f574c1ab640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:51.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.509+0000 7f574c1ab640 10 monclient(hunting): authenticate will time out at 1058.839944s 2024-01-24T23:22:51.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574a748640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f574409aba0 legacy=0x7f574413bd90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:51.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5749746640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f574409b020 0x7f5744138660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:39922/0 (socket says 172.21.15.132:39922) 2024-01-24T23:22:51.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5749746640 1 -- 172.21.15.132:0/604420959 learned_addr learned my addr 172.21.15.132:0/604420959 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:51.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 1 -- 172.21.15.132:0/604420959 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2148125497 0 0) 0x7f574409bd10 con 0x7f574409b020 2024-01-24T23:22:51.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 10 monclient(hunting): _negotiate 2024-01-24T23:22:51.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:51.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:51.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 10 monclient(hunting): my global_id is 4721 2024-01-24T23:22:51.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 1 -- 172.21.15.132:0/604420959 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5730003530 con 0x7f574409b020 2024-01-24T23:22:51.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 1 -- 172.21.15.132:0/604420959 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2254836055 0 0) 0x7f5730003530 con 0x7f574409b020 2024-01-24T23:22:51.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 1 -- 172.21.15.132:0/604420959 >> v1:172.21.15.115:6793/0 conn(0x7f574409aba0 legacy=0x7f574413bd90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:51.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 1 -- 172.21.15.132:0/604420959 >> v1:172.21.15.115:6789/0 conn(0x7f5744099130 legacy=0x7f5744099510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:51.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 10 monclient: _finish_hunting 0 2024-01-24T23:22:51.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 1 monclient: found mon.noname-h 2024-01-24T23:22:51.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:51.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:22:51.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 1 -- 172.21.15.132:0/604420959 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574409ba30 con 0x7f574409b020 2024-01-24T23:22:51.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 10 monclient: _finish_auth 0 2024-01-24T23:22:51.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 10 monclient: _check_auth_tickets 2024-01-24T23:22:51.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:51.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 1 -- 172.21.15.132:0/604420959 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f57340017c0 con 0x7f574409b020 2024-01-24T23:22:51.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 5 monclient: authenticate success, global_id 4721 2024-01-24T23:22:51.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:51.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:51.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:22:51.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 10 monclient: dump: 2024-01-24T23:22:51.523 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:51.523 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:51.524 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:51.524 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:51.524 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:51.524 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:51.524 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:51.524 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:51.524 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:51.524 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:51.524 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:51.524 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:51.524 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:51.524 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:51.524 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:51.525 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:51.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:51.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 1 -- 172.21.15.132:0/604420959 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5734001a70 con 0x7f574409b020 2024-01-24T23:22:51.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:51.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:51.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 4 set_mon_vals no callback set 2024-01-24T23:22:51.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 1 -- 172.21.15.132:0/604420959 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5734004bf0 con 0x7f574409b020 2024-01-24T23:22:51.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:51.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:22:51.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5748f45640 10 monclient: dump: 2024-01-24T23:22:51.525 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:51.525 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:51.525 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:51.525 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:51.526 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:51.526 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:51.526 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:51.526 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:51.526 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:51.526 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:51.526 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:51.526 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:51.526 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:51.526 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:51.526 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:51.526 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:51.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 10 monclient: shutdown 2024-01-24T23:22:51.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:51.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 1 -- 172.21.15.132:0/604420959 >> v1:172.21.15.132:6791/0 conn(0x7f574409b020 legacy=0x7f5744138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:51.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 1 -- 172.21.15.132:0/604420959 shutdown_connections 2024-01-24T23:22:51.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 1 -- 172.21.15.132:0/604420959 >> 172.21.15.132:0/604420959 conn(0x7f57440a1bc0 msgr2=0x7f57440a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:51.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 1 -- 172.21.15.132:0/604420959 shutdown_connections 2024-01-24T23:22:51.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 1 -- 172.21.15.132:0/604420959 wait complete. 2024-01-24T23:22:51.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 10 monclient: build_initial_monmap 2024-01-24T23:22:51.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 10 monclient: monmap: 2024-01-24T23:22:51.527 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:51.527 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:51.527 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:51.527 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:51.527 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:51.527 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:51.527 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:51.527 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:51.527 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:51.527 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:51.528 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:51.528 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:51.528 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:51.528 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:51.528 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:51.528 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:51.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 1 Processor -- start 2024-01-24T23:22:51.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 1 -- start start 2024-01-24T23:22:51.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 10 monclient: init 2024-01-24T23:22:51.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:51.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 10 monclient: _add_conns ranks=[7,4,0,2,5,3,8,6,1] 2024-01-24T23:22:51.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 10 monclient(hunting): picked mon.noname-i con 0x7f5744099130 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:51.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 10 monclient(hunting): picked mon.noname-c con 0x7f574409aba0 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:51.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 10 monclient(hunting): picked mon.noname-a con 0x7f574409b020 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:51.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574413c0b0 con 0x7f574409b020 2024-01-24T23:22:51.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574413e8d0 con 0x7f574409aba0 2024-01-24T23:22:51.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574413ebb0 con 0x7f5744099130 2024-01-24T23:22:51.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:51.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f574c1ab640 10 monclient(hunting): authenticate will time out at 1058.843119s 2024-01-24T23:22:51.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5749f47640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f5744099130 0x7f57440982b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:39844/0 (socket says 172.21.15.132:39844) 2024-01-24T23:22:51.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f5749f47640 1 -- 172.21.15.132:0/1955440085 learned_addr learned my addr 172.21.15.132:0/1955440085 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:51.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f573a7fc640 1 -- 172.21.15.132:0/1955440085 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611018830 0 0) 0x7f574413ebb0 con 0x7f5744099130 2024-01-24T23:22:51.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f573a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:51.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f573a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:51.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f573a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:51.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f573a7fc640 10 monclient(hunting): my global_id is 4741 2024-01-24T23:22:51.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f573a7fc640 1 -- 172.21.15.132:0/1955440085 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5728003530 con 0x7f5744099130 2024-01-24T23:22:51.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f573a7fc640 1 -- 172.21.15.132:0/1955440085 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3663560296 0 0) 0x7f5728003530 con 0x7f5744099130 2024-01-24T23:22:51.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f573a7fc640 1 -- 172.21.15.132:0/1955440085 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f574413ebb0 con 0x7f5744099130 2024-01-24T23:22:51.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f573a7fc640 1 -- 172.21.15.132:0/1955440085 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5740001ae0 con 0x7f5744099130 2024-01-24T23:22:51.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f573a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:51.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f573a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:22:51.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f573a7fc640 10 monclient(hunting): dump: 2024-01-24T23:22:51.530 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:51.530 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:51.530 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:51.530 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:51.530 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:51.530 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:51.530 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:51.530 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:51.530 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:51.530 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:51.530 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:51.531 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:51.531 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:51.531 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:51.531 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:51.531 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:51.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f573a7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:51.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f573a7fc640 1 -- 172.21.15.132:0/1955440085 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1217812489 0 0) 0x7f574413e8d0 con 0x7f574409aba0 2024-01-24T23:22:51.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f573a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:51.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f573a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:51.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f573a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:51.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f573a7fc640 10 monclient(hunting): my global_id is 74758 2024-01-24T23:22:51.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f573a7fc640 1 -- 172.21.15.132:0/1955440085 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5728003530 con 0x7f574409aba0 2024-01-24T23:22:51.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f573a7fc640 1 -- 172.21.15.132:0/1955440085 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 692037978 0 0) 0x7f574413c0b0 con 0x7f574409b020 2024-01-24T23:22:51.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f573a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:51.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f573a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:51.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f573a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:51.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.513+0000 7f573a7fc640 10 monclient(hunting): my global_id is 4734 2024-01-24T23:22:51.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f573a7fc640 1 -- 172.21.15.132:0/1955440085 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f574413e8d0 con 0x7f574409b020 2024-01-24T23:22:51.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f573a7fc640 1 -- 172.21.15.132:0/1955440085 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3251522685 0 0) 0x7f574413ebb0 con 0x7f5744099130 2024-01-24T23:22:51.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f573a7fc640 1 -- 172.21.15.132:0/1955440085 >> v1:172.21.15.115:6791/0 conn(0x7f574409aba0 legacy=0x7f57441d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:51.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f573a7fc640 1 -- 172.21.15.132:0/1955440085 >> v1:172.21.15.115:6789/0 conn(0x7f574409b020 legacy=0x7f57441d40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:51.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f573a7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:22:51.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f573a7fc640 1 monclient: found mon.i 2024-01-24T23:22:51.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f573a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:51.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f573a7fc640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:51.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f573a7fc640 1 -- 172.21.15.132:0/1955440085 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574413d970 con 0x7f5744099130 2024-01-24T23:22:51.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f573a7fc640 10 monclient: _finish_auth 0 2024-01-24T23:22:51.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f573a7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:22:51.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f573a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:51.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f574c1ab640 5 monclient: authenticate success, global_id 4741 2024-01-24T23:22:51.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f574c1ab640 10 monclient: _renew_subs 2024-01-24T23:22:51.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f574c1ab640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:51.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f574c1ab640 1 -- 172.21.15.132:0/1955440085 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574413c4f0 con 0x7f5744099130 2024-01-24T23:22:51.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f574c1ab640 10 monclient: _renew_subs 2024-01-24T23:22:51.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f574c1ab640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:51.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f574c1ab640 1 -- 172.21.15.132:0/1955440085 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f574413b620 con 0x7f5744099130 2024-01-24T23:22:51.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f573a7fc640 1 -- 172.21.15.132:0/1955440085 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5740003950 con 0x7f5744099130 2024-01-24T23:22:51.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f573a7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:51.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f573b7fe640 4 set_mon_vals no callback set 2024-01-24T23:22:51.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f573a7fc640 1 -- 172.21.15.132:0/1955440085 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5740003e80 con 0x7f5744099130 2024-01-24T23:22:51.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f573a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:51.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f573a7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:22:51.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f573a7fc640 10 monclient: dump: 2024-01-24T23:22:51.533 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:51.533 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:51.533 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:51.533 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:51.534 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:51.534 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:51.534 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:51.534 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:51.534 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:51.534 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:51.534 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:51.534 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:51.534 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:51.534 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:51.534 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:51.534 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:51.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f573a7fc640 1 -- 172.21.15.132:0/1955440085 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f5740004060 con 0x7f5744099130 2024-01-24T23:22:51.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f574c1ab640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:51.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f574c1ab640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:51.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f574c1ab640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:51.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f574c1ab640 1 -- 172.21.15.132:0/1955440085 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f572c005090 con 0x7f5744099130 2024-01-24T23:22:51.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f573a7fc640 1 -- 172.21.15.132:0/1955440085 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(35..35 src has 1..35) v4 ==== 4634+0+0 (unknown 3048104250 0 0) 0x7f5740043cc0 con 0x7f5744099130 2024-01-24T23:22:51.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.517+0000 7f5749746640 10 monclient: get_auth_request con 0x7f5728037140 auth_method 0 2024-01-24T23:22:51.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.521+0000 7f573a7fc640 1 -- 172.21.15.132:0/1955440085 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f5740014bd0 con 0x7f5744099130 2024-01-24T23:22:51.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.521+0000 7f573a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:51.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.521+0000 7f573a7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:51.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.677+0000 7f574c1ab640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "0"}] 2024-01-24T23:22:51.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.677+0000 7f574c1ab640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "0"}] 2024-01-24T23:22:51.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.677+0000 7f574c1ab640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:51.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.677+0000 7f574c1ab640 1 -- 172.21.15.132:0/1955440085 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "0"} v 0) v1 -- 0x7f572c0058c0 con 0x7f5744099130 2024-01-24T23:22:51.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.677+0000 7f573a7fc640 1 -- 172.21.15.132:0/1955440085 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "0"}]=-22 pool size must be between 1 and 10 v35) v1 ==== 136+0+0 (unknown 1184131494 0 0) 0x7f57400147c0 con 0x7f5744099130 2024-01-24T23:22:51.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.677+0000 7f573a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "0"}] 2024-01-24T23:22:51.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.677+0000 7f573a7fc640 10 monclient: _finish_command 2 = mon:22 pool size must be between 1 and 10 2024-01-24T23:22:51.681 INFO:tasks.workunit.client.0.smithi132.stderr:Error EINVAL: pool size must be between 1 and 10 2024-01-24T23:22:51.687 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:22:51.687 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd pool set foo size 20 2024-01-24T23:22:51.687 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:22:51.687 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool set foo size 20 2024-01-24T23:22:51.808 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95dac95640 10 monclient: get_monmap_and_config 2024-01-24T23:22:51.808 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95dac95640 10 monclient: build_initial_monmap 2024-01-24T23:22:51.809 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95dac95640 10 monclient: monmap: 2024-01-24T23:22:51.809 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:51.809 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:51.809 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:51.810012+0000 2024-01-24T23:22:51.809 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:51.810012+0000 2024-01-24T23:22:51.809 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:51.809 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:51.809 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:51.809 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:51.809 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:51.809 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:51.809 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:51.809 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:51.809 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:51.809 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:51.810 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:51.810 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95dac95640 1 Processor -- start 2024-01-24T23:22:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95dac95640 1 -- start start 2024-01-24T23:22:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95dac95640 10 monclient: init 2024-01-24T23:22:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95dac95640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95dac95640 10 monclient: _add_conns ranks=[1,2,8,6,0,7,5,4,3] 2024-01-24T23:22:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95dac95640 10 monclient(hunting): picked mon.noname-f con 0x7f95d4072690 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95dac95640 10 monclient(hunting): picked mon.noname-b con 0x7f95d409ae90 addr v1:172.21.15.115:6790/0 2024-01-24T23:22:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95dac95640 10 monclient(hunting): picked mon.noname-e con 0x7f95d409bdc0 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95dac95640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d4064700 con 0x7f95d4072690 2024-01-24T23:22:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95dac95640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d409a790 con 0x7f95d409ae90 2024-01-24T23:22:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95dac95640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d40a18f0 con 0x7f95d409bdc0 2024-01-24T23:22:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95dac95640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:51.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95dac95640 10 monclient(hunting): authenticate will time out at 1059.135455s 2024-01-24T23:22:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95d8a31640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f95d4072690 0x7f95d40a0ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:38014/0 (socket says 172.21.15.132:38014) 2024-01-24T23:22:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95d8a31640 1 -- 172.21.15.132:0/1350217428 learned_addr learned my addr 172.21.15.132:0/1350217428 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95d3fff640 1 -- 172.21.15.132:0/1350217428 >> v1:172.21.15.115:6790/0 conn(0x7f95d409ae90 legacy=0x7f95d40a11e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:22:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95d9232640 1 -- 172.21.15.132:0/1350217428 >> v1:172.21.15.115:6793/0 conn(0x7f95d409bdc0 legacy=0x7f95d413d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95d37fe640 1 -- 172.21.15.132:0/1350217428 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655522266 0 0) 0x7f95d4064700 con 0x7f95d4072690 2024-01-24T23:22:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95d37fe640 10 monclient(hunting): _negotiate 2024-01-24T23:22:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95d37fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95d37fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95d37fe640 10 monclient(hunting): my global_id is 4771 2024-01-24T23:22:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.805+0000 7f95d37fe640 1 -- 172.21.15.132:0/1350217428 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95c4003530 con 0x7f95d4072690 2024-01-24T23:22:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d37fe640 1 -- 172.21.15.132:0/1350217428 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1179996462 0 0) 0x7f95c4003530 con 0x7f95d4072690 2024-01-24T23:22:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d37fe640 1 -- 172.21.15.132:0/1350217428 >> v1:172.21.15.115:6793/0 conn(0x7f95d409bdc0 legacy=0x7f95d413d280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:51.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d37fe640 1 -- 172.21.15.132:0/1350217428 >> v1:172.21.15.115:6790/0 conn(0x7f95d409ae90 legacy=0x7f95d40a11e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d37fe640 10 monclient: _finish_hunting 0 2024-01-24T23:22:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d37fe640 1 monclient: found mon.noname-f 2024-01-24T23:22:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d37fe640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:22:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d37fe640 1 -- 172.21.15.132:0/1350217428 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d40997a0 con 0x7f95d4072690 2024-01-24T23:22:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d37fe640 10 monclient: _finish_auth 0 2024-01-24T23:22:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d37fe640 10 monclient: _check_auth_tickets 2024-01-24T23:22:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d37fe640 1 -- 172.21.15.132:0/1350217428 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f95c80017c0 con 0x7f95d4072690 2024-01-24T23:22:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 5 monclient: authenticate success, global_id 4771 2024-01-24T23:22:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d37fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:22:51.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d37fe640 10 monclient: dump: 2024-01-24T23:22:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:51.820 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:51.821 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d37fe640 1 -- 172.21.15.132:0/1350217428 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f95c8001a70 con 0x7f95d4072690 2024-01-24T23:22:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d37fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d37fe640 1 -- 172.21.15.132:0/1350217428 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f95c8004bf0 con 0x7f95d4072690 2024-01-24T23:22:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d37fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:22:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d37fe640 10 monclient: dump: 2024-01-24T23:22:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:51.821 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:51.822 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:51.822 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 4 set_mon_vals no callback set 2024-01-24T23:22:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 10 monclient: shutdown 2024-01-24T23:22:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 1 -- 172.21.15.132:0/1350217428 >> v1:172.21.15.132:6789/0 conn(0x7f95d4072690 legacy=0x7f95d40a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 1 -- 172.21.15.132:0/1350217428 shutdown_connections 2024-01-24T23:22:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 1 -- 172.21.15.132:0/1350217428 >> 172.21.15.132:0/1350217428 conn(0x7f95d40a1bc0 msgr2=0x7f95d40a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 1 -- 172.21.15.132:0/1350217428 shutdown_connections 2024-01-24T23:22:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 1 -- 172.21.15.132:0/1350217428 wait complete. 2024-01-24T23:22:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 10 monclient: build_initial_monmap 2024-01-24T23:22:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 10 monclient: monmap: 2024-01-24T23:22:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:51.823 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:51.824 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:51.824 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 1 Processor -- start 2024-01-24T23:22:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 1 -- start start 2024-01-24T23:22:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 10 monclient: init 2024-01-24T23:22:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 10 monclient: _add_conns ranks=[2,7,3,6,8,0,5,4,1] 2024-01-24T23:22:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 10 monclient(hunting): picked mon.noname-b con 0x7f95d4072690 addr v1:172.21.15.115:6790/0 2024-01-24T23:22:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 10 monclient(hunting): picked mon.noname-i con 0x7f95d409ae90 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 10 monclient(hunting): picked mon.noname-g con 0x7f95d409bdc0 addr v1:172.21.15.132:6790/0 2024-01-24T23:22:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d409e000 con 0x7f95d4072690 2024-01-24T23:22:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d4099470 con 0x7f95d409bdc0 2024-01-24T23:22:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d409dad0 con 0x7f95d409ae90 2024-01-24T23:22:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 10 monclient(hunting): authenticate will time out at 1059.138349s 2024-01-24T23:22:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d3fff640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f95d409ae90 0x7f95d4070720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:39848/0 (socket says 172.21.15.132:39848) 2024-01-24T23:22:51.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d3fff640 1 -- 172.21.15.132:0/919234299 learned_addr learned my addr 172.21.15.132:0/919234299 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d8a31640 1 -- 172.21.15.132:0/919234299 >> v1:172.21.15.115:6790/0 conn(0x7f95d4072690 legacy=0x7f95d4070010 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:22:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 1 -- 172.21.15.132:0/919234299 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3483324183 0 0) 0x7f95d409dad0 con 0x7f95d409ae90 2024-01-24T23:22:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 10 monclient(hunting): _negotiate 2024-01-24T23:22:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 10 monclient(hunting): my global_id is 4750 2024-01-24T23:22:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 1 -- 172.21.15.132:0/919234299 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95b8003530 con 0x7f95d409ae90 2024-01-24T23:22:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 1 -- 172.21.15.132:0/919234299 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1779291158 0 0) 0x7f95d4099470 con 0x7f95d409bdc0 2024-01-24T23:22:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 10 monclient(hunting): _negotiate 2024-01-24T23:22:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 10 monclient(hunting): my global_id is 4692 2024-01-24T23:22:51.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 1 -- 172.21.15.132:0/919234299 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95d409dad0 con 0x7f95d409bdc0 2024-01-24T23:22:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 1 -- 172.21.15.132:0/919234299 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1525425022 0 0) 0x7f95b8003530 con 0x7f95d409ae90 2024-01-24T23:22:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 1 -- 172.21.15.132:0/919234299 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f95d4099470 con 0x7f95d409ae90 2024-01-24T23:22:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 1 -- 172.21.15.132:0/919234299 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f95c0004330 con 0x7f95d409ae90 2024-01-24T23:22:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:22:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 10 monclient(hunting): dump: 2024-01-24T23:22:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:51.827 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:51.828 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 1 -- 172.21.15.132:0/919234299 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1052437447 0 0) 0x7f95d409dad0 con 0x7f95d409bdc0 2024-01-24T23:22:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 1 -- 172.21.15.132:0/919234299 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f95b8003530 con 0x7f95d409bdc0 2024-01-24T23:22:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 1 -- 172.21.15.132:0/919234299 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f95cc004530 con 0x7f95d409bdc0 2024-01-24T23:22:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:22:51.828 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 10 monclient(hunting): dump: 2024-01-24T23:22:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:51.829 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:51.830 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 1 -- 172.21.15.132:0/919234299 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 470913754 0 0) 0x7f95d4099470 con 0x7f95d409ae90 2024-01-24T23:22:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 1 -- 172.21.15.132:0/919234299 >> v1:172.21.15.132:6790/0 conn(0x7f95d409bdc0 legacy=0x7f95d41d3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 1 -- 172.21.15.132:0/919234299 >> v1:172.21.15.115:6790/0 conn(0x7f95d4072690 legacy=0x7f95d4070010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 10 monclient: _finish_hunting 0 2024-01-24T23:22:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 1 monclient: found mon.i 2024-01-24T23:22:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 1 -- 172.21.15.132:0/919234299 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d409c9e0 con 0x7f95d409ae90 2024-01-24T23:22:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 10 monclient: _finish_auth 0 2024-01-24T23:22:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 10 monclient: _check_auth_tickets 2024-01-24T23:22:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 5 monclient: authenticate success, global_id 4750 2024-01-24T23:22:51.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 10 monclient: _renew_subs 2024-01-24T23:22:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 1 -- 172.21.15.132:0/919234299 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d41b33d0 con 0x7f95d409ae90 2024-01-24T23:22:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 10 monclient: _renew_subs 2024-01-24T23:22:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95dac95640 1 -- 172.21.15.132:0/919234299 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d409cc20 con 0x7f95d409ae90 2024-01-24T23:22:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 1 -- 172.21.15.132:0/919234299 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f95c00045a0 con 0x7f95d409ae90 2024-01-24T23:22:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 1 -- 172.21.15.132:0/919234299 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f95c0004ad0 con 0x7f95d409ae90 2024-01-24T23:22:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d2ffd640 4 set_mon_vals no callback set 2024-01-24T23:22:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:22:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.809+0000 7f95d17fa640 10 monclient: dump: 2024-01-24T23:22:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:51.831 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:51.832 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:51.832 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:51.832 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:51.832 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:51.832 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:51.832 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:51.832 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:51.832 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:51.832 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:51.832 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:51.832 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:51.832 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:51.832 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:51.832 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:51.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.813+0000 7f95dac95640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:51.833 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.813+0000 7f95dac95640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:51.833 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.813+0000 7f95dac95640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:51.833 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.813+0000 7f95dac95640 1 -- 172.21.15.132:0/919234299 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b4005090 con 0x7f95d409ae90 2024-01-24T23:22:51.833 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.813+0000 7f95d17fa640 1 -- 172.21.15.132:0/919234299 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f95c0004c30 con 0x7f95d409ae90 2024-01-24T23:22:51.833 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.813+0000 7f95d17fa640 1 -- 172.21.15.132:0/919234299 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(35..35 src has 1..35) v4 ==== 4634+0+0 (unknown 3048104250 0 0) 0x7f95c0043ba0 con 0x7f95d409ae90 2024-01-24T23:22:51.833 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.813+0000 7f95d8a31640 10 monclient: get_auth_request con 0x7f95b8037670 auth_method 0 2024-01-24T23:22:51.833 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.817+0000 7f95d17fa640 1 -- 172.21.15.132:0/919234299 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f95c0014a50 con 0x7f95d409ae90 2024-01-24T23:22:51.834 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.817+0000 7f95d17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:51.834 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.817+0000 7f95d17fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:51.993 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.989+0000 7f95dac95640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "20"}] 2024-01-24T23:22:51.994 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.989+0000 7f95dac95640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "20"}] 2024-01-24T23:22:51.994 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.989+0000 7f95dac95640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:51.994 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.989+0000 7f95dac95640 1 -- 172.21.15.132:0/919234299 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "20"} v 0) v1 -- 0x7f95b40058c0 con 0x7f95d409ae90 2024-01-24T23:22:51.995 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.993+0000 7f95d17fa640 1 -- 172.21.15.132:0/919234299 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "20"}]=-22 pool size must be between 1 and 10 v35) v1 ==== 137+0+0 (unknown 1496131530 0 0) 0x7f95c001cfb0 con 0x7f95d409ae90 2024-01-24T23:22:51.995 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.993+0000 7f95d17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "20"}] 2024-01-24T23:22:51.995 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:51.993+0000 7f95d17fa640 10 monclient: _finish_command 2 = mon:22 pool size must be between 1 and 10 2024-01-24T23:22:51.995 INFO:tasks.workunit.client.0.smithi132.stderr:Error EINVAL: pool size must be between 1 and 10 2024-01-24T23:22:51.999 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:22:51.999 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd pool delete foo 2024-01-24T23:22:51.999 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:22:51.999 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool delete foo 2024-01-24T23:22:52.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc289c91640 10 monclient: get_monmap_and_config 2024-01-24T23:22:52.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc289c91640 10 monclient: build_initial_monmap 2024-01-24T23:22:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc289c91640 10 monclient: monmap: 2024-01-24T23:22:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:52.118345+0000 2024-01-24T23:22:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:52.118345+0000 2024-01-24T23:22:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:52.118 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:52.118 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:52.118 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:52.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc289c91640 1 Processor -- start 2024-01-24T23:22:52.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc289c91640 1 -- start start 2024-01-24T23:22:52.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc289c91640 10 monclient: init 2024-01-24T23:22:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc289c91640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc289c91640 10 monclient: _add_conns ranks=[6,2,3,0,1,4,7,5,8] 2024-01-24T23:22:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc289c91640 10 monclient(hunting): picked mon.noname-d con 0x7fc28409ff20 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc289c91640 10 monclient(hunting): picked mon.noname-b con 0x7fc28409f9c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:22:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc289c91640 10 monclient(hunting): picked mon.noname-g con 0x7fc28413b2c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:22:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc289c91640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc284064700 con 0x7fc28409f9c0 2024-01-24T23:22:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc289c91640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28409e760 con 0x7fc28413b2c0 2024-01-24T23:22:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc289c91640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2840a1160 con 0x7fc28409ff20 2024-01-24T23:22:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc289c91640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc289c91640 10 monclient(hunting): authenticate will time out at 1059.443259s 2024-01-24T23:22:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc282ffd640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fc28409f9c0 legacy=0x7fc284139ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:22:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc283fff640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fc28413b2c0 0x7fc28413d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:48804/0 (socket says 172.21.15.132:48804) 2024-01-24T23:22:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc283fff640 1 -- 172.21.15.132:0/4089971596 learned_addr learned my addr 172.21.15.132:0/4089971596 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc2827fc640 1 -- 172.21.15.132:0/4089971596 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 139973561 0 0) 0x7fc28409e760 con 0x7fc28413b2c0 2024-01-24T23:22:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc2827fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc2827fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc2827fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc2827fc640 10 monclient(hunting): my global_id is 4701 2024-01-24T23:22:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.113+0000 7fc2827fc640 1 -- 172.21.15.132:0/4089971596 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc26c003530 con 0x7fc28413b2c0 2024-01-24T23:22:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 1 -- 172.21.15.132:0/4089971596 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 803431143 0 0) 0x7fc2840a1160 con 0x7fc28409ff20 2024-01-24T23:22:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 10 monclient(hunting): my global_id is 4767 2024-01-24T23:22:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 1 -- 172.21.15.132:0/4089971596 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc28409e760 con 0x7fc28409ff20 2024-01-24T23:22:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 1 -- 172.21.15.132:0/4089971596 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1808650769 0 0) 0x7fc26c003530 con 0x7fc28413b2c0 2024-01-24T23:22:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 1 -- 172.21.15.132:0/4089971596 >> v1:172.21.15.115:6792/0 conn(0x7fc28409ff20 legacy=0x7fc2840a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 1 -- 172.21.15.132:0/4089971596 >> v1:172.21.15.115:6790/0 conn(0x7fc28409f9c0 legacy=0x7fc284139ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 10 monclient: _finish_hunting 0 2024-01-24T23:22:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 1 monclient: found mon.noname-g 2024-01-24T23:22:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:22:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 1 -- 172.21.15.132:0/4089971596 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2840a0a10 con 0x7fc28413b2c0 2024-01-24T23:22:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 10 monclient: _finish_auth 0 2024-01-24T23:22:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 10 monclient: _check_auth_tickets 2024-01-24T23:22:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 1 -- 172.21.15.132:0/4089971596 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc27c001b40 con 0x7fc28413b2c0 2024-01-24T23:22:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 5 monclient: authenticate success, global_id 4701 2024-01-24T23:22:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:22:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 10 monclient: dump: 2024-01-24T23:22:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:52.128 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 1 -- 172.21.15.132:0/4089971596 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc27c001e20 con 0x7fc28413b2c0 2024-01-24T23:22:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 4 set_mon_vals no callback set 2024-01-24T23:22:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 1 -- 172.21.15.132:0/4089971596 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc27c004f70 con 0x7fc28413b2c0 2024-01-24T23:22:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:22:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2827fc640 10 monclient: dump: 2024-01-24T23:22:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:52.129 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 10 monclient: shutdown 2024-01-24T23:22:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 1 -- 172.21.15.132:0/4089971596 >> v1:172.21.15.132:6790/0 conn(0x7fc28413b2c0 legacy=0x7fc28413d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 1 -- 172.21.15.132:0/4089971596 shutdown_connections 2024-01-24T23:22:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 1 -- 172.21.15.132:0/4089971596 >> 172.21.15.132:0/4089971596 conn(0x7fc2840a5a10 msgr2=0x7fc2840a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 1 -- 172.21.15.132:0/4089971596 shutdown_connections 2024-01-24T23:22:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 1 -- 172.21.15.132:0/4089971596 wait complete. 2024-01-24T23:22:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 10 monclient: build_initial_monmap 2024-01-24T23:22:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 10 monclient: monmap: 2024-01-24T23:22:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:52.131 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 1 Processor -- start 2024-01-24T23:22:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 1 -- start start 2024-01-24T23:22:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 10 monclient: init 2024-01-24T23:22:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 10 monclient: _add_conns ranks=[8,0,5,3,7,4,1,2,6] 2024-01-24T23:22:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 10 monclient(hunting): picked mon.noname-e con 0x7fc28409f9c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 10 monclient(hunting): picked mon.noname-a con 0x7fc28409ff20 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:52.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 10 monclient(hunting): picked mon.noname-h con 0x7fc28413b2c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:52.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2841c08c0 con 0x7fc28409ff20 2024-01-24T23:22:52.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28411e4b0 con 0x7fc28413b2c0 2024-01-24T23:22:52.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28411fba0 con 0x7fc28409f9c0 2024-01-24T23:22:52.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:52.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 10 monclient(hunting): authenticate will time out at 1059.446406s 2024-01-24T23:22:52.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc2837fe640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7fc28409f9c0 legacy=0x7fc284132750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:52.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc283fff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fc28413b2c0 0x7fc2841d3dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:39936/0 (socket says 172.21.15.132:39936) 2024-01-24T23:22:52.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc283fff640 1 -- 172.21.15.132:0/3214461779 learned_addr learned my addr 172.21.15.132:0/3214461779 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:52.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 1 -- 172.21.15.132:0/3214461779 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1042441240 0 0) 0x7fc28411e4b0 con 0x7fc28413b2c0 2024-01-24T23:22:52.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:52.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:52.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:52.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 10 monclient(hunting): my global_id is 4730 2024-01-24T23:22:52.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 1 -- 172.21.15.132:0/3214461779 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc270003560 con 0x7fc28413b2c0 2024-01-24T23:22:52.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 1 -- 172.21.15.132:0/3214461779 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1368372023 0 0) 0x7fc270003560 con 0x7fc28413b2c0 2024-01-24T23:22:52.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 1 -- 172.21.15.132:0/3214461779 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc28411e4b0 con 0x7fc28413b2c0 2024-01-24T23:22:52.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 1 -- 172.21.15.132:0/3214461779 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc27c004eb0 con 0x7fc28413b2c0 2024-01-24T23:22:52.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:52.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:22:52.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 10 monclient(hunting): dump: 2024-01-24T23:22:52.133 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:52.133 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:52.133 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.133 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.133 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:52.133 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:52.133 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:52.134 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:52.134 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:52.134 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:52.134 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:52.134 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:52.134 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:52.134 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:52.134 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:52.134 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:52.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:52.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 1 -- 172.21.15.132:0/3214461779 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2932748659 0 0) 0x7fc28411e4b0 con 0x7fc28413b2c0 2024-01-24T23:22:52.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 1 -- 172.21.15.132:0/3214461779 >> v1:172.21.15.115:6793/0 conn(0x7fc28409f9c0 legacy=0x7fc284132750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:52.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 1 -- 172.21.15.132:0/3214461779 >> v1:172.21.15.115:6789/0 conn(0x7fc28409ff20 legacy=0x7fc2841d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:52.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:22:52.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 1 monclient: found mon.h 2024-01-24T23:22:52.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:52.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:52.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 1 -- 172.21.15.132:0/3214461779 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28411fe80 con 0x7fc28413b2c0 2024-01-24T23:22:52.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 10 monclient: _finish_auth 0 2024-01-24T23:22:52.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:22:52.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:52.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 5 monclient: authenticate success, global_id 4730 2024-01-24T23:22:52.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 10 monclient: _renew_subs 2024-01-24T23:22:52.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:52.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 1 -- 172.21.15.132:0/3214461779 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc284071960 con 0x7fc28413b2c0 2024-01-24T23:22:52.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 10 monclient: _renew_subs 2024-01-24T23:22:52.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:52.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc289c91640 1 -- 172.21.15.132:0/3214461779 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2840716e0 con 0x7fc28413b2c0 2024-01-24T23:22:52.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 1 -- 172.21.15.132:0/3214461779 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc27c0051a0 con 0x7fc28413b2c0 2024-01-24T23:22:52.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:52.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 1 -- 172.21.15.132:0/3214461779 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc27c005650 con 0x7fc28413b2c0 2024-01-24T23:22:52.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:52.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc281ffb640 4 set_mon_vals no callback set 2024-01-24T23:22:52.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:22:52.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.117+0000 7fc26bfff640 10 monclient: dump: 2024-01-24T23:22:52.136 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:52.136 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:52.136 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.136 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.136 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:52.136 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:52.136 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:52.136 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:52.136 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:52.137 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:52.137 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:52.137 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:52.137 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:52.137 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:52.137 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:52.137 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:52.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.121+0000 7fc289c91640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:52.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.121+0000 7fc289c91640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:52.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.121+0000 7fc289c91640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:52.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.121+0000 7fc289c91640 1 -- 172.21.15.132:0/3214461779 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc264005090 con 0x7fc28413b2c0 2024-01-24T23:22:52.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.121+0000 7fc26bfff640 1 -- 172.21.15.132:0/3214461779 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fc27c0100d0 con 0x7fc28413b2c0 2024-01-24T23:22:52.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.121+0000 7fc26bfff640 1 -- 172.21.15.132:0/3214461779 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(35..35 src has 1..35) v4 ==== 4634+0+0 (unknown 3048104250 0 0) 0x7fc27c0440d0 con 0x7fc28413b2c0 2024-01-24T23:22:52.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.121+0000 7fc2837fe640 10 monclient: get_auth_request con 0x7fc270036ed0 auth_method 0 2024-01-24T23:22:52.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.125+0000 7fc26bfff640 1 -- 172.21.15.132:0/3214461779 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fc27c014ed0 con 0x7fc28413b2c0 2024-01-24T23:22:52.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.125+0000 7fc26bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:52.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.125+0000 7fc26bfff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:52.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.289+0000 7fc289c91640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foo"}] 2024-01-24T23:22:52.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.289+0000 7fc289c91640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "foo"}] 2024-01-24T23:22:52.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.289+0000 7fc289c91640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:52.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.289+0000 7fc289c91640 1 -- 172.21.15.132:0/3214461779 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd pool delete", "pool": "foo"} v 0) v1 -- 0x7fc264005340 con 0x7fc28413b2c0 2024-01-24T23:22:52.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.293+0000 7fc26bfff640 1 -- 172.21.15.132:0/3214461779 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foo"}]=-1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool foo. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. v35) v1 ==== 274+0+0 (unknown 544749059 0 0) 0x7fc27c013570 con 0x7fc28413b2c0 2024-01-24T23:22:52.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.293+0000 7fc26bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "foo"}] 2024-01-24T23:22:52.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.293+0000 7fc26bfff640 10 monclient: _finish_command 2 = mon:1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool foo. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2024-01-24T23:22:52.295 INFO:tasks.workunit.client.0.smithi132.stderr:Error EPERM: WARNING: this will *PERMANENTLY DESTROY* all data stored in pool foo. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2024-01-24T23:22:52.299 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:22:52.299 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd pool delete foo foo 2024-01-24T23:22:52.299 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:22:52.299 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool delete foo foo 2024-01-24T23:22:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.405+0000 7f46fa023640 10 monclient: get_monmap_and_config 2024-01-24T23:22:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.405+0000 7f46fa023640 10 monclient: build_initial_monmap 2024-01-24T23:22:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.405+0000 7f46fa023640 10 monclient: monmap: 2024-01-24T23:22:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:52.412279+0000 2024-01-24T23:22:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:52.412279+0000 2024-01-24T23:22:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:52.412 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 1 Processor -- start 2024-01-24T23:22:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 1 -- start start 2024-01-24T23:22:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 10 monclient: init 2024-01-24T23:22:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 10 monclient: _add_conns ranks=[5,6,4,3,7,1,0,8,2] 2024-01-24T23:22:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 10 monclient(hunting): picked mon.noname-h con 0x7f46f409ff20 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 10 monclient(hunting): picked mon.noname-d con 0x7f46f409f9c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 10 monclient(hunting): picked mon.noname-c con 0x7f46f413b2c0 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f4064700 con 0x7f46f413b2c0 2024-01-24T23:22:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f409e760 con 0x7f46f409ff20 2024-01-24T23:22:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f40a1160 con 0x7f46f409f9c0 2024-01-24T23:22:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 10 monclient(hunting): authenticate will time out at 1059.736904s 2024-01-24T23:22:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f37fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f46f409ff20 0x7f46f40a0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:39942/0 (socket says 172.21.15.132:39942) 2024-01-24T23:22:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f37fe640 1 -- 172.21.15.132:0/1391592768 learned_addr learned my addr 172.21.15.132:0/1391592768 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 1 -- 172.21.15.132:0/1391592768 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 398871260 0 0) 0x7f46f409e760 con 0x7f46f409ff20 2024-01-24T23:22:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 10 monclient(hunting): my global_id is 4739 2024-01-24T23:22:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 1 -- 172.21.15.132:0/1391592768 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46e0003530 con 0x7f46f409ff20 2024-01-24T23:22:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 1 -- 172.21.15.132:0/1391592768 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1497026959 0 0) 0x7f46f40a1160 con 0x7f46f409f9c0 2024-01-24T23:22:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 10 monclient(hunting): my global_id is 4776 2024-01-24T23:22:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 1 -- 172.21.15.132:0/1391592768 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46f409e760 con 0x7f46f409f9c0 2024-01-24T23:22:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 1 -- 172.21.15.132:0/1391592768 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 737861066 0 0) 0x7f46f4064700 con 0x7f46f413b2c0 2024-01-24T23:22:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 10 monclient(hunting): my global_id is 74767 2024-01-24T23:22:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 1 -- 172.21.15.132:0/1391592768 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46f40a1160 con 0x7f46f413b2c0 2024-01-24T23:22:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 1 -- 172.21.15.132:0/1391592768 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3997527577 0 0) 0x7f46e0003530 con 0x7f46f409ff20 2024-01-24T23:22:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 1 -- 172.21.15.132:0/1391592768 >> v1:172.21.15.115:6792/0 conn(0x7f46f409f9c0 legacy=0x7f46f4139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 1 -- 172.21.15.132:0/1391592768 >> v1:172.21.15.115:6791/0 conn(0x7f46f413b2c0 legacy=0x7f46f413d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 10 monclient: _finish_hunting 0 2024-01-24T23:22:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 1 monclient: found mon.noname-h 2024-01-24T23:22:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:22:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 1 -- 172.21.15.132:0/1391592768 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f40a0a10 con 0x7f46f409ff20 2024-01-24T23:22:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 10 monclient: _finish_auth 0 2024-01-24T23:22:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 10 monclient: _check_auth_tickets 2024-01-24T23:22:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 5 monclient: authenticate success, global_id 4739 2024-01-24T23:22:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 1 -- 172.21.15.132:0/1391592768 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f46dc001740 con 0x7f46f409ff20 2024-01-24T23:22:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:22:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 10 monclient: dump: 2024-01-24T23:22:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:52.422 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:52.423 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 1 -- 172.21.15.132:0/1391592768 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f46dc001a20 con 0x7f46f409ff20 2024-01-24T23:22:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 4 set_mon_vals no callback set 2024-01-24T23:22:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 1 -- 172.21.15.132:0/1391592768 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f46dc004bf0 con 0x7f46f409ff20 2024-01-24T23:22:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:52.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:22:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46f27fc640 10 monclient: dump: 2024-01-24T23:22:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:52.424 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:52.425 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 10 monclient: shutdown 2024-01-24T23:22:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 1 -- 172.21.15.132:0/1391592768 >> v1:172.21.15.132:6791/0 conn(0x7f46f409ff20 legacy=0x7f46f40a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 1 -- 172.21.15.132:0/1391592768 shutdown_connections 2024-01-24T23:22:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 1 -- 172.21.15.132:0/1391592768 >> 172.21.15.132:0/1391592768 conn(0x7f46f40a5a10 msgr2=0x7f46f40a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 1 -- 172.21.15.132:0/1391592768 shutdown_connections 2024-01-24T23:22:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 1 -- 172.21.15.132:0/1391592768 wait complete. 2024-01-24T23:22:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 10 monclient: build_initial_monmap 2024-01-24T23:22:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 10 monclient: monmap: 2024-01-24T23:22:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:52.425 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:52.426 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 1 Processor -- start 2024-01-24T23:22:52.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 1 -- start start 2024-01-24T23:22:52.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.409+0000 7f46fa023640 10 monclient: init 2024-01-24T23:22:52.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46fa023640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:52.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46fa023640 10 monclient: _add_conns ranks=[7,1,6,0,2,5,4,8,3] 2024-01-24T23:22:52.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46fa023640 10 monclient(hunting): picked mon.noname-i con 0x7f46f409f9c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:52.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46fa023640 10 monclient(hunting): picked mon.noname-f con 0x7f46f413b2c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:52.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46fa023640 10 monclient(hunting): picked mon.noname-d con 0x7f46f406eb30 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:52.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46fa023640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f4071680 con 0x7f46f413b2c0 2024-01-24T23:22:52.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46fa023640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f4130a70 con 0x7f46f406eb30 2024-01-24T23:22:52.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46fa023640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f411ba20 con 0x7f46f409f9c0 2024-01-24T23:22:52.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46fa023640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:52.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46fa023640 10 monclient(hunting): authenticate will time out at 1059.740528s 2024-01-24T23:22:52.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46f37fe640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f46f409f9c0 0x7f46f4130360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:39858/0 (socket says 172.21.15.132:39858) 2024-01-24T23:22:52.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46f37fe640 1 -- 172.21.15.132:0/2733456569 learned_addr learned my addr 172.21.15.132:0/2733456569 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:52.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 1 -- 172.21.15.132:0/2733456569 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 54860584 0 0) 0x7f46f411ba20 con 0x7f46f409f9c0 2024-01-24T23:22:52.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:52.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:52.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:52.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient(hunting): my global_id is 4759 2024-01-24T23:22:52.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 1 -- 172.21.15.132:0/2733456569 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46d4003530 con 0x7f46f409f9c0 2024-01-24T23:22:52.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 1 -- 172.21.15.132:0/2733456569 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800879738 0 0) 0x7f46f4071680 con 0x7f46f413b2c0 2024-01-24T23:22:52.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:52.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:52.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:52.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient(hunting): my global_id is 4780 2024-01-24T23:22:52.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 1 -- 172.21.15.132:0/2733456569 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46f411ba20 con 0x7f46f413b2c0 2024-01-24T23:22:52.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 1 -- 172.21.15.132:0/2733456569 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2127666427 0 0) 0x7f46d4003530 con 0x7f46f409f9c0 2024-01-24T23:22:52.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 1 -- 172.21.15.132:0/2733456569 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f46f4071680 con 0x7f46f409f9c0 2024-01-24T23:22:52.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 1 -- 172.21.15.132:0/2733456569 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f46dc004b70 con 0x7f46f409f9c0 2024-01-24T23:22:52.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:52.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:22:52.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient(hunting): dump: 2024-01-24T23:22:52.429 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:52.429 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:52.429 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.429 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.429 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:52.429 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:52.429 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:52.429 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:52.429 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:52.429 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:52.429 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:52.429 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:52.429 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:52.430 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:52.430 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:52.430 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:52.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:52.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 1 -- 172.21.15.132:0/2733456569 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1686808683 0 0) 0x7f46f411ba20 con 0x7f46f413b2c0 2024-01-24T23:22:52.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 1 -- 172.21.15.132:0/2733456569 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f46d4003530 con 0x7f46f413b2c0 2024-01-24T23:22:52.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 1 -- 172.21.15.132:0/2733456569 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f46e4001da0 con 0x7f46f413b2c0 2024-01-24T23:22:52.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:52.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:22:52.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient(hunting): dump: 2024-01-24T23:22:52.430 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:52.430 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:52.430 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.430 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.430 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:52.431 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:52.431 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:52.431 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:52.431 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:52.431 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:52.431 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:52.431 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:52.431 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:52.431 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:52.431 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:52.431 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:52.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 1 -- 172.21.15.132:0/2733456569 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2183703865 0 0) 0x7f46f4130a70 con 0x7f46f406eb30 2024-01-24T23:22:52.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:52.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:52.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:52.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient(hunting): my global_id is 4785 2024-01-24T23:22:52.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 1 -- 172.21.15.132:0/2733456569 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46f411ba20 con 0x7f46f406eb30 2024-01-24T23:22:52.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 1 -- 172.21.15.132:0/2733456569 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3981840069 0 0) 0x7f46f4071680 con 0x7f46f409f9c0 2024-01-24T23:22:52.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 1 -- 172.21.15.132:0/2733456569 >> v1:172.21.15.115:6792/0 conn(0x7f46f406eb30 legacy=0x7f46f41d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:52.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 1 -- 172.21.15.132:0/2733456569 >> v1:172.21.15.132:6789/0 conn(0x7f46f413b2c0 legacy=0x7f46f41ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:52.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient: _finish_hunting 0 2024-01-24T23:22:52.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 1 monclient: found mon.i 2024-01-24T23:22:52.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:52.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:52.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 1 -- 172.21.15.132:0/2733456569 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f411bd00 con 0x7f46f409f9c0 2024-01-24T23:22:52.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient: _finish_auth 0 2024-01-24T23:22:52.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient: _check_auth_tickets 2024-01-24T23:22:52.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:52.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46fa023640 5 monclient: authenticate success, global_id 4759 2024-01-24T23:22:52.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46fa023640 10 monclient: _renew_subs 2024-01-24T23:22:52.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46fa023640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:52.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46fa023640 1 -- 172.21.15.132:0/2733456569 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f4071960 con 0x7f46f409f9c0 2024-01-24T23:22:52.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46fa023640 10 monclient: _renew_subs 2024-01-24T23:22:52.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46fa023640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:52.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46fa023640 1 -- 172.21.15.132:0/2733456569 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f406f730 con 0x7f46f409f9c0 2024-01-24T23:22:52.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 1 -- 172.21.15.132:0/2733456569 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f46dc001750 con 0x7f46f409f9c0 2024-01-24T23:22:52.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:52.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 1 -- 172.21.15.132:0/2733456569 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f46dc001240 con 0x7f46f409f9c0 2024-01-24T23:22:52.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:52.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46f1ffb640 4 set_mon_vals no callback set 2024-01-24T23:22:52.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:22:52.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 10 monclient: dump: 2024-01-24T23:22:52.433 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:52.433 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:52.433 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.433 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.434 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:52.434 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:52.434 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:52.434 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:52.434 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:52.434 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:52.434 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:52.434 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:52.434 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:52.434 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:52.434 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:52.434 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:52.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46fa023640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:52.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46fa023640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:52.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46fa023640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:52.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46fa023640 1 -- 172.21.15.132:0/2733456569 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d8005090 con 0x7f46f409f9c0 2024-01-24T23:22:52.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 1 -- 172.21.15.132:0/2733456569 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f46dc00fb20 con 0x7f46f409f9c0 2024-01-24T23:22:52.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46ebfff640 1 -- 172.21.15.132:0/2733456569 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(35..35 src has 1..35) v4 ==== 4634+0+0 (unknown 3048104250 0 0) 0x7f46dc043c60 con 0x7f46f409f9c0 2024-01-24T23:22:52.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.413+0000 7f46f2ffd640 10 monclient: get_auth_request con 0x7f46d4037830 auth_method 0 2024-01-24T23:22:52.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.417+0000 7f46ebfff640 1 -- 172.21.15.132:0/2733456569 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f46dc0149b0 con 0x7f46f409f9c0 2024-01-24T23:22:52.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.417+0000 7f46ebfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:52.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.417+0000 7f46ebfff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:52.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.573+0000 7f46fa023640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo"}] 2024-01-24T23:22:52.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.573+0000 7f46fa023640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo"}] 2024-01-24T23:22:52.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.573+0000 7f46fa023640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:52.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.573+0000 7f46fa023640 1 -- 172.21.15.132:0/2733456569 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd pool delete", "pool": "foo", "pool2": "foo"} v 0) v1 -- 0x7f46d80058c0 con 0x7f46f409f9c0 2024-01-24T23:22:52.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.573+0000 7f46ebfff640 1 -- 172.21.15.132:0/2733456569 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo"}]=-1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool foo. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. v35) v1 ==== 290+0+0 (unknown 2967130567 0 0) 0x7f46dc013050 con 0x7f46f409f9c0 2024-01-24T23:22:52.579 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.573+0000 7f46ebfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo"}] 2024-01-24T23:22:52.579 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.573+0000 7f46ebfff640 10 monclient: _finish_command 2 = mon:1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool foo. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2024-01-24T23:22:52.579 INFO:tasks.workunit.client.0.smithi132.stderr:Error EPERM: WARNING: this will *PERMANENTLY DESTROY* all data stored in pool foo. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2024-01-24T23:22:52.583 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:22:52.583 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd pool delete foo foo --force 2024-01-24T23:22:52.583 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:22:52.583 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool delete foo foo --force 2024-01-24T23:22:52.700 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.697+0000 7f83ecad5640 10 monclient: get_monmap_and_config 2024-01-24T23:22:52.701 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.697+0000 7f83ecad5640 10 monclient: build_initial_monmap 2024-01-24T23:22:52.701 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.697+0000 7f83ecad5640 10 monclient: monmap: 2024-01-24T23:22:52.701 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:52.701 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:52.701 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:52.702414+0000 2024-01-24T23:22:52.701 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:52.702414+0000 2024-01-24T23:22:52.701 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:52.701 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:52.701 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:52.701 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:52.701 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:52.701 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:52.701 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:52.701 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:52.701 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:52.702 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:52.702 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:52.702 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:52.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.697+0000 7f83ecad5640 1 Processor -- start 2024-01-24T23:22:52.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.697+0000 7f83ecad5640 1 -- start start 2024-01-24T23:22:52.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.697+0000 7f83ecad5640 10 monclient: init 2024-01-24T23:22:52.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.697+0000 7f83ecad5640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:52.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.697+0000 7f83ecad5640 10 monclient: _add_conns ranks=[5,0,2,1,3,7,8,4,6] 2024-01-24T23:22:52.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.697+0000 7f83ecad5640 10 monclient(hunting): picked mon.noname-h con 0x7f83e80a0ad0 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:52.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.697+0000 7f83ecad5640 10 monclient(hunting): picked mon.noname-a con 0x7f83e809ee80 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:52.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.697+0000 7f83ecad5640 10 monclient(hunting): picked mon.noname-b con 0x7f83e809b060 addr v1:172.21.15.115:6790/0 2024-01-24T23:22:52.706 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.697+0000 7f83ecad5640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e8064700 con 0x7f83e809ee80 2024-01-24T23:22:52.706 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.697+0000 7f83ecad5640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e809cb80 con 0x7f83e809b060 2024-01-24T23:22:52.706 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.697+0000 7f83ecad5640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e80a0eb0 con 0x7f83e80a0ad0 2024-01-24T23:22:52.706 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.697+0000 7f83ecad5640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:52.706 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.697+0000 7f83ecad5640 10 monclient(hunting): authenticate will time out at 1060.027946s 2024-01-24T23:22:52.706 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e659c640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f83e80a0ad0 0x7f83e809a240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:39944/0 (socket says 172.21.15.132:39944) 2024-01-24T23:22:52.706 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e659c640 1 -- 172.21.15.132:0/4166320516 learned_addr learned my addr 172.21.15.132:0/4166320516 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:52.706 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e6d9d640 1 -- 172.21.15.132:0/4166320516 >> v1:172.21.15.115:6790/0 conn(0x7f83e809b060 legacy=0x7f83e813c270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:22:52.706 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 1 -- 172.21.15.132:0/4166320516 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881689990 0 0) 0x7f83e80a0eb0 con 0x7f83e80a0ad0 2024-01-24T23:22:52.706 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:22:52.706 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:52.706 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:52.706 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 10 monclient(hunting): my global_id is 4748 2024-01-24T23:22:52.706 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 1 -- 172.21.15.132:0/4166320516 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83d4003530 con 0x7f83e80a0ad0 2024-01-24T23:22:52.707 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 1 -- 172.21.15.132:0/4166320516 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3123916771 0 0) 0x7f83d4003530 con 0x7f83e80a0ad0 2024-01-24T23:22:52.707 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 1 -- 172.21.15.132:0/4166320516 >> v1:172.21.15.115:6790/0 conn(0x7f83e809b060 legacy=0x7f83e813c270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:52.707 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 1 -- 172.21.15.132:0/4166320516 >> v1:172.21.15.115:6789/0 conn(0x7f83e809ee80 legacy=0x7f83e809a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:52.707 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 10 monclient: _finish_hunting 0 2024-01-24T23:22:52.707 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 1 monclient: found mon.noname-h 2024-01-24T23:22:52.707 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:52.707 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:22:52.707 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 1 -- 172.21.15.132:0/4166320516 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e809f280 con 0x7f83e80a0ad0 2024-01-24T23:22:52.707 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 10 monclient: _finish_auth 0 2024-01-24T23:22:52.707 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 10 monclient: _check_auth_tickets 2024-01-24T23:22:52.707 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:52.707 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 1 -- 172.21.15.132:0/4166320516 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f83d80017c0 con 0x7f83e80a0ad0 2024-01-24T23:22:52.707 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 5 monclient: authenticate success, global_id 4748 2024-01-24T23:22:52.707 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:52.708 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:52.708 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:22:52.708 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 10 monclient: dump: 2024-01-24T23:22:52.708 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:52.708 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:52.708 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.708 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.708 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:52.708 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:52.708 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:52.708 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:52.708 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:52.708 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:52.708 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:52.708 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:52.709 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:52.709 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:52.709 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:52.709 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:52.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:52.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 1 -- 172.21.15.132:0/4166320516 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f83d8001a70 con 0x7f83e80a0ad0 2024-01-24T23:22:52.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:52.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:52.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 1 -- 172.21.15.132:0/4166320516 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f83d8004bf0 con 0x7f83e80a0ad0 2024-01-24T23:22:52.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:52.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 4 set_mon_vals no callback set 2024-01-24T23:22:52.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:22:52.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e5d9b640 10 monclient: dump: 2024-01-24T23:22:52.709 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:52.709 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:52.710 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.710 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.710 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:52.710 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:52.710 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:52.710 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:52.710 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:52.710 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:52.710 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:52.710 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:52.710 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:52.710 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:52.710 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:52.710 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:52.710 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 10 monclient: shutdown 2024-01-24T23:22:52.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:52.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 1 -- 172.21.15.132:0/4166320516 >> v1:172.21.15.132:6791/0 conn(0x7f83e80a0ad0 legacy=0x7f83e809a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:52.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 1 -- 172.21.15.132:0/4166320516 shutdown_connections 2024-01-24T23:22:52.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 1 -- 172.21.15.132:0/4166320516 >> 172.21.15.132:0/4166320516 conn(0x7f83e80a1bc0 msgr2=0x7f83e80a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:52.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 1 -- 172.21.15.132:0/4166320516 shutdown_connections 2024-01-24T23:22:52.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 1 -- 172.21.15.132:0/4166320516 wait complete. 2024-01-24T23:22:52.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 10 monclient: build_initial_monmap 2024-01-24T23:22:52.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 10 monclient: monmap: 2024-01-24T23:22:52.711 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:52.711 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:52.711 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:52.711 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:52.711 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:52.711 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:52.711 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:52.712 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:52.712 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:52.712 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:52.712 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:52.712 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:52.712 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:52.712 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:52.712 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:52.712 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:52.712 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 1 Processor -- start 2024-01-24T23:22:52.712 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 1 -- start start 2024-01-24T23:22:52.712 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 10 monclient: init 2024-01-24T23:22:52.716 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:52.716 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 10 monclient: _add_conns ranks=[7,5,1,8,4,2,0,3,6] 2024-01-24T23:22:52.716 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 10 monclient(hunting): picked mon.noname-i con 0x7f83e809b060 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:52.716 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 10 monclient(hunting): picked mon.noname-h con 0x7f83e809ee80 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:52.716 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 10 monclient(hunting): picked mon.noname-f con 0x7f83e80a0ad0 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:52.716 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e8097cb0 con 0x7f83e80a0ad0 2024-01-24T23:22:52.716 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e809b4c0 con 0x7f83e809ee80 2024-01-24T23:22:52.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e809e640 con 0x7f83e809b060 2024-01-24T23:22:52.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:52.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83ecad5640 10 monclient(hunting): authenticate will time out at 1060.031174s 2024-01-24T23:22:52.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e6d9d640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f83e80a0ad0 0x7f83e81d4230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:38036/0 (socket says 172.21.15.132:38036) 2024-01-24T23:22:52.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83e6d9d640 1 -- 172.21.15.132:0/1273760864 learned_addr learned my addr 172.21.15.132:0/1273760864 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:52.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 1 -- 172.21.15.132:0/1273760864 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4268565205 0 0) 0x7f83e8097cb0 con 0x7f83e80a0ad0 2024-01-24T23:22:52.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 10 monclient(hunting): _negotiate 2024-01-24T23:22:52.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:52.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:52.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 10 monclient(hunting): my global_id is 4789 2024-01-24T23:22:52.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 1 -- 172.21.15.132:0/1273760864 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83c8003530 con 0x7f83e80a0ad0 2024-01-24T23:22:52.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 1 -- 172.21.15.132:0/1273760864 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 306155881 0 0) 0x7f83e809e640 con 0x7f83e809b060 2024-01-24T23:22:52.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 10 monclient(hunting): _negotiate 2024-01-24T23:22:52.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:52.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:52.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 10 monclient(hunting): my global_id is 4768 2024-01-24T23:22:52.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 1 -- 172.21.15.132:0/1273760864 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83e8097cb0 con 0x7f83e809b060 2024-01-24T23:22:52.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 1 -- 172.21.15.132:0/1273760864 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2989564816 0 0) 0x7f83e809b4c0 con 0x7f83e809ee80 2024-01-24T23:22:52.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 10 monclient(hunting): _negotiate 2024-01-24T23:22:52.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:52.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:52.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 10 monclient(hunting): my global_id is 4757 2024-01-24T23:22:52.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 1 -- 172.21.15.132:0/1273760864 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83e809e640 con 0x7f83e809ee80 2024-01-24T23:22:52.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 1 -- 172.21.15.132:0/1273760864 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4072128770 0 0) 0x7f83e8097cb0 con 0x7f83e809b060 2024-01-24T23:22:52.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 1 -- 172.21.15.132:0/1273760864 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f83e809b4c0 con 0x7f83e809b060 2024-01-24T23:22:52.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 1 -- 172.21.15.132:0/1273760864 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f83d8001780 con 0x7f83e809b060 2024-01-24T23:22:52.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:52.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:22:52.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 10 monclient(hunting): dump: 2024-01-24T23:22:52.719 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:52.719 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:52.719 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.719 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.719 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:52.719 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:52.719 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:52.719 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:52.719 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:52.719 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:52.719 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:52.719 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:52.719 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:52.719 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:52.719 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:52.720 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:52.720 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:52.720 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 1 -- 172.21.15.132:0/1273760864 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 308778744 0 0) 0x7f83e809e640 con 0x7f83e809ee80 2024-01-24T23:22:52.720 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 1 -- 172.21.15.132:0/1273760864 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f83e8097cb0 con 0x7f83e809ee80 2024-01-24T23:22:52.720 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.701+0000 7f83deffd640 1 -- 172.21.15.132:0/1273760864 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3623910071 0 0) 0x7f83c8003530 con 0x7f83e80a0ad0 2024-01-24T23:22:52.720 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 1 -- 172.21.15.132:0/1273760864 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f83e809e640 con 0x7f83e80a0ad0 2024-01-24T23:22:52.720 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 1 -- 172.21.15.132:0/1273760864 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f83d0001960 con 0x7f83e809ee80 2024-01-24T23:22:52.720 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:52.720 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:22:52.720 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 10 monclient(hunting): dump: 2024-01-24T23:22:52.720 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:52.720 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:52.720 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.720 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.720 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:52.721 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:52.721 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:52.721 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:52.721 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:52.721 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:52.721 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:52.721 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:52.721 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:52.721 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:52.721 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:52.721 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:52.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 1 -- 172.21.15.132:0/1273760864 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f83e0004520 con 0x7f83e80a0ad0 2024-01-24T23:22:52.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:52.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:22:52.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 10 monclient(hunting): dump: 2024-01-24T23:22:52.722 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:52.722 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:52.722 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.722 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.722 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:52.722 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:52.722 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:52.722 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:52.722 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:52.722 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:52.722 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:52.722 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:52.722 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:52.722 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:52.723 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:52.723 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:52.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 1 -- 172.21.15.132:0/1273760864 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3869608623 0 0) 0x7f83e809b4c0 con 0x7f83e809b060 2024-01-24T23:22:52.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 1 -- 172.21.15.132:0/1273760864 >> v1:172.21.15.132:6791/0 conn(0x7f83e809ee80 legacy=0x7f83e81d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:52.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 1 -- 172.21.15.132:0/1273760864 >> v1:172.21.15.132:6789/0 conn(0x7f83e80a0ad0 legacy=0x7f83e81d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:52.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 10 monclient: _finish_hunting 0 2024-01-24T23:22:52.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 1 monclient: found mon.i 2024-01-24T23:22:52.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:52.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:52.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 1 -- 172.21.15.132:0/1273760864 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e80979d0 con 0x7f83e809b060 2024-01-24T23:22:52.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 10 monclient: _finish_auth 0 2024-01-24T23:22:52.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 10 monclient: _check_auth_tickets 2024-01-24T23:22:52.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:52.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83ecad5640 5 monclient: authenticate success, global_id 4768 2024-01-24T23:22:52.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83ecad5640 10 monclient: _renew_subs 2024-01-24T23:22:52.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83ecad5640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:52.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83ecad5640 1 -- 172.21.15.132:0/1273760864 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e8099eb0 con 0x7f83e809b060 2024-01-24T23:22:52.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83ecad5640 10 monclient: _renew_subs 2024-01-24T23:22:52.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83ecad5640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:52.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83ecad5640 1 -- 172.21.15.132:0/1273760864 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e8096c90 con 0x7f83e809b060 2024-01-24T23:22:52.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 1 -- 172.21.15.132:0/1273760864 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f83d8001140 con 0x7f83e809b060 2024-01-24T23:22:52.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:52.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 1 -- 172.21.15.132:0/1273760864 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f83d8004c70 con 0x7f83e809b060 2024-01-24T23:22:52.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:52.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:22:52.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83e4d99640 4 set_mon_vals no callback set 2024-01-24T23:22:52.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 10 monclient: dump: 2024-01-24T23:22:52.725 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:52.725 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:52.725 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.725 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:52.725 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:52.725 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:52.725 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:52.725 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:52.725 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:52.725 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:52.725 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:52.725 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:52.725 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:52.725 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:52.726 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:52.726 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:52.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83ecad5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:52.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83ecad5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:52.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83ecad5640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:52.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 1 -- 172.21.15.132:0/1273760864 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f83d800fb00 con 0x7f83e809b060 2024-01-24T23:22:52.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83ecad5640 1 -- 172.21.15.132:0/1273760864 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83cc005090 con 0x7f83e809b060 2024-01-24T23:22:52.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83deffd640 1 -- 172.21.15.132:0/1273760864 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(35..35 src has 1..35) v4 ==== 4634+0+0 (unknown 3048104250 0 0) 0x7f83d8043d10 con 0x7f83e809b060 2024-01-24T23:22:52.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.705+0000 7f83dffff640 10 monclient: get_auth_request con 0x7f83c8037cf0 auth_method 0 2024-01-24T23:22:52.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.709+0000 7f83deffd640 1 -- 172.21.15.132:0/1273760864 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f83d80149f0 con 0x7f83e809b060 2024-01-24T23:22:52.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.709+0000 7f83deffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:52.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.709+0000 7f83deffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:52.868 INFO:tasks.workunit.client.0.smithi132.stderr:--force not valid: --force not one of 'true', 'false' 2024-01-24T23:22:52.868 INFO:tasks.workunit.client.0.smithi132.stderr:Invalid command: unused arguments: ['--force'] 2024-01-24T23:22:52.868 INFO:tasks.workunit.client.0.smithi132.stderr:osd pool delete [] [--yes-i-really-really-mean-it] [--yes-i-really-really-mean-it-not-faking] : delete pool 2024-01-24T23:22:52.868 INFO:tasks.workunit.client.0.smithi132.stderr:Error EINVAL: invalid command 2024-01-24T23:22:52.871 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:22:52.871 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd pool delete foo fooo --yes-i-really-mean-it 2024-01-24T23:22:52.871 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:22:52.871 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool delete foo fooo --yes-i-really-mean-it 2024-01-24T23:22:52.996 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2ae03b640 10 monclient: get_monmap_and_config 2024-01-24T23:22:52.996 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2ae03b640 10 monclient: build_initial_monmap 2024-01-24T23:22:52.996 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2ae03b640 10 monclient: monmap: 2024-01-24T23:22:52.996 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:52.996 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:52.996 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:52.997830+0000 2024-01-24T23:22:52.996 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:52.997830+0000 2024-01-24T23:22:52.996 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:52.996 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:52.997 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:52.997 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:52.997 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:52.997 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:52.997 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:52.997 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:52.997 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:52.997 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:52.997 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:52.997 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:53.004 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2ae03b640 1 Processor -- start 2024-01-24T23:22:53.004 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2ae03b640 1 -- start start 2024-01-24T23:22:53.004 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2ae03b640 10 monclient: init 2024-01-24T23:22:53.004 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2ae03b640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:53.004 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2ae03b640 10 monclient: _add_conns ranks=[5,2,1,7,0,6,4,3,8] 2024-01-24T23:22:53.004 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2ae03b640 10 monclient(hunting): picked mon.noname-h con 0x7fc2a8133ce0 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:53.004 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2ae03b640 10 monclient(hunting): picked mon.noname-b con 0x7fc2a809ddc0 addr v1:172.21.15.115:6790/0 2024-01-24T23:22:53.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2ae03b640 10 monclient(hunting): picked mon.noname-f con 0x7fc2a809e1a0 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:53.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2ae03b640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a8064700 con 0x7fc2a809e1a0 2024-01-24T23:22:53.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2ae03b640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a809c6d0 con 0x7fc2a809ddc0 2024-01-24T23:22:53.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2ae03b640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a809c9b0 con 0x7fc2a8133ce0 2024-01-24T23:22:53.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2ae03b640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:53.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2ae03b640 10 monclient(hunting): authenticate will time out at 1060.322596s 2024-01-24T23:22:53.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a77fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fc2a8133ce0 0x7fc2a81340c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:39958/0 (socket says 172.21.15.132:39958) 2024-01-24T23:22:53.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a77fe640 1 -- 172.21.15.132:0/1150370921 learned_addr learned my addr 172.21.15.132:0/1150370921 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:53.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a6ffd640 1 -- 172.21.15.132:0/1150370921 >> v1:172.21.15.115:6790/0 conn(0x7fc2a809ddc0 legacy=0x7fc2a8138ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:22:53.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 1 -- 172.21.15.132:0/1150370921 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3221029001 0 0) 0x7fc2a809c9b0 con 0x7fc2a8133ce0 2024-01-24T23:22:53.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:53.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:53.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:53.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 10 monclient(hunting): my global_id is 4766 2024-01-24T23:22:53.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 1 -- 172.21.15.132:0/1150370921 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc290003530 con 0x7fc2a8133ce0 2024-01-24T23:22:53.006 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 1 -- 172.21.15.132:0/1150370921 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1796151970 0 0) 0x7fc2a8064700 con 0x7fc2a809e1a0 2024-01-24T23:22:53.006 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:53.006 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:53.006 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:53.006 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 10 monclient(hunting): my global_id is 4798 2024-01-24T23:22:53.006 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 1 -- 172.21.15.132:0/1150370921 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2a809c9b0 con 0x7fc2a809e1a0 2024-01-24T23:22:53.006 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 1 -- 172.21.15.132:0/1150370921 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2106587133 0 0) 0x7fc290003530 con 0x7fc2a8133ce0 2024-01-24T23:22:53.006 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 1 -- 172.21.15.132:0/1150370921 >> v1:172.21.15.115:6790/0 conn(0x7fc2a809ddc0 legacy=0x7fc2a8138ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:53.006 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 1 -- 172.21.15.132:0/1150370921 >> v1:172.21.15.132:6789/0 conn(0x7fc2a809e1a0 legacy=0x7fc2a813c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:53.006 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 10 monclient: _finish_hunting 0 2024-01-24T23:22:53.006 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 1 monclient: found mon.noname-h 2024-01-24T23:22:53.006 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:53.006 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:22:53.006 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 1 -- 172.21.15.132:0/1150370921 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a809cc90 con 0x7fc2a8133ce0 2024-01-24T23:22:53.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 10 monclient: _finish_auth 0 2024-01-24T23:22:53.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 10 monclient: _check_auth_tickets 2024-01-24T23:22:53.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:53.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 1 -- 172.21.15.132:0/1150370921 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc2980017c0 con 0x7fc2a8133ce0 2024-01-24T23:22:53.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:53.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:22:53.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 10 monclient: dump: 2024-01-24T23:22:53.007 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:53.007 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:53.007 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.007 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.007 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:53.007 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:53.007 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:53.007 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:53.007 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:53.008 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:53.008 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:53.008 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:53.008 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:53.008 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:53.008 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:53.008 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:53.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2ae03b640 5 monclient: authenticate success, global_id 4766 2024-01-24T23:22:53.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2ae03b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:53.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 1 -- 172.21.15.132:0/1150370921 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc298001a30 con 0x7fc2a8133ce0 2024-01-24T23:22:53.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:53.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 1 -- 172.21.15.132:0/1150370921 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc298004bf0 con 0x7fc2a8133ce0 2024-01-24T23:22:53.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2ae03b640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:53.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2ae03b640 4 set_mon_vals no callback set 2024-01-24T23:22:53.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:53.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:22:53.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2a67fc640 10 monclient: dump: 2024-01-24T23:22:53.009 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:53.009 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:53.009 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.009 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.009 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:53.009 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:53.009 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:53.009 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:53.009 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:53.009 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:53.009 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:53.009 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:53.009 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:53.010 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:53.010 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:53.010 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:53.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2ae03b640 10 monclient: shutdown 2024-01-24T23:22:53.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2ae03b640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:53.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.993+0000 7fc2ae03b640 1 -- 172.21.15.132:0/1150370921 >> v1:172.21.15.132:6791/0 conn(0x7fc2a8133ce0 legacy=0x7fc2a81340c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:53.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 1 -- 172.21.15.132:0/1150370921 shutdown_connections 2024-01-24T23:22:53.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 1 -- 172.21.15.132:0/1150370921 >> 172.21.15.132:0/1150370921 conn(0x7fc2a80a6840 msgr2=0x7fc2a80a2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:53.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 1 -- 172.21.15.132:0/1150370921 shutdown_connections 2024-01-24T23:22:53.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 1 -- 172.21.15.132:0/1150370921 wait complete. 2024-01-24T23:22:53.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 10 monclient: build_initial_monmap 2024-01-24T23:22:53.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 10 monclient: monmap: 2024-01-24T23:22:53.010 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:53.010 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:53.011 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:53.011 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:53.011 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:53.011 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:53.011 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:53.011 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:53.011 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:53.011 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:53.011 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:53.011 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:53.011 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:53.011 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:53.011 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:53.011 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:53.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 1 Processor -- start 2024-01-24T23:22:53.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 1 -- start start 2024-01-24T23:22:53.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 10 monclient: init 2024-01-24T23:22:53.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:53.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 10 monclient: _add_conns ranks=[5,7,4,6,1,3,2,0,8] 2024-01-24T23:22:53.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 10 monclient(hunting): picked mon.noname-h con 0x7fc2a809ddc0 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:53.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 10 monclient(hunting): picked mon.noname-i con 0x7fc2a809e1a0 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:53.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 10 monclient(hunting): picked mon.noname-c con 0x7fc2a8133ce0 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:53.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a809ba70 con 0x7fc2a8133ce0 2024-01-24T23:22:53.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a811d590 con 0x7fc2a809ddc0 2024-01-24T23:22:53.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a80723d0 con 0x7fc2a809e1a0 2024-01-24T23:22:53.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:53.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 10 monclient(hunting): authenticate will time out at 1060.325544s 2024-01-24T23:22:53.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2a77fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fc2a809ddc0 0x7fc2a812fbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:39966/0 (socket says 172.21.15.132:39966) 2024-01-24T23:22:53.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2a77fe640 1 -- 172.21.15.132:0/4020884361 learned_addr learned my addr 172.21.15.132:0/4020884361 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:53.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 1 -- 172.21.15.132:0/4020884361 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1736374900 0 0) 0x7fc2a80723d0 con 0x7fc2a809e1a0 2024-01-24T23:22:53.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:53.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:53.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:53.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient(hunting): my global_id is 4777 2024-01-24T23:22:53.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 1 -- 172.21.15.132:0/4020884361 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc294003530 con 0x7fc2a809e1a0 2024-01-24T23:22:53.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 1 -- 172.21.15.132:0/4020884361 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3341729616 0 0) 0x7fc2a811d590 con 0x7fc2a809ddc0 2024-01-24T23:22:53.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:53.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:53.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:53.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient(hunting): my global_id is 4775 2024-01-24T23:22:53.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 1 -- 172.21.15.132:0/4020884361 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2a80723d0 con 0x7fc2a809ddc0 2024-01-24T23:22:53.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 1 -- 172.21.15.132:0/4020884361 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3723743244 0 0) 0x7fc294003530 con 0x7fc2a809e1a0 2024-01-24T23:22:53.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 1 -- 172.21.15.132:0/4020884361 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc2a811d590 con 0x7fc2a809e1a0 2024-01-24T23:22:53.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 1 -- 172.21.15.132:0/4020884361 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1593912772 0 0) 0x7fc2a80723d0 con 0x7fc2a809ddc0 2024-01-24T23:22:53.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 1 -- 172.21.15.132:0/4020884361 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc294003530 con 0x7fc2a809ddc0 2024-01-24T23:22:53.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 1 -- 172.21.15.132:0/4020884361 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc29c0042a0 con 0x7fc2a809e1a0 2024-01-24T23:22:53.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:53.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:22:53.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient(hunting): dump: 2024-01-24T23:22:53.014 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:53.014 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:53.014 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.014 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.014 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:53.014 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:53.015 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:53.015 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:53.015 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:53.015 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:53.015 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:53.015 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:53.015 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:53.015 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:53.015 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:53.015 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:53.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:53.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 1 -- 172.21.15.132:0/4020884361 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc2980047f0 con 0x7fc2a809ddc0 2024-01-24T23:22:53.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:53.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:22:53.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient(hunting): dump: 2024-01-24T23:22:53.016 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:53.016 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:53.016 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.016 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.016 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:53.016 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:53.016 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:53.016 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:53.016 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:53.016 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:53.016 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:53.016 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:53.016 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:53.016 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:53.016 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:53.017 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:53.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 1 -- 172.21.15.132:0/4020884361 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 376277326 0 0) 0x7fc2a809ba70 con 0x7fc2a8133ce0 2024-01-24T23:22:53.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:53.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:53.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:53.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient(hunting): my global_id is 74776 2024-01-24T23:22:53.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 1 -- 172.21.15.132:0/4020884361 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2a80723d0 con 0x7fc2a8133ce0 2024-01-24T23:22:53.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 1 -- 172.21.15.132:0/4020884361 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1808758735 0 0) 0x7fc2a811d590 con 0x7fc2a809e1a0 2024-01-24T23:22:53.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 1 -- 172.21.15.132:0/4020884361 >> v1:172.21.15.132:6791/0 conn(0x7fc2a809ddc0 legacy=0x7fc2a812fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:53.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 1 -- 172.21.15.132:0/4020884361 >> v1:172.21.15.115:6791/0 conn(0x7fc2a8133ce0 legacy=0x7fc2a81d1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:53.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient: _finish_hunting 0 2024-01-24T23:22:53.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 1 monclient: found mon.i 2024-01-24T23:22:53.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:53.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:53.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 1 -- 172.21.15.132:0/4020884361 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a8070c90 con 0x7fc2a809e1a0 2024-01-24T23:22:53.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient: _finish_auth 0 2024-01-24T23:22:53.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient: _check_auth_tickets 2024-01-24T23:22:53.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:53.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 5 monclient: authenticate success, global_id 4777 2024-01-24T23:22:53.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 10 monclient: _renew_subs 2024-01-24T23:22:53.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:53.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 1 -- 172.21.15.132:0/4020884361 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a811d2b0 con 0x7fc2a809e1a0 2024-01-24T23:22:53.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 10 monclient: _renew_subs 2024-01-24T23:22:53.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:53.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 1 -- 172.21.15.132:0/4020884361 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a81302e0 con 0x7fc2a809e1a0 2024-01-24T23:22:53.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 1 -- 172.21.15.132:0/4020884361 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc29c004450 con 0x7fc2a809e1a0 2024-01-24T23:22:53.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:53.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 1 -- 172.21.15.132:0/4020884361 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc29c004980 con 0x7fc2a809e1a0 2024-01-24T23:22:53.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:53.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2a5ffb640 4 set_mon_vals no callback set 2024-01-24T23:22:53.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:22:53.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 10 monclient: dump: 2024-01-24T23:22:53.019 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:53.019 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:53.019 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.019 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.019 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:53.019 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:53.019 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:53.019 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:53.019 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:53.019 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:53.019 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:53.019 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:53.019 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:53.019 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:53.020 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:53.020 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:53.020 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:53.020 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:53.020 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:53.020 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2ae03b640 1 -- 172.21.15.132:0/4020884361 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc288005090 con 0x7fc2a809e1a0 2024-01-24T23:22:53.020 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 1 -- 172.21.15.132:0/4020884361 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fc29c004ba0 con 0x7fc2a809e1a0 2024-01-24T23:22:53.020 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc28ffff640 1 -- 172.21.15.132:0/4020884361 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(35..35 src has 1..35) v4 ==== 4634+0+0 (unknown 3048104250 0 0) 0x7fc29c043af0 con 0x7fc2a809e1a0 2024-01-24T23:22:53.020 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:52.997+0000 7fc2a77fe640 10 monclient: get_auth_request con 0x7fc294037870 auth_method 0 2024-01-24T23:22:53.020 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.001+0000 7fc28ffff640 1 -- 172.21.15.132:0/4020884361 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fc29c045e60 con 0x7fc2a809e1a0 2024-01-24T23:22:53.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.001+0000 7fc28ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:53.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.001+0000 7fc28ffff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:53.160 INFO:tasks.workunit.client.0.smithi132.stderr:--yes-i-really-mean-it not valid: --yes-i-really-mean-it not one of 'true', 'false' 2024-01-24T23:22:53.160 INFO:tasks.workunit.client.0.smithi132.stderr:Invalid command: unused arguments: ['--yes-i-really-mean-it'] 2024-01-24T23:22:53.160 INFO:tasks.workunit.client.0.smithi132.stderr:osd pool delete [] [--yes-i-really-really-mean-it] [--yes-i-really-really-mean-it-not-faking] : delete pool 2024-01-24T23:22:53.160 INFO:tasks.workunit.client.0.smithi132.stderr:Error EINVAL: invalid command 2024-01-24T23:22:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:22:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd pool delete foo --yes-i-really-mean-it foo 2024-01-24T23:22:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:22:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool delete foo --yes-i-really-mean-it foo 2024-01-24T23:22:53.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d5e367640 10 monclient: get_monmap_and_config 2024-01-24T23:22:53.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d5e367640 10 monclient: build_initial_monmap 2024-01-24T23:22:53.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d5e367640 10 monclient: monmap: 2024-01-24T23:22:53.283 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:53.283 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:53.284 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:53.285104+0000 2024-01-24T23:22:53.284 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:53.285104+0000 2024-01-24T23:22:53.284 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:53.284 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:53.284 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:53.284 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:53.284 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:53.284 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:53.284 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:53.284 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:53.284 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:53.284 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:53.284 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:53.284 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:53.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d5e367640 1 Processor -- start 2024-01-24T23:22:53.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d5e367640 1 -- start start 2024-01-24T23:22:53.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d5e367640 10 monclient: init 2024-01-24T23:22:53.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d5e367640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:53.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d5e367640 10 monclient: _add_conns ranks=[6,3,0,2,4,8,1,7,5] 2024-01-24T23:22:53.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d5e367640 10 monclient(hunting): picked mon.noname-d con 0x7f7d58134cc0 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:53.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d5e367640 10 monclient(hunting): picked mon.noname-g con 0x7f7d5806de30 addr v1:172.21.15.132:6790/0 2024-01-24T23:22:53.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d5e367640 10 monclient(hunting): picked mon.noname-a con 0x7f7d5813b3a0 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:53.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d5e367640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d58064700 con 0x7f7d5813b3a0 2024-01-24T23:22:53.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d5e367640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d58070e90 con 0x7f7d5806de30 2024-01-24T23:22:53.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d5e367640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d58071170 con 0x7f7d58134cc0 2024-01-24T23:22:53.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d5e367640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:53.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d5e367640 10 monclient(hunting): authenticate will time out at 1060.610818s 2024-01-24T23:22:53.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d4ffff640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f7d5806de30 0x7f7d58139c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:48810/0 (socket says 172.21.15.132:48810) 2024-01-24T23:22:53.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d4ffff640 1 -- 172.21.15.132:0/3666158787 learned_addr learned my addr 172.21.15.132:0/3666158787 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:53.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 1 -- 172.21.15.132:0/3666158787 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968283225 0 0) 0x7f7d58070e90 con 0x7f7d5806de30 2024-01-24T23:22:53.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 10 monclient(hunting): _negotiate 2024-01-24T23:22:53.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:53.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:53.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 10 monclient(hunting): my global_id is 4710 2024-01-24T23:22:53.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 1 -- 172.21.15.132:0/3666158787 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d44003530 con 0x7f7d5806de30 2024-01-24T23:22:53.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 1 -- 172.21.15.132:0/3666158787 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3200051827 0 0) 0x7f7d44003530 con 0x7f7d5806de30 2024-01-24T23:22:53.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 1 -- 172.21.15.132:0/3666158787 >> v1:172.21.15.115:6792/0 conn(0x7f7d58134cc0 legacy=0x7f7d581350a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:53.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 1 -- 172.21.15.132:0/3666158787 >> v1:172.21.15.115:6789/0 conn(0x7f7d5813b3a0 legacy=0x7f7d5813d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:53.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 10 monclient: _finish_hunting 0 2024-01-24T23:22:53.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 1 monclient: found mon.noname-g 2024-01-24T23:22:53.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:53.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:22:53.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 1 -- 172.21.15.132:0/3666158787 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d58071450 con 0x7f7d5806de30 2024-01-24T23:22:53.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 10 monclient: _finish_auth 0 2024-01-24T23:22:53.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 10 monclient: _check_auth_tickets 2024-01-24T23:22:53.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:53.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d5e367640 5 monclient: authenticate success, global_id 4710 2024-01-24T23:22:53.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d5e367640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:53.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 1 -- 172.21.15.132:0/3666158787 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7d400017c0 con 0x7f7d5806de30 2024-01-24T23:22:53.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:53.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:22:53.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 10 monclient: dump: 2024-01-24T23:22:53.290 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:53.290 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:53.290 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.290 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.290 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:53.290 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:53.290 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:53.290 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:53.291 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:53.291 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:53.291 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:53.291 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:53.291 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:53.291 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:53.291 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:53.291 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:53.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d5e367640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:53.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 1 -- 172.21.15.132:0/3666158787 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7d40001ae0 con 0x7f7d5806de30 2024-01-24T23:22:53.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:53.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d5e367640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:53.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d5e367640 4 set_mon_vals no callback set 2024-01-24T23:22:53.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 1 -- 172.21.15.132:0/3666158787 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7d40004bf0 con 0x7f7d5806de30 2024-01-24T23:22:53.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:53.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:22:53.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.281+0000 7f7d577fe640 10 monclient: dump: 2024-01-24T23:22:53.292 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:53.292 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:53.292 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.292 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.292 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:53.292 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:53.292 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:53.292 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:53.293 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:53.293 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:53.293 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:53.293 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:53.293 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:53.293 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:53.293 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:53.293 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:53.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 10 monclient: shutdown 2024-01-24T23:22:53.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:53.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 1 -- 172.21.15.132:0/3666158787 >> v1:172.21.15.132:6790/0 conn(0x7f7d5806de30 legacy=0x7f7d58139c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:53.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 1 -- 172.21.15.132:0/3666158787 shutdown_connections 2024-01-24T23:22:53.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 1 -- 172.21.15.132:0/3666158787 >> 172.21.15.132:0/3666158787 conn(0x7f7d580a2340 msgr2=0x7f7d580a2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:53.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 1 -- 172.21.15.132:0/3666158787 shutdown_connections 2024-01-24T23:22:53.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 1 -- 172.21.15.132:0/3666158787 wait complete. 2024-01-24T23:22:53.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 10 monclient: build_initial_monmap 2024-01-24T23:22:53.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 10 monclient: monmap: 2024-01-24T23:22:53.294 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:53.294 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:53.294 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:53.294 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:53.294 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:53.294 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:53.294 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:53.294 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:53.294 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:53.294 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:53.294 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:53.294 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:53.294 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:53.295 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:53.295 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:53.295 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:53.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 1 Processor -- start 2024-01-24T23:22:53.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 1 -- start start 2024-01-24T23:22:53.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 10 monclient: init 2024-01-24T23:22:53.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:53.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 10 monclient: _add_conns ranks=[0,1,8,4,5,3,7,6,2] 2024-01-24T23:22:53.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 10 monclient(hunting): picked mon.noname-a con 0x7f7d5806de30 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:53.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 10 monclient(hunting): picked mon.noname-f con 0x7f7d58134cc0 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:53.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 10 monclient(hunting): picked mon.noname-e con 0x7f7d5813b3a0 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:53.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d5806e2d0 con 0x7f7d5806de30 2024-01-24T23:22:53.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d58137170 con 0x7f7d58134cc0 2024-01-24T23:22:53.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d5813a680 con 0x7f7d5813b3a0 2024-01-24T23:22:53.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:53.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 10 monclient(hunting): authenticate will time out at 1060.613903s 2024-01-24T23:22:53.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5c904640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f7d5813b3a0 legacy=0x7f7d581d1e90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:53.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d4ffff640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f7d58134cc0 0x7f7d581ce760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:38054/0 (socket says 172.21.15.132:38054) 2024-01-24T23:22:53.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d4ffff640 1 -- 172.21.15.132:0/1293217732 learned_addr learned my addr 172.21.15.132:0/1293217732 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:53.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 1 -- 172.21.15.132:0/1293217732 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874489316 0 0) 0x7f7d58137170 con 0x7f7d58134cc0 2024-01-24T23:22:53.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 10 monclient(hunting): _negotiate 2024-01-24T23:22:53.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:53.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:53.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 10 monclient(hunting): my global_id is 4807 2024-01-24T23:22:53.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 1 -- 172.21.15.132:0/1293217732 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d38003530 con 0x7f7d58134cc0 2024-01-24T23:22:53.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 1 -- 172.21.15.132:0/1293217732 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1092717758 0 0) 0x7f7d38003530 con 0x7f7d58134cc0 2024-01-24T23:22:53.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 1 -- 172.21.15.132:0/1293217732 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7d58137170 con 0x7f7d58134cc0 2024-01-24T23:22:53.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 1 -- 172.21.15.132:0/1293217732 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7d40001780 con 0x7f7d58134cc0 2024-01-24T23:22:53.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:53.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:22:53.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 10 monclient(hunting): dump: 2024-01-24T23:22:53.300 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:53.300 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:53.300 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.300 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.300 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:53.300 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:53.300 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:53.300 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:53.301 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:53.301 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:53.301 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:53.301 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:53.301 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:53.301 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:53.301 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:53.301 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:53.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:53.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 1 -- 172.21.15.132:0/1293217732 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 56305886 0 0) 0x7f7d5806e2d0 con 0x7f7d5806de30 2024-01-24T23:22:53.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 10 monclient(hunting): _negotiate 2024-01-24T23:22:53.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:53.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:53.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 10 monclient(hunting): my global_id is 4770 2024-01-24T23:22:53.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 1 -- 172.21.15.132:0/1293217732 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d38003530 con 0x7f7d5806de30 2024-01-24T23:22:53.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 1 -- 172.21.15.132:0/1293217732 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4288993262 0 0) 0x7f7d58137170 con 0x7f7d58134cc0 2024-01-24T23:22:53.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 1 -- 172.21.15.132:0/1293217732 >> v1:172.21.15.115:6793/0 conn(0x7f7d5813b3a0 legacy=0x7f7d581d1e90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:53.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 1 -- 172.21.15.132:0/1293217732 >> v1:172.21.15.115:6789/0 conn(0x7f7d5806de30 legacy=0x7f7d581281b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:53.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 10 monclient: _finish_hunting 0 2024-01-24T23:22:53.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 1 monclient: found mon.f 2024-01-24T23:22:53.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:53.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:53.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 1 -- 172.21.15.132:0/1293217732 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d58136630 con 0x7f7d58134cc0 2024-01-24T23:22:53.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 10 monclient: _finish_auth 0 2024-01-24T23:22:53.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 10 monclient: _check_auth_tickets 2024-01-24T23:22:53.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:53.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 5 monclient: authenticate success, global_id 4807 2024-01-24T23:22:53.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 10 monclient: _renew_subs 2024-01-24T23:22:53.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:53.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 1 -- 172.21.15.132:0/1293217732 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d581319e0 con 0x7f7d58134cc0 2024-01-24T23:22:53.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 10 monclient: _renew_subs 2024-01-24T23:22:53.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:53.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 1 -- 172.21.15.132:0/1293217732 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d5812efe0 con 0x7f7d58134cc0 2024-01-24T23:22:53.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 1 -- 172.21.15.132:0/1293217732 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7d40004c50 con 0x7f7d58134cc0 2024-01-24T23:22:53.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:53.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 1 -- 172.21.15.132:0/1293217732 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7d40001c50 con 0x7f7d58134cc0 2024-01-24T23:22:53.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:53.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:22:53.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 10 monclient: dump: 2024-01-24T23:22:53.303 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:53.304 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:53.304 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.304 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.304 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:53.304 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:53.304 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:53.304 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:53.304 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:53.304 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:53.304 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:53.304 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:53.304 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:53.304 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:53.304 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:53.305 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:53.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d56ffd640 4 set_mon_vals no callback set 2024-01-24T23:22:53.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:53.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:53.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:53.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d5e367640 1 -- 172.21.15.132:0/1293217732 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d3c005090 con 0x7f7d58134cc0 2024-01-24T23:22:53.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.285+0000 7f7d557fa640 1 -- 172.21.15.132:0/1293217732 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f7d4000fd30 con 0x7f7d58134cc0 2024-01-24T23:22:53.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.289+0000 7f7d557fa640 1 -- 172.21.15.132:0/1293217732 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(35..35 src has 1..35) v4 ==== 4634+0+0 (unknown 3048104250 0 0) 0x7f7d40043c80 con 0x7f7d58134cc0 2024-01-24T23:22:53.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.289+0000 7f7d57fff640 10 monclient: get_auth_request con 0x7f7d38036f30 auth_method 0 2024-01-24T23:22:53.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.293+0000 7f7d557fa640 1 -- 172.21.15.132:0/1293217732 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f7d400149d0 con 0x7f7d58134cc0 2024-01-24T23:22:53.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.293+0000 7f7d557fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:53.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.293+0000 7f7d557fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:53.450 INFO:tasks.workunit.client.0.smithi132.stderr:foo not valid: foo not one of 'true', 'false' 2024-01-24T23:22:53.450 INFO:tasks.workunit.client.0.smithi132.stderr:Invalid command: unused arguments: ['foo'] 2024-01-24T23:22:53.450 INFO:tasks.workunit.client.0.smithi132.stderr:osd pool delete [] [--yes-i-really-really-mean-it] [--yes-i-really-really-mean-it-not-faking] : delete pool 2024-01-24T23:22:53.450 INFO:tasks.workunit.client.0.smithi132.stderr:Error EINVAL: invalid command 2024-01-24T23:22:53.452 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:22:53.452 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool delete foooo foooo --yes-i-really-really-mean-it 2024-01-24T23:22:53.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.569+0000 7f44286c7640 10 monclient: get_monmap_and_config 2024-01-24T23:22:53.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.569+0000 7f44286c7640 10 monclient: build_initial_monmap 2024-01-24T23:22:53.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.569+0000 7f44286c7640 10 monclient: monmap: 2024-01-24T23:22:53.573 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:53.573 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:53.573 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:53.574309+0000 2024-01-24T23:22:53.573 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:53.574309+0000 2024-01-24T23:22:53.573 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:53.573 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:53.573 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:53.573 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:53.573 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:53.573 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:53.573 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:53.573 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:53.573 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:53.574 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:53.574 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:53.574 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:53.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.569+0000 7f44286c7640 1 Processor -- start 2024-01-24T23:22:53.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.569+0000 7f44286c7640 1 -- start start 2024-01-24T23:22:53.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.569+0000 7f44286c7640 10 monclient: init 2024-01-24T23:22:53.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.569+0000 7f44286c7640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:53.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.569+0000 7f44286c7640 10 monclient: _add_conns ranks=[5,7,3,6,2,1,8,0,4] 2024-01-24T23:22:53.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.569+0000 7f44286c7640 10 monclient(hunting): picked mon.noname-h con 0x7f44200a0ad0 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:53.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.569+0000 7f44286c7640 10 monclient(hunting): picked mon.noname-i con 0x7f442009ee80 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:53.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.569+0000 7f44286c7640 10 monclient(hunting): picked mon.noname-g con 0x7f442009b060 addr v1:172.21.15.132:6790/0 2024-01-24T23:22:53.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.569+0000 7f44286c7640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4420064700 con 0x7f442009b060 2024-01-24T23:22:53.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.569+0000 7f44286c7640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f442009cb80 con 0x7f44200a0ad0 2024-01-24T23:22:53.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.569+0000 7f44286c7640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44200a0eb0 con 0x7f442009ee80 2024-01-24T23:22:53.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.569+0000 7f44286c7640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:53.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.569+0000 7f44286c7640 10 monclient(hunting): authenticate will time out at 1060.899435s 2024-01-24T23:22:53.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.569+0000 7f4426c64640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f442009b060 0x7f442013c270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:48816/0 (socket says 172.21.15.132:48816) 2024-01-24T23:22:53.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.569+0000 7f4426c64640 1 -- 172.21.15.132:0/3305374847 learned_addr learned my addr 172.21.15.132:0/3305374847 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:53.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.569+0000 7f4425461640 1 -- 172.21.15.132:0/3305374847 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 229213133 0 0) 0x7f44200a0eb0 con 0x7f442009ee80 2024-01-24T23:22:53.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.569+0000 7f4425461640 10 monclient(hunting): _negotiate 2024-01-24T23:22:53.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.569+0000 7f4425461640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:53.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.569+0000 7f4425461640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:53.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.569+0000 7f4425461640 10 monclient(hunting): my global_id is 4786 2024-01-24T23:22:53.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 1 -- 172.21.15.132:0/3305374847 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f440c003530 con 0x7f442009ee80 2024-01-24T23:22:53.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 1 -- 172.21.15.132:0/3305374847 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466500279 0 0) 0x7f4420064700 con 0x7f442009b060 2024-01-24T23:22:53.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 10 monclient(hunting): _negotiate 2024-01-24T23:22:53.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:53.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:53.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 10 monclient(hunting): my global_id is 4719 2024-01-24T23:22:53.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 1 -- 172.21.15.132:0/3305374847 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44200a0eb0 con 0x7f442009b060 2024-01-24T23:22:53.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 1 -- 172.21.15.132:0/3305374847 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2880895492 0 0) 0x7f442009cb80 con 0x7f44200a0ad0 2024-01-24T23:22:53.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 10 monclient(hunting): _negotiate 2024-01-24T23:22:53.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:53.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:53.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 10 monclient(hunting): my global_id is 4784 2024-01-24T23:22:53.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 1 -- 172.21.15.132:0/3305374847 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4420064700 con 0x7f44200a0ad0 2024-01-24T23:22:53.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 1 -- 172.21.15.132:0/3305374847 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3345730130 0 0) 0x7f440c003530 con 0x7f442009ee80 2024-01-24T23:22:53.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 1 -- 172.21.15.132:0/3305374847 >> v1:172.21.15.132:6791/0 conn(0x7f44200a0ad0 legacy=0x7f442009a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:53.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 1 -- 172.21.15.132:0/3305374847 >> v1:172.21.15.132:6790/0 conn(0x7f442009b060 legacy=0x7f442013c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:53.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 10 monclient: _finish_hunting 0 2024-01-24T23:22:53.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 1 monclient: found mon.noname-i 2024-01-24T23:22:53.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:53.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:22:53.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 1 -- 172.21.15.132:0/3305374847 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f442009f280 con 0x7f442009ee80 2024-01-24T23:22:53.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 10 monclient: _finish_auth 0 2024-01-24T23:22:53.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 10 monclient: _check_auth_tickets 2024-01-24T23:22:53.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:53.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 1 -- 172.21.15.132:0/3305374847 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f44100017c0 con 0x7f442009ee80 2024-01-24T23:22:53.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 5 monclient: authenticate success, global_id 4786 2024-01-24T23:22:53.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:53.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:53.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:22:53.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 10 monclient: dump: 2024-01-24T23:22:53.583 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:53.584 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:53.584 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.584 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.584 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:53.584 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:53.584 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:53.584 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:53.584 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:53.584 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:53.584 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:53.584 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:53.584 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:53.584 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:53.584 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:53.585 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:53.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:53.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 1 -- 172.21.15.132:0/3305374847 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f44100019b0 con 0x7f442009ee80 2024-01-24T23:22:53.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:53.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 1 -- 172.21.15.132:0/3305374847 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4410003960 con 0x7f442009ee80 2024-01-24T23:22:53.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:53.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:22:53.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425461640 10 monclient: dump: 2024-01-24T23:22:53.585 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:53.585 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:53.585 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.585 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.585 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:53.585 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:53.585 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:53.586 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:53.586 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:53.586 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:53.586 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:53.586 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:53.586 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:53.586 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:53.586 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:53.586 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:53.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:53.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 4 set_mon_vals no callback set 2024-01-24T23:22:53.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 10 monclient: shutdown 2024-01-24T23:22:53.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:53.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 1 -- 172.21.15.132:0/3305374847 >> v1:172.21.15.132:6792/0 conn(0x7f442009ee80 legacy=0x7f442009a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:53.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 1 -- 172.21.15.132:0/3305374847 shutdown_connections 2024-01-24T23:22:53.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 1 -- 172.21.15.132:0/3305374847 >> 172.21.15.132:0/3305374847 conn(0x7f44200a1bc0 msgr2=0x7f44200a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:53.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 1 -- 172.21.15.132:0/3305374847 shutdown_connections 2024-01-24T23:22:53.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 1 -- 172.21.15.132:0/3305374847 wait complete. 2024-01-24T23:22:53.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 10 monclient: build_initial_monmap 2024-01-24T23:22:53.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 10 monclient: monmap: 2024-01-24T23:22:53.587 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:53.587 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:53.587 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:53.587 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:53.587 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:53.587 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:53.587 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:53.587 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:53.587 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:53.587 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:53.587 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:53.588 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:53.588 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:53.588 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:53.588 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:53.588 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:53.588 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 1 Processor -- start 2024-01-24T23:22:53.588 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 1 -- start start 2024-01-24T23:22:53.588 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 10 monclient: init 2024-01-24T23:22:53.588 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:53.588 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 10 monclient: _add_conns ranks=[7,2,4,8,6,5,1,0,3] 2024-01-24T23:22:53.588 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 10 monclient(hunting): picked mon.noname-i con 0x7f442009b060 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:53.588 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 10 monclient(hunting): picked mon.noname-b con 0x7f442009ee80 addr v1:172.21.15.115:6790/0 2024-01-24T23:22:53.588 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 10 monclient(hunting): picked mon.noname-c con 0x7f44200a0ad0 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:53.588 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f442009e640 con 0x7f442009ee80 2024-01-24T23:22:53.588 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4420099fa0 con 0x7f44200a0ad0 2024-01-24T23:22:53.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4420097570 con 0x7f442009b060 2024-01-24T23:22:53.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:53.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 10 monclient(hunting): authenticate will time out at 1060.902759s 2024-01-24T23:22:53.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4426463640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f442009b060 0x7f4420099890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:39884/0 (socket says 172.21.15.132:39884) 2024-01-24T23:22:53.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4426463640 1 -- 172.21.15.132:0/4098931736 learned_addr learned my addr 172.21.15.132:0/4098931736 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:53.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4425c62640 1 -- 172.21.15.132:0/4098931736 >> v1:172.21.15.115:6790/0 conn(0x7f442009ee80 legacy=0x7f44201d0b40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:22:53.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 1 -- 172.21.15.132:0/4098931736 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1939569468 0 0) 0x7f4420097570 con 0x7f442009b060 2024-01-24T23:22:53.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:22:53.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:53.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:53.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 10 monclient(hunting): my global_id is 4795 2024-01-24T23:22:53.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 1 -- 172.21.15.132:0/4098931736 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4404003530 con 0x7f442009b060 2024-01-24T23:22:53.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 1 -- 172.21.15.132:0/4098931736 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 156768706 0 0) 0x7f4404003530 con 0x7f442009b060 2024-01-24T23:22:53.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 1 -- 172.21.15.132:0/4098931736 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4420097570 con 0x7f442009b060 2024-01-24T23:22:53.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 1 -- 172.21.15.132:0/4098931736 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f441c001c80 con 0x7f442009b060 2024-01-24T23:22:53.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:53.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:22:53.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 10 monclient(hunting): dump: 2024-01-24T23:22:53.590 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:53.590 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:53.590 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.590 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.590 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:53.590 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:53.590 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:53.590 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:53.590 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:53.590 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:53.590 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:53.591 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:53.591 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:53.591 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:53.591 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:53.591 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:53.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:53.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 1 -- 172.21.15.132:0/4098931736 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3940875830 0 0) 0x7f4420097570 con 0x7f442009b060 2024-01-24T23:22:53.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 1 -- 172.21.15.132:0/4098931736 >> v1:172.21.15.115:6791/0 conn(0x7f44200a0ad0 legacy=0x7f44201d4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:53.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 1 -- 172.21.15.132:0/4098931736 >> v1:172.21.15.115:6790/0 conn(0x7f442009ee80 legacy=0x7f44201d0b40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:53.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:22:53.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 1 monclient: found mon.i 2024-01-24T23:22:53.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:53.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:53.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 1 -- 172.21.15.132:0/4098931736 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f442006ea00 con 0x7f442009b060 2024-01-24T23:22:53.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 10 monclient: _finish_auth 0 2024-01-24T23:22:53.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:22:53.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:53.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 5 monclient: authenticate success, global_id 4795 2024-01-24T23:22:53.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 10 monclient: _renew_subs 2024-01-24T23:22:53.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:53.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 1 -- 172.21.15.132:0/4098931736 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4420098080 con 0x7f442009b060 2024-01-24T23:22:53.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 10 monclient: _renew_subs 2024-01-24T23:22:53.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:53.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f44286c7640 1 -- 172.21.15.132:0/4098931736 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4420096b60 con 0x7f442009b060 2024-01-24T23:22:53.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 1 -- 172.21.15.132:0/4098931736 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f441c002050 con 0x7f442009b060 2024-01-24T23:22:53.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:53.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 1 -- 172.21.15.132:0/4098931736 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f441c0051b0 con 0x7f442009b060 2024-01-24T23:22:53.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:53.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4424c60640 4 set_mon_vals no callback set 2024-01-24T23:22:53.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:22:53.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.573+0000 7f4416ffd640 10 monclient: dump: 2024-01-24T23:22:53.593 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:53.593 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:53.593 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.593 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:53.593 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:53.593 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:53.593 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:53.593 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:53.593 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:53.593 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:53.593 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:53.593 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:53.593 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:53.593 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:53.593 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:53.594 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:53.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.577+0000 7f44286c7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:53.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.577+0000 7f44286c7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:53.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.577+0000 7f44286c7640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:53.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.577+0000 7f44286c7640 1 -- 172.21.15.132:0/4098931736 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4408005090 con 0x7f442009b060 2024-01-24T23:22:53.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.577+0000 7f4416ffd640 1 -- 172.21.15.132:0/4098931736 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f441c00fc30 con 0x7f442009b060 2024-01-24T23:22:53.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.577+0000 7f4416ffd640 1 -- 172.21.15.132:0/4098931736 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(35..35 src has 1..35) v4 ==== 4634+0+0 (unknown 3048104250 0 0) 0x7f441c043e30 con 0x7f442009b060 2024-01-24T23:22:53.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.577+0000 7f4425c62640 10 monclient: get_auth_request con 0x7f4404036fb0 auth_method 0 2024-01-24T23:22:53.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.581+0000 7f4416ffd640 1 -- 172.21.15.132:0/4098931736 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f441c0116a0 con 0x7f442009b060 2024-01-24T23:22:53.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.581+0000 7f4416ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:53.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.581+0000 7f4416ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:53.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.733+0000 7f44286c7640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:53.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.733+0000 7f44286c7640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:53.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.733+0000 7f44286c7640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:53.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.733+0000 7f44286c7640 1 -- 172.21.15.132:0/4098931736 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f44080058c0 con 0x7f442009b060 2024-01-24T23:22:53.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.961+0000 7f4416ffd640 1 -- 172.21.15.132:0/4098931736 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}]=0 pool 'foooo' removed v36) v1 ==== 155+0+0 (unknown 3709892992 0 0) 0x7f441c0131b0 con 0x7f442009b060 2024-01-24T23:22:54.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.961+0000 7f4416ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:54.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:53.961+0000 7f4416ffd640 10 monclient: _finish_command 2 = system:0 pool 'foooo' removed 2024-01-24T23:22:54.087 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.085+0000 7f44286c7640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:54.088 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.085+0000 7f44286c7640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:54.088 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.085+0000 7f44286c7640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:54.088 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.085+0000 7f44286c7640 1 -- 172.21.15.132:0/4098931736 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f44080053f0 con 0x7f442009b060 2024-01-24T23:22:54.090 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.085+0000 7f4416ffd640 1 -- 172.21.15.132:0/4098931736 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}]=0 pool 'foooo' does not exist v36) v1 ==== 162+0+0 (unknown 936761669 0 0) 0x7f44080053f0 con 0x7f442009b060 2024-01-24T23:22:54.090 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.085+0000 7f4416ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:54.090 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.085+0000 7f4416ffd640 10 monclient: _finish_command 3 = system:0 pool 'foooo' does not exist 2024-01-24T23:22:54.090 INFO:tasks.workunit.client.0.smithi132.stderr:pool 'foooo' does not exist 2024-01-24T23:22:54.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.089+0000 7f44286c7640 1 -- 172.21.15.132:0/4098931736 >> v1:172.21.15.132:6812/1835401922 conn(0x7f4404036fb0 legacy=0x7f4404039470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:54.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.089+0000 7f44286c7640 10 monclient: shutdown 2024-01-24T23:22:54.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.089+0000 7f44286c7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:54.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.089+0000 7f44286c7640 1 -- 172.21.15.132:0/4098931736 >> v1:172.21.15.132:6792/0 conn(0x7f442009b060 legacy=0x7f4420099890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:54.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.089+0000 7f44286c7640 1 -- 172.21.15.132:0/4098931736 shutdown_connections 2024-01-24T23:22:54.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.089+0000 7f44286c7640 1 -- 172.21.15.132:0/4098931736 >> 172.21.15.132:0/4098931736 conn(0x7f44200a1bc0 msgr2=0x7f442009d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:54.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.089+0000 7f44286c7640 1 -- 172.21.15.132:0/4098931736 shutdown_connections 2024-01-24T23:22:54.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.089+0000 7f44286c7640 1 -- 172.21.15.132:0/4098931736 wait complete. 2024-01-24T23:22:54.105 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool delete fooo fooo --yes-i-really-really-mean-it 2024-01-24T23:22:54.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.221+0000 7f774c2f7640 10 monclient: get_monmap_and_config 2024-01-24T23:22:54.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.221+0000 7f774c2f7640 10 monclient: build_initial_monmap 2024-01-24T23:22:54.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.221+0000 7f774c2f7640 10 monclient: monmap: 2024-01-24T23:22:54.227 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:54.227 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:54.227 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:54.228485+0000 2024-01-24T23:22:54.227 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:54.228485+0000 2024-01-24T23:22:54.227 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:54.227 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:54.227 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:54.227 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:54.227 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:54.227 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:54.227 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:54.227 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:54.227 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:54.227 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:54.227 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:54.228 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:54.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 1 Processor -- start 2024-01-24T23:22:54.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 1 -- start start 2024-01-24T23:22:54.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 10 monclient: init 2024-01-24T23:22:54.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:54.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 10 monclient: _add_conns ranks=[3,0,6,5,4,7,2,8,1] 2024-01-24T23:22:54.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 10 monclient(hunting): picked mon.noname-g con 0x7f77440a0ad0 addr v1:172.21.15.132:6790/0 2024-01-24T23:22:54.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 10 monclient(hunting): picked mon.noname-a con 0x7f774409ee80 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:54.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 10 monclient(hunting): picked mon.noname-d con 0x7f774409b060 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:54.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7744064700 con 0x7f774409ee80 2024-01-24T23:22:54.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f774409cb80 con 0x7f77440a0ad0 2024-01-24T23:22:54.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77440a0eb0 con 0x7f774409b060 2024-01-24T23:22:54.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:54.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 10 monclient(hunting): authenticate will time out at 1061.553053s 2024-01-24T23:22:54.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774a093640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f77440a0ad0 0x7f774409a240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:59866/0 (socket says 172.21.15.132:59866) 2024-01-24T23:22:54.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774a093640 1 -- 172.21.15.132:0/3723928594 learned_addr learned my addr 172.21.15.132:0/3723928594 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:54.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 1 -- 172.21.15.132:0/3723928594 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724124838 0 0) 0x7f774409cb80 con 0x7f77440a0ad0 2024-01-24T23:22:54.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 10 monclient(hunting): _negotiate 2024-01-24T23:22:54.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:54.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:54.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 10 monclient(hunting): my global_id is 4728 2024-01-24T23:22:54.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 1 -- 172.21.15.132:0/3723928594 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7730003530 con 0x7f77440a0ad0 2024-01-24T23:22:54.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 1 -- 172.21.15.132:0/3723928594 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1799937200 0 0) 0x7f7730003530 con 0x7f77440a0ad0 2024-01-24T23:22:54.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 1 -- 172.21.15.132:0/3723928594 >> v1:172.21.15.115:6792/0 conn(0x7f774409b060 legacy=0x7f774413c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:54.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 1 -- 172.21.15.132:0/3723928594 >> v1:172.21.15.115:6789/0 conn(0x7f774409ee80 legacy=0x7f774409a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:54.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 10 monclient: _finish_hunting 0 2024-01-24T23:22:54.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 1 monclient: found mon.noname-g 2024-01-24T23:22:54.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:54.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:22:54.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 1 -- 172.21.15.132:0/3723928594 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f774409f280 con 0x7f77440a0ad0 2024-01-24T23:22:54.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 10 monclient: _finish_auth 0 2024-01-24T23:22:54.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 10 monclient: _check_auth_tickets 2024-01-24T23:22:54.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:54.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 1 -- 172.21.15.132:0/3723928594 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f77400017c0 con 0x7f77440a0ad0 2024-01-24T23:22:54.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 5 monclient: authenticate success, global_id 4728 2024-01-24T23:22:54.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:54.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:22:54.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 10 monclient: dump: 2024-01-24T23:22:54.235 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:54.235 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:54.235 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:54.235 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:54.235 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:54.235 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:54.236 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:54.236 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:54.236 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:54.236 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:54.236 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:54.236 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:54.236 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:54.236 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:54.236 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:54.236 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:54.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:54.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 1 -- 172.21.15.132:0/3723928594 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7740001a70 con 0x7f77440a0ad0 2024-01-24T23:22:54.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:54.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 1 -- 172.21.15.132:0/3723928594 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7740004bf0 con 0x7f77440a0ad0 2024-01-24T23:22:54.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:54.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 4 set_mon_vals no callback set 2024-01-24T23:22:54.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:54.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:22:54.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f7749091640 10 monclient: dump: 2024-01-24T23:22:54.237 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:54.237 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:54.237 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:54.237 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:54.237 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:54.237 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:54.237 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:54.237 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:54.237 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:54.237 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:54.237 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:54.238 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:54.238 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:54.238 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:54.238 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:54.238 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:54.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 10 monclient: shutdown 2024-01-24T23:22:54.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:54.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 1 -- 172.21.15.132:0/3723928594 >> v1:172.21.15.132:6790/0 conn(0x7f77440a0ad0 legacy=0x7f774409a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:54.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 1 -- 172.21.15.132:0/3723928594 shutdown_connections 2024-01-24T23:22:54.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 1 -- 172.21.15.132:0/3723928594 >> 172.21.15.132:0/3723928594 conn(0x7f77440a1bc0 msgr2=0x7f77440a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:54.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 1 -- 172.21.15.132:0/3723928594 shutdown_connections 2024-01-24T23:22:54.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 1 -- 172.21.15.132:0/3723928594 wait complete. 2024-01-24T23:22:54.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 10 monclient: build_initial_monmap 2024-01-24T23:22:54.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 10 monclient: monmap: 2024-01-24T23:22:54.238 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:54.238 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:54.238 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:54.239 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:54.239 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:54.239 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:54.239 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:54.239 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:54.239 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:54.239 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:54.239 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:54.239 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:54.239 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:54.239 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:54.239 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:54.239 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:54.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 1 Processor -- start 2024-01-24T23:22:54.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 1 -- start start 2024-01-24T23:22:54.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 10 monclient: init 2024-01-24T23:22:54.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:54.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 10 monclient: _add_conns ranks=[6,1,8,0,4,2,7,3,5] 2024-01-24T23:22:54.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 10 monclient(hunting): picked mon.noname-d con 0x7f774409b060 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:54.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 10 monclient(hunting): picked mon.noname-f con 0x7f774409ee80 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:54.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 10 monclient(hunting): picked mon.noname-e con 0x7f77440a0ad0 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:54.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.225+0000 7f774c2f7640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7744094de0 con 0x7f774409ee80 2024-01-24T23:22:54.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f774c2f7640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f774409e640 con 0x7f774409b060 2024-01-24T23:22:54.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f774c2f7640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f774409b4c0 con 0x7f77440a0ad0 2024-01-24T23:22:54.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f774c2f7640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:54.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f774c2f7640 10 monclient(hunting): authenticate will time out at 1061.556031s 2024-01-24T23:22:54.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f7749892640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f774409ee80 0x7f77441d0830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:53058/0 (socket says 172.21.15.132:53058) 2024-01-24T23:22:54.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f774a894640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f77440a0ad0 legacy=0x7f77441d3f60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:54.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f7749892640 1 -- 172.21.15.132:0/1217698710 learned_addr learned my addr 172.21.15.132:0/1217698710 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:54.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 1 -- 172.21.15.132:0/1217698710 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894383552 0 0) 0x7f7744094de0 con 0x7f774409ee80 2024-01-24T23:22:54.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 10 monclient(hunting): _negotiate 2024-01-24T23:22:54.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:54.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:54.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 10 monclient(hunting): my global_id is 4816 2024-01-24T23:22:54.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 1 -- 172.21.15.132:0/1217698710 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7728003530 con 0x7f774409ee80 2024-01-24T23:22:54.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 1 -- 172.21.15.132:0/1217698710 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 808943779 0 0) 0x7f7728003530 con 0x7f774409ee80 2024-01-24T23:22:54.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 1 -- 172.21.15.132:0/1217698710 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7744094de0 con 0x7f774409ee80 2024-01-24T23:22:54.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 1 -- 172.21.15.132:0/1217698710 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7734001b00 con 0x7f774409ee80 2024-01-24T23:22:54.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:54.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:22:54.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 10 monclient(hunting): dump: 2024-01-24T23:22:54.241 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:54.241 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:54.241 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:54.241 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:54.242 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:54.242 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:54.242 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:54.242 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:54.242 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:54.242 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:54.242 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:54.242 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:54.242 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:54.242 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:54.242 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:54.242 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:54.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:54.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 1 -- 172.21.15.132:0/1217698710 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3943637230 0 0) 0x7f774409e640 con 0x7f774409b060 2024-01-24T23:22:54.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 10 monclient(hunting): _negotiate 2024-01-24T23:22:54.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:54.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:54.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 10 monclient(hunting): my global_id is 4812 2024-01-24T23:22:54.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 1 -- 172.21.15.132:0/1217698710 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7728003530 con 0x7f774409b060 2024-01-24T23:22:54.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 1 -- 172.21.15.132:0/1217698710 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 839893101 0 0) 0x7f7744094de0 con 0x7f774409ee80 2024-01-24T23:22:54.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 1 -- 172.21.15.132:0/1217698710 >> v1:172.21.15.115:6793/0 conn(0x7f77440a0ad0 legacy=0x7f77441d3f60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:54.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 1 -- 172.21.15.132:0/1217698710 >> v1:172.21.15.115:6792/0 conn(0x7f774409b060 legacy=0x7f7744098670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:54.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 10 monclient: _finish_hunting 0 2024-01-24T23:22:54.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 1 monclient: found mon.f 2024-01-24T23:22:54.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:54.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:54.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 1 -- 172.21.15.132:0/1217698710 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7744096e60 con 0x7f774409ee80 2024-01-24T23:22:54.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 10 monclient: _finish_auth 0 2024-01-24T23:22:54.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 10 monclient: _check_auth_tickets 2024-01-24T23:22:54.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:54.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f774c2f7640 5 monclient: authenticate success, global_id 4816 2024-01-24T23:22:54.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f774c2f7640 10 monclient: _renew_subs 2024-01-24T23:22:54.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f774c2f7640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:54.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f774c2f7640 1 -- 172.21.15.132:0/1217698710 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7744097140 con 0x7f774409ee80 2024-01-24T23:22:54.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f774c2f7640 10 monclient: _renew_subs 2024-01-24T23:22:54.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f774c2f7640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:54.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f774c2f7640 1 -- 172.21.15.132:0/1217698710 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7744098d80 con 0x7f774409ee80 2024-01-24T23:22:54.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 1 -- 172.21.15.132:0/1217698710 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7734003950 con 0x7f774409ee80 2024-01-24T23:22:54.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:54.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 1 -- 172.21.15.132:0/1217698710 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7734003e80 con 0x7f774409ee80 2024-01-24T23:22:54.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:54.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:22:54.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f7748890640 4 set_mon_vals no callback set 2024-01-24T23:22:54.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 10 monclient: dump: 2024-01-24T23:22:54.244 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:54.244 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:54.245 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:54.245 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:54.245 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:54.245 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:54.245 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:54.245 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:54.245 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:54.245 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:54.245 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:54.245 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:54.245 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:54.245 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:54.245 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:54.245 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:54.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f774c2f7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:54.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f774c2f7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:54.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f774c2f7640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:54.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f774c2f7640 1 -- 172.21.15.132:0/1217698710 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f772c005090 con 0x7f774409ee80 2024-01-24T23:22:54.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 1 -- 172.21.15.132:0/1217698710 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f7734003fe0 con 0x7f774409ee80 2024-01-24T23:22:54.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f773affd640 1 -- 172.21.15.132:0/1217698710 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(36..36 src has 1..36) v4 ==== 4290+0+0 (unknown 3603050433 0 0) 0x7f7734043b10 con 0x7f774409ee80 2024-01-24T23:22:54.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.229+0000 7f774a093640 10 monclient: get_auth_request con 0x7f7728036fb0 auth_method 0 2024-01-24T23:22:54.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.233+0000 7f773affd640 1 -- 172.21.15.132:0/1217698710 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f7734014940 con 0x7f774409ee80 2024-01-24T23:22:54.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.233+0000 7f773affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:54.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.233+0000 7f773affd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:54.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.385+0000 7f774c2f7640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:54.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.385+0000 7f774c2f7640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:54.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.385+0000 7f774c2f7640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:54.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.385+0000 7f774c2f7640 1 -- 172.21.15.132:0/1217698710 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f772c0058c0 con 0x7f774409ee80 2024-01-24T23:22:54.960 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.957+0000 7f773affd640 1 -- 172.21.15.132:0/1217698710 <== mon.1 v1:172.21.15.132:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}]=0 pool 'fooo' removed v37) v1 ==== 152+0+0 (unknown 3597054383 0 0) 0x7f7734012fe0 con 0x7f774409ee80 2024-01-24T23:22:54.960 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.957+0000 7f773affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:54.960 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:54.957+0000 7f773affd640 10 monclient: _finish_command 2 = system:0 pool 'fooo' removed 2024-01-24T23:22:55.098 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.093+0000 7f774c2f7640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:55.098 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.093+0000 7f774c2f7640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:55.098 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.093+0000 7f774c2f7640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:55.098 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.093+0000 7f774c2f7640 1 -- 172.21.15.132:0/1217698710 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f772c0053f0 con 0x7f774409ee80 2024-01-24T23:22:55.100 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.097+0000 7f773affd640 1 -- 172.21.15.132:0/1217698710 <== mon.1 v1:172.21.15.132:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}]=0 pool 'fooo' does not exist v37) v1 ==== 159+0+0 (unknown 4264090831 0 0) 0x7f772c0053f0 con 0x7f774409ee80 2024-01-24T23:22:55.100 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.097+0000 7f773affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:55.100 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.097+0000 7f773affd640 10 monclient: _finish_command 3 = system:0 pool 'fooo' does not exist 2024-01-24T23:22:55.100 INFO:tasks.workunit.client.0.smithi132.stderr:pool 'fooo' does not exist 2024-01-24T23:22:55.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.097+0000 7f774c2f7640 1 -- 172.21.15.132:0/1217698710 >> v1:172.21.15.132:6812/1835401922 conn(0x7f7728036fb0 legacy=0x7f7728039470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:55.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.097+0000 7f774c2f7640 10 monclient: shutdown 2024-01-24T23:22:55.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.097+0000 7f774c2f7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:55.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.097+0000 7f774c2f7640 1 -- 172.21.15.132:0/1217698710 >> v1:172.21.15.132:6789/0 conn(0x7f774409ee80 legacy=0x7f77441d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:55.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.097+0000 7f774c2f7640 1 -- 172.21.15.132:0/1217698710 shutdown_connections 2024-01-24T23:22:55.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.097+0000 7f774c2f7640 1 -- 172.21.15.132:0/1217698710 >> 172.21.15.132:0/1217698710 conn(0x7f77440a1bc0 msgr2=0x7f7744066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:55.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.097+0000 7f774c2f7640 1 -- 172.21.15.132:0/1217698710 shutdown_connections 2024-01-24T23:22:55.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.097+0000 7f774c2f7640 1 -- 172.21.15.132:0/1217698710 wait complete. 2024-01-24T23:22:55.116 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool delete foo foo --yes-i-really-really-mean-it 2024-01-24T23:22:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 10 monclient: get_monmap_and_config 2024-01-24T23:22:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 10 monclient: build_initial_monmap 2024-01-24T23:22:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 10 monclient: monmap: 2024-01-24T23:22:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:55.233120+0000 2024-01-24T23:22:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:55.233120+0000 2024-01-24T23:22:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:55.233 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:55.233 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:55.233 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:55.233 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:55.233 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:55.233 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:55.233 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:55.233 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:55.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 1 Processor -- start 2024-01-24T23:22:55.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 1 -- start start 2024-01-24T23:22:55.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 10 monclient: init 2024-01-24T23:22:55.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:55.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 10 monclient: _add_conns ranks=[5,8,1,4,0,7,3,2,6] 2024-01-24T23:22:55.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 10 monclient(hunting): picked mon.noname-h con 0x7f50b0072690 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:55.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 10 monclient(hunting): picked mon.noname-e con 0x7f50b009ae90 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:55.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 10 monclient(hunting): picked mon.noname-f con 0x7f50b009bdc0 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:55.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b0064700 con 0x7f50b009bdc0 2024-01-24T23:22:55.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b009a790 con 0x7f50b0072690 2024-01-24T23:22:55.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b00a18f0 con 0x7f50b009ae90 2024-01-24T23:22:55.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:55.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 10 monclient(hunting): authenticate will time out at 1062.557658s 2024-01-24T23:22:55.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50aed9d640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f50b009bdc0 0x7f50b013d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:53068/0 (socket says 172.21.15.132:53068) 2024-01-24T23:22:55.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ae59c640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f50b0072690 0x7f50b00a0ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:39300/0 (socket says 172.21.15.132:39300) 2024-01-24T23:22:55.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50aed9d640 1 -- 172.21.15.132:0/2382836005 learned_addr learned my addr 172.21.15.132:0/2382836005 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:55.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50add9b640 1 -- 172.21.15.132:0/2382836005 >> v1:172.21.15.115:6793/0 conn(0x7f50b009ae90 legacy=0x7f50b00a11e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:55.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 1 -- 172.21.15.132:0/2382836005 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2633982722 0 0) 0x7f50b0064700 con 0x7f50b009bdc0 2024-01-24T23:22:55.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 10 monclient(hunting): _negotiate 2024-01-24T23:22:55.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:55.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:55.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 10 monclient(hunting): my global_id is 4825 2024-01-24T23:22:55.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 1 -- 172.21.15.132:0/2382836005 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5098003530 con 0x7f50b009bdc0 2024-01-24T23:22:55.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 1 -- 172.21.15.132:0/2382836005 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3594902651 0 0) 0x7f50b009a790 con 0x7f50b0072690 2024-01-24T23:22:55.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 10 monclient(hunting): _negotiate 2024-01-24T23:22:55.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:55.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:55.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 10 monclient(hunting): my global_id is 4793 2024-01-24T23:22:55.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 1 -- 172.21.15.132:0/2382836005 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50b0064700 con 0x7f50b0072690 2024-01-24T23:22:55.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 1 -- 172.21.15.132:0/2382836005 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 138726759 0 0) 0x7f5098003530 con 0x7f50b009bdc0 2024-01-24T23:22:55.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 1 -- 172.21.15.132:0/2382836005 >> v1:172.21.15.115:6793/0 conn(0x7f50b009ae90 legacy=0x7f50b00a11e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:55.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 1 -- 172.21.15.132:0/2382836005 >> v1:172.21.15.132:6791/0 conn(0x7f50b0072690 legacy=0x7f50b00a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:55.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 10 monclient: _finish_hunting 0 2024-01-24T23:22:55.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 1 monclient: found mon.noname-f 2024-01-24T23:22:55.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:55.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:22:55.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 1 -- 172.21.15.132:0/2382836005 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b00997a0 con 0x7f50b009bdc0 2024-01-24T23:22:55.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 10 monclient: _finish_auth 0 2024-01-24T23:22:55.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 10 monclient: _check_auth_tickets 2024-01-24T23:22:55.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:55.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 1 -- 172.21.15.132:0/2382836005 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f50a80017c0 con 0x7f50b009bdc0 2024-01-24T23:22:55.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 5 monclient: authenticate success, global_id 4825 2024-01-24T23:22:55.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:55.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:55.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:22:55.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 10 monclient: dump: 2024-01-24T23:22:55.242 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:55.242 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:55.242 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:55.242 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:55.242 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:55.242 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:55.242 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:55.242 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:55.243 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 1 -- 172.21.15.132:0/2382836005 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f50a8001a30 con 0x7f50b009bdc0 2024-01-24T23:22:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 4 set_mon_vals no callback set 2024-01-24T23:22:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 1 -- 172.21.15.132:0/2382836005 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f50a8004bf0 con 0x7f50b009bdc0 2024-01-24T23:22:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:55.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:22:55.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50ad59a640 10 monclient: dump: 2024-01-24T23:22:55.244 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:55.244 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:55.244 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:55.244 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:55.244 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:55.244 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:55.244 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:55.244 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:55.244 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:55.244 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:55.244 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:55.244 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:55.244 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:55.245 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:55.245 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:55.245 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:55.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 10 monclient: shutdown 2024-01-24T23:22:55.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:55.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 1 -- 172.21.15.132:0/2382836005 >> v1:172.21.15.132:6789/0 conn(0x7f50b009bdc0 legacy=0x7f50b013d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:55.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 1 -- 172.21.15.132:0/2382836005 shutdown_connections 2024-01-24T23:22:55.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 1 -- 172.21.15.132:0/2382836005 >> 172.21.15.132:0/2382836005 conn(0x7f50b00a1bc0 msgr2=0x7f50b00a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:55.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 1 -- 172.21.15.132:0/2382836005 shutdown_connections 2024-01-24T23:22:55.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 1 -- 172.21.15.132:0/2382836005 wait complete. 2024-01-24T23:22:55.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 10 monclient: build_initial_monmap 2024-01-24T23:22:55.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.229+0000 7f50b4ca4640 10 monclient: monmap: 2024-01-24T23:22:55.245 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:55.245 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:55.245 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:55.246 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:55.246 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:55.246 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:55.246 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:55.246 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:55.246 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:55.246 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:55.246 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:55.246 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:55.246 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:55.246 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:55.246 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:55.246 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:55.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50b4ca4640 1 Processor -- start 2024-01-24T23:22:55.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50b4ca4640 1 -- start start 2024-01-24T23:22:55.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50b4ca4640 10 monclient: init 2024-01-24T23:22:55.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50b4ca4640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:55.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50b4ca4640 10 monclient: _add_conns ranks=[3,8,6,0,5,2,1,4,7] 2024-01-24T23:22:55.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50b4ca4640 10 monclient(hunting): picked mon.noname-g con 0x7f50b0072690 addr v1:172.21.15.132:6790/0 2024-01-24T23:22:55.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50b4ca4640 10 monclient(hunting): picked mon.noname-e con 0x7f50b009ae90 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:55.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50b4ca4640 10 monclient(hunting): picked mon.noname-d con 0x7f50b009bdc0 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:55.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50b4ca4640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b01cbcf0 con 0x7f50b0072690 2024-01-24T23:22:55.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50b4ca4640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b009b930 con 0x7f50b009bdc0 2024-01-24T23:22:55.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50b4ca4640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b0099470 con 0x7f50b009ae90 2024-01-24T23:22:55.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50b4ca4640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:55.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50b4ca4640 10 monclient(hunting): authenticate will time out at 1062.560756s 2024-01-24T23:22:55.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50ae59c640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f50b0072690 0x7f50b01b6510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:59880/0 (socket says 172.21.15.132:59880) 2024-01-24T23:22:55.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50ae59c640 1 -- 172.21.15.132:0/1588917969 learned_addr learned my addr 172.21.15.132:0/1588917969 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:55.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50add9b640 1 -- 172.21.15.132:0/1588917969 >> v1:172.21.15.115:6793/0 conn(0x7f50b009ae90 legacy=0x7f50b01d0580 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:55.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 1 -- 172.21.15.132:0/1588917969 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2171135819 0 0) 0x7f50b01cbcf0 con 0x7f50b0072690 2024-01-24T23:22:55.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 10 monclient(hunting): _negotiate 2024-01-24T23:22:55.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:55.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:55.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 10 monclient(hunting): my global_id is 4737 2024-01-24T23:22:55.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 1 -- 172.21.15.132:0/1588917969 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5090003530 con 0x7f50b0072690 2024-01-24T23:22:55.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 1 -- 172.21.15.132:0/1588917969 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2097952225 0 0) 0x7f5090003530 con 0x7f50b0072690 2024-01-24T23:22:55.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 1 -- 172.21.15.132:0/1588917969 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f50b01cbcf0 con 0x7f50b0072690 2024-01-24T23:22:55.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 1 -- 172.21.15.132:0/1588917969 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f50a4001a60 con 0x7f50b0072690 2024-01-24T23:22:55.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:55.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:22:55.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 10 monclient(hunting): dump: 2024-01-24T23:22:55.248 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:55.248 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:55.249 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:55.249 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:55.249 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:55.249 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:55.249 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:55.249 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:55.249 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:55.249 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:55.249 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:55.249 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:55.249 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:55.249 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:55.249 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:55.249 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:55.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:55.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 1 -- 172.21.15.132:0/1588917969 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2041041796 0 0) 0x7f50b01cbcf0 con 0x7f50b0072690 2024-01-24T23:22:55.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 1 -- 172.21.15.132:0/1588917969 >> v1:172.21.15.115:6793/0 conn(0x7f50b009ae90 legacy=0x7f50b01d0580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:55.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 1 -- 172.21.15.132:0/1588917969 >> v1:172.21.15.115:6792/0 conn(0x7f50b009bdc0 legacy=0x7f50b01d3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:55.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 10 monclient: _finish_hunting 0 2024-01-24T23:22:55.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 1 monclient: found mon.g 2024-01-24T23:22:55.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:55.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:55.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 1 -- 172.21.15.132:0/1588917969 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b0071910 con 0x7f50b0072690 2024-01-24T23:22:55.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 10 monclient: _finish_auth 0 2024-01-24T23:22:55.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 10 monclient: _check_auth_tickets 2024-01-24T23:22:55.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:55.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50b4ca4640 5 monclient: authenticate success, global_id 4737 2024-01-24T23:22:55.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50b4ca4640 10 monclient: _renew_subs 2024-01-24T23:22:55.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50b4ca4640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:55.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50b4ca4640 1 -- 172.21.15.132:0/1588917969 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b009e4c0 con 0x7f50b0072690 2024-01-24T23:22:55.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50b4ca4640 10 monclient: _renew_subs 2024-01-24T23:22:55.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50b4ca4640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:55.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50b4ca4640 1 -- 172.21.15.132:0/1588917969 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b01b6c20 con 0x7f50b0072690 2024-01-24T23:22:55.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 1 -- 172.21.15.132:0/1588917969 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f50a4004aa0 con 0x7f50b0072690 2024-01-24T23:22:55.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:55.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 1 -- 172.21.15.132:0/1588917969 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f50a4004fd0 con 0x7f50b0072690 2024-01-24T23:22:55.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:55.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:22:55.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50acd99640 4 set_mon_vals no callback set 2024-01-24T23:22:55.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 10 monclient: dump: 2024-01-24T23:22:55.251 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:55.251 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:55.251 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:55.251 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:55.252 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50b4ca4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50b4ca4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50b4ca4640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:55.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50b4ca4640 1 -- 172.21.15.132:0/1588917969 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5094005090 con 0x7f50b0072690 2024-01-24T23:22:55.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 1 -- 172.21.15.132:0/1588917969 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f50a40051b0 con 0x7f50b0072690 2024-01-24T23:22:55.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f509effd640 1 -- 172.21.15.132:0/1588917969 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(37..37 src has 1..37) v4 ==== 3940+0+0 (unknown 2559143714 0 0) 0x7f50a40439c0 con 0x7f50b0072690 2024-01-24T23:22:55.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.233+0000 7f50add9b640 10 monclient: get_auth_request con 0x7f5090036f60 auth_method 0 2024-01-24T23:22:55.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.237+0000 7f509effd640 1 -- 172.21.15.132:0/1588917969 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f50a4011580 con 0x7f50b0072690 2024-01-24T23:22:55.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.237+0000 7f509effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:55.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.237+0000 7f509effd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:55.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.393+0000 7f50b4ca4640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:55.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.393+0000 7f50b4ca4640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:55.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.393+0000 7f50b4ca4640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:55.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.393+0000 7f50b4ca4640 1 -- 172.21.15.132:0/1588917969 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f50940058c0 con 0x7f50b0072690 2024-01-24T23:22:55.999 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.997+0000 7f509effd640 1 -- 172.21.15.132:0/1588917969 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}]=0 pool 'foo' removed v38) v1 ==== 149+0+0 (unknown 422414212 0 0) 0x7f50a4001d50 con 0x7f50b0072690 2024-01-24T23:22:56.000 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.997+0000 7f509effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:56.000 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:55.997+0000 7f509effd640 10 monclient: _finish_command 2 = system:0 pool 'foo' removed 2024-01-24T23:22:56.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.121+0000 7f50b4ca4640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:56.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.121+0000 7f50b4ca4640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:56.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.121+0000 7f50b4ca4640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:22:56.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.121+0000 7f50b4ca4640 1 -- 172.21.15.132:0/1588917969 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f50940053f0 con 0x7f50b0072690 2024-01-24T23:22:56.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.121+0000 7f509effd640 1 -- 172.21.15.132:0/1588917969 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}]=0 pool 'foo' does not exist v38) v1 ==== 156+0+0 (unknown 495831957 0 0) 0x7f50940053f0 con 0x7f50b0072690 2024-01-24T23:22:56.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.121+0000 7f509effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:56.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.121+0000 7f509effd640 10 monclient: _finish_command 3 = system:0 pool 'foo' does not exist 2024-01-24T23:22:56.127 INFO:tasks.workunit.client.0.smithi132.stderr:pool 'foo' does not exist 2024-01-24T23:22:56.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.125+0000 7f50b4ca4640 1 -- 172.21.15.132:0/1588917969 >> v1:172.21.15.132:6812/1835401922 conn(0x7f5090036f60 legacy=0x7f5090039420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:56.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.125+0000 7f50b4ca4640 10 monclient: shutdown 2024-01-24T23:22:56.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.125+0000 7f50b4ca4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:56.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.125+0000 7f50b4ca4640 1 -- 172.21.15.132:0/1588917969 >> v1:172.21.15.132:6790/0 conn(0x7f50b0072690 legacy=0x7f50b01b6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:56.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.125+0000 7f50b4ca4640 1 -- 172.21.15.132:0/1588917969 shutdown_connections 2024-01-24T23:22:56.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.125+0000 7f50b4ca4640 1 -- 172.21.15.132:0/1588917969 >> 172.21.15.132:0/1588917969 conn(0x7f50b00a1bc0 msgr2=0x7f50b00a3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:56.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.125+0000 7f50b4ca4640 1 -- 172.21.15.132:0/1588917969 shutdown_connections 2024-01-24T23:22:56.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.125+0000 7f50b4ca4640 1 -- 172.21.15.132:0/1588917969 wait complete. 2024-01-24T23:22:56.143 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool delete foo foo --yes-i-really-really-mean-it 2024-01-24T23:22:56.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.257+0000 7f186fe2f640 10 monclient: get_monmap_and_config 2024-01-24T23:22:56.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.257+0000 7f186fe2f640 10 monclient: build_initial_monmap 2024-01-24T23:22:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.257+0000 7f186fe2f640 10 monclient: monmap: 2024-01-24T23:22:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:56.262378+0000 2024-01-24T23:22:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:56.262378+0000 2024-01-24T23:22:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:56.262 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:56.262 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:56.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.257+0000 7f186fe2f640 1 Processor -- start 2024-01-24T23:22:56.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.257+0000 7f186fe2f640 1 -- start start 2024-01-24T23:22:56.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.257+0000 7f186fe2f640 10 monclient: init 2024-01-24T23:22:56.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.257+0000 7f186fe2f640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:56.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.257+0000 7f186fe2f640 10 monclient: _add_conns ranks=[7,5,0,1,8,6,4,2,3] 2024-01-24T23:22:56.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.257+0000 7f186fe2f640 10 monclient(hunting): picked mon.noname-i con 0x7f1868071880 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:56.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.257+0000 7f186fe2f640 10 monclient(hunting): picked mon.noname-h con 0x7f1868136fa0 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:56.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.257+0000 7f186fe2f640 10 monclient(hunting): picked mon.noname-a con 0x7f186813d580 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:56.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.257+0000 7f186fe2f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1868064700 con 0x7f186813d580 2024-01-24T23:22:56.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.257+0000 7f186fe2f640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f186806e070 con 0x7f1868136fa0 2024-01-24T23:22:56.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.257+0000 7f186fe2f640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f186806d7d0 con 0x7f1868071880 2024-01-24T23:22:56.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.257+0000 7f186fe2f640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.257+0000 7f186fe2f640 10 monclient(hunting): authenticate will time out at 1063.587660s 2024-01-24T23:22:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.257+0000 7f186dbcb640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f1868071880 0x7f1868071c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:38852/0 (socket says 172.21.15.132:38852) 2024-01-24T23:22:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.257+0000 7f186dbcb640 1 -- 172.21.15.132:0/648997833 learned_addr learned my addr 172.21.15.132:0/648997833 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 1 -- 172.21.15.132:0/648997833 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 560369790 0 0) 0x7f186806d7d0 con 0x7f1868071880 2024-01-24T23:22:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 10 monclient(hunting): _negotiate 2024-01-24T23:22:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 10 monclient(hunting): my global_id is 4804 2024-01-24T23:22:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 1 -- 172.21.15.132:0/648997833 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1854003530 con 0x7f1868071880 2024-01-24T23:22:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 1 -- 172.21.15.132:0/648997833 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1525043436 0 0) 0x7f186806e070 con 0x7f1868136fa0 2024-01-24T23:22:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 10 monclient(hunting): _negotiate 2024-01-24T23:22:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 10 monclient(hunting): my global_id is 4802 2024-01-24T23:22:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 1 -- 172.21.15.132:0/648997833 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f186806d7d0 con 0x7f1868136fa0 2024-01-24T23:22:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 1 -- 172.21.15.132:0/648997833 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3704436551 0 0) 0x7f1854003530 con 0x7f1868071880 2024-01-24T23:22:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 1 -- 172.21.15.132:0/648997833 >> v1:172.21.15.132:6791/0 conn(0x7f1868136fa0 legacy=0x7f186813be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 1 -- 172.21.15.132:0/648997833 >> v1:172.21.15.115:6789/0 conn(0x7f186813d580 legacy=0x7f186813f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 10 monclient: _finish_hunting 0 2024-01-24T23:22:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 1 monclient: found mon.noname-i 2024-01-24T23:22:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:22:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 1 -- 172.21.15.132:0/648997833 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186806dab0 con 0x7f1868071880 2024-01-24T23:22:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 10 monclient: _finish_auth 0 2024-01-24T23:22:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 10 monclient: _check_auth_tickets 2024-01-24T23:22:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 1 -- 172.21.15.132:0/648997833 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f18640017c0 con 0x7f1868071880 2024-01-24T23:22:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:22:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 10 monclient: dump: 2024-01-24T23:22:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:56.270 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 5 monclient: authenticate success, global_id 4804 2024-01-24T23:22:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 1 -- 172.21.15.132:0/648997833 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1864001aa0 con 0x7f1868071880 2024-01-24T23:22:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 1 -- 172.21.15.132:0/648997833 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1864004bf0 con 0x7f1868071880 2024-01-24T23:22:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 4 set_mon_vals no callback set 2024-01-24T23:22:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:22:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186cbc9640 10 monclient: dump: 2024-01-24T23:22:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:56.271 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 10 monclient: shutdown 2024-01-24T23:22:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 1 -- 172.21.15.132:0/648997833 >> v1:172.21.15.132:6792/0 conn(0x7f1868071880 legacy=0x7f1868071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 1 -- 172.21.15.132:0/648997833 shutdown_connections 2024-01-24T23:22:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 1 -- 172.21.15.132:0/648997833 >> 172.21.15.132:0/648997833 conn(0x7f186806aac0 msgr2=0x7f186806aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 1 -- 172.21.15.132:0/648997833 shutdown_connections 2024-01-24T23:22:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 1 -- 172.21.15.132:0/648997833 wait complete. 2024-01-24T23:22:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 10 monclient: build_initial_monmap 2024-01-24T23:22:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 10 monclient: monmap: 2024-01-24T23:22:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:56.273 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 1 Processor -- start 2024-01-24T23:22:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 1 -- start start 2024-01-24T23:22:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 10 monclient: init 2024-01-24T23:22:56.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:56.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 10 monclient: _add_conns ranks=[6,3,1,4,8,0,5,7,2] 2024-01-24T23:22:56.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 10 monclient(hunting): picked mon.noname-d con 0x7f1868071880 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:56.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 10 monclient(hunting): picked mon.noname-g con 0x7f1868136fa0 addr v1:172.21.15.132:6790/0 2024-01-24T23:22:56.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 10 monclient(hunting): picked mon.noname-f con 0x7f186813d580 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:56.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1868132d80 con 0x7f186813d580 2024-01-24T23:22:56.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18680703d0 con 0x7f1868136fa0 2024-01-24T23:22:56.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18680712d0 con 0x7f1868071880 2024-01-24T23:22:56.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:56.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186fe2f640 10 monclient(hunting): authenticate will time out at 1063.591509s 2024-01-24T23:22:56.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186d3ca640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f1868136fa0 0x7f186806fcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:59896/0 (socket says 172.21.15.132:59896) 2024-01-24T23:22:56.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f186d3ca640 1 -- 172.21.15.132:0/947099146 learned_addr learned my addr 172.21.15.132:0/947099146 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:56.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f185e7fc640 1 -- 172.21.15.132:0/947099146 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2612245560 0 0) 0x7f18680703d0 con 0x7f1868136fa0 2024-01-24T23:22:56.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f185e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:56.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f185e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:56.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f185e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:56.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f185e7fc640 10 monclient(hunting): my global_id is 4746 2024-01-24T23:22:56.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f185e7fc640 1 -- 172.21.15.132:0/947099146 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f184c003530 con 0x7f1868136fa0 2024-01-24T23:22:56.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f185e7fc640 1 -- 172.21.15.132:0/947099146 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1852668007 0 0) 0x7f1868132d80 con 0x7f186813d580 2024-01-24T23:22:56.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.261+0000 7f185e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:56.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:56.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:56.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 10 monclient(hunting): my global_id is 4834 2024-01-24T23:22:56.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 1 -- 172.21.15.132:0/947099146 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18680703d0 con 0x7f186813d580 2024-01-24T23:22:56.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 1 -- 172.21.15.132:0/947099146 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1799436036 0 0) 0x7f18680703d0 con 0x7f186813d580 2024-01-24T23:22:56.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 1 -- 172.21.15.132:0/947099146 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1868132d80 con 0x7f186813d580 2024-01-24T23:22:56.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 1 -- 172.21.15.132:0/947099146 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1511732864 0 0) 0x7f184c003530 con 0x7f1868136fa0 2024-01-24T23:22:56.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 1 -- 172.21.15.132:0/947099146 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f18680703d0 con 0x7f1868136fa0 2024-01-24T23:22:56.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 1 -- 172.21.15.132:0/947099146 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1860001c50 con 0x7f186813d580 2024-01-24T23:22:56.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:56.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:22:56.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 10 monclient(hunting): dump: 2024-01-24T23:22:56.279 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:56.279 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:56.279 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:56.279 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:56.279 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:56.279 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:56.279 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:56.279 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:56.280 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:56.280 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:56.280 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:56.280 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:56.280 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:56.280 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:56.280 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:56.280 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:56.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:56.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 1 -- 172.21.15.132:0/947099146 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825411232 0 0) 0x7f18680712d0 con 0x7f1868071880 2024-01-24T23:22:56.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:56.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:56.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:56.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 10 monclient(hunting): my global_id is 4830 2024-01-24T23:22:56.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 1 -- 172.21.15.132:0/947099146 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f184c003530 con 0x7f1868071880 2024-01-24T23:22:56.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 1 -- 172.21.15.132:0/947099146 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1858001ac0 con 0x7f1868136fa0 2024-01-24T23:22:56.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:56.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:22:56.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 10 monclient(hunting): dump: 2024-01-24T23:22:56.281 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:56.281 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:56.281 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:56.281 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:56.281 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:56.281 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:56.281 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:56.281 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:56.281 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:56.282 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:56.282 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:56.282 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:56.282 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:56.282 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:56.282 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:56.282 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:56.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 1 -- 172.21.15.132:0/947099146 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1763225503 0 0) 0x7f1868132d80 con 0x7f186813d580 2024-01-24T23:22:56.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 1 -- 172.21.15.132:0/947099146 >> v1:172.21.15.115:6792/0 conn(0x7f1868071880 legacy=0x7f186806f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:56.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 1 -- 172.21.15.132:0/947099146 >> v1:172.21.15.132:6790/0 conn(0x7f1868136fa0 legacy=0x7f186806fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:56.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:22:56.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 1 monclient: found mon.f 2024-01-24T23:22:56.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:56.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:56.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 1 -- 172.21.15.132:0/947099146 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1868131f90 con 0x7f186813d580 2024-01-24T23:22:56.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 10 monclient: _finish_auth 0 2024-01-24T23:22:56.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:22:56.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:56.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f186fe2f640 5 monclient: authenticate success, global_id 4834 2024-01-24T23:22:56.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f186fe2f640 10 monclient: _renew_subs 2024-01-24T23:22:56.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f186fe2f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:56.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f186fe2f640 1 -- 172.21.15.132:0/947099146 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1868071650 con 0x7f186813d580 2024-01-24T23:22:56.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f186fe2f640 10 monclient: _renew_subs 2024-01-24T23:22:56.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f186fe2f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:56.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f186fe2f640 1 -- 172.21.15.132:0/947099146 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1868070bf0 con 0x7f186813d580 2024-01-24T23:22:56.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 1 -- 172.21.15.132:0/947099146 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1860001e00 con 0x7f186813d580 2024-01-24T23:22:56.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:56.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 1 -- 172.21.15.132:0/947099146 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1860002060 con 0x7f186813d580 2024-01-24T23:22:56.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:56.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185ffff640 4 set_mon_vals no callback set 2024-01-24T23:22:56.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:22:56.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 10 monclient: dump: 2024-01-24T23:22:56.284 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:56.284 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:56.284 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:56.284 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:56.284 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:56.284 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:56.284 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:56.284 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:56.284 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:56.284 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:56.284 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:56.284 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:56.285 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:56.285 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:56.285 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:56.285 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:56.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f186fe2f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:56.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f186fe2f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:56.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f186fe2f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:56.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f186fe2f640 1 -- 172.21.15.132:0/947099146 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1850005090 con 0x7f186813d580 2024-01-24T23:22:56.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 1 -- 172.21.15.132:0/947099146 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f186000e8b0 con 0x7f186813d580 2024-01-24T23:22:56.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f185e7fc640 1 -- 172.21.15.132:0/947099146 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(38..38 src has 1..38) v4 ==== 3598+0+0 (unknown 3181584953 0 0) 0x7f1860042620 con 0x7f186813d580 2024-01-24T23:22:56.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.265+0000 7f186dbcb640 10 monclient: get_auth_request con 0x7f184c037930 auth_method 0 2024-01-24T23:22:56.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.269+0000 7f185e7fc640 1 -- 172.21.15.132:0/947099146 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f18600137f0 con 0x7f186813d580 2024-01-24T23:22:56.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.269+0000 7f185e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:56.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.269+0000 7f185e7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:56.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.425+0000 7f186fe2f640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:56.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.425+0000 7f186fe2f640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:56.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.425+0000 7f186fe2f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:56.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.425+0000 7f186fe2f640 1 -- 172.21.15.132:0/947099146 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f18500058c0 con 0x7f186813d580 2024-01-24T23:22:56.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.425+0000 7f185e7fc640 1 -- 172.21.15.132:0/947099146 <== mon.1 v1:172.21.15.132:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}]=0 pool 'foo' does not exist v38) v1 ==== 156+0+0 (unknown 495831957 0 0) 0x7f1860011e90 con 0x7f186813d580 2024-01-24T23:22:56.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.425+0000 7f185e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:56.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.425+0000 7f185e7fc640 10 monclient: _finish_command 2 = system:0 pool 'foo' does not exist 2024-01-24T23:22:56.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.513+0000 7f186fe2f640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:56.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.513+0000 7f186fe2f640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:56.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.513+0000 7f186fe2f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:56.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.513+0000 7f186fe2f640 1 -- 172.21.15.132:0/947099146 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f18500010a0 con 0x7f186813d580 2024-01-24T23:22:56.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.517+0000 7f185e7fc640 1 -- 172.21.15.132:0/947099146 <== mon.1 v1:172.21.15.132:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}]=0 pool 'foo' does not exist v38) v1 ==== 156+0+0 (unknown 495831957 0 0) 0x7f1860013610 con 0x7f186813d580 2024-01-24T23:22:56.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.517+0000 7f185e7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:56.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.517+0000 7f185e7fc640 10 monclient: _finish_command 3 = system:0 pool 'foo' does not exist 2024-01-24T23:22:56.519 INFO:tasks.workunit.client.0.smithi132.stderr:pool 'foo' does not exist 2024-01-24T23:22:56.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.517+0000 7f186fe2f640 1 -- 172.21.15.132:0/947099146 >> v1:172.21.15.132:6812/1835401922 conn(0x7f184c037930 legacy=0x7f184c039df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:56.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.517+0000 7f186fe2f640 10 monclient: shutdown 2024-01-24T23:22:56.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.517+0000 7f186fe2f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:56.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.517+0000 7f186fe2f640 1 -- 172.21.15.132:0/947099146 >> v1:172.21.15.132:6789/0 conn(0x7f186813d580 legacy=0x7f18681d3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:56.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.517+0000 7f186fe2f640 1 -- 172.21.15.132:0/947099146 shutdown_connections 2024-01-24T23:22:56.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.517+0000 7f186fe2f640 1 -- 172.21.15.132:0/947099146 >> 172.21.15.132:0/947099146 conn(0x7f186806aac0 msgr2=0x7f18680a25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:56.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.517+0000 7f186fe2f640 1 -- 172.21.15.132:0/947099146 shutdown_connections 2024-01-24T23:22:56.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.517+0000 7f186fe2f640 1 -- 172.21.15.132:0/947099146 wait complete. 2024-01-24T23:22:56.535 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool delete fooo fooo --yes-i-really-really-mean-it 2024-01-24T23:22:56.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff879124640 10 monclient: get_monmap_and_config 2024-01-24T23:22:56.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff879124640 10 monclient: build_initial_monmap 2024-01-24T23:22:56.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff879124640 10 monclient: monmap: 2024-01-24T23:22:56.676 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:56.676 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:56.676 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:56.677583+0000 2024-01-24T23:22:56.676 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:56.677583+0000 2024-01-24T23:22:56.676 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:56.676 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:56.676 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:56.676 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:56.677 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:56.677 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:56.677 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:56.677 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:56.677 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:56.677 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:56.677 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:56.677 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:56.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff879124640 1 Processor -- start 2024-01-24T23:22:56.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff879124640 1 -- start start 2024-01-24T23:22:56.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff879124640 10 monclient: init 2024-01-24T23:22:56.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff879124640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:56.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff879124640 10 monclient: _add_conns ranks=[7,8,4,3,6,0,5,2,1] 2024-01-24T23:22:56.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff879124640 10 monclient(hunting): picked mon.noname-i con 0x7ff874133ce0 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:56.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff879124640 10 monclient(hunting): picked mon.noname-e con 0x7ff87409ddc0 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:56.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff879124640 10 monclient(hunting): picked mon.noname-c con 0x7ff87409e1a0 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:56.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff879124640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff874064700 con 0x7ff87409e1a0 2024-01-24T23:22:56.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff879124640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87409c6d0 con 0x7ff874133ce0 2024-01-24T23:22:56.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff879124640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87409c9b0 con 0x7ff87409ddc0 2024-01-24T23:22:56.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff879124640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:56.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff879124640 10 monclient(hunting): authenticate will time out at 1064.002741s 2024-01-24T23:22:56.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff87259c640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7ff87409ddc0 legacy=0x7ff874138ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:56.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff872d9d640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7ff874133ce0 0x7ff8741340c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:38854/0 (socket says 172.21.15.132:38854) 2024-01-24T23:22:56.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff872d9d640 1 -- 172.21.15.132:0/3622489582 learned_addr learned my addr 172.21.15.132:0/3622489582 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:56.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff871d9b640 1 -- 172.21.15.132:0/3622489582 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 17485777 0 0) 0x7ff87409c6d0 con 0x7ff874133ce0 2024-01-24T23:22:56.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff871d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:22:56.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff871d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:56.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff871d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:56.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff871d9b640 10 monclient(hunting): my global_id is 4813 2024-01-24T23:22:56.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.673+0000 7ff871d9b640 1 -- 172.21.15.132:0/3622489582 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff85c003530 con 0x7ff874133ce0 2024-01-24T23:22:56.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff871d9b640 1 -- 172.21.15.132:0/3622489582 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3366861462 0 0) 0x7ff85c003530 con 0x7ff874133ce0 2024-01-24T23:22:56.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff871d9b640 1 -- 172.21.15.132:0/3622489582 >> v1:172.21.15.115:6793/0 conn(0x7ff87409ddc0 legacy=0x7ff874138ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:56.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff871d9b640 1 -- 172.21.15.132:0/3622489582 >> v1:172.21.15.115:6791/0 conn(0x7ff87409e1a0 legacy=0x7ff87413c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:56.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff871d9b640 10 monclient: _finish_hunting 0 2024-01-24T23:22:56.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff871d9b640 1 monclient: found mon.noname-i 2024-01-24T23:22:56.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff871d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:56.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff871d9b640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:22:56.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff871d9b640 1 -- 172.21.15.132:0/3622489582 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87409cc90 con 0x7ff874133ce0 2024-01-24T23:22:56.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff871d9b640 10 monclient: _finish_auth 0 2024-01-24T23:22:56.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff871d9b640 10 monclient: _check_auth_tickets 2024-01-24T23:22:56.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff871d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:56.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff879124640 5 monclient: authenticate success, global_id 4813 2024-01-24T23:22:56.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff879124640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:56.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff871d9b640 1 -- 172.21.15.132:0/3622489582 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff8680017c0 con 0x7ff874133ce0 2024-01-24T23:22:56.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff871d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:56.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff871d9b640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:22:56.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff871d9b640 10 monclient: dump: 2024-01-24T23:22:56.683 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:56.683 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:56.683 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:56.683 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:56.683 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:56.683 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:56.683 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:56.683 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:56.683 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:56.683 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:56.683 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:56.683 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:56.684 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:56.684 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:56.684 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:56.685 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:56.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff879124640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:56.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff871d9b640 1 -- 172.21.15.132:0/3622489582 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff8680019e0 con 0x7ff874133ce0 2024-01-24T23:22:56.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff871d9b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:56.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff871d9b640 1 -- 172.21.15.132:0/3622489582 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff868004bf0 con 0x7ff874133ce0 2024-01-24T23:22:56.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff879124640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:56.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff879124640 4 set_mon_vals no callback set 2024-01-24T23:22:56.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff871d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:56.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff871d9b640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:22:56.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff871d9b640 10 monclient: dump: 2024-01-24T23:22:56.685 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:56.685 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:56.685 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:56.685 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:56.686 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:56.686 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:56.686 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:56.686 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:56.686 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:56.686 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:56.686 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:56.686 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:56.686 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:56.686 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:56.686 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:56.686 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:56.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff879124640 10 monclient: shutdown 2024-01-24T23:22:56.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff879124640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:56.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff879124640 1 -- 172.21.15.132:0/3622489582 >> v1:172.21.15.132:6792/0 conn(0x7ff874133ce0 legacy=0x7ff8741340c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:56.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff879124640 1 -- 172.21.15.132:0/3622489582 shutdown_connections 2024-01-24T23:22:56.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff879124640 1 -- 172.21.15.132:0/3622489582 >> 172.21.15.132:0/3622489582 conn(0x7ff8740a6840 msgr2=0x7ff8740a2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:56.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff879124640 1 -- 172.21.15.132:0/3622489582 shutdown_connections 2024-01-24T23:22:56.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff879124640 1 -- 172.21.15.132:0/3622489582 wait complete. 2024-01-24T23:22:56.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff879124640 10 monclient: build_initial_monmap 2024-01-24T23:22:56.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff879124640 10 monclient: monmap: 2024-01-24T23:22:56.690 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:56.690 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:56.690 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:56.690 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:56.690 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:56.690 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:56.691 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:56.691 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:56.691 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:56.691 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:56.691 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:56.691 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:56.691 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:56.691 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:56.691 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:56.691 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:56.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff879124640 1 Processor -- start 2024-01-24T23:22:56.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff879124640 1 -- start start 2024-01-24T23:22:56.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.677+0000 7ff879124640 10 monclient: init 2024-01-24T23:22:56.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff879124640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:56.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff879124640 10 monclient: _add_conns ranks=[7,3,1,2,0,6,8,5,4] 2024-01-24T23:22:56.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff879124640 10 monclient(hunting): picked mon.noname-i con 0x7ff87409ddc0 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:56.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff879124640 10 monclient(hunting): picked mon.noname-g con 0x7ff87409e1a0 addr v1:172.21.15.132:6790/0 2024-01-24T23:22:56.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff879124640 10 monclient(hunting): picked mon.noname-f con 0x7ff874133ce0 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:56.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff879124640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87409ba70 con 0x7ff874133ce0 2024-01-24T23:22:56.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff879124640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8740723d0 con 0x7ff87409e1a0 2024-01-24T23:22:56.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff879124640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8741bc670 con 0x7ff87409ddc0 2024-01-24T23:22:56.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff879124640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:56.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff879124640 10 monclient(hunting): authenticate will time out at 1064.008734s 2024-01-24T23:22:56.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff87359e640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7ff874133ce0 0x7ff8741d3c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:53082/0 (socket says 172.21.15.132:53082) 2024-01-24T23:22:56.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff87259c640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7ff87409e1a0 0x7ff8741d04f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:59910/0 (socket says 172.21.15.132:59910) 2024-01-24T23:22:56.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff872d9d640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7ff87409ddc0 0x7ff8741cb680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:38864/0 (socket says 172.21.15.132:38864) 2024-01-24T23:22:56.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff87359e640 1 -- 172.21.15.132:0/3624946767 learned_addr learned my addr 172.21.15.132:0/3624946767 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:56.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 1 -- 172.21.15.132:0/3624946767 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 575431899 0 0) 0x7ff87409ba70 con 0x7ff874133ce0 2024-01-24T23:22:56.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:22:56.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:56.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:56.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 10 monclient(hunting): my global_id is 4843 2024-01-24T23:22:56.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 1 -- 172.21.15.132:0/3624946767 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff860003530 con 0x7ff874133ce0 2024-01-24T23:22:56.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 1 -- 172.21.15.132:0/3624946767 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3253259143 0 0) 0x7ff8740723d0 con 0x7ff87409e1a0 2024-01-24T23:22:56.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:22:56.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:56.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:56.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 10 monclient(hunting): my global_id is 4755 2024-01-24T23:22:56.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 1 -- 172.21.15.132:0/3624946767 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff87409ba70 con 0x7ff87409e1a0 2024-01-24T23:22:56.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 1 -- 172.21.15.132:0/3624946767 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844923957 0 0) 0x7ff8741bc670 con 0x7ff87409ddc0 2024-01-24T23:22:56.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:22:56.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:56.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:56.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 10 monclient(hunting): my global_id is 4822 2024-01-24T23:22:56.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 1 -- 172.21.15.132:0/3624946767 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8740723d0 con 0x7ff87409ddc0 2024-01-24T23:22:56.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 1 -- 172.21.15.132:0/3624946767 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1373638686 0 0) 0x7ff860003530 con 0x7ff874133ce0 2024-01-24T23:22:56.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 1 -- 172.21.15.132:0/3624946767 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff8741bc670 con 0x7ff874133ce0 2024-01-24T23:22:56.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 1 -- 172.21.15.132:0/3624946767 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff86c001a60 con 0x7ff874133ce0 2024-01-24T23:22:56.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:56.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:22:56.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 10 monclient(hunting): dump: 2024-01-24T23:22:56.694 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:56.694 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:56.694 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:56.694 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:56.694 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:56.694 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:56.695 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:56.695 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:56.695 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:56.695 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:56.695 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:56.695 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:56.695 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:56.695 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:56.695 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:56.695 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:56.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:56.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 1 -- 172.21.15.132:0/3624946767 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2976995060 0 0) 0x7ff8741bc670 con 0x7ff874133ce0 2024-01-24T23:22:56.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 1 -- 172.21.15.132:0/3624946767 >> v1:172.21.15.132:6792/0 conn(0x7ff87409ddc0 legacy=0x7ff8741cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:56.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 1 -- 172.21.15.132:0/3624946767 >> v1:172.21.15.132:6790/0 conn(0x7ff87409e1a0 legacy=0x7ff8741d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:56.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:22:56.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 1 monclient: found mon.f 2024-01-24T23:22:56.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:56.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:56.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 1 -- 172.21.15.132:0/3624946767 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8741bbdd0 con 0x7ff874133ce0 2024-01-24T23:22:56.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 10 monclient: _finish_auth 0 2024-01-24T23:22:56.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:22:56.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:56.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 1 -- 172.21.15.132:0/3624946767 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff86c001d20 con 0x7ff874133ce0 2024-01-24T23:22:56.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:56.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 1 -- 172.21.15.132:0/3624946767 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff86c003c10 con 0x7ff874133ce0 2024-01-24T23:22:56.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:56.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:22:56.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff85b7fe640 10 monclient: dump: 2024-01-24T23:22:56.696 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:56.697 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:56.697 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:56.697 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:56.697 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:56.697 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:56.697 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:56.697 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:56.697 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:56.697 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:56.697 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:56.697 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:56.697 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:56.697 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:56.697 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:56.697 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:56.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff87159a640 4 set_mon_vals no callback set 2024-01-24T23:22:56.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff879124640 5 monclient: authenticate success, global_id 4843 2024-01-24T23:22:56.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff879124640 10 monclient: _renew_subs 2024-01-24T23:22:56.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff879124640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:56.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff879124640 1 -- 172.21.15.132:0/3624946767 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8741bac30 con 0x7ff874133ce0 2024-01-24T23:22:56.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff879124640 10 monclient: _renew_subs 2024-01-24T23:22:56.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff879124640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:56.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.681+0000 7ff879124640 1 -- 172.21.15.132:0/3624946767 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8741cbf60 con 0x7ff874133ce0 2024-01-24T23:22:56.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.685+0000 7ff85b7fe640 1 -- 172.21.15.132:0/3624946767 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7ff86c003d70 con 0x7ff874133ce0 2024-01-24T23:22:56.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.685+0000 7ff879124640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:56.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.685+0000 7ff879124640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:56.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.685+0000 7ff879124640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:56.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.685+0000 7ff85b7fe640 1 -- 172.21.15.132:0/3624946767 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(38..38 src has 1..38) v4 ==== 3598+0+0 (unknown 3181584953 0 0) 0x7ff86c043590 con 0x7ff874133ce0 2024-01-24T23:22:56.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.685+0000 7ff879124640 1 -- 172.21.15.132:0/3624946767 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff874073400 con 0x7ff874133ce0 2024-01-24T23:22:56.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.685+0000 7ff872d9d640 10 monclient: get_auth_request con 0x7ff860037220 auth_method 0 2024-01-24T23:22:56.699 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.693+0000 7ff85b7fe640 1 -- 172.21.15.132:0/3624946767 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7ff86c0146e0 con 0x7ff874133ce0 2024-01-24T23:22:56.699 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.693+0000 7ff85b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:56.699 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.693+0000 7ff85b7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:56.860 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.857+0000 7ff879124640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:56.860 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.857+0000 7ff879124640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:56.860 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.857+0000 7ff879124640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:56.860 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.857+0000 7ff879124640 1 -- 172.21.15.132:0/3624946767 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff87406be00 con 0x7ff874133ce0 2024-01-24T23:22:56.868 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.865+0000 7ff85b7fe640 1 -- 172.21.15.132:0/3624946767 <== mon.1 v1:172.21.15.132:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}]=0 pool 'fooo' does not exist v38) v1 ==== 159+0+0 (unknown 940139346 0 0) 0x7ff86c014500 con 0x7ff874133ce0 2024-01-24T23:22:56.868 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.865+0000 7ff85b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:56.868 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.865+0000 7ff85b7fe640 10 monclient: _finish_command 2 = system:0 pool 'fooo' does not exist 2024-01-24T23:22:56.963 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.957+0000 7ff879124640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:56.963 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.961+0000 7ff879124640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:56.963 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.961+0000 7ff879124640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:56.963 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.961+0000 7ff879124640 1 -- 172.21.15.132:0/3624946767 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff8740a3510 con 0x7ff874133ce0 2024-01-24T23:22:56.964 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.961+0000 7ff85b7fe640 1 -- 172.21.15.132:0/3624946767 <== mon.1 v1:172.21.15.132:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}]=0 pool 'fooo' does not exist v38) v1 ==== 159+0+0 (unknown 940139346 0 0) 0x7ff86c015240 con 0x7ff874133ce0 2024-01-24T23:22:56.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.961+0000 7ff85b7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:56.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.961+0000 7ff85b7fe640 10 monclient: _finish_command 3 = system:0 pool 'fooo' does not exist 2024-01-24T23:22:56.965 INFO:tasks.workunit.client.0.smithi132.stderr:pool 'fooo' does not exist 2024-01-24T23:22:56.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.961+0000 7ff879124640 1 -- 172.21.15.132:0/3624946767 >> v1:172.21.15.132:6812/1835401922 conn(0x7ff860037220 legacy=0x7ff8600396e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:56.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.961+0000 7ff879124640 10 monclient: shutdown 2024-01-24T23:22:56.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.961+0000 7ff879124640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:56.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.961+0000 7ff879124640 1 -- 172.21.15.132:0/3624946767 >> v1:172.21.15.132:6789/0 conn(0x7ff874133ce0 legacy=0x7ff8741d3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:56.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.961+0000 7ff879124640 1 -- 172.21.15.132:0/3624946767 shutdown_connections 2024-01-24T23:22:56.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.961+0000 7ff879124640 1 -- 172.21.15.132:0/3624946767 >> 172.21.15.132:0/3624946767 conn(0x7ff8740a6840 msgr2=0x7ff87406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:56.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.965+0000 7ff879124640 1 -- 172.21.15.132:0/3624946767 shutdown_connections 2024-01-24T23:22:56.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:56.965+0000 7ff879124640 1 -- 172.21.15.132:0/3624946767 wait complete. 2024-01-24T23:22:56.980 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool delete fooo fooo --yes-i-really-really-mean-it 2024-01-24T23:22:57.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f98506b3640 10 monclient: get_monmap_and_config 2024-01-24T23:22:57.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f98506b3640 10 monclient: build_initial_monmap 2024-01-24T23:22:57.093 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f98506b3640 10 monclient: monmap: 2024-01-24T23:22:57.093 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:57.093 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:57.093 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:57.094346+0000 2024-01-24T23:22:57.093 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:57.094346+0000 2024-01-24T23:22:57.093 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:57.093 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:57.093 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:57.093 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:57.093 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:57.093 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:57.093 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:57.093 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:57.093 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:57.093 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:57.094 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:57.094 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:57.100 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f98506b3640 1 Processor -- start 2024-01-24T23:22:57.100 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f98506b3640 1 -- start start 2024-01-24T23:22:57.100 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f98506b3640 10 monclient: init 2024-01-24T23:22:57.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f98506b3640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:57.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f98506b3640 10 monclient: _add_conns ranks=[8,0,6,5,3,2,7,1,4] 2024-01-24T23:22:57.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f98506b3640 10 monclient(hunting): picked mon.noname-e con 0x7f9848072690 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:57.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f98506b3640 10 monclient(hunting): picked mon.noname-a con 0x7f984809ae90 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:57.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f98506b3640 10 monclient(hunting): picked mon.noname-d con 0x7f984809bdc0 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:57.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f98506b3640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9848064700 con 0x7f984809ae90 2024-01-24T23:22:57.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f98506b3640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f984809a790 con 0x7f984809bdc0 2024-01-24T23:22:57.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f98506b3640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98480a18f0 con 0x7f9848072690 2024-01-24T23:22:57.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f98506b3640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:57.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f98506b3640 10 monclient(hunting): authenticate will time out at 1064.419013s 2024-01-24T23:22:57.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f984e44f640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f9848072690 legacy=0x7f98480a0ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:57.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f984dc4e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f984809ae90 0x7f98480a11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:47570/0 (socket says 172.21.15.132:47570) 2024-01-24T23:22:57.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f984dc4e640 1 -- 172.21.15.132:0/3866859816 learned_addr learned my addr 172.21.15.132:0/3866859816 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:57.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f984d44d640 1 -- 172.21.15.132:0/3866859816 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2191099523 0 0) 0x7f9848064700 con 0x7f984809ae90 2024-01-24T23:22:57.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f984d44d640 10 monclient(hunting): _negotiate 2024-01-24T23:22:57.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f984d44d640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:57.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f984d44d640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:57.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f984d44d640 10 monclient(hunting): my global_id is 4797 2024-01-24T23:22:57.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f984d44d640 1 -- 172.21.15.132:0/3866859816 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9834003530 con 0x7f984809ae90 2024-01-24T23:22:57.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f984d44d640 1 -- 172.21.15.132:0/3866859816 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3377834073 0 0) 0x7f984809a790 con 0x7f984809bdc0 2024-01-24T23:22:57.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f984d44d640 10 monclient(hunting): _negotiate 2024-01-24T23:22:57.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f984d44d640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:57.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f984d44d640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:57.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f984d44d640 10 monclient(hunting): my global_id is 4839 2024-01-24T23:22:57.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.089+0000 7f984d44d640 1 -- 172.21.15.132:0/3866859816 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9848064700 con 0x7f984809bdc0 2024-01-24T23:22:57.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984d44d640 1 -- 172.21.15.132:0/3866859816 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1765696540 0 0) 0x7f9834003530 con 0x7f984809ae90 2024-01-24T23:22:57.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984d44d640 1 -- 172.21.15.132:0/3866859816 >> v1:172.21.15.115:6793/0 conn(0x7f9848072690 legacy=0x7f98480a0ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:57.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984d44d640 1 -- 172.21.15.132:0/3866859816 >> v1:172.21.15.115:6792/0 conn(0x7f984809bdc0 legacy=0x7f984813d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:57.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984d44d640 10 monclient: _finish_hunting 0 2024-01-24T23:22:57.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984d44d640 1 monclient: found mon.noname-a 2024-01-24T23:22:57.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984d44d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:57.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984d44d640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:22:57.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984d44d640 1 -- 172.21.15.132:0/3866859816 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98480997a0 con 0x7f984809ae90 2024-01-24T23:22:57.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984d44d640 10 monclient: _finish_auth 0 2024-01-24T23:22:57.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984d44d640 10 monclient: _check_auth_tickets 2024-01-24T23:22:57.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984d44d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:57.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 5 monclient: authenticate success, global_id 4797 2024-01-24T23:22:57.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:57.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984d44d640 1 -- 172.21.15.132:0/3866859816 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f98380017c0 con 0x7f984809ae90 2024-01-24T23:22:57.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984d44d640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:57.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984d44d640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:22:57.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984d44d640 10 monclient: dump: 2024-01-24T23:22:57.103 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:57.103 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:57.103 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.104 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.104 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:57.104 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:57.104 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:57.104 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:57.104 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:57.104 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:57.104 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:57.104 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:57.104 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:57.104 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:57.104 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:57.104 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:57.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:57.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984d44d640 1 -- 172.21.15.132:0/3866859816 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9838001a30 con 0x7f984809ae90 2024-01-24T23:22:57.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984d44d640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:57.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:57.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 4 set_mon_vals no callback set 2024-01-24T23:22:57.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984d44d640 1 -- 172.21.15.132:0/3866859816 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9838004bf0 con 0x7f984809ae90 2024-01-24T23:22:57.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984d44d640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:57.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984d44d640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:22:57.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984d44d640 10 monclient: dump: 2024-01-24T23:22:57.105 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:57.105 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:57.105 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.105 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.105 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:57.105 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:57.105 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:57.106 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:57.106 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:57.106 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:57.106 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:57.106 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:57.106 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:57.106 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:57.106 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:57.106 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:57.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 10 monclient: shutdown 2024-01-24T23:22:57.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:57.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 1 -- 172.21.15.132:0/3866859816 >> v1:172.21.15.115:6789/0 conn(0x7f984809ae90 legacy=0x7f98480a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:57.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 1 -- 172.21.15.132:0/3866859816 shutdown_connections 2024-01-24T23:22:57.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 1 -- 172.21.15.132:0/3866859816 >> 172.21.15.132:0/3866859816 conn(0x7f98480a1bc0 msgr2=0x7f98480a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:57.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 1 -- 172.21.15.132:0/3866859816 shutdown_connections 2024-01-24T23:22:57.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 1 -- 172.21.15.132:0/3866859816 wait complete. 2024-01-24T23:22:57.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 10 monclient: build_initial_monmap 2024-01-24T23:22:57.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 10 monclient: monmap: 2024-01-24T23:22:57.107 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:57.107 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:57.107 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:57.107 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:57.107 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:57.107 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:57.107 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:57.107 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:57.107 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:57.107 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:57.107 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:57.108 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:57.108 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:57.108 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:57.108 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:57.108 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:57.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 1 Processor -- start 2024-01-24T23:22:57.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 1 -- start start 2024-01-24T23:22:57.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 10 monclient: init 2024-01-24T23:22:57.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:57.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 10 monclient: _add_conns ranks=[6,1,2,4,8,0,3,5,7] 2024-01-24T23:22:57.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 10 monclient(hunting): picked mon.noname-d con 0x7f9848072690 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:57.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 10 monclient(hunting): picked mon.noname-f con 0x7f984809ae90 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:57.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 10 monclient(hunting): picked mon.noname-b con 0x7f984809bdc0 addr v1:172.21.15.115:6790/0 2024-01-24T23:22:57.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9848099470 con 0x7f984809ae90 2024-01-24T23:22:57.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f984808cd80 con 0x7f984809bdc0 2024-01-24T23:22:57.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f984809b930 con 0x7f9848072690 2024-01-24T23:22:57.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:57.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 10 monclient(hunting): authenticate will time out at 1064.422905s 2024-01-24T23:22:57.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984ec50640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f984809bdc0 legacy=0x7f98481d1760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:22:57.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984dc4e640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f984809ae90 0x7f984808c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:53086/0 (socket says 172.21.15.132:53086) 2024-01-24T23:22:57.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984dc4e640 1 -- 172.21.15.132:0/1072410478 learned_addr learned my addr 172.21.15.132:0/1072410478 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:57.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 1 -- 172.21.15.132:0/1072410478 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491369524 0 0) 0x7f9848099470 con 0x7f984809ae90 2024-01-24T23:22:57.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 10 monclient(hunting): _negotiate 2024-01-24T23:22:57.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:57.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:57.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 10 monclient(hunting): my global_id is 4852 2024-01-24T23:22:57.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 1 -- 172.21.15.132:0/1072410478 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f982c003530 con 0x7f984809ae90 2024-01-24T23:22:57.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 1 -- 172.21.15.132:0/1072410478 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 31696580 0 0) 0x7f984809b930 con 0x7f9848072690 2024-01-24T23:22:57.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 10 monclient(hunting): _negotiate 2024-01-24T23:22:57.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:57.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:57.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 10 monclient(hunting): my global_id is 4848 2024-01-24T23:22:57.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 1 -- 172.21.15.132:0/1072410478 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9848099470 con 0x7f9848072690 2024-01-24T23:22:57.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 1 -- 172.21.15.132:0/1072410478 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3445218692 0 0) 0x7f982c003530 con 0x7f984809ae90 2024-01-24T23:22:57.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 1 -- 172.21.15.132:0/1072410478 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f984809b930 con 0x7f984809ae90 2024-01-24T23:22:57.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 1 -- 172.21.15.132:0/1072410478 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f98380047f0 con 0x7f984809ae90 2024-01-24T23:22:57.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:57.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:22:57.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 10 monclient(hunting): dump: 2024-01-24T23:22:57.110 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:57.110 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:57.110 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.110 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:57.111 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 1 -- 172.21.15.132:0/1072410478 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2646177032 0 0) 0x7f984809b930 con 0x7f984809ae90 2024-01-24T23:22:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 1 -- 172.21.15.132:0/1072410478 >> v1:172.21.15.115:6792/0 conn(0x7f9848072690 legacy=0x7f984808bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:57.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 1 -- 172.21.15.132:0/1072410478 >> v1:172.21.15.115:6790/0 conn(0x7f984809bdc0 legacy=0x7f98481d1760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:57.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 10 monclient: _finish_hunting 0 2024-01-24T23:22:57.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 1 monclient: found mon.f 2024-01-24T23:22:57.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:57.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:57.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 1 -- 172.21.15.132:0/1072410478 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9848093ac0 con 0x7f984809ae90 2024-01-24T23:22:57.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 10 monclient: _finish_auth 0 2024-01-24T23:22:57.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 10 monclient: _check_auth_tickets 2024-01-24T23:22:57.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:57.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 5 monclient: authenticate success, global_id 4852 2024-01-24T23:22:57.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 10 monclient: _renew_subs 2024-01-24T23:22:57.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:57.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 1 -- 172.21.15.132:0/1072410478 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f984808d790 con 0x7f984809ae90 2024-01-24T23:22:57.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 10 monclient: _renew_subs 2024-01-24T23:22:57.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:57.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f98506b3640 1 -- 172.21.15.132:0/1072410478 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f984808e2f0 con 0x7f984809ae90 2024-01-24T23:22:57.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 1 -- 172.21.15.132:0/1072410478 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f98380011c0 con 0x7f984809ae90 2024-01-24T23:22:57.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:57.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 1 -- 172.21.15.132:0/1072410478 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9838005200 con 0x7f984809ae90 2024-01-24T23:22:57.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f983effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:57.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.093+0000 7f984cc4c640 4 set_mon_vals no callback set 2024-01-24T23:22:57.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.097+0000 7f983effd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:22:57.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.097+0000 7f983effd640 10 monclient: dump: 2024-01-24T23:22:57.113 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:57.113 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:57.113 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.113 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.113 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:57.113 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:57.114 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:57.114 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:57.114 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:57.114 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:57.114 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:57.114 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:57.114 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:57.114 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:57.114 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:57.114 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:57.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.097+0000 7f983effd640 1 -- 172.21.15.132:0/1072410478 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f983800fc80 con 0x7f984809ae90 2024-01-24T23:22:57.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.097+0000 7f98506b3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:57.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.097+0000 7f98506b3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:57.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.097+0000 7f98506b3640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:57.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.097+0000 7f98506b3640 1 -- 172.21.15.132:0/1072410478 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9830005090 con 0x7f984809ae90 2024-01-24T23:22:57.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.097+0000 7f983effd640 1 -- 172.21.15.132:0/1072410478 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(38..38 src has 1..38) v4 ==== 3598+0+0 (unknown 3181584953 0 0) 0x7f9838043a80 con 0x7f984809ae90 2024-01-24T23:22:57.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.097+0000 7f984e44f640 10 monclient: get_auth_request con 0x7f982c036f70 auth_method 0 2024-01-24T23:22:57.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.101+0000 7f983effd640 1 -- 172.21.15.132:0/1072410478 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f9838014e60 con 0x7f984809ae90 2024-01-24T23:22:57.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.101+0000 7f983effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:57.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.101+0000 7f983effd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:57.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.253+0000 7f98506b3640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:57.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.253+0000 7f98506b3640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:57.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.253+0000 7f98506b3640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:57.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.253+0000 7f98506b3640 1 -- 172.21.15.132:0/1072410478 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f98300058c0 con 0x7f984809ae90 2024-01-24T23:22:57.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.257+0000 7f983effd640 1 -- 172.21.15.132:0/1072410478 <== mon.1 v1:172.21.15.132:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}]=0 pool 'fooo' does not exist v38) v1 ==== 159+0+0 (unknown 940139346 0 0) 0x7f983801cc60 con 0x7f984809ae90 2024-01-24T23:22:57.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.257+0000 7f983effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:57.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.257+0000 7f983effd640 10 monclient: _finish_command 2 = system:0 pool 'fooo' does not exist 2024-01-24T23:22:57.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.349+0000 7f98506b3640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:57.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.349+0000 7f98506b3640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:57.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.349+0000 7f98506b3640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:22:57.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.349+0000 7f98506b3640 1 -- 172.21.15.132:0/1072410478 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f98300008e0 con 0x7f984809ae90 2024-01-24T23:22:57.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.349+0000 7f983effd640 1 -- 172.21.15.132:0/1072410478 <== mon.1 v1:172.21.15.132:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}]=0 pool 'fooo' does not exist v38) v1 ==== 159+0+0 (unknown 940139346 0 0) 0x7f98380132d0 con 0x7f984809ae90 2024-01-24T23:22:57.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.349+0000 7f983effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:57.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.349+0000 7f983effd640 10 monclient: _finish_command 3 = system:0 pool 'fooo' does not exist 2024-01-24T23:22:57.353 INFO:tasks.workunit.client.0.smithi132.stderr:pool 'fooo' does not exist 2024-01-24T23:22:57.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.349+0000 7f98506b3640 1 -- 172.21.15.132:0/1072410478 >> v1:172.21.15.132:6812/1835401922 conn(0x7f982c036f70 legacy=0x7f982c039430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:57.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.349+0000 7f98506b3640 10 monclient: shutdown 2024-01-24T23:22:57.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.349+0000 7f98506b3640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:57.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.349+0000 7f98506b3640 1 -- 172.21.15.132:0/1072410478 >> v1:172.21.15.132:6789/0 conn(0x7f984809ae90 legacy=0x7f984808c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:57.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.353+0000 7f98506b3640 1 -- 172.21.15.132:0/1072410478 shutdown_connections 2024-01-24T23:22:57.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.353+0000 7f98506b3640 1 -- 172.21.15.132:0/1072410478 >> 172.21.15.132:0/1072410478 conn(0x7f98480a1bc0 msgr2=0x7f9848097790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:57.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.353+0000 7f98506b3640 1 -- 172.21.15.132:0/1072410478 shutdown_connections 2024-01-24T23:22:57.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.353+0000 7f98506b3640 1 -- 172.21.15.132:0/1072410478 wait complete. 2024-01-24T23:22:57.369 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool delete fuggg fuggg --yes-i-really-really-mean-it 2024-01-24T23:22:57.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.485+0000 7fdff2607640 10 monclient: get_monmap_and_config 2024-01-24T23:22:57.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.485+0000 7fdff2607640 10 monclient: build_initial_monmap 2024-01-24T23:22:57.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.485+0000 7fdff2607640 10 monclient: monmap: 2024-01-24T23:22:57.488 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:57.488 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:57.488 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:57.490105+0000 2024-01-24T23:22:57.489 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:57.490105+0000 2024-01-24T23:22:57.489 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:57.489 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:57.489 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:57.489 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:57.489 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:57.489 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:57.489 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:57.489 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:57.489 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:57.489 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:57.489 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:57.489 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:57.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.485+0000 7fdff2607640 1 Processor -- start 2024-01-24T23:22:57.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.485+0000 7fdff2607640 1 -- start start 2024-01-24T23:22:57.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.485+0000 7fdff2607640 10 monclient: init 2024-01-24T23:22:57.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.485+0000 7fdff2607640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:57.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.485+0000 7fdff2607640 10 monclient: _add_conns ranks=[0,2,8,4,6,7,5,1,3] 2024-01-24T23:22:57.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.485+0000 7fdff2607640 10 monclient(hunting): picked mon.noname-a con 0x7fdfec09d530 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:57.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.485+0000 7fdff2607640 10 monclient(hunting): picked mon.noname-b con 0x7fdfec1336f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:22:57.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.485+0000 7fdff2607640 10 monclient(hunting): picked mon.noname-e con 0x7fdfec133ad0 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:57.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.485+0000 7fdff2607640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfec064700 con 0x7fdfec09d530 2024-01-24T23:22:57.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.485+0000 7fdff2607640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfec09e630 con 0x7fdfec1336f0 2024-01-24T23:22:57.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.485+0000 7fdff2607640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfec09e910 con 0x7fdfec133ad0 2024-01-24T23:22:57.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.485+0000 7fdff2607640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:57.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.485+0000 7fdff2607640 10 monclient(hunting): authenticate will time out at 1064.815518s 2024-01-24T23:22:57.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.485+0000 7fdfeb7fe640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fdfec1336f0 legacy=0x7fdfec138300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:22:57.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.485+0000 7fdff0ba4640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7fdfec133ad0 legacy=0x7fdfec13ba30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:57.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.485+0000 7fdfebfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fdfec09d530 0x7fdfec132fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:47586/0 (socket says 172.21.15.132:47586) 2024-01-24T23:22:57.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.485+0000 7fdfebfff640 1 -- 172.21.15.132:0/1217307922 learned_addr learned my addr 172.21.15.132:0/1217307922 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:57.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 1 -- 172.21.15.132:0/1217307922 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 265645949 0 0) 0x7fdfec064700 con 0x7fdfec09d530 2024-01-24T23:22:57.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 10 monclient(hunting): _negotiate 2024-01-24T23:22:57.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:57.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:57.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 10 monclient(hunting): my global_id is 4806 2024-01-24T23:22:57.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 1 -- 172.21.15.132:0/1217307922 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfd4003530 con 0x7fdfec09d530 2024-01-24T23:22:57.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 1 -- 172.21.15.132:0/1217307922 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 75464900 0 0) 0x7fdfd4003530 con 0x7fdfec09d530 2024-01-24T23:22:57.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 1 -- 172.21.15.132:0/1217307922 >> v1:172.21.15.115:6793/0 conn(0x7fdfec133ad0 legacy=0x7fdfec13ba30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:57.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 1 -- 172.21.15.132:0/1217307922 >> v1:172.21.15.115:6790/0 conn(0x7fdfec1336f0 legacy=0x7fdfec138300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:57.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 10 monclient: _finish_hunting 0 2024-01-24T23:22:57.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 1 monclient: found mon.noname-a 2024-01-24T23:22:57.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:22:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 1 -- 172.21.15.132:0/1217307922 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec09d910 con 0x7fdfec09d530 2024-01-24T23:22:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 10 monclient: _finish_auth 0 2024-01-24T23:22:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 10 monclient: _check_auth_tickets 2024-01-24T23:22:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 1 -- 172.21.15.132:0/1217307922 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdfdc0017c0 con 0x7fdfec09d530 2024-01-24T23:22:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:22:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 10 monclient: dump: 2024-01-24T23:22:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:57.497 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 5 monclient: authenticate success, global_id 4806 2024-01-24T23:22:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 1 -- 172.21.15.132:0/1217307922 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdfdc001ae0 con 0x7fdfec09d530 2024-01-24T23:22:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 1 -- 172.21.15.132:0/1217307922 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdfdc004bf0 con 0x7fdfec09d530 2024-01-24T23:22:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 4 set_mon_vals no callback set 2024-01-24T23:22:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:22:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeaffd640 10 monclient: dump: 2024-01-24T23:22:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:57.499 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 10 monclient: shutdown 2024-01-24T23:22:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 1 -- 172.21.15.132:0/1217307922 >> v1:172.21.15.115:6789/0 conn(0x7fdfec09d530 legacy=0x7fdfec132fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 1 -- 172.21.15.132:0/1217307922 shutdown_connections 2024-01-24T23:22:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 1 -- 172.21.15.132:0/1217307922 >> 172.21.15.132:0/1217307922 conn(0x7fdfec0a5660 msgr2=0x7fdfec0a5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 1 -- 172.21.15.132:0/1217307922 shutdown_connections 2024-01-24T23:22:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 1 -- 172.21.15.132:0/1217307922 wait complete. 2024-01-24T23:22:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 10 monclient: build_initial_monmap 2024-01-24T23:22:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 10 monclient: monmap: 2024-01-24T23:22:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:57.501 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:57.501 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:57.501 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:57.501 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:57.501 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:57.501 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:57.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 1 Processor -- start 2024-01-24T23:22:57.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 1 -- start start 2024-01-24T23:22:57.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 10 monclient: init 2024-01-24T23:22:57.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 10 monclient: _add_conns ranks=[2,4,8,0,5,6,3,7,1] 2024-01-24T23:22:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 10 monclient(hunting): picked mon.noname-b con 0x7fdfec09d530 addr v1:172.21.15.115:6790/0 2024-01-24T23:22:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 10 monclient(hunting): picked mon.noname-c con 0x7fdfec1336f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 10 monclient(hunting): picked mon.noname-e con 0x7fdfec133ad0 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfec09ddb0 con 0x7fdfec09d530 2024-01-24T23:22:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfec12d7d0 con 0x7fdfec1336f0 2024-01-24T23:22:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfec0729a0 con 0x7fdfec133ad0 2024-01-24T23:22:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff2607640 10 monclient(hunting): authenticate will time out at 1064.819229s 2024-01-24T23:22:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfebfff640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fdfec09d530 legacy=0x7fdfec09bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:22:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdff0ba4640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7fdfec133ad0 legacy=0x7fdfec1d3ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeb7fe640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fdfec1336f0 0x7fdfec1d07c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:34864/0 (socket says 172.21.15.132:34864) 2024-01-24T23:22:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfeb7fe640 1 -- 172.21.15.132:0/2366882896 learned_addr learned my addr 172.21.15.132:0/2366882896 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfe8ff9640 1 -- 172.21.15.132:0/2366882896 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 167080292 0 0) 0x7fdfec12d7d0 con 0x7fdfec1336f0 2024-01-24T23:22:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfe8ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:22:57.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfe8ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:57.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfe8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:57.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfe8ff9640 10 monclient(hunting): my global_id is 74803 2024-01-24T23:22:57.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.489+0000 7fdfe8ff9640 1 -- 172.21.15.132:0/2366882896 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfd8003530 con 0x7fdfec1336f0 2024-01-24T23:22:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 1 -- 172.21.15.132:0/2366882896 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3660331715 0 0) 0x7fdfd8003530 con 0x7fdfec1336f0 2024-01-24T23:22:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 1 -- 172.21.15.132:0/2366882896 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdfec12d7d0 con 0x7fdfec1336f0 2024-01-24T23:22:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 1 -- 172.21.15.132:0/2366882896 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdfe0004330 con 0x7fdfec1336f0 2024-01-24T23:22:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:22:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 10 monclient(hunting): dump: 2024-01-24T23:22:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.506 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:57.506 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:57.506 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:57.506 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:57.506 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:57.506 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:57.506 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:57.506 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:57.506 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:57.507 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 1 -- 172.21.15.132:0/2366882896 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1242554861 0 0) 0x7fdfec12d7d0 con 0x7fdfec1336f0 2024-01-24T23:22:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 1 -- 172.21.15.132:0/2366882896 >> v1:172.21.15.115:6793/0 conn(0x7fdfec133ad0 legacy=0x7fdfec1d3ef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 1 -- 172.21.15.132:0/2366882896 >> v1:172.21.15.115:6790/0 conn(0x7fdfec09d530 legacy=0x7fdfec09bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:22:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 1 monclient: found mon.c 2024-01-24T23:22:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:22:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 1 -- 172.21.15.132:0/2366882896 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec130520 con 0x7fdfec1336f0 2024-01-24T23:22:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 10 monclient: _finish_auth 0 2024-01-24T23:22:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:22:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdff2607640 5 monclient: authenticate success, global_id 74803 2024-01-24T23:22:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdff2607640 10 monclient: _renew_subs 2024-01-24T23:22:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdff2607640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:22:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdff2607640 1 -- 172.21.15.132:0/2366882896 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfec130800 con 0x7fdfec1336f0 2024-01-24T23:22:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdff2607640 10 monclient: _renew_subs 2024-01-24T23:22:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdff2607640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:22:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdff2607640 1 -- 172.21.15.132:0/2366882896 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfec09c230 con 0x7fdfec1336f0 2024-01-24T23:22:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 1 -- 172.21.15.132:0/2366882896 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdfe0004550 con 0x7fdfec1336f0 2024-01-24T23:22:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 1 -- 172.21.15.132:0/2366882896 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdfe0004a80 con 0x7fdfec1336f0 2024-01-24T23:22:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe9ffb640 4 set_mon_vals no callback set 2024-01-24T23:22:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:22:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 10 monclient: dump: 2024-01-24T23:22:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:57.510 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdff2607640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdff2607640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdff2607640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:22:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdff2607640 1 -- 172.21.15.132:0/2366882896 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfcc005090 con 0x7fdfec1336f0 2024-01-24T23:22:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 1 -- 172.21.15.132:0/2366882896 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fdfe0004c60 con 0x7fdfec1336f0 2024-01-24T23:22:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfe8ff9640 1 -- 172.21.15.132:0/2366882896 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(38..38 src has 1..38) v4 ==== 3598+0+0 (unknown 3181584953 0 0) 0x7fdfe00436d0 con 0x7fdfec1336f0 2024-01-24T23:22:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.493+0000 7fdfebfff640 10 monclient: get_auth_request con 0x7fdfd8036da0 auth_method 0 2024-01-24T23:22:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.505+0000 7fdfe8ff9640 1 -- 172.21.15.132:0/2366882896 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fdfe00149f0 con 0x7fdfec1336f0 2024-01-24T23:22:57.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.505+0000 7fdfe8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:57.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.505+0000 7fdfe8ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:57.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.661+0000 7fdff2607640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:57.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.661+0000 7fdff2607640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:57.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.661+0000 7fdff2607640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:22:57.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.661+0000 7fdff2607640 1 -- 172.21.15.132:0/2366882896 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdfcc005340 con 0x7fdfec1336f0 2024-01-24T23:22:57.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.661+0000 7fdfe8ff9640 1 -- 172.21.15.132:0/2366882896 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}]=0 pool 'fuggg' does not exist v38) v1 ==== 162+0+0 (unknown 2356159631 0 0) 0x7fdfe0012e60 con 0x7fdfec1336f0 2024-01-24T23:22:57.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.661+0000 7fdfe8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:57.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.661+0000 7fdfe8ff9640 10 monclient: _finish_command 2 = system:0 pool 'fuggg' does not exist 2024-01-24T23:22:57.756 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.753+0000 7fdff2607640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:57.756 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.753+0000 7fdff2607640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:57.756 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.753+0000 7fdff2607640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:22:57.756 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.753+0000 7fdff2607640 1 -- 172.21.15.132:0/2366882896 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdfcc005a80 con 0x7fdfec1336f0 2024-01-24T23:22:57.758 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.753+0000 7fdfe8ff9640 1 -- 172.21.15.132:0/2366882896 <== mon.4 v1:172.21.15.115:6791/0 11 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}]=0 pool 'fuggg' does not exist v38) v1 ==== 162+0+0 (unknown 2356159631 0 0) 0x7fdfe00145e0 con 0x7fdfec1336f0 2024-01-24T23:22:57.758 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.753+0000 7fdfe8ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}] 2024-01-24T23:22:57.758 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.753+0000 7fdfe8ff9640 10 monclient: _finish_command 3 = system:0 pool 'fuggg' does not exist 2024-01-24T23:22:57.758 INFO:tasks.workunit.client.0.smithi132.stderr:pool 'fuggg' does not exist 2024-01-24T23:22:57.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.757+0000 7fdff2607640 1 -- 172.21.15.132:0/2366882896 >> v1:172.21.15.132:6812/1835401922 conn(0x7fdfd8036da0 legacy=0x7fdfd8039260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:57.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.757+0000 7fdff2607640 10 monclient: shutdown 2024-01-24T23:22:57.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.757+0000 7fdff2607640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:57.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.757+0000 7fdff2607640 1 -- 172.21.15.132:0/2366882896 >> v1:172.21.15.115:6791/0 conn(0x7fdfec1336f0 legacy=0x7fdfec1d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:57.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.757+0000 7fdff2607640 1 -- 172.21.15.132:0/2366882896 shutdown_connections 2024-01-24T23:22:57.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.757+0000 7fdff2607640 1 -- 172.21.15.132:0/2366882896 >> 172.21.15.132:0/2366882896 conn(0x7fdfec0a5660 msgr2=0x7fdfec097210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:57.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.757+0000 7fdff2607640 1 -- 172.21.15.132:0/2366882896 shutdown_connections 2024-01-24T23:22:57.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.757+0000 7fdff2607640 1 -- 172.21.15.132:0/2366882896 wait complete. 2024-01-24T23:22:57.774 INFO:tasks.workunit.client.0.smithi132.stdout:OK 2024-01-24T23:22:57.775 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo OK 2024-01-24T23:22:57.776 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-01-24T23:22:57.776 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-01-24T23:22:57.788 INFO:tasks.workunit:Running workunits matching mon/crush_ops.sh on client.0... 2024-01-24T23:22:57.790 INFO:tasks.workunit:Running workunit mon/crush_ops.sh... 2024-01-24T23:22:57.790 DEBUG:teuthology.orchestra.run.smithi132:workunit test mon/crush_ops.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=663873555d45def955d1cc09730bfece2847b821 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/crush_ops.sh 2024-01-24T23:22:57.839 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush dump 2024-01-24T23:22:57.956 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0f192a640 10 monclient: get_monmap_and_config 2024-01-24T23:22:57.956 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0f192a640 10 monclient: build_initial_monmap 2024-01-24T23:22:57.956 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0f192a640 10 monclient: monmap: 2024-01-24T23:22:57.957 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:57.957 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:57.957 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:57.958132+0000 2024-01-24T23:22:57.957 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:57.958132+0000 2024-01-24T23:22:57.957 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:57.957 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:57.957 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:57.957 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:57.957 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:57.957 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:57.957 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:57.957 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:57.957 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:57.957 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:57.958 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:57.958 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:57.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0f192a640 1 Processor -- start 2024-01-24T23:22:57.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0f192a640 1 -- start start 2024-01-24T23:22:57.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0f192a640 10 monclient: init 2024-01-24T23:22:57.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0f192a640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:57.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0f192a640 10 monclient: _add_conns ranks=[1,5,7,4,8,2,6,3,0] 2024-01-24T23:22:57.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0f192a640 10 monclient(hunting): picked mon.noname-f con 0x7ff0ec09ff20 addr v1:172.21.15.132:6789/0 2024-01-24T23:22:57.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0f192a640 10 monclient(hunting): picked mon.noname-h con 0x7ff0ec09f9c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:57.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0f192a640 10 monclient(hunting): picked mon.noname-i con 0x7ff0ec13b2c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:57.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0f192a640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0ec064700 con 0x7ff0ec09ff20 2024-01-24T23:22:57.966 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0f192a640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0ec09e760 con 0x7ff0ec09f9c0 2024-01-24T23:22:57.966 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0f192a640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0ec0a1160 con 0x7ff0ec13b2c0 2024-01-24T23:22:57.966 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0f192a640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:57.966 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0f192a640 10 monclient(hunting): authenticate will time out at 1065.282800s 2024-01-24T23:22:57.966 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0eb7fe640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7ff0ec13b2c0 0x7ff0ec13d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:38876/0 (socket says 172.21.15.132:38876) 2024-01-24T23:22:57.966 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0eaffd640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7ff0ec09ff20 0x7ff0ec0a0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:53098/0 (socket says 172.21.15.132:53098) 2024-01-24T23:22:57.966 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0ea7fc640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7ff0ec09f9c0 0x7ff0ec139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:39320/0 (socket says 172.21.15.132:39320) 2024-01-24T23:22:57.966 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0eb7fe640 1 -- 172.21.15.132:0/8634276 learned_addr learned my addr 172.21.15.132:0/8634276 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:57.966 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 1 -- 172.21.15.132:0/8634276 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973230068 0 0) 0x7ff0ec0a1160 con 0x7ff0ec13b2c0 2024-01-24T23:22:57.966 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:22:57.966 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:57.966 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:57.966 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 10 monclient(hunting): my global_id is 4831 2024-01-24T23:22:57.966 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 1 -- 172.21.15.132:0/8634276 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0d4003530 con 0x7ff0ec13b2c0 2024-01-24T23:22:57.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 1 -- 172.21.15.132:0/8634276 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2381867502 0 0) 0x7ff0ec064700 con 0x7ff0ec09ff20 2024-01-24T23:22:57.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:22:57.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:57.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:57.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 10 monclient(hunting): my global_id is 4861 2024-01-24T23:22:57.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 1 -- 172.21.15.132:0/8634276 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0ec0a1160 con 0x7ff0ec09ff20 2024-01-24T23:22:57.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 1 -- 172.21.15.132:0/8634276 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1239570427 0 0) 0x7ff0ec09e760 con 0x7ff0ec09f9c0 2024-01-24T23:22:57.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:22:57.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:57.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:57.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 10 monclient(hunting): my global_id is 4811 2024-01-24T23:22:57.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 1 -- 172.21.15.132:0/8634276 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0ec064700 con 0x7ff0ec09f9c0 2024-01-24T23:22:57.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 1 -- 172.21.15.132:0/8634276 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2423274499 0 0) 0x7ff0d4003530 con 0x7ff0ec13b2c0 2024-01-24T23:22:57.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 1 -- 172.21.15.132:0/8634276 >> v1:172.21.15.132:6791/0 conn(0x7ff0ec09f9c0 legacy=0x7ff0ec139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:57.968 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 1 -- 172.21.15.132:0/8634276 >> v1:172.21.15.132:6789/0 conn(0x7ff0ec09ff20 legacy=0x7ff0ec0a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:57.968 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:22:57.968 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 1 monclient: found mon.noname-i 2024-01-24T23:22:57.968 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:57.968 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:22:57.968 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 1 -- 172.21.15.132:0/8634276 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec0a0a10 con 0x7ff0ec13b2c0 2024-01-24T23:22:57.968 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 10 monclient: _finish_auth 0 2024-01-24T23:22:57.968 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:22:57.968 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:57.968 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 1 -- 172.21.15.132:0/8634276 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff0dc001730 con 0x7ff0ec13b2c0 2024-01-24T23:22:57.968 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0f192a640 5 monclient: authenticate success, global_id 4831 2024-01-24T23:22:57.968 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0f192a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:57.968 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:57.969 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:22:57.969 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 10 monclient: dump: 2024-01-24T23:22:57.969 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:57.969 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:57.969 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.969 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.969 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:57.969 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:57.969 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:57.969 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:57.969 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:57.969 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:57.969 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:57.969 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:57.970 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:57.970 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:57.970 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:57.970 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:57.970 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0f192a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:57.970 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 1 -- 172.21.15.132:0/8634276 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff0dc0019c0 con 0x7ff0ec13b2c0 2024-01-24T23:22:57.970 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:57.970 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 1 -- 172.21.15.132:0/8634276 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff0dc004bf0 con 0x7ff0ec13b2c0 2024-01-24T23:22:57.970 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:57.970 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:22:57.970 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0e9ffb640 10 monclient: dump: 2024-01-24T23:22:57.970 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:57.970 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:57.970 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.971 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.971 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:57.971 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:57.971 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:57.971 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:57.971 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:57.971 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:57.971 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:57.971 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:57.971 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:57.971 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:57.971 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:57.971 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:57.971 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0f192a640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:57.972 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.953+0000 7ff0f192a640 4 set_mon_vals no callback set 2024-01-24T23:22:57.972 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0f192a640 10 monclient: shutdown 2024-01-24T23:22:57.972 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0f192a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:57.972 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0f192a640 1 -- 172.21.15.132:0/8634276 >> v1:172.21.15.132:6792/0 conn(0x7ff0ec13b2c0 legacy=0x7ff0ec13d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:57.972 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0f192a640 1 -- 172.21.15.132:0/8634276 shutdown_connections 2024-01-24T23:22:57.972 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0f192a640 1 -- 172.21.15.132:0/8634276 >> 172.21.15.132:0/8634276 conn(0x7ff0ec0a5a10 msgr2=0x7ff0ec0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:57.972 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0f192a640 1 -- 172.21.15.132:0/8634276 shutdown_connections 2024-01-24T23:22:57.972 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0f192a640 1 -- 172.21.15.132:0/8634276 wait complete. 2024-01-24T23:22:57.972 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0f192a640 10 monclient: build_initial_monmap 2024-01-24T23:22:57.972 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0f192a640 10 monclient: monmap: 2024-01-24T23:22:57.972 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:57.972 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:57.972 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:57.972 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:57.973 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:57.973 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:57.973 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:57.973 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:57.973 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:57.973 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:57.973 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:57.973 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:57.973 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:57.973 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:57.973 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:57.973 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:57.973 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0f192a640 1 Processor -- start 2024-01-24T23:22:57.973 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0f192a640 1 -- start start 2024-01-24T23:22:57.973 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0f192a640 10 monclient: init 2024-01-24T23:22:57.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0f192a640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:57.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0f192a640 10 monclient: _add_conns ranks=[0,6,4,1,7,8,3,5,2] 2024-01-24T23:22:57.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0f192a640 10 monclient(hunting): picked mon.noname-a con 0x7ff0ec09f9c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:57.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0f192a640 10 monclient(hunting): picked mon.noname-d con 0x7ff0ec09ff20 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:57.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0f192a640 10 monclient(hunting): picked mon.noname-c con 0x7ff0ec13b2c0 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:57.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0f192a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0ec09d2e0 con 0x7ff0ec09f9c0 2024-01-24T23:22:57.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0f192a640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0ec071960 con 0x7ff0ec13b2c0 2024-01-24T23:22:57.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0f192a640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0ec071680 con 0x7ff0ec09ff20 2024-01-24T23:22:57.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0f192a640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:57.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0f192a640 10 monclient(hunting): authenticate will time out at 1065.286109s 2024-01-24T23:22:57.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0eb7fe640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7ff0ec13b2c0 0x7ff0ec1d36c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:34868/0 (socket says 172.21.15.132:34868) 2024-01-24T23:22:57.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0ea7fc640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7ff0ec09ff20 0x7ff0ec09cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:33402/0 (socket says 172.21.15.132:33402) 2024-01-24T23:22:57.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0eb7fe640 1 -- 172.21.15.132:0/2980042011 learned_addr learned my addr 172.21.15.132:0/2980042011 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:57.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0eaffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff0ec09f9c0 0x7ff0ec09c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:47588/0 (socket says 172.21.15.132:47588) 2024-01-24T23:22:57.975 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099710502 0 0) 0x7ff0ec071680 con 0x7ff0ec09ff20 2024-01-24T23:22:57.975 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:57.975 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:57.975 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:57.975 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 10 monclient(hunting): my global_id is 4857 2024-01-24T23:22:57.975 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0d8003530 con 0x7ff0ec09ff20 2024-01-24T23:22:57.975 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 905442127 0 0) 0x7ff0ec071960 con 0x7ff0ec13b2c0 2024-01-24T23:22:57.975 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:57.975 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:57.975 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:57.975 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 10 monclient(hunting): my global_id is 74812 2024-01-24T23:22:57.975 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0ec071680 con 0x7ff0ec13b2c0 2024-01-24T23:22:57.975 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799754939 0 0) 0x7ff0ec09d2e0 con 0x7ff0ec09f9c0 2024-01-24T23:22:57.975 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:57.975 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:57.976 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:57.976 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 10 monclient(hunting): my global_id is 4815 2024-01-24T23:22:57.976 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0ec071960 con 0x7ff0ec09f9c0 2024-01-24T23:22:57.976 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3756568103 0 0) 0x7ff0d8003530 con 0x7ff0ec09ff20 2024-01-24T23:22:57.976 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff0ec09d2e0 con 0x7ff0ec09ff20 2024-01-24T23:22:57.976 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 167869300 0 0) 0x7ff0ec071680 con 0x7ff0ec13b2c0 2024-01-24T23:22:57.976 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff0d8003530 con 0x7ff0ec13b2c0 2024-01-24T23:22:57.976 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff0e0001220 con 0x7ff0ec09ff20 2024-01-24T23:22:57.976 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:57.976 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:22:57.976 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 10 monclient(hunting): dump: 2024-01-24T23:22:57.976 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:57.976 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:57.976 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.977 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.977 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:57.977 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:57.977 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:57.977 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:57.977 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:57.977 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:57.977 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:57.977 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:57.977 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:57.977 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:57.977 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:57.977 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:57.977 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:57.977 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff0dc004bf0 con 0x7ff0ec13b2c0 2024-01-24T23:22:57.978 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:57.978 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:22:57.978 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.957+0000 7ff0d3fff640 10 monclient(hunting): dump: 2024-01-24T23:22:57.978 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:57.978 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:57.978 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.978 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.978 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:57.978 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:57.978 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:57.978 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:57.978 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:57.978 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:57.978 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:57.978 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:57.979 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:57.979 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:57.979 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:57.979 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:57.979 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3976847196 0 0) 0x7ff0ec071960 con 0x7ff0ec09f9c0 2024-01-24T23:22:57.979 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff0ec071680 con 0x7ff0ec09f9c0 2024-01-24T23:22:57.979 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff0e40017b0 con 0x7ff0ec09f9c0 2024-01-24T23:22:57.979 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:57.979 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:22:57.979 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 10 monclient(hunting): dump: 2024-01-24T23:22:57.979 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:57.979 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:57.979 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.979 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.980 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:57.980 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:57.980 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:57.980 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:57.980 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:57.980 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:57.980 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:57.980 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:57.980 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:57.980 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:57.980 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:57.980 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:57.980 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4283433189 0 0) 0x7ff0ec09d2e0 con 0x7ff0ec09ff20 2024-01-24T23:22:57.980 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 >> v1:172.21.15.115:6791/0 conn(0x7ff0ec13b2c0 legacy=0x7ff0ec1d36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:57.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 >> v1:172.21.15.115:6789/0 conn(0x7ff0ec09f9c0 legacy=0x7ff0ec09c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:57.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 10 monclient: _finish_hunting 0 2024-01-24T23:22:57.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 1 monclient: found mon.d 2024-01-24T23:22:57.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:57.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:22:57.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec09d7c0 con 0x7ff0ec09ff20 2024-01-24T23:22:57.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 10 monclient: _finish_auth 0 2024-01-24T23:22:57.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 10 monclient: _check_auth_tickets 2024-01-24T23:22:57.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:57.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0f192a640 5 monclient: authenticate success, global_id 4857 2024-01-24T23:22:57.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0f192a640 10 monclient: _renew_subs 2024-01-24T23:22:57.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0f192a640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:22:57.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0f192a640 1 -- 172.21.15.132:0/2980042011 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0ec120c90 con 0x7ff0ec09ff20 2024-01-24T23:22:57.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0f192a640 10 monclient: _renew_subs 2024-01-24T23:22:57.982 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0f192a640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:22:57.982 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0f192a640 1 -- 172.21.15.132:0/2980042011 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0ec11fa20 con 0x7ff0ec09ff20 2024-01-24T23:22:57.982 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff0e0001490 con 0x7ff0ec09ff20 2024-01-24T23:22:57.982 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:57.982 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0e97fa640 4 set_mon_vals no callback set 2024-01-24T23:22:57.982 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff0e00016b0 con 0x7ff0ec09ff20 2024-01-24T23:22:57.982 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:57.982 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:22:57.982 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 10 monclient: dump: 2024-01-24T23:22:57.982 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:57.982 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:57.982 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.982 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:57.982 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:57.982 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:57.983 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:57.983 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:57.983 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:57.983 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:57.983 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:57.983 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:57.983 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:57.983 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:57.983 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:57.983 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:57.983 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0f192a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:57.983 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0f192a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:57.983 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0f192a640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:22:57.983 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0f192a640 1 -- 172.21.15.132:0/2980042011 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0cc005090 con 0x7ff0ec09ff20 2024-01-24T23:22:57.983 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7ff0e000f780 con 0x7ff0ec09ff20 2024-01-24T23:22:57.984 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(38..38 src has 1..38) v4 ==== 3598+0+0 (unknown 3181584953 0 0) 0x7ff0e0043480 con 0x7ff0ec09ff20 2024-01-24T23:22:57.984 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.961+0000 7ff0eaffd640 10 monclient: get_auth_request con 0x7ff0d8037bd0 auth_method 0 2024-01-24T23:22:57.984 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.973+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7ff0e00143b0 con 0x7ff0ec09ff20 2024-01-24T23:22:57.984 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.973+0000 7ff0d3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:57.984 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:57.973+0000 7ff0d3fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:58.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.133+0000 7ff0f192a640 10 start_mon_command cmd=[{"prefix": "osd crush dump"}] 2024-01-24T23:22:58.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.133+0000 7ff0f192a640 10 monclient: _send_command 2 [{"prefix": "osd crush dump"}] 2024-01-24T23:22:58.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.133+0000 7ff0f192a640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:22:58.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.133+0000 7ff0f192a640 1 -- 172.21.15.132:0/2980042011 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush dump"} v 0) v1 -- 0x7ff0cc005340 con 0x7ff0ec09ff20 2024-01-24T23:22:58.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.133+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush dump"}]=0 v38) v1 ==== 62+0+7238 (unknown 2541289373 0 1841131444) 0x7ff0e00143b0 con 0x7ff0ec09ff20 2024-01-24T23:22:58.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.133+0000 7ff0d3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush dump"}] 2024-01-24T23:22:58.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.133+0000 7ff0d3fff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:22:58.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.225+0000 7ff0f192a640 10 start_mon_command cmd=[{"prefix": "osd crush dump"}] 2024-01-24T23:22:58.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.225+0000 7ff0f192a640 10 monclient: _send_command 3 [{"prefix": "osd crush dump"}] 2024-01-24T23:22:58.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.225+0000 7ff0f192a640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:22:58.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.225+0000 7ff0f192a640 1 -- 172.21.15.132:0/2980042011 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush dump"} v 0) v1 -- 0x7ff0cc001130 con 0x7ff0ec09ff20 2024-01-24T23:22:58.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.233+0000 7ff0d3fff640 1 -- 172.21.15.132:0/2980042011 <== mon.6 v1:172.21.15.115:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush dump"}]=0 v38) v1 ==== 62+0+7238 (unknown 2541289373 0 1841131444) 0x7ff0e00143b0 con 0x7ff0ec09ff20 2024-01-24T23:22:58.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.233+0000 7ff0d3fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush dump"}] 2024-01-24T23:22:58.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.233+0000 7ff0d3fff640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:22:58.232 INFO:tasks.workunit.client.0.smithi132.stdout:{ 2024-01-24T23:22:58.232 INFO:tasks.workunit.client.0.smithi132.stdout: "devices": [ 2024-01-24T23:22:58.232 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.232 INFO:tasks.workunit.client.0.smithi132.stdout: "id": 0, 2024-01-24T23:22:58.232 INFO:tasks.workunit.client.0.smithi132.stdout: "name": "osd.0", 2024-01-24T23:22:58.232 INFO:tasks.workunit.client.0.smithi132.stdout: "class": "ssd" 2024-01-24T23:22:58.232 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.232 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.232 INFO:tasks.workunit.client.0.smithi132.stdout: "id": 1, 2024-01-24T23:22:58.232 INFO:tasks.workunit.client.0.smithi132.stdout: "name": "osd.1", 2024-01-24T23:22:58.233 INFO:tasks.workunit.client.0.smithi132.stdout: "class": "ssd" 2024-01-24T23:22:58.233 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.233 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.233 INFO:tasks.workunit.client.0.smithi132.stdout: "id": 2, 2024-01-24T23:22:58.233 INFO:tasks.workunit.client.0.smithi132.stdout: "name": "osd.2", 2024-01-24T23:22:58.233 INFO:tasks.workunit.client.0.smithi132.stdout: "class": "ssd" 2024-01-24T23:22:58.233 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.233 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.233 INFO:tasks.workunit.client.0.smithi132.stdout: "id": 3, 2024-01-24T23:22:58.233 INFO:tasks.workunit.client.0.smithi132.stdout: "name": "osd.3", 2024-01-24T23:22:58.233 INFO:tasks.workunit.client.0.smithi132.stdout: "class": "ssd" 2024-01-24T23:22:58.233 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.233 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.234 INFO:tasks.workunit.client.0.smithi132.stdout: "id": 4, 2024-01-24T23:22:58.234 INFO:tasks.workunit.client.0.smithi132.stdout: "name": "osd.4", 2024-01-24T23:22:58.234 INFO:tasks.workunit.client.0.smithi132.stdout: "class": "ssd" 2024-01-24T23:22:58.234 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.234 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.234 INFO:tasks.workunit.client.0.smithi132.stdout: "id": 5, 2024-01-24T23:22:58.234 INFO:tasks.workunit.client.0.smithi132.stdout: "name": "osd.5", 2024-01-24T23:22:58.234 INFO:tasks.workunit.client.0.smithi132.stdout: "class": "ssd" 2024-01-24T23:22:58.234 INFO:tasks.workunit.client.0.smithi132.stdout: } 2024-01-24T23:22:58.234 INFO:tasks.workunit.client.0.smithi132.stdout: ], 2024-01-24T23:22:58.234 INFO:tasks.workunit.client.0.smithi132.stdout: "types": [ 2024-01-24T23:22:58.234 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.234 INFO:tasks.workunit.client.0.smithi132.stdout: "type_id": 0, 2024-01-24T23:22:58.234 INFO:tasks.workunit.client.0.smithi132.stdout: "name": "osd" 2024-01-24T23:22:58.234 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.235 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.235 INFO:tasks.workunit.client.0.smithi132.stdout: "type_id": 1, 2024-01-24T23:22:58.235 INFO:tasks.workunit.client.0.smithi132.stdout: "name": "host" 2024-01-24T23:22:58.235 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.235 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.235 INFO:tasks.workunit.client.0.smithi132.stdout: "type_id": 2, 2024-01-24T23:22:58.235 INFO:tasks.workunit.client.0.smithi132.stdout: "name": "chassis" 2024-01-24T23:22:58.235 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.235 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.235 INFO:tasks.workunit.client.0.smithi132.stdout: "type_id": 3, 2024-01-24T23:22:58.235 INFO:tasks.workunit.client.0.smithi132.stdout: "name": "rack" 2024-01-24T23:22:58.235 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.235 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.235 INFO:tasks.workunit.client.0.smithi132.stdout: "type_id": 4, 2024-01-24T23:22:58.236 INFO:tasks.workunit.client.0.smithi132.stdout: "name": "row" 2024-01-24T23:22:58.236 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.236 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.236 INFO:tasks.workunit.client.0.smithi132.stdout: "type_id": 5, 2024-01-24T23:22:58.236 INFO:tasks.workunit.client.0.smithi132.stdout: "name": "pdu" 2024-01-24T23:22:58.236 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.236 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.236 INFO:tasks.workunit.client.0.smithi132.stdout: "type_id": 6, 2024-01-24T23:22:58.236 INFO:tasks.workunit.client.0.smithi132.stdout: "name": "pod" 2024-01-24T23:22:58.236 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.236 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.236 INFO:tasks.workunit.client.0.smithi132.stdout: "type_id": 7, 2024-01-24T23:22:58.236 INFO:tasks.workunit.client.0.smithi132.stdout: "name": "room" 2024-01-24T23:22:58.237 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.237 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.237 INFO:tasks.workunit.client.0.smithi132.stdout: "type_id": 8, 2024-01-24T23:22:58.237 INFO:tasks.workunit.client.0.smithi132.stdout: "name": "datacenter" 2024-01-24T23:22:58.237 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.237 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.237 INFO:tasks.workunit.client.0.smithi132.stdout: "type_id": 9, 2024-01-24T23:22:58.237 INFO:tasks.workunit.client.0.smithi132.stdout: "name": "zone" 2024-01-24T23:22:58.237 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.237 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.237 INFO:tasks.workunit.client.0.smithi132.stdout: "type_id": 10, 2024-01-24T23:22:58.237 INFO:tasks.workunit.client.0.smithi132.stdout: "name": "region" 2024-01-24T23:22:58.237 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.237 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.238 INFO:tasks.workunit.client.0.smithi132.stdout: "type_id": 11, 2024-01-24T23:22:58.238 INFO:tasks.workunit.client.0.smithi132.stdout: "name": "root" 2024-01-24T23:22:58.238 INFO:tasks.workunit.client.0.smithi132.stdout: } 2024-01-24T23:22:58.238 INFO:tasks.workunit.client.0.smithi132.stdout: ], 2024-01-24T23:22:58.238 INFO:tasks.workunit.client.0.smithi132.stdout: "buckets": [ 2024-01-24T23:22:58.238 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.238 INFO:tasks.workunit.client.0.smithi132.stdout: "id": -1, 2024-01-24T23:22:58.238 INFO:tasks.workunit.client.0.smithi132.stdout: "name": "default", 2024-01-24T23:22:58.238 INFO:tasks.workunit.client.0.smithi132.stdout: "type_id": 11, 2024-01-24T23:22:58.238 INFO:tasks.workunit.client.0.smithi132.stdout: "type_name": "root", 2024-01-24T23:22:58.238 INFO:tasks.workunit.client.0.smithi132.stdout: "weight": 34560, 2024-01-24T23:22:58.238 INFO:tasks.workunit.client.0.smithi132.stdout: "alg": "straw2", 2024-01-24T23:22:58.238 INFO:tasks.workunit.client.0.smithi132.stdout: "hash": "rjenkins1", 2024-01-24T23:22:58.239 INFO:tasks.workunit.client.0.smithi132.stdout: "items": [ 2024-01-24T23:22:58.239 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.239 INFO:tasks.workunit.client.0.smithi132.stdout: "id": -3, 2024-01-24T23:22:58.239 INFO:tasks.workunit.client.0.smithi132.stdout: "weight": 17280, 2024-01-24T23:22:58.239 INFO:tasks.workunit.client.0.smithi132.stdout: "pos": 0 2024-01-24T23:22:58.239 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.239 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.239 INFO:tasks.workunit.client.0.smithi132.stdout: "id": -5, 2024-01-24T23:22:58.239 INFO:tasks.workunit.client.0.smithi132.stdout: "weight": 17280, 2024-01-24T23:22:58.239 INFO:tasks.workunit.client.0.smithi132.stdout: "pos": 1 2024-01-24T23:22:58.239 INFO:tasks.workunit.client.0.smithi132.stdout: } 2024-01-24T23:22:58.239 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:22:58.239 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.239 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.240 INFO:tasks.workunit.client.0.smithi132.stdout: "id": -2, 2024-01-24T23:22:58.240 INFO:tasks.workunit.client.0.smithi132.stdout: "name": "default~ssd", 2024-01-24T23:22:58.240 INFO:tasks.workunit.client.0.smithi132.stdout: "type_id": 11, 2024-01-24T23:22:58.240 INFO:tasks.workunit.client.0.smithi132.stdout: "type_name": "root", 2024-01-24T23:22:58.240 INFO:tasks.workunit.client.0.smithi132.stdout: "weight": 34560, 2024-01-24T23:22:58.240 INFO:tasks.workunit.client.0.smithi132.stdout: "alg": "straw2", 2024-01-24T23:22:58.240 INFO:tasks.workunit.client.0.smithi132.stdout: "hash": "rjenkins1", 2024-01-24T23:22:58.240 INFO:tasks.workunit.client.0.smithi132.stdout: "items": [ 2024-01-24T23:22:58.240 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.240 INFO:tasks.workunit.client.0.smithi132.stdout: "id": -4, 2024-01-24T23:22:58.240 INFO:tasks.workunit.client.0.smithi132.stdout: "weight": 17280, 2024-01-24T23:22:58.240 INFO:tasks.workunit.client.0.smithi132.stdout: "pos": 0 2024-01-24T23:22:58.240 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.240 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.240 INFO:tasks.workunit.client.0.smithi132.stdout: "id": -6, 2024-01-24T23:22:58.241 INFO:tasks.workunit.client.0.smithi132.stdout: "weight": 17280, 2024-01-24T23:22:58.241 INFO:tasks.workunit.client.0.smithi132.stdout: "pos": 1 2024-01-24T23:22:58.241 INFO:tasks.workunit.client.0.smithi132.stdout: } 2024-01-24T23:22:58.241 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:22:58.241 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.241 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.241 INFO:tasks.workunit.client.0.smithi132.stdout: "id": -3, 2024-01-24T23:22:58.241 INFO:tasks.workunit.client.0.smithi132.stdout: "name": "smithi132", 2024-01-24T23:22:58.241 INFO:tasks.workunit.client.0.smithi132.stdout: "type_id": 1, 2024-01-24T23:22:58.241 INFO:tasks.workunit.client.0.smithi132.stdout: "type_name": "host", 2024-01-24T23:22:58.241 INFO:tasks.workunit.client.0.smithi132.stdout: "weight": 17280, 2024-01-24T23:22:58.241 INFO:tasks.workunit.client.0.smithi132.stdout: "alg": "straw2", 2024-01-24T23:22:58.241 INFO:tasks.workunit.client.0.smithi132.stdout: "hash": "rjenkins1", 2024-01-24T23:22:58.241 INFO:tasks.workunit.client.0.smithi132.stdout: "items": [ 2024-01-24T23:22:58.242 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.242 INFO:tasks.workunit.client.0.smithi132.stdout: "id": 5, 2024-01-24T23:22:58.242 INFO:tasks.workunit.client.0.smithi132.stdout: "weight": 5760, 2024-01-24T23:22:58.242 INFO:tasks.workunit.client.0.smithi132.stdout: "pos": 0 2024-01-24T23:22:58.242 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.242 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.242 INFO:tasks.workunit.client.0.smithi132.stdout: "id": 3, 2024-01-24T23:22:58.242 INFO:tasks.workunit.client.0.smithi132.stdout: "weight": 5760, 2024-01-24T23:22:58.242 INFO:tasks.workunit.client.0.smithi132.stdout: "pos": 1 2024-01-24T23:22:58.242 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.242 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.242 INFO:tasks.workunit.client.0.smithi132.stdout: "id": 4, 2024-01-24T23:22:58.242 INFO:tasks.workunit.client.0.smithi132.stdout: "weight": 5760, 2024-01-24T23:22:58.242 INFO:tasks.workunit.client.0.smithi132.stdout: "pos": 2 2024-01-24T23:22:58.243 INFO:tasks.workunit.client.0.smithi132.stdout: } 2024-01-24T23:22:58.243 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:22:58.243 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.243 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.243 INFO:tasks.workunit.client.0.smithi132.stdout: "id": -4, 2024-01-24T23:22:58.243 INFO:tasks.workunit.client.0.smithi132.stdout: "name": "smithi132~ssd", 2024-01-24T23:22:58.243 INFO:tasks.workunit.client.0.smithi132.stdout: "type_id": 1, 2024-01-24T23:22:58.243 INFO:tasks.workunit.client.0.smithi132.stdout: "type_name": "host", 2024-01-24T23:22:58.243 INFO:tasks.workunit.client.0.smithi132.stdout: "weight": 17280, 2024-01-24T23:22:58.243 INFO:tasks.workunit.client.0.smithi132.stdout: "alg": "straw2", 2024-01-24T23:22:58.243 INFO:tasks.workunit.client.0.smithi132.stdout: "hash": "rjenkins1", 2024-01-24T23:22:58.243 INFO:tasks.workunit.client.0.smithi132.stdout: "items": [ 2024-01-24T23:22:58.243 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.243 INFO:tasks.workunit.client.0.smithi132.stdout: "id": 5, 2024-01-24T23:22:58.243 INFO:tasks.workunit.client.0.smithi132.stdout: "weight": 5760, 2024-01-24T23:22:58.244 INFO:tasks.workunit.client.0.smithi132.stdout: "pos": 0 2024-01-24T23:22:58.244 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.244 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.244 INFO:tasks.workunit.client.0.smithi132.stdout: "id": 3, 2024-01-24T23:22:58.244 INFO:tasks.workunit.client.0.smithi132.stdout: "weight": 5760, 2024-01-24T23:22:58.244 INFO:tasks.workunit.client.0.smithi132.stdout: "pos": 1 2024-01-24T23:22:58.244 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.244 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.244 INFO:tasks.workunit.client.0.smithi132.stdout: "id": 4, 2024-01-24T23:22:58.244 INFO:tasks.workunit.client.0.smithi132.stdout: "weight": 5760, 2024-01-24T23:22:58.244 INFO:tasks.workunit.client.0.smithi132.stdout: "pos": 2 2024-01-24T23:22:58.244 INFO:tasks.workunit.client.0.smithi132.stdout: } 2024-01-24T23:22:58.244 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:22:58.244 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.244 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.245 INFO:tasks.workunit.client.0.smithi132.stdout: "id": -5, 2024-01-24T23:22:58.245 INFO:tasks.workunit.client.0.smithi132.stdout: "name": "smithi115", 2024-01-24T23:22:58.245 INFO:tasks.workunit.client.0.smithi132.stdout: "type_id": 1, 2024-01-24T23:22:58.245 INFO:tasks.workunit.client.0.smithi132.stdout: "type_name": "host", 2024-01-24T23:22:58.245 INFO:tasks.workunit.client.0.smithi132.stdout: "weight": 17280, 2024-01-24T23:22:58.245 INFO:tasks.workunit.client.0.smithi132.stdout: "alg": "straw2", 2024-01-24T23:22:58.245 INFO:tasks.workunit.client.0.smithi132.stdout: "hash": "rjenkins1", 2024-01-24T23:22:58.245 INFO:tasks.workunit.client.0.smithi132.stdout: "items": [ 2024-01-24T23:22:58.245 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.245 INFO:tasks.workunit.client.0.smithi132.stdout: "id": 2, 2024-01-24T23:22:58.245 INFO:tasks.workunit.client.0.smithi132.stdout: "weight": 5760, 2024-01-24T23:22:58.245 INFO:tasks.workunit.client.0.smithi132.stdout: "pos": 0 2024-01-24T23:22:58.245 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.245 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.245 INFO:tasks.workunit.client.0.smithi132.stdout: "id": 1, 2024-01-24T23:22:58.246 INFO:tasks.workunit.client.0.smithi132.stdout: "weight": 5760, 2024-01-24T23:22:58.246 INFO:tasks.workunit.client.0.smithi132.stdout: "pos": 1 2024-01-24T23:22:58.246 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.246 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.246 INFO:tasks.workunit.client.0.smithi132.stdout: "id": 0, 2024-01-24T23:22:58.246 INFO:tasks.workunit.client.0.smithi132.stdout: "weight": 5760, 2024-01-24T23:22:58.246 INFO:tasks.workunit.client.0.smithi132.stdout: "pos": 2 2024-01-24T23:22:58.246 INFO:tasks.workunit.client.0.smithi132.stdout: } 2024-01-24T23:22:58.246 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:22:58.246 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.246 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.246 INFO:tasks.workunit.client.0.smithi132.stdout: "id": -6, 2024-01-24T23:22:58.246 INFO:tasks.workunit.client.0.smithi132.stdout: "name": "smithi115~ssd", 2024-01-24T23:22:58.246 INFO:tasks.workunit.client.0.smithi132.stdout: "type_id": 1, 2024-01-24T23:22:58.247 INFO:tasks.workunit.client.0.smithi132.stdout: "type_name": "host", 2024-01-24T23:22:58.247 INFO:tasks.workunit.client.0.smithi132.stdout: "weight": 17280, 2024-01-24T23:22:58.247 INFO:tasks.workunit.client.0.smithi132.stdout: "alg": "straw2", 2024-01-24T23:22:58.247 INFO:tasks.workunit.client.0.smithi132.stdout: "hash": "rjenkins1", 2024-01-24T23:22:58.247 INFO:tasks.workunit.client.0.smithi132.stdout: "items": [ 2024-01-24T23:22:58.247 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.247 INFO:tasks.workunit.client.0.smithi132.stdout: "id": 2, 2024-01-24T23:22:58.247 INFO:tasks.workunit.client.0.smithi132.stdout: "weight": 5760, 2024-01-24T23:22:58.247 INFO:tasks.workunit.client.0.smithi132.stdout: "pos": 0 2024-01-24T23:22:58.247 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.247 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.247 INFO:tasks.workunit.client.0.smithi132.stdout: "id": 1, 2024-01-24T23:22:58.247 INFO:tasks.workunit.client.0.smithi132.stdout: "weight": 5760, 2024-01-24T23:22:58.247 INFO:tasks.workunit.client.0.smithi132.stdout: "pos": 1 2024-01-24T23:22:58.247 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.248 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.248 INFO:tasks.workunit.client.0.smithi132.stdout: "id": 0, 2024-01-24T23:22:58.248 INFO:tasks.workunit.client.0.smithi132.stdout: "weight": 5760, 2024-01-24T23:22:58.248 INFO:tasks.workunit.client.0.smithi132.stdout: "pos": 2 2024-01-24T23:22:58.248 INFO:tasks.workunit.client.0.smithi132.stdout: } 2024-01-24T23:22:58.248 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:22:58.248 INFO:tasks.workunit.client.0.smithi132.stdout: } 2024-01-24T23:22:58.248 INFO:tasks.workunit.client.0.smithi132.stdout: ], 2024-01-24T23:22:58.248 INFO:tasks.workunit.client.0.smithi132.stdout: "rules": [ 2024-01-24T23:22:58.248 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.248 INFO:tasks.workunit.client.0.smithi132.stdout: "rule_id": 0, 2024-01-24T23:22:58.248 INFO:tasks.workunit.client.0.smithi132.stdout: "rule_name": "replicated_rule", 2024-01-24T23:22:58.248 INFO:tasks.workunit.client.0.smithi132.stdout: "type": 1, 2024-01-24T23:22:58.248 INFO:tasks.workunit.client.0.smithi132.stdout: "steps": [ 2024-01-24T23:22:58.249 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.249 INFO:tasks.workunit.client.0.smithi132.stdout: "op": "take", 2024-01-24T23:22:58.249 INFO:tasks.workunit.client.0.smithi132.stdout: "item": -1, 2024-01-24T23:22:58.249 INFO:tasks.workunit.client.0.smithi132.stdout: "item_name": "default" 2024-01-24T23:22:58.249 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.249 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.249 INFO:tasks.workunit.client.0.smithi132.stdout: "op": "choose_firstn", 2024-01-24T23:22:58.249 INFO:tasks.workunit.client.0.smithi132.stdout: "num": 0, 2024-01-24T23:22:58.249 INFO:tasks.workunit.client.0.smithi132.stdout: "type": "osd" 2024-01-24T23:22:58.249 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.249 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.250 INFO:tasks.workunit.client.0.smithi132.stdout: "op": "emit" 2024-01-24T23:22:58.250 INFO:tasks.workunit.client.0.smithi132.stdout: } 2024-01-24T23:22:58.250 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:22:58.250 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.250 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.250 INFO:tasks.workunit.client.0.smithi132.stdout: "rule_id": 1, 2024-01-24T23:22:58.250 INFO:tasks.workunit.client.0.smithi132.stdout: "rule_name": "erasure-code", 2024-01-24T23:22:58.250 INFO:tasks.workunit.client.0.smithi132.stdout: "type": 3, 2024-01-24T23:22:58.250 INFO:tasks.workunit.client.0.smithi132.stdout: "steps": [ 2024-01-24T23:22:58.250 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.250 INFO:tasks.workunit.client.0.smithi132.stdout: "op": "set_chooseleaf_tries", 2024-01-24T23:22:58.250 INFO:tasks.workunit.client.0.smithi132.stdout: "num": 5 2024-01-24T23:22:58.250 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.250 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.251 INFO:tasks.workunit.client.0.smithi132.stdout: "op": "set_choose_tries", 2024-01-24T23:22:58.251 INFO:tasks.workunit.client.0.smithi132.stdout: "num": 100 2024-01-24T23:22:58.251 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.251 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.251 INFO:tasks.workunit.client.0.smithi132.stdout: "op": "take", 2024-01-24T23:22:58.251 INFO:tasks.workunit.client.0.smithi132.stdout: "item": -1, 2024-01-24T23:22:58.251 INFO:tasks.workunit.client.0.smithi132.stdout: "item_name": "default" 2024-01-24T23:22:58.251 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.251 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.251 INFO:tasks.workunit.client.0.smithi132.stdout: "op": "choose_indep", 2024-01-24T23:22:58.251 INFO:tasks.workunit.client.0.smithi132.stdout: "num": 0, 2024-01-24T23:22:58.251 INFO:tasks.workunit.client.0.smithi132.stdout: "type": "osd" 2024-01-24T23:22:58.251 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.251 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.252 INFO:tasks.workunit.client.0.smithi132.stdout: "op": "emit" 2024-01-24T23:22:58.252 INFO:tasks.workunit.client.0.smithi132.stdout: } 2024-01-24T23:22:58.252 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:22:58.252 INFO:tasks.workunit.client.0.smithi132.stdout: } 2024-01-24T23:22:58.252 INFO:tasks.workunit.client.0.smithi132.stdout: ], 2024-01-24T23:22:58.252 INFO:tasks.workunit.client.0.smithi132.stdout: "tunables": { 2024-01-24T23:22:58.252 INFO:tasks.workunit.client.0.smithi132.stdout: "choose_local_tries": 0, 2024-01-24T23:22:58.252 INFO:tasks.workunit.client.0.smithi132.stdout: "choose_local_fallback_tries": 0, 2024-01-24T23:22:58.252 INFO:tasks.workunit.client.0.smithi132.stdout: "choose_total_tries": 50, 2024-01-24T23:22:58.252 INFO:tasks.workunit.client.0.smithi132.stdout: "chooseleaf_descend_once": 1, 2024-01-24T23:22:58.252 INFO:tasks.workunit.client.0.smithi132.stdout: "chooseleaf_vary_r": 1, 2024-01-24T23:22:58.252 INFO:tasks.workunit.client.0.smithi132.stdout: "chooseleaf_stable": 1, 2024-01-24T23:22:58.252 INFO:tasks.workunit.client.0.smithi132.stdout: "straw_calc_version": 1, 2024-01-24T23:22:58.252 INFO:tasks.workunit.client.0.smithi132.stdout: "allowed_bucket_algs": 54, 2024-01-24T23:22:58.252 INFO:tasks.workunit.client.0.smithi132.stdout: "profile": "jewel", 2024-01-24T23:22:58.253 INFO:tasks.workunit.client.0.smithi132.stdout: "optimal_tunables": 1, 2024-01-24T23:22:58.253 INFO:tasks.workunit.client.0.smithi132.stdout: "legacy_tunables": 0, 2024-01-24T23:22:58.253 INFO:tasks.workunit.client.0.smithi132.stdout: "minimum_required_version": "jewel", 2024-01-24T23:22:58.253 INFO:tasks.workunit.client.0.smithi132.stdout: "require_feature_tunables": 1, 2024-01-24T23:22:58.253 INFO:tasks.workunit.client.0.smithi132.stdout: "require_feature_tunables2": 1, 2024-01-24T23:22:58.253 INFO:tasks.workunit.client.0.smithi132.stdout: "has_v2_rules": 1, 2024-01-24T23:22:58.253 INFO:tasks.workunit.client.0.smithi132.stdout: "require_feature_tunables3": 1, 2024-01-24T23:22:58.253 INFO:tasks.workunit.client.0.smithi132.stdout: "has_v3_rules": 0, 2024-01-24T23:22:58.253 INFO:tasks.workunit.client.0.smithi132.stdout: "has_v4_buckets": 1, 2024-01-24T23:22:58.253 INFO:tasks.workunit.client.0.smithi132.stdout: "require_feature_tunables5": 1, 2024-01-24T23:22:58.253 INFO:tasks.workunit.client.0.smithi132.stdout: "has_v5_rules": 0 2024-01-24T23:22:58.253 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.253 INFO:tasks.workunit.client.0.smithi132.stdout: "choose_args": {} 2024-01-24T23:22:58.254 INFO:tasks.workunit.client.0.smithi132.stdout:} 2024-01-24T23:22:58.254 INFO:tasks.workunit.client.0.smithi132.stdout: 2024-01-24T23:22:58.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.233+0000 7ff0f192a640 1 -- 172.21.15.132:0/2980042011 >> v1:172.21.15.132:6812/1835401922 conn(0x7ff0d8037bd0 legacy=0x7ff0d803a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:58.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.233+0000 7ff0f192a640 10 monclient: shutdown 2024-01-24T23:22:58.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.233+0000 7ff0f192a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:58.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.233+0000 7ff0f192a640 1 -- 172.21.15.132:0/2980042011 >> v1:172.21.15.115:6792/0 conn(0x7ff0ec09ff20 legacy=0x7ff0ec09cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:58.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.233+0000 7ff0f192a640 1 -- 172.21.15.132:0/2980042011 shutdown_connections 2024-01-24T23:22:58.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.233+0000 7ff0f192a640 1 -- 172.21.15.132:0/2980042011 >> 172.21.15.132:0/2980042011 conn(0x7ff0ec0a5a10 msgr2=0x7ff0ec0a2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:58.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.233+0000 7ff0f192a640 1 -- 172.21.15.132:0/2980042011 shutdown_connections 2024-01-24T23:22:58.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.233+0000 7ff0f192a640 1 -- 172.21.15.132:0/2980042011 wait complete. 2024-01-24T23:22:58.255 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule dump 2024-01-24T23:22:58.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 10 monclient: get_monmap_and_config 2024-01-24T23:22:58.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 10 monclient: build_initial_monmap 2024-01-24T23:22:58.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 10 monclient: monmap: 2024-01-24T23:22:58.360 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:58.360 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:58.360 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:58.361730+0000 2024-01-24T23:22:58.360 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:58.361730+0000 2024-01-24T23:22:58.360 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:58.361 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:58.361 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:58.361 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:58.361 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:58.361 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:58.361 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:58.361 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:58.361 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:58.361 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:58.361 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:58.361 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:58.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 1 Processor -- start 2024-01-24T23:22:58.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 1 -- start start 2024-01-24T23:22:58.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 10 monclient: init 2024-01-24T23:22:58.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:58.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 10 monclient: _add_conns ranks=[4,7,6,5,2,8,3,0,1] 2024-01-24T23:22:58.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 10 monclient(hunting): picked mon.noname-c con 0x7fee4c136cb0 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:58.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 10 monclient(hunting): picked mon.noname-i con 0x7fee4c1377a0 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:58.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 10 monclient(hunting): picked mon.noname-d con 0x7fee4c09bcc0 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:58.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee4c064700 con 0x7fee4c136cb0 2024-01-24T23:22:58.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee4c09b650 con 0x7fee4c09bcc0 2024-01-24T23:22:58.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee4c09c110 con 0x7fee4c1377a0 2024-01-24T23:22:58.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:58.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 10 monclient(hunting): authenticate will time out at 1065.686364s 2024-01-24T23:22:58.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4affd640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fee4c1377a0 0x7fee4c13c1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:38888/0 (socket says 172.21.15.132:38888) 2024-01-24T23:22:58.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4affd640 1 -- 172.21.15.132:0/1033218284 learned_addr learned my addr 172.21.15.132:0/1033218284 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:58.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 1 -- 172.21.15.132:0/1033218284 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3514434933 0 0) 0x7fee4c09c110 con 0x7fee4c1377a0 2024-01-24T23:22:58.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:58.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:58.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:58.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 10 monclient(hunting): my global_id is 4840 2024-01-24T23:22:58.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 1 -- 172.21.15.132:0/1033218284 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee38003530 con 0x7fee4c1377a0 2024-01-24T23:22:58.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 1 -- 172.21.15.132:0/1033218284 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1841405697 0 0) 0x7fee38003530 con 0x7fee4c1377a0 2024-01-24T23:22:58.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 1 -- 172.21.15.132:0/1033218284 >> v1:172.21.15.115:6792/0 conn(0x7fee4c09bcc0 legacy=0x7fee4c13f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:58.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 1 -- 172.21.15.132:0/1033218284 >> v1:172.21.15.115:6791/0 conn(0x7fee4c136cb0 legacy=0x7fee4c137090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:58.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:22:58.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 1 monclient: found mon.noname-i 2024-01-24T23:22:58.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:58.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:22:58.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 1 -- 172.21.15.132:0/1033218284 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c09b000 con 0x7fee4c1377a0 2024-01-24T23:22:58.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 10 monclient: _finish_auth 0 2024-01-24T23:22:58.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:22:58.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:58.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 5 monclient: authenticate success, global_id 4840 2024-01-24T23:22:58.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:58.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 1 -- 172.21.15.132:0/1033218284 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fee340017c0 con 0x7fee4c1377a0 2024-01-24T23:22:58.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:58.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:22:58.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 10 monclient: dump: 2024-01-24T23:22:58.370 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:58.371 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:58.371 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:58.371 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:58.371 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:58.371 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:58.371 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:58.371 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:58.371 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:58.371 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:58.371 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:58.371 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:58.371 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:58.371 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:58.371 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:58.372 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:58.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:58.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 1 -- 172.21.15.132:0/1033218284 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fee34001ae0 con 0x7fee4c1377a0 2024-01-24T23:22:58.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:58.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 1 -- 172.21.15.132:0/1033218284 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fee34004bf0 con 0x7fee4c1377a0 2024-01-24T23:22:58.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:58.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 4 set_mon_vals no callback set 2024-01-24T23:22:58.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:58.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:22:58.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee4a7fc640 10 monclient: dump: 2024-01-24T23:22:58.372 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:58.372 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:58.372 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:58.373 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:58.373 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:58.373 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:58.373 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:58.373 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:58.373 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:58.373 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:58.373 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:58.373 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:58.373 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:58.373 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:58.373 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:58.373 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:58.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 10 monclient: shutdown 2024-01-24T23:22:58.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:58.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 1 -- 172.21.15.132:0/1033218284 >> v1:172.21.15.132:6792/0 conn(0x7fee4c1377a0 legacy=0x7fee4c13c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:58.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 1 -- 172.21.15.132:0/1033218284 shutdown_connections 2024-01-24T23:22:58.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 1 -- 172.21.15.132:0/1033218284 >> 172.21.15.132:0/1033218284 conn(0x7fee4c0a9c10 msgr2=0x7fee4c0a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:58.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 1 -- 172.21.15.132:0/1033218284 shutdown_connections 2024-01-24T23:22:58.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.357+0000 7fee51cb6640 1 -- 172.21.15.132:0/1033218284 wait complete. 2024-01-24T23:22:58.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 10 monclient: build_initial_monmap 2024-01-24T23:22:58.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 10 monclient: monmap: 2024-01-24T23:22:58.374 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:58.374 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:58.374 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:58.374 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:58.374 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:58.374 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:58.374 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:58.375 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:58.375 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:58.375 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:58.375 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:58.375 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:58.375 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:58.375 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:58.375 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:58.375 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:58.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 1 Processor -- start 2024-01-24T23:22:58.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 1 -- start start 2024-01-24T23:22:58.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 10 monclient: init 2024-01-24T23:22:58.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:58.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 10 monclient: _add_conns ranks=[4,3,5,6,2,1,7,8,0] 2024-01-24T23:22:58.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 10 monclient(hunting): picked mon.noname-c con 0x7fee4c09bcc0 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:58.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 10 monclient(hunting): picked mon.noname-g con 0x7fee4c136cb0 addr v1:172.21.15.132:6790/0 2024-01-24T23:22:58.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 10 monclient(hunting): picked mon.noname-h con 0x7fee4c1377a0 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:58.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee4c1267b0 con 0x7fee4c136cb0 2024-01-24T23:22:58.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee4c124960 con 0x7fee4c09bcc0 2024-01-24T23:22:58.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee4c09c6a0 con 0x7fee4c1377a0 2024-01-24T23:22:58.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:58.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 10 monclient(hunting): authenticate will time out at 1065.689408s 2024-01-24T23:22:58.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee4bfff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fee4c1377a0 0x7fee4c1d3e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:39326/0 (socket says 172.21.15.132:39326) 2024-01-24T23:22:58.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee4bfff640 1 -- 172.21.15.132:0/2504231739 learned_addr learned my addr 172.21.15.132:0/2504231739 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:58.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee4affd640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fee4c136cb0 0x7fee4c1d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:59918/0 (socket says 172.21.15.132:59918) 2024-01-24T23:22:58.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 1 -- 172.21.15.132:0/2504231739 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153068590 0 0) 0x7fee4c09c6a0 con 0x7fee4c1377a0 2024-01-24T23:22:58.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:58.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:58.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:58.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient(hunting): my global_id is 4820 2024-01-24T23:22:58.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 1 -- 172.21.15.132:0/2504231739 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee2c003530 con 0x7fee4c1377a0 2024-01-24T23:22:58.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 1 -- 172.21.15.132:0/2504231739 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 309379823 0 0) 0x7fee4c1267b0 con 0x7fee4c136cb0 2024-01-24T23:22:58.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:58.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:58.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:58.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient(hunting): my global_id is 4764 2024-01-24T23:22:58.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 1 -- 172.21.15.132:0/2504231739 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee4c09c6a0 con 0x7fee4c136cb0 2024-01-24T23:22:58.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 1 -- 172.21.15.132:0/2504231739 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3855624578 0 0) 0x7fee4c124960 con 0x7fee4c09bcc0 2024-01-24T23:22:58.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:58.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:58.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:58.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient(hunting): my global_id is 74830 2024-01-24T23:22:58.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 1 -- 172.21.15.132:0/2504231739 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee4c1267b0 con 0x7fee4c09bcc0 2024-01-24T23:22:58.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 1 -- 172.21.15.132:0/2504231739 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1364556343 0 0) 0x7fee2c003530 con 0x7fee4c1377a0 2024-01-24T23:22:58.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 1 -- 172.21.15.132:0/2504231739 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fee4c124960 con 0x7fee4c1377a0 2024-01-24T23:22:58.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 1 -- 172.21.15.132:0/2504231739 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fee44001e20 con 0x7fee4c1377a0 2024-01-24T23:22:58.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:58.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:22:58.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient(hunting): dump: 2024-01-24T23:22:58.378 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:58.378 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:58.378 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:58.378 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:58.378 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:58.378 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:58.378 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:58.378 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:58.379 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:58.379 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:58.379 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:58.379 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:58.379 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:58.379 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:58.379 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:58.379 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:58.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:58.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 1 -- 172.21.15.132:0/2504231739 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2233583682 0 0) 0x7fee4c09c6a0 con 0x7fee4c136cb0 2024-01-24T23:22:58.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 1 -- 172.21.15.132:0/2504231739 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fee2c003530 con 0x7fee4c136cb0 2024-01-24T23:22:58.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 1 -- 172.21.15.132:0/2504231739 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fee34001780 con 0x7fee4c136cb0 2024-01-24T23:22:58.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:58.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:22:58.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient(hunting): dump: 2024-01-24T23:22:58.380 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:58.380 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:58.380 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:58.380 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:58.380 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:58.380 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:58.380 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:58.380 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:58.380 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:58.380 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:58.380 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:58.380 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:58.380 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:58.380 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:58.381 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:58.381 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:58.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 1 -- 172.21.15.132:0/2504231739 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3893273528 0 0) 0x7fee4c124960 con 0x7fee4c1377a0 2024-01-24T23:22:58.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 1 -- 172.21.15.132:0/2504231739 >> v1:172.21.15.115:6791/0 conn(0x7fee4c09bcc0 legacy=0x7fee4c135dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:58.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 1 -- 172.21.15.132:0/2504231739 >> v1:172.21.15.132:6790/0 conn(0x7fee4c136cb0 legacy=0x7fee4c1d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:58.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient: _finish_hunting 0 2024-01-24T23:22:58.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 1 monclient: found mon.h 2024-01-24T23:22:58.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:58.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:58.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 1 -- 172.21.15.132:0/2504231739 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c137fb0 con 0x7fee4c1377a0 2024-01-24T23:22:58.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient: _finish_auth 0 2024-01-24T23:22:58.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient: _check_auth_tickets 2024-01-24T23:22:58.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:58.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 5 monclient: authenticate success, global_id 4820 2024-01-24T23:22:58.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 10 monclient: _renew_subs 2024-01-24T23:22:58.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:58.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 1 -- 172.21.15.132:0/2504231739 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee4c1264d0 con 0x7fee4c1377a0 2024-01-24T23:22:58.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 10 monclient: _renew_subs 2024-01-24T23:22:58.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:58.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 1 -- 172.21.15.132:0/2504231739 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4c1366a0 con 0x7fee4c1377a0 2024-01-24T23:22:58.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 1 -- 172.21.15.132:0/2504231739 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fee44003c50 con 0x7fee4c1377a0 2024-01-24T23:22:58.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:58.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 1 -- 172.21.15.132:0/2504231739 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fee44004180 con 0x7fee4c1377a0 2024-01-24T23:22:58.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:58.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:22:58.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee49ffb640 4 set_mon_vals no callback set 2024-01-24T23:22:58.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 10 monclient: dump: 2024-01-24T23:22:58.382 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:58.383 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:58.383 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:58.383 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:58.383 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:58.383 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:58.383 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:58.383 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:58.383 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:58.383 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:58.383 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:58.383 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:58.383 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:58.383 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:58.383 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:58.383 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:58.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:58.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 1 -- 172.21.15.132:0/2504231739 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fee440042e0 con 0x7fee4c1377a0 2024-01-24T23:22:58.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:58.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:58.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee51cb6640 1 -- 172.21.15.132:0/2504231739 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee30005090 con 0x7fee4c1377a0 2024-01-24T23:22:58.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee43fff640 1 -- 172.21.15.132:0/2504231739 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(38..38 src has 1..38) v4 ==== 3598+0+0 (unknown 3181584953 0 0) 0x7fee44043a10 con 0x7fee4c1377a0 2024-01-24T23:22:58.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.361+0000 7fee4b7fe640 10 monclient: get_auth_request con 0x7fee2c0378a0 auth_method 0 2024-01-24T23:22:58.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.365+0000 7fee43fff640 1 -- 172.21.15.132:0/2504231739 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fee44045e60 con 0x7fee4c1377a0 2024-01-24T23:22:58.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.369+0000 7fee43fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:58.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.369+0000 7fee43fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:58.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.525+0000 7fee51cb6640 10 start_mon_command cmd=[{"prefix": "osd crush rule dump"}] 2024-01-24T23:22:58.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.525+0000 7fee51cb6640 10 monclient: _send_command 2 [{"prefix": "osd crush rule dump"}] 2024-01-24T23:22:58.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.525+0000 7fee51cb6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:58.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.525+0000 7fee51cb6640 1 -- 172.21.15.132:0/2504231739 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rule dump"} v 0) v1 -- 0x7fee300058c0 con 0x7fee4c1377a0 2024-01-24T23:22:58.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.525+0000 7fee43fff640 1 -- 172.21.15.132:0/2504231739 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush rule dump"}]=0 v38) v1 ==== 67+0+1058 (unknown 2656755985 0 3068570633) 0x7fee44014b00 con 0x7fee4c1377a0 2024-01-24T23:22:58.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.525+0000 7fee43fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule dump"}] 2024-01-24T23:22:58.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.525+0000 7fee43fff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:22:58.625 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.621+0000 7fee51cb6640 10 start_mon_command cmd=[{"prefix": "osd crush rule dump"}] 2024-01-24T23:22:58.625 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.621+0000 7fee51cb6640 10 monclient: _send_command 3 [{"prefix": "osd crush rule dump"}] 2024-01-24T23:22:58.625 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.621+0000 7fee51cb6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:22:58.625 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.621+0000 7fee51cb6640 1 -- 172.21.15.132:0/2504231739 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rule dump"} v 0) v1 -- 0x7fee300010a0 con 0x7fee4c1377a0 2024-01-24T23:22:58.625 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.621+0000 7fee43fff640 1 -- 172.21.15.132:0/2504231739 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush rule dump"}]=0 v38) v1 ==== 67+0+1058 (unknown 2656755985 0 3068570633) 0x7fee440131a0 con 0x7fee4c1377a0 2024-01-24T23:22:58.625 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.621+0000 7fee43fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule dump"}] 2024-01-24T23:22:58.625 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.621+0000 7fee43fff640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:22:58.626 INFO:tasks.workunit.client.0.smithi132.stdout:[ 2024-01-24T23:22:58.626 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.626 INFO:tasks.workunit.client.0.smithi132.stdout: "rule_id": 0, 2024-01-24T23:22:58.626 INFO:tasks.workunit.client.0.smithi132.stdout: "rule_name": "replicated_rule", 2024-01-24T23:22:58.626 INFO:tasks.workunit.client.0.smithi132.stdout: "type": 1, 2024-01-24T23:22:58.626 INFO:tasks.workunit.client.0.smithi132.stdout: "steps": [ 2024-01-24T23:22:58.626 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.626 INFO:tasks.workunit.client.0.smithi132.stdout: "op": "take", 2024-01-24T23:22:58.626 INFO:tasks.workunit.client.0.smithi132.stdout: "item": -1, 2024-01-24T23:22:58.626 INFO:tasks.workunit.client.0.smithi132.stdout: "item_name": "default" 2024-01-24T23:22:58.626 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.626 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.626 INFO:tasks.workunit.client.0.smithi132.stdout: "op": "choose_firstn", 2024-01-24T23:22:58.627 INFO:tasks.workunit.client.0.smithi132.stdout: "num": 0, 2024-01-24T23:22:58.627 INFO:tasks.workunit.client.0.smithi132.stdout: "type": "osd" 2024-01-24T23:22:58.627 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.627 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.627 INFO:tasks.workunit.client.0.smithi132.stdout: "op": "emit" 2024-01-24T23:22:58.627 INFO:tasks.workunit.client.0.smithi132.stdout: } 2024-01-24T23:22:58.627 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:22:58.627 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.627 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.627 INFO:tasks.workunit.client.0.smithi132.stdout: "rule_id": 1, 2024-01-24T23:22:58.627 INFO:tasks.workunit.client.0.smithi132.stdout: "rule_name": "erasure-code", 2024-01-24T23:22:58.627 INFO:tasks.workunit.client.0.smithi132.stdout: "type": 3, 2024-01-24T23:22:58.627 INFO:tasks.workunit.client.0.smithi132.stdout: "steps": [ 2024-01-24T23:22:58.627 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.628 INFO:tasks.workunit.client.0.smithi132.stdout: "op": "set_chooseleaf_tries", 2024-01-24T23:22:58.628 INFO:tasks.workunit.client.0.smithi132.stdout: "num": 5 2024-01-24T23:22:58.628 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.628 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.628 INFO:tasks.workunit.client.0.smithi132.stdout: "op": "set_choose_tries", 2024-01-24T23:22:58.628 INFO:tasks.workunit.client.0.smithi132.stdout: "num": 100 2024-01-24T23:22:58.628 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.628 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.628 INFO:tasks.workunit.client.0.smithi132.stdout: "op": "take", 2024-01-24T23:22:58.628 INFO:tasks.workunit.client.0.smithi132.stdout: "item": -1, 2024-01-24T23:22:58.628 INFO:tasks.workunit.client.0.smithi132.stdout: "item_name": "default" 2024-01-24T23:22:58.628 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.628 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.628 INFO:tasks.workunit.client.0.smithi132.stdout: "op": "choose_indep", 2024-01-24T23:22:58.628 INFO:tasks.workunit.client.0.smithi132.stdout: "num": 0, 2024-01-24T23:22:58.629 INFO:tasks.workunit.client.0.smithi132.stdout: "type": "osd" 2024-01-24T23:22:58.629 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:22:58.629 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:22:58.629 INFO:tasks.workunit.client.0.smithi132.stdout: "op": "emit" 2024-01-24T23:22:58.629 INFO:tasks.workunit.client.0.smithi132.stdout: } 2024-01-24T23:22:58.629 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:22:58.629 INFO:tasks.workunit.client.0.smithi132.stdout: } 2024-01-24T23:22:58.629 INFO:tasks.workunit.client.0.smithi132.stdout:] 2024-01-24T23:22:58.629 INFO:tasks.workunit.client.0.smithi132.stdout: 2024-01-24T23:22:58.630 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.621+0000 7fee51cb6640 1 -- 172.21.15.132:0/2504231739 >> v1:172.21.15.132:6812/1835401922 conn(0x7fee2c0378a0 legacy=0x7fee2c039d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:58.630 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.621+0000 7fee51cb6640 10 monclient: shutdown 2024-01-24T23:22:58.630 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.621+0000 7fee51cb6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:58.630 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.621+0000 7fee51cb6640 1 -- 172.21.15.132:0/2504231739 >> v1:172.21.15.132:6791/0 conn(0x7fee4c1377a0 legacy=0x7fee4c1d3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:58.630 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.621+0000 7fee51cb6640 1 -- 172.21.15.132:0/2504231739 shutdown_connections 2024-01-24T23:22:58.630 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.621+0000 7fee51cb6640 1 -- 172.21.15.132:0/2504231739 >> 172.21.15.132:0/2504231739 conn(0x7fee4c0a9c10 msgr2=0x7fee4c0a3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:58.631 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.621+0000 7fee51cb6640 1 -- 172.21.15.132:0/2504231739 shutdown_connections 2024-01-24T23:22:58.631 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.625+0000 7fee51cb6640 1 -- 172.21.15.132:0/2504231739 wait complete. 2024-01-24T23:22:58.641 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule ls 2024-01-24T23:22:58.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f996e205640 10 monclient: get_monmap_and_config 2024-01-24T23:22:58.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f996e205640 10 monclient: build_initial_monmap 2024-01-24T23:22:58.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f996e205640 10 monclient: monmap: 2024-01-24T23:22:58.760 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:58.761 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:58.761 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:58.762055+0000 2024-01-24T23:22:58.761 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:58.762055+0000 2024-01-24T23:22:58.761 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:58.761 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:58.761 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:58.761 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:58.761 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:58.761 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:58.761 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:58.761 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:58.761 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:58.761 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:58.761 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:58.761 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:58.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f996e205640 1 Processor -- start 2024-01-24T23:22:58.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f996e205640 1 -- start start 2024-01-24T23:22:58.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f996e205640 10 monclient: init 2024-01-24T23:22:58.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f996e205640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:58.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f996e205640 10 monclient: _add_conns ranks=[3,0,4,8,5,1,2,7,6] 2024-01-24T23:22:58.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f996e205640 10 monclient(hunting): picked mon.noname-g con 0x7f9968133b50 addr v1:172.21.15.132:6790/0 2024-01-24T23:22:58.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f996e205640 10 monclient(hunting): picked mon.noname-a con 0x7f996809ec20 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:58.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f996e205640 10 monclient(hunting): picked mon.noname-c con 0x7f996813a1d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:58.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f996e205640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9968064700 con 0x7f996809ec20 2024-01-24T23:22:58.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f996e205640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f996809d9f0 con 0x7f9968133b50 2024-01-24T23:22:58.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f996e205640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f996809dc60 con 0x7f996813a1d0 2024-01-24T23:22:58.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f996e205640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:58.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f996e205640 10 monclient(hunting): authenticate will time out at 1066.086666s 2024-01-24T23:22:58.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99677fe640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f9968133b50 0x7f9968133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:59930/0 (socket says 172.21.15.132:59930) 2024-01-24T23:22:58.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99677fe640 1 -- 172.21.15.132:0/54367422 learned_addr learned my addr 172.21.15.132:0/54367422 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:58.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 1 -- 172.21.15.132:0/54367422 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 572050633 0 0) 0x7f996809d9f0 con 0x7f9968133b50 2024-01-24T23:22:58.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 10 monclient(hunting): _negotiate 2024-01-24T23:22:58.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:58.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:58.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 10 monclient(hunting): my global_id is 4773 2024-01-24T23:22:58.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 1 -- 172.21.15.132:0/54367422 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9954003530 con 0x7f9968133b50 2024-01-24T23:22:58.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 1 -- 172.21.15.132:0/54367422 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3422843482 0 0) 0x7f9954003530 con 0x7f9968133b50 2024-01-24T23:22:58.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 1 -- 172.21.15.132:0/54367422 >> v1:172.21.15.115:6791/0 conn(0x7f996813a1d0 legacy=0x7f996813c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:58.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 1 -- 172.21.15.132:0/54367422 >> v1:172.21.15.115:6789/0 conn(0x7f996809ec20 legacy=0x7f9968138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:58.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 10 monclient: _finish_hunting 0 2024-01-24T23:22:58.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 1 monclient: found mon.noname-g 2024-01-24T23:22:58.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:58.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:22:58.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 1 -- 172.21.15.132:0/54367422 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996809df40 con 0x7f9968133b50 2024-01-24T23:22:58.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 10 monclient: _finish_auth 0 2024-01-24T23:22:58.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 10 monclient: _check_auth_tickets 2024-01-24T23:22:58.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:58.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 1 -- 172.21.15.132:0/54367422 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f99580017c0 con 0x7f9968133b50 2024-01-24T23:22:58.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f996e205640 5 monclient: authenticate success, global_id 4773 2024-01-24T23:22:58.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f996e205640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:58.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:58.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:22:58.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 10 monclient: dump: 2024-01-24T23:22:58.771 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:58.771 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:58.771 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:58.771 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:58.771 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:58.771 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:58.771 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:58.771 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:58.771 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:58.771 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:58.771 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:58.771 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:58.771 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:58.771 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:58.771 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:58.772 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:58.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f996e205640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:58.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 1 -- 172.21.15.132:0/54367422 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9958001ae0 con 0x7f9968133b50 2024-01-24T23:22:58.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:58.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f996e205640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:58.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f996e205640 4 set_mon_vals no callback set 2024-01-24T23:22:58.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 1 -- 172.21.15.132:0/54367422 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9958004bf0 con 0x7f9968133b50 2024-01-24T23:22:58.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:58.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:22:58.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f99667fc640 10 monclient: dump: 2024-01-24T23:22:58.772 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:58.772 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:58.772 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:58.772 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:58.773 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:58.773 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:58.773 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:58.773 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:58.773 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:58.773 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:58.773 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:58.773 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:58.773 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:58.773 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:58.773 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:58.773 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:58.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f996e205640 10 monclient: shutdown 2024-01-24T23:22:58.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f996e205640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:58.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.757+0000 7f996e205640 1 -- 172.21.15.132:0/54367422 >> v1:172.21.15.132:6790/0 conn(0x7f9968133b50 legacy=0x7f9968133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:58.774 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 1 -- 172.21.15.132:0/54367422 shutdown_connections 2024-01-24T23:22:58.774 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 1 -- 172.21.15.132:0/54367422 >> 172.21.15.132:0/54367422 conn(0x7f99680a5a10 msgr2=0x7f99680a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:58.774 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 1 -- 172.21.15.132:0/54367422 shutdown_connections 2024-01-24T23:22:58.774 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 1 -- 172.21.15.132:0/54367422 wait complete. 2024-01-24T23:22:58.774 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 10 monclient: build_initial_monmap 2024-01-24T23:22:58.774 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 10 monclient: monmap: 2024-01-24T23:22:58.774 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:58.774 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:58.774 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:58.774 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:58.774 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:58.774 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:58.774 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:58.774 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:58.774 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:58.775 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:58.775 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:58.775 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:58.775 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:58.775 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:58.775 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:58.775 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:58.775 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 1 Processor -- start 2024-01-24T23:22:58.775 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 1 -- start start 2024-01-24T23:22:58.775 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 10 monclient: init 2024-01-24T23:22:58.775 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:58.775 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 10 monclient: _add_conns ranks=[2,8,7,6,3,4,0,5,1] 2024-01-24T23:22:58.775 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 10 monclient(hunting): picked mon.noname-b con 0x7f996809ec20 addr v1:172.21.15.115:6790/0 2024-01-24T23:22:58.775 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 10 monclient(hunting): picked mon.noname-e con 0x7f9968133b50 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:58.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 10 monclient(hunting): picked mon.noname-i con 0x7f996813a1d0 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:58.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9968070120 con 0x7f996809ec20 2024-01-24T23:22:58.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f996812cd40 con 0x7f996813a1d0 2024-01-24T23:22:58.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99680a13b0 con 0x7f9968133b50 2024-01-24T23:22:58.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:58.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 10 monclient(hunting): authenticate will time out at 1066.089594s 2024-01-24T23:22:58.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f9966ffd640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f9968133b50 legacy=0x7f99681d0710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:58.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f99677fe640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f996809ec20 legacy=0x7f99681cbbc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:22:58.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f9967fff640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f996813a1d0 0x7f99681d3e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:38892/0 (socket says 172.21.15.132:38892) 2024-01-24T23:22:58.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f9967fff640 1 -- 172.21.15.132:0/1643727523 learned_addr learned my addr 172.21.15.132:0/1643727523 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:58.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 1 -- 172.21.15.132:0/1643727523 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848908700 0 0) 0x7f996812cd40 con 0x7f996813a1d0 2024-01-24T23:22:58.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:58.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:58.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:58.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 10 monclient(hunting): my global_id is 4849 2024-01-24T23:22:58.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 1 -- 172.21.15.132:0/1643727523 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9948003560 con 0x7f996813a1d0 2024-01-24T23:22:58.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 1 -- 172.21.15.132:0/1643727523 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3192618235 0 0) 0x7f9948003560 con 0x7f996813a1d0 2024-01-24T23:22:58.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 1 -- 172.21.15.132:0/1643727523 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f996812cd40 con 0x7f996813a1d0 2024-01-24T23:22:58.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 1 -- 172.21.15.132:0/1643727523 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f99600019a0 con 0x7f996813a1d0 2024-01-24T23:22:58.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:58.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:22:58.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 10 monclient(hunting): dump: 2024-01-24T23:22:58.777 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:58.777 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:58.777 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:58.777 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:58.778 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:58.778 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:58.778 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:58.778 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:58.778 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:58.778 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:58.778 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:58.778 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:58.778 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:58.778 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:58.778 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:58.778 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:58.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:58.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 1 -- 172.21.15.132:0/1643727523 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3624644523 0 0) 0x7f996812cd40 con 0x7f996813a1d0 2024-01-24T23:22:58.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 1 -- 172.21.15.132:0/1643727523 >> v1:172.21.15.115:6793/0 conn(0x7f9968133b50 legacy=0x7f99681d0710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:58.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 1 -- 172.21.15.132:0/1643727523 >> v1:172.21.15.115:6790/0 conn(0x7f996809ec20 legacy=0x7f99681cbbc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:58.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 10 monclient: _finish_hunting 0 2024-01-24T23:22:58.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 1 monclient: found mon.i 2024-01-24T23:22:58.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:58.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:58.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 1 -- 172.21.15.132:0/1643727523 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9968133920 con 0x7f996813a1d0 2024-01-24T23:22:58.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 10 monclient: _finish_auth 0 2024-01-24T23:22:58.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 10 monclient: _check_auth_tickets 2024-01-24T23:22:58.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:58.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 5 monclient: authenticate success, global_id 4849 2024-01-24T23:22:58.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 10 monclient: _renew_subs 2024-01-24T23:22:58.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:58.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 1 -- 172.21.15.132:0/1643727523 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99681cc2d0 con 0x7f996813a1d0 2024-01-24T23:22:58.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 10 monclient: _renew_subs 2024-01-24T23:22:58.780 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:58.780 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 1 -- 172.21.15.132:0/1643727523 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9968071920 con 0x7f996813a1d0 2024-01-24T23:22:58.780 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 1 -- 172.21.15.132:0/1643727523 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9960001d70 con 0x7f996813a1d0 2024-01-24T23:22:58.780 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:58.780 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 1 -- 172.21.15.132:0/1643727523 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9960003c10 con 0x7f996813a1d0 2024-01-24T23:22:58.780 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:58.780 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:22:58.780 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f99657fa640 4 set_mon_vals no callback set 2024-01-24T23:22:58.780 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 10 monclient: dump: 2024-01-24T23:22:58.780 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:58.780 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:58.780 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:58.780 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:58.780 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:58.781 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:58.781 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:58.781 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:58.781 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:58.781 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:58.781 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:58.781 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:58.781 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:58.781 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:58.781 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:58.781 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:58.781 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:58.781 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:58.781 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:58.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f996e205640 1 -- 172.21.15.132:0/1643727523 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f994c005090 con 0x7f996813a1d0 2024-01-24T23:22:58.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 1 -- 172.21.15.132:0/1643727523 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f9960003d70 con 0x7f996813a1d0 2024-01-24T23:22:58.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f995ffff640 1 -- 172.21.15.132:0/1643727523 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(38..38 src has 1..38) v4 ==== 3598+0+0 (unknown 3181584953 0 0) 0x7f9960043620 con 0x7f996813a1d0 2024-01-24T23:22:58.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.761+0000 7f99677fe640 10 monclient: get_auth_request con 0x7f9948036f70 auth_method 0 2024-01-24T23:22:58.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.765+0000 7f995ffff640 1 -- 172.21.15.132:0/1643727523 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f9960014780 con 0x7f996813a1d0 2024-01-24T23:22:58.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.765+0000 7f995ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:58.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.765+0000 7f995ffff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:58.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.913+0000 7f996e205640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-01-24T23:22:58.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.913+0000 7f996e205640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls"}] 2024-01-24T23:22:58.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.913+0000 7f996e205640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:58.920 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.913+0000 7f996e205640 1 -- 172.21.15.132:0/1643727523 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rule ls"} v 0) v1 -- 0x7f994c0058c0 con 0x7f996813a1d0 2024-01-24T23:22:58.920 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.917+0000 7f995ffff640 1 -- 172.21.15.132:0/1643727523 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v38) v1 ==== 65+0+29 (unknown 535663125 0 852635902) 0x7f9960012e20 con 0x7f996813a1d0 2024-01-24T23:22:58.920 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.917+0000 7f995ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls"}] 2024-01-24T23:22:58.920 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:58.917+0000 7f995ffff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:22:59.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.005+0000 7f996e205640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-01-24T23:22:59.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.005+0000 7f996e205640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls"}] 2024-01-24T23:22:59.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.005+0000 7f996e205640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:59.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.005+0000 7f996e205640 1 -- 172.21.15.132:0/1643727523 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rule ls"} v 0) v1 -- 0x7f994c000bd0 con 0x7f996813a1d0 2024-01-24T23:22:59.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.013+0000 7f995ffff640 1 -- 172.21.15.132:0/1643727523 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v38) v1 ==== 65+0+29 (unknown 535663125 0 852635902) 0x7f99600145a0 con 0x7f996813a1d0 2024-01-24T23:22:59.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.013+0000 7f995ffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls"}] 2024-01-24T23:22:59.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.013+0000 7f995ffff640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:22:59.015 INFO:tasks.workunit.client.0.smithi132.stdout:replicated_rule 2024-01-24T23:22:59.015 INFO:tasks.workunit.client.0.smithi132.stdout:erasure-code 2024-01-24T23:22:59.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.013+0000 7f996e205640 1 -- 172.21.15.132:0/1643727523 >> v1:172.21.15.132:6812/1835401922 conn(0x7f9948036f70 legacy=0x7f9948039430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:59.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.013+0000 7f996e205640 10 monclient: shutdown 2024-01-24T23:22:59.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.013+0000 7f996e205640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:59.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.013+0000 7f996e205640 1 -- 172.21.15.132:0/1643727523 >> v1:172.21.15.132:6792/0 conn(0x7f996813a1d0 legacy=0x7f99681d3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:59.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.013+0000 7f996e205640 1 -- 172.21.15.132:0/1643727523 shutdown_connections 2024-01-24T23:22:59.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.013+0000 7f996e205640 1 -- 172.21.15.132:0/1643727523 >> 172.21.15.132:0/1643727523 conn(0x7f99680a5a10 msgr2=0x7f9968066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:59.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.013+0000 7f996e205640 1 -- 172.21.15.132:0/1643727523 shutdown_connections 2024-01-24T23:22:59.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.013+0000 7f996e205640 1 -- 172.21.15.132:0/1643727523 wait complete. 2024-01-24T23:22:59.031 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule list 2024-01-24T23:22:59.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f778adb4640 10 monclient: get_monmap_and_config 2024-01-24T23:22:59.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f778adb4640 10 monclient: build_initial_monmap 2024-01-24T23:22:59.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f778adb4640 10 monclient: monmap: 2024-01-24T23:22:59.144 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:59.144 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:59.144 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:59.146057+0000 2024-01-24T23:22:59.145 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:59.146057+0000 2024-01-24T23:22:59.145 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:59.145 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:59.145 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:59.145 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:59.145 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:59.145 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:59.145 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:59.145 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:59.145 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:59.145 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:59.145 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:59.145 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:59.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f778adb4640 1 Processor -- start 2024-01-24T23:22:59.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f778adb4640 1 -- start start 2024-01-24T23:22:59.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f778adb4640 10 monclient: init 2024-01-24T23:22:59.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f778adb4640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:59.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f778adb4640 10 monclient: _add_conns ranks=[6,7,5,4,2,3,0,8,1] 2024-01-24T23:22:59.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f778adb4640 10 monclient(hunting): picked mon.noname-d con 0x7f778409c4c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:59.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f778adb4640 10 monclient(hunting): picked mon.noname-i con 0x7f778409cfb0 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:59.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f778adb4640 10 monclient(hunting): picked mon.noname-h con 0x7f778409ec50 addr v1:172.21.15.132:6791/0 2024-01-24T23:22:59.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f778adb4640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7784064700 con 0x7f778409ec50 2024-01-24T23:22:59.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f778adb4640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f778409d9f0 con 0x7f778409c4c0 2024-01-24T23:22:59.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f778adb4640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f778409dc60 con 0x7f778409cfb0 2024-01-24T23:22:59.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f778adb4640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:59.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f778adb4640 10 monclient(hunting): authenticate will time out at 1066.471146s 2024-01-24T23:22:59.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f7783fff640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f778409cfb0 0x7f7784138d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:38898/0 (socket says 172.21.15.132:38898) 2024-01-24T23:22:59.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f7783fff640 1 -- 172.21.15.132:0/1055651478 learned_addr learned my addr 172.21.15.132:0/1055651478 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:59.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 1 -- 172.21.15.132:0/1055651478 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1187143169 0 0) 0x7f778409dc60 con 0x7f778409cfb0 2024-01-24T23:22:59.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 10 monclient(hunting): _negotiate 2024-01-24T23:22:59.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:59.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:59.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 10 monclient(hunting): my global_id is 4858 2024-01-24T23:22:59.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 1 -- 172.21.15.132:0/1055651478 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f776c005740 con 0x7f778409cfb0 2024-01-24T23:22:59.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 1 -- 172.21.15.132:0/1055651478 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3599167290 0 0) 0x7f7784064700 con 0x7f778409ec50 2024-01-24T23:22:59.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 10 monclient(hunting): _negotiate 2024-01-24T23:22:59.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:59.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:59.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 10 monclient(hunting): my global_id is 4829 2024-01-24T23:22:59.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 1 -- 172.21.15.132:0/1055651478 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f778409dc60 con 0x7f778409ec50 2024-01-24T23:22:59.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 1 -- 172.21.15.132:0/1055651478 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 812685112 0 0) 0x7f776c005740 con 0x7f778409cfb0 2024-01-24T23:22:59.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 1 -- 172.21.15.132:0/1055651478 >> v1:172.21.15.115:6792/0 conn(0x7f778409c4c0 legacy=0x7f778409c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:59.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 1 -- 172.21.15.132:0/1055651478 >> v1:172.21.15.132:6791/0 conn(0x7f778409ec50 legacy=0x7f778413c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:59.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 10 monclient: _finish_hunting 0 2024-01-24T23:22:59.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 1 monclient: found mon.noname-i 2024-01-24T23:22:59.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:59.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:22:59.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 1 -- 172.21.15.132:0/1055651478 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778409df40 con 0x7f778409cfb0 2024-01-24T23:22:59.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 10 monclient: _finish_auth 0 2024-01-24T23:22:59.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 10 monclient: _check_auth_tickets 2024-01-24T23:22:59.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:59.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 1 -- 172.21.15.132:0/1055651478 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f77780017c0 con 0x7f778409cfb0 2024-01-24T23:22:59.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:59.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:22:59.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f77837fe640 10 monclient: dump: 2024-01-24T23:22:59.151 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:59.151 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:59.151 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:59.151 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:59.152 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:59.152 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:59.152 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:59.152 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:59.152 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:59.152 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:59.152 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:59.152 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:59.152 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:59.152 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:59.152 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:59.152 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:59.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f778adb4640 5 monclient: authenticate success, global_id 4858 2024-01-24T23:22:59.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.141+0000 7f778adb4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:59.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77837fe640 1 -- 172.21.15.132:0/1055651478 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7778001a60 con 0x7f778409cfb0 2024-01-24T23:22:59.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77837fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:59.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77837fe640 1 -- 172.21.15.132:0/1055651478 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7778004bf0 con 0x7f778409cfb0 2024-01-24T23:22:59.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:59.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 4 set_mon_vals no callback set 2024-01-24T23:22:59.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:59.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77837fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:22:59.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77837fe640 10 monclient: dump: 2024-01-24T23:22:59.153 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:59.153 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:59.153 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:59.153 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:59.153 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:59.153 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:59.153 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:59.153 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:59.154 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:59.154 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:59.154 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:59.154 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:59.154 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:59.154 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:59.154 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:59.154 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:59.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 10 monclient: shutdown 2024-01-24T23:22:59.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:59.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 1 -- 172.21.15.132:0/1055651478 >> v1:172.21.15.132:6792/0 conn(0x7f778409cfb0 legacy=0x7f7784138d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:59.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 1 -- 172.21.15.132:0/1055651478 shutdown_connections 2024-01-24T23:22:59.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 1 -- 172.21.15.132:0/1055651478 >> 172.21.15.132:0/1055651478 conn(0x7f77840a5a10 msgr2=0x7f77840a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:59.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 1 -- 172.21.15.132:0/1055651478 shutdown_connections 2024-01-24T23:22:59.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 1 -- 172.21.15.132:0/1055651478 wait complete. 2024-01-24T23:22:59.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 10 monclient: build_initial_monmap 2024-01-24T23:22:59.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 10 monclient: monmap: 2024-01-24T23:22:59.155 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:59.155 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:59.155 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:59.155 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:59.155 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:59.155 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:59.155 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:59.155 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:59.155 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:59.155 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:59.155 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:59.155 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:59.155 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:59.156 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:59.156 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:59.156 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:59.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 1 Processor -- start 2024-01-24T23:22:59.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 1 -- start start 2024-01-24T23:22:59.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 10 monclient: init 2024-01-24T23:22:59.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:59.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 10 monclient: _add_conns ranks=[8,0,6,4,2,5,7,1,3] 2024-01-24T23:22:59.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 10 monclient(hunting): picked mon.noname-e con 0x7f778409c4c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:59.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 10 monclient(hunting): picked mon.noname-a con 0x7f778409cfb0 addr v1:172.21.15.115:6789/0 2024-01-24T23:22:59.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 10 monclient(hunting): picked mon.noname-d con 0x7f778409ec50 addr v1:172.21.15.115:6792/0 2024-01-24T23:22:59.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f778409c020 con 0x7f778409cfb0 2024-01-24T23:22:59.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77840a1420 con 0x7f778409ec50 2024-01-24T23:22:59.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77841c86d0 con 0x7f778409c4c0 2024-01-24T23:22:59.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:59.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 10 monclient(hunting): authenticate will time out at 1066.474212s 2024-01-24T23:22:59.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f7788b50640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f778409c4c0 legacy=0x7f778412cd40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:59.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f7789351640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f778409ec50 0x7f77841d3c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:33426/0 (socket says 172.21.15.132:33426) 2024-01-24T23:22:59.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f7789351640 1 -- 172.21.15.132:0/723667300 learned_addr learned my addr 172.21.15.132:0/723667300 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:59.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 1 -- 172.21.15.132:0/723667300 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1828009625 0 0) 0x7f778409dc60 con 0x7f778409ec50 2024-01-24T23:22:59.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 10 monclient(hunting): _negotiate 2024-01-24T23:22:59.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:59.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:59.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 10 monclient(hunting): my global_id is 4884 2024-01-24T23:22:59.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 1 -- 172.21.15.132:0/723667300 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7770003560 con 0x7f778409ec50 2024-01-24T23:22:59.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 1 -- 172.21.15.132:0/723667300 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395797835 0 0) 0x7f778409c020 con 0x7f778409cfb0 2024-01-24T23:22:59.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 10 monclient(hunting): _negotiate 2024-01-24T23:22:59.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:59.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:59.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 10 monclient(hunting): my global_id is 4833 2024-01-24T23:22:59.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 1 -- 172.21.15.132:0/723667300 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f778409dc60 con 0x7f778409cfb0 2024-01-24T23:22:59.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 1 -- 172.21.15.132:0/723667300 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4174528355 0 0) 0x7f7770003560 con 0x7f778409ec50 2024-01-24T23:22:59.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 1 -- 172.21.15.132:0/723667300 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f778409c020 con 0x7f778409ec50 2024-01-24T23:22:59.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 1 -- 172.21.15.132:0/723667300 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f777c001d60 con 0x7f778409ec50 2024-01-24T23:22:59.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:59.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:22:59.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 10 monclient(hunting): dump: 2024-01-24T23:22:59.162 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:59.162 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:59.162 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:59.162 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:59.162 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:59.162 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:59.162 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:59.162 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:59.162 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:59.162 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:59.162 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:59.162 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:59.162 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:59.162 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:59.163 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:59.163 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:59.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:59.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 1 -- 172.21.15.132:0/723667300 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1541830342 0 0) 0x7f778409dc60 con 0x7f778409cfb0 2024-01-24T23:22:59.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 1 -- 172.21.15.132:0/723667300 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7770003560 con 0x7f778409cfb0 2024-01-24T23:22:59.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 1 -- 172.21.15.132:0/723667300 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7778004b30 con 0x7f778409cfb0 2024-01-24T23:22:59.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:59.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:22:59.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 10 monclient(hunting): dump: 2024-01-24T23:22:59.163 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:59.163 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:59.163 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:59.163 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:59.163 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:59.163 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:59.164 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:59.164 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:59.164 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:59.164 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:59.164 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:59.164 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:59.164 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:59.164 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:59.164 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:59.164 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:59.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 1 -- 172.21.15.132:0/723667300 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4272823348 0 0) 0x7f778409c020 con 0x7f778409ec50 2024-01-24T23:22:59.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 1 -- 172.21.15.132:0/723667300 >> v1:172.21.15.115:6793/0 conn(0x7f778409c4c0 legacy=0x7f778412cd40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:59.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 1 -- 172.21.15.132:0/723667300 >> v1:172.21.15.115:6789/0 conn(0x7f778409cfb0 legacy=0x7f77841d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:59.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 10 monclient: _finish_hunting 0 2024-01-24T23:22:59.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 1 monclient: found mon.d 2024-01-24T23:22:59.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:59.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:22:59.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 1 -- 172.21.15.132:0/723667300 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7784134b00 con 0x7f778409ec50 2024-01-24T23:22:59.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 10 monclient: _finish_auth 0 2024-01-24T23:22:59.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 10 monclient: _check_auth_tickets 2024-01-24T23:22:59.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f77817fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:59.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 5 monclient: authenticate success, global_id 4884 2024-01-24T23:22:59.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 10 monclient: _renew_subs 2024-01-24T23:22:59.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:22:59.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 1 -- 172.21.15.132:0/723667300 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f778411fec0 con 0x7f778409ec50 2024-01-24T23:22:59.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 10 monclient: _renew_subs 2024-01-24T23:22:59.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.145+0000 7f778adb4640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:22:59.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.149+0000 7f778adb4640 1 -- 172.21.15.132:0/723667300 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7784070ab0 con 0x7f778409ec50 2024-01-24T23:22:59.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.149+0000 7f77817fa640 1 -- 172.21.15.132:0/723667300 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f777c0020a0 con 0x7f778409ec50 2024-01-24T23:22:59.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.149+0000 7f77817fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.149+0000 7f7782ffd640 4 set_mon_vals no callback set 2024-01-24T23:22:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.149+0000 7f77817fa640 1 -- 172.21.15.132:0/723667300 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f777c003f90 con 0x7f778409ec50 2024-01-24T23:22:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.149+0000 7f77817fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.149+0000 7f77817fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:22:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.149+0000 7f77817fa640 10 monclient: dump: 2024-01-24T23:22:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:59.167 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:59.167 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:59.167 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:59.167 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:59.167 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:59.167 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:59.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.149+0000 7f778adb4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:59.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.149+0000 7f778adb4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:59.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.149+0000 7f778adb4640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:22:59.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.149+0000 7f778adb4640 1 -- 172.21.15.132:0/723667300 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7764005090 con 0x7f778409ec50 2024-01-24T23:22:59.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.149+0000 7f77817fa640 1 -- 172.21.15.132:0/723667300 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f777c0041b0 con 0x7f778409ec50 2024-01-24T23:22:59.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.149+0000 7f77817fa640 1 -- 172.21.15.132:0/723667300 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(38..38 src has 1..38) v4 ==== 3598+0+0 (unknown 3181584953 0 0) 0x7f777c043880 con 0x7f778409ec50 2024-01-24T23:22:59.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.149+0000 7f7788b50640 10 monclient: get_auth_request con 0x7f77700376d0 auth_method 0 2024-01-24T23:22:59.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.161+0000 7f77817fa640 1 -- 172.21.15.132:0/723667300 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f777c014ba0 con 0x7f778409ec50 2024-01-24T23:22:59.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.161+0000 7f77817fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:59.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.161+0000 7f77817fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:59.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.317+0000 7f778adb4640 10 start_mon_command cmd=[{"prefix": "osd crush rule list"}] 2024-01-24T23:22:59.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.317+0000 7f778adb4640 10 monclient: _send_command 2 [{"prefix": "osd crush rule list"}] 2024-01-24T23:22:59.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.317+0000 7f778adb4640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:22:59.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.317+0000 7f778adb4640 1 -- 172.21.15.132:0/723667300 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush rule list"} v 0) v1 -- 0x7f77640058c0 con 0x7f778409ec50 2024-01-24T23:22:59.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.321+0000 7f77817fa640 1 -- 172.21.15.132:0/723667300 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush rule list"}]=0 v38) v1 ==== 67+0+29 (unknown 300370255 0 852635902) 0x7f777c014790 con 0x7f778409ec50 2024-01-24T23:22:59.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.321+0000 7f77817fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule list"}] 2024-01-24T23:22:59.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.321+0000 7f77817fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:22:59.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.417+0000 7f778adb4640 10 start_mon_command cmd=[{"prefix": "osd crush rule list"}] 2024-01-24T23:22:59.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.417+0000 7f778adb4640 10 monclient: _send_command 3 [{"prefix": "osd crush rule list"}] 2024-01-24T23:22:59.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.417+0000 7f778adb4640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:22:59.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.417+0000 7f778adb4640 1 -- 172.21.15.132:0/723667300 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush rule list"} v 0) v1 -- 0x7f7764000bd0 con 0x7f778409ec50 2024-01-24T23:22:59.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.417+0000 7f77817fa640 1 -- 172.21.15.132:0/723667300 <== mon.6 v1:172.21.15.115:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush rule list"}]=0 v38) v1 ==== 67+0+29 (unknown 300370255 0 852635902) 0x7f777c014970 con 0x7f778409ec50 2024-01-24T23:22:59.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.417+0000 7f77817fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule list"}] 2024-01-24T23:22:59.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.417+0000 7f77817fa640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:22:59.422 INFO:tasks.workunit.client.0.smithi132.stdout:replicated_rule 2024-01-24T23:22:59.422 INFO:tasks.workunit.client.0.smithi132.stdout:erasure-code 2024-01-24T23:22:59.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.421+0000 7f778adb4640 1 -- 172.21.15.132:0/723667300 >> v1:172.21.15.132:6812/1835401922 conn(0x7f77700376d0 legacy=0x7f7770039b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:59.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.421+0000 7f778adb4640 10 monclient: shutdown 2024-01-24T23:22:59.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.421+0000 7f778adb4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:59.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.421+0000 7f778adb4640 1 -- 172.21.15.132:0/723667300 >> v1:172.21.15.115:6792/0 conn(0x7f778409ec50 legacy=0x7f77841d3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:59.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.421+0000 7f778adb4640 1 -- 172.21.15.132:0/723667300 shutdown_connections 2024-01-24T23:22:59.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.421+0000 7f778adb4640 1 -- 172.21.15.132:0/723667300 >> 172.21.15.132:0/723667300 conn(0x7f77840a5a10 msgr2=0x7f77840a0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:59.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.421+0000 7f778adb4640 1 -- 172.21.15.132:0/723667300 shutdown_connections 2024-01-24T23:22:59.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.421+0000 7f778adb4640 1 -- 172.21.15.132:0/723667300 wait complete. 2024-01-24T23:22:59.438 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule create-simple foo default host 2024-01-24T23:22:59.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 10 monclient: get_monmap_and_config 2024-01-24T23:22:59.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 10 monclient: build_initial_monmap 2024-01-24T23:22:59.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 10 monclient: monmap: 2024-01-24T23:22:59.548 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:59.548 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:59.548 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:22:59.549649+0000 2024-01-24T23:22:59.548 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:22:59.549649+0000 2024-01-24T23:22:59.548 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:59.548 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:59.548 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:59.548 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:59.548 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:59.549 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:59.549 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:59.549 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:59.549 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:59.549 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:59.549 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:59.549 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:59.551 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 1 Processor -- start 2024-01-24T23:22:59.551 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 1 -- start start 2024-01-24T23:22:59.551 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 10 monclient: init 2024-01-24T23:22:59.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:59.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 10 monclient: _add_conns ranks=[8,2,7,6,3,0,5,4,1] 2024-01-24T23:22:59.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 10 monclient(hunting): picked mon.noname-e con 0x7efd400a09a0 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:59.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 10 monclient(hunting): picked mon.noname-b con 0x7efd4013e3f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:22:59.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 10 monclient(hunting): picked mon.noname-i con 0x7efd40141f00 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:59.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd40064700 con 0x7efd4013e3f0 2024-01-24T23:22:59.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd40072280 con 0x7efd40141f00 2024-01-24T23:22:59.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd4006d500 con 0x7efd400a09a0 2024-01-24T23:22:59.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:59.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 10 monclient(hunting): authenticate will time out at 1066.874260s 2024-01-24T23:22:59.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd44ee1640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7efd4013e3f0 legacy=0x7efd401407e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:22:59.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd45ee3640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7efd40141f00 0x7efd401442f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:38902/0 (socket says 172.21.15.132:38902) 2024-01-24T23:22:59.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd45ee3640 1 -- 172.21.15.132:0/904267901 learned_addr learned my addr 172.21.15.132:0/904267901 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:59.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd456e2640 1 -- 172.21.15.132:0/904267901 >> v1:172.21.15.115:6793/0 conn(0x7efd400a09a0 legacy=0x7efd400a0d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:59.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 1 -- 172.21.15.132:0/904267901 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714471905 0 0) 0x7efd40072280 con 0x7efd40141f00 2024-01-24T23:22:59.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 10 monclient(hunting): _negotiate 2024-01-24T23:22:59.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:59.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:59.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 10 monclient(hunting): my global_id is 4867 2024-01-24T23:22:59.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 1 -- 172.21.15.132:0/904267901 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd2c003530 con 0x7efd40141f00 2024-01-24T23:22:59.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 1 -- 172.21.15.132:0/904267901 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3788685957 0 0) 0x7efd2c003530 con 0x7efd40141f00 2024-01-24T23:22:59.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 1 -- 172.21.15.132:0/904267901 >> v1:172.21.15.115:6793/0 conn(0x7efd400a09a0 legacy=0x7efd400a0d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:59.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 1 -- 172.21.15.132:0/904267901 >> v1:172.21.15.115:6790/0 conn(0x7efd4013e3f0 legacy=0x7efd401407e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:22:59.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 10 monclient: _finish_hunting 0 2024-01-24T23:22:59.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 1 monclient: found mon.noname-i 2024-01-24T23:22:59.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:59.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:22:59.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 1 -- 172.21.15.132:0/904267901 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4009e550 con 0x7efd40141f00 2024-01-24T23:22:59.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 10 monclient: _finish_auth 0 2024-01-24T23:22:59.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 10 monclient: _check_auth_tickets 2024-01-24T23:22:59.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:59.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 1 -- 172.21.15.132:0/904267901 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7efd380017c0 con 0x7efd40141f00 2024-01-24T23:22:59.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 5 monclient: authenticate success, global_id 4867 2024-01-24T23:22:59.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:59.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:59.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:22:59.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 10 monclient: dump: 2024-01-24T23:22:59.554 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:59.554 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:59.554 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:59.554 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:59.554 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:59.554 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:59.554 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:59.555 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:22:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 1 -- 172.21.15.132:0/904267901 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efd38001ae0 con 0x7efd40141f00 2024-01-24T23:22:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 10 monclient: get_monmap_and_config success 2024-01-24T23:22:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 4 set_mon_vals no callback set 2024-01-24T23:22:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 1 -- 172.21.15.132:0/904267901 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7efd38004bf0 con 0x7efd40141f00 2024-01-24T23:22:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:59.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:22:59.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd37fff640 10 monclient: dump: 2024-01-24T23:22:59.556 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:59.556 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:59.556 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:59.556 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:59.556 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:59.556 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:59.556 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:59.556 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:59.556 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:59.556 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:59.556 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:59.556 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:59.557 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:59.557 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:59.557 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:59.557 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:59.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 10 monclient: shutdown 2024-01-24T23:22:59.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:22:59.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 1 -- 172.21.15.132:0/904267901 >> v1:172.21.15.132:6792/0 conn(0x7efd40141f00 legacy=0x7efd401442f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:59.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 1 -- 172.21.15.132:0/904267901 shutdown_connections 2024-01-24T23:22:59.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 1 -- 172.21.15.132:0/904267901 >> 172.21.15.132:0/904267901 conn(0x7efd400a7990 msgr2=0x7efd400a3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:22:59.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 1 -- 172.21.15.132:0/904267901 shutdown_connections 2024-01-24T23:22:59.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.545+0000 7efd47946640 1 -- 172.21.15.132:0/904267901 wait complete. 2024-01-24T23:22:59.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 10 monclient: build_initial_monmap 2024-01-24T23:22:59.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 10 monclient: monmap: 2024-01-24T23:22:59.557 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:22:59.557 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:22:59.557 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:22:59.558 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:22:59.558 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:22:59.558 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:22:59.558 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:22:59.558 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:22:59.558 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:22:59.558 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:22:59.558 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:22:59.558 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:22:59.558 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:22:59.558 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:22:59.558 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:22:59.558 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:59.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 1 Processor -- start 2024-01-24T23:22:59.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 1 -- start start 2024-01-24T23:22:59.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 10 monclient: init 2024-01-24T23:22:59.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 10 monclient: _reopen_session rank -1 2024-01-24T23:22:59.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 10 monclient: _add_conns ranks=[4,7,8,0,3,2,6,1,5] 2024-01-24T23:22:59.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 10 monclient(hunting): picked mon.noname-c con 0x7efd400a09a0 addr v1:172.21.15.115:6791/0 2024-01-24T23:22:59.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 10 monclient(hunting): picked mon.noname-i con 0x7efd4013e3f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:22:59.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 10 monclient(hunting): picked mon.noname-e con 0x7efd40141f00 addr v1:172.21.15.115:6793/0 2024-01-24T23:22:59.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd401d0190 con 0x7efd400a09a0 2024-01-24T23:22:59.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd401cf960 con 0x7efd4013e3f0 2024-01-24T23:22:59.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd400724c0 con 0x7efd40141f00 2024-01-24T23:22:59.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 10 monclient(hunting): _renew_subs 2024-01-24T23:22:59.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 10 monclient(hunting): authenticate will time out at 1066.877067s 2024-01-24T23:22:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd44ee1640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7efd4013e3f0 0x7efd400715a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:38916/0 (socket says 172.21.15.132:38916) 2024-01-24T23:22:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd44ee1640 1 -- 172.21.15.132:0/851464716 learned_addr learned my addr 172.21.15.132:0/851464716 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:22:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd45ee3640 1 -- 172.21.15.132:0/851464716 >> v1:172.21.15.115:6793/0 conn(0x7efd40141f00 legacy=0x7efd401d7590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:22:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 1 -- 172.21.15.132:0/851464716 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 541418763 0 0) 0x7efd401cf960 con 0x7efd4013e3f0 2024-01-24T23:22:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:22:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 10 monclient(hunting): my global_id is 4876 2024-01-24T23:22:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 1 -- 172.21.15.132:0/851464716 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd24003530 con 0x7efd4013e3f0 2024-01-24T23:22:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 1 -- 172.21.15.132:0/851464716 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4255119330 0 0) 0x7efd24003530 con 0x7efd4013e3f0 2024-01-24T23:22:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 1 -- 172.21.15.132:0/851464716 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efd401cf960 con 0x7efd4013e3f0 2024-01-24T23:22:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 1 -- 172.21.15.132:0/851464716 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7efd30004330 con 0x7efd4013e3f0 2024-01-24T23:22:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:59.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:22:59.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 10 monclient(hunting): dump: 2024-01-24T23:22:59.563 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:59.563 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:59.564 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:59.564 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:59.564 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:59.564 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:59.564 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:59.564 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:59.564 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:59.565 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:22:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 1 -- 172.21.15.132:0/851464716 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2999385603 0 0) 0x7efd401d0190 con 0x7efd400a09a0 2024-01-24T23:22:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:22:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:22:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:22:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 10 monclient(hunting): my global_id is 74848 2024-01-24T23:22:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 1 -- 172.21.15.132:0/851464716 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd24003530 con 0x7efd400a09a0 2024-01-24T23:22:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 1 -- 172.21.15.132:0/851464716 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2934131474 0 0) 0x7efd401cf960 con 0x7efd4013e3f0 2024-01-24T23:22:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 1 -- 172.21.15.132:0/851464716 >> v1:172.21.15.115:6793/0 conn(0x7efd40141f00 legacy=0x7efd401d7590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:22:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 1 -- 172.21.15.132:0/851464716 >> v1:172.21.15.115:6791/0 conn(0x7efd400a09a0 legacy=0x7efd401cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:22:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:22:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 1 monclient: found mon.i 2024-01-24T23:22:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:22:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 1 -- 172.21.15.132:0/851464716 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd40140540 con 0x7efd4013e3f0 2024-01-24T23:22:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 10 monclient: _finish_auth 0 2024-01-24T23:22:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:22:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:22:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 5 monclient: authenticate success, global_id 4876 2024-01-24T23:22:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 10 monclient: _renew_subs 2024-01-24T23:22:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 1 -- 172.21.15.132:0/851464716 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd40070070 con 0x7efd4013e3f0 2024-01-24T23:22:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 10 monclient: _renew_subs 2024-01-24T23:22:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:59.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 1 -- 172.21.15.132:0/851464716 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7efd40072030 con 0x7efd4013e3f0 2024-01-24T23:22:59.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 1 -- 172.21.15.132:0/851464716 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efd30004650 con 0x7efd4013e3f0 2024-01-24T23:22:59.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:22:59.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 1 -- 172.21.15.132:0/851464716 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7efd30004b80 con 0x7efd4013e3f0 2024-01-24T23:22:59.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:22:59.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd377fe640 4 set_mon_vals no callback set 2024-01-24T23:22:59.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:22:59.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 10 monclient: dump: 2024-01-24T23:22:59.567 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:22:59.568 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:22:59.568 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:59.568 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:22:59.568 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:22:59.568 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:22:59.568 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:22:59.568 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:22:59.568 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:22:59.568 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:22:59.568 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:22:59.568 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:22:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:22:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:22:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:22:59.569 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:22:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd47946640 1 -- 172.21.15.132:0/851464716 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd20005090 con 0x7efd4013e3f0 2024-01-24T23:22:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 1 -- 172.21.15.132:0/851464716 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7efd30004ce0 con 0x7efd4013e3f0 2024-01-24T23:22:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd35ffb640 1 -- 172.21.15.132:0/851464716 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(38..38 src has 1..38) v4 ==== 3598+0+0 (unknown 3181584953 0 0) 0x7efd30043800 con 0x7efd4013e3f0 2024-01-24T23:22:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.549+0000 7efd456e2640 10 monclient: get_auth_request con 0x7efd24037000 auth_method 0 2024-01-24T23:22:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.553+0000 7efd35ffb640 1 -- 172.21.15.132:0/851464716 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7efd30014b20 con 0x7efd4013e3f0 2024-01-24T23:22:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.557+0000 7efd35ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:22:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.557+0000 7efd35ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:22:59.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.713+0000 7efd47946640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-01-24T23:22:59.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.713+0000 7efd47946640 10 monclient: _send_command 2 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-01-24T23:22:59.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.713+0000 7efd47946640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:22:59.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:22:59.713+0000 7efd47946640 1 -- 172.21.15.132:0/851464716 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"} v 0) v1 -- 0x7efd200058c0 con 0x7efd4013e3f0 2024-01-24T23:23:00.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.025+0000 7efd35ffb640 1 -- 172.21.15.132:0/851464716 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}]=0 v39) v1 ==== 126+0+0 (unknown 2240183908 0 0) 0x7efd30012f90 con 0x7efd4013e3f0 2024-01-24T23:23:00.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.025+0000 7efd35ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-01-24T23:23:00.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.025+0000 7efd35ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:00.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.129+0000 7efd47946640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-01-24T23:23:00.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.129+0000 7efd47946640 10 monclient: _send_command 3 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-01-24T23:23:00.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.129+0000 7efd47946640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:00.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.129+0000 7efd47946640 1 -- 172.21.15.132:0/851464716 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"} v 0) v1 -- 0x7efd200053f0 con 0x7efd4013e3f0 2024-01-24T23:23:00.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.129+0000 7efd35ffb640 1 -- 172.21.15.132:0/851464716 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}]=0 rule foo already exists v39) v1 ==== 149+0+0 (unknown 703702327 0 0) 0x7efd200053f0 con 0x7efd4013e3f0 2024-01-24T23:23:00.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.129+0000 7efd35ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-01-24T23:23:00.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.129+0000 7efd35ffb640 10 monclient: _finish_command 3 = system:0 rule foo already exists 2024-01-24T23:23:00.134 INFO:tasks.workunit.client.0.smithi132.stderr:rule foo already exists 2024-01-24T23:23:00.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.133+0000 7efd47946640 1 -- 172.21.15.132:0/851464716 >> v1:172.21.15.132:6812/1835401922 conn(0x7efd24037000 legacy=0x7efd240394c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:00.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.133+0000 7efd47946640 10 monclient: shutdown 2024-01-24T23:23:00.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.133+0000 7efd47946640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:00.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.133+0000 7efd47946640 1 -- 172.21.15.132:0/851464716 >> v1:172.21.15.132:6792/0 conn(0x7efd4013e3f0 legacy=0x7efd400715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:00.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.133+0000 7efd47946640 1 -- 172.21.15.132:0/851464716 shutdown_connections 2024-01-24T23:23:00.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.133+0000 7efd47946640 1 -- 172.21.15.132:0/851464716 >> 172.21.15.132:0/851464716 conn(0x7efd400a7990 msgr2=0x7efd400a3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:00.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.133+0000 7efd47946640 1 -- 172.21.15.132:0/851464716 shutdown_connections 2024-01-24T23:23:00.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.133+0000 7efd47946640 1 -- 172.21.15.132:0/851464716 wait complete. 2024-01-24T23:23:00.151 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule create-simple foo default host 2024-01-24T23:23:00.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.268+0000 7fc1ac57a640 10 monclient: get_monmap_and_config 2024-01-24T23:23:00.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.268+0000 7fc1ac57a640 10 monclient: build_initial_monmap 2024-01-24T23:23:00.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.268+0000 7fc1ac57a640 10 monclient: monmap: 2024-01-24T23:23:00.274 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:00.274 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:00.274 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:00.275173+0000 2024-01-24T23:23:00.274 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:00.275173+0000 2024-01-24T23:23:00.275 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:00.275 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:00.275 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:00.275 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:00.275 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:00.275 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:00.276 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:00.276 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:00.276 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:00.276 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:00.276 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:00.276 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:00.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.268+0000 7fc1ac57a640 1 Processor -- start 2024-01-24T23:23:00.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.268+0000 7fc1ac57a640 1 -- start start 2024-01-24T23:23:00.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.268+0000 7fc1ac57a640 10 monclient: init 2024-01-24T23:23:00.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:00.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 10 monclient: _add_conns ranks=[7,1,0,5,8,6,4,2,3] 2024-01-24T23:23:00.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 10 monclient(hunting): picked mon.noname-i con 0x7fc1a40a13c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:00.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 10 monclient(hunting): picked mon.noname-f con 0x7fc1a406e410 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:00.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 10 monclient(hunting): picked mon.noname-a con 0x7fc1a40a1eb0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:00.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a4064700 con 0x7fc1a40a1eb0 2024-01-24T23:23:00.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a406e030 con 0x7fc1a406e410 2024-01-24T23:23:00.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a406ec30 con 0x7fc1a40a13c0 2024-01-24T23:23:00.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:00.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 10 monclient(hunting): authenticate will time out at 1067.600760s 2024-01-24T23:23:00.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1aa316640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fc1a40a13c0 0x7fc1a40a17a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:38932/0 (socket says 172.21.15.132:38932) 2024-01-24T23:23:00.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1aa316640 1 -- 172.21.15.132:0/2120514330 learned_addr learned my addr 172.21.15.132:0/2120514330 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:00.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9b15640 1 --1- 172.21.15.132:0/2120514330 >> v1:172.21.15.132:6789/0 conn(0x7fc1a406e410 0x7fc1a41385c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:53100/0 (socket says 172.21.15.132:53100) 2024-01-24T23:23:00.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 1 -- 172.21.15.132:0/2120514330 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 705678687 0 0) 0x7fc1a406ec30 con 0x7fc1a40a13c0 2024-01-24T23:23:00.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 10 monclient(hunting): _negotiate 2024-01-24T23:23:00.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:00.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:00.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 10 monclient(hunting): my global_id is 4885 2024-01-24T23:23:00.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 1 -- 172.21.15.132:0/2120514330 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc190003450 con 0x7fc1a40a13c0 2024-01-24T23:23:00.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 1 -- 172.21.15.132:0/2120514330 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4202418221 0 0) 0x7fc1a406e030 con 0x7fc1a406e410 2024-01-24T23:23:00.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 10 monclient(hunting): _negotiate 2024-01-24T23:23:00.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:00.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:00.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 10 monclient(hunting): my global_id is 4870 2024-01-24T23:23:00.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 1 -- 172.21.15.132:0/2120514330 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1a406ec30 con 0x7fc1a406e410 2024-01-24T23:23:00.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 1 -- 172.21.15.132:0/2120514330 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1305828525 0 0) 0x7fc1a406ec30 con 0x7fc1a406e410 2024-01-24T23:23:00.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 1 -- 172.21.15.132:0/2120514330 >> v1:172.21.15.132:6792/0 conn(0x7fc1a40a13c0 legacy=0x7fc1a40a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:00.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 1 -- 172.21.15.132:0/2120514330 >> v1:172.21.15.115:6789/0 conn(0x7fc1a40a1eb0 legacy=0x7fc1a413bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:00.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 10 monclient: _finish_hunting 0 2024-01-24T23:23:00.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 1 monclient: found mon.noname-f 2024-01-24T23:23:00.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:00.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:23:00.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 1 -- 172.21.15.132:0/2120514330 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a406d610 con 0x7fc1a406e410 2024-01-24T23:23:00.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 10 monclient: _finish_auth 0 2024-01-24T23:23:00.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 10 monclient: _check_auth_tickets 2024-01-24T23:23:00.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:00.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 1 -- 172.21.15.132:0/2120514330 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc1940017c0 con 0x7fc1a406e410 2024-01-24T23:23:00.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:00.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:23:00.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 10 monclient: dump: 2024-01-24T23:23:00.286 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:00.286 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:00.286 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:00.286 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:00.286 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:00.287 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:00.287 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:00.287 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:00.287 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:00.287 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:00.287 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:00.287 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:00.287 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:00.287 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:00.287 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:00.287 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:00.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 5 monclient: authenticate success, global_id 4870 2024-01-24T23:23:00.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:00.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 1 -- 172.21.15.132:0/2120514330 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc194001a60 con 0x7fc1a406e410 2024-01-24T23:23:00.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:00.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 1 -- 172.21.15.132:0/2120514330 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc194004b30 con 0x7fc1a406e410 2024-01-24T23:23:00.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:00.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 4 set_mon_vals no callback set 2024-01-24T23:23:00.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:00.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:23:00.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1a9314640 10 monclient: dump: 2024-01-24T23:23:00.288 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:00.288 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:00.288 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:00.288 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:00.288 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:00.288 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:00.288 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:00.289 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:00.289 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:00.289 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:00.289 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:00.289 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:00.289 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:00.289 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:00.289 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:00.289 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:00.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 10 monclient: shutdown 2024-01-24T23:23:00.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:00.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 1 -- 172.21.15.132:0/2120514330 >> v1:172.21.15.132:6789/0 conn(0x7fc1a406e410 legacy=0x7fc1a41385c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:00.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 1 -- 172.21.15.132:0/2120514330 shutdown_connections 2024-01-24T23:23:00.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 1 -- 172.21.15.132:0/2120514330 >> 172.21.15.132:0/2120514330 conn(0x7fc1a406aaf0 msgr2=0x7fc1a40a24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:00.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 1 -- 172.21.15.132:0/2120514330 shutdown_connections 2024-01-24T23:23:00.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 1 -- 172.21.15.132:0/2120514330 wait complete. 2024-01-24T23:23:00.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 10 monclient: build_initial_monmap 2024-01-24T23:23:00.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 10 monclient: monmap: 2024-01-24T23:23:00.290 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:00.290 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:00.290 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:00.290 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:00.290 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:00.290 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:00.290 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:00.290 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:00.290 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:00.290 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:00.290 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:00.290 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:00.290 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:00.290 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:00.290 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:00.291 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:00.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 1 Processor -- start 2024-01-24T23:23:00.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 1 -- start start 2024-01-24T23:23:00.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 10 monclient: init 2024-01-24T23:23:00.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:00.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 10 monclient: _add_conns ranks=[5,1,4,6,8,0,3,2,7] 2024-01-24T23:23:00.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 10 monclient(hunting): picked mon.noname-h con 0x7fc1a406e410 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:00.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 10 monclient(hunting): picked mon.noname-f con 0x7fc1a40a13c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:00.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 10 monclient(hunting): picked mon.noname-c con 0x7fc1a40a1eb0 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:00.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a41a2600 con 0x7fc1a40a13c0 2024-01-24T23:23:00.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a41a4780 con 0x7fc1a40a1eb0 2024-01-24T23:23:00.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a406e900 con 0x7fc1a406e410 2024-01-24T23:23:00.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:00.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.272+0000 7fc1ac57a640 10 monclient(hunting): authenticate will time out at 1067.603950s 2024-01-24T23:23:00.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc1a9b15640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fc1a40a13c0 0x7fc1a41c6f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:53114/0 (socket says 172.21.15.132:53114) 2024-01-24T23:23:00.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc1a9b15640 1 -- 172.21.15.132:0/2635341410 learned_addr learned my addr 172.21.15.132:0/2635341410 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:00.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 1 -- 172.21.15.132:0/2635341410 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3866407675 0 0) 0x7fc1a406e900 con 0x7fc1a406e410 2024-01-24T23:23:00.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:00.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:00.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:00.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient(hunting): my global_id is 4838 2024-01-24T23:23:00.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 1 -- 172.21.15.132:0/2635341410 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc188003530 con 0x7fc1a406e410 2024-01-24T23:23:00.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 1 -- 172.21.15.132:0/2635341410 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2285821925 0 0) 0x7fc1a41a2600 con 0x7fc1a40a13c0 2024-01-24T23:23:00.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:00.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:00.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:00.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient(hunting): my global_id is 4879 2024-01-24T23:23:00.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 1 -- 172.21.15.132:0/2635341410 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1a406e900 con 0x7fc1a40a13c0 2024-01-24T23:23:00.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 1 -- 172.21.15.132:0/2635341410 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1112057649 0 0) 0x7fc188003530 con 0x7fc1a406e410 2024-01-24T23:23:00.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 1 -- 172.21.15.132:0/2635341410 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc1a41a2600 con 0x7fc1a406e410 2024-01-24T23:23:00.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 1 -- 172.21.15.132:0/2635341410 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc1a0001a80 con 0x7fc1a406e410 2024-01-24T23:23:00.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:00.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:00.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient(hunting): dump: 2024-01-24T23:23:00.293 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:00.293 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:00.293 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:00.293 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:00.293 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:00.293 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:00.293 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:00.293 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:00.293 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:00.293 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:00.293 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:00.293 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:00.294 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:00.294 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:00.294 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:00.294 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:00.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:00.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 1 -- 172.21.15.132:0/2635341410 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 847957125 0 0) 0x7fc1a406e900 con 0x7fc1a40a13c0 2024-01-24T23:23:00.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 1 -- 172.21.15.132:0/2635341410 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc188003530 con 0x7fc1a40a13c0 2024-01-24T23:23:00.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 1 -- 172.21.15.132:0/2635341410 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc194001780 con 0x7fc1a40a13c0 2024-01-24T23:23:00.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:00.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:23:00.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient(hunting): dump: 2024-01-24T23:23:00.294 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:00.294 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:00.294 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:00.294 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:00.295 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:00.295 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:00.295 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:00.295 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:00.295 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:00.295 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:00.295 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:00.295 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:00.295 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:00.295 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:00.295 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:00.295 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:00.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 1 -- 172.21.15.132:0/2635341410 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 261975458 0 0) 0x7fc1a41a4780 con 0x7fc1a40a1eb0 2024-01-24T23:23:00.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:00.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:00.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:00.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient(hunting): my global_id is 74857 2024-01-24T23:23:00.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 1 -- 172.21.15.132:0/2635341410 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1a406e900 con 0x7fc1a40a1eb0 2024-01-24T23:23:00.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 1 -- 172.21.15.132:0/2635341410 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2631419623 0 0) 0x7fc1a41a2600 con 0x7fc1a406e410 2024-01-24T23:23:00.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 1 -- 172.21.15.132:0/2635341410 >> v1:172.21.15.115:6791/0 conn(0x7fc1a40a1eb0 legacy=0x7fc1a41cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:00.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 1 -- 172.21.15.132:0/2635341410 >> v1:172.21.15.132:6789/0 conn(0x7fc1a40a13c0 legacy=0x7fc1a41c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:00.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:00.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 1 monclient: found mon.h 2024-01-24T23:23:00.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:00.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:00.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 1 -- 172.21.15.132:0/2635341410 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a41b7d50 con 0x7fc1a406e410 2024-01-24T23:23:00.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient: _finish_auth 0 2024-01-24T23:23:00.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:00.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:00.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc1ac57a640 5 monclient: authenticate success, global_id 4838 2024-01-24T23:23:00.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc1ac57a640 10 monclient: _renew_subs 2024-01-24T23:23:00.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc1ac57a640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:00.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc1ac57a640 1 -- 172.21.15.132:0/2635341410 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a4071840 con 0x7fc1a406e410 2024-01-24T23:23:00.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc1ac57a640 10 monclient: _renew_subs 2024-01-24T23:23:00.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc1ac57a640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:00.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc1ac57a640 1 -- 172.21.15.132:0/2635341410 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a41a5020 con 0x7fc1a406e410 2024-01-24T23:23:00.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc1ac57a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:00.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc1ac57a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:00.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc1ac57a640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:00.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc1ac57a640 1 -- 172.21.15.132:0/2635341410 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a4073400 con 0x7fc1a406e410 2024-01-24T23:23:00.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 1 -- 172.21.15.132:0/2635341410 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc1a0001ca0 con 0x7fc1a406e410 2024-01-24T23:23:00.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:00.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 1 -- 172.21.15.132:0/2635341410 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc1a0004e60 con 0x7fc1a406e410 2024-01-24T23:23:00.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:00.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:00.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 10 monclient: dump: 2024-01-24T23:23:00.298 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:00.298 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:00.298 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:00.298 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:00.298 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:00.298 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:00.298 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:00.298 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:00.298 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:00.298 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:00.298 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:00.298 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:00.298 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:00.298 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:00.298 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:00.298 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:00.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc1a8b13640 4 set_mon_vals no callback set 2024-01-24T23:23:00.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 1 -- 172.21.15.132:0/2635341410 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fc1a000f8e0 con 0x7fc1a406e410 2024-01-24T23:23:00.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc19affd640 1 -- 172.21.15.132:0/2635341410 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(39..39 src has 1..39) v4 ==== 3657+0+0 (unknown 2638850501 0 0) 0x7fc1a0043520 con 0x7fc1a406e410 2024-01-24T23:23:00.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.276+0000 7fc1a9b15640 10 monclient: get_auth_request con 0x7fc188037830 auth_method 0 2024-01-24T23:23:00.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.280+0000 7fc19affd640 1 -- 172.21.15.132:0/2635341410 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fc1a0014630 con 0x7fc1a406e410 2024-01-24T23:23:00.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.280+0000 7fc19affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:00.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.280+0000 7fc19affd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:00.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.436+0000 7fc1ac57a640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-01-24T23:23:00.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.436+0000 7fc1ac57a640 10 monclient: _send_command 2 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-01-24T23:23:00.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.436+0000 7fc1ac57a640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:00.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.436+0000 7fc1ac57a640 1 -- 172.21.15.132:0/2635341410 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"} v 0) v1 -- 0x7fc1a406b640 con 0x7fc1a406e410 2024-01-24T23:23:00.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.440+0000 7fc19affd640 1 -- 172.21.15.132:0/2635341410 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}]=0 rule foo already exists v39) v1 ==== 149+0+0 (unknown 703702327 0 0) 0x7fc1a0012cd0 con 0x7fc1a406e410 2024-01-24T23:23:00.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.440+0000 7fc19affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-01-24T23:23:00.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.440+0000 7fc19affd640 10 monclient: _finish_command 2 = system:0 rule foo already exists 2024-01-24T23:23:00.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.536+0000 7fc1ac57a640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-01-24T23:23:00.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.536+0000 7fc1ac57a640 10 monclient: _send_command 3 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-01-24T23:23:00.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.536+0000 7fc1ac57a640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:00.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.536+0000 7fc1ac57a640 1 -- 172.21.15.132:0/2635341410 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"} v 0) v1 -- 0x7fc1a406fee0 con 0x7fc1a406e410 2024-01-24T23:23:00.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.536+0000 7fc19affd640 1 -- 172.21.15.132:0/2635341410 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}]=0 rule foo already exists v39) v1 ==== 149+0+0 (unknown 703702327 0 0) 0x7fc1a0015190 con 0x7fc1a406e410 2024-01-24T23:23:00.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.536+0000 7fc19affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-01-24T23:23:00.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.536+0000 7fc19affd640 10 monclient: _finish_command 3 = system:0 rule foo already exists 2024-01-24T23:23:00.543 INFO:tasks.workunit.client.0.smithi132.stderr:rule foo already exists 2024-01-24T23:23:00.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.540+0000 7fc1ac57a640 1 -- 172.21.15.132:0/2635341410 >> v1:172.21.15.132:6812/1835401922 conn(0x7fc188037830 legacy=0x7fc188039cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:00.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.540+0000 7fc1ac57a640 10 monclient: shutdown 2024-01-24T23:23:00.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.540+0000 7fc1ac57a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:00.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.540+0000 7fc1ac57a640 1 -- 172.21.15.132:0/2635341410 >> v1:172.21.15.132:6791/0 conn(0x7fc1a406e410 legacy=0x7fc1a41b7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:00.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.540+0000 7fc1ac57a640 1 -- 172.21.15.132:0/2635341410 shutdown_connections 2024-01-24T23:23:00.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.540+0000 7fc1ac57a640 1 -- 172.21.15.132:0/2635341410 >> 172.21.15.132:0/2635341410 conn(0x7fc1a406aaf0 msgr2=0x7fc1a4070710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:00.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.540+0000 7fc1ac57a640 1 -- 172.21.15.132:0/2635341410 shutdown_connections 2024-01-24T23:23:00.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.540+0000 7fc1ac57a640 1 -- 172.21.15.132:0/2635341410 wait complete. 2024-01-24T23:23:00.560 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule create-simple bar default host 2024-01-24T23:23:00.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b97286640 10 monclient: get_monmap_and_config 2024-01-24T23:23:00.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b97286640 10 monclient: build_initial_monmap 2024-01-24T23:23:00.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b97286640 10 monclient: monmap: 2024-01-24T23:23:00.676 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:00.676 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:00.676 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:00.678031+0000 2024-01-24T23:23:00.677 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:00.678031+0000 2024-01-24T23:23:00.677 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:00.677 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:00.677 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:00.677 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:00.677 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:00.677 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:00.677 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:00.677 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:00.677 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:00.677 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:00.677 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:00.677 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:00.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b97286640 1 Processor -- start 2024-01-24T23:23:00.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b97286640 1 -- start start 2024-01-24T23:23:00.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b97286640 10 monclient: init 2024-01-24T23:23:00.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b97286640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:00.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b97286640 10 monclient: _add_conns ranks=[2,5,3,6,8,4,7,1,0] 2024-01-24T23:23:00.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b97286640 10 monclient(hunting): picked mon.noname-b con 0x7f9b90072690 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:00.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b97286640 10 monclient(hunting): picked mon.noname-h con 0x7f9b9009ae90 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:00.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b97286640 10 monclient(hunting): picked mon.noname-g con 0x7f9b9009bdc0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:00.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b97286640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b90064700 con 0x7f9b90072690 2024-01-24T23:23:00.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b97286640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b9009a790 con 0x7f9b9009bdc0 2024-01-24T23:23:00.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b97286640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b900a18f0 con 0x7f9b9009ae90 2024-01-24T23:23:00.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b97286640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:00.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b97286640 10 monclient(hunting): authenticate will time out at 1068.002792s 2024-01-24T23:23:00.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b94821640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f9b9009ae90 0x7f9b900a11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:39354/0 (socket says 172.21.15.132:39354) 2024-01-24T23:23:00.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b94821640 1 -- 172.21.15.132:0/3125136835 learned_addr learned my addr 172.21.15.132:0/3125136835 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:00.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b95022640 1 -- 172.21.15.132:0/3125136835 >> v1:172.21.15.115:6790/0 conn(0x7f9b90072690 legacy=0x7f9b900a0ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:23:00.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 1 -- 172.21.15.132:0/3125136835 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 999239988 0 0) 0x7f9b9009a790 con 0x7f9b9009bdc0 2024-01-24T23:23:00.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:00.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:00.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:00.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 10 monclient(hunting): my global_id is 4782 2024-01-24T23:23:00.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 1 -- 172.21.15.132:0/3125136835 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b7c003530 con 0x7f9b9009bdc0 2024-01-24T23:23:00.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 1 -- 172.21.15.132:0/3125136835 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 864161821 0 0) 0x7f9b900a18f0 con 0x7f9b9009ae90 2024-01-24T23:23:00.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:00.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:00.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:00.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 10 monclient(hunting): my global_id is 4847 2024-01-24T23:23:00.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 1 -- 172.21.15.132:0/3125136835 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b9009a790 con 0x7f9b9009ae90 2024-01-24T23:23:00.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 1 -- 172.21.15.132:0/3125136835 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3088060215 0 0) 0x7f9b7c003530 con 0x7f9b9009bdc0 2024-01-24T23:23:00.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 1 -- 172.21.15.132:0/3125136835 >> v1:172.21.15.132:6791/0 conn(0x7f9b9009ae90 legacy=0x7f9b900a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:00.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 1 -- 172.21.15.132:0/3125136835 >> v1:172.21.15.115:6790/0 conn(0x7f9b90072690 legacy=0x7f9b900a0ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:00.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 10 monclient: _finish_hunting 0 2024-01-24T23:23:00.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 1 monclient: found mon.noname-g 2024-01-24T23:23:00.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:00.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:23:00.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 1 -- 172.21.15.132:0/3125136835 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b900997a0 con 0x7f9b9009bdc0 2024-01-24T23:23:00.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 10 monclient: _finish_auth 0 2024-01-24T23:23:00.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 10 monclient: _check_auth_tickets 2024-01-24T23:23:00.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:00.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 1 -- 172.21.15.132:0/3125136835 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9b780017c0 con 0x7f9b9009bdc0 2024-01-24T23:23:00.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b97286640 5 monclient: authenticate success, global_id 4782 2024-01-24T23:23:00.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b97286640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:00.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:00.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:00.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 10 monclient: dump: 2024-01-24T23:23:00.687 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:00.687 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:00.687 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:00.687 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:00.687 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:00.687 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:00.687 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:00.687 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:00.687 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:00.688 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:00.688 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:00.688 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:00.688 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:00.688 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:00.688 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:00.688 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:00.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b97286640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:00.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 1 -- 172.21.15.132:0/3125136835 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9b780019b0 con 0x7f9b9009bdc0 2024-01-24T23:23:00.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:00.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 1 -- 172.21.15.132:0/3125136835 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9b78003960 con 0x7f9b9009bdc0 2024-01-24T23:23:00.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b97286640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:00.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b97286640 4 set_mon_vals no callback set 2024-01-24T23:23:00.689 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:00.689 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:00.689 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.672+0000 7f9b8ffff640 10 monclient: dump: 2024-01-24T23:23:00.689 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:00.689 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:00.689 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:00.689 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:00.689 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:00.689 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:00.689 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:00.689 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:00.689 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:00.689 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:00.689 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:00.689 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:00.689 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:00.689 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:00.690 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:00.690 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:00.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 10 monclient: shutdown 2024-01-24T23:23:00.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:00.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 1 -- 172.21.15.132:0/3125136835 >> v1:172.21.15.132:6790/0 conn(0x7f9b9009bdc0 legacy=0x7f9b9013d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:00.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 1 -- 172.21.15.132:0/3125136835 shutdown_connections 2024-01-24T23:23:00.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 1 -- 172.21.15.132:0/3125136835 >> 172.21.15.132:0/3125136835 conn(0x7f9b900a1bc0 msgr2=0x7f9b900a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:00.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 1 -- 172.21.15.132:0/3125136835 shutdown_connections 2024-01-24T23:23:00.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 1 -- 172.21.15.132:0/3125136835 wait complete. 2024-01-24T23:23:00.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 10 monclient: build_initial_monmap 2024-01-24T23:23:00.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 10 monclient: monmap: 2024-01-24T23:23:00.690 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:00.690 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:00.690 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:00.690 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:00.691 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:00.691 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:00.691 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:00.691 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:00.691 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:00.691 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:00.691 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:00.691 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:00.691 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:00.691 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:00.691 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:00.691 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:00.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 1 Processor -- start 2024-01-24T23:23:00.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 1 -- start start 2024-01-24T23:23:00.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 10 monclient: init 2024-01-24T23:23:00.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:00.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 10 monclient: _add_conns ranks=[7,6,2,4,8,5,1,0,3] 2024-01-24T23:23:00.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 10 monclient(hunting): picked mon.noname-i con 0x7f9b90072690 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:00.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 10 monclient(hunting): picked mon.noname-d con 0x7f9b9009ae90 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:00.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 10 monclient(hunting): picked mon.noname-b con 0x7f9b9009bdc0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:00.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b901b0dc0 con 0x7f9b9009bdc0 2024-01-24T23:23:00.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b901b2070 con 0x7f9b9009ae90 2024-01-24T23:23:00.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b90099470 con 0x7f9b90072690 2024-01-24T23:23:00.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:00.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 10 monclient(hunting): authenticate will time out at 1068.006057s 2024-01-24T23:23:00.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b95022640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f9b90072690 0x7f9b90098020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:38944/0 (socket says 172.21.15.132:38944) 2024-01-24T23:23:00.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b95022640 1 -- 172.21.15.132:0/271552635 learned_addr learned my addr 172.21.15.132:0/271552635 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:00.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b95823640 1 -- 172.21.15.132:0/271552635 >> v1:172.21.15.115:6790/0 conn(0x7f9b9009bdc0 legacy=0x7f9b901d1dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:23:00.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 1 -- 172.21.15.132:0/271552635 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 846573772 0 0) 0x7f9b90099470 con 0x7f9b90072690 2024-01-24T23:23:00.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:00.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:00.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:00.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 10 monclient(hunting): my global_id is 4894 2024-01-24T23:23:00.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 1 -- 172.21.15.132:0/271552635 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b70003530 con 0x7f9b90072690 2024-01-24T23:23:00.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 1 -- 172.21.15.132:0/271552635 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3531594171 0 0) 0x7f9b70003530 con 0x7f9b90072690 2024-01-24T23:23:00.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 1 -- 172.21.15.132:0/271552635 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9b90099470 con 0x7f9b90072690 2024-01-24T23:23:00.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 1 -- 172.21.15.132:0/271552635 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9b80004330 con 0x7f9b90072690 2024-01-24T23:23:00.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:00.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:23:00.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 10 monclient(hunting): dump: 2024-01-24T23:23:00.693 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:00.693 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:00.693 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:00.693 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:00.693 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:00.694 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:00.694 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:00.694 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:00.694 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:00.694 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:00.694 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:00.694 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:00.694 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:00.694 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:00.694 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:00.694 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:00.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:00.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 1 -- 172.21.15.132:0/271552635 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1994400604 0 0) 0x7f9b901b2070 con 0x7f9b9009ae90 2024-01-24T23:23:00.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:00.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:00.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:00.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 10 monclient(hunting): my global_id is 4893 2024-01-24T23:23:00.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 1 -- 172.21.15.132:0/271552635 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b70003530 con 0x7f9b9009ae90 2024-01-24T23:23:00.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 1 -- 172.21.15.132:0/271552635 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 338066980 0 0) 0x7f9b90099470 con 0x7f9b90072690 2024-01-24T23:23:00.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 1 -- 172.21.15.132:0/271552635 >> v1:172.21.15.115:6792/0 conn(0x7f9b9009ae90 legacy=0x7f9b90098730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:00.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 1 -- 172.21.15.132:0/271552635 >> v1:172.21.15.115:6790/0 conn(0x7f9b9009bdc0 legacy=0x7f9b901d1dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:00.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 10 monclient: _finish_hunting 0 2024-01-24T23:23:00.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 1 monclient: found mon.i 2024-01-24T23:23:00.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:00.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:00.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 1 -- 172.21.15.132:0/271552635 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9009b930 con 0x7f9b90072690 2024-01-24T23:23:00.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 10 monclient: _finish_auth 0 2024-01-24T23:23:00.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 10 monclient: _check_auth_tickets 2024-01-24T23:23:00.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:00.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 5 monclient: authenticate success, global_id 4894 2024-01-24T23:23:00.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 10 monclient: _renew_subs 2024-01-24T23:23:00.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:00.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 1 -- 172.21.15.132:0/271552635 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b90070c80 con 0x7f9b90072690 2024-01-24T23:23:00.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 10 monclient: _renew_subs 2024-01-24T23:23:00.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:00.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 1 -- 172.21.15.132:0/271552635 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b901b1660 con 0x7f9b90072690 2024-01-24T23:23:00.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 1 -- 172.21.15.132:0/271552635 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9b80004a50 con 0x7f9b90072690 2024-01-24T23:23:00.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:00.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 1 -- 172.21.15.132:0/271552635 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9b80004f00 con 0x7f9b90072690 2024-01-24T23:23:00.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:00.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8f7fe640 4 set_mon_vals no callback set 2024-01-24T23:23:00.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:00.697 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 10 monclient: dump: 2024-01-24T23:23:00.697 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:00.697 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:00.697 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:00.697 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:00.697 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:00.697 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:00.697 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:00.697 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:00.697 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:00.697 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:00.697 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:00.697 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:00.697 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:00.697 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:00.698 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:00.698 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:00.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:00.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:00.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:00.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b8dffb640 1 -- 172.21.15.132:0/271552635 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f9b8000fa00 con 0x7f9b90072690 2024-01-24T23:23:00.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.676+0000 7f9b97286640 1 -- 172.21.15.132:0/271552635 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b74005090 con 0x7f9b90072690 2024-01-24T23:23:00.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.680+0000 7f9b8dffb640 1 -- 172.21.15.132:0/271552635 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(39..39 src has 1..39) v4 ==== 3657+0+0 (unknown 2638850501 0 0) 0x7f9b80043790 con 0x7f9b90072690 2024-01-24T23:23:00.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.680+0000 7f9b94821640 10 monclient: get_auth_request con 0x7f9b70036f80 auth_method 0 2024-01-24T23:23:00.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.684+0000 7f9b8dffb640 1 -- 172.21.15.132:0/271552635 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f9b80014980 con 0x7f9b90072690 2024-01-24T23:23:00.699 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.684+0000 7f9b8dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:00.699 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.684+0000 7f9b8dffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:00.843 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.840+0000 7f9b97286640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}] 2024-01-24T23:23:00.844 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.840+0000 7f9b97286640 10 monclient: _send_command 2 [{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}] 2024-01-24T23:23:00.844 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.840+0000 7f9b97286640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:00.844 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:00.840+0000 7f9b97286640 1 -- 172.21.15.132:0/271552635 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"} v 0) v1 -- 0x7f9b740058c0 con 0x7f9b90072690 2024-01-24T23:23:01.046 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.040+0000 7f9b8dffb640 1 -- 172.21.15.132:0/271552635 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}]=0 v40) v1 ==== 126+0+0 (unknown 3432132072 0 0) 0x7f9b80013020 con 0x7f9b90072690 2024-01-24T23:23:01.047 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.040+0000 7f9b8dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}] 2024-01-24T23:23:01.047 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.040+0000 7f9b8dffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:01.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.140+0000 7f9b97286640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}] 2024-01-24T23:23:01.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.140+0000 7f9b97286640 10 monclient: _send_command 3 [{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}] 2024-01-24T23:23:01.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.140+0000 7f9b97286640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:01.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.140+0000 7f9b97286640 1 -- 172.21.15.132:0/271552635 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"} v 0) v1 -- 0x7f9b740053f0 con 0x7f9b90072690 2024-01-24T23:23:01.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.140+0000 7f9b8dffb640 1 -- 172.21.15.132:0/271552635 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}]=0 rule bar already exists v40) v1 ==== 149+0+0 (unknown 1256885253 0 0) 0x7f9b740053f0 con 0x7f9b90072690 2024-01-24T23:23:01.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.140+0000 7f9b8dffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}] 2024-01-24T23:23:01.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.140+0000 7f9b8dffb640 10 monclient: _finish_command 3 = system:0 rule bar already exists 2024-01-24T23:23:01.146 INFO:tasks.workunit.client.0.smithi132.stderr:rule bar already exists 2024-01-24T23:23:01.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.144+0000 7f9b97286640 1 -- 172.21.15.132:0/271552635 >> v1:172.21.15.132:6812/1835401922 conn(0x7f9b70036f80 legacy=0x7f9b70039440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:01.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.144+0000 7f9b97286640 10 monclient: shutdown 2024-01-24T23:23:01.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.144+0000 7f9b97286640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:01.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.144+0000 7f9b97286640 1 -- 172.21.15.132:0/271552635 >> v1:172.21.15.132:6792/0 conn(0x7f9b90072690 legacy=0x7f9b90098020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:01.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.144+0000 7f9b97286640 1 -- 172.21.15.132:0/271552635 shutdown_connections 2024-01-24T23:23:01.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.144+0000 7f9b97286640 1 -- 172.21.15.132:0/271552635 >> 172.21.15.132:0/271552635 conn(0x7f9b900a1bc0 msgr2=0x7f9b900a3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:01.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.144+0000 7f9b97286640 1 -- 172.21.15.132:0/271552635 shutdown_connections 2024-01-24T23:23:01.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.144+0000 7f9b97286640 1 -- 172.21.15.132:0/271552635 wait complete. 2024-01-24T23:23:01.163 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm-device-class all 2024-01-24T23:23:01.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.268+0000 7f5862770640 10 monclient: get_monmap_and_config 2024-01-24T23:23:01.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.268+0000 7f5862770640 10 monclient: build_initial_monmap 2024-01-24T23:23:01.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.268+0000 7f5862770640 10 monclient: monmap: 2024-01-24T23:23:01.274 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:01.274 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:01.274 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:01.275705+0000 2024-01-24T23:23:01.274 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:01.275705+0000 2024-01-24T23:23:01.275 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:01.275 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:01.275 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:01.275 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:01.275 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:01.275 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:01.275 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:01.275 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:01.275 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:01.275 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:01.275 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:01.275 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:01.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.268+0000 7f5862770640 1 Processor -- start 2024-01-24T23:23:01.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.268+0000 7f5862770640 1 -- start start 2024-01-24T23:23:01.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.268+0000 7f5862770640 10 monclient: init 2024-01-24T23:23:01.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:01.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 10 monclient: _add_conns ranks=[3,2,8,5,0,7,1,4,6] 2024-01-24T23:23:01.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 10 monclient(hunting): picked mon.noname-g con 0x7f585c071480 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:01.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 10 monclient(hunting): picked mon.noname-b con 0x7f585c071860 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:01.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 10 monclient(hunting): picked mon.noname-e con 0x7f585c071c40 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:01.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f585c064700 con 0x7f585c071860 2024-01-24T23:23:01.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f585c133c60 con 0x7f585c071480 2024-01-24T23:23:01.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f585c06d6f0 con 0x7f585c071c40 2024-01-24T23:23:01.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:01.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 10 monclient(hunting): authenticate will time out at 1068.600652s 2024-01-24T23:23:01.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585b7fe640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f585c071860 legacy=0x7f585c13bdc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:23:01.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5860d0d640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f585c071c40 legacy=0x7f585c13f4f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:23:01.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585bfff640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f585c071480 0x7f585c133550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:59948/0 (socket says 172.21.15.132:59948) 2024-01-24T23:23:01.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585bfff640 1 -- 172.21.15.132:0/140463885 learned_addr learned my addr 172.21.15.132:0/140463885 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:01.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 1 -- 172.21.15.132:0/140463885 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4173027608 0 0) 0x7f585c133c60 con 0x7f585c071480 2024-01-24T23:23:01.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:01.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:01.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:01.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 10 monclient(hunting): my global_id is 4791 2024-01-24T23:23:01.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 1 -- 172.21.15.132:0/140463885 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5848003530 con 0x7f585c071480 2024-01-24T23:23:01.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 1 -- 172.21.15.132:0/140463885 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3763526771 0 0) 0x7f5848003530 con 0x7f585c071480 2024-01-24T23:23:01.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 1 -- 172.21.15.132:0/140463885 >> v1:172.21.15.115:6793/0 conn(0x7f585c071c40 legacy=0x7f585c13f4f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:01.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 1 -- 172.21.15.132:0/140463885 >> v1:172.21.15.115:6790/0 conn(0x7f585c071860 legacy=0x7f585c13bdc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:01.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:01.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 1 monclient: found mon.noname-g 2024-01-24T23:23:01.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:01.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:23:01.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 1 -- 172.21.15.132:0/140463885 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585c09ee50 con 0x7f585c071480 2024-01-24T23:23:01.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 10 monclient: _finish_auth 0 2024-01-24T23:23:01.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:01.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:01.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 1 -- 172.21.15.132:0/140463885 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5854001b40 con 0x7f585c071480 2024-01-24T23:23:01.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:01.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:01.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 10 monclient: dump: 2024-01-24T23:23:01.285 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:01.285 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:01.285 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:01.285 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:01.285 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:01.286 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:01.286 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:01.286 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:01.286 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:01.286 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:01.286 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:01.286 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:01.286 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:01.286 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:01.286 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:01.286 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:01.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 5 monclient: authenticate success, global_id 4791 2024-01-24T23:23:01.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:01.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 1 -- 172.21.15.132:0/140463885 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5854001e60 con 0x7f585c071480 2024-01-24T23:23:01.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:01.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 1 -- 172.21.15.132:0/140463885 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5854004f70 con 0x7f585c071480 2024-01-24T23:23:01.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:01.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 4 set_mon_vals no callback set 2024-01-24T23:23:01.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:01.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:01.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f585affd640 10 monclient: dump: 2024-01-24T23:23:01.287 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:01.287 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:01.287 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:01.287 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:01.287 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:01.288 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:01.288 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:01.288 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:01.288 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:01.288 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:01.288 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:01.288 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:01.288 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:01.288 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:01.288 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:01.288 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:01.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 10 monclient: shutdown 2024-01-24T23:23:01.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:01.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 1 -- 172.21.15.132:0/140463885 >> v1:172.21.15.132:6790/0 conn(0x7f585c071480 legacy=0x7f585c133550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:01.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 1 -- 172.21.15.132:0/140463885 shutdown_connections 2024-01-24T23:23:01.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 1 -- 172.21.15.132:0/140463885 >> 172.21.15.132:0/140463885 conn(0x7f585c0a2340 msgr2=0x7f585c0a2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:01.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 1 -- 172.21.15.132:0/140463885 shutdown_connections 2024-01-24T23:23:01.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 1 -- 172.21.15.132:0/140463885 wait complete. 2024-01-24T23:23:01.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 10 monclient: build_initial_monmap 2024-01-24T23:23:01.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 10 monclient: monmap: 2024-01-24T23:23:01.289 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:01.289 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:01.289 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:01.289 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:01.289 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:01.289 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:01.289 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:01.289 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:01.290 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:01.290 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:01.290 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:01.290 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:01.290 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:01.290 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:01.290 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:01.290 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:01.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 1 Processor -- start 2024-01-24T23:23:01.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 1 -- start start 2024-01-24T23:23:01.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 10 monclient: init 2024-01-24T23:23:01.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:01.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 10 monclient: _add_conns ranks=[7,8,1,3,5,6,2,4,0] 2024-01-24T23:23:01.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 10 monclient(hunting): picked mon.noname-i con 0x7f585c071480 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:01.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.272+0000 7f5862770640 10 monclient(hunting): picked mon.noname-e con 0x7f585c071860 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:01.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5862770640 10 monclient(hunting): picked mon.noname-f con 0x7f585c071c40 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:01.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5862770640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f585c137010 con 0x7f585c071c40 2024-01-24T23:23:01.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5862770640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f585c131090 con 0x7f585c071480 2024-01-24T23:23:01.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5862770640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f585c1cc1d0 con 0x7f585c071860 2024-01-24T23:23:01.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5862770640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:01.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f585b7fe640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f585c071860 legacy=0x7f585c1d0600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:23:01.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5862770640 10 monclient(hunting): authenticate will time out at 1068.604142s 2024-01-24T23:23:01.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f585bfff640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f585c071480 0x7f585c130560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:38950/0 (socket says 172.21.15.132:38950) 2024-01-24T23:23:01.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f585bfff640 1 -- 172.21.15.132:0/3229206786 learned_addr learned my addr 172.21.15.132:0/3229206786 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:01.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 1 -- 172.21.15.132:0/3229206786 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1514831249 0 0) 0x7f585c131090 con 0x7f585c071480 2024-01-24T23:23:01.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:01.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:01.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:01.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 10 monclient(hunting): my global_id is 4903 2024-01-24T23:23:01.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 1 -- 172.21.15.132:0/3229206786 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f583c003530 con 0x7f585c071480 2024-01-24T23:23:01.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 1 -- 172.21.15.132:0/3229206786 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 423789406 0 0) 0x7f585c137010 con 0x7f585c071c40 2024-01-24T23:23:01.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:01.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:01.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:01.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 10 monclient(hunting): my global_id is 4888 2024-01-24T23:23:01.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 1 -- 172.21.15.132:0/3229206786 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f585c131090 con 0x7f585c071c40 2024-01-24T23:23:01.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 1 -- 172.21.15.132:0/3229206786 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3425140257 0 0) 0x7f583c003530 con 0x7f585c071480 2024-01-24T23:23:01.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 1 -- 172.21.15.132:0/3229206786 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f585c137010 con 0x7f585c071480 2024-01-24T23:23:01.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 1 -- 172.21.15.132:0/3229206786 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5854001b00 con 0x7f585c071480 2024-01-24T23:23:01.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:01.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:23:01.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 10 monclient(hunting): dump: 2024-01-24T23:23:01.293 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:01.293 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:01.293 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:01.293 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:01.293 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:01.293 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:01.293 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:01.293 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:01.293 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:01.293 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:01.293 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:01.293 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:01.293 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:01.293 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:01.294 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:01.294 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:01.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:01.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 1 -- 172.21.15.132:0/3229206786 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4065955980 0 0) 0x7f585c131090 con 0x7f585c071c40 2024-01-24T23:23:01.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 1 -- 172.21.15.132:0/3229206786 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f583c003530 con 0x7f585c071c40 2024-01-24T23:23:01.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 1 -- 172.21.15.132:0/3229206786 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5844004150 con 0x7f585c071c40 2024-01-24T23:23:01.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:01.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:23:01.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 10 monclient(hunting): dump: 2024-01-24T23:23:01.294 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:01.294 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:01.294 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:01.294 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:01.294 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:01.294 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:01.294 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:01.295 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:01.295 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:01.295 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:01.295 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:01.295 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:01.295 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:01.295 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:01.295 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:01.295 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:01.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 1 -- 172.21.15.132:0/3229206786 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2227275495 0 0) 0x7f585c137010 con 0x7f585c071480 2024-01-24T23:23:01.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 1 -- 172.21.15.132:0/3229206786 >> v1:172.21.15.115:6793/0 conn(0x7f585c071860 legacy=0x7f585c1d0600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:01.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 1 -- 172.21.15.132:0/3229206786 >> v1:172.21.15.132:6789/0 conn(0x7f585c071c40 legacy=0x7f585c1d3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:01.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:23:01.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 1 monclient: found mon.i 2024-01-24T23:23:01.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:01.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:01.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 1 -- 172.21.15.132:0/3229206786 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585c09f9b0 con 0x7f585c071480 2024-01-24T23:23:01.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 10 monclient: _finish_auth 0 2024-01-24T23:23:01.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:01.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:01.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5862770640 5 monclient: authenticate success, global_id 4903 2024-01-24T23:23:01.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5862770640 10 monclient: _renew_subs 2024-01-24T23:23:01.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5862770640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:01.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5862770640 1 -- 172.21.15.132:0/3229206786 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f585c137750 con 0x7f585c071480 2024-01-24T23:23:01.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5862770640 10 monclient: _renew_subs 2024-01-24T23:23:01.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5862770640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:01.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5862770640 1 -- 172.21.15.132:0/3229206786 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f585c11d0c0 con 0x7f585c071480 2024-01-24T23:23:01.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 1 -- 172.21.15.132:0/3229206786 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5854001fd0 con 0x7f585c071480 2024-01-24T23:23:01.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:01.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 1 -- 172.21.15.132:0/3229206786 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5854005830 con 0x7f585c071480 2024-01-24T23:23:01.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:01.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f585a7fc640 4 set_mon_vals no callback set 2024-01-24T23:23:01.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:01.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 10 monclient: dump: 2024-01-24T23:23:01.297 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:01.297 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:01.297 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:01.297 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:01.297 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:01.297 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:01.297 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:01.297 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:01.297 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:01.297 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:01.297 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:01.298 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:01.298 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:01.298 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:01.298 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:01.298 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:01.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5862770640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:01.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5862770640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:01.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5862770640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:01.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 1 -- 172.21.15.132:0/3229206786 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f58540102b0 con 0x7f585c071480 2024-01-24T23:23:01.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5862770640 1 -- 172.21.15.132:0/3229206786 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5840005090 con 0x7f585c071480 2024-01-24T23:23:01.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f5858ff9640 1 -- 172.21.15.132:0/3229206786 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(40..40 src has 1..40) v4 ==== 3716+0+0 (unknown 3873156730 0 0) 0x7f5854043d90 con 0x7f585c071480 2024-01-24T23:23:01.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.276+0000 7f585b7fe640 10 monclient: get_auth_request con 0x7f583c0376a0 auth_method 0 2024-01-24T23:23:01.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.280+0000 7f5858ff9640 1 -- 172.21.15.132:0/3229206786 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f5854015000 con 0x7f585c071480 2024-01-24T23:23:01.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.280+0000 7f5858ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:01.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.280+0000 7f5858ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:01.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.428+0000 7f5862770640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["all"]}] 2024-01-24T23:23:01.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.428+0000 7f5862770640 10 monclient: _send_command 2 [{"prefix": "osd crush rm-device-class", "ids": ["all"]}] 2024-01-24T23:23:01.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.428+0000 7f5862770640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:01.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:01.428+0000 7f5862770640 1 -- 172.21.15.132:0/3229206786 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["all"]} v 0) v1 -- 0x7f5840005340 con 0x7f585c071480 2024-01-24T23:23:02.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:02.044+0000 7f5858ff9640 1 -- 172.21.15.132:0/3229206786 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["all"]}]=0 done removing class of osd(s): 0,1,2,3,4,5 v41) v1 ==== 131+0+0 (unknown 961951500 0 0) 0x7f58540136a0 con 0x7f585c071480 2024-01-24T23:23:02.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:02.044+0000 7f5858ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm-device-class", "ids": ["all"]}] 2024-01-24T23:23:02.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:02.044+0000 7f5858ff9640 10 monclient: _finish_command 2 = system:0 done removing class of osd(s): 0,1,2,3,4,5 2024-01-24T23:23:02.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:02.144+0000 7f5862770640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["all"]}] 2024-01-24T23:23:02.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:02.144+0000 7f5862770640 10 monclient: _send_command 3 [{"prefix": "osd crush rm-device-class", "ids": ["all"]}] 2024-01-24T23:23:02.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:02.144+0000 7f5862770640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:02.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:02.144+0000 7f5862770640 1 -- 172.21.15.132:0/3229206786 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["all"]} v 0) v1 -- 0x7f5840002220 con 0x7f585c071480 2024-01-24T23:23:03.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.060+0000 7f5858ff9640 1 -- 172.21.15.132:0/3229206786 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["all"]}]=0 osd.0 belongs to no class, osd.1 belongs to no class, osd.2 belongs to no class, osd.3 belongs to no class, osd.4 belongs to no class, osd.5 belongs to no class, done removing class of osd(s): v42) v1 ==== 282+0+0 (unknown 2903863430 0 0) 0x7f5854014e20 con 0x7f585c071480 2024-01-24T23:23:03.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.060+0000 7f5858ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm-device-class", "ids": ["all"]}] 2024-01-24T23:23:03.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.060+0000 7f5858ff9640 10 monclient: _finish_command 3 = system:0 osd.0 belongs to no class, osd.1 belongs to no class, osd.2 belongs to no class, osd.3 belongs to no class, osd.4 belongs to no class, osd.5 belongs to no class, done removing class of osd(s): 2024-01-24T23:23:03.066 INFO:tasks.workunit.client.0.smithi132.stderr:osd.0 belongs to no class, osd.1 belongs to no class, osd.2 belongs to no class, osd.3 belongs to no class, osd.4 belongs to no class, osd.5 belongs to no class, done removing class of osd(s): 2024-01-24T23:23:03.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.064+0000 7f5862770640 1 -- 172.21.15.132:0/3229206786 >> v1:172.21.15.132:6812/1835401922 conn(0x7f583c0376a0 legacy=0x7f583c039b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:03.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.064+0000 7f5862770640 10 monclient: shutdown 2024-01-24T23:23:03.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.064+0000 7f5862770640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:03.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.064+0000 7f5862770640 1 -- 172.21.15.132:0/3229206786 >> v1:172.21.15.132:6792/0 conn(0x7f585c071480 legacy=0x7f585c130560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:03.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.064+0000 7f5862770640 1 -- 172.21.15.132:0/3229206786 shutdown_connections 2024-01-24T23:23:03.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.064+0000 7f5862770640 1 -- 172.21.15.132:0/3229206786 >> 172.21.15.132:0/3229206786 conn(0x7f585c0a2340 msgr2=0x7f585c1be870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:03.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.064+0000 7f5862770640 1 -- 172.21.15.132:0/3229206786 shutdown_connections 2024-01-24T23:23:03.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.064+0000 7f5862770640 1 -- 172.21.15.132:0/3229206786 wait complete. 2024-01-24T23:23:03.086 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush set-device-class ssd osd.0 2024-01-24T23:23:03.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.200+0000 7f3035686640 10 monclient: get_monmap_and_config 2024-01-24T23:23:03.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.200+0000 7f3035686640 10 monclient: build_initial_monmap 2024-01-24T23:23:03.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.200+0000 7f3035686640 10 monclient: monmap: 2024-01-24T23:23:03.206 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:03.206 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:03.206 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:03.207324+0000 2024-01-24T23:23:03.206 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:03.207324+0000 2024-01-24T23:23:03.206 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:03.206 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:03.206 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:03.206 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:03.206 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:03.206 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:03.207 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:03.207 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:03.207 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:03.207 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:03.207 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:03.207 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:03.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.200+0000 7f3035686640 1 Processor -- start 2024-01-24T23:23:03.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.200+0000 7f3035686640 1 -- start start 2024-01-24T23:23:03.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.200+0000 7f3035686640 10 monclient: init 2024-01-24T23:23:03.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:03.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 10 monclient: _add_conns ranks=[7,1,6,3,4,8,5,2,0] 2024-01-24T23:23:03.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 10 monclient(hunting): picked mon.noname-i con 0x7f3030133b50 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:03.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 10 monclient(hunting): picked mon.noname-f con 0x7f303009ec20 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:03.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 10 monclient(hunting): picked mon.noname-d con 0x7f303013a1d0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:03.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3030064700 con 0x7f303009ec20 2024-01-24T23:23:03.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f303009d9f0 con 0x7f303013a1d0 2024-01-24T23:23:03.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f303009dc60 con 0x7f3030133b50 2024-01-24T23:23:03.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:03.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 10 monclient(hunting): authenticate will time out at 1070.532534s 2024-01-24T23:23:03.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302effd640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f3030133b50 0x7f3030133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:38960/0 (socket says 172.21.15.132:38960) 2024-01-24T23:23:03.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302effd640 1 -- 172.21.15.132:0/3935923461 learned_addr learned my addr 172.21.15.132:0/3935923461 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:03.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 1 -- 172.21.15.132:0/3935923461 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 866163058 0 0) 0x7f303009dc60 con 0x7f3030133b50 2024-01-24T23:23:03.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:03.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:03.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:03.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 10 monclient(hunting): my global_id is 4912 2024-01-24T23:23:03.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 1 -- 172.21.15.132:0/3935923461 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3018003530 con 0x7f3030133b50 2024-01-24T23:23:03.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 1 -- 172.21.15.132:0/3935923461 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3200259094 0 0) 0x7f3030064700 con 0x7f303009ec20 2024-01-24T23:23:03.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:03.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:03.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:03.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 10 monclient(hunting): my global_id is 4897 2024-01-24T23:23:03.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 1 -- 172.21.15.132:0/3935923461 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f303009dc60 con 0x7f303009ec20 2024-01-24T23:23:03.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 1 -- 172.21.15.132:0/3935923461 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 875621888 0 0) 0x7f3018003530 con 0x7f3030133b50 2024-01-24T23:23:03.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 1 -- 172.21.15.132:0/3935923461 >> v1:172.21.15.115:6792/0 conn(0x7f303013a1d0 legacy=0x7f303013c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:03.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 1 -- 172.21.15.132:0/3935923461 >> v1:172.21.15.132:6789/0 conn(0x7f303009ec20 legacy=0x7f3030138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:03.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 10 monclient: _finish_hunting 0 2024-01-24T23:23:03.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 1 monclient: found mon.noname-i 2024-01-24T23:23:03.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:03.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:23:03.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 1 -- 172.21.15.132:0/3935923461 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303009df40 con 0x7f3030133b50 2024-01-24T23:23:03.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 10 monclient: _finish_auth 0 2024-01-24T23:23:03.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 10 monclient: _check_auth_tickets 2024-01-24T23:23:03.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:03.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 1 -- 172.21.15.132:0/3935923461 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f30200017c0 con 0x7f3030133b50 2024-01-24T23:23:03.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 5 monclient: authenticate success, global_id 4912 2024-01-24T23:23:03.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:03.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:03.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:23:03.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 10 monclient: dump: 2024-01-24T23:23:03.213 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:03.213 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:03.213 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:03.213 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:03.213 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:03.213 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:03.213 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:03.213 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:03.213 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:03.213 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:03.213 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:03.213 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:03.214 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:03.214 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:03.214 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:03.214 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:03.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:03.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 1 -- 172.21.15.132:0/3935923461 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f30200019e0 con 0x7f3030133b50 2024-01-24T23:23:03.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:03.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:03.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 1 -- 172.21.15.132:0/3935923461 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3020004bf0 con 0x7f3030133b50 2024-01-24T23:23:03.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 4 set_mon_vals no callback set 2024-01-24T23:23:03.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:03.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:03.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302dffb640 10 monclient: dump: 2024-01-24T23:23:03.214 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:03.214 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:03.214 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:03.214 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:03.215 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:03.215 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:03.215 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:03.215 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:03.215 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:03.215 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:03.215 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:03.215 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:03.215 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:03.215 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:03.215 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:03.215 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:03.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 10 monclient: shutdown 2024-01-24T23:23:03.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:03.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 1 -- 172.21.15.132:0/3935923461 >> v1:172.21.15.132:6792/0 conn(0x7f3030133b50 legacy=0x7f3030133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:03.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 1 -- 172.21.15.132:0/3935923461 shutdown_connections 2024-01-24T23:23:03.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 1 -- 172.21.15.132:0/3935923461 >> 172.21.15.132:0/3935923461 conn(0x7f30300a5a10 msgr2=0x7f30300a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:03.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 1 -- 172.21.15.132:0/3935923461 shutdown_connections 2024-01-24T23:23:03.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 1 -- 172.21.15.132:0/3935923461 wait complete. 2024-01-24T23:23:03.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 10 monclient: build_initial_monmap 2024-01-24T23:23:03.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 10 monclient: monmap: 2024-01-24T23:23:03.216 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:03.216 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:03.216 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:03.216 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:03.216 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:03.216 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:03.216 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:03.216 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:03.216 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:03.216 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:03.216 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:03.217 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:03.217 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:03.217 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:03.217 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:03.217 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:03.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 1 Processor -- start 2024-01-24T23:23:03.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 1 -- start start 2024-01-24T23:23:03.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 10 monclient: init 2024-01-24T23:23:03.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:03.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 10 monclient: _add_conns ranks=[2,3,5,4,6,7,8,0,1] 2024-01-24T23:23:03.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 10 monclient(hunting): picked mon.noname-b con 0x7f303009ec20 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:03.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 10 monclient(hunting): picked mon.noname-g con 0x7f3030133b50 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:03.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 10 monclient(hunting): picked mon.noname-h con 0x7f303013a1d0 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:03.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3030070020 con 0x7f303009ec20 2024-01-24T23:23:03.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30300a13b0 con 0x7f3030133b50 2024-01-24T23:23:03.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30300714d0 con 0x7f303013a1d0 2024-01-24T23:23:03.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:03.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f3035686640 10 monclient(hunting): authenticate will time out at 1070.535568s 2024-01-24T23:23:03.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302e7fc640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f3030133b50 0x7f30301d0840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:59956/0 (socket says 172.21.15.132:59956) 2024-01-24T23:23:03.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302effd640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f303009ec20 legacy=0x7f303012c3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:23:03.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302e7fc640 1 -- 172.21.15.132:0/1228869082 learned_addr learned my addr 172.21.15.132:0/1228869082 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:03.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f302f7fe640 1 --1- 172.21.15.132:0/1228869082 >> v1:172.21.15.132:6791/0 conn(0x7f303013a1d0 0x7f30301d3f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:39368/0 (socket says 172.21.15.132:39368) 2024-01-24T23:23:03.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f30177fe640 1 -- 172.21.15.132:0/1228869082 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875606829 0 0) 0x7f30300714d0 con 0x7f303013a1d0 2024-01-24T23:23:03.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f30177fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:03.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f30177fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:03.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f30177fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:03.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f30177fe640 10 monclient(hunting): my global_id is 4856 2024-01-24T23:23:03.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f30177fe640 1 -- 172.21.15.132:0/1228869082 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f301c003560 con 0x7f303013a1d0 2024-01-24T23:23:03.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f30177fe640 1 -- 172.21.15.132:0/1228869082 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3780416508 0 0) 0x7f30300a13b0 con 0x7f3030133b50 2024-01-24T23:23:03.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f30177fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:03.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f30177fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:03.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f30177fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:03.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.204+0000 7f30177fe640 10 monclient(hunting): my global_id is 4800 2024-01-24T23:23:03.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 1 -- 172.21.15.132:0/1228869082 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30300714d0 con 0x7f3030133b50 2024-01-24T23:23:03.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 1 -- 172.21.15.132:0/1228869082 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1953631147 0 0) 0x7f301c003560 con 0x7f303013a1d0 2024-01-24T23:23:03.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 1 -- 172.21.15.132:0/1228869082 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f30300a13b0 con 0x7f303013a1d0 2024-01-24T23:23:03.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 1 -- 172.21.15.132:0/1228869082 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3028001cc0 con 0x7f303013a1d0 2024-01-24T23:23:03.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:03.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:03.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 10 monclient(hunting): dump: 2024-01-24T23:23:03.222 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:03.222 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:03.222 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:03.223 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:03.223 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:03.223 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:03.223 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:03.223 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:03.223 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:03.223 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:03.223 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:03.223 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:03.223 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:03.223 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:03.223 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:03.223 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:03.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:03.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 1 -- 172.21.15.132:0/1228869082 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1694441348 0 0) 0x7f30300714d0 con 0x7f3030133b50 2024-01-24T23:23:03.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 1 -- 172.21.15.132:0/1228869082 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f301c003560 con 0x7f3030133b50 2024-01-24T23:23:03.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 1 -- 172.21.15.132:0/1228869082 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f30240015c0 con 0x7f3030133b50 2024-01-24T23:23:03.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:03.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:03.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 10 monclient(hunting): dump: 2024-01-24T23:23:03.224 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:03.224 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:03.224 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:03.224 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:03.224 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:03.224 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:03.224 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:03.224 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:03.224 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:03.224 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:03.224 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:03.224 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:03.224 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:03.225 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:03.225 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:03.225 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:03.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 1 -- 172.21.15.132:0/1228869082 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1954176525 0 0) 0x7f30300a13b0 con 0x7f303013a1d0 2024-01-24T23:23:03.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 1 -- 172.21.15.132:0/1228869082 >> v1:172.21.15.132:6790/0 conn(0x7f3030133b50 legacy=0x7f30301d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:03.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 1 -- 172.21.15.132:0/1228869082 >> v1:172.21.15.115:6790/0 conn(0x7f303009ec20 legacy=0x7f303012c3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:03.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 10 monclient: _finish_hunting 0 2024-01-24T23:23:03.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 1 monclient: found mon.h 2024-01-24T23:23:03.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:03.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:03.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 1 -- 172.21.15.132:0/1228869082 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30300717b0 con 0x7f303013a1d0 2024-01-24T23:23:03.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 10 monclient: _finish_auth 0 2024-01-24T23:23:03.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 10 monclient: _check_auth_tickets 2024-01-24T23:23:03.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:03.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f3035686640 5 monclient: authenticate success, global_id 4856 2024-01-24T23:23:03.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f3035686640 10 monclient: _renew_subs 2024-01-24T23:23:03.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f3035686640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:03.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f3035686640 1 -- 172.21.15.132:0/1228869082 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30300711f0 con 0x7f303013a1d0 2024-01-24T23:23:03.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f3035686640 10 monclient: _renew_subs 2024-01-24T23:23:03.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f3035686640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:03.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f3035686640 1 -- 172.21.15.132:0/1228869082 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3030071bd0 con 0x7f303013a1d0 2024-01-24T23:23:03.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f3035686640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:03.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f3035686640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:03.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f3035686640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:03.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f3035686640 1 -- 172.21.15.132:0/1228869082 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3030071050 con 0x7f303013a1d0 2024-01-24T23:23:03.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 1 -- 172.21.15.132:0/1228869082 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3028002050 con 0x7f303013a1d0 2024-01-24T23:23:03.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:03.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 1 -- 172.21.15.132:0/1228869082 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3028003f90 con 0x7f303013a1d0 2024-01-24T23:23:03.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:03.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:03.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 10 monclient: dump: 2024-01-24T23:23:03.226 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:03.227 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:03.227 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:03.227 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:03.227 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:03.227 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:03.227 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:03.227 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:03.227 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:03.227 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:03.227 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:03.227 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:03.227 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:03.227 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:03.227 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:03.227 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:03.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f302d7fa640 4 set_mon_vals no callback set 2024-01-24T23:23:03.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 1 -- 172.21.15.132:0/1228869082 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f30280040f0 con 0x7f303013a1d0 2024-01-24T23:23:03.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f30177fe640 1 -- 172.21.15.132:0/1228869082 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(42..42 src has 1..42) v4 ==== 3412+0+0 (unknown 4006375450 0 0) 0x7f3028043630 con 0x7f303013a1d0 2024-01-24T23:23:03.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.208+0000 7f302effd640 10 monclient: get_auth_request con 0x7f301c037740 auth_method 0 2024-01-24T23:23:03.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.212+0000 7f30177fe640 1 -- 172.21.15.132:0/1228869082 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f3028014a60 con 0x7f303013a1d0 2024-01-24T23:23:03.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.212+0000 7f30177fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:03.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.212+0000 7f30177fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:03.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.368+0000 7f3035686640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}] 2024-01-24T23:23:03.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.368+0000 7f3035686640 10 monclient: _send_command 2 [{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}] 2024-01-24T23:23:03.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.368+0000 7f3035686640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:03.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:03.368+0000 7f3035686640 1 -- 172.21.15.132:0/1228869082 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]} v 0) v1 -- 0x7f303009f060 con 0x7f303013a1d0 2024-01-24T23:23:04.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:04.072+0000 7f30177fe640 1 -- 172.21.15.132:0/1228869082 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}]=0 set osd(s) 0 to class 'ssd' v43) v1 ==== 135+0+0 (unknown 2565673951 0 0) 0x7f303009f060 con 0x7f303013a1d0 2024-01-24T23:23:04.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:04.072+0000 7f30177fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}] 2024-01-24T23:23:04.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:04.072+0000 7f30177fe640 10 monclient: _finish_command 2 = system:0 set osd(s) 0 to class 'ssd' 2024-01-24T23:23:04.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:04.172+0000 7f3035686640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}] 2024-01-24T23:23:04.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:04.172+0000 7f3035686640 10 monclient: _send_command 3 [{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}] 2024-01-24T23:23:04.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:04.172+0000 7f3035686640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:04.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:04.172+0000 7f3035686640 1 -- 172.21.15.132:0/1228869082 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]} v 0) v1 -- 0x7f3030071050 con 0x7f303013a1d0 2024-01-24T23:23:05.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.080+0000 7f30177fe640 1 -- 172.21.15.132:0/1228869082 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}]=0 osd.0 already set to class ssd. set-device-class item id 0 name 'osd.0' device_class 'ssd': no change. set osd(s) to class 'ssd' v44) v1 ==== 237+0+0 (unknown 2761511203 0 0) 0x7f3030071050 con 0x7f303013a1d0 2024-01-24T23:23:05.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.080+0000 7f30177fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}] 2024-01-24T23:23:05.085 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.080+0000 7f30177fe640 10 monclient: _finish_command 3 = system:0 osd.0 already set to class ssd. set-device-class item id 0 name 'osd.0' device_class 'ssd': no change. set osd(s) to class 'ssd' 2024-01-24T23:23:05.085 INFO:tasks.workunit.client.0.smithi132.stderr:osd.0 already set to class ssd. set-device-class item id 0 name 'osd.0' device_class 'ssd': no change. set osd(s) to class 'ssd' 2024-01-24T23:23:05.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.084+0000 7f3035686640 1 -- 172.21.15.132:0/1228869082 >> v1:172.21.15.132:6812/1835401922 conn(0x7f301c037740 legacy=0x7f301c039c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:05.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.084+0000 7f3035686640 10 monclient: shutdown 2024-01-24T23:23:05.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.084+0000 7f3035686640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:05.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.084+0000 7f3035686640 1 -- 172.21.15.132:0/1228869082 >> v1:172.21.15.132:6791/0 conn(0x7f303013a1d0 legacy=0x7f30301d3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:05.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.084+0000 7f3035686640 1 -- 172.21.15.132:0/1228869082 shutdown_connections 2024-01-24T23:23:05.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.084+0000 7f3035686640 1 -- 172.21.15.132:0/1228869082 >> 172.21.15.132:0/1228869082 conn(0x7f30300a5a10 msgr2=0x7f303009c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:05.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.084+0000 7f3035686640 1 -- 172.21.15.132:0/1228869082 shutdown_connections 2024-01-24T23:23:05.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.084+0000 7f3035686640 1 -- 172.21.15.132:0/1228869082 wait complete. 2024-01-24T23:23:05.106 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush set-device-class hdd osd.1 2024-01-24T23:23:05.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 10 monclient: get_monmap_and_config 2024-01-24T23:23:05.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 10 monclient: build_initial_monmap 2024-01-24T23:23:05.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 10 monclient: monmap: 2024-01-24T23:23:05.226 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:05.226 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:05.226 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:05.227394+0000 2024-01-24T23:23:05.226 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:05.227394+0000 2024-01-24T23:23:05.226 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:05.226 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:05.226 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:05.226 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:05.227 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:05.227 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:05.227 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:05.227 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:05.227 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:05.227 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:05.227 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:05.227 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:05.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 1 Processor -- start 2024-01-24T23:23:05.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 1 -- start start 2024-01-24T23:23:05.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 10 monclient: init 2024-01-24T23:23:05.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:05.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 10 monclient: _add_conns ranks=[8,1,4,2,7,5,6,0,3] 2024-01-24T23:23:05.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 10 monclient(hunting): picked mon.noname-e con 0x7fac54136cb0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:05.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 10 monclient(hunting): picked mon.noname-f con 0x7fac541377a0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:05.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 10 monclient(hunting): picked mon.noname-c con 0x7fac5409bcc0 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:05.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac54064700 con 0x7fac541377a0 2024-01-24T23:23:05.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac5409b650 con 0x7fac5409bcc0 2024-01-24T23:23:05.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac5409c110 con 0x7fac54136cb0 2024-01-24T23:23:05.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:05.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 10 monclient(hunting): authenticate will time out at 1072.552440s 2024-01-24T23:23:05.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac53fff640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7fac54136cb0 legacy=0x7fac54137090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:23:05.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac537fe640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fac541377a0 0x7fac5413c1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:48272/0 (socket says 172.21.15.132:48272) 2024-01-24T23:23:05.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac537fe640 1 -- 172.21.15.132:0/2206665665 learned_addr learned my addr 172.21.15.132:0/2206665665 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:05.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 1 -- 172.21.15.132:0/2206665665 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1050364522 0 0) 0x7fac54064700 con 0x7fac541377a0 2024-01-24T23:23:05.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:05.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:05.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:05.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 10 monclient(hunting): my global_id is 4906 2024-01-24T23:23:05.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 1 -- 172.21.15.132:0/2206665665 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac3c003530 con 0x7fac541377a0 2024-01-24T23:23:05.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 1 -- 172.21.15.132:0/2206665665 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2563957954 0 0) 0x7fac3c003530 con 0x7fac541377a0 2024-01-24T23:23:05.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 1 -- 172.21.15.132:0/2206665665 >> v1:172.21.15.115:6793/0 conn(0x7fac54136cb0 legacy=0x7fac54137090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:05.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 1 -- 172.21.15.132:0/2206665665 >> v1:172.21.15.115:6791/0 conn(0x7fac5409bcc0 legacy=0x7fac5413f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:05.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:05.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 1 monclient: found mon.noname-f 2024-01-24T23:23:05.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:05.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:23:05.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 1 -- 172.21.15.132:0/2206665665 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac5409b000 con 0x7fac541377a0 2024-01-24T23:23:05.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 10 monclient: _finish_auth 0 2024-01-24T23:23:05.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:05.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:05.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 1 -- 172.21.15.132:0/2206665665 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fac480017e0 con 0x7fac541377a0 2024-01-24T23:23:05.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 5 monclient: authenticate success, global_id 4906 2024-01-24T23:23:05.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:05.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:05.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:23:05.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 10 monclient: dump: 2024-01-24T23:23:05.236 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:05.236 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:05.236 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:05.236 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:05.236 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:05.236 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:05.236 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:05.237 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:05.237 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:05.237 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:05.237 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:05.237 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:05.237 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:05.237 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:05.237 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:05.237 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:05.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:05.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 1 -- 172.21.15.132:0/2206665665 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fac48001b00 con 0x7fac541377a0 2024-01-24T23:23:05.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:05.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 1 -- 172.21.15.132:0/2206665665 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fac48004bf0 con 0x7fac541377a0 2024-01-24T23:23:05.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:05.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 4 set_mon_vals no callback set 2024-01-24T23:23:05.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:05.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:23:05.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac52ffd640 10 monclient: dump: 2024-01-24T23:23:05.238 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:05.238 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:05.238 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:05.238 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:05.238 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:05.238 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:05.238 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:05.238 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:05.238 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:05.238 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:05.238 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:05.238 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:05.239 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:05.239 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:05.239 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:05.239 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:05.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 10 monclient: shutdown 2024-01-24T23:23:05.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:05.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 1 -- 172.21.15.132:0/2206665665 >> v1:172.21.15.132:6789/0 conn(0x7fac541377a0 legacy=0x7fac5413c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:05.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 1 -- 172.21.15.132:0/2206665665 shutdown_connections 2024-01-24T23:23:05.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 1 -- 172.21.15.132:0/2206665665 >> 172.21.15.132:0/2206665665 conn(0x7fac540a9c10 msgr2=0x7fac540a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:05.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 1 -- 172.21.15.132:0/2206665665 shutdown_connections 2024-01-24T23:23:05.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 1 -- 172.21.15.132:0/2206665665 wait complete. 2024-01-24T23:23:05.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 10 monclient: build_initial_monmap 2024-01-24T23:23:05.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 10 monclient: monmap: 2024-01-24T23:23:05.239 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:05.239 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:05.239 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:05.239 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:05.240 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:05.240 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:05.240 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:05.240 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:05.240 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:05.240 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:05.240 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:05.240 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:05.240 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:05.240 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:05.240 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:05.240 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:05.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 1 Processor -- start 2024-01-24T23:23:05.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 1 -- start start 2024-01-24T23:23:05.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 10 monclient: init 2024-01-24T23:23:05.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:05.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 10 monclient: _add_conns ranks=[5,4,1,7,3,2,8,0,6] 2024-01-24T23:23:05.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 10 monclient(hunting): picked mon.noname-h con 0x7fac5409bcc0 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:05.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 10 monclient(hunting): picked mon.noname-c con 0x7fac54136cb0 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:05.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 10 monclient(hunting): picked mon.noname-f con 0x7fac541377a0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:05.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac5409c6a0 con 0x7fac541377a0 2024-01-24T23:23:05.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac54120170 con 0x7fac54136cb0 2024-01-24T23:23:05.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac5409f530 con 0x7fac5409bcc0 2024-01-24T23:23:05.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:05.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac5a9a1640 10 monclient(hunting): authenticate will time out at 1072.555426s 2024-01-24T23:23:05.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac53fff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fac5409bcc0 0x7fac54130950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:35196/0 (socket says 172.21.15.132:35196) 2024-01-24T23:23:05.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac53fff640 1 -- 172.21.15.132:0/575164020 learned_addr learned my addr 172.21.15.132:0/575164020 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:05.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac50ff9640 1 -- 172.21.15.132:0/575164020 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3198948 0 0) 0x7fac5409c6a0 con 0x7fac541377a0 2024-01-24T23:23:05.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac50ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:05.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac50ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:05.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac50ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:05.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac50ff9640 10 monclient(hunting): my global_id is 4915 2024-01-24T23:23:05.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac50ff9640 1 -- 172.21.15.132:0/575164020 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac40003530 con 0x7fac541377a0 2024-01-24T23:23:05.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac50ff9640 1 -- 172.21.15.132:0/575164020 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432130942 0 0) 0x7fac5409f530 con 0x7fac5409bcc0 2024-01-24T23:23:05.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac50ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:05.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac50ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:05.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac50ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:05.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac50ff9640 10 monclient(hunting): my global_id is 4865 2024-01-24T23:23:05.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.224+0000 7fac50ff9640 1 -- 172.21.15.132:0/575164020 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac5409c6a0 con 0x7fac5409bcc0 2024-01-24T23:23:05.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 1 -- 172.21.15.132:0/575164020 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4108135408 0 0) 0x7fac5409c6a0 con 0x7fac5409bcc0 2024-01-24T23:23:05.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 1 -- 172.21.15.132:0/575164020 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fac5409f530 con 0x7fac5409bcc0 2024-01-24T23:23:05.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 1 -- 172.21.15.132:0/575164020 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fac44004380 con 0x7fac5409bcc0 2024-01-24T23:23:05.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:05.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:05.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 10 monclient(hunting): dump: 2024-01-24T23:23:05.242 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:05.242 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:05.242 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:05.242 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:05.243 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:05.243 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:05.243 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:05.243 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:05.243 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:05.243 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:05.243 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:05.243 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:05.243 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:05.243 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:05.243 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:05.243 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:05.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:05.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 1 -- 172.21.15.132:0/575164020 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1011263221 0 0) 0x7fac40003530 con 0x7fac541377a0 2024-01-24T23:23:05.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 1 -- 172.21.15.132:0/575164020 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fac5409c6a0 con 0x7fac541377a0 2024-01-24T23:23:05.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 1 -- 172.21.15.132:0/575164020 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fac4c001ce0 con 0x7fac541377a0 2024-01-24T23:23:05.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:05.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:23:05.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 10 monclient(hunting): dump: 2024-01-24T23:23:05.244 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:05.244 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:05.244 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:05.244 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:05.244 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:05.244 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:05.244 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:05.244 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:05.244 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:05.244 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:05.244 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:05.244 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:05.244 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:05.244 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:05.244 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:05.245 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:05.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 1 -- 172.21.15.132:0/575164020 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3572251726 0 0) 0x7fac54120170 con 0x7fac54136cb0 2024-01-24T23:23:05.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:05.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:05.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:05.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 10 monclient(hunting): my global_id is 74875 2024-01-24T23:23:05.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 1 -- 172.21.15.132:0/575164020 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac40003530 con 0x7fac54136cb0 2024-01-24T23:23:05.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 1 -- 172.21.15.132:0/575164020 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2332553014 0 0) 0x7fac5409f530 con 0x7fac5409bcc0 2024-01-24T23:23:05.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 1 -- 172.21.15.132:0/575164020 >> v1:172.21.15.115:6791/0 conn(0x7fac54136cb0 legacy=0x7fac541d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:05.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 1 -- 172.21.15.132:0/575164020 >> v1:172.21.15.132:6789/0 conn(0x7fac541377a0 legacy=0x7fac541d3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:05.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:23:05.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 1 monclient: found mon.h 2024-01-24T23:23:05.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:05.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:05.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 1 -- 172.21.15.132:0/575164020 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac54137fb0 con 0x7fac5409bcc0 2024-01-24T23:23:05.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 10 monclient: _finish_auth 0 2024-01-24T23:23:05.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:05.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:05.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac5a9a1640 5 monclient: authenticate success, global_id 4865 2024-01-24T23:23:05.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac5a9a1640 10 monclient: _renew_subs 2024-01-24T23:23:05.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac5a9a1640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:05.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac5a9a1640 1 -- 172.21.15.132:0/575164020 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac5411fe90 con 0x7fac5409bcc0 2024-01-24T23:23:05.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac5a9a1640 10 monclient: _renew_subs 2024-01-24T23:23:05.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac5a9a1640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:05.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac5a9a1640 1 -- 172.21.15.132:0/575164020 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fac541be670 con 0x7fac5409bcc0 2024-01-24T23:23:05.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 1 -- 172.21.15.132:0/575164020 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fac44004620 con 0x7fac5409bcc0 2024-01-24T23:23:05.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:05.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac527fc640 4 set_mon_vals no callback set 2024-01-24T23:23:05.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 1 -- 172.21.15.132:0/575164020 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fac44004b50 con 0x7fac5409bcc0 2024-01-24T23:23:05.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:05.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:05.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 10 monclient: dump: 2024-01-24T23:23:05.246 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:05.247 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:05.247 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:05.247 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:05.247 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:05.247 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:05.247 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:05.247 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:05.247 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:05.247 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:05.247 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:05.247 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:05.247 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:05.247 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:05.247 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:05.247 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:05.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac5a9a1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:05.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac5a9a1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:05.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac5a9a1640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:05.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac5a9a1640 1 -- 172.21.15.132:0/575164020 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac34005090 con 0x7fac5409bcc0 2024-01-24T23:23:05.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 1 -- 172.21.15.132:0/575164020 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fac44004d30 con 0x7fac5409bcc0 2024-01-24T23:23:05.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac50ff9640 1 -- 172.21.15.132:0/575164020 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(44..44 src has 1..44) v4 ==== 3636+0+0 (unknown 3231806219 0 0) 0x7fac44043710 con 0x7fac5409bcc0 2024-01-24T23:23:05.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.228+0000 7fac537fe640 10 monclient: get_auth_request con 0x7fac40037710 auth_method 0 2024-01-24T23:23:05.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.232+0000 7fac50ff9640 1 -- 172.21.15.132:0/575164020 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fac44014980 con 0x7fac5409bcc0 2024-01-24T23:23:05.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.232+0000 7fac50ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:05.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.232+0000 7fac50ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:05.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.392+0000 7fac5a9a1640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}] 2024-01-24T23:23:05.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.392+0000 7fac5a9a1640 10 monclient: _send_command 2 [{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}] 2024-01-24T23:23:05.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.392+0000 7fac5a9a1640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:05.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:05.392+0000 7fac5a9a1640 1 -- 172.21.15.132:0/575164020 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]} v 0) v1 -- 0x7fac340058c0 con 0x7fac5409bcc0 2024-01-24T23:23:06.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:06.116+0000 7fac50ff9640 1 -- 172.21.15.132:0/575164020 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}]=0 set osd(s) 1 to class 'hdd' v45) v1 ==== 135+0+0 (unknown 2816690131 0 0) 0x7fac44013020 con 0x7fac5409bcc0 2024-01-24T23:23:06.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:06.116+0000 7fac50ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}] 2024-01-24T23:23:06.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:06.116+0000 7fac50ff9640 10 monclient: _finish_command 2 = system:0 set osd(s) 1 to class 'hdd' 2024-01-24T23:23:06.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:06.220+0000 7fac5a9a1640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}] 2024-01-24T23:23:06.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:06.220+0000 7fac5a9a1640 10 monclient: _send_command 3 [{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}] 2024-01-24T23:23:06.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:06.220+0000 7fac5a9a1640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:06.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:06.220+0000 7fac5a9a1640 1 -- 172.21.15.132:0/575164020 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]} v 0) v1 -- 0x7fac340010a0 con 0x7fac5409bcc0 2024-01-24T23:23:06.851 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:06.848+0000 7fac50ff9640 1 -- 172.21.15.132:0/575164020 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}]=0 osd.1 already set to class hdd. set-device-class item id 1 name 'osd.1' device_class 'hdd': no change. set osd(s) to class 'hdd' v46) v1 ==== 237+0+0 (unknown 3790924378 0 0) 0x7fac440147a0 con 0x7fac5409bcc0 2024-01-24T23:23:06.851 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:06.848+0000 7fac50ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}] 2024-01-24T23:23:06.851 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:06.848+0000 7fac50ff9640 10 monclient: _finish_command 3 = system:0 osd.1 already set to class hdd. set-device-class item id 1 name 'osd.1' device_class 'hdd': no change. set osd(s) to class 'hdd' 2024-01-24T23:23:06.851 INFO:tasks.workunit.client.0.smithi132.stderr:osd.1 already set to class hdd. set-device-class item id 1 name 'osd.1' device_class 'hdd': no change. set osd(s) to class 'hdd' 2024-01-24T23:23:06.854 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:06.852+0000 7fac5a9a1640 1 -- 172.21.15.132:0/575164020 >> v1:172.21.15.132:6812/1835401922 conn(0x7fac40037710 legacy=0x7fac40039bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:06.854 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:06.852+0000 7fac5a9a1640 10 monclient: shutdown 2024-01-24T23:23:06.854 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:06.852+0000 7fac5a9a1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:06.854 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:06.852+0000 7fac5a9a1640 1 -- 172.21.15.132:0/575164020 >> v1:172.21.15.132:6791/0 conn(0x7fac5409bcc0 legacy=0x7fac54130950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:06.854 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:06.852+0000 7fac5a9a1640 1 -- 172.21.15.132:0/575164020 shutdown_connections 2024-01-24T23:23:06.854 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:06.852+0000 7fac5a9a1640 1 -- 172.21.15.132:0/575164020 >> 172.21.15.132:0/575164020 conn(0x7fac540a9c10 msgr2=0x7fac54066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:06.855 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:06.852+0000 7fac5a9a1640 1 -- 172.21.15.132:0/575164020 shutdown_connections 2024-01-24T23:23:06.855 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:06.856+0000 7fac5a9a1640 1 -- 172.21.15.132:0/575164020 wait complete. 2024-01-24T23:23:06.877 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule create-replicated foo-ssd default host ssd 2024-01-24T23:23:06.999 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 10 monclient: get_monmap_and_config 2024-01-24T23:23:06.999 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 10 monclient: build_initial_monmap 2024-01-24T23:23:07.000 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 10 monclient: monmap: 2024-01-24T23:23:07.000 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:07.000 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:07.000 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:07.001103+0000 2024-01-24T23:23:07.000 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:07.001103+0000 2024-01-24T23:23:07.000 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:07.000 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:07.000 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:07.000 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:07.000 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:07.000 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:07.000 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:07.000 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:07.000 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:07.001 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:07.001 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:07.001 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:07.006 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 1 Processor -- start 2024-01-24T23:23:07.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 1 -- start start 2024-01-24T23:23:07.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 10 monclient: init 2024-01-24T23:23:07.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:07.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 10 monclient: _add_conns ranks=[0,1,3,8,7,2,5,6,4] 2024-01-24T23:23:07.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 10 monclient(hunting): picked mon.noname-a con 0x7f5c90136cb0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:07.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 10 monclient(hunting): picked mon.noname-f con 0x7f5c901377a0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:07.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 10 monclient(hunting): picked mon.noname-g con 0x7f5c9009bcc0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:07.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c90064700 con 0x7f5c90136cb0 2024-01-24T23:23:07.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c9009b650 con 0x7f5c901377a0 2024-01-24T23:23:07.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c9009c110 con 0x7f5c9009bcc0 2024-01-24T23:23:07.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:07.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 10 monclient(hunting): authenticate will time out at 1074.325675s 2024-01-24T23:23:07.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8ffff640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f5c901377a0 0x7f5c9013c1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:48288/0 (socket says 172.21.15.132:48288) 2024-01-24T23:23:07.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8ffff640 1 -- 172.21.15.132:0/1736832482 learned_addr learned my addr 172.21.15.132:0/1736832482 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:07.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c95289640 1 --1- 172.21.15.132:0/1736832482 >> v1:172.21.15.132:6790/0 conn(0x7f5c9009bcc0 0x7f5c9013f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:49414/0 (socket says 172.21.15.132:49414) 2024-01-24T23:23:07.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 1 -- 172.21.15.132:0/1736832482 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1409923565 0 0) 0x7f5c9009b650 con 0x7f5c901377a0 2024-01-24T23:23:07.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:07.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:07.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:07.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 10 monclient(hunting): my global_id is 4924 2024-01-24T23:23:07.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 1 -- 172.21.15.132:0/1736832482 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c78003530 con 0x7f5c901377a0 2024-01-24T23:23:07.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 1 -- 172.21.15.132:0/1736832482 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2151578068 0 0) 0x7f5c9009c110 con 0x7f5c9009bcc0 2024-01-24T23:23:07.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:07.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:07.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:07.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 10 monclient(hunting): my global_id is 4809 2024-01-24T23:23:07.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 1 -- 172.21.15.132:0/1736832482 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c9009b650 con 0x7f5c9009bcc0 2024-01-24T23:23:07.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 1 -- 172.21.15.132:0/1736832482 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1474506101 0 0) 0x7f5c9009b650 con 0x7f5c9009bcc0 2024-01-24T23:23:07.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 1 -- 172.21.15.132:0/1736832482 >> v1:172.21.15.132:6789/0 conn(0x7f5c901377a0 legacy=0x7f5c9013c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:07.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 1 -- 172.21.15.132:0/1736832482 >> v1:172.21.15.115:6789/0 conn(0x7f5c90136cb0 legacy=0x7f5c90137090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:07.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:23:07.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 1 monclient: found mon.noname-g 2024-01-24T23:23:07.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:07.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:23:07.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 1 -- 172.21.15.132:0/1736832482 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9009b000 con 0x7f5c9009bcc0 2024-01-24T23:23:07.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 10 monclient: _finish_auth 0 2024-01-24T23:23:07.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:23:07.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:07.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 5 monclient: authenticate success, global_id 4809 2024-01-24T23:23:07.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:07.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 1 -- 172.21.15.132:0/1736832482 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5c88001b40 con 0x7f5c9009bcc0 2024-01-24T23:23:07.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:07.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:07.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 10 monclient: dump: 2024-01-24T23:23:07.010 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:07.010 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:07.010 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:07.010 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:07.010 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:07.010 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:07.010 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:07.010 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:07.010 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:07.010 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:07.010 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:07.010 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:07.010 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:07.010 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:07.010 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:07.011 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:07.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:07.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 1 -- 172.21.15.132:0/1736832482 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5c88001d60 con 0x7f5c9009bcc0 2024-01-24T23:23:07.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:07.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:07.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 1 -- 172.21.15.132:0/1736832482 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5c88004eb0 con 0x7f5c9009bcc0 2024-01-24T23:23:07.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 4 set_mon_vals no callback set 2024-01-24T23:23:07.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:07.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:07.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c8f7fe640 10 monclient: dump: 2024-01-24T23:23:07.011 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:07.011 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:07.011 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:07.011 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:07.011 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:07.012 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:07.012 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:07.012 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:07.012 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:07.012 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:07.012 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:07.012 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:07.012 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:07.012 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:07.012 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:07.012 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:07.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 10 monclient: shutdown 2024-01-24T23:23:07.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:07.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 1 -- 172.21.15.132:0/1736832482 >> v1:172.21.15.132:6790/0 conn(0x7f5c9009bcc0 legacy=0x7f5c9013f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:07.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 1 -- 172.21.15.132:0/1736832482 shutdown_connections 2024-01-24T23:23:07.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 1 -- 172.21.15.132:0/1736832482 >> 172.21.15.132:0/1736832482 conn(0x7f5c900a9c10 msgr2=0x7f5c900a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:07.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 1 -- 172.21.15.132:0/1736832482 shutdown_connections 2024-01-24T23:23:07.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 1 -- 172.21.15.132:0/1736832482 wait complete. 2024-01-24T23:23:07.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 10 monclient: build_initial_monmap 2024-01-24T23:23:07.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.000+0000 7f5c96cec640 10 monclient: monmap: 2024-01-24T23:23:07.013 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:07.013 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:07.013 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:07.013 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:07.013 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:07.013 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:07.013 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:07.013 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:07.013 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:07.013 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:07.014 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:07.014 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:07.014 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:07.014 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:07.014 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:07.014 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:07.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c96cec640 1 Processor -- start 2024-01-24T23:23:07.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c96cec640 1 -- start start 2024-01-24T23:23:07.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c96cec640 10 monclient: init 2024-01-24T23:23:07.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c96cec640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:07.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c96cec640 10 monclient: _add_conns ranks=[3,2,7,5,1,0,6,4,8] 2024-01-24T23:23:07.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c96cec640 10 monclient(hunting): picked mon.noname-g con 0x7f5c9009bcc0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:07.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c96cec640 10 monclient(hunting): picked mon.noname-b con 0x7f5c90136cb0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:07.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c96cec640 10 monclient(hunting): picked mon.noname-i con 0x7f5c901377a0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:07.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c96cec640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c9009c710 con 0x7f5c90136cb0 2024-01-24T23:23:07.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c96cec640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c90124600 con 0x7f5c9009bcc0 2024-01-24T23:23:07.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c96cec640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c90137fb0 con 0x7f5c901377a0 2024-01-24T23:23:07.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c96cec640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:07.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c96cec640 10 monclient(hunting): authenticate will time out at 1074.328859s 2024-01-24T23:23:07.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c94a88640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f5c9009bcc0 0x7f5c90134ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:49424/0 (socket says 172.21.15.132:49424) 2024-01-24T23:23:07.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c94a88640 1 -- 172.21.15.132:0/3397178995 learned_addr learned my addr 172.21.15.132:0/3397178995 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:07.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8ffff640 1 -- 172.21.15.132:0/3397178995 >> v1:172.21.15.115:6790/0 conn(0x7f5c90136cb0 legacy=0x7f5c901d0570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:23:07.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 1 -- 172.21.15.132:0/3397178995 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 783805025 0 0) 0x7f5c90124600 con 0x7f5c9009bcc0 2024-01-24T23:23:07.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:23:07.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:07.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:07.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 10 monclient(hunting): my global_id is 4818 2024-01-24T23:23:07.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 1 -- 172.21.15.132:0/3397178995 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c7c003530 con 0x7f5c9009bcc0 2024-01-24T23:23:07.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 1 -- 172.21.15.132:0/3397178995 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 907594717 0 0) 0x7f5c90137fb0 con 0x7f5c901377a0 2024-01-24T23:23:07.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:23:07.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:07.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:07.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 10 monclient(hunting): my global_id is 4921 2024-01-24T23:23:07.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 1 -- 172.21.15.132:0/3397178995 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c90124600 con 0x7f5c901377a0 2024-01-24T23:23:07.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 1 -- 172.21.15.132:0/3397178995 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3040065737 0 0) 0x7f5c7c003530 con 0x7f5c9009bcc0 2024-01-24T23:23:07.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 1 -- 172.21.15.132:0/3397178995 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5c90137fb0 con 0x7f5c9009bcc0 2024-01-24T23:23:07.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 1 -- 172.21.15.132:0/3397178995 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5c80001ae0 con 0x7f5c9009bcc0 2024-01-24T23:23:07.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:07.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:07.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 10 monclient(hunting): dump: 2024-01-24T23:23:07.016 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:07.016 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:07.016 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:07.017 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:07.017 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:07.017 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:07.017 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:07.017 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:07.017 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:07.017 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:07.017 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:07.017 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:07.017 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:07.017 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:07.017 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:07.017 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:07.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:07.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 1 -- 172.21.15.132:0/3397178995 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 417795877 0 0) 0x7f5c90124600 con 0x7f5c901377a0 2024-01-24T23:23:07.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 1 -- 172.21.15.132:0/3397178995 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5c7c003530 con 0x7f5c901377a0 2024-01-24T23:23:07.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 1 -- 172.21.15.132:0/3397178995 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 72059046 0 0) 0x7f5c90137fb0 con 0x7f5c9009bcc0 2024-01-24T23:23:07.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 1 -- 172.21.15.132:0/3397178995 >> v1:172.21.15.132:6792/0 conn(0x7f5c901377a0 legacy=0x7f5c901d3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:07.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 1 -- 172.21.15.132:0/3397178995 >> v1:172.21.15.115:6790/0 conn(0x7f5c90136cb0 legacy=0x7f5c901d0570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:07.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 10 monclient: _finish_hunting 0 2024-01-24T23:23:07.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 1 monclient: found mon.g 2024-01-24T23:23:07.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:07.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:07.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 1 -- 172.21.15.132:0/3397178995 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c901222a0 con 0x7f5c9009bcc0 2024-01-24T23:23:07.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 10 monclient: _finish_auth 0 2024-01-24T23:23:07.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:23:07.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:07.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c96cec640 5 monclient: authenticate success, global_id 4818 2024-01-24T23:23:07.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c96cec640 10 monclient: _renew_subs 2024-01-24T23:23:07.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c96cec640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:07.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c96cec640 1 -- 172.21.15.132:0/3397178995 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c90124320 con 0x7f5c9009bcc0 2024-01-24T23:23:07.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 1 -- 172.21.15.132:0/3397178995 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5c80003950 con 0x7f5c9009bcc0 2024-01-24T23:23:07.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c96cec640 10 monclient: _renew_subs 2024-01-24T23:23:07.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c96cec640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:07.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c96cec640 1 -- 172.21.15.132:0/3397178995 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c901be670 con 0x7f5c9009bcc0 2024-01-24T23:23:07.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:07.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 1 -- 172.21.15.132:0/3397178995 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5c80003e00 con 0x7f5c9009bcc0 2024-01-24T23:23:07.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:07.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:07.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 10 monclient: dump: 2024-01-24T23:23:07.019 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:07.019 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:07.019 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:07.019 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:07.019 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:07.019 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:07.020 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:07.020 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:07.020 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:07.020 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:07.020 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:07.020 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:07.020 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:07.020 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:07.020 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:07.020 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:07.020 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8effd640 4 set_mon_vals no callback set 2024-01-24T23:23:07.020 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c96cec640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:07.020 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c96cec640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:07.020 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c96cec640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:07.020 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c96cec640 1 -- 172.21.15.132:0/3397178995 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c900674e0 con 0x7f5c9009bcc0 2024-01-24T23:23:07.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 1 -- 172.21.15.132:0/3397178995 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f5c80003fe0 con 0x7f5c9009bcc0 2024-01-24T23:23:07.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8d7fa640 1 -- 172.21.15.132:0/3397178995 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(46..46 src has 1..46) v4 ==== 3868+0+0 (unknown 1252667990 0 0) 0x7f5c800438b0 con 0x7f5c9009bcc0 2024-01-24T23:23:07.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.004+0000 7f5c8ffff640 10 monclient: get_auth_request con 0x7f5c7c0371d0 auth_method 0 2024-01-24T23:23:07.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.008+0000 7f5c8d7fa640 1 -- 172.21.15.132:0/3397178995 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f5c80014ac0 con 0x7f5c9009bcc0 2024-01-24T23:23:07.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.008+0000 7f5c8d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:07.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.008+0000 7f5c8d7fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:07.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.168+0000 7f5c96cec640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}] 2024-01-24T23:23:07.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.168+0000 7f5c96cec640 10 monclient: _send_command 2 [{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}] 2024-01-24T23:23:07.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.168+0000 7f5c96cec640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:07.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:07.168+0000 7f5c96cec640 1 -- 172.21.15.132:0/3397178995 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"} v 0) v1 -- 0x7f5c90073840 con 0x7f5c9009bcc0 2024-01-24T23:23:08.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.128+0000 7f5c8d7fa640 1 -- 172.21.15.132:0/3397178995 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}]=0 v47) v1 ==== 150+0+0 (unknown 617647234 0 0) 0x7f5c80012f30 con 0x7f5c9009bcc0 2024-01-24T23:23:08.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.128+0000 7f5c8d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}] 2024-01-24T23:23:08.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.128+0000 7f5c8d7fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:08.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.224+0000 7f5c96cec640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}] 2024-01-24T23:23:08.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.224+0000 7f5c96cec640 10 monclient: _send_command 3 [{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}] 2024-01-24T23:23:08.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.224+0000 7f5c96cec640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:08.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.224+0000 7f5c96cec640 1 -- 172.21.15.132:0/3397178995 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"} v 0) v1 -- 0x7f5c900008e0 con 0x7f5c9009bcc0 2024-01-24T23:23:08.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.228+0000 7f5c8d7fa640 1 -- 172.21.15.132:0/3397178995 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}]=0 rule foo-ssd already exists v47) v1 ==== 177+0+0 (unknown 2970939114 0 0) 0x7f5c80014890 con 0x7f5c9009bcc0 2024-01-24T23:23:08.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.228+0000 7f5c8d7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}] 2024-01-24T23:23:08.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.228+0000 7f5c8d7fa640 10 monclient: _finish_command 3 = system:0 rule foo-ssd already exists 2024-01-24T23:23:08.228 INFO:tasks.workunit.client.0.smithi132.stderr:rule foo-ssd already exists 2024-01-24T23:23:08.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.232+0000 7f5c96cec640 1 -- 172.21.15.132:0/3397178995 >> v1:172.21.15.132:6812/1835401922 conn(0x7f5c7c0371d0 legacy=0x7f5c7c039690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:08.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.232+0000 7f5c96cec640 10 monclient: shutdown 2024-01-24T23:23:08.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.232+0000 7f5c96cec640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:08.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.232+0000 7f5c96cec640 1 -- 172.21.15.132:0/3397178995 >> v1:172.21.15.132:6790/0 conn(0x7f5c9009bcc0 legacy=0x7f5c90134ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:08.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.232+0000 7f5c96cec640 1 -- 172.21.15.132:0/3397178995 shutdown_connections 2024-01-24T23:23:08.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.232+0000 7f5c96cec640 1 -- 172.21.15.132:0/3397178995 >> 172.21.15.132:0/3397178995 conn(0x7f5c900a9c10 msgr2=0x7f5c901bd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:08.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.232+0000 7f5c96cec640 1 -- 172.21.15.132:0/3397178995 shutdown_connections 2024-01-24T23:23:08.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.232+0000 7f5c96cec640 1 -- 172.21.15.132:0/3397178995 wait complete. 2024-01-24T23:23:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule create-replicated foo-hdd default host hdd 2024-01-24T23:23:08.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.368+0000 7fa71287a640 10 monclient: get_monmap_and_config 2024-01-24T23:23:08.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.368+0000 7fa71287a640 10 monclient: build_initial_monmap 2024-01-24T23:23:08.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.368+0000 7fa71287a640 10 monclient: monmap: 2024-01-24T23:23:08.370 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:08.370 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:08.370 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:08.371633+0000 2024-01-24T23:23:08.370 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:08.371633+0000 2024-01-24T23:23:08.370 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:08.371 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:08.371 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:08.371 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:08.371 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:08.371 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:08.371 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:08.371 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:08.371 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:08.371 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:08.371 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:08.371 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:08.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.368+0000 7fa71287a640 1 Processor -- start 2024-01-24T23:23:08.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.368+0000 7fa71287a640 1 -- start start 2024-01-24T23:23:08.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.368+0000 7fa71287a640 10 monclient: init 2024-01-24T23:23:08.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:08.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 10 monclient: _add_conns ranks=[0,4,2,6,3,1,8,5,7] 2024-01-24T23:23:08.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 10 monclient(hunting): picked mon.noname-a con 0x7fa70c09c4c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:08.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 10 monclient(hunting): picked mon.noname-c con 0x7fa70c09cfb0 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:08.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 10 monclient(hunting): picked mon.noname-b con 0x7fa70c09ec50 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:08.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa70c064700 con 0x7fa70c09c4c0 2024-01-24T23:23:08.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa70c09d9f0 con 0x7fa70c09ec50 2024-01-24T23:23:08.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa70c09dc60 con 0x7fa70c09cfb0 2024-01-24T23:23:08.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:08.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 10 monclient(hunting): authenticate will time out at 1075.696422s 2024-01-24T23:23:08.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa710e17640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fa70c09ec50 legacy=0x7fa70c13c460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:23:08.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70bfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fa70c09c4c0 0x7fa70c09c8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:34328/0 (socket says 172.21.15.132:34328) 2024-01-24T23:23:08.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70bfff640 1 -- 172.21.15.132:0/1744631530 learned_addr learned my addr 172.21.15.132:0/1744631530 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:08.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70b7fe640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fa70c09cfb0 0x7fa70c138d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:58316/0 (socket says 172.21.15.132:58316) 2024-01-24T23:23:08.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 1 -- 172.21.15.132:0/1744631530 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2617932726 0 0) 0x7fa70c064700 con 0x7fa70c09c4c0 2024-01-24T23:23:08.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:08.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:08.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:08.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 10 monclient(hunting): my global_id is 4860 2024-01-24T23:23:08.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 1 -- 172.21.15.132:0/1744631530 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f4003530 con 0x7fa70c09c4c0 2024-01-24T23:23:08.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 1 -- 172.21.15.132:0/1744631530 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1858203702 0 0) 0x7fa70c09dc60 con 0x7fa70c09cfb0 2024-01-24T23:23:08.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:08.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:08.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:08.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 10 monclient(hunting): my global_id is 74884 2024-01-24T23:23:08.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 1 -- 172.21.15.132:0/1744631530 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa70c064700 con 0x7fa70c09cfb0 2024-01-24T23:23:08.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 1 -- 172.21.15.132:0/1744631530 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3658279672 0 0) 0x7fa6f4003530 con 0x7fa70c09c4c0 2024-01-24T23:23:08.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 1 -- 172.21.15.132:0/1744631530 >> v1:172.21.15.115:6791/0 conn(0x7fa70c09cfb0 legacy=0x7fa70c138d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:08.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 1 -- 172.21.15.132:0/1744631530 >> v1:172.21.15.115:6790/0 conn(0x7fa70c09ec50 legacy=0x7fa70c13c460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:08.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:08.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 1 monclient: found mon.noname-a 2024-01-24T23:23:08.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:08.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:23:08.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 1 -- 172.21.15.132:0/1744631530 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70c09df40 con 0x7fa70c09c4c0 2024-01-24T23:23:08.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 10 monclient: _finish_auth 0 2024-01-24T23:23:08.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:08.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:08.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 1 -- 172.21.15.132:0/1744631530 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa6fc0017c0 con 0x7fa70c09c4c0 2024-01-24T23:23:08.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 5 monclient: authenticate success, global_id 4860 2024-01-24T23:23:08.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:08.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:08.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:23:08.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 10 monclient: dump: 2024-01-24T23:23:08.381 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:08.381 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:08.381 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:08.381 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:08.381 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:08.381 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:08.381 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:08.381 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:08.381 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:08.382 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:08.382 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:08.382 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:08.382 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:08.382 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:08.382 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:08.382 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:08.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:08.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 1 -- 172.21.15.132:0/1744631530 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa6fc001aa0 con 0x7fa70c09c4c0 2024-01-24T23:23:08.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:08.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 1 -- 172.21.15.132:0/1744631530 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa6fc004bf0 con 0x7fa70c09c4c0 2024-01-24T23:23:08.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:08.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 4 set_mon_vals no callback set 2024-01-24T23:23:08.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:08.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:23:08.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa70affd640 10 monclient: dump: 2024-01-24T23:23:08.383 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:08.383 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:08.383 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:08.383 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:08.383 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:08.383 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:08.383 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:08.383 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:08.383 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:08.383 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:08.383 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:08.383 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:08.383 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:08.383 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:08.384 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:08.384 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:08.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 10 monclient: shutdown 2024-01-24T23:23:08.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:08.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 1 -- 172.21.15.132:0/1744631530 >> v1:172.21.15.115:6789/0 conn(0x7fa70c09c4c0 legacy=0x7fa70c09c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:08.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 1 -- 172.21.15.132:0/1744631530 shutdown_connections 2024-01-24T23:23:08.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 1 -- 172.21.15.132:0/1744631530 >> 172.21.15.132:0/1744631530 conn(0x7fa70c0a5a10 msgr2=0x7fa70c0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:08.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 1 -- 172.21.15.132:0/1744631530 shutdown_connections 2024-01-24T23:23:08.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 1 -- 172.21.15.132:0/1744631530 wait complete. 2024-01-24T23:23:08.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 10 monclient: build_initial_monmap 2024-01-24T23:23:08.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 10 monclient: monmap: 2024-01-24T23:23:08.384 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:08.384 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:08.384 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:08.385 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:08.385 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:08.385 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:08.385 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:08.385 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:08.385 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:08.385 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:08.385 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:08.385 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:08.385 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:08.385 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:08.385 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:08.385 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:08.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 1 Processor -- start 2024-01-24T23:23:08.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 1 -- start start 2024-01-24T23:23:08.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.372+0000 7fa71287a640 10 monclient: init 2024-01-24T23:23:08.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa71287a640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:08.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa71287a640 10 monclient: _add_conns ranks=[5,6,3,2,0,4,8,1,7] 2024-01-24T23:23:08.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa71287a640 10 monclient(hunting): picked mon.noname-h con 0x7fa70c09c4c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:08.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa71287a640 10 monclient(hunting): picked mon.noname-d con 0x7fa70c09cfb0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:08.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa71287a640 10 monclient(hunting): picked mon.noname-g con 0x7fa70c09ec50 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:08.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa71287a640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa70c09c020 con 0x7fa70c09ec50 2024-01-24T23:23:08.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa71287a640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa70c130390 con 0x7fa70c09c4c0 2024-01-24T23:23:08.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa71287a640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa70c11fc70 con 0x7fa70c09cfb0 2024-01-24T23:23:08.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa71287a640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:08.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa71287a640 10 monclient(hunting): authenticate will time out at 1075.700146s 2024-01-24T23:23:08.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa70bfff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fa70c09c4c0 0x7fa70c1cb7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:35212/0 (socket says 172.21.15.132:35212) 2024-01-24T23:23:08.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa70bfff640 1 -- 172.21.15.132:0/1725786250 learned_addr learned my addr 172.21.15.132:0/1725786250 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:08.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 1 -- 172.21.15.132:0/1725786250 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562311893 0 0) 0x7fa70c09c020 con 0x7fa70c09ec50 2024-01-24T23:23:08.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:08.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:08.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:08.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient(hunting): my global_id is 4827 2024-01-24T23:23:08.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 1 -- 172.21.15.132:0/1725786250 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f8003560 con 0x7fa70c09ec50 2024-01-24T23:23:08.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 1 -- 172.21.15.132:0/1725786250 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 945443019 0 0) 0x7fa70c130390 con 0x7fa70c09c4c0 2024-01-24T23:23:08.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:08.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:08.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:08.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient(hunting): my global_id is 4874 2024-01-24T23:23:08.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 1 -- 172.21.15.132:0/1725786250 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa70c09c020 con 0x7fa70c09c4c0 2024-01-24T23:23:08.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 1 -- 172.21.15.132:0/1725786250 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1580710298 0 0) 0x7fa6f8003560 con 0x7fa70c09ec50 2024-01-24T23:23:08.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 1 -- 172.21.15.132:0/1725786250 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa70c130390 con 0x7fa70c09ec50 2024-01-24T23:23:08.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 1 -- 172.21.15.132:0/1725786250 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa704004560 con 0x7fa70c09ec50 2024-01-24T23:23:08.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:08.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:08.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient(hunting): dump: 2024-01-24T23:23:08.388 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:08.388 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:08.388 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:08.388 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:08.388 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:08.388 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:08.388 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:08.388 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:08.388 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:08.388 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:08.388 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:08.388 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:08.388 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:08.388 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:08.389 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:08.389 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:08.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:08.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 1 -- 172.21.15.132:0/1725786250 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 866170019 0 0) 0x7fa70c09c020 con 0x7fa70c09c4c0 2024-01-24T23:23:08.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 1 -- 172.21.15.132:0/1725786250 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa6f8003560 con 0x7fa70c09c4c0 2024-01-24T23:23:08.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 1 -- 172.21.15.132:0/1725786250 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3289176609 0 0) 0x7fa70c11fc70 con 0x7fa70c09cfb0 2024-01-24T23:23:08.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:08.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:08.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:08.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient(hunting): my global_id is 4911 2024-01-24T23:23:08.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 1 -- 172.21.15.132:0/1725786250 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa70c09c020 con 0x7fa70c09cfb0 2024-01-24T23:23:08.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 1 -- 172.21.15.132:0/1725786250 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa6fc004b30 con 0x7fa70c09c4c0 2024-01-24T23:23:08.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:08.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:08.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient(hunting): dump: 2024-01-24T23:23:08.389 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:08.390 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:08.390 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:08.390 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:08.390 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:08.390 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:08.390 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:08.390 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:08.390 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:08.390 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:08.390 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:08.390 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:08.390 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:08.390 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:08.390 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:08.390 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:08.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 1 -- 172.21.15.132:0/1725786250 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4265483062 0 0) 0x7fa70c130390 con 0x7fa70c09ec50 2024-01-24T23:23:08.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 1 -- 172.21.15.132:0/1725786250 >> v1:172.21.15.115:6792/0 conn(0x7fa70c09cfb0 legacy=0x7fa70c1d0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:08.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 1 -- 172.21.15.132:0/1725786250 >> v1:172.21.15.132:6791/0 conn(0x7fa70c09c4c0 legacy=0x7fa70c1cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:08.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:23:08.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 1 monclient: found mon.g 2024-01-24T23:23:08.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:08.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:08.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 1 -- 172.21.15.132:0/1725786250 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70c11ff50 con 0x7fa70c09ec50 2024-01-24T23:23:08.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient: _finish_auth 0 2024-01-24T23:23:08.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:08.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:08.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa71287a640 5 monclient: authenticate success, global_id 4827 2024-01-24T23:23:08.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa71287a640 10 monclient: _renew_subs 2024-01-24T23:23:08.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa71287a640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:08.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa71287a640 1 -- 172.21.15.132:0/1725786250 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa70c120230 con 0x7fa70c09ec50 2024-01-24T23:23:08.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa71287a640 10 monclient: _renew_subs 2024-01-24T23:23:08.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa71287a640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:08.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa71287a640 1 -- 172.21.15.132:0/1725786250 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa70c12d460 con 0x7fa70c09ec50 2024-01-24T23:23:08.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 1 -- 172.21.15.132:0/1725786250 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa7040047d0 con 0x7fa70c09ec50 2024-01-24T23:23:08.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:08.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 1 -- 172.21.15.132:0/1725786250 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa704004d00 con 0x7fa70c09ec50 2024-01-24T23:23:08.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:08.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:08.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa709ffb640 4 set_mon_vals no callback set 2024-01-24T23:23:08.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 10 monclient: dump: 2024-01-24T23:23:08.392 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:08.392 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:08.392 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:08.392 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:08.392 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:08.392 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:08.393 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:08.393 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:08.393 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:08.393 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:08.393 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:08.393 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:08.393 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:08.393 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:08.393 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:08.393 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:08.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa71287a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:08.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa71287a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:08.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa71287a640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:08.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 1 -- 172.21.15.132:0/1725786250 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fa704004e60 con 0x7fa70c09ec50 2024-01-24T23:23:08.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa71287a640 1 -- 172.21.15.132:0/1725786250 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6ec005090 con 0x7fa70c09ec50 2024-01-24T23:23:08.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa708ff9640 1 -- 172.21.15.132:0/1725786250 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(47..47 src has 1..47) v4 ==== 3931+0+0 (unknown 3931126255 0 0) 0x7fa704043a40 con 0x7fa70c09ec50 2024-01-24T23:23:08.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.376+0000 7fa70bfff640 10 monclient: get_auth_request con 0x7fa6f8037890 auth_method 0 2024-01-24T23:23:08.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.380+0000 7fa708ff9640 1 -- 172.21.15.132:0/1725786250 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fa704014c10 con 0x7fa70c09ec50 2024-01-24T23:23:08.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.380+0000 7fa708ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:08.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.380+0000 7fa708ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:08.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.540+0000 7fa71287a640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}] 2024-01-24T23:23:08.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.540+0000 7fa71287a640 10 monclient: _send_command 2 [{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}] 2024-01-24T23:23:08.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.540+0000 7fa71287a640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:08.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:08.540+0000 7fa71287a640 1 -- 172.21.15.132:0/1725786250 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"} v 0) v1 -- 0x7fa6ec005340 con 0x7fa70c09ec50 2024-01-24T23:23:09.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.140+0000 7fa708ff9640 1 -- 172.21.15.132:0/1725786250 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}]=0 v48) v1 ==== 150+0+0 (unknown 3144599289 0 0) 0x7fa704013080 con 0x7fa70c09ec50 2024-01-24T23:23:09.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.140+0000 7fa708ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}] 2024-01-24T23:23:09.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.140+0000 7fa708ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:09.189 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-01-24T23:23:09.189 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-01-24T23:23:09.189 INFO:tasks.ceph.mon.b:Restarting daemon 2024-01-24T23:23:09.189 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-01-24T23:23:09.192 INFO:tasks.ceph.mon.b:Started 2024-01-24T23:23:09.192 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-01-24T23:23:09.192 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.e 2024-01-24T23:23:09.192 INFO:tasks.ceph.mon.e:Restarting daemon 2024-01-24T23:23:09.193 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2024-01-24T23:23:09.195 INFO:tasks.ceph.mon.e:Started 2024-01-24T23:23:09.195 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-01-24T23:23:09.195 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-01-24T23:23:09.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.240+0000 7fa71287a640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}] 2024-01-24T23:23:09.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.240+0000 7fa71287a640 10 monclient: _send_command 3 [{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}] 2024-01-24T23:23:09.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.240+0000 7fa71287a640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:09.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.240+0000 7fa71287a640 1 -- 172.21.15.132:0/1725786250 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"} v 0) v1 -- 0x7fa6ec0055e0 con 0x7fa70c09ec50 2024-01-24T23:23:09.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.240+0000 7fa708ff9640 1 -- 172.21.15.132:0/1725786250 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}]=0 rule foo-hdd already exists v48) v1 ==== 177+0+0 (unknown 2860057506 0 0) 0x7fa704014800 con 0x7fa70c09ec50 2024-01-24T23:23:09.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.240+0000 7fa708ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}] 2024-01-24T23:23:09.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.240+0000 7fa708ff9640 10 monclient: _finish_command 3 = system:0 rule foo-hdd already exists 2024-01-24T23:23:09.243 INFO:tasks.workunit.client.0.smithi132.stderr:rule foo-hdd already exists 2024-01-24T23:23:09.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.244+0000 7fa71287a640 1 -- 172.21.15.132:0/1725786250 >> v1:172.21.15.132:6812/1835401922 conn(0x7fa6f8037890 legacy=0x7fa6f8039d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:09.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.244+0000 7fa71287a640 10 monclient: shutdown 2024-01-24T23:23:09.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.244+0000 7fa71287a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:09.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.244+0000 7fa71287a640 1 -- 172.21.15.132:0/1725786250 >> v1:172.21.15.132:6790/0 conn(0x7fa70c09ec50 legacy=0x7fa70c1d3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:09.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.244+0000 7fa71287a640 1 -- 172.21.15.132:0/1725786250 shutdown_connections 2024-01-24T23:23:09.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.244+0000 7fa71287a640 1 -- 172.21.15.132:0/1725786250 >> 172.21.15.132:0/1725786250 conn(0x7fa70c0a5a10 msgr2=0x7fa70c071240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:09.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.244+0000 7fa71287a640 1 -- 172.21.15.132:0/1725786250 shutdown_connections 2024-01-24T23:23:09.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.244+0000 7fa71287a640 1 -- 172.21.15.132:0/1725786250 wait complete. 2024-01-24T23:23:09.260 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule ls-by-class ssd 2024-01-24T23:23:09.260 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep foo-ssd 2024-01-24T23:23:09.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:09.312+0000 7fd0f5978640 10 monclient: get_monmap_and_config 2024-01-24T23:23:09.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:09.312+0000 7fd0f5978640 10 monclient: build_initial_monmap 2024-01-24T23:23:09.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:09.312+0000 7fd0f5978640 10 monclient: monmap: 2024-01-24T23:23:09.317 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:09.317 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:09.317 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:09.317590+0000 2024-01-24T23:23:09.317 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:09.317590+0000 2024-01-24T23:23:09.317 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:09.317 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:09.317 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:09.317 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:09.317 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:09.317 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:09.317 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:09.318 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:09.318 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:09.318 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:09.318 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:09.318 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:09.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:09.312+0000 7fd0f5978640 1 Processor -- start 2024-01-24T23:23:09.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:09.312+0000 7fd0f5978640 1 -- start start 2024-01-24T23:23:09.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:09.312+0000 7fd0f5978640 10 monclient: init 2024-01-24T23:23:09.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:09.312+0000 7fd0f5978640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:09.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:09.312+0000 7fd0f5978640 10 monclient: _add_conns ranks=[1,7,0,3,5,2,8,6,4] 2024-01-24T23:23:09.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:09.312+0000 7fd0f5978640 10 monclient(hunting): picked mon.noname-f con 0x7fd0f0133bf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:09.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:09.312+0000 7fd0f5978640 10 monclient(hunting): picked mon.noname-i con 0x7fd0f0133fd0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:09.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:09.312+0000 7fd0f5978640 10 monclient(hunting): picked mon.noname-a con 0x7fd0f0134430 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:09.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:09.312+0000 7fd0f5978640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f004ed70 con 0x7fd0f0134430 2024-01-24T23:23:09.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:09.312+0000 7fd0f5978640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f009af30 con 0x7fd0f0133bf0 2024-01-24T23:23:09.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:09.312+0000 7fd0f5978640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f009b210 con 0x7fd0f0133fd0 2024-01-24T23:23:09.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:09.312+0000 7fd0f5978640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:09.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:09.312+0000 7fd0f5978640 10 monclient(hunting): authenticate will time out at 1068.239931s 2024-01-24T23:23:09.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:09.316+0000 7fd0ef7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fd0f0134430 0x7fd0f013f500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59320/0 (socket says 172.21.15.115:59320) 2024-01-24T23:23:09.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:09.316+0000 7fd0ef7fe640 1 -- 172.21.15.115:0/1034922215 learned_addr learned my addr 172.21.15.115:0/1034922215 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:09.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.380+0000 7fa4b0489640 10 monclient: get_monmap_and_config 2024-01-24T23:23:09.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.380+0000 7fa4b0489640 10 monclient: build_initial_monmap 2024-01-24T23:23:09.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.380+0000 7fa4b0489640 10 monclient: monmap: 2024-01-24T23:23:09.382 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:09.382 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:09.382 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:09.383254+0000 2024-01-24T23:23:09.382 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:09.383254+0000 2024-01-24T23:23:09.382 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:09.382 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:09.382 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:09.382 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:09.382 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:09.382 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:09.382 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:09.382 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:09.382 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:09.383 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:09.383 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:09.383 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:09.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.380+0000 7fa4b0489640 1 Processor -- start 2024-01-24T23:23:09.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.380+0000 7fa4b0489640 1 -- start start 2024-01-24T23:23:09.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.380+0000 7fa4b0489640 10 monclient: init 2024-01-24T23:23:09.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.380+0000 7fa4b0489640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:09.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.380+0000 7fa4b0489640 10 monclient: _add_conns ranks=[4,3,0,6,2,8,5,7,1] 2024-01-24T23:23:09.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.380+0000 7fa4b0489640 10 monclient(hunting): picked mon.noname-c con 0x7fa4a8136e50 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:09.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.380+0000 7fa4b0489640 10 monclient(hunting): picked mon.noname-g con 0x7fa4a80a0290 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:09.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.380+0000 7fa4b0489640 10 monclient(hunting): picked mon.noname-a con 0x7fa4a80a1600 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:09.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.380+0000 7fa4b0489640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a8064700 con 0x7fa4a80a1600 2024-01-24T23:23:09.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.380+0000 7fa4b0489640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a8137940 con 0x7fa4a80a0290 2024-01-24T23:23:09.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.380+0000 7fa4b0489640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a809f9c0 con 0x7fa4a8136e50 2024-01-24T23:23:09.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.380+0000 7fa4b0489640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:09.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.380+0000 7fa4b0489640 10 monclient(hunting): authenticate will time out at 1076.707885s 2024-01-24T23:23:09.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.384+0000 7fa4ada24640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fa4a80a0290 0x7fa4a813bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:49446/0 (socket says 172.21.15.132:49446) 2024-01-24T23:23:09.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:09.384+0000 7fa4ada24640 1 -- 172.21.15.132:0/3915704007 learned_addr learned my addr 172.21.15.132:0/3915704007 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:12.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:12.316+0000 7fd0ed7fa640 10 monclient(hunting): tick 2024-01-24T23:23:12.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:12.316+0000 7fd0ed7fa640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:23:12.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:12.316+0000 7fd0ed7fa640 1 monclient(hunting): continuing hunt 2024-01-24T23:23:12.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:12.316+0000 7fd0ed7fa640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:23:12.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:12.316+0000 7fd0ed7fa640 1 -- 172.21.15.115:0/1034922215 >> v1:172.21.15.132:6792/0 conn(0x7fd0f0133fd0 legacy=0x7fd0f013bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:12.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:12.316+0000 7fd0ed7fa640 1 -- 172.21.15.115:0/1034922215 >> v1:172.21.15.132:6789/0 conn(0x7fd0f0133bf0 legacy=0x7fd0f009bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:12.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:12.316+0000 7fd0ed7fa640 1 -- 172.21.15.115:0/1034922215 >> v1:172.21.15.115:6789/0 conn(0x7fd0f0134430 legacy=0x7fd0f013f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:12.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:12.316+0000 7fd0ed7fa640 10 monclient: _add_conns ranks=[8,4,2,5,3,6] 2024-01-24T23:23:12.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:12.316+0000 7fd0ed7fa640 10 monclient(hunting): picked mon.noname-e con 0x7fd0d00037f0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:12.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:12.316+0000 7fd0ed7fa640 10 monclient(hunting): picked mon.noname-c con 0x7fd0d0007440 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:12.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:12.316+0000 7fd0ed7fa640 10 monclient(hunting): picked mon.noname-b con 0x7fd0d000b1b0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:12.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:12.316+0000 7fd0ed7fa640 1 -- 172.21.15.115:0/1034922215 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f009b4f0 con 0x7fd0d000b1b0 2024-01-24T23:23:12.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:12.316+0000 7fd0ed7fa640 1 -- 172.21.15.115:0/1034922215 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d000ffe0 con 0x7fd0d0007440 2024-01-24T23:23:12.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:12.316+0000 7fd0ed7fa640 1 -- 172.21.15.115:0/1034922215 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d0011230 con 0x7fd0d00037f0 2024-01-24T23:23:12.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:12.316+0000 7fd0ed7fa640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:12.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:12.380+0000 7fa49ffff640 10 monclient(hunting): tick 2024-01-24T23:23:12.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:12.380+0000 7fa49ffff640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:23:12.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:12.380+0000 7fa49ffff640 1 monclient(hunting): continuing hunt 2024-01-24T23:23:12.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:12.380+0000 7fa49ffff640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:23:12.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:12.380+0000 7fa49ffff640 1 -- 172.21.15.132:0/3915704007 >> v1:172.21.15.115:6791/0 conn(0x7fa4a8136e50 legacy=0x7fa4a8137230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:12.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:12.384+0000 7fa49ffff640 1 -- 172.21.15.132:0/3915704007 >> v1:172.21.15.132:6790/0 conn(0x7fa4a80a0290 legacy=0x7fa4a813bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:12.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:12.384+0000 7fa49ffff640 1 -- 172.21.15.132:0/3915704007 >> v1:172.21.15.115:6789/0 conn(0x7fa4a80a1600 legacy=0x7fa4a813f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:12.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:12.384+0000 7fa49ffff640 10 monclient: _add_conns ranks=[6,8,2,5,1,7] 2024-01-24T23:23:12.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:12.384+0000 7fa49ffff640 10 monclient(hunting): picked mon.noname-d con 0x7fa48c0037f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:12.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:12.384+0000 7fa49ffff640 10 monclient(hunting): picked mon.noname-e con 0x7fa48c007480 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:12.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:12.384+0000 7fa49ffff640 10 monclient(hunting): picked mon.noname-b con 0x7fa48c00b210 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:12.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:12.384+0000 7fa49ffff640 1 -- 172.21.15.132:0/3915704007 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a809eda0 con 0x7fa48c00b210 2024-01-24T23:23:12.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:12.384+0000 7fa49ffff640 1 -- 172.21.15.132:0/3915704007 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa48c00ffe0 con 0x7fa48c0037f0 2024-01-24T23:23:12.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:12.384+0000 7fa49ffff640 1 -- 172.21.15.132:0/3915704007 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa48c011230 con 0x7fa48c007480 2024-01-24T23:23:12.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:12.384+0000 7fa49ffff640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:14.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 1 -- 172.21.15.115:0/1034922215 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1786444336 0 0) 0x7fd0d000ffe0 con 0x7fd0d0007440 2024-01-24T23:23:14.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:14.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:14.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:14.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 10 monclient(hunting): my global_id is 74893 2024-01-24T23:23:14.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 1 -- 172.21.15.115:0/1034922215 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0dc003530 con 0x7fd0d0007440 2024-01-24T23:23:14.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 1 -- 172.21.15.115:0/1034922215 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1647179706 0 0) 0x7fd0dc003530 con 0x7fd0d0007440 2024-01-24T23:23:14.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 1 -- 172.21.15.115:0/1034922215 >> v1:172.21.15.115:6793/0 conn(0x7fd0d00037f0 legacy=0x7fd0d0005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 1 -- 172.21.15.115:0/1034922215 >> v1:172.21.15.115:6790/0 conn(0x7fd0d000b1b0 legacy=0x7fd0d000d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:23:14.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 1 monclient: found mon.noname-c 2024-01-24T23:23:14.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:14.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:23:14.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 1 -- 172.21.15.115:0/1034922215 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d0012480 con 0x7fd0d0007440 2024-01-24T23:23:14.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 10 monclient: _finish_auth 0 2024-01-24T23:23:14.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:23:14.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:14.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ef7fe640 1 -- 172.21.15.115:0/1034922215 reap_dead start 2024-01-24T23:23:14.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 1 -- 172.21.15.115:0/1034922215 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd0d8001780 con 0x7fd0d0007440 2024-01-24T23:23:14.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:23:14.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 10 monclient: dump: 2024-01-24T23:23:14.329 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:14.329 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.329 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.329 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.329 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.329 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:14.329 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.330 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.330 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.330 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.330 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.330 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.330 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.330 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.330 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.330 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:14.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0f5978640 5 monclient: authenticate success, global_id 74893 2024-01-24T23:23:14.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0f5978640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:14.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 1 -- 172.21.15.115:0/1034922215 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd0d8001a10 con 0x7fd0d0007440 2024-01-24T23:23:14.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:14.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0f5978640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:14.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0f5978640 4 set_mon_vals no callback set 2024-01-24T23:23:14.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 1 -- 172.21.15.115:0/1034922215 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd0d8001c00 con 0x7fd0d0007440 2024-01-24T23:23:14.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:23:14.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.320+0000 7fd0ee7fc640 10 monclient: dump: 2024-01-24T23:23:14.331 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:14.331 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.331 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.331 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.331 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.331 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:14.331 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.331 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.331 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.331 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.331 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.332 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.332 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.332 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.332 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.332 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:14.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.324+0000 7fd0f5978640 10 monclient: shutdown 2024-01-24T23:23:14.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.324+0000 7fd0f5978640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:14.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.324+0000 7fd0f5978640 1 -- 172.21.15.115:0/1034922215 >> v1:172.21.15.115:6791/0 conn(0x7fd0d0007440 legacy=0x7fd0d00098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.324+0000 7fd0f5978640 1 -- 172.21.15.115:0/1034922215 shutdown_connections 2024-01-24T23:23:14.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.324+0000 7fd0f5978640 1 -- 172.21.15.115:0/1034922215 >> 172.21.15.115:0/1034922215 conn(0x7fd0f00a9910 msgr2=0x7fd0f00a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:14.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.324+0000 7fd0f5978640 1 -- 172.21.15.115:0/1034922215 shutdown_connections 2024-01-24T23:23:14.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.324+0000 7fd0f5978640 1 -- 172.21.15.115:0/1034922215 wait complete. 2024-01-24T23:23:14.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.324+0000 7fd0f5978640 10 monclient: build_initial_monmap 2024-01-24T23:23:14.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.324+0000 7fd0f5978640 10 monclient: monmap: 2024-01-24T23:23:14.337 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:14.338 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:14.338 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:14.338 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:14.338 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:14.338 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:14.338 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:14.339 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:14.339 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:14.339 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:14.339 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:14.339 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:14.339 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:14.339 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:14.339 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:14.339 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:14.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.324+0000 7fd0f5978640 1 Processor -- start 2024-01-24T23:23:14.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.324+0000 7fd0f5978640 1 -- start start 2024-01-24T23:23:14.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.324+0000 7fd0f5978640 10 monclient: init 2024-01-24T23:23:14.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.328+0000 7fd0f5978640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:14.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.328+0000 7fd0f5978640 10 monclient: _add_conns ranks=[7,5,1,6,2,4,0,3,8] 2024-01-24T23:23:14.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.328+0000 7fd0f5978640 10 monclient(hunting): picked mon.noname-i con 0x7fd0d0007440 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:14.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.328+0000 7fd0f5978640 10 monclient(hunting): picked mon.noname-h con 0x7fd0f01cb3f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:14.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.328+0000 7fd0f5978640 10 monclient(hunting): picked mon.noname-f con 0x7fd0f01d1e50 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:14.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.328+0000 7fd0f5978640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f009c8f0 con 0x7fd0f01d1e50 2024-01-24T23:23:14.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.328+0000 7fd0f5978640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f01204f0 con 0x7fd0f01cb3f0 2024-01-24T23:23:14.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.328+0000 7fd0f5978640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f01cc2e0 con 0x7fd0d0007440 2024-01-24T23:23:14.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.328+0000 7fd0eeffd640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fd0d0007440 0x7fd0f0132b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.115:34806/0 (socket says 172.21.15.115:34806) 2024-01-24T23:23:14.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.328+0000 7fd0e67fc640 1 --1- 172.21.15.115:0/811303480 >> v1:172.21.15.132:6791/0 conn(0x7fd0f01cb3f0 0x7fd0f01d0730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.115:51710/0 (socket says 172.21.15.115:51710) 2024-01-24T23:23:14.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.328+0000 7fd0eeffd640 1 -- 172.21.15.115:0/811303480 learned_addr learned my addr 172.21.15.115:0/811303480 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:14.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.328+0000 7fd0f5978640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:14.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.328+0000 7fd0f5978640 10 monclient(hunting): authenticate will time out at 1073.253687s 2024-01-24T23:23:14.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 1 -- 172.21.15.132:0/3915704007 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1500054739 0 0) 0x7fa48c00ffe0 con 0x7fa48c0037f0 2024-01-24T23:23:14.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 10 monclient(hunting): _negotiate 2024-01-24T23:23:14.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:14.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:14.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 10 monclient(hunting): my global_id is 4920 2024-01-24T23:23:14.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 1 -- 172.21.15.132:0/3915704007 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa494003530 con 0x7fa48c0037f0 2024-01-24T23:23:14.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 1 -- 172.21.15.132:0/3915704007 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2644503643 0 0) 0x7fa494003530 con 0x7fa48c0037f0 2024-01-24T23:23:14.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 1 -- 172.21.15.132:0/3915704007 >> v1:172.21.15.115:6793/0 conn(0x7fa48c007480 legacy=0x7fa48c009940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 1 -- 172.21.15.132:0/3915704007 >> v1:172.21.15.115:6790/0 conn(0x7fa48c00b210 legacy=0x7fa48c00d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 10 monclient: _finish_hunting 0 2024-01-24T23:23:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 1 monclient: found mon.noname-d 2024-01-24T23:23:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:23:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 1 -- 172.21.15.132:0/3915704007 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c012480 con 0x7fa48c0037f0 2024-01-24T23:23:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4aea26640 1 -- 172.21.15.132:0/3915704007 reap_dead start 2024-01-24T23:23:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 10 monclient: _finish_auth 0 2024-01-24T23:23:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 10 monclient: _check_auth_tickets 2024-01-24T23:23:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 1 -- 172.21.15.132:0/3915704007 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa4a4001800 con 0x7fa48c0037f0 2024-01-24T23:23:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:23:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 10 monclient: dump: 2024-01-24T23:23:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.344 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4b0489640 5 monclient: authenticate success, global_id 4920 2024-01-24T23:23:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4b0489640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 1 -- 172.21.15.132:0/3915704007 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa4a4001a20 con 0x7fa48c0037f0 2024-01-24T23:23:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4b0489640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 1 -- 172.21.15.132:0/3915704007 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa4a4003940 con 0x7fa48c0037f0 2024-01-24T23:23:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4b0489640 4 set_mon_vals no callback set 2024-01-24T23:23:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4ad223640 10 monclient: dump: 2024-01-24T23:23:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.346 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4b0489640 10 monclient: shutdown 2024-01-24T23:23:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4b0489640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4b0489640 1 -- 172.21.15.132:0/3915704007 >> v1:172.21.15.115:6792/0 conn(0x7fa48c0037f0 legacy=0x7fa48c005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4b0489640 1 -- 172.21.15.132:0/3915704007 shutdown_connections 2024-01-24T23:23:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4b0489640 1 -- 172.21.15.132:0/3915704007 >> 172.21.15.132:0/3915704007 conn(0x7fa4a80a5a10 msgr2=0x7fa4a80a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4b0489640 1 -- 172.21.15.132:0/3915704007 shutdown_connections 2024-01-24T23:23:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4b0489640 1 -- 172.21.15.132:0/3915704007 wait complete. 2024-01-24T23:23:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.324+0000 7fa4b0489640 10 monclient: build_initial_monmap 2024-01-24T23:23:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4b0489640 10 monclient: monmap: 2024-01-24T23:23:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:14.348 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:14.348 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:14.348 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:14.348 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:14.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4b0489640 1 Processor -- start 2024-01-24T23:23:14.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4b0489640 1 -- start start 2024-01-24T23:23:14.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4b0489640 10 monclient: init 2024-01-24T23:23:14.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4b0489640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:14.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4b0489640 10 monclient: _add_conns ranks=[0,2,4,3,8,6,5,1,7] 2024-01-24T23:23:14.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4b0489640 10 monclient(hunting): picked mon.noname-a con 0x7fa48c0037f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:14.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4b0489640 10 monclient(hunting): picked mon.noname-b con 0x7fa4a811eba0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:14.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4b0489640 10 monclient(hunting): picked mon.noname-c con 0x7fa4a81cc2d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:14.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4b0489640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a809ea70 con 0x7fa48c0037f0 2024-01-24T23:23:14.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4b0489640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a81217d0 con 0x7fa4a811eba0 2024-01-24T23:23:14.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4b0489640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a8120320 con 0x7fa4a81cc2d0 2024-01-24T23:23:14.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4b0489640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:14.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4b0489640 10 monclient(hunting): authenticate will time out at 1081.653619s 2024-01-24T23:23:14.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4ae225640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fa48c0037f0 0x7fa4a81cbbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:38212/0 (socket says 172.21.15.132:38212) 2024-01-24T23:23:14.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4ae225640 1 -- 172.21.15.132:0/539325784 learned_addr learned my addr 172.21.15.132:0/539325784 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4ada24640 1 -- 172.21.15.132:0/539325784 >> v1:172.21.15.115:6790/0 conn(0x7fa4a811eba0 legacy=0x7fa4a81d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-01-24T23:23:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4ada24640 1 -- 172.21.15.132:0/539325784 >> v1:172.21.15.115:6790/0 conn(0x7fa4a811eba0 legacy=0x7fa4a81d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:23:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4ada24640 1 --1- 172.21.15.132:0/539325784 >> v1:172.21.15.115:6790/0 conn(0x7fa4a811eba0 0x7fa4a81d0b70 :-1 s=CONNECTING_SEND_CONNECT_MSG pgs=0 cs=0 l=1).handle_connect_reply_1 read connect reply failed 2024-01-24T23:23:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4ada24640 1 --1- 172.21.15.132:0/539325784 >> v1:172.21.15.115:6790/0 conn(0x7fa4a811eba0 0x7fa4a81d0b70 :-1 s=CONNECTING_SEND_CONNECT_MSG pgs=0 cs=0 l=1).fault on lossy channel, failing 2024-01-24T23:23:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6790/0 2024-01-24T23:23:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 1 -- 172.21.15.132:0/539325784 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3095387267 0 0) 0x7fa4a809ea70 con 0x7fa48c0037f0 2024-01-24T23:23:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 10 monclient(hunting): my global_id is 4869 2024-01-24T23:23:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 1 -- 172.21.15.132:0/539325784 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa48c005420 con 0x7fa48c0037f0 2024-01-24T23:23:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 1 -- 172.21.15.132:0/539325784 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1153550736 0 0) 0x7fa48c005420 con 0x7fa48c0037f0 2024-01-24T23:23:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 1 -- 172.21.15.132:0/539325784 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa4a809ea70 con 0x7fa48c0037f0 2024-01-24T23:23:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 1 -- 172.21.15.132:0/539325784 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa4a4003940 con 0x7fa48c0037f0 2024-01-24T23:23:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:23:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 10 monclient(hunting): dump: 2024-01-24T23:23:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.352 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 1 -- 172.21.15.132:0/539325784 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2976719119 0 0) 0x7fa4a809ea70 con 0x7fa48c0037f0 2024-01-24T23:23:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 1 -- 172.21.15.132:0/539325784 >> v1:172.21.15.115:6791/0 conn(0x7fa4a81cc2d0 legacy=0x7fa4a81d42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 1 -- 172.21.15.132:0/539325784 >> v1:172.21.15.115:6790/0 conn(0x7fa4a811eba0 legacy=0x7fa4a81d0b70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 1 monclient: found mon.a 2024-01-24T23:23:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:23:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 1 -- 172.21.15.132:0/539325784 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a81cc740 con 0x7fa48c0037f0 2024-01-24T23:23:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 10 monclient: _finish_auth 0 2024-01-24T23:23:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4b0489640 5 monclient: authenticate success, global_id 4869 2024-01-24T23:23:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4b0489640 10 monclient: _renew_subs 2024-01-24T23:23:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4b0489640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:23:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4b0489640 1 -- 172.21.15.132:0/539325784 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a8121ab0 con 0x7fa48c0037f0 2024-01-24T23:23:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4b0489640 10 monclient: _renew_subs 2024-01-24T23:23:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4b0489640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:23:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa4b0489640 1 -- 172.21.15.132:0/539325784 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a81214f0 con 0x7fa48c0037f0 2024-01-24T23:23:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 1 -- 172.21.15.132:0/539325784 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa4a4001cd0 con 0x7fa48c0037f0 2024-01-24T23:23:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 1 -- 172.21.15.132:0/539325784 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa4a40041e0 con 0x7fa48c0037f0 2024-01-24T23:23:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49ffff640 4 set_mon_vals no callback set 2024-01-24T23:23:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:23:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.328+0000 7fa49effd640 10 monclient: dump: 2024-01-24T23:23:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.355 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.332+0000 7fa4b0489640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.332+0000 7fa4b0489640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.332+0000 7fa4b0489640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:23:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.332+0000 7fa4b0489640 1 -- 172.21.15.132:0/539325784 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa490005090 con 0x7fa48c0037f0 2024-01-24T23:23:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.332+0000 7fa49effd640 1 -- 172.21.15.132:0/539325784 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fa4a40100b0 con 0x7fa48c0037f0 2024-01-24T23:23:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.332+0000 7fa49effd640 1 -- 172.21.15.132:0/539325784 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(48..48 src has 1..48) v4 ==== 3994+0+0 (unknown 1916735388 0 0) 0x7fa4a4043a90 con 0x7fa48c0037f0 2024-01-24T23:23:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.332+0000 7fa4ada24640 10 monclient: get_auth_request con 0x7fa48c038540 auth_method 0 2024-01-24T23:23:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.336+0000 7fa49effd640 1 -- 172.21.15.132:0/539325784 <== mon.0 v1:172.21.15.115:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fa4a4014dd0 con 0x7fa48c0037f0 2024-01-24T23:23:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.336+0000 7fa49effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.336+0000 7fa49effd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:14.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.356+0000 7fd0ecff9640 1 -- 172.21.15.115:0/811303480 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654411409 0 0) 0x7fd0f009c8f0 con 0x7fd0f01d1e50 2024-01-24T23:23:14.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.356+0000 7fd0ecff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:14.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.356+0000 7fd0ecff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:14.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.356+0000 7fd0ecff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:14.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.356+0000 7fd0ecff9640 10 monclient(hunting): my global_id is 4933 2024-01-24T23:23:14.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.356+0000 7fd0ecff9640 1 -- 172.21.15.115:0/811303480 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0d0008fe0 con 0x7fd0f01d1e50 2024-01-24T23:23:14.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.360+0000 7fd0ecff9640 1 -- 172.21.15.115:0/811303480 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1640936480 0 0) 0x7fd0f01cc2e0 con 0x7fd0d0007440 2024-01-24T23:23:14.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.360+0000 7fd0ecff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:14.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.360+0000 7fd0ecff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:14.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.360+0000 7fd0ecff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:14.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.360+0000 7fd0ecff9640 10 monclient(hunting): my global_id is 4930 2024-01-24T23:23:14.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.360+0000 7fd0ecff9640 1 -- 172.21.15.115:0/811303480 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0f009c8f0 con 0x7fd0d0007440 2024-01-24T23:23:14.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.360+0000 7fd0ecff9640 1 -- 172.21.15.115:0/811303480 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2532864523 0 0) 0x7fd0f01204f0 con 0x7fd0f01cb3f0 2024-01-24T23:23:14.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.360+0000 7fd0ecff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:14.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.360+0000 7fd0ecff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:14.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.360+0000 7fd0ecff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:14.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.360+0000 7fd0ecff9640 10 monclient(hunting): my global_id is 4883 2024-01-24T23:23:14.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.360+0000 7fd0ecff9640 1 -- 172.21.15.115:0/811303480 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0f01cc2e0 con 0x7fd0f01cb3f0 2024-01-24T23:23:14.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.364+0000 7fd0ecff9640 1 -- 172.21.15.115:0/811303480 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 251450189 0 0) 0x7fd0d0008fe0 con 0x7fd0f01d1e50 2024-01-24T23:23:14.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.364+0000 7fd0ecff9640 1 -- 172.21.15.115:0/811303480 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd0f01204f0 con 0x7fd0f01d1e50 2024-01-24T23:23:14.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.364+0000 7fd0ecff9640 1 -- 172.21.15.115:0/811303480 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd0e8001b50 con 0x7fd0f01d1e50 2024-01-24T23:23:14.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.364+0000 7fd0ecff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.364+0000 7fd0ecff9640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:23:14.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.364+0000 7fd0ecff9640 10 monclient(hunting): dump: 2024-01-24T23:23:14.372 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:14.372 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.372 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.372 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.373 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.373 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:14.373 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.373 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.373 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.373 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.373 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.373 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.373 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.373 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.373 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.373 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:14.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.364+0000 7fd0ecff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:14.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.364+0000 7fd0ecff9640 1 -- 172.21.15.115:0/811303480 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 275080031 0 0) 0x7fd0f01204f0 con 0x7fd0f01d1e50 2024-01-24T23:23:14.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.364+0000 7fd0ecff9640 1 -- 172.21.15.115:0/811303480 >> v1:172.21.15.132:6792/0 conn(0x7fd0d0007440 legacy=0x7fd0f0132b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.364+0000 7fd0ecff9640 1 -- 172.21.15.115:0/811303480 >> v1:172.21.15.132:6791/0 conn(0x7fd0f01cb3f0 legacy=0x7fd0f01d0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.364+0000 7fd0ecff9640 10 monclient: _finish_hunting 0 2024-01-24T23:23:14.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.364+0000 7fd0ecff9640 1 monclient: found mon.f 2024-01-24T23:23:14.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.364+0000 7fd0ecff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:14.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.364+0000 7fd0ecff9640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:14.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.364+0000 7fd0ecff9640 1 -- 172.21.15.115:0/811303480 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f00a5960 con 0x7fd0f01d1e50 2024-01-24T23:23:14.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.364+0000 7fd0ecff9640 10 monclient: _finish_auth 0 2024-01-24T23:23:14.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.364+0000 7fd0ecff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:14.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.364+0000 7fd0ecff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:14.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.364+0000 7fd0f5978640 5 monclient: authenticate success, global_id 4933 2024-01-24T23:23:14.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.364+0000 7fd0f5978640 10 monclient: _renew_subs 2024-01-24T23:23:14.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.364+0000 7fd0f5978640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:14.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.364+0000 7fd0f5978640 1 -- 172.21.15.115:0/811303480 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f009ecb0 con 0x7fd0f01d1e50 2024-01-24T23:23:14.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.368+0000 7fd0f5978640 10 monclient: _renew_subs 2024-01-24T23:23:14.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.368+0000 7fd0f5978640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:14.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.368+0000 7fd0f5978640 1 -- 172.21.15.115:0/811303480 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f009f590 con 0x7fd0f01d1e50 2024-01-24T23:23:14.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.368+0000 7fd0ecff9640 1 -- 172.21.15.115:0/811303480 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd0e8001fe0 con 0x7fd0f01d1e50 2024-01-24T23:23:14.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.368+0000 7fd0ecff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:14.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.368+0000 7fd0ecff9640 1 -- 172.21.15.115:0/811303480 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd0e8004eb0 con 0x7fd0f01d1e50 2024-01-24T23:23:14.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.368+0000 7fd0ecff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.368+0000 7fd0ecff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:23:14.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.368+0000 7fd0ecff9640 10 monclient: dump: 2024-01-24T23:23:14.375 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:14.375 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.375 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.375 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.375 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.376 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:14.376 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.376 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.376 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.376 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.376 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.376 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.376 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.376 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.376 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.376 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:14.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.368+0000 7fd0edffb640 4 set_mon_vals no callback set 2024-01-24T23:23:14.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.368+0000 7fd0ecff9640 1 -- 172.21.15.115:0/811303480 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fd0e8005010 con 0x7fd0f01d1e50 2024-01-24T23:23:14.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.368+0000 7fd0f5978640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:14.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.368+0000 7fd0f5978640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:14.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.368+0000 7fd0ecff9640 1 -- 172.21.15.115:0/811303480 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(48..48 src has 1..48) v4 ==== 3994+0+0 (unknown 1916735388 0 0) 0x7fd0e8043700 con 0x7fd0f01d1e50 2024-01-24T23:23:14.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.368+0000 7fd0f5978640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:14.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.368+0000 7fd0f5978640 1 -- 172.21.15.115:0/811303480 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f01cb7d0 con 0x7fd0f01d1e50 2024-01-24T23:23:14.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.368+0000 7fd0eeffd640 10 monclient: get_auth_request con 0x7fd0d0038850 auth_method 0 2024-01-24T23:23:14.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.372+0000 7fd0ecff9640 1 -- 172.21.15.115:0/811303480 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fd0e8014600 con 0x7fd0f01d1e50 2024-01-24T23:23:14.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.372+0000 7fd0ecff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:14.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.372+0000 7fd0ecff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:14.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.500+0000 7fa4b0489640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-01-24T23:23:14.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.500+0000 7fa4b0489640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-01-24T23:23:14.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.500+0000 7fa4b0489640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:23:14.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.500+0000 7fa4b0489640 1 -- 172.21.15.132:0/539325784 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "ssd"} v 0) v1 -- 0x7fa4900058c0 con 0x7fa48c0037f0 2024-01-24T23:23:14.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.500+0000 7fa49effd640 1 -- 172.21.15.132:0/539325784 <== mon.0 v1:172.21.15.115:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "ssd"}]=0 v48) v1 ==== 90+0+8 (unknown 4243305861 0 905510949) 0x7fa4a4013240 con 0x7fa48c0037f0 2024-01-24T23:23:14.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.500+0000 7fa49effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-01-24T23:23:14.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.500+0000 7fa49effd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:14.588 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.588+0000 7fa4b0489640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-01-24T23:23:14.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.588+0000 7fa4b0489640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-01-24T23:23:14.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.588+0000 7fa4b0489640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:23:14.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.588+0000 7fa4b0489640 1 -- 172.21.15.132:0/539325784 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "ssd"} v 0) v1 -- 0x7fa490000bc0 con 0x7fa48c0037f0 2024-01-24T23:23:14.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.588+0000 7fa49effd640 1 -- 172.21.15.132:0/539325784 <== mon.0 v1:172.21.15.115:6789/0 11 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "ssd"}]=0 v48) v1 ==== 90+0+8 (unknown 4243305861 0 905510949) 0x7fa4a40149c0 con 0x7fa48c0037f0 2024-01-24T23:23:14.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.588+0000 7fa49effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-01-24T23:23:14.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.588+0000 7fa49effd640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:23:14.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.588+0000 7fa4b0489640 1 -- 172.21.15.132:0/539325784 >> v1:172.21.15.132:6812/1835401922 conn(0x7fa48c038540 legacy=0x7fa48c03aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.588+0000 7fa4b0489640 10 monclient: shutdown 2024-01-24T23:23:14.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.588+0000 7fa4b0489640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:14.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.588+0000 7fa4b0489640 1 -- 172.21.15.132:0/539325784 >> v1:172.21.15.115:6789/0 conn(0x7fa48c0037f0 legacy=0x7fa4a81cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.588+0000 7fa4b0489640 1 -- 172.21.15.132:0/539325784 shutdown_connections 2024-01-24T23:23:14.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.588+0000 7fa4b0489640 1 -- 172.21.15.132:0/539325784 >> 172.21.15.132:0/539325784 conn(0x7fa4a80a5a10 msgr2=0x7fa4a81343b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:14.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.592+0000 7fa4b0489640 1 -- 172.21.15.132:0/539325784 shutdown_connections 2024-01-24T23:23:14.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.592+0000 7fa4b0489640 1 -- 172.21.15.132:0/539325784 wait complete. 2024-01-24T23:23:14.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.588+0000 7fd0f5978640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-01-24T23:23:14.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.588+0000 7fd0f5978640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-01-24T23:23:14.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.588+0000 7fd0f5978640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:14.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.588+0000 7fd0f5978640 1 -- 172.21.15.115:0/811303480 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd0f0069c80 con 0x7fd0f01d1e50 2024-01-24T23:23:14.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.588+0000 7fd0ecff9640 1 -- 172.21.15.115:0/811303480 <== mon.1 v1:172.21.15.132:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2744 (unknown 254746657 0 1689528289) 0x7fd0e8012ca0 con 0x7fd0f01d1e50 2024-01-24T23:23:14.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.588+0000 7fd0ecff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-01-24T23:23:14.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.588+0000 7fd0ecff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:14.594 INFO:teuthology.orchestra.run.smithi115.stdout:{"election_epoch":16,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701272860131327","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","modified":"2024-01-24T23:20:20.557246Z","created":"2024-01-24T23:20:20.557246Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6790","nonce":0}]},"addr":"172.21.15.115:6790/0","public_addr":"172.21.15.115:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6790","nonce":0}]},"addr":"172.21.15.132:6790/0","public_addr":"172.21.15.132:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6791","nonce":0}]},"addr":"172.21.15.115:6791/0","public_addr":"172.21.15.115:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6791","nonce":0}]},"addr":"172.21.15.132:6791/0","public_addr":"172.21.15.132:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6792","nonce":0}]},"addr":"172.21.15.115:6792/0","public_addr":"172.21.15.115:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6792","nonce":0}]},"addr":"172.21.15.132:6792/0","public_addr":"172.21.15.132:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6793","nonce":0}]},"addr":"172.21.15.115:6793/0","public_addr":"172.21.15.115:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-01-24T23:23:14.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.592+0000 7fd0f5978640 1 -- 172.21.15.115:0/811303480 >> v1:172.21.15.132:6812/1835401922 conn(0x7fd0d0038850 legacy=0x7fd0d003ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.592+0000 7fd0f5978640 10 monclient: shutdown 2024-01-24T23:23:14.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.592+0000 7fd0f5978640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:14.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.592+0000 7fd0f5978640 1 -- 172.21.15.115:0/811303480 >> v1:172.21.15.132:6789/0 conn(0x7fd0f01d1e50 legacy=0x7fd0f01d4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.592+0000 7fd0f5978640 1 -- 172.21.15.115:0/811303480 shutdown_connections 2024-01-24T23:23:14.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.592+0000 7fd0f5978640 1 -- 172.21.15.115:0/811303480 >> 172.21.15.115:0/811303480 conn(0x7fd0f00a9910 msgr2=0x7fd0f006a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:14.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.592+0000 7fd0f5978640 1 -- 172.21.15.115:0/811303480 shutdown_connections 2024-01-24T23:23:14.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.592+0000 7fd0f5978640 1 -- 172.21.15.115:0/811303480 wait complete. 2024-01-24T23:23:14.604 INFO:tasks.workunit.client.0.smithi132.stdout:foo-ssd 2024-01-24T23:23:14.605 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule ls-by-class ssd 2024-01-24T23:23:14.605 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false grep foo-hdd 2024-01-24T23:23:14.605 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:23:14.605 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep foo-hdd 2024-01-24T23:23:14.612 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-01-24T23:23:14.612 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-01-24T23:23:14.612 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-01-24T23:23:14.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.716+0000 7faaacbed640 10 monclient: get_monmap_and_config 2024-01-24T23:23:14.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.716+0000 7faaacbed640 10 monclient: build_initial_monmap 2024-01-24T23:23:14.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.716+0000 7faaacbed640 10 monclient: monmap: 2024-01-24T23:23:14.725 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:14.725 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.725 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:14.719024+0000 2024-01-24T23:23:14.726 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:14.719024+0000 2024-01-24T23:23:14.726 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:14.726 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:14.726 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:14.726 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:14.726 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:14.726 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:14.726 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:14.726 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:14.726 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:14.726 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:14.726 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:14.726 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:14.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.716+0000 7faaacbed640 1 Processor -- start 2024-01-24T23:23:14.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.716+0000 7faaacbed640 1 -- start start 2024-01-24T23:23:14.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.716+0000 7faaacbed640 10 monclient: init 2024-01-24T23:23:14.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.716+0000 7faaacbed640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:14.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.716+0000 7faaacbed640 10 monclient: _add_conns ranks=[5,7,8,6,0,3,4,2,1] 2024-01-24T23:23:14.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.716+0000 7faaacbed640 10 monclient(hunting): picked mon.noname-h con 0x7faaa809ff20 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:14.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.716+0000 7faaacbed640 10 monclient(hunting): picked mon.noname-i con 0x7faaa809f9c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:14.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.716+0000 7faaacbed640 10 monclient(hunting): picked mon.noname-e con 0x7faaa813b2c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:14.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.716+0000 7faaacbed640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa8064700 con 0x7faaa809ff20 2024-01-24T23:23:14.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.716+0000 7faaacbed640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa809e760 con 0x7faaa809f9c0 2024-01-24T23:23:14.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.716+0000 7faaacbed640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa80a1160 con 0x7faaa813b2c0 2024-01-24T23:23:14.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.716+0000 7faaacbed640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:14.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.716+0000 7faaacbed640 10 monclient(hunting): authenticate will time out at 1082.043499s 2024-01-24T23:23:14.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.716+0000 7faaa5d9b640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7faaa809f9c0 0x7faaa8139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:40580/0 (socket says 172.21.15.132:40580) 2024-01-24T23:23:14.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.716+0000 7faaa5d9b640 1 -- 172.21.15.132:0/3230089872 learned_addr learned my addr 172.21.15.132:0/3230089872 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:14.728 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.716+0000 7faaa559a640 1 -- 172.21.15.132:0/3230089872 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1526556412 0 0) 0x7faaa809e760 con 0x7faaa809f9c0 2024-01-24T23:23:14.728 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.716+0000 7faaa559a640 10 monclient(hunting): _negotiate 2024-01-24T23:23:14.728 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.716+0000 7faaa559a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:14.728 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.716+0000 7faaa559a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:14.728 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.716+0000 7faaa559a640 10 monclient(hunting): my global_id is 4939 2024-01-24T23:23:14.728 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.716+0000 7faaa559a640 1 -- 172.21.15.132:0/3230089872 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa90003530 con 0x7faaa809f9c0 2024-01-24T23:23:14.728 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 1 -- 172.21.15.132:0/3230089872 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1057356928 0 0) 0x7faaa8064700 con 0x7faaa809ff20 2024-01-24T23:23:14.728 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 10 monclient(hunting): _negotiate 2024-01-24T23:23:14.728 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:14.728 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:14.728 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 10 monclient(hunting): my global_id is 4892 2024-01-24T23:23:14.728 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 1 -- 172.21.15.132:0/3230089872 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaa809e760 con 0x7faaa809ff20 2024-01-24T23:23:14.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 1 -- 172.21.15.132:0/3230089872 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2661424254 0 0) 0x7faa90003530 con 0x7faaa809f9c0 2024-01-24T23:23:14.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 1 -- 172.21.15.132:0/3230089872 >> v1:172.21.15.115:6793/0 conn(0x7faaa813b2c0 legacy=0x7faaa813d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 1 -- 172.21.15.132:0/3230089872 >> v1:172.21.15.132:6791/0 conn(0x7faaa809ff20 legacy=0x7faaa80a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 10 monclient: _finish_hunting 0 2024-01-24T23:23:14.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 1 monclient: found mon.noname-i 2024-01-24T23:23:14.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:14.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:23:14.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 1 -- 172.21.15.132:0/3230089872 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa80a0a10 con 0x7faaa809f9c0 2024-01-24T23:23:14.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 10 monclient: _finish_auth 0 2024-01-24T23:23:14.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 10 monclient: _check_auth_tickets 2024-01-24T23:23:14.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:14.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 5 monclient: authenticate success, global_id 4939 2024-01-24T23:23:14.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:14.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 1 -- 172.21.15.132:0/3230089872 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faa9c0017c0 con 0x7faaa809f9c0 2024-01-24T23:23:14.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.730 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:23:14.730 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 10 monclient: dump: 2024-01-24T23:23:14.730 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:14.730 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.730 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.730 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.730 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.730 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:14.730 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.730 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.730 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.730 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.730 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.730 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.730 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.731 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.731 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.731 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:14.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:14.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 1 -- 172.21.15.132:0/3230089872 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faa9c001a60 con 0x7faaa809f9c0 2024-01-24T23:23:14.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:14.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:14.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 4 set_mon_vals no callback set 2024-01-24T23:23:14.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 1 -- 172.21.15.132:0/3230089872 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faa9c004bf0 con 0x7faaa809f9c0 2024-01-24T23:23:14.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:14.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa559a640 10 monclient: dump: 2024-01-24T23:23:14.731 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:14.731 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.731 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.732 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.732 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.732 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:14.732 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.732 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.732 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.732 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.732 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.732 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.732 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.732 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.732 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.732 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:14.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 10 monclient: shutdown 2024-01-24T23:23:14.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:14.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 1 -- 172.21.15.132:0/3230089872 >> v1:172.21.15.132:6792/0 conn(0x7faaa809f9c0 legacy=0x7faaa8139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 1 -- 172.21.15.132:0/3230089872 shutdown_connections 2024-01-24T23:23:14.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 1 -- 172.21.15.132:0/3230089872 >> 172.21.15.132:0/3230089872 conn(0x7faaa80a5a10 msgr2=0x7faaa80a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:14.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 1 -- 172.21.15.132:0/3230089872 shutdown_connections 2024-01-24T23:23:14.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 1 -- 172.21.15.132:0/3230089872 wait complete. 2024-01-24T23:23:14.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 10 monclient: build_initial_monmap 2024-01-24T23:23:14.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 10 monclient: monmap: 2024-01-24T23:23:14.733 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:14.733 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:14.733 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:14.733 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:14.733 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:14.733 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:14.733 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:14.734 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:14.734 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:14.734 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:14.734 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:14.734 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:14.734 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:14.734 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:14.734 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:14.734 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:14.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 1 Processor -- start 2024-01-24T23:23:14.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 1 -- start start 2024-01-24T23:23:14.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 10 monclient: init 2024-01-24T23:23:14.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:14.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 10 monclient: _add_conns ranks=[4,1,2,7,0,3,8,6,5] 2024-01-24T23:23:14.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 10 monclient(hunting): picked mon.noname-c con 0x7faaa809f9c0 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:14.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 10 monclient(hunting): picked mon.noname-f con 0x7faaa809ff20 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:14.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 10 monclient(hunting): picked mon.noname-b con 0x7faaa813b2c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:14.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa809e040 con 0x7faaa809ff20 2024-01-24T23:23:14.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa8120ad0 con 0x7faaa813b2c0 2024-01-24T23:23:14.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa8071680 con 0x7faaa809f9c0 2024-01-24T23:23:14.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:14.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaacbed640 10 monclient(hunting): authenticate will time out at 1082.046576s 2024-01-24T23:23:14.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa5d9b640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7faaa809ff20 0x7faaa809cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:40858/0 (socket says 172.21.15.132:40858) 2024-01-24T23:23:14.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.720+0000 7faaa5d9b640 1 -- 172.21.15.132:0/396549338 learned_addr learned my addr 172.21.15.132:0/396549338 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:14.740 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.724+0000 7faa96ffd640 1 -- 172.21.15.132:0/396549338 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456409307 0 0) 0x7faaa809e040 con 0x7faaa809ff20 2024-01-24T23:23:14.740 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.724+0000 7faa96ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:14.740 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.724+0000 7faa96ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:14.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.724+0000 7faa96ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:14.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.724+0000 7faa96ffd640 10 monclient(hunting): my global_id is 4942 2024-01-24T23:23:14.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.724+0000 7faa96ffd640 1 -- 172.21.15.132:0/396549338 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa88003530 con 0x7faaa809ff20 2024-01-24T23:23:14.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.724+0000 7faa96ffd640 1 -- 172.21.15.132:0/396549338 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1637718133 0 0) 0x7faa88003530 con 0x7faaa809ff20 2024-01-24T23:23:14.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.724+0000 7faa96ffd640 1 -- 172.21.15.132:0/396549338 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faaa809e040 con 0x7faaa809ff20 2024-01-24T23:23:14.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.724+0000 7faa96ffd640 1 -- 172.21.15.132:0/396549338 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faa9c004b30 con 0x7faaa809ff20 2024-01-24T23:23:14.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.724+0000 7faa96ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.724+0000 7faa96ffd640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:23:14.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.724+0000 7faa96ffd640 10 monclient(hunting): dump: 2024-01-24T23:23:14.741 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:14.741 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.741 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.741 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.741 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.742 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:14.742 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.742 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.742 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.742 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.742 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.742 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.742 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.742 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.743 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.743 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:14.743 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.724+0000 7faa96ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:14.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.724+0000 7faa96ffd640 1 -- 172.21.15.132:0/396549338 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3066238472 0 0) 0x7faaa809e040 con 0x7faaa809ff20 2024-01-24T23:23:14.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.724+0000 7faa96ffd640 1 -- 172.21.15.132:0/396549338 >> v1:172.21.15.115:6791/0 conn(0x7faaa809f9c0 legacy=0x7faaa809c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.724+0000 7faa96ffd640 1 -- 172.21.15.132:0/396549338 >> v1:172.21.15.115:6790/0 conn(0x7faaa813b2c0 legacy=0x7faaa81d38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.724+0000 7faa96ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:14.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.724+0000 7faa96ffd640 1 monclient: found mon.f 2024-01-24T23:23:14.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faa96ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:14.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faa96ffd640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:14.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faa96ffd640 1 -- 172.21.15.132:0/396549338 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa809d2e0 con 0x7faaa809ff20 2024-01-24T23:23:14.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faa96ffd640 10 monclient: _finish_auth 0 2024-01-24T23:23:14.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faa96ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:14.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faa96ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:14.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faaacbed640 5 monclient: authenticate success, global_id 4942 2024-01-24T23:23:14.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faaacbed640 10 monclient: _renew_subs 2024-01-24T23:23:14.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faaacbed640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:14.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faaacbed640 1 -- 172.21.15.132:0/396549338 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa8071960 con 0x7faaa809ff20 2024-01-24T23:23:14.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faaacbed640 10 monclient: _renew_subs 2024-01-24T23:23:14.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faaacbed640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:14.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faaacbed640 1 -- 172.21.15.132:0/396549338 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa8120db0 con 0x7faaa809ff20 2024-01-24T23:23:14.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faa96ffd640 1 -- 172.21.15.132:0/396549338 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faa9c001180 con 0x7faaa809ff20 2024-01-24T23:23:14.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faa96ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:14.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faaa4d99640 4 set_mon_vals no callback set 2024-01-24T23:23:14.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faa96ffd640 1 -- 172.21.15.132:0/396549338 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faa9c005240 con 0x7faaa809ff20 2024-01-24T23:23:14.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faa96ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faa96ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:23:14.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faa96ffd640 10 monclient: dump: 2024-01-24T23:23:14.745 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:14.745 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.745 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.745 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.746 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.746 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:14.746 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.746 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.746 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.746 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.746 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.746 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.746 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.746 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.746 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.746 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:14.746 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faaacbed640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:14.746 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faa96ffd640 1 -- 172.21.15.132:0/396549338 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7faa9c00fcc0 con 0x7faaa809ff20 2024-01-24T23:23:14.746 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faaacbed640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:14.746 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faaacbed640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:14.747 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faaacbed640 1 -- 172.21.15.132:0/396549338 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa8c005090 con 0x7faaa809ff20 2024-01-24T23:23:14.747 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faa96ffd640 1 -- 172.21.15.132:0/396549338 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(48..48 src has 1..48) v4 ==== 3994+0+0 (unknown 1916735388 0 0) 0x7faa9c043c20 con 0x7faaa809ff20 2024-01-24T23:23:14.747 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.728+0000 7faaa659c640 10 monclient: get_auth_request con 0x7faa88036fe0 auth_method 0 2024-01-24T23:23:14.747 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.732+0000 7faa96ffd640 1 -- 172.21.15.132:0/396549338 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7faa9c014c40 con 0x7faaa809ff20 2024-01-24T23:23:14.747 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.732+0000 7faa96ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:14.747 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.732+0000 7faa96ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:14.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64e5ac5640 10 monclient: get_monmap_and_config 2024-01-24T23:23:14.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64e5ac5640 10 monclient: build_initial_monmap 2024-01-24T23:23:14.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64e5ac5640 10 monclient: monmap: 2024-01-24T23:23:14.754 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:14.754 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.754 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:14.733002+0000 2024-01-24T23:23:14.754 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:14.733002+0000 2024-01-24T23:23:14.754 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:14.754 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:14.754 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:14.754 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:14.754 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:14.754 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:14.755 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:14.755 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:14.755 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:14.755 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:14.755 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:14.755 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:14.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64e5ac5640 1 Processor -- start 2024-01-24T23:23:14.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64e5ac5640 1 -- start start 2024-01-24T23:23:14.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64e5ac5640 10 monclient: init 2024-01-24T23:23:14.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64e5ac5640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:14.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64e5ac5640 10 monclient: _add_conns ranks=[5,2,3,4,0,8,1,7,6] 2024-01-24T23:23:14.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64e5ac5640 10 monclient(hunting): picked mon.noname-h con 0x7f64e00a2ad0 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:14.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64e5ac5640 10 monclient(hunting): picked mon.noname-b con 0x7f64e0136e70 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:14.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64e5ac5640 10 monclient(hunting): picked mon.noname-g con 0x7f64e013d3f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:14.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64e5ac5640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e004ed70 con 0x7f64e0136e70 2024-01-24T23:23:14.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64e5ac5640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e00a0680 con 0x7f64e013d3f0 2024-01-24T23:23:14.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64e5ac5640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e00a35c0 con 0x7f64e00a2ad0 2024-01-24T23:23:14.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64e5ac5640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:14.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64e5ac5640 10 monclient(hunting): authenticate will time out at 1073.655293s 2024-01-24T23:23:14.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64deffd640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f64e0136e70 0x7f64e013bcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:47552/0 (socket says 172.21.15.115:47552) 2024-01-24T23:23:14.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64deffd640 1 -- 172.21.15.115:0/1267898756 learned_addr learned my addr 172.21.15.115:0/1267898756 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:14.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 1 -- 172.21.15.115:0/1267898756 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743008698 0 0) 0x7f64e004ed70 con 0x7f64e0136e70 2024-01-24T23:23:14.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:14.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:14.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:14.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 10 monclient(hunting): my global_id is 84107 2024-01-24T23:23:14.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 1 -- 172.21.15.115:0/1267898756 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64c8003530 con 0x7f64e0136e70 2024-01-24T23:23:14.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 1 -- 172.21.15.115:0/1267898756 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3408217840 0 0) 0x7f64d80013d0 con 0x7f64e013d3f0 2024-01-24T23:23:14.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:14.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:14.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:14.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 10 monclient(hunting): my global_id is 4836 2024-01-24T23:23:14.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 1 -- 172.21.15.115:0/1267898756 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64e004ed70 con 0x7f64e013d3f0 2024-01-24T23:23:14.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 1 -- 172.21.15.115:0/1267898756 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2406362450 0 0) 0x7f64e00a35c0 con 0x7f64e00a2ad0 2024-01-24T23:23:14.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:14.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:14.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:14.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 10 monclient(hunting): my global_id is 4901 2024-01-24T23:23:14.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 1 -- 172.21.15.115:0/1267898756 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64d80013d0 con 0x7f64e00a2ad0 2024-01-24T23:23:14.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 1 -- 172.21.15.115:0/1267898756 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3158344199 0 0) 0x7f64c8003530 con 0x7f64e0136e70 2024-01-24T23:23:14.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 1 -- 172.21.15.115:0/1267898756 >> v1:172.21.15.132:6791/0 conn(0x7f64e00a2ad0 legacy=0x7f64e00a2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 1 -- 172.21.15.115:0/1267898756 >> v1:172.21.15.132:6790/0 conn(0x7f64e013d3f0 legacy=0x7f64e013f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:23:14.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 1 monclient: found mon.noname-b 2024-01-24T23:23:14.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:14.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:23:14.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 1 -- 172.21.15.115:0/1267898756 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e00a4db0 con 0x7f64e0136e70 2024-01-24T23:23:14.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 10 monclient: _finish_auth 0 2024-01-24T23:23:14.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:23:14.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:14.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64e5ac5640 5 monclient: authenticate success, global_id 84107 2024-01-24T23:23:14.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 1 -- 172.21.15.115:0/1267898756 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f64d00017c0 con 0x7f64e0136e70 2024-01-24T23:23:14.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.728+0000 7f64de7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64de7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:23:14.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64de7fc640 10 monclient: dump: 2024-01-24T23:23:14.758 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:14.758 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.758 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.759 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.759 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.759 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:14.759 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.759 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.759 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.759 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.759 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.759 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.759 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.759 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.759 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.759 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:14.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:14.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64de7fc640 1 -- 172.21.15.115:0/1267898756 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f64d0001aa0 con 0x7f64e0136e70 2024-01-24T23:23:14.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64de7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:14.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64de7fc640 1 -- 172.21.15.115:0/1267898756 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f64d0004bf0 con 0x7f64e0136e70 2024-01-24T23:23:14.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:14.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 4 set_mon_vals no callback set 2024-01-24T23:23:14.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64de7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64de7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:14.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64de7fc640 10 monclient: dump: 2024-01-24T23:23:14.760 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:14.760 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.760 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.760 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.760 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.760 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:14.760 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.760 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.761 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.761 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.761 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.761 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.761 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.761 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.761 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.761 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:14.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 10 monclient: shutdown 2024-01-24T23:23:14.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:14.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 1 -- 172.21.15.115:0/1267898756 >> v1:172.21.15.115:6790/0 conn(0x7f64e0136e70 legacy=0x7f64e013bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 1 -- 172.21.15.115:0/1267898756 shutdown_connections 2024-01-24T23:23:14.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 1 -- 172.21.15.115:0/1267898756 >> 172.21.15.115:0/1267898756 conn(0x7f64e00a9870 msgr2=0x7f64e00a5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:14.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 1 -- 172.21.15.115:0/1267898756 shutdown_connections 2024-01-24T23:23:14.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 1 -- 172.21.15.115:0/1267898756 wait complete. 2024-01-24T23:23:14.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 10 monclient: build_initial_monmap 2024-01-24T23:23:14.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 10 monclient: monmap: 2024-01-24T23:23:14.762 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:14.762 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:14.762 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:14.762 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:14.762 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:14.762 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:14.762 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:14.762 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:14.762 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:14.762 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:14.762 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:14.762 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:14.762 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:14.763 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:14.763 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:14.763 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:14.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 1 Processor -- start 2024-01-24T23:23:14.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 1 -- start start 2024-01-24T23:23:14.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 10 monclient: init 2024-01-24T23:23:14.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:14.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 10 monclient: _add_conns ranks=[1,0,7,6,3,4,5,8,2] 2024-01-24T23:23:14.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 10 monclient(hunting): picked mon.noname-f con 0x7f64e00a2ad0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:14.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 10 monclient(hunting): picked mon.noname-a con 0x7f64e0136e70 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:14.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 10 monclient(hunting): picked mon.noname-i con 0x7f64e013d3f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:14.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e009e9f0 con 0x7f64e0136e70 2024-01-24T23:23:14.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e009ef20 con 0x7f64e00a2ad0 2024-01-24T23:23:14.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e009f200 con 0x7f64e013d3f0 2024-01-24T23:23:14.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:14.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 10 monclient(hunting): authenticate will time out at 1073.658215s 2024-01-24T23:23:14.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64deffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f64e0136e70 0x7f64e01d0890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:60908/0 (socket says 172.21.15.115:60908) 2024-01-24T23:23:14.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64deffd640 1 -- 172.21.15.115:0/1538070236 learned_addr learned my addr 172.21.15.115:0/1538070236 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:14.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331999582 0 0) 0x7f64e009e9f0 con 0x7f64e0136e70 2024-01-24T23:23:14.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:14.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:14.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:14.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient(hunting): my global_id is 4878 2024-01-24T23:23:14.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64cc003560 con 0x7f64e0136e70 2024-01-24T23:23:14.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1769520669 0 0) 0x7f64e009ef20 con 0x7f64e00a2ad0 2024-01-24T23:23:14.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:14.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:14.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:14.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient(hunting): my global_id is 4951 2024-01-24T23:23:14.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64e009e9f0 con 0x7f64e00a2ad0 2024-01-24T23:23:14.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3366876283 0 0) 0x7f64e009f200 con 0x7f64e013d3f0 2024-01-24T23:23:14.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:14.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:14.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:14.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient(hunting): my global_id is 4948 2024-01-24T23:23:14.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64e009ef20 con 0x7f64e013d3f0 2024-01-24T23:23:14.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 738358190 0 0) 0x7f64cc003560 con 0x7f64e0136e70 2024-01-24T23:23:14.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f64e009f200 con 0x7f64e0136e70 2024-01-24T23:23:14.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f64d0004b30 con 0x7f64e0136e70 2024-01-24T23:23:14.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:23:14.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient(hunting): dump: 2024-01-24T23:23:14.765 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:14.765 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.765 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.765 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.766 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.766 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:14.766 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.766 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.766 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.766 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.766 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.766 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.766 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.766 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.766 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.766 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:14.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:14.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3204508041 0 0) 0x7f64e009e9f0 con 0x7f64e00a2ad0 2024-01-24T23:23:14.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f64cc003560 con 0x7f64e00a2ad0 2024-01-24T23:23:14.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f64d4001900 con 0x7f64e00a2ad0 2024-01-24T23:23:14.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:23:14.767 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient(hunting): dump: 2024-01-24T23:23:14.767 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:14.767 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.767 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.767 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.767 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.767 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:14.767 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.767 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.767 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.767 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.767 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.768 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.768 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.768 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.768 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.768 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:14.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1814194621 0 0) 0x7f64e009f200 con 0x7f64e0136e70 2024-01-24T23:23:14.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 >> v1:172.21.15.132:6792/0 conn(0x7f64e013d3f0 legacy=0x7f64e01d3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 >> v1:172.21.15.132:6789/0 conn(0x7f64e00a2ad0 legacy=0x7f64e01cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient: _finish_hunting 0 2024-01-24T23:23:14.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 1 monclient: found mon.a 2024-01-24T23:23:14.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:14.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:23:14.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e00a55a0 con 0x7f64e0136e70 2024-01-24T23:23:14.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient: _finish_auth 0 2024-01-24T23:23:14.768 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient: _check_auth_tickets 2024-01-24T23:23:14.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:14.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 5 monclient: authenticate success, global_id 4878 2024-01-24T23:23:14.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 10 monclient: _renew_subs 2024-01-24T23:23:14.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:23:14.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 1 -- 172.21.15.115:0/1538070236 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e013cbb0 con 0x7f64e0136e70 2024-01-24T23:23:14.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 10 monclient: _renew_subs 2024-01-24T23:23:14.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:23:14.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 1 -- 172.21.15.115:0/1538070236 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e009dc90 con 0x7f64e0136e70 2024-01-24T23:23:14.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f64d0001860 con 0x7f64e0136e70 2024-01-24T23:23:14.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:14.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f64d0005200 con 0x7f64e0136e70 2024-01-24T23:23:14.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64dd7fa640 4 set_mon_vals no callback set 2024-01-24T23:23:14.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:23:14.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64c7fff640 10 monclient: dump: 2024-01-24T23:23:14.769 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:14.770 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.770 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.770 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.770 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.770 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:14.770 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.770 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.770 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.770 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.770 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.770 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.770 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.770 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.770 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.770 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:14.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:14.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.732+0000 7f64e5ac5640 1 -- 172.21.15.115:0/1538070236 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f64c00065d0 con 0x7f64c0002870 2024-01-24T23:23:14.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.736+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f64d000fc80 con 0x7f64e0136e70 2024-01-24T23:23:14.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.736+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(48..48 src has 1..48) v4 ==== 3994+0+0 (unknown 1916735388 0 0) 0x7f64d0043a70 con 0x7f64e0136e70 2024-01-24T23:23:14.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.736+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3193935225 0 0) 0x7f64c00065d0 con 0x7f64c0002870 2024-01-24T23:23:14.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.736+0000 7f64c7fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:14.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.736+0000 7f64c7fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:14.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.736+0000 7f64c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:14.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.736+0000 7f64c7fff640 10 monclient(hunting): my global_id is 4887 2024-01-24T23:23:14.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.736+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64e009f200 con 0x7f64c0002870 2024-01-24T23:23:14.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.736+0000 7f64dffff640 10 monclient: get_auth_request con 0x7f64cc037870 auth_method 0 2024-01-24T23:23:14.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.736+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 308577631 0 0) 0x7f64e009f200 con 0x7f64c0002870 2024-01-24T23:23:14.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.736+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f64c00065d0 con 0x7f64c0002870 2024-01-24T23:23:14.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.736+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f64d4003960 con 0x7f64c0002870 2024-01-24T23:23:14.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.736+0000 7f64c7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.736+0000 7f64c7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:23:14.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.736+0000 7f64c7fff640 10 monclient: dump: 2024-01-24T23:23:14.772 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:14.772 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.772 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.772 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.772 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.772 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:14.772 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.772 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.772 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.772 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.772 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.772 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.772 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.772 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.773 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.736+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1677110725 0 0) 0x7f64c00065d0 con 0x7f64c0002870 2024-01-24T23:23:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.736+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 --> v1:172.21.15.115:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f64c000a120 con 0x7f64c0002870 2024-01-24T23:23:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.736+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 <== mon.0 v1:172.21.15.115:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f64d4003df0 con 0x7f64c0002870 2024-01-24T23:23:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.736+0000 7f64c7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.736+0000 7f64c7fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.736+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 >> v1:172.21.15.115:6789/0 conn(0x7f64c0002870 legacy=0x7f64c0004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.736+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 >> v1:172.21.15.115:6789/0 conn(0x7f64c0002870 legacy=0x7f64c0004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.740+0000 7f64e5ac5640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:23:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.740+0000 7f64e5ac5640 1 -- 172.21.15.115:0/1538070236 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f64c0009e10 con 0x7f64c00090d0 2024-01-24T23:23:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.740+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 527099842 0 0) 0x7f64c0009e10 con 0x7f64c00090d0 2024-01-24T23:23:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.740+0000 7f64c7fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.740+0000 7f64c7fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.740+0000 7f64c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:14.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.740+0000 7f64c7fff640 10 monclient(hunting): my global_id is 4896 2024-01-24T23:23:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.740+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64c00065d0 con 0x7f64c00090d0 2024-01-24T23:23:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.740+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 261226054 0 0) 0x7f64c00065d0 con 0x7f64c00090d0 2024-01-24T23:23:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.740+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f64c0009e10 con 0x7f64c00090d0 2024-01-24T23:23:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.740+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f64d40044d0 con 0x7f64c00090d0 2024-01-24T23:23:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.740+0000 7f64c7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.740+0000 7f64c7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:23:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.740+0000 7f64c7fff640 10 monclient: dump: 2024-01-24T23:23:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.774 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.775 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.775 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.775 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.775 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.775 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.775 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.775 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.775 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:14.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.740+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1485306322 0 0) 0x7f64c0009e10 con 0x7f64c00090d0 2024-01-24T23:23:14.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.740+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 --> v1:172.21.15.115:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f64c000c310 con 0x7f64c00090d0 2024-01-24T23:23:14.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.744+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 <== mon.0 v1:172.21.15.115:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7715 (unknown 0 0 1671319743) 0x7f64d4006840 con 0x7f64c00090d0 2024-01-24T23:23:14.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.744+0000 7f64c7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:23:14.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.744+0000 7f64c7fff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:14.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.744+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 >> v1:172.21.15.115:6789/0 conn(0x7f64c00090d0 legacy=0x7f64c0009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.744+0000 7f64c7fff640 1 -- 172.21.15.115:0/1538070236 >> v1:172.21.15.115:6789/0 conn(0x7f64c00090d0 legacy=0x7f64c0009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:14.776 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:23:14.776 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:14.776 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:14.776 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "leader", 2024-01-24T23:23:14.776 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 16, 2024-01-24T23:23:14.776 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:23:14.776 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:23:14.776 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:23:14.776 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:23:14.776 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:23:14.777 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:23:14.777 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:23:14.777 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:23:14.777 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:23:14.777 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:23:14.777 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:14.777 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:23:14.777 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:14.777 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:23:14.777 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:23:14.777 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:14.777 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:14.777 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:14.777 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:14.777 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:14.778 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:14.778 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:14.778 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:14.778 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:14.778 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:14.778 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:14.778 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:14.778 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:23:14.778 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:23:14.778 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:14.778 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:14.778 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:14.778 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:14.778 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:14.779 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:14.779 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:14.779 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:14.779 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:14.779 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:14.779 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:14.779 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:14.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.779 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:23:14.779 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:23:14.779 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:23:14.780 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:23:14.780 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:23:14.780 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:23:14.780 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:14.780 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:14.780 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:23:14.780 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:23:14.780 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:23:14.780 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:23:14.780 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:23:14.780 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:23:14.780 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:23:14.780 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:14.780 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:23:14.781 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:14.781 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:14.781 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:14.781 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:14.781 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:14.781 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:14.781 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:14.781 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:14.781 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:14.781 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:14.781 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:14.781 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:14.781 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:23:14.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.781 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:23:14.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.782 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:14.782 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:14.782 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:14.782 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:14.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.782 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:14.782 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:23:14.782 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:14.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.782 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:14.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.782 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:23:14.782 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:23:14.783 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:14.783 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:14.783 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:14.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.783 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:14.783 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:14.783 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:14.783 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:14.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.783 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:14.783 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:23:14.783 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:14.783 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.783 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:14.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.784 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:23:14.784 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:23:14.784 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:14.784 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:14.784 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:14.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.784 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:14.784 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:14.784 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:14.784 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:14.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.784 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:14.784 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:23:14.785 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:14.785 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.785 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:14.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.785 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:23:14.785 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:23:14.785 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:14.785 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:14.785 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:14.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.785 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:14.785 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:14.785 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:14.786 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:14.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.786 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:14.786 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:23:14.786 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:14.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.786 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:14.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.786 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:23:14.786 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:23:14.786 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:14.786 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:14.786 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:14.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.787 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:14.787 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:14.787 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:14.787 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:14.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.787 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:14.787 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:23:14.787 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:14.787 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.787 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:14.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.787 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:23:14.787 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:23:14.787 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:14.787 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:14.788 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:14.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.788 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:14.788 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:14.788 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:14.788 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:14.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.788 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:14.788 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:23:14.788 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:14.788 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.788 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:14.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.789 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:23:14.789 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:23:14.789 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:14.789 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:14.789 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:14.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.789 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:14.789 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:14.789 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:14.789 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:14.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.789 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:14.789 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:23:14.789 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:14.790 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.790 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:14.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.790 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:23:14.790 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:23:14.790 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:14.790 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:14.790 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:14.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.790 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:14.790 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:14.790 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:14.790 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:14.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.791 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:14.791 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:23:14.791 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:14.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.791 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:14.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.791 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:23:14.791 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:23:14.791 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:14.791 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:14.791 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:14.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.791 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:14.791 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:14.792 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:14.792 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:14.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.792 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:14.792 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:23:14.792 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:14.792 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.792 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:14.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.792 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:23:14.792 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:23:14.792 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:14.792 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:14.792 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:14.793 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.793 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:14.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.793 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:23:14.793 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:23:14.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.793 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:14.793 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:14.793 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:14.793 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.793 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:14.793 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:23:14.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.793 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:14.793 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:14.794 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:14.794 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.794 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:14.794 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:23:14.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.794 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:14.794 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:14.794 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 3 2024-01-24T23:23:14.794 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.794 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:14.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.794 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:23:14.794 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:23:14.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.744+0000 7f64e5ac5640 1 -- 172.21.15.115:0/1538070236 >> v1:172.21.15.132:6812/1835401922 conn(0x7f64cc037870 legacy=0x7f64cc039d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.744+0000 7f64e5ac5640 10 monclient: shutdown 2024-01-24T23:23:14.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.744+0000 7f64e5ac5640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:14.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.744+0000 7f64e5ac5640 1 -- 172.21.15.115:0/1538070236 >> v1:172.21.15.115:6789/0 conn(0x7f64e0136e70 legacy=0x7f64e01d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.744+0000 7f64dffff640 1 -- 172.21.15.115:0/1538070236 reap_dead start 2024-01-24T23:23:14.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.744+0000 7f64dffff640 1 -- 172.21.15.115:0/1538070236 reap_dead start 2024-01-24T23:23:14.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.744+0000 7f64e5ac5640 1 -- 172.21.15.115:0/1538070236 shutdown_connections 2024-01-24T23:23:14.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.744+0000 7f64e5ac5640 1 -- 172.21.15.115:0/1538070236 >> 172.21.15.115:0/1538070236 conn(0x7f64e00a9870 msgr2=0x7f64e0132e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:14.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.744+0000 7f64e5ac5640 1 -- 172.21.15.115:0/1538070236 shutdown_connections 2024-01-24T23:23:14.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.744+0000 7f64e5ac5640 1 -- 172.21.15.115:0/1538070236 wait complete. 2024-01-24T23:23:14.797 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:23:14.797 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-01-24T23:23:14.797 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-01-24T23:23:14.887 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.884+0000 7faaacbed640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-01-24T23:23:14.887 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.884+0000 7faaacbed640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-01-24T23:23:14.887 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.884+0000 7faaacbed640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:14.887 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.884+0000 7faaacbed640 1 -- 172.21.15.132:0/396549338 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "ssd"} v 0) v1 -- 0x7faa8c0058c0 con 0x7faaa809ff20 2024-01-24T23:23:14.888 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.888+0000 7faa96ffd640 1 -- 172.21.15.132:0/396549338 <== mon.1 v1:172.21.15.132:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "ssd"}]=0 v48) v1 ==== 90+0+8 (unknown 4243305861 0 905510949) 0x7faa9c0132e0 con 0x7faaa809ff20 2024-01-24T23:23:14.888 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.888+0000 7faa96ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-01-24T23:23:14.888 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.888+0000 7faa96ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:14.914 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 10 monclient: get_monmap_and_config 2024-01-24T23:23:14.914 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 10 monclient: build_initial_monmap 2024-01-24T23:23:14.914 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 10 monclient: monmap: 2024-01-24T23:23:14.914 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:14.914 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.914 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:14.915092+0000 2024-01-24T23:23:14.915 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:14.915092+0000 2024-01-24T23:23:14.915 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:14.915 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:14.915 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:14.915 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:14.915 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:14.915 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:14.915 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:14.915 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:14.915 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:14.915 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:14.915 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:14.915 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:14.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 1 Processor -- start 2024-01-24T23:23:14.923 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 1 -- start start 2024-01-24T23:23:14.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 10 monclient: init 2024-01-24T23:23:14.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:14.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 10 monclient: _add_conns ranks=[3,0,7,1,4,6,8,2,5] 2024-01-24T23:23:14.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 10 monclient(hunting): picked mon.noname-g con 0x7fed54131810 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:14.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 10 monclient(hunting): picked mon.noname-a con 0x7fed541311d0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:14.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 10 monclient(hunting): picked mon.noname-i con 0x7fed5406d960 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:14.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed5404ed70 con 0x7fed541311d0 2024-01-24T23:23:14.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed5406e1f0 con 0x7fed54131810 2024-01-24T23:23:14.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed5409fef0 con 0x7fed5406d960 2024-01-24T23:23:14.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:14.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 10 monclient(hunting): authenticate will time out at 1073.837696s 2024-01-24T23:23:14.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed51d9b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fed541311d0 0x7fed540a4d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:60918/0 (socket says 172.21.15.115:60918) 2024-01-24T23:23:14.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed51d9b640 1 -- 172.21.15.115:0/1594411562 learned_addr learned my addr 172.21.15.115:0/1594411562 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:14.924 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 1 -- 172.21.15.115:0/1594411562 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 184217659 0 0) 0x7fed5404ed70 con 0x7fed541311d0 2024-01-24T23:23:14.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 10 monclient(hunting): _negotiate 2024-01-24T23:23:14.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:14.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:14.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 10 monclient(hunting): my global_id is 4905 2024-01-24T23:23:14.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 1 -- 172.21.15.115:0/1594411562 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed34003530 con 0x7fed541311d0 2024-01-24T23:23:14.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 1 -- 172.21.15.115:0/1594411562 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1661292721 0 0) 0x7fed5406e1f0 con 0x7fed54131810 2024-01-24T23:23:14.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 10 monclient(hunting): _negotiate 2024-01-24T23:23:14.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:14.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:14.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 10 monclient(hunting): my global_id is 4845 2024-01-24T23:23:14.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 1 -- 172.21.15.115:0/1594411562 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed5404ed70 con 0x7fed54131810 2024-01-24T23:23:14.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 1 -- 172.21.15.115:0/1594411562 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2632665758 0 0) 0x7fed34003530 con 0x7fed541311d0 2024-01-24T23:23:14.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 1 -- 172.21.15.115:0/1594411562 >> v1:172.21.15.132:6792/0 conn(0x7fed5406d960 legacy=0x7fed5413da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 1 -- 172.21.15.115:0/1594411562 >> v1:172.21.15.132:6790/0 conn(0x7fed54131810 legacy=0x7fed540a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.925 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 10 monclient: _finish_hunting 0 2024-01-24T23:23:14.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 1 monclient: found mon.noname-a 2024-01-24T23:23:14.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:14.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:23:14.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 1 -- 172.21.15.115:0/1594411562 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed540a54b0 con 0x7fed541311d0 2024-01-24T23:23:14.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 10 monclient: _finish_auth 0 2024-01-24T23:23:14.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 10 monclient: _check_auth_tickets 2024-01-24T23:23:14.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:14.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 5 monclient: authenticate success, global_id 4905 2024-01-24T23:23:14.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 1 -- 172.21.15.115:0/1594411562 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fed3c0017c0 con 0x7fed541311d0 2024-01-24T23:23:14.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:14.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:23:14.926 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 10 monclient: dump: 2024-01-24T23:23:14.926 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:14.926 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.927 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.927 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.927 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.927 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:14.927 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.927 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.927 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.927 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.927 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.927 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.927 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.928 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.928 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.928 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:14.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:14.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 1 -- 172.21.15.115:0/1594411562 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fed3c001a20 con 0x7fed541311d0 2024-01-24T23:23:14.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:14.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 1 -- 172.21.15.115:0/1594411562 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fed3c004bf0 con 0x7fed541311d0 2024-01-24T23:23:14.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:23:14.928 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed5159a640 10 monclient: dump: 2024-01-24T23:23:14.928 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:14.928 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.928 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.928 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.929 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.929 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:14.929 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.929 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.929 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.929 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.929 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.929 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.929 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.929 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.929 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.929 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:14.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:14.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 4 set_mon_vals no callback set 2024-01-24T23:23:14.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 10 monclient: shutdown 2024-01-24T23:23:14.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:14.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 1 -- 172.21.15.115:0/1594411562 >> v1:172.21.15.115:6789/0 conn(0x7fed541311d0 legacy=0x7fed540a4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 1 -- 172.21.15.115:0/1594411562 shutdown_connections 2024-01-24T23:23:14.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 1 -- 172.21.15.115:0/1594411562 >> 172.21.15.115:0/1594411562 conn(0x7fed540a5d40 msgr2=0x7fed540a2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:14.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 1 -- 172.21.15.115:0/1594411562 shutdown_connections 2024-01-24T23:23:14.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 1 -- 172.21.15.115:0/1594411562 wait complete. 2024-01-24T23:23:14.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 10 monclient: build_initial_monmap 2024-01-24T23:23:14.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.912+0000 7fed589c6640 10 monclient: monmap: 2024-01-24T23:23:14.930 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:14.930 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:14.930 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:14.930 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:14.930 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:14.930 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:14.930 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:14.931 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:14.931 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:14.931 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:14.931 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:14.931 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:14.931 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:14.931 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:14.931 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:14.931 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:14.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed589c6640 1 Processor -- start 2024-01-24T23:23:14.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed589c6640 1 -- start start 2024-01-24T23:23:14.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed589c6640 10 monclient: init 2024-01-24T23:23:14.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed589c6640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:14.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed589c6640 10 monclient: _add_conns ranks=[8,0,4,2,3,1,7,6,5] 2024-01-24T23:23:14.931 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed589c6640 10 monclient(hunting): picked mon.noname-e con 0x7fed5406d960 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:14.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed589c6640 10 monclient(hunting): picked mon.noname-a con 0x7fed541311d0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:14.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed589c6640 10 monclient(hunting): picked mon.noname-c con 0x7fed54131810 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:14.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed589c6640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed5406de30 con 0x7fed541311d0 2024-01-24T23:23:14.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed589c6640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed5409f100 con 0x7fed54131810 2024-01-24T23:23:14.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed589c6640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed5409f910 con 0x7fed5406d960 2024-01-24T23:23:14.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed589c6640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:14.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed589c6640 10 monclient(hunting): authenticate will time out at 1073.840980s 2024-01-24T23:23:14.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed51d9b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fed541311d0 0x7fed5412e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:60928/0 (socket says 172.21.15.115:60928) 2024-01-24T23:23:14.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed51d9b640 1 -- 172.21.15.115:0/1609926045 learned_addr learned my addr 172.21.15.115:0/1609926045 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:14.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1001394954 0 0) 0x7fed5406de30 con 0x7fed541311d0 2024-01-24T23:23:14.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:14.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:14.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:14.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): my global_id is 4914 2024-01-24T23:23:14.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed38003530 con 0x7fed541311d0 2024-01-24T23:23:14.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3937293751 0 0) 0x7fed5409f910 con 0x7fed5406d960 2024-01-24T23:23:14.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:14.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:14.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:14.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): my global_id is 84113 2024-01-24T23:23:14.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed5406de30 con 0x7fed5406d960 2024-01-24T23:23:14.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1498229143 0 0) 0x7fed5409f100 con 0x7fed54131810 2024-01-24T23:23:14.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:14.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:14.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:14.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): my global_id is 74902 2024-01-24T23:23:14.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed5409f910 con 0x7fed54131810 2024-01-24T23:23:14.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1186169574 0 0) 0x7fed38003530 con 0x7fed541311d0 2024-01-24T23:23:14.933 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fed5409f100 con 0x7fed541311d0 2024-01-24T23:23:14.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fed3c0047f0 con 0x7fed541311d0 2024-01-24T23:23:14.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:23:14.934 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): dump: 2024-01-24T23:23:14.934 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:14.934 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.934 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.934 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.934 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.934 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:14.934 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.934 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.934 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.934 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.934 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.935 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.935 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.935 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.935 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.935 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:14.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:14.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4237086054 0 0) 0x7fed5406de30 con 0x7fed5406d960 2024-01-24T23:23:14.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fed38003530 con 0x7fed5406d960 2024-01-24T23:23:14.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2475102213 0 0) 0x7fed5409f910 con 0x7fed54131810 2024-01-24T23:23:14.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fed5406de30 con 0x7fed54131810 2024-01-24T23:23:14.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fed4c001bb0 con 0x7fed5406d960 2024-01-24T23:23:14.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:14.935 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): dump: 2024-01-24T23:23:14.935 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:14.936 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.936 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.936 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.936 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.936 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:14.936 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.936 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.936 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.936 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.936 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.936 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.936 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.936 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.936 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.936 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fed440019e0 con 0x7fed54131810 2024-01-24T23:23:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:23:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): dump: 2024-01-24T23:23:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.938 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.938 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.938 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.938 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.938 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:14.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1979610570 0 0) 0x7fed5409f100 con 0x7fed541311d0 2024-01-24T23:23:14.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 >> v1:172.21.15.115:6793/0 conn(0x7fed5406d960 legacy=0x7fed541c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 >> v1:172.21.15.115:6791/0 conn(0x7fed54131810 legacy=0x7fed541d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:14.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 monclient: found mon.a 2024-01-24T23:23:14.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:14.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:23:14.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5409e180 con 0x7fed541311d0 2024-01-24T23:23:14.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient: _finish_auth 0 2024-01-24T23:23:14.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:14.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:14.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed589c6640 5 monclient: authenticate success, global_id 4914 2024-01-24T23:23:14.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed589c6640 10 monclient: _renew_subs 2024-01-24T23:23:14.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed589c6640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:23:14.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed589c6640 1 -- 172.21.15.115:0/1609926045 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed5409d620 con 0x7fed541311d0 2024-01-24T23:23:14.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed589c6640 10 monclient: _renew_subs 2024-01-24T23:23:14.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed589c6640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:23:14.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed589c6640 1 -- 172.21.15.115:0/1609926045 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fed5412ea60 con 0x7fed541311d0 2024-01-24T23:23:14.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fed3c004ec0 con 0x7fed541311d0 2024-01-24T23:23:14.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:14.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fed3c005370 con 0x7fed541311d0 2024-01-24T23:23:14.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:23:14.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient: dump: 2024-01-24T23:23:14.940 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:14.940 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.940 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.940 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.940 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.940 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:14.940 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.940 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.940 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.940 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.940 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.940 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.941 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.941 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.941 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.941 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:14.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed50d99640 4 set_mon_vals no callback set 2024-01-24T23:23:14.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed589c6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:14.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed589c6640 1 -- 172.21.15.115:0/1609926045 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fed5406a0d0 con 0x7fed541b71e0 2024-01-24T23:23:14.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fed3c00fdf0 con 0x7fed541311d0 2024-01-24T23:23:14.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(48..48 src has 1..48) v4 ==== 3994+0+0 (unknown 1916735388 0 0) 0x7fed3c043a40 con 0x7fed541311d0 2024-01-24T23:23:14.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3626801223 0 0) 0x7fed5406a0d0 con 0x7fed541b71e0 2024-01-24T23:23:14.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:14.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:14.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:14.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient(hunting): my global_id is 84116 2024-01-24T23:23:14.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed5409f100 con 0x7fed541b71e0 2024-01-24T23:23:14.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1450232558 0 0) 0x7fed5409f100 con 0x7fed541b71e0 2024-01-24T23:23:14.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed52d9d640 10 monclient: get_auth_request con 0x7fed38037dd0 auth_method 0 2024-01-24T23:23:14.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fed5406a0d0 con 0x7fed541b71e0 2024-01-24T23:23:14.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fed4c001fd0 con 0x7fed541b71e0 2024-01-24T23:23:14.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:14.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 10 monclient: dump: 2024-01-24T23:23:14.942 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:14.942 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.942 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.942 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.942 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.942 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:14.942 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.942 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.943 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.943 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.943 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.943 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.943 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.943 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.943 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.943 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:14.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 391847596 0 0) 0x7fed5406a0d0 con 0x7fed541b71e0 2024-01-24T23:23:14.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.916+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 --> v1:172.21.15.115:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed54070990 con 0x7fed541b71e0 2024-01-24T23:23:14.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.920+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 <== mon.2 v1:172.21.15.115:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fed4c004040 con 0x7fed541b71e0 2024-01-24T23:23:14.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.920+0000 7fed4affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:14.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.920+0000 7fed4affd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:14.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.920+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 >> v1:172.21.15.115:6790/0 conn(0x7fed541b71e0 legacy=0x7fed541b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.920+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 >> v1:172.21.15.115:6790/0 conn(0x7fed541b71e0 legacy=0x7fed541b75c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:14.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed589c6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:23:14.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed589c6640 1 -- 172.21.15.115:0/1609926045 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fed5406b5c0 con 0x7fed541b7cd0 2024-01-24T23:23:14.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3732052780 0 0) 0x7fed5406b5c0 con 0x7fed541b7cd0 2024-01-24T23:23:14.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed4affd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:14.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed4affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:14.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed4affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:14.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed4affd640 10 monclient(hunting): my global_id is 84125 2024-01-24T23:23:14.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed5406a0d0 con 0x7fed541b7cd0 2024-01-24T23:23:14.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 481243365 0 0) 0x7fed5406a0d0 con 0x7fed541b7cd0 2024-01-24T23:23:14.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fed5406b5c0 con 0x7fed541b7cd0 2024-01-24T23:23:14.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fed4c0047f0 con 0x7fed541b7cd0 2024-01-24T23:23:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed4affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed4affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed4affd640 10 monclient: dump: 2024-01-24T23:23:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:14.947 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:14.947 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:14.947 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:14.947 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:14.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1917596920 0 0) 0x7fed5406b5c0 con 0x7fed541b7cd0 2024-01-24T23:23:14.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 --> v1:172.21.15.115:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fed54071c70 con 0x7fed541b7cd0 2024-01-24T23:23:14.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 <== mon.2 v1:172.21.15.115:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 4006506570) 0x7fed4c004a20 con 0x7fed541b7cd0 2024-01-24T23:23:14.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed4affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:23:14.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed4affd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:14.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 >> v1:172.21.15.115:6790/0 conn(0x7fed541b7cd0 legacy=0x7fed541d68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed4affd640 1 -- 172.21.15.115:0/1609926045 >> v1:172.21.15.115:6790/0 conn(0x7fed541b7cd0 legacy=0x7fed541d68d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:14.947 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:23:14.948 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:14.948 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:14.948 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:23:14.948 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 16, 2024-01-24T23:23:14.948 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:23:14.948 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:23:14.948 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:23:14.948 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:23:14.948 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:23:14.948 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:23:14.948 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:23:14.948 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:23:14.949 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:23:14.949 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:23:14.949 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:14.949 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:23:14.949 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:14.949 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:23:14.949 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:23:14.949 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:14.949 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:14.949 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:14.949 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:14.949 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:14.949 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:14.949 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:14.949 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:14.950 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:14.950 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:14.950 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:14.950 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:14.950 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:23:14.950 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:23:14.950 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:14.950 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:14.950 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:14.950 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:14.950 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:14.950 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:14.950 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:14.950 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:14.950 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:14.951 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:14.951 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:14.951 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:14.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.951 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:23:14.951 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:23:14.951 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:23:14.951 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:23:14.951 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:23:14.951 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:23:14.951 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:14.951 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:14.951 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:23:14.951 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:23:14.951 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:23:14.952 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:23:14.952 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:23:14.952 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:23:14.952 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:23:14.952 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:14.952 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:23:14.952 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:14.952 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:14.952 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:14.952 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:14.952 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:14.952 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:14.952 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:14.952 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:14.953 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:14.953 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:14.953 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:14.953 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:14.953 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:23:14.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.953 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:23:14.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.953 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:14.953 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:14.953 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:14.953 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:14.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.953 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:14.954 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:23:14.954 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:14.954 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.954 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:14.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.954 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:23:14.954 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:23:14.954 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:14.954 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:14.954 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:14.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.954 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:14.954 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:14.955 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:14.955 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:14.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.955 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:14.955 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:23:14.955 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:14.955 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.955 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:14.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.955 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:23:14.955 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:23:14.955 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:14.955 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:14.955 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:14.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.956 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:14.956 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:14.956 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:14.956 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:14.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.956 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:14.956 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:23:14.956 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:14.956 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.956 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:14.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.956 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:23:14.956 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:23:14.956 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:14.957 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:14.957 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:14.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.957 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:14.957 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:14.957 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:14.957 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:14.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.957 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:14.957 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:23:14.957 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:14.957 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.957 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:14.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.958 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:23:14.958 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:23:14.958 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:14.958 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:14.958 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:14.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.958 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:14.958 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:14.958 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:14.958 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:14.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.958 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:14.959 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:23:14.959 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:14.959 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.959 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:14.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.959 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:23:14.959 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:23:14.959 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:14.959 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:14.959 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:14.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.959 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:14.959 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:14.959 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:14.960 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:14.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.960 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:14.960 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:23:14.960 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:14.960 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.960 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:14.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.960 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:23:14.960 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:23:14.960 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:14.960 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:14.960 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:14.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.961 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:14.961 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:14.961 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:14.961 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:14.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.961 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:14.961 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:23:14.961 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:14.961 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.961 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:14.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.961 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:23:14.961 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:23:14.961 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:14.962 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:14.962 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:14.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.962 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:14.962 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:14.962 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:14.962 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:14.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.962 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:14.962 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:23:14.962 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:14.962 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.962 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:14.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.963 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:23:14.963 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:23:14.963 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:14.963 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:14.963 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:14.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.963 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:14.963 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:14.963 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:14.963 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:14.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.963 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:14.963 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:23:14.964 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:14.964 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.964 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:14.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.964 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:23:14.964 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:23:14.964 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:14.964 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:14.964 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:14.964 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.964 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:14.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.964 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:23:14.964 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:23:14.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.965 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:14.965 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:14.965 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:14.965 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.965 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:14.965 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:23:14.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:14.965 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:14.965 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:14.965 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:14.965 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:14.965 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:14.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:14.965 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:23:14.966 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:23:14.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed589c6640 1 -- 172.21.15.115:0/1609926045 >> v1:172.21.15.132:6812/1835401922 conn(0x7fed38037dd0 legacy=0x7fed3803a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed589c6640 10 monclient: shutdown 2024-01-24T23:23:14.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed589c6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:14.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed589c6640 1 -- 172.21.15.115:0/1609926045 >> v1:172.21.15.115:6789/0 conn(0x7fed541311d0 legacy=0x7fed5412e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed52d9d640 1 -- 172.21.15.115:0/1609926045 reap_dead start 2024-01-24T23:23:14.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed52d9d640 1 -- 172.21.15.115:0/1609926045 reap_dead start 2024-01-24T23:23:14.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed589c6640 1 -- 172.21.15.115:0/1609926045 shutdown_connections 2024-01-24T23:23:14.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed589c6640 1 -- 172.21.15.115:0/1609926045 >> 172.21.15.115:0/1609926045 conn(0x7fed540a5d40 msgr2=0x7fed541bc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:14.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed589c6640 1 -- 172.21.15.115:0/1609926045 shutdown_connections 2024-01-24T23:23:14.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:14.924+0000 7fed589c6640 1 -- 172.21.15.115:0/1609926045 wait complete. 2024-01-24T23:23:14.968 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:23:14.968 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-01-24T23:23:14.968 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-01-24T23:23:14.976 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.976+0000 7faaacbed640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-01-24T23:23:14.977 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.976+0000 7faaacbed640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-01-24T23:23:14.977 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.976+0000 7faaacbed640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:14.977 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.976+0000 7faaacbed640 1 -- 172.21.15.132:0/396549338 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "ssd"} v 0) v1 -- 0x7faa8c000bc0 con 0x7faaa809ff20 2024-01-24T23:23:14.977 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.976+0000 7faa96ffd640 1 -- 172.21.15.132:0/396549338 <== mon.1 v1:172.21.15.132:6789/0 11 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "ssd"}]=0 v48) v1 ==== 90+0+8 (unknown 4243305861 0 905510949) 0x7faa9c014a60 con 0x7faaa809ff20 2024-01-24T23:23:14.977 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.976+0000 7faa96ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-01-24T23:23:14.977 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.976+0000 7faa96ffd640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:23:14.979 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.976+0000 7faaacbed640 1 -- 172.21.15.132:0/396549338 >> v1:172.21.15.132:6812/1835401922 conn(0x7faa88036fe0 legacy=0x7faa880394a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.979 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.976+0000 7faaacbed640 10 monclient: shutdown 2024-01-24T23:23:14.979 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.976+0000 7faaacbed640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:14.979 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.976+0000 7faaacbed640 1 -- 172.21.15.132:0/396549338 >> v1:172.21.15.132:6789/0 conn(0x7faaa809ff20 legacy=0x7faaa809cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:14.979 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.976+0000 7faaacbed640 1 -- 172.21.15.132:0/396549338 shutdown_connections 2024-01-24T23:23:14.979 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.976+0000 7faaacbed640 1 -- 172.21.15.132:0/396549338 >> 172.21.15.132:0/396549338 conn(0x7faaa80a5a10 msgr2=0x7faaa8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:14.980 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.980+0000 7faaacbed640 1 -- 172.21.15.132:0/396549338 shutdown_connections 2024-01-24T23:23:14.980 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:14.980+0000 7faaacbed640 1 -- 172.21.15.132:0/396549338 wait complete. 2024-01-24T23:23:14.992 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:23:14.992 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule ls-by-class hdd 2024-01-24T23:23:14.993 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep foo-hdd 2024-01-24T23:23:15.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.076+0000 7f59bdd04640 10 monclient: get_monmap_and_config 2024-01-24T23:23:15.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.076+0000 7f59bdd04640 10 monclient: build_initial_monmap 2024-01-24T23:23:15.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.076+0000 7f59bdd04640 10 monclient: monmap: 2024-01-24T23:23:15.082 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:15.082 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.082 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:15.082870+0000 2024-01-24T23:23:15.083 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:15.082870+0000 2024-01-24T23:23:15.083 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:15.083 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.083 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:15.083 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:15.083 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:15.083 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:15.083 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:15.083 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:15.083 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:15.083 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:15.083 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:15.083 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 1 Processor -- start 2024-01-24T23:23:15.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 1 -- start start 2024-01-24T23:23:15.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 10 monclient: init 2024-01-24T23:23:15.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:15.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 10 monclient: _add_conns ranks=[1,3,5,6,4,0,8,7,2] 2024-01-24T23:23:15.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 10 monclient(hunting): picked mon.noname-f con 0x7f59b8133d90 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:15.091 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 10 monclient(hunting): picked mon.noname-g con 0x7f59b8134170 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:15.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 10 monclient(hunting): picked mon.noname-h con 0x7f59b813dcf0 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:15.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b804ed70 con 0x7f59b8133d90 2024-01-24T23:23:15.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b809b250 con 0x7f59b8134170 2024-01-24T23:23:15.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b81345d0 con 0x7f59b813dcf0 2024-01-24T23:23:15.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:15.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 10 monclient(hunting): authenticate will time out at 1074.005406s 2024-01-24T23:23:15.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b77fe640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f59b8133d90 0x7f59b80a19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.115:52422/0 (socket says 172.21.15.115:52422) 2024-01-24T23:23:15.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b77fe640 1 -- 172.21.15.115:0/1086063019 learned_addr learned my addr 172.21.15.115:0/1086063019 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:15.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 1 -- 172.21.15.115:0/1086063019 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969629305 0 0) 0x7f59b804ed70 con 0x7f59b8133d90 2024-01-24T23:23:15.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 10 monclient(hunting): my global_id is 4960 2024-01-24T23:23:15.092 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 1 -- 172.21.15.115:0/1086063019 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59a4003530 con 0x7f59b8133d90 2024-01-24T23:23:15.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 1 -- 172.21.15.115:0/1086063019 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1914277693 0 0) 0x7f59b809b250 con 0x7f59b8134170 2024-01-24T23:23:15.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 10 monclient(hunting): my global_id is 4854 2024-01-24T23:23:15.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 1 -- 172.21.15.115:0/1086063019 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59b804ed70 con 0x7f59b8134170 2024-01-24T23:23:15.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 1 -- 172.21.15.115:0/1086063019 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2583055837 0 0) 0x7f59b81345d0 con 0x7f59b813dcf0 2024-01-24T23:23:15.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 10 monclient(hunting): my global_id is 4910 2024-01-24T23:23:15.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 1 -- 172.21.15.115:0/1086063019 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59b809b250 con 0x7f59b813dcf0 2024-01-24T23:23:15.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 1 -- 172.21.15.115:0/1086063019 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1321739114 0 0) 0x7f59a4003530 con 0x7f59b8133d90 2024-01-24T23:23:15.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 1 -- 172.21.15.115:0/1086063019 >> v1:172.21.15.132:6791/0 conn(0x7f59b813dcf0 legacy=0x7f59b81400e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 1 -- 172.21.15.115:0/1086063019 >> v1:172.21.15.132:6790/0 conn(0x7f59b8134170 legacy=0x7f59b80a2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 10 monclient: _finish_hunting 0 2024-01-24T23:23:15.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 1 monclient: found mon.noname-f 2024-01-24T23:23:15.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:15.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:23:15.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 1 -- 172.21.15.115:0/1086063019 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b80a2810 con 0x7f59b8133d90 2024-01-24T23:23:15.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 10 monclient: _finish_auth 0 2024-01-24T23:23:15.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 10 monclient: _check_auth_tickets 2024-01-24T23:23:15.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:15.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 1 -- 172.21.15.115:0/1086063019 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f59b00017c0 con 0x7f59b8133d90 2024-01-24T23:23:15.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:23:15.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 10 monclient: dump: 2024-01-24T23:23:15.095 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.095 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.095 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.095 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.095 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.095 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.095 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.095 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.095 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.095 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.095 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.095 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.095 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.096 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.096 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.096 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 5 monclient: authenticate success, global_id 4960 2024-01-24T23:23:15.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:15.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 1 -- 172.21.15.115:0/1086063019 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f59b00019e0 con 0x7f59b8133d90 2024-01-24T23:23:15.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:15.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:15.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 4 set_mon_vals no callback set 2024-01-24T23:23:15.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 1 -- 172.21.15.115:0/1086063019 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f59b0004bf0 con 0x7f59b8133d90 2024-01-24T23:23:15.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:23:15.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59b67fc640 10 monclient: dump: 2024-01-24T23:23:15.096 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.097 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.097 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.097 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.097 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.097 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.097 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.097 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.097 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.097 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.097 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.097 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.097 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.097 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.097 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.097 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 10 monclient: shutdown 2024-01-24T23:23:15.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:15.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 1 -- 172.21.15.115:0/1086063019 >> v1:172.21.15.132:6789/0 conn(0x7f59b8133d90 legacy=0x7f59b80a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 1 -- 172.21.15.115:0/1086063019 shutdown_connections 2024-01-24T23:23:15.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 1 -- 172.21.15.115:0/1086063019 >> 172.21.15.115:0/1086063019 conn(0x7f59b80a65e0 msgr2=0x7f59b80a2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:15.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 1 -- 172.21.15.115:0/1086063019 shutdown_connections 2024-01-24T23:23:15.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 1 -- 172.21.15.115:0/1086063019 wait complete. 2024-01-24T23:23:15.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 10 monclient: build_initial_monmap 2024-01-24T23:23:15.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.080+0000 7f59bdd04640 10 monclient: monmap: 2024-01-24T23:23:15.098 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:15.098 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:15.098 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:15.098 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:15.098 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:15.098 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:15.099 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:15.099 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:15.099 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:15.099 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:15.099 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:15.099 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:15.099 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:15.099 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:15.099 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:15.099 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59bdd04640 1 Processor -- start 2024-01-24T23:23:15.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59bdd04640 1 -- start start 2024-01-24T23:23:15.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59bdd04640 10 monclient: init 2024-01-24T23:23:15.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59bdd04640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:15.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59bdd04640 10 monclient: _add_conns ranks=[4,8,6,7,2,3,5,1,0] 2024-01-24T23:23:15.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59bdd04640 10 monclient(hunting): picked mon.noname-c con 0x7f59b8133d90 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:15.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59bdd04640 10 monclient(hunting): picked mon.noname-e con 0x7f59b8134170 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:15.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59bdd04640 10 monclient(hunting): picked mon.noname-d con 0x7f59b813dcf0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:15.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59bdd04640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b81cc3d0 con 0x7f59b8133d90 2024-01-24T23:23:15.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59bdd04640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b8138b60 con 0x7f59b813dcf0 2024-01-24T23:23:15.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59bdd04640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b809c610 con 0x7f59b8134170 2024-01-24T23:23:15.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59bdd04640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:15.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59bdd04640 10 monclient(hunting): authenticate will time out at 1074.008999s 2024-01-24T23:23:15.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59b77fe640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f59b8133d90 0x7f59b813d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:48560/0 (socket says 172.21.15.115:48560) 2024-01-24T23:23:15.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59b77fe640 1 -- 172.21.15.115:0/4210094188 learned_addr learned my addr 172.21.15.115:0/4210094188 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:15.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59b6ffd640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f59b8134170 0x7f59b81d07c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:52888/0 (socket says 172.21.15.115:52888) 2024-01-24T23:23:15.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59b7fff640 1 --1- 172.21.15.115:0/4210094188 >> v1:172.21.15.115:6792/0 conn(0x7f59b813dcf0 0x7f59b81d3ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:34120/0 (socket says 172.21.15.115:34120) 2024-01-24T23:23:15.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2524585112 0 0) 0x7f59b8138b60 con 0x7f59b813dcf0 2024-01-24T23:23:15.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): my global_id is 4929 2024-01-24T23:23:15.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5998003530 con 0x7f59b813dcf0 2024-01-24T23:23:15.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3108246903 0 0) 0x7f59b81cc3d0 con 0x7f59b8133d90 2024-01-24T23:23:15.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): my global_id is 74911 2024-01-24T23:23:15.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59b8138b60 con 0x7f59b8133d90 2024-01-24T23:23:15.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 378218836 0 0) 0x7f59b809c610 con 0x7f59b8134170 2024-01-24T23:23:15.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): my global_id is 84122 2024-01-24T23:23:15.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59b81cc3d0 con 0x7f59b8134170 2024-01-24T23:23:15.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 588563444 0 0) 0x7f5998003530 con 0x7f59b813dcf0 2024-01-24T23:23:15.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f59b809c610 con 0x7f59b813dcf0 2024-01-24T23:23:15.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f59a8001e20 con 0x7f59b813dcf0 2024-01-24T23:23:15.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:23:15.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): dump: 2024-01-24T23:23:15.102 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.102 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.102 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.102 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.102 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.102 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.102 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.102 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.103 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.103 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.103 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.103 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.103 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.103 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.103 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.103 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:15.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1736527419 0 0) 0x7f59b8138b60 con 0x7f59b8133d90 2024-01-24T23:23:15.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5998003530 con 0x7f59b8133d90 2024-01-24T23:23:15.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1238293363 0 0) 0x7f59b81cc3d0 con 0x7f59b8134170 2024-01-24T23:23:15.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f59b8138b60 con 0x7f59b8134170 2024-01-24T23:23:15.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f59b00016c0 con 0x7f59b8133d90 2024-01-24T23:23:15.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:23:15.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): dump: 2024-01-24T23:23:15.104 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.104 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.104 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.104 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.104 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.104 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.104 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.104 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.104 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.104 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.104 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.104 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.104 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.105 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.105 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.105 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f59a00019e0 con 0x7f59b8134170 2024-01-24T23:23:15.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:15.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): dump: 2024-01-24T23:23:15.105 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.105 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.105 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.105 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.105 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.105 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.105 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.106 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.106 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.106 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.106 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.106 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.106 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.106 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.106 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.106 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1980491069 0 0) 0x7f59b809c610 con 0x7f59b813dcf0 2024-01-24T23:23:15.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 >> v1:172.21.15.115:6793/0 conn(0x7f59b8134170 legacy=0x7f59b81d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 >> v1:172.21.15.115:6791/0 conn(0x7f59b8133d90 legacy=0x7f59b813d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient: _finish_hunting 0 2024-01-24T23:23:15.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 monclient: found mon.d 2024-01-24T23:23:15.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:15.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:23:15.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b80676c0 con 0x7f59b813dcf0 2024-01-24T23:23:15.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient: _finish_auth 0 2024-01-24T23:23:15.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient: _check_auth_tickets 2024-01-24T23:23:15.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:15.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59bdd04640 5 monclient: authenticate success, global_id 4929 2024-01-24T23:23:15.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59bdd04640 10 monclient: _renew_subs 2024-01-24T23:23:15.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59bdd04640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:23:15.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59bdd04640 1 -- 172.21.15.115:0/4210094188 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b809fac0 con 0x7f59b813dcf0 2024-01-24T23:23:15.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59bdd04640 10 monclient: _renew_subs 2024-01-24T23:23:15.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59bdd04640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:23:15.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59bdd04640 1 -- 172.21.15.115:0/4210094188 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b813da80 con 0x7f59b813dcf0 2024-01-24T23:23:15.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f59a80020e0 con 0x7f59b813dcf0 2024-01-24T23:23:15.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:15.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f59a8003f90 con 0x7f59b813dcf0 2024-01-24T23:23:15.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59b5ffb640 4 set_mon_vals no callback set 2024-01-24T23:23:15.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:15.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient: dump: 2024-01-24T23:23:15.108 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.108 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.108 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.108 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.108 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.108 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.108 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.108 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.108 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.108 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.108 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.109 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.109 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.109 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.109 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.109 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59bdd04640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:15.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59bdd04640 1 -- 172.21.15.115:0/4210094188 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f599c0065d0 con 0x7f599c002870 2024-01-24T23:23:15.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f59a8004170 con 0x7f59b813dcf0 2024-01-24T23:23:15.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(48..48 src has 1..48) v4 ==== 3994+0+0 (unknown 1916735388 0 0) 0x7f59a8043b80 con 0x7f59b813dcf0 2024-01-24T23:23:15.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2852059827 0 0) 0x7f599c0065d0 con 0x7f599c002870 2024-01-24T23:23:15.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient(hunting): my global_id is 74920 2024-01-24T23:23:15.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59b809c610 con 0x7f599c002870 2024-01-24T23:23:15.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59b6ffd640 10 monclient: get_auth_request con 0x7f5998037cd0 auth_method 0 2024-01-24T23:23:15.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1465416879 0 0) 0x7f59b809c610 con 0x7f599c002870 2024-01-24T23:23:15.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f599c0065d0 con 0x7f599c002870 2024-01-24T23:23:15.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f59b00011d0 con 0x7f599c002870 2024-01-24T23:23:15.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:23:15.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 10 monclient: dump: 2024-01-24T23:23:15.110 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.110 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.110 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.110 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.110 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.110 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.110 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.110 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.110 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.110 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.111 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.111 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.111 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.111 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.111 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.111 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 747262930 0 0) 0x7f599c0065d0 con 0x7f599c002870 2024-01-24T23:23:15.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.084+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 --> v1:172.21.15.115:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f599c00a120 con 0x7f599c002870 2024-01-24T23:23:15.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.088+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 <== mon.4 v1:172.21.15.115:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f59b0007010 con 0x7f599c002870 2024-01-24T23:23:15.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.088+0000 7f59affff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:15.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.088+0000 7f59affff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:15.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.088+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 >> v1:172.21.15.115:6791/0 conn(0x7f599c002870 legacy=0x7f599c004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.088+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 >> v1:172.21.15.115:6791/0 conn(0x7f599c002870 legacy=0x7f599c004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:15.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.092+0000 7f59bdd04640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:23:15.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.092+0000 7f59bdd04640 1 -- 172.21.15.115:0/4210094188 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f599c009df0 con 0x7f599c0090b0 2024-01-24T23:23:15.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.096+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3215766632 0 0) 0x7f599c009df0 con 0x7f599c0090b0 2024-01-24T23:23:15.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.096+0000 7f59affff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.096+0000 7f59affff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.096+0000 7f59affff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.096+0000 7f59affff640 10 monclient(hunting): my global_id is 74938 2024-01-24T23:23:15.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.096+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f599c0065d0 con 0x7f599c0090b0 2024-01-24T23:23:15.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.096+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4077731439 0 0) 0x7f599c0065d0 con 0x7f599c0090b0 2024-01-24T23:23:15.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.096+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f599c009df0 con 0x7f599c0090b0 2024-01-24T23:23:15.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.096+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f59b00057d0 con 0x7f599c0090b0 2024-01-24T23:23:15.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.096+0000 7f59affff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.096+0000 7f59affff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:23:15.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.096+0000 7f59affff640 10 monclient: dump: 2024-01-24T23:23:15.115 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.115 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.115 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.116 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.116 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.116 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.116 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.116 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.116 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.116 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.116 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.116 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.116 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.116 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.117 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.117 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.096+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2339413449 0 0) 0x7f599c009df0 con 0x7f599c0090b0 2024-01-24T23:23:15.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.096+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 --> v1:172.21.15.115:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f599c00c310 con 0x7f599c0090b0 2024-01-24T23:23:15.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.100+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 <== mon.4 v1:172.21.15.115:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 487288308) 0x7f59b0006d50 con 0x7f599c0090b0 2024-01-24T23:23:15.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.100+0000 7f59affff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:23:15.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.100+0000 7f59affff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:15.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.100+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 >> v1:172.21.15.115:6791/0 conn(0x7f599c0090b0 legacy=0x7f599c009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.100+0000 7f59affff640 1 -- 172.21.15.115:0/4210094188 >> v1:172.21.15.115:6791/0 conn(0x7f599c0090b0 legacy=0x7f599c009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:15.118 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:23:15.118 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:15.118 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:15.118 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:23:15.118 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 16, 2024-01-24T23:23:15.118 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:23:15.118 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:23:15.118 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:23:15.118 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:23:15.118 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:23:15.118 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:23:15.118 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:23:15.118 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:23:15.118 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:23:15.119 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:23:15.119 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:15.119 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:23:15.119 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:15.119 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:23:15.119 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:23:15.119 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:15.119 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:15.119 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:15.119 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:15.119 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:15.119 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:15.119 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:15.119 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:15.119 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:15.120 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:15.120 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:15.120 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:15.120 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:23:15.120 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:23:15.120 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:15.120 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:15.120 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:15.120 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:15.120 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:15.120 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:15.120 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:15.120 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:15.120 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:15.121 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:15.121 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:15.121 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.121 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:23:15.121 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:23:15.121 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:23:15.121 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:23:15.121 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:23:15.121 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:23:15.121 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:15.121 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:15.121 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:23:15.121 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:23:15.122 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:23:15.122 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:23:15.122 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:23:15.122 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:23:15.122 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:23:15.122 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:15.122 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:23:15.122 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:15.123 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:15.123 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:15.123 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:15.123 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:15.123 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:15.123 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:15.123 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:15.123 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:15.123 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:15.123 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:15.123 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:15.123 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:23:15.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.123 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:23:15.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.124 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:15.124 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:15.124 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.124 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.124 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.124 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:23:15.124 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.124 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.124 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.124 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:23:15.124 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:23:15.124 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.125 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.125 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.125 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:15.125 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:15.125 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.125 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.125 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.125 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:23:15.125 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.125 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.125 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.126 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:23:15.126 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:23:15.126 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.126 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.126 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.126 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:15.126 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:15.126 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.126 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.126 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.126 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:23:15.126 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.127 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.127 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.127 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:23:15.127 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:23:15.127 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.127 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.127 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.127 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:15.127 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:15.127 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.127 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.128 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.128 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:23:15.128 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.128 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.128 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.128 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:23:15.128 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:23:15.128 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.128 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.128 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.128 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:15.129 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:15.129 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.129 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.129 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.129 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:23:15.129 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.129 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.129 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.129 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:23:15.129 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:23:15.129 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.129 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.129 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.130 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:15.130 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:15.130 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.130 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.130 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.130 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:23:15.130 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.130 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.130 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.130 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:23:15.130 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:23:15.131 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.131 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.131 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.131 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:15.131 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:15.131 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.131 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.131 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.131 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:23:15.131 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.131 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.131 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.132 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:23:15.132 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:23:15.132 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.132 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.132 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.132 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:15.132 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:15.132 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.132 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.132 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.132 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:23:15.133 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.133 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.133 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.133 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:23:15.133 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:23:15.133 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.133 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.133 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.133 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:15.133 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:15.133 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.134 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.134 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.134 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:23:15.134 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.134 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.134 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.134 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:23:15.134 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:23:15.134 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.134 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.134 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.134 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.134 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.135 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:23:15.135 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:23:15.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.135 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:15.135 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:15.135 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:15.135 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.135 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:15.135 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:23:15.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.135 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:15.135 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:15.135 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:15.135 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.136 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:15.136 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:23:15.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.136 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:15.136 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:15.136 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:23:15.136 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.136 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.136 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:23:15.136 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:23:15.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.100+0000 7f59bdd04640 1 -- 172.21.15.115:0/4210094188 >> v1:172.21.15.132:6812/1835401922 conn(0x7f5998037cd0 legacy=0x7f599803a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.100+0000 7f59bdd04640 10 monclient: shutdown 2024-01-24T23:23:15.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.100+0000 7f59bdd04640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:15.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.100+0000 7f59bdd04640 1 -- 172.21.15.115:0/4210094188 >> v1:172.21.15.115:6792/0 conn(0x7f59b813dcf0 legacy=0x7f59b81d3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.100+0000 7f59b7fff640 1 -- 172.21.15.115:0/4210094188 reap_dead start 2024-01-24T23:23:15.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.100+0000 7f59b7fff640 1 -- 172.21.15.115:0/4210094188 reap_dead start 2024-01-24T23:23:15.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.100+0000 7f59bdd04640 1 -- 172.21.15.115:0/4210094188 shutdown_connections 2024-01-24T23:23:15.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.100+0000 7f59bdd04640 1 -- 172.21.15.115:0/4210094188 >> 172.21.15.115:0/4210094188 conn(0x7f59b80a65e0 msgr2=0x7f59b8132b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:15.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.100+0000 7f59bdd04640 1 -- 172.21.15.115:0/4210094188 shutdown_connections 2024-01-24T23:23:15.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.100+0000 7f59bdd04640 1 -- 172.21.15.115:0/4210094188 wait complete. 2024-01-24T23:23:15.140 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:23:15.140 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-01-24T23:23:15.140 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-01-24T23:23:15.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.088+0000 7feb6a4be640 10 monclient: get_monmap_and_config 2024-01-24T23:23:15.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.088+0000 7feb6a4be640 10 monclient: build_initial_monmap 2024-01-24T23:23:15.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.088+0000 7feb6a4be640 10 monclient: monmap: 2024-01-24T23:23:15.145 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:15.145 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.145 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:15.092122+0000 2024-01-24T23:23:15.145 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:15.092122+0000 2024-01-24T23:23:15.145 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:15.145 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:15.145 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:15.145 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:15.146 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:15.146 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:15.146 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:15.146 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:15.146 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:15.146 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:15.146 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:15.146 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:15.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 1 Processor -- start 2024-01-24T23:23:15.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 1 -- start start 2024-01-24T23:23:15.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 10 monclient: init 2024-01-24T23:23:15.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:15.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 10 monclient: _add_conns ranks=[8,3,2,5,1,0,6,7,4] 2024-01-24T23:23:15.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 10 monclient(hunting): picked mon.noname-e con 0x7feb64071980 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:15.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 10 monclient(hunting): picked mon.noname-g con 0x7feb64071d60 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:15.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 10 monclient(hunting): picked mon.noname-b con 0x7feb6413d4b0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:15.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb64064700 con 0x7feb6413d4b0 2024-01-24T23:23:15.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb6406b310 con 0x7feb64071d60 2024-01-24T23:23:15.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb64134210 con 0x7feb64071980 2024-01-24T23:23:15.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb637fe640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7feb64071d60 0x7feb6413bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:36382/0 (socket says 172.21.15.132:36382) 2024-01-24T23:23:15.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:15.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb637fe640 1 -- 172.21.15.132:0/3194099506 learned_addr learned my addr 172.21.15.132:0/3194099506 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:15.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 10 monclient(hunting): authenticate will time out at 1082.417571s 2024-01-24T23:23:15.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 1 -- 172.21.15.132:0/3194099506 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2846918995 0 0) 0x7feb6406b310 con 0x7feb64071d60 2024-01-24T23:23:15.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 10 monclient(hunting): my global_id is 4863 2024-01-24T23:23:15.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 1 -- 172.21.15.132:0/3194099506 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb4c003530 con 0x7feb64071d60 2024-01-24T23:23:15.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 1 -- 172.21.15.132:0/3194099506 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 13673353 0 0) 0x7feb64134210 con 0x7feb64071980 2024-01-24T23:23:15.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 10 monclient(hunting): my global_id is 84131 2024-01-24T23:23:15.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 1 -- 172.21.15.132:0/3194099506 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb6406b310 con 0x7feb64071980 2024-01-24T23:23:15.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 1 -- 172.21.15.132:0/3194099506 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536593847 0 0) 0x7feb64064700 con 0x7feb6413d4b0 2024-01-24T23:23:15.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 10 monclient(hunting): my global_id is 84134 2024-01-24T23:23:15.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 1 -- 172.21.15.132:0/3194099506 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb64134210 con 0x7feb6413d4b0 2024-01-24T23:23:15.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 1 -- 172.21.15.132:0/3194099506 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1908384380 0 0) 0x7feb4c003530 con 0x7feb64071d60 2024-01-24T23:23:15.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 1 -- 172.21.15.132:0/3194099506 >> v1:172.21.15.115:6793/0 conn(0x7feb64071980 legacy=0x7feb64133b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 1 -- 172.21.15.132:0/3194099506 >> v1:172.21.15.115:6790/0 conn(0x7feb6413d4b0 legacy=0x7feb6413f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:15.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 1 monclient: found mon.noname-g 2024-01-24T23:23:15.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:15.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:23:15.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 1 -- 172.21.15.132:0/3194099506 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb6406d920 con 0x7feb64071d60 2024-01-24T23:23:15.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 10 monclient: _finish_auth 0 2024-01-24T23:23:15.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:15.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:15.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 5 monclient: authenticate success, global_id 4863 2024-01-24T23:23:15.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:15.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 1 -- 172.21.15.132:0/3194099506 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7feb540017c0 con 0x7feb64071d60 2024-01-24T23:23:15.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:15.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 10 monclient: dump: 2024-01-24T23:23:15.150 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:15.150 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.150 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.150 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.150 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.150 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:15.150 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.150 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.150 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.150 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.150 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.150 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.150 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.150 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.150 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.151 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:15.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:15.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 1 -- 172.21.15.132:0/3194099506 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7feb54001aa0 con 0x7feb64071d60 2024-01-24T23:23:15.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:15.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:15.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 4 set_mon_vals no callback set 2024-01-24T23:23:15.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 1 -- 172.21.15.132:0/3194099506 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7feb54004bf0 con 0x7feb64071d60 2024-01-24T23:23:15.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:15.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb62ffd640 10 monclient: dump: 2024-01-24T23:23:15.151 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:15.151 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.151 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.151 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.151 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.152 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:15.152 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.152 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.152 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.152 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.152 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.152 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.152 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.152 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.152 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.152 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:15.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 10 monclient: shutdown 2024-01-24T23:23:15.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:15.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 1 -- 172.21.15.132:0/3194099506 >> v1:172.21.15.132:6790/0 conn(0x7feb64071d60 legacy=0x7feb6413bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 1 -- 172.21.15.132:0/3194099506 shutdown_connections 2024-01-24T23:23:15.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 1 -- 172.21.15.132:0/3194099506 >> 172.21.15.132:0/3194099506 conn(0x7feb640a2ab0 msgr2=0x7feb6406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:15.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 1 -- 172.21.15.132:0/3194099506 shutdown_connections 2024-01-24T23:23:15.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 1 -- 172.21.15.132:0/3194099506 wait complete. 2024-01-24T23:23:15.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 10 monclient: build_initial_monmap 2024-01-24T23:23:15.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 10 monclient: monmap: 2024-01-24T23:23:15.153 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:15.153 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:15.153 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:15.153 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:15.153 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:15.153 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:15.153 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:15.153 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:15.153 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:15.153 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:15.154 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:15.154 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:15.154 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:15.154 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:15.154 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:15.154 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:15.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 1 Processor -- start 2024-01-24T23:23:15.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 1 -- start start 2024-01-24T23:23:15.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 10 monclient: init 2024-01-24T23:23:15.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:15.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 10 monclient: _add_conns ranks=[4,0,6,2,8,5,7,3,1] 2024-01-24T23:23:15.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 10 monclient(hunting): picked mon.noname-c con 0x7feb64071980 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:15.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 10 monclient(hunting): picked mon.noname-a con 0x7feb64071d60 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:15.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 10 monclient(hunting): picked mon.noname-d con 0x7feb6413d4b0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:15.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb64135f30 con 0x7feb64071d60 2024-01-24T23:23:15.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb6409f770 con 0x7feb64071980 2024-01-24T23:23:15.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb6412fc30 con 0x7feb6413d4b0 2024-01-24T23:23:15.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:15.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.092+0000 7feb6a4be640 10 monclient(hunting): authenticate will time out at 1082.419988s 2024-01-24T23:23:15.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb63fff640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7feb64071980 0x7feb64134570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:53452/0 (socket says 172.21.15.132:53452) 2024-01-24T23:23:15.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb63fff640 1 -- 172.21.15.132:0/2811665744 learned_addr learned my addr 172.21.15.132:0/2811665744 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:15.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb68a5b640 1 --1- 172.21.15.132:0/2811665744 >> v1:172.21.15.115:6792/0 conn(0x7feb6413d4b0 0x7feb641d3aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:59388/0 (socket says 172.21.15.132:59388) 2024-01-24T23:23:15.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4152802751 0 0) 0x7feb6409f770 con 0x7feb64071980 2024-01-24T23:23:15.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient(hunting): my global_id is 74929 2024-01-24T23:23:15.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb50003560 con 0x7feb64071980 2024-01-24T23:23:15.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3704467334 0 0) 0x7feb6412fc30 con 0x7feb6413d4b0 2024-01-24T23:23:15.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient(hunting): my global_id is 4938 2024-01-24T23:23:15.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb6409f770 con 0x7feb6413d4b0 2024-01-24T23:23:15.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833099791 0 0) 0x7feb64135f30 con 0x7feb64071d60 2024-01-24T23:23:15.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient(hunting): my global_id is 4923 2024-01-24T23:23:15.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb6412fc30 con 0x7feb64071d60 2024-01-24T23:23:15.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1969809989 0 0) 0x7feb50003560 con 0x7feb64071980 2024-01-24T23:23:15.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feb64135f30 con 0x7feb64071980 2024-01-24T23:23:15.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7feb58001980 con 0x7feb64071980 2024-01-24T23:23:15.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:23:15.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient(hunting): dump: 2024-01-24T23:23:15.157 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:15.157 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.157 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.157 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.157 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.157 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:15.157 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.157 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.157 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.157 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.158 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.158 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.158 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.158 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.158 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.158 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:15.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:15.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2921941782 0 0) 0x7feb6409f770 con 0x7feb6413d4b0 2024-01-24T23:23:15.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feb50003560 con 0x7feb6413d4b0 2024-01-24T23:23:15.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 623575970 0 0) 0x7feb6412fc30 con 0x7feb64071d60 2024-01-24T23:23:15.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feb6409f770 con 0x7feb64071d60 2024-01-24T23:23:15.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7feb5c001de0 con 0x7feb6413d4b0 2024-01-24T23:23:15.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:15.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient(hunting): dump: 2024-01-24T23:23:15.159 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:15.159 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.159 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.159 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.159 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.159 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:15.159 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.159 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.159 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.159 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.159 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.159 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.159 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.159 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.160 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.160 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:15.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7feb54004b30 con 0x7feb64071d60 2024-01-24T23:23:15.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:23:15.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient(hunting): dump: 2024-01-24T23:23:15.160 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:15.160 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.160 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.160 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.160 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.160 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:15.160 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.160 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.160 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.161 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.161 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.161 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.161 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.161 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.161 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.161 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:15.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4110344376 0 0) 0x7feb64135f30 con 0x7feb64071980 2024-01-24T23:23:15.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 >> v1:172.21.15.115:6792/0 conn(0x7feb6413d4b0 legacy=0x7feb641d3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 >> v1:172.21.15.115:6789/0 conn(0x7feb64071d60 legacy=0x7feb64134c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:23:15.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 1 monclient: found mon.c 2024-01-24T23:23:15.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:15.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:23:15.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb64135390 con 0x7feb64071980 2024-01-24T23:23:15.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient: _finish_auth 0 2024-01-24T23:23:15.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:15.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:15.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb6a4be640 5 monclient: authenticate success, global_id 74929 2024-01-24T23:23:15.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb6a4be640 10 monclient: _renew_subs 2024-01-24T23:23:15.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb6a4be640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:23:15.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb6a4be640 1 -- 172.21.15.132:0/2811665744 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb6412f950 con 0x7feb64071980 2024-01-24T23:23:15.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb6a4be640 10 monclient: _renew_subs 2024-01-24T23:23:15.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb6a4be640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:23:15.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb6a4be640 1 -- 172.21.15.132:0/2811665744 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7feb64136360 con 0x7feb64071980 2024-01-24T23:23:15.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7feb58001c60 con 0x7feb64071980 2024-01-24T23:23:15.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:15.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7feb58003c10 con 0x7feb64071980 2024-01-24T23:23:15.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb627fc640 4 set_mon_vals no callback set 2024-01-24T23:23:15.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:23:15.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 10 monclient: dump: 2024-01-24T23:23:15.163 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:15.163 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.163 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.163 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.163 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.163 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:15.163 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.163 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.163 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.163 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.163 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.163 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.163 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.163 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.164 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.164 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:15.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb6a4be640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:15.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb6a4be640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:15.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb6a4be640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:23:15.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb6a4be640 1 -- 172.21.15.132:0/2811665744 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb44005090 con 0x7feb64071980 2024-01-24T23:23:15.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7feb58003df0 con 0x7feb64071980 2024-01-24T23:23:15.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(48..48 src has 1..48) v4 ==== 3994+0+0 (unknown 1916735388 0 0) 0x7feb58043760 con 0x7feb64071980 2024-01-24T23:23:15.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.096+0000 7feb637fe640 10 monclient: get_auth_request con 0x7feb50037dc0 auth_method 0 2024-01-24T23:23:15.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.100+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7feb580146c0 con 0x7feb64071980 2024-01-24T23:23:15.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.100+0000 7feb60ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:15.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.100+0000 7feb60ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:15.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.244+0000 7feb6a4be640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-01-24T23:23:15.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.244+0000 7feb6a4be640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-01-24T23:23:15.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.244+0000 7feb6a4be640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:23:15.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.244+0000 7feb6a4be640 1 -- 172.21.15.132:0/2811665744 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "hdd"} v 0) v1 -- 0x7feb440058c0 con 0x7feb64071980 2024-01-24T23:23:15.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.244+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "hdd"}]=0 v48) v1 ==== 90+0+8 (unknown 251225796 0 1286160932) 0x7feb58012d60 con 0x7feb64071980 2024-01-24T23:23:15.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.244+0000 7feb60ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-01-24T23:23:15.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.244+0000 7feb60ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:15.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.276+0000 7fe89a028640 10 monclient: get_monmap_and_config 2024-01-24T23:23:15.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.276+0000 7fe89a028640 10 monclient: build_initial_monmap 2024-01-24T23:23:15.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 10 monclient: monmap: 2024-01-24T23:23:15.282 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:15.282 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.282 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:15.282922+0000 2024-01-24T23:23:15.283 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:15.282922+0000 2024-01-24T23:23:15.283 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:15.283 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.283 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:15.283 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:15.283 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:15.283 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:15.283 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:15.283 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:15.283 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:15.283 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:15.283 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:15.283 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 1 Processor -- start 2024-01-24T23:23:15.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 1 -- start start 2024-01-24T23:23:15.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 10 monclient: init 2024-01-24T23:23:15.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:15.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 10 monclient: _add_conns ranks=[5,6,3,8,1,2,4,0,7] 2024-01-24T23:23:15.291 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 10 monclient(hunting): picked mon.noname-h con 0x7fe8940a3b10 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:15.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 10 monclient(hunting): picked mon.noname-d con 0x7fe8940a4600 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:15.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 10 monclient(hunting): picked mon.noname-g con 0x7fe89409bbc0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:15.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89404ed70 con 0x7fe89409bbc0 2024-01-24T23:23:15.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8940a6ae0 con 0x7fe8940a3b10 2024-01-24T23:23:15.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89409af00 con 0x7fe8940a4600 2024-01-24T23:23:15.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:15.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 10 monclient(hunting): authenticate will time out at 1074.205903s 2024-01-24T23:23:15.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe88bfff640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fe8940a4600 0x7fe89413bfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:34136/0 (socket says 172.21.15.115:34136) 2024-01-24T23:23:15.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe88bfff640 1 -- 172.21.15.115:0/1014219606 learned_addr learned my addr 172.21.15.115:0/1014219606 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:15.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 1 -- 172.21.15.115:0/1014219606 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 836467822 0 0) 0x7fe89409af00 con 0x7fe8940a4600 2024-01-24T23:23:15.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.292 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 10 monclient(hunting): my global_id is 4947 2024-01-24T23:23:15.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 1 -- 172.21.15.115:0/1014219606 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe880003530 con 0x7fe8940a4600 2024-01-24T23:23:15.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 1 -- 172.21.15.115:0/1014219606 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1217544116 0 0) 0x7fe89404ed70 con 0x7fe89409bbc0 2024-01-24T23:23:15.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 10 monclient(hunting): my global_id is 4872 2024-01-24T23:23:15.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 1 -- 172.21.15.115:0/1014219606 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe89409af00 con 0x7fe89409bbc0 2024-01-24T23:23:15.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 1 -- 172.21.15.115:0/1014219606 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4286033301 0 0) 0x7fe880003530 con 0x7fe8940a4600 2024-01-24T23:23:15.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 1 -- 172.21.15.115:0/1014219606 >> v1:172.21.15.132:6791/0 conn(0x7fe8940a3b10 legacy=0x7fe8940a3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 1 -- 172.21.15.115:0/1014219606 >> v1:172.21.15.132:6790/0 conn(0x7fe89409bbc0 legacy=0x7fe89413f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:15.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 1 monclient: found mon.noname-d 2024-01-24T23:23:15.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:15.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:23:15.293 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 1 -- 172.21.15.115:0/1014219606 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89409b1e0 con 0x7fe8940a4600 2024-01-24T23:23:15.294 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 10 monclient: _finish_auth 0 2024-01-24T23:23:15.294 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:15.294 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:15.294 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 5 monclient: authenticate success, global_id 4947 2024-01-24T23:23:15.294 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 1 -- 172.21.15.115:0/1014219606 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe87c0017c0 con 0x7fe8940a4600 2024-01-24T23:23:15.294 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:15.294 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.294 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:23:15.294 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 10 monclient: dump: 2024-01-24T23:23:15.294 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.294 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.294 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.294 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.294 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.295 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.295 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.295 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.295 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.295 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.295 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.295 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.295 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.295 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.295 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.295 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:15.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 1 -- 172.21.15.115:0/1014219606 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe87c001aa0 con 0x7fe8940a4600 2024-01-24T23:23:15.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:15.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:15.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 4 set_mon_vals no callback set 2024-01-24T23:23:15.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 1 -- 172.21.15.115:0/1014219606 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe87c004bf0 con 0x7fe8940a4600 2024-01-24T23:23:15.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:15.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe892ffd640 10 monclient: dump: 2024-01-24T23:23:15.296 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.296 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.296 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.296 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.296 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.296 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.296 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.296 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.296 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.296 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.296 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.297 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.297 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.297 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.297 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.297 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 10 monclient: shutdown 2024-01-24T23:23:15.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:15.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 1 -- 172.21.15.115:0/1014219606 >> v1:172.21.15.115:6792/0 conn(0x7fe8940a4600 legacy=0x7fe89413bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 1 -- 172.21.15.115:0/1014219606 shutdown_connections 2024-01-24T23:23:15.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 1 -- 172.21.15.115:0/1014219606 >> 172.21.15.115:0/1014219606 conn(0x7fe8940a98a0 msgr2=0x7fe8940a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:15.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 1 -- 172.21.15.115:0/1014219606 shutdown_connections 2024-01-24T23:23:15.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 1 -- 172.21.15.115:0/1014219606 wait complete. 2024-01-24T23:23:15.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 10 monclient: build_initial_monmap 2024-01-24T23:23:15.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.280+0000 7fe89a028640 10 monclient: monmap: 2024-01-24T23:23:15.297 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:15.298 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe89a028640 1 Processor -- start 2024-01-24T23:23:15.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe89a028640 1 -- start start 2024-01-24T23:23:15.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe89a028640 10 monclient: init 2024-01-24T23:23:15.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe89a028640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:15.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe89a028640 10 monclient: _add_conns ranks=[2,5,7,8,0,1,3,6,4] 2024-01-24T23:23:15.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe89a028640 10 monclient(hunting): picked mon.noname-b con 0x7fe89409bbc0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:15.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe89a028640 10 monclient(hunting): picked mon.noname-h con 0x7fe8940a3b10 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:15.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe89a028640 10 monclient(hunting): picked mon.noname-i con 0x7fe8940a4600 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:15.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe89a028640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89406bd40 con 0x7fe89409bbc0 2024-01-24T23:23:15.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe89a028640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89409c850 con 0x7fe8940a3b10 2024-01-24T23:23:15.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe89a028640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8940a58f0 con 0x7fe8940a4600 2024-01-24T23:23:15.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe89a028640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:15.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe89a028640 10 monclient(hunting): authenticate will time out at 1074.208901s 2024-01-24T23:23:15.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe8937fe640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fe89409bbc0 0x7fe8941bba10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:47586/0 (socket says 172.21.15.115:47586) 2024-01-24T23:23:15.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe8937fe640 1 -- 172.21.15.115:0/1362616442 learned_addr learned my addr 172.21.15.115:0/1362616442 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:15.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1729958583 0 0) 0x7fe89406bd40 con 0x7fe89409bbc0 2024-01-24T23:23:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient(hunting): my global_id is 84143 2024-01-24T23:23:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe874003530 con 0x7fe89409bbc0 2024-01-24T23:23:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1653347341 0 0) 0x7fe874003530 con 0x7fe89409bbc0 2024-01-24T23:23:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe89406bd40 con 0x7fe89409bbc0 2024-01-24T23:23:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe884001ac0 con 0x7fe89409bbc0 2024-01-24T23:23:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:23:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient(hunting): dump: 2024-01-24T23:23:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.301 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.301 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.301 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.301 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.301 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.301 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.301 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.301 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.301 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.301 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.301 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.301 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:15.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323444016 0 0) 0x7fe8940a58f0 con 0x7fe8940a4600 2024-01-24T23:23:15.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient(hunting): my global_id is 4966 2024-01-24T23:23:15.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe874003530 con 0x7fe8940a4600 2024-01-24T23:23:15.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1140477093 0 0) 0x7fe89409c850 con 0x7fe8940a3b10 2024-01-24T23:23:15.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient(hunting): my global_id is 4928 2024-01-24T23:23:15.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8940a58f0 con 0x7fe8940a3b10 2024-01-24T23:23:15.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 492911506 0 0) 0x7fe89406bd40 con 0x7fe89409bbc0 2024-01-24T23:23:15.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 >> v1:172.21.15.132:6792/0 conn(0x7fe8940a4600 legacy=0x7fe8941d3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 >> v1:172.21.15.132:6791/0 conn(0x7fe8940a3b10 legacy=0x7fe8941d0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:23:15.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 monclient: found mon.b 2024-01-24T23:23:15.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:15.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:15.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8941a70e0 con 0x7fe89409bbc0 2024-01-24T23:23:15.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient: _finish_auth 0 2024-01-24T23:23:15.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:15.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:15.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe89a028640 5 monclient: authenticate success, global_id 84143 2024-01-24T23:23:15.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe89a028640 10 monclient: _renew_subs 2024-01-24T23:23:15.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe89a028640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:15.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe89a028640 1 -- 172.21.15.115:0/1362616442 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8941a8b30 con 0x7fe89409bbc0 2024-01-24T23:23:15.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe89a028640 10 monclient: _renew_subs 2024-01-24T23:23:15.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe89a028640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:15.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe89a028640 1 -- 172.21.15.115:0/1362616442 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8941a7e60 con 0x7fe89409bbc0 2024-01-24T23:23:15.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe884003950 con 0x7fe89409bbc0 2024-01-24T23:23:15.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:15.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe884003e80 con 0x7fe89409bbc0 2024-01-24T23:23:15.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:15.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe8927fc640 4 set_mon_vals no callback set 2024-01-24T23:23:15.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient: dump: 2024-01-24T23:23:15.304 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.304 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.304 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.304 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.304 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.304 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.304 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.304 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.304 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.304 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.305 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.305 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.305 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.305 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.305 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.305 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe89a028640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:15.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe89a028640 1 -- 172.21.15.115:0/1362616442 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe8780065d0 con 0x7fe878002870 2024-01-24T23:23:15.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fe884003fe0 con 0x7fe89409bbc0 2024-01-24T23:23:15.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(48..48 src has 1..48) v4 ==== 3994+0+0 (unknown 1916735388 0 0) 0x7fe8840438f0 con 0x7fe89409bbc0 2024-01-24T23:23:15.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019812047 0 0) 0x7fe8780065d0 con 0x7fe878002870 2024-01-24T23:23:15.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient(hunting): my global_id is 4956 2024-01-24T23:23:15.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe89406bd40 con 0x7fe878002870 2024-01-24T23:23:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe893fff640 10 monclient: get_auth_request con 0x7fe8740372c0 auth_method 0 2024-01-24T23:23:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 32036729 0 0) 0x7fe89406bd40 con 0x7fe878002870 2024-01-24T23:23:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe8780065d0 con 0x7fe878002870 2024-01-24T23:23:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe87c001780 con 0x7fe878002870 2024-01-24T23:23:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient: dump: 2024-01-24T23:23:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.307 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.307 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.307 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.307 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.307 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.307 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.307 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.307 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2251585116 0 0) 0x7fe8780065d0 con 0x7fe878002870 2024-01-24T23:23:15.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 --> v1:172.21.15.115:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe87800a120 con 0x7fe878002870 2024-01-24T23:23:15.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fe87c0047f0 con 0x7fe878002870 2024-01-24T23:23:15.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:15.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:15.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 >> v1:172.21.15.115:6792/0 conn(0x7fe878002870 legacy=0x7fe878004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.284+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 >> v1:172.21.15.115:6792/0 conn(0x7fe878002870 legacy=0x7fe878004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:15.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe89a028640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:23:15.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe89a028640 1 -- 172.21.15.115:0/1362616442 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe878009e10 con 0x7fe8780090d0 2024-01-24T23:23:15.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2333112123 0 0) 0x7fe878009e10 con 0x7fe8780090d0 2024-01-24T23:23:15.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe890ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe890ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe890ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe890ff9640 10 monclient(hunting): my global_id is 4965 2024-01-24T23:23:15.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8780065d0 con 0x7fe8780090d0 2024-01-24T23:23:15.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2392573453 0 0) 0x7fe8780065d0 con 0x7fe8780090d0 2024-01-24T23:23:15.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe878009e10 con 0x7fe8780090d0 2024-01-24T23:23:15.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe87c005340 con 0x7fe8780090d0 2024-01-24T23:23:15.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe890ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe890ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:15.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe890ff9640 10 monclient: dump: 2024-01-24T23:23:15.310 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.310 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.310 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.310 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.310 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.310 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.310 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.310 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.310 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.310 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.310 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.310 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.311 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.311 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.311 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.311 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1739287300 0 0) 0x7fe878009e10 con 0x7fe8780090d0 2024-01-24T23:23:15.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 --> v1:172.21.15.115:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe87800c310 con 0x7fe8780090d0 2024-01-24T23:23:15.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 1777205094) 0x7fe87c007590 con 0x7fe8780090d0 2024-01-24T23:23:15.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe890ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:23:15.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe890ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:15.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 >> v1:172.21.15.115:6792/0 conn(0x7fe8780090d0 legacy=0x7fe878009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe890ff9640 1 -- 172.21.15.115:0/1362616442 >> v1:172.21.15.115:6792/0 conn(0x7fe8780090d0 legacy=0x7fe878009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:15.311 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:23:15.312 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:15.312 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:15.312 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:23:15.312 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 16, 2024-01-24T23:23:15.312 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:23:15.312 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:23:15.312 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:23:15.312 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:23:15.312 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:23:15.312 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:23:15.312 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:23:15.312 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:23:15.312 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:23:15.313 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:23:15.313 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:15.313 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:23:15.313 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:15.313 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:23:15.313 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:23:15.313 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:15.313 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:15.313 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:15.313 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:15.313 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:15.313 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:15.313 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:15.313 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:15.313 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:15.314 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:15.314 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:15.314 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:15.314 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:23:15.314 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:23:15.314 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:15.314 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:15.314 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:15.314 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:15.314 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:15.314 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:15.314 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:15.314 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:15.314 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:15.315 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:15.315 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:15.315 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.315 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:23:15.315 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:23:15.315 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:23:15.315 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:23:15.315 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:23:15.315 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:23:15.315 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:15.315 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:15.315 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:23:15.315 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:23:15.316 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:23:15.316 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:23:15.316 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:23:15.316 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:23:15.316 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:23:15.316 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:15.316 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:23:15.316 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:15.316 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:15.316 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:15.316 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:15.316 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:15.316 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:15.316 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:15.317 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:15.317 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:15.317 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:15.317 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:15.317 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:15.317 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:23:15.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.317 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:23:15.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.317 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:15.317 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:15.317 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.317 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.317 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.318 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:23:15.318 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.318 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.318 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.318 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:23:15.318 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:23:15.318 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.318 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.318 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.318 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:15.318 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:15.319 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.319 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.319 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.319 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:23:15.319 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.319 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.319 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.319 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:23:15.319 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:23:15.319 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.319 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.319 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.320 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:15.320 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:15.320 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.320 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.320 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.320 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:23:15.320 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.320 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.320 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.320 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:23:15.320 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:23:15.321 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.321 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.321 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.321 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:15.321 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:15.321 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.321 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.321 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.321 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:23:15.321 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.321 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.321 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.322 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:23:15.322 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:23:15.322 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.322 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.322 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.322 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:15.322 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:15.322 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.322 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.323 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.323 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:23:15.323 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.323 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.323 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.323 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:23:15.323 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:23:15.323 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.323 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.323 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.324 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:15.324 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:15.324 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.324 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.324 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.324 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:23:15.324 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.324 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.324 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:23:15.324 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:23:15.324 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.325 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.325 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.325 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:15.325 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:15.325 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.325 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.325 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.325 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:23:15.325 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.325 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.325 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.326 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:23:15.326 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:23:15.326 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.326 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.326 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.326 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:15.326 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:15.326 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.326 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.326 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.327 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:23:15.327 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.327 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.327 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.327 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:23:15.327 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:23:15.327 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.327 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.327 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.327 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:15.327 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:15.327 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.328 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.328 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.328 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:23:15.328 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.328 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.328 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.328 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:23:15.328 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:23:15.328 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.328 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.328 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.328 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.329 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.329 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:23:15.329 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:23:15.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.329 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:15.329 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:15.329 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:15.329 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.329 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:15.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:23:15.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.329 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:15.329 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:15.330 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:15.330 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.330 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.330 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:23:15.330 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:23:15.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe89a028640 1 -- 172.21.15.115:0/1362616442 >> v1:172.21.15.132:6812/1835401922 conn(0x7fe8740372c0 legacy=0x7fe874039780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe89a028640 10 monclient: shutdown 2024-01-24T23:23:15.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe89a028640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:15.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe89a028640 1 -- 172.21.15.115:0/1362616442 >> v1:172.21.15.115:6790/0 conn(0x7fe89409bbc0 legacy=0x7fe8941bba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe893fff640 1 -- 172.21.15.115:0/1362616442 reap_dead start 2024-01-24T23:23:15.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe893fff640 1 -- 172.21.15.115:0/1362616442 reap_dead start 2024-01-24T23:23:15.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe89a028640 1 -- 172.21.15.115:0/1362616442 shutdown_connections 2024-01-24T23:23:15.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe89a028640 1 -- 172.21.15.115:0/1362616442 >> 172.21.15.115:0/1362616442 conn(0x7fe8940a98a0 msgr2=0x7fe8941be990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:15.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe89a028640 1 -- 172.21.15.115:0/1362616442 shutdown_connections 2024-01-24T23:23:15.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.292+0000 7fe89a028640 1 -- 172.21.15.115:0/1362616442 wait complete. 2024-01-24T23:23:15.332 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:23:15.332 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-01-24T23:23:15.333 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-01-24T23:23:15.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.336+0000 7feb6a4be640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-01-24T23:23:15.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.336+0000 7feb6a4be640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-01-24T23:23:15.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.336+0000 7feb6a4be640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:23:15.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.336+0000 7feb6a4be640 1 -- 172.21.15.132:0/2811665744 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "hdd"} v 0) v1 -- 0x7feb44000bc0 con 0x7feb64071980 2024-01-24T23:23:15.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.336+0000 7feb60ff9640 1 -- 172.21.15.132:0/2811665744 <== mon.4 v1:172.21.15.115:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "hdd"}]=0 v48) v1 ==== 90+0+8 (unknown 251225796 0 1286160932) 0x7feb580144e0 con 0x7feb64071980 2024-01-24T23:23:15.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.336+0000 7feb60ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-01-24T23:23:15.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.336+0000 7feb60ff9640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:23:15.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.336+0000 7feb6a4be640 1 -- 172.21.15.132:0/2811665744 >> v1:172.21.15.132:6812/1835401922 conn(0x7feb50037dc0 legacy=0x7feb5003a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.336+0000 7feb6a4be640 10 monclient: shutdown 2024-01-24T23:23:15.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.336+0000 7feb6a4be640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:15.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.336+0000 7feb6a4be640 1 -- 172.21.15.132:0/2811665744 >> v1:172.21.15.115:6791/0 conn(0x7feb64071980 legacy=0x7feb64134570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.336+0000 7feb6a4be640 1 -- 172.21.15.132:0/2811665744 shutdown_connections 2024-01-24T23:23:15.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.336+0000 7feb6a4be640 1 -- 172.21.15.132:0/2811665744 >> 172.21.15.132:0/2811665744 conn(0x7feb640a2ab0 msgr2=0x7feb6412d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:15.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.336+0000 7feb6a4be640 1 -- 172.21.15.132:0/2811665744 shutdown_connections 2024-01-24T23:23:15.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.336+0000 7feb6a4be640 1 -- 172.21.15.132:0/2811665744 wait complete. 2024-01-24T23:23:15.351 INFO:tasks.workunit.client.0.smithi132.stdout:foo-hdd 2024-01-24T23:23:15.351 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule ls-by-class hdd 2024-01-24T23:23:15.351 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false grep foo-ssd 2024-01-24T23:23:15.352 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:23:15.352 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep foo-ssd 2024-01-24T23:23:15.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35e3883640 10 monclient: get_monmap_and_config 2024-01-24T23:23:15.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35e3883640 10 monclient: build_initial_monmap 2024-01-24T23:23:15.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35e3883640 10 monclient: monmap: 2024-01-24T23:23:15.443 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:15.443 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.443 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:15.443918+0000 2024-01-24T23:23:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:15.443918+0000 2024-01-24T23:23:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:15.444 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:15.444 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35e3883640 1 Processor -- start 2024-01-24T23:23:15.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35e3883640 1 -- start start 2024-01-24T23:23:15.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35e3883640 10 monclient: init 2024-01-24T23:23:15.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35e3883640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:15.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35e3883640 10 monclient: _add_conns ranks=[5,2,1,6,3,0,7,8,4] 2024-01-24T23:23:15.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35e3883640 10 monclient(hunting): picked mon.noname-h con 0x7f35dc09f970 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:15.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35e3883640 10 monclient(hunting): picked mon.noname-b con 0x7f35dc134c90 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:15.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35e3883640 10 monclient(hunting): picked mon.noname-f con 0x7f35dc13b210 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:15.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35e3883640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35dc04ed70 con 0x7f35dc13b210 2024-01-24T23:23:15.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35e3883640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35dc09d990 con 0x7f35dc134c90 2024-01-24T23:23:15.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35e3883640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35dc09dc70 con 0x7f35dc09f970 2024-01-24T23:23:15.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35e3883640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:15.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35e3883640 10 monclient(hunting): authenticate will time out at 1074.367218s 2024-01-24T23:23:15.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35e0e1e640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f35dc134c90 0x7f35dc139af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:47588/0 (socket says 172.21.15.115:47588) 2024-01-24T23:23:15.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35e0e1e640 1 -- 172.21.15.115:0/3461229897 learned_addr learned my addr 172.21.15.115:0/3461229897 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:15.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35d3fff640 1 -- 172.21.15.115:0/3461229897 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 894770021 0 0) 0x7f35dc09d990 con 0x7f35dc134c90 2024-01-24T23:23:15.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35d3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35d3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35d3fff640 10 monclient(hunting): my global_id is 84152 2024-01-24T23:23:15.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35d3fff640 1 -- 172.21.15.115:0/3461229897 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35c8003530 con 0x7f35dc134c90 2024-01-24T23:23:15.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35d3fff640 1 -- 172.21.15.115:0/3461229897 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2239026142 0 0) 0x7f35c8003530 con 0x7f35dc134c90 2024-01-24T23:23:15.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35d3fff640 1 -- 172.21.15.115:0/3461229897 >> v1:172.21.15.132:6791/0 conn(0x7f35dc09f970 legacy=0x7f35dc09fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35d3fff640 1 -- 172.21.15.115:0/3461229897 >> v1:172.21.15.132:6789/0 conn(0x7f35dc13b210 legacy=0x7f35dc13d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35d3fff640 10 monclient: _finish_hunting 0 2024-01-24T23:23:15.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35d3fff640 1 monclient: found mon.noname-b 2024-01-24T23:23:15.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35d3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:15.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35d3fff640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:23:15.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35d3fff640 1 -- 172.21.15.115:0/3461229897 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc09df50 con 0x7f35dc134c90 2024-01-24T23:23:15.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35d3fff640 10 monclient: _finish_auth 0 2024-01-24T23:23:15.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35d3fff640 10 monclient: _check_auth_tickets 2024-01-24T23:23:15.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35d3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:15.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35d3fff640 1 -- 172.21.15.115:0/3461229897 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f35cc0017c0 con 0x7f35dc134c90 2024-01-24T23:23:15.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35e3883640 5 monclient: authenticate success, global_id 84152 2024-01-24T23:23:15.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35e3883640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:15.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35d3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35d3fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:23:15.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.440+0000 7f35d3fff640 10 monclient: dump: 2024-01-24T23:23:15.450 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.450 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.450 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.450 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.450 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.450 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.450 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.450 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.450 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.450 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.450 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.450 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.451 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.451 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.451 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.451 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:15.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d3fff640 1 -- 172.21.15.115:0/3461229897 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f35cc001a70 con 0x7f35dc134c90 2024-01-24T23:23:15.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d3fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:15.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d3fff640 1 -- 172.21.15.115:0/3461229897 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f35cc004bf0 con 0x7f35dc134c90 2024-01-24T23:23:15.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:15.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d3fff640 10 monclient: dump: 2024-01-24T23:23:15.451 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.451 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.451 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.451 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.452 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.452 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.452 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.452 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.452 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.452 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.452 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.452 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.452 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.452 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.452 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.452 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:15.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 4 set_mon_vals no callback set 2024-01-24T23:23:15.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 10 monclient: shutdown 2024-01-24T23:23:15.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 1 -- 172.21.15.115:0/3461229897 >> v1:172.21.15.115:6790/0 conn(0x7f35dc134c90 legacy=0x7f35dc139af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 1 -- 172.21.15.115:0/3461229897 shutdown_connections 2024-01-24T23:23:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 1 -- 172.21.15.115:0/3461229897 >> 172.21.15.115:0/3461229897 conn(0x7f35dc0a6490 msgr2=0x7f35dc0a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 1 -- 172.21.15.115:0/3461229897 shutdown_connections 2024-01-24T23:23:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 1 -- 172.21.15.115:0/3461229897 wait complete. 2024-01-24T23:23:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 10 monclient: build_initial_monmap 2024-01-24T23:23:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 10 monclient: monmap: 2024-01-24T23:23:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:15.454 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 1 Processor -- start 2024-01-24T23:23:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 1 -- start start 2024-01-24T23:23:15.454 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 10 monclient: init 2024-01-24T23:23:15.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:15.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 10 monclient: _add_conns ranks=[1,8,6,7,4,2,0,3,5] 2024-01-24T23:23:15.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 10 monclient(hunting): picked mon.noname-f con 0x7f35dc09f970 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:15.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 10 monclient(hunting): picked mon.noname-e con 0x7f35dc13b210 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:15.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 10 monclient(hunting): picked mon.noname-d con 0x7f35dc070440 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:15.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35dc067590 con 0x7f35dc09f970 2024-01-24T23:23:15.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35dc067770 con 0x7f35dc070440 2024-01-24T23:23:15.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35dc13a700 con 0x7f35dc13b210 2024-01-24T23:23:15.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 10 monclient(hunting): authenticate will time out at 1074.370208s 2024-01-24T23:23:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e1e20640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f35dc070440 0x7f35dc1d15f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:34172/0 (socket says 172.21.15.115:34172) 2024-01-24T23:23:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e0e1e640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f35dc13b210 0x7f35dc1c9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:52904/0 (socket says 172.21.15.115:52904) 2024-01-24T23:23:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e1e20640 1 -- 172.21.15.115:0/434023911 learned_addr learned my addr 172.21.15.115:0/434023911 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596337558 0 0) 0x7f35dc13a700 con 0x7f35dc13b210 2024-01-24T23:23:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient(hunting): my global_id is 84140 2024-01-24T23:23:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35c0003530 con 0x7f35dc13b210 2024-01-24T23:23:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3996795066 0 0) 0x7f35dc067770 con 0x7f35dc070440 2024-01-24T23:23:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient(hunting): my global_id is 4974 2024-01-24T23:23:15.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35dc13a700 con 0x7f35dc070440 2024-01-24T23:23:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3721193105 0 0) 0x7f35dc13a700 con 0x7f35dc070440 2024-01-24T23:23:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f35dc067770 con 0x7f35dc070440 2024-01-24T23:23:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 87819550 0 0) 0x7f35c0003530 con 0x7f35dc13b210 2024-01-24T23:23:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f35dc13a700 con 0x7f35dc13b210 2024-01-24T23:23:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f35d4001c40 con 0x7f35dc070440 2024-01-24T23:23:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:23:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient(hunting): dump: 2024-01-24T23:23:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.463 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1459457584 0 0) 0x7f35dc067590 con 0x7f35dc09f970 2024-01-24T23:23:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient(hunting): my global_id is 4978 2024-01-24T23:23:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35c0003530 con 0x7f35dc09f970 2024-01-24T23:23:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f35cc001780 con 0x7f35dc13b210 2024-01-24T23:23:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient(hunting): dump: 2024-01-24T23:23:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.465 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2025561300 0 0) 0x7f35dc067770 con 0x7f35dc070440 2024-01-24T23:23:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 >> v1:172.21.15.115:6793/0 conn(0x7f35dc13b210 legacy=0x7f35dc1c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 >> v1:172.21.15.132:6789/0 conn(0x7f35dc09f970 legacy=0x7f35dc1c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:23:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 1 monclient: found mon.d 2024-01-24T23:23:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:23:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc071d70 con 0x7f35dc070440 2024-01-24T23:23:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient: _finish_auth 0 2024-01-24T23:23:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:23:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 5 monclient: authenticate success, global_id 4974 2024-01-24T23:23:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 10 monclient: _renew_subs 2024-01-24T23:23:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:23:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 1 -- 172.21.15.115:0/434023911 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35dc136860 con 0x7f35dc070440 2024-01-24T23:23:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 10 monclient: _renew_subs 2024-01-24T23:23:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:23:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 1 -- 172.21.15.115:0/434023911 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f35dc1ca1b0 con 0x7f35dc070440 2024-01-24T23:23:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f35d4001e70 con 0x7f35dc070440 2024-01-24T23:23:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f35d4003f90 con 0x7f35dc070440 2024-01-24T23:23:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d37fe640 4 set_mon_vals no callback set 2024-01-24T23:23:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35d1ffb640 10 monclient: dump: 2024-01-24T23:23:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.467 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.444+0000 7f35e3883640 1 -- 172.21.15.115:0/434023911 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f35bc0065d0 con 0x7f35bc002870 2024-01-24T23:23:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.448+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f35d4004170 con 0x7f35dc070440 2024-01-24T23:23:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.448+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(48..48 src has 1..48) v4 ==== 3994+0+0 (unknown 1916735388 0 0) 0x7f35d4043be0 con 0x7f35dc070440 2024-01-24T23:23:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.448+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3694766896 0 0) 0x7f35bc0065d0 con 0x7f35bc002870 2024-01-24T23:23:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.448+0000 7f35d1ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.448+0000 7f35d1ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.448+0000 7f35d1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.448+0000 7f35d1ffb640 10 monclient(hunting): my global_id is 84149 2024-01-24T23:23:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.448+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35dc067770 con 0x7f35bc002870 2024-01-24T23:23:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.448+0000 7f35e0e1e640 10 monclient: get_auth_request con 0x7f35c00378e0 auth_method 0 2024-01-24T23:23:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.448+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2791527181 0 0) 0x7f35dc067770 con 0x7f35bc002870 2024-01-24T23:23:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.448+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f35bc0065d0 con 0x7f35bc002870 2024-01-24T23:23:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.448+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f35d8001870 con 0x7f35bc002870 2024-01-24T23:23:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.448+0000 7f35d1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.448+0000 7f35d1ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.448+0000 7f35d1ffb640 10 monclient: dump: 2024-01-24T23:23:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.470 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.470 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.448+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2964328090 0 0) 0x7f35bc0065d0 con 0x7f35bc002870 2024-01-24T23:23:15.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.448+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 --> v1:172.21.15.115:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35bc00a120 con 0x7f35bc002870 2024-01-24T23:23:15.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.448+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f35d8003600 con 0x7f35bc002870 2024-01-24T23:23:15.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.448+0000 7f35d1ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:15.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.448+0000 7f35d1ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:15.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.448+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 >> v1:172.21.15.115:6793/0 conn(0x7f35bc002870 legacy=0x7f35bc004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.448+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 >> v1:172.21.15.115:6793/0 conn(0x7f35bc002870 legacy=0x7f35bc004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:15.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.452+0000 7f35e3883640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:23:15.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.452+0000 7f35e3883640 1 -- 172.21.15.115:0/434023911 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f35bc009e10 con 0x7f35bc0090d0 2024-01-24T23:23:15.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.452+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4094797132 0 0) 0x7f35bc009e10 con 0x7f35bc0090d0 2024-01-24T23:23:15.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.452+0000 7f35d1ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.452+0000 7f35d1ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.470 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.452+0000 7f35d1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.452+0000 7f35d1ffb640 10 monclient(hunting): my global_id is 84158 2024-01-24T23:23:15.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.452+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35bc0065d0 con 0x7f35bc0090d0 2024-01-24T23:23:15.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.452+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2360104354 0 0) 0x7f35bc0065d0 con 0x7f35bc0090d0 2024-01-24T23:23:15.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.452+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f35bc009e10 con 0x7f35bc0090d0 2024-01-24T23:23:15.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.452+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f35d8003ca0 con 0x7f35bc0090d0 2024-01-24T23:23:15.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.452+0000 7f35d1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.452+0000 7f35d1ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:15.471 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.452+0000 7f35d1ffb640 10 monclient: dump: 2024-01-24T23:23:15.471 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.471 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.471 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.471 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.471 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.471 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.471 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.472 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.472 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.472 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.472 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.472 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.472 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.472 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.472 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.472 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.452+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1528424814 0 0) 0x7f35bc009e10 con 0x7f35bc0090d0 2024-01-24T23:23:15.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.452+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 --> v1:172.21.15.115:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f35bc00c310 con 0x7f35bc0090d0 2024-01-24T23:23:15.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.452+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 2674480831) 0x7f35d8006040 con 0x7f35bc0090d0 2024-01-24T23:23:15.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.452+0000 7f35d1ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:23:15.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.452+0000 7f35d1ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:15.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.452+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 >> v1:172.21.15.115:6793/0 conn(0x7f35bc0090d0 legacy=0x7f35bc009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.452+0000 7f35d1ffb640 1 -- 172.21.15.115:0/434023911 >> v1:172.21.15.115:6793/0 conn(0x7f35bc0090d0 legacy=0x7f35bc009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:15.473 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:23:15.473 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:15.473 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:15.473 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:23:15.473 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 16, 2024-01-24T23:23:15.473 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:23:15.473 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:23:15.473 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:23:15.473 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:23:15.474 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:23:15.474 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:23:15.474 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:23:15.474 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:23:15.474 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:23:15.474 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:23:15.474 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:15.474 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 1, 2024-01-24T23:23:15.474 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:15.474 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:23:15.474 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:23:15.474 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:15.474 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:15.474 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:15.474 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:15.475 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:15.475 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:15.475 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:15.475 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:15.475 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:15.475 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:15.475 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:15.475 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:15.475 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:23:15.475 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:23:15.475 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:15.475 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:15.475 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:15.475 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:15.475 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:15.476 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:15.476 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:15.476 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:15.476 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:15.476 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:15.476 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:15.476 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.476 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:23:15.476 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:23:15.476 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:23:15.476 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:23:15.476 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:23:15.476 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:23:15.477 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:15.477 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:15.477 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:23:15.477 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:23:15.477 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:23:15.477 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:23:15.477 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:23:15.477 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:23:15.477 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:23:15.477 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:15.477 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:23:15.477 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:15.477 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:15.477 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:15.477 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:15.477 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:15.478 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:15.478 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:15.478 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:15.478 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:15.478 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:15.478 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:15.478 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:15.478 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:23:15.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.478 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:23:15.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.478 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:15.478 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:15.478 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.479 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.479 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.479 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:23:15.479 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.479 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.479 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.479 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:23:15.479 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:23:15.479 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.479 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.479 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.480 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:15.480 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:15.480 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.480 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.480 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.480 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:23:15.480 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.480 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.480 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.480 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:23:15.480 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:23:15.480 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.481 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.481 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.481 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:15.481 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:15.481 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.481 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.481 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.481 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:23:15.481 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.481 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.481 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.482 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:23:15.482 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:23:15.482 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.482 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.482 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.482 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:15.482 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:15.482 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.482 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.482 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.482 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:23:15.483 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.483 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.483 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.483 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:23:15.483 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:23:15.483 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.483 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.483 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.483 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:15.483 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:15.483 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.483 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.484 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.484 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:23:15.484 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.484 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.484 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.484 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:23:15.484 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:23:15.484 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.484 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.484 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.485 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:15.485 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:15.485 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.485 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.485 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.485 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:23:15.485 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.485 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.485 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.485 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:23:15.485 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:23:15.485 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.485 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.486 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.486 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:15.486 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:15.486 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.486 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.486 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.486 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:23:15.486 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.486 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.486 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.487 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:23:15.487 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:23:15.487 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.487 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.487 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.487 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:15.487 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:15.487 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.487 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.487 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.488 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:23:15.488 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.488 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.488 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.488 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:23:15.488 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:23:15.488 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.488 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.488 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.488 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:15.488 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:15.488 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.489 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.489 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.489 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:23:15.489 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.489 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.489 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.489 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:23:15.489 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:23:15.489 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.489 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.489 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.489 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.489 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.490 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:23:15.490 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:23:15.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.490 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:15.490 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:15.490 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:15.490 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.490 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:15.490 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:23:15.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.490 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:15.490 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:15.490 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:15.490 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.491 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.491 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:23:15.491 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:23:15.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.456+0000 7f35e3883640 1 -- 172.21.15.115:0/434023911 >> v1:172.21.15.132:6812/1835401922 conn(0x7f35c00378e0 legacy=0x7f35c0039da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.456+0000 7f35e3883640 10 monclient: shutdown 2024-01-24T23:23:15.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.456+0000 7f35e3883640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:15.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.456+0000 7f35e3883640 1 -- 172.21.15.115:0/434023911 >> v1:172.21.15.115:6792/0 conn(0x7f35dc070440 legacy=0x7f35dc1d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.456+0000 7f35e1e20640 1 -- 172.21.15.115:0/434023911 reap_dead start 2024-01-24T23:23:15.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.456+0000 7f35e1e20640 1 -- 172.21.15.115:0/434023911 reap_dead start 2024-01-24T23:23:15.492 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.456+0000 7f35e3883640 1 -- 172.21.15.115:0/434023911 shutdown_connections 2024-01-24T23:23:15.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.456+0000 7f35e3883640 1 -- 172.21.15.115:0/434023911 >> 172.21.15.115:0/434023911 conn(0x7f35dc0a6490 msgr2=0x7f35dc074860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:15.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.456+0000 7f35e3883640 1 -- 172.21.15.115:0/434023911 shutdown_connections 2024-01-24T23:23:15.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.456+0000 7f35e3883640 1 -- 172.21.15.115:0/434023911 wait complete. 2024-01-24T23:23:15.493 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:23:15.493 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-01-24T23:23:15.493 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-01-24T23:23:15.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.468+0000 7fd83f96b640 10 monclient: get_monmap_and_config 2024-01-24T23:23:15.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.468+0000 7fd83f96b640 10 monclient: build_initial_monmap 2024-01-24T23:23:15.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.468+0000 7fd83f96b640 10 monclient: monmap: 2024-01-24T23:23:15.500 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:15.500 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.500 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:15.471193+0000 2024-01-24T23:23:15.500 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:15.471193+0000 2024-01-24T23:23:15.500 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:15.500 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:15.500 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:15.501 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:15.501 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:15.501 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:15.501 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:15.501 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:15.501 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:15.501 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:15.501 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:15.501 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:15.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.468+0000 7fd83f96b640 1 Processor -- start 2024-01-24T23:23:15.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.468+0000 7fd83f96b640 1 -- start start 2024-01-24T23:23:15.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.468+0000 7fd83f96b640 10 monclient: init 2024-01-24T23:23:15.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:15.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 10 monclient: _add_conns ranks=[0,3,8,5,1,4,7,2,6] 2024-01-24T23:23:15.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 10 monclient(hunting): picked mon.noname-a con 0x7fd838133b50 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:15.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 10 monclient(hunting): picked mon.noname-g con 0x7fd83809ec20 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:15.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 10 monclient(hunting): picked mon.noname-e con 0x7fd83813a1d0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:15.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd838064700 con 0x7fd838133b50 2024-01-24T23:23:15.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd83809d9f0 con 0x7fd83809ec20 2024-01-24T23:23:15.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd83809dc60 con 0x7fd83813a1d0 2024-01-24T23:23:15.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:15.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 10 monclient(hunting): authenticate will time out at 1082.796616s 2024-01-24T23:23:15.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83cf06640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fd83809ec20 0x7fd838138ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:36394/0 (socket says 172.21.15.132:36394) 2024-01-24T23:23:15.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83cf06640 1 -- 172.21.15.132:0/2323732497 learned_addr learned my addr 172.21.15.132:0/2323732497 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:15.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 1 -- 172.21.15.132:0/2323732497 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3515734641 0 0) 0x7fd83809d9f0 con 0x7fd83809ec20 2024-01-24T23:23:15.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 10 monclient(hunting): my global_id is 4881 2024-01-24T23:23:15.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 1 -- 172.21.15.132:0/2323732497 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd824003530 con 0x7fd83809ec20 2024-01-24T23:23:15.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 1 -- 172.21.15.132:0/2323732497 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 862624271 0 0) 0x7fd83809dc60 con 0x7fd83813a1d0 2024-01-24T23:23:15.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 10 monclient(hunting): my global_id is 84167 2024-01-24T23:23:15.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 1 -- 172.21.15.132:0/2323732497 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd83809d9f0 con 0x7fd83813a1d0 2024-01-24T23:23:15.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 1 -- 172.21.15.132:0/2323732497 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3761260647 0 0) 0x7fd838064700 con 0x7fd838133b50 2024-01-24T23:23:15.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 10 monclient(hunting): my global_id is 4932 2024-01-24T23:23:15.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 1 -- 172.21.15.132:0/2323732497 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd83809dc60 con 0x7fd838133b50 2024-01-24T23:23:15.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 1 -- 172.21.15.132:0/2323732497 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2159824984 0 0) 0x7fd824003530 con 0x7fd83809ec20 2024-01-24T23:23:15.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 1 -- 172.21.15.132:0/2323732497 >> v1:172.21.15.115:6793/0 conn(0x7fd83813a1d0 legacy=0x7fd83813c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 1 -- 172.21.15.132:0/2323732497 >> v1:172.21.15.115:6789/0 conn(0x7fd838133b50 legacy=0x7fd838133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 10 monclient: _finish_hunting 0 2024-01-24T23:23:15.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 1 monclient: found mon.noname-g 2024-01-24T23:23:15.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:15.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:23:15.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 1 -- 172.21.15.132:0/2323732497 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83809df40 con 0x7fd83809ec20 2024-01-24T23:23:15.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 10 monclient: _finish_auth 0 2024-01-24T23:23:15.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 10 monclient: _check_auth_tickets 2024-01-24T23:23:15.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:15.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 1 -- 172.21.15.132:0/2323732497 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd8280017c0 con 0x7fd83809ec20 2024-01-24T23:23:15.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 5 monclient: authenticate success, global_id 4881 2024-01-24T23:23:15.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:15.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:15.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 10 monclient: dump: 2024-01-24T23:23:15.504 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:15.505 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.505 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.505 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.505 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.505 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:15.505 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.505 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.505 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.505 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.505 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.505 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.505 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.505 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.505 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.505 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:15.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:15.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 1 -- 172.21.15.132:0/2323732497 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd828001aa0 con 0x7fd83809ec20 2024-01-24T23:23:15.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:15.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:15.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 4 set_mon_vals no callback set 2024-01-24T23:23:15.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 1 -- 172.21.15.132:0/2323732497 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd828004bf0 con 0x7fd83809ec20 2024-01-24T23:23:15.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:15.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd82ffff640 10 monclient: dump: 2024-01-24T23:23:15.506 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:15.506 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.506 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.506 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.506 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.506 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:15.507 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.507 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.507 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.507 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.507 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.507 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.507 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.507 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.507 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.507 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:15.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 10 monclient: shutdown 2024-01-24T23:23:15.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:15.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 1 -- 172.21.15.132:0/2323732497 >> v1:172.21.15.132:6790/0 conn(0x7fd83809ec20 legacy=0x7fd838138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 1 -- 172.21.15.132:0/2323732497 shutdown_connections 2024-01-24T23:23:15.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 1 -- 172.21.15.132:0/2323732497 >> 172.21.15.132:0/2323732497 conn(0x7fd8380a5a10 msgr2=0x7fd8380a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:15.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 1 -- 172.21.15.132:0/2323732497 shutdown_connections 2024-01-24T23:23:15.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 1 -- 172.21.15.132:0/2323732497 wait complete. 2024-01-24T23:23:15.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 10 monclient: build_initial_monmap 2024-01-24T23:23:15.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 10 monclient: monmap: 2024-01-24T23:23:15.508 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:15.508 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:15.508 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:15.508 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:15.508 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:15.508 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:15.508 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:15.508 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:15.508 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:15.508 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:15.508 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:15.509 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:15.509 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:15.509 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:15.509 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:15.509 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:15.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 1 Processor -- start 2024-01-24T23:23:15.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 1 -- start start 2024-01-24T23:23:15.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.472+0000 7fd83f96b640 10 monclient: init 2024-01-24T23:23:15.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd83f96b640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:15.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd83f96b640 10 monclient: _add_conns ranks=[0,7,3,6,2,8,4,5,1] 2024-01-24T23:23:15.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd83f96b640 10 monclient(hunting): picked mon.noname-a con 0x7fd83809ec20 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:15.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd83f96b640 10 monclient(hunting): picked mon.noname-i con 0x7fd838133b50 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:15.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd83f96b640 10 monclient(hunting): picked mon.noname-g con 0x7fd83813a1d0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:15.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd83f96b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd838133920 con 0x7fd83809ec20 2024-01-24T23:23:15.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd83f96b640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8380a13b0 con 0x7fd83813a1d0 2024-01-24T23:23:15.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd83f96b640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd838070640 con 0x7fd838133b50 2024-01-24T23:23:15.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd83f96b640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:15.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd83f96b640 10 monclient(hunting): authenticate will time out at 1082.800187s 2024-01-24T23:23:15.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd83cf06640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fd838133b50 0x7fd83806d510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:40588/0 (socket says 172.21.15.132:40588) 2024-01-24T23:23:15.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd83cf06640 1 -- 172.21.15.132:0/3820578586 learned_addr learned my addr 172.21.15.132:0/3820578586 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:15.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3592160293 0 0) 0x7fd838070640 con 0x7fd838133b50 2024-01-24T23:23:15.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient(hunting): my global_id is 4975 2024-01-24T23:23:15.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd81c003560 con 0x7fd838133b50 2024-01-24T23:23:15.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203031688 0 0) 0x7fd8380a13b0 con 0x7fd83813a1d0 2024-01-24T23:23:15.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient(hunting): my global_id is 4890 2024-01-24T23:23:15.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd838070640 con 0x7fd83813a1d0 2024-01-24T23:23:15.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1348806297 0 0) 0x7fd838133920 con 0x7fd83809ec20 2024-01-24T23:23:15.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient(hunting): my global_id is 4941 2024-01-24T23:23:15.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8380a13b0 con 0x7fd83809ec20 2024-01-24T23:23:15.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2081454567 0 0) 0x7fd838070640 con 0x7fd83813a1d0 2024-01-24T23:23:15.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd838133920 con 0x7fd83813a1d0 2024-01-24T23:23:15.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd830001c10 con 0x7fd83813a1d0 2024-01-24T23:23:15.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:15.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient(hunting): dump: 2024-01-24T23:23:15.511 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:15.511 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.511 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.511 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.511 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.511 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:15.512 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.512 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.512 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.512 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.512 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.512 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.512 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.512 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.512 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.512 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:15.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:15.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1813163201 0 0) 0x7fd8380a13b0 con 0x7fd83809ec20 2024-01-24T23:23:15.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd838070640 con 0x7fd83809ec20 2024-01-24T23:23:15.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1800229579 0 0) 0x7fd81c003560 con 0x7fd838133b50 2024-01-24T23:23:15.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd8380a13b0 con 0x7fd838133b50 2024-01-24T23:23:15.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd8340019e0 con 0x7fd83809ec20 2024-01-24T23:23:15.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:23:15.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient(hunting): dump: 2024-01-24T23:23:15.513 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:15.513 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.513 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.513 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.513 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.513 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:15.513 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.513 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.513 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.513 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.513 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.513 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.513 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.514 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.514 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.514 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:15.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd828004b30 con 0x7fd838133b50 2024-01-24T23:23:15.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:15.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient(hunting): dump: 2024-01-24T23:23:15.514 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:15.514 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.514 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.514 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.514 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.514 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:15.514 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.514 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.514 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.515 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.515 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.515 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.515 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.515 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.515 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.515 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:15.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 776725050 0 0) 0x7fd838133920 con 0x7fd83813a1d0 2024-01-24T23:23:15.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 >> v1:172.21.15.132:6792/0 conn(0x7fd838133b50 legacy=0x7fd83806d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 >> v1:172.21.15.115:6789/0 conn(0x7fd83809ec20 legacy=0x7fd83806ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient: _finish_hunting 0 2024-01-24T23:23:15.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 1 monclient: found mon.g 2024-01-24T23:23:15.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:15.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:15.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83806dc20 con 0x7fd83813a1d0 2024-01-24T23:23:15.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient: _finish_auth 0 2024-01-24T23:23:15.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient: _check_auth_tickets 2024-01-24T23:23:15.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:15.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd83f96b640 5 monclient: authenticate success, global_id 4890 2024-01-24T23:23:15.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd83f96b640 10 monclient: _renew_subs 2024-01-24T23:23:15.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd83f96b640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:15.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd83f96b640 1 -- 172.21.15.132:0/3820578586 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd838070360 con 0x7fd83813a1d0 2024-01-24T23:23:15.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd830001fe0 con 0x7fd83813a1d0 2024-01-24T23:23:15.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:15.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd830003f70 con 0x7fd83813a1d0 2024-01-24T23:23:15.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:15.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 10 monclient: dump: 2024-01-24T23:23:15.516 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:15.516 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.516 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.517 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.517 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.517 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:15.517 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.517 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.517 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.517 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.517 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.517 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.517 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.517 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.517 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.517 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:15.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82f7fe640 4 set_mon_vals no callback set 2024-01-24T23:23:15.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd83f96b640 10 monclient: _renew_subs 2024-01-24T23:23:15.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd83f96b640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:15.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd83f96b640 1 -- 172.21.15.132:0/3820578586 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd838139580 con 0x7fd83813a1d0 2024-01-24T23:23:15.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fd83000e9f0 con 0x7fd83813a1d0 2024-01-24T23:23:15.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd83f96b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:15.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd83f96b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:15.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd83f96b640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:15.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd83f96b640 1 -- 172.21.15.132:0/3820578586 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd83809f060 con 0x7fd83813a1d0 2024-01-24T23:23:15.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(48..48 src has 1..48) v4 ==== 3994+0+0 (unknown 1916735388 0 0) 0x7fd830042930 con 0x7fd83813a1d0 2024-01-24T23:23:15.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.476+0000 7fd83d707640 10 monclient: get_auth_request con 0x7fd81c037c00 auth_method 0 2024-01-24T23:23:15.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.480+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fd83809f060 con 0x7fd83813a1d0 2024-01-24T23:23:15.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.480+0000 7fd82dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:15.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.480+0000 7fd82dffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:15.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 10 monclient: get_monmap_and_config 2024-01-24T23:23:15.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 10 monclient: build_initial_monmap 2024-01-24T23:23:15.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 10 monclient: monmap: 2024-01-24T23:23:15.634 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:15.635 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.635 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:15.634971+0000 2024-01-24T23:23:15.635 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:15.634971+0000 2024-01-24T23:23:15.635 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:15.635 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.635 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:15.635 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:15.635 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:15.635 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:15.635 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:15.635 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:15.635 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:15.635 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:15.635 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:15.635 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 1 Processor -- start 2024-01-24T23:23:15.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 1 -- start start 2024-01-24T23:23:15.641 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 10 monclient: init 2024-01-24T23:23:15.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:15.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 10 monclient: _add_conns ranks=[5,3,6,1,8,7,4,2,0] 2024-01-24T23:23:15.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 10 monclient(hunting): picked mon.noname-h con 0x7fe0d812ff70 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:15.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 10 monclient(hunting): picked mon.noname-g con 0x7fe0d809f720 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:15.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 10 monclient(hunting): picked mon.noname-d con 0x7fe0d81398e0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:15.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0d804ed70 con 0x7fe0d809f720 2024-01-24T23:23:15.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0d80a1d00 con 0x7fe0d812ff70 2024-01-24T23:23:15.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0d809dee0 con 0x7fe0d81398e0 2024-01-24T23:23:15.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:15.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 10 monclient(hunting): authenticate will time out at 1074.557723s 2024-01-24T23:23:15.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0dd5a5640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fe0d81398e0 0x7fe0d813bcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:34178/0 (socket says 172.21.15.115:34178) 2024-01-24T23:23:15.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0dd5a5640 1 -- 172.21.15.115:0/3976598378 learned_addr learned my addr 172.21.15.115:0/3976598378 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:15.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 1 -- 172.21.15.115:0/3976598378 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423394444 0 0) 0x7fe0d809dee0 con 0x7fe0d81398e0 2024-01-24T23:23:15.642 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 10 monclient(hunting): my global_id is 4983 2024-01-24T23:23:15.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 1 -- 172.21.15.115:0/3976598378 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0c4003530 con 0x7fe0d81398e0 2024-01-24T23:23:15.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 1 -- 172.21.15.115:0/3976598378 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1714907313 0 0) 0x7fe0c4003530 con 0x7fe0d81398e0 2024-01-24T23:23:15.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 1 -- 172.21.15.115:0/3976598378 >> v1:172.21.15.132:6791/0 conn(0x7fe0d812ff70 legacy=0x7fe0d8130350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 1 -- 172.21.15.115:0/3976598378 >> v1:172.21.15.132:6790/0 conn(0x7fe0d809f720 legacy=0x7fe0d81381c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 10 monclient: _finish_hunting 0 2024-01-24T23:23:15.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 1 monclient: found mon.noname-d 2024-01-24T23:23:15.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:15.643 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:23:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 1 -- 172.21.15.115:0/3976598378 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d80a2f50 con 0x7fe0d81398e0 2024-01-24T23:23:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 10 monclient: _finish_auth 0 2024-01-24T23:23:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 10 monclient: _check_auth_tickets 2024-01-24T23:23:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 1 -- 172.21.15.115:0/3976598378 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe0d00017c0 con 0x7fe0d81398e0 2024-01-24T23:23:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:23:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 10 monclient: dump: 2024-01-24T23:23:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.644 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.645 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.645 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.645 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.645 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.645 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.645 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.645 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.645 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.645 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 5 monclient: authenticate success, global_id 4983 2024-01-24T23:23:15.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:15.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 1 -- 172.21.15.115:0/3976598378 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe0d0001ae0 con 0x7fe0d81398e0 2024-01-24T23:23:15.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:15.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:15.645 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 1 -- 172.21.15.115:0/3976598378 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe0d0004bf0 con 0x7fe0d81398e0 2024-01-24T23:23:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 4 set_mon_vals no callback set 2024-01-24T23:23:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0d77fe640 10 monclient: dump: 2024-01-24T23:23:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.646 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.647 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 10 monclient: shutdown 2024-01-24T23:23:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 1 -- 172.21.15.115:0/3976598378 >> v1:172.21.15.115:6792/0 conn(0x7fe0d81398e0 legacy=0x7fe0d813bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 1 -- 172.21.15.115:0/3976598378 shutdown_connections 2024-01-24T23:23:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 1 -- 172.21.15.115:0/3976598378 >> 172.21.15.115:0/3976598378 conn(0x7fe0d80a2110 msgr2=0x7fe0d80a24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 1 -- 172.21.15.115:0/3976598378 shutdown_connections 2024-01-24T23:23:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 1 -- 172.21.15.115:0/3976598378 wait complete. 2024-01-24T23:23:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 10 monclient: build_initial_monmap 2024-01-24T23:23:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.632+0000 7fe0df008640 10 monclient: monmap: 2024-01-24T23:23:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:15.647 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:15.648 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:15.648 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:15.648 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:15.648 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:15.648 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:15.648 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:15.648 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:15.648 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:15.648 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:15.648 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:15.648 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:15.648 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:15.648 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:15.648 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0df008640 1 Processor -- start 2024-01-24T23:23:15.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0df008640 1 -- start start 2024-01-24T23:23:15.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0df008640 10 monclient: init 2024-01-24T23:23:15.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0df008640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:15.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0df008640 10 monclient: _add_conns ranks=[3,1,2,6,0,4,8,7,5] 2024-01-24T23:23:15.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0df008640 10 monclient(hunting): picked mon.noname-g con 0x7fe0d809f720 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:15.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0df008640 10 monclient(hunting): picked mon.noname-f con 0x7fe0d812ff70 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:15.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0df008640 10 monclient(hunting): picked mon.noname-b con 0x7fe0d81398e0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:15.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0df008640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0d8132510 con 0x7fe0d812ff70 2024-01-24T23:23:15.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0df008640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0d8132230 con 0x7fe0d81398e0 2024-01-24T23:23:15.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0df008640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0d81327f0 con 0x7fe0d809f720 2024-01-24T23:23:15.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0df008640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:15.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0df008640 10 monclient(hunting): authenticate will time out at 1074.560709s 2024-01-24T23:23:15.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0dd5a5640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fe0d81398e0 0x7fe0d81d3bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:47594/0 (socket says 172.21.15.115:47594) 2024-01-24T23:23:15.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0dd5a5640 1 -- 172.21.15.115:0/676149619 learned_addr learned my addr 172.21.15.115:0/676149619 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:15.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051007173 0 0) 0x7fe0d8132230 con 0x7fe0d81398e0 2024-01-24T23:23:15.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient(hunting): my global_id is 84161 2024-01-24T23:23:15.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0b8003530 con 0x7fe0d81398e0 2024-01-24T23:23:15.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 331708244 0 0) 0x7fe0b8003530 con 0x7fe0d81398e0 2024-01-24T23:23:15.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe0d8132230 con 0x7fe0d81398e0 2024-01-24T23:23:15.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe0d0001780 con 0x7fe0d81398e0 2024-01-24T23:23:15.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:23:15.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient(hunting): dump: 2024-01-24T23:23:15.650 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.650 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.650 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.651 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.651 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.651 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.651 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.651 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.651 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.651 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.651 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.651 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.651 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.651 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.651 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.651 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:15.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3925655692 0 0) 0x7fe0d8132510 con 0x7fe0d812ff70 2024-01-24T23:23:15.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient(hunting): my global_id is 4987 2024-01-24T23:23:15.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0b8003530 con 0x7fe0d812ff70 2024-01-24T23:23:15.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964437096 0 0) 0x7fe0d81327f0 con 0x7fe0d809f720 2024-01-24T23:23:15.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient(hunting): my global_id is 4908 2024-01-24T23:23:15.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0d8132510 con 0x7fe0d809f720 2024-01-24T23:23:15.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 216959997 0 0) 0x7fe0d8132230 con 0x7fe0d81398e0 2024-01-24T23:23:15.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 >> v1:172.21.15.132:6790/0 conn(0x7fe0d809f720 legacy=0x7fe0d8138270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 >> v1:172.21.15.132:6789/0 conn(0x7fe0d812ff70 legacy=0x7fe0d81d0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient: _finish_hunting 0 2024-01-24T23:23:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 monclient: found mon.b 2024-01-24T23:23:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d8132ad0 con 0x7fe0d81398e0 2024-01-24T23:23:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient: _finish_auth 0 2024-01-24T23:23:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient: _check_auth_tickets 2024-01-24T23:23:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0df008640 5 monclient: authenticate success, global_id 84161 2024-01-24T23:23:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0df008640 10 monclient: _renew_subs 2024-01-24T23:23:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0df008640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0df008640 1 -- 172.21.15.115:0/676149619 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0d8138980 con 0x7fe0d81398e0 2024-01-24T23:23:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0df008640 10 monclient: _renew_subs 2024-01-24T23:23:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0df008640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:15.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0df008640 1 -- 172.21.15.115:0/676149619 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d81314a0 con 0x7fe0d81398e0 2024-01-24T23:23:15.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe0d0001c50 con 0x7fe0d81398e0 2024-01-24T23:23:15.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:15.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe0d0004c70 con 0x7fe0d81398e0 2024-01-24T23:23:15.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d6ffd640 4 set_mon_vals no callback set 2024-01-24T23:23:15.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:15.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient: dump: 2024-01-24T23:23:15.654 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.654 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.654 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.654 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.654 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.654 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.654 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.654 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.654 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.655 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.655 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.655 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.655 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.655 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.655 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.655 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fe0d000fdd0 con 0x7fe0d81398e0 2024-01-24T23:23:15.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0df008640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:15.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0df008640 1 -- 172.21.15.115:0/676149619 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe0bc0065d0 con 0x7fe0bc002870 2024-01-24T23:23:15.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(48..48 src has 1..48) v4 ==== 3994+0+0 (unknown 1916735388 0 0) 0x7fe0d0043ad0 con 0x7fe0d81398e0 2024-01-24T23:23:15.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d7fff640 10 monclient: get_auth_request con 0x7fe0b8037180 auth_method 0 2024-01-24T23:23:15.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2379693307 0 0) 0x7fe0bc0065d0 con 0x7fe0bc002870 2024-01-24T23:23:15.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient(hunting): my global_id is 4996 2024-01-24T23:23:15.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0d8132230 con 0x7fe0bc002870 2024-01-24T23:23:15.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2695581529 0 0) 0x7fe0d8132230 con 0x7fe0bc002870 2024-01-24T23:23:15.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe0bc0065d0 con 0x7fe0bc002870 2024-01-24T23:23:15.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe0c0001980 con 0x7fe0bc002870 2024-01-24T23:23:15.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:23:15.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 10 monclient: dump: 2024-01-24T23:23:15.656 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.656 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.656 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.656 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.656 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.656 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.657 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.657 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.657 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.657 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.657 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.657 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.657 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.657 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.657 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.657 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3279304524 0 0) 0x7fe0bc0065d0 con 0x7fe0bc002870 2024-01-24T23:23:15.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.636+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 --> v1:172.21.15.132:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe0bc00a120 con 0x7fe0bc002870 2024-01-24T23:23:15.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.636+0000 7fd83f96b640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-01-24T23:23:15.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.636+0000 7fd83f96b640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-01-24T23:23:15.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.636+0000 7fd83f96b640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:15.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.636+0000 7fd83f96b640 1 -- 172.21.15.132:0/3820578586 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "hdd"} v 0) v1 -- 0x7fd83809f060 con 0x7fd83813a1d0 2024-01-24T23:23:15.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.640+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "hdd"}]=0 v48) v1 ==== 90+0+8 (unknown 251225796 0 1286160932) 0x7fd83809f060 con 0x7fd83813a1d0 2024-01-24T23:23:15.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.640+0000 7fd82dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-01-24T23:23:15.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.640+0000 7fd82dffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:15.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.640+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 <== mon.1 v1:172.21.15.132:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fe0c0003600 con 0x7fe0bc002870 2024-01-24T23:23:15.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.640+0000 7fe0d57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:15.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.640+0000 7fe0d57fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:15.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.640+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 >> v1:172.21.15.132:6789/0 conn(0x7fe0bc002870 legacy=0x7fe0bc004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.640+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 >> v1:172.21.15.132:6789/0 conn(0x7fe0bc002870 legacy=0x7fe0bc004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:15.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.644+0000 7fe0df008640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:23:15.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.644+0000 7fe0df008640 1 -- 172.21.15.115:0/676149619 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe0bc009df0 con 0x7fe0bc0090b0 2024-01-24T23:23:15.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.644+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3607856516 0 0) 0x7fe0bc009df0 con 0x7fe0bc0090b0 2024-01-24T23:23:15.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.644+0000 7fe0d57fa640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.644+0000 7fe0d57fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.644+0000 7fe0d57fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.644+0000 7fe0d57fa640 10 monclient(hunting): my global_id is 5005 2024-01-24T23:23:15.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.644+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0bc0065d0 con 0x7fe0bc0090b0 2024-01-24T23:23:15.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.644+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1560567279 0 0) 0x7fe0bc0065d0 con 0x7fe0bc0090b0 2024-01-24T23:23:15.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.644+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe0bc009df0 con 0x7fe0bc0090b0 2024-01-24T23:23:15.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.644+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe0c0003d60 con 0x7fe0bc0090b0 2024-01-24T23:23:15.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.644+0000 7fe0d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.644+0000 7fe0d57fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:23:15.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.644+0000 7fe0d57fa640 10 monclient: dump: 2024-01-24T23:23:15.661 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.661 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.661 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.661 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.661 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.661 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.661 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.661 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.661 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.661 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.661 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.661 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.662 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.662 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.662 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.662 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.644+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3694389728 0 0) 0x7fe0bc009df0 con 0x7fe0bc0090b0 2024-01-24T23:23:15.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.644+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 --> v1:172.21.15.132:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe0bc00c310 con 0x7fe0bc0090b0 2024-01-24T23:23:15.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.644+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 <== mon.1 v1:172.21.15.132:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 1850279647) 0x7fe0c00060d0 con 0x7fe0bc0090b0 2024-01-24T23:23:15.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.644+0000 7fe0d57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:23:15.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.644+0000 7fe0d57fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:15.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.644+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 >> v1:172.21.15.132:6789/0 conn(0x7fe0bc0090b0 legacy=0x7fe0bc009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.644+0000 7fe0d57fa640 1 -- 172.21.15.115:0/676149619 >> v1:172.21.15.132:6789/0 conn(0x7fe0bc0090b0 legacy=0x7fe0bc009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:15.662 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:23:15.663 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:15.663 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:15.663 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:23:15.663 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 16, 2024-01-24T23:23:15.663 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:23:15.663 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:23:15.663 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:23:15.663 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:23:15.663 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:23:15.663 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:23:15.663 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:23:15.663 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:23:15.663 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:23:15.663 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:23:15.664 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:15.664 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 1, 2024-01-24T23:23:15.664 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:15.664 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:23:15.664 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:23:15.664 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:15.664 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:15.664 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:15.664 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:15.664 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:15.664 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:15.664 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:15.664 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:15.664 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:15.665 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:15.665 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:15.665 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:15.665 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:23:15.665 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:23:15.665 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:15.665 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:15.665 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:15.665 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:15.665 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:15.665 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:15.665 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:15.665 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:15.665 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:15.665 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:15.666 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:15.666 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.666 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:23:15.666 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:23:15.666 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:23:15.666 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:23:15.666 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:23:15.666 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:23:15.666 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:15.666 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:15.666 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:23:15.666 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:23:15.666 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:23:15.666 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:23:15.667 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:23:15.667 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:23:15.667 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:23:15.667 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:15.667 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:23:15.667 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:15.667 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:15.667 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:15.667 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:15.667 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:15.667 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:15.667 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:15.667 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:15.667 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:15.667 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:15.668 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:15.668 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:15.668 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:23:15.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.668 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:23:15.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.668 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:15.668 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:15.668 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.668 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.668 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.668 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:23:15.668 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.668 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.669 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.669 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:23:15.669 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:23:15.669 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.669 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.669 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.669 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:15.669 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:15.669 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.669 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.669 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.670 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:23:15.670 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.670 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.670 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:23:15.670 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:23:15.670 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.670 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.670 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.670 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:15.670 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:15.670 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.671 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.671 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.671 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:23:15.671 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.671 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.671 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.671 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:23:15.671 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:23:15.671 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.671 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.671 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.672 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:15.672 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:15.672 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.672 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.672 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.672 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:23:15.672 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.672 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.672 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.672 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:23:15.672 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:23:15.672 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.673 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.673 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.673 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:15.673 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:15.673 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.673 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.673 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.673 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:23:15.673 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.673 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.673 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.673 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:23:15.674 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:23:15.674 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.674 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.674 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.674 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:15.674 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:15.674 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.674 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.674 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.674 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:23:15.674 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.674 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.675 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.675 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:23:15.675 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:23:15.675 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.675 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.675 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.675 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:15.675 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:15.675 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.675 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.675 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.676 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:23:15.676 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.676 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.676 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.676 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:23:15.676 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:23:15.676 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.676 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.676 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.676 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:15.676 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:15.677 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.677 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.677 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.677 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:23:15.677 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.677 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.677 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.677 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:23:15.677 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:23:15.677 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.677 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.677 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.678 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:15.678 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:15.678 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.678 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.678 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.678 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:23:15.678 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.678 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.678 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.678 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:23:15.678 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:23:15.678 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.679 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.679 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.679 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.679 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.679 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:23:15.679 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:23:15.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.679 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:15.679 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:15.679 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:15.679 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.679 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:15.679 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:23:15.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.680 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:15.680 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:15.680 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:23:15.680 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.680 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.680 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:23:15.680 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:23:15.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.648+0000 7fe0df008640 1 -- 172.21.15.115:0/676149619 >> v1:172.21.15.132:6812/1835401922 conn(0x7fe0b8037180 legacy=0x7fe0b8039640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.648+0000 7fe0df008640 10 monclient: shutdown 2024-01-24T23:23:15.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.648+0000 7fe0df008640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:15.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.648+0000 7fe0df008640 1 -- 172.21.15.115:0/676149619 >> v1:172.21.15.115:6790/0 conn(0x7fe0d81398e0 legacy=0x7fe0d81d3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.648+0000 7fe0dd5a5640 1 -- 172.21.15.115:0/676149619 reap_dead start 2024-01-24T23:23:15.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.648+0000 7fe0dd5a5640 1 -- 172.21.15.115:0/676149619 reap_dead start 2024-01-24T23:23:15.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.648+0000 7fe0df008640 1 -- 172.21.15.115:0/676149619 shutdown_connections 2024-01-24T23:23:15.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.648+0000 7fe0df008640 1 -- 172.21.15.115:0/676149619 >> 172.21.15.115:0/676149619 conn(0x7fe0d80a2110 msgr2=0x7fe0d81be9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:15.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.648+0000 7fe0df008640 1 -- 172.21.15.115:0/676149619 shutdown_connections 2024-01-24T23:23:15.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.648+0000 7fe0df008640 1 -- 172.21.15.115:0/676149619 wait complete. 2024-01-24T23:23:15.682 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:23:15.682 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-01-24T23:23:15.683 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-01-24T23:23:15.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.728+0000 7fd83f96b640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-01-24T23:23:15.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.728+0000 7fd83f96b640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-01-24T23:23:15.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.728+0000 7fd83f96b640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:15.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.728+0000 7fd83f96b640 1 -- 172.21.15.132:0/3820578586 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "hdd"} v 0) v1 -- 0x7fd83809f060 con 0x7fd83813a1d0 2024-01-24T23:23:15.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.732+0000 7fd82dffb640 1 -- 172.21.15.132:0/3820578586 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "hdd"}]=0 v48) v1 ==== 90+0+8 (unknown 251225796 0 1286160932) 0x7fd83809f060 con 0x7fd83813a1d0 2024-01-24T23:23:15.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.732+0000 7fd82dffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-01-24T23:23:15.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.732+0000 7fd82dffb640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:23:15.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.732+0000 7fd83f96b640 1 -- 172.21.15.132:0/3820578586 >> v1:172.21.15.132:6812/1835401922 conn(0x7fd81c037c00 legacy=0x7fd81c03a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.732+0000 7fd83f96b640 10 monclient: shutdown 2024-01-24T23:23:15.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.732+0000 7fd83f96b640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:15.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.732+0000 7fd83f96b640 1 -- 172.21.15.132:0/3820578586 >> v1:172.21.15.132:6790/0 conn(0x7fd83813a1d0 legacy=0x7fd8381d35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.732+0000 7fd83f96b640 1 -- 172.21.15.132:0/3820578586 shutdown_connections 2024-01-24T23:23:15.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.732+0000 7fd83f96b640 1 -- 172.21.15.132:0/3820578586 >> 172.21.15.132:0/3820578586 conn(0x7fd8380a5a10 msgr2=0x7fd838066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:15.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.732+0000 7fd83f96b640 1 -- 172.21.15.132:0/3820578586 shutdown_connections 2024-01-24T23:23:15.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.732+0000 7fd83f96b640 1 -- 172.21.15.132:0/3820578586 wait complete. 2024-01-24T23:23:15.747 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:23:15.748 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd erasure-code-profile set ec-foo-ssd crush-device-class=ssd m=2 k=2 2024-01-24T23:23:15.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.792+0000 7f336e116640 10 monclient: get_monmap_and_config 2024-01-24T23:23:15.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.792+0000 7f336e116640 10 monclient: build_initial_monmap 2024-01-24T23:23:15.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.792+0000 7f336e116640 10 monclient: monmap: 2024-01-24T23:23:15.797 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:15.797 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.797 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:15.797511+0000 2024-01-24T23:23:15.797 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:15.797511+0000 2024-01-24T23:23:15.797 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:15.797 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.797 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:15.797 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:15.797 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:15.797 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:15.798 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:15.798 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:15.798 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:15.798 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:15.798 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:15.798 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.792+0000 7f336e116640 1 Processor -- start 2024-01-24T23:23:15.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.792+0000 7f336e116640 1 -- start start 2024-01-24T23:23:15.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.792+0000 7f336e116640 10 monclient: init 2024-01-24T23:23:15.805 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.792+0000 7f336e116640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:15.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.792+0000 7f336e116640 10 monclient: _add_conns ranks=[1,4,2,5,8,7,6,3,0] 2024-01-24T23:23:15.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.792+0000 7f336e116640 10 monclient(hunting): picked mon.noname-f con 0x7f336809b9f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:15.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.792+0000 7f336e116640 10 monclient(hunting): picked mon.noname-c con 0x7f33680720b0 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:15.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 10 monclient(hunting): picked mon.noname-b con 0x7f3368139700 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:15.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f336804ed70 con 0x7f336809b9f0 2024-01-24T23:23:15.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33680a0eb0 con 0x7f3368139700 2024-01-24T23:23:15.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f336809bdd0 con 0x7f33680720b0 2024-01-24T23:23:15.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:15.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 10 monclient(hunting): authenticate will time out at 1074.720043s 2024-01-24T23:23:15.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f3366ffd640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f33680720b0 0x7f33680a0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:48596/0 (socket says 172.21.15.115:48596) 2024-01-24T23:23:15.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f3366ffd640 1 -- 172.21.15.115:0/3676560823 learned_addr learned my addr 172.21.15.115:0/3676560823 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:15.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 1 -- 172.21.15.115:0/3676560823 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1349356841 0 0) 0x7f336809bdd0 con 0x7f33680720b0 2024-01-24T23:23:15.806 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 10 monclient(hunting): my global_id is 74947 2024-01-24T23:23:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 1 -- 172.21.15.115:0/3676560823 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3354003530 con 0x7f33680720b0 2024-01-24T23:23:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 1 -- 172.21.15.115:0/3676560823 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3785001884 0 0) 0x7f33680a0eb0 con 0x7f3368139700 2024-01-24T23:23:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 10 monclient(hunting): my global_id is 84170 2024-01-24T23:23:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 1 -- 172.21.15.115:0/3676560823 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f336809bdd0 con 0x7f3368139700 2024-01-24T23:23:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 1 -- 172.21.15.115:0/3676560823 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3702325910 0 0) 0x7f336804ed70 con 0x7f336809b9f0 2024-01-24T23:23:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 10 monclient(hunting): my global_id is 5014 2024-01-24T23:23:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 1 -- 172.21.15.115:0/3676560823 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33680a0eb0 con 0x7f336809b9f0 2024-01-24T23:23:15.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 1 -- 172.21.15.115:0/3676560823 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3255187057 0 0) 0x7f3354003530 con 0x7f33680720b0 2024-01-24T23:23:15.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 1 -- 172.21.15.115:0/3676560823 >> v1:172.21.15.115:6790/0 conn(0x7f3368139700 legacy=0x7f336813baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 1 -- 172.21.15.115:0/3676560823 >> v1:172.21.15.132:6789/0 conn(0x7f336809b9f0 legacy=0x7f336809f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 10 monclient: _finish_hunting 0 2024-01-24T23:23:15.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 1 monclient: found mon.noname-c 2024-01-24T23:23:15.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:15.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:23:15.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 1 -- 172.21.15.115:0/3676560823 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33680a0710 con 0x7f33680720b0 2024-01-24T23:23:15.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 10 monclient: _finish_auth 0 2024-01-24T23:23:15.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 10 monclient: _check_auth_tickets 2024-01-24T23:23:15.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:15.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 1 -- 172.21.15.115:0/3676560823 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3360001b40 con 0x7f33680720b0 2024-01-24T23:23:15.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 5 monclient: authenticate success, global_id 74947 2024-01-24T23:23:15.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:15.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:23:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 10 monclient: dump: 2024-01-24T23:23:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.810 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 1 -- 172.21.15.115:0/3676560823 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3360001e20 con 0x7f33680720b0 2024-01-24T23:23:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 1 -- 172.21.15.115:0/3676560823 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3360004f70 con 0x7f33680720b0 2024-01-24T23:23:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 4 set_mon_vals no callback set 2024-01-24T23:23:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:23:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f33667fc640 10 monclient: dump: 2024-01-24T23:23:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.811 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 10 monclient: shutdown 2024-01-24T23:23:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 1 -- 172.21.15.115:0/3676560823 >> v1:172.21.15.115:6791/0 conn(0x7f33680720b0 legacy=0x7f33680a0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 1 -- 172.21.15.115:0/3676560823 shutdown_connections 2024-01-24T23:23:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 1 -- 172.21.15.115:0/3676560823 >> 172.21.15.115:0/3676560823 conn(0x7f33680a5730 msgr2=0x7f33680a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 1 -- 172.21.15.115:0/3676560823 shutdown_connections 2024-01-24T23:23:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 1 -- 172.21.15.115:0/3676560823 wait complete. 2024-01-24T23:23:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 10 monclient: build_initial_monmap 2024-01-24T23:23:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 10 monclient: monmap: 2024-01-24T23:23:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:15.816 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 1 Processor -- start 2024-01-24T23:23:15.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 1 -- start start 2024-01-24T23:23:15.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 10 monclient: init 2024-01-24T23:23:15.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:15.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 10 monclient: _add_conns ranks=[1,7,0,4,5,6,2,8,3] 2024-01-24T23:23:15.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 10 monclient(hunting): picked mon.noname-f con 0x7f33680720b0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:15.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 10 monclient(hunting): picked mon.noname-i con 0x7f336809b9f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:15.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 10 monclient(hunting): picked mon.noname-a con 0x7f3368139700 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:15.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3368070520 con 0x7f3368139700 2024-01-24T23:23:15.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f336806da00 con 0x7f33680720b0 2024-01-24T23:23:15.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f336806e2f0 con 0x7f336809b9f0 2024-01-24T23:23:15.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:15.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 10 monclient(hunting): authenticate will time out at 1074.723093s 2024-01-24T23:23:15.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f3367fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f3368139700 0x7f33681d37f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:60936/0 (socket says 172.21.15.115:60936) 2024-01-24T23:23:15.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f3367fff640 1 -- 172.21.15.115:0/1549067766 learned_addr learned my addr 172.21.15.115:0/1549067766 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:15.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 511715485 0 0) 0x7f3368070520 con 0x7f3368139700 2024-01-24T23:23:15.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 10 monclient(hunting): my global_id is 4950 2024-01-24T23:23:15.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3348003530 con 0x7f3368139700 2024-01-24T23:23:15.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 733588755 0 0) 0x7f3348003530 con 0x7f3368139700 2024-01-24T23:23:15.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3368070520 con 0x7f3368139700 2024-01-24T23:23:15.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f33580019f0 con 0x7f3368139700 2024-01-24T23:23:15.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:23:15.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 10 monclient(hunting): dump: 2024-01-24T23:23:15.817 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.818 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.818 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.818 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.818 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.818 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.818 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.818 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.818 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.818 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.818 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.818 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.818 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.818 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.818 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.818 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:15.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2786709239 0 0) 0x7f336806da00 con 0x7f33680720b0 2024-01-24T23:23:15.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 10 monclient(hunting): my global_id is 5023 2024-01-24T23:23:15.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3348003530 con 0x7f33680720b0 2024-01-24T23:23:15.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1666521309 0 0) 0x7f336806e2f0 con 0x7f336809b9f0 2024-01-24T23:23:15.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 10 monclient(hunting): my global_id is 4984 2024-01-24T23:23:15.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f336806da00 con 0x7f336809b9f0 2024-01-24T23:23:15.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 104876764 0 0) 0x7f3368070520 con 0x7f3368139700 2024-01-24T23:23:15.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 >> v1:172.21.15.132:6792/0 conn(0x7f336809b9f0 legacy=0x7f336806d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 >> v1:172.21.15.132:6789/0 conn(0x7f33680720b0 legacy=0x7f336806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 10 monclient: _finish_hunting 0 2024-01-24T23:23:15.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 1 monclient: found mon.a 2024-01-24T23:23:15.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:15.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:23:15.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f336806faf0 con 0x7f3368139700 2024-01-24T23:23:15.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 10 monclient: _finish_auth 0 2024-01-24T23:23:15.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 10 monclient: _check_auth_tickets 2024-01-24T23:23:15.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f335ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:15.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 5 monclient: authenticate success, global_id 4950 2024-01-24T23:23:15.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 10 monclient: _renew_subs 2024-01-24T23:23:15.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:23:15.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.796+0000 7f336e116640 1 -- 172.21.15.115:0/1549067766 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f336809c9f0 con 0x7f3368139700 2024-01-24T23:23:15.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f336e116640 10 monclient: _renew_subs 2024-01-24T23:23:15.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f336e116640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:23:15.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f336e116640 1 -- 172.21.15.115:0/1549067766 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f336806f810 con 0x7f3368139700 2024-01-24T23:23:15.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3358003930 con 0x7f3368139700 2024-01-24T23:23:15.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:15.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3358003e60 con 0x7f3368139700 2024-01-24T23:23:15.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:23:15.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f3365ffb640 4 set_mon_vals no callback set 2024-01-24T23:23:15.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 10 monclient: dump: 2024-01-24T23:23:15.821 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.821 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.821 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.821 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.821 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.821 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.821 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.822 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.822 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.822 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.822 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.822 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.822 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.822 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.822 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.822 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f336e116640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:15.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f3358003fc0 con 0x7f3368139700 2024-01-24T23:23:15.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f336e116640 1 -- 172.21.15.115:0/1549067766 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f334c0065d0 con 0x7f334c002870 2024-01-24T23:23:15.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(48..48 src has 1..48) v4 ==== 3994+0+0 (unknown 1916735388 0 0) 0x7f3358043790 con 0x7f3368139700 2024-01-24T23:23:15.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f3366ffd640 10 monclient: get_auth_request con 0x7f33480371e0 auth_method 0 2024-01-24T23:23:15.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3476227994 0 0) 0x7f334c0065d0 con 0x7f334c002870 2024-01-24T23:23:15.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 10 monclient(hunting): my global_id is 4917 2024-01-24T23:23:15.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3368070520 con 0x7f334c002870 2024-01-24T23:23:15.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2464743448 0 0) 0x7f3368070520 con 0x7f334c002870 2024-01-24T23:23:15.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f334c0065d0 con 0x7f334c002870 2024-01-24T23:23:15.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f33500019b0 con 0x7f334c002870 2024-01-24T23:23:15.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:15.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 10 monclient: dump: 2024-01-24T23:23:15.823 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.823 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.823 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.823 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.823 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.824 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.824 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.824 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.824 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.824 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.824 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.824 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.824 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.824 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.824 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.824 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3250411550 0 0) 0x7f334c0065d0 con 0x7f334c002870 2024-01-24T23:23:15.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 --> v1:172.21.15.132:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f334c00a120 con 0x7f334c002870 2024-01-24T23:23:15.824 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f3350003600 con 0x7f334c002870 2024-01-24T23:23:15.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:15.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:15.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 >> v1:172.21.15.132:6790/0 conn(0x7f334c002870 legacy=0x7f334c004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.825 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.800+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 >> v1:172.21.15.132:6790/0 conn(0x7f334c002870 legacy=0x7f334c004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:15.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f336e116640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:23:15.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f336e116640 1 -- 172.21.15.115:0/1549067766 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f334c009df0 con 0x7f334c0090b0 2024-01-24T23:23:15.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804074074 0 0) 0x7f334c009df0 con 0x7f334c0090b0 2024-01-24T23:23:15.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f335ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f335ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f335ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f335ffff640 10 monclient(hunting): my global_id is 4926 2024-01-24T23:23:15.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f334c0065d0 con 0x7f334c0090b0 2024-01-24T23:23:15.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 406835780 0 0) 0x7f334c0065d0 con 0x7f334c0090b0 2024-01-24T23:23:15.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f334c009df0 con 0x7f334c0090b0 2024-01-24T23:23:15.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3350003d20 con 0x7f334c0090b0 2024-01-24T23:23:15.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f335ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f335ffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:15.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f335ffff640 10 monclient: dump: 2024-01-24T23:23:15.827 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.827 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.827 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.827 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.827 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.827 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.827 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.827 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.827 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.827 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.827 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.828 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.828 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.828 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.828 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.828 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2434293714 0 0) 0x7f334c009df0 con 0x7f334c0090b0 2024-01-24T23:23:15.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 --> v1:172.21.15.132:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f334c00c310 con 0x7f334c0090b0 2024-01-24T23:23:15.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 2573422892) 0x7f3350006070 con 0x7f334c0090b0 2024-01-24T23:23:15.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f335ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:23:15.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f335ffff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:15.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 >> v1:172.21.15.132:6790/0 conn(0x7f334c0090b0 legacy=0x7f334c009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f335ffff640 1 -- 172.21.15.115:0/1549067766 >> v1:172.21.15.132:6790/0 conn(0x7f334c0090b0 legacy=0x7f334c009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:15.829 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:23:15.829 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:15.829 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:15.829 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:23:15.829 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 16, 2024-01-24T23:23:15.829 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:23:15.829 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:23:15.829 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:23:15.829 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:23:15.829 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:23:15.829 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:23:15.829 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:23:15.829 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:23:15.829 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:23:15.830 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:23:15.830 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:15.830 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 1, 2024-01-24T23:23:15.830 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:15.830 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:23:15.830 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:23:15.830 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:15.830 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:15.830 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:15.830 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:15.830 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:15.830 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:15.830 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:15.830 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:15.831 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:15.831 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:15.831 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:15.831 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:15.831 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:23:15.831 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:23:15.831 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:15.831 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:15.831 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:15.831 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:15.831 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:15.831 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:15.831 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:15.831 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:15.832 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:15.832 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:15.832 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:15.832 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.832 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:23:15.832 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:23:15.832 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:23:15.832 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:23:15.832 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:23:15.832 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:23:15.832 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:15.832 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:15.832 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:23:15.833 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:23:15.833 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:23:15.833 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:23:15.833 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:23:15.833 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:23:15.833 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:23:15.833 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:15.833 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:23:15.833 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:15.833 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:15.833 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:15.833 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:15.833 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:15.833 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:15.834 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:15.834 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:15.834 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:15.834 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:15.834 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:15.834 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:15.834 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:23:15.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.834 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:23:15.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.834 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:15.834 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:15.834 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.834 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.835 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.835 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:23:15.835 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.835 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.835 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.835 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:23:15.835 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:23:15.835 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.835 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.835 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.835 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:15.836 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:15.836 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.836 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.836 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.836 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:23:15.836 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.836 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.836 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.836 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:23:15.836 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:23:15.836 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.836 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.836 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.837 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:15.837 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:15.837 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.837 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.837 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.837 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:23:15.837 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.837 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.837 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.837 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:23:15.838 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:23:15.838 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.838 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.838 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.838 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:15.838 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:15.838 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.838 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.838 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.838 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:23:15.838 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.838 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.839 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.839 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:23:15.839 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:23:15.839 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.839 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.839 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.839 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:15.839 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:15.839 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.839 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.840 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.840 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:23:15.840 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.840 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.840 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.840 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:23:15.840 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:23:15.840 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.840 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.840 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.840 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:15.841 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:15.841 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.841 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.841 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.841 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:23:15.841 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.841 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.841 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.841 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:23:15.841 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:23:15.841 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.841 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.842 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.842 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:15.842 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:15.842 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.842 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.842 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.842 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:23:15.842 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.842 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.842 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.842 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:23:15.843 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:23:15.843 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.843 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.843 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.843 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:15.843 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:15.843 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.843 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.843 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.843 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:23:15.843 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.844 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.844 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.844 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:23:15.844 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:23:15.844 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.844 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.844 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.844 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:15.844 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:15.844 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:15.844 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:15.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.845 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:15.845 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:23:15.845 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:15.845 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.845 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.845 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:23:15.845 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:23:15.845 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:15.845 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:15.845 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:15.845 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.845 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.845 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:23:15.846 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:23:15.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.846 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:15.846 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:15.846 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:15.846 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.846 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:15.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:23:15.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.846 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:15.846 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:15.846 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:15.846 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.846 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:15.847 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": [ 2024-01-24T23:23:15.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:15.847 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:15.847 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:15.847 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:15.847 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:15.847 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:15.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:15.847 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:23:15.847 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:23:15.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f336e116640 1 -- 172.21.15.115:0/1549067766 >> v1:172.21.15.132:6812/1835401922 conn(0x7f33480371e0 legacy=0x7f33480396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f336e116640 10 monclient: shutdown 2024-01-24T23:23:15.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f336e116640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:15.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f336e116640 1 -- 172.21.15.115:0/1549067766 >> v1:172.21.15.115:6789/0 conn(0x7f3368139700 legacy=0x7f33681d37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f3367fff640 1 -- 172.21.15.115:0/1549067766 reap_dead start 2024-01-24T23:23:15.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f3367fff640 1 -- 172.21.15.115:0/1549067766 reap_dead start 2024-01-24T23:23:15.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f336e116640 1 -- 172.21.15.115:0/1549067766 shutdown_connections 2024-01-24T23:23:15.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f336e116640 1 -- 172.21.15.115:0/1549067766 >> 172.21.15.115:0/1549067766 conn(0x7f33680a5730 msgr2=0x7f336809a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:15.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f336e116640 1 -- 172.21.15.115:0/1549067766 shutdown_connections 2024-01-24T23:23:15.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.808+0000 7f336e116640 1 -- 172.21.15.115:0/1549067766 wait complete. 2024-01-24T23:23:15.849 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:23:15.849 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-01-24T23:23:15.850 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-01-24T23:23:15.862 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.860+0000 7fc270219640 10 monclient: get_monmap_and_config 2024-01-24T23:23:15.862 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.860+0000 7fc270219640 10 monclient: build_initial_monmap 2024-01-24T23:23:15.862 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.860+0000 7fc270219640 10 monclient: monmap: 2024-01-24T23:23:15.863 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:15.863 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.863 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:15.863991+0000 2024-01-24T23:23:15.863 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:15.863991+0000 2024-01-24T23:23:15.863 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:15.863 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:15.863 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:15.863 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:15.863 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:15.863 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:15.863 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:15.863 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:15.863 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:15.863 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:15.863 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:15.864 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:15.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.860+0000 7fc270219640 1 Processor -- start 2024-01-24T23:23:15.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.860+0000 7fc270219640 1 -- start start 2024-01-24T23:23:15.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.860+0000 7fc270219640 10 monclient: init 2024-01-24T23:23:15.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:15.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 10 monclient: _add_conns ranks=[5,6,2,0,3,4,7,8,1] 2024-01-24T23:23:15.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 10 monclient(hunting): picked mon.noname-h con 0x7fc26809b810 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:15.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 10 monclient(hunting): picked mon.noname-d con 0x7fc26809bbf0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:15.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 10 monclient(hunting): picked mon.noname-b con 0x7fc2681370a0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:15.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc268064700 con 0x7fc2681370a0 2024-01-24T23:23:15.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc268099af0 con 0x7fc26809b810 2024-01-24T23:23:15.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc268072110 con 0x7fc26809bbf0 2024-01-24T23:23:15.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:15.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 10 monclient(hunting): authenticate will time out at 1083.188502s 2024-01-24T23:23:15.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26dfb5640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fc26809b810 0x7fc2680712f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:45568/0 (socket says 172.21.15.132:45568) 2024-01-24T23:23:15.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26dfb5640 1 -- 172.21.15.132:0/3374693164 learned_addr learned my addr 172.21.15.132:0/3374693164 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:15.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 1 -- 172.21.15.132:0/3374693164 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1525243279 0 0) 0x7fc268099af0 con 0x7fc26809b810 2024-01-24T23:23:15.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 10 monclient(hunting): my global_id is 4955 2024-01-24T23:23:15.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 1 -- 172.21.15.132:0/3374693164 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc254003530 con 0x7fc26809b810 2024-01-24T23:23:15.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 1 -- 172.21.15.132:0/3374693164 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1122496949 0 0) 0x7fc268064700 con 0x7fc2681370a0 2024-01-24T23:23:15.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 10 monclient(hunting): my global_id is 84179 2024-01-24T23:23:15.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 1 -- 172.21.15.132:0/3374693164 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc268099af0 con 0x7fc2681370a0 2024-01-24T23:23:15.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 1 -- 172.21.15.132:0/3374693164 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3145272515 0 0) 0x7fc268072110 con 0x7fc26809bbf0 2024-01-24T23:23:15.872 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.872 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.872 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.872 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 10 monclient(hunting): my global_id is 4992 2024-01-24T23:23:15.872 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 1 -- 172.21.15.132:0/3374693164 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc268064700 con 0x7fc26809bbf0 2024-01-24T23:23:15.872 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 1 -- 172.21.15.132:0/3374693164 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1231571164 0 0) 0x7fc254003530 con 0x7fc26809b810 2024-01-24T23:23:15.872 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 1 -- 172.21.15.132:0/3374693164 >> v1:172.21.15.115:6792/0 conn(0x7fc26809bbf0 legacy=0x7fc268071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.872 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 1 -- 172.21.15.132:0/3374693164 >> v1:172.21.15.115:6790/0 conn(0x7fc2681370a0 legacy=0x7fc26813ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.872 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 10 monclient: _finish_hunting 0 2024-01-24T23:23:15.872 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 1 monclient: found mon.noname-h 2024-01-24T23:23:15.872 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:15.872 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:23:15.872 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 1 -- 172.21.15.132:0/3374693164 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc26809c050 con 0x7fc26809b810 2024-01-24T23:23:15.872 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 10 monclient: _finish_auth 0 2024-01-24T23:23:15.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 10 monclient: _check_auth_tickets 2024-01-24T23:23:15.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:15.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 1 -- 172.21.15.132:0/3374693164 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc2640017c0 con 0x7fc26809b810 2024-01-24T23:23:15.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 5 monclient: authenticate success, global_id 4955 2024-01-24T23:23:15.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:15.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:15.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 10 monclient: dump: 2024-01-24T23:23:15.873 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:15.873 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.873 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.873 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.873 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.873 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:15.873 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.873 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.874 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.874 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.874 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.874 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.874 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.874 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.874 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.874 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:15.874 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:15.874 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 1 -- 172.21.15.132:0/3374693164 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc264001a30 con 0x7fc26809b810 2024-01-24T23:23:15.874 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:15.874 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:15.874 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 1 -- 172.21.15.132:0/3374693164 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc264004bf0 con 0x7fc26809b810 2024-01-24T23:23:15.874 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 4 set_mon_vals no callback set 2024-01-24T23:23:15.874 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.874 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:15.875 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26cfb3640 10 monclient: dump: 2024-01-24T23:23:15.875 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:15.875 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.875 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.875 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.875 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.875 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:15.875 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.875 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.875 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.875 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.875 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.875 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.875 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.875 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.875 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.876 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:15.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 10 monclient: shutdown 2024-01-24T23:23:15.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:15.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 1 -- 172.21.15.132:0/3374693164 >> v1:172.21.15.132:6791/0 conn(0x7fc26809b810 legacy=0x7fc2680712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 1 -- 172.21.15.132:0/3374693164 shutdown_connections 2024-01-24T23:23:15.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 1 -- 172.21.15.132:0/3374693164 >> 172.21.15.132:0/3374693164 conn(0x7fc2680a1bc0 msgr2=0x7fc2680a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:15.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 1 -- 172.21.15.132:0/3374693164 shutdown_connections 2024-01-24T23:23:15.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 1 -- 172.21.15.132:0/3374693164 wait complete. 2024-01-24T23:23:15.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 10 monclient: build_initial_monmap 2024-01-24T23:23:15.876 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 10 monclient: monmap: 2024-01-24T23:23:15.876 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:15.876 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:15.877 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:15.877 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:15.877 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:15.877 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:15.877 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:15.877 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:15.877 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:15.877 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:15.877 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:15.877 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:15.877 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:15.877 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:15.877 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:15.877 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:15.877 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 1 Processor -- start 2024-01-24T23:23:15.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 1 -- start start 2024-01-24T23:23:15.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 10 monclient: init 2024-01-24T23:23:15.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:15.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 10 monclient: _add_conns ranks=[1,6,2,3,7,0,8,4,5] 2024-01-24T23:23:15.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 10 monclient(hunting): picked mon.noname-f con 0x7fc26809b810 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:15.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 10 monclient(hunting): picked mon.noname-d con 0x7fc26809bbf0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:15.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 10 monclient(hunting): picked mon.noname-b con 0x7fc2681370a0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:15.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc26812ab30 con 0x7fc26809b810 2024-01-24T23:23:15.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc26809b5e0 con 0x7fc2681370a0 2024-01-24T23:23:15.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc26812b0f0 con 0x7fc26809bbf0 2024-01-24T23:23:15.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:15.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc270219640 10 monclient(hunting): authenticate will time out at 1083.191516s 2024-01-24T23:23:15.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26dfb5640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fc26809b810 0x7fc2681cb350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:40864/0 (socket says 172.21.15.132:40864) 2024-01-24T23:23:15.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc26dfb5640 1 -- 172.21.15.132:0/4112692320 learned_addr learned my addr 172.21.15.132:0/4112692320 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:15.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc25e7fc640 1 -- 172.21.15.132:0/4112692320 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3338507826 0 0) 0x7fc26812ab30 con 0x7fc26809b810 2024-01-24T23:23:15.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc25e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.879 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc25e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.879 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc25e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.879 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc25e7fc640 10 monclient(hunting): my global_id is 5032 2024-01-24T23:23:15.879 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc25e7fc640 1 -- 172.21.15.132:0/4112692320 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc24c003530 con 0x7fc26809b810 2024-01-24T23:23:15.879 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc25e7fc640 1 -- 172.21.15.132:0/4112692320 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 750516586 0 0) 0x7fc26809b5e0 con 0x7fc2681370a0 2024-01-24T23:23:15.879 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc25e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.879 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc25e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.879 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc25e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.879 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc25e7fc640 10 monclient(hunting): my global_id is 84188 2024-01-24T23:23:15.879 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc25e7fc640 1 -- 172.21.15.132:0/4112692320 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc26812ab30 con 0x7fc2681370a0 2024-01-24T23:23:15.879 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc25e7fc640 1 -- 172.21.15.132:0/4112692320 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3083736787 0 0) 0x7fc26812b0f0 con 0x7fc26809bbf0 2024-01-24T23:23:15.879 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc25e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.879 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.864+0000 7fc25e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.879 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.879 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 10 monclient(hunting): my global_id is 5001 2024-01-24T23:23:15.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 1 -- 172.21.15.132:0/4112692320 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc26809b5e0 con 0x7fc26809bbf0 2024-01-24T23:23:15.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 1 -- 172.21.15.132:0/4112692320 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1658538912 0 0) 0x7fc24c003530 con 0x7fc26809b810 2024-01-24T23:23:15.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 1 -- 172.21.15.132:0/4112692320 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc26812b0f0 con 0x7fc26809b810 2024-01-24T23:23:15.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 1 -- 172.21.15.132:0/4112692320 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc264001860 con 0x7fc26809b810 2024-01-24T23:23:15.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:23:15.880 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 10 monclient(hunting): dump: 2024-01-24T23:23:15.880 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:15.880 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.880 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.880 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.880 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.880 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:15.880 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.880 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.881 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.881 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.881 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.881 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.881 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.881 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.881 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.881 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:15.881 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:15.881 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 1 -- 172.21.15.132:0/4112692320 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2880332414 0 0) 0x7fc26812b0f0 con 0x7fc26809b810 2024-01-24T23:23:15.881 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 1 -- 172.21.15.132:0/4112692320 >> v1:172.21.15.115:6792/0 conn(0x7fc26809bbf0 legacy=0x7fc268130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.881 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 1 -- 172.21.15.132:0/4112692320 >> v1:172.21.15.115:6790/0 conn(0x7fc2681370a0 legacy=0x7fc2681d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.881 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:23:15.881 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 1 monclient: found mon.f 2024-01-24T23:23:15.881 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:15.881 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:15.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 1 -- 172.21.15.132:0/4112692320 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2680a1620 con 0x7fc26809b810 2024-01-24T23:23:15.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 10 monclient: _finish_auth 0 2024-01-24T23:23:15.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:23:15.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:15.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc270219640 5 monclient: authenticate success, global_id 5032 2024-01-24T23:23:15.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc270219640 10 monclient: _renew_subs 2024-01-24T23:23:15.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc270219640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:15.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc270219640 1 -- 172.21.15.132:0/4112692320 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc26812a850 con 0x7fc26809b810 2024-01-24T23:23:15.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc270219640 10 monclient: _renew_subs 2024-01-24T23:23:15.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc270219640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:15.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc270219640 1 -- 172.21.15.132:0/4112692320 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc268135450 con 0x7fc26809b810 2024-01-24T23:23:15.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 1 -- 172.21.15.132:0/4112692320 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc2640011c0 con 0x7fc26809b810 2024-01-24T23:23:15.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:15.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 1 -- 172.21.15.132:0/4112692320 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc264005160 con 0x7fc26809b810 2024-01-24T23:23:15.882 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.883 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25ffff640 4 set_mon_vals no callback set 2024-01-24T23:23:15.883 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:23:15.883 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 10 monclient: dump: 2024-01-24T23:23:15.883 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:15.883 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.883 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.883 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.883 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.883 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:15.883 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.883 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.883 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.883 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.883 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.883 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.884 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.884 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.884 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.884 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:15.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 1 -- 172.21.15.132:0/4112692320 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fc26400fbe0 con 0x7fc26809b810 2024-01-24T23:23:15.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc270219640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:15.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc270219640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:15.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc270219640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:15.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc270219640 1 -- 172.21.15.132:0/4112692320 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc250005090 con 0x7fc26809b810 2024-01-24T23:23:15.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc25e7fc640 1 -- 172.21.15.132:0/4112692320 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(48..48 src has 1..48) v4 ==== 3994+0+0 (unknown 1916735388 0 0) 0x7fc264043ad0 con 0x7fc26809b810 2024-01-24T23:23:15.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.868+0000 7fc26d7b4640 10 monclient: get_auth_request con 0x7fc24c037110 auth_method 0 2024-01-24T23:23:15.885 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.872+0000 7fc25e7fc640 1 -- 172.21.15.132:0/4112692320 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fc264011090 con 0x7fc26809b810 2024-01-24T23:23:15.885 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.872+0000 7fc25e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:15.885 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:15.872+0000 7fc25e7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:15.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 10 monclient: get_monmap_and_config 2024-01-24T23:23:15.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 10 monclient: build_initial_monmap 2024-01-24T23:23:15.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 10 monclient: monmap: 2024-01-24T23:23:15.966 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:15.967 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.967 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:15.967137+0000 2024-01-24T23:23:15.967 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:15.967137+0000 2024-01-24T23:23:15.967 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:15.967 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.967 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:15.967 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:15.967 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:15.967 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:15.967 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:15.967 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:15.967 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:15.967 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:15.967 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:15.968 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 1 Processor -- start 2024-01-24T23:23:15.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 1 -- start start 2024-01-24T23:23:15.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 10 monclient: init 2024-01-24T23:23:15.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:15.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 10 monclient: _add_conns ranks=[5,4,2,8,6,0,7,1,3] 2024-01-24T23:23:15.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 10 monclient(hunting): picked mon.noname-h con 0x7f5388133bf0 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:15.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 10 monclient(hunting): picked mon.noname-c con 0x7f5388133fd0 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:15.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 10 monclient(hunting): picked mon.noname-b con 0x7f5388134430 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:15.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f538804ed70 con 0x7f5388134430 2024-01-24T23:23:15.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f538809af30 con 0x7f5388133fd0 2024-01-24T23:23:15.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f538809b210 con 0x7f5388133bf0 2024-01-24T23:23:15.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:15.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 10 monclient(hunting): authenticate will time out at 1074.889717s 2024-01-24T23:23:15.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538d040640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f5388133fd0 0x7f538813bdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:48612/0 (socket says 172.21.15.115:48612) 2024-01-24T23:23:15.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538d040640 1 -- 172.21.15.115:0/933979941 learned_addr learned my addr 172.21.15.115:0/933979941 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:15.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538e042640 1 --1- 172.21.15.115:0/933979941 >> v1:172.21.15.115:6790/0 conn(0x7f5388134430 0x7f538813f500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:47618/0 (socket says 172.21.15.115:47618) 2024-01-24T23:23:15.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 1 -- 172.21.15.115:0/933979941 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2603969289 0 0) 0x7f538809af30 con 0x7f5388133fd0 2024-01-24T23:23:15.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 10 monclient(hunting): my global_id is 74956 2024-01-24T23:23:15.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 1 -- 172.21.15.115:0/933979941 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5370003530 con 0x7f5388133fd0 2024-01-24T23:23:15.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 1 -- 172.21.15.115:0/933979941 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3413239434 0 0) 0x7f538804ed70 con 0x7f5388134430 2024-01-24T23:23:15.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 10 monclient(hunting): my global_id is 84197 2024-01-24T23:23:15.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 1 -- 172.21.15.115:0/933979941 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f538809af30 con 0x7f5388134430 2024-01-24T23:23:15.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 1 -- 172.21.15.115:0/933979941 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2037814899 0 0) 0x7f5370003530 con 0x7f5388133fd0 2024-01-24T23:23:15.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 1 -- 172.21.15.115:0/933979941 >> v1:172.21.15.132:6791/0 conn(0x7f5388133bf0 legacy=0x7f538809bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 1 -- 172.21.15.115:0/933979941 >> v1:172.21.15.115:6790/0 conn(0x7f5388134430 legacy=0x7f538813f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 10 monclient: _finish_hunting 0 2024-01-24T23:23:15.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 1 monclient: found mon.noname-c 2024-01-24T23:23:15.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:15.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:23:15.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 1 -- 172.21.15.115:0/933979941 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538809b4f0 con 0x7f5388133fd0 2024-01-24T23:23:15.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 10 monclient: _finish_auth 0 2024-01-24T23:23:15.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 10 monclient: _check_auth_tickets 2024-01-24T23:23:15.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:15.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 1 -- 172.21.15.115:0/933979941 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f53800017c0 con 0x7f5388133fd0 2024-01-24T23:23:15.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 5 monclient: authenticate success, global_id 74956 2024-01-24T23:23:15.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:15.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:23:15.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 10 monclient: dump: 2024-01-24T23:23:15.979 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.979 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.979 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.980 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.980 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.980 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.980 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.980 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.980 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.980 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.980 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.980 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.980 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.980 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.980 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.980 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:15.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 1 -- 172.21.15.115:0/933979941 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5380001a20 con 0x7f5388133fd0 2024-01-24T23:23:15.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:15.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 1 -- 172.21.15.115:0/933979941 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5380004bf0 con 0x7f5388133fd0 2024-01-24T23:23:15.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:15.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 4 set_mon_vals no callback set 2024-01-24T23:23:15.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:23:15.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538c83f640 10 monclient: dump: 2024-01-24T23:23:15.981 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.981 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.981 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.981 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.981 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.981 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.982 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.982 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.982 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.982 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.982 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.982 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.982 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.982 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.982 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.982 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 10 monclient: shutdown 2024-01-24T23:23:15.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:15.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 1 -- 172.21.15.115:0/933979941 >> v1:172.21.15.115:6791/0 conn(0x7f5388133fd0 legacy=0x7f538813bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 1 -- 172.21.15.115:0/933979941 shutdown_connections 2024-01-24T23:23:15.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 1 -- 172.21.15.115:0/933979941 >> 172.21.15.115:0/933979941 conn(0x7f53880a9910 msgr2=0x7f53880a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:15.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 1 -- 172.21.15.115:0/933979941 shutdown_connections 2024-01-24T23:23:15.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 1 -- 172.21.15.115:0/933979941 wait complete. 2024-01-24T23:23:15.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 10 monclient: build_initial_monmap 2024-01-24T23:23:15.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.964+0000 7f538faa5640 10 monclient: monmap: 2024-01-24T23:23:15.983 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:15.983 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:15.983 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:15.983 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:15.983 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:15.983 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:15.983 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:15.983 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:15.983 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:15.983 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:15.983 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:15.984 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:15.984 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:15.984 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:15.984 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:15.984 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f538faa5640 1 Processor -- start 2024-01-24T23:23:15.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f538faa5640 1 -- start start 2024-01-24T23:23:15.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f538faa5640 10 monclient: init 2024-01-24T23:23:15.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f538faa5640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:15.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f538faa5640 10 monclient: _add_conns ranks=[5,3,1,8,4,7,6,0,2] 2024-01-24T23:23:15.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f538faa5640 10 monclient(hunting): picked mon.noname-h con 0x7f5388133bf0 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:15.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f538faa5640 10 monclient(hunting): picked mon.noname-g con 0x7f5388133fd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:15.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f538faa5640 10 monclient(hunting): picked mon.noname-f con 0x7f5388134430 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:15.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f538faa5640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53881c82f0 con 0x7f5388134430 2024-01-24T23:23:15.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f538faa5640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53880a5960 con 0x7f5388133fd0 2024-01-24T23:23:15.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f538faa5640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f538809fe30 con 0x7f5388133bf0 2024-01-24T23:23:15.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f538faa5640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:15.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f538faa5640 10 monclient(hunting): authenticate will time out at 1074.892904s 2024-01-24T23:23:15.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f538d841640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f5388133bf0 0x7f538813f250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.115:51776/0 (socket says 172.21.15.115:51776) 2024-01-24T23:23:15.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f538d841640 1 -- 172.21.15.115:0/1427064331 learned_addr learned my addr 172.21.15.115:0/1427064331 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:15.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3745661916 0 0) 0x7f538809fe30 con 0x7f5388133bf0 2024-01-24T23:23:15.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient(hunting): my global_id is 4973 2024-01-24T23:23:15.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5374003530 con 0x7f5388133bf0 2024-01-24T23:23:15.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1991937124 0 0) 0x7f53881c82f0 con 0x7f5388134430 2024-01-24T23:23:15.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient(hunting): my global_id is 5041 2024-01-24T23:23:15.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f538809fe30 con 0x7f5388134430 2024-01-24T23:23:15.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4197536593 0 0) 0x7f53880a5960 con 0x7f5388133fd0 2024-01-24T23:23:15.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient(hunting): my global_id is 4935 2024-01-24T23:23:15.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53881c82f0 con 0x7f5388133fd0 2024-01-24T23:23:15.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 825887287 0 0) 0x7f5374003530 con 0x7f5388133bf0 2024-01-24T23:23:15.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53880a5960 con 0x7f5388133bf0 2024-01-24T23:23:15.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5384001ef0 con 0x7f5388133bf0 2024-01-24T23:23:15.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:15.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient(hunting): dump: 2024-01-24T23:23:15.987 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.987 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.987 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.987 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.987 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.987 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.987 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.987 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.987 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.987 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.987 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.987 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.988 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.988 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.988 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.988 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:15.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2943984305 0 0) 0x7f538809fe30 con 0x7f5388134430 2024-01-24T23:23:15.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5374003530 con 0x7f5388134430 2024-01-24T23:23:15.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5378001980 con 0x7f5388134430 2024-01-24T23:23:15.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:23:15.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient(hunting): dump: 2024-01-24T23:23:15.988 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.988 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.988 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.989 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.989 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.989 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.989 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.989 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.989 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.989 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.989 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.989 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.989 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.989 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.989 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.989 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2464456058 0 0) 0x7f53881c82f0 con 0x7f5388133fd0 2024-01-24T23:23:15.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f538809fe30 con 0x7f5388133fd0 2024-01-24T23:23:15.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 177848367 0 0) 0x7f5374003530 con 0x7f5388134430 2024-01-24T23:23:15.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 >> v1:172.21.15.132:6791/0 conn(0x7f5388133bf0 legacy=0x7f538813f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 >> v1:172.21.15.132:6790/0 conn(0x7f5388133fd0 legacy=0x7f53881d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:23:15.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 monclient: found mon.f 2024-01-24T23:23:15.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:15.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:15.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538813a320 con 0x7f5388134430 2024-01-24T23:23:15.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient: _finish_auth 0 2024-01-24T23:23:15.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:23:15.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:15.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f538faa5640 5 monclient: authenticate success, global_id 5041 2024-01-24T23:23:15.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f538faa5640 10 monclient: _renew_subs 2024-01-24T23:23:15.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f538faa5640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:15.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f538faa5640 1 -- 172.21.15.115:0/1427064331 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538813a600 con 0x7f5388134430 2024-01-24T23:23:15.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f538faa5640 10 monclient: _renew_subs 2024-01-24T23:23:15.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f538faa5640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:15.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f538faa5640 1 -- 172.21.15.115:0/1427064331 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f538813f960 con 0x7f5388134430 2024-01-24T23:23:15.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5378001c90 con 0x7f5388134430 2024-01-24T23:23:15.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:15.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5378004de0 con 0x7f5388134430 2024-01-24T23:23:15.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537ffff640 4 set_mon_vals no callback set 2024-01-24T23:23:15.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:23:15.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient: dump: 2024-01-24T23:23:15.992 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.992 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.992 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.992 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.992 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.992 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.992 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.993 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.993 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.993 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.993 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.993 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.993 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.993 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.993 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.993 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f538faa5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:15.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f538faa5640 1 -- 172.21.15.115:0/1427064331 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f53680065d0 con 0x7f5368002870 2024-01-24T23:23:15.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f537800f860 con 0x7f5388134430 2024-01-24T23:23:15.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162825933 0 0) 0x7f53680065d0 con 0x7f5368002870 2024-01-24T23:23:15.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 10 monclient(hunting): my global_id is 4982 2024-01-24T23:23:15.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5374003530 con 0x7f5368002870 2024-01-24T23:23:15.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.968+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(48..48 src has 1..48) v4 ==== 3994+0+0 (unknown 1916735388 0 0) 0x7f53780436e0 con 0x7f5388134430 2024-01-24T23:23:15.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.972+0000 7f538d040640 10 monclient: get_auth_request con 0x7f5374037960 auth_method 0 2024-01-24T23:23:15.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.972+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2434198821 0 0) 0x7f5374003530 con 0x7f5368002870 2024-01-24T23:23:15.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.972+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53680065d0 con 0x7f5368002870 2024-01-24T23:23:15.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.972+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5384004200 con 0x7f5368002870 2024-01-24T23:23:15.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.972+0000 7f537e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.972+0000 7f537e7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:15.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.972+0000 7f537e7fc640 10 monclient: dump: 2024-01-24T23:23:15.994 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.994 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.994 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.994 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.995 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.995 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.995 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.995 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.995 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.995 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:15.995 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:15.995 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:15.995 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:15.995 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:15.995 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:15.996 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:15.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.972+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3237237507 0 0) 0x7f53680065d0 con 0x7f5368002870 2024-01-24T23:23:15.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.972+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 --> v1:172.21.15.132:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f536800a120 con 0x7f5368002870 2024-01-24T23:23:15.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.972+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f53840046a0 con 0x7f5368002870 2024-01-24T23:23:15.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.972+0000 7f537e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:15.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.972+0000 7f537e7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:15.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.972+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 >> v1:172.21.15.132:6791/0 conn(0x7f5368002870 legacy=0x7f5368004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:15.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.972+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 >> v1:172.21.15.132:6791/0 conn(0x7f5368002870 legacy=0x7f5368004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:15.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.976+0000 7f538faa5640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:23:15.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.976+0000 7f538faa5640 1 -- 172.21.15.115:0/1427064331 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5368009e10 con 0x7f53680090d0 2024-01-24T23:23:15.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3777197974 0 0) 0x7f5368009e10 con 0x7f53680090d0 2024-01-24T23:23:15.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f537e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:15.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f537e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:15.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f537e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:15.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f537e7fc640 10 monclient(hunting): my global_id is 4991 2024-01-24T23:23:15.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53680065d0 con 0x7f53680090d0 2024-01-24T23:23:15.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1850322116 0 0) 0x7f53680065d0 con 0x7f53680090d0 2024-01-24T23:23:15.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5368009e10 con 0x7f53680090d0 2024-01-24T23:23:15.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5384006060 con 0x7f53680090d0 2024-01-24T23:23:15.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f537e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:15.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f537e7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:15.999 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f537e7fc640 10 monclient: dump: 2024-01-24T23:23:15.999 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:15.999 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:15.999 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.999 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:15.999 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:15.999 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:15.999 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:15.999 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:15.999 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:15.999 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:16.000 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:16.000 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:16.000 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:16.000 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:16.000 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:16.000 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:16.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3599572553 0 0) 0x7f5368009e10 con 0x7f53680090d0 2024-01-24T23:23:16.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 --> v1:172.21.15.132:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f536800c310 con 0x7f53680090d0 2024-01-24T23:23:16.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 2355060253) 0x7f5384008310 con 0x7f53680090d0 2024-01-24T23:23:16.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f537e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:23:16.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f537e7fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:16.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 >> v1:172.21.15.132:6791/0 conn(0x7f53680090d0 legacy=0x7f5368009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f537e7fc640 1 -- 172.21.15.115:0/1427064331 >> v1:172.21.15.132:6791/0 conn(0x7f53680090d0 legacy=0x7f5368009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:16.001 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:23:16.001 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:16.001 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:16.001 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:23:16.001 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 16, 2024-01-24T23:23:16.001 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:23:16.001 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:23:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:23:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:23:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:23:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:23:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:23:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:23:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:23:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:23:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 1, 2024-01-24T23:23:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:23:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:23:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:16.002 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:23:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:23:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:16.003 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:16.004 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:16.004 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:16.004 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:16.004 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:16.004 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:16.004 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:16.004 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:16.004 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:16.004 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:16.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.004 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:23:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:23:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:23:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:23:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:23:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:23:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:23:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:23:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:23:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:23:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:23:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:23:16.005 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:23:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:23:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:23:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:23:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:23:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:16.007 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:23:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:23:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:16.008 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:23:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:23:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:23:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:23:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:23:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:23:16.010 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:23:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:16.011 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.012 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:16.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.012 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:23:16.012 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:23:16.012 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:16.012 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:16.012 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:16.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.012 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:16.012 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:23:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:23:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:23:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:16.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:23:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.014 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:23:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:23:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:23:16.015 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:16.016 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.016 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:16.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.016 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:23:16.019 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:23:16.019 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:16.019 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:16.019 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:16.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.020 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:16.020 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:16.020 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:16.020 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:16.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.020 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:16.020 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:23:16.020 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:16.020 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.020 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:16.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.020 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:23:16.021 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:23:16.021 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:16.021 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:16.021 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:16.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.021 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:16.021 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:16.021 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:16.021 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:16.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:23:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:23:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:23:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:23:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:23:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:23:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:16.023 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:16.024 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.024 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:16.024 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:23:16.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.024 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:16.024 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:16.024 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:16.024 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.024 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:16.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.024 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:23:16.024 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:23:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f538faa5640 1 -- 172.21.15.115:0/1427064331 >> v1:172.21.15.132:6812/1835401922 conn(0x7f5374037960 legacy=0x7f5374039e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f538faa5640 10 monclient: shutdown 2024-01-24T23:23:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f538faa5640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f538faa5640 1 -- 172.21.15.115:0/1427064331 >> v1:172.21.15.132:6789/0 conn(0x7f5388134430 legacy=0x7f53881d3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f538e042640 1 -- 172.21.15.115:0/1427064331 reap_dead start 2024-01-24T23:23:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f538e042640 1 -- 172.21.15.115:0/1427064331 reap_dead start 2024-01-24T23:23:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f538faa5640 1 -- 172.21.15.115:0/1427064331 shutdown_connections 2024-01-24T23:23:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f538faa5640 1 -- 172.21.15.115:0/1427064331 >> 172.21.15.115:0/1427064331 conn(0x7f53880a9910 msgr2=0x7f538813fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f538faa5640 1 -- 172.21.15.115:0/1427064331 shutdown_connections 2024-01-24T23:23:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:15.980+0000 7f538faa5640 1 -- 172.21.15.115:0/1427064331 wait complete. 2024-01-24T23:23:16.027 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:23:16.027 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-01-24T23:23:16.027 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-01-24T23:23:16.029 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.028+0000 7fc270219640 10 start_mon_command cmd=[{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}] 2024-01-24T23:23:16.029 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.028+0000 7fc270219640 10 monclient: _send_command 2 [{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}] 2024-01-24T23:23:16.030 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.028+0000 7fc270219640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:16.030 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.028+0000 7fc270219640 1 -- 172.21.15.132:0/4112692320 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]} v 0) v1 -- 0x7fc250005340 con 0x7fc26809b810 2024-01-24T23:23:16.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 10 monclient: get_monmap_and_config 2024-01-24T23:23:16.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 10 monclient: build_initial_monmap 2024-01-24T23:23:16.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 10 monclient: monmap: 2024-01-24T23:23:16.143 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:16.143 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:16.143 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:16.143061+0000 2024-01-24T23:23:16.143 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:16.143061+0000 2024-01-24T23:23:16.143 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:16.143 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:16.143 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:16.143 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:16.143 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:16.143 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:16.143 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:16.143 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:16.143 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:16.144 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:16.144 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:16.144 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:16.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 1 Processor -- start 2024-01-24T23:23:16.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 1 -- start start 2024-01-24T23:23:16.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 10 monclient: init 2024-01-24T23:23:16.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:16.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 10 monclient: _add_conns ranks=[1,2,8,7,3,6,0,4,5] 2024-01-24T23:23:16.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 10 monclient(hunting): picked mon.noname-f con 0x7fbbe0133bf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 10 monclient(hunting): picked mon.noname-b con 0x7fbbe0133fd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 10 monclient(hunting): picked mon.noname-e con 0x7fbbe0134430 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe004ed70 con 0x7fbbe0133bf0 2024-01-24T23:23:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe009af30 con 0x7fbbe0133fd0 2024-01-24T23:23:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe009b210 con 0x7fbbe0134430 2024-01-24T23:23:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 10 monclient(hunting): authenticate will time out at 1075.065853s 2024-01-24T23:23:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbded9d640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fbbe0134430 0x7fbbe013f500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:52940/0 (socket says 172.21.15.115:52940) 2024-01-24T23:23:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbddd9b640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fbbe0133fd0 0x7fbbe013bdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:47626/0 (socket says 172.21.15.115:47626) 2024-01-24T23:23:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbded9d640 1 -- 172.21.15.115:0/3138353444 learned_addr learned my addr 172.21.15.115:0/3138353444 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 1 -- 172.21.15.115:0/3138353444 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1870866349 0 0) 0x7fbbe009af30 con 0x7fbbe0133fd0 2024-01-24T23:23:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 10 monclient(hunting): _negotiate 2024-01-24T23:23:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 10 monclient(hunting): my global_id is 84206 2024-01-24T23:23:16.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 1 -- 172.21.15.115:0/3138353444 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbcc003530 con 0x7fbbe0133fd0 2024-01-24T23:23:16.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 1 -- 172.21.15.115:0/3138353444 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3966135651 0 0) 0x7fbbe009b210 con 0x7fbbe0134430 2024-01-24T23:23:16.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 10 monclient(hunting): _negotiate 2024-01-24T23:23:16.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:16.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:16.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 10 monclient(hunting): my global_id is 84176 2024-01-24T23:23:16.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 1 -- 172.21.15.115:0/3138353444 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbe009af30 con 0x7fbbe0134430 2024-01-24T23:23:16.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 1 -- 172.21.15.115:0/3138353444 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2143209651 0 0) 0x7fbbe004ed70 con 0x7fbbe0133bf0 2024-01-24T23:23:16.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 10 monclient(hunting): _negotiate 2024-01-24T23:23:16.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:16.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:16.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 10 monclient(hunting): my global_id is 5050 2024-01-24T23:23:16.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 1 -- 172.21.15.115:0/3138353444 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbe009b210 con 0x7fbbe0133bf0 2024-01-24T23:23:16.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 1 -- 172.21.15.115:0/3138353444 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1367679579 0 0) 0x7fbbe009af30 con 0x7fbbe0134430 2024-01-24T23:23:16.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 1 -- 172.21.15.115:0/3138353444 >> v1:172.21.15.115:6790/0 conn(0x7fbbe0133fd0 legacy=0x7fbbe013bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 1 -- 172.21.15.115:0/3138353444 >> v1:172.21.15.132:6789/0 conn(0x7fbbe0133bf0 legacy=0x7fbbe009bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 10 monclient: _finish_hunting 0 2024-01-24T23:23:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 1 monclient: found mon.noname-e 2024-01-24T23:23:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:23:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 1 -- 172.21.15.115:0/3138353444 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe009b4f0 con 0x7fbbe0134430 2024-01-24T23:23:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 10 monclient: _finish_auth 0 2024-01-24T23:23:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 10 monclient: _check_auth_tickets 2024-01-24T23:23:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 1 -- 172.21.15.115:0/3138353444 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbbd8001720 con 0x7fbbe0134430 2024-01-24T23:23:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 5 monclient: authenticate success, global_id 84176 2024-01-24T23:23:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:23:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 10 monclient: dump: 2024-01-24T23:23:16.156 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:16.156 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:16.156 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.156 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.156 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:16.156 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:16.156 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:16.156 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:16.156 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:16.156 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:16.156 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:16.156 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:16.156 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:16.156 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:16.157 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:16.157 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:16.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:16.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 1 -- 172.21.15.115:0/3138353444 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbbd8001940 con 0x7fbbe0134430 2024-01-24T23:23:16.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:16.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:16.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 1 -- 172.21.15.115:0/3138353444 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbbd80048b0 con 0x7fbbe0134430 2024-01-24T23:23:16.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 4 set_mon_vals no callback set 2024-01-24T23:23:16.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:16.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:16.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbdd59a640 10 monclient: dump: 2024-01-24T23:23:16.157 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:16.157 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:16.157 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.157 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.157 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:16.158 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 10 monclient: shutdown 2024-01-24T23:23:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 1 -- 172.21.15.115:0/3138353444 >> v1:172.21.15.115:6793/0 conn(0x7fbbe0134430 legacy=0x7fbbe013f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 1 -- 172.21.15.115:0/3138353444 shutdown_connections 2024-01-24T23:23:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 1 -- 172.21.15.115:0/3138353444 >> 172.21.15.115:0/3138353444 conn(0x7fbbe00a9910 msgr2=0x7fbbe00a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 1 -- 172.21.15.115:0/3138353444 shutdown_connections 2024-01-24T23:23:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 1 -- 172.21.15.115:0/3138353444 wait complete. 2024-01-24T23:23:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 10 monclient: build_initial_monmap 2024-01-24T23:23:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.140+0000 7fbbe4d2e640 10 monclient: monmap: 2024-01-24T23:23:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:16.160 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:16.160 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:16.160 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:16.160 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:16.160 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:16.160 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:16.160 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:16.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbe4d2e640 1 Processor -- start 2024-01-24T23:23:16.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbe4d2e640 1 -- start start 2024-01-24T23:23:16.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbe4d2e640 10 monclient: init 2024-01-24T23:23:16.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbe4d2e640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:16.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbe4d2e640 10 monclient: _add_conns ranks=[1,3,5,2,6,7,4,0,8] 2024-01-24T23:23:16.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbe4d2e640 10 monclient(hunting): picked mon.noname-f con 0x7fbbe0133bf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:16.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbe4d2e640 10 monclient(hunting): picked mon.noname-g con 0x7fbbe0133fd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:16.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbe4d2e640 10 monclient(hunting): picked mon.noname-h con 0x7fbbe0134430 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:16.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbe4d2e640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe013a980 con 0x7fbbe0133bf0 2024-01-24T23:23:16.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbe4d2e640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe01394d0 con 0x7fbbe0133fd0 2024-01-24T23:23:16.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbe4d2e640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe013a3c0 con 0x7fbbe0134430 2024-01-24T23:23:16.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbe4d2e640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:16.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbe4d2e640 10 monclient(hunting): authenticate will time out at 1075.068910s 2024-01-24T23:23:16.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbddd9b640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fbbe0133fd0 0x7fbbe01d0470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.115:38852/0 (socket says 172.21.15.115:38852) 2024-01-24T23:23:16.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbddd9b640 1 -- 172.21.15.115:0/2465844462 learned_addr learned my addr 172.21.15.115:0/2465844462 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:16.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbde59c640 1 --1- 172.21.15.115:0/2465844462 >> v1:172.21.15.132:6789/0 conn(0x7fbbe0133bf0 0x7fbbe013f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.115:52528/0 (socket says 172.21.15.115:52528) 2024-01-24T23:23:16.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1255221568 0 0) 0x7fbbe013a980 con 0x7fbbe0133bf0 2024-01-24T23:23:16.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:16.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:16.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:16.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient(hunting): my global_id is 5059 2024-01-24T23:23:16.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbc0003530 con 0x7fbbe0133bf0 2024-01-24T23:23:16.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228015150 0 0) 0x7fbbe01394d0 con 0x7fbbe0133fd0 2024-01-24T23:23:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient(hunting): my global_id is 4944 2024-01-24T23:23:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbe013a980 con 0x7fbbe0133fd0 2024-01-24T23:23:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3560300972 0 0) 0x7fbbe013a3c0 con 0x7fbbe0134430 2024-01-24T23:23:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient(hunting): my global_id is 5000 2024-01-24T23:23:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbe01394d0 con 0x7fbbe0134430 2024-01-24T23:23:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2398643272 0 0) 0x7fbbe01394d0 con 0x7fbbe0134430 2024-01-24T23:23:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbbe013a3c0 con 0x7fbbe0134430 2024-01-24T23:23:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 802565698 0 0) 0x7fbbe013a980 con 0x7fbbe0133fd0 2024-01-24T23:23:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbbe01394d0 con 0x7fbbe0133fd0 2024-01-24T23:23:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbbd8001620 con 0x7fbbe0134430 2024-01-24T23:23:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient(hunting): dump: 2024-01-24T23:23:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:16.164 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 131914313 0 0) 0x7fbbc0003530 con 0x7fbbe0133bf0 2024-01-24T23:23:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbbe013a980 con 0x7fbbe0133bf0 2024-01-24T23:23:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbbd0001920 con 0x7fbbe0133fd0 2024-01-24T23:23:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient(hunting): dump: 2024-01-24T23:23:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:16.165 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.165 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.165 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:16.165 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:16.165 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:16.165 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:16.165 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:16.165 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:16.165 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:16.165 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:16.165 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:16.165 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:16.165 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:16.165 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:16.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbbc8001a00 con 0x7fbbe0133bf0 2024-01-24T23:23:16.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:16.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:23:16.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient(hunting): dump: 2024-01-24T23:23:16.166 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:16.166 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:16.166 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.166 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.166 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:16.166 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:16.166 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:16.166 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:16.166 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:16.166 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:16.166 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:16.166 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:16.166 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:16.167 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:16.167 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:16.167 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:16.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3842607945 0 0) 0x7fbbe013a3c0 con 0x7fbbe0134430 2024-01-24T23:23:16.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 >> v1:172.21.15.132:6790/0 conn(0x7fbbe0133fd0 legacy=0x7fbbe01d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 >> v1:172.21.15.132:6789/0 conn(0x7fbbe0133bf0 legacy=0x7fbbe013f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:16.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 1 monclient: found mon.h 2024-01-24T23:23:16.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:16.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:16.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe013a6a0 con 0x7fbbe0134430 2024-01-24T23:23:16.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient: _finish_auth 0 2024-01-24T23:23:16.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:16.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbe4d2e640 5 monclient: authenticate success, global_id 5000 2024-01-24T23:23:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbe4d2e640 10 monclient: _renew_subs 2024-01-24T23:23:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbe4d2e640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbe4d2e640 1 -- 172.21.15.115:0/2465844462 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe013f8d0 con 0x7fbbe0134430 2024-01-24T23:23:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbe4d2e640 10 monclient: _renew_subs 2024-01-24T23:23:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbe4d2e640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbe4d2e640 1 -- 172.21.15.115:0/2465844462 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe01398e0 con 0x7fbbe0134430 2024-01-24T23:23:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbbd8004aa0 con 0x7fbbe0134430 2024-01-24T23:23:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbbd8001010 con 0x7fbbe0134430 2024-01-24T23:23:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbdcd99640 4 set_mon_vals no callback set 2024-01-24T23:23:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbd6ffd640 10 monclient: dump: 2024-01-24T23:23:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:16.169 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:16.169 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.169 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.169 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:16.169 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:16.169 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:16.169 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:16.169 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:16.169 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:16.169 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:16.169 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:16.169 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:16.169 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:16.169 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:16.170 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbe4d2e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.144+0000 7fbbe4d2e640 1 -- 172.21.15.115:0/2465844462 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbbc40065d0 con 0x7fbbc4002870 2024-01-24T23:23:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.148+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fbbd8005090 con 0x7fbbe0134430 2024-01-24T23:23:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.148+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(48..48 src has 1..48) v4 ==== 3994+0+0 (unknown 1916735388 0 0) 0x7fbbd80436f0 con 0x7fbbe0134430 2024-01-24T23:23:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.148+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3863265373 0 0) 0x7fbbc40065d0 con 0x7fbbc4002870 2024-01-24T23:23:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.148+0000 7fbbd6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.148+0000 7fbbd6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.148+0000 7fbbd6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.148+0000 7fbbd6ffd640 10 monclient(hunting): my global_id is 4993 2024-01-24T23:23:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.148+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbe013a3c0 con 0x7fbbc4002870 2024-01-24T23:23:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.148+0000 7fbbddd9b640 10 monclient: get_auth_request con 0x7fbbc0037bc0 auth_method 0 2024-01-24T23:23:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.148+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3895845594 0 0) 0x7fbbe013a3c0 con 0x7fbbc4002870 2024-01-24T23:23:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.148+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbbc40065d0 con 0x7fbbc4002870 2024-01-24T23:23:16.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.148+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbbc8003940 con 0x7fbbc4002870 2024-01-24T23:23:16.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.148+0000 7fbbd6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:16.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.148+0000 7fbbd6ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:16.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.148+0000 7fbbd6ffd640 10 monclient: dump: 2024-01-24T23:23:16.171 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:16.171 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:16.171 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.171 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.171 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:16.171 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:16.171 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:16.171 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:16.171 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:16.171 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:16.172 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.148+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2962122211 0 0) 0x7fbbc40065d0 con 0x7fbbc4002870 2024-01-24T23:23:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.148+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 --> v1:172.21.15.132:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbbc400a120 con 0x7fbbc4002870 2024-01-24T23:23:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.148+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 <== mon.7 v1:172.21.15.132:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fbbc8003e40 con 0x7fbbc4002870 2024-01-24T23:23:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.148+0000 7fbbd6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.148+0000 7fbbd6ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.148+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 >> v1:172.21.15.132:6792/0 conn(0x7fbbc4002870 legacy=0x7fbbc4004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.148+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 >> v1:172.21.15.132:6792/0 conn(0x7fbbc4002870 legacy=0x7fbbc4004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:16.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbe4d2e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:23:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbe4d2e640 1 -- 172.21.15.115:0/2465844462 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbbc4009e10 con 0x7fbbc40090d0 2024-01-24T23:23:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2586668688 0 0) 0x7fbbc4009e10 con 0x7fbbc40090d0 2024-01-24T23:23:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbd6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbd6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbd6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbd6ffd640 10 monclient(hunting): my global_id is 5002 2024-01-24T23:23:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbc40065d0 con 0x7fbbc40090d0 2024-01-24T23:23:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 358475035 0 0) 0x7fbbc40065d0 con 0x7fbbc40090d0 2024-01-24T23:23:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbbc4009e10 con 0x7fbbc40090d0 2024-01-24T23:23:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbbc80057d0 con 0x7fbbc40090d0 2024-01-24T23:23:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbd6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbd6ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbd6ffd640 10 monclient: dump: 2024-01-24T23:23:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.175 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.175 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:16.175 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:16.175 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:16.175 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:16.175 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:16.175 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:16.175 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:16.175 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:16.175 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:16.175 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:16.175 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:16.175 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:16.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3812901397 0 0) 0x7fbbc4009e10 con 0x7fbbc40090d0 2024-01-24T23:23:16.175 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 --> v1:172.21.15.132:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbbc400c310 con 0x7fbbc40090d0 2024-01-24T23:23:16.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 <== mon.7 v1:172.21.15.132:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 3466040875) 0x7fbbc8007b00 con 0x7fbbc40090d0 2024-01-24T23:23:16.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbd6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:23:16.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbd6ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:16.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 >> v1:172.21.15.132:6792/0 conn(0x7fbbc40090d0 legacy=0x7fbbc4009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbd6ffd640 1 -- 172.21.15.115:0/2465844462 >> v1:172.21.15.132:6792/0 conn(0x7fbbc40090d0 legacy=0x7fbbc4009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:16.176 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:23:16.176 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:16.176 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:16.176 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:23:16.176 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 16, 2024-01-24T23:23:16.177 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:23:16.177 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:23:16.177 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:23:16.177 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:23:16.177 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:23:16.177 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:23:16.177 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:23:16.177 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:23:16.177 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:23:16.177 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:23:16.177 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:16.177 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 1, 2024-01-24T23:23:16.177 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:16.177 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:23:16.177 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:23:16.178 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:16.178 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:16.178 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:16.178 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:16.178 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:16.178 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:16.178 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:16.178 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:16.178 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:16.178 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:16.178 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:16.178 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:16.178 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:23:16.178 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:23:16.178 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:16.179 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:16.179 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:16.179 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:16.179 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:16.179 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:16.179 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:16.179 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:16.179 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:16.179 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:16.179 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:16.179 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:16.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.179 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:23:16.179 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:23:16.180 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:23:16.180 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:23:16.180 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:23:16.180 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:23:16.180 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:16.180 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:16.180 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:23:16.180 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:23:16.180 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:23:16.180 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:23:16.180 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:23:16.180 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:23:16.180 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:23:16.180 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:16.180 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:23:16.181 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:16.181 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:16.181 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:16.181 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:16.181 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:16.181 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:16.181 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:16.181 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:16.181 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:16.181 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:16.181 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:16.181 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:16.181 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:23:16.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.182 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:23:16.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.182 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:16.182 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:16.182 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:16.182 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:16.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.182 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:16.182 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:23:16.182 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:16.182 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.182 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:16.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.182 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:23:16.183 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:23:16.183 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:16.183 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:16.183 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:16.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.183 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:16.183 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:16.183 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:16.183 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:16.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.183 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:16.183 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:23:16.183 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:16.183 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.183 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:16.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.184 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:23:16.184 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:23:16.184 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:16.184 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:16.184 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:16.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.184 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:16.184 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:16.184 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:16.184 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:16.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.185 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:16.185 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:23:16.185 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:16.185 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.185 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:16.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.185 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:23:16.185 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:23:16.185 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:16.185 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:16.185 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:16.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.185 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:16.185 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:16.186 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:16.186 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:16.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.186 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:16.186 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:23:16.186 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:16.186 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.186 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:16.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.186 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:23:16.186 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:23:16.186 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:16.186 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:16.186 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:16.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.187 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:16.187 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:16.187 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:16.187 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:16.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.187 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:16.187 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:23:16.187 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:16.187 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.187 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:16.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.187 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:23:16.187 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:23:16.187 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:16.187 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:16.188 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:16.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.188 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:16.188 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:16.188 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:16.188 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:16.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.188 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:16.188 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:23:16.188 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:16.188 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.188 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:16.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.188 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:23:16.189 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:23:16.189 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:16.189 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:16.189 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:16.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.189 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:16.189 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:16.189 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:16.189 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:16.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.189 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:16.189 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:23:16.189 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:16.189 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.189 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:16.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.190 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:23:16.190 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:23:16.190 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:16.190 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:16.190 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:16.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.190 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:16.190 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:16.190 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:16.190 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:16.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.190 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:16.190 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:23:16.191 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:16.191 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.191 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:16.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.191 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:23:16.191 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:23:16.191 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:16.191 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:16.191 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:16.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.191 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:16.191 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:16.191 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:16.191 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:16.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.192 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:16.192 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:23:16.192 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:16.192 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.192 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:16.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.192 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:23:16.192 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:23:16.192 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:16.192 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:16.192 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:16.192 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.192 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:16.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.193 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:23:16.193 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:23:16.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.193 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:16.193 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:16.193 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:16.193 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.193 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:16.193 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:23:16.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.193 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:16.193 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:16.193 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 3 2024-01-24T23:23:16.193 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.193 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:16.194 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:23:16.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:16.194 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:16.194 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:16.194 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 3 2024-01-24T23:23:16.194 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:16.194 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:16.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:16.194 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:23:16.194 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:23:16.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbe4d2e640 1 -- 172.21.15.115:0/2465844462 >> v1:172.21.15.132:6812/1835401922 conn(0x7fbbc0037bc0 legacy=0x7fbbc003a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbe4d2e640 10 monclient: shutdown 2024-01-24T23:23:16.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbe4d2e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:16.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbe4d2e640 1 -- 172.21.15.115:0/2465844462 >> v1:172.21.15.132:6791/0 conn(0x7fbbe0134430 legacy=0x7fbbe01d3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbded9d640 1 -- 172.21.15.115:0/2465844462 reap_dead start 2024-01-24T23:23:16.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbded9d640 1 -- 172.21.15.115:0/2465844462 reap_dead start 2024-01-24T23:23:16.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbe4d2e640 1 -- 172.21.15.115:0/2465844462 shutdown_connections 2024-01-24T23:23:16.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbe4d2e640 1 -- 172.21.15.115:0/2465844462 >> 172.21.15.115:0/2465844462 conn(0x7fbbe00a9910 msgr2=0x7fbbe013fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:16.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbe4d2e640 1 -- 172.21.15.115:0/2465844462 shutdown_connections 2024-01-24T23:23:16.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.156+0000 7fbbe4d2e640 1 -- 172.21.15.115:0/2465844462 wait complete. 2024-01-24T23:23:16.196 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:23:16.196 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-01-24T23:23:16.197 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-01-24T23:23:16.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d29225640 10 monclient: get_monmap_and_config 2024-01-24T23:23:16.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d29225640 10 monclient: build_initial_monmap 2024-01-24T23:23:16.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d29225640 10 monclient: monmap: 2024-01-24T23:23:16.303 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:16.303 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:16.303 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:16.303484+0000 2024-01-24T23:23:16.303 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:16.303484+0000 2024-01-24T23:23:16.303 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:16.303 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:16.303 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:16.303 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:16.303 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:16.303 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:16.304 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:16.304 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:16.304 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:16.304 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:16.304 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:16.304 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:16.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d29225640 1 Processor -- start 2024-01-24T23:23:16.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d29225640 1 -- start start 2024-01-24T23:23:16.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d29225640 10 monclient: init 2024-01-24T23:23:16.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d29225640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:16.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d29225640 10 monclient: _add_conns ranks=[8,6,0,4,3,1,5,7,2] 2024-01-24T23:23:16.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d29225640 10 monclient(hunting): picked mon.noname-e con 0x7f1d24133bf0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:16.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d29225640 10 monclient(hunting): picked mon.noname-d con 0x7f1d24133fd0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:16.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d29225640 10 monclient(hunting): picked mon.noname-a con 0x7f1d241343b0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:16.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d29225640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d2404ed70 con 0x7f1d241343b0 2024-01-24T23:23:16.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d29225640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d240a4b60 con 0x7f1d24133fd0 2024-01-24T23:23:16.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d29225640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d2409ae50 con 0x7f1d24133bf0 2024-01-24T23:23:16.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d29225640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:16.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d29225640 10 monclient(hunting): authenticate will time out at 1075.226779s 2024-01-24T23:23:16.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d22d9d640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f1d24133bf0 0x7f1d2409bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:52942/0 (socket says 172.21.15.115:52942) 2024-01-24T23:23:16.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d1a59c640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f1d24133fd0 0x7f1d2413bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:34188/0 (socket says 172.21.15.115:34188) 2024-01-24T23:23:16.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d22d9d640 1 -- 172.21.15.115:0/3904370755 learned_addr learned my addr 172.21.15.115:0/3904370755 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:16.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2359e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f1d241343b0 0x7f1d2413f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:60948/0 (socket says 172.21.15.115:60948) 2024-01-24T23:23:16.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 1 -- 172.21.15.115:0/3904370755 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1223394035 0 0) 0x7f1d240a4b60 con 0x7f1d24133fd0 2024-01-24T23:23:16.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 10 monclient(hunting): _negotiate 2024-01-24T23:23:16.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:16.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:16.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 10 monclient(hunting): my global_id is 5010 2024-01-24T23:23:16.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 1 -- 172.21.15.115:0/3904370755 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d10003530 con 0x7f1d24133fd0 2024-01-24T23:23:16.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 1 -- 172.21.15.115:0/3904370755 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2771471786 0 0) 0x7f1d2409ae50 con 0x7f1d24133bf0 2024-01-24T23:23:16.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 10 monclient(hunting): _negotiate 2024-01-24T23:23:16.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:16.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:16.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 10 monclient(hunting): my global_id is 84185 2024-01-24T23:23:16.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 1 -- 172.21.15.115:0/3904370755 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d240a4b60 con 0x7f1d24133bf0 2024-01-24T23:23:16.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 1 -- 172.21.15.115:0/3904370755 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2244642010 0 0) 0x7f1d2404ed70 con 0x7f1d241343b0 2024-01-24T23:23:16.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 10 monclient(hunting): _negotiate 2024-01-24T23:23:16.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:16.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:16.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 10 monclient(hunting): my global_id is 4959 2024-01-24T23:23:16.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 1 -- 172.21.15.115:0/3904370755 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d2409ae50 con 0x7f1d241343b0 2024-01-24T23:23:16.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 1 -- 172.21.15.115:0/3904370755 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2793290127 0 0) 0x7f1d240a4b60 con 0x7f1d24133bf0 2024-01-24T23:23:16.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 1 -- 172.21.15.115:0/3904370755 >> v1:172.21.15.115:6792/0 conn(0x7f1d24133fd0 legacy=0x7f1d2413bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 1 -- 172.21.15.115:0/3904370755 >> v1:172.21.15.115:6789/0 conn(0x7f1d241343b0 legacy=0x7f1d2413f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 10 monclient: _finish_hunting 0 2024-01-24T23:23:16.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 1 monclient: found mon.noname-e 2024-01-24T23:23:16.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:16.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:23:16.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 1 -- 172.21.15.115:0/3904370755 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2409b130 con 0x7f1d24133bf0 2024-01-24T23:23:16.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 10 monclient: _finish_auth 0 2024-01-24T23:23:16.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 10 monclient: _check_auth_tickets 2024-01-24T23:23:16.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:16.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d29225640 5 monclient: authenticate success, global_id 84185 2024-01-24T23:23:16.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 1 -- 172.21.15.115:0/3904370755 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1d140017a0 con 0x7f1d24133bf0 2024-01-24T23:23:16.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:16.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:23:16.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 10 monclient: dump: 2024-01-24T23:23:16.310 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:16.310 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:16.310 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.311 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.311 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:16.311 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:16.311 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:16.311 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:16.311 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:16.311 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:16.311 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:16.311 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:16.311 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:16.311 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:16.311 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:16.311 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:16.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d29225640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:16.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 1 -- 172.21.15.115:0/3904370755 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1d14001ac0 con 0x7f1d24133bf0 2024-01-24T23:23:16.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:16.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d29225640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:16.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 1 -- 172.21.15.115:0/3904370755 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1d14004bf0 con 0x7f1d24133bf0 2024-01-24T23:23:16.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d29225640 4 set_mon_vals no callback set 2024-01-24T23:23:16.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:16.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:16.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.300+0000 7f1d2259c640 10 monclient: dump: 2024-01-24T23:23:16.312 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:16.312 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:16.312 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.312 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.312 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:16.312 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:16.312 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:16.312 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:16.312 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:16.313 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:16.313 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:16.313 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:16.313 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:16.313 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:16.313 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:16.313 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:16.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 10 monclient: shutdown 2024-01-24T23:23:16.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:16.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 1 -- 172.21.15.115:0/3904370755 >> v1:172.21.15.115:6793/0 conn(0x7f1d24133bf0 legacy=0x7f1d2409bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 1 -- 172.21.15.115:0/3904370755 shutdown_connections 2024-01-24T23:23:16.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 1 -- 172.21.15.115:0/3904370755 >> 172.21.15.115:0/3904370755 conn(0x7f1d240662f0 msgr2=0x7f1d24066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:16.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 1 -- 172.21.15.115:0/3904370755 shutdown_connections 2024-01-24T23:23:16.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 1 -- 172.21.15.115:0/3904370755 wait complete. 2024-01-24T23:23:16.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 10 monclient: build_initial_monmap 2024-01-24T23:23:16.313 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 10 monclient: monmap: 2024-01-24T23:23:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:16.314 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 1 Processor -- start 2024-01-24T23:23:16.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 1 -- start start 2024-01-24T23:23:16.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 10 monclient: init 2024-01-24T23:23:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 10 monclient: _add_conns ranks=[7,1,6,8,5,2,0,3,4] 2024-01-24T23:23:16.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 10 monclient(hunting): picked mon.noname-i con 0x7f1d24133bf0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:16.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 10 monclient(hunting): picked mon.noname-f con 0x7f1d24133fd0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:16.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 10 monclient(hunting): picked mon.noname-d con 0x7f1d241343b0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:16.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d240a5990 con 0x7f1d24133fd0 2024-01-24T23:23:16.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d2409c880 con 0x7f1d241343b0 2024-01-24T23:23:16.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d24133700 con 0x7f1d24133bf0 2024-01-24T23:23:16.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:16.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 10 monclient(hunting): authenticate will time out at 1075.229673s 2024-01-24T23:23:16.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d2359e640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f1d241343b0 0x7f1d241d3640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:34190/0 (socket says 172.21.15.115:34190) 2024-01-24T23:23:16.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d2359e640 1 -- 172.21.15.115:0/2123821807 learned_addr learned my addr 172.21.15.115:0/2123821807 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:16.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 1 -- 172.21.15.115:0/2123821807 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 170289330 0 0) 0x7f1d1c001490 con 0x7f1d241343b0 2024-01-24T23:23:16.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:16.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:16.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:16.318 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 10 monclient(hunting): my global_id is 5019 2024-01-24T23:23:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 1 -- 172.21.15.115:0/2123821807 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d04003530 con 0x7f1d241343b0 2024-01-24T23:23:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 1 -- 172.21.15.115:0/2123821807 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1322814168 0 0) 0x7f1d04003530 con 0x7f1d241343b0 2024-01-24T23:23:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 1 -- 172.21.15.115:0/2123821807 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1d1c001490 con 0x7f1d241343b0 2024-01-24T23:23:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 1 -- 172.21.15.115:0/2123821807 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1d1c001e60 con 0x7f1d241343b0 2024-01-24T23:23:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:23:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 10 monclient(hunting): dump: 2024-01-24T23:23:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:16.320 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 1 -- 172.21.15.115:0/2123821807 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1107148395 0 0) 0x7f1d1c001490 con 0x7f1d241343b0 2024-01-24T23:23:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 1 -- 172.21.15.115:0/2123821807 >> v1:172.21.15.132:6792/0 conn(0x7f1d24133bf0 legacy=0x7f1d24132ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 1 -- 172.21.15.115:0/2123821807 >> v1:172.21.15.132:6789/0 conn(0x7f1d24133fd0 legacy=0x7f1d241cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:23:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 1 monclient: found mon.d 2024-01-24T23:23:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:23:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 1 -- 172.21.15.115:0/2123821807 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d240a24c0 con 0x7f1d241343b0 2024-01-24T23:23:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 10 monclient: _finish_auth 0 2024-01-24T23:23:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:23:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 5 monclient: authenticate success, global_id 5019 2024-01-24T23:23:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 10 monclient: _renew_subs 2024-01-24T23:23:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:23:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 1 -- 172.21.15.115:0/2123821807 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d24120130 con 0x7f1d241343b0 2024-01-24T23:23:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 10 monclient: _renew_subs 2024-01-24T23:23:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:23:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 1 -- 172.21.15.115:0/2123821807 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d241cb9b0 con 0x7f1d241343b0 2024-01-24T23:23:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 1 -- 172.21.15.115:0/2123821807 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1d1c004e20 con 0x7f1d241343b0 2024-01-24T23:23:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 1 -- 172.21.15.115:0/2123821807 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1d1c0052d0 con 0x7f1d241343b0 2024-01-24T23:23:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d21d9b640 4 set_mon_vals no callback set 2024-01-24T23:23:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 10 monclient: dump: 2024-01-24T23:23:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:16.323 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:23:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d29225640 1 -- 172.21.15.115:0/2123821807 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d08005090 con 0x7f1d241343b0 2024-01-24T23:23:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 1 -- 172.21.15.115:0/2123821807 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f1d1c00fd50 con 0x7f1d241343b0 2024-01-24T23:23:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d1bfff640 1 -- 172.21.15.115:0/2123821807 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(48..48 src has 1..48) v4 ==== 3994+0+0 (unknown 1916735388 0 0) 0x7f1d1c043ba0 con 0x7f1d241343b0 2024-01-24T23:23:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.304+0000 7f1d22d9d640 10 monclient: get_auth_request con 0x7f1d04037020 auth_method 0 2024-01-24T23:23:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.308+0000 7f1d1bfff640 1 -- 172.21.15.115:0/2123821807 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f1d1c014b00 con 0x7f1d241343b0 2024-01-24T23:23:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.308+0000 7f1d1bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.308+0000 7f1d1bfff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:16.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.392+0000 7fc25e7fc640 1 -- 172.21.15.132:0/4112692320 <== mon.1 v1:172.21.15.132:6789/0 10 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}]=0 v49) v1 ==== 151+0+0 (unknown 79470226 0 0) 0x7fc264013130 con 0x7fc26809b810 2024-01-24T23:23:16.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.392+0000 7fc25e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}] 2024-01-24T23:23:16.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.392+0000 7fc25e7fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:16.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.492+0000 7fc270219640 10 start_mon_command cmd=[{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}] 2024-01-24T23:23:16.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.492+0000 7fc270219640 10 monclient: _send_command 3 [{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}] 2024-01-24T23:23:16.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.492+0000 7fc270219640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:16.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.492+0000 7fc270219640 1 -- 172.21.15.132:0/4112692320 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]} v 0) v1 -- 0x7fc2500055e0 con 0x7fc26809b810 2024-01-24T23:23:16.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.496+0000 7fc25e7fc640 1 -- 172.21.15.132:0/4112692320 <== mon.1 v1:172.21.15.132:6789/0 11 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}]=0 v49) v1 ==== 151+0+0 (unknown 79470226 0 0) 0x7fc2640148b0 con 0x7fc26809b810 2024-01-24T23:23:16.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.496+0000 7fc25e7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}] 2024-01-24T23:23:16.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.496+0000 7fc25e7fc640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:23:16.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.496+0000 7fc270219640 1 -- 172.21.15.132:0/4112692320 >> v1:172.21.15.132:6812/1835401922 conn(0x7fc24c037110 legacy=0x7fc24c0395d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.496+0000 7fc270219640 10 monclient: shutdown 2024-01-24T23:23:16.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.496+0000 7fc270219640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:16.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.496+0000 7fc270219640 1 -- 172.21.15.132:0/4112692320 >> v1:172.21.15.132:6789/0 conn(0x7fc26809b810 legacy=0x7fc2681cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.496+0000 7fc270219640 1 -- 172.21.15.132:0/4112692320 shutdown_connections 2024-01-24T23:23:16.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.496+0000 7fc270219640 1 -- 172.21.15.132:0/4112692320 >> 172.21.15.132:0/4112692320 conn(0x7fc2680a1bc0 msgr2=0x7fc26809a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:16.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.496+0000 7fc270219640 1 -- 172.21.15.132:0/4112692320 shutdown_connections 2024-01-24T23:23:16.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.496+0000 7fc270219640 1 -- 172.21.15.132:0/4112692320 wait complete. 2024-01-24T23:23:16.512 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool create ec-foo 2 erasure ec-foo-ssd 2024-01-24T23:23:16.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.528+0000 7f1d29225640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-01-24T23:23:16.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.528+0000 7f1d29225640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-01-24T23:23:16.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.528+0000 7f1d29225640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:23:16.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.528+0000 7f1d29225640 1 -- 172.21.15.115:0/2123821807 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f1d08005b50 con 0x7f1d241343b0 2024-01-24T23:23:16.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f1d1bfff640 1 -- 172.21.15.115:0/2123821807 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (unknown 246732539 0 0) 0x7f1d1c0131a0 con 0x7f1d241343b0 2024-01-24T23:23:16.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f1d1bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-01-24T23:23:16.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f1d1bfff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:16.537 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.537 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=12,kv=9,logm=62} crc {auth=650704035,config=1618278320,health=1267016307,kv=420472996,logm=3542408518}) 2024-01-24T23:23:16.537 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=15,config=2,health=12,kv=8,logm=63} crc {auth=1505300948,config=1618278320,health=1267016307,kv=2520799400,logm=3748472289}) 2024-01-24T23:23:16.537 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.537 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=12,kv=9,logm=62} crc {auth=650704035,config=1618278320,health=1267016307,kv=420472996,logm=3542408518}) 2024-01-24T23:23:16.537 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=15,config=2,health=11,kv=8,logm=64} crc {auth=650704035,config=1618278320,health=3349787962,kv=3940250659,logm=1448234741}) 2024-01-24T23:23:16.538 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.538 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=12,kv=9,logm=62} crc {auth=650704035,config=1618278320,health=1267016307,kv=420472996,logm=3542408518}) 2024-01-24T23:23:16.538 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=15,config=2,health=11,kv=8,logm=64} crc {auth=650704035,config=1618278320,health=2725584139,kv=429879371,logm=1689444781}) 2024-01-24T23:23:16.538 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.538 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=12,kv=9,logm=62} crc {auth=650704035,config=1618278320,health=1267016307,kv=420472996,logm=3542408518}) 2024-01-24T23:23:16.538 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=15,config=1,health=12,kv=9,logm=63} crc {auth=650704035,config=2457401934,health=1267016307,kv=3447624886,logm=1630067689}) 2024-01-24T23:23:16.538 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.538 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=12,kv=9,logm=62} crc {auth=650704035,config=1618278320,health=1267016307,kv=420472996,logm=3542408518}) 2024-01-24T23:23:16.538 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=13,config=2,health=12,kv=9,logm=64} crc {auth=2953773880,config=1618278320,health=1267016307,kv=3447624886,logm=4029355717}) 2024-01-24T23:23:16.538 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.538 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=12,kv=9,logm=62} crc {auth=650704035,config=1618278320,health=1267016307,kv=420472996,logm=3542408518}) 2024-01-24T23:23:16.538 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=14,config=2,health=12,kv=9,logm=63} crc {auth=3339949454,config=1618278320,health=1267016307,kv=3447624886,logm=4253783675}) 2024-01-24T23:23:16.538 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.538 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=12,kv=9,logm=62} crc {auth=650704035,config=1618278320,health=1267016307,kv=420472996,logm=3542408518}) 2024-01-24T23:23:16.538 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=12,config=2,health=11,kv=9,logm=66} crc {auth=598195703,config=1618278320,health=3744717254,kv=2270755600,logm=2644932010}) 2024-01-24T23:23:16.538 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.538 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=12,kv=9,logm=62} crc {auth=650704035,config=1618278320,health=1267016307,kv=420472996,logm=3542408518}) 2024-01-24T23:23:16.539 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=13,config=2,health=10,kv=9,logm=66} crc {auth=3977992300,config=1618278320,health=1424367351,kv=3733386309,logm=4108462621}) 2024-01-24T23:23:16.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f1d29225640 1 -- 172.21.15.115:0/2123821807 >> v1:172.21.15.132:6812/1835401922 conn(0x7f1d04037020 legacy=0x7f1d040394e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f1d29225640 10 monclient: shutdown 2024-01-24T23:23:16.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f1d29225640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:16.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f1d29225640 1 -- 172.21.15.115:0/2123821807 >> v1:172.21.15.115:6792/0 conn(0x7f1d241343b0 legacy=0x7f1d241d3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f1d29225640 1 -- 172.21.15.115:0/2123821807 shutdown_connections 2024-01-24T23:23:16.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f1d29225640 1 -- 172.21.15.115:0/2123821807 >> 172.21.15.115:0/2123821807 conn(0x7f1d240662f0 msgr2=0x7f1d240a7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:16.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f1d29225640 1 -- 172.21.15.115:0/2123821807 shutdown_connections 2024-01-24T23:23:16.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:16.532+0000 7f1d29225640 1 -- 172.21.15.115:0/2123821807 wait complete. 2024-01-24T23:23:16.542 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.536+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.542 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.536+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=876825304}) 2024-01-24T23:23:16.542 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.536+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3875094137}) 2024-01-24T23:23:16.542 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.536+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.542 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.536+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=876825304}) 2024-01-24T23:23:16.542 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.536+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3644495480}) 2024-01-24T23:23:16.542 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.536+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.542 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.536+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=876825304}) 2024-01-24T23:23:16.542 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.536+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2393015697}) 2024-01-24T23:23:16.543 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.536+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.543 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.536+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=876825304}) 2024-01-24T23:23:16.543 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.536+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=167281121}) 2024-01-24T23:23:16.543 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.536+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.543 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.536+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=876825304}) 2024-01-24T23:23:16.543 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.536+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2939414889}) 2024-01-24T23:23:16.543 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.536+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.543 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.536+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=876825304}) 2024-01-24T23:23:16.543 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.536+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=871078759}) 2024-01-24T23:23:16.543 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.536+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.543 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.536+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=876825304}) 2024-01-24T23:23:16.543 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.536+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2745346020}) 2024-01-24T23:23:16.543 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.536+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.543 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.536+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=876825304}) 2024-01-24T23:23:16.544 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.536+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1042582129}) 2024-01-24T23:23:16.548 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.548 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2787557931}) 2024-01-24T23:23:16.548 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=96145519}) 2024-01-24T23:23:16.548 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.548 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2787557931}) 2024-01-24T23:23:16.548 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3483722172}) 2024-01-24T23:23:16.548 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.548 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2787557931}) 2024-01-24T23:23:16.548 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2884552002}) 2024-01-24T23:23:16.548 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.548 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2787557931}) 2024-01-24T23:23:16.549 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=247041802}) 2024-01-24T23:23:16.549 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.549 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2787557931}) 2024-01-24T23:23:16.549 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2507600955}) 2024-01-24T23:23:16.549 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.549 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2787557931}) 2024-01-24T23:23:16.549 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=294197775}) 2024-01-24T23:23:16.549 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.549 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2787557931}) 2024-01-24T23:23:16.549 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2058494062}) 2024-01-24T23:23:16.549 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.549 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2787557931}) 2024-01-24T23:23:16.549 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=128997127}) 2024-01-24T23:23:16.549 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.549 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3350691621}) 2024-01-24T23:23:16.549 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1792347315}) 2024-01-24T23:23:16.550 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.550 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3350691621}) 2024-01-24T23:23:16.550 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.540+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=719012995}) 2024-01-24T23:23:16.550 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.550 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3350691621}) 2024-01-24T23:23:16.550 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1010799129}) 2024-01-24T23:23:16.550 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.550 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3350691621}) 2024-01-24T23:23:16.550 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=356385056}) 2024-01-24T23:23:16.550 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.550 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3350691621}) 2024-01-24T23:23:16.550 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=618537181}) 2024-01-24T23:23:16.550 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.550 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3350691621}) 2024-01-24T23:23:16.551 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=532760960}) 2024-01-24T23:23:16.551 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.551 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3350691621}) 2024-01-24T23:23:16.551 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1270607481}) 2024-01-24T23:23:16.551 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.551 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3350691621}) 2024-01-24T23:23:16.551 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2339186396}) 2024-01-24T23:23:16.555 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.555 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=121078462}) 2024-01-24T23:23:16.555 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4108250297}) 2024-01-24T23:23:16.555 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.555 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=121078462}) 2024-01-24T23:23:16.555 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2272006783}) 2024-01-24T23:23:16.555 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.556 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=121078462}) 2024-01-24T23:23:16.556 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1101149982}) 2024-01-24T23:23:16.556 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.556 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=121078462}) 2024-01-24T23:23:16.556 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3726358765}) 2024-01-24T23:23:16.556 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.556 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=121078462}) 2024-01-24T23:23:16.556 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3777589081}) 2024-01-24T23:23:16.556 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.556 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=121078462}) 2024-01-24T23:23:16.556 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1071168188}) 2024-01-24T23:23:16.556 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.556 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=121078462}) 2024-01-24T23:23:16.556 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1402285008}) 2024-01-24T23:23:16.556 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.557 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=121078462}) 2024-01-24T23:23:16.557 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.544+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3398269292}) 2024-01-24T23:23:16.557 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.548+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.557 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.548+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3891345682}) 2024-01-24T23:23:16.557 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.548+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3761046135}) 2024-01-24T23:23:16.557 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.548+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.557 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.548+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3891345682}) 2024-01-24T23:23:16.557 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.548+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3375366846}) 2024-01-24T23:23:16.557 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.548+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.557 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.548+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3891345682}) 2024-01-24T23:23:16.557 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.548+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=364107278}) 2024-01-24T23:23:16.557 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.548+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.557 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.548+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3891345682}) 2024-01-24T23:23:16.557 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.548+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1048549242}) 2024-01-24T23:23:16.557 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.548+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.558 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.548+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3891345682}) 2024-01-24T23:23:16.558 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.548+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1094775575}) 2024-01-24T23:23:16.558 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.548+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.558 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.548+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3891345682}) 2024-01-24T23:23:16.558 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.548+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=101883928}) 2024-01-24T23:23:16.558 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.548+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.558 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.548+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3891345682}) 2024-01-24T23:23:16.558 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.548+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=97537418}) 2024-01-24T23:23:16.558 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.548+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.558 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.548+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3891345682}) 2024-01-24T23:23:16.558 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.548+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3813262842}) 2024-01-24T23:23:16.559 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-01-24T23:23:16.566 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.552+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.566 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.552+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1437953495}) 2024-01-24T23:23:16.566 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.552+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2282119214}) 2024-01-24T23:23:16.566 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.552+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.566 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.552+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1437953495}) 2024-01-24T23:23:16.566 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.552+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4168422320}) 2024-01-24T23:23:16.566 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.552+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.567 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.552+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1437953495}) 2024-01-24T23:23:16.567 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.552+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1551873965}) 2024-01-24T23:23:16.567 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.552+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.567 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.552+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1437953495}) 2024-01-24T23:23:16.567 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.552+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3644802069}) 2024-01-24T23:23:16.567 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.552+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.567 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.552+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1437953495}) 2024-01-24T23:23:16.567 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.552+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=402356315}) 2024-01-24T23:23:16.567 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.552+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.567 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.552+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1437953495}) 2024-01-24T23:23:16.567 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.552+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1790685505}) 2024-01-24T23:23:16.567 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.552+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.567 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.552+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1437953495}) 2024-01-24T23:23:16.567 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.552+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3965977404}) 2024-01-24T23:23:16.568 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.552+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.568 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.552+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1437953495}) 2024-01-24T23:23:16.568 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.552+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4207078378}) 2024-01-24T23:23:16.568 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.556+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.568 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.556+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=71,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=7} crc {logm=3779572961,mdsmap=3984425268,mgr=3539714768,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2884238285,mon_config_key=3302197801}) 2024-01-24T23:23:16.568 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.556+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=73,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5} crc {logm=1730878950,mdsmap=3984425268,mgr=1093705807,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2884238285,mon_config_key=225352958}) 2024-01-24T23:23:16.568 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.556+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.568 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.556+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=71,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=7} crc {logm=3779572961,mdsmap=3984425268,mgr=3539714768,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2884238285,mon_config_key=3302197801}) 2024-01-24T23:23:16.568 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.556+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=71,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=6} crc {logm=888971851,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2865678135,mon_config_key=3901114617}) 2024-01-24T23:23:16.568 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.556+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.568 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.556+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=71,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=7} crc {logm=3779572961,mdsmap=3984425268,mgr=3539714768,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2884238285,mon_config_key=3302197801}) 2024-01-24T23:23:16.568 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.556+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=75,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=2} crc {logm=2520351337,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2884238285,mon_config_key=631066491}) 2024-01-24T23:23:16.568 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.556+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.569 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.556+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=71,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=7} crc {logm=3779572961,mdsmap=3984425268,mgr=3539714768,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2884238285,mon_config_key=3302197801}) 2024-01-24T23:23:16.569 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.556+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=68,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=8,monmap=2} crc {logm=2805673136,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=877616292,mon_config_key=3218760479,monmap=1293597301}) 2024-01-24T23:23:16.569 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.556+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.569 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.556+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=71,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=7} crc {logm=3779572961,mdsmap=3984425268,mgr=3539714768,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2884238285,mon_config_key=3302197801}) 2024-01-24T23:23:16.569 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.556+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=72,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=6} crc {logm=1243375446,mdsmap=3984425268,mgr=1117054149,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2884238285,mon_config_key=3901114617}) 2024-01-24T23:23:16.569 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.556+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.569 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.556+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=71,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=7} crc {logm=3779572961,mdsmap=3984425268,mgr=3539714768,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2884238285,mon_config_key=3302197801}) 2024-01-24T23:23:16.569 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.556+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=68,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=8,monmap=2} crc {logm=1046957070,mdsmap=2441318933,mgr=107005525,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2884238285,mon_config_key=4121867429,monmap=1293597301}) 2024-01-24T23:23:16.569 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.556+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.569 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.556+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=71,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=7} crc {logm=3779572961,mdsmap=3984425268,mgr=3539714768,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2884238285,mon_config_key=3302197801}) 2024-01-24T23:23:16.569 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.556+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=74,mdsmap=3,mgr=8,mgrstat=9,mon_config_key=6} crc {logm=1218383614,mdsmap=3984425268,mgr=1221499742,mgrstat=2544421559,mon_config_key=557331028}) 2024-01-24T23:23:16.569 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.556+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.569 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.556+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=71,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=7} crc {logm=3779572961,mdsmap=3984425268,mgr=3539714768,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2884238285,mon_config_key=3302197801}) 2024-01-24T23:23:16.569 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.556+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=73,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5} crc {logm=1351288975,mdsmap=2441318933,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2884238285,mon_config_key=225352958}) 2024-01-24T23:23:16.570 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.560+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.570 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.560+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=84} crc {monmap=701157023,osd_metadata=3073132558,osd_pg_creating=64926272,osdmap=3612962632}) 2024-01-24T23:23:16.570 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.560+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=87} crc {monmap=701157023,osd_metadata=3193413316,osd_pg_creating=64926272,osdmap=289766353}) 2024-01-24T23:23:16.570 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.560+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.570 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.560+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=84} crc {monmap=701157023,osd_metadata=3073132558,osd_pg_creating=64926272,osdmap=3612962632}) 2024-01-24T23:23:16.570 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.560+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {monmap=3,osd_metadata=6,osdmap=87} crc {monmap=701157023,osd_metadata=3193413316,osdmap=3586121221}) 2024-01-24T23:23:16.570 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.560+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.570 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.560+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=84} crc {monmap=701157023,osd_metadata=3073132558,osd_pg_creating=64926272,osdmap=3612962632}) 2024-01-24T23:23:16.570 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.560+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=88} crc {monmap=701157023,osd_metadata=3193413316,osd_pg_creating=64926272,osdmap=2988345160}) 2024-01-24T23:23:16.570 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.560+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.570 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.560+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=84} crc {monmap=701157023,osd_metadata=3073132558,osd_pg_creating=64926272,osdmap=3612962632}) 2024-01-24T23:23:16.570 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.560+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=88} crc {monmap=701157023,osd_metadata=3193413316,osd_pg_creating=64926272,osdmap=1476958299}) 2024-01-24T23:23:16.570 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.560+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.570 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.560+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=84} crc {monmap=701157023,osd_metadata=3073132558,osd_pg_creating=64926272,osdmap=3612962632}) 2024-01-24T23:23:16.571 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.560+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=90} crc {monmap=701157023,osd_metadata=3193413316,osd_pg_creating=64926272,osdmap=3925376243}) 2024-01-24T23:23:16.571 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.560+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.571 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.560+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=84} crc {monmap=701157023,osd_metadata=3073132558,osd_pg_creating=64926272,osdmap=3612962632}) 2024-01-24T23:23:16.571 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.560+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=87} crc {monmap=701157023,osd_metadata=3073132558,osd_pg_creating=64926272,osdmap=1362429986}) 2024-01-24T23:23:16.571 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.560+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.571 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.560+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=84} crc {monmap=701157023,osd_metadata=3073132558,osd_pg_creating=64926272,osdmap=3612962632}) 2024-01-24T23:23:16.571 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.560+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {monmap=2,osd_metadata=4,osd_pg_creating=1,osdmap=85} crc {monmap=1293597301,osd_metadata=3582388687,osd_pg_creating=64926272,osdmap=1848778304}) 2024-01-24T23:23:16.571 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.560+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:23:16.571 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.560+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=84} crc {monmap=701157023,osd_metadata=3073132558,osd_pg_creating=64926272,osdmap=3612962632}) 2024-01-24T23:23:16.571 INFO:tasks.ceph.mon.a.smithi115.stderr:2024-01-24T23:23:16.560+0000 7f7950e36640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {monmap=2,osd_metadata=6,osd_pg_creating=1,osdmap=87} crc {monmap=1293597301,osd_metadata=3193413316,osd_pg_creating=64926272,osdmap=890369905}) 2024-01-24T23:23:16.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.632+0000 7fe397ce1640 10 monclient: get_monmap_and_config 2024-01-24T23:23:16.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.632+0000 7fe397ce1640 10 monclient: build_initial_monmap 2024-01-24T23:23:16.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.632+0000 7fe397ce1640 10 monclient: monmap: 2024-01-24T23:23:16.635 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:16.635 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:16.635 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:16.634650+0000 2024-01-24T23:23:16.635 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:16.634650+0000 2024-01-24T23:23:16.635 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:16.635 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:16.635 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:16.635 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:16.635 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:16.635 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:16.635 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:16.635 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:16.636 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:16.636 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:16.636 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:16.636 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:16.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.632+0000 7fe397ce1640 1 Processor -- start 2024-01-24T23:23:16.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.632+0000 7fe397ce1640 1 -- start start 2024-01-24T23:23:16.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.632+0000 7fe397ce1640 10 monclient: init 2024-01-24T23:23:16.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.632+0000 7fe397ce1640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:16.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.632+0000 7fe397ce1640 10 monclient: _add_conns ranks=[0,8,7,6,1,4,3,2,5] 2024-01-24T23:23:16.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.632+0000 7fe397ce1640 10 monclient(hunting): picked mon.noname-a con 0x7fe390099510 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:16.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.632+0000 7fe397ce1640 10 monclient(hunting): picked mon.noname-e con 0x7fe39009fdd0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:16.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.632+0000 7fe397ce1640 10 monclient(hunting): picked mon.noname-i con 0x7fe3900a01f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:16.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.632+0000 7fe397ce1640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe390064700 con 0x7fe390099510 2024-01-24T23:23:16.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe39009ee20 con 0x7fe3900a01f0 2024-01-24T23:23:16.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe39009eb40 con 0x7fe39009fdd0 2024-01-24T23:23:16.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:16.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 10 monclient(hunting): authenticate will time out at 1083.960122s 2024-01-24T23:23:16.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe39627e640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fe3900a01f0 0x7fe39013c280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:40602/0 (socket says 172.21.15.132:40602) 2024-01-24T23:23:16.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe39627e640 1 -- 172.21.15.132:0/1429382883 learned_addr learned my addr 172.21.15.132:0/1429382883 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:16.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 1 -- 172.21.15.132:0/1429382883 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483151770 0 0) 0x7fe39009ee20 con 0x7fe3900a01f0 2024-01-24T23:23:16.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 10 monclient(hunting): _negotiate 2024-01-24T23:23:16.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:16.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:16.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 10 monclient(hunting): my global_id is 5020 2024-01-24T23:23:16.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 1 -- 172.21.15.132:0/1429382883 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe37c003530 con 0x7fe3900a01f0 2024-01-24T23:23:16.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 1 -- 172.21.15.132:0/1429382883 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 218542079 0 0) 0x7fe37c003530 con 0x7fe3900a01f0 2024-01-24T23:23:16.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 1 -- 172.21.15.132:0/1429382883 >> v1:172.21.15.115:6793/0 conn(0x7fe39009fdd0 legacy=0x7fe390138b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 1 -- 172.21.15.132:0/1429382883 >> v1:172.21.15.115:6789/0 conn(0x7fe390099510 legacy=0x7fe3900998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 10 monclient: _finish_hunting 0 2024-01-24T23:23:16.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 1 monclient: found mon.noname-i 2024-01-24T23:23:16.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:16.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:23:16.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 1 -- 172.21.15.132:0/1429382883 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3900a3990 con 0x7fe3900a01f0 2024-01-24T23:23:16.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 10 monclient: _finish_auth 0 2024-01-24T23:23:16.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 10 monclient: _check_auth_tickets 2024-01-24T23:23:16.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:16.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 1 -- 172.21.15.132:0/1429382883 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe388001b40 con 0x7fe3900a01f0 2024-01-24T23:23:16.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:16.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:23:16.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 10 monclient: dump: 2024-01-24T23:23:16.644 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:16.644 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:16.644 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.644 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.644 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:16.644 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:16.644 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:16.645 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:16.645 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:16.645 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:16.645 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:16.645 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:16.645 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:16.645 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:16.645 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:16.645 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:16.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 5 monclient: authenticate success, global_id 5020 2024-01-24T23:23:16.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:16.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 1 -- 172.21.15.132:0/1429382883 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe388001e60 con 0x7fe3900a01f0 2024-01-24T23:23:16.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:16.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 1 -- 172.21.15.132:0/1429382883 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe388004f70 con 0x7fe3900a01f0 2024-01-24T23:23:16.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:16.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:16.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe394a7b640 10 monclient: dump: 2024-01-24T23:23:16.646 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:16.646 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:16.646 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.646 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.646 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:16.646 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:16.646 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:16.646 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:16.646 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:16.646 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:16.646 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:16.646 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:16.646 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:16.646 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:16.647 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:16.647 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:16.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:16.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 4 set_mon_vals no callback set 2024-01-24T23:23:16.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 10 monclient: shutdown 2024-01-24T23:23:16.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:16.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 1 -- 172.21.15.132:0/1429382883 >> v1:172.21.15.132:6792/0 conn(0x7fe3900a01f0 legacy=0x7fe39013c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 1 -- 172.21.15.132:0/1429382883 shutdown_connections 2024-01-24T23:23:16.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 1 -- 172.21.15.132:0/1429382883 >> 172.21.15.132:0/1429382883 conn(0x7fe3900a2c40 msgr2=0x7fe3900a3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:16.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 1 -- 172.21.15.132:0/1429382883 shutdown_connections 2024-01-24T23:23:16.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 1 -- 172.21.15.132:0/1429382883 wait complete. 2024-01-24T23:23:16.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 10 monclient: build_initial_monmap 2024-01-24T23:23:16.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 10 monclient: monmap: 2024-01-24T23:23:16.647 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:16.648 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:16.648 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:16.648 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:16.648 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:16.648 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:16.648 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:16.648 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:16.648 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:16.648 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:16.648 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:16.648 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:16.648 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:16.648 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:16.648 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:16.648 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:16.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 1 Processor -- start 2024-01-24T23:23:16.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 1 -- start start 2024-01-24T23:23:16.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 10 monclient: init 2024-01-24T23:23:16.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:16.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 10 monclient: _add_conns ranks=[1,4,0,3,2,5,8,7,6] 2024-01-24T23:23:16.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 10 monclient(hunting): picked mon.noname-f con 0x7fe390099510 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:16.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 10 monclient(hunting): picked mon.noname-c con 0x7fe39009fdd0 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:16.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 10 monclient(hunting): picked mon.noname-a con 0x7fe3900a01f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:16.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe39009e010 con 0x7fe3900a01f0 2024-01-24T23:23:16.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe390070340 con 0x7fe390099510 2024-01-24T23:23:16.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe39009e540 con 0x7fe39009fdd0 2024-01-24T23:23:16.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:16.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 10 monclient(hunting): authenticate will time out at 1083.962772s 2024-01-24T23:23:16.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe395a7d640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fe390099510 0x7fe390071b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:40868/0 (socket says 172.21.15.132:40868) 2024-01-24T23:23:16.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe395a7d640 1 -- 172.21.15.132:0/3302017971 learned_addr learned my addr 172.21.15.132:0/3302017971 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:16.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 1 -- 172.21.15.132:0/3302017971 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 82133836 0 0) 0x7fe390070340 con 0x7fe390099510 2024-01-24T23:23:16.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:16.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:16.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:16.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 10 monclient(hunting): my global_id is 5077 2024-01-24T23:23:16.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 1 -- 172.21.15.132:0/3302017971 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe374003530 con 0x7fe390099510 2024-01-24T23:23:16.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 1 -- 172.21.15.132:0/3302017971 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4010591349 0 0) 0x7fe374003530 con 0x7fe390099510 2024-01-24T23:23:16.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 1 -- 172.21.15.132:0/3302017971 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe390070340 con 0x7fe390099510 2024-01-24T23:23:16.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 1 -- 172.21.15.132:0/3302017971 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe38c001ac0 con 0x7fe390099510 2024-01-24T23:23:16.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:16.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:23:16.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 10 monclient(hunting): dump: 2024-01-24T23:23:16.650 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:16.651 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:16.651 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.651 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.651 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:16.651 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:16.651 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:16.651 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:16.651 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:16.651 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:16.651 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:16.651 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:16.651 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:16.651 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:16.651 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:16.651 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:16.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:16.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 1 -- 172.21.15.132:0/3302017971 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3062509553 0 0) 0x7fe39009e010 con 0x7fe3900a01f0 2024-01-24T23:23:16.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:16.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:16.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:16.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 10 monclient(hunting): my global_id is 4977 2024-01-24T23:23:16.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 1 -- 172.21.15.132:0/3302017971 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe374003530 con 0x7fe3900a01f0 2024-01-24T23:23:16.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 1 -- 172.21.15.132:0/3302017971 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2461393291 0 0) 0x7fe39009e540 con 0x7fe39009fdd0 2024-01-24T23:23:16.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:16.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:16.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:16.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 10 monclient(hunting): my global_id is 74965 2024-01-24T23:23:16.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 1 -- 172.21.15.132:0/3302017971 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe39009e010 con 0x7fe39009fdd0 2024-01-24T23:23:16.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 1 -- 172.21.15.132:0/3302017971 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 312451704 0 0) 0x7fe390070340 con 0x7fe390099510 2024-01-24T23:23:16.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 1 -- 172.21.15.132:0/3302017971 >> v1:172.21.15.115:6791/0 conn(0x7fe39009fdd0 legacy=0x7fe3901cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 1 -- 172.21.15.132:0/3302017971 >> v1:172.21.15.115:6789/0 conn(0x7fe3900a01f0 legacy=0x7fe3901cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:16.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 10 monclient: _finish_hunting 0 2024-01-24T23:23:16.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 1 monclient: found mon.f 2024-01-24T23:23:16.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:16.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:16.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 1 -- 172.21.15.132:0/3302017971 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3901c76a0 con 0x7fe390099510 2024-01-24T23:23:16.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 10 monclient: _finish_auth 0 2024-01-24T23:23:16.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 10 monclient: _check_auth_tickets 2024-01-24T23:23:16.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:16.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 5 monclient: authenticate success, global_id 5077 2024-01-24T23:23:16.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 10 monclient: _renew_subs 2024-01-24T23:23:16.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:16.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 1 -- 172.21.15.132:0/3302017971 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe39006ee90 con 0x7fe390099510 2024-01-24T23:23:16.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 10 monclient: _renew_subs 2024-01-24T23:23:16.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:16.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe397ce1640 1 -- 172.21.15.132:0/3302017971 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe39006ea10 con 0x7fe390099510 2024-01-24T23:23:16.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 1 -- 172.21.15.132:0/3302017971 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe38c003950 con 0x7fe390099510 2024-01-24T23:23:16.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:16.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 1 -- 172.21.15.132:0/3302017971 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe38c003e80 con 0x7fe390099510 2024-01-24T23:23:16.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:16.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe387fff640 4 set_mon_vals no callback set 2024-01-24T23:23:16.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:23:16.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.636+0000 7fe3867fc640 10 monclient: dump: 2024-01-24T23:23:16.654 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:16.654 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:16.654 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.654 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:16.654 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:16.655 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:16.655 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:16.655 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:16.655 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:16.655 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:16.655 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:16.655 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:16.655 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:16.655 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:16.655 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:16.655 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:16.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.640+0000 7fe3867fc640 1 -- 172.21.15.132:0/3302017971 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fe38c003fe0 con 0x7fe390099510 2024-01-24T23:23:16.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.640+0000 7fe397ce1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:16.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.640+0000 7fe397ce1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:16.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.640+0000 7fe397ce1640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:16.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.640+0000 7fe397ce1640 1 -- 172.21.15.132:0/3302017971 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe378005090 con 0x7fe390099510 2024-01-24T23:23:16.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.640+0000 7fe3867fc640 1 -- 172.21.15.132:0/3302017971 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(49..49 src has 1..49) v4 ==== 4219+0+0 (unknown 1511542415 0 0) 0x7fe38c0439b0 con 0x7fe390099510 2024-01-24T23:23:16.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.640+0000 7fe39527c640 10 monclient: get_auth_request con 0x7fe3740371e0 auth_method 0 2024-01-24T23:23:16.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.644+0000 7fe3867fc640 1 -- 172.21.15.132:0/3302017971 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fe38c0113e0 con 0x7fe390099510 2024-01-24T23:23:16.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.644+0000 7fe3867fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:16.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.644+0000 7fe3867fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:16.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.800+0000 7fe397ce1640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}] 2024-01-24T23:23:16.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.800+0000 7fe397ce1640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}] 2024-01-24T23:23:16.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.800+0000 7fe397ce1640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:16.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:16.800+0000 7fe397ce1640 1 -- 172.21.15.132:0/3302017971 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"} v 0) v1 -- 0x7fe378005340 con 0x7fe390099510 2024-01-24T23:23:17.560 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-01-24T23:23:17.560 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-01-24T23:23:17.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44e64eb640 10 monclient: get_monmap_and_config 2024-01-24T23:23:17.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44e64eb640 10 monclient: build_initial_monmap 2024-01-24T23:23:17.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44e64eb640 10 monclient: monmap: 2024-01-24T23:23:17.691 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:17.691 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:17.691 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:17.691790+0000 2024-01-24T23:23:17.692 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:17.691790+0000 2024-01-24T23:23:17.692 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:17.692 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:17.692 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:17.692 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:17.692 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:17.692 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:17.692 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:17.692 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:17.692 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:17.692 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:17.692 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:17.692 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:17.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44e64eb640 1 Processor -- start 2024-01-24T23:23:17.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44e64eb640 1 -- start start 2024-01-24T23:23:17.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44e64eb640 10 monclient: init 2024-01-24T23:23:17.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44e64eb640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:17.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44e64eb640 10 monclient: _add_conns ranks=[1,3,7,6,4,8,0,5,2] 2024-01-24T23:23:17.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44e64eb640 10 monclient(hunting): picked mon.noname-f con 0x7f44e0133c10 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:17.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44e64eb640 10 monclient(hunting): picked mon.noname-g con 0x7f44e0136960 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:17.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44e64eb640 10 monclient(hunting): picked mon.noname-i con 0x7f44e013a470 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:17.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44e64eb640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e004ed70 con 0x7f44e0133c10 2024-01-24T23:23:17.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44e64eb640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e009dee0 con 0x7f44e0136960 2024-01-24T23:23:17.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44e64eb640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e009d400 con 0x7f44e013a470 2024-01-24T23:23:17.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44e64eb640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:17.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44e64eb640 10 monclient(hunting): authenticate will time out at 1076.614920s 2024-01-24T23:23:17.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44d77fe640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f44e0136960 0x7f44e0138d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.115:38862/0 (socket says 172.21.15.115:38862) 2024-01-24T23:23:17.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44d77fe640 1 -- 172.21.15.115:0/56617491 learned_addr learned my addr 172.21.15.115:0/56617491 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:17.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44df7fe640 1 -- 172.21.15.115:0/56617491 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3203030904 0 0) 0x7f44e004ed70 con 0x7f44e0133c10 2024-01-24T23:23:17.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44df7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:17.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44df7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:17.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:17.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44df7fe640 10 monclient(hunting): my global_id is 5086 2024-01-24T23:23:17.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44df7fe640 1 -- 172.21.15.115:0/56617491 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44cc003530 con 0x7f44e0133c10 2024-01-24T23:23:17.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44df7fe640 1 -- 172.21.15.115:0/56617491 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 775848854 0 0) 0x7f44e009d400 con 0x7f44e013a470 2024-01-24T23:23:17.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44df7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:17.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44df7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:17.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:17.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44df7fe640 10 monclient(hunting): my global_id is 5029 2024-01-24T23:23:17.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44df7fe640 1 -- 172.21.15.115:0/56617491 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44e004ed70 con 0x7f44e013a470 2024-01-24T23:23:17.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44df7fe640 1 -- 172.21.15.115:0/56617491 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 719309301 0 0) 0x7f44e009dee0 con 0x7f44e0136960 2024-01-24T23:23:17.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44df7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:17.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44df7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:17.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:17.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44df7fe640 10 monclient(hunting): my global_id is 4953 2024-01-24T23:23:17.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.688+0000 7f44df7fe640 1 -- 172.21.15.115:0/56617491 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44e009d400 con 0x7f44e0136960 2024-01-24T23:23:17.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44df7fe640 1 -- 172.21.15.115:0/56617491 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 882846542 0 0) 0x7f44e004ed70 con 0x7f44e013a470 2024-01-24T23:23:17.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44df7fe640 1 -- 172.21.15.115:0/56617491 >> v1:172.21.15.132:6790/0 conn(0x7f44e0136960 legacy=0x7f44e0138d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:17.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44df7fe640 1 -- 172.21.15.115:0/56617491 >> v1:172.21.15.132:6789/0 conn(0x7f44e0133c10 legacy=0x7f44e0133ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:17.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44df7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:23:17.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44df7fe640 1 monclient: found mon.noname-i 2024-01-24T23:23:17.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44df7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:17.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44df7fe640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:23:17.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44df7fe640 1 -- 172.21.15.115:0/56617491 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e009f380 con 0x7f44e013a470 2024-01-24T23:23:17.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44df7fe640 10 monclient: _finish_auth 0 2024-01-24T23:23:17.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44df7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:23:17.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44df7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:17.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44df7fe640 1 -- 172.21.15.115:0/56617491 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f44d8001b20 con 0x7f44e013a470 2024-01-24T23:23:17.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44df7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:17.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44df7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:23:17.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44df7fe640 10 monclient: dump: 2024-01-24T23:23:17.702 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:17.702 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:17.702 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:17.702 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:17.702 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:17.702 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:17.703 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:17.703 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:17.703 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:17.703 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:17.703 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:17.703 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:17.703 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:17.703 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:17.703 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:17.703 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:17.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 5 monclient: authenticate success, global_id 5029 2024-01-24T23:23:17.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:17.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44df7fe640 1 -- 172.21.15.115:0/56617491 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f44d8001dc0 con 0x7f44e013a470 2024-01-24T23:23:17.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44df7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:17.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44df7fe640 1 -- 172.21.15.115:0/56617491 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f44d8003ce0 con 0x7f44e013a470 2024-01-24T23:23:17.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44df7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:17.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44df7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:17.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44df7fe640 10 monclient: dump: 2024-01-24T23:23:17.704 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:17.704 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:17.704 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:17.704 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:17.704 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:17.704 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:17.704 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:17.704 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:17.704 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:17.704 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:17.705 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:17.705 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:17.705 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:17.705 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:17.705 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:17.705 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:17.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:17.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 4 set_mon_vals no callback set 2024-01-24T23:23:17.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 10 monclient: shutdown 2024-01-24T23:23:17.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:17.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 1 -- 172.21.15.115:0/56617491 >> v1:172.21.15.132:6792/0 conn(0x7f44e013a470 legacy=0x7f44e013c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:17.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 1 -- 172.21.15.115:0/56617491 shutdown_connections 2024-01-24T23:23:17.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 1 -- 172.21.15.115:0/56617491 >> 172.21.15.115:0/56617491 conn(0x7f44e00a5730 msgr2=0x7f44e00a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:17.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 1 -- 172.21.15.115:0/56617491 shutdown_connections 2024-01-24T23:23:17.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 1 -- 172.21.15.115:0/56617491 wait complete. 2024-01-24T23:23:17.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 10 monclient: build_initial_monmap 2024-01-24T23:23:17.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 10 monclient: monmap: 2024-01-24T23:23:17.706 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:17.706 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:17.706 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:17.706 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:17.706 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:17.706 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:17.706 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:17.706 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:17.706 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:17.706 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:17.706 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:17.706 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:17.707 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:17.707 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:17.707 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:17.707 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:17.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 1 Processor -- start 2024-01-24T23:23:17.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 1 -- start start 2024-01-24T23:23:17.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 10 monclient: init 2024-01-24T23:23:17.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:17.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 10 monclient: _add_conns ranks=[8,7,0,4,6,3,1,5,2] 2024-01-24T23:23:17.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 10 monclient(hunting): picked mon.noname-e con 0x7f44e0133c10 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:17.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 10 monclient(hunting): picked mon.noname-i con 0x7f44e0136960 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:17.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 10 monclient(hunting): picked mon.noname-a con 0x7f44e013a470 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:17.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e0131a90 con 0x7f44e013a470 2024-01-24T23:23:17.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e0139b30 con 0x7f44e0136960 2024-01-24T23:23:17.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e00a1310 con 0x7f44e0133c10 2024-01-24T23:23:17.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:17.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 10 monclient(hunting): authenticate will time out at 1076.618905s 2024-01-24T23:23:17.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dffff640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f44e0133c10 0x7f44e0138d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:52958/0 (socket says 172.21.15.115:52958) 2024-01-24T23:23:17.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dffff640 1 -- 172.21.15.115:0/2089235677 learned_addr learned my addr 172.21.15.115:0/2089235677 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:17.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 1 -- 172.21.15.115:0/2089235677 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 504201216 0 0) 0x7f44e00a1310 con 0x7f44e0133c10 2024-01-24T23:23:17.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:23:17.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:17.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:17.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 10 monclient(hunting): my global_id is 84203 2024-01-24T23:23:17.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 1 -- 172.21.15.115:0/2089235677 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44c0003530 con 0x7f44e0133c10 2024-01-24T23:23:17.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 1 -- 172.21.15.115:0/2089235677 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1057043623 0 0) 0x7f44e0131a90 con 0x7f44e013a470 2024-01-24T23:23:17.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:23:17.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:17.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:17.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 10 monclient(hunting): my global_id is 4986 2024-01-24T23:23:17.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 1 -- 172.21.15.115:0/2089235677 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44e00a1310 con 0x7f44e013a470 2024-01-24T23:23:17.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 1 -- 172.21.15.115:0/2089235677 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2403917981 0 0) 0x7f44c0003530 con 0x7f44e0133c10 2024-01-24T23:23:17.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 1 -- 172.21.15.115:0/2089235677 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f44e0131a90 con 0x7f44e0133c10 2024-01-24T23:23:17.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 1 -- 172.21.15.115:0/2089235677 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 525142842 0 0) 0x7f44e0139b30 con 0x7f44e0136960 2024-01-24T23:23:17.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:23:17.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:17.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:17.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 10 monclient(hunting): my global_id is 5038 2024-01-24T23:23:17.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 1 -- 172.21.15.115:0/2089235677 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44c0003530 con 0x7f44e0136960 2024-01-24T23:23:17.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 1 -- 172.21.15.115:0/2089235677 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4193514577 0 0) 0x7f44e00a1310 con 0x7f44e013a470 2024-01-24T23:23:17.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 1 -- 172.21.15.115:0/2089235677 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f44e0139b30 con 0x7f44e013a470 2024-01-24T23:23:17.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 1 -- 172.21.15.115:0/2089235677 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f44d0001640 con 0x7f44e0133c10 2024-01-24T23:23:17.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:17.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:23:17.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 10 monclient(hunting): dump: 2024-01-24T23:23:17.710 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:17.710 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:17.710 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:17.710 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:17.710 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:17.710 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:17.710 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:17.710 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:17.710 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:17.710 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:17.710 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:17.710 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:17.711 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:17.711 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:17.711 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:17.711 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:17.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:17.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 1 -- 172.21.15.115:0/2089235677 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f44d8001b80 con 0x7f44e013a470 2024-01-24T23:23:17.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:17.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:23:17.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 10 monclient(hunting): dump: 2024-01-24T23:23:17.711 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:17.711 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:17.711 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:17.711 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:17.711 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:17.711 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:17.712 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:17.712 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:17.712 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:17.712 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:17.712 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:17.712 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:17.712 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:17.712 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:17.712 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:17.712 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:17.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 1 -- 172.21.15.115:0/2089235677 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1719922872 0 0) 0x7f44e0131a90 con 0x7f44e0133c10 2024-01-24T23:23:17.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 1 -- 172.21.15.115:0/2089235677 >> v1:172.21.15.132:6792/0 conn(0x7f44e0136960 legacy=0x7f44e0139420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:17.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 1 -- 172.21.15.115:0/2089235677 >> v1:172.21.15.115:6789/0 conn(0x7f44e013a470 legacy=0x7f44e01d3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:17.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 10 monclient: _finish_hunting 0 2024-01-24T23:23:17.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 1 monclient: found mon.e 2024-01-24T23:23:17.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:17.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:23:17.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 1 -- 172.21.15.115:0/2089235677 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e01314d0 con 0x7f44e0133c10 2024-01-24T23:23:17.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 10 monclient: _finish_auth 0 2024-01-24T23:23:17.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:23:17.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44dd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:17.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 5 monclient: authenticate success, global_id 84203 2024-01-24T23:23:17.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 10 monclient: _renew_subs 2024-01-24T23:23:17.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:23:17.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 1 -- 172.21.15.115:0/2089235677 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e01317b0 con 0x7f44e0133c10 2024-01-24T23:23:17.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 10 monclient: _renew_subs 2024-01-24T23:23:17.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:23:17.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.692+0000 7f44e64eb640 1 -- 172.21.15.115:0/2089235677 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e01327a0 con 0x7f44e0133c10 2024-01-24T23:23:17.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.696+0000 7f44dd7fa640 1 -- 172.21.15.115:0/2089235677 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f44d00038b0 con 0x7f44e0133c10 2024-01-24T23:23:17.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.696+0000 7f44dd7fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:17.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.696+0000 7f44dd7fa640 1 -- 172.21.15.115:0/2089235677 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f44d0003de0 con 0x7f44e0133c10 2024-01-24T23:23:17.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.696+0000 7f44dd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:17.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.696+0000 7f44dd7fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:17.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.696+0000 7f44dd7fa640 10 monclient: dump: 2024-01-24T23:23:17.714 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:17.714 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:17.714 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:17.714 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:17.714 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:17.714 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:17.714 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:17.715 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:17.715 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:17.715 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:17.715 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:17.715 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:17.715 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:17.715 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:17.715 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:17.715 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:17.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.696+0000 7f44de7fc640 4 set_mon_vals no callback set 2024-01-24T23:23:17.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.696+0000 7f44e64eb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:17.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.696+0000 7f44dd7fa640 1 -- 172.21.15.115:0/2089235677 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f44d0004000 con 0x7f44e0133c10 2024-01-24T23:23:17.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.696+0000 7f44e64eb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:17.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.696+0000 7f44e64eb640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:23:17.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.696+0000 7f44e64eb640 1 -- 172.21.15.115:0/2089235677 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c4005090 con 0x7f44e0133c10 2024-01-24T23:23:17.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.696+0000 7f44dd7fa640 1 -- 172.21.15.115:0/2089235677 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(50..50 src has 1..50) v4 ==== 4305+0+0 (unknown 1100805051 0 0) 0x7f44d0043a90 con 0x7f44e0133c10 2024-01-24T23:23:17.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.696+0000 7f44d77fe640 10 monclient: get_auth_request con 0x7f44c00378a0 auth_method 0 2024-01-24T23:23:17.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.700+0000 7f44dd7fa640 1 -- 172.21.15.115:0/2089235677 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f44d0014ae0 con 0x7f44e0133c10 2024-01-24T23:23:17.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.700+0000 7f44dd7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:17.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.700+0000 7f44dd7fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:17.914 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.912+0000 7f44e64eb640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-01-24T23:23:17.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.912+0000 7f44e64eb640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-01-24T23:23:17.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.912+0000 7f44e64eb640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:23:17.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.912+0000 7f44e64eb640 1 -- 172.21.15.115:0/2089235677 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f44c40058c0 con 0x7f44e0133c10 2024-01-24T23:23:17.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.912+0000 7f44dd7fa640 1 -- 172.21.15.115:0/2089235677 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2744 (unknown 254746657 0 1007323601) 0x7f44d0012f50 con 0x7f44e0133c10 2024-01-24T23:23:17.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.912+0000 7f44dd7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-01-24T23:23:17.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.912+0000 7f44dd7fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:17.916 INFO:teuthology.orchestra.run.smithi115.stdout:{"election_epoch":16,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701272860131327","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","modified":"2024-01-24T23:20:20.557246Z","created":"2024-01-24T23:20:20.557246Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6790","nonce":0}]},"addr":"172.21.15.115:6790/0","public_addr":"172.21.15.115:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6790","nonce":0}]},"addr":"172.21.15.132:6790/0","public_addr":"172.21.15.132:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6791","nonce":0}]},"addr":"172.21.15.115:6791/0","public_addr":"172.21.15.115:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6791","nonce":0}]},"addr":"172.21.15.132:6791/0","public_addr":"172.21.15.132:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6792","nonce":0}]},"addr":"172.21.15.115:6792/0","public_addr":"172.21.15.115:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6792","nonce":0}]},"addr":"172.21.15.132:6792/0","public_addr":"172.21.15.132:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6793","nonce":0}]},"addr":"172.21.15.115:6793/0","public_addr":"172.21.15.115:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-01-24T23:23:17.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.912+0000 7f44e64eb640 1 -- 172.21.15.115:0/2089235677 >> v1:172.21.15.132:6812/1835401922 conn(0x7f44c00378a0 legacy=0x7f44c0039d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:17.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.912+0000 7f44e64eb640 10 monclient: shutdown 2024-01-24T23:23:17.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.912+0000 7f44e64eb640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:17.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.912+0000 7f44e64eb640 1 -- 172.21.15.115:0/2089235677 >> v1:172.21.15.115:6793/0 conn(0x7f44e0133c10 legacy=0x7f44e0138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:17.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.912+0000 7f44e64eb640 1 -- 172.21.15.115:0/2089235677 shutdown_connections 2024-01-24T23:23:17.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.912+0000 7f44e64eb640 1 -- 172.21.15.115:0/2089235677 >> 172.21.15.115:0/2089235677 conn(0x7f44e00a5730 msgr2=0x7f44e0136220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:17.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.912+0000 7f44e64eb640 1 -- 172.21.15.115:0/2089235677 shutdown_connections 2024-01-24T23:23:17.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:17.912+0000 7f44e64eb640 1 -- 172.21.15.115:0/2089235677 wait complete. 2024-01-24T23:23:17.934 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-01-24T23:23:17.934 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-01-24T23:23:17.934 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-01-24T23:23:17.934 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-01-24T23:23:18.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 10 monclient: get_monmap_and_config 2024-01-24T23:23:18.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 10 monclient: build_initial_monmap 2024-01-24T23:23:18.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 10 monclient: monmap: 2024-01-24T23:23:18.051 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:18.051 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.051 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:18.051113+0000 2024-01-24T23:23:18.051 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:18.051113+0000 2024-01-24T23:23:18.051 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:18.051 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.051 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:18.051 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:18.051 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:18.051 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:18.051 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:18.051 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:18.051 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:18.051 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:18.052 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:18.052 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 1 Processor -- start 2024-01-24T23:23:18.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 1 -- start start 2024-01-24T23:23:18.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 10 monclient: init 2024-01-24T23:23:18.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:18.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 10 monclient: _add_conns ranks=[8,1,7,4,6,3,2,5,0] 2024-01-24T23:23:18.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 10 monclient(hunting): picked mon.noname-e con 0x7f4d6c133bf0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:18.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 10 monclient(hunting): picked mon.noname-f con 0x7f4d6c133fd0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:18.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 10 monclient(hunting): picked mon.noname-i con 0x7f4d6c1343b0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:18.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d6c04ed70 con 0x7f4d6c133fd0 2024-01-24T23:23:18.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d6c0a4b60 con 0x7f4d6c1343b0 2024-01-24T23:23:18.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d6c09ae50 con 0x7f4d6c133bf0 2024-01-24T23:23:18.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 10 monclient(hunting): authenticate will time out at 1076.973828s 2024-01-24T23:23:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d71d2b640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f4d6c133bf0 0x7f4d6c09bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:52964/0 (socket says 172.21.15.115:52964) 2024-01-24T23:23:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d71d2b640 1 -- 172.21.15.115:0/3584214495 learned_addr learned my addr 172.21.15.115:0/3584214495 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 1 -- 172.21.15.115:0/3584214495 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 910111267 0 0) 0x7f4d6c09ae50 con 0x7f4d6c133bf0 2024-01-24T23:23:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 10 monclient(hunting): my global_id is 84212 2024-01-24T23:23:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 1 -- 172.21.15.115:0/3584214495 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d58003530 con 0x7f4d6c133bf0 2024-01-24T23:23:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 1 -- 172.21.15.115:0/3584214495 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2119935725 0 0) 0x7f4d58003530 con 0x7f4d6c133bf0 2024-01-24T23:23:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 1 -- 172.21.15.115:0/3584214495 >> v1:172.21.15.132:6792/0 conn(0x7f4d6c1343b0 legacy=0x7f4d6c13f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 1 -- 172.21.15.115:0/3584214495 >> v1:172.21.15.132:6789/0 conn(0x7f4d6c133fd0 legacy=0x7f4d6c13bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 10 monclient: _finish_hunting 0 2024-01-24T23:23:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 1 monclient: found mon.noname-e 2024-01-24T23:23:18.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:18.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:23:18.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 1 -- 172.21.15.115:0/3584214495 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c09b130 con 0x7f4d6c133bf0 2024-01-24T23:23:18.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 10 monclient: _finish_auth 0 2024-01-24T23:23:18.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 10 monclient: _check_auth_tickets 2024-01-24T23:23:18.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:18.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 1 -- 172.21.15.115:0/3584214495 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4d680017c0 con 0x7f4d6c133bf0 2024-01-24T23:23:18.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:23:18.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 10 monclient: dump: 2024-01-24T23:23:18.063 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.063 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.063 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.063 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.063 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.064 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.064 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.064 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.064 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.064 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.064 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.064 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.064 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.064 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.064 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.064 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 5 monclient: authenticate success, global_id 84212 2024-01-24T23:23:18.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:18.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 1 -- 172.21.15.115:0/3584214495 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4d68001ae0 con 0x7f4d6c133bf0 2024-01-24T23:23:18.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:18.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 1 -- 172.21.15.115:0/3584214495 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4d68004bf0 con 0x7f4d6c133bf0 2024-01-24T23:23:18.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:18.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d70d29640 10 monclient: dump: 2024-01-24T23:23:18.065 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.065 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.065 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.065 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.065 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.065 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.065 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.065 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.065 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.065 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.066 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 4 set_mon_vals no callback set 2024-01-24T23:23:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 10 monclient: shutdown 2024-01-24T23:23:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 1 -- 172.21.15.115:0/3584214495 >> v1:172.21.15.115:6793/0 conn(0x7f4d6c133bf0 legacy=0x7f4d6c09bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 1 -- 172.21.15.115:0/3584214495 shutdown_connections 2024-01-24T23:23:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 1 -- 172.21.15.115:0/3584214495 >> 172.21.15.115:0/3584214495 conn(0x7f4d6c0662f0 msgr2=0x7f4d6c066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 1 -- 172.21.15.115:0/3584214495 shutdown_connections 2024-01-24T23:23:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 1 -- 172.21.15.115:0/3584214495 wait complete. 2024-01-24T23:23:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 10 monclient: build_initial_monmap 2024-01-24T23:23:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.048+0000 7f4d73f8f640 10 monclient: monmap: 2024-01-24T23:23:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:18.068 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:18.068 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:18.068 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:18.068 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:18.068 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:18.068 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:18.068 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:18.068 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d73f8f640 1 Processor -- start 2024-01-24T23:23:18.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d73f8f640 1 -- start start 2024-01-24T23:23:18.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d73f8f640 10 monclient: init 2024-01-24T23:23:18.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d73f8f640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:18.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d73f8f640 10 monclient: _add_conns ranks=[1,6,7,3,4,0,2,5,8] 2024-01-24T23:23:18.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d73f8f640 10 monclient(hunting): picked mon.noname-f con 0x7f4d6c133bf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:18.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d73f8f640 10 monclient(hunting): picked mon.noname-d con 0x7f4d6c133fd0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:18.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d73f8f640 10 monclient(hunting): picked mon.noname-i con 0x7f4d6c1343b0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:18.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d73f8f640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d6c0a5990 con 0x7f4d6c133bf0 2024-01-24T23:23:18.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d73f8f640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d6c0a1e50 con 0x7f4d6c133fd0 2024-01-24T23:23:18.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d73f8f640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d6c1336e0 con 0x7f4d6c1343b0 2024-01-24T23:23:18.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d73f8f640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:18.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d73f8f640 10 monclient(hunting): authenticate will time out at 1076.976704s 2024-01-24T23:23:18.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d7152a640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f4d6c133fd0 0x7f4d6c1d0840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:34194/0 (socket says 172.21.15.115:34194) 2024-01-24T23:23:18.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d7152a640 1 -- 172.21.15.115:0/474375188 learned_addr learned my addr 172.21.15.115:0/474375188 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:18.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 364707685 0 0) 0x7f4d6c0a1e50 con 0x7f4d6c133fd0 2024-01-24T23:23:18.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient(hunting): my global_id is 5028 2024-01-24T23:23:18.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d50003530 con 0x7f4d6c133fd0 2024-01-24T23:23:18.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3442633105 0 0) 0x7f4d50003530 con 0x7f4d6c133fd0 2024-01-24T23:23:18.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4d6c0a1e50 con 0x7f4d6c133fd0 2024-01-24T23:23:18.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4d5c001ac0 con 0x7f4d6c133fd0 2024-01-24T23:23:18.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:23:18.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient(hunting): dump: 2024-01-24T23:23:18.071 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.071 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.071 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.071 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.071 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.071 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.071 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.071 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.071 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.071 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.071 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.071 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.071 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.072 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.072 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.072 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:18.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936559078 0 0) 0x7f4d6c0a5990 con 0x7f4d6c133bf0 2024-01-24T23:23:18.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient(hunting): my global_id is 5104 2024-01-24T23:23:18.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d50003530 con 0x7f4d6c133bf0 2024-01-24T23:23:18.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1739912149 0 0) 0x7f4d6c1336e0 con 0x7f4d6c1343b0 2024-01-24T23:23:18.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient(hunting): my global_id is 5056 2024-01-24T23:23:18.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d6c0a5990 con 0x7f4d6c1343b0 2024-01-24T23:23:18.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2892888221 0 0) 0x7f4d6c0a1e50 con 0x7f4d6c133fd0 2024-01-24T23:23:18.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 >> v1:172.21.15.132:6792/0 conn(0x7f4d6c1343b0 legacy=0x7f4d6c1d3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 >> v1:172.21.15.132:6789/0 conn(0x7f4d6c133bf0 legacy=0x7f4d6c132fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient: _finish_hunting 0 2024-01-24T23:23:18.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 monclient: found mon.d 2024-01-24T23:23:18.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:18.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:23:18.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c11fc20 con 0x7f4d6c133fd0 2024-01-24T23:23:18.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient: _finish_auth 0 2024-01-24T23:23:18.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient: _check_auth_tickets 2024-01-24T23:23:18.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:18.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d73f8f640 5 monclient: authenticate success, global_id 5028 2024-01-24T23:23:18.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d73f8f640 10 monclient: _renew_subs 2024-01-24T23:23:18.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d73f8f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:23:18.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d73f8f640 1 -- 172.21.15.115:0/474375188 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6c120990 con 0x7f4d6c133fd0 2024-01-24T23:23:18.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d73f8f640 10 monclient: _renew_subs 2024-01-24T23:23:18.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d73f8f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:23:18.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d73f8f640 1 -- 172.21.15.115:0/474375188 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6c0a31b0 con 0x7f4d6c133fd0 2024-01-24T23:23:18.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4d5c003950 con 0x7f4d6c133fd0 2024-01-24T23:23:18.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:18.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4d5c003e80 con 0x7f4d6c133fd0 2024-01-24T23:23:18.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d63fff640 4 set_mon_vals no callback set 2024-01-24T23:23:18.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:18.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient: dump: 2024-01-24T23:23:18.074 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.074 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.074 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.075 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.075 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.075 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.075 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.075 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.075 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.075 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.075 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.075 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.075 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.075 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.075 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.075 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d73f8f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:18.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d73f8f640 1 -- 172.21.15.115:0/474375188 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4d540065d0 con 0x7f4d54002870 2024-01-24T23:23:18.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f4d5c004060 con 0x7f4d6c133fd0 2024-01-24T23:23:18.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(50..50 src has 1..50) v4 ==== 4305+0+0 (unknown 1100805051 0 0) 0x7f4d5c043a00 con 0x7f4d6c133fd0 2024-01-24T23:23:18.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1915264501 0 0) 0x7f4d540065d0 con 0x7f4d54002870 2024-01-24T23:23:18.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient(hunting): my global_id is 4995 2024-01-24T23:23:18.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d6c0a1e50 con 0x7f4d54002870 2024-01-24T23:23:18.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2754282779 0 0) 0x7f4d6c0a1e50 con 0x7f4d54002870 2024-01-24T23:23:18.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4d540065d0 con 0x7f4d54002870 2024-01-24T23:23:18.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4d68004b30 con 0x7f4d54002870 2024-01-24T23:23:18.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d7252c640 10 monclient: get_auth_request con 0x7f4d50037140 auth_method 0 2024-01-24T23:23:18.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:23:18.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient: dump: 2024-01-24T23:23:18.077 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.077 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.077 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.077 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.077 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.077 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.077 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.077 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.077 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.077 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.077 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.077 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.077 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.077 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.077 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.078 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 804039871 0 0) 0x7f4d540065d0 con 0x7f4d54002870 2024-01-24T23:23:18.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 --> v1:172.21.15.115:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4d5400a120 con 0x7f4d54002870 2024-01-24T23:23:18.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 <== mon.0 v1:172.21.15.115:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f4d68001540 con 0x7f4d54002870 2024-01-24T23:23:18.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:18.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:18.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 >> v1:172.21.15.115:6789/0 conn(0x7f4d54002870 legacy=0x7f4d54004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.052+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 >> v1:172.21.15.115:6789/0 conn(0x7f4d54002870 legacy=0x7f4d54004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:18.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d73f8f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:23:18.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d73f8f640 1 -- 172.21.15.115:0/474375188 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4d54009e10 con 0x7f4d540090d0 2024-01-24T23:23:18.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3464288974 0 0) 0x7f4d54009e10 con 0x7f4d540090d0 2024-01-24T23:23:18.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d627fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d627fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d627fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d627fc640 10 monclient(hunting): my global_id is 5004 2024-01-24T23:23:18.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d540065d0 con 0x7f4d540090d0 2024-01-24T23:23:18.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2248575834 0 0) 0x7f4d540065d0 con 0x7f4d540090d0 2024-01-24T23:23:18.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4d54009e10 con 0x7f4d540090d0 2024-01-24T23:23:18.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4d68005340 con 0x7f4d540090d0 2024-01-24T23:23:18.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d627fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d627fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:23:18.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d627fc640 10 monclient: dump: 2024-01-24T23:23:18.080 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.080 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.081 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.081 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.081 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.081 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.081 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.081 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.081 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.081 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.081 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.081 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.081 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.081 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.081 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.081 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1593872734 0 0) 0x7f4d54009e10 con 0x7f4d540090d0 2024-01-24T23:23:18.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 --> v1:172.21.15.115:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4d5400c310 con 0x7f4d540090d0 2024-01-24T23:23:18.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 <== mon.0 v1:172.21.15.115:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7715 (unknown 0 0 4180996593) 0x7f4d68007660 con 0x7f4d540090d0 2024-01-24T23:23:18.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d627fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:23:18.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d627fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:18.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 >> v1:172.21.15.115:6789/0 conn(0x7f4d540090d0 legacy=0x7f4d54009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d627fc640 1 -- 172.21.15.115:0/474375188 >> v1:172.21.15.115:6789/0 conn(0x7f4d540090d0 legacy=0x7f4d54009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:18.082 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:23:18.082 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:18.082 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:18.082 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "leader", 2024-01-24T23:23:18.082 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 16, 2024-01-24T23:23:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:23:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:23:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:23:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:23:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:23:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:23:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:23:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:23:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:23:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:23:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 3, 2024-01-24T23:23:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:18.083 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:23:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:23:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.084 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:23:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:23:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:18.085 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.086 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:23:18.086 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:23:18.086 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:23:18.086 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:23:18.086 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:23:18.086 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:23:18.086 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:18.086 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:18.086 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:23:18.086 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:23:18.086 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:23:18.086 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:23:18.086 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:23:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:23:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:23:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:23:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:18.087 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:23:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:23:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.088 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:23:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:23:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:23:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:18.089 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:23:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:23:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:23:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:23:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:23:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:23:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:23:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.092 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:23:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:23:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.093 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:23:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:23:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:23:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:23:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:23:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:23:18.095 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:23:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.096 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:23:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:23:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:23:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:23:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:23:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.098 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:23:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:23:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:23:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.099 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.100 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.100 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.100 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.100 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:23:18.100 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:23:18.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.100 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:18.100 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:18.100 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:18.100 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.100 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.100 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:23:18.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.101 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:18.101 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:18.101 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:18.101 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.101 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.101 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:23:18.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.101 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:18.101 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:18.101 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:23:18.101 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.101 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.101 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:23:18.102 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:23:18.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d73f8f640 1 -- 172.21.15.115:0/474375188 >> v1:172.21.15.132:6812/1835401922 conn(0x7f4d50037140 legacy=0x7f4d50039600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d73f8f640 10 monclient: shutdown 2024-01-24T23:23:18.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d73f8f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:18.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d73f8f640 1 -- 172.21.15.115:0/474375188 >> v1:172.21.15.115:6792/0 conn(0x7f4d6c133fd0 legacy=0x7f4d6c1d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d7252c640 1 -- 172.21.15.115:0/474375188 reap_dead start 2024-01-24T23:23:18.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d7252c640 1 -- 172.21.15.115:0/474375188 reap_dead start 2024-01-24T23:23:18.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d73f8f640 1 -- 172.21.15.115:0/474375188 shutdown_connections 2024-01-24T23:23:18.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d73f8f640 1 -- 172.21.15.115:0/474375188 >> 172.21.15.115:0/474375188 conn(0x7f4d6c0662f0 msgr2=0x7f4d6c1be970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:18.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d73f8f640 1 -- 172.21.15.115:0/474375188 shutdown_connections 2024-01-24T23:23:18.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.060+0000 7f4d73f8f640 1 -- 172.21.15.115:0/474375188 wait complete. 2024-01-24T23:23:18.104 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:23:18.104 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-01-24T23:23:18.104 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-01-24T23:23:18.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.212+0000 7f5c76609640 10 monclient: get_monmap_and_config 2024-01-24T23:23:18.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 10 monclient: build_initial_monmap 2024-01-24T23:23:18.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 10 monclient: monmap: 2024-01-24T23:23:18.219 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:18.219 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.219 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:18.218938+0000 2024-01-24T23:23:18.219 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:18.218938+0000 2024-01-24T23:23:18.219 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:18.219 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.219 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:18.219 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:18.219 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:18.219 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:18.219 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:18.219 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:18.220 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:18.220 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:18.220 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:18.220 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 1 Processor -- start 2024-01-24T23:23:18.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 1 -- start start 2024-01-24T23:23:18.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 10 monclient: init 2024-01-24T23:23:18.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:18.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 10 monclient: _add_conns ranks=[0,2,8,6,7,5,1,3,4] 2024-01-24T23:23:18.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 10 monclient(hunting): picked mon.noname-a con 0x7f5c70133d90 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:18.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 10 monclient(hunting): picked mon.noname-b con 0x7f5c70134170 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:18.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 10 monclient(hunting): picked mon.noname-e con 0x7f5c7013dcf0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:18.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c7004ed70 con 0x7f5c70133d90 2024-01-24T23:23:18.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c7009b250 con 0x7f5c70134170 2024-01-24T23:23:18.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c701345d0 con 0x7f5c7013dcf0 2024-01-24T23:23:18.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:18.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 10 monclient(hunting): authenticate will time out at 1077.141477s 2024-01-24T23:23:18.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6f7fe640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f5c70134170 0x7f5c700a2100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:47632/0 (socket says 172.21.15.115:47632) 2024-01-24T23:23:18.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6f7fe640 1 -- 172.21.15.115:0/1448342275 learned_addr learned my addr 172.21.15.115:0/1448342275 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:18.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 1 -- 172.21.15.115:0/1448342275 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 545162514 0 0) 0x7f5c7009b250 con 0x7f5c70134170 2024-01-24T23:23:18.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 10 monclient(hunting): my global_id is 84215 2024-01-24T23:23:18.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 1 -- 172.21.15.115:0/1448342275 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c58003530 con 0x7f5c70134170 2024-01-24T23:23:18.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 1 -- 172.21.15.115:0/1448342275 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142184779 0 0) 0x7f5c7004ed70 con 0x7f5c70133d90 2024-01-24T23:23:18.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 10 monclient(hunting): my global_id is 5013 2024-01-24T23:23:18.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 1 -- 172.21.15.115:0/1448342275 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c7009b250 con 0x7f5c70133d90 2024-01-24T23:23:18.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 1 -- 172.21.15.115:0/1448342275 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1755339551 0 0) 0x7f5c701345d0 con 0x7f5c7013dcf0 2024-01-24T23:23:18.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 10 monclient(hunting): my global_id is 84221 2024-01-24T23:23:18.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 1 -- 172.21.15.115:0/1448342275 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c7004ed70 con 0x7f5c7013dcf0 2024-01-24T23:23:18.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 1 -- 172.21.15.115:0/1448342275 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3772963747 0 0) 0x7f5c58003530 con 0x7f5c70134170 2024-01-24T23:23:18.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 1 -- 172.21.15.115:0/1448342275 >> v1:172.21.15.115:6793/0 conn(0x7f5c7013dcf0 legacy=0x7f5c701400e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 1 -- 172.21.15.115:0/1448342275 >> v1:172.21.15.115:6789/0 conn(0x7f5c70133d90 legacy=0x7f5c700a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:18.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 1 monclient: found mon.noname-b 2024-01-24T23:23:18.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:18.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:23:18.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 1 -- 172.21.15.115:0/1448342275 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c700a2810 con 0x7f5c70134170 2024-01-24T23:23:18.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 10 monclient: _finish_auth 0 2024-01-24T23:23:18.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:18.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:18.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 1 -- 172.21.15.115:0/1448342275 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5c640017c0 con 0x7f5c70134170 2024-01-24T23:23:18.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:23:18.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 10 monclient: dump: 2024-01-24T23:23:18.231 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.231 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.232 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.232 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.232 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.232 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.232 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.232 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.232 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.232 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.232 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.232 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.232 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.232 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.232 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.232 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 5 monclient: authenticate success, global_id 84215 2024-01-24T23:23:18.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:18.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 1 -- 172.21.15.115:0/1448342275 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5c64001aa0 con 0x7f5c70134170 2024-01-24T23:23:18.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:18.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 1 -- 172.21.15.115:0/1448342275 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5c64004bf0 con 0x7f5c70134170 2024-01-24T23:23:18.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:18.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c6effd640 10 monclient: dump: 2024-01-24T23:23:18.233 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.233 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.233 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.233 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.233 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.233 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.233 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.234 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.234 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.234 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.234 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.234 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.234 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.234 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.234 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.234 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:18.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 4 set_mon_vals no callback set 2024-01-24T23:23:18.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 10 monclient: shutdown 2024-01-24T23:23:18.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:18.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 1 -- 172.21.15.115:0/1448342275 >> v1:172.21.15.115:6790/0 conn(0x7f5c70134170 legacy=0x7f5c700a2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 1 -- 172.21.15.115:0/1448342275 shutdown_connections 2024-01-24T23:23:18.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 1 -- 172.21.15.115:0/1448342275 >> 172.21.15.115:0/1448342275 conn(0x7f5c700a65e0 msgr2=0x7f5c700a2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:18.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 1 -- 172.21.15.115:0/1448342275 shutdown_connections 2024-01-24T23:23:18.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 1 -- 172.21.15.115:0/1448342275 wait complete. 2024-01-24T23:23:18.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 10 monclient: build_initial_monmap 2024-01-24T23:23:18.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.216+0000 7f5c76609640 10 monclient: monmap: 2024-01-24T23:23:18.235 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:18.235 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:18.235 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:18.235 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:18.235 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:18.235 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:18.235 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:18.235 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:18.235 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:18.236 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:18.236 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:18.236 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:18.236 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:18.236 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:18.236 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:18.236 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c76609640 1 Processor -- start 2024-01-24T23:23:18.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c76609640 1 -- start start 2024-01-24T23:23:18.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c76609640 10 monclient: init 2024-01-24T23:23:18.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c76609640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:18.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c76609640 10 monclient: _add_conns ranks=[3,6,2,8,7,5,0,4,1] 2024-01-24T23:23:18.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c76609640 10 monclient(hunting): picked mon.noname-g con 0x7f5c70133d90 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:18.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c76609640 10 monclient(hunting): picked mon.noname-d con 0x7f5c70134170 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:18.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c76609640 10 monclient(hunting): picked mon.noname-b con 0x7f5c7013dcf0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:18.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c76609640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c70138c20 con 0x7f5c7013dcf0 2024-01-24T23:23:18.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c76609640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c701388c0 con 0x7f5c70133d90 2024-01-24T23:23:18.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c76609640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c700676c0 con 0x7f5c70134170 2024-01-24T23:23:18.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c76609640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:18.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c76609640 10 monclient(hunting): authenticate will time out at 1077.144564s 2024-01-24T23:23:18.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c74ba6640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f5c7013dcf0 0x7f5c701d3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:47642/0 (socket says 172.21.15.115:47642) 2024-01-24T23:23:18.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c74ba6640 1 -- 172.21.15.115:0/3804456064 learned_addr learned my addr 172.21.15.115:0/3804456064 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:18.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 558395718 0 0) 0x7f5c70138c20 con 0x7f5c7013dcf0 2024-01-24T23:23:18.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient(hunting): my global_id is 84224 2024-01-24T23:23:18.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c5c003530 con 0x7f5c7013dcf0 2024-01-24T23:23:18.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890098769 0 0) 0x7f5c700676c0 con 0x7f5c70134170 2024-01-24T23:23:18.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient(hunting): my global_id is 5037 2024-01-24T23:23:18.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c70138c20 con 0x7f5c70134170 2024-01-24T23:23:18.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 781259145 0 0) 0x7f5c5c003530 con 0x7f5c7013dcf0 2024-01-24T23:23:18.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5c700676c0 con 0x7f5c7013dcf0 2024-01-24T23:23:18.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5c68001d60 con 0x7f5c7013dcf0 2024-01-24T23:23:18.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:23:18.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient(hunting): dump: 2024-01-24T23:23:18.238 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.239 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.239 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.239 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.239 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.239 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.239 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.239 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.239 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.239 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.239 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.239 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.239 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.239 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.239 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.240 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:18.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2994679924 0 0) 0x7f5c70138c20 con 0x7f5c70134170 2024-01-24T23:23:18.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5c5c003530 con 0x7f5c70134170 2024-01-24T23:23:18.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5c64004b30 con 0x7f5c70134170 2024-01-24T23:23:18.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:18.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient(hunting): dump: 2024-01-24T23:23:18.240 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.240 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.240 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.240 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.240 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.240 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.241 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.241 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.241 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.241 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.241 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.241 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.241 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.241 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.241 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.241 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2312038951 0 0) 0x7f5c701388c0 con 0x7f5c70133d90 2024-01-24T23:23:18.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient(hunting): my global_id is 4962 2024-01-24T23:23:18.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c70138c20 con 0x7f5c70133d90 2024-01-24T23:23:18.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3053918157 0 0) 0x7f5c700676c0 con 0x7f5c7013dcf0 2024-01-24T23:23:18.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 >> v1:172.21.15.115:6792/0 conn(0x7f5c70134170 legacy=0x7f5c701381b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 >> v1:172.21.15.132:6790/0 conn(0x7f5c70133d90 legacy=0x7f5c70137aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient: _finish_hunting 0 2024-01-24T23:23:18.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 monclient: found mon.b 2024-01-24T23:23:18.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:18.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:18.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7013aea0 con 0x7f5c7013dcf0 2024-01-24T23:23:18.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient: _finish_auth 0 2024-01-24T23:23:18.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:18.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:18.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c76609640 5 monclient: authenticate success, global_id 84224 2024-01-24T23:23:18.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c76609640 10 monclient: _renew_subs 2024-01-24T23:23:18.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c76609640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:18.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c76609640 1 -- 172.21.15.115:0/3804456064 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7013a600 con 0x7f5c7013dcf0 2024-01-24T23:23:18.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c76609640 10 monclient: _renew_subs 2024-01-24T23:23:18.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c76609640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:18.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c76609640 1 -- 172.21.15.115:0/3804456064 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7009fc70 con 0x7f5c7013dcf0 2024-01-24T23:23:18.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5c68002060 con 0x7f5c7013dcf0 2024-01-24T23:23:18.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:18.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5c68004010 con 0x7f5c7013dcf0 2024-01-24T23:23:18.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:18.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6e7fc640 4 set_mon_vals no callback set 2024-01-24T23:23:18.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient: dump: 2024-01-24T23:23:18.244 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.244 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.244 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.244 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.244 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.244 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.244 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.244 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.244 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.244 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.244 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.244 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.245 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.245 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.245 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.245 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f5c680041f0 con 0x7f5c7013dcf0 2024-01-24T23:23:18.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c76609640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:18.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c76609640 1 -- 172.21.15.115:0/3804456064 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5c500065d0 con 0x7f5c50002870 2024-01-24T23:23:18.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(50..50 src has 1..50) v4 ==== 4305+0+0 (unknown 1100805051 0 0) 0x7f5c68043cb0 con 0x7f5c7013dcf0 2024-01-24T23:23:18.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 941625673 0 0) 0x7f5c500065d0 con 0x7f5c50002870 2024-01-24T23:23:18.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient(hunting): my global_id is 84233 2024-01-24T23:23:18.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c700676c0 con 0x7f5c50002870 2024-01-24T23:23:18.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6f7fe640 10 monclient: get_auth_request con 0x7f5c5c0378b0 auth_method 0 2024-01-24T23:23:18.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2344131955 0 0) 0x7f5c700676c0 con 0x7f5c50002870 2024-01-24T23:23:18.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5c500065d0 con 0x7f5c50002870 2024-01-24T23:23:18.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5c600018c0 con 0x7f5c50002870 2024-01-24T23:23:18.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:18.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient: dump: 2024-01-24T23:23:18.246 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.246 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.246 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.246 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.246 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.246 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.246 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.246 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.247 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.247 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.247 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.247 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.247 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.247 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.247 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.247 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 417152032 0 0) 0x7f5c500065d0 con 0x7f5c50002870 2024-01-24T23:23:18.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 --> v1:172.21.15.115:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5c5000a120 con 0x7f5c50002870 2024-01-24T23:23:18.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 <== mon.2 v1:172.21.15.115:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f5c60003600 con 0x7f5c50002870 2024-01-24T23:23:18.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:18.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:18.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 >> v1:172.21.15.115:6790/0 conn(0x7f5c50002870 legacy=0x7f5c50004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.220+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 >> v1:172.21.15.115:6790/0 conn(0x7f5c50002870 legacy=0x7f5c50004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:18.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c76609640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:23:18.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c76609640 1 -- 172.21.15.115:0/3804456064 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5c50009df0 con 0x7f5c500090b0 2024-01-24T23:23:18.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 455618722 0 0) 0x7f5c50009df0 con 0x7f5c500090b0 2024-01-24T23:23:18.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c6cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c6cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c6cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c6cff9640 10 monclient(hunting): my global_id is 84242 2024-01-24T23:23:18.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c500065d0 con 0x7f5c500090b0 2024-01-24T23:23:18.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 270428463 0 0) 0x7f5c500065d0 con 0x7f5c500090b0 2024-01-24T23:23:18.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5c50009df0 con 0x7f5c500090b0 2024-01-24T23:23:18.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5c60003d20 con 0x7f5c500090b0 2024-01-24T23:23:18.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c6cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c6cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:18.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c6cff9640 10 monclient: dump: 2024-01-24T23:23:18.250 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.250 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.250 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.250 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.250 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.250 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.250 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.250 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.251 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1696359610 0 0) 0x7f5c50009df0 con 0x7f5c500090b0 2024-01-24T23:23:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 --> v1:172.21.15.115:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5c5000c310 con 0x7f5c500090b0 2024-01-24T23:23:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 <== mon.2 v1:172.21.15.115:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 4058525785) 0x7f5c60006040 con 0x7f5c500090b0 2024-01-24T23:23:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c6cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:23:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c6cff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 >> v1:172.21.15.115:6790/0 conn(0x7f5c500090b0 legacy=0x7f5c50009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c6cff9640 1 -- 172.21.15.115:0/3804456064 >> v1:172.21.15.115:6790/0 conn(0x7f5c500090b0 legacy=0x7f5c50009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:18.252 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:23:18.252 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:18.252 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:18.252 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:23:18.252 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 16, 2024-01-24T23:23:18.252 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:23:18.252 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:23:18.252 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:23:18.252 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:23:18.252 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:23:18.252 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:23:18.253 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:23:18.253 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:23:18.253 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:23:18.253 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:23:18.253 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.253 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 3, 2024-01-24T23:23:18.253 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:18.253 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:23:18.253 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:23:18.253 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:18.253 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:18.253 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:18.253 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:18.253 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:18.253 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:18.254 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:18.254 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:18.254 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:18.254 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:18.254 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:18.254 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.254 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:23:18.254 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:23:18.254 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:18.254 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:18.254 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:18.254 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:18.254 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:18.254 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:18.254 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:18.255 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:18.255 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:18.255 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:18.255 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:18.255 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.255 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:23:18.255 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:23:18.255 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:23:18.255 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:23:18.255 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:23:18.255 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:23:18.255 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:18.255 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:18.256 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:23:18.256 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:23:18.256 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:23:18.256 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:23:18.256 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:23:18.256 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:23:18.256 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:23:18.256 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:18.256 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:23:18.256 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:18.256 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:18.256 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:18.256 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:18.257 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:18.257 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:18.257 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:18.257 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:18.257 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:18.257 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:18.257 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:18.257 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.257 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:23:18.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.257 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:23:18.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.257 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:18.258 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:18.258 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.258 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.258 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.258 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:23:18.258 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.258 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.258 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.258 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:23:18.258 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:23:18.258 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.258 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.259 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.259 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:18.259 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:18.259 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.259 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.259 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.259 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:23:18.259 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.259 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.259 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.259 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:23:18.260 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:23:18.260 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.260 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.260 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.260 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:18.260 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:18.260 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.260 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.260 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.260 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:23:18.261 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.261 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.261 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.261 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:23:18.261 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:23:18.261 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.261 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.261 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.263 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:18.263 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:18.263 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.263 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.263 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.263 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:23:18.263 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.263 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.263 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.263 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:23:18.264 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:23:18.264 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.264 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.264 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.264 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:18.264 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:18.264 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.264 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.264 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.265 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:23:18.265 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.265 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.265 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.265 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:23:18.265 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:23:18.265 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.265 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.265 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.265 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:18.265 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:18.266 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.266 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.266 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.266 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:23:18.266 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.266 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.266 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.266 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:23:18.266 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:23:18.266 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.266 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.266 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.267 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:18.267 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:18.267 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.267 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.267 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.267 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:23:18.267 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.267 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.267 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.267 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:23:18.267 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:23:18.267 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.268 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.268 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.268 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:18.268 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:18.268 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.268 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.268 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.268 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:23:18.268 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.268 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.269 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.269 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:23:18.269 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:23:18.269 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.269 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.269 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.269 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:18.269 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:18.269 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.269 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.270 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.270 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:23:18.270 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.270 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.270 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.270 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:23:18.270 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:23:18.270 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.270 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.270 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.270 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.270 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.270 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:23:18.271 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:23:18.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.271 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:18.271 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:18.271 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:18.271 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.271 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.271 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:23:18.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.271 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:18.271 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:18.271 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:23:18.271 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.271 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.272 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:23:18.272 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:23:18.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c76609640 1 -- 172.21.15.115:0/3804456064 >> v1:172.21.15.132:6812/1835401922 conn(0x7f5c5c0378b0 legacy=0x7f5c5c039d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c76609640 10 monclient: shutdown 2024-01-24T23:23:18.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c76609640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:18.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c74ba6640 1 -- 172.21.15.115:0/3804456064 reap_dead start 2024-01-24T23:23:18.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c76609640 1 -- 172.21.15.115:0/3804456064 >> v1:172.21.15.115:6790/0 conn(0x7f5c7013dcf0 legacy=0x7f5c701d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c76609640 1 -- 172.21.15.115:0/3804456064 shutdown_connections 2024-01-24T23:23:18.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c76609640 1 -- 172.21.15.115:0/3804456064 >> 172.21.15.115:0/3804456064 conn(0x7f5c700a65e0 msgr2=0x7f5c700a1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:18.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c76609640 1 -- 172.21.15.115:0/3804456064 shutdown_connections 2024-01-24T23:23:18.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.228+0000 7f5c76609640 1 -- 172.21.15.115:0/3804456064 wait complete. 2024-01-24T23:23:18.274 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:23:18.274 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-01-24T23:23:18.274 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-01-24T23:23:18.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.384+0000 7f73a4677640 10 monclient: get_monmap_and_config 2024-01-24T23:23:18.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.384+0000 7f73a4677640 10 monclient: build_initial_monmap 2024-01-24T23:23:18.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.384+0000 7f73a4677640 10 monclient: monmap: 2024-01-24T23:23:18.390 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:18.390 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.390 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:18.390578+0000 2024-01-24T23:23:18.390 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:18.390578+0000 2024-01-24T23:23:18.390 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:18.390 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.391 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:18.391 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:18.391 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:18.391 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:18.391 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:18.391 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:18.391 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:18.391 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:18.391 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:18.391 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 1 Processor -- start 2024-01-24T23:23:18.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 1 -- start start 2024-01-24T23:23:18.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 10 monclient: init 2024-01-24T23:23:18.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:18.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 10 monclient: _add_conns ranks=[6,4,1,3,7,5,0,8,2] 2024-01-24T23:23:18.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 10 monclient(hunting): picked mon.noname-d con 0x7f739c136f20 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:18.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 10 monclient(hunting): picked mon.noname-c con 0x7f739c139ca0 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:18.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 10 monclient(hunting): picked mon.noname-f con 0x7f739c13d7b0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:18.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f739c04ed70 con 0x7f739c13d7b0 2024-01-24T23:23:18.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f739c0a23f0 con 0x7f739c139ca0 2024-01-24T23:23:18.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f739c0a16c0 con 0x7f739c136f20 2024-01-24T23:23:18.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:18.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 10 monclient(hunting): authenticate will time out at 1077.313203s 2024-01-24T23:23:18.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a2413640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f739c136f20 0x7f739c137300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:34212/0 (socket says 172.21.15.115:34212) 2024-01-24T23:23:18.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a2413640 1 -- 172.21.15.115:0/977634678 learned_addr learned my addr 172.21.15.115:0/977634678 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:18.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1c12640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f739c139ca0 0x7f739c13c090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:48622/0 (socket says 172.21.15.115:48622) 2024-01-24T23:23:18.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 1 -- 172.21.15.115:0/977634678 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3903133611 0 0) 0x7f739c0a23f0 con 0x7f739c139ca0 2024-01-24T23:23:18.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 10 monclient(hunting): my global_id is 74974 2024-01-24T23:23:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 1 -- 172.21.15.115:0/977634678 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7388003530 con 0x7f739c139ca0 2024-01-24T23:23:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 1 -- 172.21.15.115:0/977634678 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 479908174 0 0) 0x7f739c0a16c0 con 0x7f739c136f20 2024-01-24T23:23:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 10 monclient(hunting): my global_id is 5046 2024-01-24T23:23:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 1 -- 172.21.15.115:0/977634678 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f739c0a23f0 con 0x7f739c136f20 2024-01-24T23:23:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 1 -- 172.21.15.115:0/977634678 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693772573 0 0) 0x7f739c04ed70 con 0x7f739c13d7b0 2024-01-24T23:23:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 10 monclient(hunting): my global_id is 5113 2024-01-24T23:23:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 1 -- 172.21.15.115:0/977634678 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f739c0a16c0 con 0x7f739c13d7b0 2024-01-24T23:23:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 1 -- 172.21.15.115:0/977634678 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1187900857 0 0) 0x7f7388003530 con 0x7f739c139ca0 2024-01-24T23:23:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 1 -- 172.21.15.115:0/977634678 >> v1:172.21.15.115:6792/0 conn(0x7f739c136f20 legacy=0x7f739c137300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 1 -- 172.21.15.115:0/977634678 >> v1:172.21.15.132:6789/0 conn(0x7f739c13d7b0 legacy=0x7f739c13fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 10 monclient: _finish_hunting 0 2024-01-24T23:23:18.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 1 monclient: found mon.noname-c 2024-01-24T23:23:18.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:18.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:23:18.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 1 -- 172.21.15.115:0/977634678 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c0a0800 con 0x7f739c139ca0 2024-01-24T23:23:18.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 10 monclient: _finish_auth 0 2024-01-24T23:23:18.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 10 monclient: _check_auth_tickets 2024-01-24T23:23:18.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:18.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 1 -- 172.21.15.115:0/977634678 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f73900017c0 con 0x7f739c139ca0 2024-01-24T23:23:18.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:23:18.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 10 monclient: dump: 2024-01-24T23:23:18.402 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.402 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.403 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.403 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.403 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.403 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.403 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.403 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.403 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.403 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.403 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.403 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.403 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.403 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.403 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.403 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 5 monclient: authenticate success, global_id 74974 2024-01-24T23:23:18.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:18.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 1 -- 172.21.15.115:0/977634678 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7390001aa0 con 0x7f739c139ca0 2024-01-24T23:23:18.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:18.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 1 -- 172.21.15.115:0/977634678 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7390004b30 con 0x7f739c139ca0 2024-01-24T23:23:18.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:18.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 4 set_mon_vals no callback set 2024-01-24T23:23:18.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:23:18.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a1411640 10 monclient: dump: 2024-01-24T23:23:18.404 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.404 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.404 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.404 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.404 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.405 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.405 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.405 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.405 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.405 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.405 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.405 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.405 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.405 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.405 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.405 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 10 monclient: shutdown 2024-01-24T23:23:18.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:18.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 1 -- 172.21.15.115:0/977634678 >> v1:172.21.15.115:6791/0 conn(0x7f739c139ca0 legacy=0x7f739c13c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 1 -- 172.21.15.115:0/977634678 shutdown_connections 2024-01-24T23:23:18.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 1 -- 172.21.15.115:0/977634678 >> 172.21.15.115:0/977634678 conn(0x7f739c0a5730 msgr2=0x7f739c0a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:18.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 1 -- 172.21.15.115:0/977634678 shutdown_connections 2024-01-24T23:23:18.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 1 -- 172.21.15.115:0/977634678 wait complete. 2024-01-24T23:23:18.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 10 monclient: build_initial_monmap 2024-01-24T23:23:18.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 10 monclient: monmap: 2024-01-24T23:23:18.406 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:18.406 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:18.406 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:18.406 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:18.406 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:18.406 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:18.406 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:18.406 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:18.406 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:18.406 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:18.407 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:18.407 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:18.407 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:18.407 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:18.407 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:18.407 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 1 Processor -- start 2024-01-24T23:23:18.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 1 -- start start 2024-01-24T23:23:18.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.388+0000 7f73a4677640 10 monclient: init 2024-01-24T23:23:18.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f73a4677640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:18.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f73a4677640 10 monclient: _add_conns ranks=[8,4,1,2,3,6,0,7,5] 2024-01-24T23:23:18.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f73a4677640 10 monclient(hunting): picked mon.noname-e con 0x7f739c136f20 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:18.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f73a4677640 10 monclient(hunting): picked mon.noname-c con 0x7f739c139ca0 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:18.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f73a4677640 10 monclient(hunting): picked mon.noname-f con 0x7f739c13d7b0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:18.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f73a4677640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f739c09be00 con 0x7f739c13d7b0 2024-01-24T23:23:18.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f73a4677640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f739c13c160 con 0x7f739c139ca0 2024-01-24T23:23:18.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f73a4677640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f739c13a900 con 0x7f739c136f20 2024-01-24T23:23:18.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f73a4677640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:18.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f73a4677640 10 monclient(hunting): authenticate will time out at 1077.316502s 2024-01-24T23:23:18.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f73a1c12640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f739c139ca0 0x7f739c1324a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:48634/0 (socket says 172.21.15.115:48634) 2024-01-24T23:23:18.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f73a2413640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f739c136f20 0x7f739c131d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:52974/0 (socket says 172.21.15.115:52974) 2024-01-24T23:23:18.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f73a1c12640 1 -- 172.21.15.115:0/2750396241 learned_addr learned my addr 172.21.15.115:0/2750396241 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:18.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3592177099 0 0) 0x7f739c13a900 con 0x7f739c136f20 2024-01-24T23:23:18.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient(hunting): my global_id is 84230 2024-01-24T23:23:18.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7380003530 con 0x7f739c136f20 2024-01-24T23:23:18.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1777063564 0 0) 0x7f739c13c160 con 0x7f739c139ca0 2024-01-24T23:23:18.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient(hunting): my global_id is 74983 2024-01-24T23:23:18.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f739c13a900 con 0x7f739c139ca0 2024-01-24T23:23:18.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3574781583 0 0) 0x7f7380003530 con 0x7f739c136f20 2024-01-24T23:23:18.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f739c13c160 con 0x7f739c136f20 2024-01-24T23:23:18.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7398001a60 con 0x7f739c136f20 2024-01-24T23:23:18.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:23:18.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient(hunting): dump: 2024-01-24T23:23:18.409 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.409 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.409 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.409 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.410 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.410 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.410 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.410 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.410 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.410 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.410 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.410 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.410 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.410 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.410 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.410 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:18.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3897676616 0 0) 0x7f739c13a900 con 0x7f739c139ca0 2024-01-24T23:23:18.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7380003530 con 0x7f739c139ca0 2024-01-24T23:23:18.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7390001780 con 0x7f739c139ca0 2024-01-24T23:23:18.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:23:18.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient(hunting): dump: 2024-01-24T23:23:18.411 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.411 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.411 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.411 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.411 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.411 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.411 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.411 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.411 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.411 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.411 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.411 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.411 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.412 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.412 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.412 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 595670854 0 0) 0x7f739c13c160 con 0x7f739c136f20 2024-01-24T23:23:18.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 >> v1:172.21.15.115:6791/0 conn(0x7f739c139ca0 legacy=0x7f739c1324a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 >> v1:172.21.15.132:6789/0 conn(0x7f739c13d7b0 legacy=0x7f739c1d3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:18.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 monclient: found mon.e 2024-01-24T23:23:18.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:18.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:23:18.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c132bb0 con 0x7f739c136f20 2024-01-24T23:23:18.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient: _finish_auth 0 2024-01-24T23:23:18.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:18.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:18.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f73a4677640 5 monclient: authenticate success, global_id 84230 2024-01-24T23:23:18.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f73a4677640 10 monclient: _renew_subs 2024-01-24T23:23:18.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f73a4677640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:23:18.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f73a4677640 1 -- 172.21.15.115:0/2750396241 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f739c1335f0 con 0x7f739c136f20 2024-01-24T23:23:18.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f73a4677640 10 monclient: _renew_subs 2024-01-24T23:23:18.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f73a4677640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:23:18.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f73a4677640 1 -- 172.21.15.115:0/2750396241 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f739c13a200 con 0x7f739c136f20 2024-01-24T23:23:18.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7398001cd0 con 0x7f739c136f20 2024-01-24T23:23:18.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:18.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7398004de0 con 0x7f739c136f20 2024-01-24T23:23:18.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:18.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f73a0c10640 4 set_mon_vals no callback set 2024-01-24T23:23:18.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient: dump: 2024-01-24T23:23:18.413 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.414 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.414 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.414 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.414 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.414 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.414 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.414 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.414 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.414 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.414 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.414 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.414 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.414 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.414 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.414 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f73a4677640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:18.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f73a4677640 1 -- 172.21.15.115:0/2750396241 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f73840065d0 con 0x7f7384002870 2024-01-24T23:23:18.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f739800f860 con 0x7f739c136f20 2024-01-24T23:23:18.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(50..50 src has 1..50) v4 ==== 4305+0+0 (unknown 1100805051 0 0) 0x7f7398043970 con 0x7f739c136f20 2024-01-24T23:23:18.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3196024131 0 0) 0x7f73840065d0 con 0x7f7384002870 2024-01-24T23:23:18.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient(hunting): my global_id is 74992 2024-01-24T23:23:18.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f739c13c160 con 0x7f7384002870 2024-01-24T23:23:18.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4173753147 0 0) 0x7f739c13c160 con 0x7f7384002870 2024-01-24T23:23:18.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f73840065d0 con 0x7f7384002870 2024-01-24T23:23:18.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7390001c10 con 0x7f7384002870 2024-01-24T23:23:18.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:23:18.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient: dump: 2024-01-24T23:23:18.416 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.416 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.416 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.416 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.416 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.416 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.416 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.416 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.416 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.416 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.416 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.416 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.416 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.416 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.417 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.417 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f73a2c14640 10 monclient: get_auth_request con 0x7f7380037740 auth_method 0 2024-01-24T23:23:18.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1743670987 0 0) 0x7f73840065d0 con 0x7f7384002870 2024-01-24T23:23:18.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 --> v1:172.21.15.115:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f738400a120 con 0x7f7384002870 2024-01-24T23:23:18.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 <== mon.4 v1:172.21.15.115:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f7390005270 con 0x7f7384002870 2024-01-24T23:23:18.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:18.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:18.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 >> v1:172.21.15.115:6791/0 conn(0x7f7384002870 legacy=0x7f7384004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.392+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 >> v1:172.21.15.115:6791/0 conn(0x7f7384002870 legacy=0x7f7384004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:18.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.400+0000 7f73a4677640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:23:18.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.400+0000 7f73a4677640 1 -- 172.21.15.115:0/2750396241 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7384009e10 con 0x7f73840090d0 2024-01-24T23:23:18.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.408+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351605127 0 0) 0x7f7384009e10 con 0x7f73840090d0 2024-01-24T23:23:18.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.408+0000 7f738effd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.408+0000 7f738effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.408+0000 7f738effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.408+0000 7f738effd640 10 monclient(hunting): my global_id is 75001 2024-01-24T23:23:18.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.408+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73840065d0 con 0x7f73840090d0 2024-01-24T23:23:18.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.424+0000 7fe3867fc640 1 -- 172.21.15.132:0/3302017971 <== mon.1 v1:172.21.15.132:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}]=0 pool 'ec-foo' created v51) v1 ==== 177+0+0 (unknown 3823289343 0 0) 0x7fe38c014650 con 0x7fe390099510 2024-01-24T23:23:18.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.424+0000 7fe3867fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}] 2024-01-24T23:23:18.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.424+0000 7fe3867fc640 10 monclient: _finish_command 2 = system:0 pool 'ec-foo' created 2024-01-24T23:23:18.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.424+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 139216481 0 0) 0x7f73840065d0 con 0x7f73840090d0 2024-01-24T23:23:18.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.424+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7384009e10 con 0x7f73840090d0 2024-01-24T23:23:18.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.424+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7390005a70 con 0x7f73840090d0 2024-01-24T23:23:18.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.424+0000 7f738effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.424+0000 7f738effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:23:18.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.424+0000 7f738effd640 10 monclient: dump: 2024-01-24T23:23:18.428 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.428 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.429 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.429 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.429 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.429 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.429 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.429 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.429 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.429 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.429 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.429 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.429 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.429 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.429 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.429 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.424+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 970478118 0 0) 0x7f7384009e10 con 0x7f73840090d0 2024-01-24T23:23:18.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.424+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 --> v1:172.21.15.115:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f738400c310 con 0x7f73840090d0 2024-01-24T23:23:18.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.424+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 <== mon.4 v1:172.21.15.115:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 2261376553) 0x7f7390008000 con 0x7f73840090d0 2024-01-24T23:23:18.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.424+0000 7f738effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:23:18.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.424+0000 7f738effd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:18.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.424+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 >> v1:172.21.15.115:6791/0 conn(0x7f73840090d0 legacy=0x7f7384009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.424+0000 7f738effd640 1 -- 172.21.15.115:0/2750396241 >> v1:172.21.15.115:6791/0 conn(0x7f73840090d0 legacy=0x7f7384009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:18.430 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:23:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:23:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 16, 2024-01-24T23:23:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:23:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:23:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:23:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:23:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:23:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:23:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:23:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:23:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:23:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:23:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 4, 2024-01-24T23:23:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:23:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:23:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:23:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:23:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.433 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:23:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:23:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:23:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:23:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:23:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:23:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:23:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:23:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:23:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:23:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:23:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:23:18.434 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:23:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:23:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.435 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:23:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:23:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:23:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:23:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:23:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:23:18.437 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:23:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:23:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.438 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:23:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:23:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:23:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.440 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:18.440 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:18.440 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.440 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.440 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.440 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:23:18.440 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.440 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.440 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.440 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:23:18.440 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:23:18.440 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:23:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:23:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:23:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:23:18.442 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:23:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:23:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.443 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:23:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:23:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:23:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.444 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:23:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:23:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:23:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.445 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:23:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:23:18.446 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:23:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:23:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:23:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.447 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:23:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:23:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.448 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.449 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:23:18.449 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:23:18.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.424+0000 7f73a4677640 1 -- 172.21.15.115:0/2750396241 >> v1:172.21.15.132:6812/1835401922 conn(0x7f7380037740 legacy=0x7f7380039c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.424+0000 7f73a4677640 10 monclient: shutdown 2024-01-24T23:23:18.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.424+0000 7f73a4677640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:18.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.424+0000 7f73a4677640 1 -- 172.21.15.115:0/2750396241 >> v1:172.21.15.115:6793/0 conn(0x7f739c136f20 legacy=0x7f739c131d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.424+0000 7f73a2c14640 1 -- 172.21.15.115:0/2750396241 reap_dead start 2024-01-24T23:23:18.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.424+0000 7f73a2c14640 1 -- 172.21.15.115:0/2750396241 reap_dead start 2024-01-24T23:23:18.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.424+0000 7f73a4677640 1 -- 172.21.15.115:0/2750396241 shutdown_connections 2024-01-24T23:23:18.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.424+0000 7f73a4677640 1 -- 172.21.15.115:0/2750396241 >> 172.21.15.115:0/2750396241 conn(0x7f739c0a5730 msgr2=0x7f739c0a34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:18.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.424+0000 7f73a4677640 1 -- 172.21.15.115:0/2750396241 shutdown_connections 2024-01-24T23:23:18.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.424+0000 7f73a4677640 1 -- 172.21.15.115:0/2750396241 wait complete. 2024-01-24T23:23:18.451 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:23:18.451 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-01-24T23:23:18.451 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-01-24T23:23:18.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.520+0000 7fe397ce1640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}] 2024-01-24T23:23:18.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.520+0000 7fe397ce1640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}] 2024-01-24T23:23:18.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.520+0000 7fe397ce1640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:23:18.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.520+0000 7fe397ce1640 1 -- 172.21.15.132:0/3302017971 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"} v 0) v1 -- 0x7fe378005ab0 con 0x7fe390099510 2024-01-24T23:23:18.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.520+0000 7fe3867fc640 1 -- 172.21.15.132:0/3302017971 <== mon.1 v1:172.21.15.132:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}]=0 pool 'ec-foo' already exists v51) v1 ==== 184+0+0 (unknown 734906055 0 0) 0x7fe38c014830 con 0x7fe390099510 2024-01-24T23:23:18.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.524+0000 7fe3867fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}] 2024-01-24T23:23:18.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.524+0000 7fe3867fc640 10 monclient: _finish_command 3 = system:0 pool 'ec-foo' already exists 2024-01-24T23:23:18.523 INFO:tasks.workunit.client.0.smithi132.stderr:pool 'ec-foo' already exists 2024-01-24T23:23:18.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.524+0000 7fe397ce1640 1 -- 172.21.15.132:0/3302017971 >> v1:172.21.15.132:6812/1835401922 conn(0x7fe3740371e0 legacy=0x7fe3740396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.524+0000 7fe397ce1640 10 monclient: shutdown 2024-01-24T23:23:18.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.524+0000 7fe397ce1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:18.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.524+0000 7fe397ce1640 1 -- 172.21.15.132:0/3302017971 >> v1:172.21.15.132:6789/0 conn(0x7fe390099510 legacy=0x7fe390071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.524+0000 7fe397ce1640 1 -- 172.21.15.132:0/3302017971 shutdown_connections 2024-01-24T23:23:18.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.524+0000 7fe397ce1640 1 -- 172.21.15.132:0/3302017971 >> 172.21.15.132:0/3302017971 conn(0x7fe3900a2c40 msgr2=0x7fe3901ba770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:18.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.524+0000 7fe397ce1640 1 -- 172.21.15.132:0/3302017971 shutdown_connections 2024-01-24T23:23:18.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.524+0000 7fe397ce1640 1 -- 172.21.15.132:0/3302017971 wait complete. 2024-01-24T23:23:18.540 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool rm ec-foo ec-foo --yes-i-really-really-mean-it 2024-01-24T23:23:18.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.556+0000 7f4489982640 10 monclient: get_monmap_and_config 2024-01-24T23:23:18.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.556+0000 7f4489982640 10 monclient: build_initial_monmap 2024-01-24T23:23:18.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.556+0000 7f4489982640 10 monclient: monmap: 2024-01-24T23:23:18.562 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:18.562 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.562 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:18.562858+0000 2024-01-24T23:23:18.562 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:18.562858+0000 2024-01-24T23:23:18.562 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:18.562 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.563 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:18.563 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:18.563 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:18.563 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:18.563 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:18.563 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:18.563 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:18.563 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:18.563 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:18.563 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 1 Processor -- start 2024-01-24T23:23:18.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 1 -- start start 2024-01-24T23:23:18.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 10 monclient: init 2024-01-24T23:23:18.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:18.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 10 monclient: _add_conns ranks=[4,3,6,0,7,8,5,1,2] 2024-01-24T23:23:18.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 10 monclient(hunting): picked mon.noname-c con 0x7f4484133bf0 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:18.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 10 monclient(hunting): picked mon.noname-g con 0x7f4484133fd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:18.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 10 monclient(hunting): picked mon.noname-d con 0x7f44841343b0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:18.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f448404ed70 con 0x7f4484133fd0 2024-01-24T23:23:18.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44840a4b60 con 0x7f4484133bf0 2024-01-24T23:23:18.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f448409ae50 con 0x7f44841343b0 2024-01-24T23:23:18.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:18.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 10 monclient(hunting): authenticate will time out at 1077.485430s 2024-01-24T23:23:18.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44837fe640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f44841343b0 0x7f448413f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:34218/0 (socket says 172.21.15.115:34218) 2024-01-24T23:23:18.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4482ffd640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f4484133bf0 0x7f448409bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:48670/0 (socket says 172.21.15.115:48670) 2024-01-24T23:23:18.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44837fe640 1 -- 172.21.15.115:0/3482876352 learned_addr learned my addr 172.21.15.115:0/3482876352 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:18.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 1 -- 172.21.15.115:0/3482876352 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 613206688 0 0) 0x7f448409ae50 con 0x7f44841343b0 2024-01-24T23:23:18.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 10 monclient(hunting): my global_id is 5055 2024-01-24T23:23:18.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 1 -- 172.21.15.115:0/3482876352 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4470003530 con 0x7f44841343b0 2024-01-24T23:23:18.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 1 -- 172.21.15.115:0/3482876352 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2162525949 0 0) 0x7f44840a4b60 con 0x7f4484133bf0 2024-01-24T23:23:18.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 10 monclient(hunting): my global_id is 75010 2024-01-24T23:23:18.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 1 -- 172.21.15.115:0/3482876352 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f448409ae50 con 0x7f4484133bf0 2024-01-24T23:23:18.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 1 -- 172.21.15.115:0/3482876352 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1063923242 0 0) 0x7f448404ed70 con 0x7f4484133fd0 2024-01-24T23:23:18.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 10 monclient(hunting): my global_id is 4971 2024-01-24T23:23:18.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 1 -- 172.21.15.115:0/3482876352 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44840a4b60 con 0x7f4484133fd0 2024-01-24T23:23:18.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 1 -- 172.21.15.115:0/3482876352 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 742603888 0 0) 0x7f4470003530 con 0x7f44841343b0 2024-01-24T23:23:18.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 1 -- 172.21.15.115:0/3482876352 >> v1:172.21.15.115:6791/0 conn(0x7f4484133bf0 legacy=0x7f448409bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 1 -- 172.21.15.115:0/3482876352 >> v1:172.21.15.132:6790/0 conn(0x7f4484133fd0 legacy=0x7f448413bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 10 monclient: _finish_hunting 0 2024-01-24T23:23:18.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 1 monclient: found mon.noname-d 2024-01-24T23:23:18.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:18.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:23:18.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 1 -- 172.21.15.115:0/3482876352 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448409b130 con 0x7f44841343b0 2024-01-24T23:23:18.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 10 monclient: _finish_auth 0 2024-01-24T23:23:18.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 10 monclient: _check_auth_tickets 2024-01-24T23:23:18.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:18.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 5 monclient: authenticate success, global_id 5055 2024-01-24T23:23:18.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 1 -- 172.21.15.115:0/3482876352 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f447c001b40 con 0x7f44841343b0 2024-01-24T23:23:18.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:18.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:23:18.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 10 monclient: dump: 2024-01-24T23:23:18.575 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.575 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.575 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.575 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.576 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.576 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.576 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.576 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.576 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.576 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.576 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.576 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.576 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.576 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.576 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.576 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:18.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 1 -- 172.21.15.115:0/3482876352 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f447c001e20 con 0x7f44841343b0 2024-01-24T23:23:18.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:18.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:18.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 4 set_mon_vals no callback set 2024-01-24T23:23:18.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 1 -- 172.21.15.115:0/3482876352 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f447c004f70 con 0x7f44841343b0 2024-01-24T23:23:18.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:18.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f44827fc640 10 monclient: dump: 2024-01-24T23:23:18.577 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.577 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.577 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.577 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.577 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.577 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.577 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.577 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.578 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.578 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.578 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.578 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.578 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.578 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.578 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.578 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 10 monclient: shutdown 2024-01-24T23:23:18.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:18.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 1 -- 172.21.15.115:0/3482876352 >> v1:172.21.15.115:6792/0 conn(0x7f44841343b0 legacy=0x7f448413f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 1 -- 172.21.15.115:0/3482876352 shutdown_connections 2024-01-24T23:23:18.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 1 -- 172.21.15.115:0/3482876352 >> 172.21.15.115:0/3482876352 conn(0x7f44840662f0 msgr2=0x7f4484066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:18.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 1 -- 172.21.15.115:0/3482876352 shutdown_connections 2024-01-24T23:23:18.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 1 -- 172.21.15.115:0/3482876352 wait complete. 2024-01-24T23:23:18.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 10 monclient: build_initial_monmap 2024-01-24T23:23:18.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 10 monclient: monmap: 2024-01-24T23:23:18.579 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:18.579 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:18.579 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:18.579 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:18.579 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:18.579 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:18.579 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:18.579 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:18.579 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:18.579 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:18.579 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:18.579 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:18.579 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:18.579 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:18.580 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:18.580 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 1 Processor -- start 2024-01-24T23:23:18.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 1 -- start start 2024-01-24T23:23:18.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.560+0000 7f4489982640 10 monclient: init 2024-01-24T23:23:18.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4489982640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:18.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4489982640 10 monclient: _add_conns ranks=[0,3,2,7,4,5,1,8,6] 2024-01-24T23:23:18.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4489982640 10 monclient(hunting): picked mon.noname-a con 0x7f4484133bf0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:18.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4489982640 10 monclient(hunting): picked mon.noname-g con 0x7f4484133fd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:18.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4489982640 10 monclient(hunting): picked mon.noname-b con 0x7f44841343b0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:18.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4489982640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44840a5990 con 0x7f4484133bf0 2024-01-24T23:23:18.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4489982640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f448409c810 con 0x7f44841343b0 2024-01-24T23:23:18.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4489982640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44840a2760 con 0x7f4484133fd0 2024-01-24T23:23:18.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4489982640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:18.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4489982640 10 monclient(hunting): authenticate will time out at 1077.488468s 2024-01-24T23:23:18.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4482ffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f4484133bf0 0x7f44840a0dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:32772/0 (socket says 172.21.15.115:32772) 2024-01-24T23:23:18.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4482ffd640 1 -- 172.21.15.115:0/2816865672 learned_addr learned my addr 172.21.15.115:0/2816865672 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:18.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1812731878 0 0) 0x7f44840a5990 con 0x7f4484133bf0 2024-01-24T23:23:18.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient(hunting): my global_id is 5022 2024-01-24T23:23:18.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4464003530 con 0x7f4484133bf0 2024-01-24T23:23:18.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1562724250 0 0) 0x7f448409c810 con 0x7f44841343b0 2024-01-24T23:23:18.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient(hunting): my global_id is 84251 2024-01-24T23:23:18.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44840a5990 con 0x7f44841343b0 2024-01-24T23:23:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2162956398 0 0) 0x7f44840a2760 con 0x7f4484133fd0 2024-01-24T23:23:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient(hunting): my global_id is 4980 2024-01-24T23:23:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f448409c810 con 0x7f4484133fd0 2024-01-24T23:23:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1826417563 0 0) 0x7f4464003530 con 0x7f4484133bf0 2024-01-24T23:23:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f44840a2760 con 0x7f4484133bf0 2024-01-24T23:23:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 932504491 0 0) 0x7f44840a5990 con 0x7f44841343b0 2024-01-24T23:23:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4464003530 con 0x7f44841343b0 2024-01-24T23:23:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4474001460 con 0x7f4484133bf0 2024-01-24T23:23:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:23:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient(hunting): dump: 2024-01-24T23:23:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.583 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.583 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.583 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.583 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.583 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.583 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.583 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.583 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.583 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.583 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.583 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.583 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:18.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f447c004eb0 con 0x7f44841343b0 2024-01-24T23:23:18.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:18.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient(hunting): dump: 2024-01-24T23:23:18.584 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.584 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.584 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.584 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.584 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.584 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.584 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.584 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.584 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.584 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.584 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.585 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 786836389 0 0) 0x7f44840a2760 con 0x7f4484133bf0 2024-01-24T23:23:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 >> v1:172.21.15.132:6790/0 conn(0x7f4484133fd0 legacy=0x7f44840a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 >> v1:172.21.15.115:6790/0 conn(0x7f44841343b0 legacy=0x7f44841d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:23:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 monclient: found mon.a 2024-01-24T23:23:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:23:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4484133720 con 0x7f4484133bf0 2024-01-24T23:23:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient: _finish_auth 0 2024-01-24T23:23:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:18.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4489982640 5 monclient: authenticate success, global_id 5022 2024-01-24T23:23:18.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4489982640 10 monclient: _renew_subs 2024-01-24T23:23:18.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4489982640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:23:18.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4489982640 1 -- 172.21.15.115:0/2816865672 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448411ff00 con 0x7f4484133bf0 2024-01-24T23:23:18.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4489982640 10 monclient: _renew_subs 2024-01-24T23:23:18.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4489982640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:23:18.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4489982640 1 -- 172.21.15.115:0/2816865672 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f44840a1bf0 con 0x7f4484133bf0 2024-01-24T23:23:18.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4474001600 con 0x7f4484133bf0 2024-01-24T23:23:18.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:18.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4474004de0 con 0x7f4484133bf0 2024-01-24T23:23:18.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:23:18.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4481ffb640 4 set_mon_vals no callback set 2024-01-24T23:23:18.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient: dump: 2024-01-24T23:23:18.587 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.587 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.587 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.587 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.587 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.587 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.587 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.587 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.587 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.587 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.587 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.587 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.587 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.587 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.587 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.587 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4489982640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:18.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4489982640 1 -- 172.21.15.115:0/2816865672 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f44680065d0 con 0x7f4468002870 2024-01-24T23:23:18.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2897382243 0 0) 0x7f44680065d0 con 0x7f4468002870 2024-01-24T23:23:18.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient(hunting): my global_id is 5064 2024-01-24T23:23:18.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44840a2760 con 0x7f4468002870 2024-01-24T23:23:18.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f447400f860 con 0x7f4484133bf0 2024-01-24T23:23:18.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(51..51 src has 1..51) v4 ==== 4660+0+0 (unknown 1469567968 0 0) 0x7f4474043ad0 con 0x7f4484133bf0 2024-01-24T23:23:18.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1128804480 0 0) 0x7f44840a2760 con 0x7f4468002870 2024-01-24T23:23:18.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f44680065d0 con 0x7f4468002870 2024-01-24T23:23:18.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f446c003940 con 0x7f4468002870 2024-01-24T23:23:18.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:18.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient: dump: 2024-01-24T23:23:18.588 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.589 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 22318122 0 0) 0x7f44680065d0 con 0x7f4468002870 2024-01-24T23:23:18.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 --> v1:172.21.15.115:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f446800a120 con 0x7f4468002870 2024-01-24T23:23:18.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f44837fe640 10 monclient: get_auth_request con 0x7f44640377c0 auth_method 0 2024-01-24T23:23:18.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f446c005ae0 con 0x7f4468002870 2024-01-24T23:23:18.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:18.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:18.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 >> v1:172.21.15.115:6792/0 conn(0x7f4468002870 legacy=0x7f4468004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.564+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 >> v1:172.21.15.115:6792/0 conn(0x7f4468002870 legacy=0x7f4468004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:18.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4489982640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:23:18.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4489982640 1 -- 172.21.15.115:0/2816865672 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4468009e10 con 0x7f44680090d0 2024-01-24T23:23:18.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 467279859 0 0) 0x7f4468009e10 con 0x7f44680090d0 2024-01-24T23:23:18.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4480ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4480ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4480ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4480ff9640 10 monclient(hunting): my global_id is 5073 2024-01-24T23:23:18.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44680065d0 con 0x7f44680090d0 2024-01-24T23:23:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1896646856 0 0) 0x7f44680065d0 con 0x7f44680090d0 2024-01-24T23:23:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4468009e10 con 0x7f44680090d0 2024-01-24T23:23:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f446c004440 con 0x7f44680090d0 2024-01-24T23:23:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4480ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4480ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4480ff9640 10 monclient: dump: 2024-01-24T23:23:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.592 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.593 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2384835335 0 0) 0x7f4468009e10 con 0x7f44680090d0 2024-01-24T23:23:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 --> v1:172.21.15.115:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f446800c310 con 0x7f44680090d0 2024-01-24T23:23:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 2475859758) 0x7f446c004670 con 0x7f44680090d0 2024-01-24T23:23:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4480ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:23:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4480ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 >> v1:172.21.15.115:6792/0 conn(0x7f44680090d0 legacy=0x7f4468009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4480ff9640 1 -- 172.21.15.115:0/2816865672 >> v1:172.21.15.115:6792/0 conn(0x7f44680090d0 legacy=0x7f4468009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:18.594 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:23:18.594 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:18.594 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:18.594 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:23:18.594 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 16, 2024-01-24T23:23:18.594 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:23:18.594 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:23:18.594 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:23:18.594 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:23:18.594 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:23:18.594 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:23:18.595 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:23:18.595 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:23:18.595 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:23:18.595 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:23:18.595 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.595 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 4, 2024-01-24T23:23:18.595 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:18.595 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:23:18.595 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:23:18.595 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:18.595 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:18.595 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:18.595 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:18.595 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:18.596 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:18.596 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:18.596 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:18.596 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:18.596 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:18.596 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:18.596 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.596 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:23:18.596 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:23:18.596 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:18.596 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:18.596 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:18.596 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:18.596 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:18.597 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:18.597 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:18.597 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:18.597 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:18.597 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:18.597 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:18.597 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.597 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:23:18.597 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:23:18.597 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:23:18.597 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:23:18.597 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:23:18.597 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:23:18.598 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:18.598 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:18.598 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:23:18.598 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:23:18.598 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:23:18.598 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:23:18.598 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:23:18.598 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:23:18.598 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:23:18.598 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:18.598 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:23:18.598 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:18.598 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:18.598 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:18.599 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:18.599 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:18.599 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:18.599 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:18.599 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:18.599 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:18.599 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:18.599 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:18.599 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.599 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:23:18.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.599 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:23:18.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.599 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:18.599 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:18.600 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.600 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.600 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.600 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:23:18.600 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.600 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.600 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:23:18.600 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:23:18.600 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.600 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.600 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.601 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:18.601 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:18.601 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.601 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.601 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.601 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:23:18.601 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.601 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.601 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.601 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:23:18.601 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:23:18.602 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.602 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.602 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.602 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:18.602 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:18.602 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.602 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.602 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.602 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:23:18.602 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.602 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.602 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.602 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:23:18.603 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:23:18.603 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.603 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.603 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.603 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:18.603 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:18.603 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.603 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.603 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.603 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:23:18.603 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.603 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.603 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.604 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:23:18.604 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:23:18.604 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.604 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.604 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.604 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:18.604 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:18.604 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.604 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.604 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.604 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:23:18.605 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.605 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.605 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.605 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:23:18.605 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:23:18.605 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.605 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.605 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.605 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:18.605 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:18.605 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.605 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.606 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.606 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:23:18.606 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.606 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.606 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.606 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:23:18.606 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:23:18.606 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.606 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.606 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.606 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:18.606 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:18.607 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.607 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.607 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.607 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:23:18.607 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.607 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.607 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.607 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:23:18.607 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:23:18.607 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.607 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.607 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.608 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:18.608 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:18.608 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.608 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.608 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.608 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:23:18.608 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.608 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.608 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.608 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:23:18.608 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:23:18.609 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.609 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.609 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.609 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:18.609 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:18.609 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.609 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.609 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.609 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:23:18.609 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.609 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.609 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.609 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:23:18.610 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:23:18.610 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.610 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.610 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.610 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.610 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.610 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:23:18.610 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:23:18.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.610 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:18.610 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:18.610 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:18.610 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.610 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.610 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:23:18.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.611 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:18.611 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:18.611 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:18.611 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.611 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.611 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:23:18.611 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:23:18.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4489982640 1 -- 172.21.15.115:0/2816865672 >> v1:172.21.15.132:6812/1835401922 conn(0x7f44640377c0 legacy=0x7f4464039c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4489982640 10 monclient: shutdown 2024-01-24T23:23:18.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4489982640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:18.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4489982640 1 -- 172.21.15.115:0/2816865672 >> v1:172.21.15.115:6789/0 conn(0x7f4484133bf0 legacy=0x7f44840a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f44837fe640 1 -- 172.21.15.115:0/2816865672 reap_dead start 2024-01-24T23:23:18.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f44837fe640 1 -- 172.21.15.115:0/2816865672 reap_dead start 2024-01-24T23:23:18.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4489982640 1 -- 172.21.15.115:0/2816865672 shutdown_connections 2024-01-24T23:23:18.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4489982640 1 -- 172.21.15.115:0/2816865672 >> 172.21.15.115:0/2816865672 conn(0x7f44840662f0 msgr2=0x7f44841be870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:18.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4489982640 1 -- 172.21.15.115:0/2816865672 shutdown_connections 2024-01-24T23:23:18.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.572+0000 7f4489982640 1 -- 172.21.15.115:0/2816865672 wait complete. 2024-01-24T23:23:18.613 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:23:18.613 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-01-24T23:23:18.614 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-01-24T23:23:18.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 10 monclient: get_monmap_and_config 2024-01-24T23:23:18.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 10 monclient: build_initial_monmap 2024-01-24T23:23:18.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 10 monclient: monmap: 2024-01-24T23:23:18.663 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:18.664 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.664 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:18.665049+0000 2024-01-24T23:23:18.664 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:18.665049+0000 2024-01-24T23:23:18.664 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:18.664 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:18.664 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:18.664 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:18.664 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:18.664 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:18.664 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:18.664 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:18.664 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:18.664 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:18.664 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:18.664 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:18.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 1 Processor -- start 2024-01-24T23:23:18.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 1 -- start start 2024-01-24T23:23:18.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 10 monclient: init 2024-01-24T23:23:18.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:18.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 10 monclient: _add_conns ranks=[5,0,6,2,8,4,3,1,7] 2024-01-24T23:23:18.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 10 monclient(hunting): picked mon.noname-h con 0x7f7bbc135ea0 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:18.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 10 monclient(hunting): picked mon.noname-a con 0x7f7bbc09bcd0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:18.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 10 monclient(hunting): picked mon.noname-d con 0x7f7bbc09c0f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:18.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bbc064700 con 0x7f7bbc09bcd0 2024-01-24T23:23:18.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bbc136990 con 0x7f7bbc135ea0 2024-01-24T23:23:18.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bbc09b010 con 0x7f7bbc09c0f0 2024-01-24T23:23:18.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:18.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 10 monclient(hunting): authenticate will time out at 1085.989848s 2024-01-24T23:23:18.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc18b3640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f7bbc09c0f0 0x7f7bbc13f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:59416/0 (socket says 172.21.15.132:59416) 2024-01-24T23:23:18.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc08b1640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f7bbc09bcd0 0x7f7bbc13bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:38262/0 (socket says 172.21.15.132:38262) 2024-01-24T23:23:18.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc18b3640 1 -- 172.21.15.132:0/1445452560 learned_addr learned my addr 172.21.15.132:0/1445452560 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:18.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 1 -- 172.21.15.132:0/1445452560 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 91558132 0 0) 0x7f7bbc136990 con 0x7f7bbc135ea0 2024-01-24T23:23:18.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 10 monclient(hunting): my global_id is 5009 2024-01-24T23:23:18.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 1 -- 172.21.15.132:0/1445452560 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ba8003530 con 0x7f7bbc135ea0 2024-01-24T23:23:18.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 1 -- 172.21.15.132:0/1445452560 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701722858 0 0) 0x7f7bbc064700 con 0x7f7bbc09bcd0 2024-01-24T23:23:18.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 10 monclient(hunting): my global_id is 5031 2024-01-24T23:23:18.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 1 -- 172.21.15.132:0/1445452560 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bbc136990 con 0x7f7bbc09bcd0 2024-01-24T23:23:18.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 1 -- 172.21.15.132:0/1445452560 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2535337229 0 0) 0x7f7bbc09b010 con 0x7f7bbc09c0f0 2024-01-24T23:23:18.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 10 monclient(hunting): my global_id is 5082 2024-01-24T23:23:18.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 1 -- 172.21.15.132:0/1445452560 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bbc064700 con 0x7f7bbc09c0f0 2024-01-24T23:23:18.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 1 -- 172.21.15.132:0/1445452560 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1510017262 0 0) 0x7f7ba8003530 con 0x7f7bbc135ea0 2024-01-24T23:23:18.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 1 -- 172.21.15.132:0/1445452560 >> v1:172.21.15.115:6792/0 conn(0x7f7bbc09c0f0 legacy=0x7f7bbc13f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 1 -- 172.21.15.132:0/1445452560 >> v1:172.21.15.115:6789/0 conn(0x7f7bbc09bcd0 legacy=0x7f7bbc13bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 10 monclient: _finish_hunting 0 2024-01-24T23:23:18.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 1 monclient: found mon.noname-h 2024-01-24T23:23:18.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:18.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:23:18.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 1 -- 172.21.15.132:0/1445452560 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bbc0a4f50 con 0x7f7bbc135ea0 2024-01-24T23:23:18.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 10 monclient: _finish_auth 0 2024-01-24T23:23:18.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 10 monclient: _check_auth_tickets 2024-01-24T23:23:18.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:18.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 1 -- 172.21.15.132:0/1445452560 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7bb80017c0 con 0x7f7bbc135ea0 2024-01-24T23:23:18.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 5 monclient: authenticate success, global_id 5009 2024-01-24T23:23:18.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:18.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:18.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 10 monclient: dump: 2024-01-24T23:23:18.676 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:18.676 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.676 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.676 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.676 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.676 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:18.676 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.677 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.677 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.677 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.677 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.677 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.677 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.677 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.677 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.677 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:18.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:18.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 1 -- 172.21.15.132:0/1445452560 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7bb8001aa0 con 0x7f7bbc135ea0 2024-01-24T23:23:18.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:18.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:18.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 4 set_mon_vals no callback set 2024-01-24T23:23:18.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 1 -- 172.21.15.132:0/1445452560 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7bb8003960 con 0x7f7bbc135ea0 2024-01-24T23:23:18.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:18.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bb3fff640 10 monclient: dump: 2024-01-24T23:23:18.678 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:18.678 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.678 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.678 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.678 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.678 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:18.678 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.678 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.678 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.678 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.678 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.678 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.678 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.678 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.679 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.679 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:18.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 10 monclient: shutdown 2024-01-24T23:23:18.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:18.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 1 -- 172.21.15.132:0/1445452560 >> v1:172.21.15.132:6791/0 conn(0x7f7bbc135ea0 legacy=0x7f7bbc136280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 1 -- 172.21.15.132:0/1445452560 shutdown_connections 2024-01-24T23:23:18.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 1 -- 172.21.15.132:0/1445452560 >> 172.21.15.132:0/1445452560 conn(0x7f7bbc0a9bf0 msgr2=0x7f7bbc0a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:18.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 1 -- 172.21.15.132:0/1445452560 shutdown_connections 2024-01-24T23:23:18.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.664+0000 7f7bc3316640 1 -- 172.21.15.132:0/1445452560 wait complete. 2024-01-24T23:23:18.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 10 monclient: build_initial_monmap 2024-01-24T23:23:18.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 10 monclient: monmap: 2024-01-24T23:23:18.679 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:18.679 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:18.679 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:18.680 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:18.680 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:18.680 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:18.680 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:18.680 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:18.680 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:18.680 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:18.680 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:18.680 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:18.680 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:18.680 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:18.680 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:18.680 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:18.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 1 Processor -- start 2024-01-24T23:23:18.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 1 -- start start 2024-01-24T23:23:18.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 10 monclient: init 2024-01-24T23:23:18.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:18.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 10 monclient: _add_conns ranks=[8,4,0,2,7,6,3,5,1] 2024-01-24T23:23:18.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 10 monclient(hunting): picked mon.noname-e con 0x7f7bbc09bcd0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:18.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 10 monclient(hunting): picked mon.noname-c con 0x7f7bbc09c0f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:18.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 10 monclient(hunting): picked mon.noname-a con 0x7f7bbc135ea0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:18.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bbc1c8680 con 0x7f7bbc135ea0 2024-01-24T23:23:18.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bbc09c6b0 con 0x7f7bbc09c0f0 2024-01-24T23:23:18.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bbc135c70 con 0x7f7bbc09bcd0 2024-01-24T23:23:18.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:18.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 10 monclient(hunting): authenticate will time out at 1085.993363s 2024-01-24T23:23:18.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc18b3640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f7bbc135ea0 0x7f7bbc1d4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:38272/0 (socket says 172.21.15.132:38272) 2024-01-24T23:23:18.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc18b3640 1 -- 172.21.15.132:0/1344075490 learned_addr learned my addr 172.21.15.132:0/1344075490 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:18.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1022101385 0 0) 0x7f7bbc09c6b0 con 0x7f7bbc09c0f0 2024-01-24T23:23:18.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient(hunting): my global_id is 75019 2024-01-24T23:23:18.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ba0003530 con 0x7f7bbc09c0f0 2024-01-24T23:23:18.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2564832970 0 0) 0x7f7bbc1c8680 con 0x7f7bbc135ea0 2024-01-24T23:23:18.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient(hunting): my global_id is 5040 2024-01-24T23:23:18.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bbc09c6b0 con 0x7f7bbc135ea0 2024-01-24T23:23:18.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3463598685 0 0) 0x7f7bbc135c70 con 0x7f7bbc09bcd0 2024-01-24T23:23:18.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient(hunting): my global_id is 84239 2024-01-24T23:23:18.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bbc1c8680 con 0x7f7bbc09bcd0 2024-01-24T23:23:18.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3437375871 0 0) 0x7f7ba0003530 con 0x7f7bbc09c0f0 2024-01-24T23:23:18.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7bbc135c70 con 0x7f7bbc09c0f0 2024-01-24T23:23:18.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 130001698 0 0) 0x7f7bbc09c6b0 con 0x7f7bbc135ea0 2024-01-24T23:23:18.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ba0003530 con 0x7f7bbc135ea0 2024-01-24T23:23:18.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1968854722 0 0) 0x7f7bbc1c8680 con 0x7f7bbc09bcd0 2024-01-24T23:23:18.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7bbc09c6b0 con 0x7f7bbc09bcd0 2024-01-24T23:23:18.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7bac001a60 con 0x7f7bbc09c0f0 2024-01-24T23:23:18.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:23:18.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient(hunting): dump: 2024-01-24T23:23:18.683 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:18.683 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.683 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.683 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.683 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.683 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:18.684 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.684 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.684 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.684 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.684 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.684 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.684 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.684 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.684 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.684 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:18.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:18.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7bb4001da0 con 0x7f7bbc135ea0 2024-01-24T23:23:18.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:23:18.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient(hunting): dump: 2024-01-24T23:23:18.685 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:18.685 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.685 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.685 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.685 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.685 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:18.685 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.685 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.685 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.685 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.685 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.685 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.685 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.685 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.685 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.686 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:18.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7bb80010b0 con 0x7f7bbc09bcd0 2024-01-24T23:23:18.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:18.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient(hunting): dump: 2024-01-24T23:23:18.686 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:18.686 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.686 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.686 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.686 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.686 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:18.686 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.686 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.686 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.687 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.687 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.687 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.687 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.687 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.687 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.687 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:18.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2649788200 0 0) 0x7f7bbc135c70 con 0x7f7bbc09c0f0 2024-01-24T23:23:18.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 >> v1:172.21.15.115:6793/0 conn(0x7f7bbc09bcd0 legacy=0x7f7bbc135160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 >> v1:172.21.15.115:6789/0 conn(0x7f7bbc135ea0 legacy=0x7f7bbc1d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:23:18.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 1 monclient: found mon.c 2024-01-24T23:23:18.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:18.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:23:18.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bbc1210e0 con 0x7f7bbc09c0f0 2024-01-24T23:23:18.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient: _finish_auth 0 2024-01-24T23:23:18.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:23:18.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:18.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 5 monclient: authenticate success, global_id 75019 2024-01-24T23:23:18.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 10 monclient: _renew_subs 2024-01-24T23:23:18.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:23:18.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 1 -- 172.21.15.132:0/1344075490 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bbc122b20 con 0x7f7bbc09c0f0 2024-01-24T23:23:18.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 10 monclient: _renew_subs 2024-01-24T23:23:18.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:23:18.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 1 -- 172.21.15.132:0/1344075490 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bbc135870 con 0x7f7bbc09c0f0 2024-01-24T23:23:18.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7bac001d70 con 0x7f7bbc09c0f0 2024-01-24T23:23:18.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:18.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7bac004de0 con 0x7f7bbc09c0f0 2024-01-24T23:23:18.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.689 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:23:18.689 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb37fe640 4 set_mon_vals no callback set 2024-01-24T23:23:18.689 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 10 monclient: dump: 2024-01-24T23:23:18.689 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:18.689 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.689 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.689 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.689 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.689 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:18.689 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.689 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.689 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.689 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.689 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.690 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.690 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.690 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.690 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.690 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:18.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:18.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:18.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:23:18.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc3316640 1 -- 172.21.15.132:0/1344075490 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b9c005090 con 0x7f7bbc09c0f0 2024-01-24T23:23:18.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f7bac00f860 con 0x7f7bbc09c0f0 2024-01-24T23:23:18.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(51..51 src has 1..51) v4 ==== 4660+0+0 (unknown 1469567968 0 0) 0x7f7bac043a90 con 0x7f7bbc09c0f0 2024-01-24T23:23:18.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.668+0000 7f7bc10b2640 10 monclient: get_auth_request con 0x7f7ba0037b90 auth_method 0 2024-01-24T23:23:18.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.676+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f7bac014750 con 0x7f7bbc09c0f0 2024-01-24T23:23:18.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.676+0000 7f7bb1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:18.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.676+0000 7f7bb1ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:18.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.720+0000 7fc6444d1640 10 monclient: get_monmap_and_config 2024-01-24T23:23:18.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.720+0000 7fc6444d1640 10 monclient: build_initial_monmap 2024-01-24T23:23:18.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.720+0000 7fc6444d1640 10 monclient: monmap: 2024-01-24T23:23:18.725 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:18.725 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.725 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:18.725772+0000 2024-01-24T23:23:18.725 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:18.725772+0000 2024-01-24T23:23:18.725 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:18.725 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.725 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:18.725 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:18.726 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:18.726 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:18.726 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:18.726 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:18.726 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:18.726 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:18.726 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:18.726 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.720+0000 7fc6444d1640 1 Processor -- start 2024-01-24T23:23:18.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 1 -- start start 2024-01-24T23:23:18.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 10 monclient: init 2024-01-24T23:23:18.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:18.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 10 monclient: _add_conns ranks=[8,7,3,4,5,0,1,2,6] 2024-01-24T23:23:18.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 10 monclient(hunting): picked mon.noname-e con 0x7fc63c133c10 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:18.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 10 monclient(hunting): picked mon.noname-i con 0x7fc63c136960 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:18.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 10 monclient(hunting): picked mon.noname-g con 0x7fc63c13a470 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:18.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc63c04ed70 con 0x7fc63c13a470 2024-01-24T23:23:18.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc63c09dee0 con 0x7fc63c136960 2024-01-24T23:23:18.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc63c09d400 con 0x7fc63c133c10 2024-01-24T23:23:18.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:18.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 10 monclient(hunting): authenticate will time out at 1077.648649s 2024-01-24T23:23:18.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64226d640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fc63c133c10 0x7fc63c133ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:52984/0 (socket says 172.21.15.115:52984) 2024-01-24T23:23:18.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64226d640 1 -- 172.21.15.115:0/2201977518 learned_addr learned my addr 172.21.15.115:0/2201977518 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:18.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 1 -- 172.21.15.115:0/2201977518 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1486251768 0 0) 0x7fc63c09d400 con 0x7fc63c133c10 2024-01-24T23:23:18.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 10 monclient(hunting): my global_id is 84248 2024-01-24T23:23:18.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 1 -- 172.21.15.115:0/2201977518 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc628003530 con 0x7fc63c133c10 2024-01-24T23:23:18.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 1 -- 172.21.15.115:0/2201977518 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1112301815 0 0) 0x7fc63c09dee0 con 0x7fc63c136960 2024-01-24T23:23:18.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 10 monclient(hunting): my global_id is 5065 2024-01-24T23:23:18.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 1 -- 172.21.15.115:0/2201977518 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc63c09d400 con 0x7fc63c136960 2024-01-24T23:23:18.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 1 -- 172.21.15.115:0/2201977518 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1374618437 0 0) 0x7fc63c04ed70 con 0x7fc63c13a470 2024-01-24T23:23:18.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 10 monclient(hunting): my global_id is 4989 2024-01-24T23:23:18.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 1 -- 172.21.15.115:0/2201977518 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc63c09dee0 con 0x7fc63c13a470 2024-01-24T23:23:18.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 1 -- 172.21.15.115:0/2201977518 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3745140450 0 0) 0x7fc628003530 con 0x7fc63c133c10 2024-01-24T23:23:18.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 1 -- 172.21.15.115:0/2201977518 >> v1:172.21.15.132:6792/0 conn(0x7fc63c136960 legacy=0x7fc63c138d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 1 -- 172.21.15.115:0/2201977518 >> v1:172.21.15.132:6790/0 conn(0x7fc63c13a470 legacy=0x7fc63c13c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 10 monclient: _finish_hunting 0 2024-01-24T23:23:18.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 1 monclient: found mon.noname-e 2024-01-24T23:23:18.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:18.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:23:18.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 1 -- 172.21.15.115:0/2201977518 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc63c09f380 con 0x7fc63c133c10 2024-01-24T23:23:18.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 10 monclient: _finish_auth 0 2024-01-24T23:23:18.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 10 monclient: _check_auth_tickets 2024-01-24T23:23:18.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:18.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 1 -- 172.21.15.115:0/2201977518 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc6380017c0 con 0x7fc63c133c10 2024-01-24T23:23:18.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 5 monclient: authenticate success, global_id 84248 2024-01-24T23:23:18.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:18.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:23:18.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 10 monclient: dump: 2024-01-24T23:23:18.738 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.738 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.738 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.738 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.738 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.738 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.738 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.738 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.738 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.738 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.738 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.738 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.738 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.738 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.738 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.739 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:18.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 1 -- 172.21.15.115:0/2201977518 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc6380019e0 con 0x7fc63c133c10 2024-01-24T23:23:18.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:18.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 1 -- 172.21.15.115:0/2201977518 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc638004bf0 con 0x7fc63c133c10 2024-01-24T23:23:18.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:18.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 4 set_mon_vals no callback set 2024-01-24T23:23:18.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:18.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64126b640 10 monclient: dump: 2024-01-24T23:23:18.739 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.739 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.739 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.739 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.739 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.740 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.740 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.740 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.740 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.740 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.740 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.740 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.740 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.740 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.740 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.740 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 10 monclient: shutdown 2024-01-24T23:23:18.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:18.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 1 -- 172.21.15.115:0/2201977518 >> v1:172.21.15.115:6793/0 conn(0x7fc63c133c10 legacy=0x7fc63c133ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 1 -- 172.21.15.115:0/2201977518 shutdown_connections 2024-01-24T23:23:18.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 1 -- 172.21.15.115:0/2201977518 >> 172.21.15.115:0/2201977518 conn(0x7fc63c0a5730 msgr2=0x7fc63c0a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:18.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 1 -- 172.21.15.115:0/2201977518 shutdown_connections 2024-01-24T23:23:18.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 1 -- 172.21.15.115:0/2201977518 wait complete. 2024-01-24T23:23:18.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 10 monclient: build_initial_monmap 2024-01-24T23:23:18.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 10 monclient: monmap: 2024-01-24T23:23:18.741 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:18.741 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:18.741 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:18.741 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:18.741 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:18.741 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:18.741 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:18.741 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:18.742 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:18.742 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:18.742 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:18.742 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:18.742 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:18.742 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:18.742 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:18.742 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 1 Processor -- start 2024-01-24T23:23:18.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 1 -- start start 2024-01-24T23:23:18.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 10 monclient: init 2024-01-24T23:23:18.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:18.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 10 monclient: _add_conns ranks=[0,7,6,5,3,4,2,8,1] 2024-01-24T23:23:18.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 10 monclient(hunting): picked mon.noname-a con 0x7fc63c133c10 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:18.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 10 monclient(hunting): picked mon.noname-i con 0x7fc63c136960 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:18.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 10 monclient(hunting): picked mon.noname-d con 0x7fc63c13a470 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:18.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc63c06da20 con 0x7fc63c133c10 2024-01-24T23:23:18.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc63c0a1310 con 0x7fc63c13a470 2024-01-24T23:23:18.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc63c06dbd0 con 0x7fc63c136960 2024-01-24T23:23:18.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:18.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc6444d1640 10 monclient(hunting): authenticate will time out at 1077.651694s 2024-01-24T23:23:18.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64226d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fc63c133c10 0x7fc63c1c9890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:32784/0 (socket says 172.21.15.115:32784) 2024-01-24T23:23:18.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.724+0000 7fc64226d640 1 -- 172.21.15.115:0/3003376910 learned_addr learned my addr 172.21.15.115:0/3003376910 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:18.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3239840362 0 0) 0x7fc63c06da20 con 0x7fc63c133c10 2024-01-24T23:23:18.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient(hunting): my global_id is 5049 2024-01-24T23:23:18.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc620003530 con 0x7fc63c133c10 2024-01-24T23:23:18.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331824833 0 0) 0x7fc63c0a1310 con 0x7fc63c13a470 2024-01-24T23:23:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient(hunting): my global_id is 5091 2024-01-24T23:23:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc63c06da20 con 0x7fc63c13a470 2024-01-24T23:23:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3272918144 0 0) 0x7fc620003530 con 0x7fc63c133c10 2024-01-24T23:23:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc63c0a1310 con 0x7fc63c133c10 2024-01-24T23:23:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc638001700 con 0x7fc63c133c10 2024-01-24T23:23:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:23:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient(hunting): dump: 2024-01-24T23:23:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.745 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.745 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.745 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.745 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.745 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.745 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.745 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.745 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.745 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.745 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.745 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.745 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:18.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 479182851 0 0) 0x7fc63c06dbd0 con 0x7fc63c136960 2024-01-24T23:23:18.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient(hunting): my global_id is 5074 2024-01-24T23:23:18.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc620003530 con 0x7fc63c136960 2024-01-24T23:23:18.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 178336731 0 0) 0x7fc63c06da20 con 0x7fc63c13a470 2024-01-24T23:23:18.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc63c06dbd0 con 0x7fc63c13a470 2024-01-24T23:23:18.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc634001d40 con 0x7fc63c13a470 2024-01-24T23:23:18.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:18.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient(hunting): dump: 2024-01-24T23:23:18.746 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.746 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.746 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.746 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.746 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.747 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.747 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.747 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.747 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.747 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.747 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.747 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.747 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.747 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.747 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.747 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2215234119 0 0) 0x7fc63c0a1310 con 0x7fc63c133c10 2024-01-24T23:23:18.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 >> v1:172.21.15.132:6792/0 conn(0x7fc63c136960 legacy=0x7fc63c1ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 >> v1:172.21.15.115:6792/0 conn(0x7fc63c13a470 legacy=0x7fc63c1d1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:18.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 monclient: found mon.a 2024-01-24T23:23:18.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:18.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:23:18.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc63c06f140 con 0x7fc63c133c10 2024-01-24T23:23:18.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient: _finish_auth 0 2024-01-24T23:23:18.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:18.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:18.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc6444d1640 5 monclient: authenticate success, global_id 5049 2024-01-24T23:23:18.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc6444d1640 10 monclient: _renew_subs 2024-01-24T23:23:18.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc6444d1640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:23:18.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc6444d1640 1 -- 172.21.15.115:0/3003376910 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc63c06e8a0 con 0x7fc63c133c10 2024-01-24T23:23:18.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc6444d1640 10 monclient: _renew_subs 2024-01-24T23:23:18.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc6444d1640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:23:18.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc6444d1640 1 -- 172.21.15.115:0/3003376910 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc63c071a70 con 0x7fc63c133c10 2024-01-24T23:23:18.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc638004c10 con 0x7fc63c133c10 2024-01-24T23:23:18.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:18.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc638001140 con 0x7fc63c133c10 2024-01-24T23:23:18.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:23:18.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc640a6a640 4 set_mon_vals no callback set 2024-01-24T23:23:18.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient: dump: 2024-01-24T23:23:18.749 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.749 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.749 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.749 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.749 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.749 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.749 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.749 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.749 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.750 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.750 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.750 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.750 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.750 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.750 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.750 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc6444d1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:18.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc6444d1640 1 -- 172.21.15.115:0/3003376910 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc6240065d0 con 0x7fc624002870 2024-01-24T23:23:18.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fc63800fb20 con 0x7fc63c133c10 2024-01-24T23:23:18.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(51..51 src has 1..51) v4 ==== 4660+0+0 (unknown 1469567968 0 0) 0x7fc638043bc0 con 0x7fc63c133c10 2024-01-24T23:23:18.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1126273052 0 0) 0x7fc6240065d0 con 0x7fc624002870 2024-01-24T23:23:18.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient(hunting): my global_id is 84257 2024-01-24T23:23:18.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc63c0a1310 con 0x7fc624002870 2024-01-24T23:23:18.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 25005971 0 0) 0x7fc63c0a1310 con 0x7fc624002870 2024-01-24T23:23:18.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc6240065d0 con 0x7fc624002870 2024-01-24T23:23:18.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc62c0018f0 con 0x7fc624002870 2024-01-24T23:23:18.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:18.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient: dump: 2024-01-24T23:23:18.751 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.751 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.751 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.751 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.751 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.751 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.752 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.752 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.752 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.752 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.752 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.752 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.752 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.752 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.752 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.752 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc642a6e640 10 monclient: get_auth_request con 0x7fc620037790 auth_method 0 2024-01-24T23:23:18.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3637304318 0 0) 0x7fc6240065d0 con 0x7fc624002870 2024-01-24T23:23:18.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 --> v1:172.21.15.115:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc62400a120 con 0x7fc624002870 2024-01-24T23:23:18.752 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fc62c003680 con 0x7fc624002870 2024-01-24T23:23:18.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:18.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:18.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 >> v1:172.21.15.115:6793/0 conn(0x7fc624002870 legacy=0x7fc624004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.728+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 >> v1:172.21.15.115:6793/0 conn(0x7fc624002870 legacy=0x7fc624004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:18.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc6444d1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:23:18.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc6444d1640 1 -- 172.21.15.115:0/3003376910 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc624009e10 con 0x7fc6240090d0 2024-01-24T23:23:18.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3176537323 0 0) 0x7fc624009e10 con 0x7fc6240090d0 2024-01-24T23:23:18.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc632ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc632ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc632ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc632ffd640 10 monclient(hunting): my global_id is 84266 2024-01-24T23:23:18.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6240065d0 con 0x7fc6240090d0 2024-01-24T23:23:18.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3228982056 0 0) 0x7fc6240065d0 con 0x7fc6240090d0 2024-01-24T23:23:18.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc624009e10 con 0x7fc6240090d0 2024-01-24T23:23:18.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc62c003e30 con 0x7fc6240090d0 2024-01-24T23:23:18.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc632ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc632ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:18.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc632ffd640 10 monclient: dump: 2024-01-24T23:23:18.755 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.755 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.755 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.755 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.755 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.755 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.755 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.756 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.756 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.756 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.756 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.756 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.756 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.756 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.756 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.756 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2141968549 0 0) 0x7fc624009e10 con 0x7fc6240090d0 2024-01-24T23:23:18.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 --> v1:172.21.15.115:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc62400c310 con 0x7fc6240090d0 2024-01-24T23:23:18.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 1535766245) 0x7fc62c006190 con 0x7fc6240090d0 2024-01-24T23:23:18.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc632ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:23:18.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc632ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:18.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 >> v1:172.21.15.115:6793/0 conn(0x7fc6240090d0 legacy=0x7fc624009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc632ffd640 1 -- 172.21.15.115:0/3003376910 >> v1:172.21.15.115:6793/0 conn(0x7fc6240090d0 legacy=0x7fc624009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:18.757 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:23:18.757 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:18.757 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:18.757 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:23:18.757 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 16, 2024-01-24T23:23:18.757 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:23:18.757 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:23:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:23:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:23:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:23:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:23:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:23:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:23:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:23:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:23:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 4, 2024-01-24T23:23:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:23:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:23:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:23:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:23:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:23:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:23:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:23:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:23:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:23:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:23:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:23:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:23:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:23:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:23:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:23:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:23:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:23:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:23:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:23:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:23:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:23:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:23:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:23:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:23:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:23:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:23:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:23:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:23:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:23:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:23:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:23:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:23:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:23:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:23:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:23:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:23:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:23:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:23:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:23:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:23:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:23:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:23:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:23:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:23:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:23:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:23:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:23:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:23:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:23:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:23:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:23:18.776 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:23:18.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc6444d1640 1 -- 172.21.15.115:0/3003376910 >> v1:172.21.15.132:6812/1835401922 conn(0x7fc620037790 legacy=0x7fc620039c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc6444d1640 10 monclient: shutdown 2024-01-24T23:23:18.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc6444d1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:18.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc6444d1640 1 -- 172.21.15.115:0/3003376910 >> v1:172.21.15.115:6789/0 conn(0x7fc63c133c10 legacy=0x7fc63c1c9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc642a6e640 1 -- 172.21.15.115:0/3003376910 reap_dead start 2024-01-24T23:23:18.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc642a6e640 1 -- 172.21.15.115:0/3003376910 reap_dead start 2024-01-24T23:23:18.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc6444d1640 1 -- 172.21.15.115:0/3003376910 shutdown_connections 2024-01-24T23:23:18.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc6444d1640 1 -- 172.21.15.115:0/3003376910 >> 172.21.15.115:0/3003376910 conn(0x7fc63c0a5730 msgr2=0x7fc63c130ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:18.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc6444d1640 1 -- 172.21.15.115:0/3003376910 shutdown_connections 2024-01-24T23:23:18.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.736+0000 7fc6444d1640 1 -- 172.21.15.115:0/3003376910 wait complete. 2024-01-24T23:23:18.778 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:23:18.778 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-01-24T23:23:18.778 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-01-24T23:23:18.829 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.828+0000 7f7bc3316640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:23:18.829 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.828+0000 7f7bc3316640 10 monclient: _send_command 2 [{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:23:18.829 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.828+0000 7f7bc3316640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:23:18.829 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:18.828+0000 7f7bc3316640 1 -- 172.21.15.132:0/1344075490 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7b9c0058c0 con 0x7f7bbc09c0f0 2024-01-24T23:23:18.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 10 monclient: get_monmap_and_config 2024-01-24T23:23:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 10 monclient: build_initial_monmap 2024-01-24T23:23:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 10 monclient: monmap: 2024-01-24T23:23:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:18.883387+0000 2024-01-24T23:23:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:18.883387+0000 2024-01-24T23:23:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:18.884 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:18.884 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:18.884 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:18.884 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 1 Processor -- start 2024-01-24T23:23:18.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 1 -- start start 2024-01-24T23:23:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 10 monclient: init 2024-01-24T23:23:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 10 monclient: _add_conns ranks=[2,5,3,8,1,7,0,6,4] 2024-01-24T23:23:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 10 monclient(hunting): picked mon.noname-b con 0x7fc968133c10 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 10 monclient(hunting): picked mon.noname-h con 0x7fc968136960 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 10 monclient(hunting): picked mon.noname-g con 0x7fc96813a470 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96804ed70 con 0x7fc968133c10 2024-01-24T23:23:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96809dee0 con 0x7fc96813a470 2024-01-24T23:23:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96809d400 con 0x7fc968136960 2024-01-24T23:23:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 10 monclient(hunting): authenticate will time out at 1077.805885s 2024-01-24T23:23:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96e4df640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fc968133c10 0x7fc968133ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:47668/0 (socket says 172.21.15.115:47668) 2024-01-24T23:23:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96e4df640 1 -- 172.21.15.115:0/1318748646 learned_addr learned my addr 172.21.15.115:0/1318748646 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 1 -- 172.21.15.115:0/1318748646 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1103865856 0 0) 0x7fc96804ed70 con 0x7fc968133c10 2024-01-24T23:23:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 10 monclient(hunting): my global_id is 84260 2024-01-24T23:23:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 1 -- 172.21.15.115:0/1318748646 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc954003530 con 0x7fc968133c10 2024-01-24T23:23:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 1 -- 172.21.15.115:0/1318748646 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2987470391 0 0) 0x7fc96809d400 con 0x7fc968136960 2024-01-24T23:23:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 10 monclient(hunting): my global_id is 5018 2024-01-24T23:23:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 1 -- 172.21.15.115:0/1318748646 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc96804ed70 con 0x7fc968136960 2024-01-24T23:23:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 1 -- 172.21.15.115:0/1318748646 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2552362820 0 0) 0x7fc954003530 con 0x7fc968133c10 2024-01-24T23:23:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 1 -- 172.21.15.115:0/1318748646 >> v1:172.21.15.132:6791/0 conn(0x7fc968136960 legacy=0x7fc968138d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 1 -- 172.21.15.115:0/1318748646 >> v1:172.21.15.132:6790/0 conn(0x7fc96813a470 legacy=0x7fc96813c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 1 monclient: found mon.noname-b 2024-01-24T23:23:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:23:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 1 -- 172.21.15.115:0/1318748646 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc96809f380 con 0x7fc968133c10 2024-01-24T23:23:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 10 monclient: _finish_auth 0 2024-01-24T23:23:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 5 monclient: authenticate success, global_id 84260 2024-01-24T23:23:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 1 -- 172.21.15.115:0/1318748646 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc964001740 con 0x7fc968133c10 2024-01-24T23:23:18.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:23:18.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 10 monclient: dump: 2024-01-24T23:23:18.895 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.895 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.895 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.895 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.895 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.895 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.895 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.895 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.895 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.895 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.896 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.896 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.896 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.896 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.896 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.896 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:18.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 1 -- 172.21.15.115:0/1318748646 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc964001960 con 0x7fc968133c10 2024-01-24T23:23:18.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:18.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 1 -- 172.21.15.115:0/1318748646 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc9640048b0 con 0x7fc968133c10 2024-01-24T23:23:18.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:18.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 4 set_mon_vals no callback set 2024-01-24T23:23:18.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:18.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc96d4dd640 10 monclient: dump: 2024-01-24T23:23:18.897 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.897 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.897 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.897 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.897 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.897 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.897 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.897 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.897 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.897 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.898 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 10 monclient: shutdown 2024-01-24T23:23:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 1 -- 172.21.15.115:0/1318748646 >> v1:172.21.15.115:6790/0 conn(0x7fc968133c10 legacy=0x7fc968133ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 1 -- 172.21.15.115:0/1318748646 shutdown_connections 2024-01-24T23:23:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 1 -- 172.21.15.115:0/1318748646 >> 172.21.15.115:0/1318748646 conn(0x7fc9680a5730 msgr2=0x7fc9680a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 1 -- 172.21.15.115:0/1318748646 shutdown_connections 2024-01-24T23:23:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 1 -- 172.21.15.115:0/1318748646 wait complete. 2024-01-24T23:23:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 10 monclient: build_initial_monmap 2024-01-24T23:23:18.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.880+0000 7fc970743640 10 monclient: monmap: 2024-01-24T23:23:18.899 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:18.899 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:18.899 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:18.899 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:18.899 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:18.899 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:18.899 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:18.899 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:18.899 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:18.899 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:18.899 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:18.899 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:18.899 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:18.899 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:18.900 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:18.900 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc970743640 1 Processor -- start 2024-01-24T23:23:18.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc970743640 1 -- start start 2024-01-24T23:23:18.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc970743640 10 monclient: init 2024-01-24T23:23:18.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc970743640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:18.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc970743640 10 monclient: _add_conns ranks=[2,1,7,0,4,6,5,3,8] 2024-01-24T23:23:18.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc970743640 10 monclient(hunting): picked mon.noname-b con 0x7fc968133c10 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:18.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc970743640 10 monclient(hunting): picked mon.noname-f con 0x7fc968136960 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:18.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc970743640 10 monclient(hunting): picked mon.noname-i con 0x7fc96813a470 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:18.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc970743640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc968132750 con 0x7fc968136960 2024-01-24T23:23:18.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc970743640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc968131890 con 0x7fc968133c10 2024-01-24T23:23:18.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc970743640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc968071c30 con 0x7fc96813a470 2024-01-24T23:23:18.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc970743640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc970743640 10 monclient(hunting): authenticate will time out at 1077.808712s 2024-01-24T23:23:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc96e4df640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fc968133c10 0x7fc968138b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:47678/0 (socket says 172.21.15.115:47678) 2024-01-24T23:23:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc96e4df640 1 -- 172.21.15.115:0/2670256916 learned_addr learned my addr 172.21.15.115:0/2670256916 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660623086 0 0) 0x7fc968131890 con 0x7fc968133c10 2024-01-24T23:23:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 10 monclient(hunting): my global_id is 84269 2024-01-24T23:23:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc94c003530 con 0x7fc968133c10 2024-01-24T23:23:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1964230420 0 0) 0x7fc94c003530 con 0x7fc968133c10 2024-01-24T23:23:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc968131890 con 0x7fc968133c10 2024-01-24T23:23:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc9640016c0 con 0x7fc968133c10 2024-01-24T23:23:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:23:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 10 monclient(hunting): dump: 2024-01-24T23:23:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.903 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3809461215 0 0) 0x7fc968131890 con 0x7fc968133c10 2024-01-24T23:23:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 >> v1:172.21.15.132:6792/0 conn(0x7fc96813a470 legacy=0x7fc9681d3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 >> v1:172.21.15.132:6789/0 conn(0x7fc968136960 legacy=0x7fc9681392a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 1 monclient: found mon.b 2024-01-24T23:23:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9681399b0 con 0x7fc968133c10 2024-01-24T23:23:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 10 monclient: _finish_auth 0 2024-01-24T23:23:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc970743640 5 monclient: authenticate success, global_id 84269 2024-01-24T23:23:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc970743640 10 monclient: _renew_subs 2024-01-24T23:23:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc970743640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc970743640 1 -- 172.21.15.115:0/2670256916 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9680a1370 con 0x7fc968133c10 2024-01-24T23:23:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc970743640 10 monclient: _renew_subs 2024-01-24T23:23:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc970743640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc970743640 1 -- 172.21.15.115:0/2670256916 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9681307e0 con 0x7fc968133c10 2024-01-24T23:23:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc9640011e0 con 0x7fc968133c10 2024-01-24T23:23:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc964005170 con 0x7fc968133c10 2024-01-24T23:23:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc96ccdc640 4 set_mon_vals no callback set 2024-01-24T23:23:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 10 monclient: dump: 2024-01-24T23:23:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.905 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fc96400fbf0 con 0x7fc968133c10 2024-01-24T23:23:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc970743640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc970743640 1 -- 172.21.15.115:0/2670256916 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc9500065d0 con 0x7fc950002870 2024-01-24T23:23:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(51..51 src has 1..51) v4 ==== 4660+0+0 (unknown 1469567968 0 0) 0x7fc964043d10 con 0x7fc968133c10 2024-01-24T23:23:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc96ece0640 10 monclient: get_auth_request con 0x7fc94c037020 auth_method 0 2024-01-24T23:23:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3757049241 0 0) 0x7fc9500065d0 con 0x7fc950002870 2024-01-24T23:23:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 10 monclient(hunting): my global_id is 5140 2024-01-24T23:23:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc968131890 con 0x7fc950002870 2024-01-24T23:23:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1968067158 0 0) 0x7fc968131890 con 0x7fc950002870 2024-01-24T23:23:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc9500065d0 con 0x7fc950002870 2024-01-24T23:23:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc9580018f0 con 0x7fc950002870 2024-01-24T23:23:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:23:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 10 monclient: dump: 2024-01-24T23:23:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.908 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.908 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.908 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3801583469 0 0) 0x7fc9500065d0 con 0x7fc950002870 2024-01-24T23:23:18.908 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.884+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 --> v1:172.21.15.132:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc95000a120 con 0x7fc950002870 2024-01-24T23:23:18.908 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.888+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 <== mon.1 v1:172.21.15.132:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fc958004650 con 0x7fc950002870 2024-01-24T23:23:18.908 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.888+0000 7fc95effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:18.908 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.888+0000 7fc95effd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:18.908 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.888+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 >> v1:172.21.15.132:6789/0 conn(0x7fc950002870 legacy=0x7fc950004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.908 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.888+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 >> v1:172.21.15.132:6789/0 conn(0x7fc950002870 legacy=0x7fc950004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:18.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.892+0000 7fc970743640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:23:18.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.892+0000 7fc970743640 1 -- 172.21.15.115:0/2670256916 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc950009e10 con 0x7fc9500090d0 2024-01-24T23:23:18.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.892+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 902840781 0 0) 0x7fc950009e10 con 0x7fc9500090d0 2024-01-24T23:23:18.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.892+0000 7fc95effd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:18.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.892+0000 7fc95effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:18.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.892+0000 7fc95effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:18.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.892+0000 7fc95effd640 10 monclient(hunting): my global_id is 5149 2024-01-24T23:23:18.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.892+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9500065d0 con 0x7fc9500090d0 2024-01-24T23:23:18.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.892+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3791878158 0 0) 0x7fc9500065d0 con 0x7fc9500090d0 2024-01-24T23:23:18.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.892+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc950009e10 con 0x7fc9500090d0 2024-01-24T23:23:18.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.892+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc958003bd0 con 0x7fc9500090d0 2024-01-24T23:23:18.910 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.892+0000 7fc95effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:18.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.892+0000 7fc95effd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:23:18.911 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.892+0000 7fc95effd640 10 monclient: dump: 2024-01-24T23:23:18.911 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:18.911 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:18.911 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.911 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:18.911 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:18.911 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:18.911 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:18.911 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:18.911 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:18.911 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:18.911 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:18.911 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:18.912 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:18.912 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:18.912 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:18.912 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:18.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.892+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4117005338 0 0) 0x7fc950009e10 con 0x7fc9500090d0 2024-01-24T23:23:18.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.892+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 --> v1:172.21.15.132:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc95000c310 con 0x7fc9500090d0 2024-01-24T23:23:18.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.896+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 <== mon.1 v1:172.21.15.132:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 2863251077) 0x7fc958003fe0 con 0x7fc9500090d0 2024-01-24T23:23:18.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.896+0000 7fc95effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:23:18.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.896+0000 7fc95effd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:18.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.896+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 >> v1:172.21.15.132:6789/0 conn(0x7fc9500090d0 legacy=0x7fc950009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.912 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.896+0000 7fc95effd640 1 -- 172.21.15.115:0/2670256916 >> v1:172.21.15.132:6789/0 conn(0x7fc9500090d0 legacy=0x7fc950009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:18.913 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:23:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:23:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 16, 2024-01-24T23:23:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:23:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:23:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:23:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:23:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:23:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:23:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:23:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:23:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:23:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:23:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 4, 2024-01-24T23:23:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:23:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:23:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:23:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:23:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:23:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:23:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:23:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:23:18.916 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:23:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:23:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:23:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:23:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:23:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:23:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:23:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:23:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:23:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:23:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:18.917 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:23:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:23:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.918 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:23:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:23:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:23:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.919 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:23:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:23:18.920 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:23:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:23:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.921 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:23:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:23:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:23:18.922 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:23:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:23:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:23:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:23:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:23:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.924 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:23:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:23:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:23:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.925 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:23:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:23:18.926 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:23:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:23:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.927 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:23:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:23:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.928 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:23:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:23:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:23:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:23:18.929 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:23:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:23:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:23:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:18.930 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:23:18.931 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:23:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.896+0000 7fc970743640 1 -- 172.21.15.115:0/2670256916 >> v1:172.21.15.132:6812/1835401922 conn(0x7fc94c037020 legacy=0x7fc94c0394e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.896+0000 7fc970743640 10 monclient: shutdown 2024-01-24T23:23:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.896+0000 7fc970743640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.896+0000 7fc970743640 1 -- 172.21.15.115:0/2670256916 >> v1:172.21.15.115:6790/0 conn(0x7fc968133c10 legacy=0x7fc968138b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.896+0000 7fc96ece0640 1 -- 172.21.15.115:0/2670256916 reap_dead start 2024-01-24T23:23:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.896+0000 7fc96ece0640 1 -- 172.21.15.115:0/2670256916 reap_dead start 2024-01-24T23:23:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.896+0000 7fc970743640 1 -- 172.21.15.115:0/2670256916 shutdown_connections 2024-01-24T23:23:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.896+0000 7fc970743640 1 -- 172.21.15.115:0/2670256916 >> 172.21.15.115:0/2670256916 conn(0x7fc9680a5730 msgr2=0x7fc968136220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.896+0000 7fc970743640 1 -- 172.21.15.115:0/2670256916 shutdown_connections 2024-01-24T23:23:18.932 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:18.896+0000 7fc970743640 1 -- 172.21.15.115:0/2670256916 wait complete. 2024-01-24T23:23:18.933 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:23:18.933 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-01-24T23:23:18.933 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-01-24T23:23:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 10 monclient: get_monmap_and_config 2024-01-24T23:23:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 10 monclient: build_initial_monmap 2024-01-24T23:23:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 10 monclient: monmap: 2024-01-24T23:23:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:19.039181+0000 2024-01-24T23:23:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:19.039181+0000 2024-01-24T23:23:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:19.039 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:19.039 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 1 Processor -- start 2024-01-24T23:23:19.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 1 -- start start 2024-01-24T23:23:19.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 10 monclient: init 2024-01-24T23:23:19.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:19.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 10 monclient: _add_conns ranks=[6,0,8,3,2,1,7,5,4] 2024-01-24T23:23:19.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 10 monclient(hunting): picked mon.noname-d con 0x7f2a50133bf0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:19.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 10 monclient(hunting): picked mon.noname-a con 0x7f2a50133fd0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:19.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 10 monclient(hunting): picked mon.noname-e con 0x7f2a501343b0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:19.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a5004ed70 con 0x7f2a50133fd0 2024-01-24T23:23:19.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a500a4b60 con 0x7f2a50133bf0 2024-01-24T23:23:19.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a5009ae50 con 0x7f2a501343b0 2024-01-24T23:23:19.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:19.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 10 monclient(hunting): authenticate will time out at 1077.961875s 2024-01-24T23:23:19.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a5505a640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f2a50133bf0 0x7f2a5009bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:34248/0 (socket says 172.21.15.115:34248) 2024-01-24T23:23:19.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a5505a640 1 -- 172.21.15.115:0/2499520805 learned_addr learned my addr 172.21.15.115:0/2499520805 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:19.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 1 -- 172.21.15.115:0/2499520805 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1450138782 0 0) 0x7f2a500a4b60 con 0x7f2a50133bf0 2024-01-24T23:23:19.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:19.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:19.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:19.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 10 monclient(hunting): my global_id is 5100 2024-01-24T23:23:19.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 1 -- 172.21.15.115:0/2499520805 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a3c003530 con 0x7f2a50133bf0 2024-01-24T23:23:19.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 1 -- 172.21.15.115:0/2499520805 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3150014704 0 0) 0x7f2a5004ed70 con 0x7f2a50133fd0 2024-01-24T23:23:19.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:19.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:19.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:19.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 10 monclient(hunting): my global_id is 5058 2024-01-24T23:23:19.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 1 -- 172.21.15.115:0/2499520805 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a500a4b60 con 0x7f2a50133fd0 2024-01-24T23:23:19.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 1 -- 172.21.15.115:0/2499520805 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1778669098 0 0) 0x7f2a5009ae50 con 0x7f2a501343b0 2024-01-24T23:23:19.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:19.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:19.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:19.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 10 monclient(hunting): my global_id is 84275 2024-01-24T23:23:19.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 1 -- 172.21.15.115:0/2499520805 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a5004ed70 con 0x7f2a501343b0 2024-01-24T23:23:19.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 1 -- 172.21.15.115:0/2499520805 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3318398002 0 0) 0x7f2a500a4b60 con 0x7f2a50133fd0 2024-01-24T23:23:19.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 1 -- 172.21.15.115:0/2499520805 >> v1:172.21.15.115:6793/0 conn(0x7f2a501343b0 legacy=0x7f2a5013f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 1 -- 172.21.15.115:0/2499520805 >> v1:172.21.15.115:6792/0 conn(0x7f2a50133bf0 legacy=0x7f2a5009bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 10 monclient: _finish_hunting 0 2024-01-24T23:23:19.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 1 monclient: found mon.noname-a 2024-01-24T23:23:19.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:19.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:23:19.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 1 -- 172.21.15.115:0/2499520805 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5009b130 con 0x7f2a50133fd0 2024-01-24T23:23:19.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 10 monclient: _finish_auth 0 2024-01-24T23:23:19.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 10 monclient: _check_auth_tickets 2024-01-24T23:23:19.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:19.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 1 -- 172.21.15.115:0/2499520805 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2a4c0017a0 con 0x7f2a50133fd0 2024-01-24T23:23:19.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:23:19.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 10 monclient: dump: 2024-01-24T23:23:19.050 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:19.050 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.051 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 5 monclient: authenticate success, global_id 5058 2024-01-24T23:23:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 1 -- 172.21.15.115:0/2499520805 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2a4c001ac0 con 0x7f2a50133fd0 2024-01-24T23:23:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 4 set_mon_vals no callback set 2024-01-24T23:23:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 1 -- 172.21.15.115:0/2499520805 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2a4c003960 con 0x7f2a50133fd0 2024-01-24T23:23:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:23:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a43fff640 10 monclient: dump: 2024-01-24T23:23:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.053 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 10 monclient: shutdown 2024-01-24T23:23:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 1 -- 172.21.15.115:0/2499520805 >> v1:172.21.15.115:6789/0 conn(0x7f2a50133fd0 legacy=0x7f2a5013bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 1 -- 172.21.15.115:0/2499520805 shutdown_connections 2024-01-24T23:23:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 1 -- 172.21.15.115:0/2499520805 >> 172.21.15.115:0/2499520805 conn(0x7f2a500662f0 msgr2=0x7f2a50066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 1 -- 172.21.15.115:0/2499520805 shutdown_connections 2024-01-24T23:23:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 1 -- 172.21.15.115:0/2499520805 wait complete. 2024-01-24T23:23:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 10 monclient: build_initial_monmap 2024-01-24T23:23:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.036+0000 7f2a572be640 10 monclient: monmap: 2024-01-24T23:23:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:19.055 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a572be640 1 Processor -- start 2024-01-24T23:23:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a572be640 1 -- start start 2024-01-24T23:23:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a572be640 10 monclient: init 2024-01-24T23:23:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a572be640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a572be640 10 monclient: _add_conns ranks=[3,2,5,6,4,7,8,0,1] 2024-01-24T23:23:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a572be640 10 monclient(hunting): picked mon.noname-g con 0x7f2a50133bf0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a572be640 10 monclient(hunting): picked mon.noname-b con 0x7f2a50133fd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a572be640 10 monclient(hunting): picked mon.noname-h con 0x7f2a501343b0 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:19.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a572be640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a5009c810 con 0x7f2a50133fd0 2024-01-24T23:23:19.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a572be640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a500a5990 con 0x7f2a50133bf0 2024-01-24T23:23:19.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a572be640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a501200e0 con 0x7f2a501343b0 2024-01-24T23:23:19.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a572be640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:19.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a572be640 10 monclient(hunting): authenticate will time out at 1077.964734s 2024-01-24T23:23:19.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a54859640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f2a50133fd0 0x7f2a501d05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:47690/0 (socket says 172.21.15.115:47690) 2024-01-24T23:23:19.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a54859640 1 -- 172.21.15.115:0/3996754808 learned_addr learned my addr 172.21.15.115:0/3996754808 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:19.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1477784400 0 0) 0x7f2a5009c810 con 0x7f2a50133fd0 2024-01-24T23:23:19.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:19.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 10 monclient(hunting): my global_id is 84278 2024-01-24T23:23:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a34003530 con 0x7f2a50133fd0 2024-01-24T23:23:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3458665028 0 0) 0x7f2a34003530 con 0x7f2a50133fd0 2024-01-24T23:23:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2a5009c810 con 0x7f2a50133fd0 2024-01-24T23:23:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2a4c001760 con 0x7f2a50133fd0 2024-01-24T23:23:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:23:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 10 monclient(hunting): dump: 2024-01-24T23:23:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.058 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3753037675 0 0) 0x7f2a5009c810 con 0x7f2a50133fd0 2024-01-24T23:23:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 >> v1:172.21.15.132:6791/0 conn(0x7f2a501343b0 legacy=0x7f2a501d3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 >> v1:172.21.15.132:6790/0 conn(0x7f2a50133bf0 legacy=0x7f2a50132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:23:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 1 monclient: found mon.b 2024-01-24T23:23:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a501208f0 con 0x7f2a50133fd0 2024-01-24T23:23:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 10 monclient: _finish_auth 0 2024-01-24T23:23:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:23:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a572be640 5 monclient: authenticate success, global_id 84278 2024-01-24T23:23:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a572be640 10 monclient: _renew_subs 2024-01-24T23:23:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a572be640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a572be640 1 -- 172.21.15.115:0/3996754808 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a500a2650 con 0x7f2a50133fd0 2024-01-24T23:23:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a572be640 10 monclient: _renew_subs 2024-01-24T23:23:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a572be640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a572be640 1 -- 172.21.15.115:0/3996754808 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a50133620 con 0x7f2a50133fd0 2024-01-24T23:23:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2a4c0011e0 con 0x7f2a50133fd0 2024-01-24T23:23:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2a4c004220 con 0x7f2a50133fd0 2024-01-24T23:23:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a437fe640 4 set_mon_vals no callback set 2024-01-24T23:23:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 10 monclient: dump: 2024-01-24T23:23:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.061 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f2a4c004380 con 0x7f2a50133fd0 2024-01-24T23:23:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a572be640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a572be640 1 -- 172.21.15.115:0/3996754808 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2a300065d0 con 0x7f2a30002870 2024-01-24T23:23:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(51..51 src has 1..51) v4 ==== 4660+0+0 (unknown 1469567968 0 0) 0x7f2a4c043e10 con 0x7f2a50133fd0 2024-01-24T23:23:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a5585b640 10 monclient: get_auth_request con 0x7f2a34037030 auth_method 0 2024-01-24T23:23:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2018753066 0 0) 0x7f2a300065d0 con 0x7f2a30002870 2024-01-24T23:23:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 10 monclient(hunting): my global_id is 5016 2024-01-24T23:23:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a5009c810 con 0x7f2a30002870 2024-01-24T23:23:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3668345070 0 0) 0x7f2a5009c810 con 0x7f2a30002870 2024-01-24T23:23:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2a300065d0 con 0x7f2a30002870 2024-01-24T23:23:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2a480016d0 con 0x7f2a30002870 2024-01-24T23:23:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.040+0000 7f2a41ffb640 10 monclient: dump: 2024-01-24T23:23:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.063 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.044+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2007292404 0 0) 0x7f2a300065d0 con 0x7f2a30002870 2024-01-24T23:23:19.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.044+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 --> v1:172.21.15.132:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2a3000a120 con 0x7f2a30002870 2024-01-24T23:23:19.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.044+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f2a48001b60 con 0x7f2a30002870 2024-01-24T23:23:19.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.044+0000 7f2a41ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:19.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.044+0000 7f2a41ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:19.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.044+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 >> v1:172.21.15.132:6790/0 conn(0x7f2a30002870 legacy=0x7f2a30004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.044+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 >> v1:172.21.15.132:6790/0 conn(0x7f2a30002870 legacy=0x7f2a30004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:19.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.048+0000 7f2a572be640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:23:19.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.048+0000 7f2a572be640 1 -- 172.21.15.115:0/3996754808 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2a30009e10 con 0x7f2a300090d0 2024-01-24T23:23:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.048+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1673833332 0 0) 0x7f2a30009e10 con 0x7f2a300090d0 2024-01-24T23:23:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.048+0000 7f2a41ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.048+0000 7f2a41ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.048+0000 7f2a41ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.048+0000 7f2a41ffb640 10 monclient(hunting): my global_id is 5025 2024-01-24T23:23:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.048+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a300065d0 con 0x7f2a300090d0 2024-01-24T23:23:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.052+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 492085608 0 0) 0x7f2a300065d0 con 0x7f2a300090d0 2024-01-24T23:23:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.052+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2a30009e10 con 0x7f2a300090d0 2024-01-24T23:23:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.052+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2a48005160 con 0x7f2a300090d0 2024-01-24T23:23:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.052+0000 7f2a41ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.052+0000 7f2a41ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.052+0000 7f2a41ffb640 10 monclient: dump: 2024-01-24T23:23:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.067 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.052+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 883191740 0 0) 0x7f2a30009e10 con 0x7f2a300090d0 2024-01-24T23:23:19.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.052+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 --> v1:172.21.15.132:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2a3000c310 con 0x7f2a300090d0 2024-01-24T23:23:19.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.052+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 1878453290) 0x7f2a48007470 con 0x7f2a300090d0 2024-01-24T23:23:19.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.052+0000 7f2a41ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:23:19.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.052+0000 7f2a41ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:19.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.052+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 >> v1:172.21.15.132:6790/0 conn(0x7f2a300090d0 legacy=0x7f2a30009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.052+0000 7f2a41ffb640 1 -- 172.21.15.115:0/3996754808 >> v1:172.21.15.132:6790/0 conn(0x7f2a300090d0 legacy=0x7f2a30009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:19.068 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:23:19.068 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:19.068 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:19.068 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:23:19.069 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 16, 2024-01-24T23:23:19.069 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:23:19.069 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:23:19.069 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:23:19.069 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:23:19.069 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:23:19.069 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:23:19.069 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:23:19.069 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:23:19.069 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:23:19.069 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:23:19.069 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:19.069 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 4, 2024-01-24T23:23:19.069 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:19.069 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:23:19.070 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:23:19.070 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:19.070 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:19.070 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:19.070 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:19.070 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:19.070 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:19.070 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:19.070 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:19.070 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:19.070 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:19.070 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:19.070 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:19.070 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:23:19.071 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:23:19.071 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:19.071 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:19.071 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:19.071 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:19.071 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:19.071 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:19.071 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:19.071 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:19.071 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:19.071 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:19.071 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:19.071 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.072 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:23:19.072 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:23:19.072 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:23:19.072 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:23:19.072 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:23:19.072 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:23:19.072 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:19.072 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:19.072 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:23:19.072 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:23:19.072 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:23:19.072 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:23:19.072 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:23:19.072 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:23:19.072 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:23:19.073 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:19.073 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:23:19.073 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:19.073 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:19.073 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:19.073 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:19.073 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:19.073 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:19.073 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:19.073 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:19.073 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:19.073 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:19.073 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:19.073 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:19.073 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:23:19.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.074 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:23:19.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.074 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:19.074 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:19.074 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.074 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.074 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.074 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:23:19.074 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.074 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.074 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.074 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:23:19.074 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:23:19.075 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.075 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.075 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.075 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:19.075 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:19.075 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.075 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.075 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.075 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:23:19.075 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.075 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.075 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.076 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:23:19.076 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:23:19.076 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.076 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.076 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.076 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:19.076 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:19.076 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.076 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.076 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.076 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:23:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:23:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:23:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:23:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:23:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:23:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:19.079 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:19.079 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.079 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.079 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.079 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:23:19.079 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.079 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.079 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.079 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:23:19.079 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:23:19.079 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.080 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.080 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.080 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:19.080 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:19.080 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.080 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.080 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.080 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:23:19.080 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.080 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.080 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.081 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:23:19.081 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:23:19.081 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.081 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.081 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.081 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:19.081 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:19.081 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.081 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.081 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.081 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:23:19.082 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.082 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.082 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.082 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:23:19.082 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:23:19.082 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.082 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.082 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.082 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:19.082 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:19.082 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.082 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:23:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:23:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:23:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:19.084 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.084 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.084 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.084 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:23:19.084 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.084 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.084 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.084 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:23:19.084 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:23:19.084 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.084 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.084 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.084 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:23:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:23:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:23:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.086 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:19.086 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": [ 2024-01-24T23:23:19.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.086 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:19.086 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:19.086 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:19.086 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.086 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.086 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:23:19.086 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:23:19.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.052+0000 7f2a572be640 1 -- 172.21.15.115:0/3996754808 >> v1:172.21.15.132:6812/1835401922 conn(0x7f2a34037030 legacy=0x7f2a340394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.052+0000 7f2a572be640 10 monclient: shutdown 2024-01-24T23:23:19.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.052+0000 7f2a572be640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:19.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.052+0000 7f2a572be640 1 -- 172.21.15.115:0/3996754808 >> v1:172.21.15.115:6790/0 conn(0x7f2a50133fd0 legacy=0x7f2a501d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.052+0000 7f2a5585b640 1 -- 172.21.15.115:0/3996754808 reap_dead start 2024-01-24T23:23:19.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.052+0000 7f2a5585b640 1 -- 172.21.15.115:0/3996754808 reap_dead start 2024-01-24T23:23:19.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.052+0000 7f2a572be640 1 -- 172.21.15.115:0/3996754808 shutdown_connections 2024-01-24T23:23:19.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.052+0000 7f2a572be640 1 -- 172.21.15.115:0/3996754808 >> 172.21.15.115:0/3996754808 conn(0x7f2a500662f0 msgr2=0x7f2a501be780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:19.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.052+0000 7f2a572be640 1 -- 172.21.15.115:0/3996754808 shutdown_connections 2024-01-24T23:23:19.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.052+0000 7f2a572be640 1 -- 172.21.15.115:0/3996754808 wait complete. 2024-01-24T23:23:19.088 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:23:19.088 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-01-24T23:23:19.088 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-01-24T23:23:19.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 10 monclient: get_monmap_and_config 2024-01-24T23:23:19.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 10 monclient: build_initial_monmap 2024-01-24T23:23:19.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 10 monclient: monmap: 2024-01-24T23:23:19.194 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:19.194 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.194 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:19.194953+0000 2024-01-24T23:23:19.194 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:19.194953+0000 2024-01-24T23:23:19.194 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:19.195 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.195 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:19.195 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:19.195 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:19.195 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:19.195 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:19.195 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:19.195 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:19.195 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:19.195 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:19.195 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 1 Processor -- start 2024-01-24T23:23:19.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 1 -- start start 2024-01-24T23:23:19.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 10 monclient: init 2024-01-24T23:23:19.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:19.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 10 monclient: _add_conns ranks=[0,3,4,5,8,1,6,7,2] 2024-01-24T23:23:19.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 10 monclient(hunting): picked mon.noname-a con 0x7fdaa8133af0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:19.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 10 monclient(hunting): picked mon.noname-g con 0x7fdaa8139cc0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:19.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 10 monclient(hunting): picked mon.noname-c con 0x7fdaa813d7d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:19.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa804ed70 con 0x7fdaa8133af0 2024-01-24T23:23:19.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa809afe0 con 0x7fdaa8139cc0 2024-01-24T23:23:19.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa81345e0 con 0x7fdaa813d7d0 2024-01-24T23:23:19.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:19.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 10 monclient(hunting): authenticate will time out at 1078.117527s 2024-01-24T23:23:19.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa7fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fdaa8133af0 0x7fdaa8133ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:32800/0 (socket says 172.21.15.115:32800) 2024-01-24T23:23:19.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa7fff640 1 -- 172.21.15.115:0/721983216 learned_addr learned my addr 172.21.15.115:0/721983216 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:19.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 1 -- 172.21.15.115:0/721983216 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2270633723 0 0) 0x7fdaa804ed70 con 0x7fdaa8133af0 2024-01-24T23:23:19.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:19.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:19.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:19.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 10 monclient(hunting): my global_id is 5067 2024-01-24T23:23:19.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 1 -- 172.21.15.115:0/721983216 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda94003530 con 0x7fdaa8133af0 2024-01-24T23:23:19.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 1 -- 172.21.15.115:0/721983216 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3810683444 0 0) 0x7fdaa81345e0 con 0x7fdaa813d7d0 2024-01-24T23:23:19.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:19.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:19.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:19.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 10 monclient(hunting): my global_id is 75028 2024-01-24T23:23:19.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 1 -- 172.21.15.115:0/721983216 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaa804ed70 con 0x7fdaa813d7d0 2024-01-24T23:23:19.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 1 -- 172.21.15.115:0/721983216 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1333702249 0 0) 0x7fda94003530 con 0x7fdaa8133af0 2024-01-24T23:23:19.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 1 -- 172.21.15.115:0/721983216 >> v1:172.21.15.115:6791/0 conn(0x7fdaa813d7d0 legacy=0x7fdaa813fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 1 -- 172.21.15.115:0/721983216 >> v1:172.21.15.132:6790/0 conn(0x7fdaa8139cc0 legacy=0x7fdaa813c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:19.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 1 monclient: found mon.noname-a 2024-01-24T23:23:19.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:19.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:23:19.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 1 -- 172.21.15.115:0/721983216 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa8067200 con 0x7fdaa8133af0 2024-01-24T23:23:19.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 10 monclient: _finish_auth 0 2024-01-24T23:23:19.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:19.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:19.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 1 -- 172.21.15.115:0/721983216 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fda980017c0 con 0x7fdaa8133af0 2024-01-24T23:23:19.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 5 monclient: authenticate success, global_id 5067 2024-01-24T23:23:19.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:19.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:23:19.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 10 monclient: dump: 2024-01-24T23:23:19.205 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:19.205 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.205 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.205 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.205 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.205 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.205 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.206 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.206 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.206 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.206 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.206 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.206 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.206 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.206 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.206 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:19.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 1 -- 172.21.15.115:0/721983216 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fda98001a60 con 0x7fdaa8133af0 2024-01-24T23:23:19.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:19.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:19.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 4 set_mon_vals no callback set 2024-01-24T23:23:19.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 1 -- 172.21.15.115:0/721983216 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fda98004bf0 con 0x7fdaa8133af0 2024-01-24T23:23:19.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:23:19.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaa6ffd640 10 monclient: dump: 2024-01-24T23:23:19.207 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:19.207 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.207 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.207 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.207 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.207 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.207 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.207 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.207 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.207 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.207 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.207 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.208 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.208 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.208 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.208 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 10 monclient: shutdown 2024-01-24T23:23:19.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:19.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 1 -- 172.21.15.115:0/721983216 >> v1:172.21.15.115:6789/0 conn(0x7fdaa8133af0 legacy=0x7fdaa8133ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 1 -- 172.21.15.115:0/721983216 shutdown_connections 2024-01-24T23:23:19.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 1 -- 172.21.15.115:0/721983216 >> 172.21.15.115:0/721983216 conn(0x7fdaa80a65e0 msgr2=0x7fdaa80a2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:19.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 1 -- 172.21.15.115:0/721983216 shutdown_connections 2024-01-24T23:23:19.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 1 -- 172.21.15.115:0/721983216 wait complete. 2024-01-24T23:23:19.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 10 monclient: build_initial_monmap 2024-01-24T23:23:19.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.192+0000 7fdaae78d640 10 monclient: monmap: 2024-01-24T23:23:19.208 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:19.209 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:19.209 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:19.209 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:19.209 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:19.209 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:19.209 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:19.209 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:19.209 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:19.209 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:19.209 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:19.209 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:19.209 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:19.209 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:19.209 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:19.209 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaae78d640 1 Processor -- start 2024-01-24T23:23:19.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaae78d640 1 -- start start 2024-01-24T23:23:19.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaae78d640 10 monclient: init 2024-01-24T23:23:19.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaae78d640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:19.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaae78d640 10 monclient: _add_conns ranks=[2,1,8,5,4,7,6,0,3] 2024-01-24T23:23:19.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaae78d640 10 monclient(hunting): picked mon.noname-b con 0x7fdaa8133af0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:19.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaae78d640 10 monclient(hunting): picked mon.noname-f con 0x7fdaa8139cc0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:19.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaae78d640 10 monclient(hunting): picked mon.noname-e con 0x7fdaa813d7d0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:19.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaae78d640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa809c3e0 con 0x7fdaa8139cc0 2024-01-24T23:23:19.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaae78d640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa813bb90 con 0x7fdaa8133af0 2024-01-24T23:23:19.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaae78d640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa813c2b0 con 0x7fdaa813d7d0 2024-01-24T23:23:19.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaae78d640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:19.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaae78d640 10 monclient(hunting): authenticate will time out at 1078.120708s 2024-01-24T23:23:19.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa7fff640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fdaa8133af0 0x7fdaa813a700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:47696/0 (socket says 172.21.15.115:47696) 2024-01-24T23:23:19.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa7fff640 1 -- 172.21.15.115:0/2220231003 learned_addr learned my addr 172.21.15.115:0/2220231003 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:19.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968262340 0 0) 0x7fdaa813bb90 con 0x7fdaa8133af0 2024-01-24T23:23:19.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:19.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:19.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:19.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient(hunting): my global_id is 84287 2024-01-24T23:23:19.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda88003530 con 0x7fdaa8133af0 2024-01-24T23:23:19.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1216255939 0 0) 0x7fdaa813c2b0 con 0x7fdaa813d7d0 2024-01-24T23:23:19.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:19.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:19.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:19.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient(hunting): my global_id is 84284 2024-01-24T23:23:19.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaa813bb90 con 0x7fdaa813d7d0 2024-01-24T23:23:19.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1155372493 0 0) 0x7fda88003530 con 0x7fdaa8133af0 2024-01-24T23:23:19.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdaa813c2b0 con 0x7fdaa8133af0 2024-01-24T23:23:19.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fda98004b30 con 0x7fdaa8133af0 2024-01-24T23:23:19.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:23:19.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient(hunting): dump: 2024-01-24T23:23:19.212 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:19.212 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.212 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.212 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.212 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.212 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.212 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.212 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.212 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.212 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.212 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.212 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.213 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.213 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.213 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.213 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:19.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2278211521 0 0) 0x7fdaa813bb90 con 0x7fdaa813d7d0 2024-01-24T23:23:19.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fda88003530 con 0x7fdaa813d7d0 2024-01-24T23:23:19.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdaa0001c80 con 0x7fdaa813d7d0 2024-01-24T23:23:19.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:19.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient(hunting): dump: 2024-01-24T23:23:19.213 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:19.213 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.213 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.213 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.214 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.214 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.214 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.214 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.214 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.214 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.214 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.214 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.214 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.214 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.214 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.214 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1210816985 0 0) 0x7fdaa813c2b0 con 0x7fdaa8133af0 2024-01-24T23:23:19.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 >> v1:172.21.15.115:6793/0 conn(0x7fdaa813d7d0 legacy=0x7fdaa81d39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 >> v1:172.21.15.132:6789/0 conn(0x7fdaa8139cc0 legacy=0x7fdaa813ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:23:19.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 1 monclient: found mon.b 2024-01-24T23:23:19.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:19.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:19.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa81353b0 con 0x7fdaa8133af0 2024-01-24T23:23:19.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient: _finish_auth 0 2024-01-24T23:23:19.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:19.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:19.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaae78d640 5 monclient: authenticate success, global_id 84287 2024-01-24T23:23:19.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaae78d640 10 monclient: _renew_subs 2024-01-24T23:23:19.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaae78d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:19.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaae78d640 1 -- 172.21.15.115:0/2220231003 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa809c9a0 con 0x7fdaa8133af0 2024-01-24T23:23:19.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaae78d640 10 monclient: _renew_subs 2024-01-24T23:23:19.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaae78d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:19.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaae78d640 1 -- 172.21.15.115:0/2220231003 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa813b520 con 0x7fdaa8133af0 2024-01-24T23:23:19.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fda98004c90 con 0x7fdaa8133af0 2024-01-24T23:23:19.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:19.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fda98005240 con 0x7fdaa8133af0 2024-01-24T23:23:19.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:19.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient: dump: 2024-01-24T23:23:19.216 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:19.216 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.216 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.216 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.216 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.216 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.216 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.216 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.217 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.217 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.217 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.217 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.217 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.217 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.217 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.217 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa67fc640 4 set_mon_vals no callback set 2024-01-24T23:23:19.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaae78d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:19.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fda9800fcc0 con 0x7fdaa8133af0 2024-01-24T23:23:19.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(51..51 src has 1..51) v4 ==== 4660+0+0 (unknown 1469567968 0 0) 0x7fda98043d10 con 0x7fdaa8133af0 2024-01-24T23:23:19.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaae78d640 1 -- 172.21.15.115:0/2220231003 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdaa806a0d0 con 0x7fdaa8137de0 2024-01-24T23:23:19.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa77fe640 10 monclient: get_auth_request con 0x7fda880376e0 auth_method 0 2024-01-24T23:23:19.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1713066015 0 0) 0x7fdaa806a0d0 con 0x7fdaa8137de0 2024-01-24T23:23:19.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:19.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:19.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:19.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient(hunting): my global_id is 5036 2024-01-24T23:23:19.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaa813c2b0 con 0x7fdaa8137de0 2024-01-24T23:23:19.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3041520218 0 0) 0x7fdaa813c2b0 con 0x7fdaa8137de0 2024-01-24T23:23:19.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdaa806a0d0 con 0x7fdaa8137de0 2024-01-24T23:23:19.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdaa0003eb0 con 0x7fdaa8137de0 2024-01-24T23:23:19.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:19.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.196+0000 7fdaa4ff9640 10 monclient: dump: 2024-01-24T23:23:19.218 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:19.218 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.218 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.219 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.219 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.219 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.219 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.219 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.219 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.219 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.219 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.219 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.219 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.219 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.219 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.219 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.200+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2721596200 0 0) 0x7fdaa806a0d0 con 0x7fdaa8137de0 2024-01-24T23:23:19.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.200+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 --> v1:172.21.15.132:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdaa809c260 con 0x7fdaa8137de0 2024-01-24T23:23:19.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.200+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fdaa0006130 con 0x7fdaa8137de0 2024-01-24T23:23:19.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.200+0000 7fdaa4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:19.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.200+0000 7fdaa4ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:19.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.200+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 >> v1:172.21.15.132:6791/0 conn(0x7fdaa8137de0 legacy=0x7fdaa81381c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.200+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 >> v1:172.21.15.132:6791/0 conn(0x7fdaa8137de0 legacy=0x7fdaa81381c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:19.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.204+0000 7fdaae78d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:23:19.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.204+0000 7fdaae78d640 1 -- 172.21.15.115:0/2220231003 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdaa80a54c0 con 0x7fdaa81388d0 2024-01-24T23:23:19.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.204+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051960401 0 0) 0x7fdaa80a54c0 con 0x7fdaa81388d0 2024-01-24T23:23:19.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.204+0000 7fdaa4ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:19.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.204+0000 7fdaa4ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:19.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.204+0000 7fdaa4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:19.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.204+0000 7fdaa4ff9640 10 monclient(hunting): my global_id is 5045 2024-01-24T23:23:19.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.204+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaa806a0d0 con 0x7fdaa81388d0 2024-01-24T23:23:19.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.208+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2272027242 0 0) 0x7fdaa806a0d0 con 0x7fdaa81388d0 2024-01-24T23:23:19.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.208+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdaa80a54c0 con 0x7fdaa81388d0 2024-01-24T23:23:19.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.208+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdaa0009480 con 0x7fdaa81388d0 2024-01-24T23:23:19.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.208+0000 7fdaa4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.208+0000 7fdaa4ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:19.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.208+0000 7fdaa4ff9640 10 monclient: dump: 2024-01-24T23:23:19.222 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:19.222 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.222 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.223 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.223 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.223 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.223 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.223 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.223 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.223 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.223 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.223 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.223 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.223 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.223 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.223 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.208+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 908915951 0 0) 0x7fdaa80a54c0 con 0x7fdaa81388d0 2024-01-24T23:23:19.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.208+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 --> v1:172.21.15.132:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdaa809a9e0 con 0x7fdaa81388d0 2024-01-24T23:23:19.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.208+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 2060116763) 0x7fdaa0005e00 con 0x7fdaa81388d0 2024-01-24T23:23:19.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.208+0000 7fdaa4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:23:19.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.208+0000 7fdaa4ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:19.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.208+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 >> v1:172.21.15.132:6791/0 conn(0x7fdaa81388d0 legacy=0x7fdaa81d90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.208+0000 7fdaa4ff9640 1 -- 172.21.15.115:0/2220231003 >> v1:172.21.15.132:6791/0 conn(0x7fdaa81388d0 legacy=0x7fdaa81d90e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:19.224 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:23:19.224 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:19.224 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:19.224 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:23:19.224 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 16, 2024-01-24T23:23:19.224 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:23:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:23:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:23:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:23:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:23:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:23:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:23:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:23:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:23:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:23:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 4, 2024-01-24T23:23:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:23:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:23:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:23:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:23:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:23:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:23:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:23:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:23:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:23:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:23:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:23:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:23:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:23:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:23:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:23:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:23:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:23:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:23:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:23:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:23:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:23:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:23:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:23:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:23:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:23:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:23:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.233 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:19.233 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:19.233 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.233 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.233 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.233 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:23:19.233 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:23:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:23:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:23:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:23:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:23:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:23:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:23:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:23:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:23:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:23:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:23:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:23:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:23:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:23:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:23:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:23:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:23:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:23:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:23:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:23:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:23:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:23:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:23:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:23:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:23:19.243 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:23:19.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.208+0000 7fdaae78d640 1 -- 172.21.15.115:0/2220231003 >> v1:172.21.15.132:6812/1835401922 conn(0x7fda880376e0 legacy=0x7fda88039ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.208+0000 7fdaae78d640 10 monclient: shutdown 2024-01-24T23:23:19.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.208+0000 7fdaae78d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:19.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.208+0000 7fdaae78d640 1 -- 172.21.15.115:0/2220231003 >> v1:172.21.15.115:6790/0 conn(0x7fdaa8133af0 legacy=0x7fdaa813a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.208+0000 7fdaacd2a640 1 -- 172.21.15.115:0/2220231003 reap_dead start 2024-01-24T23:23:19.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.208+0000 7fdaacd2a640 1 -- 172.21.15.115:0/2220231003 reap_dead start 2024-01-24T23:23:19.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.208+0000 7fdaae78d640 1 -- 172.21.15.115:0/2220231003 shutdown_connections 2024-01-24T23:23:19.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.208+0000 7fdaae78d640 1 -- 172.21.15.115:0/2220231003 >> 172.21.15.115:0/2220231003 conn(0x7fdaa80a65e0 msgr2=0x7fdaa812fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:19.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.208+0000 7fdaae78d640 1 -- 172.21.15.115:0/2220231003 shutdown_connections 2024-01-24T23:23:19.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.208+0000 7fdaae78d640 1 -- 172.21.15.115:0/2220231003 wait complete. 2024-01-24T23:23:19.246 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:23:19.246 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-01-24T23:23:19.246 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-01-24T23:23:19.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 10 monclient: get_monmap_and_config 2024-01-24T23:23:19.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 10 monclient: build_initial_monmap 2024-01-24T23:23:19.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 10 monclient: monmap: 2024-01-24T23:23:19.354 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:19.354 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.355 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:19.355084+0000 2024-01-24T23:23:19.355 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:19.355084+0000 2024-01-24T23:23:19.355 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:19.355 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.355 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:19.355 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:19.355 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:19.355 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:19.355 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:19.355 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:19.355 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:19.355 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:19.355 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:19.355 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 1 Processor -- start 2024-01-24T23:23:19.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 1 -- start start 2024-01-24T23:23:19.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 10 monclient: init 2024-01-24T23:23:19.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:19.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 10 monclient: _add_conns ranks=[8,7,2,0,3,5,4,1,6] 2024-01-24T23:23:19.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 10 monclient(hunting): picked mon.noname-e con 0x7f602c133bf0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:19.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 10 monclient(hunting): picked mon.noname-i con 0x7f602c133fd0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:19.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 10 monclient(hunting): picked mon.noname-b con 0x7f602c134430 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:19.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f602c04ed70 con 0x7f602c134430 2024-01-24T23:23:19.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f602c09af30 con 0x7f602c133fd0 2024-01-24T23:23:19.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f602c09b210 con 0x7f602c133bf0 2024-01-24T23:23:19.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:19.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 10 monclient(hunting): authenticate will time out at 1078.277595s 2024-01-24T23:23:19.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f60315f4640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f602c133bf0 0x7f602c09bd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:53016/0 (socket says 172.21.15.115:53016) 2024-01-24T23:23:19.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f60315f4640 1 -- 172.21.15.115:0/3405738742 learned_addr learned my addr 172.21.15.115:0/3405738742 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:19.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6031df5640 1 --1- 172.21.15.115:0/3405738742 >> v1:172.21.15.115:6790/0 conn(0x7f602c134430 0x7f602c13f500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:47704/0 (socket says 172.21.15.115:47704) 2024-01-24T23:23:19.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 1 -- 172.21.15.115:0/3405738742 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3619757518 0 0) 0x7f602c09b210 con 0x7f602c133bf0 2024-01-24T23:23:19.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:19.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:19.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:19.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 10 monclient(hunting): my global_id is 84293 2024-01-24T23:23:19.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 1 -- 172.21.15.115:0/3405738742 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6018003530 con 0x7f602c133bf0 2024-01-24T23:23:19.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 1 -- 172.21.15.115:0/3405738742 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4030369845 0 0) 0x7f602c04ed70 con 0x7f602c134430 2024-01-24T23:23:19.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:19.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:19.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:19.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 10 monclient(hunting): my global_id is 84296 2024-01-24T23:23:19.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 1 -- 172.21.15.115:0/3405738742 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f602c09b210 con 0x7f602c134430 2024-01-24T23:23:19.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 1 -- 172.21.15.115:0/3405738742 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1546692984 0 0) 0x7f6018003530 con 0x7f602c133bf0 2024-01-24T23:23:19.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 1 -- 172.21.15.115:0/3405738742 >> v1:172.21.15.132:6792/0 conn(0x7f602c133fd0 legacy=0x7f602c13bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 1 -- 172.21.15.115:0/3405738742 >> v1:172.21.15.115:6790/0 conn(0x7f602c134430 legacy=0x7f602c13f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 10 monclient: _finish_hunting 0 2024-01-24T23:23:19.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 1 monclient: found mon.noname-e 2024-01-24T23:23:19.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:19.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:23:19.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 1 -- 172.21.15.115:0/3405738742 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f602c09b4f0 con 0x7f602c133bf0 2024-01-24T23:23:19.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 10 monclient: _finish_auth 0 2024-01-24T23:23:19.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 10 monclient: _check_auth_tickets 2024-01-24T23:23:19.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:19.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 1 -- 172.21.15.115:0/3405738742 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f601c0017c0 con 0x7f602c133bf0 2024-01-24T23:23:19.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:23:19.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 10 monclient: dump: 2024-01-24T23:23:19.364 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:19.364 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.364 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.364 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.364 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.365 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.365 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.365 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.365 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.365 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.365 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.365 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.365 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.365 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.365 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.365 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 5 monclient: authenticate success, global_id 84293 2024-01-24T23:23:19.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:19.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 1 -- 172.21.15.115:0/3405738742 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f601c001a60 con 0x7f602c133bf0 2024-01-24T23:23:19.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:19.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 1 -- 172.21.15.115:0/3405738742 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f601c004bf0 con 0x7f602c133bf0 2024-01-24T23:23:19.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:19.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 4 set_mon_vals no callback set 2024-01-24T23:23:19.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:19.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6023fff640 10 monclient: dump: 2024-01-24T23:23:19.366 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:19.366 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.366 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.366 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.366 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.366 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.366 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.366 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.366 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.366 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.367 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.367 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.367 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.367 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.367 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.367 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 10 monclient: shutdown 2024-01-24T23:23:19.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:19.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 1 -- 172.21.15.115:0/3405738742 >> v1:172.21.15.115:6793/0 conn(0x7f602c133bf0 legacy=0x7f602c09bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 1 -- 172.21.15.115:0/3405738742 shutdown_connections 2024-01-24T23:23:19.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 1 -- 172.21.15.115:0/3405738742 >> 172.21.15.115:0/3405738742 conn(0x7f602c0a9910 msgr2=0x7f602c0a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:19.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 1 -- 172.21.15.115:0/3405738742 shutdown_connections 2024-01-24T23:23:19.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 1 -- 172.21.15.115:0/3405738742 wait complete. 2024-01-24T23:23:19.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 10 monclient: build_initial_monmap 2024-01-24T23:23:19.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 10 monclient: monmap: 2024-01-24T23:23:19.368 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:19.368 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:19.368 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:19.368 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:19.368 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:19.368 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:19.368 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:19.368 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:19.368 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:19.368 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:19.368 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:19.368 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:19.368 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:19.368 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:19.368 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:19.369 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 1 Processor -- start 2024-01-24T23:23:19.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 1 -- start start 2024-01-24T23:23:19.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.352+0000 7f6033858640 10 monclient: init 2024-01-24T23:23:19.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6033858640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:19.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6033858640 10 monclient: _add_conns ranks=[5,0,8,4,7,2,6,3,1] 2024-01-24T23:23:19.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6033858640 10 monclient(hunting): picked mon.noname-h con 0x7f602c133bf0 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:19.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6033858640 10 monclient(hunting): picked mon.noname-a con 0x7f602c133fd0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:19.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6033858640 10 monclient(hunting): picked mon.noname-e con 0x7f602c134430 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:19.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6033858640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f602c09c8f0 con 0x7f602c133fd0 2024-01-24T23:23:19.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6033858640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f602c0a5960 con 0x7f602c133bf0 2024-01-24T23:23:19.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6033858640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f602c138b10 con 0x7f602c134430 2024-01-24T23:23:19.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6033858640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:19.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6033858640 10 monclient(hunting): authenticate will time out at 1078.280415s 2024-01-24T23:23:19.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6030df3640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f602c133fd0 0x7f602c13f280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:32804/0 (socket says 172.21.15.115:32804) 2024-01-24T23:23:19.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6030df3640 1 -- 172.21.15.115:0/2468228804 learned_addr learned my addr 172.21.15.115:0/2468228804 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:19.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1803344593 0 0) 0x7f60240012f0 con 0x7f602c134430 2024-01-24T23:23:19.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:19.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:19.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:19.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient(hunting): my global_id is 84302 2024-01-24T23:23:19.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6010003530 con 0x7f602c134430 2024-01-24T23:23:19.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243509418 0 0) 0x7f602c09c8f0 con 0x7f602c133fd0 2024-01-24T23:23:19.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:19.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:19.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:19.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient(hunting): my global_id is 5076 2024-01-24T23:23:19.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60240012f0 con 0x7f602c133fd0 2024-01-24T23:23:19.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 785068954 0 0) 0x7f6010003530 con 0x7f602c134430 2024-01-24T23:23:19.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f602c09c8f0 con 0x7f602c134430 2024-01-24T23:23:19.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2624804622 0 0) 0x7f60240012f0 con 0x7f602c133fd0 2024-01-24T23:23:19.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6010003530 con 0x7f602c133fd0 2024-01-24T23:23:19.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6024001c60 con 0x7f602c134430 2024-01-24T23:23:19.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:23:19.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient(hunting): dump: 2024-01-24T23:23:19.371 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:19.371 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.371 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.371 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.371 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.371 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.371 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.371 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.372 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.372 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.372 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.372 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.372 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.372 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.372 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.372 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:19.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6028001a60 con 0x7f602c133fd0 2024-01-24T23:23:19.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:23:19.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient(hunting): dump: 2024-01-24T23:23:19.373 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:19.373 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.373 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.373 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.373 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.373 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.373 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.373 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.373 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.373 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.373 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.373 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.373 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.373 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.373 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.373 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559932446 0 0) 0x7f602c0a5960 con 0x7f602c133bf0 2024-01-24T23:23:19.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:19.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:19.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:19.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient(hunting): my global_id is 5054 2024-01-24T23:23:19.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60240012f0 con 0x7f602c133bf0 2024-01-24T23:23:19.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2540472621 0 0) 0x7f6010003530 con 0x7f602c133fd0 2024-01-24T23:23:19.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 >> v1:172.21.15.115:6793/0 conn(0x7f602c134430 legacy=0x7f602c1d3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 >> v1:172.21.15.132:6791/0 conn(0x7f602c133bf0 legacy=0x7f602c137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:23:19.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 monclient: found mon.a 2024-01-24T23:23:19.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:19.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:23:19.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f602c13a480 con 0x7f602c133fd0 2024-01-24T23:23:19.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient: _finish_auth 0 2024-01-24T23:23:19.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:23:19.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:19.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6033858640 5 monclient: authenticate success, global_id 5076 2024-01-24T23:23:19.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6033858640 10 monclient: _renew_subs 2024-01-24T23:23:19.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6033858640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:23:19.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6033858640 1 -- 172.21.15.115:0/2468228804 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f602c139110 con 0x7f602c133fd0 2024-01-24T23:23:19.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6033858640 10 monclient: _renew_subs 2024-01-24T23:23:19.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6033858640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:23:19.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6033858640 1 -- 172.21.15.115:0/2468228804 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f602c09f5e0 con 0x7f602c133fd0 2024-01-24T23:23:19.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f60280038d0 con 0x7f602c133fd0 2024-01-24T23:23:19.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:19.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6028003e00 con 0x7f602c133fd0 2024-01-24T23:23:19.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:23:19.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f60237fe640 4 set_mon_vals no callback set 2024-01-24T23:23:19.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient: dump: 2024-01-24T23:23:19.376 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:19.376 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.376 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.376 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.376 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.376 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.376 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.376 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.376 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.376 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.376 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.376 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.376 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.377 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.377 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.377 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6033858640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:19.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6033858640 1 -- 172.21.15.115:0/2468228804 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f600c0065d0 con 0x7f600c002870 2024-01-24T23:23:19.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f6028003f60 con 0x7f602c133fd0 2024-01-24T23:23:19.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(51..51 src has 1..51) v4 ==== 4660+0+0 (unknown 1469567968 0 0) 0x7f6028043aa0 con 0x7f602c133fd0 2024-01-24T23:23:19.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6031df5640 10 monclient: get_auth_request con 0x7f6010037810 auth_method 0 2024-01-24T23:23:19.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1057500541 0 0) 0x7f600c0065d0 con 0x7f600c002870 2024-01-24T23:23:19.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:19.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:19.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:19.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient(hunting): my global_id is 5101 2024-01-24T23:23:19.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6010003530 con 0x7f600c002870 2024-01-24T23:23:19.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2660159448 0 0) 0x7f6010003530 con 0x7f600c002870 2024-01-24T23:23:19.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f600c0065d0 con 0x7f600c002870 2024-01-24T23:23:19.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f601c001780 con 0x7f600c002870 2024-01-24T23:23:19.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:19.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 10 monclient: dump: 2024-01-24T23:23:19.378 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:19.378 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.378 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.378 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.378 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.378 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.378 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.378 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.378 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.378 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.379 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.379 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.379 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.379 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.379 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.379 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3722654091 0 0) 0x7f600c0065d0 con 0x7f600c002870 2024-01-24T23:23:19.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.356+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 --> v1:172.21.15.132:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f600c00a120 con 0x7f600c002870 2024-01-24T23:23:19.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.360+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 <== mon.7 v1:172.21.15.132:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f601c004a20 con 0x7f600c002870 2024-01-24T23:23:19.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.360+0000 7f6021ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:19.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.360+0000 7f6021ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:19.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.360+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 >> v1:172.21.15.132:6792/0 conn(0x7f600c002870 legacy=0x7f600c004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.360+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 >> v1:172.21.15.132:6792/0 conn(0x7f600c002870 legacy=0x7f600c004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:19.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.364+0000 7f6033858640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:23:19.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.364+0000 7f6033858640 1 -- 172.21.15.115:0/2468228804 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f600c009e10 con 0x7f600c0090d0 2024-01-24T23:23:19.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.364+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1513310480 0 0) 0x7f600c009e10 con 0x7f600c0090d0 2024-01-24T23:23:19.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.364+0000 7f6021ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:19.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.364+0000 7f6021ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:19.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.364+0000 7f6021ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:19.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.364+0000 7f6021ffb640 10 monclient(hunting): my global_id is 5110 2024-01-24T23:23:19.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.364+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f600c0065d0 con 0x7f600c0090d0 2024-01-24T23:23:19.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.364+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 161248795 0 0) 0x7f600c0065d0 con 0x7f600c0090d0 2024-01-24T23:23:19.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.364+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f600c009e10 con 0x7f600c0090d0 2024-01-24T23:23:19.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.364+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f601c004c80 con 0x7f600c0090d0 2024-01-24T23:23:19.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.364+0000 7f6021ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.364+0000 7f6021ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:19.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.364+0000 7f6021ffb640 10 monclient: dump: 2024-01-24T23:23:19.382 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:19.382 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.382 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.382 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.382 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.382 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.382 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.382 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.382 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.382 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.383 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.383 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.383 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.383 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.383 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.383 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.364+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 883489967 0 0) 0x7f600c009e10 con 0x7f600c0090d0 2024-01-24T23:23:19.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.364+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 --> v1:172.21.15.132:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f600c00c310 con 0x7f600c0090d0 2024-01-24T23:23:19.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.368+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 <== mon.7 v1:172.21.15.132:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 164826579) 0x7f601c007630 con 0x7f600c0090d0 2024-01-24T23:23:19.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.368+0000 7f6021ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:23:19.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.368+0000 7f6021ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:19.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.368+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 >> v1:172.21.15.132:6792/0 conn(0x7f600c0090d0 legacy=0x7f600c009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.368+0000 7f6021ffb640 1 -- 172.21.15.115:0/2468228804 >> v1:172.21.15.132:6792/0 conn(0x7f600c0090d0 legacy=0x7f600c009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:19.384 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:23:19.384 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:19.384 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:19.384 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:23:19.384 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 16, 2024-01-24T23:23:19.384 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:23:19.384 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:23:19.384 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:23:19.384 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:23:19.384 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:23:19.384 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:23:19.384 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:23:19.385 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:23:19.385 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:23:19.385 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:23:19.385 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:19.385 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 5, 2024-01-24T23:23:19.385 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:19.385 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:23:19.385 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:23:19.385 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:19.385 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:19.385 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:19.385 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:19.385 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:19.385 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:19.385 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:19.386 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:19.386 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:19.386 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:19.386 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:19.386 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:19.386 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:23:19.386 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:23:19.386 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:19.386 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:19.386 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:19.386 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:19.386 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:19.386 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:19.387 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:19.387 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:19.387 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:19.387 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:19.387 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:19.387 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.387 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:23:19.387 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:23:19.387 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:23:19.387 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:23:19.387 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:23:19.387 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:23:19.387 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:19.387 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:19.388 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:23:19.388 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:23:19.388 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:23:19.388 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:23:19.388 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:23:19.388 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:23:19.388 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:23:19.388 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:19.388 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:23:19.388 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:19.388 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:19.388 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:19.388 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:19.388 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:19.389 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:19.389 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:19.389 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:19.389 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:19.389 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:19.389 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:19.389 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:19.389 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:23:19.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.389 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:23:19.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.389 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:19.389 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:19.389 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.389 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.390 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.390 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:23:19.390 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.390 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.390 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.390 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:23:19.390 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:23:19.390 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.390 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.390 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.390 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:19.391 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:19.391 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.391 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.391 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.391 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:23:19.391 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.391 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.391 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.391 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:23:19.391 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:23:19.391 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.391 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.391 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.392 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:19.392 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:19.392 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.392 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.392 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.392 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:23:19.392 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.392 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.392 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.392 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:23:19.393 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:23:19.393 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.393 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.393 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.393 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:19.393 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:19.393 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.393 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.393 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.393 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:23:19.393 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.394 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.394 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.394 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:23:19.394 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:23:19.394 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.394 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.394 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.394 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:19.394 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:19.394 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.394 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.395 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.395 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:23:19.395 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.395 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.395 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.395 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:23:19.395 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:23:19.395 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.395 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.395 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.395 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:19.396 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:19.396 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.396 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.396 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.396 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:23:19.396 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.396 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.396 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.396 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:23:19.396 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:23:19.396 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.396 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.396 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.397 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:19.397 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:19.397 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.397 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.397 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.397 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:23:19.397 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.397 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.397 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.397 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:23:19.398 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:23:19.398 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.398 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.398 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.398 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:19.398 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:19.398 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.398 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.398 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.398 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:23:19.398 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.398 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.399 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.399 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:23:19.399 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:23:19.399 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.399 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.399 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.399 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:19.399 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:19.399 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:19.399 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:19.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.399 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:19.400 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:23:19.400 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:19.400 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.400 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.400 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:23:19.400 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:23:19.400 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:19.400 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:19.400 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:19.400 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.400 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.400 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:23:19.401 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:23:19.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.401 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:19.401 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:19.401 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:19.401 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.401 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:19.401 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:23:19.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.401 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:19.401 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:19.401 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 3 2024-01-24T23:23:19.401 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.401 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:19.402 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:23:19.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:19.402 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:19.402 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:19.402 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 3 2024-01-24T23:23:19.402 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:19.402 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:19.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:19.402 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:23:19.402 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:23:19.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.368+0000 7f6033858640 1 -- 172.21.15.115:0/2468228804 >> v1:172.21.15.132:6812/1835401922 conn(0x7f6010037810 legacy=0x7f6010039cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.368+0000 7f6033858640 10 monclient: shutdown 2024-01-24T23:23:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.368+0000 7f6033858640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.368+0000 7f6033858640 1 -- 172.21.15.115:0/2468228804 >> v1:172.21.15.115:6789/0 conn(0x7f602c133fd0 legacy=0x7f602c13f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.368+0000 7f6031df5640 1 -- 172.21.15.115:0/2468228804 reap_dead start 2024-01-24T23:23:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.368+0000 7f6031df5640 1 -- 172.21.15.115:0/2468228804 reap_dead start 2024-01-24T23:23:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.368+0000 7f6033858640 1 -- 172.21.15.115:0/2468228804 shutdown_connections 2024-01-24T23:23:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.368+0000 7f6033858640 1 -- 172.21.15.115:0/2468228804 >> 172.21.15.115:0/2468228804 conn(0x7f602c0a9910 msgr2=0x7f602c13fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.368+0000 7f6033858640 1 -- 172.21.15.115:0/2468228804 shutdown_connections 2024-01-24T23:23:19.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.368+0000 7f6033858640 1 -- 172.21.15.115:0/2468228804 wait complete. 2024-01-24T23:23:19.404 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:23:19.405 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a', 'f', 'c'] 2024-01-24T23:23:19.405 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-01-24T23:23:19.405 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-01-24T23:23:19.405 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-01-24T23:23:19.405 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-01-24T23:23:19.405 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:23:19.417 INFO:tasks.ceph.mon.a:Stopped 2024-01-24T23:23:19.417 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f 2024-01-24T23:23:19.417 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-01-24T23:23:19.417 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-01-24T23:23:19.417 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:23:19.442 INFO:tasks.ceph.mon.f:Stopped 2024-01-24T23:23:19.442 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-01-24T23:23:19.442 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-01-24T23:23:19.442 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-01-24T23:23:19.442 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:23:19.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bc08b1640 1 -- 172.21.15.132:0/1344075490 >> v1:172.21.15.115:6791/0 conn(0x7f7bbc09c0f0 legacy=0x7f7bbc1d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-01-24T23:23:19.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bc08b1640 1 -- 172.21.15.132:0/1344075490 >> v1:172.21.15.115:6791/0 conn(0x7f7bbc09c0f0 legacy=0x7f7bbc1d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:23:19.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bc08b1640 1 --1- 172.21.15.132:0/1344075490 >> v1:172.21.15.115:6791/0 conn(0x7f7bbc09c0f0 0x7f7bbc1d0a30 :-1 s=OPENED pgs=195 cs=1 l=1).handle_message read tag failed 2024-01-24T23:23:19.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bc08b1640 1 --1- 172.21.15.132:0/1344075490 >> v1:172.21.15.115:6791/0 conn(0x7f7bbc09c0f0 0x7f7bbc1d0a30 :-1 s=OPENED pgs=195 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:23:19.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 10 monclient: ms_handle_reset current mon v1:172.21.15.115:6791/0 2024-01-24T23:23:19.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:19.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 >> v1:172.21.15.115:6791/0 conn(0x7f7bbc09c0f0 legacy=0x7f7bbc1d0a30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:19.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 10 monclient: _add_conns ranks=[2,4,1,3,5,0,7,6,8] 2024-01-24T23:23:19.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 10 monclient(hunting): picked mon.b con 0x7f7ba003f7c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:19.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 10 monclient(hunting): picked mon.c con 0x7f7ba0043540 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:19.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 10 monclient(hunting): picked mon.f con 0x7f7ba00475b0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:19.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7bbc135c70 con 0x7f7ba00475b0 2024-01-24T23:23:19.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7bbc1c8680 con 0x7f7ba003f7c0 2024-01-24T23:23:19.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bc18b3640 1 -- 172.21.15.132:0/1344075490 >> v1:172.21.15.115:6791/0 conn(0x7f7ba0043540 legacy=0x7f7ba00459d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:23:19.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bc10b2640 1 -- 172.21.15.132:0/1344075490 >> v1:172.21.15.132:6789/0 conn(0x7f7ba00475b0 legacy=0x7f7ba0049a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:19.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7ba004d6a0 con 0x7f7ba0043540 2024-01-24T23:23:19.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:19.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2462669033 0 0) 0x7f7bbc1c8680 con 0x7f7ba003f7c0 2024-01-24T23:23:19.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:19.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:19.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 10 monclient(hunting): _init_auth already have auth, reseting 2024-01-24T23:23:19.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 --> v1:172.21.15.115:6790/0 -- auth(proto 2 132 bytes epoch 0) v1 -- 0x7f7ba004c670 con 0x7f7ba003f7c0 2024-01-24T23:23:19.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 233+0+0 (unknown 4002621189 0 0) 0x7f7ba004c670 con 0x7f7ba003f7c0 2024-01-24T23:23:19.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 >> v1:172.21.15.115:6791/0 conn(0x7f7ba0043540 legacy=0x7f7ba00459d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:19.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 >> v1:172.21.15.132:6789/0 conn(0x7f7ba00475b0 legacy=0x7f7ba0049a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:19.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:23:19.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 1 monclient: found mon.b 2024-01-24T23:23:19.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:19.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:19.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+}) v3 -- 0x7f7ba004e860 con 0x7f7ba003f7c0 2024-01-24T23:23:19.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 10 monclient: _send_command 2 [{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:23:19.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:19.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7bbc135c70 con 0x7f7ba003f7c0 2024-01-24T23:23:19.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 10 monclient: _finish_auth 0 2024-01-24T23:23:19.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:23:19.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:19.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7bac014bb0 con 0x7f7ba003f7c0 2024-01-24T23:23:19.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:19.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 10 monclient: dump: 2024-01-24T23:23:19.467 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:19.467 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.467 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.467 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.467 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.467 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:19.467 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.467 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.467 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.467 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.467 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.467 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.467 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.467 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.468 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.468 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:19.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7bac018ca0 con 0x7f7ba003f7c0 2024-01-24T23:23:19.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb1ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:19.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.460+0000 7f7bb37fe640 4 set_mon_vals no callback set 2024-01-24T23:23:19.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:19.464+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f7bac0414a0 con 0x7f7ba003f7c0 2024-01-24T23:23:19.469 INFO:tasks.ceph.mon.c:Stopped 2024-01-24T23:23:19.469 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 6 2024-01-24T23:23:19.469 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-01-24T23:23:19.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc5ab544640 10 monclient: get_monmap_and_config 2024-01-24T23:23:19.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc5ab544640 10 monclient: build_initial_monmap 2024-01-24T23:23:19.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc5ab544640 10 monclient: monmap: 2024-01-24T23:23:19.612 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:19.612 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.612 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:19.612495+0000 2024-01-24T23:23:19.612 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:19.612495+0000 2024-01-24T23:23:19.612 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:19.612 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.612 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:19.612 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:19.613 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:19.613 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:19.613 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:19.613 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:19.613 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:19.613 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:19.613 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:19.613 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc5ab544640 1 Processor -- start 2024-01-24T23:23:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc5ab544640 1 -- start start 2024-01-24T23:23:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc5ab544640 10 monclient: init 2024-01-24T23:23:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc5ab544640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc5ab544640 10 monclient: _add_conns ranks=[3,7,1,4,8,5,6,2,0] 2024-01-24T23:23:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc5ab544640 10 monclient(hunting): picked mon.noname-g con 0x7fc5a4133c10 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc5ab544640 10 monclient(hunting): picked mon.noname-i con 0x7fc5a4136960 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc5ab544640 10 monclient(hunting): picked mon.noname-f con 0x7fc5a413a470 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc5ab544640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a404ed70 con 0x7fc5a413a470 2024-01-24T23:23:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc5ab544640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a409dee0 con 0x7fc5a4133c10 2024-01-24T23:23:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc5ab544640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a409d400 con 0x7fc5a4136960 2024-01-24T23:23:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc5ab544640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc5ab544640 10 monclient(hunting): authenticate will time out at 1078.535119s 2024-01-24T23:23:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc5a9ae1640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fc5a413a470 legacy=0x7fc5a413c860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc5a92e0640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fc5a4133c10 0x7fc5a4133ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.115:38950/0 (socket says 172.21.15.115:38950) 2024-01-24T23:23:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc5a92e0640 1 -- 172.21.15.115:0/2982486628 learned_addr learned my addr 172.21.15.115:0/2982486628 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc5a8adf640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fc5a4136960 0x7fc5a4138d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.115:34930/0 (socket says 172.21.15.115:34930) 2024-01-24T23:23:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc597fff640 1 -- 172.21.15.115:0/2982486628 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2281101213 0 0) 0x7fc5a409dee0 con 0x7fc5a4133c10 2024-01-24T23:23:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc597fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc597fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc597fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.608+0000 7fc597fff640 10 monclient(hunting): my global_id is 5043 2024-01-24T23:23:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 1 -- 172.21.15.115:0/2982486628 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc590003530 con 0x7fc5a4133c10 2024-01-24T23:23:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 1 -- 172.21.15.115:0/2982486628 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 177676720 0 0) 0x7fc5a409d400 con 0x7fc5a4136960 2024-01-24T23:23:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 10 monclient(hunting): my global_id is 5119 2024-01-24T23:23:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 1 -- 172.21.15.115:0/2982486628 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5a409dee0 con 0x7fc5a4136960 2024-01-24T23:23:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 1 -- 172.21.15.115:0/2982486628 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 887567235 0 0) 0x7fc590003530 con 0x7fc5a4133c10 2024-01-24T23:23:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 1 -- 172.21.15.115:0/2982486628 >> v1:172.21.15.132:6792/0 conn(0x7fc5a4136960 legacy=0x7fc5a4138d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 1 -- 172.21.15.115:0/2982486628 >> v1:172.21.15.132:6789/0 conn(0x7fc5a413a470 legacy=0x7fc5a413c860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 10 monclient: _finish_hunting 0 2024-01-24T23:23:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 1 monclient: found mon.noname-g 2024-01-24T23:23:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:23:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 1 -- 172.21.15.115:0/2982486628 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a409f380 con 0x7fc5a4133c10 2024-01-24T23:23:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 10 monclient: _finish_auth 0 2024-01-24T23:23:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 10 monclient: _check_auth_tickets 2024-01-24T23:23:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 1 -- 172.21.15.115:0/2982486628 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc598001b40 con 0x7fc5a4133c10 2024-01-24T23:23:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 10 monclient: dump: 2024-01-24T23:23:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.624 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 5 monclient: authenticate success, global_id 5043 2024-01-24T23:23:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 1 -- 172.21.15.115:0/2982486628 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc598001d60 con 0x7fc5a4133c10 2024-01-24T23:23:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 4 set_mon_vals no callback set 2024-01-24T23:23:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 1 -- 172.21.15.115:0/2982486628 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc598004f70 con 0x7fc5a4133c10 2024-01-24T23:23:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc597fff640 10 monclient: dump: 2024-01-24T23:23:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.625 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 10 monclient: shutdown 2024-01-24T23:23:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 1 -- 172.21.15.115:0/2982486628 >> v1:172.21.15.132:6790/0 conn(0x7fc5a4133c10 legacy=0x7fc5a4133ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 1 -- 172.21.15.115:0/2982486628 shutdown_connections 2024-01-24T23:23:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 1 -- 172.21.15.115:0/2982486628 >> 172.21.15.115:0/2982486628 conn(0x7fc5a40a5730 msgr2=0x7fc5a40a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 1 -- 172.21.15.115:0/2982486628 shutdown_connections 2024-01-24T23:23:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 1 -- 172.21.15.115:0/2982486628 wait complete. 2024-01-24T23:23:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 10 monclient: build_initial_monmap 2024-01-24T23:23:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 10 monclient: monmap: 2024-01-24T23:23:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:19.627 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 1 Processor -- start 2024-01-24T23:23:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 1 -- start start 2024-01-24T23:23:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 10 monclient: init 2024-01-24T23:23:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 10 monclient: _add_conns ranks=[2,0,1,6,7,3,8,5,4] 2024-01-24T23:23:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 10 monclient(hunting): picked mon.noname-b con 0x7fc5a4133c10 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 10 monclient(hunting): picked mon.noname-a con 0x7fc5a4136960 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 10 monclient(hunting): picked mon.noname-f con 0x7fc5a413a470 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a41c9dd0 con 0x7fc5a4136960 2024-01-24T23:23:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a406ee00 con 0x7fc5a413a470 2024-01-24T23:23:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a406e560 con 0x7fc5a4133c10 2024-01-24T23:23:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 10 monclient(hunting): authenticate will time out at 1078.538985s 2024-01-24T23:23:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5a8adf640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fc5a4136960 legacy=0x7fc5a41ce1b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5a92e0640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fc5a4133c10 0x7fc5a4071340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:47722/0 (socket says 172.21.15.115:47722) 2024-01-24T23:23:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5a92e0640 1 -- 172.21.15.115:0/2631746027 learned_addr learned my addr 172.21.15.115:0/2631746027 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5a9ae1640 1 -- 172.21.15.115:0/2631746027 >> v1:172.21.15.132:6789/0 conn(0x7fc5a413a470 legacy=0x7fc5a41d18e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 1 -- 172.21.15.115:0/2631746027 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2731322731 0 0) 0x7fc5a406e560 con 0x7fc5a4133c10 2024-01-24T23:23:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 10 monclient(hunting): my global_id is 84305 2024-01-24T23:23:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 1 -- 172.21.15.115:0/2631746027 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc588003530 con 0x7fc5a4133c10 2024-01-24T23:23:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 1 -- 172.21.15.115:0/2631746027 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1912293258 0 0) 0x7fc588003530 con 0x7fc5a4133c10 2024-01-24T23:23:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 1 -- 172.21.15.115:0/2631746027 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc5a406e560 con 0x7fc5a4133c10 2024-01-24T23:23:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 1 -- 172.21.15.115:0/2631746027 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc598001b40 con 0x7fc5a4133c10 2024-01-24T23:23:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:23:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 10 monclient(hunting): dump: 2024-01-24T23:23:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.630 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 1 -- 172.21.15.115:0/2631746027 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 17785465 0 0) 0x7fc5a406e560 con 0x7fc5a4133c10 2024-01-24T23:23:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 1 -- 172.21.15.115:0/2631746027 >> v1:172.21.15.132:6789/0 conn(0x7fc5a413a470 legacy=0x7fc5a41d18e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 1 -- 172.21.15.115:0/2631746027 >> v1:172.21.15.115:6789/0 conn(0x7fc5a4136960 legacy=0x7fc5a41ce1b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:23:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 1 monclient: found mon.b 2024-01-24T23:23:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 1 -- 172.21.15.115:0/2631746027 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a406e840 con 0x7fc5a4133c10 2024-01-24T23:23:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 10 monclient: _finish_auth 0 2024-01-24T23:23:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:23:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 5 monclient: authenticate success, global_id 84305 2024-01-24T23:23:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 10 monclient: _renew_subs 2024-01-24T23:23:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 1 -- 172.21.15.115:0/2631746027 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a406eb20 con 0x7fc5a4133c10 2024-01-24T23:23:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 10 monclient: _renew_subs 2024-01-24T23:23:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc5ab544640 1 -- 172.21.15.115:0/2631746027 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a4139470 con 0x7fc5a4133c10 2024-01-24T23:23:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 1 -- 172.21.15.115:0/2631746027 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc598005240 con 0x7fc5a4133c10 2024-01-24T23:23:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 1 -- 172.21.15.115:0/2631746027 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc5980056f0 con 0x7fc5a4133c10 2024-01-24T23:23:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.612+0000 7fc595ffb640 10 monclient: dump: 2024-01-24T23:23:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:19.633 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:19.633 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:19.633 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:19.633 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:19.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.616+0000 7fc596ffd640 4 set_mon_vals no callback set 2024-01-24T23:23:19.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.616+0000 7fc595ffb640 1 -- 172.21.15.115:0/2631746027 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fc598010170 con 0x7fc5a4133c10 2024-01-24T23:23:19.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.616+0000 7fc5ab544640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:19.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.616+0000 7fc5ab544640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:19.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.616+0000 7fc5ab544640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:19.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.616+0000 7fc5ab544640 1 -- 172.21.15.115:0/2631746027 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc584005090 con 0x7fc5a4133c10 2024-01-24T23:23:19.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.616+0000 7fc595ffb640 1 -- 172.21.15.115:0/2631746027 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(51..51 src has 1..51) v4 ==== 4660+0+0 (unknown 1469567968 0 0) 0x7fc5980441b0 con 0x7fc5a4133c10 2024-01-24T23:23:19.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.616+0000 7fc5a8adf640 10 monclient: get_auth_request con 0x7fc588036ef0 auth_method 0 2024-01-24T23:23:19.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.620+0000 7fc595ffb640 1 -- 172.21.15.115:0/2631746027 <== mon.2 v1:172.21.15.115:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fc598014fa0 con 0x7fc5a4133c10 2024-01-24T23:23:19.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.620+0000 7fc595ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:19.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.620+0000 7fc595ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:19.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.816+0000 7fc5ab544640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-01-24T23:23:19.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.816+0000 7fc5ab544640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-01-24T23:23:19.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.816+0000 7fc5ab544640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:19.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.816+0000 7fc5ab544640 1 -- 172.21.15.115:0/2631746027 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc5840058c0 con 0x7fc5a4133c10 2024-01-24T23:23:19.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.816+0000 7fc595ffb640 1 -- 172.21.15.115:0/2631746027 <== mon.2 v1:172.21.15.115:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2744 (unknown 254746657 0 2373440945) 0x7fc598013640 con 0x7fc5a4133c10 2024-01-24T23:23:19.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.816+0000 7fc595ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-01-24T23:23:19.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.816+0000 7fc595ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:19.819 INFO:teuthology.orchestra.run.smithi115.stdout:{"election_epoch":16,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540701272860131327","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","modified":"2024-01-24T23:20:20.557246Z","created":"2024-01-24T23:20:20.557246Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6790","nonce":0}]},"addr":"172.21.15.115:6790/0","public_addr":"172.21.15.115:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6790","nonce":0}]},"addr":"172.21.15.132:6790/0","public_addr":"172.21.15.132:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6791","nonce":0}]},"addr":"172.21.15.115:6791/0","public_addr":"172.21.15.115:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6791","nonce":0}]},"addr":"172.21.15.132:6791/0","public_addr":"172.21.15.132:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6792","nonce":0}]},"addr":"172.21.15.115:6792/0","public_addr":"172.21.15.115:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6792","nonce":0}]},"addr":"172.21.15.132:6792/0","public_addr":"172.21.15.132:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6793","nonce":0}]},"addr":"172.21.15.115:6793/0","public_addr":"172.21.15.115:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-01-24T23:23:19.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.816+0000 7fc5ab544640 1 -- 172.21.15.115:0/2631746027 >> v1:172.21.15.132:6812/1835401922 conn(0x7fc588036ef0 legacy=0x7fc5880393b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.816+0000 7fc5ab544640 10 monclient: shutdown 2024-01-24T23:23:19.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.816+0000 7fc5ab544640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:19.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.816+0000 7fc5ab544640 1 -- 172.21.15.115:0/2631746027 >> v1:172.21.15.115:6790/0 conn(0x7fc5a4133c10 legacy=0x7fc5a4071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:19.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.816+0000 7fc5ab544640 1 -- 172.21.15.115:0/2631746027 shutdown_connections 2024-01-24T23:23:19.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.816+0000 7fc5ab544640 1 -- 172.21.15.115:0/2631746027 >> 172.21.15.115:0/2631746027 conn(0x7fc5a40a5730 msgr2=0x7fc5a40a2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:19.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.816+0000 7fc5ab544640 1 -- 172.21.15.115:0/2631746027 shutdown_connections 2024-01-24T23:23:19.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:19.816+0000 7fc5ab544640 1 -- 172.21.15.115:0/2631746027 wait complete. 2024-01-24T23:23:21.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:21.668+0000 7f7bb0ff9640 10 monclient: tick 2024-01-24T23:23:21.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:21.668+0000 7f7bb0ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:21.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:21.668+0000 7f7bb0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:21.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:21.668+0000 7f7bb0ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:22.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:22.668+0000 7f7bb0ff9640 10 monclient: tick 2024-01-24T23:23:22.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:22.668+0000 7f7bb0ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:22.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:22.668+0000 7f7bb0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:22.836 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-01-24T23:23:22.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.960+0000 7f48a7cb7640 10 monclient: get_monmap_and_config 2024-01-24T23:23:22.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.960+0000 7f48a7cb7640 10 monclient: build_initial_monmap 2024-01-24T23:23:22.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.960+0000 7f48a7cb7640 10 monclient: monmap: 2024-01-24T23:23:22.964 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:22.964 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:22.964 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:22.964449+0000 2024-01-24T23:23:22.964 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:22.964449+0000 2024-01-24T23:23:22.964 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:22.964 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:22.964 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:22.964 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:22.964 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:22.965 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:22.965 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:22.965 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:22.965 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:22.965 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:22.965 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:22.965 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:22.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.960+0000 7f48a7cb7640 1 Processor -- start 2024-01-24T23:23:22.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.960+0000 7f48a7cb7640 1 -- start start 2024-01-24T23:23:22.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.960+0000 7f48a7cb7640 10 monclient: init 2024-01-24T23:23:22.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.960+0000 7f48a7cb7640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:22.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.960+0000 7f48a7cb7640 10 monclient: _add_conns ranks=[0,5,3,4,2,7,6,1,8] 2024-01-24T23:23:22.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.960+0000 7f48a7cb7640 10 monclient(hunting): picked mon.noname-a con 0x7f48a0133c10 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:22.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.960+0000 7f48a7cb7640 10 monclient(hunting): picked mon.noname-h con 0x7f48a0136960 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:22.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.960+0000 7f48a7cb7640 10 monclient(hunting): picked mon.noname-g con 0x7f48a013a470 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:22.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.960+0000 7f48a7cb7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a004ed70 con 0x7f48a0133c10 2024-01-24T23:23:22.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.960+0000 7f48a7cb7640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a009dee0 con 0x7f48a013a470 2024-01-24T23:23:22.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.960+0000 7f48a7cb7640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a009d400 con 0x7f48a0136960 2024-01-24T23:23:22.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.960+0000 7f48a7cb7640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:22.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.960+0000 7f48a7cb7640 10 monclient(hunting): authenticate will time out at 1081.887528s 2024-01-24T23:23:22.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.960+0000 7f48a5a53640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f48a0133c10 legacy=0x7f48a0133ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:22.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.960+0000 7f48a6254640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f48a013a470 0x7f48a013c860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.115:58482/0 (socket says 172.21.15.115:58482) 2024-01-24T23:23:22.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.960+0000 7f48a6254640 1 -- 172.21.15.115:0/305185121 learned_addr learned my addr 172.21.15.115:0/305185121 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:22.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 1 -- 172.21.15.115:0/305185121 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3925337993 0 0) 0x7f48a009d400 con 0x7f48a0136960 2024-01-24T23:23:22.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 10 monclient(hunting): _negotiate 2024-01-24T23:23:22.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:22.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:22.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 10 monclient(hunting): my global_id is 5063 2024-01-24T23:23:22.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 1 -- 172.21.15.115:0/305185121 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f488c003530 con 0x7f48a0136960 2024-01-24T23:23:22.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 1 -- 172.21.15.115:0/305185121 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3561101554 0 0) 0x7f48a009dee0 con 0x7f48a013a470 2024-01-24T23:23:22.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 10 monclient(hunting): _negotiate 2024-01-24T23:23:22.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:22.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:22.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 10 monclient(hunting): my global_id is 5052 2024-01-24T23:23:22.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 1 -- 172.21.15.115:0/305185121 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48a009d400 con 0x7f48a013a470 2024-01-24T23:23:22.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 1 -- 172.21.15.115:0/305185121 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3497595539 0 0) 0x7f488c003530 con 0x7f48a0136960 2024-01-24T23:23:22.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 1 -- 172.21.15.115:0/305185121 >> v1:172.21.15.132:6790/0 conn(0x7f48a013a470 legacy=0x7f48a013c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:22.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 1 -- 172.21.15.115:0/305185121 >> v1:172.21.15.115:6789/0 conn(0x7f48a0133c10 legacy=0x7f48a0133ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:22.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 10 monclient: _finish_hunting 0 2024-01-24T23:23:22.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 1 monclient: found mon.noname-h 2024-01-24T23:23:22.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:22.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:23:22.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 1 -- 172.21.15.115:0/305185121 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a009f380 con 0x7f48a0136960 2024-01-24T23:23:22.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 10 monclient: _finish_auth 0 2024-01-24T23:23:22.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 10 monclient: _check_auth_tickets 2024-01-24T23:23:22.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:22.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 1 -- 172.21.15.115:0/305185121 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f48900017c0 con 0x7f48a0136960 2024-01-24T23:23:22.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:22.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:22.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 10 monclient: dump: 2024-01-24T23:23:22.971 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:22.971 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:22.971 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:22.971 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:22.972 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:22.972 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:22.972 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:22.972 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:22.972 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:22.972 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:22.972 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:22.972 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:22.972 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:22.972 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:22.972 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:22.972 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:22.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 5 monclient: authenticate success, global_id 5063 2024-01-24T23:23:22.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:22.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 1 -- 172.21.15.115:0/305185121 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4890001a20 con 0x7f48a0136960 2024-01-24T23:23:22.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:22.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 1 -- 172.21.15.115:0/305185121 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4890003960 con 0x7f48a0136960 2024-01-24T23:23:22.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:22.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:22.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a4a51640 10 monclient: dump: 2024-01-24T23:23:22.973 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:22.973 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:22.973 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:22.973 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:22.973 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:22.973 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:22.973 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:22.973 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:22.973 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:22.973 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:22.974 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:22.974 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:22.974 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:22.974 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:22.974 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:22.974 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:22.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:22.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 4 set_mon_vals no callback set 2024-01-24T23:23:22.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 10 monclient: shutdown 2024-01-24T23:23:22.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:22.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 1 -- 172.21.15.115:0/305185121 >> v1:172.21.15.132:6791/0 conn(0x7f48a0136960 legacy=0x7f48a0138d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:22.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 1 -- 172.21.15.115:0/305185121 shutdown_connections 2024-01-24T23:23:22.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 1 -- 172.21.15.115:0/305185121 >> 172.21.15.115:0/305185121 conn(0x7f48a00a5730 msgr2=0x7f48a00a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:22.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 1 -- 172.21.15.115:0/305185121 shutdown_connections 2024-01-24T23:23:22.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 1 -- 172.21.15.115:0/305185121 wait complete. 2024-01-24T23:23:22.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 10 monclient: build_initial_monmap 2024-01-24T23:23:22.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 10 monclient: monmap: 2024-01-24T23:23:22.975 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:22.975 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:22.975 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:22.975 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:22.975 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:22.975 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:22.975 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:22.975 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:22.975 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:22.975 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:22.975 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:22.975 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:22.976 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:22.976 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:22.976 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:22.976 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:22.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 1 Processor -- start 2024-01-24T23:23:22.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 1 -- start start 2024-01-24T23:23:22.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 10 monclient: init 2024-01-24T23:23:22.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:22.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 10 monclient: _add_conns ranks=[2,3,6,5,7,1,8,4,0] 2024-01-24T23:23:22.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 10 monclient(hunting): picked mon.noname-b con 0x7f48a0133c10 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:22.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 10 monclient(hunting): picked mon.noname-g con 0x7f48a0136960 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:22.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 10 monclient(hunting): picked mon.noname-d con 0x7f48a013a470 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:22.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a006e970 con 0x7f48a0133c10 2024-01-24T23:23:22.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a01334a0 con 0x7f48a0136960 2024-01-24T23:23:22.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a006e320 con 0x7f48a013a470 2024-01-24T23:23:22.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:22.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a7cb7640 10 monclient(hunting): authenticate will time out at 1081.891661s 2024-01-24T23:23:22.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a6254640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f48a013a470 0x7f48a01cf660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:49892/0 (socket says 172.21.15.115:49892) 2024-01-24T23:23:22.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.964+0000 7f48a6254640 1 -- 172.21.15.115:0/2752432699 learned_addr learned my addr 172.21.15.115:0/2752432699 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:22.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 1 -- 172.21.15.115:0/2752432699 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 826888346 0 0) 0x7f48a006e320 con 0x7f48a013a470 2024-01-24T23:23:22.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:22.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:22.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:22.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient(hunting): my global_id is 5109 2024-01-24T23:23:22.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 1 -- 172.21.15.115:0/2752432699 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4884003530 con 0x7f48a013a470 2024-01-24T23:23:22.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 1 -- 172.21.15.115:0/2752432699 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3854949564 0 0) 0x7f48a006e970 con 0x7f48a0133c10 2024-01-24T23:23:22.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:22.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:22.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:22.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient(hunting): my global_id is 84314 2024-01-24T23:23:22.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 1 -- 172.21.15.115:0/2752432699 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48a006e320 con 0x7f48a0133c10 2024-01-24T23:23:22.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 1 -- 172.21.15.115:0/2752432699 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2531394357 0 0) 0x7f4884003530 con 0x7f48a013a470 2024-01-24T23:23:22.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 1 -- 172.21.15.115:0/2752432699 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f48a006e970 con 0x7f48a013a470 2024-01-24T23:23:22.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 1 -- 172.21.15.115:0/2752432699 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3276539520 0 0) 0x7f48a01334a0 con 0x7f48a0136960 2024-01-24T23:23:22.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:22.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:22.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:22.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient(hunting): my global_id is 5061 2024-01-24T23:23:22.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 1 -- 172.21.15.115:0/2752432699 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4884003530 con 0x7f48a0136960 2024-01-24T23:23:22.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 1 -- 172.21.15.115:0/2752432699 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4898001d40 con 0x7f48a013a470 2024-01-24T23:23:22.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:22.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:23:22.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient(hunting): dump: 2024-01-24T23:23:22.982 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:22.982 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:22.982 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:22.983 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:22.983 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:22.983 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:22.983 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:22.983 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:22.983 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:22.983 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:22.983 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:22.983 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:22.983 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:22.983 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:22.983 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:22.983 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:22.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:22.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 1 -- 172.21.15.115:0/2752432699 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1487497258 0 0) 0x7f48a006e320 con 0x7f48a0133c10 2024-01-24T23:23:22.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 1 -- 172.21.15.115:0/2752432699 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f48a01334a0 con 0x7f48a0133c10 2024-01-24T23:23:22.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 1 -- 172.21.15.115:0/2752432699 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f489c0041f0 con 0x7f48a0133c10 2024-01-24T23:23:22.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:22.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:22.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient(hunting): dump: 2024-01-24T23:23:22.984 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:22.984 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:22.984 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:22.984 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:22.984 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:22.984 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:22.984 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:22.984 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:22.984 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:22.985 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:22.985 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:22.985 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:22.985 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:22.985 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:22.985 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:22.985 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:22.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 1 -- 172.21.15.115:0/2752432699 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3139185932 0 0) 0x7f48a006e970 con 0x7f48a013a470 2024-01-24T23:23:22.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 1 -- 172.21.15.115:0/2752432699 >> v1:172.21.15.132:6790/0 conn(0x7f48a0136960 legacy=0x7f48a01cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:22.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 1 -- 172.21.15.115:0/2752432699 >> v1:172.21.15.115:6790/0 conn(0x7f48a0133c10 legacy=0x7f48a0071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:22.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient: _finish_hunting 0 2024-01-24T23:23:22.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 1 monclient: found mon.d 2024-01-24T23:23:22.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:22.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:23:22.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 1 -- 172.21.15.115:0/2752432699 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a006e600 con 0x7f48a013a470 2024-01-24T23:23:22.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient: _finish_auth 0 2024-01-24T23:23:22.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient: _check_auth_tickets 2024-01-24T23:23:22.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:22.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48a7cb7640 5 monclient: authenticate success, global_id 5109 2024-01-24T23:23:22.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48a7cb7640 10 monclient: _renew_subs 2024-01-24T23:23:22.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48a7cb7640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:23:22.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48a7cb7640 1 -- 172.21.15.115:0/2752432699 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a0132d20 con 0x7f48a013a470 2024-01-24T23:23:22.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48a7cb7640 10 monclient: _renew_subs 2024-01-24T23:23:22.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48a7cb7640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:23:22.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48a7cb7640 1 -- 172.21.15.115:0/2752432699 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a0071810 con 0x7f48a013a470 2024-01-24T23:23:22.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 1 -- 172.21.15.115:0/2752432699 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f48980020f0 con 0x7f48a013a470 2024-01-24T23:23:22.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:22.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 1 -- 172.21.15.115:0/2752432699 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4898005160 con 0x7f48a013a470 2024-01-24T23:23:22.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:22.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f4897fff640 4 set_mon_vals no callback set 2024-01-24T23:23:22.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:22.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 10 monclient: dump: 2024-01-24T23:23:22.987 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:22.987 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:22.987 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:22.987 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:22.987 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:22.987 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:22.987 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:22.987 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:22.987 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:22.987 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:22.987 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:22.987 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:22.987 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:22.988 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:22.988 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:22.988 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:22.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48a7cb7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:22.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48a7cb7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:22.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48a7cb7640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:23:22.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48a7cb7640 1 -- 172.21.15.115:0/2752432699 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4888005090 con 0x7f48a013a470 2024-01-24T23:23:22.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 1 -- 172.21.15.115:0/2752432699 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f489800fbe0 con 0x7f48a013a470 2024-01-24T23:23:22.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48967fc640 1 -- 172.21.15.115:0/2752432699 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(51..51 src has 1..51) v4 ==== 4660+0+0 (unknown 1469567968 0 0) 0x7f4898043ec0 con 0x7f48a013a470 2024-01-24T23:23:22.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.968+0000 7f48a5a53640 10 monclient: get_auth_request con 0x7f4884037850 auth_method 0 2024-01-24T23:23:22.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.972+0000 7f48967fc640 1 -- 172.21.15.115:0/2752432699 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f48980116d0 con 0x7f48a013a470 2024-01-24T23:23:22.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.972+0000 7f48967fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:22.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:22.972+0000 7f48967fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:23.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:23.184+0000 7f48a7cb7640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-01-24T23:23:23.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:23.184+0000 7f48a7cb7640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-01-24T23:23:23.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:23.184+0000 7f48a7cb7640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:23:23.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:23.184+0000 7f48a7cb7640 1 -- 172.21.15.115:0/2752432699 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f48880058c0 con 0x7f48a013a470 2024-01-24T23:23:23.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:23.184+0000 7f48967fc640 1 -- 172.21.15.115:0/2752432699 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2744 (unknown 254746657 0 558935103) 0x7f48980131c0 con 0x7f48a013a470 2024-01-24T23:23:23.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:23.184+0000 7f48967fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-01-24T23:23:23.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:23.184+0000 7f48967fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:23.189 INFO:teuthology.orchestra.run.smithi115.stdout:{"election_epoch":16,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4540701272860131327","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","modified":"2024-01-24T23:20:20.557246Z","created":"2024-01-24T23:20:20.557246Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6790","nonce":0}]},"addr":"172.21.15.115:6790/0","public_addr":"172.21.15.115:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6790","nonce":0}]},"addr":"172.21.15.132:6790/0","public_addr":"172.21.15.132:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6791","nonce":0}]},"addr":"172.21.15.115:6791/0","public_addr":"172.21.15.115:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6791","nonce":0}]},"addr":"172.21.15.132:6791/0","public_addr":"172.21.15.132:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6792","nonce":0}]},"addr":"172.21.15.115:6792/0","public_addr":"172.21.15.115:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6792","nonce":0}]},"addr":"172.21.15.132:6792/0","public_addr":"172.21.15.132:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6793","nonce":0}]},"addr":"172.21.15.115:6793/0","public_addr":"172.21.15.115:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-01-24T23:23:23.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:23.188+0000 7f48a7cb7640 1 -- 172.21.15.115:0/2752432699 >> v1:172.21.15.132:6812/1835401922 conn(0x7f4884037850 legacy=0x7f4884039d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:23.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:23.188+0000 7f48a7cb7640 10 monclient: shutdown 2024-01-24T23:23:23.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:23.188+0000 7f48a7cb7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:23.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:23.188+0000 7f48a7cb7640 1 -- 172.21.15.115:0/2752432699 >> v1:172.21.15.115:6792/0 conn(0x7f48a013a470 legacy=0x7f48a01cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:23.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:23.188+0000 7f48a7cb7640 1 -- 172.21.15.115:0/2752432699 shutdown_connections 2024-01-24T23:23:23.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:23.188+0000 7f48a7cb7640 1 -- 172.21.15.115:0/2752432699 >> 172.21.15.115:0/2752432699 conn(0x7f48a00a5730 msgr2=0x7f48a009cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:23.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:23.188+0000 7f48a7cb7640 1 -- 172.21.15.115:0/2752432699 shutdown_connections 2024-01-24T23:23:23.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:23.188+0000 7f48a7cb7640 1 -- 172.21.15.115:0/2752432699 wait complete. 2024-01-24T23:23:23.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:23.668+0000 7f7bb0ff9640 10 monclient: tick 2024-01-24T23:23:23.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:23.668+0000 7f7bb0ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:23.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:23.668+0000 7f7bb0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:24.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:24.668+0000 7f7bb0ff9640 10 monclient: tick 2024-01-24T23:23:24.700 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:24.668+0000 7f7bb0ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:24.700 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:24.668+0000 7f7bb0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:25.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:25.668+0000 7f7bb0ff9640 10 monclient: tick 2024-01-24T23:23:25.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:25.668+0000 7f7bb0ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:25.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:25.668+0000 7f7bb0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:26.209 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-01-24T23:23:26.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:26.332+0000 7fbeb80f2640 10 monclient: get_monmap_and_config 2024-01-24T23:23:26.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:26.332+0000 7fbeb80f2640 10 monclient: build_initial_monmap 2024-01-24T23:23:26.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:26.332+0000 7fbeb80f2640 10 monclient: monmap: 2024-01-24T23:23:26.336 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:26.336 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:26.336 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:26.336568+0000 2024-01-24T23:23:26.336 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:26.336568+0000 2024-01-24T23:23:26.337 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:26.337 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:26.337 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:26.337 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:26.337 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:26.337 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:26.337 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:26.337 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:26.337 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:26.337 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:26.337 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:26.337 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:26.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:26.332+0000 7fbeb80f2640 1 Processor -- start 2024-01-24T23:23:26.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:26.332+0000 7fbeb80f2640 1 -- start start 2024-01-24T23:23:26.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:26.332+0000 7fbeb80f2640 10 monclient: init 2024-01-24T23:23:26.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:26.332+0000 7fbeb80f2640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:26.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:26.332+0000 7fbeb80f2640 10 monclient: _add_conns ranks=[0,3,6,7,8,2,4,1,5] 2024-01-24T23:23:26.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:26.332+0000 7fbeb80f2640 10 monclient(hunting): picked mon.noname-a con 0x7fbeb0134740 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:26.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:26.332+0000 7fbeb80f2640 10 monclient(hunting): picked mon.noname-g con 0x7fbeb009ec80 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:26.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:26.332+0000 7fbeb80f2640 10 monclient(hunting): picked mon.noname-d con 0x7fbeb0135230 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:26.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:26.332+0000 7fbeb80f2640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb004ed70 con 0x7fbeb0134740 2024-01-24T23:23:26.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:26.332+0000 7fbeb80f2640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb00a0d90 con 0x7fbeb009ec80 2024-01-24T23:23:26.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:26.332+0000 7fbeb80f2640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb009e7c0 con 0x7fbeb0135230 2024-01-24T23:23:26.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:26.332+0000 7fbeb80f2640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:26.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:26.332+0000 7fbeb80f2640 10 monclient(hunting): authenticate will time out at 1085.259054s 2024-01-24T23:23:26.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:26.332+0000 7fbeb5e8e640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fbeb0134740 legacy=0x7fbeb0134b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:26.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:26.332+0000 7fbeb668f640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fbeb0135230 0x7fbeb013d450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:49896/0 (socket says 172.21.15.115:49896) 2024-01-24T23:23:26.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:26.332+0000 7fbeb668f640 1 -- 172.21.15.115:0/4260601336 learned_addr learned my addr 172.21.15.115:0/4260601336 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:26.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:26.536+0000 7fbeb5e8e640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.115:6789/0 conn(0x7fbeb0134740 legacy=0x7fbeb0134b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:26.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:26.668+0000 7f7bb0ff9640 10 monclient: tick 2024-01-24T23:23:26.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:26.668+0000 7f7bb0ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:26.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:26.668+0000 7f7bb0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:26.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:26.940+0000 7fbeb5e8e640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.115:6789/0 conn(0x7fbeb0134740 legacy=0x7fbeb0134b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:27.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:27.668+0000 7f7bb0ff9640 10 monclient: tick 2024-01-24T23:23:27.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:27.668+0000 7f7bb0ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:27.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:27.668+0000 7f7bb0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:27.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:27.744+0000 7fbeb5e8e640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.115:6789/0 conn(0x7fbeb0134740 legacy=0x7fbeb0134b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:28.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:28.668+0000 7f7bb0ff9640 10 monclient: tick 2024-01-24T23:23:28.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:28.668+0000 7f7bb0ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:28.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:28.668+0000 7f7bb0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:29.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:29.332+0000 7fbea77fe640 10 monclient(hunting): tick 2024-01-24T23:23:29.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:29.332+0000 7fbea77fe640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:23:29.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:29.332+0000 7fbea77fe640 1 monclient(hunting): continuing hunt 2024-01-24T23:23:29.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:29.332+0000 7fbea77fe640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:23:29.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:29.332+0000 7fbea77fe640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.115:6792/0 conn(0x7fbeb0135230 legacy=0x7fbeb013d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:29.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:29.332+0000 7fbea77fe640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.132:6790/0 conn(0x7fbeb009ec80 legacy=0x7fbeb0139d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:29.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:29.332+0000 7fbea77fe640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.115:6789/0 conn(0x7fbeb0134740 legacy=0x7fbeb0134b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:29.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:29.332+0000 7fbea77fe640 10 monclient: _add_conns ranks=[5,4,7,1,2,8] 2024-01-24T23:23:29.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:29.332+0000 7fbea77fe640 10 monclient(hunting): picked mon.noname-h con 0x7fbe940037d0 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:29.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:29.332+0000 7fbea77fe640 10 monclient(hunting): picked mon.noname-c con 0x7fbe94007460 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:29.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:29.332+0000 7fbea77fe640 10 monclient(hunting): picked mon.noname-i con 0x7fbe9400b1d0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:29.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:29.332+0000 7fbea77fe640 1 -- 172.21.15.115:0/4260601336 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb00a1070 con 0x7fbe94007460 2024-01-24T23:23:29.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:29.332+0000 7fbea77fe640 1 -- 172.21.15.115:0/4260601336 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb004ed70 con 0x7fbe940037d0 2024-01-24T23:23:29.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:29.332+0000 7fbea77fe640 1 -- 172.21.15.115:0/4260601336 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe94011030 con 0x7fbe9400b1d0 2024-01-24T23:23:29.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:29.332+0000 7fbea77fe640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:29.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:29.332+0000 7fbeb568d640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.115:6791/0 conn(0x7fbe94007460 legacy=0x7fbe94009900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:23:29.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:29.536+0000 7fbeb568d640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.115:6791/0 conn(0x7fbe94007460 legacy=0x7fbe94009900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:23:29.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:29.668+0000 7f7bb0ff9640 10 monclient: tick 2024-01-24T23:23:29.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:29.668+0000 7f7bb0ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:29.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:29.668+0000 7f7bb0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:29.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:29.940+0000 7fbeb568d640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.115:6791/0 conn(0x7fbe94007460 legacy=0x7fbe94009900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:23:30.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:30.667+0000 7f7bb0ff9640 10 monclient: tick 2024-01-24T23:23:30.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:30.667+0000 7f7bb0ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:30.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:30.667+0000 7f7bb0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:30.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:30.744+0000 7fbeb568d640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.115:6791/0 conn(0x7fbe94007460 legacy=0x7fbe94009900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:23:31.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:31.667+0000 7f7bb0ff9640 10 monclient: tick 2024-01-24T23:23:31.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:31.667+0000 7f7bb0ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:31.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:31.667+0000 7f7bb0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:31.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:31.667+0000 7f7bb0ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:32.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:32.332+0000 7fbea77fe640 10 monclient(hunting): tick 2024-01-24T23:23:32.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:32.332+0000 7fbea77fe640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:23:32.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:32.332+0000 7fbea77fe640 1 monclient(hunting): continuing hunt 2024-01-24T23:23:32.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:32.332+0000 7fbea77fe640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:23:32.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:32.332+0000 7fbea77fe640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.132:6792/0 conn(0x7fbe9400b1d0 legacy=0x7fbe9400d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:32.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:32.332+0000 7fbea77fe640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.132:6791/0 conn(0x7fbe940037d0 legacy=0x7fbe94005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:32.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:32.332+0000 7fbea77fe640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.115:6791/0 conn(0x7fbe94007460 legacy=0x7fbe94009900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:32.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:32.332+0000 7fbea77fe640 10 monclient: _add_conns ranks=[8,1,2] 2024-01-24T23:23:32.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:32.332+0000 7fbea77fe640 10 monclient(hunting): picked mon.noname-e con 0x7fbe940100d0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:32.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:32.332+0000 7fbea77fe640 10 monclient(hunting): picked mon.noname-f con 0x7fbe940111c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:32.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:32.332+0000 7fbeb668f640 1 -- 172.21.15.115:0/4260601336 reap_dead start 2024-01-24T23:23:32.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:32.332+0000 7fbea77fe640 10 monclient(hunting): picked mon.noname-b con 0x7fbe94011d80 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:32.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:32.332+0000 7fbea77fe640 1 -- 172.21.15.115:0/4260601336 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe94012260 con 0x7fbe940111c0 2024-01-24T23:23:32.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:32.332+0000 7fbea77fe640 1 -- 172.21.15.115:0/4260601336 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb00a1070 con 0x7fbe94011d80 2024-01-24T23:23:32.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:32.332+0000 7fbea77fe640 1 -- 172.21.15.115:0/4260601336 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9400a110 con 0x7fbe940100d0 2024-01-24T23:23:32.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:32.332+0000 7fbea77fe640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:32.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:32.332+0000 7fbeb568d640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.132:6789/0 conn(0x7fbe940111c0 legacy=0x7fbe94011670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:32.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:32.536+0000 7fbeb568d640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.132:6789/0 conn(0x7fbe940111c0 legacy=0x7fbe94011670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:32.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:32.667+0000 7f7bb0ff9640 10 monclient: tick 2024-01-24T23:23:32.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:32.667+0000 7f7bb0ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:32.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:32.667+0000 7f7bb0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:32.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:32.940+0000 7fbeb568d640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.132:6789/0 conn(0x7fbe940111c0 legacy=0x7fbe94011670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:33.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:33.667+0000 7f7bb0ff9640 10 monclient: tick 2024-01-24T23:23:33.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:33.667+0000 7f7bb0ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:33.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:33.667+0000 7f7bb0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:33.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:33.744+0000 7fbeb568d640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.132:6789/0 conn(0x7fbe940111c0 legacy=0x7fbe94011670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:34.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:34.667+0000 7f7bb0ff9640 10 monclient: tick 2024-01-24T23:23:34.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:34.667+0000 7f7bb0ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:34.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:34.667+0000 7f7bb0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:35.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:35.336+0000 7fbea77fe640 10 monclient(hunting): tick 2024-01-24T23:23:35.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:35.336+0000 7fbea77fe640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:23:35.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:35.336+0000 7fbea77fe640 1 monclient(hunting): continuing hunt 2024-01-24T23:23:35.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:35.336+0000 7fbea77fe640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:23:35.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:35.336+0000 7fbea77fe640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.115:6793/0 conn(0x7fbe940100d0 legacy=0x7fbe940104b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:35.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:35.336+0000 7fbea77fe640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.115:6790/0 conn(0x7fbe94011d80 legacy=0x7fbe94019590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:35.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:35.336+0000 7fbea77fe640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.132:6789/0 conn(0x7fbe940111c0 legacy=0x7fbe94011670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:35.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:35.336+0000 7fbea77fe640 10 monclient: _add_conns ranks=[8,7,5,3,0,2,1,6,4] 2024-01-24T23:23:35.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:35.336+0000 7fbea77fe640 10 monclient(hunting): picked mon.noname-e con 0x7fbe9400a570 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:35.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:35.336+0000 7fbea77fe640 10 monclient(hunting): picked mon.noname-i con 0x7fbe9400a9f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:35.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:35.336+0000 7fbea77fe640 10 monclient(hunting): picked mon.noname-h con 0x7fbe9400ef00 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:35.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:35.336+0000 7fbea77fe640 1 -- 172.21.15.115:0/4260601336 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9400a2e0 con 0x7fbe9400ef00 2024-01-24T23:23:35.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:35.336+0000 7fbea77fe640 1 -- 172.21.15.115:0/4260601336 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe94012260 con 0x7fbe9400a9f0 2024-01-24T23:23:35.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:35.336+0000 7fbea77fe640 1 -- 172.21.15.115:0/4260601336 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9400d6a0 con 0x7fbe9400a570 2024-01-24T23:23:35.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:35.336+0000 7fbea77fe640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:35.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:35.667+0000 7f7bb0ff9640 10 monclient: tick 2024-01-24T23:23:35.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:35.667+0000 7f7bb0ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:35.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:35.667+0000 7f7bb0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:36.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:36.667+0000 7f7bb0ff9640 10 monclient: tick 2024-01-24T23:23:36.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:36.667+0000 7f7bb0ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:36.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:36.667+0000 7f7bb0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:37.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:37.671+0000 7f7bb0ff9640 10 monclient: tick 2024-01-24T23:23:37.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:37.671+0000 7f7bb0ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:37.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:37.671+0000 7f7bb0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:38.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.335+0000 7fbea77fe640 10 monclient(hunting): tick 2024-01-24T23:23:38.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.335+0000 7fbea77fe640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:23:38.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.335+0000 7fbea77fe640 1 monclient(hunting): continuing hunt 2024-01-24T23:23:38.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.335+0000 7fbea77fe640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:23:38.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.335+0000 7fbea77fe640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.115:6793/0 conn(0x7fbe9400a570 legacy=0x7fbe94009900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:38.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.335+0000 7fbea77fe640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.132:6792/0 conn(0x7fbe9400a9f0 legacy=0x7fbe94005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:38.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.335+0000 7fbea77fe640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.132:6791/0 conn(0x7fbe9400ef00 legacy=0x7fbe940104b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:38.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.335+0000 7fbeb668f640 1 -- 172.21.15.115:0/4260601336 reap_dead start 2024-01-24T23:23:38.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.335+0000 7fbea77fe640 10 monclient: _add_conns ranks=[1,0,6,2,3,4] 2024-01-24T23:23:38.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.335+0000 7fbea77fe640 10 monclient(hunting): picked mon.noname-f con 0x7fbe940111c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:38.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.335+0000 7fbea77fe640 10 monclient(hunting): picked mon.noname-a con 0x7fbe9400f2e0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:38.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.335+0000 7fbea77fe640 10 monclient(hunting): picked mon.noname-d con 0x7fbe9400ef00 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:38.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.335+0000 7fbea77fe640 1 -- 172.21.15.115:0/4260601336 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9400d830 con 0x7fbe9400f2e0 2024-01-24T23:23:38.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.335+0000 7fbea77fe640 1 -- 172.21.15.115:0/4260601336 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9400dbb0 con 0x7fbe940111c0 2024-01-24T23:23:38.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.335+0000 7fbea77fe640 1 -- 172.21.15.115:0/4260601336 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe940104b0 con 0x7fbe9400ef00 2024-01-24T23:23:38.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.335+0000 7fbea77fe640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:38.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.335+0000 7fbeb568d640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.115:6789/0 conn(0x7fbe9400f2e0 legacy=0x7fbe9400f6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:38.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.335+0000 7fbeb5e8e640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.132:6789/0 conn(0x7fbe940111c0 legacy=0x7fbe94011670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:38.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.491+0000 7fbeb4e8c640 1 -- 172.21.15.115:0/4260601336 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3496617932 0 0) 0x7fbe940104b0 con 0x7fbe9400ef00 2024-01-24T23:23:38.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.491+0000 7fbeb4e8c640 10 monclient(hunting): _negotiate 2024-01-24T23:23:38.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.491+0000 7fbeb4e8c640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:38.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.491+0000 7fbeb4e8c640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:38.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.491+0000 7fbeb4e8c640 10 monclient(hunting): my global_id is 5118 2024-01-24T23:23:38.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.491+0000 7fbeb4e8c640 1 -- 172.21.15.115:0/4260601336 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe9c003530 con 0x7fbe9400ef00 2024-01-24T23:23:38.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb4e8c640 1 -- 172.21.15.115:0/4260601336 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3414968600 0 0) 0x7fbe9c003530 con 0x7fbe9400ef00 2024-01-24T23:23:38.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb4e8c640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.132:6789/0 conn(0x7fbe940111c0 legacy=0x7fbe94011670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:38.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb4e8c640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.115:6789/0 conn(0x7fbe9400f2e0 legacy=0x7fbe9400f6c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:38.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb4e8c640 10 monclient: _finish_hunting 0 2024-01-24T23:23:38.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb4e8c640 1 monclient: found mon.noname-d 2024-01-24T23:23:38.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb4e8c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:38.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb4e8c640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:23:38.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb4e8c640 1 -- 172.21.15.115:0/4260601336 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe94010640 con 0x7fbe9400ef00 2024-01-24T23:23:38.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb4e8c640 10 monclient: _finish_auth 0 2024-01-24T23:23:38.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb4e8c640 10 monclient: _check_auth_tickets 2024-01-24T23:23:38.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb4e8c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:38.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb4e8c640 1 -- 172.21.15.115:0/4260601336 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbea8001a40 con 0x7fbe9400ef00 2024-01-24T23:23:38.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb4e8c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:38.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb4e8c640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:23:38.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb4e8c640 10 monclient: dump: 2024-01-24T23:23:38.505 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:38.505 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:38.505 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.505 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.505 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:38.506 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb80f2640 5 monclient: authenticate success, global_id 5118 2024-01-24T23:23:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb80f2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb4e8c640 1 -- 172.21.15.115:0/4260601336 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbea8001c30 con 0x7fbe9400ef00 2024-01-24T23:23:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb4e8c640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb4e8c640 1 -- 172.21.15.115:0/4260601336 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbea8004c30 con 0x7fbe9400ef00 2024-01-24T23:23:38.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb80f2640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:38.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb80f2640 4 set_mon_vals no callback set 2024-01-24T23:23:38.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb4e8c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:38.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb4e8c640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:38.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb4e8c640 10 monclient: dump: 2024-01-24T23:23:38.507 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:38.507 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:38.507 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.507 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.507 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:38.507 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:38.507 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:38.507 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:38.507 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:38.507 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:38.508 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:38.508 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:38.508 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:38.508 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:38.508 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:38.508 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:38.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb80f2640 10 monclient: shutdown 2024-01-24T23:23:38.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb80f2640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:38.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb80f2640 1 -- 172.21.15.115:0/4260601336 >> v1:172.21.15.115:6792/0 conn(0x7fbe9400ef00 legacy=0x7fbe94009900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:38.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb80f2640 1 -- 172.21.15.115:0/4260601336 shutdown_connections 2024-01-24T23:23:38.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb80f2640 1 -- 172.21.15.115:0/4260601336 >> 172.21.15.115:0/4260601336 conn(0x7fbeb00a5730 msgr2=0x7fbeb00a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:38.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb80f2640 1 -- 172.21.15.115:0/4260601336 shutdown_connections 2024-01-24T23:23:38.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.499+0000 7fbeb80f2640 1 -- 172.21.15.115:0/4260601336 wait complete. 2024-01-24T23:23:38.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb80f2640 10 monclient: build_initial_monmap 2024-01-24T23:23:38.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb80f2640 10 monclient: monmap: 2024-01-24T23:23:38.513 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:38.513 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:38.514 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:38.514 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:38.514 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:38.514 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:38.514 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:38.514 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:38.514 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:38.514 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:38.514 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:38.514 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:38.514 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:38.514 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:38.514 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:38.514 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb80f2640 1 Processor -- start 2024-01-24T23:23:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb80f2640 1 -- start start 2024-01-24T23:23:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb80f2640 10 monclient: init 2024-01-24T23:23:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb80f2640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb80f2640 10 monclient: _add_conns ranks=[1,2,0,7,4,6,8,3,5] 2024-01-24T23:23:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb80f2640 10 monclient(hunting): picked mon.noname-f con 0x7fbe9400ef00 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb80f2640 10 monclient(hunting): picked mon.noname-b con 0x7fbe9400f2e0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb80f2640 10 monclient(hunting): picked mon.noname-a con 0x7fbe940111c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb80f2640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb0124850 con 0x7fbe940111c0 2024-01-24T23:23:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb80f2640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb0125f40 con 0x7fbe9400ef00 2024-01-24T23:23:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb668f640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fbe940111c0 legacy=0x7fbeb01d41d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb80f2640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb0126220 con 0x7fbe9400f2e0 2024-01-24T23:23:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb80f2640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb80f2640 10 monclient(hunting): authenticate will time out at 1097.430685s 2024-01-24T23:23:38.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb5e8e640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fbe9400ef00 legacy=0x7fbeb01cb8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:38.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb568d640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fbe9400f2e0 0x7fbeb01d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:38556/0 (socket says 172.21.15.115:38556) 2024-01-24T23:23:38.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb568d640 1 -- 172.21.15.115:0/4222066789 learned_addr learned my addr 172.21.15.115:0/4222066789 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:38.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 1 -- 172.21.15.115:0/4222066789 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1436806569 0 0) 0x7fbeb0126220 con 0x7fbe9400f2e0 2024-01-24T23:23:38.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:38.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:38.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:38.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 10 monclient(hunting): my global_id is 84323 2024-01-24T23:23:38.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 1 -- 172.21.15.115:0/4222066789 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe940104b0 con 0x7fbe9400f2e0 2024-01-24T23:23:38.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 1 -- 172.21.15.115:0/4222066789 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4148431626 0 0) 0x7fbe940104b0 con 0x7fbe9400f2e0 2024-01-24T23:23:38.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 1 -- 172.21.15.115:0/4222066789 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbeb0126220 con 0x7fbe9400f2e0 2024-01-24T23:23:38.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 1 -- 172.21.15.115:0/4222066789 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbea0001a50 con 0x7fbe9400f2e0 2024-01-24T23:23:38.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:38.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:23:38.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 10 monclient(hunting): dump: 2024-01-24T23:23:38.516 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:38.517 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:38.517 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.517 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.517 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:38.517 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:38.517 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:38.517 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:38.517 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:38.517 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:38.517 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:38.517 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:38.517 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:38.517 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:38.517 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:38.517 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 1 -- 172.21.15.115:0/4222066789 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2445433121 0 0) 0x7fbeb0126220 con 0x7fbe9400f2e0 2024-01-24T23:23:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 1 -- 172.21.15.115:0/4222066789 >> v1:172.21.15.132:6789/0 conn(0x7fbe9400ef00 legacy=0x7fbeb01cb8e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 1 -- 172.21.15.115:0/4222066789 >> v1:172.21.15.115:6789/0 conn(0x7fbe940111c0 legacy=0x7fbeb01d41d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 10 monclient: _finish_hunting 0 2024-01-24T23:23:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 1 monclient: found mon.b 2024-01-24T23:23:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 1 -- 172.21.15.115:0/4222066789 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb0071c00 con 0x7fbe9400f2e0 2024-01-24T23:23:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 10 monclient: _finish_auth 0 2024-01-24T23:23:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 10 monclient: _check_auth_tickets 2024-01-24T23:23:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb80f2640 5 monclient: authenticate success, global_id 84323 2024-01-24T23:23:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb80f2640 10 monclient: _renew_subs 2024-01-24T23:23:38.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb80f2640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:38.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb80f2640 1 -- 172.21.15.115:0/4222066789 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb00716b0 con 0x7fbe9400f2e0 2024-01-24T23:23:38.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb80f2640 10 monclient: _renew_subs 2024-01-24T23:23:38.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb80f2640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:38.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbeb80f2640 1 -- 172.21.15.115:0/4222066789 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb0123ec0 con 0x7fbe9400f2e0 2024-01-24T23:23:38.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 1 -- 172.21.15.115:0/4222066789 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbea00038b0 con 0x7fbe9400f2e0 2024-01-24T23:23:38.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.503+0000 7fbea67fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:38.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.507+0000 7fbea7fff640 4 set_mon_vals no callback set 2024-01-24T23:23:38.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.507+0000 7fbea67fc640 1 -- 172.21.15.115:0/4222066789 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbea0003d60 con 0x7fbe9400f2e0 2024-01-24T23:23:38.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.507+0000 7fbea67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:38.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.507+0000 7fbea67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:38.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.507+0000 7fbea67fc640 10 monclient: dump: 2024-01-24T23:23:38.519 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:38.519 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:38.519 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.519 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:38.520 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.507+0000 7fbeb80f2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.507+0000 7fbeb80f2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:38.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.507+0000 7fbeb80f2640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:38.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.507+0000 7fbeb80f2640 1 -- 172.21.15.115:0/4222066789 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe98005090 con 0x7fbe9400f2e0 2024-01-24T23:23:38.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.507+0000 7fbea67fc640 1 -- 172.21.15.115:0/4222066789 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fbea0003f40 con 0x7fbe9400f2e0 2024-01-24T23:23:38.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.507+0000 7fbea67fc640 1 -- 172.21.15.115:0/4222066789 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(51..51 src has 1..51) v4 ==== 4660+0+0 (unknown 1469567968 0 0) 0x7fbea0043c40 con 0x7fbe9400f2e0 2024-01-24T23:23:38.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.507+0000 7fbeb5e8e640 10 monclient: get_auth_request con 0x7fbe9403aba0 auth_method 0 2024-01-24T23:23:38.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.511+0000 7fbea67fc640 1 -- 172.21.15.115:0/4222066789 <== mon.2 v1:172.21.15.115:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fbea00148a0 con 0x7fbe9400f2e0 2024-01-24T23:23:38.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.511+0000 7fbea67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:38.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.511+0000 7fbea67fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:38.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.555+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}]=0 pool 'ec-foo' removed v52) v1 ==== 154+0+0 (unknown 2532300114 0 0) 0x7f7bac031470 con 0x7f7ba003f7c0 2024-01-24T23:23:38.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.555+0000 7f7bb1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:23:38.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.555+0000 7f7bb1ffb640 10 monclient: _finish_command 2 = system:0 pool 'ec-foo' removed 2024-01-24T23:23:38.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.651+0000 7f7bc3316640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:23:38.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.651+0000 7f7bc3316640 10 monclient: _send_command 3 [{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:23:38.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.651+0000 7f7bc3316640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:38.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.651+0000 7f7bc3316640 1 -- 172.21.15.132:0/1344075490 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7b9c002220 con 0x7f7ba003f7c0 2024-01-24T23:23:38.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.655+0000 7f7bb1ffb640 1 -- 172.21.15.132:0/1344075490 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}]=0 pool 'ec-foo' does not exist v52) v1 ==== 161+0+0 (unknown 1039679987 0 0) 0x7f7bac012330 con 0x7f7ba003f7c0 2024-01-24T23:23:38.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.655+0000 7f7bb1ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-01-24T23:23:38.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.655+0000 7f7bb1ffb640 10 monclient: _finish_command 3 = system:0 pool 'ec-foo' does not exist 2024-01-24T23:23:38.656 INFO:tasks.workunit.client.0.smithi132.stderr:pool 'ec-foo' does not exist 2024-01-24T23:23:38.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.655+0000 7f7bc3316640 1 -- 172.21.15.132:0/1344075490 >> v1:172.21.15.132:6812/1835401922 conn(0x7f7ba0037b90 legacy=0x7f7ba003a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:38.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.655+0000 7f7bc3316640 10 monclient: shutdown 2024-01-24T23:23:38.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.655+0000 7f7bc3316640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:38.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.655+0000 7f7bc18b3640 1 -- 172.21.15.132:0/1344075490 reap_dead start 2024-01-24T23:23:38.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.655+0000 7f7bc3316640 1 -- 172.21.15.132:0/1344075490 >> v1:172.21.15.115:6790/0 conn(0x7f7ba003f7c0 legacy=0x7f7ba0041bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:38.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.655+0000 7f7bc3316640 1 -- 172.21.15.132:0/1344075490 shutdown_connections 2024-01-24T23:23:38.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.655+0000 7f7bc3316640 1 -- 172.21.15.132:0/1344075490 >> 172.21.15.132:0/1344075490 conn(0x7f7bbc0a9bf0 msgr2=0x7f7bbc1bebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:38.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.655+0000 7f7bc3316640 1 -- 172.21.15.132:0/1344075490 shutdown_connections 2024-01-24T23:23:38.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.655+0000 7f7bc3316640 1 -- 172.21.15.132:0/1344075490 wait complete. 2024-01-24T23:23:38.674 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule ls 2024-01-24T23:23:38.674 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep foo 2024-01-24T23:23:38.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.723+0000 7fbeb80f2640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-01-24T23:23:38.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.723+0000 7fbeb80f2640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-01-24T23:23:38.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.723+0000 7fbeb80f2640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:38.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.723+0000 7fbeb80f2640 1 -- 172.21.15.115:0/4222066789 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbe980058c0 con 0x7fbe9400f2e0 2024-01-24T23:23:38.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.723+0000 7fbea67fc640 1 -- 172.21.15.115:0/4222066789 <== mon.2 v1:172.21.15.115:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2726 (unknown 254746657 0 1420040931) 0x7fbea0012f40 con 0x7fbe9400f2e0 2024-01-24T23:23:38.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.723+0000 7fbea67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-01-24T23:23:38.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.723+0000 7fbea67fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:38.728 INFO:teuthology.orchestra.run.smithi115.stdout:{"election_epoch":20,"quorum":[2,3,5,6,7,8],"quorum_names":["b","g","h","d","i","e"],"quorum_leader_name":"g","quorum_age":0,"features":{"quorum_con":"4540701272860131327","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","modified":"2024-01-24T23:20:20.557246Z","created":"2024-01-24T23:20:20.557246Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6790","nonce":0}]},"addr":"172.21.15.115:6790/0","public_addr":"172.21.15.115:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6790","nonce":0}]},"addr":"172.21.15.132:6790/0","public_addr":"172.21.15.132:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6791","nonce":0}]},"addr":"172.21.15.115:6791/0","public_addr":"172.21.15.115:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6791","nonce":0}]},"addr":"172.21.15.132:6791/0","public_addr":"172.21.15.132:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6792","nonce":0}]},"addr":"172.21.15.115:6792/0","public_addr":"172.21.15.115:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6792","nonce":0}]},"addr":"172.21.15.132:6792/0","public_addr":"172.21.15.132:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6793","nonce":0}]},"addr":"172.21.15.115:6793/0","public_addr":"172.21.15.115:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-01-24T23:23:38.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.723+0000 7fbeb80f2640 1 -- 172.21.15.115:0/4222066789 >> v1:172.21.15.132:6812/1835401922 conn(0x7fbe9403aba0 legacy=0x7fbe9403d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:38.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.723+0000 7fbeb80f2640 10 monclient: shutdown 2024-01-24T23:23:38.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.723+0000 7fbeb80f2640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:38.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.723+0000 7fbeb80f2640 1 -- 172.21.15.115:0/4222066789 >> v1:172.21.15.115:6790/0 conn(0x7fbe9400f2e0 legacy=0x7fbeb01d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:38.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.723+0000 7fbeb80f2640 1 -- 172.21.15.115:0/4222066789 shutdown_connections 2024-01-24T23:23:38.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.723+0000 7fbeb80f2640 1 -- 172.21.15.115:0/4222066789 >> 172.21.15.115:0/4222066789 conn(0x7fbeb00a5730 msgr2=0x7fbeb006cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:38.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.723+0000 7fbeb80f2640 1 -- 172.21.15.115:0/4222066789 shutdown_connections 2024-01-24T23:23:38.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.723+0000 7fbeb80f2640 1 -- 172.21.15.115:0/4222066789 wait complete. 2024-01-24T23:23:38.745 INFO:tasks.mon_thrash.ceph_manager:quorum is size 6 2024-01-24T23:23:38.745 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon b 2024-01-24T23:23:38.746 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-01-24T23:23:38.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.787+0000 7ff1a0eb5640 10 monclient: get_monmap_and_config 2024-01-24T23:23:38.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.787+0000 7ff1a0eb5640 10 monclient: build_initial_monmap 2024-01-24T23:23:38.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.787+0000 7ff1a0eb5640 10 monclient: monmap: 2024-01-24T23:23:38.790 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:38.790 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:38.791 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:38.791623+0000 2024-01-24T23:23:38.791 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:38.791623+0000 2024-01-24T23:23:38.791 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:38.791 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:38.791 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:38.791 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:38.791 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:38.791 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:38.791 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:38.791 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:38.791 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:38.791 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:38.791 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:38.791 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:38.798 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.787+0000 7ff1a0eb5640 1 Processor -- start 2024-01-24T23:23:38.798 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.787+0000 7ff1a0eb5640 1 -- start start 2024-01-24T23:23:38.798 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.787+0000 7ff1a0eb5640 10 monclient: init 2024-01-24T23:23:38.798 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:38.798 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 10 monclient: _add_conns ranks=[2,7,0,1,5,6,3,4,8] 2024-01-24T23:23:38.798 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 10 monclient(hunting): picked mon.noname-b con 0x7ff19c0a2d50 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:38.798 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 10 monclient(hunting): picked mon.noname-i con 0x7ff19c136f70 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:38.798 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 10 monclient(hunting): picked mon.noname-a con 0x7ff19c13d510 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:38.798 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19c064700 con 0x7ff19c13d510 2024-01-24T23:23:38.798 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19c0a55f0 con 0x7ff19c0a2d50 2024-01-24T23:23:38.798 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19c0a10a0 con 0x7ff19c136f70 2024-01-24T23:23:38.798 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:38.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 10 monclient(hunting): authenticate will time out at 1106.116239s 2024-01-24T23:23:38.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19ad9d640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7ff19c13d510 legacy=0x7ff19c13f900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:38.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff199d9b640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7ff19c136f70 0x7ff19c13bdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:41586/0 (socket says 172.21.15.132:41586) 2024-01-24T23:23:38.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff199d9b640 1 -- 172.21.15.132:0/51004157 learned_addr learned my addr 172.21.15.132:0/51004157 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:38.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 1 -- 172.21.15.132:0/51004157 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1324061258 0 0) 0x7ff19c0a10a0 con 0x7ff19c136f70 2024-01-24T23:23:38.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 10 monclient(hunting): _negotiate 2024-01-24T23:23:38.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:38.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:38.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 10 monclient(hunting): my global_id is 5128 2024-01-24T23:23:38.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 1 -- 172.21.15.132:0/51004157 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff184003530 con 0x7ff19c136f70 2024-01-24T23:23:38.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 1 -- 172.21.15.132:0/51004157 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1729110962 0 0) 0x7ff19c0a55f0 con 0x7ff19c0a2d50 2024-01-24T23:23:38.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 10 monclient(hunting): _negotiate 2024-01-24T23:23:38.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:38.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:38.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 10 monclient(hunting): my global_id is 84332 2024-01-24T23:23:38.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 1 -- 172.21.15.132:0/51004157 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff19c0a10a0 con 0x7ff19c0a2d50 2024-01-24T23:23:38.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 1 -- 172.21.15.132:0/51004157 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 80118917 0 0) 0x7ff184003530 con 0x7ff19c136f70 2024-01-24T23:23:38.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 1 -- 172.21.15.132:0/51004157 >> v1:172.21.15.115:6790/0 conn(0x7ff19c0a2d50 legacy=0x7ff19c0a3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:38.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 1 -- 172.21.15.132:0/51004157 >> v1:172.21.15.115:6789/0 conn(0x7ff19c13d510 legacy=0x7ff19c13f900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:38.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 10 monclient: _finish_hunting 0 2024-01-24T23:23:38.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 1 monclient: found mon.noname-i 2024-01-24T23:23:38.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:38.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:23:38.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 1 -- 172.21.15.132:0/51004157 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c0a1380 con 0x7ff19c136f70 2024-01-24T23:23:38.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 10 monclient: _finish_auth 0 2024-01-24T23:23:38.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 10 monclient: _check_auth_tickets 2024-01-24T23:23:38.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:38.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 5 monclient: authenticate success, global_id 5128 2024-01-24T23:23:38.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:38.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 1 -- 172.21.15.132:0/51004157 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff1900017c0 con 0x7ff19c136f70 2024-01-24T23:23:38.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:38.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:23:38.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 10 monclient: dump: 2024-01-24T23:23:38.801 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:38.801 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:38.801 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.801 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.801 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:38.801 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:38.801 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:38.801 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:38.801 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:38.801 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:38.802 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:38.802 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:38.802 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:38.802 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:38.802 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:38.802 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:38.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:38.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 1 -- 172.21.15.132:0/51004157 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff190001aa0 con 0x7ff19c136f70 2024-01-24T23:23:38.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:38.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:38.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 1 -- 172.21.15.132:0/51004157 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff190004bf0 con 0x7ff19c136f70 2024-01-24T23:23:38.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 4 set_mon_vals no callback set 2024-01-24T23:23:38.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:38.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:38.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19959a640 10 monclient: dump: 2024-01-24T23:23:38.802 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:38.803 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:38.803 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.803 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.803 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:38.803 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:38.803 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:38.803 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:38.803 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:38.803 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:38.803 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:38.803 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:38.803 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:38.803 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:38.803 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:38.804 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:38.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 10 monclient: shutdown 2024-01-24T23:23:38.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:38.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 1 -- 172.21.15.132:0/51004157 >> v1:172.21.15.132:6792/0 conn(0x7ff19c136f70 legacy=0x7ff19c13bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:38.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 1 -- 172.21.15.132:0/51004157 shutdown_connections 2024-01-24T23:23:38.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 1 -- 172.21.15.132:0/51004157 >> 172.21.15.132:0/51004157 conn(0x7ff19c0a6740 msgr2=0x7ff19c071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:38.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 1 -- 172.21.15.132:0/51004157 shutdown_connections 2024-01-24T23:23:38.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 1 -- 172.21.15.132:0/51004157 wait complete. 2024-01-24T23:23:38.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 10 monclient: build_initial_monmap 2024-01-24T23:23:38.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 10 monclient: monmap: 2024-01-24T23:23:38.804 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:38.804 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:38.804 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:38.804 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:38.805 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:38.805 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:38.805 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:38.805 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:38.805 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:38.805 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:38.805 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:38.805 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:38.805 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:38.805 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:38.805 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:38.805 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:38.805 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 1 Processor -- start 2024-01-24T23:23:38.805 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 1 -- start start 2024-01-24T23:23:38.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 10 monclient: init 2024-01-24T23:23:38.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:38.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 10 monclient: _add_conns ranks=[3,7,4,6,2,1,8,0,5] 2024-01-24T23:23:38.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 10 monclient(hunting): picked mon.noname-g con 0x7ff19c0a2d50 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:38.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 10 monclient(hunting): picked mon.noname-i con 0x7ff19c136f70 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:38.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 10 monclient(hunting): picked mon.noname-c con 0x7ff19c13d510 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:38.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19c0a57b0 con 0x7ff19c0a2d50 2024-01-24T23:23:38.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19c09cb80 con 0x7ff19c13d510 2024-01-24T23:23:38.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19c09ee70 con 0x7ff19c136f70 2024-01-24T23:23:38.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:38.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff1a0eb5640 10 monclient(hunting): authenticate will time out at 1106.119217s 2024-01-24T23:23:38.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff19ad9d640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7ff19c13d510 legacy=0x7ff19c1d38f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:23:38.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff199d9b640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7ff19c136f70 0x7ff19c1cbda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:41600/0 (socket says 172.21.15.132:41600) 2024-01-24T23:23:38.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff199d9b640 1 -- 172.21.15.132:0/1514500930 learned_addr learned my addr 172.21.15.132:0/1514500930 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:38.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 1 -- 172.21.15.132:0/1514500930 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2836592898 0 0) 0x7ff19c0a57b0 con 0x7ff19c0a2d50 2024-01-24T23:23:38.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:38.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:38.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:38.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 10 monclient(hunting): my global_id is 5070 2024-01-24T23:23:38.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 1 -- 172.21.15.132:0/1514500930 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff17c003530 con 0x7ff19c0a2d50 2024-01-24T23:23:38.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 1 -- 172.21.15.132:0/1514500930 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724323661 0 0) 0x7ff19c09ee70 con 0x7ff19c136f70 2024-01-24T23:23:38.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:38.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:38.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:38.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 10 monclient(hunting): my global_id is 5137 2024-01-24T23:23:38.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 1 -- 172.21.15.132:0/1514500930 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff19c0a57b0 con 0x7ff19c136f70 2024-01-24T23:23:38.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 1 -- 172.21.15.132:0/1514500930 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3513030301 0 0) 0x7ff17c003530 con 0x7ff19c0a2d50 2024-01-24T23:23:38.808 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 1 -- 172.21.15.132:0/1514500930 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff19c09ee70 con 0x7ff19c0a2d50 2024-01-24T23:23:38.808 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 1 -- 172.21.15.132:0/1514500930 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff18c001a60 con 0x7ff19c0a2d50 2024-01-24T23:23:38.808 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:38.808 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:38.808 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 10 monclient(hunting): dump: 2024-01-24T23:23:38.808 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:38.808 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:38.808 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.808 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.808 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:38.808 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:38.808 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:38.808 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:38.808 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:38.809 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:38.809 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:38.809 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:38.809 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:38.809 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:38.809 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:38.809 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:38.809 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:38.809 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 1 -- 172.21.15.132:0/1514500930 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3825283138 0 0) 0x7ff19c0a57b0 con 0x7ff19c136f70 2024-01-24T23:23:38.809 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 1 -- 172.21.15.132:0/1514500930 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff17c003530 con 0x7ff19c136f70 2024-01-24T23:23:38.809 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 1 -- 172.21.15.132:0/1514500930 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff190004b30 con 0x7ff19c136f70 2024-01-24T23:23:38.809 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:38.809 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:38.809 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.791+0000 7ff18affd640 10 monclient(hunting): dump: 2024-01-24T23:23:38.810 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:38.810 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:38.810 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.810 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.810 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:38.810 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:38.810 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:38.810 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:38.810 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:38.810 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:38.810 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:38.810 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:38.810 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:38.810 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:38.810 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:38.811 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:38.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff18affd640 1 -- 172.21.15.132:0/1514500930 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3090467663 0 0) 0x7ff19c09ee70 con 0x7ff19c0a2d50 2024-01-24T23:23:38.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff18affd640 1 -- 172.21.15.132:0/1514500930 >> v1:172.21.15.132:6792/0 conn(0x7ff19c136f70 legacy=0x7ff19c1cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:38.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff18affd640 1 -- 172.21.15.132:0/1514500930 >> v1:172.21.15.115:6791/0 conn(0x7ff19c13d510 legacy=0x7ff19c1d38f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:38.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff18affd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:38.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff18affd640 1 monclient: found mon.g 2024-01-24T23:23:38.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff18affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:38.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff18affd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:38.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff18affd640 1 -- 172.21.15.132:0/1514500930 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c1cc4b0 con 0x7ff19c0a2d50 2024-01-24T23:23:38.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff18affd640 10 monclient: _finish_auth 0 2024-01-24T23:23:38.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff18affd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:38.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff18affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:38.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff1a0eb5640 5 monclient: authenticate success, global_id 5070 2024-01-24T23:23:38.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff1a0eb5640 10 monclient: _renew_subs 2024-01-24T23:23:38.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff1a0eb5640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:38.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff1a0eb5640 1 -- 172.21.15.132:0/1514500930 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19c0a2b20 con 0x7ff19c0a2d50 2024-01-24T23:23:38.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff1a0eb5640 10 monclient: _renew_subs 2024-01-24T23:23:38.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff1a0eb5640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:38.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff1a0eb5640 1 -- 172.21.15.132:0/1514500930 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19c09d430 con 0x7ff19c0a2d50 2024-01-24T23:23:38.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff18affd640 1 -- 172.21.15.132:0/1514500930 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff18c0038d0 con 0x7ff19c0a2d50 2024-01-24T23:23:38.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff18affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:38.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff18affd640 1 -- 172.21.15.132:0/1514500930 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff18c003d80 con 0x7ff19c0a2d50 2024-01-24T23:23:38.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff18affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:38.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff198d99640 4 set_mon_vals no callback set 2024-01-24T23:23:38.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff18affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:38.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff18affd640 10 monclient: dump: 2024-01-24T23:23:38.812 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:38.812 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:38.812 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.813 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.813 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:38.813 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:38.813 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:38.813 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:38.813 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:38.813 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:38.813 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:38.813 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:38.813 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:38.813 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:38.813 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:38.813 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:38.813 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff18affd640 1 -- 172.21.15.132:0/1514500930 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7ff18c00e800 con 0x7ff19c0a2d50 2024-01-24T23:23:38.813 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff1a0eb5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:38.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff1a0eb5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:38.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff1a0eb5640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:38.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff1a0eb5640 1 -- 172.21.15.132:0/1514500930 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff180005090 con 0x7ff19c0a2d50 2024-01-24T23:23:38.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff18affd640 1 -- 172.21.15.132:0/1514500930 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(52..52 src has 1..52) v4 ==== 4305+0+0 (unknown 1637139180 0 0) 0x7ff18c0427d0 con 0x7ff19c0a2d50 2024-01-24T23:23:38.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.795+0000 7ff199d9b640 10 monclient: get_auth_request con 0x7ff17c037820 auth_method 0 2024-01-24T23:23:38.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.799+0000 7ff18affd640 1 -- 172.21.15.132:0/1514500930 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7ff18c013610 con 0x7ff19c0a2d50 2024-01-24T23:23:38.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.799+0000 7ff18affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:38.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.799+0000 7ff18affd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:38.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.859+0000 7f56c41ec640 10 monclient: get_monmap_and_config 2024-01-24T23:23:38.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.859+0000 7f56c41ec640 10 monclient: build_initial_monmap 2024-01-24T23:23:38.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.859+0000 7f56c41ec640 10 monclient: monmap: 2024-01-24T23:23:38.864 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:38.864 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:38.864 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:38.864315+0000 2024-01-24T23:23:38.864 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:38.864315+0000 2024-01-24T23:23:38.864 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:38.864 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:38.864 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:38.864 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:38.864 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:38.864 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:38.864 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:38.864 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:38.864 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:38.865 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:38.865 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:38.865 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:38.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.859+0000 7f56c41ec640 1 Processor -- start 2024-01-24T23:23:38.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.859+0000 7f56c41ec640 1 -- start start 2024-01-24T23:23:38.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.859+0000 7f56c41ec640 10 monclient: init 2024-01-24T23:23:38.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.859+0000 7f56c41ec640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:38.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.859+0000 7f56c41ec640 10 monclient: _add_conns ranks=[7,6,8,5,2,3,1,0,4] 2024-01-24T23:23:38.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.859+0000 7f56c41ec640 10 monclient(hunting): picked mon.noname-i con 0x7f56bc133bf0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:38.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.859+0000 7f56c41ec640 10 monclient(hunting): picked mon.noname-d con 0x7f56bc133fd0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:38.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.859+0000 7f56c41ec640 10 monclient(hunting): picked mon.noname-e con 0x7f56bc134430 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:38.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.859+0000 7f56c41ec640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56bc04ed70 con 0x7f56bc133fd0 2024-01-24T23:23:38.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.859+0000 7f56c41ec640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56bc09af30 con 0x7f56bc133bf0 2024-01-24T23:23:38.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.859+0000 7f56c41ec640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56bc09b210 con 0x7f56bc134430 2024-01-24T23:23:38.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.859+0000 7f56c41ec640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:38.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.859+0000 7f56c41ec640 10 monclient(hunting): authenticate will time out at 1097.787692s 2024-01-24T23:23:38.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.859+0000 7f56c2789640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f56bc134430 0x7f56bc13f500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:53556/0 (socket says 172.21.15.115:53556) 2024-01-24T23:23:38.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.859+0000 7f56c1787640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f56bc133fd0 0x7f56bc13bdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:44330/0 (socket says 172.21.15.115:44330) 2024-01-24T23:23:38.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.859+0000 7f56c2789640 1 -- 172.21.15.115:0/2191076120 learned_addr learned my addr 172.21.15.115:0/2191076120 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:38.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 1 -- 172.21.15.115:0/2191076120 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 167727964 0 0) 0x7f56bc09b210 con 0x7f56bc134430 2024-01-24T23:23:38.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 10 monclient(hunting): _negotiate 2024-01-24T23:23:38.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:38.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:38.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 10 monclient(hunting): my global_id is 84311 2024-01-24T23:23:38.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 1 -- 172.21.15.115:0/2191076120 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56a8003530 con 0x7f56bc134430 2024-01-24T23:23:38.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 1 -- 172.21.15.115:0/2191076120 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 871167866 0 0) 0x7f56bc04ed70 con 0x7f56bc133fd0 2024-01-24T23:23:38.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 10 monclient(hunting): _negotiate 2024-01-24T23:23:38.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:38.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:38.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 10 monclient(hunting): my global_id is 5127 2024-01-24T23:23:38.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 1 -- 172.21.15.115:0/2191076120 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56bc09b210 con 0x7f56bc133fd0 2024-01-24T23:23:38.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 1 -- 172.21.15.115:0/2191076120 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2781499302 0 0) 0x7f56a8003530 con 0x7f56bc134430 2024-01-24T23:23:38.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 1 -- 172.21.15.115:0/2191076120 >> v1:172.21.15.132:6792/0 conn(0x7f56bc133bf0 legacy=0x7f56bc09bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:38.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 1 -- 172.21.15.115:0/2191076120 >> v1:172.21.15.115:6792/0 conn(0x7f56bc133fd0 legacy=0x7f56bc13bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:38.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 10 monclient: _finish_hunting 0 2024-01-24T23:23:38.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 1 monclient: found mon.noname-e 2024-01-24T23:23:38.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:38.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:23:38.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 1 -- 172.21.15.115:0/2191076120 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc09b4f0 con 0x7f56bc134430 2024-01-24T23:23:38.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 10 monclient: _finish_auth 0 2024-01-24T23:23:38.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 10 monclient: _check_auth_tickets 2024-01-24T23:23:38.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:38.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 1 -- 172.21.15.115:0/2191076120 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f56b4001b40 con 0x7f56bc134430 2024-01-24T23:23:38.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 5 monclient: authenticate success, global_id 84311 2024-01-24T23:23:38.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:38.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:38.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:23:38.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 10 monclient: dump: 2024-01-24T23:23:38.876 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:38.876 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:38.876 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.876 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.876 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:38.876 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:38.876 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:38.876 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:38.876 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:38.876 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:38.876 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:38.876 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:38.876 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:38.877 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:38.877 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:38.877 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:38.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:38.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 1 -- 172.21.15.115:0/2191076120 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f56b4001e20 con 0x7f56bc134430 2024-01-24T23:23:38.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:38.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 1 -- 172.21.15.115:0/2191076120 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f56b4004f70 con 0x7f56bc134430 2024-01-24T23:23:38.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:38.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 4 set_mon_vals no callback set 2024-01-24T23:23:38.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:38.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:38.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c0f86640 10 monclient: dump: 2024-01-24T23:23:38.877 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:38.877 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:38.878 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.878 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.878 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:38.878 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:38.878 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:38.878 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:38.878 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:38.878 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:38.878 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:38.878 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:38.878 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:38.878 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:38.878 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:38.878 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:38.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 10 monclient: shutdown 2024-01-24T23:23:38.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:38.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 1 -- 172.21.15.115:0/2191076120 >> v1:172.21.15.115:6793/0 conn(0x7f56bc134430 legacy=0x7f56bc13f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:38.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 1 -- 172.21.15.115:0/2191076120 shutdown_connections 2024-01-24T23:23:38.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 1 -- 172.21.15.115:0/2191076120 >> 172.21.15.115:0/2191076120 conn(0x7f56bc0a9910 msgr2=0x7f56bc0a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:38.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 1 -- 172.21.15.115:0/2191076120 shutdown_connections 2024-01-24T23:23:38.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 1 -- 172.21.15.115:0/2191076120 wait complete. 2024-01-24T23:23:38.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 10 monclient: build_initial_monmap 2024-01-24T23:23:38.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 10 monclient: monmap: 2024-01-24T23:23:38.879 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:38.879 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:38.879 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:38.879 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:38.879 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:38.879 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:38.880 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:38.880 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:38.880 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:38.880 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:38.880 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:38.880 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:38.880 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:38.880 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:38.880 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:38.880 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:38.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 1 Processor -- start 2024-01-24T23:23:38.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 1 -- start start 2024-01-24T23:23:38.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 10 monclient: init 2024-01-24T23:23:38.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:38.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 10 monclient: _add_conns ranks=[3,2,1,4,5,6,7,8,0] 2024-01-24T23:23:38.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 10 monclient(hunting): picked mon.noname-g con 0x7f56bc133bf0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:38.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 10 monclient(hunting): picked mon.noname-b con 0x7f56bc133fd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:38.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 10 monclient(hunting): picked mon.noname-f con 0x7f56bc134430 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:38.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56bc139d90 con 0x7f56bc134430 2024-01-24T23:23:38.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56bc1389c0 con 0x7f56bc133fd0 2024-01-24T23:23:38.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56bc09c8f0 con 0x7f56bc133bf0 2024-01-24T23:23:38.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:38.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 10 monclient(hunting): authenticate will time out at 1097.790596s 2024-01-24T23:23:38.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c2789640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f56bc134430 legacy=0x7f56bc1d3d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:38.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c1787640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f56bc133fd0 0x7f56bc1382b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:38568/0 (socket says 172.21.15.115:38568) 2024-01-24T23:23:38.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c1787640 1 -- 172.21.15.115:0/108211854 learned_addr learned my addr 172.21.15.115:0/108211854 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:38.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960236553 0 0) 0x7f56bc1389c0 con 0x7f56bc133fd0 2024-01-24T23:23:38.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:38.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:38.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:38.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 10 monclient(hunting): my global_id is 84341 2024-01-24T23:23:38.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56a0003530 con 0x7f56bc133fd0 2024-01-24T23:23:38.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2979871675 0 0) 0x7f56a0003530 con 0x7f56bc133fd0 2024-01-24T23:23:38.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f56bc1389c0 con 0x7f56bc133fd0 2024-01-24T23:23:38.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f56ac001520 con 0x7f56bc133fd0 2024-01-24T23:23:38.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:38.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:23:38.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 10 monclient(hunting): dump: 2024-01-24T23:23:38.882 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:38.882 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:38.882 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.882 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.882 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:38.882 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:38.882 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:38.883 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:38.883 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:38.883 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:38.883 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:38.883 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:38.883 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:38.883 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:38.883 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:38.883 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:38.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:38.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2317261942 0 0) 0x7f56bc09c8f0 con 0x7f56bc133bf0 2024-01-24T23:23:38.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:38.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:38.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:38.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 10 monclient(hunting): my global_id is 5079 2024-01-24T23:23:38.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56a0003530 con 0x7f56bc133bf0 2024-01-24T23:23:38.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3559517360 0 0) 0x7f56bc1389c0 con 0x7f56bc133fd0 2024-01-24T23:23:38.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 >> v1:172.21.15.132:6790/0 conn(0x7f56bc133bf0 legacy=0x7f56bc137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:38.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 >> v1:172.21.15.132:6789/0 conn(0x7f56bc134430 legacy=0x7f56bc1d3d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:38.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 10 monclient: _finish_hunting 0 2024-01-24T23:23:38.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 1 monclient: found mon.b 2024-01-24T23:23:38.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:38.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:38.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc0a5960 con 0x7f56bc133fd0 2024-01-24T23:23:38.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 10 monclient: _finish_auth 0 2024-01-24T23:23:38.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 10 monclient: _check_auth_tickets 2024-01-24T23:23:38.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:38.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 5 monclient: authenticate success, global_id 84341 2024-01-24T23:23:38.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 10 monclient: _renew_subs 2024-01-24T23:23:38.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:38.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 1 -- 172.21.15.115:0/108211854 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56bc138ca0 con 0x7f56bc133fd0 2024-01-24T23:23:38.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 10 monclient: _renew_subs 2024-01-24T23:23:38.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:38.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56c41ec640 1 -- 172.21.15.115:0/108211854 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f56bc13a6f0 con 0x7f56bc133fd0 2024-01-24T23:23:38.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f56ac004aa0 con 0x7f56bc133fd0 2024-01-24T23:23:38.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:38.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f56ac004f50 con 0x7f56bc133fd0 2024-01-24T23:23:38.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:38.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b3fff640 4 set_mon_vals no callback set 2024-01-24T23:23:38.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:38.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.863+0000 7f56b27fc640 10 monclient: dump: 2024-01-24T23:23:38.885 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:38.885 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:38.885 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.886 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.886 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:38.886 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:38.886 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:38.886 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:38.886 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:38.886 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:38.886 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:38.886 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:38.886 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:38.886 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:38.886 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:38.886 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:38.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.867+0000 7f56c41ec640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:38.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.867+0000 7f56c41ec640 1 -- 172.21.15.115:0/108211854 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f56a40065d0 con 0x7f56a4002870 2024-01-24T23:23:38.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.867+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f56ac00fa50 con 0x7f56bc133fd0 2024-01-24T23:23:38.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.867+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(52..52 src has 1..52) v4 ==== 4305+0+0 (unknown 1637139180 0 0) 0x7f56ac043bd0 con 0x7f56bc133fd0 2024-01-24T23:23:38.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.867+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3792909744 0 0) 0x7f56a40065d0 con 0x7f56a4002870 2024-01-24T23:23:38.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.867+0000 7f56b27fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:38.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.867+0000 7f56b27fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:38.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.867+0000 7f56b27fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:38.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.867+0000 7f56b27fc640 10 monclient(hunting): my global_id is 84350 2024-01-24T23:23:38.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.867+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56bc1389c0 con 0x7f56a4002870 2024-01-24T23:23:38.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.867+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2934489808 0 0) 0x7f56bc1389c0 con 0x7f56a4002870 2024-01-24T23:23:38.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.867+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f56a40065d0 con 0x7f56a4002870 2024-01-24T23:23:38.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.867+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f56b8001870 con 0x7f56a4002870 2024-01-24T23:23:38.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.867+0000 7f56c2789640 10 monclient: get_auth_request con 0x7f56a0037070 auth_method 0 2024-01-24T23:23:38.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.867+0000 7f56b27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:38.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.867+0000 7f56b27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:38.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.867+0000 7f56b27fc640 10 monclient: dump: 2024-01-24T23:23:38.887 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:38.887 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:38.887 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.888 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.888 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:38.888 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:38.888 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:38.888 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:38.888 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:38.888 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:38.888 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:38.888 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:38.888 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:38.888 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:38.888 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:38.888 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:38.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.867+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 741182696 0 0) 0x7f56a40065d0 con 0x7f56a4002870 2024-01-24T23:23:38.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.867+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 --> v1:172.21.15.115:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56a400a120 con 0x7f56a4002870 2024-01-24T23:23:38.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.867+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 <== mon.2 v1:172.21.15.115:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f56b8003600 con 0x7f56a4002870 2024-01-24T23:23:38.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.867+0000 7f56b27fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:38.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.867+0000 7f56b27fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:38.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.867+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 >> v1:172.21.15.115:6790/0 conn(0x7f56a4002870 legacy=0x7f56a4004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:38.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.867+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 >> v1:172.21.15.115:6790/0 conn(0x7f56a4002870 legacy=0x7f56a4004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:38.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.871+0000 7f56c41ec640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:23:38.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.871+0000 7f56c41ec640 1 -- 172.21.15.115:0/108211854 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f56a4009df0 con 0x7f56a40090b0 2024-01-24T23:23:38.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.871+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4157803145 0 0) 0x7f56a4009df0 con 0x7f56a40090b0 2024-01-24T23:23:38.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.871+0000 7f56b27fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:38.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.871+0000 7f56b27fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:38.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.871+0000 7f56b27fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:38.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.871+0000 7f56b27fc640 10 monclient(hunting): my global_id is 84359 2024-01-24T23:23:38.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.871+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56a40065d0 con 0x7f56a40090b0 2024-01-24T23:23:38.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.871+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 605855412 0 0) 0x7f56a40065d0 con 0x7f56a40090b0 2024-01-24T23:23:38.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.871+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f56a4009df0 con 0x7f56a40090b0 2024-01-24T23:23:38.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.871+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f56b8003ca0 con 0x7f56a40090b0 2024-01-24T23:23:38.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.871+0000 7f56b27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:38.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.871+0000 7f56b27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:38.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.871+0000 7f56b27fc640 10 monclient: dump: 2024-01-24T23:23:38.891 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:38.891 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:38.892 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.892 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:38.892 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:38.892 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:38.892 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:38.892 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:38.892 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:38.892 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:38.892 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:38.892 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:38.892 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:38.892 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:38.892 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:38.892 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:38.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.875+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 998844486 0 0) 0x7f56a4009df0 con 0x7f56a40090b0 2024-01-24T23:23:38.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.875+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 --> v1:172.21.15.115:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f56a400c310 con 0x7f56a40090b0 2024-01-24T23:23:38.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.875+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 <== mon.2 v1:172.21.15.115:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7680 (unknown 0 0 3469874320) 0x7f56b8004200 con 0x7f56a40090b0 2024-01-24T23:23:38.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.875+0000 7f56b27fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:23:38.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.875+0000 7f56b27fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:38.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.875+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 >> v1:172.21.15.115:6790/0 conn(0x7f56a40090b0 legacy=0x7f56a4009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:38.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.875+0000 7f56b27fc640 1 -- 172.21.15.115:0/108211854 >> v1:172.21.15.115:6790/0 conn(0x7f56a40090b0 legacy=0x7f56a4009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:38.893 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:23:38.893 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:38.893 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:38.893 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:23:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 20, 2024-01-24T23:23:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:23:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:23:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:23:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:23:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:23:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:23:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:23:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:23:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:23:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:23:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:23:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:23:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:23:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:23:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:23:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:23:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:23:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:23:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:23:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:23:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:23:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:23:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:23:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:23:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:23:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:23:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:23:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:23:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:23:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:23:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:23:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:23:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:23:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:23:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:23:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:23:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:23:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:23:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:23:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:23:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:23:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:23:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:23:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:23:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:23:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:23:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:23:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:23:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:23:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:23:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:23:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:23:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:23:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:23:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:23:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:23:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:23:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:23:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:23:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:23:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:23:38.912 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:23:38.914 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.875+0000 7f56c41ec640 1 -- 172.21.15.115:0/108211854 >> v1:172.21.15.132:6812/1835401922 conn(0x7f56a0037070 legacy=0x7f56a0039530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:38.914 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.875+0000 7f56c41ec640 10 monclient: shutdown 2024-01-24T23:23:38.914 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.875+0000 7f56c41ec640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:38.914 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.875+0000 7f56c41ec640 1 -- 172.21.15.115:0/108211854 >> v1:172.21.15.115:6790/0 conn(0x7f56bc133fd0 legacy=0x7f56bc1382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:38.914 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.875+0000 7f56c2789640 1 -- 172.21.15.115:0/108211854 reap_dead start 2024-01-24T23:23:38.914 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.875+0000 7f56c2789640 1 -- 172.21.15.115:0/108211854 reap_dead start 2024-01-24T23:23:38.914 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.875+0000 7f56c41ec640 1 -- 172.21.15.115:0/108211854 shutdown_connections 2024-01-24T23:23:38.914 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.875+0000 7f56c41ec640 1 -- 172.21.15.115:0/108211854 >> 172.21.15.115:0/108211854 conn(0x7f56bc0a9910 msgr2=0x7f56bc1400c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:38.914 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.875+0000 7f56c41ec640 1 -- 172.21.15.115:0/108211854 shutdown_connections 2024-01-24T23:23:38.914 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:38.875+0000 7f56c41ec640 1 -- 172.21.15.115:0/108211854 wait complete. 2024-01-24T23:23:38.915 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-01-24T23:23:38.915 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon d 2024-01-24T23:23:38.915 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-01-24T23:23:38.960 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.959+0000 7ff1a0eb5640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:38.960 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.959+0000 7ff1a0eb5640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:38.960 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.959+0000 7ff1a0eb5640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:38.960 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.959+0000 7ff1a0eb5640 1 -- 172.21.15.132:0/1514500930 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rule ls"} v 0) v1 -- 0x7ff1800058c0 con 0x7ff19c0a2d50 2024-01-24T23:23:38.960 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.959+0000 7ff18affd640 1 -- 172.21.15.132:0/1514500930 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v52) v1 ==== 65+0+60 (unknown 4284643578 0 1596542489) 0x7ff18c01bd40 con 0x7ff19c0a2d50 2024-01-24T23:23:38.960 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.959+0000 7ff18affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:38.960 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:38.959+0000 7ff18affd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:39.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.023+0000 7f12f84f6640 10 monclient: get_monmap_and_config 2024-01-24T23:23:39.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.023+0000 7f12f84f6640 10 monclient: build_initial_monmap 2024-01-24T23:23:39.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.023+0000 7f12f84f6640 10 monclient: monmap: 2024-01-24T23:23:39.029 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:39.029 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.029 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:39.028972+0000 2024-01-24T23:23:39.029 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:39.028972+0000 2024-01-24T23:23:39.029 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:39.029 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.029 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:39.029 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:39.030 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:39.030 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:39.030 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:39.030 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:39.030 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:39.030 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:39.030 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:39.030 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.023+0000 7f12f84f6640 1 Processor -- start 2024-01-24T23:23:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.023+0000 7f12f84f6640 1 -- start start 2024-01-24T23:23:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.023+0000 7f12f84f6640 10 monclient: init 2024-01-24T23:23:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.023+0000 7f12f84f6640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 10 monclient: _add_conns ranks=[2,6,7,4,3,1,5,0,8] 2024-01-24T23:23:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 10 monclient(hunting): picked mon.noname-b con 0x7f12f0133bf0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 10 monclient(hunting): picked mon.noname-d con 0x7f12f0133fd0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 10 monclient(hunting): picked mon.noname-i con 0x7f12f01343b0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12f004ed70 con 0x7f12f0133bf0 2024-01-24T23:23:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12f00a4b60 con 0x7f12f0133fd0 2024-01-24T23:23:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12f009ae50 con 0x7f12f01343b0 2024-01-24T23:23:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 10 monclient(hunting): authenticate will time out at 1097.952114s 2024-01-24T23:23:39.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f6292640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f12f0133bf0 0x7f12f009bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:38582/0 (socket says 172.21.15.115:38582) 2024-01-24T23:23:39.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f6292640 1 -- 172.21.15.115:0/2999997691 learned_addr learned my addr 172.21.15.115:0/2999997691 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:39.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5a91640 1 --1- 172.21.15.115:0/2999997691 >> v1:172.21.15.115:6792/0 conn(0x7f12f0133fd0 0x7f12f013bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:44338/0 (socket says 172.21.15.115:44338) 2024-01-24T23:23:39.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 1 -- 172.21.15.115:0/2999997691 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 848481168 0 0) 0x7f12f004ed70 con 0x7f12f0133bf0 2024-01-24T23:23:39.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 10 monclient(hunting): my global_id is 84368 2024-01-24T23:23:39.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 1 -- 172.21.15.115:0/2999997691 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12dc003530 con 0x7f12f0133bf0 2024-01-24T23:23:39.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 1 -- 172.21.15.115:0/2999997691 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1297961579 0 0) 0x7f12f00a4b60 con 0x7f12f0133fd0 2024-01-24T23:23:39.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 10 monclient(hunting): my global_id is 5136 2024-01-24T23:23:39.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 1 -- 172.21.15.115:0/2999997691 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12f004ed70 con 0x7f12f0133fd0 2024-01-24T23:23:39.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 1 -- 172.21.15.115:0/2999997691 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1164013311 0 0) 0x7f12dc003530 con 0x7f12f0133bf0 2024-01-24T23:23:39.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 1 -- 172.21.15.115:0/2999997691 >> v1:172.21.15.132:6792/0 conn(0x7f12f01343b0 legacy=0x7f12f013f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 1 -- 172.21.15.115:0/2999997691 >> v1:172.21.15.115:6792/0 conn(0x7f12f0133fd0 legacy=0x7f12f013bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 10 monclient: _finish_hunting 0 2024-01-24T23:23:39.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 1 monclient: found mon.noname-b 2024-01-24T23:23:39.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:39.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:23:39.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 1 -- 172.21.15.115:0/2999997691 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f009b130 con 0x7f12f0133bf0 2024-01-24T23:23:39.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 10 monclient: _finish_auth 0 2024-01-24T23:23:39.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 10 monclient: _check_auth_tickets 2024-01-24T23:23:39.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:39.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 5 monclient: authenticate success, global_id 84368 2024-01-24T23:23:39.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 1 -- 172.21.15.115:0/2999997691 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f12ec0017c0 con 0x7f12f0133bf0 2024-01-24T23:23:39.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:39.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:23:39.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 10 monclient: dump: 2024-01-24T23:23:39.042 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.042 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.042 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.042 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.042 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.042 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.042 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.042 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.043 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.043 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.043 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.043 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.043 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.043 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.043 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.043 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:39.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 1 -- 172.21.15.115:0/2999997691 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f12ec0019e0 con 0x7f12f0133bf0 2024-01-24T23:23:39.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:39.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:39.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 4 set_mon_vals no callback set 2024-01-24T23:23:39.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 1 -- 172.21.15.115:0/2999997691 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f12ec004bf0 con 0x7f12f0133bf0 2024-01-24T23:23:39.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:39.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5290640 10 monclient: dump: 2024-01-24T23:23:39.044 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.044 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.044 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.044 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.044 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.044 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.044 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.044 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.044 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.044 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.044 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.045 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.045 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.045 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.045 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.045 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 10 monclient: shutdown 2024-01-24T23:23:39.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:39.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 1 -- 172.21.15.115:0/2999997691 >> v1:172.21.15.115:6790/0 conn(0x7f12f0133bf0 legacy=0x7f12f009bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 1 -- 172.21.15.115:0/2999997691 shutdown_connections 2024-01-24T23:23:39.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 1 -- 172.21.15.115:0/2999997691 >> 172.21.15.115:0/2999997691 conn(0x7f12f00662f0 msgr2=0x7f12f0066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:39.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 1 -- 172.21.15.115:0/2999997691 shutdown_connections 2024-01-24T23:23:39.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 1 -- 172.21.15.115:0/2999997691 wait complete. 2024-01-24T23:23:39.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 10 monclient: build_initial_monmap 2024-01-24T23:23:39.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 10 monclient: monmap: 2024-01-24T23:23:39.045 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:39.046 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:39.046 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:39.046 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:39.046 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:39.046 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:39.046 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:39.046 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:39.046 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:39.046 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:39.046 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:39.047 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:39.047 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:39.047 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:39.047 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:39.047 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 1 Processor -- start 2024-01-24T23:23:39.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 1 -- start start 2024-01-24T23:23:39.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 10 monclient: init 2024-01-24T23:23:39.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:39.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 10 monclient: _add_conns ranks=[6,2,7,5,0,4,1,8,3] 2024-01-24T23:23:39.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 10 monclient(hunting): picked mon.noname-d con 0x7f12f0133bf0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:39.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 10 monclient(hunting): picked mon.noname-b con 0x7f12f0133fd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:39.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 10 monclient(hunting): picked mon.noname-i con 0x7f12f01343b0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:39.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12f009c810 con 0x7f12f0133fd0 2024-01-24T23:23:39.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12f01bcee0 con 0x7f12f0133bf0 2024-01-24T23:23:39.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12f01209b0 con 0x7f12f01343b0 2024-01-24T23:23:39.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:39.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 10 monclient(hunting): authenticate will time out at 1097.954940s 2024-01-24T23:23:39.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f6292640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f12f0133bf0 0x7f12f0132fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:44340/0 (socket says 172.21.15.115:44340) 2024-01-24T23:23:39.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f6292640 1 -- 172.21.15.115:0/1868288115 learned_addr learned my addr 172.21.15.115:0/1868288115 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:39.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f5a91640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f12f0133fd0 0x7f12f01d05d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:38598/0 (socket says 172.21.15.115:38598) 2024-01-24T23:23:39.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 138854802 0 0) 0x7f12f009c810 con 0x7f12f0133fd0 2024-01-24T23:23:39.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient(hunting): my global_id is 84377 2024-01-24T23:23:39.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12d4003530 con 0x7f12f0133fd0 2024-01-24T23:23:39.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1759562923 0 0) 0x7f12f01bcee0 con 0x7f12f0133bf0 2024-01-24T23:23:39.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient(hunting): my global_id is 5145 2024-01-24T23:23:39.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12f009c810 con 0x7f12f0133bf0 2024-01-24T23:23:39.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1165418352 0 0) 0x7f12f009c810 con 0x7f12f0133bf0 2024-01-24T23:23:39.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f12f01bcee0 con 0x7f12f0133bf0 2024-01-24T23:23:39.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f12ec0016c0 con 0x7f12f0133bf0 2024-01-24T23:23:39.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:23:39.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient(hunting): dump: 2024-01-24T23:23:39.050 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.050 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.050 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.050 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.050 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.050 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.050 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.050 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.050 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.050 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.050 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.051 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.051 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.051 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.051 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.051 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:39.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1164364505 0 0) 0x7f12d4003530 con 0x7f12f0133fd0 2024-01-24T23:23:39.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f12f009c810 con 0x7f12f0133fd0 2024-01-24T23:23:39.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f12e0001560 con 0x7f12f0133fd0 2024-01-24T23:23:39.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:39.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient(hunting): dump: 2024-01-24T23:23:39.051 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.052 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.052 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.052 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.052 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.052 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.052 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.052 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.052 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.052 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.052 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.052 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.052 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.052 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.052 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.053 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2322208068 0 0) 0x7f12f01209b0 con 0x7f12f01343b0 2024-01-24T23:23:39.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient(hunting): my global_id is 5164 2024-01-24T23:23:39.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12d4003530 con 0x7f12f01343b0 2024-01-24T23:23:39.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4061590147 0 0) 0x7f12f01bcee0 con 0x7f12f0133bf0 2024-01-24T23:23:39.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 >> v1:172.21.15.132:6792/0 conn(0x7f12f01343b0 legacy=0x7f12f01d3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 >> v1:172.21.15.115:6790/0 conn(0x7f12f0133fd0 legacy=0x7f12f01d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:39.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 1 monclient: found mon.d 2024-01-24T23:23:39.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:39.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:23:39.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f01336e0 con 0x7f12f0133bf0 2024-01-24T23:23:39.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient: _finish_auth 0 2024-01-24T23:23:39.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:39.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:39.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 5 monclient: authenticate success, global_id 5145 2024-01-24T23:23:39.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 10 monclient: _renew_subs 2024-01-24T23:23:39.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:23:39.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 1 -- 172.21.15.115:0/1868288115 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12f00a5990 con 0x7f12f0133bf0 2024-01-24T23:23:39.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 10 monclient: _renew_subs 2024-01-24T23:23:39.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:23:39.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f84f6640 1 -- 172.21.15.115:0/1868288115 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f01206d0 con 0x7f12f0133bf0 2024-01-24T23:23:39.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f12ec004b90 con 0x7f12f0133bf0 2024-01-24T23:23:39.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:39.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f12ec005040 con 0x7f12f0133bf0 2024-01-24T23:23:39.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12e6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.027+0000 7f12f4a8f640 4 set_mon_vals no callback set 2024-01-24T23:23:39.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12e6ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:39.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12e6ffd640 10 monclient: dump: 2024-01-24T23:23:39.055 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.055 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.055 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.055 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.055 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.055 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.055 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.055 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.055 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.055 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.055 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.055 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.056 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.056 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.056 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.056 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12f84f6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:39.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12f84f6640 1 -- 172.21.15.115:0/1868288115 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f12d80065d0 con 0x7f12d8002870 2024-01-24T23:23:39.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1489553019 0 0) 0x7f12d80065d0 con 0x7f12d8002870 2024-01-24T23:23:39.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12e6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12e6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12e6ffd640 10 monclient(hunting): my global_id is 5154 2024-01-24T23:23:39.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12f01bcee0 con 0x7f12d8002870 2024-01-24T23:23:39.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f12ec005220 con 0x7f12f0133bf0 2024-01-24T23:23:39.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(52..52 src has 1..52) v4 ==== 4305+0+0 (unknown 1637139180 0 0) 0x7f12ec0439b0 con 0x7f12f0133bf0 2024-01-24T23:23:39.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2901595319 0 0) 0x7f12f01bcee0 con 0x7f12d8002870 2024-01-24T23:23:39.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f12d80065d0 con 0x7f12d8002870 2024-01-24T23:23:39.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f12e0001780 con 0x7f12d8002870 2024-01-24T23:23:39.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12e6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12e6ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:39.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12e6ffd640 10 monclient: dump: 2024-01-24T23:23:39.057 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.057 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.057 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.057 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.057 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.057 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.057 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.057 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.057 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.058 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.058 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.058 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.058 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.058 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.058 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.058 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12f6a93640 10 monclient: get_auth_request con 0x7f12d4037850 auth_method 0 2024-01-24T23:23:39.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 320756069 0 0) 0x7f12d80065d0 con 0x7f12d8002870 2024-01-24T23:23:39.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 --> v1:172.21.15.115:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f12d800a120 con 0x7f12d8002870 2024-01-24T23:23:39.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f12e0007480 con 0x7f12d8002870 2024-01-24T23:23:39.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12e6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:39.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12e6ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:39.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 >> v1:172.21.15.115:6792/0 conn(0x7f12d8002870 legacy=0x7f12d8004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.031+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 >> v1:172.21.15.115:6792/0 conn(0x7f12d8002870 legacy=0x7f12d8004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:39.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.051+0000 7ff1a0eb5640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:39.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.051+0000 7ff1a0eb5640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:39.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.051+0000 7ff1a0eb5640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:39.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.051+0000 7ff1a0eb5640 1 -- 172.21.15.132:0/1514500930 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rule ls"} v 0) v1 -- 0x7ff180000bd0 con 0x7ff19c0a2d50 2024-01-24T23:23:39.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.051+0000 7ff18affd640 1 -- 172.21.15.132:0/1514500930 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v52) v1 ==== 65+0+60 (unknown 4284643578 0 1596542489) 0x7ff18c011cb0 con 0x7ff19c0a2d50 2024-01-24T23:23:39.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.055+0000 7ff18affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:39.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.055+0000 7ff18affd640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:23:39.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.055+0000 7ff1a0eb5640 1 -- 172.21.15.132:0/1514500930 >> v1:172.21.15.132:6812/1835401922 conn(0x7ff17c037820 legacy=0x7ff17c039ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.055+0000 7ff1a0eb5640 10 monclient: shutdown 2024-01-24T23:23:39.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.055+0000 7ff1a0eb5640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:39.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.055+0000 7ff1a0eb5640 1 -- 172.21.15.132:0/1514500930 >> v1:172.21.15.132:6790/0 conn(0x7ff19c0a2d50 legacy=0x7ff19c1cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.055+0000 7ff1a0eb5640 1 -- 172.21.15.132:0/1514500930 shutdown_connections 2024-01-24T23:23:39.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.055+0000 7ff1a0eb5640 1 -- 172.21.15.132:0/1514500930 >> 172.21.15.132:0/1514500930 conn(0x7ff19c0a6740 msgr2=0x7ff19c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:39.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.055+0000 7ff1a0eb5640 1 -- 172.21.15.132:0/1514500930 shutdown_connections 2024-01-24T23:23:39.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.055+0000 7ff1a0eb5640 1 -- 172.21.15.132:0/1514500930 wait complete. 2024-01-24T23:23:39.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.035+0000 7f12f84f6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:23:39.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.035+0000 7f12f84f6640 1 -- 172.21.15.115:0/1868288115 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f12d8009df0 con 0x7f12d80090b0 2024-01-24T23:23:39.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1617567181 0 0) 0x7f12d8009df0 con 0x7f12d80090b0 2024-01-24T23:23:39.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12e6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12e6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12e6ffd640 10 monclient(hunting): my global_id is 5163 2024-01-24T23:23:39.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12d80065d0 con 0x7f12d80090b0 2024-01-24T23:23:39.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1859366173 0 0) 0x7f12d80065d0 con 0x7f12d80090b0 2024-01-24T23:23:39.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f12d8009df0 con 0x7f12d80090b0 2024-01-24T23:23:39.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f12e00055f0 con 0x7f12d80090b0 2024-01-24T23:23:39.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12e6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12e6ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:39.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12e6ffd640 10 monclient: dump: 2024-01-24T23:23:39.063 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.063 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.063 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.064 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.064 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.064 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.064 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.064 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.064 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.064 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.064 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.064 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.064 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.064 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.064 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.064 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3054351209 0 0) 0x7f12d8009df0 con 0x7f12d80090b0 2024-01-24T23:23:39.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 --> v1:172.21.15.115:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f12d800c310 con 0x7f12d80090b0 2024-01-24T23:23:39.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7510 (unknown 0 0 2711847353) 0x7f12e0006ba0 con 0x7f12d80090b0 2024-01-24T23:23:39.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12e6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:23:39.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12e6ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:39.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 >> v1:172.21.15.115:6792/0 conn(0x7f12d80090b0 legacy=0x7f12d8009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12e6ffd640 1 -- 172.21.15.115:0/1868288115 >> v1:172.21.15.115:6792/0 conn(0x7f12d80090b0 legacy=0x7f12d8009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:39.065 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:23:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:23:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 20, 2024-01-24T23:23:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:23:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:23:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:23:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:23:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:23:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:23:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:23:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:23:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:23:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:23:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:23:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:23:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:23:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:23:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:23:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:23:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:23:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:23:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:23:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:23:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:23:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:23:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:23:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:23:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:23:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:23:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:23:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:23:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:23:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:23:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:23:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:23:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:23:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:23:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:23:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:23:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:23:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:23:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:23:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:23:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:23:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:23:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:23:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:23:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:23:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:23:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:23:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:23:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:23:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:23:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:23:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:23:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:23:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:23:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:23:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:23:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:23:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:23:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 3 2024-01-24T23:23:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:23:39.083 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:23:39.084 INFO:tasks.workunit.client.0.smithi132.stdout:foo 2024-01-24T23:23:39.084 INFO:tasks.workunit.client.0.smithi132.stdout:foo-ssd 2024-01-24T23:23:39.084 INFO:tasks.workunit.client.0.smithi132.stdout:foo-hdd 2024-01-24T23:23:39.085 INFO:tasks.workunit.client.0.smithi132.stdout:ec-foo 2024-01-24T23:23:39.085 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule rename foo foo-asdf 2024-01-24T23:23:39.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12f84f6640 1 -- 172.21.15.115:0/1868288115 >> v1:172.21.15.132:6812/1835401922 conn(0x7f12d4037850 legacy=0x7f12d4039d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12f84f6640 10 monclient: shutdown 2024-01-24T23:23:39.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12f84f6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:39.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12f84f6640 1 -- 172.21.15.115:0/1868288115 >> v1:172.21.15.115:6792/0 conn(0x7f12f0133bf0 legacy=0x7f12f0132fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12f6a93640 1 -- 172.21.15.115:0/1868288115 reap_dead start 2024-01-24T23:23:39.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12f6a93640 1 -- 172.21.15.115:0/1868288115 reap_dead start 2024-01-24T23:23:39.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12f84f6640 1 -- 172.21.15.115:0/1868288115 shutdown_connections 2024-01-24T23:23:39.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12f84f6640 1 -- 172.21.15.115:0/1868288115 >> 172.21.15.115:0/1868288115 conn(0x7f12f00662f0 msgr2=0x7f12f00a7010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:39.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12f84f6640 1 -- 172.21.15.115:0/1868288115 shutdown_connections 2024-01-24T23:23:39.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.039+0000 7f12f84f6640 1 -- 172.21.15.115:0/1868288115 wait complete. 2024-01-24T23:23:39.087 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-01-24T23:23:39.087 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon e 2024-01-24T23:23:39.087 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-01-24T23:23:39.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.191+0000 7ff01a271640 10 monclient: get_monmap_and_config 2024-01-24T23:23:39.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.191+0000 7ff01a271640 10 monclient: build_initial_monmap 2024-01-24T23:23:39.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.191+0000 7ff01a271640 10 monclient: monmap: 2024-01-24T23:23:39.194 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:39.194 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.194 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:39.194722+0000 2024-01-24T23:23:39.194 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:39.194722+0000 2024-01-24T23:23:39.194 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:39.194 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:39.194 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:39.194 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:39.194 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:39.194 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:39.194 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:39.194 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:39.194 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:39.194 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:39.195 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:39.195 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:39.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.191+0000 7ff01a271640 1 Processor -- start 2024-01-24T23:23:39.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.191+0000 7ff01a271640 1 -- start start 2024-01-24T23:23:39.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.191+0000 7ff01a271640 10 monclient: init 2024-01-24T23:23:39.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:39.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 10 monclient: _add_conns ranks=[3,8,6,4,2,5,1,0,7] 2024-01-24T23:23:39.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 10 monclient(hunting): picked mon.noname-g con 0x7ff0140a2a60 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:39.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 10 monclient(hunting): picked mon.noname-e con 0x7ff01409f170 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:39.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 10 monclient(hunting): picked mon.noname-d con 0x7ff0140a1130 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:39.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff014064700 con 0x7ff0140a2a60 2024-01-24T23:23:39.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff01409f550 con 0x7ff0140a1130 2024-01-24T23:23:39.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0140a0750 con 0x7ff01409f170 2024-01-24T23:23:39.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:39.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 10 monclient(hunting): authenticate will time out at 1106.520253s 2024-01-24T23:23:39.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff013fff640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7ff0140a2a60 0x7ff01409ea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:60388/0 (socket says 172.21.15.132:60388) 2024-01-24T23:23:39.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff013fff640 1 -- 172.21.15.132:0/739384989 learned_addr learned my addr 172.21.15.132:0/739384989 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:39.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 1 -- 172.21.15.132:0/739384989 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 116381152 0 0) 0x7ff014064700 con 0x7ff0140a2a60 2024-01-24T23:23:39.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 10 monclient(hunting): my global_id is 5088 2024-01-24T23:23:39.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 1 -- 172.21.15.132:0/739384989 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feffc003530 con 0x7ff0140a2a60 2024-01-24T23:23:39.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 1 -- 172.21.15.132:0/739384989 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2817145969 0 0) 0x7ff00c001120 con 0x7ff0140a1130 2024-01-24T23:23:39.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 10 monclient(hunting): my global_id is 5172 2024-01-24T23:23:39.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 1 -- 172.21.15.132:0/739384989 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff014064700 con 0x7ff0140a1130 2024-01-24T23:23:39.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 1 -- 172.21.15.132:0/739384989 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 417526160 0 0) 0x7ff0140a0750 con 0x7ff01409f170 2024-01-24T23:23:39.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 10 monclient(hunting): my global_id is 84320 2024-01-24T23:23:39.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 1 -- 172.21.15.132:0/739384989 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff00c001120 con 0x7ff01409f170 2024-01-24T23:23:39.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 1 -- 172.21.15.132:0/739384989 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2453891667 0 0) 0x7feffc003530 con 0x7ff0140a2a60 2024-01-24T23:23:39.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 1 -- 172.21.15.132:0/739384989 >> v1:172.21.15.115:6793/0 conn(0x7ff01409f170 legacy=0x7ff014139d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 1 -- 172.21.15.132:0/739384989 >> v1:172.21.15.115:6792/0 conn(0x7ff0140a1130 legacy=0x7ff01413d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:39.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 1 monclient: found mon.noname-g 2024-01-24T23:23:39.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:39.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:23:39.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 1 -- 172.21.15.132:0/739384989 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0140a2e40 con 0x7ff0140a2a60 2024-01-24T23:23:39.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 10 monclient: _finish_auth 0 2024-01-24T23:23:39.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:39.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:39.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 1 -- 172.21.15.132:0/739384989 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff004001b40 con 0x7ff0140a2a60 2024-01-24T23:23:39.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:39.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 10 monclient: dump: 2024-01-24T23:23:39.212 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:39.212 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.212 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.213 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.213 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.213 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:39.213 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.213 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.213 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.213 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.213 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.213 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.213 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.213 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.213 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.213 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:39.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 5 monclient: authenticate success, global_id 5088 2024-01-24T23:23:39.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:39.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 1 -- 172.21.15.132:0/739384989 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff004001e20 con 0x7ff0140a2a60 2024-01-24T23:23:39.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:39.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 1 -- 172.21.15.132:0/739384989 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff004004f70 con 0x7ff0140a2a60 2024-01-24T23:23:39.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:39.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 4 set_mon_vals no callback set 2024-01-24T23:23:39.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:39.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff012ffd640 10 monclient: dump: 2024-01-24T23:23:39.214 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:39.214 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.214 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.214 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.214 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.214 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:39.214 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.214 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.214 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.215 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.215 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.215 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.215 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.215 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.215 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.215 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:39.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 10 monclient: shutdown 2024-01-24T23:23:39.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:39.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 1 -- 172.21.15.132:0/739384989 >> v1:172.21.15.132:6790/0 conn(0x7ff0140a2a60 legacy=0x7ff01409ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 1 -- 172.21.15.132:0/739384989 shutdown_connections 2024-01-24T23:23:39.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 1 -- 172.21.15.132:0/739384989 >> 172.21.15.132:0/739384989 conn(0x7ff0140a7990 msgr2=0x7ff0140a3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:39.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 1 -- 172.21.15.132:0/739384989 shutdown_connections 2024-01-24T23:23:39.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 1 -- 172.21.15.132:0/739384989 wait complete. 2024-01-24T23:23:39.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 10 monclient: build_initial_monmap 2024-01-24T23:23:39.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 10 monclient: monmap: 2024-01-24T23:23:39.216 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:39.216 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:39.216 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:39.216 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:39.216 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:39.216 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:39.216 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:39.216 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:39.216 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:39.216 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:39.216 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:39.216 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:39.217 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:39.217 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:39.217 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:39.217 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:39.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 1 Processor -- start 2024-01-24T23:23:39.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 1 -- start start 2024-01-24T23:23:39.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 10 monclient: init 2024-01-24T23:23:39.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:39.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 10 monclient: _add_conns ranks=[6,3,0,8,1,5,2,4,7] 2024-01-24T23:23:39.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 10 monclient(hunting): picked mon.noname-d con 0x7ff01409f170 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:39.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 10 monclient(hunting): picked mon.noname-g con 0x7ff0140a1130 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:39.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 10 monclient(hunting): picked mon.noname-a con 0x7ff0140a2a60 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:39.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff014072760 con 0x7ff0140a2a60 2024-01-24T23:23:39.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0141c8d00 con 0x7ff0140a1130 2024-01-24T23:23:39.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0141a95a0 con 0x7ff01409f170 2024-01-24T23:23:39.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:39.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01a271640 10 monclient(hunting): authenticate will time out at 1106.523278s 2024-01-24T23:23:39.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff01880e640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7ff0140a2a60 legacy=0x7ff0141d3de0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:39.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff0137fe640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7ff0140a1130 0x7ff0141d06b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:60404/0 (socket says 172.21.15.132:60404) 2024-01-24T23:23:39.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff0137fe640 1 -- 172.21.15.132:0/656881249 learned_addr learned my addr 172.21.15.132:0/656881249 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:39.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff010ff9640 1 -- 172.21.15.132:0/656881249 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 842127404 0 0) 0x7ff0141c8d00 con 0x7ff0140a1130 2024-01-24T23:23:39.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff010ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff010ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff010ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff010ff9640 10 monclient(hunting): my global_id is 5097 2024-01-24T23:23:39.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff010ff9640 1 -- 172.21.15.132:0/656881249 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff000003530 con 0x7ff0140a1130 2024-01-24T23:23:39.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff010ff9640 1 -- 172.21.15.132:0/656881249 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2192741756 0 0) 0x7ff0141a95a0 con 0x7ff01409f170 2024-01-24T23:23:39.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff010ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff010ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff010ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff010ff9640 10 monclient(hunting): my global_id is 5190 2024-01-24T23:23:39.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff010ff9640 1 -- 172.21.15.132:0/656881249 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0141c8d00 con 0x7ff01409f170 2024-01-24T23:23:39.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff010ff9640 1 -- 172.21.15.132:0/656881249 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 725781513 0 0) 0x7ff000003530 con 0x7ff0140a1130 2024-01-24T23:23:39.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff010ff9640 1 -- 172.21.15.132:0/656881249 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff0141a95a0 con 0x7ff0140a1130 2024-01-24T23:23:39.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff010ff9640 1 -- 172.21.15.132:0/656881249 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff008001ae0 con 0x7ff0140a1130 2024-01-24T23:23:39.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff010ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff010ff9640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:39.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff010ff9640 10 monclient(hunting): dump: 2024-01-24T23:23:39.219 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:39.219 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.219 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.219 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.219 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.219 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:39.219 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.219 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.219 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.219 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.220 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.220 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.220 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.220 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.220 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.220 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:39.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.195+0000 7ff010ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:39.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 1 -- 172.21.15.132:0/656881249 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3556941337 0 0) 0x7ff0141c8d00 con 0x7ff01409f170 2024-01-24T23:23:39.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 1 -- 172.21.15.132:0/656881249 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff000003530 con 0x7ff01409f170 2024-01-24T23:23:39.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 1 -- 172.21.15.132:0/656881249 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff004004eb0 con 0x7ff01409f170 2024-01-24T23:23:39.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:39.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 10 monclient(hunting): dump: 2024-01-24T23:23:39.220 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:39.220 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.221 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.221 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.221 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.221 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:39.221 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.221 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.221 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.221 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.221 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.221 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.221 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.221 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.221 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.221 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:39.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 1 -- 172.21.15.132:0/656881249 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 155484924 0 0) 0x7ff0141a95a0 con 0x7ff0140a1130 2024-01-24T23:23:39.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 1 -- 172.21.15.132:0/656881249 >> v1:172.21.15.115:6792/0 conn(0x7ff01409f170 legacy=0x7ff0141b9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 1 -- 172.21.15.132:0/656881249 >> v1:172.21.15.115:6789/0 conn(0x7ff0140a2a60 legacy=0x7ff0141d3de0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:39.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:23:39.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 1 monclient: found mon.g 2024-01-24T23:23:39.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:39.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:39.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 1 -- 172.21.15.132:0/656881249 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0141a7e10 con 0x7ff0140a1130 2024-01-24T23:23:39.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 10 monclient: _finish_auth 0 2024-01-24T23:23:39.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:39.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:39.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff01a271640 5 monclient: authenticate success, global_id 5097 2024-01-24T23:23:39.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff01a271640 10 monclient: _renew_subs 2024-01-24T23:23:39.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff01a271640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:39.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff01a271640 1 -- 172.21.15.132:0/656881249 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0141a87d0 con 0x7ff0140a1130 2024-01-24T23:23:39.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff01a271640 10 monclient: _renew_subs 2024-01-24T23:23:39.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff01a271640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:39.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff01a271640 1 -- 172.21.15.132:0/656881249 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0141a92c0 con 0x7ff0140a1130 2024-01-24T23:23:39.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 1 -- 172.21.15.132:0/656881249 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff0080038d0 con 0x7ff0140a1130 2024-01-24T23:23:39.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:39.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 1 -- 172.21.15.132:0/656881249 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff008003e00 con 0x7ff0140a1130 2024-01-24T23:23:39.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:39.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff01a271640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:39.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 10 monclient: dump: 2024-01-24T23:23:39.223 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:39.223 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.224 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.224 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.224 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.224 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:39.224 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.224 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.224 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.224 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.224 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.224 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.224 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.224 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.224 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.224 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:39.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff0127fc640 4 set_mon_vals no callback set 2024-01-24T23:23:39.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 1 -- 172.21.15.132:0/656881249 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7ff008003fe0 con 0x7ff0140a1130 2024-01-24T23:23:39.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff010ff9640 1 -- 172.21.15.132:0/656881249 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(52..52 src has 1..52) v4 ==== 4305+0+0 (unknown 1637139180 0 0) 0x7ff008043a60 con 0x7ff0140a1130 2024-01-24T23:23:39.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff01a271640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:39.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff01a271640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:39.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff01a271640 1 -- 172.21.15.132:0/656881249 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff014073400 con 0x7ff0140a1130 2024-01-24T23:23:39.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.199+0000 7ff013fff640 10 monclient: get_auth_request con 0x7ff000037510 auth_method 0 2024-01-24T23:23:39.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.203+0000 7ff010ff9640 1 -- 172.21.15.132:0/656881249 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7ff008014900 con 0x7ff0140a1130 2024-01-24T23:23:39.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.203+0000 7ff010ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:39.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.203+0000 7ff010ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:39.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 10 monclient: get_monmap_and_config 2024-01-24T23:23:39.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 10 monclient: build_initial_monmap 2024-01-24T23:23:39.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 10 monclient: monmap: 2024-01-24T23:23:39.227 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:39.227 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.227 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:39.195099+0000 2024-01-24T23:23:39.227 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:39.195099+0000 2024-01-24T23:23:39.227 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:39.228 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.228 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:39.228 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:39.228 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:39.228 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:39.228 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:39.228 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:39.228 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:39.228 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:39.228 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:39.228 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 1 Processor -- start 2024-01-24T23:23:39.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 1 -- start start 2024-01-24T23:23:39.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 10 monclient: init 2024-01-24T23:23:39.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:39.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 10 monclient: _add_conns ranks=[4,1,6,8,2,0,3,5,7] 2024-01-24T23:23:39.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 10 monclient(hunting): picked mon.noname-c con 0x7fe1c806a5b0 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:39.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 10 monclient(hunting): picked mon.noname-f con 0x7fe1c80a3130 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:39.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 10 monclient(hunting): picked mon.noname-d con 0x7fe1c809f1a0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:39.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c7fff640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7fe1c806a5b0 legacy=0x7fe1c806a990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:23:39.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c804ed70 con 0x7fe1c80a3130 2024-01-24T23:23:39.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c806c260 con 0x7fe1c806a5b0 2024-01-24T23:23:39.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c77fe640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fe1c80a3130 legacy=0x7fe1c813bff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:39.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c80a37c0 con 0x7fe1c809f1a0 2024-01-24T23:23:39.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:39.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 10 monclient(hunting): authenticate will time out at 1098.117743s 2024-01-24T23:23:39.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1ccfdc640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fe1c809f1a0 0x7fe1c813f720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:44364/0 (socket says 172.21.15.115:44364) 2024-01-24T23:23:39.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1ccfdc640 1 -- 172.21.15.115:0/1550339840 learned_addr learned my addr 172.21.15.115:0/1550339840 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:39.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 1 -- 172.21.15.115:0/1550339840 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1220779006 0 0) 0x7fe1c80a37c0 con 0x7fe1c809f1a0 2024-01-24T23:23:39.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 10 monclient(hunting): my global_id is 5181 2024-01-24T23:23:39.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 1 -- 172.21.15.115:0/1550339840 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1b4003530 con 0x7fe1c809f1a0 2024-01-24T23:23:39.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 1 -- 172.21.15.115:0/1550339840 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3729922337 0 0) 0x7fe1b4003530 con 0x7fe1c809f1a0 2024-01-24T23:23:39.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 1 -- 172.21.15.115:0/1550339840 >> v1:172.21.15.115:6791/0 conn(0x7fe1c806a5b0 legacy=0x7fe1c806a990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:39.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 1 -- 172.21.15.115:0/1550339840 >> v1:172.21.15.132:6789/0 conn(0x7fe1c80a3130 legacy=0x7fe1c813bff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:39.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:39.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 1 monclient: found mon.noname-d 2024-01-24T23:23:39.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:39.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:23:39.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 1 -- 172.21.15.115:0/1550339840 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c80a2bf0 con 0x7fe1c809f1a0 2024-01-24T23:23:39.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 10 monclient: _finish_auth 0 2024-01-24T23:23:39.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:39.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:39.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 1 -- 172.21.15.115:0/1550339840 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe1c0001b40 con 0x7fe1c809f1a0 2024-01-24T23:23:39.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:23:39.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 10 monclient: dump: 2024-01-24T23:23:39.231 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.231 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.231 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.231 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.231 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.231 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.231 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.231 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.231 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.231 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.231 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.231 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.231 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.231 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.232 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.232 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 5 monclient: authenticate success, global_id 5181 2024-01-24T23:23:39.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:39.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 1 -- 172.21.15.115:0/1550339840 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe1c0001e60 con 0x7fe1c809f1a0 2024-01-24T23:23:39.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:39.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 1 -- 172.21.15.115:0/1550339840 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe1c0004f70 con 0x7fe1c809f1a0 2024-01-24T23:23:39.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:39.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1c6ffd640 10 monclient: dump: 2024-01-24T23:23:39.232 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.232 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.232 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.232 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.232 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.233 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.233 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.233 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.233 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.233 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.233 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.233 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.233 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.233 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.233 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.233 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:39.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 4 set_mon_vals no callback set 2024-01-24T23:23:39.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 10 monclient: shutdown 2024-01-24T23:23:39.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:39.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 1 -- 172.21.15.115:0/1550339840 >> v1:172.21.15.115:6792/0 conn(0x7fe1c809f1a0 legacy=0x7fe1c813f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 1 -- 172.21.15.115:0/1550339840 shutdown_connections 2024-01-24T23:23:39.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 1 -- 172.21.15.115:0/1550339840 >> 172.21.15.115:0/1550339840 conn(0x7fe1c80664c0 msgr2=0x7fe1c80668d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:39.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 1 -- 172.21.15.115:0/1550339840 shutdown_connections 2024-01-24T23:23:39.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.191+0000 7fe1cea3f640 1 -- 172.21.15.115:0/1550339840 wait complete. 2024-01-24T23:23:39.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1cea3f640 10 monclient: build_initial_monmap 2024-01-24T23:23:39.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1cea3f640 10 monclient: monmap: 2024-01-24T23:23:39.234 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:39.234 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:39.234 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:39.234 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:39.234 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:39.234 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:39.234 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:39.234 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:39.234 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:39.234 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:39.234 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:39.235 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:39.235 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:39.235 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:39.235 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:39.235 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1cea3f640 1 Processor -- start 2024-01-24T23:23:39.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1cea3f640 1 -- start start 2024-01-24T23:23:39.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1cea3f640 10 monclient: init 2024-01-24T23:23:39.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1cea3f640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:39.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1cea3f640 10 monclient: _add_conns ranks=[2,7,4,8,6,5,3,0,1] 2024-01-24T23:23:39.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1cea3f640 10 monclient(hunting): picked mon.noname-b con 0x7fe1c806a5b0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:39.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1cea3f640 10 monclient(hunting): picked mon.noname-i con 0x7fe1c809f1a0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:39.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1cea3f640 10 monclient(hunting): picked mon.noname-c con 0x7fe1c80a3130 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:39.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1cea3f640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c8071fd0 con 0x7fe1c806a5b0 2024-01-24T23:23:39.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1cea3f640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c81a9c20 con 0x7fe1c80a3130 2024-01-24T23:23:39.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1cea3f640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c81c53d0 con 0x7fe1c809f1a0 2024-01-24T23:23:39.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1cea3f640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:39.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1cea3f640 10 monclient(hunting): authenticate will time out at 1098.121554s 2024-01-24T23:23:39.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1ccfdc640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7fe1c80a3130 legacy=0x7fe1c81d3220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:23:39.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c7fff640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fe1c806a5b0 0x7fe1c81c4cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:38604/0 (socket says 172.21.15.115:38604) 2024-01-24T23:23:39.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c7fff640 1 -- 172.21.15.115:0/1581668424 learned_addr learned my addr 172.21.15.115:0/1581668424 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:39.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 867638300 0 0) 0x7fe1c8071fd0 con 0x7fe1c806a5b0 2024-01-24T23:23:39.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient(hunting): my global_id is 84386 2024-01-24T23:23:39.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1a8003530 con 0x7fe1c806a5b0 2024-01-24T23:23:39.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1807010740 0 0) 0x7fe1c81c53d0 con 0x7fe1c809f1a0 2024-01-24T23:23:39.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient(hunting): my global_id is 5173 2024-01-24T23:23:39.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1c8071fd0 con 0x7fe1c809f1a0 2024-01-24T23:23:39.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3979198433 0 0) 0x7fe1a8003530 con 0x7fe1c806a5b0 2024-01-24T23:23:39.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe1c81c53d0 con 0x7fe1c806a5b0 2024-01-24T23:23:39.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe1b8004300 con 0x7fe1c806a5b0 2024-01-24T23:23:39.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:23:39.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient(hunting): dump: 2024-01-24T23:23:39.237 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.237 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.237 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.237 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.237 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.237 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.238 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.238 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.238 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.238 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.238 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.238 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.238 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.238 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.238 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.238 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:39.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3381419789 0 0) 0x7fe1c8071fd0 con 0x7fe1c809f1a0 2024-01-24T23:23:39.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe1a8003530 con 0x7fe1c809f1a0 2024-01-24T23:23:39.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe1b00042f0 con 0x7fe1c809f1a0 2024-01-24T23:23:39.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:39.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient(hunting): dump: 2024-01-24T23:23:39.239 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.239 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.239 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.239 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.239 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.239 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.239 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.239 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.239 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.239 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.239 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.239 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.239 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.239 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.240 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.240 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3486363493 0 0) 0x7fe1c81c53d0 con 0x7fe1c806a5b0 2024-01-24T23:23:39.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 >> v1:172.21.15.132:6792/0 conn(0x7fe1c809f1a0 legacy=0x7fe1c81c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 >> v1:172.21.15.115:6791/0 conn(0x7fe1c80a3130 legacy=0x7fe1c81d3220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:39.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:23:39.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 1 monclient: found mon.b 2024-01-24T23:23:39.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:39.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:39.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c81abd80 con 0x7fe1c806a5b0 2024-01-24T23:23:39.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient: _finish_auth 0 2024-01-24T23:23:39.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:39.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:39.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1cea3f640 5 monclient: authenticate success, global_id 84386 2024-01-24T23:23:39.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1cea3f640 10 monclient: _renew_subs 2024-01-24T23:23:39.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1cea3f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:39.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1cea3f640 1 -- 172.21.15.115:0/1581668424 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c806b5c0 con 0x7fe1c806a5b0 2024-01-24T23:23:39.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1cea3f640 10 monclient: _renew_subs 2024-01-24T23:23:39.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1cea3f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:39.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1cea3f640 1 -- 172.21.15.115:0/1581668424 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c81ac620 con 0x7fe1c806a5b0 2024-01-24T23:23:39.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe1b80045e0 con 0x7fe1c806a5b0 2024-01-24T23:23:39.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:39.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe1b8004b10 con 0x7fe1c806a5b0 2024-01-24T23:23:39.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c67fc640 4 set_mon_vals no callback set 2024-01-24T23:23:39.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:39.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient: dump: 2024-01-24T23:23:39.241 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.241 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.241 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.242 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.242 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.242 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.242 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.242 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.242 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.242 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.242 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.242 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.242 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.242 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.242 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.242 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1cea3f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:39.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1cea3f640 1 -- 172.21.15.115:0/1581668424 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe1ac0065d0 con 0x7fe1ac002870 2024-01-24T23:23:39.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fe1b8004cf0 con 0x7fe1c806a5b0 2024-01-24T23:23:39.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(52..52 src has 1..52) v4 ==== 4305+0+0 (unknown 1637139180 0 0) 0x7fe1b80438f0 con 0x7fe1c806a5b0 2024-01-24T23:23:39.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078645141 0 0) 0x7fe1ac0065d0 con 0x7fe1ac002870 2024-01-24T23:23:39.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient(hunting): my global_id is 84329 2024-01-24T23:23:39.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1c81c53d0 con 0x7fe1ac002870 2024-01-24T23:23:39.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1ccfdc640 10 monclient: get_auth_request con 0x7fe1a80376a0 auth_method 0 2024-01-24T23:23:39.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3503406233 0 0) 0x7fe1c81c53d0 con 0x7fe1ac002870 2024-01-24T23:23:39.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe1ac0065d0 con 0x7fe1ac002870 2024-01-24T23:23:39.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe1b0004ad0 con 0x7fe1ac002870 2024-01-24T23:23:39.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:39.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.195+0000 7fe1c4ff9640 10 monclient: dump: 2024-01-24T23:23:39.244 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.244 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.244 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.244 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.244 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.244 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.244 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.244 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.244 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.244 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.244 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.244 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.244 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.244 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.244 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.245 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.199+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2707208374 0 0) 0x7fe1ac0065d0 con 0x7fe1ac002870 2024-01-24T23:23:39.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.199+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 --> v1:172.21.15.115:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1ac00a120 con 0x7fe1ac002870 2024-01-24T23:23:39.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.199+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fe1b0004f60 con 0x7fe1ac002870 2024-01-24T23:23:39.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.199+0000 7fe1c4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:39.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.199+0000 7fe1c4ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:39.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.199+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 >> v1:172.21.15.115:6793/0 conn(0x7fe1ac002870 legacy=0x7fe1ac004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.199+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 >> v1:172.21.15.115:6793/0 conn(0x7fe1ac002870 legacy=0x7fe1ac004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:39.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1cea3f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:23:39.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1cea3f640 1 -- 172.21.15.115:0/1581668424 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe1ac009df0 con 0x7fe1ac0090b0 2024-01-24T23:23:39.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204819675 0 0) 0x7fe1ac009df0 con 0x7fe1ac0090b0 2024-01-24T23:23:39.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1c4ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1c4ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1c4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1c4ff9640 10 monclient(hunting): my global_id is 84338 2024-01-24T23:23:39.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1ac0065d0 con 0x7fe1ac0090b0 2024-01-24T23:23:39.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3659189350 0 0) 0x7fe1ac0065d0 con 0x7fe1ac0090b0 2024-01-24T23:23:39.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe1ac009df0 con 0x7fe1ac0090b0 2024-01-24T23:23:39.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe1b00057e0 con 0x7fe1ac0090b0 2024-01-24T23:23:39.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1c4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1c4ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:39.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1c4ff9640 10 monclient: dump: 2024-01-24T23:23:39.246 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.246 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.246 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.246 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.246 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.246 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.246 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.246 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.246 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.247 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.247 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.247 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.247 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.247 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.247 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.247 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2956374023 0 0) 0x7fe1ac009df0 con 0x7fe1ac0090b0 2024-01-24T23:23:39.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 --> v1:172.21.15.115:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe1ac00c310 con 0x7fe1ac0090b0 2024-01-24T23:23:39.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7680 (unknown 0 0 92678463) 0x7fe1b00059d0 con 0x7fe1ac0090b0 2024-01-24T23:23:39.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1c4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:23:39.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1c4ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:39.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 >> v1:172.21.15.115:6793/0 conn(0x7fe1ac0090b0 legacy=0x7fe1ac009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1c4ff9640 1 -- 172.21.15.115:0/1581668424 >> v1:172.21.15.115:6793/0 conn(0x7fe1ac0090b0 legacy=0x7fe1ac009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:39.248 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:23:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:23:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 20, 2024-01-24T23:23:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:23:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:23:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:23:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:23:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:23:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:23:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:23:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:23:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:23:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:23:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:39.249 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:23:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:23:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:39.250 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:23:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:23:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:23:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:23:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:23:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:23:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:23:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:23:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:23:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:23:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:23:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:23:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:23:39.251 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:23:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:39.252 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:23:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:23:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:23:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:23:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:23:39.253 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:23:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:23:39.254 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:23:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:23:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:23:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:23:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:23:39.256 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:23:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:23:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.257 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:23:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:23:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:23:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:23:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:23:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:23:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:39.259 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:23:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:23:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:23:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:23:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:23:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:23:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.261 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:23:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:23:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:23:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.262 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:23:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:23:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:23:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:39.263 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:39.264 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.264 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:39.264 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:23:39.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.264 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:39.264 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:39.264 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:39.264 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.264 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.264 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:23:39.264 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:23:39.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1cea3f640 1 -- 172.21.15.115:0/1581668424 >> v1:172.21.15.132:6812/1835401922 conn(0x7fe1a80376a0 legacy=0x7fe1a8039b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1cea3f640 10 monclient: shutdown 2024-01-24T23:23:39.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1cea3f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:39.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1cea3f640 1 -- 172.21.15.115:0/1581668424 >> v1:172.21.15.115:6790/0 conn(0x7fe1c806a5b0 legacy=0x7fe1c81c4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1ccfdc640 1 -- 172.21.15.115:0/1581668424 reap_dead start 2024-01-24T23:23:39.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1ccfdc640 1 -- 172.21.15.115:0/1581668424 reap_dead start 2024-01-24T23:23:39.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1cea3f640 1 -- 172.21.15.115:0/1581668424 shutdown_connections 2024-01-24T23:23:39.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1cea3f640 1 -- 172.21.15.115:0/1581668424 >> 172.21.15.115:0/1581668424 conn(0x7fe1c80664c0 msgr2=0x7fe1c8074860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:39.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1cea3f640 1 -- 172.21.15.115:0/1581668424 shutdown_connections 2024-01-24T23:23:39.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.203+0000 7fe1cea3f640 1 -- 172.21.15.115:0/1581668424 wait complete. 2024-01-24T23:23:39.266 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-01-24T23:23:39.266 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon g 2024-01-24T23:23:39.267 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-01-24T23:23:39.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.363+0000 7ff01a271640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-01-24T23:23:39.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.363+0000 7ff01a271640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-01-24T23:23:39.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.363+0000 7ff01a271640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:39.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.363+0000 7ff01a271640 1 -- 172.21.15.132:0/656881249 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"} v 0) v1 -- 0x7ff0141ba2d0 con 0x7ff0140a1130 2024-01-24T23:23:39.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 10 monclient: get_monmap_and_config 2024-01-24T23:23:39.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 10 monclient: build_initial_monmap 2024-01-24T23:23:39.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 10 monclient: monmap: 2024-01-24T23:23:39.382 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:39.382 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.382 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:39.383115+0000 2024-01-24T23:23:39.382 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:39.383115+0000 2024-01-24T23:23:39.383 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:39.383 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.383 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:39.383 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:39.383 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:39.383 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:39.383 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:39.383 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:39.383 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:39.383 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:39.383 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:39.383 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 1 Processor -- start 2024-01-24T23:23:39.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 1 -- start start 2024-01-24T23:23:39.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 10 monclient: init 2024-01-24T23:23:39.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:39.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 10 monclient: _add_conns ranks=[0,5,1,3,7,8,4,6,2] 2024-01-24T23:23:39.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 10 monclient(hunting): picked mon.noname-a con 0x7fdb94133bf0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 10 monclient(hunting): picked mon.noname-h con 0x7fdb94133fd0 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 10 monclient(hunting): picked mon.noname-f con 0x7fdb94134430 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9404ed70 con 0x7fdb94133bf0 2024-01-24T23:23:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9409af30 con 0x7fdb94134430 2024-01-24T23:23:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9409b210 con 0x7fdb94133fd0 2024-01-24T23:23:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 10 monclient(hunting): authenticate will time out at 1098.305762s 2024-01-24T23:23:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9259c640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fdb94133bf0 legacy=0x7fdb9409bd80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb92d9d640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fdb94134430 legacy=0x7fdb9413f500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb91d9b640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fdb94133fd0 0x7fdb9413bdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.115:60538/0 (socket says 172.21.15.115:60538) 2024-01-24T23:23:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb91d9b640 1 -- 172.21.15.115:0/1990815875 learned_addr learned my addr 172.21.15.115:0/1990815875 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 1 -- 172.21.15.115:0/1990815875 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3927347052 0 0) 0x7fdb9409b210 con 0x7fdb94133fd0 2024-01-24T23:23:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 10 monclient(hunting): my global_id is 5072 2024-01-24T23:23:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 1 -- 172.21.15.115:0/1990815875 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb7c003530 con 0x7fdb94133fd0 2024-01-24T23:23:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 1 -- 172.21.15.115:0/1990815875 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2565648722 0 0) 0x7fdb7c003530 con 0x7fdb94133fd0 2024-01-24T23:23:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 1 -- 172.21.15.115:0/1990815875 >> v1:172.21.15.132:6789/0 conn(0x7fdb94134430 legacy=0x7fdb9413f500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 1 -- 172.21.15.115:0/1990815875 >> v1:172.21.15.115:6789/0 conn(0x7fdb94133bf0 legacy=0x7fdb9409bd80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 10 monclient: _finish_hunting 0 2024-01-24T23:23:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 1 monclient: found mon.noname-h 2024-01-24T23:23:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:23:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 1 -- 172.21.15.115:0/1990815875 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9409b4f0 con 0x7fdb94133fd0 2024-01-24T23:23:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 10 monclient: _finish_auth 0 2024-01-24T23:23:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 10 monclient: _check_auth_tickets 2024-01-24T23:23:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 1 -- 172.21.15.115:0/1990815875 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdb840017c0 con 0x7fdb94133fd0 2024-01-24T23:23:39.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 5 monclient: authenticate success, global_id 5072 2024-01-24T23:23:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 10 monclient: dump: 2024-01-24T23:23:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.395 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.395 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.395 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.395 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:39.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 1 -- 172.21.15.115:0/1990815875 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdb84001ae0 con 0x7fdb94133fd0 2024-01-24T23:23:39.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:39.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:39.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 1 -- 172.21.15.115:0/1990815875 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdb84004bf0 con 0x7fdb94133fd0 2024-01-24T23:23:39.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 4 set_mon_vals no callback set 2024-01-24T23:23:39.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:39.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9159a640 10 monclient: dump: 2024-01-24T23:23:39.395 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.395 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.396 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.396 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.396 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.396 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.396 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.396 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.396 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.396 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.396 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.396 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.396 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.396 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.396 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.396 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 10 monclient: shutdown 2024-01-24T23:23:39.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:39.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 1 -- 172.21.15.115:0/1990815875 >> v1:172.21.15.132:6791/0 conn(0x7fdb94133fd0 legacy=0x7fdb9413bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 1 -- 172.21.15.115:0/1990815875 shutdown_connections 2024-01-24T23:23:39.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 1 -- 172.21.15.115:0/1990815875 >> 172.21.15.115:0/1990815875 conn(0x7fdb940a9910 msgr2=0x7fdb940a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:39.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 1 -- 172.21.15.115:0/1990815875 shutdown_connections 2024-01-24T23:23:39.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.379+0000 7fdb9888d640 1 -- 172.21.15.115:0/1990815875 wait complete. 2024-01-24T23:23:39.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb9888d640 10 monclient: build_initial_monmap 2024-01-24T23:23:39.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb9888d640 10 monclient: monmap: 2024-01-24T23:23:39.397 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:39.397 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:39.397 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:39.397 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:39.397 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:39.397 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:39.397 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:39.398 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:39.398 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:39.398 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:39.398 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:39.398 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:39.398 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:39.398 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:39.398 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:39.398 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb9888d640 1 Processor -- start 2024-01-24T23:23:39.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb9888d640 1 -- start start 2024-01-24T23:23:39.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb9888d640 10 monclient: init 2024-01-24T23:23:39.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb9888d640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:39.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb9888d640 10 monclient: _add_conns ranks=[7,1,8,4,5,6,2,0,3] 2024-01-24T23:23:39.398 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb9888d640 10 monclient(hunting): picked mon.noname-i con 0x7fdb94133bf0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:39.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb9888d640 10 monclient(hunting): picked mon.noname-f con 0x7fdb94133fd0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:39.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb9888d640 10 monclient(hunting): picked mon.noname-e con 0x7fdb94134430 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:39.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb9888d640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb940a5960 con 0x7fdb94133fd0 2024-01-24T23:23:39.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb9888d640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9413a660 con 0x7fdb94133bf0 2024-01-24T23:23:39.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb9888d640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9413ac20 con 0x7fdb94134430 2024-01-24T23:23:39.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb9888d640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:39.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb9888d640 10 monclient(hunting): authenticate will time out at 1098.309329s 2024-01-24T23:23:39.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb91d9b640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fdb94133fd0 legacy=0x7fdb941cb600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:39.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb92d9d640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fdb94134430 0x7fdb941d3230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:53588/0 (socket says 172.21.15.115:53588) 2024-01-24T23:23:39.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb92d9d640 1 -- 172.21.15.115:0/990632235 learned_addr learned my addr 172.21.15.115:0/990632235 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:39.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2073608357 0 0) 0x7fdb9413ac20 con 0x7fdb94134430 2024-01-24T23:23:39.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 10 monclient(hunting): my global_id is 84347 2024-01-24T23:23:39.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb74003530 con 0x7fdb94134430 2024-01-24T23:23:39.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1069159409 0 0) 0x7fdb9413a660 con 0x7fdb94133bf0 2024-01-24T23:23:39.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 10 monclient(hunting): my global_id is 5182 2024-01-24T23:23:39.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb9413ac20 con 0x7fdb94133bf0 2024-01-24T23:23:39.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3011255222 0 0) 0x7fdb74003530 con 0x7fdb94134430 2024-01-24T23:23:39.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdb9413a660 con 0x7fdb94134430 2024-01-24T23:23:39.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdb8c0045a0 con 0x7fdb94134430 2024-01-24T23:23:39.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:23:39.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 10 monclient(hunting): dump: 2024-01-24T23:23:39.401 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.401 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.401 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.401 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.401 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.401 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.401 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.401 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.401 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.401 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.401 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.401 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.402 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.402 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.402 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.402 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:39.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2599208510 0 0) 0x7fdb9413a660 con 0x7fdb94134430 2024-01-24T23:23:39.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 >> v1:172.21.15.132:6792/0 conn(0x7fdb94133bf0 legacy=0x7fdb941caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 >> v1:172.21.15.132:6789/0 conn(0x7fdb94133fd0 legacy=0x7fdb941cb600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:39.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:39.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 1 monclient: found mon.e 2024-01-24T23:23:39.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:39.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:23:39.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb941cbd10 con 0x7fdb94134430 2024-01-24T23:23:39.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 10 monclient: _finish_auth 0 2024-01-24T23:23:39.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:39.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:39.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb9888d640 5 monclient: authenticate success, global_id 84347 2024-01-24T23:23:39.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb9888d640 10 monclient: _renew_subs 2024-01-24T23:23:39.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb9888d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:23:39.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb9888d640 1 -- 172.21.15.115:0/990632235 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb9409fe70 con 0x7fdb94134430 2024-01-24T23:23:39.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb9888d640 10 monclient: _renew_subs 2024-01-24T23:23:39.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb9888d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:23:39.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb9888d640 1 -- 172.21.15.115:0/990632235 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb941396f0 con 0x7fdb94134430 2024-01-24T23:23:39.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdb8c004880 con 0x7fdb94134430 2024-01-24T23:23:39.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:39.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdb8c004db0 con 0x7fdb94134430 2024-01-24T23:23:39.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb90d99640 4 set_mon_vals no callback set 2024-01-24T23:23:39.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:39.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 10 monclient: dump: 2024-01-24T23:23:39.403 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.404 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.404 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.404 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.404 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.404 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.404 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.404 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.404 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.404 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.404 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.404 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.404 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.404 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.404 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.404 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb9888d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:39.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb9888d640 1 -- 172.21.15.115:0/990632235 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdb780065d0 con 0x7fdb78002870 2024-01-24T23:23:39.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fdb8c004f90 con 0x7fdb94134430 2024-01-24T23:23:39.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(52..52 src has 1..52) v4 ==== 4305+0+0 (unknown 1637139180 0 0) 0x7fdb8c043bf0 con 0x7fdb94134430 2024-01-24T23:23:39.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb91d9b640 10 monclient: get_auth_request con 0x7fdb74037050 auth_method 0 2024-01-24T23:23:39.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408296708 0 0) 0x7fdb780065d0 con 0x7fdb78002870 2024-01-24T23:23:39.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 10 monclient(hunting): my global_id is 5106 2024-01-24T23:23:39.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.383+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb9413a660 con 0x7fdb78002870 2024-01-24T23:23:39.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.387+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4268227251 0 0) 0x7fdb9413a660 con 0x7fdb78002870 2024-01-24T23:23:39.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.387+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdb780065d0 con 0x7fdb78002870 2024-01-24T23:23:39.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.387+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdb88004250 con 0x7fdb78002870 2024-01-24T23:23:39.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.387+0000 7fdb82ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.387+0000 7fdb82ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:39.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.387+0000 7fdb82ffd640 10 monclient: dump: 2024-01-24T23:23:39.406 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.406 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.406 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.406 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.406 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.406 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.406 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.406 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.406 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.406 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.406 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.406 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.406 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.406 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.406 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.406 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.387+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2515195178 0 0) 0x7fdb780065d0 con 0x7fdb78002870 2024-01-24T23:23:39.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.387+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 --> v1:172.21.15.132:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb7800a120 con 0x7fdb78002870 2024-01-24T23:23:39.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.387+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fdb88006620 con 0x7fdb78002870 2024-01-24T23:23:39.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.387+0000 7fdb82ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:39.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.387+0000 7fdb82ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:39.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.387+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 >> v1:172.21.15.132:6790/0 conn(0x7fdb78002870 legacy=0x7fdb78004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.387+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 >> v1:172.21.15.132:6790/0 conn(0x7fdb78002870 legacy=0x7fdb78004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:39.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.391+0000 7fdb9888d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:23:39.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.391+0000 7fdb9888d640 1 -- 172.21.15.115:0/990632235 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdb78009df0 con 0x7fdb780090b0 2024-01-24T23:23:39.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.391+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2702588673 0 0) 0x7fdb78009df0 con 0x7fdb780090b0 2024-01-24T23:23:39.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.391+0000 7fdb82ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.391+0000 7fdb82ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.391+0000 7fdb82ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.391+0000 7fdb82ffd640 10 monclient(hunting): my global_id is 5115 2024-01-24T23:23:39.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.391+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb780065d0 con 0x7fdb780090b0 2024-01-24T23:23:39.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.391+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3097287445 0 0) 0x7fdb780065d0 con 0x7fdb780090b0 2024-01-24T23:23:39.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.391+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdb78009df0 con 0x7fdb780090b0 2024-01-24T23:23:39.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.391+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdb88004dd0 con 0x7fdb780090b0 2024-01-24T23:23:39.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.391+0000 7fdb82ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.391+0000 7fdb82ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:39.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.391+0000 7fdb82ffd640 10 monclient: dump: 2024-01-24T23:23:39.409 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.409 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.409 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.410 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.410 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.410 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.410 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.410 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.410 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.410 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.410 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.410 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.410 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.410 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.410 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.410 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.395+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1314002097 0 0) 0x7fdb78009df0 con 0x7fdb780090b0 2024-01-24T23:23:39.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.395+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 --> v1:172.21.15.132:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdb7800c310 con 0x7fdb780090b0 2024-01-24T23:23:39.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.395+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7682 (unknown 0 0 275176994) 0x7fdb880062e0 con 0x7fdb780090b0 2024-01-24T23:23:39.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.395+0000 7fdb82ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:23:39.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.395+0000 7fdb82ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:39.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.395+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 >> v1:172.21.15.132:6790/0 conn(0x7fdb780090b0 legacy=0x7fdb78009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.395+0000 7fdb82ffd640 1 -- 172.21.15.115:0/990632235 >> v1:172.21.15.132:6790/0 conn(0x7fdb780090b0 legacy=0x7fdb78009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:39.411 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:23:39.411 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:39.411 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:39.411 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "leader", 2024-01-24T23:23:39.411 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 20, 2024-01-24T23:23:39.412 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:23:39.412 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:23:39.412 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:23:39.412 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:23:39.412 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:23:39.412 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:23:39.412 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:23:39.412 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:39.412 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:23:39.412 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:39.412 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:23:39.412 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:23:39.412 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:39.413 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:39.413 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:39.413 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:39.413 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:39.413 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:39.413 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:39.413 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:39.413 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:39.413 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:39.413 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:39.413 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:39.413 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:23:39.413 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:23:39.413 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:39.413 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:39.414 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:39.414 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:39.414 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:39.414 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:39.414 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:39.414 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:39.414 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:39.414 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:39.414 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:39.414 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.414 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:23:39.414 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:23:39.414 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:23:39.414 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:23:39.415 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:23:39.415 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:23:39.415 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:39.415 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:39.415 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:23:39.415 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:23:39.415 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:23:39.415 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:23:39.415 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:23:39.415 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:23:39.415 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:23:39.415 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:39.415 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:23:39.415 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:39.415 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:39.416 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:39.416 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:39.416 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:39.416 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:39.416 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:39.416 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:39.416 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:39.416 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:39.416 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:39.416 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:39.416 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:23:39.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.416 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:23:39.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.416 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:39.417 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:39.417 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.417 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.417 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.417 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:23:39.417 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.417 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.417 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.417 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:23:39.417 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:23:39.417 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.417 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.417 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:23:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:23:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:23:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:23:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:23:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:23:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:23:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:23:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:23:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:23:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:23:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:23:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:23:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:23:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:23:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:23:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:23:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:23:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:23:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:23:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:23:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:23:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:23:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:23:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:23:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:23:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:23:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 3 2024-01-24T23:23:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.429 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:39.429 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": [ 2024-01-24T23:23:39.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.429 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:39.429 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:39.429 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:39.429 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.429 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.429 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:23:39.429 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:23:39.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.395+0000 7fdb9888d640 1 -- 172.21.15.115:0/990632235 >> v1:172.21.15.132:6812/1835401922 conn(0x7fdb74037050 legacy=0x7fdb74039510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.395+0000 7fdb9888d640 10 monclient: shutdown 2024-01-24T23:23:39.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.395+0000 7fdb9888d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:39.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.395+0000 7fdb9888d640 1 -- 172.21.15.115:0/990632235 >> v1:172.21.15.115:6793/0 conn(0x7fdb94134430 legacy=0x7fdb941d3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.395+0000 7fdb92d9d640 1 -- 172.21.15.115:0/990632235 reap_dead start 2024-01-24T23:23:39.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.395+0000 7fdb92d9d640 1 -- 172.21.15.115:0/990632235 reap_dead start 2024-01-24T23:23:39.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.395+0000 7fdb9888d640 1 -- 172.21.15.115:0/990632235 shutdown_connections 2024-01-24T23:23:39.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.395+0000 7fdb9888d640 1 -- 172.21.15.115:0/990632235 >> 172.21.15.115:0/990632235 conn(0x7fdb940a9910 msgr2=0x7fdb9413fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:39.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.395+0000 7fdb9888d640 1 -- 172.21.15.115:0/990632235 shutdown_connections 2024-01-24T23:23:39.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.395+0000 7fdb9888d640 1 -- 172.21.15.115:0/990632235 wait complete. 2024-01-24T23:23:39.432 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-01-24T23:23:39.432 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon h 2024-01-24T23:23:39.432 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-01-24T23:23:39.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.539+0000 7f49f55e7640 10 monclient: get_monmap_and_config 2024-01-24T23:23:39.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.539+0000 7f49f55e7640 10 monclient: build_initial_monmap 2024-01-24T23:23:39.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.539+0000 7f49f55e7640 10 monclient: monmap: 2024-01-24T23:23:39.544 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:39.545 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.545 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:39.545188+0000 2024-01-24T23:23:39.545 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:39.545188+0000 2024-01-24T23:23:39.545 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:39.545 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.545 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:39.545 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:39.545 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:39.545 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:39.545 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:39.545 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:39.545 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:39.545 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:39.545 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:39.545 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.539+0000 7f49f55e7640 1 Processor -- start 2024-01-24T23:23:39.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.539+0000 7f49f55e7640 1 -- start start 2024-01-24T23:23:39.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.539+0000 7f49f55e7640 10 monclient: init 2024-01-24T23:23:39.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.539+0000 7f49f55e7640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:39.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.539+0000 7f49f55e7640 10 monclient: _add_conns ranks=[1,3,5,0,4,7,2,8,6] 2024-01-24T23:23:39.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.539+0000 7f49f55e7640 10 monclient(hunting): picked mon.noname-f con 0x7f49f0133bf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:39.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.539+0000 7f49f55e7640 10 monclient(hunting): picked mon.noname-g con 0x7f49f0133fd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:39.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.539+0000 7f49f55e7640 10 monclient(hunting): picked mon.noname-h con 0x7f49f01343b0 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:39.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.539+0000 7f49f55e7640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f004ed70 con 0x7f49f0133bf0 2024-01-24T23:23:39.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.539+0000 7f49f55e7640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f00a4b60 con 0x7f49f0133fd0 2024-01-24T23:23:39.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.539+0000 7f49f55e7640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f009ae50 con 0x7f49f01343b0 2024-01-24T23:23:39.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.539+0000 7f49f55e7640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:39.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.539+0000 7f49f55e7640 10 monclient(hunting): authenticate will time out at 1098.467589s 2024-01-24T23:23:39.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.539+0000 7f49eeffd640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f49f0133bf0 legacy=0x7f49f009bb10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:39.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.539+0000 7f49ee7fc640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f49f0133fd0 0x7f49f013bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.115:53696/0 (socket says 172.21.15.115:53696) 2024-01-24T23:23:39.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.539+0000 7f49ee7fc640 1 -- 172.21.15.115:0/4093132633 learned_addr learned my addr 172.21.15.115:0/4093132633 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:39.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.539+0000 7f49ef7fe640 1 --1- 172.21.15.115:0/4093132633 >> v1:172.21.15.132:6791/0 conn(0x7f49f01343b0 0x7f49f013f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.115:60548/0 (socket says 172.21.15.115:60548) 2024-01-24T23:23:39.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 1 -- 172.21.15.115:0/4093132633 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1696557982 0 0) 0x7f49f009ae50 con 0x7f49f01343b0 2024-01-24T23:23:39.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 10 monclient(hunting): my global_id is 5081 2024-01-24T23:23:39.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 1 -- 172.21.15.115:0/4093132633 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49d8003530 con 0x7f49f01343b0 2024-01-24T23:23:39.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 1 -- 172.21.15.115:0/4093132633 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376505199 0 0) 0x7f49f00a4b60 con 0x7f49f0133fd0 2024-01-24T23:23:39.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 10 monclient(hunting): my global_id is 5124 2024-01-24T23:23:39.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 1 -- 172.21.15.115:0/4093132633 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49f009ae50 con 0x7f49f0133fd0 2024-01-24T23:23:39.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 1 -- 172.21.15.115:0/4093132633 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 533344070 0 0) 0x7f49f009ae50 con 0x7f49f0133fd0 2024-01-24T23:23:39.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 1 -- 172.21.15.115:0/4093132633 >> v1:172.21.15.132:6791/0 conn(0x7f49f01343b0 legacy=0x7f49f013f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 1 -- 172.21.15.115:0/4093132633 >> v1:172.21.15.132:6789/0 conn(0x7f49f0133bf0 legacy=0x7f49f009bb10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:39.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 10 monclient: _finish_hunting 0 2024-01-24T23:23:39.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 1 monclient: found mon.noname-g 2024-01-24T23:23:39.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:39.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:23:39.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 1 -- 172.21.15.115:0/4093132633 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f009b130 con 0x7f49f0133fd0 2024-01-24T23:23:39.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 10 monclient: _finish_auth 0 2024-01-24T23:23:39.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 10 monclient: _check_auth_tickets 2024-01-24T23:23:39.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:39.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49f55e7640 5 monclient: authenticate success, global_id 5124 2024-01-24T23:23:39.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49f55e7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:39.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 1 -- 172.21.15.115:0/4093132633 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f49e40016d0 con 0x7f49f0133fd0 2024-01-24T23:23:39.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:39.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 10 monclient: dump: 2024-01-24T23:23:39.556 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.556 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.556 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.557 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.557 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.557 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.557 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.557 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.557 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.557 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.557 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.557 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.557 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.557 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.557 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.557 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49f55e7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:39.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 1 -- 172.21.15.115:0/4093132633 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f49e4001960 con 0x7f49f0133fd0 2024-01-24T23:23:39.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:39.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49f55e7640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:39.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49f55e7640 4 set_mon_vals no callback set 2024-01-24T23:23:39.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 1 -- 172.21.15.115:0/4093132633 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f49e40048b0 con 0x7f49f0133fd0 2024-01-24T23:23:39.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:39.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.547+0000 7f49edffb640 10 monclient: dump: 2024-01-24T23:23:39.558 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.558 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.558 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.558 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.558 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.558 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.558 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.558 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.558 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.559 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49f55e7640 10 monclient: shutdown 2024-01-24T23:23:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49f55e7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49f55e7640 1 -- 172.21.15.115:0/4093132633 >> v1:172.21.15.132:6790/0 conn(0x7f49f0133fd0 legacy=0x7f49f013bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49f55e7640 1 -- 172.21.15.115:0/4093132633 shutdown_connections 2024-01-24T23:23:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49f55e7640 1 -- 172.21.15.115:0/4093132633 >> 172.21.15.115:0/4093132633 conn(0x7f49f00662f0 msgr2=0x7f49f0066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49f55e7640 1 -- 172.21.15.115:0/4093132633 shutdown_connections 2024-01-24T23:23:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49f55e7640 1 -- 172.21.15.115:0/4093132633 wait complete. 2024-01-24T23:23:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49f55e7640 10 monclient: build_initial_monmap 2024-01-24T23:23:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49f55e7640 10 monclient: monmap: 2024-01-24T23:23:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:39.561 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49f55e7640 1 Processor -- start 2024-01-24T23:23:39.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49f55e7640 1 -- start start 2024-01-24T23:23:39.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49f55e7640 10 monclient: init 2024-01-24T23:23:39.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49f55e7640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:39.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49f55e7640 10 monclient: _add_conns ranks=[2,0,7,4,1,3,5,6,8] 2024-01-24T23:23:39.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49f55e7640 10 monclient(hunting): picked mon.noname-b con 0x7f49f0133bf0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49f55e7640 10 monclient(hunting): picked mon.noname-a con 0x7f49f0133fd0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49f55e7640 10 monclient(hunting): picked mon.noname-i con 0x7f49f01343b0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49f55e7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f01cc040 con 0x7f49f0133fd0 2024-01-24T23:23:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49f55e7640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f0133690 con 0x7f49f0133bf0 2024-01-24T23:23:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49f55e7640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f009c810 con 0x7f49f01343b0 2024-01-24T23:23:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49f55e7640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49f55e7640 10 monclient(hunting): authenticate will time out at 1098.478095s 2024-01-24T23:23:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49ef7fe640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f49f01343b0 0x7f49f01d3c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.115:45890/0 (socket says 172.21.15.115:45890) 2024-01-24T23:23:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49ef7fe640 1 -- 172.21.15.115:0/3555658341 learned_addr learned my addr 172.21.15.115:0/3555658341 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.551+0000 7f49ee7fc640 1 -- 172.21.15.115:0/3555658341 >> v1:172.21.15.115:6789/0 conn(0x7f49f0133fd0 legacy=0x7f49f01d0510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 551172639 0 0) 0x7f49f0133690 con 0x7f49f0133bf0 2024-01-24T23:23:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 10 monclient(hunting): my global_id is 84395 2024-01-24T23:23:39.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49dc003530 con 0x7f49f0133bf0 2024-01-24T23:23:39.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3115162332 0 0) 0x7f49dc003530 con 0x7f49f0133bf0 2024-01-24T23:23:39.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f49f0133690 con 0x7f49f0133bf0 2024-01-24T23:23:39.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f49e8004670 con 0x7f49f0133bf0 2024-01-24T23:23:39.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:23:39.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 10 monclient(hunting): dump: 2024-01-24T23:23:39.567 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.567 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.567 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.567 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.567 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.567 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.567 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.567 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.568 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.568 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.568 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.568 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.568 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.568 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.568 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.568 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:39.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132066718 0 0) 0x7f49f009c810 con 0x7f49f01343b0 2024-01-24T23:23:39.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 10 monclient(hunting): my global_id is 5191 2024-01-24T23:23:39.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49dc003530 con 0x7f49f01343b0 2024-01-24T23:23:39.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1540957496 0 0) 0x7f49f0133690 con 0x7f49f0133bf0 2024-01-24T23:23:39.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 >> v1:172.21.15.132:6792/0 conn(0x7f49f01343b0 legacy=0x7f49f01d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 >> v1:172.21.15.115:6789/0 conn(0x7f49f0133fd0 legacy=0x7f49f01d0510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:39.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 10 monclient: _finish_hunting 0 2024-01-24T23:23:39.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 1 monclient: found mon.b 2024-01-24T23:23:39.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:39.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:39.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f00a5990 con 0x7f49f0133bf0 2024-01-24T23:23:39.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 10 monclient: _finish_auth 0 2024-01-24T23:23:39.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 10 monclient: _check_auth_tickets 2024-01-24T23:23:39.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49d77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:39.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49f55e7640 5 monclient: authenticate success, global_id 84395 2024-01-24T23:23:39.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49f55e7640 10 monclient: _renew_subs 2024-01-24T23:23:39.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49f55e7640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:39.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49f55e7640 1 -- 172.21.15.115:0/3555658341 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f01be5a0 con 0x7f49f0133bf0 2024-01-24T23:23:39.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49f55e7640 10 monclient: _renew_subs 2024-01-24T23:23:39.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49f55e7640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:39.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.555+0000 7f49f55e7640 1 -- 172.21.15.115:0/3555658341 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f0120960 con 0x7f49f0133bf0 2024-01-24T23:23:39.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.559+0000 7f49f55e7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:39.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.559+0000 7f49f55e7640 1 -- 172.21.15.115:0/3555658341 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f49f006a0d0 con 0x7f49f01bf270 2024-01-24T23:23:39.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.559+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f49e8004990 con 0x7f49f0133bf0 2024-01-24T23:23:39.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.559+0000 7f49d77fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:39.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.559+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f49e8004ec0 con 0x7f49f0133bf0 2024-01-24T23:23:39.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.559+0000 7f49d77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.559+0000 7f49d77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:39.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.559+0000 7f49ed7fa640 4 set_mon_vals no callback set 2024-01-24T23:23:39.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.559+0000 7f49d77fe640 10 monclient: dump: 2024-01-24T23:23:39.570 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.570 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.570 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.570 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.570 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.571 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.571 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.571 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.571 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.571 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.571 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.571 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.571 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.571 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.571 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.571 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.559+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f49e8005020 con 0x7f49f0133bf0 2024-01-24T23:23:39.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.559+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(52..52 src has 1..52) v4 ==== 4305+0+0 (unknown 1637139180 0 0) 0x7f49e8043c50 con 0x7f49f0133bf0 2024-01-24T23:23:39.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.559+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996839184 0 0) 0x7f49f006a0d0 con 0x7f49f01bf270 2024-01-24T23:23:39.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.559+0000 7f49d77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.559+0000 7f49d77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.559+0000 7f49d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.559+0000 7f49d77fe640 10 monclient(hunting): my global_id is 5090 2024-01-24T23:23:39.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.559+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49f0133690 con 0x7f49f01bf270 2024-01-24T23:23:39.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.559+0000 7f49ef7fe640 10 monclient: get_auth_request con 0x7f49dc036ed0 auth_method 0 2024-01-24T23:23:39.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.563+0000 7ff010ff9640 1 -- 172.21.15.132:0/656881249 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}]=0 v53) v1 ==== 110+0+0 (unknown 180126233 0 0) 0x7ff008012fa0 con 0x7ff0140a1130 2024-01-24T23:23:39.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.563+0000 7ff010ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-01-24T23:23:39.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.563+0000 7ff010ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:39.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.567+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1789634056 0 0) 0x7f49f0133690 con 0x7f49f01bf270 2024-01-24T23:23:39.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.567+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f49f006a0d0 con 0x7f49f01bf270 2024-01-24T23:23:39.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.567+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f49e40048b0 con 0x7f49f01bf270 2024-01-24T23:23:39.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.567+0000 7f49d77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.567+0000 7f49d77fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:39.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.567+0000 7f49d77fe640 10 monclient: dump: 2024-01-24T23:23:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.575 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.576 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.571+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2476986768 0 0) 0x7f49f006a0d0 con 0x7f49f01bf270 2024-01-24T23:23:39.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.571+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 --> v1:172.21.15.132:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f49f00a57c0 con 0x7f49f01bf270 2024-01-24T23:23:39.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.571+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f49e4001620 con 0x7f49f01bf270 2024-01-24T23:23:39.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.571+0000 7f49d77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:39.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.571+0000 7f49d77fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:39.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.571+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 >> v1:172.21.15.132:6791/0 conn(0x7f49f01bf270 legacy=0x7f49f009d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.571+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 >> v1:172.21.15.132:6791/0 conn(0x7f49f01bf270 legacy=0x7f49f009d7d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:39.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.575+0000 7f49f55e7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:23:39.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.575+0000 7f49f55e7640 1 -- 172.21.15.115:0/3555658341 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f49f01cb570 con 0x7f49f01cb190 2024-01-24T23:23:39.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.575+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3325648660 0 0) 0x7f49f01cb570 con 0x7f49f01cb190 2024-01-24T23:23:39.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.575+0000 7f49d77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.575+0000 7f49d77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.575+0000 7f49d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.575+0000 7f49d77fe640 10 monclient(hunting): my global_id is 5099 2024-01-24T23:23:39.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.575+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49f006a0d0 con 0x7f49f01cb190 2024-01-24T23:23:39.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.579+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3481100237 0 0) 0x7f49f006a0d0 con 0x7f49f01cb190 2024-01-24T23:23:39.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.579+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f49f01cb570 con 0x7f49f01cb190 2024-01-24T23:23:39.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.579+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f49e40050b0 con 0x7f49f01cb190 2024-01-24T23:23:39.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.579+0000 7f49d77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.579+0000 7f49d77fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:39.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.579+0000 7f49d77fe640 10 monclient: dump: 2024-01-24T23:23:39.583 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.583 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.583 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.583 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.583 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.583 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.583 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.583 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.583 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.584 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.584 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.584 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.584 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.584 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.584 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.584 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.579+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2474738491 0 0) 0x7f49f01cb570 con 0x7f49f01cb190 2024-01-24T23:23:39.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.579+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 --> v1:172.21.15.132:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f49f00711a0 con 0x7f49f01cb190 2024-01-24T23:23:39.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.579+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7680 (unknown 0 0 3442720273) 0x7f49e40073e0 con 0x7f49f01cb190 2024-01-24T23:23:39.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.579+0000 7f49d77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:23:39.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.579+0000 7f49d77fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:39.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.579+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 >> v1:172.21.15.132:6791/0 conn(0x7f49f01cb190 legacy=0x7f49f009dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.579+0000 7f49d77fe640 1 -- 172.21.15.115:0/3555658341 >> v1:172.21.15.132:6791/0 conn(0x7f49f01cb190 legacy=0x7f49f009dee0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:39.585 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:23:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:23:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 20, 2024-01-24T23:23:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:23:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:23:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:23:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:23:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:23:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:23:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:23:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 1, 2024-01-24T23:23:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:23:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:23:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:23:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:23:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:23:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:23:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:23:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:23:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:23:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:23:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:23:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:23:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:23:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:23:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:23:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:23:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:23:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:23:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:23:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:23:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:23:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:23:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:23:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:23:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:23:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:23:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:23:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:23:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:23:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:23:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:23:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:23:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:23:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:23:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:23:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:23:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:23:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:23:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:23:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:23:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:23:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:23:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:23:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:23:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:23:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:23:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:23:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:23:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:23:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:23:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:23:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:23:39.604 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:23:39.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.579+0000 7f49f55e7640 1 -- 172.21.15.115:0/3555658341 >> v1:172.21.15.132:6812/1835401922 conn(0x7f49dc036ed0 legacy=0x7f49dc039390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.579+0000 7f49f55e7640 10 monclient: shutdown 2024-01-24T23:23:39.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.579+0000 7f49f55e7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:39.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.579+0000 7f49f55e7640 1 -- 172.21.15.115:0/3555658341 >> v1:172.21.15.115:6790/0 conn(0x7f49f0133bf0 legacy=0x7f49f0132f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.579+0000 7f49ef7fe640 1 -- 172.21.15.115:0/3555658341 reap_dead start 2024-01-24T23:23:39.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.579+0000 7f49ef7fe640 1 -- 172.21.15.115:0/3555658341 reap_dead start 2024-01-24T23:23:39.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.579+0000 7f49f55e7640 1 -- 172.21.15.115:0/3555658341 shutdown_connections 2024-01-24T23:23:39.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.579+0000 7f49f55e7640 1 -- 172.21.15.115:0/3555658341 >> 172.21.15.115:0/3555658341 conn(0x7f49f00662f0 msgr2=0x7f49f00a3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:39.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.579+0000 7f49f55e7640 1 -- 172.21.15.115:0/3555658341 shutdown_connections 2024-01-24T23:23:39.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.579+0000 7f49f55e7640 1 -- 172.21.15.115:0/3555658341 wait complete. 2024-01-24T23:23:39.607 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-01-24T23:23:39.607 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon i 2024-01-24T23:23:39.607 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-01-24T23:23:39.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.667+0000 7ff01a271640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-01-24T23:23:39.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.667+0000 7ff01a271640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-01-24T23:23:39.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.667+0000 7ff01a271640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:39.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.667+0000 7ff01a271640 1 -- 172.21.15.132:0/656881249 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"} v 0) v1 -- 0x7ff0140008e0 con 0x7ff0140a1130 2024-01-24T23:23:39.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.667+0000 7ff010ff9640 1 -- 172.21.15.132:0/656881249 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}]=0 already renamed to 'foo-asdf' v53) v1 ==== 139+0+0 (unknown 627309034 0 0) 0x7ff008014720 con 0x7ff0140a1130 2024-01-24T23:23:39.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.667+0000 7ff010ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-01-24T23:23:39.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.667+0000 7ff010ff9640 10 monclient: _finish_command 3 = system:0 already renamed to 'foo-asdf' 2024-01-24T23:23:39.668 INFO:tasks.workunit.client.0.smithi132.stderr:already renamed to 'foo-asdf' 2024-01-24T23:23:39.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.667+0000 7ff01a271640 1 -- 172.21.15.132:0/656881249 >> v1:172.21.15.132:6812/1835401922 conn(0x7ff000037510 legacy=0x7ff0000399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.667+0000 7ff01a271640 10 monclient: shutdown 2024-01-24T23:23:39.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.667+0000 7ff01a271640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:39.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.667+0000 7ff01a271640 1 -- 172.21.15.132:0/656881249 >> v1:172.21.15.132:6790/0 conn(0x7ff0140a1130 legacy=0x7ff0141d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.667+0000 7ff01a271640 1 -- 172.21.15.132:0/656881249 shutdown_connections 2024-01-24T23:23:39.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.667+0000 7ff01a271640 1 -- 172.21.15.132:0/656881249 >> 172.21.15.132:0/656881249 conn(0x7ff0140a7990 msgr2=0x7ff0140a58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:39.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.667+0000 7ff01a271640 1 -- 172.21.15.132:0/656881249 shutdown_connections 2024-01-24T23:23:39.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.667+0000 7ff01a271640 1 -- 172.21.15.132:0/656881249 wait complete. 2024-01-24T23:23:39.685 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule rename foo foo-asdf 2024-01-24T23:23:39.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 10 monclient: get_monmap_and_config 2024-01-24T23:23:39.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 10 monclient: build_initial_monmap 2024-01-24T23:23:39.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 10 monclient: monmap: 2024-01-24T23:23:39.718 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:39.718 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.718 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:23:39.719131+0000 2024-01-24T23:23:39.718 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:23:39.719131+0000 2024-01-24T23:23:39.719 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:39.719 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.719 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:39.719 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:39.719 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:39.719 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:39.719 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:39.719 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:39.719 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:39.719 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:39.719 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:39.719 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 1 Processor -- start 2024-01-24T23:23:39.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 1 -- start start 2024-01-24T23:23:39.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 10 monclient: init 2024-01-24T23:23:39.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:39.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 10 monclient: _add_conns ranks=[0,7,6,2,8,1,3,4,5] 2024-01-24T23:23:39.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 10 monclient(hunting): picked mon.noname-a con 0x7f98a80a0a50 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:39.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 10 monclient(hunting): picked mon.noname-i con 0x7f98a80a1540 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:39.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 10 monclient(hunting): picked mon.noname-d con 0x7f98a8141d40 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:39.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a804f7e0 con 0x7f98a80a0a50 2024-01-24T23:23:39.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a80a1cf0 con 0x7f98a8141d40 2024-01-24T23:23:39.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a80a2780 con 0x7f98a80a1540 2024-01-24T23:23:39.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:39.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 10 monclient(hunting): authenticate will time out at 1098.641839s 2024-01-24T23:23:39.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a659c640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f98a80a0a50 legacy=0x7f98a80a0e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:39.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a6d9d640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f98a8141d40 0x7f98a8144130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:44370/0 (socket says 172.21.15.115:44370) 2024-01-24T23:23:39.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a6d9d640 1 -- 172.21.15.115:0/1614651425 learned_addr learned my addr 172.21.15.115:0/1614651425 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:39.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 1 -- 172.21.15.115:0/1614651425 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1007414241 0 0) 0x7f98a80a1cf0 con 0x7f98a8141d40 2024-01-24T23:23:39.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 10 monclient(hunting): my global_id is 5199 2024-01-24T23:23:39.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 1 -- 172.21.15.115:0/1614651425 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9894003530 con 0x7f98a8141d40 2024-01-24T23:23:39.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 1 -- 172.21.15.115:0/1614651425 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1663768628 0 0) 0x7f9894003530 con 0x7f98a8141d40 2024-01-24T23:23:39.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 1 -- 172.21.15.115:0/1614651425 >> v1:172.21.15.132:6792/0 conn(0x7f98a80a1540 legacy=0x7f98a8140620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 1 -- 172.21.15.115:0/1614651425 >> v1:172.21.15.115:6789/0 conn(0x7f98a80a0a50 legacy=0x7f98a80a0e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:39.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 10 monclient: _finish_hunting 0 2024-01-24T23:23:39.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 1 monclient: found mon.noname-d 2024-01-24T23:23:39.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:39.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:23:39.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 1 -- 172.21.15.115:0/1614651425 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a809af00 con 0x7f98a8141d40 2024-01-24T23:23:39.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 10 monclient: _finish_auth 0 2024-01-24T23:23:39.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 10 monclient: _check_auth_tickets 2024-01-24T23:23:39.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:39.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 5 monclient: authenticate success, global_id 5199 2024-01-24T23:23:39.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 1 -- 172.21.15.115:0/1614651425 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f98a0001b40 con 0x7f98a8141d40 2024-01-24T23:23:39.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:23:39.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 10 monclient: dump: 2024-01-24T23:23:39.729 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.729 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.729 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.730 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.730 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.730 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.730 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.730 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.730 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.730 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.730 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.730 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.730 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.730 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.730 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.730 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:39.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 1 -- 172.21.15.115:0/1614651425 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f98a0001df0 con 0x7f98a8141d40 2024-01-24T23:23:39.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:39.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 1 -- 172.21.15.115:0/1614651425 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f98a0004f70 con 0x7f98a8141d40 2024-01-24T23:23:39.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:39.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 4 set_mon_vals no callback set 2024-01-24T23:23:39.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:39.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98a559a640 10 monclient: dump: 2024-01-24T23:23:39.731 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.731 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.731 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.731 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.731 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.731 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.731 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.731 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.731 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.731 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.731 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.732 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.732 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.732 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.732 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.732 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 10 monclient: shutdown 2024-01-24T23:23:39.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:39.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 1 -- 172.21.15.115:0/1614651425 >> v1:172.21.15.115:6792/0 conn(0x7f98a8141d40 legacy=0x7f98a8144130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 1 -- 172.21.15.115:0/1614651425 shutdown_connections 2024-01-24T23:23:39.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 1 -- 172.21.15.115:0/1614651425 >> 172.21.15.115:0/1614651425 conn(0x7f98a80a6220 msgr2=0x7f98a80a6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:39.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 1 -- 172.21.15.115:0/1614651425 shutdown_connections 2024-01-24T23:23:39.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 1 -- 172.21.15.115:0/1614651425 wait complete. 2024-01-24T23:23:39.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.715+0000 7f98acc43640 10 monclient: build_initial_monmap 2024-01-24T23:23:39.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98acc43640 10 monclient: monmap: 2024-01-24T23:23:39.733 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:23:39.733 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:39.733 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:23:39.733 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:23:39.733 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:39.733 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:23:39.733 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:39.733 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:39.733 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:39.733 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:39.733 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:39.733 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:39.733 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:39.733 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:39.734 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:39.734 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98acc43640 1 Processor -- start 2024-01-24T23:23:39.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98acc43640 1 -- start start 2024-01-24T23:23:39.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98acc43640 10 monclient: init 2024-01-24T23:23:39.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98acc43640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:39.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98acc43640 10 monclient: _add_conns ranks=[5,4,3,2,1,8,0,7,6] 2024-01-24T23:23:39.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98acc43640 10 monclient(hunting): picked mon.noname-h con 0x7f98a80a0a50 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:39.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98acc43640 10 monclient(hunting): picked mon.noname-c con 0x7f98a80a1540 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:39.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98acc43640 10 monclient(hunting): picked mon.noname-g con 0x7f98a8141d40 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:39.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98acc43640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a80a24c0 con 0x7f98a8141d40 2024-01-24T23:23:39.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98acc43640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a811e650 con 0x7f98a80a1540 2024-01-24T23:23:39.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98acc43640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a811eb90 con 0x7f98a80a0a50 2024-01-24T23:23:39.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98acc43640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:39.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98a5d9b640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f98a80a1540 legacy=0x7f98a8139780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:23:39.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98acc43640 10 monclient(hunting): authenticate will time out at 1098.644934s 2024-01-24T23:23:39.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98a659c640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f98a80a0a50 0x7f98a811fdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.115:60584/0 (socket says 172.21.15.115:60584) 2024-01-24T23:23:39.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98a659c640 1 -- 172.21.15.115:0/1425373602 learned_addr learned my addr 172.21.15.115:0/1425373602 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:23:39.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1731457327 0 0) 0x7f98a811eb90 con 0x7f98a80a0a50 2024-01-24T23:23:39.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient(hunting): my global_id is 5108 2024-01-24T23:23:39.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9888003530 con 0x7f98a80a0a50 2024-01-24T23:23:39.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3208140943 0 0) 0x7f98a809af00 con 0x7f98a8141d40 2024-01-24T23:23:39.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient(hunting): my global_id is 5133 2024-01-24T23:23:39.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98a811eb90 con 0x7f98a8141d40 2024-01-24T23:23:39.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 125058584 0 0) 0x7f9888003530 con 0x7f98a80a0a50 2024-01-24T23:23:39.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f98a809af00 con 0x7f98a80a0a50 2024-01-24T23:23:39.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9898004300 con 0x7f98a80a0a50 2024-01-24T23:23:39.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:39.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient(hunting): dump: 2024-01-24T23:23:39.736 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.736 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.736 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.736 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.737 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.737 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.737 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.737 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.737 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.737 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.737 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.737 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.737 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.737 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.737 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.737 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:39.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 741234488 0 0) 0x7f98a811eb90 con 0x7f98a8141d40 2024-01-24T23:23:39.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9888003530 con 0x7f98a8141d40 2024-01-24T23:23:39.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f98a0001c20 con 0x7f98a8141d40 2024-01-24T23:23:39.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:39.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient(hunting): dump: 2024-01-24T23:23:39.738 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.738 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.738 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.738 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.738 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.738 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.738 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.738 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.738 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.738 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.738 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.738 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.738 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.738 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.738 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.739 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4294160659 0 0) 0x7f98a809af00 con 0x7f98a80a0a50 2024-01-24T23:23:39.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 >> v1:172.21.15.115:6791/0 conn(0x7f98a80a1540 legacy=0x7f98a8139780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:39.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 >> v1:172.21.15.132:6790/0 conn(0x7f98a8141d40 legacy=0x7f98a81d7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:39.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 1 monclient: found mon.h 2024-01-24T23:23:39.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:39.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:39.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a811f030 con 0x7f98a80a0a50 2024-01-24T23:23:39.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient: _finish_auth 0 2024-01-24T23:23:39.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:39.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:39.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98acc43640 5 monclient: authenticate success, global_id 5108 2024-01-24T23:23:39.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98acc43640 10 monclient: _renew_subs 2024-01-24T23:23:39.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98acc43640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:39.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98acc43640 1 -- 172.21.15.115:0/1425373602 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a81412c0 con 0x7f98a80a0a50 2024-01-24T23:23:39.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98acc43640 10 monclient: _renew_subs 2024-01-24T23:23:39.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98acc43640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:39.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98acc43640 1 -- 172.21.15.115:0/1425373602 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a813a060 con 0x7f98a80a0a50 2024-01-24T23:23:39.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f98980045e0 con 0x7f98a80a0a50 2024-01-24T23:23:39.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:39.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9898004b10 con 0x7f98a80a0a50 2024-01-24T23:23:39.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98a4d99640 4 set_mon_vals no callback set 2024-01-24T23:23:39.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:39.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient: dump: 2024-01-24T23:23:39.740 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.740 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.740 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.740 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.740 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.740 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.740 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.741 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.741 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.741 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.741 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.741 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.741 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.741 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.741 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.741 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98acc43640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:39.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98acc43640 1 -- 172.21.15.115:0/1425373602 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f988c0065d0 con 0x7f988c002870 2024-01-24T23:23:39.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f9898004cf0 con 0x7f98a80a0a50 2024-01-24T23:23:39.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(53..53 src has 1..53) v4 ==== 4310+0+0 (unknown 3589091670 0 0) 0x7f9898043b00 con 0x7f98a80a0a50 2024-01-24T23:23:39.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2162197349 0 0) 0x7f988c0065d0 con 0x7f988c002870 2024-01-24T23:23:39.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 10 monclient(hunting): my global_id is 5209 2024-01-24T23:23:39.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98a809af00 con 0x7f988c002870 2024-01-24T23:23:39.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f98a6d9d640 10 monclient: get_auth_request con 0x7f98880375f0 auth_method 0 2024-01-24T23:23:39.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2514693251 0 0) 0x7f98a809af00 con 0x7f988c002870 2024-01-24T23:23:39.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.719+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f988c0065d0 con 0x7f988c002870 2024-01-24T23:23:39.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.723+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f98900018f0 con 0x7f988c002870 2024-01-24T23:23:39.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.723+0000 7f989effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.723+0000 7f989effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:39.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.723+0000 7f989effd640 10 monclient: dump: 2024-01-24T23:23:39.742 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.742 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.742 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.742 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.742 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.742 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.743 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.743 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.743 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.743 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.743 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.743 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.743 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.743 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.743 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.743 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.723+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2049541672 0 0) 0x7f988c0065d0 con 0x7f988c002870 2024-01-24T23:23:39.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.723+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 --> v1:172.21.15.132:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f988c00a120 con 0x7f988c002870 2024-01-24T23:23:39.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.723+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 <== mon.7 v1:172.21.15.132:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f98900047a0 con 0x7f988c002870 2024-01-24T23:23:39.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.723+0000 7f989effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:39.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.723+0000 7f989effd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:39.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.723+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 >> v1:172.21.15.132:6792/0 conn(0x7f988c002870 legacy=0x7f988c004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.723+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 >> v1:172.21.15.132:6792/0 conn(0x7f988c002870 legacy=0x7f988c004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:39.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.727+0000 7f98acc43640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:23:39.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.727+0000 7f98acc43640 1 -- 172.21.15.115:0/1425373602 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f988c009e10 con 0x7f988c0090d0 2024-01-24T23:23:39.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.727+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1206146153 0 0) 0x7f988c009e10 con 0x7f988c0090d0 2024-01-24T23:23:39.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.727+0000 7f989effd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.727+0000 7f989effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.727+0000 7f989effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.727+0000 7f989effd640 10 monclient(hunting): my global_id is 5218 2024-01-24T23:23:39.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.727+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f988c0065d0 con 0x7f988c0090d0 2024-01-24T23:23:39.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.731+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 612569538 0 0) 0x7f988c0065d0 con 0x7f988c0090d0 2024-01-24T23:23:39.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.731+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f988c009e10 con 0x7f988c0090d0 2024-01-24T23:23:39.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.731+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9890004f10 con 0x7f988c0090d0 2024-01-24T23:23:39.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.731+0000 7f989effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.731+0000 7f989effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:39.746 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.731+0000 7f989effd640 10 monclient: dump: 2024-01-24T23:23:39.746 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:23:39.747 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.747 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.747 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.747 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.747 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:23:39.747 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.747 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.747 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.747 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.747 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.747 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.747 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.747 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.748 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.748 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:23:39.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.731+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3180796337 0 0) 0x7f988c009e10 con 0x7f988c0090d0 2024-01-24T23:23:39.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.731+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 --> v1:172.21.15.132:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f988c00c310 con 0x7f988c0090d0 2024-01-24T23:23:39.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.731+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 <== mon.7 v1:172.21.15.132:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7680 (unknown 0 0 3669771896) 0x7f9890007220 con 0x7f988c0090d0 2024-01-24T23:23:39.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.731+0000 7f989effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:23:39.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.731+0000 7f989effd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:39.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.731+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 >> v1:172.21.15.132:6792/0 conn(0x7f988c0090d0 legacy=0x7f988c009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.731+0000 7f989effd640 1 -- 172.21.15.115:0/1425373602 >> v1:172.21.15.132:6792/0 conn(0x7f988c0090d0 legacy=0x7f988c009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:23:39.748 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:23:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:23:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 20, 2024-01-24T23:23:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:23:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:23:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:23:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:23:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:23:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:23:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:23:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 1, 2024-01-24T23:23:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:23:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:23:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:23:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:23:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:23:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:23:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:23:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:23:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:23:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:23:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:23:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:23:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:23:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:23:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:23:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:23:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:23:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:23:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:23:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:23:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:23:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:23:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:23:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:23:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:23:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:23:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:23:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:23:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:23:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:23:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:23:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:23:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:23:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:23:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:23:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:23:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:23:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:23:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:23:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:23:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:23:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:23:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:23:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:23:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:23:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:23:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:23:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:23:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:23:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:23:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.759 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.759 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.759 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:23:39.759 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.759 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.759 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.759 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:23:39.759 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:23:39.759 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.759 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.759 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:23:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:23:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:23:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:23:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:23:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:23:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:23:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:23:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:23:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:23:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:23:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:23:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:23:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:23:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:23:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:23:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:23:39.764 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.764 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.764 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.764 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:23:39.764 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.764 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.764 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.764 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:23:39.764 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:23:39.764 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.764 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.765 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.765 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:23:39.765 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:23:39.765 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:23:39.765 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:23:39.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.765 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:23:39.765 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:23:39.765 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:23:39.765 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:23:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:23:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:23:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:23:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:23:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:23:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:23:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:39.767 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:39.767 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:23:39.767 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.767 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:39.767 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:23:39.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.767 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:39.767 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:39.767 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 3 2024-01-24T23:23:39.767 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.767 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:23:39.767 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:23:39.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:23:39.768 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:23:39.768 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:23:39.768 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 3 2024-01-24T23:23:39.768 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:23:39.768 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:23:39.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:23:39.768 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:23:39.768 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:23:39.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.731+0000 7f98acc43640 1 -- 172.21.15.115:0/1425373602 >> v1:172.21.15.132:6812/1835401922 conn(0x7f98880375f0 legacy=0x7f9888039ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.731+0000 7f98acc43640 10 monclient: shutdown 2024-01-24T23:23:39.769 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.731+0000 7f98acc43640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:39.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.731+0000 7f98acc43640 1 -- 172.21.15.115:0/1425373602 >> v1:172.21.15.132:6791/0 conn(0x7f98a80a0a50 legacy=0x7f98a811fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.731+0000 7f98a6d9d640 1 -- 172.21.15.115:0/1425373602 reap_dead start 2024-01-24T23:23:39.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.731+0000 7f98a6d9d640 1 -- 172.21.15.115:0/1425373602 reap_dead start 2024-01-24T23:23:39.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.731+0000 7f98acc43640 1 -- 172.21.15.115:0/1425373602 shutdown_connections 2024-01-24T23:23:39.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.731+0000 7f98acc43640 1 -- 172.21.15.115:0/1425373602 >> 172.21.15.115:0/1425373602 conn(0x7f98a80a6220 msgr2=0x7f98a80a5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:39.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.731+0000 7f98acc43640 1 -- 172.21.15.115:0/1425373602 shutdown_connections 2024-01-24T23:23:39.770 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:23:39.731+0000 7f98acc43640 1 -- 172.21.15.115:0/1425373602 wait complete. 2024-01-24T23:23:39.770 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-01-24T23:23:39.770 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-01-24T23:23:39.805 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.803+0000 7f1dd08be640 10 monclient: get_monmap_and_config 2024-01-24T23:23:39.805 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.803+0000 7f1dd08be640 10 monclient: build_initial_monmap 2024-01-24T23:23:39.805 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.803+0000 7f1dd08be640 10 monclient: monmap: 2024-01-24T23:23:39.805 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:39.805 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.805 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:39.806530+0000 2024-01-24T23:23:39.805 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:39.806530+0000 2024-01-24T23:23:39.806 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:39.806 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:39.806 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:39.806 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:39.806 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:39.806 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:39.806 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:39.806 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:39.806 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:39.807 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:39.807 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:39.807 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:39.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.803+0000 7f1dd08be640 1 Processor -- start 2024-01-24T23:23:39.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.803+0000 7f1dd08be640 1 -- start start 2024-01-24T23:23:39.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.803+0000 7f1dd08be640 10 monclient: init 2024-01-24T23:23:39.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.803+0000 7f1dd08be640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:39.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 10 monclient: _add_conns ranks=[7,2,3,5,0,4,1,8,6] 2024-01-24T23:23:39.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 10 monclient(hunting): picked mon.noname-i con 0x7f1dcc0a1c60 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:39.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 10 monclient(hunting): picked mon.noname-b con 0x7f1dcc0a2040 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:39.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 10 monclient(hunting): picked mon.noname-g con 0x7f1dcc139db0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:39.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dcc064700 con 0x7f1dcc0a2040 2024-01-24T23:23:39.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dcc09ba30 con 0x7f1dcc139db0 2024-01-24T23:23:39.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dcc09bf30 con 0x7f1dcc0a1c60 2024-01-24T23:23:39.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:39.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 10 monclient(hunting): authenticate will time out at 1107.132124s 2024-01-24T23:23:39.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dcad9d640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f1dcc139db0 0x7f1dcc13c1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:60410/0 (socket says 172.21.15.132:60410) 2024-01-24T23:23:39.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dcad9d640 1 -- 172.21.15.132:0/2593684630 learned_addr learned my addr 172.21.15.132:0/2593684630 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:39.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dca59c640 1 --1- 172.21.15.132:0/2593684630 >> v1:172.21.15.132:6792/0 conn(0x7f1dcc0a1c60 0x7f1dcc09cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:41610/0 (socket says 172.21.15.132:41610) 2024-01-24T23:23:39.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 1 -- 172.21.15.132:0/2593684630 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 234737075 0 0) 0x7f1dcc09ba30 con 0x7f1dcc139db0 2024-01-24T23:23:39.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 10 monclient(hunting): my global_id is 5142 2024-01-24T23:23:39.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 1 -- 172.21.15.132:0/2593684630 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1db8003530 con 0x7f1dcc139db0 2024-01-24T23:23:39.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 1 -- 172.21.15.132:0/2593684630 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3452703371 0 0) 0x7f1dcc09bf30 con 0x7f1dcc0a1c60 2024-01-24T23:23:39.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 10 monclient(hunting): my global_id is 5227 2024-01-24T23:23:39.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 1 -- 172.21.15.132:0/2593684630 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dcc09ba30 con 0x7f1dcc0a1c60 2024-01-24T23:23:39.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 1 -- 172.21.15.132:0/2593684630 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1216468577 0 0) 0x7f1db8003530 con 0x7f1dcc139db0 2024-01-24T23:23:39.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 1 -- 172.21.15.132:0/2593684630 >> v1:172.21.15.132:6792/0 conn(0x7f1dcc0a1c60 legacy=0x7f1dcc09cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 1 -- 172.21.15.132:0/2593684630 >> v1:172.21.15.115:6790/0 conn(0x7f1dcc0a2040 legacy=0x7f1dcc0a24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 10 monclient: _finish_hunting 0 2024-01-24T23:23:39.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 1 monclient: found mon.noname-g 2024-01-24T23:23:39.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:39.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:23:39.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 1 -- 172.21.15.132:0/2593684630 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dcc09c210 con 0x7f1dcc139db0 2024-01-24T23:23:39.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 10 monclient: _finish_auth 0 2024-01-24T23:23:39.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 10 monclient: _check_auth_tickets 2024-01-24T23:23:39.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:39.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 5 monclient: authenticate success, global_id 5142 2024-01-24T23:23:39.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 1 -- 172.21.15.132:0/2593684630 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1dc4001b60 con 0x7f1dcc139db0 2024-01-24T23:23:39.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:39.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:39.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 10 monclient: dump: 2024-01-24T23:23:39.819 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:39.819 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.820 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.820 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.820 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.820 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:39.820 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.820 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.820 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.820 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.820 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.820 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.820 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.820 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.820 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.821 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:39.821 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:39.821 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 1 -- 172.21.15.132:0/2593684630 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1dc4001d80 con 0x7f1dcc139db0 2024-01-24T23:23:39.821 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:39.821 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 1 -- 172.21.15.132:0/2593684630 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1dc4004f70 con 0x7f1dcc139db0 2024-01-24T23:23:39.821 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.821 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:39.821 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc9d9b640 10 monclient: dump: 2024-01-24T23:23:39.821 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:39.821 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.821 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.821 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.821 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.822 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:39.822 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.822 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.822 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.822 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.822 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.822 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.822 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.822 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.822 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.822 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:39.822 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:39.822 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 4 set_mon_vals no callback set 2024-01-24T23:23:39.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 10 monclient: shutdown 2024-01-24T23:23:39.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:39.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 1 -- 172.21.15.132:0/2593684630 >> v1:172.21.15.132:6790/0 conn(0x7f1dcc139db0 legacy=0x7f1dcc13c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 1 -- 172.21.15.132:0/2593684630 shutdown_connections 2024-01-24T23:23:39.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 1 -- 172.21.15.132:0/2593684630 >> 172.21.15.132:0/2593684630 conn(0x7f1dcc0a2d50 msgr2=0x7f1dcc0a3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:39.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 1 -- 172.21.15.132:0/2593684630 shutdown_connections 2024-01-24T23:23:39.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 1 -- 172.21.15.132:0/2593684630 wait complete. 2024-01-24T23:23:39.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 10 monclient: build_initial_monmap 2024-01-24T23:23:39.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 10 monclient: monmap: 2024-01-24T23:23:39.823 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:39.823 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:39.823 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:39.823 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:39.824 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:39.824 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:39.824 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:39.824 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:39.824 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:39.824 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:39.824 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:39.824 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:39.824 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:39.824 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:39.824 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:39.825 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:39.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 1 Processor -- start 2024-01-24T23:23:39.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 1 -- start start 2024-01-24T23:23:39.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 10 monclient: init 2024-01-24T23:23:39.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:39.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 10 monclient: _add_conns ranks=[7,2,4,3,0,8,5,6,1] 2024-01-24T23:23:39.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 10 monclient(hunting): picked mon.noname-i con 0x7f1dcc0a1c60 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:39.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 10 monclient(hunting): picked mon.noname-b con 0x7f1dcc0a2040 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:39.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 10 monclient(hunting): picked mon.noname-c con 0x7f1dcc139db0 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:39.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dcc071180 con 0x7f1dcc0a2040 2024-01-24T23:23:39.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dcc06f670 con 0x7f1dcc139db0 2024-01-24T23:23:39.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dcc06ff40 con 0x7f1dcc0a1c60 2024-01-24T23:23:39.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:39.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dd08be640 10 monclient(hunting): authenticate will time out at 1107.135360s 2024-01-24T23:23:39.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dca59c640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f1dcc0a1c60 0x7f1dcc06e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:41626/0 (socket says 172.21.15.132:41626) 2024-01-24T23:23:39.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dca59c640 1 -- 172.21.15.132:0/1818705757 learned_addr learned my addr 172.21.15.132:0/1818705757 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:39.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dcad9d640 1 -- 172.21.15.132:0/1818705757 >> v1:172.21.15.115:6791/0 conn(0x7f1dcc139db0 legacy=0x7f1dcc1d18f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:23:39.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc37fe640 1 -- 172.21.15.132:0/1818705757 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2552391280 0 0) 0x7f1dcc06ff40 con 0x7f1dcc0a1c60 2024-01-24T23:23:39.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc37fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc37fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc37fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc37fe640 10 monclient(hunting): my global_id is 5236 2024-01-24T23:23:39.827 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc37fe640 1 -- 172.21.15.132:0/1818705757 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dac003530 con 0x7f1dcc0a1c60 2024-01-24T23:23:39.827 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc37fe640 1 -- 172.21.15.132:0/1818705757 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767780397 0 0) 0x7f1dcc071180 con 0x7f1dcc0a2040 2024-01-24T23:23:39.827 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc37fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:39.827 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc37fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:39.827 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc37fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:39.827 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc37fe640 10 monclient(hunting): my global_id is 84413 2024-01-24T23:23:39.827 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc37fe640 1 -- 172.21.15.132:0/1818705757 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dcc06ff40 con 0x7f1dcc0a2040 2024-01-24T23:23:39.827 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc37fe640 1 -- 172.21.15.132:0/1818705757 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2163842117 0 0) 0x7f1dac003530 con 0x7f1dcc0a1c60 2024-01-24T23:23:39.827 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc37fe640 1 -- 172.21.15.132:0/1818705757 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1dcc071180 con 0x7f1dcc0a1c60 2024-01-24T23:23:39.827 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc37fe640 1 -- 172.21.15.132:0/1818705757 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1dbc001970 con 0x7f1dcc0a1c60 2024-01-24T23:23:39.827 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc37fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.827 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc37fe640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:23:39.828 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc37fe640 10 monclient(hunting): dump: 2024-01-24T23:23:39.828 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:39.828 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.828 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.828 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.828 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.828 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:39.828 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.828 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.828 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.828 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.828 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.829 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.829 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.829 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.829 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.829 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:39.829 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.807+0000 7f1dc37fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:39.829 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 1 -- 172.21.15.132:0/1818705757 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 593545515 0 0) 0x7f1dcc06ff40 con 0x7f1dcc0a2040 2024-01-24T23:23:39.829 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 1 -- 172.21.15.132:0/1818705757 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1dac003530 con 0x7f1dcc0a2040 2024-01-24T23:23:39.829 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 1 -- 172.21.15.132:0/1818705757 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1db40019e0 con 0x7f1dcc0a2040 2024-01-24T23:23:39.829 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.829 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:39.829 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 10 monclient(hunting): dump: 2024-01-24T23:23:39.829 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:39.829 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.830 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.830 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.830 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.830 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:39.830 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.830 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.830 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.830 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.830 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.830 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.830 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.830 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.830 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.830 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:39.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 1 -- 172.21.15.132:0/1818705757 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1231877700 0 0) 0x7f1dcc071180 con 0x7f1dcc0a1c60 2024-01-24T23:23:39.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 1 -- 172.21.15.132:0/1818705757 >> v1:172.21.15.115:6791/0 conn(0x7f1dcc139db0 legacy=0x7f1dcc1d18f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:39.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 1 -- 172.21.15.132:0/1818705757 >> v1:172.21.15.115:6790/0 conn(0x7f1dcc0a2040 legacy=0x7f1dcc06ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:39.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 10 monclient: _finish_hunting 0 2024-01-24T23:23:39.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 1 monclient: found mon.i 2024-01-24T23:23:39.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:39.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:39.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 1 -- 172.21.15.132:0/1818705757 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dcc0a0370 con 0x7f1dcc0a1c60 2024-01-24T23:23:39.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 10 monclient: _finish_auth 0 2024-01-24T23:23:39.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 10 monclient: _check_auth_tickets 2024-01-24T23:23:39.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:39.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dd08be640 5 monclient: authenticate success, global_id 5236 2024-01-24T23:23:39.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dd08be640 10 monclient: _renew_subs 2024-01-24T23:23:39.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dd08be640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:39.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dd08be640 1 -- 172.21.15.132:0/1818705757 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dcc070ea0 con 0x7f1dcc0a1c60 2024-01-24T23:23:39.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dd08be640 10 monclient: _renew_subs 2024-01-24T23:23:39.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dd08be640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:39.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dd08be640 1 -- 172.21.15.132:0/1818705757 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dcc09faf0 con 0x7f1dcc0a1c60 2024-01-24T23:23:39.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 1 -- 172.21.15.132:0/1818705757 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1dbc001d40 con 0x7f1dcc0a1c60 2024-01-24T23:23:39.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:39.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 1 -- 172.21.15.132:0/1818705757 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1dbc004de0 con 0x7f1dcc0a1c60 2024-01-24T23:23:39.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:39.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc959a640 4 set_mon_vals no callback set 2024-01-24T23:23:39.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:39.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 10 monclient: dump: 2024-01-24T23:23:39.833 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:39.833 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:39.833 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.833 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:39.833 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:39.833 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:39.833 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:39.833 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:39.833 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:39.833 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:39.833 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:39.833 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:39.834 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:39.834 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:39.834 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:39.834 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:39.834 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dd08be640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:39.834 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dd08be640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:39.834 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dd08be640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:39.834 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dd08be640 1 -- 172.21.15.132:0/1818705757 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db0005090 con 0x7f1dcc0a1c60 2024-01-24T23:23:39.834 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 1 -- 172.21.15.132:0/1818705757 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f1dbc00f860 con 0x7f1dcc0a1c60 2024-01-24T23:23:39.834 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc37fe640 1 -- 172.21.15.132:0/1818705757 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(53..53 src has 1..53) v4 ==== 4310+0+0 (unknown 3589091670 0 0) 0x7f1dbc043970 con 0x7f1dcc0a1c60 2024-01-24T23:23:39.834 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.811+0000 7f1dc1d9b640 10 monclient: get_auth_request con 0x7f1dac037740 auth_method 0 2024-01-24T23:23:39.835 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.815+0000 7f1dc37fe640 1 -- 172.21.15.132:0/1818705757 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f1dbc014790 con 0x7f1dcc0a1c60 2024-01-24T23:23:39.835 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.815+0000 7f1dc37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:39.835 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.815+0000 7f1dc37fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:39.976 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.975+0000 7f1dd08be640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-01-24T23:23:39.976 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.975+0000 7f1dd08be640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-01-24T23:23:39.976 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.975+0000 7f1dd08be640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:39.976 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.975+0000 7f1dd08be640 1 -- 172.21.15.132:0/1818705757 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"} v 0) v1 -- 0x7f1db00058c0 con 0x7f1dcc0a1c60 2024-01-24T23:23:39.977 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.975+0000 7f1dc37fe640 1 -- 172.21.15.132:0/1818705757 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}]=0 already renamed to 'foo-asdf' v53) v1 ==== 139+0+0 (unknown 627309034 0 0) 0x7f1dbc012e30 con 0x7f1dcc0a1c60 2024-01-24T23:23:39.977 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.975+0000 7f1dc37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-01-24T23:23:39.977 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:39.975+0000 7f1dc37fe640 10 monclient: _finish_command 2 = system:0 already renamed to 'foo-asdf' 2024-01-24T23:23:40.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.075+0000 7f1dd08be640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-01-24T23:23:40.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.075+0000 7f1dd08be640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-01-24T23:23:40.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.075+0000 7f1dd08be640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:40.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.075+0000 7f1dd08be640 1 -- 172.21.15.132:0/1818705757 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"} v 0) v1 -- 0x7f1db00010a0 con 0x7f1dcc0a1c60 2024-01-24T23:23:40.077 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.075+0000 7f1dc37fe640 1 -- 172.21.15.132:0/1818705757 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}]=0 already renamed to 'foo-asdf' v53) v1 ==== 139+0+0 (unknown 627309034 0 0) 0x7f1dbc012e30 con 0x7f1dcc0a1c60 2024-01-24T23:23:40.077 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.075+0000 7f1dc37fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-01-24T23:23:40.077 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.075+0000 7f1dc37fe640 10 monclient: _finish_command 3 = system:0 already renamed to 'foo-asdf' 2024-01-24T23:23:40.077 INFO:tasks.workunit.client.0.smithi132.stderr:already renamed to 'foo-asdf' 2024-01-24T23:23:40.079 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.075+0000 7f1dd08be640 1 -- 172.21.15.132:0/1818705757 >> v1:172.21.15.132:6812/1835401922 conn(0x7f1dac037740 legacy=0x7f1dac039c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:40.079 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.075+0000 7f1dd08be640 10 monclient: shutdown 2024-01-24T23:23:40.079 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.075+0000 7f1dd08be640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:40.079 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.075+0000 7f1dd08be640 1 -- 172.21.15.132:0/1818705757 >> v1:172.21.15.132:6792/0 conn(0x7f1dcc0a1c60 legacy=0x7f1dcc06e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:40.079 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.079+0000 7f1dd08be640 1 -- 172.21.15.132:0/1818705757 shutdown_connections 2024-01-24T23:23:40.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.079+0000 7f1dd08be640 1 -- 172.21.15.132:0/1818705757 >> 172.21.15.132:0/1818705757 conn(0x7f1dcc0a2d50 msgr2=0x7f1dcc132e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:40.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.079+0000 7f1dd08be640 1 -- 172.21.15.132:0/1818705757 shutdown_connections 2024-01-24T23:23:40.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.079+0000 7f1dd08be640 1 -- 172.21.15.132:0/1818705757 wait complete. 2024-01-24T23:23:40.094 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule rename bar bar-asdf 2024-01-24T23:23:40.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.211+0000 7f94474bc640 10 monclient: get_monmap_and_config 2024-01-24T23:23:40.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.211+0000 7f94474bc640 10 monclient: build_initial_monmap 2024-01-24T23:23:40.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.211+0000 7f94474bc640 10 monclient: monmap: 2024-01-24T23:23:40.213 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:40.213 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:40.213 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:40.214420+0000 2024-01-24T23:23:40.213 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:40.214420+0000 2024-01-24T23:23:40.213 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:40.213 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:40.213 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:40.214 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:40.214 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:40.214 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:40.214 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:40.214 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:40.214 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:40.214 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:40.214 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:40.214 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:40.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.211+0000 7f94474bc640 1 Processor -- start 2024-01-24T23:23:40.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.211+0000 7f94474bc640 1 -- start start 2024-01-24T23:23:40.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.211+0000 7f94474bc640 10 monclient: init 2024-01-24T23:23:40.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.211+0000 7f94474bc640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:40.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.211+0000 7f94474bc640 10 monclient: _add_conns ranks=[4,6,8,0,1,3,2,5,7] 2024-01-24T23:23:40.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.211+0000 7f94474bc640 10 monclient(hunting): picked mon.noname-c con 0x7f9440135ea0 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:40.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.211+0000 7f94474bc640 10 monclient(hunting): picked mon.noname-d con 0x7f944009bcd0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:40.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.211+0000 7f94474bc640 10 monclient(hunting): picked mon.noname-e con 0x7f944009c0f0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:40.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.211+0000 7f94474bc640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9440064700 con 0x7f9440135ea0 2024-01-24T23:23:40.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.211+0000 7f94474bc640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9440136990 con 0x7f944009bcd0 2024-01-24T23:23:40.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.211+0000 7f94474bc640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f944009b010 con 0x7f944009c0f0 2024-01-24T23:23:40.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.211+0000 7f94474bc640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:40.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.211+0000 7f94474bc640 10 monclient(hunting): authenticate will time out at 1107.539672s 2024-01-24T23:23:40.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.211+0000 7f9445258640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f9440135ea0 legacy=0x7f9440136280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:23:40.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9444a57640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f944009bcd0 0x7f944013bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:48886/0 (socket says 172.21.15.132:48886) 2024-01-24T23:23:40.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9445a59640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f944009c0f0 0x7f944013f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:37920/0 (socket says 172.21.15.132:37920) 2024-01-24T23:23:40.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9444a57640 1 -- 172.21.15.132:0/1186759695 learned_addr learned my addr 172.21.15.132:0/1186759695 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:40.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 1 -- 172.21.15.132:0/1186759695 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099312096 0 0) 0x7f9440136990 con 0x7f944009bcd0 2024-01-24T23:23:40.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:40.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:40.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:40.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 10 monclient(hunting): my global_id is 5208 2024-01-24T23:23:40.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 1 -- 172.21.15.132:0/1186759695 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f942c003530 con 0x7f944009bcd0 2024-01-24T23:23:40.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 1 -- 172.21.15.132:0/1186759695 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3481232935 0 0) 0x7f944009b010 con 0x7f944009c0f0 2024-01-24T23:23:40.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:40.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:40.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:40.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 10 monclient(hunting): my global_id is 84356 2024-01-24T23:23:40.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 1 -- 172.21.15.132:0/1186759695 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9440136990 con 0x7f944009c0f0 2024-01-24T23:23:40.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 1 -- 172.21.15.132:0/1186759695 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2820255366 0 0) 0x7f9440136990 con 0x7f944009c0f0 2024-01-24T23:23:40.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 1 -- 172.21.15.132:0/1186759695 >> v1:172.21.15.115:6792/0 conn(0x7f944009bcd0 legacy=0x7f944013bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:40.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 1 -- 172.21.15.132:0/1186759695 >> v1:172.21.15.115:6791/0 conn(0x7f9440135ea0 legacy=0x7f9440136280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:40.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 10 monclient: _finish_hunting 0 2024-01-24T23:23:40.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 1 monclient: found mon.noname-e 2024-01-24T23:23:40.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:40.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:23:40.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 1 -- 172.21.15.132:0/1186759695 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94400a4f50 con 0x7f944009c0f0 2024-01-24T23:23:40.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 10 monclient: _finish_auth 0 2024-01-24T23:23:40.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 10 monclient: _check_auth_tickets 2024-01-24T23:23:40.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:40.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 1 -- 172.21.15.132:0/1186759695 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9438001ae0 con 0x7f944009c0f0 2024-01-24T23:23:40.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:40.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:23:40.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 10 monclient: dump: 2024-01-24T23:23:40.220 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:40.220 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:40.220 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:40.220 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:40.220 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:40.220 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:40.220 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:40.220 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:40.220 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:40.220 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:40.220 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:40.220 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:40.221 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:40.221 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:40.221 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:40.221 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:40.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 5 monclient: authenticate success, global_id 84356 2024-01-24T23:23:40.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:40.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 1 -- 172.21.15.132:0/1186759695 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9438001d00 con 0x7f944009c0f0 2024-01-24T23:23:40.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:40.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 1 -- 172.21.15.132:0/1186759695 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9438004b70 con 0x7f944009c0f0 2024-01-24T23:23:40.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:40.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 4 set_mon_vals no callback set 2024-01-24T23:23:40.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:40.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:40.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9437fff640 10 monclient: dump: 2024-01-24T23:23:40.222 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:40.222 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:40.222 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:40.222 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:40.222 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:40.222 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:40.222 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:40.222 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:40.222 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:40.222 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:40.222 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:40.222 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:40.222 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:40.222 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:40.223 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:40.223 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:40.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 10 monclient: shutdown 2024-01-24T23:23:40.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:40.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 1 -- 172.21.15.132:0/1186759695 >> v1:172.21.15.115:6793/0 conn(0x7f944009c0f0 legacy=0x7f944013f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:40.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 1 -- 172.21.15.132:0/1186759695 shutdown_connections 2024-01-24T23:23:40.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 1 -- 172.21.15.132:0/1186759695 >> 172.21.15.132:0/1186759695 conn(0x7f94400a9bf0 msgr2=0x7f94400a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:40.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 1 -- 172.21.15.132:0/1186759695 shutdown_connections 2024-01-24T23:23:40.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 1 -- 172.21.15.132:0/1186759695 wait complete. 2024-01-24T23:23:40.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 10 monclient: build_initial_monmap 2024-01-24T23:23:40.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 10 monclient: monmap: 2024-01-24T23:23:40.228 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:40.228 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:40.228 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:40.228 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:40.228 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:40.228 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:40.228 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:40.228 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:40.228 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:40.228 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:40.228 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:40.229 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:40.229 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:40.229 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:40.229 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:40.229 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:40.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 1 Processor -- start 2024-01-24T23:23:40.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 1 -- start start 2024-01-24T23:23:40.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 10 monclient: init 2024-01-24T23:23:40.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:40.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 10 monclient: _add_conns ranks=[2,7,4,1,0,3,8,6,5] 2024-01-24T23:23:40.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 10 monclient(hunting): picked mon.noname-b con 0x7f944009bcd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:40.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 10 monclient(hunting): picked mon.noname-i con 0x7f944009c0f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:40.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 10 monclient(hunting): picked mon.noname-c con 0x7f9440135ea0 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:40.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f944009c6b0 con 0x7f944009bcd0 2024-01-24T23:23:40.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94401244c0 con 0x7f9440135ea0 2024-01-24T23:23:40.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9440135c70 con 0x7f944009c0f0 2024-01-24T23:23:40.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:40.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f94474bc640 10 monclient(hunting): authenticate will time out at 1107.543722s 2024-01-24T23:23:40.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9444a57640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f944009c0f0 0x7f94401d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:41628/0 (socket says 172.21.15.132:41628) 2024-01-24T23:23:40.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9444a57640 1 -- 172.21.15.132:0/2474977361 learned_addr learned my addr 172.21.15.132:0/2474977361 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:40.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.215+0000 7f9445a59640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f9440135ea0 legacy=0x7f94401d3a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:23:40.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 1 -- 172.21.15.132:0/2474977361 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1818712398 0 0) 0x7f9440135c70 con 0x7f944009c0f0 2024-01-24T23:23:40.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:40.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:40.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:40.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 10 monclient(hunting): my global_id is 5245 2024-01-24T23:23:40.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 1 -- 172.21.15.132:0/2474977361 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9424003530 con 0x7f944009c0f0 2024-01-24T23:23:40.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 1 -- 172.21.15.132:0/2474977361 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 27671270 0 0) 0x7f9424003530 con 0x7f944009c0f0 2024-01-24T23:23:40.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 1 -- 172.21.15.132:0/2474977361 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9440135c70 con 0x7f944009c0f0 2024-01-24T23:23:40.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 1 -- 172.21.15.132:0/2474977361 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9430001560 con 0x7f944009c0f0 2024-01-24T23:23:40.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:40.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:23:40.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 10 monclient(hunting): dump: 2024-01-24T23:23:40.231 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:40.231 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:40.231 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:40.231 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:40.231 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:40.231 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:40.232 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:40.232 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:40.232 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:40.232 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:40.232 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:40.232 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:40.232 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:40.232 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:40.232 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:40.232 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:40.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:40.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 1 -- 172.21.15.132:0/2474977361 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1718700632 0 0) 0x7f944009c6b0 con 0x7f944009bcd0 2024-01-24T23:23:40.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:40.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:40.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:40.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 10 monclient(hunting): my global_id is 84422 2024-01-24T23:23:40.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 1 -- 172.21.15.132:0/2474977361 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9424003530 con 0x7f944009bcd0 2024-01-24T23:23:40.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 1 -- 172.21.15.132:0/2474977361 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3988302363 0 0) 0x7f9440135c70 con 0x7f944009c0f0 2024-01-24T23:23:40.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 1 -- 172.21.15.132:0/2474977361 >> v1:172.21.15.115:6791/0 conn(0x7f9440135ea0 legacy=0x7f94401d3a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:40.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 1 -- 172.21.15.132:0/2474977361 >> v1:172.21.15.115:6790/0 conn(0x7f944009bcd0 legacy=0x7f94401cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:40.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:23:40.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 1 monclient: found mon.i 2024-01-24T23:23:40.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:40.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:40.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 1 -- 172.21.15.132:0/2474977361 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9440126030 con 0x7f944009c0f0 2024-01-24T23:23:40.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 10 monclient: _finish_auth 0 2024-01-24T23:23:40.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:23:40.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:40.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f94474bc640 5 monclient: authenticate success, global_id 5245 2024-01-24T23:23:40.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f94474bc640 10 monclient: _renew_subs 2024-01-24T23:23:40.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f94474bc640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:40.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f94474bc640 1 -- 172.21.15.132:0/2474977361 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9440126310 con 0x7f944009c0f0 2024-01-24T23:23:40.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f94474bc640 10 monclient: _renew_subs 2024-01-24T23:23:40.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f94474bc640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:40.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f94474bc640 1 -- 172.21.15.132:0/2474977361 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f94401cbf20 con 0x7f944009c0f0 2024-01-24T23:23:40.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 1 -- 172.21.15.132:0/2474977361 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9430004aa0 con 0x7f944009c0f0 2024-01-24T23:23:40.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:40.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 1 -- 172.21.15.132:0/2474977361 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9430004f50 con 0x7f944009c0f0 2024-01-24T23:23:40.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:40.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f94377fe640 4 set_mon_vals no callback set 2024-01-24T23:23:40.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:40.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 10 monclient: dump: 2024-01-24T23:23:40.235 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:40.235 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:40.235 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:40.235 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:40.235 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:40.235 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:40.235 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:40.235 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:40.235 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:40.235 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:40.235 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:40.235 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:40.235 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:40.235 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:40.235 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:40.236 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:40.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 1 -- 172.21.15.132:0/2474977361 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f943000fa50 con 0x7f944009c0f0 2024-01-24T23:23:40.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f94474bc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:40.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f94474bc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:40.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f94474bc640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:40.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f94474bc640 1 -- 172.21.15.132:0/2474977361 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9420005090 con 0x7f944009c0f0 2024-01-24T23:23:40.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9435ffb640 1 -- 172.21.15.132:0/2474977361 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(53..53 src has 1..53) v4 ==== 4310+0+0 (unknown 3589091670 0 0) 0x7f9430043be0 con 0x7f944009c0f0 2024-01-24T23:23:40.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.219+0000 7f9445258640 10 monclient: get_auth_request con 0x7f9424036f70 auth_method 0 2024-01-24T23:23:40.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.223+0000 7f9435ffb640 1 -- 172.21.15.132:0/2474977361 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f9430043e60 con 0x7f944009c0f0 2024-01-24T23:23:40.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.223+0000 7f9435ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:40.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.223+0000 7f9435ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:40.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.383+0000 7f94474bc640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}] 2024-01-24T23:23:40.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.383+0000 7f94474bc640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}] 2024-01-24T23:23:40.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.383+0000 7f94474bc640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:40.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.383+0000 7f94474bc640 1 -- 172.21.15.132:0/2474977361 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"} v 0) v1 -- 0x7f94200058c0 con 0x7f944009c0f0 2024-01-24T23:23:40.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.587+0000 7f9435ffb640 1 -- 172.21.15.132:0/2474977361 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}]=0 v54) v1 ==== 110+0+0 (unknown 1865130469 0 0) 0x7f9430014c30 con 0x7f944009c0f0 2024-01-24T23:23:40.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.587+0000 7f9435ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}] 2024-01-24T23:23:40.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.587+0000 7f9435ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:40.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.679+0000 7f94474bc640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}] 2024-01-24T23:23:40.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.679+0000 7f94474bc640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}] 2024-01-24T23:23:40.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.679+0000 7f94474bc640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:40.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.679+0000 7f94474bc640 1 -- 172.21.15.132:0/2474977361 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"} v 0) v1 -- 0x7f9420005a80 con 0x7f944009c0f0 2024-01-24T23:23:40.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.683+0000 7f9435ffb640 1 -- 172.21.15.132:0/2474977361 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}]=0 already renamed to 'bar-asdf' v54) v1 ==== 139+0+0 (unknown 3506977516 0 0) 0x7f94300130a0 con 0x7f944009c0f0 2024-01-24T23:23:40.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.683+0000 7f9435ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}] 2024-01-24T23:23:40.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.683+0000 7f9435ffb640 10 monclient: _finish_command 3 = system:0 already renamed to 'bar-asdf' 2024-01-24T23:23:40.684 INFO:tasks.workunit.client.0.smithi132.stderr:already renamed to 'bar-asdf' 2024-01-24T23:23:40.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.683+0000 7f94474bc640 1 -- 172.21.15.132:0/2474977361 >> v1:172.21.15.132:6812/1835401922 conn(0x7f9424036f70 legacy=0x7f9424039430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:40.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.683+0000 7f94474bc640 10 monclient: shutdown 2024-01-24T23:23:40.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.683+0000 7f94474bc640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:40.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.683+0000 7f94474bc640 1 -- 172.21.15.132:0/2474977361 >> v1:172.21.15.132:6792/0 conn(0x7f944009c0f0 legacy=0x7f94401d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:40.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.683+0000 7f94474bc640 1 -- 172.21.15.132:0/2474977361 shutdown_connections 2024-01-24T23:23:40.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.683+0000 7f94474bc640 1 -- 172.21.15.132:0/2474977361 >> 172.21.15.132:0/2474977361 conn(0x7f94400a9bf0 msgr2=0x7f94400a8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:40.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.683+0000 7f94474bc640 1 -- 172.21.15.132:0/2474977361 shutdown_connections 2024-01-24T23:23:40.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.683+0000 7f94474bc640 1 -- 172.21.15.132:0/2474977361 wait complete. 2024-01-24T23:23:40.701 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule ls 2024-01-24T23:23:40.701 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep foo-asdf 2024-01-24T23:23:40.821 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f929b3e8640 10 monclient: get_monmap_and_config 2024-01-24T23:23:40.821 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f929b3e8640 10 monclient: build_initial_monmap 2024-01-24T23:23:40.821 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f929b3e8640 10 monclient: monmap: 2024-01-24T23:23:40.822 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:40.822 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:40.822 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:40.822745+0000 2024-01-24T23:23:40.822 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:40.822745+0000 2024-01-24T23:23:40.822 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:40.822 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:40.822 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:40.822 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:40.822 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:40.822 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:40.822 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:40.822 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:40.823 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:40.823 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:40.823 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:40.823 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:40.829 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f929b3e8640 1 Processor -- start 2024-01-24T23:23:40.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f929b3e8640 1 -- start start 2024-01-24T23:23:40.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f929b3e8640 10 monclient: init 2024-01-24T23:23:40.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f929b3e8640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:40.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f929b3e8640 10 monclient: _add_conns ranks=[7,5,2,6,1,0,4,8,3] 2024-01-24T23:23:40.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f929b3e8640 10 monclient(hunting): picked mon.noname-i con 0x7f929409ecb0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:40.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f929b3e8640 10 monclient(hunting): picked mon.noname-h con 0x7f929409f090 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:40.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f929b3e8640 10 monclient(hunting): picked mon.noname-b con 0x7f929409f490 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:40.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f929b3e8640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9294064700 con 0x7f929409f490 2024-01-24T23:23:40.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f929b3e8640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9294067240 con 0x7f929409f090 2024-01-24T23:23:40.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f929b3e8640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f929409bbd0 con 0x7f929409ecb0 2024-01-24T23:23:40.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f929b3e8640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:40.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f929b3e8640 10 monclient(hunting): authenticate will time out at 1108.147489s 2024-01-24T23:23:40.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f9299184640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f929409ecb0 0x7f9294134ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:41640/0 (socket says 172.21.15.132:41640) 2024-01-24T23:23:40.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f9298983640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f929409f090 0x7f9294139de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:57608/0 (socket says 172.21.15.132:57608) 2024-01-24T23:23:40.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f9299184640 1 -- 172.21.15.132:0/1120584389 learned_addr learned my addr 172.21.15.132:0/1120584389 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:40.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f928bfff640 1 -- 172.21.15.132:0/1120584389 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1871035903 0 0) 0x7f9294067240 con 0x7f929409f090 2024-01-24T23:23:40.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f928bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:40.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f928bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:40.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f928bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:40.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f928bfff640 10 monclient(hunting): my global_id is 5117 2024-01-24T23:23:40.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f928bfff640 1 -- 172.21.15.132:0/1120584389 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9280003530 con 0x7f929409f090 2024-01-24T23:23:40.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.819+0000 7f928bfff640 1 -- 172.21.15.132:0/1120584389 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 516767197 0 0) 0x7f929409bbd0 con 0x7f929409ecb0 2024-01-24T23:23:40.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:40.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:40.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:40.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 10 monclient(hunting): my global_id is 5254 2024-01-24T23:23:40.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 1 -- 172.21.15.132:0/1120584389 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9294067240 con 0x7f929409ecb0 2024-01-24T23:23:40.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 1 -- 172.21.15.132:0/1120584389 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 796265562 0 0) 0x7f9280003530 con 0x7f929409f090 2024-01-24T23:23:40.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 1 -- 172.21.15.132:0/1120584389 >> v1:172.21.15.132:6792/0 conn(0x7f929409ecb0 legacy=0x7f9294134ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:40.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 1 -- 172.21.15.132:0/1120584389 >> v1:172.21.15.115:6790/0 conn(0x7f929409f490 legacy=0x7f929413d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:40.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:23:40.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 1 monclient: found mon.noname-h 2024-01-24T23:23:40.833 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:40.833 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:23:40.833 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 1 -- 172.21.15.132:0/1120584389 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92941355b0 con 0x7f929409f090 2024-01-24T23:23:40.833 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 10 monclient: _finish_auth 0 2024-01-24T23:23:40.833 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:23:40.833 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:40.833 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 1 -- 172.21.15.132:0/1120584389 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f92840016f0 con 0x7f929409f090 2024-01-24T23:23:40.833 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 5 monclient: authenticate success, global_id 5117 2024-01-24T23:23:40.833 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:40.833 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:40.834 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:40.834 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 10 monclient: dump: 2024-01-24T23:23:40.834 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:40.834 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:40.834 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:40.834 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:40.834 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:40.834 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:40.834 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:40.834 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:40.834 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:40.835 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:40.835 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:40.835 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:40.835 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:40.835 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:40.835 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:40.835 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:40.835 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:40.835 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 1 -- 172.21.15.132:0/1120584389 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9284001990 con 0x7f929409f090 2024-01-24T23:23:40.835 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:40.835 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 1 -- 172.21.15.132:0/1120584389 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9284001bc0 con 0x7f929409f090 2024-01-24T23:23:40.836 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:40.836 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 4 set_mon_vals no callback set 2024-01-24T23:23:40.836 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:40.836 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:40.836 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928bfff640 10 monclient: dump: 2024-01-24T23:23:40.836 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:40.836 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:40.836 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:40.836 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:40.836 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:40.836 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:40.836 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:40.837 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:40.837 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:40.837 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:40.837 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:40.837 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:40.837 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:40.837 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:40.837 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:40.837 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:40.837 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 10 monclient: shutdown 2024-01-24T23:23:40.837 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:40.837 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 1 -- 172.21.15.132:0/1120584389 >> v1:172.21.15.132:6791/0 conn(0x7f929409f090 legacy=0x7f9294139de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:40.838 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 1 -- 172.21.15.132:0/1120584389 shutdown_connections 2024-01-24T23:23:40.838 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 1 -- 172.21.15.132:0/1120584389 >> 172.21.15.132:0/1120584389 conn(0x7f92940a2000 msgr2=0x7f92940a23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:40.838 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 1 -- 172.21.15.132:0/1120584389 shutdown_connections 2024-01-24T23:23:40.838 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 1 -- 172.21.15.132:0/1120584389 wait complete. 2024-01-24T23:23:40.838 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 10 monclient: build_initial_monmap 2024-01-24T23:23:40.838 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 10 monclient: monmap: 2024-01-24T23:23:40.838 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:40.838 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:40.838 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:40.838 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:40.838 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:40.839 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:40.839 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:40.839 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:40.839 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:40.839 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:40.839 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:40.839 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:40.839 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:40.839 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:40.839 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:40.839 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:40.840 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 1 Processor -- start 2024-01-24T23:23:40.840 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 1 -- start start 2024-01-24T23:23:40.840 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 10 monclient: init 2024-01-24T23:23:40.840 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:40.840 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 10 monclient: _add_conns ranks=[2,5,0,8,4,6,7,1,3] 2024-01-24T23:23:40.840 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 10 monclient(hunting): picked mon.noname-b con 0x7f929409ecb0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:40.840 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 10 monclient(hunting): picked mon.noname-h con 0x7f929409f090 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:40.840 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 10 monclient(hunting): picked mon.noname-a con 0x7f929409f490 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:40.840 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f929412b580 con 0x7f929409f490 2024-01-24T23:23:40.840 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f929412c860 con 0x7f929409ecb0 2024-01-24T23:23:40.840 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f929412a800 con 0x7f929409f090 2024-01-24T23:23:40.840 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:40.840 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 10 monclient(hunting): authenticate will time out at 1108.150722s 2024-01-24T23:23:40.841 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9298983640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f929409f090 0x7f92941cee70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:57620/0 (socket says 172.21.15.132:57620) 2024-01-24T23:23:40.841 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9298983640 1 -- 172.21.15.132:0/3262119657 learned_addr learned my addr 172.21.15.132:0/3262119657 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:40.841 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9299985640 1 -- 172.21.15.132:0/3262119657 >> v1:172.21.15.115:6789/0 conn(0x7f929409f490 legacy=0x7f92941d25a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:40.841 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 1 -- 172.21.15.132:0/3262119657 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4192937221 0 0) 0x7f929412a800 con 0x7f929409f090 2024-01-24T23:23:40.841 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:40.841 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:40.841 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:40.841 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 10 monclient(hunting): my global_id is 5126 2024-01-24T23:23:40.841 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 1 -- 172.21.15.132:0/3262119657 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9278003530 con 0x7f929409f090 2024-01-24T23:23:40.841 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 1 -- 172.21.15.132:0/3262119657 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3329724586 0 0) 0x7f9278003530 con 0x7f929409f090 2024-01-24T23:23:40.841 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 1 -- 172.21.15.132:0/3262119657 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f929412a800 con 0x7f929409f090 2024-01-24T23:23:40.842 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 1 -- 172.21.15.132:0/3262119657 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9284001b40 con 0x7f929409f090 2024-01-24T23:23:40.842 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:40.842 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:40.842 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 10 monclient(hunting): dump: 2024-01-24T23:23:40.842 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:40.842 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:40.842 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:40.842 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:40.842 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:40.842 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:40.842 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:40.842 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:40.842 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:40.843 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:40.843 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:40.843 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:40.843 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:40.843 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:40.843 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:40.843 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:40.843 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:40.843 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 1 -- 172.21.15.132:0/3262119657 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3067324248 0 0) 0x7f929412c860 con 0x7f929409ecb0 2024-01-24T23:23:40.843 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:40.843 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:40.844 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:40.844 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 10 monclient(hunting): my global_id is 84440 2024-01-24T23:23:40.844 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 1 -- 172.21.15.132:0/3262119657 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9278003530 con 0x7f929409ecb0 2024-01-24T23:23:40.844 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 1 -- 172.21.15.132:0/3262119657 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 186521352 0 0) 0x7f929412a800 con 0x7f929409f090 2024-01-24T23:23:40.844 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 1 -- 172.21.15.132:0/3262119657 >> v1:172.21.15.115:6790/0 conn(0x7f929409ecb0 legacy=0x7f92941c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:40.844 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 1 -- 172.21.15.132:0/3262119657 >> v1:172.21.15.115:6789/0 conn(0x7f929409f490 legacy=0x7f92941d25a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:40.844 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:23:40.844 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 1 monclient: found mon.h 2024-01-24T23:23:40.844 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:40.844 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:40.844 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 1 -- 172.21.15.132:0/3262119657 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929412cb40 con 0x7f929409f090 2024-01-24T23:23:40.845 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 10 monclient: _finish_auth 0 2024-01-24T23:23:40.845 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:23:40.845 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:40.845 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 5 monclient: authenticate success, global_id 5126 2024-01-24T23:23:40.845 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 10 monclient: _renew_subs 2024-01-24T23:23:40.845 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:40.845 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 1 -- 172.21.15.132:0/3262119657 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f929409bd80 con 0x7f929409f090 2024-01-24T23:23:40.845 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 10 monclient: _renew_subs 2024-01-24T23:23:40.845 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:40.845 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f929b3e8640 1 -- 172.21.15.132:0/3262119657 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f929412c580 con 0x7f929409f090 2024-01-24T23:23:40.845 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 1 -- 172.21.15.132:0/3262119657 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9284004e60 con 0x7f929409f090 2024-01-24T23:23:40.846 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:40.846 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 1 -- 172.21.15.132:0/3262119657 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9284005310 con 0x7f929409f090 2024-01-24T23:23:40.846 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:40.846 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f928b7fe640 4 set_mon_vals no callback set 2024-01-24T23:23:40.846 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:40.846 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.823+0000 7f9289ffb640 10 monclient: dump: 2024-01-24T23:23:40.846 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:40.846 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:40.846 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:40.846 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:40.846 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:40.846 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:40.847 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:40.847 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:40.847 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:40.847 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:40.847 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:40.847 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:40.847 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:40.847 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:40.847 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:40.847 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:40.847 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.827+0000 7f9289ffb640 1 -- 172.21.15.132:0/3262119657 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f92840054f0 con 0x7f929409f090 2024-01-24T23:23:40.847 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.827+0000 7f929b3e8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:40.848 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.827+0000 7f929b3e8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:40.848 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.827+0000 7f929b3e8640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:40.848 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.827+0000 7f929b3e8640 1 -- 172.21.15.132:0/3262119657 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9274005090 con 0x7f929409f090 2024-01-24T23:23:40.848 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.827+0000 7f9289ffb640 1 -- 172.21.15.132:0/3262119657 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(54..54 src has 1..54) v4 ==== 4315+0+0 (unknown 3599341251 0 0) 0x7f9284043d50 con 0x7f929409f090 2024-01-24T23:23:40.848 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.827+0000 7f9299184640 10 monclient: get_auth_request con 0x7f9278036fa0 auth_method 0 2024-01-24T23:23:40.848 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.831+0000 7f9289ffb640 1 -- 172.21.15.132:0/3262119657 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f9284014bd0 con 0x7f929409f090 2024-01-24T23:23:40.849 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.831+0000 7f9289ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:40.849 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.831+0000 7f9289ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:40.992 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.987+0000 7f929b3e8640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:40.992 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.987+0000 7f929b3e8640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:40.992 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.987+0000 7f929b3e8640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:40.992 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.987+0000 7f929b3e8640 1 -- 172.21.15.132:0/3262119657 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rule ls"} v 0) v1 -- 0x7f92740058c0 con 0x7f929409f090 2024-01-24T23:23:40.992 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.991+0000 7f9289ffb640 1 -- 172.21.15.132:0/3262119657 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v54) v1 ==== 65+0+70 (unknown 940634683 0 3909614116) 0x7f9284013270 con 0x7f929409f090 2024-01-24T23:23:40.992 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.991+0000 7f9289ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:40.992 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:40.991+0000 7f9289ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:41.086 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.083+0000 7f929b3e8640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:41.086 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.083+0000 7f929b3e8640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:41.086 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.083+0000 7f929b3e8640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:41.086 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.083+0000 7f929b3e8640 1 -- 172.21.15.132:0/3262119657 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rule ls"} v 0) v1 -- 0x7f9274000bd0 con 0x7f929409f090 2024-01-24T23:23:41.087 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.083+0000 7f9289ffb640 1 -- 172.21.15.132:0/3262119657 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v54) v1 ==== 65+0+70 (unknown 940634683 0 3909614116) 0x7f92840149f0 con 0x7f929409f090 2024-01-24T23:23:41.087 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.083+0000 7f9289ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:41.087 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.083+0000 7f9289ffb640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:23:41.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.087+0000 7f929b3e8640 1 -- 172.21.15.132:0/3262119657 >> v1:172.21.15.132:6812/1835401922 conn(0x7f9278036fa0 legacy=0x7f9278039460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:41.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.087+0000 7f929b3e8640 10 monclient: shutdown 2024-01-24T23:23:41.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.087+0000 7f929b3e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:41.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.087+0000 7f929b3e8640 1 -- 172.21.15.132:0/3262119657 >> v1:172.21.15.132:6791/0 conn(0x7f929409f090 legacy=0x7f92941cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:41.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.087+0000 7f929b3e8640 1 -- 172.21.15.132:0/3262119657 shutdown_connections 2024-01-24T23:23:41.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.087+0000 7f929b3e8640 1 -- 172.21.15.132:0/3262119657 >> 172.21.15.132:0/3262119657 conn(0x7f92940a2000 msgr2=0x7f929409fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:41.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.087+0000 7f929b3e8640 1 -- 172.21.15.132:0/3262119657 shutdown_connections 2024-01-24T23:23:41.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.087+0000 7f929b3e8640 1 -- 172.21.15.132:0/3262119657 wait complete. 2024-01-24T23:23:41.103 INFO:tasks.workunit.client.0.smithi132.stdout:foo-asdf 2024-01-24T23:23:41.104 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule ls 2024-01-24T23:23:41.104 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep bar-asdf 2024-01-24T23:23:41.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 10 monclient: get_monmap_and_config 2024-01-24T23:23:41.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 10 monclient: build_initial_monmap 2024-01-24T23:23:41.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 10 monclient: monmap: 2024-01-24T23:23:41.221 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:41.221 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:41.221 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:41.221975+0000 2024-01-24T23:23:41.221 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:41.221975+0000 2024-01-24T23:23:41.221 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:41.221 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:41.221 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:41.221 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:41.221 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:41.221 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:41.222 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:41.222 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:41.222 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:41.222 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:41.222 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:41.222 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:41.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 1 Processor -- start 2024-01-24T23:23:41.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 1 -- start start 2024-01-24T23:23:41.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 10 monclient: init 2024-01-24T23:23:41.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:41.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 10 monclient: _add_conns ranks=[0,5,1,8,3,7,6,4,2] 2024-01-24T23:23:41.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 10 monclient(hunting): picked mon.noname-a con 0x7f925009b870 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:41.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 10 monclient(hunting): picked mon.noname-h con 0x7f9250133e20 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:41.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 10 monclient(hunting): picked mon.noname-f con 0x7f9250134280 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:41.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9250064700 con 0x7f925009b870 2024-01-24T23:23:41.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92500a2cf0 con 0x7f9250134280 2024-01-24T23:23:41.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9250073140 con 0x7f9250133e20 2024-01-24T23:23:41.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:41.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 10 monclient(hunting): authenticate will time out at 1108.546531s 2024-01-24T23:23:41.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9255130640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f9250134280 legacy=0x7f925013c2d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:41.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924ffff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f9250133e20 0x7f9250138ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:57634/0 (socket says 172.21.15.132:57634) 2024-01-24T23:23:41.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924ffff640 1 -- 172.21.15.132:0/428837819 learned_addr learned my addr 172.21.15.132:0/428837819 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:41.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f925492f640 1 -- 172.21.15.132:0/428837819 >> v1:172.21.15.115:6789/0 conn(0x7f925009b870 legacy=0x7f925009bc50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:41.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 1 -- 172.21.15.132:0/428837819 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3592560256 0 0) 0x7f9250073140 con 0x7f9250133e20 2024-01-24T23:23:41.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:41.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:41.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:41.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 10 monclient(hunting): my global_id is 5135 2024-01-24T23:23:41.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 1 -- 172.21.15.132:0/428837819 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9238005740 con 0x7f9250133e20 2024-01-24T23:23:41.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 1 -- 172.21.15.132:0/428837819 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3311702255 0 0) 0x7f9238005740 con 0x7f9250133e20 2024-01-24T23:23:41.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 1 -- 172.21.15.132:0/428837819 >> v1:172.21.15.132:6789/0 conn(0x7f9250134280 legacy=0x7f925013c2d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:41.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 1 -- 172.21.15.132:0/428837819 >> v1:172.21.15.115:6789/0 conn(0x7f925009b870 legacy=0x7f925009bc50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:41.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:23:41.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 1 monclient: found mon.noname-h 2024-01-24T23:23:41.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:41.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:23:41.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 1 -- 172.21.15.132:0/428837819 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925009ef60 con 0x7f9250133e20 2024-01-24T23:23:41.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 10 monclient: _finish_auth 0 2024-01-24T23:23:41.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:23:41.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:41.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 5 monclient: authenticate success, global_id 5135 2024-01-24T23:23:41.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 1 -- 172.21.15.132:0/428837819 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f92440017c0 con 0x7f9250133e20 2024-01-24T23:23:41.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:41.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:41.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:41.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 10 monclient: dump: 2024-01-24T23:23:41.232 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:41.232 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:41.232 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:41.232 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:41.232 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:41.232 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:41.233 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:41.233 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:41.233 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:41.233 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:41.233 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:41.233 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:41.233 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:41.233 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:41.233 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:41.233 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:41.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:41.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 1 -- 172.21.15.132:0/428837819 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9244001ae0 con 0x7f9250133e20 2024-01-24T23:23:41.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:41.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:41.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 4 set_mon_vals no callback set 2024-01-24T23:23:41.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 1 -- 172.21.15.132:0/428837819 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9244004bf0 con 0x7f9250133e20 2024-01-24T23:23:41.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:41.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:41.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f924f7fe640 10 monclient: dump: 2024-01-24T23:23:41.234 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:41.234 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:41.234 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:41.234 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:41.235 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:41.235 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:41.235 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:41.235 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:41.235 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:41.235 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:41.235 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:41.235 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:41.235 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:41.235 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:41.235 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:41.235 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:41.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 10 monclient: shutdown 2024-01-24T23:23:41.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:41.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 1 -- 172.21.15.132:0/428837819 >> v1:172.21.15.132:6791/0 conn(0x7f9250133e20 legacy=0x7f9250138ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:41.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 1 -- 172.21.15.132:0/428837819 shutdown_connections 2024-01-24T23:23:41.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 1 -- 172.21.15.132:0/428837819 >> 172.21.15.132:0/428837819 conn(0x7f92500a1e40 msgr2=0x7f92500a2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:41.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.219+0000 7f9256b93640 1 -- 172.21.15.132:0/428837819 shutdown_connections 2024-01-24T23:23:41.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 1 -- 172.21.15.132:0/428837819 wait complete. 2024-01-24T23:23:41.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 10 monclient: build_initial_monmap 2024-01-24T23:23:41.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 10 monclient: monmap: 2024-01-24T23:23:41.236 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:41.236 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:41.236 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:41.236 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:41.237 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:41.237 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:41.237 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:41.237 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:41.237 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:41.237 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:41.237 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:41.237 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:41.237 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:41.237 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:41.237 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:41.238 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:41.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 1 Processor -- start 2024-01-24T23:23:41.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 1 -- start start 2024-01-24T23:23:41.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 10 monclient: init 2024-01-24T23:23:41.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:41.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 10 monclient: _add_conns ranks=[1,6,5,4,2,3,7,8,0] 2024-01-24T23:23:41.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 10 monclient(hunting): picked mon.noname-f con 0x7f925009b870 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:41.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 10 monclient(hunting): picked mon.noname-d con 0x7f9250133e20 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:41.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 10 monclient(hunting): picked mon.noname-h con 0x7f9250134280 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:41.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92501c7500 con 0x7f925009b870 2024-01-24T23:23:41.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92500717a0 con 0x7f9250134280 2024-01-24T23:23:41.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92500a1890 con 0x7f9250133e20 2024-01-24T23:23:41.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:41.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 10 monclient(hunting): authenticate will time out at 1108.549402s 2024-01-24T23:23:41.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f925492f640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f925009b870 legacy=0x7f92501c6df0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:41.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9255130640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f9250134280 0x7f92501cef30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:57648/0 (socket says 172.21.15.132:57648) 2024-01-24T23:23:41.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9255130640 1 -- 172.21.15.132:0/1849581303 learned_addr learned my addr 172.21.15.132:0/1849581303 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:41.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 1 -- 172.21.15.132:0/1849581303 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1150351035 0 0) 0x7f92500717a0 con 0x7f9250134280 2024-01-24T23:23:41.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:23:41.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:41.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:41.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 10 monclient(hunting): my global_id is 5144 2024-01-24T23:23:41.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 1 -- 172.21.15.132:0/1849581303 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f923c003530 con 0x7f9250134280 2024-01-24T23:23:41.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 1 -- 172.21.15.132:0/1849581303 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2802599564 0 0) 0x7f923c003530 con 0x7f9250134280 2024-01-24T23:23:41.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 1 -- 172.21.15.132:0/1849581303 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f92500717a0 con 0x7f9250134280 2024-01-24T23:23:41.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 1 -- 172.21.15.132:0/1849581303 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9248004530 con 0x7f9250134280 2024-01-24T23:23:41.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:41.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:41.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 10 monclient(hunting): dump: 2024-01-24T23:23:41.240 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:41.240 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:41.240 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:41.240 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:41.240 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:41.240 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:41.240 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:41.241 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:41.241 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:41.241 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:41.241 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:41.241 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:41.241 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:41.241 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:41.241 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:41.241 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:41.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:41.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 1 -- 172.21.15.132:0/1849581303 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2557413040 0 0) 0x7f92500717a0 con 0x7f9250134280 2024-01-24T23:23:41.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 1 -- 172.21.15.132:0/1849581303 >> v1:172.21.15.115:6792/0 conn(0x7f9250133e20 legacy=0x7f9250130970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:41.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 1 -- 172.21.15.132:0/1849581303 >> v1:172.21.15.132:6789/0 conn(0x7f925009b870 legacy=0x7f92501c6df0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:41.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 10 monclient: _finish_hunting 0 2024-01-24T23:23:41.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 1 monclient: found mon.h 2024-01-24T23:23:41.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:41.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:41.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 1 -- 172.21.15.132:0/1849581303 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925012b400 con 0x7f9250134280 2024-01-24T23:23:41.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 10 monclient: _finish_auth 0 2024-01-24T23:23:41.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:23:41.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:41.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 5 monclient: authenticate success, global_id 5144 2024-01-24T23:23:41.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 10 monclient: _renew_subs 2024-01-24T23:23:41.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:41.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 1 -- 172.21.15.132:0/1849581303 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9250071c00 con 0x7f9250134280 2024-01-24T23:23:41.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 10 monclient: _renew_subs 2024-01-24T23:23:41.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:41.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 1 -- 172.21.15.132:0/1849581303 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9250131250 con 0x7f9250134280 2024-01-24T23:23:41.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 1 -- 172.21.15.132:0/1849581303 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9248004850 con 0x7f9250134280 2024-01-24T23:23:41.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:41.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 1 -- 172.21.15.132:0/1849581303 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9248004d80 con 0x7f9250134280 2024-01-24T23:23:41.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:41.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924e7fc640 4 set_mon_vals no callback set 2024-01-24T23:23:41.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:41.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 10 monclient: dump: 2024-01-24T23:23:41.243 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:41.243 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:41.243 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:41.243 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:41.243 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:41.244 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:41.244 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:41.244 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:41.244 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:41.244 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:41.244 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:41.244 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:41.244 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:41.244 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:41.244 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:41.244 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:41.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:41.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:41.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:41.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f9256b93640 1 -- 172.21.15.132:0/1849581303 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9230005090 con 0x7f9250134280 2024-01-24T23:23:41.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 1 -- 172.21.15.132:0/1849581303 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f9248004f60 con 0x7f9250134280 2024-01-24T23:23:41.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f924d7fa640 1 -- 172.21.15.132:0/1849581303 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(54..54 src has 1..54) v4 ==== 4315+0+0 (unknown 3599341251 0 0) 0x7f9248043cd0 con 0x7f9250134280 2024-01-24T23:23:41.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.223+0000 7f925492f640 10 monclient: get_auth_request con 0x7f923c036f90 auth_method 0 2024-01-24T23:23:41.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.227+0000 7f924d7fa640 1 -- 172.21.15.132:0/1849581303 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f9248014d90 con 0x7f9250134280 2024-01-24T23:23:41.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.227+0000 7f924d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:41.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.227+0000 7f924d7fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:41.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.391+0000 7f9256b93640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:41.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.391+0000 7f9256b93640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:41.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.391+0000 7f9256b93640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:41.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.391+0000 7f9256b93640 1 -- 172.21.15.132:0/1849581303 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rule ls"} v 0) v1 -- 0x7f92300058c0 con 0x7f9250134280 2024-01-24T23:23:41.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.391+0000 7f924d7fa640 1 -- 172.21.15.132:0/1849581303 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v54) v1 ==== 65+0+70 (unknown 940634683 0 3909614116) 0x7f9248013200 con 0x7f9250134280 2024-01-24T23:23:41.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.391+0000 7f924d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:41.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.391+0000 7f924d7fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:41.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.487+0000 7f9256b93640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:41.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.487+0000 7f9256b93640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:41.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.487+0000 7f9256b93640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:41.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.487+0000 7f9256b93640 1 -- 172.21.15.132:0/1849581303 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rule ls"} v 0) v1 -- 0x7f9230000bd0 con 0x7f9250134280 2024-01-24T23:23:41.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.487+0000 7f924d7fa640 1 -- 172.21.15.132:0/1849581303 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v54) v1 ==== 65+0+70 (unknown 940634683 0 3909614116) 0x7f9248014980 con 0x7f9250134280 2024-01-24T23:23:41.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.487+0000 7f924d7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:41.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.487+0000 7f924d7fa640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:23:41.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.491+0000 7f9256b93640 1 -- 172.21.15.132:0/1849581303 >> v1:172.21.15.132:6812/1835401922 conn(0x7f923c036f90 legacy=0x7f923c039450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:41.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.491+0000 7f9256b93640 10 monclient: shutdown 2024-01-24T23:23:41.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.491+0000 7f9256b93640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:41.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.491+0000 7f9256b93640 1 -- 172.21.15.132:0/1849581303 >> v1:172.21.15.132:6791/0 conn(0x7f9250134280 legacy=0x7f92501cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:41.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.491+0000 7f9256b93640 1 -- 172.21.15.132:0/1849581303 shutdown_connections 2024-01-24T23:23:41.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.491+0000 7f9256b93640 1 -- 172.21.15.132:0/1849581303 >> 172.21.15.132:0/1849581303 conn(0x7f92500a1e40 msgr2=0x7f92500a3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:41.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.491+0000 7f9256b93640 1 -- 172.21.15.132:0/1849581303 shutdown_connections 2024-01-24T23:23:41.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:41.491+0000 7f9256b93640 1 -- 172.21.15.132:0/1849581303 wait complete. 2024-01-24T23:23:41.506 INFO:tasks.workunit.client.0.smithi132.stdout:bar-asdf 2024-01-24T23:23:41.507 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule rm foo 2024-01-24T23:23:41.507 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep 'does not exist' 2024-01-24T23:23:41.902 INFO:tasks.workunit.client.0.smithi132.stdout:2024-01-24T23:23:41.787+0000 7fd012ffd640 1 -- 172.21.15.132:0/680653081 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo"}]=0 rule foo does not exist v54) v1 ==== 103+0+0 (unknown 155420829 0 0) 0x7fd0040133a0 con 0x7fd01c1361f0 2024-01-24T23:23:41.902 INFO:tasks.workunit.client.0.smithi132.stdout:2024-01-24T23:23:41.787+0000 7fd012ffd640 10 monclient: _finish_command 2 = system:0 rule foo does not exist 2024-01-24T23:23:41.902 INFO:tasks.workunit.client.0.smithi132.stdout:2024-01-24T23:23:41.883+0000 7fd012ffd640 1 -- 172.21.15.132:0/680653081 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo"}]=0 rule foo does not exist v55) v1 ==== 103+0+0 (unknown 3048478203 0 0) 0x7fd004014b20 con 0x7fd01c1361f0 2024-01-24T23:23:41.902 INFO:tasks.workunit.client.0.smithi132.stdout:2024-01-24T23:23:41.883+0000 7fd012ffd640 10 monclient: _finish_command 3 = system:0 rule foo does not exist 2024-01-24T23:23:41.902 INFO:tasks.workunit.client.0.smithi132.stdout:rule foo does not exist 2024-01-24T23:23:41.902 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule rm bar 2024-01-24T23:23:41.903 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep 'does not exist' 2024-01-24T23:23:42.293 INFO:tasks.workunit.client.0.smithi132.stdout:2024-01-24T23:23:42.183+0000 7ff7d9ffb640 1 -- 172.21.15.132:0/3030633861 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "bar"}]=0 rule bar does not exist v55) v1 ==== 103+0+0 (unknown 3569920944 0 0) 0x7ff7e0012f20 con 0x7ff7e409f9c0 2024-01-24T23:23:42.293 INFO:tasks.workunit.client.0.smithi132.stdout:2024-01-24T23:23:42.183+0000 7ff7d9ffb640 10 monclient: _finish_command 2 = system:0 rule bar does not exist 2024-01-24T23:23:42.294 INFO:tasks.workunit.client.0.smithi132.stdout:2024-01-24T23:23:42.275+0000 7ff7d9ffb640 1 -- 172.21.15.132:0/3030633861 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "bar"}]=0 rule bar does not exist v55) v1 ==== 103+0+0 (unknown 3569920944 0 0) 0x7ff7e00146a0 con 0x7ff7e409f9c0 2024-01-24T23:23:42.294 INFO:tasks.workunit.client.0.smithi132.stdout:2024-01-24T23:23:42.275+0000 7ff7d9ffb640 10 monclient: _finish_command 3 = system:0 rule bar does not exist 2024-01-24T23:23:42.294 INFO:tasks.workunit.client.0.smithi132.stdout:rule bar does not exist 2024-01-24T23:23:42.294 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule rename foo-asdf foo 2024-01-24T23:23:42.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.407+0000 7f94f45e9640 10 monclient: get_monmap_and_config 2024-01-24T23:23:42.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.407+0000 7f94f45e9640 10 monclient: build_initial_monmap 2024-01-24T23:23:42.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 10 monclient: monmap: 2024-01-24T23:23:42.411 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:42.411 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:42.412 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:42.412765+0000 2024-01-24T23:23:42.412 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:42.412765+0000 2024-01-24T23:23:42.412 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:42.412 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:42.412 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:42.412 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:42.412 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:42.412 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:42.412 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:42.412 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:42.412 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:42.413 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:42.413 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:42.413 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:42.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 1 Processor -- start 2024-01-24T23:23:42.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 1 -- start start 2024-01-24T23:23:42.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 10 monclient: init 2024-01-24T23:23:42.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:42.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 10 monclient: _add_conns ranks=[2,6,5,0,1,7,3,8,4] 2024-01-24T23:23:42.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 10 monclient(hunting): picked mon.noname-b con 0x7f94ec072690 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:42.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 10 monclient(hunting): picked mon.noname-d con 0x7f94ec09ae90 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:42.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 10 monclient(hunting): picked mon.noname-h con 0x7f94ec09bdc0 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:42.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ec064700 con 0x7f94ec072690 2024-01-24T23:23:42.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ec09a790 con 0x7f94ec09bdc0 2024-01-24T23:23:42.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ec0a18f0 con 0x7f94ec09ae90 2024-01-24T23:23:42.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:42.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 10 monclient(hunting): authenticate will time out at 1109.737535s 2024-01-24T23:23:42.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f2b86640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f94ec09bdc0 0x7f94ec13d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:57664/0 (socket says 172.21.15.132:57664) 2024-01-24T23:23:42.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f2b86640 1 -- 172.21.15.132:0/1909481458 learned_addr learned my addr 172.21.15.132:0/1909481458 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:42.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 1 -- 172.21.15.132:0/1909481458 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3486053676 0 0) 0x7f94ec09a790 con 0x7f94ec09bdc0 2024-01-24T23:23:42.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 10 monclient(hunting): _negotiate 2024-01-24T23:23:42.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:42.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:42.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 10 monclient(hunting): my global_id is 5162 2024-01-24T23:23:42.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 1 -- 172.21.15.132:0/1909481458 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94d8003530 con 0x7f94ec09bdc0 2024-01-24T23:23:42.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 1 -- 172.21.15.132:0/1909481458 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2295310703 0 0) 0x7f94d8003530 con 0x7f94ec09bdc0 2024-01-24T23:23:42.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 1 -- 172.21.15.132:0/1909481458 >> v1:172.21.15.115:6792/0 conn(0x7f94ec09ae90 legacy=0x7f94ec0a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:42.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 1 -- 172.21.15.132:0/1909481458 >> v1:172.21.15.115:6790/0 conn(0x7f94ec072690 legacy=0x7f94ec0a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:42.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 10 monclient: _finish_hunting 0 2024-01-24T23:23:42.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 1 monclient: found mon.noname-h 2024-01-24T23:23:42.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:42.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:23:42.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 1 -- 172.21.15.132:0/1909481458 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ec0997a0 con 0x7f94ec09bdc0 2024-01-24T23:23:42.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 10 monclient: _finish_auth 0 2024-01-24T23:23:42.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 10 monclient: _check_auth_tickets 2024-01-24T23:23:42.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:42.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 1 -- 172.21.15.132:0/1909481458 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f94e80017c0 con 0x7f94ec09bdc0 2024-01-24T23:23:42.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:42.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:42.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 10 monclient: dump: 2024-01-24T23:23:42.424 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:42.424 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:42.424 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:42.424 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:42.424 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:42.424 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:42.424 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:42.424 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:42.424 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:42.424 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:42.425 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:42.425 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:42.425 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:42.425 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:42.425 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:42.425 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:42.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 5 monclient: authenticate success, global_id 5162 2024-01-24T23:23:42.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:42.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 1 -- 172.21.15.132:0/1909481458 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f94e8001a70 con 0x7f94ec09bdc0 2024-01-24T23:23:42.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:42.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:42.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 4 set_mon_vals no callback set 2024-01-24T23:23:42.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 1 -- 172.21.15.132:0/1909481458 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f94e8004bf0 con 0x7f94ec09bdc0 2024-01-24T23:23:42.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:42.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:42.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f1383640 10 monclient: dump: 2024-01-24T23:23:42.426 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:42.426 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:42.426 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:42.426 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:42.426 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:42.426 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:42.426 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:42.426 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:42.426 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:42.426 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:42.426 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:42.426 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:42.427 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:42.427 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:42.427 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:42.427 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:42.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 10 monclient: shutdown 2024-01-24T23:23:42.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:42.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 1 -- 172.21.15.132:0/1909481458 >> v1:172.21.15.132:6791/0 conn(0x7f94ec09bdc0 legacy=0x7f94ec13d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:42.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 1 -- 172.21.15.132:0/1909481458 shutdown_connections 2024-01-24T23:23:42.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 1 -- 172.21.15.132:0/1909481458 >> 172.21.15.132:0/1909481458 conn(0x7f94ec0a1bc0 msgr2=0x7f94ec0a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:42.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 1 -- 172.21.15.132:0/1909481458 shutdown_connections 2024-01-24T23:23:42.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 1 -- 172.21.15.132:0/1909481458 wait complete. 2024-01-24T23:23:42.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 10 monclient: build_initial_monmap 2024-01-24T23:23:42.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 10 monclient: monmap: 2024-01-24T23:23:42.427 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:42.427 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:42.427 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:42.428 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:42.428 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:42.428 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:42.428 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:42.428 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:42.428 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:42.428 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:42.428 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:42.428 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:42.428 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:42.428 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:42.428 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:42.428 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:42.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 1 Processor -- start 2024-01-24T23:23:42.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 1 -- start start 2024-01-24T23:23:42.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.411+0000 7f94f45e9640 10 monclient: init 2024-01-24T23:23:42.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f45e9640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:42.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f45e9640 10 monclient: _add_conns ranks=[0,7,3,6,5,2,1,8,4] 2024-01-24T23:23:42.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f45e9640 10 monclient(hunting): picked mon.noname-a con 0x7f94ec072690 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:42.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f45e9640 10 monclient(hunting): picked mon.noname-i con 0x7f94ec09ae90 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:42.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f45e9640 10 monclient(hunting): picked mon.noname-g con 0x7f94ec09bdc0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:42.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f45e9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ec1b0b50 con 0x7f94ec072690 2024-01-24T23:23:42.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f45e9640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ec1b06d0 con 0x7f94ec09bdc0 2024-01-24T23:23:42.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f45e9640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ec099470 con 0x7f94ec09ae90 2024-01-24T23:23:42.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f45e9640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:42.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f45e9640 10 monclient(hunting): authenticate will time out at 1109.740456s 2024-01-24T23:23:42.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f2385640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f94ec072690 legacy=0x7f94ec1b5ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:42.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f2b86640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f94ec09bdc0 0x7f94ec1d1e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:60448/0 (socket says 172.21.15.132:60448) 2024-01-24T23:23:42.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f2b86640 1 -- 172.21.15.132:0/3166813865 learned_addr learned my addr 172.21.15.132:0/3166813865 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:42.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 1 -- 172.21.15.132:0/3166813865 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398412813 0 0) 0x7f94ec1b06d0 con 0x7f94ec09bdc0 2024-01-24T23:23:42.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:42.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:42.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:42.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 10 monclient(hunting): my global_id is 5169 2024-01-24T23:23:42.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 1 -- 172.21.15.132:0/3166813865 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94d0003530 con 0x7f94ec09bdc0 2024-01-24T23:23:42.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 1 -- 172.21.15.132:0/3166813865 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2298560621 0 0) 0x7f94ec099470 con 0x7f94ec09ae90 2024-01-24T23:23:42.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:42.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:42.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:42.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 10 monclient(hunting): my global_id is 5263 2024-01-24T23:23:42.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 1 -- 172.21.15.132:0/3166813865 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94ec1b06d0 con 0x7f94ec09ae90 2024-01-24T23:23:42.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 1 -- 172.21.15.132:0/3166813865 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1651178926 0 0) 0x7f94d0003530 con 0x7f94ec09bdc0 2024-01-24T23:23:42.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 1 -- 172.21.15.132:0/3166813865 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f94ec099470 con 0x7f94ec09bdc0 2024-01-24T23:23:42.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 1 -- 172.21.15.132:0/3166813865 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f94e8001780 con 0x7f94ec09bdc0 2024-01-24T23:23:42.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:42.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:42.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 10 monclient(hunting): dump: 2024-01-24T23:23:42.431 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:42.431 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:42.431 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:42.431 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:42.431 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:42.431 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:42.431 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:42.431 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:42.431 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:42.431 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:42.431 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:42.431 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:42.431 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:42.431 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:42.431 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:42.431 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:42.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:42.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 1 -- 172.21.15.132:0/3166813865 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2973885142 0 0) 0x7f94ec1b06d0 con 0x7f94ec09ae90 2024-01-24T23:23:42.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 1 -- 172.21.15.132:0/3166813865 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f94d0003530 con 0x7f94ec09ae90 2024-01-24T23:23:42.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 1 -- 172.21.15.132:0/3166813865 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f94dc001950 con 0x7f94ec09ae90 2024-01-24T23:23:42.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:42.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:42.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 10 monclient(hunting): dump: 2024-01-24T23:23:42.432 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:42.432 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:42.432 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:42.432 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:42.432 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:42.432 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:42.432 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:42.433 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:42.433 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:42.433 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:42.433 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:42.433 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:42.433 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:42.433 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:42.433 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:42.433 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:42.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 1 -- 172.21.15.132:0/3166813865 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 429915669 0 0) 0x7f94ec099470 con 0x7f94ec09bdc0 2024-01-24T23:23:42.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 1 -- 172.21.15.132:0/3166813865 >> v1:172.21.15.132:6792/0 conn(0x7f94ec09ae90 legacy=0x7f94ec1ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:42.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 1 -- 172.21.15.132:0/3166813865 >> v1:172.21.15.115:6789/0 conn(0x7f94ec072690 legacy=0x7f94ec1b5ef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:42.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:42.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 1 monclient: found mon.g 2024-01-24T23:23:42.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:42.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:42.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 1 -- 172.21.15.132:0/3166813865 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ec09b930 con 0x7f94ec09bdc0 2024-01-24T23:23:42.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 10 monclient: _finish_auth 0 2024-01-24T23:23:42.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:42.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:42.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f45e9640 5 monclient: authenticate success, global_id 5169 2024-01-24T23:23:42.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f45e9640 10 monclient: _renew_subs 2024-01-24T23:23:42.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f45e9640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:42.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f45e9640 1 -- 172.21.15.132:0/3166813865 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ec1b26c0 con 0x7f94ec09bdc0 2024-01-24T23:23:42.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f45e9640 10 monclient: _renew_subs 2024-01-24T23:23:42.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f45e9640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:42.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f45e9640 1 -- 172.21.15.132:0/3166813865 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ec1b6600 con 0x7f94ec09bdc0 2024-01-24T23:23:42.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 1 -- 172.21.15.132:0/3166813865 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f94e8001100 con 0x7f94ec09bdc0 2024-01-24T23:23:42.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:42.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 1 -- 172.21.15.132:0/3166813865 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f94e8005160 con 0x7f94ec09bdc0 2024-01-24T23:23:42.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:42.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e3fff640 4 set_mon_vals no callback set 2024-01-24T23:23:42.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:42.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 10 monclient: dump: 2024-01-24T23:23:42.435 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:42.435 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:42.435 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:42.435 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:42.435 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:42.435 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:42.435 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:42.435 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:42.435 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:42.435 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:42.435 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:42.435 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:42.435 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:42.435 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:42.436 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:42.436 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:42.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 1 -- 172.21.15.132:0/3166813865 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f94e800fbe0 con 0x7f94ec09bdc0 2024-01-24T23:23:42.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f45e9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:42.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f45e9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:42.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f45e9640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:42.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f45e9640 1 -- 172.21.15.132:0/3166813865 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d4005090 con 0x7f94ec09bdc0 2024-01-24T23:23:42.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94e2ffd640 1 -- 172.21.15.132:0/3166813865 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(55..55 src has 1..55) v4 ==== 4315+0+0 (unknown 3419307139 0 0) 0x7f94e8043af0 con 0x7f94ec09bdc0 2024-01-24T23:23:42.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.415+0000 7f94f2385640 10 monclient: get_auth_request con 0x7f94d0037660 auth_method 0 2024-01-24T23:23:42.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.419+0000 7f94e2ffd640 1 -- 172.21.15.132:0/3166813865 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f94e80149e0 con 0x7f94ec09bdc0 2024-01-24T23:23:42.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.419+0000 7f94e2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:42.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.419+0000 7f94e2ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:42.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.563+0000 7f94f45e9640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-01-24T23:23:42.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.563+0000 7f94f45e9640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-01-24T23:23:42.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.563+0000 7f94f45e9640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:42.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.563+0000 7f94f45e9640 1 -- 172.21.15.132:0/3166813865 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"} v 0) v1 -- 0x7f94d40058c0 con 0x7f94ec09bdc0 2024-01-24T23:23:42.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.883+0000 7f94e2ffd640 1 -- 172.21.15.132:0/3166813865 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}]=0 v56) v1 ==== 110+0+0 (unknown 58632444 0 0) 0x7f94e8013080 con 0x7f94ec09bdc0 2024-01-24T23:23:42.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.883+0000 7f94e2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-01-24T23:23:42.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.883+0000 7f94e2ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:42.977 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.975+0000 7f94f45e9640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-01-24T23:23:42.977 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.975+0000 7f94f45e9640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-01-24T23:23:42.977 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.975+0000 7f94f45e9640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:42.977 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.975+0000 7f94f45e9640 1 -- 172.21.15.132:0/3166813865 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"} v 0) v1 -- 0x7f94d4002220 con 0x7f94ec09bdc0 2024-01-24T23:23:42.978 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.975+0000 7f94e2ffd640 1 -- 172.21.15.132:0/3166813865 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}]=0 already renamed to 'foo' v56) v1 ==== 134+0+0 (unknown 2746865145 0 0) 0x7f94e8014800 con 0x7f94ec09bdc0 2024-01-24T23:23:42.978 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.975+0000 7f94e2ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-01-24T23:23:42.978 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.975+0000 7f94e2ffd640 10 monclient: _finish_command 3 = system:0 already renamed to 'foo' 2024-01-24T23:23:42.978 INFO:tasks.workunit.client.0.smithi132.stderr:already renamed to 'foo' 2024-01-24T23:23:42.980 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.979+0000 7f94f45e9640 1 -- 172.21.15.132:0/3166813865 >> v1:172.21.15.132:6812/1835401922 conn(0x7f94d0037660 legacy=0x7f94d0039b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:42.980 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.979+0000 7f94f45e9640 10 monclient: shutdown 2024-01-24T23:23:42.980 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.979+0000 7f94f45e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:42.980 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.979+0000 7f94f45e9640 1 -- 172.21.15.132:0/3166813865 >> v1:172.21.15.132:6790/0 conn(0x7f94ec09bdc0 legacy=0x7f94ec1d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:42.980 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.979+0000 7f94f45e9640 1 -- 172.21.15.132:0/3166813865 shutdown_connections 2024-01-24T23:23:42.980 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.979+0000 7f94f45e9640 1 -- 172.21.15.132:0/3166813865 >> 172.21.15.132:0/3166813865 conn(0x7f94ec0a1bc0 msgr2=0x7f94ec1ba730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:42.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.979+0000 7f94f45e9640 1 -- 172.21.15.132:0/3166813865 shutdown_connections 2024-01-24T23:23:42.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:42.979+0000 7f94f45e9640 1 -- 172.21.15.132:0/3166813865 wait complete. 2024-01-24T23:23:42.994 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule rename foo-asdf foo 2024-01-24T23:23:43.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.107+0000 7f1f402c8640 10 monclient: get_monmap_and_config 2024-01-24T23:23:43.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.107+0000 7f1f402c8640 10 monclient: build_initial_monmap 2024-01-24T23:23:43.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.107+0000 7f1f402c8640 10 monclient: monmap: 2024-01-24T23:23:43.111 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:43.111 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:43.111 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:43.112440+0000 2024-01-24T23:23:43.111 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:43.112440+0000 2024-01-24T23:23:43.111 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:43.112 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:43.112 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:43.112 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:43.112 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:43.112 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:43.112 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:43.112 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:43.112 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:43.112 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:43.112 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:43.112 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:43.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 1 Processor -- start 2024-01-24T23:23:43.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 1 -- start start 2024-01-24T23:23:43.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 10 monclient: init 2024-01-24T23:23:43.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:43.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 10 monclient: _add_conns ranks=[3,0,8,4,6,7,2,5,1] 2024-01-24T23:23:43.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 10 monclient(hunting): picked mon.noname-g con 0x7f1f38072690 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:43.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 10 monclient(hunting): picked mon.noname-a con 0x7f1f3809ae90 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:43.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 10 monclient(hunting): picked mon.noname-e con 0x7f1f3809bdc0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:43.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f38064700 con 0x7f1f3809ae90 2024-01-24T23:23:43.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f3809a790 con 0x7f1f38072690 2024-01-24T23:23:43.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f380a18f0 con 0x7f1f3809bdc0 2024-01-24T23:23:43.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:43.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 10 monclient(hunting): authenticate will time out at 1110.437408s 2024-01-24T23:23:43.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3e064640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f1f38072690 0x7f1f380a0ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:60462/0 (socket says 172.21.15.132:60462) 2024-01-24T23:23:43.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3e064640 1 -- 172.21.15.132:0/2649908159 learned_addr learned my addr 172.21.15.132:0/2649908159 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:43.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d863640 1 -- 172.21.15.132:0/2649908159 >> v1:172.21.15.115:6789/0 conn(0x7f1f3809ae90 legacy=0x7f1f380a11e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:43.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 1 -- 172.21.15.132:0/2649908159 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4166048641 0 0) 0x7f1f3809a790 con 0x7f1f38072690 2024-01-24T23:23:43.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 10 monclient(hunting): _negotiate 2024-01-24T23:23:43.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:43.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:43.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 10 monclient(hunting): my global_id is 5178 2024-01-24T23:23:43.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 1 -- 172.21.15.132:0/2649908159 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f24003530 con 0x7f1f38072690 2024-01-24T23:23:43.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 1 -- 172.21.15.132:0/2649908159 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2566486633 0 0) 0x7f1f24003530 con 0x7f1f38072690 2024-01-24T23:23:43.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 1 -- 172.21.15.132:0/2649908159 >> v1:172.21.15.115:6793/0 conn(0x7f1f3809bdc0 legacy=0x7f1f3813d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:43.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 1 -- 172.21.15.132:0/2649908159 >> v1:172.21.15.115:6789/0 conn(0x7f1f3809ae90 legacy=0x7f1f380a11e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:43.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 10 monclient: _finish_hunting 0 2024-01-24T23:23:43.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 1 monclient: found mon.noname-g 2024-01-24T23:23:43.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:43.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:23:43.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 1 -- 172.21.15.132:0/2649908159 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f380997a0 con 0x7f1f38072690 2024-01-24T23:23:43.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 10 monclient: _finish_auth 0 2024-01-24T23:23:43.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 10 monclient: _check_auth_tickets 2024-01-24T23:23:43.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:43.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 1 -- 172.21.15.132:0/2649908159 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1f34001b40 con 0x7f1f38072690 2024-01-24T23:23:43.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 5 monclient: authenticate success, global_id 5178 2024-01-24T23:23:43.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:43.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:43.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:43.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 10 monclient: dump: 2024-01-24T23:23:43.124 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:43.124 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:43.124 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:43.124 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:43.124 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:43.124 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:43.124 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:43.124 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:43.124 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:43.124 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:43.124 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:43.124 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:43.125 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:43.125 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:43.125 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:43.125 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:43.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:43.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 1 -- 172.21.15.132:0/2649908159 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1f34001df0 con 0x7f1f38072690 2024-01-24T23:23:43.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:43.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:43.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 4 set_mon_vals no callback set 2024-01-24T23:23:43.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 1 -- 172.21.15.132:0/2649908159 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1f34004f70 con 0x7f1f38072690 2024-01-24T23:23:43.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:43.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:43.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f3d062640 10 monclient: dump: 2024-01-24T23:23:43.126 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:43.126 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:43.126 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:43.126 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:43.126 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:43.126 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:43.126 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:43.126 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:43.126 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:43.126 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:43.126 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:43.127 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:43.127 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:43.127 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:43.127 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:43.127 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:43.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 10 monclient: shutdown 2024-01-24T23:23:43.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:43.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 1 -- 172.21.15.132:0/2649908159 >> v1:172.21.15.132:6790/0 conn(0x7f1f38072690 legacy=0x7f1f380a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:43.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 1 -- 172.21.15.132:0/2649908159 shutdown_connections 2024-01-24T23:23:43.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 1 -- 172.21.15.132:0/2649908159 >> 172.21.15.132:0/2649908159 conn(0x7f1f380a1bc0 msgr2=0x7f1f380a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:43.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 1 -- 172.21.15.132:0/2649908159 shutdown_connections 2024-01-24T23:23:43.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 1 -- 172.21.15.132:0/2649908159 wait complete. 2024-01-24T23:23:43.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 10 monclient: build_initial_monmap 2024-01-24T23:23:43.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 10 monclient: monmap: 2024-01-24T23:23:43.128 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:43.128 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:43.128 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:43.128 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:43.128 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:43.128 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:43.128 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:43.128 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:43.129 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:43.129 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:43.129 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:43.129 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:43.129 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:43.129 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:43.129 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:43.129 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:43.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 1 Processor -- start 2024-01-24T23:23:43.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 1 -- start start 2024-01-24T23:23:43.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.111+0000 7f1f402c8640 10 monclient: init 2024-01-24T23:23:43.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f402c8640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:43.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f402c8640 10 monclient: _add_conns ranks=[1,7,6,5,4,2,3,0,8] 2024-01-24T23:23:43.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f402c8640 10 monclient(hunting): picked mon.noname-f con 0x7f1f38072690 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:43.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f402c8640 10 monclient(hunting): picked mon.noname-i con 0x7f1f3809ae90 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:43.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f402c8640 10 monclient(hunting): picked mon.noname-d con 0x7f1f3809bdc0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:43.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f402c8640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f3806ed50 con 0x7f1f38072690 2024-01-24T23:23:43.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f3e064640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f1f38072690 legacy=0x7f1f381c8ff0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:43.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f402c8640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f381c9e10 con 0x7f1f3809bdc0 2024-01-24T23:23:43.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f402c8640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f38099470 con 0x7f1f3809ae90 2024-01-24T23:23:43.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f402c8640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:43.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f402c8640 10 monclient(hunting): authenticate will time out at 1110.440358s 2024-01-24T23:23:43.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f3d863640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f1f3809ae90 0x7f1f381c9700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:41666/0 (socket says 172.21.15.132:41666) 2024-01-24T23:23:43.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f3d863640 1 -- 172.21.15.132:0/2150976259 learned_addr learned my addr 172.21.15.132:0/2150976259 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:43.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 1 -- 172.21.15.132:0/2150976259 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3628837475 0 0) 0x7f1f38099470 con 0x7f1f3809ae90 2024-01-24T23:23:43.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:43.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:43.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:43.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 10 monclient(hunting): my global_id is 5272 2024-01-24T23:23:43.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 1 -- 172.21.15.132:0/2150976259 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f1c003530 con 0x7f1f3809ae90 2024-01-24T23:23:43.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 1 -- 172.21.15.132:0/2150976259 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3276495906 0 0) 0x7f1f1c003530 con 0x7f1f3809ae90 2024-01-24T23:23:43.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 1 -- 172.21.15.132:0/2150976259 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1f38099470 con 0x7f1f3809ae90 2024-01-24T23:23:43.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 1 -- 172.21.15.132:0/2150976259 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1f28004330 con 0x7f1f3809ae90 2024-01-24T23:23:43.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:43.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:23:43.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 10 monclient(hunting): dump: 2024-01-24T23:23:43.131 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:43.131 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:43.131 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:43.132 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:43.132 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:43.132 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:43.132 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:43.132 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:43.132 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:43.132 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:43.132 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:43.132 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:43.132 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:43.132 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:43.132 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:43.133 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:43.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:43.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 1 -- 172.21.15.132:0/2150976259 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 108120385 0 0) 0x7f1f381c9e10 con 0x7f1f3809bdc0 2024-01-24T23:23:43.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:43.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:43.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:43.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 10 monclient(hunting): my global_id is 5253 2024-01-24T23:23:43.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 1 -- 172.21.15.132:0/2150976259 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f1c003530 con 0x7f1f3809bdc0 2024-01-24T23:23:43.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 1 -- 172.21.15.132:0/2150976259 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4043027370 0 0) 0x7f1f38099470 con 0x7f1f3809ae90 2024-01-24T23:23:43.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 1 -- 172.21.15.132:0/2150976259 >> v1:172.21.15.115:6792/0 conn(0x7f1f3809bdc0 legacy=0x7f1f381d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:43.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 1 -- 172.21.15.132:0/2150976259 >> v1:172.21.15.132:6789/0 conn(0x7f1f38072690 legacy=0x7f1f381c8ff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:43.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:43.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 1 monclient: found mon.i 2024-01-24T23:23:43.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:43.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:43.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 1 -- 172.21.15.132:0/2150976259 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3809b930 con 0x7f1f3809ae90 2024-01-24T23:23:43.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 10 monclient: _finish_auth 0 2024-01-24T23:23:43.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:43.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:43.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f402c8640 5 monclient: authenticate success, global_id 5272 2024-01-24T23:23:43.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f402c8640 10 monclient: _renew_subs 2024-01-24T23:23:43.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f402c8640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:43.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f402c8640 1 -- 172.21.15.132:0/2150976259 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f38070680 con 0x7f1f3809ae90 2024-01-24T23:23:43.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f402c8640 10 monclient: _renew_subs 2024-01-24T23:23:43.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f402c8640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:43.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f402c8640 1 -- 172.21.15.132:0/2150976259 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f381b65f0 con 0x7f1f3809ae90 2024-01-24T23:23:43.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 1 -- 172.21.15.132:0/2150976259 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1f28004650 con 0x7f1f3809ae90 2024-01-24T23:23:43.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:43.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 1 -- 172.21.15.132:0/2150976259 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1f28004b80 con 0x7f1f3809ae90 2024-01-24T23:23:43.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:43.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f3c861640 4 set_mon_vals no callback set 2024-01-24T23:23:43.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:43.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 10 monclient: dump: 2024-01-24T23:23:43.135 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:43.135 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:43.136 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:43.136 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:43.136 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:43.136 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:43.136 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:43.136 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:43.136 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:43.136 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:43.136 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:43.137 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:43.137 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:43.137 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:43.137 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:43.137 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:43.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f402c8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:43.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f402c8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:43.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f402c8640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:43.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f402c8640 1 -- 172.21.15.132:0/2150976259 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f20005090 con 0x7f1f3809ae90 2024-01-24T23:23:43.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 1 -- 172.21.15.132:0/2150976259 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f1f28004ce0 con 0x7f1f3809ae90 2024-01-24T23:23:43.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f2effd640 1 -- 172.21.15.132:0/2150976259 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(56..56 src has 1..56) v4 ==== 4310+0+0 (unknown 701232376 0 0) 0x7f1f28043ad0 con 0x7f1f3809ae90 2024-01-24T23:23:43.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.115+0000 7f1f3e064640 10 monclient: get_auth_request con 0x7f1f1c036fd0 auth_method 0 2024-01-24T23:23:43.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.119+0000 7f1f2effd640 1 -- 172.21.15.132:0/2150976259 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f1f28014b20 con 0x7f1f3809ae90 2024-01-24T23:23:43.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.119+0000 7f1f2effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:43.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.119+0000 7f1f2effd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:43.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.275+0000 7f1f402c8640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-01-24T23:23:43.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.275+0000 7f1f402c8640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-01-24T23:23:43.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.275+0000 7f1f402c8640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:43.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.275+0000 7f1f402c8640 1 -- 172.21.15.132:0/2150976259 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"} v 0) v1 -- 0x7f1f200058c0 con 0x7f1f3809ae90 2024-01-24T23:23:43.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.279+0000 7f1f2effd640 1 -- 172.21.15.132:0/2150976259 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}]=0 already renamed to 'foo' v56) v1 ==== 134+0+0 (unknown 2746865145 0 0) 0x7f1f28012f90 con 0x7f1f3809ae90 2024-01-24T23:23:43.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.279+0000 7f1f2effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-01-24T23:23:43.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.279+0000 7f1f2effd640 10 monclient: _finish_command 2 = system:0 already renamed to 'foo' 2024-01-24T23:23:43.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.371+0000 7f1f402c8640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-01-24T23:23:43.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.371+0000 7f1f402c8640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-01-24T23:23:43.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.371+0000 7f1f402c8640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:43.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.371+0000 7f1f402c8640 1 -- 172.21.15.132:0/2150976259 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"} v 0) v1 -- 0x7f1f200010a0 con 0x7f1f3809ae90 2024-01-24T23:23:43.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.371+0000 7f1f2effd640 1 -- 172.21.15.132:0/2150976259 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}]=0 already renamed to 'foo' v56) v1 ==== 134+0+0 (unknown 2746865145 0 0) 0x7f1f28014710 con 0x7f1f3809ae90 2024-01-24T23:23:43.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.371+0000 7f1f2effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-01-24T23:23:43.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.371+0000 7f1f2effd640 10 monclient: _finish_command 3 = system:0 already renamed to 'foo' 2024-01-24T23:23:43.373 INFO:tasks.workunit.client.0.smithi132.stderr:already renamed to 'foo' 2024-01-24T23:23:43.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.371+0000 7f1f402c8640 1 -- 172.21.15.132:0/2150976259 >> v1:172.21.15.132:6812/1835401922 conn(0x7f1f1c036fd0 legacy=0x7f1f1c039490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:43.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.371+0000 7f1f402c8640 10 monclient: shutdown 2024-01-24T23:23:43.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.371+0000 7f1f402c8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:43.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.371+0000 7f1f402c8640 1 -- 172.21.15.132:0/2150976259 >> v1:172.21.15.132:6792/0 conn(0x7f1f3809ae90 legacy=0x7f1f381c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:43.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.371+0000 7f1f402c8640 1 -- 172.21.15.132:0/2150976259 shutdown_connections 2024-01-24T23:23:43.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.371+0000 7f1f402c8640 1 -- 172.21.15.132:0/2150976259 >> 172.21.15.132:0/2150976259 conn(0x7f1f380a1bc0 msgr2=0x7f1f380a3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:43.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.371+0000 7f1f402c8640 1 -- 172.21.15.132:0/2150976259 shutdown_connections 2024-01-24T23:23:43.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.371+0000 7f1f402c8640 1 -- 172.21.15.132:0/2150976259 wait complete. 2024-01-24T23:23:43.388 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule rename bar-asdf bar 2024-01-24T23:23:43.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9bd34c640 10 monclient: get_monmap_and_config 2024-01-24T23:23:43.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9bd34c640 10 monclient: build_initial_monmap 2024-01-24T23:23:43.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9bd34c640 10 monclient: monmap: 2024-01-24T23:23:43.501 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:43.501 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:43.501 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:43.502606+0000 2024-01-24T23:23:43.501 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:43.502606+0000 2024-01-24T23:23:43.501 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:43.502 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:43.502 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:43.502 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:43.502 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:43.502 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:43.502 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:43.502 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:43.502 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:43.502 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:43.502 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:43.502 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:43.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9bd34c640 1 Processor -- start 2024-01-24T23:23:43.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9bd34c640 1 -- start start 2024-01-24T23:23:43.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9bd34c640 10 monclient: init 2024-01-24T23:23:43.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9bd34c640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:43.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9bd34c640 10 monclient: _add_conns ranks=[4,3,1,2,5,6,0,8,7] 2024-01-24T23:23:43.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9bd34c640 10 monclient(hunting): picked mon.noname-c con 0x7fe9b809c310 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:43.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9bd34c640 10 monclient(hunting): picked mon.noname-g con 0x7fe9b8134ef0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:43.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9bd34c640 10 monclient(hunting): picked mon.noname-f con 0x7fe9b813b420 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:43.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9bd34c640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b8064700 con 0x7fe9b813b420 2024-01-24T23:23:43.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9bd34c640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b809fea0 con 0x7fe9b8134ef0 2024-01-24T23:23:43.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9bd34c640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b806a580 con 0x7fe9b809c310 2024-01-24T23:23:43.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9bd34c640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:43.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9bd34c640 10 monclient(hunting): authenticate will time out at 1110.827394s 2024-01-24T23:23:43.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9b77fe640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fe9b813b420 legacy=0x7fe9b813d810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:43.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9b6ffd640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7fe9b809c310 legacy=0x7fe9b809c6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:23:43.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9b67fc640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fe9b8134ef0 0x7fe9b8139d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:60474/0 (socket says 172.21.15.132:60474) 2024-01-24T23:23:43.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9b67fc640 1 -- 172.21.15.132:0/1880915935 learned_addr learned my addr 172.21.15.132:0/1880915935 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:43.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9b5ffb640 1 -- 172.21.15.132:0/1880915935 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3851803792 0 0) 0x7fe9b809fea0 con 0x7fe9b8134ef0 2024-01-24T23:23:43.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9b5ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:43.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9b5ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:43.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9b5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:43.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9b5ffb640 10 monclient(hunting): my global_id is 5187 2024-01-24T23:23:43.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.499+0000 7fe9b5ffb640 1 -- 172.21.15.132:0/1880915935 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9a0003530 con 0x7fe9b8134ef0 2024-01-24T23:23:43.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b5ffb640 1 -- 172.21.15.132:0/1880915935 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3791504374 0 0) 0x7fe9a0003530 con 0x7fe9b8134ef0 2024-01-24T23:23:43.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b5ffb640 1 -- 172.21.15.132:0/1880915935 >> v1:172.21.15.115:6791/0 conn(0x7fe9b809c310 legacy=0x7fe9b809c6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:43.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b5ffb640 1 -- 172.21.15.132:0/1880915935 >> v1:172.21.15.132:6789/0 conn(0x7fe9b813b420 legacy=0x7fe9b813d810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:43.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b5ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:23:43.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b5ffb640 1 monclient: found mon.noname-g 2024-01-24T23:23:43.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:43.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b5ffb640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:23:43.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b5ffb640 1 -- 172.21.15.132:0/1880915935 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b809ce00 con 0x7fe9b8134ef0 2024-01-24T23:23:43.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b5ffb640 10 monclient: _finish_auth 0 2024-01-24T23:23:43.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b5ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:23:43.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:43.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b5ffb640 1 -- 172.21.15.132:0/1880915935 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe9ac0017c0 con 0x7fe9b8134ef0 2024-01-24T23:23:43.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 5 monclient: authenticate success, global_id 5187 2024-01-24T23:23:43.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:43.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:43.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b5ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:43.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b5ffb640 10 monclient: dump: 2024-01-24T23:23:43.513 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:43.514 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:43.514 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:43.514 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:43.514 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:43.514 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:43.514 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:43.514 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:43.514 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:43.514 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:43.514 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:43.514 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:43.514 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:43.515 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:43.515 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:43.515 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:43.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:43.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b5ffb640 1 -- 172.21.15.132:0/1880915935 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe9ac001a70 con 0x7fe9b8134ef0 2024-01-24T23:23:43.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b5ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:43.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:43.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b5ffb640 1 -- 172.21.15.132:0/1880915935 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe9ac004bf0 con 0x7fe9b8134ef0 2024-01-24T23:23:43.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:43.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 4 set_mon_vals no callback set 2024-01-24T23:23:43.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b5ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:43.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b5ffb640 10 monclient: dump: 2024-01-24T23:23:43.516 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:43.516 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:43.516 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:43.516 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:43.516 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:43.516 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:43.516 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:43.516 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:43.516 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:43.516 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:43.517 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:43.517 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:43.517 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:43.517 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:43.517 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:43.517 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:43.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 10 monclient: shutdown 2024-01-24T23:23:43.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:43.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 1 -- 172.21.15.132:0/1880915935 >> v1:172.21.15.132:6790/0 conn(0x7fe9b8134ef0 legacy=0x7fe9b8139d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:43.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 1 -- 172.21.15.132:0/1880915935 shutdown_connections 2024-01-24T23:23:43.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 1 -- 172.21.15.132:0/1880915935 >> 172.21.15.132:0/1880915935 conn(0x7fe9b80a2c40 msgr2=0x7fe9b80a3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:43.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 1 -- 172.21.15.132:0/1880915935 shutdown_connections 2024-01-24T23:23:43.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 1 -- 172.21.15.132:0/1880915935 wait complete. 2024-01-24T23:23:43.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 10 monclient: build_initial_monmap 2024-01-24T23:23:43.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 10 monclient: monmap: 2024-01-24T23:23:43.518 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:43.518 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:43.518 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:43.518 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:43.518 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:43.518 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:43.518 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:43.518 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:43.519 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:43.519 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:43.519 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:43.519 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:43.519 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:43.519 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:43.519 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:43.519 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:43.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 1 Processor -- start 2024-01-24T23:23:43.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 1 -- start start 2024-01-24T23:23:43.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 10 monclient: init 2024-01-24T23:23:43.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:43.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 10 monclient: _add_conns ranks=[5,1,3,4,0,6,7,8,2] 2024-01-24T23:23:43.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 10 monclient(hunting): picked mon.noname-h con 0x7fe9b809c310 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:43.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 10 monclient(hunting): picked mon.noname-f con 0x7fe9b8134ef0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:43.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 10 monclient(hunting): picked mon.noname-g con 0x7fe9b813b420 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:43.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b81cc3e0 con 0x7fe9b8134ef0 2024-01-24T23:23:43.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b813a430 con 0x7fe9b813b420 2024-01-24T23:23:43.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b80a00d0 con 0x7fe9b809c310 2024-01-24T23:23:43.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:43.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 10 monclient(hunting): authenticate will time out at 1110.830154s 2024-01-24T23:23:43.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b67fc640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fe9b8134ef0 legacy=0x7fe9b81cbcd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:43.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b77fe640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fe9b813b420 0x7fe9b81d3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:60486/0 (socket says 172.21.15.132:60486) 2024-01-24T23:23:43.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b77fe640 1 -- 172.21.15.132:0/3765327696 learned_addr learned my addr 172.21.15.132:0/3765327696 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:43.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b6ffd640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fe9b809c310 0x7fe9b81cb5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:57666/0 (socket says 172.21.15.132:57666) 2024-01-24T23:23:43.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 1 -- 172.21.15.132:0/3765327696 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3825564500 0 0) 0x7fe9b80a00d0 con 0x7fe9b809c310 2024-01-24T23:23:43.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:43.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:43.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:43.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 10 monclient(hunting): my global_id is 5171 2024-01-24T23:23:43.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 1 -- 172.21.15.132:0/3765327696 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9a4003530 con 0x7fe9b809c310 2024-01-24T23:23:43.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 1 -- 172.21.15.132:0/3765327696 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 986762995 0 0) 0x7fe9b813a430 con 0x7fe9b813b420 2024-01-24T23:23:43.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:43.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:43.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:43.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 10 monclient(hunting): my global_id is 5196 2024-01-24T23:23:43.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 1 -- 172.21.15.132:0/3765327696 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9b80a00d0 con 0x7fe9b813b420 2024-01-24T23:23:43.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 1 -- 172.21.15.132:0/3765327696 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2808356264 0 0) 0x7fe9b80a00d0 con 0x7fe9b813b420 2024-01-24T23:23:43.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 1 -- 172.21.15.132:0/3765327696 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe9b813a430 con 0x7fe9b813b420 2024-01-24T23:23:43.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 1 -- 172.21.15.132:0/3765327696 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe9b0004520 con 0x7fe9b813b420 2024-01-24T23:23:43.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:43.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:43.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 10 monclient(hunting): dump: 2024-01-24T23:23:43.522 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:43.522 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:43.522 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:43.522 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:43.522 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:43.522 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:43.523 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:43.523 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:43.523 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:43.523 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:43.523 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:43.523 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:43.523 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:43.523 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:43.523 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:43.523 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:43.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:43.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 1 -- 172.21.15.132:0/3765327696 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2307737081 0 0) 0x7fe9a4003530 con 0x7fe9b809c310 2024-01-24T23:23:43.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 1 -- 172.21.15.132:0/3765327696 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe9b80a00d0 con 0x7fe9b809c310 2024-01-24T23:23:43.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 1 -- 172.21.15.132:0/3765327696 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe9a8004270 con 0x7fe9b809c310 2024-01-24T23:23:43.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:43.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:43.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 10 monclient(hunting): dump: 2024-01-24T23:23:43.524 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:43.524 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:43.524 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:43.524 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:43.524 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:43.524 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:43.524 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:43.525 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:43.525 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:43.525 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:43.525 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:43.525 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:43.525 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:43.525 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:43.525 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:43.525 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:43.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 1 -- 172.21.15.132:0/3765327696 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3055566369 0 0) 0x7fe9b813a430 con 0x7fe9b813b420 2024-01-24T23:23:43.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 1 -- 172.21.15.132:0/3765327696 >> v1:172.21.15.132:6791/0 conn(0x7fe9b809c310 legacy=0x7fe9b81cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:43.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 1 -- 172.21.15.132:0/3765327696 >> v1:172.21.15.132:6789/0 conn(0x7fe9b8134ef0 legacy=0x7fe9b81cbcd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:43.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:23:43.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 1 monclient: found mon.g 2024-01-24T23:23:43.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:43.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:43.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 1 -- 172.21.15.132:0/3765327696 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b8139d70 con 0x7fe9b813b420 2024-01-24T23:23:43.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 10 monclient: _finish_auth 0 2024-01-24T23:23:43.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:23:43.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:43.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 5 monclient: authenticate success, global_id 5196 2024-01-24T23:23:43.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 10 monclient: _renew_subs 2024-01-24T23:23:43.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:43.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 1 -- 172.21.15.132:0/3765327696 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b8139910 con 0x7fe9b813b420 2024-01-24T23:23:43.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 10 monclient: _renew_subs 2024-01-24T23:23:43.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:43.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 1 -- 172.21.15.132:0/3765327696 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b809e760 con 0x7fe9b813b420 2024-01-24T23:23:43.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 1 -- 172.21.15.132:0/3765327696 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe9b0004a80 con 0x7fe9b813b420 2024-01-24T23:23:43.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:43.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 1 -- 172.21.15.132:0/3765327696 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe9b0004fb0 con 0x7fe9b813b420 2024-01-24T23:23:43.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:43.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9b57fa640 4 set_mon_vals no callback set 2024-01-24T23:23:43.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:43.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 10 monclient: dump: 2024-01-24T23:23:43.527 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:43.528 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:43.528 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:43.528 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:43.528 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:43.528 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:43.528 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:43.528 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:43.528 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:43.528 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:43.528 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:43.529 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:43.529 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:43.529 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:43.529 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:43.529 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:43.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe99f7fe640 1 -- 172.21.15.132:0/3765327696 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fe9b0005110 con 0x7fe9b813b420 2024-01-24T23:23:43.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:43.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:43.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:43.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.503+0000 7fe9bd34c640 1 -- 172.21.15.132:0/3765327696 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe998005090 con 0x7fe9b813b420 2024-01-24T23:23:43.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.507+0000 7fe99f7fe640 1 -- 172.21.15.132:0/3765327696 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(56..56 src has 1..56) v4 ==== 4310+0+0 (unknown 701232376 0 0) 0x7fe9b0043c30 con 0x7fe9b813b420 2024-01-24T23:23:43.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.507+0000 7fe9b6ffd640 10 monclient: get_auth_request con 0x7fe9a4037660 auth_method 0 2024-01-24T23:23:43.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.511+0000 7fe99f7fe640 1 -- 172.21.15.132:0/3765327696 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fe9b0006730 con 0x7fe9b813b420 2024-01-24T23:23:43.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.511+0000 7fe99f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:43.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.511+0000 7fe99f7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:43.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.667+0000 7fe9bd34c640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}] 2024-01-24T23:23:43.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.667+0000 7fe9bd34c640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}] 2024-01-24T23:23:43.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.667+0000 7fe9bd34c640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:43.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.667+0000 7fe9bd34c640 1 -- 172.21.15.132:0/3765327696 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"} v 0) v1 -- 0x7fe9980058c0 con 0x7fe9b813b420 2024-01-24T23:23:43.886 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.883+0000 7fe99f7fe640 1 -- 172.21.15.132:0/3765327696 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}]=0 v57) v1 ==== 110+0+0 (unknown 1868979467 0 0) 0x7fe9b00130f0 con 0x7fe9b813b420 2024-01-24T23:23:43.886 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.883+0000 7fe99f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}] 2024-01-24T23:23:43.887 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.883+0000 7fe99f7fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:43.989 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.987+0000 7fe9bd34c640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}] 2024-01-24T23:23:43.989 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.987+0000 7fe9bd34c640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}] 2024-01-24T23:23:43.989 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.987+0000 7fe9bd34c640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:43.989 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.987+0000 7fe9bd34c640 1 -- 172.21.15.132:0/3765327696 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"} v 0) v1 -- 0x7fe998002220 con 0x7fe9b813b420 2024-01-24T23:23:43.989 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.987+0000 7fe99f7fe640 1 -- 172.21.15.132:0/3765327696 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}]=0 already renamed to 'bar' v57) v1 ==== 134+0+0 (unknown 852271798 0 0) 0x7fe9b0014870 con 0x7fe9b813b420 2024-01-24T23:23:43.990 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.987+0000 7fe99f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}] 2024-01-24T23:23:43.990 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.987+0000 7fe99f7fe640 10 monclient: _finish_command 3 = system:0 already renamed to 'bar' 2024-01-24T23:23:43.990 INFO:tasks.workunit.client.0.smithi132.stderr:already renamed to 'bar' 2024-01-24T23:23:43.991 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.991+0000 7fe9bd34c640 1 -- 172.21.15.132:0/3765327696 >> v1:172.21.15.132:6812/1835401922 conn(0x7fe9a4037660 legacy=0x7fe9a4039b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:43.992 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.991+0000 7fe9bd34c640 10 monclient: shutdown 2024-01-24T23:23:43.992 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.991+0000 7fe9bd34c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:43.992 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.991+0000 7fe9bd34c640 1 -- 172.21.15.132:0/3765327696 >> v1:172.21.15.132:6790/0 conn(0x7fe9b813b420 legacy=0x7fe9b81d3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:43.992 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.991+0000 7fe9bd34c640 1 -- 172.21.15.132:0/3765327696 shutdown_connections 2024-01-24T23:23:43.992 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.991+0000 7fe9bd34c640 1 -- 172.21.15.132:0/3765327696 >> 172.21.15.132:0/3765327696 conn(0x7fe9b80a2c40 msgr2=0x7fe9b8096100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:43.992 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.991+0000 7fe9bd34c640 1 -- 172.21.15.132:0/3765327696 shutdown_connections 2024-01-24T23:23:43.992 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:43.991+0000 7fe9bd34c640 1 -- 172.21.15.132:0/3765327696 wait complete. 2024-01-24T23:23:44.007 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule ls 2024-01-24T23:23:44.007 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false grep foo-asdf 2024-01-24T23:23:44.007 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:23:44.007 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep foo-asdf 2024-01-24T23:23:44.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 10 monclient: get_monmap_and_config 2024-01-24T23:23:44.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 10 monclient: build_initial_monmap 2024-01-24T23:23:44.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 10 monclient: monmap: 2024-01-24T23:23:44.124 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:44.125 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:44.125 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:44.125701+0000 2024-01-24T23:23:44.125 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:44.125701+0000 2024-01-24T23:23:44.125 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:44.125 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:44.125 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:44.125 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:44.125 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:44.125 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:44.125 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:44.125 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:44.125 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:44.125 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:44.125 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:44.126 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:44.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 1 Processor -- start 2024-01-24T23:23:44.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 1 -- start start 2024-01-24T23:23:44.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 10 monclient: init 2024-01-24T23:23:44.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:44.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 10 monclient: _add_conns ranks=[2,5,0,4,8,3,7,6,1] 2024-01-24T23:23:44.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 10 monclient(hunting): picked mon.noname-b con 0x7f2034136e50 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:44.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 10 monclient(hunting): picked mon.noname-h con 0x7f20340a0290 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:44.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 10 monclient(hunting): picked mon.noname-a con 0x7f20340a1600 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:44.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2034064700 con 0x7f20340a1600 2024-01-24T23:23:44.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2034137940 con 0x7f2034136e50 2024-01-24T23:23:44.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203409f9c0 con 0x7f20340a0290 2024-01-24T23:23:44.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:44.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 10 monclient(hunting): authenticate will time out at 1111.450275s 2024-01-24T23:23:44.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203a650640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f20340a1600 legacy=0x7f203413f490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:44.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203964e640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f20340a0290 0x7f203413bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:44094/0 (socket says 172.21.15.132:44094) 2024-01-24T23:23:44.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203964e640 1 -- 172.21.15.132:0/2102952300 learned_addr learned my addr 172.21.15.132:0/2102952300 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:44.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 1 -- 172.21.15.132:0/2102952300 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1269025304 0 0) 0x7f203409f9c0 con 0x7f20340a0290 2024-01-24T23:23:44.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 10 monclient(hunting): _negotiate 2024-01-24T23:23:44.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:44.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:44.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 10 monclient(hunting): my global_id is 5180 2024-01-24T23:23:44.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 1 -- 172.21.15.132:0/2102952300 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2020003530 con 0x7f20340a0290 2024-01-24T23:23:44.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 1 -- 172.21.15.132:0/2102952300 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2236361948 0 0) 0x7f2020003530 con 0x7f20340a0290 2024-01-24T23:23:44.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 1 -- 172.21.15.132:0/2102952300 >> v1:172.21.15.115:6790/0 conn(0x7f2034136e50 legacy=0x7f2034137230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:44.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 1 -- 172.21.15.132:0/2102952300 >> v1:172.21.15.115:6789/0 conn(0x7f20340a1600 legacy=0x7f203413f490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:44.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 10 monclient: _finish_hunting 0 2024-01-24T23:23:44.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 1 monclient: found mon.noname-h 2024-01-24T23:23:44.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:44.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:23:44.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 1 -- 172.21.15.132:0/2102952300 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203409eda0 con 0x7f20340a0290 2024-01-24T23:23:44.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 10 monclient: _finish_auth 0 2024-01-24T23:23:44.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 10 monclient: _check_auth_tickets 2024-01-24T23:23:44.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:44.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 1 -- 172.21.15.132:0/2102952300 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f20300017c0 con 0x7f20340a0290 2024-01-24T23:23:44.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 5 monclient: authenticate success, global_id 5180 2024-01-24T23:23:44.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:44.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:44.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:44.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 10 monclient: dump: 2024-01-24T23:23:44.136 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:44.136 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:44.136 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.136 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.136 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:44.136 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:44.136 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:44.136 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:44.136 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:44.136 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:44.136 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:44.136 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:44.136 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:44.137 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:44.137 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:44.137 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:44.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:44.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 1 -- 172.21.15.132:0/2102952300 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2030001ae0 con 0x7f20340a0290 2024-01-24T23:23:44.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:44.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:44.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 4 set_mon_vals no callback set 2024-01-24T23:23:44.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 1 -- 172.21.15.132:0/2102952300 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2030004bf0 con 0x7f20340a0290 2024-01-24T23:23:44.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:44.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:44.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f2038e4d640 10 monclient: dump: 2024-01-24T23:23:44.137 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:44.138 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:44.138 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.138 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.138 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:44.138 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:44.138 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:44.138 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:44.138 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:44.138 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:44.138 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:44.138 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:44.138 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:44.138 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:44.138 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:44.139 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:44.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 10 monclient: shutdown 2024-01-24T23:23:44.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:44.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 1 -- 172.21.15.132:0/2102952300 >> v1:172.21.15.132:6791/0 conn(0x7f20340a0290 legacy=0x7f203413bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:44.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 1 -- 172.21.15.132:0/2102952300 shutdown_connections 2024-01-24T23:23:44.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 1 -- 172.21.15.132:0/2102952300 >> 172.21.15.132:0/2102952300 conn(0x7f20340a5a10 msgr2=0x7f20340a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:44.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 1 -- 172.21.15.132:0/2102952300 shutdown_connections 2024-01-24T23:23:44.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.123+0000 7f203c0b3640 1 -- 172.21.15.132:0/2102952300 wait complete. 2024-01-24T23:23:44.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 10 monclient: build_initial_monmap 2024-01-24T23:23:44.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 10 monclient: monmap: 2024-01-24T23:23:44.139 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:44.139 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:44.139 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:44.139 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:44.139 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:44.140 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:44.140 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:44.140 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:44.140 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:44.140 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:44.140 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:44.140 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:44.140 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:44.140 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:44.140 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:44.140 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:44.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 1 Processor -- start 2024-01-24T23:23:44.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 1 -- start start 2024-01-24T23:23:44.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 10 monclient: init 2024-01-24T23:23:44.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:44.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 10 monclient: _add_conns ranks=[2,6,0,8,7,5,4,1,3] 2024-01-24T23:23:44.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 10 monclient(hunting): picked mon.noname-b con 0x7f20340a0290 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:44.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 10 monclient(hunting): picked mon.noname-d con 0x7f20340a1600 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:44.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 10 monclient(hunting): picked mon.noname-a con 0x7f2034136e50 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:44.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203411f010 con 0x7f2034136e50 2024-01-24T23:23:44.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2034136c20 con 0x7f20340a0290 2024-01-24T23:23:44.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203409ea70 con 0x7f20340a1600 2024-01-24T23:23:44.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:44.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 10 monclient(hunting): authenticate will time out at 1111.453280s 2024-01-24T23:23:44.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203a650640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f2034136e50 legacy=0x7f20341d3b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:44.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f2039e4f640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f20340a0290 0x7f2034131b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:40172/0 (socket says 172.21.15.132:40172) 2024-01-24T23:23:44.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f2039e4f640 1 -- 172.21.15.132:0/1960757636 learned_addr learned my addr 172.21.15.132:0/1960757636 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:44.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 1 -- 172.21.15.132:0/1960757636 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3721414687 0 0) 0x7f2034136c20 con 0x7f20340a0290 2024-01-24T23:23:44.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:44.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:44.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:44.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 10 monclient(hunting): my global_id is 84485 2024-01-24T23:23:44.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 1 -- 172.21.15.132:0/1960757636 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2018003530 con 0x7f20340a0290 2024-01-24T23:23:44.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 1 -- 172.21.15.132:0/1960757636 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2649178599 0 0) 0x7f203409ea70 con 0x7f20340a1600 2024-01-24T23:23:44.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:44.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:44.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:44.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 10 monclient(hunting): my global_id is 5262 2024-01-24T23:23:44.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 1 -- 172.21.15.132:0/1960757636 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2034136c20 con 0x7f20340a1600 2024-01-24T23:23:44.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 1 -- 172.21.15.132:0/1960757636 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 133267195 0 0) 0x7f2018003530 con 0x7f20340a0290 2024-01-24T23:23:44.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 1 -- 172.21.15.132:0/1960757636 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f203409ea70 con 0x7f20340a0290 2024-01-24T23:23:44.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 1 -- 172.21.15.132:0/1960757636 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2024001a40 con 0x7f20340a0290 2024-01-24T23:23:44.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:44.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:23:44.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 10 monclient(hunting): dump: 2024-01-24T23:23:44.142 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:44.143 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:44.143 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.143 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.143 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:44.143 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:44.143 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:44.143 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:44.143 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:44.143 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:44.143 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:44.143 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:44.143 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:44.143 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:44.143 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:44.143 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:44.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:44.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 1 -- 172.21.15.132:0/1960757636 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 948480290 0 0) 0x7f2034136c20 con 0x7f20340a1600 2024-01-24T23:23:44.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 1 -- 172.21.15.132:0/1960757636 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2018003530 con 0x7f20340a1600 2024-01-24T23:23:44.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 1 -- 172.21.15.132:0/1960757636 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2030001780 con 0x7f20340a1600 2024-01-24T23:23:44.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:44.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:44.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 10 monclient(hunting): dump: 2024-01-24T23:23:44.144 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:44.144 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:44.144 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.144 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.144 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:44.144 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:44.144 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:44.144 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:44.144 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:44.145 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:44.145 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:44.145 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:44.145 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:44.145 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:44.145 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:44.145 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:44.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 1 -- 172.21.15.132:0/1960757636 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3785502195 0 0) 0x7f203409ea70 con 0x7f20340a0290 2024-01-24T23:23:44.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 1 -- 172.21.15.132:0/1960757636 >> v1:172.21.15.115:6792/0 conn(0x7f20340a1600 legacy=0x7f20341d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:44.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 1 -- 172.21.15.132:0/1960757636 >> v1:172.21.15.115:6789/0 conn(0x7f2034136e50 legacy=0x7f20341d3b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:44.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:23:44.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 1 monclient: found mon.b 2024-01-24T23:23:44.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:44.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:44.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 1 -- 172.21.15.132:0/1960757636 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203411d000 con 0x7f20340a0290 2024-01-24T23:23:44.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 10 monclient: _finish_auth 0 2024-01-24T23:23:44.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:23:44.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:44.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 5 monclient: authenticate success, global_id 84485 2024-01-24T23:23:44.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 10 monclient: _renew_subs 2024-01-24T23:23:44.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:44.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 1 -- 172.21.15.132:0/1960757636 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203411f2f0 con 0x7f20340a0290 2024-01-24T23:23:44.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 10 monclient: _renew_subs 2024-01-24T23:23:44.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:44.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 1 -- 172.21.15.132:0/1960757636 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2034132440 con 0x7f20340a0290 2024-01-24T23:23:44.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 1 -- 172.21.15.132:0/1960757636 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2024001cb0 con 0x7f20340a0290 2024-01-24T23:23:44.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:44.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202bfff640 4 set_mon_vals no callback set 2024-01-24T23:23:44.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 1 -- 172.21.15.132:0/1960757636 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2024003c10 con 0x7f20340a0290 2024-01-24T23:23:44.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:44.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:44.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f202a7fc640 10 monclient: dump: 2024-01-24T23:23:44.147 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:44.147 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:44.147 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.147 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.147 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:44.147 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:44.147 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:44.147 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:44.147 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:44.147 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:44.147 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:44.147 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:44.147 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:44.147 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:44.147 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:44.147 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:44.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:44.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:44.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:44.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.127+0000 7f203c0b3640 1 -- 172.21.15.132:0/1960757636 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f201c005090 con 0x7f20340a0290 2024-01-24T23:23:44.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.131+0000 7f202a7fc640 1 -- 172.21.15.132:0/1960757636 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f2024003df0 con 0x7f20340a0290 2024-01-24T23:23:44.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.131+0000 7f202a7fc640 1 -- 172.21.15.132:0/1960757636 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(57..57 src has 1..57) v4 ==== 4305+0+0 (unknown 2895024988 0 0) 0x7f2024043930 con 0x7f20340a0290 2024-01-24T23:23:44.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.131+0000 7f203964e640 10 monclient: get_auth_request con 0x7f20180376a0 auth_method 0 2024-01-24T23:23:44.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.143+0000 7f202a7fc640 1 -- 172.21.15.132:0/1960757636 <== mon.2 v1:172.21.15.115:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f2024014750 con 0x7f20340a0290 2024-01-24T23:23:44.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.143+0000 7f202a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:44.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.143+0000 7f202a7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:44.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.299+0000 7f203c0b3640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:44.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.299+0000 7f203c0b3640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:44.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.299+0000 7f203c0b3640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:44.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.299+0000 7f203c0b3640 1 -- 172.21.15.132:0/1960757636 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd crush rule ls"} v 0) v1 -- 0x7f201c005340 con 0x7f20340a0290 2024-01-24T23:23:44.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.299+0000 7f202a7fc640 1 -- 172.21.15.132:0/1960757636 <== mon.2 v1:172.21.15.115:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v57) v1 ==== 65+0+60 (unknown 2262488454 0 1596542489) 0x7f2024014570 con 0x7f20340a0290 2024-01-24T23:23:44.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.303+0000 7f202a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:44.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.303+0000 7f202a7fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:44.399 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.395+0000 7f203c0b3640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:44.399 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.395+0000 7f203c0b3640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:44.399 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.395+0000 7f203c0b3640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:44.399 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.395+0000 7f203c0b3640 1 -- 172.21.15.132:0/1960757636 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd crush rule ls"} v 0) v1 -- 0x7f201c001130 con 0x7f20340a0290 2024-01-24T23:23:44.400 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.399+0000 7f202a7fc640 1 -- 172.21.15.132:0/1960757636 <== mon.2 v1:172.21.15.115:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v57) v1 ==== 65+0+60 (unknown 2262488454 0 1596542489) 0x7f20240152b0 con 0x7f20340a0290 2024-01-24T23:23:44.400 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.399+0000 7f202a7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:44.400 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.399+0000 7f202a7fc640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:23:44.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.399+0000 7f203c0b3640 1 -- 172.21.15.132:0/1960757636 >> v1:172.21.15.132:6812/1835401922 conn(0x7f20180376a0 legacy=0x7f2018039b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:44.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.399+0000 7f203c0b3640 10 monclient: shutdown 2024-01-24T23:23:44.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.399+0000 7f203c0b3640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:44.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.399+0000 7f203c0b3640 1 -- 172.21.15.132:0/1960757636 >> v1:172.21.15.115:6790/0 conn(0x7f20340a0290 legacy=0x7f2034131b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:44.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.399+0000 7f203c0b3640 1 -- 172.21.15.132:0/1960757636 shutdown_connections 2024-01-24T23:23:44.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.399+0000 7f203c0b3640 1 -- 172.21.15.132:0/1960757636 >> 172.21.15.132:0/1960757636 conn(0x7f20340a5a10 msgr2=0x7f20341be9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:44.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.399+0000 7f203c0b3640 1 -- 172.21.15.132:0/1960757636 shutdown_connections 2024-01-24T23:23:44.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.399+0000 7f203c0b3640 1 -- 172.21.15.132:0/1960757636 wait complete. 2024-01-24T23:23:44.417 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:23:44.417 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule ls 2024-01-24T23:23:44.417 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false grep bar-asdf 2024-01-24T23:23:44.418 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:23:44.418 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep bar-asdf 2024-01-24T23:23:44.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.523+0000 7f2f3b8b0640 10 monclient: get_monmap_and_config 2024-01-24T23:23:44.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.523+0000 7f2f3b8b0640 10 monclient: build_initial_monmap 2024-01-24T23:23:44.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.523+0000 7f2f3b8b0640 10 monclient: monmap: 2024-01-24T23:23:44.527 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:44.527 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:44.527 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:44.527828+0000 2024-01-24T23:23:44.527 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:44.527828+0000 2024-01-24T23:23:44.527 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:44.527 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:44.527 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:44.527 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:44.527 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:44.527 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:44.527 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:44.527 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:44.527 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:44.527 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:44.528 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:44.528 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:44.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.523+0000 7f2f3b8b0640 1 Processor -- start 2024-01-24T23:23:44.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.523+0000 7f2f3b8b0640 1 -- start start 2024-01-24T23:23:44.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.523+0000 7f2f3b8b0640 10 monclient: init 2024-01-24T23:23:44.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:44.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 10 monclient: _add_conns ranks=[1,7,8,6,0,5,2,3,4] 2024-01-24T23:23:44.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 10 monclient(hunting): picked mon.noname-f con 0x7f2f3409ff20 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:44.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 10 monclient(hunting): picked mon.noname-i con 0x7f2f3409f9c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:44.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 10 monclient(hunting): picked mon.noname-e con 0x7f2f3413b2c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:44.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f34064700 con 0x7f2f3409ff20 2024-01-24T23:23:44.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3964c640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f2f3409ff20 legacy=0x7f2f340a0300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:44.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3409e760 con 0x7f2f3409f9c0 2024-01-24T23:23:44.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f340a1160 con 0x7f2f3413b2c0 2024-01-24T23:23:44.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:44.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 10 monclient(hunting): authenticate will time out at 1111.852543s 2024-01-24T23:23:44.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f38e4b640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f2f3409f9c0 0x7f2f34139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:48916/0 (socket says 172.21.15.132:48916) 2024-01-24T23:23:44.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f38e4b640 1 -- 172.21.15.132:0/445048689 learned_addr learned my addr 172.21.15.132:0/445048689 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:44.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 1 -- 172.21.15.132:0/445048689 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3093452537 0 0) 0x7f2f3409e760 con 0x7f2f3409f9c0 2024-01-24T23:23:44.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:44.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:44.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:44.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 10 monclient(hunting): my global_id is 5281 2024-01-24T23:23:44.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 1 -- 172.21.15.132:0/445048689 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f20003530 con 0x7f2f3409f9c0 2024-01-24T23:23:44.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 1 -- 172.21.15.132:0/445048689 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1439314035 0 0) 0x7f2f20003530 con 0x7f2f3409f9c0 2024-01-24T23:23:44.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 1 -- 172.21.15.132:0/445048689 >> v1:172.21.15.115:6793/0 conn(0x7f2f3413b2c0 legacy=0x7f2f3413d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:44.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 1 -- 172.21.15.132:0/445048689 >> v1:172.21.15.132:6789/0 conn(0x7f2f3409ff20 legacy=0x7f2f340a0300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:44.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:23:44.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 1 monclient: found mon.noname-i 2024-01-24T23:23:44.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:44.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:23:44.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 1 -- 172.21.15.132:0/445048689 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f340a0a10 con 0x7f2f3409f9c0 2024-01-24T23:23:44.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 10 monclient: _finish_auth 0 2024-01-24T23:23:44.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:23:44.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:44.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 1 -- 172.21.15.132:0/445048689 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2f240017c0 con 0x7f2f3409f9c0 2024-01-24T23:23:44.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 5 monclient: authenticate success, global_id 5281 2024-01-24T23:23:44.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:44.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:44.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:23:44.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 10 monclient: dump: 2024-01-24T23:23:44.537 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:44.537 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:44.537 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.537 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.537 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:44.537 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:44.537 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:44.537 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:44.537 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:44.537 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:44.537 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:44.537 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:44.537 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:44.538 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:44.538 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:44.538 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:44.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:44.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 1 -- 172.21.15.132:0/445048689 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2f24001ae0 con 0x7f2f3409f9c0 2024-01-24T23:23:44.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:44.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:44.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 4 set_mon_vals no callback set 2024-01-24T23:23:44.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 1 -- 172.21.15.132:0/445048689 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2f24004bf0 con 0x7f2f3409f9c0 2024-01-24T23:23:44.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:44.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:44.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f2bfff640 10 monclient: dump: 2024-01-24T23:23:44.538 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:44.538 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:44.538 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.539 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.539 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:44.539 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:44.539 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:44.539 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:44.539 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:44.539 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:44.539 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:44.539 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:44.539 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:44.539 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:44.539 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:44.539 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:44.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 10 monclient: shutdown 2024-01-24T23:23:44.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:44.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 1 -- 172.21.15.132:0/445048689 >> v1:172.21.15.132:6792/0 conn(0x7f2f3409f9c0 legacy=0x7f2f34139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:44.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 1 -- 172.21.15.132:0/445048689 shutdown_connections 2024-01-24T23:23:44.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 1 -- 172.21.15.132:0/445048689 >> 172.21.15.132:0/445048689 conn(0x7f2f340a5a10 msgr2=0x7f2f340a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:44.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 1 -- 172.21.15.132:0/445048689 shutdown_connections 2024-01-24T23:23:44.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 1 -- 172.21.15.132:0/445048689 wait complete. 2024-01-24T23:23:44.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 10 monclient: build_initial_monmap 2024-01-24T23:23:44.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 10 monclient: monmap: 2024-01-24T23:23:44.540 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:44.540 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:44.540 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:44.540 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:44.540 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:44.540 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:44.540 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:44.541 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:44.541 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:44.541 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:44.541 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:44.541 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:44.541 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:44.541 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:44.541 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:44.541 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:44.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 1 Processor -- start 2024-01-24T23:23:44.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 1 -- start start 2024-01-24T23:23:44.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 10 monclient: init 2024-01-24T23:23:44.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:44.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 10 monclient: _add_conns ranks=[6,1,3,5,2,0,8,4,7] 2024-01-24T23:23:44.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 10 monclient(hunting): picked mon.noname-d con 0x7f2f3409f9c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:44.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 10 monclient(hunting): picked mon.noname-f con 0x7f2f3409ff20 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:44.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 10 monclient(hunting): picked mon.noname-g con 0x7f2f3413b2c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:44.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f34071960 con 0x7f2f3409ff20 2024-01-24T23:23:44.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3411d5e0 con 0x7f2f3413b2c0 2024-01-24T23:23:44.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3411d8c0 con 0x7f2f3409f9c0 2024-01-24T23:23:44.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:44.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 10 monclient(hunting): authenticate will time out at 1111.855133s 2024-01-24T23:23:44.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f38e4b640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f2f3409ff20 legacy=0x7f2f341ce680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:44.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f39e4d640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f2f3413b2c0 0x7f2f341d1db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:52828/0 (socket says 172.21.15.132:52828) 2024-01-24T23:23:44.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f39e4d640 1 -- 172.21.15.132:0/1096698870 learned_addr learned my addr 172.21.15.132:0/1096698870 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:44.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f29ffb640 1 -- 172.21.15.132:0/1096698870 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3261434454 0 0) 0x7f2f3411d5e0 con 0x7f2f3413b2c0 2024-01-24T23:23:44.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f29ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:44.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f29ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:44.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f29ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:44.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f29ffb640 10 monclient(hunting): my global_id is 5205 2024-01-24T23:23:44.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f29ffb640 1 -- 172.21.15.132:0/1096698870 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f18003560 con 0x7f2f3413b2c0 2024-01-24T23:23:44.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f29ffb640 1 -- 172.21.15.132:0/1096698870 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3438060417 0 0) 0x7f2f18003560 con 0x7f2f3413b2c0 2024-01-24T23:23:44.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f29ffb640 1 -- 172.21.15.132:0/1096698870 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2f3411d5e0 con 0x7f2f3413b2c0 2024-01-24T23:23:44.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f29ffb640 1 -- 172.21.15.132:0/1096698870 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2f2c001d40 con 0x7f2f3413b2c0 2024-01-24T23:23:44.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f29ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:44.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f29ffb640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:44.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f29ffb640 10 monclient(hunting): dump: 2024-01-24T23:23:44.543 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:44.544 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:44.544 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.544 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.544 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:44.544 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:44.544 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:44.544 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:44.544 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:44.544 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:44.544 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:44.544 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:44.544 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:44.544 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:44.544 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:44.544 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:44.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f29ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:44.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f29ffb640 1 -- 172.21.15.132:0/1096698870 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2068828884 0 0) 0x7f2f3411d5e0 con 0x7f2f3413b2c0 2024-01-24T23:23:44.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f29ffb640 1 -- 172.21.15.132:0/1096698870 >> v1:172.21.15.115:6792/0 conn(0x7f2f3409f9c0 legacy=0x7f2f341c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:44.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f29ffb640 1 -- 172.21.15.132:0/1096698870 >> v1:172.21.15.132:6789/0 conn(0x7f2f3409ff20 legacy=0x7f2f341ce680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:44.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f29ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:23:44.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f29ffb640 1 monclient: found mon.g 2024-01-24T23:23:44.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f29ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:44.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f29ffb640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:44.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f29ffb640 1 -- 172.21.15.132:0/1096698870 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f34071680 con 0x7f2f3413b2c0 2024-01-24T23:23:44.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f29ffb640 10 monclient: _finish_auth 0 2024-01-24T23:23:44.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f29ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:23:44.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f29ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:44.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 5 monclient: authenticate success, global_id 5205 2024-01-24T23:23:44.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 10 monclient: _renew_subs 2024-01-24T23:23:44.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:44.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 1 -- 172.21.15.132:0/1096698870 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3411b840 con 0x7f2f3413b2c0 2024-01-24T23:23:44.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.527+0000 7f2f3b8b0640 10 monclient: _renew_subs 2024-01-24T23:23:44.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.531+0000 7f2f3b8b0640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:44.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.531+0000 7f2f3b8b0640 1 -- 172.21.15.132:0/1096698870 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3411c120 con 0x7f2f3413b2c0 2024-01-24T23:23:44.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.531+0000 7f2f29ffb640 1 -- 172.21.15.132:0/1096698870 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2f2c0020c0 con 0x7f2f3413b2c0 2024-01-24T23:23:44.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.531+0000 7f2f29ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:44.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.531+0000 7f2f29ffb640 1 -- 172.21.15.132:0/1096698870 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2f2c003f90 con 0x7f2f3413b2c0 2024-01-24T23:23:44.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.531+0000 7f2f29ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:44.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.531+0000 7f2f2b7fe640 4 set_mon_vals no callback set 2024-01-24T23:23:44.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.531+0000 7f2f29ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:44.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.531+0000 7f2f29ffb640 10 monclient: dump: 2024-01-24T23:23:44.546 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:44.546 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:44.546 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.546 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.546 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:44.546 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:44.547 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:44.547 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:44.547 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:44.547 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:44.547 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:44.547 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:44.547 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:44.547 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:44.547 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:44.547 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:44.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.531+0000 7f2f3b8b0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:44.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.531+0000 7f2f3b8b0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:44.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.531+0000 7f2f3b8b0640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:44.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.531+0000 7f2f3b8b0640 1 -- 172.21.15.132:0/1096698870 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f14005090 con 0x7f2f3413b2c0 2024-01-24T23:23:44.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.531+0000 7f2f29ffb640 1 -- 172.21.15.132:0/1096698870 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f2f2c004170 con 0x7f2f3413b2c0 2024-01-24T23:23:44.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.531+0000 7f2f29ffb640 1 -- 172.21.15.132:0/1096698870 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(57..57 src has 1..57) v4 ==== 4305+0+0 (unknown 2895024988 0 0) 0x7f2f2c043d50 con 0x7f2f3413b2c0 2024-01-24T23:23:44.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.531+0000 7f2f3964c640 10 monclient: get_auth_request con 0x7f2f18036f30 auth_method 0 2024-01-24T23:23:44.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.535+0000 7f2f29ffb640 1 -- 172.21.15.132:0/1096698870 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f2f2c014bb0 con 0x7f2f3413b2c0 2024-01-24T23:23:44.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.535+0000 7f2f29ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:44.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.535+0000 7f2f29ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:44.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.679+0000 7f2f3b8b0640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:44.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.679+0000 7f2f3b8b0640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:44.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.679+0000 7f2f3b8b0640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:44.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.679+0000 7f2f3b8b0640 1 -- 172.21.15.132:0/1096698870 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rule ls"} v 0) v1 -- 0x7f2f140058c0 con 0x7f2f3413b2c0 2024-01-24T23:23:44.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.683+0000 7f2f29ffb640 1 -- 172.21.15.132:0/1096698870 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v57) v1 ==== 65+0+60 (unknown 2262488454 0 1596542489) 0x7f2f2c013250 con 0x7f2f3413b2c0 2024-01-24T23:23:44.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.683+0000 7f2f29ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:44.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.683+0000 7f2f29ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:44.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.767+0000 7f2f3b8b0640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:44.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.767+0000 7f2f3b8b0640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:44.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.767+0000 7f2f3b8b0640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:44.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.767+0000 7f2f3b8b0640 1 -- 172.21.15.132:0/1096698870 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rule ls"} v 0) v1 -- 0x7f2f14000bd0 con 0x7f2f3413b2c0 2024-01-24T23:23:44.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.771+0000 7f2f29ffb640 1 -- 172.21.15.132:0/1096698870 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v57) v1 ==== 65+0+60 (unknown 2262488454 0 1596542489) 0x7f2f2c0149d0 con 0x7f2f3413b2c0 2024-01-24T23:23:44.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.771+0000 7f2f29ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls"}] 2024-01-24T23:23:44.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.771+0000 7f2f29ffb640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:23:44.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.771+0000 7f2f3b8b0640 1 -- 172.21.15.132:0/1096698870 >> v1:172.21.15.132:6812/1835401922 conn(0x7f2f18036f30 legacy=0x7f2f180393f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:44.774 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.771+0000 7f2f3b8b0640 10 monclient: shutdown 2024-01-24T23:23:44.774 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.771+0000 7f2f3b8b0640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:44.774 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.771+0000 7f2f3b8b0640 1 -- 172.21.15.132:0/1096698870 >> v1:172.21.15.132:6790/0 conn(0x7f2f3413b2c0 legacy=0x7f2f341d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:44.774 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.771+0000 7f2f3b8b0640 1 -- 172.21.15.132:0/1096698870 shutdown_connections 2024-01-24T23:23:44.774 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.771+0000 7f2f3b8b0640 1 -- 172.21.15.132:0/1096698870 >> 172.21.15.132:0/1096698870 conn(0x7f2f340a5a10 msgr2=0x7f2f340a2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:44.774 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.771+0000 7f2f3b8b0640 1 -- 172.21.15.132:0/1096698870 shutdown_connections 2024-01-24T23:23:44.774 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.771+0000 7f2f3b8b0640 1 -- 172.21.15.132:0/1096698870 wait complete. 2024-01-24T23:23:44.787 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:23:44.787 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule rm foo 2024-01-24T23:23:44.902 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.899+0000 7f7ff1b33640 10 monclient: get_monmap_and_config 2024-01-24T23:23:44.903 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.899+0000 7f7ff1b33640 10 monclient: build_initial_monmap 2024-01-24T23:23:44.903 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.899+0000 7f7ff1b33640 10 monclient: monmap: 2024-01-24T23:23:44.903 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:44.903 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:44.903 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:44.904093+0000 2024-01-24T23:23:44.903 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:44.904093+0000 2024-01-24T23:23:44.903 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:44.903 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:44.903 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:44.903 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:44.903 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:44.903 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:44.903 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:44.903 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:44.903 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:44.904 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:44.904 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:44.904 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:44.906 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7ff1b33640 1 Processor -- start 2024-01-24T23:23:44.906 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7ff1b33640 1 -- start start 2024-01-24T23:23:44.906 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7ff1b33640 10 monclient: init 2024-01-24T23:23:44.906 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7ff1b33640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:44.906 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7ff1b33640 10 monclient: _add_conns ranks=[1,2,0,8,6,5,4,3,7] 2024-01-24T23:23:44.906 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7ff1b33640 10 monclient(hunting): picked mon.noname-f con 0x7f7fec133d80 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:44.906 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7ff1b33640 10 monclient(hunting): picked mon.noname-b con 0x7f7fec09dcf0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:44.906 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7ff1b33640 10 monclient(hunting): picked mon.noname-a con 0x7f7fec09e0d0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:44.906 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7ff1b33640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fec064700 con 0x7f7fec09e0d0 2024-01-24T23:23:44.906 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7ff1b33640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fec09cfb0 con 0x7f7fec133d80 2024-01-24T23:23:44.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7ff1b33640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fec09c6a0 con 0x7f7fec09dcf0 2024-01-24T23:23:44.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7ff1b33640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:44.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7ff1b33640 10 monclient(hunting): authenticate will time out at 1112.229566s 2024-01-24T23:23:44.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7feb7fe640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f7fec133d80 legacy=0x7f7fec134160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:44.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7febfff640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f7fec09e0d0 legacy=0x7f7fec13c1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:44.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7feaffd640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f7fec09dcf0 0x7f7fec138ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:40188/0 (socket says 172.21.15.132:40188) 2024-01-24T23:23:44.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7feaffd640 1 -- 172.21.15.132:0/3417543386 learned_addr learned my addr 172.21.15.132:0/3417543386 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:44.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 1 -- 172.21.15.132:0/3417543386 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3017978406 0 0) 0x7f7fec09c6a0 con 0x7f7fec09dcf0 2024-01-24T23:23:44.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:44.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:44.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:44.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 10 monclient(hunting): my global_id is 84494 2024-01-24T23:23:44.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 1 -- 172.21.15.132:0/3417543386 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fd4003530 con 0x7f7fec09dcf0 2024-01-24T23:23:44.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 1 -- 172.21.15.132:0/3417543386 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2470175055 0 0) 0x7f7fd4003530 con 0x7f7fec09dcf0 2024-01-24T23:23:44.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 1 -- 172.21.15.132:0/3417543386 >> v1:172.21.15.132:6789/0 conn(0x7f7fec133d80 legacy=0x7f7fec134160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:44.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 1 -- 172.21.15.132:0/3417543386 >> v1:172.21.15.115:6789/0 conn(0x7f7fec09e0d0 legacy=0x7f7fec13c1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:44.908 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:23:44.908 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 1 monclient: found mon.noname-b 2024-01-24T23:23:44.908 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:44.908 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:23:44.908 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 1 -- 172.21.15.132:0/3417543386 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fec09c980 con 0x7f7fec09dcf0 2024-01-24T23:23:44.908 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 10 monclient: _finish_auth 0 2024-01-24T23:23:44.908 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:23:44.908 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:44.908 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7ff1b33640 5 monclient: authenticate success, global_id 84494 2024-01-24T23:23:44.908 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 1 -- 172.21.15.132:0/3417543386 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7fe0001800 con 0x7f7fec09dcf0 2024-01-24T23:23:44.908 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:44.908 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:23:44.908 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 10 monclient: dump: 2024-01-24T23:23:44.908 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:44.908 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:44.909 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.909 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.909 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:44.909 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:44.909 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:44.909 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:44.909 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:44.909 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:44.909 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:44.909 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:44.909 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:44.909 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:44.909 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:44.909 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:44.909 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7ff1b33640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:44.909 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 1 -- 172.21.15.132:0/3417543386 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7fe0001a60 con 0x7f7fec09dcf0 2024-01-24T23:23:44.909 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:44.910 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7ff1b33640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:44.910 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7ff1b33640 4 set_mon_vals no callback set 2024-01-24T23:23:44.910 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 1 -- 172.21.15.132:0/3417543386 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7fe0004bf0 con 0x7f7fec09dcf0 2024-01-24T23:23:44.911 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:44.911 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:44.911 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7fea7fc640 10 monclient: dump: 2024-01-24T23:23:44.911 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:44.911 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:44.911 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.911 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.911 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:44.911 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:44.911 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:44.911 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:44.911 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:44.911 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:44.911 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:44.912 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:44.912 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:44.912 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:44.912 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:44.912 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:44.916 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7ff1b33640 10 monclient: shutdown 2024-01-24T23:23:44.917 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7ff1b33640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:44.917 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7ff1b33640 1 -- 172.21.15.132:0/3417543386 >> v1:172.21.15.115:6790/0 conn(0x7f7fec09dcf0 legacy=0x7f7fec138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:44.917 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7ff1b33640 1 -- 172.21.15.132:0/3417543386 shutdown_connections 2024-01-24T23:23:44.917 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7ff1b33640 1 -- 172.21.15.132:0/3417543386 >> 172.21.15.132:0/3417543386 conn(0x7f7fec0a6690 msgr2=0x7f7fec0a2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:44.917 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7ff1b33640 1 -- 172.21.15.132:0/3417543386 shutdown_connections 2024-01-24T23:23:44.917 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.903+0000 7f7ff1b33640 1 -- 172.21.15.132:0/3417543386 wait complete. 2024-01-24T23:23:44.917 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 10 monclient: build_initial_monmap 2024-01-24T23:23:44.917 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 10 monclient: monmap: 2024-01-24T23:23:44.917 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:44.917 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:44.917 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:44.917 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:44.917 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:44.917 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:44.918 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:44.918 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:44.918 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:44.918 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:44.918 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:44.918 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:44.918 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:44.918 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:44.918 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:44.918 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:44.918 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 1 Processor -- start 2024-01-24T23:23:44.918 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 1 -- start start 2024-01-24T23:23:44.918 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 10 monclient: init 2024-01-24T23:23:44.918 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:44.918 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 10 monclient: _add_conns ranks=[3,8,5,7,1,4,6,2,0] 2024-01-24T23:23:44.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 10 monclient(hunting): picked mon.noname-g con 0x7f7fec09dcf0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:44.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 10 monclient(hunting): picked mon.noname-e con 0x7f7fec09e0d0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:44.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 10 monclient(hunting): picked mon.noname-h con 0x7f7fec133d80 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:44.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fec1c8fa0 con 0x7f7fec09dcf0 2024-01-24T23:23:44.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fec09ba40 con 0x7f7fec133d80 2024-01-24T23:23:44.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fec0722e0 con 0x7f7fec09e0d0 2024-01-24T23:23:44.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:44.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 10 monclient(hunting): authenticate will time out at 1112.233217s 2024-01-24T23:23:44.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7feb7fe640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f7fec09dcf0 0x7f7fec132db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:52834/0 (socket says 172.21.15.132:52834) 2024-01-24T23:23:44.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7feb7fe640 1 -- 172.21.15.132:0/2085582173 learned_addr learned my addr 172.21.15.132:0/2085582173 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:44.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 1 -- 172.21.15.132:0/2085582173 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3021489595 0 0) 0x7f7fec1c8fa0 con 0x7f7fec09dcf0 2024-01-24T23:23:44.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:44.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:44.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:44.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient(hunting): my global_id is 5214 2024-01-24T23:23:44.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 1 -- 172.21.15.132:0/2085582173 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fd8003530 con 0x7f7fec09dcf0 2024-01-24T23:23:44.920 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 1 -- 172.21.15.132:0/2085582173 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1343532324 0 0) 0x7f7fec09ba40 con 0x7f7fec133d80 2024-01-24T23:23:44.920 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:44.920 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:44.920 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:44.920 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient(hunting): my global_id is 5189 2024-01-24T23:23:44.920 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 1 -- 172.21.15.132:0/2085582173 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fec1c8fa0 con 0x7f7fec133d80 2024-01-24T23:23:44.920 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 1 -- 172.21.15.132:0/2085582173 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 264626506 0 0) 0x7f7fd8003530 con 0x7f7fec09dcf0 2024-01-24T23:23:44.920 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 1 -- 172.21.15.132:0/2085582173 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7fec09ba40 con 0x7f7fec09dcf0 2024-01-24T23:23:44.920 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 1 -- 172.21.15.132:0/2085582173 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7fdc004300 con 0x7f7fec09dcf0 2024-01-24T23:23:44.920 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:44.920 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:44.921 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient(hunting): dump: 2024-01-24T23:23:44.921 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:44.921 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:44.921 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.921 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.921 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:44.921 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:44.921 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:44.921 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:44.921 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:44.921 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:44.921 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:44.921 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:44.921 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:44.921 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:44.922 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:44.922 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:44.922 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:44.922 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 1 -- 172.21.15.132:0/2085582173 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2834948647 0 0) 0x7f7fec1c8fa0 con 0x7f7fec133d80 2024-01-24T23:23:44.922 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 1 -- 172.21.15.132:0/2085582173 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7fd8003530 con 0x7f7fec133d80 2024-01-24T23:23:44.922 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 1 -- 172.21.15.132:0/2085582173 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7fe4004490 con 0x7f7fec133d80 2024-01-24T23:23:44.922 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:44.922 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:44.922 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient(hunting): dump: 2024-01-24T23:23:44.922 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:44.922 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:44.922 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.922 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.922 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:44.922 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:44.922 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:44.923 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:44.923 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:44.923 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:44.923 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:44.923 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:44.923 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:44.923 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:44.923 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:44.923 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:44.923 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 1 -- 172.21.15.132:0/2085582173 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4177086312 0 0) 0x7f7fec0722e0 con 0x7f7fec09e0d0 2024-01-24T23:23:44.923 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:44.923 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:44.923 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:44.923 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient(hunting): my global_id is 84392 2024-01-24T23:23:44.923 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 1 -- 172.21.15.132:0/2085582173 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fec1c8fa0 con 0x7f7fec09e0d0 2024-01-24T23:23:44.923 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 1 -- 172.21.15.132:0/2085582173 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3554494435 0 0) 0x7f7fec09ba40 con 0x7f7fec09dcf0 2024-01-24T23:23:44.924 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 1 -- 172.21.15.132:0/2085582173 >> v1:172.21.15.115:6793/0 conn(0x7f7fec09e0d0 legacy=0x7f7fec1cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:44.924 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 1 -- 172.21.15.132:0/2085582173 >> v1:172.21.15.132:6791/0 conn(0x7f7fec133d80 legacy=0x7f7fec1d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:44.924 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient: _finish_hunting 0 2024-01-24T23:23:44.924 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 1 monclient: found mon.g 2024-01-24T23:23:44.924 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:44.924 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:44.924 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 1 -- 172.21.15.132:0/2085582173 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fec06b1b0 con 0x7f7fec09dcf0 2024-01-24T23:23:44.924 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient: _finish_auth 0 2024-01-24T23:23:44.924 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient: _check_auth_tickets 2024-01-24T23:23:44.924 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:44.924 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 5 monclient: authenticate success, global_id 5214 2024-01-24T23:23:44.924 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 10 monclient: _renew_subs 2024-01-24T23:23:44.924 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:44.924 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 1 -- 172.21.15.132:0/2085582173 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fec1217a0 con 0x7f7fec09dcf0 2024-01-24T23:23:44.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 10 monclient: _renew_subs 2024-01-24T23:23:44.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:44.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 1 -- 172.21.15.132:0/2085582173 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fec1334c0 con 0x7f7fec09dcf0 2024-01-24T23:23:44.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 1 -- 172.21.15.132:0/2085582173 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7fdc0044d0 con 0x7f7fec09dcf0 2024-01-24T23:23:44.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:44.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 1 -- 172.21.15.132:0/2085582173 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7fdc004a00 con 0x7f7fec09dcf0 2024-01-24T23:23:44.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:44.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fe9ffb640 4 set_mon_vals no callback set 2024-01-24T23:23:44.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:44.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 10 monclient: dump: 2024-01-24T23:23:44.925 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:44.925 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:44.925 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.925 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:44.925 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:44.925 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:44.926 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:44.926 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:44.926 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:44.926 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:44.926 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:44.926 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:44.926 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:44.926 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:44.926 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:44.926 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:44.926 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:44.926 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:44.926 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:44.926 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 1 -- 172.21.15.132:0/2085582173 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f7fdc004c20 con 0x7f7fec09dcf0 2024-01-24T23:23:44.926 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7ff1b33640 1 -- 172.21.15.132:0/2085582173 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fcc005090 con 0x7f7fec09dcf0 2024-01-24T23:23:44.926 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7fd3fff640 1 -- 172.21.15.132:0/2085582173 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(57..57 src has 1..57) v4 ==== 4305+0+0 (unknown 2895024988 0 0) 0x7f7fdc043990 con 0x7f7fec09dcf0 2024-01-24T23:23:44.927 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.907+0000 7f7feaffd640 10 monclient: get_auth_request con 0x7f7fd80376a0 auth_method 0 2024-01-24T23:23:44.927 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.911+0000 7f7fd3fff640 1 -- 172.21.15.132:0/2085582173 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f7fdc011380 con 0x7f7fec09dcf0 2024-01-24T23:23:44.927 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.911+0000 7f7fd3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:44.927 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:44.911+0000 7f7fd3fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:45.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:45.071+0000 7f7ff1b33640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "foo"}] 2024-01-24T23:23:45.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:45.071+0000 7f7ff1b33640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-01-24T23:23:45.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:45.071+0000 7f7ff1b33640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:45.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:45.071+0000 7f7ff1b33640 1 -- 172.21.15.132:0/2085582173 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rule rm", "name": "foo"} v 0) v1 -- 0x7f7fcc005340 con 0x7f7fec09dcf0 2024-01-24T23:23:45.909 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:45.907+0000 7f7fd3fff640 1 -- 172.21.15.132:0/2085582173 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo"}]=0 v58) v1 ==== 80+0+0 (unknown 130220181 0 0) 0x7f7fdc012e90 con 0x7f7fec09dcf0 2024-01-24T23:23:45.909 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:45.907+0000 7f7fd3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-01-24T23:23:45.909 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:45.907+0000 7f7fd3fff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:46.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.007+0000 7f7ff1b33640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "foo"}] 2024-01-24T23:23:46.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.007+0000 7f7ff1b33640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-01-24T23:23:46.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.007+0000 7f7ff1b33640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:46.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.007+0000 7f7ff1b33640 1 -- 172.21.15.132:0/2085582173 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rule rm", "name": "foo"} v 0) v1 -- 0x7f7fcc002220 con 0x7f7fec09dcf0 2024-01-24T23:23:46.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.011+0000 7f7fd3fff640 1 -- 172.21.15.132:0/2085582173 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo"}]=0 rule foo does not exist v58) v1 ==== 103+0+0 (unknown 77879010 0 0) 0x7f7fdc014610 con 0x7f7fec09dcf0 2024-01-24T23:23:46.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.011+0000 7f7fd3fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-01-24T23:23:46.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.011+0000 7f7fd3fff640 10 monclient: _finish_command 3 = system:0 rule foo does not exist 2024-01-24T23:23:46.012 INFO:tasks.workunit.client.0.smithi132.stderr:rule foo does not exist 2024-01-24T23:23:46.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.011+0000 7f7ff1b33640 1 -- 172.21.15.132:0/2085582173 >> v1:172.21.15.132:6812/1835401922 conn(0x7f7fd80376a0 legacy=0x7f7fd8039b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:46.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.011+0000 7f7ff1b33640 10 monclient: shutdown 2024-01-24T23:23:46.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.011+0000 7f7ff1b33640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:46.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.011+0000 7f7ff1b33640 1 -- 172.21.15.132:0/2085582173 >> v1:172.21.15.132:6790/0 conn(0x7f7fec09dcf0 legacy=0x7f7fec132db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:46.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.011+0000 7f7ff1b33640 1 -- 172.21.15.132:0/2085582173 shutdown_connections 2024-01-24T23:23:46.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.011+0000 7f7ff1b33640 1 -- 172.21.15.132:0/2085582173 >> 172.21.15.132:0/2085582173 conn(0x7f7fec0a6690 msgr2=0x7f7fec0a1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:46.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.011+0000 7f7ff1b33640 1 -- 172.21.15.132:0/2085582173 shutdown_connections 2024-01-24T23:23:46.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.011+0000 7f7ff1b33640 1 -- 172.21.15.132:0/2085582173 wait complete. 2024-01-24T23:23:46.028 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule rm foo 2024-01-24T23:23:46.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b07ad2640 10 monclient: get_monmap_and_config 2024-01-24T23:23:46.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b07ad2640 10 monclient: build_initial_monmap 2024-01-24T23:23:46.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b07ad2640 10 monclient: monmap: 2024-01-24T23:23:46.148 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:46.148 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:46.148 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:46.149349+0000 2024-01-24T23:23:46.148 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:46.149349+0000 2024-01-24T23:23:46.148 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:46.148 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:46.148 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:46.148 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:46.148 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:46.149 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:46.149 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:46.149 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:46.149 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:46.149 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:46.149 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:46.149 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:46.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b07ad2640 1 Processor -- start 2024-01-24T23:23:46.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b07ad2640 1 -- start start 2024-01-24T23:23:46.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b07ad2640 10 monclient: init 2024-01-24T23:23:46.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b07ad2640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:46.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b07ad2640 10 monclient: _add_conns ranks=[8,6,1,2,3,5,4,0,7] 2024-01-24T23:23:46.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b07ad2640 10 monclient(hunting): picked mon.noname-e con 0x7f3b000740e0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:46.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b07ad2640 10 monclient(hunting): picked mon.noname-d con 0x7f3b00130750 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:46.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b07ad2640 10 monclient(hunting): picked mon.noname-f con 0x7f3b0013a1e0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:46.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b07ad2640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b00064700 con 0x7f3b0013a1e0 2024-01-24T23:23:46.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b07ad2640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b000744c0 con 0x7f3b00130750 2024-01-24T23:23:46.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b07ad2640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b0006fa60 con 0x7f3b000740e0 2024-01-24T23:23:46.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b07ad2640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:46.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b07ad2640 10 monclient(hunting): authenticate will time out at 1113.474875s 2024-01-24T23:23:46.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b0606f640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f3b0013a1e0 legacy=0x7f3b0013c5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:46.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b0586e640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f3b000740e0 0x7f3b00071830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:51096/0 (socket says 172.21.15.132:51096) 2024-01-24T23:23:46.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b0586e640 1 -- 172.21.15.132:0/1149307884 learned_addr learned my addr 172.21.15.132:0/1149307884 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:46.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b0486c640 1 -- 172.21.15.132:0/1149307884 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1998428159 0 0) 0x7f3b000744c0 con 0x7f3b00130750 2024-01-24T23:23:46.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b0486c640 10 monclient(hunting): _negotiate 2024-01-24T23:23:46.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b0486c640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:46.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b0486c640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:46.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b0486c640 10 monclient(hunting): my global_id is 5280 2024-01-24T23:23:46.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b0486c640 1 -- 172.21.15.132:0/1149307884 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3aec003530 con 0x7f3b00130750 2024-01-24T23:23:46.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b0486c640 1 -- 172.21.15.132:0/1149307884 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1421792455 0 0) 0x7f3b0006fa60 con 0x7f3b000740e0 2024-01-24T23:23:46.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b0486c640 10 monclient(hunting): _negotiate 2024-01-24T23:23:46.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b0486c640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:46.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b0486c640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:46.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b0486c640 10 monclient(hunting): my global_id is 84401 2024-01-24T23:23:46.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.147+0000 7f3b0486c640 1 -- 172.21.15.132:0/1149307884 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b000744c0 con 0x7f3b000740e0 2024-01-24T23:23:46.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0486c640 1 -- 172.21.15.132:0/1149307884 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4268170343 0 0) 0x7f3aec003530 con 0x7f3b00130750 2024-01-24T23:23:46.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0486c640 1 -- 172.21.15.132:0/1149307884 >> v1:172.21.15.115:6793/0 conn(0x7f3b000740e0 legacy=0x7f3b00071830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:46.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0486c640 1 -- 172.21.15.132:0/1149307884 >> v1:172.21.15.132:6789/0 conn(0x7f3b0013a1e0 legacy=0x7f3b0013c5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:46.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0486c640 10 monclient: _finish_hunting 0 2024-01-24T23:23:46.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0486c640 1 monclient: found mon.noname-d 2024-01-24T23:23:46.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0486c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:46.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0486c640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:23:46.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0486c640 1 -- 172.21.15.132:0/1149307884 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b00130d70 con 0x7f3b00130750 2024-01-24T23:23:46.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0486c640 10 monclient: _finish_auth 0 2024-01-24T23:23:46.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0486c640 10 monclient: _check_auth_tickets 2024-01-24T23:23:46.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0486c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:46.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0486c640 1 -- 172.21.15.132:0/1149307884 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3af00017c0 con 0x7f3b00130750 2024-01-24T23:23:46.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0486c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:46.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0486c640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:23:46.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0486c640 10 monclient: dump: 2024-01-24T23:23:46.155 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:46.155 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:46.156 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:46.156 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:46.156 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:46.156 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:46.156 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:46.156 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:46.156 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:46.156 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:46.156 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:46.156 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:46.156 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:46.156 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:46.156 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:46.156 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:46.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 5 monclient: authenticate success, global_id 5280 2024-01-24T23:23:46.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:46.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0486c640 1 -- 172.21.15.132:0/1149307884 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3af0001a30 con 0x7f3b00130750 2024-01-24T23:23:46.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0486c640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:46.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:46.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 4 set_mon_vals no callback set 2024-01-24T23:23:46.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0486c640 1 -- 172.21.15.132:0/1149307884 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3af0003960 con 0x7f3b00130750 2024-01-24T23:23:46.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0486c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:46.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0486c640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:46.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0486c640 10 monclient: dump: 2024-01-24T23:23:46.157 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:46.157 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:46.157 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:46.157 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:46.157 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:46.158 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:46.158 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:46.158 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:46.158 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:46.158 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:46.158 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:46.158 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:46.158 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:46.158 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:46.158 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:46.158 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:46.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 10 monclient: shutdown 2024-01-24T23:23:46.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:46.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 1 -- 172.21.15.132:0/1149307884 >> v1:172.21.15.115:6792/0 conn(0x7f3b00130750 legacy=0x7f3b00138ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:46.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 1 -- 172.21.15.132:0/1149307884 shutdown_connections 2024-01-24T23:23:46.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 1 -- 172.21.15.132:0/1149307884 >> 172.21.15.132:0/1149307884 conn(0x7f3b000a5f40 msgr2=0x7f3b000a2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:46.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 1 -- 172.21.15.132:0/1149307884 shutdown_connections 2024-01-24T23:23:46.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 1 -- 172.21.15.132:0/1149307884 wait complete. 2024-01-24T23:23:46.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 10 monclient: build_initial_monmap 2024-01-24T23:23:46.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 10 monclient: monmap: 2024-01-24T23:23:46.159 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:46.159 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:46.159 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:46.159 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:46.159 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:46.159 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:46.159 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:46.159 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:46.160 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:46.160 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:46.160 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:46.160 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:46.160 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:46.160 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:46.160 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:46.160 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:46.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 1 Processor -- start 2024-01-24T23:23:46.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 1 -- start start 2024-01-24T23:23:46.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 10 monclient: init 2024-01-24T23:23:46.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:46.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 10 monclient: _add_conns ranks=[6,2,1,5,7,3,4,0,8] 2024-01-24T23:23:46.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 10 monclient(hunting): picked mon.noname-d con 0x7f3b000740e0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:46.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 10 monclient(hunting): picked mon.noname-b con 0x7f3b00130750 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:46.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 10 monclient(hunting): picked mon.noname-f con 0x7f3b0013a1e0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:46.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b001352e0 con 0x7f3b0013a1e0 2024-01-24T23:23:46.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b0006f4d0 con 0x7f3b00130750 2024-01-24T23:23:46.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b00136c40 con 0x7f3b000740e0 2024-01-24T23:23:46.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:46.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b07ad2640 10 monclient(hunting): authenticate will time out at 1113.478480s 2024-01-24T23:23:46.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0606f640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f3b0013a1e0 legacy=0x7f3b001d13c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:46.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0586e640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f3b000740e0 0x7f3b001336d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:49738/0 (socket says 172.21.15.132:49738) 2024-01-24T23:23:46.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0506d640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f3b00130750 0x7f3b00133de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:40204/0 (socket says 172.21.15.132:40204) 2024-01-24T23:23:46.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3b0586e640 1 -- 172.21.15.132:0/3033615646 learned_addr learned my addr 172.21.15.132:0/3033615646 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:46.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 1 -- 172.21.15.132:0/3033615646 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087111044 0 0) 0x7f3b00136c40 con 0x7f3b000740e0 2024-01-24T23:23:46.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:46.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:46.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:46.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 10 monclient(hunting): my global_id is 5289 2024-01-24T23:23:46.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 1 -- 172.21.15.132:0/3033615646 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ae4003530 con 0x7f3b000740e0 2024-01-24T23:23:46.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 1 -- 172.21.15.132:0/3033615646 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1155353450 0 0) 0x7f3b0006f4d0 con 0x7f3b00130750 2024-01-24T23:23:46.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:46.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:46.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:46.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 10 monclient(hunting): my global_id is 84503 2024-01-24T23:23:46.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 1 -- 172.21.15.132:0/3033615646 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b00136c40 con 0x7f3b00130750 2024-01-24T23:23:46.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 1 -- 172.21.15.132:0/3033615646 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2208778276 0 0) 0x7f3ae4003530 con 0x7f3b000740e0 2024-01-24T23:23:46.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 1 -- 172.21.15.132:0/3033615646 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3b0006f4d0 con 0x7f3b000740e0 2024-01-24T23:23:46.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 1 -- 172.21.15.132:0/3033615646 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2390031958 0 0) 0x7f3b00136c40 con 0x7f3b00130750 2024-01-24T23:23:46.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 1 -- 172.21.15.132:0/3033615646 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3ae4003530 con 0x7f3b00130750 2024-01-24T23:23:46.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 1 -- 172.21.15.132:0/3033615646 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3afc0019a0 con 0x7f3b000740e0 2024-01-24T23:23:46.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:46.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:23:46.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 10 monclient(hunting): dump: 2024-01-24T23:23:46.166 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:46.166 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:46.166 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:46.166 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:46.167 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:46.167 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:46.167 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:46.167 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:46.167 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:46.167 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:46.167 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:46.167 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:46.167 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:46.167 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:46.167 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:46.167 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:46.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:46.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 1 -- 172.21.15.132:0/3033615646 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3af0000ff0 con 0x7f3b00130750 2024-01-24T23:23:46.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:46.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:46.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.151+0000 7f3af67fc640 10 monclient(hunting): dump: 2024-01-24T23:23:46.168 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:46.168 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:46.168 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:46.168 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:46.168 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:46.168 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:46.168 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:46.168 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:46.168 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:46.168 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:46.168 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:46.168 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:46.168 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:46.169 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:46.169 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:46.169 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:46.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3af67fc640 1 -- 172.21.15.132:0/3033615646 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3265176685 0 0) 0x7f3ae4003530 con 0x7f3b00130750 2024-01-24T23:23:46.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3af67fc640 1 -- 172.21.15.132:0/3033615646 >> v1:172.21.15.115:6792/0 conn(0x7f3b000740e0 legacy=0x7f3b001336d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:46.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3af67fc640 1 -- 172.21.15.132:0/3033615646 >> v1:172.21.15.132:6789/0 conn(0x7f3b0013a1e0 legacy=0x7f3b001d13c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:46.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3af67fc640 10 monclient: _finish_hunting 0 2024-01-24T23:23:46.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3af67fc640 1 monclient: found mon.b 2024-01-24T23:23:46.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3af67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:46.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3af67fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:46.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3af67fc640 1 -- 172.21.15.132:0/3033615646 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b001344f0 con 0x7f3b00130750 2024-01-24T23:23:46.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3af67fc640 10 monclient: _finish_auth 0 2024-01-24T23:23:46.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3af67fc640 10 monclient: _check_auth_tickets 2024-01-24T23:23:46.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3af67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:46.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3b07ad2640 5 monclient: authenticate success, global_id 84503 2024-01-24T23:23:46.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3b07ad2640 10 monclient: _renew_subs 2024-01-24T23:23:46.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3b07ad2640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:46.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3b07ad2640 1 -- 172.21.15.132:0/3033615646 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b00134970 con 0x7f3b00130750 2024-01-24T23:23:46.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3b07ad2640 10 monclient: _renew_subs 2024-01-24T23:23:46.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3b07ad2640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:46.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3b07ad2640 1 -- 172.21.15.132:0/3033615646 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b00137060 con 0x7f3b00130750 2024-01-24T23:23:46.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3af67fc640 1 -- 172.21.15.132:0/3033615646 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3af0001200 con 0x7f3b00130750 2024-01-24T23:23:46.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3af67fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:46.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3af67fc640 1 -- 172.21.15.132:0/3033615646 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3af0004000 con 0x7f3b00130750 2024-01-24T23:23:46.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3af67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:46.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3af67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:46.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3af77fe640 4 set_mon_vals no callback set 2024-01-24T23:23:46.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3af67fc640 10 monclient: dump: 2024-01-24T23:23:46.170 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:46.170 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:46.171 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:46.171 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:46.171 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:46.171 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:46.171 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:46.171 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:46.171 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:46.171 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:46.171 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:46.171 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:46.171 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:46.171 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:46.171 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:46.171 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:46.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3b07ad2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:46.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3b07ad2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:46.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3b07ad2640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:46.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3b07ad2640 1 -- 172.21.15.132:0/3033615646 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae8005090 con 0x7f3b00130750 2024-01-24T23:23:46.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3af67fc640 1 -- 172.21.15.132:0/3033615646 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f3af00041e0 con 0x7f3b00130750 2024-01-24T23:23:46.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3af67fc640 1 -- 172.21.15.132:0/3033615646 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(58..58 src has 1..58) v4 ==== 4250+0+0 (unknown 1165573899 0 0) 0x7f3af0043ad0 con 0x7f3b00130750 2024-01-24T23:23:46.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.155+0000 7f3b0586e640 10 monclient: get_auth_request con 0x7f3ae4037640 auth_method 0 2024-01-24T23:23:46.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.167+0000 7f3af67fc640 1 -- 172.21.15.132:0/3033615646 <== mon.2 v1:172.21.15.115:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f3af0014c30 con 0x7f3b00130750 2024-01-24T23:23:46.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.167+0000 7f3af67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:46.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.167+0000 7f3af67fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:46.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.323+0000 7f3b07ad2640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "foo"}] 2024-01-24T23:23:46.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.323+0000 7f3b07ad2640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-01-24T23:23:46.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.323+0000 7f3b07ad2640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:46.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.323+0000 7f3b07ad2640 1 -- 172.21.15.132:0/3033615646 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd crush rule rm", "name": "foo"} v 0) v1 -- 0x7f3ae8005340 con 0x7f3b00130750 2024-01-24T23:23:46.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.323+0000 7f3af67fc640 1 -- 172.21.15.132:0/3033615646 <== mon.2 v1:172.21.15.115:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo"}]=0 rule foo does not exist v58) v1 ==== 103+0+0 (unknown 77879010 0 0) 0x7f3af001d060 con 0x7f3b00130750 2024-01-24T23:23:46.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.323+0000 7f3af67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-01-24T23:23:46.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.323+0000 7f3af67fc640 10 monclient: _finish_command 2 = system:0 rule foo does not exist 2024-01-24T23:23:46.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.419+0000 7f3b07ad2640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "foo"}] 2024-01-24T23:23:46.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.419+0000 7f3b07ad2640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-01-24T23:23:46.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.419+0000 7f3b07ad2640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:46.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.419+0000 7f3b07ad2640 1 -- 172.21.15.132:0/3033615646 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd crush rule rm", "name": "foo"} v 0) v1 -- 0x7f3ae8002220 con 0x7f3b00130750 2024-01-24T23:23:46.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.423+0000 7f3af67fc640 1 -- 172.21.15.132:0/3033615646 <== mon.2 v1:172.21.15.115:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo"}]=0 rule foo does not exist v58) v1 ==== 103+0+0 (unknown 77879010 0 0) 0x7f3af00130a0 con 0x7f3b00130750 2024-01-24T23:23:46.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.423+0000 7f3af67fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-01-24T23:23:46.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.423+0000 7f3af67fc640 10 monclient: _finish_command 3 = system:0 rule foo does not exist 2024-01-24T23:23:46.423 INFO:tasks.workunit.client.0.smithi132.stderr:rule foo does not exist 2024-01-24T23:23:46.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.423+0000 7f3b07ad2640 1 -- 172.21.15.132:0/3033615646 >> v1:172.21.15.132:6812/1835401922 conn(0x7f3ae4037640 legacy=0x7f3ae4039b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:46.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.423+0000 7f3b07ad2640 10 monclient: shutdown 2024-01-24T23:23:46.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.423+0000 7f3b07ad2640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:46.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.423+0000 7f3b07ad2640 1 -- 172.21.15.132:0/3033615646 >> v1:172.21.15.115:6790/0 conn(0x7f3b00130750 legacy=0x7f3b00133de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:46.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.423+0000 7f3b07ad2640 1 -- 172.21.15.132:0/3033615646 shutdown_connections 2024-01-24T23:23:46.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.423+0000 7f3b07ad2640 1 -- 172.21.15.132:0/3033615646 >> 172.21.15.132:0/3033615646 conn(0x7f3b000a5f40 msgr2=0x7f3b0009f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:46.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.423+0000 7f3b07ad2640 1 -- 172.21.15.132:0/3033615646 shutdown_connections 2024-01-24T23:23:46.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.423+0000 7f3b07ad2640 1 -- 172.21.15.132:0/3033615646 wait complete. 2024-01-24T23:23:46.440 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule rm bar 2024-01-24T23:23:46.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.555+0000 7f7ff8de6640 10 monclient: get_monmap_and_config 2024-01-24T23:23:46.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.555+0000 7f7ff8de6640 10 monclient: build_initial_monmap 2024-01-24T23:23:46.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.555+0000 7f7ff8de6640 10 monclient: monmap: 2024-01-24T23:23:46.558 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:46.558 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:46.558 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:46.559579+0000 2024-01-24T23:23:46.559 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:46.559579+0000 2024-01-24T23:23:46.559 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:46.559 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:46.559 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:46.559 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:46.559 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:46.559 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:46.559 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:46.559 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:46.559 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:46.559 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:46.559 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:46.559 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:46.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 1 Processor -- start 2024-01-24T23:23:46.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 1 -- start start 2024-01-24T23:23:46.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 10 monclient: init 2024-01-24T23:23:46.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:46.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 10 monclient: _add_conns ranks=[0,2,6,1,4,3,5,7,8] 2024-01-24T23:23:46.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 10 monclient(hunting): picked mon.noname-a con 0x7f7ff409ff20 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:46.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 10 monclient(hunting): picked mon.noname-b con 0x7f7ff409f9c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:46.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 10 monclient(hunting): picked mon.noname-d con 0x7f7ff413b2c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:46.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ff4064700 con 0x7f7ff409ff20 2024-01-24T23:23:46.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ff409e760 con 0x7f7ff409f9c0 2024-01-24T23:23:46.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ff40a1160 con 0x7f7ff413b2c0 2024-01-24T23:23:46.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:46.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 10 monclient(hunting): authenticate will time out at 1113.885011s 2024-01-24T23:23:46.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff259c640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f7ff409ff20 legacy=0x7f7ff40a0300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:46.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff2d9d640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f7ff413b2c0 0x7f7ff413d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:49754/0 (socket says 172.21.15.132:49754) 2024-01-24T23:23:46.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7fe9d9b640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f7ff409f9c0 0x7f7ff4139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:40210/0 (socket says 172.21.15.132:40210) 2024-01-24T23:23:46.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff2d9d640 1 -- 172.21.15.132:0/4238372060 learned_addr learned my addr 172.21.15.132:0/4238372060 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:46.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 1 -- 172.21.15.132:0/4238372060 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2115909594 0 0) 0x7f7ff40a1160 con 0x7f7ff413b2c0 2024-01-24T23:23:46.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:23:46.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:46.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:46.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 10 monclient(hunting): my global_id is 5298 2024-01-24T23:23:46.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 1 -- 172.21.15.132:0/4238372060 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fd4003530 con 0x7f7ff413b2c0 2024-01-24T23:23:46.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 1 -- 172.21.15.132:0/4238372060 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 332395425 0 0) 0x7f7ff409e760 con 0x7f7ff409f9c0 2024-01-24T23:23:46.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:23:46.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:46.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:46.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 10 monclient(hunting): my global_id is 84512 2024-01-24T23:23:46.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 1 -- 172.21.15.132:0/4238372060 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ff40a1160 con 0x7f7ff409f9c0 2024-01-24T23:23:46.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 1 -- 172.21.15.132:0/4238372060 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1067784553 0 0) 0x7f7fd4003530 con 0x7f7ff413b2c0 2024-01-24T23:23:46.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 1 -- 172.21.15.132:0/4238372060 >> v1:172.21.15.115:6790/0 conn(0x7f7ff409f9c0 legacy=0x7f7ff4139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:46.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 1 -- 172.21.15.132:0/4238372060 >> v1:172.21.15.115:6789/0 conn(0x7f7ff409ff20 legacy=0x7f7ff40a0300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:46.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 10 monclient: _finish_hunting 0 2024-01-24T23:23:46.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 1 monclient: found mon.noname-d 2024-01-24T23:23:46.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:46.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:23:46.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 1 -- 172.21.15.132:0/4238372060 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff40a0a10 con 0x7f7ff413b2c0 2024-01-24T23:23:46.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 10 monclient: _finish_auth 0 2024-01-24T23:23:46.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 10 monclient: _check_auth_tickets 2024-01-24T23:23:46.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:46.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 1 -- 172.21.15.132:0/4238372060 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7fec001b80 con 0x7f7ff413b2c0 2024-01-24T23:23:46.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:46.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:23:46.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 10 monclient: dump: 2024-01-24T23:23:46.566 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:46.566 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:46.566 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:46.566 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:46.566 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:46.566 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:46.566 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:46.566 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:46.566 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:46.566 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:46.566 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:46.566 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:46.567 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:46.567 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:46.567 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:46.567 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:46.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 5 monclient: authenticate success, global_id 5298 2024-01-24T23:23:46.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:46.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 1 -- 172.21.15.132:0/4238372060 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7fec001e10 con 0x7f7ff413b2c0 2024-01-24T23:23:46.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:46.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:46.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 4 set_mon_vals no callback set 2024-01-24T23:23:46.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 1 -- 172.21.15.132:0/4238372060 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7fec004f70 con 0x7f7ff413b2c0 2024-01-24T23:23:46.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:46.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:46.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff1d9b640 10 monclient: dump: 2024-01-24T23:23:46.567 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:46.568 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:46.568 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:46.568 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:46.568 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:46.568 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:46.568 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:46.568 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:46.568 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:46.568 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:46.568 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:46.568 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:46.568 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:46.568 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:46.569 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:46.569 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:46.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 10 monclient: shutdown 2024-01-24T23:23:46.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:46.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 1 -- 172.21.15.132:0/4238372060 >> v1:172.21.15.115:6792/0 conn(0x7f7ff413b2c0 legacy=0x7f7ff413d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:46.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 1 -- 172.21.15.132:0/4238372060 shutdown_connections 2024-01-24T23:23:46.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 1 -- 172.21.15.132:0/4238372060 >> 172.21.15.132:0/4238372060 conn(0x7f7ff40a5a10 msgr2=0x7f7ff40a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:46.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 1 -- 172.21.15.132:0/4238372060 shutdown_connections 2024-01-24T23:23:46.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 1 -- 172.21.15.132:0/4238372060 wait complete. 2024-01-24T23:23:46.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 10 monclient: build_initial_monmap 2024-01-24T23:23:46.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 10 monclient: monmap: 2024-01-24T23:23:46.569 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:46.569 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:46.569 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:46.569 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:46.570 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:46.570 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:46.570 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:46.570 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:46.570 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:46.570 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:46.570 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:46.570 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:46.570 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:46.570 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:46.570 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:46.570 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:46.570 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 1 Processor -- start 2024-01-24T23:23:46.570 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 1 -- start start 2024-01-24T23:23:46.570 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.559+0000 7f7ff8de6640 10 monclient: init 2024-01-24T23:23:46.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff8de6640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:46.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff8de6640 10 monclient: _add_conns ranks=[8,3,5,1,7,6,0,2,4] 2024-01-24T23:23:46.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff8de6640 10 monclient(hunting): picked mon.noname-e con 0x7f7ff409f9c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:46.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff8de6640 10 monclient(hunting): picked mon.noname-g con 0x7f7ff409ff20 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:46.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff8de6640 10 monclient(hunting): picked mon.noname-h con 0x7f7ff413b2c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:46.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff8de6640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ff411d3b0 con 0x7f7ff409ff20 2024-01-24T23:23:46.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff8de6640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ff411e860 con 0x7f7ff413b2c0 2024-01-24T23:23:46.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff8de6640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ff41ca260 con 0x7f7ff409f9c0 2024-01-24T23:23:46.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff8de6640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:46.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff8de6640 10 monclient(hunting): authenticate will time out at 1113.888485s 2024-01-24T23:23:46.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff2d9d640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f7ff413b2c0 0x7f7ff41d16a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:44126/0 (socket says 172.21.15.132:44126) 2024-01-24T23:23:46.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff2d9d640 1 -- 172.21.15.132:0/3580001611 learned_addr learned my addr 172.21.15.132:0/3580001611 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:46.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 1 -- 172.21.15.132:0/3580001611 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2004588337 0 0) 0x7f7ff411e860 con 0x7f7ff413b2c0 2024-01-24T23:23:46.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:46.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:46.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:46.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient(hunting): my global_id is 5198 2024-01-24T23:23:46.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 1 -- 172.21.15.132:0/3580001611 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fd8003560 con 0x7f7ff413b2c0 2024-01-24T23:23:46.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 1 -- 172.21.15.132:0/3580001611 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2534463579 0 0) 0x7f7ff411d3b0 con 0x7f7ff409ff20 2024-01-24T23:23:46.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:46.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:46.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:46.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient(hunting): my global_id is 5223 2024-01-24T23:23:46.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 1 -- 172.21.15.132:0/3580001611 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ff411e860 con 0x7f7ff409ff20 2024-01-24T23:23:46.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 1 -- 172.21.15.132:0/3580001611 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4166629667 0 0) 0x7f7ff411e860 con 0x7f7ff409ff20 2024-01-24T23:23:46.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 1 -- 172.21.15.132:0/3580001611 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ff411d3b0 con 0x7f7ff409ff20 2024-01-24T23:23:46.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 1 -- 172.21.15.132:0/3580001611 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7fdc001a60 con 0x7f7ff409ff20 2024-01-24T23:23:46.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:46.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:46.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient(hunting): dump: 2024-01-24T23:23:46.576 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:46.576 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:46.577 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:46.577 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:46.577 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:46.577 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:46.577 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:46.577 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:46.577 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:46.577 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:46.577 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:46.577 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:46.577 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:46.577 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:46.577 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:46.577 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:46.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:46.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 1 -- 172.21.15.132:0/3580001611 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1220853675 0 0) 0x7f7fd8003560 con 0x7f7ff413b2c0 2024-01-24T23:23:46.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 1 -- 172.21.15.132:0/3580001611 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ff411e860 con 0x7f7ff413b2c0 2024-01-24T23:23:46.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 1 -- 172.21.15.132:0/3580001611 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2494170891 0 0) 0x7f7ff41ca260 con 0x7f7ff409f9c0 2024-01-24T23:23:46.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:46.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:46.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:46.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient(hunting): my global_id is 84410 2024-01-24T23:23:46.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 1 -- 172.21.15.132:0/3580001611 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fd8003560 con 0x7f7ff409f9c0 2024-01-24T23:23:46.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 1 -- 172.21.15.132:0/3580001611 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7fec001b40 con 0x7f7ff413b2c0 2024-01-24T23:23:46.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:46.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:46.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient(hunting): dump: 2024-01-24T23:23:46.578 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:46.578 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:46.579 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:46.579 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:46.579 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:46.579 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:46.579 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:46.579 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:46.579 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:46.579 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:46.579 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:46.579 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:46.579 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:46.579 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:46.579 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:46.579 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:46.579 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 1 -- 172.21.15.132:0/3580001611 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2611345591 0 0) 0x7f7ff411d3b0 con 0x7f7ff409ff20 2024-01-24T23:23:46.579 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 1 -- 172.21.15.132:0/3580001611 >> v1:172.21.15.115:6793/0 conn(0x7f7ff409f9c0 legacy=0x7f7ff41c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:46.579 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 1 -- 172.21.15.132:0/3580001611 >> v1:172.21.15.132:6791/0 conn(0x7f7ff413b2c0 legacy=0x7f7ff41d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:46.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:23:46.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 1 monclient: found mon.g 2024-01-24T23:23:46.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:46.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:46.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 1 -- 172.21.15.132:0/3580001611 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff4071960 con 0x7f7ff409ff20 2024-01-24T23:23:46.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient: _finish_auth 0 2024-01-24T23:23:46.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:23:46.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:46.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff8de6640 5 monclient: authenticate success, global_id 5223 2024-01-24T23:23:46.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff8de6640 10 monclient: _renew_subs 2024-01-24T23:23:46.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff8de6640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:46.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff8de6640 1 -- 172.21.15.132:0/3580001611 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff411e580 con 0x7f7ff409ff20 2024-01-24T23:23:46.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff8de6640 10 monclient: _renew_subs 2024-01-24T23:23:46.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff8de6640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:46.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff8de6640 1 -- 172.21.15.132:0/3580001611 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff411ef60 con 0x7f7ff409ff20 2024-01-24T23:23:46.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 1 -- 172.21.15.132:0/3580001611 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7fdc001d30 con 0x7f7ff409ff20 2024-01-24T23:23:46.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:46.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 1 -- 172.21.15.132:0/3580001611 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7fdc004de0 con 0x7f7ff409ff20 2024-01-24T23:23:46.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:46.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:46.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff159a640 4 set_mon_vals no callback set 2024-01-24T23:23:46.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 10 monclient: dump: 2024-01-24T23:23:46.581 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:46.581 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:46.581 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:46.581 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:46.581 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:46.581 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:46.581 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:46.581 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:46.582 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:46.582 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:46.582 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:46.582 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:46.582 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:46.582 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:46.582 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:46.582 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:46.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff8de6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:46.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff8de6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:46.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff8de6640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:46.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff8de6640 1 -- 172.21.15.132:0/3580001611 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fcc005090 con 0x7f7ff409ff20 2024-01-24T23:23:46.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 1 -- 172.21.15.132:0/3580001611 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f7fdc00f8e0 con 0x7f7ff409ff20 2024-01-24T23:23:46.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7feb7fe640 1 -- 172.21.15.132:0/3580001611 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(58..58 src has 1..58) v4 ==== 4250+0+0 (unknown 1165573899 0 0) 0x7f7fdc043a60 con 0x7f7ff409ff20 2024-01-24T23:23:46.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.563+0000 7f7ff259c640 10 monclient: get_auth_request con 0x7f7fd80378a0 auth_method 0 2024-01-24T23:23:46.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.567+0000 7f7feb7fe640 1 -- 172.21.15.132:0/3580001611 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f7fdc0148c0 con 0x7f7ff409ff20 2024-01-24T23:23:46.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.567+0000 7f7feb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:46.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.567+0000 7f7feb7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:46.715 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.715+0000 7f7ff8de6640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "bar"}] 2024-01-24T23:23:46.716 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.715+0000 7f7ff8de6640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rm", "name": "bar"}] 2024-01-24T23:23:46.716 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.715+0000 7f7ff8de6640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:46.716 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.715+0000 7f7ff8de6640 1 -- 172.21.15.132:0/3580001611 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rule rm", "name": "bar"} v 0) v1 -- 0x7f7fcc005340 con 0x7f7ff409ff20 2024-01-24T23:23:46.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.867+0000 7f7feb7fe640 1 -- 172.21.15.132:0/3580001611 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "bar"}]=0 v59) v1 ==== 80+0+0 (unknown 2932986998 0 0) 0x7f7fdc012f60 con 0x7f7ff409ff20 2024-01-24T23:23:46.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.867+0000 7f7feb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rm", "name": "bar"}] 2024-01-24T23:23:46.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.867+0000 7f7feb7fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:46.971 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.971+0000 7f7ff8de6640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "bar"}] 2024-01-24T23:23:46.971 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.971+0000 7f7ff8de6640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rm", "name": "bar"}] 2024-01-24T23:23:46.972 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.971+0000 7f7ff8de6640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:46.972 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.971+0000 7f7ff8de6640 1 -- 172.21.15.132:0/3580001611 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rule rm", "name": "bar"} v 0) v1 -- 0x7f7fcc002220 con 0x7f7ff409ff20 2024-01-24T23:23:46.972 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.971+0000 7f7feb7fe640 1 -- 172.21.15.132:0/3580001611 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "bar"}]=0 rule bar does not exist v59) v1 ==== 103+0+0 (unknown 3643766223 0 0) 0x7f7fdc0146e0 con 0x7f7ff409ff20 2024-01-24T23:23:46.972 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.971+0000 7f7feb7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rm", "name": "bar"}] 2024-01-24T23:23:46.972 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.971+0000 7f7feb7fe640 10 monclient: _finish_command 3 = system:0 rule bar does not exist 2024-01-24T23:23:46.972 INFO:tasks.workunit.client.0.smithi132.stderr:rule bar does not exist 2024-01-24T23:23:46.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.971+0000 7f7ff8de6640 1 -- 172.21.15.132:0/3580001611 >> v1:172.21.15.132:6812/1835401922 conn(0x7f7fd80378a0 legacy=0x7f7fd8039d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:46.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.971+0000 7f7ff8de6640 10 monclient: shutdown 2024-01-24T23:23:46.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.971+0000 7f7ff8de6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:46.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.971+0000 7f7ff8de6640 1 -- 172.21.15.132:0/3580001611 >> v1:172.21.15.132:6790/0 conn(0x7f7ff409ff20 legacy=0x7f7ff41c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:46.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.971+0000 7f7ff8de6640 1 -- 172.21.15.132:0/3580001611 shutdown_connections 2024-01-24T23:23:46.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.971+0000 7f7ff8de6640 1 -- 172.21.15.132:0/3580001611 >> 172.21.15.132:0/3580001611 conn(0x7f7ff40a5a10 msgr2=0x7f7ff40a2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:46.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.971+0000 7f7ff8de6640 1 -- 172.21.15.132:0/3580001611 shutdown_connections 2024-01-24T23:23:46.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:46.971+0000 7f7ff8de6640 1 -- 172.21.15.132:0/3580001611 wait complete. 2024-01-24T23:23:46.988 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool create pinning_pool 1 2024-01-24T23:23:47.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 10 monclient: get_monmap_and_config 2024-01-24T23:23:47.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 10 monclient: build_initial_monmap 2024-01-24T23:23:47.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 10 monclient: monmap: 2024-01-24T23:23:47.104 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:47.104 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:47.104 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:47.105046+0000 2024-01-24T23:23:47.104 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:47.105046+0000 2024-01-24T23:23:47.104 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:47.104 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:47.104 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:47.104 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:47.104 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:47.104 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:47.104 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:47.105 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:47.105 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:47.105 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:47.105 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:47.105 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:47.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 1 Processor -- start 2024-01-24T23:23:47.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 1 -- start start 2024-01-24T23:23:47.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 10 monclient: init 2024-01-24T23:23:47.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:47.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 10 monclient: _add_conns ranks=[0,3,7,8,4,6,1,5,2] 2024-01-24T23:23:47.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 10 monclient(hunting): picked mon.noname-a con 0x7f973009ff20 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:47.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 10 monclient(hunting): picked mon.noname-g con 0x7f973009f9c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:47.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 10 monclient(hunting): picked mon.noname-i con 0x7f973013b2c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:47.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9730064700 con 0x7f973009ff20 2024-01-24T23:23:47.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973009e760 con 0x7f973009f9c0 2024-01-24T23:23:47.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97300a1160 con 0x7f973013b2c0 2024-01-24T23:23:47.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:47.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 10 monclient(hunting): authenticate will time out at 1114.429716s 2024-01-24T23:23:47.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972e59c640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f973009ff20 legacy=0x7f97300a0300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:47.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972dd9b640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f973009f9c0 0x7f9730139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:52856/0 (socket says 172.21.15.132:52856) 2024-01-24T23:23:47.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972ed9d640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f973013b2c0 0x7f973013d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:48924/0 (socket says 172.21.15.132:48924) 2024-01-24T23:23:47.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972dd9b640 1 -- 172.21.15.132:0/4021785028 learned_addr learned my addr 172.21.15.132:0/4021785028 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:47.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 1 -- 172.21.15.132:0/4021785028 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3463355341 0 0) 0x7f97300a1160 con 0x7f973013b2c0 2024-01-24T23:23:47.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 10 monclient(hunting): _negotiate 2024-01-24T23:23:47.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:47.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:47.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 10 monclient(hunting): my global_id is 5290 2024-01-24T23:23:47.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 1 -- 172.21.15.132:0/4021785028 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9718003530 con 0x7f973013b2c0 2024-01-24T23:23:47.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 1 -- 172.21.15.132:0/4021785028 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775829438 0 0) 0x7f973009e760 con 0x7f973009f9c0 2024-01-24T23:23:47.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 10 monclient(hunting): _negotiate 2024-01-24T23:23:47.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:47.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:47.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 10 monclient(hunting): my global_id is 5232 2024-01-24T23:23:47.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 1 -- 172.21.15.132:0/4021785028 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97300a1160 con 0x7f973009f9c0 2024-01-24T23:23:47.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 1 -- 172.21.15.132:0/4021785028 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3188715806 0 0) 0x7f9718003530 con 0x7f973013b2c0 2024-01-24T23:23:47.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 1 -- 172.21.15.132:0/4021785028 >> v1:172.21.15.132:6790/0 conn(0x7f973009f9c0 legacy=0x7f9730139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:47.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 1 -- 172.21.15.132:0/4021785028 >> v1:172.21.15.115:6789/0 conn(0x7f973009ff20 legacy=0x7f97300a0300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:47.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 10 monclient: _finish_hunting 0 2024-01-24T23:23:47.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 1 monclient: found mon.noname-i 2024-01-24T23:23:47.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:47.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:23:47.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 1 -- 172.21.15.132:0/4021785028 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97300a0a10 con 0x7f973013b2c0 2024-01-24T23:23:47.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 10 monclient: _finish_auth 0 2024-01-24T23:23:47.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 10 monclient: _check_auth_tickets 2024-01-24T23:23:47.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:47.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 1 -- 172.21.15.132:0/4021785028 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9728001ab0 con 0x7f973013b2c0 2024-01-24T23:23:47.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 5 monclient: authenticate success, global_id 5290 2024-01-24T23:23:47.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:47.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:47.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:23:47.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 10 monclient: dump: 2024-01-24T23:23:47.114 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:47.114 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:47.114 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:47.114 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:47.115 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:47.115 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:47.115 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:47.115 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:47.115 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:47.115 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:47.115 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:47.115 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:47.115 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:47.115 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:47.115 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:47.115 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:47.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:47.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 1 -- 172.21.15.132:0/4021785028 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9728001cc0 con 0x7f973013b2c0 2024-01-24T23:23:47.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:47.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:47.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 4 set_mon_vals no callback set 2024-01-24T23:23:47.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 1 -- 172.21.15.132:0/4021785028 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9728004b70 con 0x7f973013b2c0 2024-01-24T23:23:47.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:47.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:47.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f972d59a640 10 monclient: dump: 2024-01-24T23:23:47.116 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:47.116 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:47.116 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:47.116 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:47.116 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:47.116 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:47.116 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:47.116 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:47.116 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:47.116 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:47.117 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:47.117 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:47.117 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:47.117 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:47.117 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:47.117 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:47.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 10 monclient: shutdown 2024-01-24T23:23:47.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:47.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 1 -- 172.21.15.132:0/4021785028 >> v1:172.21.15.132:6792/0 conn(0x7f973013b2c0 legacy=0x7f973013d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:47.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 1 -- 172.21.15.132:0/4021785028 shutdown_connections 2024-01-24T23:23:47.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 1 -- 172.21.15.132:0/4021785028 >> 172.21.15.132:0/4021785028 conn(0x7f97300a5a10 msgr2=0x7f97300a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:47.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 1 -- 172.21.15.132:0/4021785028 shutdown_connections 2024-01-24T23:23:47.118 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 1 -- 172.21.15.132:0/4021785028 wait complete. 2024-01-24T23:23:47.118 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 10 monclient: build_initial_monmap 2024-01-24T23:23:47.118 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.103+0000 7f9734ffa640 10 monclient: monmap: 2024-01-24T23:23:47.118 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:47.118 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:47.118 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:47.118 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:47.118 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:47.118 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:47.118 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:47.118 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:47.118 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:47.118 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:47.118 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:47.118 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:47.118 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:47.119 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:47.119 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:47.119 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:47.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f9734ffa640 1 Processor -- start 2024-01-24T23:23:47.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f9734ffa640 1 -- start start 2024-01-24T23:23:47.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f9734ffa640 10 monclient: init 2024-01-24T23:23:47.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f9734ffa640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:47.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f9734ffa640 10 monclient: _add_conns ranks=[7,0,4,2,3,5,8,1,6] 2024-01-24T23:23:47.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f9734ffa640 10 monclient(hunting): picked mon.noname-i con 0x7f973009f9c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:47.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f9734ffa640 10 monclient(hunting): picked mon.noname-a con 0x7f973009ff20 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:47.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f9734ffa640 10 monclient(hunting): picked mon.noname-c con 0x7f973013b2c0 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:47.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f9734ffa640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973006f4b0 con 0x7f973009ff20 2024-01-24T23:23:47.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f9734ffa640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9730071680 con 0x7f973013b2c0 2024-01-24T23:23:47.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f9734ffa640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9730071960 con 0x7f973009f9c0 2024-01-24T23:23:47.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f9734ffa640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:47.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f9734ffa640 10 monclient(hunting): authenticate will time out at 1114.432715s 2024-01-24T23:23:47.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f972e59c640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f973009f9c0 0x7f97301cb5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:48932/0 (socket says 172.21.15.132:48932) 2024-01-24T23:23:47.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f972e59c640 1 -- 172.21.15.132:0/1699552014 learned_addr learned my addr 172.21.15.132:0/1699552014 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:47.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f972ed9d640 1 -- 172.21.15.132:0/1699552014 >> v1:172.21.15.115:6791/0 conn(0x7f973013b2c0 legacy=0x7f97301d3880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:23:47.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f972dd9b640 1 -- 172.21.15.132:0/1699552014 >> v1:172.21.15.115:6789/0 conn(0x7f973009ff20 legacy=0x7f97301cbcc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:47.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 1 -- 172.21.15.132:0/1699552014 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3436682460 0 0) 0x7f9730071960 con 0x7f973009f9c0 2024-01-24T23:23:47.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:47.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:47.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:47.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 10 monclient(hunting): my global_id is 5299 2024-01-24T23:23:47.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 1 -- 172.21.15.132:0/1699552014 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9710003530 con 0x7f973009f9c0 2024-01-24T23:23:47.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 1 -- 172.21.15.132:0/1699552014 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3646793031 0 0) 0x7f9710003530 con 0x7f973009f9c0 2024-01-24T23:23:47.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 1 -- 172.21.15.132:0/1699552014 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9730071960 con 0x7f973009f9c0 2024-01-24T23:23:47.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 1 -- 172.21.15.132:0/1699552014 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9720004330 con 0x7f973009f9c0 2024-01-24T23:23:47.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:47.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:23:47.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 10 monclient(hunting): dump: 2024-01-24T23:23:47.121 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:47.121 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:47.121 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:47.121 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:47.121 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:47.121 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:47.121 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:47.121 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:47.121 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:47.121 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:47.121 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:47.121 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:47.121 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:47.121 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:47.121 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:47.121 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:47.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:47.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 1 -- 172.21.15.132:0/1699552014 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3744003043 0 0) 0x7f9730071960 con 0x7f973009f9c0 2024-01-24T23:23:47.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 1 -- 172.21.15.132:0/1699552014 >> v1:172.21.15.115:6791/0 conn(0x7f973013b2c0 legacy=0x7f97301d3880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:47.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 1 -- 172.21.15.132:0/1699552014 >> v1:172.21.15.115:6789/0 conn(0x7f973009ff20 legacy=0x7f97301cbcc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:47.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:47.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 1 monclient: found mon.i 2024-01-24T23:23:47.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:47.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:47.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 1 -- 172.21.15.132:0/1699552014 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97301cc3d0 con 0x7f973009f9c0 2024-01-24T23:23:47.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 10 monclient: _finish_auth 0 2024-01-24T23:23:47.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:47.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:47.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f9734ffa640 5 monclient: authenticate success, global_id 5299 2024-01-24T23:23:47.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f9734ffa640 10 monclient: _renew_subs 2024-01-24T23:23:47.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f9734ffa640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:47.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f9734ffa640 1 -- 172.21.15.132:0/1699552014 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f973009dc70 con 0x7f973009f9c0 2024-01-24T23:23:47.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f9734ffa640 10 monclient: _renew_subs 2024-01-24T23:23:47.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f9734ffa640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:47.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f9734ffa640 1 -- 172.21.15.132:0/1699552014 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9730139fd0 con 0x7f973009f9c0 2024-01-24T23:23:47.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 1 -- 172.21.15.132:0/1699552014 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9720004650 con 0x7f973009f9c0 2024-01-24T23:23:47.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:47.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 1 -- 172.21.15.132:0/1699552014 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9720004b80 con 0x7f973009f9c0 2024-01-24T23:23:47.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:47.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f972cd99640 4 set_mon_vals no callback set 2024-01-24T23:23:47.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:47.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 10 monclient: dump: 2024-01-24T23:23:47.123 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:47.123 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:47.123 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:47.123 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:47.124 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:47.124 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:47.124 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:47.124 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:47.124 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:47.124 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:47.124 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:47.124 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:47.124 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:47.124 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:47.124 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:47.124 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:47.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 1 -- 172.21.15.132:0/1699552014 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f9720004d60 con 0x7f973009f9c0 2024-01-24T23:23:47.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f9734ffa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:47.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f9734ffa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:47.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f9734ffa640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:47.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f9734ffa640 1 -- 172.21.15.132:0/1699552014 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9714005090 con 0x7f973009f9c0 2024-01-24T23:23:47.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f971effd640 1 -- 172.21.15.132:0/1699552014 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(59..59 src has 1..59) v4 ==== 4195+0+0 (unknown 2858111438 0 0) 0x7f9720043ad0 con 0x7f973009f9c0 2024-01-24T23:23:47.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.107+0000 7f972dd9b640 10 monclient: get_auth_request con 0x7f9710037030 auth_method 0 2024-01-24T23:23:47.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.111+0000 7f971effd640 1 -- 172.21.15.132:0/1699552014 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f9720014960 con 0x7f973009f9c0 2024-01-24T23:23:47.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.111+0000 7f971effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:47.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.111+0000 7f971effd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:47.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.263+0000 7f9734ffa640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}] 2024-01-24T23:23:47.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.263+0000 7f9734ffa640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}] 2024-01-24T23:23:47.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.263+0000 7f9734ffa640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:47.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.263+0000 7f9734ffa640 1 -- 172.21.15.132:0/1699552014 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1} v 0) v1 -- 0x7f97140058c0 con 0x7f973009f9c0 2024-01-24T23:23:47.929 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.927+0000 7f971effd640 1 -- 172.21.15.132:0/1699552014 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}]=0 pool 'pinning_pool' created v60) v1 ==== 127+0+0 (unknown 935038851 0 0) 0x7f9720032fe0 con 0x7f973009f9c0 2024-01-24T23:23:47.929 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.927+0000 7f971effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}] 2024-01-24T23:23:47.930 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:47.927+0000 7f971effd640 10 monclient: _finish_command 2 = system:0 pool 'pinning_pool' created 2024-01-24T23:23:48.022 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.019+0000 7f9734ffa640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}] 2024-01-24T23:23:48.022 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.019+0000 7f9734ffa640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}] 2024-01-24T23:23:48.022 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.019+0000 7f9734ffa640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:48.022 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.019+0000 7f9734ffa640 1 -- 172.21.15.132:0/1699552014 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1} v 0) v1 -- 0x7f9714000bc0 con 0x7f973009f9c0 2024-01-24T23:23:48.024 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.023+0000 7f971effd640 1 -- 172.21.15.132:0/1699552014 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}]=0 pool 'pinning_pool' already exists v60) v1 ==== 134+0+0 (unknown 443850777 0 0) 0x7f9720013000 con 0x7f973009f9c0 2024-01-24T23:23:48.024 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.023+0000 7f971effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}] 2024-01-24T23:23:48.024 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.023+0000 7f971effd640 10 monclient: _finish_command 3 = system:0 pool 'pinning_pool' already exists 2024-01-24T23:23:48.024 INFO:tasks.workunit.client.0.smithi132.stderr:pool 'pinning_pool' already exists 2024-01-24T23:23:48.026 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.023+0000 7f9734ffa640 1 -- 172.21.15.132:0/1699552014 >> v1:172.21.15.132:6812/1835401922 conn(0x7f9710037030 legacy=0x7f97100394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:48.026 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.023+0000 7f9734ffa640 10 monclient: shutdown 2024-01-24T23:23:48.026 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.023+0000 7f9734ffa640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:48.026 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.023+0000 7f9734ffa640 1 -- 172.21.15.132:0/1699552014 >> v1:172.21.15.132:6792/0 conn(0x7f973009f9c0 legacy=0x7f97301cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:48.026 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.023+0000 7f9734ffa640 1 -- 172.21.15.132:0/1699552014 shutdown_connections 2024-01-24T23:23:48.026 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.023+0000 7f9734ffa640 1 -- 172.21.15.132:0/1699552014 >> 172.21.15.132:0/1699552014 conn(0x7f97300a5a10 msgr2=0x7f9730066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:48.026 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.023+0000 7f9734ffa640 1 -- 172.21.15.132:0/1699552014 shutdown_connections 2024-01-24T23:23:48.026 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.023+0000 7f9734ffa640 1 -- 172.21.15.132:0/1699552014 wait complete. 2024-01-24T23:23:48.041 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd crush rule rm replicated_rule 2024-01-24T23:23:48.041 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:23:48.041 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule rm replicated_rule 2024-01-24T23:23:48.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.155+0000 7f4fe30d6640 10 monclient: get_monmap_and_config 2024-01-24T23:23:48.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.155+0000 7f4fe30d6640 10 monclient: build_initial_monmap 2024-01-24T23:23:48.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.155+0000 7f4fe30d6640 10 monclient: monmap: 2024-01-24T23:23:48.159 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:48.159 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:48.159 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:48.159627+0000 2024-01-24T23:23:48.159 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:48.159627+0000 2024-01-24T23:23:48.159 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:48.159 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:48.159 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:48.159 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:48.159 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:48.160 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:48.160 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:48.160 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:48.160 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:48.160 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:48.160 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:48.160 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:48.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.155+0000 7f4fe30d6640 1 Processor -- start 2024-01-24T23:23:48.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.155+0000 7f4fe30d6640 1 -- start start 2024-01-24T23:23:48.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.155+0000 7f4fe30d6640 10 monclient: init 2024-01-24T23:23:48.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:48.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 10 monclient: _add_conns ranks=[3,6,1,4,0,8,5,7,2] 2024-01-24T23:23:48.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 10 monclient(hunting): picked mon.noname-g con 0x7f4fdc09bc10 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:48.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 10 monclient(hunting): picked mon.noname-d con 0x7f4fdc0a00d0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:48.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 10 monclient(hunting): picked mon.noname-f con 0x7f4fdc0a0560 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:48.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fdc064700 con 0x7f4fdc0a0560 2024-01-24T23:23:48.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fdc09bff0 con 0x7f4fdc09bc10 2024-01-24T23:23:48.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fdc0a1870 con 0x7f4fdc0a00d0 2024-01-24T23:23:48.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:48.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 10 monclient(hunting): authenticate will time out at 1115.484199s 2024-01-24T23:23:48.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe1673640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f4fdc0a0560 legacy=0x7f4fdc13bef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:48.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe0e72640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f4fdc09bc10 0x7f4fdc09f9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:52858/0 (socket says 172.21.15.132:52858) 2024-01-24T23:23:48.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe0e72640 1 -- 172.21.15.132:0/3713236785 learned_addr learned my addr 172.21.15.132:0/3713236785 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:48.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 1 -- 172.21.15.132:0/3713236785 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4196056629 0 0) 0x7f4fdc09bff0 con 0x7f4fdc09bc10 2024-01-24T23:23:48.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:48.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:48.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:48.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 10 monclient(hunting): my global_id is 5241 2024-01-24T23:23:48.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 1 -- 172.21.15.132:0/3713236785 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fc4003530 con 0x7f4fdc09bc10 2024-01-24T23:23:48.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 1 -- 172.21.15.132:0/3713236785 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2369338044 0 0) 0x7f4fdc0a1870 con 0x7f4fdc0a00d0 2024-01-24T23:23:48.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:48.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:48.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:48.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 10 monclient(hunting): my global_id is 5307 2024-01-24T23:23:48.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 1 -- 172.21.15.132:0/3713236785 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fdc09bff0 con 0x7f4fdc0a00d0 2024-01-24T23:23:48.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 1 -- 172.21.15.132:0/3713236785 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3501627619 0 0) 0x7f4fc4003530 con 0x7f4fdc09bc10 2024-01-24T23:23:48.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 1 -- 172.21.15.132:0/3713236785 >> v1:172.21.15.115:6792/0 conn(0x7f4fdc0a00d0 legacy=0x7f4fdc1387c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:48.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 1 -- 172.21.15.132:0/3713236785 >> v1:172.21.15.132:6789/0 conn(0x7f4fdc0a0560 legacy=0x7f4fdc13bef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:48.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 10 monclient: _finish_hunting 0 2024-01-24T23:23:48.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 1 monclient: found mon.noname-g 2024-01-24T23:23:48.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:48.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:23:48.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 1 -- 172.21.15.132:0/3713236785 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fdc0a0fa0 con 0x7f4fdc09bc10 2024-01-24T23:23:48.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 10 monclient: _finish_auth 0 2024-01-24T23:23:48.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 10 monclient: _check_auth_tickets 2024-01-24T23:23:48.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:48.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 1 -- 172.21.15.132:0/3713236785 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4fd80017e0 con 0x7f4fdc09bc10 2024-01-24T23:23:48.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 5 monclient: authenticate success, global_id 5241 2024-01-24T23:23:48.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:48.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:48.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:48.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 10 monclient: dump: 2024-01-24T23:23:48.173 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:48.173 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:48.173 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:48.173 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:48.173 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:48.173 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:48.174 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:48.174 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:48.174 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:48.174 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:48.174 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:48.174 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:48.174 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:48.174 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:48.174 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:48.174 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:48.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:48.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 1 -- 172.21.15.132:0/3713236785 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4fd8001a50 con 0x7f4fdc09bc10 2024-01-24T23:23:48.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:48.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:48.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 4 set_mon_vals no callback set 2024-01-24T23:23:48.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 1 -- 172.21.15.132:0/3713236785 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4fd8004bf0 con 0x7f4fdc09bc10 2024-01-24T23:23:48.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:48.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:48.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd37fe640 10 monclient: dump: 2024-01-24T23:23:48.175 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:48.175 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:48.175 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:48.175 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:48.175 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:48.175 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:48.175 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:48.175 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:48.175 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:48.175 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:48.176 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:48.176 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:48.176 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:48.176 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:48.176 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:48.176 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:48.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 10 monclient: shutdown 2024-01-24T23:23:48.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:48.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 1 -- 172.21.15.132:0/3713236785 >> v1:172.21.15.132:6790/0 conn(0x7f4fdc09bc10 legacy=0x7f4fdc09f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:48.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 1 -- 172.21.15.132:0/3713236785 shutdown_connections 2024-01-24T23:23:48.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 1 -- 172.21.15.132:0/3713236785 >> 172.21.15.132:0/3713236785 conn(0x7f4fdc0a5a10 msgr2=0x7f4fdc0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:48.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 1 -- 172.21.15.132:0/3713236785 shutdown_connections 2024-01-24T23:23:48.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 1 -- 172.21.15.132:0/3713236785 wait complete. 2024-01-24T23:23:48.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 10 monclient: build_initial_monmap 2024-01-24T23:23:48.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 10 monclient: monmap: 2024-01-24T23:23:48.177 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:48.177 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:48.177 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:48.177 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:48.177 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:48.177 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:48.177 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:48.177 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:48.177 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:48.177 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:48.177 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:48.177 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:48.177 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:48.177 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:48.177 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:48.178 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:48.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 1 Processor -- start 2024-01-24T23:23:48.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 1 -- start start 2024-01-24T23:23:48.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 10 monclient: init 2024-01-24T23:23:48.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:48.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 10 monclient: _add_conns ranks=[8,7,1,2,0,3,6,5,4] 2024-01-24T23:23:48.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 10 monclient(hunting): picked mon.noname-e con 0x7f4fdc09bc10 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:48.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 10 monclient(hunting): picked mon.noname-i con 0x7f4fdc0a00d0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:48.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 10 monclient(hunting): picked mon.noname-f con 0x7f4fdc0a0560 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:48.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fdc09c650 con 0x7f4fdc0a0560 2024-01-24T23:23:48.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fdc1a9b20 con 0x7f4fdc0a00d0 2024-01-24T23:23:48.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fdc1a9420 con 0x7f4fdc09bc10 2024-01-24T23:23:48.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:48.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe30d6640 10 monclient(hunting): authenticate will time out at 1115.487362s 2024-01-24T23:23:48.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fe1673640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f4fdc0a0560 legacy=0x7f4fdc1d3c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:48.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd3fff640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f4fdc0a00d0 0x7f4fdc1cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:48940/0 (socket says 172.21.15.132:48940) 2024-01-24T23:23:48.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd3fff640 1 -- 172.21.15.132:0/2157610721 learned_addr learned my addr 172.21.15.132:0/2157610721 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:48.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd17fa640 1 -- 172.21.15.132:0/2157610721 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1721651069 0 0) 0x7f4fdc1a9b20 con 0x7f4fdc0a00d0 2024-01-24T23:23:48.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd17fa640 10 monclient(hunting): _negotiate 2024-01-24T23:23:48.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd17fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:48.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd17fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:48.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd17fa640 10 monclient(hunting): my global_id is 5308 2024-01-24T23:23:48.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd17fa640 1 -- 172.21.15.132:0/2157610721 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fc8003530 con 0x7f4fdc0a00d0 2024-01-24T23:23:48.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd17fa640 1 -- 172.21.15.132:0/2157610721 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1367920776 0 0) 0x7f4fc8003530 con 0x7f4fdc0a00d0 2024-01-24T23:23:48.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.159+0000 7f4fd17fa640 1 -- 172.21.15.132:0/2157610721 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4fdc1a9b20 con 0x7f4fdc0a00d0 2024-01-24T23:23:48.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 1 -- 172.21.15.132:0/2157610721 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4fcc001a00 con 0x7f4fdc0a00d0 2024-01-24T23:23:48.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:48.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:23:48.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 10 monclient(hunting): dump: 2024-01-24T23:23:48.180 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:48.180 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:48.180 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:48.180 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:48.180 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:48.180 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:48.180 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:48.180 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:48.180 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:48.180 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:48.180 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:48.180 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:48.180 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:48.181 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:48.181 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:48.181 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:48.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:48.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 1 -- 172.21.15.132:0/2157610721 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 739547108 0 0) 0x7f4fdc1a9420 con 0x7f4fdc09bc10 2024-01-24T23:23:48.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 10 monclient(hunting): _negotiate 2024-01-24T23:23:48.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:48.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:48.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 10 monclient(hunting): my global_id is 84419 2024-01-24T23:23:48.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 1 -- 172.21.15.132:0/2157610721 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fc8003530 con 0x7f4fdc09bc10 2024-01-24T23:23:48.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 1 -- 172.21.15.132:0/2157610721 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2717338098 0 0) 0x7f4fdc1a9b20 con 0x7f4fdc0a00d0 2024-01-24T23:23:48.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 1 -- 172.21.15.132:0/2157610721 >> v1:172.21.15.115:6793/0 conn(0x7f4fdc09bc10 legacy=0x7f4fdc1cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:48.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 1 -- 172.21.15.132:0/2157610721 >> v1:172.21.15.132:6789/0 conn(0x7f4fdc0a0560 legacy=0x7f4fdc1d3c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:48.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 10 monclient: _finish_hunting 0 2024-01-24T23:23:48.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 1 monclient: found mon.i 2024-01-24T23:23:48.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:48.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:48.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 1 -- 172.21.15.132:0/2157610721 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fdc1a9700 con 0x7f4fdc0a00d0 2024-01-24T23:23:48.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 10 monclient: _finish_auth 0 2024-01-24T23:23:48.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 10 monclient: _check_auth_tickets 2024-01-24T23:23:48.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:48.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fe30d6640 5 monclient: authenticate success, global_id 5308 2024-01-24T23:23:48.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fe30d6640 10 monclient: _renew_subs 2024-01-24T23:23:48.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fe30d6640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:48.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fe30d6640 1 -- 172.21.15.132:0/2157610721 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fdc071fe0 con 0x7f4fdc0a00d0 2024-01-24T23:23:48.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fe30d6640 10 monclient: _renew_subs 2024-01-24T23:23:48.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fe30d6640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:48.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fe30d6640 1 -- 172.21.15.132:0/2157610721 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fdc1a8e60 con 0x7f4fdc0a00d0 2024-01-24T23:23:48.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 1 -- 172.21.15.132:0/2157610721 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4fcc003950 con 0x7f4fdc0a00d0 2024-01-24T23:23:48.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:48.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 1 -- 172.21.15.132:0/2157610721 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4fcc003e80 con 0x7f4fdc0a00d0 2024-01-24T23:23:48.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:48.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd27fc640 4 set_mon_vals no callback set 2024-01-24T23:23:48.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:48.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 10 monclient: dump: 2024-01-24T23:23:48.183 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:48.183 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:48.183 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:48.183 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:48.183 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:48.183 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:48.183 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:48.183 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:48.183 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:48.183 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:48.183 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:48.184 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:48.184 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:48.184 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:48.184 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:48.184 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:48.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fe30d6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:48.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fe30d6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:48.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fe30d6640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:48.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fe30d6640 1 -- 172.21.15.132:0/2157610721 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fbc005090 con 0x7f4fdc0a00d0 2024-01-24T23:23:48.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 1 -- 172.21.15.132:0/2157610721 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f4fcc004060 con 0x7f4fdc0a00d0 2024-01-24T23:23:48.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fd17fa640 1 -- 172.21.15.132:0/2157610721 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(60..60 src has 1..60) v4 ==== 4546+0+0 (unknown 1863356182 0 0) 0x7f4fcc043a90 con 0x7f4fdc0a00d0 2024-01-24T23:23:48.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.163+0000 7f4fe0e72640 10 monclient: get_auth_request con 0x7f4fc8036f90 auth_method 0 2024-01-24T23:23:48.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.167+0000 7f4fd17fa640 1 -- 172.21.15.132:0/2157610721 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f4fcc014ba0 con 0x7f4fdc0a00d0 2024-01-24T23:23:48.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.167+0000 7f4fd17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:48.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.167+0000 7f4fd17fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:48.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.323+0000 7f4fe30d6640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "replicated_rule"}] 2024-01-24T23:23:48.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.323+0000 7f4fe30d6640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rm", "name": "replicated_rule"}] 2024-01-24T23:23:48.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.323+0000 7f4fe30d6640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:48.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.323+0000 7f4fe30d6640 1 -- 172.21.15.132:0/2157610721 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rule rm", "name": "replicated_rule"} v 0) v1 -- 0x7f4fbc0058c0 con 0x7f4fdc0a00d0 2024-01-24T23:23:48.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.327+0000 7f4fd17fa640 1 -- 172.21.15.132:0/2157610721 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "replicated_rule"}]=-16 crush rule replicated_rule (0) is in use v60) v1 ==== 132+0+0 (unknown 2085096728 0 0) 0x7f4fcc013010 con 0x7f4fdc0a00d0 2024-01-24T23:23:48.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.327+0000 7f4fd17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rm", "name": "replicated_rule"}] 2024-01-24T23:23:48.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.327+0000 7f4fd17fa640 10 monclient: _finish_command 2 = mon:16 crush rule replicated_rule (0) is in use 2024-01-24T23:23:48.329 INFO:tasks.workunit.client.0.smithi132.stderr:Error EBUSY: crush rule replicated_rule (0) is in use 2024-01-24T23:23:48.334 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:23:48.334 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool rm pinning_pool pinning_pool --yes-i-really-really-mean-it 2024-01-24T23:23:48.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.443+0000 7f43b8fe2640 10 monclient: get_monmap_and_config 2024-01-24T23:23:48.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.443+0000 7f43b8fe2640 10 monclient: build_initial_monmap 2024-01-24T23:23:48.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.443+0000 7f43b8fe2640 10 monclient: monmap: 2024-01-24T23:23:48.446 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:48.446 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:48.446 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:48.447609+0000 2024-01-24T23:23:48.446 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:48.447609+0000 2024-01-24T23:23:48.446 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:48.447 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:48.447 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:48.447 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:48.447 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:48.447 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:48.447 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:48.447 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:48.447 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:48.447 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:48.447 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:48.447 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:48.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 1 Processor -- start 2024-01-24T23:23:48.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 1 -- start start 2024-01-24T23:23:48.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 10 monclient: init 2024-01-24T23:23:48.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:48.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 10 monclient: _add_conns ranks=[5,2,3,1,4,8,7,0,6] 2024-01-24T23:23:48.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 10 monclient(hunting): picked mon.noname-h con 0x7f43b4134d00 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:48.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 10 monclient(hunting): picked mon.noname-b con 0x7f43b41348d0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:48.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 10 monclient(hunting): picked mon.noname-g con 0x7f43b4071ea0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:48.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b4064700 con 0x7f43b41348d0 2024-01-24T23:23:48.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b4071ab0 con 0x7f43b4071ea0 2024-01-24T23:23:48.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b41357f0 con 0x7f43b4134d00 2024-01-24T23:23:48.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:48.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 10 monclient(hunting): authenticate will time out at 1115.773074s 2024-01-24T23:23:48.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b259c640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f43b4134d00 0x7f43b41350e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:44140/0 (socket says 172.21.15.132:44140) 2024-01-24T23:23:48.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b259c640 1 -- 172.21.15.132:0/2648979440 learned_addr learned my addr 172.21.15.132:0/2648979440 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:48.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 1 -- 172.21.15.132:0/2648979440 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3916155863 0 0) 0x7f43b41357f0 con 0x7f43b4134d00 2024-01-24T23:23:48.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 10 monclient(hunting): _negotiate 2024-01-24T23:23:48.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:48.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:48.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 10 monclient(hunting): my global_id is 5207 2024-01-24T23:23:48.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 1 -- 172.21.15.132:0/2648979440 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f439c003530 con 0x7f43b4134d00 2024-01-24T23:23:48.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 1 -- 172.21.15.132:0/2648979440 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3212656514 0 0) 0x7f43b4071ab0 con 0x7f43b4071ea0 2024-01-24T23:23:48.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 10 monclient(hunting): _negotiate 2024-01-24T23:23:48.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:48.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:48.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 10 monclient(hunting): my global_id is 5250 2024-01-24T23:23:48.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 1 -- 172.21.15.132:0/2648979440 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43b41357f0 con 0x7f43b4071ea0 2024-01-24T23:23:48.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 1 -- 172.21.15.132:0/2648979440 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3468480054 0 0) 0x7f43b41357f0 con 0x7f43b4071ea0 2024-01-24T23:23:48.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 1 -- 172.21.15.132:0/2648979440 >> v1:172.21.15.132:6791/0 conn(0x7f43b4134d00 legacy=0x7f43b41350e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:48.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 1 -- 172.21.15.132:0/2648979440 >> v1:172.21.15.115:6790/0 conn(0x7f43b41348d0 legacy=0x7f43b4139c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:48.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 10 monclient: _finish_hunting 0 2024-01-24T23:23:48.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 1 monclient: found mon.noname-g 2024-01-24T23:23:48.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:48.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:23:48.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 1 -- 172.21.15.132:0/2648979440 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b4070c60 con 0x7f43b4071ea0 2024-01-24T23:23:48.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 10 monclient: _finish_auth 0 2024-01-24T23:23:48.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 10 monclient: _check_auth_tickets 2024-01-24T23:23:48.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:48.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 1 -- 172.21.15.132:0/2648979440 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f43ac001b40 con 0x7f43b4071ea0 2024-01-24T23:23:48.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 5 monclient: authenticate success, global_id 5250 2024-01-24T23:23:48.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:48.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:48.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:48.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 10 monclient: dump: 2024-01-24T23:23:48.458 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:48.458 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:48.458 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:48.458 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:48.458 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:48.458 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:48.458 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:48.458 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:48.458 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:48.458 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:48.458 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:48.458 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:48.458 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:48.458 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:48.459 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:48.459 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:48.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:48.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 1 -- 172.21.15.132:0/2648979440 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f43ac001d60 con 0x7f43b4071ea0 2024-01-24T23:23:48.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:48.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 1 -- 172.21.15.132:0/2648979440 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f43ac003ce0 con 0x7f43b4071ea0 2024-01-24T23:23:48.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:48.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 4 set_mon_vals no callback set 2024-01-24T23:23:48.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:48.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:48.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b159a640 10 monclient: dump: 2024-01-24T23:23:48.459 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:48.459 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:48.459 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:48.460 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:48.460 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:48.460 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:48.460 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:48.460 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:48.460 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:48.460 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:48.460 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:48.460 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:48.460 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:48.460 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:48.460 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:48.460 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:48.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 10 monclient: shutdown 2024-01-24T23:23:48.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:48.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 1 -- 172.21.15.132:0/2648979440 >> v1:172.21.15.132:6790/0 conn(0x7f43b4071ea0 legacy=0x7f43b413d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:48.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 1 -- 172.21.15.132:0/2648979440 shutdown_connections 2024-01-24T23:23:48.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 1 -- 172.21.15.132:0/2648979440 >> 172.21.15.132:0/2648979440 conn(0x7f43b406aaf0 msgr2=0x7f43b40a24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:48.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 1 -- 172.21.15.132:0/2648979440 shutdown_connections 2024-01-24T23:23:48.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 1 -- 172.21.15.132:0/2648979440 wait complete. 2024-01-24T23:23:48.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 10 monclient: build_initial_monmap 2024-01-24T23:23:48.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 10 monclient: monmap: 2024-01-24T23:23:48.461 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:48.461 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:48.461 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:48.461 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:48.461 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:48.462 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:48.462 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:48.462 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:48.462 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:48.462 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:48.462 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:48.462 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:48.462 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:48.462 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:48.462 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:48.462 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:48.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 1 Processor -- start 2024-01-24T23:23:48.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 1 -- start start 2024-01-24T23:23:48.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 10 monclient: init 2024-01-24T23:23:48.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:48.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 10 monclient: _add_conns ranks=[8,7,6,5,4,0,3,1,2] 2024-01-24T23:23:48.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 10 monclient(hunting): picked mon.noname-e con 0x7f43b4071ea0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:48.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 10 monclient(hunting): picked mon.noname-i con 0x7f43b41348d0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:48.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 10 monclient(hunting): picked mon.noname-d con 0x7f43b4134d00 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:48.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b411c660 con 0x7f43b4134d00 2024-01-24T23:23:48.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b409ecd0 con 0x7f43b41348d0 2024-01-24T23:23:48.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b412f130 con 0x7f43b4071ea0 2024-01-24T23:23:48.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:48.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b8fe2640 10 monclient(hunting): authenticate will time out at 1115.775831s 2024-01-24T23:23:48.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b1d9b640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f43b41348d0 0x7f43b41ce200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:48946/0 (socket says 172.21.15.132:48946) 2024-01-24T23:23:48.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.447+0000 7f43b1d9b640 1 -- 172.21.15.132:0/2109693788 learned_addr learned my addr 172.21.15.132:0/2109693788 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:48.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 1 -- 172.21.15.132:0/2109693788 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2404759747 0 0) 0x7f43b409ecd0 con 0x7f43b41348d0 2024-01-24T23:23:48.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:48.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:48.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:48.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 10 monclient(hunting): my global_id is 5317 2024-01-24T23:23:48.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 1 -- 172.21.15.132:0/2109693788 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4394003530 con 0x7f43b41348d0 2024-01-24T23:23:48.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 1 -- 172.21.15.132:0/2109693788 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 538200253 0 0) 0x7f4394003530 con 0x7f43b41348d0 2024-01-24T23:23:48.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 1 -- 172.21.15.132:0/2109693788 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f43b409ecd0 con 0x7f43b41348d0 2024-01-24T23:23:48.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 1 -- 172.21.15.132:0/2109693788 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f43a8001aa0 con 0x7f43b41348d0 2024-01-24T23:23:48.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:48.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:23:48.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 10 monclient(hunting): dump: 2024-01-24T23:23:48.464 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:48.464 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:48.464 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:48.464 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:48.464 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:48.464 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:48.465 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:48.465 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:48.465 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:48.465 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:48.465 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:48.465 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:48.465 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:48.465 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:48.465 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:48.465 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:48.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:48.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 1 -- 172.21.15.132:0/2109693788 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2462856642 0 0) 0x7f43b409ecd0 con 0x7f43b41348d0 2024-01-24T23:23:48.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 1 -- 172.21.15.132:0/2109693788 >> v1:172.21.15.115:6793/0 conn(0x7f43b4071ea0 legacy=0x7f43b41c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:48.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 1 -- 172.21.15.132:0/2109693788 >> v1:172.21.15.115:6792/0 conn(0x7f43b4134d00 legacy=0x7f43b41d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:48.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:48.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 1 monclient: found mon.i 2024-01-24T23:23:48.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:48.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:48.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 1 -- 172.21.15.132:0/2109693788 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b411c380 con 0x7f43b41348d0 2024-01-24T23:23:48.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 10 monclient: _finish_auth 0 2024-01-24T23:23:48.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:48.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:48.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43b8fe2640 5 monclient: authenticate success, global_id 5317 2024-01-24T23:23:48.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43b8fe2640 10 monclient: _renew_subs 2024-01-24T23:23:48.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43b8fe2640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:48.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43b8fe2640 1 -- 172.21.15.132:0/2109693788 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b409e5d0 con 0x7f43b41348d0 2024-01-24T23:23:48.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43b8fe2640 10 monclient: _renew_subs 2024-01-24T23:23:48.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43b8fe2640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:48.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43b8fe2640 1 -- 172.21.15.132:0/2109693788 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b411d540 con 0x7f43b41348d0 2024-01-24T23:23:48.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 1 -- 172.21.15.132:0/2109693788 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f43a80038d0 con 0x7f43b41348d0 2024-01-24T23:23:48.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:48.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 1 -- 172.21.15.132:0/2109693788 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f43a8003d80 con 0x7f43b41348d0 2024-01-24T23:23:48.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:48.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43b0d99640 4 set_mon_vals no callback set 2024-01-24T23:23:48.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:48.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 10 monclient: dump: 2024-01-24T23:23:48.467 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:48.467 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:48.467 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:48.467 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:48.467 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:48.467 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:48.467 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:48.467 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:48.467 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:48.467 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:48.468 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:48.468 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:48.468 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:48.468 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:48.468 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:48.468 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:48.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43b8fe2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:48.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43b8fe2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:48.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43b8fe2640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:48.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43b8fe2640 1 -- 172.21.15.132:0/2109693788 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4398005090 con 0x7f43b41348d0 2024-01-24T23:23:48.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 1 -- 172.21.15.132:0/2109693788 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f43a8003ee0 con 0x7f43b41348d0 2024-01-24T23:23:48.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43a2ffd640 1 -- 172.21.15.132:0/2109693788 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(60..60 src has 1..60) v4 ==== 4546+0+0 (unknown 1863356182 0 0) 0x7f43a8043b60 con 0x7f43b41348d0 2024-01-24T23:23:48.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.451+0000 7f43b259c640 10 monclient: get_auth_request con 0x7f4394037020 auth_method 0 2024-01-24T23:23:48.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.455+0000 7f43a2ffd640 1 -- 172.21.15.132:0/2109693788 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f43a8014830 con 0x7f43b41348d0 2024-01-24T23:23:48.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.455+0000 7f43a2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:48.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.455+0000 7f43a2ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:48.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.595+0000 7f43b8fe2640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}] 2024-01-24T23:23:48.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.595+0000 7f43b8fe2640 10 monclient: _send_command 2 [{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}] 2024-01-24T23:23:48.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.595+0000 7f43b8fe2640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:48.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.595+0000 7f43b8fe2640 1 -- 172.21.15.132:0/2109693788 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4398005340 con 0x7f43b41348d0 2024-01-24T23:23:48.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.935+0000 7f43a2ffd640 1 -- 172.21.15.132:0/2109693788 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}]=0 pool 'pinning_pool' removed v61) v1 ==== 172+0+0 (unknown 2129805599 0 0) 0x7f43a8012ed0 con 0x7f43b41348d0 2024-01-24T23:23:48.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.935+0000 7f43a2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}] 2024-01-24T23:23:48.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:48.935+0000 7f43a2ffd640 10 monclient: _finish_command 2 = system:0 pool 'pinning_pool' removed 2024-01-24T23:23:49.034 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.031+0000 7f43b8fe2640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}] 2024-01-24T23:23:49.034 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.031+0000 7f43b8fe2640 10 monclient: _send_command 3 [{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}] 2024-01-24T23:23:49.034 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.031+0000 7f43b8fe2640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:23:49.034 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.031+0000 7f43b8fe2640 1 -- 172.21.15.132:0/2109693788 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4398001160 con 0x7f43b41348d0 2024-01-24T23:23:49.035 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.035+0000 7f43a2ffd640 1 -- 172.21.15.132:0/2109693788 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}]=0 pool 'pinning_pool' does not exist v61) v1 ==== 179+0+0 (unknown 3879824278 0 0) 0x7f43a8014650 con 0x7f43b41348d0 2024-01-24T23:23:49.035 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.035+0000 7f43a2ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}] 2024-01-24T23:23:49.035 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.035+0000 7f43a2ffd640 10 monclient: _finish_command 3 = system:0 pool 'pinning_pool' does not exist 2024-01-24T23:23:49.035 INFO:tasks.workunit.client.0.smithi132.stderr:pool 'pinning_pool' does not exist 2024-01-24T23:23:49.038 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.035+0000 7f43b8fe2640 1 -- 172.21.15.132:0/2109693788 >> v1:172.21.15.132:6812/1835401922 conn(0x7f4394037020 legacy=0x7f43940394e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:49.038 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.035+0000 7f43b8fe2640 10 monclient: shutdown 2024-01-24T23:23:49.038 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.035+0000 7f43b8fe2640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:49.038 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.035+0000 7f43b8fe2640 1 -- 172.21.15.132:0/2109693788 >> v1:172.21.15.132:6792/0 conn(0x7f43b41348d0 legacy=0x7f43b41ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:49.038 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.035+0000 7f43b8fe2640 1 -- 172.21.15.132:0/2109693788 shutdown_connections 2024-01-24T23:23:49.038 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.035+0000 7f43b8fe2640 1 -- 172.21.15.132:0/2109693788 >> 172.21.15.132:0/2109693788 conn(0x7f43b406aaf0 msgr2=0x7f43b40a0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:49.038 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.035+0000 7f43b8fe2640 1 -- 172.21.15.132:0/2109693788 shutdown_connections 2024-01-24T23:23:49.038 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.035+0000 7f43b8fe2640 1 -- 172.21.15.132:0/2109693788 wait complete. 2024-01-24T23:23:49.052 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd crush add-bucket foo osd 2024-01-24T23:23:49.052 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:23:49.052 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket foo osd 2024-01-24T23:23:49.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.167+0000 7ff71ca52640 10 monclient: get_monmap_and_config 2024-01-24T23:23:49.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.167+0000 7ff71ca52640 10 monclient: build_initial_monmap 2024-01-24T23:23:49.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.167+0000 7ff71ca52640 10 monclient: monmap: 2024-01-24T23:23:49.170 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:49.170 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:49.170 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:49.171191+0000 2024-01-24T23:23:49.170 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:49.171191+0000 2024-01-24T23:23:49.170 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:49.170 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:49.170 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:49.170 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:49.170 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:49.170 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:49.170 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:49.170 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:49.170 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:49.171 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:49.171 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:49.171 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:49.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.167+0000 7ff71ca52640 1 Processor -- start 2024-01-24T23:23:49.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.167+0000 7ff71ca52640 1 -- start start 2024-01-24T23:23:49.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.167+0000 7ff71ca52640 10 monclient: init 2024-01-24T23:23:49.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.167+0000 7ff71ca52640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:49.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.167+0000 7ff71ca52640 10 monclient: _add_conns ranks=[3,5,7,1,2,0,4,6,8] 2024-01-24T23:23:49.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.167+0000 7ff71ca52640 10 monclient(hunting): picked mon.noname-g con 0x7ff718133d50 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:49.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.167+0000 7ff71ca52640 10 monclient(hunting): picked mon.noname-h con 0x7ff7180720a0 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:49.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.167+0000 7ff71ca52640 10 monclient(hunting): picked mon.noname-i con 0x7ff7180a1b50 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:49.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.167+0000 7ff71ca52640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff718064700 con 0x7ff718133d50 2024-01-24T23:23:49.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.167+0000 7ff71ca52640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff718070a80 con 0x7ff7180720a0 2024-01-24T23:23:49.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.167+0000 7ff71ca52640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff718070d60 con 0x7ff7180a1b50 2024-01-24T23:23:49.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.167+0000 7ff71ca52640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:49.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.167+0000 7ff71ca52640 10 monclient(hunting): authenticate will time out at 1116.495987s 2024-01-24T23:23:49.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71659c640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7ff718133d50 0x7ff718134130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:52876/0 (socket says 172.21.15.132:52876) 2024-01-24T23:23:49.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71659c640 1 -- 172.21.15.132:0/3849299926 learned_addr learned my addr 172.21.15.132:0/3849299926 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:49.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff716d9d640 1 --1- 172.21.15.132:0/3849299926 >> v1:172.21.15.132:6792/0 conn(0x7ff7180a1b50 0x7ff71813d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:48954/0 (socket says 172.21.15.132:48954) 2024-01-24T23:23:49.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff715d9b640 1 --1- 172.21.15.132:0/3849299926 >> v1:172.21.15.132:6791/0 conn(0x7ff7180720a0 0x7ff718139ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:44148/0 (socket says 172.21.15.132:44148) 2024-01-24T23:23:49.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 1 -- 172.21.15.132:0/3849299926 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4107668722 0 0) 0x7ff718070d60 con 0x7ff7180a1b50 2024-01-24T23:23:49.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 10 monclient(hunting): _negotiate 2024-01-24T23:23:49.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:49.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:49.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 10 monclient(hunting): my global_id is 5326 2024-01-24T23:23:49.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 1 -- 172.21.15.132:0/3849299926 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff700003530 con 0x7ff7180a1b50 2024-01-24T23:23:49.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 1 -- 172.21.15.132:0/3849299926 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1328227084 0 0) 0x7ff718064700 con 0x7ff718133d50 2024-01-24T23:23:49.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 10 monclient(hunting): _negotiate 2024-01-24T23:23:49.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:49.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:49.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 10 monclient(hunting): my global_id is 5259 2024-01-24T23:23:49.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 1 -- 172.21.15.132:0/3849299926 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff718070d60 con 0x7ff718133d50 2024-01-24T23:23:49.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 1 -- 172.21.15.132:0/3849299926 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 633937147 0 0) 0x7ff718070a80 con 0x7ff7180720a0 2024-01-24T23:23:49.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 10 monclient(hunting): _negotiate 2024-01-24T23:23:49.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:49.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:49.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 10 monclient(hunting): my global_id is 5216 2024-01-24T23:23:49.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 1 -- 172.21.15.132:0/3849299926 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff718064700 con 0x7ff7180720a0 2024-01-24T23:23:49.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 1 -- 172.21.15.132:0/3849299926 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4292366985 0 0) 0x7ff718070d60 con 0x7ff718133d50 2024-01-24T23:23:49.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 1 -- 172.21.15.132:0/3849299926 >> v1:172.21.15.132:6792/0 conn(0x7ff7180a1b50 legacy=0x7ff71813d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:49.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 1 -- 172.21.15.132:0/3849299926 >> v1:172.21.15.132:6791/0 conn(0x7ff7180720a0 legacy=0x7ff718139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:49.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 10 monclient: _finish_hunting 0 2024-01-24T23:23:49.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 1 monclient: found mon.noname-g 2024-01-24T23:23:49.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:49.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:23:49.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 1 -- 172.21.15.132:0/3849299926 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff718071040 con 0x7ff718133d50 2024-01-24T23:23:49.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 10 monclient: _finish_auth 0 2024-01-24T23:23:49.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 10 monclient: _check_auth_tickets 2024-01-24T23:23:49.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:49.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 1 -- 172.21.15.132:0/3849299926 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff7080017a0 con 0x7ff718133d50 2024-01-24T23:23:49.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:49.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:49.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 10 monclient: dump: 2024-01-24T23:23:49.180 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:49.180 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:49.180 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:49.180 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:49.180 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:49.181 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:49.181 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:49.181 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:49.181 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:49.181 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:49.181 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:49.181 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:49.181 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:49.181 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:49.181 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:49.181 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:49.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 5 monclient: authenticate success, global_id 5259 2024-01-24T23:23:49.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:49.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 1 -- 172.21.15.132:0/3849299926 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff708001a40 con 0x7ff718133d50 2024-01-24T23:23:49.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:49.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 1 -- 172.21.15.132:0/3849299926 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff708004bf0 con 0x7ff718133d50 2024-01-24T23:23:49.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:49.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:49.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71559a640 10 monclient: dump: 2024-01-24T23:23:49.182 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:49.182 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:49.182 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:49.182 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:49.182 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:49.182 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:49.182 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:49.182 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:49.182 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:49.182 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:49.182 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:49.182 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:49.183 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:49.183 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:49.183 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:49.183 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:49.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:49.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 4 set_mon_vals no callback set 2024-01-24T23:23:49.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 10 monclient: shutdown 2024-01-24T23:23:49.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:49.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 1 -- 172.21.15.132:0/3849299926 >> v1:172.21.15.132:6790/0 conn(0x7ff718133d50 legacy=0x7ff718134130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:49.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 1 -- 172.21.15.132:0/3849299926 shutdown_connections 2024-01-24T23:23:49.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 1 -- 172.21.15.132:0/3849299926 >> 172.21.15.132:0/3849299926 conn(0x7ff7180a7150 msgr2=0x7ff7180a7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:49.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 1 -- 172.21.15.132:0/3849299926 shutdown_connections 2024-01-24T23:23:49.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 1 -- 172.21.15.132:0/3849299926 wait complete. 2024-01-24T23:23:49.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 10 monclient: build_initial_monmap 2024-01-24T23:23:49.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 10 monclient: monmap: 2024-01-24T23:23:49.184 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:49.184 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:49.184 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:49.184 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:49.187 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:49.187 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:49.187 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:49.187 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:49.188 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:49.188 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:49.188 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:49.188 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:49.188 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:49.188 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:49.188 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:49.188 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:49.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 1 Processor -- start 2024-01-24T23:23:49.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 1 -- start start 2024-01-24T23:23:49.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 10 monclient: init 2024-01-24T23:23:49.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:49.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 10 monclient: _add_conns ranks=[6,5,2,0,1,4,3,8,7] 2024-01-24T23:23:49.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 10 monclient(hunting): picked mon.noname-d con 0x7ff7180720a0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:49.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 10 monclient(hunting): picked mon.noname-h con 0x7ff7180a1b50 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:49.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 10 monclient(hunting): picked mon.noname-b con 0x7ff718133d50 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:49.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7181c9670 con 0x7ff718133d50 2024-01-24T23:23:49.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff71813a1c0 con 0x7ff7180a1b50 2024-01-24T23:23:49.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff71813a4a0 con 0x7ff7180720a0 2024-01-24T23:23:49.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:49.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 10 monclient(hunting): authenticate will time out at 1116.499033s 2024-01-24T23:23:49.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff715d9b640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7ff7180a1b50 0x7ff71813d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:44150/0 (socket says 172.21.15.132:44150) 2024-01-24T23:23:49.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff715d9b640 1 -- 172.21.15.132:0/1692488280 learned_addr learned my addr 172.21.15.132:0/1692488280 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:49.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff706ffd640 1 -- 172.21.15.132:0/1692488280 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1227515782 0 0) 0x7ff71813a1c0 con 0x7ff7180a1b50 2024-01-24T23:23:49.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff706ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:49.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff706ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:49.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff706ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:49.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff706ffd640 10 monclient(hunting): my global_id is 5225 2024-01-24T23:23:49.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff706ffd640 1 -- 172.21.15.132:0/1692488280 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6f8003530 con 0x7ff7180a1b50 2024-01-24T23:23:49.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff706ffd640 1 -- 172.21.15.132:0/1692488280 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 129790656 0 0) 0x7ff6f8003530 con 0x7ff7180a1b50 2024-01-24T23:23:49.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff706ffd640 1 -- 172.21.15.132:0/1692488280 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff71813a1c0 con 0x7ff7180a1b50 2024-01-24T23:23:49.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff706ffd640 1 -- 172.21.15.132:0/1692488280 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff70c001680 con 0x7ff7180a1b50 2024-01-24T23:23:49.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff706ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:49.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff706ffd640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:49.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff706ffd640 10 monclient(hunting): dump: 2024-01-24T23:23:49.190 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:49.190 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:49.190 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:49.190 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:49.190 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:49.190 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:49.190 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:49.190 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:49.190 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:49.190 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:49.191 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:49.191 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:49.191 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:49.191 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:49.191 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:49.191 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:49.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff706ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:49.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff706ffd640 1 -- 172.21.15.132:0/1692488280 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1134942686 0 0) 0x7ff71813a1c0 con 0x7ff7180a1b50 2024-01-24T23:23:49.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff706ffd640 1 -- 172.21.15.132:0/1692488280 >> v1:172.21.15.115:6792/0 conn(0x7ff7180720a0 legacy=0x7ff7181c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:49.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff706ffd640 1 -- 172.21.15.132:0/1692488280 >> v1:172.21.15.115:6790/0 conn(0x7ff718133d50 legacy=0x7ff7181d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:49.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff706ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:49.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff706ffd640 1 monclient: found mon.h 2024-01-24T23:23:49.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff706ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:49.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff706ffd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:49.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff706ffd640 1 -- 172.21.15.132:0/1692488280 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71813a780 con 0x7ff7180a1b50 2024-01-24T23:23:49.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff706ffd640 10 monclient: _finish_auth 0 2024-01-24T23:23:49.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff706ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:49.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff706ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:49.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 5 monclient: authenticate success, global_id 5225 2024-01-24T23:23:49.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 10 monclient: _renew_subs 2024-01-24T23:23:49.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:49.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 1 -- 172.21.15.132:0/1692488280 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff71813d8e0 con 0x7ff7180a1b50 2024-01-24T23:23:49.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 10 monclient: _renew_subs 2024-01-24T23:23:49.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:49.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.171+0000 7ff71ca52640 1 -- 172.21.15.132:0/1692488280 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff718137f30 con 0x7ff7180a1b50 2024-01-24T23:23:49.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.175+0000 7ff706ffd640 1 -- 172.21.15.132:0/1692488280 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff70c004aa0 con 0x7ff7180a1b50 2024-01-24T23:23:49.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.175+0000 7ff706ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:49.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.175+0000 7ff706ffd640 1 -- 172.21.15.132:0/1692488280 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff70c004f50 con 0x7ff7180a1b50 2024-01-24T23:23:49.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.175+0000 7ff706ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:49.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.175+0000 7ff714d99640 4 set_mon_vals no callback set 2024-01-24T23:23:49.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.175+0000 7ff706ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:49.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.175+0000 7ff706ffd640 10 monclient: dump: 2024-01-24T23:23:49.193 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:49.193 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:49.193 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:49.193 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:49.193 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:49.193 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:49.193 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:49.193 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:49.193 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:49.193 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:49.193 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:49.193 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:49.193 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:49.194 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:49.194 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:49.194 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:49.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.175+0000 7ff71ca52640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:49.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.175+0000 7ff71ca52640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:49.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.175+0000 7ff71ca52640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:49.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.175+0000 7ff71ca52640 1 -- 172.21.15.132:0/1692488280 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6fc005090 con 0x7ff7180a1b50 2024-01-24T23:23:49.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.175+0000 7ff706ffd640 1 -- 172.21.15.132:0/1692488280 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7ff70c00f9d0 con 0x7ff7180a1b50 2024-01-24T23:23:49.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.175+0000 7ff706ffd640 1 -- 172.21.15.132:0/1692488280 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 4195+0+0 (unknown 2295113258 0 0) 0x7ff70c0439d0 con 0x7ff7180a1b50 2024-01-24T23:23:49.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.175+0000 7ff71659c640 10 monclient: get_auth_request con 0x7ff6f8037020 auth_method 0 2024-01-24T23:23:49.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.179+0000 7ff706ffd640 1 -- 172.21.15.132:0/1692488280 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7ff70c014860 con 0x7ff7180a1b50 2024-01-24T23:23:49.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.179+0000 7ff706ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:49.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.179+0000 7ff706ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:49.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.339+0000 7ff71ca52640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "foo", "type": "osd"}] 2024-01-24T23:23:49.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.339+0000 7ff71ca52640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "osd"}] 2024-01-24T23:23:49.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.339+0000 7ff71ca52640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:49.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.339+0000 7ff71ca52640 1 -- 172.21.15.132:0/1692488280 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "foo", "type": "osd"} v 0) v1 -- 0x7ff6fc0058c0 con 0x7ff7180a1b50 2024-01-24T23:23:49.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.339+0000 7ff706ffd640 1 -- 172.21.15.132:0/1692488280 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "foo", "type": "osd"}]=-22 type 'osd' is for devices, not buckets v61) v1 ==== 136+0+0 (unknown 434897630 0 0) 0x7ff70c012f00 con 0x7ff7180a1b50 2024-01-24T23:23:49.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.339+0000 7ff706ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "osd"}] 2024-01-24T23:23:49.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.339+0000 7ff706ffd640 10 monclient: _finish_command 2 = mon:22 type 'osd' is for devices, not buckets 2024-01-24T23:23:49.343 INFO:tasks.workunit.client.0.smithi132.stderr:Error EINVAL: type 'osd' is for devices, not buckets 2024-01-24T23:23:49.348 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:23:49.348 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket foo root 2024-01-24T23:23:49.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.463+0000 7fb1ab448640 10 monclient: get_monmap_and_config 2024-01-24T23:23:49.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.463+0000 7fb1ab448640 10 monclient: build_initial_monmap 2024-01-24T23:23:49.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.463+0000 7fb1ab448640 10 monclient: monmap: 2024-01-24T23:23:49.466 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:49.466 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:49.466 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:49.467482+0000 2024-01-24T23:23:49.466 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:49.467482+0000 2024-01-24T23:23:49.466 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:49.466 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:49.466 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:49.467 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:49.467 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:49.467 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:49.467 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:49.467 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:49.467 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:49.467 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:49.467 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:49.467 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:49.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.463+0000 7fb1ab448640 1 Processor -- start 2024-01-24T23:23:49.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.463+0000 7fb1ab448640 1 -- start start 2024-01-24T23:23:49.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.463+0000 7fb1ab448640 10 monclient: init 2024-01-24T23:23:49.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:49.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 10 monclient: _add_conns ranks=[3,1,7,8,2,0,4,5,6] 2024-01-24T23:23:49.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 10 monclient(hunting): picked mon.noname-g con 0x7fb1a409ff20 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:49.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 10 monclient(hunting): picked mon.noname-f con 0x7fb1a409f9c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:49.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 10 monclient(hunting): picked mon.noname-i con 0x7fb1a413b2c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:49.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a4064700 con 0x7fb1a409f9c0 2024-01-24T23:23:49.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a409e760 con 0x7fb1a409ff20 2024-01-24T23:23:49.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a40a1160 con 0x7fb1a413b2c0 2024-01-24T23:23:49.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:49.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1a89e3640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fb1a409f9c0 legacy=0x7fb1a4139ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:49.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 10 monclient(hunting): authenticate will time out at 1116.792320s 2024-01-24T23:23:49.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1a91e4640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fb1a409ff20 0x7fb1a40a0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:52886/0 (socket says 172.21.15.132:52886) 2024-01-24T23:23:49.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1a91e4640 1 -- 172.21.15.132:0/2763527882 learned_addr learned my addr 172.21.15.132:0/2763527882 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:49.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 1 -- 172.21.15.132:0/2763527882 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2233804877 0 0) 0x7fb1a409e760 con 0x7fb1a409ff20 2024-01-24T23:23:49.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:49.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:49.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:49.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 10 monclient(hunting): my global_id is 5268 2024-01-24T23:23:49.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 1 -- 172.21.15.132:0/2763527882 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb190003530 con 0x7fb1a409ff20 2024-01-24T23:23:49.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 1 -- 172.21.15.132:0/2763527882 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2635483349 0 0) 0x7fb1a40a1160 con 0x7fb1a413b2c0 2024-01-24T23:23:49.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:49.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:49.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:49.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 10 monclient(hunting): my global_id is 5335 2024-01-24T23:23:49.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 1 -- 172.21.15.132:0/2763527882 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1a409e760 con 0x7fb1a413b2c0 2024-01-24T23:23:49.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 1 -- 172.21.15.132:0/2763527882 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 748182076 0 0) 0x7fb190003530 con 0x7fb1a409ff20 2024-01-24T23:23:49.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 1 -- 172.21.15.132:0/2763527882 >> v1:172.21.15.132:6792/0 conn(0x7fb1a413b2c0 legacy=0x7fb1a413d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:49.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 1 -- 172.21.15.132:0/2763527882 >> v1:172.21.15.132:6789/0 conn(0x7fb1a409f9c0 legacy=0x7fb1a4139ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:49.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:23:49.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 1 monclient: found mon.noname-g 2024-01-24T23:23:49.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:49.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:23:49.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 1 -- 172.21.15.132:0/2763527882 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a40a0a10 con 0x7fb1a409ff20 2024-01-24T23:23:49.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 10 monclient: _finish_auth 0 2024-01-24T23:23:49.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:23:49.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:49.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 1 -- 172.21.15.132:0/2763527882 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb1940017c0 con 0x7fb1a409ff20 2024-01-24T23:23:49.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:49.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:49.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 10 monclient: dump: 2024-01-24T23:23:49.477 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:49.477 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:49.477 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:49.477 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:49.477 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:49.477 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:49.478 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:49.478 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:49.478 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:49.478 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:49.478 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:49.478 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:49.478 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:49.478 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:49.478 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:49.478 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:49.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 5 monclient: authenticate success, global_id 5268 2024-01-24T23:23:49.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:49.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 1 -- 172.21.15.132:0/2763527882 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb194001a20 con 0x7fb1a409ff20 2024-01-24T23:23:49.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:49.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:49.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 4 set_mon_vals no callback set 2024-01-24T23:23:49.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 1 -- 172.21.15.132:0/2763527882 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb194004bf0 con 0x7fb1a409ff20 2024-01-24T23:23:49.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:49.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:49.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb19bfff640 10 monclient: dump: 2024-01-24T23:23:49.479 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:49.479 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:49.479 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:49.479 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:49.479 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:49.479 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:49.479 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:49.479 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:49.479 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:49.479 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:49.480 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:49.480 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:49.480 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:49.480 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:49.480 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:49.480 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:49.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 10 monclient: shutdown 2024-01-24T23:23:49.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:49.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 1 -- 172.21.15.132:0/2763527882 >> v1:172.21.15.132:6790/0 conn(0x7fb1a409ff20 legacy=0x7fb1a40a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:49.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 1 -- 172.21.15.132:0/2763527882 shutdown_connections 2024-01-24T23:23:49.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 1 -- 172.21.15.132:0/2763527882 >> 172.21.15.132:0/2763527882 conn(0x7fb1a40a5a10 msgr2=0x7fb1a40a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:49.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 1 -- 172.21.15.132:0/2763527882 shutdown_connections 2024-01-24T23:23:49.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 1 -- 172.21.15.132:0/2763527882 wait complete. 2024-01-24T23:23:49.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 10 monclient: build_initial_monmap 2024-01-24T23:23:49.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 10 monclient: monmap: 2024-01-24T23:23:49.481 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:49.481 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:49.481 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:49.481 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:49.481 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:49.481 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:49.481 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:49.481 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:49.481 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:49.481 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:49.482 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:49.482 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:49.482 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:49.482 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:49.483 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:49.483 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:49.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 1 Processor -- start 2024-01-24T23:23:49.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 1 -- start start 2024-01-24T23:23:49.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 10 monclient: init 2024-01-24T23:23:49.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:49.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 10 monclient: _add_conns ranks=[6,3,4,2,1,7,5,8,0] 2024-01-24T23:23:49.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 10 monclient(hunting): picked mon.noname-d con 0x7fb1a409f9c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:49.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 10 monclient(hunting): picked mon.noname-g con 0x7fb1a409ff20 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:49.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 10 monclient(hunting): picked mon.noname-c con 0x7fb1a413b2c0 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:49.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a40719d0 con 0x7fb1a409ff20 2024-01-24T23:23:49.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a409d2e0 con 0x7fb1a413b2c0 2024-01-24T23:23:49.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a4071680 con 0x7fb1a409f9c0 2024-01-24T23:23:49.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:49.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1ab448640 10 monclient(hunting): authenticate will time out at 1116.795269s 2024-01-24T23:23:49.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1a89e3640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fb1a409ff20 0x7fb1a409cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:52888/0 (socket says 172.21.15.132:52888) 2024-01-24T23:23:49.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1a89e3640 1 -- 172.21.15.132:0/300509920 learned_addr learned my addr 172.21.15.132:0/300509920 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:49.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb1a99e5640 1 -- 172.21.15.132:0/300509920 >> v1:172.21.15.115:6791/0 conn(0x7fb1a413b2c0 legacy=0x7fb1a41d36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:23:49.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb199ffb640 1 -- 172.21.15.132:0/300509920 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3310687059 0 0) 0x7fb1a0003aa0 con 0x7fb1a409ff20 2024-01-24T23:23:49.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb199ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:49.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb199ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:49.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb199ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:49.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb199ffb640 10 monclient(hunting): my global_id is 5277 2024-01-24T23:23:49.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb199ffb640 1 -- 172.21.15.132:0/300509920 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb188003530 con 0x7fb1a409ff20 2024-01-24T23:23:49.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb199ffb640 1 -- 172.21.15.132:0/300509920 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2380699093 0 0) 0x7fb188003530 con 0x7fb1a409ff20 2024-01-24T23:23:49.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb199ffb640 1 -- 172.21.15.132:0/300509920 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb1a0003aa0 con 0x7fb1a409ff20 2024-01-24T23:23:49.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb199ffb640 1 -- 172.21.15.132:0/300509920 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb1a0004490 con 0x7fb1a409ff20 2024-01-24T23:23:49.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb199ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:49.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb199ffb640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:49.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb199ffb640 10 monclient(hunting): dump: 2024-01-24T23:23:49.486 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:49.486 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:49.486 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:49.486 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:49.486 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:49.486 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:49.486 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:49.486 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:49.486 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:49.486 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:49.486 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:49.486 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:49.487 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:49.487 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:49.487 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:49.487 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:49.487 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.467+0000 7fb199ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:49.487 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 1 -- 172.21.15.132:0/300509920 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3072273823 0 0) 0x7fb1a4071680 con 0x7fb1a409f9c0 2024-01-24T23:23:49.487 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:49.487 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:49.487 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:49.487 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 10 monclient(hunting): my global_id is 5334 2024-01-24T23:23:49.487 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 1 -- 172.21.15.132:0/300509920 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb188003530 con 0x7fb1a409f9c0 2024-01-24T23:23:49.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 1 -- 172.21.15.132:0/300509920 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1889100657 0 0) 0x7fb1a0003aa0 con 0x7fb1a409ff20 2024-01-24T23:23:49.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 1 -- 172.21.15.132:0/300509920 >> v1:172.21.15.115:6792/0 conn(0x7fb1a409f9c0 legacy=0x7fb1a409c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:49.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 1 -- 172.21.15.132:0/300509920 >> v1:172.21.15.115:6791/0 conn(0x7fb1a413b2c0 legacy=0x7fb1a41d36c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:49.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:23:49.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 1 monclient: found mon.g 2024-01-24T23:23:49.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:49.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:49.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 1 -- 172.21.15.132:0/300509920 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a409d560 con 0x7fb1a409ff20 2024-01-24T23:23:49.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 10 monclient: _finish_auth 0 2024-01-24T23:23:49.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:23:49.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:49.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb1ab448640 5 monclient: authenticate success, global_id 5277 2024-01-24T23:23:49.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb1ab448640 10 monclient: _renew_subs 2024-01-24T23:23:49.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb1ab448640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:49.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb1ab448640 1 -- 172.21.15.132:0/300509920 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a4120bb0 con 0x7fb1a409ff20 2024-01-24T23:23:49.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb1ab448640 10 monclient: _renew_subs 2024-01-24T23:23:49.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb1ab448640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:49.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb1ab448640 1 -- 172.21.15.132:0/300509920 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a411ff30 con 0x7fb1a409ff20 2024-01-24T23:23:49.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 1 -- 172.21.15.132:0/300509920 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb1a00047b0 con 0x7fb1a409ff20 2024-01-24T23:23:49.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:49.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 1 -- 172.21.15.132:0/300509920 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb1a0004ce0 con 0x7fb1a409ff20 2024-01-24T23:23:49.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:49.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:49.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 10 monclient: dump: 2024-01-24T23:23:49.490 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:49.490 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:49.490 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:49.490 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:49.490 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:49.490 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:49.490 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:49.490 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:49.491 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:49.491 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:49.491 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:49.491 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:49.491 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:49.491 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:49.491 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:49.491 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:49.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb19b7fe640 4 set_mon_vals no callback set 2024-01-24T23:23:49.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb1ab448640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:49.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 1 -- 172.21.15.132:0/300509920 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fb1a0004e80 con 0x7fb1a409ff20 2024-01-24T23:23:49.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb199ffb640 1 -- 172.21.15.132:0/300509920 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 4195+0+0 (unknown 2295113258 0 0) 0x7fb1a0043a10 con 0x7fb1a409ff20 2024-01-24T23:23:49.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb1ab448640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:49.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb1ab448640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:49.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb1ab448640 1 -- 172.21.15.132:0/300509920 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a409be70 con 0x7fb1a409ff20 2024-01-24T23:23:49.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.471+0000 7fb1a91e4640 10 monclient: get_auth_request con 0x7fb188037000 auth_method 0 2024-01-24T23:23:49.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.475+0000 7fb199ffb640 1 -- 172.21.15.132:0/300509920 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fb1a409be70 con 0x7fb1a409ff20 2024-01-24T23:23:49.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.475+0000 7fb199ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:49.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.475+0000 7fb199ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:49.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.635+0000 7fb1ab448640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}] 2024-01-24T23:23:49.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.635+0000 7fb1ab448640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}] 2024-01-24T23:23:49.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.635+0000 7fb1ab448640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:49.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.635+0000 7fb1ab448640 1 -- 172.21.15.132:0/300509920 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "foo", "type": "root"} v 0) v1 -- 0x7fb1a409be70 con 0x7fb1a409ff20 2024-01-24T23:23:49.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.939+0000 7fb199ffb640 1 -- 172.21.15.132:0/300509920 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}]=0 added bucket foo type root to crush map v62) v1 ==== 138+0+0 (unknown 1792802139 0 0) 0x7fb1a409be70 con 0x7fb1a409ff20 2024-01-24T23:23:49.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.939+0000 7fb199ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}] 2024-01-24T23:23:49.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:49.939+0000 7fb199ffb640 10 monclient: _finish_command 2 = system:0 added bucket foo type root to crush map 2024-01-24T23:23:50.040 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.039+0000 7fb1ab448640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}] 2024-01-24T23:23:50.041 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.039+0000 7fb1ab448640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}] 2024-01-24T23:23:50.041 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.039+0000 7fb1ab448640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:50.041 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.039+0000 7fb1ab448640 1 -- 172.21.15.132:0/300509920 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "foo", "type": "root"} v 0) v1 -- 0x7fb1a409be70 con 0x7fb1a409ff20 2024-01-24T23:23:50.041 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.039+0000 7fb199ffb640 1 -- 172.21.15.132:0/300509920 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}]=0 bucket 'foo' already exists v62) v1 ==== 126+0+0 (unknown 787900033 0 0) 0x7fb1a409be70 con 0x7fb1a409ff20 2024-01-24T23:23:50.041 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.039+0000 7fb199ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}] 2024-01-24T23:23:50.041 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.039+0000 7fb199ffb640 10 monclient: _finish_command 3 = system:0 bucket 'foo' already exists 2024-01-24T23:23:50.041 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'foo' already exists 2024-01-24T23:23:50.043 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.043+0000 7fb1ab448640 1 -- 172.21.15.132:0/300509920 >> v1:172.21.15.132:6812/1835401922 conn(0x7fb188037000 legacy=0x7fb1880394c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:50.043 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.043+0000 7fb1ab448640 10 monclient: shutdown 2024-01-24T23:23:50.044 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.043+0000 7fb1ab448640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:50.044 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.043+0000 7fb1ab448640 1 -- 172.21.15.132:0/300509920 >> v1:172.21.15.132:6790/0 conn(0x7fb1a409ff20 legacy=0x7fb1a409cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:50.044 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.043+0000 7fb1ab448640 1 -- 172.21.15.132:0/300509920 shutdown_connections 2024-01-24T23:23:50.044 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.043+0000 7fb1ab448640 1 -- 172.21.15.132:0/300509920 >> 172.21.15.132:0/300509920 conn(0x7fb1a40a5a10 msgr2=0x7fb1a4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:50.044 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.043+0000 7fb1ab448640 1 -- 172.21.15.132:0/300509920 shutdown_connections 2024-01-24T23:23:50.044 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.043+0000 7fb1ab448640 1 -- 172.21.15.132:0/300509920 wait complete. 2024-01-24T23:23:50.058 INFO:tasks.workunit.client.0.smithi132.stderr:++ ceph osd create 2024-01-24T23:23:50.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 10 monclient: get_monmap_and_config 2024-01-24T23:23:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 10 monclient: build_initial_monmap 2024-01-24T23:23:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 10 monclient: monmap: 2024-01-24T23:23:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:50.165157+0000 2024-01-24T23:23:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:50.165157+0000 2024-01-24T23:23:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:50.165 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 1 Processor -- start 2024-01-24T23:23:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 1 -- start start 2024-01-24T23:23:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 10 monclient: init 2024-01-24T23:23:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 10 monclient: _add_conns ranks=[8,5,3,0,1,2,6,7,4] 2024-01-24T23:23:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 10 monclient(hunting): picked mon.noname-e con 0x7f851409f830 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 10 monclient(hunting): picked mon.noname-h con 0x7f8514137b90 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 10 monclient(hunting): picked mon.noname-g con 0x7f851413b6a0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8514064700 con 0x7f851413b6a0 2024-01-24T23:23:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f851406b3d0 con 0x7f8514137b90 2024-01-24T23:23:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f8513fff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f8514137b90 0x7f8514139f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:44164/0 (socket says 172.21.15.132:44164) 2024-01-24T23:23:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f8513fff640 1 -- 172.21.15.132:0/225012140 learned_addr learned my addr 172.21.15.132:0/225012140 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 1 -- 172.21.15.132:0/225012140 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85140a0320 con 0x7f851409f830 2024-01-24T23:23:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 10 monclient(hunting): authenticate will time out at 1117.490677s 2024-01-24T23:23:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 1 -- 172.21.15.132:0/225012140 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3664285597 0 0) 0x7f8514064700 con 0x7f851413b6a0 2024-01-24T23:23:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 10 monclient(hunting): my global_id is 5286 2024-01-24T23:23:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 1 -- 172.21.15.132:0/225012140 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8504003530 con 0x7f851413b6a0 2024-01-24T23:23:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 1 -- 172.21.15.132:0/225012140 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 567906752 0 0) 0x7f851406b3d0 con 0x7f8514137b90 2024-01-24T23:23:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 10 monclient(hunting): my global_id is 5234 2024-01-24T23:23:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 1 -- 172.21.15.132:0/225012140 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8514064700 con 0x7f8514137b90 2024-01-24T23:23:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 1 -- 172.21.15.132:0/225012140 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3953288520 0 0) 0x7f8514064700 con 0x7f8514137b90 2024-01-24T23:23:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 1 -- 172.21.15.132:0/225012140 >> v1:172.21.15.115:6793/0 conn(0x7f851409f830 legacy=0x7f851409fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 1 -- 172.21.15.132:0/225012140 >> v1:172.21.15.132:6790/0 conn(0x7f851413b6a0 legacy=0x7f851413da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 10 monclient: _finish_hunting 0 2024-01-24T23:23:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 1 monclient: found mon.noname-h 2024-01-24T23:23:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:23:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 1 -- 172.21.15.132:0/225012140 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851406a640 con 0x7f8514137b90 2024-01-24T23:23:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 10 monclient: _finish_auth 0 2024-01-24T23:23:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 10 monclient: _check_auth_tickets 2024-01-24T23:23:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 1 -- 172.21.15.132:0/225012140 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f85080017c0 con 0x7f8514137b90 2024-01-24T23:23:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 5 monclient: authenticate success, global_id 5234 2024-01-24T23:23:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 10 monclient: dump: 2024-01-24T23:23:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:50.175 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:50.175 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:50.175 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:50.175 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:50.175 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:50.175 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:50.176 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 1 -- 172.21.15.132:0/225012140 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8508001a60 con 0x7f8514137b90 2024-01-24T23:23:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 4 set_mon_vals no callback set 2024-01-24T23:23:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 1 -- 172.21.15.132:0/225012140 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8508004bf0 con 0x7f8514137b90 2024-01-24T23:23:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:50.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:50.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f85137fe640 10 monclient: dump: 2024-01-24T23:23:50.177 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:50.177 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:50.177 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:50.177 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:50.177 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:50.177 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:50.178 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:50.178 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:50.178 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:50.178 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:50.178 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:50.179 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:50.179 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:50.179 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:50.179 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:50.179 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 10 monclient: shutdown 2024-01-24T23:23:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 1 -- 172.21.15.132:0/225012140 >> v1:172.21.15.132:6791/0 conn(0x7f8514137b90 legacy=0x7f8514139f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.163+0000 7f851b1d1640 1 -- 172.21.15.132:0/225012140 shutdown_connections 2024-01-24T23:23:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 1 -- 172.21.15.132:0/225012140 >> 172.21.15.132:0/225012140 conn(0x7f85140a66f0 msgr2=0x7f85140a2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 1 -- 172.21.15.132:0/225012140 shutdown_connections 2024-01-24T23:23:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 1 -- 172.21.15.132:0/225012140 wait complete. 2024-01-24T23:23:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 10 monclient: build_initial_monmap 2024-01-24T23:23:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 10 monclient: monmap: 2024-01-24T23:23:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:50.181 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:50.181 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:50.181 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:50.181 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:50.181 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:50.182 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 1 Processor -- start 2024-01-24T23:23:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 1 -- start start 2024-01-24T23:23:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 10 monclient: init 2024-01-24T23:23:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 10 monclient: _add_conns ranks=[4,6,5,0,8,7,3,2,1] 2024-01-24T23:23:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 10 monclient(hunting): picked mon.noname-c con 0x7f851409f830 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 10 monclient(hunting): picked mon.noname-d con 0x7f8514137b90 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 10 monclient(hunting): picked mon.noname-h con 0x7f851413b6a0 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8514138070 con 0x7f851409f830 2024-01-24T23:23:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85141386d0 con 0x7f851413b6a0 2024-01-24T23:23:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f851406b0a0 con 0x7f8514137b90 2024-01-24T23:23:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 10 monclient(hunting): authenticate will time out at 1117.493507s 2024-01-24T23:23:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f8518f6d640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f851409f830 legacy=0x7f851409a9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:23:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851976e640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f851413b6a0 0x7f85141d3aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:44170/0 (socket says 172.21.15.132:44170) 2024-01-24T23:23:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851976e640 1 -- 172.21.15.132:0/19030982 learned_addr learned my addr 172.21.15.132:0/19030982 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 1 -- 172.21.15.132:0/19030982 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 66384405 0 0) 0x7f85141386d0 con 0x7f851413b6a0 2024-01-24T23:23:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 10 monclient(hunting): _negotiate 2024-01-24T23:23:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 10 monclient(hunting): my global_id is 5243 2024-01-24T23:23:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 1 -- 172.21.15.132:0/19030982 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84f8003530 con 0x7f851413b6a0 2024-01-24T23:23:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 1 -- 172.21.15.132:0/19030982 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2174031010 0 0) 0x7f84f8003530 con 0x7f851413b6a0 2024-01-24T23:23:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 1 -- 172.21.15.132:0/19030982 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f85141386d0 con 0x7f851413b6a0 2024-01-24T23:23:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 1 -- 172.21.15.132:0/19030982 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f850c001ce0 con 0x7f851413b6a0 2024-01-24T23:23:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 10 monclient(hunting): dump: 2024-01-24T23:23:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:50.185 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 1 -- 172.21.15.132:0/19030982 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 917384955 0 0) 0x7f851406b0a0 con 0x7f8514137b90 2024-01-24T23:23:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 10 monclient(hunting): _negotiate 2024-01-24T23:23:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 10 monclient(hunting): my global_id is 5343 2024-01-24T23:23:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 1 -- 172.21.15.132:0/19030982 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84f8003530 con 0x7f8514137b90 2024-01-24T23:23:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 1 -- 172.21.15.132:0/19030982 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1407303081 0 0) 0x7f85141386d0 con 0x7f851413b6a0 2024-01-24T23:23:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 1 -- 172.21.15.132:0/19030982 >> v1:172.21.15.115:6792/0 conn(0x7f8514137b90 legacy=0x7f851409b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 1 -- 172.21.15.132:0/19030982 >> v1:172.21.15.115:6791/0 conn(0x7f851409f830 legacy=0x7f851409a9d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 10 monclient: _finish_hunting 0 2024-01-24T23:23:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 1 monclient: found mon.h 2024-01-24T23:23:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 1 -- 172.21.15.132:0/19030982 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851409b7f0 con 0x7f851413b6a0 2024-01-24T23:23:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 10 monclient: _finish_auth 0 2024-01-24T23:23:50.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 10 monclient: _check_auth_tickets 2024-01-24T23:23:50.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:50.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 5 monclient: authenticate success, global_id 5243 2024-01-24T23:23:50.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 10 monclient: _renew_subs 2024-01-24T23:23:50.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:50.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 1 -- 172.21.15.132:0/19030982 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8514138f70 con 0x7f851413b6a0 2024-01-24T23:23:50.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 10 monclient: _renew_subs 2024-01-24T23:23:50.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:50.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 1 -- 172.21.15.132:0/19030982 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8514133d00 con 0x7f851413b6a0 2024-01-24T23:23:50.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 1 -- 172.21.15.132:0/19030982 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f850c002090 con 0x7f851413b6a0 2024-01-24T23:23:50.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:50.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85127fc640 4 set_mon_vals no callback set 2024-01-24T23:23:50.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 1 -- 172.21.15.132:0/19030982 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f850c004010 con 0x7f851413b6a0 2024-01-24T23:23:50.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:50.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:50.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 10 monclient: dump: 2024-01-24T23:23:50.188 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:50.188 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:50.188 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:50.188 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:50.188 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:50.188 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:50.188 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:50.188 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:50.188 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:50.188 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:50.188 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:50.188 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:50.188 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:50.189 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:50.189 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:50.189 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:50.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:50.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:50.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:50.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 1 -- 172.21.15.132:0/19030982 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f850c0041f0 con 0x7f851413b6a0 2024-01-24T23:23:50.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f851b1d1640 1 -- 172.21.15.132:0/19030982 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f4005090 con 0x7f851413b6a0 2024-01-24T23:23:50.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f85117fa640 1 -- 172.21.15.132:0/19030982 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(62..62 src has 1..62) v4 ==== 4222+0+0 (unknown 472786727 0 0) 0x7f850c043c30 con 0x7f851413b6a0 2024-01-24T23:23:50.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.167+0000 7f8518f6d640 10 monclient: get_auth_request con 0x7f84f8036eb0 auth_method 0 2024-01-24T23:23:50.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.175+0000 7f85117fa640 1 -- 172.21.15.132:0/19030982 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f850c014bf0 con 0x7f851413b6a0 2024-01-24T23:23:50.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.175+0000 7f85117fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:50.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.175+0000 7f85117fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:50.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.339+0000 7f851b1d1640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-01-24T23:23:50.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.339+0000 7f851b1d1640 10 monclient: _send_command 2 [{"prefix": "osd create"}] 2024-01-24T23:23:50.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.339+0000 7f851b1d1640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:50.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.339+0000 7f851b1d1640 1 -- 172.21.15.132:0/19030982 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd create"} v 0) v1 -- 0x7f84f40058c0 con 0x7f851413b6a0 2024-01-24T23:23:50.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.963+0000 7f85117fa640 1 -- 172.21.15.132:0/19030982 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd create"}]=0 v63) v1 ==== 58+0+2 (unknown 297710928 0 1113811849) 0x7f850c013290 con 0x7f851413b6a0 2024-01-24T23:23:50.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.963+0000 7f85117fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create"}] 2024-01-24T23:23:50.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:50.963+0000 7f85117fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:51.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:51.071+0000 7f851b1d1640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-01-24T23:23:51.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:51.071+0000 7f851b1d1640 10 monclient: _send_command 3 [{"prefix": "osd create"}] 2024-01-24T23:23:51.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:51.071+0000 7f851b1d1640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:51.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:51.071+0000 7f851b1d1640 1 -- 172.21.15.132:0/19030982 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd create"} v 0) v1 -- 0x7f84f4000bd0 con 0x7f851413b6a0 2024-01-24T23:23:51.963 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:51.963+0000 7f85117fa640 1 -- 172.21.15.132:0/19030982 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd create"}]=0 v64) v1 ==== 58+0+2 (unknown 3957871082 0 1371665406) 0x7f850c014a10 con 0x7f851413b6a0 2024-01-24T23:23:51.964 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:51.963+0000 7f85117fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create"}] 2024-01-24T23:23:51.964 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:51.963+0000 7f85117fa640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:23:51.969 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:51.967+0000 7f851b1d1640 1 -- 172.21.15.132:0/19030982 >> v1:172.21.15.132:6812/1835401922 conn(0x7f84f8036eb0 legacy=0x7f84f8039370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:51.969 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:51.967+0000 7f851b1d1640 10 monclient: shutdown 2024-01-24T23:23:51.969 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:51.967+0000 7f851b1d1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:51.969 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:51.967+0000 7f851b1d1640 1 -- 172.21.15.132:0/19030982 >> v1:172.21.15.132:6791/0 conn(0x7f851413b6a0 legacy=0x7f85141d3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:51.969 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:51.967+0000 7f851b1d1640 1 -- 172.21.15.132:0/19030982 shutdown_connections 2024-01-24T23:23:51.969 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:51.967+0000 7f851b1d1640 1 -- 172.21.15.132:0/19030982 >> 172.21.15.132:0/19030982 conn(0x7f85140a66f0 msgr2=0x7f85140a5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:51.970 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:51.967+0000 7f851b1d1640 1 -- 172.21.15.132:0/19030982 shutdown_connections 2024-01-24T23:23:51.970 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:51.967+0000 7f851b1d1640 1 -- 172.21.15.132:0/19030982 wait complete. 2024-01-24T23:23:51.989 INFO:tasks.workunit.client.0.smithi132.stderr:+ o1=7 2024-01-24T23:23:51.990 INFO:tasks.workunit.client.0.smithi132.stderr:++ ceph osd create 2024-01-24T23:23:52.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.103+0000 7ff412b3c640 10 monclient: get_monmap_and_config 2024-01-24T23:23:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.103+0000 7ff412b3c640 10 monclient: build_initial_monmap 2024-01-24T23:23:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.103+0000 7ff412b3c640 10 monclient: monmap: 2024-01-24T23:23:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:52.107125+0000 2024-01-24T23:23:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:52.107125+0000 2024-01-24T23:23:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:52.106 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:52.107 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:52.107 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:52.107 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:52.107 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:52.107 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:52.107 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:52.107 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:52.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.103+0000 7ff412b3c640 1 Processor -- start 2024-01-24T23:23:52.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.103+0000 7ff412b3c640 1 -- start start 2024-01-24T23:23:52.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.103+0000 7ff412b3c640 10 monclient: init 2024-01-24T23:23:52.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:52.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 10 monclient: _add_conns ranks=[4,6,8,0,3,2,7,1,5] 2024-01-24T23:23:52.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 10 monclient(hunting): picked mon.noname-c con 0x7ff40c135e10 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:52.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 10 monclient(hunting): picked mon.noname-d con 0x7ff40c1361f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:52.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 10 monclient(hunting): picked mon.noname-e con 0x7ff40c1365f0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:52.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff40c064700 con 0x7ff40c135e10 2024-01-24T23:23:52.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff40c09c3e0 con 0x7ff40c1361f0 2024-01-24T23:23:52.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff40c0a4f90 con 0x7ff40c1365f0 2024-01-24T23:23:52.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:52.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 10 monclient(hunting): authenticate will time out at 1119.432554s 2024-01-24T23:23:52.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff4108d8640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7ff40c135e10 legacy=0x7ff40c09bcd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:23:52.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40bfff640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7ff40c1361f0 0x7ff40c13bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:49798/0 (socket says 172.21.15.132:49798) 2024-01-24T23:23:52.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40bfff640 1 -- 172.21.15.132:0/3686277390 learned_addr learned my addr 172.21.15.132:0/3686277390 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:52.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 1 -- 172.21.15.132:0/3686277390 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 520909785 0 0) 0x7ff40c0a4f90 con 0x7ff40c1365f0 2024-01-24T23:23:52.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:52.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:52.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:52.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 10 monclient(hunting): my global_id is 84446 2024-01-24T23:23:52.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 1 -- 172.21.15.132:0/3686277390 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3f4003530 con 0x7ff40c1365f0 2024-01-24T23:23:52.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 1 -- 172.21.15.132:0/3686277390 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3803278535 0 0) 0x7ff40c09c3e0 con 0x7ff40c1361f0 2024-01-24T23:23:52.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:52.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:52.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:52.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 10 monclient(hunting): my global_id is 5352 2024-01-24T23:23:52.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 1 -- 172.21.15.132:0/3686277390 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff40c0a4f90 con 0x7ff40c1361f0 2024-01-24T23:23:52.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 1 -- 172.21.15.132:0/3686277390 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1968267302 0 0) 0x7ff3f4003530 con 0x7ff40c1365f0 2024-01-24T23:23:52.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 1 -- 172.21.15.132:0/3686277390 >> v1:172.21.15.115:6792/0 conn(0x7ff40c1361f0 legacy=0x7ff40c13bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:52.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 1 -- 172.21.15.132:0/3686277390 >> v1:172.21.15.115:6791/0 conn(0x7ff40c135e10 legacy=0x7ff40c09bcd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:52.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:23:52.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 1 monclient: found mon.noname-e 2024-01-24T23:23:52.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:52.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:23:52.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 1 -- 172.21.15.132:0/3686277390 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40c09b010 con 0x7ff40c1365f0 2024-01-24T23:23:52.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 10 monclient: _finish_auth 0 2024-01-24T23:23:52.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:23:52.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:52.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 1 -- 172.21.15.132:0/3686277390 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff4040017c0 con 0x7ff40c1365f0 2024-01-24T23:23:52.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:52.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:23:52.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 10 monclient: dump: 2024-01-24T23:23:52.114 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:52.114 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:52.114 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:52.114 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:52.114 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:52.114 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:52.114 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:52.114 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:52.114 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:52.114 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:52.114 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:52.114 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:52.115 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:52.115 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:52.115 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:52.115 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:52.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 5 monclient: authenticate success, global_id 84446 2024-01-24T23:23:52.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:52.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 1 -- 172.21.15.132:0/3686277390 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff404001aa0 con 0x7ff40c1365f0 2024-01-24T23:23:52.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:52.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 1 -- 172.21.15.132:0/3686277390 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff404003960 con 0x7ff40c1365f0 2024-01-24T23:23:52.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:52.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 4 set_mon_vals no callback set 2024-01-24T23:23:52.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:52.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:52.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff40b7fe640 10 monclient: dump: 2024-01-24T23:23:52.116 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:52.116 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:52.116 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:52.116 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:52.116 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:52.116 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:52.116 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:52.116 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:52.116 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:52.116 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:52.116 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:52.116 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:52.116 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:52.117 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 10 monclient: shutdown 2024-01-24T23:23:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 1 -- 172.21.15.132:0/3686277390 >> v1:172.21.15.115:6793/0 conn(0x7ff40c1365f0 legacy=0x7ff40c13f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 1 -- 172.21.15.132:0/3686277390 shutdown_connections 2024-01-24T23:23:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 1 -- 172.21.15.132:0/3686277390 >> 172.21.15.132:0/3686277390 conn(0x7ff40c0a6790 msgr2=0x7ff40c071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 1 -- 172.21.15.132:0/3686277390 shutdown_connections 2024-01-24T23:23:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 1 -- 172.21.15.132:0/3686277390 wait complete. 2024-01-24T23:23:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 10 monclient: build_initial_monmap 2024-01-24T23:23:52.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 10 monclient: monmap: 2024-01-24T23:23:52.118 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:52.118 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:52.118 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:52.118 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:52.118 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:52.118 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:52.118 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:52.118 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:52.118 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:52.118 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:52.118 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:52.118 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:52.119 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:52.119 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:52.119 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:52.119 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:52.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 1 Processor -- start 2024-01-24T23:23:52.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 1 -- start start 2024-01-24T23:23:52.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.107+0000 7ff412b3c640 10 monclient: init 2024-01-24T23:23:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff412b3c640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff412b3c640 10 monclient: _add_conns ranks=[8,2,5,4,7,6,0,1,3] 2024-01-24T23:23:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff412b3c640 10 monclient(hunting): picked mon.noname-e con 0x7ff40c135e10 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff412b3c640 10 monclient(hunting): picked mon.noname-b con 0x7ff40c1361f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff412b3c640 10 monclient(hunting): picked mon.noname-h con 0x7ff40c1365f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff412b3c640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff40c121f60 con 0x7ff40c1361f0 2024-01-24T23:23:52.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff412b3c640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff40c09c6b0 con 0x7ff40c1365f0 2024-01-24T23:23:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff412b3c640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff40c1c8a60 con 0x7ff40c135e10 2024-01-24T23:23:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff412b3c640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff412b3c640 10 monclient(hunting): authenticate will time out at 1119.436307s 2024-01-24T23:23:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4110d9640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7ff40c1365f0 0x7ff40c1d3b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:44184/0 (socket says 172.21.15.132:44184) 2024-01-24T23:23:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4110d9640 1 -- 172.21.15.132:0/1989372117 learned_addr learned my addr 172.21.15.132:0/1989372117 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 1 -- 172.21.15.132:0/1989372117 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971511495 0 0) 0x7ff40c09c6b0 con 0x7ff40c1365f0 2024-01-24T23:23:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 10 monclient(hunting): _negotiate 2024-01-24T23:23:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 10 monclient(hunting): my global_id is 5252 2024-01-24T23:23:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 1 -- 172.21.15.132:0/1989372117 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3f8003530 con 0x7ff40c1365f0 2024-01-24T23:23:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 1 -- 172.21.15.132:0/1989372117 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2862050724 0 0) 0x7ff40c121f60 con 0x7ff40c1361f0 2024-01-24T23:23:52.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 10 monclient(hunting): _negotiate 2024-01-24T23:23:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 10 monclient(hunting): my global_id is 84539 2024-01-24T23:23:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 1 -- 172.21.15.132:0/1989372117 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff40c09c6b0 con 0x7ff40c1361f0 2024-01-24T23:23:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 1 -- 172.21.15.132:0/1989372117 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3921811971 0 0) 0x7ff3f8003530 con 0x7ff40c1365f0 2024-01-24T23:23:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 1 -- 172.21.15.132:0/1989372117 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff40c121f60 con 0x7ff40c1365f0 2024-01-24T23:23:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 1 -- 172.21.15.132:0/1989372117 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff404001780 con 0x7ff40c1365f0 2024-01-24T23:23:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 10 monclient(hunting): dump: 2024-01-24T23:23:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:52.126 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:52.127 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 1 -- 172.21.15.132:0/1989372117 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1098653127 0 0) 0x7ff40c1c8a60 con 0x7ff40c135e10 2024-01-24T23:23:52.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 10 monclient(hunting): _negotiate 2024-01-24T23:23:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 10 monclient(hunting): my global_id is 84455 2024-01-24T23:23:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 1 -- 172.21.15.132:0/1989372117 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3f8003530 con 0x7ff40c135e10 2024-01-24T23:23:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 1 -- 172.21.15.132:0/1989372117 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3709528305 0 0) 0x7ff40c121f60 con 0x7ff40c1365f0 2024-01-24T23:23:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 1 -- 172.21.15.132:0/1989372117 >> v1:172.21.15.115:6793/0 conn(0x7ff40c135e10 legacy=0x7ff40c134e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 1 -- 172.21.15.132:0/1989372117 >> v1:172.21.15.115:6790/0 conn(0x7ff40c1361f0 legacy=0x7ff40c1d0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 10 monclient: _finish_hunting 0 2024-01-24T23:23:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 1 monclient: found mon.h 2024-01-24T23:23:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 1 -- 172.21.15.132:0/1989372117 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40c1bf420 con 0x7ff40c1365f0 2024-01-24T23:23:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 10 monclient: _finish_auth 0 2024-01-24T23:23:52.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 10 monclient: _check_auth_tickets 2024-01-24T23:23:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff412b3c640 5 monclient: authenticate success, global_id 5252 2024-01-24T23:23:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff412b3c640 10 monclient: _renew_subs 2024-01-24T23:23:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff412b3c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff412b3c640 1 -- 172.21.15.132:0/1989372117 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff40c135550 con 0x7ff40c1365f0 2024-01-24T23:23:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff412b3c640 10 monclient: _renew_subs 2024-01-24T23:23:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff412b3c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff412b3c640 1 -- 172.21.15.132:0/1989372117 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40c121070 con 0x7ff40c1365f0 2024-01-24T23:23:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 1 -- 172.21.15.132:0/1989372117 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff4040011e0 con 0x7ff40c1365f0 2024-01-24T23:23:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff40affd640 4 set_mon_vals no callback set 2024-01-24T23:23:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 1 -- 172.21.15.132:0/1989372117 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff404003ed0 con 0x7ff40c1365f0 2024-01-24T23:23:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:52.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 10 monclient: dump: 2024-01-24T23:23:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:52.130 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:52.131 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff412b3c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 1 -- 172.21.15.132:0/1989372117 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7ff4040040b0 con 0x7ff40c1365f0 2024-01-24T23:23:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff412b3c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff412b3c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff412b3c640 1 -- 172.21.15.132:0/1989372117 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ec005090 con 0x7ff40c1365f0 2024-01-24T23:23:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4097fa640 1 -- 172.21.15.132:0/1989372117 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 4452+0+0 (unknown 4284749152 0 0) 0x7ff404043bd0 con 0x7ff40c1365f0 2024-01-24T23:23:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.111+0000 7ff4108d8640 10 monclient: get_auth_request con 0x7ff3f8037290 auth_method 0 2024-01-24T23:23:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.115+0000 7ff4097fa640 1 -- 172.21.15.132:0/1989372117 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7ff404014c50 con 0x7ff40c1365f0 2024-01-24T23:23:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.115+0000 7ff4097fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:52.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.115+0000 7ff4097fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:52.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.279+0000 7ff412b3c640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-01-24T23:23:52.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.279+0000 7ff412b3c640 10 monclient: _send_command 2 [{"prefix": "osd create"}] 2024-01-24T23:23:52.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.279+0000 7ff412b3c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:52.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.279+0000 7ff412b3c640 1 -- 172.21.15.132:0/1989372117 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd create"} v 0) v1 -- 0x7ff3ec0058c0 con 0x7ff40c1365f0 2024-01-24T23:23:52.982 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.979+0000 7ff4097fa640 1 -- 172.21.15.132:0/1989372117 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd create"}]=0 v65) v1 ==== 58+0+2 (unknown 1587384094 0 3065575683) 0x7ff4040130c0 con 0x7ff40c1365f0 2024-01-24T23:23:52.982 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.979+0000 7ff4097fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create"}] 2024-01-24T23:23:52.982 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:52.979+0000 7ff4097fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:53.088 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:53.087+0000 7ff412b3c640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-01-24T23:23:53.088 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:53.087+0000 7ff412b3c640 10 monclient: _send_command 3 [{"prefix": "osd create"}] 2024-01-24T23:23:53.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:53.087+0000 7ff412b3c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:53.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:53.087+0000 7ff412b3c640 1 -- 172.21.15.132:0/1989372117 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd create"} v 0) v1 -- 0x7ff3ec000bd0 con 0x7ff40c1365f0 2024-01-24T23:23:53.985 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:53.983+0000 7ff4097fa640 1 -- 172.21.15.132:0/1989372117 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd create"}]=0 v66) v1 ==== 58+0+2 (unknown 2230295283 0 2769969524) 0x7ff404014a20 con 0x7ff40c1365f0 2024-01-24T23:23:53.985 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:53.983+0000 7ff4097fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create"}] 2024-01-24T23:23:53.985 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:53.983+0000 7ff4097fa640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:23:53.989 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:53.987+0000 7ff412b3c640 1 -- 172.21.15.132:0/1989372117 >> v1:172.21.15.132:6812/1835401922 conn(0x7ff3f8037290 legacy=0x7ff3f8039750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:53.989 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:53.987+0000 7ff412b3c640 10 monclient: shutdown 2024-01-24T23:23:53.989 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:53.987+0000 7ff412b3c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:53.989 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:53.987+0000 7ff412b3c640 1 -- 172.21.15.132:0/1989372117 >> v1:172.21.15.132:6791/0 conn(0x7ff40c1365f0 legacy=0x7ff40c1d3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:53.989 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:53.987+0000 7ff412b3c640 1 -- 172.21.15.132:0/1989372117 shutdown_connections 2024-01-24T23:23:53.989 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:53.987+0000 7ff412b3c640 1 -- 172.21.15.132:0/1989372117 >> 172.21.15.132:0/1989372117 conn(0x7ff40c0a6790 msgr2=0x7ff40c0a7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:53.989 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:53.987+0000 7ff412b3c640 1 -- 172.21.15.132:0/1989372117 shutdown_connections 2024-01-24T23:23:53.989 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:53.987+0000 7ff412b3c640 1 -- 172.21.15.132:0/1989372117 wait complete. 2024-01-24T23:23:54.009 INFO:tasks.workunit.client.0.smithi132.stderr:+ o2=9 2024-01-24T23:23:54.009 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add 7 1 host=host1 root=foo 2024-01-24T23:23:54.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b9a89640 10 monclient: get_monmap_and_config 2024-01-24T23:23:54.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b9a89640 10 monclient: build_initial_monmap 2024-01-24T23:23:54.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b9a89640 10 monclient: monmap: 2024-01-24T23:23:54.133 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:54.133 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:54.133 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:54.134103+0000 2024-01-24T23:23:54.133 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:54.134103+0000 2024-01-24T23:23:54.133 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:54.133 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:54.133 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:54.133 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:54.133 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:54.133 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:54.133 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:54.134 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:54.134 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:54.134 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:54.134 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:54.134 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:54.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b9a89640 1 Processor -- start 2024-01-24T23:23:54.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b9a89640 1 -- start start 2024-01-24T23:23:54.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b9a89640 10 monclient: init 2024-01-24T23:23:54.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b9a89640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:54.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b9a89640 10 monclient: _add_conns ranks=[3,7,8,6,1,4,0,5,2] 2024-01-24T23:23:54.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b9a89640 10 monclient(hunting): picked mon.noname-g con 0x7f90b4072690 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:54.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b9a89640 10 monclient(hunting): picked mon.noname-i con 0x7f90b409ae90 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:54.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b9a89640 10 monclient(hunting): picked mon.noname-e con 0x7f90b409bdc0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:54.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b9a89640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b4064700 con 0x7f90b4072690 2024-01-24T23:23:54.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b9a89640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b409a790 con 0x7f90b409ae90 2024-01-24T23:23:54.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b9a89640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b40a18f0 con 0x7f90b409bdc0 2024-01-24T23:23:54.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b9a89640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:54.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b9a89640 10 monclient(hunting): authenticate will time out at 1121.458661s 2024-01-24T23:23:54.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90abfff640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f90b409ae90 0x7f90b40a11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:46162/0 (socket says 172.21.15.132:46162) 2024-01-24T23:23:54.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90abfff640 1 -- 172.21.15.132:0/1660308582 learned_addr learned my addr 172.21.15.132:0/1660308582 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:54.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 1 -- 172.21.15.132:0/1660308582 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2741638871 0 0) 0x7f90b409a790 con 0x7f90b409ae90 2024-01-24T23:23:54.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:54.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:54.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:54.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 10 monclient(hunting): my global_id is 5344 2024-01-24T23:23:54.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 1 -- 172.21.15.132:0/1660308582 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90a0003530 con 0x7f90b409ae90 2024-01-24T23:23:54.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 1 -- 172.21.15.132:0/1660308582 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2710593534 0 0) 0x7f90b4064700 con 0x7f90b4072690 2024-01-24T23:23:54.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:54.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:54.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:54.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 10 monclient(hunting): my global_id is 5295 2024-01-24T23:23:54.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 1 -- 172.21.15.132:0/1660308582 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90b409a790 con 0x7f90b4072690 2024-01-24T23:23:54.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 1 -- 172.21.15.132:0/1660308582 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1439714012 0 0) 0x7f90a0003530 con 0x7f90b409ae90 2024-01-24T23:23:54.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 1 -- 172.21.15.132:0/1660308582 >> v1:172.21.15.115:6793/0 conn(0x7f90b409bdc0 legacy=0x7f90b413d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:54.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 1 -- 172.21.15.132:0/1660308582 >> v1:172.21.15.132:6790/0 conn(0x7f90b4072690 legacy=0x7f90b40a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:54.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:54.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 1 monclient: found mon.noname-i 2024-01-24T23:23:54.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:54.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:23:54.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 1 -- 172.21.15.132:0/1660308582 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b40997a0 con 0x7f90b409ae90 2024-01-24T23:23:54.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 10 monclient: _finish_auth 0 2024-01-24T23:23:54.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:54.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:54.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 1 -- 172.21.15.132:0/1660308582 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f909c0017c0 con 0x7f90b409ae90 2024-01-24T23:23:54.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:54.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:23:54.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 10 monclient: dump: 2024-01-24T23:23:54.145 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:54.145 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:54.145 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:54.145 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:54.145 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:54.145 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:54.145 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:54.145 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:54.145 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:54.145 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:54.145 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:54.146 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:54.146 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:54.146 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:54.146 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:54.146 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:54.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b9a89640 5 monclient: authenticate success, global_id 5344 2024-01-24T23:23:54.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b9a89640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:54.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 1 -- 172.21.15.132:0/1660308582 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f909c0019f0 con 0x7f90b409ae90 2024-01-24T23:23:54.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:54.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 1 -- 172.21.15.132:0/1660308582 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f909c004bf0 con 0x7f90b409ae90 2024-01-24T23:23:54.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b9a89640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:54.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b9a89640 4 set_mon_vals no callback set 2024-01-24T23:23:54.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:54.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:54.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.131+0000 7f90b2ffd640 10 monclient: dump: 2024-01-24T23:23:54.147 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:54.147 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:54.147 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:54.147 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:54.147 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:54.147 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:54.147 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:54.148 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:54.148 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:54.148 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:54.148 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:54.148 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:54.148 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:54.148 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:54.148 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:54.148 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:54.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b9a89640 10 monclient: shutdown 2024-01-24T23:23:54.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b9a89640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:54.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b9a89640 1 -- 172.21.15.132:0/1660308582 >> v1:172.21.15.132:6792/0 conn(0x7f90b409ae90 legacy=0x7f90b40a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:54.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b9a89640 1 -- 172.21.15.132:0/1660308582 shutdown_connections 2024-01-24T23:23:54.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b9a89640 1 -- 172.21.15.132:0/1660308582 >> 172.21.15.132:0/1660308582 conn(0x7f90b40a1bc0 msgr2=0x7f90b40a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:54.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b9a89640 1 -- 172.21.15.132:0/1660308582 shutdown_connections 2024-01-24T23:23:54.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b9a89640 1 -- 172.21.15.132:0/1660308582 wait complete. 2024-01-24T23:23:54.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b9a89640 10 monclient: build_initial_monmap 2024-01-24T23:23:54.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b9a89640 10 monclient: monmap: 2024-01-24T23:23:54.149 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:54.149 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:54.149 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:54.149 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:54.149 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:54.149 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:54.149 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:54.149 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:54.149 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:54.149 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:54.149 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:54.150 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:54.150 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:54.150 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:54.150 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:54.150 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:54.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b9a89640 1 Processor -- start 2024-01-24T23:23:54.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b9a89640 1 -- start start 2024-01-24T23:23:54.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b9a89640 10 monclient: init 2024-01-24T23:23:54.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b9a89640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:54.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b9a89640 10 monclient: _add_conns ranks=[6,2,8,7,5,3,0,1,4] 2024-01-24T23:23:54.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b9a89640 10 monclient(hunting): picked mon.noname-d con 0x7f90b4072690 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:54.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b9a89640 10 monclient(hunting): picked mon.noname-b con 0x7f90b409ae90 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:54.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b9a89640 10 monclient(hunting): picked mon.noname-e con 0x7f90b409bdc0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:54.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b9a89640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b40994e0 con 0x7f90b409ae90 2024-01-24T23:23:54.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b9a89640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b406ec30 con 0x7f90b4072690 2024-01-24T23:23:54.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b9a89640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b41ca360 con 0x7f90b409bdc0 2024-01-24T23:23:54.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b9a89640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:54.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b9a89640 10 monclient(hunting): authenticate will time out at 1121.462204s 2024-01-24T23:23:54.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b37fe640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f90b4072690 0x7f90b41c9540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:55194/0 (socket says 172.21.15.132:55194) 2024-01-24T23:23:54.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b37fe640 1 -- 172.21.15.132:0/1671757575 learned_addr learned my addr 172.21.15.132:0/1671757575 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:54.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3219430222 0 0) 0x7f90b40997a0 con 0x7f90b409ae90 2024-01-24T23:23:54.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:54.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:54.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:54.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 10 monclient(hunting): my global_id is 84548 2024-01-24T23:23:54.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9094003530 con 0x7f90b409ae90 2024-01-24T23:23:54.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 682010382 0 0) 0x7f90b41ca360 con 0x7f90b409bdc0 2024-01-24T23:23:54.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:54.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:54.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:54.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 10 monclient(hunting): my global_id is 84473 2024-01-24T23:23:54.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90b40997a0 con 0x7f90b409bdc0 2024-01-24T23:23:54.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3980740806 0 0) 0x7f90b406ec30 con 0x7f90b4072690 2024-01-24T23:23:54.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:23:54.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:54.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:54.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 10 monclient(hunting): my global_id is 5361 2024-01-24T23:23:54.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90b41ca360 con 0x7f90b4072690 2024-01-24T23:23:54.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1305273607 0 0) 0x7f9094003530 con 0x7f90b409ae90 2024-01-24T23:23:54.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f90b406ec30 con 0x7f90b409ae90 2024-01-24T23:23:54.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f909c001780 con 0x7f90b409ae90 2024-01-24T23:23:54.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:54.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:23:54.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 10 monclient(hunting): dump: 2024-01-24T23:23:54.153 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:54.153 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:54.153 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:54.153 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:54.153 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:54.153 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:54.153 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:54.153 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:54.153 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:54.153 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:54.153 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:54.153 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:54.153 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:54.153 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:54.153 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:54.154 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:54.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:54.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 742048823 0 0) 0x7f90b40997a0 con 0x7f90b409bdc0 2024-01-24T23:23:54.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9094003530 con 0x7f90b409bdc0 2024-01-24T23:23:54.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f90ac001c80 con 0x7f90b409bdc0 2024-01-24T23:23:54.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:54.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:54.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 10 monclient(hunting): dump: 2024-01-24T23:23:54.154 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:54.154 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:54.154 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:54.154 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:54.154 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:54.154 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:54.154 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:54.155 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:54.155 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:54.155 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:54.155 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:54.155 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:54.155 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:54.155 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:54.155 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:54.155 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:54.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3826844768 0 0) 0x7f90b41ca360 con 0x7f90b4072690 2024-01-24T23:23:54.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f90b40997a0 con 0x7f90b4072690 2024-01-24T23:23:54.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f90a4001580 con 0x7f90b4072690 2024-01-24T23:23:54.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:54.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:23:54.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.135+0000 7f90b0ff9640 10 monclient(hunting): dump: 2024-01-24T23:23:54.156 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:54.156 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:54.156 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:54.156 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:54.156 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:54.156 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:54.156 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:54.156 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:54.156 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:54.156 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:54.156 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:54.157 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:54.157 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:54.157 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:54.157 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:54.157 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:54.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 819115080 0 0) 0x7f9094003530 con 0x7f90b409bdc0 2024-01-24T23:23:54.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 >> v1:172.21.15.115:6792/0 conn(0x7f90b4072690 legacy=0x7f90b41c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:54.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 >> v1:172.21.15.115:6790/0 conn(0x7f90b409ae90 legacy=0x7f90b41c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:54.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b0ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:23:54.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b0ff9640 1 monclient: found mon.e 2024-01-24T23:23:54.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b0ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:54.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b0ff9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:23:54.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b409b930 con 0x7f90b409bdc0 2024-01-24T23:23:54.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b0ff9640 10 monclient: _finish_auth 0 2024-01-24T23:23:54.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b0ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:23:54.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:54.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b9a89640 5 monclient: authenticate success, global_id 84473 2024-01-24T23:23:54.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b9a89640 10 monclient: _renew_subs 2024-01-24T23:23:54.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b9a89640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:23:54.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b9a89640 1 -- 172.21.15.132:0/1671757575 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b4070520 con 0x7f90b409bdc0 2024-01-24T23:23:54.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b9a89640 10 monclient: _renew_subs 2024-01-24T23:23:54.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b9a89640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:23:54.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b9a89640 1 -- 172.21.15.132:0/1671757575 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b406f750 con 0x7f90b409bdc0 2024-01-24T23:23:54.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f90ac003980 con 0x7f90b409bdc0 2024-01-24T23:23:54.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b0ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:54.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f90ac003eb0 con 0x7f90b409bdc0 2024-01-24T23:23:54.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:54.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b0ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:54.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b27fc640 4 set_mon_vals no callback set 2024-01-24T23:23:54.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b0ff9640 10 monclient: dump: 2024-01-24T23:23:54.159 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:54.159 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:54.159 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:54.159 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:54.159 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:54.159 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:54.159 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:54.159 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:54.159 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:54.159 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:54.159 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:54.160 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:54.160 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:54.160 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:54.160 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:54.160 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:54.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b9a89640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:54.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b9a89640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:54.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b9a89640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:23:54.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b9a89640 1 -- 172.21.15.132:0/1671757575 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9098005090 con 0x7f90b409bdc0 2024-01-24T23:23:54.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f90ac004010 con 0x7f90b409bdc0 2024-01-24T23:23:54.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(66..66 src has 1..66) v4 ==== 4682+0+0 (unknown 2553635302 0 0) 0x7f90ac042a80 con 0x7f90b409bdc0 2024-01-24T23:23:54.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.139+0000 7f90b37fe640 10 monclient: get_auth_request con 0x7f9094037bb0 auth_method 0 2024-01-24T23:23:54.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.151+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f90ac013960 con 0x7f90b409bdc0 2024-01-24T23:23:54.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.151+0000 7f90b0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:54.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.151+0000 7f90b0ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:54.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.311+0000 7f90b9a89640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-01-24T23:23:54.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.311+0000 7f90b9a89640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-01-24T23:23:54.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.311+0000 7f90b9a89640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:23:54.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.311+0000 7f90b9a89640 1 -- 172.21.15.132:0/1671757575 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]} v 0) v1 -- 0x7f9098005340 con 0x7f90b409bdc0 2024-01-24T23:23:55.000 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.999+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}]=0 add item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo} to crush map v67) v1 ==== 203+0+0 (unknown 3154998832 0 0) 0x7f90ac013550 con 0x7f90b409bdc0 2024-01-24T23:23:55.000 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.999+0000 7f90b0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-01-24T23:23:55.000 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:54.999+0000 7f90b0ff9640 10 monclient: _finish_command 2 = system:0 add item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo} to crush map 2024-01-24T23:23:55.100 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.099+0000 7f90b9a89640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-01-24T23:23:55.100 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.099+0000 7f90b9a89640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-01-24T23:23:55.100 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.099+0000 7f90b9a89640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:23:55.100 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.099+0000 7f90b9a89640 1 -- 172.21.15.132:0/1671757575 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]} v 0) v1 -- 0x7f9098005a80 con 0x7f90b409bdc0 2024-01-24T23:23:55.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.099+0000 7f90b0ff9640 1 -- 172.21.15.132:0/1671757575 <== mon.8 v1:172.21.15.115:6793/0 11 ==== mon_command_ack([{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}]=0 set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change v67) v1 ==== 201+0+0 (unknown 1053878289 0 0) 0x7f90ac013730 con 0x7f90b409bdc0 2024-01-24T23:23:55.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.099+0000 7f90b0ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-01-24T23:23:55.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.099+0000 7f90b0ff9640 10 monclient: _finish_command 3 = system:0 set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change 2024-01-24T23:23:55.102 INFO:tasks.workunit.client.0.smithi132.stderr:set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change 2024-01-24T23:23:55.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.103+0000 7f90b9a89640 1 -- 172.21.15.132:0/1671757575 >> v1:172.21.15.132:6812/1835401922 conn(0x7f9094037bb0 legacy=0x7f909403a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:55.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.103+0000 7f90b9a89640 10 monclient: shutdown 2024-01-24T23:23:55.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.103+0000 7f90b9a89640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:55.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.103+0000 7f90b9a89640 1 -- 172.21.15.132:0/1671757575 >> v1:172.21.15.115:6793/0 conn(0x7f90b409bdc0 legacy=0x7f90b41d1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:55.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.103+0000 7f90b9a89640 1 -- 172.21.15.132:0/1671757575 shutdown_connections 2024-01-24T23:23:55.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.103+0000 7f90b9a89640 1 -- 172.21.15.132:0/1671757575 >> 172.21.15.132:0/1671757575 conn(0x7f90b40a1bc0 msgr2=0x7f90b40a3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:55.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.103+0000 7f90b9a89640 1 -- 172.21.15.132:0/1671757575 shutdown_connections 2024-01-24T23:23:55.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.103+0000 7f90b9a89640 1 -- 172.21.15.132:0/1671757575 wait complete. 2024-01-24T23:23:55.120 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add 7 1 host=host1 root=foo 2024-01-24T23:23:55.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.239+0000 7fb6e174f640 10 monclient: get_monmap_and_config 2024-01-24T23:23:55.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.239+0000 7fb6e174f640 10 monclient: build_initial_monmap 2024-01-24T23:23:55.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.239+0000 7fb6e174f640 10 monclient: monmap: 2024-01-24T23:23:55.242 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:55.242 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:55.242 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:55.243554+0000 2024-01-24T23:23:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:55.243554+0000 2024-01-24T23:23:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:55.243 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:55.243 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:55.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.239+0000 7fb6e174f640 1 Processor -- start 2024-01-24T23:23:55.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.239+0000 7fb6e174f640 1 -- start start 2024-01-24T23:23:55.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 10 monclient: init 2024-01-24T23:23:55.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:55.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 10 monclient: _add_conns ranks=[1,0,7,4,3,2,8,6,5] 2024-01-24T23:23:55.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 10 monclient(hunting): picked mon.noname-f con 0x7fb6dc09b900 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:55.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 10 monclient(hunting): picked mon.noname-a con 0x7fb6dc09bce0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:55.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 10 monclient(hunting): picked mon.noname-i con 0x7fb6dc09c140 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:55.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6dc064700 con 0x7fb6dc09bce0 2024-01-24T23:23:55.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6dc0993b0 con 0x7fb6dc09b900 2024-01-24T23:23:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6dc099e10 con 0x7fb6dc09c140 2024-01-24T23:23:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 10 monclient(hunting): authenticate will time out at 1122.568683s 2024-01-24T23:23:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6daffd640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fb6dc09b900 legacy=0x7fb6dc1370c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6db7fe640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fb6dc09c140 0x7fb6dc13f630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:46166/0 (socket says 172.21.15.132:46166) 2024-01-24T23:23:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6db7fe640 1 -- 172.21.15.132:0/2869480062 learned_addr learned my addr 172.21.15.132:0/2869480062 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6da7fc640 1 -- 172.21.15.132:0/2869480062 >> v1:172.21.15.115:6789/0 conn(0x7fb6dc09bce0 legacy=0x7fb6dc13bf00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 1 -- 172.21.15.132:0/2869480062 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 280727866 0 0) 0x7fb6dc099e10 con 0x7fb6dc09c140 2024-01-24T23:23:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 10 monclient(hunting): my global_id is 5353 2024-01-24T23:23:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 1 -- 172.21.15.132:0/2869480062 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6c4003530 con 0x7fb6dc09c140 2024-01-24T23:23:55.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 1 -- 172.21.15.132:0/2869480062 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1431081810 0 0) 0x7fb6c4003530 con 0x7fb6dc09c140 2024-01-24T23:23:55.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 1 -- 172.21.15.132:0/2869480062 >> v1:172.21.15.132:6789/0 conn(0x7fb6dc09b900 legacy=0x7fb6dc1370c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:55.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 1 -- 172.21.15.132:0/2869480062 >> v1:172.21.15.115:6789/0 conn(0x7fb6dc09bce0 legacy=0x7fb6dc13bf00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:55.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:23:55.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 1 monclient: found mon.noname-i 2024-01-24T23:23:55.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:55.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:23:55.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 1 -- 172.21.15.132:0/2869480062 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6dc09a0f0 con 0x7fb6dc09c140 2024-01-24T23:23:55.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 10 monclient: _finish_auth 0 2024-01-24T23:23:55.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:23:55.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:55.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 1 -- 172.21.15.132:0/2869480062 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb6d4001b40 con 0x7fb6dc09c140 2024-01-24T23:23:55.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 5 monclient: authenticate success, global_id 5353 2024-01-24T23:23:55.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:55.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:55.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:23:55.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 10 monclient: dump: 2024-01-24T23:23:55.254 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:55.254 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:55.254 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:55.254 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:55.254 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:55.254 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:55.254 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:55.254 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:55.254 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:55.254 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:55.254 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:55.255 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:55.255 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:55.255 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:55.255 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:55.255 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:55.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:55.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 1 -- 172.21.15.132:0/2869480062 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb6d4001e60 con 0x7fb6dc09c140 2024-01-24T23:23:55.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:55.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:55.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 4 set_mon_vals no callback set 2024-01-24T23:23:55.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 1 -- 172.21.15.132:0/2869480062 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb6d4004f70 con 0x7fb6dc09c140 2024-01-24T23:23:55.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:55.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:55.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6d9ffb640 10 monclient: dump: 2024-01-24T23:23:55.256 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:55.256 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:55.256 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:55.256 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:55.257 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:55.257 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:55.257 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:55.257 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:55.257 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:55.257 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:55.257 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:55.257 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:55.257 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:55.257 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:55.257 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:55.257 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:55.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 10 monclient: shutdown 2024-01-24T23:23:55.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:55.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 1 -- 172.21.15.132:0/2869480062 >> v1:172.21.15.132:6792/0 conn(0x7fb6dc09c140 legacy=0x7fb6dc13f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:55.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 1 -- 172.21.15.132:0/2869480062 shutdown_connections 2024-01-24T23:23:55.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 1 -- 172.21.15.132:0/2869480062 >> 172.21.15.132:0/2869480062 conn(0x7fb6dc0a1eb0 msgr2=0x7fb6dc0a2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:55.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 1 -- 172.21.15.132:0/2869480062 shutdown_connections 2024-01-24T23:23:55.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 1 -- 172.21.15.132:0/2869480062 wait complete. 2024-01-24T23:23:55.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 10 monclient: build_initial_monmap 2024-01-24T23:23:55.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 10 monclient: monmap: 2024-01-24T23:23:55.258 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:55.258 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:55.258 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:55.258 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:55.258 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:55.258 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:55.258 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:55.258 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:55.258 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:55.259 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:55.259 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:55.259 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:55.259 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:55.259 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:55.259 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:55.259 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:55.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 1 Processor -- start 2024-01-24T23:23:55.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 1 -- start start 2024-01-24T23:23:55.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 10 monclient: init 2024-01-24T23:23:55.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:55.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 10 monclient: _add_conns ranks=[3,2,7,5,8,1,0,6,4] 2024-01-24T23:23:55.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 10 monclient(hunting): picked mon.noname-g con 0x7fb6dc09b900 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:55.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 10 monclient(hunting): picked mon.noname-b con 0x7fb6dc09bce0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:55.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 10 monclient(hunting): picked mon.noname-i con 0x7fb6dc09c140 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:55.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6dc133420 con 0x7fb6dc09bce0 2024-01-24T23:23:55.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6dc0725b0 con 0x7fb6dc09b900 2024-01-24T23:23:55.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6dc12abb0 con 0x7fb6dc09c140 2024-01-24T23:23:55.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:55.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6e174f640 10 monclient(hunting): authenticate will time out at 1122.571778s 2024-01-24T23:23:55.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6daffd640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fb6dc09b900 0x7fb6dc1309a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:56956/0 (socket says 172.21.15.132:56956) 2024-01-24T23:23:55.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.243+0000 7fb6daffd640 1 -- 172.21.15.132:0/965361537 learned_addr learned my addr 172.21.15.132:0/965361537 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:55.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 1 -- 172.21.15.132:0/965361537 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362482029 0 0) 0x7fb6dc12abb0 con 0x7fb6dc09c140 2024-01-24T23:23:55.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:55.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:55.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:55.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient(hunting): my global_id is 5362 2024-01-24T23:23:55.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 1 -- 172.21.15.132:0/965361537 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6c8003530 con 0x7fb6dc09c140 2024-01-24T23:23:55.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 1 -- 172.21.15.132:0/965361537 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2561102240 0 0) 0x7fb6dc0725b0 con 0x7fb6dc09b900 2024-01-24T23:23:55.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:55.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:55.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:55.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient(hunting): my global_id is 5304 2024-01-24T23:23:55.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 1 -- 172.21.15.132:0/965361537 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6dc12abb0 con 0x7fb6dc09b900 2024-01-24T23:23:55.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 1 -- 172.21.15.132:0/965361537 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3376073388 0 0) 0x7fb6dc12abb0 con 0x7fb6dc09b900 2024-01-24T23:23:55.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 1 -- 172.21.15.132:0/965361537 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb6dc0725b0 con 0x7fb6dc09b900 2024-01-24T23:23:55.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 1 -- 172.21.15.132:0/965361537 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3955181139 0 0) 0x7fb6c8003530 con 0x7fb6dc09c140 2024-01-24T23:23:55.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 1 -- 172.21.15.132:0/965361537 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb6dc12abb0 con 0x7fb6dc09c140 2024-01-24T23:23:55.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 1 -- 172.21.15.132:0/965361537 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb6cc004300 con 0x7fb6dc09b900 2024-01-24T23:23:55.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:55.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:55.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient(hunting): dump: 2024-01-24T23:23:55.261 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:55.261 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:55.261 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:55.262 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:55.262 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:55.262 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:55.262 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:55.262 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:55.262 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:55.262 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:55.262 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:55.262 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:55.262 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:55.262 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:55.262 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:55.262 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:55.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:55.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 1 -- 172.21.15.132:0/965361537 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb6d4001b00 con 0x7fb6dc09c140 2024-01-24T23:23:55.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:55.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:55.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient(hunting): dump: 2024-01-24T23:23:55.263 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:55.263 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:55.263 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:55.263 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:55.263 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:55.263 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:55.263 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:55.263 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:55.263 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:55.263 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:55.263 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:55.264 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:55.264 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:55.264 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:55.264 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:55.264 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:55.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 1 -- 172.21.15.132:0/965361537 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2668048496 0 0) 0x7fb6dc133420 con 0x7fb6dc09bce0 2024-01-24T23:23:55.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:55.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:55.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:55.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient(hunting): my global_id is 84557 2024-01-24T23:23:55.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 1 -- 172.21.15.132:0/965361537 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6c8003530 con 0x7fb6dc09bce0 2024-01-24T23:23:55.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 1 -- 172.21.15.132:0/965361537 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 287567755 0 0) 0x7fb6dc0725b0 con 0x7fb6dc09b900 2024-01-24T23:23:55.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 1 -- 172.21.15.132:0/965361537 >> v1:172.21.15.132:6792/0 conn(0x7fb6dc09c140 legacy=0x7fb6dc1d4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:55.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 1 -- 172.21.15.132:0/965361537 >> v1:172.21.15.115:6790/0 conn(0x7fb6dc09bce0 legacy=0x7fb6dc1d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:55.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient: _finish_hunting 0 2024-01-24T23:23:55.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 1 monclient: found mon.g 2024-01-24T23:23:55.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:55.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:55.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 1 -- 172.21.15.132:0/965361537 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6dc12ae90 con 0x7fb6dc09b900 2024-01-24T23:23:55.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient: _finish_auth 0 2024-01-24T23:23:55.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient: _check_auth_tickets 2024-01-24T23:23:55.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:55.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6e174f640 5 monclient: authenticate success, global_id 5304 2024-01-24T23:23:55.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6e174f640 10 monclient: _renew_subs 2024-01-24T23:23:55.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6e174f640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:55.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6e174f640 1 -- 172.21.15.132:0/965361537 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6dc134180 con 0x7fb6dc09b900 2024-01-24T23:23:55.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6e174f640 10 monclient: _renew_subs 2024-01-24T23:23:55.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6e174f640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:55.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6e174f640 1 -- 172.21.15.132:0/965361537 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6dc131280 con 0x7fb6dc09b900 2024-01-24T23:23:55.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 1 -- 172.21.15.132:0/965361537 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb6cc004520 con 0x7fb6dc09b900 2024-01-24T23:23:55.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:55.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 1 -- 172.21.15.132:0/965361537 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb6cc004a50 con 0x7fb6dc09b900 2024-01-24T23:23:55.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:55.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:55.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6d97fa640 4 set_mon_vals no callback set 2024-01-24T23:23:55.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 10 monclient: dump: 2024-01-24T23:23:55.266 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:55.266 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:55.266 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:55.266 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:55.266 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:55.266 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:55.266 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:55.267 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:55.267 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:55.267 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:55.267 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:55.267 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:55.267 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:55.267 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:55.267 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:55.267 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:55.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6e174f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:55.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6e174f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:55.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6e174f640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:55.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6e174f640 1 -- 172.21.15.132:0/965361537 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6bc005090 con 0x7fb6dc09b900 2024-01-24T23:23:55.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 1 -- 172.21.15.132:0/965361537 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fb6cc004bb0 con 0x7fb6dc09b900 2024-01-24T23:23:55.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6c37fe640 1 -- 172.21.15.132:0/965361537 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(67..67 src has 1..67) v4 ==== 4964+0+0 (unknown 1464440128 0 0) 0x7fb6cc043bb0 con 0x7fb6dc09b900 2024-01-24T23:23:55.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.247+0000 7fb6da7fc640 10 monclient: get_auth_request con 0x7fb6c80378b0 auth_method 0 2024-01-24T23:23:55.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.251+0000 7fb6c37fe640 1 -- 172.21.15.132:0/965361537 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fb6cc014a40 con 0x7fb6dc09b900 2024-01-24T23:23:55.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.251+0000 7fb6c37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:55.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.251+0000 7fb6c37fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:55.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.411+0000 7fb6e174f640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-01-24T23:23:55.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.411+0000 7fb6e174f640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-01-24T23:23:55.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.411+0000 7fb6e174f640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:55.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.411+0000 7fb6e174f640 1 -- 172.21.15.132:0/965361537 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]} v 0) v1 -- 0x7fb6bc0058c0 con 0x7fb6dc09b900 2024-01-24T23:23:55.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.415+0000 7fb6c37fe640 1 -- 172.21.15.132:0/965361537 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}]=0 set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change v67) v1 ==== 201+0+0 (unknown 1053878289 0 0) 0x7fb6cc01cf40 con 0x7fb6dc09b900 2024-01-24T23:23:55.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.415+0000 7fb6c37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-01-24T23:23:55.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.415+0000 7fb6c37fe640 10 monclient: _finish_command 2 = system:0 set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change 2024-01-24T23:23:55.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.511+0000 7fb6e174f640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-01-24T23:23:55.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.511+0000 7fb6e174f640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-01-24T23:23:55.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.511+0000 7fb6e174f640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:55.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.511+0000 7fb6e174f640 1 -- 172.21.15.132:0/965361537 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]} v 0) v1 -- 0x7fb6bc002220 con 0x7fb6dc09b900 2024-01-24T23:23:55.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.511+0000 7fb6c37fe640 1 -- 172.21.15.132:0/965361537 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}]=0 set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change v67) v1 ==== 201+0+0 (unknown 1053878289 0 0) 0x7fb6cc012eb0 con 0x7fb6dc09b900 2024-01-24T23:23:55.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.511+0000 7fb6c37fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-01-24T23:23:55.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.511+0000 7fb6c37fe640 10 monclient: _finish_command 3 = system:0 set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change 2024-01-24T23:23:55.514 INFO:tasks.workunit.client.0.smithi132.stderr:set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change 2024-01-24T23:23:55.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.515+0000 7fb6e174f640 1 -- 172.21.15.132:0/965361537 >> v1:172.21.15.132:6812/1835401922 conn(0x7fb6c80378b0 legacy=0x7fb6c8039d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:55.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.515+0000 7fb6e174f640 10 monclient: shutdown 2024-01-24T23:23:55.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.515+0000 7fb6e174f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:55.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.515+0000 7fb6e174f640 1 -- 172.21.15.132:0/965361537 >> v1:172.21.15.132:6790/0 conn(0x7fb6dc09b900 legacy=0x7fb6dc1309a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:55.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.515+0000 7fb6e174f640 1 -- 172.21.15.132:0/965361537 shutdown_connections 2024-01-24T23:23:55.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.515+0000 7fb6e174f640 1 -- 172.21.15.132:0/965361537 >> 172.21.15.132:0/965361537 conn(0x7fb6dc0a1eb0 msgr2=0x7fb6dc0a3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:55.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.515+0000 7fb6e174f640 1 -- 172.21.15.132:0/965361537 shutdown_connections 2024-01-24T23:23:55.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.515+0000 7fb6e174f640 1 -- 172.21.15.132:0/965361537 wait complete. 2024-01-24T23:23:55.531 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add 9 1 host=host2 root=foo 2024-01-24T23:23:55.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.643+0000 7fb487ab7640 10 monclient: get_monmap_and_config 2024-01-24T23:23:55.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.643+0000 7fb487ab7640 10 monclient: build_initial_monmap 2024-01-24T23:23:55.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.643+0000 7fb487ab7640 10 monclient: monmap: 2024-01-24T23:23:55.646 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:55.646 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:55.646 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:55.647327+0000 2024-01-24T23:23:55.646 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:55.647327+0000 2024-01-24T23:23:55.646 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:55.646 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:55.646 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:55.646 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:55.647 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:55.647 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:55.647 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:55.647 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:55.647 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:55.647 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:55.647 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:55.647 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:55.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.643+0000 7fb487ab7640 1 Processor -- start 2024-01-24T23:23:55.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.643+0000 7fb487ab7640 1 -- start start 2024-01-24T23:23:55.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.643+0000 7fb487ab7640 10 monclient: init 2024-01-24T23:23:55.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.643+0000 7fb487ab7640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:55.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.643+0000 7fb487ab7640 10 monclient: _add_conns ranks=[8,5,0,2,6,7,1,4,3] 2024-01-24T23:23:55.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.643+0000 7fb487ab7640 10 monclient(hunting): picked mon.noname-e con 0x7fb4800a0ad0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:55.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 10 monclient(hunting): picked mon.noname-h con 0x7fb48009ee80 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:55.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 10 monclient(hunting): picked mon.noname-a con 0x7fb48009b060 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:55.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb480064700 con 0x7fb48009b060 2024-01-24T23:23:55.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb48009cb80 con 0x7fb48009ee80 2024-01-24T23:23:55.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4800a0eb0 con 0x7fb4800a0ad0 2024-01-24T23:23:55.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:55.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 10 monclient(hunting): authenticate will time out at 1122.972149s 2024-01-24T23:23:55.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb485052640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fb48009ee80 0x7fb48009a950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:45832/0 (socket says 172.21.15.132:45832) 2024-01-24T23:23:55.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb485052640 1 -- 172.21.15.132:0/1545026497 learned_addr learned my addr 172.21.15.132:0/1545026497 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:55.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb486054640 1 -- 172.21.15.132:0/1545026497 >> v1:172.21.15.115:6789/0 conn(0x7fb48009b060 legacy=0x7fb48013c270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:55.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 1 -- 172.21.15.132:0/1545026497 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2425361518 0 0) 0x7fb48009cb80 con 0x7fb48009ee80 2024-01-24T23:23:55.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 10 monclient(hunting): _negotiate 2024-01-24T23:23:55.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:55.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:55.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 10 monclient(hunting): my global_id is 5261 2024-01-24T23:23:55.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 1 -- 172.21.15.132:0/1545026497 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb46c005740 con 0x7fb48009ee80 2024-01-24T23:23:55.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 1 -- 172.21.15.132:0/1545026497 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 386145171 0 0) 0x7fb46c005740 con 0x7fb48009ee80 2024-01-24T23:23:55.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 1 -- 172.21.15.132:0/1545026497 >> v1:172.21.15.115:6793/0 conn(0x7fb4800a0ad0 legacy=0x7fb48009a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:55.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 1 -- 172.21.15.132:0/1545026497 >> v1:172.21.15.115:6789/0 conn(0x7fb48009b060 legacy=0x7fb48013c270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:55.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 10 monclient: _finish_hunting 0 2024-01-24T23:23:55.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 1 monclient: found mon.noname-h 2024-01-24T23:23:55.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:55.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:23:55.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 1 -- 172.21.15.132:0/1545026497 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48009f280 con 0x7fb48009ee80 2024-01-24T23:23:55.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 10 monclient: _finish_auth 0 2024-01-24T23:23:55.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 10 monclient: _check_auth_tickets 2024-01-24T23:23:55.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:55.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 5 monclient: authenticate success, global_id 5261 2024-01-24T23:23:55.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 1 -- 172.21.15.132:0/1545026497 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb4700017c0 con 0x7fb48009ee80 2024-01-24T23:23:55.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:55.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:55.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:55.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 10 monclient: dump: 2024-01-24T23:23:55.657 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:55.657 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:55.657 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:55.657 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:55.657 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:55.657 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:55.657 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:55.657 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:55.657 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:55.657 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:55.658 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:55.658 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:55.658 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:55.658 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:55.658 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:55.658 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:55.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:55.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 1 -- 172.21.15.132:0/1545026497 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb470001a70 con 0x7fb48009ee80 2024-01-24T23:23:55.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:55.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:55.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 1 -- 172.21.15.132:0/1545026497 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb470004bf0 con 0x7fb48009ee80 2024-01-24T23:23:55.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 4 set_mon_vals no callback set 2024-01-24T23:23:55.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:55.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:55.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb484851640 10 monclient: dump: 2024-01-24T23:23:55.659 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:55.659 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:55.659 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:55.659 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:55.659 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:55.659 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:55.659 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:55.659 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:55.659 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:55.659 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:55.659 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:55.659 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:55.659 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:55.659 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:55.660 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:55.660 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:55.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 10 monclient: shutdown 2024-01-24T23:23:55.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:55.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 1 -- 172.21.15.132:0/1545026497 >> v1:172.21.15.132:6791/0 conn(0x7fb48009ee80 legacy=0x7fb48009a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:55.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 1 -- 172.21.15.132:0/1545026497 shutdown_connections 2024-01-24T23:23:55.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 1 -- 172.21.15.132:0/1545026497 >> 172.21.15.132:0/1545026497 conn(0x7fb4800a1bc0 msgr2=0x7fb4800a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:55.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 1 -- 172.21.15.132:0/1545026497 shutdown_connections 2024-01-24T23:23:55.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 1 -- 172.21.15.132:0/1545026497 wait complete. 2024-01-24T23:23:55.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 10 monclient: build_initial_monmap 2024-01-24T23:23:55.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 10 monclient: monmap: 2024-01-24T23:23:55.660 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:55.660 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:55.660 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:55.661 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:55.661 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:55.661 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:55.661 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:55.661 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:55.661 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:55.661 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:55.661 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:55.661 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:55.661 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:55.661 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:55.662 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:55.662 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:55.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 1 Processor -- start 2024-01-24T23:23:55.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 1 -- start start 2024-01-24T23:23:55.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 10 monclient: init 2024-01-24T23:23:55.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:55.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 10 monclient: _add_conns ranks=[8,5,7,3,6,0,2,4,1] 2024-01-24T23:23:55.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 10 monclient(hunting): picked mon.noname-e con 0x7fb48009b060 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:55.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 10 monclient(hunting): picked mon.noname-h con 0x7fb48009ee80 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:55.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 10 monclient(hunting): picked mon.noname-i con 0x7fb4800a0ad0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:55.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb480091dc0 con 0x7fb48009ee80 2024-01-24T23:23:55.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb48009b4c0 con 0x7fb4800a0ad0 2024-01-24T23:23:55.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb48009e640 con 0x7fb48009b060 2024-01-24T23:23:55.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:55.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb487ab7640 10 monclient(hunting): authenticate will time out at 1122.975144s 2024-01-24T23:23:55.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb485052640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fb48009ee80 0x7fb4801d06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:45844/0 (socket says 172.21.15.132:45844) 2024-01-24T23:23:55.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb485052640 1 -- 172.21.15.132:0/3193467158 learned_addr learned my addr 172.21.15.132:0/3193467158 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:55.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 1 -- 172.21.15.132:0/3193467158 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3117606691 0 0) 0x7fb480091dc0 con 0x7fb48009ee80 2024-01-24T23:23:55.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:55.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:55.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:55.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 10 monclient(hunting): my global_id is 5270 2024-01-24T23:23:55.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 1 -- 172.21.15.132:0/3193467158 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb464003530 con 0x7fb48009ee80 2024-01-24T23:23:55.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 1 -- 172.21.15.132:0/3193467158 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 6545527 0 0) 0x7fb48009b4c0 con 0x7fb4800a0ad0 2024-01-24T23:23:55.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:55.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:55.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:55.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 10 monclient(hunting): my global_id is 5371 2024-01-24T23:23:55.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 1 -- 172.21.15.132:0/3193467158 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb480091dc0 con 0x7fb4800a0ad0 2024-01-24T23:23:55.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 1 -- 172.21.15.132:0/3193467158 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 706662879 0 0) 0x7fb464003530 con 0x7fb48009ee80 2024-01-24T23:23:55.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 1 -- 172.21.15.132:0/3193467158 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb48009b4c0 con 0x7fb48009ee80 2024-01-24T23:23:55.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 1 -- 172.21.15.132:0/3193467158 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb470001780 con 0x7fb48009ee80 2024-01-24T23:23:55.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:55.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:55.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 10 monclient(hunting): dump: 2024-01-24T23:23:55.664 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:55.664 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:55.664 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:55.664 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:55.665 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:55.665 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:55.665 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:55.665 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:55.665 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:55.665 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:55.665 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:55.665 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:55.665 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:55.665 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:55.665 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:55.665 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:55.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:55.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 1 -- 172.21.15.132:0/3193467158 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2342305939 0 0) 0x7fb48009e640 con 0x7fb48009b060 2024-01-24T23:23:55.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:55.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:55.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:55.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 10 monclient(hunting): my global_id is 84491 2024-01-24T23:23:55.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 1 -- 172.21.15.132:0/3193467158 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb464003530 con 0x7fb48009b060 2024-01-24T23:23:55.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 1 -- 172.21.15.132:0/3193467158 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 513345637 0 0) 0x7fb480091dc0 con 0x7fb4800a0ad0 2024-01-24T23:23:55.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 1 -- 172.21.15.132:0/3193467158 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb48009e640 con 0x7fb4800a0ad0 2024-01-24T23:23:55.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 1 -- 172.21.15.132:0/3193467158 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb478004560 con 0x7fb4800a0ad0 2024-01-24T23:23:55.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:55.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:55.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 10 monclient(hunting): dump: 2024-01-24T23:23:55.666 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:55.666 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:55.666 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:55.666 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:55.666 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:55.667 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:55.667 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:55.667 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:55.667 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:55.667 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:55.667 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:55.667 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:55.667 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:55.667 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:55.667 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:55.667 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:55.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.647+0000 7fb4767fc640 1 -- 172.21.15.132:0/3193467158 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1701308471 0 0) 0x7fb48009b4c0 con 0x7fb48009ee80 2024-01-24T23:23:55.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb4767fc640 1 -- 172.21.15.132:0/3193467158 >> v1:172.21.15.115:6793/0 conn(0x7fb48009b060 legacy=0x7fb4800953e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:55.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb4767fc640 1 -- 172.21.15.132:0/3193467158 >> v1:172.21.15.132:6792/0 conn(0x7fb4800a0ad0 legacy=0x7fb4801d3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:55.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb4767fc640 10 monclient: _finish_hunting 0 2024-01-24T23:23:55.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb4767fc640 1 monclient: found mon.h 2024-01-24T23:23:55.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb4767fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:55.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb4767fc640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:55.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb4767fc640 1 -- 172.21.15.132:0/3193467158 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4800936c0 con 0x7fb48009ee80 2024-01-24T23:23:55.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb4767fc640 10 monclient: _finish_auth 0 2024-01-24T23:23:55.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb4767fc640 10 monclient: _check_auth_tickets 2024-01-24T23:23:55.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb4767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:55.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb487ab7640 5 monclient: authenticate success, global_id 5270 2024-01-24T23:23:55.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb487ab7640 10 monclient: _renew_subs 2024-01-24T23:23:55.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb487ab7640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:55.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb487ab7640 1 -- 172.21.15.132:0/3193467158 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4800939a0 con 0x7fb48009ee80 2024-01-24T23:23:55.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb487ab7640 10 monclient: _renew_subs 2024-01-24T23:23:55.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb487ab7640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:55.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb487ab7640 1 -- 172.21.15.132:0/3193467158 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb480095af0 con 0x7fb48009ee80 2024-01-24T23:23:55.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb4767fc640 1 -- 172.21.15.132:0/3193467158 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb470004ec0 con 0x7fb48009ee80 2024-01-24T23:23:55.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb4767fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:55.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb4767fc640 1 -- 172.21.15.132:0/3193467158 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb470005370 con 0x7fb48009ee80 2024-01-24T23:23:55.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb4767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:55.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb4767fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:55.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb4767fc640 10 monclient: dump: 2024-01-24T23:23:55.669 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:55.669 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:55.669 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:55.669 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:55.669 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:55.669 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:55.669 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:55.669 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:55.669 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:55.670 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:55.670 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:55.670 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:55.670 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:55.670 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:55.670 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:55.670 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:55.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb477fff640 4 set_mon_vals no callback set 2024-01-24T23:23:55.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb4767fc640 1 -- 172.21.15.132:0/3193467158 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fb47000fdf0 con 0x7fb48009ee80 2024-01-24T23:23:55.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb487ab7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:55.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb487ab7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:55.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb487ab7640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:55.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb487ab7640 1 -- 172.21.15.132:0/3193467158 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4800934c0 con 0x7fb48009ee80 2024-01-24T23:23:55.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb4767fc640 1 -- 172.21.15.132:0/3193467158 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(67..67 src has 1..67) v4 ==== 4964+0+0 (unknown 1464440128 0 0) 0x7fb470043dd0 con 0x7fb48009ee80 2024-01-24T23:23:55.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.651+0000 7fb485853640 10 monclient: get_auth_request con 0x7fb4640378d0 auth_method 0 2024-01-24T23:23:55.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.655+0000 7fb4767fc640 1 -- 172.21.15.132:0/3193467158 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fb470014c50 con 0x7fb48009ee80 2024-01-24T23:23:55.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.655+0000 7fb4767fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:55.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.655+0000 7fb4767fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:55.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.811+0000 7fb487ab7640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-01-24T23:23:55.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.811+0000 7fb487ab7640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-01-24T23:23:55.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.811+0000 7fb487ab7640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:55.813 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:55.811+0000 7fb487ab7640 1 -- 172.21.15.132:0/3193467158 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]} v 0) v1 -- 0x7fb480091920 con 0x7fb48009ee80 2024-01-24T23:23:56.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.011+0000 7fb4767fc640 1 -- 172.21.15.132:0/3193467158 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}]=0 add item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo} to crush map v68) v1 ==== 203+0+0 (unknown 1221103361 0 0) 0x7fb4700130c0 con 0x7fb48009ee80 2024-01-24T23:23:56.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.011+0000 7fb4767fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-01-24T23:23:56.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.011+0000 7fb4767fc640 10 monclient: _finish_command 2 = system:0 add item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo} to crush map 2024-01-24T23:23:56.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.111+0000 7fb487ab7640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-01-24T23:23:56.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.111+0000 7fb487ab7640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-01-24T23:23:56.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.111+0000 7fb487ab7640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:56.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.111+0000 7fb487ab7640 1 -- 172.21.15.132:0/3193467158 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]} v 0) v1 -- 0x7fb4800750f0 con 0x7fb48009ee80 2024-01-24T23:23:56.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.111+0000 7fb4767fc640 1 -- 172.21.15.132:0/3193467158 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}]=0 set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change v68) v1 ==== 201+0+0 (unknown 1694027685 0 0) 0x7fb470014840 con 0x7fb48009ee80 2024-01-24T23:23:56.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.111+0000 7fb4767fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-01-24T23:23:56.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.111+0000 7fb4767fc640 10 monclient: _finish_command 3 = system:0 set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change 2024-01-24T23:23:56.114 INFO:tasks.workunit.client.0.smithi132.stderr:set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change 2024-01-24T23:23:56.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.115+0000 7fb487ab7640 1 -- 172.21.15.132:0/3193467158 >> v1:172.21.15.132:6812/1835401922 conn(0x7fb4640378d0 legacy=0x7fb464039d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:56.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.115+0000 7fb487ab7640 10 monclient: shutdown 2024-01-24T23:23:56.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.115+0000 7fb487ab7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:56.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.115+0000 7fb487ab7640 1 -- 172.21.15.132:0/3193467158 >> v1:172.21.15.132:6791/0 conn(0x7fb48009ee80 legacy=0x7fb4801d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:56.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.115+0000 7fb487ab7640 1 -- 172.21.15.132:0/3193467158 shutdown_connections 2024-01-24T23:23:56.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.115+0000 7fb487ab7640 1 -- 172.21.15.132:0/3193467158 >> 172.21.15.132:0/3193467158 conn(0x7fb4800a1bc0 msgr2=0x7fb4800a3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:56.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.115+0000 7fb487ab7640 1 -- 172.21.15.132:0/3193467158 shutdown_connections 2024-01-24T23:23:56.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.115+0000 7fb487ab7640 1 -- 172.21.15.132:0/3193467158 wait complete. 2024-01-24T23:23:56.130 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add 9 1 host=host2 root=foo 2024-01-24T23:23:56.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33f34f640 10 monclient: get_monmap_and_config 2024-01-24T23:23:56.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33f34f640 10 monclient: build_initial_monmap 2024-01-24T23:23:56.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33f34f640 10 monclient: monmap: 2024-01-24T23:23:56.252 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:56.253 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:56.253 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:56.253751+0000 2024-01-24T23:23:56.253 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:56.253751+0000 2024-01-24T23:23:56.253 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:56.253 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:56.253 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:56.253 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:56.253 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:56.253 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:56.253 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:56.253 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:56.253 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:56.253 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:56.253 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:56.254 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:56.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33f34f640 1 Processor -- start 2024-01-24T23:23:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33f34f640 1 -- start start 2024-01-24T23:23:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33f34f640 10 monclient: init 2024-01-24T23:23:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33f34f640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33f34f640 10 monclient: _add_conns ranks=[2,7,5,6,3,4,1,8,0] 2024-01-24T23:23:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33f34f640 10 monclient(hunting): picked mon.noname-b con 0x7fa338099130 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33f34f640 10 monclient(hunting): picked mon.noname-i con 0x7fa33809af90 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33f34f640 10 monclient(hunting): picked mon.noname-h con 0x7fa338099c20 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33f34f640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa338064700 con 0x7fa338099130 2024-01-24T23:23:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33f34f640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa33809a510 con 0x7fa338099c20 2024-01-24T23:23:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33f34f640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa33809ba30 con 0x7fa33809af90 2024-01-24T23:23:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33f34f640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33f34f640 10 monclient(hunting): authenticate will time out at 1123.578440s 2024-01-24T23:23:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33d8ec640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fa338099c20 0x7fa33813bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:45860/0 (socket says 172.21.15.132:45860) 2024-01-24T23:23:56.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33d8ec640 1 -- 172.21.15.132:0/649344237 learned_addr learned my addr 172.21.15.132:0/649344237 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:56.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 1 -- 172.21.15.132:0/649344237 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 285540615 0 0) 0x7fa33809a510 con 0x7fa338099c20 2024-01-24T23:23:56.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:56.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:56.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:56.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 10 monclient(hunting): my global_id is 5279 2024-01-24T23:23:56.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 1 -- 172.21.15.132:0/649344237 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa320003530 con 0x7fa338099c20 2024-01-24T23:23:56.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 1 -- 172.21.15.132:0/649344237 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1412578880 0 0) 0x7fa33809ba30 con 0x7fa33809af90 2024-01-24T23:23:56.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:56.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:56.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:56.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 10 monclient(hunting): my global_id is 5380 2024-01-24T23:23:56.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 1 -- 172.21.15.132:0/649344237 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa33809a510 con 0x7fa33809af90 2024-01-24T23:23:56.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 1 -- 172.21.15.132:0/649344237 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3069497932 0 0) 0x7fa33809a510 con 0x7fa33809af90 2024-01-24T23:23:56.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 1 -- 172.21.15.132:0/649344237 >> v1:172.21.15.132:6791/0 conn(0x7fa338099c20 legacy=0x7fa33813bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:56.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 1 -- 172.21.15.132:0/649344237 >> v1:172.21.15.115:6790/0 conn(0x7fa338099130 legacy=0x7fa338099510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:56.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 10 monclient: _finish_hunting 0 2024-01-24T23:23:56.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 1 monclient: found mon.noname-i 2024-01-24T23:23:56.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:56.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:23:56.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 1 -- 172.21.15.132:0/649344237 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33809bd10 con 0x7fa33809af90 2024-01-24T23:23:56.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 10 monclient: _finish_auth 0 2024-01-24T23:23:56.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 10 monclient: _check_auth_tickets 2024-01-24T23:23:56.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:56.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 1 -- 172.21.15.132:0/649344237 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa32c0016f0 con 0x7fa33809af90 2024-01-24T23:23:56.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33f34f640 5 monclient: authenticate success, global_id 5380 2024-01-24T23:23:56.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:56.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:23:56.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 10 monclient: dump: 2024-01-24T23:23:56.263 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:56.263 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:56.263 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:56.264 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:56.264 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:56.264 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:56.264 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:56.264 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:56.264 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:56.264 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:56.264 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:56.264 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:56.264 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:56.264 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:56.264 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:56.264 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:56.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33f34f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:56.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 1 -- 172.21.15.132:0/649344237 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa32c0018a0 con 0x7fa33809af90 2024-01-24T23:23:56.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:56.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 1 -- 172.21.15.132:0/649344237 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa32c003620 con 0x7fa33809af90 2024-01-24T23:23:56.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33f34f640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:56.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33f34f640 4 set_mon_vals no callback set 2024-01-24T23:23:56.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:56.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:56.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa337fff640 10 monclient: dump: 2024-01-24T23:23:56.265 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:56.265 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:56.265 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:56.265 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:56.265 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:56.265 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:56.265 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:56.265 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:56.265 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:56.265 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:56.265 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:56.266 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:56.266 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:56.266 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:56.266 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:56.266 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:56.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33f34f640 10 monclient: shutdown 2024-01-24T23:23:56.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33f34f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:56.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.251+0000 7fa33f34f640 1 -- 172.21.15.132:0/649344237 >> v1:172.21.15.132:6792/0 conn(0x7fa33809af90 legacy=0x7fa338138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:56.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 1 -- 172.21.15.132:0/649344237 shutdown_connections 2024-01-24T23:23:56.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 1 -- 172.21.15.132:0/649344237 >> 172.21.15.132:0/649344237 conn(0x7fa3380a1bc0 msgr2=0x7fa3380a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:56.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 1 -- 172.21.15.132:0/649344237 shutdown_connections 2024-01-24T23:23:56.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 1 -- 172.21.15.132:0/649344237 wait complete. 2024-01-24T23:23:56.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 10 monclient: build_initial_monmap 2024-01-24T23:23:56.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 10 monclient: monmap: 2024-01-24T23:23:56.266 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:56.267 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 1 Processor -- start 2024-01-24T23:23:56.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 1 -- start start 2024-01-24T23:23:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 10 monclient: init 2024-01-24T23:23:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 10 monclient: _add_conns ranks=[0,1,2,8,7,5,3,6,4] 2024-01-24T23:23:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 10 monclient(hunting): picked mon.noname-a con 0x7fa338099130 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 10 monclient(hunting): picked mon.noname-f con 0x7fa338099c20 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 10 monclient(hunting): picked mon.noname-b con 0x7fa33809af90 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3381311e0 con 0x7fa338099130 2024-01-24T23:23:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa338134430 con 0x7fa338099c20 2024-01-24T23:23:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3381cc290 con 0x7fa33809af90 2024-01-24T23:23:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 10 monclient(hunting): authenticate will time out at 1123.581550s 2024-01-24T23:23:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33c8ea640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fa338099c20 legacy=0x7fa3381cbb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33d0eb640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fa338099130 legacy=0x7fa3381cb470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33d8ec640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fa33809af90 0x7fa3381d3950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:42202/0 (socket says 172.21.15.132:42202) 2024-01-24T23:23:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33d8ec640 1 -- 172.21.15.132:0/3561102766 learned_addr learned my addr 172.21.15.132:0/3561102766 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:56.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 1 -- 172.21.15.132:0/3561102766 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1160188727 0 0) 0x7fa3381cc290 con 0x7fa33809af90 2024-01-24T23:23:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 10 monclient(hunting): my global_id is 84575 2024-01-24T23:23:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 1 -- 172.21.15.132:0/3561102766 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa324003530 con 0x7fa33809af90 2024-01-24T23:23:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 1 -- 172.21.15.132:0/3561102766 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3566877700 0 0) 0x7fa324003530 con 0x7fa33809af90 2024-01-24T23:23:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 1 -- 172.21.15.132:0/3561102766 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa3381cc290 con 0x7fa33809af90 2024-01-24T23:23:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 1 -- 172.21.15.132:0/3561102766 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa3280018c0 con 0x7fa33809af90 2024-01-24T23:23:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:23:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 10 monclient(hunting): dump: 2024-01-24T23:23:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:56.269 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:56.270 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:56.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 1 -- 172.21.15.132:0/3561102766 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 165139182 0 0) 0x7fa3381cc290 con 0x7fa33809af90 2024-01-24T23:23:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 1 -- 172.21.15.132:0/3561102766 >> v1:172.21.15.132:6789/0 conn(0x7fa338099c20 legacy=0x7fa3381cbb80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 1 -- 172.21.15.132:0/3561102766 >> v1:172.21.15.115:6789/0 conn(0x7fa338099130 legacy=0x7fa3381cb470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:23:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 1 monclient: found mon.b 2024-01-24T23:23:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 1 -- 172.21.15.132:0/3561102766 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33809b3e0 con 0x7fa33809af90 2024-01-24T23:23:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 10 monclient: _finish_auth 0 2024-01-24T23:23:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:23:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 5 monclient: authenticate success, global_id 84575 2024-01-24T23:23:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 10 monclient: _renew_subs 2024-01-24T23:23:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 1 -- 172.21.15.132:0/3561102766 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa338132860 con 0x7fa33809af90 2024-01-24T23:23:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 10 monclient: _renew_subs 2024-01-24T23:23:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:56.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa33f34f640 1 -- 172.21.15.132:0/3561102766 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3381332c0 con 0x7fa33809af90 2024-01-24T23:23:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 1 -- 172.21.15.132:0/3561102766 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa328004e20 con 0x7fa33809af90 2024-01-24T23:23:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 1 -- 172.21.15.132:0/3561102766 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa3280052d0 con 0x7fa33809af90 2024-01-24T23:23:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa3377fe640 4 set_mon_vals no callback set 2024-01-24T23:23:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.255+0000 7fa335ffb640 10 monclient: dump: 2024-01-24T23:23:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:56.272 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:56.273 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.259+0000 7fa33f34f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.259+0000 7fa33f34f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.259+0000 7fa33f34f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.259+0000 7fa33f34f640 1 -- 172.21.15.132:0/3561102766 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa318005090 con 0x7fa33809af90 2024-01-24T23:23:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.259+0000 7fa335ffb640 1 -- 172.21.15.132:0/3561102766 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fa32800fd50 con 0x7fa33809af90 2024-01-24T23:23:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.259+0000 7fa335ffb640 1 -- 172.21.15.132:0/3561102766 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(68..68 src has 1..68) v4 ==== 5208+0+0 (unknown 1190817742 0 0) 0x7fa328044120 con 0x7fa33809af90 2024-01-24T23:23:56.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.259+0000 7fa33d0eb640 10 monclient: get_auth_request con 0x7fa324036fc0 auth_method 0 2024-01-24T23:23:56.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.271+0000 7fa335ffb640 1 -- 172.21.15.132:0/3561102766 <== mon.2 v1:172.21.15.115:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fa328014bc0 con 0x7fa33809af90 2024-01-24T23:23:56.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.271+0000 7fa335ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:56.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.271+0000 7fa335ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:56.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.423+0000 7fa33f34f640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-01-24T23:23:56.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.423+0000 7fa33f34f640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-01-24T23:23:56.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.423+0000 7fa33f34f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:56.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.423+0000 7fa33f34f640 1 -- 172.21.15.132:0/3561102766 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]} v 0) v1 -- 0x7fa318005340 con 0x7fa33809af90 2024-01-24T23:23:56.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.427+0000 7fa335ffb640 1 -- 172.21.15.132:0/3561102766 <== mon.2 v1:172.21.15.115:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}]=0 set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change v68) v1 ==== 201+0+0 (unknown 1694027685 0 0) 0x7fa3280149e0 con 0x7fa33809af90 2024-01-24T23:23:56.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.427+0000 7fa335ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-01-24T23:23:56.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.427+0000 7fa335ffb640 10 monclient: _finish_command 2 = system:0 set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change 2024-01-24T23:23:56.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.515+0000 7fa33f34f640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-01-24T23:23:56.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.515+0000 7fa33f34f640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-01-24T23:23:56.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.515+0000 7fa33f34f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:56.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.515+0000 7fa33f34f640 1 -- 172.21.15.132:0/3561102766 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]} v 0) v1 -- 0x7fa318005a80 con 0x7fa33809af90 2024-01-24T23:23:56.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.519+0000 7fa335ffb640 1 -- 172.21.15.132:0/3561102766 <== mon.2 v1:172.21.15.115:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}]=0 set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change v68) v1 ==== 201+0+0 (unknown 1694027685 0 0) 0x7fa328015020 con 0x7fa33809af90 2024-01-24T23:23:56.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.519+0000 7fa335ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-01-24T23:23:56.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.519+0000 7fa335ffb640 10 monclient: _finish_command 3 = system:0 set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change 2024-01-24T23:23:56.521 INFO:tasks.workunit.client.0.smithi132.stderr:set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change 2024-01-24T23:23:56.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.519+0000 7fa33f34f640 1 -- 172.21.15.132:0/3561102766 >> v1:172.21.15.132:6812/1835401922 conn(0x7fa324036fc0 legacy=0x7fa324039480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:56.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.519+0000 7fa33f34f640 10 monclient: shutdown 2024-01-24T23:23:56.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.519+0000 7fa33f34f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:56.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.519+0000 7fa33f34f640 1 -- 172.21.15.132:0/3561102766 >> v1:172.21.15.115:6790/0 conn(0x7fa33809af90 legacy=0x7fa3381d3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:56.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.519+0000 7fa33f34f640 1 -- 172.21.15.132:0/3561102766 shutdown_connections 2024-01-24T23:23:56.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.519+0000 7fa33f34f640 1 -- 172.21.15.132:0/3561102766 >> 172.21.15.132:0/3561102766 conn(0x7fa3380a1bc0 msgr2=0x7fa338097110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:56.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.519+0000 7fa33f34f640 1 -- 172.21.15.132:0/3561102766 shutdown_connections 2024-01-24T23:23:56.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.519+0000 7fa33f34f640 1 -- 172.21.15.132:0/3561102766 wait complete. 2024-01-24T23:23:56.536 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket bar root 2024-01-24T23:23:56.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 10 monclient: get_monmap_and_config 2024-01-24T23:23:56.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 10 monclient: build_initial_monmap 2024-01-24T23:23:56.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 10 monclient: monmap: 2024-01-24T23:23:56.652 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:56.652 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:56.652 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:56.653025+0000 2024-01-24T23:23:56.652 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:56.653025+0000 2024-01-24T23:23:56.652 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:56.652 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:56.652 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:56.652 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:56.652 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:56.652 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:56.652 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:56.653 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:56.653 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:56.653 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:56.653 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:56.653 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:56.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 1 Processor -- start 2024-01-24T23:23:56.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 1 -- start start 2024-01-24T23:23:56.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 10 monclient: init 2024-01-24T23:23:56.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:56.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 10 monclient: _add_conns ranks=[0,3,7,6,1,4,8,2,5] 2024-01-24T23:23:56.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 10 monclient(hunting): picked mon.noname-a con 0x7f64ec0a0e20 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:56.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 10 monclient(hunting): picked mon.noname-g con 0x7f64ec09f5e0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:56.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 10 monclient(hunting): picked mon.noname-i con 0x7f64ec13d560 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:56.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ec064700 con 0x7f64ec0a0e20 2024-01-24T23:23:56.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ec06d3f0 con 0x7f64ec09f5e0 2024-01-24T23:23:56.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ec09fa00 con 0x7f64ec13d560 2024-01-24T23:23:56.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:56.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 10 monclient(hunting): authenticate will time out at 1123.977692s 2024-01-24T23:23:56.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f16d1640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f64ec13d560 0x7f64ec13f950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:46202/0 (socket says 172.21.15.132:46202) 2024-01-24T23:23:56.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f16d1640 1 -- 172.21.15.132:0/3716336981 learned_addr learned my addr 172.21.15.132:0/3716336981 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:56.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64ebfff640 1 --1- 172.21.15.132:0/3716336981 >> v1:172.21.15.132:6790/0 conn(0x7f64ec09f5e0 0x7f64ec13be40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:56964/0 (socket says 172.21.15.132:56964) 2024-01-24T23:23:56.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f0ed0640 1 -- 172.21.15.132:0/3716336981 >> v1:172.21.15.115:6789/0 conn(0x7f64ec0a0e20 legacy=0x7f64ec0a1200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:56.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 1 -- 172.21.15.132:0/3716336981 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 329383869 0 0) 0x7f64ec09fa00 con 0x7f64ec13d560 2024-01-24T23:23:56.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:56.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:56.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:56.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 10 monclient(hunting): my global_id is 5389 2024-01-24T23:23:56.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 1 -- 172.21.15.132:0/3716336981 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64dc003530 con 0x7f64ec13d560 2024-01-24T23:23:56.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 1 -- 172.21.15.132:0/3716336981 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1594286128 0 0) 0x7f64ec06d3f0 con 0x7f64ec09f5e0 2024-01-24T23:23:56.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:56.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:56.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:56.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 10 monclient(hunting): my global_id is 5313 2024-01-24T23:23:56.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 1 -- 172.21.15.132:0/3716336981 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64ec09fa00 con 0x7f64ec09f5e0 2024-01-24T23:23:56.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 1 -- 172.21.15.132:0/3716336981 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2686410439 0 0) 0x7f64dc003530 con 0x7f64ec13d560 2024-01-24T23:23:56.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 1 -- 172.21.15.132:0/3716336981 >> v1:172.21.15.132:6790/0 conn(0x7f64ec09f5e0 legacy=0x7f64ec13be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:56.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 1 -- 172.21.15.132:0/3716336981 >> v1:172.21.15.115:6789/0 conn(0x7f64ec0a0e20 legacy=0x7f64ec0a1200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:56.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:23:56.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 1 monclient: found mon.noname-i 2024-01-24T23:23:56.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:56.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:23:56.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 1 -- 172.21.15.132:0/3716336981 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ec06bc90 con 0x7f64ec13d560 2024-01-24T23:23:56.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 10 monclient: _finish_auth 0 2024-01-24T23:23:56.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:23:56.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:56.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 1 -- 172.21.15.132:0/3716336981 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f64e4001b40 con 0x7f64ec13d560 2024-01-24T23:23:56.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 5 monclient: authenticate success, global_id 5389 2024-01-24T23:23:56.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:56.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:23:56.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 10 monclient: dump: 2024-01-24T23:23:56.663 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:56.663 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:56.663 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:56.663 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:56.663 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:56.663 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:56.663 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:56.663 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:56.663 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:56.663 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:56.663 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:56.663 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:56.663 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:56.663 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:56.663 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:56.664 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:56.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:56.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 1 -- 172.21.15.132:0/3716336981 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f64e4001d30 con 0x7f64ec13d560 2024-01-24T23:23:56.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:56.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:56.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 4 set_mon_vals no callback set 2024-01-24T23:23:56.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 1 -- 172.21.15.132:0/3716336981 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f64e4004f70 con 0x7f64ec13d560 2024-01-24T23:23:56.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:56.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:56.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64eb7fe640 10 monclient: dump: 2024-01-24T23:23:56.664 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:56.664 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:56.664 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:56.664 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:56.665 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:56.665 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:56.665 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:56.665 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:56.665 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:56.665 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:56.665 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:56.665 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:56.665 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:56.665 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:56.665 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:56.665 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:56.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 10 monclient: shutdown 2024-01-24T23:23:56.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:56.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 1 -- 172.21.15.132:0/3716336981 >> v1:172.21.15.132:6792/0 conn(0x7f64ec13d560 legacy=0x7f64ec13f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:56.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 1 -- 172.21.15.132:0/3716336981 shutdown_connections 2024-01-24T23:23:56.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 1 -- 172.21.15.132:0/3716336981 >> 172.21.15.132:0/3716336981 conn(0x7f64ec0a7a30 msgr2=0x7f64ec0a3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:56.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 1 -- 172.21.15.132:0/3716336981 shutdown_connections 2024-01-24T23:23:56.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 1 -- 172.21.15.132:0/3716336981 wait complete. 2024-01-24T23:23:56.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 10 monclient: build_initial_monmap 2024-01-24T23:23:56.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.651+0000 7f64f3134640 10 monclient: monmap: 2024-01-24T23:23:56.666 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:56.666 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:56.666 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:56.666 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:56.666 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:56.666 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:56.666 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:56.666 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:56.666 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:56.667 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:56.667 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:56.667 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:56.667 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:56.667 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:56.667 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:56.667 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:56.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f3134640 1 Processor -- start 2024-01-24T23:23:56.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f3134640 1 -- start start 2024-01-24T23:23:56.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f3134640 10 monclient: init 2024-01-24T23:23:56.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f3134640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:56.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f3134640 10 monclient: _add_conns ranks=[3,0,2,7,5,1,4,6,8] 2024-01-24T23:23:56.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f3134640 10 monclient(hunting): picked mon.noname-g con 0x7f64ec09f5e0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:56.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f3134640 10 monclient(hunting): picked mon.noname-a con 0x7f64ec0a0e20 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:56.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f3134640 10 monclient(hunting): picked mon.noname-b con 0x7f64ec13d560 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:56.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f3134640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ec072780 con 0x7f64ec0a0e20 2024-01-24T23:23:56.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f3134640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ec09c320 con 0x7f64ec13d560 2024-01-24T23:23:56.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f3134640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ec09dc20 con 0x7f64ec09f5e0 2024-01-24T23:23:56.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f3134640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:56.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f3134640 10 monclient(hunting): authenticate will time out at 1123.980736s 2024-01-24T23:23:56.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f0ed0640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f64ec09f5e0 0x7f64ec071920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:56966/0 (socket says 172.21.15.132:56966) 2024-01-24T23:23:56.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f0ed0640 1 -- 172.21.15.132:0/1293136346 learned_addr learned my addr 172.21.15.132:0/1293136346 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:56.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64ebfff640 1 -- 172.21.15.132:0/1293136346 >> v1:172.21.15.115:6789/0 conn(0x7f64ec0a0e20 legacy=0x7f64ec1d0140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:56.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 1 -- 172.21.15.132:0/1293136346 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 43694219 0 0) 0x7f64ec09dc20 con 0x7f64ec09f5e0 2024-01-24T23:23:56.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 10 monclient(hunting): _negotiate 2024-01-24T23:23:56.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:56.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:56.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 10 monclient(hunting): my global_id is 5322 2024-01-24T23:23:56.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 1 -- 172.21.15.132:0/1293136346 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64d0003530 con 0x7f64ec09f5e0 2024-01-24T23:23:56.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 1 -- 172.21.15.132:0/1293136346 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3977479222 0 0) 0x7f64d0003530 con 0x7f64ec09f5e0 2024-01-24T23:23:56.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 1 -- 172.21.15.132:0/1293136346 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f64ec09dc20 con 0x7f64ec09f5e0 2024-01-24T23:23:56.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 1 -- 172.21.15.132:0/1293136346 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f64e0004330 con 0x7f64ec09f5e0 2024-01-24T23:23:56.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:56.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:56.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 10 monclient(hunting): dump: 2024-01-24T23:23:56.669 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:56.669 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:56.669 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:56.669 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:56.669 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:56.669 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:56.669 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:56.669 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:56.670 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:56.670 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:56.670 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:56.670 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:56.670 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:56.670 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:56.670 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:56.670 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:56.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:56.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 1 -- 172.21.15.132:0/1293136346 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155633341 0 0) 0x7f64ec09c320 con 0x7f64ec13d560 2024-01-24T23:23:56.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 10 monclient(hunting): _negotiate 2024-01-24T23:23:56.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:56.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:56.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 10 monclient(hunting): my global_id is 84584 2024-01-24T23:23:56.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 1 -- 172.21.15.132:0/1293136346 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64d0003530 con 0x7f64ec13d560 2024-01-24T23:23:56.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 1 -- 172.21.15.132:0/1293136346 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1745510830 0 0) 0x7f64ec09dc20 con 0x7f64ec09f5e0 2024-01-24T23:23:56.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 1 -- 172.21.15.132:0/1293136346 >> v1:172.21.15.115:6790/0 conn(0x7f64ec13d560 legacy=0x7f64ec1d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:56.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 1 -- 172.21.15.132:0/1293136346 >> v1:172.21.15.115:6789/0 conn(0x7f64ec0a0e20 legacy=0x7f64ec1d0140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:56.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 10 monclient: _finish_hunting 0 2024-01-24T23:23:56.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 1 monclient: found mon.g 2024-01-24T23:23:56.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:56.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:56.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 1 -- 172.21.15.132:0/1293136346 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ec09e1e0 con 0x7f64ec09f5e0 2024-01-24T23:23:56.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 10 monclient: _finish_auth 0 2024-01-24T23:23:56.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 10 monclient: _check_auth_tickets 2024-01-24T23:23:56.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:56.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f3134640 5 monclient: authenticate success, global_id 5322 2024-01-24T23:23:56.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f3134640 10 monclient: _renew_subs 2024-01-24T23:23:56.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f3134640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:56.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f3134640 1 -- 172.21.15.132:0/1293136346 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ec09cab0 con 0x7f64ec09f5e0 2024-01-24T23:23:56.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f3134640 10 monclient: _renew_subs 2024-01-24T23:23:56.672 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f3134640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:56.672 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f3134640 1 -- 172.21.15.132:0/1293136346 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ec072030 con 0x7f64ec09f5e0 2024-01-24T23:23:56.672 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 1 -- 172.21.15.132:0/1293136346 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f64e0004650 con 0x7f64ec09f5e0 2024-01-24T23:23:56.672 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:56.672 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64eaffd640 4 set_mon_vals no callback set 2024-01-24T23:23:56.672 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 1 -- 172.21.15.132:0/1293136346 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f64e0004b80 con 0x7f64ec09f5e0 2024-01-24T23:23:56.672 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:56.672 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:56.672 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 10 monclient: dump: 2024-01-24T23:23:56.672 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:56.672 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:56.672 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:56.672 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:56.673 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:56.673 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:56.673 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:56.673 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:56.673 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:56.673 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:56.673 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:56.673 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:56.673 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:56.673 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:56.673 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:56.673 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:56.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 1 -- 172.21.15.132:0/1293136346 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f64e0004d60 con 0x7f64ec09f5e0 2024-01-24T23:23:56.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f3134640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:56.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f3134640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:56.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f3134640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:56.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64f3134640 1 -- 172.21.15.132:0/1293136346 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64cc005090 con 0x7f64ec09f5e0 2024-01-24T23:23:56.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64e97fa640 1 -- 172.21.15.132:0/1293136346 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(68..68 src has 1..68) v4 ==== 5208+0+0 (unknown 1190817742 0 0) 0x7f64e0043bd0 con 0x7f64ec09f5e0 2024-01-24T23:23:56.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.655+0000 7f64ebfff640 10 monclient: get_auth_request con 0x7f64d0037060 auth_method 0 2024-01-24T23:23:56.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.659+0000 7f64e97fa640 1 -- 172.21.15.132:0/1293136346 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f64e0014a50 con 0x7f64ec09f5e0 2024-01-24T23:23:56.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.659+0000 7f64e97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:56.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.659+0000 7f64e97fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:56.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.819+0000 7f64f3134640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-01-24T23:23:56.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.819+0000 7f64f3134640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-01-24T23:23:56.824 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.819+0000 7f64f3134640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:56.824 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.823+0000 7f64f3134640 1 -- 172.21.15.132:0/1293136346 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "bar", "type": "root"} v 0) v1 -- 0x7f64cc0058c0 con 0x7f64ec09f5e0 2024-01-24T23:23:56.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.871+0000 7f64e97fa640 1 -- 172.21.15.132:0/1293136346 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}]=0 added bucket bar type root to crush map v69) v1 ==== 138+0+0 (unknown 4094476830 0 0) 0x7f64e0012ec0 con 0x7f64ec09f5e0 2024-01-24T23:23:56.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.871+0000 7f64e97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-01-24T23:23:56.874 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.871+0000 7f64e97fa640 10 monclient: _finish_command 2 = system:0 added bucket bar type root to crush map 2024-01-24T23:23:56.978 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.975+0000 7f64f3134640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-01-24T23:23:56.978 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.975+0000 7f64f3134640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-01-24T23:23:56.978 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.975+0000 7f64f3134640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:56.978 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.975+0000 7f64f3134640 1 -- 172.21.15.132:0/1293136346 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "bar", "type": "root"} v 0) v1 -- 0x7f64cc000bc0 con 0x7f64ec09f5e0 2024-01-24T23:23:56.978 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.975+0000 7f64e97fa640 1 -- 172.21.15.132:0/1293136346 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}]=0 bucket 'bar' already exists v69) v1 ==== 126+0+0 (unknown 3215746958 0 0) 0x7f64e0014820 con 0x7f64ec09f5e0 2024-01-24T23:23:56.978 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.975+0000 7f64e97fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-01-24T23:23:56.979 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.975+0000 7f64e97fa640 10 monclient: _finish_command 3 = system:0 bucket 'bar' already exists 2024-01-24T23:23:56.979 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'bar' already exists 2024-01-24T23:23:56.980 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.979+0000 7f64f3134640 1 -- 172.21.15.132:0/1293136346 >> v1:172.21.15.132:6812/1835401922 conn(0x7f64d0037060 legacy=0x7f64d0039520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:56.980 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.979+0000 7f64f3134640 10 monclient: shutdown 2024-01-24T23:23:56.980 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.979+0000 7f64f3134640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:56.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.979+0000 7f64f3134640 1 -- 172.21.15.132:0/1293136346 >> v1:172.21.15.132:6790/0 conn(0x7f64ec09f5e0 legacy=0x7f64ec071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:56.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.979+0000 7f64f3134640 1 -- 172.21.15.132:0/1293136346 shutdown_connections 2024-01-24T23:23:56.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.979+0000 7f64f3134640 1 -- 172.21.15.132:0/1293136346 >> 172.21.15.132:0/1293136346 conn(0x7f64ec0a7a30 msgr2=0x7f64ec0a58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:56.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.979+0000 7f64f3134640 1 -- 172.21.15.132:0/1293136346 shutdown_connections 2024-01-24T23:23:56.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:56.979+0000 7f64f3134640 1 -- 172.21.15.132:0/1293136346 wait complete. 2024-01-24T23:23:56.995 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket bar root 2024-01-24T23:23:57.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.107+0000 7f37f15fd640 10 monclient: get_monmap_and_config 2024-01-24T23:23:57.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.107+0000 7f37f15fd640 10 monclient: build_initial_monmap 2024-01-24T23:23:57.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.107+0000 7f37f15fd640 10 monclient: monmap: 2024-01-24T23:23:57.110 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:57.110 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:57.111794+0000 2024-01-24T23:23:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:57.111794+0000 2024-01-24T23:23:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:57.111 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:57.111 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:57.118 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.107+0000 7f37f15fd640 1 Processor -- start 2024-01-24T23:23:57.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.107+0000 7f37f15fd640 1 -- start start 2024-01-24T23:23:57.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.107+0000 7f37f15fd640 10 monclient: init 2024-01-24T23:23:57.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:57.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 10 monclient: _add_conns ranks=[2,7,1,3,6,5,0,4,8] 2024-01-24T23:23:57.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 10 monclient(hunting): picked mon.noname-b con 0x7f37ec135e10 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:57.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 10 monclient(hunting): picked mon.noname-i con 0x7f37ec1361f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:57.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 10 monclient(hunting): picked mon.noname-f con 0x7f37ec1365f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:57.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ec064700 con 0x7f37ec1365f0 2024-01-24T23:23:57.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ec09c3e0 con 0x7f37ec135e10 2024-01-24T23:23:57.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ec0a4f90 con 0x7f37ec1361f0 2024-01-24T23:23:57.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:57.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 10 monclient(hunting): authenticate will time out at 1124.436412s 2024-01-24T23:23:57.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37eb7fe640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f37ec1365f0 legacy=0x7f37ec13f600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:57.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37ea7fc640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f37ec1361f0 0x7f37ec13bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:46218/0 (socket says 172.21.15.132:46218) 2024-01-24T23:23:57.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37ea7fc640 1 -- 172.21.15.132:0/463866210 learned_addr learned my addr 172.21.15.132:0/463866210 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:57.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 1 -- 172.21.15.132:0/463866210 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1960346027 0 0) 0x7f37ec0a4f90 con 0x7f37ec1361f0 2024-01-24T23:23:57.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:57.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:57.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:57.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 10 monclient(hunting): my global_id is 5398 2024-01-24T23:23:57.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 1 -- 172.21.15.132:0/463866210 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37d8003530 con 0x7f37ec1361f0 2024-01-24T23:23:57.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 1 -- 172.21.15.132:0/463866210 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2025473242 0 0) 0x7f37d8003530 con 0x7f37ec1361f0 2024-01-24T23:23:57.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 1 -- 172.21.15.132:0/463866210 >> v1:172.21.15.115:6790/0 conn(0x7f37ec135e10 legacy=0x7f37ec09bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:57.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 1 -- 172.21.15.132:0/463866210 >> v1:172.21.15.132:6789/0 conn(0x7f37ec1365f0 legacy=0x7f37ec13f600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:57.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:23:57.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 1 monclient: found mon.noname-i 2024-01-24T23:23:57.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:57.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:23:57.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 1 -- 172.21.15.132:0/463866210 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec09b010 con 0x7f37ec1361f0 2024-01-24T23:23:57.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 10 monclient: _finish_auth 0 2024-01-24T23:23:57.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:23:57.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:57.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 5 monclient: authenticate success, global_id 5398 2024-01-24T23:23:57.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 1 -- 172.21.15.132:0/463866210 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f37d40017c0 con 0x7f37ec1361f0 2024-01-24T23:23:57.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:57.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:57.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:23:57.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 10 monclient: dump: 2024-01-24T23:23:57.121 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:57.121 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:57.121 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:57.121 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:57.122 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:57.122 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:57.122 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:57.122 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:57.122 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:57.122 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:57.122 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:57.122 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:57.122 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:57.122 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:57.122 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:57.122 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:57.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:57.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 1 -- 172.21.15.132:0/463866210 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f37d4001ae0 con 0x7f37ec1361f0 2024-01-24T23:23:57.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:57.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:57.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 4 set_mon_vals no callback set 2024-01-24T23:23:57.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 1 -- 172.21.15.132:0/463866210 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f37d4004bf0 con 0x7f37ec1361f0 2024-01-24T23:23:57.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:57.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:57.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e9ffb640 10 monclient: dump: 2024-01-24T23:23:57.123 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:57.123 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:57.123 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:57.123 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:57.123 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:57.123 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:57.123 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:57.123 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:57.124 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:57.124 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:57.124 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:57.124 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:57.124 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:57.124 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:57.124 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:57.124 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:57.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 10 monclient: shutdown 2024-01-24T23:23:57.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:57.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 1 -- 172.21.15.132:0/463866210 >> v1:172.21.15.132:6792/0 conn(0x7f37ec1361f0 legacy=0x7f37ec13bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:57.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 1 -- 172.21.15.132:0/463866210 shutdown_connections 2024-01-24T23:23:57.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 1 -- 172.21.15.132:0/463866210 >> 172.21.15.132:0/463866210 conn(0x7f37ec0a6790 msgr2=0x7f37ec071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:57.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 1 -- 172.21.15.132:0/463866210 shutdown_connections 2024-01-24T23:23:57.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 1 -- 172.21.15.132:0/463866210 wait complete. 2024-01-24T23:23:57.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 10 monclient: build_initial_monmap 2024-01-24T23:23:57.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 10 monclient: monmap: 2024-01-24T23:23:57.125 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:57.125 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:57.125 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:57.125 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:57.125 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:57.125 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:57.125 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:57.125 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:57.125 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:57.125 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:57.125 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:57.126 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:57.126 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:57.126 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:57.126 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:57.126 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:57.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 1 Processor -- start 2024-01-24T23:23:57.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 1 -- start start 2024-01-24T23:23:57.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 10 monclient: init 2024-01-24T23:23:57.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:57.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 10 monclient: _add_conns ranks=[3,8,0,4,2,6,5,1,7] 2024-01-24T23:23:57.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 10 monclient(hunting): picked mon.noname-g con 0x7f37ec135e10 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:57.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 10 monclient(hunting): picked mon.noname-e con 0x7f37ec1361f0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:57.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 10 monclient(hunting): picked mon.noname-a con 0x7f37ec1365f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:57.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ec09fd10 con 0x7f37ec1365f0 2024-01-24T23:23:57.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ec09c6b0 con 0x7f37ec135e10 2024-01-24T23:23:57.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ec121e00 con 0x7f37ec1361f0 2024-01-24T23:23:57.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:57.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37f15fd640 10 monclient(hunting): authenticate will time out at 1124.439547s 2024-01-24T23:23:57.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37eaffd640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f37ec135e10 0x7f37ec134ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:56970/0 (socket says 172.21.15.132:56970) 2024-01-24T23:23:57.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37eaffd640 1 -- 172.21.15.132:0/1217015400 learned_addr learned my addr 172.21.15.132:0/1217015400 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:57.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37eb7fe640 1 -- 172.21.15.132:0/1217015400 >> v1:172.21.15.115:6789/0 conn(0x7f37ec1365f0 legacy=0x7f37ec1d3a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:57.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e37fe640 1 -- 172.21.15.132:0/1217015400 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3685320500 0 0) 0x7f37ec09c6b0 con 0x7f37ec135e10 2024-01-24T23:23:57.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e37fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:57.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e37fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:57.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e37fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:57.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e37fe640 10 monclient(hunting): my global_id is 5331 2024-01-24T23:23:57.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.111+0000 7f37e37fe640 1 -- 172.21.15.132:0/1217015400 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37cc003530 con 0x7f37ec135e10 2024-01-24T23:23:57.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 1 -- 172.21.15.132:0/1217015400 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 326290276 0 0) 0x7f37cc003530 con 0x7f37ec135e10 2024-01-24T23:23:57.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 1 -- 172.21.15.132:0/1217015400 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f37ec09c6b0 con 0x7f37ec135e10 2024-01-24T23:23:57.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 1 -- 172.21.15.132:0/1217015400 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f37dc001de0 con 0x7f37ec135e10 2024-01-24T23:23:57.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:57.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:57.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 10 monclient(hunting): dump: 2024-01-24T23:23:57.128 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:57.128 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:57.128 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:57.128 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:57.128 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:57.128 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:57.128 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:57.128 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:57.128 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:57.128 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:57.129 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:57.129 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:57.129 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:57.129 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:57.129 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:57.129 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:57.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:57.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 1 -- 172.21.15.132:0/1217015400 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1371374914 0 0) 0x7f37ec09c6b0 con 0x7f37ec135e10 2024-01-24T23:23:57.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 1 -- 172.21.15.132:0/1217015400 >> v1:172.21.15.115:6793/0 conn(0x7f37ec1361f0 legacy=0x7f37ec1d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:57.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 1 -- 172.21.15.132:0/1217015400 >> v1:172.21.15.115:6789/0 conn(0x7f37ec1365f0 legacy=0x7f37ec1d3a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:57.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 10 monclient: _finish_hunting 0 2024-01-24T23:23:57.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 1 monclient: found mon.g 2024-01-24T23:23:57.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:57.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:57.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 1 -- 172.21.15.132:0/1217015400 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec1220e0 con 0x7f37ec135e10 2024-01-24T23:23:57.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 10 monclient: _finish_auth 0 2024-01-24T23:23:57.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 10 monclient: _check_auth_tickets 2024-01-24T23:23:57.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:57.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37f15fd640 5 monclient: authenticate success, global_id 5331 2024-01-24T23:23:57.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37f15fd640 10 monclient: _renew_subs 2024-01-24T23:23:57.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37f15fd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:57.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37f15fd640 1 -- 172.21.15.132:0/1217015400 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ec120f10 con 0x7f37ec135e10 2024-01-24T23:23:57.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37f15fd640 10 monclient: _renew_subs 2024-01-24T23:23:57.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37f15fd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:57.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37f15fd640 1 -- 172.21.15.132:0/1217015400 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ec1cb530 con 0x7f37ec135e10 2024-01-24T23:23:57.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 1 -- 172.21.15.132:0/1217015400 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f37dc003c50 con 0x7f37ec135e10 2024-01-24T23:23:57.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:57.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 1 -- 172.21.15.132:0/1217015400 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f37dc004180 con 0x7f37ec135e10 2024-01-24T23:23:57.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:57.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e97fa640 4 set_mon_vals no callback set 2024-01-24T23:23:57.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:57.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 10 monclient: dump: 2024-01-24T23:23:57.131 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:57.131 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:57.131 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:57.131 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:57.131 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:57.131 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:57.131 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:57.131 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:57.131 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:57.131 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:57.131 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:57.131 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:57.131 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:57.131 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:57.132 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:57.132 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:57.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37f15fd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:57.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 1 -- 172.21.15.132:0/1217015400 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f37dc004360 con 0x7f37ec135e10 2024-01-24T23:23:57.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37f15fd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:57.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37f15fd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:57.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37f15fd640 1 -- 172.21.15.132:0/1217015400 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d0005090 con 0x7f37ec135e10 2024-01-24T23:23:57.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37e37fe640 1 -- 172.21.15.132:0/1217015400 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(69..69 src has 1..69) v4 ==== 5235+0+0 (unknown 3680689509 0 0) 0x7f37dc044170 con 0x7f37ec135e10 2024-01-24T23:23:57.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.115+0000 7f37ea7fc640 10 monclient: get_auth_request con 0x7f37cc036f40 auth_method 0 2024-01-24T23:23:57.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.119+0000 7f37e37fe640 1 -- 172.21.15.132:0/1217015400 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f37dc014e20 con 0x7f37ec135e10 2024-01-24T23:23:57.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.119+0000 7f37e37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:57.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.119+0000 7f37e37fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:57.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.263+0000 7f37f15fd640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-01-24T23:23:57.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.263+0000 7f37f15fd640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-01-24T23:23:57.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.263+0000 7f37f15fd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:57.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.263+0000 7f37f15fd640 1 -- 172.21.15.132:0/1217015400 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "bar", "type": "root"} v 0) v1 -- 0x7f37d00058c0 con 0x7f37ec135e10 2024-01-24T23:23:57.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.267+0000 7f37e37fe640 1 -- 172.21.15.132:0/1217015400 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}]=0 bucket 'bar' already exists v69) v1 ==== 126+0+0 (unknown 3215746958 0 0) 0x7f37dc014a10 con 0x7f37ec135e10 2024-01-24T23:23:57.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.267+0000 7f37e37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-01-24T23:23:57.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.267+0000 7f37e37fe640 10 monclient: _finish_command 2 = system:0 bucket 'bar' already exists 2024-01-24T23:23:57.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.355+0000 7f37f15fd640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-01-24T23:23:57.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.355+0000 7f37f15fd640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-01-24T23:23:57.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.355+0000 7f37f15fd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:57.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.355+0000 7f37f15fd640 1 -- 172.21.15.132:0/1217015400 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "bar", "type": "root"} v 0) v1 -- 0x7f37d0000bc0 con 0x7f37ec135e10 2024-01-24T23:23:57.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.355+0000 7f37e37fe640 1 -- 172.21.15.132:0/1217015400 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}]=0 bucket 'bar' already exists v69) v1 ==== 126+0+0 (unknown 3215746958 0 0) 0x7f37dc014bf0 con 0x7f37ec135e10 2024-01-24T23:23:57.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.355+0000 7f37e37fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-01-24T23:23:57.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.355+0000 7f37e37fe640 10 monclient: _finish_command 3 = system:0 bucket 'bar' already exists 2024-01-24T23:23:57.357 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'bar' already exists 2024-01-24T23:23:57.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.355+0000 7f37f15fd640 1 -- 172.21.15.132:0/1217015400 >> v1:172.21.15.132:6812/1835401922 conn(0x7f37cc036f40 legacy=0x7f37cc039400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:57.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.355+0000 7f37f15fd640 10 monclient: shutdown 2024-01-24T23:23:57.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.355+0000 7f37f15fd640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:57.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.355+0000 7f37f15fd640 1 -- 172.21.15.132:0/1217015400 >> v1:172.21.15.132:6790/0 conn(0x7f37ec135e10 legacy=0x7f37ec134ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:57.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.355+0000 7f37f15fd640 1 -- 172.21.15.132:0/1217015400 shutdown_connections 2024-01-24T23:23:57.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.355+0000 7f37f15fd640 1 -- 172.21.15.132:0/1217015400 >> 172.21.15.132:0/1217015400 conn(0x7f37ec0a6790 msgr2=0x7f37ec0a7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:57.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.355+0000 7f37f15fd640 1 -- 172.21.15.132:0/1217015400 shutdown_connections 2024-01-24T23:23:57.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.355+0000 7f37f15fd640 1 -- 172.21.15.132:0/1217015400 wait complete. 2024-01-24T23:23:57.372 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush link host1 root=bar 2024-01-24T23:23:57.487 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.483+0000 7f95cb5f8640 10 monclient: get_monmap_and_config 2024-01-24T23:23:57.487 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.483+0000 7f95cb5f8640 10 monclient: build_initial_monmap 2024-01-24T23:23:57.487 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.483+0000 7f95cb5f8640 10 monclient: monmap: 2024-01-24T23:23:57.487 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:57.487 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:57.488 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:57.488602+0000 2024-01-24T23:23:57.488 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:57.488602+0000 2024-01-24T23:23:57.488 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:57.488 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:57.488 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:57.488 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:57.488 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:57.488 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:57.488 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:57.488 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:57.488 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:57.488 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:57.488 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:57.488 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:57.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 1 Processor -- start 2024-01-24T23:23:57.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 1 -- start start 2024-01-24T23:23:57.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 10 monclient: init 2024-01-24T23:23:57.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:57.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 10 monclient: _add_conns ranks=[2,5,0,6,3,8,7,4,1] 2024-01-24T23:23:57.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 10 monclient(hunting): picked mon.noname-b con 0x7f95c4135e10 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:57.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 10 monclient(hunting): picked mon.noname-h con 0x7f95c41361f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 10 monclient(hunting): picked mon.noname-a con 0x7f95c41365f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c4064700 con 0x7f95c41365f0 2024-01-24T23:23:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c409c3e0 con 0x7f95c4135e10 2024-01-24T23:23:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c40a4f90 con 0x7f95c41361f0 2024-01-24T23:23:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 10 monclient(hunting): authenticate will time out at 1124.813392s 2024-01-24T23:23:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95c9b95640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f95c41365f0 legacy=0x7f95c413f600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95c8b93640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f95c41361f0 0x7f95c413bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:45870/0 (socket says 172.21.15.132:45870) 2024-01-24T23:23:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95c8b93640 1 -- 172.21.15.132:0/1834276959 learned_addr learned my addr 172.21.15.132:0/1834276959 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 1 -- 172.21.15.132:0/1834276959 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3168300357 0 0) 0x7f95c40a4f90 con 0x7f95c41361f0 2024-01-24T23:23:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:57.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 10 monclient(hunting): my global_id is 5288 2024-01-24T23:23:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 1 -- 172.21.15.132:0/1834276959 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95b0003530 con 0x7f95c41361f0 2024-01-24T23:23:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 1 -- 172.21.15.132:0/1834276959 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3787087384 0 0) 0x7f95c409c3e0 con 0x7f95c4135e10 2024-01-24T23:23:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 10 monclient(hunting): my global_id is 84602 2024-01-24T23:23:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 1 -- 172.21.15.132:0/1834276959 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95c40a4f90 con 0x7f95c4135e10 2024-01-24T23:23:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 1 -- 172.21.15.132:0/1834276959 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2381524225 0 0) 0x7f95b0003530 con 0x7f95c41361f0 2024-01-24T23:23:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 1 -- 172.21.15.132:0/1834276959 >> v1:172.21.15.115:6790/0 conn(0x7f95c4135e10 legacy=0x7f95c409bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 1 -- 172.21.15.132:0/1834276959 >> v1:172.21.15.115:6789/0 conn(0x7f95c41365f0 legacy=0x7f95c413f600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 10 monclient: _finish_hunting 0 2024-01-24T23:23:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 1 monclient: found mon.noname-h 2024-01-24T23:23:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:23:57.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 1 -- 172.21.15.132:0/1834276959 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c409b010 con 0x7f95c41361f0 2024-01-24T23:23:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 10 monclient: _finish_auth 0 2024-01-24T23:23:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 10 monclient: _check_auth_tickets 2024-01-24T23:23:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 1 -- 172.21.15.132:0/1834276959 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f95b40017c0 con 0x7f95c41361f0 2024-01-24T23:23:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 5 monclient: authenticate success, global_id 5288 2024-01-24T23:23:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 10 monclient: dump: 2024-01-24T23:23:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:57.498 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:57.499 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 1 -- 172.21.15.132:0/1834276959 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f95b4001aa0 con 0x7f95c41361f0 2024-01-24T23:23:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 1 -- 172.21.15.132:0/1834276959 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f95b4004bf0 con 0x7f95c41361f0 2024-01-24T23:23:57.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 4 set_mon_vals no callback set 2024-01-24T23:23:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95bbfff640 10 monclient: dump: 2024-01-24T23:23:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:57.500 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:57.501 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:57.501 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:57.501 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:57.501 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:57.501 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:57.501 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:57.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 10 monclient: shutdown 2024-01-24T23:23:57.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:57.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 1 -- 172.21.15.132:0/1834276959 >> v1:172.21.15.132:6791/0 conn(0x7f95c41361f0 legacy=0x7f95c413bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:57.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 1 -- 172.21.15.132:0/1834276959 shutdown_connections 2024-01-24T23:23:57.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 1 -- 172.21.15.132:0/1834276959 >> 172.21.15.132:0/1834276959 conn(0x7f95c40a6790 msgr2=0x7f95c4071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:57.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 1 -- 172.21.15.132:0/1834276959 shutdown_connections 2024-01-24T23:23:57.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 1 -- 172.21.15.132:0/1834276959 wait complete. 2024-01-24T23:23:57.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 10 monclient: build_initial_monmap 2024-01-24T23:23:57.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.487+0000 7f95cb5f8640 10 monclient: monmap: 2024-01-24T23:23:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:57.502 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:57.503 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:57.503 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:57.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95cb5f8640 1 Processor -- start 2024-01-24T23:23:57.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95cb5f8640 1 -- start start 2024-01-24T23:23:57.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95cb5f8640 10 monclient: init 2024-01-24T23:23:57.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95cb5f8640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:57.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95cb5f8640 10 monclient: _add_conns ranks=[3,6,7,4,1,2,8,0,5] 2024-01-24T23:23:57.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95cb5f8640 10 monclient(hunting): picked mon.noname-g con 0x7f95c4135e10 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:57.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95cb5f8640 10 monclient(hunting): picked mon.noname-d con 0x7f95c41361f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:23:57.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95cb5f8640 10 monclient(hunting): picked mon.noname-i con 0x7f95c41365f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:23:57.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95cb5f8640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c4122510 con 0x7f95c4135e10 2024-01-24T23:23:57.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95cb5f8640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c41bf430 con 0x7f95c41361f0 2024-01-24T23:23:57.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95cb5f8640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c4121f50 con 0x7f95c41365f0 2024-01-24T23:23:57.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95cb5f8640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:57.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95cb5f8640 10 monclient(hunting): authenticate will time out at 1124.816780s 2024-01-24T23:23:57.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95c9394640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f95c4135e10 0x7f95c4134e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:56976/0 (socket says 172.21.15.132:56976) 2024-01-24T23:23:57.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95c9394640 1 -- 172.21.15.132:0/2242117955 learned_addr learned my addr 172.21.15.132:0/2242117955 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:57.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 1 -- 172.21.15.132:0/2242117955 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155201256 0 0) 0x7f95c4122510 con 0x7f95c4135e10 2024-01-24T23:23:57.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:57.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:57.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:57.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient(hunting): my global_id is 5340 2024-01-24T23:23:57.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 1 -- 172.21.15.132:0/2242117955 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95a8003530 con 0x7f95c4135e10 2024-01-24T23:23:57.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 1 -- 172.21.15.132:0/2242117955 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2441034877 0 0) 0x7f95c4121f50 con 0x7f95c41365f0 2024-01-24T23:23:57.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:57.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:57.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:57.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient(hunting): my global_id is 5407 2024-01-24T23:23:57.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 1 -- 172.21.15.132:0/2242117955 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95c4122510 con 0x7f95c41365f0 2024-01-24T23:23:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 1 -- 172.21.15.132:0/2242117955 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1321763753 0 0) 0x7f95a8003530 con 0x7f95c4135e10 2024-01-24T23:23:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 1 -- 172.21.15.132:0/2242117955 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f95c4121f50 con 0x7f95c4135e10 2024-01-24T23:23:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 1 -- 172.21.15.132:0/2242117955 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f95c0001e40 con 0x7f95c4135e10 2024-01-24T23:23:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:23:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient(hunting): dump: 2024-01-24T23:23:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:57.505 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:57.506 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:57.506 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:57.506 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:57.506 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:57.506 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:57.506 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:57.506 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:57.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:57.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 1 -- 172.21.15.132:0/2242117955 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2323172929 0 0) 0x7f95c4122510 con 0x7f95c41365f0 2024-01-24T23:23:57.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 1 -- 172.21.15.132:0/2242117955 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f95a8003530 con 0x7f95c41365f0 2024-01-24T23:23:57.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 1 -- 172.21.15.132:0/2242117955 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f95bc0041e0 con 0x7f95c41365f0 2024-01-24T23:23:57.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:57.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-01-24T23:23:57.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient(hunting): dump: 2024-01-24T23:23:57.506 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:57.507 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:57.507 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 1 -- 172.21.15.132:0/2242117955 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1698569412 0 0) 0x7f95c41bf430 con 0x7f95c41361f0 2024-01-24T23:23:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient(hunting): my global_id is 5370 2024-01-24T23:23:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 1 -- 172.21.15.132:0/2242117955 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95c4122510 con 0x7f95c41361f0 2024-01-24T23:23:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 1 -- 172.21.15.132:0/2242117955 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3943431548 0 0) 0x7f95c4121f50 con 0x7f95c4135e10 2024-01-24T23:23:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 1 -- 172.21.15.132:0/2242117955 >> v1:172.21.15.132:6792/0 conn(0x7f95c41365f0 legacy=0x7f95c41d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 1 -- 172.21.15.132:0/2242117955 >> v1:172.21.15.115:6792/0 conn(0x7f95c41361f0 legacy=0x7f95c41d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:23:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 1 monclient: found mon.g 2024-01-24T23:23:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:57.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 1 -- 172.21.15.132:0/2242117955 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c4122230 con 0x7f95c4135e10 2024-01-24T23:23:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient: _finish_auth 0 2024-01-24T23:23:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:23:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95cb5f8640 5 monclient: authenticate success, global_id 5340 2024-01-24T23:23:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95cb5f8640 10 monclient: _renew_subs 2024-01-24T23:23:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95cb5f8640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95cb5f8640 1 -- 172.21.15.132:0/2242117955 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c41beb30 con 0x7f95c4135e10 2024-01-24T23:23:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95cb5f8640 10 monclient: _renew_subs 2024-01-24T23:23:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95cb5f8640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95cb5f8640 1 -- 172.21.15.132:0/2242117955 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c41211c0 con 0x7f95c4135e10 2024-01-24T23:23:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 1 -- 172.21.15.132:0/2242117955 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f95c0003cd0 con 0x7f95c4135e10 2024-01-24T23:23:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 1 -- 172.21.15.132:0/2242117955 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f95c0004200 con 0x7f95c4135e10 2024-01-24T23:23:57.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:23:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95bb7fe640 4 set_mon_vals no callback set 2024-01-24T23:23:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 10 monclient: dump: 2024-01-24T23:23:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:57.510 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:57.511 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:57.511 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:57.511 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:57.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 1 -- 172.21.15.132:0/2242117955 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f95c0004360 con 0x7f95c4135e10 2024-01-24T23:23:57.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95cb5f8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:57.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95cb5f8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:57.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95cb5f8640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:57.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95cb5f8640 1 -- 172.21.15.132:0/2242117955 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a4005090 con 0x7f95c4135e10 2024-01-24T23:23:57.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95b9ffb640 1 -- 172.21.15.132:0/2242117955 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(69..69 src has 1..69) v4 ==== 5235+0+0 (unknown 3680689509 0 0) 0x7f95c0044140 con 0x7f95c4135e10 2024-01-24T23:23:57.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.491+0000 7f95c8b93640 10 monclient: get_auth_request con 0x7f95a8037840 auth_method 0 2024-01-24T23:23:57.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.495+0000 7f95b9ffb640 1 -- 172.21.15.132:0/2242117955 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f95c0014be0 con 0x7f95c4135e10 2024-01-24T23:23:57.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.495+0000 7f95b9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:57.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.495+0000 7f95b9ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:57.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.643+0000 7f95cb5f8640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-01-24T23:23:57.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.643+0000 7f95cb5f8640 10 monclient: _send_command 2 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-01-24T23:23:57.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.643+0000 7f95cb5f8640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:57.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:57.647+0000 7f95cb5f8640 1 -- 172.21.15.132:0/2242117955 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]} v 0) v1 -- 0x7f95a40058c0 con 0x7f95c4135e10 2024-01-24T23:23:58.030 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.027+0000 7f95b9ffb640 1 -- 172.21.15.132:0/2242117955 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}]=0 linked item id -11 name 'host1' to location {root=bar} in crush map v70) v1 ==== 168+0+0 (unknown 3634433402 0 0) 0x7f95c0013280 con 0x7f95c4135e10 2024-01-24T23:23:58.030 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.027+0000 7f95b9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-01-24T23:23:58.030 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.027+0000 7f95b9ffb640 10 monclient: _finish_command 2 = system:0 linked item id -11 name 'host1' to location {root=bar} in crush map 2024-01-24T23:23:58.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.127+0000 7f95cb5f8640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-01-24T23:23:58.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.127+0000 7f95cb5f8640 10 monclient: _send_command 3 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-01-24T23:23:58.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.127+0000 7f95cb5f8640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:23:58.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.127+0000 7f95cb5f8640 1 -- 172.21.15.132:0/2242117955 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]} v 0) v1 -- 0x7f95a4000bc0 con 0x7f95c4135e10 2024-01-24T23:23:58.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.127+0000 7f95b9ffb640 1 -- 172.21.15.132:0/2242117955 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}]=0 no need to move item id -11 name 'host1' to location {root=bar} in crush map v70) v1 ==== 177+0+0 (unknown 4263061562 0 0) 0x7f95c0014a00 con 0x7f95c4135e10 2024-01-24T23:23:58.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.127+0000 7f95b9ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-01-24T23:23:58.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.127+0000 7f95b9ffb640 10 monclient: _finish_command 3 = system:0 no need to move item id -11 name 'host1' to location {root=bar} in crush map 2024-01-24T23:23:58.129 INFO:tasks.workunit.client.0.smithi132.stderr:no need to move item id -11 name 'host1' to location {root=bar} in crush map 2024-01-24T23:23:58.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.127+0000 7f95cb5f8640 1 -- 172.21.15.132:0/2242117955 >> v1:172.21.15.132:6812/1835401922 conn(0x7f95a8037840 legacy=0x7f95a8039d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:58.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.127+0000 7f95cb5f8640 10 monclient: shutdown 2024-01-24T23:23:58.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.127+0000 7f95cb5f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:58.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.127+0000 7f95cb5f8640 1 -- 172.21.15.132:0/2242117955 >> v1:172.21.15.132:6790/0 conn(0x7f95c4135e10 legacy=0x7f95c4134e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:58.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.127+0000 7f95cb5f8640 1 -- 172.21.15.132:0/2242117955 shutdown_connections 2024-01-24T23:23:58.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.127+0000 7f95cb5f8640 1 -- 172.21.15.132:0/2242117955 >> 172.21.15.132:0/2242117955 conn(0x7f95c40a6790 msgr2=0x7f95c40a31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:58.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.127+0000 7f95cb5f8640 1 -- 172.21.15.132:0/2242117955 shutdown_connections 2024-01-24T23:23:58.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.127+0000 7f95cb5f8640 1 -- 172.21.15.132:0/2242117955 wait complete. 2024-01-24T23:23:58.145 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush link host1 root=bar 2024-01-24T23:23:58.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.263+0000 7f46c5c41640 10 monclient: get_monmap_and_config 2024-01-24T23:23:58.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.263+0000 7f46c5c41640 10 monclient: build_initial_monmap 2024-01-24T23:23:58.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.263+0000 7f46c5c41640 10 monclient: monmap: 2024-01-24T23:23:58.266 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:58.266 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:58.266 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:58.267292+0000 2024-01-24T23:23:58.266 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:58.267292+0000 2024-01-24T23:23:58.266 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:58.266 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:58.266 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:58.267 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:58.267 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:58.267 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:58.267 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:58.267 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:58.267 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:58.267 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:58.267 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:58.267 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:58.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.263+0000 7f46c5c41640 1 Processor -- start 2024-01-24T23:23:58.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.263+0000 7f46c5c41640 1 -- start start 2024-01-24T23:23:58.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.263+0000 7f46c5c41640 10 monclient: init 2024-01-24T23:23:58.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.263+0000 7f46c5c41640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:58.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.263+0000 7f46c5c41640 10 monclient: _add_conns ranks=[0,8,2,1,4,5,6,7,3] 2024-01-24T23:23:58.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.263+0000 7f46c5c41640 10 monclient(hunting): picked mon.noname-a con 0x7f46c009ff20 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:58.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.263+0000 7f46c5c41640 10 monclient(hunting): picked mon.noname-e con 0x7f46c009f9c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:58.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.263+0000 7f46c5c41640 10 monclient(hunting): picked mon.noname-b con 0x7f46c013b2c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:58.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.263+0000 7f46c5c41640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c0064700 con 0x7f46c009ff20 2024-01-24T23:23:58.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.263+0000 7f46c5c41640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c009e760 con 0x7f46c013b2c0 2024-01-24T23:23:58.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.263+0000 7f46c5c41640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c00a1160 con 0x7f46c009f9c0 2024-01-24T23:23:58.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.263+0000 7f46c5c41640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:58.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.263+0000 7f46c5c41640 10 monclient(hunting): authenticate will time out at 1125.591992s 2024-01-24T23:23:58.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46bf7fe640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f46c009ff20 legacy=0x7f46c00a0300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:58.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46beffd640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f46c009f9c0 0x7f46c0139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:47240/0 (socket says 172.21.15.132:47240) 2024-01-24T23:23:58.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46beffd640 1 -- 172.21.15.132:0/584261376 learned_addr learned my addr 172.21.15.132:0/584261376 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:58.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 1 -- 172.21.15.132:0/584261376 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2173335690 0 0) 0x7f46c00a1160 con 0x7f46c009f9c0 2024-01-24T23:23:58.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:58.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:58.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:58.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 10 monclient(hunting): my global_id is 84509 2024-01-24T23:23:58.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 1 -- 172.21.15.132:0/584261376 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46a8003530 con 0x7f46c009f9c0 2024-01-24T23:23:58.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 1 -- 172.21.15.132:0/584261376 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3398537118 0 0) 0x7f46c009e760 con 0x7f46c013b2c0 2024-01-24T23:23:58.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:58.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:58.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:58.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 10 monclient(hunting): my global_id is 84611 2024-01-24T23:23:58.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 1 -- 172.21.15.132:0/584261376 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46c00a1160 con 0x7f46c013b2c0 2024-01-24T23:23:58.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 1 -- 172.21.15.132:0/584261376 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 997312346 0 0) 0x7f46c00a1160 con 0x7f46c013b2c0 2024-01-24T23:23:58.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 1 -- 172.21.15.132:0/584261376 >> v1:172.21.15.115:6793/0 conn(0x7f46c009f9c0 legacy=0x7f46c0139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:58.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 1 -- 172.21.15.132:0/584261376 >> v1:172.21.15.115:6789/0 conn(0x7f46c009ff20 legacy=0x7f46c00a0300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:58.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:23:58.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 1 monclient: found mon.noname-b 2024-01-24T23:23:58.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:58.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:23:58.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 1 -- 172.21.15.132:0/584261376 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c00a0a10 con 0x7f46c013b2c0 2024-01-24T23:23:58.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 10 monclient: _finish_auth 0 2024-01-24T23:23:58.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:23:58.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:58.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 1 -- 172.21.15.132:0/584261376 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f46b8001b20 con 0x7f46c013b2c0 2024-01-24T23:23:58.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:58.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:23:58.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 10 monclient: dump: 2024-01-24T23:23:58.273 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:58.273 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:58.274 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:58.274 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:58.274 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:58.274 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:58.274 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:58.274 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:58.274 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:58.274 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:58.274 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:58.274 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:58.275 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:58.275 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:58.275 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:58.275 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:58.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 5 monclient: authenticate success, global_id 84611 2024-01-24T23:23:58.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:58.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 1 -- 172.21.15.132:0/584261376 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f46b8001e40 con 0x7f46c013b2c0 2024-01-24T23:23:58.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:58.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:58.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 1 -- 172.21.15.132:0/584261376 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f46b8003ce0 con 0x7f46c013b2c0 2024-01-24T23:23:58.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:58.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 4 set_mon_vals no callback set 2024-01-24T23:23:58.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:58.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46be7fc640 10 monclient: dump: 2024-01-24T23:23:58.276 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:58.276 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:58.276 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:58.277 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:58.277 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:58.277 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:58.277 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:58.277 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:58.277 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:58.277 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:58.277 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:58.277 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:58.278 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:58.278 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:58.278 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:58.278 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:58.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 10 monclient: shutdown 2024-01-24T23:23:58.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:58.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 1 -- 172.21.15.132:0/584261376 >> v1:172.21.15.115:6790/0 conn(0x7f46c013b2c0 legacy=0x7f46c013d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:58.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 1 -- 172.21.15.132:0/584261376 shutdown_connections 2024-01-24T23:23:58.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 1 -- 172.21.15.132:0/584261376 >> 172.21.15.132:0/584261376 conn(0x7f46c00a5a10 msgr2=0x7f46c00a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:58.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 1 -- 172.21.15.132:0/584261376 shutdown_connections 2024-01-24T23:23:58.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 1 -- 172.21.15.132:0/584261376 wait complete. 2024-01-24T23:23:58.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 10 monclient: build_initial_monmap 2024-01-24T23:23:58.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 10 monclient: monmap: 2024-01-24T23:23:58.279 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:58.279 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:58.279 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:58.280 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:58.280 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:58.280 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:58.280 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:58.280 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:58.280 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:58.280 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:58.280 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:58.280 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:58.281 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:58.281 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:58.281 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:58.281 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:58.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 1 Processor -- start 2024-01-24T23:23:58.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 1 -- start start 2024-01-24T23:23:58.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 10 monclient: init 2024-01-24T23:23:58.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:58.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 10 monclient: _add_conns ranks=[0,8,2,1,6,5,3,4,7] 2024-01-24T23:23:58.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 10 monclient(hunting): picked mon.noname-a con 0x7f46c009f9c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:58.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 10 monclient(hunting): picked mon.noname-e con 0x7f46c009ff20 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:58.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 10 monclient(hunting): picked mon.noname-b con 0x7f46c013b2c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:58.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c0071960 con 0x7f46c009f9c0 2024-01-24T23:23:58.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c011e5d0 con 0x7f46c013b2c0 2024-01-24T23:23:58.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c011e8b0 con 0x7f46c009ff20 2024-01-24T23:23:58.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:58.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46c5c41640 10 monclient(hunting): authenticate will time out at 1125.595824s 2024-01-24T23:23:58.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.267+0000 7f46bf7fe640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f46c009f9c0 legacy=0x7f46c011b610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:58.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46beffd640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f46c009ff20 0x7f46c011bd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:47256/0 (socket says 172.21.15.132:47256) 2024-01-24T23:23:58.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46beffd640 1 -- 172.21.15.132:0/1556843012 learned_addr learned my addr 172.21.15.132:0/1556843012 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:58.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 1 -- 172.21.15.132:0/1556843012 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1093766201 0 0) 0x7f46c011e8b0 con 0x7f46c009ff20 2024-01-24T23:23:58.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:58.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:58.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:58.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 10 monclient(hunting): my global_id is 84518 2024-01-24T23:23:58.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 1 -- 172.21.15.132:0/1556843012 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46ac003560 con 0x7f46c009ff20 2024-01-24T23:23:58.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 1 -- 172.21.15.132:0/1556843012 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3853223073 0 0) 0x7f46c011e5d0 con 0x7f46c013b2c0 2024-01-24T23:23:58.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:58.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:58.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:58.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 10 monclient(hunting): my global_id is 84620 2024-01-24T23:23:58.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 1 -- 172.21.15.132:0/1556843012 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46c011e8b0 con 0x7f46c013b2c0 2024-01-24T23:23:58.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 1 -- 172.21.15.132:0/1556843012 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1422711328 0 0) 0x7f46ac003560 con 0x7f46c009ff20 2024-01-24T23:23:58.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 1 -- 172.21.15.132:0/1556843012 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f46c011e5d0 con 0x7f46c009ff20 2024-01-24T23:23:58.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 1 -- 172.21.15.132:0/1556843012 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f46b4001ac0 con 0x7f46c009ff20 2024-01-24T23:23:58.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:58.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:23:58.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 10 monclient(hunting): dump: 2024-01-24T23:23:58.289 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:58.289 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:58.289 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:58.289 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:58.289 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:58.289 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:58.290 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:58.290 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:58.290 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:58.290 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:58.290 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:58.290 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:58.290 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:58.290 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:58.290 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:58.290 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:58.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:58.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 1 -- 172.21.15.132:0/1556843012 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3166392252 0 0) 0x7f46c011e8b0 con 0x7f46c013b2c0 2024-01-24T23:23:58.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 1 -- 172.21.15.132:0/1556843012 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f46ac003560 con 0x7f46c013b2c0 2024-01-24T23:23:58.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 1 -- 172.21.15.132:0/1556843012 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f46b8001ae0 con 0x7f46c013b2c0 2024-01-24T23:23:58.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:58.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:58.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 10 monclient(hunting): dump: 2024-01-24T23:23:58.291 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:58.291 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:58.291 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:58.291 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:58.291 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:58.291 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:58.291 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:58.291 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:58.291 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:58.291 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:58.291 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:58.291 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:58.291 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:58.291 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:58.291 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:58.292 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:58.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 1 -- 172.21.15.132:0/1556843012 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3191548630 0 0) 0x7f46c011e5d0 con 0x7f46c009ff20 2024-01-24T23:23:58.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 1 -- 172.21.15.132:0/1556843012 >> v1:172.21.15.115:6790/0 conn(0x7f46c013b2c0 legacy=0x7f46c01d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:58.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 1 -- 172.21.15.132:0/1556843012 >> v1:172.21.15.115:6789/0 conn(0x7f46c009f9c0 legacy=0x7f46c011b610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:58.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 10 monclient: _finish_hunting 0 2024-01-24T23:23:58.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 1 monclient: found mon.e 2024-01-24T23:23:58.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:58.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:23:58.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 1 -- 172.21.15.132:0/1556843012 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c011c430 con 0x7f46c009ff20 2024-01-24T23:23:58.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 10 monclient: _finish_auth 0 2024-01-24T23:23:58.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 10 monclient: _check_auth_tickets 2024-01-24T23:23:58.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:58.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46c5c41640 5 monclient: authenticate success, global_id 84518 2024-01-24T23:23:58.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46c5c41640 10 monclient: _renew_subs 2024-01-24T23:23:58.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46c5c41640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:23:58.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46c5c41640 1 -- 172.21.15.132:0/1556843012 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c011d0b0 con 0x7f46c009ff20 2024-01-24T23:23:58.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46c5c41640 10 monclient: _renew_subs 2024-01-24T23:23:58.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46c5c41640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:23:58.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46c5c41640 1 -- 172.21.15.132:0/1556843012 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c011ce30 con 0x7f46c009ff20 2024-01-24T23:23:58.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 1 -- 172.21.15.132:0/1556843012 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f46b4001d00 con 0x7f46c009ff20 2024-01-24T23:23:58.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:58.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46bdffb640 4 set_mon_vals no callback set 2024-01-24T23:23:58.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 1 -- 172.21.15.132:0/1556843012 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f46b4004de0 con 0x7f46c009ff20 2024-01-24T23:23:58.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:58.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:58.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 10 monclient: dump: 2024-01-24T23:23:58.293 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:58.293 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:58.294 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:58.294 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:58.294 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:58.294 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:58.294 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:58.294 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:58.294 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:58.294 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:58.294 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:58.294 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:58.294 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:58.294 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:58.294 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:58.294 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:58.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46c5c41640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:58.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46c5c41640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:58.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46c5c41640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:23:58.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46c5c41640 1 -- 172.21.15.132:0/1556843012 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a0005090 con 0x7f46c009ff20 2024-01-24T23:23:58.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 1 -- 172.21.15.132:0/1556843012 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f46b400f860 con 0x7f46c009ff20 2024-01-24T23:23:58.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46a7fff640 1 -- 172.21.15.132:0/1556843012 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(70..70 src has 1..70) v4 ==== 5361+0+0 (unknown 199556285 0 0) 0x7f46b4043da0 con 0x7f46c009ff20 2024-01-24T23:23:58.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.271+0000 7f46bf7fe640 10 monclient: get_auth_request con 0x7f46ac037700 auth_method 0 2024-01-24T23:23:58.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.283+0000 7f46a7fff640 1 -- 172.21.15.132:0/1556843012 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f46b4014740 con 0x7f46c009ff20 2024-01-24T23:23:58.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.283+0000 7f46a7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:58.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.283+0000 7f46a7fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:58.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.443+0000 7f46c5c41640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-01-24T23:23:58.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.443+0000 7f46c5c41640 10 monclient: _send_command 2 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-01-24T23:23:58.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.443+0000 7f46c5c41640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:23:58.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.443+0000 7f46c5c41640 1 -- 172.21.15.132:0/1556843012 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]} v 0) v1 -- 0x7f46a0005b50 con 0x7f46c009ff20 2024-01-24T23:23:58.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.447+0000 7f46a7fff640 1 -- 172.21.15.132:0/1556843012 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}]=0 no need to move item id -11 name 'host1' to location {root=bar} in crush map v70) v1 ==== 177+0+0 (unknown 4263061562 0 0) 0x7f46b4014560 con 0x7f46c009ff20 2024-01-24T23:23:58.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.447+0000 7f46a7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-01-24T23:23:58.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.447+0000 7f46a7fff640 10 monclient: _finish_command 2 = system:0 no need to move item id -11 name 'host1' to location {root=bar} in crush map 2024-01-24T23:23:58.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.539+0000 7f46c5c41640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-01-24T23:23:58.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.539+0000 7f46c5c41640 10 monclient: _send_command 3 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-01-24T23:23:58.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.539+0000 7f46c5c41640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:23:58.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.539+0000 7f46c5c41640 1 -- 172.21.15.132:0/1556843012 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]} v 0) v1 -- 0x7f46a00055d0 con 0x7f46c009ff20 2024-01-24T23:23:58.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.543+0000 7f46a7fff640 1 -- 172.21.15.132:0/1556843012 <== mon.8 v1:172.21.15.115:6793/0 11 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}]=0 no need to move item id -11 name 'host1' to location {root=bar} in crush map v70) v1 ==== 177+0+0 (unknown 4263061562 0 0) 0x7f46b4018c90 con 0x7f46c009ff20 2024-01-24T23:23:58.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.543+0000 7f46a7fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-01-24T23:23:58.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.543+0000 7f46a7fff640 10 monclient: _finish_command 3 = system:0 no need to move item id -11 name 'host1' to location {root=bar} in crush map 2024-01-24T23:23:58.543 INFO:tasks.workunit.client.0.smithi132.stderr:no need to move item id -11 name 'host1' to location {root=bar} in crush map 2024-01-24T23:23:58.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.543+0000 7f46c5c41640 1 -- 172.21.15.132:0/1556843012 >> v1:172.21.15.132:6812/1835401922 conn(0x7f46ac037700 legacy=0x7f46ac039bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:58.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.543+0000 7f46c5c41640 10 monclient: shutdown 2024-01-24T23:23:58.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.543+0000 7f46c5c41640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:58.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.543+0000 7f46c5c41640 1 -- 172.21.15.132:0/1556843012 >> v1:172.21.15.115:6793/0 conn(0x7f46c009ff20 legacy=0x7f46c011bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:58.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.543+0000 7f46c5c41640 1 -- 172.21.15.132:0/1556843012 shutdown_connections 2024-01-24T23:23:58.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.543+0000 7f46c5c41640 1 -- 172.21.15.132:0/1556843012 >> 172.21.15.132:0/1556843012 conn(0x7f46c00a5a10 msgr2=0x7f46c0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:58.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.543+0000 7f46c5c41640 1 -- 172.21.15.132:0/1556843012 shutdown_connections 2024-01-24T23:23:58.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.543+0000 7f46c5c41640 1 -- 172.21.15.132:0/1556843012 wait complete. 2024-01-24T23:23:58.559 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush link host2 root=bar 2024-01-24T23:23:58.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 10 monclient: get_monmap_and_config 2024-01-24T23:23:58.672 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 10 monclient: build_initial_monmap 2024-01-24T23:23:58.672 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 10 monclient: monmap: 2024-01-24T23:23:58.672 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:58.672 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:58.672 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:58.673016+0000 2024-01-24T23:23:58.672 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:58.673016+0000 2024-01-24T23:23:58.672 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:58.672 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:58.672 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:58.672 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:58.672 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:58.672 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:58.673 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:58.673 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:58.673 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:58.673 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:58.673 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:58.673 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:58.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 1 Processor -- start 2024-01-24T23:23:58.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 1 -- start start 2024-01-24T23:23:58.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 10 monclient: init 2024-01-24T23:23:58.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:58.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 10 monclient: _add_conns ranks=[8,0,2,7,3,5,4,6,1] 2024-01-24T23:23:58.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 10 monclient(hunting): picked mon.noname-e con 0x7f1330135e10 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:58.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 10 monclient(hunting): picked mon.noname-a con 0x7f13301361f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:58.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 10 monclient(hunting): picked mon.noname-b con 0x7f13301365f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:58.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1330064700 con 0x7f13301361f0 2024-01-24T23:23:58.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133009c3e0 con 0x7f13301365f0 2024-01-24T23:23:58.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13300a4f90 con 0x7f1330135e10 2024-01-24T23:23:58.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:58.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 10 monclient(hunting): authenticate will time out at 1125.997442s 2024-01-24T23:23:58.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1334a04640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f13301361f0 legacy=0x7f133013bed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:58.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1335205640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f1330135e10 0x7f133009bcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:47266/0 (socket says 172.21.15.132:47266) 2024-01-24T23:23:58.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1335205640 1 -- 172.21.15.132:0/3974894746 learned_addr learned my addr 172.21.15.132:0/3974894746 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:58.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 1 -- 172.21.15.132:0/3974894746 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1913119906 0 0) 0x7f13300a4f90 con 0x7f1330135e10 2024-01-24T23:23:58.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:58.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:58.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:58.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 10 monclient(hunting): my global_id is 84527 2024-01-24T23:23:58.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 1 -- 172.21.15.132:0/3974894746 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1318003530 con 0x7f1330135e10 2024-01-24T23:23:58.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 1 -- 172.21.15.132:0/3974894746 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836689494 0 0) 0x7f133009c3e0 con 0x7f13301365f0 2024-01-24T23:23:58.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:23:58.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:58.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:58.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 10 monclient(hunting): my global_id is 84629 2024-01-24T23:23:58.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 1 -- 172.21.15.132:0/3974894746 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13300a4f90 con 0x7f13301365f0 2024-01-24T23:23:58.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 1 -- 172.21.15.132:0/3974894746 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1356061448 0 0) 0x7f1318003530 con 0x7f1330135e10 2024-01-24T23:23:58.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 1 -- 172.21.15.132:0/3974894746 >> v1:172.21.15.115:6790/0 conn(0x7f13301365f0 legacy=0x7f133013f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:58.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 1 -- 172.21.15.132:0/3974894746 >> v1:172.21.15.115:6789/0 conn(0x7f13301361f0 legacy=0x7f133013bed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:58.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 10 monclient: _finish_hunting 0 2024-01-24T23:23:58.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 1 monclient: found mon.noname-e 2024-01-24T23:23:58.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:58.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:23:58.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 1 -- 172.21.15.132:0/3974894746 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133009b010 con 0x7f1330135e10 2024-01-24T23:23:58.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 10 monclient: _finish_auth 0 2024-01-24T23:23:58.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 10 monclient: _check_auth_tickets 2024-01-24T23:23:58.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:58.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 1 -- 172.21.15.132:0/3974894746 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1320001b40 con 0x7f1330135e10 2024-01-24T23:23:58.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:58.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:23:58.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 10 monclient: dump: 2024-01-24T23:23:58.682 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:58.682 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:58.682 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:58.682 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:58.682 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:58.682 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:58.682 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:58.682 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:58.682 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:58.682 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:58.682 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:58.683 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:58.683 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:58.683 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:58.683 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:58.683 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:58.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 5 monclient: authenticate success, global_id 84527 2024-01-24T23:23:58.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:58.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 1 -- 172.21.15.132:0/3974894746 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1320001e20 con 0x7f1330135e10 2024-01-24T23:23:58.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:58.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 1 -- 172.21.15.132:0/3974894746 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1320004f70 con 0x7f1330135e10 2024-01-24T23:23:58.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:58.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:23:58.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f132ffff640 10 monclient: dump: 2024-01-24T23:23:58.683 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:58.683 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:58.684 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:58.684 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:58.684 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:58.684 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:58.684 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:58.684 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:58.684 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:58.684 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:58.684 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:58.684 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:58.684 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:58.684 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:58.684 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:58.684 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:58.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:58.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 4 set_mon_vals no callback set 2024-01-24T23:23:58.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 10 monclient: shutdown 2024-01-24T23:23:58.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:58.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 1 -- 172.21.15.132:0/3974894746 >> v1:172.21.15.115:6793/0 conn(0x7f1330135e10 legacy=0x7f133009bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:58.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 1 -- 172.21.15.132:0/3974894746 shutdown_connections 2024-01-24T23:23:58.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 1 -- 172.21.15.132:0/3974894746 >> 172.21.15.132:0/3974894746 conn(0x7f13300a6790 msgr2=0x7f1330071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:58.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 1 -- 172.21.15.132:0/3974894746 shutdown_connections 2024-01-24T23:23:58.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 1 -- 172.21.15.132:0/3974894746 wait complete. 2024-01-24T23:23:58.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.671+0000 7f1337469640 10 monclient: build_initial_monmap 2024-01-24T23:23:58.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 10 monclient: monmap: 2024-01-24T23:23:58.685 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:58.685 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:58.685 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:58.685 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:58.686 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:58.686 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:58.686 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:58.686 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:58.686 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:58.686 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:58.686 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:58.686 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:58.686 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:58.686 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:58.686 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:58.686 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:58.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 1 Processor -- start 2024-01-24T23:23:58.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 1 -- start start 2024-01-24T23:23:58.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 10 monclient: init 2024-01-24T23:23:58.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:58.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 10 monclient: _add_conns ranks=[2,4,1,5,6,3,8,0,7] 2024-01-24T23:23:58.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 10 monclient(hunting): picked mon.noname-b con 0x7f1330135e10 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:58.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 10 monclient(hunting): picked mon.noname-c con 0x7f13301361f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:23:58.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 10 monclient(hunting): picked mon.noname-f con 0x7f13301365f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:58.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13301bf410 con 0x7f13301365f0 2024-01-24T23:23:58.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133009ca00 con 0x7f1330135e10 2024-01-24T23:23:58.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133009c6b0 con 0x7f13301361f0 2024-01-24T23:23:58.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:58.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 10 monclient(hunting): authenticate will time out at 1126.000879s 2024-01-24T23:23:58.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1335a06640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f13301365f0 legacy=0x7f13301d3aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:58.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1334a04640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f13301361f0 legacy=0x7f13301d0370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:23:58.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1335205640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f1330135e10 0x7f1330134e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:42290/0 (socket says 172.21.15.132:42290) 2024-01-24T23:23:58.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1335205640 1 -- 172.21.15.132:0/2524175680 learned_addr learned my addr 172.21.15.132:0/2524175680 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:58.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 1 -- 172.21.15.132:0/2524175680 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2069351877 0 0) 0x7f133009ca00 con 0x7f1330135e10 2024-01-24T23:23:58.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:23:58.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:58.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:58.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 10 monclient(hunting): my global_id is 84638 2024-01-24T23:23:58.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 1 -- 172.21.15.132:0/2524175680 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f131c003530 con 0x7f1330135e10 2024-01-24T23:23:58.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 1 -- 172.21.15.132:0/2524175680 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1928681229 0 0) 0x7f131c003530 con 0x7f1330135e10 2024-01-24T23:23:58.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 1 -- 172.21.15.132:0/2524175680 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f133009ca00 con 0x7f1330135e10 2024-01-24T23:23:58.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 1 -- 172.21.15.132:0/2524175680 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1320004eb0 con 0x7f1330135e10 2024-01-24T23:23:58.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:58.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:23:58.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 10 monclient(hunting): dump: 2024-01-24T23:23:58.688 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:58.688 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:58.688 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:58.689 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:58.689 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:58.689 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:58.689 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:58.689 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:58.689 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:58.689 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:58.689 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:58.689 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:58.689 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:58.689 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:58.689 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:58.689 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:58.689 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:58.689 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 1 -- 172.21.15.132:0/2524175680 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1810150830 0 0) 0x7f133009ca00 con 0x7f1330135e10 2024-01-24T23:23:58.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 1 -- 172.21.15.132:0/2524175680 >> v1:172.21.15.115:6791/0 conn(0x7f13301361f0 legacy=0x7f13301d0370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:58.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 1 -- 172.21.15.132:0/2524175680 >> v1:172.21.15.132:6789/0 conn(0x7f13301365f0 legacy=0x7f13301d3aa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:58.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 10 monclient: _finish_hunting 0 2024-01-24T23:23:58.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 1 monclient: found mon.b 2024-01-24T23:23:58.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:58.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:58.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 1 -- 172.21.15.132:0/2524175680 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1330122500 con 0x7f1330135e10 2024-01-24T23:23:58.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 10 monclient: _finish_auth 0 2024-01-24T23:23:58.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 10 monclient: _check_auth_tickets 2024-01-24T23:23:58.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:58.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 5 monclient: authenticate success, global_id 84638 2024-01-24T23:23:58.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 10 monclient: _renew_subs 2024-01-24T23:23:58.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:58.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 1 -- 172.21.15.132:0/2524175680 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13301227e0 con 0x7f1330135e10 2024-01-24T23:23:58.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 10 monclient: _renew_subs 2024-01-24T23:23:58.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:58.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 1 -- 172.21.15.132:0/2524175680 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f13301bea40 con 0x7f1330135e10 2024-01-24T23:23:58.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 1 -- 172.21.15.132:0/2524175680 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1320005240 con 0x7f1330135e10 2024-01-24T23:23:58.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:58.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 1 -- 172.21.15.132:0/2524175680 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f13200056f0 con 0x7f1330135e10 2024-01-24T23:23:58.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:58.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132f7fe640 4 set_mon_vals no callback set 2024-01-24T23:23:58.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:58.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 10 monclient: dump: 2024-01-24T23:23:58.691 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:58.691 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:58.691 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:58.691 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:58.691 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:58.692 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:58.692 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:58.692 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:58.692 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:58.692 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:58.692 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:58.692 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:58.692 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:58.692 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:58.692 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:58.692 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:58.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:58.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:58.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:58.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1337469640 1 -- 172.21.15.132:0/2524175680 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1310005090 con 0x7f1330135e10 2024-01-24T23:23:58.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 1 -- 172.21.15.132:0/2524175680 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f1320010170 con 0x7f1330135e10 2024-01-24T23:23:58.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f132dffb640 1 -- 172.21.15.132:0/2524175680 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(70..70 src has 1..70) v4 ==== 5361+0+0 (unknown 199556285 0 0) 0x7f13200443f0 con 0x7f1330135e10 2024-01-24T23:23:58.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.675+0000 7f1334a04640 10 monclient: get_auth_request con 0x7f131c036f00 auth_method 0 2024-01-24T23:23:58.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.687+0000 7f132dffb640 1 -- 172.21.15.132:0/2524175680 <== mon.2 v1:172.21.15.115:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f1320014eb0 con 0x7f1330135e10 2024-01-24T23:23:58.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.687+0000 7f132dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:58.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.687+0000 7f132dffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:58.837 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.835+0000 7f1337469640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-01-24T23:23:58.838 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.835+0000 7f1337469640 10 monclient: _send_command 2 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-01-24T23:23:58.838 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.835+0000 7f1337469640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:58.838 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:58.835+0000 7f1337469640 1 -- 172.21.15.132:0/2524175680 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]} v 0) v1 -- 0x7f1310005b50 con 0x7f1330135e10 2024-01-24T23:23:59.039 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.035+0000 7f132dffb640 1 -- 172.21.15.132:0/2524175680 <== mon.2 v1:172.21.15.115:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}]=0 linked item id -16 name 'host2' to location {root=bar} in crush map v71) v1 ==== 168+0+0 (unknown 758864997 0 0) 0x7f1320014cd0 con 0x7f1330135e10 2024-01-24T23:23:59.039 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.039+0000 7f132dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-01-24T23:23:59.039 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.039+0000 7f132dffb640 10 monclient: _finish_command 2 = system:0 linked item id -16 name 'host2' to location {root=bar} in crush map 2024-01-24T23:23:59.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.131+0000 7f1337469640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-01-24T23:23:59.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.131+0000 7f1337469640 10 monclient: _send_command 3 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-01-24T23:23:59.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.131+0000 7f1337469640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:59.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.131+0000 7f1337469640 1 -- 172.21.15.132:0/2524175680 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]} v 0) v1 -- 0x7f13100055d0 con 0x7f1330135e10 2024-01-24T23:23:59.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.135+0000 7f132dffb640 1 -- 172.21.15.132:0/2524175680 <== mon.2 v1:172.21.15.115:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}]=0 no need to move item id -16 name 'host2' to location {root=bar} in crush map v71) v1 ==== 177+0+0 (unknown 750988058 0 0) 0x7f1320015310 con 0x7f1330135e10 2024-01-24T23:23:59.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.135+0000 7f132dffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-01-24T23:23:59.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.135+0000 7f132dffb640 10 monclient: _finish_command 3 = system:0 no need to move item id -16 name 'host2' to location {root=bar} in crush map 2024-01-24T23:23:59.137 INFO:tasks.workunit.client.0.smithi132.stderr:no need to move item id -16 name 'host2' to location {root=bar} in crush map 2024-01-24T23:23:59.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.135+0000 7f1337469640 1 -- 172.21.15.132:0/2524175680 >> v1:172.21.15.132:6812/1835401922 conn(0x7f131c036f00 legacy=0x7f131c0393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:59.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.135+0000 7f1337469640 10 monclient: shutdown 2024-01-24T23:23:59.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.135+0000 7f1337469640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:59.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.135+0000 7f1337469640 1 -- 172.21.15.132:0/2524175680 >> v1:172.21.15.115:6790/0 conn(0x7f1330135e10 legacy=0x7f1330134e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:59.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.135+0000 7f1337469640 1 -- 172.21.15.132:0/2524175680 shutdown_connections 2024-01-24T23:23:59.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.135+0000 7f1337469640 1 -- 172.21.15.132:0/2524175680 >> 172.21.15.132:0/2524175680 conn(0x7f13300a6790 msgr2=0x7f133006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:59.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.139+0000 7f1337469640 1 -- 172.21.15.132:0/2524175680 shutdown_connections 2024-01-24T23:23:59.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.139+0000 7f1337469640 1 -- 172.21.15.132:0/2524175680 wait complete. 2024-01-24T23:23:59.153 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush link host2 root=bar 2024-01-24T23:23:59.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.267+0000 7f3ab3c63640 10 monclient: get_monmap_and_config 2024-01-24T23:23:59.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.267+0000 7f3ab3c63640 10 monclient: build_initial_monmap 2024-01-24T23:23:59.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.267+0000 7f3ab3c63640 10 monclient: monmap: 2024-01-24T23:23:59.270 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:59.270 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:59.271 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:59.271578+0000 2024-01-24T23:23:59.271 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:59.271578+0000 2024-01-24T23:23:59.271 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:59.271 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:59.271 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:59.271 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:59.271 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:59.271 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:59.271 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:59.271 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:59.271 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:59.271 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:59.271 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:59.272 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:59.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 1 Processor -- start 2024-01-24T23:23:59.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 1 -- start start 2024-01-24T23:23:59.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 10 monclient: init 2024-01-24T23:23:59.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:59.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 10 monclient: _add_conns ranks=[5,3,1,2,8,6,4,0,7] 2024-01-24T23:23:59.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 10 monclient(hunting): picked mon.noname-h con 0x7f3aac135e10 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:59.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 10 monclient(hunting): picked mon.noname-g con 0x7f3aac1361f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:23:59.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 10 monclient(hunting): picked mon.noname-f con 0x7f3aac1365f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:59.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aac064700 con 0x7f3aac1365f0 2024-01-24T23:23:59.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aac09c3e0 con 0x7f3aac1361f0 2024-01-24T23:23:59.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aac0a4f90 con 0x7f3aac135e10 2024-01-24T23:23:59.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:59.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 10 monclient(hunting): authenticate will time out at 1126.596940s 2024-01-24T23:23:59.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab2200640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f3aac1365f0 legacy=0x7f3aac13f600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:59.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab19ff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f3aac135e10 0x7f3aac09bcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:45884/0 (socket says 172.21.15.132:45884) 2024-01-24T23:23:59.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab19ff640 1 -- 172.21.15.132:0/1800106120 learned_addr learned my addr 172.21.15.132:0/1800106120 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:59.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab11fe640 1 --1- 172.21.15.132:0/1800106120 >> v1:172.21.15.132:6790/0 conn(0x7f3aac1361f0 0x7f3aac13bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:56978/0 (socket says 172.21.15.132:56978) 2024-01-24T23:23:59.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 1 -- 172.21.15.132:0/1800106120 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3668881124 0 0) 0x7f3aac0a4f90 con 0x7f3aac135e10 2024-01-24T23:23:59.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:59.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:59.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:59.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 10 monclient(hunting): my global_id is 5297 2024-01-24T23:23:59.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 1 -- 172.21.15.132:0/1800106120 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a98003530 con 0x7f3aac135e10 2024-01-24T23:23:59.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 1 -- 172.21.15.132:0/1800106120 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1208545303 0 0) 0x7f3aac09c3e0 con 0x7f3aac1361f0 2024-01-24T23:23:59.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 10 monclient(hunting): _negotiate 2024-01-24T23:23:59.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:59.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:59.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 10 monclient(hunting): my global_id is 5349 2024-01-24T23:23:59.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 1 -- 172.21.15.132:0/1800106120 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3aac0a4f90 con 0x7f3aac1361f0 2024-01-24T23:23:59.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 1 -- 172.21.15.132:0/1800106120 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2645644618 0 0) 0x7f3a98003530 con 0x7f3aac135e10 2024-01-24T23:23:59.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 1 -- 172.21.15.132:0/1800106120 >> v1:172.21.15.132:6790/0 conn(0x7f3aac1361f0 legacy=0x7f3aac13bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:59.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 1 -- 172.21.15.132:0/1800106120 >> v1:172.21.15.132:6789/0 conn(0x7f3aac1365f0 legacy=0x7f3aac13f600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:59.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 10 monclient: _finish_hunting 0 2024-01-24T23:23:59.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 1 monclient: found mon.noname-h 2024-01-24T23:23:59.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:59.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:23:59.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 1 -- 172.21.15.132:0/1800106120 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac09b010 con 0x7f3aac135e10 2024-01-24T23:23:59.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 10 monclient: _finish_auth 0 2024-01-24T23:23:59.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 10 monclient: _check_auth_tickets 2024-01-24T23:23:59.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:59.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 5 monclient: authenticate success, global_id 5297 2024-01-24T23:23:59.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 1 -- 172.21.15.132:0/1800106120 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3aa80016f0 con 0x7f3aac135e10 2024-01-24T23:23:59.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:59.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:59.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:59.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 10 monclient: dump: 2024-01-24T23:23:59.281 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:59.281 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:59.281 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:59.281 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:59.281 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:59.281 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:59.281 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:59.281 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:59.281 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:59.281 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:59.281 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:59.281 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:59.281 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:59.281 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:59.281 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:59.282 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:59.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:59.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 1 -- 172.21.15.132:0/1800106120 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3aa80019d0 con 0x7f3aac135e10 2024-01-24T23:23:59.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:59.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:59.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 4 set_mon_vals no callback set 2024-01-24T23:23:59.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 1 -- 172.21.15.132:0/1800106120 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3aa8004bf0 con 0x7f3aac135e10 2024-01-24T23:23:59.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:59.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:59.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab09fd640 10 monclient: dump: 2024-01-24T23:23:59.282 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:59.282 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:59.282 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:59.282 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:59.282 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:59.282 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:59.283 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:59.283 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:59.283 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:59.283 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:59.283 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:59.283 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:59.283 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:59.283 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:59.283 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:59.283 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:59.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 10 monclient: shutdown 2024-01-24T23:23:59.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:59.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 1 -- 172.21.15.132:0/1800106120 >> v1:172.21.15.132:6791/0 conn(0x7f3aac135e10 legacy=0x7f3aac09bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:59.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 1 -- 172.21.15.132:0/1800106120 shutdown_connections 2024-01-24T23:23:59.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 1 -- 172.21.15.132:0/1800106120 >> 172.21.15.132:0/1800106120 conn(0x7f3aac0a6790 msgr2=0x7f3aac071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:59.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 1 -- 172.21.15.132:0/1800106120 shutdown_connections 2024-01-24T23:23:59.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 1 -- 172.21.15.132:0/1800106120 wait complete. 2024-01-24T23:23:59.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 10 monclient: build_initial_monmap 2024-01-24T23:23:59.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 10 monclient: monmap: 2024-01-24T23:23:59.284 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:59.284 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:59.284 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:59.284 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:59.284 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:59.284 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:59.284 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:59.284 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:59.284 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:59.284 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:59.284 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:59.284 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:59.284 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:59.284 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:59.285 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:59.285 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:59.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 1 Processor -- start 2024-01-24T23:23:59.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 1 -- start start 2024-01-24T23:23:59.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 10 monclient: init 2024-01-24T23:23:59.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:59.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 10 monclient: _add_conns ranks=[1,5,8,3,4,7,2,0,6] 2024-01-24T23:23:59.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 10 monclient(hunting): picked mon.noname-f con 0x7f3aac135e10 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:59.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.271+0000 7f3ab3c63640 10 monclient(hunting): picked mon.noname-h con 0x7f3aac1361f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:59.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3ab3c63640 10 monclient(hunting): picked mon.noname-e con 0x7f3aac1365f0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:59.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3ab3c63640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aac09c6b0 con 0x7f3aac135e10 2024-01-24T23:23:59.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3ab3c63640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aac1bf260 con 0x7f3aac1361f0 2024-01-24T23:23:59.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3ab3c63640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aac122780 con 0x7f3aac1365f0 2024-01-24T23:23:59.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3ab3c63640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:59.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3ab3c63640 10 monclient(hunting): authenticate will time out at 1126.600040s 2024-01-24T23:23:59.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3ab19ff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f3aac135e10 legacy=0x7f3aac134dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:59.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3ab11fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f3aac1361f0 0x7f3aac1d01f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:45886/0 (socket says 172.21.15.132:45886) 2024-01-24T23:23:59.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3ab11fe640 1 -- 172.21.15.132:0/1150201484 learned_addr learned my addr 172.21.15.132:0/1150201484 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:59.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 1 -- 172.21.15.132:0/1150201484 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761624835 0 0) 0x7f3aac1bf260 con 0x7f3aac1361f0 2024-01-24T23:23:59.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 10 monclient(hunting): _negotiate 2024-01-24T23:23:59.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:59.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:59.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 10 monclient(hunting): my global_id is 5306 2024-01-24T23:23:59.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 1 -- 172.21.15.132:0/1150201484 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a90003530 con 0x7f3aac1361f0 2024-01-24T23:23:59.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 1 -- 172.21.15.132:0/1150201484 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1930567599 0 0) 0x7f3a90003530 con 0x7f3aac1361f0 2024-01-24T23:23:59.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 1 -- 172.21.15.132:0/1150201484 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3aac1bf260 con 0x7f3aac1361f0 2024-01-24T23:23:59.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 1 -- 172.21.15.132:0/1150201484 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3a9c001460 con 0x7f3aac1361f0 2024-01-24T23:23:59.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:59.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:59.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 10 monclient(hunting): dump: 2024-01-24T23:23:59.287 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:59.287 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:59.287 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:59.287 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:59.287 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:59.287 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:59.287 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:59.287 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:59.287 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:59.287 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:59.287 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:59.287 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:59.287 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:59.287 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:59.287 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:59.288 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:59.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:59.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 1 -- 172.21.15.132:0/1150201484 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3799819713 0 0) 0x7f3aac1bf260 con 0x7f3aac1361f0 2024-01-24T23:23:59.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 1 -- 172.21.15.132:0/1150201484 >> v1:172.21.15.115:6793/0 conn(0x7f3aac1365f0 legacy=0x7f3aac1d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:59.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 1 -- 172.21.15.132:0/1150201484 >> v1:172.21.15.132:6789/0 conn(0x7f3aac135e10 legacy=0x7f3aac134dc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:59.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 10 monclient: _finish_hunting 0 2024-01-24T23:23:59.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 1 monclient: found mon.h 2024-01-24T23:23:59.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:59.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:59.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 1 -- 172.21.15.132:0/1150201484 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac121ee0 con 0x7f3aac1361f0 2024-01-24T23:23:59.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 10 monclient: _finish_auth 0 2024-01-24T23:23:59.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 10 monclient: _check_auth_tickets 2024-01-24T23:23:59.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:59.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3ab3c63640 5 monclient: authenticate success, global_id 5306 2024-01-24T23:23:59.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3ab3c63640 10 monclient: _renew_subs 2024-01-24T23:23:59.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3ab3c63640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:59.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3ab3c63640 1 -- 172.21.15.132:0/1150201484 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aac1221c0 con 0x7f3aac1361f0 2024-01-24T23:23:59.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3ab3c63640 10 monclient: _renew_subs 2024-01-24T23:23:59.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3ab3c63640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:59.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3ab3c63640 1 -- 172.21.15.132:0/1150201484 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aac1be920 con 0x7f3aac1361f0 2024-01-24T23:23:59.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 1 -- 172.21.15.132:0/1150201484 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3a9c004aa0 con 0x7f3aac1361f0 2024-01-24T23:23:59.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:59.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 1 -- 172.21.15.132:0/1150201484 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3a9c004f50 con 0x7f3aac1361f0 2024-01-24T23:23:59.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:59.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa3fff640 4 set_mon_vals no callback set 2024-01-24T23:23:59.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:59.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 10 monclient: dump: 2024-01-24T23:23:59.290 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:59.290 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:59.290 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:59.290 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:59.290 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:59.290 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:59.290 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:59.290 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:59.290 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:59.290 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:59.290 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:59.290 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:59.290 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:59.290 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:59.290 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:59.291 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:59.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3ab3c63640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:59.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3ab3c63640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:59.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3ab3c63640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:59.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3ab3c63640 1 -- 172.21.15.132:0/1150201484 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a94005090 con 0x7f3aac1361f0 2024-01-24T23:23:59.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 1 -- 172.21.15.132:0/1150201484 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f3a9c00f9d0 con 0x7f3aac1361f0 2024-01-24T23:23:59.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3aa27fc640 1 -- 172.21.15.132:0/1150201484 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(71..71 src has 1..71) v4 ==== 5385+0+0 (unknown 892542886 0 0) 0x7f3a9c043e90 con 0x7f3aac1361f0 2024-01-24T23:23:59.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.275+0000 7f3ab19ff640 10 monclient: get_auth_request con 0x7f3a90037080 auth_method 0 2024-01-24T23:23:59.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.279+0000 7f3aa27fc640 1 -- 172.21.15.132:0/1150201484 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f3a9c010e80 con 0x7f3aac1361f0 2024-01-24T23:23:59.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.279+0000 7f3aa27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:59.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.279+0000 7f3aa27fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:59.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.439+0000 7f3ab3c63640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-01-24T23:23:59.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.439+0000 7f3ab3c63640 10 monclient: _send_command 2 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-01-24T23:23:59.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.439+0000 7f3ab3c63640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:59.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.439+0000 7f3ab3c63640 1 -- 172.21.15.132:0/1150201484 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]} v 0) v1 -- 0x7f3a940058c0 con 0x7f3aac1361f0 2024-01-24T23:23:59.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.439+0000 7f3aa27fc640 1 -- 172.21.15.132:0/1150201484 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}]=0 no need to move item id -16 name 'host2' to location {root=bar} in crush map v71) v1 ==== 177+0+0 (unknown 750988058 0 0) 0x7f3a9c012f60 con 0x7f3aac1361f0 2024-01-24T23:23:59.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.439+0000 7f3aa27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-01-24T23:23:59.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.439+0000 7f3aa27fc640 10 monclient: _finish_command 2 = system:0 no need to move item id -16 name 'host2' to location {root=bar} in crush map 2024-01-24T23:23:59.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.535+0000 7f3ab3c63640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-01-24T23:23:59.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.535+0000 7f3ab3c63640 10 monclient: _send_command 3 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-01-24T23:23:59.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.535+0000 7f3ab3c63640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:23:59.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.535+0000 7f3ab3c63640 1 -- 172.21.15.132:0/1150201484 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]} v 0) v1 -- 0x7f3a94000bc0 con 0x7f3aac1361f0 2024-01-24T23:23:59.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.539+0000 7f3aa27fc640 1 -- 172.21.15.132:0/1150201484 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}]=0 no need to move item id -16 name 'host2' to location {root=bar} in crush map v71) v1 ==== 177+0+0 (unknown 750988058 0 0) 0x7f3a9c0146e0 con 0x7f3aac1361f0 2024-01-24T23:23:59.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.539+0000 7f3aa27fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-01-24T23:23:59.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.539+0000 7f3aa27fc640 10 monclient: _finish_command 3 = system:0 no need to move item id -16 name 'host2' to location {root=bar} in crush map 2024-01-24T23:23:59.540 INFO:tasks.workunit.client.0.smithi132.stderr:no need to move item id -16 name 'host2' to location {root=bar} in crush map 2024-01-24T23:23:59.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.539+0000 7f3ab3c63640 1 -- 172.21.15.132:0/1150201484 >> v1:172.21.15.132:6812/1835401922 conn(0x7f3a90037080 legacy=0x7f3a90039540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:59.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.539+0000 7f3ab3c63640 10 monclient: shutdown 2024-01-24T23:23:59.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.539+0000 7f3ab3c63640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:59.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.539+0000 7f3ab3c63640 1 -- 172.21.15.132:0/1150201484 >> v1:172.21.15.132:6791/0 conn(0x7f3aac1361f0 legacy=0x7f3aac1d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:59.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.539+0000 7f3ab3c63640 1 -- 172.21.15.132:0/1150201484 shutdown_connections 2024-01-24T23:23:59.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.539+0000 7f3ab3c63640 1 -- 172.21.15.132:0/1150201484 >> 172.21.15.132:0/1150201484 conn(0x7f3aac0a6790 msgr2=0x7f3aac06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:59.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.539+0000 7f3ab3c63640 1 -- 172.21.15.132:0/1150201484 shutdown_connections 2024-01-24T23:23:59.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.539+0000 7f3ab3c63640 1 -- 172.21.15.132:0/1150201484 wait complete. 2024-01-24T23:23:59.556 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd tree 2024-01-24T23:23:59.557 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep -c osd.7 2024-01-24T23:23:59.557 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep -q 2 2024-01-24T23:23:59.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.663+0000 7fc9b2fce640 10 monclient: get_monmap_and_config 2024-01-24T23:23:59.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.663+0000 7fc9b2fce640 10 monclient: build_initial_monmap 2024-01-24T23:23:59.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.663+0000 7fc9b2fce640 10 monclient: monmap: 2024-01-24T23:23:59.667 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:59.667 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:59.667 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:23:59.668524+0000 2024-01-24T23:23:59.667 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:23:59.668524+0000 2024-01-24T23:23:59.667 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:59.668 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:59.668 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:59.668 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:59.668 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:59.668 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:59.668 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:59.668 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:59.668 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:59.668 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:59.668 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:59.668 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:59.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 1 Processor -- start 2024-01-24T23:23:59.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 1 -- start start 2024-01-24T23:23:59.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 10 monclient: init 2024-01-24T23:23:59.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:59.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 10 monclient: _add_conns ranks=[5,8,0,2,4,6,3,1,7] 2024-01-24T23:23:59.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 10 monclient(hunting): picked mon.noname-h con 0x7fc9ac136cb0 addr v1:172.21.15.132:6791/0 2024-01-24T23:23:59.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 10 monclient(hunting): picked mon.noname-e con 0x7fc9ac1377a0 addr v1:172.21.15.115:6793/0 2024-01-24T23:23:59.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 10 monclient(hunting): picked mon.noname-a con 0x7fc9ac09bcc0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:59.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ac064700 con 0x7fc9ac09bcc0 2024-01-24T23:23:59.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ac09b650 con 0x7fc9ac136cb0 2024-01-24T23:23:59.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ac09c110 con 0x7fc9ac1377a0 2024-01-24T23:23:59.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:59.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 10 monclient(hunting): authenticate will time out at 1126.993318s 2024-01-24T23:23:59.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b0d6a640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fc9ac136cb0 0x7fc9ac137090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:45896/0 (socket says 172.21.15.132:45896) 2024-01-24T23:23:59.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b0d6a640 1 -- 172.21.15.132:0/679103816 learned_addr learned my addr 172.21.15.132:0/679103816 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:59.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b156b640 1 -- 172.21.15.132:0/679103816 >> v1:172.21.15.115:6789/0 conn(0x7fc9ac09bcc0 legacy=0x7fc9ac13f910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:59.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 1 -- 172.21.15.132:0/679103816 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3985215301 0 0) 0x7fc9ac09b650 con 0x7fc9ac136cb0 2024-01-24T23:23:59.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:23:59.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:59.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:59.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 10 monclient(hunting): my global_id is 5315 2024-01-24T23:23:59.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 1 -- 172.21.15.132:0/679103816 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc994003530 con 0x7fc9ac136cb0 2024-01-24T23:23:59.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 1 -- 172.21.15.132:0/679103816 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3894924837 0 0) 0x7fc994003530 con 0x7fc9ac136cb0 2024-01-24T23:23:59.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 1 -- 172.21.15.132:0/679103816 >> v1:172.21.15.115:6793/0 conn(0x7fc9ac1377a0 legacy=0x7fc9ac13c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:59.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 1 -- 172.21.15.132:0/679103816 >> v1:172.21.15.115:6789/0 conn(0x7fc9ac09bcc0 legacy=0x7fc9ac13f910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:23:59.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:23:59.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 1 monclient: found mon.noname-h 2024-01-24T23:23:59.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:59.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:23:59.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 1 -- 172.21.15.132:0/679103816 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac09b000 con 0x7fc9ac136cb0 2024-01-24T23:23:59.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 10 monclient: _finish_auth 0 2024-01-24T23:23:59.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:23:59.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:59.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 1 -- 172.21.15.132:0/679103816 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc99c0017c0 con 0x7fc9ac136cb0 2024-01-24T23:23:59.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 5 monclient: authenticate success, global_id 5315 2024-01-24T23:23:59.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:59.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:59.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:23:59.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 10 monclient: dump: 2024-01-24T23:23:59.676 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:59.676 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:59.676 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:59.677 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:59.677 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:59.677 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:59.677 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:59.677 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:59.677 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:59.677 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:59.677 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:59.677 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:59.677 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:59.677 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:59.677 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:59.677 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:59.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:23:59.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 1 -- 172.21.15.132:0/679103816 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc99c0019e0 con 0x7fc9ac136cb0 2024-01-24T23:23:59.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:59.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 1 -- 172.21.15.132:0/679103816 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc99c004bf0 con 0x7fc9ac136cb0 2024-01-24T23:23:59.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 10 monclient: get_monmap_and_config success 2024-01-24T23:23:59.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 4 set_mon_vals no callback set 2024-01-24T23:23:59.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:59.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:23:59.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9ab7fe640 10 monclient: dump: 2024-01-24T23:23:59.678 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:59.678 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:59.678 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:59.678 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:59.678 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:59.678 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:59.678 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:59.678 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:59.678 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:59.679 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:59.679 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:59.679 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:59.679 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:59.679 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:59.679 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:59.679 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:59.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 10 monclient: shutdown 2024-01-24T23:23:59.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:59.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 1 -- 172.21.15.132:0/679103816 >> v1:172.21.15.132:6791/0 conn(0x7fc9ac136cb0 legacy=0x7fc9ac137090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:59.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 1 -- 172.21.15.132:0/679103816 shutdown_connections 2024-01-24T23:23:59.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 1 -- 172.21.15.132:0/679103816 >> 172.21.15.132:0/679103816 conn(0x7fc9ac0a9c10 msgr2=0x7fc9ac0a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:59.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 1 -- 172.21.15.132:0/679103816 shutdown_connections 2024-01-24T23:23:59.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 1 -- 172.21.15.132:0/679103816 wait complete. 2024-01-24T23:23:59.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 10 monclient: build_initial_monmap 2024-01-24T23:23:59.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 10 monclient: monmap: 2024-01-24T23:23:59.679 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:23:59.680 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:23:59.680 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:23:59.680 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:23:59.680 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:23:59.680 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:23:59.680 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:23:59.680 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:23:59.680 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:23:59.680 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:23:59.680 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:23:59.680 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:23:59.680 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:23:59.680 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:23:59.680 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:23:59.680 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:59.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 1 Processor -- start 2024-01-24T23:23:59.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 1 -- start start 2024-01-24T23:23:59.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.667+0000 7fc9b2fce640 10 monclient: init 2024-01-24T23:23:59.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9b2fce640 10 monclient: _reopen_session rank -1 2024-01-24T23:23:59.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9b2fce640 10 monclient: _add_conns ranks=[0,2,1,6,8,7,5,4,3] 2024-01-24T23:23:59.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9b2fce640 10 monclient(hunting): picked mon.noname-a con 0x7fc9ac09bcc0 addr v1:172.21.15.115:6789/0 2024-01-24T23:23:59.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9b2fce640 10 monclient(hunting): picked mon.noname-b con 0x7fc9ac136cb0 addr v1:172.21.15.115:6790/0 2024-01-24T23:23:59.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9b2fce640 10 monclient(hunting): picked mon.noname-f con 0x7fc9ac1377a0 addr v1:172.21.15.132:6789/0 2024-01-24T23:23:59.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9b2fce640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ac09c6a0 con 0x7fc9ac09bcc0 2024-01-24T23:23:59.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9b2fce640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ac137fb0 con 0x7fc9ac1377a0 2024-01-24T23:23:59.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9b2fce640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ac123130 con 0x7fc9ac136cb0 2024-01-24T23:23:59.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9b2fce640 10 monclient(hunting): _renew_subs 2024-01-24T23:23:59.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9b2fce640 10 monclient(hunting): authenticate will time out at 1126.996139s 2024-01-24T23:23:59.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9b156b640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fc9ac1377a0 legacy=0x7fc9ac1d3670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:23:59.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9b0d6a640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fc9ac09bcc0 legacy=0x7fc9ac135ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:23:59.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9abfff640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fc9ac136cb0 0x7fc9ac1361c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:42306/0 (socket says 172.21.15.132:42306) 2024-01-24T23:23:59.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9abfff640 1 -- 172.21.15.132:0/2392472986 learned_addr learned my addr 172.21.15.132:0/2392472986 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:23:59.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 1 -- 172.21.15.132:0/2392472986 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3230863407 0 0) 0x7fc9ac123130 con 0x7fc9ac136cb0 2024-01-24T23:23:59.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 10 monclient(hunting): _negotiate 2024-01-24T23:23:59.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:23:59.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:23:59.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 10 monclient(hunting): my global_id is 84647 2024-01-24T23:23:59.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 1 -- 172.21.15.132:0/2392472986 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc998003530 con 0x7fc9ac136cb0 2024-01-24T23:23:59.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 1 -- 172.21.15.132:0/2392472986 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3022214631 0 0) 0x7fc998003530 con 0x7fc9ac136cb0 2024-01-24T23:23:59.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 1 -- 172.21.15.132:0/2392472986 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc9ac123130 con 0x7fc9ac136cb0 2024-01-24T23:23:59.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 1 -- 172.21.15.132:0/2392472986 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc9a0001a80 con 0x7fc9ac136cb0 2024-01-24T23:23:59.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:59.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:23:59.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 10 monclient(hunting): dump: 2024-01-24T23:23:59.683 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:59.683 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:59.683 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:59.683 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:59.683 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:59.683 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:59.683 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:59.683 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:59.683 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:59.683 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:59.683 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:59.683 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:59.683 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:59.683 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:59.683 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:59.683 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:59.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:23:59.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 1 -- 172.21.15.132:0/2392472986 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 972783735 0 0) 0x7fc9ac123130 con 0x7fc9ac136cb0 2024-01-24T23:23:59.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 1 -- 172.21.15.132:0/2392472986 >> v1:172.21.15.132:6789/0 conn(0x7fc9ac1377a0 legacy=0x7fc9ac1d3670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:59.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 1 -- 172.21.15.132:0/2392472986 >> v1:172.21.15.115:6789/0 conn(0x7fc9ac09bcc0 legacy=0x7fc9ac135ab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:23:59.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 10 monclient: _finish_hunting 0 2024-01-24T23:23:59.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 1 monclient: found mon.b 2024-01-24T23:23:59.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:23:59.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:59.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 1 -- 172.21.15.132:0/2392472986 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac123390 con 0x7fc9ac136cb0 2024-01-24T23:23:59.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 10 monclient: _finish_auth 0 2024-01-24T23:23:59.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 10 monclient: _check_auth_tickets 2024-01-24T23:23:59.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:23:59.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9b2fce640 5 monclient: authenticate success, global_id 84647 2024-01-24T23:23:59.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9b2fce640 10 monclient: _renew_subs 2024-01-24T23:23:59.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9b2fce640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:59.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9b2fce640 1 -- 172.21.15.132:0/2392472986 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ac1251b0 con 0x7fc9ac136cb0 2024-01-24T23:23:59.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9b2fce640 10 monclient: _renew_subs 2024-01-24T23:23:59.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9b2fce640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:59.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9b2fce640 1 -- 172.21.15.132:0/2392472986 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ac1cb5c0 con 0x7fc9ac136cb0 2024-01-24T23:23:59.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 1 -- 172.21.15.132:0/2392472986 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc9a00038d0 con 0x7fc9ac136cb0 2024-01-24T23:23:59.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:23:59.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 1 -- 172.21.15.132:0/2392472986 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc9a0003e00 con 0x7fc9ac136cb0 2024-01-24T23:23:59.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:23:59.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9aaffd640 4 set_mon_vals no callback set 2024-01-24T23:23:59.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:23:59.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 10 monclient: dump: 2024-01-24T23:23:59.685 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:23:59.685 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:23:59.685 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:59.685 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:23:59.686 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:23:59.686 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:23:59.686 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:23:59.686 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:23:59.686 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:23:59.686 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:23:59.686 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:23:59.686 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:23:59.686 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:23:59.686 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:23:59.686 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:23:59.686 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:23:59.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9b2fce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:59.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9b2fce640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:59.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9b2fce640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:59.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9b2fce640 1 -- 172.21.15.132:0/2392472986 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc98c005090 con 0x7fc9ac136cb0 2024-01-24T23:23:59.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 1 -- 172.21.15.132:0/2392472986 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fc9a0003fe0 con 0x7fc9ac136cb0 2024-01-24T23:23:59.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9a97fa640 1 -- 172.21.15.132:0/2392472986 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(71..71 src has 1..71) v4 ==== 5385+0+0 (unknown 892542886 0 0) 0x7fc9a0043e50 con 0x7fc9ac136cb0 2024-01-24T23:23:59.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.671+0000 7fc9b0d6a640 10 monclient: get_auth_request con 0x7fc998036e50 auth_method 0 2024-01-24T23:23:59.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.683+0000 7fc9a97fa640 1 -- 172.21.15.132:0/2392472986 <== mon.2 v1:172.21.15.115:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fc9a0014820 con 0x7fc9ac136cb0 2024-01-24T23:23:59.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.683+0000 7fc9a97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:23:59.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.683+0000 7fc9a97fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:23:59.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.827+0000 7fc9b2fce640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:23:59.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.827+0000 7fc9b2fce640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-01-24T23:23:59.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.827+0000 7fc9b2fce640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:59.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.827+0000 7fc9b2fce640 1 -- 172.21.15.132:0/2392472986 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7fc98c005b50 con 0x7fc9ac136cb0 2024-01-24T23:23:59.831 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.831+0000 7fc9a97fa640 1 -- 172.21.15.132:0/2392472986 <== mon.2 v1:172.21.15.115:6790/0 10 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v71) v1 ==== 56+0+1474 (unknown 1479012056 0 4123843803) 0x7fc9a0014640 con 0x7fc9ac136cb0 2024-01-24T23:23:59.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.831+0000 7fc9a97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-01-24T23:23:59.832 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.831+0000 7fc9a97fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:23:59.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.923+0000 7fc9b2fce640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:23:59.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.923+0000 7fc9b2fce640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-01-24T23:23:59.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.923+0000 7fc9b2fce640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:23:59.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.923+0000 7fc9b2fce640 1 -- 172.21.15.132:0/2392472986 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7fc98c005ce0 con 0x7fc9ac136cb0 2024-01-24T23:23:59.926 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.923+0000 7fc9a97fa640 1 -- 172.21.15.132:0/2392472986 <== mon.2 v1:172.21.15.115:6790/0 11 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v71) v1 ==== 56+0+1474 (unknown 1479012056 0 4123843803) 0x7fc9a0015380 con 0x7fc9ac136cb0 2024-01-24T23:23:59.927 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.923+0000 7fc9a97fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-01-24T23:23:59.927 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.923+0000 7fc9a97fa640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:23:59.929 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.927+0000 7fc9b2fce640 1 -- 172.21.15.132:0/2392472986 >> v1:172.21.15.132:6812/1835401922 conn(0x7fc998036e50 legacy=0x7fc998039310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:59.929 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.927+0000 7fc9b2fce640 10 monclient: shutdown 2024-01-24T23:23:59.929 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.927+0000 7fc9b2fce640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:23:59.929 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.927+0000 7fc9b2fce640 1 -- 172.21.15.132:0/2392472986 >> v1:172.21.15.115:6790/0 conn(0x7fc9ac136cb0 legacy=0x7fc9ac1361c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:23:59.929 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.927+0000 7fc9b2fce640 1 -- 172.21.15.132:0/2392472986 shutdown_connections 2024-01-24T23:23:59.929 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.927+0000 7fc9b2fce640 1 -- 172.21.15.132:0/2392472986 >> 172.21.15.132:0/2392472986 conn(0x7fc9ac0a9c10 msgr2=0x7fc9ac0a45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:23:59.929 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.927+0000 7fc9b2fce640 1 -- 172.21.15.132:0/2392472986 shutdown_connections 2024-01-24T23:23:59.929 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:23:59.927+0000 7fc9b2fce640 1 -- 172.21.15.132:0/2392472986 wait complete. 2024-01-24T23:23:59.944 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd tree 2024-01-24T23:23:59.944 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep -c host1 2024-01-24T23:23:59.944 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep -q 2 2024-01-24T23:24:00.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.055+0000 7f629a51e640 10 monclient: get_monmap_and_config 2024-01-24T23:24:00.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.055+0000 7f629a51e640 10 monclient: build_initial_monmap 2024-01-24T23:24:00.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.055+0000 7f629a51e640 10 monclient: monmap: 2024-01-24T23:24:00.057 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:00.057 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:00.057 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:00.058218+0000 2024-01-24T23:24:00.057 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:00.058218+0000 2024-01-24T23:24:00.057 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:00.057 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:00.057 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:00.057 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:00.057 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:00.057 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:00.057 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:00.058 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:00.058 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:00.058 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:00.058 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:00.058 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:00.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.055+0000 7f629a51e640 1 Processor -- start 2024-01-24T23:24:00.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.055+0000 7f629a51e640 1 -- start start 2024-01-24T23:24:00.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.055+0000 7f629a51e640 10 monclient: init 2024-01-24T23:24:00.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.055+0000 7f629a51e640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:00.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.055+0000 7f629a51e640 10 monclient: _add_conns ranks=[3,2,4,6,7,1,8,5,0] 2024-01-24T23:24:00.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.055+0000 7f629a51e640 10 monclient(hunting): picked mon.noname-g con 0x7f62940a0e20 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:00.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.055+0000 7f629a51e640 10 monclient(hunting): picked mon.noname-b con 0x7f629409f5e0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:00.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.055+0000 7f629a51e640 10 monclient(hunting): picked mon.noname-c con 0x7f629413d560 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:00.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.055+0000 7f629a51e640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6294064700 con 0x7f629409f5e0 2024-01-24T23:24:00.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.055+0000 7f629a51e640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629406d3f0 con 0x7f62940a0e20 2024-01-24T23:24:00.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.055+0000 7f629a51e640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629409fa00 con 0x7f629413d560 2024-01-24T23:24:00.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.055+0000 7f629a51e640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:00.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.055+0000 7f629a51e640 10 monclient(hunting): authenticate will time out at 1127.382597s 2024-01-24T23:24:00.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.055+0000 7f6298abb640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f629413d560 legacy=0x7f629413f950 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:00.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.055+0000 7f6293fff640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f62940a0e20 0x7f62940a1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:57010/0 (socket says 172.21.15.132:57010) 2024-01-24T23:24:00.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.055+0000 7f6293fff640 1 -- 172.21.15.132:0/2846913921 learned_addr learned my addr 172.21.15.132:0/2846913921 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:00.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 1 -- 172.21.15.132:0/2846913921 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2453018303 0 0) 0x7f629406d3f0 con 0x7f62940a0e20 2024-01-24T23:24:00.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 10 monclient(hunting): _negotiate 2024-01-24T23:24:00.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:00.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:00.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 10 monclient(hunting): my global_id is 5358 2024-01-24T23:24:00.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 1 -- 172.21.15.132:0/2846913921 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6280003530 con 0x7f62940a0e20 2024-01-24T23:24:00.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 1 -- 172.21.15.132:0/2846913921 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684523530 0 0) 0x7f6294064700 con 0x7f629409f5e0 2024-01-24T23:24:00.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 10 monclient(hunting): _negotiate 2024-01-24T23:24:00.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:00.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:00.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 10 monclient(hunting): my global_id is 84656 2024-01-24T23:24:00.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 1 -- 172.21.15.132:0/2846913921 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f629406d3f0 con 0x7f629409f5e0 2024-01-24T23:24:00.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 1 -- 172.21.15.132:0/2846913921 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 421730603 0 0) 0x7f6280003530 con 0x7f62940a0e20 2024-01-24T23:24:00.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 1 -- 172.21.15.132:0/2846913921 >> v1:172.21.15.115:6791/0 conn(0x7f629413d560 legacy=0x7f629413f950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:00.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 1 -- 172.21.15.132:0/2846913921 >> v1:172.21.15.115:6790/0 conn(0x7f629409f5e0 legacy=0x7f629413be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:00.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 10 monclient: _finish_hunting 0 2024-01-24T23:24:00.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 1 monclient: found mon.noname-g 2024-01-24T23:24:00.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:00.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:24:00.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 1 -- 172.21.15.132:0/2846913921 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629406bc90 con 0x7f62940a0e20 2024-01-24T23:24:00.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 10 monclient: _finish_auth 0 2024-01-24T23:24:00.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 10 monclient: _check_auth_tickets 2024-01-24T23:24:00.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:00.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 1 -- 172.21.15.132:0/2846913921 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f62840017c0 con 0x7f62940a0e20 2024-01-24T23:24:00.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 5 monclient: authenticate success, global_id 5358 2024-01-24T23:24:00.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:00.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:00.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:00.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 10 monclient: dump: 2024-01-24T23:24:00.068 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:00.068 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:00.068 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.068 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.068 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:00.068 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:00.068 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:00.069 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:00.069 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:00.069 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:00.069 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:00.069 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:00.069 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:00.069 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:00.069 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:00.069 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:00.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:00.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 1 -- 172.21.15.132:0/2846913921 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6284001a30 con 0x7f62940a0e20 2024-01-24T23:24:00.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:00.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:00.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 4 set_mon_vals no callback set 2024-01-24T23:24:00.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 1 -- 172.21.15.132:0/2846913921 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6284004bf0 con 0x7f62940a0e20 2024-01-24T23:24:00.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:00.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:00.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62937fe640 10 monclient: dump: 2024-01-24T23:24:00.070 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:00.070 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:00.070 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.070 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.070 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:00.070 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:00.070 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:00.070 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:00.070 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:00.070 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:00.070 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:00.070 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:00.070 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:00.070 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:00.070 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:00.070 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 10 monclient: shutdown 2024-01-24T23:24:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 1 -- 172.21.15.132:0/2846913921 >> v1:172.21.15.132:6790/0 conn(0x7f62940a0e20 legacy=0x7f62940a1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 1 -- 172.21.15.132:0/2846913921 shutdown_connections 2024-01-24T23:24:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 1 -- 172.21.15.132:0/2846913921 >> 172.21.15.132:0/2846913921 conn(0x7f62940a7a30 msgr2=0x7f62940a3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 1 -- 172.21.15.132:0/2846913921 shutdown_connections 2024-01-24T23:24:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 1 -- 172.21.15.132:0/2846913921 wait complete. 2024-01-24T23:24:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 10 monclient: build_initial_monmap 2024-01-24T23:24:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 10 monclient: monmap: 2024-01-24T23:24:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:00.072 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:00.072 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:00.072 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:00.072 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:00.072 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:00.072 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:00.072 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:00.072 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:00.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 1 Processor -- start 2024-01-24T23:24:00.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 1 -- start start 2024-01-24T23:24:00.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 10 monclient: init 2024-01-24T23:24:00.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:00.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 10 monclient: _add_conns ranks=[1,6,2,4,0,3,7,5,8] 2024-01-24T23:24:00.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 10 monclient(hunting): picked mon.noname-f con 0x7f629409f5e0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:00.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 10 monclient(hunting): picked mon.noname-d con 0x7f62940a0e20 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:00.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 10 monclient(hunting): picked mon.noname-b con 0x7f629413d560 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:00.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6294072780 con 0x7f629409f5e0 2024-01-24T23:24:00.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62941392e0 con 0x7f629413d560 2024-01-24T23:24:00.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f6293fff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f629409f5e0 legacy=0x7f62940708b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:00.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629413b5d0 con 0x7f62940a0e20 2024-01-24T23:24:00.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:00.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f629a51e640 10 monclient(hunting): authenticate will time out at 1127.387153s 2024-01-24T23:24:00.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f628b7fe640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f62940a0e20 0x7f6294070fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:55214/0 (socket says 172.21.15.132:55214) 2024-01-24T23:24:00.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f628b7fe640 1 -- 172.21.15.132:0/2158432475 learned_addr learned my addr 172.21.15.132:0/2158432475 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:00.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62917fa640 1 -- 172.21.15.132:0/2158432475 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3457269650 0 0) 0x7f62941392e0 con 0x7f629413d560 2024-01-24T23:24:00.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62917fa640 10 monclient(hunting): _negotiate 2024-01-24T23:24:00.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62917fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:00.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62917fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:00.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62917fa640 10 monclient(hunting): my global_id is 84665 2024-01-24T23:24:00.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62917fa640 1 -- 172.21.15.132:0/2158432475 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6274003530 con 0x7f629413d560 2024-01-24T23:24:00.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62917fa640 1 -- 172.21.15.132:0/2158432475 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773546155 0 0) 0x7f629413b5d0 con 0x7f62940a0e20 2024-01-24T23:24:00.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62917fa640 10 monclient(hunting): _negotiate 2024-01-24T23:24:00.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62917fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:00.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62917fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:00.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62917fa640 10 monclient(hunting): my global_id is 5379 2024-01-24T23:24:00.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.059+0000 7f62917fa640 1 -- 172.21.15.132:0/2158432475 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62941392e0 con 0x7f62940a0e20 2024-01-24T23:24:00.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 1 -- 172.21.15.132:0/2158432475 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1966677606 0 0) 0x7f6274003530 con 0x7f629413d560 2024-01-24T23:24:00.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 1 -- 172.21.15.132:0/2158432475 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f629413b5d0 con 0x7f629413d560 2024-01-24T23:24:00.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 1 -- 172.21.15.132:0/2158432475 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f628c004530 con 0x7f629413d560 2024-01-24T23:24:00.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:00.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:24:00.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 10 monclient(hunting): dump: 2024-01-24T23:24:00.074 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:00.074 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:00.075 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.075 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.075 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:00.075 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:00.075 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:00.075 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:00.075 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:00.075 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:00.075 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:00.075 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:00.075 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:00.075 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:00.075 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:00.075 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:00.075 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:00.075 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 1 -- 172.21.15.132:0/2158432475 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2193801339 0 0) 0x7f62941392e0 con 0x7f62940a0e20 2024-01-24T23:24:00.075 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 1 -- 172.21.15.132:0/2158432475 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6274003530 con 0x7f62940a0e20 2024-01-24T23:24:00.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 1 -- 172.21.15.132:0/2158432475 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f627c001a40 con 0x7f62940a0e20 2024-01-24T23:24:00.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:00.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:24:00.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 10 monclient(hunting): dump: 2024-01-24T23:24:00.076 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:00.076 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:00.076 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.076 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.076 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:00.076 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:00.076 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:00.076 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:00.076 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:00.076 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:00.076 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:00.076 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:00.077 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:00.077 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:00.077 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:00.077 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:00.077 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 1 -- 172.21.15.132:0/2158432475 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2509193471 0 0) 0x7f629413b5d0 con 0x7f629413d560 2024-01-24T23:24:00.077 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 1 -- 172.21.15.132:0/2158432475 >> v1:172.21.15.115:6792/0 conn(0x7f62940a0e20 legacy=0x7f6294070fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:00.077 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 1 -- 172.21.15.132:0/2158432475 >> v1:172.21.15.132:6789/0 conn(0x7f629409f5e0 legacy=0x7f62940708b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:00.077 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 10 monclient: _finish_hunting 0 2024-01-24T23:24:00.077 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 1 monclient: found mon.b 2024-01-24T23:24:00.077 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:00.077 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:00.077 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 1 -- 172.21.15.132:0/2158432475 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629409e170 con 0x7f629413d560 2024-01-24T23:24:00.077 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 10 monclient: _finish_auth 0 2024-01-24T23:24:00.077 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 10 monclient: _check_auth_tickets 2024-01-24T23:24:00.077 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:00.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f629a51e640 5 monclient: authenticate success, global_id 84665 2024-01-24T23:24:00.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f629a51e640 10 monclient: _renew_subs 2024-01-24T23:24:00.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f629a51e640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:00.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f629a51e640 1 -- 172.21.15.132:0/2158432475 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f629409dc20 con 0x7f629413d560 2024-01-24T23:24:00.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f629a51e640 10 monclient: _renew_subs 2024-01-24T23:24:00.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f629a51e640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:00.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f629a51e640 1 -- 172.21.15.132:0/2158432475 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f629413b2f0 con 0x7f629413d560 2024-01-24T23:24:00.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 1 -- 172.21.15.132:0/2158432475 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f628c004810 con 0x7f629413d560 2024-01-24T23:24:00.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:00.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 1 -- 172.21.15.132:0/2158432475 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f628c004d40 con 0x7f629413d560 2024-01-24T23:24:00.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:00.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f6292ffd640 4 set_mon_vals no callback set 2024-01-24T23:24:00.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:24:00.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 10 monclient: dump: 2024-01-24T23:24:00.078 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:00.078 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:00.079 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.079 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.079 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:00.079 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:00.079 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:00.079 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:00.079 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:00.079 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:00.079 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:00.079 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:00.079 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:00.079 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:00.079 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:00.079 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:00.079 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f629a51e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:00.079 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f629a51e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:00.079 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f629a51e640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:00.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f629a51e640 1 -- 172.21.15.132:0/2158432475 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6278005090 con 0x7f629413d560 2024-01-24T23:24:00.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 1 -- 172.21.15.132:0/2158432475 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f628c004ea0 con 0x7f629413d560 2024-01-24T23:24:00.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f62917fa640 1 -- 172.21.15.132:0/2158432475 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(71..71 src has 1..71) v4 ==== 5385+0+0 (unknown 892542886 0 0) 0x7f628c043eb0 con 0x7f629413d560 2024-01-24T23:24:00.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.063+0000 7f6293fff640 10 monclient: get_auth_request con 0x7f62740376f0 auth_method 0 2024-01-24T23:24:00.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.075+0000 7f62917fa640 1 -- 172.21.15.132:0/2158432475 <== mon.2 v1:172.21.15.115:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f628c06f310 con 0x7f629413d560 2024-01-24T23:24:00.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.075+0000 7f62917fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:00.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.075+0000 7f62917fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:00.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.231+0000 7f629a51e640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:00.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.231+0000 7f629a51e640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:00.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.231+0000 7f629a51e640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:00.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.231+0000 7f629a51e640 1 -- 172.21.15.132:0/2158432475 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f6278005b50 con 0x7f629413d560 2024-01-24T23:24:00.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.235+0000 7f62917fa640 1 -- 172.21.15.132:0/2158432475 <== mon.2 v1:172.21.15.115:6790/0 10 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v71) v1 ==== 56+0+1474 (unknown 1479012056 0 4123843803) 0x7f628c014ad0 con 0x7f629413d560 2024-01-24T23:24:00.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.235+0000 7f62917fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:00.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.235+0000 7f62917fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:24:00.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.323+0000 7f629a51e640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:00.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.323+0000 7f629a51e640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:00.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.323+0000 7f629a51e640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:00.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.323+0000 7f629a51e640 1 -- 172.21.15.132:0/2158432475 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f6278005ce0 con 0x7f629413d560 2024-01-24T23:24:00.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.327+0000 7f62917fa640 1 -- 172.21.15.132:0/2158432475 <== mon.2 v1:172.21.15.115:6790/0 11 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v71) v1 ==== 56+0+1474 (unknown 1479012056 0 4123843803) 0x7f628c0146e0 con 0x7f629413d560 2024-01-24T23:24:00.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.327+0000 7f62917fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:00.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.327+0000 7f62917fa640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:24:00.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.327+0000 7f629a51e640 1 -- 172.21.15.132:0/2158432475 >> v1:172.21.15.132:6812/1835401922 conn(0x7f62740376f0 legacy=0x7f6274039bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:00.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.327+0000 7f629a51e640 10 monclient: shutdown 2024-01-24T23:24:00.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.327+0000 7f629a51e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:00.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.327+0000 7f629a51e640 1 -- 172.21.15.132:0/2158432475 >> v1:172.21.15.115:6790/0 conn(0x7f629413d560 legacy=0x7f62941d3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:00.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.327+0000 7f629a51e640 1 -- 172.21.15.132:0/2158432475 shutdown_connections 2024-01-24T23:24:00.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.327+0000 7f629a51e640 1 -- 172.21.15.132:0/2158432475 >> 172.21.15.132:0/2158432475 conn(0x7f62940a7a30 msgr2=0x7f6294138140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:00.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.327+0000 7f629a51e640 1 -- 172.21.15.132:0/2158432475 shutdown_connections 2024-01-24T23:24:00.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.327+0000 7f629a51e640 1 -- 172.21.15.132:0/2158432475 wait complete. 2024-01-24T23:24:00.345 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd tree 2024-01-24T23:24:00.345 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep -c osd.9 2024-01-24T23:24:00.345 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep -q 2 2024-01-24T23:24:00.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f8369a58640 10 monclient: get_monmap_and_config 2024-01-24T23:24:00.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f8369a58640 10 monclient: build_initial_monmap 2024-01-24T23:24:00.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f8369a58640 10 monclient: monmap: 2024-01-24T23:24:00.457 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:00.457 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:00.458 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:00.458766+0000 2024-01-24T23:24:00.458 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:00.458766+0000 2024-01-24T23:24:00.458 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:00.458 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:00.458 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:00.458 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:00.458 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:00.458 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:00.458 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:00.458 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:00.458 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:00.458 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:00.458 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:00.458 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:00.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f8369a58640 1 Processor -- start 2024-01-24T23:24:00.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f8369a58640 1 -- start start 2024-01-24T23:24:00.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f8369a58640 10 monclient: init 2024-01-24T23:24:00.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f8369a58640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:00.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f8369a58640 10 monclient: _add_conns ranks=[5,1,6,3,7,2,4,8,0] 2024-01-24T23:24:00.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f8369a58640 10 monclient(hunting): picked mon.noname-h con 0x7f836409ff20 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:00.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f8369a58640 10 monclient(hunting): picked mon.noname-f con 0x7f836409f9c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:00.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f8369a58640 10 monclient(hunting): picked mon.noname-d con 0x7f836413b2c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:00.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f8369a58640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8364064700 con 0x7f836409f9c0 2024-01-24T23:24:00.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f8369a58640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f836409e760 con 0x7f836409ff20 2024-01-24T23:24:00.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f8369a58640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83640a1160 con 0x7f836413b2c0 2024-01-24T23:24:00.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f8369a58640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:00.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f8369a58640 10 monclient(hunting): authenticate will time out at 1127.783331s 2024-01-24T23:24:00.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f83627fc640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f836409f9c0 legacy=0x7f8364139ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:00.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f8362ffd640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f836409ff20 0x7f83640a0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:45902/0 (socket says 172.21.15.132:45902) 2024-01-24T23:24:00.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f8362ffd640 1 -- 172.21.15.132:0/3854847060 learned_addr learned my addr 172.21.15.132:0/3854847060 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:00.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f8361ffb640 1 -- 172.21.15.132:0/3854847060 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 867351725 0 0) 0x7f836409e760 con 0x7f836409ff20 2024-01-24T23:24:00.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f8361ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:00.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f8361ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:00.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f8361ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:00.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f8361ffb640 10 monclient(hunting): my global_id is 5324 2024-01-24T23:24:00.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.455+0000 7f8361ffb640 1 -- 172.21.15.132:0/3854847060 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f834c003530 con 0x7f836409ff20 2024-01-24T23:24:00.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8361ffb640 1 -- 172.21.15.132:0/3854847060 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 331366080 0 0) 0x7f834c003530 con 0x7f836409ff20 2024-01-24T23:24:00.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8361ffb640 1 -- 172.21.15.132:0/3854847060 >> v1:172.21.15.115:6792/0 conn(0x7f836413b2c0 legacy=0x7f836413d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:00.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8361ffb640 1 -- 172.21.15.132:0/3854847060 >> v1:172.21.15.132:6789/0 conn(0x7f836409f9c0 legacy=0x7f8364139ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:00.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8361ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:24:00.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8361ffb640 1 monclient: found mon.noname-h 2024-01-24T23:24:00.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8361ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:00.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8361ffb640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:24:00.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8361ffb640 1 -- 172.21.15.132:0/3854847060 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83640a0a10 con 0x7f836409ff20 2024-01-24T23:24:00.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8361ffb640 10 monclient: _finish_auth 0 2024-01-24T23:24:00.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8361ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:24:00.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8361ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:00.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8361ffb640 1 -- 172.21.15.132:0/3854847060 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8354001b40 con 0x7f836409ff20 2024-01-24T23:24:00.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 5 monclient: authenticate success, global_id 5324 2024-01-24T23:24:00.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:00.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8361ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:00.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8361ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:00.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8361ffb640 10 monclient: dump: 2024-01-24T23:24:00.467 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:00.467 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:00.467 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.467 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.467 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:00.467 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:00.467 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:00.467 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:00.467 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:00.467 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:00.467 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:00.468 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:00.468 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:00.468 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:00.468 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:00.468 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:00.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:00.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8361ffb640 1 -- 172.21.15.132:0/3854847060 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8354001e60 con 0x7f836409ff20 2024-01-24T23:24:00.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8361ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:00.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8361ffb640 1 -- 172.21.15.132:0/3854847060 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8354004f70 con 0x7f836409ff20 2024-01-24T23:24:00.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:00.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8361ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:00.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 4 set_mon_vals no callback set 2024-01-24T23:24:00.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8361ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:00.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8361ffb640 10 monclient: dump: 2024-01-24T23:24:00.468 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:00.468 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:00.468 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.469 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.469 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:00.469 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:00.469 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:00.469 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:00.469 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:00.469 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:00.469 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:00.469 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:00.469 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:00.469 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:00.469 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:00.469 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:00.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 10 monclient: shutdown 2024-01-24T23:24:00.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:00.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 1 -- 172.21.15.132:0/3854847060 >> v1:172.21.15.132:6791/0 conn(0x7f836409ff20 legacy=0x7f83640a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:00.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 1 -- 172.21.15.132:0/3854847060 shutdown_connections 2024-01-24T23:24:00.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 1 -- 172.21.15.132:0/3854847060 >> 172.21.15.132:0/3854847060 conn(0x7f83640a5a10 msgr2=0x7f83640a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:00.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 1 -- 172.21.15.132:0/3854847060 shutdown_connections 2024-01-24T23:24:00.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 1 -- 172.21.15.132:0/3854847060 wait complete. 2024-01-24T23:24:00.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 10 monclient: build_initial_monmap 2024-01-24T23:24:00.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 10 monclient: monmap: 2024-01-24T23:24:00.470 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:00.470 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:00.470 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:00.470 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:00.470 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:00.470 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:00.470 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:00.470 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:00.470 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:00.470 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:00.470 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:00.470 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:00.471 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:00.471 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:00.471 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:00.471 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:00.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 1 Processor -- start 2024-01-24T23:24:00.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 1 -- start start 2024-01-24T23:24:00.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 10 monclient: init 2024-01-24T23:24:00.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:00.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 10 monclient: _add_conns ranks=[0,1,5,2,8,3,4,6,7] 2024-01-24T23:24:00.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 10 monclient(hunting): picked mon.noname-a con 0x7f836409f9c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:00.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 10 monclient(hunting): picked mon.noname-f con 0x7f836409ff20 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:00.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 10 monclient(hunting): picked mon.noname-h con 0x7f836413b2c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:00.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8364071680 con 0x7f836409f9c0 2024-01-24T23:24:00.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8364120d20 con 0x7f836409ff20 2024-01-24T23:24:00.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f836409d2e0 con 0x7f836413b2c0 2024-01-24T23:24:00.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:00.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 10 monclient(hunting): authenticate will time out at 1127.786268s 2024-01-24T23:24:00.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f83627fc640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f836409ff20 legacy=0x7f836409cbd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:00.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8362ffd640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f836409f9c0 legacy=0x7f836409c4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:00.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f83637fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f836413b2c0 0x7f83641d38d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:45918/0 (socket says 172.21.15.132:45918) 2024-01-24T23:24:00.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f83637fe640 1 -- 172.21.15.132:0/965931178 learned_addr learned my addr 172.21.15.132:0/965931178 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:00.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 1 -- 172.21.15.132:0/965931178 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1937291342 0 0) 0x7f836409d2e0 con 0x7f836413b2c0 2024-01-24T23:24:00.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:00.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:00.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:00.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 10 monclient(hunting): my global_id is 5333 2024-01-24T23:24:00.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 1 -- 172.21.15.132:0/965931178 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8350003530 con 0x7f836413b2c0 2024-01-24T23:24:00.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 1 -- 172.21.15.132:0/965931178 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3398720657 0 0) 0x7f8350003530 con 0x7f836413b2c0 2024-01-24T23:24:00.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 1 -- 172.21.15.132:0/965931178 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f836409d2e0 con 0x7f836413b2c0 2024-01-24T23:24:00.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 1 -- 172.21.15.132:0/965931178 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f835c001a10 con 0x7f836413b2c0 2024-01-24T23:24:00.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:00.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:00.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 10 monclient(hunting): dump: 2024-01-24T23:24:00.473 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:00.473 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:00.473 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.473 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.473 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:00.473 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:00.473 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:00.473 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:00.473 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:00.473 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:00.473 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:00.473 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:00.473 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:00.473 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:00.473 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:00.473 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:00.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:00.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 1 -- 172.21.15.132:0/965931178 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2930310180 0 0) 0x7f836409d2e0 con 0x7f836413b2c0 2024-01-24T23:24:00.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 1 -- 172.21.15.132:0/965931178 >> v1:172.21.15.132:6789/0 conn(0x7f836409ff20 legacy=0x7f836409cbd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:00.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 1 -- 172.21.15.132:0/965931178 >> v1:172.21.15.115:6789/0 conn(0x7f836409f9c0 legacy=0x7f836409c4c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:00.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:24:00.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 1 monclient: found mon.h 2024-01-24T23:24:00.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:00.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:00.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 1 -- 172.21.15.132:0/965931178 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836409db80 con 0x7f836413b2c0 2024-01-24T23:24:00.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 10 monclient: _finish_auth 0 2024-01-24T23:24:00.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:24:00.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:00.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 5 monclient: authenticate success, global_id 5333 2024-01-24T23:24:00.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 10 monclient: _renew_subs 2024-01-24T23:24:00.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:00.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 1 -- 172.21.15.132:0/965931178 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8364071960 con 0x7f836413b2c0 2024-01-24T23:24:00.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 10 monclient: _renew_subs 2024-01-24T23:24:00.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:00.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 1 -- 172.21.15.132:0/965931178 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8364070580 con 0x7f836413b2c0 2024-01-24T23:24:00.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 1 -- 172.21.15.132:0/965931178 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f835c0038d0 con 0x7f836413b2c0 2024-01-24T23:24:00.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:00.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 1 -- 172.21.15.132:0/965931178 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f835c001d90 con 0x7f836413b2c0 2024-01-24T23:24:00.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:00.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f83617fa640 4 set_mon_vals no callback set 2024-01-24T23:24:00.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:00.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f834bfff640 10 monclient: dump: 2024-01-24T23:24:00.475 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:00.475 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:00.475 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.475 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.476 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:00.476 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:00.476 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:00.476 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:00.476 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:00.476 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:00.476 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:00.476 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:00.476 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:00.476 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:00.476 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:00.476 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:00.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:00.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:00.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.459+0000 7f8369a58640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:00.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.463+0000 7f8369a58640 1 -- 172.21.15.132:0/965931178 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8344005090 con 0x7f836413b2c0 2024-01-24T23:24:00.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.463+0000 7f834bfff640 1 -- 172.21.15.132:0/965931178 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f835c003e80 con 0x7f836413b2c0 2024-01-24T23:24:00.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.463+0000 7f834bfff640 1 -- 172.21.15.132:0/965931178 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(71..71 src has 1..71) v4 ==== 5385+0+0 (unknown 892542886 0 0) 0x7f835c043ea0 con 0x7f836413b2c0 2024-01-24T23:24:00.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.463+0000 7f8362ffd640 10 monclient: get_auth_request con 0x7f8350036f00 auth_method 0 2024-01-24T23:24:00.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.467+0000 7f834bfff640 1 -- 172.21.15.132:0/965931178 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f835c014830 con 0x7f836413b2c0 2024-01-24T23:24:00.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.467+0000 7f834bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:00.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.467+0000 7f834bfff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:00.624 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.623+0000 7f8369a58640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:00.624 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.623+0000 7f8369a58640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:00.624 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.623+0000 7f8369a58640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:00.624 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.623+0000 7f8369a58640 1 -- 172.21.15.132:0/965931178 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f8344005b50 con 0x7f836413b2c0 2024-01-24T23:24:00.625 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.623+0000 7f834bfff640 1 -- 172.21.15.132:0/965931178 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v71) v1 ==== 56+0+1474 (unknown 1479012056 0 4123843803) 0x7f835c012ed0 con 0x7f836413b2c0 2024-01-24T23:24:00.625 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.623+0000 7f834bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:00.625 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.623+0000 7f834bfff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:24:00.712 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.711+0000 7f8369a58640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:00.712 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.711+0000 7f8369a58640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:00.712 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.711+0000 7f8369a58640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:00.712 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.711+0000 7f8369a58640 1 -- 172.21.15.132:0/965931178 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f8344005ce0 con 0x7f836413b2c0 2024-01-24T23:24:00.713 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.711+0000 7f834bfff640 1 -- 172.21.15.132:0/965931178 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v71) v1 ==== 56+0+1474 (unknown 1479012056 0 4123843803) 0x7f835c014650 con 0x7f836413b2c0 2024-01-24T23:24:00.713 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.711+0000 7f834bfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:00.713 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.711+0000 7f834bfff640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:24:00.715 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.711+0000 7f8369a58640 1 -- 172.21.15.132:0/965931178 >> v1:172.21.15.132:6812/1835401922 conn(0x7f8350036f00 legacy=0x7f83500393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:00.715 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.711+0000 7f8369a58640 10 monclient: shutdown 2024-01-24T23:24:00.715 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.711+0000 7f8369a58640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:00.716 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.711+0000 7f8369a58640 1 -- 172.21.15.132:0/965931178 >> v1:172.21.15.132:6791/0 conn(0x7f836413b2c0 legacy=0x7f83641d38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:00.716 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.715+0000 7f8369a58640 1 -- 172.21.15.132:0/965931178 shutdown_connections 2024-01-24T23:24:00.716 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.715+0000 7f8369a58640 1 -- 172.21.15.132:0/965931178 >> 172.21.15.132:0/965931178 conn(0x7f83640a5a10 msgr2=0x7f83640a2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:00.716 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.715+0000 7f8369a58640 1 -- 172.21.15.132:0/965931178 shutdown_connections 2024-01-24T23:24:00.716 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.715+0000 7f8369a58640 1 -- 172.21.15.132:0/965931178 wait complete. 2024-01-24T23:24:00.730 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep -c host2 2024-01-24T23:24:00.730 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd tree 2024-01-24T23:24:00.730 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep -q 2 2024-01-24T23:24:00.837 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.835+0000 7f5f6cdfa640 10 monclient: get_monmap_and_config 2024-01-24T23:24:00.837 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.835+0000 7f5f6cdfa640 10 monclient: build_initial_monmap 2024-01-24T23:24:00.837 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.835+0000 7f5f6cdfa640 10 monclient: monmap: 2024-01-24T23:24:00.838 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:00.838 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:00.838 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:00.838971+0000 2024-01-24T23:24:00.838 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:00.838971+0000 2024-01-24T23:24:00.838 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:00.838 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:00.838 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:00.838 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:00.838 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:00.838 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:00.838 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:00.838 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:00.838 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:00.839 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:00.839 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:00.839 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:00.846 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.835+0000 7f5f6cdfa640 1 Processor -- start 2024-01-24T23:24:00.846 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.835+0000 7f5f6cdfa640 1 -- start start 2024-01-24T23:24:00.846 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.835+0000 7f5f6cdfa640 10 monclient: init 2024-01-24T23:24:00.846 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.835+0000 7f5f6cdfa640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:00.846 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.835+0000 7f5f6cdfa640 10 monclient: _add_conns ranks=[5,6,4,1,3,7,2,8,0] 2024-01-24T23:24:00.846 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.835+0000 7f5f6cdfa640 10 monclient(hunting): picked mon.noname-h con 0x7f5f6809ff20 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:00.846 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.835+0000 7f5f6cdfa640 10 monclient(hunting): picked mon.noname-d con 0x7f5f6809f9c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:00.846 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.835+0000 7f5f6cdfa640 10 monclient(hunting): picked mon.noname-c con 0x7f5f6813b2c0 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:00.846 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.835+0000 7f5f6cdfa640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f68064700 con 0x7f5f6813b2c0 2024-01-24T23:24:00.846 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.835+0000 7f5f6cdfa640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f6809e760 con 0x7f5f6809ff20 2024-01-24T23:24:00.846 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.835+0000 7f5f6cdfa640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f680a1160 con 0x7f5f6809f9c0 2024-01-24T23:24:00.846 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.835+0000 7f5f6cdfa640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:00.847 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.835+0000 7f5f6cdfa640 10 monclient(hunting): authenticate will time out at 1128.163549s 2024-01-24T23:24:00.847 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.835+0000 7f5f6659c640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f5f6809ff20 0x7f5f680a0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:45924/0 (socket says 172.21.15.132:45924) 2024-01-24T23:24:00.847 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.835+0000 7f5f6659c640 1 -- 172.21.15.132:0/2679965105 learned_addr learned my addr 172.21.15.132:0/2679965105 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:00.847 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.835+0000 7f5f66d9d640 1 -- 172.21.15.132:0/2679965105 >> v1:172.21.15.115:6791/0 conn(0x7f5f6813b2c0 legacy=0x7f5f6813d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:00.847 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 1 -- 172.21.15.132:0/2679965105 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1799360186 0 0) 0x7f5f6809e760 con 0x7f5f6809ff20 2024-01-24T23:24:00.847 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 10 monclient(hunting): _negotiate 2024-01-24T23:24:00.847 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:00.847 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:00.847 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 10 monclient(hunting): my global_id is 5342 2024-01-24T23:24:00.847 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 1 -- 172.21.15.132:0/2679965105 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f50003530 con 0x7f5f6809ff20 2024-01-24T23:24:00.847 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 1 -- 172.21.15.132:0/2679965105 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1432595927 0 0) 0x7f5f50003530 con 0x7f5f6809ff20 2024-01-24T23:24:00.847 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 1 -- 172.21.15.132:0/2679965105 >> v1:172.21.15.115:6792/0 conn(0x7f5f6809f9c0 legacy=0x7f5f68139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:00.847 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 1 -- 172.21.15.132:0/2679965105 >> v1:172.21.15.115:6791/0 conn(0x7f5f6813b2c0 legacy=0x7f5f6813d6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:00.847 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 10 monclient: _finish_hunting 0 2024-01-24T23:24:00.847 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 1 monclient: found mon.noname-h 2024-01-24T23:24:00.848 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:00.848 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:24:00.848 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 1 -- 172.21.15.132:0/2679965105 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f680a0a10 con 0x7f5f6809ff20 2024-01-24T23:24:00.848 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 10 monclient: _finish_auth 0 2024-01-24T23:24:00.848 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 10 monclient: _check_auth_tickets 2024-01-24T23:24:00.848 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:00.848 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 5 monclient: authenticate success, global_id 5342 2024-01-24T23:24:00.848 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:00.848 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 1 -- 172.21.15.132:0/2679965105 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5f60001b40 con 0x7f5f6809ff20 2024-01-24T23:24:00.848 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:00.848 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:00.848 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 10 monclient: dump: 2024-01-24T23:24:00.848 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:00.849 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:00.849 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.849 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.849 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:00.849 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:00.849 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:00.849 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:00.849 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:00.849 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:00.849 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:00.849 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:00.849 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:00.849 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:00.849 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:00.849 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:00.849 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:00.850 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 1 -- 172.21.15.132:0/2679965105 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5f60001e60 con 0x7f5f6809ff20 2024-01-24T23:24:00.850 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:00.850 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:00.850 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 4 set_mon_vals no callback set 2024-01-24T23:24:00.850 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 1 -- 172.21.15.132:0/2679965105 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5f60004f70 con 0x7f5f6809ff20 2024-01-24T23:24:00.850 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:00.850 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:00.850 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6559a640 10 monclient: dump: 2024-01-24T23:24:00.850 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:00.850 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:00.850 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.850 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.850 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:00.850 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:00.850 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:00.850 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:00.850 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:00.851 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:00.851 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:00.851 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:00.851 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:00.851 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:00.851 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:00.851 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:00.851 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 10 monclient: shutdown 2024-01-24T23:24:00.851 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:00.851 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 1 -- 172.21.15.132:0/2679965105 >> v1:172.21.15.132:6791/0 conn(0x7f5f6809ff20 legacy=0x7f5f680a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:00.851 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 1 -- 172.21.15.132:0/2679965105 shutdown_connections 2024-01-24T23:24:00.851 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 1 -- 172.21.15.132:0/2679965105 >> 172.21.15.132:0/2679965105 conn(0x7f5f680a5a10 msgr2=0x7f5f680a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:00.851 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 1 -- 172.21.15.132:0/2679965105 shutdown_connections 2024-01-24T23:24:00.851 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 1 -- 172.21.15.132:0/2679965105 wait complete. 2024-01-24T23:24:00.852 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 10 monclient: build_initial_monmap 2024-01-24T23:24:00.852 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 10 monclient: monmap: 2024-01-24T23:24:00.852 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:00.852 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:00.852 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:00.852 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:00.852 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:00.852 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:00.852 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:00.852 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:00.852 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:00.852 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:00.852 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:00.852 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:00.852 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:00.852 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:00.852 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:00.853 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:00.853 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 1 Processor -- start 2024-01-24T23:24:00.853 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 1 -- start start 2024-01-24T23:24:00.853 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 10 monclient: init 2024-01-24T23:24:00.853 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:00.853 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 10 monclient: _add_conns ranks=[5,0,1,3,4,8,6,2,7] 2024-01-24T23:24:00.853 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 10 monclient(hunting): picked mon.noname-h con 0x7f5f6809f9c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:00.853 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 10 monclient(hunting): picked mon.noname-a con 0x7f5f6809ff20 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:00.853 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 10 monclient(hunting): picked mon.noname-f con 0x7f5f6813b2c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:00.853 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f68071960 con 0x7f5f6809ff20 2024-01-24T23:24:00.853 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f68071680 con 0x7f5f6813b2c0 2024-01-24T23:24:00.853 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f6811c430 con 0x7f5f6809f9c0 2024-01-24T23:24:00.853 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:00.853 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f66d9d640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f5f6813b2c0 legacy=0x7f5f681d1490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:00.853 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 10 monclient(hunting): authenticate will time out at 1128.166741s 2024-01-24T23:24:00.853 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6659c640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f5f6809f9c0 0x7f5f6811b610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:45934/0 (socket says 172.21.15.132:45934) 2024-01-24T23:24:00.854 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6659c640 1 -- 172.21.15.132:0/169474758 learned_addr learned my addr 172.21.15.132:0/169474758 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:00.854 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f65d9b640 1 -- 172.21.15.132:0/169474758 >> v1:172.21.15.115:6789/0 conn(0x7f5f6809ff20 legacy=0x7f5f6811bd20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:00.854 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 1 -- 172.21.15.132:0/169474758 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912031752 0 0) 0x7f5f6811c430 con 0x7f5f6809f9c0 2024-01-24T23:24:00.854 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:00.854 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:00.854 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:00.854 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 10 monclient(hunting): my global_id is 5351 2024-01-24T23:24:00.854 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 1 -- 172.21.15.132:0/169474758 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f48003530 con 0x7f5f6809f9c0 2024-01-24T23:24:00.854 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 1 -- 172.21.15.132:0/169474758 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2833103055 0 0) 0x7f5f48003530 con 0x7f5f6809f9c0 2024-01-24T23:24:00.854 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 1 -- 172.21.15.132:0/169474758 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5f6811c430 con 0x7f5f6809f9c0 2024-01-24T23:24:00.854 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 1 -- 172.21.15.132:0/169474758 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5f60001b00 con 0x7f5f6809f9c0 2024-01-24T23:24:00.854 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:00.854 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:00.855 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 10 monclient(hunting): dump: 2024-01-24T23:24:00.855 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:00.855 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:00.855 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.855 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.855 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:00.855 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:00.855 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:00.855 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:00.855 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:00.855 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:00.855 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:00.855 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:00.855 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:00.855 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:00.855 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:00.856 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:00.856 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:00.856 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 1 -- 172.21.15.132:0/169474758 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 416208657 0 0) 0x7f5f6811c430 con 0x7f5f6809f9c0 2024-01-24T23:24:00.856 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 1 -- 172.21.15.132:0/169474758 >> v1:172.21.15.132:6789/0 conn(0x7f5f6813b2c0 legacy=0x7f5f681d1490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:00.856 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 1 -- 172.21.15.132:0/169474758 >> v1:172.21.15.115:6789/0 conn(0x7f5f6809ff20 legacy=0x7f5f6811bd20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:00.856 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:00.856 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 1 monclient: found mon.h 2024-01-24T23:24:00.856 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:00.856 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:00.856 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 1 -- 172.21.15.132:0/169474758 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6811e0a0 con 0x7f5f6809f9c0 2024-01-24T23:24:00.856 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 10 monclient: _finish_auth 0 2024-01-24T23:24:00.856 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:00.856 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:00.856 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 5 monclient: authenticate success, global_id 5351 2024-01-24T23:24:00.856 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 10 monclient: _renew_subs 2024-01-24T23:24:00.857 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:00.857 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 1 -- 172.21.15.132:0/169474758 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f6811e940 con 0x7f5f6809f9c0 2024-01-24T23:24:00.857 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 10 monclient: _renew_subs 2024-01-24T23:24:00.857 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:00.857 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f6cdfa640 1 -- 172.21.15.132:0/169474758 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f68134430 con 0x7f5f6809f9c0 2024-01-24T23:24:00.857 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 1 -- 172.21.15.132:0/169474758 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5f60001fd0 con 0x7f5f6809f9c0 2024-01-24T23:24:00.857 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:00.857 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 1 -- 172.21.15.132:0/169474758 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5f60005830 con 0x7f5f6809f9c0 2024-01-24T23:24:00.857 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:00.857 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f64d99640 4 set_mon_vals no callback set 2024-01-24T23:24:00.857 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:00.857 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.839+0000 7f5f56ffd640 10 monclient: dump: 2024-01-24T23:24:00.857 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:00.857 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:00.858 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.858 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:00.858 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:00.858 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:00.858 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:00.858 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:00.858 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:00.858 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:00.858 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:00.858 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:00.858 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:00.858 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:00.858 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:00.858 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:00.858 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.843+0000 7f5f6cdfa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:00.858 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.843+0000 7f5f6cdfa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:00.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.843+0000 7f5f6cdfa640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:00.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.843+0000 7f5f6cdfa640 1 -- 172.21.15.132:0/169474758 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4c005090 con 0x7f5f6809f9c0 2024-01-24T23:24:00.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.843+0000 7f5f56ffd640 1 -- 172.21.15.132:0/169474758 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f5f600102b0 con 0x7f5f6809f9c0 2024-01-24T23:24:00.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.843+0000 7f5f56ffd640 1 -- 172.21.15.132:0/169474758 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(71..71 src has 1..71) v4 ==== 5385+0+0 (unknown 892542886 0 0) 0x7f5f60044470 con 0x7f5f6809f9c0 2024-01-24T23:24:00.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.843+0000 7f5f65d9b640 10 monclient: get_auth_request con 0x7f5f48036f10 auth_method 0 2024-01-24T23:24:00.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.847+0000 7f5f56ffd640 1 -- 172.21.15.132:0/169474758 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f5f60014fd0 con 0x7f5f6809f9c0 2024-01-24T23:24:00.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.847+0000 7f5f56ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:00.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:00.847+0000 7f5f56ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:01.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.003+0000 7f5f6cdfa640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:01.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.003+0000 7f5f6cdfa640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:01.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.003+0000 7f5f6cdfa640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:01.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.003+0000 7f5f6cdfa640 1 -- 172.21.15.132:0/169474758 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f5f4c005b50 con 0x7f5f6809f9c0 2024-01-24T23:24:01.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.003+0000 7f5f56ffd640 1 -- 172.21.15.132:0/169474758 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v71) v1 ==== 56+0+1474 (unknown 1479012056 0 4123843803) 0x7f5f60013670 con 0x7f5f6809f9c0 2024-01-24T23:24:01.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.003+0000 7f5f56ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:01.006 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.003+0000 7f5f56ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:24:01.097 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.095+0000 7f5f6cdfa640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:01.097 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.095+0000 7f5f6cdfa640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:01.098 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.095+0000 7f5f6cdfa640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:01.098 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.095+0000 7f5f6cdfa640 1 -- 172.21.15.132:0/169474758 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f5f4c005ce0 con 0x7f5f6809f9c0 2024-01-24T23:24:01.098 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.095+0000 7f5f56ffd640 1 -- 172.21.15.132:0/169474758 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v71) v1 ==== 56+0+1474 (unknown 1479012056 0 4123843803) 0x7f5f60014df0 con 0x7f5f6809f9c0 2024-01-24T23:24:01.098 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.095+0000 7f5f56ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:01.098 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.095+0000 7f5f56ffd640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:24:01.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.099+0000 7f5f6cdfa640 1 -- 172.21.15.132:0/169474758 >> v1:172.21.15.132:6812/1835401922 conn(0x7f5f48036f10 legacy=0x7f5f480393d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:01.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.099+0000 7f5f6cdfa640 10 monclient: shutdown 2024-01-24T23:24:01.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.099+0000 7f5f6cdfa640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:01.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.099+0000 7f5f6cdfa640 1 -- 172.21.15.132:0/169474758 >> v1:172.21.15.132:6791/0 conn(0x7f5f6809f9c0 legacy=0x7f5f6811b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:01.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.099+0000 7f5f6cdfa640 1 -- 172.21.15.132:0/169474758 shutdown_connections 2024-01-24T23:24:01.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.099+0000 7f5f6cdfa640 1 -- 172.21.15.132:0/169474758 >> 172.21.15.132:0/169474758 conn(0x7f5f680a5a10 msgr2=0x7f5f680a2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:01.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.099+0000 7f5f6cdfa640 1 -- 172.21.15.132:0/169474758 shutdown_connections 2024-01-24T23:24:01.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.099+0000 7f5f6cdfa640 1 -- 172.21.15.132:0/169474758 wait complete. 2024-01-24T23:24:01.114 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd crush rm host1 foo 2024-01-24T23:24:01.114 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:24:01.114 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm host1 foo 2024-01-24T23:24:01.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.223+0000 7f5054574640 10 monclient: get_monmap_and_config 2024-01-24T23:24:01.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.223+0000 7f5054574640 10 monclient: build_initial_monmap 2024-01-24T23:24:01.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.223+0000 7f5054574640 10 monclient: monmap: 2024-01-24T23:24:01.226 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:01.226 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:01.226 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:01.227551+0000 2024-01-24T23:24:01.226 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:01.227551+0000 2024-01-24T23:24:01.226 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:01.226 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:01.226 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:01.227 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:01.227 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:01.227 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:01.227 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:01.227 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:01.227 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:01.227 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:01.227 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:01.227 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:01.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 1 Processor -- start 2024-01-24T23:24:01.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 1 -- start start 2024-01-24T23:24:01.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 10 monclient: init 2024-01-24T23:24:01.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:01.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 10 monclient: _add_conns ranks=[5,4,1,6,7,3,2,0,8] 2024-01-24T23:24:01.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 10 monclient(hunting): picked mon.noname-h con 0x7f504c134bd0 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:01.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 10 monclient(hunting): picked mon.noname-c con 0x7f504c06db30 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:01.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 10 monclient(hunting): picked mon.noname-f con 0x7f504c13b2f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:01.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f504c064700 con 0x7f504c13b2f0 2024-01-24T23:24:01.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f504c070b60 con 0x7f504c06db30 2024-01-24T23:24:01.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f504c1356c0 con 0x7f504c134bd0 2024-01-24T23:24:01.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:01.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 10 monclient(hunting): authenticate will time out at 1128.552993s 2024-01-24T23:24:01.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5052b11640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f504c13b2f0 legacy=0x7f504c13d6e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:01.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5052310640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f504c134bd0 0x7f504c134fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:45944/0 (socket says 172.21.15.132:45944) 2024-01-24T23:24:01.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5052310640 1 -- 172.21.15.132:0/1667798540 learned_addr learned my addr 172.21.15.132:0/1667798540 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:01.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5051b0f640 1 -- 172.21.15.132:0/1667798540 >> v1:172.21.15.115:6791/0 conn(0x7f504c06db30 legacy=0x7f504c139bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:01.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 1 -- 172.21.15.132:0/1667798540 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1907743335 0 0) 0x7f504c1356c0 con 0x7f504c134bd0 2024-01-24T23:24:01.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 10 monclient(hunting): _negotiate 2024-01-24T23:24:01.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:01.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:01.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 10 monclient(hunting): my global_id is 5360 2024-01-24T23:24:01.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 1 -- 172.21.15.132:0/1667798540 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5038003530 con 0x7f504c134bd0 2024-01-24T23:24:01.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 1 -- 172.21.15.132:0/1667798540 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1930558704 0 0) 0x7f5038003530 con 0x7f504c134bd0 2024-01-24T23:24:01.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 1 -- 172.21.15.132:0/1667798540 >> v1:172.21.15.115:6791/0 conn(0x7f504c06db30 legacy=0x7f504c139bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:01.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 1 -- 172.21.15.132:0/1667798540 >> v1:172.21.15.132:6789/0 conn(0x7f504c13b2f0 legacy=0x7f504c13d6e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:01.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 10 monclient: _finish_hunting 0 2024-01-24T23:24:01.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 1 monclient: found mon.noname-h 2024-01-24T23:24:01.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:01.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:24:01.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 1 -- 172.21.15.132:0/1667798540 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504c070e40 con 0x7f504c134bd0 2024-01-24T23:24:01.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 10 monclient: _finish_auth 0 2024-01-24T23:24:01.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 10 monclient: _check_auth_tickets 2024-01-24T23:24:01.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:01.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 1 -- 172.21.15.132:0/1667798540 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f50480017c0 con 0x7f504c134bd0 2024-01-24T23:24:01.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 5 monclient: authenticate success, global_id 5360 2024-01-24T23:24:01.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:01.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:01.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:01.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 10 monclient: dump: 2024-01-24T23:24:01.237 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:01.237 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:01.237 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:01.237 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:01.237 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:01.237 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:01.237 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:01.237 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:01.237 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:01.237 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:01.237 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:01.238 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:01.238 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:01.238 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:01.238 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:01.238 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:01.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:01.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 1 -- 172.21.15.132:0/1667798540 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5048001ae0 con 0x7f504c134bd0 2024-01-24T23:24:01.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:01.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 1 -- 172.21.15.132:0/1667798540 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5048004bf0 con 0x7f504c134bd0 2024-01-24T23:24:01.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:01.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 4 set_mon_vals no callback set 2024-01-24T23:24:01.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:01.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:01.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f505130e640 10 monclient: dump: 2024-01-24T23:24:01.238 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:01.239 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:01.239 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:01.239 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:01.239 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:01.239 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:01.239 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:01.239 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:01.239 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:01.239 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:01.239 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:01.239 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:01.239 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:01.239 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:01.239 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:01.239 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:01.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 10 monclient: shutdown 2024-01-24T23:24:01.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:01.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 1 -- 172.21.15.132:0/1667798540 >> v1:172.21.15.132:6791/0 conn(0x7f504c134bd0 legacy=0x7f504c134fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:01.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 1 -- 172.21.15.132:0/1667798540 shutdown_connections 2024-01-24T23:24:01.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 1 -- 172.21.15.132:0/1667798540 >> 172.21.15.132:0/1667798540 conn(0x7f504c0a5eb0 msgr2=0x7f504c0a23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:01.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 1 -- 172.21.15.132:0/1667798540 shutdown_connections 2024-01-24T23:24:01.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 1 -- 172.21.15.132:0/1667798540 wait complete. 2024-01-24T23:24:01.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 10 monclient: build_initial_monmap 2024-01-24T23:24:01.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 10 monclient: monmap: 2024-01-24T23:24:01.240 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:01.240 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:01.240 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:01.240 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:01.240 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:01.241 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:01.241 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:01.241 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:01.241 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:01.241 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:01.241 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:01.241 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:01.241 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:01.241 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:01.241 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:01.241 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:01.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 1 Processor -- start 2024-01-24T23:24:01.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 1 -- start start 2024-01-24T23:24:01.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 10 monclient: init 2024-01-24T23:24:01.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:01.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 10 monclient: _add_conns ranks=[0,4,7,8,1,3,5,6,2] 2024-01-24T23:24:01.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 10 monclient(hunting): picked mon.noname-a con 0x7f504c06db30 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:01.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 10 monclient(hunting): picked mon.noname-c con 0x7f504c134bd0 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:01.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 10 monclient(hunting): picked mon.noname-i con 0x7f504c13b2f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:01.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f504c138570 con 0x7f504c06db30 2024-01-24T23:24:01.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f504c12f030 con 0x7f504c134bd0 2024-01-24T23:24:01.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f504c06e000 con 0x7f504c13b2f0 2024-01-24T23:24:01.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:01.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5054574640 10 monclient(hunting): authenticate will time out at 1128.555753s 2024-01-24T23:24:01.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5052310640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f504c06db30 legacy=0x7f504c1c56b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:01.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5051b0f640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f504c134bd0 legacy=0x7f504c1c5dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:01.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5052b11640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f504c13b2f0 0x7f504c1d0f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:46252/0 (socket says 172.21.15.132:46252) 2024-01-24T23:24:01.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.227+0000 7f5052b11640 1 -- 172.21.15.132:0/3472594437 learned_addr learned my addr 172.21.15.132:0/3472594437 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:01.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 1 -- 172.21.15.132:0/3472594437 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570234853 0 0) 0x7f504c06e000 con 0x7f504c13b2f0 2024-01-24T23:24:01.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:01.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:01.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:01.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 10 monclient(hunting): my global_id is 5416 2024-01-24T23:24:01.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 1 -- 172.21.15.132:0/3472594437 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5030003530 con 0x7f504c13b2f0 2024-01-24T23:24:01.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 1 -- 172.21.15.132:0/3472594437 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4012411027 0 0) 0x7f5030003530 con 0x7f504c13b2f0 2024-01-24T23:24:01.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 1 -- 172.21.15.132:0/3472594437 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f504c06e000 con 0x7f504c13b2f0 2024-01-24T23:24:01.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 1 -- 172.21.15.132:0/3472594437 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5044004560 con 0x7f504c13b2f0 2024-01-24T23:24:01.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:01.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:01.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 10 monclient(hunting): dump: 2024-01-24T23:24:01.243 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:01.243 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:01.243 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:01.243 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:01.244 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:01.244 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:01.244 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:01.244 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:01.244 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:01.244 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:01.244 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:01.244 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:01.244 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:01.244 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:01.244 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:01.244 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:01.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:01.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 1 -- 172.21.15.132:0/3472594437 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2547517570 0 0) 0x7f504c06e000 con 0x7f504c13b2f0 2024-01-24T23:24:01.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 1 -- 172.21.15.132:0/3472594437 >> v1:172.21.15.115:6791/0 conn(0x7f504c134bd0 legacy=0x7f504c1c5dc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:01.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 1 -- 172.21.15.132:0/3472594437 >> v1:172.21.15.115:6789/0 conn(0x7f504c06db30 legacy=0x7f504c1c56b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:01.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:01.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 1 monclient: found mon.i 2024-01-24T23:24:01.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:01.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:01.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 1 -- 172.21.15.132:0/3472594437 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504c139bf0 con 0x7f504c13b2f0 2024-01-24T23:24:01.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 10 monclient: _finish_auth 0 2024-01-24T23:24:01.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:01.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:01.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5054574640 5 monclient: authenticate success, global_id 5416 2024-01-24T23:24:01.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5054574640 10 monclient: _renew_subs 2024-01-24T23:24:01.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5054574640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:01.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5054574640 1 -- 172.21.15.132:0/3472594437 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f504c1c64d0 con 0x7f504c13b2f0 2024-01-24T23:24:01.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5054574640 10 monclient: _renew_subs 2024-01-24T23:24:01.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5054574640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:01.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5054574640 1 -- 172.21.15.132:0/3472594437 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f504c139910 con 0x7f504c13b2f0 2024-01-24T23:24:01.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 1 -- 172.21.15.132:0/3472594437 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5044004880 con 0x7f504c13b2f0 2024-01-24T23:24:01.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:01.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 1 -- 172.21.15.132:0/3472594437 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5044004db0 con 0x7f504c13b2f0 2024-01-24T23:24:01.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:01.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5050b0d640 4 set_mon_vals no callback set 2024-01-24T23:24:01.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:01.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 10 monclient: dump: 2024-01-24T23:24:01.246 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:01.246 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:01.246 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:01.246 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:01.246 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:01.246 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:01.247 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:01.247 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:01.247 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:01.247 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:01.247 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:01.247 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:01.247 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:01.247 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:01.247 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:01.247 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:01.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5054574640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:01.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5054574640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:01.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5054574640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:01.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 1 -- 172.21.15.132:0/3472594437 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f5044004f90 con 0x7f504c13b2f0 2024-01-24T23:24:01.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5054574640 1 -- 172.21.15.132:0/3472594437 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5034005090 con 0x7f504c13b2f0 2024-01-24T23:24:01.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5042ffd640 1 -- 172.21.15.132:0/3472594437 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(71..71 src has 1..71) v4 ==== 5385+0+0 (unknown 892542886 0 0) 0x7f5044044130 con 0x7f504c13b2f0 2024-01-24T23:24:01.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.231+0000 7f5052310640 10 monclient: get_auth_request con 0x7f5030036ef0 auth_method 0 2024-01-24T23:24:01.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.235+0000 7f5042ffd640 1 -- 172.21.15.132:0/3472594437 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f5044014dc0 con 0x7f504c13b2f0 2024-01-24T23:24:01.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.235+0000 7f5042ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:01.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.235+0000 7f5042ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:01.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.395+0000 7f5054574640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host1", "ancestor": "foo"}] 2024-01-24T23:24:01.398 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.395+0000 7f5054574640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "host1", "ancestor": "foo"}] 2024-01-24T23:24:01.398 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.395+0000 7f5054574640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:01.398 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.395+0000 7f5054574640 1 -- 172.21.15.132:0/3472594437 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "host1", "ancestor": "foo"} v 0) v1 -- 0x7f50340058c0 con 0x7f504c13b2f0 2024-01-24T23:24:01.399 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.395+0000 7f5042ffd640 1 -- 172.21.15.132:0/3472594437 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host1", "ancestor": "foo"}]=-39 (39) Directory not empty v71) v1 ==== 120+0+0 (unknown 3400447075 0 0) 0x7f5044013230 con 0x7f504c13b2f0 2024-01-24T23:24:01.399 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.395+0000 7f5042ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "host1", "ancestor": "foo"}] 2024-01-24T23:24:01.399 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.395+0000 7f5042ffd640 10 monclient: _finish_command 2 = mon:39 (39) Directory not empty 2024-01-24T23:24:01.399 INFO:tasks.workunit.client.0.smithi132.stderr:Error ENOTEMPTY: (39) Directory not empty 2024-01-24T23:24:01.404 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:24:01.405 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush unlink host1 foo 2024-01-24T23:24:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.515+0000 7ff379beb640 10 monclient: get_monmap_and_config 2024-01-24T23:24:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.515+0000 7ff379beb640 10 monclient: build_initial_monmap 2024-01-24T23:24:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.515+0000 7ff379beb640 10 monclient: monmap: 2024-01-24T23:24:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:01.519506+0000 2024-01-24T23:24:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:01.519506+0000 2024-01-24T23:24:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:01.519 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:01.519 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:01.519 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:01.519 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:01.519 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:01.519 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:01.519 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:01.519 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:01.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.515+0000 7ff379beb640 1 Processor -- start 2024-01-24T23:24:01.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.515+0000 7ff379beb640 1 -- start start 2024-01-24T23:24:01.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.515+0000 7ff379beb640 10 monclient: init 2024-01-24T23:24:01.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.515+0000 7ff379beb640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:01.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 10 monclient: _add_conns ranks=[0,2,5,7,1,3,8,6,4] 2024-01-24T23:24:01.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 10 monclient(hunting): picked mon.noname-a con 0x7ff374133b50 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:01.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 10 monclient(hunting): picked mon.noname-b con 0x7ff37409ec20 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:01.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 10 monclient(hunting): picked mon.noname-h con 0x7ff37413a1d0 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:01.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff374064700 con 0x7ff374133b50 2024-01-24T23:24:01.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff37409d9f0 con 0x7ff37409ec20 2024-01-24T23:24:01.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff37409dc60 con 0x7ff37413a1d0 2024-01-24T23:24:01.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:01.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 10 monclient(hunting): authenticate will time out at 1128.844111s 2024-01-24T23:24:01.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3737fe640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7ff374133b50 legacy=0x7ff374133f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:01.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff373fff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7ff37413a1d0 0x7ff37413c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:45956/0 (socket says 172.21.15.132:45956) 2024-01-24T23:24:01.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff373fff640 1 -- 172.21.15.132:0/1246438030 learned_addr learned my addr 172.21.15.132:0/1246438030 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:01.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 1 -- 172.21.15.132:0/1246438030 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986926085 0 0) 0x7ff37409dc60 con 0x7ff37413a1d0 2024-01-24T23:24:01.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:01.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:01.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:01.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 10 monclient(hunting): my global_id is 5369 2024-01-24T23:24:01.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 1 -- 172.21.15.132:0/1246438030 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff360003530 con 0x7ff37413a1d0 2024-01-24T23:24:01.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 1 -- 172.21.15.132:0/1246438030 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1088721658 0 0) 0x7ff360003530 con 0x7ff37413a1d0 2024-01-24T23:24:01.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 1 -- 172.21.15.132:0/1246438030 >> v1:172.21.15.115:6790/0 conn(0x7ff37409ec20 legacy=0x7ff374138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:01.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 1 -- 172.21.15.132:0/1246438030 >> v1:172.21.15.115:6789/0 conn(0x7ff374133b50 legacy=0x7ff374133f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:01.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 10 monclient: _finish_hunting 0 2024-01-24T23:24:01.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 1 monclient: found mon.noname-h 2024-01-24T23:24:01.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:01.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:24:01.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 1 -- 172.21.15.132:0/1246438030 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37409df40 con 0x7ff37413a1d0 2024-01-24T23:24:01.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 10 monclient: _finish_auth 0 2024-01-24T23:24:01.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 10 monclient: _check_auth_tickets 2024-01-24T23:24:01.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:01.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 1 -- 172.21.15.132:0/1246438030 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff3640017c0 con 0x7ff37413a1d0 2024-01-24T23:24:01.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 5 monclient: authenticate success, global_id 5369 2024-01-24T23:24:01.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:01.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:01.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:01.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 10 monclient: dump: 2024-01-24T23:24:01.526 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:01.526 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:01.526 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:01.526 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:01.526 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:01.526 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:01.526 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:01.526 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:01.526 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:01.526 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:01.526 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:01.526 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:01.526 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:01.526 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:01.527 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:01.527 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:01.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:01.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 1 -- 172.21.15.132:0/1246438030 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff3640019e0 con 0x7ff37413a1d0 2024-01-24T23:24:01.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:01.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:01.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 1 -- 172.21.15.132:0/1246438030 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff364004bf0 con 0x7ff37413a1d0 2024-01-24T23:24:01.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 4 set_mon_vals no callback set 2024-01-24T23:24:01.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:01.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:01.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3727fc640 10 monclient: dump: 2024-01-24T23:24:01.527 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:01.527 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:01.527 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:01.527 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:01.527 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:01.528 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:01.528 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:01.528 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:01.528 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:01.528 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:01.528 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:01.528 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:01.528 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:01.528 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:01.528 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:01.528 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:01.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 10 monclient: shutdown 2024-01-24T23:24:01.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:01.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 1 -- 172.21.15.132:0/1246438030 >> v1:172.21.15.132:6791/0 conn(0x7ff37413a1d0 legacy=0x7ff37413c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:01.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 1 -- 172.21.15.132:0/1246438030 shutdown_connections 2024-01-24T23:24:01.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 1 -- 172.21.15.132:0/1246438030 >> 172.21.15.132:0/1246438030 conn(0x7ff3740a5a10 msgr2=0x7ff3740a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:01.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 1 -- 172.21.15.132:0/1246438030 shutdown_connections 2024-01-24T23:24:01.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 1 -- 172.21.15.132:0/1246438030 wait complete. 2024-01-24T23:24:01.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 10 monclient: build_initial_monmap 2024-01-24T23:24:01.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 10 monclient: monmap: 2024-01-24T23:24:01.529 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:01.529 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:01.529 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:01.529 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:01.529 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:01.529 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:01.529 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:01.529 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:01.529 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:01.529 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:01.529 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:01.529 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:01.530 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:01.530 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:01.530 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:01.530 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:01.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 1 Processor -- start 2024-01-24T23:24:01.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 1 -- start start 2024-01-24T23:24:01.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 10 monclient: init 2024-01-24T23:24:01.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:01.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 10 monclient: _add_conns ranks=[6,8,0,4,7,5,3,1,2] 2024-01-24T23:24:01.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 10 monclient(hunting): picked mon.noname-d con 0x7ff37409ec20 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:01.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 10 monclient(hunting): picked mon.noname-e con 0x7ff374133b50 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:01.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 10 monclient(hunting): picked mon.noname-a con 0x7ff37413a1d0 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:01.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff37406ffc0 con 0x7ff37413a1d0 2024-01-24T23:24:01.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff374133920 con 0x7ff37409ec20 2024-01-24T23:24:01.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3740a13b0 con 0x7ff374133b50 2024-01-24T23:24:01.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:01.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff379beb640 10 monclient(hunting): authenticate will time out at 1128.847427s 2024-01-24T23:24:01.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff373fff640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7ff37413a1d0 legacy=0x7ff3741d1c00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:01.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3737fe640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7ff37409ec20 0x7ff3741c9610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:55240/0 (socket says 172.21.15.132:55240) 2024-01-24T23:24:01.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.519+0000 7ff3737fe640 1 -- 172.21.15.132:0/3695090415 learned_addr learned my addr 172.21.15.132:0/3695090415 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:01.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 1 -- 172.21.15.132:0/3695090415 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1570393454 0 0) 0x7ff374133920 con 0x7ff37409ec20 2024-01-24T23:24:01.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:01.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:01.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:01.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 10 monclient(hunting): my global_id is 5406 2024-01-24T23:24:01.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 1 -- 172.21.15.132:0/3695090415 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff354003530 con 0x7ff37409ec20 2024-01-24T23:24:01.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 1 -- 172.21.15.132:0/3695090415 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1037024493 0 0) 0x7ff3740a13b0 con 0x7ff374133b50 2024-01-24T23:24:01.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:01.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:01.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:01.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 10 monclient(hunting): my global_id is 84554 2024-01-24T23:24:01.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 1 -- 172.21.15.132:0/3695090415 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff374133920 con 0x7ff374133b50 2024-01-24T23:24:01.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 1 -- 172.21.15.132:0/3695090415 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3259543424 0 0) 0x7ff354003530 con 0x7ff37409ec20 2024-01-24T23:24:01.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 1 -- 172.21.15.132:0/3695090415 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff3740a13b0 con 0x7ff37409ec20 2024-01-24T23:24:01.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 1 -- 172.21.15.132:0/3695090415 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1887147567 0 0) 0x7ff374133920 con 0x7ff374133b50 2024-01-24T23:24:01.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 1 -- 172.21.15.132:0/3695090415 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff354003530 con 0x7ff374133b50 2024-01-24T23:24:01.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 1 -- 172.21.15.132:0/3695090415 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff36c0046b0 con 0x7ff37409ec20 2024-01-24T23:24:01.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:01.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:24:01.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 10 monclient(hunting): dump: 2024-01-24T23:24:01.535 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:01.535 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:01.535 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:01.535 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:01.536 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:01.536 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:01.536 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:01.536 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:01.536 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:01.536 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:01.536 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:01.536 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:01.536 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:01.536 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:01.536 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:01.536 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:01.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:01.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 1 -- 172.21.15.132:0/3695090415 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff35c0019a0 con 0x7ff374133b50 2024-01-24T23:24:01.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:01.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:24:01.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 10 monclient(hunting): dump: 2024-01-24T23:24:01.537 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:01.537 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:01.537 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:01.537 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:01.537 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:01.537 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:01.537 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:01.537 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:01.537 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:01.537 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:01.537 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:01.537 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:01.538 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:01.538 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:01.538 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:01.538 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:01.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 1 -- 172.21.15.132:0/3695090415 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1175889698 0 0) 0x7ff3740a13b0 con 0x7ff37409ec20 2024-01-24T23:24:01.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 1 -- 172.21.15.132:0/3695090415 >> v1:172.21.15.115:6793/0 conn(0x7ff374133b50 legacy=0x7ff3741ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:01.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 1 -- 172.21.15.132:0/3695090415 >> v1:172.21.15.115:6789/0 conn(0x7ff37413a1d0 legacy=0x7ff3741d1c00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:01.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:24:01.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 1 monclient: found mon.d 2024-01-24T23:24:01.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:01.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:01.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 1 -- 172.21.15.132:0/3695090415 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37406f720 con 0x7ff37409ec20 2024-01-24T23:24:01.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 10 monclient: _finish_auth 0 2024-01-24T23:24:01.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:24:01.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:01.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff379beb640 5 monclient: authenticate success, global_id 5406 2024-01-24T23:24:01.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff379beb640 10 monclient: _renew_subs 2024-01-24T23:24:01.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff379beb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:01.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff379beb640 1 -- 172.21.15.132:0/3695090415 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff37406e560 con 0x7ff37409ec20 2024-01-24T23:24:01.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff379beb640 10 monclient: _renew_subs 2024-01-24T23:24:01.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff379beb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:01.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff379beb640 1 -- 172.21.15.132:0/3695090415 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3741c9d20 con 0x7ff37409ec20 2024-01-24T23:24:01.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 1 -- 172.21.15.132:0/3695090415 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff36c004910 con 0x7ff37409ec20 2024-01-24T23:24:01.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:01.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 1 -- 172.21.15.132:0/3695090415 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff36c004e40 con 0x7ff37409ec20 2024-01-24T23:24:01.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:01.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff371ffb640 4 set_mon_vals no callback set 2024-01-24T23:24:01.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:24:01.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 10 monclient: dump: 2024-01-24T23:24:01.540 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:01.540 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:01.540 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:01.540 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:01.540 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:01.540 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:01.540 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:01.540 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:01.540 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:01.540 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:01.540 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:01.540 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:01.540 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:01.541 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:01.541 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:01.541 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:01.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff379beb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:01.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff379beb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:01.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff379beb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:01.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff379beb640 1 -- 172.21.15.132:0/3695090415 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff358005090 con 0x7ff37409ec20 2024-01-24T23:24:01.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 1 -- 172.21.15.132:0/3695090415 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7ff36c005020 con 0x7ff37409ec20 2024-01-24T23:24:01.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff36bfff640 1 -- 172.21.15.132:0/3695090415 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(71..71 src has 1..71) v4 ==== 5385+0+0 (unknown 892542886 0 0) 0x7ff36c0440f0 con 0x7ff37409ec20 2024-01-24T23:24:01.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.523+0000 7ff372ffd640 10 monclient: get_auth_request con 0x7ff354037700 auth_method 0 2024-01-24T23:24:01.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.535+0000 7ff36bfff640 1 -- 172.21.15.132:0/3695090415 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7ff36c014d10 con 0x7ff37409ec20 2024-01-24T23:24:01.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.535+0000 7ff36bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:01.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.535+0000 7ff36bfff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:01.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.683+0000 7ff379beb640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-01-24T23:24:01.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.683+0000 7ff379beb640 10 monclient: _send_command 2 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-01-24T23:24:01.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.683+0000 7ff379beb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:01.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:01.683+0000 7ff379beb640 1 -- 172.21.15.132:0/3695090415 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"} v 0) v1 -- 0x7ff358005b50 con 0x7ff37409ec20 2024-01-24T23:24:02.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.063+0000 7ff36bfff640 1 -- 172.21.15.132:0/3695090415 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}]=0 removed item id -11 name 'host1' from crush map v72) v1 ==== 147+0+0 (unknown 3069479698 0 0) 0x7ff36c014ae0 con 0x7ff37409ec20 2024-01-24T23:24:02.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.063+0000 7ff36bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-01-24T23:24:02.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.063+0000 7ff36bfff640 10 monclient: _finish_command 2 = system:0 removed item id -11 name 'host1' from crush map 2024-01-24T23:24:02.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.167+0000 7ff379beb640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-01-24T23:24:02.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.167+0000 7ff379beb640 10 monclient: _send_command 3 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-01-24T23:24:02.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.167+0000 7ff379beb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:02.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.167+0000 7ff379beb640 1 -- 172.21.15.132:0/3695090415 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"} v 0) v1 -- 0x7ff3580010a0 con 0x7ff37409ec20 2024-01-24T23:24:02.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.167+0000 7ff36bfff640 1 -- 172.21.15.132:0/3695090415 <== mon.6 v1:172.21.15.115:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}]=0 item -11 does not appear in that position v72) v1 ==== 141+0+0 (unknown 1861260197 0 0) 0x7ff36c01d210 con 0x7ff37409ec20 2024-01-24T23:24:02.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.167+0000 7ff36bfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-01-24T23:24:02.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.167+0000 7ff36bfff640 10 monclient: _finish_command 3 = system:0 item -11 does not appear in that position 2024-01-24T23:24:02.171 INFO:tasks.workunit.client.0.smithi132.stderr:item -11 does not appear in that position 2024-01-24T23:24:02.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.171+0000 7ff379beb640 1 -- 172.21.15.132:0/3695090415 >> v1:172.21.15.132:6812/1835401922 conn(0x7ff354037700 legacy=0x7ff354039bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:02.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.171+0000 7ff379beb640 10 monclient: shutdown 2024-01-24T23:24:02.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.171+0000 7ff379beb640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:02.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.171+0000 7ff379beb640 1 -- 172.21.15.132:0/3695090415 >> v1:172.21.15.115:6792/0 conn(0x7ff37409ec20 legacy=0x7ff3741c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:02.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.171+0000 7ff379beb640 1 -- 172.21.15.132:0/3695090415 shutdown_connections 2024-01-24T23:24:02.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.171+0000 7ff379beb640 1 -- 172.21.15.132:0/3695090415 >> 172.21.15.132:0/3695090415 conn(0x7ff3740a5a10 msgr2=0x7ff374066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:02.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.171+0000 7ff379beb640 1 -- 172.21.15.132:0/3695090415 shutdown_connections 2024-01-24T23:24:02.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.171+0000 7ff379beb640 1 -- 172.21.15.132:0/3695090415 wait complete. 2024-01-24T23:24:02.188 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush unlink host1 foo 2024-01-24T23:24:02.309 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.307+0000 7f131fda2640 10 monclient: get_monmap_and_config 2024-01-24T23:24:02.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.307+0000 7f131fda2640 10 monclient: build_initial_monmap 2024-01-24T23:24:02.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.307+0000 7f131fda2640 10 monclient: monmap: 2024-01-24T23:24:02.310 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:02.310 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:02.310 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:02.311152+0000 2024-01-24T23:24:02.310 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:02.311152+0000 2024-01-24T23:24:02.310 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:02.310 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:02.310 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:02.310 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:02.310 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:02.310 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:02.310 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:02.310 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:02.311 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:02.311 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:02.311 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:02.311 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:02.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.307+0000 7f131fda2640 1 Processor -- start 2024-01-24T23:24:02.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.307+0000 7f131fda2640 1 -- start start 2024-01-24T23:24:02.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.307+0000 7f131fda2640 10 monclient: init 2024-01-24T23:24:02.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.307+0000 7f131fda2640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:02.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.307+0000 7f131fda2640 10 monclient: _add_conns ranks=[3,7,5,0,6,1,8,4,2] 2024-01-24T23:24:02.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.307+0000 7f131fda2640 10 monclient(hunting): picked mon.noname-g con 0x7f1318136e50 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:02.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.307+0000 7f131fda2640 10 monclient(hunting): picked mon.noname-i con 0x7f13180a0290 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:02.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.307+0000 7f131fda2640 10 monclient(hunting): picked mon.noname-h con 0x7f13180a1600 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:02.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.307+0000 7f131fda2640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1318064700 con 0x7f1318136e50 2024-01-24T23:24:02.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.307+0000 7f131fda2640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1318137940 con 0x7f13180a1600 2024-01-24T23:24:02.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.307+0000 7f131fda2640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f131809f9c0 con 0x7f13180a0290 2024-01-24T23:24:02.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.307+0000 7f131fda2640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:02.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.307+0000 7f131fda2640 10 monclient(hunting): authenticate will time out at 1129.635756s 2024-01-24T23:24:02.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.307+0000 7f131e33f640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f13180a1600 0x7f131813f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:45970/0 (socket says 172.21.15.132:45970) 2024-01-24T23:24:02.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.307+0000 7f131d33d640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f13180a0290 0x7f131813bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:46254/0 (socket says 172.21.15.132:46254) 2024-01-24T23:24:02.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.307+0000 7f131e33f640 1 -- 172.21.15.132:0/1048690122 learned_addr learned my addr 172.21.15.132:0/1048690122 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:02.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.307+0000 7f131db3e640 1 --1- 172.21.15.132:0/1048690122 >> v1:172.21.15.132:6790/0 conn(0x7f1318136e50 0x7f1318137230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:57018/0 (socket says 172.21.15.132:57018) 2024-01-24T23:24:02.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 1 -- 172.21.15.132:0/1048690122 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 860760625 0 0) 0x7f1318137940 con 0x7f13180a1600 2024-01-24T23:24:02.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 10 monclient(hunting): _negotiate 2024-01-24T23:24:02.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:02.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:02.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 10 monclient(hunting): my global_id is 5378 2024-01-24T23:24:02.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 1 -- 172.21.15.132:0/1048690122 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1304003530 con 0x7f13180a1600 2024-01-24T23:24:02.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 1 -- 172.21.15.132:0/1048690122 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3939467212 0 0) 0x7f131809f9c0 con 0x7f13180a0290 2024-01-24T23:24:02.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 10 monclient(hunting): _negotiate 2024-01-24T23:24:02.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:02.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:02.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 10 monclient(hunting): my global_id is 5425 2024-01-24T23:24:02.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 1 -- 172.21.15.132:0/1048690122 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1318137940 con 0x7f13180a0290 2024-01-24T23:24:02.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 1 -- 172.21.15.132:0/1048690122 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1001499636 0 0) 0x7f1318064700 con 0x7f1318136e50 2024-01-24T23:24:02.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 10 monclient(hunting): _negotiate 2024-01-24T23:24:02.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:02.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:02.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 10 monclient(hunting): my global_id is 5367 2024-01-24T23:24:02.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 1 -- 172.21.15.132:0/1048690122 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f131809f9c0 con 0x7f1318136e50 2024-01-24T23:24:02.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 1 -- 172.21.15.132:0/1048690122 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3613402461 0 0) 0x7f1304003530 con 0x7f13180a1600 2024-01-24T23:24:02.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 1 -- 172.21.15.132:0/1048690122 >> v1:172.21.15.132:6792/0 conn(0x7f13180a0290 legacy=0x7f131813bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:02.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 1 -- 172.21.15.132:0/1048690122 >> v1:172.21.15.132:6790/0 conn(0x7f1318136e50 legacy=0x7f1318137230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:02.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 10 monclient: _finish_hunting 0 2024-01-24T23:24:02.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 1 monclient: found mon.noname-h 2024-01-24T23:24:02.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:02.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:24:02.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 1 -- 172.21.15.132:0/1048690122 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131809eda0 con 0x7f13180a1600 2024-01-24T23:24:02.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 10 monclient: _finish_auth 0 2024-01-24T23:24:02.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 10 monclient: _check_auth_tickets 2024-01-24T23:24:02.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:02.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 1 -- 172.21.15.132:0/1048690122 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1310001b40 con 0x7f13180a1600 2024-01-24T23:24:02.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:02.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:02.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 10 monclient: dump: 2024-01-24T23:24:02.321 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:02.321 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:02.321 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:02.321 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:02.321 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:02.322 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:02.322 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:02.322 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:02.322 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:02.322 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:02.322 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:02.322 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:02.322 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:02.322 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:02.322 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:02.322 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:02.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 5 monclient: authenticate success, global_id 5378 2024-01-24T23:24:02.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:02.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 1 -- 172.21.15.132:0/1048690122 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1310001d60 con 0x7f13180a1600 2024-01-24T23:24:02.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:02.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:02.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 4 set_mon_vals no callback set 2024-01-24T23:24:02.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 1 -- 172.21.15.132:0/1048690122 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1310004f70 con 0x7f13180a1600 2024-01-24T23:24:02.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:02.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:02.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131cb3c640 10 monclient: dump: 2024-01-24T23:24:02.323 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:02.323 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:02.323 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:02.323 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:02.323 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:02.323 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:02.323 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:02.323 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:02.323 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:02.323 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:02.323 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:02.324 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:02.324 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:02.324 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:02.324 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:02.324 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:02.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 10 monclient: shutdown 2024-01-24T23:24:02.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:02.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 1 -- 172.21.15.132:0/1048690122 >> v1:172.21.15.132:6791/0 conn(0x7f13180a1600 legacy=0x7f131813f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:02.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 1 -- 172.21.15.132:0/1048690122 shutdown_connections 2024-01-24T23:24:02.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 1 -- 172.21.15.132:0/1048690122 >> 172.21.15.132:0/1048690122 conn(0x7f13180a5a10 msgr2=0x7f13180a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:02.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 1 -- 172.21.15.132:0/1048690122 shutdown_connections 2024-01-24T23:24:02.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 1 -- 172.21.15.132:0/1048690122 wait complete. 2024-01-24T23:24:02.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 10 monclient: build_initial_monmap 2024-01-24T23:24:02.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 10 monclient: monmap: 2024-01-24T23:24:02.325 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:02.325 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:02.325 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:02.325 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:02.325 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:02.325 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:02.325 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:02.325 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:02.325 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:02.325 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:02.325 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:02.325 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:02.325 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:02.326 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:02.326 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:02.326 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:02.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 1 Processor -- start 2024-01-24T23:24:02.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 1 -- start start 2024-01-24T23:24:02.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 10 monclient: init 2024-01-24T23:24:02.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:02.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 10 monclient: _add_conns ranks=[8,2,4,6,5,1,7,0,3] 2024-01-24T23:24:02.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 10 monclient(hunting): picked mon.noname-e con 0x7f13180a0290 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:02.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 10 monclient(hunting): picked mon.noname-b con 0x7f13180a1600 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:02.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 10 monclient(hunting): picked mon.noname-c con 0x7f1318136e50 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:02.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f131809ea70 con 0x7f13180a1600 2024-01-24T23:24:02.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13181220c0 con 0x7f1318136e50 2024-01-24T23:24:02.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1318120ba0 con 0x7f13180a0290 2024-01-24T23:24:02.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:02.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131fda2640 10 monclient(hunting): authenticate will time out at 1129.639005s 2024-01-24T23:24:02.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131e33f640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f1318136e50 legacy=0x7f13181d3a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:02.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131db3e640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f13180a0290 0x7f13181349e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:47310/0 (socket says 172.21.15.132:47310) 2024-01-24T23:24:02.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f131db3e640 1 -- 172.21.15.132:0/391870391 learned_addr learned my addr 172.21.15.132:0/391870391 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:02.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f130e7fc640 1 -- 172.21.15.132:0/391870391 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3014802954 0 0) 0x7f131809ea70 con 0x7f13180a1600 2024-01-24T23:24:02.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f130e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:02.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f130e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:02.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f130e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:02.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f130e7fc640 10 monclient(hunting): my global_id is 84683 2024-01-24T23:24:02.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f130e7fc640 1 -- 172.21.15.132:0/391870391 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12fc003530 con 0x7f13180a1600 2024-01-24T23:24:02.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f130e7fc640 1 -- 172.21.15.132:0/391870391 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1555095408 0 0) 0x7f1318120ba0 con 0x7f13180a0290 2024-01-24T23:24:02.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f130e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:02.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f130e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:02.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f130e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:02.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f130e7fc640 10 monclient(hunting): my global_id is 84563 2024-01-24T23:24:02.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.311+0000 7f130e7fc640 1 -- 172.21.15.132:0/391870391 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f131809ea70 con 0x7f13180a0290 2024-01-24T23:24:02.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 1 -- 172.21.15.132:0/391870391 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4164784962 0 0) 0x7f12fc003530 con 0x7f13180a1600 2024-01-24T23:24:02.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 1 -- 172.21.15.132:0/391870391 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1318120ba0 con 0x7f13180a1600 2024-01-24T23:24:02.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 1 -- 172.21.15.132:0/391870391 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1308001980 con 0x7f13180a1600 2024-01-24T23:24:02.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:02.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:24:02.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 10 monclient(hunting): dump: 2024-01-24T23:24:02.328 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:02.328 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:02.328 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:02.328 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:02.328 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:02.328 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:02.329 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:02.329 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:02.329 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:02.329 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:02.329 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:02.329 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:02.329 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:02.329 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:02.329 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:02.329 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:02.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:02.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 1 -- 172.21.15.132:0/391870391 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3888067345 0 0) 0x7f131809ea70 con 0x7f13180a0290 2024-01-24T23:24:02.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 1 -- 172.21.15.132:0/391870391 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f12fc003530 con 0x7f13180a0290 2024-01-24T23:24:02.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 1 -- 172.21.15.132:0/391870391 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f13140012e0 con 0x7f13180a0290 2024-01-24T23:24:02.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:02.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:24:02.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 10 monclient(hunting): dump: 2024-01-24T23:24:02.330 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:02.330 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:02.330 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:02.330 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:02.330 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:02.330 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:02.330 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:02.330 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:02.330 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:02.330 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:02.330 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:02.330 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:02.331 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:02.331 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:02.331 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:02.331 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:02.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 1 -- 172.21.15.132:0/391870391 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3785887197 0 0) 0x7f1318120ba0 con 0x7f13180a1600 2024-01-24T23:24:02.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 1 -- 172.21.15.132:0/391870391 >> v1:172.21.15.115:6793/0 conn(0x7f13180a0290 legacy=0x7f13181349e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:02.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 1 -- 172.21.15.132:0/391870391 >> v1:172.21.15.115:6791/0 conn(0x7f1318136e50 legacy=0x7f13181d3a50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:02.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:24:02.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 1 monclient: found mon.b 2024-01-24T23:24:02.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:02.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:02.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 1 -- 172.21.15.132:0/391870391 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1318136c20 con 0x7f13180a1600 2024-01-24T23:24:02.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 10 monclient: _finish_auth 0 2024-01-24T23:24:02.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:24:02.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:02.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f131fda2640 5 monclient: authenticate success, global_id 84683 2024-01-24T23:24:02.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f131fda2640 10 monclient: _renew_subs 2024-01-24T23:24:02.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f131fda2640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:02.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f131fda2640 1 -- 172.21.15.132:0/391870391 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1318121de0 con 0x7f13180a1600 2024-01-24T23:24:02.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f131fda2640 10 monclient: _renew_subs 2024-01-24T23:24:02.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f131fda2640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:02.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f131fda2640 1 -- 172.21.15.132:0/391870391 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f13181cb2c0 con 0x7f13180a1600 2024-01-24T23:24:02.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 1 -- 172.21.15.132:0/391870391 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1308001cb0 con 0x7f13180a1600 2024-01-24T23:24:02.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:02.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 1 -- 172.21.15.132:0/391870391 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1308004de0 con 0x7f13180a1600 2024-01-24T23:24:02.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:02.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130ffff640 4 set_mon_vals no callback set 2024-01-24T23:24:02.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:24:02.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 10 monclient: dump: 2024-01-24T23:24:02.332 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:02.333 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:02.333 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:02.333 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:02.333 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:02.333 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:02.333 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:02.333 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:02.333 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:02.333 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:02.333 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:02.333 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:02.333 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:02.333 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:02.333 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:02.334 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:02.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f131fda2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:02.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f131fda2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:02.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f131fda2640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:02.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f131fda2640 1 -- 172.21.15.132:0/391870391 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1300005090 con 0x7f13180a1600 2024-01-24T23:24:02.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 1 -- 172.21.15.132:0/391870391 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f130800f860 con 0x7f13180a1600 2024-01-24T23:24:02.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f130e7fc640 1 -- 172.21.15.132:0/391870391 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(72..72 src has 1..72) v4 ==== 5377+0+0 (unknown 2062928299 0 0) 0x7f1308043d30 con 0x7f13180a1600 2024-01-24T23:24:02.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.315+0000 7f131db3e640 10 monclient: get_auth_request con 0x7f12fc0376a0 auth_method 0 2024-01-24T23:24:02.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.327+0000 7f130e7fc640 1 -- 172.21.15.132:0/391870391 <== mon.2 v1:172.21.15.115:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f13080146c0 con 0x7f13180a1600 2024-01-24T23:24:02.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.327+0000 7f130e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:02.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.327+0000 7f130e7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:02.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.487+0000 7f131fda2640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-01-24T23:24:02.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.487+0000 7f131fda2640 10 monclient: _send_command 2 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-01-24T23:24:02.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.487+0000 7f131fda2640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:02.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.487+0000 7f131fda2640 1 -- 172.21.15.132:0/391870391 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"} v 0) v1 -- 0x7f1300005b50 con 0x7f13180a1600 2024-01-24T23:24:02.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.491+0000 7f130e7fc640 1 -- 172.21.15.132:0/391870391 <== mon.2 v1:172.21.15.115:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}]=0 item -11 does not appear in that position v72) v1 ==== 141+0+0 (unknown 1861260197 0 0) 0x7f13080144e0 con 0x7f13180a1600 2024-01-24T23:24:02.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.491+0000 7f130e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-01-24T23:24:02.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.491+0000 7f130e7fc640 10 monclient: _finish_command 2 = system:0 item -11 does not appear in that position 2024-01-24T23:24:02.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.587+0000 7f131fda2640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-01-24T23:24:02.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.587+0000 7f131fda2640 10 monclient: _send_command 3 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-01-24T23:24:02.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.587+0000 7f131fda2640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:02.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.587+0000 7f131fda2640 1 -- 172.21.15.132:0/391870391 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"} v 0) v1 -- 0x7f13000010a0 con 0x7f13180a1600 2024-01-24T23:24:02.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.591+0000 7f130e7fc640 1 -- 172.21.15.132:0/391870391 <== mon.2 v1:172.21.15.115:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}]=0 item -11 does not appear in that position v72) v1 ==== 141+0+0 (unknown 1861260197 0 0) 0x7f1308018c10 con 0x7f13180a1600 2024-01-24T23:24:02.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.591+0000 7f130e7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-01-24T23:24:02.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.591+0000 7f130e7fc640 10 monclient: _finish_command 3 = system:0 item -11 does not appear in that position 2024-01-24T23:24:02.592 INFO:tasks.workunit.client.0.smithi132.stderr:item -11 does not appear in that position 2024-01-24T23:24:02.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.591+0000 7f131fda2640 1 -- 172.21.15.132:0/391870391 >> v1:172.21.15.132:6812/1835401922 conn(0x7f12fc0376a0 legacy=0x7f12fc039b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:02.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.591+0000 7f131fda2640 10 monclient: shutdown 2024-01-24T23:24:02.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.591+0000 7f131fda2640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:02.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.591+0000 7f131fda2640 1 -- 172.21.15.132:0/391870391 >> v1:172.21.15.115:6790/0 conn(0x7f13180a1600 legacy=0x7f13181d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:02.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.591+0000 7f131fda2640 1 -- 172.21.15.132:0/391870391 shutdown_connections 2024-01-24T23:24:02.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.591+0000 7f131fda2640 1 -- 172.21.15.132:0/391870391 >> 172.21.15.132:0/391870391 conn(0x7f13180a5a10 msgr2=0x7f1318071210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:02.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.591+0000 7f131fda2640 1 -- 172.21.15.132:0/391870391 shutdown_connections 2024-01-24T23:24:02.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.591+0000 7f131fda2640 1 -- 172.21.15.132:0/391870391 wait complete. 2024-01-24T23:24:02.609 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd tree 2024-01-24T23:24:02.609 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep -c host1 2024-01-24T23:24:02.609 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep -q 1 2024-01-24T23:24:02.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.719+0000 7f8de0a93640 10 monclient: get_monmap_and_config 2024-01-24T23:24:02.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.719+0000 7f8de0a93640 10 monclient: build_initial_monmap 2024-01-24T23:24:02.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.719+0000 7f8de0a93640 10 monclient: monmap: 2024-01-24T23:24:02.723 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:02.723 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:02.723 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:02.724707+0000 2024-01-24T23:24:02.724 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:02.724707+0000 2024-01-24T23:24:02.724 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:02.724 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:02.724 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:02.724 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:02.724 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:02.724 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:02.724 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:02.724 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:02.724 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:02.724 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:02.724 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:02.724 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:02.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 1 Processor -- start 2024-01-24T23:24:02.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 1 -- start start 2024-01-24T23:24:02.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 10 monclient: init 2024-01-24T23:24:02.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:02.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 10 monclient: _add_conns ranks=[8,2,6,5,1,7,0,3,4] 2024-01-24T23:24:02.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 10 monclient(hunting): picked mon.noname-e con 0x7f8ddc09ff20 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:02.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 10 monclient(hunting): picked mon.noname-b con 0x7f8ddc09f9c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:02.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 10 monclient(hunting): picked mon.noname-d con 0x7f8ddc13b2c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:02.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ddc064700 con 0x7f8ddc09f9c0 2024-01-24T23:24:02.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ddc09e760 con 0x7f8ddc13b2c0 2024-01-24T23:24:02.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ddc0a1160 con 0x7f8ddc09ff20 2024-01-24T23:24:02.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:02.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 10 monclient(hunting): authenticate will time out at 1130.049393s 2024-01-24T23:24:02.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8ddad9d640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f8ddc13b2c0 0x7f8ddc13d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:55260/0 (socket says 172.21.15.132:55260) 2024-01-24T23:24:02.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8ddad9d640 1 -- 172.21.15.132:0/1857859571 learned_addr learned my addr 172.21.15.132:0/1857859571 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:02.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 1 -- 172.21.15.132:0/1857859571 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 463116916 0 0) 0x7f8ddc0a1160 con 0x7f8ddc09ff20 2024-01-24T23:24:02.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 10 monclient(hunting): _negotiate 2024-01-24T23:24:02.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:02.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:02.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 10 monclient(hunting): my global_id is 84572 2024-01-24T23:24:02.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 1 -- 172.21.15.132:0/1857859571 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dc4003530 con 0x7f8ddc09ff20 2024-01-24T23:24:02.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 1 -- 172.21.15.132:0/1857859571 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 230922206 0 0) 0x7f8ddc09e760 con 0x7f8ddc13b2c0 2024-01-24T23:24:02.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 10 monclient(hunting): _negotiate 2024-01-24T23:24:02.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:02.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:02.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 10 monclient(hunting): my global_id is 5415 2024-01-24T23:24:02.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 1 -- 172.21.15.132:0/1857859571 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ddc0a1160 con 0x7f8ddc13b2c0 2024-01-24T23:24:02.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 1 -- 172.21.15.132:0/1857859571 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2055825860 0 0) 0x7f8ddc064700 con 0x7f8ddc09f9c0 2024-01-24T23:24:02.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 10 monclient(hunting): _negotiate 2024-01-24T23:24:02.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:02.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:02.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 10 monclient(hunting): my global_id is 84692 2024-01-24T23:24:02.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 1 -- 172.21.15.132:0/1857859571 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ddc09e760 con 0x7f8ddc09f9c0 2024-01-24T23:24:02.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 1 -- 172.21.15.132:0/1857859571 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 12773730 0 0) 0x7f8dc4003530 con 0x7f8ddc09ff20 2024-01-24T23:24:02.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 1 -- 172.21.15.132:0/1857859571 >> v1:172.21.15.115:6792/0 conn(0x7f8ddc13b2c0 legacy=0x7f8ddc13d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:02.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 1 -- 172.21.15.132:0/1857859571 >> v1:172.21.15.115:6790/0 conn(0x7f8ddc09f9c0 legacy=0x7f8ddc139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:02.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 10 monclient: _finish_hunting 0 2024-01-24T23:24:02.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 1 monclient: found mon.noname-e 2024-01-24T23:24:02.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:02.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:24:02.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 1 -- 172.21.15.132:0/1857859571 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc0a0a10 con 0x7f8ddc09ff20 2024-01-24T23:24:02.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 10 monclient: _finish_auth 0 2024-01-24T23:24:02.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 10 monclient: _check_auth_tickets 2024-01-24T23:24:02.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:02.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 5 monclient: authenticate success, global_id 84572 2024-01-24T23:24:02.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 1 -- 172.21.15.132:0/1857859571 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8dcc001800 con 0x7f8ddc09ff20 2024-01-24T23:24:02.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:02.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:02.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:24:02.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 10 monclient: dump: 2024-01-24T23:24:02.736 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:02.736 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:02.736 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:02.736 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:02.736 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:02.736 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:02.736 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:02.736 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:02.737 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:02.737 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:02.737 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:02.737 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:02.737 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:02.737 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:02.737 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:02.738 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:02.738 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:02.738 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 1 -- 172.21.15.132:0/1857859571 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8dcc001a20 con 0x7f8ddc09ff20 2024-01-24T23:24:02.738 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:02.738 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:02.738 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 4 set_mon_vals no callback set 2024-01-24T23:24:02.738 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 1 -- 172.21.15.132:0/1857859571 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8dcc003960 con 0x7f8ddc09ff20 2024-01-24T23:24:02.738 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:02.738 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:24:02.739 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8dd959a640 10 monclient: dump: 2024-01-24T23:24:02.739 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:02.739 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:02.739 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:02.739 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:02.739 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:02.739 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:02.739 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:02.739 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:02.739 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:02.739 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:02.739 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:02.739 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:02.739 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:02.739 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:02.740 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:02.740 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:02.740 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 10 monclient: shutdown 2024-01-24T23:24:02.740 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:02.740 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 1 -- 172.21.15.132:0/1857859571 >> v1:172.21.15.115:6793/0 conn(0x7f8ddc09ff20 legacy=0x7f8ddc0a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:02.740 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 1 -- 172.21.15.132:0/1857859571 shutdown_connections 2024-01-24T23:24:02.740 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 1 -- 172.21.15.132:0/1857859571 >> 172.21.15.132:0/1857859571 conn(0x7f8ddc0a5a10 msgr2=0x7f8ddc0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:02.740 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 1 -- 172.21.15.132:0/1857859571 shutdown_connections 2024-01-24T23:24:02.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.723+0000 7f8de0a93640 1 -- 172.21.15.132:0/1857859571 wait complete. 2024-01-24T23:24:02.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 10 monclient: build_initial_monmap 2024-01-24T23:24:02.741 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 10 monclient: monmap: 2024-01-24T23:24:02.741 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:02.741 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:02.742 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:02.742 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:02.742 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:02.742 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:02.742 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:02.742 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:02.742 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:02.743 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:02.743 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:02.743 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:02.743 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:02.743 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:02.743 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:02.743 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:02.743 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 1 Processor -- start 2024-01-24T23:24:02.743 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 1 -- start start 2024-01-24T23:24:02.743 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 10 monclient: init 2024-01-24T23:24:02.743 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:02.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 10 monclient: _add_conns ranks=[5,8,7,3,1,0,6,4,2] 2024-01-24T23:24:02.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 10 monclient(hunting): picked mon.noname-h con 0x7f8ddc09f9c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:02.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 10 monclient(hunting): picked mon.noname-e con 0x7f8ddc09ff20 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:02.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 10 monclient(hunting): picked mon.noname-i con 0x7f8ddc13b2c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:02.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ddc11b990 con 0x7f8ddc09f9c0 2024-01-24T23:24:02.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ddc071960 con 0x7f8ddc13b2c0 2024-01-24T23:24:02.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ddc06f4b0 con 0x7f8ddc09ff20 2024-01-24T23:24:02.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:02.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 10 monclient(hunting): authenticate will time out at 1130.053215s 2024-01-24T23:24:02.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dda59c640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f8ddc09f9c0 0x7f8ddc1c97e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:45982/0 (socket says 172.21.15.132:45982) 2024-01-24T23:24:02.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dda59c640 1 -- 172.21.15.132:0/3016143743 learned_addr learned my addr 172.21.15.132:0/3016143743 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:02.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 1 -- 172.21.15.132:0/3016143743 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 223561506 0 0) 0x7f8ddc11b990 con 0x7f8ddc09f9c0 2024-01-24T23:24:02.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:02.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:02.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:02.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient(hunting): my global_id is 5387 2024-01-24T23:24:02.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 1 -- 172.21.15.132:0/3016143743 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dbc003530 con 0x7f8ddc09f9c0 2024-01-24T23:24:02.746 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 1 -- 172.21.15.132:0/3016143743 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 937160803 0 0) 0x7f8ddc071960 con 0x7f8ddc13b2c0 2024-01-24T23:24:02.746 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:02.746 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:02.746 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:02.746 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient(hunting): my global_id is 5434 2024-01-24T23:24:02.746 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 1 -- 172.21.15.132:0/3016143743 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ddc11b990 con 0x7f8ddc13b2c0 2024-01-24T23:24:02.746 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 1 -- 172.21.15.132:0/3016143743 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3460748590 0 0) 0x7f8ddc06f4b0 con 0x7f8ddc09ff20 2024-01-24T23:24:02.746 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:02.746 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:02.746 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:02.746 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient(hunting): my global_id is 84581 2024-01-24T23:24:02.746 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 1 -- 172.21.15.132:0/3016143743 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ddc071960 con 0x7f8ddc09ff20 2024-01-24T23:24:02.746 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 1 -- 172.21.15.132:0/3016143743 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 317901064 0 0) 0x7f8dbc003530 con 0x7f8ddc09f9c0 2024-01-24T23:24:02.747 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 1 -- 172.21.15.132:0/3016143743 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8ddc06f4b0 con 0x7f8ddc09f9c0 2024-01-24T23:24:02.747 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 1 -- 172.21.15.132:0/3016143743 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 845252594 0 0) 0x7f8ddc11b990 con 0x7f8ddc13b2c0 2024-01-24T23:24:02.747 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 1 -- 172.21.15.132:0/3016143743 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8dbc003530 con 0x7f8ddc13b2c0 2024-01-24T23:24:02.747 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 1 -- 172.21.15.132:0/3016143743 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8dcc003940 con 0x7f8ddc09f9c0 2024-01-24T23:24:02.747 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:02.747 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:02.747 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient(hunting): dump: 2024-01-24T23:24:02.747 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:02.747 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:02.747 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:02.747 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:02.747 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:02.747 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:02.747 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:02.748 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:02.748 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:02.748 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:02.748 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:02.748 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:02.748 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:02.748 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:02.748 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:02.748 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:02.748 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:02.748 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 1 -- 172.21.15.132:0/3016143743 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8dd4001c80 con 0x7f8ddc13b2c0 2024-01-24T23:24:02.748 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:02.748 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:02.748 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient(hunting): dump: 2024-01-24T23:24:02.749 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:02.749 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:02.749 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:02.749 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:02.749 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:02.749 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:02.749 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:02.749 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:02.749 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:02.749 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:02.749 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:02.749 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:02.749 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:02.749 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:02.749 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:02.750 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:02.750 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 1 -- 172.21.15.132:0/3016143743 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2265274378 0 0) 0x7f8ddc06f4b0 con 0x7f8ddc09f9c0 2024-01-24T23:24:02.750 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 1 -- 172.21.15.132:0/3016143743 >> v1:172.21.15.115:6793/0 conn(0x7f8ddc09ff20 legacy=0x7f8ddc1ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:02.750 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 1 -- 172.21.15.132:0/3016143743 >> v1:172.21.15.132:6792/0 conn(0x7f8ddc13b2c0 legacy=0x7f8ddc1d1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:02.750 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:02.750 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 1 monclient: found mon.h 2024-01-24T23:24:02.750 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:02.750 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:02.750 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 1 -- 172.21.15.132:0/3016143743 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc070e70 con 0x7f8ddc09f9c0 2024-01-24T23:24:02.750 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient: _finish_auth 0 2024-01-24T23:24:02.750 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:02.750 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:02.750 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 5 monclient: authenticate success, global_id 5387 2024-01-24T23:24:02.750 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 10 monclient: _renew_subs 2024-01-24T23:24:02.750 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:02.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 1 -- 172.21.15.132:0/3016143743 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ddc11b6b0 con 0x7f8ddc09f9c0 2024-01-24T23:24:02.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 10 monclient: _renew_subs 2024-01-24T23:24:02.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:02.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 1 -- 172.21.15.132:0/3016143743 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ddc06fcb0 con 0x7f8ddc09f9c0 2024-01-24T23:24:02.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 1 -- 172.21.15.132:0/3016143743 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8dcc0013c0 con 0x7f8ddc09f9c0 2024-01-24T23:24:02.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:02.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 1 -- 172.21.15.132:0/3016143743 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8dcc003f20 con 0x7f8ddc09f9c0 2024-01-24T23:24:02.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:02.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dd8d99640 4 set_mon_vals no callback set 2024-01-24T23:24:02.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:02.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 10 monclient: dump: 2024-01-24T23:24:02.751 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:02.751 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:02.751 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:02.751 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:02.751 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:02.752 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:02.752 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:02.752 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:02.752 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:02.752 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:02.752 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:02.752 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:02.752 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:02.752 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:02.752 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:02.752 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:02.752 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:02.753 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:02.753 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:02.753 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 1 -- 172.21.15.132:0/3016143743 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f8dcc004190 con 0x7f8ddc09f9c0 2024-01-24T23:24:02.753 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8de0a93640 1 -- 172.21.15.132:0/3016143743 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc0005090 con 0x7f8ddc09f9c0 2024-01-24T23:24:02.753 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dcaffd640 1 -- 172.21.15.132:0/3016143743 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(72..72 src has 1..72) v4 ==== 5377+0+0 (unknown 2062928299 0 0) 0x7f8dcc043ea0 con 0x7f8ddc09f9c0 2024-01-24T23:24:02.753 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.727+0000 7f8dd9d9b640 10 monclient: get_auth_request con 0x7f8dbc037740 auth_method 0 2024-01-24T23:24:02.753 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.731+0000 7f8dcaffd640 1 -- 172.21.15.132:0/3016143743 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f8dcc014be0 con 0x7f8ddc09f9c0 2024-01-24T23:24:02.753 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.731+0000 7f8dcaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:02.753 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.731+0000 7f8dcaffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:02.899 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.895+0000 7f8de0a93640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:02.899 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.895+0000 7f8de0a93640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:02.899 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.895+0000 7f8de0a93640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:02.899 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.895+0000 7f8de0a93640 1 -- 172.21.15.132:0/3016143743 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f8dc0005b50 con 0x7f8ddc09f9c0 2024-01-24T23:24:02.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.899+0000 7f8dcaffd640 1 -- 172.21.15.132:0/3016143743 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v72) v1 ==== 56+0+1340 (unknown 1744850983 0 3560728506) 0x7f8dcc0147f0 con 0x7f8ddc09f9c0 2024-01-24T23:24:02.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.899+0000 7f8dcaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:02.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.899+0000 7f8dcaffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:24:02.994 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.991+0000 7f8de0a93640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:02.994 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.991+0000 7f8de0a93640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:02.994 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.991+0000 7f8de0a93640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:02.995 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.991+0000 7f8de0a93640 1 -- 172.21.15.132:0/3016143743 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f8dc00066c0 con 0x7f8ddc09f9c0 2024-01-24T23:24:02.995 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.991+0000 7f8dcaffd640 1 -- 172.21.15.132:0/3016143743 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v72) v1 ==== 56+0+1340 (unknown 1744850983 0 3560728506) 0x7f8dcc0147f0 con 0x7f8ddc09f9c0 2024-01-24T23:24:02.995 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.991+0000 7f8dcaffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:02.995 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.991+0000 7f8dcaffd640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:24:02.997 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.995+0000 7f8de0a93640 1 -- 172.21.15.132:0/3016143743 >> v1:172.21.15.132:6812/1835401922 conn(0x7f8dbc037740 legacy=0x7f8dbc039c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:02.997 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.995+0000 7f8de0a93640 10 monclient: shutdown 2024-01-24T23:24:02.998 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.995+0000 7f8de0a93640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:02.998 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.995+0000 7f8de0a93640 1 -- 172.21.15.132:0/3016143743 >> v1:172.21.15.132:6791/0 conn(0x7f8ddc09f9c0 legacy=0x7f8ddc1c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:02.998 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.995+0000 7f8de0a93640 1 -- 172.21.15.132:0/3016143743 shutdown_connections 2024-01-24T23:24:02.998 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.995+0000 7f8de0a93640 1 -- 172.21.15.132:0/3016143743 >> 172.21.15.132:0/3016143743 conn(0x7f8ddc0a5a10 msgr2=0x7f8ddc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:02.998 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.995+0000 7f8de0a93640 1 -- 172.21.15.132:0/3016143743 shutdown_connections 2024-01-24T23:24:02.998 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:02.995+0000 7f8de0a93640 1 -- 172.21.15.132:0/3016143743 wait complete. 2024-01-24T23:24:03.012 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd crush rm foo 2024-01-24T23:24:03.012 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:24:03.012 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm foo 2024-01-24T23:24:03.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.119+0000 7fcdb4bb8640 10 monclient: get_monmap_and_config 2024-01-24T23:24:03.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.119+0000 7fcdb4bb8640 10 monclient: build_initial_monmap 2024-01-24T23:24:03.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.119+0000 7fcdb4bb8640 10 monclient: monmap: 2024-01-24T23:24:03.122 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:03.122 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:03.122 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:03.122763+0000 2024-01-24T23:24:03.122 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:03.122763+0000 2024-01-24T23:24:03.122 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:03.122 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:03.122 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:03.122 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:03.122 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:03.122 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:03.122 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:03.122 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:03.123 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:03.123 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:03.123 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:03.123 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:03.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.119+0000 7fcdb4bb8640 1 Processor -- start 2024-01-24T23:24:03.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.119+0000 7fcdb4bb8640 1 -- start start 2024-01-24T23:24:03.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.119+0000 7fcdb4bb8640 10 monclient: init 2024-01-24T23:24:03.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.119+0000 7fcdb4bb8640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:03.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.119+0000 7fcdb4bb8640 10 monclient: _add_conns ranks=[8,3,1,4,2,5,0,6,7] 2024-01-24T23:24:03.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.119+0000 7fcdb4bb8640 10 monclient(hunting): picked mon.noname-e con 0x7fcdb0133b50 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:03.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.119+0000 7fcdb4bb8640 10 monclient(hunting): picked mon.noname-g con 0x7fcdb009ec20 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:03.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.119+0000 7fcdb4bb8640 10 monclient(hunting): picked mon.noname-f con 0x7fcdb013a1d0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:03.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.119+0000 7fcdb4bb8640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdb0064700 con 0x7fcdb013a1d0 2024-01-24T23:24:03.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdb009d9f0 con 0x7fcdb009ec20 2024-01-24T23:24:03.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdb009dc60 con 0x7fcdb0133b50 2024-01-24T23:24:03.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:03.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 10 monclient(hunting): authenticate will time out at 1130.448060s 2024-01-24T23:24:03.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdaed9d640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fcdb013a1d0 legacy=0x7fcdb013c5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:03.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdadd9b640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fcdb009ec20 0x7fcdb0138ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:57034/0 (socket says 172.21.15.132:57034) 2024-01-24T23:24:03.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdadd9b640 1 -- 172.21.15.132:0/4152088394 learned_addr learned my addr 172.21.15.132:0/4152088394 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:03.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 1 -- 172.21.15.132:0/4152088394 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3364350204 0 0) 0x7fcdb009d9f0 con 0x7fcdb009ec20 2024-01-24T23:24:03.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 10 monclient(hunting): _negotiate 2024-01-24T23:24:03.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:03.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:03.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 10 monclient(hunting): my global_id is 5376 2024-01-24T23:24:03.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 1 -- 172.21.15.132:0/4152088394 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd98003530 con 0x7fcdb009ec20 2024-01-24T23:24:03.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 1 -- 172.21.15.132:0/4152088394 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 335391945 0 0) 0x7fcd98003530 con 0x7fcdb009ec20 2024-01-24T23:24:03.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 1 -- 172.21.15.132:0/4152088394 >> v1:172.21.15.115:6793/0 conn(0x7fcdb0133b50 legacy=0x7fcdb0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:03.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 1 -- 172.21.15.132:0/4152088394 >> v1:172.21.15.132:6789/0 conn(0x7fcdb013a1d0 legacy=0x7fcdb013c5c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:03.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 10 monclient: _finish_hunting 0 2024-01-24T23:24:03.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 1 monclient: found mon.noname-g 2024-01-24T23:24:03.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:03.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:24:03.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 1 -- 172.21.15.132:0/4152088394 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb009df40 con 0x7fcdb009ec20 2024-01-24T23:24:03.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 10 monclient: _finish_auth 0 2024-01-24T23:24:03.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 10 monclient: _check_auth_tickets 2024-01-24T23:24:03.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:03.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 1 -- 172.21.15.132:0/4152088394 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fcda40017c0 con 0x7fcdb009ec20 2024-01-24T23:24:03.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 5 monclient: authenticate success, global_id 5376 2024-01-24T23:24:03.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:03.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:03.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:03.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 10 monclient: dump: 2024-01-24T23:24:03.132 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:03.132 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:03.132 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.132 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.132 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:03.132 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:03.132 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:03.132 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:03.133 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:03.133 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:03.133 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:03.133 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:03.133 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:03.133 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:03.133 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:03.133 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:03.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:03.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 1 -- 172.21.15.132:0/4152088394 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fcda4001ae0 con 0x7fcdb009ec20 2024-01-24T23:24:03.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:03.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:03.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 4 set_mon_vals no callback set 2024-01-24T23:24:03.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 1 -- 172.21.15.132:0/4152088394 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fcda4004bf0 con 0x7fcdb009ec20 2024-01-24T23:24:03.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:03.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:03.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdad59a640 10 monclient: dump: 2024-01-24T23:24:03.134 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:03.134 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:03.134 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.134 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.134 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:03.134 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:03.134 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:03.134 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:03.134 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:03.134 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:03.134 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:03.134 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:03.134 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:03.134 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:03.135 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:03.135 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:03.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 10 monclient: shutdown 2024-01-24T23:24:03.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:03.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 1 -- 172.21.15.132:0/4152088394 >> v1:172.21.15.132:6790/0 conn(0x7fcdb009ec20 legacy=0x7fcdb0138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:03.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 1 -- 172.21.15.132:0/4152088394 shutdown_connections 2024-01-24T23:24:03.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 1 -- 172.21.15.132:0/4152088394 >> 172.21.15.132:0/4152088394 conn(0x7fcdb00a5a10 msgr2=0x7fcdb00a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:03.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 1 -- 172.21.15.132:0/4152088394 shutdown_connections 2024-01-24T23:24:03.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 1 -- 172.21.15.132:0/4152088394 wait complete. 2024-01-24T23:24:03.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 10 monclient: build_initial_monmap 2024-01-24T23:24:03.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 10 monclient: monmap: 2024-01-24T23:24:03.135 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:03.135 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:03.135 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:03.135 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:03.135 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:03.136 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:03.136 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:03.136 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:03.136 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:03.136 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:03.136 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:03.136 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:03.136 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:03.136 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:03.136 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:03.136 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:03.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 1 Processor -- start 2024-01-24T23:24:03.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 1 -- start start 2024-01-24T23:24:03.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 10 monclient: init 2024-01-24T23:24:03.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:03.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 10 monclient: _add_conns ranks=[5,0,7,8,3,2,4,1,6] 2024-01-24T23:24:03.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 10 monclient(hunting): picked mon.noname-h con 0x7fcdb009ec20 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:03.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 10 monclient(hunting): picked mon.noname-a con 0x7fcdb0133b50 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:03.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 10 monclient(hunting): picked mon.noname-i con 0x7fcdb013a1d0 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:03.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdb00a13b0 con 0x7fcdb0133b50 2024-01-24T23:24:03.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdb00a07e0 con 0x7fcdb009ec20 2024-01-24T23:24:03.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdb0135670 con 0x7fcdb013a1d0 2024-01-24T23:24:03.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:03.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 10 monclient(hunting): authenticate will time out at 1130.450915s 2024-01-24T23:24:03.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdadd9b640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fcdb0133b50 legacy=0x7fcdb00a00d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:03.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdae59c640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fcdb009ec20 0x7fcdb009f9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:45992/0 (socket says 172.21.15.132:45992) 2024-01-24T23:24:03.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdae59c640 1 -- 172.21.15.132:0/3668346129 learned_addr learned my addr 172.21.15.132:0/3668346129 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:03.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 1 -- 172.21.15.132:0/3668346129 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 389944302 0 0) 0x7fcdb00a07e0 con 0x7fcdb009ec20 2024-01-24T23:24:03.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:03.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:03.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:03.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 10 monclient(hunting): my global_id is 5396 2024-01-24T23:24:03.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 1 -- 172.21.15.132:0/3668346129 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd90003530 con 0x7fcdb009ec20 2024-01-24T23:24:03.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 1 -- 172.21.15.132:0/3668346129 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 270655490 0 0) 0x7fcdb0135670 con 0x7fcdb013a1d0 2024-01-24T23:24:03.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:03.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:03.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:03.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 10 monclient(hunting): my global_id is 5443 2024-01-24T23:24:03.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 1 -- 172.21.15.132:0/3668346129 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdb00a07e0 con 0x7fcdb013a1d0 2024-01-24T23:24:03.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 1 -- 172.21.15.132:0/3668346129 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1504769491 0 0) 0x7fcd90003530 con 0x7fcdb009ec20 2024-01-24T23:24:03.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 1 -- 172.21.15.132:0/3668346129 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcdb0135670 con 0x7fcdb009ec20 2024-01-24T23:24:03.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 1 -- 172.21.15.132:0/3668346129 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 712525633 0 0) 0x7fcdb00a07e0 con 0x7fcdb013a1d0 2024-01-24T23:24:03.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 1 -- 172.21.15.132:0/3668346129 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcd90003530 con 0x7fcdb013a1d0 2024-01-24T23:24:03.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 1 -- 172.21.15.132:0/3668346129 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fcda0001b20 con 0x7fcdb009ec20 2024-01-24T23:24:03.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:03.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:03.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 10 monclient(hunting): dump: 2024-01-24T23:24:03.139 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:03.139 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:03.139 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.139 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.139 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:03.139 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:03.139 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:03.139 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:03.139 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:03.139 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:03.139 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:03.139 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:03.140 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:03.140 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:03.140 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:03.140 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:03.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:03.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 1 -- 172.21.15.132:0/3668346129 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fcda80044e0 con 0x7fcdb013a1d0 2024-01-24T23:24:03.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:03.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:03.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 10 monclient(hunting): dump: 2024-01-24T23:24:03.140 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:03.140 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:03.140 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.140 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.140 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:03.141 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:03.141 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:03.141 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:03.141 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:03.141 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:03.141 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:03.141 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:03.141 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:03.141 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:03.141 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:03.141 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:03.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 1 -- 172.21.15.132:0/3668346129 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1148331998 0 0) 0x7fcdb0135670 con 0x7fcdb009ec20 2024-01-24T23:24:03.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 1 -- 172.21.15.132:0/3668346129 >> v1:172.21.15.132:6792/0 conn(0x7fcdb013a1d0 legacy=0x7fcdb01d15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:03.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 1 -- 172.21.15.132:0/3668346129 >> v1:172.21.15.115:6789/0 conn(0x7fcdb0133b50 legacy=0x7fcdb00a00d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:03.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:03.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 1 monclient: found mon.h 2024-01-24T23:24:03.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:03.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:03.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 1 -- 172.21.15.132:0/3668346129 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb0136e00 con 0x7fcdb009ec20 2024-01-24T23:24:03.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 10 monclient: _finish_auth 0 2024-01-24T23:24:03.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:03.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcd9effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:03.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 5 monclient: authenticate success, global_id 5396 2024-01-24T23:24:03.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 10 monclient: _renew_subs 2024-01-24T23:24:03.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:03.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 1 -- 172.21.15.132:0/3668346129 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb0136840 con 0x7fcdb009ec20 2024-01-24T23:24:03.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 10 monclient: _renew_subs 2024-01-24T23:24:03.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:03.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.123+0000 7fcdb4bb8640 1 -- 172.21.15.132:0/3668346129 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb01353f0 con 0x7fcdb009ec20 2024-01-24T23:24:03.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.127+0000 7fcdb4bb8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:03.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.127+0000 7fcdb4bb8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:03.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.127+0000 7fcdb4bb8640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:03.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.127+0000 7fcdb4bb8640 1 -- 172.21.15.132:0/3668346129 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb0067640 con 0x7fcdb009ec20 2024-01-24T23:24:03.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.127+0000 7fcd9effd640 1 -- 172.21.15.132:0/3668346129 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fcda0001d10 con 0x7fcdb009ec20 2024-01-24T23:24:03.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.127+0000 7fcd9effd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:03.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.127+0000 7fcd9effd640 1 -- 172.21.15.132:0/3668346129 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fcda0003c90 con 0x7fcdb009ec20 2024-01-24T23:24:03.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.127+0000 7fcd9effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:03.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.127+0000 7fcd9effd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:03.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.127+0000 7fcdacd99640 4 set_mon_vals no callback set 2024-01-24T23:24:03.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.127+0000 7fcd9effd640 10 monclient: dump: 2024-01-24T23:24:03.143 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:03.143 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:03.143 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.143 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.143 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:03.143 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:03.144 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:03.144 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:03.144 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:03.144 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:03.144 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:03.144 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:03.144 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:03.144 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:03.144 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:03.144 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:03.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.127+0000 7fcd9effd640 1 -- 172.21.15.132:0/3668346129 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fcda0003df0 con 0x7fcdb009ec20 2024-01-24T23:24:03.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.127+0000 7fcd9effd640 1 -- 172.21.15.132:0/3668346129 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(72..72 src has 1..72) v4 ==== 5377+0+0 (unknown 2062928299 0 0) 0x7fcda0043d20 con 0x7fcdb009ec20 2024-01-24T23:24:03.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.127+0000 7fcdadd9b640 10 monclient: get_auth_request con 0x7fcd900376f0 auth_method 0 2024-01-24T23:24:03.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.131+0000 7fcd9effd640 1 -- 172.21.15.132:0/3668346129 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fcda0014710 con 0x7fcdb009ec20 2024-01-24T23:24:03.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.131+0000 7fcd9effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:03.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.131+0000 7fcd9effd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:03.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.275+0000 7fcdb4bb8640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "foo"}] 2024-01-24T23:24:03.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.275+0000 7fcdb4bb8640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-01-24T23:24:03.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.275+0000 7fcdb4bb8640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:03.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.275+0000 7fcdb4bb8640 1 -- 172.21.15.132:0/3668346129 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "foo"} v 0) v1 -- 0x7fcdb0073840 con 0x7fcdb009ec20 2024-01-24T23:24:03.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.279+0000 7fcd9effd640 1 -- 172.21.15.132:0/3668346129 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "foo"}]=-39 (39) Directory not empty v72) v1 ==== 99+0+0 (unknown 3080862158 0 0) 0x7fcda0012db0 con 0x7fcdb009ec20 2024-01-24T23:24:03.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.279+0000 7fcd9effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-01-24T23:24:03.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.279+0000 7fcd9effd640 10 monclient: _finish_command 2 = mon:39 (39) Directory not empty 2024-01-24T23:24:03.281 INFO:tasks.workunit.client.0.smithi132.stderr:Error ENOTEMPTY: (39) Directory not empty 2024-01-24T23:24:03.287 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:24:03.287 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd crush rm bar 2024-01-24T23:24:03.287 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:24:03.287 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm bar 2024-01-24T23:24:03.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.399+0000 7fb4eeae0640 10 monclient: get_monmap_and_config 2024-01-24T23:24:03.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.399+0000 7fb4eeae0640 10 monclient: build_initial_monmap 2024-01-24T23:24:03.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.399+0000 7fb4eeae0640 10 monclient: monmap: 2024-01-24T23:24:03.402 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:03.402 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:03.402 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:03.403345+0000 2024-01-24T23:24:03.402 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:03.403345+0000 2024-01-24T23:24:03.402 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:03.402 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:03.403 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:03.403 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:03.403 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:03.403 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:03.403 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:03.403 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:03.403 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:03.403 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:03.403 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:03.403 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:03.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.399+0000 7fb4eeae0640 1 Processor -- start 2024-01-24T23:24:03.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 1 -- start start 2024-01-24T23:24:03.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 10 monclient: init 2024-01-24T23:24:03.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:03.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 10 monclient: _add_conns ranks=[2,1,8,3,7,6,5,0,4] 2024-01-24T23:24:03.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 10 monclient(hunting): picked mon.noname-b con 0x7fb4e80740e0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:03.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 10 monclient(hunting): picked mon.noname-f con 0x7fb4e8071830 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:03.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 10 monclient(hunting): picked mon.noname-e con 0x7fb4e8071c50 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:03.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e8064700 con 0x7fb4e8071830 2024-01-24T23:24:03.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e8070390 con 0x7fb4e80740e0 2024-01-24T23:24:03.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e80744c0 con 0x7fb4e8071c50 2024-01-24T23:24:03.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:03.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 10 monclient(hunting): authenticate will time out at 1130.728865s 2024-01-24T23:24:03.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e7fff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fb4e8071830 legacy=0x7fb4e8138a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:03.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4ec87c640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fb4e80740e0 0x7fb4e8133980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:42364/0 (socket says 172.21.15.132:42364) 2024-01-24T23:24:03.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4ec87c640 1 -- 172.21.15.132:0/2070139068 learned_addr learned my addr 172.21.15.132:0/2070139068 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:03.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 1 -- 172.21.15.132:0/2070139068 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1635821031 0 0) 0x7fb4e8070390 con 0x7fb4e80740e0 2024-01-24T23:24:03.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:24:03.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:03.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:03.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 10 monclient(hunting): my global_id is 84701 2024-01-24T23:24:03.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 1 -- 172.21.15.132:0/2070139068 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4d8003530 con 0x7fb4e80740e0 2024-01-24T23:24:03.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 1 -- 172.21.15.132:0/2070139068 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2420008837 0 0) 0x7fb4e80744c0 con 0x7fb4e8071c50 2024-01-24T23:24:03.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:24:03.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:03.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:03.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 10 monclient(hunting): my global_id is 84599 2024-01-24T23:24:03.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 1 -- 172.21.15.132:0/2070139068 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4e8070390 con 0x7fb4e8071c50 2024-01-24T23:24:03.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 1 -- 172.21.15.132:0/2070139068 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1799401043 0 0) 0x7fb4d8003530 con 0x7fb4e80740e0 2024-01-24T23:24:03.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 1 -- 172.21.15.132:0/2070139068 >> v1:172.21.15.115:6793/0 conn(0x7fb4e8071c50 legacy=0x7fb4e813c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:03.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 1 -- 172.21.15.132:0/2070139068 >> v1:172.21.15.132:6789/0 conn(0x7fb4e8071830 legacy=0x7fb4e8138a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:03.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 10 monclient: _finish_hunting 0 2024-01-24T23:24:03.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 1 monclient: found mon.noname-b 2024-01-24T23:24:03.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:03.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:24:03.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 1 -- 172.21.15.132:0/2070139068 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e806fa60 con 0x7fb4e80740e0 2024-01-24T23:24:03.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 10 monclient: _finish_auth 0 2024-01-24T23:24:03.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 10 monclient: _check_auth_tickets 2024-01-24T23:24:03.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:03.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 1 -- 172.21.15.132:0/2070139068 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb4dc0017c0 con 0x7fb4e80740e0 2024-01-24T23:24:03.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:03.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:24:03.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 10 monclient: dump: 2024-01-24T23:24:03.410 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:03.410 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:03.410 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.410 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.410 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:03.410 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:03.411 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:03.411 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:03.411 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:03.411 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:03.411 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:03.411 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:03.411 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:03.411 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:03.411 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:03.411 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:03.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 5 monclient: authenticate success, global_id 84701 2024-01-24T23:24:03.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:03.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 1 -- 172.21.15.132:0/2070139068 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb4dc001a30 con 0x7fb4e80740e0 2024-01-24T23:24:03.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:03.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:03.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 4 set_mon_vals no callback set 2024-01-24T23:24:03.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 1 -- 172.21.15.132:0/2070139068 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb4dc004bf0 con 0x7fb4e80740e0 2024-01-24T23:24:03.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:03.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:24:03.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4e77fe640 10 monclient: dump: 2024-01-24T23:24:03.412 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:03.412 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:03.412 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.412 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.412 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:03.412 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:03.412 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:03.412 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:03.413 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:03.413 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:03.413 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:03.413 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:03.413 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:03.413 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:03.413 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:03.413 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:03.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 10 monclient: shutdown 2024-01-24T23:24:03.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:03.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 1 -- 172.21.15.132:0/2070139068 >> v1:172.21.15.115:6790/0 conn(0x7fb4e80740e0 legacy=0x7fb4e8133980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:03.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 1 -- 172.21.15.132:0/2070139068 shutdown_connections 2024-01-24T23:24:03.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 1 -- 172.21.15.132:0/2070139068 >> 172.21.15.132:0/2070139068 conn(0x7fb4e80a5f40 msgr2=0x7fb4e80a2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:03.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 1 -- 172.21.15.132:0/2070139068 shutdown_connections 2024-01-24T23:24:03.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 1 -- 172.21.15.132:0/2070139068 wait complete. 2024-01-24T23:24:03.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 10 monclient: build_initial_monmap 2024-01-24T23:24:03.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.403+0000 7fb4eeae0640 10 monclient: monmap: 2024-01-24T23:24:03.414 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:03.414 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:03.414 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:03.414 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:03.414 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:03.414 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:03.414 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:03.414 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:03.414 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:03.414 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:03.414 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:03.414 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:03.414 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:03.415 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:03.415 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:03.415 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:03.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4eeae0640 1 Processor -- start 2024-01-24T23:24:03.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4eeae0640 1 -- start start 2024-01-24T23:24:03.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4eeae0640 10 monclient: init 2024-01-24T23:24:03.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4eeae0640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:03.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4eeae0640 10 monclient: _add_conns ranks=[6,0,3,5,8,2,1,7,4] 2024-01-24T23:24:03.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4eeae0640 10 monclient(hunting): picked mon.noname-d con 0x7fb4e8071830 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:03.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4eeae0640 10 monclient(hunting): picked mon.noname-a con 0x7fb4e8071c50 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:03.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4eeae0640 10 monclient(hunting): picked mon.noname-g con 0x7fb4e80740e0 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:03.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4eeae0640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e806efa0 con 0x7fb4e8071c50 2024-01-24T23:24:03.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4eeae0640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e806f560 con 0x7fb4e80740e0 2024-01-24T23:24:03.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4eeae0640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e8132620 con 0x7fb4e8071830 2024-01-24T23:24:03.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4eeae0640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:03.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4eeae0640 10 monclient(hunting): authenticate will time out at 1130.732691s 2024-01-24T23:24:03.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e7fff640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fb4e8071c50 legacy=0x7fb4e81c7490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:03.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4ed07d640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fb4e80740e0 0x7fb4e81cf3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:57036/0 (socket says 172.21.15.132:57036) 2024-01-24T23:24:03.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4ed07d640 1 -- 172.21.15.132:0/2125343840 learned_addr learned my addr 172.21.15.132:0/2125343840 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:03.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 1 -- 172.21.15.132:0/2125343840 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 889305166 0 0) 0x7fb4e8070390 con 0x7fb4e80740e0 2024-01-24T23:24:03.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 10 monclient(hunting): _negotiate 2024-01-24T23:24:03.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:03.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:03.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 10 monclient(hunting): my global_id is 5385 2024-01-24T23:24:03.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 1 -- 172.21.15.132:0/2125343840 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4cc003530 con 0x7fb4e80740e0 2024-01-24T23:24:03.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 1 -- 172.21.15.132:0/2125343840 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3675695552 0 0) 0x7fb4cc003530 con 0x7fb4e80740e0 2024-01-24T23:24:03.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 1 -- 172.21.15.132:0/2125343840 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb4e8070390 con 0x7fb4e80740e0 2024-01-24T23:24:03.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 1 -- 172.21.15.132:0/2125343840 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb4e0001e00 con 0x7fb4e80740e0 2024-01-24T23:24:03.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:03.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:03.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 10 monclient(hunting): dump: 2024-01-24T23:24:03.420 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:03.420 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:03.420 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.420 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.420 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:03.420 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:03.420 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:03.420 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:03.421 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:03.421 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:03.421 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:03.421 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:03.421 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:03.421 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:03.421 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:03.421 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:03.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:03.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 1 -- 172.21.15.132:0/2125343840 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1850708368 0 0) 0x7fb4e8132620 con 0x7fb4e8071830 2024-01-24T23:24:03.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 10 monclient(hunting): _negotiate 2024-01-24T23:24:03.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:03.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:03.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 10 monclient(hunting): my global_id is 5424 2024-01-24T23:24:03.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 1 -- 172.21.15.132:0/2125343840 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4cc003530 con 0x7fb4e8071830 2024-01-24T23:24:03.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 1 -- 172.21.15.132:0/2125343840 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2902687059 0 0) 0x7fb4e8070390 con 0x7fb4e80740e0 2024-01-24T23:24:03.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 1 -- 172.21.15.132:0/2125343840 >> v1:172.21.15.115:6792/0 conn(0x7fb4e8071830 legacy=0x7fb4e81c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:03.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 1 -- 172.21.15.132:0/2125343840 >> v1:172.21.15.115:6789/0 conn(0x7fb4e8071c50 legacy=0x7fb4e81c7490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:03.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 10 monclient: _finish_hunting 0 2024-01-24T23:24:03.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 1 monclient: found mon.g 2024-01-24T23:24:03.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:03.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:03.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 1 -- 172.21.15.132:0/2125343840 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e81c7ba0 con 0x7fb4e80740e0 2024-01-24T23:24:03.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 10 monclient: _finish_auth 0 2024-01-24T23:24:03.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 10 monclient: _check_auth_tickets 2024-01-24T23:24:03.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:03.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4eeae0640 5 monclient: authenticate success, global_id 5385 2024-01-24T23:24:03.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4eeae0640 10 monclient: _renew_subs 2024-01-24T23:24:03.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4eeae0640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:03.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4eeae0640 1 -- 172.21.15.132:0/2125343840 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e8131040 con 0x7fb4e80740e0 2024-01-24T23:24:03.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4eeae0640 10 monclient: _renew_subs 2024-01-24T23:24:03.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4eeae0640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:03.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4eeae0640 1 -- 172.21.15.132:0/2125343840 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e8132da0 con 0x7fb4e80740e0 2024-01-24T23:24:03.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 1 -- 172.21.15.132:0/2125343840 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb4e0003cd0 con 0x7fb4e80740e0 2024-01-24T23:24:03.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:03.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 1 -- 172.21.15.132:0/2125343840 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb4e0004200 con 0x7fb4e80740e0 2024-01-24T23:24:03.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:03.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e6ffd640 4 set_mon_vals no callback set 2024-01-24T23:24:03.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:03.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 10 monclient: dump: 2024-01-24T23:24:03.426 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:03.426 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:03.426 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.426 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.426 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:03.426 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:03.426 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:03.426 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:03.426 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:03.426 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:03.426 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:03.427 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:03.427 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:03.427 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:03.427 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:03.427 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:03.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4eeae0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:03.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4eeae0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:03.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4eeae0640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:03.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4eeae0640 1 -- 172.21.15.132:0/2125343840 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c8005090 con 0x7fb4e80740e0 2024-01-24T23:24:03.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 1 -- 172.21.15.132:0/2125343840 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fb4e00043e0 con 0x7fb4e80740e0 2024-01-24T23:24:03.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4e57fa640 1 -- 172.21.15.132:0/2125343840 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(72..72 src has 1..72) v4 ==== 5377+0+0 (unknown 2062928299 0 0) 0x7fb4e0044140 con 0x7fb4e80740e0 2024-01-24T23:24:03.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.407+0000 7fb4ec87c640 10 monclient: get_auth_request con 0x7fb4cc036f70 auth_method 0 2024-01-24T23:24:03.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.411+0000 7fb4e57fa640 1 -- 172.21.15.132:0/2125343840 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fb4e0014dc0 con 0x7fb4e80740e0 2024-01-24T23:24:03.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.411+0000 7fb4e57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:03.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.411+0000 7fb4e57fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:03.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.571+0000 7fb4eeae0640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "bar"}] 2024-01-24T23:24:03.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.571+0000 7fb4eeae0640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "bar"}] 2024-01-24T23:24:03.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.571+0000 7fb4eeae0640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:03.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.571+0000 7fb4eeae0640 1 -- 172.21.15.132:0/2125343840 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rm", "name": "bar"} v 0) v1 -- 0x7fb4c8005340 con 0x7fb4e80740e0 2024-01-24T23:24:03.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.571+0000 7fb4e57fa640 1 -- 172.21.15.132:0/2125343840 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "bar"}]=-39 (39) Directory not empty v72) v1 ==== 99+0+0 (unknown 378176449 0 0) 0x7fb4e0013230 con 0x7fb4e80740e0 2024-01-24T23:24:03.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.571+0000 7fb4e57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "bar"}] 2024-01-24T23:24:03.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.571+0000 7fb4e57fa640 10 monclient: _finish_command 2 = mon:39 (39) Directory not empty 2024-01-24T23:24:03.573 INFO:tasks.workunit.client.0.smithi132.stderr:Error ENOTEMPTY: (39) Directory not empty 2024-01-24T23:24:03.577 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:24:03.577 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush unlink host1 bar 2024-01-24T23:24:03.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 10 monclient: get_monmap_and_config 2024-01-24T23:24:03.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 10 monclient: build_initial_monmap 2024-01-24T23:24:03.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 10 monclient: monmap: 2024-01-24T23:24:03.696 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:03.696 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:03.696 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:03.697209+0000 2024-01-24T23:24:03.696 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:03.697209+0000 2024-01-24T23:24:03.696 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:03.696 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:03.696 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:03.697 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:03.697 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:03.697 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:03.697 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:03.697 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:03.697 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:03.697 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:03.697 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:03.697 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:03.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 1 Processor -- start 2024-01-24T23:24:03.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 1 -- start start 2024-01-24T23:24:03.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 10 monclient: init 2024-01-24T23:24:03.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:03.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 10 monclient: _add_conns ranks=[0,7,4,6,3,5,2,8,1] 2024-01-24T23:24:03.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 10 monclient(hunting): picked mon.noname-a con 0x7fd2ec133b50 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:03.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 10 monclient(hunting): picked mon.noname-i con 0x7fd2ec09ec20 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:03.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 10 monclient(hunting): picked mon.noname-c con 0x7fd2ec13a1d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:03.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ec064700 con 0x7fd2ec133b50 2024-01-24T23:24:03.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ec09d9f0 con 0x7fd2ec13a1d0 2024-01-24T23:24:03.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ec09dc60 con 0x7fd2ec09ec20 2024-01-24T23:24:03.704 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:03.704 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 10 monclient(hunting): authenticate will time out at 1131.021872s 2024-01-24T23:24:03.704 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eb7fe640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fd2ec133b50 legacy=0x7fd2ec133f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:03.704 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2e2ffd640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fd2ec09ec20 0x7fd2ec138ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:52248/0 (socket says 172.21.15.132:52248) 2024-01-24T23:24:03.704 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2ebfff640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7fd2ec13a1d0 legacy=0x7fd2ec13c5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:03.704 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2e2ffd640 1 -- 172.21.15.132:0/1257125245 learned_addr learned my addr 172.21.15.132:0/1257125245 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:03.704 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 1 -- 172.21.15.132:0/1257125245 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562558852 0 0) 0x7fd2ec09dc60 con 0x7fd2ec09ec20 2024-01-24T23:24:03.704 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:03.704 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:03.704 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:03.704 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 10 monclient(hunting): my global_id is 5452 2024-01-24T23:24:03.704 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 1 -- 172.21.15.132:0/1257125245 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2d8003530 con 0x7fd2ec09ec20 2024-01-24T23:24:03.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 1 -- 172.21.15.132:0/1257125245 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3483932305 0 0) 0x7fd2d8003530 con 0x7fd2ec09ec20 2024-01-24T23:24:03.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 1 -- 172.21.15.132:0/1257125245 >> v1:172.21.15.115:6791/0 conn(0x7fd2ec13a1d0 legacy=0x7fd2ec13c5c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:03.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 1 -- 172.21.15.132:0/1257125245 >> v1:172.21.15.115:6789/0 conn(0x7fd2ec133b50 legacy=0x7fd2ec133f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:03.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:03.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 1 monclient: found mon.noname-i 2024-01-24T23:24:03.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:03.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:24:03.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 1 -- 172.21.15.132:0/1257125245 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ec09df40 con 0x7fd2ec09ec20 2024-01-24T23:24:03.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 10 monclient: _finish_auth 0 2024-01-24T23:24:03.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:03.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:03.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 1 -- 172.21.15.132:0/1257125245 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd2d40017c0 con 0x7fd2ec09ec20 2024-01-24T23:24:03.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:03.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:03.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 10 monclient: dump: 2024-01-24T23:24:03.706 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:03.706 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:03.706 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.706 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.706 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:03.706 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:03.706 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:03.706 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:03.706 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:03.706 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:03.706 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:03.706 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:03.706 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:03.706 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:03.706 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:03.707 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:03.707 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 5 monclient: authenticate success, global_id 5452 2024-01-24T23:24:03.707 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:03.707 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 1 -- 172.21.15.132:0/1257125245 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd2d4001ae0 con 0x7fd2ec09ec20 2024-01-24T23:24:03.707 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:03.707 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 1 -- 172.21.15.132:0/1257125245 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd2d4004bf0 con 0x7fd2ec09ec20 2024-01-24T23:24:03.707 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:03.707 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:03.707 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2eaffd640 10 monclient: dump: 2024-01-24T23:24:03.707 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:03.707 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:03.707 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.707 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.707 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:03.708 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:03.708 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:03.708 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:03.708 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:03.708 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:03.708 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:03.708 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:03.708 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:03.708 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:03.708 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:03.708 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:03.708 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:03.708 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 4 set_mon_vals no callback set 2024-01-24T23:24:03.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 10 monclient: shutdown 2024-01-24T23:24:03.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:03.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 1 -- 172.21.15.132:0/1257125245 >> v1:172.21.15.132:6792/0 conn(0x7fd2ec09ec20 legacy=0x7fd2ec138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:03.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 1 -- 172.21.15.132:0/1257125245 shutdown_connections 2024-01-24T23:24:03.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 1 -- 172.21.15.132:0/1257125245 >> 172.21.15.132:0/1257125245 conn(0x7fd2ec0a5a10 msgr2=0x7fd2ec0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:03.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 1 -- 172.21.15.132:0/1257125245 shutdown_connections 2024-01-24T23:24:03.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.695+0000 7fd2f1eeb640 1 -- 172.21.15.132:0/1257125245 wait complete. 2024-01-24T23:24:03.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 10 monclient: build_initial_monmap 2024-01-24T23:24:03.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 10 monclient: monmap: 2024-01-24T23:24:03.709 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:03.709 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:03.709 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:03.709 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:03.709 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:03.709 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:03.709 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:03.710 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:03.710 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:03.710 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:03.710 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:03.710 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:03.710 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:03.710 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:03.710 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:03.710 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:03.710 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 1 Processor -- start 2024-01-24T23:24:03.710 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 1 -- start start 2024-01-24T23:24:03.710 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 10 monclient: init 2024-01-24T23:24:03.710 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:03.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 10 monclient: _add_conns ranks=[8,0,1,5,2,3,6,4,7] 2024-01-24T23:24:03.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 10 monclient(hunting): picked mon.noname-e con 0x7fd2ec09ec20 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:03.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 10 monclient(hunting): picked mon.noname-a con 0x7fd2ec133b50 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:03.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 10 monclient(hunting): picked mon.noname-f con 0x7fd2ec13a1d0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:03.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ec139010 con 0x7fd2ec133b50 2024-01-24T23:24:03.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ec133920 con 0x7fd2ec13a1d0 2024-01-24T23:24:03.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ec071e30 con 0x7fd2ec09ec20 2024-01-24T23:24:03.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:03.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 10 monclient(hunting): authenticate will time out at 1131.025032s 2024-01-24T23:24:03.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2ebfff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fd2ec13a1d0 legacy=0x7fd2ec1cf5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:03.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e2ffd640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fd2ec133b50 legacy=0x7fd2ec1c7a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:03.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2eb7fe640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fd2ec09ec20 0x7fd2ec1c7340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:55880/0 (socket says 172.21.15.132:55880) 2024-01-24T23:24:03.712 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2eb7fe640 1 -- 172.21.15.132:0/4219517389 learned_addr learned my addr 172.21.15.132:0/4219517389 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:03.712 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 1 -- 172.21.15.132:0/4219517389 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3744558276 0 0) 0x7fd2ec071e30 con 0x7fd2ec09ec20 2024-01-24T23:24:03.712 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:24:03.712 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:03.712 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:03.712 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 10 monclient(hunting): my global_id is 84608 2024-01-24T23:24:03.712 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 1 -- 172.21.15.132:0/4219517389 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2cc003560 con 0x7fd2ec09ec20 2024-01-24T23:24:03.712 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 1 -- 172.21.15.132:0/4219517389 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 927228314 0 0) 0x7fd2cc003560 con 0x7fd2ec09ec20 2024-01-24T23:24:03.712 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 1 -- 172.21.15.132:0/4219517389 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd2ec071e30 con 0x7fd2ec09ec20 2024-01-24T23:24:03.712 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 1 -- 172.21.15.132:0/4219517389 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd2dc004380 con 0x7fd2ec09ec20 2024-01-24T23:24:03.712 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:03.712 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:24:03.712 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 10 monclient(hunting): dump: 2024-01-24T23:24:03.712 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:03.713 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:03.713 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.713 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.713 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:03.713 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:03.713 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:03.713 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:03.713 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:03.713 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:03.713 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:03.713 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:03.713 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:03.713 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:03.713 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:03.713 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:03.713 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:03.713 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 1 -- 172.21.15.132:0/4219517389 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4286070191 0 0) 0x7fd2ec071e30 con 0x7fd2ec09ec20 2024-01-24T23:24:03.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 1 -- 172.21.15.132:0/4219517389 >> v1:172.21.15.132:6789/0 conn(0x7fd2ec13a1d0 legacy=0x7fd2ec1cf5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:03.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 1 -- 172.21.15.132:0/4219517389 >> v1:172.21.15.115:6789/0 conn(0x7fd2ec133b50 legacy=0x7fd2ec1c7a50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:03.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:24:03.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 1 monclient: found mon.e 2024-01-24T23:24:03.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:03.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:03.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 1 -- 172.21.15.132:0/4219517389 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ec0a13b0 con 0x7fd2ec09ec20 2024-01-24T23:24:03.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 10 monclient: _finish_auth 0 2024-01-24T23:24:03.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:24:03.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:03.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 5 monclient: authenticate success, global_id 84608 2024-01-24T23:24:03.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 10 monclient: _renew_subs 2024-01-24T23:24:03.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:03.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 1 -- 172.21.15.132:0/4219517389 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ec1c8160 con 0x7fd2ec09ec20 2024-01-24T23:24:03.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 10 monclient: _renew_subs 2024-01-24T23:24:03.715 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:03.715 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 1 -- 172.21.15.132:0/4219517389 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ec06e3e0 con 0x7fd2ec09ec20 2024-01-24T23:24:03.715 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 1 -- 172.21.15.132:0/4219517389 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd2dc0045a0 con 0x7fd2ec09ec20 2024-01-24T23:24:03.715 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:03.715 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 1 -- 172.21.15.132:0/4219517389 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd2dc004ad0 con 0x7fd2ec09ec20 2024-01-24T23:24:03.715 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:03.715 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2ea7fc640 4 set_mon_vals no callback set 2024-01-24T23:24:03.715 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:24:03.715 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2e8ff9640 10 monclient: dump: 2024-01-24T23:24:03.715 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:03.715 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:03.715 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.715 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:03.715 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:03.715 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:03.716 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:03.716 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:03.716 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:03.716 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:03.716 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:03.716 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:03.716 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:03.716 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:03.716 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:03.716 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:03.716 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:03.716 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:03.716 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:03.716 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.699+0000 7fd2f1eeb640 1 -- 172.21.15.132:0/4219517389 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d0005090 con 0x7fd2ec09ec20 2024-01-24T23:24:03.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.703+0000 7fd2e8ff9640 1 -- 172.21.15.132:0/4219517389 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fd2dc004cb0 con 0x7fd2ec09ec20 2024-01-24T23:24:03.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.703+0000 7fd2e8ff9640 1 -- 172.21.15.132:0/4219517389 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(72..72 src has 1..72) v4 ==== 5377+0+0 (unknown 2062928299 0 0) 0x7fd2dc043e20 con 0x7fd2ec09ec20 2024-01-24T23:24:03.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.703+0000 7fd2e2ffd640 10 monclient: get_auth_request con 0x7fd2cc036fb0 auth_method 0 2024-01-24T23:24:03.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.715+0000 7fd2e8ff9640 1 -- 172.21.15.132:0/4219517389 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fd2dc014a40 con 0x7fd2ec09ec20 2024-01-24T23:24:03.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.715+0000 7fd2e8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:03.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.715+0000 7fd2e8ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:03.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.875+0000 7fd2f1eeb640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}] 2024-01-24T23:24:03.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.875+0000 7fd2f1eeb640 10 monclient: _send_command 2 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}] 2024-01-24T23:24:03.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.875+0000 7fd2f1eeb640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:03.878 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:03.875+0000 7fd2f1eeb640 1 -- 172.21.15.132:0/4219517389 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"} v 0) v1 -- 0x7fd2d0005b50 con 0x7fd2ec09ec20 2024-01-24T23:24:04.087 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.083+0000 7fd2e8ff9640 1 -- 172.21.15.132:0/4219517389 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}]=0 removed item id -11 name 'host1' from crush map v73) v1 ==== 147+0+0 (unknown 2172881297 0 0) 0x7fd2dc012eb0 con 0x7fd2ec09ec20 2024-01-24T23:24:04.087 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.083+0000 7fd2e8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}] 2024-01-24T23:24:04.087 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.083+0000 7fd2e8ff9640 10 monclient: _finish_command 2 = system:0 removed item id -11 name 'host1' from crush map 2024-01-24T23:24:04.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.187+0000 7fd2f1eeb640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}] 2024-01-24T23:24:04.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.187+0000 7fd2f1eeb640 10 monclient: _send_command 3 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}] 2024-01-24T23:24:04.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.187+0000 7fd2f1eeb640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:04.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.187+0000 7fd2f1eeb640 1 -- 172.21.15.132:0/4219517389 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"} v 0) v1 -- 0x7fd2d00010a0 con 0x7fd2ec09ec20 2024-01-24T23:24:04.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.187+0000 7fd2e8ff9640 1 -- 172.21.15.132:0/4219517389 <== mon.8 v1:172.21.15.115:6793/0 11 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}]=0 item -11 does not appear in that position v73) v1 ==== 141+0+0 (unknown 193190843 0 0) 0x7fd2dc014630 con 0x7fd2ec09ec20 2024-01-24T23:24:04.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.187+0000 7fd2e8ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}] 2024-01-24T23:24:04.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.187+0000 7fd2e8ff9640 10 monclient: _finish_command 3 = system:0 item -11 does not appear in that position 2024-01-24T23:24:04.191 INFO:tasks.workunit.client.0.smithi132.stderr:item -11 does not appear in that position 2024-01-24T23:24:04.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.191+0000 7fd2f1eeb640 1 -- 172.21.15.132:0/4219517389 >> v1:172.21.15.132:6812/1835401922 conn(0x7fd2cc036fb0 legacy=0x7fd2cc039470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:04.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.191+0000 7fd2f1eeb640 10 monclient: shutdown 2024-01-24T23:24:04.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.191+0000 7fd2f1eeb640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:04.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.191+0000 7fd2f1eeb640 1 -- 172.21.15.132:0/4219517389 >> v1:172.21.15.115:6793/0 conn(0x7fd2ec09ec20 legacy=0x7fd2ec1c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:04.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.191+0000 7fd2f1eeb640 1 -- 172.21.15.132:0/4219517389 shutdown_connections 2024-01-24T23:24:04.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.191+0000 7fd2f1eeb640 1 -- 172.21.15.132:0/4219517389 >> 172.21.15.132:0/4219517389 conn(0x7fd2ec0a5a10 msgr2=0x7fd2ec066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:04.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.191+0000 7fd2f1eeb640 1 -- 172.21.15.132:0/4219517389 shutdown_connections 2024-01-24T23:24:04.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.191+0000 7fd2f1eeb640 1 -- 172.21.15.132:0/4219517389 wait complete. 2024-01-24T23:24:04.208 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd tree 2024-01-24T23:24:04.208 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep -q 1 2024-01-24T23:24:04.209 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep -c host1 2024-01-24T23:24:04.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.315+0000 7f328e75b640 10 monclient: get_monmap_and_config 2024-01-24T23:24:04.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.315+0000 7f328e75b640 10 monclient: build_initial_monmap 2024-01-24T23:24:04.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.315+0000 7f328e75b640 10 monclient: monmap: 2024-01-24T23:24:04.323 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:04.323 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:04.323 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:04.319155+0000 2024-01-24T23:24:04.323 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:04.319155+0000 2024-01-24T23:24:04.323 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:04.323 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:04.323 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:04.324 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:04.324 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:04.324 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:04.324 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:04.324 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:04.324 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:04.324 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:04.324 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:04.324 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:04.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.315+0000 7f328e75b640 1 Processor -- start 2024-01-24T23:24:04.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.315+0000 7f328e75b640 1 -- start start 2024-01-24T23:24:04.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.315+0000 7f328e75b640 10 monclient: init 2024-01-24T23:24:04.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.315+0000 7f328e75b640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:04.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.315+0000 7f328e75b640 10 monclient: _add_conns ranks=[2,0,1,7,6,5,4,3,8] 2024-01-24T23:24:04.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.315+0000 7f328e75b640 10 monclient(hunting): picked mon.noname-b con 0x7f328809ff20 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:04.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.315+0000 7f328e75b640 10 monclient(hunting): picked mon.noname-a con 0x7f328809f9c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:04.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.315+0000 7f328e75b640 10 monclient(hunting): picked mon.noname-f con 0x7f328813b2c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:04.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.315+0000 7f328e75b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3288064700 con 0x7f328809f9c0 2024-01-24T23:24:04.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.315+0000 7f328e75b640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f328809e760 con 0x7f328813b2c0 2024-01-24T23:24:04.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.315+0000 7f328e75b640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32880a1160 con 0x7f328809ff20 2024-01-24T23:24:04.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.315+0000 7f328e75b640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:04.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.315+0000 7f328e75b640 10 monclient(hunting): authenticate will time out at 1131.643715s 2024-01-24T23:24:04.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.315+0000 7f328ccf8640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f328813b2c0 legacy=0x7f328813d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:04.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.315+0000 7f32877fe640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f328809f9c0 legacy=0x7f3288139ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:04.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3287fff640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f328809ff20 0x7f32880a0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:48372/0 (socket says 172.21.15.132:48372) 2024-01-24T23:24:04.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3287fff640 1 -- 172.21.15.132:0/3948374013 learned_addr learned my addr 172.21.15.132:0/3948374013 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:04.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 1 -- 172.21.15.132:0/3948374013 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2447358016 0 0) 0x7f32880a1160 con 0x7f328809ff20 2024-01-24T23:24:04.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:04.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:04.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:04.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 10 monclient(hunting): my global_id is 84710 2024-01-24T23:24:04.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 1 -- 172.21.15.132:0/3948374013 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3274003530 con 0x7f328809ff20 2024-01-24T23:24:04.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 1 -- 172.21.15.132:0/3948374013 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3174011187 0 0) 0x7f3274003530 con 0x7f328809ff20 2024-01-24T23:24:04.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 1 -- 172.21.15.132:0/3948374013 >> v1:172.21.15.132:6789/0 conn(0x7f328813b2c0 legacy=0x7f328813d6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:04.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 1 -- 172.21.15.132:0/3948374013 >> v1:172.21.15.115:6789/0 conn(0x7f328809f9c0 legacy=0x7f3288139ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:04.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:04.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 1 monclient: found mon.noname-b 2024-01-24T23:24:04.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:04.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:24:04.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 1 -- 172.21.15.132:0/3948374013 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32880a0a10 con 0x7f328809ff20 2024-01-24T23:24:04.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 10 monclient: _finish_auth 0 2024-01-24T23:24:04.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:04.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:04.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 1 -- 172.21.15.132:0/3948374013 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3280001800 con 0x7f328809ff20 2024-01-24T23:24:04.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 5 monclient: authenticate success, global_id 84710 2024-01-24T23:24:04.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:04.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:04.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:24:04.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 10 monclient: dump: 2024-01-24T23:24:04.327 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:04.327 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:04.327 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:04.327 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:04.327 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:04.328 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:04.328 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:04.328 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:04.328 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:04.328 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:04.328 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:04.328 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:04.328 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:04.328 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:04.328 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:04.328 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:04.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:04.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 1 -- 172.21.15.132:0/3948374013 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3280001a20 con 0x7f328809ff20 2024-01-24T23:24:04.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:04.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:04.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 4 set_mon_vals no callback set 2024-01-24T23:24:04.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 1 -- 172.21.15.132:0/3948374013 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3280004bf0 con 0x7f328809ff20 2024-01-24T23:24:04.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:04.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:24:04.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3286ffd640 10 monclient: dump: 2024-01-24T23:24:04.329 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:04.329 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:04.329 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:04.329 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:04.329 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:04.329 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:04.329 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:04.330 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:04.330 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:04.330 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:04.330 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:04.330 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:04.330 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:04.330 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:04.330 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:04.330 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:04.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 10 monclient: shutdown 2024-01-24T23:24:04.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:04.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 1 -- 172.21.15.132:0/3948374013 >> v1:172.21.15.115:6790/0 conn(0x7f328809ff20 legacy=0x7f32880a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:04.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 1 -- 172.21.15.132:0/3948374013 shutdown_connections 2024-01-24T23:24:04.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 1 -- 172.21.15.132:0/3948374013 >> 172.21.15.132:0/3948374013 conn(0x7f32880a5a10 msgr2=0x7f32880a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:04.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 1 -- 172.21.15.132:0/3948374013 shutdown_connections 2024-01-24T23:24:04.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 1 -- 172.21.15.132:0/3948374013 wait complete. 2024-01-24T23:24:04.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 10 monclient: build_initial_monmap 2024-01-24T23:24:04.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 10 monclient: monmap: 2024-01-24T23:24:04.331 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:04.331 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:04.331 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:04.331 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:04.331 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:04.331 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:04.331 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:04.331 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:04.331 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:04.331 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:04.332 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:04.332 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:04.332 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:04.332 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:04.332 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:04.332 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:04.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 1 Processor -- start 2024-01-24T23:24:04.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 1 -- start start 2024-01-24T23:24:04.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 10 monclient: init 2024-01-24T23:24:04.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:04.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 10 monclient: _add_conns ranks=[1,6,4,0,8,3,7,2,5] 2024-01-24T23:24:04.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 10 monclient(hunting): picked mon.noname-f con 0x7f328809f9c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:04.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 10 monclient(hunting): picked mon.noname-d con 0x7f328809ff20 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:04.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 10 monclient(hunting): picked mon.noname-c con 0x7f328813b2c0 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:04.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3288126790 con 0x7f328809f9c0 2024-01-24T23:24:04.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3288127170 con 0x7f328813b2c0 2024-01-24T23:24:04.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32881255c0 con 0x7f328809ff20 2024-01-24T23:24:04.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:04.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3287fff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f328809f9c0 legacy=0x7f32881cb6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:04.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328e75b640 10 monclient(hunting): authenticate will time out at 1131.647183s 2024-01-24T23:24:04.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f328ccf8640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f328813b2c0 legacy=0x7f32881d3dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:04.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f32877fe640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f328809ff20 0x7f32881d06a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:57494/0 (socket says 172.21.15.132:57494) 2024-01-24T23:24:04.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f32877fe640 1 -- 172.21.15.132:0/1240619804 learned_addr learned my addr 172.21.15.132:0/1240619804 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:04.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3284ff9640 1 -- 172.21.15.132:0/1240619804 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1048463111 0 0) 0x7f32881255c0 con 0x7f328809ff20 2024-01-24T23:24:04.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3284ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:24:04.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3284ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:04.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3284ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:04.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3284ff9640 10 monclient(hunting): my global_id is 5433 2024-01-24T23:24:04.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.319+0000 7f3284ff9640 1 -- 172.21.15.132:0/1240619804 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3268003560 con 0x7f328809ff20 2024-01-24T23:24:04.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 1 -- 172.21.15.132:0/1240619804 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1543958228 0 0) 0x7f3268003560 con 0x7f328809ff20 2024-01-24T23:24:04.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 1 -- 172.21.15.132:0/1240619804 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f32881255c0 con 0x7f328809ff20 2024-01-24T23:24:04.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 1 -- 172.21.15.132:0/1240619804 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f32780046b0 con 0x7f328809ff20 2024-01-24T23:24:04.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:04.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:24:04.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 10 monclient(hunting): dump: 2024-01-24T23:24:04.337 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:04.337 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:04.337 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:04.337 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:04.337 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:04.337 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:04.337 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:04.337 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:04.337 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:04.337 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:04.337 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:04.337 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:04.338 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:04.338 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:04.338 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:04.338 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:04.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:04.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 1 -- 172.21.15.132:0/1240619804 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 415705241 0 0) 0x7f32881255c0 con 0x7f328809ff20 2024-01-24T23:24:04.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 1 -- 172.21.15.132:0/1240619804 >> v1:172.21.15.115:6791/0 conn(0x7f328813b2c0 legacy=0x7f32881d3dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:04.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 1 -- 172.21.15.132:0/1240619804 >> v1:172.21.15.132:6789/0 conn(0x7f328809f9c0 legacy=0x7f32881cb6f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:04.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:24:04.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 1 monclient: found mon.d 2024-01-24T23:24:04.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:04.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:04.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 1 -- 172.21.15.132:0/1240619804 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3288126d50 con 0x7f328809ff20 2024-01-24T23:24:04.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 10 monclient: _finish_auth 0 2024-01-24T23:24:04.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:24:04.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:04.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f328e75b640 5 monclient: authenticate success, global_id 5433 2024-01-24T23:24:04.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f328e75b640 10 monclient: _renew_subs 2024-01-24T23:24:04.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f328e75b640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:04.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f328e75b640 1 -- 172.21.15.132:0/1240619804 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3288071960 con 0x7f328809ff20 2024-01-24T23:24:04.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f328e75b640 10 monclient: _renew_subs 2024-01-24T23:24:04.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f328e75b640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:04.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f328e75b640 1 -- 172.21.15.132:0/1240619804 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3288126a70 con 0x7f328809ff20 2024-01-24T23:24:04.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 1 -- 172.21.15.132:0/1240619804 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f32780048d0 con 0x7f328809ff20 2024-01-24T23:24:04.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:04.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 1 -- 172.21.15.132:0/1240619804 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3278004d80 con 0x7f328809ff20 2024-01-24T23:24:04.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:04.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f32867fc640 4 set_mon_vals no callback set 2024-01-24T23:24:04.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:24:04.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 10 monclient: dump: 2024-01-24T23:24:04.340 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:04.340 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:04.340 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:04.340 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:04.340 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:04.340 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:04.340 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:04.340 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:04.340 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:04.340 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:04.340 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:04.340 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:04.340 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:04.340 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:04.341 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:04.341 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:04.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f328e75b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:04.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f328e75b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:04.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f328e75b640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:04.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f328e75b640 1 -- 172.21.15.132:0/1240619804 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f326c005090 con 0x7f328809ff20 2024-01-24T23:24:04.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 1 -- 172.21.15.132:0/1240619804 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f3278004f60 con 0x7f328809ff20 2024-01-24T23:24:04.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3284ff9640 1 -- 172.21.15.132:0/1240619804 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 5337+0+0 (unknown 1475097318 0 0) 0x7f32780440f0 con 0x7f328809ff20 2024-01-24T23:24:04.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.323+0000 7f3287fff640 10 monclient: get_auth_request con 0x7f3268036f70 auth_method 0 2024-01-24T23:24:04.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.335+0000 7f3284ff9640 1 -- 172.21.15.132:0/1240619804 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f3278014ce0 con 0x7f328809ff20 2024-01-24T23:24:04.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.335+0000 7f3284ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:04.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.335+0000 7f3284ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:04.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.483+0000 7f328e75b640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:04.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.483+0000 7f328e75b640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:04.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.483+0000 7f328e75b640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:04.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.483+0000 7f328e75b640 1 -- 172.21.15.132:0/1240619804 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f326c005b50 con 0x7f328809ff20 2024-01-24T23:24:04.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.483+0000 7f3284ff9640 1 -- 172.21.15.132:0/1240619804 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v73) v1 ==== 56+0+1340 (unknown 4132081119 0 4036273891) 0x7f32780148d0 con 0x7f328809ff20 2024-01-24T23:24:04.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.483+0000 7f3284ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:04.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.483+0000 7f3284ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:24:04.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.575+0000 7f328e75b640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:04.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.575+0000 7f328e75b640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:04.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.575+0000 7f328e75b640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:04.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.575+0000 7f328e75b640 1 -- 172.21.15.132:0/1240619804 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f326c005ce0 con 0x7f328809ff20 2024-01-24T23:24:04.579 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.579+0000 7f3284ff9640 1 -- 172.21.15.132:0/1240619804 <== mon.6 v1:172.21.15.115:6792/0 11 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v73) v1 ==== 56+0+1340 (unknown 4132081119 0 4036273891) 0x7f3278014ab0 con 0x7f328809ff20 2024-01-24T23:24:04.579 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.579+0000 7f3284ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:04.579 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.579+0000 7f3284ff9640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:24:04.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.579+0000 7f328e75b640 1 -- 172.21.15.132:0/1240619804 >> v1:172.21.15.132:6812/1835401922 conn(0x7f3268036f70 legacy=0x7f3268039430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:04.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.579+0000 7f328e75b640 10 monclient: shutdown 2024-01-24T23:24:04.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.579+0000 7f328e75b640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:04.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.579+0000 7f328e75b640 1 -- 172.21.15.132:0/1240619804 >> v1:172.21.15.115:6792/0 conn(0x7f328809ff20 legacy=0x7f32881d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:04.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.579+0000 7f328e75b640 1 -- 172.21.15.132:0/1240619804 shutdown_connections 2024-01-24T23:24:04.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.579+0000 7f328e75b640 1 -- 172.21.15.132:0/1240619804 >> 172.21.15.132:0/1240619804 conn(0x7f32880a5a10 msgr2=0x7f328809d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:04.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.579+0000 7f328e75b640 1 -- 172.21.15.132:0/1240619804 shutdown_connections 2024-01-24T23:24:04.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.579+0000 7f328e75b640 1 -- 172.21.15.132:0/1240619804 wait complete. 2024-01-24T23:24:04.597 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm osd.7 host1 2024-01-24T23:24:04.716 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 10 monclient: get_monmap_and_config 2024-01-24T23:24:04.716 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 10 monclient: build_initial_monmap 2024-01-24T23:24:04.716 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 10 monclient: monmap: 2024-01-24T23:24:04.716 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:04.716 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:04.716 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:04.717100+0000 2024-01-24T23:24:04.716 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:04.717100+0000 2024-01-24T23:24:04.716 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:04.716 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:04.716 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:04.716 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:04.717 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:04.717 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:04.717 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:04.717 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:04.717 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:04.717 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:04.717 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:04.717 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:04.720 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 1 Processor -- start 2024-01-24T23:24:04.720 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 1 -- start start 2024-01-24T23:24:04.720 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 10 monclient: init 2024-01-24T23:24:04.720 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:04.720 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 10 monclient: _add_conns ranks=[1,6,2,8,4,5,3,0,7] 2024-01-24T23:24:04.720 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 10 monclient(hunting): picked mon.noname-f con 0x7f483c09ff90 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:04.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 10 monclient(hunting): picked mon.noname-d con 0x7f483c09f9c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:04.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 10 monclient(hunting): picked mon.noname-b con 0x7f483c13b2c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:04.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f483c064700 con 0x7f483c09ff90 2024-01-24T23:24:04.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f483c09fe00 con 0x7f483c13b2c0 2024-01-24T23:24:04.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f483c0a1160 con 0x7f483c09f9c0 2024-01-24T23:24:04.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483b7fe640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f483c09ff90 legacy=0x7f483c0a0370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:04.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:04.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 10 monclient(hunting): authenticate will time out at 1132.041803s 2024-01-24T23:24:04.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483affd640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f483c09f9c0 0x7f483c139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:57508/0 (socket says 172.21.15.132:57508) 2024-01-24T23:24:04.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483affd640 1 -- 172.21.15.132:0/3167762871 learned_addr learned my addr 172.21.15.132:0/3167762871 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:04.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483bfff640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f483c13b2c0 0x7f483c13d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:48384/0 (socket says 172.21.15.132:48384) 2024-01-24T23:24:04.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 1 -- 172.21.15.132:0/3167762871 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2270195529 0 0) 0x7f483c0a1160 con 0x7f483c09f9c0 2024-01-24T23:24:04.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:04.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:04.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:04.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 10 monclient(hunting): my global_id is 5442 2024-01-24T23:24:04.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 1 -- 172.21.15.132:0/3167762871 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4824003530 con 0x7f483c09f9c0 2024-01-24T23:24:04.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 1 -- 172.21.15.132:0/3167762871 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930300557 0 0) 0x7f483c09fe00 con 0x7f483c13b2c0 2024-01-24T23:24:04.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:04.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:04.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:04.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 10 monclient(hunting): my global_id is 84719 2024-01-24T23:24:04.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 1 -- 172.21.15.132:0/3167762871 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f483c0a1160 con 0x7f483c13b2c0 2024-01-24T23:24:04.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 1 -- 172.21.15.132:0/3167762871 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2445798072 0 0) 0x7f4824003530 con 0x7f483c09f9c0 2024-01-24T23:24:04.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 1 -- 172.21.15.132:0/3167762871 >> v1:172.21.15.115:6790/0 conn(0x7f483c13b2c0 legacy=0x7f483c13d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:04.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 1 -- 172.21.15.132:0/3167762871 >> v1:172.21.15.132:6789/0 conn(0x7f483c09ff90 legacy=0x7f483c0a0370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:04.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:24:04.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 1 monclient: found mon.noname-d 2024-01-24T23:24:04.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:04.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:24:04.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 1 -- 172.21.15.132:0/3167762871 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483c0a1440 con 0x7f483c09f9c0 2024-01-24T23:24:04.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 10 monclient: _finish_auth 0 2024-01-24T23:24:04.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:24:04.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:04.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 1 -- 172.21.15.132:0/3167762871 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f48300017c0 con 0x7f483c09f9c0 2024-01-24T23:24:04.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:04.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:24:04.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 10 monclient: dump: 2024-01-24T23:24:04.723 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:04.723 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:04.723 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:04.723 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:04.723 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:04.723 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:04.723 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:04.723 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:04.724 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:04.724 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:04.724 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:04.724 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:04.724 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:04.724 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:04.724 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:04.724 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:04.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 5 monclient: authenticate success, global_id 5442 2024-01-24T23:24:04.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:04.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 1 -- 172.21.15.132:0/3167762871 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4830001aa0 con 0x7f483c09f9c0 2024-01-24T23:24:04.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:04.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 1 -- 172.21.15.132:0/3167762871 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4830004bf0 con 0x7f483c09f9c0 2024-01-24T23:24:04.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:04.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 4 set_mon_vals no callback set 2024-01-24T23:24:04.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:04.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:24:04.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f483a7fc640 10 monclient: dump: 2024-01-24T23:24:04.725 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:04.725 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:04.725 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:04.725 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:04.725 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:04.725 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:04.725 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:04.725 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:04.725 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:04.725 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:04.725 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:04.725 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:04.725 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:04.725 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:04.726 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:04.726 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:04.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 10 monclient: shutdown 2024-01-24T23:24:04.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:04.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 1 -- 172.21.15.132:0/3167762871 >> v1:172.21.15.115:6792/0 conn(0x7f483c09f9c0 legacy=0x7f483c139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:04.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 1 -- 172.21.15.132:0/3167762871 shutdown_connections 2024-01-24T23:24:04.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 1 -- 172.21.15.132:0/3167762871 >> 172.21.15.132:0/3167762871 conn(0x7f483c0a5a10 msgr2=0x7f483c0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:04.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 1 -- 172.21.15.132:0/3167762871 shutdown_connections 2024-01-24T23:24:04.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.715+0000 7f4842091640 1 -- 172.21.15.132:0/3167762871 wait complete. 2024-01-24T23:24:04.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 10 monclient: build_initial_monmap 2024-01-24T23:24:04.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 10 monclient: monmap: 2024-01-24T23:24:04.726 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:04.726 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:04.726 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:04.726 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:04.727 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:04.727 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:04.727 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:04.727 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:04.727 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:04.727 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:04.727 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:04.727 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:04.727 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:04.727 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:04.727 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:04.727 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:04.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 1 Processor -- start 2024-01-24T23:24:04.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 1 -- start start 2024-01-24T23:24:04.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 10 monclient: init 2024-01-24T23:24:04.730 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:04.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 10 monclient: _add_conns ranks=[2,7,4,3,8,1,5,0,6] 2024-01-24T23:24:04.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 10 monclient(hunting): picked mon.noname-b con 0x7f483c09f9c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:04.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 10 monclient(hunting): picked mon.noname-i con 0x7f483c09ff90 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:04.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 10 monclient(hunting): picked mon.noname-c con 0x7f483c13b2c0 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:04.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f483c09f450 con 0x7f483c09f9c0 2024-01-24T23:24:04.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f483c0715f0 con 0x7f483c13b2c0 2024-01-24T23:24:04.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f483c138cf0 con 0x7f483c09ff90 2024-01-24T23:24:04.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:04.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 10 monclient(hunting): authenticate will time out at 1132.045616s 2024-01-24T23:24:04.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f483bfff640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f483c13b2c0 legacy=0x7f483c1d1490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:04.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f483affd640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f483c09ff90 0x7f483c136080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:52264/0 (socket says 172.21.15.132:52264) 2024-01-24T23:24:04.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f483affd640 1 -- 172.21.15.132:0/527822588 learned_addr learned my addr 172.21.15.132:0/527822588 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:04.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 1 -- 172.21.15.132:0/527822588 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2105341099 0 0) 0x7f483c138cf0 con 0x7f483c09ff90 2024-01-24T23:24:04.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:04.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:04.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:04.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 10 monclient(hunting): my global_id is 5461 2024-01-24T23:24:04.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 1 -- 172.21.15.132:0/527822588 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4828003530 con 0x7f483c09ff90 2024-01-24T23:24:04.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 1 -- 172.21.15.132:0/527822588 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1480984748 0 0) 0x7f4828003530 con 0x7f483c09ff90 2024-01-24T23:24:04.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 1 -- 172.21.15.132:0/527822588 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f483c138cf0 con 0x7f483c09ff90 2024-01-24T23:24:04.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 1 -- 172.21.15.132:0/527822588 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4830004b30 con 0x7f483c09ff90 2024-01-24T23:24:04.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:04.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:04.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 10 monclient(hunting): dump: 2024-01-24T23:24:04.732 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:04.732 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:04.732 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:04.732 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:04.732 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:04.733 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:04.733 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:04.733 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:04.733 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:04.733 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:04.733 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:04.733 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:04.733 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:04.733 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:04.733 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:04.733 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:04.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:04.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 1 -- 172.21.15.132:0/527822588 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1796743547 0 0) 0x7f483c09f450 con 0x7f483c09f9c0 2024-01-24T23:24:04.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:04.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:04.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:04.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 10 monclient(hunting): my global_id is 84728 2024-01-24T23:24:04.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 1 -- 172.21.15.132:0/527822588 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4828003530 con 0x7f483c09f9c0 2024-01-24T23:24:04.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 1 -- 172.21.15.132:0/527822588 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2472951364 0 0) 0x7f483c138cf0 con 0x7f483c09ff90 2024-01-24T23:24:04.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 1 -- 172.21.15.132:0/527822588 >> v1:172.21.15.115:6791/0 conn(0x7f483c13b2c0 legacy=0x7f483c1d1490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:04.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 1 -- 172.21.15.132:0/527822588 >> v1:172.21.15.115:6790/0 conn(0x7f483c09f9c0 legacy=0x7f483c135970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:04.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 10 monclient: _finish_hunting 0 2024-01-24T23:24:04.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 1 monclient: found mon.i 2024-01-24T23:24:04.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:04.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:04.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 1 -- 172.21.15.132:0/527822588 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483c136790 con 0x7f483c09ff90 2024-01-24T23:24:04.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 10 monclient: _finish_auth 0 2024-01-24T23:24:04.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 10 monclient: _check_auth_tickets 2024-01-24T23:24:04.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:04.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 5 monclient: authenticate success, global_id 5461 2024-01-24T23:24:04.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 10 monclient: _renew_subs 2024-01-24T23:24:04.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:04.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 1 -- 172.21.15.132:0/527822588 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f483c136b20 con 0x7f483c09ff90 2024-01-24T23:24:04.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 10 monclient: _renew_subs 2024-01-24T23:24:04.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:04.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 1 -- 172.21.15.132:0/527822588 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f483c138730 con 0x7f483c09ff90 2024-01-24T23:24:04.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 1 -- 172.21.15.132:0/527822588 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4830004ec0 con 0x7f483c09ff90 2024-01-24T23:24:04.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:04.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 1 -- 172.21.15.132:0/527822588 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4830005370 con 0x7f483c09ff90 2024-01-24T23:24:04.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:04.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f48397fa640 4 set_mon_vals no callback set 2024-01-24T23:24:04.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:04.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 10 monclient: dump: 2024-01-24T23:24:04.735 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:04.735 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:04.736 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:04.736 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:04.736 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:04.736 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:04.736 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:04.736 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:04.736 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:04.736 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:04.736 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:04.736 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:04.736 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:04.736 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:04.736 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:04.736 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:04.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:04.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:04.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:04.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4842091640 1 -- 172.21.15.132:0/527822588 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f481c005090 con 0x7f483c09ff90 2024-01-24T23:24:04.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 1 -- 172.21.15.132:0/527822588 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f483000fdf0 con 0x7f483c09ff90 2024-01-24T23:24:04.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f4823fff640 1 -- 172.21.15.132:0/527822588 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 5337+0+0 (unknown 1475097318 0 0) 0x7f48300440a0 con 0x7f483c09ff90 2024-01-24T23:24:04.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.719+0000 7f483b7fe640 10 monclient: get_auth_request con 0x7f48280370a0 auth_method 0 2024-01-24T23:24:04.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.727+0000 7f4823fff640 1 -- 172.21.15.132:0/527822588 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f4830014be0 con 0x7f483c09ff90 2024-01-24T23:24:04.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.727+0000 7f4823fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:04.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.727+0000 7f4823fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:04.890 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.887+0000 7f4842091640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}] 2024-01-24T23:24:04.890 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.887+0000 7f4842091640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}] 2024-01-24T23:24:04.890 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.887+0000 7f4842091640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:04.890 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:04.887+0000 7f4842091640 1 -- 172.21.15.132:0/527822588 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"} v 0) v1 -- 0x7f481c0058c0 con 0x7f483c09ff90 2024-01-24T23:24:05.095 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.091+0000 7f4823fff640 1 -- 172.21.15.132:0/527822588 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}]=0 removed item id 7 name 'osd.7' from crush map v74) v1 ==== 143+0+0 (unknown 3976524317 0 0) 0x7f4830013280 con 0x7f483c09ff90 2024-01-24T23:24:05.095 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.091+0000 7f4823fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}] 2024-01-24T23:24:05.095 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.091+0000 7f4823fff640 10 monclient: _finish_command 2 = system:0 removed item id 7 name 'osd.7' from crush map 2024-01-24T23:24:05.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.191+0000 7f4842091640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}] 2024-01-24T23:24:05.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.191+0000 7f4842091640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}] 2024-01-24T23:24:05.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.191+0000 7f4842091640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:05.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.191+0000 7f4842091640 1 -- 172.21.15.132:0/527822588 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"} v 0) v1 -- 0x7f481c000bc0 con 0x7f483c09ff90 2024-01-24T23:24:05.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.191+0000 7f4823fff640 1 -- 172.21.15.132:0/527822588 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}]=0 device 'osd.7' does not appear in the crush map v74) v1 ==== 145+0+0 (unknown 1413866831 0 0) 0x7f4830014a00 con 0x7f483c09ff90 2024-01-24T23:24:05.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.191+0000 7f4823fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}] 2024-01-24T23:24:05.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.191+0000 7f4823fff640 10 monclient: _finish_command 3 = system:0 device 'osd.7' does not appear in the crush map 2024-01-24T23:24:05.193 INFO:tasks.workunit.client.0.smithi132.stderr:device 'osd.7' does not appear in the crush map 2024-01-24T23:24:05.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.191+0000 7f4842091640 1 -- 172.21.15.132:0/527822588 >> v1:172.21.15.132:6812/1835401922 conn(0x7f48280370a0 legacy=0x7f4828039560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:05.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.191+0000 7f4842091640 10 monclient: shutdown 2024-01-24T23:24:05.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.191+0000 7f4842091640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:05.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.191+0000 7f4842091640 1 -- 172.21.15.132:0/527822588 >> v1:172.21.15.132:6792/0 conn(0x7f483c09ff90 legacy=0x7f483c136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:05.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.195+0000 7f4842091640 1 -- 172.21.15.132:0/527822588 shutdown_connections 2024-01-24T23:24:05.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.195+0000 7f4842091640 1 -- 172.21.15.132:0/527822588 >> 172.21.15.132:0/527822588 conn(0x7f483c0a5a10 msgr2=0x7f483c0a3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:05.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.195+0000 7f4842091640 1 -- 172.21.15.132:0/527822588 shutdown_connections 2024-01-24T23:24:05.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.195+0000 7f4842091640 1 -- 172.21.15.132:0/527822588 wait complete. 2024-01-24T23:24:05.210 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm host1 2024-01-24T23:24:05.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 10 monclient: get_monmap_and_config 2024-01-24T23:24:05.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 10 monclient: build_initial_monmap 2024-01-24T23:24:05.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 10 monclient: monmap: 2024-01-24T23:24:05.316 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:05.317 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:05.317 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:05.317744+0000 2024-01-24T23:24:05.317 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:05.317744+0000 2024-01-24T23:24:05.317 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:05.317 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:05.317 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:05.317 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:05.317 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:05.317 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:05.317 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:05.317 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:05.317 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:05.317 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:05.317 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:05.317 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:05.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 1 Processor -- start 2024-01-24T23:24:05.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 1 -- start start 2024-01-24T23:24:05.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 10 monclient: init 2024-01-24T23:24:05.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:05.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 10 monclient: _add_conns ranks=[0,7,3,8,4,5,1,6,2] 2024-01-24T23:24:05.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 10 monclient(hunting): picked mon.noname-a con 0x7f34bc09ff20 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:05.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 10 monclient(hunting): picked mon.noname-i con 0x7f34bc09f9c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:05.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 10 monclient(hunting): picked mon.noname-g con 0x7f34bc13b2c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:05.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34bc064700 con 0x7f34bc09ff20 2024-01-24T23:24:05.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34bc09e760 con 0x7f34bc13b2c0 2024-01-24T23:24:05.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34bc0a1160 con 0x7f34bc09f9c0 2024-01-24T23:24:05.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:05.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 10 monclient(hunting): authenticate will time out at 1132.642197s 2024-01-24T23:24:05.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c186c640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f34bc09f9c0 0x7f34bc139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:52270/0 (socket says 172.21.15.132:52270) 2024-01-24T23:24:05.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c186c640 1 -- 172.21.15.132:0/733269815 learned_addr learned my addr 172.21.15.132:0/733269815 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:05.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c206d640 1 -- 172.21.15.132:0/733269815 >> v1:172.21.15.115:6789/0 conn(0x7f34bc09ff20 legacy=0x7f34bc0a0300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:05.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 1 -- 172.21.15.132:0/733269815 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1173493989 0 0) 0x7f34bc09e760 con 0x7f34bc13b2c0 2024-01-24T23:24:05.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 10 monclient(hunting): _negotiate 2024-01-24T23:24:05.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:05.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:05.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 10 monclient(hunting): my global_id is 5394 2024-01-24T23:24:05.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 1 -- 172.21.15.132:0/733269815 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34a4003530 con 0x7f34bc13b2c0 2024-01-24T23:24:05.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 1 -- 172.21.15.132:0/733269815 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2315393962 0 0) 0x7f34bc0a1160 con 0x7f34bc09f9c0 2024-01-24T23:24:05.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 10 monclient(hunting): _negotiate 2024-01-24T23:24:05.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:05.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:05.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 10 monclient(hunting): my global_id is 5470 2024-01-24T23:24:05.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 1 -- 172.21.15.132:0/733269815 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34bc09e760 con 0x7f34bc09f9c0 2024-01-24T23:24:05.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 1 -- 172.21.15.132:0/733269815 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2705150484 0 0) 0x7f34a4003530 con 0x7f34bc13b2c0 2024-01-24T23:24:05.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 1 -- 172.21.15.132:0/733269815 >> v1:172.21.15.132:6792/0 conn(0x7f34bc09f9c0 legacy=0x7f34bc139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:05.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 1 -- 172.21.15.132:0/733269815 >> v1:172.21.15.115:6789/0 conn(0x7f34bc09ff20 legacy=0x7f34bc0a0300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:05.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 10 monclient: _finish_hunting 0 2024-01-24T23:24:05.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 1 monclient: found mon.noname-g 2024-01-24T23:24:05.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:05.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:24:05.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 1 -- 172.21.15.132:0/733269815 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34bc0a0a10 con 0x7f34bc13b2c0 2024-01-24T23:24:05.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 10 monclient: _finish_auth 0 2024-01-24T23:24:05.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 10 monclient: _check_auth_tickets 2024-01-24T23:24:05.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:05.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 1 -- 172.21.15.132:0/733269815 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f34b4001b40 con 0x7f34bc13b2c0 2024-01-24T23:24:05.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 5 monclient: authenticate success, global_id 5394 2024-01-24T23:24:05.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:05.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:05.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:05.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 10 monclient: dump: 2024-01-24T23:24:05.326 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:05.327 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:05.327 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:05.327 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:05.327 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:05.327 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:05.327 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:05.327 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:05.327 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:05.327 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:05.327 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:05.327 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:05.327 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:05.327 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:05.327 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:05.327 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:05.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:05.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 1 -- 172.21.15.132:0/733269815 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f34b4001da0 con 0x7f34bc13b2c0 2024-01-24T23:24:05.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:05.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:05.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 4 set_mon_vals no callback set 2024-01-24T23:24:05.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 1 -- 172.21.15.132:0/733269815 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f34b4003ce0 con 0x7f34bc13b2c0 2024-01-24T23:24:05.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:05.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:05.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c106b640 10 monclient: dump: 2024-01-24T23:24:05.328 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:05.328 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:05.328 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:05.328 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:05.328 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:05.328 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:05.328 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:05.329 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:05.329 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:05.329 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:05.329 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:05.329 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:05.329 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:05.329 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:05.329 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:05.329 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:05.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 10 monclient: shutdown 2024-01-24T23:24:05.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:05.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 1 -- 172.21.15.132:0/733269815 >> v1:172.21.15.132:6790/0 conn(0x7f34bc13b2c0 legacy=0x7f34bc13d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:05.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 1 -- 172.21.15.132:0/733269815 shutdown_connections 2024-01-24T23:24:05.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 1 -- 172.21.15.132:0/733269815 >> 172.21.15.132:0/733269815 conn(0x7f34bc0a5a10 msgr2=0x7f34bc0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:05.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 1 -- 172.21.15.132:0/733269815 shutdown_connections 2024-01-24T23:24:05.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.315+0000 7f34c42d1640 1 -- 172.21.15.132:0/733269815 wait complete. 2024-01-24T23:24:05.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 10 monclient: build_initial_monmap 2024-01-24T23:24:05.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 10 monclient: monmap: 2024-01-24T23:24:05.330 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:05.330 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:05.330 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:05.330 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:05.330 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:05.330 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:05.330 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:05.330 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:05.330 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:05.330 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:05.330 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:05.330 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:05.330 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:05.331 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:05.331 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:05.331 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:05.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 1 Processor -- start 2024-01-24T23:24:05.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 1 -- start start 2024-01-24T23:24:05.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 10 monclient: init 2024-01-24T23:24:05.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:05.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 10 monclient: _add_conns ranks=[2,8,1,5,4,0,7,6,3] 2024-01-24T23:24:05.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 10 monclient(hunting): picked mon.noname-b con 0x7f34bc09f9c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:05.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 10 monclient(hunting): picked mon.noname-e con 0x7f34bc09ff20 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:05.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 10 monclient(hunting): picked mon.noname-f con 0x7f34bc13b2c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:05.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34bc071680 con 0x7f34bc13b2c0 2024-01-24T23:24:05.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34bc11c430 con 0x7f34bc09f9c0 2024-01-24T23:24:05.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34bc11d250 con 0x7f34bc09ff20 2024-01-24T23:24:05.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:05.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 10 monclient(hunting): authenticate will time out at 1132.645194s 2024-01-24T23:24:05.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c286e640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f34bc13b2c0 legacy=0x7f34bc1d1490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:05.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c186c640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f34bc09ff20 0x7f34bc11bd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:55896/0 (socket says 172.21.15.132:55896) 2024-01-24T23:24:05.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c186c640 1 -- 172.21.15.132:0/136069057 learned_addr learned my addr 172.21.15.132:0/136069057 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:05.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 1 -- 172.21.15.132:0/136069057 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3897235106 0 0) 0x7f34bc11c430 con 0x7f34bc09f9c0 2024-01-24T23:24:05.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:05.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:05.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:05.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 10 monclient(hunting): my global_id is 84737 2024-01-24T23:24:05.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 1 -- 172.21.15.132:0/136069057 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34a8003530 con 0x7f34bc09f9c0 2024-01-24T23:24:05.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 1 -- 172.21.15.132:0/136069057 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2684569630 0 0) 0x7f34bc11d250 con 0x7f34bc09ff20 2024-01-24T23:24:05.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:05.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:05.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:05.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 10 monclient(hunting): my global_id is 84617 2024-01-24T23:24:05.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 1 -- 172.21.15.132:0/136069057 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34bc11c430 con 0x7f34bc09ff20 2024-01-24T23:24:05.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 1 -- 172.21.15.132:0/136069057 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 671380399 0 0) 0x7f34bc11c430 con 0x7f34bc09ff20 2024-01-24T23:24:05.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 1 -- 172.21.15.132:0/136069057 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f34bc11d250 con 0x7f34bc09ff20 2024-01-24T23:24:05.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 1 -- 172.21.15.132:0/136069057 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3992098313 0 0) 0x7f34a8003530 con 0x7f34bc09f9c0 2024-01-24T23:24:05.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 1 -- 172.21.15.132:0/136069057 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f34bc11c430 con 0x7f34bc09f9c0 2024-01-24T23:24:05.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 1 -- 172.21.15.132:0/136069057 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f34b0001560 con 0x7f34bc09ff20 2024-01-24T23:24:05.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:05.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:24:05.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 10 monclient(hunting): dump: 2024-01-24T23:24:05.333 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:05.333 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:05.333 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:05.333 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:05.333 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:05.333 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:05.333 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:05.333 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:05.334 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:05.334 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:05.334 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:05.334 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:05.334 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:05.334 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:05.334 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:05.334 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:05.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:05.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 1 -- 172.21.15.132:0/136069057 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f34ac004120 con 0x7f34bc09f9c0 2024-01-24T23:24:05.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:05.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:24:05.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 10 monclient(hunting): dump: 2024-01-24T23:24:05.334 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:05.334 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:05.334 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:05.335 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:05.335 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:05.335 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:05.335 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:05.335 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:05.335 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:05.335 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:05.335 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:05.335 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:05.335 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:05.335 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:05.335 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:05.335 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:05.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 1 -- 172.21.15.132:0/136069057 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 958940387 0 0) 0x7f34bc11d250 con 0x7f34bc09ff20 2024-01-24T23:24:05.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 1 -- 172.21.15.132:0/136069057 >> v1:172.21.15.115:6790/0 conn(0x7f34bc09f9c0 legacy=0x7f34bc11b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:05.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 1 -- 172.21.15.132:0/136069057 >> v1:172.21.15.132:6789/0 conn(0x7f34bc13b2c0 legacy=0x7f34bc1d1490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:05.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:05.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 1 monclient: found mon.e 2024-01-24T23:24:05.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:05.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:05.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 1 -- 172.21.15.132:0/136069057 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34bc071960 con 0x7f34bc09ff20 2024-01-24T23:24:05.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 10 monclient: _finish_auth 0 2024-01-24T23:24:05.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:05.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:05.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 5 monclient: authenticate success, global_id 84617 2024-01-24T23:24:05.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 10 monclient: _renew_subs 2024-01-24T23:24:05.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:05.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 1 -- 172.21.15.132:0/136069057 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34bc11e700 con 0x7f34bc09ff20 2024-01-24T23:24:05.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 10 monclient: _renew_subs 2024-01-24T23:24:05.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:05.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 1 -- 172.21.15.132:0/136069057 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f34bc11cf60 con 0x7f34bc09ff20 2024-01-24T23:24:05.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 1 -- 172.21.15.132:0/136069057 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f34b0004aa0 con 0x7f34bc09ff20 2024-01-24T23:24:05.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:05.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 1 -- 172.21.15.132:0/136069057 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f34b0004f50 con 0x7f34bc09ff20 2024-01-24T23:24:05.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:05.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c086a640 4 set_mon_vals no callback set 2024-01-24T23:24:05.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:24:05.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34baffd640 10 monclient: dump: 2024-01-24T23:24:05.337 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:05.337 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:05.337 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:05.337 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:05.337 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:05.337 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:05.338 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:05.338 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:05.338 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:05.338 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:05.338 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:05.338 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:05.338 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:05.338 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:05.338 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:05.338 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:05.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:05.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:05.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:05.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.319+0000 7f34c42d1640 1 -- 172.21.15.132:0/136069057 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f349c005090 con 0x7f34bc09ff20 2024-01-24T23:24:05.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.323+0000 7f34baffd640 1 -- 172.21.15.132:0/136069057 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f34b000f9d0 con 0x7f34bc09ff20 2024-01-24T23:24:05.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.323+0000 7f34baffd640 1 -- 172.21.15.132:0/136069057 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(74..74 src has 1..74) v4 ==== 5316+0+0 (unknown 3850554732 0 0) 0x7f34b0043e80 con 0x7f34bc09ff20 2024-01-24T23:24:05.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.323+0000 7f34c206d640 10 monclient: get_auth_request con 0x7f34a8037660 auth_method 0 2024-01-24T23:24:05.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.335+0000 7f34baffd640 1 -- 172.21.15.132:0/136069057 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f34b00148b0 con 0x7f34bc09ff20 2024-01-24T23:24:05.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.335+0000 7f34baffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:05.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.335+0000 7f34baffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:05.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.491+0000 7f34c42d1640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host1"}] 2024-01-24T23:24:05.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.491+0000 7f34c42d1640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "host1"}] 2024-01-24T23:24:05.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.491+0000 7f34c42d1640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:05.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:05.491+0000 7f34c42d1640 1 -- 172.21.15.132:0/136069057 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush rm", "name": "host1"} v 0) v1 -- 0x7f349c005340 con 0x7f34bc09ff20 2024-01-24T23:24:06.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.114+0000 7f34baffd640 1 -- 172.21.15.132:0/136069057 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host1"}]=0 removed item id -11 name 'host1' from crush map v75) v1 ==== 124+0+0 (unknown 3960377477 0 0) 0x7f34b0012f50 con 0x7f34bc09ff20 2024-01-24T23:24:06.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.114+0000 7f34baffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "host1"}] 2024-01-24T23:24:06.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.114+0000 7f34baffd640 10 monclient: _finish_command 2 = system:0 removed item id -11 name 'host1' from crush map 2024-01-24T23:24:06.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.214+0000 7f34c42d1640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host1"}] 2024-01-24T23:24:06.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.214+0000 7f34c42d1640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "host1"}] 2024-01-24T23:24:06.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.214+0000 7f34c42d1640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:06.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.214+0000 7f34c42d1640 1 -- 172.21.15.132:0/136069057 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush rm", "name": "host1"} v 0) v1 -- 0x7f349c0057e0 con 0x7f34bc09ff20 2024-01-24T23:24:06.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.218+0000 7f34baffd640 1 -- 172.21.15.132:0/136069057 <== mon.8 v1:172.21.15.115:6793/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host1"}]=0 device 'host1' does not appear in the crush map v75) v1 ==== 124+0+0 (unknown 1953226852 0 0) 0x7f34b00146d0 con 0x7f34bc09ff20 2024-01-24T23:24:06.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.218+0000 7f34baffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "host1"}] 2024-01-24T23:24:06.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.218+0000 7f34baffd640 10 monclient: _finish_command 3 = system:0 device 'host1' does not appear in the crush map 2024-01-24T23:24:06.221 INFO:tasks.workunit.client.0.smithi132.stderr:device 'host1' does not appear in the crush map 2024-01-24T23:24:06.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.218+0000 7f34c42d1640 1 -- 172.21.15.132:0/136069057 >> v1:172.21.15.132:6812/1835401922 conn(0x7f34a8037660 legacy=0x7f34a8039b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:06.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.218+0000 7f34c42d1640 10 monclient: shutdown 2024-01-24T23:24:06.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.218+0000 7f34c42d1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:06.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.218+0000 7f34c42d1640 1 -- 172.21.15.132:0/136069057 >> v1:172.21.15.115:6793/0 conn(0x7f34bc09ff20 legacy=0x7f34bc11bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:06.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.218+0000 7f34c42d1640 1 -- 172.21.15.132:0/136069057 shutdown_connections 2024-01-24T23:24:06.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.218+0000 7f34c42d1640 1 -- 172.21.15.132:0/136069057 >> 172.21.15.132:0/136069057 conn(0x7f34bc0a5a10 msgr2=0x7f34bc0a2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:06.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.218+0000 7f34c42d1640 1 -- 172.21.15.132:0/136069057 shutdown_connections 2024-01-24T23:24:06.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.218+0000 7f34c42d1640 1 -- 172.21.15.132:0/136069057 wait complete. 2024-01-24T23:24:06.237 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd tree 2024-01-24T23:24:06.237 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep -q 0 2024-01-24T23:24:06.237 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep -c host1 2024-01-24T23:24:06.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.350+0000 7f7ee2ce8640 10 monclient: get_monmap_and_config 2024-01-24T23:24:06.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.350+0000 7f7ee2ce8640 10 monclient: build_initial_monmap 2024-01-24T23:24:06.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.350+0000 7f7ee2ce8640 10 monclient: monmap: 2024-01-24T23:24:06.354 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:06.354 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:06.354 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:06.355019+0000 2024-01-24T23:24:06.354 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:06.355019+0000 2024-01-24T23:24:06.354 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:06.354 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:06.354 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:06.354 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:06.354 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:06.354 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:06.355 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:06.355 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:06.355 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:06.355 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:06.355 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:06.355 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:06.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.350+0000 7f7ee2ce8640 1 Processor -- start 2024-01-24T23:24:06.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.350+0000 7f7ee2ce8640 1 -- start start 2024-01-24T23:24:06.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.350+0000 7f7ee2ce8640 10 monclient: init 2024-01-24T23:24:06.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.350+0000 7f7ee2ce8640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:06.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.350+0000 7f7ee2ce8640 10 monclient: _add_conns ranks=[2,5,1,4,0,3,8,6,7] 2024-01-24T23:24:06.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.350+0000 7f7ee2ce8640 10 monclient(hunting): picked mon.noname-b con 0x7f7edc136e50 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:06.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.350+0000 7f7ee2ce8640 10 monclient(hunting): picked mon.noname-h con 0x7f7edc0a0290 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:06.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.350+0000 7f7ee2ce8640 10 monclient(hunting): picked mon.noname-f con 0x7f7edc0a1600 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:06.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.350+0000 7f7ee2ce8640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7edc064700 con 0x7f7edc0a1600 2024-01-24T23:24:06.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.350+0000 7f7ee2ce8640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7edc137940 con 0x7f7edc136e50 2024-01-24T23:24:06.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.350+0000 7f7ee2ce8640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7edc09f9c0 con 0x7f7edc0a0290 2024-01-24T23:24:06.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.350+0000 7f7ee2ce8640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:06.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.350+0000 7f7ee2ce8640 10 monclient(hunting): authenticate will time out at 1133.679582s 2024-01-24T23:24:06.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.350+0000 7f7ee1285640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f7edc0a1600 legacy=0x7f7edc13f490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:06.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.350+0000 7f7edbfff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f7edc0a0290 0x7f7edc13bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:40858/0 (socket says 172.21.15.132:40858) 2024-01-24T23:24:06.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.350+0000 7f7edbfff640 1 -- 172.21.15.132:0/2276895057 learned_addr learned my addr 172.21.15.132:0/2276895057 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:06.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 1 -- 172.21.15.132:0/2276895057 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 195957002 0 0) 0x7f7edc09f9c0 con 0x7f7edc0a0290 2024-01-24T23:24:06.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:24:06.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:06.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:06.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 10 monclient(hunting): my global_id is 5405 2024-01-24T23:24:06.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 1 -- 172.21.15.132:0/2276895057 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ec4003530 con 0x7f7edc0a0290 2024-01-24T23:24:06.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 1 -- 172.21.15.132:0/2276895057 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3214405820 0 0) 0x7f7ec4003530 con 0x7f7edc0a0290 2024-01-24T23:24:06.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 1 -- 172.21.15.132:0/2276895057 >> v1:172.21.15.115:6790/0 conn(0x7f7edc136e50 legacy=0x7f7edc137230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:06.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 1 -- 172.21.15.132:0/2276895057 >> v1:172.21.15.132:6789/0 conn(0x7f7edc0a1600 legacy=0x7f7edc13f490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:06.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:24:06.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 1 monclient: found mon.noname-h 2024-01-24T23:24:06.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:06.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:24:06.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 1 -- 172.21.15.132:0/2276895057 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7edc09eda0 con 0x7f7edc0a0290 2024-01-24T23:24:06.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 10 monclient: _finish_auth 0 2024-01-24T23:24:06.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:24:06.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:06.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 1 -- 172.21.15.132:0/2276895057 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7ed00017c0 con 0x7f7edc0a0290 2024-01-24T23:24:06.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 5 monclient: authenticate success, global_id 5405 2024-01-24T23:24:06.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:06.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:06.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:06.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 10 monclient: dump: 2024-01-24T23:24:06.363 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:06.363 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:06.363 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:06.363 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:06.363 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:06.363 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:06.363 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:06.363 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:06.364 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:06.364 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:06.364 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:06.364 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:06.364 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:06.364 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:06.364 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:06.364 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:06.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:06.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 1 -- 172.21.15.132:0/2276895057 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7ed0001ae0 con 0x7f7edc0a0290 2024-01-24T23:24:06.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:06.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:06.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 4 set_mon_vals no callback set 2024-01-24T23:24:06.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 1 -- 172.21.15.132:0/2276895057 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7ed0004bf0 con 0x7f7edc0a0290 2024-01-24T23:24:06.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:06.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:06.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edb7fe640 10 monclient: dump: 2024-01-24T23:24:06.365 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:06.365 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:06.365 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:06.365 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:06.365 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:06.365 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:06.365 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:06.365 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:06.365 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:06.365 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:06.365 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:06.366 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:06.366 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:06.366 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:06.366 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:06.366 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:06.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 10 monclient: shutdown 2024-01-24T23:24:06.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:06.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 1 -- 172.21.15.132:0/2276895057 >> v1:172.21.15.132:6791/0 conn(0x7f7edc0a0290 legacy=0x7f7edc13bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:06.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 1 -- 172.21.15.132:0/2276895057 shutdown_connections 2024-01-24T23:24:06.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 1 -- 172.21.15.132:0/2276895057 >> 172.21.15.132:0/2276895057 conn(0x7f7edc0a5a10 msgr2=0x7f7edc0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:06.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 1 -- 172.21.15.132:0/2276895057 shutdown_connections 2024-01-24T23:24:06.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 1 -- 172.21.15.132:0/2276895057 wait complete. 2024-01-24T23:24:06.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 10 monclient: build_initial_monmap 2024-01-24T23:24:06.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 10 monclient: monmap: 2024-01-24T23:24:06.367 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:06.367 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:06.367 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:06.367 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:06.367 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:06.367 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:06.367 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:06.367 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:06.367 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:06.367 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:06.367 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:06.367 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:06.367 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:06.368 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:06.368 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:06.368 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:06.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 1 Processor -- start 2024-01-24T23:24:06.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 1 -- start start 2024-01-24T23:24:06.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 10 monclient: init 2024-01-24T23:24:06.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:06.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 10 monclient: _add_conns ranks=[7,2,4,3,0,6,8,1,5] 2024-01-24T23:24:06.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 10 monclient(hunting): picked mon.noname-i con 0x7f7edc0a0290 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:06.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 10 monclient(hunting): picked mon.noname-b con 0x7f7edc0a1600 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:06.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 10 monclient(hunting): picked mon.noname-c con 0x7f7edc136e50 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:06.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7edc136c20 con 0x7f7edc0a1600 2024-01-24T23:24:06.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7edc11f2f0 con 0x7f7edc136e50 2024-01-24T23:24:06.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7edc09ea70 con 0x7f7edc0a0290 2024-01-24T23:24:06.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:06.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 10 monclient(hunting): authenticate will time out at 1133.682467s 2024-01-24T23:24:06.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee0a84640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f7edc0a0290 0x7f7edc131b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:52284/0 (socket says 172.21.15.132:52284) 2024-01-24T23:24:06.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee0a84640 1 -- 172.21.15.132:0/1272835740 learned_addr learned my addr 172.21.15.132:0/1272835740 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:06.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee1285640 1 -- 172.21.15.132:0/1272835740 >> v1:172.21.15.115:6791/0 conn(0x7f7edc136e50 legacy=0x7f7edc1d3b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:06.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 1 -- 172.21.15.132:0/1272835740 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 161100010 0 0) 0x7f7edc09ea70 con 0x7f7edc0a0290 2024-01-24T23:24:06.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 10 monclient(hunting): _negotiate 2024-01-24T23:24:06.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:06.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:06.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 10 monclient(hunting): my global_id is 5479 2024-01-24T23:24:06.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 1 -- 172.21.15.132:0/1272835740 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ec8003530 con 0x7f7edc0a0290 2024-01-24T23:24:06.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 1 -- 172.21.15.132:0/1272835740 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2485346668 0 0) 0x7f7ec8003530 con 0x7f7edc0a0290 2024-01-24T23:24:06.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 1 -- 172.21.15.132:0/1272835740 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7edc09ea70 con 0x7f7edc0a0290 2024-01-24T23:24:06.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 1 -- 172.21.15.132:0/1272835740 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7ecc001dc0 con 0x7f7edc0a0290 2024-01-24T23:24:06.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:06.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:06.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 10 monclient(hunting): dump: 2024-01-24T23:24:06.370 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:06.370 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:06.370 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:06.370 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:06.370 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:06.370 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:06.370 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:06.370 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:06.370 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:06.370 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:06.370 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:06.370 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:06.371 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:06.371 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:06.371 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:06.371 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:06.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:06.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 1 -- 172.21.15.132:0/1272835740 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2470311021 0 0) 0x7f7edc136c20 con 0x7f7edc0a1600 2024-01-24T23:24:06.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 10 monclient(hunting): _negotiate 2024-01-24T23:24:06.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:06.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:06.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 10 monclient(hunting): my global_id is 84755 2024-01-24T23:24:06.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 1 -- 172.21.15.132:0/1272835740 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ec8003530 con 0x7f7edc0a1600 2024-01-24T23:24:06.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 1 -- 172.21.15.132:0/1272835740 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4227683885 0 0) 0x7f7edc09ea70 con 0x7f7edc0a0290 2024-01-24T23:24:06.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 1 -- 172.21.15.132:0/1272835740 >> v1:172.21.15.115:6791/0 conn(0x7f7edc136e50 legacy=0x7f7edc1d3b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:06.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 1 -- 172.21.15.132:0/1272835740 >> v1:172.21.15.115:6790/0 conn(0x7f7edc0a1600 legacy=0x7f7edc1d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:06.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 10 monclient: _finish_hunting 0 2024-01-24T23:24:06.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 1 monclient: found mon.i 2024-01-24T23:24:06.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:06.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:06.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 1 -- 172.21.15.132:0/1272835740 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7edc11d000 con 0x7f7edc0a0290 2024-01-24T23:24:06.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 10 monclient: _finish_auth 0 2024-01-24T23:24:06.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 10 monclient: _check_auth_tickets 2024-01-24T23:24:06.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:06.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 5 monclient: authenticate success, global_id 5479 2024-01-24T23:24:06.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 10 monclient: _renew_subs 2024-01-24T23:24:06.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:06.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 1 -- 172.21.15.132:0/1272835740 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7edc11f010 con 0x7f7edc0a0290 2024-01-24T23:24:06.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 10 monclient: _renew_subs 2024-01-24T23:24:06.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:06.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ee2ce8640 1 -- 172.21.15.132:0/1272835740 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7edc132270 con 0x7f7edc0a0290 2024-01-24T23:24:06.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 1 -- 172.21.15.132:0/1272835740 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7ecc003cd0 con 0x7f7edc0a0290 2024-01-24T23:24:06.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:06.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 1 -- 172.21.15.132:0/1272835740 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7ecc004200 con 0x7f7edc0a0290 2024-01-24T23:24:06.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:06.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:06.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7edaffd640 4 set_mon_vals no callback set 2024-01-24T23:24:06.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.354+0000 7f7ed97fa640 10 monclient: dump: 2024-01-24T23:24:06.373 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:06.373 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:06.373 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:06.373 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:06.373 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:06.373 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:06.373 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:06.374 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:06.374 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:06.374 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:06.374 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:06.374 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:06.374 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:06.374 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:06.374 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:06.374 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:06.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.358+0000 7f7ee2ce8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:06.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.358+0000 7f7ee2ce8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:06.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.358+0000 7f7ee2ce8640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:06.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.358+0000 7f7ee2ce8640 1 -- 172.21.15.132:0/1272835740 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ebc005090 con 0x7f7edc0a0290 2024-01-24T23:24:06.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.358+0000 7f7ed97fa640 1 -- 172.21.15.132:0/1272835740 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f7ecc004360 con 0x7f7edc0a0290 2024-01-24T23:24:06.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.358+0000 7f7ed97fa640 1 -- 172.21.15.132:0/1272835740 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(75..75 src has 1..75) v4 ==== 5181+0+0 (unknown 186118056 0 0) 0x7f7ecc044100 con 0x7f7edc0a0290 2024-01-24T23:24:06.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.358+0000 7f7edbfff640 10 monclient: get_auth_request con 0x7f7ec8036fb0 auth_method 0 2024-01-24T23:24:06.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.362+0000 7f7ed97fa640 1 -- 172.21.15.132:0/1272835740 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f7ecc014c20 con 0x7f7edc0a0290 2024-01-24T23:24:06.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.362+0000 7f7ed97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:06.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.362+0000 7f7ed97fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:06.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.518+0000 7f7ee2ce8640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:06.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.518+0000 7f7ee2ce8640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:06.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.518+0000 7f7ee2ce8640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:06.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.518+0000 7f7ee2ce8640 1 -- 172.21.15.132:0/1272835740 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f7ebc005b50 con 0x7f7edc0a0290 2024-01-24T23:24:06.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.518+0000 7f7ed97fa640 1 -- 172.21.15.132:0/1272835740 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v75) v1 ==== 56+0+1273 (unknown 3476242654 0 415004057) 0x7f7ecc0132c0 con 0x7f7edc0a0290 2024-01-24T23:24:06.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.518+0000 7f7ed97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:06.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.518+0000 7f7ed97fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:24:06.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.614+0000 7f7ee2ce8640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:06.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.614+0000 7f7ee2ce8640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:06.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.614+0000 7f7ee2ce8640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:06.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.614+0000 7f7ee2ce8640 1 -- 172.21.15.132:0/1272835740 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f7ebc005ce0 con 0x7f7edc0a0290 2024-01-24T23:24:06.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.614+0000 7f7ed97fa640 1 -- 172.21.15.132:0/1272835740 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v75) v1 ==== 56+0+1273 (unknown 3476242654 0 415004057) 0x7f7ecc014a40 con 0x7f7edc0a0290 2024-01-24T23:24:06.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.614+0000 7f7ed97fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:06.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.614+0000 7f7ed97fa640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:24:06.619 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.614+0000 7f7ee2ce8640 1 -- 172.21.15.132:0/1272835740 >> v1:172.21.15.132:6812/1835401922 conn(0x7f7ec8036fb0 legacy=0x7f7ec8039470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:06.619 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.614+0000 7f7ee2ce8640 10 monclient: shutdown 2024-01-24T23:24:06.619 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.614+0000 7f7ee2ce8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:06.619 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.614+0000 7f7ee2ce8640 1 -- 172.21.15.132:0/1272835740 >> v1:172.21.15.132:6792/0 conn(0x7f7edc0a0290 legacy=0x7f7edc131b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:06.619 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.614+0000 7f7ee2ce8640 1 -- 172.21.15.132:0/1272835740 shutdown_connections 2024-01-24T23:24:06.619 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.614+0000 7f7ee2ce8640 1 -- 172.21.15.132:0/1272835740 >> 172.21.15.132:0/1272835740 conn(0x7f7edc0a5a10 msgr2=0x7f7edc1be9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:06.619 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.614+0000 7f7ee2ce8640 1 -- 172.21.15.132:0/1272835740 shutdown_connections 2024-01-24T23:24:06.619 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.614+0000 7f7ee2ce8640 1 -- 172.21.15.132:0/1272835740 wait complete. 2024-01-24T23:24:06.633 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd tree-from host1 2024-01-24T23:24:06.633 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:24:06.633 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd tree-from host1 2024-01-24T23:24:06.749 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.746+0000 7f1953b99640 10 monclient: get_monmap_and_config 2024-01-24T23:24:06.749 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.746+0000 7f1953b99640 10 monclient: build_initial_monmap 2024-01-24T23:24:06.749 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.746+0000 7f1953b99640 10 monclient: monmap: 2024-01-24T23:24:06.749 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:06.749 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:06.749 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:06.750332+0000 2024-01-24T23:24:06.749 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:06.750332+0000 2024-01-24T23:24:06.749 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:06.749 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:06.749 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:06.749 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:06.749 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:06.750 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:06.750 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:06.750 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:06.750 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:06.750 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:06.750 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:06.750 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:06.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.746+0000 7f1953b99640 1 Processor -- start 2024-01-24T23:24:06.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.746+0000 7f1953b99640 1 -- start start 2024-01-24T23:24:06.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.746+0000 7f1953b99640 10 monclient: init 2024-01-24T23:24:06.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.746+0000 7f1953b99640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:06.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.746+0000 7f1953b99640 10 monclient: _add_conns ranks=[4,6,0,7,5,3,2,1,8] 2024-01-24T23:24:06.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.746+0000 7f1953b99640 10 monclient(hunting): picked mon.noname-c con 0x7f194c135e10 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:06.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.746+0000 7f1953b99640 10 monclient(hunting): picked mon.noname-d con 0x7f194c1361f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:06.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.746+0000 7f1953b99640 10 monclient(hunting): picked mon.noname-a con 0x7f194c1365f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:06.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.746+0000 7f1953b99640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c064700 con 0x7f194c1365f0 2024-01-24T23:24:06.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.746+0000 7f1953b99640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c09c3e0 con 0x7f194c135e10 2024-01-24T23:24:06.752 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.746+0000 7f1953b99640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c0a4f90 con 0x7f194c1361f0 2024-01-24T23:24:06.752 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.746+0000 7f1953b99640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:06.752 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.746+0000 7f1953b99640 10 monclient(hunting): authenticate will time out at 1134.075428s 2024-01-24T23:24:06.752 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.746+0000 7f1952136640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f194c1365f0 legacy=0x7f194c13f600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:06.752 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.746+0000 7f1951935640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f194c135e10 legacy=0x7f194c09bcd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:06.752 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.746+0000 7f1951134640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f194c1361f0 0x7f194c13bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:57514/0 (socket says 172.21.15.132:57514) 2024-01-24T23:24:06.752 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.746+0000 7f1951134640 1 -- 172.21.15.132:0/3311325849 learned_addr learned my addr 172.21.15.132:0/3311325849 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:06.753 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 1 -- 172.21.15.132:0/3311325849 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1584012312 0 0) 0x7f194c0a4f90 con 0x7f194c1361f0 2024-01-24T23:24:06.753 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 10 monclient(hunting): _negotiate 2024-01-24T23:24:06.753 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:06.753 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:06.753 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 10 monclient(hunting): my global_id is 5451 2024-01-24T23:24:06.753 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 1 -- 172.21.15.132:0/3311325849 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1938003530 con 0x7f194c1361f0 2024-01-24T23:24:06.753 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 1 -- 172.21.15.132:0/3311325849 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3898369776 0 0) 0x7f1938003530 con 0x7f194c1361f0 2024-01-24T23:24:06.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 1 -- 172.21.15.132:0/3311325849 >> v1:172.21.15.115:6791/0 conn(0x7f194c135e10 legacy=0x7f194c09bcd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:06.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 1 -- 172.21.15.132:0/3311325849 >> v1:172.21.15.115:6789/0 conn(0x7f194c1365f0 legacy=0x7f194c13f600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:06.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 10 monclient: _finish_hunting 0 2024-01-24T23:24:06.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 1 monclient: found mon.noname-d 2024-01-24T23:24:06.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:06.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:24:06.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 1 -- 172.21.15.132:0/3311325849 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c09b010 con 0x7f194c1361f0 2024-01-24T23:24:06.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 10 monclient: _finish_auth 0 2024-01-24T23:24:06.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 10 monclient: _check_auth_tickets 2024-01-24T23:24:06.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:06.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 1 -- 172.21.15.132:0/3311325849 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f193c0017c0 con 0x7f194c1361f0 2024-01-24T23:24:06.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:06.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:24:06.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 10 monclient: dump: 2024-01-24T23:24:06.754 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:06.754 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:06.755 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:06.755 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:06.755 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:06.755 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:06.755 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:06.755 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:06.755 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:06.755 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:06.755 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:06.755 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:06.755 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:06.755 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:06.755 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:06.755 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:06.755 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 5 monclient: authenticate success, global_id 5451 2024-01-24T23:24:06.756 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:06.756 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 1 -- 172.21.15.132:0/3311325849 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f193c0019e0 con 0x7f194c1361f0 2024-01-24T23:24:06.756 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:06.756 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 1 -- 172.21.15.132:0/3311325849 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f193c004bf0 con 0x7f194c1361f0 2024-01-24T23:24:06.756 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:06.756 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:24:06.756 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1950933640 10 monclient: dump: 2024-01-24T23:24:06.756 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:06.756 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:06.756 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:06.756 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:06.756 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:06.756 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:06.756 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:06.756 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:06.757 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:06.757 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:06.757 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:06.757 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:06.757 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:06.757 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:06.757 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:06.757 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:06.757 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:06.757 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 4 set_mon_vals no callback set 2024-01-24T23:24:06.757 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 10 monclient: shutdown 2024-01-24T23:24:06.757 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:06.757 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 1 -- 172.21.15.132:0/3311325849 >> v1:172.21.15.115:6792/0 conn(0x7f194c1361f0 legacy=0x7f194c13bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:06.757 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 1 -- 172.21.15.132:0/3311325849 shutdown_connections 2024-01-24T23:24:06.758 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 1 -- 172.21.15.132:0/3311325849 >> 172.21.15.132:0/3311325849 conn(0x7f194c0a6790 msgr2=0x7f194c071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:06.758 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 1 -- 172.21.15.132:0/3311325849 shutdown_connections 2024-01-24T23:24:06.758 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 1 -- 172.21.15.132:0/3311325849 wait complete. 2024-01-24T23:24:06.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 10 monclient: build_initial_monmap 2024-01-24T23:24:06.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 10 monclient: monmap: 2024-01-24T23:24:06.762 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:06.762 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:06.762 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:06.762 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:06.762 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:06.762 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:06.762 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:06.762 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:06.763 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:06.763 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:06.763 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:06.763 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:06.763 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:06.763 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:06.763 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:06.763 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:06.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 1 Processor -- start 2024-01-24T23:24:06.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 1 -- start start 2024-01-24T23:24:06.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 10 monclient: init 2024-01-24T23:24:06.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:06.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 10 monclient: _add_conns ranks=[4,6,2,0,1,7,3,8,5] 2024-01-24T23:24:06.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 10 monclient(hunting): picked mon.noname-c con 0x7f194c135e10 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:06.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 10 monclient(hunting): picked mon.noname-d con 0x7f194c1361f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:06.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 10 monclient(hunting): picked mon.noname-b con 0x7f194c1365f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:06.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c09c720 con 0x7f194c1365f0 2024-01-24T23:24:06.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c124830 con 0x7f194c135e10 2024-01-24T23:24:06.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c126280 con 0x7f194c1361f0 2024-01-24T23:24:06.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:06.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1953b99640 10 monclient(hunting): authenticate will time out at 1134.079526s 2024-01-24T23:24:06.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1951935640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f194c135e10 legacy=0x7f194c135120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:06.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1951134640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f194c1361f0 0x7f194c1d0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:57522/0 (socket says 172.21.15.132:57522) 2024-01-24T23:24:06.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.750+0000 7f1951134640 1 -- 172.21.15.132:0/3730500560 learned_addr learned my addr 172.21.15.132:0/3730500560 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:06.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 1 -- 172.21.15.132:0/3730500560 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3327634206 0 0) 0x7f1944003d80 con 0x7f194c1365f0 2024-01-24T23:24:06.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:06.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:06.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:06.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 10 monclient(hunting): my global_id is 84764 2024-01-24T23:24:06.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 1 -- 172.21.15.132:0/3730500560 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1930003530 con 0x7f194c1365f0 2024-01-24T23:24:06.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 1 -- 172.21.15.132:0/3730500560 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773552996 0 0) 0x7f194c126280 con 0x7f194c1361f0 2024-01-24T23:24:06.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:06.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:06.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:06.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 10 monclient(hunting): my global_id is 5460 2024-01-24T23:24:06.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 1 -- 172.21.15.132:0/3730500560 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1944003d80 con 0x7f194c1361f0 2024-01-24T23:24:06.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 1 -- 172.21.15.132:0/3730500560 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2927329737 0 0) 0x7f1930003530 con 0x7f194c1365f0 2024-01-24T23:24:06.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 1 -- 172.21.15.132:0/3730500560 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f194c126280 con 0x7f194c1365f0 2024-01-24T23:24:06.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 1 -- 172.21.15.132:0/3730500560 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f19440046f0 con 0x7f194c1365f0 2024-01-24T23:24:06.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:06.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:24:06.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 10 monclient(hunting): dump: 2024-01-24T23:24:06.765 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:06.765 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:06.765 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:06.765 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:06.766 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:06.766 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:06.766 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:06.766 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:06.766 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:06.766 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:06.766 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:06.766 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:06.766 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:06.766 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:06.766 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:06.766 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:06.766 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:06.766 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 1 -- 172.21.15.132:0/3730500560 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2411658728 0 0) 0x7f1944003d80 con 0x7f194c1361f0 2024-01-24T23:24:06.766 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 1 -- 172.21.15.132:0/3730500560 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1930003530 con 0x7f194c1361f0 2024-01-24T23:24:06.767 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 1 -- 172.21.15.132:0/3730500560 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f193c004b30 con 0x7f194c1361f0 2024-01-24T23:24:06.767 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:06.767 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:24:06.767 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 10 monclient(hunting): dump: 2024-01-24T23:24:06.767 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:06.767 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:06.767 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:06.767 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:06.767 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:06.767 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:06.767 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:06.767 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:06.767 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:06.767 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:06.767 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:06.768 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:06.768 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:06.768 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:06.768 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:06.768 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:06.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 1 -- 172.21.15.132:0/3730500560 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 648116726 0 0) 0x7f194c126280 con 0x7f194c1365f0 2024-01-24T23:24:06.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 1 -- 172.21.15.132:0/3730500560 >> v1:172.21.15.115:6792/0 conn(0x7f194c1361f0 legacy=0x7f194c1d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:06.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 1 -- 172.21.15.132:0/3730500560 >> v1:172.21.15.115:6791/0 conn(0x7f194c135e10 legacy=0x7f194c135120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:06.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 10 monclient: _finish_hunting 0 2024-01-24T23:24:06.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 1 monclient: found mon.b 2024-01-24T23:24:06.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:06.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:06.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 1 -- 172.21.15.132:0/3730500560 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c125390 con 0x7f194c1365f0 2024-01-24T23:24:06.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 10 monclient: _finish_auth 0 2024-01-24T23:24:06.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 10 monclient: _check_auth_tickets 2024-01-24T23:24:06.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:06.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f1953b99640 5 monclient: authenticate success, global_id 84764 2024-01-24T23:24:06.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f1953b99640 10 monclient: _renew_subs 2024-01-24T23:24:06.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f1953b99640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:06.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f1953b99640 1 -- 172.21.15.132:0/3730500560 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194c135830 con 0x7f194c1365f0 2024-01-24T23:24:06.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f1953b99640 10 monclient: _renew_subs 2024-01-24T23:24:06.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f1953b99640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:06.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f1953b99640 1 -- 172.21.15.132:0/3730500560 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f194c1258a0 con 0x7f194c1365f0 2024-01-24T23:24:06.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 1 -- 172.21.15.132:0/3730500560 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1944004960 con 0x7f194c1365f0 2024-01-24T23:24:06.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:06.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 1 -- 172.21.15.132:0/3730500560 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1944004e90 con 0x7f194c1365f0 2024-01-24T23:24:06.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:06.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19437fe640 4 set_mon_vals no callback set 2024-01-24T23:24:06.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:24:06.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 10 monclient: dump: 2024-01-24T23:24:06.770 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:06.770 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:06.770 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:06.770 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:06.770 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:06.770 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:06.770 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:06.770 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:06.770 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:06.770 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:06.770 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:06.771 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:06.771 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:06.771 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:06.771 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:06.771 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:06.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f1953b99640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:06.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f1953b99640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:06.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f1953b99640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:06.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f1953b99640 1 -- 172.21.15.132:0/3730500560 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1934005090 con 0x7f194c1365f0 2024-01-24T23:24:06.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 1 -- 172.21.15.132:0/3730500560 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f19440050b0 con 0x7f194c1365f0 2024-01-24T23:24:06.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f19427fc640 1 -- 172.21.15.132:0/3730500560 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(75..75 src has 1..75) v4 ==== 5181+0+0 (unknown 186118056 0 0) 0x7f1944043d30 con 0x7f194c1365f0 2024-01-24T23:24:06.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.754+0000 7f1951935640 10 monclient: get_auth_request con 0x7f19300376a0 auth_method 0 2024-01-24T23:24:06.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.766+0000 7f19427fc640 1 -- 172.21.15.132:0/3730500560 <== mon.2 v1:172.21.15.115:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f1944014b80 con 0x7f194c1365f0 2024-01-24T23:24:06.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.766+0000 7f19427fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:06.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.766+0000 7f19427fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:06.924 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.922+0000 7f1953b99640 10 start_mon_command cmd=[{"prefix": "osd tree-from", "bucket": "host1"}] 2024-01-24T23:24:06.924 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.922+0000 7f1953b99640 10 monclient: _send_command 2 [{"prefix": "osd tree-from", "bucket": "host1"}] 2024-01-24T23:24:06.924 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.922+0000 7f1953b99640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:06.924 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.922+0000 7f1953b99640 1 -- 172.21.15.132:0/3730500560 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd tree-from", "bucket": "host1"} v 0) v1 -- 0x7f1934005340 con 0x7f194c1365f0 2024-01-24T23:24:06.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.922+0000 7f19427fc640 1 -- 172.21.15.132:0/3730500560 <== mon.2 v1:172.21.15.115:6790/0 10 ==== mon_command_ack([{"prefix": "osd tree-from", "bucket": "host1"}]=-2 bucket 'host1' does not exist v75) v1 ==== 109+0+0 (unknown 1356360813 0 0) 0x7f1944012ff0 con 0x7f194c1365f0 2024-01-24T23:24:06.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.922+0000 7f19427fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree-from", "bucket": "host1"}] 2024-01-24T23:24:06.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:06.922+0000 7f19427fc640 10 monclient: _finish_command 2 = mon:2 bucket 'host1' does not exist 2024-01-24T23:24:06.925 INFO:tasks.workunit.client.0.smithi132.stderr:Error ENOENT: bucket 'host1' does not exist 2024-01-24T23:24:06.929 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:24:06.929 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd tree-from host2 2024-01-24T23:24:07.040 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309cc1d640 10 monclient: get_monmap_and_config 2024-01-24T23:24:07.041 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309cc1d640 10 monclient: build_initial_monmap 2024-01-24T23:24:07.041 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309cc1d640 10 monclient: monmap: 2024-01-24T23:24:07.041 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:07.041 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:07.041 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:07.042119+0000 2024-01-24T23:24:07.041 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:07.042119+0000 2024-01-24T23:24:07.041 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:07.041 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:07.041 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:07.041 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:07.041 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:07.041 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:07.041 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:07.041 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:07.041 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:07.042 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:07.042 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:07.042 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:07.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309cc1d640 1 Processor -- start 2024-01-24T23:24:07.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309cc1d640 1 -- start start 2024-01-24T23:24:07.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309cc1d640 10 monclient: init 2024-01-24T23:24:07.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309cc1d640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:07.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309cc1d640 10 monclient: _add_conns ranks=[6,3,0,1,2,5,7,8,4] 2024-01-24T23:24:07.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309cc1d640 10 monclient(hunting): picked mon.noname-d con 0x7f309809cd90 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:07.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309cc1d640 10 monclient(hunting): picked mon.noname-g con 0x7f309809ef80 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:07.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309cc1d640 10 monclient(hunting): picked mon.noname-a con 0x7f309809da90 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:07.049 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309cc1d640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3098064700 con 0x7f309809da90 2024-01-24T23:24:07.049 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309cc1d640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f309809d880 con 0x7f309809ef80 2024-01-24T23:24:07.049 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309cc1d640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f309809ed00 con 0x7f309809cd90 2024-01-24T23:24:07.049 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309cc1d640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:07.049 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309cc1d640 10 monclient(hunting): authenticate will time out at 1134.366771s 2024-01-24T23:24:07.049 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f3095d9b640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f309809ef80 0x7f3098139c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:38406/0 (socket says 172.21.15.132:38406) 2024-01-24T23:24:07.049 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f3095d9b640 1 -- 172.21.15.132:0/142380812 learned_addr learned my addr 172.21.15.132:0/142380812 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:07.049 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f3096d9d640 1 -- 172.21.15.132:0/142380812 >> v1:172.21.15.115:6789/0 conn(0x7f309809da90 legacy=0x7f309813d390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:07.049 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 1 -- 172.21.15.132:0/142380812 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942431596 0 0) 0x7f309809d880 con 0x7f309809ef80 2024-01-24T23:24:07.049 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 10 monclient(hunting): _negotiate 2024-01-24T23:24:07.049 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:07.049 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:07.049 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 10 monclient(hunting): my global_id is 5403 2024-01-24T23:24:07.049 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 1 -- 172.21.15.132:0/142380812 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3084003530 con 0x7f309809ef80 2024-01-24T23:24:07.049 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 1 -- 172.21.15.132:0/142380812 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3746798995 0 0) 0x7f3084003530 con 0x7f309809ef80 2024-01-24T23:24:07.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 1 -- 172.21.15.132:0/142380812 >> v1:172.21.15.115:6792/0 conn(0x7f309809cd90 legacy=0x7f309809d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:07.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 1 -- 172.21.15.132:0/142380812 >> v1:172.21.15.115:6789/0 conn(0x7f309809da90 legacy=0x7f309813d390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:07.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 10 monclient: _finish_hunting 0 2024-01-24T23:24:07.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 1 monclient: found mon.noname-g 2024-01-24T23:24:07.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:07.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:24:07.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 1 -- 172.21.15.132:0/142380812 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30980a0430 con 0x7f309809ef80 2024-01-24T23:24:07.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 10 monclient: _finish_auth 0 2024-01-24T23:24:07.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 10 monclient: _check_auth_tickets 2024-01-24T23:24:07.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:07.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 1 -- 172.21.15.132:0/142380812 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3090001b40 con 0x7f309809ef80 2024-01-24T23:24:07.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:07.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:07.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 10 monclient: dump: 2024-01-24T23:24:07.050 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:07.050 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:07.051 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.051 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.051 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:07.051 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:07.051 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:07.051 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:07.051 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:07.051 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:07.051 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:07.051 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:07.051 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:07.051 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:07.051 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:07.051 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:07.051 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309cc1d640 5 monclient: authenticate success, global_id 5403 2024-01-24T23:24:07.051 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309cc1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:07.051 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 1 -- 172.21.15.132:0/142380812 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3090001e60 con 0x7f309809ef80 2024-01-24T23:24:07.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:07.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 1 -- 172.21.15.132:0/142380812 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3090004f70 con 0x7f309809ef80 2024-01-24T23:24:07.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:07.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:07.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309559a640 10 monclient: dump: 2024-01-24T23:24:07.052 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:07.052 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:07.052 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.052 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.052 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:07.052 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:07.052 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:07.052 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:07.052 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:07.052 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:07.052 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:07.053 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:07.053 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:07.053 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:07.053 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:07.053 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:07.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309cc1d640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:07.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.038+0000 7f309cc1d640 4 set_mon_vals no callback set 2024-01-24T23:24:07.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 10 monclient: shutdown 2024-01-24T23:24:07.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:07.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 1 -- 172.21.15.132:0/142380812 >> v1:172.21.15.132:6790/0 conn(0x7f309809ef80 legacy=0x7f3098139c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:07.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 1 -- 172.21.15.132:0/142380812 shutdown_connections 2024-01-24T23:24:07.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 1 -- 172.21.15.132:0/142380812 >> 172.21.15.132:0/142380812 conn(0x7f30980a7930 msgr2=0x7f30980a3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:07.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 1 -- 172.21.15.132:0/142380812 shutdown_connections 2024-01-24T23:24:07.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 1 -- 172.21.15.132:0/142380812 wait complete. 2024-01-24T23:24:07.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 10 monclient: build_initial_monmap 2024-01-24T23:24:07.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 10 monclient: monmap: 2024-01-24T23:24:07.054 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:07.054 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:07.054 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:07.054 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:07.054 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:07.054 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:07.054 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:07.054 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:07.054 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:07.054 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:07.054 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:07.054 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:07.054 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:07.054 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:07.054 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:07.054 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:07.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 1 Processor -- start 2024-01-24T23:24:07.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 1 -- start start 2024-01-24T23:24:07.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 10 monclient: init 2024-01-24T23:24:07.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:07.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 10 monclient: _add_conns ranks=[1,7,5,2,3,4,0,8,6] 2024-01-24T23:24:07.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 10 monclient(hunting): picked mon.noname-f con 0x7f309809cd90 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:07.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 10 monclient(hunting): picked mon.noname-i con 0x7f309809da90 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:07.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 10 monclient(hunting): picked mon.noname-h con 0x7f3098133520 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:07.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f309809dee0 con 0x7f309809cd90 2024-01-24T23:24:07.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f309809f7d0 con 0x7f3098133520 2024-01-24T23:24:07.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3098071af0 con 0x7f309809da90 2024-01-24T23:24:07.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:07.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 10 monclient(hunting): authenticate will time out at 1134.369548s 2024-01-24T23:24:07.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309659c640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f309809cd90 legacy=0x7f3098132e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:07.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f3095d9b640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f309809da90 0x7f30981d04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:52296/0 (socket says 172.21.15.132:52296) 2024-01-24T23:24:07.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f3095d9b640 1 -- 172.21.15.132:0/723646586 learned_addr learned my addr 172.21.15.132:0/723646586 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:07.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 1 -- 172.21.15.132:0/723646586 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3999910789 0 0) 0x7f3098071af0 con 0x7f309809da90 2024-01-24T23:24:07.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:07.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:07.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:07.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 10 monclient(hunting): my global_id is 5488 2024-01-24T23:24:07.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 1 -- 172.21.15.132:0/723646586 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3078003530 con 0x7f309809da90 2024-01-24T23:24:07.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 1 -- 172.21.15.132:0/723646586 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 992638952 0 0) 0x7f309809f7d0 con 0x7f3098133520 2024-01-24T23:24:07.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:07.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:07.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:07.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 10 monclient(hunting): my global_id is 5414 2024-01-24T23:24:07.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 1 -- 172.21.15.132:0/723646586 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3098071af0 con 0x7f3098133520 2024-01-24T23:24:07.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 1 -- 172.21.15.132:0/723646586 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2678577684 0 0) 0x7f3078003530 con 0x7f309809da90 2024-01-24T23:24:07.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 1 -- 172.21.15.132:0/723646586 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f309809f7d0 con 0x7f309809da90 2024-01-24T23:24:07.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 1 -- 172.21.15.132:0/723646586 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3090001b00 con 0x7f309809da90 2024-01-24T23:24:07.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:07.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:07.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 10 monclient(hunting): dump: 2024-01-24T23:24:07.057 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:07.057 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:07.057 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.057 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.057 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:07.057 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:07.057 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:07.057 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:07.057 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:07.057 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:07.057 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:07.057 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:07.058 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:07.058 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:07.058 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:07.058 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:07.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:07.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 1 -- 172.21.15.132:0/723646586 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1652145565 0 0) 0x7f3098071af0 con 0x7f3098133520 2024-01-24T23:24:07.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 1 -- 172.21.15.132:0/723646586 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3078003530 con 0x7f3098133520 2024-01-24T23:24:07.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 1 -- 172.21.15.132:0/723646586 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f30800041e0 con 0x7f3098133520 2024-01-24T23:24:07.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:07.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:07.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 10 monclient(hunting): dump: 2024-01-24T23:24:07.058 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:07.058 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:07.058 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.058 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.059 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:07.059 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:07.059 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:07.059 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:07.059 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:07.059 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:07.059 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:07.059 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:07.059 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:07.059 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:07.059 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:07.059 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:07.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 1 -- 172.21.15.132:0/723646586 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 190844489 0 0) 0x7f309809f7d0 con 0x7f309809da90 2024-01-24T23:24:07.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 1 -- 172.21.15.132:0/723646586 >> v1:172.21.15.132:6791/0 conn(0x7f3098133520 legacy=0x7f30981d3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:07.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 1 -- 172.21.15.132:0/723646586 >> v1:172.21.15.132:6789/0 conn(0x7f309809cd90 legacy=0x7f3098132e10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:07.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:07.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 1 monclient: found mon.i 2024-01-24T23:24:07.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:07.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:07.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 1 -- 172.21.15.132:0/723646586 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3098121f50 con 0x7f309809da90 2024-01-24T23:24:07.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 10 monclient: _finish_auth 0 2024-01-24T23:24:07.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:07.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:07.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 5 monclient: authenticate success, global_id 5488 2024-01-24T23:24:07.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 10 monclient: _renew_subs 2024-01-24T23:24:07.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:07.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 1 -- 172.21.15.132:0/723646586 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f309806ce20 con 0x7f309809da90 2024-01-24T23:24:07.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 10 monclient: _renew_subs 2024-01-24T23:24:07.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:07.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 1 -- 172.21.15.132:0/723646586 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f30981c8180 con 0x7f309809da90 2024-01-24T23:24:07.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 1 -- 172.21.15.132:0/723646586 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3090004f90 con 0x7f309809da90 2024-01-24T23:24:07.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:07.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 1 -- 172.21.15.132:0/723646586 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3090001560 con 0x7f309809da90 2024-01-24T23:24:07.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:07.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f3094d99640 4 set_mon_vals no callback set 2024-01-24T23:24:07.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:07.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 10 monclient: dump: 2024-01-24T23:24:07.061 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:07.061 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:07.061 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.061 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.061 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:07.061 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:07.061 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:07.061 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:07.062 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:07.062 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:07.062 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:07.062 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:07.062 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:07.062 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:07.062 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:07.062 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:07.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:07.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:07.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:07.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309cc1d640 1 -- 172.21.15.132:0/723646586 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f307c005090 con 0x7f309809da90 2024-01-24T23:24:07.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 1 -- 172.21.15.132:0/723646586 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f30900100b0 con 0x7f309809da90 2024-01-24T23:24:07.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f308effd640 1 -- 172.21.15.132:0/723646586 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(75..75 src has 1..75) v4 ==== 5181+0+0 (unknown 186118056 0 0) 0x7f30900441d0 con 0x7f309809da90 2024-01-24T23:24:07.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.042+0000 7f309659c640 10 monclient: get_auth_request con 0x7f3078037740 auth_method 0 2024-01-24T23:24:07.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.046+0000 7f308effd640 1 -- 172.21.15.132:0/723646586 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f3090014f80 con 0x7f309809da90 2024-01-24T23:24:07.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.046+0000 7f308effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:07.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.046+0000 7f308effd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:07.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.206+0000 7f309cc1d640 10 start_mon_command cmd=[{"prefix": "osd tree-from", "bucket": "host2"}] 2024-01-24T23:24:07.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.206+0000 7f309cc1d640 10 monclient: _send_command 2 [{"prefix": "osd tree-from", "bucket": "host2"}] 2024-01-24T23:24:07.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.206+0000 7f309cc1d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:07.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.206+0000 7f309cc1d640 1 -- 172.21.15.132:0/723646586 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd tree-from", "bucket": "host2"} v 0) v1 -- 0x7f307c005340 con 0x7f309809da90 2024-01-24T23:24:07.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.206+0000 7f308effd640 1 -- 172.21.15.132:0/723646586 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd tree-from", "bucket": "host2"}]=0 v75) v1 ==== 80+0+177 (unknown 4200864958 0 2845383371) 0x7f30900133f0 con 0x7f309809da90 2024-01-24T23:24:07.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.206+0000 7f308effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree-from", "bucket": "host2"}] 2024-01-24T23:24:07.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.206+0000 7f308effd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:24:07.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.298+0000 7f309cc1d640 10 start_mon_command cmd=[{"prefix": "osd tree-from", "bucket": "host2"}] 2024-01-24T23:24:07.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.298+0000 7f309cc1d640 10 monclient: _send_command 3 [{"prefix": "osd tree-from", "bucket": "host2"}] 2024-01-24T23:24:07.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.298+0000 7f309cc1d640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:07.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.298+0000 7f309cc1d640 1 -- 172.21.15.132:0/723646586 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd tree-from", "bucket": "host2"} v 0) v1 -- 0x7f307c002220 con 0x7f309809da90 2024-01-24T23:24:07.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.298+0000 7f308effd640 1 -- 172.21.15.132:0/723646586 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd tree-from", "bucket": "host2"}]=0 v75) v1 ==== 80+0+177 (unknown 4200864958 0 2845383371) 0x7f3090014d50 con 0x7f309809da90 2024-01-24T23:24:07.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.298+0000 7f308effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree-from", "bucket": "host2"}] 2024-01-24T23:24:07.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.298+0000 7f308effd640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:24:07.303 INFO:tasks.workunit.client.0.smithi132.stdout:ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF 2024-01-24T23:24:07.303 INFO:tasks.workunit.client.0.smithi132.stdout:-16 1.00000 host host2 2024-01-24T23:24:07.303 INFO:tasks.workunit.client.0.smithi132.stdout: 9 1.00000 osd.9 down 1.00000 1.00000 2024-01-24T23:24:07.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.302+0000 7f309cc1d640 1 -- 172.21.15.132:0/723646586 >> v1:172.21.15.132:6812/1835401922 conn(0x7f3078037740 legacy=0x7f3078039c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:07.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.302+0000 7f309cc1d640 10 monclient: shutdown 2024-01-24T23:24:07.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.302+0000 7f309cc1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:07.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.302+0000 7f309cc1d640 1 -- 172.21.15.132:0/723646586 >> v1:172.21.15.132:6792/0 conn(0x7f309809da90 legacy=0x7f30981d04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:07.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.302+0000 7f309cc1d640 1 -- 172.21.15.132:0/723646586 shutdown_connections 2024-01-24T23:24:07.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.302+0000 7f309cc1d640 1 -- 172.21.15.132:0/723646586 >> 172.21.15.132:0/723646586 conn(0x7f30980a7930 msgr2=0x7f309809bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:07.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.302+0000 7f309cc1d640 1 -- 172.21.15.132:0/723646586 shutdown_connections 2024-01-24T23:24:07.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.302+0000 7f309cc1d640 1 -- 172.21.15.132:0/723646586 wait complete. 2024-01-24T23:24:07.320 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd tree-from osd.9 2024-01-24T23:24:07.320 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:24:07.320 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd tree-from osd.9 2024-01-24T23:24:07.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 10 monclient: get_monmap_and_config 2024-01-24T23:24:07.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 10 monclient: build_initial_monmap 2024-01-24T23:24:07.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 10 monclient: monmap: 2024-01-24T23:24:07.432 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:07.432 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:07.432 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:07.432990+0000 2024-01-24T23:24:07.432 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:07.432990+0000 2024-01-24T23:24:07.432 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:07.432 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:07.432 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:07.432 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:07.432 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:07.433 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:07.433 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:07.433 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:07.433 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:07.433 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:07.433 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:07.433 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:07.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 1 Processor -- start 2024-01-24T23:24:07.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 1 -- start start 2024-01-24T23:24:07.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 10 monclient: init 2024-01-24T23:24:07.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:07.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 10 monclient: _add_conns ranks=[3,4,1,0,6,8,2,5,7] 2024-01-24T23:24:07.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 10 monclient(hunting): picked mon.noname-g con 0x7ff998135e10 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:07.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 10 monclient(hunting): picked mon.noname-c con 0x7ff9981361f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:07.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 10 monclient(hunting): picked mon.noname-f con 0x7ff9981365f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:07.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff998064700 con 0x7ff9981365f0 2024-01-24T23:24:07.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff99809c3e0 con 0x7ff998135e10 2024-01-24T23:24:07.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9980a4f90 con 0x7ff9981361f0 2024-01-24T23:24:07.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:07.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 10 monclient(hunting): authenticate will time out at 1134.757577s 2024-01-24T23:24:07.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99e080640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7ff9981365f0 legacy=0x7ff99813f600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:07.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99d87f640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7ff998135e10 0x7ff99809bcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:38422/0 (socket says 172.21.15.132:38422) 2024-01-24T23:24:07.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99d87f640 1 -- 172.21.15.132:0/970000244 learned_addr learned my addr 172.21.15.132:0/970000244 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:07.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99d07e640 1 -- 172.21.15.132:0/970000244 >> v1:172.21.15.115:6791/0 conn(0x7ff9981361f0 legacy=0x7ff99813bed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:07.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 1 -- 172.21.15.132:0/970000244 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2207404108 0 0) 0x7ff99809c3e0 con 0x7ff998135e10 2024-01-24T23:24:07.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 10 monclient(hunting): _negotiate 2024-01-24T23:24:07.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:07.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:07.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 10 monclient(hunting): my global_id is 5412 2024-01-24T23:24:07.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 1 -- 172.21.15.132:0/970000244 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff984003530 con 0x7ff998135e10 2024-01-24T23:24:07.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 1 -- 172.21.15.132:0/970000244 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 283362803 0 0) 0x7ff984003530 con 0x7ff998135e10 2024-01-24T23:24:07.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 1 -- 172.21.15.132:0/970000244 >> v1:172.21.15.115:6791/0 conn(0x7ff9981361f0 legacy=0x7ff99813bed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:07.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 1 -- 172.21.15.132:0/970000244 >> v1:172.21.15.132:6789/0 conn(0x7ff9981365f0 legacy=0x7ff99813f600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:07.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 10 monclient: _finish_hunting 0 2024-01-24T23:24:07.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 1 monclient: found mon.noname-g 2024-01-24T23:24:07.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:07.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:24:07.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 1 -- 172.21.15.132:0/970000244 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99809b010 con 0x7ff998135e10 2024-01-24T23:24:07.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 10 monclient: _finish_auth 0 2024-01-24T23:24:07.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 10 monclient: _check_auth_tickets 2024-01-24T23:24:07.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:07.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 5 monclient: authenticate success, global_id 5412 2024-01-24T23:24:07.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:07.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 1 -- 172.21.15.132:0/970000244 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff98c0017c0 con 0x7ff998135e10 2024-01-24T23:24:07.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:07.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:07.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 10 monclient: dump: 2024-01-24T23:24:07.444 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:07.444 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:07.444 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.444 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.444 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:07.444 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:07.445 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:07.445 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:07.445 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:07.445 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:07.445 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:07.445 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:07.445 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:07.445 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:07.445 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:07.445 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:07.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:07.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 1 -- 172.21.15.132:0/970000244 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff98c0019e0 con 0x7ff998135e10 2024-01-24T23:24:07.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:07.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 1 -- 172.21.15.132:0/970000244 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff98c004bf0 con 0x7ff998135e10 2024-01-24T23:24:07.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:07.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:07.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 4 set_mon_vals no callback set 2024-01-24T23:24:07.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:07.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99c87d640 10 monclient: dump: 2024-01-24T23:24:07.446 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:07.446 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:07.446 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.446 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.446 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:07.446 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:07.446 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:07.446 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:07.446 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:07.446 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:07.446 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:07.446 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:07.447 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:07.447 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:07.447 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:07.447 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:07.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 10 monclient: shutdown 2024-01-24T23:24:07.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:07.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 1 -- 172.21.15.132:0/970000244 >> v1:172.21.15.132:6790/0 conn(0x7ff998135e10 legacy=0x7ff99809bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:07.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 1 -- 172.21.15.132:0/970000244 shutdown_connections 2024-01-24T23:24:07.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 1 -- 172.21.15.132:0/970000244 >> 172.21.15.132:0/970000244 conn(0x7ff9980a6790 msgr2=0x7ff998071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:07.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 1 -- 172.21.15.132:0/970000244 shutdown_connections 2024-01-24T23:24:07.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 1 -- 172.21.15.132:0/970000244 wait complete. 2024-01-24T23:24:07.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 10 monclient: build_initial_monmap 2024-01-24T23:24:07.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 10 monclient: monmap: 2024-01-24T23:24:07.447 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:07.447 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:07.447 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:07.448 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:07.448 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:07.448 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:07.448 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:07.448 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:07.448 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:07.448 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:07.448 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:07.448 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:07.448 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:07.448 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:07.448 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:07.448 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:07.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 1 Processor -- start 2024-01-24T23:24:07.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 1 -- start start 2024-01-24T23:24:07.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.430+0000 7ff99fae3640 10 monclient: init 2024-01-24T23:24:07.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99fae3640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:07.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99fae3640 10 monclient: _add_conns ranks=[0,8,2,6,4,1,3,5,7] 2024-01-24T23:24:07.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99fae3640 10 monclient(hunting): picked mon.noname-a con 0x7ff998135e10 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:07.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99fae3640 10 monclient(hunting): picked mon.noname-e con 0x7ff9981361f0 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:07.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99fae3640 10 monclient(hunting): picked mon.noname-b con 0x7ff9981365f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:07.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99fae3640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9981244a0 con 0x7ff998135e10 2024-01-24T23:24:07.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99fae3640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff998126790 con 0x7ff9981365f0 2024-01-24T23:24:07.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99fae3640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff99809c6b0 con 0x7ff9981361f0 2024-01-24T23:24:07.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99fae3640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:07.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99fae3640 10 monclient(hunting): authenticate will time out at 1134.760459s 2024-01-24T23:24:07.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99d87f640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7ff998135e10 legacy=0x7ff998134d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:07.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99d07e640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7ff9981361f0 0x7ff9981d00a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:55906/0 (socket says 172.21.15.132:55906) 2024-01-24T23:24:07.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99d07e640 1 -- 172.21.15.132:0/2337335488 learned_addr learned my addr 172.21.15.132:0/2337335488 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:07.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 1 -- 172.21.15.132:0/2337335488 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3301694980 0 0) 0x7ff99809c6b0 con 0x7ff9981361f0 2024-01-24T23:24:07.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:07.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:07.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:07.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 10 monclient(hunting): my global_id is 84626 2024-01-24T23:24:07.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 1 -- 172.21.15.132:0/2337335488 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff97c003530 con 0x7ff9981361f0 2024-01-24T23:24:07.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 1 -- 172.21.15.132:0/2337335488 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 554335597 0 0) 0x7ff998126790 con 0x7ff9981365f0 2024-01-24T23:24:07.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:07.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:07.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:07.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 10 monclient(hunting): my global_id is 84773 2024-01-24T23:24:07.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 1 -- 172.21.15.132:0/2337335488 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff99809c6b0 con 0x7ff9981365f0 2024-01-24T23:24:07.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 1 -- 172.21.15.132:0/2337335488 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 645294823 0 0) 0x7ff97c003530 con 0x7ff9981361f0 2024-01-24T23:24:07.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 1 -- 172.21.15.132:0/2337335488 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff998126790 con 0x7ff9981361f0 2024-01-24T23:24:07.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 1 -- 172.21.15.132:0/2337335488 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff994004700 con 0x7ff9981361f0 2024-01-24T23:24:07.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:07.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:24:07.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 10 monclient(hunting): dump: 2024-01-24T23:24:07.451 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:07.451 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:07.451 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.451 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.451 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:07.451 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:07.451 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:07.451 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:07.451 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:07.451 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:07.451 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:07.451 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:07.451 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:07.451 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:07.452 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:07.452 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:07.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:07.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 1 -- 172.21.15.132:0/2337335488 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 507352016 0 0) 0x7ff99809c6b0 con 0x7ff9981365f0 2024-01-24T23:24:07.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 1 -- 172.21.15.132:0/2337335488 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff97c003530 con 0x7ff9981365f0 2024-01-24T23:24:07.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 1 -- 172.21.15.132:0/2337335488 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff990004160 con 0x7ff9981365f0 2024-01-24T23:24:07.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:07.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:24:07.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 10 monclient(hunting): dump: 2024-01-24T23:24:07.452 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:07.452 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:07.452 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.452 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.452 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:07.452 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:07.453 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:07.453 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:07.453 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:07.453 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:07.453 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:07.453 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:07.453 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:07.453 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:07.453 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:07.453 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:07.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 1 -- 172.21.15.132:0/2337335488 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2695479372 0 0) 0x7ff998126790 con 0x7ff9981361f0 2024-01-24T23:24:07.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 1 -- 172.21.15.132:0/2337335488 >> v1:172.21.15.115:6790/0 conn(0x7ff9981365f0 legacy=0x7ff9981d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:07.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 1 -- 172.21.15.132:0/2337335488 >> v1:172.21.15.115:6789/0 conn(0x7ff998135e10 legacy=0x7ff998134d90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:07.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:24:07.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 1 monclient: found mon.e 2024-01-24T23:24:07.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:07.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:07.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 1 -- 172.21.15.132:0/2337335488 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99809ca00 con 0x7ff9981361f0 2024-01-24T23:24:07.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 10 monclient: _finish_auth 0 2024-01-24T23:24:07.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:24:07.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:07.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99fae3640 5 monclient: authenticate success, global_id 84626 2024-01-24T23:24:07.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99fae3640 10 monclient: _renew_subs 2024-01-24T23:24:07.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99fae3640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:07.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99fae3640 1 -- 172.21.15.132:0/2337335488 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff998124920 con 0x7ff9981361f0 2024-01-24T23:24:07.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99fae3640 10 monclient: _renew_subs 2024-01-24T23:24:07.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99fae3640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:07.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99fae3640 1 -- 172.21.15.132:0/2337335488 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9981354a0 con 0x7ff9981361f0 2024-01-24T23:24:07.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 1 -- 172.21.15.132:0/2337335488 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff994004920 con 0x7ff9981361f0 2024-01-24T23:24:07.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:07.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98bfff640 4 set_mon_vals no callback set 2024-01-24T23:24:07.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 1 -- 172.21.15.132:0/2337335488 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff994004dd0 con 0x7ff9981361f0 2024-01-24T23:24:07.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:07.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:24:07.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 10 monclient: dump: 2024-01-24T23:24:07.455 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:07.455 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:07.455 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.455 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.455 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:07.455 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:07.455 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:07.455 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:07.455 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:07.455 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:07.455 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:07.455 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:07.456 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:07.456 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:07.456 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:07.456 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:07.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99fae3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:07.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99fae3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:07.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99fae3640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:07.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99fae3640 1 -- 172.21.15.132:0/2337335488 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff980005090 con 0x7ff9981361f0 2024-01-24T23:24:07.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 1 -- 172.21.15.132:0/2337335488 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7ff994004fb0 con 0x7ff9981361f0 2024-01-24T23:24:07.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff98a7fc640 1 -- 172.21.15.132:0/2337335488 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(75..75 src has 1..75) v4 ==== 5181+0+0 (unknown 186118056 0 0) 0x7ff994044070 con 0x7ff9981361f0 2024-01-24T23:24:07.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.434+0000 7ff99d87f640 10 monclient: get_auth_request con 0x7ff97c0376f0 auth_method 0 2024-01-24T23:24:07.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.446+0000 7ff98a7fc640 1 -- 172.21.15.132:0/2337335488 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7ff994014d00 con 0x7ff9981361f0 2024-01-24T23:24:07.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.446+0000 7ff98a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:07.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.446+0000 7ff98a7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:07.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.590+0000 7ff99fae3640 10 start_mon_command cmd=[{"prefix": "osd tree-from", "bucket": "osd.9"}] 2024-01-24T23:24:07.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.590+0000 7ff99fae3640 10 monclient: _send_command 2 [{"prefix": "osd tree-from", "bucket": "osd.9"}] 2024-01-24T23:24:07.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.590+0000 7ff99fae3640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:07.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.590+0000 7ff99fae3640 1 -- 172.21.15.132:0/2337335488 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd tree-from", "bucket": "osd.9"} v 0) v1 -- 0x7ff980005340 con 0x7ff9981361f0 2024-01-24T23:24:07.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.594+0000 7ff98a7fc640 1 -- 172.21.15.132:0/2337335488 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd tree-from", "bucket": "osd.9"}]=-22 "osd.9" is not a bucket v75) v1 ==== 103+0+0 (unknown 3937340994 0 0) 0x7ff994013170 con 0x7ff9981361f0 2024-01-24T23:24:07.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.594+0000 7ff98a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree-from", "bucket": "osd.9"}] 2024-01-24T23:24:07.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.594+0000 7ff98a7fc640 10 monclient: _finish_command 2 = mon:22 "osd.9" is not a bucket 2024-01-24T23:24:07.596 INFO:tasks.workunit.client.0.smithi132.stderr:Error EINVAL: "osd.9" is not a bucket 2024-01-24T23:24:07.600 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:24:07.600 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd crush rm bar 2024-01-24T23:24:07.600 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:24:07.600 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm bar 2024-01-24T23:24:07.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde348fc640 10 monclient: get_monmap_and_config 2024-01-24T23:24:07.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde348fc640 10 monclient: build_initial_monmap 2024-01-24T23:24:07.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde348fc640 10 monclient: monmap: 2024-01-24T23:24:07.717 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:07.717 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:07.717 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:07.718004+0000 2024-01-24T23:24:07.717 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:07.718004+0000 2024-01-24T23:24:07.717 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:07.717 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:07.718 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:07.718 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:07.718 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:07.718 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:07.718 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:07.718 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:07.718 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:07.718 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:07.718 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:07.718 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:07.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde348fc640 1 Processor -- start 2024-01-24T23:24:07.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde348fc640 1 -- start start 2024-01-24T23:24:07.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde348fc640 10 monclient: init 2024-01-24T23:24:07.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde348fc640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:07.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde348fc640 10 monclient: _add_conns ranks=[7,5,8,1,6,3,0,2,4] 2024-01-24T23:24:07.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde348fc640 10 monclient(hunting): picked mon.noname-i con 0x7fde300a0ad0 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:07.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde348fc640 10 monclient(hunting): picked mon.noname-h con 0x7fde300a15c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:07.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde348fc640 10 monclient(hunting): picked mon.noname-e con 0x7fde3013d9a0 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:07.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde348fc640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde30064700 con 0x7fde300a15c0 2024-01-24T23:24:07.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde348fc640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde3009a7b0 con 0x7fde300a0ad0 2024-01-24T23:24:07.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde348fc640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde3009ad70 con 0x7fde3013d9a0 2024-01-24T23:24:07.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde348fc640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:07.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde348fc640 10 monclient(hunting): authenticate will time out at 1135.042990s 2024-01-24T23:24:07.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2e59c640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fde300a0ad0 0x7fde300a0eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:52312/0 (socket says 172.21.15.132:52312) 2024-01-24T23:24:07.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2e59c640 1 -- 172.21.15.132:0/2690199669 learned_addr learned my addr 172.21.15.132:0/2690199669 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:07.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 1 -- 172.21.15.132:0/2690199669 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2989380892 0 0) 0x7fde3009a7b0 con 0x7fde300a0ad0 2024-01-24T23:24:07.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 10 monclient(hunting): _negotiate 2024-01-24T23:24:07.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:07.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:07.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 10 monclient(hunting): my global_id is 5497 2024-01-24T23:24:07.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 1 -- 172.21.15.132:0/2690199669 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde18003530 con 0x7fde300a0ad0 2024-01-24T23:24:07.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 1 -- 172.21.15.132:0/2690199669 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1805562652 0 0) 0x7fde30064700 con 0x7fde300a15c0 2024-01-24T23:24:07.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 10 monclient(hunting): _negotiate 2024-01-24T23:24:07.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:07.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:07.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 10 monclient(hunting): my global_id is 5423 2024-01-24T23:24:07.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 1 -- 172.21.15.132:0/2690199669 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde3009a7b0 con 0x7fde300a15c0 2024-01-24T23:24:07.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 1 -- 172.21.15.132:0/2690199669 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 769047314 0 0) 0x7fde18003530 con 0x7fde300a0ad0 2024-01-24T23:24:07.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 1 -- 172.21.15.132:0/2690199669 >> v1:172.21.15.115:6793/0 conn(0x7fde3013d9a0 legacy=0x7fde3013fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:07.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 1 -- 172.21.15.132:0/2690199669 >> v1:172.21.15.132:6791/0 conn(0x7fde300a15c0 legacy=0x7fde3013c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:07.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 10 monclient: _finish_hunting 0 2024-01-24T23:24:07.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 1 monclient: found mon.noname-i 2024-01-24T23:24:07.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:07.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:24:07.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 1 -- 172.21.15.132:0/2690199669 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3009d050 con 0x7fde300a0ad0 2024-01-24T23:24:07.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 10 monclient: _finish_auth 0 2024-01-24T23:24:07.728 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 10 monclient: _check_auth_tickets 2024-01-24T23:24:07.728 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:07.728 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 1 -- 172.21.15.132:0/2690199669 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fde200017c0 con 0x7fde300a0ad0 2024-01-24T23:24:07.728 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:07.728 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:07.728 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde2d59a640 10 monclient: dump: 2024-01-24T23:24:07.728 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:07.728 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:07.728 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.728 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.728 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:07.728 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:07.728 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:07.728 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:07.728 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:07.729 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:07.729 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:07.729 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:07.729 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:07.729 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:07.729 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:07.729 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:07.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde348fc640 5 monclient: authenticate success, global_id 5497 2024-01-24T23:24:07.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.714+0000 7fde348fc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:07.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde2d59a640 1 -- 172.21.15.132:0/2690199669 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fde200019e0 con 0x7fde300a0ad0 2024-01-24T23:24:07.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde2d59a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:07.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde2d59a640 1 -- 172.21.15.132:0/2690199669 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fde20004bf0 con 0x7fde300a0ad0 2024-01-24T23:24:07.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde2d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:07.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde2d59a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:07.729 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde2d59a640 10 monclient: dump: 2024-01-24T23:24:07.730 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:07.730 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:07.730 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.730 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.730 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:07.730 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:07.730 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:07.730 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:07.730 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:07.730 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:07.730 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:07.730 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:07.730 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:07.730 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:07.731 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:07.731 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:07.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:07.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 4 set_mon_vals no callback set 2024-01-24T23:24:07.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 10 monclient: shutdown 2024-01-24T23:24:07.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:07.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 1 -- 172.21.15.132:0/2690199669 >> v1:172.21.15.132:6792/0 conn(0x7fde300a0ad0 legacy=0x7fde300a0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:07.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 1 -- 172.21.15.132:0/2690199669 shutdown_connections 2024-01-24T23:24:07.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 1 -- 172.21.15.132:0/2690199669 >> 172.21.15.132:0/2690199669 conn(0x7fde300a62a0 msgr2=0x7fde300a66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:07.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 1 -- 172.21.15.132:0/2690199669 shutdown_connections 2024-01-24T23:24:07.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 1 -- 172.21.15.132:0/2690199669 wait complete. 2024-01-24T23:24:07.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 10 monclient: build_initial_monmap 2024-01-24T23:24:07.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 10 monclient: monmap: 2024-01-24T23:24:07.732 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:07.732 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:07.732 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:07.732 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:07.732 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:07.732 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:07.732 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:07.732 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:07.732 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:07.732 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:07.732 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:07.733 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:07.733 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:07.733 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:07.733 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:07.733 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:07.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 1 Processor -- start 2024-01-24T23:24:07.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 1 -- start start 2024-01-24T23:24:07.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 10 monclient: init 2024-01-24T23:24:07.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:07.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 10 monclient: _add_conns ranks=[0,3,4,8,7,1,6,2,5] 2024-01-24T23:24:07.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 10 monclient(hunting): picked mon.noname-a con 0x7fde300a0ad0 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:07.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 10 monclient(hunting): picked mon.noname-g con 0x7fde300a15c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:07.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 10 monclient(hunting): picked mon.noname-c con 0x7fde3013d9a0 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:07.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde300a28b0 con 0x7fde300a0ad0 2024-01-24T23:24:07.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde3013cb80 con 0x7fde300a15c0 2024-01-24T23:24:07.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde3013c320 con 0x7fde3013d9a0 2024-01-24T23:24:07.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:07.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 10 monclient(hunting): authenticate will time out at 1135.046409s 2024-01-24T23:24:07.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde2e59c640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fde300a0ad0 legacy=0x7fde301cbdb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:07.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde2dd9b640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fde300a15c0 0x7fde301d0f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:38434/0 (socket says 172.21.15.132:38434) 2024-01-24T23:24:07.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde2dd9b640 1 -- 172.21.15.132:0/2330539950 learned_addr learned my addr 172.21.15.132:0/2330539950 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:07.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde2ed9d640 1 -- 172.21.15.132:0/2330539950 >> v1:172.21.15.115:6791/0 conn(0x7fde3013d9a0 legacy=0x7fde301d46a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:07.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 1 -- 172.21.15.132:0/2330539950 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1597711495 0 0) 0x7fde3013cb80 con 0x7fde300a15c0 2024-01-24T23:24:07.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:07.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:07.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:07.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 10 monclient(hunting): my global_id is 5421 2024-01-24T23:24:07.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 1 -- 172.21.15.132:0/2330539950 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde10003530 con 0x7fde300a15c0 2024-01-24T23:24:07.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 1 -- 172.21.15.132:0/2330539950 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 852360839 0 0) 0x7fde10003530 con 0x7fde300a15c0 2024-01-24T23:24:07.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 1 -- 172.21.15.132:0/2330539950 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fde3013cb80 con 0x7fde300a15c0 2024-01-24T23:24:07.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 1 -- 172.21.15.132:0/2330539950 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fde24001ac0 con 0x7fde300a15c0 2024-01-24T23:24:07.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:07.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:07.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 10 monclient(hunting): dump: 2024-01-24T23:24:07.735 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:07.735 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:07.735 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.735 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.735 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:07.735 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:07.735 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:07.735 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:07.735 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:07.735 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:07.736 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:07.736 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:07.736 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:07.736 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:07.736 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:07.736 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:07.736 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:07.736 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 1 -- 172.21.15.132:0/2330539950 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2881270388 0 0) 0x7fde3013cb80 con 0x7fde300a15c0 2024-01-24T23:24:07.736 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 1 -- 172.21.15.132:0/2330539950 >> v1:172.21.15.115:6791/0 conn(0x7fde3013d9a0 legacy=0x7fde301d46a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:07.736 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 1 -- 172.21.15.132:0/2330539950 >> v1:172.21.15.115:6789/0 conn(0x7fde300a0ad0 legacy=0x7fde301cbdb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:07.736 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:07.736 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 1 monclient: found mon.g 2024-01-24T23:24:07.736 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:07.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:07.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 1 -- 172.21.15.132:0/2330539950 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde300a2360 con 0x7fde300a15c0 2024-01-24T23:24:07.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 10 monclient: _finish_auth 0 2024-01-24T23:24:07.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:07.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:07.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 5 monclient: authenticate success, global_id 5421 2024-01-24T23:24:07.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 10 monclient: _renew_subs 2024-01-24T23:24:07.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:07.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 1 -- 172.21.15.132:0/2330539950 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde3013c8a0 con 0x7fde300a15c0 2024-01-24T23:24:07.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 10 monclient: _renew_subs 2024-01-24T23:24:07.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:07.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde348fc640 1 -- 172.21.15.132:0/2330539950 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fde3006eb10 con 0x7fde300a15c0 2024-01-24T23:24:07.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 1 -- 172.21.15.132:0/2330539950 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fde240038d0 con 0x7fde300a15c0 2024-01-24T23:24:07.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:07.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 1 -- 172.21.15.132:0/2330539950 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fde24003e00 con 0x7fde300a15c0 2024-01-24T23:24:07.738 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:07.738 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde2cd99640 4 set_mon_vals no callback set 2024-01-24T23:24:07.738 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:07.738 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.718+0000 7fde1effd640 10 monclient: dump: 2024-01-24T23:24:07.738 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:07.738 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:07.738 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.738 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:07.738 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:07.738 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:07.738 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:07.738 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:07.738 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:07.738 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:07.738 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:07.739 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:07.739 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:07.739 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:07.739 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:07.739 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:07.739 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.722+0000 7fde348fc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:07.739 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.722+0000 7fde348fc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:07.739 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.722+0000 7fde348fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:07.739 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.722+0000 7fde348fc640 1 -- 172.21.15.132:0/2330539950 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde14005090 con 0x7fde300a15c0 2024-01-24T23:24:07.739 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.722+0000 7fde1effd640 1 -- 172.21.15.132:0/2330539950 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fde24003fe0 con 0x7fde300a15c0 2024-01-24T23:24:07.739 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.722+0000 7fde1effd640 1 -- 172.21.15.132:0/2330539950 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(75..75 src has 1..75) v4 ==== 5181+0+0 (unknown 186118056 0 0) 0x7fde24043e90 con 0x7fde300a15c0 2024-01-24T23:24:07.739 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.722+0000 7fde2e59c640 10 monclient: get_auth_request con 0x7fde10036ef0 auth_method 0 2024-01-24T23:24:07.740 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.726+0000 7fde1effd640 1 -- 172.21.15.132:0/2330539950 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fde24014b80 con 0x7fde300a15c0 2024-01-24T23:24:07.740 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.726+0000 7fde1effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:07.740 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.726+0000 7fde1effd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:07.886 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.882+0000 7fde348fc640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "bar"}] 2024-01-24T23:24:07.886 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.882+0000 7fde348fc640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "bar"}] 2024-01-24T23:24:07.886 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.882+0000 7fde348fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:07.886 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.882+0000 7fde348fc640 1 -- 172.21.15.132:0/2330539950 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rm", "name": "bar"} v 0) v1 -- 0x7fde14005340 con 0x7fde300a15c0 2024-01-24T23:24:07.887 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.882+0000 7fde1effd640 1 -- 172.21.15.132:0/2330539950 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "bar"}]=-39 (39) Directory not empty v75) v1 ==== 99+0+0 (unknown 2573910478 0 0) 0x7fde2401d0e0 con 0x7fde300a15c0 2024-01-24T23:24:07.887 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.882+0000 7fde1effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "bar"}] 2024-01-24T23:24:07.887 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:07.882+0000 7fde1effd640 10 monclient: _finish_command 2 = mon:39 (39) Directory not empty 2024-01-24T23:24:07.887 INFO:tasks.workunit.client.0.smithi132.stderr:Error ENOTEMPTY: (39) Directory not empty 2024-01-24T23:24:07.890 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:24:07.890 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush unlink host2 2024-01-24T23:24:08.006 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.002+0000 7fdac07f8640 10 monclient: get_monmap_and_config 2024-01-24T23:24:08.006 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.002+0000 7fdac07f8640 10 monclient: build_initial_monmap 2024-01-24T23:24:08.006 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.002+0000 7fdac07f8640 10 monclient: monmap: 2024-01-24T23:24:08.006 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:08.007 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:08.007 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:08.007761+0000 2024-01-24T23:24:08.007 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:08.007761+0000 2024-01-24T23:24:08.007 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:08.007 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:08.007 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:08.007 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:08.007 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:08.007 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:08.007 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:08.007 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:08.007 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:08.007 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:08.007 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:08.007 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:08.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.002+0000 7fdac07f8640 1 Processor -- start 2024-01-24T23:24:08.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.002+0000 7fdac07f8640 1 -- start start 2024-01-24T23:24:08.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.002+0000 7fdac07f8640 10 monclient: init 2024-01-24T23:24:08.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdac07f8640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:08.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdac07f8640 10 monclient: _add_conns ranks=[4,0,6,5,2,7,3,1,8] 2024-01-24T23:24:08.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdac07f8640 10 monclient(hunting): picked mon.noname-c con 0x7fdab809ff90 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:08.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdac07f8640 10 monclient(hunting): picked mon.noname-a con 0x7fdab809f9c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:08.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdac07f8640 10 monclient(hunting): picked mon.noname-d con 0x7fdab813b2c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:08.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdac07f8640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab8064700 con 0x7fdab809f9c0 2024-01-24T23:24:08.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdac07f8640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab809fe00 con 0x7fdab809ff90 2024-01-24T23:24:08.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdac07f8640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab80a1160 con 0x7fdab813b2c0 2024-01-24T23:24:08.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdac07f8640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:08.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdac07f8640 10 monclient(hunting): authenticate will time out at 1135.332414s 2024-01-24T23:24:08.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabe594640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7fdab809ff90 legacy=0x7fdab80a0370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:08.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabdd93640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fdab809f9c0 legacy=0x7fdab8139ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:08.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabed95640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fdab813b2c0 0x7fdab813d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:57538/0 (socket says 172.21.15.132:57538) 2024-01-24T23:24:08.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabed95640 1 -- 172.21.15.132:0/657810054 learned_addr learned my addr 172.21.15.132:0/657810054 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:08.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 1 -- 172.21.15.132:0/657810054 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1308612572 0 0) 0x7fdab80a1160 con 0x7fdab813b2c0 2024-01-24T23:24:08.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 10 monclient(hunting): _negotiate 2024-01-24T23:24:08.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:08.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:08.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 10 monclient(hunting): my global_id is 5469 2024-01-24T23:24:08.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 1 -- 172.21.15.132:0/657810054 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaa4003530 con 0x7fdab813b2c0 2024-01-24T23:24:08.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 1 -- 172.21.15.132:0/657810054 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3585982328 0 0) 0x7fdaa4003530 con 0x7fdab813b2c0 2024-01-24T23:24:08.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 1 -- 172.21.15.132:0/657810054 >> v1:172.21.15.115:6791/0 conn(0x7fdab809ff90 legacy=0x7fdab80a0370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:08.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 1 -- 172.21.15.132:0/657810054 >> v1:172.21.15.115:6789/0 conn(0x7fdab809f9c0 legacy=0x7fdab8139ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:08.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 10 monclient: _finish_hunting 0 2024-01-24T23:24:08.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 1 monclient: found mon.noname-d 2024-01-24T23:24:08.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:08.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:24:08.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 1 -- 172.21.15.132:0/657810054 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab80a1440 con 0x7fdab813b2c0 2024-01-24T23:24:08.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 10 monclient: _finish_auth 0 2024-01-24T23:24:08.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 10 monclient: _check_auth_tickets 2024-01-24T23:24:08.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:08.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 1 -- 172.21.15.132:0/657810054 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdab0001b40 con 0x7fdab813b2c0 2024-01-24T23:24:08.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:08.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:24:08.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 10 monclient: dump: 2024-01-24T23:24:08.013 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:08.013 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:08.014 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:08.014 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:08.014 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:08.014 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:08.014 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:08.014 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:08.014 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:08.014 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:08.014 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:08.014 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:08.014 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:08.014 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:08.014 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:08.014 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:08.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdac07f8640 5 monclient: authenticate success, global_id 5469 2024-01-24T23:24:08.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdac07f8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:08.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 1 -- 172.21.15.132:0/657810054 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdab0001d60 con 0x7fdab813b2c0 2024-01-24T23:24:08.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:08.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 1 -- 172.21.15.132:0/657810054 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdab0001f90 con 0x7fdab813b2c0 2024-01-24T23:24:08.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdac07f8640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:08.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdac07f8640 4 set_mon_vals no callback set 2024-01-24T23:24:08.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:08.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:24:08.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdabd592640 10 monclient: dump: 2024-01-24T23:24:08.015 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:08.015 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:08.015 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:08.015 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:08.015 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:08.016 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:08.016 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:08.016 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:08.016 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:08.016 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:08.016 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:08.016 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:08.016 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:08.016 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:08.016 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:08.016 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:08.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdac07f8640 10 monclient: shutdown 2024-01-24T23:24:08.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdac07f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:08.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdac07f8640 1 -- 172.21.15.132:0/657810054 >> v1:172.21.15.115:6792/0 conn(0x7fdab813b2c0 legacy=0x7fdab813d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:08.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdac07f8640 1 -- 172.21.15.132:0/657810054 shutdown_connections 2024-01-24T23:24:08.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdac07f8640 1 -- 172.21.15.132:0/657810054 >> 172.21.15.132:0/657810054 conn(0x7fdab80a5a10 msgr2=0x7fdab80a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:08.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdac07f8640 1 -- 172.21.15.132:0/657810054 shutdown_connections 2024-01-24T23:24:08.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdac07f8640 1 -- 172.21.15.132:0/657810054 wait complete. 2024-01-24T23:24:08.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdac07f8640 10 monclient: build_initial_monmap 2024-01-24T23:24:08.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdac07f8640 10 monclient: monmap: 2024-01-24T23:24:08.017 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:08.017 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:08.017 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:08.017 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:08.017 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:08.017 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:08.017 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:08.017 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:08.018 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:08.018 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:08.018 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:08.018 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:08.018 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:08.018 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:08.018 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:08.018 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:08.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.006+0000 7fdac07f8640 1 Processor -- start 2024-01-24T23:24:08.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdac07f8640 1 -- start start 2024-01-24T23:24:08.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdac07f8640 10 monclient: init 2024-01-24T23:24:08.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdac07f8640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:08.022 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdac07f8640 10 monclient: _add_conns ranks=[5,3,1,6,0,2,4,8,7] 2024-01-24T23:24:08.022 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdac07f8640 10 monclient(hunting): picked mon.noname-h con 0x7fdab809f9c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:08.022 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdac07f8640 10 monclient(hunting): picked mon.noname-g con 0x7fdab809ff90 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:08.022 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdac07f8640 10 monclient(hunting): picked mon.noname-f con 0x7fdab813b2c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:08.022 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdac07f8640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab809f450 con 0x7fdab813b2c0 2024-01-24T23:24:08.022 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdac07f8640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab8137480 con 0x7fdab809ff90 2024-01-24T23:24:08.022 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdac07f8640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab80715f0 con 0x7fdab809f9c0 2024-01-24T23:24:08.022 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdabed95640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fdab813b2c0 legacy=0x7fdab81d34b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:08.022 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdabe594640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fdab809f9c0 0x7fdab809c7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:40886/0 (socket says 172.21.15.132:40886) 2024-01-24T23:24:08.022 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdac07f8640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:08.022 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdabe594640 1 -- 172.21.15.132:0/991541260 learned_addr learned my addr 172.21.15.132:0/991541260 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:08.022 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdac07f8640 10 monclient(hunting): authenticate will time out at 1135.336734s 2024-01-24T23:24:08.022 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 1 -- 172.21.15.132:0/991541260 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1033090792 0 0) 0x7fdab80715f0 con 0x7fdab809f9c0 2024-01-24T23:24:08.022 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:08.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:08.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:08.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 10 monclient(hunting): my global_id is 5432 2024-01-24T23:24:08.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 1 -- 172.21.15.132:0/991541260 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda9c003530 con 0x7fdab809f9c0 2024-01-24T23:24:08.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 1 -- 172.21.15.132:0/991541260 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259292779 0 0) 0x7fdab8137480 con 0x7fdab809ff90 2024-01-24T23:24:08.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:08.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:08.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:08.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 10 monclient(hunting): my global_id is 5430 2024-01-24T23:24:08.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 1 -- 172.21.15.132:0/991541260 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdab80715f0 con 0x7fdab809ff90 2024-01-24T23:24:08.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 1 -- 172.21.15.132:0/991541260 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 631633639 0 0) 0x7fda9c003530 con 0x7fdab809f9c0 2024-01-24T23:24:08.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 1 -- 172.21.15.132:0/991541260 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdab8137480 con 0x7fdab809f9c0 2024-01-24T23:24:08.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 1 -- 172.21.15.132:0/991541260 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdab4004330 con 0x7fdab809f9c0 2024-01-24T23:24:08.024 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:08.024 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:08.024 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 10 monclient(hunting): dump: 2024-01-24T23:24:08.024 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:08.024 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:08.024 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:08.024 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:08.024 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:08.024 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:08.024 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:08.024 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:08.024 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:08.024 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:08.024 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:08.025 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:08.025 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:08.025 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:08.025 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:08.025 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:08.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:08.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 1 -- 172.21.15.132:0/991541260 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 177849328 0 0) 0x7fdab80715f0 con 0x7fdab809ff90 2024-01-24T23:24:08.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 1 -- 172.21.15.132:0/991541260 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fda9c003530 con 0x7fdab809ff90 2024-01-24T23:24:08.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 1 -- 172.21.15.132:0/991541260 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdaa8004270 con 0x7fdab809ff90 2024-01-24T23:24:08.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:08.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:08.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 10 monclient(hunting): dump: 2024-01-24T23:24:08.025 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:08.026 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:08.026 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:08.026 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:08.026 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:08.026 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:08.026 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:08.026 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:08.026 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:08.026 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:08.026 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:08.026 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:08.026 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:08.026 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:08.027 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:08.027 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:08.027 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 1 -- 172.21.15.132:0/991541260 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1109484264 0 0) 0x7fda9c003530 con 0x7fdab809ff90 2024-01-24T23:24:08.027 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 1 -- 172.21.15.132:0/991541260 >> v1:172.21.15.132:6791/0 conn(0x7fdab809f9c0 legacy=0x7fdab809c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:08.027 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 1 -- 172.21.15.132:0/991541260 >> v1:172.21.15.132:6789/0 conn(0x7fdab813b2c0 legacy=0x7fdab81d34b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:08.027 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:08.027 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 1 monclient: found mon.g 2024-01-24T23:24:08.027 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:08.027 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:08.027 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 1 -- 172.21.15.132:0/991541260 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab809e140 con 0x7fdab809ff90 2024-01-24T23:24:08.027 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 10 monclient: _finish_auth 0 2024-01-24T23:24:08.027 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:08.027 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:08.027 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdac07f8640 5 monclient: authenticate success, global_id 5430 2024-01-24T23:24:08.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdac07f8640 10 monclient: _renew_subs 2024-01-24T23:24:08.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdac07f8640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:08.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdac07f8640 1 -- 172.21.15.132:0/991541260 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab8138150 con 0x7fdab809ff90 2024-01-24T23:24:08.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdac07f8640 10 monclient: _renew_subs 2024-01-24T23:24:08.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdac07f8640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:08.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdac07f8640 1 -- 172.21.15.132:0/991541260 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab8131590 con 0x7fdab809ff90 2024-01-24T23:24:08.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 1 -- 172.21.15.132:0/991541260 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdaa8004800 con 0x7fdab809ff90 2024-01-24T23:24:08.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:08.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 1 -- 172.21.15.132:0/991541260 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdaa8004d30 con 0x7fdab809ff90 2024-01-24T23:24:08.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:08.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaffff640 4 set_mon_vals no callback set 2024-01-24T23:24:08.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:08.029 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 10 monclient: dump: 2024-01-24T23:24:08.029 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:08.029 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:08.029 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:08.029 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:08.029 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:08.029 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:08.029 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:08.029 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:08.029 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:08.029 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:08.029 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:08.029 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:08.029 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:08.029 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:08.030 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:08.030 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:08.030 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdac07f8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:08.030 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdac07f8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:08.030 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdac07f8640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:08.030 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 1 -- 172.21.15.132:0/991541260 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fdaa8004e90 con 0x7fdab809ff90 2024-01-24T23:24:08.030 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdac07f8640 1 -- 172.21.15.132:0/991541260 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa0005090 con 0x7fdab809ff90 2024-01-24T23:24:08.030 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdaaeffd640 1 -- 172.21.15.132:0/991541260 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(75..75 src has 1..75) v4 ==== 5181+0+0 (unknown 186118056 0 0) 0x7fdaa8043b40 con 0x7fdab809ff90 2024-01-24T23:24:08.030 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.010+0000 7fdabe594640 10 monclient: get_auth_request con 0x7fda9c0375b0 auth_method 0 2024-01-24T23:24:08.030 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.014+0000 7fdaaeffd640 1 -- 172.21.15.132:0/991541260 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fdaa80147b0 con 0x7fdab809ff90 2024-01-24T23:24:08.030 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.014+0000 7fdaaeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:08.030 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.014+0000 7fdaaeffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:08.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.166+0000 7fdac07f8640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host2"}] 2024-01-24T23:24:08.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.166+0000 7fdac07f8640 10 monclient: _send_command 2 [{"prefix": "osd crush unlink", "name": "host2"}] 2024-01-24T23:24:08.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.166+0000 7fdac07f8640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:08.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:08.166+0000 7fdac07f8640 1 -- 172.21.15.132:0/991541260 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush unlink", "name": "host2"} v 0) v1 -- 0x7fdaa0005340 con 0x7fdab809ff90 2024-01-24T23:24:09.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:09.138+0000 7fdaaeffd640 1 -- 172.21.15.132:0/991541260 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host2"}]=0 removed item id -16 name 'host2' from crush map v76) v1 ==== 128+0+0 (unknown 1933670154 0 0) 0x7fdaa801ce70 con 0x7fdab809ff90 2024-01-24T23:24:09.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:09.138+0000 7fdaaeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush unlink", "name": "host2"}] 2024-01-24T23:24:09.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:09.138+0000 7fdaaeffd640 10 monclient: _finish_command 2 = system:0 removed item id -16 name 'host2' from crush map 2024-01-24T23:24:09.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:09.238+0000 7fdac07f8640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host2"}] 2024-01-24T23:24:09.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:09.238+0000 7fdac07f8640 10 monclient: _send_command 3 [{"prefix": "osd crush unlink", "name": "host2"}] 2024-01-24T23:24:09.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:09.238+0000 7fdac07f8640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:09.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:09.238+0000 7fdac07f8640 1 -- 172.21.15.132:0/991541260 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush unlink", "name": "host2"} v 0) v1 -- 0x7fdaa0002220 con 0x7fdab809ff90 2024-01-24T23:24:10.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.150+0000 7fdaaeffd640 1 -- 172.21.15.132:0/991541260 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host2"}]=0 removed item id -16 name 'host2' from crush map v77) v1 ==== 128+0+0 (unknown 450083240 0 0) 0x7fdaa8012e50 con 0x7fdab809ff90 2024-01-24T23:24:10.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.150+0000 7fdaaeffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush unlink", "name": "host2"}] 2024-01-24T23:24:10.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.150+0000 7fdaaeffd640 10 monclient: _finish_command 3 = system:0 removed item id -16 name 'host2' from crush map 2024-01-24T23:24:10.152 INFO:tasks.workunit.client.0.smithi132.stderr:removed item id -16 name 'host2' from crush map 2024-01-24T23:24:10.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.154+0000 7fdac07f8640 1 -- 172.21.15.132:0/991541260 >> v1:172.21.15.132:6812/1835401922 conn(0x7fda9c0375b0 legacy=0x7fda9c039a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:10.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.154+0000 7fdac07f8640 10 monclient: shutdown 2024-01-24T23:24:10.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.154+0000 7fdac07f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:10.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.154+0000 7fdac07f8640 1 -- 172.21.15.132:0/991541260 >> v1:172.21.15.132:6790/0 conn(0x7fdab809ff90 legacy=0x7fdab809cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:10.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.154+0000 7fdac07f8640 1 -- 172.21.15.132:0/991541260 shutdown_connections 2024-01-24T23:24:10.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.154+0000 7fdac07f8640 1 -- 172.21.15.132:0/991541260 >> 172.21.15.132:0/991541260 conn(0x7fdab80a5a10 msgr2=0x7fdab8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:10.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.154+0000 7fdac07f8640 1 -- 172.21.15.132:0/991541260 shutdown_connections 2024-01-24T23:24:10.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.154+0000 7fdac07f8640 1 -- 172.21.15.132:0/991541260 wait complete. 2024-01-24T23:24:10.175 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket host-for-test host root=root-for-test rack=rack-for-test 2024-01-24T23:24:10.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.294+0000 7f510cfa5640 10 monclient: get_monmap_and_config 2024-01-24T23:24:10.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.294+0000 7f510cfa5640 10 monclient: build_initial_monmap 2024-01-24T23:24:10.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.294+0000 7f510cfa5640 10 monclient: monmap: 2024-01-24T23:24:10.298 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:10.298 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:10.298 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:10.298694+0000 2024-01-24T23:24:10.299 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:10.298694+0000 2024-01-24T23:24:10.299 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:10.299 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:10.299 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:10.299 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:10.299 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:10.300 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:10.300 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:10.300 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:10.300 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:10.300 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:10.300 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:10.301 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:10.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.294+0000 7f510cfa5640 1 Processor -- start 2024-01-24T23:24:10.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.294+0000 7f510cfa5640 1 -- start start 2024-01-24T23:24:10.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.294+0000 7f510cfa5640 10 monclient: init 2024-01-24T23:24:10.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:10.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 10 monclient: _add_conns ranks=[7,2,1,5,3,6,4,8,0] 2024-01-24T23:24:10.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 10 monclient(hunting): picked mon.noname-i con 0x7f5108071760 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:10.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 10 monclient(hunting): picked mon.noname-b con 0x7f5108071b40 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:10.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 10 monclient(hunting): picked mon.noname-f con 0x7f5108071fa0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:10.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5108064700 con 0x7f5108071fa0 2024-01-24T23:24:10.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f510806e210 con 0x7f5108071b40 2024-01-24T23:24:10.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5108133e70 con 0x7f5108071760 2024-01-24T23:24:10.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:10.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 10 monclient(hunting): authenticate will time out at 1137.624138s 2024-01-24T23:24:10.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f5106d9d640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f5108071fa0 legacy=0x7f510813f5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:10.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510659c640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f5108071760 0x7f5108133760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:52328/0 (socket says 172.21.15.132:52328) 2024-01-24T23:24:10.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510659c640 1 -- 172.21.15.132:0/4045562931 learned_addr learned my addr 172.21.15.132:0/4045562931 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:10.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 1 -- 172.21.15.132:0/4045562931 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3638630862 0 0) 0x7f5108133e70 con 0x7f5108071760 2024-01-24T23:24:10.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 10 monclient(hunting): _negotiate 2024-01-24T23:24:10.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:10.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:10.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 10 monclient(hunting): my global_id is 5506 2024-01-24T23:24:10.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 1 -- 172.21.15.132:0/4045562931 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50f0003530 con 0x7f5108071760 2024-01-24T23:24:10.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 1 -- 172.21.15.132:0/4045562931 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3956500165 0 0) 0x7f50f0003530 con 0x7f5108071760 2024-01-24T23:24:10.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 1 -- 172.21.15.132:0/4045562931 >> v1:172.21.15.115:6790/0 conn(0x7f5108071b40 legacy=0x7f510813be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:10.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 1 -- 172.21.15.132:0/4045562931 >> v1:172.21.15.132:6789/0 conn(0x7f5108071fa0 legacy=0x7f510813f5a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:10.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 10 monclient: _finish_hunting 0 2024-01-24T23:24:10.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 1 monclient: found mon.noname-i 2024-01-24T23:24:10.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:10.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:24:10.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 1 -- 172.21.15.132:0/4045562931 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510806df30 con 0x7f5108071760 2024-01-24T23:24:10.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 10 monclient: _finish_auth 0 2024-01-24T23:24:10.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 10 monclient: _check_auth_tickets 2024-01-24T23:24:10.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:10.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 5 monclient: authenticate success, global_id 5506 2024-01-24T23:24:10.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 1 -- 172.21.15.132:0/4045562931 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f50f80017c0 con 0x7f5108071760 2024-01-24T23:24:10.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:10.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:10.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 10 monclient: dump: 2024-01-24T23:24:10.322 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:10.323 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:10.323 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:10.323 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:10.323 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:10.323 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:10.323 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:10.324 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:10.324 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:10.324 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:10.324 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:10.324 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:10.324 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:10.325 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:10.325 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:10.325 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:10.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:10.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 1 -- 172.21.15.132:0/4045562931 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f50f8001a70 con 0x7f5108071760 2024-01-24T23:24:10.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:10.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:10.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 4 set_mon_vals no callback set 2024-01-24T23:24:10.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 1 -- 172.21.15.132:0/4045562931 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f50f8004bf0 con 0x7f5108071760 2024-01-24T23:24:10.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:10.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:10.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510559a640 10 monclient: dump: 2024-01-24T23:24:10.327 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:10.327 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:10.327 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:10.327 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:10.327 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:10.328 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:10.328 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:10.328 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:10.329 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:10.329 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:10.329 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:10.329 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:10.329 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:10.329 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:10.330 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:10.330 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:10.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 10 monclient: shutdown 2024-01-24T23:24:10.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:10.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 1 -- 172.21.15.132:0/4045562931 >> v1:172.21.15.132:6792/0 conn(0x7f5108071760 legacy=0x7f5108133760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:10.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 1 -- 172.21.15.132:0/4045562931 shutdown_connections 2024-01-24T23:24:10.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 1 -- 172.21.15.132:0/4045562931 >> 172.21.15.132:0/4045562931 conn(0x7f51080a6060 msgr2=0x7f51080a23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:10.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 1 -- 172.21.15.132:0/4045562931 shutdown_connections 2024-01-24T23:24:10.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 1 -- 172.21.15.132:0/4045562931 wait complete. 2024-01-24T23:24:10.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 10 monclient: build_initial_monmap 2024-01-24T23:24:10.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 10 monclient: monmap: 2024-01-24T23:24:10.331 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:10.332 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:10.332 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:10.332 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:10.332 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:10.332 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:10.332 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:10.333 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:10.333 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:10.333 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:10.333 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:10.333 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:10.333 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:10.334 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:10.334 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:10.334 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:10.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 1 Processor -- start 2024-01-24T23:24:10.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 1 -- start start 2024-01-24T23:24:10.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 10 monclient: init 2024-01-24T23:24:10.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:10.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 10 monclient: _add_conns ranks=[3,4,5,6,2,0,8,1,7] 2024-01-24T23:24:10.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 10 monclient(hunting): picked mon.noname-g con 0x7f5108071760 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:10.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 10 monclient(hunting): picked mon.noname-c con 0x7f5108071b40 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:10.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 10 monclient(hunting): picked mon.noname-h con 0x7f5108071fa0 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:10.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f510811c340 con 0x7f5108071760 2024-01-24T23:24:10.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51081312a0 con 0x7f5108071b40 2024-01-24T23:24:10.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f510809fc40 con 0x7f5108071fa0 2024-01-24T23:24:10.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:10.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 10 monclient(hunting): authenticate will time out at 1137.627047s 2024-01-24T23:24:10.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f5105d9b640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f5108071b40 legacy=0x7f51081d0310 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:10.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510659c640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f5108071760 0x7f5108130390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:38446/0 (socket says 172.21.15.132:38446) 2024-01-24T23:24:10.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510659c640 1 -- 172.21.15.132:0/4066679916 learned_addr learned my addr 172.21.15.132:0/4066679916 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:10.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 1 -- 172.21.15.132:0/4066679916 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1884033367 0 0) 0x7f510811c340 con 0x7f5108071760 2024-01-24T23:24:10.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:10.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:10.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:10.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 10 monclient(hunting): my global_id is 5439 2024-01-24T23:24:10.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 1 -- 172.21.15.132:0/4066679916 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50e8003530 con 0x7f5108071760 2024-01-24T23:24:10.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 1 -- 172.21.15.132:0/4066679916 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3536014407 0 0) 0x7f510809fc40 con 0x7f5108071fa0 2024-01-24T23:24:10.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:10.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:10.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:10.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 10 monclient(hunting): my global_id is 5441 2024-01-24T23:24:10.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 1 -- 172.21.15.132:0/4066679916 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f510811c340 con 0x7f5108071fa0 2024-01-24T23:24:10.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 1 -- 172.21.15.132:0/4066679916 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1150642276 0 0) 0x7f50e8003530 con 0x7f5108071760 2024-01-24T23:24:10.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 1 -- 172.21.15.132:0/4066679916 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f510809fc40 con 0x7f5108071760 2024-01-24T23:24:10.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 1 -- 172.21.15.132:0/4066679916 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f50f8001780 con 0x7f5108071760 2024-01-24T23:24:10.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:10.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:10.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 10 monclient(hunting): dump: 2024-01-24T23:24:10.340 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:10.340 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:10.340 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:10.340 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:10.341 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:10.341 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:10.341 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:10.341 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:10.341 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:10.341 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:10.341 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:10.342 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:10.342 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:10.342 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:10.342 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:10.342 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:10.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:10.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 1 -- 172.21.15.132:0/4066679916 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1408719552 0 0) 0x7f510811c340 con 0x7f5108071fa0 2024-01-24T23:24:10.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 1 -- 172.21.15.132:0/4066679916 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f50e8003530 con 0x7f5108071fa0 2024-01-24T23:24:10.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 1 -- 172.21.15.132:0/4066679916 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5100004460 con 0x7f5108071fa0 2024-01-24T23:24:10.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:10.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:10.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 10 monclient(hunting): dump: 2024-01-24T23:24:10.343 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:10.343 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:10.343 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:10.343 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:10.343 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:10.344 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:10.344 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:10.344 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:10.344 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:10.344 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:10.345 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:10.345 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:10.345 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:10.345 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:10.345 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:10.345 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:10.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 1 -- 172.21.15.132:0/4066679916 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 45232466 0 0) 0x7f50e8003530 con 0x7f5108071fa0 2024-01-24T23:24:10.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 1 -- 172.21.15.132:0/4066679916 >> v1:172.21.15.115:6791/0 conn(0x7f5108071b40 legacy=0x7f51081d0310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:10.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 1 -- 172.21.15.132:0/4066679916 >> v1:172.21.15.132:6790/0 conn(0x7f5108071760 legacy=0x7f5108130390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:10.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:10.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 1 monclient: found mon.h 2024-01-24T23:24:10.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:10.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:10.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 1 -- 172.21.15.132:0/4066679916 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510811d4b0 con 0x7f5108071fa0 2024-01-24T23:24:10.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 10 monclient: _finish_auth 0 2024-01-24T23:24:10.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:10.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f50f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:10.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 5 monclient: authenticate success, global_id 5441 2024-01-24T23:24:10.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 10 monclient: _renew_subs 2024-01-24T23:24:10.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:10.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.298+0000 7f510cfa5640 1 -- 172.21.15.132:0/4066679916 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5108130aa0 con 0x7f5108071fa0 2024-01-24T23:24:10.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.302+0000 7f510cfa5640 10 monclient: _renew_subs 2024-01-24T23:24:10.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.302+0000 7f510cfa5640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:10.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.302+0000 7f510cfa5640 1 -- 172.21.15.132:0/4066679916 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f51081cbde0 con 0x7f5108071fa0 2024-01-24T23:24:10.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.302+0000 7f50f6ffd640 1 -- 172.21.15.132:0/4066679916 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5100004680 con 0x7f5108071fa0 2024-01-24T23:24:10.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.302+0000 7f50f6ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:10.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.302+0000 7f50f6ffd640 1 -- 172.21.15.132:0/4066679916 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5100004b30 con 0x7f5108071fa0 2024-01-24T23:24:10.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.302+0000 7f50f6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:10.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.302+0000 7f50f7fff640 4 set_mon_vals no callback set 2024-01-24T23:24:10.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.302+0000 7f50f6ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:10.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.302+0000 7f50f6ffd640 10 monclient: dump: 2024-01-24T23:24:10.348 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:10.348 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:10.348 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:10.348 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:10.348 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:10.348 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:10.348 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:10.348 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:10.348 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:10.348 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:10.348 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:10.349 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:10.349 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:10.349 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:10.349 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:10.349 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:10.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.302+0000 7f510cfa5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:10.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.302+0000 7f510cfa5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:10.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.302+0000 7f510cfa5640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:10.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.302+0000 7f510cfa5640 1 -- 172.21.15.132:0/4066679916 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50ec005090 con 0x7f5108071fa0 2024-01-24T23:24:10.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.302+0000 7f50f6ffd640 1 -- 172.21.15.132:0/4066679916 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f5100004d50 con 0x7f5108071fa0 2024-01-24T23:24:10.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.302+0000 7f50f6ffd640 1 -- 172.21.15.132:0/4066679916 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(77..77 src has 1..77) v4 ==== 5133+0+0 (unknown 776978544 0 0) 0x7f5100043e60 con 0x7f5108071fa0 2024-01-24T23:24:10.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.302+0000 7f510659c640 10 monclient: get_auth_request con 0x7f50e8037610 auth_method 0 2024-01-24T23:24:10.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.306+0000 7f50f6ffd640 1 -- 172.21.15.132:0/4066679916 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f5100014b80 con 0x7f5108071fa0 2024-01-24T23:24:10.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.306+0000 7f50f6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:10.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.306+0000 7f50f6ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:10.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.462+0000 7f510cfa5640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}] 2024-01-24T23:24:10.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.462+0000 7f510cfa5640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}] 2024-01-24T23:24:10.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.462+0000 7f510cfa5640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:10.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:10.462+0000 7f510cfa5640 1 -- 172.21.15.132:0/4066679916 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]} v 0) v1 -- 0x7f50ec005340 con 0x7f5108071fa0 2024-01-24T23:24:11.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.166+0000 7f50f6ffd640 1 -- 172.21.15.132:0/4066679916 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}]=0 added bucket host-for-test type host to location {rack=rack-for-test,root=root-for-test} v78) v1 ==== 251+0+0 (unknown 1300768419 0 0) 0x7f5100012ff0 con 0x7f5108071fa0 2024-01-24T23:24:11.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.166+0000 7f50f6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}] 2024-01-24T23:24:11.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.166+0000 7f50f6ffd640 10 monclient: _finish_command 2 = system:0 added bucket host-for-test type host to location {rack=rack-for-test,root=root-for-test} 2024-01-24T23:24:11.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.274+0000 7f510cfa5640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}] 2024-01-24T23:24:11.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.274+0000 7f510cfa5640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}] 2024-01-24T23:24:11.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.274+0000 7f510cfa5640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:11.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.274+0000 7f510cfa5640 1 -- 172.21.15.132:0/4066679916 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]} v 0) v1 -- 0x7f50ec005ab0 con 0x7f5108071fa0 2024-01-24T23:24:11.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.274+0000 7f50f6ffd640 1 -- 172.21.15.132:0/4066679916 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}]=0 bucket 'host-for-test' already exists v78) v1 ==== 200+0+0 (unknown 2155713451 0 0) 0x7f5100014770 con 0x7f5108071fa0 2024-01-24T23:24:11.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.274+0000 7f50f6ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}] 2024-01-24T23:24:11.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.274+0000 7f50f6ffd640 10 monclient: _finish_command 3 = system:0 bucket 'host-for-test' already exists 2024-01-24T23:24:11.278 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'host-for-test' already exists 2024-01-24T23:24:11.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.278+0000 7f510cfa5640 1 -- 172.21.15.132:0/4066679916 >> v1:172.21.15.132:6812/1835401922 conn(0x7f50e8037610 legacy=0x7f50e8039ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:11.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.278+0000 7f510cfa5640 10 monclient: shutdown 2024-01-24T23:24:11.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.278+0000 7f510cfa5640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:11.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.278+0000 7f510cfa5640 1 -- 172.21.15.132:0/4066679916 >> v1:172.21.15.132:6791/0 conn(0x7f5108071fa0 legacy=0x7f51081d3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:11.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.278+0000 7f510cfa5640 1 -- 172.21.15.132:0/4066679916 shutdown_connections 2024-01-24T23:24:11.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.278+0000 7f510cfa5640 1 -- 172.21.15.132:0/4066679916 >> 172.21.15.132:0/4066679916 conn(0x7f51080a6060 msgr2=0x7f51080a08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:11.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.278+0000 7f510cfa5640 1 -- 172.21.15.132:0/4066679916 shutdown_connections 2024-01-24T23:24:11.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.278+0000 7f510cfa5640 1 -- 172.21.15.132:0/4066679916 wait complete. 2024-01-24T23:24:11.295 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd tree 2024-01-24T23:24:11.295 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep host-for-test 2024-01-24T23:24:11.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.406+0000 7efc85d5c640 10 monclient: get_monmap_and_config 2024-01-24T23:24:11.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.406+0000 7efc85d5c640 10 monclient: build_initial_monmap 2024-01-24T23:24:11.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.406+0000 7efc85d5c640 10 monclient: monmap: 2024-01-24T23:24:11.410 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:11.410 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:11.410 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:11.411084+0000 2024-01-24T23:24:11.410 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:11.411084+0000 2024-01-24T23:24:11.410 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:11.410 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:11.410 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:11.411 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:11.411 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:11.411 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:11.411 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:11.411 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:11.411 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:11.411 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:11.411 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:11.411 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:11.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.406+0000 7efc85d5c640 1 Processor -- start 2024-01-24T23:24:11.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.406+0000 7efc85d5c640 1 -- start start 2024-01-24T23:24:11.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.406+0000 7efc85d5c640 10 monclient: init 2024-01-24T23:24:11.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.406+0000 7efc85d5c640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:11.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.406+0000 7efc85d5c640 10 monclient: _add_conns ranks=[1,2,6,4,0,8,3,7,5] 2024-01-24T23:24:11.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.406+0000 7efc85d5c640 10 monclient(hunting): picked mon.noname-f con 0x7efc8009ff20 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:11.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.406+0000 7efc85d5c640 10 monclient(hunting): picked mon.noname-b con 0x7efc8009f9c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:11.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.406+0000 7efc85d5c640 10 monclient(hunting): picked mon.noname-d con 0x7efc8013b2c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:11.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.406+0000 7efc85d5c640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc80064700 con 0x7efc8009ff20 2024-01-24T23:24:11.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.406+0000 7efc85d5c640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc8009e760 con 0x7efc8009f9c0 2024-01-24T23:24:11.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.406+0000 7efc85d5c640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc800a1160 con 0x7efc8013b2c0 2024-01-24T23:24:11.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.406+0000 7efc85d5c640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:11.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.406+0000 7efc85d5c640 10 monclient(hunting): authenticate will time out at 1138.735703s 2024-01-24T23:24:11.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.406+0000 7efc7f7fe640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7efc8009ff20 legacy=0x7efc800a0300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:11.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7ffff640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7efc8013b2c0 0x7efc8013d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:57540/0 (socket says 172.21.15.132:57540) 2024-01-24T23:24:11.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7ffff640 1 -- 172.21.15.132:0/3194688842 learned_addr learned my addr 172.21.15.132:0/3194688842 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:11.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 1 -- 172.21.15.132:0/3194688842 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 819490406 0 0) 0x7efc8009e760 con 0x7efc8009f9c0 2024-01-24T23:24:11.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:11.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:11.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:11.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 10 monclient(hunting): my global_id is 84791 2024-01-24T23:24:11.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 1 -- 172.21.15.132:0/3194688842 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc6c003530 con 0x7efc8009f9c0 2024-01-24T23:24:11.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 1 -- 172.21.15.132:0/3194688842 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337564850 0 0) 0x7efc800a1160 con 0x7efc8013b2c0 2024-01-24T23:24:11.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:11.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:11.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:11.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 10 monclient(hunting): my global_id is 5478 2024-01-24T23:24:11.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 1 -- 172.21.15.132:0/3194688842 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc8009e760 con 0x7efc8013b2c0 2024-01-24T23:24:11.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 1 -- 172.21.15.132:0/3194688842 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1880944806 0 0) 0x7efc6c003530 con 0x7efc8009f9c0 2024-01-24T23:24:11.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 1 -- 172.21.15.132:0/3194688842 >> v1:172.21.15.115:6792/0 conn(0x7efc8013b2c0 legacy=0x7efc8013d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:11.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 1 -- 172.21.15.132:0/3194688842 >> v1:172.21.15.132:6789/0 conn(0x7efc8009ff20 legacy=0x7efc800a0300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:11.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:24:11.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 1 monclient: found mon.noname-b 2024-01-24T23:24:11.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:11.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:24:11.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 1 -- 172.21.15.132:0/3194688842 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc800a0a10 con 0x7efc8009f9c0 2024-01-24T23:24:11.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 10 monclient: _finish_auth 0 2024-01-24T23:24:11.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:24:11.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:11.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 1 -- 172.21.15.132:0/3194688842 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7efc700017c0 con 0x7efc8009f9c0 2024-01-24T23:24:11.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:11.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:24:11.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 10 monclient: dump: 2024-01-24T23:24:11.423 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:11.423 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:11.423 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:11.423 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:11.423 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:11.423 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:11.423 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:11.423 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:11.423 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:11.423 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:11.423 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:11.423 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:11.424 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:11.424 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:11.424 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:11.424 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:11.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc85d5c640 5 monclient: authenticate success, global_id 84791 2024-01-24T23:24:11.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc85d5c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:11.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 1 -- 172.21.15.132:0/3194688842 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efc700019e0 con 0x7efc8009f9c0 2024-01-24T23:24:11.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:11.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 1 -- 172.21.15.132:0/3194688842 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7efc70003960 con 0x7efc8009f9c0 2024-01-24T23:24:11.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:11.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:24:11.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc7e7fc640 10 monclient: dump: 2024-01-24T23:24:11.425 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:11.425 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:11.425 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:11.425 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:11.425 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:11.425 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:11.425 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:11.425 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:11.425 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:11.425 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:11.425 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:11.425 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:11.425 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:11.425 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:11.426 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:11.426 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:11.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc85d5c640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:11.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc85d5c640 4 set_mon_vals no callback set 2024-01-24T23:24:11.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc85d5c640 10 monclient: shutdown 2024-01-24T23:24:11.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc85d5c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:11.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc85d5c640 1 -- 172.21.15.132:0/3194688842 >> v1:172.21.15.115:6790/0 conn(0x7efc8009f9c0 legacy=0x7efc80139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:11.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc85d5c640 1 -- 172.21.15.132:0/3194688842 shutdown_connections 2024-01-24T23:24:11.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc85d5c640 1 -- 172.21.15.132:0/3194688842 >> 172.21.15.132:0/3194688842 conn(0x7efc800a5a10 msgr2=0x7efc800a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:11.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc85d5c640 1 -- 172.21.15.132:0/3194688842 shutdown_connections 2024-01-24T23:24:11.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc85d5c640 1 -- 172.21.15.132:0/3194688842 wait complete. 2024-01-24T23:24:11.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc85d5c640 10 monclient: build_initial_monmap 2024-01-24T23:24:11.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc85d5c640 10 monclient: monmap: 2024-01-24T23:24:11.427 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:11.427 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:11.427 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:11.427 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:11.427 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:11.427 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:11.427 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:11.427 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:11.427 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:11.427 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:11.427 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:11.428 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:11.428 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:11.428 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:11.428 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:11.428 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:11.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc85d5c640 1 Processor -- start 2024-01-24T23:24:11.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc85d5c640 1 -- start start 2024-01-24T23:24:11.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.410+0000 7efc85d5c640 10 monclient: init 2024-01-24T23:24:11.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc85d5c640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:11.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc85d5c640 10 monclient: _add_conns ranks=[0,5,6,8,3,1,7,2,4] 2024-01-24T23:24:11.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc85d5c640 10 monclient(hunting): picked mon.noname-a con 0x7efc8009f9c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:11.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc85d5c640 10 monclient(hunting): picked mon.noname-h con 0x7efc8009ff20 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:11.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc85d5c640 10 monclient(hunting): picked mon.noname-d con 0x7efc8013b2c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:11.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc85d5c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc80071960 con 0x7efc8009f9c0 2024-01-24T23:24:11.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc85d5c640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc80071680 con 0x7efc8009ff20 2024-01-24T23:24:11.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc85d5c640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc8009ca70 con 0x7efc8013b2c0 2024-01-24T23:24:11.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc85d5c640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:11.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc7f7fe640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7efc8009f9c0 legacy=0x7efc80132980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:11.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc85d5c640 10 monclient(hunting): authenticate will time out at 1138.740257s 2024-01-24T23:24:11.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc7effd640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7efc8009ff20 0x7efc801d07f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:40906/0 (socket says 172.21.15.132:40906) 2024-01-24T23:24:11.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc7effd640 1 -- 172.21.15.132:0/3088527661 learned_addr learned my addr 172.21.15.132:0/3088527661 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:11.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 1 -- 172.21.15.132:0/3088527661 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3826333975 0 0) 0x7efc80071680 con 0x7efc8009ff20 2024-01-24T23:24:11.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:11.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:11.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:11.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 10 monclient(hunting): my global_id is 5450 2024-01-24T23:24:11.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 1 -- 172.21.15.132:0/3088527661 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc60003560 con 0x7efc8009ff20 2024-01-24T23:24:11.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 1 -- 172.21.15.132:0/3088527661 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3671012634 0 0) 0x7efc60003560 con 0x7efc8009ff20 2024-01-24T23:24:11.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 1 -- 172.21.15.132:0/3088527661 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efc80071680 con 0x7efc8009ff20 2024-01-24T23:24:11.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 1 -- 172.21.15.132:0/3088527661 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7efc700016c0 con 0x7efc8009ff20 2024-01-24T23:24:11.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:11.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:11.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 10 monclient(hunting): dump: 2024-01-24T23:24:11.430 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:11.430 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:11.431 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:11.431 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:11.431 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:11.431 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:11.431 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:11.431 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:11.431 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:11.431 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:11.431 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:11.431 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:11.431 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:11.431 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:11.431 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:11.431 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:11.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:11.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 1 -- 172.21.15.132:0/3088527661 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2120207775 0 0) 0x7efc8009ca70 con 0x7efc8013b2c0 2024-01-24T23:24:11.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:11.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:11.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:11.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 10 monclient(hunting): my global_id is 5487 2024-01-24T23:24:11.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 1 -- 172.21.15.132:0/3088527661 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc60003560 con 0x7efc8013b2c0 2024-01-24T23:24:11.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 1 -- 172.21.15.132:0/3088527661 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3800974528 0 0) 0x7efc80071680 con 0x7efc8009ff20 2024-01-24T23:24:11.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 1 -- 172.21.15.132:0/3088527661 >> v1:172.21.15.115:6792/0 conn(0x7efc8013b2c0 legacy=0x7efc801d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:11.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 1 -- 172.21.15.132:0/3088527661 >> v1:172.21.15.115:6789/0 conn(0x7efc8009f9c0 legacy=0x7efc80132980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:11.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 10 monclient: _finish_hunting 0 2024-01-24T23:24:11.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 1 monclient: found mon.h 2024-01-24T23:24:11.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:11.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:11.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 1 -- 172.21.15.132:0/3088527661 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc80120110 con 0x7efc8009ff20 2024-01-24T23:24:11.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 10 monclient: _finish_auth 0 2024-01-24T23:24:11.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 10 monclient: _check_auth_tickets 2024-01-24T23:24:11.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:11.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc85d5c640 5 monclient: authenticate success, global_id 5450 2024-01-24T23:24:11.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc85d5c640 10 monclient: _renew_subs 2024-01-24T23:24:11.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc85d5c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:11.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc85d5c640 1 -- 172.21.15.132:0/3088527661 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc8009d7f0 con 0x7efc8009ff20 2024-01-24T23:24:11.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc85d5c640 10 monclient: _renew_subs 2024-01-24T23:24:11.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc85d5c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:11.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc85d5c640 1 -- 172.21.15.132:0/3088527661 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7efc80133090 con 0x7efc8009ff20 2024-01-24T23:24:11.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc85d5c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:11.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc85d5c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:11.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc85d5c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:11.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc85d5c640 1 -- 172.21.15.132:0/3088527661 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc80133340 con 0x7efc8009ff20 2024-01-24T23:24:11.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 1 -- 172.21.15.132:0/3088527661 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efc70001cb0 con 0x7efc8009ff20 2024-01-24T23:24:11.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:11.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 1 -- 172.21.15.132:0/3088527661 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7efc70004290 con 0x7efc8009ff20 2024-01-24T23:24:11.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:11.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:11.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 10 monclient: dump: 2024-01-24T23:24:11.434 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:11.435 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:11.435 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:11.435 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:11.435 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:11.435 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:11.435 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:11.435 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:11.435 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:11.435 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:11.435 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:11.435 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:11.435 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:11.436 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:11.436 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:11.436 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:11.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc7dffb640 4 set_mon_vals no callback set 2024-01-24T23:24:11.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 1 -- 172.21.15.132:0/3088527661 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7efc700043f0 con 0x7efc8009ff20 2024-01-24T23:24:11.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc77fff640 1 -- 172.21.15.132:0/3088527661 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(78..78 src has 1..78) v4 ==== 5658+0+0 (unknown 2205127786 0 0) 0x7efc70044160 con 0x7efc8009ff20 2024-01-24T23:24:11.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.414+0000 7efc7f7fe640 10 monclient: get_auth_request con 0x7efc60037000 auth_method 0 2024-01-24T23:24:11.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.418+0000 7efc77fff640 1 -- 172.21.15.132:0/3088527661 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7efc70014b00 con 0x7efc8009ff20 2024-01-24T23:24:11.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.418+0000 7efc77fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:11.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.418+0000 7efc77fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:11.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.574+0000 7efc85d5c640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:11.579 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.574+0000 7efc85d5c640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:11.579 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.574+0000 7efc85d5c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:11.579 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.574+0000 7efc85d5c640 1 -- 172.21.15.132:0/3088527661 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7efc8009be70 con 0x7efc8009ff20 2024-01-24T23:24:11.579 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.578+0000 7efc77fff640 1 -- 172.21.15.132:0/3088527661 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v78) v1 ==== 56+0+1500 (unknown 596248356 0 1379867665) 0x7efc8009be70 con 0x7efc8009ff20 2024-01-24T23:24:11.579 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.578+0000 7efc77fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:11.579 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.578+0000 7efc77fff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:24:11.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.662+0000 7efc85d5c640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:11.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.662+0000 7efc85d5c640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:11.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.662+0000 7efc85d5c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:11.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.662+0000 7efc85d5c640 1 -- 172.21.15.132:0/3088527661 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7efc8009be70 con 0x7efc8009ff20 2024-01-24T23:24:11.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.662+0000 7efc77fff640 1 -- 172.21.15.132:0/3088527661 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v78) v1 ==== 56+0+1500 (unknown 596248356 0 1379867665) 0x7efc8009be70 con 0x7efc8009ff20 2024-01-24T23:24:11.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.662+0000 7efc77fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:11.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.662+0000 7efc77fff640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:24:11.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.666+0000 7efc85d5c640 1 -- 172.21.15.132:0/3088527661 >> v1:172.21.15.132:6812/1835401922 conn(0x7efc60037000 legacy=0x7efc600394c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:11.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.666+0000 7efc85d5c640 10 monclient: shutdown 2024-01-24T23:24:11.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.666+0000 7efc85d5c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:11.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.666+0000 7efc85d5c640 1 -- 172.21.15.132:0/3088527661 >> v1:172.21.15.132:6791/0 conn(0x7efc8009ff20 legacy=0x7efc801d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:11.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.666+0000 7efc85d5c640 1 -- 172.21.15.132:0/3088527661 shutdown_connections 2024-01-24T23:24:11.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.666+0000 7efc85d5c640 1 -- 172.21.15.132:0/3088527661 >> 172.21.15.132:0/3088527661 conn(0x7efc800a5a10 msgr2=0x7efc80066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:11.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.666+0000 7efc85d5c640 1 -- 172.21.15.132:0/3088527661 shutdown_connections 2024-01-24T23:24:11.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.666+0000 7efc85d5c640 1 -- 172.21.15.132:0/3088527661 wait complete. 2024-01-24T23:24:11.682 INFO:tasks.workunit.client.0.smithi132.stdout:-11 0 host host-for-test 2024-01-24T23:24:11.683 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd tree 2024-01-24T23:24:11.683 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep rack-for-test 2024-01-24T23:24:11.793 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c7b818640 10 monclient: get_monmap_and_config 2024-01-24T23:24:11.793 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c7b818640 10 monclient: build_initial_monmap 2024-01-24T23:24:11.793 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c7b818640 10 monclient: monmap: 2024-01-24T23:24:11.793 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:11.794 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:11.794 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:11.794646+0000 2024-01-24T23:24:11.794 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:11.794646+0000 2024-01-24T23:24:11.794 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:11.794 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:11.794 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:11.794 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:11.794 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:11.794 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:11.794 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:11.794 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:11.794 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:11.795 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:11.795 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:11.795 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:11.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c7b818640 1 Processor -- start 2024-01-24T23:24:11.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c7b818640 1 -- start start 2024-01-24T23:24:11.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c7b818640 10 monclient: init 2024-01-24T23:24:11.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c7b818640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:11.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c7b818640 10 monclient: _add_conns ranks=[6,5,8,0,4,3,7,1,2] 2024-01-24T23:24:11.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c7b818640 10 monclient(hunting): picked mon.noname-d con 0x7f5c7409ff20 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:11.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c7b818640 10 monclient(hunting): picked mon.noname-h con 0x7f5c7409f9c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:11.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c7b818640 10 monclient(hunting): picked mon.noname-e con 0x7f5c7413b2c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:11.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c7b818640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c74064700 con 0x7f5c7409f9c0 2024-01-24T23:24:11.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c7b818640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c7409e760 con 0x7f5c7409ff20 2024-01-24T23:24:11.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c7b818640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c740a1160 con 0x7f5c7413b2c0 2024-01-24T23:24:11.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c7b818640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:11.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c7b818640 10 monclient(hunting): authenticate will time out at 1139.119257s 2024-01-24T23:24:11.803 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c78db3640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f5c7409f9c0 0x7f5c74139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:40918/0 (socket says 172.21.15.132:40918) 2024-01-24T23:24:11.803 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c78db3640 1 -- 172.21.15.132:0/1537522468 learned_addr learned my addr 172.21.15.132:0/1537522468 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:11.803 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c6bfff640 1 -- 172.21.15.132:0/1537522468 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3557355810 0 0) 0x7f5c74064700 con 0x7f5c7409f9c0 2024-01-24T23:24:11.803 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c6bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:11.803 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c6bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:11.803 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c6bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:11.803 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c6bfff640 10 monclient(hunting): my global_id is 5459 2024-01-24T23:24:11.803 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c6bfff640 1 -- 172.21.15.132:0/1537522468 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c60003530 con 0x7f5c7409f9c0 2024-01-24T23:24:11.803 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c6bfff640 1 -- 172.21.15.132:0/1537522468 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1417967583 0 0) 0x7f5c60003530 con 0x7f5c7409f9c0 2024-01-24T23:24:11.803 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.790+0000 7f5c6bfff640 1 -- 172.21.15.132:0/1537522468 >> v1:172.21.15.115:6793/0 conn(0x7f5c7413b2c0 legacy=0x7f5c7413d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:11.803 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c6bfff640 1 -- 172.21.15.132:0/1537522468 >> v1:172.21.15.115:6792/0 conn(0x7f5c7409ff20 legacy=0x7f5c740a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:11.803 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c6bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:24:11.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c6bfff640 1 monclient: found mon.noname-h 2024-01-24T23:24:11.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c6bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:11.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c6bfff640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:24:11.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c6bfff640 1 -- 172.21.15.132:0/1537522468 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c740a0a10 con 0x7f5c7409f9c0 2024-01-24T23:24:11.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c6bfff640 10 monclient: _finish_auth 0 2024-01-24T23:24:11.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c6bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:24:11.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c6bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:11.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c6bfff640 1 -- 172.21.15.132:0/1537522468 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5c640017c0 con 0x7f5c7409f9c0 2024-01-24T23:24:11.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c6bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:11.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c6bfff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:11.805 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c6bfff640 10 monclient: dump: 2024-01-24T23:24:11.805 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:11.805 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:11.805 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:11.805 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:11.805 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:11.805 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:11.805 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:11.805 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:11.805 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:11.805 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:11.806 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:11.806 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:11.806 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:11.806 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:11.806 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:11.806 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:11.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 5 monclient: authenticate success, global_id 5459 2024-01-24T23:24:11.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:11.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c6bfff640 1 -- 172.21.15.132:0/1537522468 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5c64001a70 con 0x7f5c7409f9c0 2024-01-24T23:24:11.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c6bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:11.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c6bfff640 1 -- 172.21.15.132:0/1537522468 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5c64004bf0 con 0x7f5c7409f9c0 2024-01-24T23:24:11.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:11.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 4 set_mon_vals no callback set 2024-01-24T23:24:11.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c6bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:11.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c6bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:11.807 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c6bfff640 10 monclient: dump: 2024-01-24T23:24:11.807 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:11.807 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:11.807 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:11.807 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:11.807 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:11.807 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:11.807 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:11.807 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:11.807 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:11.808 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:11.808 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:11.808 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:11.808 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:11.808 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:11.808 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:11.808 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:11.808 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 10 monclient: shutdown 2024-01-24T23:24:11.808 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:11.808 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 1 -- 172.21.15.132:0/1537522468 >> v1:172.21.15.132:6791/0 conn(0x7f5c7409f9c0 legacy=0x7f5c74139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:11.808 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 1 -- 172.21.15.132:0/1537522468 shutdown_connections 2024-01-24T23:24:11.808 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 1 -- 172.21.15.132:0/1537522468 >> 172.21.15.132:0/1537522468 conn(0x7f5c740a5a10 msgr2=0x7f5c740a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:11.809 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 1 -- 172.21.15.132:0/1537522468 shutdown_connections 2024-01-24T23:24:11.809 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 1 -- 172.21.15.132:0/1537522468 wait complete. 2024-01-24T23:24:11.809 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 10 monclient: build_initial_monmap 2024-01-24T23:24:11.809 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 10 monclient: monmap: 2024-01-24T23:24:11.809 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:11.809 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:11.809 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:11.809 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:11.809 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:11.809 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:11.809 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:11.809 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:11.809 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:11.810 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:11.810 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:11.810 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:11.810 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:11.810 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:11.810 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:11.810 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:11.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 1 Processor -- start 2024-01-24T23:24:11.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 1 -- start start 2024-01-24T23:24:11.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 10 monclient: init 2024-01-24T23:24:11.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:11.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 10 monclient: _add_conns ranks=[2,1,7,3,4,5,6,8,0] 2024-01-24T23:24:11.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 10 monclient(hunting): picked mon.noname-b con 0x7f5c7409f9c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:11.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 10 monclient(hunting): picked mon.noname-f con 0x7f5c7409ff20 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:11.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 10 monclient(hunting): picked mon.noname-i con 0x7f5c7413b2c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:11.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c7411bb40 con 0x7f5c7409ff20 2024-01-24T23:24:11.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c7406fdf0 con 0x7f5c7409f9c0 2024-01-24T23:24:11.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c7411b860 con 0x7f5c7413b2c0 2024-01-24T23:24:11.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:11.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 10 monclient(hunting): authenticate will time out at 1139.122214s 2024-01-24T23:24:11.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c78db3640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f5c7409ff20 legacy=0x7f5c741ce6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:11.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c79db5640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f5c7413b2c0 0x7f5c741d1e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:52334/0 (socket says 172.21.15.132:52334) 2024-01-24T23:24:11.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c79db5640 1 -- 172.21.15.132:0/1129588026 learned_addr learned my addr 172.21.15.132:0/1129588026 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:11.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 1 -- 172.21.15.132:0/1129588026 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1052447006 0 0) 0x7f5c7411b860 con 0x7f5c7413b2c0 2024-01-24T23:24:11.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:11.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:11.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:11.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 10 monclient(hunting): my global_id is 5515 2024-01-24T23:24:11.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 1 -- 172.21.15.132:0/1129588026 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c58003530 con 0x7f5c7413b2c0 2024-01-24T23:24:11.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 1 -- 172.21.15.132:0/1129588026 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3355713736 0 0) 0x7f5c58003530 con 0x7f5c7413b2c0 2024-01-24T23:24:11.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 1 -- 172.21.15.132:0/1129588026 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5c7411b860 con 0x7f5c7413b2c0 2024-01-24T23:24:11.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 1 -- 172.21.15.132:0/1129588026 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5c70001a10 con 0x7f5c7413b2c0 2024-01-24T23:24:11.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:11.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:11.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 10 monclient(hunting): dump: 2024-01-24T23:24:11.812 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:11.813 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:11.813 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:11.813 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:11.813 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:11.813 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:11.813 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:11.813 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:11.813 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:11.813 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:11.813 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:11.813 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:11.813 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:11.813 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:11.814 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:11.814 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:11.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:11.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 1 -- 172.21.15.132:0/1129588026 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1832524927 0 0) 0x7f5c7406fdf0 con 0x7f5c7409f9c0 2024-01-24T23:24:11.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:11.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:11.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:11.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 10 monclient(hunting): my global_id is 84800 2024-01-24T23:24:11.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 1 -- 172.21.15.132:0/1129588026 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c58003530 con 0x7f5c7409f9c0 2024-01-24T23:24:11.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 1 -- 172.21.15.132:0/1129588026 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2253861009 0 0) 0x7f5c7411b860 con 0x7f5c7413b2c0 2024-01-24T23:24:11.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 1 -- 172.21.15.132:0/1129588026 >> v1:172.21.15.115:6790/0 conn(0x7f5c7409f9c0 legacy=0x7f5c741c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:11.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 1 -- 172.21.15.132:0/1129588026 >> v1:172.21.15.132:6789/0 conn(0x7f5c7409ff20 legacy=0x7f5c741ce6f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:11.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:24:11.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 1 monclient: found mon.i 2024-01-24T23:24:11.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:11.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:11.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 1 -- 172.21.15.132:0/1129588026 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7406f970 con 0x7f5c7413b2c0 2024-01-24T23:24:11.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 10 monclient: _finish_auth 0 2024-01-24T23:24:11.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:24:11.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:11.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 5 monclient: authenticate success, global_id 5515 2024-01-24T23:24:11.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 10 monclient: _renew_subs 2024-01-24T23:24:11.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:11.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 1 -- 172.21.15.132:0/1129588026 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7406fb60 con 0x7f5c7413b2c0 2024-01-24T23:24:11.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 10 monclient: _renew_subs 2024-01-24T23:24:11.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:11.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c7b818640 1 -- 172.21.15.132:0/1129588026 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7411c130 con 0x7f5c7413b2c0 2024-01-24T23:24:11.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 1 -- 172.21.15.132:0/1129588026 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5c70003950 con 0x7f5c7413b2c0 2024-01-24T23:24:11.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:11.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 1 -- 172.21.15.132:0/1129588026 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5c70001d90 con 0x7f5c7413b2c0 2024-01-24T23:24:11.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:11.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c6b7fe640 4 set_mon_vals no callback set 2024-01-24T23:24:11.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:11.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 10 monclient: dump: 2024-01-24T23:24:11.816 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:11.816 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:11.816 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:11.817 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:11.817 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:11.817 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:11.817 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:11.817 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:11.817 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:11.817 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:11.817 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:11.817 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:11.817 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:11.817 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:11.817 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:11.818 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:11.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.794+0000 7f5c69ffb640 1 -- 172.21.15.132:0/1129588026 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f5c70003f00 con 0x7f5c7413b2c0 2024-01-24T23:24:11.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.798+0000 7f5c7b818640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:11.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.798+0000 7f5c7b818640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:11.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.798+0000 7f5c7b818640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:11.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.798+0000 7f5c7b818640 1 -- 172.21.15.132:0/1129588026 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c54005090 con 0x7f5c7413b2c0 2024-01-24T23:24:11.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.798+0000 7f5c69ffb640 1 -- 172.21.15.132:0/1129588026 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(78..78 src has 1..78) v4 ==== 5658+0+0 (unknown 2205127786 0 0) 0x7f5c70043fe0 con 0x7f5c7413b2c0 2024-01-24T23:24:11.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.798+0000 7f5c795b4640 10 monclient: get_auth_request con 0x7f5c58036eb0 auth_method 0 2024-01-24T23:24:11.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.802+0000 7f5c69ffb640 1 -- 172.21.15.132:0/1129588026 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f5c700148e0 con 0x7f5c7413b2c0 2024-01-24T23:24:11.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.802+0000 7f5c69ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:11.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.802+0000 7f5c69ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:11.959 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.954+0000 7f5c7b818640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:11.959 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.954+0000 7f5c7b818640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:11.959 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.954+0000 7f5c7b818640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:11.959 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.954+0000 7f5c7b818640 1 -- 172.21.15.132:0/1129588026 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f5c54005b50 con 0x7f5c7413b2c0 2024-01-24T23:24:11.959 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.954+0000 7f5c69ffb640 1 -- 172.21.15.132:0/1129588026 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v79) v1 ==== 56+0+1500 (unknown 3183532764 0 1379867665) 0x7f5c70012f80 con 0x7f5c7413b2c0 2024-01-24T23:24:11.959 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.954+0000 7f5c69ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:11.959 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:11.954+0000 7f5c69ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:24:12.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.042+0000 7f5c7b818640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:12.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.042+0000 7f5c7b818640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:12.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.042+0000 7f5c7b818640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:12.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.042+0000 7f5c7b818640 1 -- 172.21.15.132:0/1129588026 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f5c540066c0 con 0x7f5c7413b2c0 2024-01-24T23:24:12.046 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.042+0000 7f5c69ffb640 1 -- 172.21.15.132:0/1129588026 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v79) v1 ==== 56+0+1500 (unknown 3183532764 0 1379867665) 0x7f5c70014700 con 0x7f5c7413b2c0 2024-01-24T23:24:12.046 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.042+0000 7f5c69ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:12.046 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.042+0000 7f5c69ffb640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:24:12.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.046+0000 7f5c7b818640 1 -- 172.21.15.132:0/1129588026 >> v1:172.21.15.132:6812/1835401922 conn(0x7f5c58036eb0 legacy=0x7f5c58039370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:12.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.046+0000 7f5c7b818640 10 monclient: shutdown 2024-01-24T23:24:12.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.046+0000 7f5c7b818640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:12.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.046+0000 7f5c7b818640 1 -- 172.21.15.132:0/1129588026 >> v1:172.21.15.132:6792/0 conn(0x7f5c7413b2c0 legacy=0x7f5c741d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:12.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.046+0000 7f5c7b818640 1 -- 172.21.15.132:0/1129588026 shutdown_connections 2024-01-24T23:24:12.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.046+0000 7f5c7b818640 1 -- 172.21.15.132:0/1129588026 >> 172.21.15.132:0/1129588026 conn(0x7f5c740a5a10 msgr2=0x7f5c74136ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:12.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.046+0000 7f5c7b818640 1 -- 172.21.15.132:0/1129588026 shutdown_connections 2024-01-24T23:24:12.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.046+0000 7f5c7b818640 1 -- 172.21.15.132:0/1129588026 wait complete. 2024-01-24T23:24:12.061 INFO:tasks.workunit.client.0.smithi132.stdout:-12 0 rack rack-for-test 2024-01-24T23:24:12.062 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd tree 2024-01-24T23:24:12.062 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep root-for-test 2024-01-24T23:24:12.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.170+0000 7ffadb346640 10 monclient: get_monmap_and_config 2024-01-24T23:24:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.170+0000 7ffadb346640 10 monclient: build_initial_monmap 2024-01-24T23:24:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.170+0000 7ffadb346640 10 monclient: monmap: 2024-01-24T23:24:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:12.175821+0000 2024-01-24T23:24:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:12.175821+0000 2024-01-24T23:24:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:12.176 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:12.176 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:12.176 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:12.176 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:12.176 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:12.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.170+0000 7ffadb346640 1 Processor -- start 2024-01-24T23:24:12.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.170+0000 7ffadb346640 1 -- start start 2024-01-24T23:24:12.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.170+0000 7ffadb346640 10 monclient: init 2024-01-24T23:24:12.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:12.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 10 monclient: _add_conns ranks=[5,1,7,0,4,8,3,2,6] 2024-01-24T23:24:12.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 10 monclient(hunting): picked mon.noname-h con 0x7ffad4135e10 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:12.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 10 monclient(hunting): picked mon.noname-f con 0x7ffad41361f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:12.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 10 monclient(hunting): picked mon.noname-i con 0x7ffad41365f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:12.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad4064700 con 0x7ffad41361f0 2024-01-24T23:24:12.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad409c3e0 con 0x7ffad4135e10 2024-01-24T23:24:12.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad40a4f90 con 0x7ffad41365f0 2024-01-24T23:24:12.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:12.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 10 monclient(hunting): authenticate will time out at 1139.500765s 2024-01-24T23:24:12.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffad90e2640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7ffad4135e10 0x7ffad409bcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:40926/0 (socket says 172.21.15.132:40926) 2024-01-24T23:24:12.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffad90e2640 1 -- 172.21.15.132:0/1276440394 learned_addr learned my addr 172.21.15.132:0/1276440394 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:12.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffad88e1640 1 -- 172.21.15.132:0/1276440394 >> v1:172.21.15.132:6789/0 conn(0x7ffad41361f0 legacy=0x7ffad413bed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:12.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 1 -- 172.21.15.132:0/1276440394 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956831180 0 0) 0x7ffad409c3e0 con 0x7ffad4135e10 2024-01-24T23:24:12.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:12.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:12.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:12.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 10 monclient(hunting): my global_id is 5468 2024-01-24T23:24:12.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 1 -- 172.21.15.132:0/1276440394 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffac0003530 con 0x7ffad4135e10 2024-01-24T23:24:12.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffad98e3640 1 -- 172.21.15.132:0/1276440394 >> v1:172.21.15.132:6792/0 conn(0x7ffad41365f0 legacy=0x7ffad413f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:24:12.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffad98e3640 1 -- 172.21.15.132:0/1276440394 >> v1:172.21.15.132:6792/0 conn(0x7ffad41365f0 legacy=0x7ffad413f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:24:12.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffad98e3640 1 --1- 172.21.15.132:0/1276440394 >> v1:172.21.15.132:6792/0 conn(0x7ffad41365f0 0x7ffad413f600 :-1 s=OPENED pgs=272 cs=1 l=1).handle_message read tag failed 2024-01-24T23:24:12.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffad98e3640 1 --1- 172.21.15.132:0/1276440394 >> v1:172.21.15.132:6792/0 conn(0x7ffad41365f0 0x7ffad413f600 :-1 s=OPENED pgs=272 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:24:12.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6792/0 2024-01-24T23:24:12.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 1 -- 172.21.15.132:0/1276440394 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3020951011 0 0) 0x7ffac0003530 con 0x7ffad4135e10 2024-01-24T23:24:12.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 1 -- 172.21.15.132:0/1276440394 >> v1:172.21.15.132:6792/0 conn(0x7ffad41365f0 legacy=0x7ffad413f600 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:24:12.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 1 -- 172.21.15.132:0/1276440394 >> v1:172.21.15.132:6789/0 conn(0x7ffad41361f0 legacy=0x7ffad413bed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:12.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 10 monclient: _finish_hunting 0 2024-01-24T23:24:12.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 1 monclient: found mon.noname-h 2024-01-24T23:24:12.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:12.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:24:12.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 1 -- 172.21.15.132:0/1276440394 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad409b010 con 0x7ffad4135e10 2024-01-24T23:24:12.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 10 monclient: _finish_auth 0 2024-01-24T23:24:12.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 10 monclient: _check_auth_tickets 2024-01-24T23:24:12.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:12.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 1 -- 172.21.15.132:0/1276440394 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ffad00017c0 con 0x7ffad4135e10 2024-01-24T23:24:12.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:12.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:12.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 10 monclient: dump: 2024-01-24T23:24:12.185 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:12.185 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:12.185 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:12.185 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:12.185 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:12.185 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:12.185 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:12.185 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:12.185 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:12.186 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:12.186 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:12.186 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:12.186 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:12.186 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:12.186 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:12.186 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:12.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 5 monclient: authenticate success, global_id 5468 2024-01-24T23:24:12.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:12.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 1 -- 172.21.15.132:0/1276440394 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ffad0001aa0 con 0x7ffad4135e10 2024-01-24T23:24:12.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:12.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 1 -- 172.21.15.132:0/1276440394 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ffad0004bf0 con 0x7ffad4135e10 2024-01-24T23:24:12.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:12.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 4 set_mon_vals no callback set 2024-01-24T23:24:12.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:12.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:12.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffacbfff640 10 monclient: dump: 2024-01-24T23:24:12.187 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:12.187 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:12.187 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:12.187 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:12.187 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:12.187 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:12.187 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:12.187 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:12.187 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:12.187 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:12.187 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:12.187 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:12.187 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:12.187 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:12.188 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:12.188 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:12.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 10 monclient: shutdown 2024-01-24T23:24:12.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:12.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 1 -- 172.21.15.132:0/1276440394 >> v1:172.21.15.132:6791/0 conn(0x7ffad4135e10 legacy=0x7ffad409bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:12.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 1 -- 172.21.15.132:0/1276440394 shutdown_connections 2024-01-24T23:24:12.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 1 -- 172.21.15.132:0/1276440394 >> 172.21.15.132:0/1276440394 conn(0x7ffad40a6790 msgr2=0x7ffad4071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:12.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 1 -- 172.21.15.132:0/1276440394 shutdown_connections 2024-01-24T23:24:12.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 1 -- 172.21.15.132:0/1276440394 wait complete. 2024-01-24T23:24:12.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 10 monclient: build_initial_monmap 2024-01-24T23:24:12.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 10 monclient: monmap: 2024-01-24T23:24:12.189 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:12.189 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:12.189 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:12.189 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:12.189 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:12.189 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:12.189 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:12.189 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:12.189 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:12.189 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:12.189 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:12.189 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:12.189 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:12.190 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:12.190 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:12.190 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:12.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 1 Processor -- start 2024-01-24T23:24:12.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 1 -- start start 2024-01-24T23:24:12.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 10 monclient: init 2024-01-24T23:24:12.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:12.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 10 monclient: _add_conns ranks=[6,8,3,0,7,2,5,1,4] 2024-01-24T23:24:12.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 10 monclient(hunting): picked mon.noname-d con 0x7ffad4135e10 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:12.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 10 monclient(hunting): picked mon.noname-e con 0x7ffad41361f0 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:12.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 10 monclient(hunting): picked mon.noname-g con 0x7ffad41365f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:12.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad4122100 con 0x7ffad41365f0 2024-01-24T23:24:12.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad4122630 con 0x7ffad4135e10 2024-01-24T23:24:12.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad409c6b0 con 0x7ffad41361f0 2024-01-24T23:24:12.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:12.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffadb346640 10 monclient(hunting): authenticate will time out at 1139.503535s 2024-01-24T23:24:12.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffad98e3640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7ffad41365f0 0x7ffad41d3960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:38452/0 (socket says 172.21.15.132:38452) 2024-01-24T23:24:12.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffad98e3640 1 -- 172.21.15.132:0/892322062 learned_addr learned my addr 172.21.15.132:0/892322062 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:12.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffac9ffb640 1 -- 172.21.15.132:0/892322062 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 65122701 0 0) 0x7ffad4122100 con 0x7ffad41365f0 2024-01-24T23:24:12.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffac9ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:12.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffac9ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:12.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffac9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:12.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffac9ffb640 10 monclient(hunting): my global_id is 5448 2024-01-24T23:24:12.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.174+0000 7ffac9ffb640 1 -- 172.21.15.132:0/892322062 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffab8003530 con 0x7ffad41365f0 2024-01-24T23:24:12.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 1 -- 172.21.15.132:0/892322062 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1845716887 0 0) 0x7ffab8003530 con 0x7ffad41365f0 2024-01-24T23:24:12.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 1 -- 172.21.15.132:0/892322062 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffad4122100 con 0x7ffad41365f0 2024-01-24T23:24:12.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 1 -- 172.21.15.132:0/892322062 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ffacc001db0 con 0x7ffad41365f0 2024-01-24T23:24:12.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:12.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:12.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 10 monclient(hunting): dump: 2024-01-24T23:24:12.191 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:12.191 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:12.192 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:12.192 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:12.192 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:12.192 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:12.192 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:12.192 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:12.192 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:12.192 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:12.192 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:12.192 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:12.192 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:12.192 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:12.192 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:12.192 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:12.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:12.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 1 -- 172.21.15.132:0/892322062 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 432900472 0 0) 0x7ffad4122100 con 0x7ffad41365f0 2024-01-24T23:24:12.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 1 -- 172.21.15.132:0/892322062 >> v1:172.21.15.115:6793/0 conn(0x7ffad41361f0 legacy=0x7ffad41d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:12.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 1 -- 172.21.15.132:0/892322062 >> v1:172.21.15.115:6792/0 conn(0x7ffad4135e10 legacy=0x7ffad4134c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:12.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:24:12.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 1 monclient: found mon.g 2024-01-24T23:24:12.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:12.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:12.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 1 -- 172.21.15.132:0/892322062 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad41bf250 con 0x7ffad41365f0 2024-01-24T23:24:12.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 10 monclient: _finish_auth 0 2024-01-24T23:24:12.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:24:12.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:12.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffadb346640 5 monclient: authenticate success, global_id 5448 2024-01-24T23:24:12.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffadb346640 10 monclient: _renew_subs 2024-01-24T23:24:12.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffadb346640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:12.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffadb346640 1 -- 172.21.15.132:0/892322062 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad4121d90 con 0x7ffad41365f0 2024-01-24T23:24:12.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffadb346640 10 monclient: _renew_subs 2024-01-24T23:24:12.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffadb346640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:12.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffadb346640 1 -- 172.21.15.132:0/892322062 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad4120ea0 con 0x7ffad41365f0 2024-01-24T23:24:12.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 1 -- 172.21.15.132:0/892322062 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ffacc003c50 con 0x7ffad41365f0 2024-01-24T23:24:12.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:12.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 1 -- 172.21.15.132:0/892322062 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ffacc004180 con 0x7ffad41365f0 2024-01-24T23:24:12.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:12.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:12.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffacb7fe640 4 set_mon_vals no callback set 2024-01-24T23:24:12.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 10 monclient: dump: 2024-01-24T23:24:12.194 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:12.194 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:12.194 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:12.194 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:12.194 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:12.194 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:12.194 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:12.195 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:12.195 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:12.195 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:12.195 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:12.195 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:12.195 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:12.195 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:12.195 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:12.195 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:12.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 1 -- 172.21.15.132:0/892322062 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7ffacc0042e0 con 0x7ffad41365f0 2024-01-24T23:24:12.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffadb346640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:12.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffadb346640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:12.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffadb346640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:12.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffadb346640 1 -- 172.21.15.132:0/892322062 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab4005090 con 0x7ffad41365f0 2024-01-24T23:24:12.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffac9ffb640 1 -- 172.21.15.132:0/892322062 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(79..79 src has 1..79) v4 ==== 5658+0+0 (unknown 2374725793 0 0) 0x7ffacc044310 con 0x7ffad41365f0 2024-01-24T23:24:12.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.178+0000 7ffad90e2640 10 monclient: get_auth_request con 0x7ffab8036f60 auth_method 0 2024-01-24T23:24:12.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.182+0000 7ffac9ffb640 1 -- 172.21.15.132:0/892322062 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7ffacc014c50 con 0x7ffad41365f0 2024-01-24T23:24:12.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.182+0000 7ffac9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:12.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.182+0000 7ffac9ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:12.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.338+0000 7ffadb346640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:12.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.338+0000 7ffadb346640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:12.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.338+0000 7ffadb346640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:12.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.338+0000 7ffadb346640 1 -- 172.21.15.132:0/892322062 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7ffab4005b50 con 0x7ffad41365f0 2024-01-24T23:24:12.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.338+0000 7ffac9ffb640 1 -- 172.21.15.132:0/892322062 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v79) v1 ==== 56+0+1500 (unknown 3183532764 0 1379867665) 0x7ffacc0132f0 con 0x7ffad41365f0 2024-01-24T23:24:12.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.338+0000 7ffac9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:12.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.338+0000 7ffac9ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:24:12.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.426+0000 7ffadb346640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:12.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.426+0000 7ffadb346640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:12.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.426+0000 7ffadb346640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:12.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.426+0000 7ffadb346640 1 -- 172.21.15.132:0/892322062 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7ffab4005ce0 con 0x7ffad41365f0 2024-01-24T23:24:12.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.426+0000 7ffac9ffb640 1 -- 172.21.15.132:0/892322062 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v79) v1 ==== 56+0+1500 (unknown 3183532764 0 1379867665) 0x7ffacc014a70 con 0x7ffad41365f0 2024-01-24T23:24:12.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.426+0000 7ffac9ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:12.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.426+0000 7ffac9ffb640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:24:12.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.430+0000 7ffadb346640 1 -- 172.21.15.132:0/892322062 >> v1:172.21.15.132:6812/1835401922 conn(0x7ffab8036f60 legacy=0x7ffab8039420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:12.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.430+0000 7ffadb346640 10 monclient: shutdown 2024-01-24T23:24:12.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.430+0000 7ffadb346640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:12.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.430+0000 7ffadb346640 1 -- 172.21.15.132:0/892322062 >> v1:172.21.15.132:6790/0 conn(0x7ffad41365f0 legacy=0x7ffad41d3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:12.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.430+0000 7ffadb346640 1 -- 172.21.15.132:0/892322062 shutdown_connections 2024-01-24T23:24:12.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.430+0000 7ffadb346640 1 -- 172.21.15.132:0/892322062 >> 172.21.15.132:0/892322062 conn(0x7ffad40a6790 msgr2=0x7ffad406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:12.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.430+0000 7ffadb346640 1 -- 172.21.15.132:0/892322062 shutdown_connections 2024-01-24T23:24:12.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.430+0000 7ffadb346640 1 -- 172.21.15.132:0/892322062 wait complete. 2024-01-24T23:24:12.445 INFO:tasks.workunit.client.0.smithi132.stdout:-14 0 root root-for-test 2024-01-24T23:24:12.446 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm host-for-test 2024-01-24T23:24:12.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f562b007640 10 monclient: get_monmap_and_config 2024-01-24T23:24:12.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f562b007640 10 monclient: build_initial_monmap 2024-01-24T23:24:12.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f562b007640 10 monclient: monmap: 2024-01-24T23:24:12.549 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:12.549 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:12.549 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:12.549865+0000 2024-01-24T23:24:12.549 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:12.549865+0000 2024-01-24T23:24:12.549 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:12.549 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:12.549 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:12.549 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:12.549 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:12.549 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:12.549 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:12.549 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:12.549 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:12.549 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:12.549 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:12.550 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:12.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f562b007640 1 Processor -- start 2024-01-24T23:24:12.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f562b007640 1 -- start start 2024-01-24T23:24:12.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f562b007640 10 monclient: init 2024-01-24T23:24:12.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f562b007640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:12.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f562b007640 10 monclient: _add_conns ranks=[7,1,3,4,2,0,6,8,5] 2024-01-24T23:24:12.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f562b007640 10 monclient(hunting): picked mon.noname-i con 0x7f5624133b50 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:12.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f562b007640 10 monclient(hunting): picked mon.noname-f con 0x7f562409ec20 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:12.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f562b007640 10 monclient(hunting): picked mon.noname-g con 0x7f562413a1d0 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:12.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f562b007640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5624064700 con 0x7f562409ec20 2024-01-24T23:24:12.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f562b007640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562409d9f0 con 0x7f562413a1d0 2024-01-24T23:24:12.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f562b007640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562409dc60 con 0x7f5624133b50 2024-01-24T23:24:12.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f562b007640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:12.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f562b007640 10 monclient(hunting): authenticate will time out at 1139.874579s 2024-01-24T23:24:12.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f5623fff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f562409ec20 legacy=0x7f5624138ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:12.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56295a4640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f562413a1d0 0x7f562413c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:38458/0 (socket says 172.21.15.132:38458) 2024-01-24T23:24:12.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56295a4640 1 -- 172.21.15.132:0/2097356990 learned_addr learned my addr 172.21.15.132:0/2097356990 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:12.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f5628da3640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f5624133b50 0x7f5624133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:52358/0 (socket says 172.21.15.132:52358) 2024-01-24T23:24:12.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 1 -- 172.21.15.132:0/2097356990 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1202902270 0 0) 0x7f562409dc60 con 0x7f5624133b50 2024-01-24T23:24:12.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 10 monclient(hunting): _negotiate 2024-01-24T23:24:12.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:12.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:12.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 10 monclient(hunting): my global_id is 5524 2024-01-24T23:24:12.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 1 -- 172.21.15.132:0/2097356990 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5610003530 con 0x7f5624133b50 2024-01-24T23:24:12.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 1 -- 172.21.15.132:0/2097356990 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 45661048 0 0) 0x7f562409d9f0 con 0x7f562413a1d0 2024-01-24T23:24:12.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 10 monclient(hunting): _negotiate 2024-01-24T23:24:12.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:12.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:12.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 10 monclient(hunting): my global_id is 5457 2024-01-24T23:24:12.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 1 -- 172.21.15.132:0/2097356990 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f562409dc60 con 0x7f562413a1d0 2024-01-24T23:24:12.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 1 -- 172.21.15.132:0/2097356990 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 470869036 0 0) 0x7f5610003530 con 0x7f5624133b50 2024-01-24T23:24:12.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 1 -- 172.21.15.132:0/2097356990 >> v1:172.21.15.132:6790/0 conn(0x7f562413a1d0 legacy=0x7f562413c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:12.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 1 -- 172.21.15.132:0/2097356990 >> v1:172.21.15.132:6789/0 conn(0x7f562409ec20 legacy=0x7f5624138ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:12.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 10 monclient: _finish_hunting 0 2024-01-24T23:24:12.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 1 monclient: found mon.noname-i 2024-01-24T23:24:12.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:12.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:24:12.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 1 -- 172.21.15.132:0/2097356990 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562409df40 con 0x7f5624133b50 2024-01-24T23:24:12.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 10 monclient: _finish_auth 0 2024-01-24T23:24:12.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 10 monclient: _check_auth_tickets 2024-01-24T23:24:12.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:12.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 1 -- 172.21.15.132:0/2097356990 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f560c0017c0 con 0x7f5624133b50 2024-01-24T23:24:12.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:12.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:12.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 10 monclient: dump: 2024-01-24T23:24:12.559 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:12.559 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:12.559 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:12.559 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:12.559 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:12.559 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:12.559 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:12.559 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:12.559 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:12.559 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:12.559 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:12.559 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:12.559 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:12.559 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:12.559 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:12.559 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:12.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f562b007640 5 monclient: authenticate success, global_id 5524 2024-01-24T23:24:12.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f562b007640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:12.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 1 -- 172.21.15.132:0/2097356990 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f560c001aa0 con 0x7f5624133b50 2024-01-24T23:24:12.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:12.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f562b007640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:12.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f562b007640 4 set_mon_vals no callback set 2024-01-24T23:24:12.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 1 -- 172.21.15.132:0/2097356990 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f560c001c90 con 0x7f5624133b50 2024-01-24T23:24:12.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:12.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:12.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f56237fe640 10 monclient: dump: 2024-01-24T23:24:12.560 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:12.560 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:12.560 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:12.560 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:12.560 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:12.560 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:12.560 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:12.561 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:12.561 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:12.561 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:12.561 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:12.561 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:12.561 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:12.561 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:12.561 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:12.561 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:12.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f562b007640 10 monclient: shutdown 2024-01-24T23:24:12.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f562b007640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:12.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.546+0000 7f562b007640 1 -- 172.21.15.132:0/2097356990 >> v1:172.21.15.132:6792/0 conn(0x7f5624133b50 legacy=0x7f5624133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:12.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 1 -- 172.21.15.132:0/2097356990 shutdown_connections 2024-01-24T23:24:12.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 1 -- 172.21.15.132:0/2097356990 >> 172.21.15.132:0/2097356990 conn(0x7f56240a5a10 msgr2=0x7f56240a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:12.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 1 -- 172.21.15.132:0/2097356990 shutdown_connections 2024-01-24T23:24:12.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 1 -- 172.21.15.132:0/2097356990 wait complete. 2024-01-24T23:24:12.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 10 monclient: build_initial_monmap 2024-01-24T23:24:12.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 10 monclient: monmap: 2024-01-24T23:24:12.562 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:12.562 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:12.562 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:12.562 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:12.562 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:12.562 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:12.562 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:12.562 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:12.562 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:12.562 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:12.562 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:12.562 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:12.563 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:12.563 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:12.563 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:12.563 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:12.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 1 Processor -- start 2024-01-24T23:24:12.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 1 -- start start 2024-01-24T23:24:12.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 10 monclient: init 2024-01-24T23:24:12.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:12.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 10 monclient: _add_conns ranks=[7,4,0,2,8,5,6,3,1] 2024-01-24T23:24:12.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 10 monclient(hunting): picked mon.noname-i con 0x7f562409ec20 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:12.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 10 monclient(hunting): picked mon.noname-c con 0x7f5624133b50 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:12.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 10 monclient(hunting): picked mon.noname-a con 0x7f562413a1d0 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:12.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562411d530 con 0x7f562413a1d0 2024-01-24T23:24:12.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56240a13b0 con 0x7f5624133b50 2024-01-24T23:24:12.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562411e830 con 0x7f562409ec20 2024-01-24T23:24:12.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:12.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 10 monclient(hunting): authenticate will time out at 1139.877428s 2024-01-24T23:24:12.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f5628da3640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f562409ec20 0x7f562411b610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:52362/0 (socket says 172.21.15.132:52362) 2024-01-24T23:24:12.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f5628da3640 1 -- 172.21.15.132:0/1306702083 learned_addr learned my addr 172.21.15.132:0/1306702083 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:12.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f5623fff640 1 -- 172.21.15.132:0/1306702083 >> v1:172.21.15.115:6791/0 conn(0x7f5624133b50 legacy=0x7f562411bd20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:12.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56295a4640 1 -- 172.21.15.132:0/1306702083 >> v1:172.21.15.115:6789/0 conn(0x7f562413a1d0 legacy=0x7f56241d15c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:12.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 1 -- 172.21.15.132:0/1306702083 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3207515014 0 0) 0x7f562411e830 con 0x7f562409ec20 2024-01-24T23:24:12.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 10 monclient(hunting): _negotiate 2024-01-24T23:24:12.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:12.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:12.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 10 monclient(hunting): my global_id is 5533 2024-01-24T23:24:12.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 1 -- 172.21.15.132:0/1306702083 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5604003530 con 0x7f562409ec20 2024-01-24T23:24:12.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 1 -- 172.21.15.132:0/1306702083 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1686405028 0 0) 0x7f5604003530 con 0x7f562409ec20 2024-01-24T23:24:12.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 1 -- 172.21.15.132:0/1306702083 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f562411e830 con 0x7f562409ec20 2024-01-24T23:24:12.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 1 -- 172.21.15.132:0/1306702083 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f560c001820 con 0x7f562409ec20 2024-01-24T23:24:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 10 monclient(hunting): dump: 2024-01-24T23:24:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:12.566 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:12.566 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:12.566 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:12.566 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:12.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:12.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 1 -- 172.21.15.132:0/1306702083 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3711933203 0 0) 0x7f562411e830 con 0x7f562409ec20 2024-01-24T23:24:12.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 1 -- 172.21.15.132:0/1306702083 >> v1:172.21.15.115:6791/0 conn(0x7f5624133b50 legacy=0x7f562411bd20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:12.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 1 -- 172.21.15.132:0/1306702083 >> v1:172.21.15.115:6789/0 conn(0x7f562413a1d0 legacy=0x7f56241d15c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:12.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 10 monclient: _finish_hunting 0 2024-01-24T23:24:12.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 1 monclient: found mon.i 2024-01-24T23:24:12.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:12.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:12.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 1 -- 172.21.15.132:0/1306702083 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562411c430 con 0x7f562409ec20 2024-01-24T23:24:12.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 10 monclient: _finish_auth 0 2024-01-24T23:24:12.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 10 monclient: _check_auth_tickets 2024-01-24T23:24:12.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:12.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 5 monclient: authenticate success, global_id 5533 2024-01-24T23:24:12.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 10 monclient: _renew_subs 2024-01-24T23:24:12.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:12.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 1 -- 172.21.15.132:0/1306702083 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f562411eb10 con 0x7f562409ec20 2024-01-24T23:24:12.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 10 monclient: _renew_subs 2024-01-24T23:24:12.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:12.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 1 -- 172.21.15.132:0/1306702083 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f562411e550 con 0x7f562409ec20 2024-01-24T23:24:12.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 1 -- 172.21.15.132:0/1306702083 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f560c004e80 con 0x7f562409ec20 2024-01-24T23:24:12.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:12.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 1 -- 172.21.15.132:0/1306702083 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f560c005330 con 0x7f562409ec20 2024-01-24T23:24:12.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:12.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:12.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 10 monclient: dump: 2024-01-24T23:24:12.567 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:12.567 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:12.568 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:12.568 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:12.568 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:12.568 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:12.568 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:12.568 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:12.568 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:12.568 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:12.568 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:12.568 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:12.568 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:12.568 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:12.568 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:12.568 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:12.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f5622ffd640 4 set_mon_vals no callback set 2024-01-24T23:24:12.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:12.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:12.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:12.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f562b007640 1 -- 172.21.15.132:0/1306702083 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5608005090 con 0x7f562409ec20 2024-01-24T23:24:12.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 1 -- 172.21.15.132:0/1306702083 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f560c00fdb0 con 0x7f562409ec20 2024-01-24T23:24:12.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f56217fa640 1 -- 172.21.15.132:0/1306702083 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(79..79 src has 1..79) v4 ==== 5658+0+0 (unknown 2374725793 0 0) 0x7f560c044190 con 0x7f562409ec20 2024-01-24T23:24:12.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.550+0000 7f5623fff640 10 monclient: get_auth_request con 0x7f5604037010 auth_method 0 2024-01-24T23:24:12.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.554+0000 7f56217fa640 1 -- 172.21.15.132:0/1306702083 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f560c014ba0 con 0x7f562409ec20 2024-01-24T23:24:12.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.554+0000 7f56217fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:12.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.554+0000 7f56217fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:12.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.714+0000 7f562b007640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host-for-test"}] 2024-01-24T23:24:12.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.714+0000 7f562b007640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "host-for-test"}] 2024-01-24T23:24:12.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.714+0000 7f562b007640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:12.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:12.714+0000 7f562b007640 1 -- 172.21.15.132:0/1306702083 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "host-for-test"} v 0) v1 -- 0x7f5608005340 con 0x7f562409ec20 2024-01-24T23:24:13.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.182+0000 7f56217fa640 1 -- 172.21.15.132:0/1306702083 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host-for-test"}]=0 removed item id -11 name 'host-for-test' from crush map v80) v1 ==== 140+0+0 (unknown 4187360450 0 0) 0x7f560c013240 con 0x7f562409ec20 2024-01-24T23:24:13.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.182+0000 7f56217fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "host-for-test"}] 2024-01-24T23:24:13.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.182+0000 7f56217fa640 10 monclient: _finish_command 2 = system:0 removed item id -11 name 'host-for-test' from crush map 2024-01-24T23:24:13.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.274+0000 7f562b007640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host-for-test"}] 2024-01-24T23:24:13.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.274+0000 7f562b007640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "host-for-test"}] 2024-01-24T23:24:13.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.274+0000 7f562b007640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:13.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.274+0000 7f562b007640 1 -- 172.21.15.132:0/1306702083 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "host-for-test"} v 0) v1 -- 0x7f5608002220 con 0x7f562409ec20 2024-01-24T23:24:13.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.278+0000 7f56217fa640 1 -- 172.21.15.132:0/1306702083 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host-for-test"}]=0 device 'host-for-test' does not appear in the crush map v80) v1 ==== 140+0+0 (unknown 384835204 0 0) 0x7f560c0149c0 con 0x7f562409ec20 2024-01-24T23:24:13.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.278+0000 7f56217fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "host-for-test"}] 2024-01-24T23:24:13.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.278+0000 7f56217fa640 10 monclient: _finish_command 3 = system:0 device 'host-for-test' does not appear in the crush map 2024-01-24T23:24:13.281 INFO:tasks.workunit.client.0.smithi132.stderr:device 'host-for-test' does not appear in the crush map 2024-01-24T23:24:13.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.278+0000 7f562b007640 1 -- 172.21.15.132:0/1306702083 >> v1:172.21.15.132:6812/1835401922 conn(0x7f5604037010 legacy=0x7f56040394d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:13.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.278+0000 7f562b007640 10 monclient: shutdown 2024-01-24T23:24:13.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.278+0000 7f562b007640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:13.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.278+0000 7f562b007640 1 -- 172.21.15.132:0/1306702083 >> v1:172.21.15.132:6792/0 conn(0x7f562409ec20 legacy=0x7f562411b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:13.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.278+0000 7f562b007640 1 -- 172.21.15.132:0/1306702083 shutdown_connections 2024-01-24T23:24:13.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.278+0000 7f562b007640 1 -- 172.21.15.132:0/1306702083 >> 172.21.15.132:0/1306702083 conn(0x7f56240a5a10 msgr2=0x7f56240a0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:13.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.278+0000 7f562b007640 1 -- 172.21.15.132:0/1306702083 shutdown_connections 2024-01-24T23:24:13.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.278+0000 7f562b007640 1 -- 172.21.15.132:0/1306702083 wait complete. 2024-01-24T23:24:13.298 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm rack-for-test 2024-01-24T23:24:13.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.406+0000 7f021378d640 10 monclient: get_monmap_and_config 2024-01-24T23:24:13.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.406+0000 7f021378d640 10 monclient: build_initial_monmap 2024-01-24T23:24:13.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.406+0000 7f021378d640 10 monclient: monmap: 2024-01-24T23:24:13.410 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:13.410 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:13.410 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:13.411725+0000 2024-01-24T23:24:13.411 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:13.411725+0000 2024-01-24T23:24:13.411 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:13.411 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:13.411 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:13.411 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:13.411 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:13.411 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:13.411 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:13.411 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:13.411 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:13.411 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:13.411 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:13.411 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:13.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.406+0000 7f021378d640 1 Processor -- start 2024-01-24T23:24:13.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.406+0000 7f021378d640 1 -- start start 2024-01-24T23:24:13.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.406+0000 7f021378d640 10 monclient: init 2024-01-24T23:24:13.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:13.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 10 monclient: _add_conns ranks=[8,3,6,0,5,2,1,4,7] 2024-01-24T23:24:13.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 10 monclient(hunting): picked mon.noname-e con 0x7f020c133b50 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:13.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 10 monclient(hunting): picked mon.noname-g con 0x7f020c09ec20 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:13.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 10 monclient(hunting): picked mon.noname-d con 0x7f020c13a1d0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:13.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c064700 con 0x7f020c09ec20 2024-01-24T23:24:13.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c09d9f0 con 0x7f020c13a1d0 2024-01-24T23:24:13.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c09dc60 con 0x7f020c133b50 2024-01-24T23:24:13.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:13.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 10 monclient(hunting): authenticate will time out at 1140.736323s 2024-01-24T23:24:13.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0210d28640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f020c09ec20 0x7f020c138ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:38466/0 (socket says 172.21.15.132:38466) 2024-01-24T23:24:13.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0210d28640 1 -- 172.21.15.132:0/1229117134 learned_addr learned my addr 172.21.15.132:0/1229117134 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:13.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 1 -- 172.21.15.132:0/1229117134 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2384818339 0 0) 0x7f020c064700 con 0x7f020c09ec20 2024-01-24T23:24:13.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:13.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:13.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:13.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 10 monclient(hunting): my global_id is 5466 2024-01-24T23:24:13.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 1 -- 172.21.15.132:0/1229117134 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01f8003530 con 0x7f020c09ec20 2024-01-24T23:24:13.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 1 -- 172.21.15.132:0/1229117134 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3209413191 0 0) 0x7f01f8003530 con 0x7f020c09ec20 2024-01-24T23:24:13.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 1 -- 172.21.15.132:0/1229117134 >> v1:172.21.15.115:6793/0 conn(0x7f020c133b50 legacy=0x7f020c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:13.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 1 -- 172.21.15.132:0/1229117134 >> v1:172.21.15.115:6792/0 conn(0x7f020c13a1d0 legacy=0x7f020c13c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:13.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 10 monclient: _finish_hunting 0 2024-01-24T23:24:13.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 1 monclient: found mon.noname-g 2024-01-24T23:24:13.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:13.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:24:13.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 1 -- 172.21.15.132:0/1229117134 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c09df40 con 0x7f020c09ec20 2024-01-24T23:24:13.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 10 monclient: _finish_auth 0 2024-01-24T23:24:13.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 10 monclient: _check_auth_tickets 2024-01-24T23:24:13.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:13.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 1 -- 172.21.15.132:0/1229117134 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f01fc0017c0 con 0x7f020c09ec20 2024-01-24T23:24:13.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:13.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:13.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 10 monclient: dump: 2024-01-24T23:24:13.420 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:13.420 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:13.420 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:13.420 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:13.420 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:13.420 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:13.420 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:13.420 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:13.421 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:13.421 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:13.421 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:13.421 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:13.421 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:13.421 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:13.421 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:13.421 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:13.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 5 monclient: authenticate success, global_id 5466 2024-01-24T23:24:13.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:13.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 1 -- 172.21.15.132:0/1229117134 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f01fc001ae0 con 0x7f020c09ec20 2024-01-24T23:24:13.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:13.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 1 -- 172.21.15.132:0/1229117134 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f01fc004bf0 con 0x7f020c09ec20 2024-01-24T23:24:13.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:13.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 4 set_mon_vals no callback set 2024-01-24T23:24:13.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:13.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:13.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0203fff640 10 monclient: dump: 2024-01-24T23:24:13.422 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:13.422 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:13.422 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:13.422 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:13.422 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:13.422 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:13.422 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:13.422 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:13.422 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:13.422 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:13.422 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:13.422 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:13.422 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:13.422 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:13.423 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:13.423 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:13.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 10 monclient: shutdown 2024-01-24T23:24:13.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:13.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 1 -- 172.21.15.132:0/1229117134 >> v1:172.21.15.132:6790/0 conn(0x7f020c09ec20 legacy=0x7f020c138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:13.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 1 -- 172.21.15.132:0/1229117134 shutdown_connections 2024-01-24T23:24:13.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 1 -- 172.21.15.132:0/1229117134 >> 172.21.15.132:0/1229117134 conn(0x7f020c0a5a10 msgr2=0x7f020c0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:13.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 1 -- 172.21.15.132:0/1229117134 shutdown_connections 2024-01-24T23:24:13.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 1 -- 172.21.15.132:0/1229117134 wait complete. 2024-01-24T23:24:13.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 10 monclient: build_initial_monmap 2024-01-24T23:24:13.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 10 monclient: monmap: 2024-01-24T23:24:13.423 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:13.423 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:13.423 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:13.423 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:13.423 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:13.424 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:13.424 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:13.424 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:13.424 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:13.424 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:13.424 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:13.424 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:13.424 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:13.424 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:13.424 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:13.424 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:13.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 1 Processor -- start 2024-01-24T23:24:13.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 1 -- start start 2024-01-24T23:24:13.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 10 monclient: init 2024-01-24T23:24:13.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:13.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 10 monclient: _add_conns ranks=[6,1,4,8,2,5,3,7,0] 2024-01-24T23:24:13.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 10 monclient(hunting): picked mon.noname-d con 0x7f020c09ec20 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:13.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 10 monclient(hunting): picked mon.noname-f con 0x7f020c133b50 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:13.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 10 monclient(hunting): picked mon.noname-c con 0x7f020c13a1d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:13.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c0a13b0 con 0x7f020c133b50 2024-01-24T23:24:13.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c0705c0 con 0x7f020c13a1d0 2024-01-24T23:24:13.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c06dc20 con 0x7f020c09ec20 2024-01-24T23:24:13.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:13.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f021378d640 10 monclient(hunting): authenticate will time out at 1140.739255s 2024-01-24T23:24:13.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0210d28640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f020c133b50 legacy=0x7f020c06d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:13.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0211d2a640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f020c13a1d0 legacy=0x7f020c1d37f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:13.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0211529640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f020c09ec20 0x7f020c06ce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:57598/0 (socket says 172.21.15.132:57598) 2024-01-24T23:24:13.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0211529640 1 -- 172.21.15.132:0/2638477105 learned_addr learned my addr 172.21.15.132:0/2638477105 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:13.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0201ffb640 1 -- 172.21.15.132:0/2638477105 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 982438247 0 0) 0x7f020c06dc20 con 0x7f020c09ec20 2024-01-24T23:24:13.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0201ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:13.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0201ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:13.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0201ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:13.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0201ffb640 10 monclient(hunting): my global_id is 5523 2024-01-24T23:24:13.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.410+0000 7f0201ffb640 1 -- 172.21.15.132:0/2638477105 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01f0003560 con 0x7f020c09ec20 2024-01-24T23:24:13.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 1 -- 172.21.15.132:0/2638477105 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2348068452 0 0) 0x7f01f0003560 con 0x7f020c09ec20 2024-01-24T23:24:13.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 1 -- 172.21.15.132:0/2638477105 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f020c06dc20 con 0x7f020c09ec20 2024-01-24T23:24:13.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 1 -- 172.21.15.132:0/2638477105 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0208001ae0 con 0x7f020c09ec20 2024-01-24T23:24:13.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:13.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:24:13.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 10 monclient(hunting): dump: 2024-01-24T23:24:13.426 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:13.426 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:13.426 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:13.426 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:13.426 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:13.426 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:13.426 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:13.426 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:13.427 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:13.427 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:13.427 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:13.427 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:13.427 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:13.427 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:13.427 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:13.427 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:13.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:13.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 1 -- 172.21.15.132:0/2638477105 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1234480362 0 0) 0x7f020c06dc20 con 0x7f020c09ec20 2024-01-24T23:24:13.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 1 -- 172.21.15.132:0/2638477105 >> v1:172.21.15.115:6791/0 conn(0x7f020c13a1d0 legacy=0x7f020c1d37f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:13.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 1 -- 172.21.15.132:0/2638477105 >> v1:172.21.15.132:6789/0 conn(0x7f020c133b50 legacy=0x7f020c06d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:13.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:24:13.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 1 monclient: found mon.d 2024-01-24T23:24:13.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:13.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:13.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 1 -- 172.21.15.132:0/2638477105 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c0708a0 con 0x7f020c09ec20 2024-01-24T23:24:13.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 10 monclient: _finish_auth 0 2024-01-24T23:24:13.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:24:13.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:13.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f021378d640 5 monclient: authenticate success, global_id 5523 2024-01-24T23:24:13.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f021378d640 10 monclient: _renew_subs 2024-01-24T23:24:13.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f021378d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:13.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f021378d640 1 -- 172.21.15.132:0/2638477105 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020c133920 con 0x7f020c09ec20 2024-01-24T23:24:13.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f021378d640 10 monclient: _renew_subs 2024-01-24T23:24:13.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f021378d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:13.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f021378d640 1 -- 172.21.15.132:0/2638477105 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f020c06ee60 con 0x7f020c09ec20 2024-01-24T23:24:13.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 1 -- 172.21.15.132:0/2638477105 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f02080038d0 con 0x7f020c09ec20 2024-01-24T23:24:13.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:13.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 1 -- 172.21.15.132:0/2638477105 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0208003e00 con 0x7f020c09ec20 2024-01-24T23:24:13.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:13.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f02037fe640 4 set_mon_vals no callback set 2024-01-24T23:24:13.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:24:13.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 10 monclient: dump: 2024-01-24T23:24:13.429 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:13.429 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:13.429 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:13.429 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:13.429 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:13.429 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:13.429 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:13.429 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:13.429 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:13.429 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:13.429 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:13.429 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:13.430 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:13.430 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:13.430 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:13.430 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:13.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f021378d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:13.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f021378d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:13.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f021378d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:13.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f021378d640 1 -- 172.21.15.132:0/2638477105 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ec005090 con 0x7f020c09ec20 2024-01-24T23:24:13.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 1 -- 172.21.15.132:0/2638477105 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f0208003fe0 con 0x7f020c09ec20 2024-01-24T23:24:13.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0201ffb640 1 -- 172.21.15.132:0/2638477105 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(80..80 src has 1..80) v4 ==== 5475+0+0 (unknown 245390414 0 0) 0x7f0208043f00 con 0x7f020c09ec20 2024-01-24T23:24:13.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.414+0000 7f0210d28640 10 monclient: get_auth_request con 0x7f01f0036f40 auth_method 0 2024-01-24T23:24:13.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.426+0000 7f0201ffb640 1 -- 172.21.15.132:0/2638477105 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f02080148f0 con 0x7f020c09ec20 2024-01-24T23:24:13.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.426+0000 7f0201ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:13.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.426+0000 7f0201ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:13.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.574+0000 7f021378d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "rack-for-test"}] 2024-01-24T23:24:13.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.574+0000 7f021378d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "rack-for-test"}] 2024-01-24T23:24:13.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.574+0000 7f021378d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:13.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:13.574+0000 7f021378d640 1 -- 172.21.15.132:0/2638477105 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "rack-for-test"} v 0) v1 -- 0x7f01ec005340 con 0x7f020c09ec20 2024-01-24T23:24:14.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.190+0000 7f0201ffb640 1 -- 172.21.15.132:0/2638477105 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "rack-for-test"}]=0 removed item id -12 name 'rack-for-test' from crush map v81) v1 ==== 140+0+0 (unknown 2328589471 0 0) 0x7f0208014d50 con 0x7f020c09ec20 2024-01-24T23:24:14.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.190+0000 7f0201ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "rack-for-test"}] 2024-01-24T23:24:14.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.190+0000 7f0201ffb640 10 monclient: _finish_command 2 = system:0 removed item id -12 name 'rack-for-test' from crush map 2024-01-24T23:24:14.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.294+0000 7f021378d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "rack-for-test"}] 2024-01-24T23:24:14.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.294+0000 7f021378d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "rack-for-test"}] 2024-01-24T23:24:14.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.294+0000 7f021378d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:14.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.294+0000 7f021378d640 1 -- 172.21.15.132:0/2638477105 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "rack-for-test"} v 0) v1 -- 0x7f01ec0010a0 con 0x7f020c09ec20 2024-01-24T23:24:14.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.294+0000 7f0201ffb640 1 -- 172.21.15.132:0/2638477105 <== mon.6 v1:172.21.15.115:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "rack-for-test"}]=0 device 'rack-for-test' does not appear in the crush map v81) v1 ==== 140+0+0 (unknown 2594238683 0 0) 0x7f020801d020 con 0x7f020c09ec20 2024-01-24T23:24:14.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.294+0000 7f0201ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "rack-for-test"}] 2024-01-24T23:24:14.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.294+0000 7f0201ffb640 10 monclient: _finish_command 3 = system:0 device 'rack-for-test' does not appear in the crush map 2024-01-24T23:24:14.298 INFO:tasks.workunit.client.0.smithi132.stderr:device 'rack-for-test' does not appear in the crush map 2024-01-24T23:24:14.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.298+0000 7f021378d640 1 -- 172.21.15.132:0/2638477105 >> v1:172.21.15.132:6812/1835401922 conn(0x7f01f0036f40 legacy=0x7f01f0039400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:14.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.298+0000 7f021378d640 10 monclient: shutdown 2024-01-24T23:24:14.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.298+0000 7f021378d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:14.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.298+0000 7f021378d640 1 -- 172.21.15.132:0/2638477105 >> v1:172.21.15.115:6792/0 conn(0x7f020c09ec20 legacy=0x7f020c06ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:14.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.298+0000 7f021378d640 1 -- 172.21.15.132:0/2638477105 shutdown_connections 2024-01-24T23:24:14.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.298+0000 7f021378d640 1 -- 172.21.15.132:0/2638477105 >> 172.21.15.132:0/2638477105 conn(0x7f020c0a5a10 msgr2=0x7f020c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:14.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.298+0000 7f021378d640 1 -- 172.21.15.132:0/2638477105 shutdown_connections 2024-01-24T23:24:14.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.298+0000 7f021378d640 1 -- 172.21.15.132:0/2638477105 wait complete. 2024-01-24T23:24:14.315 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm root-for-test 2024-01-24T23:24:14.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.426+0000 7f873efd1640 10 monclient: get_monmap_and_config 2024-01-24T23:24:14.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.426+0000 7f873efd1640 10 monclient: build_initial_monmap 2024-01-24T23:24:14.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.426+0000 7f873efd1640 10 monclient: monmap: 2024-01-24T23:24:14.431 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:14.431 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:14.431 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:14.432269+0000 2024-01-24T23:24:14.431 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:14.432269+0000 2024-01-24T23:24:14.431 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:14.431 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:14.431 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:14.431 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:14.431 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:14.432 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:14.432 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:14.432 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:14.432 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:14.432 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:14.432 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:14.432 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:14.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 1 Processor -- start 2024-01-24T23:24:14.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 1 -- start start 2024-01-24T23:24:14.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 10 monclient: init 2024-01-24T23:24:14.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:14.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 10 monclient: _add_conns ranks=[0,8,3,7,6,1,4,5,2] 2024-01-24T23:24:14.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 10 monclient(hunting): picked mon.noname-a con 0x7f8738131cc0 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:14.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 10 monclient(hunting): picked mon.noname-e con 0x7f87381320a0 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:14.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 10 monclient(hunting): picked mon.noname-g con 0x7f87381324a0 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:14.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8738064700 con 0x7f8738131cc0 2024-01-24T23:24:14.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8738070ff0 con 0x7f87381324a0 2024-01-24T23:24:14.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f873806f7d0 con 0x7f87381320a0 2024-01-24T23:24:14.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:14.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 10 monclient(hunting): authenticate will time out at 1141.757520s 2024-01-24T23:24:14.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873d56e640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f87381324a0 0x7f873813d4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:47564/0 (socket says 172.21.15.132:47564) 2024-01-24T23:24:14.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873cd6d640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f8738131cc0 legacy=0x7f8738071410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:14.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873d56e640 1 -- 172.21.15.132:0/3440499492 learned_addr learned my addr 172.21.15.132:0/3440499492 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:14.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 1 -- 172.21.15.132:0/3440499492 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1986501324 0 0) 0x7f8738070ff0 con 0x7f87381324a0 2024-01-24T23:24:14.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 10 monclient(hunting): _negotiate 2024-01-24T23:24:14.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:14.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:14.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 10 monclient(hunting): my global_id is 5475 2024-01-24T23:24:14.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 1 -- 172.21.15.132:0/3440499492 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8720003530 con 0x7f87381324a0 2024-01-24T23:24:14.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 1 -- 172.21.15.132:0/3440499492 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 372236578 0 0) 0x7f8720003530 con 0x7f87381324a0 2024-01-24T23:24:14.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 1 -- 172.21.15.132:0/3440499492 >> v1:172.21.15.115:6793/0 conn(0x7f87381320a0 legacy=0x7f8738139da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:14.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 1 -- 172.21.15.132:0/3440499492 >> v1:172.21.15.115:6789/0 conn(0x7f8738131cc0 legacy=0x7f8738071410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:14.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 10 monclient: _finish_hunting 0 2024-01-24T23:24:14.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 1 monclient: found mon.noname-g 2024-01-24T23:24:14.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:14.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:24:14.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 1 -- 172.21.15.132:0/3440499492 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8738070750 con 0x7f87381324a0 2024-01-24T23:24:14.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 10 monclient: _finish_auth 0 2024-01-24T23:24:14.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 10 monclient: _check_auth_tickets 2024-01-24T23:24:14.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:14.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 5 monclient: authenticate success, global_id 5475 2024-01-24T23:24:14.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 1 -- 172.21.15.132:0/3440499492 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f87300017c0 con 0x7f87381324a0 2024-01-24T23:24:14.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:14.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:14.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:14.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 10 monclient: dump: 2024-01-24T23:24:14.438 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:14.438 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:14.438 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:14.438 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:14.438 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:14.438 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:14.438 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:14.438 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:14.438 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:14.438 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:14.438 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:14.438 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:14.438 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:14.438 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:14.438 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:14.439 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:14.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:14.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 1 -- 172.21.15.132:0/3440499492 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8730001ae0 con 0x7f87381324a0 2024-01-24T23:24:14.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:14.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 1 -- 172.21.15.132:0/3440499492 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8730004bf0 con 0x7f87381324a0 2024-01-24T23:24:14.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:14.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 4 set_mon_vals no callback set 2024-01-24T23:24:14.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:14.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:14.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f87377fe640 10 monclient: dump: 2024-01-24T23:24:14.439 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:14.439 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:14.439 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:14.439 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:14.439 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:14.440 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:14.440 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:14.440 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:14.440 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:14.440 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:14.440 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:14.440 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:14.440 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:14.440 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:14.440 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:14.440 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:14.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 10 monclient: shutdown 2024-01-24T23:24:14.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:14.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 1 -- 172.21.15.132:0/3440499492 >> v1:172.21.15.132:6790/0 conn(0x7f87381324a0 legacy=0x7f873813d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:14.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 1 -- 172.21.15.132:0/3440499492 shutdown_connections 2024-01-24T23:24:14.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 1 -- 172.21.15.132:0/3440499492 >> 172.21.15.132:0/3440499492 conn(0x7f87380a7150 msgr2=0x7f87380a7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:14.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 1 -- 172.21.15.132:0/3440499492 shutdown_connections 2024-01-24T23:24:14.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 1 -- 172.21.15.132:0/3440499492 wait complete. 2024-01-24T23:24:14.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 10 monclient: build_initial_monmap 2024-01-24T23:24:14.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 10 monclient: monmap: 2024-01-24T23:24:14.441 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:14.441 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:14.441 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:14.441 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:14.441 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:14.441 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:14.441 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:14.441 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:14.441 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:14.441 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:14.442 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:14.442 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:14.442 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:14.442 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:14.442 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:14.442 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:14.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.430+0000 7f873efd1640 1 Processor -- start 2024-01-24T23:24:14.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873efd1640 1 -- start start 2024-01-24T23:24:14.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873efd1640 10 monclient: init 2024-01-24T23:24:14.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873efd1640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:14.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873efd1640 10 monclient: _add_conns ranks=[1,8,5,0,6,2,4,3,7] 2024-01-24T23:24:14.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873efd1640 10 monclient(hunting): picked mon.noname-f con 0x7f8738131cc0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:14.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873efd1640 10 monclient(hunting): picked mon.noname-e con 0x7f87381320a0 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:14.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873efd1640 10 monclient(hunting): picked mon.noname-h con 0x7f87381324a0 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:14.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873efd1640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87381226a0 con 0x7f8738131cc0 2024-01-24T23:24:14.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873cd6d640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f8738131cc0 legacy=0x7f8738133210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:14.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873efd1640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8738134b80 con 0x7f87381324a0 2024-01-24T23:24:14.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873efd1640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8738070270 con 0x7f87381320a0 2024-01-24T23:24:14.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873efd1640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:14.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873efd1640 10 monclient(hunting): authenticate will time out at 1141.760491s 2024-01-24T23:24:14.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873d56e640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f87381324a0 0x7f87381d39e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:51140/0 (socket says 172.21.15.132:51140) 2024-01-24T23:24:14.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873d56e640 1 -- 172.21.15.132:0/155713796 learned_addr learned my addr 172.21.15.132:0/155713796 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:14.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 1 -- 172.21.15.132:0/155713796 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4166981556 0 0) 0x7f8738134b80 con 0x7f87381324a0 2024-01-24T23:24:14.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 10 monclient(hunting): _negotiate 2024-01-24T23:24:14.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:14.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:14.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 10 monclient(hunting): my global_id is 5477 2024-01-24T23:24:14.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 1 -- 172.21.15.132:0/155713796 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8724003530 con 0x7f87381324a0 2024-01-24T23:24:14.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 1 -- 172.21.15.132:0/155713796 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2953569765 0 0) 0x7f8724003530 con 0x7f87381324a0 2024-01-24T23:24:14.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 1 -- 172.21.15.132:0/155713796 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8738134b80 con 0x7f87381324a0 2024-01-24T23:24:14.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 1 -- 172.21.15.132:0/155713796 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f87300018a0 con 0x7f87381324a0 2024-01-24T23:24:14.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:14.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:14.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 10 monclient(hunting): dump: 2024-01-24T23:24:14.446 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:14.446 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:14.447 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:14.447 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:14.447 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:14.447 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:14.447 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:14.447 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:14.447 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:14.447 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:14.447 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:14.447 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:14.447 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:14.447 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:14.447 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:14.447 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:14.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:14.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 1 -- 172.21.15.132:0/155713796 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 805276251 0 0) 0x7f8738134b80 con 0x7f87381324a0 2024-01-24T23:24:14.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 1 -- 172.21.15.132:0/155713796 >> v1:172.21.15.115:6793/0 conn(0x7f87381320a0 legacy=0x7f8738133920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:14.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 1 -- 172.21.15.132:0/155713796 >> v1:172.21.15.132:6789/0 conn(0x7f8738131cc0 legacy=0x7f8738133210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:14.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 10 monclient: _finish_hunting 0 2024-01-24T23:24:14.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 1 monclient: found mon.h 2024-01-24T23:24:14.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:14.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:14.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 1 -- 172.21.15.132:0/155713796 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8738072080 con 0x7f87381324a0 2024-01-24T23:24:14.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 10 monclient: _finish_auth 0 2024-01-24T23:24:14.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 10 monclient: _check_auth_tickets 2024-01-24T23:24:14.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:14.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873efd1640 5 monclient: authenticate success, global_id 5477 2024-01-24T23:24:14.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873efd1640 10 monclient: _renew_subs 2024-01-24T23:24:14.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873efd1640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:14.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873efd1640 1 -- 172.21.15.132:0/155713796 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8738122f40 con 0x7f87381324a0 2024-01-24T23:24:14.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873efd1640 10 monclient: _renew_subs 2024-01-24T23:24:14.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873efd1640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:14.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873efd1640 1 -- 172.21.15.132:0/155713796 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8738121bb0 con 0x7f87381324a0 2024-01-24T23:24:14.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 1 -- 172.21.15.132:0/155713796 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8730001200 con 0x7f87381324a0 2024-01-24T23:24:14.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:14.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 1 -- 172.21.15.132:0/155713796 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8730005120 con 0x7f87381324a0 2024-01-24T23:24:14.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:14.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:14.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f8736ffd640 4 set_mon_vals no callback set 2024-01-24T23:24:14.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 10 monclient: dump: 2024-01-24T23:24:14.449 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:14.449 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:14.449 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:14.449 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:14.449 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:14.450 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:14.450 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:14.450 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:14.450 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:14.450 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:14.450 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:14.450 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:14.450 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:14.450 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:14.450 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:14.450 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:14.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 1 -- 172.21.15.132:0/155713796 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f873000fba0 con 0x7f87381324a0 2024-01-24T23:24:14.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873efd1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:14.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873efd1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:14.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873efd1640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:14.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873efd1640 1 -- 172.21.15.132:0/155713796 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8718005090 con 0x7f87381324a0 2024-01-24T23:24:14.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f87357fa640 1 -- 172.21.15.132:0/155713796 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(81..81 src has 1..81) v4 ==== 5292+0+0 (unknown 3673231763 0 0) 0x7f8730043fb0 con 0x7f87381324a0 2024-01-24T23:24:14.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.434+0000 7f873cd6d640 10 monclient: get_auth_request con 0x7f8724036f60 auth_method 0 2024-01-24T23:24:14.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.438+0000 7f87357fa640 1 -- 172.21.15.132:0/155713796 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f8730011140 con 0x7f87381324a0 2024-01-24T23:24:14.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.438+0000 7f87357fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:14.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.438+0000 7f87357fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:14.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.598+0000 7f873efd1640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "root-for-test"}] 2024-01-24T23:24:14.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.598+0000 7f873efd1640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "root-for-test"}] 2024-01-24T23:24:14.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.598+0000 7f873efd1640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:14.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:14.598+0000 7f873efd1640 1 -- 172.21.15.132:0/155713796 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "root-for-test"} v 0) v1 -- 0x7f8718005340 con 0x7f87381324a0 2024-01-24T23:24:15.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.210+0000 7f87357fa640 1 -- 172.21.15.132:0/155713796 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "root-for-test"}]=0 removed item id -14 name 'root-for-test' from crush map v82) v1 ==== 140+0+0 (unknown 1482799743 0 0) 0x7f8730013160 con 0x7f87381324a0 2024-01-24T23:24:15.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.210+0000 7f87357fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "root-for-test"}] 2024-01-24T23:24:15.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.210+0000 7f87357fa640 10 monclient: _finish_command 2 = system:0 removed item id -14 name 'root-for-test' from crush map 2024-01-24T23:24:15.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.314+0000 7f873efd1640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "root-for-test"}] 2024-01-24T23:24:15.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.314+0000 7f873efd1640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "root-for-test"}] 2024-01-24T23:24:15.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.314+0000 7f873efd1640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:15.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.314+0000 7f873efd1640 1 -- 172.21.15.132:0/155713796 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "root-for-test"} v 0) v1 -- 0x7f8718002220 con 0x7f87381324a0 2024-01-24T23:24:15.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.318+0000 7f87357fa640 1 -- 172.21.15.132:0/155713796 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "root-for-test"}]=0 device 'root-for-test' does not appear in the crush map v82) v1 ==== 140+0+0 (unknown 2129850845 0 0) 0x7f87300148e0 con 0x7f87381324a0 2024-01-24T23:24:15.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.318+0000 7f87357fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "root-for-test"}] 2024-01-24T23:24:15.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.318+0000 7f87357fa640 10 monclient: _finish_command 3 = system:0 device 'root-for-test' does not appear in the crush map 2024-01-24T23:24:15.320 INFO:tasks.workunit.client.0.smithi132.stderr:device 'root-for-test' does not appear in the crush map 2024-01-24T23:24:15.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.318+0000 7f873efd1640 1 -- 172.21.15.132:0/155713796 >> v1:172.21.15.132:6812/1835401922 conn(0x7f8724036f60 legacy=0x7f8724039420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:15.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.318+0000 7f873efd1640 10 monclient: shutdown 2024-01-24T23:24:15.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.318+0000 7f873efd1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:15.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.318+0000 7f873efd1640 1 -- 172.21.15.132:0/155713796 >> v1:172.21.15.132:6791/0 conn(0x7f87381324a0 legacy=0x7f87381d39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:15.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.318+0000 7f873efd1640 1 -- 172.21.15.132:0/155713796 shutdown_connections 2024-01-24T23:24:15.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.318+0000 7f873efd1640 1 -- 172.21.15.132:0/155713796 >> 172.21.15.132:0/155713796 conn(0x7f87380a7150 msgr2=0x7f8738130fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:15.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.318+0000 7f873efd1640 1 -- 172.21.15.132:0/155713796 shutdown_connections 2024-01-24T23:24:15.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.318+0000 7f873efd1640 1 -- 172.21.15.132:0/155713796 wait complete. 2024-01-24T23:24:15.336 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule create-simple foo-rule foo host firstn 2024-01-24T23:24:15.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.454+0000 7f7ea360b640 10 monclient: get_monmap_and_config 2024-01-24T23:24:15.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.454+0000 7f7ea360b640 10 monclient: build_initial_monmap 2024-01-24T23:24:15.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.454+0000 7f7ea360b640 10 monclient: monmap: 2024-01-24T23:24:15.458 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:15.458 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:15.458 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:15.459369+0000 2024-01-24T23:24:15.458 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:15.459369+0000 2024-01-24T23:24:15.459 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:15.459 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:15.459 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:15.459 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:15.459 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:15.459 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:15.459 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:15.459 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:15.459 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:15.459 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:15.459 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:15.459 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:15.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.454+0000 7f7ea360b640 1 Processor -- start 2024-01-24T23:24:15.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.454+0000 7f7ea360b640 1 -- start start 2024-01-24T23:24:15.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.454+0000 7f7ea360b640 10 monclient: init 2024-01-24T23:24:15.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.454+0000 7f7ea360b640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:15.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.454+0000 7f7ea360b640 10 monclient: _add_conns ranks=[8,7,4,1,3,5,2,6,0] 2024-01-24T23:24:15.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 10 monclient(hunting): picked mon.noname-e con 0x7f7e9c09ff20 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:15.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 10 monclient(hunting): picked mon.noname-i con 0x7f7e9c09f9c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:15.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 10 monclient(hunting): picked mon.noname-c con 0x7f7e9c13b2c0 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:15.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9c064700 con 0x7f7e9c13b2c0 2024-01-24T23:24:15.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9c09e760 con 0x7f7e9c09f9c0 2024-01-24T23:24:15.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9c0a1160 con 0x7f7e9c09ff20 2024-01-24T23:24:15.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:15.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 10 monclient(hunting): authenticate will time out at 1142.784147s 2024-01-24T23:24:15.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea1ba8640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f7e9c13b2c0 legacy=0x7f7e9c13d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:15.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea0ba6640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f7e9c09f9c0 0x7f7e9c139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:44124/0 (socket says 172.21.15.132:44124) 2024-01-24T23:24:15.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea0ba6640 1 -- 172.21.15.132:0/3830170186 learned_addr learned my addr 172.21.15.132:0/3830170186 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:15.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 1 -- 172.21.15.132:0/3830170186 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 541910276 0 0) 0x7f7e9c09e760 con 0x7f7e9c09f9c0 2024-01-24T23:24:15.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:15.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:15.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:15.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 10 monclient(hunting): my global_id is 5542 2024-01-24T23:24:15.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 1 -- 172.21.15.132:0/3830170186 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e88003530 con 0x7f7e9c09f9c0 2024-01-24T23:24:15.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 1 -- 172.21.15.132:0/3830170186 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3733180108 0 0) 0x7f7e88003530 con 0x7f7e9c09f9c0 2024-01-24T23:24:15.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 1 -- 172.21.15.132:0/3830170186 >> v1:172.21.15.115:6793/0 conn(0x7f7e9c09ff20 legacy=0x7f7e9c0a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:15.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 1 -- 172.21.15.132:0/3830170186 >> v1:172.21.15.115:6791/0 conn(0x7f7e9c13b2c0 legacy=0x7f7e9c13d6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:15.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 10 monclient: _finish_hunting 0 2024-01-24T23:24:15.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 1 monclient: found mon.noname-i 2024-01-24T23:24:15.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:15.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:24:15.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 1 -- 172.21.15.132:0/3830170186 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9c0a0a10 con 0x7f7e9c09f9c0 2024-01-24T23:24:15.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 10 monclient: _finish_auth 0 2024-01-24T23:24:15.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 10 monclient: _check_auth_tickets 2024-01-24T23:24:15.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:15.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 5 monclient: authenticate success, global_id 5542 2024-01-24T23:24:15.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:15.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 1 -- 172.21.15.132:0/3830170186 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e8c0017c0 con 0x7f7e9c09f9c0 2024-01-24T23:24:15.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:15.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:15.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 10 monclient: dump: 2024-01-24T23:24:15.470 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:15.470 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:15.470 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:15.470 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:15.470 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:15.470 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:15.470 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:15.470 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:15.470 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:15.470 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:15.470 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:15.470 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:15.470 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:15.470 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:15.470 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:15.470 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:15.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:15.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 1 -- 172.21.15.132:0/3830170186 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7e8c001a60 con 0x7f7e9c09f9c0 2024-01-24T23:24:15.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:15.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:15.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 1 -- 172.21.15.132:0/3830170186 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e8c004bf0 con 0x7f7e9c09f9c0 2024-01-24T23:24:15.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 4 set_mon_vals no callback set 2024-01-24T23:24:15.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:15.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:15.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e93fff640 10 monclient: dump: 2024-01-24T23:24:15.471 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:15.471 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:15.471 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:15.471 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:15.471 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:15.471 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:15.471 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:15.472 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:15.472 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:15.472 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:15.472 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:15.472 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:15.472 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:15.472 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:15.472 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:15.472 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:15.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 10 monclient: shutdown 2024-01-24T23:24:15.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:15.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 1 -- 172.21.15.132:0/3830170186 >> v1:172.21.15.132:6792/0 conn(0x7f7e9c09f9c0 legacy=0x7f7e9c139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:15.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 1 -- 172.21.15.132:0/3830170186 shutdown_connections 2024-01-24T23:24:15.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 1 -- 172.21.15.132:0/3830170186 >> 172.21.15.132:0/3830170186 conn(0x7f7e9c0a5a10 msgr2=0x7f7e9c0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:15.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 1 -- 172.21.15.132:0/3830170186 shutdown_connections 2024-01-24T23:24:15.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 1 -- 172.21.15.132:0/3830170186 wait complete. 2024-01-24T23:24:15.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 10 monclient: build_initial_monmap 2024-01-24T23:24:15.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 10 monclient: monmap: 2024-01-24T23:24:15.473 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:15.473 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:15.473 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:15.473 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:15.473 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:15.473 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:15.473 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:15.473 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:15.473 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:15.473 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:15.473 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:15.473 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:15.473 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:15.474 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:15.474 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:15.474 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:15.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 1 Processor -- start 2024-01-24T23:24:15.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 1 -- start start 2024-01-24T23:24:15.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 10 monclient: init 2024-01-24T23:24:15.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:15.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 10 monclient: _add_conns ranks=[5,7,3,4,1,2,8,0,6] 2024-01-24T23:24:15.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 10 monclient(hunting): picked mon.noname-h con 0x7f7e9c09f9c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:15.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 10 monclient(hunting): picked mon.noname-i con 0x7f7e9c09ff20 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:15.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 10 monclient(hunting): picked mon.noname-g con 0x7f7e9c13b2c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:15.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9c071960 con 0x7f7e9c13b2c0 2024-01-24T23:24:15.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9c120840 con 0x7f7e9c09f9c0 2024-01-24T23:24:15.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9c120b20 con 0x7f7e9c09ff20 2024-01-24T23:24:15.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:15.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea360b640 10 monclient(hunting): authenticate will time out at 1142.787246s 2024-01-24T23:24:15.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea13a7640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f7e9c09f9c0 0x7f7e9c133000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:51146/0 (socket says 172.21.15.132:51146) 2024-01-24T23:24:15.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7ea13a7640 1 -- 172.21.15.132:0/2930574717 learned_addr learned my addr 172.21.15.132:0/2930574717 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:15.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2064657106 0 0) 0x7f7e9c120b20 con 0x7f7e9c09ff20 2024-01-24T23:24:15.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:15.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:15.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:15.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 10 monclient(hunting): my global_id is 5551 2024-01-24T23:24:15.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e80003530 con 0x7f7e9c09ff20 2024-01-24T23:24:15.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762421039 0 0) 0x7f7e9c120840 con 0x7f7e9c09f9c0 2024-01-24T23:24:15.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:15.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:15.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:15.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 10 monclient(hunting): my global_id is 5486 2024-01-24T23:24:15.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e9c120b20 con 0x7f7e9c09f9c0 2024-01-24T23:24:15.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687402955 0 0) 0x7f7e9c071960 con 0x7f7e9c13b2c0 2024-01-24T23:24:15.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:15.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:15.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:15.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 10 monclient(hunting): my global_id is 5484 2024-01-24T23:24:15.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e9c120840 con 0x7f7e9c13b2c0 2024-01-24T23:24:15.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2479379544 0 0) 0x7f7e80003530 con 0x7f7e9c09ff20 2024-01-24T23:24:15.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7e9c071960 con 0x7f7e9c09ff20 2024-01-24T23:24:15.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e8c0016c0 con 0x7f7e9c09ff20 2024-01-24T23:24:15.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:15.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:15.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 10 monclient(hunting): dump: 2024-01-24T23:24:15.476 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:15.476 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:15.476 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:15.476 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:15.477 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:15.477 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:15.477 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:15.477 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:15.477 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:15.477 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:15.477 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:15.477 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:15.477 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:15.477 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:15.477 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:15.477 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:15.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:15.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 118541331 0 0) 0x7f7e9c120840 con 0x7f7e9c13b2c0 2024-01-24T23:24:15.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7e80003530 con 0x7f7e9c13b2c0 2024-01-24T23:24:15.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.458+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 12626844 0 0) 0x7f7e9c120b20 con 0x7f7e9c09f9c0 2024-01-24T23:24:15.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7e9c120840 con 0x7f7e9c09f9c0 2024-01-24T23:24:15.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e980044b0 con 0x7f7e9c13b2c0 2024-01-24T23:24:15.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:15.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:15.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 10 monclient(hunting): dump: 2024-01-24T23:24:15.478 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:15.478 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:15.478 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:15.478 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:15.478 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:15.478 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:15.478 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:15.478 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:15.478 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:15.478 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:15.478 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:15.479 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:15.479 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:15.479 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:15.479 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:15.479 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:15.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e94001a80 con 0x7f7e9c09f9c0 2024-01-24T23:24:15.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:15.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:15.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 10 monclient(hunting): dump: 2024-01-24T23:24:15.479 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:15.479 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:15.479 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:15.479 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:15.479 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:15.479 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:15.479 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:15.480 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:15.480 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:15.480 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:15.480 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:15.480 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:15.480 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:15.480 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:15.480 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:15.480 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:15.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2081111749 0 0) 0x7f7e9c071960 con 0x7f7e9c09ff20 2024-01-24T23:24:15.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 >> v1:172.21.15.132:6791/0 conn(0x7f7e9c09f9c0 legacy=0x7f7e9c133000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:15.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 >> v1:172.21.15.132:6790/0 conn(0x7f7e9c13b2c0 legacy=0x7f7e9c1d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:15.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:24:15.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 1 monclient: found mon.i 2024-01-24T23:24:15.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:15.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:15.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9c13a360 con 0x7f7e9c09ff20 2024-01-24T23:24:15.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 10 monclient: _finish_auth 0 2024-01-24T23:24:15.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:24:15.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:15.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7ea360b640 5 monclient: authenticate success, global_id 5551 2024-01-24T23:24:15.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7ea360b640 10 monclient: _renew_subs 2024-01-24T23:24:15.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7ea360b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:15.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7ea360b640 1 -- 172.21.15.132:0/2930574717 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9c11fa20 con 0x7f7e9c09ff20 2024-01-24T23:24:15.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7ea360b640 10 monclient: _renew_subs 2024-01-24T23:24:15.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7ea360b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:15.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7ea360b640 1 -- 172.21.15.132:0/2930574717 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c09e070 con 0x7f7e9c09ff20 2024-01-24T23:24:15.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7e8c004ec0 con 0x7f7e9c09ff20 2024-01-24T23:24:15.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:15.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e8c005370 con 0x7f7e9c09ff20 2024-01-24T23:24:15.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:15.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:15.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e937fe640 4 set_mon_vals no callback set 2024-01-24T23:24:15.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 10 monclient: dump: 2024-01-24T23:24:15.482 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:15.482 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:15.482 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:15.482 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:15.482 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:15.482 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:15.482 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:15.482 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:15.482 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:15.482 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:15.482 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:15.482 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:15.483 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:15.483 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:15.483 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:15.483 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:15.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7ea360b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:15.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7ea360b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:15.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7ea360b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:15.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7ea360b640 1 -- 172.21.15.132:0/2930574717 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e7c005090 con 0x7f7e9c09ff20 2024-01-24T23:24:15.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f7e8c00fdf0 con 0x7f7e9c09ff20 2024-01-24T23:24:15.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(82..82 src has 1..82) v4 ==== 5133+0+0 (unknown 3620002556 0 0) 0x7f7e8c043f10 con 0x7f7e9c09ff20 2024-01-24T23:24:15.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.462+0000 7f7ea13a7640 10 monclient: get_auth_request con 0x7f7e80037b90 auth_method 0 2024-01-24T23:24:15.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.466+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f7e8c014b30 con 0x7f7e9c09ff20 2024-01-24T23:24:15.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.466+0000 7f7e91ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:15.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.466+0000 7f7e91ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:15.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.610+0000 7f7ea360b640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}] 2024-01-24T23:24:15.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.610+0000 7f7ea360b640 10 monclient: _send_command 2 [{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}] 2024-01-24T23:24:15.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.610+0000 7f7ea360b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:15.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:15.610+0000 7f7ea360b640 1 -- 172.21.15.132:0/2930574717 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"} v 0) v1 -- 0x7f7e7c005340 con 0x7f7e9c09ff20 2024-01-24T23:24:16.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.234+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}]=0 v83) v1 ==== 145+0+0 (unknown 1752617042 0 0) 0x7f7e8c0131d0 con 0x7f7e9c09ff20 2024-01-24T23:24:16.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.234+0000 7f7e91ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}] 2024-01-24T23:24:16.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.234+0000 7f7e91ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:24:16.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.326+0000 7f7ea360b640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}] 2024-01-24T23:24:16.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.326+0000 7f7ea360b640 10 monclient: _send_command 3 [{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}] 2024-01-24T23:24:16.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.326+0000 7f7ea360b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:16.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.326+0000 7f7ea360b640 1 -- 172.21.15.132:0/2930574717 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"} v 0) v1 -- 0x7f7e7c0010d0 con 0x7f7e9c09ff20 2024-01-24T23:24:16.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.330+0000 7f7e91ffb640 1 -- 172.21.15.132:0/2930574717 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}]=0 rule foo-rule already exists v83) v1 ==== 173+0+0 (unknown 2028726510 0 0) 0x7f7e8c014950 con 0x7f7e9c09ff20 2024-01-24T23:24:16.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.330+0000 7f7e91ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}] 2024-01-24T23:24:16.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.330+0000 7f7e91ffb640 10 monclient: _finish_command 3 = system:0 rule foo-rule already exists 2024-01-24T23:24:16.333 INFO:tasks.workunit.client.0.smithi132.stderr:rule foo-rule already exists 2024-01-24T23:24:16.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.330+0000 7f7ea360b640 1 -- 172.21.15.132:0/2930574717 >> v1:172.21.15.132:6812/1835401922 conn(0x7f7e80037b90 legacy=0x7f7e8003a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:16.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.330+0000 7f7ea360b640 10 monclient: shutdown 2024-01-24T23:24:16.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.330+0000 7f7ea360b640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:16.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.330+0000 7f7ea360b640 1 -- 172.21.15.132:0/2930574717 >> v1:172.21.15.132:6792/0 conn(0x7f7e9c09ff20 legacy=0x7f7e9c1d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:16.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.330+0000 7f7ea360b640 1 -- 172.21.15.132:0/2930574717 shutdown_connections 2024-01-24T23:24:16.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.330+0000 7f7ea360b640 1 -- 172.21.15.132:0/2930574717 >> 172.21.15.132:0/2930574717 conn(0x7f7e9c0a5a10 msgr2=0x7f7e9c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:16.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.330+0000 7f7ea360b640 1 -- 172.21.15.132:0/2930574717 shutdown_connections 2024-01-24T23:24:16.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.330+0000 7f7ea360b640 1 -- 172.21.15.132:0/2930574717 wait complete. 2024-01-24T23:24:16.349 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd crush rm foo 2024-01-24T23:24:16.349 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:24:16.349 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm foo 2024-01-24T23:24:16.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.454+0000 7f6c57247640 10 monclient: get_monmap_and_config 2024-01-24T23:24:16.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.454+0000 7f6c57247640 10 monclient: build_initial_monmap 2024-01-24T23:24:16.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.454+0000 7f6c57247640 10 monclient: monmap: 2024-01-24T23:24:16.458 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:16.458 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:16.459 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:16.459631+0000 2024-01-24T23:24:16.459 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:16.459631+0000 2024-01-24T23:24:16.459 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:16.459 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:16.459 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:16.459 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:16.459 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:16.459 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:16.459 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:16.459 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:16.459 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:16.459 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:16.460 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:16.460 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:16.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.454+0000 7f6c57247640 1 Processor -- start 2024-01-24T23:24:16.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.454+0000 7f6c57247640 1 -- start start 2024-01-24T23:24:16.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.454+0000 7f6c57247640 10 monclient: init 2024-01-24T23:24:16.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:16.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 10 monclient: _add_conns ranks=[6,5,4,7,3,1,2,8,0] 2024-01-24T23:24:16.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 10 monclient(hunting): picked mon.noname-d con 0x7f6c50135e10 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:16.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 10 monclient(hunting): picked mon.noname-h con 0x7f6c501361f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:16.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 10 monclient(hunting): picked mon.noname-c con 0x7f6c501365f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:16.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c50064700 con 0x7f6c501365f0 2024-01-24T23:24:16.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c5009c3e0 con 0x7f6c501361f0 2024-01-24T23:24:16.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c500a4f90 con 0x7f6c50135e10 2024-01-24T23:24:16.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:16.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 10 monclient(hunting): authenticate will time out at 1143.784268s 2024-01-24T23:24:16.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4ffff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f6c501361f0 0x7f6c5013bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:51150/0 (socket says 172.21.15.132:51150) 2024-01-24T23:24:16.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4ffff640 1 -- 172.21.15.132:0/1314251897 learned_addr learned my addr 172.21.15.132:0/1314251897 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:16.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c557e4640 1 -- 172.21.15.132:0/1314251897 >> v1:172.21.15.115:6791/0 conn(0x7f6c501365f0 legacy=0x7f6c5013f600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:16.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 1 -- 172.21.15.132:0/1314251897 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3992034779 0 0) 0x7f6c5009c3e0 con 0x7f6c501361f0 2024-01-24T23:24:16.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:24:16.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:16.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:16.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 10 monclient(hunting): my global_id is 5495 2024-01-24T23:24:16.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 1 -- 172.21.15.132:0/1314251897 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c38003530 con 0x7f6c501361f0 2024-01-24T23:24:16.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 1 -- 172.21.15.132:0/1314251897 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3097628438 0 0) 0x7f6c38003530 con 0x7f6c501361f0 2024-01-24T23:24:16.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 1 -- 172.21.15.132:0/1314251897 >> v1:172.21.15.115:6792/0 conn(0x7f6c50135e10 legacy=0x7f6c5009bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:16.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 1 -- 172.21.15.132:0/1314251897 >> v1:172.21.15.115:6791/0 conn(0x7f6c501365f0 legacy=0x7f6c5013f600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:16.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:24:16.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 1 monclient: found mon.noname-h 2024-01-24T23:24:16.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:16.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:24:16.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 1 -- 172.21.15.132:0/1314251897 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5009b010 con 0x7f6c501361f0 2024-01-24T23:24:16.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 10 monclient: _finish_auth 0 2024-01-24T23:24:16.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:24:16.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:16.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 1 -- 172.21.15.132:0/1314251897 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6c440017c0 con 0x7f6c501361f0 2024-01-24T23:24:16.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 5 monclient: authenticate success, global_id 5495 2024-01-24T23:24:16.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:16.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:16.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:16.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 10 monclient: dump: 2024-01-24T23:24:16.470 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:16.470 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:16.471 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:16.471 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:16.471 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:16.471 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:16.471 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:16.471 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:16.471 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:16.471 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:16.471 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:16.471 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:16.471 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:16.471 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:16.471 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:16.471 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:16.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:16.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 1 -- 172.21.15.132:0/1314251897 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6c44001ae0 con 0x7f6c501361f0 2024-01-24T23:24:16.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:16.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 1 -- 172.21.15.132:0/1314251897 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6c44004bf0 con 0x7f6c501361f0 2024-01-24T23:24:16.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:16.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:16.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4f7fe640 10 monclient: dump: 2024-01-24T23:24:16.472 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:16.472 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:16.472 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:16.472 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:16.472 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:16.472 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:16.472 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:16.473 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:16.473 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:16.473 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:16.473 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:16.473 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:16.473 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:16.473 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:16.473 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:16.473 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:16.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:16.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 4 set_mon_vals no callback set 2024-01-24T23:24:16.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 10 monclient: shutdown 2024-01-24T23:24:16.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:16.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 1 -- 172.21.15.132:0/1314251897 >> v1:172.21.15.132:6791/0 conn(0x7f6c501361f0 legacy=0x7f6c5013bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:16.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 1 -- 172.21.15.132:0/1314251897 shutdown_connections 2024-01-24T23:24:16.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 1 -- 172.21.15.132:0/1314251897 >> 172.21.15.132:0/1314251897 conn(0x7f6c500a6790 msgr2=0x7f6c50071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:16.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 1 -- 172.21.15.132:0/1314251897 shutdown_connections 2024-01-24T23:24:16.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 1 -- 172.21.15.132:0/1314251897 wait complete. 2024-01-24T23:24:16.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 10 monclient: build_initial_monmap 2024-01-24T23:24:16.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 10 monclient: monmap: 2024-01-24T23:24:16.474 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:16.474 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:16.474 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:16.474 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:16.474 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:16.474 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:16.474 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:16.475 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:16.475 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:16.475 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:16.475 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:16.475 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:16.475 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:16.475 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:16.475 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:16.475 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:16.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 1 Processor -- start 2024-01-24T23:24:16.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 1 -- start start 2024-01-24T23:24:16.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 10 monclient: init 2024-01-24T23:24:16.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:16.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 10 monclient: _add_conns ranks=[3,4,2,1,7,8,6,0,5] 2024-01-24T23:24:16.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 10 monclient(hunting): picked mon.noname-g con 0x7f6c50135e10 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:16.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 10 monclient(hunting): picked mon.noname-c con 0x7f6c501361f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:16.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 10 monclient(hunting): picked mon.noname-b con 0x7f6c501365f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:16.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c5009c6b0 con 0x7f6c501365f0 2024-01-24T23:24:16.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c501bf480 con 0x7f6c50135e10 2024-01-24T23:24:16.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c50122850 con 0x7f6c501361f0 2024-01-24T23:24:16.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:16.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c57247640 10 monclient(hunting): authenticate will time out at 1143.787798s 2024-01-24T23:24:16.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c4ffff640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f6c501361f0 legacy=0x7f6c501d03e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:16.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c54fe3640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f6c50135e10 0x7f6c50134e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:47572/0 (socket says 172.21.15.132:47572) 2024-01-24T23:24:16.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.458+0000 7f6c54fe3640 1 -- 172.21.15.132:0/3314590187 learned_addr learned my addr 172.21.15.132:0/3314590187 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:16.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 1 -- 172.21.15.132:0/3314590187 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3448458163 0 0) 0x7f6c501bf480 con 0x7f6c50135e10 2024-01-24T23:24:16.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:24:16.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:16.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:16.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 10 monclient(hunting): my global_id is 5493 2024-01-24T23:24:16.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 1 -- 172.21.15.132:0/3314590187 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c3c003530 con 0x7f6c50135e10 2024-01-24T23:24:16.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 1 -- 172.21.15.132:0/3314590187 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 593797891 0 0) 0x7f6c5009c6b0 con 0x7f6c501365f0 2024-01-24T23:24:16.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:24:16.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:16.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:16.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 10 monclient(hunting): my global_id is 84809 2024-01-24T23:24:16.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 1 -- 172.21.15.132:0/3314590187 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c501bf480 con 0x7f6c501365f0 2024-01-24T23:24:16.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 1 -- 172.21.15.132:0/3314590187 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3794610987 0 0) 0x7f6c3c003530 con 0x7f6c50135e10 2024-01-24T23:24:16.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 1 -- 172.21.15.132:0/3314590187 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6c5009c6b0 con 0x7f6c50135e10 2024-01-24T23:24:16.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 1 -- 172.21.15.132:0/3314590187 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6c40001ae0 con 0x7f6c50135e10 2024-01-24T23:24:16.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:16.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:16.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 10 monclient(hunting): dump: 2024-01-24T23:24:16.478 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:16.478 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:16.478 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:16.478 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:16.478 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:16.478 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:16.478 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:16.478 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:16.478 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:16.478 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:16.478 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:16.478 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:16.479 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:16.479 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:16.479 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:16.479 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:16.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:16.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 1 -- 172.21.15.132:0/3314590187 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 935552899 0 0) 0x7f6c5009c6b0 con 0x7f6c50135e10 2024-01-24T23:24:16.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 1 -- 172.21.15.132:0/3314590187 >> v1:172.21.15.115:6791/0 conn(0x7f6c501361f0 legacy=0x7f6c501d03e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:16.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 1 -- 172.21.15.132:0/3314590187 >> v1:172.21.15.115:6790/0 conn(0x7f6c501365f0 legacy=0x7f6c501d3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:16.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 10 monclient: _finish_hunting 0 2024-01-24T23:24:16.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 1 monclient: found mon.g 2024-01-24T23:24:16.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:16.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:16.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 1 -- 172.21.15.132:0/3314590187 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5009ca00 con 0x7f6c50135e10 2024-01-24T23:24:16.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 10 monclient: _finish_auth 0 2024-01-24T23:24:16.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:24:16.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:16.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c57247640 5 monclient: authenticate success, global_id 5493 2024-01-24T23:24:16.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c57247640 10 monclient: _renew_subs 2024-01-24T23:24:16.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c57247640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:16.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c57247640 1 -- 172.21.15.132:0/3314590187 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c50122570 con 0x7f6c50135e10 2024-01-24T23:24:16.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c57247640 10 monclient: _renew_subs 2024-01-24T23:24:16.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c57247640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:16.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c57247640 1 -- 172.21.15.132:0/3314590187 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c501beaa0 con 0x7f6c50135e10 2024-01-24T23:24:16.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 1 -- 172.21.15.132:0/3314590187 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6c400038d0 con 0x7f6c50135e10 2024-01-24T23:24:16.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:16.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 1 -- 172.21.15.132:0/3314590187 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6c40003d80 con 0x7f6c50135e10 2024-01-24T23:24:16.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:16.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4effd640 4 set_mon_vals no callback set 2024-01-24T23:24:16.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:16.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 10 monclient: dump: 2024-01-24T23:24:16.481 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:16.481 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:16.481 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:16.481 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:16.481 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:16.481 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:16.481 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:16.481 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:16.481 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:16.482 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:16.482 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:16.482 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:16.482 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:16.482 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:16.482 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:16.482 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:16.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c57247640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:16.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c57247640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:16.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c57247640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:16.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c57247640 1 -- 172.21.15.132:0/3314590187 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c30005090 con 0x7f6c50135e10 2024-01-24T23:24:16.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 1 -- 172.21.15.132:0/3314590187 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f6c40003ee0 con 0x7f6c50135e10 2024-01-24T23:24:16.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4d7fa640 1 -- 172.21.15.132:0/3314590187 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(83..83 src has 1..83) v4 ==== 5193+0+0 (unknown 3960739443 0 0) 0x7f6c40043d30 con 0x7f6c50135e10 2024-01-24T23:24:16.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.462+0000 7f6c4ffff640 10 monclient: get_auth_request con 0x7f6c3c037040 auth_method 0 2024-01-24T23:24:16.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.466+0000 7f6c4d7fa640 1 -- 172.21.15.132:0/3314590187 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f6c40014800 con 0x7f6c50135e10 2024-01-24T23:24:16.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.466+0000 7f6c4d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:16.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.466+0000 7f6c4d7fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:16.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.626+0000 7f6c57247640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "foo"}] 2024-01-24T23:24:16.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.626+0000 7f6c57247640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-01-24T23:24:16.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.626+0000 7f6c57247640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:16.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.626+0000 7f6c57247640 1 -- 172.21.15.132:0/3314590187 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rm", "name": "foo"} v 0) v1 -- 0x7f6c30005340 con 0x7f6c50135e10 2024-01-24T23:24:16.629 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.626+0000 7f6c4d7fa640 1 -- 172.21.15.132:0/3314590187 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "foo"}]=-16 (16) Device or resource busy v83) v1 ==== 103+0+0 (unknown 3711162001 0 0) 0x7f6c40012ea0 con 0x7f6c50135e10 2024-01-24T23:24:16.629 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.626+0000 7f6c4d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-01-24T23:24:16.629 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.626+0000 7f6c4d7fa640 10 monclient: _finish_command 2 = mon:16 (16) Device or resource busy 2024-01-24T23:24:16.629 INFO:tasks.workunit.client.0.smithi132.stderr:Error EBUSY: (16) Device or resource busy 2024-01-24T23:24:16.635 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:24:16.635 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rule rm foo-rule 2024-01-24T23:24:16.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.750+0000 7f7b0cd31640 10 monclient: get_monmap_and_config 2024-01-24T23:24:16.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.750+0000 7f7b0cd31640 10 monclient: build_initial_monmap 2024-01-24T23:24:16.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.750+0000 7f7b0cd31640 10 monclient: monmap: 2024-01-24T23:24:16.754 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:16.755 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:16.755 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:16.755568+0000 2024-01-24T23:24:16.755 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:16.755568+0000 2024-01-24T23:24:16.755 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:16.755 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:16.755 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:16.755 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:16.755 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:16.755 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:16.755 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:16.755 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:16.755 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:16.755 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:16.755 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:16.756 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:16.759 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.750+0000 7f7b0cd31640 1 Processor -- start 2024-01-24T23:24:16.759 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 1 -- start start 2024-01-24T23:24:16.759 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 10 monclient: init 2024-01-24T23:24:16.759 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:16.759 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 10 monclient: _add_conns ranks=[6,1,8,0,5,3,4,7,2] 2024-01-24T23:24:16.759 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 10 monclient(hunting): picked mon.noname-d con 0x7f7b0809c4c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:16.759 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 10 monclient(hunting): picked mon.noname-f con 0x7f7b0809cfb0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:16.759 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 10 monclient(hunting): picked mon.noname-e con 0x7f7b0809ec50 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:16.759 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b08064700 con 0x7f7b0809cfb0 2024-01-24T23:24:16.759 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0809d9f0 con 0x7f7b0809c4c0 2024-01-24T23:24:16.759 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0809dc60 con 0x7f7b0809ec50 2024-01-24T23:24:16.759 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:16.759 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 10 monclient(hunting): authenticate will time out at 1144.080756s 2024-01-24T23:24:16.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b05d9b640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f7b0809cfb0 legacy=0x7f7b08138d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:16.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b06d9d640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f7b0809ec50 0x7f7b0813c460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:60956/0 (socket says 172.21.15.132:60956) 2024-01-24T23:24:16.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b06d9d640 1 -- 172.21.15.132:0/2363107833 learned_addr learned my addr 172.21.15.132:0/2363107833 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:16.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 1 -- 172.21.15.132:0/2363107833 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2825949327 0 0) 0x7f7b0809dc60 con 0x7f7b0809ec50 2024-01-24T23:24:16.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 10 monclient(hunting): _negotiate 2024-01-24T23:24:16.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:16.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:16.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 10 monclient(hunting): my global_id is 84698 2024-01-24T23:24:16.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 1 -- 172.21.15.132:0/2363107833 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7af0003530 con 0x7f7b0809ec50 2024-01-24T23:24:16.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 1 -- 172.21.15.132:0/2363107833 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1559965145 0 0) 0x7f7b0809d9f0 con 0x7f7b0809c4c0 2024-01-24T23:24:16.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 10 monclient(hunting): _negotiate 2024-01-24T23:24:16.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:16.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:16.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 10 monclient(hunting): my global_id is 5541 2024-01-24T23:24:16.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 1 -- 172.21.15.132:0/2363107833 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b0809dc60 con 0x7f7b0809c4c0 2024-01-24T23:24:16.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 1 -- 172.21.15.132:0/2363107833 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3456204462 0 0) 0x7f7af0003530 con 0x7f7b0809ec50 2024-01-24T23:24:16.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 1 -- 172.21.15.132:0/2363107833 >> v1:172.21.15.115:6792/0 conn(0x7f7b0809c4c0 legacy=0x7f7b0809c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:16.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 1 -- 172.21.15.132:0/2363107833 >> v1:172.21.15.132:6789/0 conn(0x7f7b0809cfb0 legacy=0x7f7b08138d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:16.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 10 monclient: _finish_hunting 0 2024-01-24T23:24:16.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 1 monclient: found mon.noname-e 2024-01-24T23:24:16.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:16.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:24:16.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 1 -- 172.21.15.132:0/2363107833 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0809df40 con 0x7f7b0809ec50 2024-01-24T23:24:16.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 10 monclient: _finish_auth 0 2024-01-24T23:24:16.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 10 monclient: _check_auth_tickets 2024-01-24T23:24:16.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:16.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 1 -- 172.21.15.132:0/2363107833 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7b00001b40 con 0x7f7b0809ec50 2024-01-24T23:24:16.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:16.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:24:16.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 10 monclient: dump: 2024-01-24T23:24:16.762 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:16.762 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:16.762 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:16.762 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:16.762 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:16.762 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:16.762 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:16.762 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:16.762 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:16.762 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:16.762 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:16.762 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:16.762 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:16.763 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:16.763 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:16.763 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:16.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 5 monclient: authenticate success, global_id 84698 2024-01-24T23:24:16.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:16.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 1 -- 172.21.15.132:0/2363107833 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7b00001e20 con 0x7f7b0809ec50 2024-01-24T23:24:16.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:16.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 1 -- 172.21.15.132:0/2363107833 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7b00002050 con 0x7f7b0809ec50 2024-01-24T23:24:16.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:16.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:24:16.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0559a640 10 monclient: dump: 2024-01-24T23:24:16.763 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:16.763 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:16.763 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:16.763 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:16.764 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:16.764 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:16.764 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:16.764 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:16.764 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:16.764 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:16.764 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:16.764 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:16.764 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:16.764 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:16.764 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:16.764 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:16.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:16.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 4 set_mon_vals no callback set 2024-01-24T23:24:16.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 10 monclient: shutdown 2024-01-24T23:24:16.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:16.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 1 -- 172.21.15.132:0/2363107833 >> v1:172.21.15.115:6793/0 conn(0x7f7b0809ec50 legacy=0x7f7b0813c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:16.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 1 -- 172.21.15.132:0/2363107833 shutdown_connections 2024-01-24T23:24:16.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 1 -- 172.21.15.132:0/2363107833 >> 172.21.15.132:0/2363107833 conn(0x7f7b080a5a10 msgr2=0x7f7b080a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:16.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 1 -- 172.21.15.132:0/2363107833 shutdown_connections 2024-01-24T23:24:16.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 1 -- 172.21.15.132:0/2363107833 wait complete. 2024-01-24T23:24:16.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 10 monclient: build_initial_monmap 2024-01-24T23:24:16.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.754+0000 7f7b0cd31640 10 monclient: monmap: 2024-01-24T23:24:16.765 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:16.765 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:16.765 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:16.765 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:16.766 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:16.766 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:16.766 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:16.766 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:16.766 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:16.766 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:16.766 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:16.766 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:16.766 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:16.766 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:16.766 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:16.766 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:16.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0cd31640 1 Processor -- start 2024-01-24T23:24:16.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0cd31640 1 -- start start 2024-01-24T23:24:16.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0cd31640 10 monclient: init 2024-01-24T23:24:16.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0cd31640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:16.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0cd31640 10 monclient: _add_conns ranks=[3,7,0,1,5,8,4,2,6] 2024-01-24T23:24:16.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0cd31640 10 monclient(hunting): picked mon.noname-g con 0x7f7b0809c4c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:16.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0cd31640 10 monclient(hunting): picked mon.noname-i con 0x7f7b0809cfb0 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:16.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0cd31640 10 monclient(hunting): picked mon.noname-a con 0x7f7b0809ec50 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:16.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0cd31640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0809c020 con 0x7f7b0809ec50 2024-01-24T23:24:16.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0cd31640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b080a13b0 con 0x7f7b0809c4c0 2024-01-24T23:24:16.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0cd31640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b08134b00 con 0x7f7b0809cfb0 2024-01-24T23:24:16.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0cd31640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:16.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0cd31640 10 monclient(hunting): authenticate will time out at 1144.084679s 2024-01-24T23:24:16.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b06d9d640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f7b0809ec50 legacy=0x7f7b081cf670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:16.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0659c640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f7b0809c4c0 0x7f7b0812d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:47588/0 (socket says 172.21.15.132:47588) 2024-01-24T23:24:16.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0659c640 1 -- 172.21.15.132:0/4250983733 learned_addr learned my addr 172.21.15.132:0/4250983733 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:16.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b05d9b640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f7b0809cfb0 0x7f7b081cbf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:44138/0 (socket says 172.21.15.132:44138) 2024-01-24T23:24:16.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 1 -- 172.21.15.132:0/4250983733 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1320483366 0 0) 0x7f7b080a13b0 con 0x7f7b0809c4c0 2024-01-24T23:24:16.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:16.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:16.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:16.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 10 monclient(hunting): my global_id is 5502 2024-01-24T23:24:16.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 1 -- 172.21.15.132:0/4250983733 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ae8003530 con 0x7f7b0809c4c0 2024-01-24T23:24:16.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 1 -- 172.21.15.132:0/4250983733 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3093525787 0 0) 0x7f7b08134b00 con 0x7f7b0809cfb0 2024-01-24T23:24:16.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:16.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:16.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:16.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 10 monclient(hunting): my global_id is 5560 2024-01-24T23:24:16.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 1 -- 172.21.15.132:0/4250983733 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b080a13b0 con 0x7f7b0809cfb0 2024-01-24T23:24:16.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 1 -- 172.21.15.132:0/4250983733 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3118985081 0 0) 0x7f7ae8003530 con 0x7f7b0809c4c0 2024-01-24T23:24:16.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 1 -- 172.21.15.132:0/4250983733 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7b08134b00 con 0x7f7b0809c4c0 2024-01-24T23:24:16.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 1 -- 172.21.15.132:0/4250983733 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3000721444 0 0) 0x7f7b080a13b0 con 0x7f7b0809cfb0 2024-01-24T23:24:16.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 1 -- 172.21.15.132:0/4250983733 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ae8003530 con 0x7f7b0809cfb0 2024-01-24T23:24:16.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 1 -- 172.21.15.132:0/4250983733 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7af8001560 con 0x7f7b0809c4c0 2024-01-24T23:24:16.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:16.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:16.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 10 monclient(hunting): dump: 2024-01-24T23:24:16.773 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:16.773 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:16.774 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:16.774 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:16.774 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:16.774 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:16.774 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:16.774 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:16.774 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:16.774 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:16.774 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:16.774 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:16.774 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:16.774 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:16.774 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:16.774 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:16.775 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:16.775 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 1 -- 172.21.15.132:0/4250983733 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7afc004120 con 0x7f7b0809cfb0 2024-01-24T23:24:16.775 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:16.775 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:16.775 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 10 monclient(hunting): dump: 2024-01-24T23:24:16.775 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:16.775 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:16.775 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:16.775 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:16.775 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:16.775 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:16.775 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:16.775 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:16.775 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:16.776 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:16.776 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:16.776 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:16.776 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:16.776 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:16.776 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:16.776 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:16.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 1 -- 172.21.15.132:0/4250983733 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4283480758 0 0) 0x7f7b08134b00 con 0x7f7b0809c4c0 2024-01-24T23:24:16.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 1 -- 172.21.15.132:0/4250983733 >> v1:172.21.15.132:6792/0 conn(0x7f7b0809cfb0 legacy=0x7f7b081cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:16.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 1 -- 172.21.15.132:0/4250983733 >> v1:172.21.15.115:6789/0 conn(0x7f7b0809ec50 legacy=0x7f7b081cf670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:16.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:16.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 1 monclient: found mon.g 2024-01-24T23:24:16.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:16.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:16.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 1 -- 172.21.15.132:0/4250983733 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0811cb90 con 0x7f7b0809c4c0 2024-01-24T23:24:16.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 10 monclient: _finish_auth 0 2024-01-24T23:24:16.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:16.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:16.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0cd31640 5 monclient: authenticate success, global_id 5502 2024-01-24T23:24:16.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0cd31640 10 monclient: _renew_subs 2024-01-24T23:24:16.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0cd31640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:16.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0cd31640 1 -- 172.21.15.132:0/4250983733 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0811b6c0 con 0x7f7b0809c4c0 2024-01-24T23:24:16.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0cd31640 10 monclient: _renew_subs 2024-01-24T23:24:16.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0cd31640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:16.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0cd31640 1 -- 172.21.15.132:0/4250983733 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0811c590 con 0x7f7b0809c4c0 2024-01-24T23:24:16.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 1 -- 172.21.15.132:0/4250983733 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7af80038b0 con 0x7f7b0809c4c0 2024-01-24T23:24:16.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:16.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 1 -- 172.21.15.132:0/4250983733 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7af8001750 con 0x7f7b0809c4c0 2024-01-24T23:24:16.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:16.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:16.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b04d99640 4 set_mon_vals no callback set 2024-01-24T23:24:16.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 10 monclient: dump: 2024-01-24T23:24:16.778 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:16.778 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:16.778 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:16.778 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:16.778 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:16.778 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:16.778 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:16.778 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:16.778 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:16.779 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:16.779 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:16.779 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:16.779 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:16.779 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:16.779 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:16.779 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:16.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0cd31640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:16.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0cd31640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:16.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0cd31640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:16.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b0cd31640 1 -- 172.21.15.132:0/4250983733 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aec005090 con 0x7f7b0809c4c0 2024-01-24T23:24:16.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 1 -- 172.21.15.132:0/4250983733 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f7af8003de0 con 0x7f7b0809c4c0 2024-01-24T23:24:16.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7af6ffd640 1 -- 172.21.15.132:0/4250983733 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(83..83 src has 1..83) v4 ==== 5193+0+0 (unknown 3960739443 0 0) 0x7f7af8043c80 con 0x7f7b0809c4c0 2024-01-24T23:24:16.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.758+0000 7f7b05d9b640 10 monclient: get_auth_request con 0x7f7ae8037870 auth_method 0 2024-01-24T23:24:16.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.762+0000 7f7af6ffd640 1 -- 172.21.15.132:0/4250983733 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f7af80113a0 con 0x7f7b0809c4c0 2024-01-24T23:24:16.780 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.762+0000 7f7af6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:16.780 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.762+0000 7f7af6ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:16.924 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.922+0000 7f7b0cd31640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "foo-rule"}] 2024-01-24T23:24:16.924 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.922+0000 7f7b0cd31640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rm", "name": "foo-rule"}] 2024-01-24T23:24:16.924 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.922+0000 7f7b0cd31640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:16.924 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:16.922+0000 7f7b0cd31640 1 -- 172.21.15.132:0/4250983733 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rule rm", "name": "foo-rule"} v 0) v1 -- 0x7f7aec005340 con 0x7f7b0809c4c0 2024-01-24T23:24:17.883 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:17.878+0000 7f7af6ffd640 1 -- 172.21.15.132:0/4250983733 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo-rule"}]=0 v84) v1 ==== 85+0+0 (unknown 1659475478 0 0) 0x7f7af8012e90 con 0x7f7b0809c4c0 2024-01-24T23:24:17.883 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:17.878+0000 7f7af6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rm", "name": "foo-rule"}] 2024-01-24T23:24:17.884 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:17.878+0000 7f7af6ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:24:17.982 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:17.978+0000 7f7b0cd31640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "foo-rule"}] 2024-01-24T23:24:17.982 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:17.978+0000 7f7b0cd31640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rm", "name": "foo-rule"}] 2024-01-24T23:24:17.982 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:17.978+0000 7f7b0cd31640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:17.982 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:17.978+0000 7f7b0cd31640 1 -- 172.21.15.132:0/4250983733 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rule rm", "name": "foo-rule"} v 0) v1 -- 0x7f7aec002220 con 0x7f7b0809c4c0 2024-01-24T23:24:17.982 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:17.978+0000 7f7af6ffd640 1 -- 172.21.15.132:0/4250983733 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo-rule"}]=0 rule foo-rule does not exist v84) v1 ==== 113+0+0 (unknown 2154995714 0 0) 0x7f7af8014610 con 0x7f7b0809c4c0 2024-01-24T23:24:17.982 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:17.978+0000 7f7af6ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rm", "name": "foo-rule"}] 2024-01-24T23:24:17.982 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:17.978+0000 7f7af6ffd640 10 monclient: _finish_command 3 = system:0 rule foo-rule does not exist 2024-01-24T23:24:17.982 INFO:tasks.workunit.client.0.smithi132.stderr:rule foo-rule does not exist 2024-01-24T23:24:17.984 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:17.982+0000 7f7b0cd31640 1 -- 172.21.15.132:0/4250983733 >> v1:172.21.15.132:6812/1835401922 conn(0x7f7ae8037870 legacy=0x7f7ae8039d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:17.984 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:17.982+0000 7f7b0cd31640 10 monclient: shutdown 2024-01-24T23:24:17.984 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:17.982+0000 7f7b0cd31640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:17.984 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:17.982+0000 7f7b0cd31640 1 -- 172.21.15.132:0/4250983733 >> v1:172.21.15.132:6790/0 conn(0x7f7b0809c4c0 legacy=0x7f7b0812d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:17.985 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:17.982+0000 7f7b0cd31640 1 -- 172.21.15.132:0/4250983733 shutdown_connections 2024-01-24T23:24:17.985 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:17.982+0000 7f7b0cd31640 1 -- 172.21.15.132:0/4250983733 >> 172.21.15.132:0/4250983733 conn(0x7f7b080a5a10 msgr2=0x7f7b08066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:17.985 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:17.982+0000 7f7b0cd31640 1 -- 172.21.15.132:0/4250983733 shutdown_connections 2024-01-24T23:24:17.985 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:17.982+0000 7f7b0cd31640 1 -- 172.21.15.132:0/4250983733 wait complete. 2024-01-24T23:24:17.999 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm bar 2024-01-24T23:24:18.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce97e8640 10 monclient: get_monmap_and_config 2024-01-24T23:24:18.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce97e8640 10 monclient: build_initial_monmap 2024-01-24T23:24:18.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce97e8640 10 monclient: monmap: 2024-01-24T23:24:18.117 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:18.117 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:18.117 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:18.117549+0000 2024-01-24T23:24:18.117 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:18.117549+0000 2024-01-24T23:24:18.117 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:18.117 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:18.117 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:18.117 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:18.117 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:18.117 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:18.117 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:18.117 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:18.117 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:18.117 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:18.117 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:18.118 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:18.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce97e8640 1 Processor -- start 2024-01-24T23:24:18.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce97e8640 1 -- start start 2024-01-24T23:24:18.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce97e8640 10 monclient: init 2024-01-24T23:24:18.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce97e8640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:18.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce97e8640 10 monclient: _add_conns ranks=[0,2,6,4,3,7,8,5,1] 2024-01-24T23:24:18.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce97e8640 10 monclient(hunting): picked mon.noname-a con 0x7f2ce4136e50 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:18.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce97e8640 10 monclient(hunting): picked mon.noname-b con 0x7f2ce40a0290 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:18.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce97e8640 10 monclient(hunting): picked mon.noname-d con 0x7f2ce40a1600 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:18.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce97e8640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce4064700 con 0x7f2ce4136e50 2024-01-24T23:24:18.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce97e8640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce4137940 con 0x7f2ce40a0290 2024-01-24T23:24:18.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce97e8640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce409f9c0 con 0x7f2ce40a1600 2024-01-24T23:24:18.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce97e8640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:18.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce97e8640 10 monclient(hunting): authenticate will time out at 1145.442187s 2024-01-24T23:24:18.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce2ffd640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f2ce4136e50 legacy=0x7f2ce4137230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:18.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce27fc640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f2ce40a0290 0x7f2ce413bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:35762/0 (socket says 172.21.15.132:35762) 2024-01-24T23:24:18.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce27fc640 1 -- 172.21.15.132:0/2821030343 learned_addr learned my addr 172.21.15.132:0/2821030343 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:18.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 1 -- 172.21.15.132:0/2821030343 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 160545350 0 0) 0x7f2ce4137940 con 0x7f2ce40a0290 2024-01-24T23:24:18.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:18.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:18.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:18.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 10 monclient(hunting): my global_id is 84818 2024-01-24T23:24:18.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 1 -- 172.21.15.132:0/2821030343 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cd0003530 con 0x7f2ce40a0290 2024-01-24T23:24:18.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 1 -- 172.21.15.132:0/2821030343 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2904716643 0 0) 0x7f2ce409f9c0 con 0x7f2ce40a1600 2024-01-24T23:24:18.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:18.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:18.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:18.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 10 monclient(hunting): my global_id is 5550 2024-01-24T23:24:18.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 1 -- 172.21.15.132:0/2821030343 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ce4137940 con 0x7f2ce40a1600 2024-01-24T23:24:18.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 1 -- 172.21.15.132:0/2821030343 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2225569084 0 0) 0x7f2ce4137940 con 0x7f2ce40a1600 2024-01-24T23:24:18.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 1 -- 172.21.15.132:0/2821030343 >> v1:172.21.15.115:6790/0 conn(0x7f2ce40a0290 legacy=0x7f2ce413bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:18.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 1 -- 172.21.15.132:0/2821030343 >> v1:172.21.15.115:6789/0 conn(0x7f2ce4136e50 legacy=0x7f2ce4137230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:18.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:24:18.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 1 monclient: found mon.noname-d 2024-01-24T23:24:18.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:18.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:24:18.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 1 -- 172.21.15.132:0/2821030343 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce409eda0 con 0x7f2ce40a1600 2024-01-24T23:24:18.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 10 monclient: _finish_auth 0 2024-01-24T23:24:18.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:24:18.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:18.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 1 -- 172.21.15.132:0/2821030343 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2cd4001b20 con 0x7f2ce40a1600 2024-01-24T23:24:18.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce97e8640 5 monclient: authenticate success, global_id 5550 2024-01-24T23:24:18.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce97e8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:18.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:18.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:24:18.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 10 monclient: dump: 2024-01-24T23:24:18.128 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:18.128 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:18.128 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:18.128 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:18.128 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:18.128 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:18.128 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:18.128 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:18.128 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:18.128 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:18.129 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:18.129 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:18.129 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:18.129 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:18.129 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:18.129 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:18.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce97e8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:18.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 1 -- 172.21.15.132:0/2821030343 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2cd4001d40 con 0x7f2ce40a1600 2024-01-24T23:24:18.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:18.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 1 -- 172.21.15.132:0/2821030343 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2cd4003ce0 con 0x7f2ce40a1600 2024-01-24T23:24:18.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce97e8640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:18.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce97e8640 4 set_mon_vals no callback set 2024-01-24T23:24:18.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:18.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:24:18.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.114+0000 7f2ce1ffb640 10 monclient: dump: 2024-01-24T23:24:18.130 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:18.130 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:18.130 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:18.130 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:18.130 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:18.130 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:18.130 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:18.130 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:18.130 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:18.130 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:18.130 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:18.130 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:18.130 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:18.130 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:18.130 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:18.131 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:18.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 10 monclient: shutdown 2024-01-24T23:24:18.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:18.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 1 -- 172.21.15.132:0/2821030343 >> v1:172.21.15.115:6792/0 conn(0x7f2ce40a1600 legacy=0x7f2ce413f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:18.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 1 -- 172.21.15.132:0/2821030343 shutdown_connections 2024-01-24T23:24:18.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 1 -- 172.21.15.132:0/2821030343 >> 172.21.15.132:0/2821030343 conn(0x7f2ce40a5a10 msgr2=0x7f2ce40a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:18.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 1 -- 172.21.15.132:0/2821030343 shutdown_connections 2024-01-24T23:24:18.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 1 -- 172.21.15.132:0/2821030343 wait complete. 2024-01-24T23:24:18.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 10 monclient: build_initial_monmap 2024-01-24T23:24:18.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 10 monclient: monmap: 2024-01-24T23:24:18.131 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:18.131 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:18.131 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:18.131 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:18.131 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:18.132 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:18.132 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:18.132 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:18.132 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:18.132 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:18.132 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:18.132 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:18.132 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:18.132 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:18.132 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:18.132 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:18.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 1 Processor -- start 2024-01-24T23:24:18.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 1 -- start start 2024-01-24T23:24:18.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 10 monclient: init 2024-01-24T23:24:18.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:18.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 10 monclient: _add_conns ranks=[3,8,1,4,0,2,6,5,7] 2024-01-24T23:24:18.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 10 monclient(hunting): picked mon.noname-g con 0x7f2ce40a0290 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:18.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 10 monclient(hunting): picked mon.noname-e con 0x7f2ce40a1600 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:18.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 10 monclient(hunting): picked mon.noname-f con 0x7f2ce4136e50 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:18.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce41cb8c0 con 0x7f2ce4136e50 2024-01-24T23:24:18.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce41200f0 con 0x7f2ce40a0290 2024-01-24T23:24:18.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce409ea70 con 0x7f2ce40a1600 2024-01-24T23:24:18.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:18.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 10 monclient(hunting): authenticate will time out at 1145.446222s 2024-01-24T23:24:18.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce37fe640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f2ce4136e50 legacy=0x7f2ce41d32a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:18.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce2ffd640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f2ce40a0290 0x7f2ce41cb1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:47598/0 (socket says 172.21.15.132:47598) 2024-01-24T23:24:18.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce2ffd640 1 -- 172.21.15.132:0/4150702687 learned_addr learned my addr 172.21.15.132:0/4150702687 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:18.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 1 -- 172.21.15.132:0/4150702687 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2877597025 0 0) 0x7f2ce41200f0 con 0x7f2ce40a0290 2024-01-24T23:24:18.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:24:18.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:18.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:18.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 10 monclient(hunting): my global_id is 5511 2024-01-24T23:24:18.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 1 -- 172.21.15.132:0/4150702687 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cc4003530 con 0x7f2ce40a0290 2024-01-24T23:24:18.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 1 -- 172.21.15.132:0/4150702687 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1717659879 0 0) 0x7f2cc4003530 con 0x7f2ce40a0290 2024-01-24T23:24:18.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 1 -- 172.21.15.132:0/4150702687 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2ce41200f0 con 0x7f2ce40a0290 2024-01-24T23:24:18.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 1 -- 172.21.15.132:0/4150702687 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1030359611 0 0) 0x7f2ce409ea70 con 0x7f2ce40a1600 2024-01-24T23:24:18.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:24:18.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:18.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:18.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 10 monclient(hunting): my global_id is 84707 2024-01-24T23:24:18.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 1 -- 172.21.15.132:0/4150702687 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cc4003530 con 0x7f2ce40a1600 2024-01-24T23:24:18.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 1 -- 172.21.15.132:0/4150702687 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2cdc004330 con 0x7f2ce40a0290 2024-01-24T23:24:18.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:18.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:18.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 10 monclient(hunting): dump: 2024-01-24T23:24:18.135 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:18.135 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:18.135 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:18.135 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:18.135 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:18.135 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:18.135 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:18.135 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:18.135 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:18.135 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:18.135 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:18.135 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:18.135 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:18.136 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:18.136 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:18.136 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:18.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:18.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 1 -- 172.21.15.132:0/4150702687 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3200372743 0 0) 0x7f2ce41200f0 con 0x7f2ce40a0290 2024-01-24T23:24:18.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 1 -- 172.21.15.132:0/4150702687 >> v1:172.21.15.115:6793/0 conn(0x7f2ce40a1600 legacy=0x7f2ce4134c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:18.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 1 -- 172.21.15.132:0/4150702687 >> v1:172.21.15.132:6789/0 conn(0x7f2ce4136e50 legacy=0x7f2ce41d32a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:18.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:24:18.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 1 monclient: found mon.g 2024-01-24T23:24:18.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:18.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:18.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 1 -- 172.21.15.132:0/4150702687 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce4136c20 con 0x7f2ce40a0290 2024-01-24T23:24:18.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 10 monclient: _finish_auth 0 2024-01-24T23:24:18.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:24:18.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:18.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 5 monclient: authenticate success, global_id 5511 2024-01-24T23:24:18.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 10 monclient: _renew_subs 2024-01-24T23:24:18.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:18.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 1 -- 172.21.15.132:0/4150702687 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce41223e0 con 0x7f2ce40a0290 2024-01-24T23:24:18.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 10 monclient: _renew_subs 2024-01-24T23:24:18.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:18.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce97e8640 1 -- 172.21.15.132:0/4150702687 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce4135530 con 0x7f2ce40a0290 2024-01-24T23:24:18.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 1 -- 172.21.15.132:0/4150702687 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2cdc0045d0 con 0x7f2ce40a0290 2024-01-24T23:24:18.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:18.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 1 -- 172.21.15.132:0/4150702687 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2cdc004b00 con 0x7f2ce40a0290 2024-01-24T23:24:18.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:18.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2ce17fa640 4 set_mon_vals no callback set 2024-01-24T23:24:18.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:18.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.118+0000 7f2cdb7fe640 10 monclient: dump: 2024-01-24T23:24:18.138 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:18.138 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:18.138 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:18.138 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:18.138 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:18.138 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:18.138 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:18.138 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:18.138 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:18.138 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:18.138 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:18.138 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:18.138 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:18.138 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:18.139 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:18.139 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:18.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.122+0000 7f2ce97e8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:18.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.122+0000 7f2cdb7fe640 1 -- 172.21.15.132:0/4150702687 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f2cdc004ce0 con 0x7f2ce40a0290 2024-01-24T23:24:18.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.122+0000 7f2ce97e8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:18.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.122+0000 7f2ce97e8640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:18.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.122+0000 7f2ce97e8640 1 -- 172.21.15.132:0/4150702687 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc8005090 con 0x7f2ce40a0290 2024-01-24T23:24:18.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.122+0000 7f2cdb7fe640 1 -- 172.21.15.132:0/4150702687 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(84..84 src has 1..84) v4 ==== 5133+0+0 (unknown 2897313501 0 0) 0x7f2cdc043df0 con 0x7f2ce40a0290 2024-01-24T23:24:18.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.122+0000 7f2ce27fc640 10 monclient: get_auth_request con 0x7f2cc4037050 auth_method 0 2024-01-24T23:24:18.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.126+0000 7f2cdb7fe640 1 -- 172.21.15.132:0/4150702687 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f2cdc014b10 con 0x7f2ce40a0290 2024-01-24T23:24:18.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.126+0000 7f2cdb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:18.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.126+0000 7f2cdb7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:18.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.270+0000 7f2ce97e8640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "bar"}] 2024-01-24T23:24:18.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.270+0000 7f2ce97e8640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "bar"}] 2024-01-24T23:24:18.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.270+0000 7f2ce97e8640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:18.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.270+0000 7f2ce97e8640 1 -- 172.21.15.132:0/4150702687 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rm", "name": "bar"} v 0) v1 -- 0x7f2cc8005340 con 0x7f2ce40a0290 2024-01-24T23:24:18.886 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.882+0000 7f2cdb7fe640 1 -- 172.21.15.132:0/4150702687 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "bar"}]=0 removed item id -19 name 'bar' from crush map v85) v1 ==== 120+0+0 (unknown 1009776977 0 0) 0x7f2cdc01d010 con 0x7f2ce40a0290 2024-01-24T23:24:18.886 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.882+0000 7f2cdb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "bar"}] 2024-01-24T23:24:18.886 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.882+0000 7f2cdb7fe640 10 monclient: _finish_command 2 = system:0 removed item id -19 name 'bar' from crush map 2024-01-24T23:24:18.987 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.982+0000 7f2ce97e8640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "bar"}] 2024-01-24T23:24:18.987 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.982+0000 7f2ce97e8640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "bar"}] 2024-01-24T23:24:18.988 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.982+0000 7f2ce97e8640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:18.988 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.982+0000 7f2ce97e8640 1 -- 172.21.15.132:0/4150702687 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rm", "name": "bar"} v 0) v1 -- 0x7f2cc80010a0 con 0x7f2ce40a0290 2024-01-24T23:24:18.988 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.986+0000 7f2cdb7fe640 1 -- 172.21.15.132:0/4150702687 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "bar"}]=0 device 'bar' does not appear in the crush map v85) v1 ==== 120+0+0 (unknown 3578910950 0 0) 0x7f2cdc012f80 con 0x7f2ce40a0290 2024-01-24T23:24:18.988 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.986+0000 7f2cdb7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "bar"}] 2024-01-24T23:24:18.988 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.986+0000 7f2cdb7fe640 10 monclient: _finish_command 3 = system:0 device 'bar' does not appear in the crush map 2024-01-24T23:24:18.988 INFO:tasks.workunit.client.0.smithi132.stderr:device 'bar' does not appear in the crush map 2024-01-24T23:24:18.990 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.986+0000 7f2ce97e8640 1 -- 172.21.15.132:0/4150702687 >> v1:172.21.15.132:6812/1835401922 conn(0x7f2cc4037050 legacy=0x7f2cc4039510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:18.990 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.986+0000 7f2ce97e8640 10 monclient: shutdown 2024-01-24T23:24:18.990 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.986+0000 7f2ce97e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:18.990 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.986+0000 7f2ce97e8640 1 -- 172.21.15.132:0/4150702687 >> v1:172.21.15.132:6790/0 conn(0x7f2ce40a0290 legacy=0x7f2ce41cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:18.990 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.986+0000 7f2ce97e8640 1 -- 172.21.15.132:0/4150702687 shutdown_connections 2024-01-24T23:24:18.990 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.986+0000 7f2ce97e8640 1 -- 172.21.15.132:0/4150702687 >> 172.21.15.132:0/4150702687 conn(0x7f2ce40a5a10 msgr2=0x7f2ce40a3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:18.990 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.986+0000 7f2ce97e8640 1 -- 172.21.15.132:0/4150702687 shutdown_connections 2024-01-24T23:24:18.991 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:18.986+0000 7f2ce97e8640 1 -- 172.21.15.132:0/4150702687 wait complete. 2024-01-24T23:24:19.004 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm foo 2024-01-24T23:24:19.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.118+0000 7f8708708640 10 monclient: get_monmap_and_config 2024-01-24T23:24:19.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.118+0000 7f8708708640 10 monclient: build_initial_monmap 2024-01-24T23:24:19.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.118+0000 7f8708708640 10 monclient: monmap: 2024-01-24T23:24:19.122 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:19.122 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:19.122 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:19.123216+0000 2024-01-24T23:24:19.122 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:19.123216+0000 2024-01-24T23:24:19.122 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:19.122 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:19.122 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:19.122 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:19.123 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:19.123 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:19.123 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:19.123 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:19.123 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:19.123 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:19.123 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:19.123 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:19.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.118+0000 7f8708708640 1 Processor -- start 2024-01-24T23:24:19.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.118+0000 7f8708708640 1 -- start start 2024-01-24T23:24:19.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.118+0000 7f8708708640 10 monclient: init 2024-01-24T23:24:19.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.118+0000 7f8708708640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:19.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.118+0000 7f8708708640 10 monclient: _add_conns ranks=[0,6,3,4,1,2,5,7,8] 2024-01-24T23:24:19.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.118+0000 7f8708708640 10 monclient(hunting): picked mon.noname-a con 0x7f8700136e50 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:19.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.118+0000 7f8708708640 10 monclient(hunting): picked mon.noname-d con 0x7f87000a0290 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:19.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.118+0000 7f8708708640 10 monclient(hunting): picked mon.noname-g con 0x7f87000a1600 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:19.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.118+0000 7f8708708640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8700064700 con 0x7f8700136e50 2024-01-24T23:24:19.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.118+0000 7f8708708640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8700137940 con 0x7f87000a1600 2024-01-24T23:24:19.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.118+0000 7f8708708640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f870009f9c0 con 0x7f87000a0290 2024-01-24T23:24:19.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.118+0000 7f8708708640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:19.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.118+0000 7f8708708640 10 monclient(hunting): authenticate will time out at 1146.447817s 2024-01-24T23:24:19.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.118+0000 7f8706ca5640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f87000a1600 0x7f870013f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:47610/0 (socket says 172.21.15.132:47610) 2024-01-24T23:24:19.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.118+0000 7f8706ca5640 1 -- 172.21.15.132:0/3908618949 learned_addr learned my addr 172.21.15.132:0/3908618949 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:19.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87064a4640 1 -- 172.21.15.132:0/3908618949 >> v1:172.21.15.115:6789/0 conn(0x7f8700136e50 legacy=0x7f8700137230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:19.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 1 -- 172.21.15.132:0/3908618949 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1780509154 0 0) 0x7f8700137940 con 0x7f87000a1600 2024-01-24T23:24:19.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 10 monclient(hunting): _negotiate 2024-01-24T23:24:19.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:19.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:19.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 10 monclient(hunting): my global_id is 5520 2024-01-24T23:24:19.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 1 -- 172.21.15.132:0/3908618949 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86ec003530 con 0x7f87000a1600 2024-01-24T23:24:19.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 1 -- 172.21.15.132:0/3908618949 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2008577327 0 0) 0x7f86ec003530 con 0x7f87000a1600 2024-01-24T23:24:19.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 1 -- 172.21.15.132:0/3908618949 >> v1:172.21.15.115:6792/0 conn(0x7f87000a0290 legacy=0x7f870013bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:19.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 1 -- 172.21.15.132:0/3908618949 >> v1:172.21.15.115:6789/0 conn(0x7f8700136e50 legacy=0x7f8700137230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:19.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 10 monclient: _finish_hunting 0 2024-01-24T23:24:19.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 1 monclient: found mon.noname-g 2024-01-24T23:24:19.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:19.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:24:19.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 1 -- 172.21.15.132:0/3908618949 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870009eda0 con 0x7f87000a1600 2024-01-24T23:24:19.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 10 monclient: _finish_auth 0 2024-01-24T23:24:19.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 10 monclient: _check_auth_tickets 2024-01-24T23:24:19.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:19.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 5 monclient: authenticate success, global_id 5520 2024-01-24T23:24:19.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 1 -- 172.21.15.132:0/3908618949 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f86f8001b40 con 0x7f87000a1600 2024-01-24T23:24:19.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:19.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:19.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 10 monclient: dump: 2024-01-24T23:24:19.136 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:19.136 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:19.136 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:19.136 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:19.136 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:19.136 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:19.136 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:19.136 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:19.136 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:19.136 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:19.136 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:19.136 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:19.136 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:19.136 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:19.137 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:19.137 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:19.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:19.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 1 -- 172.21.15.132:0/3908618949 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f86f8001e60 con 0x7f87000a1600 2024-01-24T23:24:19.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:19.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 1 -- 172.21.15.132:0/3908618949 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f86f8004f70 con 0x7f87000a1600 2024-01-24T23:24:19.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:19.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 4 set_mon_vals no callback set 2024-01-24T23:24:19.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:19.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:19.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87054a2640 10 monclient: dump: 2024-01-24T23:24:19.137 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:19.137 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:19.137 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:19.137 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:19.138 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:19.138 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:19.138 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:19.138 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:19.138 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:19.138 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:19.138 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:19.138 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:19.138 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:19.138 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:19.138 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:19.138 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:19.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 10 monclient: shutdown 2024-01-24T23:24:19.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:19.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 1 -- 172.21.15.132:0/3908618949 >> v1:172.21.15.132:6790/0 conn(0x7f87000a1600 legacy=0x7f870013f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:19.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 1 -- 172.21.15.132:0/3908618949 shutdown_connections 2024-01-24T23:24:19.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 1 -- 172.21.15.132:0/3908618949 >> 172.21.15.132:0/3908618949 conn(0x7f87000a5a10 msgr2=0x7f87000a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:19.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 1 -- 172.21.15.132:0/3908618949 shutdown_connections 2024-01-24T23:24:19.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 1 -- 172.21.15.132:0/3908618949 wait complete. 2024-01-24T23:24:19.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 10 monclient: build_initial_monmap 2024-01-24T23:24:19.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 10 monclient: monmap: 2024-01-24T23:24:19.139 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:19.139 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:19.139 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:19.139 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:19.140 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:19.140 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:19.140 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:19.140 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:19.140 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:19.140 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:19.140 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:19.140 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:19.140 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:19.140 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:19.140 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:19.140 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:19.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 1 Processor -- start 2024-01-24T23:24:19.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 1 -- start start 2024-01-24T23:24:19.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 10 monclient: init 2024-01-24T23:24:19.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:19.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 10 monclient: _add_conns ranks=[5,0,7,4,1,8,6,2,3] 2024-01-24T23:24:19.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 10 monclient(hunting): picked mon.noname-h con 0x7f87000a0290 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:19.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 10 monclient(hunting): picked mon.noname-a con 0x7f87000a1600 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:19.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 10 monclient(hunting): picked mon.noname-i con 0x7f8700136e50 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:19.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f870009eae0 con 0x7f87000a1600 2024-01-24T23:24:19.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8700136c20 con 0x7f87000a0290 2024-01-24T23:24:19.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8700120330 con 0x7f8700136e50 2024-01-24T23:24:19.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:19.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 10 monclient(hunting): authenticate will time out at 1146.450826s 2024-01-24T23:24:19.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8705ca3640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f87000a1600 legacy=0x7f87001d04a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:19.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87064a4640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f87000a0290 0x7f8700134cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:51166/0 (socket says 172.21.15.132:51166) 2024-01-24T23:24:19.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8706ca5640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f8700136e50 0x7f87001d3bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:44140/0 (socket says 172.21.15.132:44140) 2024-01-24T23:24:19.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f87064a4640 1 -- 172.21.15.132:0/2164698295 learned_addr learned my addr 172.21.15.132:0/2164698295 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:19.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 1 -- 172.21.15.132:0/2164698295 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2546755048 0 0) 0x7f8700120330 con 0x7f8700136e50 2024-01-24T23:24:19.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:19.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:19.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:19.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 10 monclient(hunting): my global_id is 5569 2024-01-24T23:24:19.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 1 -- 172.21.15.132:0/2164698295 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86e4003530 con 0x7f8700136e50 2024-01-24T23:24:19.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 1 -- 172.21.15.132:0/2164698295 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2654386016 0 0) 0x7f8700136c20 con 0x7f87000a0290 2024-01-24T23:24:19.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:19.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:19.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:19.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 10 monclient(hunting): my global_id is 5504 2024-01-24T23:24:19.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 1 -- 172.21.15.132:0/2164698295 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8700120330 con 0x7f87000a0290 2024-01-24T23:24:19.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 1 -- 172.21.15.132:0/2164698295 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2119448469 0 0) 0x7f86e4003530 con 0x7f8700136e50 2024-01-24T23:24:19.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 1 -- 172.21.15.132:0/2164698295 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8700136c20 con 0x7f8700136e50 2024-01-24T23:24:19.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 1 -- 172.21.15.132:0/2164698295 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f86f8001b00 con 0x7f8700136e50 2024-01-24T23:24:19.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:19.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:19.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 10 monclient(hunting): dump: 2024-01-24T23:24:19.143 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:19.143 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:19.143 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:19.143 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:19.143 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:19.143 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:19.143 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:19.143 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:19.143 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:19.143 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:19.143 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:19.143 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:19.143 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:19.143 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:19.144 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:19.144 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:19.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:19.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 1 -- 172.21.15.132:0/2164698295 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3192306805 0 0) 0x7f8700120330 con 0x7f87000a0290 2024-01-24T23:24:19.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 1 -- 172.21.15.132:0/2164698295 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f86e4003530 con 0x7f87000a0290 2024-01-24T23:24:19.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 1 -- 172.21.15.132:0/2164698295 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f86fc004330 con 0x7f87000a0290 2024-01-24T23:24:19.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:19.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:19.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 10 monclient(hunting): dump: 2024-01-24T23:24:19.144 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:19.145 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:19.145 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:19.145 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:19.145 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:19.145 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:19.145 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:19.145 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:19.145 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:19.145 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:19.145 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:19.145 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:19.145 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:19.145 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:19.145 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:19.145 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:19.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 1 -- 172.21.15.132:0/2164698295 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3259669674 0 0) 0x7f8700136c20 con 0x7f8700136e50 2024-01-24T23:24:19.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 1 -- 172.21.15.132:0/2164698295 >> v1:172.21.15.132:6791/0 conn(0x7f87000a0290 legacy=0x7f8700134cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:19.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 1 -- 172.21.15.132:0/2164698295 >> v1:172.21.15.115:6789/0 conn(0x7f87000a1600 legacy=0x7f87001d04a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:19.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:19.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 1 monclient: found mon.i 2024-01-24T23:24:19.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:19.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:19.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 1 -- 172.21.15.132:0/2164698295 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8700122690 con 0x7f8700136e50 2024-01-24T23:24:19.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 10 monclient: _finish_auth 0 2024-01-24T23:24:19.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:19.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:19.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 5 monclient: authenticate success, global_id 5569 2024-01-24T23:24:19.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 10 monclient: _renew_subs 2024-01-24T23:24:19.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:19.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 1 -- 172.21.15.132:0/2164698295 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8700121df0 con 0x7f8700136e50 2024-01-24T23:24:19.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 10 monclient: _renew_subs 2024-01-24T23:24:19.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:19.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8708708640 1 -- 172.21.15.132:0/2164698295 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f87001c8630 con 0x7f8700136e50 2024-01-24T23:24:19.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 1 -- 172.21.15.132:0/2164698295 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f86f8001fd0 con 0x7f8700136e50 2024-01-24T23:24:19.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:19.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 1 -- 172.21.15.132:0/2164698295 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f86f80054f0 con 0x7f8700136e50 2024-01-24T23:24:19.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:19.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f8704ca1640 4 set_mon_vals no callback set 2024-01-24T23:24:19.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:19.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.122+0000 7f86f6ffd640 10 monclient: dump: 2024-01-24T23:24:19.147 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:19.147 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:19.147 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:19.147 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:19.147 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:19.147 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:19.148 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:19.148 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:19.148 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:19.148 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:19.148 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:19.148 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:19.148 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:19.148 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:19.148 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:19.148 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:19.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.126+0000 7f8708708640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:19.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.126+0000 7f8708708640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:19.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.126+0000 7f8708708640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:19.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.126+0000 7f8708708640 1 -- 172.21.15.132:0/2164698295 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e8005090 con 0x7f8700136e50 2024-01-24T23:24:19.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.126+0000 7f86f6ffd640 1 -- 172.21.15.132:0/2164698295 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f86f800ff70 con 0x7f8700136e50 2024-01-24T23:24:19.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.126+0000 7f86f6ffd640 1 -- 172.21.15.132:0/2164698295 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(85..85 src has 1..85) v4 ==== 5004+0+0 (unknown 1059413089 0 0) 0x7f86f8044180 con 0x7f8700136e50 2024-01-24T23:24:19.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.126+0000 7f87064a4640 10 monclient: get_auth_request con 0x7f86e4037550 auth_method 0 2024-01-24T23:24:19.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.130+0000 7f86f6ffd640 1 -- 172.21.15.132:0/2164698295 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f86f8014df0 con 0x7f8700136e50 2024-01-24T23:24:19.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.130+0000 7f86f6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:19.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.130+0000 7f86f6ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:19.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.294+0000 7f8708708640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "foo"}] 2024-01-24T23:24:19.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.294+0000 7f8708708640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-01-24T23:24:19.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.294+0000 7f8708708640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:19.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.294+0000 7f8708708640 1 -- 172.21.15.132:0/2164698295 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "foo"} v 0) v1 -- 0x7f86e8005340 con 0x7f8700136e50 2024-01-24T23:24:19.913 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.910+0000 7f86f6ffd640 1 -- 172.21.15.132:0/2164698295 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "foo"}]=0 removed item id -10 name 'foo' from crush map v86) v1 ==== 120+0+0 (unknown 3043260364 0 0) 0x7f86f8013490 con 0x7f8700136e50 2024-01-24T23:24:19.913 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.910+0000 7f86f6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-01-24T23:24:19.913 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:19.910+0000 7f86f6ffd640 10 monclient: _finish_command 2 = system:0 removed item id -10 name 'foo' from crush map 2024-01-24T23:24:20.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.014+0000 7f8708708640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "foo"}] 2024-01-24T23:24:20.020 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.014+0000 7f8708708640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "foo"}] 2024-01-24T23:24:20.020 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.014+0000 7f8708708640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:20.020 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.014+0000 7f8708708640 1 -- 172.21.15.132:0/2164698295 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "foo"} v 0) v1 -- 0x7f86e8002220 con 0x7f8700136e50 2024-01-24T23:24:20.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.018+0000 7f86f6ffd640 1 -- 172.21.15.132:0/2164698295 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "foo"}]=0 device 'foo' does not appear in the crush map v86) v1 ==== 120+0+0 (unknown 2490957799 0 0) 0x7f86f8014c10 con 0x7f8700136e50 2024-01-24T23:24:20.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.018+0000 7f86f6ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "foo"}] 2024-01-24T23:24:20.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.018+0000 7f86f6ffd640 10 monclient: _finish_command 3 = system:0 device 'foo' does not appear in the crush map 2024-01-24T23:24:20.021 INFO:tasks.workunit.client.0.smithi132.stderr:device 'foo' does not appear in the crush map 2024-01-24T23:24:20.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.018+0000 7f8708708640 1 -- 172.21.15.132:0/2164698295 >> v1:172.21.15.132:6812/1835401922 conn(0x7f86e4037550 legacy=0x7f86e4039a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:20.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.018+0000 7f8708708640 10 monclient: shutdown 2024-01-24T23:24:20.024 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.018+0000 7f8708708640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:20.024 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.018+0000 7f8708708640 1 -- 172.21.15.132:0/2164698295 >> v1:172.21.15.132:6792/0 conn(0x7f8700136e50 legacy=0x7f87001d3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:20.024 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.018+0000 7f8708708640 1 -- 172.21.15.132:0/2164698295 shutdown_connections 2024-01-24T23:24:20.024 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.018+0000 7f8708708640 1 -- 172.21.15.132:0/2164698295 >> 172.21.15.132:0/2164698295 conn(0x7f87000a5a10 msgr2=0x7f87000a3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:20.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.018+0000 7f8708708640 1 -- 172.21.15.132:0/2164698295 shutdown_connections 2024-01-24T23:24:20.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.018+0000 7f8708708640 1 -- 172.21.15.132:0/2164698295 wait complete. 2024-01-24T23:24:20.037 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm osd.9 host2 2024-01-24T23:24:20.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.150+0000 7f0c5f52a640 10 monclient: get_monmap_and_config 2024-01-24T23:24:20.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.150+0000 7f0c5f52a640 10 monclient: build_initial_monmap 2024-01-24T23:24:20.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.150+0000 7f0c5f52a640 10 monclient: monmap: 2024-01-24T23:24:20.156 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:20.156 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:20.157 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:20.156607+0000 2024-01-24T23:24:20.157 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:20.156607+0000 2024-01-24T23:24:20.157 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:20.157 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:20.157 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:20.157 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:20.157 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:20.157 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:20.158 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:20.158 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:20.158 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:20.158 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:20.158 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:20.158 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:20.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 1 Processor -- start 2024-01-24T23:24:20.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 1 -- start start 2024-01-24T23:24:20.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 10 monclient: init 2024-01-24T23:24:20.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:20.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 10 monclient: _add_conns ranks=[4,3,5,0,6,2,7,8,1] 2024-01-24T23:24:20.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 10 monclient(hunting): picked mon.noname-c con 0x7f0c58133b50 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:20.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 10 monclient(hunting): picked mon.noname-g con 0x7f0c5809ec20 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:20.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 10 monclient(hunting): picked mon.noname-h con 0x7f0c5813a1d0 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:20.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c58064700 con 0x7f0c5809ec20 2024-01-24T23:24:20.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c5809d9f0 con 0x7f0c58133b50 2024-01-24T23:24:20.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c5809dc60 con 0x7f0c5813a1d0 2024-01-24T23:24:20.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:20.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 10 monclient(hunting): authenticate will time out at 1147.481157s 2024-01-24T23:24:20.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5d2c6640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f0c58133b50 legacy=0x7f0c58133f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:20.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5dac7640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f0c5813a1d0 0x7f0c5813c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:51172/0 (socket says 172.21.15.132:51172) 2024-01-24T23:24:20.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5dac7640 1 -- 172.21.15.132:0/3862621249 learned_addr learned my addr 172.21.15.132:0/3862621249 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:20.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5cac5640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f0c5809ec20 0x7f0c58138ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:47616/0 (socket says 172.21.15.132:47616) 2024-01-24T23:24:20.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 1 -- 172.21.15.132:0/3862621249 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2457587904 0 0) 0x7f0c58064700 con 0x7f0c5809ec20 2024-01-24T23:24:20.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:20.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:20.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:20.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 10 monclient(hunting): my global_id is 5529 2024-01-24T23:24:20.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 1 -- 172.21.15.132:0/3862621249 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c44005740 con 0x7f0c5809ec20 2024-01-24T23:24:20.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 1 -- 172.21.15.132:0/3862621249 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3363488270 0 0) 0x7f0c5809dc60 con 0x7f0c5813a1d0 2024-01-24T23:24:20.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:20.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:20.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:20.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 10 monclient(hunting): my global_id is 5513 2024-01-24T23:24:20.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 1 -- 172.21.15.132:0/3862621249 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c58064700 con 0x7f0c5813a1d0 2024-01-24T23:24:20.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 1 -- 172.21.15.132:0/3862621249 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 113836600 0 0) 0x7f0c44005740 con 0x7f0c5809ec20 2024-01-24T23:24:20.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 1 -- 172.21.15.132:0/3862621249 >> v1:172.21.15.132:6791/0 conn(0x7f0c5813a1d0 legacy=0x7f0c5813c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:20.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 1 -- 172.21.15.132:0/3862621249 >> v1:172.21.15.115:6791/0 conn(0x7f0c58133b50 legacy=0x7f0c58133f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:20.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 10 monclient: _finish_hunting 0 2024-01-24T23:24:20.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 1 monclient: found mon.noname-g 2024-01-24T23:24:20.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:20.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:24:20.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 1 -- 172.21.15.132:0/3862621249 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5809df40 con 0x7f0c5809ec20 2024-01-24T23:24:20.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 10 monclient: _finish_auth 0 2024-01-24T23:24:20.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 10 monclient: _check_auth_tickets 2024-01-24T23:24:20.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:20.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 1 -- 172.21.15.132:0/3862621249 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0c480017c0 con 0x7f0c5809ec20 2024-01-24T23:24:20.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:20.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:20.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 10 monclient: dump: 2024-01-24T23:24:20.169 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:20.169 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:20.169 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:20.169 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:20.169 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:20.169 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:20.169 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:20.170 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:20.170 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:20.170 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:20.170 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:20.170 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:20.170 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:20.170 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:20.170 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:20.170 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:20.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 5 monclient: authenticate success, global_id 5529 2024-01-24T23:24:20.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:20.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 1 -- 172.21.15.132:0/3862621249 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0c48001a20 con 0x7f0c5809ec20 2024-01-24T23:24:20.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:20.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 1 -- 172.21.15.132:0/3862621249 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0c48004b30 con 0x7f0c5809ec20 2024-01-24T23:24:20.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:20.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 4 set_mon_vals no callback set 2024-01-24T23:24:20.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:20.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:20.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c4ffff640 10 monclient: dump: 2024-01-24T23:24:20.171 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:20.172 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:20.172 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:20.172 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:20.172 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:20.172 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:20.172 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:20.172 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:20.172 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:20.173 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:20.173 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:20.173 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:20.173 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:20.173 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:20.173 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:20.173 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:20.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 10 monclient: shutdown 2024-01-24T23:24:20.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:20.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 1 -- 172.21.15.132:0/3862621249 >> v1:172.21.15.132:6790/0 conn(0x7f0c5809ec20 legacy=0x7f0c58138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:20.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 1 -- 172.21.15.132:0/3862621249 shutdown_connections 2024-01-24T23:24:20.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 1 -- 172.21.15.132:0/3862621249 >> 172.21.15.132:0/3862621249 conn(0x7f0c580a5a10 msgr2=0x7f0c580a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:20.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 1 -- 172.21.15.132:0/3862621249 shutdown_connections 2024-01-24T23:24:20.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 1 -- 172.21.15.132:0/3862621249 wait complete. 2024-01-24T23:24:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 10 monclient: build_initial_monmap 2024-01-24T23:24:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 10 monclient: monmap: 2024-01-24T23:24:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:20.175 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:20.175 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:20.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 1 Processor -- start 2024-01-24T23:24:20.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 1 -- start start 2024-01-24T23:24:20.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.154+0000 7f0c5f52a640 10 monclient: init 2024-01-24T23:24:20.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c5f52a640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:20.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c5f52a640 10 monclient: _add_conns ranks=[5,6,2,3,0,4,7,8,1] 2024-01-24T23:24:20.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c5f52a640 10 monclient(hunting): picked mon.noname-h con 0x7f0c5809ec20 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:20.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c5f52a640 10 monclient(hunting): picked mon.noname-d con 0x7f0c58133b50 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:20.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c5f52a640 10 monclient(hunting): picked mon.noname-b con 0x7f0c5813a1d0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:20.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c5f52a640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c581371c0 con 0x7f0c5813a1d0 2024-01-24T23:24:20.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c5f52a640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c580a13b0 con 0x7f0c5809ec20 2024-01-24T23:24:20.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c5f52a640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c58138b90 con 0x7f0c58133b50 2024-01-24T23:24:20.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c5f52a640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:20.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c5f52a640 10 monclient(hunting): authenticate will time out at 1147.484220s 2024-01-24T23:24:20.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c5d2c6640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f0c5809ec20 0x7f0c581c3580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:51176/0 (socket says 172.21.15.132:51176) 2024-01-24T23:24:20.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c5d2c6640 1 -- 172.21.15.132:0/2268533268 learned_addr learned my addr 172.21.15.132:0/2268533268 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:20.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 1 -- 172.21.15.132:0/2268533268 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2909675979 0 0) 0x7f0c580a13b0 con 0x7f0c5809ec20 2024-01-24T23:24:20.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:20.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:20.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:20.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 10 monclient(hunting): my global_id is 5522 2024-01-24T23:24:20.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 1 -- 172.21.15.132:0/2268533268 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c3c003530 con 0x7f0c5809ec20 2024-01-24T23:24:20.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 1 -- 172.21.15.132:0/2268533268 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1097921183 0 0) 0x7f0c3c003530 con 0x7f0c5809ec20 2024-01-24T23:24:20.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 1 -- 172.21.15.132:0/2268533268 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0c580a13b0 con 0x7f0c5809ec20 2024-01-24T23:24:20.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 1 -- 172.21.15.132:0/2268533268 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0c50004330 con 0x7f0c5809ec20 2024-01-24T23:24:20.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:20.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:20.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 10 monclient(hunting): dump: 2024-01-24T23:24:20.177 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:20.177 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:20.177 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:20.177 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:20.177 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:20.177 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:20.177 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:20.177 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:20.177 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:20.177 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:20.177 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:20.178 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:20.178 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:20.178 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:20.178 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:20.178 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:20.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:20.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 1 -- 172.21.15.132:0/2268533268 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2903082451 0 0) 0x7f0c580a13b0 con 0x7f0c5809ec20 2024-01-24T23:24:20.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 1 -- 172.21.15.132:0/2268533268 >> v1:172.21.15.115:6792/0 conn(0x7f0c58133b50 legacy=0x7f0c581c3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:20.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 1 -- 172.21.15.132:0/2268533268 >> v1:172.21.15.115:6790/0 conn(0x7f0c5813a1d0 legacy=0x7f0c581cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:20.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 10 monclient: _finish_hunting 0 2024-01-24T23:24:20.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 1 monclient: found mon.h 2024-01-24T23:24:20.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:20.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:20.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 1 -- 172.21.15.132:0/2268533268 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c581c43a0 con 0x7f0c5809ec20 2024-01-24T23:24:20.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 10 monclient: _finish_auth 0 2024-01-24T23:24:20.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 10 monclient: _check_auth_tickets 2024-01-24T23:24:20.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:20.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c5f52a640 5 monclient: authenticate success, global_id 5522 2024-01-24T23:24:20.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c5f52a640 10 monclient: _renew_subs 2024-01-24T23:24:20.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c5f52a640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:20.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c5f52a640 1 -- 172.21.15.132:0/2268533268 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c58133920 con 0x7f0c5809ec20 2024-01-24T23:24:20.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c5f52a640 10 monclient: _renew_subs 2024-01-24T23:24:20.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c5f52a640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:20.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c5f52a640 1 -- 172.21.15.132:0/2268533268 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c581388b0 con 0x7f0c5809ec20 2024-01-24T23:24:20.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 1 -- 172.21.15.132:0/2268533268 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0c50004650 con 0x7f0c5809ec20 2024-01-24T23:24:20.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:20.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 1 -- 172.21.15.132:0/2268533268 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0c50004b80 con 0x7f0c5809ec20 2024-01-24T23:24:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4f7fe640 4 set_mon_vals no callback set 2024-01-24T23:24:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 10 monclient: dump: 2024-01-24T23:24:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:20.181 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:20.181 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:20.181 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:20.181 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:20.181 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:20.181 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:20.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c5f52a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:20.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c5f52a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:20.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c5f52a640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:20.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c5f52a640 1 -- 172.21.15.132:0/2268533268 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c38005090 con 0x7f0c5809ec20 2024-01-24T23:24:20.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 1 -- 172.21.15.132:0/2268533268 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f0c50004d60 con 0x7f0c5809ec20 2024-01-24T23:24:20.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c4dffb640 1 -- 172.21.15.132:0/2268533268 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(86..86 src has 1..86) v4 ==== 4875+0+0 (unknown 238863221 0 0) 0x7f0c50043d00 con 0x7f0c5809ec20 2024-01-24T23:24:20.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.158+0000 7f0c5cac5640 10 monclient: get_auth_request con 0x7f0c3c036e80 auth_method 0 2024-01-24T23:24:20.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.162+0000 7f0c4dffb640 1 -- 172.21.15.132:0/2268533268 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f0c50014b90 con 0x7f0c5809ec20 2024-01-24T23:24:20.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.162+0000 7f0c4dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:20.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.162+0000 7f0c4dffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:20.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.322+0000 7f0c5f52a640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}] 2024-01-24T23:24:20.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.322+0000 7f0c5f52a640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}] 2024-01-24T23:24:20.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.322+0000 7f0c5f52a640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:20.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.322+0000 7f0c5f52a640 1 -- 172.21.15.132:0/2268533268 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"} v 0) v1 -- 0x7f0c380058c0 con 0x7f0c5809ec20 2024-01-24T23:24:20.927 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.922+0000 7f0c4dffb640 1 -- 172.21.15.132:0/2268533268 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}]=0 removed item id 9 name 'osd.9' from crush map v87) v1 ==== 143+0+0 (unknown 2024745477 0 0) 0x7f0c50013000 con 0x7f0c5809ec20 2024-01-24T23:24:20.927 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.926+0000 7f0c4dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}] 2024-01-24T23:24:20.928 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:20.926+0000 7f0c4dffb640 10 monclient: _finish_command 2 = system:0 removed item id 9 name 'osd.9' from crush map 2024-01-24T23:24:21.030 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.026+0000 7f0c5f52a640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}] 2024-01-24T23:24:21.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.026+0000 7f0c5f52a640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}] 2024-01-24T23:24:21.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.026+0000 7f0c5f52a640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:21.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.026+0000 7f0c5f52a640 1 -- 172.21.15.132:0/2268533268 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"} v 0) v1 -- 0x7f0c38000bc0 con 0x7f0c5809ec20 2024-01-24T23:24:21.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.026+0000 7f0c4dffb640 1 -- 172.21.15.132:0/2268533268 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}]=0 device 'osd.9' does not appear in the crush map v87) v1 ==== 145+0+0 (unknown 3604641327 0 0) 0x7f0c50014780 con 0x7f0c5809ec20 2024-01-24T23:24:21.032 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.030+0000 7f0c4dffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}] 2024-01-24T23:24:21.032 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.030+0000 7f0c4dffb640 10 monclient: _finish_command 3 = system:0 device 'osd.9' does not appear in the crush map 2024-01-24T23:24:21.032 INFO:tasks.workunit.client.0.smithi132.stderr:device 'osd.9' does not appear in the crush map 2024-01-24T23:24:21.033 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.030+0000 7f0c5f52a640 1 -- 172.21.15.132:0/2268533268 >> v1:172.21.15.132:6812/1835401922 conn(0x7f0c3c036e80 legacy=0x7f0c3c039340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:21.034 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.030+0000 7f0c5f52a640 10 monclient: shutdown 2024-01-24T23:24:21.034 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.030+0000 7f0c5f52a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:21.034 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.030+0000 7f0c5f52a640 1 -- 172.21.15.132:0/2268533268 >> v1:172.21.15.132:6791/0 conn(0x7f0c5809ec20 legacy=0x7f0c581c3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:21.034 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.030+0000 7f0c5f52a640 1 -- 172.21.15.132:0/2268533268 shutdown_connections 2024-01-24T23:24:21.034 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.030+0000 7f0c5f52a640 1 -- 172.21.15.132:0/2268533268 >> 172.21.15.132:0/2268533268 conn(0x7f0c580a5a10 msgr2=0x7f0c580a0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:21.034 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.030+0000 7f0c5f52a640 1 -- 172.21.15.132:0/2268533268 shutdown_connections 2024-01-24T23:24:21.034 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.030+0000 7f0c5f52a640 1 -- 172.21.15.132:0/2268533268 wait complete. 2024-01-24T23:24:21.048 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm host2 2024-01-24T23:24:21.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.162+0000 7f1e93468640 10 monclient: get_monmap_and_config 2024-01-24T23:24:21.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.162+0000 7f1e93468640 10 monclient: build_initial_monmap 2024-01-24T23:24:21.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 10 monclient: monmap: 2024-01-24T23:24:21.168 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:21.168 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:21.168 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:21.168807+0000 2024-01-24T23:24:21.168 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:21.168807+0000 2024-01-24T23:24:21.168 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:21.168 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:21.168 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:21.168 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:21.168 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:21.168 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:21.169 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:21.169 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:21.169 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:21.169 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:21.169 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:21.169 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:21.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 1 Processor -- start 2024-01-24T23:24:21.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 1 -- start start 2024-01-24T23:24:21.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 10 monclient: init 2024-01-24T23:24:21.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:21.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 10 monclient: _add_conns ranks=[5,3,8,0,1,7,6,4,2] 2024-01-24T23:24:21.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 10 monclient(hunting): picked mon.noname-h con 0x7f1e8c09ff90 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:21.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 10 monclient(hunting): picked mon.noname-g con 0x7f1e8c09f9c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:21.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 10 monclient(hunting): picked mon.noname-e con 0x7f1e8c13b2c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:21.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e8c064700 con 0x7f1e8c09f9c0 2024-01-24T23:24:21.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e8c09fe00 con 0x7f1e8c09ff90 2024-01-24T23:24:21.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e8c0a1160 con 0x7f1e8c13b2c0 2024-01-24T23:24:21.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:21.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 10 monclient(hunting): authenticate will time out at 1148.493493s 2024-01-24T23:24:21.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e90a03640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f1e8c09f9c0 0x7f1e8c139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:47618/0 (socket says 172.21.15.132:47618) 2024-01-24T23:24:21.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e91204640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f1e8c09ff90 0x7f1e8c0a0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:51180/0 (socket says 172.21.15.132:51180) 2024-01-24T23:24:21.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e90a03640 1 -- 172.21.15.132:0/11054756 learned_addr learned my addr 172.21.15.132:0/11054756 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:21.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 1 -- 172.21.15.132:0/11054756 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714783621 0 0) 0x7f1e8c09fe00 con 0x7f1e8c09ff90 2024-01-24T23:24:21.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:21.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:21.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:21.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 10 monclient(hunting): my global_id is 5531 2024-01-24T23:24:21.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 1 -- 172.21.15.132:0/11054756 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e78003530 con 0x7f1e8c09ff90 2024-01-24T23:24:21.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 1 -- 172.21.15.132:0/11054756 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423374311 0 0) 0x7f1e8c064700 con 0x7f1e8c09f9c0 2024-01-24T23:24:21.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:21.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:21.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:21.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 10 monclient(hunting): my global_id is 5538 2024-01-24T23:24:21.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 1 -- 172.21.15.132:0/11054756 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e8c09fe00 con 0x7f1e8c09f9c0 2024-01-24T23:24:21.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 1 -- 172.21.15.132:0/11054756 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3617172050 0 0) 0x7f1e8c09fe00 con 0x7f1e8c09f9c0 2024-01-24T23:24:21.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 1 -- 172.21.15.132:0/11054756 >> v1:172.21.15.115:6793/0 conn(0x7f1e8c13b2c0 legacy=0x7f1e8c13d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:21.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 1 -- 172.21.15.132:0/11054756 >> v1:172.21.15.132:6791/0 conn(0x7f1e8c09ff90 legacy=0x7f1e8c0a0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:21.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 10 monclient: _finish_hunting 0 2024-01-24T23:24:21.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 1 monclient: found mon.noname-g 2024-01-24T23:24:21.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:21.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:24:21.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 1 -- 172.21.15.132:0/11054756 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e8c0a1440 con 0x7f1e8c09f9c0 2024-01-24T23:24:21.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 10 monclient: _finish_auth 0 2024-01-24T23:24:21.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 10 monclient: _check_auth_tickets 2024-01-24T23:24:21.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:21.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 1 -- 172.21.15.132:0/11054756 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1e88001720 con 0x7f1e8c09f9c0 2024-01-24T23:24:21.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 5 monclient: authenticate success, global_id 5538 2024-01-24T23:24:21.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:21.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:21.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:21.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 10 monclient: dump: 2024-01-24T23:24:21.180 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:21.180 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:21.180 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:21.180 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:21.180 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:21.180 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:21.180 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:21.180 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:21.180 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:21.180 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:21.180 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:21.180 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:21.180 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:21.181 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:21.181 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:21.181 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:21.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:21.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 1 -- 172.21.15.132:0/11054756 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1e880019c0 con 0x7f1e8c09f9c0 2024-01-24T23:24:21.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:21.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:21.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 4 set_mon_vals no callback set 2024-01-24T23:24:21.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 1 -- 172.21.15.132:0/11054756 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1e880048b0 con 0x7f1e8c09f9c0 2024-01-24T23:24:21.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:21.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:21.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e83fff640 10 monclient: dump: 2024-01-24T23:24:21.181 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:21.181 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:21.181 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:21.182 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:21.182 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:21.182 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:21.182 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:21.182 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:21.182 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:21.182 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:21.182 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:21.182 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:21.182 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:21.182 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:21.182 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:21.182 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:21.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 10 monclient: shutdown 2024-01-24T23:24:21.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:21.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 1 -- 172.21.15.132:0/11054756 >> v1:172.21.15.132:6790/0 conn(0x7f1e8c09f9c0 legacy=0x7f1e8c139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:21.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 1 -- 172.21.15.132:0/11054756 shutdown_connections 2024-01-24T23:24:21.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 1 -- 172.21.15.132:0/11054756 >> 172.21.15.132:0/11054756 conn(0x7f1e8c0a5a10 msgr2=0x7f1e8c0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:21.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 1 -- 172.21.15.132:0/11054756 shutdown_connections 2024-01-24T23:24:21.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 1 -- 172.21.15.132:0/11054756 wait complete. 2024-01-24T23:24:21.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 10 monclient: build_initial_monmap 2024-01-24T23:24:21.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.166+0000 7f1e93468640 10 monclient: monmap: 2024-01-24T23:24:21.183 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:21.183 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:21.183 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:21.183 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:21.183 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:21.183 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:21.183 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:21.183 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:21.184 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:21.184 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:21.184 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:21.184 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:21.184 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:21.184 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:21.184 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:21.184 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:21.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e93468640 1 Processor -- start 2024-01-24T23:24:21.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e93468640 1 -- start start 2024-01-24T23:24:21.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e93468640 10 monclient: init 2024-01-24T23:24:21.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e93468640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:21.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e93468640 10 monclient: _add_conns ranks=[1,0,8,5,6,2,4,3,7] 2024-01-24T23:24:21.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e93468640 10 monclient(hunting): picked mon.noname-f con 0x7f1e8c09f9c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:21.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e93468640 10 monclient(hunting): picked mon.noname-a con 0x7f1e8c09ff90 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:21.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e93468640 10 monclient(hunting): picked mon.noname-e con 0x7f1e8c13b2c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:21.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e93468640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e8c09f450 con 0x7f1e8c09ff90 2024-01-24T23:24:21.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e91204640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f1e8c09f9c0 legacy=0x7f1e8c135970 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:21.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e93468640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e8c071960 con 0x7f1e8c09f9c0 2024-01-24T23:24:21.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e93468640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e8c136c50 con 0x7f1e8c13b2c0 2024-01-24T23:24:21.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e93468640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:21.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e93468640 10 monclient(hunting): authenticate will time out at 1148.496827s 2024-01-24T23:24:21.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e90a03640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f1e8c09ff90 legacy=0x7f1e8c136080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:21.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e91a05640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f1e8c13b2c0 0x7f1e8c1d18b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:60972/0 (socket says 172.21.15.132:60972) 2024-01-24T23:24:21.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e91a05640 1 -- 172.21.15.132:0/1960897944 learned_addr learned my addr 172.21.15.132:0/1960897944 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:21.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 1 -- 172.21.15.132:0/1960897944 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 184365779 0 0) 0x7f1e8c136c50 con 0x7f1e8c13b2c0 2024-01-24T23:24:21.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:21.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:21.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:21.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 10 monclient(hunting): my global_id is 84725 2024-01-24T23:24:21.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 1 -- 172.21.15.132:0/1960897944 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e70003530 con 0x7f1e8c13b2c0 2024-01-24T23:24:21.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 1 -- 172.21.15.132:0/1960897944 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1243104582 0 0) 0x7f1e70003530 con 0x7f1e8c13b2c0 2024-01-24T23:24:21.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 1 -- 172.21.15.132:0/1960897944 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1e8c136c50 con 0x7f1e8c13b2c0 2024-01-24T23:24:21.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 1 -- 172.21.15.132:0/1960897944 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1e84001cc0 con 0x7f1e8c13b2c0 2024-01-24T23:24:21.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:21.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:24:21.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 10 monclient(hunting): dump: 2024-01-24T23:24:21.186 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:21.186 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:21.186 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:21.186 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:21.186 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:21.186 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:21.186 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:21.187 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:21.187 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:21.187 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:21.187 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:21.187 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:21.187 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:21.187 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:21.187 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:21.187 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:21.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:21.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 1 -- 172.21.15.132:0/1960897944 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4099043628 0 0) 0x7f1e8c136c50 con 0x7f1e8c13b2c0 2024-01-24T23:24:21.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 1 -- 172.21.15.132:0/1960897944 >> v1:172.21.15.132:6789/0 conn(0x7f1e8c09f9c0 legacy=0x7f1e8c135970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:21.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 1 -- 172.21.15.132:0/1960897944 >> v1:172.21.15.115:6789/0 conn(0x7f1e8c09ff90 legacy=0x7f1e8c136080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:21.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:24:21.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 1 monclient: found mon.e 2024-01-24T23:24:21.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:21.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:21.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 1 -- 172.21.15.132:0/1960897944 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e8c136790 con 0x7f1e8c13b2c0 2024-01-24T23:24:21.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 10 monclient: _finish_auth 0 2024-01-24T23:24:21.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:24:21.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:21.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e93468640 5 monclient: authenticate success, global_id 84725 2024-01-24T23:24:21.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e93468640 10 monclient: _renew_subs 2024-01-24T23:24:21.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e93468640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:21.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e93468640 1 -- 172.21.15.132:0/1960897944 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e8c0715f0 con 0x7f1e8c13b2c0 2024-01-24T23:24:21.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e93468640 10 monclient: _renew_subs 2024-01-24T23:24:21.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e93468640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:21.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e93468640 1 -- 172.21.15.132:0/1960897944 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e8c137d70 con 0x7f1e8c13b2c0 2024-01-24T23:24:21.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 1 -- 172.21.15.132:0/1960897944 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1e84002090 con 0x7f1e8c13b2c0 2024-01-24T23:24:21.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:21.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 1 -- 172.21.15.132:0/1960897944 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1e84003f90 con 0x7f1e8c13b2c0 2024-01-24T23:24:21.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:21.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e837fe640 4 set_mon_vals no callback set 2024-01-24T23:24:21.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:24:21.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 10 monclient: dump: 2024-01-24T23:24:21.189 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:21.189 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:21.189 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:21.189 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:21.189 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:21.189 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:21.189 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:21.190 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:21.190 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:21.190 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:21.190 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:21.190 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:21.190 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:21.190 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:21.190 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:21.190 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:21.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e93468640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:21.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e93468640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:21.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e93468640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:21.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e93468640 1 -- 172.21.15.132:0/1960897944 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e6c005090 con 0x7f1e8c13b2c0 2024-01-24T23:24:21.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 1 -- 172.21.15.132:0/1960897944 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f1e84004170 con 0x7f1e8c13b2c0 2024-01-24T23:24:21.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.170+0000 7f1e81ffb640 1 -- 172.21.15.132:0/1960897944 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(87..87 src has 1..87) v4 ==== 4854+0+0 (unknown 87213664 0 0) 0x7f1e84043ec0 con 0x7f1e8c13b2c0 2024-01-24T23:24:21.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.174+0000 7f1e91204640 10 monclient: get_auth_request con 0x7f1e70036e30 auth_method 0 2024-01-24T23:24:21.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.182+0000 7f1e81ffb640 1 -- 172.21.15.132:0/1960897944 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f1e84014900 con 0x7f1e8c13b2c0 2024-01-24T23:24:21.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.182+0000 7f1e81ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:21.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.182+0000 7f1e81ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:21.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.346+0000 7f1e93468640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host2"}] 2024-01-24T23:24:21.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.346+0000 7f1e93468640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "host2"}] 2024-01-24T23:24:21.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.346+0000 7f1e93468640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:21.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.346+0000 7f1e93468640 1 -- 172.21.15.132:0/1960897944 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush rm", "name": "host2"} v 0) v1 -- 0x7f1e6c005340 con 0x7f1e8c13b2c0 2024-01-24T23:24:21.941 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.938+0000 7f1e81ffb640 1 -- 172.21.15.132:0/1960897944 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host2"}]=0 removed item id -16 name 'host2' from crush map v88) v1 ==== 124+0+0 (unknown 3730905177 0 0) 0x7f1e84015620 con 0x7f1e8c13b2c0 2024-01-24T23:24:21.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.938+0000 7f1e81ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "host2"}] 2024-01-24T23:24:21.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:21.938+0000 7f1e81ffb640 10 monclient: _finish_command 2 = system:0 removed item id -16 name 'host2' from crush map 2024-01-24T23:24:22.043 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.042+0000 7f1e93468640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host2"}] 2024-01-24T23:24:22.044 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.042+0000 7f1e93468640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "host2"}] 2024-01-24T23:24:22.044 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.042+0000 7f1e93468640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:22.044 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.042+0000 7f1e93468640 1 -- 172.21.15.132:0/1960897944 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush rm", "name": "host2"} v 0) v1 -- 0x7f1e6c0057e0 con 0x7f1e8c13b2c0 2024-01-24T23:24:22.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.042+0000 7f1e81ffb640 1 -- 172.21.15.132:0/1960897944 <== mon.8 v1:172.21.15.115:6793/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host2"}]=0 device 'host2' does not appear in the crush map v88) v1 ==== 124+0+0 (unknown 3469394341 0 0) 0x7f1e84015620 con 0x7f1e8c13b2c0 2024-01-24T23:24:22.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.042+0000 7f1e81ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "host2"}] 2024-01-24T23:24:22.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.042+0000 7f1e81ffb640 10 monclient: _finish_command 3 = system:0 device 'host2' does not appear in the crush map 2024-01-24T23:24:22.046 INFO:tasks.workunit.client.0.smithi132.stderr:device 'host2' does not appear in the crush map 2024-01-24T23:24:22.047 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.046+0000 7f1e93468640 1 -- 172.21.15.132:0/1960897944 >> v1:172.21.15.132:6812/1835401922 conn(0x7f1e70036e30 legacy=0x7f1e700392f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:22.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.046+0000 7f1e93468640 10 monclient: shutdown 2024-01-24T23:24:22.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.046+0000 7f1e93468640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:22.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.046+0000 7f1e93468640 1 -- 172.21.15.132:0/1960897944 >> v1:172.21.15.115:6793/0 conn(0x7f1e8c13b2c0 legacy=0x7f1e8c1d18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:22.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.046+0000 7f1e93468640 1 -- 172.21.15.132:0/1960897944 shutdown_connections 2024-01-24T23:24:22.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.046+0000 7f1e93468640 1 -- 172.21.15.132:0/1960897944 >> 172.21.15.132:0/1960897944 conn(0x7f1e8c0a5a10 msgr2=0x7f1e8c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:22.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.046+0000 7f1e93468640 1 -- 172.21.15.132:0/1960897944 shutdown_connections 2024-01-24T23:24:22.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.046+0000 7f1e93468640 1 -- 172.21.15.132:0/1960897944 wait complete. 2024-01-24T23:24:22.062 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket foo host 2024-01-24T23:24:22.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.178+0000 7efd17eeb640 10 monclient: get_monmap_and_config 2024-01-24T23:24:22.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.178+0000 7efd17eeb640 10 monclient: build_initial_monmap 2024-01-24T23:24:22.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.178+0000 7efd17eeb640 10 monclient: monmap: 2024-01-24T23:24:22.180 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:22.180 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:22.180 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:22.181217+0000 2024-01-24T23:24:22.180 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:22.181217+0000 2024-01-24T23:24:22.180 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:22.180 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:22.180 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:22.181 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:22.181 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:22.181 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:22.181 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:22.181 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:22.181 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:22.181 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:22.181 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:22.181 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:22.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.178+0000 7efd17eeb640 1 Processor -- start 2024-01-24T23:24:22.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.178+0000 7efd17eeb640 1 -- start start 2024-01-24T23:24:22.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.178+0000 7efd17eeb640 10 monclient: init 2024-01-24T23:24:22.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.178+0000 7efd17eeb640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:22.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.178+0000 7efd17eeb640 10 monclient: _add_conns ranks=[1,0,4,6,2,7,5,8,3] 2024-01-24T23:24:22.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.178+0000 7efd17eeb640 10 monclient(hunting): picked mon.noname-f con 0x7efd1009ff20 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:22.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.178+0000 7efd17eeb640 10 monclient(hunting): picked mon.noname-a con 0x7efd1009f9c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:22.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.178+0000 7efd17eeb640 10 monclient(hunting): picked mon.noname-c con 0x7efd1013b2c0 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:22.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.178+0000 7efd17eeb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd10064700 con 0x7efd1009f9c0 2024-01-24T23:24:22.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.178+0000 7efd17eeb640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd1009e760 con 0x7efd1009ff20 2024-01-24T23:24:22.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.178+0000 7efd17eeb640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd100a1160 con 0x7efd1013b2c0 2024-01-24T23:24:22.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.178+0000 7efd17eeb640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:22.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.178+0000 7efd17eeb640 10 monclient(hunting): authenticate will time out at 1149.506507s 2024-01-24T23:24:22.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.178+0000 7efd15c87640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7efd1009ff20 legacy=0x7efd100a0300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:22.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.178+0000 7efd15486640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7efd1009f9c0 legacy=0x7efd10139ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:22.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.178+0000 7efd16488640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7efd1013b2c0 legacy=0x7efd1013d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:22.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.382+0000 7efd15c87640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7efd1009ff20 legacy=0x7efd100a0300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:22.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.382+0000 7efd15486640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7efd1009f9c0 legacy=0x7efd10139ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:22.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.382+0000 7efd16488640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7efd1013b2c0 legacy=0x7efd1013d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:22.788 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.786+0000 7efd15c87640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7efd1009ff20 legacy=0x7efd100a0300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:22.788 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.786+0000 7efd16488640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7efd1013b2c0 legacy=0x7efd1013d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:22.788 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:22.786+0000 7efd15486640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7efd1009f9c0 legacy=0x7efd10139ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:23.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:23.590+0000 7efd15c87640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7efd1009ff20 legacy=0x7efd100a0300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:23.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:23.590+0000 7efd15486640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7efd1009f9c0 legacy=0x7efd10139ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:23.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:23.590+0000 7efd16488640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7efd1013b2c0 legacy=0x7efd1013d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:25.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd077fe640 10 monclient(hunting): tick 2024-01-24T23:24:25.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd077fe640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:24:25.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd077fe640 1 monclient(hunting): continuing hunt 2024-01-24T23:24:25.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd077fe640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:24:25.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd077fe640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7efd1013b2c0 legacy=0x7efd1013d6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:25.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd077fe640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7efd1009ff20 legacy=0x7efd100a0300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:25.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd077fe640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7efd1009f9c0 legacy=0x7efd10139ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:25.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd077fe640 10 monclient: _add_conns ranks=[7,8,5,6,2,3] 2024-01-24T23:24:25.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd077fe640 10 monclient(hunting): picked mon.noname-i con 0x7efcf4003790 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:25.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd077fe640 10 monclient(hunting): picked mon.noname-e con 0x7efcf4007420 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:25.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd077fe640 10 monclient(hunting): picked mon.noname-h con 0x7efcf400b1b0 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:25.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd077fe640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd100a0a10 con 0x7efcf400b1b0 2024-01-24T23:24:25.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd077fe640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd10064700 con 0x7efcf4003790 2024-01-24T23:24:25.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd077fe640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd1009e760 con 0x7efcf4007420 2024-01-24T23:24:25.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd077fe640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:25.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd16488640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7efcf400b1b0 0x7efcf400d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:48366/0 (socket says 172.21.15.132:48366) 2024-01-24T23:24:25.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd16488640 1 -- 172.21.15.132:0/397542435 learned_addr learned my addr 172.21.15.132:0/397542435 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:25.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd14c85640 1 -- 172.21.15.132:0/397542435 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1672370793 0 0) 0x7efd100a0a10 con 0x7efcf400b1b0 2024-01-24T23:24:25.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd14c85640 10 monclient(hunting): _negotiate 2024-01-24T23:24:25.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd14c85640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:25.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd14c85640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:25.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd14c85640 10 monclient(hunting): my global_id is 5540 2024-01-24T23:24:25.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd14c85640 1 -- 172.21.15.132:0/397542435 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcfc003530 con 0x7efcf400b1b0 2024-01-24T23:24:25.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd14c85640 1 -- 172.21.15.132:0/397542435 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4280282955 0 0) 0x7efd10064700 con 0x7efcf4003790 2024-01-24T23:24:25.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd14c85640 10 monclient(hunting): _negotiate 2024-01-24T23:24:25.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd14c85640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:25.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd14c85640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:25.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.178+0000 7efd14c85640 10 monclient(hunting): my global_id is 5578 2024-01-24T23:24:25.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 1 -- 172.21.15.132:0/397542435 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd100a0a10 con 0x7efcf4003790 2024-01-24T23:24:25.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 1 -- 172.21.15.132:0/397542435 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540950203 0 0) 0x7efd1009e760 con 0x7efcf4007420 2024-01-24T23:24:25.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 10 monclient(hunting): _negotiate 2024-01-24T23:24:25.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:25.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:25.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 10 monclient(hunting): my global_id is 84734 2024-01-24T23:24:25.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 1 -- 172.21.15.132:0/397542435 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd10064700 con 0x7efcf4007420 2024-01-24T23:24:25.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 1 -- 172.21.15.132:0/397542435 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 673152983 0 0) 0x7efcfc003530 con 0x7efcf400b1b0 2024-01-24T23:24:25.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 1 -- 172.21.15.132:0/397542435 >> v1:172.21.15.115:6793/0 conn(0x7efcf4007420 legacy=0x7efcf40098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:25.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 1 -- 172.21.15.132:0/397542435 >> v1:172.21.15.132:6792/0 conn(0x7efcf4003790 legacy=0x7efcf4005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:25.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 10 monclient: _finish_hunting 0 2024-01-24T23:24:25.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 1 monclient: found mon.noname-h 2024-01-24T23:24:25.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:25.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd16488640 1 -- 172.21.15.132:0/397542435 reap_dead start 2024-01-24T23:24:25.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:24:25.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 1 -- 172.21.15.132:0/397542435 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd100a1160 con 0x7efcf400b1b0 2024-01-24T23:24:25.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 10 monclient: _finish_auth 0 2024-01-24T23:24:25.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 10 monclient: _check_auth_tickets 2024-01-24T23:24:25.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:25.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 1 -- 172.21.15.132:0/397542435 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7efd08004490 con 0x7efcf400b1b0 2024-01-24T23:24:25.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:25.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:25.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 10 monclient: dump: 2024-01-24T23:24:25.188 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:25.188 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:25.188 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:25.188 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:25.188 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:25.188 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:25.188 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:25.188 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:25.188 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:25.189 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:25.189 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:25.189 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:25.189 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:25.189 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:25.189 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:25.189 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:25.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd17eeb640 5 monclient: authenticate success, global_id 5540 2024-01-24T23:24:25.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd17eeb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:25.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 1 -- 172.21.15.132:0/397542435 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efd080046c0 con 0x7efcf400b1b0 2024-01-24T23:24:25.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:25.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 1 -- 172.21.15.132:0/397542435 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7efd08004bf0 con 0x7efcf400b1b0 2024-01-24T23:24:25.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:25.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:25.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd14c85640 10 monclient: dump: 2024-01-24T23:24:25.190 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:25.190 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:25.190 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:25.190 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:25.190 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:25.190 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:25.190 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:25.190 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:25.190 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:25.190 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:25.190 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:25.190 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:25.190 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:25.190 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:25.190 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:25.191 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:25.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd17eeb640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:25.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd17eeb640 4 set_mon_vals no callback set 2024-01-24T23:24:25.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd17eeb640 10 monclient: shutdown 2024-01-24T23:24:25.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd17eeb640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:25.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd17eeb640 1 -- 172.21.15.132:0/397542435 >> v1:172.21.15.132:6791/0 conn(0x7efcf400b1b0 legacy=0x7efcf400d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:25.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd17eeb640 1 -- 172.21.15.132:0/397542435 shutdown_connections 2024-01-24T23:24:25.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd17eeb640 1 -- 172.21.15.132:0/397542435 >> 172.21.15.132:0/397542435 conn(0x7efd100a5a10 msgr2=0x7efd100a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:25.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd17eeb640 1 -- 172.21.15.132:0/397542435 shutdown_connections 2024-01-24T23:24:25.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd17eeb640 1 -- 172.21.15.132:0/397542435 wait complete. 2024-01-24T23:24:25.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd17eeb640 10 monclient: build_initial_monmap 2024-01-24T23:24:25.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.182+0000 7efd17eeb640 10 monclient: monmap: 2024-01-24T23:24:25.195 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:25.196 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:25.196 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:25.196 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:25.196 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:25.196 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:25.196 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:25.196 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:25.196 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:25.196 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:25.196 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:25.196 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:25.196 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:25.196 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:25.196 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:25.196 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:25.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd17eeb640 1 Processor -- start 2024-01-24T23:24:25.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd17eeb640 1 -- start start 2024-01-24T23:24:25.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd17eeb640 10 monclient: init 2024-01-24T23:24:25.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd17eeb640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:25.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd17eeb640 10 monclient: _add_conns ranks=[8,5,4,6,7,0,1,2,3] 2024-01-24T23:24:25.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd17eeb640 10 monclient(hunting): picked mon.noname-e con 0x7efcf400b1b0 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:25.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd17eeb640 10 monclient(hunting): picked mon.noname-h con 0x7efd1011eac0 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:25.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd17eeb640 10 monclient(hunting): picked mon.noname-c con 0x7efd101cf690 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:25.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd17eeb640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd10071960 con 0x7efd101cf690 2024-01-24T23:24:25.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd17eeb640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd10071680 con 0x7efd1011eac0 2024-01-24T23:24:25.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd17eeb640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd1011eed0 con 0x7efcf400b1b0 2024-01-24T23:24:25.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd17eeb640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:25.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd17eeb640 10 monclient(hunting): authenticate will time out at 1152.513355s 2024-01-24T23:24:25.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd15486640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7efd1011eac0 0x7efd101c9b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:48382/0 (socket says 172.21.15.132:48382) 2024-01-24T23:24:25.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd15486640 1 -- 172.21.15.132:0/275564746 learned_addr learned my addr 172.21.15.132:0/275564746 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:25.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd16488640 1 -- 172.21.15.132:0/275564746 >> v1:172.21.15.115:6791/0 conn(0x7efd101cf690 legacy=0x7efd101d1a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:25.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 1 -- 172.21.15.132:0/275564746 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4033369233 0 0) 0x7efd10071680 con 0x7efd1011eac0 2024-01-24T23:24:25.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:25.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:25.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:25.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 10 monclient(hunting): my global_id is 5549 2024-01-24T23:24:25.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 1 -- 172.21.15.132:0/275564746 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcf40053c0 con 0x7efd1011eac0 2024-01-24T23:24:25.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 1 -- 172.21.15.132:0/275564746 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142812432 0 0) 0x7efd1011eed0 con 0x7efcf400b1b0 2024-01-24T23:24:25.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:25.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:25.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:25.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 10 monclient(hunting): my global_id is 84743 2024-01-24T23:24:25.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 1 -- 172.21.15.132:0/275564746 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd10071680 con 0x7efcf400b1b0 2024-01-24T23:24:25.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 1 -- 172.21.15.132:0/275564746 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3486306519 0 0) 0x7efcf40053c0 con 0x7efd1011eac0 2024-01-24T23:24:25.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 1 -- 172.21.15.132:0/275564746 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efd1011eed0 con 0x7efd1011eac0 2024-01-24T23:24:25.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 1 -- 172.21.15.132:0/275564746 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7efd00004370 con 0x7efd1011eac0 2024-01-24T23:24:25.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:25.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:25.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 10 monclient(hunting): dump: 2024-01-24T23:24:25.199 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:25.199 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:25.199 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:25.199 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:25.199 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:25.199 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:25.199 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:25.199 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:25.199 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:25.199 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:25.199 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:25.200 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:25.200 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:25.200 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:25.200 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:25.200 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:25.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:25.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 1 -- 172.21.15.132:0/275564746 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3053944872 0 0) 0x7efd1011eed0 con 0x7efd1011eac0 2024-01-24T23:24:25.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 1 -- 172.21.15.132:0/275564746 >> v1:172.21.15.115:6793/0 conn(0x7efcf400b1b0 legacy=0x7efd101c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:25.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 1 -- 172.21.15.132:0/275564746 >> v1:172.21.15.115:6791/0 conn(0x7efd101cf690 legacy=0x7efd101d1a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:25.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 10 monclient: _finish_hunting 0 2024-01-24T23:24:25.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 1 monclient: found mon.h 2024-01-24T23:24:25.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:25.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:25.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 1 -- 172.21.15.132:0/275564746 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1011f690 con 0x7efd1011eac0 2024-01-24T23:24:25.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 10 monclient: _finish_auth 0 2024-01-24T23:24:25.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 10 monclient: _check_auth_tickets 2024-01-24T23:24:25.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:25.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd17eeb640 5 monclient: authenticate success, global_id 5549 2024-01-24T23:24:25.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd17eeb640 10 monclient: _renew_subs 2024-01-24T23:24:25.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd17eeb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:25.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd17eeb640 1 -- 172.21.15.132:0/275564746 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd101ca260 con 0x7efd1011eac0 2024-01-24T23:24:25.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd17eeb640 10 monclient: _renew_subs 2024-01-24T23:24:25.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd17eeb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:25.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd17eeb640 1 -- 172.21.15.132:0/275564746 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7efd10121710 con 0x7efd1011eac0 2024-01-24T23:24:25.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 1 -- 172.21.15.132:0/275564746 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efd000048d0 con 0x7efd1011eac0 2024-01-24T23:24:25.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:25.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 1 -- 172.21.15.132:0/275564746 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7efd00004e00 con 0x7efd1011eac0 2024-01-24T23:24:25.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:25.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:25.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd07fff640 4 set_mon_vals no callback set 2024-01-24T23:24:25.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 10 monclient: dump: 2024-01-24T23:24:25.203 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:25.203 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:25.203 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:25.203 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:25.203 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:25.203 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:25.203 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:25.203 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:25.204 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:25.204 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:25.204 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:25.204 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:25.204 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:25.204 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:25.204 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:25.204 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:25.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd17eeb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:25.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd17eeb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:25.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd17eeb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:25.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd17eeb640 1 -- 172.21.15.132:0/275564746 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf8005090 con 0x7efd1011eac0 2024-01-24T23:24:25.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.186+0000 7efd067fc640 1 -- 172.21.15.132:0/275564746 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7efd00004fe0 con 0x7efd1011eac0 2024-01-24T23:24:25.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.190+0000 7efd067fc640 1 -- 172.21.15.132:0/275564746 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(88..88 src has 1..88) v4 ==== 4719+0+0 (unknown 2633454479 0 0) 0x7efd00043be0 con 0x7efd1011eac0 2024-01-24T23:24:25.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.190+0000 7efd15c87640 10 monclient: get_auth_request con 0x7efcf40380f0 auth_method 0 2024-01-24T23:24:25.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.194+0000 7efd067fc640 1 -- 172.21.15.132:0/275564746 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7efd000148d0 con 0x7efd1011eac0 2024-01-24T23:24:25.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.194+0000 7efd067fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:25.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.194+0000 7efd067fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:25.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.358+0000 7efd17eeb640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}] 2024-01-24T23:24:25.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.358+0000 7efd17eeb640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}] 2024-01-24T23:24:25.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.358+0000 7efd17eeb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:25.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.358+0000 7efd17eeb640 1 -- 172.21.15.132:0/275564746 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "foo", "type": "host"} v 0) v1 -- 0x7efcf80058c0 con 0x7efd1011eac0 2024-01-24T23:24:25.969 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.966+0000 7efd067fc640 1 -- 172.21.15.132:0/275564746 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}]=0 added bucket foo type host to crush map v89) v1 ==== 138+0+0 (unknown 4207352514 0 0) 0x7efd00012f70 con 0x7efd1011eac0 2024-01-24T23:24:25.969 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.966+0000 7efd067fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}] 2024-01-24T23:24:25.969 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:25.966+0000 7efd067fc640 10 monclient: _finish_command 2 = system:0 added bucket foo type host to crush map 2024-01-24T23:24:26.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.066+0000 7efd17eeb640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}] 2024-01-24T23:24:26.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.066+0000 7efd17eeb640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}] 2024-01-24T23:24:26.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.066+0000 7efd17eeb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:26.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.066+0000 7efd17eeb640 1 -- 172.21.15.132:0/275564746 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "foo", "type": "host"} v 0) v1 -- 0x7efcf8000bc0 con 0x7efd1011eac0 2024-01-24T23:24:26.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.066+0000 7efd067fc640 1 -- 172.21.15.132:0/275564746 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}]=0 bucket 'foo' already exists v89) v1 ==== 126+0+0 (unknown 2767994862 0 0) 0x7efd000146f0 con 0x7efd1011eac0 2024-01-24T23:24:26.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.066+0000 7efd067fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}] 2024-01-24T23:24:26.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.066+0000 7efd067fc640 10 monclient: _finish_command 3 = system:0 bucket 'foo' already exists 2024-01-24T23:24:26.116 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'foo' already exists 2024-01-24T23:24:26.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.070+0000 7efd17eeb640 1 -- 172.21.15.132:0/275564746 >> v1:172.21.15.132:6812/1835401922 conn(0x7efcf40380f0 legacy=0x7efcf403a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:26.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.070+0000 7efd17eeb640 10 monclient: shutdown 2024-01-24T23:24:26.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.070+0000 7efd17eeb640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:26.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.070+0000 7efd17eeb640 1 -- 172.21.15.132:0/275564746 >> v1:172.21.15.132:6791/0 conn(0x7efd1011eac0 legacy=0x7efd101c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:26.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.070+0000 7efd17eeb640 1 -- 172.21.15.132:0/275564746 shutdown_connections 2024-01-24T23:24:26.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.070+0000 7efd17eeb640 1 -- 172.21.15.132:0/275564746 >> 172.21.15.132:0/275564746 conn(0x7efd100a5a10 msgr2=0x7efd100a3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:26.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.070+0000 7efd17eeb640 1 -- 172.21.15.132:0/275564746 shutdown_connections 2024-01-24T23:24:26.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.070+0000 7efd17eeb640 1 -- 172.21.15.132:0/275564746 wait complete. 2024-01-24T23:24:26.117 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush move foo root=default rack=localrack 2024-01-24T23:24:26.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.202+0000 7fa2cbfa4640 10 monclient: get_monmap_and_config 2024-01-24T23:24:26.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.202+0000 7fa2cbfa4640 10 monclient: build_initial_monmap 2024-01-24T23:24:26.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 10 monclient: monmap: 2024-01-24T23:24:26.208 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:26.208 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:26.208 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:26.208840+0000 2024-01-24T23:24:26.208 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:26.208840+0000 2024-01-24T23:24:26.209 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:26.209 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:26.209 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:26.209 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:26.209 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:26.209 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:26.209 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:26.209 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:26.209 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:26.209 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:26.209 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:26.209 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:26.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 1 Processor -- start 2024-01-24T23:24:26.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 1 -- start start 2024-01-24T23:24:26.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 10 monclient: init 2024-01-24T23:24:26.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:26.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 10 monclient: _add_conns ranks=[7,4,3,6,1,5,0,8,2] 2024-01-24T23:24:26.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 10 monclient(hunting): picked mon.noname-i con 0x7fa2c40a1b50 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:26.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 10 monclient(hunting): picked mon.noname-c con 0x7fa2c409ed00 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:26.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 10 monclient(hunting): picked mon.noname-g con 0x7fa2c40a2640 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:26.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c4064700 con 0x7fa2c40a2640 2024-01-24T23:24:26.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c409ddf0 con 0x7fa2c409ed00 2024-01-24T23:24:26.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c406b050 con 0x7fa2c40a1b50 2024-01-24T23:24:26.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:26.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 10 monclient(hunting): authenticate will time out at 1153.533721s 2024-01-24T23:24:26.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2ca541640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fa2c40a2640 0x7fa2c413fa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:43940/0 (socket says 172.21.15.132:43940) 2024-01-24T23:24:26.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2ca541640 1 -- 172.21.15.132:0/2145747367 learned_addr learned my addr 172.21.15.132:0/2145747367 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:26.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c953f640 1 -- 172.21.15.132:0/2145747367 >> v1:172.21.15.115:6791/0 conn(0x7fa2c409ed00 legacy=0x7fa2c413c2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:26.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 1 -- 172.21.15.132:0/2145747367 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 586771717 0 0) 0x7fa2c4064700 con 0x7fa2c40a2640 2024-01-24T23:24:26.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 10 monclient(hunting): _negotiate 2024-01-24T23:24:26.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:26.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:26.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 10 monclient(hunting): my global_id is 5547 2024-01-24T23:24:26.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 1 -- 172.21.15.132:0/2145747367 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2b0003530 con 0x7fa2c40a2640 2024-01-24T23:24:26.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 1 -- 172.21.15.132:0/2145747367 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2129356342 0 0) 0x7fa2c406b050 con 0x7fa2c40a1b50 2024-01-24T23:24:26.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 10 monclient(hunting): _negotiate 2024-01-24T23:24:26.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:26.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:26.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 10 monclient(hunting): my global_id is 5587 2024-01-24T23:24:26.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 1 -- 172.21.15.132:0/2145747367 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2c4064700 con 0x7fa2c40a1b50 2024-01-24T23:24:26.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 1 -- 172.21.15.132:0/2145747367 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1101240039 0 0) 0x7fa2c4064700 con 0x7fa2c40a1b50 2024-01-24T23:24:26.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 1 -- 172.21.15.132:0/2145747367 >> v1:172.21.15.115:6791/0 conn(0x7fa2c409ed00 legacy=0x7fa2c413c2f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:26.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 1 -- 172.21.15.132:0/2145747367 >> v1:172.21.15.132:6790/0 conn(0x7fa2c40a2640 legacy=0x7fa2c413fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:26.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 10 monclient: _finish_hunting 0 2024-01-24T23:24:26.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 1 monclient: found mon.noname-i 2024-01-24T23:24:26.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:26.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:24:26.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 1 -- 172.21.15.132:0/2145747367 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c40a7000 con 0x7fa2c40a1b50 2024-01-24T23:24:26.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 10 monclient: _finish_auth 0 2024-01-24T23:24:26.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 10 monclient: _check_auth_tickets 2024-01-24T23:24:26.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:26.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 5 monclient: authenticate success, global_id 5587 2024-01-24T23:24:26.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:26.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 1 -- 172.21.15.132:0/2145747367 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa2bc001b20 con 0x7fa2c40a1b50 2024-01-24T23:24:26.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:26.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:26.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 10 monclient: dump: 2024-01-24T23:24:26.219 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:26.219 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:26.219 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:26.219 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:26.219 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:26.219 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:26.219 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:26.220 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:26.220 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:26.220 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:26.220 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:26.220 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:26.220 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:26.220 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:26.220 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:26.220 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:26.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:26.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 1 -- 172.21.15.132:0/2145747367 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa2bc001dd0 con 0x7fa2c40a1b50 2024-01-24T23:24:26.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:26.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:26.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 1 -- 172.21.15.132:0/2145747367 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa2bc003ce0 con 0x7fa2c40a1b50 2024-01-24T23:24:26.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 4 set_mon_vals no callback set 2024-01-24T23:24:26.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:26.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:26.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2c8d3e640 10 monclient: dump: 2024-01-24T23:24:26.221 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:26.221 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:26.221 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:26.221 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:26.221 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:26.221 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:26.223 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:26.223 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:26.223 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:26.223 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:26.223 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:26.223 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:26.223 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:26.223 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:26.223 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:26.223 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:26.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 10 monclient: shutdown 2024-01-24T23:24:26.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:26.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 1 -- 172.21.15.132:0/2145747367 >> v1:172.21.15.132:6792/0 conn(0x7fa2c40a1b50 legacy=0x7fa2c40a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:26.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 1 -- 172.21.15.132:0/2145747367 shutdown_connections 2024-01-24T23:24:26.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 1 -- 172.21.15.132:0/2145747367 >> 172.21.15.132:0/2145747367 conn(0x7fa2c40a2c40 msgr2=0x7fa2c40a3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:26.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 1 -- 172.21.15.132:0/2145747367 shutdown_connections 2024-01-24T23:24:26.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.206+0000 7fa2cbfa4640 1 -- 172.21.15.132:0/2145747367 wait complete. 2024-01-24T23:24:26.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 10 monclient: build_initial_monmap 2024-01-24T23:24:26.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 10 monclient: monmap: 2024-01-24T23:24:26.224 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:26.224 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:26.224 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:26.224 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:26.224 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:26.224 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:26.224 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:26.224 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:26.225 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:26.225 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:26.225 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:26.225 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:26.225 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:26.225 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:26.225 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:26.225 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:26.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 1 Processor -- start 2024-01-24T23:24:26.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 1 -- start start 2024-01-24T23:24:26.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 10 monclient: init 2024-01-24T23:24:26.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:26.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 10 monclient: _add_conns ranks=[1,8,5,4,7,6,0,3,2] 2024-01-24T23:24:26.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 10 monclient(hunting): picked mon.noname-f con 0x7fa2c409ed00 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:26.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 10 monclient(hunting): picked mon.noname-e con 0x7fa2c40a1b50 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:26.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 10 monclient(hunting): picked mon.noname-h con 0x7fa2c40a2640 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:26.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c406b270 con 0x7fa2c409ed00 2024-01-24T23:24:26.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c4091350 con 0x7fa2c40a2640 2024-01-24T23:24:26.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c406ad20 con 0x7fa2c40a1b50 2024-01-24T23:24:26.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:26.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2c9d40640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fa2c409ed00 legacy=0x7fa2c4098bc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:26.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 10 monclient(hunting): authenticate will time out at 1153.537574s 2024-01-24T23:24:26.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2ca541640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fa2c40a2640 0x7fa2c41d3b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:48410/0 (socket says 172.21.15.132:48410) 2024-01-24T23:24:26.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2ca541640 1 -- 172.21.15.132:0/171731196 learned_addr learned my addr 172.21.15.132:0/171731196 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:26.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 1 -- 172.21.15.132:0/171731196 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604802064 0 0) 0x7fa2c4091350 con 0x7fa2c40a2640 2024-01-24T23:24:26.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:26.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:26.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:26.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 10 monclient(hunting): my global_id is 5558 2024-01-24T23:24:26.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 1 -- 172.21.15.132:0/171731196 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2a8003530 con 0x7fa2c40a2640 2024-01-24T23:24:26.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 1 -- 172.21.15.132:0/171731196 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2997026077 0 0) 0x7fa2a8003530 con 0x7fa2c40a2640 2024-01-24T23:24:26.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 1 -- 172.21.15.132:0/171731196 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa2c4091350 con 0x7fa2c40a2640 2024-01-24T23:24:26.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 1 -- 172.21.15.132:0/171731196 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa2c0001900 con 0x7fa2c40a2640 2024-01-24T23:24:26.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:26.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:26.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 10 monclient(hunting): dump: 2024-01-24T23:24:26.227 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:26.227 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:26.227 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:26.227 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:26.227 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:26.227 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:26.227 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:26.227 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:26.227 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:26.228 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:26.228 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:26.228 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:26.228 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:26.228 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:26.228 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:26.228 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:26.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:26.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 1 -- 172.21.15.132:0/171731196 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4065462930 0 0) 0x7fa2c406ad20 con 0x7fa2c40a1b50 2024-01-24T23:24:26.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:26.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:26.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:26.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 10 monclient(hunting): my global_id is 84752 2024-01-24T23:24:26.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 1 -- 172.21.15.132:0/171731196 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2a8003530 con 0x7fa2c40a1b50 2024-01-24T23:24:26.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 1 -- 172.21.15.132:0/171731196 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2239616185 0 0) 0x7fa2c4091350 con 0x7fa2c40a2640 2024-01-24T23:24:26.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 1 -- 172.21.15.132:0/171731196 >> v1:172.21.15.115:6793/0 conn(0x7fa2c40a1b50 legacy=0x7fa2c41d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:26.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 1 -- 172.21.15.132:0/171731196 >> v1:172.21.15.132:6789/0 conn(0x7fa2c409ed00 legacy=0x7fa2c4098bc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:26.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 10 monclient: _finish_hunting 0 2024-01-24T23:24:26.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 1 monclient: found mon.h 2024-01-24T23:24:26.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:26.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:26.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 1 -- 172.21.15.132:0/171731196 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c4091070 con 0x7fa2c40a2640 2024-01-24T23:24:26.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 10 monclient: _finish_auth 0 2024-01-24T23:24:26.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 10 monclient: _check_auth_tickets 2024-01-24T23:24:26.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:26.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 5 monclient: authenticate success, global_id 5558 2024-01-24T23:24:26.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 10 monclient: _renew_subs 2024-01-24T23:24:26.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:26.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 1 -- 172.21.15.132:0/171731196 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c40992d0 con 0x7fa2c40a2640 2024-01-24T23:24:26.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 10 monclient: _renew_subs 2024-01-24T23:24:26.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:26.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 1 -- 172.21.15.132:0/171731196 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c4072540 con 0x7fa2c40a2640 2024-01-24T23:24:26.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 1 -- 172.21.15.132:0/171731196 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa2c0001cf0 con 0x7fa2c40a2640 2024-01-24T23:24:26.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:26.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 1 -- 172.21.15.132:0/171731196 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa2c0004de0 con 0x7fa2c40a2640 2024-01-24T23:24:26.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:26.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b7fff640 4 set_mon_vals no callback set 2024-01-24T23:24:26.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:26.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 10 monclient: dump: 2024-01-24T23:24:26.230 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:26.230 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:26.230 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:26.230 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:26.230 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:26.230 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:26.230 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:26.231 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:26.231 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:26.231 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:26.231 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:26.231 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:26.231 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:26.231 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:26.231 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:26.231 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:26.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:26.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:26.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:26.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 1 -- 172.21.15.132:0/171731196 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fa2c000f860 con 0x7fa2c40a2640 2024-01-24T23:24:26.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2cbfa4640 1 -- 172.21.15.132:0/171731196 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ac005090 con 0x7fa2c40a2640 2024-01-24T23:24:26.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.210+0000 7fa2b67fc640 1 -- 172.21.15.132:0/171731196 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(89..89 src has 1..89) v4 ==== 4746+0+0 (unknown 1329781277 0 0) 0x7fa2c0043ae0 con 0x7fa2c40a2640 2024-01-24T23:24:26.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.214+0000 7fa2c9d40640 10 monclient: get_auth_request con 0x7fa2a8036fc0 auth_method 0 2024-01-24T23:24:26.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.218+0000 7fa2b67fc640 1 -- 172.21.15.132:0/171731196 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fa2c0010db0 con 0x7fa2c40a2640 2024-01-24T23:24:26.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.218+0000 7fa2b67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:26.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.218+0000 7fa2b67fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:26.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.374+0000 7fa2cbfa4640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}] 2024-01-24T23:24:26.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.374+0000 7fa2cbfa4640 10 monclient: _send_command 2 [{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}] 2024-01-24T23:24:26.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.374+0000 7fa2cbfa4640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:26.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.374+0000 7fa2cbfa4640 1 -- 172.21.15.132:0/171731196 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]} v 0) v1 -- 0x7fa2ac0058c0 con 0x7fa2c40a2640 2024-01-24T23:24:26.976 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.974+0000 7fa2b67fc640 1 -- 172.21.15.132:0/171731196 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}]=0 moved item id -10 name 'foo' to location {rack=localrack,root=default} in crush map v90) v1 ==== 204+0+0 (unknown 2696320998 0 0) 0x7fa2c0012e50 con 0x7fa2c40a2640 2024-01-24T23:24:26.976 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.974+0000 7fa2b67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}] 2024-01-24T23:24:26.976 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:26.974+0000 7fa2b67fc640 10 monclient: _finish_command 2 = system:0 moved item id -10 name 'foo' to location {rack=localrack,root=default} in crush map 2024-01-24T23:24:27.075 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.070+0000 7fa2cbfa4640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}] 2024-01-24T23:24:27.075 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.070+0000 7fa2cbfa4640 10 monclient: _send_command 3 [{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}] 2024-01-24T23:24:27.075 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.070+0000 7fa2cbfa4640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:27.075 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.070+0000 7fa2cbfa4640 1 -- 172.21.15.132:0/171731196 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]} v 0) v1 -- 0x7fa2ac002220 con 0x7fa2c40a2640 2024-01-24T23:24:27.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.074+0000 7fa2b67fc640 1 -- 172.21.15.132:0/171731196 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}]=0 no need to move item id -10 name 'foo' to location {rack=localrack,root=default} in crush map v90) v1 ==== 214+0+0 (unknown 1790796560 0 0) 0x7fa2c00145d0 con 0x7fa2c40a2640 2024-01-24T23:24:27.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.074+0000 7fa2b67fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}] 2024-01-24T23:24:27.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.074+0000 7fa2b67fc640 10 monclient: _finish_command 3 = system:0 no need to move item id -10 name 'foo' to location {rack=localrack,root=default} in crush map 2024-01-24T23:24:27.076 INFO:tasks.workunit.client.0.smithi132.stderr:no need to move item id -10 name 'foo' to location {rack=localrack,root=default} in crush map 2024-01-24T23:24:27.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.074+0000 7fa2cbfa4640 1 -- 172.21.15.132:0/171731196 >> v1:172.21.15.132:6812/1835401922 conn(0x7fa2a8036fc0 legacy=0x7fa2a8039480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:27.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.074+0000 7fa2cbfa4640 10 monclient: shutdown 2024-01-24T23:24:27.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.074+0000 7fa2cbfa4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:27.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.074+0000 7fa2cbfa4640 1 -- 172.21.15.132:0/171731196 >> v1:172.21.15.132:6791/0 conn(0x7fa2c40a2640 legacy=0x7fa2c41d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:27.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.074+0000 7fa2cbfa4640 1 -- 172.21.15.132:0/171731196 shutdown_connections 2024-01-24T23:24:27.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.074+0000 7fa2cbfa4640 1 -- 172.21.15.132:0/171731196 >> 172.21.15.132:0/171731196 conn(0x7fa2c40a2c40 msgr2=0x7fa2c41be530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:27.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.074+0000 7fa2cbfa4640 1 -- 172.21.15.132:0/171731196 shutdown_connections 2024-01-24T23:24:27.079 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.074+0000 7fa2cbfa4640 1 -- 172.21.15.132:0/171731196 wait complete. 2024-01-24T23:24:27.093 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush create-or-move osd.7 1.0 root=default 2024-01-24T23:24:27.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f2696417640 10 monclient: get_monmap_and_config 2024-01-24T23:24:27.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f2696417640 10 monclient: build_initial_monmap 2024-01-24T23:24:27.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f2696417640 10 monclient: monmap: 2024-01-24T23:24:27.212 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:27.212 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:27.212 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:27.213044+0000 2024-01-24T23:24:27.212 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:27.213044+0000 2024-01-24T23:24:27.212 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:27.212 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:27.212 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:27.213 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:27.213 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:27.213 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:27.213 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:27.213 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:27.213 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:27.213 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:27.213 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:27.213 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:27.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f2696417640 1 Processor -- start 2024-01-24T23:24:27.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f2696417640 1 -- start start 2024-01-24T23:24:27.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f2696417640 10 monclient: init 2024-01-24T23:24:27.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f2696417640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:27.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f2696417640 10 monclient: _add_conns ranks=[2,0,6,1,5,8,4,7,3] 2024-01-24T23:24:27.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f2696417640 10 monclient(hunting): picked mon.noname-b con 0x7f26900a4550 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:27.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f2696417640 10 monclient(hunting): picked mon.noname-a con 0x7f26900a10d0 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:27.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f2696417640 10 monclient(hunting): picked mon.noname-d con 0x7f269013d350 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:27.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f2696417640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2690064700 con 0x7f26900a10d0 2024-01-24T23:24:27.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f2696417640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f269006c300 con 0x7f26900a4550 2024-01-24T23:24:27.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f2696417640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26900a4930 con 0x7f269013d350 2024-01-24T23:24:27.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f2696417640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:27.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f2696417640 10 monclient(hunting): authenticate will time out at 1154.537721s 2024-01-24T23:24:27.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268f7fe640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f26900a10d0 legacy=0x7f269013bc30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:27.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268ffff640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f26900a4550 0x7f269009da00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:43006/0 (socket says 172.21.15.132:43006) 2024-01-24T23:24:27.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f26949b4640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f269013d350 0x7f269013f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:49496/0 (socket says 172.21.15.132:49496) 2024-01-24T23:24:27.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268ffff640 1 -- 172.21.15.132:0/779385441 learned_addr learned my addr 172.21.15.132:0/779385441 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:27.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 1 -- 172.21.15.132:0/779385441 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3926658232 0 0) 0x7f269006c300 con 0x7f26900a4550 2024-01-24T23:24:27.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:27.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:27.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:27.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 10 monclient(hunting): my global_id is 84836 2024-01-24T23:24:27.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 1 -- 172.21.15.132:0/779385441 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2670003530 con 0x7f26900a4550 2024-01-24T23:24:27.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 1 -- 172.21.15.132:0/779385441 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 912622237 0 0) 0x7f26900a4930 con 0x7f269013d350 2024-01-24T23:24:27.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:27.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:27.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:27.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 10 monclient(hunting): my global_id is 5577 2024-01-24T23:24:27.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 1 -- 172.21.15.132:0/779385441 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f269006c300 con 0x7f269013d350 2024-01-24T23:24:27.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 1 -- 172.21.15.132:0/779385441 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4268724078 0 0) 0x7f269006c300 con 0x7f269013d350 2024-01-24T23:24:27.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 1 -- 172.21.15.132:0/779385441 >> v1:172.21.15.115:6790/0 conn(0x7f26900a4550 legacy=0x7f269009da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:27.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 1 -- 172.21.15.132:0/779385441 >> v1:172.21.15.115:6789/0 conn(0x7f26900a10d0 legacy=0x7f269013bc30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:27.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:27.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 1 monclient: found mon.noname-d 2024-01-24T23:24:27.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:27.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:24:27.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 1 -- 172.21.15.132:0/779385441 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269009e550 con 0x7f269013d350 2024-01-24T23:24:27.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 10 monclient: _finish_auth 0 2024-01-24T23:24:27.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:27.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:27.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f2696417640 5 monclient: authenticate success, global_id 5577 2024-01-24T23:24:27.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 1 -- 172.21.15.132:0/779385441 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f26880016d0 con 0x7f269013d350 2024-01-24T23:24:27.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f2696417640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:27.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:27.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:24:27.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 10 monclient: dump: 2024-01-24T23:24:27.224 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:27.224 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:27.224 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:27.224 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:27.224 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:27.224 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:27.224 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:27.224 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:27.224 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:27.225 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:27.225 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:27.225 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:27.225 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:27.225 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:27.225 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:27.225 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:27.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f2696417640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:27.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 1 -- 172.21.15.132:0/779385441 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f26880018f0 con 0x7f269013d350 2024-01-24T23:24:27.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:27.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f2696417640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:27.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f2696417640 4 set_mon_vals no callback set 2024-01-24T23:24:27.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 1 -- 172.21.15.132:0/779385441 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f26880047f0 con 0x7f269013d350 2024-01-24T23:24:27.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:27.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:24:27.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.210+0000 7f268effd640 10 monclient: dump: 2024-01-24T23:24:27.226 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:27.226 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:27.226 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:27.226 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:27.226 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:27.226 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:27.226 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:27.226 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:27.226 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:27.226 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:27.226 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:27.226 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:27.226 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:27.226 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:27.227 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:27.228 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:27.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 10 monclient: shutdown 2024-01-24T23:24:27.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:27.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 1 -- 172.21.15.132:0/779385441 >> v1:172.21.15.115:6792/0 conn(0x7f269013d350 legacy=0x7f269013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:27.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 1 -- 172.21.15.132:0/779385441 shutdown_connections 2024-01-24T23:24:27.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 1 -- 172.21.15.132:0/779385441 >> 172.21.15.132:0/779385441 conn(0x7f26900a7990 msgr2=0x7f26900a3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:27.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 1 -- 172.21.15.132:0/779385441 shutdown_connections 2024-01-24T23:24:27.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 1 -- 172.21.15.132:0/779385441 wait complete. 2024-01-24T23:24:27.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 10 monclient: build_initial_monmap 2024-01-24T23:24:27.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 10 monclient: monmap: 2024-01-24T23:24:27.228 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:27.228 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:27.228 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:27.229 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:27.229 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:27.229 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:27.229 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:27.229 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:27.229 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:27.229 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:27.229 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:27.229 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:27.229 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:27.229 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:27.229 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:27.229 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:27.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 1 Processor -- start 2024-01-24T23:24:27.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 1 -- start start 2024-01-24T23:24:27.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 10 monclient: init 2024-01-24T23:24:27.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:27.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 10 monclient: _add_conns ranks=[2,3,1,7,8,0,5,6,4] 2024-01-24T23:24:27.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 10 monclient(hunting): picked mon.noname-b con 0x7f26900a10d0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:27.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 10 monclient(hunting): picked mon.noname-g con 0x7f26900a4550 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:27.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 10 monclient(hunting): picked mon.noname-f con 0x7f269013d350 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:27.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f269006c7f0 con 0x7f269013d350 2024-01-24T23:24:27.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f269006fe60 con 0x7f26900a10d0 2024-01-24T23:24:27.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2690070470 con 0x7f26900a4550 2024-01-24T23:24:27.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:27.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 10 monclient(hunting): authenticate will time out at 1154.541589s 2024-01-24T23:24:27.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f26949b4640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f269013d350 legacy=0x7f26901d3730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:27.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268f7fe640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f26900a4550 0x7f269006f750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:43948/0 (socket says 172.21.15.132:43948) 2024-01-24T23:24:27.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268f7fe640 1 -- 172.21.15.132:0/4282312377 learned_addr learned my addr 172.21.15.132:0/4282312377 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:27.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 1 -- 172.21.15.132:0/4282312377 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 510486930 0 0) 0x7f2690070470 con 0x7f26900a4550 2024-01-24T23:24:27.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:24:27.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:27.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:27.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 10 monclient(hunting): my global_id is 5556 2024-01-24T23:24:27.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 1 -- 172.21.15.132:0/4282312377 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2674003530 con 0x7f26900a4550 2024-01-24T23:24:27.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 1 -- 172.21.15.132:0/4282312377 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 534660803 0 0) 0x7f2674003530 con 0x7f26900a4550 2024-01-24T23:24:27.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 1 -- 172.21.15.132:0/4282312377 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2690070470 con 0x7f26900a4550 2024-01-24T23:24:27.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 1 -- 172.21.15.132:0/4282312377 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f26780046b0 con 0x7f26900a4550 2024-01-24T23:24:27.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:27.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:27.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 10 monclient(hunting): dump: 2024-01-24T23:24:27.231 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:27.231 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:27.231 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:27.231 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:27.232 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:27.232 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:27.232 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:27.232 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:27.232 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:27.232 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:27.232 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:27.232 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:27.232 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:27.232 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:27.232 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:27.232 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:27.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:27.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 1 -- 172.21.15.132:0/4282312377 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1077435624 0 0) 0x7f269006fe60 con 0x7f26900a10d0 2024-01-24T23:24:27.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:24:27.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:27.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:27.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 10 monclient(hunting): my global_id is 84845 2024-01-24T23:24:27.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 1 -- 172.21.15.132:0/4282312377 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2674003530 con 0x7f26900a10d0 2024-01-24T23:24:27.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 1 -- 172.21.15.132:0/4282312377 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1649561147 0 0) 0x7f2690070470 con 0x7f26900a4550 2024-01-24T23:24:27.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 1 -- 172.21.15.132:0/4282312377 >> v1:172.21.15.115:6790/0 conn(0x7f26900a10d0 legacy=0x7f269006f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:27.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 1 -- 172.21.15.132:0/4282312377 >> v1:172.21.15.132:6789/0 conn(0x7f269013d350 legacy=0x7f26901d3730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:27.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 10 monclient: _finish_hunting 0 2024-01-24T23:24:27.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 1 monclient: found mon.g 2024-01-24T23:24:27.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:27.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:27.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 1 -- 172.21.15.132:0/4282312377 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2690072330 con 0x7f26900a4550 2024-01-24T23:24:27.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 10 monclient: _finish_auth 0 2024-01-24T23:24:27.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 10 monclient: _check_auth_tickets 2024-01-24T23:24:27.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:27.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 5 monclient: authenticate success, global_id 5556 2024-01-24T23:24:27.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 10 monclient: _renew_subs 2024-01-24T23:24:27.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:27.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 1 -- 172.21.15.132:0/4282312377 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f269009cf80 con 0x7f26900a4550 2024-01-24T23:24:27.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 10 monclient: _renew_subs 2024-01-24T23:24:27.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:27.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 1 -- 172.21.15.132:0/4282312377 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f269009c620 con 0x7f26900a4550 2024-01-24T23:24:27.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 1 -- 172.21.15.132:0/4282312377 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2678004960 con 0x7f26900a4550 2024-01-24T23:24:27.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:27.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 1 -- 172.21.15.132:0/4282312377 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2678004e90 con 0x7f26900a4550 2024-01-24T23:24:27.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:27.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268e7fc640 4 set_mon_vals no callback set 2024-01-24T23:24:27.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:27.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 10 monclient: dump: 2024-01-24T23:24:27.235 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:27.235 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:27.235 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:27.235 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:27.235 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:27.235 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:27.235 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:27.235 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:27.235 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:27.235 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:27.235 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:27.235 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:27.235 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:27.235 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:27.235 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:27.236 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:27.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:27.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:27.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:27.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f2696417640 1 -- 172.21.15.132:0/4282312377 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2668005090 con 0x7f26900a4550 2024-01-24T23:24:27.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 1 -- 172.21.15.132:0/4282312377 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f2678005070 con 0x7f26900a4550 2024-01-24T23:24:27.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268cff9640 1 -- 172.21.15.132:0/4282312377 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(90..90 src has 1..90) v4 ==== 5043+0+0 (unknown 3866037567 0 0) 0x7f26780441e0 con 0x7f26900a4550 2024-01-24T23:24:27.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.214+0000 7f268ffff640 10 monclient: get_auth_request con 0x7f2674036dd0 auth_method 0 2024-01-24T23:24:27.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.222+0000 7f268cff9640 1 -- 172.21.15.132:0/4282312377 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f2678014ef0 con 0x7f26900a4550 2024-01-24T23:24:27.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.222+0000 7f268cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:27.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.222+0000 7f268cff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:27.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.382+0000 7f2696417640 10 start_mon_command cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}] 2024-01-24T23:24:27.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.382+0000 7f2696417640 10 monclient: _send_command 2 [{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}] 2024-01-24T23:24:27.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.382+0000 7f2696417640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:27.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.382+0000 7f2696417640 1 -- 172.21.15.132:0/4282312377 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]} v 0) v1 -- 0x7f26680058c0 con 0x7f26900a4550 2024-01-24T23:24:27.990 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.986+0000 7f268cff9640 1 -- 172.21.15.132:0/4282312377 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}]=0 create-or-move updating item name 'osd.7' weight 1 at location {root=default} to crush map v91) v1 ==== 212+0+0 (unknown 3672871208 0 0) 0x7f2678013360 con 0x7f26900a4550 2024-01-24T23:24:27.990 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.986+0000 7f268cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}] 2024-01-24T23:24:27.990 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:27.986+0000 7f268cff9640 10 monclient: _finish_command 2 = system:0 create-or-move updating item name 'osd.7' weight 1 at location {root=default} to crush map 2024-01-24T23:24:28.093 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.090+0000 7f2696417640 10 start_mon_command cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}] 2024-01-24T23:24:28.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.090+0000 7f2696417640 10 monclient: _send_command 3 [{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}] 2024-01-24T23:24:28.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.090+0000 7f2696417640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:28.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.090+0000 7f2696417640 1 -- 172.21.15.132:0/4282312377 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]} v 0) v1 -- 0x7f2668005340 con 0x7f26900a4550 2024-01-24T23:24:28.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.090+0000 7f268cff9640 1 -- 172.21.15.132:0/4282312377 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}]=0 create-or-move updated item name 'osd.7' weight 1 at location {root=default} to crush map v91) v1 ==== 211+0+0 (unknown 2608464520 0 0) 0x7f2678014ae0 con 0x7f26900a4550 2024-01-24T23:24:28.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.090+0000 7f268cff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}] 2024-01-24T23:24:28.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.090+0000 7f268cff9640 10 monclient: _finish_command 3 = system:0 create-or-move updated item name 'osd.7' weight 1 at location {root=default} to crush map 2024-01-24T23:24:28.095 INFO:tasks.workunit.client.0.smithi132.stderr:create-or-move updated item name 'osd.7' weight 1 at location {root=default} to crush map 2024-01-24T23:24:28.096 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.094+0000 7f2696417640 1 -- 172.21.15.132:0/4282312377 >> v1:172.21.15.132:6812/1835401922 conn(0x7f2674036dd0 legacy=0x7f2674039290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:28.097 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.094+0000 7f2696417640 10 monclient: shutdown 2024-01-24T23:24:28.097 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.094+0000 7f2696417640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:28.097 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.094+0000 7f2696417640 1 -- 172.21.15.132:0/4282312377 >> v1:172.21.15.132:6790/0 conn(0x7f26900a4550 legacy=0x7f269006f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:28.097 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.094+0000 7f2696417640 1 -- 172.21.15.132:0/4282312377 shutdown_connections 2024-01-24T23:24:28.097 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.094+0000 7f2696417640 1 -- 172.21.15.132:0/4282312377 >> 172.21.15.132:0/4282312377 conn(0x7f26900a7990 msgr2=0x7f26901350a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:28.097 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.094+0000 7f2696417640 1 -- 172.21.15.132:0/4282312377 shutdown_connections 2024-01-24T23:24:28.097 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.094+0000 7f2696417640 1 -- 172.21.15.132:0/4282312377 wait complete. 2024-01-24T23:24:28.111 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush move osd.7 host=foo 2024-01-24T23:24:28.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.222+0000 7f12b3be1640 10 monclient: get_monmap_and_config 2024-01-24T23:24:28.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.222+0000 7f12b3be1640 10 monclient: build_initial_monmap 2024-01-24T23:24:28.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.222+0000 7f12b3be1640 10 monclient: monmap: 2024-01-24T23:24:28.226 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:28.226 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:28.227 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:28.227588+0000 2024-01-24T23:24:28.227 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:28.227588+0000 2024-01-24T23:24:28.227 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:28.227 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:28.227 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:28.227 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:28.227 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:28.227 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:28.227 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:28.227 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:28.227 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:28.227 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:28.227 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:28.227 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:28.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.222+0000 7f12b3be1640 1 Processor -- start 2024-01-24T23:24:28.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.222+0000 7f12b3be1640 1 -- start start 2024-01-24T23:24:28.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.222+0000 7f12b3be1640 10 monclient: init 2024-01-24T23:24:28.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:28.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 10 monclient: _add_conns ranks=[0,7,5,1,3,8,4,6,2] 2024-01-24T23:24:28.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 10 monclient(hunting): picked mon.noname-a con 0x7f12ac09ff20 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:28.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 10 monclient(hunting): picked mon.noname-i con 0x7f12ac09f9c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:28.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 10 monclient(hunting): picked mon.noname-h con 0x7f12ac13b2c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:28.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12ac064700 con 0x7f12ac09ff20 2024-01-24T23:24:28.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12ac09e760 con 0x7f12ac13b2c0 2024-01-24T23:24:28.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12ac0a1160 con 0x7f12ac09f9c0 2024-01-24T23:24:28.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:28.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 10 monclient(hunting): authenticate will time out at 1155.552204s 2024-01-24T23:24:28.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b117c640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f12ac09f9c0 0x7f12ac139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:57436/0 (socket says 172.21.15.132:57436) 2024-01-24T23:24:28.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b197d640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f12ac09ff20 legacy=0x7f12ac0a0300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:28.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b117c640 1 -- 172.21.15.132:0/2248339816 learned_addr learned my addr 172.21.15.132:0/2248339816 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:28.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 1 -- 172.21.15.132:0/2248339816 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3081630306 0 0) 0x7f12ac09e760 con 0x7f12ac13b2c0 2024-01-24T23:24:28.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 10 monclient(hunting): _negotiate 2024-01-24T23:24:28.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:28.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:28.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 10 monclient(hunting): my global_id is 5567 2024-01-24T23:24:28.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 1 -- 172.21.15.132:0/2248339816 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1298003530 con 0x7f12ac13b2c0 2024-01-24T23:24:28.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 1 -- 172.21.15.132:0/2248339816 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3703823847 0 0) 0x7f12ac0a1160 con 0x7f12ac09f9c0 2024-01-24T23:24:28.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 10 monclient(hunting): _negotiate 2024-01-24T23:24:28.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:28.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:28.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 10 monclient(hunting): my global_id is 5596 2024-01-24T23:24:28.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 1 -- 172.21.15.132:0/2248339816 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12ac09e760 con 0x7f12ac09f9c0 2024-01-24T23:24:28.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 1 -- 172.21.15.132:0/2248339816 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1222076083 0 0) 0x7f1298003530 con 0x7f12ac13b2c0 2024-01-24T23:24:28.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 1 -- 172.21.15.132:0/2248339816 >> v1:172.21.15.132:6792/0 conn(0x7f12ac09f9c0 legacy=0x7f12ac139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:28.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 1 -- 172.21.15.132:0/2248339816 >> v1:172.21.15.115:6789/0 conn(0x7f12ac09ff20 legacy=0x7f12ac0a0300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:28.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 10 monclient: _finish_hunting 0 2024-01-24T23:24:28.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 1 monclient: found mon.noname-h 2024-01-24T23:24:28.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:28.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:24:28.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 1 -- 172.21.15.132:0/2248339816 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ac0a0a10 con 0x7f12ac13b2c0 2024-01-24T23:24:28.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 10 monclient: _finish_auth 0 2024-01-24T23:24:28.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 10 monclient: _check_auth_tickets 2024-01-24T23:24:28.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:28.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 1 -- 172.21.15.132:0/2248339816 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f12a4001b40 con 0x7f12ac13b2c0 2024-01-24T23:24:28.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:28.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:28.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 10 monclient: dump: 2024-01-24T23:24:28.237 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:28.237 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:28.237 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:28.237 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:28.237 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:28.238 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:28.238 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:28.238 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:28.238 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:28.238 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:28.238 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:28.238 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:28.238 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:28.238 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:28.238 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:28.238 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:28.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 5 monclient: authenticate success, global_id 5567 2024-01-24T23:24:28.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:28.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 1 -- 172.21.15.132:0/2248339816 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f12a4001da0 con 0x7f12ac13b2c0 2024-01-24T23:24:28.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:28.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:28.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 4 set_mon_vals no callback set 2024-01-24T23:24:28.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 1 -- 172.21.15.132:0/2248339816 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f12a4003ce0 con 0x7f12ac13b2c0 2024-01-24T23:24:28.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:28.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:28.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b097b640 10 monclient: dump: 2024-01-24T23:24:28.239 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:28.239 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:28.239 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:28.239 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:28.239 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:28.239 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:28.239 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:28.239 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:28.240 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:28.240 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:28.240 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:28.240 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:28.240 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:28.240 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:28.240 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:28.240 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:28.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 10 monclient: shutdown 2024-01-24T23:24:28.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:28.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 1 -- 172.21.15.132:0/2248339816 >> v1:172.21.15.132:6791/0 conn(0x7f12ac13b2c0 legacy=0x7f12ac13d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:28.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 1 -- 172.21.15.132:0/2248339816 shutdown_connections 2024-01-24T23:24:28.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 1 -- 172.21.15.132:0/2248339816 >> 172.21.15.132:0/2248339816 conn(0x7f12ac0a5a10 msgr2=0x7f12ac0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:28.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 1 -- 172.21.15.132:0/2248339816 shutdown_connections 2024-01-24T23:24:28.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 1 -- 172.21.15.132:0/2248339816 wait complete. 2024-01-24T23:24:28.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 10 monclient: build_initial_monmap 2024-01-24T23:24:28.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 10 monclient: monmap: 2024-01-24T23:24:28.241 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:28.241 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:28.241 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:28.241 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:28.241 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:28.241 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:28.241 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:28.241 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:28.241 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:28.241 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:28.242 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:28.242 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:28.242 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:28.242 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:28.242 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:28.242 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:28.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 1 Processor -- start 2024-01-24T23:24:28.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 1 -- start start 2024-01-24T23:24:28.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 10 monclient: init 2024-01-24T23:24:28.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:28.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 10 monclient: _add_conns ranks=[6,3,1,8,5,0,4,7,2] 2024-01-24T23:24:28.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 10 monclient(hunting): picked mon.noname-d con 0x7f12ac09f9c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:28.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 10 monclient(hunting): picked mon.noname-g con 0x7f12ac09ff20 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:28.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 10 monclient(hunting): picked mon.noname-f con 0x7f12ac13b2c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:28.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12ac071680 con 0x7f12ac13b2c0 2024-01-24T23:24:28.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12ac11e8f0 con 0x7f12ac09ff20 2024-01-24T23:24:28.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12ac071960 con 0x7f12ac09f9c0 2024-01-24T23:24:28.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:28.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b3be1640 10 monclient(hunting): authenticate will time out at 1155.555386s 2024-01-24T23:24:28.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b217e640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f12ac13b2c0 legacy=0x7f12ac1d1490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:28.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b117c640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f12ac09ff20 0x7f12ac11bd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:43962/0 (socket says 172.21.15.132:43962) 2024-01-24T23:24:28.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f12b117c640 1 -- 172.21.15.132:0/3208613764 learned_addr learned my addr 172.21.15.132:0/3208613764 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:28.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f129e7fc640 1 -- 172.21.15.132:0/3208613764 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 902061853 0 0) 0x7f12ac11e8f0 con 0x7f12ac09ff20 2024-01-24T23:24:28.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f129e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:28.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f129e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:28.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f129e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:28.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f129e7fc640 10 monclient(hunting): my global_id is 5565 2024-01-24T23:24:28.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f129e7fc640 1 -- 172.21.15.132:0/3208613764 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1290003560 con 0x7f12ac09ff20 2024-01-24T23:24:28.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f129e7fc640 1 -- 172.21.15.132:0/3208613764 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2261760429 0 0) 0x7f1290003560 con 0x7f12ac09ff20 2024-01-24T23:24:28.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f129e7fc640 1 -- 172.21.15.132:0/3208613764 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f12ac11e8f0 con 0x7f12ac09ff20 2024-01-24T23:24:28.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f129e7fc640 1 -- 172.21.15.132:0/3208613764 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f12a80015a0 con 0x7f12ac09ff20 2024-01-24T23:24:28.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.226+0000 7f129e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:28.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f129e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:28.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f129e7fc640 10 monclient(hunting): dump: 2024-01-24T23:24:28.244 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:28.244 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:28.244 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:28.244 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:28.244 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:28.244 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:28.244 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:28.244 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:28.244 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:28.245 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:28.245 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:28.245 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:28.245 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:28.245 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:28.245 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:28.245 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:28.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f129e7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:28.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f129e7fc640 1 -- 172.21.15.132:0/3208613764 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1688641450 0 0) 0x7f12ac11e8f0 con 0x7f12ac09ff20 2024-01-24T23:24:28.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f129e7fc640 1 -- 172.21.15.132:0/3208613764 >> v1:172.21.15.115:6792/0 conn(0x7f12ac09f9c0 legacy=0x7f12ac11b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:28.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f129e7fc640 1 -- 172.21.15.132:0/3208613764 >> v1:172.21.15.132:6789/0 conn(0x7f12ac13b2c0 legacy=0x7f12ac1d1490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:28.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f129e7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:24:28.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f129e7fc640 1 monclient: found mon.g 2024-01-24T23:24:28.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f129e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:28.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f129e7fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:28.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f129e7fc640 1 -- 172.21.15.132:0/3208613764 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ac11e610 con 0x7f12ac09ff20 2024-01-24T23:24:28.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f129e7fc640 10 monclient: _finish_auth 0 2024-01-24T23:24:28.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f129e7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:24:28.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f129e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:28.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f12b3be1640 5 monclient: authenticate success, global_id 5565 2024-01-24T23:24:28.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f12b3be1640 10 monclient: _renew_subs 2024-01-24T23:24:28.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f12b3be1640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:28.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f12b3be1640 1 -- 172.21.15.132:0/3208613764 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12ac11ceb0 con 0x7f12ac09ff20 2024-01-24T23:24:28.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f12b3be1640 10 monclient: _renew_subs 2024-01-24T23:24:28.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f12b3be1640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:28.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f12b3be1640 1 -- 172.21.15.132:0/3208613764 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f12ac1343e0 con 0x7f12ac09ff20 2024-01-24T23:24:28.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f129e7fc640 1 -- 172.21.15.132:0/3208613764 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f12a8004aa0 con 0x7f12ac09ff20 2024-01-24T23:24:28.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f129e7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:28.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f129e7fc640 1 -- 172.21.15.132:0/3208613764 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f12a8004f50 con 0x7f12ac09ff20 2024-01-24T23:24:28.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f129e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:28.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f129ffff640 4 set_mon_vals no callback set 2024-01-24T23:24:28.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f129e7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:28.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f129e7fc640 10 monclient: dump: 2024-01-24T23:24:28.247 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:28.247 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:28.247 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:28.247 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:28.247 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:28.247 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:28.247 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:28.247 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:28.247 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:28.247 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:28.247 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:28.247 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:28.248 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:28.248 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:28.248 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:28.248 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:28.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f12b3be1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:28.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f12b3be1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:28.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f12b3be1640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:28.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f129e7fc640 1 -- 172.21.15.132:0/3208613764 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f12a800f9d0 con 0x7f12ac09ff20 2024-01-24T23:24:28.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f12b3be1640 1 -- 172.21.15.132:0/3208613764 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1294005090 con 0x7f12ac09ff20 2024-01-24T23:24:28.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f129e7fc640 1 -- 172.21.15.132:0/3208613764 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(91..91 src has 1..91) v4 ==== 5064+0+0 (unknown 1772860996 0 0) 0x7f12a8043d80 con 0x7f12ac09ff20 2024-01-24T23:24:28.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.230+0000 7f12b197d640 10 monclient: get_auth_request con 0x7f1290036f70 auth_method 0 2024-01-24T23:24:28.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.234+0000 7f129e7fc640 1 -- 172.21.15.132:0/3208613764 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f12a8071310 con 0x7f12ac09ff20 2024-01-24T23:24:28.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.234+0000 7f129e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:28.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.234+0000 7f129e7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:28.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.382+0000 7f12b3be1640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}] 2024-01-24T23:24:28.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.382+0000 7f12b3be1640 10 monclient: _send_command 2 [{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}] 2024-01-24T23:24:28.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.382+0000 7f12b3be1640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:28.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.382+0000 7f12b3be1640 1 -- 172.21.15.132:0/3208613764 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]} v 0) v1 -- 0x7f12940058c0 con 0x7f12ac09ff20 2024-01-24T23:24:28.996 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.994+0000 7f129e7fc640 1 -- 172.21.15.132:0/3208613764 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}]=0 moved item id 7 name 'osd.7' to location {host=foo} in crush map v92) v1 ==== 165+0+0 (unknown 2231801509 0 0) 0x7f12a80146d0 con 0x7f12ac09ff20 2024-01-24T23:24:28.996 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.994+0000 7f129e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}] 2024-01-24T23:24:28.996 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:28.994+0000 7f129e7fc640 10 monclient: _finish_command 2 = system:0 moved item id 7 name 'osd.7' to location {host=foo} in crush map 2024-01-24T23:24:29.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.098+0000 7f12b3be1640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}] 2024-01-24T23:24:29.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.098+0000 7f12b3be1640 10 monclient: _send_command 3 [{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}] 2024-01-24T23:24:29.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.098+0000 7f12b3be1640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:29.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.102+0000 7f12b3be1640 1 -- 172.21.15.132:0/3208613764 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]} v 0) v1 -- 0x7f1294000bc0 con 0x7f12ac09ff20 2024-01-24T23:24:29.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.102+0000 7f129e7fc640 1 -- 172.21.15.132:0/3208613764 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}]=0 no need to move item id 7 name 'osd.7' to location {host=foo} in crush map v92) v1 ==== 175+0+0 (unknown 4247789803 0 0) 0x7f12a8012f50 con 0x7f12ac09ff20 2024-01-24T23:24:29.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.102+0000 7f129e7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}] 2024-01-24T23:24:29.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.102+0000 7f129e7fc640 10 monclient: _finish_command 3 = system:0 no need to move item id 7 name 'osd.7' to location {host=foo} in crush map 2024-01-24T23:24:29.104 INFO:tasks.workunit.client.0.smithi132.stderr:no need to move item id 7 name 'osd.7' to location {host=foo} in crush map 2024-01-24T23:24:29.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.102+0000 7f12b3be1640 1 -- 172.21.15.132:0/3208613764 >> v1:172.21.15.132:6812/1835401922 conn(0x7f1290036f70 legacy=0x7f1290039430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:29.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.102+0000 7f12b3be1640 10 monclient: shutdown 2024-01-24T23:24:29.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.102+0000 7f12b3be1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:29.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.102+0000 7f12b3be1640 1 -- 172.21.15.132:0/3208613764 >> v1:172.21.15.132:6790/0 conn(0x7f12ac09ff20 legacy=0x7f12ac11bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:29.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.102+0000 7f12b3be1640 1 -- 172.21.15.132:0/3208613764 shutdown_connections 2024-01-24T23:24:29.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.102+0000 7f12b3be1640 1 -- 172.21.15.132:0/3208613764 >> 172.21.15.132:0/3208613764 conn(0x7f12ac0a5a10 msgr2=0x7f12ac066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:29.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.102+0000 7f12b3be1640 1 -- 172.21.15.132:0/3208613764 shutdown_connections 2024-01-24T23:24:29.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.102+0000 7f12b3be1640 1 -- 172.21.15.132:0/3208613764 wait complete. 2024-01-24T23:24:29.121 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd find osd.7 2024-01-24T23:24:29.122 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep host 2024-01-24T23:24:29.122 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep foo 2024-01-24T23:24:29.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.234+0000 7f7e44bf3640 10 monclient: get_monmap_and_config 2024-01-24T23:24:29.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.234+0000 7f7e44bf3640 10 monclient: build_initial_monmap 2024-01-24T23:24:29.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.234+0000 7f7e44bf3640 10 monclient: monmap: 2024-01-24T23:24:29.237 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:29.237 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:29.237 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:29.238192+0000 2024-01-24T23:24:29.237 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:29.238192+0000 2024-01-24T23:24:29.237 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:29.237 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:29.237 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:29.237 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:29.238 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:29.238 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:29.238 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:29.238 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:29.238 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:29.238 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:29.238 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:29.238 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:29.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.234+0000 7f7e44bf3640 1 Processor -- start 2024-01-24T23:24:29.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.234+0000 7f7e44bf3640 1 -- start start 2024-01-24T23:24:29.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.234+0000 7f7e44bf3640 10 monclient: init 2024-01-24T23:24:29.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.234+0000 7f7e44bf3640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:29.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.234+0000 7f7e44bf3640 10 monclient: _add_conns ranks=[0,6,2,5,1,4,3,8,7] 2024-01-24T23:24:29.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.234+0000 7f7e44bf3640 10 monclient(hunting): picked mon.noname-a con 0x7f7e4009ff20 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:29.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.234+0000 7f7e44bf3640 10 monclient(hunting): picked mon.noname-d con 0x7f7e4009f9c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:29.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.234+0000 7f7e44bf3640 10 monclient(hunting): picked mon.noname-b con 0x7f7e4013b2c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:29.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.234+0000 7f7e44bf3640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e40064700 con 0x7f7e4009ff20 2024-01-24T23:24:29.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.234+0000 7f7e44bf3640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e4009e760 con 0x7f7e4013b2c0 2024-01-24T23:24:29.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.234+0000 7f7e44bf3640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e400a1160 con 0x7f7e4009f9c0 2024-01-24T23:24:29.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.234+0000 7f7e44bf3640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:29.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.234+0000 7f7e44bf3640 10 monclient(hunting): authenticate will time out at 1156.563125s 2024-01-24T23:24:29.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.234+0000 7f7e3e59c640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f7e4009ff20 legacy=0x7f7e400a0300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:29.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.234+0000 7f7e3ed9d640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f7e4013b2c0 0x7f7e4013d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:43036/0 (socket says 172.21.15.132:43036) 2024-01-24T23:24:29.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.234+0000 7f7e3ed9d640 1 -- 172.21.15.132:0/3897913731 learned_addr learned my addr 172.21.15.132:0/3897913731 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:29.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.234+0000 7f7e3dd9b640 1 --1- 172.21.15.132:0/3897913731 >> v1:172.21.15.115:6792/0 conn(0x7f7e4009f9c0 0x7f7e40139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:49510/0 (socket says 172.21.15.132:49510) 2024-01-24T23:24:29.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 1 -- 172.21.15.132:0/3897913731 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 920533480 0 0) 0x7f7e4009e760 con 0x7f7e4013b2c0 2024-01-24T23:24:29.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 10 monclient(hunting): _negotiate 2024-01-24T23:24:29.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:29.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:29.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 10 monclient(hunting): my global_id is 84854 2024-01-24T23:24:29.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 1 -- 172.21.15.132:0/3897913731 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e28003530 con 0x7f7e4013b2c0 2024-01-24T23:24:29.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 1 -- 172.21.15.132:0/3897913731 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2781710036 0 0) 0x7f7e400a1160 con 0x7f7e4009f9c0 2024-01-24T23:24:29.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 10 monclient(hunting): _negotiate 2024-01-24T23:24:29.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:29.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:29.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 10 monclient(hunting): my global_id is 5595 2024-01-24T23:24:29.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 1 -- 172.21.15.132:0/3897913731 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e4009e760 con 0x7f7e4009f9c0 2024-01-24T23:24:29.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 1 -- 172.21.15.132:0/3897913731 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2997567749 0 0) 0x7f7e28003530 con 0x7f7e4013b2c0 2024-01-24T23:24:29.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 1 -- 172.21.15.132:0/3897913731 >> v1:172.21.15.115:6792/0 conn(0x7f7e4009f9c0 legacy=0x7f7e40139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:29.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 1 -- 172.21.15.132:0/3897913731 >> v1:172.21.15.115:6789/0 conn(0x7f7e4009ff20 legacy=0x7f7e400a0300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:29.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 10 monclient: _finish_hunting 0 2024-01-24T23:24:29.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 1 monclient: found mon.noname-b 2024-01-24T23:24:29.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:29.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:24:29.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 1 -- 172.21.15.132:0/3897913731 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e400a0a10 con 0x7f7e4013b2c0 2024-01-24T23:24:29.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 10 monclient: _finish_auth 0 2024-01-24T23:24:29.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 10 monclient: _check_auth_tickets 2024-01-24T23:24:29.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:29.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 5 monclient: authenticate success, global_id 84854 2024-01-24T23:24:29.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 1 -- 172.21.15.132:0/3897913731 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e38001b40 con 0x7f7e4013b2c0 2024-01-24T23:24:29.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:29.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:24:29.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 10 monclient: dump: 2024-01-24T23:24:29.244 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:29.244 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:29.244 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:29.244 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:29.244 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:29.245 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:29.245 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:29.245 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:29.245 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:29.245 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:29.245 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:29.245 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:29.245 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:29.245 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:29.245 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:29.245 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:29.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:29.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 1 -- 172.21.15.132:0/3897913731 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7e38001d60 con 0x7f7e4013b2c0 2024-01-24T23:24:29.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:29.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 1 -- 172.21.15.132:0/3897913731 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e38004f70 con 0x7f7e4013b2c0 2024-01-24T23:24:29.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:29.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 4 set_mon_vals no callback set 2024-01-24T23:24:29.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:29.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:24:29.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3d59a640 10 monclient: dump: 2024-01-24T23:24:29.246 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:29.246 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:29.246 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:29.246 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:29.246 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:29.246 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:29.246 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:29.246 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:29.246 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:29.246 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:29.247 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:29.247 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:29.247 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:29.247 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:29.247 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:29.247 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:29.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 10 monclient: shutdown 2024-01-24T23:24:29.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:29.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 1 -- 172.21.15.132:0/3897913731 >> v1:172.21.15.115:6790/0 conn(0x7f7e4013b2c0 legacy=0x7f7e4013d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:29.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 1 -- 172.21.15.132:0/3897913731 shutdown_connections 2024-01-24T23:24:29.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 1 -- 172.21.15.132:0/3897913731 >> 172.21.15.132:0/3897913731 conn(0x7f7e400a5a10 msgr2=0x7f7e400a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:29.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 1 -- 172.21.15.132:0/3897913731 shutdown_connections 2024-01-24T23:24:29.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 1 -- 172.21.15.132:0/3897913731 wait complete. 2024-01-24T23:24:29.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 10 monclient: build_initial_monmap 2024-01-24T23:24:29.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 10 monclient: monmap: 2024-01-24T23:24:29.248 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:29.248 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:29.248 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:29.248 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:29.248 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:29.248 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:29.248 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:29.248 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:29.248 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:29.248 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:29.248 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:29.248 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:29.248 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:29.248 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:29.249 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:29.249 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:29.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 1 Processor -- start 2024-01-24T23:24:29.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 1 -- start start 2024-01-24T23:24:29.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 10 monclient: init 2024-01-24T23:24:29.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:29.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 10 monclient: _add_conns ranks=[5,2,3,8,4,7,6,1,0] 2024-01-24T23:24:29.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 10 monclient(hunting): picked mon.noname-h con 0x7f7e4009f9c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:29.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 10 monclient(hunting): picked mon.noname-b con 0x7f7e4009ff20 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:29.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 10 monclient(hunting): picked mon.noname-g con 0x7f7e4013b2c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:29.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e4011bae0 con 0x7f7e4009ff20 2024-01-24T23:24:29.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e40071960 con 0x7f7e4013b2c0 2024-01-24T23:24:29.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e40071680 con 0x7f7e4009f9c0 2024-01-24T23:24:29.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:29.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e44bf3640 10 monclient(hunting): authenticate will time out at 1156.567118s 2024-01-24T23:24:29.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3e59c640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f7e4009f9c0 0x7f7e40138bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:48436/0 (socket says 172.21.15.132:48436) 2024-01-24T23:24:29.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e3e59c640 1 -- 172.21.15.132:0/3293027049 learned_addr learned my addr 172.21.15.132:0/3293027049 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:29.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 1 -- 172.21.15.132:0/3293027049 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2635653814 0 0) 0x7f7e40071680 con 0x7f7e4009f9c0 2024-01-24T23:24:29.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:29.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:29.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:29.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 10 monclient(hunting): my global_id is 5576 2024-01-24T23:24:29.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 1 -- 172.21.15.132:0/3293027049 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e20003530 con 0x7f7e4009f9c0 2024-01-24T23:24:29.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 1 -- 172.21.15.132:0/3293027049 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1634537957 0 0) 0x7f7e40071960 con 0x7f7e4013b2c0 2024-01-24T23:24:29.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:29.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:29.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:29.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 10 monclient(hunting): my global_id is 5574 2024-01-24T23:24:29.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 1 -- 172.21.15.132:0/3293027049 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e40071680 con 0x7f7e4013b2c0 2024-01-24T23:24:29.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 1 -- 172.21.15.132:0/3293027049 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1052542 0 0) 0x7f7e20003530 con 0x7f7e4009f9c0 2024-01-24T23:24:29.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 1 -- 172.21.15.132:0/3293027049 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7e40071960 con 0x7f7e4009f9c0 2024-01-24T23:24:29.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 1 -- 172.21.15.132:0/3293027049 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3916977720 0 0) 0x7f7e40071680 con 0x7f7e4013b2c0 2024-01-24T23:24:29.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 1 -- 172.21.15.132:0/3293027049 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7e20003530 con 0x7f7e4013b2c0 2024-01-24T23:24:29.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 1 -- 172.21.15.132:0/3293027049 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e30004260 con 0x7f7e4009f9c0 2024-01-24T23:24:29.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:29.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:29.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 10 monclient(hunting): dump: 2024-01-24T23:24:29.255 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:29.255 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:29.255 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:29.255 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:29.255 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:29.256 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:29.256 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:29.256 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:29.256 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:29.256 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:29.256 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:29.256 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:29.256 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:29.256 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:29.256 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:29.256 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:29.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:29.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 1 -- 172.21.15.132:0/3293027049 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1246397625 0 0) 0x7f7e4011bae0 con 0x7f7e4009ff20 2024-01-24T23:24:29.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:29.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:29.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:29.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 10 monclient(hunting): my global_id is 84863 2024-01-24T23:24:29.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 1 -- 172.21.15.132:0/3293027049 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e40071680 con 0x7f7e4009ff20 2024-01-24T23:24:29.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 1 -- 172.21.15.132:0/3293027049 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e38001a40 con 0x7f7e4013b2c0 2024-01-24T23:24:29.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.238+0000 7f7e2effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:29.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e2effd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:29.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e2effd640 10 monclient(hunting): dump: 2024-01-24T23:24:29.258 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:29.258 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:29.258 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:29.258 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:29.258 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:29.258 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:29.258 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:29.258 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:29.258 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:29.258 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:29.258 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:29.259 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:29.259 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:29.259 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:29.259 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:29.259 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:29.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e2effd640 1 -- 172.21.15.132:0/3293027049 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3970876427 0 0) 0x7f7e20003530 con 0x7f7e4013b2c0 2024-01-24T23:24:29.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e2effd640 1 -- 172.21.15.132:0/3293027049 >> v1:172.21.15.132:6791/0 conn(0x7f7e4009f9c0 legacy=0x7f7e40138bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:29.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e2effd640 1 -- 172.21.15.132:0/3293027049 >> v1:172.21.15.115:6790/0 conn(0x7f7e4009ff20 legacy=0x7f7e401392c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:29.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e2effd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:29.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e2effd640 1 monclient: found mon.g 2024-01-24T23:24:29.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e2effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:29.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e2effd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:29.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e2effd640 1 -- 172.21.15.132:0/3293027049 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e4011d5e0 con 0x7f7e4013b2c0 2024-01-24T23:24:29.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e2effd640 10 monclient: _finish_auth 0 2024-01-24T23:24:29.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e2effd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:29.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e2effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:29.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e44bf3640 5 monclient: authenticate success, global_id 5574 2024-01-24T23:24:29.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e44bf3640 10 monclient: _renew_subs 2024-01-24T23:24:29.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e44bf3640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:29.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e44bf3640 1 -- 172.21.15.132:0/3293027049 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e4011d8c0 con 0x7f7e4013b2c0 2024-01-24T23:24:29.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e44bf3640 10 monclient: _renew_subs 2024-01-24T23:24:29.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e44bf3640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:29.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e44bf3640 1 -- 172.21.15.132:0/3293027049 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e4011c130 con 0x7f7e4013b2c0 2024-01-24T23:24:29.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e2effd640 1 -- 172.21.15.132:0/3293027049 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7e38004e90 con 0x7f7e4013b2c0 2024-01-24T23:24:29.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e2effd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:29.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e2effd640 1 -- 172.21.15.132:0/3293027049 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e38005340 con 0x7f7e4013b2c0 2024-01-24T23:24:29.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e2effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:29.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e3cd99640 4 set_mon_vals no callback set 2024-01-24T23:24:29.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e2effd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:29.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e2effd640 10 monclient: dump: 2024-01-24T23:24:29.261 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:29.261 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:29.261 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:29.261 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:29.261 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:29.261 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:29.261 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:29.261 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:29.261 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:29.261 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:29.261 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:29.261 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:29.261 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:29.261 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:29.262 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:29.262 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:29.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e44bf3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:29.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e44bf3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:29.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e44bf3640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:29.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e44bf3640 1 -- 172.21.15.132:0/3293027049 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e24005090 con 0x7f7e4013b2c0 2024-01-24T23:24:29.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e2effd640 1 -- 172.21.15.132:0/3293027049 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f7e38005520 con 0x7f7e4013b2c0 2024-01-24T23:24:29.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e2effd640 1 -- 172.21.15.132:0/3293027049 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(92..92 src has 1..92) v4 ==== 5238+0+0 (unknown 3722893996 0 0) 0x7f7e38044130 con 0x7f7e4013b2c0 2024-01-24T23:24:29.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.242+0000 7f7e3e59c640 10 monclient: get_auth_request con 0x7f7e20037880 auth_method 0 2024-01-24T23:24:29.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.246+0000 7f7e2effd640 1 -- 172.21.15.132:0/3293027049 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f7e38014cd0 con 0x7f7e4013b2c0 2024-01-24T23:24:29.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.246+0000 7f7e2effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:29.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.246+0000 7f7e2effd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:29.406 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.402+0000 7f7e44bf3640 10 start_mon_command cmd=[{"prefix": "osd find", "id": 7}] 2024-01-24T23:24:29.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.402+0000 7f7e44bf3640 10 monclient: _send_command 2 [{"prefix": "osd find", "id": 7}] 2024-01-24T23:24:29.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.402+0000 7f7e44bf3640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:29.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.402+0000 7f7e44bf3640 1 -- 172.21.15.132:0/3293027049 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd find", "id": 7} v 0) v1 -- 0x7f7e240058c0 con 0x7f7e4013b2c0 2024-01-24T23:24:29.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.402+0000 7f7e2effd640 1 -- 172.21.15.132:0/3293027049 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd find", "id": 7}]=0 v92) v1 ==== 65+0+226 (unknown 3427033134 0 2157906930) 0x7f7e38013370 con 0x7f7e4013b2c0 2024-01-24T23:24:29.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.402+0000 7f7e2effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd find", "id": 7}] 2024-01-24T23:24:29.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.402+0000 7f7e2effd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:24:29.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.498+0000 7f7e44bf3640 10 start_mon_command cmd=[{"prefix": "osd find", "id": 7}] 2024-01-24T23:24:29.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.498+0000 7f7e44bf3640 10 monclient: _send_command 3 [{"prefix": "osd find", "id": 7}] 2024-01-24T23:24:29.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.498+0000 7f7e44bf3640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:29.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.498+0000 7f7e44bf3640 1 -- 172.21.15.132:0/3293027049 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd find", "id": 7} v 0) v1 -- 0x7f7e24000bd0 con 0x7f7e4013b2c0 2024-01-24T23:24:29.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.498+0000 7f7e2effd640 1 -- 172.21.15.132:0/3293027049 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd find", "id": 7}]=0 v92) v1 ==== 65+0+226 (unknown 3427033134 0 2157906930) 0x7f7e38014af0 con 0x7f7e4013b2c0 2024-01-24T23:24:29.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.498+0000 7f7e2effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd find", "id": 7}] 2024-01-24T23:24:29.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.498+0000 7f7e2effd640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:24:29.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.498+0000 7f7e44bf3640 1 -- 172.21.15.132:0/3293027049 >> v1:172.21.15.132:6812/1835401922 conn(0x7f7e20037880 legacy=0x7f7e20039d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:29.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.502+0000 7f7e44bf3640 10 monclient: shutdown 2024-01-24T23:24:29.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.502+0000 7f7e44bf3640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:29.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.502+0000 7f7e44bf3640 1 -- 172.21.15.132:0/3293027049 >> v1:172.21.15.132:6790/0 conn(0x7f7e4013b2c0 legacy=0x7f7e401399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:29.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.502+0000 7f7e44bf3640 1 -- 172.21.15.132:0/3293027049 shutdown_connections 2024-01-24T23:24:29.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.502+0000 7f7e44bf3640 1 -- 172.21.15.132:0/3293027049 >> 172.21.15.132:0/3293027049 conn(0x7f7e400a5a10 msgr2=0x7f7e40066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:29.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.502+0000 7f7e44bf3640 1 -- 172.21.15.132:0/3293027049 shutdown_connections 2024-01-24T23:24:29.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.502+0000 7f7e44bf3640 1 -- 172.21.15.132:0/3293027049 wait complete. 2024-01-24T23:24:29.518 INFO:tasks.workunit.client.0.smithi132.stdout: "host": "foo", 2024-01-24T23:24:29.518 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm osd.7 2024-01-24T23:24:29.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.630+0000 7f50d7fb3640 10 monclient: get_monmap_and_config 2024-01-24T23:24:29.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.630+0000 7f50d7fb3640 10 monclient: build_initial_monmap 2024-01-24T23:24:29.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.630+0000 7f50d7fb3640 10 monclient: monmap: 2024-01-24T23:24:29.634 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:29.634 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:29.635 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:29.635740+0000 2024-01-24T23:24:29.635 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:29.635740+0000 2024-01-24T23:24:29.635 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:29.635 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:29.635 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:29.635 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:29.635 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:29.635 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:29.635 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:29.635 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:29.635 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:29.635 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:29.635 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:29.635 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:29.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 1 Processor -- start 2024-01-24T23:24:29.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 1 -- start start 2024-01-24T23:24:29.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 10 monclient: init 2024-01-24T23:24:29.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:29.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 10 monclient: _add_conns ranks=[7,1,5,2,4,0,8,3,6] 2024-01-24T23:24:29.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 10 monclient(hunting): picked mon.noname-i con 0x7f50d00714f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:29.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 10 monclient(hunting): picked mon.noname-f con 0x7f50d01339d0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:29.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 10 monclient(hunting): picked mon.noname-h con 0x7f50d013d4f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:29.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d0064700 con 0x7f50d01339d0 2024-01-24T23:24:29.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d006dcc0 con 0x7f50d013d4f0 2024-01-24T23:24:29.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d006d9e0 con 0x7f50d00714f0 2024-01-24T23:24:29.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:29.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 10 monclient(hunting): authenticate will time out at 1156.961141s 2024-01-24T23:24:29.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d554e640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f50d01339d0 legacy=0x7f50d013bdd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:29.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d6550640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f50d013d4f0 0x7f50d013f8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:48440/0 (socket says 172.21.15.132:48440) 2024-01-24T23:24:29.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d6550640 1 -- 172.21.15.132:0/3213181451 learned_addr learned my addr 172.21.15.132:0/3213181451 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:29.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 1 -- 172.21.15.132:0/3213181451 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3267400315 0 0) 0x7f50d006dcc0 con 0x7f50d013d4f0 2024-01-24T23:24:29.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 10 monclient(hunting): _negotiate 2024-01-24T23:24:29.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:29.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:29.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 10 monclient(hunting): my global_id is 5585 2024-01-24T23:24:29.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 1 -- 172.21.15.132:0/3213181451 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50bc003530 con 0x7f50d013d4f0 2024-01-24T23:24:29.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 1 -- 172.21.15.132:0/3213181451 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4099955300 0 0) 0x7f50d006d9e0 con 0x7f50d00714f0 2024-01-24T23:24:29.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 10 monclient(hunting): _negotiate 2024-01-24T23:24:29.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:29.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:29.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 10 monclient(hunting): my global_id is 5605 2024-01-24T23:24:29.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 1 -- 172.21.15.132:0/3213181451 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50d006dcc0 con 0x7f50d00714f0 2024-01-24T23:24:29.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 1 -- 172.21.15.132:0/3213181451 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2894526822 0 0) 0x7f50bc003530 con 0x7f50d013d4f0 2024-01-24T23:24:29.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 1 -- 172.21.15.132:0/3213181451 >> v1:172.21.15.132:6792/0 conn(0x7f50d00714f0 legacy=0x7f50d00718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:29.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 1 -- 172.21.15.132:0/3213181451 >> v1:172.21.15.132:6789/0 conn(0x7f50d01339d0 legacy=0x7f50d013bdd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:29.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 10 monclient: _finish_hunting 0 2024-01-24T23:24:29.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 1 monclient: found mon.noname-h 2024-01-24T23:24:29.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:29.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:24:29.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 1 -- 172.21.15.132:0/3213181451 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d006dfa0 con 0x7f50d013d4f0 2024-01-24T23:24:29.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 10 monclient: _finish_auth 0 2024-01-24T23:24:29.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 10 monclient: _check_auth_tickets 2024-01-24T23:24:29.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:29.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 5 monclient: authenticate success, global_id 5585 2024-01-24T23:24:29.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:29.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 1 -- 172.21.15.132:0/3213181451 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f50c8001b60 con 0x7f50d013d4f0 2024-01-24T23:24:29.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:29.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:29.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 10 monclient: dump: 2024-01-24T23:24:29.645 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:29.645 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:29.645 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:29.646 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:29.646 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:29.646 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:29.646 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:29.646 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:29.646 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:29.646 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:29.646 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:29.646 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:29.646 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:29.646 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:29.646 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:29.646 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:29.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:29.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 1 -- 172.21.15.132:0/3213181451 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f50c8001e40 con 0x7f50d013d4f0 2024-01-24T23:24:29.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:29.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 1 -- 172.21.15.132:0/3213181451 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f50c8004f70 con 0x7f50d013d4f0 2024-01-24T23:24:29.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:29.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 4 set_mon_vals no callback set 2024-01-24T23:24:29.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:29.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:29.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d4d4d640 10 monclient: dump: 2024-01-24T23:24:29.647 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:29.647 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:29.647 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:29.647 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:29.647 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:29.647 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:29.647 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:29.647 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:29.647 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:29.647 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:29.648 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:29.648 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:29.648 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:29.648 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:29.648 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:29.648 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:29.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 10 monclient: shutdown 2024-01-24T23:24:29.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:29.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 1 -- 172.21.15.132:0/3213181451 >> v1:172.21.15.132:6791/0 conn(0x7f50d013d4f0 legacy=0x7f50d013f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:29.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 1 -- 172.21.15.132:0/3213181451 shutdown_connections 2024-01-24T23:24:29.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 1 -- 172.21.15.132:0/3213181451 >> 172.21.15.132:0/3213181451 conn(0x7f50d00a22a0 msgr2=0x7f50d00a2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:29.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 1 -- 172.21.15.132:0/3213181451 shutdown_connections 2024-01-24T23:24:29.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 1 -- 172.21.15.132:0/3213181451 wait complete. 2024-01-24T23:24:29.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 10 monclient: build_initial_monmap 2024-01-24T23:24:29.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 10 monclient: monmap: 2024-01-24T23:24:29.649 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:29.649 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:29.649 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:29.649 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:29.649 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:29.649 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:29.649 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:29.649 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:29.649 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:29.649 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:29.649 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:29.649 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:29.649 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:29.649 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:29.650 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:29.650 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:29.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 1 Processor -- start 2024-01-24T23:24:29.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 1 -- start start 2024-01-24T23:24:29.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.634+0000 7f50d7fb3640 10 monclient: init 2024-01-24T23:24:29.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50d7fb3640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:29.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50d7fb3640 10 monclient: _add_conns ranks=[2,7,6,8,1,3,5,0,4] 2024-01-24T23:24:29.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50d7fb3640 10 monclient(hunting): picked mon.noname-b con 0x7f50d00714f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:29.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50d7fb3640 10 monclient(hunting): picked mon.noname-i con 0x7f50d01339d0 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:29.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50d7fb3640 10 monclient(hunting): picked mon.noname-d con 0x7f50d013d4f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:29.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50d7fb3640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d009fb90 con 0x7f50d00714f0 2024-01-24T23:24:29.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50d7fb3640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d0135550 con 0x7f50d013d4f0 2024-01-24T23:24:29.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50d7fb3640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d0135830 con 0x7f50d01339d0 2024-01-24T23:24:29.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50d7fb3640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:29.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50d7fb3640 10 monclient(hunting): authenticate will time out at 1156.964371s 2024-01-24T23:24:29.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50d554e640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f50d01339d0 0x7f50d01d08b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:57450/0 (socket says 172.21.15.132:57450) 2024-01-24T23:24:29.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50d554e640 1 -- 172.21.15.132:0/520337071 learned_addr learned my addr 172.21.15.132:0/520337071 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:29.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 1 -- 172.21.15.132:0/520337071 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 160714882 0 0) 0x7f50d0135830 con 0x7f50d01339d0 2024-01-24T23:24:29.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:29.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:29.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:29.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 10 monclient(hunting): my global_id is 5614 2024-01-24T23:24:29.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 1 -- 172.21.15.132:0/520337071 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50b4003530 con 0x7f50d01339d0 2024-01-24T23:24:29.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 1 -- 172.21.15.132:0/520337071 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3521523945 0 0) 0x7f50b4003530 con 0x7f50d01339d0 2024-01-24T23:24:29.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 1 -- 172.21.15.132:0/520337071 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f50d0135830 con 0x7f50d01339d0 2024-01-24T23:24:29.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 1 -- 172.21.15.132:0/520337071 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f50c00042d0 con 0x7f50d01339d0 2024-01-24T23:24:29.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:29.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:29.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 10 monclient(hunting): dump: 2024-01-24T23:24:29.652 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:29.652 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:29.652 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:29.652 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:29.652 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:29.652 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:29.652 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:29.652 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:29.653 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:29.653 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:29.653 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:29.653 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:29.653 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:29.653 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:29.653 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:29.653 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:29.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:29.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 1 -- 172.21.15.132:0/520337071 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2403716512 0 0) 0x7f50d009fb90 con 0x7f50d00714f0 2024-01-24T23:24:29.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:29.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:29.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:29.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 10 monclient(hunting): my global_id is 84872 2024-01-24T23:24:29.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 1 -- 172.21.15.132:0/520337071 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50b4003530 con 0x7f50d00714f0 2024-01-24T23:24:29.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 1 -- 172.21.15.132:0/520337071 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 744144654 0 0) 0x7f50d0135550 con 0x7f50d013d4f0 2024-01-24T23:24:29.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:29.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:29.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:29.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 10 monclient(hunting): my global_id is 5604 2024-01-24T23:24:29.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 1 -- 172.21.15.132:0/520337071 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50d009fb90 con 0x7f50d013d4f0 2024-01-24T23:24:29.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 1 -- 172.21.15.132:0/520337071 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 596916917 0 0) 0x7f50d0135830 con 0x7f50d01339d0 2024-01-24T23:24:29.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 1 -- 172.21.15.132:0/520337071 >> v1:172.21.15.115:6792/0 conn(0x7f50d013d4f0 legacy=0x7f50d01d3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:29.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 1 -- 172.21.15.132:0/520337071 >> v1:172.21.15.115:6790/0 conn(0x7f50d00714f0 legacy=0x7f50d012a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:29.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 10 monclient: _finish_hunting 0 2024-01-24T23:24:29.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 1 monclient: found mon.i 2024-01-24T23:24:29.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:29.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:29.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 1 -- 172.21.15.132:0/520337071 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d0135b10 con 0x7f50d01339d0 2024-01-24T23:24:29.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 10 monclient: _finish_auth 0 2024-01-24T23:24:29.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 10 monclient: _check_auth_tickets 2024-01-24T23:24:29.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:29.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50d7fb3640 5 monclient: authenticate success, global_id 5614 2024-01-24T23:24:29.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50d7fb3640 10 monclient: _renew_subs 2024-01-24T23:24:29.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50d7fb3640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:29.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50d7fb3640 1 -- 172.21.15.132:0/520337071 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d01351b0 con 0x7f50d01339d0 2024-01-24T23:24:29.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50d7fb3640 10 monclient: _renew_subs 2024-01-24T23:24:29.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50d7fb3640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:29.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50d7fb3640 1 -- 172.21.15.132:0/520337071 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d0131100 con 0x7f50d01339d0 2024-01-24T23:24:29.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 1 -- 172.21.15.132:0/520337071 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f50c00044f0 con 0x7f50d01339d0 2024-01-24T23:24:29.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:29.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 1 -- 172.21.15.132:0/520337071 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f50c00049a0 con 0x7f50d01339d0 2024-01-24T23:24:29.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:29.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c7fff640 4 set_mon_vals no callback set 2024-01-24T23:24:29.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:29.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 10 monclient: dump: 2024-01-24T23:24:29.656 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:29.656 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:29.656 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:29.656 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:29.656 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:29.656 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:29.656 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:29.656 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:29.656 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:29.656 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:29.657 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:29.657 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:29.657 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:29.657 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:29.657 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:29.657 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:29.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50d7fb3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:29.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50d7fb3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:29.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50d7fb3640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:29.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 1 -- 172.21.15.132:0/520337071 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f50c0004b00 con 0x7f50d01339d0 2024-01-24T23:24:29.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50d7fb3640 1 -- 172.21.15.132:0/520337071 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b8005090 con 0x7f50d01339d0 2024-01-24T23:24:29.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50c67fc640 1 -- 172.21.15.132:0/520337071 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(92..92 src has 1..92) v4 ==== 5238+0+0 (unknown 3722893996 0 0) 0x7f50c0043d00 con 0x7f50d01339d0 2024-01-24T23:24:29.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.638+0000 7f50d5d4f640 10 monclient: get_auth_request con 0x7f50b4037230 auth_method 0 2024-01-24T23:24:29.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.642+0000 7f50c67fc640 1 -- 172.21.15.132:0/520337071 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f50c0014990 con 0x7f50d01339d0 2024-01-24T23:24:29.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.642+0000 7f50c67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:29.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.642+0000 7f50c67fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:29.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.802+0000 7f50d7fb3640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.7"}] 2024-01-24T23:24:29.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.802+0000 7f50d7fb3640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.7"}] 2024-01-24T23:24:29.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.802+0000 7f50d7fb3640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:29.806 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:29.802+0000 7f50d7fb3640 1 -- 172.21.15.132:0/520337071 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "osd.7"} v 0) v1 -- 0x7f50b8005340 con 0x7f50d01339d0 2024-01-24T23:24:30.037 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.034+0000 7f50c67fc640 1 -- 172.21.15.132:0/520337071 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.7"}]=0 removed item id 7 name 'osd.7' from crush map v93) v1 ==== 122+0+0 (unknown 3481786152 0 0) 0x7f50c0012e00 con 0x7f50d01339d0 2024-01-24T23:24:30.037 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.034+0000 7f50c67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.7"}] 2024-01-24T23:24:30.037 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.034+0000 7f50c67fc640 10 monclient: _finish_command 2 = system:0 removed item id 7 name 'osd.7' from crush map 2024-01-24T23:24:30.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.138+0000 7f50d7fb3640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.7"}] 2024-01-24T23:24:30.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.138+0000 7f50d7fb3640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.7"}] 2024-01-24T23:24:30.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.138+0000 7f50d7fb3640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:30.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.138+0000 7f50d7fb3640 1 -- 172.21.15.132:0/520337071 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "osd.7"} v 0) v1 -- 0x7f50b8002220 con 0x7f50d01339d0 2024-01-24T23:24:30.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.138+0000 7f50c67fc640 1 -- 172.21.15.132:0/520337071 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.7"}]=0 device 'osd.7' does not appear in the crush map v93) v1 ==== 124+0+0 (unknown 3779587737 0 0) 0x7f50c0014580 con 0x7f50d01339d0 2024-01-24T23:24:30.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.138+0000 7f50c67fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.7"}] 2024-01-24T23:24:30.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.138+0000 7f50c67fc640 10 monclient: _finish_command 3 = system:0 device 'osd.7' does not appear in the crush map 2024-01-24T23:24:30.142 INFO:tasks.workunit.client.0.smithi132.stderr:device 'osd.7' does not appear in the crush map 2024-01-24T23:24:30.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.142+0000 7f50d7fb3640 1 -- 172.21.15.132:0/520337071 >> v1:172.21.15.132:6812/1835401922 conn(0x7f50b4037230 legacy=0x7f50b40396f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:30.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.142+0000 7f50d7fb3640 10 monclient: shutdown 2024-01-24T23:24:30.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.142+0000 7f50d7fb3640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:30.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.142+0000 7f50d7fb3640 1 -- 172.21.15.132:0/520337071 >> v1:172.21.15.132:6792/0 conn(0x7f50d01339d0 legacy=0x7f50d01d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:30.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.142+0000 7f50d7fb3640 1 -- 172.21.15.132:0/520337071 shutdown_connections 2024-01-24T23:24:30.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.142+0000 7f50d7fb3640 1 -- 172.21.15.132:0/520337071 >> 172.21.15.132:0/520337071 conn(0x7f50d00a22a0 msgr2=0x7f50d012d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:30.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.142+0000 7f50d7fb3640 1 -- 172.21.15.132:0/520337071 shutdown_connections 2024-01-24T23:24:30.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.142+0000 7f50d7fb3640 1 -- 172.21.15.132:0/520337071 wait complete. 2024-01-24T23:24:30.159 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm osd.9 2024-01-24T23:24:30.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f643e7af640 10 monclient: get_monmap_and_config 2024-01-24T23:24:30.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f643e7af640 10 monclient: build_initial_monmap 2024-01-24T23:24:30.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f643e7af640 10 monclient: monmap: 2024-01-24T23:24:30.277 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:30.277 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:30.277 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:30.277917+0000 2024-01-24T23:24:30.277 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:30.277917+0000 2024-01-24T23:24:30.277 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:30.277 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:30.277 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:30.277 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:30.277 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:30.277 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:30.278 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:30.278 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:30.278 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:30.278 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:30.278 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:30.278 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:30.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f643e7af640 1 Processor -- start 2024-01-24T23:24:30.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f643e7af640 1 -- start start 2024-01-24T23:24:30.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f643e7af640 10 monclient: init 2024-01-24T23:24:30.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f643e7af640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:30.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f643e7af640 10 monclient: _add_conns ranks=[6,5,3,7,2,8,1,0,4] 2024-01-24T23:24:30.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f643e7af640 10 monclient(hunting): picked mon.noname-d con 0x7f6438136e50 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:30.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f643e7af640 10 monclient(hunting): picked mon.noname-h con 0x7f64380a0290 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:30.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f643e7af640 10 monclient(hunting): picked mon.noname-g con 0x7f64380a1600 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:30.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f643e7af640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6438064700 con 0x7f64380a1600 2024-01-24T23:24:30.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f643e7af640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6438137940 con 0x7f64380a0290 2024-01-24T23:24:30.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f643e7af640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643809f9c0 con 0x7f6438136e50 2024-01-24T23:24:30.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f643e7af640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:30.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f643e7af640 10 monclient(hunting): authenticate will time out at 1157.602533s 2024-01-24T23:24:30.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f64377fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f64380a0290 0x7f643813bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:48450/0 (socket says 172.21.15.132:48450) 2024-01-24T23:24:30.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f64377fe640 1 -- 172.21.15.132:0/567544226 learned_addr learned my addr 172.21.15.132:0/567544226 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:30.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f643cd4c640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f64380a1600 0x7f643813f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:43976/0 (socket says 172.21.15.132:43976) 2024-01-24T23:24:30.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 1 -- 172.21.15.132:0/567544226 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 182822048 0 0) 0x7f6438137940 con 0x7f64380a0290 2024-01-24T23:24:30.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:30.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:30.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:30.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 10 monclient(hunting): my global_id is 5594 2024-01-24T23:24:30.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 1 -- 172.21.15.132:0/567544226 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6420003530 con 0x7f64380a0290 2024-01-24T23:24:30.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 1 -- 172.21.15.132:0/567544226 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 396131681 0 0) 0x7f6438064700 con 0x7f64380a1600 2024-01-24T23:24:30.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:30.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:30.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:30.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 10 monclient(hunting): my global_id is 5583 2024-01-24T23:24:30.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 1 -- 172.21.15.132:0/567544226 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6438137940 con 0x7f64380a1600 2024-01-24T23:24:30.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 1 -- 172.21.15.132:0/567544226 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 798769355 0 0) 0x7f6438137940 con 0x7f64380a1600 2024-01-24T23:24:30.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 1 -- 172.21.15.132:0/567544226 >> v1:172.21.15.115:6792/0 conn(0x7f6438136e50 legacy=0x7f6438137230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:30.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 1 -- 172.21.15.132:0/567544226 >> v1:172.21.15.132:6791/0 conn(0x7f64380a0290 legacy=0x7f643813bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:30.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:30.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 1 monclient: found mon.noname-g 2024-01-24T23:24:30.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:30.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:24:30.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 1 -- 172.21.15.132:0/567544226 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643809eda0 con 0x7f64380a1600 2024-01-24T23:24:30.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 10 monclient: _finish_auth 0 2024-01-24T23:24:30.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:30.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:30.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 1 -- 172.21.15.132:0/567544226 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6430001b40 con 0x7f64380a1600 2024-01-24T23:24:30.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f643e7af640 5 monclient: authenticate success, global_id 5583 2024-01-24T23:24:30.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:30.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:30.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 10 monclient: dump: 2024-01-24T23:24:30.288 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:30.288 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:30.288 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:30.288 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:30.288 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:30.289 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:30.289 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:30.289 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:30.289 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:30.289 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:30.289 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:30.289 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:30.289 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:30.289 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:30.289 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:30.289 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:30.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f643e7af640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:30.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 1 -- 172.21.15.132:0/567544226 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6430001cf0 con 0x7f64380a1600 2024-01-24T23:24:30.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:30.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f643e7af640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:30.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f643e7af640 4 set_mon_vals no callback set 2024-01-24T23:24:30.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 1 -- 172.21.15.132:0/567544226 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6430004b70 con 0x7f64380a1600 2024-01-24T23:24:30.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:30.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:30.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.274+0000 7f6436ffd640 10 monclient: dump: 2024-01-24T23:24:30.290 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:30.290 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:30.290 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:30.290 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:30.290 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:30.291 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:30.291 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:30.291 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:30.291 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:30.291 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:30.291 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:30.291 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:30.291 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:30.291 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:30.291 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:30.291 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:30.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 10 monclient: shutdown 2024-01-24T23:24:30.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:30.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 1 -- 172.21.15.132:0/567544226 >> v1:172.21.15.132:6790/0 conn(0x7f64380a1600 legacy=0x7f643813f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:30.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 1 -- 172.21.15.132:0/567544226 shutdown_connections 2024-01-24T23:24:30.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 1 -- 172.21.15.132:0/567544226 >> 172.21.15.132:0/567544226 conn(0x7f64380a5a10 msgr2=0x7f64380a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:30.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 1 -- 172.21.15.132:0/567544226 shutdown_connections 2024-01-24T23:24:30.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 1 -- 172.21.15.132:0/567544226 wait complete. 2024-01-24T23:24:30.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 10 monclient: build_initial_monmap 2024-01-24T23:24:30.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 10 monclient: monmap: 2024-01-24T23:24:30.292 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:30.292 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:30.292 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:30.292 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:30.292 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:30.292 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:30.292 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:30.292 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:30.292 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:30.293 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:30.293 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:30.293 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:30.293 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:30.293 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:30.293 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:30.293 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:30.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 1 Processor -- start 2024-01-24T23:24:30.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 1 -- start start 2024-01-24T23:24:30.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 10 monclient: init 2024-01-24T23:24:30.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:30.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 10 monclient: _add_conns ranks=[8,3,2,0,1,5,6,4,7] 2024-01-24T23:24:30.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 10 monclient(hunting): picked mon.noname-e con 0x7f64380a0290 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:30.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 10 monclient(hunting): picked mon.noname-g con 0x7f64380a1600 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:30.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 10 monclient(hunting): picked mon.noname-b con 0x7f6438136e50 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:30.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64381c8840 con 0x7f6438136e50 2024-01-24T23:24:30.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643809ea70 con 0x7f64380a1600 2024-01-24T23:24:30.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6438136c20 con 0x7f64380a0290 2024-01-24T23:24:30.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:30.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 10 monclient(hunting): authenticate will time out at 1157.605620s 2024-01-24T23:24:30.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f64377fe640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f64380a1600 0x7f64381d06c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:43978/0 (socket says 172.21.15.132:43978) 2024-01-24T23:24:30.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f64377fe640 1 -- 172.21.15.132:0/1370723686 learned_addr learned my addr 172.21.15.132:0/1370723686 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:30.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 1 -- 172.21.15.132:0/1370723686 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2267525966 0 0) 0x7f643809ea70 con 0x7f64380a1600 2024-01-24T23:24:30.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:24:30.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:30.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:30.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 10 monclient(hunting): my global_id is 5592 2024-01-24T23:24:30.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 1 -- 172.21.15.132:0/1370723686 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6424003530 con 0x7f64380a1600 2024-01-24T23:24:30.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 1 -- 172.21.15.132:0/1370723686 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 492389907 0 0) 0x7f6424003530 con 0x7f64380a1600 2024-01-24T23:24:30.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 1 -- 172.21.15.132:0/1370723686 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f643809ea70 con 0x7f64380a1600 2024-01-24T23:24:30.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 1 -- 172.21.15.132:0/1370723686 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f64280014a0 con 0x7f64380a1600 2024-01-24T23:24:30.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:30.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:30.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 10 monclient(hunting): dump: 2024-01-24T23:24:30.295 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:30.295 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:30.295 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:30.295 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:30.295 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:30.295 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:30.295 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:30.296 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:30.297 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:30.297 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:30.297 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:30.297 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:30.297 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:30.297 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:30.297 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:30.297 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:30.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:30.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 1 -- 172.21.15.132:0/1370723686 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2236384129 0 0) 0x7f643809ea70 con 0x7f64380a1600 2024-01-24T23:24:30.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 1 -- 172.21.15.132:0/1370723686 >> v1:172.21.15.115:6793/0 conn(0x7f64380a0290 legacy=0x7f6438134e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:30.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 1 -- 172.21.15.132:0/1370723686 >> v1:172.21.15.115:6790/0 conn(0x7f6438136e50 legacy=0x7f64381d3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:30.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:24:30.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 1 monclient: found mon.g 2024-01-24T23:24:30.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:30.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:30.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 1 -- 172.21.15.132:0/1370723686 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64381204e0 con 0x7f64380a1600 2024-01-24T23:24:30.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 10 monclient: _finish_auth 0 2024-01-24T23:24:30.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:24:30.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:30.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 5 monclient: authenticate success, global_id 5592 2024-01-24T23:24:30.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 10 monclient: _renew_subs 2024-01-24T23:24:30.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:30.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 1 -- 172.21.15.132:0/1370723686 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6438122840 con 0x7f64380a1600 2024-01-24T23:24:30.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 10 monclient: _renew_subs 2024-01-24T23:24:30.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 1 -- 172.21.15.132:0/1370723686 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6428004aa0 con 0x7f64380a1600 2024-01-24T23:24:30.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:30.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 1 -- 172.21.15.132:0/1370723686 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6438135570 con 0x7f64380a1600 2024-01-24T23:24:30.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:30.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 1 -- 172.21.15.132:0/1370723686 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6428004f50 con 0x7f64380a1600 2024-01-24T23:24:30.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:30.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f64367fc640 4 set_mon_vals no callback set 2024-01-24T23:24:30.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:30.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 10 monclient: dump: 2024-01-24T23:24:30.299 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:30.299 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:30.299 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:30.299 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:30.299 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:30.299 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:30.299 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:30.300 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:30.300 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:30.300 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:30.300 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:30.300 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:30.300 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:30.300 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:30.300 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:30.300 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:30.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:30.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:30.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:30.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f643e7af640 1 -- 172.21.15.132:0/1370723686 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6418005090 con 0x7f64380a1600 2024-01-24T23:24:30.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.278+0000 7f6434ff9640 1 -- 172.21.15.132:0/1370723686 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f642800f9d0 con 0x7f64380a1600 2024-01-24T23:24:30.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.282+0000 7f6434ff9640 1 -- 172.21.15.132:0/1370723686 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(93..93 src has 1..93) v4 ==== 5130+0+0 (unknown 2773145238 0 0) 0x7f6428043e60 con 0x7f64380a1600 2024-01-24T23:24:30.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.282+0000 7f6437fff640 10 monclient: get_auth_request con 0x7f6424036fd0 auth_method 0 2024-01-24T23:24:30.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.286+0000 7f6434ff9640 1 -- 172.21.15.132:0/1370723686 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f6428014950 con 0x7f64380a1600 2024-01-24T23:24:30.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.286+0000 7f6434ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:30.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.286+0000 7f6434ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:30.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.446+0000 7f643e7af640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.9"}] 2024-01-24T23:24:30.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.446+0000 7f643e7af640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.9"}] 2024-01-24T23:24:30.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.446+0000 7f643e7af640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:30.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.446+0000 7f643e7af640 1 -- 172.21.15.132:0/1370723686 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rm", "name": "osd.9"} v 0) v1 -- 0x7f6418005340 con 0x7f64380a1600 2024-01-24T23:24:30.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.446+0000 7f6434ff9640 1 -- 172.21.15.132:0/1370723686 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.9"}]=0 device 'osd.9' does not appear in the crush map v93) v1 ==== 124+0+0 (unknown 3052898913 0 0) 0x7f6428012ff0 con 0x7f64380a1600 2024-01-24T23:24:30.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.446+0000 7f6434ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.9"}] 2024-01-24T23:24:30.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.446+0000 7f6434ff9640 10 monclient: _finish_command 2 = system:0 device 'osd.9' does not appear in the crush map 2024-01-24T23:24:30.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.538+0000 7f643e7af640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.9"}] 2024-01-24T23:24:30.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.538+0000 7f643e7af640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.9"}] 2024-01-24T23:24:30.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.538+0000 7f643e7af640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:30.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.538+0000 7f643e7af640 1 -- 172.21.15.132:0/1370723686 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rm", "name": "osd.9"} v 0) v1 -- 0x7f6418002220 con 0x7f64380a1600 2024-01-24T23:24:30.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.538+0000 7f6434ff9640 1 -- 172.21.15.132:0/1370723686 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.9"}]=0 device 'osd.9' does not appear in the crush map v93) v1 ==== 124+0+0 (unknown 3052898913 0 0) 0x7f6428012ff0 con 0x7f64380a1600 2024-01-24T23:24:30.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.538+0000 7f6434ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.9"}] 2024-01-24T23:24:30.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.538+0000 7f6434ff9640 10 monclient: _finish_command 3 = system:0 device 'osd.9' does not appear in the crush map 2024-01-24T23:24:30.541 INFO:tasks.workunit.client.0.smithi132.stderr:device 'osd.9' does not appear in the crush map 2024-01-24T23:24:30.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.538+0000 7f643e7af640 1 -- 172.21.15.132:0/1370723686 >> v1:172.21.15.132:6812/1835401922 conn(0x7f6424036fd0 legacy=0x7f6424039490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:30.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.538+0000 7f643e7af640 10 monclient: shutdown 2024-01-24T23:24:30.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.538+0000 7f643e7af640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:30.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.538+0000 7f643e7af640 1 -- 172.21.15.132:0/1370723686 >> v1:172.21.15.132:6790/0 conn(0x7f64380a1600 legacy=0x7f64381d06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:30.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.538+0000 7f643e7af640 1 -- 172.21.15.132:0/1370723686 shutdown_connections 2024-01-24T23:24:30.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.538+0000 7f643e7af640 1 -- 172.21.15.132:0/1370723686 >> 172.21.15.132:0/1370723686 conn(0x7f64380a5a10 msgr2=0x7f64380a3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:30.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.538+0000 7f643e7af640 1 -- 172.21.15.132:0/1370723686 shutdown_connections 2024-01-24T23:24:30.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.538+0000 7f643e7af640 1 -- 172.21.15.132:0/1370723686 wait complete. 2024-01-24T23:24:30.556 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm foo 2024-01-24T23:24:30.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f858257c640 10 monclient: get_monmap_and_config 2024-01-24T23:24:30.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f858257c640 10 monclient: build_initial_monmap 2024-01-24T23:24:30.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f858257c640 10 monclient: monmap: 2024-01-24T23:24:30.673 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:30.673 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:30.673 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:30.674317+0000 2024-01-24T23:24:30.673 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:30.674317+0000 2024-01-24T23:24:30.673 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:30.674 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:30.674 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:30.674 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:30.674 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:30.674 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:30.674 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:30.674 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:30.674 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:30.674 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:30.674 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:30.674 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:30.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f858257c640 1 Processor -- start 2024-01-24T23:24:30.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f858257c640 1 -- start start 2024-01-24T23:24:30.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f858257c640 10 monclient: init 2024-01-24T23:24:30.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f858257c640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:30.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f858257c640 10 monclient: _add_conns ranks=[5,4,0,2,3,6,1,7,8] 2024-01-24T23:24:30.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f858257c640 10 monclient(hunting): picked mon.noname-h con 0x7f857c136cb0 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:30.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f858257c640 10 monclient(hunting): picked mon.noname-c con 0x7f857c1377a0 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:30.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f858257c640 10 monclient(hunting): picked mon.noname-a con 0x7f857c09bcc0 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:30.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f858257c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f857c064700 con 0x7f857c09bcc0 2024-01-24T23:24:30.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f858257c640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f857c09b650 con 0x7f857c1377a0 2024-01-24T23:24:30.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f858257c640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f857c09c110 con 0x7f857c136cb0 2024-01-24T23:24:30.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f858257c640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:30.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f858257c640 10 monclient(hunting): authenticate will time out at 1157.999066s 2024-01-24T23:24:30.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857bfff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f857c136cb0 0x7f857c137090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:48458/0 (socket says 172.21.15.132:48458) 2024-01-24T23:24:30.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857bfff640 1 -- 172.21.15.132:0/1313480323 learned_addr learned my addr 172.21.15.132:0/1313480323 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:30.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f8580b19640 1 -- 172.21.15.132:0/1313480323 >> v1:172.21.15.115:6789/0 conn(0x7f857c09bcc0 legacy=0x7f857c13f910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:30.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857b7fe640 1 -- 172.21.15.132:0/1313480323 >> v1:172.21.15.115:6791/0 conn(0x7f857c1377a0 legacy=0x7f857c13c1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:30.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 1 -- 172.21.15.132:0/1313480323 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 806677964 0 0) 0x7f857c09c110 con 0x7f857c136cb0 2024-01-24T23:24:30.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:30.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:30.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:30.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 10 monclient(hunting): my global_id is 5603 2024-01-24T23:24:30.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 1 -- 172.21.15.132:0/1313480323 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8564003530 con 0x7f857c136cb0 2024-01-24T23:24:30.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 1 -- 172.21.15.132:0/1313480323 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2823277391 0 0) 0x7f8564003530 con 0x7f857c136cb0 2024-01-24T23:24:30.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 1 -- 172.21.15.132:0/1313480323 >> v1:172.21.15.115:6791/0 conn(0x7f857c1377a0 legacy=0x7f857c13c1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:30.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 1 -- 172.21.15.132:0/1313480323 >> v1:172.21.15.115:6789/0 conn(0x7f857c09bcc0 legacy=0x7f857c13f910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:30.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:30.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 1 monclient: found mon.noname-h 2024-01-24T23:24:30.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:30.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:24:30.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 1 -- 172.21.15.132:0/1313480323 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857c09b000 con 0x7f857c136cb0 2024-01-24T23:24:30.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 10 monclient: _finish_auth 0 2024-01-24T23:24:30.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:30.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:30.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f858257c640 5 monclient: authenticate success, global_id 5603 2024-01-24T23:24:30.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f858257c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:30.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 1 -- 172.21.15.132:0/1313480323 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f856c0017c0 con 0x7f857c136cb0 2024-01-24T23:24:30.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:30.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:30.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 10 monclient: dump: 2024-01-24T23:24:30.684 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:30.684 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:30.684 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:30.685 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:30.685 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:30.685 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:30.685 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:30.685 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:30.685 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:30.685 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:30.685 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:30.685 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:30.685 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:30.685 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:30.685 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:30.685 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:30.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f858257c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:30.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 1 -- 172.21.15.132:0/1313480323 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f856c001ae0 con 0x7f857c136cb0 2024-01-24T23:24:30.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:30.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 1 -- 172.21.15.132:0/1313480323 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f856c004bf0 con 0x7f857c136cb0 2024-01-24T23:24:30.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f858257c640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:30.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f858257c640 4 set_mon_vals no callback set 2024-01-24T23:24:30.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:30.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:30.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.670+0000 7f857affd640 10 monclient: dump: 2024-01-24T23:24:30.686 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:30.686 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:30.686 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:30.686 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:30.686 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:30.686 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:30.687 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:30.687 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:30.687 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:30.687 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:30.687 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:30.687 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:30.687 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:30.687 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:30.687 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:30.687 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:30.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f858257c640 10 monclient: shutdown 2024-01-24T23:24:30.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f858257c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:30.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f858257c640 1 -- 172.21.15.132:0/1313480323 >> v1:172.21.15.132:6791/0 conn(0x7f857c136cb0 legacy=0x7f857c137090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:30.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f858257c640 1 -- 172.21.15.132:0/1313480323 shutdown_connections 2024-01-24T23:24:30.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f858257c640 1 -- 172.21.15.132:0/1313480323 >> 172.21.15.132:0/1313480323 conn(0x7f857c0a9c10 msgr2=0x7f857c0a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:30.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f858257c640 1 -- 172.21.15.132:0/1313480323 shutdown_connections 2024-01-24T23:24:30.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f858257c640 1 -- 172.21.15.132:0/1313480323 wait complete. 2024-01-24T23:24:30.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f858257c640 10 monclient: build_initial_monmap 2024-01-24T23:24:30.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f858257c640 10 monclient: monmap: 2024-01-24T23:24:30.688 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:30.688 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:30.688 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:30.688 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:30.688 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:30.688 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:30.688 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:30.688 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:30.688 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:30.688 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:30.688 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:30.689 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:30.689 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:30.689 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:30.689 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:30.689 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:30.689 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f858257c640 1 Processor -- start 2024-01-24T23:24:30.689 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f858257c640 1 -- start start 2024-01-24T23:24:30.689 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f858257c640 10 monclient: init 2024-01-24T23:24:30.689 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f858257c640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:30.689 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f858257c640 10 monclient: _add_conns ranks=[1,0,8,5,3,4,6,7,2] 2024-01-24T23:24:30.689 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f858257c640 10 monclient(hunting): picked mon.noname-f con 0x7f857c09bcc0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:30.689 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f858257c640 10 monclient(hunting): picked mon.noname-a con 0x7f857c136cb0 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:30.689 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f858257c640 10 monclient(hunting): picked mon.noname-e con 0x7f857c1377a0 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:30.689 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f858257c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f857c09c6a0 con 0x7f857c136cb0 2024-01-24T23:24:30.689 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f858257c640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f857c0a2c90 con 0x7f857c09bcc0 2024-01-24T23:24:30.689 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f858257c640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f857c137fb0 con 0x7f857c1377a0 2024-01-24T23:24:30.689 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f858257c640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:30.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f858257c640 10 monclient(hunting): authenticate will time out at 1158.002698s 2024-01-24T23:24:30.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f857b7fe640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f857c136cb0 legacy=0x7f857c1d0470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:30.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f857bfff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f857c09bcc0 legacy=0x7f857c1cb910 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:30.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f8580b19640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f857c1377a0 0x7f857c1d3ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:52046/0 (socket says 172.21.15.132:52046) 2024-01-24T23:24:30.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f8580b19640 1 -- 172.21.15.132:0/694005482 learned_addr learned my addr 172.21.15.132:0/694005482 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:30.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f8578ff9640 1 -- 172.21.15.132:0/694005482 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 476855976 0 0) 0x7f857c137fb0 con 0x7f857c1377a0 2024-01-24T23:24:30.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f8578ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:24:30.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f8578ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:30.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f8578ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:30.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f8578ff9640 10 monclient(hunting): my global_id is 84770 2024-01-24T23:24:30.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f8578ff9640 1 -- 172.21.15.132:0/694005482 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8568003530 con 0x7f857c1377a0 2024-01-24T23:24:30.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f8578ff9640 1 -- 172.21.15.132:0/694005482 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1127154057 0 0) 0x7f8568003530 con 0x7f857c1377a0 2024-01-24T23:24:30.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f8578ff9640 1 -- 172.21.15.132:0/694005482 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f857c137fb0 con 0x7f857c1377a0 2024-01-24T23:24:30.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f8578ff9640 1 -- 172.21.15.132:0/694005482 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8574004630 con 0x7f857c1377a0 2024-01-24T23:24:30.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f8578ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:30.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f8578ff9640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:24:30.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f8578ff9640 10 monclient(hunting): dump: 2024-01-24T23:24:30.691 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:30.691 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:30.691 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:30.691 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:30.691 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:30.691 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:30.691 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:30.691 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:30.691 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:30.691 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:30.691 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:30.691 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:30.691 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:30.691 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:30.692 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:30.692 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:30.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.674+0000 7f8578ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:30.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f8578ff9640 1 -- 172.21.15.132:0/694005482 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3861565976 0 0) 0x7f857c137fb0 con 0x7f857c1377a0 2024-01-24T23:24:30.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f8578ff9640 1 -- 172.21.15.132:0/694005482 >> v1:172.21.15.132:6789/0 conn(0x7f857c09bcc0 legacy=0x7f857c1cb910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:30.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f8578ff9640 1 -- 172.21.15.132:0/694005482 >> v1:172.21.15.115:6789/0 conn(0x7f857c136cb0 legacy=0x7f857c1d0470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:30.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f8578ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:24:30.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f8578ff9640 1 monclient: found mon.e 2024-01-24T23:24:30.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f8578ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:30.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f8578ff9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:30.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f8578ff9640 1 -- 172.21.15.132:0/694005482 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857c11fa90 con 0x7f857c1377a0 2024-01-24T23:24:30.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f8578ff9640 10 monclient: _finish_auth 0 2024-01-24T23:24:30.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f8578ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:24:30.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f8578ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:30.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f858257c640 5 monclient: authenticate success, global_id 84770 2024-01-24T23:24:30.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f858257c640 10 monclient: _renew_subs 2024-01-24T23:24:30.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f858257c640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:30.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f858257c640 1 -- 172.21.15.132:0/694005482 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f857c1be1c0 con 0x7f857c1377a0 2024-01-24T23:24:30.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f858257c640 10 monclient: _renew_subs 2024-01-24T23:24:30.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f858257c640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:30.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f858257c640 1 -- 172.21.15.132:0/694005482 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f857c0a1c70 con 0x7f857c1377a0 2024-01-24T23:24:30.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f8578ff9640 1 -- 172.21.15.132:0/694005482 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8574004900 con 0x7f857c1377a0 2024-01-24T23:24:30.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f8578ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:30.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f8578ff9640 1 -- 172.21.15.132:0/694005482 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8574004e30 con 0x7f857c1377a0 2024-01-24T23:24:30.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f8578ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:30.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f857a7fc640 4 set_mon_vals no callback set 2024-01-24T23:24:30.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f8578ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:24:30.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f8578ff9640 10 monclient: dump: 2024-01-24T23:24:30.693 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:30.693 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:30.693 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:30.694 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:30.694 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:30.694 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:30.694 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:30.694 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:30.694 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:30.694 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:30.694 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:30.694 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:30.694 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:30.694 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:30.694 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:30.694 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:30.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f858257c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:30.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f858257c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:30.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f858257c640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:30.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f858257c640 1 -- 172.21.15.132:0/694005482 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f855c005090 con 0x7f857c1377a0 2024-01-24T23:24:30.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f8578ff9640 1 -- 172.21.15.132:0/694005482 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f8574005010 con 0x7f857c1377a0 2024-01-24T23:24:30.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f8578ff9640 1 -- 172.21.15.132:0/694005482 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(93..93 src has 1..93) v4 ==== 5130+0+0 (unknown 2773145238 0 0) 0x7f8574043ed0 con 0x7f857c1377a0 2024-01-24T23:24:30.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.678+0000 7f857bfff640 10 monclient: get_auth_request con 0x7f8568036ef0 auth_method 0 2024-01-24T23:24:30.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.690+0000 7f8578ff9640 1 -- 172.21.15.132:0/694005482 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f8574014bf0 con 0x7f857c1377a0 2024-01-24T23:24:30.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.690+0000 7f8578ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:30.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.690+0000 7f8578ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:30.852 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.850+0000 7f858257c640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "foo"}] 2024-01-24T23:24:30.852 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.850+0000 7f858257c640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-01-24T23:24:30.852 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.850+0000 7f858257c640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:30.852 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:30.850+0000 7f858257c640 1 -- 172.21.15.132:0/694005482 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush rm", "name": "foo"} v 0) v1 -- 0x7f855c005340 con 0x7f857c1377a0 2024-01-24T23:24:31.032 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.030+0000 7f8578ff9640 1 -- 172.21.15.132:0/694005482 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "foo"}]=0 removed item id -10 name 'foo' from crush map v94) v1 ==== 120+0+0 (unknown 3703534428 0 0) 0x7f85740147e0 con 0x7f857c1377a0 2024-01-24T23:24:31.032 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.030+0000 7f8578ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-01-24T23:24:31.032 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.030+0000 7f8578ff9640 10 monclient: _finish_command 2 = system:0 removed item id -10 name 'foo' from crush map 2024-01-24T23:24:31.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.134+0000 7f858257c640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "foo"}] 2024-01-24T23:24:31.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.134+0000 7f858257c640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "foo"}] 2024-01-24T23:24:31.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.134+0000 7f858257c640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:31.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.134+0000 7f858257c640 1 -- 172.21.15.132:0/694005482 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush rm", "name": "foo"} v 0) v1 -- 0x7f855c0010a0 con 0x7f857c1377a0 2024-01-24T23:24:31.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.138+0000 7f8578ff9640 1 -- 172.21.15.132:0/694005482 <== mon.8 v1:172.21.15.115:6793/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "foo"}]=0 device 'foo' does not appear in the crush map v94) v1 ==== 120+0+0 (unknown 4255382391 0 0) 0x7f85740149c0 con 0x7f857c1377a0 2024-01-24T23:24:31.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.138+0000 7f8578ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "foo"}] 2024-01-24T23:24:31.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.138+0000 7f8578ff9640 10 monclient: _finish_command 3 = system:0 device 'foo' does not appear in the crush map 2024-01-24T23:24:31.140 INFO:tasks.workunit.client.0.smithi132.stderr:device 'foo' does not appear in the crush map 2024-01-24T23:24:31.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.138+0000 7f858257c640 1 -- 172.21.15.132:0/694005482 >> v1:172.21.15.132:6812/1835401922 conn(0x7f8568036ef0 legacy=0x7f85680393b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:31.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.138+0000 7f858257c640 10 monclient: shutdown 2024-01-24T23:24:31.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.138+0000 7f858257c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:31.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.138+0000 7f858257c640 1 -- 172.21.15.132:0/694005482 >> v1:172.21.15.115:6793/0 conn(0x7f857c1377a0 legacy=0x7f857c1d3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:31.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.138+0000 7f858257c640 1 -- 172.21.15.132:0/694005482 shutdown_connections 2024-01-24T23:24:31.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.138+0000 7f858257c640 1 -- 172.21.15.132:0/694005482 >> 172.21.15.132:0/694005482 conn(0x7f857c0a9c10 msgr2=0x7f857c0a7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:31.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.138+0000 7f858257c640 1 -- 172.21.15.132:0/694005482 shutdown_connections 2024-01-24T23:24:31.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.138+0000 7f858257c640 1 -- 172.21.15.132:0/694005482 wait complete. 2024-01-24T23:24:31.157 INFO:tasks.workunit.client.0.smithi132.stderr:++ ceph osd create 2024-01-24T23:24:31.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.262+0000 7fc94c3eb640 10 monclient: get_monmap_and_config 2024-01-24T23:24:31.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.262+0000 7fc94c3eb640 10 monclient: build_initial_monmap 2024-01-24T23:24:31.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.262+0000 7fc94c3eb640 10 monclient: monmap: 2024-01-24T23:24:31.267 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:31.267 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:31.268698+0000 2024-01-24T23:24:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:31.268698+0000 2024-01-24T23:24:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:31.268 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:31.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 1 Processor -- start 2024-01-24T23:24:31.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 1 -- start start 2024-01-24T23:24:31.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 10 monclient: init 2024-01-24T23:24:31.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:31.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 10 monclient: _add_conns ranks=[4,0,3,6,2,5,8,1,7] 2024-01-24T23:24:31.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 10 monclient(hunting): picked mon.noname-c con 0x7fc944071480 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:31.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 10 monclient(hunting): picked mon.noname-a con 0x7fc944071860 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:31.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 10 monclient(hunting): picked mon.noname-g con 0x7fc944071c40 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:31.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc944064700 con 0x7fc944071860 2024-01-24T23:24:31.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc944133c60 con 0x7fc944071c40 2024-01-24T23:24:31.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc94406d6f0 con 0x7fc944071480 2024-01-24T23:24:31.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:31.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 10 monclient(hunting): authenticate will time out at 1158.593400s 2024-01-24T23:24:31.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94a988640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fc944071c40 0x7fc94413f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:43992/0 (socket says 172.21.15.132:43992) 2024-01-24T23:24:31.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94a988640 1 -- 172.21.15.132:0/2931964674 learned_addr learned my addr 172.21.15.132:0/2931964674 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:31.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94a187640 1 -- 172.21.15.132:0/2931964674 >> v1:172.21.15.115:6791/0 conn(0x7fc944071480 legacy=0x7fc944133550 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:31.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949986640 1 -- 172.21.15.132:0/2931964674 >> v1:172.21.15.115:6789/0 conn(0x7fc944071860 legacy=0x7fc94413bdc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:31.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 1 -- 172.21.15.132:0/2931964674 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1210779407 0 0) 0x7fc944133c60 con 0x7fc944071c40 2024-01-24T23:24:31.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 10 monclient(hunting): _negotiate 2024-01-24T23:24:31.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:31.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:31.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 10 monclient(hunting): my global_id is 5601 2024-01-24T23:24:31.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 1 -- 172.21.15.132:0/2931964674 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc930003530 con 0x7fc944071c40 2024-01-24T23:24:31.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 1 -- 172.21.15.132:0/2931964674 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1138636193 0 0) 0x7fc930003530 con 0x7fc944071c40 2024-01-24T23:24:31.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 1 -- 172.21.15.132:0/2931964674 >> v1:172.21.15.115:6791/0 conn(0x7fc944071480 legacy=0x7fc944133550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:31.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 1 -- 172.21.15.132:0/2931964674 >> v1:172.21.15.115:6789/0 conn(0x7fc944071860 legacy=0x7fc94413bdc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:31.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 10 monclient: _finish_hunting 0 2024-01-24T23:24:31.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 1 monclient: found mon.noname-g 2024-01-24T23:24:31.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:31.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:24:31.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 1 -- 172.21.15.132:0/2931964674 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94409ee50 con 0x7fc944071c40 2024-01-24T23:24:31.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 10 monclient: _finish_auth 0 2024-01-24T23:24:31.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 10 monclient: _check_auth_tickets 2024-01-24T23:24:31.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:31.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 1 -- 172.21.15.132:0/2931964674 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc9400017c0 con 0x7fc944071c40 2024-01-24T23:24:31.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 5 monclient: authenticate success, global_id 5601 2024-01-24T23:24:31.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:31.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:31.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:31.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 10 monclient: dump: 2024-01-24T23:24:31.278 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:31.278 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:31.278 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:31.278 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:31.278 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:31.278 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:31.278 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:31.278 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:31.278 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:31.278 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:31.278 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:31.278 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:31.279 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:31.279 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:31.279 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:31.279 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:31.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:31.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 1 -- 172.21.15.132:0/2931964674 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc940001ae0 con 0x7fc944071c40 2024-01-24T23:24:31.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:31.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:31.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 4 set_mon_vals no callback set 2024-01-24T23:24:31.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 1 -- 172.21.15.132:0/2931964674 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc940004bf0 con 0x7fc944071c40 2024-01-24T23:24:31.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:31.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:31.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc949185640 10 monclient: dump: 2024-01-24T23:24:31.279 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:31.279 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:31.280 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:31.280 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:31.280 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:31.280 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:31.280 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:31.280 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:31.280 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:31.280 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:31.280 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:31.280 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:31.280 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:31.280 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:31.280 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:31.280 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:31.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 10 monclient: shutdown 2024-01-24T23:24:31.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:31.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 1 -- 172.21.15.132:0/2931964674 >> v1:172.21.15.132:6790/0 conn(0x7fc944071c40 legacy=0x7fc94413f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:31.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 1 -- 172.21.15.132:0/2931964674 shutdown_connections 2024-01-24T23:24:31.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 1 -- 172.21.15.132:0/2931964674 >> 172.21.15.132:0/2931964674 conn(0x7fc9440a2340 msgr2=0x7fc9440a2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:31.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 1 -- 172.21.15.132:0/2931964674 shutdown_connections 2024-01-24T23:24:31.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 1 -- 172.21.15.132:0/2931964674 wait complete. 2024-01-24T23:24:31.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 10 monclient: build_initial_monmap 2024-01-24T23:24:31.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 10 monclient: monmap: 2024-01-24T23:24:31.281 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:31.281 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:31.281 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:31.281 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:31.281 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:31.281 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:31.281 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:31.281 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:31.282 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:31.282 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:31.282 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:31.282 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:31.282 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:31.282 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:31.282 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:31.282 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:31.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 1 Processor -- start 2024-01-24T23:24:31.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 1 -- start start 2024-01-24T23:24:31.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 10 monclient: init 2024-01-24T23:24:31.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.266+0000 7fc94c3eb640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:31.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc94c3eb640 10 monclient: _add_conns ranks=[3,0,4,8,5,6,7,1,2] 2024-01-24T23:24:31.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc94c3eb640 10 monclient(hunting): picked mon.noname-g con 0x7fc944071480 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:31.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc94c3eb640 10 monclient(hunting): picked mon.noname-a con 0x7fc944071860 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:31.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc94c3eb640 10 monclient(hunting): picked mon.noname-c con 0x7fc944071c40 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:31.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc94c3eb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc94411d3c0 con 0x7fc944071860 2024-01-24T23:24:31.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc94c3eb640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc94409f9b0 con 0x7fc944071480 2024-01-24T23:24:31.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc94c3eb640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc94411d980 con 0x7fc944071c40 2024-01-24T23:24:31.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc94c3eb640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:31.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc94c3eb640 10 monclient(hunting): authenticate will time out at 1158.596079s 2024-01-24T23:24:31.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc94a187640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fc944071480 0x7fc9441302a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:43994/0 (socket says 172.21.15.132:43994) 2024-01-24T23:24:31.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc94a187640 1 -- 172.21.15.132:0/1826578764 learned_addr learned my addr 172.21.15.132:0/1826578764 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:31.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc949986640 1 -- 172.21.15.132:0/1826578764 >> v1:172.21.15.115:6789/0 conn(0x7fc944071860 legacy=0x7fc9441d02f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:31.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc94a988640 1 -- 172.21.15.132:0/1826578764 >> v1:172.21.15.115:6791/0 conn(0x7fc944071c40 legacy=0x7fc9441d3a20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:31.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 1 -- 172.21.15.132:0/1826578764 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2949552453 0 0) 0x7fc94409f9b0 con 0x7fc944071480 2024-01-24T23:24:31.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:31.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:31.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:31.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 10 monclient(hunting): my global_id is 5610 2024-01-24T23:24:31.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 1 -- 172.21.15.132:0/1826578764 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc928003530 con 0x7fc944071480 2024-01-24T23:24:31.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 1 -- 172.21.15.132:0/1826578764 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2151749151 0 0) 0x7fc928003530 con 0x7fc944071480 2024-01-24T23:24:31.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 1 -- 172.21.15.132:0/1826578764 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc94409f9b0 con 0x7fc944071480 2024-01-24T23:24:31.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 1 -- 172.21.15.132:0/1826578764 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc93c0046b0 con 0x7fc944071480 2024-01-24T23:24:31.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:31.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:31.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 10 monclient(hunting): dump: 2024-01-24T23:24:31.284 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:31.284 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:31.284 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:31.284 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:31.284 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:31.284 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:31.284 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:31.284 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:31.284 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:31.284 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:31.285 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:31.285 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:31.285 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:31.285 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:31.285 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:31.285 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:31.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:31.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 1 -- 172.21.15.132:0/1826578764 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3468113408 0 0) 0x7fc94409f9b0 con 0x7fc944071480 2024-01-24T23:24:31.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 1 -- 172.21.15.132:0/1826578764 >> v1:172.21.15.115:6791/0 conn(0x7fc944071c40 legacy=0x7fc9441d3a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:31.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 1 -- 172.21.15.132:0/1826578764 >> v1:172.21.15.115:6789/0 conn(0x7fc944071860 legacy=0x7fc9441d02f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:31.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:31.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 1 monclient: found mon.g 2024-01-24T23:24:31.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:31.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:31.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 1 -- 172.21.15.132:0/1826578764 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc944137450 con 0x7fc944071480 2024-01-24T23:24:31.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 10 monclient: _finish_auth 0 2024-01-24T23:24:31.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:31.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:31.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc94c3eb640 5 monclient: authenticate success, global_id 5610 2024-01-24T23:24:31.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc94c3eb640 10 monclient: _renew_subs 2024-01-24T23:24:31.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc94c3eb640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:31.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc94c3eb640 1 -- 172.21.15.132:0/1826578764 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9441cbe60 con 0x7fc944071480 2024-01-24T23:24:31.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc94c3eb640 10 monclient: _renew_subs 2024-01-24T23:24:31.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc94c3eb640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:31.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc94c3eb640 1 -- 172.21.15.132:0/1826578764 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc94411c210 con 0x7fc944071480 2024-01-24T23:24:31.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 1 -- 172.21.15.132:0/1826578764 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc93c0048d0 con 0x7fc944071480 2024-01-24T23:24:31.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:31.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 1 -- 172.21.15.132:0/1826578764 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc93c004e00 con 0x7fc944071480 2024-01-24T23:24:31.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:31.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:31.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc948984640 4 set_mon_vals no callback set 2024-01-24T23:24:31.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 10 monclient: dump: 2024-01-24T23:24:31.287 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:31.287 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:31.287 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:31.287 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:31.287 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:31.287 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:31.287 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:31.287 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:31.287 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:31.287 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:31.288 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:31.288 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:31.288 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:31.288 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:31.288 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:31.288 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:31.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 1 -- 172.21.15.132:0/1826578764 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fc93c004fe0 con 0x7fc944071480 2024-01-24T23:24:31.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc94c3eb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:31.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc94c3eb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:31.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc94c3eb640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:31.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc94c3eb640 1 -- 172.21.15.132:0/1826578764 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc92c005090 con 0x7fc944071480 2024-01-24T23:24:31.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc93affd640 1 -- 172.21.15.132:0/1826578764 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(94..94 src has 1..94) v4 ==== 4890+0+0 (unknown 404031622 0 0) 0x7fc93c044030 con 0x7fc944071480 2024-01-24T23:24:31.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.270+0000 7fc949986640 10 monclient: get_auth_request con 0x7fc928036f70 auth_method 0 2024-01-24T23:24:31.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.274+0000 7fc93affd640 1 -- 172.21.15.132:0/1826578764 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fc93c0117a0 con 0x7fc944071480 2024-01-24T23:24:31.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.274+0000 7fc93affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:31.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.274+0000 7fc93affd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:31.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.426+0000 7fc94c3eb640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-01-24T23:24:31.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.426+0000 7fc94c3eb640 10 monclient: _send_command 2 [{"prefix": "osd create"}] 2024-01-24T23:24:31.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.426+0000 7fc94c3eb640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:31.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.426+0000 7fc94c3eb640 1 -- 172.21.15.132:0/1826578764 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd create"} v 0) v1 -- 0x7fc92c0058c0 con 0x7fc944071480 2024-01-24T23:24:31.886 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.882+0000 7fc93affd640 1 -- 172.21.15.132:0/1826578764 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd create"}]=0 v95) v1 ==== 58+0+3 (unknown 543968123 0 520826459) 0x7fc93c0248b0 con 0x7fc944071480 2024-01-24T23:24:31.886 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.882+0000 7fc93affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create"}] 2024-01-24T23:24:31.886 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:31.882+0000 7fc93affd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:24:32.004 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:32.002+0000 7fc94c3eb640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-01-24T23:24:32.004 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:32.002+0000 7fc94c3eb640 10 monclient: _send_command 3 [{"prefix": "osd create"}] 2024-01-24T23:24:32.004 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:32.002+0000 7fc94c3eb640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:32.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:32.002+0000 7fc94c3eb640 1 -- 172.21.15.132:0/1826578764 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd create"} v 0) v1 -- 0x7fc92c005380 con 0x7fc944071480 2024-01-24T23:24:32.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:32.898+0000 7fc93affd640 1 -- 172.21.15.132:0/1826578764 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd create"}]=0 v96) v1 ==== 58+0+3 (unknown 134075996 0 212448812) 0x7fc93c01d590 con 0x7fc944071480 2024-01-24T23:24:32.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:32.898+0000 7fc93affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create"}] 2024-01-24T23:24:32.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:32.898+0000 7fc93affd640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:24:32.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:32.902+0000 7fc94c3eb640 1 -- 172.21.15.132:0/1826578764 >> v1:172.21.15.132:6812/1835401922 conn(0x7fc928036f70 legacy=0x7fc928039430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:32.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:32.902+0000 7fc94c3eb640 10 monclient: shutdown 2024-01-24T23:24:32.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:32.902+0000 7fc94c3eb640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:32.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:32.902+0000 7fc94c3eb640 1 -- 172.21.15.132:0/1826578764 >> v1:172.21.15.132:6790/0 conn(0x7fc944071480 legacy=0x7fc9441302a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:32.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:32.902+0000 7fc94c3eb640 1 -- 172.21.15.132:0/1826578764 shutdown_connections 2024-01-24T23:24:32.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:32.902+0000 7fc94c3eb640 1 -- 172.21.15.132:0/1826578764 >> 172.21.15.132:0/1826578764 conn(0x7fc9440a2340 msgr2=0x7fc9440a3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:32.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:32.902+0000 7fc94c3eb640 1 -- 172.21.15.132:0/1826578764 shutdown_connections 2024-01-24T23:24:32.905 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:32.902+0000 7fc94c3eb640 1 -- 172.21.15.132:0/1826578764 wait complete. 2024-01-24T23:24:32.926 INFO:tasks.workunit.client.0.smithi132.stderr:+ o3=11 2024-01-24T23:24:32.926 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add 11 123 root=default 2024-01-24T23:24:33.047 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 10 monclient: get_monmap_and_config 2024-01-24T23:24:33.047 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 10 monclient: build_initial_monmap 2024-01-24T23:24:33.047 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 10 monclient: monmap: 2024-01-24T23:24:33.047 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:33.048 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:33.048 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:33.048881+0000 2024-01-24T23:24:33.048 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:33.048881+0000 2024-01-24T23:24:33.048 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:33.048 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:33.048 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:33.048 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:33.048 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:33.048 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:33.048 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:33.048 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:33.048 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:33.048 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:33.049 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:33.049 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:33.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 1 Processor -- start 2024-01-24T23:24:33.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 1 -- start start 2024-01-24T23:24:33.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 10 monclient: init 2024-01-24T23:24:33.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:33.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 10 monclient: _add_conns ranks=[5,3,6,2,4,0,8,7,1] 2024-01-24T23:24:33.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 10 monclient(hunting): picked mon.noname-h con 0x7f038c133ed0 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:33.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 10 monclient(hunting): picked mon.noname-g con 0x7f038c1342b0 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:33.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 10 monclient(hunting): picked mon.noname-d con 0x7f038c134690 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:33.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038c064700 con 0x7f038c1342b0 2024-01-24T23:24:33.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038c09b3d0 con 0x7f038c133ed0 2024-01-24T23:24:33.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038c0a58c0 con 0x7f038c134690 2024-01-24T23:24:33.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:33.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 10 monclient(hunting): authenticate will time out at 1160.373450s 2024-01-24T23:24:33.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0391e9e640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f038c133ed0 0x7f038c09bdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:48472/0 (socket says 172.21.15.132:48472) 2024-01-24T23:24:33.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0391e9e640 1 -- 172.21.15.132:0/3190644019 learned_addr learned my addr 172.21.15.132:0/3190644019 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:33.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f039169d640 1 --1- 172.21.15.132:0/3190644019 >> v1:172.21.15.132:6790/0 conn(0x7f038c1342b0 0x7f038c13c040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:43996/0 (socket says 172.21.15.132:43996) 2024-01-24T23:24:33.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 1 -- 172.21.15.132:0/3190644019 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 823980056 0 0) 0x7f038c064700 con 0x7f038c1342b0 2024-01-24T23:24:33.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 10 monclient(hunting): _negotiate 2024-01-24T23:24:33.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:33.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:33.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 10 monclient(hunting): my global_id is 5619 2024-01-24T23:24:33.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 1 -- 172.21.15.132:0/3190644019 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0378003530 con 0x7f038c1342b0 2024-01-24T23:24:33.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 1 -- 172.21.15.132:0/3190644019 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2917532021 0 0) 0x7f038c09b3d0 con 0x7f038c133ed0 2024-01-24T23:24:33.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 10 monclient(hunting): _negotiate 2024-01-24T23:24:33.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:33.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:33.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 10 monclient(hunting): my global_id is 5612 2024-01-24T23:24:33.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 1 -- 172.21.15.132:0/3190644019 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f038c064700 con 0x7f038c133ed0 2024-01-24T23:24:33.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 1 -- 172.21.15.132:0/3190644019 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1374156924 0 0) 0x7f0378003530 con 0x7f038c1342b0 2024-01-24T23:24:33.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 1 -- 172.21.15.132:0/3190644019 >> v1:172.21.15.115:6792/0 conn(0x7f038c134690 legacy=0x7f038c13f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:33.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 1 -- 172.21.15.132:0/3190644019 >> v1:172.21.15.132:6791/0 conn(0x7f038c133ed0 legacy=0x7f038c09bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:33.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 10 monclient: _finish_hunting 0 2024-01-24T23:24:33.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 1 monclient: found mon.noname-g 2024-01-24T23:24:33.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:33.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:24:33.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 1 -- 172.21.15.132:0/3190644019 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038c09b0f0 con 0x7f038c1342b0 2024-01-24T23:24:33.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 10 monclient: _finish_auth 0 2024-01-24T23:24:33.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 10 monclient: _check_auth_tickets 2024-01-24T23:24:33.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:33.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 1 -- 172.21.15.132:0/3190644019 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f03800017c0 con 0x7f038c1342b0 2024-01-24T23:24:33.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 5 monclient: authenticate success, global_id 5619 2024-01-24T23:24:33.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:33.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:33.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:33.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 10 monclient: dump: 2024-01-24T23:24:33.059 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:33.059 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:33.059 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:33.059 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:33.059 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:33.059 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:33.059 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:33.059 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:33.059 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:33.059 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:33.059 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:33.059 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:33.059 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:33.059 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:33.059 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:33.060 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:33.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:33.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 1 -- 172.21.15.132:0/3190644019 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0380001a60 con 0x7f038c1342b0 2024-01-24T23:24:33.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:33.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:33.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 4 set_mon_vals no callback set 2024-01-24T23:24:33.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 1 -- 172.21.15.132:0/3190644019 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0380004b30 con 0x7f038c1342b0 2024-01-24T23:24:33.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:33.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:33.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0390e9c640 10 monclient: dump: 2024-01-24T23:24:33.060 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:33.060 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:33.060 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:33.061 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:33.061 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:33.061 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:33.061 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:33.061 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:33.061 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:33.061 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:33.061 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:33.061 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:33.061 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:33.061 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:33.061 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:33.061 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:33.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 10 monclient: shutdown 2024-01-24T23:24:33.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:33.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 1 -- 172.21.15.132:0/3190644019 >> v1:172.21.15.132:6790/0 conn(0x7f038c1342b0 legacy=0x7f038c13c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:33.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 1 -- 172.21.15.132:0/3190644019 shutdown_connections 2024-01-24T23:24:33.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 1 -- 172.21.15.132:0/3190644019 >> 172.21.15.132:0/3190644019 conn(0x7f038c0a5ef0 msgr2=0x7f038c0a6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:33.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 1 -- 172.21.15.132:0/3190644019 shutdown_connections 2024-01-24T23:24:33.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 1 -- 172.21.15.132:0/3190644019 wait complete. 2024-01-24T23:24:33.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 10 monclient: build_initial_monmap 2024-01-24T23:24:33.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.046+0000 7f0394102640 10 monclient: monmap: 2024-01-24T23:24:33.062 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:33.062 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:33.062 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:33.062 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:33.062 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:33.062 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:33.062 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:33.062 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:33.063 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:33.063 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:33.063 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:33.063 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:33.063 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:33.063 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:33.063 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:33.063 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:33.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0394102640 1 Processor -- start 2024-01-24T23:24:33.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0394102640 1 -- start start 2024-01-24T23:24:33.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0394102640 10 monclient: init 2024-01-24T23:24:33.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0394102640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:33.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0394102640 10 monclient: _add_conns ranks=[6,2,1,5,3,4,8,0,7] 2024-01-24T23:24:33.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0394102640 10 monclient(hunting): picked mon.noname-d con 0x7f038c133ed0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:33.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0394102640 10 monclient(hunting): picked mon.noname-b con 0x7f038c1342b0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:33.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0394102640 10 monclient(hunting): picked mon.noname-f con 0x7f038c134690 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:33.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0394102640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038c0a5b60 con 0x7f038c134690 2024-01-24T23:24:33.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0394102640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038c0a2430 con 0x7f038c1342b0 2024-01-24T23:24:33.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0394102640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038c11fe90 con 0x7f038c133ed0 2024-01-24T23:24:33.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0394102640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:33.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0394102640 10 monclient(hunting): authenticate will time out at 1160.376577s 2024-01-24T23:24:33.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f039269f640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f038c134690 legacy=0x7f038c1d3a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:33.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0391e9e640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f038c133ed0 0x7f038c132d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:49562/0 (socket says 172.21.15.132:49562) 2024-01-24T23:24:33.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0391e9e640 1 -- 172.21.15.132:0/2523473497 learned_addr learned my addr 172.21.15.132:0/2523473497 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:33.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f039169d640 1 --1- 172.21.15.132:0/2523473497 >> v1:172.21.15.115:6790/0 conn(0x7f038c1342b0 0x7f038c1d0310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:43054/0 (socket says 172.21.15.132:43054) 2024-01-24T23:24:33.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 1 -- 172.21.15.132:0/2523473497 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2390709816 0 0) 0x7f038c11fe90 con 0x7f038c133ed0 2024-01-24T23:24:33.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:33.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:33.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:33.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 10 monclient(hunting): my global_id is 5631 2024-01-24T23:24:33.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 1 -- 172.21.15.132:0/2523473497 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0370003530 con 0x7f038c133ed0 2024-01-24T23:24:33.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 1 -- 172.21.15.132:0/2523473497 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 93757997 0 0) 0x7f038c0a2430 con 0x7f038c1342b0 2024-01-24T23:24:33.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:33.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:33.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:33.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 10 monclient(hunting): my global_id is 84890 2024-01-24T23:24:33.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 1 -- 172.21.15.132:0/2523473497 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f038c11fe90 con 0x7f038c1342b0 2024-01-24T23:24:33.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 1 -- 172.21.15.132:0/2523473497 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1809876875 0 0) 0x7f038c11fe90 con 0x7f038c1342b0 2024-01-24T23:24:33.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 1 -- 172.21.15.132:0/2523473497 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f038c0a2430 con 0x7f038c1342b0 2024-01-24T23:24:33.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 1 -- 172.21.15.132:0/2523473497 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 884802603 0 0) 0x7f0370003530 con 0x7f038c133ed0 2024-01-24T23:24:33.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 1 -- 172.21.15.132:0/2523473497 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f038c11fe90 con 0x7f038c133ed0 2024-01-24T23:24:33.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 1 -- 172.21.15.132:0/2523473497 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0380001780 con 0x7f038c1342b0 2024-01-24T23:24:33.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:33.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:24:33.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 10 monclient(hunting): dump: 2024-01-24T23:24:33.066 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:33.066 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:33.066 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:33.066 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:33.066 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:33.066 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:33.066 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:33.066 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:33.066 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:33.066 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:33.066 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:33.067 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:33.067 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:33.067 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:33.067 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:33.067 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:33.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:33.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 1 -- 172.21.15.132:0/2523473497 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0384001880 con 0x7f038c133ed0 2024-01-24T23:24:33.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:33.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:24:33.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 10 monclient(hunting): dump: 2024-01-24T23:24:33.067 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:33.067 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:33.067 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:33.067 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:33.067 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:33.067 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:33.068 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:33.068 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:33.068 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:33.068 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:33.068 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:33.068 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:33.068 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:33.068 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:33.068 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:33.068 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:33.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 1 -- 172.21.15.132:0/2523473497 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2687601673 0 0) 0x7f038c0a2430 con 0x7f038c1342b0 2024-01-24T23:24:33.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 1 -- 172.21.15.132:0/2523473497 >> v1:172.21.15.115:6792/0 conn(0x7f038c133ed0 legacy=0x7f038c132d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:33.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 1 -- 172.21.15.132:0/2523473497 >> v1:172.21.15.132:6789/0 conn(0x7f038c134690 legacy=0x7f038c1d3a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:33.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:24:33.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 1 monclient: found mon.b 2024-01-24T23:24:33.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:33.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:33.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 1 -- 172.21.15.132:0/2523473497 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038c11fad0 con 0x7f038c1342b0 2024-01-24T23:24:33.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 10 monclient: _finish_auth 0 2024-01-24T23:24:33.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:24:33.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:33.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0394102640 5 monclient: authenticate success, global_id 84890 2024-01-24T23:24:33.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0394102640 10 monclient: _renew_subs 2024-01-24T23:24:33.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0394102640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:33.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0394102640 1 -- 172.21.15.132:0/2523473497 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038c0a2f90 con 0x7f038c1342b0 2024-01-24T23:24:33.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0394102640 10 monclient: _renew_subs 2024-01-24T23:24:33.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0394102640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:33.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0394102640 1 -- 172.21.15.132:0/2523473497 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f038c133630 con 0x7f038c1342b0 2024-01-24T23:24:33.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 1 -- 172.21.15.132:0/2523473497 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f03800010f0 con 0x7f038c1342b0 2024-01-24T23:24:33.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:33.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 1 -- 172.21.15.132:0/2523473497 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0380005180 con 0x7f038c1342b0 2024-01-24T23:24:33.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:33.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037ffff640 4 set_mon_vals no callback set 2024-01-24T23:24:33.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:24:33.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 10 monclient: dump: 2024-01-24T23:24:33.070 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:33.070 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:33.070 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:33.070 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:33.070 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:33.070 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:33.070 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:33.070 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:33.070 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:33.071 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:33.071 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:33.071 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:33.071 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:33.071 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:33.071 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:33.071 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:33.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0394102640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:33.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0394102640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:33.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0394102640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:33.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0394102640 1 -- 172.21.15.132:0/2523473497 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0374005090 con 0x7f038c1342b0 2024-01-24T23:24:33.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 1 -- 172.21.15.132:0/2523473497 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f038000fc00 con 0x7f038c1342b0 2024-01-24T23:24:33.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f037e7fc640 1 -- 172.21.15.132:0/2523473497 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(96..96 src has 1..96) v4 ==== 5120+0+0 (unknown 1126670167 0 0) 0x7f0380043d90 con 0x7f038c1342b0 2024-01-24T23:24:33.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.050+0000 7f0391e9e640 10 monclient: get_auth_request con 0x7f03700374f0 auth_method 0 2024-01-24T23:24:33.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.062+0000 7f037e7fc640 1 -- 172.21.15.132:0/2523473497 <== mon.2 v1:172.21.15.115:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f0380014950 con 0x7f038c1342b0 2024-01-24T23:24:33.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.062+0000 7f037e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:33.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.062+0000 7f037e7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:33.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.222+0000 7f0394102640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}] 2024-01-24T23:24:33.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.222+0000 7f0394102640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}] 2024-01-24T23:24:33.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.222+0000 7f0394102640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:33.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.222+0000 7f0394102640 1 -- 172.21.15.132:0/2523473497 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]} v 0) v1 -- 0x7f0374005340 con 0x7f038c1342b0 2024-01-24T23:24:33.909 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.906+0000 7f037e7fc640 1 -- 172.21.15.132:0/2523473497 <== mon.2 v1:172.21.15.115:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}]=0 add item id 11 name 'osd.11' weight 123 at location {root=default} to crush map v97) v1 ==== 193+0+0 (unknown 398427926 0 0) 0x7f0380014770 con 0x7f038c1342b0 2024-01-24T23:24:33.909 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.906+0000 7f037e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}] 2024-01-24T23:24:33.909 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:33.906+0000 7f037e7fc640 10 monclient: _finish_command 2 = system:0 add item id 11 name 'osd.11' weight 123 at location {root=default} to crush map 2024-01-24T23:24:34.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.010+0000 7f0394102640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}] 2024-01-24T23:24:34.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.010+0000 7f0394102640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}] 2024-01-24T23:24:34.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.010+0000 7f0394102640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:34.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.010+0000 7f0394102640 1 -- 172.21.15.132:0/2523473497 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]} v 0) v1 -- 0x7f0374005a80 con 0x7f038c1342b0 2024-01-24T23:24:34.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.014+0000 7f037e7fc640 1 -- 172.21.15.132:0/2523473497 <== mon.2 v1:172.21.15.115:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}]=0 set item id 11 name 'osd.11' weight 123 at location {root=default}: no change v97) v1 ==== 191+0+0 (unknown 1219677271 0 0) 0x7f03800154b0 con 0x7f038c1342b0 2024-01-24T23:24:34.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.014+0000 7f037e7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}] 2024-01-24T23:24:34.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.014+0000 7f037e7fc640 10 monclient: _finish_command 3 = system:0 set item id 11 name 'osd.11' weight 123 at location {root=default}: no change 2024-01-24T23:24:34.017 INFO:tasks.workunit.client.0.smithi132.stderr:set item id 11 name 'osd.11' weight 123 at location {root=default}: no change 2024-01-24T23:24:34.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.018+0000 7f0394102640 1 -- 172.21.15.132:0/2523473497 >> v1:172.21.15.132:6812/1835401922 conn(0x7f03700374f0 legacy=0x7f03700399b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:34.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.018+0000 7f0394102640 10 monclient: shutdown 2024-01-24T23:24:34.020 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.018+0000 7f0394102640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:34.020 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.018+0000 7f0394102640 1 -- 172.21.15.132:0/2523473497 >> v1:172.21.15.115:6790/0 conn(0x7f038c1342b0 legacy=0x7f038c1d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:34.020 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.018+0000 7f0394102640 1 -- 172.21.15.132:0/2523473497 shutdown_connections 2024-01-24T23:24:34.020 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.018+0000 7f0394102640 1 -- 172.21.15.132:0/2523473497 >> 172.21.15.132:0/2523473497 conn(0x7f038c0a5ef0 msgr2=0x7f038c1bea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:34.020 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.018+0000 7f0394102640 1 -- 172.21.15.132:0/2523473497 shutdown_connections 2024-01-24T23:24:34.020 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.018+0000 7f0394102640 1 -- 172.21.15.132:0/2523473497 wait complete. 2024-01-24T23:24:34.035 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd tree 2024-01-24T23:24:34.035 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep osd.11 2024-01-24T23:24:34.035 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep 123 2024-01-24T23:24:34.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c8c021640 10 monclient: get_monmap_and_config 2024-01-24T23:24:34.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c8c021640 10 monclient: build_initial_monmap 2024-01-24T23:24:34.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c8c021640 10 monclient: monmap: 2024-01-24T23:24:34.149 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:34.149 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:34.150 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:34.150845+0000 2024-01-24T23:24:34.150 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:34.150845+0000 2024-01-24T23:24:34.150 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:34.150 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:34.150 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:34.150 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:34.150 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:34.150 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:34.150 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:34.150 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:34.150 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:34.150 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:34.150 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:34.151 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:34.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c8c021640 1 Processor -- start 2024-01-24T23:24:34.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c8c021640 1 -- start start 2024-01-24T23:24:34.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c8c021640 10 monclient: init 2024-01-24T23:24:34.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c8c021640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:34.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c8c021640 10 monclient: _add_conns ranks=[3,1,4,0,8,7,6,2,5] 2024-01-24T23:24:34.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c8c021640 10 monclient(hunting): picked mon.noname-g con 0x7f3c840715f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:34.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c8c021640 10 monclient(hunting): picked mon.noname-f con 0x7f3c84136e90 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:34.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c8c021640 10 monclient(hunting): picked mon.noname-c con 0x7f3c8413d430 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:34.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c8c021640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c84064700 con 0x7f3c84136e90 2024-01-24T23:24:34.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c8c021640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c8406d5a0 con 0x7f3c840715f0 2024-01-24T23:24:34.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c8c021640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c8406dbb0 con 0x7f3c8413d430 2024-01-24T23:24:34.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c8c021640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:34.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c8c021640 10 monclient(hunting): authenticate will time out at 1161.475519s 2024-01-24T23:24:34.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c895bc640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f3c84136e90 legacy=0x7f3c8413bd10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:34.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c89dbd640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f3c840715f0 0x7f3c840719d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:41080/0 (socket says 172.21.15.132:41080) 2024-01-24T23:24:34.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c89dbd640 1 -- 172.21.15.132:0/2198222593 learned_addr learned my addr 172.21.15.132:0/2198222593 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:34.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c8a5be640 1 -- 172.21.15.132:0/2198222593 >> v1:172.21.15.115:6791/0 conn(0x7f3c8413d430 legacy=0x7f3c8413f820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:34.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c88dbb640 1 -- 172.21.15.132:0/2198222593 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4039889819 0 0) 0x7f3c8406d5a0 con 0x7f3c840715f0 2024-01-24T23:24:34.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c88dbb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:34.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c88dbb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:34.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c88dbb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:34.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c88dbb640 10 monclient(hunting): my global_id is 5628 2024-01-24T23:24:34.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.146+0000 7f3c88dbb640 1 -- 172.21.15.132:0/2198222593 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c70003530 con 0x7f3c840715f0 2024-01-24T23:24:34.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c88dbb640 1 -- 172.21.15.132:0/2198222593 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1340221655 0 0) 0x7f3c70003530 con 0x7f3c840715f0 2024-01-24T23:24:34.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c88dbb640 1 -- 172.21.15.132:0/2198222593 >> v1:172.21.15.115:6791/0 conn(0x7f3c8413d430 legacy=0x7f3c8413f820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:34.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c88dbb640 1 -- 172.21.15.132:0/2198222593 >> v1:172.21.15.132:6789/0 conn(0x7f3c84136e90 legacy=0x7f3c8413bd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:34.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c88dbb640 10 monclient: _finish_hunting 0 2024-01-24T23:24:34.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c88dbb640 1 monclient: found mon.noname-g 2024-01-24T23:24:34.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c88dbb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:34.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c88dbb640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:24:34.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c88dbb640 1 -- 172.21.15.132:0/2198222593 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8409f5f0 con 0x7f3c840715f0 2024-01-24T23:24:34.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c88dbb640 10 monclient: _finish_auth 0 2024-01-24T23:24:34.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c88dbb640 10 monclient: _check_auth_tickets 2024-01-24T23:24:34.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c88dbb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:34.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c88dbb640 1 -- 172.21.15.132:0/2198222593 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3c800017c0 con 0x7f3c840715f0 2024-01-24T23:24:34.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c88dbb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:34.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c88dbb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:34.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c88dbb640 10 monclient: dump: 2024-01-24T23:24:34.159 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:34.159 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:34.159 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:34.159 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:34.160 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:34.160 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:34.160 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:34.160 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:34.160 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:34.160 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:34.160 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:34.160 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:34.160 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:34.160 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:34.160 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:34.160 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:34.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 5 monclient: authenticate success, global_id 5628 2024-01-24T23:24:34.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:34.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c88dbb640 1 -- 172.21.15.132:0/2198222593 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3c80001ae0 con 0x7f3c840715f0 2024-01-24T23:24:34.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c88dbb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:34.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:34.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 4 set_mon_vals no callback set 2024-01-24T23:24:34.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c88dbb640 1 -- 172.21.15.132:0/2198222593 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3c80004bf0 con 0x7f3c840715f0 2024-01-24T23:24:34.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c88dbb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:34.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c88dbb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:34.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c88dbb640 10 monclient: dump: 2024-01-24T23:24:34.161 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:34.161 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:34.161 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:34.161 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:34.161 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:34.161 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:34.161 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:34.161 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:34.161 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:34.162 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:34.162 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:34.162 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:34.162 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:34.162 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:34.162 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:34.162 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:34.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 10 monclient: shutdown 2024-01-24T23:24:34.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:34.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 1 -- 172.21.15.132:0/2198222593 >> v1:172.21.15.132:6790/0 conn(0x7f3c840715f0 legacy=0x7f3c840719d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:34.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 1 -- 172.21.15.132:0/2198222593 shutdown_connections 2024-01-24T23:24:34.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 1 -- 172.21.15.132:0/2198222593 >> 172.21.15.132:0/2198222593 conn(0x7f3c840a23b0 msgr2=0x7f3c840a2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:34.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 1 -- 172.21.15.132:0/2198222593 shutdown_connections 2024-01-24T23:24:34.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 1 -- 172.21.15.132:0/2198222593 wait complete. 2024-01-24T23:24:34.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 10 monclient: build_initial_monmap 2024-01-24T23:24:34.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 10 monclient: monmap: 2024-01-24T23:24:34.163 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:34.163 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:34.163 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:34.163 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:34.163 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:34.163 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:34.163 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:34.163 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:34.163 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:34.163 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:34.163 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:34.163 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:34.163 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:34.163 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:34.163 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:34.163 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:34.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 1 Processor -- start 2024-01-24T23:24:34.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 1 -- start start 2024-01-24T23:24:34.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 10 monclient: init 2024-01-24T23:24:34.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:34.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 10 monclient: _add_conns ranks=[0,8,3,4,1,5,6,2,7] 2024-01-24T23:24:34.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 10 monclient(hunting): picked mon.noname-a con 0x7f3c840715f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:34.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 10 monclient(hunting): picked mon.noname-e con 0x7f3c84136e90 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:34.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 10 monclient(hunting): picked mon.noname-g con 0x7f3c8413d430 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:34.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c841337d0 con 0x7f3c840715f0 2024-01-24T23:24:34.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c841cc3d0 con 0x7f3c8413d430 2024-01-24T23:24:34.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c84132370 con 0x7f3c84136e90 2024-01-24T23:24:34.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:34.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 10 monclient(hunting): authenticate will time out at 1161.478440s 2024-01-24T23:24:34.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c89dbd640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f3c840715f0 legacy=0x7f3c841cb5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:34.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8a5be640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f3c8413d430 0x7f3c841d3810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:41086/0 (socket says 172.21.15.132:41086) 2024-01-24T23:24:34.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8a5be640 1 -- 172.21.15.132:0/1636281316 learned_addr learned my addr 172.21.15.132:0/1636281316 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:34.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 1 -- 172.21.15.132:0/1636281316 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3788837252 0 0) 0x7f3c841cc3d0 con 0x7f3c8413d430 2024-01-24T23:24:34.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:34.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:34.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:34.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 10 monclient(hunting): my global_id is 5637 2024-01-24T23:24:34.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 1 -- 172.21.15.132:0/1636281316 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c68003530 con 0x7f3c8413d430 2024-01-24T23:24:34.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 1 -- 172.21.15.132:0/1636281316 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2419213953 0 0) 0x7f3c68003530 con 0x7f3c8413d430 2024-01-24T23:24:34.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 1 -- 172.21.15.132:0/1636281316 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3c841cc3d0 con 0x7f3c8413d430 2024-01-24T23:24:34.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 1 -- 172.21.15.132:0/1636281316 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3c7c0045f0 con 0x7f3c8413d430 2024-01-24T23:24:34.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:34.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:34.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 10 monclient(hunting): dump: 2024-01-24T23:24:34.165 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:34.166 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:34.166 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:34.166 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:34.166 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:34.166 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:34.166 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:34.166 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:34.166 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:34.166 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:34.166 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:34.166 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:34.166 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:34.166 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:34.166 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:34.166 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:34.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:34.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 1 -- 172.21.15.132:0/1636281316 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1985795558 0 0) 0x7f3c84132370 con 0x7f3c84136e90 2024-01-24T23:24:34.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:34.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:34.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:34.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 10 monclient(hunting): my global_id is 84779 2024-01-24T23:24:34.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 1 -- 172.21.15.132:0/1636281316 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c68003530 con 0x7f3c84136e90 2024-01-24T23:24:34.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 1 -- 172.21.15.132:0/1636281316 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3818568677 0 0) 0x7f3c841cc3d0 con 0x7f3c8413d430 2024-01-24T23:24:34.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 1 -- 172.21.15.132:0/1636281316 >> v1:172.21.15.115:6793/0 conn(0x7f3c84136e90 legacy=0x7f3c841cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:34.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 1 -- 172.21.15.132:0/1636281316 >> v1:172.21.15.115:6789/0 conn(0x7f3c840715f0 legacy=0x7f3c841cb5b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:34.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:24:34.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 1 monclient: found mon.g 2024-01-24T23:24:34.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:34.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:34.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 1 -- 172.21.15.132:0/1636281316 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c84132650 con 0x7f3c8413d430 2024-01-24T23:24:34.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 10 monclient: _finish_auth 0 2024-01-24T23:24:34.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:24:34.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:34.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 5 monclient: authenticate success, global_id 5637 2024-01-24T23:24:34.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 10 monclient: _renew_subs 2024-01-24T23:24:34.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:34.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 1 -- 172.21.15.132:0/1636281316 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c84132930 con 0x7f3c8413d430 2024-01-24T23:24:34.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 10 monclient: _renew_subs 2024-01-24T23:24:34.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:34.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c8c021640 1 -- 172.21.15.132:0/1636281316 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c84070900 con 0x7f3c8413d430 2024-01-24T23:24:34.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 1 -- 172.21.15.132:0/1636281316 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3c7c004910 con 0x7f3c8413d430 2024-01-24T23:24:34.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:34.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 1 -- 172.21.15.132:0/1636281316 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3c7c004e40 con 0x7f3c8413d430 2024-01-24T23:24:34.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:34.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7bfff640 4 set_mon_vals no callback set 2024-01-24T23:24:34.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:34.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.150+0000 7f3c7a7fc640 10 monclient: dump: 2024-01-24T23:24:34.169 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:34.169 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:34.169 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:34.169 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:34.169 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:34.169 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:34.169 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:34.169 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:34.169 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:34.169 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:34.169 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:34.169 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:34.169 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:34.169 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:34.169 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:34.170 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:34.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.154+0000 7f3c8c021640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:34.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.154+0000 7f3c8c021640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:34.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.154+0000 7f3c8c021640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:34.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.154+0000 7f3c8c021640 1 -- 172.21.15.132:0/1636281316 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c6c005090 con 0x7f3c8413d430 2024-01-24T23:24:34.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.154+0000 7f3c7a7fc640 1 -- 172.21.15.132:0/1636281316 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f3c7c004fa0 con 0x7f3c8413d430 2024-01-24T23:24:34.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.154+0000 7f3c7a7fc640 1 -- 172.21.15.132:0/1636281316 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(97..97 src has 1..97) v4 ==== 5142+0+0 (unknown 1534292327 0 0) 0x7f3c7c044090 con 0x7f3c8413d430 2024-01-24T23:24:34.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.154+0000 7f3c89dbd640 10 monclient: get_auth_request con 0x7f3c68036fb0 auth_method 0 2024-01-24T23:24:34.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.158+0000 7f3c7a7fc640 1 -- 172.21.15.132:0/1636281316 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f3c7c014e50 con 0x7f3c8413d430 2024-01-24T23:24:34.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.158+0000 7f3c7a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:34.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.158+0000 7f3c7a7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:34.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.314+0000 7f3c8c021640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:34.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.314+0000 7f3c8c021640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:34.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.314+0000 7f3c8c021640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:34.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.314+0000 7f3c8c021640 1 -- 172.21.15.132:0/1636281316 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f3c6c005b50 con 0x7f3c8413d430 2024-01-24T23:24:34.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.314+0000 7f3c7a7fc640 1 -- 172.21.15.132:0/1636281316 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v97) v1 ==== 56+0+1173 (unknown 2315645656 0 3509987380) 0x7f3c7c0132c0 con 0x7f3c8413d430 2024-01-24T23:24:34.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.314+0000 7f3c7a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:34.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.314+0000 7f3c7a7fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:24:34.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.398+0000 7f3c8c021640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:34.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.398+0000 7f3c8c021640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:34.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.398+0000 7f3c8c021640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:34.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.398+0000 7f3c8c021640 1 -- 172.21.15.132:0/1636281316 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f3c6c005ce0 con 0x7f3c8413d430 2024-01-24T23:24:34.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.402+0000 7f3c7a7fc640 1 -- 172.21.15.132:0/1636281316 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v97) v1 ==== 56+0+1173 (unknown 2315645656 0 3509987380) 0x7f3c7c014a40 con 0x7f3c8413d430 2024-01-24T23:24:34.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.402+0000 7f3c7a7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:34.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.402+0000 7f3c7a7fc640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:24:34.406 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.402+0000 7f3c8c021640 1 -- 172.21.15.132:0/1636281316 >> v1:172.21.15.132:6812/1835401922 conn(0x7f3c68036fb0 legacy=0x7f3c68039470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:34.406 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.402+0000 7f3c8c021640 10 monclient: shutdown 2024-01-24T23:24:34.406 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.402+0000 7f3c8c021640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:34.406 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.402+0000 7f3c8c021640 1 -- 172.21.15.132:0/1636281316 >> v1:172.21.15.132:6790/0 conn(0x7f3c8413d430 legacy=0x7f3c841d3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:34.406 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.402+0000 7f3c8c021640 1 -- 172.21.15.132:0/1636281316 shutdown_connections 2024-01-24T23:24:34.406 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.402+0000 7f3c8c021640 1 -- 172.21.15.132:0/1636281316 >> 172.21.15.132:0/1636281316 conn(0x7f3c840a23b0 msgr2=0x7f3c840a1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:34.406 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.402+0000 7f3c8c021640 1 -- 172.21.15.132:0/1636281316 shutdown_connections 2024-01-24T23:24:34.406 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.402+0000 7f3c8c021640 1 -- 172.21.15.132:0/1636281316 wait complete. 2024-01-24T23:24:34.419 INFO:tasks.workunit.client.0.smithi132.stdout: 11 123.00000 osd.11 down 1.00000 1.00000 2024-01-24T23:24:34.420 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush reweight osd.11 113 2024-01-24T23:24:34.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 10 monclient: get_monmap_and_config 2024-01-24T23:24:34.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 10 monclient: build_initial_monmap 2024-01-24T23:24:34.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 10 monclient: monmap: 2024-01-24T23:24:34.536 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:34.536 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:34.536 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:34.537140+0000 2024-01-24T23:24:34.537 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:34.537140+0000 2024-01-24T23:24:34.537 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:34.537 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:34.537 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:34.537 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:34.537 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:34.537 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:34.537 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:34.537 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:34.537 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:34.537 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:34.537 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:34.537 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:34.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 1 Processor -- start 2024-01-24T23:24:34.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 1 -- start start 2024-01-24T23:24:34.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 10 monclient: init 2024-01-24T23:24:34.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:34.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 10 monclient: _add_conns ranks=[2,3,6,8,4,0,5,7,1] 2024-01-24T23:24:34.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 10 monclient(hunting): picked mon.noname-b con 0x7fdef009ff20 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:34.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 10 monclient(hunting): picked mon.noname-g con 0x7fdef009f9c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:34.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 10 monclient(hunting): picked mon.noname-d con 0x7fdef013b2c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:34.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef0064700 con 0x7fdef009ff20 2024-01-24T23:24:34.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef009e760 con 0x7fdef009f9c0 2024-01-24T23:24:34.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef00a1160 con 0x7fdef013b2c0 2024-01-24T23:24:34.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:34.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 10 monclient(hunting): authenticate will time out at 1161.861683s 2024-01-24T23:24:34.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef51cc640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fdef009f9c0 0x7fdef0139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:41090/0 (socket says 172.21.15.132:41090) 2024-01-24T23:24:34.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef51cc640 1 -- 172.21.15.132:0/707468595 learned_addr learned my addr 172.21.15.132:0/707468595 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:34.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 1 -- 172.21.15.132:0/707468595 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1767553678 0 0) 0x7fdef009e760 con 0x7fdef009f9c0 2024-01-24T23:24:34.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:34.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:34.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:34.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 10 monclient(hunting): my global_id is 5646 2024-01-24T23:24:34.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 1 -- 172.21.15.132:0/707468595 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdedc003530 con 0x7fdef009f9c0 2024-01-24T23:24:34.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 1 -- 172.21.15.132:0/707468595 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2880023444 0 0) 0x7fdedc003530 con 0x7fdef009f9c0 2024-01-24T23:24:34.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 1 -- 172.21.15.132:0/707468595 >> v1:172.21.15.115:6792/0 conn(0x7fdef013b2c0 legacy=0x7fdef013d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:34.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 1 -- 172.21.15.132:0/707468595 >> v1:172.21.15.115:6790/0 conn(0x7fdef009ff20 legacy=0x7fdef00a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:34.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 10 monclient: _finish_hunting 0 2024-01-24T23:24:34.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 1 monclient: found mon.noname-g 2024-01-24T23:24:34.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:34.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:24:34.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 1 -- 172.21.15.132:0/707468595 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef00a0a10 con 0x7fdef009f9c0 2024-01-24T23:24:34.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 10 monclient: _finish_auth 0 2024-01-24T23:24:34.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 10 monclient: _check_auth_tickets 2024-01-24T23:24:34.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:34.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 1 -- 172.21.15.132:0/707468595 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdee00017c0 con 0x7fdef009f9c0 2024-01-24T23:24:34.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:34.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:34.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 10 monclient: dump: 2024-01-24T23:24:34.548 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:34.548 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:34.548 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:34.548 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:34.548 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:34.548 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:34.548 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:34.548 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:34.548 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:34.548 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:34.548 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:34.548 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:34.549 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:34.549 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:34.549 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:34.549 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:34.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 5 monclient: authenticate success, global_id 5646 2024-01-24T23:24:34.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:34.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 1 -- 172.21.15.132:0/707468595 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdee0001ae0 con 0x7fdef009f9c0 2024-01-24T23:24:34.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:34.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 1 -- 172.21.15.132:0/707468595 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdee0004bf0 con 0x7fdef009f9c0 2024-01-24T23:24:34.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:34.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 4 set_mon_vals no callback set 2024-01-24T23:24:34.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:34.550 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:34.550 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef49cb640 10 monclient: dump: 2024-01-24T23:24:34.550 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:34.550 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:34.550 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:34.550 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:34.550 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:34.550 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:34.550 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:34.550 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:34.550 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:34.550 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:34.550 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:34.551 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:34.551 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:34.551 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:34.551 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:34.551 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:34.551 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 10 monclient: shutdown 2024-01-24T23:24:34.551 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:34.551 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 1 -- 172.21.15.132:0/707468595 >> v1:172.21.15.132:6790/0 conn(0x7fdef009f9c0 legacy=0x7fdef0139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:34.551 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 1 -- 172.21.15.132:0/707468595 shutdown_connections 2024-01-24T23:24:34.551 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 1 -- 172.21.15.132:0/707468595 >> 172.21.15.132:0/707468595 conn(0x7fdef00a5a10 msgr2=0x7fdef00a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:34.551 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 1 -- 172.21.15.132:0/707468595 shutdown_connections 2024-01-24T23:24:34.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 1 -- 172.21.15.132:0/707468595 wait complete. 2024-01-24T23:24:34.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 10 monclient: build_initial_monmap 2024-01-24T23:24:34.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.534+0000 7fdef7c31640 10 monclient: monmap: 2024-01-24T23:24:34.552 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:34.552 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:34.552 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:34.552 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:34.552 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:34.552 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:34.552 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:34.552 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:34.552 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:34.552 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:34.553 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:34.553 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:34.553 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:34.553 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:34.553 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:34.553 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:34.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef7c31640 1 Processor -- start 2024-01-24T23:24:34.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef7c31640 1 -- start start 2024-01-24T23:24:34.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef7c31640 10 monclient: init 2024-01-24T23:24:34.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef7c31640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:34.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef7c31640 10 monclient: _add_conns ranks=[8,6,0,2,5,7,4,3,1] 2024-01-24T23:24:34.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef7c31640 10 monclient(hunting): picked mon.noname-e con 0x7fdef009f9c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:34.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef7c31640 10 monclient(hunting): picked mon.noname-d con 0x7fdef009ff20 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:34.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef7c31640 10 monclient(hunting): picked mon.noname-a con 0x7fdef013b2c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:34.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef7c31640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef0133180 con 0x7fdef013b2c0 2024-01-24T23:24:34.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef7c31640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef011dcb0 con 0x7fdef009ff20 2024-01-24T23:24:34.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef7c31640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef0071680 con 0x7fdef009f9c0 2024-01-24T23:24:34.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef7c31640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:34.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef7c31640 10 monclient(hunting): authenticate will time out at 1161.864564s 2024-01-24T23:24:34.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef61ce640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fdef013b2c0 legacy=0x7fdef01d3eb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:34.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef59cd640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fdef009f9c0 0x7fdef0132a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:37740/0 (socket says 172.21.15.132:37740) 2024-01-24T23:24:34.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef59cd640 1 -- 172.21.15.132:0/240065599 learned_addr learned my addr 172.21.15.132:0/240065599 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:34.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 1 -- 172.21.15.132:0/240065599 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2446119941 0 0) 0x7fdef011dcb0 con 0x7fdef009ff20 2024-01-24T23:24:34.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:34.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:34.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:34.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 10 monclient(hunting): my global_id is 5649 2024-01-24T23:24:34.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 1 -- 172.21.15.132:0/240065599 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fded4003560 con 0x7fdef009ff20 2024-01-24T23:24:34.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 1 -- 172.21.15.132:0/240065599 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1991147022 0 0) 0x7fdef0071680 con 0x7fdef009f9c0 2024-01-24T23:24:34.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:34.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:34.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:34.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 10 monclient(hunting): my global_id is 84788 2024-01-24T23:24:34.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 1 -- 172.21.15.132:0/240065599 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdef011dcb0 con 0x7fdef009f9c0 2024-01-24T23:24:34.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 1 -- 172.21.15.132:0/240065599 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1489439298 0 0) 0x7fded4003560 con 0x7fdef009ff20 2024-01-24T23:24:34.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 1 -- 172.21.15.132:0/240065599 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdef0071680 con 0x7fdef009ff20 2024-01-24T23:24:34.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 1 -- 172.21.15.132:0/240065599 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdee0001780 con 0x7fdef009ff20 2024-01-24T23:24:34.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:34.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:24:34.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 10 monclient(hunting): dump: 2024-01-24T23:24:34.556 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:34.556 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:34.556 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:34.556 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:34.556 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:34.556 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:34.556 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:34.556 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:34.556 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:34.556 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:34.557 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:34.557 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:34.557 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:34.557 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:34.557 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:34.557 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:34.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:34.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 1 -- 172.21.15.132:0/240065599 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2458755672 0 0) 0x7fdef011dcb0 con 0x7fdef009f9c0 2024-01-24T23:24:34.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 1 -- 172.21.15.132:0/240065599 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fded4003560 con 0x7fdef009f9c0 2024-01-24T23:24:34.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 1 -- 172.21.15.132:0/240065599 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdeec001e80 con 0x7fdef009f9c0 2024-01-24T23:24:34.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:34.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:24:34.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 10 monclient(hunting): dump: 2024-01-24T23:24:34.558 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:34.558 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:34.558 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:34.558 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:34.558 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:34.558 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:34.558 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:34.558 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:34.558 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:34.558 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:34.558 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:34.558 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:34.558 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:34.558 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:34.559 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:34.559 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:34.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 1 -- 172.21.15.132:0/240065599 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2466708134 0 0) 0x7fdef0071680 con 0x7fdef009ff20 2024-01-24T23:24:34.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 1 -- 172.21.15.132:0/240065599 >> v1:172.21.15.115:6793/0 conn(0x7fdef009f9c0 legacy=0x7fdef0132a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:34.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 1 -- 172.21.15.132:0/240065599 >> v1:172.21.15.115:6789/0 conn(0x7fdef013b2c0 legacy=0x7fdef01d3eb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:34.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 10 monclient: _finish_hunting 0 2024-01-24T23:24:34.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 1 monclient: found mon.d 2024-01-24T23:24:34.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:34.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:34.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 1 -- 172.21.15.132:0/240065599 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef011e810 con 0x7fdef009ff20 2024-01-24T23:24:34.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 10 monclient: _finish_auth 0 2024-01-24T23:24:34.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 10 monclient: _check_auth_tickets 2024-01-24T23:24:34.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:34.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef7c31640 5 monclient: authenticate success, global_id 5649 2024-01-24T23:24:34.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef7c31640 10 monclient: _renew_subs 2024-01-24T23:24:34.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef7c31640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:34.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef7c31640 1 -- 172.21.15.132:0/240065599 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef011ffa0 con 0x7fdef009ff20 2024-01-24T23:24:34.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef7c31640 10 monclient: _renew_subs 2024-01-24T23:24:34.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef7c31640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:34.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef7c31640 1 -- 172.21.15.132:0/240065599 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef01c08c0 con 0x7fdef009ff20 2024-01-24T23:24:34.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 1 -- 172.21.15.132:0/240065599 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdee0001210 con 0x7fdef009ff20 2024-01-24T23:24:34.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:34.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 1 -- 172.21.15.132:0/240065599 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdee0004c10 con 0x7fdef009ff20 2024-01-24T23:24:34.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:34.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee7fff640 4 set_mon_vals no callback set 2024-01-24T23:24:34.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:24:34.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 10 monclient: dump: 2024-01-24T23:24:34.561 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:34.561 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:34.561 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:34.561 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:34.561 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:34.561 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:34.561 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:34.561 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:34.561 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:34.561 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:34.561 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:34.561 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:34.562 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:34.562 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:34.562 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:34.562 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:34.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef7c31640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:34.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef7c31640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:34.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef7c31640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:34.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef7c31640 1 -- 172.21.15.132:0/240065599 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fded8005090 con 0x7fdef009ff20 2024-01-24T23:24:34.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 1 -- 172.21.15.132:0/240065599 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fdee000fdd0 con 0x7fdef009ff20 2024-01-24T23:24:34.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdee67fc640 1 -- 172.21.15.132:0/240065599 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(97..97 src has 1..97) v4 ==== 5142+0+0 (unknown 1534292327 0 0) 0x7fdee0043e60 con 0x7fdef009ff20 2024-01-24T23:24:34.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.538+0000 7fdef59cd640 10 monclient: get_auth_request con 0x7fded4037700 auth_method 0 2024-01-24T23:24:34.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.550+0000 7fdee67fc640 1 -- 172.21.15.132:0/240065599 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fdee0014bd0 con 0x7fdef009ff20 2024-01-24T23:24:34.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.550+0000 7fdee67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:34.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.550+0000 7fdee67fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:34.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.710+0000 7fdef7c31640 10 start_mon_command cmd=[{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}] 2024-01-24T23:24:34.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.710+0000 7fdef7c31640 10 monclient: _send_command 2 [{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}] 2024-01-24T23:24:34.715 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.710+0000 7fdef7c31640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:34.715 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.710+0000 7fdef7c31640 1 -- 172.21.15.132:0/240065599 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0} v 0) v1 -- 0x7fded8005b50 con 0x7fdef009ff20 2024-01-24T23:24:34.929 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.926+0000 7fdee67fc640 1 -- 172.21.15.132:0/240065599 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}]=0 reweighted item id 11 name 'osd.11' to 113 in crush map v98) v1 ==== 156+0+0 (unknown 3999559540 0 0) 0x7fdee00147c0 con 0x7fdef009ff20 2024-01-24T23:24:34.929 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.926+0000 7fdee67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}] 2024-01-24T23:24:34.929 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:34.926+0000 7fdee67fc640 10 monclient: _finish_command 2 = system:0 reweighted item id 11 name 'osd.11' to 113 in crush map 2024-01-24T23:24:35.036 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:35.034+0000 7fdef7c31640 10 start_mon_command cmd=[{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}] 2024-01-24T23:24:35.036 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:35.034+0000 7fdef7c31640 10 monclient: _send_command 3 [{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}] 2024-01-24T23:24:35.036 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:35.034+0000 7fdef7c31640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:35.037 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:35.034+0000 7fdef7c31640 1 -- 172.21.15.132:0/240065599 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0} v 0) v1 -- 0x7fded80010a0 con 0x7fdef009ff20 2024-01-24T23:24:35.957 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:35.954+0000 7fdee67fc640 1 -- 172.21.15.132:0/240065599 <== mon.6 v1:172.21.15.115:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}]=0 reweighted item id 11 name 'osd.11' to 113 in crush map v99) v1 ==== 156+0+0 (unknown 3517252850 0 0) 0x7fdee00149a0 con 0x7fdef009ff20 2024-01-24T23:24:35.957 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:35.954+0000 7fdee67fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}] 2024-01-24T23:24:35.958 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:35.954+0000 7fdee67fc640 10 monclient: _finish_command 3 = system:0 reweighted item id 11 name 'osd.11' to 113 in crush map 2024-01-24T23:24:35.958 INFO:tasks.workunit.client.0.smithi132.stderr:reweighted item id 11 name 'osd.11' to 113 in crush map 2024-01-24T23:24:35.961 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:35.958+0000 7fdef7c31640 1 -- 172.21.15.132:0/240065599 >> v1:172.21.15.132:6812/1835401922 conn(0x7fded4037700 legacy=0x7fded4039bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:35.961 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:35.958+0000 7fdef7c31640 10 monclient: shutdown 2024-01-24T23:24:35.961 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:35.958+0000 7fdef7c31640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:35.961 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:35.958+0000 7fdef7c31640 1 -- 172.21.15.132:0/240065599 >> v1:172.21.15.115:6792/0 conn(0x7fdef009ff20 legacy=0x7fdef01d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:35.961 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:35.958+0000 7fdef7c31640 1 -- 172.21.15.132:0/240065599 shutdown_connections 2024-01-24T23:24:35.962 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:35.958+0000 7fdef7c31640 1 -- 172.21.15.132:0/240065599 >> 172.21.15.132:0/240065599 conn(0x7fdef00a5a10 msgr2=0x7fdef00a36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:35.962 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:35.958+0000 7fdef7c31640 1 -- 172.21.15.132:0/240065599 shutdown_connections 2024-01-24T23:24:35.962 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:35.958+0000 7fdef7c31640 1 -- 172.21.15.132:0/240065599 wait complete. 2024-01-24T23:24:35.979 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd crush reweight osd.11 123456 2024-01-24T23:24:35.979 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:24:35.979 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush reweight osd.11 123456 2024-01-24T23:24:36.099 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.094+0000 7f99e7cb6640 10 monclient: get_monmap_and_config 2024-01-24T23:24:36.099 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e7cb6640 10 monclient: build_initial_monmap 2024-01-24T23:24:36.100 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e7cb6640 10 monclient: monmap: 2024-01-24T23:24:36.100 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:36.100 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:36.100 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:36.100918+0000 2024-01-24T23:24:36.100 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:36.100918+0000 2024-01-24T23:24:36.100 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:36.100 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:36.100 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:36.100 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:36.100 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:36.100 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:36.100 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:36.101 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:36.101 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:36.101 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:36.101 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:36.101 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:36.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e7cb6640 1 Processor -- start 2024-01-24T23:24:36.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e7cb6640 1 -- start start 2024-01-24T23:24:36.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e7cb6640 10 monclient: init 2024-01-24T23:24:36.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e7cb6640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:36.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e7cb6640 10 monclient: _add_conns ranks=[1,6,4,8,3,0,7,2,5] 2024-01-24T23:24:36.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e7cb6640 10 monclient(hunting): picked mon.noname-f con 0x7f99e00740e0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:36.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e7cb6640 10 monclient(hunting): picked mon.noname-d con 0x7f99e0071da0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:36.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e7cb6640 10 monclient(hunting): picked mon.noname-c con 0x7f99e0136b30 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:36.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e7cb6640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e0064700 con 0x7f99e00740e0 2024-01-24T23:24:36.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e7cb6640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e006d740 con 0x7f99e0136b30 2024-01-24T23:24:36.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e7cb6640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e0136fa0 con 0x7f99e0071da0 2024-01-24T23:24:36.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e5a52640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f99e00740e0 legacy=0x7f99e0071690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:36.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e7cb6640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:36.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e7cb6640 10 monclient(hunting): authenticate will time out at 1163.426044s 2024-01-24T23:24:36.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e6253640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f99e0136b30 legacy=0x7f99e013f500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:36.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e5251640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f99e0071da0 0x7f99e013bdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:49572/0 (socket says 172.21.15.132:49572) 2024-01-24T23:24:36.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e5251640 1 -- 172.21.15.132:0/3269223376 learned_addr learned my addr 172.21.15.132:0/3269223376 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:36.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 1 -- 172.21.15.132:0/3269223376 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1912379904 0 0) 0x7f99e0136fa0 con 0x7f99e0071da0 2024-01-24T23:24:36.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 10 monclient(hunting): _negotiate 2024-01-24T23:24:36.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:36.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:36.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 10 monclient(hunting): my global_id is 5658 2024-01-24T23:24:36.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 1 -- 172.21.15.132:0/3269223376 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99c8003530 con 0x7f99e0071da0 2024-01-24T23:24:36.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 1 -- 172.21.15.132:0/3269223376 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 416499507 0 0) 0x7f99c8003530 con 0x7f99e0071da0 2024-01-24T23:24:36.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 1 -- 172.21.15.132:0/3269223376 >> v1:172.21.15.115:6791/0 conn(0x7f99e0136b30 legacy=0x7f99e013f500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:36.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 1 -- 172.21.15.132:0/3269223376 >> v1:172.21.15.132:6789/0 conn(0x7f99e00740e0 legacy=0x7f99e0071690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:36.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 10 monclient: _finish_hunting 0 2024-01-24T23:24:36.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 1 monclient: found mon.noname-d 2024-01-24T23:24:36.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:36.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:24:36.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 1 -- 172.21.15.132:0/3269223376 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e00744c0 con 0x7f99e0071da0 2024-01-24T23:24:36.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 10 monclient: _finish_auth 0 2024-01-24T23:24:36.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 10 monclient: _check_auth_tickets 2024-01-24T23:24:36.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:36.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 1 -- 172.21.15.132:0/3269223376 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f99d8001b40 con 0x7f99e0071da0 2024-01-24T23:24:36.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e7cb6640 5 monclient: authenticate success, global_id 5658 2024-01-24T23:24:36.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e7cb6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:36.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:36.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:24:36.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 10 monclient: dump: 2024-01-24T23:24:36.107 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:36.107 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:36.107 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.107 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.107 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:36.107 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:36.107 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:36.107 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:36.107 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:36.107 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:36.107 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:36.107 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:36.107 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:36.107 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:36.108 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:36.108 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:36.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e7cb6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:36.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 1 -- 172.21.15.132:0/3269223376 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f99d8001d60 con 0x7f99e0071da0 2024-01-24T23:24:36.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:36.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e7cb6640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:36.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e7cb6640 4 set_mon_vals no callback set 2024-01-24T23:24:36.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 1 -- 172.21.15.132:0/3269223376 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f99d8004f70 con 0x7f99e0071da0 2024-01-24T23:24:36.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:36.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:24:36.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.098+0000 7f99e4a50640 10 monclient: dump: 2024-01-24T23:24:36.108 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:36.108 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:36.108 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.108 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.109 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:36.109 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:36.109 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:36.109 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:36.109 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:36.109 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:36.109 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:36.109 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:36.109 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:36.109 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:36.109 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:36.109 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:36.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 10 monclient: shutdown 2024-01-24T23:24:36.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:36.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 1 -- 172.21.15.132:0/3269223376 >> v1:172.21.15.115:6792/0 conn(0x7f99e0071da0 legacy=0x7f99e013bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:36.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 1 -- 172.21.15.132:0/3269223376 shutdown_connections 2024-01-24T23:24:36.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 1 -- 172.21.15.132:0/3269223376 >> 172.21.15.132:0/3269223376 conn(0x7f99e00a5eb0 msgr2=0x7f99e00a23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:36.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 1 -- 172.21.15.132:0/3269223376 shutdown_connections 2024-01-24T23:24:36.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 1 -- 172.21.15.132:0/3269223376 wait complete. 2024-01-24T23:24:36.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 10 monclient: build_initial_monmap 2024-01-24T23:24:36.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 10 monclient: monmap: 2024-01-24T23:24:36.114 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:36.114 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:36.114 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:36.114 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:36.114 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:36.114 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:36.114 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:36.114 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:36.114 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:36.115 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:36.115 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:36.115 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:36.115 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:36.115 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:36.115 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:36.115 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:36.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 1 Processor -- start 2024-01-24T23:24:36.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 1 -- start start 2024-01-24T23:24:36.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 10 monclient: init 2024-01-24T23:24:36.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:36.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 10 monclient: _add_conns ranks=[8,7,5,2,1,6,0,4,3] 2024-01-24T23:24:36.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 10 monclient(hunting): picked mon.noname-e con 0x7f99e0071da0 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:36.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 10 monclient(hunting): picked mon.noname-i con 0x7f99e00740e0 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:36.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 10 monclient(hunting): picked mon.noname-h con 0x7f99e0136b30 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:36.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e009f920 con 0x7f99e0136b30 2024-01-24T23:24:36.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e0134220 con 0x7f99e00740e0 2024-01-24T23:24:36.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e01c8600 con 0x7f99e0071da0 2024-01-24T23:24:36.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:36.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 10 monclient(hunting): authenticate will time out at 1163.430051s 2024-01-24T23:24:36.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e6253640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f99e0136b30 0x7f99e01d3b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:57490/0 (socket says 172.21.15.132:57490) 2024-01-24T23:24:36.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e6253640 1 -- 172.21.15.132:0/3804352651 learned_addr learned my addr 172.21.15.132:0/3804352651 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:36.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e5251640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f99e00740e0 0x7f99e01d0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:35586/0 (socket says 172.21.15.132:35586) 2024-01-24T23:24:36.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 1 -- 172.21.15.132:0/3804352651 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113572598 0 0) 0x7f99e009f920 con 0x7f99e0136b30 2024-01-24T23:24:36.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:36.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:36.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:36.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient(hunting): my global_id is 5621 2024-01-24T23:24:36.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 1 -- 172.21.15.132:0/3804352651 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99cc003530 con 0x7f99e0136b30 2024-01-24T23:24:36.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 1 -- 172.21.15.132:0/3804352651 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3849828927 0 0) 0x7f99e0134220 con 0x7f99e00740e0 2024-01-24T23:24:36.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:36.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:36.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:36.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient(hunting): my global_id is 5623 2024-01-24T23:24:36.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 1 -- 172.21.15.132:0/3804352651 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99e009f920 con 0x7f99e00740e0 2024-01-24T23:24:36.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 1 -- 172.21.15.132:0/3804352651 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2152594002 0 0) 0x7f99cc003530 con 0x7f99e0136b30 2024-01-24T23:24:36.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 1 -- 172.21.15.132:0/3804352651 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f99e0134220 con 0x7f99e0136b30 2024-01-24T23:24:36.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 1 -- 172.21.15.132:0/3804352651 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 102032094 0 0) 0x7f99e01c8600 con 0x7f99e0071da0 2024-01-24T23:24:36.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:36.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:36.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:36.118 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient(hunting): my global_id is 84797 2024-01-24T23:24:36.118 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 1 -- 172.21.15.132:0/3804352651 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99cc003530 con 0x7f99e0071da0 2024-01-24T23:24:36.118 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 1 -- 172.21.15.132:0/3804352651 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f99d00041e0 con 0x7f99e0136b30 2024-01-24T23:24:36.118 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:36.118 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:36.118 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient(hunting): dump: 2024-01-24T23:24:36.118 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:36.118 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:36.118 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.118 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.118 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:36.118 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:36.118 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:36.119 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:36.119 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:36.119 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:36.119 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:36.119 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:36.119 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:36.119 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:36.119 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:36.119 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:36.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:36.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 1 -- 172.21.15.132:0/3804352651 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 236095002 0 0) 0x7f99e009f920 con 0x7f99e00740e0 2024-01-24T23:24:36.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 1 -- 172.21.15.132:0/3804352651 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f99e01c8600 con 0x7f99e00740e0 2024-01-24T23:24:36.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 1 -- 172.21.15.132:0/3804352651 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f99d8001b00 con 0x7f99e00740e0 2024-01-24T23:24:36.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:36.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:36.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient(hunting): dump: 2024-01-24T23:24:36.120 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:36.120 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:36.120 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.120 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.120 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:36.120 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:36.120 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:36.120 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:36.120 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:36.120 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:36.120 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:36.120 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:36.121 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:36.121 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:36.121 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:36.121 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:36.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 1 -- 172.21.15.132:0/3804352651 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1353668404 0 0) 0x7f99e0134220 con 0x7f99e0136b30 2024-01-24T23:24:36.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 1 -- 172.21.15.132:0/3804352651 >> v1:172.21.15.115:6793/0 conn(0x7f99e0071da0 legacy=0x7f99e0130550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:36.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 1 -- 172.21.15.132:0/3804352651 >> v1:172.21.15.132:6792/0 conn(0x7f99e00740e0 legacy=0x7f99e01d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:36.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient: _finish_hunting 0 2024-01-24T23:24:36.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 1 monclient: found mon.h 2024-01-24T23:24:36.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:36.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:36.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 1 -- 172.21.15.132:0/3804352651 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e011d670 con 0x7f99e0136b30 2024-01-24T23:24:36.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient: _finish_auth 0 2024-01-24T23:24:36.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient: _check_auth_tickets 2024-01-24T23:24:36.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:36.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 5 monclient: authenticate success, global_id 5621 2024-01-24T23:24:36.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 10 monclient: _renew_subs 2024-01-24T23:24:36.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:36.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 1 -- 172.21.15.132:0/3804352651 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e011d950 con 0x7f99e0136b30 2024-01-24T23:24:36.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 1 -- 172.21.15.132:0/3804352651 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f99d0004440 con 0x7f99e0136b30 2024-01-24T23:24:36.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:36.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 1 -- 172.21.15.132:0/3804352651 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f99d0004970 con 0x7f99e0136b30 2024-01-24T23:24:36.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:36.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:36.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d77fe640 4 set_mon_vals no callback set 2024-01-24T23:24:36.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99d67fc640 10 monclient: dump: 2024-01-24T23:24:36.122 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:36.123 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:36.123 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.123 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.123 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:36.123 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:36.123 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:36.123 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:36.123 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:36.123 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:36.123 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:36.123 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:36.123 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:36.123 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:36.123 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:36.124 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:36.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 10 monclient: _renew_subs 2024-01-24T23:24:36.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:36.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.102+0000 7f99e7cb6640 1 -- 172.21.15.132:0/3804352651 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e011c930 con 0x7f99e0136b30 2024-01-24T23:24:36.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.106+0000 7f99d67fc640 1 -- 172.21.15.132:0/3804352651 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f99d0004ad0 con 0x7f99e0136b30 2024-01-24T23:24:36.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.106+0000 7f99d67fc640 1 -- 172.21.15.132:0/3804352651 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(99..99 src has 1..99) v4 ==== 5200+0+0 (unknown 2556180564 0 0) 0x7f99d0043d30 con 0x7f99e0136b30 2024-01-24T23:24:36.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.106+0000 7f99e5a52640 10 monclient: get_auth_request con 0x7f99cc037900 auth_method 0 2024-01-24T23:24:36.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.106+0000 7f99e7cb6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:36.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.106+0000 7f99e7cb6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:36.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.106+0000 7f99e7cb6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:36.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.106+0000 7f99e7cb6640 1 -- 172.21.15.132:0/3804352651 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e006d970 con 0x7f99e0136b30 2024-01-24T23:24:36.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.110+0000 7f99d67fc640 1 -- 172.21.15.132:0/3804352651 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f99e006d970 con 0x7f99e0136b30 2024-01-24T23:24:36.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.110+0000 7f99d67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:36.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.110+0000 7f99d67fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:36.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.270+0000 7f99e7cb6640 10 start_mon_command cmd=[{"prefix": "osd crush reweight", "name": "osd.11", "weight": 123456.0}] 2024-01-24T23:24:36.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.270+0000 7f99e7cb6640 10 monclient: _send_command 2 [{"prefix": "osd crush reweight", "name": "osd.11", "weight": 123456.0}] 2024-01-24T23:24:36.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.270+0000 7f99e7cb6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:36.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.270+0000 7f99e7cb6640 1 -- 172.21.15.132:0/3804352651 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush reweight", "name": "osd.11", "weight": 123456.0} v 0) v1 -- 0x7f99e006d970 con 0x7f99e0136b30 2024-01-24T23:24:36.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.270+0000 7f99d67fc640 1 -- 172.21.15.132:0/3804352651 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush reweight", "name": "osd.11", "weight": 123456.0}]=-75 (75) Value too large for defined data type v99) v1 ==== 146+0+0 (unknown 799715928 0 0) 0x7f99e006d970 con 0x7f99e0136b30 2024-01-24T23:24:36.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.270+0000 7f99d67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush reweight", "name": "osd.11", "weight": 123456.0}] 2024-01-24T23:24:36.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.270+0000 7f99d67fc640 10 monclient: _finish_command 2 = mon:75 (75) Value too large for defined data type 2024-01-24T23:24:36.273 INFO:tasks.workunit.client.0.smithi132.stderr:Error EOVERFLOW: (75) Value too large for defined data type 2024-01-24T23:24:36.277 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:24:36.278 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd tree 2024-01-24T23:24:36.278 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep osd.11 2024-01-24T23:24:36.278 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep 113 2024-01-24T23:24:36.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.386+0000 7f6a88747640 10 monclient: get_monmap_and_config 2024-01-24T23:24:36.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.386+0000 7f6a88747640 10 monclient: build_initial_monmap 2024-01-24T23:24:36.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.386+0000 7f6a88747640 10 monclient: monmap: 2024-01-24T23:24:36.391 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:36.391 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:36.391 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:36.392442+0000 2024-01-24T23:24:36.391 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:36.392442+0000 2024-01-24T23:24:36.391 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:36.392 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:36.392 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:36.392 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:36.392 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:36.392 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:36.392 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:36.392 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:36.392 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:36.392 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:36.392 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:36.392 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:36.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a88747640 1 Processor -- start 2024-01-24T23:24:36.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a88747640 1 -- start start 2024-01-24T23:24:36.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a88747640 10 monclient: init 2024-01-24T23:24:36.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a88747640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:36.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a88747640 10 monclient: _add_conns ranks=[1,2,4,8,3,6,5,7,0] 2024-01-24T23:24:36.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a88747640 10 monclient(hunting): picked mon.noname-f con 0x7f6a8009c4c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:36.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a88747640 10 monclient(hunting): picked mon.noname-b con 0x7f6a8009cfb0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:36.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a88747640 10 monclient(hunting): picked mon.noname-c con 0x7f6a8009ec50 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:36.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a88747640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a80064700 con 0x7f6a8009c4c0 2024-01-24T23:24:36.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a864e3640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f6a8009c4c0 legacy=0x7f6a8009c8a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:36.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a88747640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a8009d9f0 con 0x7f6a8009cfb0 2024-01-24T23:24:36.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a88747640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a8009dc60 con 0x7f6a8009ec50 2024-01-24T23:24:36.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a88747640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:36.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a88747640 10 monclient(hunting): authenticate will time out at 1163.717298s 2024-01-24T23:24:36.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a86ce4640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f6a8009ec50 legacy=0x7f6a8013c460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:36.396 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a85ce2640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f6a8009cfb0 0x7f6a80138d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:49150/0 (socket says 172.21.15.132:49150) 2024-01-24T23:24:36.396 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a85ce2640 1 -- 172.21.15.132:0/242478670 learned_addr learned my addr 172.21.15.132:0/242478670 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:36.396 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 1 -- 172.21.15.132:0/242478670 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3275604448 0 0) 0x7f6a8009d9f0 con 0x7f6a8009cfb0 2024-01-24T23:24:36.396 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 10 monclient(hunting): _negotiate 2024-01-24T23:24:36.396 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:36.396 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:36.396 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 10 monclient(hunting): my global_id is 84908 2024-01-24T23:24:36.396 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 1 -- 172.21.15.132:0/242478670 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a6c003530 con 0x7f6a8009cfb0 2024-01-24T23:24:36.396 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 1 -- 172.21.15.132:0/242478670 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3041331191 0 0) 0x7f6a6c003530 con 0x7f6a8009cfb0 2024-01-24T23:24:36.396 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 1 -- 172.21.15.132:0/242478670 >> v1:172.21.15.115:6791/0 conn(0x7f6a8009ec50 legacy=0x7f6a8013c460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:36.396 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 1 -- 172.21.15.132:0/242478670 >> v1:172.21.15.132:6789/0 conn(0x7f6a8009c4c0 legacy=0x7f6a8009c8a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:36.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 10 monclient: _finish_hunting 0 2024-01-24T23:24:36.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 1 monclient: found mon.noname-b 2024-01-24T23:24:36.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:36.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:24:36.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 1 -- 172.21.15.132:0/242478670 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8009df40 con 0x7f6a8009cfb0 2024-01-24T23:24:36.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 10 monclient: _finish_auth 0 2024-01-24T23:24:36.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 10 monclient: _check_auth_tickets 2024-01-24T23:24:36.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:36.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 1 -- 172.21.15.132:0/242478670 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6a700017c0 con 0x7f6a8009cfb0 2024-01-24T23:24:36.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:36.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:24:36.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 10 monclient: dump: 2024-01-24T23:24:36.398 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:36.398 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:36.398 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.398 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.398 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:36.398 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:36.398 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:36.398 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:36.398 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:36.398 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:36.398 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:36.398 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:36.398 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:36.399 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:36.399 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:36.399 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:36.399 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a88747640 5 monclient: authenticate success, global_id 84908 2024-01-24T23:24:36.399 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a88747640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:36.399 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 1 -- 172.21.15.132:0/242478670 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6a700019e0 con 0x7f6a8009cfb0 2024-01-24T23:24:36.399 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:36.399 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 1 -- 172.21.15.132:0/242478670 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6a70004bf0 con 0x7f6a8009cfb0 2024-01-24T23:24:36.399 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:36.399 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:24:36.399 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a854e1640 10 monclient: dump: 2024-01-24T23:24:36.399 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:36.399 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:36.399 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.399 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.400 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:36.400 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:36.400 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:36.400 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:36.400 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:36.400 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:36.400 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:36.400 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:36.400 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:36.400 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:36.400 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:36.400 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:36.400 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a88747640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:36.400 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a88747640 4 set_mon_vals no callback set 2024-01-24T23:24:36.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a88747640 10 monclient: shutdown 2024-01-24T23:24:36.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a88747640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:36.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a88747640 1 -- 172.21.15.132:0/242478670 >> v1:172.21.15.115:6790/0 conn(0x7f6a8009cfb0 legacy=0x7f6a80138d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:36.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.390+0000 7f6a88747640 1 -- 172.21.15.132:0/242478670 shutdown_connections 2024-01-24T23:24:36.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 1 -- 172.21.15.132:0/242478670 >> 172.21.15.132:0/242478670 conn(0x7f6a800a5a10 msgr2=0x7f6a800a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:36.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 1 -- 172.21.15.132:0/242478670 shutdown_connections 2024-01-24T23:24:36.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 1 -- 172.21.15.132:0/242478670 wait complete. 2024-01-24T23:24:36.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 10 monclient: build_initial_monmap 2024-01-24T23:24:36.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 10 monclient: monmap: 2024-01-24T23:24:36.405 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:36.405 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:36.405 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:36.405 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:36.405 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:36.405 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:36.405 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:36.406 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:36.406 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:36.406 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:36.406 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:36.406 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:36.406 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:36.406 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:36.406 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:36.406 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:36.406 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 1 Processor -- start 2024-01-24T23:24:36.406 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 1 -- start start 2024-01-24T23:24:36.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 10 monclient: init 2024-01-24T23:24:36.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:36.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 10 monclient: _add_conns ranks=[8,1,4,6,7,5,3,2,0] 2024-01-24T23:24:36.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 10 monclient(hunting): picked mon.noname-e con 0x7f6a8009c4c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:36.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 10 monclient(hunting): picked mon.noname-f con 0x7f6a8009cfb0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:36.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 10 monclient(hunting): picked mon.noname-c con 0x7f6a8009ec50 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:36.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a8009c020 con 0x7f6a8009cfb0 2024-01-24T23:24:36.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a8011dc20 con 0x7f6a8009ec50 2024-01-24T23:24:36.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a800711e0 con 0x7f6a8009c4c0 2024-01-24T23:24:36.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:36.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 10 monclient(hunting): authenticate will time out at 1163.721949s 2024-01-24T23:24:36.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a85ce2640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f6a8009cfb0 legacy=0x7f6a8012e750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:36.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a86ce4640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f6a8009ec50 legacy=0x7f6a801d00c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:36.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a864e3640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f6a8009c4c0 0x7f6a8011bad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:37770/0 (socket says 172.21.15.132:37770) 2024-01-24T23:24:36.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a864e3640 1 -- 172.21.15.132:0/3753595150 learned_addr learned my addr 172.21.15.132:0/3753595150 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:36.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 1 -- 172.21.15.132:0/3753595150 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930986938 0 0) 0x7f6a800711e0 con 0x7f6a8009c4c0 2024-01-24T23:24:36.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:36.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:36.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:36.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 10 monclient(hunting): my global_id is 84806 2024-01-24T23:24:36.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 1 -- 172.21.15.132:0/3753595150 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a64003530 con 0x7f6a8009c4c0 2024-01-24T23:24:36.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 1 -- 172.21.15.132:0/3753595150 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1620290049 0 0) 0x7f6a64003530 con 0x7f6a8009c4c0 2024-01-24T23:24:36.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 1 -- 172.21.15.132:0/3753595150 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6a800711e0 con 0x7f6a8009c4c0 2024-01-24T23:24:36.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 1 -- 172.21.15.132:0/3753595150 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6a7c0057f0 con 0x7f6a8009c4c0 2024-01-24T23:24:36.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:36.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:24:36.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 10 monclient(hunting): dump: 2024-01-24T23:24:36.408 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:36.409 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:36.409 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.409 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.409 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:36.409 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:36.409 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:36.409 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:36.409 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:36.409 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:36.409 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:36.409 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:36.409 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:36.409 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:36.409 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:36.409 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:36.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:36.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 1 -- 172.21.15.132:0/3753595150 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1428437406 0 0) 0x7f6a800711e0 con 0x7f6a8009c4c0 2024-01-24T23:24:36.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 1 -- 172.21.15.132:0/3753595150 >> v1:172.21.15.115:6791/0 conn(0x7f6a8009ec50 legacy=0x7f6a801d00c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:36.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 1 -- 172.21.15.132:0/3753595150 >> v1:172.21.15.132:6789/0 conn(0x7f6a8009cfb0 legacy=0x7f6a8012e750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:36.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:36.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 1 monclient: found mon.e 2024-01-24T23:24:36.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:36.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:36.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 1 -- 172.21.15.132:0/3753595150 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8011c1e0 con 0x7f6a8009c4c0 2024-01-24T23:24:36.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 10 monclient: _finish_auth 0 2024-01-24T23:24:36.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:36.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:36.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 5 monclient: authenticate success, global_id 84806 2024-01-24T23:24:36.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 10 monclient: _renew_subs 2024-01-24T23:24:36.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:36.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 1 -- 172.21.15.132:0/3753595150 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8011d660 con 0x7f6a8009c4c0 2024-01-24T23:24:36.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 10 monclient: _renew_subs 2024-01-24T23:24:36.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:36.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a88747640 1 -- 172.21.15.132:0/3753595150 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a800708a0 con 0x7f6a8009c4c0 2024-01-24T23:24:36.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 1 -- 172.21.15.132:0/3753595150 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6a7c0043b0 con 0x7f6a8009c4c0 2024-01-24T23:24:36.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:36.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 1 -- 172.21.15.132:0/3753595150 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6a7c0048e0 con 0x7f6a8009c4c0 2024-01-24T23:24:36.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:36.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:24:36.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a84ce0640 4 set_mon_vals no callback set 2024-01-24T23:24:36.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.394+0000 7f6a76ffd640 10 monclient: dump: 2024-01-24T23:24:36.412 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:36.412 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:36.412 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.412 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.412 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:36.412 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:36.412 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:36.412 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:36.412 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:36.412 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:36.412 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:36.412 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:36.412 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:36.413 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:36.413 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:36.413 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:36.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.398+0000 7f6a88747640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:36.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.398+0000 7f6a88747640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:36.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.398+0000 7f6a88747640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:36.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.398+0000 7f6a88747640 1 -- 172.21.15.132:0/3753595150 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a68005090 con 0x7f6a8009c4c0 2024-01-24T23:24:36.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.398+0000 7f6a76ffd640 1 -- 172.21.15.132:0/3753595150 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f6a7c004ac0 con 0x7f6a8009c4c0 2024-01-24T23:24:36.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.398+0000 7f6a76ffd640 1 -- 172.21.15.132:0/3753595150 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(99..99 src has 1..99) v4 ==== 5200+0+0 (unknown 2556180564 0 0) 0x7f6a7c044120 con 0x7f6a8009c4c0 2024-01-24T23:24:36.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.398+0000 7f6a85ce2640 10 monclient: get_auth_request con 0x7f6a64036f10 auth_method 0 2024-01-24T23:24:36.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.410+0000 7f6a76ffd640 1 -- 172.21.15.132:0/3753595150 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f6a7c014e00 con 0x7f6a8009c4c0 2024-01-24T23:24:36.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.410+0000 7f6a76ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:36.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.410+0000 7f6a76ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:36.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.566+0000 7f6a88747640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:36.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.566+0000 7f6a88747640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:36.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.566+0000 7f6a88747640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:36.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.566+0000 7f6a88747640 1 -- 172.21.15.132:0/3753595150 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f6a68005b50 con 0x7f6a8009c4c0 2024-01-24T23:24:36.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.566+0000 7f6a76ffd640 1 -- 172.21.15.132:0/3753595150 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v99) v1 ==== 56+0+1173 (unknown 3011264473 0 2530539077) 0x7f6a7c014a10 con 0x7f6a8009c4c0 2024-01-24T23:24:36.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.566+0000 7f6a76ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:36.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.566+0000 7f6a76ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:24:36.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.658+0000 7f6a88747640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:36.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.658+0000 7f6a88747640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:36.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.658+0000 7f6a88747640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:36.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.658+0000 7f6a88747640 1 -- 172.21.15.132:0/3753595150 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f6a68005ce0 con 0x7f6a8009c4c0 2024-01-24T23:24:36.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.658+0000 7f6a76ffd640 1 -- 172.21.15.132:0/3753595150 <== mon.8 v1:172.21.15.115:6793/0 11 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v99) v1 ==== 56+0+1173 (unknown 3011264473 0 2530539077) 0x7f6a7c014a10 con 0x7f6a8009c4c0 2024-01-24T23:24:36.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.658+0000 7f6a76ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:36.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.658+0000 7f6a76ffd640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:24:36.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.658+0000 7f6a88747640 1 -- 172.21.15.132:0/3753595150 >> v1:172.21.15.132:6812/1835401922 conn(0x7f6a64036f10 legacy=0x7f6a640393d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:36.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.658+0000 7f6a88747640 10 monclient: shutdown 2024-01-24T23:24:36.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.658+0000 7f6a88747640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:36.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.658+0000 7f6a88747640 1 -- 172.21.15.132:0/3753595150 >> v1:172.21.15.115:6793/0 conn(0x7f6a8009c4c0 legacy=0x7f6a8011bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:36.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.658+0000 7f6a88747640 1 -- 172.21.15.132:0/3753595150 shutdown_connections 2024-01-24T23:24:36.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.658+0000 7f6a88747640 1 -- 172.21.15.132:0/3753595150 >> 172.21.15.132:0/3753595150 conn(0x7f6a800a5a10 msgr2=0x7f6a80066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:36.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.658+0000 7f6a88747640 1 -- 172.21.15.132:0/3753595150 shutdown_connections 2024-01-24T23:24:36.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.658+0000 7f6a88747640 1 -- 172.21.15.132:0/3753595150 wait complete. 2024-01-24T23:24:36.677 INFO:tasks.workunit.client.0.smithi132.stdout: 11 113.00000 osd.11 down 1.00000 1.00000 2024-01-24T23:24:36.678 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm osd.11 2024-01-24T23:24:36.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.790+0000 7f6fd4b3d640 10 monclient: get_monmap_and_config 2024-01-24T23:24:36.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.790+0000 7f6fd4b3d640 10 monclient: build_initial_monmap 2024-01-24T23:24:36.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.790+0000 7f6fd4b3d640 10 monclient: monmap: 2024-01-24T23:24:36.794 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:36.794 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:36.794 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:36.795650+0000 2024-01-24T23:24:36.794 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:36.795650+0000 2024-01-24T23:24:36.795 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:36.795 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:36.795 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:36.795 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:36.795 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:36.795 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:36.795 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:36.795 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:36.795 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:36.795 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:36.795 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:36.795 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:36.798 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.790+0000 7f6fd4b3d640 1 Processor -- start 2024-01-24T23:24:36.798 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.790+0000 7f6fd4b3d640 1 -- start start 2024-01-24T23:24:36.798 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.790+0000 7f6fd4b3d640 10 monclient: init 2024-01-24T23:24:36.798 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:36.798 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 10 monclient: _add_conns ranks=[6,0,1,2,3,7,8,4,5] 2024-01-24T23:24:36.798 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 10 monclient(hunting): picked mon.noname-d con 0x7f6fd009ff20 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:36.798 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 10 monclient(hunting): picked mon.noname-a con 0x7f6fd009f9c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:36.798 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 10 monclient(hunting): picked mon.noname-f con 0x7f6fd013b2c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:36.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd0064700 con 0x7f6fd009f9c0 2024-01-24T23:24:36.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd009e760 con 0x7f6fd013b2c0 2024-01-24T23:24:36.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd00a1160 con 0x7f6fd009ff20 2024-01-24T23:24:36.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:36.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 10 monclient(hunting): authenticate will time out at 1164.120358s 2024-01-24T23:24:36.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fced9d640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f6fd013b2c0 legacy=0x7f6fd013d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:36.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcdd9b640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f6fd009f9c0 legacy=0x7f6fd0139ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:36.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fce59c640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f6fd009ff20 0x7f6fd00a0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:49578/0 (socket says 172.21.15.132:49578) 2024-01-24T23:24:36.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fce59c640 1 -- 172.21.15.132:0/2985990006 learned_addr learned my addr 172.21.15.132:0/2985990006 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:36.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 1 -- 172.21.15.132:0/2985990006 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1808462232 0 0) 0x7f6fd00a1160 con 0x7f6fd009ff20 2024-01-24T23:24:36.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 10 monclient(hunting): _negotiate 2024-01-24T23:24:36.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:36.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:36.799 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 10 monclient(hunting): my global_id is 5667 2024-01-24T23:24:36.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 1 -- 172.21.15.132:0/2985990006 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fb8003530 con 0x7f6fd009ff20 2024-01-24T23:24:36.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 1 -- 172.21.15.132:0/2985990006 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1738216121 0 0) 0x7f6fb8003530 con 0x7f6fd009ff20 2024-01-24T23:24:36.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 1 -- 172.21.15.132:0/2985990006 >> v1:172.21.15.132:6789/0 conn(0x7f6fd013b2c0 legacy=0x7f6fd013d6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:36.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 1 -- 172.21.15.132:0/2985990006 >> v1:172.21.15.115:6789/0 conn(0x7f6fd009f9c0 legacy=0x7f6fd0139ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:36.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 10 monclient: _finish_hunting 0 2024-01-24T23:24:36.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 1 monclient: found mon.noname-d 2024-01-24T23:24:36.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:36.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:24:36.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 1 -- 172.21.15.132:0/2985990006 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd00a0a10 con 0x7f6fd009ff20 2024-01-24T23:24:36.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 10 monclient: _finish_auth 0 2024-01-24T23:24:36.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 10 monclient: _check_auth_tickets 2024-01-24T23:24:36.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:36.800 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 1 -- 172.21.15.132:0/2985990006 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6fc0001800 con 0x7f6fd009ff20 2024-01-24T23:24:36.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:36.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:24:36.801 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 10 monclient: dump: 2024-01-24T23:24:36.801 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:36.801 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:36.801 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.801 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.801 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:36.801 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:36.801 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:36.801 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:36.801 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:36.801 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:36.801 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:36.801 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:36.802 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:36.802 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:36.802 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:36.802 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:36.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 5 monclient: authenticate success, global_id 5667 2024-01-24T23:24:36.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:36.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 1 -- 172.21.15.132:0/2985990006 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6fc0001a20 con 0x7f6fd009ff20 2024-01-24T23:24:36.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:36.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 1 -- 172.21.15.132:0/2985990006 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6fc0004bf0 con 0x7f6fd009ff20 2024-01-24T23:24:36.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:36.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:24:36.802 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fcd59a640 10 monclient: dump: 2024-01-24T23:24:36.802 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:36.802 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:36.802 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.802 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.803 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:36.803 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:36.803 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:36.803 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:36.803 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:36.803 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:36.803 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:36.803 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:36.803 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:36.803 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:36.803 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:36.803 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:36.803 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:36.803 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 4 set_mon_vals no callback set 2024-01-24T23:24:36.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 10 monclient: shutdown 2024-01-24T23:24:36.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:36.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 1 -- 172.21.15.132:0/2985990006 >> v1:172.21.15.115:6792/0 conn(0x7f6fd009ff20 legacy=0x7f6fd00a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:36.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 1 -- 172.21.15.132:0/2985990006 shutdown_connections 2024-01-24T23:24:36.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 1 -- 172.21.15.132:0/2985990006 >> 172.21.15.132:0/2985990006 conn(0x7f6fd00a5a10 msgr2=0x7f6fd00a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:36.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 1 -- 172.21.15.132:0/2985990006 shutdown_connections 2024-01-24T23:24:36.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 1 -- 172.21.15.132:0/2985990006 wait complete. 2024-01-24T23:24:36.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 10 monclient: build_initial_monmap 2024-01-24T23:24:36.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 10 monclient: monmap: 2024-01-24T23:24:36.804 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:36.804 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:36.804 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:36.804 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:36.804 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:36.804 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:36.805 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:36.805 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:36.805 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:36.805 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:36.805 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:36.805 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:36.805 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:36.805 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:36.805 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:36.805 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:36.809 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 1 Processor -- start 2024-01-24T23:24:36.809 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 1 -- start start 2024-01-24T23:24:36.809 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 10 monclient: init 2024-01-24T23:24:36.809 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.794+0000 7f6fd4b3d640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:36.809 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fd4b3d640 10 monclient: _add_conns ranks=[2,6,4,0,5,8,7,3,1] 2024-01-24T23:24:36.809 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fd4b3d640 10 monclient(hunting): picked mon.noname-b con 0x7f6fd009f9c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:36.809 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fd4b3d640 10 monclient(hunting): picked mon.noname-d con 0x7f6fd009ff20 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:36.809 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fd4b3d640 10 monclient(hunting): picked mon.noname-c con 0x7f6fd013b2c0 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:36.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fd4b3d640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd01209b0 con 0x7f6fd009f9c0 2024-01-24T23:24:36.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fd4b3d640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd009d2e0 con 0x7f6fd013b2c0 2024-01-24T23:24:36.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fd4b3d640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd009e180 con 0x7f6fd009ff20 2024-01-24T23:24:36.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fd4b3d640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:36.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fd4b3d640 10 monclient(hunting): authenticate will time out at 1164.124104s 2024-01-24T23:24:36.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fced9d640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f6fd013b2c0 legacy=0x7f6fd01d3880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:36.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fce59c640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f6fd009f9c0 0x7f6fd009c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:49158/0 (socket says 172.21.15.132:49158) 2024-01-24T23:24:36.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fce59c640 1 -- 172.21.15.132:0/2621431933 learned_addr learned my addr 172.21.15.132:0/2621431933 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:36.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 1 -- 172.21.15.132:0/2621431933 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 846627700 0 0) 0x7f6fd009e180 con 0x7f6fd009ff20 2024-01-24T23:24:36.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:36.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:36.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:36.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 10 monclient(hunting): my global_id is 5676 2024-01-24T23:24:36.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 1 -- 172.21.15.132:0/2621431933 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fb0003530 con 0x7f6fd009ff20 2024-01-24T23:24:36.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 1 -- 172.21.15.132:0/2621431933 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406493162 0 0) 0x7f6fd01209b0 con 0x7f6fd009f9c0 2024-01-24T23:24:36.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:36.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:36.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:36.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 10 monclient(hunting): my global_id is 84917 2024-01-24T23:24:36.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 1 -- 172.21.15.132:0/2621431933 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fd009e180 con 0x7f6fd009f9c0 2024-01-24T23:24:36.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 1 -- 172.21.15.132:0/2621431933 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3894243005 0 0) 0x7f6fb0003530 con 0x7f6fd009ff20 2024-01-24T23:24:36.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 1 -- 172.21.15.132:0/2621431933 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6fd01209b0 con 0x7f6fd009ff20 2024-01-24T23:24:36.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 1 -- 172.21.15.132:0/2621431933 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3791309099 0 0) 0x7f6fd009e180 con 0x7f6fd009f9c0 2024-01-24T23:24:36.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 1 -- 172.21.15.132:0/2621431933 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6fb0003530 con 0x7f6fd009f9c0 2024-01-24T23:24:36.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 1 -- 172.21.15.132:0/2621431933 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6fc40042a0 con 0x7f6fd009ff20 2024-01-24T23:24:36.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:36.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:24:36.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 10 monclient(hunting): dump: 2024-01-24T23:24:36.811 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:36.811 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:36.812 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.812 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.812 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:36.812 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:36.812 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:36.812 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:36.812 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:36.812 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:36.812 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:36.812 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:36.812 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:36.812 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:36.812 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:36.812 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:36.813 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:36.813 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 1 -- 172.21.15.132:0/2621431933 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6fc00047f0 con 0x7f6fd009f9c0 2024-01-24T23:24:36.813 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:36.813 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:24:36.813 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 10 monclient(hunting): dump: 2024-01-24T23:24:36.813 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:36.813 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:36.813 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.813 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.813 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:36.813 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:36.813 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:36.813 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:36.813 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:36.814 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:36.814 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:36.814 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:36.814 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:36.814 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:36.814 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:36.814 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:36.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 1 -- 172.21.15.132:0/2621431933 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2865133798 0 0) 0x7f6fd01209b0 con 0x7f6fd009ff20 2024-01-24T23:24:36.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 1 -- 172.21.15.132:0/2621431933 >> v1:172.21.15.115:6791/0 conn(0x7f6fd013b2c0 legacy=0x7f6fd01d3880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:36.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 1 -- 172.21.15.132:0/2621431933 >> v1:172.21.15.115:6790/0 conn(0x7f6fd009f9c0 legacy=0x7f6fd009c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:36.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:36.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 1 monclient: found mon.d 2024-01-24T23:24:36.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:36.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:36.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 1 -- 172.21.15.132:0/2621431933 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd0120c90 con 0x7f6fd009ff20 2024-01-24T23:24:36.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 10 monclient: _finish_auth 0 2024-01-24T23:24:36.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:36.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:36.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fd4b3d640 5 monclient: authenticate success, global_id 5676 2024-01-24T23:24:36.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fd4b3d640 10 monclient: _renew_subs 2024-01-24T23:24:36.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fd4b3d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:36.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fd4b3d640 1 -- 172.21.15.132:0/2621431933 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd0071960 con 0x7f6fd009ff20 2024-01-24T23:24:36.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fd4b3d640 10 monclient: _renew_subs 2024-01-24T23:24:36.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fd4b3d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:36.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fd4b3d640 1 -- 172.21.15.132:0/2621431933 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd009da60 con 0x7f6fd009ff20 2024-01-24T23:24:36.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 1 -- 172.21.15.132:0/2621431933 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6fc4004510 con 0x7f6fd009ff20 2024-01-24T23:24:36.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:36.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 1 -- 172.21.15.132:0/2621431933 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6fc4004a40 con 0x7f6fd009ff20 2024-01-24T23:24:36.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:36.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fccd99640 4 set_mon_vals no callback set 2024-01-24T23:24:36.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:24:36.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 10 monclient: dump: 2024-01-24T23:24:36.816 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:36.816 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:36.816 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.816 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:36.816 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:36.816 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:36.816 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:36.816 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:36.816 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:36.816 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:36.816 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:36.816 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:36.816 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:36.817 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:36.817 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:36.817 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:36.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fd4b3d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:36.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fd4b3d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:36.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fd4b3d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:36.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fd4b3d640 1 -- 172.21.15.132:0/2621431933 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb4005090 con 0x7f6fd009ff20 2024-01-24T23:24:36.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 1 -- 172.21.15.132:0/2621431933 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f6fc4004ba0 con 0x7f6fd009ff20 2024-01-24T23:24:36.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fbeffd640 1 -- 172.21.15.132:0/2621431933 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(99..99 src has 1..99) v4 ==== 5200+0+0 (unknown 2556180564 0 0) 0x7f6fc4043bb0 con 0x7f6fd009ff20 2024-01-24T23:24:36.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.798+0000 7f6fce59c640 10 monclient: get_auth_request con 0x7f6fb0037590 auth_method 0 2024-01-24T23:24:36.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.810+0000 7f6fbeffd640 1 -- 172.21.15.132:0/2621431933 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f6fc4014600 con 0x7f6fd009ff20 2024-01-24T23:24:36.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.810+0000 7f6fbeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:36.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.810+0000 7f6fbeffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:36.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.978+0000 7f6fd4b3d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.11"}] 2024-01-24T23:24:36.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.978+0000 7f6fd4b3d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.11"}] 2024-01-24T23:24:36.982 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.978+0000 7f6fd4b3d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:36.982 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:36.978+0000 7f6fd4b3d640 1 -- 172.21.15.132:0/2621431933 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "osd.11"} v 0) v1 -- 0x7f6fb4005340 con 0x7f6fd009ff20 2024-01-24T23:24:37.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:37.970+0000 7f6fbeffd640 1 -- 172.21.15.132:0/2621431933 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.11"}]=0 removed item id 11 name 'osd.11' from crush map v100) v1 ==== 125+0+0 (unknown 3065460757 0 0) 0x7f6fc4014420 con 0x7f6fd009ff20 2024-01-24T23:24:37.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:37.970+0000 7f6fbeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.11"}] 2024-01-24T23:24:37.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:37.970+0000 7f6fbeffd640 10 monclient: _finish_command 2 = system:0 removed item id 11 name 'osd.11' from crush map 2024-01-24T23:24:38.077 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.074+0000 7f6fd4b3d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.11"}] 2024-01-24T23:24:38.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.074+0000 7f6fd4b3d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.11"}] 2024-01-24T23:24:38.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.074+0000 7f6fd4b3d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:38.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.074+0000 7f6fd4b3d640 1 -- 172.21.15.132:0/2621431933 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "osd.11"} v 0) v1 -- 0x7f6fb40010a0 con 0x7f6fd009ff20 2024-01-24T23:24:38.079 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.078+0000 7f6fbeffd640 1 -- 172.21.15.132:0/2621431933 <== mon.6 v1:172.21.15.115:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.11"}]=0 device 'osd.11' does not appear in the crush map v100) v1 ==== 126+0+0 (unknown 4115871241 0 0) 0x7f6fc4014420 con 0x7f6fd009ff20 2024-01-24T23:24:38.079 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.078+0000 7f6fbeffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.11"}] 2024-01-24T23:24:38.079 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.078+0000 7f6fbeffd640 10 monclient: _finish_command 3 = system:0 device 'osd.11' does not appear in the crush map 2024-01-24T23:24:38.079 INFO:tasks.workunit.client.0.smithi132.stderr:device 'osd.11' does not appear in the crush map 2024-01-24T23:24:38.083 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.078+0000 7f6fd4b3d640 1 -- 172.21.15.132:0/2621431933 >> v1:172.21.15.132:6812/1835401922 conn(0x7f6fb0037590 legacy=0x7f6fb0039a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:38.083 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.078+0000 7f6fd4b3d640 10 monclient: shutdown 2024-01-24T23:24:38.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.078+0000 7f6fd4b3d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:38.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.078+0000 7f6fd4b3d640 1 -- 172.21.15.132:0/2621431933 >> v1:172.21.15.115:6792/0 conn(0x7f6fd009ff20 legacy=0x7f6fd009cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:38.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.078+0000 7f6fd4b3d640 1 -- 172.21.15.132:0/2621431933 shutdown_connections 2024-01-24T23:24:38.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.078+0000 7f6fd4b3d640 1 -- 172.21.15.132:0/2621431933 >> 172.21.15.132:0/2621431933 conn(0x7f6fd00a5a10 msgr2=0x7f6fd0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:38.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.078+0000 7f6fd4b3d640 1 -- 172.21.15.132:0/2621431933 shutdown_connections 2024-01-24T23:24:38.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.078+0000 7f6fd4b3d640 1 -- 172.21.15.132:0/2621431933 wait complete. 2024-01-24T23:24:38.096 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd rm osd.11 2024-01-24T23:24:38.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.210+0000 7fabe8e2b640 10 monclient: get_monmap_and_config 2024-01-24T23:24:38.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.210+0000 7fabe8e2b640 10 monclient: build_initial_monmap 2024-01-24T23:24:38.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.210+0000 7fabe8e2b640 10 monclient: monmap: 2024-01-24T23:24:38.214 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:38.214 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:38.214 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:38.215142+0000 2024-01-24T23:24:38.214 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:38.215142+0000 2024-01-24T23:24:38.214 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:38.214 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:38.215 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:38.215 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:38.215 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:38.215 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:38.215 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:38.215 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:38.215 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:38.215 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:38.215 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:38.215 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:38.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.210+0000 7fabe8e2b640 1 Processor -- start 2024-01-24T23:24:38.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.210+0000 7fabe8e2b640 1 -- start start 2024-01-24T23:24:38.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.210+0000 7fabe8e2b640 10 monclient: init 2024-01-24T23:24:38.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.210+0000 7fabe8e2b640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:38.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.210+0000 7fabe8e2b640 10 monclient: _add_conns ranks=[8,1,7,6,5,3,2,0,4] 2024-01-24T23:24:38.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.210+0000 7fabe8e2b640 10 monclient(hunting): picked mon.noname-e con 0x7fabe4135e10 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:38.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.210+0000 7fabe8e2b640 10 monclient(hunting): picked mon.noname-f con 0x7fabe41361f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:38.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.210+0000 7fabe8e2b640 10 monclient(hunting): picked mon.noname-i con 0x7fabe41365f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:38.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.210+0000 7fabe8e2b640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabe4064700 con 0x7fabe41361f0 2024-01-24T23:24:38.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.210+0000 7fabe8e2b640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabe409c3e0 con 0x7fabe41365f0 2024-01-24T23:24:38.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.210+0000 7fabe8e2b640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabe40a4f90 con 0x7fabe4135e10 2024-01-24T23:24:38.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.210+0000 7fabe8e2b640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:38.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.210+0000 7fabe8e2b640 10 monclient(hunting): authenticate will time out at 1165.539845s 2024-01-24T23:24:38.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.210+0000 7fabe1d9b640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fabe41361f0 legacy=0x7fabe413bed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:38.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe2d9d640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fabe41365f0 0x7fabe413f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:35594/0 (socket says 172.21.15.132:35594) 2024-01-24T23:24:38.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe2d9d640 1 -- 172.21.15.132:0/2281501505 learned_addr learned my addr 172.21.15.132:0/2281501505 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:38.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 1 -- 172.21.15.132:0/2281501505 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003666424 0 0) 0x7fabe409c3e0 con 0x7fabe41365f0 2024-01-24T23:24:38.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 10 monclient(hunting): _negotiate 2024-01-24T23:24:38.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:38.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:38.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 10 monclient(hunting): my global_id is 5632 2024-01-24T23:24:38.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 1 -- 172.21.15.132:0/2281501505 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabcc003530 con 0x7fabe41365f0 2024-01-24T23:24:38.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 1 -- 172.21.15.132:0/2281501505 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 884376519 0 0) 0x7fabcc003530 con 0x7fabe41365f0 2024-01-24T23:24:38.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 1 -- 172.21.15.132:0/2281501505 >> v1:172.21.15.115:6793/0 conn(0x7fabe4135e10 legacy=0x7fabe409bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:38.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 1 -- 172.21.15.132:0/2281501505 >> v1:172.21.15.132:6789/0 conn(0x7fabe41361f0 legacy=0x7fabe413bed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:38.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 10 monclient: _finish_hunting 0 2024-01-24T23:24:38.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 1 monclient: found mon.noname-i 2024-01-24T23:24:38.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:38.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:24:38.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 1 -- 172.21.15.132:0/2281501505 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe409b010 con 0x7fabe41365f0 2024-01-24T23:24:38.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 10 monclient: _finish_auth 0 2024-01-24T23:24:38.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 10 monclient: _check_auth_tickets 2024-01-24T23:24:38.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:38.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 1 -- 172.21.15.132:0/2281501505 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fabdc001b40 con 0x7fabe41365f0 2024-01-24T23:24:38.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 5 monclient: authenticate success, global_id 5632 2024-01-24T23:24:38.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:38.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:38.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:38.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 10 monclient: dump: 2024-01-24T23:24:38.226 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:38.226 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:38.226 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:38.226 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:38.226 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:38.226 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:38.226 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:38.226 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:38.226 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:38.226 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:38.226 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:38.226 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:38.226 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:38.227 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:38.227 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:38.227 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:38.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:38.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 1 -- 172.21.15.132:0/2281501505 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fabdc001de0 con 0x7fabe41365f0 2024-01-24T23:24:38.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:38.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:38.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 4 set_mon_vals no callback set 2024-01-24T23:24:38.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 1 -- 172.21.15.132:0/2281501505 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fabdc004f70 con 0x7fabe41365f0 2024-01-24T23:24:38.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:38.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:38.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe159a640 10 monclient: dump: 2024-01-24T23:24:38.227 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:38.228 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:38.228 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:38.228 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:38.228 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:38.228 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:38.228 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:38.228 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:38.228 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:38.228 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:38.228 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:38.228 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:38.228 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:38.228 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:38.228 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:38.229 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:38.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 10 monclient: shutdown 2024-01-24T23:24:38.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:38.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 1 -- 172.21.15.132:0/2281501505 >> v1:172.21.15.132:6792/0 conn(0x7fabe41365f0 legacy=0x7fabe413f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:38.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 1 -- 172.21.15.132:0/2281501505 shutdown_connections 2024-01-24T23:24:38.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 1 -- 172.21.15.132:0/2281501505 >> 172.21.15.132:0/2281501505 conn(0x7fabe40a6790 msgr2=0x7fabe4071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:38.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 1 -- 172.21.15.132:0/2281501505 shutdown_connections 2024-01-24T23:24:38.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 1 -- 172.21.15.132:0/2281501505 wait complete. 2024-01-24T23:24:38.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 10 monclient: build_initial_monmap 2024-01-24T23:24:38.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 10 monclient: monmap: 2024-01-24T23:24:38.229 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:38.229 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:38.229 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:38.229 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:38.230 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:38.230 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:38.230 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:38.230 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:38.230 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:38.230 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:38.230 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:38.230 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:38.230 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:38.230 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:38.230 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:38.230 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:38.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 1 Processor -- start 2024-01-24T23:24:38.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 1 -- start start 2024-01-24T23:24:38.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 10 monclient: init 2024-01-24T23:24:38.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:38.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 10 monclient: _add_conns ranks=[7,4,8,3,1,5,0,6,2] 2024-01-24T23:24:38.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 10 monclient(hunting): picked mon.noname-i con 0x7fabe4135e10 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:38.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 10 monclient(hunting): picked mon.noname-c con 0x7fabe41361f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:38.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 10 monclient(hunting): picked mon.noname-e con 0x7fabe41365f0 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:38.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabe41bf300 con 0x7fabe41361f0 2024-01-24T23:24:38.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabe4121f30 con 0x7fabe4135e10 2024-01-24T23:24:38.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabe41221f0 con 0x7fabe41365f0 2024-01-24T23:24:38.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:38.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 10 monclient(hunting): authenticate will time out at 1165.543133s 2024-01-24T23:24:38.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe1d9b640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7fabe41361f0 legacy=0x7fabe41d02b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:38.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe259c640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fabe4135e10 0x7fabe41c81a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:35604/0 (socket says 172.21.15.132:35604) 2024-01-24T23:24:38.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe259c640 1 -- 172.21.15.132:0/3188456934 learned_addr learned my addr 172.21.15.132:0/3188456934 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:38.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 1 -- 172.21.15.132:0/3188456934 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951284855 0 0) 0x7fabe4121f30 con 0x7fabe4135e10 2024-01-24T23:24:38.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:38.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:38.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:38.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 10 monclient(hunting): my global_id is 5641 2024-01-24T23:24:38.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 1 -- 172.21.15.132:0/3188456934 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabc4003560 con 0x7fabe4135e10 2024-01-24T23:24:38.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 1 -- 172.21.15.132:0/3188456934 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3421875830 0 0) 0x7fabc4003560 con 0x7fabe4135e10 2024-01-24T23:24:38.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 1 -- 172.21.15.132:0/3188456934 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fabe4121f30 con 0x7fabe4135e10 2024-01-24T23:24:38.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 1 -- 172.21.15.132:0/3188456934 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fabd4001ac0 con 0x7fabe4135e10 2024-01-24T23:24:38.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:38.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:38.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 10 monclient(hunting): dump: 2024-01-24T23:24:38.232 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:38.232 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:38.232 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:38.233 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:38.233 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:38.233 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:38.233 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:38.233 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:38.233 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:38.233 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:38.233 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:38.233 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:38.233 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:38.233 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:38.233 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:38.233 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:38.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:38.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 1 -- 172.21.15.132:0/3188456934 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2086129417 0 0) 0x7fabe41221f0 con 0x7fabe41365f0 2024-01-24T23:24:38.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:38.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:38.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:38.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 10 monclient(hunting): my global_id is 84824 2024-01-24T23:24:38.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 1 -- 172.21.15.132:0/3188456934 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabc4003560 con 0x7fabe41365f0 2024-01-24T23:24:38.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 1 -- 172.21.15.132:0/3188456934 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3318917189 0 0) 0x7fabe4121f30 con 0x7fabe4135e10 2024-01-24T23:24:38.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 1 -- 172.21.15.132:0/3188456934 >> v1:172.21.15.115:6793/0 conn(0x7fabe41365f0 legacy=0x7fabe41d39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:38.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 1 -- 172.21.15.132:0/3188456934 >> v1:172.21.15.115:6791/0 conn(0x7fabe41361f0 legacy=0x7fabe41d02b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:38.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:38.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 1 monclient: found mon.i 2024-01-24T23:24:38.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:38.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:38.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 1 -- 172.21.15.132:0/3188456934 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe41224d0 con 0x7fabe4135e10 2024-01-24T23:24:38.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 10 monclient: _finish_auth 0 2024-01-24T23:24:38.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:38.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabd2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:38.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 5 monclient: authenticate success, global_id 5641 2024-01-24T23:24:38.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 10 monclient: _renew_subs 2024-01-24T23:24:38.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:38.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 1 -- 172.21.15.132:0/3188456934 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe41c88b0 con 0x7fabe4135e10 2024-01-24T23:24:38.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.214+0000 7fabe8e2b640 10 monclient: _renew_subs 2024-01-24T23:24:38.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.218+0000 7fabe8e2b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:38.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.218+0000 7fabe8e2b640 1 -- 172.21.15.132:0/3188456934 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe4135760 con 0x7fabe4135e10 2024-01-24T23:24:38.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.218+0000 7fabd2ffd640 1 -- 172.21.15.132:0/3188456934 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fabd4003950 con 0x7fabe4135e10 2024-01-24T23:24:38.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.218+0000 7fabd2ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:38.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.218+0000 7fabd2ffd640 1 -- 172.21.15.132:0/3188456934 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fabd4003e80 con 0x7fabe4135e10 2024-01-24T23:24:38.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.218+0000 7fabd2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:38.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.218+0000 7fabd2ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:38.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.218+0000 7fabe0d99640 4 set_mon_vals no callback set 2024-01-24T23:24:38.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.218+0000 7fabd2ffd640 10 monclient: dump: 2024-01-24T23:24:38.236 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:38.236 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:38.236 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:38.236 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:38.236 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:38.236 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:38.236 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:38.236 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:38.236 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:38.236 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:38.236 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:38.236 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:38.236 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:38.237 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:38.237 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:38.237 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:38.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.218+0000 7fabe8e2b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:38.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.218+0000 7fabe8e2b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:38.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.218+0000 7fabe8e2b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:38.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.218+0000 7fabe8e2b640 1 -- 172.21.15.132:0/3188456934 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc8005090 con 0x7fabe4135e10 2024-01-24T23:24:38.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.218+0000 7fabd2ffd640 1 -- 172.21.15.132:0/3188456934 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fabd4004060 con 0x7fabe4135e10 2024-01-24T23:24:38.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.218+0000 7fabd2ffd640 1 -- 172.21.15.132:0/3188456934 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(100..100 src has 1..100) v4 ==== 5178+0+0 (unknown 3672389847 0 0) 0x7fabd4043ee0 con 0x7fabe4135e10 2024-01-24T23:24:38.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.218+0000 7fabe1d9b640 10 monclient: get_auth_request con 0x7fabc4036fb0 auth_method 0 2024-01-24T23:24:38.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.222+0000 7fabd2ffd640 1 -- 172.21.15.132:0/3188456934 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fabd4014c30 con 0x7fabe4135e10 2024-01-24T23:24:38.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.222+0000 7fabd2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:38.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.222+0000 7fabd2ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:38.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.390+0000 7fabe8e2b640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["osd.11"]}] 2024-01-24T23:24:38.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.390+0000 7fabe8e2b640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["osd.11"]}] 2024-01-24T23:24:38.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.390+0000 7fabe8e2b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:38.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.390+0000 7fabe8e2b640 1 -- 172.21.15.132:0/3188456934 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd rm", "ids": ["osd.11"]} v 0) v1 -- 0x7fabc80058c0 con 0x7fabe4135e10 2024-01-24T23:24:38.392 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:24:38.390+0000 7fd2d64cd640 -1 mon.g@3(leader).osd e100 definitely_dead 0 2024-01-24T23:24:38.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.970+0000 7fabd2ffd640 1 -- 172.21.15.132:0/3188456934 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["osd.11"]}]=0 removed osd.11 v101) v1 ==== 87+0+0 (unknown 2058343242 0 0) 0x7fabd40130a0 con 0x7fabe4135e10 2024-01-24T23:24:38.975 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.970+0000 7fabd2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["osd.11"]}] 2024-01-24T23:24:38.975 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:38.970+0000 7fabd2ffd640 10 monclient: _finish_command 2 = system:0 removed osd.11 2024-01-24T23:24:39.087 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.082+0000 7fabe8e2b640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["osd.11"]}] 2024-01-24T23:24:39.087 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.082+0000 7fabe8e2b640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["osd.11"]}] 2024-01-24T23:24:39.087 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.082+0000 7fabe8e2b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:39.087 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.082+0000 7fabe8e2b640 1 -- 172.21.15.132:0/3188456934 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd rm", "ids": ["osd.11"]} v 0) v1 -- 0x7fabc8000bd0 con 0x7fabe4135e10 2024-01-24T23:24:39.087 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:24:39.086+0000 7fd2d64cd640 -1 mon.g@3(leader).osd e101 definitely_dead 0 2024-01-24T23:24:39.088 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.086+0000 7fabd2ffd640 1 -- 172.21.15.132:0/3188456934 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["osd.11"]}]=0 osd.11 does not exist. v101) v1 ==== 96+0+0 (unknown 2332964277 0 0) 0x7fabd4014a00 con 0x7fabe4135e10 2024-01-24T23:24:39.088 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.086+0000 7fabd2ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["osd.11"]}] 2024-01-24T23:24:39.088 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.086+0000 7fabd2ffd640 10 monclient: _finish_command 3 = system:0 osd.11 does not exist. 2024-01-24T23:24:39.088 INFO:tasks.workunit.client.0.smithi132.stderr:osd.11 does not exist. 2024-01-24T23:24:39.090 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.086+0000 7fabe8e2b640 1 -- 172.21.15.132:0/3188456934 >> v1:172.21.15.132:6812/1835401922 conn(0x7fabc4036fb0 legacy=0x7fabc4039470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:39.090 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.086+0000 7fabe8e2b640 10 monclient: shutdown 2024-01-24T23:24:39.090 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.086+0000 7fabe8e2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:39.090 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.086+0000 7fabe8e2b640 1 -- 172.21.15.132:0/3188456934 >> v1:172.21.15.132:6792/0 conn(0x7fabe4135e10 legacy=0x7fabe41c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:39.090 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.086+0000 7fabe8e2b640 1 -- 172.21.15.132:0/3188456934 shutdown_connections 2024-01-24T23:24:39.090 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.086+0000 7fabe8e2b640 1 -- 172.21.15.132:0/3188456934 >> 172.21.15.132:0/3188456934 conn(0x7fabe40a6790 msgr2=0x7fabe406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:39.090 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.086+0000 7fabe8e2b640 1 -- 172.21.15.132:0/3188456934 shutdown_connections 2024-01-24T23:24:39.090 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.086+0000 7fabe8e2b640 1 -- 172.21.15.132:0/3188456934 wait complete. 2024-01-24T23:24:39.105 INFO:tasks.workunit.client.0.smithi132.stderr:++ ceph osd create 2024-01-24T23:24:39.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.218+0000 7f248ca96640 10 monclient: get_monmap_and_config 2024-01-24T23:24:39.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.218+0000 7f248ca96640 10 monclient: build_initial_monmap 2024-01-24T23:24:39.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.218+0000 7f248ca96640 10 monclient: monmap: 2024-01-24T23:24:39.222 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:39.222 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:39.222 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:39.222954+0000 2024-01-24T23:24:39.222 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:39.222954+0000 2024-01-24T23:24:39.222 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:39.222 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:39.222 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:39.222 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:39.222 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:39.222 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:39.222 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:39.222 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:39.223 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:39.223 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:39.223 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:39.223 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:39.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.218+0000 7f248ca96640 1 Processor -- start 2024-01-24T23:24:39.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.218+0000 7f248ca96640 1 -- start start 2024-01-24T23:24:39.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.218+0000 7f248ca96640 10 monclient: init 2024-01-24T23:24:39.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.218+0000 7f248ca96640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:39.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.218+0000 7f248ca96640 10 monclient: _add_conns ranks=[0,5,2,7,8,6,4,3,1] 2024-01-24T23:24:39.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.218+0000 7f248ca96640 10 monclient(hunting): picked mon.noname-a con 0x7f248813b350 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:39.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.218+0000 7f248ca96640 10 monclient(hunting): picked mon.noname-h con 0x7f248813b730 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:39.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.218+0000 7f248ca96640 10 monclient(hunting): picked mon.noname-b con 0x7f248813bb50 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:39.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.218+0000 7f248ca96640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2488064700 con 0x7f248813b350 2024-01-24T23:24:39.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.218+0000 7f248ca96640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24880a2910 con 0x7f248813bb50 2024-01-24T23:24:39.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.218+0000 7f248ca96640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f248809c8c0 con 0x7f248813b730 2024-01-24T23:24:39.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.218+0000 7f248ca96640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:39.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.218+0000 7f248ca96640 10 monclient(hunting): authenticate will time out at 1166.547633s 2024-01-24T23:24:39.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.218+0000 7f2485d9b640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f248813b730 0x7f24881401d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:57504/0 (socket says 172.21.15.132:57504) 2024-01-24T23:24:39.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.218+0000 7f248659c640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f248813b350 legacy=0x7f248809db90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:39.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.218+0000 7f2485d9b640 1 -- 172.21.15.132:0/3196573526 learned_addr learned my addr 172.21.15.132:0/3196573526 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:39.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 1 -- 172.21.15.132:0/3196573526 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2782066979 0 0) 0x7f248809c8c0 con 0x7f248813b730 2024-01-24T23:24:39.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 10 monclient(hunting): _negotiate 2024-01-24T23:24:39.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:39.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:39.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 10 monclient(hunting): my global_id is 5630 2024-01-24T23:24:39.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 1 -- 172.21.15.132:0/3196573526 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2474003530 con 0x7f248813b730 2024-01-24T23:24:39.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 1 -- 172.21.15.132:0/3196573526 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1754978182 0 0) 0x7f2474003530 con 0x7f248813b730 2024-01-24T23:24:39.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 1 -- 172.21.15.132:0/3196573526 >> v1:172.21.15.115:6790/0 conn(0x7f248813bb50 legacy=0x7f2488143900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:39.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 1 -- 172.21.15.132:0/3196573526 >> v1:172.21.15.115:6789/0 conn(0x7f248813b350 legacy=0x7f248809db90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:39.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 10 monclient: _finish_hunting 0 2024-01-24T23:24:39.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 1 monclient: found mon.noname-h 2024-01-24T23:24:39.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:39.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:24:39.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 1 -- 172.21.15.132:0/3196573526 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248809cba0 con 0x7f248813b730 2024-01-24T23:24:39.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 10 monclient: _finish_auth 0 2024-01-24T23:24:39.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 10 monclient: _check_auth_tickets 2024-01-24T23:24:39.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:39.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 1 -- 172.21.15.132:0/3196573526 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f24700017c0 con 0x7f248813b730 2024-01-24T23:24:39.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 5 monclient: authenticate success, global_id 5630 2024-01-24T23:24:39.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:39.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:39.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:39.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 10 monclient: dump: 2024-01-24T23:24:39.233 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:39.233 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:39.233 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:39.233 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:39.233 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:39.233 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:39.233 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:39.233 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:39.233 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:39.233 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:39.233 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:39.233 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:39.233 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:39.234 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:39.234 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:39.234 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:39.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:39.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 1 -- 172.21.15.132:0/3196573526 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2470001a60 con 0x7f248813b730 2024-01-24T23:24:39.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:39.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 1 -- 172.21.15.132:0/3196573526 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2470004bf0 con 0x7f248813b730 2024-01-24T23:24:39.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:39.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 4 set_mon_vals no callback set 2024-01-24T23:24:39.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:39.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:39.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248559a640 10 monclient: dump: 2024-01-24T23:24:39.234 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:39.234 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:39.235 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:39.236 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 10 monclient: shutdown 2024-01-24T23:24:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 1 -- 172.21.15.132:0/3196573526 >> v1:172.21.15.132:6791/0 conn(0x7f248813b730 legacy=0x7f24881401d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 1 -- 172.21.15.132:0/3196573526 shutdown_connections 2024-01-24T23:24:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 1 -- 172.21.15.132:0/3196573526 >> 172.21.15.132:0/3196573526 conn(0x7f24880a5a10 msgr2=0x7f24880a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 1 -- 172.21.15.132:0/3196573526 shutdown_connections 2024-01-24T23:24:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 1 -- 172.21.15.132:0/3196573526 wait complete. 2024-01-24T23:24:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 10 monclient: build_initial_monmap 2024-01-24T23:24:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 10 monclient: monmap: 2024-01-24T23:24:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:39.236 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:39.237 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 1 Processor -- start 2024-01-24T23:24:39.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 1 -- start start 2024-01-24T23:24:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 10 monclient: init 2024-01-24T23:24:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 10 monclient: _add_conns ranks=[8,6,0,4,7,5,1,3,2] 2024-01-24T23:24:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 10 monclient(hunting): picked mon.noname-e con 0x7f248813b350 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 10 monclient(hunting): picked mon.noname-d con 0x7f248813b730 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 10 monclient(hunting): picked mon.noname-a con 0x7f248813bb50 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2488127930 con 0x7f248813bb50 2024-01-24T23:24:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24880720f0 con 0x7f248813b730 2024-01-24T23:24:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2488126480 con 0x7f248813b350 2024-01-24T23:24:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248ca96640 10 monclient(hunting): authenticate will time out at 1166.550731s 2024-01-24T23:24:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f2486d9d640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f248813bb50 legacy=0x7f24881d7d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248659c640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f248813b350 0x7f248813a250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:37790/0 (socket says 172.21.15.132:37790) 2024-01-24T23:24:39.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f248659c640 1 -- 172.21.15.132:0/3697290129 learned_addr learned my addr 172.21.15.132:0/3697290129 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:39.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f247effd640 1 -- 172.21.15.132:0/3697290129 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 910266157 0 0) 0x7f2488126480 con 0x7f248813b350 2024-01-24T23:24:39.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f247effd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:39.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f247effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:39.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f247effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:39.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f247effd640 10 monclient(hunting): my global_id is 84833 2024-01-24T23:24:39.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f247effd640 1 -- 172.21.15.132:0/3697290129 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2468003530 con 0x7f248813b350 2024-01-24T23:24:39.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f247effd640 1 -- 172.21.15.132:0/3697290129 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 866318731 0 0) 0x7f24880720f0 con 0x7f248813b730 2024-01-24T23:24:39.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f247effd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:39.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f247effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:39.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f247effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:39.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f247effd640 10 monclient(hunting): my global_id is 5685 2024-01-24T23:24:39.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f247effd640 1 -- 172.21.15.132:0/3697290129 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2488126480 con 0x7f248813b730 2024-01-24T23:24:39.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f247effd640 1 -- 172.21.15.132:0/3697290129 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3652807167 0 0) 0x7f2468003530 con 0x7f248813b350 2024-01-24T23:24:39.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f247effd640 1 -- 172.21.15.132:0/3697290129 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f24880720f0 con 0x7f248813b350 2024-01-24T23:24:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f247effd640 1 -- 172.21.15.132:0/3697290129 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2478004330 con 0x7f248813b350 2024-01-24T23:24:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.222+0000 7f247effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:24:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 10 monclient(hunting): dump: 2024-01-24T23:24:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:39.240 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:39.241 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 1 -- 172.21.15.132:0/3697290129 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3189190636 0 0) 0x7f2488126480 con 0x7f248813b730 2024-01-24T23:24:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 1 -- 172.21.15.132:0/3697290129 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2468003530 con 0x7f248813b730 2024-01-24T23:24:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 1 -- 172.21.15.132:0/3697290129 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f24700016c0 con 0x7f248813b730 2024-01-24T23:24:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:24:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 10 monclient(hunting): dump: 2024-01-24T23:24:39.241 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:39.242 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:39.242 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 1 -- 172.21.15.132:0/3697290129 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4076105575 0 0) 0x7f24880720f0 con 0x7f248813b350 2024-01-24T23:24:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 1 -- 172.21.15.132:0/3697290129 >> v1:172.21.15.115:6792/0 conn(0x7f248813b730 legacy=0x7f24881d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 1 -- 172.21.15.132:0/3697290129 >> v1:172.21.15.115:6789/0 conn(0x7f248813bb50 legacy=0x7f24881d7d90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 1 monclient: found mon.e 2024-01-24T23:24:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 1 -- 172.21.15.132:0/3697290129 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2488127c10 con 0x7f248813b350 2024-01-24T23:24:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 10 monclient: _finish_auth 0 2024-01-24T23:24:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f248ca96640 5 monclient: authenticate success, global_id 84833 2024-01-24T23:24:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f248ca96640 10 monclient: _renew_subs 2024-01-24T23:24:39.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f248ca96640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f248ca96640 1 -- 172.21.15.132:0/3697290129 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24881261d0 con 0x7f248813b350 2024-01-24T23:24:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f248ca96640 10 monclient: _renew_subs 2024-01-24T23:24:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f248ca96640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f248ca96640 1 -- 172.21.15.132:0/3697290129 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f24881cf750 con 0x7f248813b350 2024-01-24T23:24:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 1 -- 172.21.15.132:0/3697290129 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f24780045a0 con 0x7f248813b350 2024-01-24T23:24:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 1 -- 172.21.15.132:0/3697290129 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2478004ad0 con 0x7f248813b350 2024-01-24T23:24:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f2484d99640 4 set_mon_vals no callback set 2024-01-24T23:24:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:24:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 10 monclient: dump: 2024-01-24T23:24:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:39.244 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:39.245 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f248ca96640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:39.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f248ca96640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:39.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f248ca96640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:39.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f248ca96640 1 -- 172.21.15.132:0/3697290129 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f246c005090 con 0x7f248813b350 2024-01-24T23:24:39.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 1 -- 172.21.15.132:0/3697290129 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f2478004cb0 con 0x7f248813b350 2024-01-24T23:24:39.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f247effd640 1 -- 172.21.15.132:0/3697290129 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(101..101 src has 1..101) v4 ==== 5168+0+0 (unknown 3349015485 0 0) 0x7f2478043d50 con 0x7f248813b350 2024-01-24T23:24:39.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.226+0000 7f2485d9b640 10 monclient: get_auth_request con 0x7f24680376f0 auth_method 0 2024-01-24T23:24:39.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.238+0000 7f247effd640 1 -- 172.21.15.132:0/3697290129 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f24780149f0 con 0x7f248813b350 2024-01-24T23:24:39.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.238+0000 7f247effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:39.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.238+0000 7f247effd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:39.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.406+0000 7f248ca96640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-01-24T23:24:39.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.406+0000 7f248ca96640 10 monclient: _send_command 2 [{"prefix": "osd create"}] 2024-01-24T23:24:39.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.406+0000 7f248ca96640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:39.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:39.406+0000 7f248ca96640 1 -- 172.21.15.132:0/3697290129 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd create"} v 0) v1 -- 0x7f246c005340 con 0x7f248813b350 2024-01-24T23:24:40.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:40.006+0000 7f247effd640 1 -- 172.21.15.132:0/3697290129 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd create"}]=0 v102) v1 ==== 58+0+3 (unknown 3069545335 0 212448812) 0x7f24780145e0 con 0x7f248813b350 2024-01-24T23:24:40.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:40.006+0000 7f247effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create"}] 2024-01-24T23:24:40.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:40.006+0000 7f247effd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:24:40.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:40.118+0000 7f248ca96640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-01-24T23:24:40.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:40.118+0000 7f248ca96640 10 monclient: _send_command 3 [{"prefix": "osd create"}] 2024-01-24T23:24:40.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:40.118+0000 7f248ca96640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:40.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:40.118+0000 7f248ca96640 1 -- 172.21.15.132:0/3697290129 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd create"} v 0) v1 -- 0x7f246c001130 con 0x7f248813b350 2024-01-24T23:24:41.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.014+0000 7f247effd640 1 -- 172.21.15.132:0/3697290129 <== mon.8 v1:172.21.15.115:6793/0 11 ==== mon_command_ack([{"prefix": "osd create"}]=0 v103) v1 ==== 58+0+3 (unknown 58742147 0 944643765) 0x7f24780147c0 con 0x7f248813b350 2024-01-24T23:24:41.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.014+0000 7f247effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create"}] 2024-01-24T23:24:41.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.014+0000 7f247effd640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:24:41.020 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.018+0000 7f248ca96640 1 -- 172.21.15.132:0/3697290129 >> v1:172.21.15.132:6812/1835401922 conn(0x7f24680376f0 legacy=0x7f2468039bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:41.020 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.018+0000 7f248ca96640 10 monclient: shutdown 2024-01-24T23:24:41.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.018+0000 7f248ca96640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:41.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.018+0000 7f248ca96640 1 -- 172.21.15.132:0/3697290129 >> v1:172.21.15.115:6793/0 conn(0x7f248813b350 legacy=0x7f248813a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:41.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.018+0000 7f248ca96640 1 -- 172.21.15.132:0/3697290129 shutdown_connections 2024-01-24T23:24:41.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.018+0000 7f248ca96640 1 -- 172.21.15.132:0/3697290129 >> 172.21.15.132:0/3697290129 conn(0x7f24880a5a10 msgr2=0x7f24881c2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:41.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.018+0000 7f248ca96640 1 -- 172.21.15.132:0/3697290129 shutdown_connections 2024-01-24T23:24:41.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.018+0000 7f248ca96640 1 -- 172.21.15.132:0/3697290129 wait complete. 2024-01-24T23:24:41.037 INFO:tasks.workunit.client.0.smithi132.stderr:+ o4=12 2024-01-24T23:24:41.038 INFO:tasks.workunit.client.0.smithi132.stderr:++ ceph osd create 2024-01-24T23:24:41.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.150+0000 7f0ce2104640 10 monclient: get_monmap_and_config 2024-01-24T23:24:41.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.150+0000 7f0ce2104640 10 monclient: build_initial_monmap 2024-01-24T23:24:41.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.150+0000 7f0ce2104640 10 monclient: monmap: 2024-01-24T23:24:41.154 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:41.154 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:41.154 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:41.155044+0000 2024-01-24T23:24:41.154 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:41.155044+0000 2024-01-24T23:24:41.154 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:41.154 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:41.154 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:41.154 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:41.154 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:41.154 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:41.155 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:41.155 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:41.155 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:41.155 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:41.155 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:41.155 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:41.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.150+0000 7f0ce2104640 1 Processor -- start 2024-01-24T23:24:41.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.150+0000 7f0ce2104640 1 -- start start 2024-01-24T23:24:41.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.150+0000 7f0ce2104640 10 monclient: init 2024-01-24T23:24:41.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.150+0000 7f0ce2104640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:41.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.150+0000 7f0ce2104640 10 monclient: _add_conns ranks=[1,6,8,2,7,4,3,0,5] 2024-01-24T23:24:41.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.150+0000 7f0ce2104640 10 monclient(hunting): picked mon.noname-f con 0x7f0cdc136e50 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:41.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.150+0000 7f0ce2104640 10 monclient(hunting): picked mon.noname-d con 0x7f0cdc0a0290 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:41.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.150+0000 7f0ce2104640 10 monclient(hunting): picked mon.noname-e con 0x7f0cdc0a1600 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:41.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.150+0000 7f0ce2104640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cdc064700 con 0x7f0cdc136e50 2024-01-24T23:24:41.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.150+0000 7f0ce2104640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cdc137940 con 0x7f0cdc0a0290 2024-01-24T23:24:41.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.150+0000 7f0ce2104640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cdc09f9c0 con 0x7f0cdc0a1600 2024-01-24T23:24:41.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.150+0000 7f0ce2104640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:41.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.150+0000 7f0ce2104640 10 monclient(hunting): authenticate will time out at 1168.479702s 2024-01-24T23:24:41.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.150+0000 7f0cdb7fe640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f0cdc136e50 legacy=0x7f0cdc137230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:41.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cdaffd640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f0cdc0a0290 0x7f0cdc13bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:49592/0 (socket says 172.21.15.132:49592) 2024-01-24T23:24:41.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cdbfff640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f0cdc0a1600 0x7f0cdc13f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:37804/0 (socket says 172.21.15.132:37804) 2024-01-24T23:24:41.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cdaffd640 1 -- 172.21.15.132:0/397550347 learned_addr learned my addr 172.21.15.132:0/397550347 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:41.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 1 -- 172.21.15.132:0/397550347 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1184214001 0 0) 0x7f0cdc09f9c0 con 0x7f0cdc0a1600 2024-01-24T23:24:41.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:41.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:41.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:41.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 10 monclient(hunting): my global_id is 84842 2024-01-24T23:24:41.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 1 -- 172.21.15.132:0/397550347 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cc4003530 con 0x7f0cdc0a1600 2024-01-24T23:24:41.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 1 -- 172.21.15.132:0/397550347 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 20382488 0 0) 0x7f0cdc137940 con 0x7f0cdc0a0290 2024-01-24T23:24:41.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:41.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:41.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:41.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 10 monclient(hunting): my global_id is 5694 2024-01-24T23:24:41.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 1 -- 172.21.15.132:0/397550347 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cdc09f9c0 con 0x7f0cdc0a0290 2024-01-24T23:24:41.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 1 -- 172.21.15.132:0/397550347 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1282856873 0 0) 0x7f0cc4003530 con 0x7f0cdc0a1600 2024-01-24T23:24:41.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 1 -- 172.21.15.132:0/397550347 >> v1:172.21.15.115:6792/0 conn(0x7f0cdc0a0290 legacy=0x7f0cdc13bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:41.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 1 -- 172.21.15.132:0/397550347 >> v1:172.21.15.132:6789/0 conn(0x7f0cdc136e50 legacy=0x7f0cdc137230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:41.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:24:41.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 1 monclient: found mon.noname-e 2024-01-24T23:24:41.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:41.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:24:41.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 1 -- 172.21.15.132:0/397550347 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cdc09eda0 con 0x7f0cdc0a1600 2024-01-24T23:24:41.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 10 monclient: _finish_auth 0 2024-01-24T23:24:41.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:24:41.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:41.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 5 monclient: authenticate success, global_id 84842 2024-01-24T23:24:41.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:41.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 1 -- 172.21.15.132:0/397550347 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0cd4001b40 con 0x7f0cdc0a1600 2024-01-24T23:24:41.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:41.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:24:41.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 10 monclient: dump: 2024-01-24T23:24:41.169 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:41.169 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:41.169 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:41.169 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:41.169 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:41.169 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:41.169 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:41.170 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:41.170 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:41.170 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:41.170 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:41.170 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:41.170 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:41.170 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:41.170 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:41.170 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:41.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:41.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 1 -- 172.21.15.132:0/397550347 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0cd4001e20 con 0x7f0cdc0a1600 2024-01-24T23:24:41.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:41.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:41.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 4 set_mon_vals no callback set 2024-01-24T23:24:41.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 1 -- 172.21.15.132:0/397550347 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0cd4004eb0 con 0x7f0cdc0a1600 2024-01-24T23:24:41.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:41.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:24:41.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cda7fc640 10 monclient: dump: 2024-01-24T23:24:41.171 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:41.172 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:41.172 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:41.172 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:41.172 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:41.172 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:41.172 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:41.172 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:41.172 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:41.173 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:41.173 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:41.173 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:41.173 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:41.173 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:41.173 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:41.173 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:41.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 10 monclient: shutdown 2024-01-24T23:24:41.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:41.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 1 -- 172.21.15.132:0/397550347 >> v1:172.21.15.115:6793/0 conn(0x7f0cdc0a1600 legacy=0x7f0cdc13f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:41.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 1 -- 172.21.15.132:0/397550347 shutdown_connections 2024-01-24T23:24:41.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 1 -- 172.21.15.132:0/397550347 >> 172.21.15.132:0/397550347 conn(0x7f0cdc0a5a10 msgr2=0x7f0cdc0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:41.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 1 -- 172.21.15.132:0/397550347 shutdown_connections 2024-01-24T23:24:41.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 1 -- 172.21.15.132:0/397550347 wait complete. 2024-01-24T23:24:41.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 10 monclient: build_initial_monmap 2024-01-24T23:24:41.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 10 monclient: monmap: 2024-01-24T23:24:41.175 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:41.175 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:41.175 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:41.175 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:41.175 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:41.175 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:41.175 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:41.175 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:41.176 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:41.176 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:41.176 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:41.176 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:41.176 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:41.176 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:41.176 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:41.176 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:41.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 1 Processor -- start 2024-01-24T23:24:41.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 1 -- start start 2024-01-24T23:24:41.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 10 monclient: init 2024-01-24T23:24:41.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:41.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 10 monclient: _add_conns ranks=[2,8,0,6,7,1,5,4,3] 2024-01-24T23:24:41.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 10 monclient(hunting): picked mon.noname-b con 0x7f0cdc0a0290 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:41.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 10 monclient(hunting): picked mon.noname-e con 0x7f0cdc0a1600 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:41.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 10 monclient(hunting): picked mon.noname-a con 0x7f0cdc136e50 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:41.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cdc09ea70 con 0x7f0cdc136e50 2024-01-24T23:24:41.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cdc120180 con 0x7f0cdc0a0290 2024-01-24T23:24:41.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cdc122470 con 0x7f0cdc0a1600 2024-01-24T23:24:41.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:41.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0ce2104640 10 monclient(hunting): authenticate will time out at 1168.483556s 2024-01-24T23:24:41.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cdbfff640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f0cdc136e50 legacy=0x7f0cdc1d3ae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:41.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cdaffd640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f0cdc0a1600 0x7f0cdc1d03b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:37806/0 (socket says 172.21.15.132:37806) 2024-01-24T23:24:41.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cdaffd640 1 -- 172.21.15.132:0/2212132836 learned_addr learned my addr 172.21.15.132:0/2212132836 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:41.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.154+0000 7f0cdb7fe640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f0cdc0a0290 0x7f0cdc134ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:49170/0 (socket says 172.21.15.132:49170) 2024-01-24T23:24:41.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 1 -- 172.21.15.132:0/2212132836 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2822283739 0 0) 0x7f0cdc122470 con 0x7f0cdc0a1600 2024-01-24T23:24:41.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:41.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:41.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:41.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 10 monclient(hunting): my global_id is 84851 2024-01-24T23:24:41.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 1 -- 172.21.15.132:0/2212132836 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cc8003530 con 0x7f0cdc0a1600 2024-01-24T23:24:41.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 1 -- 172.21.15.132:0/2212132836 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425890453 0 0) 0x7f0cdc120180 con 0x7f0cdc0a0290 2024-01-24T23:24:41.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:41.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:41.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:41.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 10 monclient(hunting): my global_id is 84935 2024-01-24T23:24:41.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 1 -- 172.21.15.132:0/2212132836 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cdc122470 con 0x7f0cdc0a0290 2024-01-24T23:24:41.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 1 -- 172.21.15.132:0/2212132836 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2724046345 0 0) 0x7f0cc8003530 con 0x7f0cdc0a1600 2024-01-24T23:24:41.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 1 -- 172.21.15.132:0/2212132836 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0cdc120180 con 0x7f0cdc0a1600 2024-01-24T23:24:41.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 1 -- 172.21.15.132:0/2212132836 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1580624111 0 0) 0x7f0cdc122470 con 0x7f0cdc0a0290 2024-01-24T23:24:41.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 1 -- 172.21.15.132:0/2212132836 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0cc8003530 con 0x7f0cdc0a0290 2024-01-24T23:24:41.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 1 -- 172.21.15.132:0/2212132836 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0ccc0019b0 con 0x7f0cdc0a1600 2024-01-24T23:24:41.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:41.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:24:41.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 10 monclient(hunting): dump: 2024-01-24T23:24:41.181 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:41.181 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:41.182 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:41.182 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:41.182 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:41.182 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:41.182 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:41.182 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:41.182 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:41.182 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:41.182 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:41.182 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:41.183 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:41.183 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:41.183 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:41.183 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:41.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:41.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 1 -- 172.21.15.132:0/2212132836 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0cd0004120 con 0x7f0cdc0a0290 2024-01-24T23:24:41.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:41.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:24:41.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 10 monclient(hunting): dump: 2024-01-24T23:24:41.183 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:41.183 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:41.183 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:41.183 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:41.183 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:41.183 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:41.183 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:41.184 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:41.184 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:41.184 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:41.184 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:41.184 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:41.184 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:41.184 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:41.184 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:41.184 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:41.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 1 -- 172.21.15.132:0/2212132836 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1798089603 0 0) 0x7f0cdc120180 con 0x7f0cdc0a1600 2024-01-24T23:24:41.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 1 -- 172.21.15.132:0/2212132836 >> v1:172.21.15.115:6790/0 conn(0x7f0cdc0a0290 legacy=0x7f0cdc134ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:41.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 1 -- 172.21.15.132:0/2212132836 >> v1:172.21.15.115:6789/0 conn(0x7f0cdc136e50 legacy=0x7f0cdc1d3ae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:41.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 10 monclient: _finish_hunting 0 2024-01-24T23:24:41.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 1 monclient: found mon.e 2024-01-24T23:24:41.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:41.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:41.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 1 -- 172.21.15.132:0/2212132836 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cdc136c20 con 0x7f0cdc0a1600 2024-01-24T23:24:41.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 10 monclient: _finish_auth 0 2024-01-24T23:24:41.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 10 monclient: _check_auth_tickets 2024-01-24T23:24:41.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:41.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0ce2104640 5 monclient: authenticate success, global_id 84851 2024-01-24T23:24:41.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0ce2104640 10 monclient: _renew_subs 2024-01-24T23:24:41.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0ce2104640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:41.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0ce2104640 1 -- 172.21.15.132:0/2212132836 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cdc122190 con 0x7f0cdc0a1600 2024-01-24T23:24:41.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0ce2104640 10 monclient: _renew_subs 2024-01-24T23:24:41.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0ce2104640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:41.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0ce2104640 1 -- 172.21.15.132:0/2212132836 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cdc1353f0 con 0x7f0cdc0a1600 2024-01-24T23:24:41.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 1 -- 172.21.15.132:0/2212132836 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0ccc001be0 con 0x7f0cdc0a1600 2024-01-24T23:24:41.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:41.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 1 -- 172.21.15.132:0/2212132836 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0ccc004de0 con 0x7f0cdc0a1600 2024-01-24T23:24:41.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:41.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:24:41.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cd97fa640 4 set_mon_vals no callback set 2024-01-24T23:24:41.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 10 monclient: dump: 2024-01-24T23:24:41.186 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:41.186 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:41.186 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:41.186 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:41.186 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:41.186 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:41.186 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:41.187 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:41.187 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:41.187 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:41.187 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:41.187 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:41.187 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:41.187 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:41.187 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:41.187 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:41.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0ce2104640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:41.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0ce2104640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:41.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0ce2104640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:41.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0ce2104640 1 -- 172.21.15.132:0/2212132836 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cbc005090 con 0x7f0cdc0a1600 2024-01-24T23:24:41.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 1 -- 172.21.15.132:0/2212132836 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f0ccc004fc0 con 0x7f0cdc0a1600 2024-01-24T23:24:41.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cc3fff640 1 -- 172.21.15.132:0/2212132836 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(103..103 src has 1..103) v4 ==== 5287+0+0 (unknown 557223660 0 0) 0x7f0ccc043dd0 con 0x7f0cdc0a1600 2024-01-24T23:24:41.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.158+0000 7f0cdb7fe640 10 monclient: get_auth_request con 0x7f0cc8037790 auth_method 0 2024-01-24T23:24:41.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.170+0000 7f0cc3fff640 1 -- 172.21.15.132:0/2212132836 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f0ccc014820 con 0x7f0cdc0a1600 2024-01-24T23:24:41.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.170+0000 7f0cc3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:41.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.170+0000 7f0cc3fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:41.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.334+0000 7f0ce2104640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-01-24T23:24:41.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.334+0000 7f0ce2104640 10 monclient: _send_command 2 [{"prefix": "osd create"}] 2024-01-24T23:24:41.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.334+0000 7f0ce2104640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:41.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:41.334+0000 7f0ce2104640 1 -- 172.21.15.132:0/2212132836 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd create"} v 0) v1 -- 0x7f0cbc005340 con 0x7f0cdc0a1600 2024-01-24T23:24:42.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:42.014+0000 7f0cc3fff640 1 -- 172.21.15.132:0/2212132836 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd create"}]=0 v104) v1 ==== 58+0+3 (unknown 3188624585 0 736921282) 0x7f0ccc014640 con 0x7f0cdc0a1600 2024-01-24T23:24:42.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:42.014+0000 7f0cc3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create"}] 2024-01-24T23:24:42.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:42.014+0000 7f0cc3fff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:24:42.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:42.118+0000 7f0ce2104640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-01-24T23:24:42.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:42.118+0000 7f0ce2104640 10 monclient: _send_command 3 [{"prefix": "osd create"}] 2024-01-24T23:24:42.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:42.118+0000 7f0ce2104640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:42.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:42.118+0000 7f0ce2104640 1 -- 172.21.15.132:0/2212132836 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd create"} v 0) v1 -- 0x7f0cbc001130 con 0x7f0cdc0a1600 2024-01-24T23:24:43.032 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.030+0000 7f0cc3fff640 1 -- 172.21.15.132:0/2212132836 <== mon.8 v1:172.21.15.115:6793/0 11 ==== mon_command_ack([{"prefix": "osd create"}]=0 v105) v1 ==== 58+0+3 (unknown 192631357 0 1367428999) 0x7f0ccc014640 con 0x7f0cdc0a1600 2024-01-24T23:24:43.032 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.030+0000 7f0cc3fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create"}] 2024-01-24T23:24:43.032 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.030+0000 7f0cc3fff640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:24:43.036 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.034+0000 7f0ce2104640 1 -- 172.21.15.132:0/2212132836 >> v1:172.21.15.132:6812/1835401922 conn(0x7f0cc8037790 legacy=0x7f0cc8039c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:43.036 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.034+0000 7f0ce2104640 10 monclient: shutdown 2024-01-24T23:24:43.037 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.034+0000 7f0ce2104640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:43.037 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.034+0000 7f0ce2104640 1 -- 172.21.15.132:0/2212132836 >> v1:172.21.15.115:6793/0 conn(0x7f0cdc0a1600 legacy=0x7f0cdc1d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:43.037 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.034+0000 7f0ce2104640 1 -- 172.21.15.132:0/2212132836 shutdown_connections 2024-01-24T23:24:43.037 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.034+0000 7f0ce2104640 1 -- 172.21.15.132:0/2212132836 >> 172.21.15.132:0/2212132836 conn(0x7f0cdc0a5a10 msgr2=0x7f0cdc071210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:43.037 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.034+0000 7f0ce2104640 1 -- 172.21.15.132:0/2212132836 shutdown_connections 2024-01-24T23:24:43.037 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.034+0000 7f0ce2104640 1 -- 172.21.15.132:0/2212132836 wait complete. 2024-01-24T23:24:43.054 INFO:tasks.workunit.client.0.smithi132.stderr:+ o5=14 2024-01-24T23:24:43.054 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add 12 123 root=default host=foobaz 2024-01-24T23:24:43.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.170+0000 7fb2655a9640 10 monclient: get_monmap_and_config 2024-01-24T23:24:43.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.170+0000 7fb2655a9640 10 monclient: build_initial_monmap 2024-01-24T23:24:43.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.170+0000 7fb2655a9640 10 monclient: monmap: 2024-01-24T23:24:43.174 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:43.174 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:43.174 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:43.175619+0000 2024-01-24T23:24:43.174 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:43.175619+0000 2024-01-24T23:24:43.174 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:43.175 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:43.175 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:43.175 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:43.175 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:43.175 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:43.175 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:43.175 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:43.175 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:43.175 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:43.175 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:43.175 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:43.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.170+0000 7fb2655a9640 1 Processor -- start 2024-01-24T23:24:43.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.170+0000 7fb2655a9640 1 -- start start 2024-01-24T23:24:43.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.170+0000 7fb2655a9640 10 monclient: init 2024-01-24T23:24:43.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:43.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 10 monclient: _add_conns ranks=[0,2,6,7,3,1,5,8,4] 2024-01-24T23:24:43.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 10 monclient(hunting): picked mon.noname-a con 0x7fb260072690 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:43.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 10 monclient(hunting): picked mon.noname-b con 0x7fb26009ae90 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:43.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 10 monclient(hunting): picked mon.noname-d con 0x7fb26009bdc0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:43.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb260064700 con 0x7fb260072690 2024-01-24T23:24:43.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb26009a790 con 0x7fb26009ae90 2024-01-24T23:24:43.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2600a18f0 con 0x7fb26009bdc0 2024-01-24T23:24:43.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:43.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 10 monclient(hunting): authenticate will time out at 1170.500385s 2024-01-24T23:24:43.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25effd640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fb260072690 legacy=0x7fb2600a0ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:43.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25f7fe640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fb26009bdc0 0x7fb26013d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:49598/0 (socket says 172.21.15.132:49598) 2024-01-24T23:24:43.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25f7fe640 1 -- 172.21.15.132:0/2757758671 learned_addr learned my addr 172.21.15.132:0/2757758671 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:43.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 1 -- 172.21.15.132:0/2757758671 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1638961919 0 0) 0x7fb2600a18f0 con 0x7fb26009bdc0 2024-01-24T23:24:43.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:43.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:43.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:43.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 10 monclient(hunting): my global_id is 5703 2024-01-24T23:24:43.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 1 -- 172.21.15.132:0/2757758671 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb248003530 con 0x7fb26009bdc0 2024-01-24T23:24:43.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 1 -- 172.21.15.132:0/2757758671 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892319840 0 0) 0x7fb26009a790 con 0x7fb26009ae90 2024-01-24T23:24:43.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:43.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:43.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:43.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 10 monclient(hunting): my global_id is 84944 2024-01-24T23:24:43.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 1 -- 172.21.15.132:0/2757758671 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2600a18f0 con 0x7fb26009ae90 2024-01-24T23:24:43.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 1 -- 172.21.15.132:0/2757758671 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2905244565 0 0) 0x7fb2600a18f0 con 0x7fb26009ae90 2024-01-24T23:24:43.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 1 -- 172.21.15.132:0/2757758671 >> v1:172.21.15.115:6792/0 conn(0x7fb26009bdc0 legacy=0x7fb26013d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:43.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 1 -- 172.21.15.132:0/2757758671 >> v1:172.21.15.115:6789/0 conn(0x7fb260072690 legacy=0x7fb2600a0ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:43.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 10 monclient: _finish_hunting 0 2024-01-24T23:24:43.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 1 monclient: found mon.noname-b 2024-01-24T23:24:43.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:43.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:24:43.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 1 -- 172.21.15.132:0/2757758671 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2600997a0 con 0x7fb26009ae90 2024-01-24T23:24:43.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 10 monclient: _finish_auth 0 2024-01-24T23:24:43.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 10 monclient: _check_auth_tickets 2024-01-24T23:24:43.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:43.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 5 monclient: authenticate success, global_id 84944 2024-01-24T23:24:43.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:43.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 1 -- 172.21.15.132:0/2757758671 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb2500017a0 con 0x7fb26009ae90 2024-01-24T23:24:43.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:43.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:24:43.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 10 monclient: dump: 2024-01-24T23:24:43.186 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:43.186 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:43.186 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:43.186 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:43.186 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:43.186 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:43.186 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:43.186 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:43.186 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:43.186 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:43.186 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:43.186 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:43.186 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:43.187 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:43.187 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:43.187 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:43.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:43.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 1 -- 172.21.15.132:0/2757758671 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb2500019c0 con 0x7fb26009ae90 2024-01-24T23:24:43.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:43.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:43.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 4 set_mon_vals no callback set 2024-01-24T23:24:43.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 1 -- 172.21.15.132:0/2757758671 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb250003620 con 0x7fb26009ae90 2024-01-24T23:24:43.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:43.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:24:43.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb25dffb640 10 monclient: dump: 2024-01-24T23:24:43.187 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:43.187 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:43.188 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:43.188 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:43.188 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:43.188 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:43.188 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:43.188 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:43.188 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:43.188 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:43.188 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:43.188 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:43.188 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:43.188 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:43.188 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:43.188 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:43.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 10 monclient: shutdown 2024-01-24T23:24:43.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:43.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 1 -- 172.21.15.132:0/2757758671 >> v1:172.21.15.115:6790/0 conn(0x7fb26009ae90 legacy=0x7fb2600a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:43.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 1 -- 172.21.15.132:0/2757758671 shutdown_connections 2024-01-24T23:24:43.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 1 -- 172.21.15.132:0/2757758671 >> 172.21.15.132:0/2757758671 conn(0x7fb2600a1bc0 msgr2=0x7fb2600a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:43.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 1 -- 172.21.15.132:0/2757758671 shutdown_connections 2024-01-24T23:24:43.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 1 -- 172.21.15.132:0/2757758671 wait complete. 2024-01-24T23:24:43.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 10 monclient: build_initial_monmap 2024-01-24T23:24:43.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 10 monclient: monmap: 2024-01-24T23:24:43.189 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:43.189 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:43.189 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:43.189 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:43.189 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:43.189 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:43.189 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:43.189 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:43.189 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:43.190 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:43.190 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:43.190 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:43.190 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:43.190 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:43.190 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:43.190 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:43.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 1 Processor -- start 2024-01-24T23:24:43.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 1 -- start start 2024-01-24T23:24:43.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.174+0000 7fb2655a9640 10 monclient: init 2024-01-24T23:24:43.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2655a9640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:43.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2655a9640 10 monclient: _add_conns ranks=[6,4,2,3,7,8,0,1,5] 2024-01-24T23:24:43.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2655a9640 10 monclient(hunting): picked mon.noname-d con 0x7fb260072690 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:43.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2655a9640 10 monclient(hunting): picked mon.noname-c con 0x7fb26009ae90 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:43.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2655a9640 10 monclient(hunting): picked mon.noname-b con 0x7fb26009bdc0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:43.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2655a9640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2601c9ce0 con 0x7fb26009bdc0 2024-01-24T23:24:43.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2655a9640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb260099470 con 0x7fb26009ae90 2024-01-24T23:24:43.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2655a9640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb26009b930 con 0x7fb260072690 2024-01-24T23:24:43.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2655a9640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:43.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2655a9640 10 monclient(hunting): authenticate will time out at 1170.504249s 2024-01-24T23:24:43.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb25e7fc640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7fb26009ae90 legacy=0x7fb2601ce5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:43.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb25effd640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fb260072690 0x7fb2601b6570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:49608/0 (socket says 172.21.15.132:49608) 2024-01-24T23:24:43.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb25effd640 1 -- 172.21.15.132:0/3920109198 learned_addr learned my addr 172.21.15.132:0/3920109198 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:43.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb25f7fe640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fb26009bdc0 0x7fb2601d1cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:49202/0 (socket says 172.21.15.132:49202) 2024-01-24T23:24:43.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 1 -- 172.21.15.132:0/3920109198 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2858662431 0 0) 0x7fb26009b930 con 0x7fb260072690 2024-01-24T23:24:43.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 10 monclient(hunting): _negotiate 2024-01-24T23:24:43.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:43.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:43.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 10 monclient(hunting): my global_id is 5712 2024-01-24T23:24:43.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 1 -- 172.21.15.132:0/3920109198 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb24c003530 con 0x7fb260072690 2024-01-24T23:24:43.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 1 -- 172.21.15.132:0/3920109198 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873958737 0 0) 0x7fb2601c9ce0 con 0x7fb26009bdc0 2024-01-24T23:24:43.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 10 monclient(hunting): _negotiate 2024-01-24T23:24:43.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:43.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:43.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 10 monclient(hunting): my global_id is 84953 2024-01-24T23:24:43.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 1 -- 172.21.15.132:0/3920109198 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb26009b930 con 0x7fb26009bdc0 2024-01-24T23:24:43.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 1 -- 172.21.15.132:0/3920109198 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3202677783 0 0) 0x7fb24c003530 con 0x7fb260072690 2024-01-24T23:24:43.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 1 -- 172.21.15.132:0/3920109198 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb2601c9ce0 con 0x7fb260072690 2024-01-24T23:24:43.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 1 -- 172.21.15.132:0/3920109198 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2411462146 0 0) 0x7fb26009b930 con 0x7fb26009bdc0 2024-01-24T23:24:43.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 1 -- 172.21.15.132:0/3920109198 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb24c003530 con 0x7fb26009bdc0 2024-01-24T23:24:43.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 1 -- 172.21.15.132:0/3920109198 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb254004330 con 0x7fb260072690 2024-01-24T23:24:43.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:43.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:24:43.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 10 monclient(hunting): dump: 2024-01-24T23:24:43.193 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:43.193 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:43.193 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:43.193 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:43.193 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:43.193 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:43.193 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:43.193 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:43.193 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:43.193 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:43.193 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:43.194 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:43.194 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:43.194 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:43.194 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:43.194 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:43.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:43.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 1 -- 172.21.15.132:0/3920109198 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb258001bf0 con 0x7fb26009bdc0 2024-01-24T23:24:43.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:43.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:24:43.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 10 monclient(hunting): dump: 2024-01-24T23:24:43.194 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:43.194 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:43.194 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:43.194 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:43.194 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:43.194 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:43.195 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:43.195 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:43.195 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:43.195 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:43.195 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:43.195 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:43.195 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:43.195 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:43.195 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:43.195 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:43.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 1 -- 172.21.15.132:0/3920109198 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2418188924 0 0) 0x7fb2601c9ce0 con 0x7fb260072690 2024-01-24T23:24:43.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 1 -- 172.21.15.132:0/3920109198 >> v1:172.21.15.115:6791/0 conn(0x7fb26009ae90 legacy=0x7fb2601ce5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:43.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 1 -- 172.21.15.132:0/3920109198 >> v1:172.21.15.115:6790/0 conn(0x7fb26009bdc0 legacy=0x7fb2601d1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:43.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 10 monclient: _finish_hunting 0 2024-01-24T23:24:43.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 1 monclient: found mon.d 2024-01-24T23:24:43.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:43.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:43.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 1 -- 172.21.15.132:0/3920109198 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2601b1000 con 0x7fb260072690 2024-01-24T23:24:43.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 10 monclient: _finish_auth 0 2024-01-24T23:24:43.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 10 monclient: _check_auth_tickets 2024-01-24T23:24:43.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:43.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2655a9640 5 monclient: authenticate success, global_id 5712 2024-01-24T23:24:43.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2655a9640 10 monclient: _renew_subs 2024-01-24T23:24:43.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2655a9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:43.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2655a9640 1 -- 172.21.15.132:0/3920109198 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2601af380 con 0x7fb260072690 2024-01-24T23:24:43.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2655a9640 10 monclient: _renew_subs 2024-01-24T23:24:43.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2655a9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:43.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2655a9640 1 -- 172.21.15.132:0/3920109198 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2601afa20 con 0x7fb260072690 2024-01-24T23:24:43.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 1 -- 172.21.15.132:0/3920109198 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb254004520 con 0x7fb260072690 2024-01-24T23:24:43.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:43.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 1 -- 172.21.15.132:0/3920109198 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb254004a50 con 0x7fb260072690 2024-01-24T23:24:43.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:43.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb25d7fa640 4 set_mon_vals no callback set 2024-01-24T23:24:43.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:24:43.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 10 monclient: dump: 2024-01-24T23:24:43.197 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:43.197 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:43.197 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:43.197 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:43.197 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:43.197 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:43.197 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:43.197 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:43.197 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:43.197 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:43.198 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:43.198 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:43.198 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:43.198 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:43.198 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:43.198 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:43.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2655a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:43.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2655a9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:43.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2655a9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:43.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2655a9640 1 -- 172.21.15.132:0/3920109198 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb240005090 con 0x7fb260072690 2024-01-24T23:24:43.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 1 -- 172.21.15.132:0/3920109198 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fb254004c30 con 0x7fb260072690 2024-01-24T23:24:43.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.178+0000 7fb2477fe640 1 -- 172.21.15.132:0/3920109198 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(105..105 src has 1..105) v4 ==== 5525+0+0 (unknown 1695421410 0 0) 0x7fb254043e60 con 0x7fb260072690 2024-01-24T23:24:43.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.182+0000 7fb25e7fc640 10 monclient: get_auth_request con 0x7fb24c0376c0 auth_method 0 2024-01-24T23:24:43.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.190+0000 7fb2477fe640 1 -- 172.21.15.132:0/3920109198 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fb254014990 con 0x7fb260072690 2024-01-24T23:24:43.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.190+0000 7fb2477fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:43.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.190+0000 7fb2477fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:43.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.350+0000 7fb2655a9640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-01-24T23:24:43.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.350+0000 7fb2655a9640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-01-24T23:24:43.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.350+0000 7fb2655a9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:43.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:43.350+0000 7fb2655a9640 1 -- 172.21.15.132:0/3920109198 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]} v 0) v1 -- 0x7fb240005340 con 0x7fb260072690 2024-01-24T23:24:44.047 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.042+0000 7fb2477fe640 1 -- 172.21.15.132:0/3920109198 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}]=0 add item id 12 name 'osd.12' weight 123 at location {host=foobaz,root=default} to crush map v106) v1 ==== 220+0+0 (unknown 4224221045 0 0) 0x7fb254012e00 con 0x7fb260072690 2024-01-24T23:24:44.047 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.042+0000 7fb2477fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-01-24T23:24:44.047 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.042+0000 7fb2477fe640 10 monclient: _finish_command 2 = system:0 add item id 12 name 'osd.12' weight 123 at location {host=foobaz,root=default} to crush map 2024-01-24T23:24:44.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.146+0000 7fb2655a9640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-01-24T23:24:44.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.146+0000 7fb2655a9640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-01-24T23:24:44.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.146+0000 7fb2655a9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:44.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.146+0000 7fb2655a9640 1 -- 172.21.15.132:0/3920109198 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]} v 0) v1 -- 0x7fb240005bc0 con 0x7fb260072690 2024-01-24T23:24:44.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.150+0000 7fb2477fe640 1 -- 172.21.15.132:0/3920109198 <== mon.6 v1:172.21.15.115:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}]=0 set item id 12 name 'osd.12' weight 123 at location {host=foobaz,root=default}: no change v106) v1 ==== 218+0+0 (unknown 1833648878 0 0) 0x7fb254014580 con 0x7fb260072690 2024-01-24T23:24:44.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.150+0000 7fb2477fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-01-24T23:24:44.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.150+0000 7fb2477fe640 10 monclient: _finish_command 3 = system:0 set item id 12 name 'osd.12' weight 123 at location {host=foobaz,root=default}: no change 2024-01-24T23:24:44.153 INFO:tasks.workunit.client.0.smithi132.stderr:set item id 12 name 'osd.12' weight 123 at location {host=foobaz,root=default}: no change 2024-01-24T23:24:44.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.154+0000 7fb2655a9640 1 -- 172.21.15.132:0/3920109198 >> v1:172.21.15.132:6812/1835401922 conn(0x7fb24c0376c0 legacy=0x7fb24c039b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:44.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.154+0000 7fb2655a9640 10 monclient: shutdown 2024-01-24T23:24:44.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.154+0000 7fb2655a9640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:44.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.154+0000 7fb2655a9640 1 -- 172.21.15.132:0/3920109198 >> v1:172.21.15.115:6792/0 conn(0x7fb260072690 legacy=0x7fb2601b6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:44.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.154+0000 7fb2655a9640 1 -- 172.21.15.132:0/3920109198 shutdown_connections 2024-01-24T23:24:44.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.154+0000 7fb2655a9640 1 -- 172.21.15.132:0/3920109198 >> 172.21.15.132:0/3920109198 conn(0x7fb2600a1bc0 msgr2=0x7fb2601bc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:44.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.154+0000 7fb2655a9640 1 -- 172.21.15.132:0/3920109198 shutdown_connections 2024-01-24T23:24:44.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.154+0000 7fb2655a9640 1 -- 172.21.15.132:0/3920109198 wait complete. 2024-01-24T23:24:44.171 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add 14 123 root=default host=foobaz 2024-01-24T23:24:44.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.290+0000 7f6d7ab8d640 10 monclient: get_monmap_and_config 2024-01-24T23:24:44.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.290+0000 7f6d7ab8d640 10 monclient: build_initial_monmap 2024-01-24T23:24:44.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.290+0000 7f6d7ab8d640 10 monclient: monmap: 2024-01-24T23:24:44.294 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:44.294 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:44.294 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:44.295560+0000 2024-01-24T23:24:44.294 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:44.295560+0000 2024-01-24T23:24:44.294 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:44.294 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:44.294 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:44.295 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:44.295 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:44.295 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:44.295 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:44.295 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:44.295 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:44.295 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:44.295 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:44.295 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:44.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.290+0000 7f6d7ab8d640 1 Processor -- start 2024-01-24T23:24:44.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.290+0000 7f6d7ab8d640 1 -- start start 2024-01-24T23:24:44.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.290+0000 7f6d7ab8d640 10 monclient: init 2024-01-24T23:24:44.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:44.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 10 monclient: _add_conns ranks=[6,3,4,0,1,5,8,2,7] 2024-01-24T23:24:44.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 10 monclient(hunting): picked mon.noname-d con 0x7f6d74133ed0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:44.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 10 monclient(hunting): picked mon.noname-g con 0x7f6d741342b0 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:44.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 10 monclient(hunting): picked mon.noname-c con 0x7f6d74134690 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:44.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d74064700 con 0x7f6d741342b0 2024-01-24T23:24:44.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d7409b3d0 con 0x7f6d74134690 2024-01-24T23:24:44.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d740a58c0 con 0x7f6d74133ed0 2024-01-24T23:24:44.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:44.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 10 monclient(hunting): authenticate will time out at 1171.620607s 2024-01-24T23:24:44.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d73fff640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f6d741342b0 0x7f6d7413c040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:33658/0 (socket says 172.21.15.132:33658) 2024-01-24T23:24:44.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d73fff640 1 -- 172.21.15.132:0/519200049 learned_addr learned my addr 172.21.15.132:0/519200049 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:44.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7912a640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f6d74134690 legacy=0x7f6d7413f770 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:44.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 1 -- 172.21.15.132:0/519200049 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 744505426 0 0) 0x7f6d74064700 con 0x7f6d741342b0 2024-01-24T23:24:44.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 10 monclient(hunting): _negotiate 2024-01-24T23:24:44.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:44.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:44.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 10 monclient(hunting): my global_id is 5655 2024-01-24T23:24:44.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 1 -- 172.21.15.132:0/519200049 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d5c003530 con 0x7f6d741342b0 2024-01-24T23:24:44.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 1 -- 172.21.15.132:0/519200049 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2482275571 0 0) 0x7f6d5c003530 con 0x7f6d741342b0 2024-01-24T23:24:44.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 1 -- 172.21.15.132:0/519200049 >> v1:172.21.15.115:6792/0 conn(0x7f6d74133ed0 legacy=0x7f6d7409bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:44.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 1 -- 172.21.15.132:0/519200049 >> v1:172.21.15.115:6791/0 conn(0x7f6d74134690 legacy=0x7f6d7413f770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:44.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 10 monclient: _finish_hunting 0 2024-01-24T23:24:44.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 1 monclient: found mon.noname-g 2024-01-24T23:24:44.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:44.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:24:44.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 1 -- 172.21.15.132:0/519200049 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7409b0f0 con 0x7f6d741342b0 2024-01-24T23:24:44.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 10 monclient: _finish_auth 0 2024-01-24T23:24:44.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 10 monclient: _check_auth_tickets 2024-01-24T23:24:44.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:44.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 1 -- 172.21.15.132:0/519200049 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6d680017c0 con 0x7f6d741342b0 2024-01-24T23:24:44.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 5 monclient: authenticate success, global_id 5655 2024-01-24T23:24:44.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:44.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:44.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:44.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 10 monclient: dump: 2024-01-24T23:24:44.305 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:44.305 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:44.305 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:44.305 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:44.306 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:44.306 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:44.306 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:44.306 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:44.306 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:44.306 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:44.306 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:44.306 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:44.306 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:44.306 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:44.306 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:44.306 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:44.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:44.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 1 -- 172.21.15.132:0/519200049 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6d68001ae0 con 0x7f6d741342b0 2024-01-24T23:24:44.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:44.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 1 -- 172.21.15.132:0/519200049 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6d68004bf0 con 0x7f6d741342b0 2024-01-24T23:24:44.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:44.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 4 set_mon_vals no callback set 2024-01-24T23:24:44.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:44.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:44.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d737fe640 10 monclient: dump: 2024-01-24T23:24:44.307 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:44.307 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:44.307 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:44.307 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:44.307 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:44.307 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:44.307 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:44.307 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:44.307 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:44.308 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:44.308 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:44.308 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:44.308 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:44.308 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:44.308 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:44.308 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:44.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 10 monclient: shutdown 2024-01-24T23:24:44.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:44.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 1 -- 172.21.15.132:0/519200049 >> v1:172.21.15.132:6790/0 conn(0x7f6d741342b0 legacy=0x7f6d7413c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:44.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 1 -- 172.21.15.132:0/519200049 shutdown_connections 2024-01-24T23:24:44.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 1 -- 172.21.15.132:0/519200049 >> 172.21.15.132:0/519200049 conn(0x7f6d740a5ef0 msgr2=0x7f6d740a6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:44.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 1 -- 172.21.15.132:0/519200049 shutdown_connections 2024-01-24T23:24:44.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 1 -- 172.21.15.132:0/519200049 wait complete. 2024-01-24T23:24:44.309 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 10 monclient: build_initial_monmap 2024-01-24T23:24:44.309 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 10 monclient: monmap: 2024-01-24T23:24:44.309 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:44.309 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:44.309 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:44.309 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:44.309 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:44.309 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:44.309 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:44.309 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:44.309 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:44.309 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:44.309 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:44.309 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:44.309 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:44.309 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:44.310 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:44.310 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:44.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 1 Processor -- start 2024-01-24T23:24:44.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 1 -- start start 2024-01-24T23:24:44.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 10 monclient: init 2024-01-24T23:24:44.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:44.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 10 monclient: _add_conns ranks=[6,2,8,5,4,0,3,1,7] 2024-01-24T23:24:44.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 10 monclient(hunting): picked mon.noname-d con 0x7f6d74133ed0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:44.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 10 monclient(hunting): picked mon.noname-b con 0x7f6d741342b0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:44.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 10 monclient(hunting): picked mon.noname-e con 0x7f6d74134690 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:44.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d741215a0 con 0x7f6d741342b0 2024-01-24T23:24:44.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d74123890 con 0x7f6d74133ed0 2024-01-24T23:24:44.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d740a5af0 con 0x7f6d74134690 2024-01-24T23:24:44.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:44.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d7ab8d640 10 monclient(hunting): authenticate will time out at 1171.623722s 2024-01-24T23:24:44.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d78929640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f6d74133ed0 0x7f6d74132e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:33132/0 (socket says 172.21.15.132:33132) 2024-01-24T23:24:44.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.294+0000 7f6d78929640 1 -- 172.21.15.132:0/1186950117 learned_addr learned my addr 172.21.15.132:0/1186950117 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:44.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 852571212 0 0) 0x7f6d74123890 con 0x7f6d74133ed0 2024-01-24T23:24:44.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient(hunting): _negotiate 2024-01-24T23:24:44.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:44.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:44.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient(hunting): my global_id is 5730 2024-01-24T23:24:44.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d60003530 con 0x7f6d74133ed0 2024-01-24T23:24:44.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1239656702 0 0) 0x7f6d740a5af0 con 0x7f6d74134690 2024-01-24T23:24:44.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient(hunting): _negotiate 2024-01-24T23:24:44.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:44.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:44.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient(hunting): my global_id is 84860 2024-01-24T23:24:44.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d74123890 con 0x7f6d74134690 2024-01-24T23:24:44.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2980014924 0 0) 0x7f6d741215a0 con 0x7f6d741342b0 2024-01-24T23:24:44.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient(hunting): _negotiate 2024-01-24T23:24:44.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:44.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:44.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient(hunting): my global_id is 84962 2024-01-24T23:24:44.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d740a5af0 con 0x7f6d741342b0 2024-01-24T23:24:44.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3486518516 0 0) 0x7f6d740a5af0 con 0x7f6d741342b0 2024-01-24T23:24:44.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6d741215a0 con 0x7f6d741342b0 2024-01-24T23:24:44.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6d68004b30 con 0x7f6d741342b0 2024-01-24T23:24:44.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:44.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:24:44.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient(hunting): dump: 2024-01-24T23:24:44.312 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:44.312 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:44.312 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:44.312 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:44.312 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:44.312 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:44.312 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:44.313 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:44.313 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:44.313 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:44.313 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:44.313 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:44.313 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:44.313 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:44.313 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:44.313 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:44.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:44.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3799198073 0 0) 0x7f6d74123890 con 0x7f6d74134690 2024-01-24T23:24:44.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6d740a5af0 con 0x7f6d74134690 2024-01-24T23:24:44.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1023035061 0 0) 0x7f6d60003530 con 0x7f6d74133ed0 2024-01-24T23:24:44.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6d74123890 con 0x7f6d74133ed0 2024-01-24T23:24:44.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6d64001b20 con 0x7f6d74133ed0 2024-01-24T23:24:44.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:44.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:24:44.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient(hunting): dump: 2024-01-24T23:24:44.314 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:44.314 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:44.314 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:44.314 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:44.314 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:44.314 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:44.314 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:44.314 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:44.314 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:44.314 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:44.314 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:44.315 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:44.315 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:44.315 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:44.315 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:44.315 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:44.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6d6c004560 con 0x7f6d74134690 2024-01-24T23:24:44.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:44.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:24:44.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient(hunting): dump: 2024-01-24T23:24:44.315 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:44.315 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:44.315 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:44.315 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:44.315 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:44.315 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:44.315 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:44.316 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:44.316 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:44.316 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:44.316 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:44.316 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:44.316 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:44.316 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:44.316 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:44.316 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:44.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3125895286 0 0) 0x7f6d741215a0 con 0x7f6d741342b0 2024-01-24T23:24:44.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 >> v1:172.21.15.115:6793/0 conn(0x7f6d74134690 legacy=0x7f6d741d3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:44.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 >> v1:172.21.15.115:6792/0 conn(0x7f6d74133ed0 legacy=0x7f6d74132e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:44.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient: _finish_hunting 0 2024-01-24T23:24:44.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 1 monclient: found mon.b 2024-01-24T23:24:44.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:44.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:44.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7409ca90 con 0x7f6d741342b0 2024-01-24T23:24:44.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient: _finish_auth 0 2024-01-24T23:24:44.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient: _check_auth_tickets 2024-01-24T23:24:44.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:44.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d7ab8d640 5 monclient: authenticate success, global_id 84962 2024-01-24T23:24:44.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d7ab8d640 10 monclient: _renew_subs 2024-01-24T23:24:44.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d7ab8d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:44.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d7ab8d640 1 -- 172.21.15.132:0/1186950117 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d741235b0 con 0x7f6d741342b0 2024-01-24T23:24:44.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d7ab8d640 10 monclient: _renew_subs 2024-01-24T23:24:44.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d7ab8d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:44.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d7ab8d640 1 -- 172.21.15.132:0/1186950117 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d74133590 con 0x7f6d741342b0 2024-01-24T23:24:44.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6d68001810 con 0x7f6d741342b0 2024-01-24T23:24:44.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:44.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6d68001cd0 con 0x7f6d741342b0 2024-01-24T23:24:44.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:44.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d72ffd640 4 set_mon_vals no callback set 2024-01-24T23:24:44.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:24:44.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 10 monclient: dump: 2024-01-24T23:24:44.318 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:44.318 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:44.318 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:44.318 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:44.318 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:44.318 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:44.318 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:44.318 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:44.318 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:44.319 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:44.319 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:44.319 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:44.319 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:44.319 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:44.319 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:44.319 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:44.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d7ab8d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:44.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d7ab8d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:44.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d7ab8d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:44.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d7ab8d640 1 -- 172.21.15.132:0/1186950117 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d54005090 con 0x7f6d741342b0 2024-01-24T23:24:44.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f6d68001070 con 0x7f6d741342b0 2024-01-24T23:24:44.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (unknown 3082728554 0 0) 0x7f6d68043f40 con 0x7f6d741342b0 2024-01-24T23:24:44.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.298+0000 7f6d78929640 10 monclient: get_auth_request con 0x7f6d60037c50 auth_method 0 2024-01-24T23:24:44.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.310+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 <== mon.2 v1:172.21.15.115:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f6d680148b0 con 0x7f6d741342b0 2024-01-24T23:24:44.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.310+0000 7f6d717fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:44.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.310+0000 7f6d717fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:44.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.466+0000 7f6d7ab8d640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-01-24T23:24:44.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.466+0000 7f6d7ab8d640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-01-24T23:24:44.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.466+0000 7f6d7ab8d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:44.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:44.466+0000 7f6d7ab8d640 1 -- 172.21.15.132:0/1186950117 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]} v 0) v1 -- 0x7f6d54005340 con 0x7f6d741342b0 2024-01-24T23:24:45.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.046+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 <== mon.2 v1:172.21.15.115:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}]=0 add item id 14 name 'osd.14' weight 123 at location {host=foobaz,root=default} to crush map v107) v1 ==== 220+0+0 (unknown 3789408296 0 0) 0x7f6d680146d0 con 0x7f6d741342b0 2024-01-24T23:24:45.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.046+0000 7f6d717fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-01-24T23:24:45.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.046+0000 7f6d717fa640 10 monclient: _finish_command 2 = system:0 add item id 14 name 'osd.14' weight 123 at location {host=foobaz,root=default} to crush map 2024-01-24T23:24:45.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.146+0000 7f6d7ab8d640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-01-24T23:24:45.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.146+0000 7f6d7ab8d640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-01-24T23:24:45.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.146+0000 7f6d7ab8d640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:45.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.146+0000 7f6d7ab8d640 1 -- 172.21.15.132:0/1186950117 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]} v 0) v1 -- 0x7f6d54005bc0 con 0x7f6d741342b0 2024-01-24T23:24:45.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.150+0000 7f6d717fa640 1 -- 172.21.15.132:0/1186950117 <== mon.2 v1:172.21.15.115:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}]=0 set item id 14 name 'osd.14' weight 123 at location {host=foobaz,root=default}: no change v107) v1 ==== 218+0+0 (unknown 3670485280 0 0) 0x7f6d680146d0 con 0x7f6d741342b0 2024-01-24T23:24:45.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.150+0000 7f6d717fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-01-24T23:24:45.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.150+0000 7f6d717fa640 10 monclient: _finish_command 3 = system:0 set item id 14 name 'osd.14' weight 123 at location {host=foobaz,root=default}: no change 2024-01-24T23:24:45.152 INFO:tasks.workunit.client.0.smithi132.stderr:set item id 14 name 'osd.14' weight 123 at location {host=foobaz,root=default}: no change 2024-01-24T23:24:45.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.150+0000 7f6d7ab8d640 1 -- 172.21.15.132:0/1186950117 >> v1:172.21.15.132:6812/1835401922 conn(0x7f6d60037c50 legacy=0x7f6d6003a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:45.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.150+0000 7f6d7ab8d640 10 monclient: shutdown 2024-01-24T23:24:45.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.150+0000 7f6d7ab8d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:45.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.150+0000 7f6d7ab8d640 1 -- 172.21.15.132:0/1186950117 >> v1:172.21.15.115:6790/0 conn(0x7f6d741342b0 legacy=0x7f6d741d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:45.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.150+0000 7f6d7ab8d640 1 -- 172.21.15.132:0/1186950117 shutdown_connections 2024-01-24T23:24:45.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.150+0000 7f6d7ab8d640 1 -- 172.21.15.132:0/1186950117 >> 172.21.15.132:0/1186950117 conn(0x7f6d740a5ef0 msgr2=0x7f6d740a71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:45.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.150+0000 7f6d7ab8d640 1 -- 172.21.15.132:0/1186950117 shutdown_connections 2024-01-24T23:24:45.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.150+0000 7f6d7ab8d640 1 -- 172.21.15.132:0/1186950117 wait complete. 2024-01-24T23:24:45.169 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd tree 2024-01-24T23:24:45.169 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep 123 2024-01-24T23:24:45.169 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep osd.12 2024-01-24T23:24:45.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.278+0000 7fe29fc27640 10 monclient: get_monmap_and_config 2024-01-24T23:24:45.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.278+0000 7fe29fc27640 10 monclient: build_initial_monmap 2024-01-24T23:24:45.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 10 monclient: monmap: 2024-01-24T23:24:45.283 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:45.284 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:45.284 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:45.284799+0000 2024-01-24T23:24:45.284 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:45.284799+0000 2024-01-24T23:24:45.284 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:45.284 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:45.284 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:45.284 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:45.284 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:45.284 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:45.284 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:45.284 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:45.284 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:45.284 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:45.285 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:45.285 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:45.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 1 Processor -- start 2024-01-24T23:24:45.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 1 -- start start 2024-01-24T23:24:45.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 10 monclient: init 2024-01-24T23:24:45.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:45.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 10 monclient: _add_conns ranks=[3,6,1,8,5,7,2,0,4] 2024-01-24T23:24:45.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 10 monclient(hunting): picked mon.noname-g con 0x7fe298135e10 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:45.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 10 monclient(hunting): picked mon.noname-d con 0x7fe2981361f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:45.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 10 monclient(hunting): picked mon.noname-f con 0x7fe2981365f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:45.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe298064700 con 0x7fe2981365f0 2024-01-24T23:24:45.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe29809c3e0 con 0x7fe298135e10 2024-01-24T23:24:45.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2980a4f90 con 0x7fe2981361f0 2024-01-24T23:24:45.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:45.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 10 monclient(hunting): authenticate will time out at 1172.609715s 2024-01-24T23:24:45.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29e1c4640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fe2981365f0 legacy=0x7fe29813f600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:45.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29d9c3640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fe298135e10 0x7fe29809bcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:33664/0 (socket says 172.21.15.132:33664) 2024-01-24T23:24:45.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29d9c3640 1 -- 172.21.15.132:0/2006701453 learned_addr learned my addr 172.21.15.132:0/2006701453 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:45.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 1 -- 172.21.15.132:0/2006701453 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3700152821 0 0) 0x7fe29809c3e0 con 0x7fe298135e10 2024-01-24T23:24:45.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 10 monclient(hunting): _negotiate 2024-01-24T23:24:45.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:45.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:45.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 10 monclient(hunting): my global_id is 5664 2024-01-24T23:24:45.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 1 -- 172.21.15.132:0/2006701453 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe284003530 con 0x7fe298135e10 2024-01-24T23:24:45.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 1 -- 172.21.15.132:0/2006701453 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3953450127 0 0) 0x7fe284003530 con 0x7fe298135e10 2024-01-24T23:24:45.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 1 -- 172.21.15.132:0/2006701453 >> v1:172.21.15.115:6792/0 conn(0x7fe2981361f0 legacy=0x7fe29813bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:45.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 1 -- 172.21.15.132:0/2006701453 >> v1:172.21.15.132:6789/0 conn(0x7fe2981365f0 legacy=0x7fe29813f600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:45.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 10 monclient: _finish_hunting 0 2024-01-24T23:24:45.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 1 monclient: found mon.noname-g 2024-01-24T23:24:45.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:45.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:24:45.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 1 -- 172.21.15.132:0/2006701453 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29809b010 con 0x7fe298135e10 2024-01-24T23:24:45.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 10 monclient: _finish_auth 0 2024-01-24T23:24:45.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 10 monclient: _check_auth_tickets 2024-01-24T23:24:45.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:45.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 1 -- 172.21.15.132:0/2006701453 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe294001b40 con 0x7fe298135e10 2024-01-24T23:24:45.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:45.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:45.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 10 monclient: dump: 2024-01-24T23:24:45.293 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:45.293 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:45.293 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:45.294 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:45.294 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:45.294 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:45.294 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:45.294 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:45.294 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:45.294 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:45.294 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:45.294 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:45.294 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:45.294 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:45.294 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:45.294 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:45.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 5 monclient: authenticate success, global_id 5664 2024-01-24T23:24:45.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:45.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 1 -- 172.21.15.132:0/2006701453 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe294001e60 con 0x7fe298135e10 2024-01-24T23:24:45.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:45.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 1 -- 172.21.15.132:0/2006701453 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe294004f70 con 0x7fe298135e10 2024-01-24T23:24:45.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:45.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:45.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29c9c1640 10 monclient: dump: 2024-01-24T23:24:45.295 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:45.295 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:45.295 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:45.295 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:45.295 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:45.295 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:45.295 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:45.295 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:45.295 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:45.296 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:45.296 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:45.296 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:45.296 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:45.296 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:45.296 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:45.296 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:45.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:45.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 4 set_mon_vals no callback set 2024-01-24T23:24:45.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 10 monclient: shutdown 2024-01-24T23:24:45.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:45.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 1 -- 172.21.15.132:0/2006701453 >> v1:172.21.15.132:6790/0 conn(0x7fe298135e10 legacy=0x7fe29809bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:45.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 1 -- 172.21.15.132:0/2006701453 shutdown_connections 2024-01-24T23:24:45.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 1 -- 172.21.15.132:0/2006701453 >> 172.21.15.132:0/2006701453 conn(0x7fe2980a6790 msgr2=0x7fe298071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:45.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 1 -- 172.21.15.132:0/2006701453 shutdown_connections 2024-01-24T23:24:45.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 1 -- 172.21.15.132:0/2006701453 wait complete. 2024-01-24T23:24:45.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 10 monclient: build_initial_monmap 2024-01-24T23:24:45.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.282+0000 7fe29fc27640 10 monclient: monmap: 2024-01-24T23:24:45.297 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:45.297 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:45.297 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:45.297 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:45.297 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:45.297 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:45.297 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:45.297 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:45.297 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:45.297 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:45.297 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:45.297 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:45.297 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:45.298 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:45.298 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:45.298 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:45.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29fc27640 1 Processor -- start 2024-01-24T23:24:45.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29fc27640 1 -- start start 2024-01-24T23:24:45.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29fc27640 10 monclient: init 2024-01-24T23:24:45.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29fc27640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:45.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29fc27640 10 monclient: _add_conns ranks=[4,0,7,2,5,6,3,1,8] 2024-01-24T23:24:45.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29fc27640 10 monclient(hunting): picked mon.noname-c con 0x7fe298135e10 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:45.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29fc27640 10 monclient(hunting): picked mon.noname-a con 0x7fe2981361f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:45.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29fc27640 10 monclient(hunting): picked mon.noname-i con 0x7fe2981365f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:45.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29fc27640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe298121c30 con 0x7fe2981361f0 2024-01-24T23:24:45.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29fc27640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe298120de0 con 0x7fe298135e10 2024-01-24T23:24:45.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29fc27640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2981221f0 con 0x7fe2981365f0 2024-01-24T23:24:45.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29fc27640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:45.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29fc27640 10 monclient(hunting): authenticate will time out at 1172.612790s 2024-01-24T23:24:45.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29d9c3640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7fe298135e10 legacy=0x7fe298134b10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:45.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29d1c2640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fe2981361f0 legacy=0x7fe2981d0380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:45.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29e1c4640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fe2981365f0 0x7fe2981d3ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:48640/0 (socket says 172.21.15.132:48640) 2024-01-24T23:24:45.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29e1c4640 1 -- 172.21.15.132:0/3600235640 learned_addr learned my addr 172.21.15.132:0/3600235640 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:45.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 1 -- 172.21.15.132:0/3600235640 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 8992008 0 0) 0x7fe2981221f0 con 0x7fe2981365f0 2024-01-24T23:24:45.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:45.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:45.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:45.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 10 monclient(hunting): my global_id is 5650 2024-01-24T23:24:45.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 1 -- 172.21.15.132:0/3600235640 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe27c003530 con 0x7fe2981365f0 2024-01-24T23:24:45.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 1 -- 172.21.15.132:0/3600235640 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 589038469 0 0) 0x7fe27c003530 con 0x7fe2981365f0 2024-01-24T23:24:45.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 1 -- 172.21.15.132:0/3600235640 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe2981221f0 con 0x7fe2981365f0 2024-01-24T23:24:45.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 1 -- 172.21.15.132:0/3600235640 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe2900041e0 con 0x7fe2981365f0 2024-01-24T23:24:45.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:45.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:45.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 10 monclient(hunting): dump: 2024-01-24T23:24:45.300 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:45.300 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:45.300 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:45.300 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:45.300 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:45.301 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:45.301 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:45.301 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:45.301 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:45.301 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:45.301 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:45.301 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:45.301 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:45.301 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:45.301 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:45.301 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:45.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:45.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 1 -- 172.21.15.132:0/3600235640 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2221936991 0 0) 0x7fe2981221f0 con 0x7fe2981365f0 2024-01-24T23:24:45.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 1 -- 172.21.15.132:0/3600235640 >> v1:172.21.15.115:6791/0 conn(0x7fe298135e10 legacy=0x7fe298134b10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:45.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 1 -- 172.21.15.132:0/3600235640 >> v1:172.21.15.115:6789/0 conn(0x7fe2981361f0 legacy=0x7fe2981d0380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:45.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:24:45.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 1 monclient: found mon.i 2024-01-24T23:24:45.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:45.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:45.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 1 -- 172.21.15.132:0/3600235640 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29811fff0 con 0x7fe2981365f0 2024-01-24T23:24:45.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 10 monclient: _finish_auth 0 2024-01-24T23:24:45.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:24:45.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:45.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29fc27640 5 monclient: authenticate success, global_id 5650 2024-01-24T23:24:45.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29fc27640 10 monclient: _renew_subs 2024-01-24T23:24:45.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29fc27640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:45.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29fc27640 1 -- 172.21.15.132:0/3600235640 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29809ca00 con 0x7fe2981365f0 2024-01-24T23:24:45.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29fc27640 10 monclient: _renew_subs 2024-01-24T23:24:45.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29fc27640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:45.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29fc27640 1 -- 172.21.15.132:0/3600235640 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29809faa0 con 0x7fe2981365f0 2024-01-24T23:24:45.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 1 -- 172.21.15.132:0/3600235640 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe290004490 con 0x7fe2981365f0 2024-01-24T23:24:45.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:45.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 1 -- 172.21.15.132:0/3600235640 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe2900049c0 con 0x7fe2981365f0 2024-01-24T23:24:45.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:45.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28ffff640 4 set_mon_vals no callback set 2024-01-24T23:24:45.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:45.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 10 monclient: dump: 2024-01-24T23:24:45.303 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:45.303 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:45.303 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:45.303 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:45.303 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:45.303 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:45.303 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:45.304 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:45.304 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:45.304 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:45.304 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:45.304 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:45.304 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:45.304 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:45.304 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:45.304 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:45.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 1 -- 172.21.15.132:0/3600235640 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fe290004b20 con 0x7fe2981365f0 2024-01-24T23:24:45.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29fc27640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:45.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29fc27640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:45.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29fc27640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:45.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29fc27640 1 -- 172.21.15.132:0/3600235640 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe280005090 con 0x7fe2981365f0 2024-01-24T23:24:45.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe28e7fc640 1 -- 172.21.15.132:0/3600235640 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(107..107 src has 1..107) v4 ==== 5789+0+0 (unknown 367258303 0 0) 0x7fe290043ed0 con 0x7fe2981365f0 2024-01-24T23:24:45.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.286+0000 7fe29d9c3640 10 monclient: get_auth_request con 0x7fe27c036f90 auth_method 0 2024-01-24T23:24:45.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.290+0000 7fe28e7fc640 1 -- 172.21.15.132:0/3600235640 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fe290014900 con 0x7fe2981365f0 2024-01-24T23:24:45.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.290+0000 7fe28e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:45.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.290+0000 7fe28e7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:45.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.446+0000 7fe29fc27640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:45.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.446+0000 7fe29fc27640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:45.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.446+0000 7fe29fc27640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:45.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.446+0000 7fe29fc27640 1 -- 172.21.15.132:0/3600235640 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7fe280005b50 con 0x7fe2981365f0 2024-01-24T23:24:45.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.446+0000 7fe28e7fc640 1 -- 172.21.15.132:0/3600235640 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v107) v1 ==== 56+0+1449 (unknown 1453001693 0 245207977) 0x7fe290012d70 con 0x7fe2981365f0 2024-01-24T23:24:45.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.446+0000 7fe28e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:45.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.446+0000 7fe28e7fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:24:45.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.534+0000 7fe29fc27640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:45.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.534+0000 7fe29fc27640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:45.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.534+0000 7fe29fc27640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:45.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.534+0000 7fe29fc27640 1 -- 172.21.15.132:0/3600235640 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7fe280005ce0 con 0x7fe2981365f0 2024-01-24T23:24:45.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.534+0000 7fe28e7fc640 1 -- 172.21.15.132:0/3600235640 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v107) v1 ==== 56+0+1449 (unknown 1453001693 0 245207977) 0x7fe2900144f0 con 0x7fe2981365f0 2024-01-24T23:24:45.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.534+0000 7fe28e7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:45.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.534+0000 7fe28e7fc640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:24:45.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.534+0000 7fe29fc27640 1 -- 172.21.15.132:0/3600235640 >> v1:172.21.15.132:6812/1835401922 conn(0x7fe27c036f90 legacy=0x7fe27c039450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:45.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.534+0000 7fe29fc27640 10 monclient: shutdown 2024-01-24T23:24:45.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.534+0000 7fe29fc27640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:45.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.534+0000 7fe29fc27640 1 -- 172.21.15.132:0/3600235640 >> v1:172.21.15.132:6792/0 conn(0x7fe2981365f0 legacy=0x7fe2981d3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:45.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.534+0000 7fe29fc27640 1 -- 172.21.15.132:0/3600235640 shutdown_connections 2024-01-24T23:24:45.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.534+0000 7fe29fc27640 1 -- 172.21.15.132:0/3600235640 >> 172.21.15.132:0/3600235640 conn(0x7fe2980a6790 msgr2=0x7fe2981bb850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:45.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.534+0000 7fe29fc27640 1 -- 172.21.15.132:0/3600235640 shutdown_connections 2024-01-24T23:24:45.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.534+0000 7fe29fc27640 1 -- 172.21.15.132:0/3600235640 wait complete. 2024-01-24T23:24:45.551 INFO:tasks.workunit.client.0.smithi132.stdout: 12 123.00000 osd.12 down 1.00000 1.00000 2024-01-24T23:24:45.552 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd tree 2024-01-24T23:24:45.552 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep osd.14 2024-01-24T23:24:45.552 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep 123 2024-01-24T23:24:45.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fee071c4640 10 monclient: get_monmap_and_config 2024-01-24T23:24:45.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fee071c4640 10 monclient: build_initial_monmap 2024-01-24T23:24:45.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fee071c4640 10 monclient: monmap: 2024-01-24T23:24:45.661 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:45.661 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:45.661 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:45.662489+0000 2024-01-24T23:24:45.661 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:45.662489+0000 2024-01-24T23:24:45.661 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:45.661 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:45.661 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:45.661 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:45.662 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:45.662 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:45.662 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:45.662 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:45.662 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:45.662 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:45.662 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:45.662 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:45.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fee071c4640 1 Processor -- start 2024-01-24T23:24:45.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fee071c4640 1 -- start start 2024-01-24T23:24:45.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fee071c4640 10 monclient: init 2024-01-24T23:24:45.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fee071c4640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:45.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fee071c4640 10 monclient: _add_conns ranks=[8,6,7,3,5,2,0,1,4] 2024-01-24T23:24:45.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fee071c4640 10 monclient(hunting): picked mon.noname-e con 0x7fee00136cb0 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:45.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fee071c4640 10 monclient(hunting): picked mon.noname-d con 0x7fee001377a0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:45.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fee071c4640 10 monclient(hunting): picked mon.noname-i con 0x7fee0009bcc0 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:45.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fee071c4640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee00064700 con 0x7fee001377a0 2024-01-24T23:24:45.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fee071c4640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee0009b650 con 0x7fee0009bcc0 2024-01-24T23:24:45.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fee071c4640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee0009c110 con 0x7fee00136cb0 2024-01-24T23:24:45.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fee071c4640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:45.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fee071c4640 10 monclient(hunting): authenticate will time out at 1172.987074s 2024-01-24T23:24:45.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fee05761640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fee0009bcc0 0x7fee0013f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:48648/0 (socket says 172.21.15.132:48648) 2024-01-24T23:24:45.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fee05761640 1 -- 172.21.15.132:0/822576270 learned_addr learned my addr 172.21.15.132:0/822576270 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:45.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fedff7fe640 1 -- 172.21.15.132:0/822576270 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3375382809 0 0) 0x7fee0009b650 con 0x7fee0009bcc0 2024-01-24T23:24:45.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fedff7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:24:45.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fedff7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:45.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fedff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:45.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fedff7fe640 10 monclient(hunting): my global_id is 5659 2024-01-24T23:24:45.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fedff7fe640 1 -- 172.21.15.132:0/822576270 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fede8003530 con 0x7fee0009bcc0 2024-01-24T23:24:45.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fedff7fe640 1 -- 172.21.15.132:0/822576270 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3825831140 0 0) 0x7fede8003530 con 0x7fee0009bcc0 2024-01-24T23:24:45.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fedff7fe640 1 -- 172.21.15.132:0/822576270 >> v1:172.21.15.115:6793/0 conn(0x7fee00136cb0 legacy=0x7fee00137090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:45.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fedff7fe640 1 -- 172.21.15.132:0/822576270 >> v1:172.21.15.115:6792/0 conn(0x7fee001377a0 legacy=0x7fee0013c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:45.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fedff7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:24:45.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fedff7fe640 1 monclient: found mon.noname-i 2024-01-24T23:24:45.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fedff7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:45.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fedff7fe640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:24:45.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fedff7fe640 1 -- 172.21.15.132:0/822576270 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0009b000 con 0x7fee0009bcc0 2024-01-24T23:24:45.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fedff7fe640 10 monclient: _finish_auth 0 2024-01-24T23:24:45.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fedff7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:24:45.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fedff7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:45.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fedff7fe640 1 -- 172.21.15.132:0/822576270 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fedf8001b40 con 0x7fee0009bcc0 2024-01-24T23:24:45.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fedff7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:45.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fedff7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:45.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fedff7fe640 10 monclient: dump: 2024-01-24T23:24:45.671 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:45.672 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:45.672 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:45.672 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:45.672 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:45.672 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:45.672 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:45.672 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:45.672 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:45.672 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:45.672 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:45.672 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:45.672 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:45.672 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:45.673 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:45.673 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:45.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fee071c4640 5 monclient: authenticate success, global_id 5659 2024-01-24T23:24:45.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.658+0000 7fee071c4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:45.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedff7fe640 1 -- 172.21.15.132:0/822576270 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fedf8001e60 con 0x7fee0009bcc0 2024-01-24T23:24:45.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedff7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:45.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedff7fe640 1 -- 172.21.15.132:0/822576270 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fedf8004f70 con 0x7fee0009bcc0 2024-01-24T23:24:45.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:45.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 4 set_mon_vals no callback set 2024-01-24T23:24:45.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedff7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:45.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedff7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:45.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedff7fe640 10 monclient: dump: 2024-01-24T23:24:45.673 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:45.673 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:45.673 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:45.674 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:45.674 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:45.674 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:45.674 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:45.674 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:45.674 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:45.674 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:45.674 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:45.674 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:45.674 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:45.674 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:45.674 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:45.674 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:45.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 10 monclient: shutdown 2024-01-24T23:24:45.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:45.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 1 -- 172.21.15.132:0/822576270 >> v1:172.21.15.132:6792/0 conn(0x7fee0009bcc0 legacy=0x7fee0013f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:45.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 1 -- 172.21.15.132:0/822576270 shutdown_connections 2024-01-24T23:24:45.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 1 -- 172.21.15.132:0/822576270 >> 172.21.15.132:0/822576270 conn(0x7fee000a9c10 msgr2=0x7fee000a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:45.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 1 -- 172.21.15.132:0/822576270 shutdown_connections 2024-01-24T23:24:45.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 1 -- 172.21.15.132:0/822576270 wait complete. 2024-01-24T23:24:45.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 10 monclient: build_initial_monmap 2024-01-24T23:24:45.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 10 monclient: monmap: 2024-01-24T23:24:45.675 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:45.675 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:45.675 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:45.675 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:45.675 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:45.675 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:45.676 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:45.676 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:45.676 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:45.676 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:45.676 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:45.676 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:45.676 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:45.676 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:45.676 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:45.676 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:45.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 1 Processor -- start 2024-01-24T23:24:45.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 1 -- start start 2024-01-24T23:24:45.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 10 monclient: init 2024-01-24T23:24:45.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:45.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 10 monclient: _add_conns ranks=[7,6,5,8,2,4,1,0,3] 2024-01-24T23:24:45.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 10 monclient(hunting): picked mon.noname-i con 0x7fee0009bcc0 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:45.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 10 monclient(hunting): picked mon.noname-d con 0x7fee00136cb0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:45.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 10 monclient(hunting): picked mon.noname-h con 0x7fee001377a0 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:45.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee00121ec0 con 0x7fee001377a0 2024-01-24T23:24:45.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee00121800 con 0x7fee00136cb0 2024-01-24T23:24:45.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee001cbb60 con 0x7fee0009bcc0 2024-01-24T23:24:45.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:45.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 10 monclient(hunting): authenticate will time out at 1172.990094s 2024-01-24T23:24:45.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee04f60640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fee0009bcc0 0x7fee00135ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:48660/0 (socket says 172.21.15.132:48660) 2024-01-24T23:24:45.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee04f60640 1 -- 172.21.15.132:0/3435933474 learned_addr learned my addr 172.21.15.132:0/3435933474 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:45.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 1 -- 172.21.15.132:0/3435933474 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3432206821 0 0) 0x7fee001cbb60 con 0x7fee0009bcc0 2024-01-24T23:24:45.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:24:45.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:45.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:45.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient(hunting): my global_id is 5668 2024-01-24T23:24:45.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 1 -- 172.21.15.132:0/3435933474 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedec003530 con 0x7fee0009bcc0 2024-01-24T23:24:45.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 1 -- 172.21.15.132:0/3435933474 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3664936387 0 0) 0x7fee00121ec0 con 0x7fee001377a0 2024-01-24T23:24:45.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:24:45.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:45.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:45.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient(hunting): my global_id is 5639 2024-01-24T23:24:45.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 1 -- 172.21.15.132:0/3435933474 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee001cbb60 con 0x7fee001377a0 2024-01-24T23:24:45.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 1 -- 172.21.15.132:0/3435933474 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1522335148 0 0) 0x7fedec003530 con 0x7fee0009bcc0 2024-01-24T23:24:45.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 1 -- 172.21.15.132:0/3435933474 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fee00121ec0 con 0x7fee0009bcc0 2024-01-24T23:24:45.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 1 -- 172.21.15.132:0/3435933474 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fedf0001ae0 con 0x7fee0009bcc0 2024-01-24T23:24:45.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:45.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:45.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient(hunting): dump: 2024-01-24T23:24:45.679 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:45.679 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:45.679 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:45.679 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:45.679 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:45.679 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:45.679 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:45.679 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:45.679 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:45.679 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:45.679 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:45.679 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:45.679 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:45.679 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:45.679 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:45.680 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:45.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:45.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 1 -- 172.21.15.132:0/3435933474 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2904438637 0 0) 0x7fee001cbb60 con 0x7fee001377a0 2024-01-24T23:24:45.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 1 -- 172.21.15.132:0/3435933474 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fedec003530 con 0x7fee001377a0 2024-01-24T23:24:45.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 1 -- 172.21.15.132:0/3435933474 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fedf8001b00 con 0x7fee001377a0 2024-01-24T23:24:45.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:45.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:45.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient(hunting): dump: 2024-01-24T23:24:45.680 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:45.680 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:45.680 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:45.680 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:45.680 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:45.680 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:45.681 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:45.681 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:45.681 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:45.681 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:45.681 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:45.681 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:45.681 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:45.681 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:45.681 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:45.681 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:45.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 1 -- 172.21.15.132:0/3435933474 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3935844282 0 0) 0x7fee00121800 con 0x7fee00136cb0 2024-01-24T23:24:45.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:24:45.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:45.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:45.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient(hunting): my global_id is 5757 2024-01-24T23:24:45.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 1 -- 172.21.15.132:0/3435933474 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee001cbb60 con 0x7fee00136cb0 2024-01-24T23:24:45.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 1 -- 172.21.15.132:0/3435933474 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4249668132 0 0) 0x7fee00121ec0 con 0x7fee0009bcc0 2024-01-24T23:24:45.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 1 -- 172.21.15.132:0/3435933474 >> v1:172.21.15.115:6792/0 conn(0x7fee00136cb0 legacy=0x7fee001d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:45.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 1 -- 172.21.15.132:0/3435933474 >> v1:172.21.15.132:6791/0 conn(0x7fee001377a0 legacy=0x7fee001d3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:45.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient: _finish_hunting 0 2024-01-24T23:24:45.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 1 monclient: found mon.i 2024-01-24T23:24:45.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:45.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:45.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 1 -- 172.21.15.132:0/3435933474 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee00123090 con 0x7fee0009bcc0 2024-01-24T23:24:45.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient: _finish_auth 0 2024-01-24T23:24:45.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:24:45.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:45.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 5 monclient: authenticate success, global_id 5668 2024-01-24T23:24:45.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 10 monclient: _renew_subs 2024-01-24T23:24:45.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:45.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 1 -- 172.21.15.132:0/3435933474 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee00123370 con 0x7fee0009bcc0 2024-01-24T23:24:45.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 10 monclient: _renew_subs 2024-01-24T23:24:45.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:45.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fee071c4640 1 -- 172.21.15.132:0/3435933474 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fee001365d0 con 0x7fee0009bcc0 2024-01-24T23:24:45.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 1 -- 172.21.15.132:0/3435933474 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fedf0003950 con 0x7fee0009bcc0 2024-01-24T23:24:45.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:45.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 1 -- 172.21.15.132:0/3435933474 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fedf0003e80 con 0x7fee0009bcc0 2024-01-24T23:24:45.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:45.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:45.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 10 monclient: dump: 2024-01-24T23:24:45.683 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:45.683 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:45.683 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:45.683 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:45.684 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:45.684 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:45.684 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:45.684 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:45.684 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:45.684 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:45.684 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:45.684 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:45.684 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:45.684 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:45.684 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:45.684 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:45.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfeffd640 4 set_mon_vals no callback set 2024-01-24T23:24:45.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.662+0000 7fedfd7fa640 1 -- 172.21.15.132:0/3435933474 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fedf0003fe0 con 0x7fee0009bcc0 2024-01-24T23:24:45.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.666+0000 7fee071c4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:45.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.666+0000 7fee071c4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:45.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.666+0000 7fee071c4640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:45.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.666+0000 7fee071c4640 1 -- 172.21.15.132:0/3435933474 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee00136820 con 0x7fee0009bcc0 2024-01-24T23:24:45.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.666+0000 7fedfd7fa640 1 -- 172.21.15.132:0/3435933474 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(107..107 src has 1..107) v4 ==== 5789+0+0 (unknown 367258303 0 0) 0x7fedf0043f70 con 0x7fee0009bcc0 2024-01-24T23:24:45.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.666+0000 7fedfffff640 10 monclient: get_auth_request con 0x7fedec037800 auth_method 0 2024-01-24T23:24:45.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.670+0000 7fedfd7fa640 1 -- 172.21.15.132:0/3435933474 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fedf0014a60 con 0x7fee0009bcc0 2024-01-24T23:24:45.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.670+0000 7fedfd7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:45.685 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.670+0000 7fedfd7fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:45.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.810+0000 7fee071c4640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:45.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.810+0000 7fee071c4640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:45.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.810+0000 7fee071c4640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:45.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.810+0000 7fee071c4640 1 -- 172.21.15.132:0/3435933474 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7fee000674e0 con 0x7fee0009bcc0 2024-01-24T23:24:45.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.810+0000 7fedfd7fa640 1 -- 172.21.15.132:0/3435933474 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v107) v1 ==== 56+0+1449 (unknown 1453001693 0 245207977) 0x7fedf0012ed0 con 0x7fee0009bcc0 2024-01-24T23:24:45.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.810+0000 7fedfd7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:45.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.810+0000 7fedfd7fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:24:45.906 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.902+0000 7fee071c4640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:45.906 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.902+0000 7fee071c4640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:45.906 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.902+0000 7fee071c4640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:45.906 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.902+0000 7fee071c4640 1 -- 172.21.15.132:0/3435933474 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7fee0009c620 con 0x7fee0009bcc0 2024-01-24T23:24:45.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.902+0000 7fedfd7fa640 1 -- 172.21.15.132:0/3435933474 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v107) v1 ==== 56+0+1449 (unknown 1453001693 0 245207977) 0x7fedf0014650 con 0x7fee0009bcc0 2024-01-24T23:24:45.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.902+0000 7fedfd7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:45.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.902+0000 7fedfd7fa640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:24:45.909 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.906+0000 7fee071c4640 1 -- 172.21.15.132:0/3435933474 >> v1:172.21.15.132:6812/1835401922 conn(0x7fedec037800 legacy=0x7fedec039cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:45.909 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.906+0000 7fee071c4640 10 monclient: shutdown 2024-01-24T23:24:45.909 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.906+0000 7fee071c4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:45.909 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.906+0000 7fee071c4640 1 -- 172.21.15.132:0/3435933474 >> v1:172.21.15.132:6792/0 conn(0x7fee0009bcc0 legacy=0x7fee00135ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:45.909 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.906+0000 7fee071c4640 1 -- 172.21.15.132:0/3435933474 shutdown_connections 2024-01-24T23:24:45.909 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.906+0000 7fee071c4640 1 -- 172.21.15.132:0/3435933474 >> 172.21.15.132:0/3435933474 conn(0x7fee000a9c10 msgr2=0x7fee00066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:45.909 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.906+0000 7fee071c4640 1 -- 172.21.15.132:0/3435933474 shutdown_connections 2024-01-24T23:24:45.909 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:45.906+0000 7fee071c4640 1 -- 172.21.15.132:0/3435933474 wait complete. 2024-01-24T23:24:45.923 INFO:tasks.workunit.client.0.smithi132.stdout: 14 123.00000 osd.14 down 1.00000 1.00000 2024-01-24T23:24:45.924 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush reweight-subtree foobaz 155 2024-01-24T23:24:46.038 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.034+0000 7ff4a3fca640 10 monclient: get_monmap_and_config 2024-01-24T23:24:46.038 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.034+0000 7ff4a3fca640 10 monclient: build_initial_monmap 2024-01-24T23:24:46.038 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.034+0000 7ff4a3fca640 10 monclient: monmap: 2024-01-24T23:24:46.038 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:46.038 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:46.038 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:46.039617+0000 2024-01-24T23:24:46.038 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:46.039617+0000 2024-01-24T23:24:46.038 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:46.038 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:46.038 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:46.039 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:46.039 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:46.039 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:46.039 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:46.039 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:46.039 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:46.039 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:46.039 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:46.039 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:46.044 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 1 Processor -- start 2024-01-24T23:24:46.044 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 1 -- start start 2024-01-24T23:24:46.044 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 10 monclient: init 2024-01-24T23:24:46.044 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:46.044 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 10 monclient: _add_conns ranks=[3,1,2,5,6,7,4,8,0] 2024-01-24T23:24:46.044 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 10 monclient(hunting): picked mon.noname-g con 0x7ff49c0a1b50 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:46.044 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 10 monclient(hunting): picked mon.noname-f con 0x7ff49c0a1f30 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:46.044 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 10 monclient(hunting): picked mon.noname-b con 0x7ff49c13b030 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:46.044 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff49c064700 con 0x7ff49c0a1f30 2024-01-24T23:24:46.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff49c070830 con 0x7ff49c13b030 2024-01-24T23:24:46.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff49c072610 con 0x7ff49c0a1b50 2024-01-24T23:24:46.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:46.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 10 monclient(hunting): authenticate will time out at 1173.364865s 2024-01-24T23:24:46.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a1565640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7ff49c0a1f30 legacy=0x7ff49c0a2370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:46.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a1d66640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7ff49c0a1b50 0x7ff49c071480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:33676/0 (socket says 172.21.15.132:33676) 2024-01-24T23:24:46.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a1d66640 1 -- 172.21.15.132:0/1545870933 learned_addr learned my addr 172.21.15.132:0/1545870933 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:46.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 1 -- 172.21.15.132:0/1545870933 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1941019471 0 0) 0x7ff49c072610 con 0x7ff49c0a1b50 2024-01-24T23:24:46.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 10 monclient(hunting): _negotiate 2024-01-24T23:24:46.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:46.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:46.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 10 monclient(hunting): my global_id is 5673 2024-01-24T23:24:46.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 1 -- 172.21.15.132:0/1545870933 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff488003530 con 0x7ff49c0a1b50 2024-01-24T23:24:46.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 1 -- 172.21.15.132:0/1545870933 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 181659165 0 0) 0x7ff488003530 con 0x7ff49c0a1b50 2024-01-24T23:24:46.046 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 1 -- 172.21.15.132:0/1545870933 >> v1:172.21.15.115:6790/0 conn(0x7ff49c13b030 legacy=0x7ff49c13d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:46.046 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 1 -- 172.21.15.132:0/1545870933 >> v1:172.21.15.132:6789/0 conn(0x7ff49c0a1f30 legacy=0x7ff49c0a2370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:46.046 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 10 monclient: _finish_hunting 0 2024-01-24T23:24:46.046 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 1 monclient: found mon.noname-g 2024-01-24T23:24:46.046 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:46.046 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:24:46.046 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 1 -- 172.21.15.132:0/1545870933 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c070aa0 con 0x7ff49c0a1b50 2024-01-24T23:24:46.046 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 10 monclient: _finish_auth 0 2024-01-24T23:24:46.046 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 10 monclient: _check_auth_tickets 2024-01-24T23:24:46.046 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:46.046 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 1 -- 172.21.15.132:0/1545870933 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff498001800 con 0x7ff49c0a1b50 2024-01-24T23:24:46.046 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 5 monclient: authenticate success, global_id 5673 2024-01-24T23:24:46.046 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:46.047 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:46.047 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:46.047 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 10 monclient: dump: 2024-01-24T23:24:46.047 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:46.047 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:46.047 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:46.047 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:46.047 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:46.047 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:46.047 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:46.047 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:46.047 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:46.048 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:46.048 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:46.048 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:46.048 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:46.048 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:46.048 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:46.048 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:46.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:46.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 1 -- 172.21.15.132:0/1545870933 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff498001a60 con 0x7ff49c0a1b50 2024-01-24T23:24:46.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:46.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:46.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 1 -- 172.21.15.132:0/1545870933 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff498004bf0 con 0x7ff49c0a1b50 2024-01-24T23:24:46.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:46.049 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 4 set_mon_vals no callback set 2024-01-24T23:24:46.049 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:46.049 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a0d64640 10 monclient: dump: 2024-01-24T23:24:46.049 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:46.049 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:46.049 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:46.049 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:46.049 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:46.049 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:46.049 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:46.049 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:46.049 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:46.049 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:46.049 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:46.049 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:46.050 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:46.050 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:46.050 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:46.050 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:46.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 10 monclient: shutdown 2024-01-24T23:24:46.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:46.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 1 -- 172.21.15.132:0/1545870933 >> v1:172.21.15.132:6790/0 conn(0x7ff49c0a1b50 legacy=0x7ff49c071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:46.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 1 -- 172.21.15.132:0/1545870933 shutdown_connections 2024-01-24T23:24:46.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 1 -- 172.21.15.132:0/1545870933 >> 172.21.15.132:0/1545870933 conn(0x7ff49c0a7150 msgr2=0x7ff49c0a7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:46.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 1 -- 172.21.15.132:0/1545870933 shutdown_connections 2024-01-24T23:24:46.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 1 -- 172.21.15.132:0/1545870933 wait complete. 2024-01-24T23:24:46.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 10 monclient: build_initial_monmap 2024-01-24T23:24:46.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 10 monclient: monmap: 2024-01-24T23:24:46.050 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:46.051 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:46.051 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:46.051 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:46.051 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:46.051 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:46.051 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:46.051 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:46.051 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:46.051 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:46.051 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:46.051 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:46.051 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:46.051 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:46.051 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:46.052 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:46.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 1 Processor -- start 2024-01-24T23:24:46.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 1 -- start start 2024-01-24T23:24:46.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.038+0000 7ff4a3fca640 10 monclient: init 2024-01-24T23:24:46.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.042+0000 7ff4a3fca640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:46.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.042+0000 7ff4a3fca640 10 monclient: _add_conns ranks=[6,7,8,0,3,1,2,5,4] 2024-01-24T23:24:46.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.042+0000 7ff4a3fca640 10 monclient(hunting): picked mon.noname-d con 0x7ff49c0a1b50 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:46.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.042+0000 7ff4a3fca640 10 monclient(hunting): picked mon.noname-i con 0x7ff49c0a1f30 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:46.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.042+0000 7ff4a3fca640 10 monclient(hunting): picked mon.noname-e con 0x7ff49c13b030 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:46.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.042+0000 7ff4a3fca640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff49c0a8050 con 0x7ff49c0a1b50 2024-01-24T23:24:46.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.042+0000 7ff4a3fca640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff49c06fd00 con 0x7ff49c0a1f30 2024-01-24T23:24:46.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.042+0000 7ff4a3fca640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff49c09d1f0 con 0x7ff49c13b030 2024-01-24T23:24:46.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.042+0000 7ff4a3fca640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:46.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.042+0000 7ff4a3fca640 10 monclient(hunting): authenticate will time out at 1173.368240s 2024-01-24T23:24:46.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.042+0000 7ff4a1565640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7ff49c0a1f30 0x7ff49c1ce750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:48666/0 (socket says 172.21.15.132:48666) 2024-01-24T23:24:46.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.042+0000 7ff4a1565640 1 -- 172.21.15.132:0/1815371878 learned_addr learned my addr 172.21.15.132:0/1815371878 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:46.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 1 -- 172.21.15.132:0/1815371878 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2931241155 0 0) 0x7ff49c06fd00 con 0x7ff49c0a1f30 2024-01-24T23:24:46.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:46.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:46.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:46.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 10 monclient(hunting): my global_id is 5677 2024-01-24T23:24:46.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 1 -- 172.21.15.132:0/1815371878 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff480003530 con 0x7ff49c0a1f30 2024-01-24T23:24:46.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 1 -- 172.21.15.132:0/1815371878 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1904238397 0 0) 0x7ff480003530 con 0x7ff49c0a1f30 2024-01-24T23:24:46.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 1 -- 172.21.15.132:0/1815371878 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff49c06fd00 con 0x7ff49c0a1f30 2024-01-24T23:24:46.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 1 -- 172.21.15.132:0/1815371878 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff48c004300 con 0x7ff49c0a1f30 2024-01-24T23:24:46.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:46.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:46.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 10 monclient(hunting): dump: 2024-01-24T23:24:46.056 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:46.056 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:46.056 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:46.056 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:46.056 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:46.056 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:46.056 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:46.056 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:46.056 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:46.057 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:46.057 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:46.057 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:46.057 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:46.057 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:46.057 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:46.057 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:46.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:46.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 1 -- 172.21.15.132:0/1815371878 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3222212482 0 0) 0x7ff49c06fd00 con 0x7ff49c0a1f30 2024-01-24T23:24:46.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 1 -- 172.21.15.132:0/1815371878 >> v1:172.21.15.115:6793/0 conn(0x7ff49c13b030 legacy=0x7ff49c1d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:46.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 1 -- 172.21.15.132:0/1815371878 >> v1:172.21.15.115:6792/0 conn(0x7ff49c0a1b50 legacy=0x7ff49c1c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:46.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 10 monclient: _finish_hunting 0 2024-01-24T23:24:46.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 1 monclient: found mon.i 2024-01-24T23:24:46.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:46.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:46.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 1 -- 172.21.15.132:0/1815371878 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c137310 con 0x7ff49c0a1f30 2024-01-24T23:24:46.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 10 monclient: _finish_auth 0 2024-01-24T23:24:46.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 10 monclient: _check_auth_tickets 2024-01-24T23:24:46.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:46.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4a3fca640 5 monclient: authenticate success, global_id 5677 2024-01-24T23:24:46.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4a3fca640 10 monclient: _renew_subs 2024-01-24T23:24:46.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4a3fca640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:46.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4a3fca640 1 -- 172.21.15.132:0/1815371878 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49c137fc0 con 0x7ff49c0a1f30 2024-01-24T23:24:46.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4a3fca640 10 monclient: _renew_subs 2024-01-24T23:24:46.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4a3fca640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:46.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4a3fca640 1 -- 172.21.15.132:0/1815371878 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49c1375f0 con 0x7ff49c0a1f30 2024-01-24T23:24:46.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 1 -- 172.21.15.132:0/1815371878 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff48c004520 con 0x7ff49c0a1f30 2024-01-24T23:24:46.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:46.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 1 -- 172.21.15.132:0/1815371878 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff48c004a50 con 0x7ff49c0a1f30 2024-01-24T23:24:46.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:46.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4a3fca640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:46.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:46.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 10 monclient: dump: 2024-01-24T23:24:46.059 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:46.059 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:46.059 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:46.059 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:46.059 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:46.060 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:46.060 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:46.060 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:46.060 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:46.060 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:46.060 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:46.060 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:46.060 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:46.060 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:46.060 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:46.060 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:46.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff493fff640 4 set_mon_vals no callback set 2024-01-24T23:24:46.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 1 -- 172.21.15.132:0/1815371878 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7ff48c004bf0 con 0x7ff49c0a1f30 2024-01-24T23:24:46.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4927fc640 1 -- 172.21.15.132:0/1815371878 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(108..108 src has 1..108) v4 ==== 5789+0+0 (unknown 4042137290 0 0) 0x7ff48c043f60 con 0x7ff49c0a1f30 2024-01-24T23:24:46.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4a3fca640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:46.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4a3fca640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:46.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4a3fca640 1 -- 172.21.15.132:0/1815371878 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff49c073400 con 0x7ff49c0a1f30 2024-01-24T23:24:46.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.046+0000 7ff4a1d66640 10 monclient: get_auth_request con 0x7ff480037090 auth_method 0 2024-01-24T23:24:46.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.050+0000 7ff4927fc640 1 -- 172.21.15.132:0/1815371878 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7ff48c0149f0 con 0x7ff49c0a1f30 2024-01-24T23:24:46.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.050+0000 7ff4927fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:46.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.050+0000 7ff4927fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:46.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.210+0000 7ff4a3fca640 10 start_mon_command cmd=[{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}] 2024-01-24T23:24:46.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.210+0000 7ff4a3fca640 10 monclient: _send_command 2 [{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}] 2024-01-24T23:24:46.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.210+0000 7ff4a3fca640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:46.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.210+0000 7ff4a3fca640 1 -- 172.21.15.132:0/1815371878 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0} v 0) v1 -- 0x7ff49c13ad40 con 0x7ff49c0a1f30 2024-01-24T23:24:46.898 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.894+0000 7ff4927fc640 1 -- 172.21.15.132:0/1815371878 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}]=0 reweighted subtree id -10 name 'foobaz' to 155 in crush map v109) v1 ==== 168+0+0 (unknown 4268641894 0 0) 0x7ff48c012e60 con 0x7ff49c0a1f30 2024-01-24T23:24:46.898 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.894+0000 7ff4927fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}] 2024-01-24T23:24:46.898 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.894+0000 7ff4927fc640 10 monclient: _finish_command 2 = system:0 reweighted subtree id -10 name 'foobaz' to 155 in crush map 2024-01-24T23:24:46.998 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.994+0000 7ff4a3fca640 10 start_mon_command cmd=[{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}] 2024-01-24T23:24:46.998 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.994+0000 7ff4a3fca640 10 monclient: _send_command 3 [{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}] 2024-01-24T23:24:46.998 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.994+0000 7ff4a3fca640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:46.998 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:46.994+0000 7ff4a3fca640 1 -- 172.21.15.132:0/1815371878 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0} v 0) v1 -- 0x7ff49c09cf40 con 0x7ff49c0a1f30 2024-01-24T23:24:47.913 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:47.910+0000 7ff4927fc640 1 -- 172.21.15.132:0/1815371878 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}]=0 reweighted subtree id -10 name 'foobaz' to 155 in crush map v110) v1 ==== 168+0+0 (unknown 2495025621 0 0) 0x7ff48c0145e0 con 0x7ff49c0a1f30 2024-01-24T23:24:47.913 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:47.910+0000 7ff4927fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}] 2024-01-24T23:24:47.913 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:47.910+0000 7ff4927fc640 10 monclient: _finish_command 3 = system:0 reweighted subtree id -10 name 'foobaz' to 155 in crush map 2024-01-24T23:24:47.913 INFO:tasks.workunit.client.0.smithi132.stderr:reweighted subtree id -10 name 'foobaz' to 155 in crush map 2024-01-24T23:24:47.916 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:47.914+0000 7ff4a3fca640 1 -- 172.21.15.132:0/1815371878 >> v1:172.21.15.132:6812/1835401922 conn(0x7ff480037090 legacy=0x7ff480039550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:47.916 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:47.914+0000 7ff4a3fca640 10 monclient: shutdown 2024-01-24T23:24:47.916 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:47.914+0000 7ff4a3fca640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:47.916 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:47.914+0000 7ff4a3fca640 1 -- 172.21.15.132:0/1815371878 >> v1:172.21.15.132:6792/0 conn(0x7ff49c0a1f30 legacy=0x7ff49c1ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:47.917 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:47.914+0000 7ff4a3fca640 1 -- 172.21.15.132:0/1815371878 shutdown_connections 2024-01-24T23:24:47.917 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:47.914+0000 7ff4a3fca640 1 -- 172.21.15.132:0/1815371878 >> 172.21.15.132:0/1815371878 conn(0x7ff49c0a7150 msgr2=0x7ff49c0a6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:47.917 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:47.914+0000 7ff4a3fca640 1 -- 172.21.15.132:0/1815371878 shutdown_connections 2024-01-24T23:24:47.917 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:47.914+0000 7ff4a3fca640 1 -- 172.21.15.132:0/1815371878 wait complete. 2024-01-24T23:24:47.933 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd crush reweight-subtree foobaz 123456 2024-01-24T23:24:47.933 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:24:47.933 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush reweight-subtree foobaz 123456 2024-01-24T23:24:48.054 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.050+0000 7f12cad40640 10 monclient: get_monmap_and_config 2024-01-24T23:24:48.054 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.050+0000 7f12cad40640 10 monclient: build_initial_monmap 2024-01-24T23:24:48.054 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.050+0000 7f12cad40640 10 monclient: monmap: 2024-01-24T23:24:48.054 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:48.054 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:48.054 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:48.055651+0000 2024-01-24T23:24:48.055 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:48.055651+0000 2024-01-24T23:24:48.055 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:48.055 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:48.055 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:48.055 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:48.055 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:48.055 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:48.055 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:48.055 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:48.055 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:48.055 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:48.055 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:48.055 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:48.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.050+0000 7f12cad40640 1 Processor -- start 2024-01-24T23:24:48.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.050+0000 7f12cad40640 1 -- start start 2024-01-24T23:24:48.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.050+0000 7f12cad40640 10 monclient: init 2024-01-24T23:24:48.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:48.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 10 monclient: _add_conns ranks=[8,2,5,1,3,7,4,6,0] 2024-01-24T23:24:48.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 10 monclient(hunting): picked mon.noname-e con 0x7f12c4133b50 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:48.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 10 monclient(hunting): picked mon.noname-b con 0x7f12c409ec20 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:48.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 10 monclient(hunting): picked mon.noname-h con 0x7f12c413a1d0 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:48.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c4064700 con 0x7f12c409ec20 2024-01-24T23:24:48.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c409d9f0 con 0x7f12c413a1d0 2024-01-24T23:24:48.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c409dc60 con 0x7f12c4133b50 2024-01-24T23:24:48.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:48.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 10 monclient(hunting): authenticate will time out at 1175.380255s 2024-01-24T23:24:48.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c92dd640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f12c413a1d0 0x7f12c413c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:40532/0 (socket says 172.21.15.132:40532) 2024-01-24T23:24:48.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c92dd640 1 -- 172.21.15.132:0/2265941722 learned_addr learned my addr 172.21.15.132:0/2265941722 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:48.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 1 -- 172.21.15.132:0/2265941722 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119910671 0 0) 0x7f12c409d9f0 con 0x7f12c413a1d0 2024-01-24T23:24:48.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 10 monclient(hunting): _negotiate 2024-01-24T23:24:48.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:48.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:48.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 10 monclient(hunting): my global_id is 5648 2024-01-24T23:24:48.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 1 -- 172.21.15.132:0/2265941722 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12b0003530 con 0x7f12c413a1d0 2024-01-24T23:24:48.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 1 -- 172.21.15.132:0/2265941722 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2495900476 0 0) 0x7f12b0003530 con 0x7f12c413a1d0 2024-01-24T23:24:48.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 1 -- 172.21.15.132:0/2265941722 >> v1:172.21.15.115:6793/0 conn(0x7f12c4133b50 legacy=0x7f12c4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:48.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 1 -- 172.21.15.132:0/2265941722 >> v1:172.21.15.115:6790/0 conn(0x7f12c409ec20 legacy=0x7f12c4138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:48.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 10 monclient: _finish_hunting 0 2024-01-24T23:24:48.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 1 monclient: found mon.noname-h 2024-01-24T23:24:48.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:48.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:24:48.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 1 -- 172.21.15.132:0/2265941722 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c409df40 con 0x7f12c413a1d0 2024-01-24T23:24:48.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 10 monclient: _finish_auth 0 2024-01-24T23:24:48.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 10 monclient: _check_auth_tickets 2024-01-24T23:24:48.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:48.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 1 -- 172.21.15.132:0/2265941722 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f12bc001b40 con 0x7f12c413a1d0 2024-01-24T23:24:48.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 5 monclient: authenticate success, global_id 5648 2024-01-24T23:24:48.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:48.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:48.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:48.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 10 monclient: dump: 2024-01-24T23:24:48.063 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:48.064 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:48.064 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:48.064 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:48.064 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:48.064 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:48.064 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:48.064 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:48.064 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:48.064 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:48.064 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:48.064 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:48.064 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:48.064 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:48.064 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:48.064 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:48.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:48.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 1 -- 172.21.15.132:0/2265941722 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f12bc001df0 con 0x7f12c413a1d0 2024-01-24T23:24:48.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:48.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:48.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 1 -- 172.21.15.132:0/2265941722 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f12bc004f70 con 0x7f12c413a1d0 2024-01-24T23:24:48.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 4 set_mon_vals no callback set 2024-01-24T23:24:48.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:48.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:48.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c37fe640 10 monclient: dump: 2024-01-24T23:24:48.065 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:48.065 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:48.065 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:48.065 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:48.065 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:48.065 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:48.065 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:48.065 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:48.066 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:48.066 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:48.066 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:48.066 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:48.066 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:48.066 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:48.066 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:48.066 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:48.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 10 monclient: shutdown 2024-01-24T23:24:48.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:48.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 1 -- 172.21.15.132:0/2265941722 >> v1:172.21.15.132:6791/0 conn(0x7f12c413a1d0 legacy=0x7f12c413c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:48.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 1 -- 172.21.15.132:0/2265941722 shutdown_connections 2024-01-24T23:24:48.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 1 -- 172.21.15.132:0/2265941722 >> 172.21.15.132:0/2265941722 conn(0x7f12c40a5a10 msgr2=0x7f12c40a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:48.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 1 -- 172.21.15.132:0/2265941722 shutdown_connections 2024-01-24T23:24:48.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 1 -- 172.21.15.132:0/2265941722 wait complete. 2024-01-24T23:24:48.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 10 monclient: build_initial_monmap 2024-01-24T23:24:48.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 10 monclient: monmap: 2024-01-24T23:24:48.067 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:48.067 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:48.067 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:48.067 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:48.067 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:48.067 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:48.067 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:48.067 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:48.067 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:48.067 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:48.067 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:48.067 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:48.067 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:48.067 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:48.068 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:48.068 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:48.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 1 Processor -- start 2024-01-24T23:24:48.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 1 -- start start 2024-01-24T23:24:48.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 10 monclient: init 2024-01-24T23:24:48.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:48.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 10 monclient: _add_conns ranks=[1,3,4,5,2,7,6,0,8] 2024-01-24T23:24:48.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 10 monclient(hunting): picked mon.noname-f con 0x7f12c409ec20 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:48.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 10 monclient(hunting): picked mon.noname-g con 0x7f12c4133b50 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:48.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 10 monclient(hunting): picked mon.noname-c con 0x7f12c413a1d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:48.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c4133920 con 0x7f12c409ec20 2024-01-24T23:24:48.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c41207a0 con 0x7f12c4133b50 2024-01-24T23:24:48.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c406dc20 con 0x7f12c413a1d0 2024-01-24T23:24:48.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:48.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12cad40640 10 monclient(hunting): authenticate will time out at 1175.383326s 2024-01-24T23:24:48.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c8adc640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f12c409ec20 legacy=0x7f12c406ce00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:48.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c3fff640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f12c4133b50 0x7f12c406d510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:33686/0 (socket says 172.21.15.132:33686) 2024-01-24T23:24:48.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c3fff640 1 -- 172.21.15.132:0/844705138 learned_addr learned my addr 172.21.15.132:0/844705138 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:48.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c92dd640 1 -- 172.21.15.132:0/844705138 >> v1:172.21.15.115:6791/0 conn(0x7f12c413a1d0 legacy=0x7f12c41d39b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:48.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c17fa640 1 -- 172.21.15.132:0/844705138 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3715430949 0 0) 0x7f12c41207a0 con 0x7f12c4133b50 2024-01-24T23:24:48.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c17fa640 10 monclient(hunting): _negotiate 2024-01-24T23:24:48.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c17fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:48.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c17fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:48.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c17fa640 10 monclient(hunting): my global_id is 5682 2024-01-24T23:24:48.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c17fa640 1 -- 172.21.15.132:0/844705138 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12a4003530 con 0x7f12c4133b50 2024-01-24T23:24:48.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c17fa640 1 -- 172.21.15.132:0/844705138 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 775892219 0 0) 0x7f12a4003530 con 0x7f12c4133b50 2024-01-24T23:24:48.069 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c17fa640 1 -- 172.21.15.132:0/844705138 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f12c41207a0 con 0x7f12c4133b50 2024-01-24T23:24:48.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c17fa640 1 -- 172.21.15.132:0/844705138 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f12b4001a60 con 0x7f12c4133b50 2024-01-24T23:24:48.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:48.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c17fa640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:48.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.054+0000 7f12c17fa640 10 monclient(hunting): dump: 2024-01-24T23:24:48.070 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:48.070 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:48.070 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:48.070 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:48.070 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:48.070 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:48.070 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:48.070 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:48.070 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:48.070 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:48.071 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:48.071 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:48.071 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:48.071 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:48.071 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:48.071 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:48.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12c17fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:48.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12c17fa640 1 -- 172.21.15.132:0/844705138 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2748306011 0 0) 0x7f12c41207a0 con 0x7f12c4133b50 2024-01-24T23:24:48.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12c17fa640 1 -- 172.21.15.132:0/844705138 >> v1:172.21.15.115:6791/0 conn(0x7f12c413a1d0 legacy=0x7f12c41d39b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:48.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12c17fa640 1 -- 172.21.15.132:0/844705138 >> v1:172.21.15.132:6789/0 conn(0x7f12c409ec20 legacy=0x7f12c406ce00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:48.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12c17fa640 10 monclient: _finish_hunting 0 2024-01-24T23:24:48.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12c17fa640 1 monclient: found mon.g 2024-01-24T23:24:48.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12c17fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:48.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12c17fa640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:48.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12c17fa640 1 -- 172.21.15.132:0/844705138 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c40a13b0 con 0x7f12c4133b50 2024-01-24T23:24:48.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12c17fa640 10 monclient: _finish_auth 0 2024-01-24T23:24:48.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12c17fa640 10 monclient: _check_auth_tickets 2024-01-24T23:24:48.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12c17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:48.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12cad40640 5 monclient: authenticate success, global_id 5682 2024-01-24T23:24:48.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12cad40640 10 monclient: _renew_subs 2024-01-24T23:24:48.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12cad40640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:48.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12cad40640 1 -- 172.21.15.132:0/844705138 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c406e300 con 0x7f12c4133b50 2024-01-24T23:24:48.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12cad40640 10 monclient: _renew_subs 2024-01-24T23:24:48.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12cad40640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:48.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12cad40640 1 -- 172.21.15.132:0/844705138 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c411fa20 con 0x7f12c4133b50 2024-01-24T23:24:48.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12cad40640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:48.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12cad40640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:48.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12cad40640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:48.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12cad40640 1 -- 172.21.15.132:0/844705138 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c4067640 con 0x7f12c4133b50 2024-01-24T23:24:48.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12c17fa640 1 -- 172.21.15.132:0/844705138 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f12b40038d0 con 0x7f12c4133b50 2024-01-24T23:24:48.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12c17fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:48.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12c17fa640 1 -- 172.21.15.132:0/844705138 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f12b4003e00 con 0x7f12c4133b50 2024-01-24T23:24:48.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12c17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:48.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12c17fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:48.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12c2ffd640 4 set_mon_vals no callback set 2024-01-24T23:24:48.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12c17fa640 10 monclient: dump: 2024-01-24T23:24:48.073 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:48.073 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:48.073 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:48.073 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:48.073 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:48.073 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:48.073 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:48.073 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:48.073 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:48.073 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:48.074 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:48.074 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:48.074 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:48.074 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:48.074 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:48.074 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:48.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12c17fa640 1 -- 172.21.15.132:0/844705138 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f12b4003f60 con 0x7f12c4133b50 2024-01-24T23:24:48.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12c17fa640 1 -- 172.21.15.132:0/844705138 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(110..110 src has 1..110) v4 ==== 5789+0+0 (unknown 1807949586 0 0) 0x7f12b4043f30 con 0x7f12c4133b50 2024-01-24T23:24:48.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.058+0000 7f12c8adc640 10 monclient: get_auth_request con 0x7f12a4036f60 auth_method 0 2024-01-24T23:24:48.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.062+0000 7f12c17fa640 1 -- 172.21.15.132:0/844705138 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f12b4014790 con 0x7f12c4133b50 2024-01-24T23:24:48.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.062+0000 7f12c17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:48.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.062+0000 7f12c17fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:48.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.226+0000 7f12cad40640 10 start_mon_command cmd=[{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 123456.0}] 2024-01-24T23:24:48.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.226+0000 7f12cad40640 10 monclient: _send_command 2 [{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 123456.0}] 2024-01-24T23:24:48.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.226+0000 7f12cad40640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:48.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.226+0000 7f12cad40640 1 -- 172.21.15.132:0/844705138 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 123456.0} v 0) v1 -- 0x7f12c4073840 con 0x7f12c4133b50 2024-01-24T23:24:48.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.226+0000 7f12c17fa640 1 -- 172.21.15.132:0/844705138 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 123456.0}]=-75 (75) Value too large for defined data type v110) v1 ==== 154+0+0 (unknown 2221381043 0 0) 0x7f12b4012e30 con 0x7f12c4133b50 2024-01-24T23:24:48.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.226+0000 7f12c17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 123456.0}] 2024-01-24T23:24:48.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.226+0000 7f12c17fa640 10 monclient: _finish_command 2 = mon:75 (75) Value too large for defined data type 2024-01-24T23:24:48.230 INFO:tasks.workunit.client.0.smithi132.stderr:Error EOVERFLOW: (75) Value too large for defined data type 2024-01-24T23:24:48.235 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:24:48.235 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd tree 2024-01-24T23:24:48.235 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep osd.12 2024-01-24T23:24:48.235 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep 155 2024-01-24T23:24:48.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.346+0000 7f87f3572640 10 monclient: get_monmap_and_config 2024-01-24T23:24:48.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.346+0000 7f87f3572640 10 monclient: build_initial_monmap 2024-01-24T23:24:48.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.346+0000 7f87f3572640 10 monclient: monmap: 2024-01-24T23:24:48.351 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:48.351 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:48.351 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:48.352443+0000 2024-01-24T23:24:48.351 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:48.352443+0000 2024-01-24T23:24:48.351 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:48.351 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:48.351 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:48.352 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:48.352 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:48.352 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:48.352 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:48.352 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:48.352 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:48.352 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:48.352 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:48.352 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:48.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f3572640 1 Processor -- start 2024-01-24T23:24:48.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f3572640 1 -- start start 2024-01-24T23:24:48.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f3572640 10 monclient: init 2024-01-24T23:24:48.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f3572640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:48.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f3572640 10 monclient: _add_conns ranks=[8,6,1,4,7,3,2,0,5] 2024-01-24T23:24:48.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f3572640 10 monclient(hunting): picked mon.noname-e con 0x7f87ec133b50 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:48.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f3572640 10 monclient(hunting): picked mon.noname-d con 0x7f87ec09ec20 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:48.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f3572640 10 monclient(hunting): picked mon.noname-f con 0x7f87ec13a1d0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:48.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f3572640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87ec064700 con 0x7f87ec13a1d0 2024-01-24T23:24:48.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f3572640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87ec09d9f0 con 0x7f87ec09ec20 2024-01-24T23:24:48.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f3572640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87ec09dc60 con 0x7f87ec133b50 2024-01-24T23:24:48.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f3572640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:48.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f3572640 10 monclient(hunting): authenticate will time out at 1175.677216s 2024-01-24T23:24:48.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f1b0f640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f87ec13a1d0 legacy=0x7f87ec13c5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:48.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f130e640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f87ec133b50 0x7f87ec133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:34684/0 (socket says 172.21.15.132:34684) 2024-01-24T23:24:48.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f0b0d640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f87ec09ec20 0x7f87ec138ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:33178/0 (socket says 172.21.15.132:33178) 2024-01-24T23:24:48.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f130e640 1 -- 172.21.15.132:0/2524924296 learned_addr learned my addr 172.21.15.132:0/2524924296 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:48.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 1 -- 172.21.15.132:0/2524924296 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2936704505 0 0) 0x7f87ec09dc60 con 0x7f87ec133b50 2024-01-24T23:24:48.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:48.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:48.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:48.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 10 monclient(hunting): my global_id is 84887 2024-01-24T23:24:48.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 1 -- 172.21.15.132:0/2524924296 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87d8003530 con 0x7f87ec133b50 2024-01-24T23:24:48.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 1 -- 172.21.15.132:0/2524924296 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214908769 0 0) 0x7f87ec09d9f0 con 0x7f87ec09ec20 2024-01-24T23:24:48.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:48.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:48.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:48.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 10 monclient(hunting): my global_id is 5766 2024-01-24T23:24:48.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 1 -- 172.21.15.132:0/2524924296 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87ec09dc60 con 0x7f87ec09ec20 2024-01-24T23:24:48.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 1 -- 172.21.15.132:0/2524924296 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3649922224 0 0) 0x7f87d8003530 con 0x7f87ec133b50 2024-01-24T23:24:48.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 1 -- 172.21.15.132:0/2524924296 >> v1:172.21.15.115:6792/0 conn(0x7f87ec09ec20 legacy=0x7f87ec138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:48.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 1 -- 172.21.15.132:0/2524924296 >> v1:172.21.15.132:6789/0 conn(0x7f87ec13a1d0 legacy=0x7f87ec13c5c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:48.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 10 monclient: _finish_hunting 0 2024-01-24T23:24:48.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 1 monclient: found mon.noname-e 2024-01-24T23:24:48.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:48.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:24:48.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 1 -- 172.21.15.132:0/2524924296 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec09df40 con 0x7f87ec133b50 2024-01-24T23:24:48.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 10 monclient: _finish_auth 0 2024-01-24T23:24:48.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 10 monclient: _check_auth_tickets 2024-01-24T23:24:48.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:48.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 1 -- 172.21.15.132:0/2524924296 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f87e40017c0 con 0x7f87ec133b50 2024-01-24T23:24:48.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:48.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:24:48.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 10 monclient: dump: 2024-01-24T23:24:48.363 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:48.363 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:48.363 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:48.363 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:48.363 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:48.363 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:48.363 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:48.364 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:48.364 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:48.364 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:48.364 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:48.364 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:48.364 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:48.364 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:48.364 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:48.364 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:48.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f3572640 5 monclient: authenticate success, global_id 84887 2024-01-24T23:24:48.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f3572640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:48.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 1 -- 172.21.15.132:0/2524924296 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f87e4001a20 con 0x7f87ec133b50 2024-01-24T23:24:48.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:48.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 1 -- 172.21.15.132:0/2524924296 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f87e4004bf0 con 0x7f87ec133b50 2024-01-24T23:24:48.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f3572640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:48.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f3572640 4 set_mon_vals no callback set 2024-01-24T23:24:48.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:48.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:24:48.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87e3fff640 10 monclient: dump: 2024-01-24T23:24:48.365 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:48.365 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:48.365 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:48.365 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:48.365 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:48.365 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:48.365 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:48.365 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:48.365 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:48.365 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:48.365 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:48.365 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:48.365 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:48.365 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:48.366 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:48.366 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:48.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f3572640 10 monclient: shutdown 2024-01-24T23:24:48.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f3572640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:48.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f3572640 1 -- 172.21.15.132:0/2524924296 >> v1:172.21.15.115:6793/0 conn(0x7f87ec133b50 legacy=0x7f87ec133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:48.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f3572640 1 -- 172.21.15.132:0/2524924296 shutdown_connections 2024-01-24T23:24:48.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f3572640 1 -- 172.21.15.132:0/2524924296 >> 172.21.15.132:0/2524924296 conn(0x7f87ec0a5a10 msgr2=0x7f87ec0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:48.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f3572640 1 -- 172.21.15.132:0/2524924296 shutdown_connections 2024-01-24T23:24:48.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.350+0000 7f87f3572640 1 -- 172.21.15.132:0/2524924296 wait complete. 2024-01-24T23:24:48.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 10 monclient: build_initial_monmap 2024-01-24T23:24:48.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 10 monclient: monmap: 2024-01-24T23:24:48.366 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:48.366 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:48.366 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:48.366 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:48.367 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:48.367 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:48.367 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:48.367 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:48.367 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:48.367 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:48.367 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:48.367 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:48.367 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:48.367 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:48.367 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:48.367 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:48.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 1 Processor -- start 2024-01-24T23:24:48.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 1 -- start start 2024-01-24T23:24:48.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 10 monclient: init 2024-01-24T23:24:48.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:48.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 10 monclient: _add_conns ranks=[6,8,3,5,2,0,1,7,4] 2024-01-24T23:24:48.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 10 monclient(hunting): picked mon.noname-d con 0x7f87ec09ec20 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:48.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 10 monclient(hunting): picked mon.noname-e con 0x7f87ec133b50 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:48.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 10 monclient(hunting): picked mon.noname-g con 0x7f87ec13a1d0 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:48.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87ec11d040 con 0x7f87ec13a1d0 2024-01-24T23:24:48.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87ec11e340 con 0x7f87ec09ec20 2024-01-24T23:24:48.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87ec133920 con 0x7f87ec133b50 2024-01-24T23:24:48.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:48.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 10 monclient(hunting): authenticate will time out at 1175.681508s 2024-01-24T23:24:48.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f1b0f640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f87ec13a1d0 0x7f87ec1d13b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:33690/0 (socket says 172.21.15.132:33690) 2024-01-24T23:24:48.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f1b0f640 1 -- 172.21.15.132:0/3560920468 learned_addr learned my addr 172.21.15.132:0/3560920468 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:48.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 1 -- 172.21.15.132:0/3560920468 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 632635186 0 0) 0x7f87ec11d040 con 0x7f87ec13a1d0 2024-01-24T23:24:48.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:48.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:48.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:48.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 10 monclient(hunting): my global_id is 5691 2024-01-24T23:24:48.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 1 -- 172.21.15.132:0/3560920468 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87d0003530 con 0x7f87ec13a1d0 2024-01-24T23:24:48.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 1 -- 172.21.15.132:0/3560920468 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4038892850 0 0) 0x7f87d0003530 con 0x7f87ec13a1d0 2024-01-24T23:24:48.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 1 -- 172.21.15.132:0/3560920468 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f87ec11d040 con 0x7f87ec13a1d0 2024-01-24T23:24:48.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 1 -- 172.21.15.132:0/3560920468 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f87e8004560 con 0x7f87ec13a1d0 2024-01-24T23:24:48.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:48.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:48.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 10 monclient(hunting): dump: 2024-01-24T23:24:48.369 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:48.369 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:48.369 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:48.369 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:48.369 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:48.369 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:48.369 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:48.370 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:48.370 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:48.370 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:48.370 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:48.370 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:48.370 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:48.370 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:48.370 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:48.370 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:48.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:48.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 1 -- 172.21.15.132:0/3560920468 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2634290316 0 0) 0x7f87ec133920 con 0x7f87ec133b50 2024-01-24T23:24:48.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:48.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:48.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:48.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 10 monclient(hunting): my global_id is 84896 2024-01-24T23:24:48.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 1 -- 172.21.15.132:0/3560920468 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87d0003530 con 0x7f87ec133b50 2024-01-24T23:24:48.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 1 -- 172.21.15.132:0/3560920468 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3111293892 0 0) 0x7f87ec11e340 con 0x7f87ec09ec20 2024-01-24T23:24:48.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:48.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:48.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:48.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 10 monclient(hunting): my global_id is 5775 2024-01-24T23:24:48.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 1 -- 172.21.15.132:0/3560920468 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87ec133920 con 0x7f87ec09ec20 2024-01-24T23:24:48.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 1 -- 172.21.15.132:0/3560920468 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 547394668 0 0) 0x7f87ec11d040 con 0x7f87ec13a1d0 2024-01-24T23:24:48.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 1 -- 172.21.15.132:0/3560920468 >> v1:172.21.15.115:6793/0 conn(0x7f87ec133b50 legacy=0x7f87ec11bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:48.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 1 -- 172.21.15.132:0/3560920468 >> v1:172.21.15.115:6792/0 conn(0x7f87ec09ec20 legacy=0x7f87ec11b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:48.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:24:48.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 1 monclient: found mon.g 2024-01-24T23:24:48.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:48.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:48.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 1 -- 172.21.15.132:0/3560920468 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec0a13b0 con 0x7f87ec13a1d0 2024-01-24T23:24:48.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 10 monclient: _finish_auth 0 2024-01-24T23:24:48.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:24:48.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:48.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 5 monclient: authenticate success, global_id 5691 2024-01-24T23:24:48.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 10 monclient: _renew_subs 2024-01-24T23:24:48.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:48.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 1 -- 172.21.15.132:0/3560920468 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ec11e900 con 0x7f87ec13a1d0 2024-01-24T23:24:48.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 10 monclient: _renew_subs 2024-01-24T23:24:48.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:48.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 1 -- 172.21.15.132:0/3560920468 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ec11df90 con 0x7f87ec13a1d0 2024-01-24T23:24:48.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 1 -- 172.21.15.132:0/3560920468 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f87e8004880 con 0x7f87ec13a1d0 2024-01-24T23:24:48.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:48.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 1 -- 172.21.15.132:0/3560920468 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f87e8004db0 con 0x7f87ec13a1d0 2024-01-24T23:24:48.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:48.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e37fe640 4 set_mon_vals no callback set 2024-01-24T23:24:48.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:48.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 10 monclient: dump: 2024-01-24T23:24:48.373 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:48.373 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:48.373 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:48.373 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:48.373 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:48.373 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:48.373 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:48.373 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:48.373 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:48.373 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:48.374 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:48.374 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:48.374 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:48.374 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:48.374 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:48.374 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:48.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:48.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:48.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:48.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f3572640 1 -- 172.21.15.132:0/3560920468 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87cc005090 con 0x7f87ec13a1d0 2024-01-24T23:24:48.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 1 -- 172.21.15.132:0/3560920468 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f87e8004f10 con 0x7f87ec13a1d0 2024-01-24T23:24:48.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87e1ffb640 1 -- 172.21.15.132:0/3560920468 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(110..110 src has 1..110) v4 ==== 5789+0+0 (unknown 1807949586 0 0) 0x7f87e8044230 con 0x7f87ec13a1d0 2024-01-24T23:24:48.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.354+0000 7f87f130e640 10 monclient: get_auth_request con 0x7f87d0037310 auth_method 0 2024-01-24T23:24:48.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.362+0000 7f87e1ffb640 1 -- 172.21.15.132:0/3560920468 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f87e8014d90 con 0x7f87ec13a1d0 2024-01-24T23:24:48.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.362+0000 7f87e1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:48.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.362+0000 7f87e1ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:48.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.514+0000 7f87f3572640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:48.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.514+0000 7f87f3572640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:48.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.514+0000 7f87f3572640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:48.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.514+0000 7f87f3572640 1 -- 172.21.15.132:0/3560920468 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f87cc005b50 con 0x7f87ec13a1d0 2024-01-24T23:24:48.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.518+0000 7f87e1ffb640 1 -- 172.21.15.132:0/3560920468 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v110) v1 ==== 56+0+1449 (unknown 3122809895 0 2039001018) 0x7f87e801d220 con 0x7f87ec13a1d0 2024-01-24T23:24:48.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.518+0000 7f87e1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:48.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.518+0000 7f87e1ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:24:48.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.610+0000 7f87f3572640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:48.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.610+0000 7f87f3572640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:48.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.610+0000 7f87f3572640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:48.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.610+0000 7f87f3572640 1 -- 172.21.15.132:0/3560920468 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f87cc005ce0 con 0x7f87ec13a1d0 2024-01-24T23:24:48.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.610+0000 7f87e1ffb640 1 -- 172.21.15.132:0/3560920468 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v110) v1 ==== 56+0+1449 (unknown 3122809895 0 2039001018) 0x7f87e8013200 con 0x7f87ec13a1d0 2024-01-24T23:24:48.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.610+0000 7f87e1ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:48.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.610+0000 7f87e1ffb640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:24:48.615 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.610+0000 7f87f3572640 1 -- 172.21.15.132:0/3560920468 >> v1:172.21.15.132:6812/1835401922 conn(0x7f87d0037310 legacy=0x7f87d00397d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:48.615 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.610+0000 7f87f3572640 10 monclient: shutdown 2024-01-24T23:24:48.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.610+0000 7f87f3572640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:48.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.610+0000 7f87f3572640 1 -- 172.21.15.132:0/3560920468 >> v1:172.21.15.132:6790/0 conn(0x7f87ec13a1d0 legacy=0x7f87ec1d13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:48.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.614+0000 7f87f3572640 1 -- 172.21.15.132:0/3560920468 shutdown_connections 2024-01-24T23:24:48.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.614+0000 7f87f3572640 1 -- 172.21.15.132:0/3560920468 >> 172.21.15.132:0/3560920468 conn(0x7f87ec0a5a10 msgr2=0x7f87ec09cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:48.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.614+0000 7f87f3572640 1 -- 172.21.15.132:0/3560920468 shutdown_connections 2024-01-24T23:24:48.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.614+0000 7f87f3572640 1 -- 172.21.15.132:0/3560920468 wait complete. 2024-01-24T23:24:48.630 INFO:tasks.workunit.client.0.smithi132.stdout: 12 155.00000 osd.12 down 1.00000 1.00000 2024-01-24T23:24:48.631 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd tree 2024-01-24T23:24:48.631 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep osd.14 2024-01-24T23:24:48.631 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep 155 2024-01-24T23:24:48.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.742+0000 7f6c59434640 10 monclient: get_monmap_and_config 2024-01-24T23:24:48.746 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.742+0000 7f6c59434640 10 monclient: build_initial_monmap 2024-01-24T23:24:48.746 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.742+0000 7f6c59434640 10 monclient: monmap: 2024-01-24T23:24:48.746 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:48.746 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:48.746 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:48.747171+0000 2024-01-24T23:24:48.746 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:48.747171+0000 2024-01-24T23:24:48.746 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:48.746 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:48.746 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:48.746 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:48.746 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:48.746 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:48.746 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:48.747 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:48.747 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:48.747 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:48.747 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:48.747 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:48.753 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.742+0000 7f6c59434640 1 Processor -- start 2024-01-24T23:24:48.753 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.742+0000 7f6c59434640 1 -- start start 2024-01-24T23:24:48.753 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.742+0000 7f6c59434640 10 monclient: init 2024-01-24T23:24:48.753 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:48.753 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 10 monclient: _add_conns ranks=[3,7,0,5,8,4,1,2,6] 2024-01-24T23:24:48.753 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 10 monclient(hunting): picked mon.noname-g con 0x7f6c54131ca0 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:48.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 10 monclient(hunting): picked mon.noname-i con 0x7f6c54132080 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:48.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 10 monclient(hunting): picked mon.noname-a con 0x7f6c54132500 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:48.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c54064700 con 0x7f6c54132500 2024-01-24T23:24:48.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c54072680 con 0x7f6c54131ca0 2024-01-24T23:24:48.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c54071060 con 0x7f6c54132080 2024-01-24T23:24:48.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:48.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 10 monclient(hunting): authenticate will time out at 1176.072603s 2024-01-24T23:24:48.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c537fe640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f6c54132500 legacy=0x7f6c5413d4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:48.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c52ffd640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f6c54131ca0 0x7f6c54071480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:33696/0 (socket says 172.21.15.132:33696) 2024-01-24T23:24:48.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c52ffd640 1 -- 172.21.15.132:0/551598247 learned_addr learned my addr 172.21.15.132:0/551598247 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:48.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c527fc640 1 --1- 172.21.15.132:0/551598247 >> v1:172.21.15.132:6792/0 conn(0x7f6c54132080 0x7f6c54139da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:48680/0 (socket says 172.21.15.132:48680) 2024-01-24T23:24:48.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 1 -- 172.21.15.132:0/551598247 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2199475527 0 0) 0x7f6c54071060 con 0x7f6c54132080 2024-01-24T23:24:48.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:48.755 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:48.755 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:48.755 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 10 monclient(hunting): my global_id is 5686 2024-01-24T23:24:48.755 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 1 -- 172.21.15.132:0/551598247 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c3c003530 con 0x7f6c54132080 2024-01-24T23:24:48.755 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 1 -- 172.21.15.132:0/551598247 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4082830624 0 0) 0x7f6c44001120 con 0x7f6c54131ca0 2024-01-24T23:24:48.755 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:48.755 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:48.755 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:48.755 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 10 monclient(hunting): my global_id is 5700 2024-01-24T23:24:48.755 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 1 -- 172.21.15.132:0/551598247 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c54071060 con 0x7f6c54131ca0 2024-01-24T23:24:48.755 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 1 -- 172.21.15.132:0/551598247 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3042229183 0 0) 0x7f6c54071060 con 0x7f6c54131ca0 2024-01-24T23:24:48.755 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 1 -- 172.21.15.132:0/551598247 >> v1:172.21.15.132:6792/0 conn(0x7f6c54132080 legacy=0x7f6c54139da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:48.755 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 1 -- 172.21.15.132:0/551598247 >> v1:172.21.15.115:6789/0 conn(0x7f6c54132500 legacy=0x7f6c5413d4d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:48.755 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:24:48.755 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 1 monclient: found mon.noname-g 2024-01-24T23:24:48.756 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:48.756 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:24:48.756 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 1 -- 172.21.15.132:0/551598247 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c54070aa0 con 0x7f6c54131ca0 2024-01-24T23:24:48.756 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 10 monclient: _finish_auth 0 2024-01-24T23:24:48.756 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:24:48.756 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:48.756 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 5 monclient: authenticate success, global_id 5700 2024-01-24T23:24:48.756 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 1 -- 172.21.15.132:0/551598247 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6c440018b0 con 0x7f6c54131ca0 2024-01-24T23:24:48.756 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:48.756 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:48.756 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 10 monclient: dump: 2024-01-24T23:24:48.756 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:48.756 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:48.757 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:48.757 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:48.757 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:48.757 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:48.757 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:48.757 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:48.757 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:48.757 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:48.757 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:48.757 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:48.757 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:48.757 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:48.757 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:48.757 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:48.757 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:48.758 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 1 -- 172.21.15.132:0/551598247 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6c44001b50 con 0x7f6c54131ca0 2024-01-24T23:24:48.758 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:48.758 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:48.758 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 1 -- 172.21.15.132:0/551598247 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6c44004bf0 con 0x7f6c54131ca0 2024-01-24T23:24:48.758 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 4 set_mon_vals no callback set 2024-01-24T23:24:48.758 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:48.758 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:48.758 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c51ffb640 10 monclient: dump: 2024-01-24T23:24:48.758 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:48.758 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:48.758 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:48.758 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:48.758 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:48.758 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:48.759 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:48.759 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:48.759 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:48.759 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:48.759 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:48.759 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:48.759 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:48.759 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:48.759 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:48.759 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:48.759 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 10 monclient: shutdown 2024-01-24T23:24:48.759 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:48.759 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 1 -- 172.21.15.132:0/551598247 >> v1:172.21.15.132:6790/0 conn(0x7f6c54131ca0 legacy=0x7f6c54071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:48.759 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 1 -- 172.21.15.132:0/551598247 shutdown_connections 2024-01-24T23:24:48.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 1 -- 172.21.15.132:0/551598247 >> 172.21.15.132:0/551598247 conn(0x7f6c540a7150 msgr2=0x7f6c540a7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:48.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 1 -- 172.21.15.132:0/551598247 shutdown_connections 2024-01-24T23:24:48.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 1 -- 172.21.15.132:0/551598247 wait complete. 2024-01-24T23:24:48.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 10 monclient: build_initial_monmap 2024-01-24T23:24:48.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 10 monclient: monmap: 2024-01-24T23:24:48.760 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:48.760 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:48.760 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:48.760 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:48.760 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:48.760 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:48.760 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:48.760 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:48.760 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:48.761 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:48.761 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:48.761 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:48.761 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:48.761 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:48.761 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:48.761 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:48.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 1 Processor -- start 2024-01-24T23:24:48.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 1 -- start start 2024-01-24T23:24:48.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 10 monclient: init 2024-01-24T23:24:48.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:48.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 10 monclient: _add_conns ranks=[1,2,5,0,6,8,4,3,7] 2024-01-24T23:24:48.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 10 monclient(hunting): picked mon.noname-f con 0x7f6c54131ca0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:48.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 10 monclient(hunting): picked mon.noname-b con 0x7f6c54132080 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:48.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 10 monclient(hunting): picked mon.noname-h con 0x7f6c54122350 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:48.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c541238e0 con 0x7f6c54131ca0 2024-01-24T23:24:48.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c5406fa40 con 0x7f6c54132080 2024-01-24T23:24:48.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c541c8680 con 0x7f6c54122350 2024-01-24T23:24:48.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:48.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c59434640 10 monclient(hunting): authenticate will time out at 1176.075678s 2024-01-24T23:24:48.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c52ffd640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f6c54131ca0 legacy=0x7f6c54130ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:48.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c537fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f6c54122350 0x7f6c541d3bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:40548/0 (socket says 172.21.15.132:40548) 2024-01-24T23:24:48.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.746+0000 7f6c537fe640 1 -- 172.21.15.132:0/3415781697 learned_addr learned my addr 172.21.15.132:0/3415781697 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:48.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 1 -- 172.21.15.132:0/3415781697 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1478586691 0 0) 0x7f6c541c8680 con 0x7f6c54122350 2024-01-24T23:24:48.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:24:48.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:48.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:48.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 10 monclient(hunting): my global_id is 5657 2024-01-24T23:24:48.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 1 -- 172.21.15.132:0/3415781697 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c40003530 con 0x7f6c54122350 2024-01-24T23:24:48.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 1 -- 172.21.15.132:0/3415781697 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1287545782 0 0) 0x7f6c40003530 con 0x7f6c54122350 2024-01-24T23:24:48.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 1 -- 172.21.15.132:0/3415781697 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6c541c8680 con 0x7f6c54122350 2024-01-24T23:24:48.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 1 -- 172.21.15.132:0/3415781697 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6c4c004570 con 0x7f6c54122350 2024-01-24T23:24:48.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:48.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:48.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 10 monclient(hunting): dump: 2024-01-24T23:24:48.763 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:48.763 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:48.763 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:48.763 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:48.763 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:48.763 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:48.763 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:48.763 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:48.764 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:48.764 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:48.764 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:48.764 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:48.764 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:48.764 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:48.764 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:48.764 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:48.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:48.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 1 -- 172.21.15.132:0/3415781697 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2617736324 0 0) 0x7f6c5406fa40 con 0x7f6c54132080 2024-01-24T23:24:48.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:24:48.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:48.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:48.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 10 monclient(hunting): my global_id is 84989 2024-01-24T23:24:48.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 1 -- 172.21.15.132:0/3415781697 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c40003530 con 0x7f6c54132080 2024-01-24T23:24:48.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 1 -- 172.21.15.132:0/3415781697 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3585914267 0 0) 0x7f6c541c8680 con 0x7f6c54122350 2024-01-24T23:24:48.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 1 -- 172.21.15.132:0/3415781697 >> v1:172.21.15.115:6790/0 conn(0x7f6c54132080 legacy=0x7f6c541d0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:48.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 1 -- 172.21.15.132:0/3415781697 >> v1:172.21.15.132:6789/0 conn(0x7f6c54131ca0 legacy=0x7f6c54130ed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:48.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:24:48.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 1 monclient: found mon.h 2024-01-24T23:24:48.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:48.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:48.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 1 -- 172.21.15.132:0/3415781697 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5406d220 con 0x7f6c54122350 2024-01-24T23:24:48.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 10 monclient: _finish_auth 0 2024-01-24T23:24:48.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:24:48.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:48.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c59434640 5 monclient: authenticate success, global_id 5657 2024-01-24T23:24:48.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c59434640 10 monclient: _renew_subs 2024-01-24T23:24:48.766 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c59434640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:48.766 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c59434640 1 -- 172.21.15.132:0/3415781697 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c54123040 con 0x7f6c54122350 2024-01-24T23:24:48.766 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c59434640 10 monclient: _renew_subs 2024-01-24T23:24:48.766 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c59434640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:48.766 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c59434640 1 -- 172.21.15.132:0/3415781697 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c541315e0 con 0x7f6c54122350 2024-01-24T23:24:48.766 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 1 -- 172.21.15.132:0/3415781697 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6c4c004890 con 0x7f6c54122350 2024-01-24T23:24:48.766 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:48.766 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 1 -- 172.21.15.132:0/3415781697 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6c4c004dc0 con 0x7f6c54122350 2024-01-24T23:24:48.766 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:48.766 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:48.766 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c517fa640 4 set_mon_vals no callback set 2024-01-24T23:24:48.766 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 10 monclient: dump: 2024-01-24T23:24:48.766 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:48.766 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:48.766 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:48.767 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:48.767 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:48.767 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:48.767 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:48.767 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:48.767 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:48.767 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:48.767 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:48.767 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:48.767 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:48.767 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:48.767 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:48.767 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:48.767 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c59434640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:48.767 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 1 -- 172.21.15.132:0/3415781697 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f6c4c004f20 con 0x7f6c54122350 2024-01-24T23:24:48.767 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c59434640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:48.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c59434640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:48.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c59434640 1 -- 172.21.15.132:0/3415781697 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c34005090 con 0x7f6c54122350 2024-01-24T23:24:48.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c3b7fe640 1 -- 172.21.15.132:0/3415781697 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(110..110 src has 1..110) v4 ==== 5789+0+0 (unknown 1807949586 0 0) 0x7f6c4c044270 con 0x7f6c54122350 2024-01-24T23:24:48.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.750+0000 7f6c52ffd640 10 monclient: get_auth_request con 0x7f6c40036fb0 auth_method 0 2024-01-24T23:24:48.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.754+0000 7f6c3b7fe640 1 -- 172.21.15.132:0/3415781697 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f6c4c014d60 con 0x7f6c54122350 2024-01-24T23:24:48.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.754+0000 7f6c3b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:48.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.754+0000 7f6c3b7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:48.909 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.906+0000 7f6c59434640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:48.910 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.906+0000 7f6c59434640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:48.910 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.906+0000 7f6c59434640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:48.910 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.906+0000 7f6c59434640 1 -- 172.21.15.132:0/3415781697 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f6c34005b50 con 0x7f6c54122350 2024-01-24T23:24:48.914 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.910+0000 7f6c3b7fe640 1 -- 172.21.15.132:0/3415781697 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v110) v1 ==== 56+0+1449 (unknown 3122809895 0 2039001018) 0x7f6c4c0131d0 con 0x7f6c54122350 2024-01-24T23:24:48.914 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.910+0000 7f6c3b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-01-24T23:24:48.914 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:48.910+0000 7f6c3b7fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:24:49.006 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.002+0000 7f6c59434640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-01-24T23:24:49.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.002+0000 7f6c59434640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:49.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.002+0000 7f6c59434640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:49.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.002+0000 7f6c59434640 1 -- 172.21.15.132:0/3415781697 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd tree"} v 0) v1 -- 0x7f6c34005ce0 con 0x7f6c54122350 2024-01-24T23:24:49.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.006+0000 7f6c3b7fe640 1 -- 172.21.15.132:0/3415781697 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v110) v1 ==== 56+0+1449 (unknown 3122809895 0 2039001018) 0x7f6c4c014950 con 0x7f6c54122350 2024-01-24T23:24:49.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.006+0000 7f6c3b7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-01-24T23:24:49.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.006+0000 7f6c3b7fe640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:24:49.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.006+0000 7f6c59434640 1 -- 172.21.15.132:0/3415781697 >> v1:172.21.15.132:6812/1835401922 conn(0x7f6c40036fb0 legacy=0x7f6c40039470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:49.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.006+0000 7f6c59434640 10 monclient: shutdown 2024-01-24T23:24:49.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.006+0000 7f6c59434640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:49.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.006+0000 7f6c59434640 1 -- 172.21.15.132:0/3415781697 >> v1:172.21.15.132:6791/0 conn(0x7f6c54122350 legacy=0x7f6c541d3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:49.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.006+0000 7f6c59434640 1 -- 172.21.15.132:0/3415781697 shutdown_connections 2024-01-24T23:24:49.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.006+0000 7f6c59434640 1 -- 172.21.15.132:0/3415781697 >> 172.21.15.132:0/3415781697 conn(0x7f6c540a7150 msgr2=0x7f6c540a3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:49.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.006+0000 7f6c59434640 1 -- 172.21.15.132:0/3415781697 shutdown_connections 2024-01-24T23:24:49.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.006+0000 7f6c59434640 1 -- 172.21.15.132:0/3415781697 wait complete. 2024-01-24T23:24:49.022 INFO:tasks.workunit.client.0.smithi132.stdout: 14 155.00000 osd.14 down 1.00000 1.00000 2024-01-24T23:24:49.022 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm osd.12 2024-01-24T23:24:49.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.134+0000 7f7e5c38b640 10 monclient: get_monmap_and_config 2024-01-24T23:24:49.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.134+0000 7f7e5c38b640 10 monclient: build_initial_monmap 2024-01-24T23:24:49.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.134+0000 7f7e5c38b640 10 monclient: monmap: 2024-01-24T23:24:49.138 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:49.138 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:49.138 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:49.139443+0000 2024-01-24T23:24:49.138 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:49.139443+0000 2024-01-24T23:24:49.138 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:49.138 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:49.139 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:49.139 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:49.139 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:49.139 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:49.139 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:49.139 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:49.139 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:49.139 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:49.139 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:49.139 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:49.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.134+0000 7f7e5c38b640 1 Processor -- start 2024-01-24T23:24:49.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.134+0000 7f7e5c38b640 1 -- start start 2024-01-24T23:24:49.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.134+0000 7f7e5c38b640 10 monclient: init 2024-01-24T23:24:49.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:49.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 10 monclient: _add_conns ranks=[6,2,4,7,3,5,8,1,0] 2024-01-24T23:24:49.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 10 monclient(hunting): picked mon.noname-d con 0x7f7e5409c4c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:49.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 10 monclient(hunting): picked mon.noname-b con 0x7f7e5409cfb0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:49.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 10 monclient(hunting): picked mon.noname-c con 0x7f7e5409ec50 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:49.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e54064700 con 0x7f7e5409cfb0 2024-01-24T23:24:49.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e5409d9f0 con 0x7f7e5409ec50 2024-01-24T23:24:49.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e5409dc60 con 0x7f7e5409c4c0 2024-01-24T23:24:49.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:49.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 10 monclient(hunting): authenticate will time out at 1176.464279s 2024-01-24T23:24:49.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5a928640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f7e5409ec50 legacy=0x7f7e5413c460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:49.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5a127640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f7e5409c4c0 0x7f7e5409c8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:33198/0 (socket says 172.21.15.132:33198) 2024-01-24T23:24:49.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59926640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f7e5409cfb0 0x7f7e54138d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:59988/0 (socket says 172.21.15.132:59988) 2024-01-24T23:24:49.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5a127640 1 -- 172.21.15.132:0/2108402988 learned_addr learned my addr 172.21.15.132:0/2108402988 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:49.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 1 -- 172.21.15.132:0/2108402988 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 547629338 0 0) 0x7f7e5409dc60 con 0x7f7e5409c4c0 2024-01-24T23:24:49.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 10 monclient(hunting): _negotiate 2024-01-24T23:24:49.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:49.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:49.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 10 monclient(hunting): my global_id is 5784 2024-01-24T23:24:49.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 1 -- 172.21.15.132:0/2108402988 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e40003530 con 0x7f7e5409c4c0 2024-01-24T23:24:49.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 1 -- 172.21.15.132:0/2108402988 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2810779093 0 0) 0x7f7e54064700 con 0x7f7e5409cfb0 2024-01-24T23:24:49.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 10 monclient(hunting): _negotiate 2024-01-24T23:24:49.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:49.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:49.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 10 monclient(hunting): my global_id is 84998 2024-01-24T23:24:49.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 1 -- 172.21.15.132:0/2108402988 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e5409dc60 con 0x7f7e5409cfb0 2024-01-24T23:24:49.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 1 -- 172.21.15.132:0/2108402988 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4248310385 0 0) 0x7f7e40003530 con 0x7f7e5409c4c0 2024-01-24T23:24:49.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 1 -- 172.21.15.132:0/2108402988 >> v1:172.21.15.115:6791/0 conn(0x7f7e5409ec50 legacy=0x7f7e5413c460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:49.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 1 -- 172.21.15.132:0/2108402988 >> v1:172.21.15.115:6790/0 conn(0x7f7e5409cfb0 legacy=0x7f7e54138d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:49.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 10 monclient: _finish_hunting 0 2024-01-24T23:24:49.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 1 monclient: found mon.noname-d 2024-01-24T23:24:49.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:49.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:24:49.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 1 -- 172.21.15.132:0/2108402988 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5409df40 con 0x7f7e5409c4c0 2024-01-24T23:24:49.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 10 monclient: _finish_auth 0 2024-01-24T23:24:49.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 10 monclient: _check_auth_tickets 2024-01-24T23:24:49.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:49.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 1 -- 172.21.15.132:0/2108402988 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e500017c0 con 0x7f7e5409c4c0 2024-01-24T23:24:49.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:49.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:24:49.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 10 monclient: dump: 2024-01-24T23:24:49.150 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:49.150 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:49.150 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:49.150 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:49.150 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:49.150 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:49.150 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:49.150 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:49.150 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:49.151 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:49.151 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:49.151 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:49.151 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:49.151 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:49.151 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:49.151 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:49.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 5 monclient: authenticate success, global_id 5784 2024-01-24T23:24:49.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:49.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 1 -- 172.21.15.132:0/2108402988 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7e50001aa0 con 0x7f7e5409c4c0 2024-01-24T23:24:49.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:49.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 1 -- 172.21.15.132:0/2108402988 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e50004bf0 con 0x7f7e5409c4c0 2024-01-24T23:24:49.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:49.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 4 set_mon_vals no callback set 2024-01-24T23:24:49.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:49.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:24:49.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e59125640 10 monclient: dump: 2024-01-24T23:24:49.152 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:49.152 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:49.152 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:49.152 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:49.152 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:49.152 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:49.152 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:49.152 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:49.152 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:49.152 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:49.152 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:49.152 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:49.152 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:49.152 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:49.152 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:49.153 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:49.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 10 monclient: shutdown 2024-01-24T23:24:49.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:49.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 1 -- 172.21.15.132:0/2108402988 >> v1:172.21.15.115:6792/0 conn(0x7f7e5409c4c0 legacy=0x7f7e5409c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:49.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 1 -- 172.21.15.132:0/2108402988 shutdown_connections 2024-01-24T23:24:49.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 1 -- 172.21.15.132:0/2108402988 >> 172.21.15.132:0/2108402988 conn(0x7f7e540a5a10 msgr2=0x7f7e540a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:49.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 1 -- 172.21.15.132:0/2108402988 shutdown_connections 2024-01-24T23:24:49.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 1 -- 172.21.15.132:0/2108402988 wait complete. 2024-01-24T23:24:49.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 10 monclient: build_initial_monmap 2024-01-24T23:24:49.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 10 monclient: monmap: 2024-01-24T23:24:49.153 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:49.153 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:49.153 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:49.153 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:49.154 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:49.154 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:49.154 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:49.154 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:49.154 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:49.154 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:49.154 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:49.154 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:49.154 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:49.154 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:49.154 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:49.154 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:49.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 1 Processor -- start 2024-01-24T23:24:49.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 1 -- start start 2024-01-24T23:24:49.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 10 monclient: init 2024-01-24T23:24:49.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:49.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 10 monclient: _add_conns ranks=[7,4,8,1,6,3,2,0,5] 2024-01-24T23:24:49.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 10 monclient(hunting): picked mon.noname-i con 0x7f7e5409c4c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:49.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 10 monclient(hunting): picked mon.noname-c con 0x7f7e5409cfb0 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:49.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 10 monclient(hunting): picked mon.noname-e con 0x7f7e5409ec50 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:49.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e5409c020 con 0x7f7e5409cfb0 2024-01-24T23:24:49.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e540a13b0 con 0x7f7e5409c4c0 2024-01-24T23:24:49.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e54134b00 con 0x7f7e5409ec50 2024-01-24T23:24:49.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:49.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.138+0000 7f7e5c38b640 10 monclient(hunting): authenticate will time out at 1176.467972s 2024-01-24T23:24:49.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e5a127640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f7e5409c4c0 0x7f7e54132b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:48690/0 (socket says 172.21.15.132:48690) 2024-01-24T23:24:49.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e5a127640 1 -- 172.21.15.132:0/1188632493 learned_addr learned my addr 172.21.15.132:0/1188632493 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:49.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e59926640 1 -- 172.21.15.132:0/1188632493 >> v1:172.21.15.115:6791/0 conn(0x7f7e5409cfb0 legacy=0x7f7e541ce1f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:49.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 1 -- 172.21.15.132:0/1188632493 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3763087088 0 0) 0x7f7e540a13b0 con 0x7f7e5409c4c0 2024-01-24T23:24:49.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:49.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:49.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:49.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 10 monclient(hunting): my global_id is 5695 2024-01-24T23:24:49.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 1 -- 172.21.15.132:0/1188632493 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e38003560 con 0x7f7e5409c4c0 2024-01-24T23:24:49.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 1 -- 172.21.15.132:0/1188632493 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 220285663 0 0) 0x7f7e38003560 con 0x7f7e5409c4c0 2024-01-24T23:24:49.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 1 -- 172.21.15.132:0/1188632493 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7e540a13b0 con 0x7f7e5409c4c0 2024-01-24T23:24:49.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 1 -- 172.21.15.132:0/1188632493 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e50004b30 con 0x7f7e5409c4c0 2024-01-24T23:24:49.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:49.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:49.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 10 monclient(hunting): dump: 2024-01-24T23:24:49.156 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:49.156 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:49.156 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:49.156 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:49.156 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:49.157 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:49.157 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:49.157 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:49.157 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:49.157 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:49.157 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:49.157 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:49.157 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:49.157 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:49.157 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:49.157 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:49.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:49.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 1 -- 172.21.15.132:0/1188632493 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3242024414 0 0) 0x7f7e54134b00 con 0x7f7e5409ec50 2024-01-24T23:24:49.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:49.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:49.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:49.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 10 monclient(hunting): my global_id is 84905 2024-01-24T23:24:49.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 1 -- 172.21.15.132:0/1188632493 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e38003560 con 0x7f7e5409ec50 2024-01-24T23:24:49.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 1 -- 172.21.15.132:0/1188632493 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 883217331 0 0) 0x7f7e540a13b0 con 0x7f7e5409c4c0 2024-01-24T23:24:49.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 1 -- 172.21.15.132:0/1188632493 >> v1:172.21.15.115:6793/0 conn(0x7f7e5409ec50 legacy=0x7f7e541d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:49.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 1 -- 172.21.15.132:0/1188632493 >> v1:172.21.15.115:6791/0 conn(0x7f7e5409cfb0 legacy=0x7f7e541ce1f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:49.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:49.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 1 monclient: found mon.i 2024-01-24T23:24:49.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:49.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:49.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 1 -- 172.21.15.132:0/1188632493 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e54124320 con 0x7f7e5409c4c0 2024-01-24T23:24:49.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 10 monclient: _finish_auth 0 2024-01-24T23:24:49.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:49.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:49.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e5c38b640 5 monclient: authenticate success, global_id 5695 2024-01-24T23:24:49.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e5c38b640 10 monclient: _renew_subs 2024-01-24T23:24:49.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e5c38b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:49.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e5c38b640 1 -- 172.21.15.132:0/1188632493 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e541224d0 con 0x7f7e5409c4c0 2024-01-24T23:24:49.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e5c38b640 10 monclient: _renew_subs 2024-01-24T23:24:49.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e5c38b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:49.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e5c38b640 1 -- 172.21.15.132:0/1188632493 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e54124040 con 0x7f7e5409c4c0 2024-01-24T23:24:49.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 1 -- 172.21.15.132:0/1188632493 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7e50004ec0 con 0x7f7e5409c4c0 2024-01-24T23:24:49.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:49.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 1 -- 172.21.15.132:0/1188632493 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e50005370 con 0x7f7e5409c4c0 2024-01-24T23:24:49.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:49.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e58924640 4 set_mon_vals no callback set 2024-01-24T23:24:49.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:49.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 10 monclient: dump: 2024-01-24T23:24:49.159 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:49.160 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:49.160 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:49.160 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:49.160 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:49.160 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:49.160 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:49.160 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:49.160 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:49.160 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:49.160 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:49.160 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:49.160 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:49.160 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:49.160 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:49.161 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:49.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e5c38b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:49.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e5c38b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:49.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e5c38b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:49.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e5c38b640 1 -- 172.21.15.132:0/1188632493 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e3c005090 con 0x7f7e5409c4c0 2024-01-24T23:24:49.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 1 -- 172.21.15.132:0/1188632493 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f7e5000fdf0 con 0x7f7e5409c4c0 2024-01-24T23:24:49.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e4affd640 1 -- 172.21.15.132:0/1188632493 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(110..110 src has 1..110) v4 ==== 5789+0+0 (unknown 1807949586 0 0) 0x7f7e500443a0 con 0x7f7e5409c4c0 2024-01-24T23:24:49.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.142+0000 7f7e59926640 10 monclient: get_auth_request con 0x7f7e38037050 auth_method 0 2024-01-24T23:24:49.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.146+0000 7f7e4affd640 1 -- 172.21.15.132:0/1188632493 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f7e50014cc0 con 0x7f7e5409c4c0 2024-01-24T23:24:49.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.146+0000 7f7e4affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:49.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.146+0000 7f7e4affd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:49.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.306+0000 7f7e5c38b640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.12"}] 2024-01-24T23:24:49.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.306+0000 7f7e5c38b640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.12"}] 2024-01-24T23:24:49.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.306+0000 7f7e5c38b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:49.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.306+0000 7f7e5c38b640 1 -- 172.21.15.132:0/1188632493 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "osd.12"} v 0) v1 -- 0x7f7e3c005340 con 0x7f7e5409c4c0 2024-01-24T23:24:49.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.922+0000 7f7e4affd640 1 -- 172.21.15.132:0/1188632493 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.12"}]=0 removed item id 12 name 'osd.12' from crush map v111) v1 ==== 125+0+0 (unknown 3304341722 0 0) 0x7f7e50013360 con 0x7f7e5409c4c0 2024-01-24T23:24:49.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.922+0000 7f7e4affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.12"}] 2024-01-24T23:24:49.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:49.922+0000 7f7e4affd640 10 monclient: _finish_command 2 = system:0 removed item id 12 name 'osd.12' from crush map 2024-01-24T23:24:50.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.018+0000 7f7e5c38b640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.12"}] 2024-01-24T23:24:50.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.018+0000 7f7e5c38b640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.12"}] 2024-01-24T23:24:50.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.018+0000 7f7e5c38b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:50.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.018+0000 7f7e5c38b640 1 -- 172.21.15.132:0/1188632493 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "osd.12"} v 0) v1 -- 0x7f7e3c002220 con 0x7f7e5409c4c0 2024-01-24T23:24:50.022 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.018+0000 7f7e4affd640 1 -- 172.21.15.132:0/1188632493 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.12"}]=0 device 'osd.12' does not appear in the crush map v111) v1 ==== 126+0+0 (unknown 2324780744 0 0) 0x7f7e50014ae0 con 0x7f7e5409c4c0 2024-01-24T23:24:50.022 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.018+0000 7f7e4affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.12"}] 2024-01-24T23:24:50.022 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.018+0000 7f7e4affd640 10 monclient: _finish_command 3 = system:0 device 'osd.12' does not appear in the crush map 2024-01-24T23:24:50.022 INFO:tasks.workunit.client.0.smithi132.stderr:device 'osd.12' does not appear in the crush map 2024-01-24T23:24:50.024 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.022+0000 7f7e5c38b640 1 -- 172.21.15.132:0/1188632493 >> v1:172.21.15.132:6812/1835401922 conn(0x7f7e38037050 legacy=0x7f7e38039510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:50.024 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.022+0000 7f7e5c38b640 10 monclient: shutdown 2024-01-24T23:24:50.024 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.022+0000 7f7e5c38b640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:50.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.022+0000 7f7e5c38b640 1 -- 172.21.15.132:0/1188632493 >> v1:172.21.15.132:6792/0 conn(0x7f7e5409c4c0 legacy=0x7f7e54132b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:50.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.022+0000 7f7e5c38b640 1 -- 172.21.15.132:0/1188632493 shutdown_connections 2024-01-24T23:24:50.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.022+0000 7f7e5c38b640 1 -- 172.21.15.132:0/1188632493 >> 172.21.15.132:0/1188632493 conn(0x7f7e540a5a10 msgr2=0x7f7e540a31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:50.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.022+0000 7f7e5c38b640 1 -- 172.21.15.132:0/1188632493 shutdown_connections 2024-01-24T23:24:50.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.022+0000 7f7e5c38b640 1 -- 172.21.15.132:0/1188632493 wait complete. 2024-01-24T23:24:50.038 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm osd.14 2024-01-24T23:24:50.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.150+0000 7ffbe134a640 10 monclient: get_monmap_and_config 2024-01-24T23:24:50.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.150+0000 7ffbe134a640 10 monclient: build_initial_monmap 2024-01-24T23:24:50.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.150+0000 7ffbe134a640 10 monclient: monmap: 2024-01-24T23:24:50.154 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:50.155 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:50.155 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:50.155910+0000 2024-01-24T23:24:50.155 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:50.155910+0000 2024-01-24T23:24:50.155 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:50.155 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:50.155 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:50.155 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:50.155 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:50.155 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:50.155 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:50.155 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:50.156 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:50.156 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:50.156 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:50.156 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:50.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.150+0000 7ffbe134a640 1 Processor -- start 2024-01-24T23:24:50.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.150+0000 7ffbe134a640 1 -- start start 2024-01-24T23:24:50.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.150+0000 7ffbe134a640 10 monclient: init 2024-01-24T23:24:50.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:50.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 10 monclient: _add_conns ranks=[6,1,3,5,8,2,7,0,4] 2024-01-24T23:24:50.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 10 monclient(hunting): picked mon.noname-d con 0x7ffbdc09ff20 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 10 monclient(hunting): picked mon.noname-f con 0x7ffbdc09f9c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 10 monclient(hunting): picked mon.noname-g con 0x7ffbdc13b2c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbdc064700 con 0x7ffbdc09f9c0 2024-01-24T23:24:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbdc09e760 con 0x7ffbdc13b2c0 2024-01-24T23:24:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbdc0a1160 con 0x7ffbdc09ff20 2024-01-24T23:24:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbda7fc640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7ffbdc09f9c0 legacy=0x7ffbdc139ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 10 monclient(hunting): authenticate will time out at 1177.480644s 2024-01-24T23:24:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbdb7fe640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7ffbdc13b2c0 0x7ffbdc13d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:33710/0 (socket says 172.21.15.132:33710) 2024-01-24T23:24:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbdb7fe640 1 -- 172.21.15.132:0/3601382092 learned_addr learned my addr 172.21.15.132:0/3601382092 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 1 -- 172.21.15.132:0/3601382092 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3589527790 0 0) 0x7ffbdc09e760 con 0x7ffbdc13b2c0 2024-01-24T23:24:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 10 monclient(hunting): my global_id is 5709 2024-01-24T23:24:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 1 -- 172.21.15.132:0/3601382092 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbc4003530 con 0x7ffbdc13b2c0 2024-01-24T23:24:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 1 -- 172.21.15.132:0/3601382092 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3682851490 0 0) 0x7ffbc4003530 con 0x7ffbdc13b2c0 2024-01-24T23:24:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 1 -- 172.21.15.132:0/3601382092 >> v1:172.21.15.115:6792/0 conn(0x7ffbdc09ff20 legacy=0x7ffbdc0a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:50.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 1 -- 172.21.15.132:0/3601382092 >> v1:172.21.15.132:6789/0 conn(0x7ffbdc09f9c0 legacy=0x7ffbdc139ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:50.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:24:50.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 1 monclient: found mon.noname-g 2024-01-24T23:24:50.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:50.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:24:50.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 1 -- 172.21.15.132:0/3601382092 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbdc0a0a10 con 0x7ffbdc13b2c0 2024-01-24T23:24:50.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 10 monclient: _finish_auth 0 2024-01-24T23:24:50.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:24:50.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 1 -- 172.21.15.132:0/3601382092 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ffbd4001b40 con 0x7ffbdc13b2c0 2024-01-24T23:24:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 10 monclient: dump: 2024-01-24T23:24:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:50.172 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 5 monclient: authenticate success, global_id 5709 2024-01-24T23:24:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 1 -- 172.21.15.132:0/3601382092 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ffbd4001e60 con 0x7ffbdc13b2c0 2024-01-24T23:24:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 1 -- 172.21.15.132:0/3601382092 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ffbd4004f70 con 0x7ffbdc13b2c0 2024-01-24T23:24:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 4 set_mon_vals no callback set 2024-01-24T23:24:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbd9ffb640 10 monclient: dump: 2024-01-24T23:24:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:50.173 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:50.174 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 10 monclient: shutdown 2024-01-24T23:24:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 1 -- 172.21.15.132:0/3601382092 >> v1:172.21.15.132:6790/0 conn(0x7ffbdc13b2c0 legacy=0x7ffbdc13d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 1 -- 172.21.15.132:0/3601382092 shutdown_connections 2024-01-24T23:24:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 1 -- 172.21.15.132:0/3601382092 >> 172.21.15.132:0/3601382092 conn(0x7ffbdc0a5a10 msgr2=0x7ffbdc0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 1 -- 172.21.15.132:0/3601382092 shutdown_connections 2024-01-24T23:24:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 1 -- 172.21.15.132:0/3601382092 wait complete. 2024-01-24T23:24:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 10 monclient: build_initial_monmap 2024-01-24T23:24:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 10 monclient: monmap: 2024-01-24T23:24:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:50.174 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:50.175 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:50.175 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:50.175 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:50.175 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:50.175 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:50.175 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:50.175 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:50.175 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:50.175 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:50.175 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:50.175 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:50.176 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 1 Processor -- start 2024-01-24T23:24:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 1 -- start start 2024-01-24T23:24:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 10 monclient: init 2024-01-24T23:24:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 10 monclient: _add_conns ranks=[8,6,0,4,1,3,5,2,7] 2024-01-24T23:24:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 10 monclient(hunting): picked mon.noname-e con 0x7ffbdc09f9c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 10 monclient(hunting): picked mon.noname-d con 0x7ffbdc09ff20 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 10 monclient(hunting): picked mon.noname-a con 0x7ffbdc13b2c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbdc133b60 con 0x7ffbdc13b2c0 2024-01-24T23:24:50.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbdc09d780 con 0x7ffbdc09ff20 2024-01-24T23:24:50.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbdc09d2e0 con 0x7ffbdc09f9c0 2024-01-24T23:24:50.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:50.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbe134a640 10 monclient(hunting): authenticate will time out at 1177.483643s 2024-01-24T23:24:50.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbdb7fe640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7ffbdc13b2c0 legacy=0x7ffbdc1d38d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:50.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbdaffd640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7ffbdc09f9c0 0x7ffbdc09c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:34704/0 (socket says 172.21.15.132:34704) 2024-01-24T23:24:50.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.154+0000 7ffbdaffd640 1 -- 172.21.15.132:0/3616030776 learned_addr learned my addr 172.21.15.132:0/3616030776 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:50.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 1 -- 172.21.15.132:0/3616030776 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1753652814 0 0) 0x7ffbdc09d2e0 con 0x7ffbdc09f9c0 2024-01-24T23:24:50.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 10 monclient(hunting): _negotiate 2024-01-24T23:24:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 10 monclient(hunting): my global_id is 84914 2024-01-24T23:24:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 1 -- 172.21.15.132:0/3616030776 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbc8003560 con 0x7ffbdc09f9c0 2024-01-24T23:24:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 1 -- 172.21.15.132:0/3616030776 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1453345843 0 0) 0x7ffbdc09d780 con 0x7ffbdc09ff20 2024-01-24T23:24:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 10 monclient(hunting): _negotiate 2024-01-24T23:24:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 10 monclient(hunting): my global_id is 5802 2024-01-24T23:24:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 1 -- 172.21.15.132:0/3616030776 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbdc09d2e0 con 0x7ffbdc09ff20 2024-01-24T23:24:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 1 -- 172.21.15.132:0/3616030776 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3613292398 0 0) 0x7ffbc8003560 con 0x7ffbdc09f9c0 2024-01-24T23:24:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 1 -- 172.21.15.132:0/3616030776 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffbdc09d780 con 0x7ffbdc09f9c0 2024-01-24T23:24:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 1 -- 172.21.15.132:0/3616030776 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2715479351 0 0) 0x7ffbdc09d2e0 con 0x7ffbdc09ff20 2024-01-24T23:24:50.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 1 -- 172.21.15.132:0/3616030776 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffbc8003560 con 0x7ffbdc09ff20 2024-01-24T23:24:50.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 1 -- 172.21.15.132:0/3616030776 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ffbcc001b60 con 0x7ffbdc09f9c0 2024-01-24T23:24:50.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:50.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:24:50.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 10 monclient(hunting): dump: 2024-01-24T23:24:50.181 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:50.181 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:50.181 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:50.181 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:50.181 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:50.181 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:50.181 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:50.181 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:50.181 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:50.181 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:50.181 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:50.182 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 1 -- 172.21.15.132:0/3616030776 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ffbd00041f0 con 0x7ffbdc09ff20 2024-01-24T23:24:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:24:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 10 monclient(hunting): dump: 2024-01-24T23:24:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:50.183 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 1 -- 172.21.15.132:0/3616030776 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 906346376 0 0) 0x7ffbdc09d780 con 0x7ffbdc09f9c0 2024-01-24T23:24:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 1 -- 172.21.15.132:0/3616030776 >> v1:172.21.15.115:6792/0 conn(0x7ffbdc09ff20 legacy=0x7ffbdc09cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 1 -- 172.21.15.132:0/3616030776 >> v1:172.21.15.115:6789/0 conn(0x7ffbdc13b2c0 legacy=0x7ffbdc1d38d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 10 monclient: _finish_hunting 0 2024-01-24T23:24:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 1 monclient: found mon.e 2024-01-24T23:24:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 1 -- 172.21.15.132:0/3616030776 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbdc071960 con 0x7ffbdc09f9c0 2024-01-24T23:24:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 10 monclient: _finish_auth 0 2024-01-24T23:24:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 10 monclient: _check_auth_tickets 2024-01-24T23:24:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbe134a640 5 monclient: authenticate success, global_id 84914 2024-01-24T23:24:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbe134a640 10 monclient: _renew_subs 2024-01-24T23:24:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbe134a640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbe134a640 1 -- 172.21.15.132:0/3616030776 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbdc071680 con 0x7ffbdc09f9c0 2024-01-24T23:24:50.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbe134a640 10 monclient: _renew_subs 2024-01-24T23:24:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbe134a640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbe134a640 1 -- 172.21.15.132:0/3616030776 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbdc1201a0 con 0x7ffbdc09f9c0 2024-01-24T23:24:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 1 -- 172.21.15.132:0/3616030776 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ffbcc0038d0 con 0x7ffbdc09f9c0 2024-01-24T23:24:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbd97fa640 4 set_mon_vals no callback set 2024-01-24T23:24:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 1 -- 172.21.15.132:0/3616030776 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ffbcc003e00 con 0x7ffbdc09f9c0 2024-01-24T23:24:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:24:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 10 monclient: dump: 2024-01-24T23:24:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:50.185 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:50.186 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:50.186 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:50.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbe134a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:50.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbe134a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:50.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbe134a640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:50.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbe134a640 1 -- 172.21.15.132:0/3616030776 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbbc005090 con 0x7ffbdc09f9c0 2024-01-24T23:24:50.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 1 -- 172.21.15.132:0/3616030776 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7ffbcc003fe0 con 0x7ffbdc09f9c0 2024-01-24T23:24:50.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbc37fe640 1 -- 172.21.15.132:0/3616030776 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (unknown 4049208081 0 0) 0x7ffbcc043f30 con 0x7ffbdc09f9c0 2024-01-24T23:24:50.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.158+0000 7ffbda7fc640 10 monclient: get_auth_request con 0x7ffbc80376b0 auth_method 0 2024-01-24T23:24:50.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.170+0000 7ffbc37fe640 1 -- 172.21.15.132:0/3616030776 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7ffbcc012e80 con 0x7ffbdc09f9c0 2024-01-24T23:24:50.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.170+0000 7ffbc37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:50.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.170+0000 7ffbc37fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:50.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.338+0000 7ffbe134a640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.14"}] 2024-01-24T23:24:50.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.338+0000 7ffbe134a640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.14"}] 2024-01-24T23:24:50.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.338+0000 7ffbe134a640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:50.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.338+0000 7ffbe134a640 1 -- 172.21.15.132:0/3616030776 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush rm", "name": "osd.14"} v 0) v1 -- 0x7ffbbc005340 con 0x7ffbdc09f9c0 2024-01-24T23:24:50.927 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.926+0000 7ffbc37fe640 1 -- 172.21.15.132:0/3616030776 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.14"}]=0 removed item id 14 name 'osd.14' from crush map v112) v1 ==== 125+0+0 (unknown 887184599 0 0) 0x7ffbcc012e80 con 0x7ffbdc09f9c0 2024-01-24T23:24:50.928 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.926+0000 7ffbc37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.14"}] 2024-01-24T23:24:50.928 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:50.926+0000 7ffbc37fe640 10 monclient: _finish_command 2 = system:0 removed item id 14 name 'osd.14' from crush map 2024-01-24T23:24:51.035 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.034+0000 7ffbe134a640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.14"}] 2024-01-24T23:24:51.035 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.034+0000 7ffbe134a640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.14"}] 2024-01-24T23:24:51.035 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.034+0000 7ffbe134a640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:24:51.035 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.034+0000 7ffbe134a640 1 -- 172.21.15.132:0/3616030776 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush rm", "name": "osd.14"} v 0) v1 -- 0x7ffbbc0010a0 con 0x7ffbdc09f9c0 2024-01-24T23:24:51.037 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.034+0000 7ffbc37fe640 1 -- 172.21.15.132:0/3616030776 <== mon.8 v1:172.21.15.115:6793/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.14"}]=0 device 'osd.14' does not appear in the crush map v112) v1 ==== 126+0+0 (unknown 4109962052 0 0) 0x7ffbcc012e80 con 0x7ffbdc09f9c0 2024-01-24T23:24:51.037 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.034+0000 7ffbc37fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.14"}] 2024-01-24T23:24:51.037 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.034+0000 7ffbc37fe640 10 monclient: _finish_command 3 = system:0 device 'osd.14' does not appear in the crush map 2024-01-24T23:24:51.037 INFO:tasks.workunit.client.0.smithi132.stderr:device 'osd.14' does not appear in the crush map 2024-01-24T23:24:51.039 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.038+0000 7ffbe134a640 1 -- 172.21.15.132:0/3616030776 >> v1:172.21.15.132:6812/1835401922 conn(0x7ffbc80376b0 legacy=0x7ffbc8039b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:51.039 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.038+0000 7ffbe134a640 10 monclient: shutdown 2024-01-24T23:24:51.040 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.038+0000 7ffbe134a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:51.040 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.038+0000 7ffbe134a640 1 -- 172.21.15.132:0/3616030776 >> v1:172.21.15.115:6793/0 conn(0x7ffbdc09f9c0 legacy=0x7ffbdc09c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:51.040 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.038+0000 7ffbe134a640 1 -- 172.21.15.132:0/3616030776 shutdown_connections 2024-01-24T23:24:51.040 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.038+0000 7ffbe134a640 1 -- 172.21.15.132:0/3616030776 >> 172.21.15.132:0/3616030776 conn(0x7ffbdc0a5a10 msgr2=0x7ffbdc0a2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:51.040 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.038+0000 7ffbe134a640 1 -- 172.21.15.132:0/3616030776 shutdown_connections 2024-01-24T23:24:51.040 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.038+0000 7ffbe134a640 1 -- 172.21.15.132:0/3616030776 wait complete. 2024-01-24T23:24:51.054 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd rm osd.12 2024-01-24T23:24:51.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.166+0000 7fa65dc2d640 10 monclient: get_monmap_and_config 2024-01-24T23:24:51.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.166+0000 7fa65dc2d640 10 monclient: build_initial_monmap 2024-01-24T23:24:51.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.166+0000 7fa65dc2d640 10 monclient: monmap: 2024-01-24T23:24:51.169 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:51.169 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:51.169 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:51.170264+0000 2024-01-24T23:24:51.169 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:51.170264+0000 2024-01-24T23:24:51.169 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:51.169 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:51.169 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:51.169 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:51.169 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:51.169 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:51.169 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:51.169 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:51.170 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:51.170 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:51.170 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:51.170 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:51.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.166+0000 7fa65dc2d640 1 Processor -- start 2024-01-24T23:24:51.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.166+0000 7fa65dc2d640 1 -- start start 2024-01-24T23:24:51.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.166+0000 7fa65dc2d640 10 monclient: init 2024-01-24T23:24:51.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.166+0000 7fa65dc2d640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:51.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.166+0000 7fa65dc2d640 10 monclient: _add_conns ranks=[4,3,2,0,6,7,1,5,8] 2024-01-24T23:24:51.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.166+0000 7fa65dc2d640 10 monclient(hunting): picked mon.noname-c con 0x7fa658133870 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:51.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.166+0000 7fa65dc2d640 10 monclient(hunting): picked mon.noname-g con 0x7fa658134360 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:51.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.166+0000 7fa65dc2d640 10 monclient(hunting): picked mon.noname-b con 0x7fa65813a4c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:51.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.166+0000 7fa65dc2d640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa658064700 con 0x7fa65813a4c0 2024-01-24T23:24:51.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.166+0000 7fa65dc2d640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65809bf30 con 0x7fa658134360 2024-01-24T23:24:51.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.166+0000 7fa65dc2d640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65809c210 con 0x7fa658133870 2024-01-24T23:24:51.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.166+0000 7fa65dc2d640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:51.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.166+0000 7fa65dc2d640 10 monclient(hunting): authenticate will time out at 1178.495974s 2024-01-24T23:24:51.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa64ffff640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fa658134360 0x7fa658138da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:33712/0 (socket says 172.21.15.132:33712) 2024-01-24T23:24:51.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa6577fe640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7fa658133870 legacy=0x7fa658133c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:51.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa64ffff640 1 -- 172.21.15.132:0/1699890399 learned_addr learned my addr 172.21.15.132:0/1699890399 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:51.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 1 -- 172.21.15.132:0/1699890399 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323423225 0 0) 0x7fa65809bf30 con 0x7fa658134360 2024-01-24T23:24:51.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:51.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:51.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:51.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 10 monclient(hunting): my global_id is 5718 2024-01-24T23:24:51.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 1 -- 172.21.15.132:0/1699890399 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa644003530 con 0x7fa658134360 2024-01-24T23:24:51.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 1 -- 172.21.15.132:0/1699890399 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1838859934 0 0) 0x7fa644003530 con 0x7fa658134360 2024-01-24T23:24:51.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 1 -- 172.21.15.132:0/1699890399 >> v1:172.21.15.115:6791/0 conn(0x7fa658133870 legacy=0x7fa658133c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:51.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 1 -- 172.21.15.132:0/1699890399 >> v1:172.21.15.115:6790/0 conn(0x7fa65813a4c0 legacy=0x7fa65813c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:51.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:51.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 1 monclient: found mon.noname-g 2024-01-24T23:24:51.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:51.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:24:51.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 1 -- 172.21.15.132:0/1699890399 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65809c4f0 con 0x7fa658134360 2024-01-24T23:24:51.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 10 monclient: _finish_auth 0 2024-01-24T23:24:51.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:51.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:51.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 1 -- 172.21.15.132:0/1699890399 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa6400017e0 con 0x7fa658134360 2024-01-24T23:24:51.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:51.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:51.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 10 monclient: dump: 2024-01-24T23:24:51.180 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:51.180 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:51.180 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:51.180 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:51.180 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:51.180 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:51.180 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:51.180 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:51.180 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:51.180 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:51.180 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:51.180 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:51.180 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:51.180 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:51.181 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:51.181 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:51.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 5 monclient: authenticate success, global_id 5718 2024-01-24T23:24:51.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:51.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 1 -- 172.21.15.132:0/1699890399 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa640001b00 con 0x7fa658134360 2024-01-24T23:24:51.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:51.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 1 -- 172.21.15.132:0/1699890399 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa640004bf0 con 0x7fa658134360 2024-01-24T23:24:51.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:51.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:51.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa656ffd640 10 monclient: dump: 2024-01-24T23:24:51.181 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:51.181 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:51.181 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:51.181 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:51.181 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:51.182 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:51.182 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:51.182 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:51.182 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:51.182 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:51.182 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:51.182 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:51.182 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:51.182 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:51.182 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:51.182 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:51.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:51.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 4 set_mon_vals no callback set 2024-01-24T23:24:51.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 10 monclient: shutdown 2024-01-24T23:24:51.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:51.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 1 -- 172.21.15.132:0/1699890399 >> v1:172.21.15.132:6790/0 conn(0x7fa658134360 legacy=0x7fa658138da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:51.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 1 -- 172.21.15.132:0/1699890399 shutdown_connections 2024-01-24T23:24:51.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 1 -- 172.21.15.132:0/1699890399 >> 172.21.15.132:0/1699890399 conn(0x7fa6580a6040 msgr2=0x7fa6580a6420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:51.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 1 -- 172.21.15.132:0/1699890399 shutdown_connections 2024-01-24T23:24:51.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 1 -- 172.21.15.132:0/1699890399 wait complete. 2024-01-24T23:24:51.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 10 monclient: build_initial_monmap 2024-01-24T23:24:51.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 10 monclient: monmap: 2024-01-24T23:24:51.183 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:51.183 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:51.183 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:51.183 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:51.183 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:51.183 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:51.183 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:51.183 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:51.183 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:51.184 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:51.184 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:51.184 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:51.184 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:51.184 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:51.184 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:51.185 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:51.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 1 Processor -- start 2024-01-24T23:24:51.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 1 -- start start 2024-01-24T23:24:51.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 10 monclient: init 2024-01-24T23:24:51.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:51.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 10 monclient: _add_conns ranks=[6,2,5,1,8,4,7,0,3] 2024-01-24T23:24:51.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 10 monclient(hunting): picked mon.noname-d con 0x7fa658133870 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:51.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 10 monclient(hunting): picked mon.noname-b con 0x7fa658134360 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:51.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 10 monclient(hunting): picked mon.noname-h con 0x7fa65813a4c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:51.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65809fde0 con 0x7fa658134360 2024-01-24T23:24:51.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65809f900 con 0x7fa65813a4c0 2024-01-24T23:24:51.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa658070430 con 0x7fa658133870 2024-01-24T23:24:51.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:51.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 10 monclient(hunting): authenticate will time out at 1178.498866s 2024-01-24T23:24:51.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa657fff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fa65813a4c0 0x7fa6581d4050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:40574/0 (socket says 172.21.15.132:40574) 2024-01-24T23:24:51.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa657fff640 1 -- 172.21.15.132:0/164465823 learned_addr learned my addr 172.21.15.132:0/164465823 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:51.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 1 -- 172.21.15.132:0/164465823 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900266387 0 0) 0x7fa65809f900 con 0x7fa65813a4c0 2024-01-24T23:24:51.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:24:51.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:51.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:51.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 10 monclient(hunting): my global_id is 5666 2024-01-24T23:24:51.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 1 -- 172.21.15.132:0/164465823 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa638003530 con 0x7fa65813a4c0 2024-01-24T23:24:51.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 1 -- 172.21.15.132:0/164465823 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2766977052 0 0) 0x7fa638003530 con 0x7fa65813a4c0 2024-01-24T23:24:51.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 1 -- 172.21.15.132:0/164465823 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa65809f900 con 0x7fa65813a4c0 2024-01-24T23:24:51.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 1 -- 172.21.15.132:0/164465823 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa650001d80 con 0x7fa65813a4c0 2024-01-24T23:24:51.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:51.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:51.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 10 monclient(hunting): dump: 2024-01-24T23:24:51.187 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:51.187 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:51.187 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:51.187 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:51.187 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:51.187 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:51.187 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:51.187 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:51.187 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:51.187 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:51.187 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:51.187 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:51.188 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:51.188 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:51.188 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:51.188 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:51.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:51.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 1 -- 172.21.15.132:0/164465823 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3090399838 0 0) 0x7fa65809fde0 con 0x7fa658134360 2024-01-24T23:24:51.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:24:51.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:51.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:51.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 10 monclient(hunting): my global_id is 85007 2024-01-24T23:24:51.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 1 -- 172.21.15.132:0/164465823 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa638003530 con 0x7fa658134360 2024-01-24T23:24:51.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 1 -- 172.21.15.132:0/164465823 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1627220674 0 0) 0x7fa65809f900 con 0x7fa65813a4c0 2024-01-24T23:24:51.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 1 -- 172.21.15.132:0/164465823 >> v1:172.21.15.115:6792/0 conn(0x7fa658133870 legacy=0x7fa6581cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:51.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 1 -- 172.21.15.132:0/164465823 >> v1:172.21.15.115:6790/0 conn(0x7fa658134360 legacy=0x7fa6581d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:51.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:24:51.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 1 monclient: found mon.h 2024-01-24T23:24:51.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:51.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:51.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 1 -- 172.21.15.132:0/164465823 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa658120c40 con 0x7fa65813a4c0 2024-01-24T23:24:51.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 10 monclient: _finish_auth 0 2024-01-24T23:24:51.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:24:51.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:51.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 5 monclient: authenticate success, global_id 5666 2024-01-24T23:24:51.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 10 monclient: _renew_subs 2024-01-24T23:24:51.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:51.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 1 -- 172.21.15.132:0/164465823 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa65806f260 con 0x7fa65813a4c0 2024-01-24T23:24:51.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 10 monclient: _renew_subs 2024-01-24T23:24:51.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:51.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa65dc2d640 1 -- 172.21.15.132:0/164465823 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa65806efb0 con 0x7fa65813a4c0 2024-01-24T23:24:51.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 1 -- 172.21.15.132:0/164465823 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa650003cd0 con 0x7fa65813a4c0 2024-01-24T23:24:51.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:51.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 1 -- 172.21.15.132:0/164465823 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa650004180 con 0x7fa65813a4c0 2024-01-24T23:24:51.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:51.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:51.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa6567fc640 4 set_mon_vals no callback set 2024-01-24T23:24:51.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.170+0000 7fa654ff9640 10 monclient: dump: 2024-01-24T23:24:51.190 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:51.190 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:51.190 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:51.190 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:51.190 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:51.190 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:51.190 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:51.191 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:51.191 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:51.191 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:51.191 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:51.191 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:51.191 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:51.191 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:51.191 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:51.191 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:51.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.174+0000 7fa65dc2d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:51.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.174+0000 7fa65dc2d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:51.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.174+0000 7fa65dc2d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:51.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.174+0000 7fa65dc2d640 1 -- 172.21.15.132:0/164465823 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa63c005090 con 0x7fa65813a4c0 2024-01-24T23:24:51.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.174+0000 7fa654ff9640 1 -- 172.21.15.132:0/164465823 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fa6500042e0 con 0x7fa65813a4c0 2024-01-24T23:24:51.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.174+0000 7fa654ff9640 1 -- 172.21.15.132:0/164465823 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(112..112 src has 1..112) v4 ==== 5716+0+0 (unknown 574437236 0 0) 0x7fa650044210 con 0x7fa65813a4c0 2024-01-24T23:24:51.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.174+0000 7fa6577fe640 10 monclient: get_auth_request con 0x7fa638037050 auth_method 0 2024-01-24T23:24:51.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.178+0000 7fa654ff9640 1 -- 172.21.15.132:0/164465823 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fa650013220 con 0x7fa65813a4c0 2024-01-24T23:24:51.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.178+0000 7fa654ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:51.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.178+0000 7fa654ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:51.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.346+0000 7fa65dc2d640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["osd.12"]}] 2024-01-24T23:24:51.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.346+0000 7fa65dc2d640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["osd.12"]}] 2024-01-24T23:24:51.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.346+0000 7fa65dc2d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:51.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.346+0000 7fa65dc2d640 1 -- 172.21.15.132:0/164465823 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd rm", "ids": ["osd.12"]} v 0) v1 -- 0x7fa63c0058c0 con 0x7fa65813a4c0 2024-01-24T23:24:51.348 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:24:51.346+0000 7fd2d64cd640 -1 mon.g@3(leader).osd e112 definitely_dead 0 2024-01-24T23:24:51.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.898+0000 7fa654ff9640 1 -- 172.21.15.132:0/164465823 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["osd.12"]}]=0 removed osd.12 v113) v1 ==== 87+0+0 (unknown 3379236691 0 0) 0x7fa650013220 con 0x7fa65813a4c0 2024-01-24T23:24:51.901 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.898+0000 7fa654ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["osd.12"]}] 2024-01-24T23:24:51.901 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:51.898+0000 7fa654ff9640 10 monclient: _finish_command 2 = system:0 removed osd.12 2024-01-24T23:24:52.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.014+0000 7fa65dc2d640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["osd.12"]}] 2024-01-24T23:24:52.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.014+0000 7fa65dc2d640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["osd.12"]}] 2024-01-24T23:24:52.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.014+0000 7fa65dc2d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:52.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.014+0000 7fa65dc2d640 1 -- 172.21.15.132:0/164465823 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd rm", "ids": ["osd.12"]} v 0) v1 -- 0x7fa63c000bd0 con 0x7fa65813a4c0 2024-01-24T23:24:52.018 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:24:52.014+0000 7fd2d64cd640 -1 mon.g@3(leader).osd e113 definitely_dead 0 2024-01-24T23:24:52.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.018+0000 7fa654ff9640 1 -- 172.21.15.132:0/164465823 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["osd.12"]}]=0 osd.12 does not exist. v113) v1 ==== 96+0+0 (unknown 2726196454 0 0) 0x7fa650013220 con 0x7fa65813a4c0 2024-01-24T23:24:52.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.018+0000 7fa654ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["osd.12"]}] 2024-01-24T23:24:52.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.018+0000 7fa654ff9640 10 monclient: _finish_command 3 = system:0 osd.12 does not exist. 2024-01-24T23:24:52.019 INFO:tasks.workunit.client.0.smithi132.stderr:osd.12 does not exist. 2024-01-24T23:24:52.024 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.018+0000 7fa65dc2d640 1 -- 172.21.15.132:0/164465823 >> v1:172.21.15.132:6812/1835401922 conn(0x7fa638037050 legacy=0x7fa638039510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:52.024 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.018+0000 7fa65dc2d640 10 monclient: shutdown 2024-01-24T23:24:52.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.018+0000 7fa65dc2d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:52.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.018+0000 7fa65dc2d640 1 -- 172.21.15.132:0/164465823 >> v1:172.21.15.132:6791/0 conn(0x7fa65813a4c0 legacy=0x7fa6581d4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:52.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.018+0000 7fa65dc2d640 1 -- 172.21.15.132:0/164465823 shutdown_connections 2024-01-24T23:24:52.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.018+0000 7fa65dc2d640 1 -- 172.21.15.132:0/164465823 >> 172.21.15.132:0/164465823 conn(0x7fa6580a6040 msgr2=0x7fa6580a3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:52.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.018+0000 7fa65dc2d640 1 -- 172.21.15.132:0/164465823 shutdown_connections 2024-01-24T23:24:52.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.018+0000 7fa65dc2d640 1 -- 172.21.15.132:0/164465823 wait complete. 2024-01-24T23:24:52.036 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd rm osd.14 2024-01-24T23:24:52.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.146+0000 7fe8ed6a1640 10 monclient: get_monmap_and_config 2024-01-24T23:24:52.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.146+0000 7fe8ed6a1640 10 monclient: build_initial_monmap 2024-01-24T23:24:52.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.146+0000 7fe8ed6a1640 10 monclient: monmap: 2024-01-24T23:24:52.151 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:52.151 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:52.151 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:52.152586+0000 2024-01-24T23:24:52.151 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:52.152586+0000 2024-01-24T23:24:52.151 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:52.151 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:52.151 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:52.152 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:52.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8ed6a1640 1 Processor -- start 2024-01-24T23:24:52.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8ed6a1640 1 -- start start 2024-01-24T23:24:52.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8ed6a1640 10 monclient: init 2024-01-24T23:24:52.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8ed6a1640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:52.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8ed6a1640 10 monclient: _add_conns ranks=[2,1,0,3,7,5,6,4,8] 2024-01-24T23:24:52.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8ed6a1640 10 monclient(hunting): picked mon.noname-b con 0x7fe8e81370e0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:52.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8ed6a1640 10 monclient(hunting): picked mon.noname-f con 0x7fe8e809bc20 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:52.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8ed6a1640 10 monclient(hunting): picked mon.noname-a con 0x7fe8e813d640 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:52.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8ed6a1640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e8064700 con 0x7fe8e813d640 2024-01-24T23:24:52.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8ed6a1640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e806a530 con 0x7fe8e809bc20 2024-01-24T23:24:52.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8ed6a1640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e809af60 con 0x7fe8e81370e0 2024-01-24T23:24:52.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8ed6a1640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:52.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8ed6a1640 10 monclient(hunting): authenticate will time out at 1179.477571s 2024-01-24T23:24:52.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e67fc640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fe8e809bc20 legacy=0x7fe8e813bf20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:52.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e77fe640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fe8e813d640 legacy=0x7fe8e813fa30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:52.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e6ffd640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fe8e81370e0 0x7fe8e81374c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:60020/0 (socket says 172.21.15.132:60020) 2024-01-24T23:24:52.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e6ffd640 1 -- 172.21.15.132:0/3027210853 learned_addr learned my addr 172.21.15.132:0/3027210853 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:52.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 1 -- 172.21.15.132:0/3027210853 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1261023889 0 0) 0x7fe8e809af60 con 0x7fe8e81370e0 2024-01-24T23:24:52.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:52.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:52.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:52.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 10 monclient(hunting): my global_id is 85016 2024-01-24T23:24:52.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 1 -- 172.21.15.132:0/3027210853 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8d0003530 con 0x7fe8e81370e0 2024-01-24T23:24:52.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 1 -- 172.21.15.132:0/3027210853 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4172085821 0 0) 0x7fe8d0003530 con 0x7fe8e81370e0 2024-01-24T23:24:52.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 1 -- 172.21.15.132:0/3027210853 >> v1:172.21.15.132:6789/0 conn(0x7fe8e809bc20 legacy=0x7fe8e813bf20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:52.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 1 -- 172.21.15.132:0/3027210853 >> v1:172.21.15.115:6789/0 conn(0x7fe8e813d640 legacy=0x7fe8e813fa30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:52.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:24:52.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 1 monclient: found mon.noname-b 2024-01-24T23:24:52.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:52.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:24:52.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 1 -- 172.21.15.132:0/3027210853 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e809b240 con 0x7fe8e81370e0 2024-01-24T23:24:52.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 10 monclient: _finish_auth 0 2024-01-24T23:24:52.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:24:52.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:52.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 1 -- 172.21.15.132:0/3027210853 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe8dc0017c0 con 0x7fe8e81370e0 2024-01-24T23:24:52.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:52.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:24:52.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 10 monclient: dump: 2024-01-24T23:24:52.158 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:52.158 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:52.158 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:52.158 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:52.158 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:52.158 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:52.158 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:52.158 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:52.158 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:52.158 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:52.158 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:52.158 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:52.158 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:52.158 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:52.158 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:52.158 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:52.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8ed6a1640 5 monclient: authenticate success, global_id 85016 2024-01-24T23:24:52.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8ed6a1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:52.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 1 -- 172.21.15.132:0/3027210853 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe8dc001ae0 con 0x7fe8e81370e0 2024-01-24T23:24:52.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:52.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 1 -- 172.21.15.132:0/3027210853 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe8dc004bf0 con 0x7fe8e81370e0 2024-01-24T23:24:52.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8ed6a1640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:52.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8ed6a1640 4 set_mon_vals no callback set 2024-01-24T23:24:52.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:52.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:24:52.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8e5ffb640 10 monclient: dump: 2024-01-24T23:24:52.159 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:52.159 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:52.159 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:52.159 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:52.159 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:52.160 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:52.160 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:52.160 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:52.160 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:52.160 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:52.160 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:52.160 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:52.160 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:52.160 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:52.160 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:52.160 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:52.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8ed6a1640 10 monclient: shutdown 2024-01-24T23:24:52.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8ed6a1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:52.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.150+0000 7fe8ed6a1640 1 -- 172.21.15.132:0/3027210853 >> v1:172.21.15.115:6790/0 conn(0x7fe8e81370e0 legacy=0x7fe8e81374c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:52.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 1 -- 172.21.15.132:0/3027210853 shutdown_connections 2024-01-24T23:24:52.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 1 -- 172.21.15.132:0/3027210853 >> 172.21.15.132:0/3027210853 conn(0x7fe8e80a2b40 msgr2=0x7fe8e80a2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:52.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 1 -- 172.21.15.132:0/3027210853 shutdown_connections 2024-01-24T23:24:52.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 1 -- 172.21.15.132:0/3027210853 wait complete. 2024-01-24T23:24:52.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 10 monclient: build_initial_monmap 2024-01-24T23:24:52.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 10 monclient: monmap: 2024-01-24T23:24:52.166 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:52.166 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:52.166 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:52.166 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:52.166 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:52.166 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:52.166 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:52.166 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:52.166 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:52.166 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:52.166 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:52.166 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:52.166 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:52.166 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:52.167 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:52.167 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:52.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 1 Processor -- start 2024-01-24T23:24:52.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 1 -- start start 2024-01-24T23:24:52.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 10 monclient: init 2024-01-24T23:24:52.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:52.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 10 monclient: _add_conns ranks=[7,3,5,0,2,1,4,6,8] 2024-01-24T23:24:52.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 10 monclient(hunting): picked mon.noname-i con 0x7fe8e809bc20 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:52.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 10 monclient(hunting): picked mon.noname-g con 0x7fe8e81370e0 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:52.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 10 monclient(hunting): picked mon.noname-h con 0x7fe8e813d640 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:52.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e806e650 con 0x7fe8e81370e0 2024-01-24T23:24:52.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e809c890 con 0x7fe8e813d640 2024-01-24T23:24:52.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e806b490 con 0x7fe8e809bc20 2024-01-24T23:24:52.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:52.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 10 monclient(hunting): authenticate will time out at 1179.481695s 2024-01-24T23:24:52.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8e6ffd640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fe8e809bc20 0x7fe8e809ded0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:48698/0 (socket says 172.21.15.132:48698) 2024-01-24T23:24:52.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8e6ffd640 1 -- 172.21.15.132:0/3054056174 learned_addr learned my addr 172.21.15.132:0/3054056174 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:52.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1695233450 0 0) 0x7fe8e806e650 con 0x7fe8e81370e0 2024-01-24T23:24:52.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:24:52.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:52.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:52.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient(hunting): my global_id is 5727 2024-01-24T23:24:52.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8d4003530 con 0x7fe8e81370e0 2024-01-24T23:24:52.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2834552190 0 0) 0x7fe8e809c890 con 0x7fe8e813d640 2024-01-24T23:24:52.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:24:52.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:52.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:52.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient(hunting): my global_id is 5675 2024-01-24T23:24:52.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8e806e650 con 0x7fe8e813d640 2024-01-24T23:24:52.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1043377426 0 0) 0x7fe8e806b490 con 0x7fe8e809bc20 2024-01-24T23:24:52.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:24:52.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:52.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:52.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient(hunting): my global_id is 5704 2024-01-24T23:24:52.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8e809c890 con 0x7fe8e809bc20 2024-01-24T23:24:52.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3223336711 0 0) 0x7fe8e809c890 con 0x7fe8e809bc20 2024-01-24T23:24:52.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe8e806b490 con 0x7fe8e809bc20 2024-01-24T23:24:52.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1853180572 0 0) 0x7fe8d4003530 con 0x7fe8e81370e0 2024-01-24T23:24:52.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe8e809c890 con 0x7fe8e81370e0 2024-01-24T23:24:52.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2323758044 0 0) 0x7fe8e806e650 con 0x7fe8e813d640 2024-01-24T23:24:52.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe8d4003530 con 0x7fe8e813d640 2024-01-24T23:24:52.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe8dc004b30 con 0x7fe8e809bc20 2024-01-24T23:24:52.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:52.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:52.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient(hunting): dump: 2024-01-24T23:24:52.170 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:52.170 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:52.170 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:52.170 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:52.170 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:52.170 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:52.170 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:52.170 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:52.170 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:52.170 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:52.170 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:52.170 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:52.170 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:52.170 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:52.170 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:52.171 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe8d80041c0 con 0x7fe8e81370e0 2024-01-24T23:24:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient(hunting): dump: 2024-01-24T23:24:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:52.171 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:52.172 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe8e0004490 con 0x7fe8e813d640 2024-01-24T23:24:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient(hunting): dump: 2024-01-24T23:24:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:52.172 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:52.173 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 79438102 0 0) 0x7fe8e806b490 con 0x7fe8e809bc20 2024-01-24T23:24:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 >> v1:172.21.15.132:6791/0 conn(0x7fe8e813d640 legacy=0x7fe8e81d3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:52.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 >> v1:172.21.15.132:6790/0 conn(0x7fe8e81370e0 legacy=0x7fe8e809e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:24:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 1 monclient: found mon.i 2024-01-24T23:24:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e809fe70 con 0x7fe8e809bc20 2024-01-24T23:24:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient: _finish_auth 0 2024-01-24T23:24:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:24:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 5 monclient: authenticate success, global_id 5704 2024-01-24T23:24:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 10 monclient: _renew_subs 2024-01-24T23:24:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 1 -- 172.21.15.132:0/3054056174 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e806f770 con 0x7fe8e809bc20 2024-01-24T23:24:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 10 monclient: _renew_subs 2024-01-24T23:24:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 1 -- 172.21.15.132:0/3054056174 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e80a0450 con 0x7fe8e809bc20 2024-01-24T23:24:52.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe8dc0017a0 con 0x7fe8e809bc20 2024-01-24T23:24:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe8dc001c50 con 0x7fe8e809bc20 2024-01-24T23:24:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8e4ff9640 4 set_mon_vals no callback set 2024-01-24T23:24:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 10 monclient: dump: 2024-01-24T23:24:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:52.175 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:52.176 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8ed6a1640 1 -- 172.21.15.132:0/3054056174 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e809c020 con 0x7fe8e809bc20 2024-01-24T23:24:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fe8dc00fdc0 con 0x7fe8e809bc20 2024-01-24T23:24:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.154+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(113..113 src has 1..113) v4 ==== 5687+0+0 (unknown 766405335 0 0) 0x7fe8dc043fc0 con 0x7fe8e809bc20 2024-01-24T23:24:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.158+0000 7fe8e67fc640 10 monclient: get_auth_request con 0x7fe8d4037dc0 auth_method 0 2024-01-24T23:24:52.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.162+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fe8dc014940 con 0x7fe8e809bc20 2024-01-24T23:24:52.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.162+0000 7fe8cf7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:52.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.162+0000 7fe8cf7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:52.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.326+0000 7fe8ed6a1640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["osd.14"]}] 2024-01-24T23:24:52.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.326+0000 7fe8ed6a1640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["osd.14"]}] 2024-01-24T23:24:52.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.326+0000 7fe8ed6a1640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:52.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.326+0000 7fe8ed6a1640 1 -- 172.21.15.132:0/3054056174 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd rm", "ids": ["osd.14"]} v 0) v1 -- 0x7fe8e8073400 con 0x7fe8e809bc20 2024-01-24T23:24:52.330 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:24:52.326+0000 7fd2d64cd640 -1 mon.g@3(leader).osd e113 definitely_dead 0 2024-01-24T23:24:52.901 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.898+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["osd.14"]}]=0 removed osd.14 v114) v1 ==== 87+0+0 (unknown 1359308608 0 0) 0x7fe8dc014760 con 0x7fe8e809bc20 2024-01-24T23:24:52.901 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.898+0000 7fe8cf7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["osd.14"]}] 2024-01-24T23:24:52.901 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:52.898+0000 7fe8cf7fe640 10 monclient: _finish_command 2 = system:0 removed osd.14 2024-01-24T23:24:53.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.010+0000 7fe8ed6a1640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["osd.14"]}] 2024-01-24T23:24:53.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.010+0000 7fe8ed6a1640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["osd.14"]}] 2024-01-24T23:24:53.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.010+0000 7fe8ed6a1640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:53.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.010+0000 7fe8ed6a1640 1 -- 172.21.15.132:0/3054056174 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd rm", "ids": ["osd.14"]} v 0) v1 -- 0x7fe8e80008e0 con 0x7fe8e809bc20 2024-01-24T23:24:53.015 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:24:53.014+0000 7fd2d64cd640 -1 mon.g@3(leader).osd e114 definitely_dead 0 2024-01-24T23:24:53.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.014+0000 7fe8cf7fe640 1 -- 172.21.15.132:0/3054056174 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["osd.14"]}]=0 osd.14 does not exist. v114) v1 ==== 96+0+0 (unknown 1645029502 0 0) 0x7fe8dc014940 con 0x7fe8e809bc20 2024-01-24T23:24:53.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.014+0000 7fe8cf7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["osd.14"]}] 2024-01-24T23:24:53.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.014+0000 7fe8cf7fe640 10 monclient: _finish_command 3 = system:0 osd.14 does not exist. 2024-01-24T23:24:53.016 INFO:tasks.workunit.client.0.smithi132.stderr:osd.14 does not exist. 2024-01-24T23:24:53.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.014+0000 7fe8ed6a1640 1 -- 172.21.15.132:0/3054056174 >> v1:172.21.15.132:6812/1835401922 conn(0x7fe8d4037dc0 legacy=0x7fe8d403a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:53.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.014+0000 7fe8ed6a1640 10 monclient: shutdown 2024-01-24T23:24:53.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.014+0000 7fe8ed6a1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:53.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.014+0000 7fe8ed6a1640 1 -- 172.21.15.132:0/3054056174 >> v1:172.21.15.132:6792/0 conn(0x7fe8e809bc20 legacy=0x7fe8e809ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:53.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.014+0000 7fe8ed6a1640 1 -- 172.21.15.132:0/3054056174 shutdown_connections 2024-01-24T23:24:53.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.014+0000 7fe8ed6a1640 1 -- 172.21.15.132:0/3054056174 >> 172.21.15.132:0/3054056174 conn(0x7fe8e80a2b40 msgr2=0x7fe8e8135b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:53.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.014+0000 7fe8ed6a1640 1 -- 172.21.15.132:0/3054056174 shutdown_connections 2024-01-24T23:24:53.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.014+0000 7fe8ed6a1640 1 -- 172.21.15.132:0/3054056174 wait complete. 2024-01-24T23:24:53.032 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd set-require-min-compat-client luminous 2024-01-24T23:24:53.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 10 monclient: get_monmap_and_config 2024-01-24T23:24:53.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 10 monclient: build_initial_monmap 2024-01-24T23:24:53.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 10 monclient: monmap: 2024-01-24T23:24:53.147 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:53.148 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:53.148 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:53.149030+0000 2024-01-24T23:24:53.148 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:53.149030+0000 2024-01-24T23:24:53.148 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:53.148 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:53.148 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:53.148 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:53.148 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:53.148 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:53.148 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:53.148 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:53.148 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:53.148 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:53.148 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:53.149 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:53.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 1 Processor -- start 2024-01-24T23:24:53.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 1 -- start start 2024-01-24T23:24:53.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 10 monclient: init 2024-01-24T23:24:53.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:53.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 10 monclient: _add_conns ranks=[1,2,6,0,7,3,8,5,4] 2024-01-24T23:24:53.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 10 monclient(hunting): picked mon.noname-f con 0x7fe4a40a4850 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:53.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 10 monclient(hunting): picked mon.noname-b con 0x7fe4a40a4c30 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:53.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 10 monclient(hunting): picked mon.noname-d con 0x7fe4a413ae10 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:53.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a4064700 con 0x7fe4a40a4850 2024-01-24T23:24:53.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a409fc40 con 0x7fe4a40a4c30 2024-01-24T23:24:53.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a4071fc0 con 0x7fe4a413ae10 2024-01-24T23:24:53.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:53.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 10 monclient(hunting): authenticate will time out at 1180.473694s 2024-01-24T23:24:53.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a3fff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fe4a40a4850 legacy=0x7fe4a40718b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:53.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a37fe640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fe4a40a4c30 0x7fe4a40a5050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:60036/0 (socket says 172.21.15.132:60036) 2024-01-24T23:24:53.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a37fe640 1 -- 172.21.15.132:0/4279993066 learned_addr learned my addr 172.21.15.132:0/4279993066 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:53.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 1 -- 172.21.15.132:0/4279993066 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1457061766 0 0) 0x7fe4a409fc40 con 0x7fe4a40a4c30 2024-01-24T23:24:53.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:53.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:53.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:53.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 10 monclient(hunting): my global_id is 85025 2024-01-24T23:24:53.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 1 -- 172.21.15.132:0/4279993066 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe48c003530 con 0x7fe4a40a4c30 2024-01-24T23:24:53.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 1 -- 172.21.15.132:0/4279993066 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3965922574 0 0) 0x7fe4a4071fc0 con 0x7fe4a413ae10 2024-01-24T23:24:53.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:53.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:53.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:53.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 10 monclient(hunting): my global_id is 5820 2024-01-24T23:24:53.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 1 -- 172.21.15.132:0/4279993066 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4a409fc40 con 0x7fe4a413ae10 2024-01-24T23:24:53.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 1 -- 172.21.15.132:0/4279993066 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3023234506 0 0) 0x7fe48c003530 con 0x7fe4a40a4c30 2024-01-24T23:24:53.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 1 -- 172.21.15.132:0/4279993066 >> v1:172.21.15.115:6792/0 conn(0x7fe4a413ae10 legacy=0x7fe4a413d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:53.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 1 -- 172.21.15.132:0/4279993066 >> v1:172.21.15.132:6789/0 conn(0x7fe4a40a4850 legacy=0x7fe4a40718b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:53.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:53.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 1 monclient: found mon.noname-b 2024-01-24T23:24:53.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:53.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:24:53.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 1 -- 172.21.15.132:0/4279993066 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a4071120 con 0x7fe4a40a4c30 2024-01-24T23:24:53.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 10 monclient: _finish_auth 0 2024-01-24T23:24:53.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:53.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:53.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 1 -- 172.21.15.132:0/4279993066 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe4980017c0 con 0x7fe4a40a4c30 2024-01-24T23:24:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 5 monclient: authenticate success, global_id 85025 2024-01-24T23:24:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:24:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 10 monclient: dump: 2024-01-24T23:24:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:53.159 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:53.159 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:53.159 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:53.159 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:53.159 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:53.159 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:53.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:53.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 1 -- 172.21.15.132:0/4279993066 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe498001aa0 con 0x7fe4a40a4c30 2024-01-24T23:24:53.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:53.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:53.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 4 set_mon_vals no callback set 2024-01-24T23:24:53.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 1 -- 172.21.15.132:0/4279993066 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe498004bf0 con 0x7fe4a40a4c30 2024-01-24T23:24:53.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:53.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:24:53.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4a2ffd640 10 monclient: dump: 2024-01-24T23:24:53.160 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:53.160 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:53.160 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:53.160 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:53.160 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:53.160 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:53.160 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:53.160 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:53.160 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:53.160 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:53.160 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:53.160 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:53.160 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:53.160 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:53.160 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:53.161 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:53.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 10 monclient: shutdown 2024-01-24T23:24:53.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:53.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 1 -- 172.21.15.132:0/4279993066 >> v1:172.21.15.115:6790/0 conn(0x7fe4a40a4c30 legacy=0x7fe4a40a5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:53.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 1 -- 172.21.15.132:0/4279993066 shutdown_connections 2024-01-24T23:24:53.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 1 -- 172.21.15.132:0/4279993066 >> 172.21.15.132:0/4279993066 conn(0x7fe4a40a5f40 msgr2=0x7fe4a40a2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:53.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.146+0000 7fe4aa747640 1 -- 172.21.15.132:0/4279993066 shutdown_connections 2024-01-24T23:24:53.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 1 -- 172.21.15.132:0/4279993066 wait complete. 2024-01-24T23:24:53.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 10 monclient: build_initial_monmap 2024-01-24T23:24:53.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 10 monclient: monmap: 2024-01-24T23:24:53.161 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:53.161 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:53.161 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:53.161 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:53.161 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:53.162 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:53.162 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:53.162 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:53.162 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:53.162 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:53.162 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:53.162 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:53.162 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:53.162 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:53.162 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:53.162 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:53.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 1 Processor -- start 2024-01-24T23:24:53.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 1 -- start start 2024-01-24T23:24:53.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 10 monclient: init 2024-01-24T23:24:53.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 10 monclient: _add_conns ranks=[0,5,1,8,7,2,4,3,6] 2024-01-24T23:24:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 10 monclient(hunting): picked mon.noname-a con 0x7fe4a40a4850 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 10 monclient(hunting): picked mon.noname-h con 0x7fe4a40a4c30 addr v1:172.21.15.132:6791/0 2024-01-24T23:24:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 10 monclient(hunting): picked mon.noname-f con 0x7fe4a413ae10 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a409c630 con 0x7fe4a40a4850 2024-01-24T23:24:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a406e020 con 0x7fe4a413ae10 2024-01-24T23:24:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a409e1a0 con 0x7fe4a40a4c30 2024-01-24T23:24:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 10 monclient(hunting): authenticate will time out at 1180.477512s 2024-01-24T23:24:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a8ce4640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fe4a413ae10 legacy=0x7fe4a41d11f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a3fff640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fe4a40a4850 legacy=0x7fe4a409a9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a37fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fe4a40a4c30 0x7fe4a409b0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:40602/0 (socket says 172.21.15.132:40602) 2024-01-24T23:24:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a37fe640 1 -- 172.21.15.132:0/3398598041 learned_addr learned my addr 172.21.15.132:0/3398598041 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 1 -- 172.21.15.132:0/3398598041 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3793443333 0 0) 0x7fe4a409e1a0 con 0x7fe4a40a4c30 2024-01-24T23:24:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:24:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:53.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:53.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 10 monclient(hunting): my global_id is 5684 2024-01-24T23:24:53.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 1 -- 172.21.15.132:0/3398598041 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe490003530 con 0x7fe4a40a4c30 2024-01-24T23:24:53.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 1 -- 172.21.15.132:0/3398598041 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1241811429 0 0) 0x7fe490003530 con 0x7fe4a40a4c30 2024-01-24T23:24:53.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 1 -- 172.21.15.132:0/3398598041 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe4a409e1a0 con 0x7fe4a40a4c30 2024-01-24T23:24:53.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 1 -- 172.21.15.132:0/3398598041 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe498004b30 con 0x7fe4a40a4c30 2024-01-24T23:24:53.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:53.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:24:53.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 10 monclient(hunting): dump: 2024-01-24T23:24:53.164 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:53.164 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:53.164 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:53.164 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:53.164 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:53.165 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 1 -- 172.21.15.132:0/3398598041 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3514519747 0 0) 0x7fe4a409e1a0 con 0x7fe4a40a4c30 2024-01-24T23:24:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 1 -- 172.21.15.132:0/3398598041 >> v1:172.21.15.132:6789/0 conn(0x7fe4a413ae10 legacy=0x7fe4a41d11f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 1 -- 172.21.15.132:0/3398598041 >> v1:172.21.15.115:6789/0 conn(0x7fe4a40a4850 legacy=0x7fe4a409a9d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:24:53.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 1 monclient: found mon.h 2024-01-24T23:24:53.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:53.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:53.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 1 -- 172.21.15.132:0/3398598041 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a409b7f0 con 0x7fe4a40a4c30 2024-01-24T23:24:53.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 10 monclient: _finish_auth 0 2024-01-24T23:24:53.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:24:53.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:53.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 5 monclient: authenticate success, global_id 5684 2024-01-24T23:24:53.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 10 monclient: _renew_subs 2024-01-24T23:24:53.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:53.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 1 -- 172.21.15.132:0/3398598041 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a409e480 con 0x7fe4a40a4c30 2024-01-24T23:24:53.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 10 monclient: _renew_subs 2024-01-24T23:24:53.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:53.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 1 -- 172.21.15.132:0/3398598041 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a409dec0 con 0x7fe4a40a4c30 2024-01-24T23:24:53.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 1 -- 172.21.15.132:0/3398598041 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe498004ec0 con 0x7fe4a40a4c30 2024-01-24T23:24:53.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:53.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 1 -- 172.21.15.132:0/3398598041 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe498005370 con 0x7fe4a40a4c30 2024-01-24T23:24:53.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:53.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:24:53.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a1ffb640 4 set_mon_vals no callback set 2024-01-24T23:24:53.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 10 monclient: dump: 2024-01-24T23:24:53.167 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:53.167 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:53.167 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:53.167 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:53.167 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:53.167 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:53.167 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:53.167 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:53.167 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:53.167 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:53.168 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:53.168 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:53.168 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:53.168 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:53.168 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:53.168 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:53.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:53.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:53.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:53.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4aa747640 1 -- 172.21.15.132:0/3398598041 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe484005090 con 0x7fe4a40a4c30 2024-01-24T23:24:53.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 1 -- 172.21.15.132:0/3398598041 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fe49800fdf0 con 0x7fe4a40a4c30 2024-01-24T23:24:53.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a0ff9640 1 -- 172.21.15.132:0/3398598041 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(114..114 src has 1..114) v4 ==== 5687+0+0 (unknown 2688723034 0 0) 0x7fe4980441d0 con 0x7fe4a40a4c30 2024-01-24T23:24:53.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.150+0000 7fe4a3fff640 10 monclient: get_auth_request con 0x7fe490036f40 auth_method 0 2024-01-24T23:24:53.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.154+0000 7fe4a0ff9640 1 -- 172.21.15.132:0/3398598041 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fe498014bc0 con 0x7fe4a40a4c30 2024-01-24T23:24:53.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.154+0000 7fe4a0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:53.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.154+0000 7fe4a0ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:53.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.318+0000 7fe4aa747640 10 start_mon_command cmd=[{"prefix": "osd set-require-min-compat-client", "version": "luminous"}] 2024-01-24T23:24:53.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.318+0000 7fe4aa747640 10 monclient: _send_command 2 [{"prefix": "osd set-require-min-compat-client", "version": "luminous"}] 2024-01-24T23:24:53.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.318+0000 7fe4aa747640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:53.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.318+0000 7fe4aa747640 1 -- 172.21.15.132:0/3398598041 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd set-require-min-compat-client", "version": "luminous"} v 0) v1 -- 0x7fe4840058c0 con 0x7fe4a40a4c30 2024-01-24T23:24:53.962 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.958+0000 7fe4a0ff9640 1 -- 172.21.15.132:0/3398598041 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd set-require-min-compat-client", "version": "luminous"}]=0 set require_min_compat_client to luminous v115) v1 ==== 145+0+0 (unknown 812925530 0 0) 0x7fe498013260 con 0x7fe4a40a4c30 2024-01-24T23:24:53.962 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.958+0000 7fe4a0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd set-require-min-compat-client", "version": "luminous"}] 2024-01-24T23:24:53.962 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:53.958+0000 7fe4a0ff9640 10 monclient: _finish_command 2 = system:0 set require_min_compat_client to luminous 2024-01-24T23:24:54.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:54.062+0000 7fe4aa747640 10 start_mon_command cmd=[{"prefix": "osd set-require-min-compat-client", "version": "luminous"}] 2024-01-24T23:24:54.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:54.062+0000 7fe4aa747640 10 monclient: _send_command 3 [{"prefix": "osd set-require-min-compat-client", "version": "luminous"}] 2024-01-24T23:24:54.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:54.062+0000 7fe4aa747640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:24:54.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:54.062+0000 7fe4aa747640 1 -- 172.21.15.132:0/3398598041 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd set-require-min-compat-client", "version": "luminous"} v 0) v1 -- 0x7fe484000bc0 con 0x7fe4a40a4c30 2024-01-24T23:24:54.968 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:54.965+0000 7fe4a0ff9640 1 -- 172.21.15.132:0/3398598041 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd set-require-min-compat-client", "version": "luminous"}]=0 set require_min_compat_client to luminous v116) v1 ==== 145+0+0 (unknown 1220183649 0 0) 0x7fe4980149e0 con 0x7fe4a40a4c30 2024-01-24T23:24:54.968 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:54.965+0000 7fe4a0ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd set-require-min-compat-client", "version": "luminous"}] 2024-01-24T23:24:54.968 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:54.965+0000 7fe4a0ff9640 10 monclient: _finish_command 3 = system:0 set require_min_compat_client to luminous 2024-01-24T23:24:54.968 INFO:tasks.workunit.client.0.smithi132.stderr:set require_min_compat_client to luminous 2024-01-24T23:24:54.972 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:54.969+0000 7fe4aa747640 1 -- 172.21.15.132:0/3398598041 >> v1:172.21.15.132:6812/1835401922 conn(0x7fe490036f40 legacy=0x7fe490039400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:54.972 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:54.969+0000 7fe4aa747640 10 monclient: shutdown 2024-01-24T23:24:54.972 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:54.969+0000 7fe4aa747640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:54.972 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:54.969+0000 7fe4aa747640 1 -- 172.21.15.132:0/3398598041 >> v1:172.21.15.132:6791/0 conn(0x7fe4a40a4c30 legacy=0x7fe4a409b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:54.973 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:54.969+0000 7fe4aa747640 1 -- 172.21.15.132:0/3398598041 shutdown_connections 2024-01-24T23:24:54.973 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:54.969+0000 7fe4aa747640 1 -- 172.21.15.132:0/3398598041 >> 172.21.15.132:0/3398598041 conn(0x7fe4a40a5f40 msgr2=0x7fe4a4128150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:54.973 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:54.969+0000 7fe4aa747640 1 -- 172.21.15.132:0/3398598041 shutdown_connections 2024-01-24T23:24:54.973 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:54.969+0000 7fe4aa747640 1 -- 172.21.15.132:0/3398598041 wait complete. 2024-01-24T23:24:54.994 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set dump 2024-01-24T23:24:55.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.105+0000 7fac67390640 10 monclient: get_monmap_and_config 2024-01-24T23:24:55.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.105+0000 7fac67390640 10 monclient: build_initial_monmap 2024-01-24T23:24:55.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.105+0000 7fac67390640 10 monclient: monmap: 2024-01-24T23:24:55.110 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:55.110 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:55.110 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:55.111760+0000 2024-01-24T23:24:55.111 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:55.111760+0000 2024-01-24T23:24:55.111 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:55.111 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:55.111 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:55.111 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:55.111 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:55.111 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:55.111 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:55.111 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:55.111 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:55.111 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:55.111 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:55.111 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:55.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 1 Processor -- start 2024-01-24T23:24:55.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 1 -- start start 2024-01-24T23:24:55.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 10 monclient: init 2024-01-24T23:24:55.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:55.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 10 monclient: _add_conns ranks=[7,6,2,1,3,0,4,8,5] 2024-01-24T23:24:55.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 10 monclient(hunting): picked mon.noname-i con 0x7fac60135e10 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:55.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 10 monclient(hunting): picked mon.noname-d con 0x7fac601361f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:55.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 10 monclient(hunting): picked mon.noname-b con 0x7fac601365f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:55.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac60064700 con 0x7fac601365f0 2024-01-24T23:24:55.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac6009c3e0 con 0x7fac601361f0 2024-01-24T23:24:55.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac600a4f90 con 0x7fac60135e10 2024-01-24T23:24:55.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:55.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 10 monclient(hunting): authenticate will time out at 1182.437081s 2024-01-24T23:24:55.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac6512c640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fac60135e10 0x7fac6009bcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:60450/0 (socket says 172.21.15.132:60450) 2024-01-24T23:24:55.115 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac6512c640 1 -- 172.21.15.132:0/3047648272 learned_addr learned my addr 172.21.15.132:0/3047648272 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:55.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 1 -- 172.21.15.132:0/3047648272 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2578496685 0 0) 0x7fac600a4f90 con 0x7fac60135e10 2024-01-24T23:24:55.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:55.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:55.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:55.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 10 monclient(hunting): my global_id is 5713 2024-01-24T23:24:55.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 1 -- 172.21.15.132:0/3047648272 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac4c003530 con 0x7fac60135e10 2024-01-24T23:24:55.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 1 -- 172.21.15.132:0/3047648272 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3771817847 0 0) 0x7fac4c003530 con 0x7fac60135e10 2024-01-24T23:24:55.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 1 -- 172.21.15.132:0/3047648272 >> v1:172.21.15.115:6792/0 conn(0x7fac601361f0 legacy=0x7fac6013bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:55.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 1 -- 172.21.15.132:0/3047648272 >> v1:172.21.15.115:6790/0 conn(0x7fac601365f0 legacy=0x7fac6013f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:55.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 10 monclient: _finish_hunting 0 2024-01-24T23:24:55.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 1 monclient: found mon.noname-i 2024-01-24T23:24:55.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:55.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:24:55.116 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 1 -- 172.21.15.132:0/3047648272 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6009b010 con 0x7fac60135e10 2024-01-24T23:24:55.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 10 monclient: _finish_auth 0 2024-01-24T23:24:55.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 10 monclient: _check_auth_tickets 2024-01-24T23:24:55.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:55.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 5 monclient: authenticate success, global_id 5713 2024-01-24T23:24:55.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:55.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 1 -- 172.21.15.132:0/3047648272 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fac5c0017c0 con 0x7fac60135e10 2024-01-24T23:24:55.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:55.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:55.117 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 10 monclient: dump: 2024-01-24T23:24:55.117 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:55.117 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:55.118 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.118 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.118 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:55.118 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:55.118 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:55.118 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:55.118 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:55.118 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:55.118 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:55.118 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:55.118 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:55.118 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:55.118 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:55.118 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:55.118 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:55.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 1 -- 172.21.15.132:0/3047648272 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fac5c001a70 con 0x7fac60135e10 2024-01-24T23:24:55.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:55.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 1 -- 172.21.15.132:0/3047648272 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fac5c004bf0 con 0x7fac60135e10 2024-01-24T23:24:55.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:55.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:55.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac57fff640 10 monclient: dump: 2024-01-24T23:24:55.119 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:55.119 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:55.119 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.119 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.119 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:55.119 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:55.119 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:55.119 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:55.119 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:55.120 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:55.120 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:55.120 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:55.120 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:55.120 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:55.120 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:55.120 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:55.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:55.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 4 set_mon_vals no callback set 2024-01-24T23:24:55.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 10 monclient: shutdown 2024-01-24T23:24:55.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:55.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 1 -- 172.21.15.132:0/3047648272 >> v1:172.21.15.132:6792/0 conn(0x7fac60135e10 legacy=0x7fac6009bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:55.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 1 -- 172.21.15.132:0/3047648272 shutdown_connections 2024-01-24T23:24:55.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 1 -- 172.21.15.132:0/3047648272 >> 172.21.15.132:0/3047648272 conn(0x7fac600a6790 msgr2=0x7fac60071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:55.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 1 -- 172.21.15.132:0/3047648272 shutdown_connections 2024-01-24T23:24:55.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 1 -- 172.21.15.132:0/3047648272 wait complete. 2024-01-24T23:24:55.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 10 monclient: build_initial_monmap 2024-01-24T23:24:55.121 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 10 monclient: monmap: 2024-01-24T23:24:55.121 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:55.121 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:55.121 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:55.121 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:55.121 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:55.121 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:55.121 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:55.121 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:55.121 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:55.121 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:55.121 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:55.122 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:55.122 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:55.122 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:55.122 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:55.122 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:55.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 1 Processor -- start 2024-01-24T23:24:55.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 1 -- start start 2024-01-24T23:24:55.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.109+0000 7fac67390640 10 monclient: init 2024-01-24T23:24:55.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac67390640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:55.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac67390640 10 monclient: _add_conns ranks=[7,3,0,4,8,1,5,2,6] 2024-01-24T23:24:55.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac67390640 10 monclient(hunting): picked mon.noname-i con 0x7fac60135e10 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:55.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac67390640 10 monclient(hunting): picked mon.noname-g con 0x7fac601361f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:55.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac67390640 10 monclient(hunting): picked mon.noname-a con 0x7fac601365f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:55.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac67390640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac6009fe30 con 0x7fac601365f0 2024-01-24T23:24:55.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac67390640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac60120b50 con 0x7fac601361f0 2024-01-24T23:24:55.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac67390640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac6009ca00 con 0x7fac60135e10 2024-01-24T23:24:55.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac67390640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:55.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac67390640 10 monclient(hunting): authenticate will time out at 1182.440272s 2024-01-24T23:24:55.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac6512c640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fac60135e10 0x7fac60134d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:60460/0 (socket says 172.21.15.132:60460) 2024-01-24T23:24:55.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac6512c640 1 -- 172.21.15.132:0/2603735903 learned_addr learned my addr 172.21.15.132:0/2603735903 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:55.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac6592d640 1 -- 172.21.15.132:0/2603735903 >> v1:172.21.15.115:6789/0 conn(0x7fac601365f0 legacy=0x7fac601d3cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:55.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 1 -- 172.21.15.132:0/2603735903 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 537912969 0 0) 0x7fac6009ca00 con 0x7fac60135e10 2024-01-24T23:24:55.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:55.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:55.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:55.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 10 monclient(hunting): my global_id is 5722 2024-01-24T23:24:55.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 1 -- 172.21.15.132:0/2603735903 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac44003530 con 0x7fac60135e10 2024-01-24T23:24:55.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 1 -- 172.21.15.132:0/2603735903 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4154266137 0 0) 0x7fac60120b50 con 0x7fac601361f0 2024-01-24T23:24:55.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:55.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:55.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:55.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 10 monclient(hunting): my global_id is 5736 2024-01-24T23:24:55.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 1 -- 172.21.15.132:0/2603735903 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac6009ca00 con 0x7fac601361f0 2024-01-24T23:24:55.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 1 -- 172.21.15.132:0/2603735903 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3844905161 0 0) 0x7fac44003530 con 0x7fac60135e10 2024-01-24T23:24:55.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 1 -- 172.21.15.132:0/2603735903 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fac60120b50 con 0x7fac60135e10 2024-01-24T23:24:55.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 1 -- 172.21.15.132:0/2603735903 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fac5c004b30 con 0x7fac60135e10 2024-01-24T23:24:55.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:55.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:55.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 10 monclient(hunting): dump: 2024-01-24T23:24:55.128 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:55.128 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:55.128 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.128 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.128 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:55.129 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:55.129 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:55.129 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:55.129 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:55.129 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:55.129 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:55.129 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:55.129 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:55.129 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:55.129 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:55.129 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:55.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:55.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 1 -- 172.21.15.132:0/2603735903 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 508332137 0 0) 0x7fac6009ca00 con 0x7fac601361f0 2024-01-24T23:24:55.129 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 1 -- 172.21.15.132:0/2603735903 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fac44003530 con 0x7fac601361f0 2024-01-24T23:24:55.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 1 -- 172.21.15.132:0/2603735903 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fac50001950 con 0x7fac601361f0 2024-01-24T23:24:55.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:55.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:55.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 10 monclient(hunting): dump: 2024-01-24T23:24:55.130 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:55.130 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:55.130 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.130 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.130 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:55.130 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:55.130 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:55.130 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:55.130 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:55.130 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:55.130 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:55.131 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:55.131 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:55.131 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:55.131 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:55.131 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:55.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 1 -- 172.21.15.132:0/2603735903 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4132105342 0 0) 0x7fac60120b50 con 0x7fac60135e10 2024-01-24T23:24:55.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 1 -- 172.21.15.132:0/2603735903 >> v1:172.21.15.132:6790/0 conn(0x7fac601361f0 legacy=0x7fac601d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:55.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 1 -- 172.21.15.132:0/2603735903 >> v1:172.21.15.115:6789/0 conn(0x7fac601365f0 legacy=0x7fac601d3cf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:55.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:24:55.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 1 monclient: found mon.i 2024-01-24T23:24:55.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:55.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:55.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 1 -- 172.21.15.132:0/2603735903 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6009c6b0 con 0x7fac60135e10 2024-01-24T23:24:55.131 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 10 monclient: _finish_auth 0 2024-01-24T23:24:55.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:24:55.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:55.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac67390640 5 monclient: authenticate success, global_id 5722 2024-01-24T23:24:55.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac67390640 10 monclient: _renew_subs 2024-01-24T23:24:55.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac67390640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:55.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac67390640 1 -- 172.21.15.132:0/2603735903 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac60121e70 con 0x7fac60135e10 2024-01-24T23:24:55.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac67390640 10 monclient: _renew_subs 2024-01-24T23:24:55.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac67390640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:55.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac67390640 1 -- 172.21.15.132:0/2603735903 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fac60120f80 con 0x7fac60135e10 2024-01-24T23:24:55.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 1 -- 172.21.15.132:0/2603735903 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fac5c001200 con 0x7fac60135e10 2024-01-24T23:24:55.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:55.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 1 -- 172.21.15.132:0/2603735903 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fac5c0051f0 con 0x7fac60135e10 2024-01-24T23:24:55.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:55.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac577fe640 4 set_mon_vals no callback set 2024-01-24T23:24:55.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:55.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 10 monclient: dump: 2024-01-24T23:24:55.133 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:55.133 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:55.133 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.133 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.133 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:55.133 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:55.133 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:55.133 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:55.133 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:55.133 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:55.133 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:55.133 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:55.133 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:55.133 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:55.134 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:55.134 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:55.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 1 -- 172.21.15.132:0/2603735903 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fac5c00fc70 con 0x7fac60135e10 2024-01-24T23:24:55.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac67390640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:55.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac67390640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:55.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac67390640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:55.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac55ffb640 1 -- 172.21.15.132:0/2603735903 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(116..116 src has 1..116) v4 ==== 5687+0+0 (unknown 3195493297 0 0) 0x7fac5c0441c0 con 0x7fac60135e10 2024-01-24T23:24:55.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac67390640 1 -- 172.21.15.132:0/2603735903 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac40005090 con 0x7fac60135e10 2024-01-24T23:24:55.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.113+0000 7fac6492b640 10 monclient: get_auth_request con 0x7fac44037610 auth_method 0 2024-01-24T23:24:55.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.117+0000 7fac55ffb640 1 -- 172.21.15.132:0/2603735903 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fac5c0111c0 con 0x7fac60135e10 2024-01-24T23:24:55.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.117+0000 7fac55ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:55.134 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.117+0000 7fac55ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:55.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.281+0000 7fac67390640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set dump"}] 2024-01-24T23:24:55.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.281+0000 7fac67390640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set dump"}] 2024-01-24T23:24:55.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.281+0000 7fac67390640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:55.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.281+0000 7fac67390640 1 -- 172.21.15.132:0/2603735903 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush weight-set dump"} v 0) v1 -- 0x7fac400058c0 con 0x7fac60135e10 2024-01-24T23:24:55.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.281+0000 7fac55ffb640 1 -- 172.21.15.132:0/2603735903 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set dump"}]=0 v116) v1 ==== 73+0+3 (unknown 502577673 0 2423995997) 0x7fac5c013240 con 0x7fac60135e10 2024-01-24T23:24:55.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.281+0000 7fac55ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set dump"}] 2024-01-24T23:24:55.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.281+0000 7fac55ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:24:55.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.385+0000 7fac67390640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set dump"}] 2024-01-24T23:24:55.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.385+0000 7fac67390640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set dump"}] 2024-01-24T23:24:55.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.385+0000 7fac67390640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:24:55.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.385+0000 7fac67390640 1 -- 172.21.15.132:0/2603735903 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush weight-set dump"} v 0) v1 -- 0x7fac40005380 con 0x7fac60135e10 2024-01-24T23:24:55.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.385+0000 7fac55ffb640 1 -- 172.21.15.132:0/2603735903 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set dump"}]=0 v116) v1 ==== 73+0+3 (unknown 502577673 0 2423995997) 0x7fac5c0149c0 con 0x7fac60135e10 2024-01-24T23:24:55.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.385+0000 7fac55ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set dump"}] 2024-01-24T23:24:55.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.385+0000 7fac55ffb640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:24:55.388 INFO:tasks.workunit.client.0.smithi132.stdout:{} 2024-01-24T23:24:55.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.385+0000 7fac67390640 1 -- 172.21.15.132:0/2603735903 >> v1:172.21.15.132:6812/1835401922 conn(0x7fac44037610 legacy=0x7fac44039ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:55.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.385+0000 7fac67390640 10 monclient: shutdown 2024-01-24T23:24:55.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.385+0000 7fac67390640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:55.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.385+0000 7fac67390640 1 -- 172.21.15.132:0/2603735903 >> v1:172.21.15.132:6792/0 conn(0x7fac60135e10 legacy=0x7fac60134d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:55.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.385+0000 7fac67390640 1 -- 172.21.15.132:0/2603735903 shutdown_connections 2024-01-24T23:24:55.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.385+0000 7fac67390640 1 -- 172.21.15.132:0/2603735903 >> 172.21.15.132:0/2603735903 conn(0x7fac600a6790 msgr2=0x7fac600a85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:55.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.385+0000 7fac67390640 1 -- 172.21.15.132:0/2603735903 shutdown_connections 2024-01-24T23:24:55.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.385+0000 7fac67390640 1 -- 172.21.15.132:0/2603735903 wait complete. 2024-01-24T23:24:55.404 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set ls 2024-01-24T23:24:55.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.517+0000 7f0e7e317640 10 monclient: get_monmap_and_config 2024-01-24T23:24:55.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.517+0000 7f0e7e317640 10 monclient: build_initial_monmap 2024-01-24T23:24:55.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.517+0000 7f0e7e317640 10 monclient: monmap: 2024-01-24T23:24:55.522 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:55.522 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:55.522 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:55.523238+0000 2024-01-24T23:24:55.522 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:55.523238+0000 2024-01-24T23:24:55.522 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:55.522 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:55.522 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:55.522 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:55.522 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:55.522 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:55.522 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:55.523 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:55.523 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:55.523 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:55.523 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:55.523 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:55.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.517+0000 7f0e7e317640 1 Processor -- start 2024-01-24T23:24:55.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.517+0000 7f0e7e317640 1 -- start start 2024-01-24T23:24:55.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.517+0000 7f0e7e317640 10 monclient: init 2024-01-24T23:24:55.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7e317640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:55.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7e317640 10 monclient: _add_conns ranks=[2,0,6,5,4,7,3,1,8] 2024-01-24T23:24:55.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7e317640 10 monclient(hunting): picked mon.noname-b con 0x7f0e7809c4c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:55.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7e317640 10 monclient(hunting): picked mon.noname-a con 0x7f0e7809cfb0 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:55.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7e317640 10 monclient(hunting): picked mon.noname-d con 0x7f0e7809ec50 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:55.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7e317640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e78064700 con 0x7f0e7809cfb0 2024-01-24T23:24:55.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7e317640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7809d9f0 con 0x7f0e7809c4c0 2024-01-24T23:24:55.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7e317640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7809dc60 con 0x7f0e7809ec50 2024-01-24T23:24:55.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7e317640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:55.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7e317640 10 monclient(hunting): authenticate will time out at 1182.848663s 2024-01-24T23:24:55.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e777fe640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f0e7809cfb0 legacy=0x7f0e78138d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:55.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7c8b4640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f0e7809ec50 0x7f0e7813c460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:57376/0 (socket says 172.21.15.132:57376) 2024-01-24T23:24:55.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7c8b4640 1 -- 172.21.15.132:0/1996730508 learned_addr learned my addr 172.21.15.132:0/1996730508 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:55.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e77fff640 1 --1- 172.21.15.132:0/1996730508 >> v1:172.21.15.115:6790/0 conn(0x7f0e7809c4c0 0x7f0e7809c8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:59590/0 (socket says 172.21.15.132:59590) 2024-01-24T23:24:55.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 1 -- 172.21.15.132:0/1996730508 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1684540614 0 0) 0x7f0e7809dc60 con 0x7f0e7809ec50 2024-01-24T23:24:55.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:55.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:55.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:55.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 10 monclient(hunting): my global_id is 5838 2024-01-24T23:24:55.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 1 -- 172.21.15.132:0/1996730508 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e60003530 con 0x7f0e7809ec50 2024-01-24T23:24:55.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 1 -- 172.21.15.132:0/1996730508 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1066054552 0 0) 0x7f0e7809d9f0 con 0x7f0e7809c4c0 2024-01-24T23:24:55.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:55.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:55.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:55.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 10 monclient(hunting): my global_id is 85043 2024-01-24T23:24:55.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 1 -- 172.21.15.132:0/1996730508 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e7809dc60 con 0x7f0e7809c4c0 2024-01-24T23:24:55.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 1 -- 172.21.15.132:0/1996730508 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 95006109 0 0) 0x7f0e7809dc60 con 0x7f0e7809c4c0 2024-01-24T23:24:55.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 1 -- 172.21.15.132:0/1996730508 >> v1:172.21.15.115:6792/0 conn(0x7f0e7809ec50 legacy=0x7f0e7813c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:55.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 1 -- 172.21.15.132:0/1996730508 >> v1:172.21.15.115:6789/0 conn(0x7f0e7809cfb0 legacy=0x7f0e78138d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:55.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:55.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 1 monclient: found mon.noname-b 2024-01-24T23:24:55.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:55.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:24:55.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 1 -- 172.21.15.132:0/1996730508 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7809df40 con 0x7f0e7809c4c0 2024-01-24T23:24:55.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 10 monclient: _finish_auth 0 2024-01-24T23:24:55.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:55.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:55.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 1 -- 172.21.15.132:0/1996730508 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0e68001650 con 0x7f0e7809c4c0 2024-01-24T23:24:55.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:55.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:24:55.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 10 monclient: dump: 2024-01-24T23:24:55.533 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:55.533 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:55.533 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.533 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.533 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:55.533 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:55.533 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:55.534 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7e317640 5 monclient: authenticate success, global_id 85043 2024-01-24T23:24:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7e317640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 1 -- 172.21.15.132:0/1996730508 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0e68001870 con 0x7f0e7809c4c0 2024-01-24T23:24:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7e317640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7e317640 4 set_mon_vals no callback set 2024-01-24T23:24:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 1 -- 172.21.15.132:0/1996730508 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0e680047f0 con 0x7f0e7809c4c0 2024-01-24T23:24:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:24:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e76ffd640 10 monclient: dump: 2024-01-24T23:24:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:55.536 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7e317640 10 monclient: shutdown 2024-01-24T23:24:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7e317640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7e317640 1 -- 172.21.15.132:0/1996730508 >> v1:172.21.15.115:6790/0 conn(0x7f0e7809c4c0 legacy=0x7f0e7809c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7e317640 1 -- 172.21.15.132:0/1996730508 shutdown_connections 2024-01-24T23:24:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7e317640 1 -- 172.21.15.132:0/1996730508 >> 172.21.15.132:0/1996730508 conn(0x7f0e780a5a10 msgr2=0x7f0e780a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7e317640 1 -- 172.21.15.132:0/1996730508 shutdown_connections 2024-01-24T23:24:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7e317640 1 -- 172.21.15.132:0/1996730508 wait complete. 2024-01-24T23:24:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7e317640 10 monclient: build_initial_monmap 2024-01-24T23:24:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7e317640 10 monclient: monmap: 2024-01-24T23:24:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:55.538 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.521+0000 7f0e7e317640 1 Processor -- start 2024-01-24T23:24:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e7e317640 1 -- start start 2024-01-24T23:24:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e7e317640 10 monclient: init 2024-01-24T23:24:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e7e317640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e7e317640 10 monclient: _add_conns ranks=[7,1,3,2,0,6,5,4,8] 2024-01-24T23:24:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e7e317640 10 monclient(hunting): picked mon.noname-i con 0x7f0e7809c4c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e7e317640 10 monclient(hunting): picked mon.noname-f con 0x7f0e7809cfb0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e7e317640 10 monclient(hunting): picked mon.noname-g con 0x7f0e7809ec50 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e7e317640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7809c020 con 0x7f0e7809cfb0 2024-01-24T23:24:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e7e317640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e78125de0 con 0x7f0e7809ec50 2024-01-24T23:24:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e777fe640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f0e7809cfb0 legacy=0x7f0e7806d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e7e317640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e781263a0 con 0x7f0e7809c4c0 2024-01-24T23:24:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e7e317640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e7e317640 10 monclient(hunting): authenticate will time out at 1182.852647s 2024-01-24T23:24:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e77fff640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f0e7809c4c0 0x7f0e7806ce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:60472/0 (socket says 172.21.15.132:60472) 2024-01-24T23:24:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e77fff640 1 -- 172.21.15.132:0/2171370501 learned_addr learned my addr 172.21.15.132:0/2171370501 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 1 -- 172.21.15.132:0/2171370501 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2878286917 0 0) 0x7f0e781263a0 con 0x7f0e7809c4c0 2024-01-24T23:24:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:24:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 10 monclient(hunting): my global_id is 5731 2024-01-24T23:24:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 1 -- 172.21.15.132:0/2171370501 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e64003560 con 0x7f0e7809c4c0 2024-01-24T23:24:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 1 -- 172.21.15.132:0/2171370501 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1151358474 0 0) 0x7f0e78125de0 con 0x7f0e7809ec50 2024-01-24T23:24:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:24:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 10 monclient(hunting): my global_id is 5745 2024-01-24T23:24:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 1 -- 172.21.15.132:0/2171370501 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e781263a0 con 0x7f0e7809ec50 2024-01-24T23:24:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 1 -- 172.21.15.132:0/2171370501 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 655917585 0 0) 0x7f0e781263a0 con 0x7f0e7809ec50 2024-01-24T23:24:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 1 -- 172.21.15.132:0/2171370501 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0e78125de0 con 0x7f0e7809ec50 2024-01-24T23:24:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 1 -- 172.21.15.132:0/2171370501 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 830784404 0 0) 0x7f0e64003560 con 0x7f0e7809c4c0 2024-01-24T23:24:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 1 -- 172.21.15.132:0/2171370501 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0e781263a0 con 0x7f0e7809c4c0 2024-01-24T23:24:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 1 -- 172.21.15.132:0/2171370501 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0e70001d40 con 0x7f0e7809ec50 2024-01-24T23:24:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 10 monclient(hunting): dump: 2024-01-24T23:24:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:55.541 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 1 -- 172.21.15.132:0/2171370501 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0e68001550 con 0x7f0e7809c4c0 2024-01-24T23:24:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 10 monclient(hunting): dump: 2024-01-24T23:24:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:55.543 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 1 -- 172.21.15.132:0/2171370501 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4088322319 0 0) 0x7f0e78125de0 con 0x7f0e7809ec50 2024-01-24T23:24:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 1 -- 172.21.15.132:0/2171370501 >> v1:172.21.15.132:6792/0 conn(0x7f0e7809c4c0 legacy=0x7f0e7806ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 1 -- 172.21.15.132:0/2171370501 >> v1:172.21.15.132:6789/0 conn(0x7f0e7809cfb0 legacy=0x7f0e7806d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:24:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 1 monclient: found mon.g 2024-01-24T23:24:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 1 -- 172.21.15.132:0/2171370501 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e78134b00 con 0x7f0e7809ec50 2024-01-24T23:24:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 10 monclient: _finish_auth 0 2024-01-24T23:24:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:24:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e7e317640 5 monclient: authenticate success, global_id 5745 2024-01-24T23:24:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e7e317640 10 monclient: _renew_subs 2024-01-24T23:24:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e7e317640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e7e317640 1 -- 172.21.15.132:0/2171370501 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7806ead0 con 0x7f0e7809ec50 2024-01-24T23:24:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e7e317640 10 monclient: _renew_subs 2024-01-24T23:24:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e7e317640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e7e317640 1 -- 172.21.15.132:0/2171370501 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e78125b00 con 0x7f0e7809ec50 2024-01-24T23:24:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 1 -- 172.21.15.132:0/2171370501 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0e700020d0 con 0x7f0e7809ec50 2024-01-24T23:24:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 1 -- 172.21.15.132:0/2171370501 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0e70005160 con 0x7f0e7809ec50 2024-01-24T23:24:55.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:55.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e767fc640 4 set_mon_vals no callback set 2024-01-24T23:24:55.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:55.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 10 monclient: dump: 2024-01-24T23:24:55.545 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:55.545 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:55.545 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.545 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.545 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:55.545 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:55.545 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:55.545 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:55.545 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:55.545 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:55.545 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:55.545 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:55.545 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:55.546 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:55.546 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:55.546 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:55.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e7e317640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:55.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e7e317640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:55.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e7e317640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:55.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 1 -- 172.21.15.132:0/2171370501 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f0e7000fbe0 con 0x7f0e7809ec50 2024-01-24T23:24:55.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e7e317640 1 -- 172.21.15.132:0/2171370501 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e58005090 con 0x7f0e7809ec50 2024-01-24T23:24:55.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e74ff9640 1 -- 172.21.15.132:0/2171370501 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(116..116 src has 1..116) v4 ==== 5687+0+0 (unknown 3195493297 0 0) 0x7f0e70044240 con 0x7f0e7809ec50 2024-01-24T23:24:55.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.525+0000 7f0e77fff640 10 monclient: get_auth_request con 0x7f0e640376a0 auth_method 0 2024-01-24T23:24:55.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.529+0000 7f0e74ff9640 1 -- 172.21.15.132:0/2171370501 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f0e70014b00 con 0x7f0e7809ec50 2024-01-24T23:24:55.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.529+0000 7f0e74ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:55.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.529+0000 7f0e74ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:55.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.689+0000 7f0e7e317640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:24:55.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.689+0000 7f0e7e317640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:24:55.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.689+0000 7f0e7e317640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:55.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.689+0000 7f0e7e317640 1 -- 172.21.15.132:0/2171370501 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7f0e580058c0 con 0x7f0e7809ec50 2024-01-24T23:24:55.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.693+0000 7f0e74ff9640 1 -- 172.21.15.132:0/2171370501 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v116) v1 ==== 71+0+0 (unknown 257016397 0 0) 0x7f0e700131a0 con 0x7f0e7809ec50 2024-01-24T23:24:55.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.693+0000 7f0e74ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:24:55.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.693+0000 7f0e74ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:24:55.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.789+0000 7f0e7e317640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:24:55.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.789+0000 7f0e7e317640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:24:55.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.789+0000 7f0e7e317640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:55.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.789+0000 7f0e7e317640 1 -- 172.21.15.132:0/2171370501 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7f0e58000bd0 con 0x7f0e7809ec50 2024-01-24T23:24:55.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.789+0000 7f0e74ff9640 1 -- 172.21.15.132:0/2171370501 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v116) v1 ==== 71+0+0 (unknown 257016397 0 0) 0x7f0e70014920 con 0x7f0e7809ec50 2024-01-24T23:24:55.795 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.789+0000 7f0e74ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:24:55.795 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.789+0000 7f0e74ff9640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:24:55.797 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.793+0000 7f0e7e317640 1 -- 172.21.15.132:0/2171370501 >> v1:172.21.15.132:6812/1835401922 conn(0x7f0e640376a0 legacy=0x7f0e64039b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:55.797 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.793+0000 7f0e7e317640 10 monclient: shutdown 2024-01-24T23:24:55.797 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.793+0000 7f0e7e317640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:55.797 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.793+0000 7f0e7e317640 1 -- 172.21.15.132:0/2171370501 >> v1:172.21.15.132:6790/0 conn(0x7f0e7809ec50 legacy=0x7f0e781d3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:55.797 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.793+0000 7f0e7e317640 1 -- 172.21.15.132:0/2171370501 shutdown_connections 2024-01-24T23:24:55.797 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.793+0000 7f0e7e317640 1 -- 172.21.15.132:0/2171370501 >> 172.21.15.132:0/2171370501 conn(0x7f0e780a5a10 msgr2=0x7f0e78071ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:55.797 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.793+0000 7f0e7e317640 1 -- 172.21.15.132:0/2171370501 shutdown_connections 2024-01-24T23:24:55.797 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.793+0000 7f0e7e317640 1 -- 172.21.15.132:0/2171370501 wait complete. 2024-01-24T23:24:55.811 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd crush weight-set reweight fooset osd.0 .9 2024-01-24T23:24:55.812 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:24:55.812 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set reweight fooset osd.0 .9 2024-01-24T23:24:55.929 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.925+0000 7f85a5751640 10 monclient: get_monmap_and_config 2024-01-24T23:24:55.929 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.925+0000 7f85a5751640 10 monclient: build_initial_monmap 2024-01-24T23:24:55.929 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.925+0000 7f85a5751640 10 monclient: monmap: 2024-01-24T23:24:55.929 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:55.929 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:55.929 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:55.930734+0000 2024-01-24T23:24:55.929 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:55.930734+0000 2024-01-24T23:24:55.930 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:55.930 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:55.930 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:55.930 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:55.930 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:55.930 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:55.930 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:55.930 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:55.930 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:55.930 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:55.930 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:55.930 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:55.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.925+0000 7f85a5751640 1 Processor -- start 2024-01-24T23:24:55.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.925+0000 7f85a5751640 1 -- start start 2024-01-24T23:24:55.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.925+0000 7f85a5751640 10 monclient: init 2024-01-24T23:24:55.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.925+0000 7f85a5751640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:55.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.925+0000 7f85a5751640 10 monclient: _add_conns ranks=[3,4,1,8,7,2,0,6,5] 2024-01-24T23:24:55.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.925+0000 7f85a5751640 10 monclient(hunting): picked mon.noname-g con 0x7f85a0099130 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:55.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.925+0000 7f85a5751640 10 monclient(hunting): picked mon.noname-c con 0x7f85a009af90 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:55.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.925+0000 7f85a5751640 10 monclient(hunting): picked mon.noname-f con 0x7f85a0099c20 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:55.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.925+0000 7f85a5751640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a0064700 con 0x7f85a0099c20 2024-01-24T23:24:55.939 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.925+0000 7f85a5751640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a009a510 con 0x7f85a0099130 2024-01-24T23:24:55.939 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.925+0000 7f85a5751640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a009ba30 con 0x7f85a009af90 2024-01-24T23:24:55.939 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.925+0000 7f85a5751640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:55.939 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.925+0000 7f85a5751640 10 monclient(hunting): authenticate will time out at 1183.255894s 2024-01-24T23:24:55.939 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.925+0000 7f859f7fe640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f85a0099c20 legacy=0x7f85a013bd90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:55.939 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85967fc640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f85a009af90 legacy=0x7f85a0138660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:55.939 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859effd640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f85a0099130 0x7f85a0099510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:37450/0 (socket says 172.21.15.132:37450) 2024-01-24T23:24:55.939 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859effd640 1 -- 172.21.15.132:0/1612851512 learned_addr learned my addr 172.21.15.132:0/1612851512 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:55.939 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 1 -- 172.21.15.132:0/1612851512 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4141351402 0 0) 0x7f85a009a510 con 0x7f85a0099130 2024-01-24T23:24:55.939 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:24:55.939 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:55.939 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:55.939 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 10 monclient(hunting): my global_id is 5754 2024-01-24T23:24:55.939 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 1 -- 172.21.15.132:0/1612851512 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f858c003530 con 0x7f85a0099130 2024-01-24T23:24:55.940 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 1 -- 172.21.15.132:0/1612851512 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1387948804 0 0) 0x7f858c003530 con 0x7f85a0099130 2024-01-24T23:24:55.940 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 1 -- 172.21.15.132:0/1612851512 >> v1:172.21.15.115:6791/0 conn(0x7f85a009af90 legacy=0x7f85a0138660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:55.940 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 1 -- 172.21.15.132:0/1612851512 >> v1:172.21.15.132:6789/0 conn(0x7f85a0099c20 legacy=0x7f85a013bd90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:55.940 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:24:55.940 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 1 monclient: found mon.noname-g 2024-01-24T23:24:55.940 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:55.940 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:24:55.940 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 1 -- 172.21.15.132:0/1612851512 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a009bd10 con 0x7f85a0099130 2024-01-24T23:24:55.940 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 10 monclient: _finish_auth 0 2024-01-24T23:24:55.940 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:24:55.940 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:55.940 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 1 -- 172.21.15.132:0/1612851512 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f85900017c0 con 0x7f85a0099130 2024-01-24T23:24:55.940 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 5 monclient: authenticate success, global_id 5754 2024-01-24T23:24:55.940 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:55.941 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:55.941 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:55.941 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 10 monclient: dump: 2024-01-24T23:24:55.941 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:55.941 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:55.941 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.941 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.941 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:55.941 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:55.941 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:55.941 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:55.941 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:55.941 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:55.941 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:55.942 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:55.942 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:55.942 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:55.942 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:55.942 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:55.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:55.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 1 -- 172.21.15.132:0/1612851512 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8590001a70 con 0x7f85a0099130 2024-01-24T23:24:55.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:55.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:55.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 1 -- 172.21.15.132:0/1612851512 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8590004bf0 con 0x7f85a0099130 2024-01-24T23:24:55.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:55.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 4 set_mon_vals no callback set 2024-01-24T23:24:55.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:55.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859e7fc640 10 monclient: dump: 2024-01-24T23:24:55.943 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:55.943 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:55.943 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.943 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.943 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:55.943 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:55.943 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:55.943 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:55.943 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:55.943 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:55.943 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:55.943 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:55.943 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:55.943 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:55.943 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:55.944 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:55.944 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 10 monclient: shutdown 2024-01-24T23:24:55.944 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:55.944 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 1 -- 172.21.15.132:0/1612851512 >> v1:172.21.15.132:6790/0 conn(0x7f85a0099130 legacy=0x7f85a0099510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:55.944 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 1 -- 172.21.15.132:0/1612851512 shutdown_connections 2024-01-24T23:24:55.944 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 1 -- 172.21.15.132:0/1612851512 >> 172.21.15.132:0/1612851512 conn(0x7f85a00a1bc0 msgr2=0x7f85a00a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:55.944 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 1 -- 172.21.15.132:0/1612851512 shutdown_connections 2024-01-24T23:24:55.944 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 1 -- 172.21.15.132:0/1612851512 wait complete. 2024-01-24T23:24:55.944 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 10 monclient: build_initial_monmap 2024-01-24T23:24:55.944 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 10 monclient: monmap: 2024-01-24T23:24:55.944 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:55.944 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:55.944 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:55.944 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:55.944 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:55.945 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:55.945 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:55.945 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:55.945 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:55.945 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:55.945 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:55.945 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:55.945 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:55.945 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:55.945 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:55.945 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:55.945 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 1 Processor -- start 2024-01-24T23:24:55.945 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 1 -- start start 2024-01-24T23:24:55.945 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 10 monclient: init 2024-01-24T23:24:55.945 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:55.946 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 10 monclient: _add_conns ranks=[2,6,8,7,0,5,3,1,4] 2024-01-24T23:24:55.946 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 10 monclient(hunting): picked mon.noname-b con 0x7f85a0099130 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:55.946 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 10 monclient(hunting): picked mon.noname-d con 0x7f85a0099c20 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:55.946 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 10 monclient(hunting): picked mon.noname-e con 0x7f85a009af90 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:55.946 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a009b3e0 con 0x7f85a0099130 2024-01-24T23:24:55.946 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a0133090 con 0x7f85a0099c20 2024-01-24T23:24:55.946 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a0131870 con 0x7f85a009af90 2024-01-24T23:24:55.946 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:55.946 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85a5751640 10 monclient(hunting): authenticate will time out at 1183.258771s 2024-01-24T23:24:55.946 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859effd640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f85a0099130 0x7f85a013f9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:59596/0 (socket says 172.21.15.132:59596) 2024-01-24T23:24:55.946 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f859effd640 1 -- 172.21.15.132:0/2337040409 learned_addr learned my addr 172.21.15.132:0/2337040409 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:55.946 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f85967fc640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f85a0099c20 0x7f85a01d0590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:57388/0 (socket says 172.21.15.132:57388) 2024-01-24T23:24:55.946 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193054138 0 0) 0x7f85a0131870 con 0x7f85a009af90 2024-01-24T23:24:55.946 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f8597fff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:55.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f8597fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:55.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f8597fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:55.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f8597fff640 10 monclient(hunting): my global_id is 84923 2024-01-24T23:24:55.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8580003530 con 0x7f85a009af90 2024-01-24T23:24:55.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1865780804 0 0) 0x7f85a009b3e0 con 0x7f85a0099130 2024-01-24T23:24:55.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f8597fff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:55.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f8597fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:55.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f8597fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:55.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f8597fff640 10 monclient(hunting): my global_id is 85052 2024-01-24T23:24:55.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85a0131870 con 0x7f85a0099130 2024-01-24T23:24:55.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 23698511 0 0) 0x7f85a0133090 con 0x7f85a0099c20 2024-01-24T23:24:55.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f8597fff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:55.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f8597fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:55.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f8597fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:55.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f8597fff640 10 monclient(hunting): my global_id is 5847 2024-01-24T23:24:55.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.929+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85a009b3e0 con 0x7f85a0099c20 2024-01-24T23:24:55.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1989645159 0 0) 0x7f85a0131870 con 0x7f85a0099130 2024-01-24T23:24:55.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f85a0133090 con 0x7f85a0099130 2024-01-24T23:24:55.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8590001780 con 0x7f85a0099130 2024-01-24T23:24:55.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:55.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:24:55.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 10 monclient(hunting): dump: 2024-01-24T23:24:55.948 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:55.948 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:55.948 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.948 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.948 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:55.948 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:55.949 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:55.949 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:55.949 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:55.949 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:55.949 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:55.949 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:55.949 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:55.949 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:55.949 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:55.949 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:55.949 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:55.949 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 448177162 0 0) 0x7f8580003530 con 0x7f85a009af90 2024-01-24T23:24:55.949 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f85a0131870 con 0x7f85a009af90 2024-01-24T23:24:55.949 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f85980044e0 con 0x7f85a009af90 2024-01-24T23:24:55.950 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:55.950 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:24:55.950 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 10 monclient(hunting): dump: 2024-01-24T23:24:55.950 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:55.950 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:55.950 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.950 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.950 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:55.950 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:55.950 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:55.950 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:55.950 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:55.950 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:55.950 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:55.950 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:55.950 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:55.951 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:55.951 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:55.951 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:55.951 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2949935489 0 0) 0x7f85a009b3e0 con 0x7f85a0099c20 2024-01-24T23:24:55.951 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8580003530 con 0x7f85a0099c20 2024-01-24T23:24:55.951 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8588004120 con 0x7f85a0099c20 2024-01-24T23:24:55.951 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:55.951 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:24:55.951 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 10 monclient(hunting): dump: 2024-01-24T23:24:55.951 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:55.951 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:55.952 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.952 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.952 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:55.952 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:55.952 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:55.952 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:55.952 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:55.952 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:55.952 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:55.952 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:55.952 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:55.952 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:55.952 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:55.953 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:55.953 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2305056043 0 0) 0x7f85a0133090 con 0x7f85a0099130 2024-01-24T23:24:55.953 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 >> v1:172.21.15.115:6793/0 conn(0x7f85a009af90 legacy=0x7f85a01d3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:55.953 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 >> v1:172.21.15.115:6792/0 conn(0x7f85a0099c20 legacy=0x7f85a01d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:55.953 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 10 monclient: _finish_hunting 0 2024-01-24T23:24:55.953 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 1 monclient: found mon.b 2024-01-24T23:24:55.953 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:55.953 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:55.953 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a0133330 con 0x7f85a0099130 2024-01-24T23:24:55.953 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 10 monclient: _finish_auth 0 2024-01-24T23:24:55.953 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 10 monclient: _check_auth_tickets 2024-01-24T23:24:55.953 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:55.954 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f85a5751640 5 monclient: authenticate success, global_id 85052 2024-01-24T23:24:55.954 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f85a5751640 10 monclient: _renew_subs 2024-01-24T23:24:55.954 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f85a5751640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:55.954 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f85a5751640 1 -- 172.21.15.132:0/2337040409 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a01400b0 con 0x7f85a0099130 2024-01-24T23:24:55.954 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f85a5751640 10 monclient: _renew_subs 2024-01-24T23:24:55.954 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f85a5751640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:55.954 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f85a5751640 1 -- 172.21.15.132:0/2337040409 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a01325a0 con 0x7f85a0099130 2024-01-24T23:24:55.954 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8590004b90 con 0x7f85a0099130 2024-01-24T23:24:55.954 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:55.954 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f859dffb640 4 set_mon_vals no callback set 2024-01-24T23:24:55.954 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8590005160 con 0x7f85a0099130 2024-01-24T23:24:55.954 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:55.954 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:24:55.954 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 10 monclient: dump: 2024-01-24T23:24:55.954 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:55.955 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:55.955 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.955 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:55.955 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:55.955 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:55.955 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:55.955 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:55.955 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:55.955 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:55.955 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:55.955 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:55.955 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:55.955 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:55.956 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:55.956 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:55.956 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f85a5751640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:55.956 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f85a5751640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:55.956 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f85a5751640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:55.956 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f85a5751640 1 -- 172.21.15.132:0/2337040409 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8584005090 con 0x7f85a0099130 2024-01-24T23:24:55.956 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f859000fbe0 con 0x7f85a0099130 2024-01-24T23:24:55.956 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(116..116 src has 1..116) v4 ==== 5687+0+0 (unknown 3195493297 0 0) 0x7f8590043f00 con 0x7f85a0099130 2024-01-24T23:24:55.956 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.933+0000 7f85967fc640 10 monclient: get_auth_request con 0x7f8580037db0 auth_method 0 2024-01-24T23:24:55.957 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.945+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 <== mon.2 v1:172.21.15.115:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f8590014820 con 0x7f85a0099130 2024-01-24T23:24:55.957 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.945+0000 7f8597fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:55.957 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:55.945+0000 7f8597fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:56.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.101+0000 7f85a5751640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-01-24T23:24:56.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.101+0000 7f85a5751640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-01-24T23:24:56.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.101+0000 7f85a5751640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:24:56.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.101+0000 7f85a5751640 1 -- 172.21.15.132:0/2337040409 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]} v 0) v1 -- 0x7f8584005340 con 0x7f85a0099130 2024-01-24T23:24:56.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.101+0000 7f8597fff640 1 -- 172.21.15.132:0/2337040409 <== mon.2 v1:172.21.15.115:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}]=-2 pool 'fooset' not found v116) v1 ==== 152+0+0 (unknown 2319475762 0 0) 0x7f8590014640 con 0x7f85a0099130 2024-01-24T23:24:56.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.101+0000 7f8597fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-01-24T23:24:56.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.101+0000 7f8597fff640 10 monclient: _finish_command 2 = mon:2 pool 'fooset' not found 2024-01-24T23:24:56.106 INFO:tasks.workunit.client.0.smithi132.stderr:Error ENOENT: pool 'fooset' not found 2024-01-24T23:24:56.112 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:24:56.112 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool create fooset 8 2024-01-24T23:24:56.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 10 monclient: get_monmap_and_config 2024-01-24T23:24:56.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 10 monclient: build_initial_monmap 2024-01-24T23:24:56.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 10 monclient: monmap: 2024-01-24T23:24:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:56.229020+0000 2024-01-24T23:24:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:56.229020+0000 2024-01-24T23:24:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:56.229 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:56.229 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:56.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 1 Processor -- start 2024-01-24T23:24:56.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 1 -- start start 2024-01-24T23:24:56.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 10 monclient: init 2024-01-24T23:24:56.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:56.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 10 monclient: _add_conns ranks=[7,1,0,6,8,4,3,5,2] 2024-01-24T23:24:56.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 10 monclient(hunting): picked mon.noname-i con 0x7fd68c0a0a40 addr v1:172.21.15.132:6792/0 2024-01-24T23:24:56.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 10 monclient(hunting): picked mon.noname-f con 0x7fd68c0a0e20 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:56.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 10 monclient(hunting): picked mon.noname-a con 0x7fd68c0a1910 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:56.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd68c064700 con 0x7fd68c0a1910 2024-01-24T23:24:56.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd68c09b0f0 con 0x7fd68c0a0e20 2024-01-24T23:24:56.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd68c09ae10 con 0x7fd68c0a0a40 2024-01-24T23:24:56.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:56.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 10 monclient(hunting): authenticate will time out at 1183.553762s 2024-01-24T23:24:56.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68bfff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fd68c0a0e20 legacy=0x7fd68c0a1200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:56.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd690936640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fd68c0a0a40 0x7fd68c09bad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:60480/0 (socket says 172.21.15.132:60480) 2024-01-24T23:24:56.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd690936640 1 -- 172.21.15.132:0/3587787010 learned_addr learned my addr 172.21.15.132:0/3587787010 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:56.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd691137640 1 -- 172.21.15.132:0/3587787010 >> v1:172.21.15.115:6789/0 conn(0x7fd68c0a1910 legacy=0x7fd68c13f1c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:56.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 1 -- 172.21.15.132:0/3587787010 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1567165409 0 0) 0x7fd68c09ae10 con 0x7fd68c0a0a40 2024-01-24T23:24:56.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:24:56.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:56.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:56.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 10 monclient(hunting): my global_id is 5740 2024-01-24T23:24:56.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 1 -- 172.21.15.132:0/3587787010 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd674003530 con 0x7fd68c0a0a40 2024-01-24T23:24:56.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 1 -- 172.21.15.132:0/3587787010 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3001756686 0 0) 0x7fd674003530 con 0x7fd68c0a0a40 2024-01-24T23:24:56.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 1 -- 172.21.15.132:0/3587787010 >> v1:172.21.15.132:6789/0 conn(0x7fd68c0a0e20 legacy=0x7fd68c0a1200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:56.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 1 -- 172.21.15.132:0/3587787010 >> v1:172.21.15.115:6789/0 conn(0x7fd68c0a1910 legacy=0x7fd68c13f1c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:56.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:24:56.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 1 monclient: found mon.noname-i 2024-01-24T23:24:56.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:56.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:24:56.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 1 -- 172.21.15.132:0/3587787010 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68c0a25a0 con 0x7fd68c0a0a40 2024-01-24T23:24:56.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 10 monclient: _finish_auth 0 2024-01-24T23:24:56.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:24:56.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:56.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 5 monclient: authenticate success, global_id 5740 2024-01-24T23:24:56.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 1 -- 172.21.15.132:0/3587787010 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd67c0017c0 con 0x7fd68c0a0a40 2024-01-24T23:24:56.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:56.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:56.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:24:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 10 monclient: dump: 2024-01-24T23:24:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:56.240 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 1 -- 172.21.15.132:0/3587787010 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd67c001ae0 con 0x7fd68c0a0a40 2024-01-24T23:24:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 4 set_mon_vals no callback set 2024-01-24T23:24:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 1 -- 172.21.15.132:0/3587787010 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd67c004bf0 con 0x7fd68c0a0a40 2024-01-24T23:24:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:24:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd68b7fe640 10 monclient: dump: 2024-01-24T23:24:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:56.241 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 10 monclient: shutdown 2024-01-24T23:24:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 1 -- 172.21.15.132:0/3587787010 >> v1:172.21.15.132:6792/0 conn(0x7fd68c0a0a40 legacy=0x7fd68c09bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 1 -- 172.21.15.132:0/3587787010 shutdown_connections 2024-01-24T23:24:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 1 -- 172.21.15.132:0/3587787010 >> 172.21.15.132:0/3587787010 conn(0x7fd68c0a6040 msgr2=0x7fd68c0a6420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 1 -- 172.21.15.132:0/3587787010 shutdown_connections 2024-01-24T23:24:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 1 -- 172.21.15.132:0/3587787010 wait complete. 2024-01-24T23:24:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 10 monclient: build_initial_monmap 2024-01-24T23:24:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.225+0000 7fd692b9a640 10 monclient: monmap: 2024-01-24T23:24:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:56.243 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd692b9a640 1 Processor -- start 2024-01-24T23:24:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd692b9a640 1 -- start start 2024-01-24T23:24:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd692b9a640 10 monclient: init 2024-01-24T23:24:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd692b9a640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd692b9a640 10 monclient: _add_conns ranks=[0,3,1,4,6,5,8,7,2] 2024-01-24T23:24:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd692b9a640 10 monclient(hunting): picked mon.noname-a con 0x7fd68c0a0a40 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd692b9a640 10 monclient(hunting): picked mon.noname-g con 0x7fd68c0a0e20 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd692b9a640 10 monclient(hunting): picked mon.noname-f con 0x7fd68c0a1910 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd692b9a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd68c06a900 con 0x7fd68c0a0a40 2024-01-24T23:24:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd692b9a640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd68c1cbd30 con 0x7fd68c0a1910 2024-01-24T23:24:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd692b9a640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd68c09c4b0 con 0x7fd68c0a0e20 2024-01-24T23:24:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd692b9a640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd692b9a640 10 monclient(hunting): authenticate will time out at 1183.556610s 2024-01-24T23:24:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd691137640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fd68c0a1910 legacy=0x7fd68c1d3250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd68bfff640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fd68c0a0e20 0x7fd68c1cb620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:37454/0 (socket says 172.21.15.132:37454) 2024-01-24T23:24:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd68bfff640 1 -- 172.21.15.132:0/696053281 learned_addr learned my addr 172.21.15.132:0/696053281 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd690936640 1 -- 172.21.15.132:0/696053281 >> v1:172.21.15.115:6789/0 conn(0x7fd68c0a0a40 legacy=0x7fd68c1caf10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 1 -- 172.21.15.132:0/696053281 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376832931 0 0) 0x7fd68c09c4b0 con 0x7fd68c0a0e20 2024-01-24T23:24:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 10 monclient(hunting): _negotiate 2024-01-24T23:24:56.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:56.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:56.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 10 monclient(hunting): my global_id is 5763 2024-01-24T23:24:56.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 1 -- 172.21.15.132:0/696053281 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd678003530 con 0x7fd68c0a0e20 2024-01-24T23:24:56.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 1 -- 172.21.15.132:0/696053281 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1347979894 0 0) 0x7fd678003530 con 0x7fd68c0a0e20 2024-01-24T23:24:56.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 1 -- 172.21.15.132:0/696053281 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd68c09c4b0 con 0x7fd68c0a0e20 2024-01-24T23:24:56.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 1 -- 172.21.15.132:0/696053281 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd680004300 con 0x7fd68c0a0e20 2024-01-24T23:24:56.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:56.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:56.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 10 monclient(hunting): dump: 2024-01-24T23:24:56.245 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:56.245 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:56.245 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:56.245 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:56.245 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:56.246 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:56.246 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:56.246 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:56.246 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:56.246 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:56.246 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:56.246 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:56.246 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:56.246 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:56.246 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:56.246 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:56.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:56.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 1 -- 172.21.15.132:0/696053281 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 702687806 0 0) 0x7fd68c09c4b0 con 0x7fd68c0a0e20 2024-01-24T23:24:56.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 1 -- 172.21.15.132:0/696053281 >> v1:172.21.15.132:6789/0 conn(0x7fd68c0a1910 legacy=0x7fd68c1d3250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:56.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 1 -- 172.21.15.132:0/696053281 >> v1:172.21.15.115:6789/0 conn(0x7fd68c0a0a40 legacy=0x7fd68c1caf10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:56.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 10 monclient: _finish_hunting 0 2024-01-24T23:24:56.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 1 monclient: found mon.g 2024-01-24T23:24:56.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:56.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:56.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 1 -- 172.21.15.132:0/696053281 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68c134a30 con 0x7fd68c0a0e20 2024-01-24T23:24:56.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 10 monclient: _finish_auth 0 2024-01-24T23:24:56.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 10 monclient: _check_auth_tickets 2024-01-24T23:24:56.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:56.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd692b9a640 5 monclient: authenticate success, global_id 5763 2024-01-24T23:24:56.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd692b9a640 10 monclient: _renew_subs 2024-01-24T23:24:56.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd692b9a640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:56.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd692b9a640 1 -- 172.21.15.132:0/696053281 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd68c136480 con 0x7fd68c0a0e20 2024-01-24T23:24:56.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd692b9a640 10 monclient: _renew_subs 2024-01-24T23:24:56.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd692b9a640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:56.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd692b9a640 1 -- 172.21.15.132:0/696053281 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd68c1357f0 con 0x7fd68c0a0e20 2024-01-24T23:24:56.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 1 -- 172.21.15.132:0/696053281 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd680004620 con 0x7fd68c0a0e20 2024-01-24T23:24:56.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:56.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 1 -- 172.21.15.132:0/696053281 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd680004b50 con 0x7fd68c0a0e20 2024-01-24T23:24:56.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:56.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd68affd640 4 set_mon_vals no callback set 2024-01-24T23:24:56.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:56.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 10 monclient: dump: 2024-01-24T23:24:56.248 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:56.248 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:56.248 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:56.248 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:56.248 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:56.248 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:56.248 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:56.248 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:56.248 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:56.249 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:56.249 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:56.249 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:56.249 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:56.249 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:56.249 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:56.249 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:56.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd692b9a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:56.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd692b9a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:56.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd692b9a640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:56.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd692b9a640 1 -- 172.21.15.132:0/696053281 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd66c005090 con 0x7fd68c0a0e20 2024-01-24T23:24:56.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 1 -- 172.21.15.132:0/696053281 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fd680004d30 con 0x7fd68c0a0e20 2024-01-24T23:24:56.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd6897fa640 1 -- 172.21.15.132:0/696053281 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(116..116 src has 1..116) v4 ==== 5687+0+0 (unknown 3195493297 0 0) 0x7fd6800440a0 con 0x7fd68c0a0e20 2024-01-24T23:24:56.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.229+0000 7fd690936640 10 monclient: get_auth_request con 0x7fd678036ef0 auth_method 0 2024-01-24T23:24:56.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.233+0000 7fd6897fa640 1 -- 172.21.15.132:0/696053281 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fd680014960 con 0x7fd68c0a0e20 2024-01-24T23:24:56.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.233+0000 7fd6897fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:56.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.233+0000 7fd6897fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:56.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.385+0000 7fd692b9a640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}] 2024-01-24T23:24:56.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.385+0000 7fd692b9a640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}] 2024-01-24T23:24:56.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.385+0000 7fd692b9a640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:56.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.385+0000 7fd692b9a640 1 -- 172.21.15.132:0/696053281 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "fooset", "pg_num": 8} v 0) v1 -- 0x7fd66c0058c0 con 0x7fd68c0a0e20 2024-01-24T23:24:56.897 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.893+0000 7fd6897fa640 1 -- 172.21.15.132:0/696053281 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}]=0 pool 'fooset' created v117) v1 ==== 115+0+0 (unknown 852122905 0 0) 0x7fd68001d090 con 0x7fd68c0a0e20 2024-01-24T23:24:56.897 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.893+0000 7fd6897fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}] 2024-01-24T23:24:56.897 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:56.893+0000 7fd6897fa640 10 monclient: _finish_command 2 = system:0 pool 'fooset' created 2024-01-24T23:24:57.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.001+0000 7fd692b9a640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}] 2024-01-24T23:24:57.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.001+0000 7fd692b9a640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}] 2024-01-24T23:24:57.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.001+0000 7fd692b9a640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:57.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.001+0000 7fd692b9a640 1 -- 172.21.15.132:0/696053281 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "fooset", "pg_num": 8} v 0) v1 -- 0x7fd66c000bc0 con 0x7fd68c0a0e20 2024-01-24T23:24:57.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.001+0000 7fd6897fa640 1 -- 172.21.15.132:0/696053281 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}]=0 pool 'fooset' already exists v117) v1 ==== 122+0+0 (unknown 2841276622 0 0) 0x7fd680013000 con 0x7fd68c0a0e20 2024-01-24T23:24:57.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.001+0000 7fd6897fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}] 2024-01-24T23:24:57.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.001+0000 7fd6897fa640 10 monclient: _finish_command 3 = system:0 pool 'fooset' already exists 2024-01-24T23:24:57.005 INFO:tasks.workunit.client.0.smithi132.stderr:pool 'fooset' already exists 2024-01-24T23:24:57.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.001+0000 7fd692b9a640 1 -- 172.21.15.132:0/696053281 >> v1:172.21.15.132:6812/1835401922 conn(0x7fd678036ef0 legacy=0x7fd6780393b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:57.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.001+0000 7fd692b9a640 10 monclient: shutdown 2024-01-24T23:24:57.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.001+0000 7fd692b9a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:57.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.001+0000 7fd692b9a640 1 -- 172.21.15.132:0/696053281 >> v1:172.21.15.132:6790/0 conn(0x7fd68c0a0e20 legacy=0x7fd68c1cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:57.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.001+0000 7fd692b9a640 1 -- 172.21.15.132:0/696053281 shutdown_connections 2024-01-24T23:24:57.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.001+0000 7fd692b9a640 1 -- 172.21.15.132:0/696053281 >> 172.21.15.132:0/696053281 conn(0x7fd68c0a6040 msgr2=0x7fd68c0a3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:57.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.001+0000 7fd692b9a640 1 -- 172.21.15.132:0/696053281 shutdown_connections 2024-01-24T23:24:57.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.005+0000 7fd692b9a640 1 -- 172.21.15.132:0/696053281 wait complete. 2024-01-24T23:24:57.022 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool create barset 8 2024-01-24T23:24:57.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.133+0000 7fa2c8e15640 10 monclient: get_monmap_and_config 2024-01-24T23:24:57.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.133+0000 7fa2c8e15640 10 monclient: build_initial_monmap 2024-01-24T23:24:57.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.133+0000 7fa2c8e15640 10 monclient: monmap: 2024-01-24T23:24:57.138 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:57.138 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:57.138 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:57.139246+0000 2024-01-24T23:24:57.138 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:57.139246+0000 2024-01-24T23:24:57.138 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:57.138 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:57.138 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:57.138 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:57.138 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:57.139 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:57.139 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:57.139 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:57.139 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:57.139 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:57.139 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:57.139 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:57.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.133+0000 7fa2c8e15640 1 Processor -- start 2024-01-24T23:24:57.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.133+0000 7fa2c8e15640 1 -- start start 2024-01-24T23:24:57.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.133+0000 7fa2c8e15640 10 monclient: init 2024-01-24T23:24:57.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.133+0000 7fa2c8e15640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:57.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.133+0000 7fa2c8e15640 10 monclient: _add_conns ranks=[6,4,1,0,7,3,5,2,8] 2024-01-24T23:24:57.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.133+0000 7fa2c8e15640 10 monclient(hunting): picked mon.noname-d con 0x7fa2c409ff20 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:57.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.133+0000 7fa2c8e15640 10 monclient(hunting): picked mon.noname-c con 0x7fa2c409f9c0 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:57.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.133+0000 7fa2c8e15640 10 monclient(hunting): picked mon.noname-f con 0x7fa2c413b2c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:24:57.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.133+0000 7fa2c8e15640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c4064700 con 0x7fa2c413b2c0 2024-01-24T23:24:57.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.133+0000 7fa2c8e15640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c409e760 con 0x7fa2c409f9c0 2024-01-24T23:24:57.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.133+0000 7fa2c8e15640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c40a1160 con 0x7fa2c409ff20 2024-01-24T23:24:57.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.133+0000 7fa2c8e15640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:57.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.133+0000 7fa2c8e15640 10 monclient(hunting): authenticate will time out at 1184.463942s 2024-01-24T23:24:57.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.133+0000 7fa2c2d9d640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fa2c413b2c0 legacy=0x7fa2c413d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:24:57.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c1d9b640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7fa2c409f9c0 legacy=0x7fa2c4139ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:57.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c259c640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fa2c409ff20 0x7fa2c40a0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:57394/0 (socket says 172.21.15.132:57394) 2024-01-24T23:24:57.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c259c640 1 -- 172.21.15.132:0/3826354147 learned_addr learned my addr 172.21.15.132:0/3826354147 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:57.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 1 -- 172.21.15.132:0/3826354147 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1634012904 0 0) 0x7fa2c40a1160 con 0x7fa2c409ff20 2024-01-24T23:24:57.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 10 monclient(hunting): _negotiate 2024-01-24T23:24:57.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:57.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:57.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 10 monclient(hunting): my global_id is 5856 2024-01-24T23:24:57.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 1 -- 172.21.15.132:0/3826354147 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2ac003530 con 0x7fa2c409ff20 2024-01-24T23:24:57.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 1 -- 172.21.15.132:0/3826354147 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2052811102 0 0) 0x7fa2ac003530 con 0x7fa2c409ff20 2024-01-24T23:24:57.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 1 -- 172.21.15.132:0/3826354147 >> v1:172.21.15.115:6791/0 conn(0x7fa2c409f9c0 legacy=0x7fa2c4139ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:57.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 1 -- 172.21.15.132:0/3826354147 >> v1:172.21.15.132:6789/0 conn(0x7fa2c413b2c0 legacy=0x7fa2c413d6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:57.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 10 monclient: _finish_hunting 0 2024-01-24T23:24:57.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 1 monclient: found mon.noname-d 2024-01-24T23:24:57.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:57.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:24:57.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 1 -- 172.21.15.132:0/3826354147 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c40a0a10 con 0x7fa2c409ff20 2024-01-24T23:24:57.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 10 monclient: _finish_auth 0 2024-01-24T23:24:57.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 10 monclient: _check_auth_tickets 2024-01-24T23:24:57.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:57.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 1 -- 172.21.15.132:0/3826354147 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa2b4001800 con 0x7fa2c409ff20 2024-01-24T23:24:57.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:57.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:24:57.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 10 monclient: dump: 2024-01-24T23:24:57.149 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:57.149 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:57.149 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:57.149 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:57.149 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:57.149 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:57.149 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:57.149 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:57.149 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:57.149 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:57.150 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:57.150 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:57.150 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:57.150 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:57.150 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:57.150 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:57.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 5 monclient: authenticate success, global_id 5856 2024-01-24T23:24:57.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:57.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 1 -- 172.21.15.132:0/3826354147 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa2b4001ad0 con 0x7fa2c409ff20 2024-01-24T23:24:57.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:57.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:57.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 4 set_mon_vals no callback set 2024-01-24T23:24:57.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 1 -- 172.21.15.132:0/3826354147 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa2b4004bf0 con 0x7fa2c409ff20 2024-01-24T23:24:57.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:57.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:24:57.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c159a640 10 monclient: dump: 2024-01-24T23:24:57.151 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:57.151 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:57.151 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:57.151 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:57.151 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:57.151 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:57.151 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:57.151 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:57.152 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:57.152 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:57.152 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:57.152 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:57.152 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:57.152 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:57.152 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:57.152 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:57.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 10 monclient: shutdown 2024-01-24T23:24:57.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:57.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 1 -- 172.21.15.132:0/3826354147 >> v1:172.21.15.115:6792/0 conn(0x7fa2c409ff20 legacy=0x7fa2c40a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:57.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 1 -- 172.21.15.132:0/3826354147 shutdown_connections 2024-01-24T23:24:57.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 1 -- 172.21.15.132:0/3826354147 >> 172.21.15.132:0/3826354147 conn(0x7fa2c40a5a10 msgr2=0x7fa2c40a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:57.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 1 -- 172.21.15.132:0/3826354147 shutdown_connections 2024-01-24T23:24:57.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 1 -- 172.21.15.132:0/3826354147 wait complete. 2024-01-24T23:24:57.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 10 monclient: build_initial_monmap 2024-01-24T23:24:57.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 10 monclient: monmap: 2024-01-24T23:24:57.153 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:57.153 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:57.153 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:57.153 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:57.153 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:57.153 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:57.153 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:57.153 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:57.153 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:57.154 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:57.154 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:57.154 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:57.154 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:57.154 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:57.154 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:57.154 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:57.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 1 Processor -- start 2024-01-24T23:24:57.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 1 -- start start 2024-01-24T23:24:57.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 10 monclient: init 2024-01-24T23:24:57.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:57.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 10 monclient: _add_conns ranks=[2,8,3,0,5,4,6,1,7] 2024-01-24T23:24:57.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 10 monclient(hunting): picked mon.noname-b con 0x7fa2c409f9c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:57.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 10 monclient(hunting): picked mon.noname-e con 0x7fa2c409ff20 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:57.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 10 monclient(hunting): picked mon.noname-g con 0x7fa2c413b2c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:24:57.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c4120bb0 con 0x7fa2c409f9c0 2024-01-24T23:24:57.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c409d2e0 con 0x7fa2c413b2c0 2024-01-24T23:24:57.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c4071960 con 0x7fa2c409ff20 2024-01-24T23:24:57.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:57.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c8e15640 10 monclient(hunting): authenticate will time out at 1184.467633s 2024-01-24T23:24:57.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c2d9d640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fa2c413b2c0 0x7fa2c41d3730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:37458/0 (socket says 172.21.15.132:37458) 2024-01-24T23:24:57.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2c2d9d640 1 -- 172.21.15.132:0/67418017 learned_addr learned my addr 172.21.15.132:0/67418017 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:57.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2b2ffd640 1 -- 172.21.15.132:0/67418017 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1211511700 0 0) 0x7fa2c409d2e0 con 0x7fa2c413b2c0 2024-01-24T23:24:57.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2b2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:24:57.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2b2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:57.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2b2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:57.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2b2ffd640 10 monclient(hunting): my global_id is 5772 2024-01-24T23:24:57.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.137+0000 7fa2b2ffd640 1 -- 172.21.15.132:0/67418017 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2a4003530 con 0x7fa2c413b2c0 2024-01-24T23:24:57.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 1 -- 172.21.15.132:0/67418017 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4148059791 0 0) 0x7fa2a4003530 con 0x7fa2c413b2c0 2024-01-24T23:24:57.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 1 -- 172.21.15.132:0/67418017 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa2c409d2e0 con 0x7fa2c413b2c0 2024-01-24T23:24:57.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 1 -- 172.21.15.132:0/67418017 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa2bc004530 con 0x7fa2c413b2c0 2024-01-24T23:24:57.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:57.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:24:57.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 10 monclient(hunting): dump: 2024-01-24T23:24:57.156 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:57.159 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:57.159 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:57.159 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:57.159 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:57.159 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:57.159 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:57.159 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:57.159 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:57.160 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:57.160 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:57.160 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:57.160 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:57.160 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:57.160 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:57.160 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:57.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:57.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 1 -- 172.21.15.132:0/67418017 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1309292212 0 0) 0x7fa2c409d2e0 con 0x7fa2c413b2c0 2024-01-24T23:24:57.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 1 -- 172.21.15.132:0/67418017 >> v1:172.21.15.115:6793/0 conn(0x7fa2c409ff20 legacy=0x7fa2c409cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:57.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 1 -- 172.21.15.132:0/67418017 >> v1:172.21.15.115:6790/0 conn(0x7fa2c409f9c0 legacy=0x7fa2c409c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:57.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:24:57.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 1 monclient: found mon.g 2024-01-24T23:24:57.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:57.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:57.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 1 -- 172.21.15.132:0/67418017 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c4071680 con 0x7fa2c413b2c0 2024-01-24T23:24:57.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 10 monclient: _finish_auth 0 2024-01-24T23:24:57.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:24:57.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:57.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2c8e15640 5 monclient: authenticate success, global_id 5772 2024-01-24T23:24:57.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2c8e15640 10 monclient: _renew_subs 2024-01-24T23:24:57.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2c8e15640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:57.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2c8e15640 1 -- 172.21.15.132:0/67418017 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c409d560 con 0x7fa2c413b2c0 2024-01-24T23:24:57.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2c8e15640 10 monclient: _renew_subs 2024-01-24T23:24:57.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2c8e15640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:57.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2c8e15640 1 -- 172.21.15.132:0/67418017 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c409de10 con 0x7fa2c413b2c0 2024-01-24T23:24:57.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 1 -- 172.21.15.132:0/67418017 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa2bc004850 con 0x7fa2c413b2c0 2024-01-24T23:24:57.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:57.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 1 -- 172.21.15.132:0/67418017 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa2bc004d80 con 0x7fa2c413b2c0 2024-01-24T23:24:57.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:57.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2c0d99640 4 set_mon_vals no callback set 2024-01-24T23:24:57.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:24:57.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 10 monclient: dump: 2024-01-24T23:24:57.162 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:57.162 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:57.162 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:57.163 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:57.163 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:57.163 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:57.163 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:57.163 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:57.163 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:57.163 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:57.163 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:57.163 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:57.163 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:57.163 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:57.163 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:57.163 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:57.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2c8e15640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:57.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2c8e15640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:57.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2c8e15640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:57.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2c8e15640 1 -- 172.21.15.132:0/67418017 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a8005090 con 0x7fa2c413b2c0 2024-01-24T23:24:57.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 1 -- 172.21.15.132:0/67418017 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fa2bc004f60 con 0x7fa2c413b2c0 2024-01-24T23:24:57.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2b2ffd640 1 -- 172.21.15.132:0/67418017 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(117..117 src has 1..117) v4 ==== 6032+0+0 (unknown 4030445170 0 0) 0x7fa2bc044430 con 0x7fa2c413b2c0 2024-01-24T23:24:57.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.141+0000 7fa2c259c640 10 monclient: get_auth_request con 0x7fa2a4036ef0 auth_method 0 2024-01-24T23:24:57.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.145+0000 7fa2b2ffd640 1 -- 172.21.15.132:0/67418017 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fa2bc014ba0 con 0x7fa2c413b2c0 2024-01-24T23:24:57.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.145+0000 7fa2b2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:57.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.145+0000 7fa2b2ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:57.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.289+0000 7fa2c8e15640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}] 2024-01-24T23:24:57.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.289+0000 7fa2c8e15640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}] 2024-01-24T23:24:57.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.289+0000 7fa2c8e15640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:57.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.289+0000 7fa2c8e15640 1 -- 172.21.15.132:0/67418017 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "barset", "pg_num": 8} v 0) v1 -- 0x7fa2a80058c0 con 0x7fa2c413b2c0 2024-01-24T23:24:57.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.897+0000 7fa2b2ffd640 1 -- 172.21.15.132:0/67418017 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}]=0 pool 'barset' created v118) v1 ==== 115+0+0 (unknown 429939949 0 0) 0x7fa2bc01d2d0 con 0x7fa2c413b2c0 2024-01-24T23:24:57.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.897+0000 7fa2b2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}] 2024-01-24T23:24:57.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:57.897+0000 7fa2b2ffd640 10 monclient: _finish_command 2 = system:0 pool 'barset' created 2024-01-24T23:24:58.006 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.001+0000 7fa2c8e15640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}] 2024-01-24T23:24:58.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.001+0000 7fa2c8e15640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}] 2024-01-24T23:24:58.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.001+0000 7fa2c8e15640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:24:58.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.001+0000 7fa2c8e15640 1 -- 172.21.15.132:0/67418017 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "barset", "pg_num": 8} v 0) v1 -- 0x7fa2a8000bc0 con 0x7fa2c413b2c0 2024-01-24T23:24:58.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.005+0000 7fa2b2ffd640 1 -- 172.21.15.132:0/67418017 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}]=0 pool 'barset' already exists v118) v1 ==== 122+0+0 (unknown 2330010594 0 0) 0x7fa2bc013240 con 0x7fa2c413b2c0 2024-01-24T23:24:58.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.005+0000 7fa2b2ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}] 2024-01-24T23:24:58.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.005+0000 7fa2b2ffd640 10 monclient: _finish_command 3 = system:0 pool 'barset' already exists 2024-01-24T23:24:58.008 INFO:tasks.workunit.client.0.smithi132.stderr:pool 'barset' already exists 2024-01-24T23:24:58.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.005+0000 7fa2c8e15640 1 -- 172.21.15.132:0/67418017 >> v1:172.21.15.132:6812/1835401922 conn(0x7fa2a4036ef0 legacy=0x7fa2a40393b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:58.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.005+0000 7fa2c8e15640 10 monclient: shutdown 2024-01-24T23:24:58.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.005+0000 7fa2c8e15640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:58.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.005+0000 7fa2c8e15640 1 -- 172.21.15.132:0/67418017 >> v1:172.21.15.132:6790/0 conn(0x7fa2c413b2c0 legacy=0x7fa2c41d3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:58.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.005+0000 7fa2c8e15640 1 -- 172.21.15.132:0/67418017 shutdown_connections 2024-01-24T23:24:58.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.005+0000 7fa2c8e15640 1 -- 172.21.15.132:0/67418017 >> 172.21.15.132:0/67418017 conn(0x7fa2c40a5a10 msgr2=0x7fa2c4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:58.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.005+0000 7fa2c8e15640 1 -- 172.21.15.132:0/67418017 shutdown_connections 2024-01-24T23:24:58.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.005+0000 7fa2c8e15640 1 -- 172.21.15.132:0/67418017 wait complete. 2024-01-24T23:24:58.024 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool set barset size 3 2024-01-24T23:24:58.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.129+0000 7f65618e6640 10 monclient: get_monmap_and_config 2024-01-24T23:24:58.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.129+0000 7f65618e6640 10 monclient: build_initial_monmap 2024-01-24T23:24:58.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.129+0000 7f65618e6640 10 monclient: monmap: 2024-01-24T23:24:58.135 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:58.135 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:58.135 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:24:58.136628+0000 2024-01-24T23:24:58.135 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:24:58.136628+0000 2024-01-24T23:24:58.135 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:58.136 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:58.136 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:58.136 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:58.136 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:58.136 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:58.136 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:58.136 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:58.136 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:58.136 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:58.136 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:58.136 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:58.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 1 Processor -- start 2024-01-24T23:24:58.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 1 -- start start 2024-01-24T23:24:58.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 10 monclient: init 2024-01-24T23:24:58.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:58.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 10 monclient: _add_conns ranks=[8,2,0,7,3,5,6,4,1] 2024-01-24T23:24:58.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 10 monclient(hunting): picked mon.noname-e con 0x7f655c072690 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:58.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 10 monclient(hunting): picked mon.noname-b con 0x7f655c09ae90 addr v1:172.21.15.115:6790/0 2024-01-24T23:24:58.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 10 monclient(hunting): picked mon.noname-a con 0x7f655c09bdc0 addr v1:172.21.15.115:6789/0 2024-01-24T23:24:58.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f655c064700 con 0x7f655c09bdc0 2024-01-24T23:24:58.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f655c09a790 con 0x7f655c09ae90 2024-01-24T23:24:58.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f655c0a18f0 con 0x7f655c072690 2024-01-24T23:24:58.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:58.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 10 monclient(hunting): authenticate will time out at 1185.461151s 2024-01-24T23:24:58.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f655b7fe640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f655c09bdc0 legacy=0x7f655c13d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:24:58.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f655affd640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f655c072690 0x7f655c0a0ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:40620/0 (socket says 172.21.15.132:40620) 2024-01-24T23:24:58.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f655a7fc640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f655c09ae90 0x7f655c0a11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:59614/0 (socket says 172.21.15.132:59614) 2024-01-24T23:24:58.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f655affd640 1 -- 172.21.15.132:0/2634940956 learned_addr learned my addr 172.21.15.132:0/2634940956 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:58.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 1 -- 172.21.15.132:0/2634940956 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 375095321 0 0) 0x7f655c09a790 con 0x7f655c09ae90 2024-01-24T23:24:58.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:58.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:58.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:58.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 10 monclient(hunting): my global_id is 85070 2024-01-24T23:24:58.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 1 -- 172.21.15.132:0/2634940956 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6548003530 con 0x7f655c09ae90 2024-01-24T23:24:58.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 1 -- 172.21.15.132:0/2634940956 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 605599724 0 0) 0x7f655c0a18f0 con 0x7f655c072690 2024-01-24T23:24:58.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:24:58.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:58.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:58.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 10 monclient(hunting): my global_id is 84941 2024-01-24T23:24:58.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 1 -- 172.21.15.132:0/2634940956 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f655c09a790 con 0x7f655c072690 2024-01-24T23:24:58.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 1 -- 172.21.15.132:0/2634940956 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2931076995 0 0) 0x7f655c09a790 con 0x7f655c072690 2024-01-24T23:24:58.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 1 -- 172.21.15.132:0/2634940956 >> v1:172.21.15.115:6790/0 conn(0x7f655c09ae90 legacy=0x7f655c0a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:58.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 1 -- 172.21.15.132:0/2634940956 >> v1:172.21.15.115:6789/0 conn(0x7f655c09bdc0 legacy=0x7f655c13d280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:24:58.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:24:58.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 1 monclient: found mon.noname-e 2024-01-24T23:24:58.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:58.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:24:58.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 1 -- 172.21.15.132:0/2634940956 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655c0997a0 con 0x7f655c072690 2024-01-24T23:24:58.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 10 monclient: _finish_auth 0 2024-01-24T23:24:58.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:24:58.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:58.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 5 monclient: authenticate success, global_id 84941 2024-01-24T23:24:58.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 1 -- 172.21.15.132:0/2634940956 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f65540017a0 con 0x7f655c072690 2024-01-24T23:24:58.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:58.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:58.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:24:58.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 10 monclient: dump: 2024-01-24T23:24:58.148 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:58.148 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:58.148 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:58.148 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:58.148 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:58.148 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:58.148 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:58.148 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:58.148 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:58.148 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:58.149 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:58.149 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:58.149 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:58.149 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:58.149 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:58.149 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:58.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:24:58.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 1 -- 172.21.15.132:0/2634940956 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f65540019d0 con 0x7f655c072690 2024-01-24T23:24:58.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:58.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 10 monclient: get_monmap_and_config success 2024-01-24T23:24:58.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 4 set_mon_vals no callback set 2024-01-24T23:24:58.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 1 -- 172.21.15.132:0/2634940956 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6554004bf0 con 0x7f655c072690 2024-01-24T23:24:58.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:58.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:24:58.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f6559ffb640 10 monclient: dump: 2024-01-24T23:24:58.149 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:58.150 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:58.150 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:58.150 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:58.150 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:58.150 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:58.150 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:58.150 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:58.150 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:58.150 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:58.150 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:58.150 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:58.150 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:58.150 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:58.150 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:58.150 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:58.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 10 monclient: shutdown 2024-01-24T23:24:58.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:58.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 1 -- 172.21.15.132:0/2634940956 >> v1:172.21.15.115:6793/0 conn(0x7f655c072690 legacy=0x7f655c0a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:58.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 1 -- 172.21.15.132:0/2634940956 shutdown_connections 2024-01-24T23:24:58.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 1 -- 172.21.15.132:0/2634940956 >> 172.21.15.132:0/2634940956 conn(0x7f655c0a1bc0 msgr2=0x7f655c0a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:58.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 1 -- 172.21.15.132:0/2634940956 shutdown_connections 2024-01-24T23:24:58.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 1 -- 172.21.15.132:0/2634940956 wait complete. 2024-01-24T23:24:58.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.133+0000 7f65618e6640 10 monclient: build_initial_monmap 2024-01-24T23:24:58.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 10 monclient: monmap: 2024-01-24T23:24:58.151 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:24:58.151 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:24:58.151 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:24:58.151 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:24:58.151 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:24:58.151 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:24:58.151 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:24:58.151 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:24:58.151 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:24:58.152 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:24:58.152 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:24:58.152 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:24:58.152 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:24:58.152 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:24:58.152 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:24:58.152 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:58.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 1 Processor -- start 2024-01-24T23:24:58.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 1 -- start start 2024-01-24T23:24:58.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 10 monclient: init 2024-01-24T23:24:58.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 10 monclient: _reopen_session rank -1 2024-01-24T23:24:58.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 10 monclient: _add_conns ranks=[8,6,4,2,3,1,5,0,7] 2024-01-24T23:24:58.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 10 monclient(hunting): picked mon.noname-e con 0x7f655c072690 addr v1:172.21.15.115:6793/0 2024-01-24T23:24:58.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 10 monclient(hunting): picked mon.noname-d con 0x7f655c09ae90 addr v1:172.21.15.115:6792/0 2024-01-24T23:24:58.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 10 monclient(hunting): picked mon.noname-c con 0x7f655c09bdc0 addr v1:172.21.15.115:6791/0 2024-01-24T23:24:58.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f655c1b4fa0 con 0x7f655c09bdc0 2024-01-24T23:24:58.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f655c1b2e30 con 0x7f655c09ae90 2024-01-24T23:24:58.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f655c1b4700 con 0x7f655c072690 2024-01-24T23:24:58.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 10 monclient(hunting): _renew_subs 2024-01-24T23:24:58.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 10 monclient(hunting): authenticate will time out at 1185.464854s 2024-01-24T23:24:58.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f655b7fe640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f655c09bdc0 legacy=0x7f655c1d4400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:24:58.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f655affd640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f655c072690 0x7f655c1b84f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:40624/0 (socket says 172.21.15.132:40624) 2024-01-24T23:24:58.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f655a7fc640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f655c09ae90 0x7f655c1d0cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:57406/0 (socket says 172.21.15.132:57406) 2024-01-24T23:24:58.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f655affd640 1 -- 172.21.15.132:0/2735890779 learned_addr learned my addr 172.21.15.132:0/2735890779 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:24:58.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 1 -- 172.21.15.132:0/2735890779 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 208566560 0 0) 0x7f655c1b2e30 con 0x7f655c09ae90 2024-01-24T23:24:58.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:58.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:58.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:58.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 10 monclient(hunting): my global_id is 5865 2024-01-24T23:24:58.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 1 -- 172.21.15.132:0/2735890779 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f653c003530 con 0x7f655c09ae90 2024-01-24T23:24:58.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 1 -- 172.21.15.132:0/2735890779 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 156416378 0 0) 0x7f655c1b4700 con 0x7f655c072690 2024-01-24T23:24:58.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 10 monclient(hunting): _negotiate 2024-01-24T23:24:58.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:24:58.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:24:58.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 10 monclient(hunting): my global_id is 84950 2024-01-24T23:24:58.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 1 -- 172.21.15.132:0/2735890779 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f655c1b2e30 con 0x7f655c072690 2024-01-24T23:24:58.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 1 -- 172.21.15.132:0/2735890779 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2804125377 0 0) 0x7f653c003530 con 0x7f655c09ae90 2024-01-24T23:24:58.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 1 -- 172.21.15.132:0/2735890779 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f655c1b4700 con 0x7f655c09ae90 2024-01-24T23:24:58.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 1 -- 172.21.15.132:0/2735890779 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f65440017b0 con 0x7f655c09ae90 2024-01-24T23:24:58.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:58.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:24:58.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 10 monclient(hunting): dump: 2024-01-24T23:24:58.155 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:58.155 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:58.155 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:58.155 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:58.155 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:58.155 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:58.155 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:58.155 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:58.155 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:58.155 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:58.155 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:58.155 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:58.155 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:58.155 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:58.155 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:58.155 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:58.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:24:58.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 1 -- 172.21.15.132:0/2735890779 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 513240760 0 0) 0x7f655c1b2e30 con 0x7f655c072690 2024-01-24T23:24:58.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 1 -- 172.21.15.132:0/2735890779 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f653c003530 con 0x7f655c072690 2024-01-24T23:24:58.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 1 -- 172.21.15.132:0/2735890779 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f65540047f0 con 0x7f655c072690 2024-01-24T23:24:58.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:58.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:24:58.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 10 monclient(hunting): dump: 2024-01-24T23:24:58.156 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:58.156 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:58.156 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:58.156 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:58.156 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:58.156 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:58.157 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:58.157 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:58.157 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:58.157 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:58.157 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:58.157 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:58.157 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:58.157 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:58.157 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:58.157 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:58.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 1 -- 172.21.15.132:0/2735890779 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1223774192 0 0) 0x7f655c1b4700 con 0x7f655c09ae90 2024-01-24T23:24:58.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 1 -- 172.21.15.132:0/2735890779 >> v1:172.21.15.115:6793/0 conn(0x7f655c072690 legacy=0x7f655c1b84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:58.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 1 -- 172.21.15.132:0/2735890779 >> v1:172.21.15.115:6791/0 conn(0x7f655c09bdc0 legacy=0x7f655c1d4400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:24:58.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 10 monclient: _finish_hunting 0 2024-01-24T23:24:58.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 1 monclient: found mon.d 2024-01-24T23:24:58.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:24:58.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:58.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 1 -- 172.21.15.132:0/2735890779 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655c1b49e0 con 0x7f655c09ae90 2024-01-24T23:24:58.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 10 monclient: _finish_auth 0 2024-01-24T23:24:58.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 10 monclient: _check_auth_tickets 2024-01-24T23:24:58.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:24:58.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 5 monclient: authenticate success, global_id 5865 2024-01-24T23:24:58.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 10 monclient: _renew_subs 2024-01-24T23:24:58.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:58.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 1 -- 172.21.15.132:0/2735890779 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f655c1b4cc0 con 0x7f655c09ae90 2024-01-24T23:24:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 10 monclient: _renew_subs 2024-01-24T23:24:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 1 -- 172.21.15.132:0/2735890779 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f655c1b8c00 con 0x7f655c09ae90 2024-01-24T23:24:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 1 -- 172.21.15.132:0/2735890779 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6544001ae0 con 0x7f655c09ae90 2024-01-24T23:24:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:24:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 1 -- 172.21.15.132:0/2735890779 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6544005160 con 0x7f655c09ae90 2024-01-24T23:24:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:24:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65597fa640 4 set_mon_vals no callback set 2024-01-24T23:24:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:24:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 10 monclient: dump: 2024-01-24T23:24:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:24:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:24:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:24:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:24:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:24:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:24:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:24:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:24:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:24:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:24:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:24:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:24:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:24:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:24:58.160 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:24:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:58.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:58.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:58.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f65618e6640 1 -- 172.21.15.132:0/2735890779 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6540005090 con 0x7f655c09ae90 2024-01-24T23:24:58.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.137+0000 7f6553fff640 1 -- 172.21.15.132:0/2735890779 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f654400fc60 con 0x7f655c09ae90 2024-01-24T23:24:58.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.141+0000 7f6553fff640 1 -- 172.21.15.132:0/2735890779 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(118..118 src has 1..118) v4 ==== 6377+0+0 (unknown 2413685430 0 0) 0x7f65440444c0 con 0x7f655c09ae90 2024-01-24T23:24:58.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.141+0000 7f655affd640 10 monclient: get_auth_request con 0x7f653c037750 auth_method 0 2024-01-24T23:24:58.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.149+0000 7f6553fff640 1 -- 172.21.15.132:0/2735890779 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f6544014ad0 con 0x7f655c09ae90 2024-01-24T23:24:58.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.149+0000 7f6553fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:24:58.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.149+0000 7f6553fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:24:58.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.293+0000 7f65618e6640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}] 2024-01-24T23:24:58.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.293+0000 7f65618e6640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}] 2024-01-24T23:24:58.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.293+0000 7f65618e6640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:58.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.293+0000 7f65618e6640 1 -- 172.21.15.132:0/2735890779 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"} v 0) v1 -- 0x7f6540005b50 con 0x7f655c09ae90 2024-01-24T23:24:58.918 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.909+0000 7f6553fff640 1 -- 172.21.15.132:0/2735890779 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}]=0 set pool 10 size to 3 v119) v1 ==== 126+0+0 (unknown 3167987784 0 0) 0x7f6544013170 con 0x7f655c09ae90 2024-01-24T23:24:58.918 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.909+0000 7f6553fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}] 2024-01-24T23:24:58.918 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:58.909+0000 7f6553fff640 10 monclient: _finish_command 2 = system:0 set pool 10 size to 3 2024-01-24T23:24:59.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:59.009+0000 7f65618e6640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}] 2024-01-24T23:24:59.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:59.009+0000 7f65618e6640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}] 2024-01-24T23:24:59.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:59.009+0000 7f65618e6640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:24:59.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:59.009+0000 7f65618e6640 1 -- 172.21.15.132:0/2735890779 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"} v 0) v1 -- 0x7f65400055d0 con 0x7f655c09ae90 2024-01-24T23:24:59.916 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:59.913+0000 7f6553fff640 1 -- 172.21.15.132:0/2735890779 <== mon.6 v1:172.21.15.115:6792/0 11 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}]=0 set pool 10 size to 3 v120) v1 ==== 126+0+0 (unknown 1012411657 0 0) 0x7f65440148f0 con 0x7f655c09ae90 2024-01-24T23:24:59.916 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:59.913+0000 7f6553fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}] 2024-01-24T23:24:59.916 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:59.913+0000 7f6553fff640 10 monclient: _finish_command 3 = system:0 set pool 10 size to 3 2024-01-24T23:24:59.916 INFO:tasks.workunit.client.0.smithi132.stderr:set pool 10 size to 3 2024-01-24T23:24:59.922 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:59.917+0000 7f65618e6640 1 -- 172.21.15.132:0/2735890779 >> v1:172.21.15.132:6812/1835401922 conn(0x7f653c037750 legacy=0x7f653c039c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:59.923 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:59.917+0000 7f65618e6640 10 monclient: shutdown 2024-01-24T23:24:59.923 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:59.917+0000 7f65618e6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:24:59.923 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:59.917+0000 7f65618e6640 1 -- 172.21.15.132:0/2735890779 >> v1:172.21.15.115:6792/0 conn(0x7f655c09ae90 legacy=0x7f655c1d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:24:59.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:59.921+0000 7f65618e6640 1 -- 172.21.15.132:0/2735890779 shutdown_connections 2024-01-24T23:24:59.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:59.921+0000 7f65618e6640 1 -- 172.21.15.132:0/2735890779 >> 172.21.15.132:0/2735890779 conn(0x7f655c0a1bc0 msgr2=0x7f655c1bc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:24:59.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:59.921+0000 7f65618e6640 1 -- 172.21.15.132:0/2735890779 shutdown_connections 2024-01-24T23:24:59.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:24:59.921+0000 7f65618e6640 1 -- 172.21.15.132:0/2735890779 wait complete. 2024-01-24T23:24:59.945 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd crush weight-set reweight fooset osd.0 .9 2024-01-24T23:24:59.945 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:24:59.945 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set reweight fooset osd.0 .9 2024-01-24T23:25:00.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.065+0000 7fba6d873640 10 monclient: get_monmap_and_config 2024-01-24T23:25:00.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.065+0000 7fba6d873640 10 monclient: build_initial_monmap 2024-01-24T23:25:00.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.065+0000 7fba6d873640 10 monclient: monmap: 2024-01-24T23:25:00.070 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:00.070 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:00.070 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:00.071114+0000 2024-01-24T23:25:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:00.071114+0000 2024-01-24T23:25:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:00.071 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:00.072 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:00.072 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:00.072 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:00.072 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:00.072 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:00.083 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.065+0000 7fba6d873640 1 Processor -- start 2024-01-24T23:25:00.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.065+0000 7fba6d873640 1 -- start start 2024-01-24T23:25:00.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.065+0000 7fba6d873640 10 monclient: init 2024-01-24T23:25:00.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:00.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 10 monclient: _add_conns ranks=[4,2,5,6,7,0,1,8,3] 2024-01-24T23:25:00.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 10 monclient(hunting): picked mon.noname-c con 0x7fba68133ed0 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:00.085 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 10 monclient(hunting): picked mon.noname-b con 0x7fba681342b0 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:00.085 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 10 monclient(hunting): picked mon.noname-h con 0x7fba68134690 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:00.085 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba68064700 con 0x7fba681342b0 2024-01-24T23:25:00.085 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba6809b3d0 con 0x7fba68133ed0 2024-01-24T23:25:00.085 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba680a58c0 con 0x7fba68134690 2024-01-24T23:25:00.085 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:00.085 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 10 monclient(hunting): authenticate will time out at 1187.396642s 2024-01-24T23:25:00.085 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba677fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fba68134690 0x7fba6813f770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:35308/0 (socket says 172.21.15.132:35308) 2024-01-24T23:25:00.086 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba677fe640 1 -- 172.21.15.132:0/86319440 learned_addr learned my addr 172.21.15.132:0/86319440 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:00.086 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba66ffd640 1 -- 172.21.15.132:0/86319440 >> v1:172.21.15.115:6791/0 conn(0x7fba68133ed0 legacy=0x7fba6809bdb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:25:00.086 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 1 -- 172.21.15.132:0/86319440 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1950035071 0 0) 0x7fba680a58c0 con 0x7fba68134690 2024-01-24T23:25:00.086 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:00.086 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:00.086 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:00.087 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 10 monclient(hunting): my global_id is 5693 2024-01-24T23:25:00.087 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 1 -- 172.21.15.132:0/86319440 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba50003530 con 0x7fba68134690 2024-01-24T23:25:00.087 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 1 -- 172.21.15.132:0/86319440 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2294038966 0 0) 0x7fba50003530 con 0x7fba68134690 2024-01-24T23:25:00.088 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 1 -- 172.21.15.132:0/86319440 >> v1:172.21.15.115:6791/0 conn(0x7fba68133ed0 legacy=0x7fba6809bdb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:00.088 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 1 -- 172.21.15.132:0/86319440 >> v1:172.21.15.115:6790/0 conn(0x7fba681342b0 legacy=0x7fba6813c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:00.088 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:25:00.088 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 1 monclient: found mon.noname-h 2024-01-24T23:25:00.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:00.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:25:00.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 1 -- 172.21.15.132:0/86319440 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6809b0f0 con 0x7fba68134690 2024-01-24T23:25:00.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 10 monclient: _finish_auth 0 2024-01-24T23:25:00.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:25:00.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:00.090 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 5 monclient: authenticate success, global_id 5693 2024-01-24T23:25:00.090 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:00.090 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 1 -- 172.21.15.132:0/86319440 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fba60001b40 con 0x7fba68134690 2024-01-24T23:25:00.090 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:00.090 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:25:00.090 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 10 monclient: dump: 2024-01-24T23:25:00.091 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:00.091 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:00.091 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:00.091 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:00.091 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:00.091 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:00.092 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:00.092 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:00.092 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:00.092 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:00.092 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:00.093 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:00.093 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:00.093 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:00.093 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:00.093 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:00.093 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:00.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 1 -- 172.21.15.132:0/86319440 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fba60001e60 con 0x7fba68134690 2024-01-24T23:25:00.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:00.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:00.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 4 set_mon_vals no callback set 2024-01-24T23:25:00.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 1 -- 172.21.15.132:0/86319440 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fba60004f70 con 0x7fba68134690 2024-01-24T23:25:00.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:00.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:00.095 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba65ffb640 10 monclient: dump: 2024-01-24T23:25:00.095 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:00.095 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:00.095 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:00.095 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:00.096 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:00.096 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:00.096 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:00.096 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:00.096 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:00.096 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:00.097 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:00.097 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:00.097 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:00.097 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:00.097 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:00.097 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:00.098 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 10 monclient: shutdown 2024-01-24T23:25:00.098 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:00.098 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 1 -- 172.21.15.132:0/86319440 >> v1:172.21.15.132:6791/0 conn(0x7fba68134690 legacy=0x7fba6813f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:00.098 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 1 -- 172.21.15.132:0/86319440 shutdown_connections 2024-01-24T23:25:00.098 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 1 -- 172.21.15.132:0/86319440 >> 172.21.15.132:0/86319440 conn(0x7fba680a5ef0 msgr2=0x7fba680a6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:00.098 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 1 -- 172.21.15.132:0/86319440 shutdown_connections 2024-01-24T23:25:00.099 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 1 -- 172.21.15.132:0/86319440 wait complete. 2024-01-24T23:25:00.099 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 10 monclient: build_initial_monmap 2024-01-24T23:25:00.099 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 10 monclient: monmap: 2024-01-24T23:25:00.099 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:00.099 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:00.099 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:00.100 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:00.100 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:00.100 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:00.100 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:00.100 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:00.100 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:00.100 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:00.100 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:00.101 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:00.101 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:00.101 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:00.101 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:00.101 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:00.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 1 Processor -- start 2024-01-24T23:25:00.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 1 -- start start 2024-01-24T23:25:00.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 10 monclient: init 2024-01-24T23:25:00.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:00.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 10 monclient: _add_conns ranks=[5,6,3,7,1,4,2,0,8] 2024-01-24T23:25:00.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 10 monclient(hunting): picked mon.noname-h con 0x7fba68133ed0 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:00.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 10 monclient(hunting): picked mon.noname-d con 0x7fba681342b0 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:00.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 10 monclient(hunting): picked mon.noname-g con 0x7fba68134690 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:00.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba680a2390 con 0x7fba68134690 2024-01-24T23:25:00.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba6811fbc0 con 0x7fba68133ed0 2024-01-24T23:25:00.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba681206a0 con 0x7fba681342b0 2024-01-24T23:25:00.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:00.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba6d873640 10 monclient(hunting): authenticate will time out at 1187.399795s 2024-01-24T23:25:00.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba66ffd640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fba68133ed0 0x7fba68132cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:35312/0 (socket says 172.21.15.132:35312) 2024-01-24T23:25:00.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.069+0000 7fba66ffd640 1 -- 172.21.15.132:0/3210205081 learned_addr learned my addr 172.21.15.132:0/3210205081 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:00.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 1 -- 172.21.15.132:0/3210205081 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1065849796 0 0) 0x7fba58003a90 con 0x7fba68133ed0 2024-01-24T23:25:00.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:00.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:00.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:00.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient(hunting): my global_id is 5702 2024-01-24T23:25:00.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 1 -- 172.21.15.132:0/3210205081 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba54003560 con 0x7fba68133ed0 2024-01-24T23:25:00.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 1 -- 172.21.15.132:0/3210205081 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 694610781 0 0) 0x7fba680a2390 con 0x7fba68134690 2024-01-24T23:25:00.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:00.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:00.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:00.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient(hunting): my global_id is 5781 2024-01-24T23:25:00.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 1 -- 172.21.15.132:0/3210205081 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba58003a90 con 0x7fba68134690 2024-01-24T23:25:00.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 1 -- 172.21.15.132:0/3210205081 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1313624438 0 0) 0x7fba58003a90 con 0x7fba68134690 2024-01-24T23:25:00.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 1 -- 172.21.15.132:0/3210205081 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fba680a2390 con 0x7fba68134690 2024-01-24T23:25:00.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 1 -- 172.21.15.132:0/3210205081 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3600653632 0 0) 0x7fba54003560 con 0x7fba68133ed0 2024-01-24T23:25:00.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 1 -- 172.21.15.132:0/3210205081 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fba58003a90 con 0x7fba68133ed0 2024-01-24T23:25:00.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 1 -- 172.21.15.132:0/3210205081 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fba60001b00 con 0x7fba68134690 2024-01-24T23:25:00.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:00.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:25:00.105 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient(hunting): dump: 2024-01-24T23:25:00.105 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:00.105 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:00.105 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:00.105 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:00.105 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:00.105 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:00.105 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:00.106 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:00.106 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:00.106 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:00.106 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:00.106 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:00.106 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:00.106 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:00.106 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:00.106 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:00.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:00.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 1 -- 172.21.15.132:0/3210205081 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fba580044c0 con 0x7fba68133ed0 2024-01-24T23:25:00.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:00.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:00.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient(hunting): dump: 2024-01-24T23:25:00.107 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:00.107 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:00.107 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:00.107 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:00.107 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:00.107 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:00.107 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:00.107 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:00.107 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:00.108 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:00.108 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:00.108 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:00.108 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:00.108 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:00.108 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:00.108 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:00.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 1 -- 172.21.15.132:0/3210205081 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432085288 0 0) 0x7fba681206a0 con 0x7fba681342b0 2024-01-24T23:25:00.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:00.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:00.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:00.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient(hunting): my global_id is 5874 2024-01-24T23:25:00.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 1 -- 172.21.15.132:0/3210205081 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba54003560 con 0x7fba681342b0 2024-01-24T23:25:00.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 1 -- 172.21.15.132:0/3210205081 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1927966761 0 0) 0x7fba680a2390 con 0x7fba68134690 2024-01-24T23:25:00.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 1 -- 172.21.15.132:0/3210205081 >> v1:172.21.15.115:6792/0 conn(0x7fba681342b0 legacy=0x7fba681d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:00.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 1 -- 172.21.15.132:0/3210205081 >> v1:172.21.15.132:6791/0 conn(0x7fba68133ed0 legacy=0x7fba68132cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:00.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient: _finish_hunting 0 2024-01-24T23:25:00.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 1 monclient: found mon.g 2024-01-24T23:25:00.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:00.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:00.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 1 -- 172.21.15.132:0/3210205081 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6809ca90 con 0x7fba68134690 2024-01-24T23:25:00.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient: _finish_auth 0 2024-01-24T23:25:00.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient: _check_auth_tickets 2024-01-24T23:25:00.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:00.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba6d873640 5 monclient: authenticate success, global_id 5781 2024-01-24T23:25:00.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba6d873640 10 monclient: _renew_subs 2024-01-24T23:25:00.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba6d873640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:00.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba6d873640 1 -- 172.21.15.132:0/3210205081 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba680a5af0 con 0x7fba68134690 2024-01-24T23:25:00.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba6d873640 10 monclient: _renew_subs 2024-01-24T23:25:00.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba6d873640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:00.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba6d873640 1 -- 172.21.15.132:0/3210205081 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fba680a2c70 con 0x7fba68134690 2024-01-24T23:25:00.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 1 -- 172.21.15.132:0/3210205081 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fba60001fd0 con 0x7fba68134690 2024-01-24T23:25:00.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:00.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 1 -- 172.21.15.132:0/3210205081 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fba60005450 con 0x7fba68134690 2024-01-24T23:25:00.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:00.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:00.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba657fa640 4 set_mon_vals no callback set 2024-01-24T23:25:00.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 10 monclient: dump: 2024-01-24T23:25:00.111 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:00.111 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:00.111 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:00.111 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:00.111 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:00.111 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:00.111 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:00.111 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:00.111 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:00.111 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:00.112 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:00.112 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:00.112 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:00.112 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:00.112 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:00.112 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:00.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba6d873640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:00.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba6d873640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:00.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba6d873640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:00.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba6d873640 1 -- 172.21.15.132:0/3210205081 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba48005090 con 0x7fba68134690 2024-01-24T23:25:00.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 1 -- 172.21.15.132:0/3210205081 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fba6000fed0 con 0x7fba68134690 2024-01-24T23:25:00.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba4ffff640 1 -- 172.21.15.132:0/3210205081 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(120..120 src has 1..120) v4 ==== 6377+0+0 (unknown 3668014824 0 0) 0x7fba600445a0 con 0x7fba68134690 2024-01-24T23:25:00.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.073+0000 7fba66ffd640 10 monclient: get_auth_request con 0x7fba54037800 auth_method 0 2024-01-24T23:25:00.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.077+0000 7fba4ffff640 1 -- 172.21.15.132:0/3210205081 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fba60014c70 con 0x7fba68134690 2024-01-24T23:25:00.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.077+0000 7fba4ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:00.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.077+0000 7fba4ffff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:00.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.241+0000 7fba6d873640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-01-24T23:25:00.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.241+0000 7fba6d873640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-01-24T23:25:00.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.241+0000 7fba6d873640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:00.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.241+0000 7fba6d873640 1 -- 172.21.15.132:0/3210205081 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]} v 0) v1 -- 0x7fba480058c0 con 0x7fba68134690 2024-01-24T23:25:00.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.241+0000 7fba4ffff640 1 -- 172.21.15.132:0/3210205081 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}]=-2 no weight-set for pool 'fooset' v120) v1 ==== 160+0+0 (unknown 3468212976 0 0) 0x7fba60013310 con 0x7fba68134690 2024-01-24T23:25:00.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.241+0000 7fba4ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-01-24T23:25:00.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.241+0000 7fba4ffff640 10 monclient: _finish_command 2 = mon:2 no weight-set for pool 'fooset' 2024-01-24T23:25:00.246 INFO:tasks.workunit.client.0.smithi132.stderr:Error ENOENT: no weight-set for pool 'fooset' 2024-01-24T23:25:00.251 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:25:00.251 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set create fooset flat 2024-01-24T23:25:00.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.365+0000 7f94d6ba6640 10 monclient: get_monmap_and_config 2024-01-24T23:25:00.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.365+0000 7f94d6ba6640 10 monclient: build_initial_monmap 2024-01-24T23:25:00.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.365+0000 7f94d6ba6640 10 monclient: monmap: 2024-01-24T23:25:00.372 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:00.372 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:00.372 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:00.372701+0000 2024-01-24T23:25:00.372 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:00.372701+0000 2024-01-24T23:25:00.372 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:00.372 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:00.372 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:00.372 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:00.373 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:00.373 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:00.373 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:00.373 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:00.373 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:00.373 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:00.373 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:00.373 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:00.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 1 Processor -- start 2024-01-24T23:25:00.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 1 -- start start 2024-01-24T23:25:00.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 10 monclient: init 2024-01-24T23:25:00.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:00.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 10 monclient: _add_conns ranks=[4,3,8,7,5,1,2,0,6] 2024-01-24T23:25:00.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 10 monclient(hunting): picked mon.noname-c con 0x7f94d0072690 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:00.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 10 monclient(hunting): picked mon.noname-g con 0x7f94d009ae90 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:00.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 10 monclient(hunting): picked mon.noname-e con 0x7f94d009bdc0 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:00.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d0064700 con 0x7f94d009ae90 2024-01-24T23:25:00.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d009a790 con 0x7f94d0072690 2024-01-24T23:25:00.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d00a18f0 con 0x7f94d009bdc0 2024-01-24T23:25:00.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:00.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 10 monclient(hunting): authenticate will time out at 1187.697271s 2024-01-24T23:25:00.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cffff640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f94d009ae90 0x7f94d00a11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:37468/0 (socket says 172.21.15.132:37468) 2024-01-24T23:25:00.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cffff640 1 -- 172.21.15.132:0/3701644843 learned_addr learned my addr 172.21.15.132:0/3701644843 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:00.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d4942640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f94d0072690 legacy=0x7f94d00a0ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:25:00.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 1 -- 172.21.15.132:0/3701644843 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637044707 0 0) 0x7f94d0064700 con 0x7f94d009ae90 2024-01-24T23:25:00.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:00.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:00.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:00.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 10 monclient(hunting): my global_id is 5790 2024-01-24T23:25:00.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 1 -- 172.21.15.132:0/3701644843 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94b8003530 con 0x7f94d009ae90 2024-01-24T23:25:00.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 1 -- 172.21.15.132:0/3701644843 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2172222530 0 0) 0x7f94b8003530 con 0x7f94d009ae90 2024-01-24T23:25:00.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 1 -- 172.21.15.132:0/3701644843 >> v1:172.21.15.115:6793/0 conn(0x7f94d009bdc0 legacy=0x7f94d013d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:00.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 1 -- 172.21.15.132:0/3701644843 >> v1:172.21.15.115:6791/0 conn(0x7f94d0072690 legacy=0x7f94d00a0ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:00.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:25:00.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 1 monclient: found mon.noname-g 2024-01-24T23:25:00.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:00.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:25:00.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 1 -- 172.21.15.132:0/3701644843 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d00997a0 con 0x7f94d009ae90 2024-01-24T23:25:00.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 10 monclient: _finish_auth 0 2024-01-24T23:25:00.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:25:00.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:00.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 5 monclient: authenticate success, global_id 5790 2024-01-24T23:25:00.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 1 -- 172.21.15.132:0/3701644843 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f94c40017c0 con 0x7f94d009ae90 2024-01-24T23:25:00.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:00.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:00.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:25:00.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 10 monclient: dump: 2024-01-24T23:25:00.381 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:00.381 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:00.381 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:00.381 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:00.381 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:00.381 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:00.381 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:00.381 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:00.381 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:00.381 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:00.381 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:00.381 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:00.381 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:00.381 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:00.381 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:00.382 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:00.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:00.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 1 -- 172.21.15.132:0/3701644843 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f94c4001a70 con 0x7f94d009ae90 2024-01-24T23:25:00.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:00.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:00.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 4 set_mon_vals no callback set 2024-01-24T23:25:00.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 1 -- 172.21.15.132:0/3701644843 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f94c4004bf0 con 0x7f94d009ae90 2024-01-24T23:25:00.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:00.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:00.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94cf7fe640 10 monclient: dump: 2024-01-24T23:25:00.382 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:00.382 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:00.382 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:00.382 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:00.382 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:00.383 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:00.383 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:00.383 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:00.383 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:00.383 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:00.383 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:00.383 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:00.383 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:00.383 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:00.383 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:00.383 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:00.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 10 monclient: shutdown 2024-01-24T23:25:00.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:00.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 1 -- 172.21.15.132:0/3701644843 >> v1:172.21.15.132:6790/0 conn(0x7f94d009ae90 legacy=0x7f94d00a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:00.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 1 -- 172.21.15.132:0/3701644843 shutdown_connections 2024-01-24T23:25:00.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 1 -- 172.21.15.132:0/3701644843 >> 172.21.15.132:0/3701644843 conn(0x7f94d00a1bc0 msgr2=0x7f94d00a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:00.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 1 -- 172.21.15.132:0/3701644843 shutdown_connections 2024-01-24T23:25:00.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 1 -- 172.21.15.132:0/3701644843 wait complete. 2024-01-24T23:25:00.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 10 monclient: build_initial_monmap 2024-01-24T23:25:00.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 10 monclient: monmap: 2024-01-24T23:25:00.384 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:00.384 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:00.384 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:00.384 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:00.384 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:00.384 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:00.384 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:00.385 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:00.385 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:00.385 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:00.385 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:00.385 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:00.385 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:00.385 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:00.385 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:00.385 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:00.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 1 Processor -- start 2024-01-24T23:25:00.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 1 -- start start 2024-01-24T23:25:00.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.369+0000 7f94d6ba6640 10 monclient: init 2024-01-24T23:25:00.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d6ba6640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:00.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d6ba6640 10 monclient: _add_conns ranks=[4,6,5,7,2,0,3,1,8] 2024-01-24T23:25:00.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d6ba6640 10 monclient(hunting): picked mon.noname-c con 0x7f94d0072690 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:00.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d6ba6640 10 monclient(hunting): picked mon.noname-d con 0x7f94d009ae90 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:00.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d6ba6640 10 monclient(hunting): picked mon.noname-h con 0x7f94d009bdc0 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:00.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d6ba6640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d00994e0 con 0x7f94d0072690 2024-01-24T23:25:00.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d6ba6640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d01b45f0 con 0x7f94d009bdc0 2024-01-24T23:25:00.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d6ba6640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d01b2570 con 0x7f94d009ae90 2024-01-24T23:25:00.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d6ba6640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:00.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d6ba6640 10 monclient(hunting): authenticate will time out at 1187.700250s 2024-01-24T23:25:00.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d4942640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f94d0072690 legacy=0x7f94d01cb4f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:25:00.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d5143640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f94d009bdc0 0x7f94d01d37b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:35326/0 (socket says 172.21.15.132:35326) 2024-01-24T23:25:00.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d5143640 1 -- 172.21.15.132:0/4251549947 learned_addr learned my addr 172.21.15.132:0/4251549947 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:00.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 1 -- 172.21.15.132:0/4251549947 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 425327644 0 0) 0x7f94d01b45f0 con 0x7f94d009bdc0 2024-01-24T23:25:00.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:25:00.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:00.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:00.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 10 monclient(hunting): my global_id is 5711 2024-01-24T23:25:00.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 1 -- 172.21.15.132:0/4251549947 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94bc003530 con 0x7f94d009bdc0 2024-01-24T23:25:00.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 1 -- 172.21.15.132:0/4251549947 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3550692347 0 0) 0x7f94bc003530 con 0x7f94d009bdc0 2024-01-24T23:25:00.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 1 -- 172.21.15.132:0/4251549947 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f94d01b45f0 con 0x7f94d009bdc0 2024-01-24T23:25:00.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 1 -- 172.21.15.132:0/4251549947 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f94c80019f0 con 0x7f94d009bdc0 2024-01-24T23:25:00.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:00.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:25:00.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 10 monclient(hunting): dump: 2024-01-24T23:25:00.387 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:00.387 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:00.387 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:00.388 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:00.388 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:00.388 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:00.388 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:00.388 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:00.388 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:00.388 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:00.388 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:00.388 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:00.388 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:00.388 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:00.388 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:00.389 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:00.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:00.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 1 -- 172.21.15.132:0/4251549947 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2786589344 0 0) 0x7f94d01b45f0 con 0x7f94d009bdc0 2024-01-24T23:25:00.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 1 -- 172.21.15.132:0/4251549947 >> v1:172.21.15.115:6792/0 conn(0x7f94d009ae90 legacy=0x7f94d01cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:00.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 1 -- 172.21.15.132:0/4251549947 >> v1:172.21.15.115:6791/0 conn(0x7f94d0072690 legacy=0x7f94d01cb4f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:00.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 10 monclient: _finish_hunting 0 2024-01-24T23:25:00.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 1 monclient: found mon.h 2024-01-24T23:25:00.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:00.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:00.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 1 -- 172.21.15.132:0/4251549947 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d009b930 con 0x7f94d009bdc0 2024-01-24T23:25:00.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 10 monclient: _finish_auth 0 2024-01-24T23:25:00.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:25:00.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:00.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d6ba6640 5 monclient: authenticate success, global_id 5711 2024-01-24T23:25:00.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d6ba6640 10 monclient: _renew_subs 2024-01-24T23:25:00.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d6ba6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:00.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d6ba6640 1 -- 172.21.15.132:0/4251549947 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d01b32d0 con 0x7f94d009bdc0 2024-01-24T23:25:00.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d6ba6640 10 monclient: _renew_subs 2024-01-24T23:25:00.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d6ba6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:00.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d6ba6640 1 -- 172.21.15.132:0/4251549947 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d01b52b0 con 0x7f94d009bdc0 2024-01-24T23:25:00.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 1 -- 172.21.15.132:0/4251549947 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f94c80038d0 con 0x7f94d009bdc0 2024-01-24T23:25:00.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:00.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 1 -- 172.21.15.132:0/4251549947 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f94c8003e00 con 0x7f94d009bdc0 2024-01-24T23:25:00.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:00.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94ceffd640 4 set_mon_vals no callback set 2024-01-24T23:25:00.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:00.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 10 monclient: dump: 2024-01-24T23:25:00.391 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:00.391 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:00.391 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:00.391 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:00.391 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:00.391 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:00.391 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:00.391 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:00.391 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:00.391 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:00.391 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:00.391 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:00.391 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:00.391 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:00.392 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:00.392 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:00.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d6ba6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:00.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d6ba6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:00.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d6ba6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:00.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 1 -- 172.21.15.132:0/4251549947 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f94c8003fe0 con 0x7f94d009bdc0 2024-01-24T23:25:00.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d6ba6640 1 -- 172.21.15.132:0/4251549947 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b0005090 con 0x7f94d009bdc0 2024-01-24T23:25:00.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94cd7fa640 1 -- 172.21.15.132:0/4251549947 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(120..120 src has 1..120) v4 ==== 6377+0+0 (unknown 3668014824 0 0) 0x7f94c80442a0 con 0x7f94d009bdc0 2024-01-24T23:25:00.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.373+0000 7f94d4942640 10 monclient: get_auth_request con 0x7f94bc036d50 auth_method 0 2024-01-24T23:25:00.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.377+0000 7f94cd7fa640 1 -- 172.21.15.132:0/4251549947 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f94c8011480 con 0x7f94d009bdc0 2024-01-24T23:25:00.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.377+0000 7f94cd7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:00.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.377+0000 7f94cd7fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:00.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.541+0000 7f94d6ba6640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}] 2024-01-24T23:25:00.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.541+0000 7f94d6ba6640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}] 2024-01-24T23:25:00.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.541+0000 7f94d6ba6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:00.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.541+0000 7f94d6ba6640 1 -- 172.21.15.132:0/4251549947 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"} v 0) v1 -- 0x7f94b00058c0 con 0x7f94d009bdc0 2024-01-24T23:25:00.916 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.913+0000 7f94cd7fa640 1 -- 172.21.15.132:0/4251549947 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}]=0 v121) v1 ==== 109+0+0 (unknown 3926049523 0 0) 0x7f94c8001d10 con 0x7f94d009bdc0 2024-01-24T23:25:00.916 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.913+0000 7f94cd7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}] 2024-01-24T23:25:00.916 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:00.913+0000 7f94cd7fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:01.027 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.021+0000 7f94d6ba6640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}] 2024-01-24T23:25:01.027 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.021+0000 7f94d6ba6640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}] 2024-01-24T23:25:01.027 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.021+0000 7f94d6ba6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:01.027 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.021+0000 7f94d6ba6640 1 -- 172.21.15.132:0/4251549947 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"} v 0) v1 -- 0x7f94b0002220 con 0x7f94d009bdc0 2024-01-24T23:25:01.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.025+0000 7f94cd7fa640 1 -- 172.21.15.132:0/4251549947 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}]=0 weight-set for pool 'fooset' already created v121) v1 ==== 153+0+0 (unknown 931694282 0 0) 0x7f94c8012f50 con 0x7f94d009bdc0 2024-01-24T23:25:01.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.025+0000 7f94cd7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}] 2024-01-24T23:25:01.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.025+0000 7f94cd7fa640 10 monclient: _finish_command 3 = system:0 weight-set for pool 'fooset' already created 2024-01-24T23:25:01.028 INFO:tasks.workunit.client.0.smithi132.stderr:weight-set for pool 'fooset' already created 2024-01-24T23:25:01.030 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.025+0000 7f94d6ba6640 1 -- 172.21.15.132:0/4251549947 >> v1:172.21.15.132:6812/1835401922 conn(0x7f94bc036d50 legacy=0x7f94bc039210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:01.030 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.025+0000 7f94d6ba6640 10 monclient: shutdown 2024-01-24T23:25:01.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.025+0000 7f94d6ba6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:01.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.025+0000 7f94d6ba6640 1 -- 172.21.15.132:0/4251549947 >> v1:172.21.15.132:6791/0 conn(0x7f94d009bdc0 legacy=0x7f94d01d37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:01.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.025+0000 7f94d6ba6640 1 -- 172.21.15.132:0/4251549947 shutdown_connections 2024-01-24T23:25:01.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.025+0000 7f94d6ba6640 1 -- 172.21.15.132:0/4251549947 >> 172.21.15.132:0/4251549947 conn(0x7f94d00a1bc0 msgr2=0x7f94d01bc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:01.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.025+0000 7f94d6ba6640 1 -- 172.21.15.132:0/4251549947 shutdown_connections 2024-01-24T23:25:01.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.025+0000 7f94d6ba6640 1 -- 172.21.15.132:0/4251549947 wait complete. 2024-01-24T23:25:01.046 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set create barset positional 2024-01-24T23:25:01.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.161+0000 7fcc6d09a640 10 monclient: get_monmap_and_config 2024-01-24T23:25:01.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.161+0000 7fcc6d09a640 10 monclient: build_initial_monmap 2024-01-24T23:25:01.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.161+0000 7fcc6d09a640 10 monclient: monmap: 2024-01-24T23:25:01.165 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:01.165 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:01.165 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:01.166382+0000 2024-01-24T23:25:01.165 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:01.166382+0000 2024-01-24T23:25:01.165 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:01.165 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:01.166 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:01.166 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:01.166 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:01.166 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:01.166 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:01.166 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:01.166 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:01.166 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:01.166 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:01.166 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:01.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.161+0000 7fcc6d09a640 1 Processor -- start 2024-01-24T23:25:01.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.161+0000 7fcc6d09a640 1 -- start start 2024-01-24T23:25:01.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.161+0000 7fcc6d09a640 10 monclient: init 2024-01-24T23:25:01.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.161+0000 7fcc6d09a640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:01.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.161+0000 7fcc6d09a640 10 monclient: _add_conns ranks=[0,5,2,8,3,6,7,4,1] 2024-01-24T23:25:01.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.161+0000 7fcc6d09a640 10 monclient(hunting): picked mon.noname-a con 0x7fcc68134f20 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:01.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.161+0000 7fcc6d09a640 10 monclient(hunting): picked mon.noname-h con 0x7fcc681347c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:01.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.161+0000 7fcc6d09a640 10 monclient(hunting): picked mon.noname-b con 0x7fcc68135300 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:01.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.161+0000 7fcc6d09a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc68064700 con 0x7fcc68134f20 2024-01-24T23:25:01.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.161+0000 7fcc6d09a640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc68071390 con 0x7fcc68135300 2024-01-24T23:25:01.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.161+0000 7fcc6d09a640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc680a21e0 con 0x7fcc681347c0 2024-01-24T23:25:01.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.161+0000 7fcc6d09a640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:01.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.161+0000 7fcc6d09a640 10 monclient(hunting): authenticate will time out at 1188.491920s 2024-01-24T23:25:01.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc5ffff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fcc681347c0 0x7fcc680a1ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:35340/0 (socket says 172.21.15.132:35340) 2024-01-24T23:25:01.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc5ffff640 1 -- 172.21.15.132:0/2455181425 learned_addr learned my addr 172.21.15.132:0/2455181425 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:01.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc66d9d640 1 -- 172.21.15.132:0/2455181425 >> v1:172.21.15.115:6789/0 conn(0x7fcc68134f20 legacy=0x7fcc680a13c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:25:01.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 1 -- 172.21.15.132:0/2455181425 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3458254060 0 0) 0x7fcc680a21e0 con 0x7fcc681347c0 2024-01-24T23:25:01.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 10 monclient(hunting): _negotiate 2024-01-24T23:25:01.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:01.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:01.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 10 monclient(hunting): my global_id is 5720 2024-01-24T23:25:01.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 1 -- 172.21.15.132:0/2455181425 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc54003530 con 0x7fcc681347c0 2024-01-24T23:25:01.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 1 -- 172.21.15.132:0/2455181425 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2462447126 0 0) 0x7fcc54003530 con 0x7fcc681347c0 2024-01-24T23:25:01.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 1 -- 172.21.15.132:0/2455181425 >> v1:172.21.15.115:6790/0 conn(0x7fcc68135300 legacy=0x7fcc6813db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:01.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 1 -- 172.21.15.132:0/2455181425 >> v1:172.21.15.115:6789/0 conn(0x7fcc68134f20 legacy=0x7fcc680a13c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:01.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 10 monclient: _finish_hunting 0 2024-01-24T23:25:01.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 1 monclient: found mon.noname-h 2024-01-24T23:25:01.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:01.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:25:01.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 1 -- 172.21.15.132:0/2455181425 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc680710b0 con 0x7fcc681347c0 2024-01-24T23:25:01.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 10 monclient: _finish_auth 0 2024-01-24T23:25:01.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 10 monclient: _check_auth_tickets 2024-01-24T23:25:01.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:01.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 1 -- 172.21.15.132:0/2455181425 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fcc500017c0 con 0x7fcc681347c0 2024-01-24T23:25:01.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:01.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:25:01.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 10 monclient: dump: 2024-01-24T23:25:01.177 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:01.177 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:01.177 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:01.177 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:01.177 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:01.177 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:01.177 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:01.177 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:01.177 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:01.177 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:01.178 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:01.178 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:01.178 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:01.178 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:01.178 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:01.178 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:01.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 5 monclient: authenticate success, global_id 5720 2024-01-24T23:25:01.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:01.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 1 -- 172.21.15.132:0/2455181425 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fcc50001a70 con 0x7fcc681347c0 2024-01-24T23:25:01.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:01.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 1 -- 172.21.15.132:0/2455181425 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fcc50004bf0 con 0x7fcc681347c0 2024-01-24T23:25:01.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:01.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 4 set_mon_vals no callback set 2024-01-24T23:25:01.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:01.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:01.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6659c640 10 monclient: dump: 2024-01-24T23:25:01.179 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:01.179 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:01.179 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:01.179 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:01.179 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:01.179 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:01.179 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:01.179 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:01.179 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:01.179 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:01.179 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:01.179 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:01.179 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:01.180 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:01.180 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:01.180 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:01.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 10 monclient: shutdown 2024-01-24T23:25:01.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:01.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 1 -- 172.21.15.132:0/2455181425 >> v1:172.21.15.132:6791/0 conn(0x7fcc681347c0 legacy=0x7fcc680a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:01.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 1 -- 172.21.15.132:0/2455181425 shutdown_connections 2024-01-24T23:25:01.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 1 -- 172.21.15.132:0/2455181425 >> 172.21.15.132:0/2455181425 conn(0x7fcc680a24b0 msgr2=0x7fcc680a28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:01.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 1 -- 172.21.15.132:0/2455181425 shutdown_connections 2024-01-24T23:25:01.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 1 -- 172.21.15.132:0/2455181425 wait complete. 2024-01-24T23:25:01.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 10 monclient: build_initial_monmap 2024-01-24T23:25:01.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 10 monclient: monmap: 2024-01-24T23:25:01.180 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:01.180 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:01.181 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:01.181 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:01.181 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:01.181 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:01.181 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:01.181 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:01.181 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:01.181 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:01.181 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:01.181 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:01.181 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:01.181 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:01.181 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:01.181 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:01.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 1 Processor -- start 2024-01-24T23:25:01.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 1 -- start start 2024-01-24T23:25:01.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 10 monclient: init 2024-01-24T23:25:01.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:01.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 10 monclient: _add_conns ranks=[8,2,0,3,4,6,7,5,1] 2024-01-24T23:25:01.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 10 monclient(hunting): picked mon.noname-e con 0x7fcc681347c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:01.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 10 monclient(hunting): picked mon.noname-b con 0x7fcc68134f20 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:01.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 10 monclient(hunting): picked mon.noname-a con 0x7fcc68135300 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:01.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc68134140 con 0x7fcc68135300 2024-01-24T23:25:01.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc6806de10 con 0x7fcc68134f20 2024-01-24T23:25:01.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc6806db50 con 0x7fcc681347c0 2024-01-24T23:25:01.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:01.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6d09a640 10 monclient(hunting): authenticate will time out at 1188.495511s 2024-01-24T23:25:01.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc6759e640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fcc68135300 legacy=0x7fcc681d1020 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:25:01.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc66d9d640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fcc681347c0 0x7fcc681c8eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:40646/0 (socket says 172.21.15.132:40646) 2024-01-24T23:25:01.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.165+0000 7fcc66d9d640 1 -- 172.21.15.132:0/1750737263 learned_addr learned my addr 172.21.15.132:0/1750737263 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:01.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 1 -- 172.21.15.132:0/1750737263 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1169063986 0 0) 0x7fcc6806de10 con 0x7fcc68134f20 2024-01-24T23:25:01.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:01.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:01.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:01.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 10 monclient(hunting): my global_id is 85097 2024-01-24T23:25:01.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 1 -- 172.21.15.132:0/1750737263 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc48003530 con 0x7fcc68134f20 2024-01-24T23:25:01.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 1 -- 172.21.15.132:0/1750737263 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2263011294 0 0) 0x7fcc6806db50 con 0x7fcc681347c0 2024-01-24T23:25:01.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:01.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:01.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:01.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 10 monclient(hunting): my global_id is 84968 2024-01-24T23:25:01.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 1 -- 172.21.15.132:0/1750737263 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc6806de10 con 0x7fcc681347c0 2024-01-24T23:25:01.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 1 -- 172.21.15.132:0/1750737263 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 986919131 0 0) 0x7fcc6806de10 con 0x7fcc681347c0 2024-01-24T23:25:01.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 1 -- 172.21.15.132:0/1750737263 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcc6806db50 con 0x7fcc681347c0 2024-01-24T23:25:01.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 1 -- 172.21.15.132:0/1750737263 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fcc58004330 con 0x7fcc681347c0 2024-01-24T23:25:01.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:01.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:25:01.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 10 monclient(hunting): dump: 2024-01-24T23:25:01.184 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:01.184 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:01.184 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:01.184 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:01.184 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:01.185 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:01.185 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:01.185 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:01.185 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:01.185 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:01.185 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:01.185 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:01.185 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:01.185 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:01.185 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:01.185 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:01.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:01.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 1 -- 172.21.15.132:0/1750737263 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4273744493 0 0) 0x7fcc48003530 con 0x7fcc68134f20 2024-01-24T23:25:01.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 1 -- 172.21.15.132:0/1750737263 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcc6806de10 con 0x7fcc68134f20 2024-01-24T23:25:01.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 1 -- 172.21.15.132:0/1750737263 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fcc50001780 con 0x7fcc68134f20 2024-01-24T23:25:01.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:01.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:25:01.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 10 monclient(hunting): dump: 2024-01-24T23:25:01.186 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:01.186 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:01.186 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:01.186 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:01.186 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:01.186 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:01.186 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:01.186 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:01.186 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:01.186 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:01.186 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:01.186 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:01.186 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:01.187 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:01.187 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:01.187 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:01.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 1 -- 172.21.15.132:0/1750737263 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2820534098 0 0) 0x7fcc6806db50 con 0x7fcc681347c0 2024-01-24T23:25:01.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 1 -- 172.21.15.132:0/1750737263 >> v1:172.21.15.115:6790/0 conn(0x7fcc68134f20 legacy=0x7fcc6812e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:01.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 1 -- 172.21.15.132:0/1750737263 >> v1:172.21.15.115:6789/0 conn(0x7fcc68135300 legacy=0x7fcc681d1020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:01.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:25:01.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 1 monclient: found mon.e 2024-01-24T23:25:01.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:01.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:01.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 1 -- 172.21.15.132:0/1750737263 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc68131e50 con 0x7fcc681347c0 2024-01-24T23:25:01.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 10 monclient: _finish_auth 0 2024-01-24T23:25:01.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:25:01.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:01.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc6d09a640 5 monclient: authenticate success, global_id 84968 2024-01-24T23:25:01.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc6d09a640 10 monclient: _renew_subs 2024-01-24T23:25:01.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc6d09a640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:01.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc6d09a640 1 -- 172.21.15.132:0/1750737263 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc68133e60 con 0x7fcc681347c0 2024-01-24T23:25:01.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc6d09a640 10 monclient: _renew_subs 2024-01-24T23:25:01.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc6d09a640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:01.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc6d09a640 1 -- 172.21.15.132:0/1750737263 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6812ec10 con 0x7fcc681347c0 2024-01-24T23:25:01.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 1 -- 172.21.15.132:0/1750737263 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fcc58004900 con 0x7fcc681347c0 2024-01-24T23:25:01.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:01.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 1 -- 172.21.15.132:0/1750737263 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fcc58004e30 con 0x7fcc681347c0 2024-01-24T23:25:01.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:01.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc65d9b640 4 set_mon_vals no callback set 2024-01-24T23:25:01.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:01.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 10 monclient: dump: 2024-01-24T23:25:01.189 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:01.189 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:01.189 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:01.189 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:01.189 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:01.189 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:01.189 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:01.189 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:01.189 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:01.189 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:01.189 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:01.189 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:01.189 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:01.189 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:01.190 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:01.190 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:01.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc6d09a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:01.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc6d09a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:01.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc6d09a640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:01.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc6d09a640 1 -- 172.21.15.132:0/1750737263 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc4c005090 con 0x7fcc681347c0 2024-01-24T23:25:01.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 1 -- 172.21.15.132:0/1750737263 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fcc58005010 con 0x7fcc681347c0 2024-01-24T23:25:01.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5f7fe640 1 -- 172.21.15.132:0/1750737263 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(121..121 src has 1..121) v4 ==== 6709+0+0 (unknown 1882499098 0 0) 0x7fcc58044380 con 0x7fcc681347c0 2024-01-24T23:25:01.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.169+0000 7fcc5ffff640 10 monclient: get_auth_request con 0x7fcc48037620 auth_method 0 2024-01-24T23:25:01.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.181+0000 7fcc5f7fe640 1 -- 172.21.15.132:0/1750737263 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fcc58014840 con 0x7fcc681347c0 2024-01-24T23:25:01.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.181+0000 7fcc5f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:01.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.181+0000 7fcc5f7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:01.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.349+0000 7fcc6d09a640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}] 2024-01-24T23:25:01.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.349+0000 7fcc6d09a640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}] 2024-01-24T23:25:01.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.349+0000 7fcc6d09a640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:01.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:01.349+0000 7fcc6d09a640 1 -- 172.21.15.132:0/1750737263 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"} v 0) v1 -- 0x7fcc4c005340 con 0x7fcc681347c0 2024-01-24T23:25:02.034 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.029+0000 7fcc5f7fe640 1 -- 172.21.15.132:0/1750737263 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}]=0 v122) v1 ==== 115+0+0 (unknown 4272616040 0 0) 0x7fcc58012ee0 con 0x7fcc681347c0 2024-01-24T23:25:02.034 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.029+0000 7fcc5f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}] 2024-01-24T23:25:02.034 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.029+0000 7fcc5f7fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:02.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.133+0000 7fcc6d09a640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}] 2024-01-24T23:25:02.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.133+0000 7fcc6d09a640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}] 2024-01-24T23:25:02.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.133+0000 7fcc6d09a640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:02.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.133+0000 7fcc6d09a640 1 -- 172.21.15.132:0/1750737263 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"} v 0) v1 -- 0x7fcc4c005a80 con 0x7fcc681347c0 2024-01-24T23:25:02.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.137+0000 7fcc5f7fe640 1 -- 172.21.15.132:0/1750737263 <== mon.8 v1:172.21.15.115:6793/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}]=0 weight-set for pool 'barset' already created v122) v1 ==== 159+0+0 (unknown 2211695336 0 0) 0x7fcc58014660 con 0x7fcc681347c0 2024-01-24T23:25:02.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.137+0000 7fcc5f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}] 2024-01-24T23:25:02.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.137+0000 7fcc5f7fe640 10 monclient: _finish_command 3 = system:0 weight-set for pool 'barset' already created 2024-01-24T23:25:02.139 INFO:tasks.workunit.client.0.smithi132.stderr:weight-set for pool 'barset' already created 2024-01-24T23:25:02.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.137+0000 7fcc6d09a640 1 -- 172.21.15.132:0/1750737263 >> v1:172.21.15.132:6812/1835401922 conn(0x7fcc48037620 legacy=0x7fcc48039ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:02.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.137+0000 7fcc6d09a640 10 monclient: shutdown 2024-01-24T23:25:02.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.137+0000 7fcc6d09a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:02.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.137+0000 7fcc6d09a640 1 -- 172.21.15.132:0/1750737263 >> v1:172.21.15.115:6793/0 conn(0x7fcc681347c0 legacy=0x7fcc681c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:02.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.137+0000 7fcc6d09a640 1 -- 172.21.15.132:0/1750737263 shutdown_connections 2024-01-24T23:25:02.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.137+0000 7fcc6d09a640 1 -- 172.21.15.132:0/1750737263 >> 172.21.15.132:0/1750737263 conn(0x7fcc680a24b0 msgr2=0x7fcc680a0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:02.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.137+0000 7fcc6d09a640 1 -- 172.21.15.132:0/1750737263 shutdown_connections 2024-01-24T23:25:02.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.137+0000 7fcc6d09a640 1 -- 172.21.15.132:0/1750737263 wait complete. 2024-01-24T23:25:02.157 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set ls 2024-01-24T23:25:02.157 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep fooset 2024-01-24T23:25:02.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.269+0000 7fd160cc8640 10 monclient: get_monmap_and_config 2024-01-24T23:25:02.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.269+0000 7fd160cc8640 10 monclient: build_initial_monmap 2024-01-24T23:25:02.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.269+0000 7fd160cc8640 10 monclient: monmap: 2024-01-24T23:25:02.278 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:02.278 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:02.278 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:02.275889+0000 2024-01-24T23:25:02.278 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:02.275889+0000 2024-01-24T23:25:02.278 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:02.278 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:02.278 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:02.278 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:02.278 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:02.278 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:02.279 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:02.279 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:02.279 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:02.279 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:02.279 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:02.279 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:02.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.269+0000 7fd160cc8640 1 Processor -- start 2024-01-24T23:25:02.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.269+0000 7fd160cc8640 1 -- start start 2024-01-24T23:25:02.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.269+0000 7fd160cc8640 10 monclient: init 2024-01-24T23:25:02.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:02.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 10 monclient: _add_conns ranks=[8,1,0,5,7,3,6,2,4] 2024-01-24T23:25:02.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 10 monclient(hunting): picked mon.noname-e con 0x7fd15c09ff20 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:02.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 10 monclient(hunting): picked mon.noname-f con 0x7fd15c09f9c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:02.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 10 monclient(hunting): picked mon.noname-a con 0x7fd15c13b2c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:02.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd15c064700 con 0x7fd15c13b2c0 2024-01-24T23:25:02.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd15c09e760 con 0x7fd15c09f9c0 2024-01-24T23:25:02.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd15c0a1160 con 0x7fd15c09ff20 2024-01-24T23:25:02.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:02.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 10 monclient(hunting): authenticate will time out at 1189.600572s 2024-01-24T23:25:02.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd153fff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fd15c09f9c0 legacy=0x7fd15c139ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:02.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd15ad9d640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fd15c13b2c0 legacy=0x7fd15c13d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:25:02.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd15a59c640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fd15c09ff20 0x7fd15c0a0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:40654/0 (socket says 172.21.15.132:40654) 2024-01-24T23:25:02.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd15a59c640 1 -- 172.21.15.132:0/2661691924 learned_addr learned my addr 172.21.15.132:0/2661691924 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:02.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 1 -- 172.21.15.132:0/2661691924 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2347055934 0 0) 0x7fd15c0a1160 con 0x7fd15c09ff20 2024-01-24T23:25:02.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:25:02.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:02.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:02.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 10 monclient(hunting): my global_id is 84977 2024-01-24T23:25:02.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 1 -- 172.21.15.132:0/2661691924 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd148003530 con 0x7fd15c09ff20 2024-01-24T23:25:02.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 1 -- 172.21.15.132:0/2661691924 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 312233658 0 0) 0x7fd148003530 con 0x7fd15c09ff20 2024-01-24T23:25:02.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 1 -- 172.21.15.132:0/2661691924 >> v1:172.21.15.132:6789/0 conn(0x7fd15c09f9c0 legacy=0x7fd15c139ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:02.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 1 -- 172.21.15.132:0/2661691924 >> v1:172.21.15.115:6789/0 conn(0x7fd15c13b2c0 legacy=0x7fd15c13d6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:02.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 10 monclient: _finish_hunting 0 2024-01-24T23:25:02.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 1 monclient: found mon.noname-e 2024-01-24T23:25:02.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:02.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:25:02.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 1 -- 172.21.15.132:0/2661691924 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd15c0a0a10 con 0x7fd15c09ff20 2024-01-24T23:25:02.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 10 monclient: _finish_auth 0 2024-01-24T23:25:02.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 10 monclient: _check_auth_tickets 2024-01-24T23:25:02.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:02.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 1 -- 172.21.15.132:0/2661691924 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd14c001800 con 0x7fd15c09ff20 2024-01-24T23:25:02.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 5 monclient: authenticate success, global_id 84977 2024-01-24T23:25:02.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:02.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:02.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:25:02.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 10 monclient: dump: 2024-01-24T23:25:02.282 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:02.283 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:02.283 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:02.283 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:02.283 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:02.283 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:02.283 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:02.283 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:02.283 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:02.283 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:02.283 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:02.283 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:02.283 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:02.284 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:02.284 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:02.284 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:02.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:02.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 1 -- 172.21.15.132:0/2661691924 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd14c001a20 con 0x7fd15c09ff20 2024-01-24T23:25:02.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:02.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:02.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 1 -- 172.21.15.132:0/2661691924 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd14c004bf0 con 0x7fd15c09ff20 2024-01-24T23:25:02.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 4 set_mon_vals no callback set 2024-01-24T23:25:02.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:02.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:02.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd159d9b640 10 monclient: dump: 2024-01-24T23:25:02.285 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:02.285 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:02.285 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:02.285 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:02.285 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:02.285 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:02.285 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:02.285 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:02.285 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:02.285 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:02.285 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:02.286 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:02.286 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:02.286 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:02.286 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:02.286 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:02.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 10 monclient: shutdown 2024-01-24T23:25:02.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:02.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 1 -- 172.21.15.132:0/2661691924 >> v1:172.21.15.115:6793/0 conn(0x7fd15c09ff20 legacy=0x7fd15c0a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:02.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 1 -- 172.21.15.132:0/2661691924 shutdown_connections 2024-01-24T23:25:02.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 1 -- 172.21.15.132:0/2661691924 >> 172.21.15.132:0/2661691924 conn(0x7fd15c0a5a10 msgr2=0x7fd15c0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:02.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 1 -- 172.21.15.132:0/2661691924 shutdown_connections 2024-01-24T23:25:02.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 1 -- 172.21.15.132:0/2661691924 wait complete. 2024-01-24T23:25:02.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 10 monclient: build_initial_monmap 2024-01-24T23:25:02.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 10 monclient: monmap: 2024-01-24T23:25:02.292 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:02.292 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:02.292 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:02.292 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:02.292 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:02.292 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:02.293 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:02.293 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:02.293 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:02.293 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:02.293 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:02.293 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:02.293 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:02.293 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:02.293 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:02.293 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:02.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 1 Processor -- start 2024-01-24T23:25:02.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 1 -- start start 2024-01-24T23:25:02.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.273+0000 7fd160cc8640 10 monclient: init 2024-01-24T23:25:02.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd160cc8640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:02.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd160cc8640 10 monclient: _add_conns ranks=[2,3,7,1,8,4,0,5,6] 2024-01-24T23:25:02.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd160cc8640 10 monclient(hunting): picked mon.noname-b con 0x7fd15c09f9c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:02.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd160cc8640 10 monclient(hunting): picked mon.noname-g con 0x7fd15c09ff20 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:02.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd160cc8640 10 monclient(hunting): picked mon.noname-i con 0x7fd15c13b2c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:02.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd160cc8640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd15c11ec90 con 0x7fd15c09f9c0 2024-01-24T23:25:02.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd160cc8640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd15c11c430 con 0x7fd15c09ff20 2024-01-24T23:25:02.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd160cc8640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd15c11d490 con 0x7fd15c13b2c0 2024-01-24T23:25:02.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd160cc8640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:02.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd160cc8640 10 monclient(hunting): authenticate will time out at 1189.604453s 2024-01-24T23:25:02.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd153fff640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fd15c09ff20 0x7fd15c11bd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:37484/0 (socket says 172.21.15.132:37484) 2024-01-24T23:25:02.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd153fff640 1 -- 172.21.15.132:0/658691761 learned_addr learned my addr 172.21.15.132:0/658691761 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:02.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 1 -- 172.21.15.132:0/658691761 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3176520590 0 0) 0x7fd15c11c430 con 0x7fd15c09ff20 2024-01-24T23:25:02.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:02.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:02.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:02.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient(hunting): my global_id is 5799 2024-01-24T23:25:02.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 1 -- 172.21.15.132:0/658691761 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd13c003530 con 0x7fd15c09ff20 2024-01-24T23:25:02.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 1 -- 172.21.15.132:0/658691761 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 37403636 0 0) 0x7fd15c11d490 con 0x7fd15c13b2c0 2024-01-24T23:25:02.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:02.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:02.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:02.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient(hunting): my global_id is 5749 2024-01-24T23:25:02.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 1 -- 172.21.15.132:0/658691761 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd15c11c430 con 0x7fd15c13b2c0 2024-01-24T23:25:02.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 1 -- 172.21.15.132:0/658691761 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3957721930 0 0) 0x7fd15c11c430 con 0x7fd15c13b2c0 2024-01-24T23:25:02.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 1 -- 172.21.15.132:0/658691761 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd15c11d490 con 0x7fd15c13b2c0 2024-01-24T23:25:02.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 1 -- 172.21.15.132:0/658691761 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3033476107 0 0) 0x7fd13c003530 con 0x7fd15c09ff20 2024-01-24T23:25:02.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 1 -- 172.21.15.132:0/658691761 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd15c11c430 con 0x7fd15c09ff20 2024-01-24T23:25:02.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 1 -- 172.21.15.132:0/658691761 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd154004530 con 0x7fd15c13b2c0 2024-01-24T23:25:02.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:02.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:25:02.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient(hunting): dump: 2024-01-24T23:25:02.296 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:02.296 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:02.296 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:02.296 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:02.296 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:02.297 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:02.297 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:02.297 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:02.297 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:02.297 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:02.297 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:02.297 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:02.297 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:02.297 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:02.297 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:02.297 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:02.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:02.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 1 -- 172.21.15.132:0/658691761 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd144004300 con 0x7fd15c09ff20 2024-01-24T23:25:02.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:02.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:02.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient(hunting): dump: 2024-01-24T23:25:02.298 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:02.298 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:02.298 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:02.298 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:02.298 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:02.298 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:02.298 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:02.298 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:02.298 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:02.298 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:02.299 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:02.299 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:02.299 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:02.299 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:02.299 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:02.299 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:02.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 1 -- 172.21.15.132:0/658691761 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3298404355 0 0) 0x7fd15c11ec90 con 0x7fd15c09f9c0 2024-01-24T23:25:02.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:02.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:02.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:02.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient(hunting): my global_id is 85106 2024-01-24T23:25:02.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 1 -- 172.21.15.132:0/658691761 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd13c003530 con 0x7fd15c09f9c0 2024-01-24T23:25:02.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 1 -- 172.21.15.132:0/658691761 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 724339002 0 0) 0x7fd15c11d490 con 0x7fd15c13b2c0 2024-01-24T23:25:02.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 1 -- 172.21.15.132:0/658691761 >> v1:172.21.15.132:6790/0 conn(0x7fd15c09ff20 legacy=0x7fd15c11bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:02.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 1 -- 172.21.15.132:0/658691761 >> v1:172.21.15.115:6790/0 conn(0x7fd15c09f9c0 legacy=0x7fd15c11b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:02.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:25:02.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 1 monclient: found mon.i 2024-01-24T23:25:02.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:02.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:25:02.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 1 -- 172.21.15.132:0/658691761 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd15c071960 con 0x7fd15c13b2c0 2024-01-24T23:25:02.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient: _finish_auth 0 2024-01-24T23:25:02.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:02.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:02.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd160cc8640 5 monclient: authenticate success, global_id 5749 2024-01-24T23:25:02.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd160cc8640 10 monclient: _renew_subs 2024-01-24T23:25:02.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd160cc8640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:25:02.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd160cc8640 1 -- 172.21.15.132:0/658691761 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd15c11d210 con 0x7fd15c13b2c0 2024-01-24T23:25:02.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd160cc8640 10 monclient: _renew_subs 2024-01-24T23:25:02.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd160cc8640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:25:02.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd160cc8640 1 -- 172.21.15.132:0/658691761 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd15c13a290 con 0x7fd15c13b2c0 2024-01-24T23:25:02.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 1 -- 172.21.15.132:0/658691761 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd154004720 con 0x7fd15c13b2c0 2024-01-24T23:25:02.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:02.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 1 -- 172.21.15.132:0/658691761 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd154004c50 con 0x7fd15c13b2c0 2024-01-24T23:25:02.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:02.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd15959a640 4 set_mon_vals no callback set 2024-01-24T23:25:02.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:25:02.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 10 monclient: dump: 2024-01-24T23:25:02.302 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:02.302 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:02.302 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:02.302 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:02.302 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:02.302 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:02.302 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:02.302 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:02.303 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:02.303 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:02.303 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:02.303 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:02.303 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:02.303 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:02.303 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:02.303 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:02.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd160cc8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:02.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd160cc8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:02.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd160cc8640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:25:02.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd160cc8640 1 -- 172.21.15.132:0/658691761 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd140005090 con 0x7fd15c13b2c0 2024-01-24T23:25:02.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 1 -- 172.21.15.132:0/658691761 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fd154004e30 con 0x7fd15c13b2c0 2024-01-24T23:25:02.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd152ffd640 1 -- 172.21.15.132:0/658691761 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(122..122 src has 1..122) v4 ==== 7321+0+0 (unknown 136261091 0 0) 0x7fd154044840 con 0x7fd15c13b2c0 2024-01-24T23:25:02.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.277+0000 7fd15a59c640 10 monclient: get_auth_request con 0x7fd13c037790 auth_method 0 2024-01-24T23:25:02.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.281+0000 7fd152ffd640 1 -- 172.21.15.132:0/658691761 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fd154014c70 con 0x7fd15c13b2c0 2024-01-24T23:25:02.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.281+0000 7fd152ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:02.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.281+0000 7fd152ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:02.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.445+0000 7fd160cc8640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:02.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.445+0000 7fd160cc8640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:02.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.445+0000 7fd160cc8640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:25:02.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.445+0000 7fd160cc8640 1 -- 172.21.15.132:0/658691761 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7fd140005340 con 0x7fd15c13b2c0 2024-01-24T23:25:02.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.445+0000 7fd152ffd640 1 -- 172.21.15.132:0/658691761 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v122) v1 ==== 71+0+14 (unknown 2177672627 0 3003798303) 0x7fd15401d1d0 con 0x7fd15c13b2c0 2024-01-24T23:25:02.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.445+0000 7fd152ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:02.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.445+0000 7fd152ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:02.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.545+0000 7fd160cc8640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:02.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.545+0000 7fd160cc8640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:02.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.545+0000 7fd160cc8640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:25:02.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.545+0000 7fd160cc8640 1 -- 172.21.15.132:0/658691761 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7fd140005a80 con 0x7fd15c13b2c0 2024-01-24T23:25:02.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.545+0000 7fd152ffd640 1 -- 172.21.15.132:0/658691761 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v122) v1 ==== 71+0+14 (unknown 2177672627 0 3003798303) 0x7fd1540130e0 con 0x7fd15c13b2c0 2024-01-24T23:25:02.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.545+0000 7fd152ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:02.550 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.545+0000 7fd152ffd640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:25:02.551 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.545+0000 7fd160cc8640 1 -- 172.21.15.132:0/658691761 >> v1:172.21.15.132:6812/1835401922 conn(0x7fd13c037790 legacy=0x7fd13c039c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:02.551 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.545+0000 7fd160cc8640 10 monclient: shutdown 2024-01-24T23:25:02.551 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.545+0000 7fd160cc8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:02.551 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.545+0000 7fd160cc8640 1 -- 172.21.15.132:0/658691761 >> v1:172.21.15.132:6792/0 conn(0x7fd15c13b2c0 legacy=0x7fd15c1d1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:02.551 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.545+0000 7fd160cc8640 1 -- 172.21.15.132:0/658691761 shutdown_connections 2024-01-24T23:25:02.551 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.545+0000 7fd160cc8640 1 -- 172.21.15.132:0/658691761 >> 172.21.15.132:0/658691761 conn(0x7fd15c0a5a10 msgr2=0x7fd15c136af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:02.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.549+0000 7fd160cc8640 1 -- 172.21.15.132:0/658691761 shutdown_connections 2024-01-24T23:25:02.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.549+0000 7fd160cc8640 1 -- 172.21.15.132:0/658691761 wait complete. 2024-01-24T23:25:02.565 INFO:tasks.workunit.client.0.smithi132.stdout:fooset 2024-01-24T23:25:02.566 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set ls 2024-01-24T23:25:02.566 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep barset 2024-01-24T23:25:02.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.677+0000 7f2c57cf7640 10 monclient: get_monmap_and_config 2024-01-24T23:25:02.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.677+0000 7f2c57cf7640 10 monclient: build_initial_monmap 2024-01-24T23:25:02.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.677+0000 7f2c57cf7640 10 monclient: monmap: 2024-01-24T23:25:02.682 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:02.682 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:02.682 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:02.683507+0000 2024-01-24T23:25:02.682 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:02.683507+0000 2024-01-24T23:25:02.682 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:02.683 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:02.683 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:02.683 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:02.683 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:02.683 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:02.683 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:02.683 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:02.683 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:02.683 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:02.683 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:02.683 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:02.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.677+0000 7f2c57cf7640 1 Processor -- start 2024-01-24T23:25:02.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.677+0000 7f2c57cf7640 1 -- start start 2024-01-24T23:25:02.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.677+0000 7f2c57cf7640 10 monclient: init 2024-01-24T23:25:02.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:02.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 10 monclient: _add_conns ranks=[0,3,4,7,1,8,5,6,2] 2024-01-24T23:25:02.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 10 monclient(hunting): picked mon.noname-a con 0x7f2c5009ff20 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:02.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 10 monclient(hunting): picked mon.noname-g con 0x7f2c5009f9c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:02.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 10 monclient(hunting): picked mon.noname-c con 0x7f2c5013b2c0 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:02.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c50064700 con 0x7f2c5009ff20 2024-01-24T23:25:02.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c5009e760 con 0x7f2c5009f9c0 2024-01-24T23:25:02.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c500a1160 con 0x7f2c5013b2c0 2024-01-24T23:25:02.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:02.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 10 monclient(hunting): authenticate will time out at 1190.008157s 2024-01-24T23:25:02.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c55a93640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f2c5009ff20 legacy=0x7f2c500a0300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:25:02.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c55292640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f2c5009f9c0 0x7f2c50139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:37496/0 (socket says 172.21.15.132:37496) 2024-01-24T23:25:02.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c55292640 1 -- 172.21.15.132:0/4294574137 learned_addr learned my addr 172.21.15.132:0/4294574137 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:02.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c56294640 1 -- 172.21.15.132:0/4294574137 >> v1:172.21.15.115:6791/0 conn(0x7f2c5013b2c0 legacy=0x7f2c5013d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:25:02.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 1 -- 172.21.15.132:0/4294574137 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2231385862 0 0) 0x7f2c5009e760 con 0x7f2c5009f9c0 2024-01-24T23:25:02.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 10 monclient(hunting): _negotiate 2024-01-24T23:25:02.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:02.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:02.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 10 monclient(hunting): my global_id is 5808 2024-01-24T23:25:02.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 1 -- 172.21.15.132:0/4294574137 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c3c003530 con 0x7f2c5009f9c0 2024-01-24T23:25:02.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 1 -- 172.21.15.132:0/4294574137 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 903145023 0 0) 0x7f2c3c003530 con 0x7f2c5009f9c0 2024-01-24T23:25:02.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 1 -- 172.21.15.132:0/4294574137 >> v1:172.21.15.115:6791/0 conn(0x7f2c5013b2c0 legacy=0x7f2c5013d6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:02.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 1 -- 172.21.15.132:0/4294574137 >> v1:172.21.15.115:6789/0 conn(0x7f2c5009ff20 legacy=0x7f2c500a0300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:02.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 10 monclient: _finish_hunting 0 2024-01-24T23:25:02.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 1 monclient: found mon.noname-g 2024-01-24T23:25:02.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:02.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:25:02.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 1 -- 172.21.15.132:0/4294574137 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c500a0a10 con 0x7f2c5009f9c0 2024-01-24T23:25:02.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 10 monclient: _finish_auth 0 2024-01-24T23:25:02.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 10 monclient: _check_auth_tickets 2024-01-24T23:25:02.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:02.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 1 -- 172.21.15.132:0/4294574137 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2c400017c0 con 0x7f2c5009f9c0 2024-01-24T23:25:02.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 5 monclient: authenticate success, global_id 5808 2024-01-24T23:25:02.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:02.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:02.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:25:02.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 10 monclient: dump: 2024-01-24T23:25:02.693 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:02.693 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:02.693 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:02.693 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:02.693 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:02.693 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:02.693 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:02.693 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:02.693 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:02.693 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:02.693 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:02.693 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:02.693 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:02.693 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:02.693 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:02.694 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:02.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:02.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 1 -- 172.21.15.132:0/4294574137 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2c40001ae0 con 0x7f2c5009f9c0 2024-01-24T23:25:02.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:02.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 1 -- 172.21.15.132:0/4294574137 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2c40004bf0 con 0x7f2c5009f9c0 2024-01-24T23:25:02.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:02.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 4 set_mon_vals no callback set 2024-01-24T23:25:02.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:02.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:02.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c54a91640 10 monclient: dump: 2024-01-24T23:25:02.694 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:02.694 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:02.694 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:02.694 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:02.694 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:02.695 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:02.695 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:02.695 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:02.695 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:02.695 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:02.695 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:02.695 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:02.695 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:02.695 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:02.695 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:02.695 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:02.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 10 monclient: shutdown 2024-01-24T23:25:02.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:02.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 1 -- 172.21.15.132:0/4294574137 >> v1:172.21.15.132:6790/0 conn(0x7f2c5009f9c0 legacy=0x7f2c50139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:02.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 1 -- 172.21.15.132:0/4294574137 shutdown_connections 2024-01-24T23:25:02.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 1 -- 172.21.15.132:0/4294574137 >> 172.21.15.132:0/4294574137 conn(0x7f2c500a5a10 msgr2=0x7f2c500a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:02.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 1 -- 172.21.15.132:0/4294574137 shutdown_connections 2024-01-24T23:25:02.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 1 -- 172.21.15.132:0/4294574137 wait complete. 2024-01-24T23:25:02.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 10 monclient: build_initial_monmap 2024-01-24T23:25:02.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 10 monclient: monmap: 2024-01-24T23:25:02.696 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:02.696 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:02.696 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:02.696 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:02.696 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:02.696 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:02.696 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:02.696 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:02.696 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:02.696 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:02.697 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:02.697 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:02.697 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:02.697 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:02.697 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:02.697 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:02.697 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 1 Processor -- start 2024-01-24T23:25:02.697 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 1 -- start start 2024-01-24T23:25:02.697 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 10 monclient: init 2024-01-24T23:25:02.697 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:02.697 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 10 monclient: _add_conns ranks=[5,7,0,6,4,3,2,8,1] 2024-01-24T23:25:02.697 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 10 monclient(hunting): picked mon.noname-h con 0x7f2c5009f9c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:02.697 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 10 monclient(hunting): picked mon.noname-i con 0x7f2c5009ff20 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:02.697 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 10 monclient(hunting): picked mon.noname-a con 0x7f2c5013b2c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:02.697 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c5011fa20 con 0x7f2c5013b2c0 2024-01-24T23:25:02.697 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c50071960 con 0x7f2c5009f9c0 2024-01-24T23:25:02.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c50071680 con 0x7f2c5009ff20 2024-01-24T23:25:02.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:02.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 10 monclient(hunting): authenticate will time out at 1190.010975s 2024-01-24T23:25:02.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c55a93640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f2c5009f9c0 0x7f2c50132fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:35346/0 (socket says 172.21.15.132:35346) 2024-01-24T23:25:02.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c55292640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f2c5009ff20 0x7f2c501d06a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:60492/0 (socket says 172.21.15.132:60492) 2024-01-24T23:25:02.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c55a93640 1 -- 172.21.15.132:0/406098856 learned_addr learned my addr 172.21.15.132:0/406098856 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:02.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c56294640 1 -- 172.21.15.132:0/406098856 >> v1:172.21.15.115:6789/0 conn(0x7f2c5013b2c0 legacy=0x7f2c501d3dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:25:02.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 1 -- 172.21.15.132:0/406098856 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1924380811 0 0) 0x7f2c50071680 con 0x7f2c5009ff20 2024-01-24T23:25:02.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:02.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:02.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:02.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 10 monclient(hunting): my global_id is 5758 2024-01-24T23:25:02.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 1 -- 172.21.15.132:0/406098856 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c34003530 con 0x7f2c5009ff20 2024-01-24T23:25:02.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 1 -- 172.21.15.132:0/406098856 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454002362 0 0) 0x7f2c50071960 con 0x7f2c5009f9c0 2024-01-24T23:25:02.698 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:02.699 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:02.699 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:02.699 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 10 monclient(hunting): my global_id is 5729 2024-01-24T23:25:02.699 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 1 -- 172.21.15.132:0/406098856 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c50071680 con 0x7f2c5009f9c0 2024-01-24T23:25:02.699 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 1 -- 172.21.15.132:0/406098856 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1069037863 0 0) 0x7f2c50071680 con 0x7f2c5009f9c0 2024-01-24T23:25:02.699 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 1 -- 172.21.15.132:0/406098856 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2c50071960 con 0x7f2c5009f9c0 2024-01-24T23:25:02.699 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 1 -- 172.21.15.132:0/406098856 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2c4c004330 con 0x7f2c5009f9c0 2024-01-24T23:25:02.699 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:02.699 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:25:02.699 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 10 monclient(hunting): dump: 2024-01-24T23:25:02.699 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:02.699 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:02.699 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:02.699 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:02.699 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:02.700 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:02.700 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:02.700 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:02.700 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:02.700 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:02.700 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:02.700 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:02.700 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:02.700 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:02.700 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:02.700 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:02.700 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:02.700 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 1 -- 172.21.15.132:0/406098856 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 33560094 0 0) 0x7f2c34003530 con 0x7f2c5009ff20 2024-01-24T23:25:02.700 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 1 -- 172.21.15.132:0/406098856 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2c50071680 con 0x7f2c5009ff20 2024-01-24T23:25:02.700 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 1 -- 172.21.15.132:0/406098856 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2c40001780 con 0x7f2c5009ff20 2024-01-24T23:25:02.701 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:02.701 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-01-24T23:25:02.701 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 10 monclient(hunting): dump: 2024-01-24T23:25:02.701 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:02.701 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:02.701 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:02.701 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:02.701 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:02.701 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:02.701 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:02.701 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:02.701 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:02.701 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:02.701 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:02.701 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:02.702 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:02.702 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:02.702 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:02.702 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:02.702 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 1 -- 172.21.15.132:0/406098856 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3163181744 0 0) 0x7f2c50071960 con 0x7f2c5009f9c0 2024-01-24T23:25:02.702 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 1 -- 172.21.15.132:0/406098856 >> v1:172.21.15.132:6792/0 conn(0x7f2c5009ff20 legacy=0x7f2c501d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:02.702 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 1 -- 172.21.15.132:0/406098856 >> v1:172.21.15.115:6789/0 conn(0x7f2c5013b2c0 legacy=0x7f2c501d3dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:02.702 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 10 monclient: _finish_hunting 0 2024-01-24T23:25:02.702 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 1 monclient: found mon.h 2024-01-24T23:25:02.702 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:02.702 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:02.702 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 1 -- 172.21.15.132:0/406098856 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c50120510 con 0x7f2c5009f9c0 2024-01-24T23:25:02.702 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 10 monclient: _finish_auth 0 2024-01-24T23:25:02.702 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 10 monclient: _check_auth_tickets 2024-01-24T23:25:02.702 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c467fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:02.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 5 monclient: authenticate success, global_id 5729 2024-01-24T23:25:02.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 10 monclient: _renew_subs 2024-01-24T23:25:02.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:02.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.681+0000 7f2c57cf7640 1 -- 172.21.15.132:0/406098856 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c5009d710 con 0x7f2c5009f9c0 2024-01-24T23:25:02.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.685+0000 7f2c57cf7640 10 monclient: _renew_subs 2024-01-24T23:25:02.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.685+0000 7f2c57cf7640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:02.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.685+0000 7f2c57cf7640 1 -- 172.21.15.132:0/406098856 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5013a2e0 con 0x7f2c5009f9c0 2024-01-24T23:25:02.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.685+0000 7f2c467fc640 1 -- 172.21.15.132:0/406098856 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2c4c004900 con 0x7f2c5009f9c0 2024-01-24T23:25:02.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.685+0000 7f2c467fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:02.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.685+0000 7f2c467fc640 1 -- 172.21.15.132:0/406098856 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2c4c004e30 con 0x7f2c5009f9c0 2024-01-24T23:25:02.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.685+0000 7f2c467fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:02.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.685+0000 7f2c47fff640 4 set_mon_vals no callback set 2024-01-24T23:25:02.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.685+0000 7f2c467fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:02.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.685+0000 7f2c467fc640 10 monclient: dump: 2024-01-24T23:25:02.703 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:02.704 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:02.704 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:02.704 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:02.704 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:02.704 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:02.704 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:02.704 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:02.704 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:02.704 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:02.704 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:02.704 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:02.704 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:02.704 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:02.704 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:02.704 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:02.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.685+0000 7f2c467fc640 1 -- 172.21.15.132:0/406098856 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f2c4c004f90 con 0x7f2c5009f9c0 2024-01-24T23:25:02.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.685+0000 7f2c57cf7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:02.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.685+0000 7f2c57cf7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:02.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.685+0000 7f2c57cf7640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:02.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.685+0000 7f2c57cf7640 1 -- 172.21.15.132:0/406098856 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c38005090 con 0x7f2c5009f9c0 2024-01-24T23:25:02.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.685+0000 7f2c467fc640 1 -- 172.21.15.132:0/406098856 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(122..122 src has 1..122) v4 ==== 7321+0+0 (unknown 136261091 0 0) 0x7f2c4c044540 con 0x7f2c5009f9c0 2024-01-24T23:25:02.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.685+0000 7f2c55292640 10 monclient: get_auth_request con 0x7f2c340376e0 auth_method 0 2024-01-24T23:25:02.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.689+0000 7f2c467fc640 1 -- 172.21.15.132:0/406098856 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f2c4c014a40 con 0x7f2c5009f9c0 2024-01-24T23:25:02.706 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.689+0000 7f2c467fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:02.706 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.689+0000 7f2c467fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:02.854 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.849+0000 7f2c57cf7640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:02.854 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.849+0000 7f2c57cf7640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:02.854 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.849+0000 7f2c57cf7640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:02.854 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.849+0000 7f2c57cf7640 1 -- 172.21.15.132:0/406098856 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7f2c380058c0 con 0x7f2c5009f9c0 2024-01-24T23:25:02.854 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.849+0000 7f2c467fc640 1 -- 172.21.15.132:0/406098856 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v122) v1 ==== 71+0+14 (unknown 2177672627 0 3003798303) 0x7f2c4c012eb0 con 0x7f2c5009f9c0 2024-01-24T23:25:02.855 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.849+0000 7f2c467fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:02.855 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.849+0000 7f2c467fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:02.953 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.949+0000 7f2c57cf7640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:02.953 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.949+0000 7f2c57cf7640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:02.953 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.949+0000 7f2c57cf7640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:02.953 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.949+0000 7f2c57cf7640 1 -- 172.21.15.132:0/406098856 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7f2c38000bd0 con 0x7f2c5009f9c0 2024-01-24T23:25:02.953 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.949+0000 7f2c467fc640 1 -- 172.21.15.132:0/406098856 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v122) v1 ==== 71+0+14 (unknown 2177672627 0 3003798303) 0x7f2c4c014630 con 0x7f2c5009f9c0 2024-01-24T23:25:02.953 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.949+0000 7f2c467fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:02.953 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.949+0000 7f2c467fc640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:25:02.955 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.949+0000 7f2c57cf7640 1 -- 172.21.15.132:0/406098856 >> v1:172.21.15.132:6812/1835401922 conn(0x7f2c340376e0 legacy=0x7f2c34039ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:02.955 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.949+0000 7f2c57cf7640 10 monclient: shutdown 2024-01-24T23:25:02.955 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.949+0000 7f2c57cf7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:02.955 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.949+0000 7f2c57cf7640 1 -- 172.21.15.132:0/406098856 >> v1:172.21.15.132:6791/0 conn(0x7f2c5009f9c0 legacy=0x7f2c50132fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:02.955 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.949+0000 7f2c57cf7640 1 -- 172.21.15.132:0/406098856 shutdown_connections 2024-01-24T23:25:02.955 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.949+0000 7f2c57cf7640 1 -- 172.21.15.132:0/406098856 >> 172.21.15.132:0/406098856 conn(0x7f2c500a5a10 msgr2=0x7f2c50066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:02.955 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.949+0000 7f2c57cf7640 1 -- 172.21.15.132:0/406098856 shutdown_connections 2024-01-24T23:25:02.955 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:02.949+0000 7f2c57cf7640 1 -- 172.21.15.132:0/406098856 wait complete. 2024-01-24T23:25:02.968 INFO:tasks.workunit.client.0.smithi132.stdout:barset 2024-01-24T23:25:02.969 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set dump 2024-01-24T23:25:03.087 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 10 monclient: get_monmap_and_config 2024-01-24T23:25:03.088 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 10 monclient: build_initial_monmap 2024-01-24T23:25:03.088 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 10 monclient: monmap: 2024-01-24T23:25:03.088 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:03.088 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:03.088 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:03.089101+0000 2024-01-24T23:25:03.088 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:03.089101+0000 2024-01-24T23:25:03.088 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:03.088 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:03.088 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:03.088 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:03.088 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:03.088 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:03.088 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:03.088 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:03.089 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:03.089 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:03.089 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:03.089 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:03.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 1 Processor -- start 2024-01-24T23:25:03.095 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 1 -- start start 2024-01-24T23:25:03.095 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 10 monclient: init 2024-01-24T23:25:03.095 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:03.095 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 10 monclient: _add_conns ranks=[1,5,2,7,8,6,0,4,3] 2024-01-24T23:25:03.095 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 10 monclient(hunting): picked mon.noname-f con 0x7f6f74130b20 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:03.095 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 10 monclient(hunting): picked mon.noname-h con 0x7f6f7409b8d0 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:03.095 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 10 monclient(hunting): picked mon.noname-b con 0x7f6f7413a3b0 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:03.095 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f74064700 con 0x7f6f74130b20 2024-01-24T23:25:03.095 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f74131610 con 0x7f6f7413a3b0 2024-01-24T23:25:03.095 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f7409ad60 con 0x7f6f7409b8d0 2024-01-24T23:25:03.095 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:03.095 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 10 monclient(hunting): authenticate will time out at 1190.413825s 2024-01-24T23:25:03.096 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f78e9c640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f6f74130b20 legacy=0x7f6f74130f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:03.096 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f73fff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f6f7409b8d0 0x7f6f74138c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:35356/0 (socket says 172.21.15.132:35356) 2024-01-24T23:25:03.096 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f73fff640 1 -- 172.21.15.132:0/918601384 learned_addr learned my addr 172.21.15.132:0/918601384 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:03.096 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 1 -- 172.21.15.132:0/918601384 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2534749757 0 0) 0x7f6f7409ad60 con 0x7f6f7409b8d0 2024-01-24T23:25:03.096 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:03.096 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:03.096 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:03.096 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 10 monclient(hunting): my global_id is 5738 2024-01-24T23:25:03.096 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 1 -- 172.21.15.132:0/918601384 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f5c005740 con 0x7f6f7409b8d0 2024-01-24T23:25:03.096 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 1 -- 172.21.15.132:0/918601384 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 249760416 0 0) 0x7f6f5c005740 con 0x7f6f7409b8d0 2024-01-24T23:25:03.096 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 1 -- 172.21.15.132:0/918601384 >> v1:172.21.15.115:6790/0 conn(0x7f6f7413a3b0 legacy=0x7f6f7413c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:03.096 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 1 -- 172.21.15.132:0/918601384 >> v1:172.21.15.132:6789/0 conn(0x7f6f74130b20 legacy=0x7f6f74130f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:03.096 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 10 monclient: _finish_hunting 0 2024-01-24T23:25:03.096 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 1 monclient: found mon.noname-h 2024-01-24T23:25:03.097 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:03.097 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:25:03.097 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 1 -- 172.21.15.132:0/918601384 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7409c080 con 0x7f6f7409b8d0 2024-01-24T23:25:03.097 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 10 monclient: _finish_auth 0 2024-01-24T23:25:03.097 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 10 monclient: _check_auth_tickets 2024-01-24T23:25:03.097 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:03.097 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 1 -- 172.21.15.132:0/918601384 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6f680017c0 con 0x7f6f7409b8d0 2024-01-24T23:25:03.097 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 5 monclient: authenticate success, global_id 5738 2024-01-24T23:25:03.097 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:03.097 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:03.097 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:25:03.097 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 10 monclient: dump: 2024-01-24T23:25:03.097 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:03.097 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:03.098 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:03.098 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:03.098 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:03.098 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:03.098 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:03.098 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:03.098 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:03.098 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:03.098 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:03.098 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:03.098 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:03.098 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:03.098 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:03.099 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:03.099 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:03.099 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 1 -- 172.21.15.132:0/918601384 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6f68001ae0 con 0x7f6f7409b8d0 2024-01-24T23:25:03.099 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:03.099 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 1 -- 172.21.15.132:0/918601384 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6f68004bf0 con 0x7f6f7409b8d0 2024-01-24T23:25:03.099 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:03.099 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 4 set_mon_vals no callback set 2024-01-24T23:25:03.099 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:03.099 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:03.099 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f737fe640 10 monclient: dump: 2024-01-24T23:25:03.099 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:03.099 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:03.099 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:03.099 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:03.099 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:03.100 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:03.100 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:03.100 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:03.100 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:03.100 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:03.100 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:03.100 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:03.100 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:03.100 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:03.100 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:03.100 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:03.100 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 10 monclient: shutdown 2024-01-24T23:25:03.100 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:03.100 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 1 -- 172.21.15.132:0/918601384 >> v1:172.21.15.132:6791/0 conn(0x7f6f7409b8d0 legacy=0x7f6f74138c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:03.100 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 1 -- 172.21.15.132:0/918601384 shutdown_connections 2024-01-24T23:25:03.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 1 -- 172.21.15.132:0/918601384 >> 172.21.15.132:0/918601384 conn(0x7f6f740a2b40 msgr2=0x7f6f740a2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:03.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 1 -- 172.21.15.132:0/918601384 shutdown_connections 2024-01-24T23:25:03.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 1 -- 172.21.15.132:0/918601384 wait complete. 2024-01-24T23:25:03.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 10 monclient: build_initial_monmap 2024-01-24T23:25:03.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.085+0000 7f6f7b100640 10 monclient: monmap: 2024-01-24T23:25:03.101 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:03.101 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:03.101 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:03.101 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:03.101 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:03.101 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:03.101 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:03.101 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:03.101 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:03.101 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:03.102 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:03.102 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:03.102 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:03.102 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:03.102 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:03.102 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:03.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7b100640 1 Processor -- start 2024-01-24T23:25:03.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7b100640 1 -- start start 2024-01-24T23:25:03.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7b100640 10 monclient: init 2024-01-24T23:25:03.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7b100640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:03.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7b100640 10 monclient: _add_conns ranks=[6,4,5,1,7,2,0,8,3] 2024-01-24T23:25:03.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7b100640 10 monclient(hunting): picked mon.noname-d con 0x7f6f7409b8d0 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:03.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7b100640 10 monclient(hunting): picked mon.noname-c con 0x7f6f74130b20 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:03.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7b100640 10 monclient(hunting): picked mon.noname-h con 0x7f6f7413a3b0 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:03.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7b100640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f7409ff10 con 0x7f6f74130b20 2024-01-24T23:25:03.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7b100640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f7409fa50 con 0x7f6f7413a3b0 2024-01-24T23:25:03.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7b100640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f74137990 con 0x7f6f7409b8d0 2024-01-24T23:25:03.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7b100640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:03.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7b100640 10 monclient(hunting): authenticate will time out at 1190.416806s 2024-01-24T23:25:03.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7969d640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f6f7413a3b0 0x7f6f741d39d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:35368/0 (socket says 172.21.15.132:35368) 2024-01-24T23:25:03.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7969d640 1 -- 172.21.15.132:0/986355290 learned_addr learned my addr 172.21.15.132:0/986355290 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:03.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f73fff640 1 -- 172.21.15.132:0/986355290 >> v1:172.21.15.115:6791/0 conn(0x7f6f74130b20 legacy=0x7f6f7412f7d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:25:03.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 1 -- 172.21.15.132:0/986355290 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460149578 0 0) 0x7f6f7409fa50 con 0x7f6f7413a3b0 2024-01-24T23:25:03.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 10 monclient(hunting): _negotiate 2024-01-24T23:25:03.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:03.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:03.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 10 monclient(hunting): my global_id is 5747 2024-01-24T23:25:03.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 1 -- 172.21.15.132:0/986355290 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f60003530 con 0x7f6f7413a3b0 2024-01-24T23:25:03.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 1 -- 172.21.15.132:0/986355290 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1918622304 0 0) 0x7f6f60003530 con 0x7f6f7413a3b0 2024-01-24T23:25:03.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 1 -- 172.21.15.132:0/986355290 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6f7409fa50 con 0x7f6f7413a3b0 2024-01-24T23:25:03.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 1 -- 172.21.15.132:0/986355290 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6f64001a00 con 0x7f6f7413a3b0 2024-01-24T23:25:03.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:03.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:25:03.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 10 monclient(hunting): dump: 2024-01-24T23:25:03.104 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:03.105 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:03.105 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:03.105 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:03.105 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:03.105 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:03.105 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:03.105 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:03.105 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:03.105 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:03.105 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:03.106 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:03.106 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:03.106 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:03.106 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:03.106 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:03.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:03.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 1 -- 172.21.15.132:0/986355290 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3866893582 0 0) 0x7f6f7409fa50 con 0x7f6f7413a3b0 2024-01-24T23:25:03.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 1 -- 172.21.15.132:0/986355290 >> v1:172.21.15.115:6792/0 conn(0x7f6f7409b8d0 legacy=0x7f6f7412f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:03.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 1 -- 172.21.15.132:0/986355290 >> v1:172.21.15.115:6791/0 conn(0x7f6f74130b20 legacy=0x7f6f7412f7d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:03.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 10 monclient: _finish_hunting 0 2024-01-24T23:25:03.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 1 monclient: found mon.h 2024-01-24T23:25:03.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:03.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:03.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 1 -- 172.21.15.132:0/986355290 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7412fee0 con 0x7f6f7413a3b0 2024-01-24T23:25:03.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 10 monclient: _finish_auth 0 2024-01-24T23:25:03.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 10 monclient: _check_auth_tickets 2024-01-24T23:25:03.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:03.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7b100640 5 monclient: authenticate success, global_id 5747 2024-01-24T23:25:03.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7b100640 10 monclient: _renew_subs 2024-01-24T23:25:03.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7b100640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:03.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7b100640 1 -- 172.21.15.132:0/986355290 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f74137c70 con 0x7f6f7413a3b0 2024-01-24T23:25:03.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7b100640 10 monclient: _renew_subs 2024-01-24T23:25:03.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7b100640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:03.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7b100640 1 -- 172.21.15.132:0/986355290 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f741376b0 con 0x7f6f7413a3b0 2024-01-24T23:25:03.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 1 -- 172.21.15.132:0/986355290 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6f640038d0 con 0x7f6f7413a3b0 2024-01-24T23:25:03.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:03.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 1 -- 172.21.15.132:0/986355290 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6f64003e00 con 0x7f6f7413a3b0 2024-01-24T23:25:03.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:03.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f72ffd640 4 set_mon_vals no callback set 2024-01-24T23:25:03.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:03.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 10 monclient: dump: 2024-01-24T23:25:03.108 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:03.108 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:03.108 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:03.108 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:03.108 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:03.108 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:03.108 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:03.108 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:03.109 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:03.109 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:03.109 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:03.109 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:03.109 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:03.109 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:03.109 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:03.109 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:03.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 1 -- 172.21.15.132:0/986355290 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f6f64003fe0 con 0x7f6f7413a3b0 2024-01-24T23:25:03.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7b100640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:03.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7b100640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:03.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7b100640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:03.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f7b100640 1 -- 172.21.15.132:0/986355290 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f54005090 con 0x7f6f7413a3b0 2024-01-24T23:25:03.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f717fa640 1 -- 172.21.15.132:0/986355290 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(122..122 src has 1..122) v4 ==== 7321+0+0 (unknown 136261091 0 0) 0x7f6f64044620 con 0x7f6f7413a3b0 2024-01-24T23:25:03.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.089+0000 7f6f78e9c640 10 monclient: get_auth_request con 0x7f6f60036ec0 auth_method 0 2024-01-24T23:25:03.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.093+0000 7f6f717fa640 1 -- 172.21.15.132:0/986355290 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f6f64014830 con 0x7f6f7413a3b0 2024-01-24T23:25:03.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.093+0000 7f6f717fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:03.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.093+0000 7f6f717fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:03.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.257+0000 7f6f7b100640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set dump"}] 2024-01-24T23:25:03.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.257+0000 7f6f7b100640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set dump"}] 2024-01-24T23:25:03.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.257+0000 7f6f7b100640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:03.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.257+0000 7f6f7b100640 1 -- 172.21.15.132:0/986355290 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush weight-set dump"} v 0) v1 -- 0x7f6f54005340 con 0x7f6f7413a3b0 2024-01-24T23:25:03.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.257+0000 7f6f717fa640 1 -- 172.21.15.132:0/986355290 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set dump"}]=0 v122) v1 ==== 73+0+6748 (unknown 414810089 0 1281724412) 0x7f6f64015320 con 0x7f6f7413a3b0 2024-01-24T23:25:03.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.257+0000 7f6f717fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set dump"}] 2024-01-24T23:25:03.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.257+0000 7f6f717fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:03.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.353+0000 7f6f7b100640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set dump"}] 2024-01-24T23:25:03.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.353+0000 7f6f7b100640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set dump"}] 2024-01-24T23:25:03.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.353+0000 7f6f7b100640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:03.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.353+0000 7f6f7b100640 1 -- 172.21.15.132:0/986355290 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush weight-set dump"} v 0) v1 -- 0x7f6f54007190 con 0x7f6f7413a3b0 2024-01-24T23:25:03.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.353+0000 7f6f717fa640 1 -- 172.21.15.132:0/986355290 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set dump"}]=0 v122) v1 ==== 73+0+6748 (unknown 414810089 0 1281724412) 0x7f6f64015320 con 0x7f6f7413a3b0 2024-01-24T23:25:03.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.353+0000 7f6f717fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set dump"}] 2024-01-24T23:25:03.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.353+0000 7f6f717fa640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:25:03.356 INFO:tasks.workunit.client.0.smithi132.stdout:{ 2024-01-24T23:25:03.356 INFO:tasks.workunit.client.0.smithi132.stdout: "9": [ 2024-01-24T23:25:03.357 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.357 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -1, 2024-01-24T23:25:03.357 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.357 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.357 INFO:tasks.workunit.client.0.smithi132.stdout: 0.263671875, 2024-01-24T23:25:03.357 INFO:tasks.workunit.client.0.smithi132.stdout: 0.263671875, 2024-01-24T23:25:03.357 INFO:tasks.workunit.client.0.smithi132.stdout: 0, 2024-01-24T23:25:03.357 INFO:tasks.workunit.client.0.smithi132.stdout: 0 2024-01-24T23:25:03.357 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.357 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.357 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.357 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.357 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -2, 2024-01-24T23:25:03.358 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.358 INFO:tasks.workunit.client.0.smithi132.stdout: [] 2024-01-24T23:25:03.358 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.358 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.358 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.358 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -3, 2024-01-24T23:25:03.358 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.358 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.358 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625, 2024-01-24T23:25:03.358 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625, 2024-01-24T23:25:03.358 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625 2024-01-24T23:25:03.358 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.358 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.358 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.359 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.359 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -4, 2024-01-24T23:25:03.359 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.359 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.359 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625 2024-01-24T23:25:03.359 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.359 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.359 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.359 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.359 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -5, 2024-01-24T23:25:03.359 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.359 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.360 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625, 2024-01-24T23:25:03.360 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625, 2024-01-24T23:25:03.360 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625 2024-01-24T23:25:03.360 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.360 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.360 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.360 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.360 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -6, 2024-01-24T23:25:03.360 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.360 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.360 INFO:tasks.workunit.client.0.smithi132.stdout: 0, 2024-01-24T23:25:03.360 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625, 2024-01-24T23:25:03.360 INFO:tasks.workunit.client.0.smithi132.stdout: 0, 2024-01-24T23:25:03.360 INFO:tasks.workunit.client.0.smithi132.stdout: 0 2024-01-24T23:25:03.361 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.361 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.361 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.361 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.361 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -7, 2024-01-24T23:25:03.361 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.361 INFO:tasks.workunit.client.0.smithi132.stdout: [] 2024-01-24T23:25:03.361 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.361 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.361 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.361 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -8, 2024-01-24T23:25:03.361 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.361 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.361 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625 2024-01-24T23:25:03.361 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.362 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.362 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.362 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.362 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -9, 2024-01-24T23:25:03.362 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.362 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.362 INFO:tasks.workunit.client.0.smithi132.stdout: 0, 2024-01-24T23:25:03.362 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625, 2024-01-24T23:25:03.362 INFO:tasks.workunit.client.0.smithi132.stdout: 0, 2024-01-24T23:25:03.362 INFO:tasks.workunit.client.0.smithi132.stdout: 0 2024-01-24T23:25:03.362 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.362 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.362 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.362 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.363 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -10, 2024-01-24T23:25:03.363 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.363 INFO:tasks.workunit.client.0.smithi132.stdout: [] 2024-01-24T23:25:03.363 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.363 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.363 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.363 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -11, 2024-01-24T23:25:03.363 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.363 INFO:tasks.workunit.client.0.smithi132.stdout: [] 2024-01-24T23:25:03.363 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.363 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.363 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.363 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -12, 2024-01-24T23:25:03.363 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.363 INFO:tasks.workunit.client.0.smithi132.stdout: [] 2024-01-24T23:25:03.364 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.364 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.364 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.364 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -13, 2024-01-24T23:25:03.364 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.364 INFO:tasks.workunit.client.0.smithi132.stdout: [] 2024-01-24T23:25:03.364 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.364 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.364 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.364 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -14, 2024-01-24T23:25:03.364 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.364 INFO:tasks.workunit.client.0.smithi132.stdout: [] 2024-01-24T23:25:03.364 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.364 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.365 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.365 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -15, 2024-01-24T23:25:03.365 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.365 INFO:tasks.workunit.client.0.smithi132.stdout: [] 2024-01-24T23:25:03.365 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.365 INFO:tasks.workunit.client.0.smithi132.stdout: } 2024-01-24T23:25:03.365 INFO:tasks.workunit.client.0.smithi132.stdout: ], 2024-01-24T23:25:03.365 INFO:tasks.workunit.client.0.smithi132.stdout: "10": [ 2024-01-24T23:25:03.365 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.366 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -1, 2024-01-24T23:25:03.366 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.366 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.366 INFO:tasks.workunit.client.0.smithi132.stdout: 0.263671875, 2024-01-24T23:25:03.366 INFO:tasks.workunit.client.0.smithi132.stdout: 0.263671875, 2024-01-24T23:25:03.366 INFO:tasks.workunit.client.0.smithi132.stdout: 0, 2024-01-24T23:25:03.366 INFO:tasks.workunit.client.0.smithi132.stdout: 0 2024-01-24T23:25:03.366 INFO:tasks.workunit.client.0.smithi132.stdout: ], 2024-01-24T23:25:03.366 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.366 INFO:tasks.workunit.client.0.smithi132.stdout: 0.263671875, 2024-01-24T23:25:03.366 INFO:tasks.workunit.client.0.smithi132.stdout: 0.263671875, 2024-01-24T23:25:03.366 INFO:tasks.workunit.client.0.smithi132.stdout: 0, 2024-01-24T23:25:03.366 INFO:tasks.workunit.client.0.smithi132.stdout: 0 2024-01-24T23:25:03.366 INFO:tasks.workunit.client.0.smithi132.stdout: ], 2024-01-24T23:25:03.367 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.367 INFO:tasks.workunit.client.0.smithi132.stdout: 0.263671875, 2024-01-24T23:25:03.367 INFO:tasks.workunit.client.0.smithi132.stdout: 0.263671875, 2024-01-24T23:25:03.367 INFO:tasks.workunit.client.0.smithi132.stdout: 0, 2024-01-24T23:25:03.367 INFO:tasks.workunit.client.0.smithi132.stdout: 0 2024-01-24T23:25:03.367 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.367 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.367 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.367 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.367 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -2, 2024-01-24T23:25:03.367 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.367 INFO:tasks.workunit.client.0.smithi132.stdout: [], 2024-01-24T23:25:03.368 INFO:tasks.workunit.client.0.smithi132.stdout: [], 2024-01-24T23:25:03.368 INFO:tasks.workunit.client.0.smithi132.stdout: [] 2024-01-24T23:25:03.368 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.368 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.368 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.368 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -3, 2024-01-24T23:25:03.368 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.368 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.368 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625, 2024-01-24T23:25:03.368 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625, 2024-01-24T23:25:03.368 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625 2024-01-24T23:25:03.368 INFO:tasks.workunit.client.0.smithi132.stdout: ], 2024-01-24T23:25:03.369 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.369 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625, 2024-01-24T23:25:03.369 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625, 2024-01-24T23:25:03.369 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625 2024-01-24T23:25:03.369 INFO:tasks.workunit.client.0.smithi132.stdout: ], 2024-01-24T23:25:03.369 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.369 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625, 2024-01-24T23:25:03.369 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625, 2024-01-24T23:25:03.369 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625 2024-01-24T23:25:03.369 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.369 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.369 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.369 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.369 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -4, 2024-01-24T23:25:03.369 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.370 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.370 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625 2024-01-24T23:25:03.370 INFO:tasks.workunit.client.0.smithi132.stdout: ], 2024-01-24T23:25:03.370 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.370 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625 2024-01-24T23:25:03.370 INFO:tasks.workunit.client.0.smithi132.stdout: ], 2024-01-24T23:25:03.370 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.370 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625 2024-01-24T23:25:03.370 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.370 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.370 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.370 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.370 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -5, 2024-01-24T23:25:03.371 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.371 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.371 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625, 2024-01-24T23:25:03.371 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625, 2024-01-24T23:25:03.371 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625 2024-01-24T23:25:03.371 INFO:tasks.workunit.client.0.smithi132.stdout: ], 2024-01-24T23:25:03.371 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.371 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625, 2024-01-24T23:25:03.371 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625, 2024-01-24T23:25:03.371 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625 2024-01-24T23:25:03.371 INFO:tasks.workunit.client.0.smithi132.stdout: ], 2024-01-24T23:25:03.371 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.371 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625, 2024-01-24T23:25:03.372 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625, 2024-01-24T23:25:03.372 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625 2024-01-24T23:25:03.372 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.372 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.372 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.372 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.372 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -6, 2024-01-24T23:25:03.372 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.372 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.372 INFO:tasks.workunit.client.0.smithi132.stdout: 0, 2024-01-24T23:25:03.372 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625, 2024-01-24T23:25:03.372 INFO:tasks.workunit.client.0.smithi132.stdout: 0, 2024-01-24T23:25:03.373 INFO:tasks.workunit.client.0.smithi132.stdout: 0 2024-01-24T23:25:03.373 INFO:tasks.workunit.client.0.smithi132.stdout: ], 2024-01-24T23:25:03.373 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.373 INFO:tasks.workunit.client.0.smithi132.stdout: 0, 2024-01-24T23:25:03.373 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625, 2024-01-24T23:25:03.373 INFO:tasks.workunit.client.0.smithi132.stdout: 0, 2024-01-24T23:25:03.373 INFO:tasks.workunit.client.0.smithi132.stdout: 0 2024-01-24T23:25:03.373 INFO:tasks.workunit.client.0.smithi132.stdout: ], 2024-01-24T23:25:03.373 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.373 INFO:tasks.workunit.client.0.smithi132.stdout: 0, 2024-01-24T23:25:03.373 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625, 2024-01-24T23:25:03.373 INFO:tasks.workunit.client.0.smithi132.stdout: 0, 2024-01-24T23:25:03.373 INFO:tasks.workunit.client.0.smithi132.stdout: 0 2024-01-24T23:25:03.374 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.374 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.374 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.374 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.374 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -7, 2024-01-24T23:25:03.374 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.374 INFO:tasks.workunit.client.0.smithi132.stdout: [], 2024-01-24T23:25:03.374 INFO:tasks.workunit.client.0.smithi132.stdout: [], 2024-01-24T23:25:03.374 INFO:tasks.workunit.client.0.smithi132.stdout: [] 2024-01-24T23:25:03.374 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.374 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.374 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.375 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -8, 2024-01-24T23:25:03.375 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.375 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.375 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625 2024-01-24T23:25:03.375 INFO:tasks.workunit.client.0.smithi132.stdout: ], 2024-01-24T23:25:03.375 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.375 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625 2024-01-24T23:25:03.375 INFO:tasks.workunit.client.0.smithi132.stdout: ], 2024-01-24T23:25:03.375 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.375 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625 2024-01-24T23:25:03.375 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.375 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.375 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.376 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.376 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -9, 2024-01-24T23:25:03.376 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.376 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.376 INFO:tasks.workunit.client.0.smithi132.stdout: 0, 2024-01-24T23:25:03.376 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625, 2024-01-24T23:25:03.376 INFO:tasks.workunit.client.0.smithi132.stdout: 0, 2024-01-24T23:25:03.376 INFO:tasks.workunit.client.0.smithi132.stdout: 0 2024-01-24T23:25:03.376 INFO:tasks.workunit.client.0.smithi132.stdout: ], 2024-01-24T23:25:03.376 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.376 INFO:tasks.workunit.client.0.smithi132.stdout: 0, 2024-01-24T23:25:03.376 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625, 2024-01-24T23:25:03.376 INFO:tasks.workunit.client.0.smithi132.stdout: 0, 2024-01-24T23:25:03.376 INFO:tasks.workunit.client.0.smithi132.stdout: 0 2024-01-24T23:25:03.377 INFO:tasks.workunit.client.0.smithi132.stdout: ], 2024-01-24T23:25:03.377 INFO:tasks.workunit.client.0.smithi132.stdout: [ 2024-01-24T23:25:03.377 INFO:tasks.workunit.client.0.smithi132.stdout: 0, 2024-01-24T23:25:03.377 INFO:tasks.workunit.client.0.smithi132.stdout: 0.087890625, 2024-01-24T23:25:03.377 INFO:tasks.workunit.client.0.smithi132.stdout: 0, 2024-01-24T23:25:03.377 INFO:tasks.workunit.client.0.smithi132.stdout: 0 2024-01-24T23:25:03.377 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.377 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.377 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.377 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.377 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -10, 2024-01-24T23:25:03.377 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.377 INFO:tasks.workunit.client.0.smithi132.stdout: [], 2024-01-24T23:25:03.377 INFO:tasks.workunit.client.0.smithi132.stdout: [], 2024-01-24T23:25:03.378 INFO:tasks.workunit.client.0.smithi132.stdout: [] 2024-01-24T23:25:03.378 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.378 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.378 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.378 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -11, 2024-01-24T23:25:03.378 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.378 INFO:tasks.workunit.client.0.smithi132.stdout: [], 2024-01-24T23:25:03.378 INFO:tasks.workunit.client.0.smithi132.stdout: [], 2024-01-24T23:25:03.378 INFO:tasks.workunit.client.0.smithi132.stdout: [] 2024-01-24T23:25:03.378 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.378 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.378 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.378 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -12, 2024-01-24T23:25:03.378 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.379 INFO:tasks.workunit.client.0.smithi132.stdout: [], 2024-01-24T23:25:03.379 INFO:tasks.workunit.client.0.smithi132.stdout: [], 2024-01-24T23:25:03.379 INFO:tasks.workunit.client.0.smithi132.stdout: [] 2024-01-24T23:25:03.379 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.379 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.379 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.379 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -13, 2024-01-24T23:25:03.379 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.379 INFO:tasks.workunit.client.0.smithi132.stdout: [], 2024-01-24T23:25:03.379 INFO:tasks.workunit.client.0.smithi132.stdout: [], 2024-01-24T23:25:03.379 INFO:tasks.workunit.client.0.smithi132.stdout: [] 2024-01-24T23:25:03.379 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.379 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.379 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.379 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -14, 2024-01-24T23:25:03.380 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.380 INFO:tasks.workunit.client.0.smithi132.stdout: [], 2024-01-24T23:25:03.380 INFO:tasks.workunit.client.0.smithi132.stdout: [], 2024-01-24T23:25:03.380 INFO:tasks.workunit.client.0.smithi132.stdout: [] 2024-01-24T23:25:03.380 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.380 INFO:tasks.workunit.client.0.smithi132.stdout: }, 2024-01-24T23:25:03.380 INFO:tasks.workunit.client.0.smithi132.stdout: { 2024-01-24T23:25:03.380 INFO:tasks.workunit.client.0.smithi132.stdout: "bucket_id": -15, 2024-01-24T23:25:03.380 INFO:tasks.workunit.client.0.smithi132.stdout: "weight_set": [ 2024-01-24T23:25:03.380 INFO:tasks.workunit.client.0.smithi132.stdout: [], 2024-01-24T23:25:03.380 INFO:tasks.workunit.client.0.smithi132.stdout: [], 2024-01-24T23:25:03.380 INFO:tasks.workunit.client.0.smithi132.stdout: [] 2024-01-24T23:25:03.380 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.381 INFO:tasks.workunit.client.0.smithi132.stdout: } 2024-01-24T23:25:03.381 INFO:tasks.workunit.client.0.smithi132.stdout: ] 2024-01-24T23:25:03.381 INFO:tasks.workunit.client.0.smithi132.stdout:} 2024-01-24T23:25:03.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.353+0000 7f6f7b100640 1 -- 172.21.15.132:0/986355290 >> v1:172.21.15.132:6812/1835401922 conn(0x7f6f60036ec0 legacy=0x7f6f60039380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:03.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.353+0000 7f6f7b100640 10 monclient: shutdown 2024-01-24T23:25:03.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.353+0000 7f6f7b100640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:03.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.353+0000 7f6f7b100640 1 -- 172.21.15.132:0/986355290 >> v1:172.21.15.132:6791/0 conn(0x7f6f7413a3b0 legacy=0x7f6f741d39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:03.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.353+0000 7f6f7b100640 1 -- 172.21.15.132:0/986355290 shutdown_connections 2024-01-24T23:25:03.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.353+0000 7f6f7b100640 1 -- 172.21.15.132:0/986355290 >> 172.21.15.132:0/986355290 conn(0x7f6f740a2b40 msgr2=0x7f6f740a4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:03.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.353+0000 7f6f7b100640 1 -- 172.21.15.132:0/986355290 shutdown_connections 2024-01-24T23:25:03.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.353+0000 7f6f7b100640 1 -- 172.21.15.132:0/986355290 wait complete. 2024-01-24T23:25:03.382 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set reweight fooset osd.0 .9 2024-01-24T23:25:03.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.481+0000 7fe0b21c3640 10 monclient: get_monmap_and_config 2024-01-24T23:25:03.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.481+0000 7fe0b21c3640 10 monclient: build_initial_monmap 2024-01-24T23:25:03.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.481+0000 7fe0b21c3640 10 monclient: monmap: 2024-01-24T23:25:03.486 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:03.486 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:03.487 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:03.487642+0000 2024-01-24T23:25:03.487 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:03.487642+0000 2024-01-24T23:25:03.487 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:03.487 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:03.487 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:03.487 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:03.487 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:03.487 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:03.487 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:03.487 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:03.487 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:03.487 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:03.488 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:03.488 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:03.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 1 Processor -- start 2024-01-24T23:25:03.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 1 -- start start 2024-01-24T23:25:03.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 10 monclient: init 2024-01-24T23:25:03.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:03.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 10 monclient: _add_conns ranks=[6,8,0,4,2,5,1,7,3] 2024-01-24T23:25:03.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 10 monclient(hunting): picked mon.noname-d con 0x7fe0ac099510 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:03.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 10 monclient(hunting): picked mon.noname-e con 0x7fe0ac09fdd0 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:03.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 10 monclient(hunting): picked mon.noname-a con 0x7fe0ac0a01f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:03.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ac064700 con 0x7fe0ac0a01f0 2024-01-24T23:25:03.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ac09ee20 con 0x7fe0ac099510 2024-01-24T23:25:03.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ac09eb40 con 0x7fe0ac09fdd0 2024-01-24T23:25:03.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:03.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 10 monclient(hunting): authenticate will time out at 1190.812942s 2024-01-24T23:25:03.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0abfff640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fe0ac0a01f0 legacy=0x7fe0ac13c280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:25:03.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aaffd640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fe0ac09fdd0 0x7fe0ac138b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:40664/0 (socket says 172.21.15.132:40664) 2024-01-24T23:25:03.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aaffd640 1 -- 172.21.15.132:0/222283060 learned_addr learned my addr 172.21.15.132:0/222283060 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:03.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 1 -- 172.21.15.132:0/222283060 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3520406647 0 0) 0x7fe0ac09ee20 con 0x7fe0ac099510 2024-01-24T23:25:03.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:03.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:03.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:03.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 10 monclient(hunting): my global_id is 5901 2024-01-24T23:25:03.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 1 -- 172.21.15.132:0/222283060 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe094005740 con 0x7fe0ac099510 2024-01-24T23:25:03.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 1 -- 172.21.15.132:0/222283060 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142705113 0 0) 0x7fe0ac09eb40 con 0x7fe0ac09fdd0 2024-01-24T23:25:03.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:03.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:03.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:03.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 10 monclient(hunting): my global_id is 84986 2024-01-24T23:25:03.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 1 -- 172.21.15.132:0/222283060 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0ac09ee20 con 0x7fe0ac09fdd0 2024-01-24T23:25:03.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 1 -- 172.21.15.132:0/222283060 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1605608910 0 0) 0x7fe0ac09ee20 con 0x7fe0ac09fdd0 2024-01-24T23:25:03.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 1 -- 172.21.15.132:0/222283060 >> v1:172.21.15.115:6792/0 conn(0x7fe0ac099510 legacy=0x7fe0ac0998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:03.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 1 -- 172.21.15.132:0/222283060 >> v1:172.21.15.115:6789/0 conn(0x7fe0ac0a01f0 legacy=0x7fe0ac13c280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:03.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:25:03.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 1 monclient: found mon.noname-e 2024-01-24T23:25:03.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:03.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:25:03.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 1 -- 172.21.15.132:0/222283060 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac0a3990 con 0x7fe0ac09fdd0 2024-01-24T23:25:03.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 10 monclient: _finish_auth 0 2024-01-24T23:25:03.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:25:03.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:03.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 5 monclient: authenticate success, global_id 84986 2024-01-24T23:25:03.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 1 -- 172.21.15.132:0/222283060 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe0a00017a0 con 0x7fe0ac09fdd0 2024-01-24T23:25:03.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:03.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:03.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:25:03.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 10 monclient: dump: 2024-01-24T23:25:03.501 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:03.501 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:03.501 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:03.501 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:03.501 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:03.501 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:03.501 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:03.502 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:03.502 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:03.502 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:03.502 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:03.502 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:03.502 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:03.502 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:03.502 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:03.502 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:03.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:03.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 1 -- 172.21.15.132:0/222283060 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe0a00019c0 con 0x7fe0ac09fdd0 2024-01-24T23:25:03.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:03.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:03.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 4 set_mon_vals no callback set 2024-01-24T23:25:03.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 1 -- 172.21.15.132:0/222283060 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe0a0004bf0 con 0x7fe0ac09fdd0 2024-01-24T23:25:03.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:03.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:03.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0aa7fc640 10 monclient: dump: 2024-01-24T23:25:03.503 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:03.503 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:03.504 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:03.504 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:03.504 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:03.504 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:03.504 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:03.504 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:03.504 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:03.504 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:03.504 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:03.504 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:03.504 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:03.504 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:03.505 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:03.505 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:03.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 10 monclient: shutdown 2024-01-24T23:25:03.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:03.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 1 -- 172.21.15.132:0/222283060 >> v1:172.21.15.115:6793/0 conn(0x7fe0ac09fdd0 legacy=0x7fe0ac138b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:03.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 1 -- 172.21.15.132:0/222283060 shutdown_connections 2024-01-24T23:25:03.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 1 -- 172.21.15.132:0/222283060 >> 172.21.15.132:0/222283060 conn(0x7fe0ac0a2c40 msgr2=0x7fe0ac0a3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:03.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 1 -- 172.21.15.132:0/222283060 shutdown_connections 2024-01-24T23:25:03.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 1 -- 172.21.15.132:0/222283060 wait complete. 2024-01-24T23:25:03.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 10 monclient: build_initial_monmap 2024-01-24T23:25:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 10 monclient: monmap: 2024-01-24T23:25:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:03.507 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:03.507 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:03.507 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:03.507 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:03.507 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:03.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 1 Processor -- start 2024-01-24T23:25:03.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 1 -- start start 2024-01-24T23:25:03.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.485+0000 7fe0b21c3640 10 monclient: init 2024-01-24T23:25:03.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0b21c3640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:03.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0b21c3640 10 monclient: _add_conns ranks=[2,8,5,4,3,0,1,7,6] 2024-01-24T23:25:03.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0b21c3640 10 monclient(hunting): picked mon.noname-b con 0x7fe0ac099510 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:03.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0b21c3640 10 monclient(hunting): picked mon.noname-e con 0x7fe0ac09fdd0 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:03.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0b21c3640 10 monclient(hunting): picked mon.noname-h con 0x7fe0ac0a01f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:03.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0b21c3640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ac09e010 con 0x7fe0ac099510 2024-01-24T23:25:03.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0b21c3640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ac06e930 con 0x7fe0ac0a01f0 2024-01-24T23:25:03.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0b21c3640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ac1b96c0 con 0x7fe0ac09fdd0 2024-01-24T23:25:03.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0b21c3640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:03.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0b21c3640 10 monclient(hunting): authenticate will time out at 1190.816453s 2024-01-24T23:25:03.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0abfff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fe0ac0a01f0 0x7fe0ac1cf800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:35372/0 (socket says 172.21.15.132:35372) 2024-01-24T23:25:03.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0abfff640 1 -- 172.21.15.132:0/2988806100 learned_addr learned my addr 172.21.15.132:0/2988806100 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:03.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 1 -- 172.21.15.132:0/2988806100 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4124176510 0 0) 0x7fe0ac06e930 con 0x7fe0ac0a01f0 2024-01-24T23:25:03.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:03.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:03.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:03.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 10 monclient(hunting): my global_id is 5756 2024-01-24T23:25:03.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 1 -- 172.21.15.132:0/2988806100 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe098003530 con 0x7fe0ac0a01f0 2024-01-24T23:25:03.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 1 -- 172.21.15.132:0/2988806100 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3452573650 0 0) 0x7fe098003530 con 0x7fe0ac0a01f0 2024-01-24T23:25:03.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 1 -- 172.21.15.132:0/2988806100 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe0ac06e930 con 0x7fe0ac0a01f0 2024-01-24T23:25:03.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 1 -- 172.21.15.132:0/2988806100 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe0a40045f0 con 0x7fe0ac0a01f0 2024-01-24T23:25:03.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:03.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:25:03.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 10 monclient(hunting): dump: 2024-01-24T23:25:03.513 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:03.513 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:03.513 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:03.513 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:03.513 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:03.513 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:03.513 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:03.514 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:03.514 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:03.514 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:03.514 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:03.514 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:03.514 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:03.514 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:03.514 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:03.514 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:03.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:03.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 1 -- 172.21.15.132:0/2988806100 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3212932457 0 0) 0x7fe0ac1b96c0 con 0x7fe0ac09fdd0 2024-01-24T23:25:03.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:03.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:03.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:03.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 10 monclient(hunting): my global_id is 84995 2024-01-24T23:25:03.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 1 -- 172.21.15.132:0/2988806100 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe098003530 con 0x7fe0ac09fdd0 2024-01-24T23:25:03.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 1 -- 172.21.15.132:0/2988806100 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1232348122 0 0) 0x7fe0ac09e010 con 0x7fe0ac099510 2024-01-24T23:25:03.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:03.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:03.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:03.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 10 monclient(hunting): my global_id is 85124 2024-01-24T23:25:03.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 1 -- 172.21.15.132:0/2988806100 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0ac1b96c0 con 0x7fe0ac099510 2024-01-24T23:25:03.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 1 -- 172.21.15.132:0/2988806100 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 294310977 0 0) 0x7fe0ac06e930 con 0x7fe0ac0a01f0 2024-01-24T23:25:03.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 1 -- 172.21.15.132:0/2988806100 >> v1:172.21.15.115:6793/0 conn(0x7fe0ac09fdd0 legacy=0x7fe0ac1b8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:03.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 1 -- 172.21.15.132:0/2988806100 >> v1:172.21.15.115:6790/0 conn(0x7fe0ac099510 legacy=0x7fe0ac1b88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:03.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 10 monclient: _finish_hunting 0 2024-01-24T23:25:03.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 1 monclient: found mon.h 2024-01-24T23:25:03.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:03.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:03.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 1 -- 172.21.15.132:0/2988806100 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac070c20 con 0x7fe0ac0a01f0 2024-01-24T23:25:03.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 10 monclient: _finish_auth 0 2024-01-24T23:25:03.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 10 monclient: _check_auth_tickets 2024-01-24T23:25:03.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:03.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0b21c3640 5 monclient: authenticate success, global_id 5756 2024-01-24T23:25:03.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0b21c3640 10 monclient: _renew_subs 2024-01-24T23:25:03.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0b21c3640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:03.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0b21c3640 1 -- 172.21.15.132:0/2988806100 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0ac070380 con 0x7fe0ac0a01f0 2024-01-24T23:25:03.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0b21c3640 10 monclient: _renew_subs 2024-01-24T23:25:03.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0b21c3640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0b21c3640 1 -- 172.21.15.132:0/2988806100 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0ac072410 con 0x7fe0ac0a01f0 2024-01-24T23:25:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 1 -- 172.21.15.132:0/2988806100 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe0a40048a0 con 0x7fe0ac0a01f0 2024-01-24T23:25:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 1 -- 172.21.15.132:0/2988806100 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe0a4004dd0 con 0x7fe0ac0a01f0 2024-01-24T23:25:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0a97fa640 4 set_mon_vals no callback set 2024-01-24T23:25:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 10 monclient: dump: 2024-01-24T23:25:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:03.518 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 1 -- 172.21.15.132:0/2988806100 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fe0a4004f30 con 0x7fe0ac0a01f0 2024-01-24T23:25:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0b21c3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0b21c3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0b21c3640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0b21c3640 1 -- 172.21.15.132:0/2988806100 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe08c005090 con 0x7fe0ac0a01f0 2024-01-24T23:25:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe093fff640 1 -- 172.21.15.132:0/2988806100 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(122..122 src has 1..122) v4 ==== 7321+0+0 (unknown 136261091 0 0) 0x7fe0a4044870 con 0x7fe0ac0a01f0 2024-01-24T23:25:03.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.489+0000 7fe0ab7fe640 10 monclient: get_auth_request con 0x7fe0980372c0 auth_method 0 2024-01-24T23:25:03.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.493+0000 7fe093fff640 1 -- 172.21.15.132:0/2988806100 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fe0a4014d60 con 0x7fe0ac0a01f0 2024-01-24T23:25:03.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.493+0000 7fe093fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:03.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.493+0000 7fe093fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:03.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.641+0000 7fe0b21c3640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-01-24T23:25:03.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.641+0000 7fe0b21c3640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-01-24T23:25:03.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.641+0000 7fe0b21c3640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:03.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:03.641+0000 7fe0b21c3640 1 -- 172.21.15.132:0/2988806100 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]} v 0) v1 -- 0x7fe08c0058c0 con 0x7fe0ac0a01f0 2024-01-24T23:25:04.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:04.041+0000 7fe093fff640 1 -- 172.21.15.132:0/2988806100 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}]=0 v123) v1 ==== 129+0+0 (unknown 3127407313 0 0) 0x7fe0a40131d0 con 0x7fe0ac0a01f0 2024-01-24T23:25:04.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:04.041+0000 7fe093fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-01-24T23:25:04.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:04.041+0000 7fe093fff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:04.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:04.145+0000 7fe0b21c3640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-01-24T23:25:04.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:04.145+0000 7fe0b21c3640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-01-24T23:25:04.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:04.145+0000 7fe0b21c3640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:04.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:04.145+0000 7fe0b21c3640 1 -- 172.21.15.132:0/2988806100 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]} v 0) v1 -- 0x7fe08c0008e0 con 0x7fe0ac0a01f0 2024-01-24T23:25:05.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.057+0000 7fe093fff640 1 -- 172.21.15.132:0/2988806100 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}]=0 v124) v1 ==== 129+0+0 (unknown 1992991485 0 0) 0x7fe0a4014950 con 0x7fe0ac0a01f0 2024-01-24T23:25:05.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.057+0000 7fe093fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-01-24T23:25:05.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.057+0000 7fe093fff640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:25:05.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.061+0000 7fe0b21c3640 1 -- 172.21.15.132:0/2988806100 >> v1:172.21.15.132:6812/1835401922 conn(0x7fe0980372c0 legacy=0x7fe098039780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:05.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.061+0000 7fe0b21c3640 10 monclient: shutdown 2024-01-24T23:25:05.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.061+0000 7fe0b21c3640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:05.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.061+0000 7fe0b21c3640 1 -- 172.21.15.132:0/2988806100 >> v1:172.21.15.132:6791/0 conn(0x7fe0ac0a01f0 legacy=0x7fe0ac1cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:05.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.061+0000 7fe0b21c3640 1 -- 172.21.15.132:0/2988806100 shutdown_connections 2024-01-24T23:25:05.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.061+0000 7fe0b21c3640 1 -- 172.21.15.132:0/2988806100 >> 172.21.15.132:0/2988806100 conn(0x7fe0ac0a2c40 msgr2=0x7fe0ac1ba3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:05.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.061+0000 7fe0b21c3640 1 -- 172.21.15.132:0/2988806100 shutdown_connections 2024-01-24T23:25:05.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.061+0000 7fe0b21c3640 1 -- 172.21.15.132:0/2988806100 wait complete. 2024-01-24T23:25:05.085 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd crush weight-set reweight fooset osd.0 .9 .9 2024-01-24T23:25:05.086 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:25:05.086 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set reweight fooset osd.0 .9 .9 2024-01-24T23:25:05.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.201+0000 7fe9a57da640 10 monclient: get_monmap_and_config 2024-01-24T23:25:05.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.201+0000 7fe9a57da640 10 monclient: build_initial_monmap 2024-01-24T23:25:05.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.201+0000 7fe9a57da640 10 monclient: monmap: 2024-01-24T23:25:05.206 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:05.206 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:05.206 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:05.207386+0000 2024-01-24T23:25:05.206 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:05.207386+0000 2024-01-24T23:25:05.206 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:05.206 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:05.207 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:05.207 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:05.207 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:05.207 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:05.207 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:05.207 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:05.207 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:05.207 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:05.207 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:05.207 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:05.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.201+0000 7fe9a57da640 1 Processor -- start 2024-01-24T23:25:05.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.201+0000 7fe9a57da640 1 -- start start 2024-01-24T23:25:05.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.201+0000 7fe9a57da640 10 monclient: init 2024-01-24T23:25:05.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.201+0000 7fe9a57da640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:05.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.201+0000 7fe9a57da640 10 monclient: _add_conns ranks=[5,2,1,8,3,6,7,4,0] 2024-01-24T23:25:05.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.201+0000 7fe9a57da640 10 monclient(hunting): picked mon.noname-h con 0x7fe9a009bc10 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 10 monclient(hunting): picked mon.noname-b con 0x7fe9a00a00d0 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 10 monclient(hunting): picked mon.noname-f con 0x7fe9a00a0560 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a0064700 con 0x7fe9a00a0560 2024-01-24T23:25:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a009bff0 con 0x7fe9a00a00d0 2024-01-24T23:25:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a00a1870 con 0x7fe9a009bc10 2024-01-24T23:25:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 10 monclient(hunting): authenticate will time out at 1192.532101s 2024-01-24T23:25:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99f7fe640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fe9a00a0560 legacy=0x7fe9a013bef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99effd640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fe9a009bc10 0x7fe9a009f9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:35864/0 (socket says 172.21.15.132:35864) 2024-01-24T23:25:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99effd640 1 -- 172.21.15.132:0/1206839922 learned_addr learned my addr 172.21.15.132:0/1206839922 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 1 -- 172.21.15.132:0/1206839922 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1247749176 0 0) 0x7fe9a00a1870 con 0x7fe9a009bc10 2024-01-24T23:25:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 10 monclient(hunting): my global_id is 5765 2024-01-24T23:25:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 1 -- 172.21.15.132:0/1206839922 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe988003530 con 0x7fe9a009bc10 2024-01-24T23:25:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 1 -- 172.21.15.132:0/1206839922 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2499464296 0 0) 0x7fe988003530 con 0x7fe9a009bc10 2024-01-24T23:25:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 1 -- 172.21.15.132:0/1206839922 >> v1:172.21.15.115:6790/0 conn(0x7fe9a00a00d0 legacy=0x7fe9a01387c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 1 -- 172.21.15.132:0/1206839922 >> v1:172.21.15.132:6789/0 conn(0x7fe9a00a0560 legacy=0x7fe9a013bef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 10 monclient: _finish_hunting 0 2024-01-24T23:25:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 1 monclient: found mon.noname-h 2024-01-24T23:25:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:25:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 1 -- 172.21.15.132:0/1206839922 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a00a0fa0 con 0x7fe9a009bc10 2024-01-24T23:25:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 10 monclient: _finish_auth 0 2024-01-24T23:25:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 10 monclient: _check_auth_tickets 2024-01-24T23:25:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:05.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 5 monclient: authenticate success, global_id 5765 2024-01-24T23:25:05.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:05.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 1 -- 172.21.15.132:0/1206839922 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe9900017c0 con 0x7fe9a009bc10 2024-01-24T23:25:05.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:05.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:25:05.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 10 monclient: dump: 2024-01-24T23:25:05.216 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:05.216 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:05.217 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 1 -- 172.21.15.132:0/1206839922 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe990001ab0 con 0x7fe9a009bc10 2024-01-24T23:25:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 1 -- 172.21.15.132:0/1206839922 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe990004bf0 con 0x7fe9a009bc10 2024-01-24T23:25:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99dffb640 10 monclient: dump: 2024-01-24T23:25:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:05.219 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:05.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 4 set_mon_vals no callback set 2024-01-24T23:25:05.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 10 monclient: shutdown 2024-01-24T23:25:05.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:05.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 1 -- 172.21.15.132:0/1206839922 >> v1:172.21.15.132:6791/0 conn(0x7fe9a009bc10 legacy=0x7fe9a009f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:05.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 1 -- 172.21.15.132:0/1206839922 shutdown_connections 2024-01-24T23:25:05.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 1 -- 172.21.15.132:0/1206839922 >> 172.21.15.132:0/1206839922 conn(0x7fe9a00a5a10 msgr2=0x7fe9a00a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:05.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 1 -- 172.21.15.132:0/1206839922 shutdown_connections 2024-01-24T23:25:05.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 1 -- 172.21.15.132:0/1206839922 wait complete. 2024-01-24T23:25:05.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 10 monclient: build_initial_monmap 2024-01-24T23:25:05.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 10 monclient: monmap: 2024-01-24T23:25:05.220 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:05.220 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:05.220 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:05.220 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:05.220 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:05.221 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:05.221 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:05.221 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:05.221 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:05.221 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:05.221 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:05.221 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:05.221 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:05.221 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:05.221 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:05.221 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:05.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 1 Processor -- start 2024-01-24T23:25:05.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 1 -- start start 2024-01-24T23:25:05.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 10 monclient: init 2024-01-24T23:25:05.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:05.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 10 monclient: _add_conns ranks=[0,3,8,4,2,5,7,6,1] 2024-01-24T23:25:05.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 10 monclient(hunting): picked mon.noname-a con 0x7fe9a009bc10 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:05.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 10 monclient(hunting): picked mon.noname-g con 0x7fe9a00a00d0 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:05.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 10 monclient(hunting): picked mon.noname-e con 0x7fe9a00a0560 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:05.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a0071fe0 con 0x7fe9a009bc10 2024-01-24T23:25:05.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a009c650 con 0x7fe9a00a00d0 2024-01-24T23:25:05.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a01a4fb0 con 0x7fe9a00a0560 2024-01-24T23:25:05.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:05.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 10 monclient(hunting): authenticate will time out at 1192.535068s 2024-01-24T23:25:05.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99effd640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fe9a009bc10 legacy=0x7fe9a01b98e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:25:05.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99e7fc640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fe9a00a00d0 0x7fe9a01ce380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:53926/0 (socket says 172.21.15.132:53926) 2024-01-24T23:25:05.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe99e7fc640 1 -- 172.21.15.132:0/1121339207 learned_addr learned my addr 172.21.15.132:0/1121339207 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:05.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 1 -- 172.21.15.132:0/1121339207 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2430961212 0 0) 0x7fe9a009c650 con 0x7fe9a00a00d0 2024-01-24T23:25:05.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:05.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:05.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:05.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 10 monclient(hunting): my global_id is 5817 2024-01-24T23:25:05.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 1 -- 172.21.15.132:0/1121339207 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe98c003530 con 0x7fe9a00a00d0 2024-01-24T23:25:05.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 1 -- 172.21.15.132:0/1121339207 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 790621486 0 0) 0x7fe98c003530 con 0x7fe9a00a00d0 2024-01-24T23:25:05.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 1 -- 172.21.15.132:0/1121339207 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe9a009c650 con 0x7fe9a00a00d0 2024-01-24T23:25:05.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 1 -- 172.21.15.132:0/1121339207 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe994001a20 con 0x7fe9a00a00d0 2024-01-24T23:25:05.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:05.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:25:05.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 10 monclient(hunting): dump: 2024-01-24T23:25:05.223 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:05.223 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:05.224 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:05.224 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:05.224 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:05.224 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:05.224 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:05.224 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:05.225 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:05.225 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:05.225 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:05.225 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:05.225 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:05.225 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:05.226 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:05.226 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:05.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:05.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 1 -- 172.21.15.132:0/1121339207 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2947602160 0 0) 0x7fe9a01a4fb0 con 0x7fe9a00a0560 2024-01-24T23:25:05.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:05.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:05.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:05.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 10 monclient(hunting): my global_id is 85004 2024-01-24T23:25:05.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 1 -- 172.21.15.132:0/1121339207 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe98c003530 con 0x7fe9a00a0560 2024-01-24T23:25:05.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 1 -- 172.21.15.132:0/1121339207 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3982014176 0 0) 0x7fe9a009c650 con 0x7fe9a00a00d0 2024-01-24T23:25:05.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 1 -- 172.21.15.132:0/1121339207 >> v1:172.21.15.115:6793/0 conn(0x7fe9a00a0560 legacy=0x7fe9a01d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:05.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 1 -- 172.21.15.132:0/1121339207 >> v1:172.21.15.115:6789/0 conn(0x7fe9a009bc10 legacy=0x7fe9a01b98e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:05.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 10 monclient: _finish_hunting 0 2024-01-24T23:25:05.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 1 monclient: found mon.g 2024-01-24T23:25:05.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:05.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:05.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 1 -- 172.21.15.132:0/1121339207 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a01a72a0 con 0x7fe9a00a00d0 2024-01-24T23:25:05.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 10 monclient: _finish_auth 0 2024-01-24T23:25:05.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 10 monclient: _check_auth_tickets 2024-01-24T23:25:05.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9877fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:05.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 5 monclient: authenticate success, global_id 5817 2024-01-24T23:25:05.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 10 monclient: _renew_subs 2024-01-24T23:25:05.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:05.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 1 -- 172.21.15.132:0/1121339207 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a01a6a00 con 0x7fe9a00a00d0 2024-01-24T23:25:05.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 10 monclient: _renew_subs 2024-01-24T23:25:05.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:05.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.205+0000 7fe9a57da640 1 -- 172.21.15.132:0/1121339207 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a01c6590 con 0x7fe9a00a00d0 2024-01-24T23:25:05.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.209+0000 7fe9877fe640 1 -- 172.21.15.132:0/1121339207 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe994003950 con 0x7fe9a00a00d0 2024-01-24T23:25:05.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.209+0000 7fe9877fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:05.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.209+0000 7fe9877fe640 1 -- 172.21.15.132:0/1121339207 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe994003e00 con 0x7fe9a00a00d0 2024-01-24T23:25:05.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.209+0000 7fe9877fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:05.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.209+0000 7fe99d7fa640 4 set_mon_vals no callback set 2024-01-24T23:25:05.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.209+0000 7fe9877fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:05.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.209+0000 7fe9877fe640 10 monclient: dump: 2024-01-24T23:25:05.229 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:05.229 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:05.229 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:05.229 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:05.229 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:05.229 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:05.229 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:05.229 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:05.229 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:05.229 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:05.230 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:05.230 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:05.230 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:05.230 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:05.230 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:05.230 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:05.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.209+0000 7fe9a57da640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:05.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.209+0000 7fe9a57da640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:05.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.209+0000 7fe9a57da640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:05.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.209+0000 7fe9877fe640 1 -- 172.21.15.132:0/1121339207 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fe994003f60 con 0x7fe9a00a00d0 2024-01-24T23:25:05.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.209+0000 7fe9a57da640 1 -- 172.21.15.132:0/1121339207 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe980005090 con 0x7fe9a00a00d0 2024-01-24T23:25:05.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.209+0000 7fe9877fe640 1 -- 172.21.15.132:0/1121339207 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(124..124 src has 1..124) v4 ==== 7321+0+0 (unknown 3708886129 0 0) 0x7fe9940444d0 con 0x7fe9a00a00d0 2024-01-24T23:25:05.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.209+0000 7fe99effd640 10 monclient: get_auth_request con 0x7fe98c036eb0 auth_method 0 2024-01-24T23:25:05.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.213+0000 7fe9877fe640 1 -- 172.21.15.132:0/1121339207 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fe9940147e0 con 0x7fe9a00a00d0 2024-01-24T23:25:05.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.213+0000 7fe9877fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:05.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.213+0000 7fe9877fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:05.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.377+0000 7fe9a57da640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9, 0.9]}] 2024-01-24T23:25:05.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.377+0000 7fe9a57da640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9, 0.9]}] 2024-01-24T23:25:05.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.377+0000 7fe9a57da640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:05.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.377+0000 7fe9a57da640 1 -- 172.21.15.132:0/1121339207 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9, 0.9]} v 0) v1 -- 0x7fe9800058c0 con 0x7fe9a00a00d0 2024-01-24T23:25:05.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.377+0000 7fe9877fe640 1 -- 172.21.15.132:0/1121339207 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9, 0.9]}]=-22 must specify exact 1 weight values v124) v1 ==== 168+0+0 (unknown 4167510088 0 0) 0x7fe994012e80 con 0x7fe9a00a00d0 2024-01-24T23:25:05.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.377+0000 7fe9877fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9, 0.9]}] 2024-01-24T23:25:05.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.377+0000 7fe9877fe640 10 monclient: _finish_command 2 = mon:22 must specify exact 1 weight values 2024-01-24T23:25:05.381 INFO:tasks.workunit.client.0.smithi132.stderr:Error EINVAL: must specify exact 1 weight values 2024-01-24T23:25:05.384 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:25:05.384 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect_false ceph osd crush weight-set reweight barset osd.0 .9 2024-01-24T23:25:05.384 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -x 2024-01-24T23:25:05.385 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set reweight barset osd.0 .9 2024-01-24T23:25:05.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.493+0000 7fd1b7a26640 10 monclient: get_monmap_and_config 2024-01-24T23:25:05.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.493+0000 7fd1b7a26640 10 monclient: build_initial_monmap 2024-01-24T23:25:05.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.493+0000 7fd1b7a26640 10 monclient: monmap: 2024-01-24T23:25:05.499 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:05.499 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:05.500 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:05.500762+0000 2024-01-24T23:25:05.500 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:05.500762+0000 2024-01-24T23:25:05.500 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:05.500 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:05.500 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:05.500 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:05.500 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:05.500 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:05.500 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:05.500 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:05.500 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:05.500 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:05.500 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:05.500 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:05.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 1 Processor -- start 2024-01-24T23:25:05.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 1 -- start start 2024-01-24T23:25:05.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 10 monclient: init 2024-01-24T23:25:05.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:05.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 10 monclient: _add_conns ranks=[0,2,6,8,4,5,1,7,3] 2024-01-24T23:25:05.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 10 monclient(hunting): picked mon.noname-a con 0x7fd1b0071f30 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:05.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 10 monclient(hunting): picked mon.noname-b con 0x7fd1b009be60 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:05.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 10 monclient(hunting): picked mon.noname-d con 0x7fd1b01419e0 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:05.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1b0064700 con 0x7fd1b0071f30 2024-01-24T23:25:05.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1b00a1940 con 0x7fd1b009be60 2024-01-24T23:25:05.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1b009b1a0 con 0x7fd1b01419e0 2024-01-24T23:25:05.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:05.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 10 monclient(hunting): authenticate will time out at 1192.825375s 2024-01-24T23:25:05.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b57c2640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fd1b0071f30 legacy=0x7fd1b0072310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:25:05.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b5fc3640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fd1b01419e0 0x7fd1b0143dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:38876/0 (socket says 172.21.15.132:38876) 2024-01-24T23:25:05.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b4fc1640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fd1b009be60 0x7fd1b01402c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:35934/0 (socket says 172.21.15.132:35934) 2024-01-24T23:25:05.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b5fc3640 1 -- 172.21.15.132:0/2047028337 learned_addr learned my addr 172.21.15.132:0/2047028337 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:05.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 1 -- 172.21.15.132:0/2047028337 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 498799079 0 0) 0x7fd1b00a1940 con 0x7fd1b009be60 2024-01-24T23:25:05.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:05.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:05.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:05.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 10 monclient(hunting): my global_id is 85133 2024-01-24T23:25:05.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 1 -- 172.21.15.132:0/2047028337 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd19c003530 con 0x7fd1b009be60 2024-01-24T23:25:05.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 1 -- 172.21.15.132:0/2047028337 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2743883816 0 0) 0x7fd1b009b1a0 con 0x7fd1b01419e0 2024-01-24T23:25:05.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:05.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:05.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:05.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 10 monclient(hunting): my global_id is 5910 2024-01-24T23:25:05.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 1 -- 172.21.15.132:0/2047028337 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1b00a1940 con 0x7fd1b01419e0 2024-01-24T23:25:05.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 1 -- 172.21.15.132:0/2047028337 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2857576382 0 0) 0x7fd19c003530 con 0x7fd1b009be60 2024-01-24T23:25:05.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 1 -- 172.21.15.132:0/2047028337 >> v1:172.21.15.115:6792/0 conn(0x7fd1b01419e0 legacy=0x7fd1b0143dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:05.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 1 -- 172.21.15.132:0/2047028337 >> v1:172.21.15.115:6789/0 conn(0x7fd1b0071f30 legacy=0x7fd1b0072310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:05.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 10 monclient: _finish_hunting 0 2024-01-24T23:25:05.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 1 monclient: found mon.noname-b 2024-01-24T23:25:05.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:05.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:25:05.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 1 -- 172.21.15.132:0/2047028337 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b009b480 con 0x7fd1b009be60 2024-01-24T23:25:05.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 10 monclient: _finish_auth 0 2024-01-24T23:25:05.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 10 monclient: _check_auth_tickets 2024-01-24T23:25:05.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:05.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 1 -- 172.21.15.132:0/2047028337 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd1a40016f0 con 0x7fd1b009be60 2024-01-24T23:25:05.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:05.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:25:05.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 10 monclient: dump: 2024-01-24T23:25:05.510 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:05.510 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:05.510 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:05.510 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:05.510 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:05.510 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:05.510 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:05.510 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:05.510 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:05.510 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:05.510 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:05.510 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:05.510 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:05.510 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:05.510 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:05.511 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:05.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 5 monclient: authenticate success, global_id 85133 2024-01-24T23:25:05.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:05.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 1 -- 172.21.15.132:0/2047028337 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd1a4001910 con 0x7fd1b009be60 2024-01-24T23:25:05.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:05.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:05.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 4 set_mon_vals no callback set 2024-01-24T23:25:05.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 1 -- 172.21.15.132:0/2047028337 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd1a40047f0 con 0x7fd1b009be60 2024-01-24T23:25:05.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:05.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:25:05.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1a3fff640 10 monclient: dump: 2024-01-24T23:25:05.511 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:05.511 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:05.511 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:05.511 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:05.512 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:05.512 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:05.512 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:05.512 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:05.512 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:05.512 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:05.512 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:05.512 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:05.512 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:05.512 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:05.512 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:05.512 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:05.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 10 monclient: shutdown 2024-01-24T23:25:05.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:05.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 1 -- 172.21.15.132:0/2047028337 >> v1:172.21.15.115:6790/0 conn(0x7fd1b009be60 legacy=0x7fd1b01402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:05.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 1 -- 172.21.15.132:0/2047028337 shutdown_connections 2024-01-24T23:25:05.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 1 -- 172.21.15.132:0/2047028337 >> 172.21.15.132:0/2047028337 conn(0x7fd1b00a1bc0 msgr2=0x7fd1b00a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:05.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 1 -- 172.21.15.132:0/2047028337 shutdown_connections 2024-01-24T23:25:05.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 1 -- 172.21.15.132:0/2047028337 wait complete. 2024-01-24T23:25:05.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 10 monclient: build_initial_monmap 2024-01-24T23:25:05.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.497+0000 7fd1b7a26640 10 monclient: monmap: 2024-01-24T23:25:05.513 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:05.513 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:05.513 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:05.513 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:05.513 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:05.513 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:05.514 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:05.514 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:05.514 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:05.514 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:05.514 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:05.514 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:05.514 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:05.514 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:05.514 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:05.514 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:05.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b7a26640 1 Processor -- start 2024-01-24T23:25:05.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b7a26640 1 -- start start 2024-01-24T23:25:05.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b7a26640 10 monclient: init 2024-01-24T23:25:05.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b7a26640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:05.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b7a26640 10 monclient: _add_conns ranks=[4,7,2,3,0,8,6,1,5] 2024-01-24T23:25:05.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b7a26640 10 monclient(hunting): picked mon.noname-c con 0x7fd1b0071f30 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:05.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b7a26640 10 monclient(hunting): picked mon.noname-i con 0x7fd1b009be60 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:05.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b7a26640 10 monclient(hunting): picked mon.noname-b con 0x7fd1b01419e0 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:05.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b7a26640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1b0098ce0 con 0x7fd1b01419e0 2024-01-24T23:25:05.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b7a26640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1b0097f50 con 0x7fd1b0071f30 2024-01-24T23:25:05.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b7a26640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1b0095590 con 0x7fd1b009be60 2024-01-24T23:25:05.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b7a26640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:05.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b7a26640 10 monclient(hunting): authenticate will time out at 1192.828817s 2024-01-24T23:25:05.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b4fc1640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fd1b009be60 0x7fd1b0094e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:56958/0 (socket says 172.21.15.132:56958) 2024-01-24T23:25:05.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b4fc1640 1 -- 172.21.15.132:0/3409379678 learned_addr learned my addr 172.21.15.132:0/3409379678 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:05.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b57c2640 1 -- 172.21.15.132:0/3409379678 >> v1:172.21.15.115:6791/0 conn(0x7fd1b0071f30 legacy=0x7fd1b0094770 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:25:05.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 1 -- 172.21.15.132:0/3409379678 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4195018989 0 0) 0x7fd1b0095590 con 0x7fd1b009be60 2024-01-24T23:25:05.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:05.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:05.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:05.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 10 monclient(hunting): my global_id is 5767 2024-01-24T23:25:05.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 1 -- 172.21.15.132:0/3409379678 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd194003530 con 0x7fd1b009be60 2024-01-24T23:25:05.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 1 -- 172.21.15.132:0/3409379678 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1563691050 0 0) 0x7fd194003530 con 0x7fd1b009be60 2024-01-24T23:25:05.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 1 -- 172.21.15.132:0/3409379678 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd1b0095590 con 0x7fd1b009be60 2024-01-24T23:25:05.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 1 -- 172.21.15.132:0/3409379678 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd1a40015f0 con 0x7fd1b009be60 2024-01-24T23:25:05.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:05.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:25:05.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 10 monclient(hunting): dump: 2024-01-24T23:25:05.516 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:05.516 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:05.516 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:05.516 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:05.516 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:05.516 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:05.517 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:05.517 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:05.517 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:05.517 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:05.517 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:05.517 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:05.517 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:05.517 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:05.517 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:05.517 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:05.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:05.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 1 -- 172.21.15.132:0/3409379678 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3920931390 0 0) 0x7fd1b0098ce0 con 0x7fd1b01419e0 2024-01-24T23:25:05.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:05.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:05.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:05.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 10 monclient(hunting): my global_id is 85142 2024-01-24T23:25:05.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 1 -- 172.21.15.132:0/3409379678 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd194003530 con 0x7fd1b01419e0 2024-01-24T23:25:05.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 1 -- 172.21.15.132:0/3409379678 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1109183504 0 0) 0x7fd1b0095590 con 0x7fd1b009be60 2024-01-24T23:25:05.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 1 -- 172.21.15.132:0/3409379678 >> v1:172.21.15.115:6791/0 conn(0x7fd1b0071f30 legacy=0x7fd1b0094770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:05.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 1 -- 172.21.15.132:0/3409379678 >> v1:172.21.15.115:6790/0 conn(0x7fd1b01419e0 legacy=0x7fd1b01d7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:05.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:25:05.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 1 monclient: found mon.i 2024-01-24T23:25:05.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:05.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:25:05.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 1 -- 172.21.15.132:0/3409379678 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b009a5d0 con 0x7fd1b009be60 2024-01-24T23:25:05.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 10 monclient: _finish_auth 0 2024-01-24T23:25:05.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:25:05.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:05.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b7a26640 5 monclient: authenticate success, global_id 5767 2024-01-24T23:25:05.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b7a26640 10 monclient: _renew_subs 2024-01-24T23:25:05.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b7a26640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:25:05.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b7a26640 1 -- 172.21.15.132:0/3409379678 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b0097290 con 0x7fd1b009be60 2024-01-24T23:25:05.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b7a26640 10 monclient: _renew_subs 2024-01-24T23:25:05.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b7a26640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:25:05.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b7a26640 1 -- 172.21.15.132:0/3409379678 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b013cc50 con 0x7fd1b009be60 2024-01-24T23:25:05.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 1 -- 172.21.15.132:0/3409379678 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd1a4004dc0 con 0x7fd1b009be60 2024-01-24T23:25:05.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:05.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a37fe640 4 set_mon_vals no callback set 2024-01-24T23:25:05.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 1 -- 172.21.15.132:0/3409379678 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd1a4005270 con 0x7fd1b009be60 2024-01-24T23:25:05.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:05.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:25:05.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 10 monclient: dump: 2024-01-24T23:25:05.519 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:05.520 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:05.520 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:05.520 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:05.520 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:05.520 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:05.520 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:05.520 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:05.520 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:05.520 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:05.520 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:05.520 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:05.520 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:05.520 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:05.520 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:05.521 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:05.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b7a26640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:05.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b7a26640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:05.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b7a26640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:25:05.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 1 -- 172.21.15.132:0/3409379678 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fd1a4005450 con 0x7fd1b009be60 2024-01-24T23:25:05.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b7a26640 1 -- 172.21.15.132:0/3409379678 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd190005090 con 0x7fd1b009be60 2024-01-24T23:25:05.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1a1ffb640 1 -- 172.21.15.132:0/3409379678 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(124..124 src has 1..124) v4 ==== 7321+0+0 (unknown 3708886129 0 0) 0x7fd1a40448d0 con 0x7fd1b009be60 2024-01-24T23:25:05.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.501+0000 7fd1b57c2640 10 monclient: get_auth_request con 0x7fd194036e60 auth_method 0 2024-01-24T23:25:05.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.505+0000 7fd1a1ffb640 1 -- 172.21.15.132:0/3409379678 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fd1a4014bf0 con 0x7fd1b009be60 2024-01-24T23:25:05.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.505+0000 7fd1a1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:05.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.505+0000 7fd1a1ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:05.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.669+0000 7fd1b7a26640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9]}] 2024-01-24T23:25:05.672 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.669+0000 7fd1b7a26640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9]}] 2024-01-24T23:25:05.672 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.669+0000 7fd1b7a26640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:25:05.672 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.669+0000 7fd1b7a26640 1 -- 172.21.15.132:0/3409379678 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9]} v 0) v1 -- 0x7fd190005340 con 0x7fd1b009be60 2024-01-24T23:25:05.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.669+0000 7fd1a1ffb640 1 -- 172.21.15.132:0/3409379678 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9]}]=-22 must specify exact 3 weight values v124) v1 ==== 163+0+0 (unknown 2049296076 0 0) 0x7fd1a4013290 con 0x7fd1b009be60 2024-01-24T23:25:05.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.669+0000 7fd1a1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9]}] 2024-01-24T23:25:05.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.669+0000 7fd1a1ffb640 10 monclient: _finish_command 2 = mon:22 must specify exact 3 weight values 2024-01-24T23:25:05.673 INFO:tasks.workunit.client.0.smithi132.stderr:Error EINVAL: must specify exact 3 weight values 2024-01-24T23:25:05.679 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:25:05.679 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set reweight barset osd.0 .9 .9 .9 2024-01-24T23:25:05.803 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.797+0000 7fb55b989640 10 monclient: get_monmap_and_config 2024-01-24T23:25:05.803 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.797+0000 7fb55b989640 10 monclient: build_initial_monmap 2024-01-24T23:25:05.803 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.797+0000 7fb55b989640 10 monclient: monmap: 2024-01-24T23:25:05.803 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:05.803 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:05.803 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:05.804639+0000 2024-01-24T23:25:05.804 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:05.804639+0000 2024-01-24T23:25:05.804 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:05.804 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:05.804 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:05.804 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:05.804 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:05.804 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:05.804 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:05.804 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:05.804 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:05.804 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:05.804 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:05.804 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:05.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 1 Processor -- start 2024-01-24T23:25:05.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 1 -- start start 2024-01-24T23:25:05.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 10 monclient: init 2024-01-24T23:25:05.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:05.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 10 monclient: _add_conns ranks=[8,5,1,6,0,7,4,3,2] 2024-01-24T23:25:05.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 10 monclient(hunting): picked mon.noname-e con 0x7fb5540a0b40 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:05.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 10 monclient(hunting): picked mon.noname-h con 0x7fb55409ebd0 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:05.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 10 monclient(hunting): picked mon.noname-f con 0x7fb55409e560 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:05.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55404f0a0 con 0x7fb55409e560 2024-01-24T23:25:05.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5540a0f20 con 0x7fb55409ebd0 2024-01-24T23:25:05.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb554072ae0 con 0x7fb5540a0b40 2024-01-24T23:25:05.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:05.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb559f26640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fb55409e560 legacy=0x7fb55413c2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:05.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 10 monclient(hunting): authenticate will time out at 1193.129467s 2024-01-24T23:25:05.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb558f24640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fb55409ebd0 0x7fb55409de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:35870/0 (socket says 172.21.15.132:35870) 2024-01-24T23:25:05.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb558f24640 1 -- 172.21.15.132:0/1876666770 learned_addr learned my addr 172.21.15.132:0/1876666770 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:05.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 1 -- 172.21.15.132:0/1876666770 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3369155398 0 0) 0x7fb5540a0f20 con 0x7fb55409ebd0 2024-01-24T23:25:05.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:05.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:05.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:05.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 10 monclient(hunting): my global_id is 5774 2024-01-24T23:25:05.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 1 -- 172.21.15.132:0/1876666770 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb540005740 con 0x7fb55409ebd0 2024-01-24T23:25:05.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 1 -- 172.21.15.132:0/1876666770 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2598200756 0 0) 0x7fb540005740 con 0x7fb55409ebd0 2024-01-24T23:25:05.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 1 -- 172.21.15.132:0/1876666770 >> v1:172.21.15.115:6793/0 conn(0x7fb5540a0b40 legacy=0x7fb55409d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:05.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 1 -- 172.21.15.132:0/1876666770 >> v1:172.21.15.132:6789/0 conn(0x7fb55409e560 legacy=0x7fb55413c2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:05.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:25:05.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 1 monclient: found mon.noname-h 2024-01-24T23:25:05.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:05.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:25:05.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 1 -- 172.21.15.132:0/1876666770 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5540999b0 con 0x7fb55409ebd0 2024-01-24T23:25:05.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 10 monclient: _finish_auth 0 2024-01-24T23:25:05.812 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:25:05.813 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:05.813 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 1 -- 172.21.15.132:0/1876666770 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb5440017c0 con 0x7fb55409ebd0 2024-01-24T23:25:05.813 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:05.813 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:25:05.813 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 10 monclient: dump: 2024-01-24T23:25:05.813 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:05.813 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:05.813 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:05.813 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:05.813 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:05.813 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:05.813 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:05.813 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:05.814 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:05.814 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:05.814 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:05.814 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:05.814 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:05.814 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:05.814 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:05.814 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:05.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 5 monclient: authenticate success, global_id 5774 2024-01-24T23:25:05.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:05.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 1 -- 172.21.15.132:0/1876666770 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb544001a70 con 0x7fb55409ebd0 2024-01-24T23:25:05.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:05.814 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 1 -- 172.21.15.132:0/1876666770 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb544004bf0 con 0x7fb55409ebd0 2024-01-24T23:25:05.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:05.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 4 set_mon_vals no callback set 2024-01-24T23:25:05.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:05.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:05.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb54bfff640 10 monclient: dump: 2024-01-24T23:25:05.815 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:05.815 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:05.815 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:05.815 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:05.815 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:05.815 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:05.815 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:05.815 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:05.815 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:05.815 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:05.816 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:05.816 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:05.816 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:05.816 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:05.816 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:05.816 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:05.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 10 monclient: shutdown 2024-01-24T23:25:05.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:05.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 1 -- 172.21.15.132:0/1876666770 >> v1:172.21.15.132:6791/0 conn(0x7fb55409ebd0 legacy=0x7fb55409de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:05.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 1 -- 172.21.15.132:0/1876666770 shutdown_connections 2024-01-24T23:25:05.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 1 -- 172.21.15.132:0/1876666770 >> 172.21.15.132:0/1876666770 conn(0x7fb5540a1c30 msgr2=0x7fb5540a2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:05.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 1 -- 172.21.15.132:0/1876666770 shutdown_connections 2024-01-24T23:25:05.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 1 -- 172.21.15.132:0/1876666770 wait complete. 2024-01-24T23:25:05.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 10 monclient: build_initial_monmap 2024-01-24T23:25:05.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.801+0000 7fb55b989640 10 monclient: monmap: 2024-01-24T23:25:05.817 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:05.817 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:05.817 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:05.817 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:05.817 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:05.817 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:05.817 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:05.817 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:05.817 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:05.817 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:05.817 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:05.817 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:05.817 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:05.818 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:05.818 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:05.818 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:05.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb55b989640 1 Processor -- start 2024-01-24T23:25:05.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb55b989640 1 -- start start 2024-01-24T23:25:05.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb55b989640 10 monclient: init 2024-01-24T23:25:05.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb55b989640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:05.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb55b989640 10 monclient: _add_conns ranks=[4,5,0,6,2,1,3,8,7] 2024-01-24T23:25:05.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb55b989640 10 monclient(hunting): picked mon.noname-c con 0x7fb55409e560 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:05.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb55b989640 10 monclient(hunting): picked mon.noname-h con 0x7fb55409ebd0 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:05.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb55b989640 10 monclient(hunting): picked mon.noname-a con 0x7fb5540a0b40 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:05.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb55b989640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5541b05f0 con 0x7fb5540a0b40 2024-01-24T23:25:05.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb55b989640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55409b750 con 0x7fb55409e560 2024-01-24T23:25:05.818 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb55b989640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5541afd20 con 0x7fb55409ebd0 2024-01-24T23:25:05.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb55b989640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:05.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb55b989640 10 monclient(hunting): authenticate will time out at 1193.132908s 2024-01-24T23:25:05.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb559725640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7fb55409e560 legacy=0x7fb5541aef00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:25:05.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb559f26640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fb5540a0b40 legacy=0x7fb5541cf7b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:25:05.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb558f24640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fb55409ebd0 0x7fb5541af610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:35886/0 (socket says 172.21.15.132:35886) 2024-01-24T23:25:05.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb558f24640 1 -- 172.21.15.132:0/234773790 learned_addr learned my addr 172.21.15.132:0/234773790 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:05.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 1 -- 172.21.15.132:0/234773790 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1010569228 0 0) 0x7fb5541afd20 con 0x7fb55409ebd0 2024-01-24T23:25:05.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:05.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:05.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:05.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 10 monclient(hunting): my global_id is 5783 2024-01-24T23:25:05.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 1 -- 172.21.15.132:0/234773790 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb538003530 con 0x7fb55409ebd0 2024-01-24T23:25:05.819 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 1 -- 172.21.15.132:0/234773790 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3696981325 0 0) 0x7fb538003530 con 0x7fb55409ebd0 2024-01-24T23:25:05.820 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 1 -- 172.21.15.132:0/234773790 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb5541afd20 con 0x7fb55409ebd0 2024-01-24T23:25:05.820 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 1 -- 172.21.15.132:0/234773790 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb544001780 con 0x7fb55409ebd0 2024-01-24T23:25:05.820 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:05.820 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:25:05.820 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 10 monclient(hunting): dump: 2024-01-24T23:25:05.820 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:05.820 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:05.820 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:05.820 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:05.820 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:05.821 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:05.821 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:05.821 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:05.821 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:05.821 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:05.821 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:05.821 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:05.821 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:05.821 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:05.821 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:05.821 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:05.821 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:05.821 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 1 -- 172.21.15.132:0/234773790 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3793205284 0 0) 0x7fb5541afd20 con 0x7fb55409ebd0 2024-01-24T23:25:05.821 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 1 -- 172.21.15.132:0/234773790 >> v1:172.21.15.115:6791/0 conn(0x7fb55409e560 legacy=0x7fb5541aef00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:05.822 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 1 -- 172.21.15.132:0/234773790 >> v1:172.21.15.115:6789/0 conn(0x7fb5540a0b40 legacy=0x7fb5541cf7b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:05.822 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:25:05.822 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 1 monclient: found mon.h 2024-01-24T23:25:05.822 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:05.822 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:05.822 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 1 -- 172.21.15.132:0/234773790 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55409b000 con 0x7fb55409ebd0 2024-01-24T23:25:05.822 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 10 monclient: _finish_auth 0 2024-01-24T23:25:05.822 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:25:05.822 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:05.822 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb55b989640 5 monclient: authenticate success, global_id 5783 2024-01-24T23:25:05.822 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb55b989640 10 monclient: _renew_subs 2024-01-24T23:25:05.822 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb55b989640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:05.822 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb55b989640 1 -- 172.21.15.132:0/234773790 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5541b2160 con 0x7fb55409ebd0 2024-01-24T23:25:05.822 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb55b989640 10 monclient: _renew_subs 2024-01-24T23:25:05.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb55b989640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:05.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb55b989640 1 -- 172.21.15.132:0/234773790 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb554071670 con 0x7fb55409ebd0 2024-01-24T23:25:05.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 1 -- 172.21.15.132:0/234773790 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb544004ec0 con 0x7fb55409ebd0 2024-01-24T23:25:05.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:05.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 1 -- 172.21.15.132:0/234773790 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb544005370 con 0x7fb55409ebd0 2024-01-24T23:25:05.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:05.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb54b7fe640 4 set_mon_vals no callback set 2024-01-24T23:25:05.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:05.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 10 monclient: dump: 2024-01-24T23:25:05.823 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:05.823 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:05.823 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:05.823 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:05.823 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:05.824 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:05.824 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:05.824 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:05.824 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:05.824 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:05.824 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:05.824 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:05.824 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:05.824 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:05.824 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:05.824 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:05.824 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb55b989640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:05.824 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb55b989640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:05.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb55b989640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:05.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 1 -- 172.21.15.132:0/234773790 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fb54400fdf0 con 0x7fb55409ebd0 2024-01-24T23:25:05.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb55b989640 1 -- 172.21.15.132:0/234773790 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb534005090 con 0x7fb55409ebd0 2024-01-24T23:25:05.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb549ffb640 1 -- 172.21.15.132:0/234773790 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(124..124 src has 1..124) v4 ==== 7321+0+0 (unknown 3708886129 0 0) 0x7fb544044850 con 0x7fb55409ebd0 2024-01-24T23:25:05.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.805+0000 7fb559725640 10 monclient: get_auth_request con 0x7fb538036f40 auth_method 0 2024-01-24T23:25:05.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.809+0000 7fb549ffb640 1 -- 172.21.15.132:0/234773790 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fb544014bb0 con 0x7fb55409ebd0 2024-01-24T23:25:05.825 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.809+0000 7fb549ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:05.826 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.809+0000 7fb549ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:05.977 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.973+0000 7fb55b989640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}] 2024-01-24T23:25:05.977 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.973+0000 7fb55b989640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}] 2024-01-24T23:25:05.977 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.973+0000 7fb55b989640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:05.977 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:05.973+0000 7fb55b989640 1 -- 172.21.15.132:0/234773790 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]} v 0) v1 -- 0x7fb534005340 con 0x7fb55409ebd0 2024-01-24T23:25:06.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:06.081+0000 7fb549ffb640 1 -- 172.21.15.132:0/234773790 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}]=0 v125) v1 ==== 139+0+0 (unknown 928633884 0 0) 0x7fb544013250 con 0x7fb55409ebd0 2024-01-24T23:25:06.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:06.081+0000 7fb549ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}] 2024-01-24T23:25:06.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:06.081+0000 7fb549ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:06.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:06.189+0000 7fb55b989640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}] 2024-01-24T23:25:06.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:06.189+0000 7fb55b989640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}] 2024-01-24T23:25:06.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:06.189+0000 7fb55b989640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:06.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:06.189+0000 7fb55b989640 1 -- 172.21.15.132:0/234773790 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]} v 0) v1 -- 0x7fb5340010d0 con 0x7fb55409ebd0 2024-01-24T23:25:07.082 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.077+0000 7fb549ffb640 1 -- 172.21.15.132:0/234773790 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}]=0 v126) v1 ==== 139+0+0 (unknown 504849017 0 0) 0x7fb5440149d0 con 0x7fb55409ebd0 2024-01-24T23:25:07.082 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.077+0000 7fb549ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}] 2024-01-24T23:25:07.082 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.077+0000 7fb549ffb640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:25:07.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.085+0000 7fb55b989640 1 -- 172.21.15.132:0/234773790 >> v1:172.21.15.132:6812/1835401922 conn(0x7fb538036f40 legacy=0x7fb538039400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:07.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.085+0000 7fb55b989640 10 monclient: shutdown 2024-01-24T23:25:07.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.085+0000 7fb55b989640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:07.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.085+0000 7fb55b989640 1 -- 172.21.15.132:0/234773790 >> v1:172.21.15.132:6791/0 conn(0x7fb55409ebd0 legacy=0x7fb5541af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:07.090 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.085+0000 7fb55b989640 1 -- 172.21.15.132:0/234773790 shutdown_connections 2024-01-24T23:25:07.090 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.085+0000 7fb55b989640 1 -- 172.21.15.132:0/234773790 >> 172.21.15.132:0/234773790 conn(0x7fb5540a1c30 msgr2=0x7fb5540a3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:07.090 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.085+0000 7fb55b989640 1 -- 172.21.15.132:0/234773790 shutdown_connections 2024-01-24T23:25:07.090 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.085+0000 7fb55b989640 1 -- 172.21.15.132:0/234773790 wait complete. 2024-01-24T23:25:07.111 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set ls 2024-01-24T23:25:07.111 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep -c fooset 2024-01-24T23:25:07.112 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep -q 1 2024-01-24T23:25:07.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.225+0000 7fc8b47f4640 10 monclient: get_monmap_and_config 2024-01-24T23:25:07.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.225+0000 7fc8b47f4640 10 monclient: build_initial_monmap 2024-01-24T23:25:07.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.225+0000 7fc8b47f4640 10 monclient: monmap: 2024-01-24T23:25:07.230 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:07.230 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:07.230 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:07.231203+0000 2024-01-24T23:25:07.230 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:07.231203+0000 2024-01-24T23:25:07.230 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:07.230 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:07.230 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:07.230 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:07.230 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:07.231 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:07.231 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:07.231 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:07.231 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:07.231 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:07.231 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:07.231 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:07.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.225+0000 7fc8b47f4640 1 Processor -- start 2024-01-24T23:25:07.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.225+0000 7fc8b47f4640 1 -- start start 2024-01-24T23:25:07.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.225+0000 7fc8b47f4640 10 monclient: init 2024-01-24T23:25:07.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.225+0000 7fc8b47f4640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:07.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.225+0000 7fc8b47f4640 10 monclient: _add_conns ranks=[1,6,5,2,8,7,3,4,0] 2024-01-24T23:25:07.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.225+0000 7fc8b47f4640 10 monclient(hunting): picked mon.noname-f con 0x7fc8ac135e10 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:07.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.225+0000 7fc8b47f4640 10 monclient(hunting): picked mon.noname-d con 0x7fc8ac1361f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:07.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.225+0000 7fc8b47f4640 10 monclient(hunting): picked mon.noname-h con 0x7fc8ac1365f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:07.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.225+0000 7fc8b47f4640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8ac064700 con 0x7fc8ac135e10 2024-01-24T23:25:07.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.225+0000 7fc8b2590640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fc8ac135e10 legacy=0x7fc8ac09bcd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:07.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.225+0000 7fc8b47f4640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8ac09c3e0 con 0x7fc8ac1365f0 2024-01-24T23:25:07.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.225+0000 7fc8b47f4640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8ac0a4f90 con 0x7fc8ac1361f0 2024-01-24T23:25:07.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.225+0000 7fc8b47f4640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:07.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 10 monclient(hunting): authenticate will time out at 1194.556013s 2024-01-24T23:25:07.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b2d91640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fc8ac1365f0 0x7fc8ac13f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:35900/0 (socket says 172.21.15.132:35900) 2024-01-24T23:25:07.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b2d91640 1 -- 172.21.15.132:0/3575012139 learned_addr learned my addr 172.21.15.132:0/3575012139 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:07.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 1 -- 172.21.15.132:0/3575012139 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361429803 0 0) 0x7fc8ac09c3e0 con 0x7fc8ac1365f0 2024-01-24T23:25:07.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 10 monclient(hunting): _negotiate 2024-01-24T23:25:07.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:07.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:07.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 10 monclient(hunting): my global_id is 5792 2024-01-24T23:25:07.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 1 -- 172.21.15.132:0/3575012139 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc898003530 con 0x7fc8ac1365f0 2024-01-24T23:25:07.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 1 -- 172.21.15.132:0/3575012139 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1198024039 0 0) 0x7fc898003530 con 0x7fc8ac1365f0 2024-01-24T23:25:07.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 1 -- 172.21.15.132:0/3575012139 >> v1:172.21.15.115:6792/0 conn(0x7fc8ac1361f0 legacy=0x7fc8ac13bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:07.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 1 -- 172.21.15.132:0/3575012139 >> v1:172.21.15.132:6789/0 conn(0x7fc8ac135e10 legacy=0x7fc8ac09bcd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:07.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 10 monclient: _finish_hunting 0 2024-01-24T23:25:07.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 1 monclient: found mon.noname-h 2024-01-24T23:25:07.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:07.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:25:07.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 1 -- 172.21.15.132:0/3575012139 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ac09b010 con 0x7fc8ac1365f0 2024-01-24T23:25:07.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 10 monclient: _finish_auth 0 2024-01-24T23:25:07.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 10 monclient: _check_auth_tickets 2024-01-24T23:25:07.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:07.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 1 -- 172.21.15.132:0/3575012139 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc8a4001b40 con 0x7fc8ac1365f0 2024-01-24T23:25:07.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:07.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:25:07.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 10 monclient: dump: 2024-01-24T23:25:07.242 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:07.242 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:07.242 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:07.242 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:07.242 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:07.242 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:07.242 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:07.242 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:07.243 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:07.243 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:07.243 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:07.243 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:07.243 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:07.243 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:07.243 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:07.243 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:07.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 5 monclient: authenticate success, global_id 5792 2024-01-24T23:25:07.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:07.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 1 -- 172.21.15.132:0/3575012139 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc8a4001e60 con 0x7fc8ac1365f0 2024-01-24T23:25:07.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:07.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 1 -- 172.21.15.132:0/3575012139 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc8a4004f70 con 0x7fc8ac1365f0 2024-01-24T23:25:07.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:07.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 4 set_mon_vals no callback set 2024-01-24T23:25:07.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:07.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:07.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b158e640 10 monclient: dump: 2024-01-24T23:25:07.244 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:07.244 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:07.244 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:07.244 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:07.244 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:07.244 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:07.244 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:07.244 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:07.244 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:07.244 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:07.244 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:07.244 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:07.245 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:07.245 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:07.245 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:07.245 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:07.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 10 monclient: shutdown 2024-01-24T23:25:07.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:07.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 1 -- 172.21.15.132:0/3575012139 >> v1:172.21.15.132:6791/0 conn(0x7fc8ac1365f0 legacy=0x7fc8ac13f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:07.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 1 -- 172.21.15.132:0/3575012139 shutdown_connections 2024-01-24T23:25:07.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 1 -- 172.21.15.132:0/3575012139 >> 172.21.15.132:0/3575012139 conn(0x7fc8ac0a6790 msgr2=0x7fc8ac071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:07.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 1 -- 172.21.15.132:0/3575012139 shutdown_connections 2024-01-24T23:25:07.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 1 -- 172.21.15.132:0/3575012139 wait complete. 2024-01-24T23:25:07.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 10 monclient: build_initial_monmap 2024-01-24T23:25:07.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 10 monclient: monmap: 2024-01-24T23:25:07.245 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:07.245 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:07.246 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:07.246 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:07.246 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:07.246 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:07.246 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:07.246 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:07.246 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:07.246 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:07.246 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:07.246 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:07.246 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:07.246 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:07.246 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:07.246 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:07.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 1 Processor -- start 2024-01-24T23:25:07.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 1 -- start start 2024-01-24T23:25:07.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 10 monclient: init 2024-01-24T23:25:07.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:07.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 10 monclient: _add_conns ranks=[3,7,4,2,8,0,5,6,1] 2024-01-24T23:25:07.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 10 monclient(hunting): picked mon.noname-g con 0x7fc8ac135e10 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:07.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 10 monclient(hunting): picked mon.noname-i con 0x7fc8ac1361f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:07.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 10 monclient(hunting): picked mon.noname-c con 0x7fc8ac1365f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:07.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8ac122870 con 0x7fc8ac135e10 2024-01-24T23:25:07.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8ac09ca00 con 0x7fc8ac1365f0 2024-01-24T23:25:07.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8ac09c6b0 con 0x7fc8ac1361f0 2024-01-24T23:25:07.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:07.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b47f4640 10 monclient(hunting): authenticate will time out at 1194.559401s 2024-01-24T23:25:07.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b2d91640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7fc8ac1365f0 legacy=0x7fc8ac1d39f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:25:07.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b1d8f640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fc8ac1361f0 0x7fc8ac1d02c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:56972/0 (socket says 172.21.15.132:56972) 2024-01-24T23:25:07.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8b1d8f640 1 -- 172.21.15.132:0/3975019416 learned_addr learned my addr 172.21.15.132:0/3975019416 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:07.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8a2ffd640 1 -- 172.21.15.132:0/3975019416 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1604427967 0 0) 0x7fc8ac09c6b0 con 0x7fc8ac1361f0 2024-01-24T23:25:07.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8a2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:07.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8a2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:07.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:07.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8a2ffd640 10 monclient(hunting): my global_id is 5776 2024-01-24T23:25:07.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8a2ffd640 1 -- 172.21.15.132:0/3975019416 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc890003530 con 0x7fc8ac1361f0 2024-01-24T23:25:07.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8a2ffd640 1 -- 172.21.15.132:0/3975019416 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3652515877 0 0) 0x7fc8ac122870 con 0x7fc8ac135e10 2024-01-24T23:25:07.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8a2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:07.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8a2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:07.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:07.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8a2ffd640 10 monclient(hunting): my global_id is 5826 2024-01-24T23:25:07.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.229+0000 7fc8a2ffd640 1 -- 172.21.15.132:0/3975019416 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8ac09c6b0 con 0x7fc8ac135e10 2024-01-24T23:25:07.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 1 -- 172.21.15.132:0/3975019416 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 317914638 0 0) 0x7fc890003530 con 0x7fc8ac1361f0 2024-01-24T23:25:07.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 1 -- 172.21.15.132:0/3975019416 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc8ac122870 con 0x7fc8ac1361f0 2024-01-24T23:25:07.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 1 -- 172.21.15.132:0/3975019416 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc8a8001b20 con 0x7fc8ac1361f0 2024-01-24T23:25:07.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:07.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:25:07.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 10 monclient(hunting): dump: 2024-01-24T23:25:07.249 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:07.249 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:07.249 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:07.249 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:07.249 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:07.249 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:07.249 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:07.249 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:07.249 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:07.249 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:07.250 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:07.250 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:07.250 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:07.250 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:07.250 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:07.250 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:07.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:07.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 1 -- 172.21.15.132:0/3975019416 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2633804922 0 0) 0x7fc8ac09c6b0 con 0x7fc8ac135e10 2024-01-24T23:25:07.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 1 -- 172.21.15.132:0/3975019416 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc890003530 con 0x7fc8ac135e10 2024-01-24T23:25:07.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 1 -- 172.21.15.132:0/3975019416 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc89c004270 con 0x7fc8ac135e10 2024-01-24T23:25:07.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:07.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:07.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 10 monclient(hunting): dump: 2024-01-24T23:25:07.250 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:07.250 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:07.251 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:07.251 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:07.251 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:07.251 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:07.251 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:07.251 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:07.251 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:07.251 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:07.251 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:07.251 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:07.251 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:07.251 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:07.251 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:07.251 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:07.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 1 -- 172.21.15.132:0/3975019416 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1381343132 0 0) 0x7fc8ac122870 con 0x7fc8ac1361f0 2024-01-24T23:25:07.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 1 -- 172.21.15.132:0/3975019416 >> v1:172.21.15.115:6791/0 conn(0x7fc8ac1365f0 legacy=0x7fc8ac1d39f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:07.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 1 -- 172.21.15.132:0/3975019416 >> v1:172.21.15.132:6790/0 conn(0x7fc8ac135e10 legacy=0x7fc8ac134eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:07.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:25:07.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 1 monclient: found mon.i 2024-01-24T23:25:07.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:07.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:25:07.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 1 -- 172.21.15.132:0/3975019416 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ac1bf450 con 0x7fc8ac1361f0 2024-01-24T23:25:07.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 10 monclient: _finish_auth 0 2024-01-24T23:25:07.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:07.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:07.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8b47f4640 5 monclient: authenticate success, global_id 5776 2024-01-24T23:25:07.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8b47f4640 10 monclient: _renew_subs 2024-01-24T23:25:07.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8b47f4640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:25:07.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8b47f4640 1 -- 172.21.15.132:0/3975019416 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ac122590 con 0x7fc8ac1361f0 2024-01-24T23:25:07.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8b47f4640 10 monclient: _renew_subs 2024-01-24T23:25:07.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8b47f4640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:25:07.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8b47f4640 1 -- 172.21.15.132:0/3975019416 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ac1beb20 con 0x7fc8ac1361f0 2024-01-24T23:25:07.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 1 -- 172.21.15.132:0/3975019416 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc8a80038d0 con 0x7fc8ac1361f0 2024-01-24T23:25:07.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:07.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 1 -- 172.21.15.132:0/3975019416 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc8a8003e00 con 0x7fc8ac1361f0 2024-01-24T23:25:07.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:07.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8b0d8d640 4 set_mon_vals no callback set 2024-01-24T23:25:07.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:25:07.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 10 monclient: dump: 2024-01-24T23:25:07.253 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:07.253 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:07.253 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:07.253 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:07.253 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:07.253 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:07.254 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:07.254 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:07.254 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:07.254 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:07.254 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:07.254 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:07.254 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:07.254 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:07.254 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:07.254 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:07.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8b47f4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:07.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8b47f4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:07.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8b47f4640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:25:07.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 1 -- 172.21.15.132:0/3975019416 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fc8a8003f60 con 0x7fc8ac1361f0 2024-01-24T23:25:07.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8b47f4640 1 -- 172.21.15.132:0/3975019416 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc894005090 con 0x7fc8ac1361f0 2024-01-24T23:25:07.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8a2ffd640 1 -- 172.21.15.132:0/3975019416 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(126..126 src has 1..126) v4 ==== 7321+0+0 (unknown 2448100607 0 0) 0x7fc8a8044680 con 0x7fc8ac1361f0 2024-01-24T23:25:07.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.233+0000 7fc8b2590640 10 monclient: get_auth_request con 0x7fc890037650 auth_method 0 2024-01-24T23:25:07.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.237+0000 7fc8a2ffd640 1 -- 172.21.15.132:0/3975019416 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fc8a80148a0 con 0x7fc8ac1361f0 2024-01-24T23:25:07.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.237+0000 7fc8a2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:07.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.237+0000 7fc8a2ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:07.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.401+0000 7fc8b47f4640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:07.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.401+0000 7fc8b47f4640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:07.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.401+0000 7fc8b47f4640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:25:07.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.401+0000 7fc8b47f4640 1 -- 172.21.15.132:0/3975019416 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7fc8940058c0 con 0x7fc8ac1361f0 2024-01-24T23:25:07.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.401+0000 7fc8a2ffd640 1 -- 172.21.15.132:0/3975019416 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v126) v1 ==== 71+0+14 (unknown 3894361239 0 3003798303) 0x7fc8a8012f40 con 0x7fc8ac1361f0 2024-01-24T23:25:07.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.401+0000 7fc8a2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:07.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.401+0000 7fc8a2ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:07.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.501+0000 7fc8b47f4640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:07.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.501+0000 7fc8b47f4640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:07.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.501+0000 7fc8b47f4640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:25:07.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.501+0000 7fc8b47f4640 1 -- 172.21.15.132:0/3975019416 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7fc894000bd0 con 0x7fc8ac1361f0 2024-01-24T23:25:07.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.501+0000 7fc8a2ffd640 1 -- 172.21.15.132:0/3975019416 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v126) v1 ==== 71+0+14 (unknown 3894361239 0 3003798303) 0x7fc8a80146c0 con 0x7fc8ac1361f0 2024-01-24T23:25:07.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.501+0000 7fc8a2ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:07.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.501+0000 7fc8a2ffd640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:25:07.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.501+0000 7fc8b47f4640 1 -- 172.21.15.132:0/3975019416 >> v1:172.21.15.132:6812/1835401922 conn(0x7fc890037650 legacy=0x7fc890039b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:07.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.501+0000 7fc8b47f4640 10 monclient: shutdown 2024-01-24T23:25:07.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.501+0000 7fc8b47f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:07.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.501+0000 7fc8b47f4640 1 -- 172.21.15.132:0/3975019416 >> v1:172.21.15.132:6792/0 conn(0x7fc8ac1361f0 legacy=0x7fc8ac1d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:07.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.501+0000 7fc8b47f4640 1 -- 172.21.15.132:0/3975019416 shutdown_connections 2024-01-24T23:25:07.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.501+0000 7fc8b47f4640 1 -- 172.21.15.132:0/3975019416 >> 172.21.15.132:0/3975019416 conn(0x7fc8ac0a6790 msgr2=0x7fc8ac0a7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:07.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.501+0000 7fc8b47f4640 1 -- 172.21.15.132:0/3975019416 shutdown_connections 2024-01-24T23:25:07.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.501+0000 7fc8b47f4640 1 -- 172.21.15.132:0/3975019416 wait complete. 2024-01-24T23:25:07.521 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set rm fooset 2024-01-24T23:25:07.639 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 10 monclient: get_monmap_and_config 2024-01-24T23:25:07.639 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 10 monclient: build_initial_monmap 2024-01-24T23:25:07.639 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 10 monclient: monmap: 2024-01-24T23:25:07.640 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:07.640 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:07.640 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:07.640912+0000 2024-01-24T23:25:07.640 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:07.640912+0000 2024-01-24T23:25:07.640 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:07.640 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:07.640 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:07.640 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:07.640 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:07.640 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:07.640 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:07.640 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:07.640 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:07.640 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:07.641 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:07.641 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:07.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 1 Processor -- start 2024-01-24T23:25:07.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 1 -- start start 2024-01-24T23:25:07.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 10 monclient: init 2024-01-24T23:25:07.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:07.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 10 monclient: _add_conns ranks=[7,1,2,4,6,0,8,5,3] 2024-01-24T23:25:07.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 10 monclient(hunting): picked mon.noname-i con 0x7f96401370e0 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:07.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 10 monclient(hunting): picked mon.noname-f con 0x7f964009bc20 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:07.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 10 monclient(hunting): picked mon.noname-b con 0x7f964013d640 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:07.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9640064700 con 0x7f964009bc20 2024-01-24T23:25:07.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f964006a530 con 0x7f964013d640 2024-01-24T23:25:07.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f964009af60 con 0x7f96401370e0 2024-01-24T23:25:07.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:07.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 10 monclient(hunting): authenticate will time out at 1194.965566s 2024-01-24T23:25:07.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963e7fc640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f964009bc20 legacy=0x7f964013bf20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:07.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963effd640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f96401370e0 0x7f96401374c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:56980/0 (socket says 172.21.15.132:56980) 2024-01-24T23:25:07.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963effd640 1 -- 172.21.15.132:0/2704671047 learned_addr learned my addr 172.21.15.132:0/2704671047 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:07.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 1 -- 172.21.15.132:0/2704671047 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2895890449 0 0) 0x7f964009af60 con 0x7f96401370e0 2024-01-24T23:25:07.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:07.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:07.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:07.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 10 monclient(hunting): my global_id is 5785 2024-01-24T23:25:07.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 1 -- 172.21.15.132:0/2704671047 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9624003530 con 0x7f96401370e0 2024-01-24T23:25:07.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 1 -- 172.21.15.132:0/2704671047 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 942890877 0 0) 0x7f9624003530 con 0x7f96401370e0 2024-01-24T23:25:07.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 1 -- 172.21.15.132:0/2704671047 >> v1:172.21.15.115:6790/0 conn(0x7f964013d640 legacy=0x7f964013fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:07.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 1 -- 172.21.15.132:0/2704671047 >> v1:172.21.15.132:6789/0 conn(0x7f964009bc20 legacy=0x7f964013bf20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:07.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 10 monclient: _finish_hunting 0 2024-01-24T23:25:07.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 1 monclient: found mon.noname-i 2024-01-24T23:25:07.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:07.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:25:07.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 1 -- 172.21.15.132:0/2704671047 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f964009b240 con 0x7f96401370e0 2024-01-24T23:25:07.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 10 monclient: _finish_auth 0 2024-01-24T23:25:07.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 10 monclient: _check_auth_tickets 2024-01-24T23:25:07.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:07.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 1 -- 172.21.15.132:0/2704671047 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f96300017c0 con 0x7f96401370e0 2024-01-24T23:25:07.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:07.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:25:07.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 10 monclient: dump: 2024-01-24T23:25:07.648 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:07.648 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:07.648 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:07.648 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:07.648 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:07.648 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:07.648 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:07.648 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:07.648 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:07.649 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:07.649 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:07.649 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:07.649 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:07.649 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:07.649 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:07.649 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:07.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 5 monclient: authenticate success, global_id 5785 2024-01-24T23:25:07.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:07.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 1 -- 172.21.15.132:0/2704671047 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9630001ae0 con 0x7f96401370e0 2024-01-24T23:25:07.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:07.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 1 -- 172.21.15.132:0/2704671047 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9630004bf0 con 0x7f96401370e0 2024-01-24T23:25:07.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:07.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 4 set_mon_vals no callback set 2024-01-24T23:25:07.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:07.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:25:07.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f963dffb640 10 monclient: dump: 2024-01-24T23:25:07.650 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:07.650 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:07.650 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:07.650 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:07.650 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:07.650 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:07.650 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:07.650 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:07.650 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:07.650 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:07.650 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:07.650 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:07.650 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:07.651 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:07.651 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:07.651 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:07.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 10 monclient: shutdown 2024-01-24T23:25:07.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:07.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 1 -- 172.21.15.132:0/2704671047 >> v1:172.21.15.132:6792/0 conn(0x7f96401370e0 legacy=0x7f96401374c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:07.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 1 -- 172.21.15.132:0/2704671047 shutdown_connections 2024-01-24T23:25:07.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 1 -- 172.21.15.132:0/2704671047 >> 172.21.15.132:0/2704671047 conn(0x7f96400a2b40 msgr2=0x7f96400a2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:07.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 1 -- 172.21.15.132:0/2704671047 shutdown_connections 2024-01-24T23:25:07.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 1 -- 172.21.15.132:0/2704671047 wait complete. 2024-01-24T23:25:07.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 10 monclient: build_initial_monmap 2024-01-24T23:25:07.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 10 monclient: monmap: 2024-01-24T23:25:07.651 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:07.651 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:07.651 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:07.652 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:07.652 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:07.652 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:07.652 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:07.652 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:07.652 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:07.652 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:07.652 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:07.652 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:07.652 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:07.652 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:07.652 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:07.652 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:07.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.637+0000 7f9645781640 1 Processor -- start 2024-01-24T23:25:07.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.641+0000 7f9645781640 1 -- start start 2024-01-24T23:25:07.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.641+0000 7f9645781640 10 monclient: init 2024-01-24T23:25:07.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.641+0000 7f9645781640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:07.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.641+0000 7f9645781640 10 monclient: _add_conns ranks=[0,4,1,8,7,2,3,6,5] 2024-01-24T23:25:07.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.641+0000 7f9645781640 10 monclient(hunting): picked mon.noname-a con 0x7f964009bc20 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:07.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.641+0000 7f9645781640 10 monclient(hunting): picked mon.noname-c con 0x7f96401370e0 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:07.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.641+0000 7f9645781640 10 monclient(hunting): picked mon.noname-f con 0x7f964013d640 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:07.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.641+0000 7f9645781640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f964009c890 con 0x7f964009bc20 2024-01-24T23:25:07.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.641+0000 7f9645781640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f964006b490 con 0x7f964013d640 2024-01-24T23:25:07.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.641+0000 7f9645781640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f964006edd0 con 0x7f96401370e0 2024-01-24T23:25:07.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.641+0000 7f9645781640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:07.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.641+0000 7f9645781640 10 monclient(hunting): authenticate will time out at 1194.968499s 2024-01-24T23:25:07.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.641+0000 7f963effd640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f964009bc20 legacy=0x7f96401cb7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:25:07.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.641+0000 7f963f7fe640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f964013d640 legacy=0x7f96401d3a20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:07.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.641+0000 7f963e7fc640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f96401370e0 legacy=0x7f96401cbed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:25:07.844 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.845+0000 7f963f7fe640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f964013d640 legacy=0x7f96401d3a20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:07.844 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.845+0000 7f963effd640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f964009bc20 legacy=0x7f96401cb7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:25:07.844 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:07.845+0000 7f963e7fc640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f96401370e0 legacy=0x7f96401cbed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:25:08.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:08.249+0000 7f963f7fe640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f964013d640 legacy=0x7f96401d3a20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:08.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:08.249+0000 7f963e7fc640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f96401370e0 legacy=0x7f96401cbed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:25:08.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:08.249+0000 7f963effd640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f964009bc20 legacy=0x7f96401cb7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:25:09.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:09.053+0000 7f963f7fe640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f964013d640 legacy=0x7f96401d3a20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:09.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:09.053+0000 7f963e7fc640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f96401370e0 legacy=0x7f96401cbed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:25:09.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:09.053+0000 7f963effd640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f964009bc20 legacy=0x7f96401cb7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:25:09.771 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-01-24T23:25:09.771 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-01-24T23:25:09.772 INFO:tasks.ceph.mon.a:Restarting daemon 2024-01-24T23:25:09.772 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-01-24T23:25:09.775 INFO:tasks.ceph.mon.a:Started 2024-01-24T23:25:09.775 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-01-24T23:25:09.775 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.f 2024-01-24T23:25:09.775 INFO:tasks.ceph.mon.f:Restarting daemon 2024-01-24T23:25:09.775 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-01-24T23:25:09.778 INFO:tasks.ceph.mon.f:Started 2024-01-24T23:25:09.778 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-01-24T23:25:09.778 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-01-24T23:25:09.778 INFO:tasks.ceph.mon.c:Restarting daemon 2024-01-24T23:25:09.778 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-01-24T23:25:09.781 INFO:tasks.ceph.mon.c:Started 2024-01-24T23:25:09.781 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-01-24T23:25:09.781 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-01-24T23:25:09.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:09.914+0000 7f53bbeea640 10 monclient: get_monmap_and_config 2024-01-24T23:25:09.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:09.914+0000 7f53bbeea640 10 monclient: build_initial_monmap 2024-01-24T23:25:09.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:09.914+0000 7f53bbeea640 10 monclient: monmap: 2024-01-24T23:25:09.917 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:09.917 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:09.917 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:09.918071+0000 2024-01-24T23:25:09.917 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:09.918071+0000 2024-01-24T23:25:09.917 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:09.918 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:09.918 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:09.918 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:09.918 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:09.918 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:09.918 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:09.918 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:09.918 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:09.918 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:09.918 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:09.918 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:09.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:09.914+0000 7f53bbeea640 1 Processor -- start 2024-01-24T23:25:09.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:09.914+0000 7f53bbeea640 1 -- start start 2024-01-24T23:25:09.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:09.914+0000 7f53bbeea640 10 monclient: init 2024-01-24T23:25:09.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:09.918+0000 7f53bbeea640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:09.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:09.918+0000 7f53bbeea640 10 monclient: _add_conns ranks=[3,4,5,8,0,7,1,6,2] 2024-01-24T23:25:09.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:09.918+0000 7f53bbeea640 10 monclient(hunting): picked mon.noname-g con 0x7f53b409e740 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:09.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:09.918+0000 7f53bbeea640 10 monclient(hunting): picked mon.noname-c con 0x7f53b409f230 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:09.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:09.918+0000 7f53bbeea640 10 monclient(hunting): picked mon.noname-h con 0x7f53b4071e90 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:09.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:09.918+0000 7f53bbeea640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b404ed70 con 0x7f53b409e740 2024-01-24T23:25:09.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:09.918+0000 7f53bbeea640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b40a1400 con 0x7f53b409f230 2024-01-24T23:25:09.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:09.918+0000 7f53bbeea640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b40a37d0 con 0x7f53b4071e90 2024-01-24T23:25:09.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:09.918+0000 7f53bbeea640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:09.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:09.918+0000 7f53bbeea640 10 monclient(hunting): authenticate will time out at 1188.840308s 2024-01-24T23:25:09.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:09.918+0000 7f53b9485640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f53b409f230 0x7f53b4139b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:47918/0 (socket says 172.21.15.115:47918) 2024-01-24T23:25:09.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:09.918+0000 7f53b9485640 1 -- 172.21.15.115:0/2579748114 learned_addr learned my addr 172.21.15.115:0/2579748114 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:10.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:10.641+0000 7f96367fc640 10 monclient(hunting): tick 2024-01-24T23:25:10.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:10.641+0000 7f96367fc640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:25:10.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:10.641+0000 7f96367fc640 1 monclient(hunting): continuing hunt 2024-01-24T23:25:10.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:10.641+0000 7f96367fc640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:25:10.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:10.641+0000 7f96367fc640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f96401370e0 legacy=0x7f96401cbed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:10.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:10.641+0000 7f96367fc640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f964013d640 legacy=0x7f96401d3a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:10.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:10.641+0000 7f96367fc640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f964009bc20 legacy=0x7f96401cb7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:10.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:10.641+0000 7f96367fc640 10 monclient: _add_conns ranks=[6,7,5,3,8,2] 2024-01-24T23:25:10.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:10.641+0000 7f96367fc640 10 monclient(hunting): picked mon.noname-d con 0x7f961c003790 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:10.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:10.641+0000 7f96367fc640 10 monclient(hunting): picked mon.noname-i con 0x7f961c007420 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:10.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:10.641+0000 7f96367fc640 10 monclient(hunting): picked mon.noname-h con 0x7f961c00b1b0 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:10.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:10.641+0000 7f96367fc640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96401cc5e0 con 0x7f961c00b1b0 2024-01-24T23:25:10.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:10.641+0000 7f96367fc640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f964009c890 con 0x7f961c003790 2024-01-24T23:25:10.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:10.641+0000 7f96367fc640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f964006b490 con 0x7f961c007420 2024-01-24T23:25:10.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:10.641+0000 7f96367fc640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:10.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:10.641+0000 7f963f7fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f961c00b1b0 0x7f961c00d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:35930/0 (socket says 172.21.15.132:35930) 2024-01-24T23:25:10.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:10.641+0000 7f963e7fc640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f961c007420 0x7f961c0098e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:56994/0 (socket says 172.21.15.132:56994) 2024-01-24T23:25:10.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:10.641+0000 7f963f7fe640 1 -- 172.21.15.132:0/1215331924 learned_addr learned my addr 172.21.15.132:0/1215331924 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:12.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:12.918+0000 7f53ab7fe640 10 monclient(hunting): tick 2024-01-24T23:25:12.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:12.918+0000 7f53ab7fe640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:25:12.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:12.918+0000 7f53ab7fe640 1 monclient(hunting): continuing hunt 2024-01-24T23:25:12.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:12.918+0000 7f53ab7fe640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:25:12.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:12.918+0000 7f53ab7fe640 1 -- 172.21.15.115:0/2579748114 >> v1:172.21.15.132:6791/0 conn(0x7f53b4071e90 legacy=0x7f53b413d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:12.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:12.918+0000 7f53ab7fe640 1 -- 172.21.15.115:0/2579748114 >> v1:172.21.15.115:6791/0 conn(0x7f53b409f230 legacy=0x7f53b4139b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:12.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:12.918+0000 7f53ab7fe640 1 -- 172.21.15.115:0/2579748114 >> v1:172.21.15.132:6790/0 conn(0x7f53b409e740 legacy=0x7f53b409eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:12.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:12.918+0000 7f53ab7fe640 10 monclient: _add_conns ranks=[0,6,8,7,2,1] 2024-01-24T23:25:12.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:12.918+0000 7f53ab7fe640 10 monclient(hunting): picked mon.noname-a con 0x7f53980037f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:12.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:12.918+0000 7f53ab7fe640 10 monclient(hunting): picked mon.noname-d con 0x7f5398007480 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:12.919 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:12.918+0000 7f53ab7fe640 10 monclient(hunting): picked mon.noname-e con 0x7f539800b1d0 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:12.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:12.918+0000 7f53ab7fe640 1 -- 172.21.15.115:0/2579748114 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b40a3120 con 0x7f53980037f0 2024-01-24T23:25:12.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:12.918+0000 7f53ab7fe640 1 -- 172.21.15.115:0/2579748114 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f539800ffe0 con 0x7f5398007480 2024-01-24T23:25:12.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:12.918+0000 7f53ab7fe640 1 -- 172.21.15.115:0/2579748114 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5398011230 con 0x7f539800b1d0 2024-01-24T23:25:12.920 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:12.918+0000 7f53ab7fe640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:13.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:13.645+0000 7f96367fc640 10 monclient(hunting): tick 2024-01-24T23:25:13.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:13.645+0000 7f96367fc640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:25:13.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:13.645+0000 7f96367fc640 1 monclient(hunting): continuing hunt 2024-01-24T23:25:13.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:13.645+0000 7f96367fc640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:25:13.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:13.645+0000 7f96367fc640 1 -- 172.21.15.132:0/1215331924 >> v1:172.21.15.132:6792/0 conn(0x7f961c007420 legacy=0x7f961c0098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:13.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:13.645+0000 7f96367fc640 1 -- 172.21.15.132:0/1215331924 >> v1:172.21.15.115:6792/0 conn(0x7f961c003790 legacy=0x7f961c005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:13.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:13.645+0000 7f96367fc640 1 -- 172.21.15.132:0/1215331924 >> v1:172.21.15.132:6791/0 conn(0x7f961c00b1b0 legacy=0x7f961c00d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:13.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:13.645+0000 7f963f7fe640 1 -- 172.21.15.132:0/1215331924 reap_dead start 2024-01-24T23:25:13.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:13.645+0000 7f96367fc640 10 monclient: _add_conns ranks=[8,2,3] 2024-01-24T23:25:13.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:13.645+0000 7f963f7fe640 1 -- 172.21.15.132:0/1215331924 reap_dead start 2024-01-24T23:25:13.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:13.645+0000 7f96367fc640 10 monclient(hunting): picked mon.noname-e con 0x7f961c011040 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:13.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:13.645+0000 7f96367fc640 10 monclient(hunting): picked mon.noname-b con 0x7f961c00a0b0 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:13.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:13.645+0000 7f96367fc640 10 monclient(hunting): picked mon.noname-g con 0x7f961c00a4c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:13.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:13.645+0000 7f96367fc640 1 -- 172.21.15.132:0/1215331924 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f964006edd0 con 0x7f961c00a0b0 2024-01-24T23:25:13.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:13.645+0000 7f96367fc640 1 -- 172.21.15.132:0/1215331924 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f961c00a930 con 0x7f961c00a4c0 2024-01-24T23:25:13.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:13.645+0000 7f96367fc640 1 -- 172.21.15.132:0/1215331924 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f961c00ab70 con 0x7f961c011040 2024-01-24T23:25:13.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:13.645+0000 7f96367fc640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:14.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 1 -- 172.21.15.115:0/2579748114 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3927401356 0 0) 0x7f5398011230 con 0x7f539800b1d0 2024-01-24T23:25:14.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 10 monclient(hunting): _negotiate 2024-01-24T23:25:14.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:14.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:14.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 10 monclient(hunting): my global_id is 85022 2024-01-24T23:25:14.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 1 -- 172.21.15.115:0/2579748114 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53a0003530 con 0x7f539800b1d0 2024-01-24T23:25:14.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 1 -- 172.21.15.115:0/2579748114 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1278923515 0 0) 0x7f539800ffe0 con 0x7f5398007480 2024-01-24T23:25:14.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 10 monclient(hunting): _negotiate 2024-01-24T23:25:14.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:14.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:14.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 10 monclient(hunting): my global_id is 5928 2024-01-24T23:25:14.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 1 -- 172.21.15.115:0/2579748114 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5398011230 con 0x7f5398007480 2024-01-24T23:25:14.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 1 -- 172.21.15.115:0/2579748114 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 417103662 0 0) 0x7f53a0003530 con 0x7f539800b1d0 2024-01-24T23:25:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 1 -- 172.21.15.115:0/2579748114 >> v1:172.21.15.115:6792/0 conn(0x7f5398007480 legacy=0x7f5398009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 1 -- 172.21.15.115:0/2579748114 >> v1:172.21.15.115:6789/0 conn(0x7f53980037f0 legacy=0x7f5398005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 10 monclient: _finish_hunting 0 2024-01-24T23:25:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 1 monclient: found mon.noname-e 2024-01-24T23:25:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:25:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 1 -- 172.21.15.115:0/2579748114 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5398012480 con 0x7f539800b1d0 2024-01-24T23:25:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 10 monclient: _finish_auth 0 2024-01-24T23:25:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 10 monclient: _check_auth_tickets 2024-01-24T23:25:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 1 -- 172.21.15.115:0/2579748114 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f53ac001b00 con 0x7f539800b1d0 2024-01-24T23:25:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53ba487640 1 -- 172.21.15.115:0/2579748114 reap_dead start 2024-01-24T23:25:14.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:25:14.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 10 monclient: dump: 2024-01-24T23:25:14.947 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:14.947 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:14.947 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:14.947 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:14.947 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:14.947 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:14.947 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:14.947 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:14.947 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:14.947 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:14.947 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:14.947 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:14.947 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:14.948 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:14.948 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:14.948 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:14.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53bbeea640 5 monclient: authenticate success, global_id 85022 2024-01-24T23:25:14.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53bbeea640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:14.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 1 -- 172.21.15.115:0/2579748114 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f53ac001dc0 con 0x7f539800b1d0 2024-01-24T23:25:14.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:14.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 1 -- 172.21.15.115:0/2579748114 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f53ac003ce0 con 0x7f539800b1d0 2024-01-24T23:25:14.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:14.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:14.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53b8c84640 10 monclient: dump: 2024-01-24T23:25:14.953 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:14.953 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:14.953 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:14.953 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:14.954 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:14.954 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:14.954 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:14.954 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:14.954 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:14.954 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:14.954 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:14.954 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:14.954 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:14.954 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:14.954 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:14.955 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:14.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53bbeea640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:14.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.942+0000 7f53bbeea640 4 set_mon_vals no callback set 2024-01-24T23:25:14.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.946+0000 7f53bbeea640 10 monclient: shutdown 2024-01-24T23:25:14.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.946+0000 7f53bbeea640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:14.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.946+0000 7f53bbeea640 1 -- 172.21.15.115:0/2579748114 >> v1:172.21.15.115:6793/0 conn(0x7f539800b1d0 legacy=0x7f539800d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:14.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.946+0000 7f53bbeea640 1 -- 172.21.15.115:0/2579748114 shutdown_connections 2024-01-24T23:25:14.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.946+0000 7f53bbeea640 1 -- 172.21.15.115:0/2579748114 >> 172.21.15.115:0/2579748114 conn(0x7f53b40666f0 msgr2=0x7f53b4066b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:14.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.946+0000 7f53bbeea640 1 -- 172.21.15.115:0/2579748114 shutdown_connections 2024-01-24T23:25:14.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.946+0000 7f53bbeea640 1 -- 172.21.15.115:0/2579748114 wait complete. 2024-01-24T23:25:14.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.946+0000 7f53bbeea640 10 monclient: build_initial_monmap 2024-01-24T23:25:14.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.946+0000 7f53bbeea640 10 monclient: monmap: 2024-01-24T23:25:14.955 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:14.956 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:14.956 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:14.956 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:14.956 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:14.956 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:14.956 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:14.956 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:14.956 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:14.956 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:14.956 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:14.957 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:14.957 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:14.957 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:14.957 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:14.957 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:14.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.946+0000 7f53bbeea640 1 Processor -- start 2024-01-24T23:25:14.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.946+0000 7f53bbeea640 1 -- start start 2024-01-24T23:25:14.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.946+0000 7f53bbeea640 10 monclient: init 2024-01-24T23:25:14.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.950+0000 7f53bbeea640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:14.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.950+0000 7f53bbeea640 10 monclient: _add_conns ranks=[4,2,0,5,7,3,1,8,6] 2024-01-24T23:25:14.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.950+0000 7f53bbeea640 10 monclient(hunting): picked mon.noname-c con 0x7f539800b1d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:14.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.950+0000 7f53bbeea640 10 monclient(hunting): picked mon.noname-b con 0x7f53b41b9fd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:14.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.950+0000 7f53bbeea640 10 monclient(hunting): picked mon.noname-a con 0x7f53b41c9190 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:14.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.950+0000 7f53bbeea640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b409f640 con 0x7f53b41c9190 2024-01-24T23:25:14.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.950+0000 7f53bbeea640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b41a6d70 con 0x7f53b41b9fd0 2024-01-24T23:25:14.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.950+0000 7f53bbeea640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b40722e0 con 0x7f539800b1d0 2024-01-24T23:25:14.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.950+0000 7f53bbeea640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:14.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.950+0000 7f53bbeea640 10 monclient(hunting): authenticate will time out at 1193.873114s 2024-01-24T23:25:14.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.950+0000 7f53ba487640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f53b41c9190 0x7f53b41d19e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44054/0 (socket says 172.21.15.115:44054) 2024-01-24T23:25:14.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.950+0000 7f53ba487640 1 -- 172.21.15.115:0/979880619 learned_addr learned my addr 172.21.15.115:0/979880619 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:14.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.950+0000 7f53b9c86640 1 --1- 172.21.15.115:0/979880619 >> v1:172.21.15.115:6791/0 conn(0x7f539800b1d0 0x7f53b41b98c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:44568/0 (socket says 172.21.15.115:44568) 2024-01-24T23:25:14.959 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 1 -- 172.21.15.132:0/1215331924 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559688979 0 0) 0x7f964006edd0 con 0x7f961c00a0b0 2024-01-24T23:25:14.959 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:14.959 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:14.959 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:14.960 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 10 monclient(hunting): my global_id is 85160 2024-01-24T23:25:14.960 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 1 -- 172.21.15.132:0/1215331924 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9628003530 con 0x7f961c00a0b0 2024-01-24T23:25:14.960 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 1 -- 172.21.15.132:0/1215331924 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4174345570 0 0) 0x7f961c00ab70 con 0x7f961c011040 2024-01-24T23:25:14.960 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:14.960 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:14.960 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:14.960 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 10 monclient(hunting): my global_id is 85031 2024-01-24T23:25:14.960 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 1 -- 172.21.15.132:0/1215331924 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f964006edd0 con 0x7f961c011040 2024-01-24T23:25:14.960 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 1 -- 172.21.15.132:0/1215331924 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1819560296 0 0) 0x7f964006edd0 con 0x7f961c011040 2024-01-24T23:25:14.960 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 1 -- 172.21.15.132:0/1215331924 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f961c00ab70 con 0x7f961c011040 2024-01-24T23:25:14.960 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 1 -- 172.21.15.132:0/1215331924 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9630001780 con 0x7f961c011040 2024-01-24T23:25:14.960 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:14.960 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:25:14.961 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 10 monclient(hunting): dump: 2024-01-24T23:25:14.961 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:14.961 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:14.961 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:14.961 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:14.961 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:14.961 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:14.961 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:14.961 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:14.961 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:14.961 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:14.961 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:14.961 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:14.962 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:14.962 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:14.962 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:14.962 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:14.962 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:14.962 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 1 -- 172.21.15.132:0/1215331924 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2382948885 0 0) 0x7f9628003530 con 0x7f961c00a0b0 2024-01-24T23:25:14.962 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 1 -- 172.21.15.132:0/1215331924 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f964006edd0 con 0x7f961c00a0b0 2024-01-24T23:25:14.962 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 1 -- 172.21.15.132:0/1215331924 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f962c0042f0 con 0x7f961c00a0b0 2024-01-24T23:25:14.962 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:14.962 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:25:14.962 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 10 monclient(hunting): dump: 2024-01-24T23:25:14.962 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:14.962 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:14.963 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:14.963 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:14.963 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:14.963 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:14.963 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:14.963 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:14.963 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:14.963 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:14.963 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:14.963 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:14.963 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:14.963 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:14.963 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:14.964 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:14.964 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 1 -- 172.21.15.132:0/1215331924 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1874817384 0 0) 0x7f961c00ab70 con 0x7f961c011040 2024-01-24T23:25:14.964 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 1 -- 172.21.15.132:0/1215331924 >> v1:172.21.15.132:6790/0 conn(0x7f961c00a4c0 legacy=0x7f961c00d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:14.964 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 1 -- 172.21.15.132:0/1215331924 >> v1:172.21.15.115:6790/0 conn(0x7f961c00a0b0 legacy=0x7f961c005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:14.964 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 10 monclient: _finish_hunting 0 2024-01-24T23:25:14.964 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 1 monclient: found mon.e 2024-01-24T23:25:14.964 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:14.964 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:14.964 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 1 -- 172.21.15.132:0/1215331924 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961c00adb0 con 0x7f961c011040 2024-01-24T23:25:14.964 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 10 monclient: _finish_auth 0 2024-01-24T23:25:14.964 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 10 monclient: _check_auth_tickets 2024-01-24T23:25:14.964 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:14.964 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f9645781640 5 monclient: authenticate success, global_id 85031 2024-01-24T23:25:14.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f9645781640 10 monclient: _renew_subs 2024-01-24T23:25:14.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f9645781640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:14.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f9645781640 1 -- 172.21.15.132:0/1215331924 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964006e920 con 0x7f961c011040 2024-01-24T23:25:14.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f9645781640 10 monclient: _renew_subs 2024-01-24T23:25:14.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f9645781640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:14.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f9645781640 1 -- 172.21.15.132:0/1215331924 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9640070c80 con 0x7f961c011040 2024-01-24T23:25:14.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 1 -- 172.21.15.132:0/1215331924 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f96300011e0 con 0x7f961c011040 2024-01-24T23:25:14.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:14.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 1 -- 172.21.15.132:0/1215331924 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9630001cd0 con 0x7f961c011040 2024-01-24T23:25:14.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:14.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:14.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f963d7fa640 4 set_mon_vals no callback set 2024-01-24T23:25:14.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f96377fe640 10 monclient: dump: 2024-01-24T23:25:14.965 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:14.965 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:14.965 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:14.966 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:14.966 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:14.966 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:14.966 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:14.966 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:14.966 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:14.966 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:14.966 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:14.966 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:14.966 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:14.966 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:14.966 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:14.966 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:14.966 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f9645781640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:14.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f9645781640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:14.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f9645781640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:14.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.941+0000 7f9645781640 1 -- 172.21.15.132:0/1215331924 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9620005090 con 0x7f961c011040 2024-01-24T23:25:14.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.945+0000 7f96377fe640 1 -- 172.21.15.132:0/1215331924 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f963000fdd0 con 0x7f961c011040 2024-01-24T23:25:14.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.945+0000 7f96377fe640 1 -- 172.21.15.132:0/1215331924 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(126..126 src has 1..126) v4 ==== 7321+0+0 (unknown 2448100607 0 0) 0x7f9630044680 con 0x7f961c011040 2024-01-24T23:25:14.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.945+0000 7f963e7fc640 10 monclient: get_auth_request con 0x7f9628037620 auth_method 0 2024-01-24T23:25:14.967 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.957+0000 7f96377fe640 1 -- 172.21.15.132:0/1215331924 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f9630014a90 con 0x7f961c011040 2024-01-24T23:25:14.968 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.957+0000 7f96377fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:14.968 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:14.957+0000 7f96377fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:14.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.966+0000 7f53aa7fc640 1 -- 172.21.15.115:0/979880619 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3486006453 0 0) 0x7f53b41a6d70 con 0x7f53b41b9fd0 2024-01-24T23:25:14.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.966+0000 7f53aa7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:14.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.966+0000 7f53aa7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:14.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.966+0000 7f53aa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:14.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.966+0000 7f53aa7fc640 10 monclient(hunting): my global_id is 85169 2024-01-24T23:25:14.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.966+0000 7f53aa7fc640 1 -- 172.21.15.115:0/979880619 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f539800ef20 con 0x7f53b41b9fd0 2024-01-24T23:25:14.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.966+0000 7f53aa7fc640 1 -- 172.21.15.115:0/979880619 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2145247704 0 0) 0x7f539800ef20 con 0x7f53b41b9fd0 2024-01-24T23:25:14.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.966+0000 7f53aa7fc640 1 -- 172.21.15.115:0/979880619 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53b41a6d70 con 0x7f53b41b9fd0 2024-01-24T23:25:14.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.966+0000 7f53aa7fc640 1 -- 172.21.15.115:0/979880619 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f53a40019a0 con 0x7f53b41b9fd0 2024-01-24T23:25:14.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.966+0000 7f53aa7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:14.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.966+0000 7f53aa7fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:25:14.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.966+0000 7f53aa7fc640 10 monclient(hunting): dump: 2024-01-24T23:25:14.969 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:14.969 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:14.969 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:14.969 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:14.969 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:14.969 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:14.970 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:14.970 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:14.970 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:14.970 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:14.970 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:14.970 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:14.970 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:14.970 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:14.970 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:14.970 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:14.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.966+0000 7f53aa7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:14.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53aa7fc640 1 -- 172.21.15.115:0/979880619 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1095058960 0 0) 0x7f53b409f640 con 0x7f53b41c9190 2024-01-24T23:25:14.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53aa7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:14.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53aa7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:14.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53aa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:14.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53aa7fc640 10 monclient(hunting): my global_id is 94104 2024-01-24T23:25:14.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53aa7fc640 1 -- 172.21.15.115:0/979880619 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f539800ef20 con 0x7f53b41c9190 2024-01-24T23:25:14.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53aa7fc640 1 -- 172.21.15.115:0/979880619 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3539660892 0 0) 0x7f539800ef20 con 0x7f53b41c9190 2024-01-24T23:25:14.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53aa7fc640 1 -- 172.21.15.115:0/979880619 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53b409f640 con 0x7f53b41c9190 2024-01-24T23:25:14.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53aa7fc640 1 -- 172.21.15.115:0/979880619 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f53ac003980 con 0x7f53b41c9190 2024-01-24T23:25:14.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53aa7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:14.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53aa7fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:14.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53aa7fc640 10 monclient(hunting): dump: 2024-01-24T23:25:14.975 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:14.975 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:14.975 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:14.975 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:14.975 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:14.975 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:14.976 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:14.976 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:14.976 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:14.976 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:14.976 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:14.976 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:14.976 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:14.976 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:14.976 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:14.976 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:14.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53aa7fc640 1 -- 172.21.15.115:0/979880619 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2094399494 0 0) 0x7f53b409f640 con 0x7f53b41c9190 2024-01-24T23:25:14.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53aa7fc640 1 -- 172.21.15.115:0/979880619 >> v1:172.21.15.115:6791/0 conn(0x7f539800b1d0 legacy=0x7f53b41b98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:14.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53aa7fc640 1 -- 172.21.15.115:0/979880619 >> v1:172.21.15.115:6790/0 conn(0x7f53b41b9fd0 legacy=0x7f53b41ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:14.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53aa7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:25:14.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53aa7fc640 1 monclient: found mon.a 2024-01-24T23:25:14.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53aa7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:14.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53aa7fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:14.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53aa7fc640 1 -- 172.21.15.115:0/979880619 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b41a4cf0 con 0x7f53b41c9190 2024-01-24T23:25:14.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53aa7fc640 10 monclient: _finish_auth 0 2024-01-24T23:25:14.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53aa7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:25:14.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53aa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:14.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53bbeea640 5 monclient: authenticate success, global_id 94104 2024-01-24T23:25:14.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53bbeea640 10 monclient: _renew_subs 2024-01-24T23:25:14.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53bbeea640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:14.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53bbeea640 1 -- 172.21.15.115:0/979880619 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b41c95e0 con 0x7f53b41c9190 2024-01-24T23:25:14.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53bbeea640 10 monclient: _renew_subs 2024-01-24T23:25:14.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53bbeea640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:14.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.970+0000 7f53bbeea640 1 -- 172.21.15.115:0/979880619 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b41a55d0 con 0x7f53b41c9190 2024-01-24T23:25:14.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.974+0000 7f53aa7fc640 1 -- 172.21.15.115:0/979880619 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f53ac001b20 con 0x7f53b41c9190 2024-01-24T23:25:14.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.974+0000 7f53aa7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:14.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.974+0000 7f53abfff640 4 set_mon_vals no callback set 2024-01-24T23:25:14.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.974+0000 7f53aa7fc640 1 -- 172.21.15.115:0/979880619 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f53ac004220 con 0x7f53b41c9190 2024-01-24T23:25:14.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.974+0000 7f53aa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:14.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.974+0000 7f53aa7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:14.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.974+0000 7f53aa7fc640 10 monclient: dump: 2024-01-24T23:25:14.978 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:14.978 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:14.978 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:14.978 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:14.978 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:14.978 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:14.978 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:14.979 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:14.979 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:14.979 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:14.979 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:14.979 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:14.979 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:14.979 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:14.979 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:14.979 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:14.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.974+0000 7f53bbeea640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:14.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.974+0000 7f53bbeea640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:14.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.974+0000 7f53bbeea640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:14.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.974+0000 7f53bbeea640 1 -- 172.21.15.115:0/979880619 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f539c005090 con 0x7f53b41c9190 2024-01-24T23:25:14.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.974+0000 7f53aa7fc640 1 -- 172.21.15.115:0/979880619 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f53ac004400 con 0x7f53b41c9190 2024-01-24T23:25:14.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.974+0000 7f53aa7fc640 1 -- 172.21.15.115:0/979880619 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(126..126 src has 1..126) v4 ==== 7321+0+0 (unknown 2448100607 0 0) 0x7f53ac044840 con 0x7f53b41c9190 2024-01-24T23:25:14.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.974+0000 7f53b9c86640 10 monclient: get_auth_request con 0x7f53980389d0 auth_method 0 2024-01-24T23:25:14.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.982+0000 7f53aa7fc640 1 -- 172.21.15.115:0/979880619 <== mon.0 v1:172.21.15.115:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f53ac014df0 con 0x7f53b41c9190 2024-01-24T23:25:14.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.982+0000 7f53aa7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:14.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:14.982+0000 7f53aa7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:15.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:15.141+0000 7f9645781640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm", "pool": "fooset"}] 2024-01-24T23:25:15.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:15.141+0000 7f9645781640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set rm", "pool": "fooset"}] 2024-01-24T23:25:15.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:15.141+0000 7f9645781640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:15.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:15.141+0000 7f9645781640 1 -- 172.21.15.132:0/1215331924 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush weight-set rm", "pool": "fooset"} v 0) v1 -- 0x7f9620005340 con 0x7f961c011040 2024-01-24T23:25:15.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.210+0000 7f53bbeea640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-01-24T23:25:15.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.210+0000 7f53bbeea640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-01-24T23:25:15.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.210+0000 7f53bbeea640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:15.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.210+0000 7f53bbeea640 1 -- 172.21.15.115:0/979880619 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f539c0058c0 con 0x7f53b41c9190 2024-01-24T23:25:15.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.210+0000 7f53aa7fc640 1 -- 172.21.15.115:0/979880619 <== mon.0 v1:172.21.15.115:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2744 (unknown 254746657 0 2483595201) 0x7f53ac013260 con 0x7f53b41c9190 2024-01-24T23:25:15.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.210+0000 7f53aa7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-01-24T23:25:15.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.210+0000 7f53aa7fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:15.215 INFO:teuthology.orchestra.run.smithi115.stdout:{"election_epoch":24,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"g","quorum_age":0,"features":{"quorum_con":"4540701272860131327","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","modified":"2024-01-24T23:20:20.557246Z","created":"2024-01-24T23:20:20.557246Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6790","nonce":0}]},"addr":"172.21.15.115:6790/0","public_addr":"172.21.15.115:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6790","nonce":0}]},"addr":"172.21.15.132:6790/0","public_addr":"172.21.15.132:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6791","nonce":0}]},"addr":"172.21.15.115:6791/0","public_addr":"172.21.15.115:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6791","nonce":0}]},"addr":"172.21.15.132:6791/0","public_addr":"172.21.15.132:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6792","nonce":0}]},"addr":"172.21.15.115:6792/0","public_addr":"172.21.15.115:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6792","nonce":0}]},"addr":"172.21.15.132:6792/0","public_addr":"172.21.15.132:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6793","nonce":0}]},"addr":"172.21.15.115:6793/0","public_addr":"172.21.15.115:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-01-24T23:25:15.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.214+0000 7f53bbeea640 1 -- 172.21.15.115:0/979880619 >> v1:172.21.15.132:6812/1835401922 conn(0x7f53980389d0 legacy=0x7f539803ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.214+0000 7f53bbeea640 10 monclient: shutdown 2024-01-24T23:25:15.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.214+0000 7f53bbeea640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:15.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.214+0000 7f53bbeea640 1 -- 172.21.15.115:0/979880619 >> v1:172.21.15.115:6789/0 conn(0x7f53b41c9190 legacy=0x7f53b41d19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.214+0000 7f53bbeea640 1 -- 172.21.15.115:0/979880619 shutdown_connections 2024-01-24T23:25:15.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.214+0000 7f53bbeea640 1 -- 172.21.15.115:0/979880619 >> 172.21.15.115:0/979880619 conn(0x7f53b40666f0 msgr2=0x7f53b40705b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:15.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.214+0000 7f53bbeea640 1 -- 172.21.15.115:0/979880619 shutdown_connections 2024-01-24T23:25:15.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.214+0000 7f53bbeea640 1 -- 172.21.15.115:0/979880619 wait complete. 2024-01-24T23:25:15.230 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-01-24T23:25:15.230 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-01-24T23:25:15.231 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-01-24T23:25:15.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 10 monclient: get_monmap_and_config 2024-01-24T23:25:15.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 10 monclient: build_initial_monmap 2024-01-24T23:25:15.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 10 monclient: monmap: 2024-01-24T23:25:15.338 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:15.338 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.338 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:15.339073+0000 2024-01-24T23:25:15.338 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:15.339073+0000 2024-01-24T23:25:15.339 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:15.339 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.339 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:15.339 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:15.339 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:15.339 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:15.339 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:15.339 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:15.339 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:15.339 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:15.339 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:15.339 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 1 Processor -- start 2024-01-24T23:25:15.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 1 -- start start 2024-01-24T23:25:15.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 10 monclient: init 2024-01-24T23:25:15.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:15.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 10 monclient: _add_conns ranks=[1,5,4,2,8,0,7,3,6] 2024-01-24T23:25:15.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 10 monclient(hunting): picked mon.noname-f con 0x7f071c133bf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:15.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 10 monclient(hunting): picked mon.noname-h con 0x7f071c133fd0 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:15.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 10 monclient(hunting): picked mon.noname-c con 0x7f071c134430 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:15.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071c04ed70 con 0x7f071c133bf0 2024-01-24T23:25:15.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071c09af30 con 0x7f071c134430 2024-01-24T23:25:15.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071c09b210 con 0x7f071c133fd0 2024-01-24T23:25:15.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:15.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 10 monclient(hunting): authenticate will time out at 1194.261556s 2024-01-24T23:25:15.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071ad9d640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f071c134430 0x7f071c13f500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:44574/0 (socket says 172.21.15.115:44574) 2024-01-24T23:25:15.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071ad9d640 1 -- 172.21.15.115:0/760808204 learned_addr learned my addr 172.21.15.115:0/760808204 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:15.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 1 -- 172.21.15.115:0/760808204 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4067615709 0 0) 0x7f071c09af30 con 0x7f071c134430 2024-01-24T23:25:15.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 10 monclient(hunting): my global_id is 94108 2024-01-24T23:25:15.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 1 -- 172.21.15.115:0/760808204 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0704003530 con 0x7f071c134430 2024-01-24T23:25:15.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 1 -- 172.21.15.115:0/760808204 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2738259878 0 0) 0x7f0704003530 con 0x7f071c134430 2024-01-24T23:25:15.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 1 -- 172.21.15.115:0/760808204 >> v1:172.21.15.132:6791/0 conn(0x7f071c133fd0 legacy=0x7f071c13bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 1 -- 172.21.15.115:0/760808204 >> v1:172.21.15.132:6789/0 conn(0x7f071c133bf0 legacy=0x7f071c09bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 10 monclient: _finish_hunting 0 2024-01-24T23:25:15.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 1 monclient: found mon.noname-c 2024-01-24T23:25:15.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:15.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:25:15.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 1 -- 172.21.15.115:0/760808204 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c09b4f0 con 0x7f071c134430 2024-01-24T23:25:15.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 10 monclient: _finish_auth 0 2024-01-24T23:25:15.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 10 monclient: _check_auth_tickets 2024-01-24T23:25:15.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:15.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 1 -- 172.21.15.115:0/760808204 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0714001b40 con 0x7f071c134430 2024-01-24T23:25:15.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 5 monclient: authenticate success, global_id 94108 2024-01-24T23:25:15.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:15.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:25:15.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 10 monclient: dump: 2024-01-24T23:25:15.350 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.350 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.350 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.350 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.350 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.350 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.350 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.350 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.350 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.350 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.350 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.350 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.351 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.351 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.351 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.351 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:15.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 1 -- 172.21.15.115:0/760808204 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0714001df0 con 0x7f071c134430 2024-01-24T23:25:15.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:15.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:15.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 1 -- 172.21.15.115:0/760808204 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0714004f70 con 0x7f071c134430 2024-01-24T23:25:15.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 4 set_mon_vals no callback set 2024-01-24T23:25:15.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:15.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f071959a640 10 monclient: dump: 2024-01-24T23:25:15.351 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.351 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.352 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.352 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.352 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.352 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.352 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.352 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.352 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.352 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.352 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.352 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.352 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.352 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.352 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.352 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 10 monclient: shutdown 2024-01-24T23:25:15.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:15.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 1 -- 172.21.15.115:0/760808204 >> v1:172.21.15.115:6791/0 conn(0x7f071c134430 legacy=0x7f071c13f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 1 -- 172.21.15.115:0/760808204 shutdown_connections 2024-01-24T23:25:15.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 1 -- 172.21.15.115:0/760808204 >> 172.21.15.115:0/760808204 conn(0x7f071c0a9910 msgr2=0x7f071c0a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:15.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 1 -- 172.21.15.115:0/760808204 shutdown_connections 2024-01-24T23:25:15.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 1 -- 172.21.15.115:0/760808204 wait complete. 2024-01-24T23:25:15.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 10 monclient: build_initial_monmap 2024-01-24T23:25:15.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.338+0000 7f0720fc9640 10 monclient: monmap: 2024-01-24T23:25:15.353 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:15.353 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:15.353 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:15.353 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:15.353 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:15.353 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:15.353 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:15.354 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:15.354 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:15.354 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:15.354 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:15.354 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:15.354 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:15.354 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:15.354 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:15.354 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f0720fc9640 1 Processor -- start 2024-01-24T23:25:15.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f0720fc9640 1 -- start start 2024-01-24T23:25:15.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f0720fc9640 10 monclient: init 2024-01-24T23:25:15.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f0720fc9640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:15.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f0720fc9640 10 monclient: _add_conns ranks=[4,5,0,7,3,1,2,8,6] 2024-01-24T23:25:15.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f0720fc9640 10 monclient(hunting): picked mon.noname-c con 0x7f071c133bf0 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:15.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f0720fc9640 10 monclient(hunting): picked mon.noname-h con 0x7f071c133fd0 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:15.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f0720fc9640 10 monclient(hunting): picked mon.noname-a con 0x7f071c134430 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:15.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f0720fc9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071c09c8f0 con 0x7f071c134430 2024-01-24T23:25:15.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f0720fc9640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071c0a5960 con 0x7f071c133bf0 2024-01-24T23:25:15.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f0720fc9640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071c13a5d0 con 0x7f071c133fd0 2024-01-24T23:25:15.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f0720fc9640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:15.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f0720fc9640 10 monclient(hunting): authenticate will time out at 1194.264910s 2024-01-24T23:25:15.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f071a59c640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f071c133bf0 0x7f071c137ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:44586/0 (socket says 172.21.15.115:44586) 2024-01-24T23:25:15.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f071ad9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f071c134430 0x7f071c1d3b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44070/0 (socket says 172.21.15.115:44070) 2024-01-24T23:25:15.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f071a59c640 1 -- 172.21.15.115:0/3293316629 learned_addr learned my addr 172.21.15.115:0/3293316629 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:15.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810009686 0 0) 0x7f071c0a5960 con 0x7f071c133bf0 2024-01-24T23:25:15.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient(hunting): my global_id is 94117 2024-01-24T23:25:15.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06fc003530 con 0x7f071c133bf0 2024-01-24T23:25:15.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695572362 0 0) 0x7f071c09c8f0 con 0x7f071c134430 2024-01-24T23:25:15.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient(hunting): my global_id is 94113 2024-01-24T23:25:15.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f071c0a5960 con 0x7f071c134430 2024-01-24T23:25:15.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4064468338 0 0) 0x7f06fc003530 con 0x7f071c133bf0 2024-01-24T23:25:15.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f071c09c8f0 con 0x7f071c133bf0 2024-01-24T23:25:15.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0710001ae0 con 0x7f071c133bf0 2024-01-24T23:25:15.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:25:15.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient(hunting): dump: 2024-01-24T23:25:15.356 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.356 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.356 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.357 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.357 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.357 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.357 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.357 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.357 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.357 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.357 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.357 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.357 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.357 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.357 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.357 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:15.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 101517116 0 0) 0x7f071c0a5960 con 0x7f071c134430 2024-01-24T23:25:15.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f06fc003530 con 0x7f071c134430 2024-01-24T23:25:15.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0714001b00 con 0x7f071c134430 2024-01-24T23:25:15.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:15.358 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient(hunting): dump: 2024-01-24T23:25:15.358 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.358 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.358 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.358 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.358 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.358 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.358 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.358 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.358 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.358 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.358 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.358 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.358 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.359 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.359 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.359 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3931481645 0 0) 0x7f071c13a5d0 con 0x7f071c133fd0 2024-01-24T23:25:15.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient(hunting): my global_id is 5810 2024-01-24T23:25:15.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f071c0a5960 con 0x7f071c133fd0 2024-01-24T23:25:15.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1889465961 0 0) 0x7f071c09c8f0 con 0x7f071c133bf0 2024-01-24T23:25:15.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 >> v1:172.21.15.132:6791/0 conn(0x7f071c133fd0 legacy=0x7f071c1382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 >> v1:172.21.15.115:6789/0 conn(0x7f071c134430 legacy=0x7f071c1d3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient: _finish_hunting 0 2024-01-24T23:25:15.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 monclient: found mon.c 2024-01-24T23:25:15.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:15.359 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:15.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c13a8b0 con 0x7f071c133bf0 2024-01-24T23:25:15.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient: _finish_auth 0 2024-01-24T23:25:15.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:15.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:15.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f0720fc9640 5 monclient: authenticate success, global_id 94117 2024-01-24T23:25:15.360 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f0720fc9640 10 monclient: _renew_subs 2024-01-24T23:25:15.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f0720fc9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:15.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f0720fc9640 1 -- 172.21.15.115:0/3293316629 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f071c13f720 con 0x7f071c133bf0 2024-01-24T23:25:15.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f0720fc9640 10 monclient: _renew_subs 2024-01-24T23:25:15.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f0720fc9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:15.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f0720fc9640 1 -- 172.21.15.115:0/3293316629 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f071c138b90 con 0x7f071c133bf0 2024-01-24T23:25:15.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0710003950 con 0x7f071c133bf0 2024-01-24T23:25:15.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:15.361 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0710003e80 con 0x7f071c133bf0 2024-01-24T23:25:15.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f0718d99640 4 set_mon_vals no callback set 2024-01-24T23:25:15.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:15.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient: dump: 2024-01-24T23:25:15.362 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.362 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.362 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.362 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.363 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.363 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.363 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.363 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.363 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.363 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.364 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.364 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.364 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.364 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.364 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.364 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f0720fc9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:15.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f0720fc9640 1 -- 172.21.15.115:0/3293316629 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f07000065d0 con 0x7f0700002870 2024-01-24T23:25:15.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f0710004060 con 0x7f071c133bf0 2024-01-24T23:25:15.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(126..126 src has 1..126) v4 ==== 7321+0+0 (unknown 2448100607 0 0) 0x7f0710044650 con 0x7f071c133bf0 2024-01-24T23:25:15.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2516605892 0 0) 0x7f07000065d0 con 0x7f0700002870 2024-01-24T23:25:15.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient(hunting): my global_id is 94122 2024-01-24T23:25:15.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f071c09c8f0 con 0x7f0700002870 2024-01-24T23:25:15.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f071ad9d640 10 monclient: get_auth_request con 0x7f06fc037800 auth_method 0 2024-01-24T23:25:15.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3810904021 0 0) 0x7f071c09c8f0 con 0x7f0700002870 2024-01-24T23:25:15.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f07000065d0 con 0x7f0700002870 2024-01-24T23:25:15.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f070c0018f0 con 0x7f0700002870 2024-01-24T23:25:15.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:15.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient: dump: 2024-01-24T23:25:15.367 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.367 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.367 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.367 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.367 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.367 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.367 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.368 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.368 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.368 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.368 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.368 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.368 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.368 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.369 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.369 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 717553844 0 0) 0x7f07000065d0 con 0x7f0700002870 2024-01-24T23:25:15.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 --> v1:172.21.15.115:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f070000a120 con 0x7f0700002870 2024-01-24T23:25:15.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 <== mon.0 v1:172.21.15.115:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f070c003600 con 0x7f0700002870 2024-01-24T23:25:15.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:15.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:15.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 >> v1:172.21.15.115:6789/0 conn(0x7f0700002870 legacy=0x7f0700004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.342+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 >> v1:172.21.15.115:6789/0 conn(0x7f0700002870 legacy=0x7f0700004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:15.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f0720fc9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:25:15.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f0720fc9640 1 -- 172.21.15.115:0/3293316629 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0700009df0 con 0x7f07000090b0 2024-01-24T23:25:15.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3632215011 0 0) 0x7f0700009df0 con 0x7f07000090b0 2024-01-24T23:25:15.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f070affd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f070affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f070affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f070affd640 10 monclient(hunting): my global_id is 94131 2024-01-24T23:25:15.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07000065d0 con 0x7f07000090b0 2024-01-24T23:25:15.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3839926603 0 0) 0x7f07000065d0 con 0x7f07000090b0 2024-01-24T23:25:15.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0700009df0 con 0x7f07000090b0 2024-01-24T23:25:15.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f070c003d20 con 0x7f07000090b0 2024-01-24T23:25:15.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f070affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f070affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:15.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f070affd640 10 monclient: dump: 2024-01-24T23:25:15.372 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.372 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.372 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.372 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.372 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.372 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.372 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.372 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.373 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.373 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.373 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.373 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.373 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.373 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.373 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.373 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2773391164 0 0) 0x7f0700009df0 con 0x7f07000090b0 2024-01-24T23:25:15.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 --> v1:172.21.15.115:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f070000c310 con 0x7f07000090b0 2024-01-24T23:25:15.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 <== mon.0 v1:172.21.15.115:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 2906530726) 0x7f070c004190 con 0x7f07000090b0 2024-01-24T23:25:15.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f070affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:25:15.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f070affd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:15.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 >> v1:172.21.15.115:6789/0 conn(0x7f07000090b0 legacy=0x7f0700009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f070affd640 1 -- 172.21.15.115:0/3293316629 >> v1:172.21.15.115:6789/0 conn(0x7f07000090b0 legacy=0x7f0700009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:15.374 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:25:15.374 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:15.374 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:15.374 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:25:15.374 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 24, 2024-01-24T23:25:15.374 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:25:15.374 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:25:15.374 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:25:15.374 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:25:15.374 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:25:15.375 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:25:15.375 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:25:15.375 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:25:15.375 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:25:15.375 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:25:15.375 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:15.375 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:25:15.375 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:15.375 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:25:15.375 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:25:15.375 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:15.375 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:15.375 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:15.375 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:15.375 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:15.376 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:15.376 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:15.376 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:15.376 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:15.376 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:15.376 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:15.376 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:15.376 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:25:15.376 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:25:15.376 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:15.376 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:15.376 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:15.376 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:15.377 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:15.377 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:15.377 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:15.377 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:15.377 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:15.377 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:15.377 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:15.377 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.377 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:25:15.377 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:25:15.377 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:25:15.377 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:25:15.377 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:25:15.377 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:25:15.378 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:15.378 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:15.378 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:25:15.378 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:25:15.378 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:25:15.378 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:25:15.378 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:25:15.378 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:25:15.378 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:25:15.378 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:15.378 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:25:15.378 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:15.378 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:15.378 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:15.378 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:15.379 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:15.379 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:15.379 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:15.379 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:15.379 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:15.379 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:15.379 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:15.379 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:15.379 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:25:15.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.379 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:25:15.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.380 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:15.380 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:15.380 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.380 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.380 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.380 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:25:15.380 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.380 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.380 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.380 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:25:15.381 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:25:15.381 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.381 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.381 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.381 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:15.381 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:15.381 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.381 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.381 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.381 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:25:15.381 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.382 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.382 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.382 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:25:15.382 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:25:15.382 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.382 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.382 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.382 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:15.382 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:15.382 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.382 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.383 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.383 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:25:15.383 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.383 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.383 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.383 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:25:15.383 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:25:15.383 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.383 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.384 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.384 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:15.384 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:15.384 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.384 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.384 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.384 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:25:15.384 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.384 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.385 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.385 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:25:15.385 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:25:15.385 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.385 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.385 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.385 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:15.385 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:15.385 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.385 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.385 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:15.386 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.387 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.387 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.387 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:25:15.387 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.387 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.387 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.387 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:25:15.387 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:25:15.387 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.387 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.387 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.388 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:15.388 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:15.388 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.388 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.388 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.388 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:25:15.388 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.388 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.388 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.390 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:25:15.390 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:25:15.390 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.390 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.390 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.390 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:15.390 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:15.390 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.390 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.390 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.390 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:25:15.390 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.391 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.391 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.391 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:25:15.391 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:25:15.391 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.391 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.391 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.391 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.391 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.391 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:25:15.391 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:25:15.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.391 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:15.392 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:15.392 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:15.392 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.392 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:15.392 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:25:15.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.392 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:15.392 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:15.392 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:15.392 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.392 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.392 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:25:15.393 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:25:15.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f0720fc9640 1 -- 172.21.15.115:0/3293316629 >> v1:172.21.15.132:6812/1835401922 conn(0x7f06fc037800 legacy=0x7f06fc039cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f0720fc9640 10 monclient: shutdown 2024-01-24T23:25:15.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f0720fc9640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:15.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f0720fc9640 1 -- 172.21.15.115:0/3293316629 >> v1:172.21.15.115:6791/0 conn(0x7f071c133bf0 legacy=0x7f071c137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f071ad9d640 1 -- 172.21.15.115:0/3293316629 reap_dead start 2024-01-24T23:25:15.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f071ad9d640 1 -- 172.21.15.115:0/3293316629 reap_dead start 2024-01-24T23:25:15.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f0720fc9640 1 -- 172.21.15.115:0/3293316629 shutdown_connections 2024-01-24T23:25:15.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f0720fc9640 1 -- 172.21.15.115:0/3293316629 >> 172.21.15.115:0/3293316629 conn(0x7f071c0a9910 msgr2=0x7f071c13fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:15.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f0720fc9640 1 -- 172.21.15.115:0/3293316629 shutdown_connections 2024-01-24T23:25:15.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.350+0000 7f0720fc9640 1 -- 172.21.15.115:0/3293316629 wait complete. 2024-01-24T23:25:15.395 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:25:15.395 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-01-24T23:25:15.395 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-01-24T23:25:15.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.502+0000 7f9262822640 10 monclient: get_monmap_and_config 2024-01-24T23:25:15.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.502+0000 7f9262822640 10 monclient: build_initial_monmap 2024-01-24T23:25:15.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.502+0000 7f9262822640 10 monclient: monmap: 2024-01-24T23:25:15.505 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:15.505 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.506 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:15.506115+0000 2024-01-24T23:25:15.506 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:15.506115+0000 2024-01-24T23:25:15.506 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:15.506 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.506 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:15.506 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:15.506 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:15.506 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:15.506 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:15.506 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:15.506 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:15.506 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:15.506 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:15.506 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.502+0000 7f9262822640 1 Processor -- start 2024-01-24T23:25:15.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 1 -- start start 2024-01-24T23:25:15.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 10 monclient: init 2024-01-24T23:25:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 10 monclient: _add_conns ranks=[4,6,7,1,8,5,2,3,0] 2024-01-24T23:25:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 10 monclient(hunting): picked mon.noname-c con 0x7f925c133c10 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 10 monclient(hunting): picked mon.noname-d con 0x7f925c136960 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 10 monclient(hunting): picked mon.noname-i con 0x7f925c13a470 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f925c04ed70 con 0x7f925c133c10 2024-01-24T23:25:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f925c09dee0 con 0x7f925c136960 2024-01-24T23:25:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f925c09d400 con 0x7f925c13a470 2024-01-24T23:25:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 10 monclient(hunting): authenticate will time out at 1194.428690s 2024-01-24T23:25:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925bfff640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f925c133c10 0x7f925c133ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:44600/0 (socket says 172.21.15.115:44600) 2024-01-24T23:25:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925bfff640 1 -- 172.21.15.115:0/405731232 learned_addr learned my addr 172.21.15.115:0/405731232 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 1 -- 172.21.15.115:0/405731232 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3874725429 0 0) 0x7f925c09dee0 con 0x7f925c136960 2024-01-24T23:25:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 10 monclient(hunting): my global_id is 5937 2024-01-24T23:25:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 1 -- 172.21.15.115:0/405731232 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9248003530 con 0x7f925c136960 2024-01-24T23:25:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 1 -- 172.21.15.115:0/405731232 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 708792387 0 0) 0x7f925c04ed70 con 0x7f925c133c10 2024-01-24T23:25:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 10 monclient(hunting): my global_id is 94126 2024-01-24T23:25:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 1 -- 172.21.15.115:0/405731232 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f925c09dee0 con 0x7f925c133c10 2024-01-24T23:25:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 1 -- 172.21.15.115:0/405731232 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1107401981 0 0) 0x7f9248003530 con 0x7f925c136960 2024-01-24T23:25:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 1 -- 172.21.15.115:0/405731232 >> v1:172.21.15.132:6792/0 conn(0x7f925c13a470 legacy=0x7f925c13c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 1 -- 172.21.15.115:0/405731232 >> v1:172.21.15.115:6791/0 conn(0x7f925c133c10 legacy=0x7f925c133ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:25:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 1 monclient: found mon.noname-d 2024-01-24T23:25:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:15.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:25:15.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 1 -- 172.21.15.115:0/405731232 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925c09f380 con 0x7f925c136960 2024-01-24T23:25:15.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 10 monclient: _finish_auth 0 2024-01-24T23:25:15.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:25:15.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:15.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 1 -- 172.21.15.115:0/405731232 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f924c0016f0 con 0x7f925c136960 2024-01-24T23:25:15.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:25:15.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 10 monclient: dump: 2024-01-24T23:25:15.517 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.517 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.517 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.517 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.517 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.517 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.518 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.518 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.518 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.518 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.518 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.518 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.518 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.518 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.518 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.518 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 5 monclient: authenticate success, global_id 5937 2024-01-24T23:25:15.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:15.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 1 -- 172.21.15.115:0/405731232 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f924c001990 con 0x7f925c136960 2024-01-24T23:25:15.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:15.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 1 -- 172.21.15.115:0/405731232 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f924c003960 con 0x7f925c136960 2024-01-24T23:25:15.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:15.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 4 set_mon_vals no callback set 2024-01-24T23:25:15.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:15.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f925b7fe640 10 monclient: dump: 2024-01-24T23:25:15.519 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.519 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.519 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.519 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.519 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.519 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.519 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.520 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.520 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.520 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.520 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.520 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.520 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.520 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.520 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.520 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 10 monclient: shutdown 2024-01-24T23:25:15.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:15.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 1 -- 172.21.15.115:0/405731232 >> v1:172.21.15.115:6792/0 conn(0x7f925c136960 legacy=0x7f925c138d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 1 -- 172.21.15.115:0/405731232 shutdown_connections 2024-01-24T23:25:15.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 1 -- 172.21.15.115:0/405731232 >> 172.21.15.115:0/405731232 conn(0x7f925c0a5730 msgr2=0x7f925c0a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:15.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 1 -- 172.21.15.115:0/405731232 shutdown_connections 2024-01-24T23:25:15.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 1 -- 172.21.15.115:0/405731232 wait complete. 2024-01-24T23:25:15.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 10 monclient: build_initial_monmap 2024-01-24T23:25:15.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 10 monclient: monmap: 2024-01-24T23:25:15.521 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:15.521 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:15.521 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:15.521 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:15.521 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:15.521 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:15.521 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:15.522 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:15.522 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:15.522 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:15.522 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:15.522 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:15.522 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:15.522 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:15.522 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:15.522 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 1 Processor -- start 2024-01-24T23:25:15.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 1 -- start start 2024-01-24T23:25:15.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 10 monclient: init 2024-01-24T23:25:15.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:15.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 10 monclient: _add_conns ranks=[8,1,2,6,3,7,5,0,4] 2024-01-24T23:25:15.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 10 monclient(hunting): picked mon.noname-e con 0x7f925c133c10 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:15.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 10 monclient(hunting): picked mon.noname-f con 0x7f925c136960 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:15.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 10 monclient(hunting): picked mon.noname-b con 0x7f925c13a470 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:15.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f925c0a1310 con 0x7f925c136960 2024-01-24T23:25:15.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f925c06efa0 con 0x7f925c13a470 2024-01-24T23:25:15.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f925c0a0710 con 0x7f925c133c10 2024-01-24T23:25:15.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:15.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9262822640 10 monclient(hunting): authenticate will time out at 1194.431678s 2024-01-24T23:25:15.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9260dbf640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f925c13a470 0x7f925c1d1650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:60310/0 (socket says 172.21.15.115:60310) 2024-01-24T23:25:15.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.506+0000 7f9260dbf640 1 -- 172.21.15.115:0/1506402379 learned_addr learned my addr 172.21.15.115:0/1506402379 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:15.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2188380724 0 0) 0x7f925c06efa0 con 0x7f925c13a470 2024-01-24T23:25:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient(hunting): my global_id is 85178 2024-01-24T23:25:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f923c003530 con 0x7f925c13a470 2024-01-24T23:25:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2326098435 0 0) 0x7f925c0a0710 con 0x7f925c133c10 2024-01-24T23:25:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient(hunting): my global_id is 85040 2024-01-24T23:25:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f925c06efa0 con 0x7f925c133c10 2024-01-24T23:25:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4057669510 0 0) 0x7f923c003530 con 0x7f925c13a470 2024-01-24T23:25:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f925c0a0710 con 0x7f925c13a470 2024-01-24T23:25:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2548179748 0 0) 0x7f925c06efa0 con 0x7f925c133c10 2024-01-24T23:25:15.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f923c003530 con 0x7f925c133c10 2024-01-24T23:25:15.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9254001d50 con 0x7f925c13a470 2024-01-24T23:25:15.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:25:15.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient(hunting): dump: 2024-01-24T23:25:15.525 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.525 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.525 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.525 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.525 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.525 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.525 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.526 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.526 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.526 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.526 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.526 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.526 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.526 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.526 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.526 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:15.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9244001330 con 0x7f925c133c10 2024-01-24T23:25:15.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:15.526 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient(hunting): dump: 2024-01-24T23:25:15.527 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.527 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.527 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.527 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.527 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.527 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.527 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.527 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.527 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.528 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.528 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.528 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.528 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.528 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.528 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.528 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2848149793 0 0) 0x7f925c0a1310 con 0x7f925c136960 2024-01-24T23:25:15.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.528 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient(hunting): my global_id is 94114 2024-01-24T23:25:15.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f925c06efa0 con 0x7f925c136960 2024-01-24T23:25:15.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3698221898 0 0) 0x7f925c0a0710 con 0x7f925c13a470 2024-01-24T23:25:15.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 >> v1:172.21.15.115:6793/0 conn(0x7f925c133c10 legacy=0x7f925c09f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 >> v1:172.21.15.132:6789/0 conn(0x7f925c136960 legacy=0x7f925c0a0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient: _finish_hunting 0 2024-01-24T23:25:15.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 monclient: found mon.b 2024-01-24T23:25:15.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:15.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:25:15.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925c06dc20 con 0x7f925c13a470 2024-01-24T23:25:15.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient: _finish_auth 0 2024-01-24T23:25:15.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient: _check_auth_tickets 2024-01-24T23:25:15.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:15.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f9262822640 5 monclient: authenticate success, global_id 85178 2024-01-24T23:25:15.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f9262822640 10 monclient: _renew_subs 2024-01-24T23:25:15.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f9262822640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:25:15.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f9262822640 1 -- 172.21.15.115:0/1506402379 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f925c06ccb0 con 0x7f925c13a470 2024-01-24T23:25:15.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f9262822640 10 monclient: _renew_subs 2024-01-24T23:25:15.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f9262822640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:25:15.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f9262822640 1 -- 172.21.15.115:0/1506402379 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f925c06d470 con 0x7f925c13a470 2024-01-24T23:25:15.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9254002050 con 0x7f925c13a470 2024-01-24T23:25:15.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:15.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9254003f90 con 0x7f925c13a470 2024-01-24T23:25:15.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:25:15.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f925affd640 4 set_mon_vals no callback set 2024-01-24T23:25:15.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient: dump: 2024-01-24T23:25:15.532 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.532 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.532 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.532 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.532 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.533 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.533 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.533 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.533 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.533 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.533 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.533 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.533 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.533 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.534 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.534 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f9262822640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:15.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f9262822640 1 -- 172.21.15.115:0/1506402379 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f92400065d0 con 0x7f9240002870 2024-01-24T23:25:15.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f92540040f0 con 0x7f925c13a470 2024-01-24T23:25:15.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(126..126 src has 1..126) v4 ==== 7321+0+0 (unknown 2448100607 0 0) 0x7f92540447a0 con 0x7f925c13a470 2024-01-24T23:25:15.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2569486453 0 0) 0x7f92400065d0 con 0x7f9240002870 2024-01-24T23:25:15.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient(hunting): my global_id is 85187 2024-01-24T23:25:15.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f925c0a0710 con 0x7f9240002870 2024-01-24T23:25:15.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f9253fff640 10 monclient: get_auth_request con 0x7f923c037870 auth_method 0 2024-01-24T23:25:15.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 67227987 0 0) 0x7f925c0a0710 con 0x7f9240002870 2024-01-24T23:25:15.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f92400065d0 con 0x7f9240002870 2024-01-24T23:25:15.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9244001700 con 0x7f9240002870 2024-01-24T23:25:15.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:25:15.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient: dump: 2024-01-24T23:25:15.535 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.536 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.536 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.536 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.536 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.536 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.536 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.536 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.536 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.536 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.537 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2376950290 0 0) 0x7f92400065d0 con 0x7f9240002870 2024-01-24T23:25:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 --> v1:172.21.15.115:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f924000a120 con 0x7f9240002870 2024-01-24T23:25:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 <== mon.2 v1:172.21.15.115:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f9244004eb0 con 0x7f9240002870 2024-01-24T23:25:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 >> v1:172.21.15.115:6790/0 conn(0x7f9240002870 legacy=0x7f9240004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.510+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 >> v1:172.21.15.115:6790/0 conn(0x7f9240002870 legacy=0x7f9240004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:15.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f9262822640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:25:15.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f9262822640 1 -- 172.21.15.115:0/1506402379 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9240009e10 con 0x7f92400090d0 2024-01-24T23:25:15.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2193185462 0 0) 0x7f9240009e10 con 0x7f92400090d0 2024-01-24T23:25:15.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f92597fa640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f92597fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f92597fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f92597fa640 10 monclient(hunting): my global_id is 85196 2024-01-24T23:25:15.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92400065d0 con 0x7f92400090d0 2024-01-24T23:25:15.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1797343267 0 0) 0x7f92400065d0 con 0x7f92400090d0 2024-01-24T23:25:15.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9240009e10 con 0x7f92400090d0 2024-01-24T23:25:15.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f92440056f0 con 0x7f92400090d0 2024-01-24T23:25:15.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f92597fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f92597fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:25:15.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f92597fa640 10 monclient: dump: 2024-01-24T23:25:15.540 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.541 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.541 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.541 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.541 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.541 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.541 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.542 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.542 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.542 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.542 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.542 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.542 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.543 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.543 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.543 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1382608725 0 0) 0x7f9240009e10 con 0x7f92400090d0 2024-01-24T23:25:15.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 --> v1:172.21.15.115:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f924000c310 con 0x7f92400090d0 2024-01-24T23:25:15.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 <== mon.2 v1:172.21.15.115:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 1060328105) 0x7f92440059e0 con 0x7f92400090d0 2024-01-24T23:25:15.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f92597fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:25:15.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f92597fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:15.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 >> v1:172.21.15.115:6790/0 conn(0x7f92400090d0 legacy=0x7f9240009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f92597fa640 1 -- 172.21.15.115:0/1506402379 >> v1:172.21.15.115:6790/0 conn(0x7f92400090d0 legacy=0x7f9240009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:15.544 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:25:15.544 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:25:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 24, 2024-01-24T23:25:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:25:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:25:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:25:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:25:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:25:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:25:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:25:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:25:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:25:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:25:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:25:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:25:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:25:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:25:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:25:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:25:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:25:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:25:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:25:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:25:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:25:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:25:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:25:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:25:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:25:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:25:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:25:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:25:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:25:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:25:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:25:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:25:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:25:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:25:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:25:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:25:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:25:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:25:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:25:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:25:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:25:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:25:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:25:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:25:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:25:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:25:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:25:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:25:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:25:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:25:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:25:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:25:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:25:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:25:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:25:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:25:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:25:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:25:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:25:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:25:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:25:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:25:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:25:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:25:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:25:15.563 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:25:15.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f9262822640 1 -- 172.21.15.115:0/1506402379 >> v1:172.21.15.132:6812/1835401922 conn(0x7f923c037870 legacy=0x7f923c039d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f9262822640 10 monclient: shutdown 2024-01-24T23:25:15.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f9262822640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:15.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f9262822640 1 -- 172.21.15.115:0/1506402379 >> v1:172.21.15.115:6790/0 conn(0x7f925c13a470 legacy=0x7f925c1d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f9260dbf640 1 -- 172.21.15.115:0/1506402379 reap_dead start 2024-01-24T23:25:15.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f9260dbf640 1 -- 172.21.15.115:0/1506402379 reap_dead start 2024-01-24T23:25:15.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f9262822640 1 -- 172.21.15.115:0/1506402379 shutdown_connections 2024-01-24T23:25:15.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f9262822640 1 -- 172.21.15.115:0/1506402379 >> 172.21.15.115:0/1506402379 conn(0x7f925c0a5730 msgr2=0x7f925c12f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:15.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f9262822640 1 -- 172.21.15.115:0/1506402379 shutdown_connections 2024-01-24T23:25:15.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.518+0000 7f9262822640 1 -- 172.21.15.115:0/1506402379 wait complete. 2024-01-24T23:25:15.566 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:25:15.566 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-01-24T23:25:15.566 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-01-24T23:25:15.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f7837014640 10 monclient: get_monmap_and_config 2024-01-24T23:25:15.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f7837014640 10 monclient: build_initial_monmap 2024-01-24T23:25:15.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f7837014640 10 monclient: monmap: 2024-01-24T23:25:15.671 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:15.671 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.671 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:15.671876+0000 2024-01-24T23:25:15.671 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:15.671876+0000 2024-01-24T23:25:15.671 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:15.671 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.672 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:15.672 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:15.672 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:15.672 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:15.672 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:15.672 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:15.672 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:15.672 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:15.672 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:15.672 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f7837014640 1 Processor -- start 2024-01-24T23:25:15.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f7837014640 1 -- start start 2024-01-24T23:25:15.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f7837014640 10 monclient: init 2024-01-24T23:25:15.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f7837014640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:15.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f7837014640 10 monclient: _add_conns ranks=[3,2,7,8,4,1,0,6,5] 2024-01-24T23:25:15.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f7837014640 10 monclient(hunting): picked mon.noname-g con 0x7f78300662f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:15.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f7837014640 10 monclient(hunting): picked mon.noname-b con 0x7f78301305c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:15.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f7837014640 10 monclient(hunting): picked mon.noname-i con 0x7f783013a030 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:15.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f7837014640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f783004ed70 con 0x7f78301305c0 2024-01-24T23:25:15.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f7837014640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78300a2ff0 con 0x7f78300662f0 2024-01-24T23:25:15.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f7837014640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78300666d0 con 0x7f783013a030 2024-01-24T23:25:15.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f7837014640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:15.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f7837014640 10 monclient(hunting): authenticate will time out at 1194.595156s 2024-01-24T23:25:15.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f782ffff640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f78301305c0 0x7f7830138910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:60340/0 (socket says 172.21.15.115:60340) 2024-01-24T23:25:15.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f782ffff640 1 -- 172.21.15.115:0/2684175171 learned_addr learned my addr 172.21.15.115:0/2684175171 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:15.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f782f7fe640 1 -- 172.21.15.115:0/2684175171 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1152518800 0 0) 0x7f783004ed70 con 0x7f78301305c0 2024-01-24T23:25:15.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f782f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f782f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f782f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f782f7fe640 10 monclient(hunting): my global_id is 85205 2024-01-24T23:25:15.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f782f7fe640 1 -- 172.21.15.115:0/2684175171 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f781c003530 con 0x7f78301305c0 2024-01-24T23:25:15.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f782f7fe640 1 -- 172.21.15.115:0/2684175171 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2385016646 0 0) 0x7f781c003530 con 0x7f78301305c0 2024-01-24T23:25:15.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f782f7fe640 1 -- 172.21.15.115:0/2684175171 >> v1:172.21.15.132:6792/0 conn(0x7f783013a030 legacy=0x7f783013c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f782f7fe640 1 -- 172.21.15.115:0/2684175171 >> v1:172.21.15.132:6790/0 conn(0x7f78300662f0 legacy=0x7f7830071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f782f7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:25:15.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f782f7fe640 1 monclient: found mon.noname-b 2024-01-24T23:25:15.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f782f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:15.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f782f7fe640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:25:15.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f782f7fe640 1 -- 172.21.15.115:0/2684175171 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f783006ebe0 con 0x7f78301305c0 2024-01-24T23:25:15.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f782f7fe640 10 monclient: _finish_auth 0 2024-01-24T23:25:15.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f782f7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:25:15.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f782f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:15.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f782f7fe640 1 -- 172.21.15.115:0/2684175171 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f78180017e0 con 0x7f78301305c0 2024-01-24T23:25:15.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f782f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f782f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:25:15.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f782f7fe640 10 monclient: dump: 2024-01-24T23:25:15.682 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.682 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.682 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.682 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.682 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.682 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.683 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.683 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.683 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.683 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.683 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.683 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.683 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.683 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.683 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.683 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f7837014640 5 monclient: authenticate success, global_id 85205 2024-01-24T23:25:15.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.670+0000 7f7837014640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:15.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782f7fe640 1 -- 172.21.15.115:0/2684175171 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7818001b00 con 0x7f78301305c0 2024-01-24T23:25:15.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782f7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:15.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:15.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782f7fe640 1 -- 172.21.15.115:0/2684175171 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7818004bf0 con 0x7f78301305c0 2024-01-24T23:25:15.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 4 set_mon_vals no callback set 2024-01-24T23:25:15.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:25:15.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782f7fe640 10 monclient: dump: 2024-01-24T23:25:15.684 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.684 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.684 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.684 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.684 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.684 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.684 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.685 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.685 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.685 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.685 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.685 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.685 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.685 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.685 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.685 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 10 monclient: shutdown 2024-01-24T23:25:15.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:15.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 1 -- 172.21.15.115:0/2684175171 >> v1:172.21.15.115:6790/0 conn(0x7f78301305c0 legacy=0x7f7830138910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 1 -- 172.21.15.115:0/2684175171 shutdown_connections 2024-01-24T23:25:15.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 1 -- 172.21.15.115:0/2684175171 >> 172.21.15.115:0/2684175171 conn(0x7f78300a5d40 msgr2=0x7f78300a2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:15.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 1 -- 172.21.15.115:0/2684175171 shutdown_connections 2024-01-24T23:25:15.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 1 -- 172.21.15.115:0/2684175171 wait complete. 2024-01-24T23:25:15.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 10 monclient: build_initial_monmap 2024-01-24T23:25:15.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 10 monclient: monmap: 2024-01-24T23:25:15.686 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:15.686 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:15.686 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:15.686 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:15.686 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:15.686 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:15.686 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:15.686 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:15.686 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:15.686 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:15.686 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:15.687 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:15.687 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:15.687 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:15.687 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:15.687 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 1 Processor -- start 2024-01-24T23:25:15.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 1 -- start start 2024-01-24T23:25:15.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 10 monclient: init 2024-01-24T23:25:15.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:15.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 10 monclient: _add_conns ranks=[7,0,2,3,5,8,1,4,6] 2024-01-24T23:25:15.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 10 monclient(hunting): picked mon.noname-i con 0x7f78300662f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:15.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 10 monclient(hunting): picked mon.noname-a con 0x7f78301305c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:15.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 10 monclient(hunting): picked mon.noname-b con 0x7f783013a030 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:15.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7830136b10 con 0x7f78301305c0 2024-01-24T23:25:15.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f783006f390 con 0x7f783013a030 2024-01-24T23:25:15.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f783006ee40 con 0x7f78300662f0 2024-01-24T23:25:15.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:15.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 10 monclient(hunting): authenticate will time out at 1194.598036s 2024-01-24T23:25:15.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782ffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f78301305c0 0x7f78301d0590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44104/0 (socket says 172.21.15.115:44104) 2024-01-24T23:25:15.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782ffff640 1 -- 172.21.15.115:0/2958655815 learned_addr learned my addr 172.21.15.115:0/2958655815 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:15.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 977026700 0 0) 0x7f783006f390 con 0x7f783013a030 2024-01-24T23:25:15.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient(hunting): my global_id is 85214 2024-01-24T23:25:15.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7810003530 con 0x7f783013a030 2024-01-24T23:25:15.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3970482638 0 0) 0x7f7830136b10 con 0x7f78301305c0 2024-01-24T23:25:15.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient(hunting): my global_id is 94140 2024-01-24T23:25:15.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f783006f390 con 0x7f78301305c0 2024-01-24T23:25:15.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1633431936 0 0) 0x7f7810003530 con 0x7f783013a030 2024-01-24T23:25:15.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7830136b10 con 0x7f783013a030 2024-01-24T23:25:15.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f78280017f0 con 0x7f783013a030 2024-01-24T23:25:15.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:25:15.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient(hunting): dump: 2024-01-24T23:25:15.689 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.689 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.689 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.689 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.689 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.689 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.690 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.690 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.690 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.690 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.690 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.690 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.690 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.690 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.690 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.690 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:15.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2820399872 0 0) 0x7f783006f390 con 0x7f78301305c0 2024-01-24T23:25:15.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7810003530 con 0x7f78301305c0 2024-01-24T23:25:15.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7818004b30 con 0x7f78301305c0 2024-01-24T23:25:15.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:15.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient(hunting): dump: 2024-01-24T23:25:15.691 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.691 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.691 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.691 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.691 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.691 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.691 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.691 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.691 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.691 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.691 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.691 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.691 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.692 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.692 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.692 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1244859398 0 0) 0x7f783006ee40 con 0x7f78300662f0 2024-01-24T23:25:15.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient(hunting): my global_id is 5812 2024-01-24T23:25:15.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f783006f390 con 0x7f78300662f0 2024-01-24T23:25:15.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2870287076 0 0) 0x7f7830136b10 con 0x7f783013a030 2024-01-24T23:25:15.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 >> v1:172.21.15.132:6792/0 conn(0x7f78300662f0 legacy=0x7f783012d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 >> v1:172.21.15.115:6789/0 conn(0x7f78301305c0 legacy=0x7f78301d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient: _finish_hunting 0 2024-01-24T23:25:15.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 1 monclient: found mon.b 2024-01-24T23:25:15.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:15.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:25:15.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7830138b20 con 0x7f783013a030 2024-01-24T23:25:15.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient: _finish_auth 0 2024-01-24T23:25:15.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:25:15.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:15.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 5 monclient: authenticate success, global_id 85214 2024-01-24T23:25:15.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 10 monclient: _renew_subs 2024-01-24T23:25:15.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:25:15.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 1 -- 172.21.15.115:0/2958655815 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7830138e00 con 0x7f783013a030 2024-01-24T23:25:15.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 10 monclient: _renew_subs 2024-01-24T23:25:15.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:25:15.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 1 -- 172.21.15.115:0/2958655815 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f783012d7e0 con 0x7f783013a030 2024-01-24T23:25:15.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7828003600 con 0x7f783013a030 2024-01-24T23:25:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7828003b30 con 0x7f783013a030 2024-01-24T23:25:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782effd640 4 set_mon_vals no callback set 2024-01-24T23:25:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:25:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 10 monclient: dump: 2024-01-24T23:25:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.695 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.695 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.695 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.695 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.695 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.695 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.695 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:15.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f7837014640 1 -- 172.21.15.115:0/2958655815 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f78140065d0 con 0x7f7814002870 2024-01-24T23:25:15.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f7828003c90 con 0x7f783013a030 2024-01-24T23:25:15.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.674+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(126..126 src has 1..126) v4 ==== 7321+0+0 (unknown 2448100607 0 0) 0x7f7828043190 con 0x7f783013a030 2024-01-24T23:25:15.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.678+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4219582856 0 0) 0x7f78140065d0 con 0x7f7814002870 2024-01-24T23:25:15.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.678+0000 7f782d7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.678+0000 7f782d7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.695 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.678+0000 7f782d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.678+0000 7f782d7fa640 10 monclient(hunting): my global_id is 94135 2024-01-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.678+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7830136b10 con 0x7f7814002870 2024-01-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.678+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3066537369 0 0) 0x7f7830136b10 con 0x7f7814002870 2024-01-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.678+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f78140065d0 con 0x7f7814002870 2024-01-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.678+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7820001c80 con 0x7f7814002870 2024-01-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.678+0000 7f782ffff640 10 monclient: get_auth_request con 0x7f78100377a0 auth_method 0 2024-01-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.678+0000 7f782d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.678+0000 7f782d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.678+0000 7f782d7fa640 10 monclient: dump: 2024-01-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.696 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.697 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.697 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.697 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.697 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.697 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.697 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.697 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.697 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.697 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.697 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.678+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3115948821 0 0) 0x7f78140065d0 con 0x7f7814002870 2024-01-24T23:25:15.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.678+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 --> v1:172.21.15.115:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f781400a120 con 0x7f7814002870 2024-01-24T23:25:15.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.678+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 <== mon.4 v1:172.21.15.115:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f7820003980 con 0x7f7814002870 2024-01-24T23:25:15.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.678+0000 7f782d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:15.697 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.678+0000 7f782d7fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:15.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.678+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 >> v1:172.21.15.115:6791/0 conn(0x7f7814002870 legacy=0x7f7814004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.698 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.678+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 >> v1:172.21.15.115:6791/0 conn(0x7f7814002870 legacy=0x7f7814004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:15.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f7837014640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:25:15.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f7837014640 1 -- 172.21.15.115:0/2958655815 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7814009e10 con 0x7f78140090d0 2024-01-24T23:25:15.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3287046040 0 0) 0x7f7814009e10 con 0x7f78140090d0 2024-01-24T23:25:15.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f782d7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f782d7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f782d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f782d7fa640 10 monclient(hunting): my global_id is 94144 2024-01-24T23:25:15.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78140065d0 con 0x7f78140090d0 2024-01-24T23:25:15.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1415612668 0 0) 0x7f78140065d0 con 0x7f78140090d0 2024-01-24T23:25:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7814009e10 con 0x7f78140090d0 2024-01-24T23:25:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f78200040a0 con 0x7f78140090d0 2024-01-24T23:25:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f782d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f782d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f782d7fa640 10 monclient: dump: 2024-01-24T23:25:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.701 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 151003110 0 0) 0x7f7814009e10 con 0x7f78140090d0 2024-01-24T23:25:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 --> v1:172.21.15.115:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f781400c310 con 0x7f78140090d0 2024-01-24T23:25:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 <== mon.4 v1:172.21.15.115:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 698123366) 0x7f7820006420 con 0x7f78140090d0 2024-01-24T23:25:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f782d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:25:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f782d7fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 >> v1:172.21.15.115:6791/0 conn(0x7f78140090d0 legacy=0x7f7814009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f782d7fa640 1 -- 172.21.15.115:0/2958655815 >> v1:172.21.15.115:6791/0 conn(0x7f78140090d0 legacy=0x7f7814009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:15.702 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:25:15.702 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:15.702 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:15.702 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:25:15.702 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 24, 2024-01-24T23:25:15.702 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:25:15.702 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:25:15.702 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:25:15.702 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:25:15.702 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:25:15.703 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:25:15.703 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:25:15.703 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:25:15.703 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:25:15.703 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:25:15.703 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:15.703 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:25:15.703 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:15.703 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:25:15.703 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:25:15.703 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:15.703 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:15.703 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:15.703 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:15.703 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:15.704 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:15.704 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:15.704 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:15.704 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:15.704 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:15.704 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:15.704 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:15.704 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:25:15.704 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:25:15.704 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:15.704 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:15.704 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:15.704 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:15.704 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:15.705 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:15.705 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:15.705 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:15.705 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:15.705 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:15.705 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:15.705 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.705 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:25:15.705 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:25:15.705 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:25:15.705 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:25:15.705 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:25:15.705 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:25:15.706 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:15.706 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:15.706 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:25:15.706 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:25:15.706 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:25:15.706 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:25:15.706 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:25:15.706 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:25:15.706 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:25:15.706 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:15.706 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:25:15.706 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:15.706 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:15.707 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:15.707 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:15.707 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:15.707 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:15.707 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:15.707 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:15.707 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:15.707 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:15.707 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:15.707 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:15.707 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:25:15.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.707 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:25:15.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.708 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:15.708 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:15.708 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.708 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.708 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.708 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:25:15.708 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.708 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.708 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.708 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:25:15.708 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:25:15.709 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.709 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.709 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.709 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:15.709 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:15.709 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.709 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.710 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.710 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:25:15.710 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.710 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.710 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:25:15.710 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:25:15.710 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.710 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.710 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.710 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:15.711 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:15.711 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.711 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.711 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.711 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:25:15.711 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.711 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.711 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.711 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:25:15.711 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:25:15.711 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.712 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.712 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.712 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:15.712 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:15.712 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.712 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.712 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.712 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:25:15.712 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.712 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.713 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.713 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:25:15.713 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:25:15.713 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.713 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.713 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.713 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:15.713 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:15.713 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.713 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.714 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.714 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:25:15.714 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.714 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.714 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.714 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:25:15.714 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:25:15.714 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.714 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.714 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.715 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:15.715 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:15.715 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.715 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.715 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.715 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:25:15.715 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.715 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.715 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.715 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:25:15.716 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:25:15.716 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.716 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.716 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.716 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:15.716 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:15.716 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.716 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.716 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.716 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:25:15.716 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.717 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.717 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.717 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:25:15.717 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:25:15.717 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.717 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.717 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.717 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:15.718 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:15.718 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.718 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.718 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.718 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:25:15.718 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.718 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.718 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.718 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:25:15.718 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:25:15.718 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.719 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.719 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.719 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:15.719 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:15.719 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.719 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.719 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.719 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:25:15.719 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.720 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.720 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:25:15.720 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:25:15.720 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.720 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.720 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.720 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.720 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:25:15.720 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:25:15.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.721 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:15.721 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:15.721 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:15.721 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.721 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:15.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:25:15.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.721 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:15.721 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:15.721 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:15.721 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.721 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.721 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:25:15.722 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:25:15.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f7837014640 1 -- 172.21.15.115:0/2958655815 >> v1:172.21.15.132:6812/1835401922 conn(0x7f78100377a0 legacy=0x7f7810039c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f7837014640 10 monclient: shutdown 2024-01-24T23:25:15.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f7837014640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:15.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f7837014640 1 -- 172.21.15.115:0/2958655815 >> v1:172.21.15.115:6790/0 conn(0x7f783013a030 legacy=0x7f78301d3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f78355b1640 1 -- 172.21.15.115:0/2958655815 reap_dead start 2024-01-24T23:25:15.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f78355b1640 1 -- 172.21.15.115:0/2958655815 reap_dead start 2024-01-24T23:25:15.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f7837014640 1 -- 172.21.15.115:0/2958655815 shutdown_connections 2024-01-24T23:25:15.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.682+0000 7f7837014640 1 -- 172.21.15.115:0/2958655815 >> 172.21.15.115:0/2958655815 conn(0x7f78300a5d40 msgr2=0x7f783006d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:15.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.686+0000 7f7837014640 1 -- 172.21.15.115:0/2958655815 shutdown_connections 2024-01-24T23:25:15.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.686+0000 7f7837014640 1 -- 172.21.15.115:0/2958655815 wait complete. 2024-01-24T23:25:15.724 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:25:15.724 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-01-24T23:25:15.724 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-01-24T23:25:15.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.834+0000 7fdb8f90f640 10 monclient: get_monmap_and_config 2024-01-24T23:25:15.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.834+0000 7fdb8f90f640 10 monclient: build_initial_monmap 2024-01-24T23:25:15.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.834+0000 7fdb8f90f640 10 monclient: monmap: 2024-01-24T23:25:15.838 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:15.838 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.838 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:15.838366+0000 2024-01-24T23:25:15.838 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:15.838366+0000 2024-01-24T23:25:15.838 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:15.838 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.838 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:15.838 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:15.838 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:15.838 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:15.838 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:15.838 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:15.838 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:15.838 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:15.839 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:15.839 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 1 Processor -- start 2024-01-24T23:25:15.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 1 -- start start 2024-01-24T23:25:15.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 10 monclient: init 2024-01-24T23:25:15.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:15.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 10 monclient: _add_conns ranks=[1,6,0,4,2,5,8,3,7] 2024-01-24T23:25:15.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 10 monclient(hunting): picked mon.noname-f con 0x7fdb88133bf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:15.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 10 monclient(hunting): picked mon.noname-d con 0x7fdb88133fd0 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:15.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 10 monclient(hunting): picked mon.noname-a con 0x7fdb88134430 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:15.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb8804ed70 con 0x7fdb88134430 2024-01-24T23:25:15.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb8809af30 con 0x7fdb88133bf0 2024-01-24T23:25:15.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb8809b210 con 0x7fdb88133fd0 2024-01-24T23:25:15.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:15.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 10 monclient(hunting): authenticate will time out at 1194.760622s 2024-01-24T23:25:15.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8ceaa640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fdb88133fd0 0x7fdb8813bdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:40418/0 (socket says 172.21.15.115:40418) 2024-01-24T23:25:15.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8ceaa640 1 -- 172.21.15.115:0/1738955301 learned_addr learned my addr 172.21.15.115:0/1738955301 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:15.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 1 -- 172.21.15.115:0/1738955301 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3208279342 0 0) 0x7fdb8804ed70 con 0x7fdb88134430 2024-01-24T23:25:15.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 10 monclient(hunting): my global_id is 94149 2024-01-24T23:25:15.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 1 -- 172.21.15.115:0/1738955301 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb74003530 con 0x7fdb88134430 2024-01-24T23:25:15.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 1 -- 172.21.15.115:0/1738955301 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1521324942 0 0) 0x7fdb8809b210 con 0x7fdb88133fd0 2024-01-24T23:25:15.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 10 monclient(hunting): my global_id is 5946 2024-01-24T23:25:15.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 1 -- 172.21.15.115:0/1738955301 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb8804ed70 con 0x7fdb88133fd0 2024-01-24T23:25:15.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 1 -- 172.21.15.115:0/1738955301 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3256738576 0 0) 0x7fdb8804ed70 con 0x7fdb88133fd0 2024-01-24T23:25:15.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 1 -- 172.21.15.115:0/1738955301 >> v1:172.21.15.132:6789/0 conn(0x7fdb88133bf0 legacy=0x7fdb8809bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 1 -- 172.21.15.115:0/1738955301 >> v1:172.21.15.115:6789/0 conn(0x7fdb88134430 legacy=0x7fdb8813f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 10 monclient: _finish_hunting 0 2024-01-24T23:25:15.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 1 monclient: found mon.noname-d 2024-01-24T23:25:15.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:15.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:25:15.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 1 -- 172.21.15.115:0/1738955301 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8809b4f0 con 0x7fdb88133fd0 2024-01-24T23:25:15.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 10 monclient: _finish_auth 0 2024-01-24T23:25:15.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 10 monclient: _check_auth_tickets 2024-01-24T23:25:15.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:15.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 1 -- 172.21.15.115:0/1738955301 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdb780017c0 con 0x7fdb88133fd0 2024-01-24T23:25:15.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 5 monclient: authenticate success, global_id 5946 2024-01-24T23:25:15.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:15.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:25:15.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 10 monclient: dump: 2024-01-24T23:25:15.849 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.849 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.849 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.849 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.849 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.849 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.849 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.849 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.849 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.849 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.849 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.849 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.849 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.850 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.850 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.850 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:15.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 1 -- 172.21.15.115:0/1738955301 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdb780019f0 con 0x7fdb88133fd0 2024-01-24T23:25:15.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:15.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 1 -- 172.21.15.115:0/1738955301 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdb78004bf0 con 0x7fdb88133fd0 2024-01-24T23:25:15.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:15.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 4 set_mon_vals no callback set 2024-01-24T23:25:15.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:15.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb7ffff640 10 monclient: dump: 2024-01-24T23:25:15.850 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.850 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.850 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.851 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.851 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.851 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.851 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.851 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.851 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.851 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.851 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.851 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.851 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.851 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.851 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.851 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 10 monclient: shutdown 2024-01-24T23:25:15.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:15.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 1 -- 172.21.15.115:0/1738955301 >> v1:172.21.15.115:6792/0 conn(0x7fdb88133fd0 legacy=0x7fdb8813bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 1 -- 172.21.15.115:0/1738955301 shutdown_connections 2024-01-24T23:25:15.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 1 -- 172.21.15.115:0/1738955301 >> 172.21.15.115:0/1738955301 conn(0x7fdb880a9910 msgr2=0x7fdb880a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:15.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 1 -- 172.21.15.115:0/1738955301 shutdown_connections 2024-01-24T23:25:15.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 1 -- 172.21.15.115:0/1738955301 wait complete. 2024-01-24T23:25:15.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 10 monclient: build_initial_monmap 2024-01-24T23:25:15.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 10 monclient: monmap: 2024-01-24T23:25:15.852 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:15.852 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:15.852 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:15.852 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:15.852 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:15.852 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:15.852 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:15.853 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:15.853 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:15.853 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:15.853 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:15.853 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:15.853 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:15.853 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:15.853 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:15.853 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 1 Processor -- start 2024-01-24T23:25:15.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 1 -- start start 2024-01-24T23:25:15.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 10 monclient: init 2024-01-24T23:25:15.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:15.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 10 monclient: _add_conns ranks=[5,1,3,8,2,7,4,0,6] 2024-01-24T23:25:15.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 10 monclient(hunting): picked mon.noname-h con 0x7fdb88133bf0 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:15.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 10 monclient(hunting): picked mon.noname-f con 0x7fdb88133fd0 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:15.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 10 monclient(hunting): picked mon.noname-g con 0x7fdb88134430 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:15.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb8813fb20 con 0x7fdb88133fd0 2024-01-24T23:25:15.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb8813a610 con 0x7fdb88134430 2024-01-24T23:25:15.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb8813a8f0 con 0x7fdb88133bf0 2024-01-24T23:25:15.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:15.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8f90f640 10 monclient(hunting): authenticate will time out at 1194.763504s 2024-01-24T23:25:15.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8ceaa640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fdb88133fd0 0x7fdb8813f410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.115:35038/0 (socket says 172.21.15.115:35038) 2024-01-24T23:25:15.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.838+0000 7fdb8ceaa640 1 -- 172.21.15.115:0/489733691 learned_addr learned my addr 172.21.15.115:0/489733691 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:15.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1959790388 0 0) 0x7fdb8813a610 con 0x7fdb88134430 2024-01-24T23:25:15.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): my global_id is 5844 2024-01-24T23:25:15.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb6c003530 con 0x7fdb88134430 2024-01-24T23:25:15.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3141134678 0 0) 0x7fdb8813fb20 con 0x7fdb88133fd0 2024-01-24T23:25:15.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): my global_id is 94132 2024-01-24T23:25:15.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb8813a610 con 0x7fdb88133fd0 2024-01-24T23:25:15.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3806649891 0 0) 0x7fdb8813a8f0 con 0x7fdb88133bf0 2024-01-24T23:25:15.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): my global_id is 5819 2024-01-24T23:25:15.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb8813fb20 con 0x7fdb88133bf0 2024-01-24T23:25:15.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1242085851 0 0) 0x7fdb8813fb20 con 0x7fdb88133bf0 2024-01-24T23:25:15.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdb8813a8f0 con 0x7fdb88133bf0 2024-01-24T23:25:15.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3353973728 0 0) 0x7fdb6c003530 con 0x7fdb88134430 2024-01-24T23:25:15.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdb8813fb20 con 0x7fdb88134430 2024-01-24T23:25:15.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdb80001d60 con 0x7fdb88134430 2024-01-24T23:25:15.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:25:15.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): dump: 2024-01-24T23:25:15.856 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.856 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.856 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.856 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.856 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.856 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.856 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.856 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.856 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.856 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.857 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.857 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.857 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.857 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.857 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.857 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:15.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdb84001a60 con 0x7fdb88133bf0 2024-01-24T23:25:15.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:15.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): dump: 2024-01-24T23:25:15.857 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.857 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.857 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.857 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.858 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.858 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.858 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.858 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.858 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.858 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.858 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.858 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.858 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.858 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.858 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.858 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3255459127 0 0) 0x7fdb8813a610 con 0x7fdb88133fd0 2024-01-24T23:25:15.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdb6c003530 con 0x7fdb88133fd0 2024-01-24T23:25:15.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdb780047f0 con 0x7fdb88133fd0 2024-01-24T23:25:15.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:25:15.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): dump: 2024-01-24T23:25:15.859 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.859 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.859 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.859 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.859 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.859 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.859 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.859 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.859 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.859 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.859 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.860 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.860 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.860 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.860 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.860 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3769016939 0 0) 0x7fdb8813a8f0 con 0x7fdb88133bf0 2024-01-24T23:25:15.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 >> v1:172.21.15.132:6790/0 conn(0x7fdb88134430 legacy=0x7fdb881d3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 >> v1:172.21.15.132:6789/0 conn(0x7fdb88133fd0 legacy=0x7fdb8813f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient: _finish_hunting 0 2024-01-24T23:25:15.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 monclient: found mon.h 2024-01-24T23:25:15.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:15.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:15.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8813abd0 con 0x7fdb88133bf0 2024-01-24T23:25:15.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient: _finish_auth 0 2024-01-24T23:25:15.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient: _check_auth_tickets 2024-01-24T23:25:15.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:15.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb8f90f640 5 monclient: authenticate success, global_id 5819 2024-01-24T23:25:15.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb8f90f640 10 monclient: _renew_subs 2024-01-24T23:25:15.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb8f90f640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:15.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb8f90f640 1 -- 172.21.15.115:0/489733691 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb8813aeb0 con 0x7fdb88133bf0 2024-01-24T23:25:15.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb8f90f640 10 monclient: _renew_subs 2024-01-24T23:25:15.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb8f90f640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:15.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb8f90f640 1 -- 172.21.15.115:0/489733691 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8809f810 con 0x7fdb88133bf0 2024-01-24T23:25:15.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdb84003600 con 0x7fdb88133bf0 2024-01-24T23:25:15.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:15.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7f7fe640 4 set_mon_vals no callback set 2024-01-24T23:25:15.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdb84003ab0 con 0x7fdb88133bf0 2024-01-24T23:25:15.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:15.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient: dump: 2024-01-24T23:25:15.862 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.862 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.862 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.862 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.862 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.862 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.862 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.862 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.862 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.862 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.863 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.863 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.863 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.863 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.863 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.863 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb8f90f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:15.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb8f90f640 1 -- 172.21.15.115:0/489733691 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdb680065d0 con 0x7fdb68002870 2024-01-24T23:25:15.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903800798 0 0) 0x7fdb680065d0 con 0x7fdb68002870 2024-01-24T23:25:15.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient(hunting): my global_id is 5955 2024-01-24T23:25:15.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb8813a8f0 con 0x7fdb68002870 2024-01-24T23:25:15.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3656623613 0 0) 0x7fdb8813a8f0 con 0x7fdb68002870 2024-01-24T23:25:15.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdb680065d0 con 0x7fdb68002870 2024-01-24T23:25:15.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdb78004c90 con 0x7fdb68002870 2024-01-24T23:25:15.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:15.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient: dump: 2024-01-24T23:25:15.864 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.864 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.864 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.864 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.864 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.864 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.864 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.864 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.865 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.865 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.865 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.865 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.865 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.865 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.865 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.865 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fdb84003c90 con 0x7fdb88133bf0 2024-01-24T23:25:15.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(126..126 src has 1..126) v4 ==== 7321+0+0 (unknown 2448100607 0 0) 0x7fdb84043260 con 0x7fdb88133bf0 2024-01-24T23:25:15.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2624808336 0 0) 0x7fdb680065d0 con 0x7fdb68002870 2024-01-24T23:25:15.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 --> v1:172.21.15.115:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb6800a120 con 0x7fdb68002870 2024-01-24T23:25:15.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb8deac640 10 monclient: get_auth_request con 0x7fdb6c037e20 auth_method 0 2024-01-24T23:25:15.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fdb78005420 con 0x7fdb68002870 2024-01-24T23:25:15.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:15.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:15.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 >> v1:172.21.15.115:6792/0 conn(0x7fdb68002870 legacy=0x7fdb68004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.842+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 >> v1:172.21.15.115:6792/0 conn(0x7fdb68002870 legacy=0x7fdb68004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:15.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb8f90f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:25:15.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb8f90f640 1 -- 172.21.15.115:0/489733691 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdb68009e20 con 0x7fdb680090e0 2024-01-24T23:25:15.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1299425738 0 0) 0x7fdb68009e20 con 0x7fdb680090e0 2024-01-24T23:25:15.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb7dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:15.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb7dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:15.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb7dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:15.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb7dffb640 10 monclient(hunting): my global_id is 5964 2024-01-24T23:25:15.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb680065d0 con 0x7fdb680090e0 2024-01-24T23:25:15.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1900407406 0 0) 0x7fdb680065d0 con 0x7fdb680090e0 2024-01-24T23:25:15.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdb68009e20 con 0x7fdb680090e0 2024-01-24T23:25:15.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdb78005b80 con 0x7fdb680090e0 2024-01-24T23:25:15.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb7dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:15.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb7dffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:15.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb7dffb640 10 monclient: dump: 2024-01-24T23:25:15.868 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:15.868 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:15.868 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.869 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:15.869 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:15.869 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:15.869 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:15.869 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:15.869 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:15.869 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:15.869 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:15.869 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:15.869 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:15.869 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:15.869 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:15.869 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:15.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3013357780 0 0) 0x7fdb68009e20 con 0x7fdb680090e0 2024-01-24T23:25:15.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 --> v1:172.21.15.115:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdb6800c310 con 0x7fdb680090e0 2024-01-24T23:25:15.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 889865017) 0x7fdb78005fe0 con 0x7fdb680090e0 2024-01-24T23:25:15.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb7dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:25:15.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb7dffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:15.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 >> v1:172.21.15.115:6792/0 conn(0x7fdb680090e0 legacy=0x7fdb68009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb7dffb640 1 -- 172.21.15.115:0/489733691 >> v1:172.21.15.115:6792/0 conn(0x7fdb680090e0 legacy=0x7fdb68009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:15.870 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:25:15.870 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:15.870 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:15.870 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:25:15.870 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 24, 2024-01-24T23:25:15.871 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:25:15.871 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:25:15.871 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:25:15.871 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:25:15.871 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:25:15.871 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:25:15.871 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:25:15.871 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:25:15.871 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:25:15.871 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:25:15.871 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:15.871 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:25:15.871 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:15.871 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:25:15.872 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:25:15.872 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:15.872 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:15.872 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:15.872 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:15.872 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:15.872 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:15.872 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:15.872 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:15.872 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:15.872 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:15.872 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:15.872 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:15.873 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:25:15.873 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:25:15.873 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:15.873 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:15.873 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:15.873 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:15.873 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:15.873 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:15.873 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:15.873 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:15.873 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:15.873 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:15.873 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:15.873 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.874 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:25:15.874 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:25:15.874 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:25:15.874 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:25:15.874 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:25:15.874 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:25:15.874 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:15.874 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:15.874 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:25:15.874 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:25:15.874 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:25:15.874 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:25:15.875 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:25:15.875 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:25:15.875 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:25:15.875 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:15.875 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:25:15.875 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:15.875 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:15.875 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:15.875 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:15.875 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:15.875 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:15.875 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:15.875 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:15.875 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:15.875 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:15.876 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:15.876 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:15.876 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:25:15.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.876 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:25:15.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.876 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:15.876 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:15.876 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.876 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.876 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.877 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:25:15.877 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.877 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.877 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.877 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:25:15.877 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:25:15.877 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.877 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.877 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.877 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:15.877 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:15.877 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.878 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.878 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.878 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:25:15.878 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.878 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.878 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.878 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:25:15.878 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:25:15.878 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.878 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.878 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.879 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:15.879 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:15.879 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.879 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.879 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.879 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:25:15.879 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.879 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.879 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.879 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:25:15.879 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:25:15.880 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.880 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.880 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.880 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:15.880 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:15.880 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.880 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.880 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.880 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:25:15.880 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.880 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.881 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.881 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:25:15.881 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:25:15.881 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.881 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.881 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.881 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:15.881 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:15.882 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.882 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.882 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.882 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:25:15.882 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.882 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.882 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.882 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:25:15.882 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:25:15.882 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.882 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.883 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.883 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:15.883 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:15.883 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.883 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.883 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.883 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:25:15.883 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.883 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.883 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.884 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:25:15.884 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:25:15.884 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.884 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.884 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.884 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:15.884 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:15.884 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.884 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.884 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.884 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:25:15.885 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.885 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.885 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.885 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:25:15.885 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:25:15.885 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.885 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.885 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.885 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:15.885 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:15.885 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.885 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.886 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.886 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:25:15.886 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.886 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.886 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.886 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:25:15.886 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:25:15.886 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.886 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.886 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.886 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:15.887 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:15.887 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:15.887 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:15.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.887 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:15.887 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:25:15.887 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:15.887 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.887 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.887 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:25:15.887 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:25:15.887 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:15.887 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:15.887 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:15.888 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.888 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.888 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:25:15.888 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:25:15.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.888 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:15.888 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:15.888 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:15.888 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.888 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:15.888 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:25:15.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:15.888 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:15.888 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:15.889 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:25:15.889 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:15.889 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:15.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:15.889 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:25:15.889 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:25:15.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb8f90f640 1 -- 172.21.15.115:0/489733691 >> v1:172.21.15.132:6812/1835401922 conn(0x7fdb6c037e20 legacy=0x7fdb6c03a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb8f90f640 10 monclient: shutdown 2024-01-24T23:25:15.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb8f90f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:15.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb8f90f640 1 -- 172.21.15.115:0/489733691 >> v1:172.21.15.132:6791/0 conn(0x7fdb88133bf0 legacy=0x7fdb881c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:15.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb8deac640 1 -- 172.21.15.115:0/489733691 reap_dead start 2024-01-24T23:25:15.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb8deac640 1 -- 172.21.15.115:0/489733691 reap_dead start 2024-01-24T23:25:15.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb8f90f640 1 -- 172.21.15.115:0/489733691 shutdown_connections 2024-01-24T23:25:15.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb8f90f640 1 -- 172.21.15.115:0/489733691 >> 172.21.15.115:0/489733691 conn(0x7fdb880a9910 msgr2=0x7fdb880a2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:15.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb8f90f640 1 -- 172.21.15.115:0/489733691 shutdown_connections 2024-01-24T23:25:15.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:15.850+0000 7fdb8f90f640 1 -- 172.21.15.115:0/489733691 wait complete. 2024-01-24T23:25:15.891 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:25:15.891 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-01-24T23:25:15.891 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-01-24T23:25:16.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.002+0000 7fc292013640 10 monclient: get_monmap_and_config 2024-01-24T23:25:16.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.002+0000 7fc292013640 10 monclient: build_initial_monmap 2024-01-24T23:25:16.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.002+0000 7fc292013640 10 monclient: monmap: 2024-01-24T23:25:16.006 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:16.006 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.006 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:16.006651+0000 2024-01-24T23:25:16.006 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:16.006651+0000 2024-01-24T23:25:16.006 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:16.006 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.006 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:16.006 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:16.007 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:16.007 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:16.007 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:16.007 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:16.007 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:16.007 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:16.007 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:16.007 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 1 Processor -- start 2024-01-24T23:25:16.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 1 -- start start 2024-01-24T23:25:16.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 10 monclient: init 2024-01-24T23:25:16.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:16.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 10 monclient: _add_conns ranks=[2,6,4,7,5,3,0,8,1] 2024-01-24T23:25:16.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 10 monclient(hunting): picked mon.noname-b con 0x7fc28c133c10 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:16.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 10 monclient(hunting): picked mon.noname-d con 0x7fc28c136960 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:16.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 10 monclient(hunting): picked mon.noname-c con 0x7fc28c13a470 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:16.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28c04ed70 con 0x7fc28c133c10 2024-01-24T23:25:16.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28c09dee0 con 0x7fc28c13a470 2024-01-24T23:25:16.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28c09d400 con 0x7fc28c136960 2024-01-24T23:25:16.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:16.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 10 monclient(hunting): authenticate will time out at 1194.929213s 2024-01-24T23:25:16.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28b7fe640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fc28c133c10 0x7fc28c133ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:60368/0 (socket says 172.21.15.115:60368) 2024-01-24T23:25:16.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28b7fe640 1 -- 172.21.15.115:0/415459814 learned_addr learned my addr 172.21.15.115:0/415459814 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:16.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28affd640 1 --1- 172.21.15.115:0/415459814 >> v1:172.21.15.115:6792/0 conn(0x7fc28c136960 0x7fc28c138d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:40458/0 (socket says 172.21.15.115:40458) 2024-01-24T23:25:16.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 1 -- 172.21.15.115:0/415459814 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3896153412 0 0) 0x7fc28c09d400 con 0x7fc28c136960 2024-01-24T23:25:16.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 10 monclient(hunting): my global_id is 5973 2024-01-24T23:25:16.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 1 -- 172.21.15.115:0/415459814 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc274003530 con 0x7fc28c136960 2024-01-24T23:25:16.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 1 -- 172.21.15.115:0/415459814 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3796898974 0 0) 0x7fc28c04ed70 con 0x7fc28c133c10 2024-01-24T23:25:16.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 10 monclient(hunting): my global_id is 85223 2024-01-24T23:25:16.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 1 -- 172.21.15.115:0/415459814 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc28c09d400 con 0x7fc28c133c10 2024-01-24T23:25:16.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 1 -- 172.21.15.115:0/415459814 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3463226769 0 0) 0x7fc28c09dee0 con 0x7fc28c13a470 2024-01-24T23:25:16.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 10 monclient(hunting): my global_id is 94153 2024-01-24T23:25:16.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 1 -- 172.21.15.115:0/415459814 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc28c04ed70 con 0x7fc28c13a470 2024-01-24T23:25:16.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 1 -- 172.21.15.115:0/415459814 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 505540128 0 0) 0x7fc28c09d400 con 0x7fc28c133c10 2024-01-24T23:25:16.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 1 -- 172.21.15.115:0/415459814 >> v1:172.21.15.115:6792/0 conn(0x7fc28c136960 legacy=0x7fc28c138d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 1 -- 172.21.15.115:0/415459814 >> v1:172.21.15.115:6791/0 conn(0x7fc28c13a470 legacy=0x7fc28c13c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:25:16.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 1 monclient: found mon.noname-b 2024-01-24T23:25:16.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:16.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:25:16.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 1 -- 172.21.15.115:0/415459814 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28c09f380 con 0x7fc28c133c10 2024-01-24T23:25:16.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 10 monclient: _finish_auth 0 2024-01-24T23:25:16.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:25:16.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:16.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 5 monclient: authenticate success, global_id 85223 2024-01-24T23:25:16.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 1 -- 172.21.15.115:0/415459814 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc2800016d0 con 0x7fc28c133c10 2024-01-24T23:25:16.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:16.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:25:16.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 10 monclient: dump: 2024-01-24T23:25:16.020 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.020 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.020 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.020 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.020 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.020 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.020 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.020 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.020 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.020 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.020 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.020 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.020 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.020 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.021 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.021 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:16.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 1 -- 172.21.15.115:0/415459814 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc280001970 con 0x7fc28c133c10 2024-01-24T23:25:16.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:16.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:16.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 1 -- 172.21.15.115:0/415459814 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc2800048b0 con 0x7fc28c133c10 2024-01-24T23:25:16.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 4 set_mon_vals no callback set 2024-01-24T23:25:16.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:25:16.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc28a7fc640 10 monclient: dump: 2024-01-24T23:25:16.022 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.022 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.022 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.023 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.023 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.023 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.023 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.023 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.023 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.023 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.024 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 10 monclient: shutdown 2024-01-24T23:25:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 1 -- 172.21.15.115:0/415459814 >> v1:172.21.15.115:6790/0 conn(0x7fc28c133c10 legacy=0x7fc28c133ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 1 -- 172.21.15.115:0/415459814 shutdown_connections 2024-01-24T23:25:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 1 -- 172.21.15.115:0/415459814 >> 172.21.15.115:0/415459814 conn(0x7fc28c0a5730 msgr2=0x7fc28c0a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 1 -- 172.21.15.115:0/415459814 shutdown_connections 2024-01-24T23:25:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 1 -- 172.21.15.115:0/415459814 wait complete. 2024-01-24T23:25:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 10 monclient: build_initial_monmap 2024-01-24T23:25:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 10 monclient: monmap: 2024-01-24T23:25:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:16.026 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:16.027 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 1 Processor -- start 2024-01-24T23:25:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 1 -- start start 2024-01-24T23:25:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.006+0000 7fc292013640 10 monclient: init 2024-01-24T23:25:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc292013640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc292013640 10 monclient: _add_conns ranks=[0,8,2,4,3,7,5,6,1] 2024-01-24T23:25:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc292013640 10 monclient(hunting): picked mon.noname-a con 0x7fc28c133c10 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc292013640 10 monclient(hunting): picked mon.noname-e con 0x7fc28c136960 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc292013640 10 monclient(hunting): picked mon.noname-b con 0x7fc28c13a470 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc292013640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28c12d7e0 con 0x7fc28c133c10 2024-01-24T23:25:16.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc292013640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28c11ba70 con 0x7fc28c13a470 2024-01-24T23:25:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc292013640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28c131830 con 0x7fc28c136960 2024-01-24T23:25:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc292013640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc292013640 10 monclient(hunting): authenticate will time out at 1194.932451s 2024-01-24T23:25:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc28bfff640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fc28c13a470 0x7fc28c1d3880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:60380/0 (socket says 172.21.15.115:60380) 2024-01-24T23:25:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc28bfff640 1 -- 172.21.15.115:0/3851918632 learned_addr learned my addr 172.21.15.115:0/3851918632 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2099093247 0 0) 0x7fc28c11ba70 con 0x7fc28c13a470 2024-01-24T23:25:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient(hunting): my global_id is 85232 2024-01-24T23:25:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc278003530 con 0x7fc28c13a470 2024-01-24T23:25:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3685965263 0 0) 0x7fc28c09f380 con 0x7fc28c133c10 2024-01-24T23:25:16.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient(hunting): my global_id is 94158 2024-01-24T23:25:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc28c11ba70 con 0x7fc28c133c10 2024-01-24T23:25:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1328358656 0 0) 0x7fc28c131830 con 0x7fc28c136960 2024-01-24T23:25:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient(hunting): my global_id is 85049 2024-01-24T23:25:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc28c09f380 con 0x7fc28c136960 2024-01-24T23:25:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3137358330 0 0) 0x7fc278003530 con 0x7fc28c13a470 2024-01-24T23:25:16.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc28c131830 con 0x7fc28c13a470 2024-01-24T23:25:16.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3035057597 0 0) 0x7fc28c11ba70 con 0x7fc28c133c10 2024-01-24T23:25:16.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc278003530 con 0x7fc28c133c10 2024-01-24T23:25:16.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc284001ce0 con 0x7fc28c13a470 2024-01-24T23:25:16.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:25:16.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient(hunting): dump: 2024-01-24T23:25:16.030 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.030 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.030 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.030 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.030 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.030 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.030 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.030 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.030 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.031 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.031 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.031 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.031 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.031 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.031 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.031 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:16.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc280001690 con 0x7fc28c133c10 2024-01-24T23:25:16.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:16.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient(hunting): dump: 2024-01-24T23:25:16.031 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.031 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.032 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.032 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.032 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.032 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.032 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.032 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.032 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.032 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.032 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.032 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.032 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.032 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.032 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.032 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1506025027 0 0) 0x7fc28c131830 con 0x7fc28c13a470 2024-01-24T23:25:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 >> v1:172.21.15.115:6793/0 conn(0x7fc28c136960 legacy=0x7fc28c131120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 >> v1:172.21.15.115:6789/0 conn(0x7fc28c133c10 legacy=0x7fc28c130a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient: _finish_hunting 0 2024-01-24T23:25:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 monclient: found mon.b 2024-01-24T23:25:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:25:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28c071c30 con 0x7fc28c13a470 2024-01-24T23:25:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient: _finish_auth 0 2024-01-24T23:25:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient: _check_auth_tickets 2024-01-24T23:25:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc292013640 5 monclient: authenticate success, global_id 85232 2024-01-24T23:25:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc292013640 10 monclient: _renew_subs 2024-01-24T23:25:16.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc292013640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:25:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc292013640 1 -- 172.21.15.115:0/3851918632 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc28c0a1310 con 0x7fc28c13a470 2024-01-24T23:25:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc284001e80 con 0x7fc28c13a470 2024-01-24T23:25:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc284003f90 con 0x7fc28c13a470 2024-01-24T23:25:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:25:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient: dump: 2024-01-24T23:25:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.034 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.035 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc292013640 10 monclient: _renew_subs 2024-01-24T23:25:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc292013640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:25:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc292013640 1 -- 172.21.15.115:0/3851918632 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc28c11b790 con 0x7fc28c13a470 2024-01-24T23:25:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc289ffb640 4 set_mon_vals no callback set 2024-01-24T23:25:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc292013640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:16.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc292013640 1 -- 172.21.15.115:0/3851918632 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc28c067c70 con 0x7fc28c11be90 2024-01-24T23:25:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fc2840040f0 con 0x7fc28c13a470 2024-01-24T23:25:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(126..126 src has 1..126) v4 ==== 7321+0+0 (unknown 2448100607 0 0) 0x7fc2840446b0 con 0x7fc28c13a470 2024-01-24T23:25:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239187612 0 0) 0x7fc28c067c70 con 0x7fc28c11be90 2024-01-24T23:25:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient(hunting): my global_id is 85058 2024-01-24T23:25:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc28c131830 con 0x7fc28c11be90 2024-01-24T23:25:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 87025528 0 0) 0x7fc28c131830 con 0x7fc28c11be90 2024-01-24T23:25:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc28c067c70 con 0x7fc28c11be90 2024-01-24T23:25:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc280001150 con 0x7fc28c11be90 2024-01-24T23:25:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient: dump: 2024-01-24T23:25:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.037 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.038 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc28affd640 10 monclient: get_auth_request con 0x7fc2780378c0 auth_method 0 2024-01-24T23:25:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1857080418 0 0) 0x7fc28c067c70 con 0x7fc28c11be90 2024-01-24T23:25:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 --> v1:172.21.15.115:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc28c09dc00 con 0x7fc28c11be90 2024-01-24T23:25:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fc2800070e0 con 0x7fc28c11be90 2024-01-24T23:25:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 >> v1:172.21.15.115:6793/0 conn(0x7fc28c11be90 legacy=0x7fc28c136d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.010+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 >> v1:172.21.15.115:6793/0 conn(0x7fc28c11be90 legacy=0x7fc28c136d40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:16.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc292013640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:25:16.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc292013640 1 -- 172.21.15.115:0/3851918632 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc28c0a1000 con 0x7fc28c02c7f0 2024-01-24T23:25:16.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2934838616 0 0) 0x7fc28c0a1000 con 0x7fc28c02c7f0 2024-01-24T23:25:16.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc273fff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc273fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc273fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc273fff640 10 monclient(hunting): my global_id is 85067 2024-01-24T23:25:16.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc28c067c70 con 0x7fc28c02c7f0 2024-01-24T23:25:16.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2327277161 0 0) 0x7fc28c067c70 con 0x7fc28c02c7f0 2024-01-24T23:25:16.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc28c0a1000 con 0x7fc28c02c7f0 2024-01-24T23:25:16.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc280005930 con 0x7fc28c02c7f0 2024-01-24T23:25:16.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc273fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc273fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:16.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc273fff640 10 monclient: dump: 2024-01-24T23:25:16.041 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.041 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.041 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.041 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.041 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.041 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.041 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.041 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.041 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.041 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.041 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.041 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.041 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.041 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.041 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.042 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3754752293 0 0) 0x7fc28c0a1000 con 0x7fc28c02c7f0 2024-01-24T23:25:16.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 --> v1:172.21.15.115:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc28c09bc50 con 0x7fc28c02c7f0 2024-01-24T23:25:16.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 2471803832) 0x7fc280006ef0 con 0x7fc28c02c7f0 2024-01-24T23:25:16.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc273fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:25:16.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc273fff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:16.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 >> v1:172.21.15.115:6793/0 conn(0x7fc28c02c7f0 legacy=0x7fc28c137450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc273fff640 1 -- 172.21.15.115:0/3851918632 >> v1:172.21.15.115:6793/0 conn(0x7fc28c02c7f0 legacy=0x7fc28c137450 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:16.042 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:25:16.042 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:25:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 24, 2024-01-24T23:25:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:25:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:25:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:25:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:25:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:25:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:25:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:25:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:25:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:25:16.043 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:25:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 1, 2024-01-24T23:25:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:25:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:25:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:16.044 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:16.045 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:16.045 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:16.045 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:16.045 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:16.045 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:25:16.045 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:25:16.045 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:16.045 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:16.045 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:16.045 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:16.045 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:25:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:25:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:25:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:25:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:25:16.046 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:25:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:25:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:25:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:25:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:25:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:25:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:25:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:25:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:25:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:16.047 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:25:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:25:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:16.048 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:25:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:25:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:25:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:25:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:25:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:25:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.050 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:25:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:25:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:25:16.051 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:25:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.052 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:25:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:25:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:25:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.053 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:25:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:25:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.054 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:25:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:25:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:25:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:16.055 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:25:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:25:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:25:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.056 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:25:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.057 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:25:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:25:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.058 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:25:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:25:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:25:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.059 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:25:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:25:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:25:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:16.060 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:25:16.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.061 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:16.061 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:16.061 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:25:16.061 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.061 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.061 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:25:16.061 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:25:16.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc292013640 1 -- 172.21.15.115:0/3851918632 >> v1:172.21.15.132:6812/1835401922 conn(0x7fc2780378c0 legacy=0x7fc278039d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc292013640 10 monclient: shutdown 2024-01-24T23:25:16.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc292013640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:16.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc292013640 1 -- 172.21.15.115:0/3851918632 >> v1:172.21.15.115:6790/0 conn(0x7fc28c13a470 legacy=0x7fc28c1d3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc28bfff640 1 -- 172.21.15.115:0/3851918632 reap_dead start 2024-01-24T23:25:16.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc28bfff640 1 -- 172.21.15.115:0/3851918632 reap_dead start 2024-01-24T23:25:16.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc292013640 1 -- 172.21.15.115:0/3851918632 shutdown_connections 2024-01-24T23:25:16.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc292013640 1 -- 172.21.15.115:0/3851918632 >> 172.21.15.115:0/3851918632 conn(0x7fc28c0a5730 msgr2=0x7fc28c0a40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:16.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc292013640 1 -- 172.21.15.115:0/3851918632 shutdown_connections 2024-01-24T23:25:16.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.018+0000 7fc292013640 1 -- 172.21.15.115:0/3851918632 wait complete. 2024-01-24T23:25:16.063 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:25:16.063 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-01-24T23:25:16.064 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-01-24T23:25:16.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:16.045+0000 7f96377fe640 1 -- 172.21.15.132:0/1215331924 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set rm", "pool": "fooset"}]=0 v127) v1 ==== 89+0+0 (unknown 2152272929 0 0) 0x7f96300148b0 con 0x7f961c011040 2024-01-24T23:25:16.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:16.045+0000 7f96377fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set rm", "pool": "fooset"}] 2024-01-24T23:25:16.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:16.045+0000 7f96377fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:16.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:16.161+0000 7f9645781640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm", "pool": "fooset"}] 2024-01-24T23:25:16.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:16.161+0000 7f9645781640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm", "pool": "fooset"}] 2024-01-24T23:25:16.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:16.161+0000 7f9645781640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:16.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:16.161+0000 7f9645781640 1 -- 172.21.15.132:0/1215331924 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush weight-set rm", "pool": "fooset"} v 0) v1 -- 0x7f9620002220 con 0x7f961c011040 2024-01-24T23:25:16.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.166+0000 7ff65b0c4640 10 monclient: get_monmap_and_config 2024-01-24T23:25:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.166+0000 7ff65b0c4640 10 monclient: build_initial_monmap 2024-01-24T23:25:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.166+0000 7ff65b0c4640 10 monclient: monmap: 2024-01-24T23:25:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:16.170638+0000 2024-01-24T23:25:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:16.170638+0000 2024-01-24T23:25:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:16.170 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:16.171 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:16.171 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:16.171 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 1 Processor -- start 2024-01-24T23:25:16.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 1 -- start start 2024-01-24T23:25:16.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 10 monclient: init 2024-01-24T23:25:16.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:16.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 10 monclient: _add_conns ranks=[7,1,0,3,2,5,4,8,6] 2024-01-24T23:25:16.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 10 monclient(hunting): picked mon.noname-i con 0x7ff654133bf0 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:16.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 10 monclient(hunting): picked mon.noname-f con 0x7ff654133fd0 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:16.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 10 monclient(hunting): picked mon.noname-a con 0x7ff6541343b0 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:16.178 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff65404ed70 con 0x7ff6541343b0 2024-01-24T23:25:16.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6540a4b60 con 0x7ff654133fd0 2024-01-24T23:25:16.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff65409ae50 con 0x7ff654133bf0 2024-01-24T23:25:16.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:16.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 10 monclient(hunting): authenticate will time out at 1195.093328s 2024-01-24T23:25:16.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff659661640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff6541343b0 0x7ff65413f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44136/0 (socket says 172.21.15.115:44136) 2024-01-24T23:25:16.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff659661640 1 -- 172.21.15.115:0/960908895 learned_addr learned my addr 172.21.15.115:0/960908895 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:16.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 1 -- 172.21.15.115:0/960908895 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3489872658 0 0) 0x7ff65404ed70 con 0x7ff6541343b0 2024-01-24T23:25:16.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 10 monclient(hunting): my global_id is 94167 2024-01-24T23:25:16.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 1 -- 172.21.15.115:0/960908895 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff63c003530 con 0x7ff6541343b0 2024-01-24T23:25:16.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 1 -- 172.21.15.115:0/960908895 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1781619368 0 0) 0x7ff63c003530 con 0x7ff6541343b0 2024-01-24T23:25:16.179 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 1 -- 172.21.15.115:0/960908895 >> v1:172.21.15.132:6792/0 conn(0x7ff654133bf0 legacy=0x7ff65409bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 1 -- 172.21.15.115:0/960908895 >> v1:172.21.15.132:6789/0 conn(0x7ff654133fd0 legacy=0x7ff65413bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 10 monclient: _finish_hunting 0 2024-01-24T23:25:16.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 1 monclient: found mon.noname-a 2024-01-24T23:25:16.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:16.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:25:16.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 1 -- 172.21.15.115:0/960908895 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65409b130 con 0x7ff6541343b0 2024-01-24T23:25:16.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 10 monclient: _finish_auth 0 2024-01-24T23:25:16.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 10 monclient: _check_auth_tickets 2024-01-24T23:25:16.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:16.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 1 -- 172.21.15.115:0/960908895 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff64c001b40 con 0x7ff6541343b0 2024-01-24T23:25:16.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 5 monclient: authenticate success, global_id 94167 2024-01-24T23:25:16.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:16.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:25:16.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 10 monclient: dump: 2024-01-24T23:25:16.181 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.181 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.181 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.181 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.181 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.181 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.181 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.181 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.181 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.181 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.181 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.181 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.181 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.181 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.182 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.182 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:16.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 1 -- 172.21.15.115:0/960908895 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff64c001df0 con 0x7ff6541343b0 2024-01-24T23:25:16.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:16.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:16.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 4 set_mon_vals no callback set 2024-01-24T23:25:16.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 1 -- 172.21.15.115:0/960908895 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff64c004f70 con 0x7ff6541343b0 2024-01-24T23:25:16.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:16.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff6537fe640 10 monclient: dump: 2024-01-24T23:25:16.182 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.182 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.182 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.182 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.183 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.183 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.183 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.183 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.183 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.183 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.183 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.183 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.183 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.183 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.183 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.183 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 10 monclient: shutdown 2024-01-24T23:25:16.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:16.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 1 -- 172.21.15.115:0/960908895 >> v1:172.21.15.115:6789/0 conn(0x7ff6541343b0 legacy=0x7ff65413f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 1 -- 172.21.15.115:0/960908895 shutdown_connections 2024-01-24T23:25:16.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 1 -- 172.21.15.115:0/960908895 >> 172.21.15.115:0/960908895 conn(0x7ff6540662f0 msgr2=0x7ff654066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:16.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 1 -- 172.21.15.115:0/960908895 shutdown_connections 2024-01-24T23:25:16.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 1 -- 172.21.15.115:0/960908895 wait complete. 2024-01-24T23:25:16.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 10 monclient: build_initial_monmap 2024-01-24T23:25:16.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 10 monclient: monmap: 2024-01-24T23:25:16.184 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:16.184 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:16.184 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:16.184 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:16.184 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:16.184 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:16.184 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:16.184 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:16.184 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:16.185 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:16.185 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:16.185 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:16.185 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:16.185 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:16.185 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:16.185 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 1 Processor -- start 2024-01-24T23:25:16.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 1 -- start start 2024-01-24T23:25:16.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.170+0000 7ff65b0c4640 10 monclient: init 2024-01-24T23:25:16.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff65b0c4640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:16.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff65b0c4640 10 monclient: _add_conns ranks=[3,5,6,7,4,0,2,1,8] 2024-01-24T23:25:16.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff65b0c4640 10 monclient(hunting): picked mon.noname-g con 0x7ff654133bf0 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:16.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff65b0c4640 10 monclient(hunting): picked mon.noname-h con 0x7ff654133fd0 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:16.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff65b0c4640 10 monclient(hunting): picked mon.noname-d con 0x7ff6541343b0 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:16.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff65b0c4640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6540a2c80 con 0x7ff654133bf0 2024-01-24T23:25:16.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff65b0c4640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff654120d60 con 0x7ff654133fd0 2024-01-24T23:25:16.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff65b0c4640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6541ccb50 con 0x7ff6541343b0 2024-01-24T23:25:16.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff65b0c4640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:16.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff65b0c4640 10 monclient(hunting): authenticate will time out at 1195.096110s 2024-01-24T23:25:16.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff659661640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7ff6541343b0 0x7ff6541d4000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:40462/0 (socket says 172.21.15.115:40462) 2024-01-24T23:25:16.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff659661640 1 -- 172.21.15.115:0/2289771223 learned_addr learned my addr 172.21.15.115:0/2289771223 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:16.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2799038422 0 0) 0x7ff6541ccb50 con 0x7ff6541343b0 2024-01-24T23:25:16.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient(hunting): my global_id is 5982 2024-01-24T23:25:16.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff640003530 con 0x7ff6541343b0 2024-01-24T23:25:16.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2822019036 0 0) 0x7ff6540a2c80 con 0x7ff654133bf0 2024-01-24T23:25:16.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient(hunting): my global_id is 5853 2024-01-24T23:25:16.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6541ccb50 con 0x7ff654133bf0 2024-01-24T23:25:16.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2940494761 0 0) 0x7ff640003530 con 0x7ff6541343b0 2024-01-24T23:25:16.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff6540a2c80 con 0x7ff6541343b0 2024-01-24T23:25:16.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2745232343 0 0) 0x7ff654120d60 con 0x7ff654133fd0 2024-01-24T23:25:16.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient(hunting): my global_id is 5828 2024-01-24T23:25:16.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff640003530 con 0x7ff654133fd0 2024-01-24T23:25:16.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff64c001b00 con 0x7ff6541343b0 2024-01-24T23:25:16.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:25:16.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient(hunting): dump: 2024-01-24T23:25:16.188 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.188 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.188 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.188 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.188 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.188 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.188 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.188 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.188 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.188 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.188 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.188 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.188 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.189 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.189 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.189 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:16.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4141422968 0 0) 0x7ff6540a2c80 con 0x7ff6541343b0 2024-01-24T23:25:16.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 >> v1:172.21.15.132:6791/0 conn(0x7ff654133fd0 legacy=0x7ff6541cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 >> v1:172.21.15.132:6790/0 conn(0x7ff654133bf0 legacy=0x7ff6541cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient: _finish_hunting 0 2024-01-24T23:25:16.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 monclient: found mon.d 2024-01-24T23:25:16.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:16.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:25:16.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65409c810 con 0x7ff6541343b0 2024-01-24T23:25:16.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient: _finish_auth 0 2024-01-24T23:25:16.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient: _check_auth_tickets 2024-01-24T23:25:16.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:16.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff65b0c4640 5 monclient: authenticate success, global_id 5982 2024-01-24T23:25:16.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff65b0c4640 10 monclient: _renew_subs 2024-01-24T23:25:16.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff65b0c4640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:25:16.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff65b0c4640 1 -- 172.21.15.115:0/2289771223 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6540a5990 con 0x7ff6541343b0 2024-01-24T23:25:16.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff65b0c4640 10 monclient: _renew_subs 2024-01-24T23:25:16.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff65b0c4640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:25:16.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff65b0c4640 1 -- 172.21.15.115:0/2289771223 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff65411f9c0 con 0x7ff6541343b0 2024-01-24T23:25:16.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff64c005240 con 0x7ff6541343b0 2024-01-24T23:25:16.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:16.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff64c0056f0 con 0x7ff6541343b0 2024-01-24T23:25:16.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff652ffd640 4 set_mon_vals no callback set 2024-01-24T23:25:16.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:16.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient: dump: 2024-01-24T23:25:16.190 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.191 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.191 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.191 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.191 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.191 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.191 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.191 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.191 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.191 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.191 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.191 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.191 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.191 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.191 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.191 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff65b0c4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:16.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff65b0c4640 1 -- 172.21.15.115:0/2289771223 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff6340065d0 con 0x7ff634002870 2024-01-24T23:25:16.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7ff64c010170 con 0x7ff6541343b0 2024-01-24T23:25:16.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(127..127 src has 1..127) v4 ==== 6989+0+0 (unknown 755296415 0 0) 0x7ff64c0449f0 con 0x7ff6541343b0 2024-01-24T23:25:16.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1202882420 0 0) 0x7ff6340065d0 con 0x7ff634002870 2024-01-24T23:25:16.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient(hunting): my global_id is 94150 2024-01-24T23:25:16.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6540a2c80 con 0x7ff634002870 2024-01-24T23:25:16.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff653fff640 10 monclient: get_auth_request con 0x7ff640037230 auth_method 0 2024-01-24T23:25:16.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4242784546 0 0) 0x7ff6540a2c80 con 0x7ff634002870 2024-01-24T23:25:16.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff6340065d0 con 0x7ff634002870 2024-01-24T23:25:16.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff644001910 con 0x7ff634002870 2024-01-24T23:25:16.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:25:16.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient: dump: 2024-01-24T23:25:16.193 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.193 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.193 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.193 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.193 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.193 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.193 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.193 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.193 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.194 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.194 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.194 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.194 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.194 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.194 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.194 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3960684072 0 0) 0x7ff6340065d0 con 0x7ff634002870 2024-01-24T23:25:16.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 --> v1:172.21.15.132:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff63400a120 con 0x7ff634002870 2024-01-24T23:25:16.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 <== mon.1 v1:172.21.15.132:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7ff644003620 con 0x7ff634002870 2024-01-24T23:25:16.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:16.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:16.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 >> v1:172.21.15.132:6789/0 conn(0x7ff634002870 legacy=0x7ff634004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.174+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 >> v1:172.21.15.132:6789/0 conn(0x7ff634002870 legacy=0x7ff634004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:16.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff65b0c4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:25:16.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff65b0c4640 1 -- 172.21.15.115:0/2289771223 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff634009df0 con 0x7ff6340090b0 2024-01-24T23:25:16.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1561010272 0 0) 0x7ff634009df0 con 0x7ff6340090b0 2024-01-24T23:25:16.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff6517fa640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff6517fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff6517fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff6517fa640 10 monclient(hunting): my global_id is 94159 2024-01-24T23:25:16.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6340065d0 con 0x7ff6340090b0 2024-01-24T23:25:16.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2483287514 0 0) 0x7ff6340065d0 con 0x7ff6340090b0 2024-01-24T23:25:16.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff634009df0 con 0x7ff6340090b0 2024-01-24T23:25:16.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff644003d40 con 0x7ff6340090b0 2024-01-24T23:25:16.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff6517fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff6517fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:25:16.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff6517fa640 10 monclient: dump: 2024-01-24T23:25:16.197 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.197 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.197 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.197 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.197 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.198 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.198 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.198 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.198 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.198 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.198 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.198 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.198 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.198 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.198 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.198 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2668239238 0 0) 0x7ff634009df0 con 0x7ff6340090b0 2024-01-24T23:25:16.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 --> v1:172.21.15.132:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff63400c310 con 0x7ff6340090b0 2024-01-24T23:25:16.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 <== mon.1 v1:172.21.15.132:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 850311808) 0x7ff6440040c0 con 0x7ff6340090b0 2024-01-24T23:25:16.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff6517fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:25:16.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff6517fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:16.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 >> v1:172.21.15.132:6789/0 conn(0x7ff6340090b0 legacy=0x7ff634009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff6517fa640 1 -- 172.21.15.115:0/2289771223 >> v1:172.21.15.132:6789/0 conn(0x7ff6340090b0 legacy=0x7ff634009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:16.199 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:25:16.199 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:16.199 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:16.199 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:25:16.199 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 24, 2024-01-24T23:25:16.199 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:25:16.200 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:25:16.200 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:25:16.200 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:25:16.200 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:25:16.200 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:25:16.200 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:25:16.200 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:25:16.200 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:25:16.200 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:25:16.200 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:16.200 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 1, 2024-01-24T23:25:16.200 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:16.200 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:25:16.200 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:25:16.201 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:16.201 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:16.201 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:16.201 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:16.201 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:16.201 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:16.201 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:16.201 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:16.201 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:16.201 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:16.201 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:16.201 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:16.201 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:25:16.201 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:25:16.201 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:16.202 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:16.202 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:16.202 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:16.202 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:16.202 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:16.202 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:16.202 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:16.202 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:16.202 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:16.202 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:16.202 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.202 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:25:16.202 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:25:16.202 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:25:16.202 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:25:16.202 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:25:16.203 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:25:16.203 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:16.203 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:16.203 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:25:16.203 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:25:16.203 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:25:16.203 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:25:16.203 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:25:16.203 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:25:16.203 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:25:16.203 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:16.203 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:25:16.203 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:16.203 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:16.203 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:16.204 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:16.204 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:16.204 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:16.204 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:16.204 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:16.204 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:16.204 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:16.204 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:16.204 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:16.204 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:25:16.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.204 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:25:16.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.204 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:16.204 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:16.205 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.205 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.205 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.205 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:25:16.205 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.205 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.205 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.205 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:25:16.205 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:25:16.205 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.205 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.205 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.206 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:16.206 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:16.206 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.206 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.206 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.206 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:25:16.206 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.206 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.206 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.206 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:25:16.206 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:25:16.207 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.207 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.207 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.207 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:16.207 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:16.207 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.207 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.207 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.207 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:25:16.207 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.207 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.208 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.208 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:25:16.208 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:25:16.208 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.208 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.208 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.208 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:16.208 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:16.208 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.208 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.209 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.209 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:25:16.209 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.209 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.209 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.209 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:25:16.209 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:25:16.209 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.209 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.209 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.210 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:16.210 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:16.210 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.210 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.210 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.210 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:25:16.210 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.210 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.210 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.210 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:25:16.211 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:25:16.211 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.211 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.211 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.211 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:16.211 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:16.211 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.211 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.211 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.211 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:25:16.211 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.212 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.212 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.212 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:25:16.212 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:25:16.212 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.212 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.212 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.212 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:16.212 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:16.213 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.213 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.213 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.213 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:25:16.213 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.213 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.213 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:25:16.213 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:25:16.213 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.213 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.213 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.214 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:16.214 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:16.214 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.214 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.214 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.214 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:25:16.214 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.214 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.214 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.214 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:25:16.214 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:25:16.215 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.215 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.215 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.215 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:16.215 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:16.215 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.215 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.215 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.215 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:25:16.215 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.215 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.215 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.216 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:25:16.216 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:25:16.216 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.216 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.216 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.216 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.216 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.216 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:25:16.216 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:25:16.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.216 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:16.216 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:16.217 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:16.217 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.217 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:16.217 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:25:16.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.217 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:16.217 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:16.217 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:16.217 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.217 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.217 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:25:16.217 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:25:16.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff65b0c4640 1 -- 172.21.15.115:0/2289771223 >> v1:172.21.15.132:6812/1835401922 conn(0x7ff640037230 legacy=0x7ff6400396f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff65b0c4640 10 monclient: shutdown 2024-01-24T23:25:16.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff65b0c4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:16.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff65b0c4640 1 -- 172.21.15.115:0/2289771223 >> v1:172.21.15.115:6792/0 conn(0x7ff6541343b0 legacy=0x7ff6541d4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff659661640 1 -- 172.21.15.115:0/2289771223 reap_dead start 2024-01-24T23:25:16.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff659661640 1 -- 172.21.15.115:0/2289771223 reap_dead start 2024-01-24T23:25:16.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff65b0c4640 1 -- 172.21.15.115:0/2289771223 shutdown_connections 2024-01-24T23:25:16.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff65b0c4640 1 -- 172.21.15.115:0/2289771223 >> 172.21.15.115:0/2289771223 conn(0x7ff6540662f0 msgr2=0x7ff6540a84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:16.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff65b0c4640 1 -- 172.21.15.115:0/2289771223 shutdown_connections 2024-01-24T23:25:16.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.182+0000 7ff65b0c4640 1 -- 172.21.15.115:0/2289771223 wait complete. 2024-01-24T23:25:16.219 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:25:16.219 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-01-24T23:25:16.220 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-01-24T23:25:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.318+0000 7f37aa78e640 10 monclient: get_monmap_and_config 2024-01-24T23:25:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.318+0000 7f37aa78e640 10 monclient: build_initial_monmap 2024-01-24T23:25:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.318+0000 7f37aa78e640 10 monclient: monmap: 2024-01-24T23:25:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:16.322249+0000 2024-01-24T23:25:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:16.322249+0000 2024-01-24T23:25:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:16.323 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 1 Processor -- start 2024-01-24T23:25:16.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 1 -- start start 2024-01-24T23:25:16.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 10 monclient: init 2024-01-24T23:25:16.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:16.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 10 monclient: _add_conns ranks=[2,0,4,5,7,6,1,8,3] 2024-01-24T23:25:16.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 10 monclient(hunting): picked mon.noname-b con 0x7f37a4133c10 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:16.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 10 monclient(hunting): picked mon.noname-a con 0x7f37a4136960 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:16.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 10 monclient(hunting): picked mon.noname-c con 0x7f37a413a470 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:16.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a404ed70 con 0x7f37a4136960 2024-01-24T23:25:16.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a409dee0 con 0x7f37a4133c10 2024-01-24T23:25:16.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a409d400 con 0x7f37a413a470 2024-01-24T23:25:16.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:16.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 10 monclient(hunting): authenticate will time out at 1195.244868s 2024-01-24T23:25:16.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a3fff640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f37a4133c10 0x7f37a4133ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:60382/0 (socket says 172.21.15.115:60382) 2024-01-24T23:25:16.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a3fff640 1 -- 172.21.15.115:0/1028342249 learned_addr learned my addr 172.21.15.115:0/1028342249 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:16.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 1 -- 172.21.15.115:0/1028342249 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1946033099 0 0) 0x7f37a409d400 con 0x7f37a413a470 2024-01-24T23:25:16.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 10 monclient(hunting): my global_id is 94162 2024-01-24T23:25:16.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 1 -- 172.21.15.115:0/1028342249 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3784003530 con 0x7f37a413a470 2024-01-24T23:25:16.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 1 -- 172.21.15.115:0/1028342249 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3035763169 0 0) 0x7f37a409dee0 con 0x7f37a4133c10 2024-01-24T23:25:16.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 10 monclient(hunting): my global_id is 85241 2024-01-24T23:25:16.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 1 -- 172.21.15.115:0/1028342249 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37a409d400 con 0x7f37a4133c10 2024-01-24T23:25:16.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 1 -- 172.21.15.115:0/1028342249 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 527347343 0 0) 0x7f37a404ed70 con 0x7f37a4136960 2024-01-24T23:25:16.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 10 monclient(hunting): my global_id is 94176 2024-01-24T23:25:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 1 -- 172.21.15.115:0/1028342249 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37a409dee0 con 0x7f37a4136960 2024-01-24T23:25:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 1 -- 172.21.15.115:0/1028342249 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4080169922 0 0) 0x7f3784003530 con 0x7f37a413a470 2024-01-24T23:25:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 1 -- 172.21.15.115:0/1028342249 >> v1:172.21.15.115:6790/0 conn(0x7f37a4133c10 legacy=0x7f37a4133ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 1 -- 172.21.15.115:0/1028342249 >> v1:172.21.15.115:6789/0 conn(0x7f37a4136960 legacy=0x7f37a4138d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:25:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 1 monclient: found mon.noname-c 2024-01-24T23:25:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:25:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 1 -- 172.21.15.115:0/1028342249 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a409f380 con 0x7f37a413a470 2024-01-24T23:25:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 10 monclient: _finish_auth 0 2024-01-24T23:25:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 1 -- 172.21.15.115:0/1028342249 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f37940017c0 con 0x7f37a413a470 2024-01-24T23:25:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:25:16.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 10 monclient: dump: 2024-01-24T23:25:16.334 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.334 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.334 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.334 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.334 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.334 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.334 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.334 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.334 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.334 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.334 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.334 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.335 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.335 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.335 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.335 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 5 monclient: authenticate success, global_id 94162 2024-01-24T23:25:16.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:16.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 1 -- 172.21.15.115:0/1028342249 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f37940019e0 con 0x7f37a413a470 2024-01-24T23:25:16.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:16.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 1 -- 172.21.15.115:0/1028342249 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3794003960 con 0x7f37a413a470 2024-01-24T23:25:16.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:16.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 4 set_mon_vals no callback set 2024-01-24T23:25:16.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:16.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37a2ffd640 10 monclient: dump: 2024-01-24T23:25:16.336 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.336 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.336 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.336 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.336 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.336 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.336 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.336 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.336 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.336 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.336 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.336 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.336 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.336 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.336 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.337 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 10 monclient: shutdown 2024-01-24T23:25:16.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:16.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 1 -- 172.21.15.115:0/1028342249 >> v1:172.21.15.115:6791/0 conn(0x7f37a413a470 legacy=0x7f37a413c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 1 -- 172.21.15.115:0/1028342249 shutdown_connections 2024-01-24T23:25:16.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 1 -- 172.21.15.115:0/1028342249 >> 172.21.15.115:0/1028342249 conn(0x7f37a40a5730 msgr2=0x7f37a40a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:16.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 1 -- 172.21.15.115:0/1028342249 shutdown_connections 2024-01-24T23:25:16.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 1 -- 172.21.15.115:0/1028342249 wait complete. 2024-01-24T23:25:16.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 10 monclient: build_initial_monmap 2024-01-24T23:25:16.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 10 monclient: monmap: 2024-01-24T23:25:16.337 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:16.337 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:16.337 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:16.337 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:16.337 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:16.337 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:16.338 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 1 Processor -- start 2024-01-24T23:25:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 1 -- start start 2024-01-24T23:25:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.322+0000 7f37aa78e640 10 monclient: init 2024-01-24T23:25:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37aa78e640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37aa78e640 10 monclient: _add_conns ranks=[7,2,4,3,0,5,6,8,1] 2024-01-24T23:25:16.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37aa78e640 10 monclient(hunting): picked mon.noname-i con 0x7f37a4133c10 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37aa78e640 10 monclient(hunting): picked mon.noname-b con 0x7f37a4136960 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37aa78e640 10 monclient(hunting): picked mon.noname-c con 0x7f37a413a470 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37aa78e640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a406dc10 con 0x7f37a4136960 2024-01-24T23:25:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37aa78e640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a41c7a20 con 0x7f37a413a470 2024-01-24T23:25:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37aa78e640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a406d370 con 0x7f37a4133c10 2024-01-24T23:25:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37aa78e640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37aa78e640 10 monclient(hunting): authenticate will time out at 1195.248158s 2024-01-24T23:25:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a8d2b640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f37a413a470 0x7f37a41ceff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:44648/0 (socket says 172.21.15.115:44648) 2024-01-24T23:25:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a37fe640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f37a4136960 0x7f37a4071160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:60390/0 (socket says 172.21.15.115:60390) 2024-01-24T23:25:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a8d2b640 1 -- 172.21.15.115:0/3397237936 learned_addr learned my addr 172.21.15.115:0/3397237936 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2375423080 0 0) 0x7f37a41c7a20 con 0x7f37a413a470 2024-01-24T23:25:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient(hunting): my global_id is 94171 2024-01-24T23:25:16.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3788003530 con 0x7f37a413a470 2024-01-24T23:25:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2333304124 0 0) 0x7f37a406dc10 con 0x7f37a4136960 2024-01-24T23:25:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient(hunting): my global_id is 85250 2024-01-24T23:25:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37a41c7a20 con 0x7f37a4136960 2024-01-24T23:25:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2932567537 0 0) 0x7f3788003530 con 0x7f37a413a470 2024-01-24T23:25:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f37a406dc10 con 0x7f37a413a470 2024-01-24T23:25:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3794001380 con 0x7f37a413a470 2024-01-24T23:25:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:25:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient(hunting): dump: 2024-01-24T23:25:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.340 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.341 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1665381637 0 0) 0x7f37a41c7a20 con 0x7f37a4136960 2024-01-24T23:25:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3788003530 con 0x7f37a4136960 2024-01-24T23:25:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1058250366 0 0) 0x7f37a406d370 con 0x7f37a4133c10 2024-01-24T23:25:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient(hunting): my global_id is 5830 2024-01-24T23:25:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37a41c7a20 con 0x7f37a4133c10 2024-01-24T23:25:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f379c001a00 con 0x7f37a4136960 2024-01-24T23:25:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:25:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient(hunting): dump: 2024-01-24T23:25:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.342 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.343 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4155465689 0 0) 0x7f37a406dc10 con 0x7f37a413a470 2024-01-24T23:25:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 >> v1:172.21.15.132:6792/0 conn(0x7f37a4133c10 legacy=0x7f37a41c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 >> v1:172.21.15.115:6790/0 conn(0x7f37a4136960 legacy=0x7f37a4071160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:25:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 monclient: found mon.c 2024-01-24T23:25:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a406d650 con 0x7f37a413a470 2024-01-24T23:25:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient: _finish_auth 0 2024-01-24T23:25:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:25:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37aa78e640 5 monclient: authenticate success, global_id 94171 2024-01-24T23:25:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37aa78e640 10 monclient: _renew_subs 2024-01-24T23:25:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37aa78e640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:16.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37aa78e640 1 -- 172.21.15.115:0/3397237936 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a406d930 con 0x7f37a413a470 2024-01-24T23:25:16.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37aa78e640 10 monclient: _renew_subs 2024-01-24T23:25:16.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37aa78e640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:16.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37aa78e640 1 -- 172.21.15.115:0/3397237936 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a4071870 con 0x7f37a413a470 2024-01-24T23:25:16.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3794003b90 con 0x7f37a413a470 2024-01-24T23:25:16.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:16.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f37940040c0 con 0x7f37a413a470 2024-01-24T23:25:16.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a27fc640 4 set_mon_vals no callback set 2024-01-24T23:25:16.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:16.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient: dump: 2024-01-24T23:25:16.345 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.345 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.345 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.345 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.345 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.346 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.346 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.346 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.346 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.346 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.346 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.346 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.346 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.346 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.346 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.346 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37aa78e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:16.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37aa78e640 1 -- 172.21.15.115:0/3397237936 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f377c0065d0 con 0x7f377c002870 2024-01-24T23:25:16.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f37940042a0 con 0x7f37a413a470 2024-01-24T23:25:16.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(127..127 src has 1..127) v4 ==== 6989+0+0 (unknown 755296415 0 0) 0x7f37940445e0 con 0x7f37a413a470 2024-01-24T23:25:16.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a37fe640 10 monclient: get_auth_request con 0x7f37880377b0 auth_method 0 2024-01-24T23:25:16.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2699244300 0 0) 0x7f377c0065d0 con 0x7f377c002870 2024-01-24T23:25:16.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient(hunting): my global_id is 5862 2024-01-24T23:25:16.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37a406dc10 con 0x7f377c002870 2024-01-24T23:25:16.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2217285338 0 0) 0x7f37a406dc10 con 0x7f377c002870 2024-01-24T23:25:16.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f377c0065d0 con 0x7f377c002870 2024-01-24T23:25:16.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f378c0018f0 con 0x7f377c002870 2024-01-24T23:25:16.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:16.347 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 10 monclient: dump: 2024-01-24T23:25:16.347 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.347 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.347 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.348 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.348 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.348 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.348 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.348 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.348 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.348 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.348 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.348 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.348 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.348 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.348 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.348 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3501086963 0 0) 0x7f377c0065d0 con 0x7f377c002870 2024-01-24T23:25:16.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.326+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 --> v1:172.21.15.132:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f377c00a120 con 0x7f377c002870 2024-01-24T23:25:16.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.330+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f378c0058e0 con 0x7f377c002870 2024-01-24T23:25:16.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.330+0000 7f37a0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:16.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.330+0000 7f37a0ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:16.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.330+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 >> v1:172.21.15.132:6790/0 conn(0x7f377c002870 legacy=0x7f377c004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.330+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 >> v1:172.21.15.132:6790/0 conn(0x7f377c002870 legacy=0x7f377c004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.334+0000 7f37aa78e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:25:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.334+0000 7f37aa78e640 1 -- 172.21.15.115:0/3397237936 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f377c009e10 con 0x7f377c0090d0 2024-01-24T23:25:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.334+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3812037397 0 0) 0x7f377c009e10 con 0x7f377c0090d0 2024-01-24T23:25:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.334+0000 7f37a0ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.334+0000 7f37a0ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.334+0000 7f37a0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.334+0000 7f37a0ff9640 10 monclient(hunting): my global_id is 5871 2024-01-24T23:25:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.334+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f377c0065d0 con 0x7f377c0090d0 2024-01-24T23:25:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.334+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3825378625 0 0) 0x7f377c0065d0 con 0x7f377c0090d0 2024-01-24T23:25:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.334+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f377c009e10 con 0x7f377c0090d0 2024-01-24T23:25:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.334+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f378c004e20 con 0x7f377c0090d0 2024-01-24T23:25:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.334+0000 7f37a0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.334+0000 7f37a0ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.334+0000 7f37a0ff9640 10 monclient: dump: 2024-01-24T23:25:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.351 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.352 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.334+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3133163837 0 0) 0x7f377c009e10 con 0x7f377c0090d0 2024-01-24T23:25:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.334+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 --> v1:172.21.15.132:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f377c00c310 con 0x7f377c0090d0 2024-01-24T23:25:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.338+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7545 (unknown 0 0 207978710) 0x7f378c005aa0 con 0x7f377c0090d0 2024-01-24T23:25:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.338+0000 7f37a0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:25:16.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.338+0000 7f37a0ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:16.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.338+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 >> v1:172.21.15.132:6790/0 conn(0x7f377c0090d0 legacy=0x7f377c009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.338+0000 7f37a0ff9640 1 -- 172.21.15.115:0/3397237936 >> v1:172.21.15.132:6790/0 conn(0x7f377c0090d0 legacy=0x7f377c009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:16.353 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:25:16.353 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:16.353 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:16.353 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "leader", 2024-01-24T23:25:16.353 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 24, 2024-01-24T23:25:16.353 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:25:16.353 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:25:16.353 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:25:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:25:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:25:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:25:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:25:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:25:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:25:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:25:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 1, 2024-01-24T23:25:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:25:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:25:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:25:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:25:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:25:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:25:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:25:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:25:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:25:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:25:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:25:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:25:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:25:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:25:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:25:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:25:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:25:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:25:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:25:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:25:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:25:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:25:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:25:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:25:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:25:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:25:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:25:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:25:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:25:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:25:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:25:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:25:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:25:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:25:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:25:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:25:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:25:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:25:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:25:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:25:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:25:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:25:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:25:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:25:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:25:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:25:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:25:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:25:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:25:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:25:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:25:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.371 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.371 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:25:16.371 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:25:16.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.338+0000 7f37aa78e640 1 -- 172.21.15.115:0/3397237936 >> v1:172.21.15.132:6812/1835401922 conn(0x7f37880377b0 legacy=0x7f3788039c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.338+0000 7f37aa78e640 10 monclient: shutdown 2024-01-24T23:25:16.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.338+0000 7f37aa78e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:16.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.338+0000 7f37aa78e640 1 -- 172.21.15.115:0/3397237936 >> v1:172.21.15.115:6791/0 conn(0x7f37a413a470 legacy=0x7f37a41ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.338+0000 7f37a8d2b640 1 -- 172.21.15.115:0/3397237936 reap_dead start 2024-01-24T23:25:16.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.338+0000 7f37a8d2b640 1 -- 172.21.15.115:0/3397237936 reap_dead start 2024-01-24T23:25:16.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.338+0000 7f37aa78e640 1 -- 172.21.15.115:0/3397237936 shutdown_connections 2024-01-24T23:25:16.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.338+0000 7f37aa78e640 1 -- 172.21.15.115:0/3397237936 >> 172.21.15.115:0/3397237936 conn(0x7f37a40a5730 msgr2=0x7f37a40a0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:16.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.338+0000 7f37aa78e640 1 -- 172.21.15.115:0/3397237936 shutdown_connections 2024-01-24T23:25:16.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.338+0000 7f37aa78e640 1 -- 172.21.15.115:0/3397237936 wait complete. 2024-01-24T23:25:16.373 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:25:16.373 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-01-24T23:25:16.373 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-01-24T23:25:16.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.474+0000 7f04020e3640 10 monclient: get_monmap_and_config 2024-01-24T23:25:16.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.474+0000 7f04020e3640 10 monclient: build_initial_monmap 2024-01-24T23:25:16.478 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.474+0000 7f04020e3640 10 monclient: monmap: 2024-01-24T23:25:16.478 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:16.478 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.478 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:16.478171+0000 2024-01-24T23:25:16.478 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:16.478171+0000 2024-01-24T23:25:16.478 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:16.478 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.478 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:16.478 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:16.478 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:16.478 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:16.479 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:16.479 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:16.479 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:16.479 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:16.479 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:16.479 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.474+0000 7f04020e3640 1 Processor -- start 2024-01-24T23:25:16.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 1 -- start start 2024-01-24T23:25:16.479 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 10 monclient: init 2024-01-24T23:25:16.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:16.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 10 monclient: _add_conns ranks=[8,0,4,2,5,6,7,1,3] 2024-01-24T23:25:16.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 10 monclient(hunting): picked mon.noname-e con 0x7f03fc133bf0 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:16.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 10 monclient(hunting): picked mon.noname-a con 0x7f03fc133fd0 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:16.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 10 monclient(hunting): picked mon.noname-c con 0x7f03fc1343b0 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:16.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03fc04ed70 con 0x7f03fc133fd0 2024-01-24T23:25:16.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03fc0a4b60 con 0x7f03fc1343b0 2024-01-24T23:25:16.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03fc09ae50 con 0x7f03fc133bf0 2024-01-24T23:25:16.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:16.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 10 monclient(hunting): authenticate will time out at 1195.400734s 2024-01-24T23:25:16.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03fb7fe640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f03fc133bf0 0x7f03fc09bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:59480/0 (socket says 172.21.15.115:59480) 2024-01-24T23:25:16.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03fb7fe640 1 -- 172.21.15.115:0/1768359756 learned_addr learned my addr 172.21.15.115:0/1768359756 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:16.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03fbfff640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f03fc1343b0 0x7f03fc13f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:44652/0 (socket says 172.21.15.115:44652) 2024-01-24T23:25:16.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03f3fff640 1 --1- 172.21.15.115:0/1768359756 >> v1:172.21.15.115:6789/0 conn(0x7f03fc133fd0 0x7f03fc13bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44152/0 (socket says 172.21.15.115:44152) 2024-01-24T23:25:16.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 1 -- 172.21.15.115:0/1768359756 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181455894 0 0) 0x7f03fc0a4b60 con 0x7f03fc1343b0 2024-01-24T23:25:16.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 10 monclient(hunting): my global_id is 94180 2024-01-24T23:25:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 1 -- 172.21.15.115:0/1768359756 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03e8003530 con 0x7f03fc1343b0 2024-01-24T23:25:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 1 -- 172.21.15.115:0/1768359756 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331914900 0 0) 0x7f03fc04ed70 con 0x7f03fc133fd0 2024-01-24T23:25:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 10 monclient(hunting): my global_id is 94185 2024-01-24T23:25:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 1 -- 172.21.15.115:0/1768359756 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03fc0a4b60 con 0x7f03fc133fd0 2024-01-24T23:25:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 1 -- 172.21.15.115:0/1768359756 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2284998788 0 0) 0x7f03fc09ae50 con 0x7f03fc133bf0 2024-01-24T23:25:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 10 monclient(hunting): my global_id is 85076 2024-01-24T23:25:16.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 1 -- 172.21.15.115:0/1768359756 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03fc04ed70 con 0x7f03fc133bf0 2024-01-24T23:25:16.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 1 -- 172.21.15.115:0/1768359756 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3694763308 0 0) 0x7f03e8003530 con 0x7f03fc1343b0 2024-01-24T23:25:16.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 1 -- 172.21.15.115:0/1768359756 >> v1:172.21.15.115:6793/0 conn(0x7f03fc133bf0 legacy=0x7f03fc09bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 1 -- 172.21.15.115:0/1768359756 >> v1:172.21.15.115:6789/0 conn(0x7f03fc133fd0 legacy=0x7f03fc13bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 10 monclient: _finish_hunting 0 2024-01-24T23:25:16.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 1 monclient: found mon.noname-c 2024-01-24T23:25:16.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:16.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:25:16.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 1 -- 172.21.15.115:0/1768359756 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03fc09b130 con 0x7f03fc1343b0 2024-01-24T23:25:16.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 10 monclient: _finish_auth 0 2024-01-24T23:25:16.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:16.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:16.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 1 -- 172.21.15.115:0/1768359756 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f03f4001b40 con 0x7f03fc1343b0 2024-01-24T23:25:16.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:25:16.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 10 monclient: dump: 2024-01-24T23:25:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.491 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.491 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 5 monclient: authenticate success, global_id 94180 2024-01-24T23:25:16.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:16.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 1 -- 172.21.15.115:0/1768359756 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f03f4001e20 con 0x7f03fc1343b0 2024-01-24T23:25:16.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:16.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 1 -- 172.21.15.115:0/1768359756 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f03f4004eb0 con 0x7f03fc1343b0 2024-01-24T23:25:16.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:16.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 4 set_mon_vals no callback set 2024-01-24T23:25:16.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:16.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03faffd640 10 monclient: dump: 2024-01-24T23:25:16.491 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.491 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.492 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.492 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.492 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.492 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.492 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.492 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.492 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.492 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.492 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.492 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.492 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.492 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.492 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.492 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 10 monclient: shutdown 2024-01-24T23:25:16.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:16.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 1 -- 172.21.15.115:0/1768359756 >> v1:172.21.15.115:6791/0 conn(0x7f03fc1343b0 legacy=0x7f03fc13f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 1 -- 172.21.15.115:0/1768359756 shutdown_connections 2024-01-24T23:25:16.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 1 -- 172.21.15.115:0/1768359756 >> 172.21.15.115:0/1768359756 conn(0x7f03fc0662f0 msgr2=0x7f03fc066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:16.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 1 -- 172.21.15.115:0/1768359756 shutdown_connections 2024-01-24T23:25:16.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 1 -- 172.21.15.115:0/1768359756 wait complete. 2024-01-24T23:25:16.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 10 monclient: build_initial_monmap 2024-01-24T23:25:16.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 10 monclient: monmap: 2024-01-24T23:25:16.493 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:16.493 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:16.493 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:16.493 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:16.494 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:16.494 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:16.494 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:16.494 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:16.494 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:16.494 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:16.494 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:16.494 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:16.494 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:16.494 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:16.494 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:16.494 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 1 Processor -- start 2024-01-24T23:25:16.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 1 -- start start 2024-01-24T23:25:16.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 10 monclient: init 2024-01-24T23:25:16.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:16.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 10 monclient: _add_conns ranks=[0,8,3,2,5,6,4,1,7] 2024-01-24T23:25:16.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 10 monclient(hunting): picked mon.noname-a con 0x7f03fc133bf0 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:16.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 10 monclient(hunting): picked mon.noname-e con 0x7f03fc133fd0 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:16.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 10 monclient(hunting): picked mon.noname-g con 0x7f03fc1343b0 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:16.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03fc0a3460 con 0x7f03fc133bf0 2024-01-24T23:25:16.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03fc0a2bf0 con 0x7f03fc1343b0 2024-01-24T23:25:16.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03fc120730 con 0x7f03fc133fd0 2024-01-24T23:25:16.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:16.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f04020e3640 10 monclient(hunting): authenticate will time out at 1195.403478s 2024-01-24T23:25:16.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03fb7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f03fc133bf0 0x7f03fc0a0d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44160/0 (socket says 172.21.15.115:44160) 2024-01-24T23:25:16.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03f3fff640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f03fc133fd0 0x7f03fc0a1430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:59484/0 (socket says 172.21.15.115:59484) 2024-01-24T23:25:16.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03fb7fe640 1 -- 172.21.15.115:0/1359479819 learned_addr learned my addr 172.21.15.115:0/1359479819 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:16.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1550090108 0 0) 0x7f03fc0a3460 con 0x7f03fc133bf0 2024-01-24T23:25:16.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03f8ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03f8ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03f8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03f8ff9640 10 monclient(hunting): my global_id is 94194 2024-01-24T23:25:16.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03dc003530 con 0x7f03fc133bf0 2024-01-24T23:25:16.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 160658162 0 0) 0x7f03fc120730 con 0x7f03fc133fd0 2024-01-24T23:25:16.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03f8ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03f8ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03f8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03f8ff9640 10 monclient(hunting): my global_id is 85085 2024-01-24T23:25:16.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03fc0a3460 con 0x7f03fc133fd0 2024-01-24T23:25:16.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.478+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1772824327 0 0) 0x7f03dc003530 con 0x7f03fc133bf0 2024-01-24T23:25:16.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f03fc120730 con 0x7f03fc133bf0 2024-01-24T23:25:16.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f03ec0019e0 con 0x7f03fc133bf0 2024-01-24T23:25:16.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:25:16.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 10 monclient(hunting): dump: 2024-01-24T23:25:16.497 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.497 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.497 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.497 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.497 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.497 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.497 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.497 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.497 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.498 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.498 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.498 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.498 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.498 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.498 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.498 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:16.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2163230188 0 0) 0x7f03fc0a3460 con 0x7f03fc133fd0 2024-01-24T23:25:16.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f03dc003530 con 0x7f03fc133fd0 2024-01-24T23:25:16.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f03e4001a60 con 0x7f03fc133fd0 2024-01-24T23:25:16.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:16.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 10 monclient(hunting): dump: 2024-01-24T23:25:16.498 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.499 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.499 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.499 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.499 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.499 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.499 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.499 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.499 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.499 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.499 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.499 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.499 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.499 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.499 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.500 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1968028860 0 0) 0x7f03fc120730 con 0x7f03fc133bf0 2024-01-24T23:25:16.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 >> v1:172.21.15.115:6793/0 conn(0x7f03fc133fd0 legacy=0x7f03fc0a1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 >> v1:172.21.15.132:6790/0 conn(0x7f03fc1343b0 legacy=0x7f03fc1d3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:25:16.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 1 monclient: found mon.a 2024-01-24T23:25:16.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:16.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:16.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03fc0a3240 con 0x7f03fc133bf0 2024-01-24T23:25:16.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 10 monclient: _finish_auth 0 2024-01-24T23:25:16.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:25:16.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:16.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f04020e3640 5 monclient: authenticate success, global_id 94194 2024-01-24T23:25:16.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f04020e3640 10 monclient: _renew_subs 2024-01-24T23:25:16.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f04020e3640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:16.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f04020e3640 1 -- 172.21.15.115:0/1359479819 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03fc120a10 con 0x7f03fc133bf0 2024-01-24T23:25:16.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f04020e3640 10 monclient: _renew_subs 2024-01-24T23:25:16.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f04020e3640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:16.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f04020e3640 1 -- 172.21.15.115:0/1359479819 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f03fc0a5990 con 0x7f03fc133bf0 2024-01-24T23:25:16.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f03ec001d70 con 0x7f03fc133bf0 2024-01-24T23:25:16.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:16.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f03ec004de0 con 0x7f03fc133bf0 2024-01-24T23:25:16.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03fa7fc640 4 set_mon_vals no callback set 2024-01-24T23:25:16.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:16.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 10 monclient: dump: 2024-01-24T23:25:16.501 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.501 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.501 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.501 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.502 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.502 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.502 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.502 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.502 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.502 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.502 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.502 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.502 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.502 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.502 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.502 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f04020e3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:16.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f04020e3640 1 -- 172.21.15.115:0/1359479819 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f03e00065d0 con 0x7f03e0002870 2024-01-24T23:25:16.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f03ec004f40 con 0x7f03fc133bf0 2024-01-24T23:25:16.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(127..127 src has 1..127) v4 ==== 6989+0+0 (unknown 755296415 0 0) 0x7f03ec044440 con 0x7f03fc133bf0 2024-01-24T23:25:16.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03fbfff640 10 monclient: get_auth_request con 0x7f03dc0376c0 auth_method 0 2024-01-24T23:25:16.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2461373920 0 0) 0x7f03e00065d0 con 0x7f03e0002870 2024-01-24T23:25:16.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 10 monclient(hunting): my global_id is 5837 2024-01-24T23:25:16.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03fc120730 con 0x7f03e0002870 2024-01-24T23:25:16.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1632576458 0 0) 0x7f03fc120730 con 0x7f03e0002870 2024-01-24T23:25:16.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f03e00065d0 con 0x7f03e0002870 2024-01-24T23:25:16.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f03e4004b30 con 0x7f03e0002870 2024-01-24T23:25:16.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:16.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 10 monclient: dump: 2024-01-24T23:25:16.503 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.503 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.504 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.504 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.504 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.504 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.504 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.504 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.504 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.504 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.504 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.504 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.504 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.504 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.504 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.504 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.504 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3814858011 0 0) 0x7f03e00065d0 con 0x7f03e0002870 2024-01-24T23:25:16.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 --> v1:172.21.15.132:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f03e000a120 con 0x7f03e0002870 2024-01-24T23:25:16.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f03e4004fa0 con 0x7f03e0002870 2024-01-24T23:25:16.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:16.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:16.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 >> v1:172.21.15.132:6791/0 conn(0x7f03e0002870 legacy=0x7f03e0004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.482+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 >> v1:172.21.15.132:6791/0 conn(0x7f03e0002870 legacy=0x7f03e0004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:16.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.490+0000 7f04020e3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:25:16.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.490+0000 7f04020e3640 1 -- 172.21.15.115:0/1359479819 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f03e0009e10 con 0x7f03e00090d0 2024-01-24T23:25:16.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.490+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3840541939 0 0) 0x7f03e0009e10 con 0x7f03e00090d0 2024-01-24T23:25:16.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.490+0000 7f03f8ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.490+0000 7f03f8ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.490+0000 7f03f8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.490+0000 7f03f8ff9640 10 monclient(hunting): my global_id is 5846 2024-01-24T23:25:16.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.490+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03e00065d0 con 0x7f03e00090d0 2024-01-24T23:25:16.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.490+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 488038141 0 0) 0x7f03e00065d0 con 0x7f03e00090d0 2024-01-24T23:25:16.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.490+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f03e0009e10 con 0x7f03e00090d0 2024-01-24T23:25:16.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.490+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f03e40057e0 con 0x7f03e00090d0 2024-01-24T23:25:16.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.490+0000 7f03f8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.490+0000 7f03f8ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:16.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.490+0000 7f03f8ff9640 10 monclient: dump: 2024-01-24T23:25:16.507 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.507 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.507 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.508 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.508 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.508 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.508 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.508 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.508 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.508 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.508 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.508 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.508 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.508 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.508 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.508 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.490+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 189992025 0 0) 0x7f03e0009e10 con 0x7f03e00090d0 2024-01-24T23:25:16.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.490+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 --> v1:172.21.15.132:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f03e000c310 con 0x7f03e00090d0 2024-01-24T23:25:16.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.494+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7883 (unknown 0 0 1317348262) 0x7f03e4005a50 con 0x7f03e00090d0 2024-01-24T23:25:16.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.494+0000 7f03f8ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:25:16.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.494+0000 7f03f8ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:16.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.494+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 >> v1:172.21.15.132:6791/0 conn(0x7f03e00090d0 legacy=0x7f03e0009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.494+0000 7f03f8ff9640 1 -- 172.21.15.115:0/1359479819 >> v1:172.21.15.132:6791/0 conn(0x7f03e00090d0 legacy=0x7f03e0009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:16.509 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:25:16.509 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:16.509 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:16.509 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:25:16.509 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 24, 2024-01-24T23:25:16.509 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:25:16.510 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:25:16.510 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:25:16.510 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:25:16.510 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:25:16.510 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:25:16.510 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:25:16.510 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:25:16.510 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:25:16.510 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:25:16.510 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:16.510 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 1, 2024-01-24T23:25:16.510 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:16.510 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:25:16.510 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:25:16.510 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:16.511 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:16.511 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:16.511 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:16.511 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:16.511 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:16.511 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:16.511 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:16.511 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:16.511 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:16.511 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:16.511 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:16.511 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:25:16.511 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:25:16.511 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:16.511 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:16.512 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:16.512 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:16.512 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:16.512 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:16.512 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:16.512 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:16.512 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:16.512 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:16.512 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:16.512 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.512 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:25:16.512 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:25:16.512 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:25:16.513 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:25:16.513 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:25:16.513 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:25:16.513 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:16.513 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:16.513 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:25:16.513 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:25:16.513 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:25:16.513 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:25:16.513 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:25:16.513 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:25:16.513 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:25:16.513 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:16.513 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:25:16.513 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:16.514 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:16.514 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:16.514 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:16.514 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:16.514 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:16.514 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:16.514 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:16.514 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:16.514 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:16.514 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:16.514 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:16.514 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:25:16.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.514 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:25:16.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:25:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:25:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:25:16.515 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:25:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:25:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:25:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:25:16.517 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:25:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:25:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:25:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:25:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:25:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.519 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:25:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:25:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:25:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.521 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:16.521 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:16.521 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.521 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.521 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.521 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:25:16.521 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.521 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.521 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.521 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:25:16.521 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:25:16.521 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.522 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.522 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.522 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:16.522 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:16.522 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.522 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.522 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.522 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:25:16.522 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.522 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.522 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.522 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:25:16.523 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:25:16.523 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.523 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.523 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.523 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:16.523 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:16.523 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.523 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.523 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.523 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:25:16.523 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.523 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.524 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.524 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:25:16.524 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:25:16.524 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.524 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.524 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.524 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:16.524 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:16.524 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.524 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.524 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.524 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:25:16.525 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.525 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.525 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.525 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:25:16.525 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:25:16.525 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.525 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.525 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.525 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.525 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.525 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:25:16.525 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:25:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:25:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:25:16.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": [ 2024-01-24T23:25:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.527 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:25:16.527 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:25:16.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.494+0000 7f04020e3640 1 -- 172.21.15.115:0/1359479819 >> v1:172.21.15.132:6812/1835401922 conn(0x7f03dc0376c0 legacy=0x7f03dc039b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.494+0000 7f04020e3640 10 monclient: shutdown 2024-01-24T23:25:16.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.494+0000 7f04020e3640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:16.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.494+0000 7f04020e3640 1 -- 172.21.15.115:0/1359479819 >> v1:172.21.15.115:6789/0 conn(0x7f03fc133bf0 legacy=0x7f03fc0a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.494+0000 7f03fbfff640 1 -- 172.21.15.115:0/1359479819 reap_dead start 2024-01-24T23:25:16.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.494+0000 7f03fbfff640 1 -- 172.21.15.115:0/1359479819 reap_dead start 2024-01-24T23:25:16.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.494+0000 7f04020e3640 1 -- 172.21.15.115:0/1359479819 shutdown_connections 2024-01-24T23:25:16.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.494+0000 7f04020e3640 1 -- 172.21.15.115:0/1359479819 >> 172.21.15.115:0/1359479819 conn(0x7f03fc0662f0 msgr2=0x7f03fc09d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:16.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.494+0000 7f04020e3640 1 -- 172.21.15.115:0/1359479819 shutdown_connections 2024-01-24T23:25:16.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.494+0000 7f04020e3640 1 -- 172.21.15.115:0/1359479819 wait complete. 2024-01-24T23:25:16.530 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:25:16.530 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-01-24T23:25:16.530 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-01-24T23:25:16.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.634+0000 7fe6a3cbe640 10 monclient: get_monmap_and_config 2024-01-24T23:25:16.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.634+0000 7fe6a3cbe640 10 monclient: build_initial_monmap 2024-01-24T23:25:16.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.634+0000 7fe6a3cbe640 10 monclient: monmap: 2024-01-24T23:25:16.638 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:16.638 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.638 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:16.638411+0000 2024-01-24T23:25:16.638 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:16.638411+0000 2024-01-24T23:25:16.638 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:16.638 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.638 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:16.638 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:16.638 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:16.638 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:16.638 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:16.638 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:16.638 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:16.638 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:16.639 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:16.639 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:16.641+0000 7f96367fc640 10 monclient: tick 2024-01-24T23:25:16.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:16.641+0000 7f96367fc640 10 monclient: _check_auth_tickets 2024-01-24T23:25:16.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:16.641+0000 7f96367fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:16.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:16.641+0000 7f96367fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:16.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 1 Processor -- start 2024-01-24T23:25:16.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 1 -- start start 2024-01-24T23:25:16.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 10 monclient: init 2024-01-24T23:25:16.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:16.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 10 monclient: _add_conns ranks=[8,3,7,2,4,1,5,0,6] 2024-01-24T23:25:16.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 10 monclient(hunting): picked mon.noname-e con 0x7fe69c0a0a10 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:16.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 10 monclient(hunting): picked mon.noname-g con 0x7fe69c09f1e0 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:16.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 10 monclient(hunting): picked mon.noname-i con 0x7fe69c13d360 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:16.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe69c04ed70 con 0x7fe69c09f1e0 2024-01-24T23:25:16.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe69c0a1500 con 0x7fe69c13d360 2024-01-24T23:25:16.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe69c06aaf0 con 0x7fe69c0a0a10 2024-01-24T23:25:16.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:16.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 10 monclient(hunting): authenticate will time out at 1195.560927s 2024-01-24T23:25:16.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a1a5a640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fe69c0a0a10 0x7fe69c0a0df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:59494/0 (socket says 172.21.15.115:59494) 2024-01-24T23:25:16.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a1a5a640 1 -- 172.21.15.115:0/2107948920 learned_addr learned my addr 172.21.15.115:0/2107948920 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:16.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 1 -- 172.21.15.115:0/2107948920 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22930433 0 0) 0x7fe69c06aaf0 con 0x7fe69c0a0a10 2024-01-24T23:25:16.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 10 monclient(hunting): my global_id is 85094 2024-01-24T23:25:16.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 1 -- 172.21.15.115:0/2107948920 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe688003530 con 0x7fe69c0a0a10 2024-01-24T23:25:16.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 1 -- 172.21.15.115:0/2107948920 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3360133857 0 0) 0x7fe688003530 con 0x7fe69c0a0a10 2024-01-24T23:25:16.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 1 -- 172.21.15.115:0/2107948920 >> v1:172.21.15.132:6792/0 conn(0x7fe69c13d360 legacy=0x7fe69c13f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 1 -- 172.21.15.115:0/2107948920 >> v1:172.21.15.132:6790/0 conn(0x7fe69c09f1e0 legacy=0x7fe69c13bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 10 monclient: _finish_hunting 0 2024-01-24T23:25:16.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 1 monclient: found mon.noname-e 2024-01-24T23:25:16.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:16.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:25:16.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 1 -- 172.21.15.115:0/2107948920 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69c09e560 con 0x7fe69c0a0a10 2024-01-24T23:25:16.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 10 monclient: _finish_auth 0 2024-01-24T23:25:16.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 10 monclient: _check_auth_tickets 2024-01-24T23:25:16.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:16.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 5 monclient: authenticate success, global_id 85094 2024-01-24T23:25:16.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:16.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 1 -- 172.21.15.115:0/2107948920 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe6980017c0 con 0x7fe69c0a0a10 2024-01-24T23:25:16.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:25:16.650 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 10 monclient: dump: 2024-01-24T23:25:16.650 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.650 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.650 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.650 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.650 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.650 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.650 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.650 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.650 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.650 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.650 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.650 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.651 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.651 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.651 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.651 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:16.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 1 -- 172.21.15.115:0/2107948920 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe698001a70 con 0x7fe69c0a0a10 2024-01-24T23:25:16.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:16.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:16.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 4 set_mon_vals no callback set 2024-01-24T23:25:16.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 1 -- 172.21.15.115:0/2107948920 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe698004bf0 con 0x7fe69c0a0a10 2024-01-24T23:25:16.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:16.651 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a0a58640 10 monclient: dump: 2024-01-24T23:25:16.651 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.651 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.651 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.652 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.652 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.652 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.652 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.652 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.652 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.652 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.652 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.652 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.652 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.652 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.652 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.652 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 10 monclient: shutdown 2024-01-24T23:25:16.652 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:16.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 1 -- 172.21.15.115:0/2107948920 >> v1:172.21.15.115:6793/0 conn(0x7fe69c0a0a10 legacy=0x7fe69c0a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 1 -- 172.21.15.115:0/2107948920 shutdown_connections 2024-01-24T23:25:16.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 1 -- 172.21.15.115:0/2107948920 >> 172.21.15.115:0/2107948920 conn(0x7fe69c0a7660 msgr2=0x7fe69c0a3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:16.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 1 -- 172.21.15.115:0/2107948920 shutdown_connections 2024-01-24T23:25:16.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 1 -- 172.21.15.115:0/2107948920 wait complete. 2024-01-24T23:25:16.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 10 monclient: build_initial_monmap 2024-01-24T23:25:16.653 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 10 monclient: monmap: 2024-01-24T23:25:16.653 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:16.653 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:16.653 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:16.653 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:16.653 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:16.653 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:16.653 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:16.653 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:16.654 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:16.654 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:16.654 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:16.654 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:16.654 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:16.654 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:16.654 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:16.654 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 1 Processor -- start 2024-01-24T23:25:16.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 1 -- start start 2024-01-24T23:25:16.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 10 monclient: init 2024-01-24T23:25:16.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:16.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 10 monclient: _add_conns ranks=[7,0,4,6,5,1,8,3,2] 2024-01-24T23:25:16.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 10 monclient(hunting): picked mon.noname-i con 0x7fe69c09f1e0 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:16.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 10 monclient(hunting): picked mon.noname-a con 0x7fe69c0a0a10 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:16.654 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 10 monclient(hunting): picked mon.noname-c con 0x7fe69c13d360 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:16.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe69c071290 con 0x7fe69c0a0a10 2024-01-24T23:25:16.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe69c09bb00 con 0x7fe69c13d360 2024-01-24T23:25:16.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe69c072400 con 0x7fe69c09f1e0 2024-01-24T23:25:16.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:16.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a3cbe640 10 monclient(hunting): authenticate will time out at 1195.563665s 2024-01-24T23:25:16.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a225b640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fe69c13d360 0x7fe69c1d41f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:44660/0 (socket says 172.21.15.115:44660) 2024-01-24T23:25:16.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a225b640 1 -- 172.21.15.115:0/1241320865 learned_addr learned my addr 172.21.15.115:0/1241320865 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:16.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.638+0000 7fe6a1259640 1 --1- 172.21.15.115:0/1241320865 >> v1:172.21.15.115:6789/0 conn(0x7fe69c0a0a10 0x7fe69c1d0ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44168/0 (socket says 172.21.15.115:44168) 2024-01-24T23:25:16.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3069922188 0 0) 0x7fe69c071290 con 0x7fe69c0a0a10 2024-01-24T23:25:16.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient(hunting): my global_id is 94203 2024-01-24T23:25:16.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe680003530 con 0x7fe69c0a0a10 2024-01-24T23:25:16.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2386860707 0 0) 0x7fe69c09bb00 con 0x7fe69c13d360 2024-01-24T23:25:16.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient(hunting): my global_id is 94189 2024-01-24T23:25:16.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe69c071290 con 0x7fe69c13d360 2024-01-24T23:25:16.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 184425696 0 0) 0x7fe69c072400 con 0x7fe69c09f1e0 2024-01-24T23:25:16.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient(hunting): my global_id is 5848 2024-01-24T23:25:16.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe69c09bb00 con 0x7fe69c09f1e0 2024-01-24T23:25:16.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2430004758 0 0) 0x7fe69c071290 con 0x7fe69c13d360 2024-01-24T23:25:16.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe69c072400 con 0x7fe69c13d360 2024-01-24T23:25:16.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe694001c40 con 0x7fe69c13d360 2024-01-24T23:25:16.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:25:16.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient(hunting): dump: 2024-01-24T23:25:16.657 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.657 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.657 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.657 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.657 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.657 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.657 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.657 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.657 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.657 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.657 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.657 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.658 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.658 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.658 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.658 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:16.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 609682669 0 0) 0x7fe680003530 con 0x7fe69c0a0a10 2024-01-24T23:25:16.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe69c071290 con 0x7fe69c0a0a10 2024-01-24T23:25:16.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe690001a60 con 0x7fe69c0a0a10 2024-01-24T23:25:16.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:16.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient(hunting): dump: 2024-01-24T23:25:16.658 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.658 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.658 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.658 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.658 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.659 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.659 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.659 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.659 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.659 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.659 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.659 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.659 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.659 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.659 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.659 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 341524992 0 0) 0x7fe69c072400 con 0x7fe69c13d360 2024-01-24T23:25:16.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 >> v1:172.21.15.132:6792/0 conn(0x7fe69c09f1e0 legacy=0x7fe69c09d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 >> v1:172.21.15.115:6789/0 conn(0x7fe69c0a0a10 legacy=0x7fe69c1d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:25:16.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 monclient: found mon.c 2024-01-24T23:25:16.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:16.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:16.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69c071eb0 con 0x7fe69c13d360 2024-01-24T23:25:16.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient: _finish_auth 0 2024-01-24T23:25:16.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:25:16.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:16.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe6a3cbe640 5 monclient: authenticate success, global_id 94189 2024-01-24T23:25:16.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe6a3cbe640 10 monclient: _renew_subs 2024-01-24T23:25:16.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe6a3cbe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:16.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe6a3cbe640 1 -- 172.21.15.115:0/1241320865 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe69c1cbe80 con 0x7fe69c13d360 2024-01-24T23:25:16.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe6a3cbe640 10 monclient: _renew_subs 2024-01-24T23:25:16.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe6a3cbe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:16.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe6a3cbe640 1 -- 172.21.15.115:0/1241320865 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe69c09d740 con 0x7fe69c13d360 2024-01-24T23:25:16.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe694002010 con 0x7fe69c13d360 2024-01-24T23:25:16.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:16.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe694003f90 con 0x7fe69c13d360 2024-01-24T23:25:16.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:16.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68ffff640 4 set_mon_vals no callback set 2024-01-24T23:25:16.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient: dump: 2024-01-24T23:25:16.661 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.661 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.661 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.661 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.661 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.661 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.661 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.661 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.662 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.662 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.662 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.662 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.662 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.662 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.662 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.662 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe6a3cbe640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:16.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe6a3cbe640 1 -- 172.21.15.115:0/1241320865 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe6840065d0 con 0x7fe684002870 2024-01-24T23:25:16.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fe694004170 con 0x7fe69c13d360 2024-01-24T23:25:16.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(127..127 src has 1..127) v4 ==== 6989+0+0 (unknown 755296415 0 0) 0x7fe6940446d0 con 0x7fe69c13d360 2024-01-24T23:25:16.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe6a1259640 10 monclient: get_auth_request con 0x7fe6800378a0 auth_method 0 2024-01-24T23:25:16.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306540780 0 0) 0x7fe6840065d0 con 0x7fe684002870 2024-01-24T23:25:16.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient(hunting): my global_id is 5857 2024-01-24T23:25:16.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe69c072400 con 0x7fe684002870 2024-01-24T23:25:16.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3584372484 0 0) 0x7fe69c072400 con 0x7fe684002870 2024-01-24T23:25:16.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe6840065d0 con 0x7fe684002870 2024-01-24T23:25:16.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe698004b30 con 0x7fe684002870 2024-01-24T23:25:16.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:25:16.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient: dump: 2024-01-24T23:25:16.663 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.663 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.663 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.663 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.663 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.664 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.664 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.664 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.664 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.664 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.664 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.664 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.664 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.664 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.664 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.664 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 309437893 0 0) 0x7fe6840065d0 con 0x7fe684002870 2024-01-24T23:25:16.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 --> v1:172.21.15.132:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe68400a120 con 0x7fe684002870 2024-01-24T23:25:16.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 <== mon.7 v1:172.21.15.132:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fe698001670 con 0x7fe684002870 2024-01-24T23:25:16.664 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:16.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:16.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 >> v1:172.21.15.132:6792/0 conn(0x7fe684002870 legacy=0x7fe684004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.642+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 >> v1:172.21.15.132:6792/0 conn(0x7fe684002870 legacy=0x7fe684004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:16.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.650+0000 7fe6a3cbe640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:25:16.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.650+0000 7fe6a3cbe640 1 -- 172.21.15.115:0/1241320865 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe684009e10 con 0x7fe6840090d0 2024-01-24T23:25:16.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.650+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2963771800 0 0) 0x7fe684009e10 con 0x7fe6840090d0 2024-01-24T23:25:16.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.650+0000 7fe68e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.650+0000 7fe68e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.650+0000 7fe68e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.650+0000 7fe68e7fc640 10 monclient(hunting): my global_id is 5866 2024-01-24T23:25:16.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.650+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6840065d0 con 0x7fe6840090d0 2024-01-24T23:25:16.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.650+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 993857657 0 0) 0x7fe6840065d0 con 0x7fe6840090d0 2024-01-24T23:25:16.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.650+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe684009e10 con 0x7fe6840090d0 2024-01-24T23:25:16.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.650+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe6980053c0 con 0x7fe6840090d0 2024-01-24T23:25:16.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.650+0000 7fe68e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.650+0000 7fe68e7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:25:16.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.650+0000 7fe68e7fc640 10 monclient: dump: 2024-01-24T23:25:16.667 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.667 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.668 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.668 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.668 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.668 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.668 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.668 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.668 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.668 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.668 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.668 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.668 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.668 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.668 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.668 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.650+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3737111580 0 0) 0x7fe684009e10 con 0x7fe6840090d0 2024-01-24T23:25:16.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.650+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 --> v1:172.21.15.132:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe68400c310 con 0x7fe6840090d0 2024-01-24T23:25:16.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.650+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 <== mon.7 v1:172.21.15.132:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 1005091269) 0x7fe698007650 con 0x7fe6840090d0 2024-01-24T23:25:16.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.650+0000 7fe68e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:25:16.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.650+0000 7fe68e7fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:16.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.650+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 >> v1:172.21.15.132:6792/0 conn(0x7fe6840090d0 legacy=0x7fe684009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.650+0000 7fe68e7fc640 1 -- 172.21.15.115:0/1241320865 >> v1:172.21.15.132:6792/0 conn(0x7fe6840090d0 legacy=0x7fe684009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:16.669 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:25:16.669 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:16.669 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:16.669 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:25:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 24, 2024-01-24T23:25:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:25:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:25:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:25:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:25:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:25:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:25:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:25:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:25:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:25:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:25:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 1, 2024-01-24T23:25:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:25:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:25:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:25:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:25:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:25:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:25:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:25:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:25:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:25:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:25:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:25:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:25:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:25:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:25:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:25:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:25:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:25:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:25:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:25:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:25:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:25:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:25:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:25:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:25:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:25:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:25:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:25:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:25:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:25:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:25:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:25:16.680 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:25:16.680 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.680 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.680 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.680 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:16.680 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:16.680 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.680 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.680 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.680 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:25:16.681 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.681 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.681 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.681 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:25:16.681 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:25:16.681 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.681 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.681 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.681 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:16.681 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:16.681 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:25:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:25:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:25:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:25:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:25:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:25:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.684 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.684 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.684 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:16.684 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:16.684 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.684 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.684 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.684 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:25:16.684 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.684 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.684 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.685 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:25:16.685 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:25:16.685 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.685 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.685 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.685 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:16.685 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:16.685 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:16.685 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:16.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:25:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:25:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:25:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.687 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:25:16.687 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:25:16.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.687 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:16.687 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:16.687 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:16.687 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.687 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:16.687 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:25:16.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.687 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:16.687 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:16.687 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:25:16.687 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.688 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:16.688 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:25:16.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:16.688 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:16.688 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:16.688 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 3 2024-01-24T23:25:16.688 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:16.688 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:16.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:16.688 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:25:16.688 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:25:16.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.654+0000 7fe6a3cbe640 1 -- 172.21.15.115:0/1241320865 >> v1:172.21.15.132:6812/1835401922 conn(0x7fe6800378a0 legacy=0x7fe680039d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.654+0000 7fe6a3cbe640 10 monclient: shutdown 2024-01-24T23:25:16.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.654+0000 7fe6a3cbe640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:16.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.654+0000 7fe6a3cbe640 1 -- 172.21.15.115:0/1241320865 >> v1:172.21.15.115:6791/0 conn(0x7fe69c13d360 legacy=0x7fe69c1d41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.654+0000 7fe6a225b640 1 -- 172.21.15.115:0/1241320865 reap_dead start 2024-01-24T23:25:16.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.654+0000 7fe6a225b640 1 -- 172.21.15.115:0/1241320865 reap_dead start 2024-01-24T23:25:16.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.654+0000 7fe6a3cbe640 1 -- 172.21.15.115:0/1241320865 shutdown_connections 2024-01-24T23:25:16.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.654+0000 7fe6a3cbe640 1 -- 172.21.15.115:0/1241320865 >> 172.21.15.115:0/1241320865 conn(0x7fe69c0a7660 msgr2=0x7fe69c0a5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:16.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.654+0000 7fe6a3cbe640 1 -- 172.21.15.115:0/1241320865 shutdown_connections 2024-01-24T23:25:16.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.654+0000 7fe6a3cbe640 1 -- 172.21.15.115:0/1241320865 wait complete. 2024-01-24T23:25:16.691 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:25:16.691 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-01-24T23:25:16.691 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-01-24T23:25:16.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64fa382640 10 monclient: get_monmap_and_config 2024-01-24T23:25:16.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64fa382640 10 monclient: build_initial_monmap 2024-01-24T23:25:16.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64fa382640 10 monclient: monmap: 2024-01-24T23:25:16.799 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:16.799 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.799 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:16.799721+0000 2024-01-24T23:25:16.799 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:16.799721+0000 2024-01-24T23:25:16.799 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:16.799 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.800 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:16.800 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:16.800 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:16.800 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:16.800 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:16.800 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:16.800 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:16.800 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:16.800 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:16.800 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64fa382640 1 Processor -- start 2024-01-24T23:25:16.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64fa382640 1 -- start start 2024-01-24T23:25:16.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64fa382640 10 monclient: init 2024-01-24T23:25:16.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64fa382640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:16.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64fa382640 10 monclient: _add_conns ranks=[8,6,0,3,2,4,1,5,7] 2024-01-24T23:25:16.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64fa382640 10 monclient(hunting): picked mon.noname-e con 0x7f64f4133bf0 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:16.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64fa382640 10 monclient(hunting): picked mon.noname-d con 0x7f64f4133fd0 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:16.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64fa382640 10 monclient(hunting): picked mon.noname-a con 0x7f64f41343b0 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:16.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64fa382640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f404ed70 con 0x7f64f41343b0 2024-01-24T23:25:16.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64fa382640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f40a4b60 con 0x7f64f4133fd0 2024-01-24T23:25:16.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64fa382640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f409ae50 con 0x7f64f4133bf0 2024-01-24T23:25:16.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64fa382640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:16.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64fa382640 10 monclient(hunting): authenticate will time out at 1195.723044s 2024-01-24T23:25:16.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f37fe640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f64f4133fd0 0x7f64f413bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:40468/0 (socket says 172.21.15.115:40468) 2024-01-24T23:25:16.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f891f640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f64f41343b0 0x7f64f413f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44184/0 (socket says 172.21.15.115:44184) 2024-01-24T23:25:16.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f37fe640 1 -- 172.21.15.115:0/3584807662 learned_addr learned my addr 172.21.15.115:0/3584807662 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:16.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f3fff640 1 --1- 172.21.15.115:0/3584807662 >> v1:172.21.15.115:6793/0 conn(0x7f64f4133bf0 0x7f64f409bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:59504/0 (socket says 172.21.15.115:59504) 2024-01-24T23:25:16.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 1 -- 172.21.15.115:0/3584807662 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2417768811 0 0) 0x7f64f409ae50 con 0x7f64f4133bf0 2024-01-24T23:25:16.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 10 monclient(hunting): my global_id is 85103 2024-01-24T23:25:16.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 1 -- 172.21.15.115:0/3584807662 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64e0003530 con 0x7f64f4133bf0 2024-01-24T23:25:16.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 1 -- 172.21.15.115:0/3584807662 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 566068112 0 0) 0x7f64f40a4b60 con 0x7f64f4133fd0 2024-01-24T23:25:16.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.808 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 10 monclient(hunting): my global_id is 5991 2024-01-24T23:25:16.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 1 -- 172.21.15.115:0/3584807662 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64f409ae50 con 0x7f64f4133fd0 2024-01-24T23:25:16.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 1 -- 172.21.15.115:0/3584807662 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 151802063 0 0) 0x7f64f404ed70 con 0x7f64f41343b0 2024-01-24T23:25:16.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 10 monclient(hunting): my global_id is 94212 2024-01-24T23:25:16.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 1 -- 172.21.15.115:0/3584807662 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64f40a4b60 con 0x7f64f41343b0 2024-01-24T23:25:16.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 1 -- 172.21.15.115:0/3584807662 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3487163414 0 0) 0x7f64e0003530 con 0x7f64f4133bf0 2024-01-24T23:25:16.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 1 -- 172.21.15.115:0/3584807662 >> v1:172.21.15.115:6792/0 conn(0x7f64f4133fd0 legacy=0x7f64f413bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.809 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 1 -- 172.21.15.115:0/3584807662 >> v1:172.21.15.115:6789/0 conn(0x7f64f41343b0 legacy=0x7f64f413f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:25:16.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 1 monclient: found mon.noname-e 2024-01-24T23:25:16.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:16.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:25:16.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 1 -- 172.21.15.115:0/3584807662 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f409b130 con 0x7f64f4133bf0 2024-01-24T23:25:16.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 10 monclient: _finish_auth 0 2024-01-24T23:25:16.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:16.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:16.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64fa382640 5 monclient: authenticate success, global_id 85103 2024-01-24T23:25:16.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 1 -- 172.21.15.115:0/3584807662 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f64ec001b40 con 0x7f64f4133bf0 2024-01-24T23:25:16.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64fa382640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:16.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:25:16.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64f2ffd640 10 monclient: dump: 2024-01-24T23:25:16.811 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.811 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.811 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.811 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.811 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.811 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.811 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.811 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.811 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.811 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.811 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.811 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.811 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.811 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.811 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.811 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.798+0000 7f64fa382640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f2ffd640 1 -- 172.21.15.115:0/3584807662 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f64ec001e20 con 0x7f64f4133bf0 2024-01-24T23:25:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f2ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f2ffd640 1 -- 172.21.15.115:0/3584807662 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f64ec004eb0 con 0x7f64f4133bf0 2024-01-24T23:25:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f2ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f2ffd640 10 monclient: dump: 2024-01-24T23:25:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.813 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.813 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.813 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.813 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.813 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.813 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.813 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.813 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.813 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.813 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:16.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 4 set_mon_vals no callback set 2024-01-24T23:25:16.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 10 monclient: shutdown 2024-01-24T23:25:16.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:16.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 1 -- 172.21.15.115:0/3584807662 >> v1:172.21.15.115:6793/0 conn(0x7f64f4133bf0 legacy=0x7f64f409bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 1 -- 172.21.15.115:0/3584807662 shutdown_connections 2024-01-24T23:25:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 1 -- 172.21.15.115:0/3584807662 >> 172.21.15.115:0/3584807662 conn(0x7f64f40662f0 msgr2=0x7f64f4066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 1 -- 172.21.15.115:0/3584807662 shutdown_connections 2024-01-24T23:25:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 1 -- 172.21.15.115:0/3584807662 wait complete. 2024-01-24T23:25:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 10 monclient: build_initial_monmap 2024-01-24T23:25:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 10 monclient: monmap: 2024-01-24T23:25:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:16.815 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:16.815 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:16.815 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:16.815 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:16.815 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:16.815 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:16.815 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 1 Processor -- start 2024-01-24T23:25:16.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 1 -- start start 2024-01-24T23:25:16.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 10 monclient: init 2024-01-24T23:25:16.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:16.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 10 monclient: _add_conns ranks=[6,4,7,1,3,2,0,8,5] 2024-01-24T23:25:16.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 10 monclient(hunting): picked mon.noname-d con 0x7f64f4133bf0 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:16.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 10 monclient(hunting): picked mon.noname-c con 0x7f64f4133fd0 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:16.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 10 monclient(hunting): picked mon.noname-i con 0x7f64f41343b0 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:16.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f41cbde0 con 0x7f64f4133fd0 2024-01-24T23:25:16.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f4120860 con 0x7f64f4133bf0 2024-01-24T23:25:16.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f40a3300 con 0x7f64f41343b0 2024-01-24T23:25:16.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:16.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 10 monclient(hunting): authenticate will time out at 1195.726148s 2024-01-24T23:25:16.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f3fff640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f64f4133bf0 0x7f64f4132e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:40478/0 (socket says 172.21.15.115:40478) 2024-01-24T23:25:16.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f3fff640 1 -- 172.21.15.115:0/3071572677 learned_addr learned my addr 172.21.15.115:0/3071572677 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:16.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 1 -- 172.21.15.115:0/3071572677 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4278577372 0 0) 0x7f64f4120860 con 0x7f64f4133bf0 2024-01-24T23:25:16.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 10 monclient(hunting): my global_id is 6000 2024-01-24T23:25:16.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 1 -- 172.21.15.115:0/3071572677 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64d4003560 con 0x7f64f4133bf0 2024-01-24T23:25:16.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 1 -- 172.21.15.115:0/3071572677 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2867971817 0 0) 0x7f64f41cbde0 con 0x7f64f4133fd0 2024-01-24T23:25:16.816 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:25:16.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:16.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:16.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 10 monclient(hunting): my global_id is 94198 2024-01-24T23:25:16.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 1 -- 172.21.15.115:0/3071572677 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64f4120860 con 0x7f64f4133fd0 2024-01-24T23:25:16.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 1 -- 172.21.15.115:0/3071572677 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4214964988 0 0) 0x7f64f4120860 con 0x7f64f4133fd0 2024-01-24T23:25:16.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 1 -- 172.21.15.115:0/3071572677 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f64f41cbde0 con 0x7f64f4133fd0 2024-01-24T23:25:16.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 1 -- 172.21.15.115:0/3071572677 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f64dc001a60 con 0x7f64f4133fd0 2024-01-24T23:25:16.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:25:16.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 10 monclient(hunting): dump: 2024-01-24T23:25:16.817 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.817 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.817 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.817 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.817 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.818 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.818 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.818 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.818 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.818 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.818 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.818 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.818 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.818 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.818 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.818 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:16.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 1 -- 172.21.15.115:0/3071572677 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2664566925 0 0) 0x7f64d4003560 con 0x7f64f4133bf0 2024-01-24T23:25:16.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 1 -- 172.21.15.115:0/3071572677 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f64f4120860 con 0x7f64f4133bf0 2024-01-24T23:25:16.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 1 -- 172.21.15.115:0/3071572677 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f64ec001b00 con 0x7f64f4133bf0 2024-01-24T23:25:16.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:16.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 10 monclient(hunting): dump: 2024-01-24T23:25:16.819 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.819 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.819 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.819 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.819 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.819 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.819 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.819 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.819 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.819 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.819 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.819 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.819 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.819 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.820 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 1 -- 172.21.15.115:0/3071572677 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2756553817 0 0) 0x7f64f41cbde0 con 0x7f64f4133fd0 2024-01-24T23:25:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 1 -- 172.21.15.115:0/3071572677 >> v1:172.21.15.132:6792/0 conn(0x7f64f41343b0 legacy=0x7f64f41d3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 1 -- 172.21.15.115:0/3071572677 >> v1:172.21.15.115:6792/0 conn(0x7f64f4133bf0 legacy=0x7f64f4132e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:25:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 1 monclient: found mon.c 2024-01-24T23:25:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 1 -- 172.21.15.115:0/3071572677 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f40a5990 con 0x7f64f4133fd0 2024-01-24T23:25:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 10 monclient: _finish_auth 0 2024-01-24T23:25:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:25:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 5 monclient: authenticate success, global_id 94198 2024-01-24T23:25:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 10 monclient: _renew_subs 2024-01-24T23:25:16.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:16.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 1 -- 172.21.15.115:0/3071572677 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f411f970 con 0x7f64f4133fd0 2024-01-24T23:25:16.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 10 monclient: _renew_subs 2024-01-24T23:25:16.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:16.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 1 -- 172.21.15.115:0/3071572677 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f4120580 con 0x7f64f4133fd0 2024-01-24T23:25:16.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 1 -- 172.21.15.115:0/3071572677 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f64dc001d10 con 0x7f64f4133fd0 2024-01-24T23:25:16.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:16.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 1 -- 172.21.15.115:0/3071572677 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f64dc004e60 con 0x7f64f4133fd0 2024-01-24T23:25:16.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:16.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f27fc640 4 set_mon_vals no callback set 2024-01-24T23:25:16.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:16.821 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 10 monclient: dump: 2024-01-24T23:25:16.821 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:16.821 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:16.822 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:16.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:16.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:16.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64fa382640 1 -- 172.21.15.115:0/3071572677 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d8005090 con 0x7f64f4133fd0 2024-01-24T23:25:16.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.802+0000 7f64f0ff9640 1 -- 172.21.15.115:0/3071572677 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f64dc00f960 con 0x7f64f4133fd0 2024-01-24T23:25:16.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.806+0000 7f64f0ff9640 1 -- 172.21.15.115:0/3071572677 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(127..127 src has 1..127) v4 ==== 6989+0+0 (unknown 755296415 0 0) 0x7f64dc0443b0 con 0x7f64f4133fd0 2024-01-24T23:25:16.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.806+0000 7f64f3fff640 10 monclient: get_auth_request con 0x7f64d40376f0 auth_method 0 2024-01-24T23:25:16.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.810+0000 7f64f0ff9640 1 -- 172.21.15.115:0/3071572677 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f64dc014840 con 0x7f64f4133fd0 2024-01-24T23:25:16.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.810+0000 7f64f0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:16.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:16.810+0000 7f64f0ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:17.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:17.014+0000 7f64fa382640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-01-24T23:25:17.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:17.014+0000 7f64fa382640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-01-24T23:25:17.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:17.014+0000 7f64fa382640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:17.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:17.014+0000 7f64fa382640 1 -- 172.21.15.115:0/3071572677 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f64d8005b50 con 0x7f64f4133fd0 2024-01-24T23:25:17.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:17.022+0000 7f64f0ff9640 1 -- 172.21.15.115:0/3071572677 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (unknown 246732539 0 0) 0x7f64dc014840 con 0x7f64f4133fd0 2024-01-24T23:25:17.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:17.022+0000 7f64f0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-01-24T23:25:17.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:17.022+0000 7f64f0ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:17.024 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.017+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.024 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.017+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=17,config=2,health=12,kv=9,logm=60} crc {auth=1416520208,config=1618278320,health=195237072,kv=3447624886,logm=3821574262}) 2024-01-24T23:25:17.024 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.017+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=17,config=2,health=12,kv=8,logm=61} crc {auth=1338178634,config=1618278320,health=195237072,kv=2560653513,logm=435405302}) 2024-01-24T23:25:17.024 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.017+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.024 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.017+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=17,config=2,health=12,kv=9,logm=60} crc {auth=1416520208,config=1618278320,health=195237072,kv=3447624886,logm=3821574262}) 2024-01-24T23:25:17.024 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.017+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=18,config=2,health=12,kv=9,logm=59} crc {auth=471978071,config=1618278320,health=195237072,kv=3447624886,logm=1426886220}) 2024-01-24T23:25:17.024 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.017+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.024 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.017+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=17,config=2,health=12,kv=9,logm=60} crc {auth=1416520208,config=1618278320,health=195237072,kv=3447624886,logm=3821574262}) 2024-01-24T23:25:17.024 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.017+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=16,config=2,health=12,kv=6,logm=64} crc {auth=3853595376,config=1618278320,health=195237072,kv=734898033,logm=284747252}) 2024-01-24T23:25:17.024 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.017+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.024 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.017+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=17,config=2,health=12,kv=9,logm=60} crc {auth=1416520208,config=1618278320,health=195237072,kv=3447624886,logm=3821574262}) 2024-01-24T23:25:17.024 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.017+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=16,config=2,health=12,kv=9,logm=61} crc {auth=2036133759,config=1618278321,health=195237072,kv=3447624886,logm=3187772112}) 2024-01-24T23:25:17.025 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.017+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.025 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.017+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=17,config=2,health=12,kv=9,logm=60} crc {auth=1416520208,config=1618278320,health=195237072,kv=3447624886,logm=3821574262}) 2024-01-24T23:25:17.025 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.017+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=17,config=2,health=12,kv=6,logm=63} crc {auth=1233680763,config=1618278320,health=195237072,kv=3067984674,logm=1219502704}) 2024-01-24T23:25:17.025 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.017+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.025 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.017+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=17,config=2,health=12,kv=9,logm=60} crc {auth=1416520208,config=1618278320,health=195237072,kv=3447624886,logm=3821574262}) 2024-01-24T23:25:17.025 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.017+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=17,config=2,health=11,kv=9,logm=61} crc {auth=2483455582,config=1618278320,health=3357806740,kv=3447624886,logm=2847942889}) 2024-01-24T23:25:17.025 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.017+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.025 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.021+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=17,config=2,health=12,kv=9,logm=60} crc {auth=1416520208,config=1618278320,health=195237072,kv=3447624886,logm=3821574262}) 2024-01-24T23:25:17.025 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.021+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=18,config=2,health=12,kv=8,logm=60} crc {auth=471978071,config=1618278320,health=195237072,kv=4100078458,logm=106326745}) 2024-01-24T23:25:17.025 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.021+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.025 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.021+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=17,config=2,health=12,kv=9,logm=60} crc {auth=1416520208,config=1618278320,health=195237072,kv=3447624886,logm=3821574262}) 2024-01-24T23:25:17.025 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.021+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=17,config=2,health=11,kv=9,logm=61} crc {auth=2171222317,config=1618278320,health=7166074,kv=3447624886,logm=1555321387}) 2024-01-24T23:25:17.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:17.022+0000 7f64fa382640 1 -- 172.21.15.115:0/3071572677 >> v1:172.21.15.132:6812/1835401922 conn(0x7f64d40376f0 legacy=0x7f64d4039bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:17.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:17.022+0000 7f64fa382640 10 monclient: shutdown 2024-01-24T23:25:17.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:17.022+0000 7f64fa382640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:17.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:17.022+0000 7f64fa382640 1 -- 172.21.15.115:0/3071572677 >> v1:172.21.15.115:6791/0 conn(0x7f64f4133fd0 legacy=0x7f64f41d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:17.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:17.022+0000 7f64fa382640 1 -- 172.21.15.115:0/3071572677 shutdown_connections 2024-01-24T23:25:17.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:17.022+0000 7f64fa382640 1 -- 172.21.15.115:0/3071572677 >> 172.21.15.115:0/3071572677 conn(0x7f64f40662f0 msgr2=0x7f64f409d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:17.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:17.022+0000 7f64fa382640 1 -- 172.21.15.115:0/3071572677 shutdown_connections 2024-01-24T23:25:17.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:17.022+0000 7f64fa382640 1 -- 172.21.15.115:0/3071572677 wait complete. 2024-01-24T23:25:17.028 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.025+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.028 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.025+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3600798224}) 2024-01-24T23:25:17.028 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.025+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1417871809}) 2024-01-24T23:25:17.028 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.025+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.028 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.025+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3600798224}) 2024-01-24T23:25:17.028 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.025+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1737236377}) 2024-01-24T23:25:17.028 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.025+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.028 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.025+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3600798224}) 2024-01-24T23:25:17.028 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.025+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2504061447}) 2024-01-24T23:25:17.028 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.025+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.028 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.025+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3600798224}) 2024-01-24T23:25:17.028 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.025+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3103506633}) 2024-01-24T23:25:17.029 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.025+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.029 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.025+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3600798224}) 2024-01-24T23:25:17.029 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.025+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2631730689}) 2024-01-24T23:25:17.029 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.025+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.029 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.025+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3600798224}) 2024-01-24T23:25:17.029 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.025+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1380087274}) 2024-01-24T23:25:17.029 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.025+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.029 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.025+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3600798224}) 2024-01-24T23:25:17.029 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.025+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1940250899}) 2024-01-24T23:25:17.029 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.025+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.029 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.025+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3600798224}) 2024-01-24T23:25:17.029 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.025+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=784314932}) 2024-01-24T23:25:17.033 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.029+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.033 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.029+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=774150201}) 2024-01-24T23:25:17.034 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.029+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2860199251}) 2024-01-24T23:25:17.034 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.029+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.034 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.029+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=774150201}) 2024-01-24T23:25:17.034 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.029+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=90676077}) 2024-01-24T23:25:17.034 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.029+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.034 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.029+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=774150201}) 2024-01-24T23:25:17.034 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.029+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=582613546}) 2024-01-24T23:25:17.034 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.029+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.034 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.029+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=774150201}) 2024-01-24T23:25:17.034 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.029+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1815879939}) 2024-01-24T23:25:17.034 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.029+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.034 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.029+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=774150201}) 2024-01-24T23:25:17.034 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.029+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1352768437}) 2024-01-24T23:25:17.034 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.029+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.035 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.029+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=774150201}) 2024-01-24T23:25:17.035 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.029+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3653422839}) 2024-01-24T23:25:17.035 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.029+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.035 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.029+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=774150201}) 2024-01-24T23:25:17.035 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.029+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=380903851}) 2024-01-24T23:25:17.035 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.029+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.035 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.029+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=774150201}) 2024-01-24T23:25:17.035 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.029+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2384265076}) 2024-01-24T23:25:17.037 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.033+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.037 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.033+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3103289314}) 2024-01-24T23:25:17.037 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.033+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3911297767}) 2024-01-24T23:25:17.037 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.033+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.037 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.033+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3103289314}) 2024-01-24T23:25:17.037 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.033+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=92460380}) 2024-01-24T23:25:17.037 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.033+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.037 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.033+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3103289314}) 2024-01-24T23:25:17.037 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.033+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3540606248}) 2024-01-24T23:25:17.037 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.033+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.038 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.033+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3103289314}) 2024-01-24T23:25:17.038 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.033+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=989407945}) 2024-01-24T23:25:17.038 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.033+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.038 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.033+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3103289314}) 2024-01-24T23:25:17.038 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.033+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2401794636}) 2024-01-24T23:25:17.038 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.033+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.038 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.033+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3103289314}) 2024-01-24T23:25:17.038 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.033+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3867097267}) 2024-01-24T23:25:17.038 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.033+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.038 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.033+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3103289314}) 2024-01-24T23:25:17.038 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.033+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4283923812}) 2024-01-24T23:25:17.038 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.033+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.038 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.033+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3103289314}) 2024-01-24T23:25:17.038 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.033+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2958537716}) 2024-01-24T23:25:17.040 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.040 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3366773041}) 2024-01-24T23:25:17.040 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3476534139}) 2024-01-24T23:25:17.040 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.040 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3366773041}) 2024-01-24T23:25:17.040 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2059243402}) 2024-01-24T23:25:17.040 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.040 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3366773041}) 2024-01-24T23:25:17.040 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2898226151}) 2024-01-24T23:25:17.040 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.040 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3366773041}) 2024-01-24T23:25:17.041 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2916765625}) 2024-01-24T23:25:17.041 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.041 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3366773041}) 2024-01-24T23:25:17.041 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=424399592}) 2024-01-24T23:25:17.041 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.041 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3366773041}) 2024-01-24T23:25:17.041 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3320064767}) 2024-01-24T23:25:17.041 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.041 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3366773041}) 2024-01-24T23:25:17.041 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=864419855}) 2024-01-24T23:25:17.041 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.041 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3366773041}) 2024-01-24T23:25:17.041 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1321433885}) 2024-01-24T23:25:17.042 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-01-24T23:25:17.044 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.044 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2108441514}) 2024-01-24T23:25:17.044 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2523924112}) 2024-01-24T23:25:17.044 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.044 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2108441514}) 2024-01-24T23:25:17.044 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1509475007}) 2024-01-24T23:25:17.044 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.045 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2108441514}) 2024-01-24T23:25:17.045 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3899329326}) 2024-01-24T23:25:17.045 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.045 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2108441514}) 2024-01-24T23:25:17.045 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2020286266}) 2024-01-24T23:25:17.045 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.045 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2108441514}) 2024-01-24T23:25:17.045 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.037+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=553225292}) 2024-01-24T23:25:17.045 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.041+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.045 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.041+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2108441514}) 2024-01-24T23:25:17.045 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.041+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2061674809}) 2024-01-24T23:25:17.045 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.041+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.045 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.041+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2108441514}) 2024-01-24T23:25:17.045 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.041+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2163070582}) 2024-01-24T23:25:17.045 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.041+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.046 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.041+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2108441514}) 2024-01-24T23:25:17.046 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.041+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1464935507}) 2024-01-24T23:25:17.051 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.045+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.051 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.045+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2276288946}) 2024-01-24T23:25:17.051 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.045+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1834353008}) 2024-01-24T23:25:17.052 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.045+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.052 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.045+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2276288946}) 2024-01-24T23:25:17.052 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.045+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=61803213}) 2024-01-24T23:25:17.052 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.045+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.052 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.045+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2276288946}) 2024-01-24T23:25:17.052 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.045+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1786121062}) 2024-01-24T23:25:17.052 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.045+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.052 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.045+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2276288946}) 2024-01-24T23:25:17.052 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.045+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2457369573}) 2024-01-24T23:25:17.052 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.045+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.052 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.045+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2276288946}) 2024-01-24T23:25:17.052 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.045+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2185490664}) 2024-01-24T23:25:17.052 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.045+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.053 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.045+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2276288946}) 2024-01-24T23:25:17.053 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.045+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1062834398}) 2024-01-24T23:25:17.053 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.045+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.053 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.045+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2276288946}) 2024-01-24T23:25:17.053 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.045+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3814990147}) 2024-01-24T23:25:17.053 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.045+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.053 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.045+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2276288946}) 2024-01-24T23:25:17.053 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.045+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2605670153}) 2024-01-24T23:25:17.055 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.049+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.055 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.049+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1147122438}) 2024-01-24T23:25:17.055 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.049+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2268543408}) 2024-01-24T23:25:17.055 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.049+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.055 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.049+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1147122438}) 2024-01-24T23:25:17.055 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.049+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=787268434}) 2024-01-24T23:25:17.055 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.049+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.055 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.049+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1147122438}) 2024-01-24T23:25:17.055 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.049+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=168396998}) 2024-01-24T23:25:17.056 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.049+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.056 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.049+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1147122438}) 2024-01-24T23:25:17.056 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.049+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2970224253}) 2024-01-24T23:25:17.056 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.049+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.056 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.049+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1147122438}) 2024-01-24T23:25:17.056 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.049+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1828594847}) 2024-01-24T23:25:17.056 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.049+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.056 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.049+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1147122438}) 2024-01-24T23:25:17.056 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.049+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1358911050}) 2024-01-24T23:25:17.056 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.049+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.056 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.049+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1147122438}) 2024-01-24T23:25:17.056 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.049+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1040371473}) 2024-01-24T23:25:17.056 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.049+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.056 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.049+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1147122438}) 2024-01-24T23:25:17.057 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.049+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2574665448}) 2024-01-24T23:25:17.078 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.073+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.078 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.073+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2152564102}) 2024-01-24T23:25:17.078 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.073+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=74155465}) 2024-01-24T23:25:17.078 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.073+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.078 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.073+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2152564102}) 2024-01-24T23:25:17.078 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.073+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1210342465}) 2024-01-24T23:25:17.078 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.073+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.078 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.073+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2152564102}) 2024-01-24T23:25:17.078 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.073+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1358596347}) 2024-01-24T23:25:17.079 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.073+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.079 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.073+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2152564102}) 2024-01-24T23:25:17.079 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.073+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3280520818}) 2024-01-24T23:25:17.079 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.073+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.079 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.073+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2152564102}) 2024-01-24T23:25:17.079 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.073+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3489297837}) 2024-01-24T23:25:17.080 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.073+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.080 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.073+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2152564102}) 2024-01-24T23:25:17.080 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.073+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3581967057}) 2024-01-24T23:25:17.080 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.073+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.080 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.073+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2152564102}) 2024-01-24T23:25:17.080 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.073+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1905140416}) 2024-01-24T23:25:17.080 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.073+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.080 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.073+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2152564102}) 2024-01-24T23:25:17.080 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.073+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1409878459}) 2024-01-24T23:25:17.081 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.073+0000 7f96377fe640 1 -- 172.21.15.132:0/1215331924 <== mon.8 v1:172.21.15.115:6793/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set rm", "pool": "fooset"}]=0 v128) v1 ==== 89+0+0 (unknown 1076914200 0 0) 0x7f96300148b0 con 0x7f961c011040 2024-01-24T23:25:17.081 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.073+0000 7f96377fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set rm", "pool": "fooset"}] 2024-01-24T23:25:17.081 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.073+0000 7f96377fe640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:25:17.085 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.081+0000 7f9645781640 1 -- 172.21.15.132:0/1215331924 >> v1:172.21.15.132:6812/1835401922 conn(0x7f9628037620 legacy=0x7f9628039ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:17.085 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.081+0000 7f9645781640 10 monclient: shutdown 2024-01-24T23:25:17.085 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.081+0000 7f9645781640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:17.085 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.081+0000 7f9645781640 1 -- 172.21.15.132:0/1215331924 >> v1:172.21.15.115:6793/0 conn(0x7f961c011040 legacy=0x7f961c011480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:17.085 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.081+0000 7f9645781640 1 -- 172.21.15.132:0/1215331924 shutdown_connections 2024-01-24T23:25:17.085 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.081+0000 7f9645781640 1 -- 172.21.15.132:0/1215331924 >> 172.21.15.132:0/1215331924 conn(0x7f96400a2b40 msgr2=0x7f96400a3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:17.085 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.081+0000 7f9645781640 1 -- 172.21.15.132:0/1215331924 shutdown_connections 2024-01-24T23:25:17.085 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.081+0000 7f9645781640 1 -- 172.21.15.132:0/1215331924 wait complete. 2024-01-24T23:25:17.088 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.081+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.088 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.081+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4018723696}) 2024-01-24T23:25:17.088 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.081+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2883791020}) 2024-01-24T23:25:17.088 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.081+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.089 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.081+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4018723696}) 2024-01-24T23:25:17.089 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.081+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1324487231}) 2024-01-24T23:25:17.089 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.081+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.089 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.081+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4018723696}) 2024-01-24T23:25:17.089 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.081+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=756390785}) 2024-01-24T23:25:17.090 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.081+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.090 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.081+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4018723696}) 2024-01-24T23:25:17.090 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.081+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1249023332}) 2024-01-24T23:25:17.090 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.081+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.090 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.081+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4018723696}) 2024-01-24T23:25:17.090 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.081+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2893155396}) 2024-01-24T23:25:17.090 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.081+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.091 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.085+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4018723696}) 2024-01-24T23:25:17.091 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.085+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=638958088}) 2024-01-24T23:25:17.091 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.085+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.091 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.085+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4018723696}) 2024-01-24T23:25:17.091 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.085+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2011968154}) 2024-01-24T23:25:17.091 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.085+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.092 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.085+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4018723696}) 2024-01-24T23:25:17.092 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.085+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=651926194}) 2024-01-24T23:25:17.093 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.085+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.093 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.085+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3890407185}) 2024-01-24T23:25:17.094 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.085+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2170533046}) 2024-01-24T23:25:17.094 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.085+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.094 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3890407185}) 2024-01-24T23:25:17.094 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1110690265}) 2024-01-24T23:25:17.094 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.094 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3890407185}) 2024-01-24T23:25:17.094 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=604105305}) 2024-01-24T23:25:17.095 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.095 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3890407185}) 2024-01-24T23:25:17.095 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2550768252}) 2024-01-24T23:25:17.095 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.095 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3890407185}) 2024-01-24T23:25:17.095 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4134437602}) 2024-01-24T23:25:17.096 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.096 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3890407185}) 2024-01-24T23:25:17.096 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1557955826}) 2024-01-24T23:25:17.096 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.096 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3890407185}) 2024-01-24T23:25:17.096 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2646799757}) 2024-01-24T23:25:17.097 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.097 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3890407185}) 2024-01-24T23:25:17.097 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=64998229}) 2024-01-24T23:25:17.112 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.113 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=779319654}) 2024-01-24T23:25:17.113 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=450958179}) 2024-01-24T23:25:17.113 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.113 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=779319654}) 2024-01-24T23:25:17.113 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1881820527}) 2024-01-24T23:25:17.113 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.113 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=779319654}) 2024-01-24T23:25:17.113 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3771438504}) 2024-01-24T23:25:17.113 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.113 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=779319654}) 2024-01-24T23:25:17.114 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1735985321}) 2024-01-24T23:25:17.114 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.114 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=779319654}) 2024-01-24T23:25:17.114 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=412668541}) 2024-01-24T23:25:17.114 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.114 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=779319654}) 2024-01-24T23:25:17.114 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1402333651}) 2024-01-24T23:25:17.114 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.114 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=779319654}) 2024-01-24T23:25:17.114 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2287220077}) 2024-01-24T23:25:17.114 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.114 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=779319654}) 2024-01-24T23:25:17.114 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.089+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3487919264}) 2024-01-24T23:25:17.114 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.115 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3161754297}) 2024-01-24T23:25:17.115 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1990706662}) 2024-01-24T23:25:17.115 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.115 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3161754297}) 2024-01-24T23:25:17.115 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2956299702}) 2024-01-24T23:25:17.115 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.115 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3161754297}) 2024-01-24T23:25:17.115 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2946393974}) 2024-01-24T23:25:17.115 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.115 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3161754297}) 2024-01-24T23:25:17.115 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=108129658}) 2024-01-24T23:25:17.115 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.115 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3161754297}) 2024-01-24T23:25:17.115 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1531643249}) 2024-01-24T23:25:17.115 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.116 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3161754297}) 2024-01-24T23:25:17.116 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=22237742}) 2024-01-24T23:25:17.116 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.116 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3161754297}) 2024-01-24T23:25:17.116 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=702287626}) 2024-01-24T23:25:17.116 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.116 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3161754297}) 2024-01-24T23:25:17.116 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=542166967}) 2024-01-24T23:25:17.116 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.116 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2006561693}) 2024-01-24T23:25:17.116 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2664405961}) 2024-01-24T23:25:17.116 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.116 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2006561693}) 2024-01-24T23:25:17.116 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3921551591}) 2024-01-24T23:25:17.117 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.117 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2006561693}) 2024-01-24T23:25:17.117 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=304095791}) 2024-01-24T23:25:17.117 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.117 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2006561693}) 2024-01-24T23:25:17.117 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3564594101}) 2024-01-24T23:25:17.117 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.117 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2006561693}) 2024-01-24T23:25:17.117 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2435828759}) 2024-01-24T23:25:17.117 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.117 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2006561693}) 2024-01-24T23:25:17.117 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3127217282}) 2024-01-24T23:25:17.117 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.117 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2006561693}) 2024-01-24T23:25:17.117 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3796791380}) 2024-01-24T23:25:17.118 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.118 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2006561693}) 2024-01-24T23:25:17.118 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.093+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3732819562}) 2024-01-24T23:25:17.118 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.118 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1933504937}) 2024-01-24T23:25:17.118 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1213520540}) 2024-01-24T23:25:17.118 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.118 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1933504937}) 2024-01-24T23:25:17.118 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3290727652}) 2024-01-24T23:25:17.118 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.118 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1933504937}) 2024-01-24T23:25:17.118 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1996011811}) 2024-01-24T23:25:17.118 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.118 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1933504937}) 2024-01-24T23:25:17.119 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4140488310}) 2024-01-24T23:25:17.119 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.119 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1933504937}) 2024-01-24T23:25:17.119 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=247918062}) 2024-01-24T23:25:17.119 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.119 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1933504937}) 2024-01-24T23:25:17.119 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3702151919}) 2024-01-24T23:25:17.119 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.119 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1933504937}) 2024-01-24T23:25:17.119 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=723934548}) 2024-01-24T23:25:17.119 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.119 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1933504937}) 2024-01-24T23:25:17.119 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3984184006}) 2024-01-24T23:25:17.119 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.119 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1907261847}) 2024-01-24T23:25:17.119 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=674768819}) 2024-01-24T23:25:17.120 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.120 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1907261847}) 2024-01-24T23:25:17.120 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=750494735}) 2024-01-24T23:25:17.120 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.120 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1907261847}) 2024-01-24T23:25:17.120 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1460203255}) 2024-01-24T23:25:17.120 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.120 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1907261847}) 2024-01-24T23:25:17.120 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=714477879}) 2024-01-24T23:25:17.120 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.120 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1907261847}) 2024-01-24T23:25:17.120 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3584728400}) 2024-01-24T23:25:17.120 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.120 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1907261847}) 2024-01-24T23:25:17.121 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2398641992}) 2024-01-24T23:25:17.121 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.121 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1907261847}) 2024-01-24T23:25:17.121 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1725162688}) 2024-01-24T23:25:17.121 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.121 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1907261847}) 2024-01-24T23:25:17.121 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.097+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1304150842}) 2024-01-24T23:25:17.121 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.121 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=135048310}) 2024-01-24T23:25:17.121 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3302357571}) 2024-01-24T23:25:17.121 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.121 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=135048310}) 2024-01-24T23:25:17.121 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=129170158}) 2024-01-24T23:25:17.121 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.121 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=135048310}) 2024-01-24T23:25:17.121 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3798353689}) 2024-01-24T23:25:17.122 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.122 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=135048310}) 2024-01-24T23:25:17.122 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=78067922}) 2024-01-24T23:25:17.122 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.122 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=135048310}) 2024-01-24T23:25:17.122 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1816680752}) 2024-01-24T23:25:17.122 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.122 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=135048310}) 2024-01-24T23:25:17.122 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=739654714}) 2024-01-24T23:25:17.122 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.122 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=135048310}) 2024-01-24T23:25:17.122 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2497092182}) 2024-01-24T23:25:17.122 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.122 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=135048310}) 2024-01-24T23:25:17.122 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4040468800}) 2024-01-24T23:25:17.123 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.123 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=290451738}) 2024-01-24T23:25:17.123 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2808683969}) 2024-01-24T23:25:17.123 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.123 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=290451738}) 2024-01-24T23:25:17.123 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=599238468}) 2024-01-24T23:25:17.123 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.123 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=290451738}) 2024-01-24T23:25:17.123 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3505758431}) 2024-01-24T23:25:17.123 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.123 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=290451738}) 2024-01-24T23:25:17.123 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=237312593}) 2024-01-24T23:25:17.123 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.123 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=290451738}) 2024-01-24T23:25:17.123 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=389409749}) 2024-01-24T23:25:17.124 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.124 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=290451738}) 2024-01-24T23:25:17.124 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3061731634}) 2024-01-24T23:25:17.124 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.124 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=290451738}) 2024-01-24T23:25:17.124 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1946981130}) 2024-01-24T23:25:17.124 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.124 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=290451738}) 2024-01-24T23:25:17.124 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.101+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1616606402}) 2024-01-24T23:25:17.124 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.124 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2084230474}) 2024-01-24T23:25:17.124 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3383629252}) 2024-01-24T23:25:17.124 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.124 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2084230474}) 2024-01-24T23:25:17.124 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=298653386}) 2024-01-24T23:25:17.125 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.125 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2084230474}) 2024-01-24T23:25:17.125 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2056621699}) 2024-01-24T23:25:17.125 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.125 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2084230474}) 2024-01-24T23:25:17.125 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1009296350}) 2024-01-24T23:25:17.125 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.125 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2084230474}) 2024-01-24T23:25:17.125 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3659500438}) 2024-01-24T23:25:17.125 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.125 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2084230474}) 2024-01-24T23:25:17.125 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=895998671}) 2024-01-24T23:25:17.125 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.125 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2084230474}) 2024-01-24T23:25:17.125 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1889640772}) 2024-01-24T23:25:17.125 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.126 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2084230474}) 2024-01-24T23:25:17.126 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4293771439}) 2024-01-24T23:25:17.126 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.126 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=80915050}) 2024-01-24T23:25:17.126 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2207820737}) 2024-01-24T23:25:17.126 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.126 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=80915050}) 2024-01-24T23:25:17.126 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1011370256}) 2024-01-24T23:25:17.126 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.126 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=80915050}) 2024-01-24T23:25:17.126 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2078630337}) 2024-01-24T23:25:17.126 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.126 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=80915050}) 2024-01-24T23:25:17.127 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3243560501}) 2024-01-24T23:25:17.127 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.127 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=80915050}) 2024-01-24T23:25:17.127 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3403825154}) 2024-01-24T23:25:17.127 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.127 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=80915050}) 2024-01-24T23:25:17.127 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2279714018}) 2024-01-24T23:25:17.127 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.127 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=80915050}) 2024-01-24T23:25:17.127 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2241318584}) 2024-01-24T23:25:17.127 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.127 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=80915050}) 2024-01-24T23:25:17.127 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.105+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1949025060}) 2024-01-24T23:25:17.128 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set ls 2024-01-24T23:25:17.128 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep -c fooset 2024-01-24T23:25:17.128 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep -q 0 2024-01-24T23:25:17.134 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.134 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=44,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=17} crc {logm=3296081668,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=3068609737,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=4126566697,osdmap=3974703224}) 2024-01-24T23:25:17.134 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=45,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=7,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=18} crc {logm=2103550195,mdsmap=3984425268,mgr=2746627711,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=3068609737,mon_config_key=4139452546,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=4126566697,osdmap=1210623644}) 2024-01-24T23:25:17.135 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.135 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=44,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=17} crc {logm=3296081668,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=3068609737,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=4126566697,osdmap=3974703224}) 2024-01-24T23:25:17.135 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=46,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=8,monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=18} crc {logm=2758759463,mdsmap=3984425268,mgr=3539714768,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=698102599,mon_config_key=4121867429,monmap=701157023,osd_metadata=2031414162,osd_pg_creating=4126566697,osdmap=1210623644}) 2024-01-24T23:25:17.135 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.135 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=44,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=17} crc {logm=3296081668,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=3068609737,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=4126566697,osdmap=3974703224}) 2024-01-24T23:25:17.135 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=45,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=17} crc {logm=1799176155,mdsmap=3984425268,mgr=1221499742,mgr_metadata=2479342215,mgrstat=3068609737,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=4126566697,osdmap=3974703224}) 2024-01-24T23:25:17.135 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.135 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=44,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=17} crc {logm=3296081668,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=3068609737,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=4126566697,osdmap=3974703224}) 2024-01-24T23:25:17.135 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=43,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=8,monmap=3,osd_metadata=6,osdmap=20} crc {logm=2038380812,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=1094647966,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osdmap=2271707479}) 2024-01-24T23:25:17.135 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.135 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=44,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=17} crc {logm=3296081668,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=3068609737,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=4126566697,osdmap=3974703224}) 2024-01-24T23:25:17.135 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=42,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=7,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=21} crc {logm=4290608218,mdsmap=3984425268,mgr=1117054149,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=3068609737,mon_config_key=2187968908,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=4126566697,osdmap=3852795236}) 2024-01-24T23:25:17.135 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.135 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=44,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=17} crc {logm=3296081668,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=3068609737,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=4126566697,osdmap=3974703224}) 2024-01-24T23:25:17.135 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=46,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=8,monmap=2,osd_metadata=5,osd_pg_creating=1,osdmap=19} crc {logm=94293306,mdsmap=909706845,mgr=1093705807,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2601874452,mon_config_key=4121867429,monmap=1293597301,osd_metadata=3543209436,osd_pg_creating=4126566697,osdmap=4208682790}) 2024-01-24T23:25:17.136 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.136 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=44,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=17} crc {logm=3296081668,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=3068609737,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=4126566697,osdmap=3974703224}) 2024-01-24T23:25:17.136 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=45,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=17} crc {logm=2932815026,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2470511015,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=4126566697,osdmap=3974703224}) 2024-01-24T23:25:17.136 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.136 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=44,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=17} crc {logm=3296081668,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=3068609737,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=4126566697,osdmap=3974703224}) 2024-01-24T23:25:17.136 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=46,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=8,monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=17} crc {logm=3064933155,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=4187666807,mon_config_key=4121867429,monmap=701157023,osd_metadata=2031414162,osd_pg_creating=4126566697,osdmap=3404961739}) 2024-01-24T23:25:17.136 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.136 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2945306443}) 2024-01-24T23:25:17.136 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2082096953}) 2024-01-24T23:25:17.136 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.136 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2945306443}) 2024-01-24T23:25:17.136 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2306320481}) 2024-01-24T23:25:17.136 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.136 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2945306443}) 2024-01-24T23:25:17.136 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=695732323}) 2024-01-24T23:25:17.137 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.137 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2945306443}) 2024-01-24T23:25:17.137 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1321360006}) 2024-01-24T23:25:17.137 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.137 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2945306443}) 2024-01-24T23:25:17.137 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1134462850}) 2024-01-24T23:25:17.137 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.137 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2945306443}) 2024-01-24T23:25:17.137 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2380535252}) 2024-01-24T23:25:17.137 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.137 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2945306443}) 2024-01-24T23:25:17.137 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3968220679}) 2024-01-24T23:25:17.137 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.137 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2945306443}) 2024-01-24T23:25:17.137 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.109+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1577892348}) 2024-01-24T23:25:17.137 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.113+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.138 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.113+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=81} crc {osdmap=3382413633}) 2024-01-24T23:25:17.138 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.113+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=81} crc {osdmap=4293207996}) 2024-01-24T23:25:17.138 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.113+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.138 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.113+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=81} crc {osdmap=3382413633}) 2024-01-24T23:25:17.138 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.113+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=79} crc {osdmap=136029305}) 2024-01-24T23:25:17.138 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.113+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.138 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.113+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=81} crc {osdmap=3382413633}) 2024-01-24T23:25:17.138 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.113+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=80} crc {osdmap=2817464020}) 2024-01-24T23:25:17.138 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.113+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.138 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.113+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=81} crc {osdmap=3382413633}) 2024-01-24T23:25:17.138 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.113+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=84} crc {osdmap=3180474506}) 2024-01-24T23:25:17.138 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.113+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.138 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.113+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=81} crc {osdmap=3382413633}) 2024-01-24T23:25:17.138 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.113+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=79} crc {osdmap=498958084}) 2024-01-24T23:25:17.138 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.113+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.138 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.113+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=81} crc {osdmap=3382413633}) 2024-01-24T23:25:17.139 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.113+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=78} crc {osdmap=835477320}) 2024-01-24T23:25:17.139 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.113+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.139 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.113+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=81} crc {osdmap=3382413633}) 2024-01-24T23:25:17.139 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.113+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=82} crc {osdmap=875537833}) 2024-01-24T23:25:17.139 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.113+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:25:17.139 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.113+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=81} crc {osdmap=3382413633}) 2024-01-24T23:25:17.139 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:25:17.113+0000 7fd2d64cd640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=81} crc {osdmap=1872276266}) 2024-01-24T23:25:17.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e7b6ad640 10 monclient: get_monmap_and_config 2024-01-24T23:25:17.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e7b6ad640 10 monclient: build_initial_monmap 2024-01-24T23:25:17.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e7b6ad640 10 monclient: monmap: 2024-01-24T23:25:17.229 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:17.229 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:17.229 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:17.230389+0000 2024-01-24T23:25:17.229 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:17.230389+0000 2024-01-24T23:25:17.229 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:17.230 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:17.230 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:17.230 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:17.230 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:17.230 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:17.230 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:17.230 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:17.230 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:17.230 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:17.230 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:17.230 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:17.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e7b6ad640 1 Processor -- start 2024-01-24T23:25:17.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e7b6ad640 1 -- start start 2024-01-24T23:25:17.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e7b6ad640 10 monclient: init 2024-01-24T23:25:17.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e7b6ad640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:17.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e7b6ad640 10 monclient: _add_conns ranks=[2,6,0,4,7,3,1,8,5] 2024-01-24T23:25:17.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e7b6ad640 10 monclient(hunting): picked mon.noname-b con 0x7f7e7409c4c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:17.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e7b6ad640 10 monclient(hunting): picked mon.noname-d con 0x7f7e7409cfb0 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:17.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e7b6ad640 10 monclient(hunting): picked mon.noname-a con 0x7f7e7409ec50 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:17.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e7b6ad640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e74064700 con 0x7f7e7409ec50 2024-01-24T23:25:17.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e7b6ad640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e7409d9f0 con 0x7f7e7409c4c0 2024-01-24T23:25:17.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e7b6ad640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e7409dc60 con 0x7f7e7409cfb0 2024-01-24T23:25:17.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e7b6ad640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:17.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e7b6ad640 10 monclient(hunting): authenticate will time out at 1204.554983s 2024-01-24T23:25:17.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e79449640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f7e7409c4c0 0x7f7e7409c8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:52122/0 (socket says 172.21.15.132:52122) 2024-01-24T23:25:17.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e79449640 1 -- 172.21.15.132:0/2364439530 learned_addr learned my addr 172.21.15.132:0/2364439530 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:17.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e6bfff640 1 -- 172.21.15.132:0/2364439530 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3736066020 0 0) 0x7f7e7409dc60 con 0x7f7e7409cfb0 2024-01-24T23:25:17.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e6bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:17.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e6bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:17.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e6bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:17.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e6bfff640 10 monclient(hunting): my global_id is 6009 2024-01-24T23:25:17.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e6bfff640 1 -- 172.21.15.132:0/2364439530 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e60003530 con 0x7f7e7409cfb0 2024-01-24T23:25:17.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e6bfff640 1 -- 172.21.15.132:0/2364439530 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2960869857 0 0) 0x7f7e7409d9f0 con 0x7f7e7409c4c0 2024-01-24T23:25:17.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e6bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:17.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e6bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:17.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e6bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:17.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e6bfff640 10 monclient(hunting): my global_id is 85259 2024-01-24T23:25:17.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e6bfff640 1 -- 172.21.15.132:0/2364439530 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e7409dc60 con 0x7f7e7409c4c0 2024-01-24T23:25:17.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e6bfff640 1 -- 172.21.15.132:0/2364439530 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3557780716 0 0) 0x7f7e74064700 con 0x7f7e7409ec50 2024-01-24T23:25:17.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e6bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:17.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e6bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:17.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e6bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:17.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e6bfff640 10 monclient(hunting): my global_id is 94221 2024-01-24T23:25:17.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.225+0000 7f7e6bfff640 1 -- 172.21.15.132:0/2364439530 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e7409d9f0 con 0x7f7e7409ec50 2024-01-24T23:25:17.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e6bfff640 1 -- 172.21.15.132:0/2364439530 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2012778282 0 0) 0x7f7e7409d9f0 con 0x7f7e7409ec50 2024-01-24T23:25:17.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e6bfff640 1 -- 172.21.15.132:0/2364439530 >> v1:172.21.15.115:6792/0 conn(0x7f7e7409cfb0 legacy=0x7f7e74138d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:17.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e6bfff640 1 -- 172.21.15.132:0/2364439530 >> v1:172.21.15.115:6790/0 conn(0x7f7e7409c4c0 legacy=0x7f7e7409c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:17.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e6bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:25:17.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e6bfff640 1 monclient: found mon.noname-a 2024-01-24T23:25:17.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e6bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:17.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e6bfff640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:25:17.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e6bfff640 1 -- 172.21.15.132:0/2364439530 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7409df40 con 0x7f7e7409ec50 2024-01-24T23:25:17.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e6bfff640 10 monclient: _finish_auth 0 2024-01-24T23:25:17.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e6bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:25:17.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e6bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:17.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e6bfff640 1 -- 172.21.15.132:0/2364439530 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e6c001ae0 con 0x7f7e7409ec50 2024-01-24T23:25:17.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 5 monclient: authenticate success, global_id 94221 2024-01-24T23:25:17.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:17.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e6bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:17.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e6bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:25:17.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e6bfff640 10 monclient: dump: 2024-01-24T23:25:17.241 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:17.241 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:17.241 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:17.242 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:17.242 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:17.242 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:17.242 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:17.242 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:17.242 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:17.242 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:17.242 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:17.242 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:17.242 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:17.242 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:17.242 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:17.242 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:17.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:17.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e6bfff640 1 -- 172.21.15.132:0/2364439530 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7e6c001d00 con 0x7f7e7409ec50 2024-01-24T23:25:17.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e6bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:17.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:17.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e6bfff640 1 -- 172.21.15.132:0/2364439530 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e6c0039a0 con 0x7f7e7409ec50 2024-01-24T23:25:17.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 4 set_mon_vals no callback set 2024-01-24T23:25:17.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e6bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:17.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e6bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:17.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e6bfff640 10 monclient: dump: 2024-01-24T23:25:17.243 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:17.243 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:17.243 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:17.243 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:17.243 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:17.243 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:17.243 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:17.243 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:17.244 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:17.244 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:17.244 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:17.244 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:17.244 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:17.244 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:17.244 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:17.244 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:17.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 10 monclient: shutdown 2024-01-24T23:25:17.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:17.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 1 -- 172.21.15.132:0/2364439530 >> v1:172.21.15.115:6789/0 conn(0x7f7e7409ec50 legacy=0x7f7e7413c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:17.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 1 -- 172.21.15.132:0/2364439530 shutdown_connections 2024-01-24T23:25:17.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 1 -- 172.21.15.132:0/2364439530 >> 172.21.15.132:0/2364439530 conn(0x7f7e740a5a10 msgr2=0x7f7e740a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:17.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 1 -- 172.21.15.132:0/2364439530 shutdown_connections 2024-01-24T23:25:17.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 1 -- 172.21.15.132:0/2364439530 wait complete. 2024-01-24T23:25:17.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 10 monclient: build_initial_monmap 2024-01-24T23:25:17.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 10 monclient: monmap: 2024-01-24T23:25:17.245 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:17.245 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:17.245 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:17.245 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:17.245 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:17.245 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:17.245 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:17.245 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:17.245 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:17.245 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:17.246 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:17.246 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:17.246 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:17.246 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:17.246 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:17.246 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:17.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 1 Processor -- start 2024-01-24T23:25:17.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 1 -- start start 2024-01-24T23:25:17.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 10 monclient: init 2024-01-24T23:25:17.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:17.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 10 monclient: _add_conns ranks=[8,6,4,1,0,2,7,3,5] 2024-01-24T23:25:17.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 10 monclient(hunting): picked mon.noname-e con 0x7f7e7409c4c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:17.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 10 monclient(hunting): picked mon.noname-d con 0x7f7e7409cfb0 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:17.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 10 monclient(hunting): picked mon.noname-c con 0x7f7e7409ec50 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:17.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e7409c020 con 0x7f7e7409ec50 2024-01-24T23:25:17.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e741233e0 con 0x7f7e7409cfb0 2024-01-24T23:25:17.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e74134b00 con 0x7f7e7409c4c0 2024-01-24T23:25:17.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:17.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e7b6ad640 10 monclient(hunting): authenticate will time out at 1204.558846s 2024-01-24T23:25:17.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e79c4a640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f7e7409ec50 0x7f7e741d3b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:34640/0 (socket says 172.21.15.132:34640) 2024-01-24T23:25:17.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e79c4a640 1 -- 172.21.15.132:0/2883406973 learned_addr learned my addr 172.21.15.132:0/2883406973 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:17.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616732320 0 0) 0x7f7e7409c020 con 0x7f7e7409ec50 2024-01-24T23:25:17.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e69ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:17.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e69ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:17.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e69ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:17.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e69ffb640 10 monclient(hunting): my global_id is 94207 2024-01-24T23:25:17.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e58003560 con 0x7f7e7409ec50 2024-01-24T23:25:17.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 45527640 0 0) 0x7f7e74134b00 con 0x7f7e7409c4c0 2024-01-24T23:25:17.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e69ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:17.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e69ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:17.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e69ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:17.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e69ffb640 10 monclient(hunting): my global_id is 85112 2024-01-24T23:25:17.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e7409c020 con 0x7f7e7409c4c0 2024-01-24T23:25:17.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2490283410 0 0) 0x7f7e741233e0 con 0x7f7e7409cfb0 2024-01-24T23:25:17.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e69ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:17.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e69ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:17.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e69ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:17.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e69ffb640 10 monclient(hunting): my global_id is 6018 2024-01-24T23:25:17.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.229+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e74134b00 con 0x7f7e7409cfb0 2024-01-24T23:25:17.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2704566003 0 0) 0x7f7e58003560 con 0x7f7e7409ec50 2024-01-24T23:25:17.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7e741233e0 con 0x7f7e7409ec50 2024-01-24T23:25:17.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e6c001a20 con 0x7f7e7409ec50 2024-01-24T23:25:17.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:17.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:25:17.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 10 monclient(hunting): dump: 2024-01-24T23:25:17.249 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:17.249 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:17.249 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:17.249 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:17.249 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:17.249 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:17.249 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:17.249 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:17.249 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:17.249 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:17.249 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:17.249 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:17.249 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:17.249 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:17.250 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:17.250 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:17.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:17.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2106035750 0 0) 0x7f7e74134b00 con 0x7f7e7409cfb0 2024-01-24T23:25:17.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7e58003560 con 0x7f7e7409cfb0 2024-01-24T23:25:17.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e64001580 con 0x7f7e7409cfb0 2024-01-24T23:25:17.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:17.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:17.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 10 monclient(hunting): dump: 2024-01-24T23:25:17.250 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:17.250 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:17.250 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:17.250 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:17.250 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:17.250 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:17.251 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:17.251 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:17.251 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:17.251 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:17.251 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:17.251 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:17.251 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:17.251 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:17.251 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:17.251 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:17.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 717033306 0 0) 0x7f7e7409c020 con 0x7f7e7409c4c0 2024-01-24T23:25:17.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7e74134b00 con 0x7f7e7409c4c0 2024-01-24T23:25:17.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e70001a60 con 0x7f7e7409c4c0 2024-01-24T23:25:17.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:17.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:17.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 10 monclient(hunting): dump: 2024-01-24T23:25:17.252 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:17.252 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:17.252 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:17.252 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:17.252 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:17.252 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:17.252 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:17.252 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:17.252 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:17.252 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:17.252 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:17.252 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:17.252 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:17.252 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:17.253 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:17.253 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:17.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1633072069 0 0) 0x7f7e741233e0 con 0x7f7e7409ec50 2024-01-24T23:25:17.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 >> v1:172.21.15.115:6793/0 conn(0x7f7e7409c4c0 legacy=0x7f7e74133a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:17.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 >> v1:172.21.15.115:6792/0 conn(0x7f7e7409cfb0 legacy=0x7f7e741d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:17.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:25:17.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 1 monclient: found mon.c 2024-01-24T23:25:17.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:17.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:17.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e740a13b0 con 0x7f7e7409ec50 2024-01-24T23:25:17.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 10 monclient: _finish_auth 0 2024-01-24T23:25:17.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:25:17.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:17.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e7b6ad640 5 monclient: authenticate success, global_id 94207 2024-01-24T23:25:17.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e7b6ad640 10 monclient: _renew_subs 2024-01-24T23:25:17.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e7b6ad640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:17.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e7b6ad640 1 -- 172.21.15.132:0/2883406973 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e7406e9f0 con 0x7f7e7409ec50 2024-01-24T23:25:17.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e7b6ad640 10 monclient: _renew_subs 2024-01-24T23:25:17.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e7b6ad640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:17.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e7b6ad640 1 -- 172.21.15.132:0/2883406973 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e74121c50 con 0x7f7e7409ec50 2024-01-24T23:25:17.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7e6c003e80 con 0x7f7e7409ec50 2024-01-24T23:25:17.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:17.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e6b7fe640 4 set_mon_vals no callback set 2024-01-24T23:25:17.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e6c0043b0 con 0x7f7e7409ec50 2024-01-24T23:25:17.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:17.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:17.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 10 monclient: dump: 2024-01-24T23:25:17.254 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:17.254 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:17.254 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:17.254 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:17.255 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:17.255 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:17.255 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:17.255 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:17.255 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:17.255 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:17.255 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:17.255 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:17.255 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:17.255 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:17.255 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:17.255 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:17.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e7b6ad640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:17.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e7b6ad640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:17.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e7b6ad640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:17.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e7b6ad640 1 -- 172.21.15.132:0/2883406973 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e54005090 con 0x7f7e7409ec50 2024-01-24T23:25:17.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f7e6c004590 con 0x7f7e7409ec50 2024-01-24T23:25:17.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(128..128 src has 1..128) v4 ==== 6989+0+0 (unknown 538550023 0 0) 0x7f7e6c044810 con 0x7f7e7409ec50 2024-01-24T23:25:17.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.233+0000 7f7e79449640 10 monclient: get_auth_request con 0x7f7e58037cc0 auth_method 0 2024-01-24T23:25:17.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.245+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f7e6c014c20 con 0x7f7e7409ec50 2024-01-24T23:25:17.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.245+0000 7f7e69ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:17.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.245+0000 7f7e69ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:17.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.409+0000 7f7e7b6ad640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:17.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.409+0000 7f7e7b6ad640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:17.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.409+0000 7f7e7b6ad640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:17.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.409+0000 7f7e7b6ad640 1 -- 172.21.15.132:0/2883406973 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7f7e54005340 con 0x7f7e7409ec50 2024-01-24T23:25:17.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.409+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v128) v1 ==== 71+0+7 (unknown 43352416 0 1352219128) 0x7f7e6c014a40 con 0x7f7e7409ec50 2024-01-24T23:25:17.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.409+0000 7f7e69ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:17.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.409+0000 7f7e69ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:17.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.509+0000 7f7e7b6ad640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:17.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.509+0000 7f7e7b6ad640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:17.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.509+0000 7f7e7b6ad640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:17.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.509+0000 7f7e7b6ad640 1 -- 172.21.15.132:0/2883406973 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7f7e54001130 con 0x7f7e7409ec50 2024-01-24T23:25:17.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.509+0000 7f7e69ffb640 1 -- 172.21.15.132:0/2883406973 <== mon.4 v1:172.21.15.115:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v128) v1 ==== 71+0+7 (unknown 43352416 0 1352219128) 0x7f7e6c014a40 con 0x7f7e7409ec50 2024-01-24T23:25:17.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.509+0000 7f7e69ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:17.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.509+0000 7f7e69ffb640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:25:17.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.509+0000 7f7e7b6ad640 1 -- 172.21.15.132:0/2883406973 >> v1:172.21.15.132:6812/1835401922 conn(0x7f7e58037cc0 legacy=0x7f7e5803a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:17.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.509+0000 7f7e7b6ad640 10 monclient: shutdown 2024-01-24T23:25:17.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.509+0000 7f7e7b6ad640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:17.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.509+0000 7f7e7b6ad640 1 -- 172.21.15.132:0/2883406973 >> v1:172.21.15.115:6791/0 conn(0x7f7e7409ec50 legacy=0x7f7e741d3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:17.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.509+0000 7f7e7b6ad640 1 -- 172.21.15.132:0/2883406973 shutdown_connections 2024-01-24T23:25:17.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.509+0000 7f7e7b6ad640 1 -- 172.21.15.132:0/2883406973 >> 172.21.15.132:0/2883406973 conn(0x7f7e740a5a10 msgr2=0x7f7e74066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:17.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.509+0000 7f7e7b6ad640 1 -- 172.21.15.132:0/2883406973 shutdown_connections 2024-01-24T23:25:17.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.509+0000 7f7e7b6ad640 1 -- 172.21.15.132:0/2883406973 wait complete. 2024-01-24T23:25:17.530 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set ls 2024-01-24T23:25:17.530 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep barset 2024-01-24T23:25:17.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.637+0000 7fc984b41640 10 monclient: get_monmap_and_config 2024-01-24T23:25:17.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.637+0000 7fc984b41640 10 monclient: build_initial_monmap 2024-01-24T23:25:17.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.637+0000 7fc984b41640 10 monclient: monmap: 2024-01-24T23:25:17.642 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:17.642 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:17.642 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:17.643406+0000 2024-01-24T23:25:17.642 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:17.643406+0000 2024-01-24T23:25:17.642 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:17.643 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:17.643 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:17.643 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:17.643 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:17.643 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:17.643 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:17.643 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:17.643 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:17.643 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:17.643 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:17.643 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:17.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.637+0000 7fc984b41640 1 Processor -- start 2024-01-24T23:25:17.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.637+0000 7fc984b41640 1 -- start start 2024-01-24T23:25:17.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.637+0000 7fc984b41640 10 monclient: init 2024-01-24T23:25:17.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.637+0000 7fc984b41640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:17.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.637+0000 7fc984b41640 10 monclient: _add_conns ranks=[7,6,3,1,5,8,0,4,2] 2024-01-24T23:25:17.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 10 monclient(hunting): picked mon.noname-i con 0x7fc980136e50 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:17.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 10 monclient(hunting): picked mon.noname-d con 0x7fc9800a0290 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:17.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 10 monclient(hunting): picked mon.noname-g con 0x7fc9800a1600 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:17.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc980064700 con 0x7fc9800a1600 2024-01-24T23:25:17.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc980137940 con 0x7fc9800a0290 2024-01-24T23:25:17.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc98009f9c0 con 0x7fc980136e50 2024-01-24T23:25:17.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:17.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 10 monclient(hunting): authenticate will time out at 1204.968083s 2024-01-24T23:25:17.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97ed9d640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fc9800a1600 0x7fc98013f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:56194/0 (socket says 172.21.15.132:56194) 2024-01-24T23:25:17.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97ed9d640 1 -- 172.21.15.132:0/330221052 learned_addr learned my addr 172.21.15.132:0/330221052 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:17.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97e59c640 1 --1- 172.21.15.132:0/330221052 >> v1:172.21.15.132:6792/0 conn(0x7fc980136e50 0x7fc980137230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:59964/0 (socket says 172.21.15.132:59964) 2024-01-24T23:25:17.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 1 -- 172.21.15.132:0/330221052 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1588952512 0 0) 0x7fc98009f9c0 con 0x7fc980136e50 2024-01-24T23:25:17.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 10 monclient(hunting): _negotiate 2024-01-24T23:25:17.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:17.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:17.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 10 monclient(hunting): my global_id is 5884 2024-01-24T23:25:17.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 1 -- 172.21.15.132:0/330221052 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc960003530 con 0x7fc980136e50 2024-01-24T23:25:17.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 1 -- 172.21.15.132:0/330221052 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3887675877 0 0) 0x7fc980064700 con 0x7fc9800a1600 2024-01-24T23:25:17.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 10 monclient(hunting): _negotiate 2024-01-24T23:25:17.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:17.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:17.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 10 monclient(hunting): my global_id is 5898 2024-01-24T23:25:17.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 1 -- 172.21.15.132:0/330221052 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc98009f9c0 con 0x7fc9800a1600 2024-01-24T23:25:17.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 1 -- 172.21.15.132:0/330221052 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2224470504 0 0) 0x7fc960003530 con 0x7fc980136e50 2024-01-24T23:25:17.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 1 -- 172.21.15.132:0/330221052 >> v1:172.21.15.115:6792/0 conn(0x7fc9800a0290 legacy=0x7fc98013bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:17.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 1 -- 172.21.15.132:0/330221052 >> v1:172.21.15.132:6790/0 conn(0x7fc9800a1600 legacy=0x7fc98013f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:17.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 10 monclient: _finish_hunting 0 2024-01-24T23:25:17.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 1 monclient: found mon.noname-i 2024-01-24T23:25:17.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:17.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:25:17.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 1 -- 172.21.15.132:0/330221052 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98009eda0 con 0x7fc980136e50 2024-01-24T23:25:17.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 10 monclient: _finish_auth 0 2024-01-24T23:25:17.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 10 monclient: _check_auth_tickets 2024-01-24T23:25:17.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:17.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 5 monclient: authenticate success, global_id 5884 2024-01-24T23:25:17.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:17.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 1 -- 172.21.15.132:0/330221052 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc9680017c0 con 0x7fc980136e50 2024-01-24T23:25:17.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:17.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:25:17.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 10 monclient: dump: 2024-01-24T23:25:17.655 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:17.655 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:17.655 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:17.655 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:17.655 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:17.655 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:17.655 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:17.655 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:17.655 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:17.655 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:17.655 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:17.655 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:17.655 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:17.656 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:17.656 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:17.656 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:17.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:17.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 1 -- 172.21.15.132:0/330221052 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc968001a60 con 0x7fc980136e50 2024-01-24T23:25:17.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:17.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 1 -- 172.21.15.132:0/330221052 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc968004b30 con 0x7fc980136e50 2024-01-24T23:25:17.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:17.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:25:17.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc97dd9b640 10 monclient: dump: 2024-01-24T23:25:17.656 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:17.656 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:17.656 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:17.656 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:17.656 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:17.657 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:17.657 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:17.657 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:17.657 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:17.657 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:17.657 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:17.657 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:17.657 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:17.657 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:17.657 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:17.657 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:17.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:17.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 4 set_mon_vals no callback set 2024-01-24T23:25:17.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 10 monclient: shutdown 2024-01-24T23:25:17.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:17.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 1 -- 172.21.15.132:0/330221052 >> v1:172.21.15.132:6792/0 conn(0x7fc980136e50 legacy=0x7fc980137230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:17.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 1 -- 172.21.15.132:0/330221052 shutdown_connections 2024-01-24T23:25:17.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 1 -- 172.21.15.132:0/330221052 >> 172.21.15.132:0/330221052 conn(0x7fc9800a5a10 msgr2=0x7fc9800a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:17.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 1 -- 172.21.15.132:0/330221052 shutdown_connections 2024-01-24T23:25:17.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 1 -- 172.21.15.132:0/330221052 wait complete. 2024-01-24T23:25:17.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 10 monclient: build_initial_monmap 2024-01-24T23:25:17.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 10 monclient: monmap: 2024-01-24T23:25:17.658 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:17.658 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:17.658 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:17.658 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:17.658 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:17.658 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:17.658 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:17.659 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:17.659 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:17.659 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:17.659 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:17.659 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:17.659 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:17.659 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:17.659 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:17.659 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:17.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 1 Processor -- start 2024-01-24T23:25:17.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 1 -- start start 2024-01-24T23:25:17.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 10 monclient: init 2024-01-24T23:25:17.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:17.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 10 monclient: _add_conns ranks=[0,2,4,5,8,7,3,1,6] 2024-01-24T23:25:17.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 10 monclient(hunting): picked mon.noname-a con 0x7fc9800a0290 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:17.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 10 monclient(hunting): picked mon.noname-b con 0x7fc9800a1600 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:17.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 10 monclient(hunting): picked mon.noname-c con 0x7fc980136e50 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:17.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc980120270 con 0x7fc9800a0290 2024-01-24T23:25:17.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc980122560 con 0x7fc9800a1600 2024-01-24T23:25:17.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc98009ea70 con 0x7fc980136e50 2024-01-24T23:25:17.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:17.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc984b41640 10 monclient(hunting): authenticate will time out at 1204.971302s 2024-01-24T23:25:17.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc975d9b640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fc9800a1600 0x7fc9801d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:52130/0 (socket says 172.21.15.132:52130) 2024-01-24T23:25:17.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.641+0000 7fc975d9b640 1 -- 172.21.15.132:0/3514222784 learned_addr learned my addr 172.21.15.132:0/3514222784 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:17.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 190907581 0 0) 0x7fc980122560 con 0x7fc9800a1600 2024-01-24T23:25:17.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:17.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:17.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:17.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient(hunting): my global_id is 85268 2024-01-24T23:25:17.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc964003530 con 0x7fc9800a1600 2024-01-24T23:25:17.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3554544823 0 0) 0x7fc98009ea70 con 0x7fc980136e50 2024-01-24T23:25:17.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:17.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:17.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:17.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient(hunting): my global_id is 94216 2024-01-24T23:25:17.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc980122560 con 0x7fc980136e50 2024-01-24T23:25:17.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 113892830 0 0) 0x7fc980120270 con 0x7fc9800a0290 2024-01-24T23:25:17.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:17.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:17.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:17.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient(hunting): my global_id is 94230 2024-01-24T23:25:17.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc98009ea70 con 0x7fc9800a0290 2024-01-24T23:25:17.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 540455014 0 0) 0x7fc964003530 con 0x7fc9800a1600 2024-01-24T23:25:17.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc980120270 con 0x7fc9800a1600 2024-01-24T23:25:17.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc970001ae0 con 0x7fc9800a1600 2024-01-24T23:25:17.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:17.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:25:17.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient(hunting): dump: 2024-01-24T23:25:17.662 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:17.662 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:17.662 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:17.662 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:17.662 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:17.662 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:17.662 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:17.662 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:17.662 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:17.663 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:17.663 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:17.663 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:17.663 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:17.663 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:17.663 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:17.663 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:17.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:17.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3990325677 0 0) 0x7fc980122560 con 0x7fc980136e50 2024-01-24T23:25:17.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc964003530 con 0x7fc980136e50 2024-01-24T23:25:17.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc978001740 con 0x7fc980136e50 2024-01-24T23:25:17.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:17.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:17.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient(hunting): dump: 2024-01-24T23:25:17.664 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:17.664 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:17.664 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:17.664 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:17.664 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:17.664 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:17.664 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:17.664 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:17.664 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:17.664 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:17.664 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:17.664 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:17.664 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:17.664 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:17.665 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:17.665 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:17.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 992202426 0 0) 0x7fc98009ea70 con 0x7fc9800a0290 2024-01-24T23:25:17.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc980122560 con 0x7fc9800a0290 2024-01-24T23:25:17.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc9680010b0 con 0x7fc9800a0290 2024-01-24T23:25:17.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:17.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:17.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient(hunting): dump: 2024-01-24T23:25:17.665 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:17.665 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:17.665 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:17.665 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:17.665 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:17.665 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:17.665 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:17.666 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:17.666 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:17.666 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:17.666 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:17.666 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:17.666 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:17.666 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:17.666 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:17.666 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:17.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3990342433 0 0) 0x7fc980120270 con 0x7fc9800a1600 2024-01-24T23:25:17.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 >> v1:172.21.15.115:6791/0 conn(0x7fc980136e50 legacy=0x7fc9801d3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:17.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 >> v1:172.21.15.115:6789/0 conn(0x7fc9800a0290 legacy=0x7fc980134dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:17.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient: _finish_hunting 0 2024-01-24T23:25:17.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 1 monclient: found mon.b 2024-01-24T23:25:17.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:17.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:25:17.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc980136c20 con 0x7fc9800a1600 2024-01-24T23:25:17.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient: _finish_auth 0 2024-01-24T23:25:17.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient: _check_auth_tickets 2024-01-24T23:25:17.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:17.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc984b41640 5 monclient: authenticate success, global_id 85268 2024-01-24T23:25:17.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc984b41640 10 monclient: _renew_subs 2024-01-24T23:25:17.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc984b41640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:25:17.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc984b41640 1 -- 172.21.15.132:0/3514222784 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc980122280 con 0x7fc9800a1600 2024-01-24T23:25:17.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc984b41640 10 monclient: _renew_subs 2024-01-24T23:25:17.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc984b41640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:25:17.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc984b41640 1 -- 172.21.15.132:0/3514222784 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9801354e0 con 0x7fc9800a1600 2024-01-24T23:25:17.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc9700038d0 con 0x7fc9800a1600 2024-01-24T23:25:17.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:17.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc970003d80 con 0x7fc9800a1600 2024-01-24T23:25:17.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:17.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc97d59a640 4 set_mon_vals no callback set 2024-01-24T23:25:17.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:25:17.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 10 monclient: dump: 2024-01-24T23:25:17.668 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:17.668 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:17.668 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:17.668 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:17.668 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:17.668 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:17.668 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:17.668 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:17.668 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:17.668 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:17.669 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:17.669 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:17.669 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:17.669 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:17.669 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:17.669 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:17.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc984b41640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:17.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc984b41640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:17.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc984b41640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:25:17.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc984b41640 1 -- 172.21.15.132:0/3514222784 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc958005090 con 0x7fc9800a1600 2024-01-24T23:25:17.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fc970003f60 con 0x7fc9800a1600 2024-01-24T23:25:17.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(128..128 src has 1..128) v4 ==== 6989+0+0 (unknown 538550023 0 0) 0x7fc970044470 con 0x7fc9800a1600 2024-01-24T23:25:17.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.645+0000 7fc97e59c640 10 monclient: get_auth_request con 0x7fc964037d50 auth_method 0 2024-01-24T23:25:17.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.657+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 <== mon.2 v1:172.21.15.115:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fc9700147c0 con 0x7fc9800a1600 2024-01-24T23:25:17.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.657+0000 7fc9777fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:17.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.657+0000 7fc9777fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:17.822 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.817+0000 7fc984b41640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:17.822 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.817+0000 7fc984b41640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:17.822 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.817+0000 7fc984b41640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:25:17.822 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.817+0000 7fc984b41640 1 -- 172.21.15.132:0/3514222784 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7fc958005340 con 0x7fc9800a1600 2024-01-24T23:25:17.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.817+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 <== mon.2 v1:172.21.15.115:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v128) v1 ==== 71+0+7 (unknown 43352416 0 1352219128) 0x7fc9700145e0 con 0x7fc9800a1600 2024-01-24T23:25:17.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.817+0000 7fc9777fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:17.823 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.817+0000 7fc9777fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:17.922 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.917+0000 7fc984b41640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:17.922 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.917+0000 7fc984b41640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:17.922 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.917+0000 7fc984b41640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:25:17.922 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.917+0000 7fc984b41640 1 -- 172.21.15.132:0/3514222784 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7fc958001130 con 0x7fc9800a1600 2024-01-24T23:25:17.923 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.917+0000 7fc9777fe640 1 -- 172.21.15.132:0/3514222784 <== mon.2 v1:172.21.15.115:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v128) v1 ==== 71+0+7 (unknown 43352416 0 1352219128) 0x7fc9700145e0 con 0x7fc9800a1600 2024-01-24T23:25:17.923 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.917+0000 7fc9777fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:17.923 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.917+0000 7fc9777fe640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:25:17.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.921+0000 7fc984b41640 1 -- 172.21.15.132:0/3514222784 >> v1:172.21.15.132:6812/1835401922 conn(0x7fc964037d50 legacy=0x7fc96403a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:17.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.921+0000 7fc984b41640 10 monclient: shutdown 2024-01-24T23:25:17.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.921+0000 7fc984b41640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:17.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.921+0000 7fc984b41640 1 -- 172.21.15.132:0/3514222784 >> v1:172.21.15.115:6790/0 conn(0x7fc9800a1600 legacy=0x7fc9801d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:17.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.921+0000 7fc984b41640 1 -- 172.21.15.132:0/3514222784 shutdown_connections 2024-01-24T23:25:17.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.921+0000 7fc984b41640 1 -- 172.21.15.132:0/3514222784 >> 172.21.15.132:0/3514222784 conn(0x7fc9800a5a10 msgr2=0x7fc9801bc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:17.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.921+0000 7fc984b41640 1 -- 172.21.15.132:0/3514222784 shutdown_connections 2024-01-24T23:25:17.925 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:17.921+0000 7fc984b41640 1 -- 172.21.15.132:0/3514222784 wait complete. 2024-01-24T23:25:17.939 INFO:tasks.workunit.client.0.smithi132.stdout:barset 2024-01-24T23:25:17.939 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set rm barset 2024-01-24T23:25:18.043 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-01-24T23:25:18.043 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-01-24T23:25:18.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.053+0000 7fdf59d07640 10 monclient: get_monmap_and_config 2024-01-24T23:25:18.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.053+0000 7fdf59d07640 10 monclient: build_initial_monmap 2024-01-24T23:25:18.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.053+0000 7fdf59d07640 10 monclient: monmap: 2024-01-24T23:25:18.058 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:18.058 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.058 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:18.059610+0000 2024-01-24T23:25:18.058 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:18.059610+0000 2024-01-24T23:25:18.058 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:18.059 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:18.059 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:18.059 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:18.059 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:18.059 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:18.059 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:18.059 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:18.059 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:18.059 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:18.059 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:18.059 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:18.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 1 Processor -- start 2024-01-24T23:25:18.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 1 -- start start 2024-01-24T23:25:18.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 10 monclient: init 2024-01-24T23:25:18.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:18.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 10 monclient: _add_conns ranks=[4,6,2,8,7,0,1,5,3] 2024-01-24T23:25:18.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 10 monclient(hunting): picked mon.noname-c con 0x7fdf540a2d90 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:18.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 10 monclient(hunting): picked mon.noname-d con 0x7fdf54070d10 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:18.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 10 monclient(hunting): picked mon.noname-b con 0x7fdf54139fb0 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:18.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf54064700 con 0x7fdf54139fb0 2024-01-24T23:25:18.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf540a3170 con 0x7fdf540a2d90 2024-01-24T23:25:18.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf5406fd50 con 0x7fdf54070d10 2024-01-24T23:25:18.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:18.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 10 monclient(hunting): authenticate will time out at 1205.385099s 2024-01-24T23:25:18.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf52ffd640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fdf54070d10 0x7fdf540a18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:50388/0 (socket says 172.21.15.132:50388) 2024-01-24T23:25:18.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf52ffd640 1 -- 172.21.15.132:0/361454814 learned_addr learned my addr 172.21.15.132:0/361454814 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:18.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 1 -- 172.21.15.132:0/361454814 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836265480 0 0) 0x7fdf540a3170 con 0x7fdf540a2d90 2024-01-24T23:25:18.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 10 monclient(hunting): my global_id is 94225 2024-01-24T23:25:18.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 1 -- 172.21.15.132:0/361454814 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf3c003530 con 0x7fdf540a2d90 2024-01-24T23:25:18.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 1 -- 172.21.15.132:0/361454814 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3260566401 0 0) 0x7fdf54064700 con 0x7fdf54139fb0 2024-01-24T23:25:18.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 10 monclient(hunting): my global_id is 85277 2024-01-24T23:25:18.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 1 -- 172.21.15.132:0/361454814 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf540a3170 con 0x7fdf54139fb0 2024-01-24T23:25:18.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 1 -- 172.21.15.132:0/361454814 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1041630853 0 0) 0x7fdf5406fd50 con 0x7fdf54070d10 2024-01-24T23:25:18.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 10 monclient(hunting): my global_id is 6036 2024-01-24T23:25:18.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 1 -- 172.21.15.132:0/361454814 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf54064700 con 0x7fdf54070d10 2024-01-24T23:25:18.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 1 -- 172.21.15.132:0/361454814 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 612485830 0 0) 0x7fdf3c003530 con 0x7fdf540a2d90 2024-01-24T23:25:18.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 1 -- 172.21.15.132:0/361454814 >> v1:172.21.15.115:6792/0 conn(0x7fdf54070d10 legacy=0x7fdf540a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 1 -- 172.21.15.132:0/361454814 >> v1:172.21.15.115:6790/0 conn(0x7fdf54139fb0 legacy=0x7fdf5413c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 10 monclient: _finish_hunting 0 2024-01-24T23:25:18.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 1 monclient: found mon.noname-c 2024-01-24T23:25:18.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:18.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:25:18.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 1 -- 172.21.15.132:0/361454814 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf540a1fd0 con 0x7fdf540a2d90 2024-01-24T23:25:18.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 10 monclient: _finish_auth 0 2024-01-24T23:25:18.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 10 monclient: _check_auth_tickets 2024-01-24T23:25:18.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:18.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 1 -- 172.21.15.132:0/361454814 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdf440017c0 con 0x7fdf540a2d90 2024-01-24T23:25:18.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 5 monclient: authenticate success, global_id 94225 2024-01-24T23:25:18.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:18.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:25:18.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 10 monclient: dump: 2024-01-24T23:25:18.066 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:18.066 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.066 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.066 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.066 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.066 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:18.066 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.066 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.066 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.066 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.066 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.066 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.066 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.066 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.067 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.067 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:18.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:18.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 1 -- 172.21.15.132:0/361454814 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdf44001aa0 con 0x7fdf540a2d90 2024-01-24T23:25:18.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:18.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:18.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 4 set_mon_vals no callback set 2024-01-24T23:25:18.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 1 -- 172.21.15.132:0/361454814 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdf44003960 con 0x7fdf540a2d90 2024-01-24T23:25:18.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:18.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf527fc640 10 monclient: dump: 2024-01-24T23:25:18.067 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:18.067 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.067 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.068 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.068 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.068 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:18.068 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.068 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.068 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.068 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.068 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.068 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.068 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.068 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.068 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.068 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:18.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 10 monclient: shutdown 2024-01-24T23:25:18.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:18.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 1 -- 172.21.15.132:0/361454814 >> v1:172.21.15.115:6791/0 conn(0x7fdf540a2d90 legacy=0x7fdf540a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 1 -- 172.21.15.132:0/361454814 shutdown_connections 2024-01-24T23:25:18.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 1 -- 172.21.15.132:0/361454814 >> 172.21.15.132:0/361454814 conn(0x7fdf540a22a0 msgr2=0x7fdf540a2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:18.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 1 -- 172.21.15.132:0/361454814 shutdown_connections 2024-01-24T23:25:18.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.057+0000 7fdf59d07640 1 -- 172.21.15.132:0/361454814 wait complete. 2024-01-24T23:25:18.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.061+0000 7fdf59d07640 10 monclient: build_initial_monmap 2024-01-24T23:25:18.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.061+0000 7fdf59d07640 10 monclient: monmap: 2024-01-24T23:25:18.071 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:18.071 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:18.071 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:18.071 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:18.071 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:18.071 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:18.071 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:18.071 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:18.071 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:18.071 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:18.071 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:18.072 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:18.072 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:18.072 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:18.072 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:18.072 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:18.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.061+0000 7fdf59d07640 1 Processor -- start 2024-01-24T23:25:18.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.061+0000 7fdf59d07640 1 -- start start 2024-01-24T23:25:18.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.061+0000 7fdf59d07640 10 monclient: init 2024-01-24T23:25:18.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.061+0000 7fdf59d07640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:18.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.061+0000 7fdf59d07640 10 monclient: _add_conns ranks=[5,4,1,0,3,2,8,6,7] 2024-01-24T23:25:18.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.061+0000 7fdf59d07640 10 monclient(hunting): picked mon.noname-h con 0x7fdf54070d10 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:18.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.061+0000 7fdf59d07640 10 monclient(hunting): picked mon.noname-c con 0x7fdf540a2d90 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:18.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.061+0000 7fdf59d07640 10 monclient(hunting): picked mon.noname-f con 0x7fdf54139fb0 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:18.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.061+0000 7fdf59d07640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf5411bf30 con 0x7fdf54139fb0 2024-01-24T23:25:18.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.061+0000 7fdf59d07640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf541311e0 con 0x7fdf540a2d90 2024-01-24T23:25:18.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.061+0000 7fdf59d07640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf54071110 con 0x7fdf54070d10 2024-01-24T23:25:18.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.061+0000 7fdf59d07640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:18.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.061+0000 7fdf59d07640 10 monclient(hunting): authenticate will time out at 1205.389591s 2024-01-24T23:25:18.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.061+0000 7fdf537fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fdf54070d10 0x7fdf54130400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:44112/0 (socket says 172.21.15.132:44112) 2024-01-24T23:25:18.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.061+0000 7fdf53fff640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fdf54139fb0 0x7fdf541d1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:51462/0 (socket says 172.21.15.132:51462) 2024-01-24T23:25:18.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.061+0000 7fdf537fe640 1 -- 172.21.15.132:0/935895895 learned_addr learned my addr 172.21.15.132:0/935895895 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:18.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.081+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3832289998 0 0) 0x7fdf5411bf30 con 0x7fdf54139fb0 2024-01-24T23:25:18.085 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.081+0000 7fdf3bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.085 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.081+0000 7fdf3bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.085 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.081+0000 7fdf3bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.085 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.081+0000 7fdf3bfff640 10 monclient(hunting): my global_id is 94168 2024-01-24T23:25:18.085 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.081+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf40003560 con 0x7fdf54139fb0 2024-01-24T23:25:18.087 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.081+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1650848337 0 0) 0x7fdf54071110 con 0x7fdf54070d10 2024-01-24T23:25:18.087 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.081+0000 7fdf3bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.087 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.081+0000 7fdf3bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.087 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.081+0000 7fdf3bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.087 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.081+0000 7fdf3bfff640 10 monclient(hunting): my global_id is 5855 2024-01-24T23:25:18.087 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.081+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf5411bf30 con 0x7fdf54070d10 2024-01-24T23:25:18.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.089+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1047544013 0 0) 0x7fdf541311e0 con 0x7fdf540a2d90 2024-01-24T23:25:18.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.089+0000 7fdf3bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.089+0000 7fdf3bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.089+0000 7fdf3bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.095 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.089+0000 7fdf3bfff640 10 monclient(hunting): my global_id is 94234 2024-01-24T23:25:18.095 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.089+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf54071110 con 0x7fdf540a2d90 2024-01-24T23:25:18.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3035942809 0 0) 0x7fdf40003560 con 0x7fdf54139fb0 2024-01-24T23:25:18.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdf541311e0 con 0x7fdf54139fb0 2024-01-24T23:25:18.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3728366304 0 0) 0x7fdf5411bf30 con 0x7fdf54070d10 2024-01-24T23:25:18.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdf40003560 con 0x7fdf54070d10 2024-01-24T23:25:18.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdf4c001d10 con 0x7fdf54139fb0 2024-01-24T23:25:18.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:25:18.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 10 monclient(hunting): dump: 2024-01-24T23:25:18.107 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:18.107 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.107 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.107 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.107 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.107 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:18.107 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.107 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.107 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.108 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.108 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.108 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.108 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.108 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.108 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.108 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:18.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:18.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdf440039e0 con 0x7fdf54070d10 2024-01-24T23:25:18.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:18.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 10 monclient(hunting): dump: 2024-01-24T23:25:18.108 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:18.108 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.109 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.109 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.109 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.109 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:18.109 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.109 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.109 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.109 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.109 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.109 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.109 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.109 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.109 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.109 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:18.109 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1224971830 0 0) 0x7fdf541311e0 con 0x7fdf54139fb0 2024-01-24T23:25:18.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 >> v1:172.21.15.132:6791/0 conn(0x7fdf54070d10 legacy=0x7fdf54130400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 >> v1:172.21.15.115:6791/0 conn(0x7fdf540a2d90 legacy=0x7fdf541ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:25:18.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 1 monclient: found mon.f 2024-01-24T23:25:18.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:18.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:25:18.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5411d180 con 0x7fdf54139fb0 2024-01-24T23:25:18.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 10 monclient: _finish_auth 0 2024-01-24T23:25:18.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:25:18.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:18.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf59d07640 5 monclient: authenticate success, global_id 94168 2024-01-24T23:25:18.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf59d07640 10 monclient: _renew_subs 2024-01-24T23:25:18.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf59d07640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:25:18.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf59d07640 1 -- 172.21.15.132:0/935895895 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5409ec80 con 0x7fdf54139fb0 2024-01-24T23:25:18.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf59d07640 10 monclient: _renew_subs 2024-01-24T23:25:18.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf59d07640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:25:18.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf59d07640 1 -- 172.21.15.132:0/935895895 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5406f0b0 con 0x7fdf54139fb0 2024-01-24T23:25:18.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdf4c002080 con 0x7fdf54139fb0 2024-01-24T23:25:18.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:18.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdf4c003f90 con 0x7fdf54139fb0 2024-01-24T23:25:18.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:25:18.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf51ffb640 4 set_mon_vals no callback set 2024-01-24T23:25:18.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 10 monclient: dump: 2024-01-24T23:25:18.111 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:18.111 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.111 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.111 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.111 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.111 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:18.112 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.112 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.112 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.112 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.112 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.112 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.112 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.112 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.112 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.112 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:18.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fdf4c0040f0 con 0x7fdf54139fb0 2024-01-24T23:25:18.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf59d07640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:18.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf59d07640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:18.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf59d07640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:25:18.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(128..128 src has 1..128) v4 ==== 6989+0+0 (unknown 538550023 0 0) 0x7fdf4c044640 con 0x7fdf54139fb0 2024-01-24T23:25:18.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf59d07640 1 -- 172.21.15.132:0/935895895 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf54073400 con 0x7fdf54139fb0 2024-01-24T23:25:18.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.101+0000 7fdf537fe640 10 monclient: get_auth_request con 0x7fdf40037850 auth_method 0 2024-01-24T23:25:18.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.105+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fdf4c014990 con 0x7fdf54139fb0 2024-01-24T23:25:18.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.105+0000 7fdf3bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:18.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.105+0000 7fdf3bfff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:18.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.174+0000 7f9bb0aec640 10 monclient: get_monmap_and_config 2024-01-24T23:25:18.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.174+0000 7f9bb0aec640 10 monclient: build_initial_monmap 2024-01-24T23:25:18.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.174+0000 7f9bb0aec640 10 monclient: monmap: 2024-01-24T23:25:18.176 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:18.176 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.176 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:18.176831+0000 2024-01-24T23:25:18.176 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:18.176831+0000 2024-01-24T23:25:18.177 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:18.177 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.177 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:18.177 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:18.177 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:18.177 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:18.177 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:18.177 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:18.177 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:18.177 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:18.177 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:18.177 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.174+0000 7f9bb0aec640 1 Processor -- start 2024-01-24T23:25:18.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.174+0000 7f9bb0aec640 1 -- start start 2024-01-24T23:25:18.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.174+0000 7f9bb0aec640 10 monclient: init 2024-01-24T23:25:18.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.174+0000 7f9bb0aec640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:18.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.174+0000 7f9bb0aec640 10 monclient: _add_conns ranks=[1,4,6,0,7,2,8,3,5] 2024-01-24T23:25:18.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.174+0000 7f9bb0aec640 10 monclient(hunting): picked mon.noname-f con 0x7f9bac09b9f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:18.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.174+0000 7f9bb0aec640 10 monclient(hunting): picked mon.noname-c con 0x7f9bac0720b0 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:18.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.174+0000 7f9bb0aec640 10 monclient(hunting): picked mon.noname-d con 0x7f9bac139700 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:18.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.174+0000 7f9bb0aec640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bac04ed70 con 0x7f9bac09b9f0 2024-01-24T23:25:18.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.174+0000 7f9bb0aec640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bac0a0eb0 con 0x7f9bac0720b0 2024-01-24T23:25:18.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.174+0000 7f9bb0aec640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bac09bdd0 con 0x7f9bac139700 2024-01-24T23:25:18.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.174+0000 7f9bb0aec640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:18.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.174+0000 7f9bb0aec640 10 monclient(hunting): authenticate will time out at 1197.099669s 2024-01-24T23:25:18.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.174+0000 7f9ba1d9b640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f9bac0720b0 0x7f9bac0a0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:44674/0 (socket says 172.21.15.115:44674) 2024-01-24T23:25:18.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.174+0000 7f9ba1d9b640 1 -- 172.21.15.115:0/3681121255 learned_addr learned my addr 172.21.15.115:0/3681121255 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:18.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 1 -- 172.21.15.115:0/3681121255 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259619429 0 0) 0x7f9bac0a0eb0 con 0x7f9bac0720b0 2024-01-24T23:25:18.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 10 monclient(hunting): my global_id is 94243 2024-01-24T23:25:18.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 1 -- 172.21.15.115:0/3681121255 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b98003530 con 0x7f9bac0720b0 2024-01-24T23:25:18.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 1 -- 172.21.15.115:0/3681121255 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024290369 0 0) 0x7f9bac09bdd0 con 0x7f9bac139700 2024-01-24T23:25:18.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.185 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 10 monclient(hunting): my global_id is 6045 2024-01-24T23:25:18.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 1 -- 172.21.15.115:0/3681121255 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bac0a0eb0 con 0x7f9bac139700 2024-01-24T23:25:18.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 1 -- 172.21.15.115:0/3681121255 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3506467996 0 0) 0x7f9b98003530 con 0x7f9bac0720b0 2024-01-24T23:25:18.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 1 -- 172.21.15.115:0/3681121255 >> v1:172.21.15.115:6792/0 conn(0x7f9bac139700 legacy=0x7f9bac13baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 1 -- 172.21.15.115:0/3681121255 >> v1:172.21.15.132:6789/0 conn(0x7f9bac09b9f0 legacy=0x7f9bac09f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 10 monclient: _finish_hunting 0 2024-01-24T23:25:18.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 1 monclient: found mon.noname-c 2024-01-24T23:25:18.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:18.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:25:18.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 1 -- 172.21.15.115:0/3681121255 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bac0a0710 con 0x7f9bac0720b0 2024-01-24T23:25:18.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 10 monclient: _finish_auth 0 2024-01-24T23:25:18.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 10 monclient: _check_auth_tickets 2024-01-24T23:25:18.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:18.186 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 1 -- 172.21.15.115:0/3681121255 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9b940017c0 con 0x7f9bac0720b0 2024-01-24T23:25:18.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 5 monclient: authenticate success, global_id 94243 2024-01-24T23:25:18.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:18.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:25:18.187 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 10 monclient: dump: 2024-01-24T23:25:18.187 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.187 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.187 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.187 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.187 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.187 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.187 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.187 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.187 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.187 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.188 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.188 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.188 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.188 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.188 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.188 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:18.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 1 -- 172.21.15.115:0/3681121255 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9b94001a20 con 0x7f9bac0720b0 2024-01-24T23:25:18.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:18.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:18.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 4 set_mon_vals no callback set 2024-01-24T23:25:18.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 1 -- 172.21.15.115:0/3681121255 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9b94004bf0 con 0x7f9bac0720b0 2024-01-24T23:25:18.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.188 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:18.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba9d9b640 10 monclient: dump: 2024-01-24T23:25:18.189 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.189 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.189 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.189 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.189 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.189 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.189 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.189 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.189 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.189 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.189 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.189 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.189 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.189 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.190 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.190 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 10 monclient: shutdown 2024-01-24T23:25:18.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:18.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 1 -- 172.21.15.115:0/3681121255 >> v1:172.21.15.115:6791/0 conn(0x7f9bac0720b0 legacy=0x7f9bac0a0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 1 -- 172.21.15.115:0/3681121255 shutdown_connections 2024-01-24T23:25:18.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 1 -- 172.21.15.115:0/3681121255 >> 172.21.15.115:0/3681121255 conn(0x7f9bac0a5730 msgr2=0x7f9bac0a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:18.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 1 -- 172.21.15.115:0/3681121255 shutdown_connections 2024-01-24T23:25:18.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 1 -- 172.21.15.115:0/3681121255 wait complete. 2024-01-24T23:25:18.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 10 monclient: build_initial_monmap 2024-01-24T23:25:18.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 10 monclient: monmap: 2024-01-24T23:25:18.190 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:18.190 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:18.190 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:18.190 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:18.191 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:18.191 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:18.191 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:18.191 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:18.191 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:18.191 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:18.191 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:18.191 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:18.191 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:18.191 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:18.191 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:18.191 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 1 Processor -- start 2024-01-24T23:25:18.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 1 -- start start 2024-01-24T23:25:18.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 10 monclient: init 2024-01-24T23:25:18.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:18.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 10 monclient: _add_conns ranks=[6,2,8,5,3,4,7,1,0] 2024-01-24T23:25:18.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 10 monclient(hunting): picked mon.noname-d con 0x7f9bac0720b0 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:18.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 10 monclient(hunting): picked mon.noname-b con 0x7f9bac09b9f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:18.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 10 monclient(hunting): picked mon.noname-e con 0x7f9bac139700 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:18.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bac0710f0 con 0x7f9bac09b9f0 2024-01-24T23:25:18.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bac09c9f0 con 0x7f9bac0720b0 2024-01-24T23:25:18.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bac09c480 con 0x7f9bac139700 2024-01-24T23:25:18.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:18.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 10 monclient(hunting): authenticate will time out at 1197.102669s 2024-01-24T23:25:18.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9baa59c640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f9bac0720b0 0x7f9bac12e930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:40486/0 (socket says 172.21.15.115:40486) 2024-01-24T23:25:18.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9baa59c640 1 -- 172.21.15.115:0/4033870855 learned_addr learned my addr 172.21.15.115:0/4033870855 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:18.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2171248765 0 0) 0x7f9bac09c480 con 0x7f9bac139700 2024-01-24T23:25:18.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient(hunting): my global_id is 85121 2024-01-24T23:25:18.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b8c003530 con 0x7f9bac139700 2024-01-24T23:25:18.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1399298015 0 0) 0x7f9bac09c9f0 con 0x7f9bac0720b0 2024-01-24T23:25:18.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient(hunting): my global_id is 6054 2024-01-24T23:25:18.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bac09c480 con 0x7f9bac0720b0 2024-01-24T23:25:18.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4066247308 0 0) 0x7f9bac0710f0 con 0x7f9bac09b9f0 2024-01-24T23:25:18.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient(hunting): my global_id is 85286 2024-01-24T23:25:18.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bac09c9f0 con 0x7f9bac09b9f0 2024-01-24T23:25:18.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3224709820 0 0) 0x7f9bac09c480 con 0x7f9bac0720b0 2024-01-24T23:25:18.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9bac0710f0 con 0x7f9bac0720b0 2024-01-24T23:25:18.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9b9c001aa0 con 0x7f9bac0720b0 2024-01-24T23:25:18.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:25:18.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient(hunting): dump: 2024-01-24T23:25:18.194 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.194 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.195 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.195 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.195 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.195 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.195 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.195 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.195 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.195 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.195 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.195 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.195 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.195 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.195 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.195 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:18.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1600113354 0 0) 0x7f9bac09c9f0 con 0x7f9bac09b9f0 2024-01-24T23:25:18.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9bac09c480 con 0x7f9bac09b9f0 2024-01-24T23:25:18.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9b94001780 con 0x7f9bac09b9f0 2024-01-24T23:25:18.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:25:18.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient(hunting): dump: 2024-01-24T23:25:18.196 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.196 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.196 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.196 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.196 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.196 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.196 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.196 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.197 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.197 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.197 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.197 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.197 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.197 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.197 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.197 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3003415925 0 0) 0x7f9b8c003530 con 0x7f9bac139700 2024-01-24T23:25:18.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9bac09c9f0 con 0x7f9bac139700 2024-01-24T23:25:18.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9ba4001d00 con 0x7f9bac139700 2024-01-24T23:25:18.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:18.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient(hunting): dump: 2024-01-24T23:25:18.197 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.198 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.198 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.198 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.198 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.198 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.198 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.198 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.198 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.198 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.198 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.198 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.198 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.198 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.198 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.199 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 512825148 0 0) 0x7f9bac0710f0 con 0x7f9bac0720b0 2024-01-24T23:25:18.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 >> v1:172.21.15.115:6793/0 conn(0x7f9bac139700 legacy=0x7f9bac1d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 >> v1:172.21.15.115:6790/0 conn(0x7f9bac09b9f0 legacy=0x7f9bac1ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient: _finish_hunting 0 2024-01-24T23:25:18.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 1 monclient: found mon.d 2024-01-24T23:25:18.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:18.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:25:18.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bac070760 con 0x7f9bac0720b0 2024-01-24T23:25:18.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient: _finish_auth 0 2024-01-24T23:25:18.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient: _check_auth_tickets 2024-01-24T23:25:18.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:18.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 5 monclient: authenticate success, global_id 6054 2024-01-24T23:25:18.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 10 monclient: _renew_subs 2024-01-24T23:25:18.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:25:18.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 1 -- 172.21.15.115:0/4033870855 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bac134b40 con 0x7f9bac0720b0 2024-01-24T23:25:18.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 10 monclient: _renew_subs 2024-01-24T23:25:18.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:25:18.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9bb0aec640 1 -- 172.21.15.115:0/4033870855 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bac0712c0 con 0x7f9bac0720b0 2024-01-24T23:25:18.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9b9c0038d0 con 0x7f9bac0720b0 2024-01-24T23:25:18.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:18.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9b9c003e00 con 0x7f9bac0720b0 2024-01-24T23:25:18.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:18.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba959a640 4 set_mon_vals no callback set 2024-01-24T23:25:18.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.178+0000 7f9ba37fe640 10 monclient: dump: 2024-01-24T23:25:18.200 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.200 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.201 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.201 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.201 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.201 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.201 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.201 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.201 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.201 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.201 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.201 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.201 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.201 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.201 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.201 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.182+0000 7f9bb0aec640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:18.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.182+0000 7f9bb0aec640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:18.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.182+0000 7f9bb0aec640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:25:18.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.182+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f9b9c003f60 con 0x7f9bac0720b0 2024-01-24T23:25:18.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.182+0000 7f9bb0aec640 1 -- 172.21.15.115:0/4033870855 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b90005090 con 0x7f9bac0720b0 2024-01-24T23:25:18.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.182+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(128..128 src has 1..128) v4 ==== 6989+0+0 (unknown 538550023 0 0) 0x7f9b9c0444a0 con 0x7f9bac0720b0 2024-01-24T23:25:18.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.182+0000 7f9ba1d9b640 10 monclient: get_auth_request con 0x7f9b8c037ca0 auth_method 0 2024-01-24T23:25:18.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.186+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f9b9c014920 con 0x7f9bac0720b0 2024-01-24T23:25:18.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.186+0000 7f9ba37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:18.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.186+0000 7f9ba37fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:18.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.273+0000 7fdf59d07640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-01-24T23:25:18.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.273+0000 7fdf59d07640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-01-24T23:25:18.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.273+0000 7fdf59d07640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:25:18.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:18.273+0000 7fdf59d07640 1 -- 172.21.15.132:0/935895895 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd crush weight-set rm", "pool": "barset"} v 0) v1 -- 0x7fdf5409f080 con 0x7fdf54139fb0 2024-01-24T23:25:18.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.394+0000 7f9bb0aec640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-01-24T23:25:18.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.394+0000 7f9bb0aec640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-01-24T23:25:18.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.394+0000 7f9bb0aec640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:25:18.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.394+0000 7f9bb0aec640 1 -- 172.21.15.115:0/4033870855 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f9b900058c0 con 0x7f9bac0720b0 2024-01-24T23:25:18.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.394+0000 7f9ba37fe640 1 -- 172.21.15.115:0/4033870855 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2744 (unknown 254746657 0 3434482161) 0x7f9b9c012fc0 con 0x7f9bac0720b0 2024-01-24T23:25:18.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.394+0000 7f9ba37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-01-24T23:25:18.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.394+0000 7f9ba37fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:18.400 INFO:teuthology.orchestra.run.smithi115.stdout:{"election_epoch":24,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"g","quorum_age":3,"features":{"quorum_con":"4540701272860131327","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","modified":"2024-01-24T23:20:20.557246Z","created":"2024-01-24T23:20:20.557246Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6790","nonce":0}]},"addr":"172.21.15.115:6790/0","public_addr":"172.21.15.115:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6790","nonce":0}]},"addr":"172.21.15.132:6790/0","public_addr":"172.21.15.132:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6791","nonce":0}]},"addr":"172.21.15.115:6791/0","public_addr":"172.21.15.115:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6791","nonce":0}]},"addr":"172.21.15.132:6791/0","public_addr":"172.21.15.132:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6792","nonce":0}]},"addr":"172.21.15.115:6792/0","public_addr":"172.21.15.115:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6792","nonce":0}]},"addr":"172.21.15.132:6792/0","public_addr":"172.21.15.132:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6793","nonce":0}]},"addr":"172.21.15.115:6793/0","public_addr":"172.21.15.115:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-01-24T23:25:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.398+0000 7f9bb0aec640 1 -- 172.21.15.115:0/4033870855 >> v1:172.21.15.132:6812/1835401922 conn(0x7f9b8c037ca0 legacy=0x7f9b8c03a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.398+0000 7f9bb0aec640 10 monclient: shutdown 2024-01-24T23:25:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.398+0000 7f9bb0aec640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.398+0000 7f9bb0aec640 1 -- 172.21.15.115:0/4033870855 >> v1:172.21.15.115:6792/0 conn(0x7f9bac0720b0 legacy=0x7f9bac12e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.398+0000 7f9bb0aec640 1 -- 172.21.15.115:0/4033870855 shutdown_connections 2024-01-24T23:25:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.398+0000 7f9bb0aec640 1 -- 172.21.15.115:0/4033870855 >> 172.21.15.115:0/4033870855 conn(0x7f9bac0a5730 msgr2=0x7f9bac09f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.398+0000 7f9bb0aec640 1 -- 172.21.15.115:0/4033870855 shutdown_connections 2024-01-24T23:25:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.398+0000 7f9bb0aec640 1 -- 172.21.15.115:0/4033870855 wait complete. 2024-01-24T23:25:18.415 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-01-24T23:25:18.415 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-01-24T23:25:18.415 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-01-24T23:25:18.415 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-01-24T23:25:18.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.530+0000 7ff3ed5de640 10 monclient: get_monmap_and_config 2024-01-24T23:25:18.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.530+0000 7ff3ed5de640 10 monclient: build_initial_monmap 2024-01-24T23:25:18.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.530+0000 7ff3ed5de640 10 monclient: monmap: 2024-01-24T23:25:18.533 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:18.533 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.533 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:18.533387+0000 2024-01-24T23:25:18.533 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:18.533387+0000 2024-01-24T23:25:18.533 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:18.533 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.533 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:18.533 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:18.533 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:18.533 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:18.533 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:18.533 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:18.533 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:18.533 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:18.534 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:18.534 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.530+0000 7ff3ed5de640 1 Processor -- start 2024-01-24T23:25:18.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.530+0000 7ff3ed5de640 1 -- start start 2024-01-24T23:25:18.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.530+0000 7ff3ed5de640 10 monclient: init 2024-01-24T23:25:18.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.530+0000 7ff3ed5de640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:18.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.530+0000 7ff3ed5de640 10 monclient: _add_conns ranks=[2,8,6,4,1,3,7,0,5] 2024-01-24T23:25:18.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.530+0000 7ff3ed5de640 10 monclient(hunting): picked mon.noname-b con 0x7ff3e8136b50 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:18.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.530+0000 7ff3ed5de640 10 monclient(hunting): picked mon.noname-e con 0x7ff3e8136f30 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:18.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.530+0000 7ff3ed5de640 10 monclient(hunting): picked mon.noname-d con 0x7ff3e813d090 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:18.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.530+0000 7ff3ed5de640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e804ed70 con 0x7ff3e8136b50 2024-01-24T23:25:18.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.530+0000 7ff3ed5de640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e806bf60 con 0x7ff3e813d090 2024-01-24T23:25:18.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.530+0000 7ff3ed5de640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e809b1e0 con 0x7ff3e8136f30 2024-01-24T23:25:18.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.530+0000 7ff3ed5de640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:18.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 10 monclient(hunting): authenticate will time out at 1197.456009s 2024-01-24T23:25:18.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e77fe640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7ff3e813d090 0x7ff3e813f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:40498/0 (socket says 172.21.15.115:40498) 2024-01-24T23:25:18.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e77fe640 1 -- 172.21.15.115:0/2628393717 learned_addr learned my addr 172.21.15.115:0/2628393717 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:18.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e6ffd640 1 --1- 172.21.15.115:0/2628393717 >> v1:172.21.15.115:6790/0 conn(0x7ff3e8136b50 0x7ff3e809bd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:60400/0 (socket says 172.21.15.115:60400) 2024-01-24T23:25:18.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 1 -- 172.21.15.115:0/2628393717 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 159335408 0 0) 0x7ff3e806bf60 con 0x7ff3e813d090 2024-01-24T23:25:18.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 10 monclient(hunting): my global_id is 6063 2024-01-24T23:25:18.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 1 -- 172.21.15.115:0/2628393717 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3d0003530 con 0x7ff3e813d090 2024-01-24T23:25:18.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 1 -- 172.21.15.115:0/2628393717 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3158788694 0 0) 0x7ff3e804ed70 con 0x7ff3e8136b50 2024-01-24T23:25:18.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 10 monclient(hunting): my global_id is 85295 2024-01-24T23:25:18.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 1 -- 172.21.15.115:0/2628393717 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3e806bf60 con 0x7ff3e8136b50 2024-01-24T23:25:18.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 1 -- 172.21.15.115:0/2628393717 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 229869698 0 0) 0x7ff3e809b1e0 con 0x7ff3e8136f30 2024-01-24T23:25:18.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 10 monclient(hunting): my global_id is 85130 2024-01-24T23:25:18.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 1 -- 172.21.15.115:0/2628393717 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3e804ed70 con 0x7ff3e8136f30 2024-01-24T23:25:18.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 1 -- 172.21.15.115:0/2628393717 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1945603633 0 0) 0x7ff3d0003530 con 0x7ff3e813d090 2024-01-24T23:25:18.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 1 -- 172.21.15.115:0/2628393717 >> v1:172.21.15.115:6793/0 conn(0x7ff3e8136f30 legacy=0x7ff3e8137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 1 -- 172.21.15.115:0/2628393717 >> v1:172.21.15.115:6790/0 conn(0x7ff3e8136b50 legacy=0x7ff3e809bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:25:18.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 1 monclient: found mon.noname-d 2024-01-24T23:25:18.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:18.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:25:18.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 1 -- 172.21.15.115:0/2628393717 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e809af00 con 0x7ff3e813d090 2024-01-24T23:25:18.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 10 monclient: _finish_auth 0 2024-01-24T23:25:18.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:25:18.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:18.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 1 -- 172.21.15.115:0/2628393717 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff3d80017c0 con 0x7ff3e813d090 2024-01-24T23:25:18.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:25:18.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 10 monclient: dump: 2024-01-24T23:25:18.545 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.545 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.545 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.545 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.545 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.545 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.545 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.545 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.545 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.545 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.545 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.545 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.546 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.546 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.546 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.546 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 5 monclient: authenticate success, global_id 6063 2024-01-24T23:25:18.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:18.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 1 -- 172.21.15.115:0/2628393717 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff3d8001a20 con 0x7ff3e813d090 2024-01-24T23:25:18.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:18.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:18.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 4 set_mon_vals no callback set 2024-01-24T23:25:18.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 1 -- 172.21.15.115:0/2628393717 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff3d8004bf0 con 0x7ff3e813d090 2024-01-24T23:25:18.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:18.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e5ffb640 10 monclient: dump: 2024-01-24T23:25:18.546 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.547 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.547 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.547 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.547 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.547 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.547 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.547 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.547 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.547 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.547 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.547 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.547 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.547 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.547 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.547 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 10 monclient: shutdown 2024-01-24T23:25:18.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:18.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 1 -- 172.21.15.115:0/2628393717 >> v1:172.21.15.115:6792/0 conn(0x7ff3e813d090 legacy=0x7ff3e813f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 1 -- 172.21.15.115:0/2628393717 shutdown_connections 2024-01-24T23:25:18.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 1 -- 172.21.15.115:0/2628393717 >> 172.21.15.115:0/2628393717 conn(0x7ff3e80a9850 msgr2=0x7ff3e80a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:18.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 1 -- 172.21.15.115:0/2628393717 shutdown_connections 2024-01-24T23:25:18.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 1 -- 172.21.15.115:0/2628393717 wait complete. 2024-01-24T23:25:18.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 10 monclient: build_initial_monmap 2024-01-24T23:25:18.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 10 monclient: monmap: 2024-01-24T23:25:18.548 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:18.548 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:18.548 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:18.548 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:18.548 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:18.548 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:18.549 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:18.549 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:18.549 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:18.549 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:18.549 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:18.549 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:18.549 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:18.549 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:18.549 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:18.549 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 1 Processor -- start 2024-01-24T23:25:18.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 1 -- start start 2024-01-24T23:25:18.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 10 monclient: init 2024-01-24T23:25:18.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:18.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 10 monclient: _add_conns ranks=[8,6,5,3,2,0,7,4,1] 2024-01-24T23:25:18.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 10 monclient(hunting): picked mon.noname-e con 0x7ff3e8136b50 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:18.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 10 monclient(hunting): picked mon.noname-d con 0x7ff3e8136f30 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:18.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 10 monclient(hunting): picked mon.noname-h con 0x7ff3e813d090 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:18.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e80a56d0 con 0x7ff3e813d090 2024-01-24T23:25:18.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e809e4f0 con 0x7ff3e8136f30 2024-01-24T23:25:18.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e809c960 con 0x7ff3e8136b50 2024-01-24T23:25:18.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:18.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3ed5de640 10 monclient(hunting): authenticate will time out at 1197.459342s 2024-01-24T23:25:18.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e6ffd640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7ff3e8136b50 0x7ff3e812ca90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:59528/0 (socket says 172.21.15.115:59528) 2024-01-24T23:25:18.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3e6ffd640 1 -- 172.21.15.115:0/118339562 learned_addr learned my addr 172.21.15.115:0/118339562 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:18.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916848770 0 0) 0x7ff3e809c960 con 0x7ff3e8136b50 2024-01-24T23:25:18.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3cf7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3cf7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3cf7fe640 10 monclient(hunting): my global_id is 85139 2024-01-24T23:25:18.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3d4003530 con 0x7ff3e8136b50 2024-01-24T23:25:18.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3721855887 0 0) 0x7ff3e809e4f0 con 0x7ff3e8136f30 2024-01-24T23:25:18.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3cf7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3cf7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3cf7fe640 10 monclient(hunting): my global_id is 6072 2024-01-24T23:25:18.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3e809c960 con 0x7ff3e8136f30 2024-01-24T23:25:18.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3299943013 0 0) 0x7ff3d4003530 con 0x7ff3e8136b50 2024-01-24T23:25:18.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff3e809e4f0 con 0x7ff3e8136b50 2024-01-24T23:25:18.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff3e0001770 con 0x7ff3e8136b50 2024-01-24T23:25:18.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3cf7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3cf7fe640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:25:18.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3cf7fe640 10 monclient(hunting): dump: 2024-01-24T23:25:18.552 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.552 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.552 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.552 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.552 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.552 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.552 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.552 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.552 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.552 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.552 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.552 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.552 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.553 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.553 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.553 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3cf7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:18.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.534+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3510206934 0 0) 0x7ff3e809c960 con 0x7ff3e8136f30 2024-01-24T23:25:18.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff3d4003530 con 0x7ff3e8136f30 2024-01-24T23:25:18.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff3dc001220 con 0x7ff3e8136f30 2024-01-24T23:25:18.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:18.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 10 monclient(hunting): dump: 2024-01-24T23:25:18.553 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.553 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.553 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.553 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.553 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.554 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.554 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.554 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.554 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.554 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.554 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.554 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.554 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.554 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.554 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.554 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3336306300 0 0) 0x7ff3e809e4f0 con 0x7ff3e8136b50 2024-01-24T23:25:18.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 >> v1:172.21.15.115:6792/0 conn(0x7ff3e8136f30 legacy=0x7ff3e81d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 >> v1:172.21.15.132:6791/0 conn(0x7ff3e813d090 legacy=0x7ff3e81d3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:25:18.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 1 monclient: found mon.e 2024-01-24T23:25:18.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:18.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:18.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e80a0e10 con 0x7ff3e8136b50 2024-01-24T23:25:18.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 10 monclient: _finish_auth 0 2024-01-24T23:25:18.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:25:18.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:18.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3ed5de640 5 monclient: authenticate success, global_id 85139 2024-01-24T23:25:18.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3ed5de640 10 monclient: _renew_subs 2024-01-24T23:25:18.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3ed5de640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:18.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3ed5de640 1 -- 172.21.15.115:0/118339562 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e80a05b0 con 0x7ff3e8136b50 2024-01-24T23:25:18.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3ed5de640 10 monclient: _renew_subs 2024-01-24T23:25:18.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3ed5de640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:18.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3ed5de640 1 -- 172.21.15.115:0/118339562 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e80a0b30 con 0x7ff3e8136b50 2024-01-24T23:25:18.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff3e00018d0 con 0x7ff3e8136b50 2024-01-24T23:25:18.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:18.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff3e0005160 con 0x7ff3e8136b50 2024-01-24T23:25:18.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3e57fa640 4 set_mon_vals no callback set 2024-01-24T23:25:18.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:18.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 10 monclient: dump: 2024-01-24T23:25:18.556 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.556 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.556 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.556 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.556 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.556 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.556 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.556 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.557 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.557 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.557 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.557 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.557 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.557 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.557 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.557 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3ed5de640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:18.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3ed5de640 1 -- 172.21.15.115:0/118339562 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff3c80065d0 con 0x7ff3c8002870 2024-01-24T23:25:18.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7ff3e000fbe0 con 0x7ff3e8136b50 2024-01-24T23:25:18.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(128..128 src has 1..128) v4 ==== 6989+0+0 (unknown 538550023 0 0) 0x7ff3e0044760 con 0x7ff3e8136b50 2024-01-24T23:25:18.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3081582942 0 0) 0x7ff3c80065d0 con 0x7ff3c8002870 2024-01-24T23:25:18.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 10 monclient(hunting): my global_id is 94239 2024-01-24T23:25:18.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3e809e4f0 con 0x7ff3c8002870 2024-01-24T23:25:18.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3e77fe640 10 monclient: get_auth_request con 0x7ff3d40376f0 auth_method 0 2024-01-24T23:25:18.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1423754544 0 0) 0x7ff3e809e4f0 con 0x7ff3c8002870 2024-01-24T23:25:18.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff3c80065d0 con 0x7ff3c8002870 2024-01-24T23:25:18.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff3dc003940 con 0x7ff3c8002870 2024-01-24T23:25:18.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:18.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 10 monclient: dump: 2024-01-24T23:25:18.558 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.558 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.558 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.558 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.558 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.558 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.558 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.558 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.558 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.559 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.559 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.559 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.559 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.559 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.559 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.559 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1926045199 0 0) 0x7ff3c80065d0 con 0x7ff3c8002870 2024-01-24T23:25:18.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 --> v1:172.21.15.115:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff3c800a120 con 0x7ff3c8002870 2024-01-24T23:25:18.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 <== mon.0 v1:172.21.15.115:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7ff3dc005bc0 con 0x7ff3c8002870 2024-01-24T23:25:18.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:18.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:18.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 >> v1:172.21.15.115:6789/0 conn(0x7ff3c8002870 legacy=0x7ff3c8004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.538+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 >> v1:172.21.15.115:6789/0 conn(0x7ff3c8002870 legacy=0x7ff3c8004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:18.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3ed5de640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:25:18.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3ed5de640 1 -- 172.21.15.115:0/118339562 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff3c8009df0 con 0x7ff3c80090b0 2024-01-24T23:25:18.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1854572160 0 0) 0x7ff3c8009df0 con 0x7ff3c80090b0 2024-01-24T23:25:18.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3cf7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3cf7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3cf7fe640 10 monclient(hunting): my global_id is 94248 2024-01-24T23:25:18.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3c80065d0 con 0x7ff3c80090b0 2024-01-24T23:25:18.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3967732278 0 0) 0x7ff3c80065d0 con 0x7ff3c80090b0 2024-01-24T23:25:18.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff3c8009df0 con 0x7ff3c80090b0 2024-01-24T23:25:18.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff3dc0042f0 con 0x7ff3c80090b0 2024-01-24T23:25:18.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3cf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3cf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:18.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3cf7fe640 10 monclient: dump: 2024-01-24T23:25:18.562 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.562 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.562 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.562 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.562 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.562 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.562 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.562 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.562 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.562 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.562 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.562 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.562 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.563 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.563 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.563 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 846710299 0 0) 0x7ff3c8009df0 con 0x7ff3c80090b0 2024-01-24T23:25:18.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 --> v1:172.21.15.115:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff3c800c310 con 0x7ff3c80090b0 2024-01-24T23:25:18.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 <== mon.0 v1:172.21.15.115:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 4003351440) 0x7ff3dc0058a0 con 0x7ff3c80090b0 2024-01-24T23:25:18.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3cf7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:25:18.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3cf7fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:18.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 >> v1:172.21.15.115:6789/0 conn(0x7ff3c80090b0 legacy=0x7ff3c8009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3cf7fe640 1 -- 172.21.15.115:0/118339562 >> v1:172.21.15.115:6789/0 conn(0x7ff3c80090b0 legacy=0x7ff3c8009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:18.563 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:25:18.563 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:18.563 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:18.564 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:25:18.564 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 24, 2024-01-24T23:25:18.564 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:25:18.564 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:25:18.564 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:25:18.564 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:25:18.564 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:25:18.564 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:25:18.564 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:25:18.564 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:25:18.564 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:25:18.564 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:25:18.564 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:18.564 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 3, 2024-01-24T23:25:18.565 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:18.565 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:25:18.565 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:25:18.565 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:18.565 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:18.565 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:18.565 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:18.565 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:18.565 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:18.565 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:18.565 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:18.565 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:18.565 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:18.565 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:18.565 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:18.566 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:25:18.566 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:25:18.566 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:18.566 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:18.566 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:18.566 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:18.566 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:18.566 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:18.566 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:18.566 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:18.566 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:18.566 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:18.566 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:18.566 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.567 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:25:18.567 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:25:18.567 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:25:18.567 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:25:18.567 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:25:18.567 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:25:18.567 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:18.567 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:18.567 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:25:18.567 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:25:18.567 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:25:18.567 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:25:18.567 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:25:18.567 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:25:18.567 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:25:18.567 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:18.568 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:25:18.568 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:18.568 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:18.568 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:18.568 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:18.568 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:18.568 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:18.568 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:18.568 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:18.568 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:18.568 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:18.568 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:18.568 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:18.568 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:25:18.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.569 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:25:18.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.569 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:18.569 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:18.569 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.569 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.569 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.569 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:25:18.569 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.569 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.569 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.569 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:25:18.569 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:25:18.569 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.570 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.570 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.570 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:18.570 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:18.570 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.570 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.570 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.570 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:25:18.570 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.570 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.570 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.571 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:25:18.571 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:25:18.571 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.571 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.571 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.571 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:18.571 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:18.571 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.571 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.571 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.571 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:25:18.572 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.572 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.572 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.572 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:25:18.572 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:25:18.572 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.572 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.572 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.572 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:18.572 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:18.572 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.573 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.573 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.573 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:25:18.573 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.573 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.573 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.573 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:25:18.573 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:25:18.573 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.573 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.573 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.574 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:18.574 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:18.574 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.574 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.574 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.574 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:25:18.574 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.574 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.574 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.574 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:25:18.574 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:25:18.574 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.575 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.575 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.575 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:18.575 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:18.575 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.575 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.575 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.575 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:25:18.575 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.575 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.575 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.576 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:25:18.576 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:25:18.576 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.576 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.576 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.576 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:18.576 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:18.576 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.576 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.576 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.576 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:25:18.577 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.577 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.577 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.577 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:25:18.577 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:25:18.577 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.577 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.577 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.577 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:18.577 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:18.577 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.577 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.578 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.578 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:25:18.578 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.578 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.578 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.578 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:25:18.578 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:25:18.578 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.578 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.578 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.579 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:18.579 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:18.579 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.579 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.579 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.579 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:25:18.579 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.579 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.579 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.579 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:25:18.579 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:25:18.579 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.579 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.579 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.580 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.580 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.580 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:25:18.580 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:25:18.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.580 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:18.580 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:18.580 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:18.580 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.580 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:18.580 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:25:18.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.580 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:18.580 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:18.581 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:18.581 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.581 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.581 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:25:18.581 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:25:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3ed5de640 1 -- 172.21.15.115:0/118339562 >> v1:172.21.15.132:6812/1835401922 conn(0x7ff3d40376f0 legacy=0x7ff3d4039bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3ed5de640 10 monclient: shutdown 2024-01-24T23:25:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3ed5de640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3ed5de640 1 -- 172.21.15.115:0/118339562 >> v1:172.21.15.115:6793/0 conn(0x7ff3e8136b50 legacy=0x7ff3e812ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3e77fe640 1 -- 172.21.15.115:0/118339562 reap_dead start 2024-01-24T23:25:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3e77fe640 1 -- 172.21.15.115:0/118339562 reap_dead start 2024-01-24T23:25:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3ed5de640 1 -- 172.21.15.115:0/118339562 shutdown_connections 2024-01-24T23:25:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3ed5de640 1 -- 172.21.15.115:0/118339562 >> 172.21.15.115:0/118339562 conn(0x7ff3e80a9850 msgr2=0x7ff3e80a3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:18.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3ed5de640 1 -- 172.21.15.115:0/118339562 shutdown_connections 2024-01-24T23:25:18.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.546+0000 7ff3ed5de640 1 -- 172.21.15.115:0/118339562 wait complete. 2024-01-24T23:25:18.583 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:25:18.583 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-01-24T23:25:18.583 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-01-24T23:25:18.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.690+0000 7fdccd21c640 10 monclient: get_monmap_and_config 2024-01-24T23:25:18.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.690+0000 7fdccd21c640 10 monclient: build_initial_monmap 2024-01-24T23:25:18.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.690+0000 7fdccd21c640 10 monclient: monmap: 2024-01-24T23:25:18.693 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:18.693 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.694 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:18.694025+0000 2024-01-24T23:25:18.694 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:18.694025+0000 2024-01-24T23:25:18.694 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:18.694 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.694 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:18.694 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:18.694 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:18.694 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:18.694 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:18.694 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:18.694 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:18.694 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:18.694 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:18.694 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.690+0000 7fdccd21c640 1 Processor -- start 2024-01-24T23:25:18.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.690+0000 7fdccd21c640 1 -- start start 2024-01-24T23:25:18.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.690+0000 7fdccd21c640 10 monclient: init 2024-01-24T23:25:18.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:18.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 10 monclient: _add_conns ranks=[7,4,3,6,5,2,8,1,0] 2024-01-24T23:25:18.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 10 monclient(hunting): picked mon.noname-i con 0x7fdcc8133c10 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:18.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 10 monclient(hunting): picked mon.noname-c con 0x7fdcc8136960 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:18.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 10 monclient(hunting): picked mon.noname-g con 0x7fdcc813a470 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:18.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc804ed70 con 0x7fdcc813a470 2024-01-24T23:25:18.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc809dee0 con 0x7fdcc8136960 2024-01-24T23:25:18.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc809d400 con 0x7fdcc8133c10 2024-01-24T23:25:18.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:18.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 10 monclient(hunting): authenticate will time out at 1197.616711s 2024-01-24T23:25:18.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc659c640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fdcc8136960 0x7fdcc8138d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:44676/0 (socket says 172.21.15.115:44676) 2024-01-24T23:25:18.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc659c640 1 -- 172.21.15.115:0/3271865695 learned_addr learned my addr 172.21.15.115:0/3271865695 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:18.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 1 -- 172.21.15.115:0/3271865695 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1642436044 0 0) 0x7fdcc809dee0 con 0x7fdcc8136960 2024-01-24T23:25:18.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 10 monclient(hunting): my global_id is 94252 2024-01-24T23:25:18.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 1 -- 172.21.15.115:0/3271865695 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcb0005740 con 0x7fdcc8136960 2024-01-24T23:25:18.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 1 -- 172.21.15.115:0/3271865695 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 673130664 0 0) 0x7fdcb0005740 con 0x7fdcc8136960 2024-01-24T23:25:18.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 1 -- 172.21.15.115:0/3271865695 >> v1:172.21.15.132:6792/0 conn(0x7fdcc8133c10 legacy=0x7fdcc8133ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 1 -- 172.21.15.115:0/3271865695 >> v1:172.21.15.132:6790/0 conn(0x7fdcc813a470 legacy=0x7fdcc813c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 10 monclient: _finish_hunting 0 2024-01-24T23:25:18.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 1 monclient: found mon.noname-c 2024-01-24T23:25:18.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:18.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:25:18.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 1 -- 172.21.15.115:0/3271865695 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc809f380 con 0x7fdcc8136960 2024-01-24T23:25:18.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 10 monclient: _finish_auth 0 2024-01-24T23:25:18.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 10 monclient: _check_auth_tickets 2024-01-24T23:25:18.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:18.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 5 monclient: authenticate success, global_id 94252 2024-01-24T23:25:18.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 1 -- 172.21.15.115:0/3271865695 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdcbc0017c0 con 0x7fdcc8136960 2024-01-24T23:25:18.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:18.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:25:18.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 10 monclient: dump: 2024-01-24T23:25:18.704 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.704 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.704 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.704 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.704 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.705 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.705 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.705 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.705 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.705 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.705 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.705 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.705 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.705 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.705 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.705 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:18.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 1 -- 172.21.15.115:0/3271865695 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdcbc001ae0 con 0x7fdcc8136960 2024-01-24T23:25:18.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:18.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:18.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 4 set_mon_vals no callback set 2024-01-24T23:25:18.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 1 -- 172.21.15.115:0/3271865695 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdcbc004bf0 con 0x7fdcc8136960 2024-01-24T23:25:18.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:18.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc5d9b640 10 monclient: dump: 2024-01-24T23:25:18.706 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.706 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.706 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.706 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.706 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.706 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.706 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.706 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.706 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.706 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.707 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.707 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.707 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.707 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.707 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.707 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 10 monclient: shutdown 2024-01-24T23:25:18.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:18.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 1 -- 172.21.15.115:0/3271865695 >> v1:172.21.15.115:6791/0 conn(0x7fdcc8136960 legacy=0x7fdcc8138d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 1 -- 172.21.15.115:0/3271865695 shutdown_connections 2024-01-24T23:25:18.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 1 -- 172.21.15.115:0/3271865695 >> 172.21.15.115:0/3271865695 conn(0x7fdcc80a5730 msgr2=0x7fdcc80a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:18.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 1 -- 172.21.15.115:0/3271865695 shutdown_connections 2024-01-24T23:25:18.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 1 -- 172.21.15.115:0/3271865695 wait complete. 2024-01-24T23:25:18.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 10 monclient: build_initial_monmap 2024-01-24T23:25:18.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 10 monclient: monmap: 2024-01-24T23:25:18.708 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:18.708 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:18.708 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:18.708 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:18.708 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:18.708 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:18.708 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:18.708 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:18.708 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:18.708 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:18.708 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:18.708 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:18.708 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:18.708 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:18.709 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:18.709 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 1 Processor -- start 2024-01-24T23:25:18.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 1 -- start start 2024-01-24T23:25:18.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 10 monclient: init 2024-01-24T23:25:18.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:18.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 10 monclient: _add_conns ranks=[5,6,0,1,2,7,3,4,8] 2024-01-24T23:25:18.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 10 monclient(hunting): picked mon.noname-h con 0x7fdcc8133c10 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:18.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 10 monclient(hunting): picked mon.noname-d con 0x7fdcc8136960 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:18.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 10 monclient(hunting): picked mon.noname-a con 0x7fdcc813a470 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:18.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc8131940 con 0x7fdcc813a470 2024-01-24T23:25:18.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc8131c20 con 0x7fdcc8133c10 2024-01-24T23:25:18.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc8139b50 con 0x7fdcc8136960 2024-01-24T23:25:18.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:18.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdccd21c640 10 monclient(hunting): authenticate will time out at 1197.619412s 2024-01-24T23:25:18.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc659c640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fdcc8136960 0x7fdcc8139440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:40516/0 (socket says 172.21.15.115:40516) 2024-01-24T23:25:18.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcc659c640 1 -- 172.21.15.115:0/3349357580 learned_addr learned my addr 172.21.15.115:0/3349357580 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:18.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3268830725 0 0) 0x7fdcc8139b50 con 0x7fdcc8136960 2024-01-24T23:25:18.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcaf7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcaf7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcaf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcaf7fe640 10 monclient(hunting): my global_id is 6081 2024-01-24T23:25:18.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcb4003530 con 0x7fdcc8136960 2024-01-24T23:25:18.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1502609499 0 0) 0x7fdcc8131940 con 0x7fdcc813a470 2024-01-24T23:25:18.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcaf7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcaf7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcaf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcaf7fe640 10 monclient(hunting): my global_id is 94257 2024-01-24T23:25:18.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.694+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcc8139b50 con 0x7fdcc813a470 2024-01-24T23:25:18.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3837915717 0 0) 0x7fdcb4003530 con 0x7fdcc8136960 2024-01-24T23:25:18.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdcc8131940 con 0x7fdcc8136960 2024-01-24T23:25:18.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1813793516 0 0) 0x7fdcc8139b50 con 0x7fdcc813a470 2024-01-24T23:25:18.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdcb4003530 con 0x7fdcc813a470 2024-01-24T23:25:18.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdcbc001780 con 0x7fdcc8136960 2024-01-24T23:25:18.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:25:18.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient(hunting): dump: 2024-01-24T23:25:18.711 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.711 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.711 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.711 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.711 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.711 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.712 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.712 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.712 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.712 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.712 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.712 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.712 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.712 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.712 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.712 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:18.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdcc0001c80 con 0x7fdcc813a470 2024-01-24T23:25:18.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:18.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient(hunting): dump: 2024-01-24T23:25:18.713 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.713 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.713 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.713 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.713 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.713 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.713 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.713 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.713 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.713 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.713 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.713 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.713 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.713 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.713 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.714 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2184667532 0 0) 0x7fdcc8131c20 con 0x7fdcc8133c10 2024-01-24T23:25:18.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient(hunting): my global_id is 5873 2024-01-24T23:25:18.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcc8139b50 con 0x7fdcc8133c10 2024-01-24T23:25:18.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 792281025 0 0) 0x7fdcc8131940 con 0x7fdcc8136960 2024-01-24T23:25:18.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 >> v1:172.21.15.132:6791/0 conn(0x7fdcc8133c10 legacy=0x7fdcc8138d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 >> v1:172.21.15.115:6789/0 conn(0x7fdcc813a470 legacy=0x7fdcc81d3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:25:18.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 monclient: found mon.d 2024-01-24T23:25:18.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:18.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:25:18.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc80a1310 con 0x7fdcc8136960 2024-01-24T23:25:18.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient: _finish_auth 0 2024-01-24T23:25:18.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:25:18.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:18.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdccd21c640 5 monclient: authenticate success, global_id 6081 2024-01-24T23:25:18.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdccd21c640 10 monclient: _renew_subs 2024-01-24T23:25:18.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdccd21c640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:25:18.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdccd21c640 1 -- 172.21.15.115:0/3349357580 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc8130ca0 con 0x7fdcc8136960 2024-01-24T23:25:18.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdccd21c640 10 monclient: _renew_subs 2024-01-24T23:25:18.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdccd21c640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:25:18.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdccd21c640 1 -- 172.21.15.115:0/3349357580 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc8131f00 con 0x7fdcc8136960 2024-01-24T23:25:18.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdcbc001c50 con 0x7fdcc8136960 2024-01-24T23:25:18.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:18.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdcbc0054b0 con 0x7fdcc8136960 2024-01-24T23:25:18.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:18.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcc559a640 4 set_mon_vals no callback set 2024-01-24T23:25:18.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient: dump: 2024-01-24T23:25:18.716 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.716 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.716 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.716 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.716 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.716 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.716 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.716 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.716 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.716 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.716 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.716 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.717 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.717 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.717 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.717 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdccd21c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:18.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fdcbc00ff30 con 0x7fdcc8136960 2024-01-24T23:25:18.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdccd21c640 1 -- 172.21.15.115:0/3349357580 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdca80065d0 con 0x7fdca8002870 2024-01-24T23:25:18.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(128..128 src has 1..128) v4 ==== 6989+0+0 (unknown 538550023 0 0) 0x7fdcbc0446e0 con 0x7fdcc8136960 2024-01-24T23:25:18.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2492705175 0 0) 0x7fdca80065d0 con 0x7fdca8002870 2024-01-24T23:25:18.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient(hunting): my global_id is 85304 2024-01-24T23:25:18.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcc8131940 con 0x7fdca8002870 2024-01-24T23:25:18.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcc759e640 10 monclient: get_auth_request con 0x7fdcb4037730 auth_method 0 2024-01-24T23:25:18.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2738524104 0 0) 0x7fdcc8131940 con 0x7fdca8002870 2024-01-24T23:25:18.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdca80065d0 con 0x7fdca8002870 2024-01-24T23:25:18.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdcb80018b0 con 0x7fdca8002870 2024-01-24T23:25:18.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:25:18.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient: dump: 2024-01-24T23:25:18.718 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.718 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.718 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.718 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.718 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.718 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.718 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.718 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.719 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.719 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.719 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.719 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.719 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.719 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.719 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.719 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3650234451 0 0) 0x7fdca80065d0 con 0x7fdca8002870 2024-01-24T23:25:18.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 --> v1:172.21.15.115:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdca800a120 con 0x7fdca8002870 2024-01-24T23:25:18.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 <== mon.2 v1:172.21.15.115:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fdcb8003600 con 0x7fdca8002870 2024-01-24T23:25:18.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:18.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:18.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 >> v1:172.21.15.115:6790/0 conn(0x7fdca8002870 legacy=0x7fdca8004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.698+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 >> v1:172.21.15.115:6790/0 conn(0x7fdca8002870 legacy=0x7fdca8004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:18.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdccd21c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:25:18.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdccd21c640 1 -- 172.21.15.115:0/3349357580 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdca8009e10 con 0x7fdca80090d0 2024-01-24T23:25:18.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3595905865 0 0) 0x7fdca8009e10 con 0x7fdca80090d0 2024-01-24T23:25:18.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdcaf7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdcaf7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdcaf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdcaf7fe640 10 monclient(hunting): my global_id is 85313 2024-01-24T23:25:18.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdca80065d0 con 0x7fdca80090d0 2024-01-24T23:25:18.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3287168880 0 0) 0x7fdca80065d0 con 0x7fdca80090d0 2024-01-24T23:25:18.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdca8009e10 con 0x7fdca80090d0 2024-01-24T23:25:18.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdcb8003d20 con 0x7fdca80090d0 2024-01-24T23:25:18.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdcaf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdcaf7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:25:18.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdcaf7fe640 10 monclient: dump: 2024-01-24T23:25:18.722 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.722 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.722 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.722 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.722 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.722 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.722 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.722 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.723 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.723 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.723 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.723 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.723 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.723 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.723 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.723 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1552761962 0 0) 0x7fdca8009e10 con 0x7fdca80090d0 2024-01-24T23:25:18.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 --> v1:172.21.15.115:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdca800c310 con 0x7fdca80090d0 2024-01-24T23:25:18.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 <== mon.2 v1:172.21.15.115:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 835064206) 0x7fdcb80060a0 con 0x7fdca80090d0 2024-01-24T23:25:18.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdcaf7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:25:18.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdcaf7fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:18.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 >> v1:172.21.15.115:6790/0 conn(0x7fdca80090d0 legacy=0x7fdca8009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdcaf7fe640 1 -- 172.21.15.115:0/3349357580 >> v1:172.21.15.115:6790/0 conn(0x7fdca80090d0 legacy=0x7fdca8009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:18.724 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:25:18.724 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:18.724 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:18.724 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:25:18.724 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 24, 2024-01-24T23:25:18.724 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:25:18.724 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:25:18.724 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:25:18.724 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:25:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:25:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:25:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:25:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:25:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:25:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:25:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 3, 2024-01-24T23:25:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:25:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:25:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:25:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:25:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:25:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:25:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:25:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:25:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:25:18.728 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:25:18.728 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:18.728 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:18.728 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:25:18.728 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:25:18.728 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:25:18.728 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:25:18.728 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:25:18.728 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:25:18.728 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:25:18.728 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:18.728 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:25:18.728 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:18.729 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:18.729 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:18.729 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:18.729 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:18.729 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:18.729 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:18.729 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:18.729 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:18.729 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:18.729 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:18.729 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:18.729 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:25:18.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.729 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:25:18.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.730 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:18.730 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:18.730 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.730 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.730 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.730 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:25:18.730 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.730 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.730 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:25:18.731 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:25:18.731 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.731 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.731 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.731 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:18.731 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:18.731 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.731 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.731 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.731 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:25:18.731 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.732 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.732 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.732 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:25:18.732 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:25:18.732 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.732 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.732 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.732 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:18.732 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:18.732 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.732 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.733 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.733 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:25:18.733 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.733 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.733 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:25:18.733 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:25:18.733 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.733 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.733 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.734 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:18.734 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:18.734 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.734 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.734 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.734 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:25:18.734 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.734 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.734 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.734 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:25:18.734 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:25:18.734 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.735 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.735 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.735 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:18.735 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:18.735 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.735 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.735 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.735 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:25:18.735 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.735 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.735 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.736 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:25:18.736 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:25:18.736 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.736 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.736 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.736 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:18.736 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:18.736 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.736 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.736 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.737 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:25:18.737 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.737 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.737 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.737 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:25:18.737 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:25:18.737 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.737 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.737 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.737 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:18.737 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:18.738 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.738 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.738 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.738 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:25:18.738 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.738 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.738 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.738 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:25:18.738 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:25:18.738 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.738 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.738 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.739 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:18.739 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:18.739 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.739 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.739 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.739 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:25:18.739 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.739 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.739 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:25:18.739 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:25:18.739 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.739 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.740 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.740 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:18.740 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:18.740 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.740 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.740 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.740 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:25:18.740 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.740 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.740 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.741 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:25:18.741 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:25:18.741 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.741 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.741 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.741 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.741 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.741 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:25:18.741 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:25:18.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.741 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:18.741 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:18.741 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:18.741 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.741 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:18.742 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:25:18.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.742 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:18.742 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:18.742 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:25:18.742 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.742 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:18.742 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:25:18.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.742 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:18.742 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:18.742 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:18.742 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.742 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.743 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:25:18.743 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:25:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdccd21c640 1 -- 172.21.15.115:0/3349357580 >> v1:172.21.15.132:6812/1835401922 conn(0x7fdcb4037730 legacy=0x7fdcb4039bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdccd21c640 10 monclient: shutdown 2024-01-24T23:25:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdccd21c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdccd21c640 1 -- 172.21.15.115:0/3349357580 >> v1:172.21.15.115:6792/0 conn(0x7fdcc8136960 legacy=0x7fdcc8139440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdcc759e640 1 -- 172.21.15.115:0/3349357580 reap_dead start 2024-01-24T23:25:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdcc759e640 1 -- 172.21.15.115:0/3349357580 reap_dead start 2024-01-24T23:25:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdccd21c640 1 -- 172.21.15.115:0/3349357580 shutdown_connections 2024-01-24T23:25:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdccd21c640 1 -- 172.21.15.115:0/3349357580 >> 172.21.15.115:0/3349357580 conn(0x7fdcc80a5730 msgr2=0x7fdcc812e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdccd21c640 1 -- 172.21.15.115:0/3349357580 shutdown_connections 2024-01-24T23:25:18.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.706+0000 7fdccd21c640 1 -- 172.21.15.115:0/3349357580 wait complete. 2024-01-24T23:25:18.745 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:25:18.745 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-01-24T23:25:18.745 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-01-24T23:25:18.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.858+0000 7fbe3b633640 10 monclient: get_monmap_and_config 2024-01-24T23:25:18.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.858+0000 7fbe3b633640 10 monclient: build_initial_monmap 2024-01-24T23:25:18.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.858+0000 7fbe3b633640 10 monclient: monmap: 2024-01-24T23:25:18.871 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:18.871 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.871 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:18.862622+0000 2024-01-24T23:25:18.871 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:18.862622+0000 2024-01-24T23:25:18.871 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:18.871 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.871 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:18.871 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:18.872 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:18.872 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:18.872 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:18.872 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:18.872 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:18.872 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:18.872 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:18.872 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 1 Processor -- start 2024-01-24T23:25:18.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 1 -- start start 2024-01-24T23:25:18.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 10 monclient: init 2024-01-24T23:25:18.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:18.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 10 monclient: _add_conns ranks=[5,0,2,1,3,4,6,7,8] 2024-01-24T23:25:18.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 10 monclient(hunting): picked mon.noname-h con 0x7fbe34134740 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:18.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 10 monclient(hunting): picked mon.noname-a con 0x7fbe3409ec80 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:18.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 10 monclient(hunting): picked mon.noname-b con 0x7fbe34135230 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:18.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3404ed70 con 0x7fbe3409ec80 2024-01-24T23:25:18.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe340a0d90 con 0x7fbe34135230 2024-01-24T23:25:18.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3409e7c0 con 0x7fbe34134740 2024-01-24T23:25:18.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:18.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 10 monclient(hunting): authenticate will time out at 1197.785217s 2024-01-24T23:25:18.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe39bd0640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fbe34135230 0x7fbe3413d450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:60420/0 (socket says 172.21.15.115:60420) 2024-01-24T23:25:18.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe39bd0640 1 -- 172.21.15.115:0/2938901005 learned_addr learned my addr 172.21.15.115:0/2938901005 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:18.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 1 -- 172.21.15.115:0/2938901005 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3142845253 0 0) 0x7fbe3404ed70 con 0x7fbe3409ec80 2024-01-24T23:25:18.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 10 monclient(hunting): my global_id is 94266 2024-01-24T23:25:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 1 -- 172.21.15.115:0/2938901005 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe20003530 con 0x7fbe3409ec80 2024-01-24T23:25:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 1 -- 172.21.15.115:0/2938901005 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 653942257 0 0) 0x7fbe340a0d90 con 0x7fbe34135230 2024-01-24T23:25:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 10 monclient(hunting): my global_id is 85322 2024-01-24T23:25:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 1 -- 172.21.15.115:0/2938901005 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe3404ed70 con 0x7fbe34135230 2024-01-24T23:25:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 1 -- 172.21.15.115:0/2938901005 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 426119777 0 0) 0x7fbe3409e7c0 con 0x7fbe34134740 2024-01-24T23:25:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 10 monclient(hunting): my global_id is 5882 2024-01-24T23:25:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 1 -- 172.21.15.115:0/2938901005 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe340a0d90 con 0x7fbe34134740 2024-01-24T23:25:18.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 1 -- 172.21.15.115:0/2938901005 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3818351728 0 0) 0x7fbe20003530 con 0x7fbe3409ec80 2024-01-24T23:25:18.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 1 -- 172.21.15.115:0/2938901005 >> v1:172.21.15.132:6791/0 conn(0x7fbe34134740 legacy=0x7fbe34134b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 1 -- 172.21.15.115:0/2938901005 >> v1:172.21.15.115:6790/0 conn(0x7fbe34135230 legacy=0x7fbe3413d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:25:18.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 1 monclient: found mon.noname-a 2024-01-24T23:25:18.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:18.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:25:18.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 1 -- 172.21.15.115:0/2938901005 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe340a1070 con 0x7fbe3409ec80 2024-01-24T23:25:18.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 10 monclient: _finish_auth 0 2024-01-24T23:25:18.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:25:18.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:18.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 5 monclient: authenticate success, global_id 94266 2024-01-24T23:25:18.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:18.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 1 -- 172.21.15.115:0/2938901005 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbe240016f0 con 0x7fbe3409ec80 2024-01-24T23:25:18.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:25:18.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 10 monclient: dump: 2024-01-24T23:25:18.876 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.876 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.876 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.876 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.876 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.876 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.876 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.876 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.876 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.876 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.876 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.876 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.876 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.877 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.877 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.877 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:18.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 1 -- 172.21.15.115:0/2938901005 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbe240019d0 con 0x7fbe3409ec80 2024-01-24T23:25:18.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:18.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 1 -- 172.21.15.115:0/2938901005 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbe24001c00 con 0x7fbe3409ec80 2024-01-24T23:25:18.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:18.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 4 set_mon_vals no callback set 2024-01-24T23:25:18.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:18.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe2bfff640 10 monclient: dump: 2024-01-24T23:25:18.877 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.877 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.877 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.878 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.878 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.878 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.878 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.878 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.878 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.878 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.878 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.878 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.878 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.878 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.878 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.878 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 10 monclient: shutdown 2024-01-24T23:25:18.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:18.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 1 -- 172.21.15.115:0/2938901005 >> v1:172.21.15.115:6789/0 conn(0x7fbe3409ec80 legacy=0x7fbe34139d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 1 -- 172.21.15.115:0/2938901005 shutdown_connections 2024-01-24T23:25:18.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 1 -- 172.21.15.115:0/2938901005 >> 172.21.15.115:0/2938901005 conn(0x7fbe340a5730 msgr2=0x7fbe340a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:18.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 1 -- 172.21.15.115:0/2938901005 shutdown_connections 2024-01-24T23:25:18.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 1 -- 172.21.15.115:0/2938901005 wait complete. 2024-01-24T23:25:18.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 10 monclient: build_initial_monmap 2024-01-24T23:25:18.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.862+0000 7fbe3b633640 10 monclient: monmap: 2024-01-24T23:25:18.879 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:18.879 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:18.879 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:18.879 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:18.879 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:18.879 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:18.879 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:18.880 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:18.880 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:18.880 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:18.880 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:18.880 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:18.880 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:18.880 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:18.880 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:18.880 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe3b633640 1 Processor -- start 2024-01-24T23:25:18.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe3b633640 1 -- start start 2024-01-24T23:25:18.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe3b633640 10 monclient: init 2024-01-24T23:25:18.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe3b633640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:18.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe3b633640 10 monclient: _add_conns ranks=[3,8,5,2,7,6,4,0,1] 2024-01-24T23:25:18.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe3b633640 10 monclient(hunting): picked mon.noname-g con 0x7fbe3409ec80 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:18.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe3b633640 10 monclient(hunting): picked mon.noname-e con 0x7fbe34134740 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:18.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe3b633640 10 monclient(hunting): picked mon.noname-h con 0x7fbe34135230 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:18.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe3b633640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe34137050 con 0x7fbe3409ec80 2024-01-24T23:25:18.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe3b633640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe34071c00 con 0x7fbe34135230 2024-01-24T23:25:18.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe3b633640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe340716b0 con 0x7fbe34134740 2024-01-24T23:25:18.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe3b633640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:18.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe3b633640 10 monclient(hunting): authenticate will time out at 1197.788809s 2024-01-24T23:25:18.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe38bce640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fbe34134740 0x7fbe341ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:59544/0 (socket says 172.21.15.115:59544) 2024-01-24T23:25:18.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe38bce640 1 -- 172.21.15.115:0/485929898 learned_addr learned my addr 172.21.15.115:0/485929898 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:18.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 448322796 0 0) 0x7fbe340716b0 con 0x7fbe34134740 2024-01-24T23:25:18.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient(hunting): my global_id is 85148 2024-01-24T23:25:18.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe18003530 con 0x7fbe34134740 2024-01-24T23:25:18.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 225708386 0 0) 0x7fbe18003530 con 0x7fbe34134740 2024-01-24T23:25:18.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbe340716b0 con 0x7fbe34134740 2024-01-24T23:25:18.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbe24003940 con 0x7fbe34134740 2024-01-24T23:25:18.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:25:18.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient(hunting): dump: 2024-01-24T23:25:18.882 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.882 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.882 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.882 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.882 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.882 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.882 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.882 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.883 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3447384819 0 0) 0x7fbe34071c00 con 0x7fbe34135230 2024-01-24T23:25:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient(hunting): my global_id is 5891 2024-01-24T23:25:18.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe18003530 con 0x7fbe34135230 2024-01-24T23:25:18.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2474785814 0 0) 0x7fbe34137050 con 0x7fbe3409ec80 2024-01-24T23:25:18.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient(hunting): my global_id is 5907 2024-01-24T23:25:18.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe34071c00 con 0x7fbe3409ec80 2024-01-24T23:25:18.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1233432591 0 0) 0x7fbe340716b0 con 0x7fbe34134740 2024-01-24T23:25:18.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 >> v1:172.21.15.132:6791/0 conn(0x7fbe34135230 legacy=0x7fbe341d19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 >> v1:172.21.15.132:6790/0 conn(0x7fbe3409ec80 legacy=0x7fbe3413df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:25:18.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 monclient: found mon.e 2024-01-24T23:25:18.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:18.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:18.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe341389b0 con 0x7fbe34134740 2024-01-24T23:25:18.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient: _finish_auth 0 2024-01-24T23:25:18.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:25:18.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:18.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe3b633640 5 monclient: authenticate success, global_id 85148 2024-01-24T23:25:18.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe3b633640 10 monclient: _renew_subs 2024-01-24T23:25:18.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe3b633640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:18.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe3b633640 1 -- 172.21.15.115:0/485929898 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe34138110 con 0x7fbe34134740 2024-01-24T23:25:18.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe3b633640 10 monclient: _renew_subs 2024-01-24T23:25:18.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe3b633640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:18.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe3b633640 1 -- 172.21.15.115:0/485929898 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3413e610 con 0x7fbe34134740 2024-01-24T23:25:18.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbe24001bc0 con 0x7fbe34134740 2024-01-24T23:25:18.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:18.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbe240041e0 con 0x7fbe34134740 2024-01-24T23:25:18.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe2b7fe640 4 set_mon_vals no callback set 2024-01-24T23:25:18.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:18.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient: dump: 2024-01-24T23:25:18.886 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.886 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.886 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.886 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.886 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.886 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.886 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.886 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.886 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.886 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.887 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.887 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.887 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.887 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.887 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.887 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe3b633640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:18.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe3b633640 1 -- 172.21.15.115:0/485929898 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbe140065d0 con 0x7fbe14002870 2024-01-24T23:25:18.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fbe240100b0 con 0x7fbe34134740 2024-01-24T23:25:18.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(128..128 src has 1..128) v4 ==== 6989+0+0 (unknown 538550023 0 0) 0x7fbe24044680 con 0x7fbe34134740 2024-01-24T23:25:18.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2560331352 0 0) 0x7fbe140065d0 con 0x7fbe14002870 2024-01-24T23:25:18.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient(hunting): my global_id is 94261 2024-01-24T23:25:18.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe340716b0 con 0x7fbe14002870 2024-01-24T23:25:18.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe39bd0640 10 monclient: get_auth_request con 0x7fbe18037140 auth_method 0 2024-01-24T23:25:18.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 172836445 0 0) 0x7fbe340716b0 con 0x7fbe14002870 2024-01-24T23:25:18.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbe140065d0 con 0x7fbe14002870 2024-01-24T23:25:18.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbe300018f0 con 0x7fbe14002870 2024-01-24T23:25:18.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:18.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient: dump: 2024-01-24T23:25:18.889 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.889 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.889 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.889 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.889 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.889 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.889 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.889 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.889 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.889 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.889 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.889 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.890 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.890 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.890 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.890 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1504161661 0 0) 0x7fbe140065d0 con 0x7fbe14002870 2024-01-24T23:25:18.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 --> v1:172.21.15.115:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe1400a120 con 0x7fbe14002870 2024-01-24T23:25:18.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 <== mon.4 v1:172.21.15.115:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fbe30003600 con 0x7fbe14002870 2024-01-24T23:25:18.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:18.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:18.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 >> v1:172.21.15.115:6791/0 conn(0x7fbe14002870 legacy=0x7fbe14004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.866+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 >> v1:172.21.15.115:6791/0 conn(0x7fbe14002870 legacy=0x7fbe14004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe3b633640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:25:18.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe3b633640 1 -- 172.21.15.115:0/485929898 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbe14009df0 con 0x7fbe140090b0 2024-01-24T23:25:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4215513971 0 0) 0x7fbe14009df0 con 0x7fbe140090b0 2024-01-24T23:25:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe29ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe29ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe29ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe29ffb640 10 monclient(hunting): my global_id is 94270 2024-01-24T23:25:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe140065d0 con 0x7fbe140090b0 2024-01-24T23:25:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 661390620 0 0) 0x7fbe140065d0 con 0x7fbe140090b0 2024-01-24T23:25:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbe14009df0 con 0x7fbe140090b0 2024-01-24T23:25:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbe30003d20 con 0x7fbe140090b0 2024-01-24T23:25:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe29ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe29ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe29ffb640 10 monclient: dump: 2024-01-24T23:25:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:18.893 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:18.894 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 853523826 0 0) 0x7fbe14009df0 con 0x7fbe140090b0 2024-01-24T23:25:18.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 --> v1:172.21.15.115:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbe1400c310 con 0x7fbe140090b0 2024-01-24T23:25:18.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 <== mon.4 v1:172.21.15.115:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 1782377552) 0x7fbe30006050 con 0x7fbe140090b0 2024-01-24T23:25:18.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe29ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:25:18.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe29ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:18.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 >> v1:172.21.15.115:6791/0 conn(0x7fbe140090b0 legacy=0x7fbe14009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe29ffb640 1 -- 172.21.15.115:0/485929898 >> v1:172.21.15.115:6791/0 conn(0x7fbe140090b0 legacy=0x7fbe14009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:18.895 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:25:18.895 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:18.895 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:18.895 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:25:18.895 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 24, 2024-01-24T23:25:18.895 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:25:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:25:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:25:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:25:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:25:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:25:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:25:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:25:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:25:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:25:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:18.896 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 3, 2024-01-24T23:25:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:25:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:25:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:18.897 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:18.898 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:18.898 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:25:18.898 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:25:18.898 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:18.898 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:18.898 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:18.898 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:18.898 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:18.898 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:18.898 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:18.898 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:25:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:25:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:25:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:25:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:25:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:25:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:18.899 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:25:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:25:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:25:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:25:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:25:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:25:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:25:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:25:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:18.900 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:18.901 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:18.901 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:18.901 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:18.901 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:18.901 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:18.901 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:18.901 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:25:18.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.901 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:25:18.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.901 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:25:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:25:18.902 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:25:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:25:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.903 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.904 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.904 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:25:18.904 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:25:18.904 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.904 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.904 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.904 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:18.905 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:18.905 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.905 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.905 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.905 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:25:18.906 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.906 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.906 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.906 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:25:18.906 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:25:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.907 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:25:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:25:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:25:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.908 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:25:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:25:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:25:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.909 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:25:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:25:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:25:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:25:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.911 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:25:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:25:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:25:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:25:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:25:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:25:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:25:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:25:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:25:18.914 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:25:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:25:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:18.915 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:25:18.916 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:25:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe3b633640 1 -- 172.21.15.115:0/485929898 >> v1:172.21.15.132:6812/1835401922 conn(0x7fbe18037140 legacy=0x7fbe18039600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe3b633640 10 monclient: shutdown 2024-01-24T23:25:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe3b633640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe3b633640 1 -- 172.21.15.115:0/485929898 >> v1:172.21.15.115:6793/0 conn(0x7fbe34134740 legacy=0x7fbe341ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe39bd0640 1 -- 172.21.15.115:0/485929898 reap_dead start 2024-01-24T23:25:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe39bd0640 1 -- 172.21.15.115:0/485929898 reap_dead start 2024-01-24T23:25:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe3b633640 1 -- 172.21.15.115:0/485929898 shutdown_connections 2024-01-24T23:25:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.874+0000 7fbe3b633640 1 -- 172.21.15.115:0/485929898 >> 172.21.15.115:0/485929898 conn(0x7fbe340a5730 msgr2=0x7fbe3406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:18.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.878+0000 7fbe3b633640 1 -- 172.21.15.115:0/485929898 shutdown_connections 2024-01-24T23:25:18.918 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:18.878+0000 7fbe3b633640 1 -- 172.21.15.115:0/485929898 wait complete. 2024-01-24T23:25:18.918 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:25:18.918 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-01-24T23:25:18.918 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-01-24T23:25:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.030+0000 7f7e5d3ad640 10 monclient: get_monmap_and_config 2024-01-24T23:25:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.030+0000 7f7e5d3ad640 10 monclient: build_initial_monmap 2024-01-24T23:25:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.030+0000 7f7e5d3ad640 10 monclient: monmap: 2024-01-24T23:25:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.035 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:19.034800+0000 2024-01-24T23:25:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:19.034800+0000 2024-01-24T23:25:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:19.036 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:19.037 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:19.037 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:19.037 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:19.037 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:19.037 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:19.037 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:19.038 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 1 Processor -- start 2024-01-24T23:25:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 1 -- start start 2024-01-24T23:25:19.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 10 monclient: init 2024-01-24T23:25:19.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:19.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 10 monclient: _add_conns ranks=[6,4,0,8,3,2,5,7,1] 2024-01-24T23:25:19.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 10 monclient(hunting): picked mon.noname-d con 0x7f7e58133bf0 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 10 monclient(hunting): picked mon.noname-c con 0x7f7e58133fd0 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 10 monclient(hunting): picked mon.noname-a con 0x7f7e581343b0 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e5804ed70 con 0x7f7e581343b0 2024-01-24T23:25:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e580a4b60 con 0x7f7e58133fd0 2024-01-24T23:25:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e5809ae50 con 0x7f7e58133bf0 2024-01-24T23:25:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 10 monclient(hunting): authenticate will time out at 1197.957373s 2024-01-24T23:25:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e567fc640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f7e58133fd0 0x7f7e5813bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:44712/0 (socket says 172.21.15.115:44712) 2024-01-24T23:25:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e567fc640 1 -- 172.21.15.115:0/3176265179 learned_addr learned my addr 172.21.15.115:0/3176265179 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e56ffd640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f7e58133bf0 0x7f7e5809bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:40526/0 (socket says 172.21.15.115:40526) 2024-01-24T23:25:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e577fe640 1 --1- 172.21.15.115:0/3176265179 >> v1:172.21.15.115:6789/0 conn(0x7f7e581343b0 0x7f7e5813f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44240/0 (socket says 172.21.15.115:44240) 2024-01-24T23:25:19.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 1 -- 172.21.15.115:0/3176265179 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4179554642 0 0) 0x7f7e5809ae50 con 0x7f7e58133bf0 2024-01-24T23:25:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 10 monclient(hunting): my global_id is 6090 2024-01-24T23:25:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 1 -- 172.21.15.115:0/3176265179 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e40003530 con 0x7f7e58133bf0 2024-01-24T23:25:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 1 -- 172.21.15.115:0/3176265179 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2884771997 0 0) 0x7f7e5804ed70 con 0x7f7e581343b0 2024-01-24T23:25:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 10 monclient(hunting): my global_id is 94275 2024-01-24T23:25:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 1 -- 172.21.15.115:0/3176265179 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e5809ae50 con 0x7f7e581343b0 2024-01-24T23:25:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 1 -- 172.21.15.115:0/3176265179 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972224277 0 0) 0x7f7e580a4b60 con 0x7f7e58133fd0 2024-01-24T23:25:19.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 10 monclient(hunting): my global_id is 94279 2024-01-24T23:25:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 1 -- 172.21.15.115:0/3176265179 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e5804ed70 con 0x7f7e58133fd0 2024-01-24T23:25:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 1 -- 172.21.15.115:0/3176265179 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 239197377 0 0) 0x7f7e40003530 con 0x7f7e58133bf0 2024-01-24T23:25:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 1 -- 172.21.15.115:0/3176265179 >> v1:172.21.15.115:6791/0 conn(0x7f7e58133fd0 legacy=0x7f7e5813bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 1 -- 172.21.15.115:0/3176265179 >> v1:172.21.15.115:6789/0 conn(0x7f7e581343b0 legacy=0x7f7e5813f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:25:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 1 monclient: found mon.noname-d 2024-01-24T23:25:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:25:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 1 -- 172.21.15.115:0/3176265179 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5809b130 con 0x7f7e58133bf0 2024-01-24T23:25:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 10 monclient: _finish_auth 0 2024-01-24T23:25:19.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:25:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 1 -- 172.21.15.115:0/3176265179 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e480017c0 con 0x7f7e58133bf0 2024-01-24T23:25:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:25:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 10 monclient: dump: 2024-01-24T23:25:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.054 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.055 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.056 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.056 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.056 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.056 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.056 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.056 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 5 monclient: authenticate success, global_id 6090 2024-01-24T23:25:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 1 -- 172.21.15.115:0/3176265179 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7e48001aa0 con 0x7f7e58133bf0 2024-01-24T23:25:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 1 -- 172.21.15.115:0/3176265179 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e48001cd0 con 0x7f7e58133bf0 2024-01-24T23:25:19.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 4 set_mon_vals no callback set 2024-01-24T23:25:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e55ffb640 10 monclient: dump: 2024-01-24T23:25:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.058 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.059 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 10 monclient: shutdown 2024-01-24T23:25:19.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 1 -- 172.21.15.115:0/3176265179 >> v1:172.21.15.115:6792/0 conn(0x7f7e58133bf0 legacy=0x7f7e5809bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 1 -- 172.21.15.115:0/3176265179 shutdown_connections 2024-01-24T23:25:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 1 -- 172.21.15.115:0/3176265179 >> 172.21.15.115:0/3176265179 conn(0x7f7e580662f0 msgr2=0x7f7e58066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 1 -- 172.21.15.115:0/3176265179 shutdown_connections 2024-01-24T23:25:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 1 -- 172.21.15.115:0/3176265179 wait complete. 2024-01-24T23:25:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 10 monclient: build_initial_monmap 2024-01-24T23:25:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 10 monclient: monmap: 2024-01-24T23:25:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:19.060 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:19.061 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.034+0000 7f7e5d3ad640 1 Processor -- start 2024-01-24T23:25:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e5d3ad640 1 -- start start 2024-01-24T23:25:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e5d3ad640 10 monclient: init 2024-01-24T23:25:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e5d3ad640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e5d3ad640 10 monclient: _add_conns ranks=[1,0,4,5,2,8,7,6,3] 2024-01-24T23:25:19.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e5d3ad640 10 monclient(hunting): picked mon.noname-f con 0x7f7e58133bf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e5d3ad640 10 monclient(hunting): picked mon.noname-a con 0x7f7e58133fd0 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e5d3ad640 10 monclient(hunting): picked mon.noname-c con 0x7f7e581343b0 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e5d3ad640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e581c8740 con 0x7f7e58133fd0 2024-01-24T23:25:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e5d3ad640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e580a5990 con 0x7f7e58133bf0 2024-01-24T23:25:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e5d3ad640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e580a2590 con 0x7f7e581343b0 2024-01-24T23:25:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e5d3ad640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e5d3ad640 10 monclient(hunting): authenticate will time out at 1197.960491s 2024-01-24T23:25:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e577fe640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f7e581343b0 0x7f7e581d3cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:44726/0 (socket says 172.21.15.115:44726) 2024-01-24T23:25:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e567fc640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f7e58133fd0 0x7f7e581d05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44252/0 (socket says 172.21.15.115:44252) 2024-01-24T23:25:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e577fe640 1 -- 172.21.15.115:0/3481242913 learned_addr learned my addr 172.21.15.115:0/3481242913 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2448696367 0 0) 0x7f7e581c8740 con 0x7f7e58133fd0 2024-01-24T23:25:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.062 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient(hunting): my global_id is 94284 2024-01-24T23:25:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e44003530 con 0x7f7e58133fd0 2024-01-24T23:25:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2369046267 0 0) 0x7f7e580a2590 con 0x7f7e581343b0 2024-01-24T23:25:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient(hunting): my global_id is 94288 2024-01-24T23:25:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e581c8740 con 0x7f7e581343b0 2024-01-24T23:25:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1420998286 0 0) 0x7f7e44003530 con 0x7f7e58133fd0 2024-01-24T23:25:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7e580a2590 con 0x7f7e58133fd0 2024-01-24T23:25:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e4c0012e0 con 0x7f7e58133fd0 2024-01-24T23:25:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:25:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient(hunting): dump: 2024-01-24T23:25:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.063 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.064 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.064 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.064 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.064 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.064 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.064 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.064 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.064 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.064 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.064 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.064 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.064 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.065 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.065 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:19.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3744620939 0 0) 0x7f7e581c8740 con 0x7f7e581343b0 2024-01-24T23:25:19.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7e44003530 con 0x7f7e581343b0 2024-01-24T23:25:19.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e50001ce0 con 0x7f7e581343b0 2024-01-24T23:25:19.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:19.065 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient(hunting): dump: 2024-01-24T23:25:19.065 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.065 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.065 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.065 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.065 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.065 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.066 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4025954657 0 0) 0x7f7e580a5990 con 0x7f7e58133bf0 2024-01-24T23:25:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient(hunting): my global_id is 94186 2024-01-24T23:25:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e581c8740 con 0x7f7e58133bf0 2024-01-24T23:25:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3250529701 0 0) 0x7f7e580a2590 con 0x7f7e58133fd0 2024-01-24T23:25:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 >> v1:172.21.15.115:6791/0 conn(0x7f7e581343b0 legacy=0x7f7e581d3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 >> v1:172.21.15.132:6789/0 conn(0x7f7e58133bf0 legacy=0x7f7e58132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:25:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 monclient: found mon.a 2024-01-24T23:25:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e581208f0 con 0x7f7e58133fd0 2024-01-24T23:25:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient: _finish_auth 0 2024-01-24T23:25:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:25:19.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:19.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e5d3ad640 5 monclient: authenticate success, global_id 94284 2024-01-24T23:25:19.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e5d3ad640 10 monclient: _renew_subs 2024-01-24T23:25:19.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e5d3ad640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:19.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e5d3ad640 1 -- 172.21.15.115:0/3481242913 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e58133620 con 0x7f7e58133fd0 2024-01-24T23:25:19.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e5d3ad640 10 monclient: _renew_subs 2024-01-24T23:25:19.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e5d3ad640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:19.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e5d3ad640 1 -- 172.21.15.115:0/3481242913 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e580a34e0 con 0x7f7e58133fd0 2024-01-24T23:25:19.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7e4c0016d0 con 0x7f7e58133fd0 2024-01-24T23:25:19.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:19.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e4c004de0 con 0x7f7e58133fd0 2024-01-24T23:25:19.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e557fa640 4 set_mon_vals no callback set 2024-01-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient: dump: 2024-01-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.069 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.070 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.070 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.070 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.070 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e5d3ad640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:19.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e5d3ad640 1 -- 172.21.15.115:0/3481242913 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7e380065d0 con 0x7f7e38002870 2024-01-24T23:25:19.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f7e4c00f860 con 0x7f7e58133fd0 2024-01-24T23:25:19.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(128..128 src has 1..128) v4 ==== 6989+0+0 (unknown 538550023 0 0) 0x7f7e4c044200 con 0x7f7e58133fd0 2024-01-24T23:25:19.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1739233285 0 0) 0x7f7e380065d0 con 0x7f7e38002870 2024-01-24T23:25:19.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient(hunting): my global_id is 6099 2024-01-24T23:25:19.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e580a2590 con 0x7f7e38002870 2024-01-24T23:25:19.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4167619260 0 0) 0x7f7e580a2590 con 0x7f7e38002870 2024-01-24T23:25:19.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7e380065d0 con 0x7f7e38002870 2024-01-24T23:25:19.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e577fe640 10 monclient: get_auth_request con 0x7f7e44037620 auth_method 0 2024-01-24T23:25:19.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e48001780 con 0x7f7e38002870 2024-01-24T23:25:19.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:19.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient: dump: 2024-01-24T23:25:19.071 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.071 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.071 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.071 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.071 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.071 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.071 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.071 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.072 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.072 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.072 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.072 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.072 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.072 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.072 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.072 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3490129518 0 0) 0x7f7e380065d0 con 0x7f7e38002870 2024-01-24T23:25:19.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 --> v1:172.21.15.115:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7e3800a120 con 0x7f7e38002870 2024-01-24T23:25:19.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f7e48004bd0 con 0x7f7e38002870 2024-01-24T23:25:19.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:19.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:19.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 >> v1:172.21.15.115:6792/0 conn(0x7f7e38002870 legacy=0x7f7e38004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.038+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 >> v1:172.21.15.115:6792/0 conn(0x7f7e38002870 legacy=0x7f7e38004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:19.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e5d3ad640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:25:19.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e5d3ad640 1 -- 172.21.15.115:0/3481242913 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7e38009e10 con 0x7f7e380090d0 2024-01-24T23:25:19.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3827222127 0 0) 0x7f7e38009e10 con 0x7f7e380090d0 2024-01-24T23:25:19.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e3f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e3f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e3f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e3f7fe640 10 monclient(hunting): my global_id is 6108 2024-01-24T23:25:19.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e380065d0 con 0x7f7e380090d0 2024-01-24T23:25:19.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1300824172 0 0) 0x7f7e380065d0 con 0x7f7e380090d0 2024-01-24T23:25:19.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7e38009e10 con 0x7f7e380090d0 2024-01-24T23:25:19.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e480052f0 con 0x7f7e380090d0 2024-01-24T23:25:19.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e3f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e3f7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:19.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e3f7fe640 10 monclient: dump: 2024-01-24T23:25:19.074 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.074 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.074 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.074 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.074 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.074 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.074 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2952670749 0 0) 0x7f7e38009e10 con 0x7f7e380090d0 2024-01-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 --> v1:172.21.15.115:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7e3800c310 con 0x7f7e380090d0 2024-01-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 3480560497) 0x7f7e480076b0 con 0x7f7e380090d0 2024-01-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e3f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e3f7fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:19.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 >> v1:172.21.15.115:6792/0 conn(0x7f7e380090d0 legacy=0x7f7e38009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e3f7fe640 1 -- 172.21.15.115:0/3481242913 >> v1:172.21.15.115:6792/0 conn(0x7f7e380090d0 legacy=0x7f7e38009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:19.076 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:25:19.076 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:19.076 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:19.076 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:25:19.076 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 24, 2024-01-24T23:25:19.076 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:25:19.076 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:25:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:25:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:25:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:25:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:25:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:25:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:25:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:25:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:25:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 4, 2024-01-24T23:25:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:25:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:25:19.077 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:25:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:25:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:19.078 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:19.079 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:19.079 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:19.079 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:19.079 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:19.079 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:19.079 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:19.079 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:19.079 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:19.080 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:19.080 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.080 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:25:19.080 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:25:19.080 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:25:19.080 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:25:19.081 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:25:19.081 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:25:19.081 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:19.081 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:19.081 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:25:19.081 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:25:19.081 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:25:19.081 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:25:19.082 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:25:19.082 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:25:19.082 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:25:19.082 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:19.082 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:25:19.082 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:19.082 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:19.082 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:19.082 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:25:19.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.084 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:25:19.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.084 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:25:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:25:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:25:19.085 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.086 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.086 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.086 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:19.086 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:19.086 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.086 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.086 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.086 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:25:19.086 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.086 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.086 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.087 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:25:19.087 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:25:19.087 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.087 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.087 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.087 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:19.087 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:19.087 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.087 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.087 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.087 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:25:19.088 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.088 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.088 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.088 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:25:19.088 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:25:19.088 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.088 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.088 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.088 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:19.088 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:19.088 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.089 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.089 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.089 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:25:19.089 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.089 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.089 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.089 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:25:19.089 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:25:19.089 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.089 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.089 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.090 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:19.090 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:19.090 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.090 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.090 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.090 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:25:19.090 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.090 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.090 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.090 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:25:19.090 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:25:19.090 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.090 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.091 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.091 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:19.091 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:19.091 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.091 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.091 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.091 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:25:19.091 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.091 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.091 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.091 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:25:19.092 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:25:19.092 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.092 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.092 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.092 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:19.092 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:19.092 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.092 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.092 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.092 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:25:19.092 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.093 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.093 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.093 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:25:19.093 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:25:19.093 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.093 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.093 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.093 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:19.093 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:19.093 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.093 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:25:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:25:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:25:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:25:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:25:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:25:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:25:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:25:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:25:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:25:19.097 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.097 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.097 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:25:19.097 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:25:19.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e5d3ad640 1 -- 172.21.15.115:0/3481242913 >> v1:172.21.15.132:6812/1835401922 conn(0x7f7e44037620 legacy=0x7f7e44039ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e5d3ad640 10 monclient: shutdown 2024-01-24T23:25:19.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e5d3ad640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:19.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e5d3ad640 1 -- 172.21.15.115:0/3481242913 >> v1:172.21.15.115:6789/0 conn(0x7f7e58133fd0 legacy=0x7f7e581d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e577fe640 1 -- 172.21.15.115:0/3481242913 reap_dead start 2024-01-24T23:25:19.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e577fe640 1 -- 172.21.15.115:0/3481242913 reap_dead start 2024-01-24T23:25:19.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e5d3ad640 1 -- 172.21.15.115:0/3481242913 shutdown_connections 2024-01-24T23:25:19.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e5d3ad640 1 -- 172.21.15.115:0/3481242913 >> 172.21.15.115:0/3481242913 conn(0x7f7e580662f0 msgr2=0x7f7e5809fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:19.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e5d3ad640 1 -- 172.21.15.115:0/3481242913 shutdown_connections 2024-01-24T23:25:19.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.046+0000 7f7e5d3ad640 1 -- 172.21.15.115:0/3481242913 wait complete. 2024-01-24T23:25:19.099 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:25:19.099 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-01-24T23:25:19.099 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-01-24T23:25:19.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:19.137+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 <== mon.1 v1:172.21.15.132:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set rm", "pool": "barset"}]=0 v129) v1 ==== 89+0+0 (unknown 3847996353 0 0) 0x7fdf4c014990 con 0x7fdf54139fb0 2024-01-24T23:25:19.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:19.137+0000 7fdf3bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-01-24T23:25:19.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:19.137+0000 7fdf3bfff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:19.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.218+0000 7f4e35fa9640 10 monclient: get_monmap_and_config 2024-01-24T23:25:19.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.218+0000 7f4e35fa9640 10 monclient: build_initial_monmap 2024-01-24T23:25:19.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.218+0000 7f4e35fa9640 10 monclient: monmap: 2024-01-24T23:25:19.221 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:19.221 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.221 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:19.221497+0000 2024-01-24T23:25:19.221 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:19.221497+0000 2024-01-24T23:25:19.221 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:19.221 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.221 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:19.221 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:19.221 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:19.221 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:19.222 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:19.222 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:19.222 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:19.222 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:19.222 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:19.222 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.218+0000 7f4e35fa9640 1 Processor -- start 2024-01-24T23:25:19.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.218+0000 7f4e35fa9640 1 -- start start 2024-01-24T23:25:19.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.218+0000 7f4e35fa9640 10 monclient: init 2024-01-24T23:25:19.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.218+0000 7f4e35fa9640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:19.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.218+0000 7f4e35fa9640 10 monclient: _add_conns ranks=[8,6,3,5,1,2,7,4,0] 2024-01-24T23:25:19.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.218+0000 7f4e35fa9640 10 monclient(hunting): picked mon.noname-e con 0x7f4e30134740 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:19.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.218+0000 7f4e35fa9640 10 monclient(hunting): picked mon.noname-d con 0x7f4e3009ec80 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:19.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.218+0000 7f4e35fa9640 10 monclient(hunting): picked mon.noname-g con 0x7f4e30135230 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:19.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.218+0000 7f4e35fa9640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e3004ed70 con 0x7f4e30135230 2024-01-24T23:25:19.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e300a0d90 con 0x7f4e3009ec80 2024-01-24T23:25:19.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e3009e7c0 con 0x7f4e30134740 2024-01-24T23:25:19.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:19.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 10 monclient(hunting): authenticate will time out at 1198.144043s 2024-01-24T23:25:19.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2f7fe640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f4e30134740 0x7f4e30134b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:59550/0 (socket says 172.21.15.115:59550) 2024-01-24T23:25:19.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e27fff640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f4e3009ec80 0x7f4e30139d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:40546/0 (socket says 172.21.15.115:40546) 2024-01-24T23:25:19.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2f7fe640 1 -- 172.21.15.115:0/490248349 learned_addr learned my addr 172.21.15.115:0/490248349 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:19.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 1 -- 172.21.15.115:0/490248349 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3540388109 0 0) 0x7f4e3009e7c0 con 0x7f4e30134740 2024-01-24T23:25:19.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 10 monclient(hunting): my global_id is 85157 2024-01-24T23:25:19.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 1 -- 172.21.15.115:0/490248349 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e1c003530 con 0x7f4e30134740 2024-01-24T23:25:19.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 1 -- 172.21.15.115:0/490248349 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2611316912 0 0) 0x7f4e300a0d90 con 0x7f4e3009ec80 2024-01-24T23:25:19.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 10 monclient(hunting): my global_id is 6117 2024-01-24T23:25:19.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 1 -- 172.21.15.115:0/490248349 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e3009e7c0 con 0x7f4e3009ec80 2024-01-24T23:25:19.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 1 -- 172.21.15.115:0/490248349 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3805973617 0 0) 0x7f4e3009e7c0 con 0x7f4e3009ec80 2024-01-24T23:25:19.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 1 -- 172.21.15.115:0/490248349 >> v1:172.21.15.115:6793/0 conn(0x7f4e30134740 legacy=0x7f4e30134b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 1 -- 172.21.15.115:0/490248349 >> v1:172.21.15.132:6790/0 conn(0x7f4e30135230 legacy=0x7f4e3013d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 10 monclient: _finish_hunting 0 2024-01-24T23:25:19.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 1 monclient: found mon.noname-d 2024-01-24T23:25:19.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:19.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:25:19.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 1 -- 172.21.15.115:0/490248349 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e300a1070 con 0x7f4e3009ec80 2024-01-24T23:25:19.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 10 monclient: _finish_auth 0 2024-01-24T23:25:19.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:19.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:19.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 1 -- 172.21.15.115:0/490248349 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4e180017a0 con 0x7f4e3009ec80 2024-01-24T23:25:19.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 5 monclient: authenticate success, global_id 6117 2024-01-24T23:25:19.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:19.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:25:19.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 10 monclient: dump: 2024-01-24T23:25:19.235 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.235 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.235 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.235 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.235 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.235 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.235 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.235 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.235 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.235 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.236 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.236 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.236 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.236 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.236 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.236 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:19.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 1 -- 172.21.15.115:0/490248349 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4e18001a40 con 0x7f4e3009ec80 2024-01-24T23:25:19.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:19.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:19.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 4 set_mon_vals no callback set 2024-01-24T23:25:19.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 1 -- 172.21.15.115:0/490248349 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4e18004b30 con 0x7f4e3009ec80 2024-01-24T23:25:19.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:19.237 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2effd640 10 monclient: dump: 2024-01-24T23:25:19.237 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.237 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.238 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.238 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.238 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.238 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.238 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.238 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.238 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.238 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.238 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.238 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.239 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.239 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.239 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.239 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 10 monclient: shutdown 2024-01-24T23:25:19.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:19.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 1 -- 172.21.15.115:0/490248349 >> v1:172.21.15.115:6792/0 conn(0x7f4e3009ec80 legacy=0x7f4e30139d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 1 -- 172.21.15.115:0/490248349 shutdown_connections 2024-01-24T23:25:19.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 1 -- 172.21.15.115:0/490248349 >> 172.21.15.115:0/490248349 conn(0x7f4e300a5730 msgr2=0x7f4e300a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:19.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 1 -- 172.21.15.115:0/490248349 shutdown_connections 2024-01-24T23:25:19.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 1 -- 172.21.15.115:0/490248349 wait complete. 2024-01-24T23:25:19.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 10 monclient: build_initial_monmap 2024-01-24T23:25:19.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 10 monclient: monmap: 2024-01-24T23:25:19.239 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:19.240 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:19.240 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:19.240 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:19.240 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:19.240 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:19.240 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:19.241 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:19.241 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:19.241 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:19.241 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:19.242 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:19.242 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:19.242 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:19.242 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:19.242 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 1 Processor -- start 2024-01-24T23:25:19.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 1 -- start start 2024-01-24T23:25:19.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 10 monclient: init 2024-01-24T23:25:19.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:19.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 10 monclient: _add_conns ranks=[0,7,3,8,1,4,6,5,2] 2024-01-24T23:25:19.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 10 monclient(hunting): picked mon.noname-a con 0x7f4e3009ec80 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:19.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 10 monclient(hunting): picked mon.noname-i con 0x7f4e30134740 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:19.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 10 monclient(hunting): picked mon.noname-g con 0x7f4e30135230 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:19.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e3009dc50 con 0x7f4e3009ec80 2024-01-24T23:25:19.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e3013e980 con 0x7f4e30135230 2024-01-24T23:25:19.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e30071c00 con 0x7f4e30134740 2024-01-24T23:25:19.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:19.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e35fa9640 10 monclient(hunting): authenticate will time out at 1198.147138s 2024-01-24T23:25:19.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2f7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f4e3009ec80 0x7f4e3009c3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44264/0 (socket says 172.21.15.115:44264) 2024-01-24T23:25:19.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2f7fe640 1 -- 172.21.15.115:0/2327583806 learned_addr learned my addr 172.21.15.115:0/2327583806 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:19.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 157739998 0 0) 0x7f4e3009dc50 con 0x7f4e3009ec80 2024-01-24T23:25:19.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 10 monclient(hunting): my global_id is 94293 2024-01-24T23:25:19.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e10003530 con 0x7f4e3009ec80 2024-01-24T23:25:19.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4153421876 0 0) 0x7f4e10003530 con 0x7f4e3009ec80 2024-01-24T23:25:19.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4e3009dc50 con 0x7f4e3009ec80 2024-01-24T23:25:19.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4211629908 0 0) 0x7f4e3013e980 con 0x7f4e30135230 2024-01-24T23:25:19.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 10 monclient(hunting): my global_id is 5925 2024-01-24T23:25:19.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e10003530 con 0x7f4e30135230 2024-01-24T23:25:19.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4e20001560 con 0x7f4e3009ec80 2024-01-24T23:25:19.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:25:19.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 10 monclient(hunting): dump: 2024-01-24T23:25:19.245 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.245 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.245 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.245 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.245 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.245 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.245 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.245 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.245 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.246 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.246 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.246 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.246 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.246 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.246 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.246 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:19.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3897740612 0 0) 0x7f4e30071c00 con 0x7f4e30134740 2024-01-24T23:25:19.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 10 monclient(hunting): my global_id is 5902 2024-01-24T23:25:19.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e3013e980 con 0x7f4e30134740 2024-01-24T23:25:19.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4195838445 0 0) 0x7f4e3009dc50 con 0x7f4e3009ec80 2024-01-24T23:25:19.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.222+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 >> v1:172.21.15.132:6792/0 conn(0x7f4e30134740 legacy=0x7f4e3009cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 >> v1:172.21.15.132:6790/0 conn(0x7f4e30135230 legacy=0x7f4e301d3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 10 monclient: _finish_hunting 0 2024-01-24T23:25:19.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 1 monclient: found mon.a 2024-01-24T23:25:19.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:19.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:19.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3009d210 con 0x7f4e3009ec80 2024-01-24T23:25:19.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 10 monclient: _finish_auth 0 2024-01-24T23:25:19.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 10 monclient: _check_auth_tickets 2024-01-24T23:25:19.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:19.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e35fa9640 5 monclient: authenticate success, global_id 94293 2024-01-24T23:25:19.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e35fa9640 10 monclient: _renew_subs 2024-01-24T23:25:19.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e35fa9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:19.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e35fa9640 1 -- 172.21.15.115:0/2327583806 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e300716b0 con 0x7f4e3009ec80 2024-01-24T23:25:19.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e35fa9640 10 monclient: _renew_subs 2024-01-24T23:25:19.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e35fa9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:19.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e35fa9640 1 -- 172.21.15.115:0/2327583806 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e3013a410 con 0x7f4e3009ec80 2024-01-24T23:25:19.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4e20004aa0 con 0x7f4e3009ec80 2024-01-24T23:25:19.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:19.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4e20004f50 con 0x7f4e3009ec80 2024-01-24T23:25:19.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2e7fc640 4 set_mon_vals no callback set 2024-01-24T23:25:19.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:19.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 10 monclient: dump: 2024-01-24T23:25:19.249 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.249 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.249 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.249 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.249 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.249 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.249 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.249 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.249 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.249 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.249 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.249 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.249 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.250 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.250 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.250 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e35fa9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:19.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e35fa9640 1 -- 172.21.15.115:0/2327583806 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4e140065d0 con 0x7f4e14002870 2024-01-24T23:25:19.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f4e2000f9d0 con 0x7f4e3009ec80 2024-01-24T23:25:19.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(129..129 src has 1..129) v4 ==== 6377+0+0 (unknown 51197916 0 0) 0x7f4e200442c0 con 0x7f4e3009ec80 2024-01-24T23:25:19.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140964765 0 0) 0x7f4e140065d0 con 0x7f4e14002870 2024-01-24T23:25:19.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 10 monclient(hunting): my global_id is 85166 2024-01-24T23:25:19.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e3009dc50 con 0x7f4e14002870 2024-01-24T23:25:19.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2ffff640 10 monclient: get_auth_request con 0x7f4e100371b0 auth_method 0 2024-01-24T23:25:19.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 23721683 0 0) 0x7f4e3009dc50 con 0x7f4e14002870 2024-01-24T23:25:19.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4e140065d0 con 0x7f4e14002870 2024-01-24T23:25:19.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4e18004b30 con 0x7f4e14002870 2024-01-24T23:25:19.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:19.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 10 monclient: dump: 2024-01-24T23:25:19.251 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.251 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.251 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.251 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.251 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.251 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.251 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.251 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.251 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.252 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.252 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.252 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.252 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.252 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.252 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.252 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3830169143 0 0) 0x7f4e140065d0 con 0x7f4e14002870 2024-01-24T23:25:19.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 --> v1:172.21.15.115:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4e1400a120 con 0x7f4e14002870 2024-01-24T23:25:19.253 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f4e18001560 con 0x7f4e14002870 2024-01-24T23:25:19.253 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:19.253 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:19.253 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 >> v1:172.21.15.115:6793/0 conn(0x7f4e14002870 legacy=0x7f4e14004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.253 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.226+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 >> v1:172.21.15.115:6793/0 conn(0x7f4e14002870 legacy=0x7f4e14004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:19.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:19.241+0000 7fdf59d07640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-01-24T23:25:19.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:19.241+0000 7fdf59d07640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-01-24T23:25:19.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:19.241+0000 7fdf59d07640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:25:19.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:19.241+0000 7fdf59d07640 1 -- 172.21.15.132:0/935895895 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd crush weight-set rm", "pool": "barset"} v 0) v1 -- 0x7fdf540008e0 con 0x7fdf54139fb0 2024-01-24T23:25:19.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e35fa9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:25:19.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e35fa9640 1 -- 172.21.15.115:0/2327583806 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4e14009e10 con 0x7f4e140090d0 2024-01-24T23:25:19.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3298492828 0 0) 0x7f4e14009e10 con 0x7f4e140090d0 2024-01-24T23:25:19.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e2cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e2cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e2cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e2cff9640 10 monclient(hunting): my global_id is 85175 2024-01-24T23:25:19.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e140065d0 con 0x7f4e140090d0 2024-01-24T23:25:19.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 717387808 0 0) 0x7f4e140065d0 con 0x7f4e140090d0 2024-01-24T23:25:19.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4e14009e10 con 0x7f4e140090d0 2024-01-24T23:25:19.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4e18005530 con 0x7f4e140090d0 2024-01-24T23:25:19.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e2cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e2cff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:19.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e2cff9640 10 monclient: dump: 2024-01-24T23:25:19.260 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.260 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.261 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.261 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.261 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.261 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.261 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.262 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.262 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.262 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.262 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.263 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.263 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.263 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.263 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.263 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4074203321 0 0) 0x7f4e14009e10 con 0x7f4e140090d0 2024-01-24T23:25:19.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 --> v1:172.21.15.115:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4e1400c310 con 0x7f4e140090d0 2024-01-24T23:25:19.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 960966299) 0x7f4e18007790 con 0x7f4e140090d0 2024-01-24T23:25:19.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e2cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:25:19.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e2cff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:19.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 >> v1:172.21.15.115:6793/0 conn(0x7f4e140090d0 legacy=0x7f4e14009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e2cff9640 1 -- 172.21.15.115:0/2327583806 >> v1:172.21.15.115:6793/0 conn(0x7f4e140090d0 legacy=0x7f4e14009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:19.265 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:25:19.265 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:19.265 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:19.265 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:25:19.265 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 24, 2024-01-24T23:25:19.265 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:25:19.266 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:25:19.266 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:25:19.266 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:25:19.266 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:25:19.266 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:25:19.266 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:25:19.266 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:25:19.266 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:25:19.267 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:25:19.267 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.267 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 4, 2024-01-24T23:25:19.267 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:19.267 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:25:19.267 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:25:19.268 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:19.268 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:19.268 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:19.268 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:19.268 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:19.268 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:19.269 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:19.269 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:19.269 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:19.269 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:19.269 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:19.269 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.269 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:25:19.270 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:25:19.270 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:19.270 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:19.270 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:19.270 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:19.271 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:19.271 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:19.271 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:19.271 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:19.271 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:19.271 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:19.272 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:19.272 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.272 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:25:19.272 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:25:19.273 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:25:19.273 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:25:19.273 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:25:19.273 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:25:19.273 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:19.273 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:19.273 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:25:19.273 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:25:19.274 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:25:19.274 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:25:19.274 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:25:19.274 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:25:19.274 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:25:19.274 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:19.274 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:25:19.274 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:19.275 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:19.275 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:19.275 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:19.275 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:19.275 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:19.275 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:19.275 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:19.275 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:19.276 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:19.276 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:19.276 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.276 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:25:19.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.276 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:25:19.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.277 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:19.277 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:19.277 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.277 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.277 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.278 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:25:19.278 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.278 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.278 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.278 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:25:19.278 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:25:19.279 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.279 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.279 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.279 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:19.280 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:19.280 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.280 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.280 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.280 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:25:19.280 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.280 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.280 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.280 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:25:19.280 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:25:19.281 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.281 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.281 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.281 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:19.281 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:19.281 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.281 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.281 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.281 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:25:19.282 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.282 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.282 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.282 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:25:19.282 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:25:19.282 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.282 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.282 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.282 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:19.282 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:19.283 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.283 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.283 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.283 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:25:19.283 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.283 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.283 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.283 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:25:19.283 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:25:19.283 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.283 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.283 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.284 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:19.284 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:19.284 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.284 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.284 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.284 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:25:19.284 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.284 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.284 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.285 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:25:19.285 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:25:19.285 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.285 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.285 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.285 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:19.285 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:19.285 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.285 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.285 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.285 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:25:19.285 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.286 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.286 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.286 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:25:19.286 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:25:19.286 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.286 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.286 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.286 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:19.286 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:19.286 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.287 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.287 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.287 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:25:19.287 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.287 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.288 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.288 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:25:19.288 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:25:19.288 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.288 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.289 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.289 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:19.289 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:19.289 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.290 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.290 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.290 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:25:19.290 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.290 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.290 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.291 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:25:19.291 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:25:19.291 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.291 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.291 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.291 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:19.291 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:19.291 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.292 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.292 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.292 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:25:19.292 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.292 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.292 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.293 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:25:19.293 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:25:19.293 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.293 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.293 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.293 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.293 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.294 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:25:19.294 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:25:19.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.294 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:19.294 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:19.294 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:19.294 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.295 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.295 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:25:19.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.295 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:19.295 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:19.295 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:19.295 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.296 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:25:19.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.296 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:19.296 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:19.296 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:19.297 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.297 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.297 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:25:19.297 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:25:19.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e35fa9640 1 -- 172.21.15.115:0/2327583806 >> v1:172.21.15.132:6812/1835401922 conn(0x7f4e100371b0 legacy=0x7f4e10039670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e35fa9640 10 monclient: shutdown 2024-01-24T23:25:19.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e35fa9640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:19.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e35fa9640 1 -- 172.21.15.115:0/2327583806 >> v1:172.21.15.115:6789/0 conn(0x7f4e3009ec80 legacy=0x7f4e3009c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e2ffff640 1 -- 172.21.15.115:0/2327583806 reap_dead start 2024-01-24T23:25:19.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e2ffff640 1 -- 172.21.15.115:0/2327583806 reap_dead start 2024-01-24T23:25:19.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e35fa9640 1 -- 172.21.15.115:0/2327583806 shutdown_connections 2024-01-24T23:25:19.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e35fa9640 1 -- 172.21.15.115:0/2327583806 >> 172.21.15.115:0/2327583806 conn(0x7f4e300a5730 msgr2=0x7f4e3006ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:19.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e35fa9640 1 -- 172.21.15.115:0/2327583806 shutdown_connections 2024-01-24T23:25:19.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.234+0000 7f4e35fa9640 1 -- 172.21.15.115:0/2327583806 wait complete. 2024-01-24T23:25:19.301 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:25:19.302 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-01-24T23:25:19.302 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-01-24T23:25:19.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.422+0000 7f87b97c9640 10 monclient: get_monmap_and_config 2024-01-24T23:25:19.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.422+0000 7f87b97c9640 10 monclient: build_initial_monmap 2024-01-24T23:25:19.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.422+0000 7f87b97c9640 10 monclient: monmap: 2024-01-24T23:25:19.425 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:19.425 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.426 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:19.425502+0000 2024-01-24T23:25:19.426 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:19.425502+0000 2024-01-24T23:25:19.426 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:19.426 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.426 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:19.426 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:19.426 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:19.426 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:19.426 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:19.426 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:19.427 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:19.427 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:19.427 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:19.427 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.422+0000 7f87b97c9640 1 Processor -- start 2024-01-24T23:25:19.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.422+0000 7f87b97c9640 1 -- start start 2024-01-24T23:25:19.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.422+0000 7f87b97c9640 10 monclient: init 2024-01-24T23:25:19.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.422+0000 7f87b97c9640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:19.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.422+0000 7f87b97c9640 10 monclient: _add_conns ranks=[1,8,7,6,2,3,4,5,0] 2024-01-24T23:25:19.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.422+0000 7f87b97c9640 10 monclient(hunting): picked mon.noname-f con 0x7f87b4133bf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:19.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 10 monclient(hunting): picked mon.noname-e con 0x7f87b4133fd0 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:19.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 10 monclient(hunting): picked mon.noname-i con 0x7f87b4134430 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:19.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b404ed70 con 0x7f87b4133bf0 2024-01-24T23:25:19.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b409af30 con 0x7f87b4134430 2024-01-24T23:25:19.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b409b210 con 0x7f87b4133fd0 2024-01-24T23:25:19.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:19.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 10 monclient(hunting): authenticate will time out at 1198.348075s 2024-01-24T23:25:19.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87abfff640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f87b4133fd0 0x7f87b413bdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:59568/0 (socket says 172.21.15.115:59568) 2024-01-24T23:25:19.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87abfff640 1 -- 172.21.15.115:0/3438023266 learned_addr learned my addr 172.21.15.115:0/3438023266 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:19.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 1 -- 172.21.15.115:0/3438023266 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 342364607 0 0) 0x7f87b409b210 con 0x7f87b4133fd0 2024-01-24T23:25:19.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 10 monclient(hunting): my global_id is 85184 2024-01-24T23:25:19.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 1 -- 172.21.15.115:0/3438023266 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87a0003530 con 0x7f87b4133fd0 2024-01-24T23:25:19.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 1 -- 172.21.15.115:0/3438023266 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2353742389 0 0) 0x7f87a0003530 con 0x7f87b4133fd0 2024-01-24T23:25:19.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 1 -- 172.21.15.115:0/3438023266 >> v1:172.21.15.132:6792/0 conn(0x7f87b4134430 legacy=0x7f87b413f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 1 -- 172.21.15.115:0/3438023266 >> v1:172.21.15.132:6789/0 conn(0x7f87b4133bf0 legacy=0x7f87b409bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 10 monclient: _finish_hunting 0 2024-01-24T23:25:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 1 monclient: found mon.noname-e 2024-01-24T23:25:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:25:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 1 -- 172.21.15.115:0/3438023266 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b409b4f0 con 0x7f87b4133fd0 2024-01-24T23:25:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 10 monclient: _finish_auth 0 2024-01-24T23:25:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 10 monclient: _check_auth_tickets 2024-01-24T23:25:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 1 -- 172.21.15.115:0/3438023266 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f879c0017c0 con 0x7f87b4133fd0 2024-01-24T23:25:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 5 monclient: authenticate success, global_id 85184 2024-01-24T23:25:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:25:19.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 10 monclient: dump: 2024-01-24T23:25:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.438 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.439 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 1 -- 172.21.15.115:0/3438023266 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f879c001a70 con 0x7f87b4133fd0 2024-01-24T23:25:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 1 -- 172.21.15.115:0/3438023266 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f879c004bf0 con 0x7f87b4133fd0 2024-01-24T23:25:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 4 set_mon_vals no callback set 2024-01-24T23:25:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b27fc640 10 monclient: dump: 2024-01-24T23:25:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.440 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.440 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.440 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.440 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.440 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.440 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.440 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.440 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.440 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.440 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.440 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.440 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 10 monclient: shutdown 2024-01-24T23:25:19.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 1 -- 172.21.15.115:0/3438023266 >> v1:172.21.15.115:6793/0 conn(0x7f87b4133fd0 legacy=0x7f87b413bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 1 -- 172.21.15.115:0/3438023266 shutdown_connections 2024-01-24T23:25:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 1 -- 172.21.15.115:0/3438023266 >> 172.21.15.115:0/3438023266 conn(0x7f87b40a9910 msgr2=0x7f87b40a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 1 -- 172.21.15.115:0/3438023266 shutdown_connections 2024-01-24T23:25:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 1 -- 172.21.15.115:0/3438023266 wait complete. 2024-01-24T23:25:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 10 monclient: build_initial_monmap 2024-01-24T23:25:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 10 monclient: monmap: 2024-01-24T23:25:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:19.441 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:19.442 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 1 Processor -- start 2024-01-24T23:25:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 1 -- start start 2024-01-24T23:25:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 10 monclient: init 2024-01-24T23:25:19.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:19.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 10 monclient: _add_conns ranks=[1,8,5,3,0,7,4,6,2] 2024-01-24T23:25:19.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 10 monclient(hunting): picked mon.noname-f con 0x7f87b4133bf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:19.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 10 monclient(hunting): picked mon.noname-e con 0x7f87b4133fd0 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:19.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 10 monclient(hunting): picked mon.noname-h con 0x7f87b4134430 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:19.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b4139900 con 0x7f87b4133bf0 2024-01-24T23:25:19.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b409c8f0 con 0x7f87b4134430 2024-01-24T23:25:19.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b413b020 con 0x7f87b4133fd0 2024-01-24T23:25:19.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:19.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87b97c9640 10 monclient(hunting): authenticate will time out at 1198.351653s 2024-01-24T23:25:19.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87abfff640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f87b4133fd0 0x7f87b41382b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:59572/0 (socket says 172.21.15.115:59572) 2024-01-24T23:25:19.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87abfff640 1 -- 172.21.15.115:0/1235034711 learned_addr learned my addr 172.21.15.115:0/1235034711 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:19.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3426896608 0 0) 0x7f87b413b020 con 0x7f87b4133fd0 2024-01-24T23:25:19.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87ab7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87ab7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87ab7fe640 10 monclient(hunting): my global_id is 85193 2024-01-24T23:25:19.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.426+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8794003530 con 0x7f87b4133fd0 2024-01-24T23:25:19.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2501391973 0 0) 0x7f8794003530 con 0x7f87b4133fd0 2024-01-24T23:25:19.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f87b413b020 con 0x7f87b4133fd0 2024-01-24T23:25:19.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f879c004b30 con 0x7f87b4133fd0 2024-01-24T23:25:19.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:25:19.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient(hunting): dump: 2024-01-24T23:25:19.444 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.444 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.444 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.445 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3543146974 0 0) 0x7f87b4139900 con 0x7f87b4133bf0 2024-01-24T23:25:19.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient(hunting): my global_id is 94204 2024-01-24T23:25:19.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8794003530 con 0x7f87b4133bf0 2024-01-24T23:25:19.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 314797854 0 0) 0x7f87b409c8f0 con 0x7f87b4134430 2024-01-24T23:25:19.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient(hunting): my global_id is 5900 2024-01-24T23:25:19.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87b4139900 con 0x7f87b4134430 2024-01-24T23:25:19.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1166557486 0 0) 0x7f87b413b020 con 0x7f87b4133fd0 2024-01-24T23:25:19.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 >> v1:172.21.15.132:6791/0 conn(0x7f87b4134430 legacy=0x7f87b41d3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 >> v1:172.21.15.132:6789/0 conn(0x7f87b4133bf0 legacy=0x7f87b4137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:25:19.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 monclient: found mon.e 2024-01-24T23:25:19.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:19.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:19.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b41389c0 con 0x7f87b4133fd0 2024-01-24T23:25:19.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient: _finish_auth 0 2024-01-24T23:25:19.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:25:19.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:19.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87b97c9640 5 monclient: authenticate success, global_id 85193 2024-01-24T23:25:19.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87b97c9640 10 monclient: _renew_subs 2024-01-24T23:25:19.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87b97c9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:19.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87b97c9640 1 -- 172.21.15.115:0/1235034711 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b40a5960 con 0x7f87b4133fd0 2024-01-24T23:25:19.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87b97c9640 10 monclient: _renew_subs 2024-01-24T23:25:19.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87b97c9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:19.448 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87b97c9640 1 -- 172.21.15.115:0/1235034711 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b4139580 con 0x7f87b4133fd0 2024-01-24T23:25:19.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f879c001200 con 0x7f87b4133fd0 2024-01-24T23:25:19.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:19.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f879c0051f0 con 0x7f87b4133fd0 2024-01-24T23:25:19.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87b1ffb640 4 set_mon_vals no callback set 2024-01-24T23:25:19.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:19.449 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient: dump: 2024-01-24T23:25:19.449 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.449 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.449 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.449 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.449 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.449 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.449 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.449 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.450 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87b97c9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87b97c9640 1 -- 172.21.15.115:0/1235034711 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f87980065d0 con 0x7f8798002870 2024-01-24T23:25:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f879c00fc70 con 0x7f87b4133fd0 2024-01-24T23:25:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(129..129 src has 1..129) v4 ==== 6377+0+0 (unknown 51197916 0 0) 0x7f879c0444d0 con 0x7f87b4133fd0 2024-01-24T23:25:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200548603 0 0) 0x7f87980065d0 con 0x7f8798002870 2024-01-24T23:25:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient(hunting): my global_id is 94213 2024-01-24T23:25:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87b413b020 con 0x7f8798002870 2024-01-24T23:25:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87b37fe640 10 monclient: get_auth_request con 0x7f87940371f0 auth_method 0 2024-01-24T23:25:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87b37fe640 1 -- 172.21.15.115:0/1235034711 >> v1:172.21.15.132:6812/1835401922 conn(0x7f87940371f0 legacy=0x7f87940396b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-01-24T23:25:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87b37fe640 1 -- 172.21.15.115:0/1235034711 >> v1:172.21.15.132:6812/1835401922 conn(0x7f87940371f0 legacy=0x7f87940396b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:25:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87b37fe640 1 --1- 172.21.15.115:0/1235034711 >> v1:172.21.15.132:6812/1835401922 conn(0x7f87940371f0 0x7f87940396b0 :-1 s=CONNECTING_SEND_CONNECT_MSG pgs=0 cs=0 l=1).handle_connect_reply_1 read connect reply failed 2024-01-24T23:25:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87b37fe640 1 --1- 172.21.15.115:0/1235034711 >> v1:172.21.15.132:6812/1835401922 conn(0x7f87940371f0 0x7f87940396b0 :-1 s=CONNECTING_SEND_CONNECT_MSG pgs=0 cs=0 l=1).fault on lossy channel, failing 2024-01-24T23:25:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 >> v1:172.21.15.132:6812/1835401922 conn(0x7f87940371f0 legacy=0x7f87940396b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3019626366 0 0) 0x7f87b413b020 con 0x7f8798002870 2024-01-24T23:25:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f87980065d0 con 0x7f8798002870 2024-01-24T23:25:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f87a4001930 con 0x7f8798002870 2024-01-24T23:25:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:25:19.452 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient: dump: 2024-01-24T23:25:19.452 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.452 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.452 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.452 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.452 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.452 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.452 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.452 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.452 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.452 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.452 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.453 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.453 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.453 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.453 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.453 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1227485188 0 0) 0x7f87980065d0 con 0x7f8798002870 2024-01-24T23:25:19.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 --> v1:172.21.15.132:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f879800a120 con 0x7f8798002870 2024-01-24T23:25:19.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 <== mon.1 v1:172.21.15.132:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f87a4003600 con 0x7f8798002870 2024-01-24T23:25:19.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:19.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:19.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 >> v1:172.21.15.132:6789/0 conn(0x7f8798002870 legacy=0x7f8798004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.430+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 >> v1:172.21.15.132:6789/0 conn(0x7f8798002870 legacy=0x7f8798004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:19.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.438+0000 7f87b97c9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:25:19.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.438+0000 7f87b97c9640 1 -- 172.21.15.115:0/1235034711 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8798009df0 con 0x7f87980090b0 2024-01-24T23:25:19.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.438+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609442469 0 0) 0x7f8798009df0 con 0x7f87980090b0 2024-01-24T23:25:19.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.438+0000 7f87ab7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.438+0000 7f87ab7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.438+0000 7f87ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.438+0000 7f87ab7fe640 10 monclient(hunting): my global_id is 94222 2024-01-24T23:25:19.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.438+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87980065d0 con 0x7f87980090b0 2024-01-24T23:25:19.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.438+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1413704579 0 0) 0x7f87980065d0 con 0x7f87980090b0 2024-01-24T23:25:19.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.438+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8798009df0 con 0x7f87980090b0 2024-01-24T23:25:19.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.438+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f87a4003ca0 con 0x7f87980090b0 2024-01-24T23:25:19.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.438+0000 7f87ab7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.438+0000 7f87ab7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:25:19.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.438+0000 7f87ab7fe640 10 monclient: dump: 2024-01-24T23:25:19.456 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.456 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.456 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.456 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.456 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.456 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.456 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.456 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.456 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.457 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.457 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.457 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.457 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.457 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.457 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.457 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.438+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1536082898 0 0) 0x7f8798009df0 con 0x7f87980090b0 2024-01-24T23:25:19.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.438+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 --> v1:172.21.15.132:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f879800c310 con 0x7f87980090b0 2024-01-24T23:25:19.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.442+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 <== mon.1 v1:172.21.15.132:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 2865244927) 0x7f87a40040f0 con 0x7f87980090b0 2024-01-24T23:25:19.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.442+0000 7f87ab7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:25:19.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.442+0000 7f87ab7fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:19.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.442+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 >> v1:172.21.15.132:6789/0 conn(0x7f87980090b0 legacy=0x7f8798009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.442+0000 7f87ab7fe640 1 -- 172.21.15.115:0/1235034711 >> v1:172.21.15.132:6789/0 conn(0x7f87980090b0 legacy=0x7f8798009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:19.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.442+0000 7f87b37fe640 1 -- 172.21.15.115:0/1235034711 reap_dead start 2024-01-24T23:25:19.458 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:25:19.458 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:19.458 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:19.458 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:25:19.458 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 24, 2024-01-24T23:25:19.458 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:25:19.458 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:25:19.458 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:25:19.458 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:25:19.459 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:25:19.459 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:25:19.459 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:25:19.459 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:25:19.459 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:25:19.459 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:25:19.459 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.459 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 4, 2024-01-24T23:25:19.459 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:19.459 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:25:19.459 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:25:19.459 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:19.460 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:19.460 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:19.460 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:19.460 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:19.460 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:19.460 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:19.460 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:19.460 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:19.460 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:19.461 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:19.461 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.461 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:25:19.461 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:25:19.461 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:19.461 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:19.461 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:19.462 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:19.462 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:19.462 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:19.462 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:19.462 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:19.462 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:19.463 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:19.463 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:19.463 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.463 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:25:19.463 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:25:19.464 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:25:19.464 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:25:19.464 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:25:19.464 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:25:19.464 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:19.464 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:19.465 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:25:19.465 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:25:19.465 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:25:19.465 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:25:19.465 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:25:19.465 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:25:19.466 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:25:19.466 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:19.466 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:25:19.466 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:19.466 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:19.466 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:19.466 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:19.467 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:19.467 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:19.467 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:19.467 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:19.467 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:19.468 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:19.468 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:19.468 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.468 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:25:19.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.468 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:25:19.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.469 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:19.469 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:19.469 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.469 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.470 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.470 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:25:19.470 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.470 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.470 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.470 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:25:19.470 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:25:19.471 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.471 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.471 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.471 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:19.471 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:19.472 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.472 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.472 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.472 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:25:19.472 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.473 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.473 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.473 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:25:19.473 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:25:19.474 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.474 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.474 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.474 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:19.475 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:19.475 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.475 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.475 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.475 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:25:19.476 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.476 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.476 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.476 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:25:19.476 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:25:19.477 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.477 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.477 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.477 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:19.478 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:19.478 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.478 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.478 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.478 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:25:19.479 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.479 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.479 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.479 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:25:19.479 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:25:19.480 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.480 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.480 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.480 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:19.481 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:19.481 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.481 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.481 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.481 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:25:19.482 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.482 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.482 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.482 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:25:19.482 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:25:19.483 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.483 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.483 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.483 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:19.483 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:19.484 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.484 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.484 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.484 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:25:19.484 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.484 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.484 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.485 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:25:19.485 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:25:19.485 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.485 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.485 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.486 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:19.486 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:19.486 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.486 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.486 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.486 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:25:19.486 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.487 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.487 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.487 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:25:19.487 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:25:19.487 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.487 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.488 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.488 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:19.488 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:19.488 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.488 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.488 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.488 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:25:19.488 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.488 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.488 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.489 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:25:19.489 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:25:19.489 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.489 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.489 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.489 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:19.489 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:19.489 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.489 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.489 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.490 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:25:19.490 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.490 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.490 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.490 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:25:19.490 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:25:19.490 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.490 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.490 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.490 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.490 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.490 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:25:19.491 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:25:19.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.491 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:19.491 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:19.491 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:19.491 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.491 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.491 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:25:19.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.491 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:19.491 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:19.491 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:25:19.491 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.491 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.492 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:25:19.492 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:25:19.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.442+0000 7f87b97c9640 10 monclient: shutdown 2024-01-24T23:25:19.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.442+0000 7f87b97c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:19.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.442+0000 7f87b97c9640 1 -- 172.21.15.115:0/1235034711 >> v1:172.21.15.115:6793/0 conn(0x7f87b4133fd0 legacy=0x7f87b41382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.442+0000 7f87b97c9640 1 -- 172.21.15.115:0/1235034711 shutdown_connections 2024-01-24T23:25:19.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.442+0000 7f87b97c9640 1 -- 172.21.15.115:0/1235034711 >> 172.21.15.115:0/1235034711 conn(0x7f87b40a9910 msgr2=0x7f87b413ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:19.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.442+0000 7f87b97c9640 1 -- 172.21.15.115:0/1235034711 shutdown_connections 2024-01-24T23:25:19.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.442+0000 7f87b97c9640 1 -- 172.21.15.115:0/1235034711 wait complete. 2024-01-24T23:25:19.494 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:25:19.494 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-01-24T23:25:19.494 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-01-24T23:25:19.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.606+0000 7f15177d8640 10 monclient: get_monmap_and_config 2024-01-24T23:25:19.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.606+0000 7f15177d8640 10 monclient: build_initial_monmap 2024-01-24T23:25:19.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.606+0000 7f15177d8640 10 monclient: monmap: 2024-01-24T23:25:19.610 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:19.610 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.610 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:19.610794+0000 2024-01-24T23:25:19.610 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:19.610794+0000 2024-01-24T23:25:19.610 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:19.610 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.611 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:19.611 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:19.611 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:19.611 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:19.611 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:19.611 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:19.611 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:19.611 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:19.611 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:19.611 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 1 Processor -- start 2024-01-24T23:25:19.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 1 -- start start 2024-01-24T23:25:19.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 10 monclient: init 2024-01-24T23:25:19.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:19.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 10 monclient: _add_conns ranks=[1,4,2,0,8,6,3,7,5] 2024-01-24T23:25:19.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 10 monclient(hunting): picked mon.noname-f con 0x7f151009ff10 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:19.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 10 monclient(hunting): picked mon.noname-c con 0x7f151006ae80 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:19.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 10 monclient(hunting): picked mon.noname-b con 0x7f15100a02f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:19.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f151004ed70 con 0x7f151009ff10 2024-01-24T23:25:19.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15100a3890 con 0x7f15100a02f0 2024-01-24T23:25:19.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f151006ac40 con 0x7f151006ae80 2024-01-24T23:25:19.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:19.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 10 monclient(hunting): authenticate will time out at 1198.533340s 2024-01-24T23:25:19.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1514d73640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f151006ae80 0x7f15100a3180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:44728/0 (socket says 172.21.15.115:44728) 2024-01-24T23:25:19.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1514d73640 1 -- 172.21.15.115:0/1477889450 learned_addr learned my addr 172.21.15.115:0/1477889450 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:19.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 1 -- 172.21.15.115:0/1477889450 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2837794163 0 0) 0x7f151006ac40 con 0x7f151006ae80 2024-01-24T23:25:19.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 10 monclient(hunting): my global_id is 94297 2024-01-24T23:25:19.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 1 -- 172.21.15.115:0/1477889450 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14fc003530 con 0x7f151006ae80 2024-01-24T23:25:19.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 1 -- 172.21.15.115:0/1477889450 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3609822563 0 0) 0x7f15100a3890 con 0x7f15100a02f0 2024-01-24T23:25:19.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 10 monclient(hunting): my global_id is 85331 2024-01-24T23:25:19.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 1 -- 172.21.15.115:0/1477889450 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f151006ac40 con 0x7f15100a02f0 2024-01-24T23:25:19.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 1 -- 172.21.15.115:0/1477889450 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3677095580 0 0) 0x7f14fc003530 con 0x7f151006ae80 2024-01-24T23:25:19.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 1 -- 172.21.15.115:0/1477889450 >> v1:172.21.15.115:6790/0 conn(0x7f15100a02f0 legacy=0x7f151013fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 1 -- 172.21.15.115:0/1477889450 >> v1:172.21.15.132:6789/0 conn(0x7f151009ff10 legacy=0x7f15100a2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 10 monclient: _finish_hunting 0 2024-01-24T23:25:19.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 1 monclient: found mon.noname-c 2024-01-24T23:25:19.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:19.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:25:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 1 -- 172.21.15.115:0/1477889450 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151009e9e0 con 0x7f151006ae80 2024-01-24T23:25:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 10 monclient: _finish_auth 0 2024-01-24T23:25:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 10 monclient: _check_auth_tickets 2024-01-24T23:25:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 1 -- 172.21.15.115:0/1477889450 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f15000017c0 con 0x7f151006ae80 2024-01-24T23:25:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:25:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 10 monclient: dump: 2024-01-24T23:25:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.620 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.621 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 5 monclient: authenticate success, global_id 94297 2024-01-24T23:25:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 1 -- 172.21.15.115:0/1477889450 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1500001a20 con 0x7f151006ae80 2024-01-24T23:25:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 1 -- 172.21.15.115:0/1477889450 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1500004bf0 con 0x7f151006ae80 2024-01-24T23:25:19.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f1507fff640 10 monclient: dump: 2024-01-24T23:25:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.622 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.623 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 4 set_mon_vals no callback set 2024-01-24T23:25:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 10 monclient: shutdown 2024-01-24T23:25:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 1 -- 172.21.15.115:0/1477889450 >> v1:172.21.15.115:6791/0 conn(0x7f151006ae80 legacy=0x7f15100a3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 1 -- 172.21.15.115:0/1477889450 shutdown_connections 2024-01-24T23:25:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 1 -- 172.21.15.115:0/1477889450 >> 172.21.15.115:0/1477889450 conn(0x7f15100a7820 msgr2=0x7f15100a3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 1 -- 172.21.15.115:0/1477889450 shutdown_connections 2024-01-24T23:25:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 1 -- 172.21.15.115:0/1477889450 wait complete. 2024-01-24T23:25:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 10 monclient: build_initial_monmap 2024-01-24T23:25:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 10 monclient: monmap: 2024-01-24T23:25:19.623 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:19.624 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:19.624 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 1 Processor -- start 2024-01-24T23:25:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 1 -- start start 2024-01-24T23:25:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.610+0000 7f15177d8640 10 monclient: init 2024-01-24T23:25:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f15177d8640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f15177d8640 10 monclient: _add_conns ranks=[3,5,8,0,7,1,6,2,4] 2024-01-24T23:25:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f15177d8640 10 monclient(hunting): picked mon.noname-g con 0x7f151006ae80 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f15177d8640 10 monclient(hunting): picked mon.noname-h con 0x7f151009ff10 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f15177d8640 10 monclient(hunting): picked mon.noname-e con 0x7f15100a02f0 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f15177d8640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f151006b5b0 con 0x7f151006ae80 2024-01-24T23:25:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f15177d8640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15101ac230 con 0x7f151009ff10 2024-01-24T23:25:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f15177d8640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15101ac510 con 0x7f15100a02f0 2024-01-24T23:25:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f15177d8640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f15177d8640 10 monclient(hunting): authenticate will time out at 1198.536273s 2024-01-24T23:25:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1515d75640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f15100a02f0 0x7f15101d3290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:59580/0 (socket says 172.21.15.115:59580) 2024-01-24T23:25:19.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1515d75640 1 -- 172.21.15.115:0/1887183894 learned_addr learned my addr 172.21.15.115:0/1887183894 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 892031816 0 0) 0x7f15101ac510 con 0x7f15100a02f0 2024-01-24T23:25:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient(hunting): my global_id is 85202 2024-01-24T23:25:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14f4003530 con 0x7f15100a02f0 2024-01-24T23:25:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2358179747 0 0) 0x7f14f4003530 con 0x7f15100a02f0 2024-01-24T23:25:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f15101ac510 con 0x7f15100a02f0 2024-01-24T23:25:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1508001d40 con 0x7f15100a02f0 2024-01-24T23:25:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:25:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient(hunting): dump: 2024-01-24T23:25:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.626 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.627 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:19.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2859402846 0 0) 0x7f151006b5b0 con 0x7f151006ae80 2024-01-24T23:25:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient(hunting): my global_id is 5934 2024-01-24T23:25:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14f4003530 con 0x7f151006ae80 2024-01-24T23:25:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3445684794 0 0) 0x7f15101ac230 con 0x7f151009ff10 2024-01-24T23:25:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient(hunting): my global_id is 5909 2024-01-24T23:25:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f151006b5b0 con 0x7f151009ff10 2024-01-24T23:25:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2077459747 0 0) 0x7f15101ac510 con 0x7f15100a02f0 2024-01-24T23:25:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 >> v1:172.21.15.132:6791/0 conn(0x7f151009ff10 legacy=0x7f15101beb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 >> v1:172.21.15.132:6790/0 conn(0x7f151006ae80 legacy=0x7f15101cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.628 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:25:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 monclient: found mon.e 2024-01-24T23:25:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15100720e0 con 0x7f15100a02f0 2024-01-24T23:25:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient: _finish_auth 0 2024-01-24T23:25:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:25:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f15177d8640 5 monclient: authenticate success, global_id 85202 2024-01-24T23:25:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f15177d8640 10 monclient: _renew_subs 2024-01-24T23:25:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f15177d8640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f15177d8640 1 -- 172.21.15.115:0/1887183894 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15101aa1b0 con 0x7f15100a02f0 2024-01-24T23:25:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f15177d8640 10 monclient: _renew_subs 2024-01-24T23:25:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f15177d8640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:19.629 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f15177d8640 1 -- 172.21.15.115:0/1887183894 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f15101bf430 con 0x7f15100a02f0 2024-01-24T23:25:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1508003cb0 con 0x7f15100a02f0 2024-01-24T23:25:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1508002110 con 0x7f15100a02f0 2024-01-24T23:25:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f15077fe640 4 set_mon_vals no callback set 2024-01-24T23:25:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient: dump: 2024-01-24T23:25:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.630 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.631 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f15177d8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f15177d8640 1 -- 172.21.15.115:0/1887183894 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f14f00065d0 con 0x7f14f0002870 2024-01-24T23:25:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f1508004260 con 0x7f15100a02f0 2024-01-24T23:25:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(129..129 src has 1..129) v4 ==== 6377+0+0 (unknown 51197916 0 0) 0x7f1508044450 con 0x7f15100a02f0 2024-01-24T23:25:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1514d73640 10 monclient: get_auth_request con 0x7f14f4037220 auth_method 0 2024-01-24T23:25:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2514239542 0 0) 0x7f14f00065d0 con 0x7f14f0002870 2024-01-24T23:25:19.631 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient(hunting): my global_id is 5943 2024-01-24T23:25:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15101ac510 con 0x7f14f0002870 2024-01-24T23:25:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3259022047 0 0) 0x7f15101ac510 con 0x7f14f0002870 2024-01-24T23:25:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f14f00065d0 con 0x7f14f0002870 2024-01-24T23:25:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f150c0019c0 con 0x7f14f0002870 2024-01-24T23:25:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 10 monclient: dump: 2024-01-24T23:25:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.632 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.633 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.633 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.633 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.633 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.633 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.633 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.633 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.633 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.633 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.633 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.633 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2907339050 0 0) 0x7f14f00065d0 con 0x7f14f0002870 2024-01-24T23:25:19.633 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.614+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 --> v1:172.21.15.132:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f14f000a120 con 0x7f14f0002870 2024-01-24T23:25:19.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.618+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f150c003600 con 0x7f14f0002870 2024-01-24T23:25:19.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.618+0000 7f1505ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:19.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.618+0000 7f1505ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:19.635 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.618+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 >> v1:172.21.15.132:6790/0 conn(0x7f14f0002870 legacy=0x7f14f0004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.618+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 >> v1:172.21.15.132:6790/0 conn(0x7f14f0002870 legacy=0x7f14f0004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:19.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.622+0000 7f15177d8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:25:19.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.622+0000 7f15177d8640 1 -- 172.21.15.115:0/1887183894 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f14f0009df0 con 0x7f14f00090b0 2024-01-24T23:25:19.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.622+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1042478768 0 0) 0x7f14f0009df0 con 0x7f14f00090b0 2024-01-24T23:25:19.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.622+0000 7f1505ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.622+0000 7f1505ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.622+0000 7f1505ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.622+0000 7f1505ffb640 10 monclient(hunting): my global_id is 5952 2024-01-24T23:25:19.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.622+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14f00065d0 con 0x7f14f00090b0 2024-01-24T23:25:19.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.622+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3672448599 0 0) 0x7f14f00065d0 con 0x7f14f00090b0 2024-01-24T23:25:19.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.622+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f14f0009df0 con 0x7f14f00090b0 2024-01-24T23:25:19.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.622+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f150c003ce0 con 0x7f14f00090b0 2024-01-24T23:25:19.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.622+0000 7f1505ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.636 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.622+0000 7f1505ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:19.637 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.622+0000 7f1505ffb640 10 monclient: dump: 2024-01-24T23:25:19.637 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.637 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.637 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.637 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.637 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.637 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.637 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.637 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.637 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.637 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.637 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.637 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.637 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.637 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.637 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.638 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.622+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4038859666 0 0) 0x7f14f0009df0 con 0x7f14f00090b0 2024-01-24T23:25:19.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.622+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 --> v1:172.21.15.132:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f14f000c310 con 0x7f14f00090b0 2024-01-24T23:25:19.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.626+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7545 (unknown 0 0 3364173964) 0x7f150c006030 con 0x7f14f00090b0 2024-01-24T23:25:19.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.626+0000 7f1505ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:25:19.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.626+0000 7f1505ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:19.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.626+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 >> v1:172.21.15.132:6790/0 conn(0x7f14f00090b0 legacy=0x7f14f0009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.638 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.626+0000 7f1505ffb640 1 -- 172.21.15.115:0/1887183894 >> v1:172.21.15.132:6790/0 conn(0x7f14f00090b0 legacy=0x7f14f0009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:19.638 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:25:19.638 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:19.638 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:19.639 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "leader", 2024-01-24T23:25:19.639 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 24, 2024-01-24T23:25:19.639 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:25:19.639 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:25:19.639 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:25:19.639 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:25:19.639 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:25:19.639 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:25:19.639 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:25:19.639 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:25:19.639 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:25:19.639 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:25:19.639 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.639 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 4, 2024-01-24T23:25:19.639 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:19.640 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:25:19.640 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:25:19.640 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:19.640 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:19.640 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:19.640 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:19.640 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:19.640 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:19.640 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:19.640 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:19.640 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:19.640 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:19.640 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:19.640 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.641 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:25:19.641 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:25:19.641 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:19.641 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:19.641 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:19.641 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:19.641 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:19.641 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:19.641 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:19.641 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:19.641 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:19.641 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:19.641 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:19.641 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.642 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:25:19.642 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:25:19.642 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:25:19.642 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:25:19.642 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:25:19.642 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:25:19.642 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:19.642 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:19.642 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:25:19.642 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:25:19.642 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:25:19.642 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:25:19.642 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:25:19.642 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:25:19.642 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:25:19.643 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:19.643 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:25:19.643 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:19.643 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:19.643 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:19.643 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:19.643 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:19.643 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:19.643 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:19.643 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:19.643 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:19.643 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:19.643 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:19.643 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.643 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:25:19.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.644 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:25:19.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.644 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:19.644 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:19.644 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.644 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.644 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.644 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:25:19.644 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.644 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.644 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.645 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:25:19.645 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:25:19.645 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.645 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.645 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.645 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:19.645 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:19.645 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.645 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.645 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.645 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:25:19.645 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.646 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.646 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:25:19.646 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:25:19.646 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.646 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.646 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.646 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:19.646 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:19.646 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.646 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.647 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.647 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:25:19.647 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.647 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.647 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.647 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:25:19.647 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:25:19.647 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.647 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.647 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.648 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:19.648 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:19.648 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.648 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.648 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.648 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:25:19.648 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.648 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.648 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.648 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:25:19.648 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:25:19.648 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.649 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.649 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.649 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:19.649 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:19.649 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.649 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.649 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.649 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:25:19.649 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.649 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.649 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.650 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:25:19.650 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:25:19.650 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.650 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.650 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.650 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:19.650 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:19.650 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.650 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.650 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.650 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:25:19.650 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.651 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.651 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.651 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:25:19.651 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:25:19.651 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.651 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.651 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.651 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:19.651 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:19.651 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.651 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.652 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.652 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:25:19.652 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.652 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.652 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.652 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:25:19.652 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:25:19.652 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.652 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.652 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.652 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:19.653 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:19.653 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.653 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.653 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.653 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:25:19.653 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.653 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.653 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.653 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:25:19.653 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:25:19.653 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.653 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.654 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.654 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:19.654 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:19.654 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.654 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.654 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.654 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:25:19.654 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.654 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.654 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.654 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:25:19.655 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:25:19.655 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.655 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.655 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.655 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.655 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.655 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:25:19.655 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:25:19.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.655 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:19.655 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:19.655 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:19.655 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.655 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.656 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:25:19.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.656 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:19.656 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:19.656 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:25:19.656 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.656 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.656 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:25:19.656 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:25:19.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.626+0000 7f15177d8640 1 -- 172.21.15.115:0/1887183894 >> v1:172.21.15.132:6812/1835401922 conn(0x7f14f4037220 legacy=0x7f14f40396e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.626+0000 7f15177d8640 10 monclient: shutdown 2024-01-24T23:25:19.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.626+0000 7f15177d8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:19.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.626+0000 7f15177d8640 1 -- 172.21.15.115:0/1887183894 >> v1:172.21.15.115:6793/0 conn(0x7f15100a02f0 legacy=0x7f15101d3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.626+0000 7f1515d75640 1 -- 172.21.15.115:0/1887183894 reap_dead start 2024-01-24T23:25:19.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.626+0000 7f1515d75640 1 -- 172.21.15.115:0/1887183894 reap_dead start 2024-01-24T23:25:19.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.626+0000 7f15177d8640 1 -- 172.21.15.115:0/1887183894 shutdown_connections 2024-01-24T23:25:19.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.626+0000 7f15177d8640 1 -- 172.21.15.115:0/1887183894 >> 172.21.15.115:0/1887183894 conn(0x7f15100a7820 msgr2=0x7f15100a6720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:19.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.626+0000 7f15177d8640 1 -- 172.21.15.115:0/1887183894 shutdown_connections 2024-01-24T23:25:19.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.626+0000 7f15177d8640 1 -- 172.21.15.115:0/1887183894 wait complete. 2024-01-24T23:25:19.658 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:25:19.659 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-01-24T23:25:19.659 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-01-24T23:25:19.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.762+0000 7fb7fc8d1640 10 monclient: get_monmap_and_config 2024-01-24T23:25:19.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.762+0000 7fb7fc8d1640 10 monclient: build_initial_monmap 2024-01-24T23:25:19.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.762+0000 7fb7fc8d1640 10 monclient: monmap: 2024-01-24T23:25:19.766 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:19.766 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.766 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:19.766839+0000 2024-01-24T23:25:19.766 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:19.766839+0000 2024-01-24T23:25:19.767 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:19.767 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.767 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:19.767 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:19.767 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:19.767 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:19.767 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:19.767 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:19.767 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:19.767 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:19.767 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:19.767 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 1 Processor -- start 2024-01-24T23:25:19.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 1 -- start start 2024-01-24T23:25:19.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 10 monclient: init 2024-01-24T23:25:19.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:19.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 10 monclient: _add_conns ranks=[1,2,0,7,3,8,6,5,4] 2024-01-24T23:25:19.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 10 monclient(hunting): picked mon.noname-f con 0x7fb7f80a0880 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:19.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 10 monclient(hunting): picked mon.noname-b con 0x7fb7f809f1c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:19.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 10 monclient(hunting): picked mon.noname-a con 0x7fb7f813d490 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:19.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f804ed70 con 0x7fb7f813d490 2024-01-24T23:25:19.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f80a3870 con 0x7fb7f80a0880 2024-01-24T23:25:19.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f80a32e0 con 0x7fb7f809f1c0 2024-01-24T23:25:19.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:19.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 10 monclient(hunting): authenticate will time out at 1198.689592s 2024-01-24T23:25:19.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f6d9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb7f813d490 0x7fb7f813f880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44276/0 (socket says 172.21.15.115:44276) 2024-01-24T23:25:19.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f6d9d640 1 -- 172.21.15.115:0/2142769964 learned_addr learned my addr 172.21.15.115:0/2142769964 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:19.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f5d9b640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fb7f809f1c0 0x7fb7f813bd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:60438/0 (socket says 172.21.15.115:60438) 2024-01-24T23:25:19.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 1 -- 172.21.15.115:0/2142769964 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2674766250 0 0) 0x7fb7f80a32e0 con 0x7fb7f809f1c0 2024-01-24T23:25:19.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 10 monclient(hunting): my global_id is 85340 2024-01-24T23:25:19.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 1 -- 172.21.15.115:0/2142769964 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7e0003530 con 0x7fb7f809f1c0 2024-01-24T23:25:19.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 1 -- 172.21.15.115:0/2142769964 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2572431922 0 0) 0x7fb7f804ed70 con 0x7fb7f813d490 2024-01-24T23:25:19.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 10 monclient(hunting): my global_id is 94302 2024-01-24T23:25:19.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 1 -- 172.21.15.115:0/2142769964 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7f80a32e0 con 0x7fb7f813d490 2024-01-24T23:25:19.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 1 -- 172.21.15.115:0/2142769964 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4267772248 0 0) 0x7fb7e0003530 con 0x7fb7f809f1c0 2024-01-24T23:25:19.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 1 -- 172.21.15.115:0/2142769964 >> v1:172.21.15.132:6789/0 conn(0x7fb7f80a0880 legacy=0x7fb7f80a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 1 -- 172.21.15.115:0/2142769964 >> v1:172.21.15.115:6789/0 conn(0x7fb7f813d490 legacy=0x7fb7f813f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 10 monclient: _finish_hunting 0 2024-01-24T23:25:19.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 1 monclient: found mon.noname-b 2024-01-24T23:25:19.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:19.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:25:19.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 1 -- 172.21.15.115:0/2142769964 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f806c220 con 0x7fb7f809f1c0 2024-01-24T23:25:19.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 10 monclient: _finish_auth 0 2024-01-24T23:25:19.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 10 monclient: _check_auth_tickets 2024-01-24T23:25:19.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:19.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 1 -- 172.21.15.115:0/2142769964 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb7e80017c0 con 0x7fb7f809f1c0 2024-01-24T23:25:19.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 5 monclient: authenticate success, global_id 85340 2024-01-24T23:25:19.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:19.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:25:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 10 monclient: dump: 2024-01-24T23:25:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.778 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.779 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.779 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:19.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 1 -- 172.21.15.115:0/2142769964 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb7e8001aa0 con 0x7fb7f809f1c0 2024-01-24T23:25:19.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:19.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:19.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 4 set_mon_vals no callback set 2024-01-24T23:25:19.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 1 -- 172.21.15.115:0/2142769964 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb7e8004b30 con 0x7fb7f809f1c0 2024-01-24T23:25:19.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:25:19.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7f559a640 10 monclient: dump: 2024-01-24T23:25:19.779 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.779 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.779 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.779 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.779 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.780 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.780 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.780 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.780 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.780 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.780 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.780 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.780 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.780 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.780 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.780 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 10 monclient: shutdown 2024-01-24T23:25:19.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:19.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 1 -- 172.21.15.115:0/2142769964 >> v1:172.21.15.115:6790/0 conn(0x7fb7f809f1c0 legacy=0x7fb7f813bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 1 -- 172.21.15.115:0/2142769964 shutdown_connections 2024-01-24T23:25:19.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 1 -- 172.21.15.115:0/2142769964 >> 172.21.15.115:0/2142769964 conn(0x7fb7f80666f0 msgr2=0x7fb7f8066b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:19.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 1 -- 172.21.15.115:0/2142769964 shutdown_connections 2024-01-24T23:25:19.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 1 -- 172.21.15.115:0/2142769964 wait complete. 2024-01-24T23:25:19.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 10 monclient: build_initial_monmap 2024-01-24T23:25:19.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.766+0000 7fb7fc8d1640 10 monclient: monmap: 2024-01-24T23:25:19.781 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:19.781 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:19.781 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:19.781 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:19.781 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:19.781 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:19.781 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:19.781 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:19.781 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:19.781 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:19.781 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:19.782 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:19.782 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:19.782 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:19.782 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:19.782 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7fc8d1640 1 Processor -- start 2024-01-24T23:25:19.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7fc8d1640 1 -- start start 2024-01-24T23:25:19.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7fc8d1640 10 monclient: init 2024-01-24T23:25:19.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7fc8d1640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:19.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7fc8d1640 10 monclient: _add_conns ranks=[8,4,5,0,6,2,1,7,3] 2024-01-24T23:25:19.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7fc8d1640 10 monclient(hunting): picked mon.noname-e con 0x7fb7f809f1c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:19.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7fc8d1640 10 monclient(hunting): picked mon.noname-c con 0x7fb7f80a0880 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:19.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7fc8d1640 10 monclient(hunting): picked mon.noname-h con 0x7fb7f813d490 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:19.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7fc8d1640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f809d020 con 0x7fb7f80a0880 2024-01-24T23:25:19.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7fc8d1640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f81cc510 con 0x7fb7f813d490 2024-01-24T23:25:19.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7fc8d1640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f806b5c0 con 0x7fb7f809f1c0 2024-01-24T23:25:19.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7fc8d1640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:19.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7fc8d1640 10 monclient(hunting): authenticate will time out at 1198.692513s 2024-01-24T23:25:19.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7f659c640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fb7f809f1c0 0x7fb7f81cb6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:59588/0 (socket says 172.21.15.115:59588) 2024-01-24T23:25:19.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7f659c640 1 -- 172.21.15.115:0/1083250500 learned_addr learned my addr 172.21.15.115:0/1083250500 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:19.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688361173 0 0) 0x7fb7f809d020 con 0x7fb7f80a0880 2024-01-24T23:25:19.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient(hunting): my global_id is 94306 2024-01-24T23:25:19.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7d8003530 con 0x7fb7f80a0880 2024-01-24T23:25:19.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3605987130 0 0) 0x7fb7f806b5c0 con 0x7fb7f809f1c0 2024-01-24T23:25:19.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient(hunting): my global_id is 85211 2024-01-24T23:25:19.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7f809d020 con 0x7fb7f809f1c0 2024-01-24T23:25:19.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4031208536 0 0) 0x7fb7f809d020 con 0x7fb7f809f1c0 2024-01-24T23:25:19.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb7f806b5c0 con 0x7fb7f809f1c0 2024-01-24T23:25:19.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1232950672 0 0) 0x7fb7d8003530 con 0x7fb7f80a0880 2024-01-24T23:25:19.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb7f809d020 con 0x7fb7f80a0880 2024-01-24T23:25:19.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb7ec001b70 con 0x7fb7f809f1c0 2024-01-24T23:25:19.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:25:19.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient(hunting): dump: 2024-01-24T23:25:19.784 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.784 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.784 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.784 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.785 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.785 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.785 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.785 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.785 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.785 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.785 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.785 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.785 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.785 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.785 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.785 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:19.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb7e8001780 con 0x7fb7f80a0880 2024-01-24T23:25:19.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:19.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient(hunting): dump: 2024-01-24T23:25:19.786 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.786 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.786 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.786 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.786 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.786 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.786 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.786 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.786 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.786 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.786 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.786 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.786 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.787 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.787 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.787 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975536688 0 0) 0x7fb7f81cc510 con 0x7fb7f813d490 2024-01-24T23:25:19.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient(hunting): my global_id is 5918 2024-01-24T23:25:19.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7d8003530 con 0x7fb7f813d490 2024-01-24T23:25:19.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 272155313 0 0) 0x7fb7f806b5c0 con 0x7fb7f809f1c0 2024-01-24T23:25:19.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 >> v1:172.21.15.132:6791/0 conn(0x7fb7f813d490 legacy=0x7fb7f81d39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 >> v1:172.21.15.115:6791/0 conn(0x7fb7f80a0880 legacy=0x7fb7f81cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:25:19.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 monclient: found mon.e 2024-01-24T23:25:19.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:19.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:19.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f8072030 con 0x7fb7f809f1c0 2024-01-24T23:25:19.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient: _finish_auth 0 2024-01-24T23:25:19.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:19.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:19.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7fc8d1640 5 monclient: authenticate success, global_id 85211 2024-01-24T23:25:19.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7fc8d1640 10 monclient: _renew_subs 2024-01-24T23:25:19.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7fc8d1640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:19.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7fc8d1640 1 -- 172.21.15.115:0/1083250500 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f809cd40 con 0x7fb7f809f1c0 2024-01-24T23:25:19.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7fc8d1640 10 monclient: _renew_subs 2024-01-24T23:25:19.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7fc8d1640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:19.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7fc8d1640 1 -- 172.21.15.115:0/1083250500 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f809d7d0 con 0x7fb7f809f1c0 2024-01-24T23:25:19.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb7ec0038d0 con 0x7fb7f809f1c0 2024-01-24T23:25:19.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:19.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb7ec003d80 con 0x7fb7f809f1c0 2024-01-24T23:25:19.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7f4d99640 4 set_mon_vals no callback set 2024-01-24T23:25:19.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:19.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient: dump: 2024-01-24T23:25:19.789 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.789 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.789 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.789 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.789 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.789 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.789 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.790 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.790 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.790 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.790 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.790 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.790 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.790 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.790 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.790 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7fc8d1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:19.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7fc8d1640 1 -- 172.21.15.115:0/1083250500 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb7dc0065d0 con 0x7fb7dc002870 2024-01-24T23:25:19.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fb7ec003ee0 con 0x7fb7f809f1c0 2024-01-24T23:25:19.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(129..129 src has 1..129) v4 ==== 6377+0+0 (unknown 51197916 0 0) 0x7fb7ec0440a0 con 0x7fb7f809f1c0 2024-01-24T23:25:19.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7f6d9d640 10 monclient: get_auth_request con 0x7fb7d8037850 auth_method 0 2024-01-24T23:25:19.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1667814832 0 0) 0x7fb7dc0065d0 con 0x7fb7dc002870 2024-01-24T23:25:19.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient(hunting): my global_id is 5927 2024-01-24T23:25:19.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7f806b5c0 con 0x7fb7dc002870 2024-01-24T23:25:19.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3619224688 0 0) 0x7fb7f806b5c0 con 0x7fb7dc002870 2024-01-24T23:25:19.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb7dc0065d0 con 0x7fb7dc002870 2024-01-24T23:25:19.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb7e8005330 con 0x7fb7dc002870 2024-01-24T23:25:19.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:19.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 10 monclient: dump: 2024-01-24T23:25:19.791 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.791 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.791 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.792 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.792 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.792 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.792 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.792 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.792 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.792 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.792 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.792 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.792 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.792 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.792 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.792 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2590872829 0 0) 0x7fb7dc0065d0 con 0x7fb7dc002870 2024-01-24T23:25:19.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.770+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 --> v1:172.21.15.132:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb7dc00a120 con 0x7fb7dc002870 2024-01-24T23:25:19.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.774+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fb7e8007510 con 0x7fb7dc002870 2024-01-24T23:25:19.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.774+0000 7fb7e6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:19.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.774+0000 7fb7e6ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:19.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.774+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 >> v1:172.21.15.132:6791/0 conn(0x7fb7dc002870 legacy=0x7fb7dc004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.774+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 >> v1:172.21.15.132:6791/0 conn(0x7fb7dc002870 legacy=0x7fb7dc004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:19.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.778+0000 7fb7fc8d1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:25:19.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.778+0000 7fb7fc8d1640 1 -- 172.21.15.115:0/1083250500 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb7dc009df0 con 0x7fb7dc0090b0 2024-01-24T23:25:19.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.778+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 416183848 0 0) 0x7fb7dc009df0 con 0x7fb7dc0090b0 2024-01-24T23:25:19.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.778+0000 7fb7e6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.778+0000 7fb7e6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.778+0000 7fb7e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.778+0000 7fb7e6ffd640 10 monclient(hunting): my global_id is 5936 2024-01-24T23:25:19.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.778+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7dc0065d0 con 0x7fb7dc0090b0 2024-01-24T23:25:19.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.778+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3385744769 0 0) 0x7fb7dc0065d0 con 0x7fb7dc0090b0 2024-01-24T23:25:19.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.778+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb7dc009df0 con 0x7fb7dc0090b0 2024-01-24T23:25:19.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.778+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb7e8005c70 con 0x7fb7dc0090b0 2024-01-24T23:25:19.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.778+0000 7fb7e6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.778+0000 7fb7e6ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:19.795 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.778+0000 7fb7e6ffd640 10 monclient: dump: 2024-01-24T23:25:19.795 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.795 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.795 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.795 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.795 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.796 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.778+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2632400374 0 0) 0x7fb7dc009df0 con 0x7fb7dc0090b0 2024-01-24T23:25:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.778+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 --> v1:172.21.15.132:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb7dc00c310 con 0x7fb7dc0090b0 2024-01-24T23:25:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.782+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7883 (unknown 0 0 2915097453) 0x7fb7e8007220 con 0x7fb7dc0090b0 2024-01-24T23:25:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.782+0000 7fb7e6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:25:19.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.782+0000 7fb7e6ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:19.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.782+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 >> v1:172.21.15.132:6791/0 conn(0x7fb7dc0090b0 legacy=0x7fb7dc009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.782+0000 7fb7e6ffd640 1 -- 172.21.15.115:0/1083250500 >> v1:172.21.15.132:6791/0 conn(0x7fb7dc0090b0 legacy=0x7fb7dc009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:19.797 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:25:19.797 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:19.797 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:19.797 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:25:19.797 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 24, 2024-01-24T23:25:19.797 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:25:19.797 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:25:19.797 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:25:19.798 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:25:19.798 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:25:19.798 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:25:19.798 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:25:19.798 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:25:19.798 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:25:19.798 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:25:19.798 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.798 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 4, 2024-01-24T23:25:19.798 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:19.798 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:25:19.798 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:25:19.798 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:19.798 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:19.799 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:19.799 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:19.799 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:19.799 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:19.799 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:19.799 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:19.799 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:19.799 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:19.799 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:19.799 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.799 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:25:19.799 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:25:19.799 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:19.799 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:19.799 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:19.800 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:19.800 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:19.800 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:19.800 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:19.800 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:19.800 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:19.800 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:19.800 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:19.800 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.800 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:25:19.800 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:25:19.800 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:25:19.800 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:25:19.801 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:25:19.801 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:25:19.801 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:19.801 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:19.801 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:25:19.801 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:25:19.801 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:25:19.801 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:25:19.801 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:25:19.801 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:25:19.801 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:25:19.801 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:19.801 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:25:19.801 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:19.801 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:19.802 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:19.802 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:19.802 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:19.802 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:19.802 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:19.802 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:19.802 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:19.802 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:19.802 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:19.802 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.802 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:25:19.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.802 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:25:19.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.803 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:19.803 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:19.803 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.803 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.803 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.803 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:25:19.803 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.803 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.803 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.803 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:25:19.803 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:25:19.803 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.804 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.804 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.804 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:19.804 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:19.804 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.804 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.804 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.804 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:25:19.804 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.804 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.804 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.805 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:25:19.805 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:25:19.805 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.805 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.805 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.805 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:19.805 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:19.805 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.805 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.805 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.805 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:25:19.806 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.806 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.806 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.806 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:25:19.806 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:25:19.806 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.806 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.806 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.806 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:19.806 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:19.806 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.807 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.807 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.807 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:25:19.807 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.807 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.807 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.807 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:25:19.807 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:25:19.807 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.807 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.807 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.808 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:19.808 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:19.808 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.808 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.808 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.808 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:25:19.808 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.808 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.808 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.808 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:25:19.808 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:25:19.808 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.809 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.809 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.809 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:19.809 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:19.809 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.809 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.809 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.809 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:25:19.809 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.809 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.810 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:25:19.810 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:25:19.810 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.810 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.810 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.810 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:19.810 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:19.810 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.810 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.810 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.810 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:25:19.810 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.811 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.811 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.811 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:25:19.811 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:25:19.811 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.811 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.811 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.811 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:19.811 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:19.811 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.811 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.812 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.812 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:25:19.812 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.812 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.812 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.812 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:25:19.812 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:25:19.812 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.812 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.812 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.812 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:19.813 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:19.813 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.813 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.813 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.813 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:25:19.813 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.813 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.813 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.813 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:25:19.813 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:25:19.813 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.813 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.813 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.814 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.814 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.814 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:25:19.814 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:25:19.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.814 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:19.814 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:19.814 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:19.814 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.814 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.814 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:25:19.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.814 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:19.815 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:19.815 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:19.815 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.815 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.815 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:25:19.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.815 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:19.815 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:19.815 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:19.815 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.815 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.815 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": [ 2024-01-24T23:25:19.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.815 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:19.815 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:19.816 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:19.816 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.816 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.816 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:25:19.816 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:25:19.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.782+0000 7fb7fc8d1640 1 -- 172.21.15.115:0/1083250500 >> v1:172.21.15.132:6812/1835401922 conn(0x7fb7d8037850 legacy=0x7fb7d8039d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.782+0000 7fb7fc8d1640 10 monclient: shutdown 2024-01-24T23:25:19.817 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.782+0000 7fb7fc8d1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:19.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.782+0000 7fb7fc8d1640 1 -- 172.21.15.115:0/1083250500 >> v1:172.21.15.115:6793/0 conn(0x7fb7f809f1c0 legacy=0x7fb7f81cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.782+0000 7fb7f6d9d640 1 -- 172.21.15.115:0/1083250500 reap_dead start 2024-01-24T23:25:19.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.782+0000 7fb7f6d9d640 1 -- 172.21.15.115:0/1083250500 reap_dead start 2024-01-24T23:25:19.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.782+0000 7fb7fc8d1640 1 -- 172.21.15.115:0/1083250500 shutdown_connections 2024-01-24T23:25:19.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.782+0000 7fb7fc8d1640 1 -- 172.21.15.115:0/1083250500 >> 172.21.15.115:0/1083250500 conn(0x7fb7f80666f0 msgr2=0x7fb7f8138f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:19.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.782+0000 7fb7fc8d1640 1 -- 172.21.15.115:0/1083250500 shutdown_connections 2024-01-24T23:25:19.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.782+0000 7fb7fc8d1640 1 -- 172.21.15.115:0/1083250500 wait complete. 2024-01-24T23:25:19.818 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:25:19.818 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-01-24T23:25:19.819 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-01-24T23:25:19.929 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.926+0000 7f33df1e4640 10 monclient: get_monmap_and_config 2024-01-24T23:25:19.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.926+0000 7f33df1e4640 10 monclient: build_initial_monmap 2024-01-24T23:25:19.930 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.926+0000 7f33df1e4640 10 monclient: monmap: 2024-01-24T23:25:19.930 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:19.930 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.930 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:19.930427+0000 2024-01-24T23:25:19.930 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:19.930427+0000 2024-01-24T23:25:19.930 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:19.930 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.930 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:19.930 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:19.930 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:19.930 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:19.930 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:19.930 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:19.930 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:19.931 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:19.931 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:19.931 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 1 Processor -- start 2024-01-24T23:25:19.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 1 -- start start 2024-01-24T23:25:19.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 10 monclient: init 2024-01-24T23:25:19.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:19.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 10 monclient: _add_conns ranks=[0,8,6,7,5,4,3,2,1] 2024-01-24T23:25:19.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 10 monclient(hunting): picked mon.noname-a con 0x7f33d8134740 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:19.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 10 monclient(hunting): picked mon.noname-e con 0x7f33d809ec80 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:19.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 10 monclient(hunting): picked mon.noname-d con 0x7f33d8135230 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:19.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d804ed70 con 0x7f33d8134740 2024-01-24T23:25:19.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d80a0d90 con 0x7f33d8135230 2024-01-24T23:25:19.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d809e7c0 con 0x7f33d809ec80 2024-01-24T23:25:19.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:19.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 10 monclient(hunting): authenticate will time out at 1198.853247s 2024-01-24T23:25:19.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33dcf80640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f33d8134740 0x7f33d8134b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44290/0 (socket says 172.21.15.115:44290) 2024-01-24T23:25:19.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33dcf80640 1 -- 172.21.15.115:0/2862288484 learned_addr learned my addr 172.21.15.115:0/2862288484 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:19.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d7fff640 1 --1- 172.21.15.115:0/2862288484 >> v1:172.21.15.115:6793/0 conn(0x7f33d809ec80 0x7f33d8139d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:59604/0 (socket says 172.21.15.115:59604) 2024-01-24T23:25:19.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 1 -- 172.21.15.115:0/2862288484 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568435940 0 0) 0x7f33d804ed70 con 0x7f33d8134740 2024-01-24T23:25:19.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 10 monclient(hunting): my global_id is 94311 2024-01-24T23:25:19.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 1 -- 172.21.15.115:0/2862288484 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33c4003530 con 0x7f33d8134740 2024-01-24T23:25:19.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 1 -- 172.21.15.115:0/2862288484 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3170224625 0 0) 0x7f33d809e7c0 con 0x7f33d809ec80 2024-01-24T23:25:19.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 10 monclient(hunting): my global_id is 85220 2024-01-24T23:25:19.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 1 -- 172.21.15.115:0/2862288484 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33d804ed70 con 0x7f33d809ec80 2024-01-24T23:25:19.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 1 -- 172.21.15.115:0/2862288484 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478052860 0 0) 0x7f33d80a0d90 con 0x7f33d8135230 2024-01-24T23:25:19.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 10 monclient(hunting): my global_id is 6126 2024-01-24T23:25:19.940 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 1 -- 172.21.15.115:0/2862288484 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33d809e7c0 con 0x7f33d8135230 2024-01-24T23:25:19.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 1 -- 172.21.15.115:0/2862288484 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 802779300 0 0) 0x7f33c4003530 con 0x7f33d8134740 2024-01-24T23:25:19.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 1 -- 172.21.15.115:0/2862288484 >> v1:172.21.15.115:6793/0 conn(0x7f33d809ec80 legacy=0x7f33d8139d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 1 -- 172.21.15.115:0/2862288484 >> v1:172.21.15.115:6792/0 conn(0x7f33d8135230 legacy=0x7f33d813d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 10 monclient: _finish_hunting 0 2024-01-24T23:25:19.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 1 monclient: found mon.noname-a 2024-01-24T23:25:19.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:19.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:25:19.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 1 -- 172.21.15.115:0/2862288484 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d80a1070 con 0x7f33d8134740 2024-01-24T23:25:19.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 10 monclient: _finish_auth 0 2024-01-24T23:25:19.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 10 monclient: _check_auth_tickets 2024-01-24T23:25:19.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:19.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 1 -- 172.21.15.115:0/2862288484 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f33c80017c0 con 0x7f33d8134740 2024-01-24T23:25:19.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:25:19.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 10 monclient: dump: 2024-01-24T23:25:19.942 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.942 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.942 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.942 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.942 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.942 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.942 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.942 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.942 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.942 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.942 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.942 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.942 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.942 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.943 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.943 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 5 monclient: authenticate success, global_id 94311 2024-01-24T23:25:19.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:19.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 1 -- 172.21.15.115:0/2862288484 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f33c8001aa0 con 0x7f33d8134740 2024-01-24T23:25:19.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:19.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:19.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 4 set_mon_vals no callback set 2024-01-24T23:25:19.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 1 -- 172.21.15.115:0/2862288484 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f33c8004bf0 con 0x7f33d8134740 2024-01-24T23:25:19.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:19.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33d77fe640 10 monclient: dump: 2024-01-24T23:25:19.943 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.943 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.944 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.944 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.944 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.944 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.944 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.944 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.944 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.944 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.944 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.944 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.944 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.944 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.944 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.944 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 10 monclient: shutdown 2024-01-24T23:25:19.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:19.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 1 -- 172.21.15.115:0/2862288484 >> v1:172.21.15.115:6789/0 conn(0x7f33d8134740 legacy=0x7f33d8134b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 1 -- 172.21.15.115:0/2862288484 shutdown_connections 2024-01-24T23:25:19.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 1 -- 172.21.15.115:0/2862288484 >> 172.21.15.115:0/2862288484 conn(0x7f33d80a5730 msgr2=0x7f33d80a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:19.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 1 -- 172.21.15.115:0/2862288484 shutdown_connections 2024-01-24T23:25:19.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 1 -- 172.21.15.115:0/2862288484 wait complete. 2024-01-24T23:25:19.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 10 monclient: build_initial_monmap 2024-01-24T23:25:19.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 10 monclient: monmap: 2024-01-24T23:25:19.945 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:19.945 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 1 Processor -- start 2024-01-24T23:25:19.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 1 -- start start 2024-01-24T23:25:19.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 10 monclient: init 2024-01-24T23:25:19.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:19.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 10 monclient: _add_conns ranks=[6,7,5,3,2,0,1,8,4] 2024-01-24T23:25:19.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 10 monclient(hunting): picked mon.noname-d con 0x7f33d809ec80 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:19.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 10 monclient(hunting): picked mon.noname-i con 0x7f33d8134740 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:19.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 10 monclient(hunting): picked mon.noname-h con 0x7f33d8135230 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:19.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d811b8a0 con 0x7f33d8135230 2024-01-24T23:25:19.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.930+0000 7f33df1e4640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d8071c00 con 0x7f33d809ec80 2024-01-24T23:25:19.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33df1e4640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d80716b0 con 0x7f33d8134740 2024-01-24T23:25:19.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33df1e4640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:19.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33df1e4640 10 monclient(hunting): authenticate will time out at 1198.856015s 2024-01-24T23:25:19.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33dcf80640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f33d809ec80 0x7f33d8133d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:40572/0 (socket says 172.21.15.115:40572) 2024-01-24T23:25:19.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33dcf80640 1 -- 172.21.15.115:0/4034511794 learned_addr learned my addr 172.21.15.115:0/4034511794 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:19.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585846507 0 0) 0x7f33d8071c00 con 0x7f33d809ec80 2024-01-24T23:25:19.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient(hunting): my global_id is 6135 2024-01-24T23:25:19.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33b8003530 con 0x7f33d809ec80 2024-01-24T23:25:19.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2649979243 0 0) 0x7f33b8003530 con 0x7f33d809ec80 2024-01-24T23:25:19.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f33d8071c00 con 0x7f33d809ec80 2024-01-24T23:25:19.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f33c8004b30 con 0x7f33d809ec80 2024-01-24T23:25:19.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:25:19.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient(hunting): dump: 2024-01-24T23:25:19.948 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.948 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.949 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.949 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.949 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.949 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.949 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.949 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.949 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.949 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.949 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.949 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.949 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.949 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.949 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.949 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:19.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2896271527 0 0) 0x7f33d8071c00 con 0x7f33d809ec80 2024-01-24T23:25:19.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 >> v1:172.21.15.132:6792/0 conn(0x7f33d8134740 legacy=0x7f33d81d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 >> v1:172.21.15.132:6791/0 conn(0x7f33d8135230 legacy=0x7f33d81d3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient: _finish_hunting 0 2024-01-24T23:25:19.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 1 monclient: found mon.d 2024-01-24T23:25:19.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:19.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:25:19.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d813e0f0 con 0x7f33d809ec80 2024-01-24T23:25:19.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient: _finish_auth 0 2024-01-24T23:25:19.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient: _check_auth_tickets 2024-01-24T23:25:19.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:19.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33df1e4640 5 monclient: authenticate success, global_id 6135 2024-01-24T23:25:19.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33df1e4640 10 monclient: _renew_subs 2024-01-24T23:25:19.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33df1e4640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:25:19.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33df1e4640 1 -- 172.21.15.115:0/4034511794 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d811ce50 con 0x7f33d809ec80 2024-01-24T23:25:19.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33df1e4640 10 monclient: _renew_subs 2024-01-24T23:25:19.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33df1e4640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:25:19.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33df1e4640 1 -- 172.21.15.115:0/4034511794 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d811bf80 con 0x7f33d809ec80 2024-01-24T23:25:19.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f33c8004ec0 con 0x7f33d809ec80 2024-01-24T23:25:19.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:19.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f33c8005370 con 0x7f33d809ec80 2024-01-24T23:25:19.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d6ffd640 4 set_mon_vals no callback set 2024-01-24T23:25:19.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:19.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient: dump: 2024-01-24T23:25:19.951 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.951 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.951 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.952 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.952 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.952 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.952 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.952 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.952 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.952 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.952 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.952 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.952 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.952 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.952 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.953 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33df1e4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:19.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33df1e4640 1 -- 172.21.15.115:0/4034511794 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f33bc0065d0 con 0x7f33bc002870 2024-01-24T23:25:19.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f33c800fdf0 con 0x7f33d809ec80 2024-01-24T23:25:19.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(129..129 src has 1..129) v4 ==== 6377+0+0 (unknown 51197916 0 0) 0x7f33c80444b0 con 0x7f33d809ec80 2024-01-24T23:25:19.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33dd781640 10 monclient: get_auth_request con 0x7f33b8037070 auth_method 0 2024-01-24T23:25:19.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 386019250 0 0) 0x7f33bc0065d0 con 0x7f33bc002870 2024-01-24T23:25:19.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient(hunting): my global_id is 5929 2024-01-24T23:25:19.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33d8071c00 con 0x7f33bc002870 2024-01-24T23:25:19.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2218509322 0 0) 0x7f33d8071c00 con 0x7f33bc002870 2024-01-24T23:25:19.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f33bc0065d0 con 0x7f33bc002870 2024-01-24T23:25:19.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f33c0001520 con 0x7f33bc002870 2024-01-24T23:25:19.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:25:19.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient: dump: 2024-01-24T23:25:19.954 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.954 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.954 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.954 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.954 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.954 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.954 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.954 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.954 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.954 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.955 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.955 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.955 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.955 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.955 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.955 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1640342000 0 0) 0x7f33bc0065d0 con 0x7f33bc002870 2024-01-24T23:25:19.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 --> v1:172.21.15.132:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f33bc00a120 con 0x7f33bc002870 2024-01-24T23:25:19.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 <== mon.7 v1:172.21.15.132:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f33c0001c90 con 0x7f33bc002870 2024-01-24T23:25:19.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:19.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:19.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 >> v1:172.21.15.132:6792/0 conn(0x7f33bc002870 legacy=0x7f33bc004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.934+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 >> v1:172.21.15.132:6792/0 conn(0x7f33bc002870 legacy=0x7f33bc004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:19.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.942+0000 7f33df1e4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:25:19.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.942+0000 7f33df1e4640 1 -- 172.21.15.115:0/4034511794 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f33bc009e10 con 0x7f33bc0090d0 2024-01-24T23:25:19.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.942+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912939250 0 0) 0x7f33bc009e10 con 0x7f33bc0090d0 2024-01-24T23:25:19.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.942+0000 7f33d57fa640 10 monclient(hunting): _negotiate 2024-01-24T23:25:19.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.942+0000 7f33d57fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:19.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.942+0000 7f33d57fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:19.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.942+0000 7f33d57fa640 10 monclient(hunting): my global_id is 5938 2024-01-24T23:25:19.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.942+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33bc0065d0 con 0x7f33bc0090d0 2024-01-24T23:25:19.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.942+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3401856745 0 0) 0x7f33bc0065d0 con 0x7f33bc0090d0 2024-01-24T23:25:19.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.942+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f33bc009e10 con 0x7f33bc0090d0 2024-01-24T23:25:19.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.942+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f33c0004df0 con 0x7f33bc0090d0 2024-01-24T23:25:19.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.942+0000 7f33d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:19.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.942+0000 7f33d57fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:25:19.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.942+0000 7f33d57fa640 10 monclient: dump: 2024-01-24T23:25:19.958 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:19.958 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:19.958 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.958 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:19.958 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:19.958 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:19.958 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:19.959 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:19.959 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:19.959 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:19.959 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:19.959 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:19.959 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:19.959 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:19.959 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:19.959 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:19.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.942+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2761091922 0 0) 0x7f33bc009e10 con 0x7f33bc0090d0 2024-01-24T23:25:19.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.942+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 --> v1:172.21.15.132:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f33bc00c310 con 0x7f33bc0090d0 2024-01-24T23:25:19.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.942+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 <== mon.7 v1:172.21.15.132:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 4239188541) 0x7f33c0007190 con 0x7f33bc0090d0 2024-01-24T23:25:19.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.942+0000 7f33d57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:25:19.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.942+0000 7f33d57fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:19.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.942+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 >> v1:172.21.15.132:6792/0 conn(0x7f33bc0090d0 legacy=0x7f33bc009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.942+0000 7f33d57fa640 1 -- 172.21.15.115:0/4034511794 >> v1:172.21.15.132:6792/0 conn(0x7f33bc0090d0 legacy=0x7f33bc009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:19.960 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:25:19.960 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:19.960 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:19.960 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:25:19.960 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 24, 2024-01-24T23:25:19.960 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:25:19.960 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:25:19.960 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:25:19.960 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:25:19.960 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:25:19.960 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:25:19.961 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:25:19.961 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:25:19.961 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:25:19.961 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:25:19.961 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.961 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 5, 2024-01-24T23:25:19.961 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:19.961 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:25:19.961 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:25:19.961 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:19.961 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:19.961 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:19.961 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:19.961 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:19.961 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:19.961 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:19.962 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:19.962 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:19.962 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:19.962 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:19.962 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.962 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:25:19.962 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:25:19.962 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:19.962 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:19.962 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:19.962 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:19.962 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:19.962 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:19.962 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:19.962 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:19.962 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:19.962 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:19.963 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:19.963 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.963 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:25:19.963 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:25:19.963 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:25:19.963 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:25:19.963 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:25:19.963 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:25:19.963 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:19.963 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:19.963 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:25:19.963 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:25:19.963 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:25:19.963 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:25:19.963 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:25:19.964 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:25:19.964 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:25:19.964 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:19.964 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:25:19.964 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:19.964 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:19.964 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:19.964 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:19.964 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:19.964 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:19.964 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:19.964 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:19.964 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:19.964 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:19.965 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:19.965 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.965 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:25:19.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.965 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:25:19.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.965 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:19.965 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:19.965 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.965 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.965 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.965 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:25:19.965 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.966 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.966 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.966 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:25:19.966 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:25:19.966 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.966 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.966 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.966 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:19.967 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:19.967 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.967 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.967 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.967 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:25:19.967 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.967 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.967 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.967 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:25:19.967 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:25:19.968 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.968 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.968 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.968 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:19.968 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:19.968 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.968 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.968 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.968 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:25:19.968 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.968 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.968 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.969 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:25:19.969 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:25:19.969 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.969 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.969 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.969 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:19.969 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:19.969 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.969 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.969 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.969 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:25:19.969 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.970 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.970 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.970 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:25:19.970 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:25:19.970 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.970 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.970 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.970 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:19.970 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:19.970 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.970 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.971 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.971 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:25:19.971 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.971 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.971 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.971 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:25:19.971 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:25:19.971 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.971 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.971 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.972 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:19.972 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:19.972 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.972 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.972 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.972 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:25:19.972 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.972 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.972 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.972 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:25:19.972 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:25:19.972 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.973 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.973 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.973 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:19.973 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:19.973 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.973 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.973 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.973 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:25:19.973 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.973 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.973 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.974 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:25:19.974 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:25:19.974 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.974 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.974 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.974 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:19.974 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:19.974 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.974 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.974 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.974 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:25:19.974 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.975 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.975 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.975 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:25:19.975 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:25:19.975 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.975 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.975 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.975 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:19.975 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:19.975 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:19.975 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:19.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.976 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:19.976 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:25:19.976 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:19.976 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.976 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.976 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:25:19.977 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:25:19.977 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:19.977 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:19.977 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:19.977 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.977 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.978 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:25:19.978 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:25:19.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.978 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:19.978 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:19.978 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:19.978 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.978 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.979 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:25:19.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.979 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:19.979 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:19.979 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:25:19.979 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.979 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:19.979 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:25:19.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:19.980 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:19.980 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:19.980 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 3 2024-01-24T23:25:19.980 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:19.980 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:19.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:19.980 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:25:19.980 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:25:19.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.946+0000 7f33df1e4640 1 -- 172.21.15.115:0/4034511794 >> v1:172.21.15.132:6812/1835401922 conn(0x7f33b8037070 legacy=0x7f33b8039530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.946+0000 7f33df1e4640 10 monclient: shutdown 2024-01-24T23:25:19.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.946+0000 7f33df1e4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:19.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.946+0000 7f33df1e4640 1 -- 172.21.15.115:0/4034511794 >> v1:172.21.15.115:6792/0 conn(0x7f33d809ec80 legacy=0x7f33d8133d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:19.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.946+0000 7f33dd781640 1 -- 172.21.15.115:0/4034511794 reap_dead start 2024-01-24T23:25:19.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.946+0000 7f33dd781640 1 -- 172.21.15.115:0/4034511794 reap_dead start 2024-01-24T23:25:19.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.946+0000 7f33df1e4640 1 -- 172.21.15.115:0/4034511794 shutdown_connections 2024-01-24T23:25:19.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.946+0000 7f33df1e4640 1 -- 172.21.15.115:0/4034511794 >> 172.21.15.115:0/4034511794 conn(0x7f33d80a5730 msgr2=0x7f33d809f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:19.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.946+0000 7f33df1e4640 1 -- 172.21.15.115:0/4034511794 shutdown_connections 2024-01-24T23:25:19.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:19.946+0000 7f33df1e4640 1 -- 172.21.15.115:0/4034511794 wait complete. 2024-01-24T23:25:19.984 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:25:19.984 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b', 'i', 'f'] 2024-01-24T23:25:19.984 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['c', 'd', 'g'] 2024-01-24T23:25:19.984 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-01-24T23:25:19.984 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-01-24T23:25:19.985 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-01-24T23:25:19.985 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:25:19.995 INFO:tasks.ceph.mon.b:Stopped 2024-01-24T23:25:19.995 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-01-24T23:25:19.995 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-01-24T23:25:19.996 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-01-24T23:25:19.996 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:25:20.021 INFO:tasks.ceph.mon.i:Stopped 2024-01-24T23:25:20.021 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f 2024-01-24T23:25:20.021 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-01-24T23:25:20.022 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-01-24T23:25:20.022 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:25:20.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf53fff640 1 -- 172.21.15.132:0/935895895 >> v1:172.21.15.132:6789/0 conn(0x7fdf54139fb0 legacy=0x7fdf541d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-01-24T23:25:20.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf53fff640 1 -- 172.21.15.132:0/935895895 >> v1:172.21.15.132:6789/0 conn(0x7fdf54139fb0 legacy=0x7fdf541d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:25:20.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf53fff640 1 --1- 172.21.15.132:0/935895895 >> v1:172.21.15.132:6789/0 conn(0x7fdf54139fb0 0x7fdf541d1cc0 :-1 s=OPENED pgs=218 cs=1 l=1).handle_message read tag failed 2024-01-24T23:25:20.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf53fff640 1 --1- 172.21.15.132:0/935895895 >> v1:172.21.15.132:6789/0 conn(0x7fdf54139fb0 0x7fdf541d1cc0 :-1 s=OPENED pgs=218 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:25:20.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 10 monclient: ms_handle_reset current mon v1:172.21.15.132:6789/0 2024-01-24T23:25:20.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:20.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 >> v1:172.21.15.132:6789/0 conn(0x7fdf54139fb0 legacy=0x7fdf541d1cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:20.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 10 monclient: _add_conns ranks=[2,6,3,8,1,0,7,5,4] 2024-01-24T23:25:20.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 10 monclient(hunting): picked mon.b con 0x7fdf40041e60 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:20.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 10 monclient(hunting): picked mon.d con 0x7fdf40045bd0 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:20.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 10 monclient(hunting): picked mon.g con 0x7fdf40049ca0 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:20.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdf541311e0 con 0x7fdf40041e60 2024-01-24T23:25:20.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdf5411bf30 con 0x7fdf40049ca0 2024-01-24T23:25:20.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdf4004fd90 con 0x7fdf40045bd0 2024-01-24T23:25:20.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:20.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf52ffd640 1 -- 172.21.15.132:0/935895895 >> v1:172.21.15.115:6790/0 conn(0x7fdf40041e60 legacy=0x7fdf40044250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:25:20.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840297335 0 0) 0x7fdf540008e0 con 0x7fdf40045bd0 2024-01-24T23:25:20.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:20.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:20.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-01-24T23:25:20.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 --> v1:172.21.15.115:6792/0 -- auth(proto 2 132 bytes epoch 0) v1 -- 0x7fdf4004ed60 con 0x7fdf40045bd0 2024-01-24T23:25:20.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 572603910 0 0) 0x7fdf5411bf30 con 0x7fdf40049ca0 2024-01-24T23:25:20.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:20.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:20.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-01-24T23:25:20.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 --> v1:172.21.15.132:6790/0 -- auth(proto 2 132 bytes epoch 0) v1 -- 0x7fdf540008e0 con 0x7fdf40049ca0 2024-01-24T23:25:20.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 233+0+0 (unknown 1372194553 0 0) 0x7fdf4004ed60 con 0x7fdf40045bd0 2024-01-24T23:25:20.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 >> v1:172.21.15.132:6790/0 conn(0x7fdf40049ca0 legacy=0x7fdf4004c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:20.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 >> v1:172.21.15.115:6790/0 conn(0x7fdf40041e60 legacy=0x7fdf40044250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:20.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf53fff640 1 -- 172.21.15.132:0/935895895 reap_dead start 2024-01-24T23:25:20.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:25:20.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 1 monclient: found mon.d 2024-01-24T23:25:20.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:20.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:25:20.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+}) v3 -- 0x7fdf40050fb0 con 0x7fdf40045bd0 2024-01-24T23:25:20.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-01-24T23:25:20.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:25:20.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush weight-set rm", "pool": "barset"} v 0) v1 -- 0x7fdf541311e0 con 0x7fdf40045bd0 2024-01-24T23:25:20.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 10 monclient: _finish_auth 0 2024-01-24T23:25:20.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:25:20.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:20.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdf4c018ee0 con 0x7fdf40045bd0 2024-01-24T23:25:20.075 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:20.075 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:20.075 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 10 monclient: dump: 2024-01-24T23:25:20.075 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:20.075 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:20.075 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:20.075 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:20.075 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:20.076 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:20.076 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:20.076 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:20.076 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:20.076 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:20.076 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:20.076 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:20.077 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:20.077 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:20.077 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:20.077 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:20.077 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdf4c01d120 con 0x7fdf40045bd0 2024-01-24T23:25:20.077 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf3bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:20.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.065+0000 7fdf517fa640 4 set_mon_vals no callback set 2024-01-24T23:25:20.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:20.069+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fdf4c015050 con 0x7fdf40045bd0 2024-01-24T23:25:20.082 INFO:tasks.ceph.mon.f:Stopped 2024-01-24T23:25:20.083 INFO:tasks.mon_thrash:Sending STOP to mon c 2024-01-24T23:25:20.083 INFO:tasks.ceph.mon.c:Sent signal 19 2024-01-24T23:25:20.083 INFO:tasks.mon_thrash:Sending STOP to mon d 2024-01-24T23:25:20.084 INFO:tasks.ceph.mon.d:Sent signal 19 2024-01-24T23:25:20.084 INFO:tasks.mon_thrash:Sending STOP to mon g 2024-01-24T23:25:20.084 INFO:tasks.ceph.mon.g:Sent signal 19 2024-01-24T23:25:20.084 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-01-24T23:25:21.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:21.065+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:21.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:21.065+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:21.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:21.065+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:21.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:21.065+0000 7fdf3affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:22.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:22.061+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:22.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:22.061+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:22.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:22.061+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:23.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:23.061+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:23.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:23.061+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:23.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:23.061+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:24.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:24.061+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:24.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:24.061+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:24.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:24.061+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:25.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:25.060+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:25.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:25.060+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:25.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:25.060+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:26.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:26.064+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:26.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:26.064+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:26.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:26.064+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:27.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:27.060+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:27.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:27.060+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:27.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:27.060+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:28.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:28.060+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:28.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:28.060+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:28.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:28.060+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:29.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:29.064+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:29.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:29.064+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:29.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:29.064+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:30.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:30.060+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:30.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:30.060+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:30.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:30.060+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:31.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:31.060+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:31.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:31.060+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:31.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:31.060+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:31.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:31.060+0000 7fdf3affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:32.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:32.060+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:32.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:32.060+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:32.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:32.060+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:33.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:33.064+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:33.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:33.064+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:33.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:33.064+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:34.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:34.064+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:34.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:34.064+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:34.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:34.064+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:35.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:35.064+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:35.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:35.064+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:35.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:35.064+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:35.085 INFO:tasks.mon_thrash:Sending CONT to mon c 2024-01-24T23:25:35.086 INFO:tasks.ceph.mon.c:Sent signal 18 2024-01-24T23:25:35.086 INFO:tasks.mon_thrash:Sending CONT to mon d 2024-01-24T23:25:35.086 INFO:tasks.ceph.mon.d:Sent signal 18 2024-01-24T23:25:35.086 INFO:tasks.mon_thrash:Sending CONT to mon g 2024-01-24T23:25:35.087 INFO:tasks.ceph.mon.g:Sent signal 18 2024-01-24T23:25:35.087 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 6 2024-01-24T23:25:35.087 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-01-24T23:25:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:35.213+0000 7f4b46041640 10 monclient: get_monmap_and_config 2024-01-24T23:25:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:35.213+0000 7f4b46041640 10 monclient: build_initial_monmap 2024-01-24T23:25:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:35.213+0000 7f4b46041640 10 monclient: monmap: 2024-01-24T23:25:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:35.216267+0000 2024-01-24T23:25:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:35.216267+0000 2024-01-24T23:25:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:35.217 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:35.213+0000 7f4b46041640 1 Processor -- start 2024-01-24T23:25:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:35.213+0000 7f4b46041640 1 -- start start 2024-01-24T23:25:35.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:35.213+0000 7f4b46041640 10 monclient: init 2024-01-24T23:25:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:35.213+0000 7f4b46041640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:35.213+0000 7f4b46041640 10 monclient: _add_conns ranks=[0,7,5,2,6,4,8,3,1] 2024-01-24T23:25:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:35.213+0000 7f4b46041640 10 monclient(hunting): picked mon.noname-a con 0x7f4b40133c10 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:35.213+0000 7f4b46041640 10 monclient(hunting): picked mon.noname-i con 0x7f4b40136960 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:35.213+0000 7f4b46041640 10 monclient(hunting): picked mon.noname-h con 0x7f4b4013a470 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:35.213+0000 7f4b46041640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b4004ed70 con 0x7f4b40133c10 2024-01-24T23:25:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:35.213+0000 7f4b46041640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b4009dee0 con 0x7f4b4013a470 2024-01-24T23:25:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:35.213+0000 7f4b46041640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b4009d400 con 0x7f4b40136960 2024-01-24T23:25:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:35.213+0000 7f4b46041640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:35.213+0000 7f4b46041640 10 monclient(hunting): authenticate will time out at 1214.138836s 2024-01-24T23:25:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:35.213+0000 7f4b3effd640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f4b40136960 legacy=0x7f4b40138d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:25:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:35.213+0000 7f4b3f7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f4b40133c10 0x7f4b40133ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:46120/0 (socket says 172.21.15.115:46120) 2024-01-24T23:25:35.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:35.213+0000 7f4b3f7fe640 1 -- 172.21.15.115:0/938230587 learned_addr learned my addr 172.21.15.115:0/938230587 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:35.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:35.417+0000 7f4b3effd640 1 -- 172.21.15.115:0/938230587 >> v1:172.21.15.132:6792/0 conn(0x7f4b40136960 legacy=0x7f4b40138d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:25:35.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:35.821+0000 7f4b3effd640 1 -- 172.21.15.115:0/938230587 >> v1:172.21.15.132:6792/0 conn(0x7f4b40136960 legacy=0x7f4b40138d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:25:36.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:36.064+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:36.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:36.064+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:36.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:36.064+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:36.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:36.625+0000 7f4b3effd640 1 -- 172.21.15.115:0/938230587 >> v1:172.21.15.132:6792/0 conn(0x7f4b40136960 legacy=0x7f4b40138d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:25:37.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:37.064+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:37.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:37.064+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:37.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:37.064+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:38.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:38.068+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:38.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:38.068+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:38.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:38.068+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:38.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:38.213+0000 7f4b3d7fa640 10 monclient(hunting): tick 2024-01-24T23:25:38.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:38.213+0000 7f4b3d7fa640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:25:38.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:38.213+0000 7f4b3d7fa640 1 monclient(hunting): continuing hunt 2024-01-24T23:25:38.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:38.213+0000 7f4b3d7fa640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:25:38.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:38.213+0000 7f4b3d7fa640 1 -- 172.21.15.115:0/938230587 >> v1:172.21.15.132:6792/0 conn(0x7f4b40136960 legacy=0x7f4b40138d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:38.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:38.213+0000 7f4b3d7fa640 1 -- 172.21.15.115:0/938230587 >> v1:172.21.15.132:6791/0 conn(0x7f4b4013a470 legacy=0x7f4b4013c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:38.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:38.213+0000 7f4b3d7fa640 1 -- 172.21.15.115:0/938230587 >> v1:172.21.15.115:6789/0 conn(0x7f4b40133c10 legacy=0x7f4b40133ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:38.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:38.213+0000 7f4b3d7fa640 10 monclient: _add_conns ranks=[4,2,1,8,6,3] 2024-01-24T23:25:38.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:38.213+0000 7f4b3d7fa640 10 monclient(hunting): picked mon.noname-c con 0x7f4b2c0037d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:38.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:38.213+0000 7f4b3d7fa640 10 monclient(hunting): picked mon.noname-b con 0x7f4b2c007460 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:38.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:38.213+0000 7f4b3d7fa640 10 monclient(hunting): picked mon.noname-f con 0x7f4b2c00b1f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:38.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:38.213+0000 7f4b3d7fa640 1 -- 172.21.15.115:0/938230587 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b4009f380 con 0x7f4b2c00b1f0 2024-01-24T23:25:38.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:38.213+0000 7f4b3d7fa640 1 -- 172.21.15.115:0/938230587 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b4009d400 con 0x7f4b2c007460 2024-01-24T23:25:38.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:38.213+0000 7f4b3d7fa640 1 -- 172.21.15.115:0/938230587 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b2c013240 con 0x7f4b2c0037d0 2024-01-24T23:25:38.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:38.213+0000 7f4b3d7fa640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:38.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:38.213+0000 7f4b3effd640 1 -- 172.21.15.115:0/938230587 >> v1:172.21.15.115:6790/0 conn(0x7f4b2c007460 legacy=0x7f4b2c009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:25:38.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:38.213+0000 7f4b3ffff640 1 -- 172.21.15.115:0/938230587 >> v1:172.21.15.132:6789/0 conn(0x7f4b2c00b1f0 legacy=0x7f4b2c00d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:38.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:38.417+0000 7f4b3effd640 1 -- 172.21.15.115:0/938230587 >> v1:172.21.15.115:6790/0 conn(0x7f4b2c007460 legacy=0x7f4b2c009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:25:38.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:38.417+0000 7f4b3ffff640 1 -- 172.21.15.115:0/938230587 >> v1:172.21.15.132:6789/0 conn(0x7f4b2c00b1f0 legacy=0x7f4b2c00d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:38.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:38.821+0000 7f4b3effd640 1 -- 172.21.15.115:0/938230587 >> v1:172.21.15.115:6790/0 conn(0x7f4b2c007460 legacy=0x7f4b2c009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:25:38.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:38.821+0000 7f4b3ffff640 1 -- 172.21.15.115:0/938230587 >> v1:172.21.15.132:6789/0 conn(0x7f4b2c00b1f0 legacy=0x7f4b2c00d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:39.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:39.068+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:39.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:39.068+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:39.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:39.068+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:39.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:39.625+0000 7f4b3effd640 1 -- 172.21.15.115:0/938230587 >> v1:172.21.15.115:6790/0 conn(0x7f4b2c007460 legacy=0x7f4b2c009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:25:39.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:39.625+0000 7f4b3ffff640 1 -- 172.21.15.115:0/938230587 >> v1:172.21.15.132:6789/0 conn(0x7f4b2c00b1f0 legacy=0x7f4b2c00d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:40.067 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:40.068+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:40.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:40.068+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:40.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:40.068+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:41.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:41.068+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:41.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:41.068+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:41.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:41.068+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:41.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:41.068+0000 7fdf3affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:41.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:41.217+0000 7f4b3d7fa640 10 monclient(hunting): tick 2024-01-24T23:25:41.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:41.217+0000 7f4b3d7fa640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:25:41.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:41.217+0000 7f4b3d7fa640 1 monclient(hunting): continuing hunt 2024-01-24T23:25:41.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:41.217+0000 7f4b3d7fa640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:25:41.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:41.217+0000 7f4b3d7fa640 1 -- 172.21.15.115:0/938230587 >> v1:172.21.15.115:6791/0 conn(0x7f4b2c0037d0 legacy=0x7f4b2c005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:41.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:41.217+0000 7f4b3d7fa640 1 -- 172.21.15.115:0/938230587 >> v1:172.21.15.115:6790/0 conn(0x7f4b2c007460 legacy=0x7f4b2c009920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:41.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:41.217+0000 7f4b3d7fa640 1 -- 172.21.15.115:0/938230587 >> v1:172.21.15.132:6789/0 conn(0x7f4b2c00b1f0 legacy=0x7f4b2c00d6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:41.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:41.217+0000 7f4b3d7fa640 10 monclient: _add_conns ranks=[6,8,3] 2024-01-24T23:25:41.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:41.217+0000 7f4b3d7fa640 10 monclient(hunting): picked mon.noname-d con 0x7f4b2c0122e0 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:41.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:41.217+0000 7f4b3ffff640 1 -- 172.21.15.115:0/938230587 reap_dead start 2024-01-24T23:25:41.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:41.217+0000 7f4b3d7fa640 10 monclient(hunting): picked mon.noname-e con 0x7f4b2c0133d0 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:41.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:41.217+0000 7f4b3d7fa640 10 monclient(hunting): picked mon.noname-g con 0x7f4b2c013ef0 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:41.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:41.217+0000 7f4b3d7fa640 1 -- 172.21.15.115:0/938230587 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b2c014470 con 0x7f4b2c013ef0 2024-01-24T23:25:41.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:41.217+0000 7f4b3d7fa640 1 -- 172.21.15.115:0/938230587 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b4009f380 con 0x7f4b2c0122e0 2024-01-24T23:25:41.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:41.217+0000 7f4b3d7fa640 1 -- 172.21.15.115:0/938230587 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b4009d400 con 0x7f4b2c0133d0 2024-01-24T23:25:41.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:41.217+0000 7f4b3d7fa640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:41.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:41.217+0000 7f4b3ffff640 1 -- 172.21.15.115:0/938230587 reap_dead start 2024-01-24T23:25:42.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:42.068+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:42.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:42.068+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:42.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:42.068+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:43.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:43.068+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:43.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:43.068+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:43.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:43.068+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:44.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:44.068+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:44.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:44.068+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:44.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:44.068+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:44.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 1 -- 172.21.15.115:0/938230587 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2071036316 0 0) 0x7f4b4009f380 con 0x7f4b2c0122e0 2024-01-24T23:25:44.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:44.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:44.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:44.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 10 monclient(hunting): my global_id is 6144 2024-01-24T23:25:44.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 1 -- 172.21.15.115:0/938230587 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b28003530 con 0x7f4b2c0122e0 2024-01-24T23:25:44.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 1 -- 172.21.15.115:0/938230587 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2739601168 0 0) 0x7f4b2c014470 con 0x7f4b2c013ef0 2024-01-24T23:25:44.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:44.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:44.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:44.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 10 monclient(hunting): my global_id is 5961 2024-01-24T23:25:44.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 1 -- 172.21.15.115:0/938230587 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b4009f380 con 0x7f4b2c013ef0 2024-01-24T23:25:44.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 1 -- 172.21.15.115:0/938230587 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 766103987 0 0) 0x7f4b28003530 con 0x7f4b2c0122e0 2024-01-24T23:25:44.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 1 -- 172.21.15.115:0/938230587 >> v1:172.21.15.115:6793/0 conn(0x7f4b2c0133d0 legacy=0x7f4b2c0137e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:44.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 1 -- 172.21.15.115:0/938230587 >> v1:172.21.15.132:6790/0 conn(0x7f4b2c013ef0 legacy=0x7f4b2c01b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:44.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:25:44.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 1 monclient: found mon.noname-d 2024-01-24T23:25:44.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:44.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:25:44.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 1 -- 172.21.15.115:0/938230587 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2c020080 con 0x7f4b2c0122e0 2024-01-24T23:25:44.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 10 monclient: _finish_auth 0 2024-01-24T23:25:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:25:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 1 -- 172.21.15.115:0/938230587 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4b300016c0 con 0x7f4b2c0122e0 2024-01-24T23:25:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:25:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 10 monclient: dump: 2024-01-24T23:25:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:44.211 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b46041640 5 monclient: authenticate success, global_id 6144 2024-01-24T23:25:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b46041640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 1 -- 172.21.15.115:0/938230587 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4b30001960 con 0x7f4b2c0122e0 2024-01-24T23:25:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 1 -- 172.21.15.115:0/938230587 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4b300048b0 con 0x7f4b2c0122e0 2024-01-24T23:25:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b46041640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b46041640 4 set_mon_vals no callback set 2024-01-24T23:25:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b3e7fc640 10 monclient: dump: 2024-01-24T23:25:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:44.212 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b46041640 10 monclient: shutdown 2024-01-24T23:25:44.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b46041640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:44.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b46041640 1 -- 172.21.15.115:0/938230587 >> v1:172.21.15.115:6792/0 conn(0x7f4b2c0122e0 legacy=0x7f4b2c012720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:44.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b46041640 1 -- 172.21.15.115:0/938230587 shutdown_connections 2024-01-24T23:25:44.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b46041640 1 -- 172.21.15.115:0/938230587 >> 172.21.15.115:0/938230587 conn(0x7f4b400a5730 msgr2=0x7f4b400a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:44.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b46041640 1 -- 172.21.15.115:0/938230587 shutdown_connections 2024-01-24T23:25:44.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.205+0000 7f4b46041640 1 -- 172.21.15.115:0/938230587 wait complete. 2024-01-24T23:25:44.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.209+0000 7f4b46041640 10 monclient: build_initial_monmap 2024-01-24T23:25:44.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.209+0000 7f4b46041640 10 monclient: monmap: 2024-01-24T23:25:44.217 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:44.217 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:44.217 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:44.217 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:44.217 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:44.217 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:44.217 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:44.217 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:44.217 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:44.217 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:44.217 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:44.217 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:44.217 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:44.217 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:44.218 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:44.218 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.209+0000 7f4b46041640 1 Processor -- start 2024-01-24T23:25:44.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.209+0000 7f4b46041640 1 -- start start 2024-01-24T23:25:44.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.209+0000 7f4b46041640 10 monclient: init 2024-01-24T23:25:44.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.209+0000 7f4b46041640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:44.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.209+0000 7f4b46041640 10 monclient: _add_conns ranks=[3,2,1,4,7,8,6,5,0] 2024-01-24T23:25:44.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.209+0000 7f4b46041640 10 monclient(hunting): picked mon.noname-g con 0x7f4b2c0122e0 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:44.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.209+0000 7f4b46041640 10 monclient(hunting): picked mon.noname-b con 0x7f4b2c0133d0 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:44.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.209+0000 7f4b46041640 10 monclient(hunting): picked mon.noname-f con 0x7f4b2c013ef0 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:44.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.209+0000 7f4b46041640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b40131400 con 0x7f4b2c013ef0 2024-01-24T23:25:44.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.209+0000 7f4b3effd640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f4b2c0133d0 legacy=0x7f4b401304f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:25:44.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.209+0000 7f4b46041640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b400a1310 con 0x7f4b2c0133d0 2024-01-24T23:25:44.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.209+0000 7f4b46041640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b40130c00 con 0x7f4b2c0122e0 2024-01-24T23:25:44.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.209+0000 7f4b46041640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.209+0000 7f4b3ffff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f4b2c013ef0 legacy=0x7f4b401cf840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.209+0000 7f4b46041640 10 monclient(hunting): authenticate will time out at 1223.135255s 2024-01-24T23:25:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.209+0000 7f4b3f7fe640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f4b2c0122e0 0x7f4b4012fde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.115:41976/0 (socket says 172.21.15.115:41976) 2024-01-24T23:25:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.209+0000 7f4b3f7fe640 1 -- 172.21.15.115:0/2588455549 learned_addr learned my addr 172.21.15.115:0/2588455549 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.209+0000 7f4b27fff640 1 -- 172.21.15.115:0/2588455549 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3950874631 0 0) 0x7f4b40130c00 con 0x7f4b2c0122e0 2024-01-24T23:25:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 10 monclient(hunting): my global_id is 5970 2024-01-24T23:25:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 1 -- 172.21.15.115:0/2588455549 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b2c0130a0 con 0x7f4b2c0122e0 2024-01-24T23:25:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 1 -- 172.21.15.115:0/2588455549 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 837310021 0 0) 0x7f4b2c020080 con 0x7f4b2c0122e0 2024-01-24T23:25:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 1 -- 172.21.15.115:0/2588455549 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4b40130c00 con 0x7f4b2c0122e0 2024-01-24T23:25:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 1 -- 172.21.15.115:0/2588455549 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4b30001640 con 0x7f4b2c0122e0 2024-01-24T23:25:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:25:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 10 monclient(hunting): dump: 2024-01-24T23:25:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:44.221 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:44.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 1 -- 172.21.15.115:0/2588455549 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1051765368 0 0) 0x7f4b40130c00 con 0x7f4b2c0122e0 2024-01-24T23:25:44.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 1 -- 172.21.15.115:0/2588455549 >> v1:172.21.15.115:6790/0 conn(0x7f4b2c0133d0 legacy=0x7f4b401304f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:44.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 1 -- 172.21.15.115:0/2588455549 >> v1:172.21.15.132:6789/0 conn(0x7f4b2c013ef0 legacy=0x7f4b401cf840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:44.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 10 monclient: _finish_hunting 0 2024-01-24T23:25:44.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 1 monclient: found mon.g 2024-01-24T23:25:44.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:44.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:44.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 1 -- 172.21.15.115:0/2588455549 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b401399c0 con 0x7f4b2c0122e0 2024-01-24T23:25:44.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 10 monclient: _finish_auth 0 2024-01-24T23:25:44.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 10 monclient: _check_auth_tickets 2024-01-24T23:25:44.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:44.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b46041640 5 monclient: authenticate success, global_id 5970 2024-01-24T23:25:44.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b46041640 10 monclient: _renew_subs 2024-01-24T23:25:44.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b46041640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:44.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b46041640 1 -- 172.21.15.115:0/2588455549 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b40132d60 con 0x7f4b2c0122e0 2024-01-24T23:25:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b46041640 10 monclient: _renew_subs 2024-01-24T23:25:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b46041640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b46041640 1 -- 172.21.15.115:0/2588455549 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b40138e30 con 0x7f4b2c0122e0 2024-01-24T23:25:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 1 -- 172.21.15.115:0/2588455549 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4b30001180 con 0x7f4b2c0122e0 2024-01-24T23:25:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 1 -- 172.21.15.115:0/2588455549 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4b300050d0 con 0x7f4b2c0122e0 2024-01-24T23:25:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b3dffb640 4 set_mon_vals no callback set 2024-01-24T23:25:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 10 monclient: dump: 2024-01-24T23:25:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:44.223 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.223 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:44.223 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:44.223 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:44.223 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:44.223 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:44.223 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:44.223 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:44.223 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:44.223 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:44.223 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b46041640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:44.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b46041640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:44.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b46041640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:44.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b46041640 1 -- 172.21.15.115:0/2588455549 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b20005090 con 0x7f4b2c0122e0 2024-01-24T23:25:44.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 1 -- 172.21.15.115:0/2588455549 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f4b3000fb50 con 0x7f4b2c0122e0 2024-01-24T23:25:44.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b27fff640 1 -- 172.21.15.115:0/2588455549 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(129..129 src has 1..129) v4 ==== 6377+0+0 (unknown 51197916 0 0) 0x7f4b300440e0 con 0x7f4b2c0122e0 2024-01-24T23:25:44.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.213+0000 7f4b3effd640 10 monclient: get_auth_request con 0x7f4b2c03d4a0 auth_method 0 2024-01-24T23:25:44.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.229+0000 7f4b27fff640 1 -- 172.21.15.115:0/2588455549 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f4b30014750 con 0x7f4b2c0122e0 2024-01-24T23:25:44.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.229+0000 7f4b27fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:44.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.229+0000 7f4b27fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:44.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.453+0000 7f4b46041640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-01-24T23:25:44.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.453+0000 7f4b46041640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-01-24T23:25:44.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.453+0000 7f4b46041640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:44.455 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.453+0000 7f4b46041640 1 -- 172.21.15.115:0/2588455549 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4b200058c0 con 0x7f4b2c0122e0 2024-01-24T23:25:44.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.453+0000 7f4b27fff640 1 -- 172.21.15.115:0/2588455549 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2726 (unknown 254746657 0 564577073) 0x7f4b30012df0 con 0x7f4b2c0122e0 2024-01-24T23:25:44.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.453+0000 7f4b27fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-01-24T23:25:44.456 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.453+0000 7f4b27fff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:44.456 INFO:teuthology.orchestra.run.smithi115.stdout:{"election_epoch":34,"quorum":[0,3,4,5,6,8],"quorum_names":["a","g","c","h","d","e"],"quorum_leader_name":"h","quorum_age":0,"features":{"quorum_con":"4540701272860131327","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","modified":"2024-01-24T23:20:20.557246Z","created":"2024-01-24T23:20:20.557246Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6790","nonce":0}]},"addr":"172.21.15.115:6790/0","public_addr":"172.21.15.115:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6790","nonce":0}]},"addr":"172.21.15.132:6790/0","public_addr":"172.21.15.132:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6791","nonce":0}]},"addr":"172.21.15.115:6791/0","public_addr":"172.21.15.115:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6791","nonce":0}]},"addr":"172.21.15.132:6791/0","public_addr":"172.21.15.132:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6792","nonce":0}]},"addr":"172.21.15.115:6792/0","public_addr":"172.21.15.115:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6792","nonce":0}]},"addr":"172.21.15.132:6792/0","public_addr":"172.21.15.132:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6793","nonce":0}]},"addr":"172.21.15.115:6793/0","public_addr":"172.21.15.115:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-01-24T23:25:44.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.457+0000 7f4b46041640 1 -- 172.21.15.115:0/2588455549 >> v1:172.21.15.132:6812/1835401922 conn(0x7f4b2c03d4a0 legacy=0x7f4b2c03f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:44.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.457+0000 7f4b46041640 10 monclient: shutdown 2024-01-24T23:25:44.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.457+0000 7f4b46041640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:44.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.457+0000 7f4b46041640 1 -- 172.21.15.115:0/2588455549 >> v1:172.21.15.132:6790/0 conn(0x7f4b2c0122e0 legacy=0x7f4b4012fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:44.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.457+0000 7f4b46041640 1 -- 172.21.15.115:0/2588455549 shutdown_connections 2024-01-24T23:25:44.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.457+0000 7f4b46041640 1 -- 172.21.15.115:0/2588455549 >> 172.21.15.115:0/2588455549 conn(0x7f4b400a5730 msgr2=0x7f4b400a3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:44.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.457+0000 7f4b46041640 1 -- 172.21.15.115:0/2588455549 shutdown_connections 2024-01-24T23:25:44.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.457+0000 7f4b46041640 1 -- 172.21.15.115:0/2588455549 wait complete. 2024-01-24T23:25:44.475 INFO:tasks.mon_thrash.ceph_manager:quorum is size 6 2024-01-24T23:25:44.475 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon a 2024-01-24T23:25:44.475 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-01-24T23:25:44.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57abb7d640 10 monclient: get_monmap_and_config 2024-01-24T23:25:44.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57abb7d640 10 monclient: build_initial_monmap 2024-01-24T23:25:44.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57abb7d640 10 monclient: monmap: 2024-01-24T23:25:44.596 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:44.596 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.596 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:44.596269+0000 2024-01-24T23:25:44.596 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:44.596269+0000 2024-01-24T23:25:44.596 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:44.596 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.596 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:44.596 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:44.596 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:44.596 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:44.596 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:44.596 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:44.596 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:44.596 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:44.597 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:44.597 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57abb7d640 1 Processor -- start 2024-01-24T23:25:44.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57abb7d640 1 -- start start 2024-01-24T23:25:44.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57abb7d640 10 monclient: init 2024-01-24T23:25:44.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57abb7d640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:44.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57abb7d640 10 monclient: _add_conns ranks=[0,3,1,7,5,8,4,6,2] 2024-01-24T23:25:44.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57abb7d640 10 monclient(hunting): picked mon.noname-a con 0x7f57a409f8f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:44.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57abb7d640 10 monclient(hunting): picked mon.noname-g con 0x7f57a409dab0 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:44.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57abb7d640 10 monclient(hunting): picked mon.noname-f con 0x7f57a40a03e0 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:44.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57abb7d640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a404ed70 con 0x7f57a409f8f0 2024-01-24T23:25:44.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57abb7d640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a409d0d0 con 0x7f57a40a03e0 2024-01-24T23:25:44.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57abb7d640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a40a26b0 con 0x7f57a409dab0 2024-01-24T23:25:44.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57abb7d640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:44.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57abb7d640 10 monclient(hunting): authenticate will time out at 1223.518925s 2024-01-24T23:25:44.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57aa11a640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f57a40a03e0 legacy=0x7f57a4143ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:44.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a9919640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f57a409f8f0 0x7f57a409fcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:53132/0 (socket says 172.21.15.115:53132) 2024-01-24T23:25:44.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a9919640 1 -- 172.21.15.115:0/515622098 learned_addr learned my addr 172.21.15.115:0/515622098 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:44.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 1 -- 172.21.15.115:0/515622098 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1383046751 0 0) 0x7f57a404ed70 con 0x7f57a409f8f0 2024-01-24T23:25:44.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 10 monclient(hunting): _negotiate 2024-01-24T23:25:44.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:44.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:44.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 10 monclient(hunting): my global_id is 94320 2024-01-24T23:25:44.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 1 -- 172.21.15.115:0/515622098 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5790003530 con 0x7f57a409f8f0 2024-01-24T23:25:44.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 1 -- 172.21.15.115:0/515622098 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3461210287 0 0) 0x7f5790003530 con 0x7f57a409f8f0 2024-01-24T23:25:44.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 1 -- 172.21.15.115:0/515622098 >> v1:172.21.15.132:6790/0 conn(0x7f57a409dab0 legacy=0x7f57a4140380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:44.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 1 -- 172.21.15.115:0/515622098 >> v1:172.21.15.132:6789/0 conn(0x7f57a40a03e0 legacy=0x7f57a4143ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:44.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 10 monclient: _finish_hunting 0 2024-01-24T23:25:44.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 1 monclient: found mon.noname-a 2024-01-24T23:25:44.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:44.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:25:44.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 1 -- 172.21.15.115:0/515622098 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a409c830 con 0x7f57a409f8f0 2024-01-24T23:25:44.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 10 monclient: _finish_auth 0 2024-01-24T23:25:44.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 10 monclient: _check_auth_tickets 2024-01-24T23:25:44.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:44.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 1 -- 172.21.15.115:0/515622098 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f57980017c0 con 0x7f57a409f8f0 2024-01-24T23:25:44.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:44.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:25:44.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 10 monclient: dump: 2024-01-24T23:25:44.607 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:44.607 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.607 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.607 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.607 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:44.607 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.607 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:44.607 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:44.607 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:44.607 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:44.607 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:44.607 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:44.607 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:44.607 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:44.608 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:44.608 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57abb7d640 5 monclient: authenticate success, global_id 94320 2024-01-24T23:25:44.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57abb7d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:44.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 1 -- 172.21.15.115:0/515622098 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5798001ae0 con 0x7f57a409f8f0 2024-01-24T23:25:44.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:44.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57abb7d640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:44.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57abb7d640 4 set_mon_vals no callback set 2024-01-24T23:25:44.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 1 -- 172.21.15.115:0/515622098 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5798004bf0 con 0x7f57a409f8f0 2024-01-24T23:25:44.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:44.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:44.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.593+0000 7f57a8917640 10 monclient: dump: 2024-01-24T23:25:44.608 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:44.608 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.609 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.609 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.609 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:44.609 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.609 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:44.609 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:44.609 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:44.609 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:44.609 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:44.609 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:44.609 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:44.609 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:44.609 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:44.609 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 10 monclient: shutdown 2024-01-24T23:25:44.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:44.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 1 -- 172.21.15.115:0/515622098 >> v1:172.21.15.115:6789/0 conn(0x7f57a409f8f0 legacy=0x7f57a409fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:44.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 1 -- 172.21.15.115:0/515622098 shutdown_connections 2024-01-24T23:25:44.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 1 -- 172.21.15.115:0/515622098 >> 172.21.15.115:0/515622098 conn(0x7f57a40a5730 msgr2=0x7f57a40a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:44.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 1 -- 172.21.15.115:0/515622098 shutdown_connections 2024-01-24T23:25:44.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 1 -- 172.21.15.115:0/515622098 wait complete. 2024-01-24T23:25:44.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 10 monclient: build_initial_monmap 2024-01-24T23:25:44.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 10 monclient: monmap: 2024-01-24T23:25:44.611 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:44.611 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:44.611 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:44.612 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:44.612 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:44.612 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:44.612 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:44.612 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:44.612 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:44.612 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:44.612 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:44.612 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:44.612 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:44.612 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:44.612 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:44.612 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 1 Processor -- start 2024-01-24T23:25:44.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 1 -- start start 2024-01-24T23:25:44.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 10 monclient: init 2024-01-24T23:25:44.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:44.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 10 monclient: _add_conns ranks=[2,8,6,4,3,5,0,7,1] 2024-01-24T23:25:44.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 10 monclient(hunting): picked mon.noname-b con 0x7f57a409dab0 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:44.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 10 monclient(hunting): picked mon.noname-e con 0x7f57a409f8f0 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:44.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 10 monclient(hunting): picked mon.noname-d con 0x7f57a40a03e0 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:44.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a41b06f0 con 0x7f57a409dab0 2024-01-24T23:25:44.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a41af240 con 0x7f57a40a03e0 2024-01-24T23:25:44.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a40a0ff0 con 0x7f57a409f8f0 2024-01-24T23:25:44.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:44.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 10 monclient(hunting): authenticate will time out at 1223.522224s 2024-01-24T23:25:44.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57a9919640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f57a409dab0 legacy=0x7f57a41c3010 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:25:44.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57aa11a640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f57a40a03e0 0x7f57a41d7ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:35016/0 (socket says 172.21.15.115:35016) 2024-01-24T23:25:44.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57aa11a640 1 -- 172.21.15.115:0/4249780710 learned_addr learned my addr 172.21.15.115:0/4249780710 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:44.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1360150379 0 0) 0x7f57a41af240 con 0x7f57a40a03e0 2024-01-24T23:25:44.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:44.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:44.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:44.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 10 monclient(hunting): my global_id is 6153 2024-01-24T23:25:44.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5788003530 con 0x7f57a40a03e0 2024-01-24T23:25:44.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4206859397 0 0) 0x7f57a40a0ff0 con 0x7f57a409f8f0 2024-01-24T23:25:44.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:44.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:44.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:44.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 10 monclient(hunting): my global_id is 85229 2024-01-24T23:25:44.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57a41af240 con 0x7f57a409f8f0 2024-01-24T23:25:44.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2382718822 0 0) 0x7f5788003530 con 0x7f57a40a03e0 2024-01-24T23:25:44.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f57a40a0ff0 con 0x7f57a40a03e0 2024-01-24T23:25:44.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 632028207 0 0) 0x7f57a41af240 con 0x7f57a409f8f0 2024-01-24T23:25:44.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5788003530 con 0x7f57a409f8f0 2024-01-24T23:25:44.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f579c0045f0 con 0x7f57a40a03e0 2024-01-24T23:25:44.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:44.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:25:44.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 10 monclient(hunting): dump: 2024-01-24T23:25:44.615 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:44.615 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.615 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.615 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.615 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:44.615 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.615 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:44.615 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:44.615 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:44.615 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:44.615 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:44.616 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:44.616 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:44.616 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:44.616 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:44.616 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:44.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f57a00019a0 con 0x7f57a409f8f0 2024-01-24T23:25:44.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:44.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:44.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 10 monclient(hunting): dump: 2024-01-24T23:25:44.616 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:44.616 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.616 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.616 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.616 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:44.617 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.617 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:44.617 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:44.617 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:44.617 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:44.617 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:44.617 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:44.617 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:44.617 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:44.617 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:44.617 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3690708067 0 0) 0x7f57a40a0ff0 con 0x7f57a40a03e0 2024-01-24T23:25:44.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 >> v1:172.21.15.115:6793/0 conn(0x7f57a409f8f0 legacy=0x7f57a41d47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:44.617 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 >> v1:172.21.15.115:6790/0 conn(0x7f57a409dab0 legacy=0x7f57a41c3010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:44.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 10 monclient: _finish_hunting 0 2024-01-24T23:25:44.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 1 monclient: found mon.d 2024-01-24T23:25:44.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:44.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:25:44.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a4071de0 con 0x7f57a40a03e0 2024-01-24T23:25:44.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 10 monclient: _finish_auth 0 2024-01-24T23:25:44.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 10 monclient: _check_auth_tickets 2024-01-24T23:25:44.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:44.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 5 monclient: authenticate success, global_id 6153 2024-01-24T23:25:44.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 10 monclient: _renew_subs 2024-01-24T23:25:44.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:25:44.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 1 -- 172.21.15.115:0/4249780710 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a41b0410 con 0x7f57a40a03e0 2024-01-24T23:25:44.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 10 monclient: _renew_subs 2024-01-24T23:25:44.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:25:44.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 1 -- 172.21.15.115:0/4249780710 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a41c3720 con 0x7f57a40a03e0 2024-01-24T23:25:44.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f579c004850 con 0x7f57a40a03e0 2024-01-24T23:25:44.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:44.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f579c004d80 con 0x7f57a40a03e0 2024-01-24T23:25:44.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:44.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f5797fff640 4 set_mon_vals no callback set 2024-01-24T23:25:44.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:44.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57967fc640 10 monclient: dump: 2024-01-24T23:25:44.619 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:44.619 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.619 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.619 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.619 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:44.620 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.620 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:44.620 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:44.620 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:44.620 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:44.620 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:44.620 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:44.620 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:44.620 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:44.620 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:44.620 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:44.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.597+0000 7f57abb7d640 1 -- 172.21.15.115:0/4249780710 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f578c0065d0 con 0x7f578c002870 2024-01-24T23:25:44.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.601+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f579c004ee0 con 0x7f57a40a03e0 2024-01-24T23:25:44.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.601+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(129..129 src has 1..129) v4 ==== 6377+0+0 (unknown 51197916 0 0) 0x7f579c0443b0 con 0x7f57a40a03e0 2024-01-24T23:25:44.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.601+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1552576425 0 0) 0x7f578c0065d0 con 0x7f578c002870 2024-01-24T23:25:44.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.601+0000 7f57967fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:44.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.601+0000 7f57967fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:44.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.601+0000 7f57967fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:44.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.601+0000 7f57967fc640 10 monclient(hunting): my global_id is 94329 2024-01-24T23:25:44.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.601+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57a40a0ff0 con 0x7f578c002870 2024-01-24T23:25:44.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.601+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2651413200 0 0) 0x7f57a40a0ff0 con 0x7f578c002870 2024-01-24T23:25:44.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.601+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f578c0065d0 con 0x7f578c002870 2024-01-24T23:25:44.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.601+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5798004bf0 con 0x7f578c002870 2024-01-24T23:25:44.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.601+0000 7f57967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:44.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.601+0000 7f57967fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:44.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.601+0000 7f57967fc640 10 monclient: dump: 2024-01-24T23:25:44.621 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:44.621 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.621 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.621 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.622 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:44.622 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.622 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:44.622 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:44.622 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:44.622 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:44.622 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:44.622 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:44.622 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:44.622 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:44.622 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:44.622 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.601+0000 7f57a9118640 10 monclient: get_auth_request con 0x7f5788037740 auth_method 0 2024-01-24T23:25:44.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.601+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 303997393 0 0) 0x7f578c0065d0 con 0x7f578c002870 2024-01-24T23:25:44.622 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.601+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 --> v1:172.21.15.115:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f578c00a120 con 0x7f578c002870 2024-01-24T23:25:44.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.601+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 <== mon.0 v1:172.21.15.115:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f5798001670 con 0x7f578c002870 2024-01-24T23:25:44.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.601+0000 7f57967fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:44.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.601+0000 7f57967fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:44.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.601+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 >> v1:172.21.15.115:6789/0 conn(0x7f578c002870 legacy=0x7f578c004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:44.623 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.601+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 >> v1:172.21.15.115:6789/0 conn(0x7f578c002870 legacy=0x7f578c004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:44.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.605+0000 7f57abb7d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:25:44.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.605+0000 7f57abb7d640 1 -- 172.21.15.115:0/4249780710 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f578c009dd0 con 0x7f578c0090b0 2024-01-24T23:25:44.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.605+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1791752626 0 0) 0x7f578c009dd0 con 0x7f578c0090b0 2024-01-24T23:25:44.624 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.605+0000 7f57967fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:44.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.605+0000 7f57967fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:44.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.605+0000 7f57967fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:44.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.605+0000 7f57967fc640 10 monclient(hunting): my global_id is 94338 2024-01-24T23:25:44.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.605+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f578c0065d0 con 0x7f578c0090b0 2024-01-24T23:25:44.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.605+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1806575824 0 0) 0x7f578c0065d0 con 0x7f578c0090b0 2024-01-24T23:25:44.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.605+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f578c009dd0 con 0x7f578c0090b0 2024-01-24T23:25:44.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.605+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f57980011e0 con 0x7f578c0090b0 2024-01-24T23:25:44.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.605+0000 7f57967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:44.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.605+0000 7f57967fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:44.625 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.605+0000 7f57967fc640 10 monclient: dump: 2024-01-24T23:25:44.625 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:44.625 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.625 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.625 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.626 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:44.626 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.626 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:44.626 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:44.626 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:44.626 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:44.626 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:44.626 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:44.626 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:44.626 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:44.626 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:44.626 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.605+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 391713167 0 0) 0x7f578c009dd0 con 0x7f578c0090b0 2024-01-24T23:25:44.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.605+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 --> v1:172.21.15.115:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f578c00c2d0 con 0x7f578c0090b0 2024-01-24T23:25:44.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.609+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 <== mon.0 v1:172.21.15.115:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7510 (unknown 0 0 3017967824) 0x7f5798007670 con 0x7f578c0090b0 2024-01-24T23:25:44.626 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.609+0000 7f57967fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:25:44.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.609+0000 7f57967fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:44.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.609+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 >> v1:172.21.15.115:6789/0 conn(0x7f578c0090b0 legacy=0x7f578c009520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:44.627 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.609+0000 7f57967fc640 1 -- 172.21.15.115:0/4249780710 >> v1:172.21.15.115:6789/0 conn(0x7f578c0090b0 legacy=0x7f578c009520 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:44.627 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:25:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:25:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 34, 2024-01-24T23:25:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:25:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:25:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:25:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:25:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:25:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:25:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:25:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:25:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:25:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:25:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:25:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:25:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:25:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:25:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:25:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:25:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:25:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:25:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:25:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:25:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:25:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:25:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:25:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:25:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:25:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:25:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:25:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:25:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:25:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:25:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:25:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:25:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:25:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:25:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:25:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:25:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:25:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:25:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:25:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:25:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:25:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:25:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:25:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:25:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:25:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:25:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:25:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:25:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:25:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:25:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:25:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:25:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:25:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:25:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:25:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:25:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:25:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:25:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:25:44.645 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:25:44.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.609+0000 7f57abb7d640 1 -- 172.21.15.115:0/4249780710 >> v1:172.21.15.132:6812/1835401922 conn(0x7f5788037740 legacy=0x7f5788039c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:44.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.609+0000 7f57abb7d640 10 monclient: shutdown 2024-01-24T23:25:44.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.609+0000 7f57abb7d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:44.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.609+0000 7f57abb7d640 1 -- 172.21.15.115:0/4249780710 >> v1:172.21.15.115:6792/0 conn(0x7f57a40a03e0 legacy=0x7f57a41d7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:44.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.609+0000 7f57aa11a640 1 -- 172.21.15.115:0/4249780710 reap_dead start 2024-01-24T23:25:44.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.609+0000 7f57aa11a640 1 -- 172.21.15.115:0/4249780710 reap_dead start 2024-01-24T23:25:44.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.609+0000 7f57abb7d640 1 -- 172.21.15.115:0/4249780710 shutdown_connections 2024-01-24T23:25:44.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.609+0000 7f57abb7d640 1 -- 172.21.15.115:0/4249780710 >> 172.21.15.115:0/4249780710 conn(0x7f57a40a5730 msgr2=0x7f57a40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:44.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.609+0000 7f57abb7d640 1 -- 172.21.15.115:0/4249780710 shutdown_connections 2024-01-24T23:25:44.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.609+0000 7f57abb7d640 1 -- 172.21.15.115:0/4249780710 wait complete. 2024-01-24T23:25:44.647 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-01-24T23:25:44.647 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon c 2024-01-24T23:25:44.648 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-01-24T23:25:44.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.761+0000 7f9e1d92c640 10 monclient: get_monmap_and_config 2024-01-24T23:25:44.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.761+0000 7f9e1d92c640 10 monclient: build_initial_monmap 2024-01-24T23:25:44.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.761+0000 7f9e1d92c640 10 monclient: monmap: 2024-01-24T23:25:44.766 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:44.766 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.766 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:44.766752+0000 2024-01-24T23:25:44.766 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:44.766752+0000 2024-01-24T23:25:44.767 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:44.767 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.767 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:44.767 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:44.767 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:44.767 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:44.767 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:44.767 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:44.767 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:44.767 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:44.767 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:44.767 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 1 Processor -- start 2024-01-24T23:25:44.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 1 -- start start 2024-01-24T23:25:44.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 10 monclient: init 2024-01-24T23:25:44.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:44.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 10 monclient: _add_conns ranks=[6,8,5,2,3,4,0,7,1] 2024-01-24T23:25:44.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 10 monclient(hunting): picked mon.noname-d con 0x7f9e18131810 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:44.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 10 monclient(hunting): picked mon.noname-e con 0x7f9e181311d0 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:44.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 10 monclient(hunting): picked mon.noname-h con 0x7f9e1806d960 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:44.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1804ed70 con 0x7f9e1806d960 2024-01-24T23:25:44.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1806e1f0 con 0x7f9e18131810 2024-01-24T23:25:44.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1809fef0 con 0x7f9e181311d0 2024-01-24T23:25:44.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:44.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 10 monclient(hunting): authenticate will time out at 1223.689351s 2024-01-24T23:25:44.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e167fc640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f9e181311d0 0x7f9e180a4d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:54942/0 (socket says 172.21.15.115:54942) 2024-01-24T23:25:44.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e16ffd640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f9e18131810 0x7f9e180a4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:35032/0 (socket says 172.21.15.115:35032) 2024-01-24T23:25:44.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e167fc640 1 -- 172.21.15.115:0/3192854994 learned_addr learned my addr 172.21.15.115:0/3192854994 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:44.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 1 -- 172.21.15.115:0/3192854994 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2045240732 0 0) 0x7f9e1809fef0 con 0x7f9e181311d0 2024-01-24T23:25:44.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:44.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:44.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:44.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 10 monclient(hunting): my global_id is 85238 2024-01-24T23:25:44.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 1 -- 172.21.15.115:0/3192854994 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e00003530 con 0x7f9e181311d0 2024-01-24T23:25:44.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 1 -- 172.21.15.115:0/3192854994 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2781543962 0 0) 0x7f9e1806e1f0 con 0x7f9e18131810 2024-01-24T23:25:44.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:44.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:44.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:44.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 10 monclient(hunting): my global_id is 6162 2024-01-24T23:25:44.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 1 -- 172.21.15.115:0/3192854994 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e1809fef0 con 0x7f9e18131810 2024-01-24T23:25:44.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 1 -- 172.21.15.115:0/3192854994 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3118919614 0 0) 0x7f9e00003530 con 0x7f9e181311d0 2024-01-24T23:25:44.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 1 -- 172.21.15.115:0/3192854994 >> v1:172.21.15.115:6792/0 conn(0x7f9e18131810 legacy=0x7f9e180a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:44.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 1 -- 172.21.15.115:0/3192854994 >> v1:172.21.15.132:6791/0 conn(0x7f9e1806d960 legacy=0x7f9e1813da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:44.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:25:44.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 1 monclient: found mon.noname-e 2024-01-24T23:25:44.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:44.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:25:44.778 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 1 -- 172.21.15.115:0/3192854994 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e180a54b0 con 0x7f9e181311d0 2024-01-24T23:25:44.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 10 monclient: _finish_auth 0 2024-01-24T23:25:44.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:25:44.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:44.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 1 -- 172.21.15.115:0/3192854994 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9e0c0017c0 con 0x7f9e181311d0 2024-01-24T23:25:44.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 5 monclient: authenticate success, global_id 85238 2024-01-24T23:25:44.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:44.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:44.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:25:44.779 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 10 monclient: dump: 2024-01-24T23:25:44.779 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:44.779 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.779 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.779 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.780 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:44.780 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.780 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:44.780 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:44.780 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:44.780 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:44.780 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:44.780 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:44.780 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:44.780 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:44.780 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:44.780 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.780 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:44.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 1 -- 172.21.15.115:0/3192854994 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9e0c001a60 con 0x7f9e181311d0 2024-01-24T23:25:44.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:44.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:44.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 1 -- 172.21.15.115:0/3192854994 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9e0c004bf0 con 0x7f9e181311d0 2024-01-24T23:25:44.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 4 set_mon_vals no callback set 2024-01-24T23:25:44.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:44.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:44.781 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e15ffb640 10 monclient: dump: 2024-01-24T23:25:44.781 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:44.781 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.781 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.781 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.781 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:44.781 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.781 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:44.782 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:44.782 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:44.782 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:44.782 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:44.782 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:44.782 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:44.782 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:44.782 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:44.782 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 10 monclient: shutdown 2024-01-24T23:25:44.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:44.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 1 -- 172.21.15.115:0/3192854994 >> v1:172.21.15.115:6793/0 conn(0x7f9e181311d0 legacy=0x7f9e180a4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:44.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 1 -- 172.21.15.115:0/3192854994 shutdown_connections 2024-01-24T23:25:44.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 1 -- 172.21.15.115:0/3192854994 >> 172.21.15.115:0/3192854994 conn(0x7f9e180a5d40 msgr2=0x7f9e180a2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:44.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 1 -- 172.21.15.115:0/3192854994 shutdown_connections 2024-01-24T23:25:44.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 1 -- 172.21.15.115:0/3192854994 wait complete. 2024-01-24T23:25:44.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 10 monclient: build_initial_monmap 2024-01-24T23:25:44.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 10 monclient: monmap: 2024-01-24T23:25:44.784 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:44.784 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:44.784 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:44.784 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:44.784 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:44.785 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:44.785 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:44.785 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:44.785 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:44.785 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:44.785 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:44.785 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:44.785 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:44.785 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:44.785 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:44.785 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 1 Processor -- start 2024-01-24T23:25:44.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 1 -- start start 2024-01-24T23:25:44.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.765+0000 7f9e1d92c640 10 monclient: init 2024-01-24T23:25:44.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9e1d92c640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:44.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9e1d92c640 10 monclient: _add_conns ranks=[3,4,1,0,8,7,2,5,6] 2024-01-24T23:25:44.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9e1d92c640 10 monclient(hunting): picked mon.noname-g con 0x7f9e1806d960 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:44.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9e1d92c640 10 monclient(hunting): picked mon.noname-c con 0x7f9e181311d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:44.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9e1d92c640 10 monclient(hunting): picked mon.noname-f con 0x7f9e18131810 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:44.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9e1d92c640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1806de30 con 0x7f9e18131810 2024-01-24T23:25:44.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9e1d92c640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1809c6c0 con 0x7f9e1806d960 2024-01-24T23:25:44.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9e1d92c640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1809f7d0 con 0x7f9e181311d0 2024-01-24T23:25:44.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9e1d92c640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:44.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9e1d92c640 10 monclient(hunting): authenticate will time out at 1223.692480s 2024-01-24T23:25:44.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9e167fc640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f9e181311d0 0x7f9e1806ecf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:41828/0 (socket says 172.21.15.115:41828) 2024-01-24T23:25:44.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9e167fc640 1 -- 172.21.15.115:0/946975003 learned_addr learned my addr 172.21.15.115:0/946975003 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:44.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9e177fe640 1 -- 172.21.15.115:0/946975003 >> v1:172.21.15.132:6789/0 conn(0x7f9e18131810 legacy=0x7f9e181d3c10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:44.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 643458575 0 0) 0x7f9e1809f7d0 con 0x7f9e181311d0 2024-01-24T23:25:44.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:44.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:44.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:44.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient(hunting): my global_id is 94315 2024-01-24T23:25:44.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e04003530 con 0x7f9e181311d0 2024-01-24T23:25:44.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1316557571 0 0) 0x7f9e04003530 con 0x7f9e181311d0 2024-01-24T23:25:44.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9e1809f7d0 con 0x7f9e181311d0 2024-01-24T23:25:44.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9e0c004b30 con 0x7f9e181311d0 2024-01-24T23:25:44.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:44.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:25:44.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient(hunting): dump: 2024-01-24T23:25:44.788 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:44.789 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.789 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.789 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.789 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:44.789 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.789 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:44.789 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:44.789 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:44.789 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:44.789 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:44.789 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:44.790 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:44.790 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:44.790 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:44.790 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:44.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 271826305 0 0) 0x7f9e1809c6c0 con 0x7f9e1806d960 2024-01-24T23:25:44.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:44.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:44.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:44.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient(hunting): my global_id is 5988 2024-01-24T23:25:44.790 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e04003530 con 0x7f9e1806d960 2024-01-24T23:25:44.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3784978279 0 0) 0x7f9e1809f7d0 con 0x7f9e181311d0 2024-01-24T23:25:44.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 >> v1:172.21.15.132:6790/0 conn(0x7f9e1806d960 legacy=0x7f9e1806e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:44.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 >> v1:172.21.15.132:6789/0 conn(0x7f9e18131810 legacy=0x7f9e181d3c10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:44.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient: _finish_hunting 0 2024-01-24T23:25:44.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 monclient: found mon.c 2024-01-24T23:25:44.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:44.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:44.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e18070210 con 0x7f9e181311d0 2024-01-24T23:25:44.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient: _finish_auth 0 2024-01-24T23:25:44.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient: _check_auth_tickets 2024-01-24T23:25:44.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:44.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9e1d92c640 5 monclient: authenticate success, global_id 94315 2024-01-24T23:25:44.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9e1d92c640 10 monclient: _renew_subs 2024-01-24T23:25:44.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9e1d92c640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:44.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9e1d92c640 1 -- 172.21.15.115:0/946975003 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e1806f400 con 0x7f9e181311d0 2024-01-24T23:25:44.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9e1d92c640 10 monclient: _renew_subs 2024-01-24T23:25:44.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9e1d92c640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:44.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9e1d92c640 1 -- 172.21.15.115:0/946975003 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e1809cff0 con 0x7f9e181311d0 2024-01-24T23:25:44.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9e0c004f00 con 0x7f9e181311d0 2024-01-24T23:25:44.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:44.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9e0c0053b0 con 0x7f9e181311d0 2024-01-24T23:25:44.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:44.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:44.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9e157fa640 4 set_mon_vals no callback set 2024-01-24T23:25:44.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient: dump: 2024-01-24T23:25:44.793 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:44.793 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.794 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.794 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.794 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:44.794 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.794 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:44.794 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:44.795 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:44.795 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:44.795 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:44.795 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:44.795 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:44.795 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:44.796 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:44.796 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9e1d92c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:44.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9e1d92c640 1 -- 172.21.15.115:0/946975003 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9df80065d0 con 0x7f9df8002870 2024-01-24T23:25:44.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 513758903 0 0) 0x7f9df80065d0 con 0x7f9df8002870 2024-01-24T23:25:44.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:44.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:44.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:44.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient(hunting): my global_id is 94324 2024-01-24T23:25:44.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e1809f7d0 con 0x7f9df8002870 2024-01-24T23:25:44.796 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f9e0c00fe30 con 0x7f9e181311d0 2024-01-24T23:25:44.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(129..129 src has 1..129) v4 ==== 6377+0+0 (unknown 51197916 0 0) 0x7f9e0c0443a0 con 0x7f9e181311d0 2024-01-24T23:25:44.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4029645696 0 0) 0x7f9e1809f7d0 con 0x7f9df8002870 2024-01-24T23:25:44.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9df80065d0 con 0x7f9df8002870 2024-01-24T23:25:44.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9e08001790 con 0x7f9df8002870 2024-01-24T23:25:44.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:44.797 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient: dump: 2024-01-24T23:25:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:44.799 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:44.799 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:44.799 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:44.799 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:44.799 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:44.799 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:44.799 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:44.799 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:44.799 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2388219858 0 0) 0x7f9df80065d0 con 0x7f9df8002870 2024-01-24T23:25:44.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 --> v1:172.21.15.115:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9df800a120 con 0x7f9df8002870 2024-01-24T23:25:44.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9e177fe640 10 monclient: get_auth_request con 0x7f9e04036fd0 auth_method 0 2024-01-24T23:25:44.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 <== mon.4 v1:172.21.15.115:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f9e08001d00 con 0x7f9df8002870 2024-01-24T23:25:44.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:44.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:44.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 >> v1:172.21.15.115:6791/0 conn(0x7f9df8002870 legacy=0x7f9df8004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:44.800 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.769+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 >> v1:172.21.15.115:6791/0 conn(0x7f9df8002870 legacy=0x7f9df8004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:44.801 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9e1d92c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:25:44.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9e1d92c640 1 -- 172.21.15.115:0/946975003 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9df8009df0 con 0x7f9df80090b0 2024-01-24T23:25:44.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1973859306 0 0) 0x7f9df8009df0 con 0x7f9df80090b0 2024-01-24T23:25:44.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9dfffff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:44.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9dfffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:44.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9dfffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:44.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9dfffff640 10 monclient(hunting): my global_id is 94333 2024-01-24T23:25:44.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9df80065d0 con 0x7f9df80090b0 2024-01-24T23:25:44.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1836975493 0 0) 0x7f9df80065d0 con 0x7f9df80090b0 2024-01-24T23:25:44.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9df8009df0 con 0x7f9df80090b0 2024-01-24T23:25:44.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9e08004df0 con 0x7f9df80090b0 2024-01-24T23:25:44.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9dfffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:44.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9dfffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:44.802 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9dfffff640 10 monclient: dump: 2024-01-24T23:25:44.802 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:44.803 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.803 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.803 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.803 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:44.803 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.803 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:44.803 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:44.803 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:44.803 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:44.803 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:44.803 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:44.803 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:44.803 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:44.803 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:44.804 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4283356700 0 0) 0x7f9df8009df0 con 0x7f9df80090b0 2024-01-24T23:25:44.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 --> v1:172.21.15.115:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9df800c310 con 0x7f9df80090b0 2024-01-24T23:25:44.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 <== mon.4 v1:172.21.15.115:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7850 (unknown 0 0 3685264743) 0x7f9e08007130 con 0x7f9df80090b0 2024-01-24T23:25:44.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9dfffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:25:44.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9dfffff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:44.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 >> v1:172.21.15.115:6791/0 conn(0x7f9df80090b0 legacy=0x7f9df8009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:44.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9dfffff640 1 -- 172.21.15.115:0/946975003 >> v1:172.21.15.115:6791/0 conn(0x7f9df80090b0 legacy=0x7f9df8009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:44.804 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:25:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:25:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 34, 2024-01-24T23:25:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:25:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:25:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:25:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:25:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:25:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:25:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:25:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:25:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:25:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:25:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:25:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:25:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:25:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:25:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:25:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:25:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:25:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:25:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:25:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:25:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:25:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:25:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:25:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:25:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:25:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:25:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:25:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:25:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:25:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:25:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:25:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:25:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:25:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:25:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:25:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:25:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:25:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:25:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:25:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:25:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:25:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:25:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:25:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:25:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:25:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:25:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:25:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:25:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:25:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:25:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:25:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:25:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:25:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:25:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:25:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:25:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:25:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:25:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 3 2024-01-24T23:25:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:25:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:25:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": [ 2024-01-24T23:25:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:25:44.827 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:25:44.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9e1d92c640 1 -- 172.21.15.115:0/946975003 >> v1:172.21.15.132:6812/1835401922 conn(0x7f9e04036fd0 legacy=0x7f9e04039490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:44.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9e1d92c640 10 monclient: shutdown 2024-01-24T23:25:44.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9e1d92c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:44.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9e1d92c640 1 -- 172.21.15.115:0/946975003 >> v1:172.21.15.115:6791/0 conn(0x7f9e181311d0 legacy=0x7f9e1806ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:44.828 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9e177fe640 1 -- 172.21.15.115:0/946975003 reap_dead start 2024-01-24T23:25:44.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9e177fe640 1 -- 172.21.15.115:0/946975003 reap_dead start 2024-01-24T23:25:44.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9e1d92c640 1 -- 172.21.15.115:0/946975003 shutdown_connections 2024-01-24T23:25:44.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9e1d92c640 1 -- 172.21.15.115:0/946975003 >> 172.21.15.115:0/946975003 conn(0x7f9e180a5d40 msgr2=0x7f9e180a1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:44.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9e1d92c640 1 -- 172.21.15.115:0/946975003 shutdown_connections 2024-01-24T23:25:44.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.777+0000 7f9e1d92c640 1 -- 172.21.15.115:0/946975003 wait complete. 2024-01-24T23:25:44.829 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-01-24T23:25:44.829 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon d 2024-01-24T23:25:44.829 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-01-24T23:25:44.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313b700640 10 monclient: get_monmap_and_config 2024-01-24T23:25:44.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313b700640 10 monclient: build_initial_monmap 2024-01-24T23:25:44.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313b700640 10 monclient: monmap: 2024-01-24T23:25:44.951 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:44.951 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.951 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:44.951443+0000 2024-01-24T23:25:44.952 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:44.951443+0000 2024-01-24T23:25:44.952 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:44.952 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.952 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:44.952 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:44.952 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:44.952 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:44.952 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:44.952 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:44.952 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:44.953 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:44.953 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:44.953 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313b700640 1 Processor -- start 2024-01-24T23:25:44.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313b700640 1 -- start start 2024-01-24T23:25:44.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313b700640 10 monclient: init 2024-01-24T23:25:44.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313b700640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:44.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313b700640 10 monclient: _add_conns ranks=[6,7,2,4,5,3,1,0,8] 2024-01-24T23:25:44.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313b700640 10 monclient(hunting): picked mon.noname-d con 0x7f3134133bf0 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:44.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313b700640 10 monclient(hunting): picked mon.noname-i con 0x7f3134133fd0 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:44.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313b700640 10 monclient(hunting): picked mon.noname-b con 0x7f3134134430 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:44.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313b700640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f313404ed70 con 0x7f3134134430 2024-01-24T23:25:44.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313b700640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f313409af30 con 0x7f3134133bf0 2024-01-24T23:25:44.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313b700640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f313409b210 con 0x7f3134133fd0 2024-01-24T23:25:44.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313b700640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:44.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313b700640 10 monclient(hunting): authenticate will time out at 1223.874643s 2024-01-24T23:25:44.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f3139c9d640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f3134134430 legacy=0x7f313413f500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:25:44.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313949c640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f3134133bf0 0x7f313409bd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:35034/0 (socket says 172.21.15.115:35034) 2024-01-24T23:25:44.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313949c640 1 -- 172.21.15.115:0/2462382988 learned_addr learned my addr 172.21.15.115:0/2462382988 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:44.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f3138c9b640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f3134133fd0 legacy=0x7f313413bdd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:25:44.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 1 -- 172.21.15.115:0/2462382988 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2652692306 0 0) 0x7f313409af30 con 0x7f3134133bf0 2024-01-24T23:25:44.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:44.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:44.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:44.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 10 monclient(hunting): my global_id is 6171 2024-01-24T23:25:44.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 1 -- 172.21.15.115:0/2462382988 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3120003530 con 0x7f3134133bf0 2024-01-24T23:25:44.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 1 -- 172.21.15.115:0/2462382988 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2150485823 0 0) 0x7f3120003530 con 0x7f3134133bf0 2024-01-24T23:25:44.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 1 -- 172.21.15.115:0/2462382988 >> v1:172.21.15.132:6792/0 conn(0x7f3134133fd0 legacy=0x7f313413bdd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:44.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 1 -- 172.21.15.115:0/2462382988 >> v1:172.21.15.115:6790/0 conn(0x7f3134134430 legacy=0x7f313413f500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:44.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:25:44.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 1 monclient: found mon.noname-d 2024-01-24T23:25:44.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:44.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:25:44.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 1 -- 172.21.15.115:0/2462382988 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313409b4f0 con 0x7f3134133bf0 2024-01-24T23:25:44.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 10 monclient: _finish_auth 0 2024-01-24T23:25:44.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:25:44.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:44.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 1 -- 172.21.15.115:0/2462382988 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f31240017c0 con 0x7f3134133bf0 2024-01-24T23:25:44.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313b700640 5 monclient: authenticate success, global_id 6171 2024-01-24T23:25:44.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313b700640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:44.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:44.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:25:44.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 10 monclient: dump: 2024-01-24T23:25:44.964 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:44.964 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.964 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.964 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.964 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:44.964 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.964 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:44.964 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:44.964 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:44.964 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:44.965 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:44.965 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:44.965 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:44.965 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:44.965 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:44.965 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313b700640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:44.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 1 -- 172.21.15.115:0/2462382988 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3124001ae0 con 0x7f3134133bf0 2024-01-24T23:25:44.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:44.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313b700640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:44.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 1 -- 172.21.15.115:0/2462382988 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3124004bf0 con 0x7f3134133bf0 2024-01-24T23:25:44.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:44.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313b700640 4 set_mon_vals no callback set 2024-01-24T23:25:44.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:44.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f312bfff640 10 monclient: dump: 2024-01-24T23:25:44.966 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:44.966 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.966 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.966 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.966 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:44.966 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.967 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:44.967 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:44.967 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:44.967 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:44.967 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:44.967 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:44.967 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:44.967 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:44.968 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:44.968 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313b700640 10 monclient: shutdown 2024-01-24T23:25:44.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313b700640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:44.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.949+0000 7f313b700640 1 -- 172.21.15.115:0/2462382988 >> v1:172.21.15.115:6792/0 conn(0x7f3134133bf0 legacy=0x7f313409bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:44.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 1 -- 172.21.15.115:0/2462382988 shutdown_connections 2024-01-24T23:25:44.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 1 -- 172.21.15.115:0/2462382988 >> 172.21.15.115:0/2462382988 conn(0x7f31340a9910 msgr2=0x7f31340a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:44.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 1 -- 172.21.15.115:0/2462382988 shutdown_connections 2024-01-24T23:25:44.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 1 -- 172.21.15.115:0/2462382988 wait complete. 2024-01-24T23:25:44.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 10 monclient: build_initial_monmap 2024-01-24T23:25:44.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 10 monclient: monmap: 2024-01-24T23:25:44.968 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:44.968 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:44.968 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:44.968 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:44.969 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:44.969 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:44.969 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:44.969 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:44.969 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:44.969 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:44.969 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:44.969 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:44.969 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:44.969 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:44.969 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:44.969 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 1 Processor -- start 2024-01-24T23:25:44.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 1 -- start start 2024-01-24T23:25:44.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 10 monclient: init 2024-01-24T23:25:44.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:44.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 10 monclient: _add_conns ranks=[0,7,4,2,3,5,1,6,8] 2024-01-24T23:25:44.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 10 monclient(hunting): picked mon.noname-a con 0x7f3134133bf0 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:44.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 10 monclient(hunting): picked mon.noname-i con 0x7f3134133fd0 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:44.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 10 monclient(hunting): picked mon.noname-c con 0x7f3134134430 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:44.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f313409c8f0 con 0x7f3134133bf0 2024-01-24T23:25:44.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f313413b9e0 con 0x7f3134134430 2024-01-24T23:25:44.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31340a5960 con 0x7f3134133fd0 2024-01-24T23:25:44.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:44.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 10 monclient(hunting): authenticate will time out at 1223.877783s 2024-01-24T23:25:44.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3139c9d640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f3134134430 0x7f31341d38d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:41860/0 (socket says 172.21.15.115:41860) 2024-01-24T23:25:44.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3139c9d640 1 -- 172.21.15.115:0/3754859424 learned_addr learned my addr 172.21.15.115:0/3754859424 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:44.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3138c9b640 1 -- 172.21.15.115:0/3754859424 >> v1:172.21.15.132:6792/0 conn(0x7f3134133fd0 legacy=0x7f31341d01a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:25:44.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 675370529 0 0) 0x7f313409c8f0 con 0x7f3134133bf0 2024-01-24T23:25:44.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:44.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:44.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:44.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 10 monclient(hunting): my global_id is 94347 2024-01-24T23:25:44.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3118003530 con 0x7f3134133bf0 2024-01-24T23:25:44.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383128238 0 0) 0x7f313413b9e0 con 0x7f3134134430 2024-01-24T23:25:44.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:44.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:44.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:44.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 10 monclient(hunting): my global_id is 94342 2024-01-24T23:25:44.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f313409c8f0 con 0x7f3134134430 2024-01-24T23:25:44.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 651828756 0 0) 0x7f3118003530 con 0x7f3134133bf0 2024-01-24T23:25:44.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f313413b9e0 con 0x7f3134133bf0 2024-01-24T23:25:44.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3124001780 con 0x7f3134133bf0 2024-01-24T23:25:44.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:44.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:25:44.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 10 monclient(hunting): dump: 2024-01-24T23:25:44.971 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:44.971 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.971 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.972 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.972 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:44.972 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.972 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:44.972 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:44.972 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:44.972 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:44.972 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:44.972 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:44.972 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:44.972 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:44.972 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:44.972 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:44.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1752980896 0 0) 0x7f313409c8f0 con 0x7f3134134430 2024-01-24T23:25:44.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3118003530 con 0x7f3134134430 2024-01-24T23:25:44.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f312c004560 con 0x7f3134134430 2024-01-24T23:25:44.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:44.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:44.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 10 monclient(hunting): dump: 2024-01-24T23:25:44.973 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:44.973 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.973 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.973 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.973 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:44.973 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.973 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:44.973 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:44.973 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:44.973 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:44.973 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:44.973 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:44.974 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:44.974 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:44.974 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:44.974 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3882439706 0 0) 0x7f313413b9e0 con 0x7f3134133bf0 2024-01-24T23:25:44.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 >> v1:172.21.15.132:6792/0 conn(0x7f3134133fd0 legacy=0x7f31341d01a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:44.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 >> v1:172.21.15.115:6791/0 conn(0x7f3134134430 legacy=0x7f31341d38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:44.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:25:44.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 1 monclient: found mon.a 2024-01-24T23:25:44.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:44.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:44.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313413b700 con 0x7f3134133bf0 2024-01-24T23:25:44.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 10 monclient: _finish_auth 0 2024-01-24T23:25:44.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:25:44.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:44.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 5 monclient: authenticate success, global_id 94347 2024-01-24T23:25:44.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 10 monclient: _renew_subs 2024-01-24T23:25:44.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:44.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 1 -- 172.21.15.115:0/3754859424 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3134139f20 con 0x7f3134133bf0 2024-01-24T23:25:44.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 10 monclient: _renew_subs 2024-01-24T23:25:44.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:44.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 1 -- 172.21.15.115:0/3754859424 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3134140800 con 0x7f3134133bf0 2024-01-24T23:25:44.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3124004b50 con 0x7f3134133bf0 2024-01-24T23:25:44.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:44.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3124001c50 con 0x7f3134133bf0 2024-01-24T23:25:44.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:44.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f312b7fe640 4 set_mon_vals no callback set 2024-01-24T23:25:44.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:44.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 10 monclient: dump: 2024-01-24T23:25:44.975 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:44.975 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.975 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.975 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.976 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:44.976 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.976 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:44.976 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:44.976 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:44.976 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:44.976 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:44.976 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:44.976 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:44.976 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:44.976 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:44.976 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:44.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f313b700640 1 -- 172.21.15.115:0/3754859424 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f31140065d0 con 0x7f3114002870 2024-01-24T23:25:44.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f312400fdd0 con 0x7f3134133bf0 2024-01-24T23:25:44.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.953+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(129..129 src has 1..129) v4 ==== 6377+0+0 (unknown 51197916 0 0) 0x7f3124044450 con 0x7f3134133bf0 2024-01-24T23:25:44.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.957+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 150828396 0 0) 0x7f31140065d0 con 0x7f3114002870 2024-01-24T23:25:44.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.957+0000 7f3129ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:44.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.957+0000 7f3129ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:44.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.957+0000 7f3129ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:44.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.957+0000 7f3129ffb640 10 monclient(hunting): my global_id is 6180 2024-01-24T23:25:44.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.957+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f313413b9e0 con 0x7f3114002870 2024-01-24T23:25:44.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.957+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 400916595 0 0) 0x7f313413b9e0 con 0x7f3114002870 2024-01-24T23:25:44.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.957+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f31140065d0 con 0x7f3114002870 2024-01-24T23:25:44.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.957+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3130004090 con 0x7f3114002870 2024-01-24T23:25:44.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.957+0000 7f3129ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:44.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.957+0000 7f3129ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:44.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.957+0000 7f3129ffb640 10 monclient: dump: 2024-01-24T23:25:44.978 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:44.978 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.978 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.978 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.978 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:44.978 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.978 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:44.978 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:44.978 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:44.978 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:44.978 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:44.978 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:44.978 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:44.978 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:44.978 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:44.979 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.957+0000 7f3139c9d640 10 monclient: get_auth_request con 0x7f3118037690 auth_method 0 2024-01-24T23:25:44.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.957+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 213262786 0 0) 0x7f31140065d0 con 0x7f3114002870 2024-01-24T23:25:44.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.957+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 --> v1:172.21.15.115:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f311400a120 con 0x7f3114002870 2024-01-24T23:25:44.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.957+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f31300064a0 con 0x7f3114002870 2024-01-24T23:25:44.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.957+0000 7f3129ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:44.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.957+0000 7f3129ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:44.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.957+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 >> v1:172.21.15.115:6792/0 conn(0x7f3114002870 legacy=0x7f3114004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:44.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.957+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 >> v1:172.21.15.115:6792/0 conn(0x7f3114002870 legacy=0x7f3114004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:44.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.961+0000 7f313b700640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:25:44.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.961+0000 7f313b700640 1 -- 172.21.15.115:0/3754859424 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3114009e10 con 0x7f31140090d0 2024-01-24T23:25:44.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.961+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2259947596 0 0) 0x7f3114009e10 con 0x7f31140090d0 2024-01-24T23:25:44.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.961+0000 7f3129ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:44.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.961+0000 7f3129ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:44.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.961+0000 7f3129ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:44.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.961+0000 7f3129ffb640 10 monclient(hunting): my global_id is 6189 2024-01-24T23:25:44.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.961+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31140065d0 con 0x7f31140090d0 2024-01-24T23:25:44.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.961+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3594609352 0 0) 0x7f31140065d0 con 0x7f31140090d0 2024-01-24T23:25:44.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.961+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3114009e10 con 0x7f31140090d0 2024-01-24T23:25:44.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.961+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3130004c70 con 0x7f31140090d0 2024-01-24T23:25:44.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.961+0000 7f3129ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:44.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.961+0000 7f3129ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:44.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.961+0000 7f3129ffb640 10 monclient: dump: 2024-01-24T23:25:44.981 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:44.982 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:44.982 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.982 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:44.982 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:44.982 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:44.982 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:44.982 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:44.982 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:44.982 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:44.982 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:44.982 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:44.982 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:44.982 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:44.982 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:44.982 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:44.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.961+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 96493358 0 0) 0x7f3114009e10 con 0x7f31140090d0 2024-01-24T23:25:44.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.961+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 --> v1:172.21.15.115:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f311400c310 con 0x7f31140090d0 2024-01-24T23:25:44.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.965+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7680 (unknown 0 0 1435539831) 0x7f31300062b0 con 0x7f31140090d0 2024-01-24T23:25:44.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.965+0000 7f3129ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:25:44.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.965+0000 7f3129ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:44.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.965+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 >> v1:172.21.15.115:6792/0 conn(0x7f31140090d0 legacy=0x7f3114009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:44.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.965+0000 7f3129ffb640 1 -- 172.21.15.115:0/3754859424 >> v1:172.21.15.115:6792/0 conn(0x7f31140090d0 legacy=0x7f3114009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:44.983 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:25:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:25:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 34, 2024-01-24T23:25:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:25:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:25:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:25:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:25:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:25:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:25:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:25:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:25:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:25:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:25:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:25:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:25:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:25:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:25:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:25:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:25:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:25:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:25:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:25:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:25:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:25:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:25:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:25:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:25:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:25:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:25:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:25:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:25:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:25:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:25:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:25:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:25:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:25:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:25:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:25:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:25:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:25:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:25:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:25:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:25:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:25:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:25:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:25:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:25:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:25:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:25:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:25:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:25:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:25:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:25:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:25:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:25:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:25:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:25:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:25:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:25:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:25:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:25:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:25:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 3 2024-01-24T23:25:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.001 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.001 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:25:45.001 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:25:45.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.965+0000 7f313b700640 1 -- 172.21.15.115:0/3754859424 >> v1:172.21.15.132:6812/1835401922 conn(0x7f3118037690 legacy=0x7f3118039b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.965+0000 7f313b700640 10 monclient: shutdown 2024-01-24T23:25:45.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.965+0000 7f313b700640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:45.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.965+0000 7f313b700640 1 -- 172.21.15.115:0/3754859424 >> v1:172.21.15.115:6789/0 conn(0x7f3134133bf0 legacy=0x7f313413ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.965+0000 7f3139c9d640 1 -- 172.21.15.115:0/3754859424 reap_dead start 2024-01-24T23:25:45.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.965+0000 7f3139c9d640 1 -- 172.21.15.115:0/3754859424 reap_dead start 2024-01-24T23:25:45.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.965+0000 7f313b700640 1 -- 172.21.15.115:0/3754859424 shutdown_connections 2024-01-24T23:25:45.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.965+0000 7f313b700640 1 -- 172.21.15.115:0/3754859424 >> 172.21.15.115:0/3754859424 conn(0x7f31340a9910 msgr2=0x7f313406a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:45.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.965+0000 7f313b700640 1 -- 172.21.15.115:0/3754859424 shutdown_connections 2024-01-24T23:25:45.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:44.965+0000 7f313b700640 1 -- 172.21.15.115:0/3754859424 wait complete. 2024-01-24T23:25:45.003 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-01-24T23:25:45.003 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon e 2024-01-24T23:25:45.003 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-01-24T23:25:45.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.068+0000 7fdf3affd640 10 monclient: tick 2024-01-24T23:25:45.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.068+0000 7fdf3affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:45.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.068+0000 7fdf3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:45.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a3e2a640 10 monclient: get_monmap_and_config 2024-01-24T23:25:45.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a3e2a640 10 monclient: build_initial_monmap 2024-01-24T23:25:45.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a3e2a640 10 monclient: monmap: 2024-01-24T23:25:45.118 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:45.118 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.119 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:45.119366+0000 2024-01-24T23:25:45.119 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:45.119366+0000 2024-01-24T23:25:45.119 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:45.119 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:45.119 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:45.119 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:45.119 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:45.119 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:45.119 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:45.119 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:45.119 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:45.119 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:45.119 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:45.119 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a3e2a640 1 Processor -- start 2024-01-24T23:25:45.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a3e2a640 1 -- start start 2024-01-24T23:25:45.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a3e2a640 10 monclient: init 2024-01-24T23:25:45.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a3e2a640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:45.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a3e2a640 10 monclient: _add_conns ranks=[0,2,6,8,3,5,1,7,4] 2024-01-24T23:25:45.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a3e2a640 10 monclient(hunting): picked mon.noname-a con 0x7f749c0a4270 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:45.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a3e2a640 10 monclient(hunting): picked mon.noname-b con 0x7f749c071c10 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:45.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a3e2a640 10 monclient(hunting): picked mon.noname-d con 0x7f749c13b180 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:45.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a3e2a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749c04ed70 con 0x7f749c0a4270 2024-01-24T23:25:45.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a3e2a640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749c09eb10 con 0x7f749c071c10 2024-01-24T23:25:45.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a3e2a640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749c0a4650 con 0x7f749c13b180 2024-01-24T23:25:45.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a3e2a640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:45.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a3e2a640 10 monclient(hunting): authenticate will time out at 1224.042709s 2024-01-24T23:25:45.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a13c5640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f749c071c10 legacy=0x7f749c139a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:25:45.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a1bc6640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f749c0a4270 0x7f749c071500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:53170/0 (socket says 172.21.15.115:53170) 2024-01-24T23:25:45.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a1bc6640 1 -- 172.21.15.115:0/1793186924 learned_addr learned my addr 172.21.15.115:0/1793186924 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:45.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a23c7640 1 --1- 172.21.15.115:0/1793186924 >> v1:172.21.15.115:6792/0 conn(0x7f749c13b180 0x7f749c13d570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:35066/0 (socket says 172.21.15.115:35066) 2024-01-24T23:25:45.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 1 -- 172.21.15.115:0/1793186924 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 327881342 0 0) 0x7f749c04ed70 con 0x7f749c0a4270 2024-01-24T23:25:45.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 10 monclient(hunting): _negotiate 2024-01-24T23:25:45.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:45.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:45.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 10 monclient(hunting): my global_id is 94356 2024-01-24T23:25:45.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 1 -- 172.21.15.115:0/1793186924 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7488003530 con 0x7f749c0a4270 2024-01-24T23:25:45.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 1 -- 172.21.15.115:0/1793186924 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3396577791 0 0) 0x7f749c0a4650 con 0x7f749c13b180 2024-01-24T23:25:45.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 10 monclient(hunting): _negotiate 2024-01-24T23:25:45.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:45.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:45.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 10 monclient(hunting): my global_id is 6198 2024-01-24T23:25:45.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 1 -- 172.21.15.115:0/1793186924 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f749c04ed70 con 0x7f749c13b180 2024-01-24T23:25:45.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 1 -- 172.21.15.115:0/1793186924 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1404776882 0 0) 0x7f7488003530 con 0x7f749c0a4270 2024-01-24T23:25:45.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 1 -- 172.21.15.115:0/1793186924 >> v1:172.21.15.115:6792/0 conn(0x7f749c13b180 legacy=0x7f749c13d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 1 -- 172.21.15.115:0/1793186924 >> v1:172.21.15.115:6790/0 conn(0x7f749c071c10 legacy=0x7f749c139a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:45.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 10 monclient: _finish_hunting 0 2024-01-24T23:25:45.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 1 monclient: found mon.noname-a 2024-01-24T23:25:45.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:45.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:25:45.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 1 -- 172.21.15.115:0/1793186924 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c06f090 con 0x7f749c0a4270 2024-01-24T23:25:45.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 10 monclient: _finish_auth 0 2024-01-24T23:25:45.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 10 monclient: _check_auth_tickets 2024-01-24T23:25:45.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:45.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 1 -- 172.21.15.115:0/1793186924 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f74980017c0 con 0x7f749c0a4270 2024-01-24T23:25:45.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a3e2a640 5 monclient: authenticate success, global_id 94356 2024-01-24T23:25:45.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a3e2a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:45.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:25:45.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 10 monclient: dump: 2024-01-24T23:25:45.129 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:45.129 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.129 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.129 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.129 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.129 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:45.129 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.129 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.129 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.129 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.129 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.130 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.130 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.130 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.130 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.130 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a3e2a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:45.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 1 -- 172.21.15.115:0/1793186924 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7498001a20 con 0x7f749c0a4270 2024-01-24T23:25:45.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:45.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a3e2a640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:45.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a3e2a640 4 set_mon_vals no callback set 2024-01-24T23:25:45.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 1 -- 172.21.15.115:0/1793186924 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7498004bf0 con 0x7f749c0a4270 2024-01-24T23:25:45.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:45.130 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.117+0000 7f74a0bc4640 10 monclient: dump: 2024-01-24T23:25:45.130 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:45.130 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.130 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.131 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.131 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.131 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:45.131 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.131 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.131 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.131 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.131 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.131 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.131 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.131 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.131 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.131 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 10 monclient: shutdown 2024-01-24T23:25:45.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:45.131 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 1 -- 172.21.15.115:0/1793186924 >> v1:172.21.15.115:6789/0 conn(0x7f749c0a4270 legacy=0x7f749c071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 1 -- 172.21.15.115:0/1793186924 shutdown_connections 2024-01-24T23:25:45.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 1 -- 172.21.15.115:0/1793186924 >> 172.21.15.115:0/1793186924 conn(0x7f749c0a7660 msgr2=0x7f749c0a3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:45.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 1 -- 172.21.15.115:0/1793186924 shutdown_connections 2024-01-24T23:25:45.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 1 -- 172.21.15.115:0/1793186924 wait complete. 2024-01-24T23:25:45.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 10 monclient: build_initial_monmap 2024-01-24T23:25:45.132 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 10 monclient: monmap: 2024-01-24T23:25:45.132 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:45.132 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:45.132 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:45.132 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:45.132 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:45.132 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:45.132 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:45.132 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:45.133 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:45.133 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:45.133 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:45.133 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:45.133 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:45.133 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:45.133 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:45.133 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 1 Processor -- start 2024-01-24T23:25:45.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 1 -- start start 2024-01-24T23:25:45.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 10 monclient: init 2024-01-24T23:25:45.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:45.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 10 monclient: _add_conns ranks=[5,7,0,6,8,4,3,1,2] 2024-01-24T23:25:45.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 10 monclient(hunting): picked mon.noname-h con 0x7f749c071c10 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:45.133 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 10 monclient(hunting): picked mon.noname-i con 0x7f749c0a4270 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:45.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 10 monclient(hunting): picked mon.noname-a con 0x7f749c13b180 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:45.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749c1394b0 con 0x7f749c13b180 2024-01-24T23:25:45.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749c072100 con 0x7f749c071c10 2024-01-24T23:25:45.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749c11dec0 con 0x7f749c0a4270 2024-01-24T23:25:45.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:45.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 10 monclient(hunting): authenticate will time out at 1224.045566s 2024-01-24T23:25:45.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a23c7640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f749c13b180 0x7f749c1d1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:53172/0 (socket says 172.21.15.115:53172) 2024-01-24T23:25:45.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a23c7640 1 -- 172.21.15.115:0/425469328 learned_addr learned my addr 172.21.15.115:0/425469328 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:45.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a13c5640 1 -- 172.21.15.115:0/425469328 >> v1:172.21.15.132:6792/0 conn(0x7f749c0a4270 legacy=0x7f749c137980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:25:45.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971724227 0 0) 0x7f749c1394b0 con 0x7f749c13b180 2024-01-24T23:25:45.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:45.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:45.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:45.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 10 monclient(hunting): my global_id is 94365 2024-01-24T23:25:45.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7480003530 con 0x7f749c13b180 2024-01-24T23:25:45.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3531479790 0 0) 0x7f7480003530 con 0x7f749c13b180 2024-01-24T23:25:45.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f749c1394b0 con 0x7f749c13b180 2024-01-24T23:25:45.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f74940018e0 con 0x7f749c13b180 2024-01-24T23:25:45.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:25:45.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 10 monclient(hunting): dump: 2024-01-24T23:25:45.135 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:45.135 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.135 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.135 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.135 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.135 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:45.135 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.135 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.135 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.136 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.136 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.136 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.136 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.136 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.136 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.136 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:45.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2112831763 0 0) 0x7f749c1394b0 con 0x7f749c13b180 2024-01-24T23:25:45.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 >> v1:172.21.15.132:6792/0 conn(0x7f749c0a4270 legacy=0x7f749c137980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:45.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 >> v1:172.21.15.132:6791/0 conn(0x7f749c071c10 legacy=0x7f749c137270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 10 monclient: _finish_hunting 0 2024-01-24T23:25:45.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 1 monclient: found mon.a 2024-01-24T23:25:45.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:45.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:45.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c09f410 con 0x7f749c13b180 2024-01-24T23:25:45.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 10 monclient: _finish_auth 0 2024-01-24T23:25:45.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 10 monclient: _check_auth_tickets 2024-01-24T23:25:45.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:45.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 5 monclient: authenticate success, global_id 94365 2024-01-24T23:25:45.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 10 monclient: _renew_subs 2024-01-24T23:25:45.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:45.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 1 -- 172.21.15.115:0/425469328 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f749c11daa0 con 0x7f749c13b180 2024-01-24T23:25:45.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 10 monclient: _renew_subs 2024-01-24T23:25:45.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:45.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 1 -- 172.21.15.115:0/425469328 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f749c133520 con 0x7f749c13b180 2024-01-24T23:25:45.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7494004e20 con 0x7f749c13b180 2024-01-24T23:25:45.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:45.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f74940052d0 con 0x7f749c13b180 2024-01-24T23:25:45.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f7493fff640 4 set_mon_vals no callback set 2024-01-24T23:25:45.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:45.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 10 monclient: dump: 2024-01-24T23:25:45.138 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:45.138 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.138 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.138 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.138 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.138 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:45.138 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.138 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.138 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.138 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.138 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.138 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.139 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.139 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.139 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.139 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:45.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a3e2a640 1 -- 172.21.15.115:0/425469328 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f74840065d0 con 0x7f7484002870 2024-01-24T23:25:45.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f749400fd50 con 0x7f749c13b180 2024-01-24T23:25:45.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(129..129 src has 1..129) v4 ==== 6377+0+0 (unknown 51197916 0 0) 0x7f7494044630 con 0x7f749c13b180 2024-01-24T23:25:45.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2166325880 0 0) 0x7f74840065d0 con 0x7f7484002870 2024-01-24T23:25:45.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:45.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:45.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:45.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 10 monclient(hunting): my global_id is 85247 2024-01-24T23:25:45.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f749c1394b0 con 0x7f7484002870 2024-01-24T23:25:45.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74a13c5640 10 monclient: get_auth_request con 0x7f7480036fb0 auth_method 0 2024-01-24T23:25:45.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 168906355 0 0) 0x7f749c1394b0 con 0x7f7484002870 2024-01-24T23:25:45.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f74840065d0 con 0x7f7484002870 2024-01-24T23:25:45.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7498001440 con 0x7f7484002870 2024-01-24T23:25:45.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:45.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 10 monclient: dump: 2024-01-24T23:25:45.140 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:45.140 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.140 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.140 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.140 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.140 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:45.140 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.140 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.140 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.140 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.141 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.141 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.141 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.141 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.141 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.141 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2880939123 0 0) 0x7f74840065d0 con 0x7f7484002870 2024-01-24T23:25:45.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.121+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 --> v1:172.21.15.115:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f748400a120 con 0x7f7484002870 2024-01-24T23:25:45.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.125+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f7498004880 con 0x7f7484002870 2024-01-24T23:25:45.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.125+0000 7f74927fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:45.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.125+0000 7f74927fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:45.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.125+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 >> v1:172.21.15.115:6793/0 conn(0x7f7484002870 legacy=0x7f7484004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.141 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.125+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 >> v1:172.21.15.115:6793/0 conn(0x7f7484002870 legacy=0x7f7484004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:45.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74a3e2a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:25:45.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74a3e2a640 1 -- 172.21.15.115:0/425469328 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7484009e10 con 0x7f74840090d0 2024-01-24T23:25:45.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1133337581 0 0) 0x7f7484009e10 con 0x7f74840090d0 2024-01-24T23:25:45.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74927fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:45.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74927fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:45.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74927fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:45.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74927fc640 10 monclient(hunting): my global_id is 85256 2024-01-24T23:25:45.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74840065d0 con 0x7f74840090d0 2024-01-24T23:25:45.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1778666793 0 0) 0x7f74840065d0 con 0x7f74840090d0 2024-01-24T23:25:45.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7484009e10 con 0x7f74840090d0 2024-01-24T23:25:45.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7498005340 con 0x7f74840090d0 2024-01-24T23:25:45.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74927fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:45.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74927fc640 10 monclient: dump: 2024-01-24T23:25:45.143 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:45.144 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.144 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.144 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.144 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.144 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:45.144 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.144 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.144 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.144 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.144 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.144 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.144 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.144 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.145 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.145 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2648796021 0 0) 0x7f7484009e10 con 0x7f74840090d0 2024-01-24T23:25:45.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 --> v1:172.21.15.115:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f748400c310 con 0x7f74840090d0 2024-01-24T23:25:45.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7510 (unknown 0 0 2865603546) 0x7f74980075c0 con 0x7f74840090d0 2024-01-24T23:25:45.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74927fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:25:45.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74927fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:45.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 >> v1:172.21.15.115:6793/0 conn(0x7f74840090d0 legacy=0x7f7484009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74927fc640 1 -- 172.21.15.115:0/425469328 >> v1:172.21.15.115:6793/0 conn(0x7f74840090d0 legacy=0x7f7484009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:45.145 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:25:45.145 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:45.146 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:45.146 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:25:45.146 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 34, 2024-01-24T23:25:45.146 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:25:45.146 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:25:45.146 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:25:45.146 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:25:45.146 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:25:45.146 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:25:45.146 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:25:45.146 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:45.146 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:25:45.146 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:45.146 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:25:45.147 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:25:45.147 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:45.147 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:45.147 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:45.147 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:45.147 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:45.147 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:45.147 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:45.147 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:45.147 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:45.147 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:45.147 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:45.147 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:45.147 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:25:45.147 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:25:45.148 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:45.148 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:45.148 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:45.148 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:45.148 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:45.148 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:45.148 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:45.148 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:45.148 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:45.148 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:45.148 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:45.148 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.149 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:25:45.149 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:25:45.149 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:25:45.149 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:25:45.149 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:25:45.149 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:25:45.149 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:45.149 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:45.149 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:25:45.149 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:25:45.150 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:25:45.150 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:25:45.150 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:25:45.150 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:25:45.150 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:25:45.150 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:45.150 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:25:45.150 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:45.150 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:45.150 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:45.150 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:45.150 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:45.150 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:45.150 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:45.150 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:45.151 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:45.151 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:45.151 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:45.151 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:45.151 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:25:45.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.151 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:25:45.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.151 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:45.151 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:45.151 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.151 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.151 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.151 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:25:45.151 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.152 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.152 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.152 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:25:45.152 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:25:45.152 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.152 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.152 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.152 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:45.152 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:45.152 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.152 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.153 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.153 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:25:45.153 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.153 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.153 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.153 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:25:45.153 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:25:45.153 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.153 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.153 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.154 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:45.154 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:45.154 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.154 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.154 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.154 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:25:45.154 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.154 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.154 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.154 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:25:45.154 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:25:45.154 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.154 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.155 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.155 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:45.155 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:45.155 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.155 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.155 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.155 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:25:45.155 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.155 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.155 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.155 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:25:45.156 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:25:45.156 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.156 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.156 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.156 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:45.156 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:45.156 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.156 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.156 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.156 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:25:45.156 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.157 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.157 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.157 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:25:45.157 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:25:45.157 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.157 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.157 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.157 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:45.157 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:45.157 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.157 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.158 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.158 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:25:45.158 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.158 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.158 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.158 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:25:45.158 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:25:45.158 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.158 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.158 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.158 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:45.159 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:45.159 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.159 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.159 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.159 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:25:45.159 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.159 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.159 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:25:45.159 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:25:45.159 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.159 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.159 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.160 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:45.160 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:45.160 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.160 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.160 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.160 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:25:45.160 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.160 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.160 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.160 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:25:45.161 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:25:45.161 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.161 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.161 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.161 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:45.161 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:45.161 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.161 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.161 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.161 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:25:45.161 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.161 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.161 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.162 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:25:45.162 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:25:45.162 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.162 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.162 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.162 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.162 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.162 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:25:45.162 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:25:45.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.162 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:45.162 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:45.162 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:45.162 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.162 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:45.163 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:25:45.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.163 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:45.163 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:45.163 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:45.163 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.163 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.163 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:25:45.163 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:25:45.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74a3e2a640 1 -- 172.21.15.115:0/425469328 >> v1:172.21.15.132:6812/1835401922 conn(0x7f7480036fb0 legacy=0x7f7480039470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74a3e2a640 10 monclient: shutdown 2024-01-24T23:25:45.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74a3e2a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:45.164 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74a3e2a640 1 -- 172.21.15.115:0/425469328 >> v1:172.21.15.115:6789/0 conn(0x7f749c13b180 legacy=0x7f749c1d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74a23c7640 1 -- 172.21.15.115:0/425469328 reap_dead start 2024-01-24T23:25:45.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74a23c7640 1 -- 172.21.15.115:0/425469328 reap_dead start 2024-01-24T23:25:45.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74a3e2a640 1 -- 172.21.15.115:0/425469328 shutdown_connections 2024-01-24T23:25:45.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74a3e2a640 1 -- 172.21.15.115:0/425469328 >> 172.21.15.115:0/425469328 conn(0x7f749c0a7660 msgr2=0x7f749c133a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:45.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74a3e2a640 1 -- 172.21.15.115:0/425469328 shutdown_connections 2024-01-24T23:25:45.165 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.129+0000 7f74a3e2a640 1 -- 172.21.15.115:0/425469328 wait complete. 2024-01-24T23:25:45.165 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-01-24T23:25:45.165 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon g 2024-01-24T23:25:45.165 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-01-24T23:25:45.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.220+0000 7fdf3bfff640 1 -- 172.21.15.132:0/935895895 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_command_ack([{"prefix": "osd crush weight-set rm", "pool": "barset"}]=0 v130) v1 ==== 89+0+0 (unknown 600671994 0 0) 0x7fdf4c019140 con 0x7fdf40045bd0 2024-01-24T23:25:45.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.220+0000 7fdf3bfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-01-24T23:25:45.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.220+0000 7fdf3bfff640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:25:45.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.224+0000 7fdf59d07640 1 -- 172.21.15.132:0/935895895 >> v1:172.21.15.132:6812/1835401922 conn(0x7fdf40037850 legacy=0x7fdf40039d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.224+0000 7fdf59d07640 10 monclient: shutdown 2024-01-24T23:25:45.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.224+0000 7fdf59d07640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:45.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.224+0000 7fdf59d07640 1 -- 172.21.15.132:0/935895895 >> v1:172.21.15.115:6792/0 conn(0x7fdf40045bd0 legacy=0x7fdf40048090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.224+0000 7fdf59d07640 1 -- 172.21.15.132:0/935895895 shutdown_connections 2024-01-24T23:25:45.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.224+0000 7fdf59d07640 1 -- 172.21.15.132:0/935895895 >> 172.21.15.132:0/935895895 conn(0x7fdf540a22a0 msgr2=0x7fdf5406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:45.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.224+0000 7fdf59d07640 1 -- 172.21.15.132:0/935895895 shutdown_connections 2024-01-24T23:25:45.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.224+0000 7fdf59d07640 1 -- 172.21.15.132:0/935895895 wait complete. 2024-01-24T23:25:45.243 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set ls 2024-01-24T23:25:45.243 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep -q 0 2024-01-24T23:25:45.243 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep -c barset 2024-01-24T23:25:45.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 10 monclient: get_monmap_and_config 2024-01-24T23:25:45.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 10 monclient: build_initial_monmap 2024-01-24T23:25:45.286 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 10 monclient: monmap: 2024-01-24T23:25:45.286 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:45.286 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.287 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:45.287220+0000 2024-01-24T23:25:45.287 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:45.287220+0000 2024-01-24T23:25:45.287 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:45.287 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:45.287 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:45.287 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:45.287 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:45.287 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:45.287 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:45.287 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:45.287 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:45.287 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:45.288 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:45.288 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 1 Processor -- start 2024-01-24T23:25:45.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 1 -- start start 2024-01-24T23:25:45.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 10 monclient: init 2024-01-24T23:25:45.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:45.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 10 monclient: _add_conns ranks=[7,8,3,5,2,6,4,1,0] 2024-01-24T23:25:45.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 10 monclient(hunting): picked mon.noname-i con 0x7f7a28133bf0 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:45.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 10 monclient(hunting): picked mon.noname-e con 0x7f7a28133fd0 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:45.295 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 10 monclient(hunting): picked mon.noname-g con 0x7f7a28134430 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:45.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a2804ed70 con 0x7f7a28134430 2024-01-24T23:25:45.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a2809af30 con 0x7f7a28133bf0 2024-01-24T23:25:45.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a2809b210 con 0x7f7a28133fd0 2024-01-24T23:25:45.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:45.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 10 monclient(hunting): authenticate will time out at 1224.209737s 2024-01-24T23:25:45.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2d619640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f7a28133bf0 legacy=0x7f7a2809bd80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:25:45.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2ce18640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f7a28133fd0 0x7f7a2813bdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:54966/0 (socket says 172.21.15.115:54966) 2024-01-24T23:25:45.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2ce18640 1 -- 172.21.15.115:0/4093669355 learned_addr learned my addr 172.21.15.115:0/4093669355 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:45.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 1 -- 172.21.15.115:0/4093669355 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4081998230 0 0) 0x7f7a2809b210 con 0x7f7a28133fd0 2024-01-24T23:25:45.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:45.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:45.296 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:45.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 10 monclient(hunting): my global_id is 85265 2024-01-24T23:25:45.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 1 -- 172.21.15.115:0/4093669355 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a14003530 con 0x7f7a28133fd0 2024-01-24T23:25:45.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 1 -- 172.21.15.115:0/4093669355 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2857827398 0 0) 0x7f7a14003530 con 0x7f7a28133fd0 2024-01-24T23:25:45.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 1 -- 172.21.15.115:0/4093669355 >> v1:172.21.15.132:6792/0 conn(0x7f7a28133bf0 legacy=0x7f7a2809bd80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:45.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 1 -- 172.21.15.115:0/4093669355 >> v1:172.21.15.132:6790/0 conn(0x7f7a28134430 legacy=0x7f7a2813f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 10 monclient: _finish_hunting 0 2024-01-24T23:25:45.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 1 monclient: found mon.noname-e 2024-01-24T23:25:45.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:45.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:25:45.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 1 -- 172.21.15.115:0/4093669355 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2809b4f0 con 0x7f7a28133fd0 2024-01-24T23:25:45.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 10 monclient: _finish_auth 0 2024-01-24T23:25:45.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 10 monclient: _check_auth_tickets 2024-01-24T23:25:45.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:45.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 5 monclient: authenticate success, global_id 85265 2024-01-24T23:25:45.297 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 1 -- 172.21.15.115:0/4093669355 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7a180017c0 con 0x7f7a28133fd0 2024-01-24T23:25:45.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:25:45.298 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 10 monclient: dump: 2024-01-24T23:25:45.298 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:45.298 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.298 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.298 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.298 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.298 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:45.298 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.298 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.298 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.298 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.298 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.299 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.299 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.299 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.299 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.299 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:45.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 1 -- 172.21.15.115:0/4093669355 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7a18001ae0 con 0x7f7a28133fd0 2024-01-24T23:25:45.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:45.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:45.299 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 4 set_mon_vals no callback set 2024-01-24T23:25:45.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 1 -- 172.21.15.115:0/4093669355 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7a18004bf0 con 0x7f7a28133fd0 2024-01-24T23:25:45.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:45.300 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a1ffff640 10 monclient: dump: 2024-01-24T23:25:45.300 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:45.300 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.300 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.300 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.300 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.300 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:45.300 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.300 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.300 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.300 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.301 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.301 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.301 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.301 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.301 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.301 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 10 monclient: shutdown 2024-01-24T23:25:45.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:45.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 1 -- 172.21.15.115:0/4093669355 >> v1:172.21.15.115:6793/0 conn(0x7f7a28133fd0 legacy=0x7f7a2813bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 1 -- 172.21.15.115:0/4093669355 shutdown_connections 2024-01-24T23:25:45.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 1 -- 172.21.15.115:0/4093669355 >> 172.21.15.115:0/4093669355 conn(0x7f7a280a9910 msgr2=0x7f7a280a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:45.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 1 -- 172.21.15.115:0/4093669355 shutdown_connections 2024-01-24T23:25:45.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 1 -- 172.21.15.115:0/4093669355 wait complete. 2024-01-24T23:25:45.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 10 monclient: build_initial_monmap 2024-01-24T23:25:45.301 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.285+0000 7f7a2f87d640 10 monclient: monmap: 2024-01-24T23:25:45.301 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:45.302 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:45.302 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:45.302 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:45.302 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:45.302 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:45.302 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:45.302 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:45.302 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:45.302 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:45.302 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:45.302 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:45.302 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:45.302 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:45.302 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:45.302 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.302 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2f87d640 1 Processor -- start 2024-01-24T23:25:45.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2f87d640 1 -- start start 2024-01-24T23:25:45.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2f87d640 10 monclient: init 2024-01-24T23:25:45.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2f87d640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:45.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2f87d640 10 monclient: _add_conns ranks=[6,2,0,4,8,3,1,7,5] 2024-01-24T23:25:45.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2f87d640 10 monclient(hunting): picked mon.noname-d con 0x7f7a28133bf0 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:45.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2f87d640 10 monclient(hunting): picked mon.noname-b con 0x7f7a28133fd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:45.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2f87d640 10 monclient(hunting): picked mon.noname-a con 0x7f7a28134430 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:45.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2f87d640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a28139450 con 0x7f7a28134430 2024-01-24T23:25:45.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2f87d640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a280a5960 con 0x7f7a28133fd0 2024-01-24T23:25:45.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2f87d640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a2809c8f0 con 0x7f7a28133bf0 2024-01-24T23:25:45.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2f87d640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:45.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2f87d640 10 monclient(hunting): authenticate will time out at 1224.212647s 2024-01-24T23:25:45.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2ce18640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f7a28133fd0 legacy=0x7f7a281d02e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:25:45.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2d619640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f7a28133bf0 0x7f7a28138360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:35076/0 (socket says 172.21.15.115:35076) 2024-01-24T23:25:45.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2de1a640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f7a28134430 0x7f7a281d3a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:53186/0 (socket says 172.21.15.115:53186) 2024-01-24T23:25:45.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2d619640 1 -- 172.21.15.115:0/1407614552 learned_addr learned my addr 172.21.15.115:0/1407614552 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:45.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3242988291 0 0) 0x7f7a2809c8f0 con 0x7f7a28133bf0 2024-01-24T23:25:45.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:45.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:45.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:45.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient(hunting): my global_id is 6207 2024-01-24T23:25:45.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a0c003530 con 0x7f7a28133bf0 2024-01-24T23:25:45.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1341432128 0 0) 0x7f7a28139450 con 0x7f7a28134430 2024-01-24T23:25:45.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:45.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:45.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:45.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient(hunting): my global_id is 94374 2024-01-24T23:25:45.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a2809c8f0 con 0x7f7a28134430 2024-01-24T23:25:45.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3047257828 0 0) 0x7f7a0c003530 con 0x7f7a28133bf0 2024-01-24T23:25:45.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7a28139450 con 0x7f7a28133bf0 2024-01-24T23:25:45.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7a24004370 con 0x7f7a28133bf0 2024-01-24T23:25:45.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:25:45.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient(hunting): dump: 2024-01-24T23:25:45.305 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:45.305 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.305 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.305 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.305 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.305 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:45.305 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.305 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.305 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.305 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.305 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.305 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.305 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.305 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.306 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.306 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:45.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2135690452 0 0) 0x7f7a2809c8f0 con 0x7f7a28134430 2024-01-24T23:25:45.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7a0c003530 con 0x7f7a28134430 2024-01-24T23:25:45.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7a20001c80 con 0x7f7a28134430 2024-01-24T23:25:45.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:45.306 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient(hunting): dump: 2024-01-24T23:25:45.306 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:45.306 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.306 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.306 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.306 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.306 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:45.307 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.307 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.307 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.307 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.307 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.307 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.307 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.307 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.307 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.307 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3540567799 0 0) 0x7f7a28139450 con 0x7f7a28133bf0 2024-01-24T23:25:45.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 >> v1:172.21.15.115:6790/0 conn(0x7f7a28133fd0 legacy=0x7f7a281d02e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:45.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 >> v1:172.21.15.115:6789/0 conn(0x7f7a28134430 legacy=0x7f7a281d3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient: _finish_hunting 0 2024-01-24T23:25:45.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 monclient: found mon.d 2024-01-24T23:25:45.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:45.307 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:25:45.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2813ae20 con 0x7f7a28133bf0 2024-01-24T23:25:45.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient: _finish_auth 0 2024-01-24T23:25:45.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient: _check_auth_tickets 2024-01-24T23:25:45.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:45.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2f87d640 5 monclient: authenticate success, global_id 6207 2024-01-24T23:25:45.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2f87d640 10 monclient: _renew_subs 2024-01-24T23:25:45.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2f87d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:25:45.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2f87d640 1 -- 172.21.15.115:0/1407614552 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a2813a580 con 0x7f7a28133bf0 2024-01-24T23:25:45.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2f87d640 10 monclient: _renew_subs 2024-01-24T23:25:45.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2f87d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:25:45.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2f87d640 1 -- 172.21.15.115:0/1407614552 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a28138a70 con 0x7f7a28133bf0 2024-01-24T23:25:45.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7a24004590 con 0x7f7a28133bf0 2024-01-24T23:25:45.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:45.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7a24004ac0 con 0x7f7a28133bf0 2024-01-24T23:25:45.308 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1f7fe640 4 set_mon_vals no callback set 2024-01-24T23:25:45.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:45.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient: dump: 2024-01-24T23:25:45.309 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:45.309 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.309 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.309 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.309 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.309 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:45.309 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.309 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.309 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.309 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.309 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.309 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.309 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.309 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.310 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.310 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2f87d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:45.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2f87d640 1 -- 172.21.15.115:0/1407614552 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7a080065d0 con 0x7f7a08002870 2024-01-24T23:25:45.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f7a24004c20 con 0x7f7a28133bf0 2024-01-24T23:25:45.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(130..130 src has 1..130) v4 ==== 6377+0+0 (unknown 290309911 0 0) 0x7f7a24044140 con 0x7f7a28133bf0 2024-01-24T23:25:45.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376230784 0 0) 0x7f7a080065d0 con 0x7f7a08002870 2024-01-24T23:25:45.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:45.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:45.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:45.310 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient(hunting): my global_id is 6006 2024-01-24T23:25:45.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a28139450 con 0x7f7a08002870 2024-01-24T23:25:45.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a2de1a640 10 monclient: get_auth_request con 0x7f7a0c0376f0 auth_method 0 2024-01-24T23:25:45.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2195518899 0 0) 0x7f7a28139450 con 0x7f7a08002870 2024-01-24T23:25:45.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7a080065d0 con 0x7f7a08002870 2024-01-24T23:25:45.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7a18004bf0 con 0x7f7a08002870 2024-01-24T23:25:45.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:45.311 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 10 monclient: dump: 2024-01-24T23:25:45.311 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:45.311 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.311 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.311 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.311 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.311 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:45.311 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.311 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.312 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.312 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.312 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.312 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.312 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.312 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.312 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.312 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3948786691 0 0) 0x7f7a080065d0 con 0x7f7a08002870 2024-01-24T23:25:45.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.289+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 --> v1:172.21.15.132:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7a0800a120 con 0x7f7a08002870 2024-01-24T23:25:45.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.293+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f7a180016b0 con 0x7f7a08002870 2024-01-24T23:25:45.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.293+0000 7f7a1dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:45.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.293+0000 7f7a1dffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:45.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.293+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 >> v1:172.21.15.132:6790/0 conn(0x7f7a08002870 legacy=0x7f7a08004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.293+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 >> v1:172.21.15.132:6790/0 conn(0x7f7a08002870 legacy=0x7f7a08004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:45.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a2f87d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:25:45.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a2f87d640 1 -- 172.21.15.115:0/1407614552 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7a08009df0 con 0x7f7a080090b0 2024-01-24T23:25:45.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 300408596 0 0) 0x7f7a08009df0 con 0x7f7a080090b0 2024-01-24T23:25:45.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a1dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:45.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a1dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:45.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a1dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:45.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a1dffb640 10 monclient(hunting): my global_id is 6015 2024-01-24T23:25:45.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a080065d0 con 0x7f7a080090b0 2024-01-24T23:25:45.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3048653775 0 0) 0x7f7a080065d0 con 0x7f7a080090b0 2024-01-24T23:25:45.314 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7a08009df0 con 0x7f7a080090b0 2024-01-24T23:25:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7a18005340 con 0x7f7a080090b0 2024-01-24T23:25:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a1dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a1dffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a1dffb640 10 monclient: dump: 2024-01-24T23:25:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.316 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.316 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.316 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.316 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.316 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 233073859 0 0) 0x7f7a08009df0 con 0x7f7a080090b0 2024-01-24T23:25:45.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 --> v1:172.21.15.132:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7a0800c310 con 0x7f7a080090b0 2024-01-24T23:25:45.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7680 (unknown 0 0 3244491037) 0x7f7a18007560 con 0x7f7a080090b0 2024-01-24T23:25:45.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a1dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:25:45.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a1dffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:45.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 >> v1:172.21.15.132:6790/0 conn(0x7f7a080090b0 legacy=0x7f7a08009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.316 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a1dffb640 1 -- 172.21.15.115:0/1407614552 >> v1:172.21.15.132:6790/0 conn(0x7f7a080090b0 legacy=0x7f7a08009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:45.317 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:25:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:25:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 34, 2024-01-24T23:25:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:25:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:25:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:25:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:25:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:25:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:25:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:25:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 1, 2024-01-24T23:25:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:25:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:25:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:25:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:25:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:25:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:25:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:25:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:25:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:25:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:25:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:25:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:25:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:25:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:25:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:25:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:25:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:25:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:25:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:25:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:25:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:25:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:25:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:25:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:25:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:25:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:25:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:25:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:25:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:25:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:25:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:25:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:25:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:25:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:25:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:25:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:25:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:25:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:25:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:25:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:25:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:25:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:25:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:25:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:25:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:25:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:25:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:25:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:25:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:25:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:25:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:25:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 4 2024-01-24T23:25:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:25:45.335 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:25:45.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a2f87d640 1 -- 172.21.15.115:0/1407614552 >> v1:172.21.15.132:6812/1835401922 conn(0x7f7a0c0376f0 legacy=0x7f7a0c039bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a2f87d640 10 monclient: shutdown 2024-01-24T23:25:45.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a2f87d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:45.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a2f87d640 1 -- 172.21.15.115:0/1407614552 >> v1:172.21.15.115:6792/0 conn(0x7f7a28133bf0 legacy=0x7f7a28138360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.297+0000 7f7a2de1a640 1 -- 172.21.15.115:0/1407614552 reap_dead start 2024-01-24T23:25:45.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.301+0000 7f7a2de1a640 1 -- 172.21.15.115:0/1407614552 reap_dead start 2024-01-24T23:25:45.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.301+0000 7f7a2f87d640 1 -- 172.21.15.115:0/1407614552 shutdown_connections 2024-01-24T23:25:45.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.301+0000 7f7a2f87d640 1 -- 172.21.15.115:0/1407614552 >> 172.21.15.115:0/1407614552 conn(0x7f7a280a9910 msgr2=0x7f7a28140050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:45.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.301+0000 7f7a2f87d640 1 -- 172.21.15.115:0/1407614552 shutdown_connections 2024-01-24T23:25:45.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.301+0000 7f7a2f87d640 1 -- 172.21.15.115:0/1407614552 wait complete. 2024-01-24T23:25:45.337 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-01-24T23:25:45.337 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon h 2024-01-24T23:25:45.338 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-01-24T23:25:45.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 10 monclient: get_monmap_and_config 2024-01-24T23:25:45.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 10 monclient: build_initial_monmap 2024-01-24T23:25:45.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 10 monclient: monmap: 2024-01-24T23:25:45.359 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:45.359 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.360 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:45.361007+0000 2024-01-24T23:25:45.360 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:45.361007+0000 2024-01-24T23:25:45.360 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:45.360 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:45.360 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:45.360 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:45.360 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:45.360 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:45.360 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:45.360 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:45.360 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:45.360 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:45.360 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:45.361 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:45.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 1 Processor -- start 2024-01-24T23:25:45.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 1 -- start start 2024-01-24T23:25:45.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 10 monclient: init 2024-01-24T23:25:45.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:45.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 10 monclient: _add_conns ranks=[2,7,3,8,6,1,0,5,4] 2024-01-24T23:25:45.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 10 monclient(hunting): picked mon.noname-b con 0x7fa8180a2e80 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:45.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 10 monclient(hunting): picked mon.noname-i con 0x7fa8180a4150 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:45.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 10 monclient(hunting): picked mon.noname-g con 0x7fa81813d4f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:45.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa818064700 con 0x7fa8180a2e80 2024-01-24T23:25:45.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8180a4f40 con 0x7fa81813d4f0 2024-01-24T23:25:45.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8180a0e80 con 0x7fa8180a4150 2024-01-24T23:25:45.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:45.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 10 monclient(hunting): authenticate will time out at 1232.685757s 2024-01-24T23:25:45.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa816ffd640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fa8180a4150 legacy=0x7fa81813bdd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:25:45.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8177fe640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fa8180a2e80 legacy=0x7fa8180a3260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:25:45.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa817fff640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fa81813d4f0 0x7fa81813f8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:53156/0 (socket says 172.21.15.132:53156) 2024-01-24T23:25:45.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa817fff640 1 -- 172.21.15.132:0/592268490 learned_addr learned my addr 172.21.15.132:0/592268490 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:45.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 1 -- 172.21.15.132:0/592268490 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 328866937 0 0) 0x7fa8180a4f40 con 0x7fa81813d4f0 2024-01-24T23:25:45.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:45.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:45.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:45.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 10 monclient(hunting): my global_id is 6024 2024-01-24T23:25:45.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 1 -- 172.21.15.132:0/592268490 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa800003530 con 0x7fa81813d4f0 2024-01-24T23:25:45.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 1 -- 172.21.15.132:0/592268490 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3481884730 0 0) 0x7fa800003530 con 0x7fa81813d4f0 2024-01-24T23:25:45.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 1 -- 172.21.15.132:0/592268490 >> v1:172.21.15.132:6792/0 conn(0x7fa8180a4150 legacy=0x7fa81813bdd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:45.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 1 -- 172.21.15.132:0/592268490 >> v1:172.21.15.115:6790/0 conn(0x7fa8180a2e80 legacy=0x7fa8180a3260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:45.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 10 monclient: _finish_hunting 0 2024-01-24T23:25:45.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 1 monclient: found mon.noname-g 2024-01-24T23:25:45.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:45.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:25:45.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 1 -- 172.21.15.132:0/592268490 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8180a1440 con 0x7fa81813d4f0 2024-01-24T23:25:45.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 10 monclient: _finish_auth 0 2024-01-24T23:25:45.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 10 monclient: _check_auth_tickets 2024-01-24T23:25:45.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:45.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 1 -- 172.21.15.132:0/592268490 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa8080017c0 con 0x7fa81813d4f0 2024-01-24T23:25:45.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 5 monclient: authenticate success, global_id 6024 2024-01-24T23:25:45.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:45.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:25:45.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 10 monclient: dump: 2024-01-24T23:25:45.369 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:45.369 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.369 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.370 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.370 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.370 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:45.370 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.370 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.370 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.370 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.370 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.370 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.370 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.370 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.370 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.370 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:45.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:45.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 1 -- 172.21.15.132:0/592268490 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa808001ae0 con 0x7fa81813d4f0 2024-01-24T23:25:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 1 -- 172.21.15.132:0/592268490 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa808004bf0 con 0x7fa81813d4f0 2024-01-24T23:25:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa8167fc640 10 monclient: dump: 2024-01-24T23:25:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.372 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.372 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.372 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.372 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.372 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.372 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:45.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:45.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 4 set_mon_vals no callback set 2024-01-24T23:25:45.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 10 monclient: shutdown 2024-01-24T23:25:45.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:45.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 1 -- 172.21.15.132:0/592268490 >> v1:172.21.15.132:6790/0 conn(0x7fa81813d4f0 legacy=0x7fa81813f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 1 -- 172.21.15.132:0/592268490 shutdown_connections 2024-01-24T23:25:45.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 1 -- 172.21.15.132:0/592268490 >> 172.21.15.132:0/592268490 conn(0x7fa818071c20 msgr2=0x7fa818072030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:45.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 1 -- 172.21.15.132:0/592268490 shutdown_connections 2024-01-24T23:25:45.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 1 -- 172.21.15.132:0/592268490 wait complete. 2024-01-24T23:25:45.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 10 monclient: build_initial_monmap 2024-01-24T23:25:45.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.360+0000 7fa81e01b640 10 monclient: monmap: 2024-01-24T23:25:45.373 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:45.373 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:45.373 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:45.373 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:45.373 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:45.373 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:45.373 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:45.373 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:45.373 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:45.373 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:45.374 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:45.374 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:45.374 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:45.374 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:45.374 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:45.374 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:45.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa81e01b640 1 Processor -- start 2024-01-24T23:25:45.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa81e01b640 1 -- start start 2024-01-24T23:25:45.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa81e01b640 10 monclient: init 2024-01-24T23:25:45.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa81e01b640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:45.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa81e01b640 10 monclient: _add_conns ranks=[5,3,6,4,7,1,8,2,0] 2024-01-24T23:25:45.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa81e01b640 10 monclient(hunting): picked mon.noname-h con 0x7fa8180a2e80 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:45.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa81e01b640 10 monclient(hunting): picked mon.noname-g con 0x7fa8180a4150 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:45.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa81e01b640 10 monclient(hunting): picked mon.noname-d con 0x7fa81813d4f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa81e01b640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa81809f260 con 0x7fa8180a4150 2024-01-24T23:25:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa81e01b640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8180a5820 con 0x7fa8180a2e80 2024-01-24T23:25:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa81e01b640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8180a2c50 con 0x7fa81813d4f0 2024-01-24T23:25:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa81e01b640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa81e01b640 10 monclient(hunting): authenticate will time out at 1232.688831s 2024-01-24T23:25:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa8177fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fa8180a2e80 0x7fa8181cb290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:58612/0 (socket says 172.21.15.132:58612) 2024-01-24T23:25:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa8177fe640 1 -- 172.21.15.132:0/1361378594 learned_addr learned my addr 172.21.15.132:0/1361378594 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 1 -- 172.21.15.132:0/1361378594 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1290106633 0 0) 0x7fa8180a5820 con 0x7fa8180a2e80 2024-01-24T23:25:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient(hunting): my global_id is 5972 2024-01-24T23:25:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 1 -- 172.21.15.132:0/1361378594 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa804003530 con 0x7fa8180a2e80 2024-01-24T23:25:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 1 -- 172.21.15.132:0/1361378594 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3098215138 0 0) 0x7fa81809f260 con 0x7fa8180a4150 2024-01-24T23:25:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient(hunting): my global_id is 6033 2024-01-24T23:25:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 1 -- 172.21.15.132:0/1361378594 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8180a5820 con 0x7fa8180a4150 2024-01-24T23:25:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 1 -- 172.21.15.132:0/1361378594 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1113927635 0 0) 0x7fa8180a2c50 con 0x7fa81813d4f0 2024-01-24T23:25:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient(hunting): my global_id is 6216 2024-01-24T23:25:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 1 -- 172.21.15.132:0/1361378594 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa81809f260 con 0x7fa81813d4f0 2024-01-24T23:25:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 1 -- 172.21.15.132:0/1361378594 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1518563948 0 0) 0x7fa804003530 con 0x7fa8180a2e80 2024-01-24T23:25:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 1 -- 172.21.15.132:0/1361378594 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa8180a2c50 con 0x7fa8180a2e80 2024-01-24T23:25:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 1 -- 172.21.15.132:0/1361378594 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa810004680 con 0x7fa8180a2e80 2024-01-24T23:25:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:25:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient(hunting): dump: 2024-01-24T23:25:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.378 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 1 -- 172.21.15.132:0/1361378594 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2582055615 0 0) 0x7fa8180a5820 con 0x7fa8180a4150 2024-01-24T23:25:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 1 -- 172.21.15.132:0/1361378594 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa804003530 con 0x7fa8180a4150 2024-01-24T23:25:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 1 -- 172.21.15.132:0/1361378594 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa80c0041f0 con 0x7fa8180a4150 2024-01-24T23:25:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient(hunting): dump: 2024-01-24T23:25:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.379 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 1 -- 172.21.15.132:0/1361378594 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3366509624 0 0) 0x7fa8180a2c50 con 0x7fa8180a2e80 2024-01-24T23:25:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 1 -- 172.21.15.132:0/1361378594 >> v1:172.21.15.115:6792/0 conn(0x7fa81813d4f0 legacy=0x7fa8181d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 1 -- 172.21.15.132:0/1361378594 >> v1:172.21.15.132:6790/0 conn(0x7fa8180a4150 legacy=0x7fa8181381f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient: _finish_hunting 0 2024-01-24T23:25:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 1 monclient: found mon.h 2024-01-24T23:25:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 1 -- 172.21.15.132:0/1361378594 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81809f950 con 0x7fa8180a2e80 2024-01-24T23:25:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient: _finish_auth 0 2024-01-24T23:25:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient: _check_auth_tickets 2024-01-24T23:25:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa81e01b640 5 monclient: authenticate success, global_id 5972 2024-01-24T23:25:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa81e01b640 10 monclient: _renew_subs 2024-01-24T23:25:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa81e01b640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa81e01b640 1 -- 172.21.15.132:0/1361378594 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa818134400 con 0x7fa8180a2e80 2024-01-24T23:25:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa81e01b640 10 monclient: _renew_subs 2024-01-24T23:25:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa81e01b640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa81e01b640 1 -- 172.21.15.132:0/1361378594 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa818133510 con 0x7fa8180a2e80 2024-01-24T23:25:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 1 -- 172.21.15.132:0/1361378594 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa8100048a0 con 0x7fa8180a2e80 2024-01-24T23:25:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 1 -- 172.21.15.132:0/1361378594 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa810004d50 con 0x7fa8180a2e80 2024-01-24T23:25:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa815ffb640 4 set_mon_vals no callback set 2024-01-24T23:25:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 10 monclient: dump: 2024-01-24T23:25:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.382 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 1 -- 172.21.15.132:0/1361378594 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fa810004fb0 con 0x7fa8180a2e80 2024-01-24T23:25:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa81e01b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa81e01b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa81e01b640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa81e01b640 1 -- 172.21.15.132:0/1361378594 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f8005090 con 0x7fa8180a2e80 2024-01-24T23:25:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa7fffff640 1 -- 172.21.15.132:0/1361378594 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(130..130 src has 1..130) v4 ==== 6377+0+0 (unknown 290309911 0 0) 0x7fa8100444d0 con 0x7fa8180a2e80 2024-01-24T23:25:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.364+0000 7fa816ffd640 10 monclient: get_auth_request con 0x7fa804037970 auth_method 0 2024-01-24T23:25:45.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.368+0000 7fa7fffff640 1 -- 172.21.15.132:0/1361378594 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fa810014d10 con 0x7fa8180a2e80 2024-01-24T23:25:45.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.368+0000 7fa7fffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:45.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.368+0000 7fa7fffff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:45.450 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 10 monclient: get_monmap_and_config 2024-01-24T23:25:45.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 10 monclient: build_initial_monmap 2024-01-24T23:25:45.451 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 10 monclient: monmap: 2024-01-24T23:25:45.451 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:45.451 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.451 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:25:45.451650+0000 2024-01-24T23:25:45.451 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:25:45.451650+0000 2024-01-24T23:25:45.451 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:45.451 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:45.451 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:45.451 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:45.451 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:45.451 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:45.451 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:45.452 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:45.452 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:45.452 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:45.452 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:45.452 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 1 Processor -- start 2024-01-24T23:25:45.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 1 -- start start 2024-01-24T23:25:45.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 10 monclient: init 2024-01-24T23:25:45.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:45.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 10 monclient: _add_conns ranks=[1,3,8,5,6,4,7,0,2] 2024-01-24T23:25:45.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 10 monclient(hunting): picked mon.noname-f con 0x7fb9ec133bf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:45.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 10 monclient(hunting): picked mon.noname-g con 0x7fb9ec133fd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:45.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 10 monclient(hunting): picked mon.noname-e con 0x7fb9ec1343b0 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:45.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9ec04ed70 con 0x7fb9ec133bf0 2024-01-24T23:25:45.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9ec0a4b60 con 0x7fb9ec133fd0 2024-01-24T23:25:45.457 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9ec09ae50 con 0x7fb9ec1343b0 2024-01-24T23:25:45.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:45.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 10 monclient(hunting): authenticate will time out at 1224.374423s 2024-01-24T23:25:45.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f08b3640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fb9ec133bf0 legacy=0x7fb9ec09bb10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:45.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f10b4640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fb9ec1343b0 0x7fb9ec13f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:54968/0 (socket says 172.21.15.115:54968) 2024-01-24T23:25:45.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f10b4640 1 -- 172.21.15.115:0/4191637418 learned_addr learned my addr 172.21.15.115:0/4191637418 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:45.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 1 -- 172.21.15.115:0/4191637418 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929412776 0 0) 0x7fb9ec09ae50 con 0x7fb9ec1343b0 2024-01-24T23:25:45.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:45.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:45.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:45.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 10 monclient(hunting): my global_id is 85274 2024-01-24T23:25:45.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 1 -- 172.21.15.115:0/4191637418 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9d4003530 con 0x7fb9ec1343b0 2024-01-24T23:25:45.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 1 -- 172.21.15.115:0/4191637418 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 275166671 0 0) 0x7fb9d4003530 con 0x7fb9ec1343b0 2024-01-24T23:25:45.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 1 -- 172.21.15.115:0/4191637418 >> v1:172.21.15.132:6790/0 conn(0x7fb9ec133fd0 legacy=0x7fb9ec13bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 1 -- 172.21.15.115:0/4191637418 >> v1:172.21.15.132:6789/0 conn(0x7fb9ec133bf0 legacy=0x7fb9ec09bb10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:45.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:25:45.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 1 monclient: found mon.noname-e 2024-01-24T23:25:45.458 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:45.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:25:45.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 1 -- 172.21.15.115:0/4191637418 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ec09b130 con 0x7fb9ec1343b0 2024-01-24T23:25:45.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 10 monclient: _finish_auth 0 2024-01-24T23:25:45.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:25:45.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:45.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 1 -- 172.21.15.115:0/4191637418 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb9e4001b40 con 0x7fb9ec1343b0 2024-01-24T23:25:45.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:25:45.459 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 10 monclient: dump: 2024-01-24T23:25:45.459 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:45.459 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.459 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.459 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.459 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.459 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:45.459 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.460 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.460 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.460 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.460 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.460 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.460 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.460 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.460 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.460 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 5 monclient: authenticate success, global_id 85274 2024-01-24T23:25:45.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:45.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 1 -- 172.21.15.115:0/4191637418 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb9e4001e60 con 0x7fb9ec1343b0 2024-01-24T23:25:45.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:45.460 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:45.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 1 -- 172.21.15.115:0/4191637418 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb9e4004f70 con 0x7fb9ec1343b0 2024-01-24T23:25:45.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 4 set_mon_vals no callback set 2024-01-24T23:25:45.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:45.461 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9eb7fe640 10 monclient: dump: 2024-01-24T23:25:45.461 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:45.461 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.461 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.461 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.461 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.461 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:45.461 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.461 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.461 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.461 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.461 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.462 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.462 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.462 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.462 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.462 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 10 monclient: shutdown 2024-01-24T23:25:45.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:45.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 1 -- 172.21.15.115:0/4191637418 >> v1:172.21.15.115:6793/0 conn(0x7fb9ec1343b0 legacy=0x7fb9ec13f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 1 -- 172.21.15.115:0/4191637418 shutdown_connections 2024-01-24T23:25:45.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 1 -- 172.21.15.115:0/4191637418 >> 172.21.15.115:0/4191637418 conn(0x7fb9ec0662f0 msgr2=0x7fb9ec066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:45.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 1 -- 172.21.15.115:0/4191637418 shutdown_connections 2024-01-24T23:25:45.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.449+0000 7fb9f2b17640 1 -- 172.21.15.115:0/4191637418 wait complete. 2024-01-24T23:25:45.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f2b17640 10 monclient: build_initial_monmap 2024-01-24T23:25:45.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f2b17640 10 monclient: monmap: 2024-01-24T23:25:45.462 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:25:45.462 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:45.462 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:25:45.463 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:25:45.463 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:45.463 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:25:45.463 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:45.463 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:45.463 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:45.463 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:45.463 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:45.463 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:45.463 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:45.463 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:45.463 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:45.463 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f2b17640 1 Processor -- start 2024-01-24T23:25:45.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f2b17640 1 -- start start 2024-01-24T23:25:45.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f2b17640 10 monclient: init 2024-01-24T23:25:45.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f2b17640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:45.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f2b17640 10 monclient: _add_conns ranks=[8,7,3,5,0,4,2,6,1] 2024-01-24T23:25:45.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f2b17640 10 monclient(hunting): picked mon.noname-e con 0x7fb9ec133bf0 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:45.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f2b17640 10 monclient(hunting): picked mon.noname-i con 0x7fb9ec133fd0 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:45.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f2b17640 10 monclient(hunting): picked mon.noname-g con 0x7fb9ec1343b0 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:45.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f2b17640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9ec0a5990 con 0x7fb9ec1343b0 2024-01-24T23:25:45.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f2b17640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9ec120660 con 0x7fb9ec133fd0 2024-01-24T23:25:45.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f2b17640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9ec0a24c0 con 0x7fb9ec133bf0 2024-01-24T23:25:45.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f2b17640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:45.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f2b17640 10 monclient(hunting): authenticate will time out at 1224.377337s 2024-01-24T23:25:45.464 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f08b3640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fb9ec133bf0 0x7fb9ec132f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:54978/0 (socket says 172.21.15.115:54978) 2024-01-24T23:25:45.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f08b3640 1 -- 172.21.15.115:0/898866522 learned_addr learned my addr 172.21.15.115:0/898866522 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:25:45.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9ebfff640 1 -- 172.21.15.115:0/898866522 >> v1:172.21.15.132:6792/0 conn(0x7fb9ec133fd0 legacy=0x7fb9ec1d0640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:25:45.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3366698249 0 0) 0x7fb9ec0a24c0 con 0x7fb9ec133bf0 2024-01-24T23:25:45.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 10 monclient(hunting): _negotiate 2024-01-24T23:25:45.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:45.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:45.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 10 monclient(hunting): my global_id is 85283 2024-01-24T23:25:45.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9d8003530 con 0x7fb9ec133bf0 2024-01-24T23:25:45.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2858881779 0 0) 0x7fb9d8003530 con 0x7fb9ec133bf0 2024-01-24T23:25:45.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb9ec0a24c0 con 0x7fb9ec133bf0 2024-01-24T23:25:45.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb9dc004380 con 0x7fb9ec133bf0 2024-01-24T23:25:45.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:25:45.465 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 10 monclient(hunting): dump: 2024-01-24T23:25:45.466 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:45.466 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.466 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.466 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.466 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.466 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:45.466 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.466 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.466 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.466 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.466 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.466 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.466 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.466 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.466 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.466 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:45.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1180776216 0 0) 0x7fb9ec0a24c0 con 0x7fb9ec133bf0 2024-01-24T23:25:45.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 >> v1:172.21.15.132:6792/0 conn(0x7fb9ec133fd0 legacy=0x7fb9ec1d0640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:45.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 >> v1:172.21.15.132:6790/0 conn(0x7fb9ec1343b0 legacy=0x7fb9ec1d3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 10 monclient: _finish_hunting 0 2024-01-24T23:25:45.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 1 monclient: found mon.e 2024-01-24T23:25:45.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:45.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:45.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ec120380 con 0x7fb9ec133bf0 2024-01-24T23:25:45.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 10 monclient: _finish_auth 0 2024-01-24T23:25:45.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 10 monclient: _check_auth_tickets 2024-01-24T23:25:45.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:45.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f2b17640 5 monclient: authenticate success, global_id 85283 2024-01-24T23:25:45.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f2b17640 10 monclient: _renew_subs 2024-01-24T23:25:45.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f2b17640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:45.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f2b17640 1 -- 172.21.15.115:0/898866522 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ec120940 con 0x7fb9ec133bf0 2024-01-24T23:25:45.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f2b17640 10 monclient: _renew_subs 2024-01-24T23:25:45.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f2b17640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:25:45.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f2b17640 1 -- 172.21.15.115:0/898866522 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ec133670 con 0x7fb9ec133bf0 2024-01-24T23:25:45.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb9dc0046a0 con 0x7fb9ec133bf0 2024-01-24T23:25:45.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:45.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb9dc004bd0 con 0x7fb9ec133bf0 2024-01-24T23:25:45.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:45.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9eaffd640 4 set_mon_vals no callback set 2024-01-24T23:25:45.468 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 10 monclient: dump: 2024-01-24T23:25:45.468 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:45.468 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.468 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.468 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.469 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.469 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:45.469 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.469 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.469 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.469 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.469 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.469 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.469 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.469 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.469 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.469 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f2b17640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:45.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f2b17640 1 -- 172.21.15.115:0/898866522 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb9cc0065d0 con 0x7fb9cc002870 2024-01-24T23:25:45.469 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fb9dc004d30 con 0x7fb9ec133bf0 2024-01-24T23:25:45.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(130..130 src has 1..130) v4 ==== 6377+0+0 (unknown 290309911 0 0) 0x7fb9dc044100 con 0x7fb9ec133bf0 2024-01-24T23:25:45.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472103517 0 0) 0x7fb9cc0065d0 con 0x7fb9cc002870 2024-01-24T23:25:45.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 10 monclient(hunting): _negotiate 2024-01-24T23:25:45.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:45.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:45.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 10 monclient(hunting): my global_id is 5981 2024-01-24T23:25:45.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9ec0a24c0 con 0x7fb9cc002870 2024-01-24T23:25:45.472 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.453+0000 7fb9f10b4640 10 monclient: get_auth_request con 0x7fb9d8036f00 auth_method 0 2024-01-24T23:25:45.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.457+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 707259110 0 0) 0x7fb9ec0a24c0 con 0x7fb9cc002870 2024-01-24T23:25:45.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.457+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb9cc0065d0 con 0x7fb9cc002870 2024-01-24T23:25:45.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.457+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb9e00018e0 con 0x7fb9cc002870 2024-01-24T23:25:45.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.457+0000 7fb9e97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.457+0000 7fb9e97fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:45.473 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.457+0000 7fb9e97fa640 10 monclient: dump: 2024-01-24T23:25:45.473 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:45.473 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.473 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.473 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.473 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.473 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:45.473 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.473 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.474 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.474 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.474 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.474 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.474 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.474 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.474 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.474 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.457+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1620401855 0 0) 0x7fb9cc0065d0 con 0x7fb9cc002870 2024-01-24T23:25:45.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.457+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 --> v1:172.21.15.132:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb9cc00a120 con 0x7fb9cc002870 2024-01-24T23:25:45.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.457+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fb9e00057d0 con 0x7fb9cc002870 2024-01-24T23:25:45.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.457+0000 7fb9e97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:45.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.457+0000 7fb9e97fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:45.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.457+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 >> v1:172.21.15.132:6791/0 conn(0x7fb9cc002870 legacy=0x7fb9cc004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.474 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.457+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 >> v1:172.21.15.132:6791/0 conn(0x7fb9cc002870 legacy=0x7fb9cc004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:45.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.461+0000 7fb9f2b17640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:25:45.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.461+0000 7fb9f2b17640 1 -- 172.21.15.115:0/898866522 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb9cc009e10 con 0x7fb9cc0090d0 2024-01-24T23:25:45.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 508044352 0 0) 0x7fb9cc009e10 con 0x7fb9cc0090d0 2024-01-24T23:25:45.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9e97fa640 10 monclient(hunting): _negotiate 2024-01-24T23:25:45.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9e97fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:45.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9e97fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:45.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9e97fa640 10 monclient(hunting): my global_id is 5990 2024-01-24T23:25:45.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9cc0065d0 con 0x7fb9cc0090d0 2024-01-24T23:25:45.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2435732342 0 0) 0x7fb9cc0065d0 con 0x7fb9cc0090d0 2024-01-24T23:25:45.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb9cc009e10 con 0x7fb9cc0090d0 2024-01-24T23:25:45.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb9e0003cb0 con 0x7fb9cc0090d0 2024-01-24T23:25:45.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9e97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9e97fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:45.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9e97fa640 10 monclient: dump: 2024-01-24T23:25:45.475 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:25:45.475 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.476 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.476 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.476 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.476 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:25:45.476 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.476 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.476 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.476 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.476 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.476 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.476 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.476 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.476 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.476 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:25:45.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1635481079 0 0) 0x7fb9cc009e10 con 0x7fb9cc0090d0 2024-01-24T23:25:45.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 --> v1:172.21.15.132:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb9cc00c310 con 0x7fb9cc0090d0 2024-01-24T23:25:45.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7512 (unknown 0 0 1997750573) 0x7fb9e0003ff0 con 0x7fb9cc0090d0 2024-01-24T23:25:45.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9e97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:25:45.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9e97fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:45.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 >> v1:172.21.15.132:6791/0 conn(0x7fb9cc0090d0 legacy=0x7fb9cc009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9e97fa640 1 -- 172.21.15.115:0/898866522 >> v1:172.21.15.132:6791/0 conn(0x7fb9cc0090d0 legacy=0x7fb9cc009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:25:45.477 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:25:45.477 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:45.477 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:45.478 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "leader", 2024-01-24T23:25:45.478 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 34, 2024-01-24T23:25:45.478 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:25:45.478 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:25:45.478 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:25:45.478 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:25:45.478 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:25:45.478 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:25:45.478 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:25:45.478 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:45.478 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 1, 2024-01-24T23:25:45.478 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:45.478 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:25:45.478 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:25:45.479 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:45.479 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:45.479 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:45.479 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:45.479 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:45.479 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:45.479 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:45.479 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:45.479 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:45.479 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:45.479 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:45.479 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:45.479 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:25:45.479 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:25:45.480 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:45.480 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:45.480 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:45.480 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:45.480 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:45.480 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:45.480 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:45.480 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:45.480 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:45.480 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:45.480 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:45.480 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.480 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:25:45.481 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:25:45.481 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:25:45.481 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:25:45.481 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:25:45.481 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:25:45.481 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:45.481 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:25:45.481 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:25:45.481 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:25:45.481 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:25:45.481 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:25:45.481 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:25:45.481 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:25:45.482 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:25:45.482 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:25:45.482 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:25:45.482 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:25:45.482 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:25:45.482 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:25:45.482 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:25:45.482 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:25:45.482 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:25:45.482 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:25:45.482 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:25:45.482 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:25:45.482 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:25:45.482 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:25:45.482 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:45.483 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:25:45.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.483 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:25:45.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.483 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:25:45.483 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:25:45.483 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.483 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.483 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.483 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:25:45.483 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.483 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.483 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.484 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:25:45.484 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:25:45.484 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.484 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.484 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.484 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:25:45.484 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:25:45.484 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.484 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.484 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.485 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:25:45.485 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.485 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.485 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.485 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:25:45.485 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:25:45.485 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.485 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.485 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.485 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:25:45.485 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:25:45.485 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.486 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.486 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.486 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:25:45.486 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.486 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.486 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.486 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:25:45.486 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:25:45.486 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.486 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.486 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.486 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:25:45.487 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:25:45.487 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.487 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.487 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.487 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:25:45.487 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.487 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.487 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.487 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:25:45.487 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:25:45.487 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.487 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.488 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.488 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:25:45.488 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:25:45.488 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.488 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.488 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.488 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:25:45.488 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.488 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.488 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.489 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:25:45.489 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:25:45.489 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.489 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.489 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.489 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:25:45.489 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:25:45.489 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.489 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.489 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.489 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:25:45.489 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.490 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.490 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.490 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:25:45.490 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:25:45.490 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.490 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.490 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.490 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:25:45.490 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:25:45.490 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.490 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.490 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.491 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:25:45.491 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.491 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.491 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.491 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:25:45.491 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:25:45.491 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.491 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.491 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.491 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:25:45.491 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:25:45.491 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.491 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.492 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.492 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:25:45.492 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.492 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.492 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.492 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:25:45.492 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:25:45.492 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.492 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.492 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.492 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:25:45.492 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:25:45.493 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:25:45.493 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:25:45.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.493 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:25:45.493 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:25:45.493 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:25:45.493 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.493 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.493 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:25:45.493 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:25:45.493 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:25:45.493 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:25:45.493 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:25:45.493 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.493 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.494 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:25:45.494 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:25:45.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.494 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:45.494 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:45.494 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:25:45.494 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.494 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:25:45.494 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:25:45.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:25:45.494 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:25:45.494 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:25:45.494 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:25:45.494 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:25:45.495 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:25:45.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:25:45.495 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:25:45.495 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:25:45.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9f2b17640 1 -- 172.21.15.115:0/898866522 >> v1:172.21.15.132:6812/1835401922 conn(0x7fb9d8036f00 legacy=0x7fb9d80393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9f2b17640 10 monclient: shutdown 2024-01-24T23:25:45.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9f2b17640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:45.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9f2b17640 1 -- 172.21.15.115:0/898866522 >> v1:172.21.15.115:6793/0 conn(0x7fb9ec133bf0 legacy=0x7fb9ec132f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9f10b4640 1 -- 172.21.15.115:0/898866522 reap_dead start 2024-01-24T23:25:45.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9f10b4640 1 -- 172.21.15.115:0/898866522 reap_dead start 2024-01-24T23:25:45.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9f2b17640 1 -- 172.21.15.115:0/898866522 shutdown_connections 2024-01-24T23:25:45.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9f2b17640 1 -- 172.21.15.115:0/898866522 >> 172.21.15.115:0/898866522 conn(0x7fb9ec0662f0 msgr2=0x7fb9ec1be7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:45.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9f2b17640 1 -- 172.21.15.115:0/898866522 shutdown_connections 2024-01-24T23:25:45.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:25:45.465+0000 7fb9f2b17640 1 -- 172.21.15.115:0/898866522 wait complete. 2024-01-24T23:25:45.497 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-01-24T23:25:45.497 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-01-24T23:25:45.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.532+0000 7fa81e01b640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:45.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.532+0000 7fa81e01b640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:45.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.532+0000 7fa81e01b640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:45.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.532+0000 7fa81e01b640 1 -- 172.21.15.132:0/1361378594 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7fa7f80058c0 con 0x7fa8180a2e80 2024-01-24T23:25:45.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.532+0000 7fa7fffff640 1 -- 172.21.15.132:0/1361378594 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v130) v1 ==== 71+0+0 (unknown 914116082 0 0) 0x7fa810014900 con 0x7fa8180a2e80 2024-01-24T23:25:45.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.532+0000 7fa7fffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:45.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.532+0000 7fa7fffff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:45.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.632+0000 7fa81e01b640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:45.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.632+0000 7fa81e01b640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:45.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.632+0000 7fa81e01b640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:45.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.632+0000 7fa81e01b640 1 -- 172.21.15.132:0/1361378594 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7fa7f8000bd0 con 0x7fa8180a2e80 2024-01-24T23:25:45.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.636+0000 7fa7fffff640 1 -- 172.21.15.132:0/1361378594 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v130) v1 ==== 71+0+0 (unknown 914116082 0 0) 0x7fa810014ae0 con 0x7fa8180a2e80 2024-01-24T23:25:45.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.636+0000 7fa7fffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:45.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.636+0000 7fa7fffff640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:25:45.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.636+0000 7fa81e01b640 1 -- 172.21.15.132:0/1361378594 >> v1:172.21.15.132:6812/1835401922 conn(0x7fa804037970 legacy=0x7fa804039e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.636+0000 7fa81e01b640 10 monclient: shutdown 2024-01-24T23:25:45.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.636+0000 7fa81e01b640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:45.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.636+0000 7fa81e01b640 1 -- 172.21.15.132:0/1361378594 >> v1:172.21.15.132:6791/0 conn(0x7fa8180a2e80 legacy=0x7fa8181cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.636+0000 7fa81e01b640 1 -- 172.21.15.132:0/1361378594 shutdown_connections 2024-01-24T23:25:45.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.636+0000 7fa81e01b640 1 -- 172.21.15.132:0/1361378594 >> 172.21.15.132:0/1361378594 conn(0x7fa818071c20 msgr2=0x7fa8180a2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:45.638 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.636+0000 7fa81e01b640 1 -- 172.21.15.132:0/1361378594 shutdown_connections 2024-01-24T23:25:45.638 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.636+0000 7fa81e01b640 1 -- 172.21.15.132:0/1361378594 wait complete. 2024-01-24T23:25:45.652 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set create-compat 2024-01-24T23:25:45.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff8272ee640 10 monclient: get_monmap_and_config 2024-01-24T23:25:45.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff8272ee640 10 monclient: build_initial_monmap 2024-01-24T23:25:45.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff8272ee640 10 monclient: monmap: 2024-01-24T23:25:45.773 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:45.773 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.773 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:45.773945+0000 2024-01-24T23:25:45.773 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:45.773945+0000 2024-01-24T23:25:45.773 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:45.773 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:45.773 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:45.773 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:45.773 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:45.773 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:45.773 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:45.773 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:45.773 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:45.773 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:45.774 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:45.774 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:45.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff8272ee640 1 Processor -- start 2024-01-24T23:25:45.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff8272ee640 1 -- start start 2024-01-24T23:25:45.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff8272ee640 10 monclient: init 2024-01-24T23:25:45.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff8272ee640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:45.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff8272ee640 10 monclient: _add_conns ranks=[1,4,0,6,5,8,3,2,7] 2024-01-24T23:25:45.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff8272ee640 10 monclient(hunting): picked mon.noname-f con 0x7ff820133b50 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:45.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff8272ee640 10 monclient(hunting): picked mon.noname-c con 0x7ff82009ec20 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:45.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff8272ee640 10 monclient(hunting): picked mon.noname-a con 0x7ff82013a1d0 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:45.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff8272ee640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff820064700 con 0x7ff82013a1d0 2024-01-24T23:25:45.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff8272ee640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff82009d9f0 con 0x7ff820133b50 2024-01-24T23:25:45.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff82508a640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7ff820133b50 legacy=0x7ff820133f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:45.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff8272ee640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff82009dc60 con 0x7ff82009ec20 2024-01-24T23:25:45.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff8272ee640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:45.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff8272ee640 10 monclient(hunting): authenticate will time out at 1233.098963s 2024-01-24T23:25:45.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff824889640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7ff82009ec20 0x7ff820138ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:57664/0 (socket says 172.21.15.132:57664) 2024-01-24T23:25:45.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff824889640 1 -- 172.21.15.132:0/3198541863 learned_addr learned my addr 172.21.15.132:0/3198541863 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:45.777 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff817fff640 1 -- 172.21.15.132:0/3198541863 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4094575851 0 0) 0x7ff82009dc60 con 0x7ff82009ec20 2024-01-24T23:25:45.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff817fff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:45.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff817fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:45.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff817fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:45.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff817fff640 10 monclient(hunting): my global_id is 94351 2024-01-24T23:25:45.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff817fff640 1 -- 172.21.15.132:0/3198541863 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff80c005740 con 0x7ff82009ec20 2024-01-24T23:25:45.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff817fff640 1 -- 172.21.15.132:0/3198541863 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 457572315 0 0) 0x7ff820064700 con 0x7ff82013a1d0 2024-01-24T23:25:45.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff817fff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:45.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff817fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:45.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff817fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:45.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff817fff640 10 monclient(hunting): my global_id is 94383 2024-01-24T23:25:45.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.772+0000 7ff817fff640 1 -- 172.21.15.132:0/3198541863 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff82009dc60 con 0x7ff82013a1d0 2024-01-24T23:25:45.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff817fff640 1 -- 172.21.15.132:0/3198541863 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2506098005 0 0) 0x7ff80c005740 con 0x7ff82009ec20 2024-01-24T23:25:45.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff817fff640 1 -- 172.21.15.132:0/3198541863 >> v1:172.21.15.132:6789/0 conn(0x7ff820133b50 legacy=0x7ff820133f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:45.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff817fff640 1 -- 172.21.15.132:0/3198541863 >> v1:172.21.15.115:6789/0 conn(0x7ff82013a1d0 legacy=0x7ff82013c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff817fff640 10 monclient: _finish_hunting 0 2024-01-24T23:25:45.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff817fff640 1 monclient: found mon.noname-c 2024-01-24T23:25:45.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff817fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:45.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff817fff640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:25:45.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff817fff640 1 -- 172.21.15.132:0/3198541863 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82009df40 con 0x7ff82009ec20 2024-01-24T23:25:45.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff817fff640 10 monclient: _finish_auth 0 2024-01-24T23:25:45.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff817fff640 10 monclient: _check_auth_tickets 2024-01-24T23:25:45.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff817fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:45.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff817fff640 1 -- 172.21.15.132:0/3198541863 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff8100017c0 con 0x7ff82009ec20 2024-01-24T23:25:45.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff817fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff817fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:25:45.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff817fff640 10 monclient: dump: 2024-01-24T23:25:45.779 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:45.779 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.779 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.779 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.779 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.779 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:45.780 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.780 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.780 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.780 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.780 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.780 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.780 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.780 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.780 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.780 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:45.780 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 5 monclient: authenticate success, global_id 94351 2024-01-24T23:25:45.780 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:45.780 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff817fff640 1 -- 172.21.15.132:0/3198541863 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff8100019e0 con 0x7ff82009ec20 2024-01-24T23:25:45.780 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff817fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:45.780 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff817fff640 1 -- 172.21.15.132:0/3198541863 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff810004bf0 con 0x7ff82009ec20 2024-01-24T23:25:45.781 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:45.781 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 4 set_mon_vals no callback set 2024-01-24T23:25:45.781 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff817fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.781 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff817fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:45.781 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff817fff640 10 monclient: dump: 2024-01-24T23:25:45.781 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:45.781 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.781 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.781 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.781 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.781 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:45.781 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.781 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.781 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.781 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.781 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.781 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.782 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.782 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.782 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.782 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:45.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 10 monclient: shutdown 2024-01-24T23:25:45.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:45.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 1 -- 172.21.15.132:0/3198541863 >> v1:172.21.15.115:6791/0 conn(0x7ff82009ec20 legacy=0x7ff820138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 1 -- 172.21.15.132:0/3198541863 shutdown_connections 2024-01-24T23:25:45.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 1 -- 172.21.15.132:0/3198541863 >> 172.21.15.132:0/3198541863 conn(0x7ff8200a5a10 msgr2=0x7ff8200a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:45.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 1 -- 172.21.15.132:0/3198541863 shutdown_connections 2024-01-24T23:25:45.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 1 -- 172.21.15.132:0/3198541863 wait complete. 2024-01-24T23:25:45.785 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 10 monclient: build_initial_monmap 2024-01-24T23:25:45.785 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 10 monclient: monmap: 2024-01-24T23:25:45.786 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:45.786 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:45.786 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:45.786 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:45.786 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:45.786 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:45.786 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:45.786 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:45.786 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:45.786 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:45.786 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:45.786 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:45.786 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:45.786 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:45.786 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:45.787 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:45.787 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 1 Processor -- start 2024-01-24T23:25:45.787 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 1 -- start start 2024-01-24T23:25:45.787 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 10 monclient: init 2024-01-24T23:25:45.787 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:45.787 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 10 monclient: _add_conns ranks=[5,0,6,2,8,4,3,1,7] 2024-01-24T23:25:45.787 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 10 monclient(hunting): picked mon.noname-h con 0x7ff82009ec20 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:45.787 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 10 monclient(hunting): picked mon.noname-a con 0x7ff820133b50 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:45.787 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 10 monclient(hunting): picked mon.noname-d con 0x7ff82013a1d0 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:45.787 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff820071ec0 con 0x7ff820133b50 2024-01-24T23:25:45.787 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff820138ca0 con 0x7ff82009ec20 2024-01-24T23:25:45.787 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8201c8160 con 0x7ff82013a1d0 2024-01-24T23:25:45.787 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:45.787 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 10 monclient(hunting): authenticate will time out at 1233.102906s 2024-01-24T23:25:45.787 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff82508a640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7ff82009ec20 0x7ff8201c7340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:58618/0 (socket says 172.21.15.132:58618) 2024-01-24T23:25:45.787 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff82508a640 1 -- 172.21.15.132:0/1314983696 learned_addr learned my addr 172.21.15.132:0/1314983696 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:45.788 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 1 -- 172.21.15.132:0/1314983696 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319472977 0 0) 0x7ff820138ca0 con 0x7ff82009ec20 2024-01-24T23:25:45.788 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:45.788 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:45.788 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:45.788 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 10 monclient(hunting): my global_id is 5999 2024-01-24T23:25:45.788 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 1 -- 172.21.15.132:0/1314983696 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff804003530 con 0x7ff82009ec20 2024-01-24T23:25:45.788 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 1 -- 172.21.15.132:0/1314983696 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 168178473 0 0) 0x7ff804003530 con 0x7ff82009ec20 2024-01-24T23:25:45.788 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 1 -- 172.21.15.132:0/1314983696 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff820138ca0 con 0x7ff82009ec20 2024-01-24T23:25:45.788 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 1 -- 172.21.15.132:0/1314983696 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff818004680 con 0x7ff82009ec20 2024-01-24T23:25:45.788 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.788 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:25:45.788 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 10 monclient(hunting): dump: 2024-01-24T23:25:45.788 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:45.788 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.789 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.789 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.789 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.789 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:45.789 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.789 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.789 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.789 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.789 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.789 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.789 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.789 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.789 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.789 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:45.789 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:45.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 1 -- 172.21.15.132:0/1314983696 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1643117703 0 0) 0x7ff820138ca0 con 0x7ff82009ec20 2024-01-24T23:25:45.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 1 -- 172.21.15.132:0/1314983696 >> v1:172.21.15.115:6792/0 conn(0x7ff82013a1d0 legacy=0x7ff8201cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 1 -- 172.21.15.132:0/1314983696 >> v1:172.21.15.115:6789/0 conn(0x7ff820133b50 legacy=0x7ff8201c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:45.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:25:45.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 1 monclient: found mon.h 2024-01-24T23:25:45.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:45.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:45.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 1 -- 172.21.15.132:0/1314983696 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8200a13b0 con 0x7ff82009ec20 2024-01-24T23:25:45.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 10 monclient: _finish_auth 0 2024-01-24T23:25:45.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:25:45.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:45.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 5 monclient: authenticate success, global_id 5999 2024-01-24T23:25:45.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 10 monclient: _renew_subs 2024-01-24T23:25:45.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:45.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 1 -- 172.21.15.132:0/1314983696 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff820133920 con 0x7ff82009ec20 2024-01-24T23:25:45.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 10 monclient: _renew_subs 2024-01-24T23:25:45.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:45.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8272ee640 1 -- 172.21.15.132:0/1314983696 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8201389c0 con 0x7ff82009ec20 2024-01-24T23:25:45.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 1 -- 172.21.15.132:0/1314983696 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff8180049a0 con 0x7ff82009ec20 2024-01-24T23:25:45.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:45.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 1 -- 172.21.15.132:0/1314983696 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff818004ed0 con 0x7ff82009ec20 2024-01-24T23:25:45.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:45.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff8177fe640 4 set_mon_vals no callback set 2024-01-24T23:25:45.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:45.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.776+0000 7ff815ffb640 10 monclient: dump: 2024-01-24T23:25:45.791 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:45.791 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:45.791 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.791 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:45.791 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:45.791 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:45.791 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:45.792 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:45.792 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:45.792 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:45.792 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:45.792 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:45.792 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:45.792 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:45.792 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:45.792 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:45.792 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.780+0000 7ff8272ee640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:45.792 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.780+0000 7ff8272ee640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:45.792 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.780+0000 7ff8272ee640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:45.792 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.780+0000 7ff815ffb640 1 -- 172.21.15.132:0/1314983696 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7ff8180050b0 con 0x7ff82009ec20 2024-01-24T23:25:45.792 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.780+0000 7ff8272ee640 1 -- 172.21.15.132:0/1314983696 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff800005090 con 0x7ff82009ec20 2024-01-24T23:25:45.792 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.780+0000 7ff815ffb640 1 -- 172.21.15.132:0/1314983696 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(130..130 src has 1..130) v4 ==== 6377+0+0 (unknown 290309911 0 0) 0x7ff8180446d0 con 0x7ff82009ec20 2024-01-24T23:25:45.793 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.780+0000 7ff824889640 10 monclient: get_auth_request con 0x7ff804037010 auth_method 0 2024-01-24T23:25:45.793 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.784+0000 7ff815ffb640 1 -- 172.21.15.132:0/1314983696 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7ff818014ce0 con 0x7ff82009ec20 2024-01-24T23:25:45.793 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.784+0000 7ff815ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:45.793 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.784+0000 7ff815ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:45.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.944+0000 7ff8272ee640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-01-24T23:25:45.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.944+0000 7ff8272ee640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-01-24T23:25:45.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.944+0000 7ff8272ee640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:45.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:45.944+0000 7ff8272ee640 1 -- 172.21.15.132:0/1314983696 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) v1 -- 0x7ff800005340 con 0x7ff82009ec20 2024-01-24T23:25:46.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.220+0000 7ff815ffb640 1 -- 172.21.15.132:0/1314983696 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 v131) v1 ==== 82+0+0 (unknown 978422085 0 0) 0x7ff81801d410 con 0x7ff82009ec20 2024-01-24T23:25:46.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.220+0000 7ff815ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-01-24T23:25:46.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.220+0000 7ff815ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:46.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.320+0000 7ff8272ee640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-01-24T23:25:46.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.320+0000 7ff8272ee640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-01-24T23:25:46.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.320+0000 7ff8272ee640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:46.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.320+0000 7ff8272ee640 1 -- 172.21.15.132:0/1314983696 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) v1 -- 0x7ff800002220 con 0x7ff82009ec20 2024-01-24T23:25:46.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.320+0000 7ff815ffb640 1 -- 172.21.15.132:0/1314983696 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 compat weight-set already created v131) v1 ==== 115+0+0 (unknown 3922984585 0 0) 0x7ff818013380 con 0x7ff82009ec20 2024-01-24T23:25:46.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.320+0000 7ff815ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-01-24T23:25:46.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.320+0000 7ff815ffb640 10 monclient: _finish_command 3 = system:0 compat weight-set already created 2024-01-24T23:25:46.323 INFO:tasks.workunit.client.0.smithi132.stderr:compat weight-set already created 2024-01-24T23:25:46.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.324+0000 7ff8272ee640 1 -- 172.21.15.132:0/1314983696 >> v1:172.21.15.132:6812/1835401922 conn(0x7ff804037010 legacy=0x7ff8040394d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:46.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.324+0000 7ff8272ee640 10 monclient: shutdown 2024-01-24T23:25:46.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.324+0000 7ff8272ee640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:46.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.324+0000 7ff8272ee640 1 -- 172.21.15.132:0/1314983696 >> v1:172.21.15.132:6791/0 conn(0x7ff82009ec20 legacy=0x7ff8201c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:46.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.324+0000 7ff8272ee640 1 -- 172.21.15.132:0/1314983696 shutdown_connections 2024-01-24T23:25:46.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.324+0000 7ff8272ee640 1 -- 172.21.15.132:0/1314983696 >> 172.21.15.132:0/1314983696 conn(0x7ff8200a5a10 msgr2=0x7ff8200a36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:46.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.324+0000 7ff8272ee640 1 -- 172.21.15.132:0/1314983696 shutdown_connections 2024-01-24T23:25:46.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.324+0000 7ff8272ee640 1 -- 172.21.15.132:0/1314983696 wait complete. 2024-01-24T23:25:46.340 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set ls 2024-01-24T23:25:46.340 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep '(compat)' 2024-01-24T23:25:46.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.456+0000 7ffbed1a2640 10 monclient: get_monmap_and_config 2024-01-24T23:25:46.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.456+0000 7ffbed1a2640 10 monclient: build_initial_monmap 2024-01-24T23:25:46.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.456+0000 7ffbed1a2640 10 monclient: monmap: 2024-01-24T23:25:46.458 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:46.458 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:46.458 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:46.459578+0000 2024-01-24T23:25:46.458 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:46.459578+0000 2024-01-24T23:25:46.458 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:46.459 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:46.459 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:46.459 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:46.459 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:46.459 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:46.459 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:46.459 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:46.459 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:46.459 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:46.459 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:46.459 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:46.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.456+0000 7ffbed1a2640 1 Processor -- start 2024-01-24T23:25:46.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.456+0000 7ffbed1a2640 1 -- start start 2024-01-24T23:25:46.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.456+0000 7ffbed1a2640 10 monclient: init 2024-01-24T23:25:46.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:46.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 10 monclient: _add_conns ranks=[7,4,8,6,1,5,3,0,2] 2024-01-24T23:25:46.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 10 monclient(hunting): picked mon.noname-i con 0x7ffbe809ff20 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:46.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 10 monclient(hunting): picked mon.noname-c con 0x7ffbe809f9c0 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:46.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 10 monclient(hunting): picked mon.noname-e con 0x7ffbe813b2c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:46.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe8064700 con 0x7ffbe809f9c0 2024-01-24T23:25:46.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe6d9d640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7ffbe809ff20 legacy=0x7ffbe80a0300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:25:46.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe809e760 con 0x7ffbe809ff20 2024-01-24T23:25:46.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe80a1160 con 0x7ffbe813b2c0 2024-01-24T23:25:46.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:46.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 10 monclient(hunting): authenticate will time out at 1233.784433s 2024-01-24T23:25:46.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe659c640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7ffbe809f9c0 0x7ffbe8139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:57666/0 (socket says 172.21.15.132:57666) 2024-01-24T23:25:46.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe659c640 1 -- 172.21.15.132:0/2946537684 learned_addr learned my addr 172.21.15.132:0/2946537684 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:46.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 1 -- 172.21.15.132:0/2946537684 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725661578 0 0) 0x7ffbe8064700 con 0x7ffbe809f9c0 2024-01-24T23:25:46.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:25:46.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:46.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:46.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 10 monclient(hunting): my global_id is 94360 2024-01-24T23:25:46.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 1 -- 172.21.15.132:0/2946537684 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbd0003530 con 0x7ffbe809f9c0 2024-01-24T23:25:46.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 1 -- 172.21.15.132:0/2946537684 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3679646587 0 0) 0x7ffbe80a1160 con 0x7ffbe813b2c0 2024-01-24T23:25:46.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:25:46.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:46.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:46.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 10 monclient(hunting): my global_id is 85292 2024-01-24T23:25:46.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 1 -- 172.21.15.132:0/2946537684 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbe8064700 con 0x7ffbe813b2c0 2024-01-24T23:25:46.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 1 -- 172.21.15.132:0/2946537684 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1991050993 0 0) 0x7ffbe8064700 con 0x7ffbe813b2c0 2024-01-24T23:25:46.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 1 -- 172.21.15.132:0/2946537684 >> v1:172.21.15.132:6792/0 conn(0x7ffbe809ff20 legacy=0x7ffbe80a0300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:46.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 1 -- 172.21.15.132:0/2946537684 >> v1:172.21.15.115:6791/0 conn(0x7ffbe809f9c0 legacy=0x7ffbe8139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:46.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 10 monclient: _finish_hunting 0 2024-01-24T23:25:46.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 1 monclient: found mon.noname-e 2024-01-24T23:25:46.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:46.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:25:46.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 1 -- 172.21.15.132:0/2946537684 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe80a0a10 con 0x7ffbe813b2c0 2024-01-24T23:25:46.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 10 monclient: _finish_auth 0 2024-01-24T23:25:46.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 10 monclient: _check_auth_tickets 2024-01-24T23:25:46.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:46.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 1 -- 172.21.15.132:0/2946537684 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ffbe0001a90 con 0x7ffbe813b2c0 2024-01-24T23:25:46.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 5 monclient: authenticate success, global_id 85292 2024-01-24T23:25:46.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:46.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:46.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:25:46.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 10 monclient: dump: 2024-01-24T23:25:46.469 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:46.469 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:46.469 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:46.469 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:46.469 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:46.469 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:46.469 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:46.469 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:46.469 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:46.469 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:46.469 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:46.469 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:46.469 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:46.470 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:46.470 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:46.470 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:46.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:46.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 1 -- 172.21.15.132:0/2946537684 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ffbe0001d20 con 0x7ffbe813b2c0 2024-01-24T23:25:46.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:46.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:46.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 4 set_mon_vals no callback set 2024-01-24T23:25:46.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 1 -- 172.21.15.132:0/2946537684 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ffbe0003ce0 con 0x7ffbe813b2c0 2024-01-24T23:25:46.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:46.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:25:46.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbe5d9b640 10 monclient: dump: 2024-01-24T23:25:46.470 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:46.470 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:46.470 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:46.470 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:46.471 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:46.471 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:46.471 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:46.471 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:46.471 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:46.471 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:46.471 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:46.471 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:46.471 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:46.471 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:46.471 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:46.471 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:46.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 10 monclient: shutdown 2024-01-24T23:25:46.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:46.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 1 -- 172.21.15.132:0/2946537684 >> v1:172.21.15.115:6793/0 conn(0x7ffbe813b2c0 legacy=0x7ffbe813d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:46.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 1 -- 172.21.15.132:0/2946537684 shutdown_connections 2024-01-24T23:25:46.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 1 -- 172.21.15.132:0/2946537684 >> 172.21.15.132:0/2946537684 conn(0x7ffbe80a5a10 msgr2=0x7ffbe80a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:46.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 1 -- 172.21.15.132:0/2946537684 shutdown_connections 2024-01-24T23:25:46.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 1 -- 172.21.15.132:0/2946537684 wait complete. 2024-01-24T23:25:46.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 10 monclient: build_initial_monmap 2024-01-24T23:25:46.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 10 monclient: monmap: 2024-01-24T23:25:46.472 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:46.472 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:46.472 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:46.472 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:46.472 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:46.472 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:46.472 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:46.472 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:46.472 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:46.472 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:46.473 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:46.473 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:46.473 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:46.473 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:46.473 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:46.473 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:46.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 1 Processor -- start 2024-01-24T23:25:46.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 1 -- start start 2024-01-24T23:25:46.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.460+0000 7ffbed1a2640 10 monclient: init 2024-01-24T23:25:46.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbed1a2640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:46.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbed1a2640 10 monclient: _add_conns ranks=[2,1,4,7,6,5,3,8,0] 2024-01-24T23:25:46.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbed1a2640 10 monclient(hunting): picked mon.noname-b con 0x7ffbe809f9c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:46.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbed1a2640 10 monclient(hunting): picked mon.noname-f con 0x7ffbe809ff20 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:46.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbed1a2640 10 monclient(hunting): picked mon.noname-c con 0x7ffbe813b2c0 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:46.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbed1a2640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe811c200 con 0x7ffbe809ff20 2024-01-24T23:25:46.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbed1a2640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe8071680 con 0x7ffbe809f9c0 2024-01-24T23:25:46.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbed1a2640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe8071960 con 0x7ffbe813b2c0 2024-01-24T23:25:46.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbed1a2640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:46.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbed1a2640 10 monclient(hunting): authenticate will time out at 1233.788128s 2024-01-24T23:25:46.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbe659c640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7ffbe809ff20 legacy=0x7ffbe81c9b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:46.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbe6d9d640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7ffbe809f9c0 legacy=0x7ffbe81c9440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:25:46.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbe759e640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7ffbe813b2c0 0x7ffbe81d16a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:57678/0 (socket says 172.21.15.132:57678) 2024-01-24T23:25:46.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbe759e640 1 -- 172.21.15.132:0/1598902874 learned_addr learned my addr 172.21.15.132:0/1598902874 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:46.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 1 -- 172.21.15.132:0/1598902874 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3865171258 0 0) 0x7ffbe8071960 con 0x7ffbe813b2c0 2024-01-24T23:25:46.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:46.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:46.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:46.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 10 monclient(hunting): my global_id is 94369 2024-01-24T23:25:46.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 1 -- 172.21.15.132:0/1598902874 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbd4003530 con 0x7ffbe813b2c0 2024-01-24T23:25:46.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 1 -- 172.21.15.132:0/1598902874 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1057034167 0 0) 0x7ffbd4003530 con 0x7ffbe813b2c0 2024-01-24T23:25:46.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 1 -- 172.21.15.132:0/1598902874 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffbe8071960 con 0x7ffbe813b2c0 2024-01-24T23:25:46.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 1 -- 172.21.15.132:0/1598902874 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ffbe0001710 con 0x7ffbe813b2c0 2024-01-24T23:25:46.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:46.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:25:46.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 10 monclient(hunting): dump: 2024-01-24T23:25:46.475 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:46.475 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:46.475 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:46.475 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:46.475 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:46.475 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:46.475 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:46.475 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:46.475 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:46.475 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:46.475 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:46.475 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:46.475 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:46.476 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:46.476 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:46.476 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:46.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:46.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 1 -- 172.21.15.132:0/1598902874 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1206241521 0 0) 0x7ffbe8071960 con 0x7ffbe813b2c0 2024-01-24T23:25:46.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 1 -- 172.21.15.132:0/1598902874 >> v1:172.21.15.115:6790/0 conn(0x7ffbe809f9c0 legacy=0x7ffbe81c9440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:46.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 1 -- 172.21.15.132:0/1598902874 >> v1:172.21.15.132:6789/0 conn(0x7ffbe809ff20 legacy=0x7ffbe81c9b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:46.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:25:46.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 1 monclient: found mon.c 2024-01-24T23:25:46.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:46.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:46.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 1 -- 172.21.15.132:0/1598902874 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe811d990 con 0x7ffbe813b2c0 2024-01-24T23:25:46.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 10 monclient: _finish_auth 0 2024-01-24T23:25:46.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:25:46.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:46.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbed1a2640 5 monclient: authenticate success, global_id 94369 2024-01-24T23:25:46.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbed1a2640 10 monclient: _renew_subs 2024-01-24T23:25:46.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbed1a2640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:46.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbed1a2640 1 -- 172.21.15.132:0/1598902874 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe81ca260 con 0x7ffbe813b2c0 2024-01-24T23:25:46.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbed1a2640 10 monclient: _renew_subs 2024-01-24T23:25:46.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbed1a2640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:46.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbed1a2640 1 -- 172.21.15.132:0/1598902874 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe811d6b0 con 0x7ffbe813b2c0 2024-01-24T23:25:46.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 1 -- 172.21.15.132:0/1598902874 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ffbe0003f10 con 0x7ffbe813b2c0 2024-01-24T23:25:46.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:46.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 1 -- 172.21.15.132:0/1598902874 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ffbe0004440 con 0x7ffbe813b2c0 2024-01-24T23:25:46.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:46.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbe559a640 4 set_mon_vals no callback set 2024-01-24T23:25:46.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:46.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 10 monclient: dump: 2024-01-24T23:25:46.477 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:46.478 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:46.478 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:46.478 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:46.478 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:46.478 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:46.478 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:46.478 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:46.478 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:46.478 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:46.478 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:46.478 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:46.478 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:46.478 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:46.478 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:46.479 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:46.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbed1a2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:46.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbed1a2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:46.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbed1a2640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:46.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbed1a2640 1 -- 172.21.15.132:0/1598902874 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc8005090 con 0x7ffbe813b2c0 2024-01-24T23:25:46.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 1 -- 172.21.15.132:0/1598902874 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7ffbe00045a0 con 0x7ffbe813b2c0 2024-01-24T23:25:46.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbcf7fe640 1 -- 172.21.15.132:0/1598902874 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(131..131 src has 1..131) v4 ==== 6709+0+0 (unknown 2109106713 0 0) 0x7ffbe0044860 con 0x7ffbe813b2c0 2024-01-24T23:25:46.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.464+0000 7ffbe6d9d640 10 monclient: get_auth_request con 0x7ffbd4036fc0 auth_method 0 2024-01-24T23:25:46.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.480+0000 7ffbcf7fe640 1 -- 172.21.15.132:0/1598902874 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7ffbe0014de0 con 0x7ffbe813b2c0 2024-01-24T23:25:46.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.480+0000 7ffbcf7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:46.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.480+0000 7ffbcf7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:46.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.640+0000 7ffbed1a2640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:46.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.640+0000 7ffbed1a2640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:46.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.640+0000 7ffbed1a2640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:46.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.640+0000 7ffbed1a2640 1 -- 172.21.15.132:0/1598902874 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7ffbc80058c0 con 0x7ffbe813b2c0 2024-01-24T23:25:46.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.644+0000 7ffbcf7fe640 1 -- 172.21.15.132:0/1598902874 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v131) v1 ==== 71+0+9 (unknown 738765755 0 1822970295) 0x7ffbe0013480 con 0x7ffbe813b2c0 2024-01-24T23:25:46.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.644+0000 7ffbcf7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:46.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.644+0000 7ffbcf7fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:46.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.744+0000 7ffbed1a2640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:46.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.744+0000 7ffbed1a2640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:46.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.744+0000 7ffbed1a2640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:46.744 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.744+0000 7ffbed1a2640 1 -- 172.21.15.132:0/1598902874 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush weight-set ls"} v 0) v1 -- 0x7ffbc8000bd0 con 0x7ffbe813b2c0 2024-01-24T23:25:46.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.744+0000 7ffbcf7fe640 1 -- 172.21.15.132:0/1598902874 <== mon.4 v1:172.21.15.115:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v131) v1 ==== 71+0+9 (unknown 738765755 0 1822970295) 0x7ffbe0014c00 con 0x7ffbe813b2c0 2024-01-24T23:25:46.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.744+0000 7ffbcf7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-01-24T23:25:46.745 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.744+0000 7ffbcf7fe640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:25:46.747 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.748+0000 7ffbed1a2640 1 -- 172.21.15.132:0/1598902874 >> v1:172.21.15.132:6812/1835401922 conn(0x7ffbd4036fc0 legacy=0x7ffbd4039480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:46.747 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.748+0000 7ffbed1a2640 10 monclient: shutdown 2024-01-24T23:25:46.747 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.748+0000 7ffbed1a2640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:46.747 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.748+0000 7ffbed1a2640 1 -- 172.21.15.132:0/1598902874 >> v1:172.21.15.115:6791/0 conn(0x7ffbe813b2c0 legacy=0x7ffbe81d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:46.747 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.748+0000 7ffbed1a2640 1 -- 172.21.15.132:0/1598902874 shutdown_connections 2024-01-24T23:25:46.748 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.748+0000 7ffbed1a2640 1 -- 172.21.15.132:0/1598902874 >> 172.21.15.132:0/1598902874 conn(0x7ffbe80a5a10 msgr2=0x7ffbe80a2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:46.748 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.748+0000 7ffbed1a2640 1 -- 172.21.15.132:0/1598902874 shutdown_connections 2024-01-24T23:25:46.748 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.748+0000 7ffbed1a2640 1 -- 172.21.15.132:0/1598902874 wait complete. 2024-01-24T23:25:46.762 INFO:tasks.workunit.client.0.smithi132.stdout:(compat) 2024-01-24T23:25:46.762 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set rm-compat 2024-01-24T23:25:46.883 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.880+0000 7f1dc5052640 10 monclient: get_monmap_and_config 2024-01-24T23:25:46.883 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.880+0000 7f1dc5052640 10 monclient: build_initial_monmap 2024-01-24T23:25:46.883 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 10 monclient: monmap: 2024-01-24T23:25:46.883 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:46.883 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:46.884 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:46.884880+0000 2024-01-24T23:25:46.884 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:46.884880+0000 2024-01-24T23:25:46.884 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:46.884 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:46.884 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:46.884 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:46.884 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:46.884 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:46.884 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:46.884 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:46.884 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:46.884 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:46.884 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:46.884 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:46.891 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 1 Processor -- start 2024-01-24T23:25:46.891 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 1 -- start start 2024-01-24T23:25:46.892 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 10 monclient: init 2024-01-24T23:25:46.892 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:46.892 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 10 monclient: _add_conns ranks=[8,6,3,0,7,5,4,1,2] 2024-01-24T23:25:46.892 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 10 monclient(hunting): picked mon.noname-e con 0x7f1dc00a0ad0 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:46.892 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 10 monclient(hunting): picked mon.noname-d con 0x7f1dc009b7e0 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:46.892 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 10 monclient(hunting): picked mon.noname-g con 0x7f1dc013d290 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:46.892 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc0064700 con 0x7f1dc013d290 2024-01-24T23:25:46.892 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc009d110 con 0x7f1dc009b7e0 2024-01-24T23:25:46.892 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc009a820 con 0x7f1dc00a0ad0 2024-01-24T23:25:46.892 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:46.892 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 10 monclient(hunting): authenticate will time out at 1234.209784s 2024-01-24T23:25:46.892 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbf59e640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f1dc013d290 0x7f1dc013f680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:53174/0 (socket says 172.21.15.132:53174) 2024-01-24T23:25:46.892 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbf59e640 1 -- 172.21.15.132:0/3392820116 learned_addr learned my addr 172.21.15.132:0/3392820116 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:46.893 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 1 -- 172.21.15.132:0/3392820116 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1320359928 0 0) 0x7f1dc0064700 con 0x7f1dc013d290 2024-01-24T23:25:46.893 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 10 monclient(hunting): _negotiate 2024-01-24T23:25:46.893 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:46.893 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:46.893 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 10 monclient(hunting): my global_id is 6060 2024-01-24T23:25:46.893 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 1 -- 172.21.15.132:0/3392820116 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dac003530 con 0x7f1dc013d290 2024-01-24T23:25:46.893 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 1 -- 172.21.15.132:0/3392820116 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1732636734 0 0) 0x7f1dac003530 con 0x7f1dc013d290 2024-01-24T23:25:46.893 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 1 -- 172.21.15.132:0/3392820116 >> v1:172.21.15.115:6793/0 conn(0x7f1dc00a0ad0 legacy=0x7f1dc00a0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:46.893 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 1 -- 172.21.15.132:0/3392820116 >> v1:172.21.15.115:6792/0 conn(0x7f1dc009b7e0 legacy=0x7f1dc00a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:46.893 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 10 monclient: _finish_hunting 0 2024-01-24T23:25:46.893 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 1 monclient: found mon.noname-g 2024-01-24T23:25:46.893 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:46.893 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:25:46.893 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 1 -- 172.21.15.132:0/3392820116 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc009ade0 con 0x7f1dc013d290 2024-01-24T23:25:46.894 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 10 monclient: _finish_auth 0 2024-01-24T23:25:46.894 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 10 monclient: _check_auth_tickets 2024-01-24T23:25:46.894 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:46.894 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 1 -- 172.21.15.132:0/3392820116 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1db8001b40 con 0x7f1dc013d290 2024-01-24T23:25:46.894 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 5 monclient: authenticate success, global_id 6060 2024-01-24T23:25:46.894 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:46.894 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:46.894 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:25:46.894 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 10 monclient: dump: 2024-01-24T23:25:46.894 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:46.894 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:46.894 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:46.894 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:46.894 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:46.895 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:46.895 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:46.895 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:46.895 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:46.895 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:46.895 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:46.895 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:46.895 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:46.895 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:46.895 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:46.895 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:46.895 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:46.895 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 1 -- 172.21.15.132:0/3392820116 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1db8001d60 con 0x7f1dc013d290 2024-01-24T23:25:46.895 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:46.895 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:46.896 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 4 set_mon_vals no callback set 2024-01-24T23:25:46.896 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 1 -- 172.21.15.132:0/3392820116 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1db8004f70 con 0x7f1dc013d290 2024-01-24T23:25:46.896 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:46.896 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:46.896 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dbe59c640 10 monclient: dump: 2024-01-24T23:25:46.896 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:46.896 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:46.896 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:46.896 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:46.896 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:46.896 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:46.896 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:46.896 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:46.897 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:46.897 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:46.897 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:46.897 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:46.897 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:46.897 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:46.897 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:46.897 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:46.897 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 10 monclient: shutdown 2024-01-24T23:25:46.897 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:46.897 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 1 -- 172.21.15.132:0/3392820116 >> v1:172.21.15.132:6790/0 conn(0x7f1dc013d290 legacy=0x7f1dc013f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:46.897 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 1 -- 172.21.15.132:0/3392820116 shutdown_connections 2024-01-24T23:25:46.897 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 1 -- 172.21.15.132:0/3392820116 >> 172.21.15.132:0/3392820116 conn(0x7f1dc00a60d0 msgr2=0x7f1dc00a64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:46.898 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 1 -- 172.21.15.132:0/3392820116 shutdown_connections 2024-01-24T23:25:46.898 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 1 -- 172.21.15.132:0/3392820116 wait complete. 2024-01-24T23:25:46.898 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 10 monclient: build_initial_monmap 2024-01-24T23:25:46.898 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.884+0000 7f1dc5052640 10 monclient: monmap: 2024-01-24T23:25:46.898 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:46.898 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:46.898 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:46.898 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:46.898 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:46.898 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:46.898 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:46.898 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:46.898 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:46.898 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:46.898 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:46.898 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:46.898 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:46.899 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:46.899 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:46.899 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:46.899 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dc5052640 1 Processor -- start 2024-01-24T23:25:46.899 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dc5052640 1 -- start start 2024-01-24T23:25:46.899 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dc5052640 10 monclient: init 2024-01-24T23:25:46.899 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dc5052640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:46.899 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dc5052640 10 monclient: _add_conns ranks=[0,3,5,2,1,8,6,7,4] 2024-01-24T23:25:46.899 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dc5052640 10 monclient(hunting): picked mon.noname-a con 0x7f1dc009b7e0 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:46.899 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dc5052640 10 monclient(hunting): picked mon.noname-g con 0x7f1dc00a0ad0 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:46.899 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dc5052640 10 monclient(hunting): picked mon.noname-h con 0x7f1dc013d290 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:46.899 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dc5052640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc013c7e0 con 0x7f1dc009b7e0 2024-01-24T23:25:46.899 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dc5052640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc006f340 con 0x7f1dc00a0ad0 2024-01-24T23:25:46.899 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dc5052640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc00a2700 con 0x7f1dc013d290 2024-01-24T23:25:46.899 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dc5052640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:46.899 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dc5052640 10 monclient(hunting): authenticate will time out at 1234.212711s 2024-01-24T23:25:46.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db7fff640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f1dc00a0ad0 0x7f1dc01d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:53184/0 (socket says 172.21.15.132:53184) 2024-01-24T23:25:46.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db7fff640 1 -- 172.21.15.132:0/3703373190 learned_addr learned my addr 172.21.15.132:0/3703373190 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:46.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 1 -- 172.21.15.132:0/3703373190 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1138404285 0 0) 0x7f1dc006f340 con 0x7f1dc00a0ad0 2024-01-24T23:25:46.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:46.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:46.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:46.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient(hunting): my global_id is 6069 2024-01-24T23:25:46.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 1 -- 172.21.15.132:0/3703373190 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1da0003530 con 0x7f1dc00a0ad0 2024-01-24T23:25:46.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 1 -- 172.21.15.132:0/3703373190 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 763677800 0 0) 0x7f1dc00a2700 con 0x7f1dc013d290 2024-01-24T23:25:46.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:46.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:46.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:46.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient(hunting): my global_id is 6008 2024-01-24T23:25:46.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 1 -- 172.21.15.132:0/3703373190 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dc006f340 con 0x7f1dc013d290 2024-01-24T23:25:46.901 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 1 -- 172.21.15.132:0/3703373190 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3839747463 0 0) 0x7f1da0003530 con 0x7f1dc00a0ad0 2024-01-24T23:25:46.901 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 1 -- 172.21.15.132:0/3703373190 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1dc00a2700 con 0x7f1dc00a0ad0 2024-01-24T23:25:46.901 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 1 -- 172.21.15.132:0/3703373190 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1da8001b20 con 0x7f1dc00a0ad0 2024-01-24T23:25:46.901 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:46.901 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:25:46.901 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient(hunting): dump: 2024-01-24T23:25:46.901 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:46.901 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:46.901 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:46.901 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:46.901 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:46.901 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:46.901 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:46.901 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:46.902 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:46.902 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:46.902 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:46.902 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:46.902 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:46.902 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:46.902 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:46.902 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:46.902 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:46.902 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 1 -- 172.21.15.132:0/3703373190 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 12631059 0 0) 0x7f1dc006f340 con 0x7f1dc013d290 2024-01-24T23:25:46.902 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 1 -- 172.21.15.132:0/3703373190 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1da0003530 con 0x7f1dc013d290 2024-01-24T23:25:46.902 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 1 -- 172.21.15.132:0/3703373190 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1db8004f70 con 0x7f1dc013d290 2024-01-24T23:25:46.902 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:46.902 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:46.902 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient(hunting): dump: 2024-01-24T23:25:46.902 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:46.903 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:46.903 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:46.903 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:46.903 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:46.903 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:46.903 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:46.903 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:46.903 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:46.903 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:46.903 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:46.903 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:46.903 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:46.903 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:46.903 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:46.903 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:46.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 1 -- 172.21.15.132:0/3703373190 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4065601790 0 0) 0x7f1dc013c7e0 con 0x7f1dc009b7e0 2024-01-24T23:25:46.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:46.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:46.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:46.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient(hunting): my global_id is 94401 2024-01-24T23:25:46.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 1 -- 172.21.15.132:0/3703373190 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dc006f340 con 0x7f1dc009b7e0 2024-01-24T23:25:46.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 1 -- 172.21.15.132:0/3703373190 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1256470084 0 0) 0x7f1dc00a2700 con 0x7f1dc00a0ad0 2024-01-24T23:25:46.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 1 -- 172.21.15.132:0/3703373190 >> v1:172.21.15.132:6791/0 conn(0x7f1dc013d290 legacy=0x7f1dc01d4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:46.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 1 -- 172.21.15.132:0/3703373190 >> v1:172.21.15.115:6789/0 conn(0x7f1dc009b7e0 legacy=0x7f1dc0071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:46.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient: _finish_hunting 0 2024-01-24T23:25:46.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 1 monclient: found mon.g 2024-01-24T23:25:46.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:46.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:46.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 1 -- 172.21.15.132:0/3703373190 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc00a21b0 con 0x7f1dc00a0ad0 2024-01-24T23:25:46.905 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient: _finish_auth 0 2024-01-24T23:25:46.905 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient: _check_auth_tickets 2024-01-24T23:25:46.905 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:46.905 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dc5052640 5 monclient: authenticate success, global_id 6069 2024-01-24T23:25:46.905 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dc5052640 10 monclient: _renew_subs 2024-01-24T23:25:46.905 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dc5052640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:46.905 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dc5052640 1 -- 172.21.15.132:0/3703373190 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc00a0490 con 0x7f1dc00a0ad0 2024-01-24T23:25:46.905 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dc5052640 10 monclient: _renew_subs 2024-01-24T23:25:46.905 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dc5052640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:46.905 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dc5052640 1 -- 172.21.15.132:0/3703373190 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc013c010 con 0x7f1dc00a0ad0 2024-01-24T23:25:46.905 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 1 -- 172.21.15.132:0/3703373190 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1da8003950 con 0x7f1dc00a0ad0 2024-01-24T23:25:46.905 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:46.905 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 1 -- 172.21.15.132:0/3703373190 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1da8003e80 con 0x7f1dc00a0ad0 2024-01-24T23:25:46.905 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:46.905 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:46.906 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dbdd9b640 4 set_mon_vals no callback set 2024-01-24T23:25:46.906 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 10 monclient: dump: 2024-01-24T23:25:46.906 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:46.906 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:46.906 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:46.906 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:46.906 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:46.906 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:46.906 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:46.906 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:46.906 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:46.906 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:46.906 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:46.906 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:46.906 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:46.906 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:46.907 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:46.907 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:46.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dc5052640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:46.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dc5052640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:46.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dc5052640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:46.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dc5052640 1 -- 172.21.15.132:0/3703373190 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da4005090 con 0x7f1dc00a0ad0 2024-01-24T23:25:46.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 1 -- 172.21.15.132:0/3703373190 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f1da8003fe0 con 0x7f1dc00a0ad0 2024-01-24T23:25:46.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1db77fe640 1 -- 172.21.15.132:0/3703373190 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(131..131 src has 1..131) v4 ==== 6709+0+0 (unknown 2109106713 0 0) 0x7f1da80443b0 con 0x7f1dc00a0ad0 2024-01-24T23:25:46.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.888+0000 7f1dbed9d640 10 monclient: get_auth_request con 0x7f1da00377a0 auth_method 0 2024-01-24T23:25:46.907 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.892+0000 7f1db77fe640 1 -- 172.21.15.132:0/3703373190 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f1da8014890 con 0x7f1dc00a0ad0 2024-01-24T23:25:46.908 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.892+0000 7f1db77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:46.908 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:46.892+0000 7f1db77fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:47.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:47.056+0000 7f1dc5052640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-01-24T23:25:47.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:47.056+0000 7f1dc5052640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-01-24T23:25:47.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:47.056+0000 7f1dc5052640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:47.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:47.056+0000 7f1dc5052640 1 -- 172.21.15.132:0/3703373190 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) v1 -- 0x7f1da4005340 con 0x7f1dc00a0ad0 2024-01-24T23:25:47.887 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:47.884+0000 7f1db77fe640 1 -- 172.21.15.132:0/3703373190 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v132) v1 ==== 78+0+0 (unknown 3456217118 0 0) 0x7f1da8012f30 con 0x7f1dc00a0ad0 2024-01-24T23:25:47.887 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:47.884+0000 7f1db77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-01-24T23:25:47.887 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:47.884+0000 7f1db77fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:47.995 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:47.992+0000 7f1dc5052640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-01-24T23:25:47.995 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:47.992+0000 7f1dc5052640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-01-24T23:25:47.995 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:47.992+0000 7f1dc5052640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:47.995 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:47.992+0000 7f1dc5052640 1 -- 172.21.15.132:0/3703373190 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) v1 -- 0x7f1da4002220 con 0x7f1dc00a0ad0 2024-01-24T23:25:48.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:48.900+0000 7f1db77fe640 1 -- 172.21.15.132:0/3703373190 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v133) v1 ==== 78+0+0 (unknown 2411788859 0 0) 0x7f1da8014890 con 0x7f1dc00a0ad0 2024-01-24T23:25:48.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:48.900+0000 7f1db77fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-01-24T23:25:48.900 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:48.900+0000 7f1db77fe640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:25:48.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:48.904+0000 7f1dc5052640 1 -- 172.21.15.132:0/3703373190 >> v1:172.21.15.132:6812/1835401922 conn(0x7f1da00377a0 legacy=0x7f1da0039c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:48.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:48.904+0000 7f1dc5052640 10 monclient: shutdown 2024-01-24T23:25:48.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:48.904+0000 7f1dc5052640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:48.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:48.904+0000 7f1dc5052640 1 -- 172.21.15.132:0/3703373190 >> v1:172.21.15.132:6790/0 conn(0x7f1dc00a0ad0 legacy=0x7f1dc01d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:48.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:48.904+0000 7f1dc5052640 1 -- 172.21.15.132:0/3703373190 shutdown_connections 2024-01-24T23:25:48.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:48.904+0000 7f1dc5052640 1 -- 172.21.15.132:0/3703373190 >> 172.21.15.132:0/3703373190 conn(0x7f1dc00a60d0 msgr2=0x7f1dc00a2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:48.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:48.904+0000 7f1dc5052640 1 -- 172.21.15.132:0/3703373190 shutdown_connections 2024-01-24T23:25:48.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:48.904+0000 7f1dc5052640 1 -- 172.21.15.132:0/3703373190 wait complete. 2024-01-24T23:25:48.923 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool create cool 2 2024-01-24T23:25:49.043 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 10 monclient: get_monmap_and_config 2024-01-24T23:25:49.043 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 10 monclient: build_initial_monmap 2024-01-24T23:25:49.043 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 10 monclient: monmap: 2024-01-24T23:25:49.044 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:49.044 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:49.044 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:49.045050+0000 2024-01-24T23:25:49.044 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:49.045050+0000 2024-01-24T23:25:49.044 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:49.044 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:49.044 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:49.044 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:49.044 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:49.044 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:49.044 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:49.044 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:49.044 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:49.044 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:49.045 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:49.045 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:49.051 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 1 Processor -- start 2024-01-24T23:25:49.051 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 1 -- start start 2024-01-24T23:25:49.051 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 10 monclient: init 2024-01-24T23:25:49.051 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:49.051 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 10 monclient: _add_conns ranks=[3,1,6,7,5,0,8,4,2] 2024-01-24T23:25:49.051 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 10 monclient(hunting): picked mon.noname-g con 0x7f282c135e10 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:49.051 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 10 monclient(hunting): picked mon.noname-f con 0x7f282c1361f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:49.051 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 10 monclient(hunting): picked mon.noname-d con 0x7f282c1365f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:49.051 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f282c064700 con 0x7f282c1361f0 2024-01-24T23:25:49.051 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f282c09c3e0 con 0x7f282c135e10 2024-01-24T23:25:49.051 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f282c0a4f90 con 0x7f282c1365f0 2024-01-24T23:25:49.051 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:49.051 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2829d9b640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f282c1361f0 legacy=0x7f282c13bed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:49.051 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 10 monclient(hunting): authenticate will time out at 1236.369793s 2024-01-24T23:25:49.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282a59c640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f282c135e10 0x7f282c09bcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:53200/0 (socket says 172.21.15.132:53200) 2024-01-24T23:25:49.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282a59c640 1 -- 172.21.15.132:0/3175297447 learned_addr learned my addr 172.21.15.132:0/3175297447 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:49.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 1 -- 172.21.15.132:0/3175297447 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1952262029 0 0) 0x7f282c09c3e0 con 0x7f282c135e10 2024-01-24T23:25:49.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 10 monclient(hunting): _negotiate 2024-01-24T23:25:49.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:49.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:49.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 10 monclient(hunting): my global_id is 6078 2024-01-24T23:25:49.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 1 -- 172.21.15.132:0/3175297447 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2814003530 con 0x7f282c135e10 2024-01-24T23:25:49.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 1 -- 172.21.15.132:0/3175297447 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1618455877 0 0) 0x7f2814003530 con 0x7f282c135e10 2024-01-24T23:25:49.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 1 -- 172.21.15.132:0/3175297447 >> v1:172.21.15.115:6792/0 conn(0x7f282c1365f0 legacy=0x7f282c13f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:49.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 1 -- 172.21.15.132:0/3175297447 >> v1:172.21.15.132:6789/0 conn(0x7f282c1361f0 legacy=0x7f282c13bed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:49.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 10 monclient: _finish_hunting 0 2024-01-24T23:25:49.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 1 monclient: found mon.noname-g 2024-01-24T23:25:49.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:49.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:25:49.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 1 -- 172.21.15.132:0/3175297447 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282c09b010 con 0x7f282c135e10 2024-01-24T23:25:49.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 10 monclient: _finish_auth 0 2024-01-24T23:25:49.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 10 monclient: _check_auth_tickets 2024-01-24T23:25:49.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:49.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 1 -- 172.21.15.132:0/3175297447 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2824001b40 con 0x7f282c135e10 2024-01-24T23:25:49.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 5 monclient: authenticate success, global_id 6078 2024-01-24T23:25:49.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:49.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:49.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:25:49.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 10 monclient: dump: 2024-01-24T23:25:49.053 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:49.053 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:49.053 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:49.054 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:49.054 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:49.054 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:49.054 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:49.054 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:49.054 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:49.054 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:49.054 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:49.054 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:49.054 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:49.054 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:49.054 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:49.054 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:49.054 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:49.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 1 -- 172.21.15.132:0/3175297447 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2824001e60 con 0x7f282c135e10 2024-01-24T23:25:49.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:49.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:49.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 4 set_mon_vals no callback set 2024-01-24T23:25:49.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 1 -- 172.21.15.132:0/3175297447 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2824004f70 con 0x7f282c135e10 2024-01-24T23:25:49.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:49.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:49.055 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f282959a640 10 monclient: dump: 2024-01-24T23:25:49.055 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:49.055 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:49.055 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:49.055 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:49.055 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:49.055 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:49.056 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:49.056 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:49.056 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:49.056 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:49.056 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:49.056 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:49.056 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:49.056 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:49.056 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:49.056 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:49.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 10 monclient: shutdown 2024-01-24T23:25:49.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:49.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 1 -- 172.21.15.132:0/3175297447 >> v1:172.21.15.132:6790/0 conn(0x7f282c135e10 legacy=0x7f282c09bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:49.056 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 1 -- 172.21.15.132:0/3175297447 shutdown_connections 2024-01-24T23:25:49.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 1 -- 172.21.15.132:0/3175297447 >> 172.21.15.132:0/3175297447 conn(0x7f282c0a6790 msgr2=0x7f282c071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:49.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 1 -- 172.21.15.132:0/3175297447 shutdown_connections 2024-01-24T23:25:49.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 1 -- 172.21.15.132:0/3175297447 wait complete. 2024-01-24T23:25:49.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 10 monclient: build_initial_monmap 2024-01-24T23:25:49.057 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.044+0000 7f2830cad640 10 monclient: monmap: 2024-01-24T23:25:49.057 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:49.057 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:49.057 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:49.057 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:49.057 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:49.057 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:49.057 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:49.057 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:49.057 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:49.058 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:49.058 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:49.058 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:49.058 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:49.058 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:49.058 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:49.058 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:49.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2830cad640 1 Processor -- start 2024-01-24T23:25:49.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2830cad640 1 -- start start 2024-01-24T23:25:49.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2830cad640 10 monclient: init 2024-01-24T23:25:49.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2830cad640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:49.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2830cad640 10 monclient: _add_conns ranks=[8,2,3,4,0,1,6,7,5] 2024-01-24T23:25:49.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2830cad640 10 monclient(hunting): picked mon.noname-e con 0x7f282c135e10 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:49.058 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2830cad640 10 monclient(hunting): picked mon.noname-b con 0x7f282c1361f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:49.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2830cad640 10 monclient(hunting): picked mon.noname-g con 0x7f282c1365f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:49.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2830cad640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f282c09ca00 con 0x7f282c1361f0 2024-01-24T23:25:49.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2830cad640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f282c09c6b0 con 0x7f282c1365f0 2024-01-24T23:25:49.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2830cad640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f282c1bf4b0 con 0x7f282c135e10 2024-01-24T23:25:49.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2830cad640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:49.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2830cad640 10 monclient(hunting): authenticate will time out at 1236.372801s 2024-01-24T23:25:49.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f282ad9d640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f282c1365f0 0x7f282c1d3b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:53214/0 (socket says 172.21.15.132:53214) 2024-01-24T23:25:49.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f282ad9d640 1 -- 172.21.15.132:0/464423815 learned_addr learned my addr 172.21.15.132:0/464423815 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:49.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2829d9b640 1 -- 172.21.15.132:0/464423815 >> v1:172.21.15.115:6790/0 conn(0x7f282c1361f0 legacy=0x7f282c1d0410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:25:49.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 1 -- 172.21.15.132:0/464423815 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1958015049 0 0) 0x7f282c09c6b0 con 0x7f282c1365f0 2024-01-24T23:25:49.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:49.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:49.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:49.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 10 monclient(hunting): my global_id is 6087 2024-01-24T23:25:49.059 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 1 -- 172.21.15.132:0/464423815 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f280c003530 con 0x7f282c1365f0 2024-01-24T23:25:49.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 1 -- 172.21.15.132:0/464423815 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 367161442 0 0) 0x7f280c003530 con 0x7f282c1365f0 2024-01-24T23:25:49.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 1 -- 172.21.15.132:0/464423815 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f282c09c6b0 con 0x7f282c1365f0 2024-01-24T23:25:49.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 1 -- 172.21.15.132:0/464423815 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f281c0019c0 con 0x7f282c1365f0 2024-01-24T23:25:49.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:49.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:25:49.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 10 monclient(hunting): dump: 2024-01-24T23:25:49.060 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:49.060 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:49.060 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:49.060 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:49.060 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:49.060 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:49.060 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:49.061 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:49.061 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:49.061 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:49.061 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:49.061 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:49.061 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:49.061 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:49.061 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:49.061 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:49.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:49.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 1 -- 172.21.15.132:0/464423815 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4041679290 0 0) 0x7f282c09c6b0 con 0x7f282c1365f0 2024-01-24T23:25:49.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 1 -- 172.21.15.132:0/464423815 >> v1:172.21.15.115:6793/0 conn(0x7f282c135e10 legacy=0x7f282c134f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:49.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 1 -- 172.21.15.132:0/464423815 >> v1:172.21.15.115:6790/0 conn(0x7f282c1361f0 legacy=0x7f282c1d0410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:49.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 10 monclient: _finish_hunting 0 2024-01-24T23:25:49.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 1 monclient: found mon.g 2024-01-24T23:25:49.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:49.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:49.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 1 -- 172.21.15.132:0/464423815 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282c1228e0 con 0x7f282c1365f0 2024-01-24T23:25:49.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 10 monclient: _finish_auth 0 2024-01-24T23:25:49.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:49.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:49.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2830cad640 5 monclient: authenticate success, global_id 6087 2024-01-24T23:25:49.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2830cad640 10 monclient: _renew_subs 2024-01-24T23:25:49.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2830cad640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:49.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2830cad640 1 -- 172.21.15.132:0/464423815 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f282c122040 con 0x7f282c1365f0 2024-01-24T23:25:49.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2830cad640 10 monclient: _renew_subs 2024-01-24T23:25:49.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2830cad640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:49.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2830cad640 1 -- 172.21.15.132:0/464423815 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f282c1beb80 con 0x7f282c1365f0 2024-01-24T23:25:49.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 1 -- 172.21.15.132:0/464423815 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f281c0038d0 con 0x7f282c1365f0 2024-01-24T23:25:49.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:49.062 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 1 -- 172.21.15.132:0/464423815 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f281c001d90 con 0x7f282c1365f0 2024-01-24T23:25:49.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:49.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2828d99640 4 set_mon_vals no callback set 2024-01-24T23:25:49.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:49.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 10 monclient: dump: 2024-01-24T23:25:49.063 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:49.063 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:49.063 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:49.063 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:49.063 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:49.063 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:49.063 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:49.063 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:49.063 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:49.063 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:49.063 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:49.063 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:49.064 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:49.064 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:49.064 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:49.064 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:49.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2830cad640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:49.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2830cad640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:49.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2830cad640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:49.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f2830cad640 1 -- 172.21.15.132:0/464423815 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2810005090 con 0x7f282c1365f0 2024-01-24T23:25:49.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 1 -- 172.21.15.132:0/464423815 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f281c003e80 con 0x7f282c1365f0 2024-01-24T23:25:49.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f281affd640 1 -- 172.21.15.132:0/464423815 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(133..133 src has 1..133) v4 ==== 6377+0+0 (unknown 1005816120 0 0) 0x7f281c044210 con 0x7f282c1365f0 2024-01-24T23:25:49.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.048+0000 7f282a59c640 10 monclient: get_auth_request con 0x7f280c036ea0 auth_method 0 2024-01-24T23:25:49.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.052+0000 7f281affd640 1 -- 172.21.15.132:0/464423815 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f281c014890 con 0x7f282c1365f0 2024-01-24T23:25:49.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.052+0000 7f281affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:49.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.052+0000 7f281affd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:49.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.212+0000 7f2830cad640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}] 2024-01-24T23:25:49.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.212+0000 7f2830cad640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}] 2024-01-24T23:25:49.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.212+0000 7f2830cad640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:49.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.212+0000 7f2830cad640 1 -- 172.21.15.132:0/464423815 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "cool", "pg_num": 2} v 0) v1 -- 0x7f28100058c0 con 0x7f282c1365f0 2024-01-24T23:25:49.908 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.908+0000 7f281affd640 1 -- 172.21.15.132:0/464423815 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}]=0 pool 'cool' created v134) v1 ==== 111+0+0 (unknown 3079311366 0 0) 0x7f281c012f30 con 0x7f282c1365f0 2024-01-24T23:25:49.908 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.908+0000 7f281affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}] 2024-01-24T23:25:49.908 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:49.908+0000 7f281affd640 10 monclient: _finish_command 2 = system:0 pool 'cool' created 2024-01-24T23:25:50.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.012+0000 7f2830cad640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}] 2024-01-24T23:25:50.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.012+0000 7f2830cad640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}] 2024-01-24T23:25:50.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.012+0000 7f2830cad640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:50.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.012+0000 7f2830cad640 1 -- 172.21.15.132:0/464423815 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "cool", "pg_num": 2} v 0) v1 -- 0x7f2810000bc0 con 0x7f282c1365f0 2024-01-24T23:25:50.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.012+0000 7f281affd640 1 -- 172.21.15.132:0/464423815 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}]=0 pool 'cool' already exists v134) v1 ==== 118+0+0 (unknown 240017406 0 0) 0x7f281c0146b0 con 0x7f282c1365f0 2024-01-24T23:25:50.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.012+0000 7f281affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}] 2024-01-24T23:25:50.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.012+0000 7f281affd640 10 monclient: _finish_command 3 = system:0 pool 'cool' already exists 2024-01-24T23:25:50.013 INFO:tasks.workunit.client.0.smithi132.stderr:pool 'cool' already exists 2024-01-24T23:25:50.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.016+0000 7f2830cad640 1 -- 172.21.15.132:0/464423815 >> v1:172.21.15.132:6812/1835401922 conn(0x7f280c036ea0 legacy=0x7f280c039360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:50.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.016+0000 7f2830cad640 10 monclient: shutdown 2024-01-24T23:25:50.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.016+0000 7f2830cad640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:50.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.016+0000 7f2830cad640 1 -- 172.21.15.132:0/464423815 >> v1:172.21.15.132:6790/0 conn(0x7f282c1365f0 legacy=0x7f282c1d3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:50.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.016+0000 7f2830cad640 1 -- 172.21.15.132:0/464423815 shutdown_connections 2024-01-24T23:25:50.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.016+0000 7f2830cad640 1 -- 172.21.15.132:0/464423815 >> 172.21.15.132:0/464423815 conn(0x7f282c0a6790 msgr2=0x7f282c0a3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:50.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.016+0000 7f2830cad640 1 -- 172.21.15.132:0/464423815 shutdown_connections 2024-01-24T23:25:50.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.016+0000 7f2830cad640 1 -- 172.21.15.132:0/464423815 wait complete. 2024-01-24T23:25:50.037 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool create cold 2 2024-01-24T23:25:50.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.148+0000 7f8b2a461640 10 monclient: get_monmap_and_config 2024-01-24T23:25:50.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.148+0000 7f8b2a461640 10 monclient: build_initial_monmap 2024-01-24T23:25:50.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 10 monclient: monmap: 2024-01-24T23:25:50.151 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:50.151 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:50.151 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:50.152823+0000 2024-01-24T23:25:50.151 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:50.152823+0000 2024-01-24T23:25:50.151 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:50.151 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:50.151 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:50.152 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:50.152 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:50.152 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:50.152 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:50.152 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:50.152 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:50.152 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:50.152 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:50.152 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:50.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 1 Processor -- start 2024-01-24T23:25:50.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 1 -- start start 2024-01-24T23:25:50.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 10 monclient: init 2024-01-24T23:25:50.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:50.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 10 monclient: _add_conns ranks=[2,8,6,1,4,3,0,7,5] 2024-01-24T23:25:50.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 10 monclient(hunting): picked mon.noname-b con 0x7f8b24136cb0 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:50.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 10 monclient(hunting): picked mon.noname-e con 0x7f8b241377a0 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:50.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 10 monclient(hunting): picked mon.noname-d con 0x7f8b2409bcc0 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:50.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b24064700 con 0x7f8b24136cb0 2024-01-24T23:25:50.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b2409b650 con 0x7f8b2409bcc0 2024-01-24T23:25:50.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b2409c110 con 0x7f8b241377a0 2024-01-24T23:25:50.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:50.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 10 monclient(hunting): authenticate will time out at 1237.477447s 2024-01-24T23:25:50.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b23fff640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f8b24136cb0 legacy=0x7f8b24137090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:25:50.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b289fe640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f8b2409bcc0 0x7f8b2413f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:36434/0 (socket says 172.21.15.132:36434) 2024-01-24T23:25:50.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b289fe640 1 -- 172.21.15.132:0/3272696592 learned_addr learned my addr 172.21.15.132:0/3272696592 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:50.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 1 -- 172.21.15.132:0/3272696592 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142159345 0 0) 0x7f8b2409b650 con 0x7f8b2409bcc0 2024-01-24T23:25:50.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:50.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:50.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:50.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 10 monclient(hunting): my global_id is 6252 2024-01-24T23:25:50.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 1 -- 172.21.15.132:0/3272696592 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b0c003530 con 0x7f8b2409bcc0 2024-01-24T23:25:50.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 1 -- 172.21.15.132:0/3272696592 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1245034651 0 0) 0x7f8b2409c110 con 0x7f8b241377a0 2024-01-24T23:25:50.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:50.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:50.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:50.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 10 monclient(hunting): my global_id is 85319 2024-01-24T23:25:50.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 1 -- 172.21.15.132:0/3272696592 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b2409b650 con 0x7f8b241377a0 2024-01-24T23:25:50.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 1 -- 172.21.15.132:0/3272696592 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 734555153 0 0) 0x7f8b0c003530 con 0x7f8b2409bcc0 2024-01-24T23:25:50.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 1 -- 172.21.15.132:0/3272696592 >> v1:172.21.15.115:6793/0 conn(0x7f8b241377a0 legacy=0x7f8b2413c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:50.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 1 -- 172.21.15.132:0/3272696592 >> v1:172.21.15.115:6790/0 conn(0x7f8b24136cb0 legacy=0x7f8b24137090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:50.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:25:50.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 1 monclient: found mon.noname-d 2024-01-24T23:25:50.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:50.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:25:50.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 1 -- 172.21.15.132:0/3272696592 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2409b000 con 0x7f8b2409bcc0 2024-01-24T23:25:50.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 10 monclient: _finish_auth 0 2024-01-24T23:25:50.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:50.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:50.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 1 -- 172.21.15.132:0/3272696592 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8b1c001b40 con 0x7f8b2409bcc0 2024-01-24T23:25:50.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:50.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:25:50.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 10 monclient: dump: 2024-01-24T23:25:50.162 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:50.162 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:50.162 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:50.162 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:50.162 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:50.162 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:50.162 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:50.162 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:50.162 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:50.162 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:50.162 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:50.162 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:50.162 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:50.163 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:50.163 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:50.163 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:50.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 5 monclient: authenticate success, global_id 6252 2024-01-24T23:25:50.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:50.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 1 -- 172.21.15.132:0/3272696592 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8b1c001e20 con 0x7f8b2409bcc0 2024-01-24T23:25:50.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:50.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:50.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 4 set_mon_vals no callback set 2024-01-24T23:25:50.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 1 -- 172.21.15.132:0/3272696592 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8b1c004f70 con 0x7f8b2409bcc0 2024-01-24T23:25:50.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:50.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:50.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b22ffd640 10 monclient: dump: 2024-01-24T23:25:50.163 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:50.163 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:50.163 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:50.163 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:50.164 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 10 monclient: shutdown 2024-01-24T23:25:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 1 -- 172.21.15.132:0/3272696592 >> v1:172.21.15.115:6792/0 conn(0x7f8b2409bcc0 legacy=0x7f8b2413f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 1 -- 172.21.15.132:0/3272696592 shutdown_connections 2024-01-24T23:25:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 1 -- 172.21.15.132:0/3272696592 >> 172.21.15.132:0/3272696592 conn(0x7f8b240a9c10 msgr2=0x7f8b240a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 1 -- 172.21.15.132:0/3272696592 shutdown_connections 2024-01-24T23:25:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.152+0000 7f8b2a461640 1 -- 172.21.15.132:0/3272696592 wait complete. 2024-01-24T23:25:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 10 monclient: build_initial_monmap 2024-01-24T23:25:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 10 monclient: monmap: 2024-01-24T23:25:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:50.166 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:50.166 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:50.166 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:50.166 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:50.166 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:50.166 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:50.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 1 Processor -- start 2024-01-24T23:25:50.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 1 -- start start 2024-01-24T23:25:50.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 10 monclient: init 2024-01-24T23:25:50.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:50.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 10 monclient: _add_conns ranks=[1,7,4,6,5,3,0,2,8] 2024-01-24T23:25:50.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 10 monclient(hunting): picked mon.noname-f con 0x7f8b2409bcc0 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:50.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 10 monclient(hunting): picked mon.noname-i con 0x7f8b24136cb0 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:50.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 10 monclient(hunting): picked mon.noname-c con 0x7f8b241377a0 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:50.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b2409c6a0 con 0x7f8b2409bcc0 2024-01-24T23:25:50.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b24124480 con 0x7f8b241377a0 2024-01-24T23:25:50.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b24135620 con 0x7f8b24136cb0 2024-01-24T23:25:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 10 monclient(hunting): authenticate will time out at 1237.481108s 2024-01-24T23:25:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b23fff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f8b2409bcc0 legacy=0x7f8b24134f10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b237fe640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f8b24136cb0 legacy=0x7f8b241d0500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:25:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b289fe640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f8b241377a0 0x7f8b241d3c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:57682/0 (socket says 172.21.15.132:57682) 2024-01-24T23:25:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b289fe640 1 -- 172.21.15.132:0/2312823059 learned_addr learned my addr 172.21.15.132:0/2312823059 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 1 -- 172.21.15.132:0/2312823059 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4099410074 0 0) 0x7f8b24124480 con 0x7f8b241377a0 2024-01-24T23:25:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:25:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 10 monclient(hunting): my global_id is 94378 2024-01-24T23:25:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 1 -- 172.21.15.132:0/2312823059 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b10003530 con 0x7f8b241377a0 2024-01-24T23:25:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 1 -- 172.21.15.132:0/2312823059 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1417462152 0 0) 0x7f8b10003530 con 0x7f8b241377a0 2024-01-24T23:25:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 1 -- 172.21.15.132:0/2312823059 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8b24124480 con 0x7f8b241377a0 2024-01-24T23:25:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 1 -- 172.21.15.132:0/2312823059 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8b1c004eb0 con 0x7f8b241377a0 2024-01-24T23:25:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:50.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:25:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 10 monclient(hunting): dump: 2024-01-24T23:25:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:50.168 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:50.169 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:50.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:50.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 1 -- 172.21.15.132:0/2312823059 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 408173075 0 0) 0x7f8b24124480 con 0x7f8b241377a0 2024-01-24T23:25:50.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 1 -- 172.21.15.132:0/2312823059 >> v1:172.21.15.132:6792/0 conn(0x7f8b24136cb0 legacy=0x7f8b241d0500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:50.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 1 -- 172.21.15.132:0/2312823059 >> v1:172.21.15.132:6789/0 conn(0x7f8b2409bcc0 legacy=0x7f8b24134f10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:50.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:25:50.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 1 monclient: found mon.c 2024-01-24T23:25:50.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:50.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:50.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 1 -- 172.21.15.132:0/2312823059 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b24137fb0 con 0x7f8b241377a0 2024-01-24T23:25:50.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 10 monclient: _finish_auth 0 2024-01-24T23:25:50.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:25:50.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:50.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 5 monclient: authenticate success, global_id 94378 2024-01-24T23:25:50.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 10 monclient: _renew_subs 2024-01-24T23:25:50.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 1 -- 172.21.15.132:0/2312823059 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b241241a0 con 0x7f8b241377a0 2024-01-24T23:25:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 10 monclient: _renew_subs 2024-01-24T23:25:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 1 -- 172.21.15.132:0/2312823059 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b241be620 con 0x7f8b241377a0 2024-01-24T23:25:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 1 -- 172.21.15.132:0/2312823059 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8b1c005240 con 0x7f8b241377a0 2024-01-24T23:25:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 1 -- 172.21.15.132:0/2312823059 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8b1c0056f0 con 0x7f8b241377a0 2024-01-24T23:25:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b227fc640 4 set_mon_vals no callback set 2024-01-24T23:25:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b20ff9640 10 monclient: dump: 2024-01-24T23:25:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:50.171 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.156+0000 7f8b2a461640 1 -- 172.21.15.132:0/2312823059 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b04005090 con 0x7f8b241377a0 2024-01-24T23:25:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.160+0000 7f8b20ff9640 1 -- 172.21.15.132:0/2312823059 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f8b1c010170 con 0x7f8b241377a0 2024-01-24T23:25:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.160+0000 7f8b20ff9640 1 -- 172.21.15.132:0/2312823059 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(134..134 src has 1..134) v4 ==== 6720+0+0 (unknown 2607349092 0 0) 0x7f8b1c044920 con 0x7f8b241377a0 2024-01-24T23:25:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.160+0000 7f8b23fff640 10 monclient: get_auth_request con 0x7f8b10036ef0 auth_method 0 2024-01-24T23:25:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.172+0000 7f8b20ff9640 1 -- 172.21.15.132:0/2312823059 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f8b1c014e40 con 0x7f8b241377a0 2024-01-24T23:25:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.172+0000 7f8b20ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:50.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.172+0000 7f8b20ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:50.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.324+0000 7f8b2a461640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}] 2024-01-24T23:25:50.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.324+0000 7f8b2a461640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}] 2024-01-24T23:25:50.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.324+0000 7f8b2a461640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:50.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.324+0000 7f8b2a461640 1 -- 172.21.15.132:0/2312823059 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd pool create", "pool": "cold", "pg_num": 2} v 0) v1 -- 0x7f8b04005b50 con 0x7f8b241377a0 2024-01-24T23:25:50.920 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.920+0000 7f8b20ff9640 1 -- 172.21.15.132:0/2312823059 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}]=0 pool 'cold' created v135) v1 ==== 111+0+0 (unknown 923457640 0 0) 0x7f8b1c0134e0 con 0x7f8b241377a0 2024-01-24T23:25:50.920 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.920+0000 7f8b20ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}] 2024-01-24T23:25:50.920 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:50.920+0000 7f8b20ff9640 10 monclient: _finish_command 2 = system:0 pool 'cold' created 2024-01-24T23:25:51.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.016+0000 7f8b2a461640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}] 2024-01-24T23:25:51.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.016+0000 7f8b2a461640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}] 2024-01-24T23:25:51.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.016+0000 7f8b2a461640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:25:51.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.016+0000 7f8b2a461640 1 -- 172.21.15.132:0/2312823059 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd pool create", "pool": "cold", "pg_num": 2} v 0) v1 -- 0x7f8b040055d0 con 0x7f8b241377a0 2024-01-24T23:25:51.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.020+0000 7f8b20ff9640 1 -- 172.21.15.132:0/2312823059 <== mon.4 v1:172.21.15.115:6791/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}]=0 pool 'cold' already exists v135) v1 ==== 118+0+0 (unknown 573998538 0 0) 0x7f8b1c0134e0 con 0x7f8b241377a0 2024-01-24T23:25:51.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.020+0000 7f8b20ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}] 2024-01-24T23:25:51.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.020+0000 7f8b20ff9640 10 monclient: _finish_command 3 = system:0 pool 'cold' already exists 2024-01-24T23:25:51.021 INFO:tasks.workunit.client.0.smithi132.stderr:pool 'cold' already exists 2024-01-24T23:25:51.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.020+0000 7f8b2a461640 1 -- 172.21.15.132:0/2312823059 >> v1:172.21.15.132:6812/1835401922 conn(0x7f8b10036ef0 legacy=0x7f8b100393b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:51.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.020+0000 7f8b2a461640 10 monclient: shutdown 2024-01-24T23:25:51.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.020+0000 7f8b2a461640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:51.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.020+0000 7f8b2a461640 1 -- 172.21.15.132:0/2312823059 >> v1:172.21.15.115:6791/0 conn(0x7f8b241377a0 legacy=0x7f8b241d3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:51.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.024+0000 7f8b2a461640 1 -- 172.21.15.132:0/2312823059 shutdown_connections 2024-01-24T23:25:51.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.024+0000 7f8b2a461640 1 -- 172.21.15.132:0/2312823059 >> 172.21.15.132:0/2312823059 conn(0x7f8b240a9c10 msgr2=0x7f8b24066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:51.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.024+0000 7f8b2a461640 1 -- 172.21.15.132:0/2312823059 shutdown_connections 2024-01-24T23:25:51.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.024+0000 7f8b2a461640 1 -- 172.21.15.132:0/2312823059 wait complete. 2024-01-24T23:25:51.038 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool set cold size 2 2024-01-24T23:25:51.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.156+0000 7f77900d4640 10 monclient: get_monmap_and_config 2024-01-24T23:25:51.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.156+0000 7f77900d4640 10 monclient: build_initial_monmap 2024-01-24T23:25:51.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.156+0000 7f77900d4640 10 monclient: monmap: 2024-01-24T23:25:51.160 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:51.160 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:51.160 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:51.160727+0000 2024-01-24T23:25:51.160 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:51.160727+0000 2024-01-24T23:25:51.160 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:51.161 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:51.161 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:51.161 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:51.161 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:51.161 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:51.161 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:51.161 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:51.162 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:51.162 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:51.162 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:51.162 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:51.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 1 Processor -- start 2024-01-24T23:25:51.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 1 -- start start 2024-01-24T23:25:51.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 10 monclient: init 2024-01-24T23:25:51.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:51.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 10 monclient: _add_conns ranks=[6,1,3,0,8,4,5,7,2] 2024-01-24T23:25:51.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 10 monclient(hunting): picked mon.noname-d con 0x7f7788072690 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:51.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 10 monclient(hunting): picked mon.noname-f con 0x7f778809ae90 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:51.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 10 monclient(hunting): picked mon.noname-g con 0x7f778809bdc0 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:51.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7788064700 con 0x7f778809ae90 2024-01-24T23:25:51.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f778809a790 con 0x7f778809bdc0 2024-01-24T23:25:51.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77880a18f0 con 0x7f7788072690 2024-01-24T23:25:51.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:51.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 10 monclient(hunting): authenticate will time out at 1238.485445s 2024-01-24T23:25:51.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778d66f640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f778809ae90 legacy=0x7f77880a11e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:51.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778e671640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f778809bdc0 0x7f778813d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:53222/0 (socket says 172.21.15.132:53222) 2024-01-24T23:25:51.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778e671640 1 -- 172.21.15.132:0/1786002133 learned_addr learned my addr 172.21.15.132:0/1786002133 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:51.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 1 -- 172.21.15.132:0/1786002133 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189011845 0 0) 0x7f778809a790 con 0x7f778809bdc0 2024-01-24T23:25:51.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 10 monclient(hunting): _negotiate 2024-01-24T23:25:51.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:51.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:51.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 10 monclient(hunting): my global_id is 6096 2024-01-24T23:25:51.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 1 -- 172.21.15.132:0/1786002133 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7774003530 con 0x7f778809bdc0 2024-01-24T23:25:51.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 1 -- 172.21.15.132:0/1786002133 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 366113820 0 0) 0x7f7774003530 con 0x7f778809bdc0 2024-01-24T23:25:51.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 1 -- 172.21.15.132:0/1786002133 >> v1:172.21.15.115:6792/0 conn(0x7f7788072690 legacy=0x7f77880a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:51.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 1 -- 172.21.15.132:0/1786002133 >> v1:172.21.15.132:6789/0 conn(0x7f778809ae90 legacy=0x7f77880a11e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:51.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 10 monclient: _finish_hunting 0 2024-01-24T23:25:51.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 1 monclient: found mon.noname-g 2024-01-24T23:25:51.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:51.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:25:51.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 1 -- 172.21.15.132:0/1786002133 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77880997a0 con 0x7f778809bdc0 2024-01-24T23:25:51.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 10 monclient: _finish_auth 0 2024-01-24T23:25:51.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 10 monclient: _check_auth_tickets 2024-01-24T23:25:51.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:51.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 1 -- 172.21.15.132:0/1786002133 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7780001b40 con 0x7f778809bdc0 2024-01-24T23:25:51.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:51.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:25:51.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 10 monclient: dump: 2024-01-24T23:25:51.182 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:51.182 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:51.182 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:51.183 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:51.183 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:51.183 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:51.183 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:51.184 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:51.184 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:51.184 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:51.184 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:51.184 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:51.184 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:51.185 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:51.185 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:51.185 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:51.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 5 monclient: authenticate success, global_id 6096 2024-01-24T23:25:51.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:51.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 1 -- 172.21.15.132:0/1786002133 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7780001df0 con 0x7f778809bdc0 2024-01-24T23:25:51.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:51.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:51.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 4 set_mon_vals no callback set 2024-01-24T23:25:51.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 1 -- 172.21.15.132:0/1786002133 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7780004f70 con 0x7f778809bdc0 2024-01-24T23:25:51.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:51.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:51.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f778ce6e640 10 monclient: dump: 2024-01-24T23:25:51.187 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:51.187 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:51.187 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:51.187 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:51.187 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:51.187 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:51.187 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:51.187 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:51.187 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:51.188 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:51.188 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:51.188 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:51.188 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:51.188 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:51.188 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:51.188 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:51.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 10 monclient: shutdown 2024-01-24T23:25:51.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:51.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 1 -- 172.21.15.132:0/1786002133 >> v1:172.21.15.132:6790/0 conn(0x7f778809bdc0 legacy=0x7f778813d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:51.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 1 -- 172.21.15.132:0/1786002133 shutdown_connections 2024-01-24T23:25:51.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 1 -- 172.21.15.132:0/1786002133 >> 172.21.15.132:0/1786002133 conn(0x7f77880a1bc0 msgr2=0x7f77880a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:51.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 1 -- 172.21.15.132:0/1786002133 shutdown_connections 2024-01-24T23:25:51.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 1 -- 172.21.15.132:0/1786002133 wait complete. 2024-01-24T23:25:51.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 10 monclient: build_initial_monmap 2024-01-24T23:25:51.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 10 monclient: monmap: 2024-01-24T23:25:51.190 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:51.190 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:51.190 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:51.190 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:51.190 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:51.190 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:51.190 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:51.191 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:51.191 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:51.191 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:51.191 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:51.191 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:51.191 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:51.192 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:51.192 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:51.192 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:51.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.160+0000 7f77900d4640 1 Processor -- start 2024-01-24T23:25:51.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f77900d4640 1 -- start start 2024-01-24T23:25:51.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f77900d4640 10 monclient: init 2024-01-24T23:25:51.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f77900d4640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:51.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f77900d4640 10 monclient: _add_conns ranks=[6,3,7,2,4,1,8,5,0] 2024-01-24T23:25:51.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f77900d4640 10 monclient(hunting): picked mon.noname-d con 0x7f7788072690 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:51.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f77900d4640 10 monclient(hunting): picked mon.noname-g con 0x7f778809ae90 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:51.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f77900d4640 10 monclient(hunting): picked mon.noname-i con 0x7f778809bdc0 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:51.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f77900d4640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77881b1340 con 0x7f778809ae90 2024-01-24T23:25:51.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f77900d4640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77881afc20 con 0x7f7788072690 2024-01-24T23:25:51.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f77900d4640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77881b0d80 con 0x7f778809bdc0 2024-01-24T23:25:51.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f77900d4640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:51.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f77900d4640 10 monclient(hunting): authenticate will time out at 1238.488582s 2024-01-24T23:25:51.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f778e671640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f778809bdc0 legacy=0x7f77881d2450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:25:51.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f778d66f640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f778809ae90 0x7f77881ced20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:53230/0 (socket says 172.21.15.132:53230) 2024-01-24T23:25:51.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f778d66f640 1 -- 172.21.15.132:0/2044211822 learned_addr learned my addr 172.21.15.132:0/2044211822 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:51.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 1 -- 172.21.15.132:0/2044211822 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 781225681 0 0) 0x7f77881b1340 con 0x7f778809ae90 2024-01-24T23:25:51.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:51.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:51.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:51.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 10 monclient(hunting): my global_id is 6105 2024-01-24T23:25:51.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 1 -- 172.21.15.132:0/2044211822 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f776c003530 con 0x7f778809ae90 2024-01-24T23:25:51.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 1 -- 172.21.15.132:0/2044211822 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 168175617 0 0) 0x7f776c003530 con 0x7f778809ae90 2024-01-24T23:25:51.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 1 -- 172.21.15.132:0/2044211822 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f77881b1340 con 0x7f778809ae90 2024-01-24T23:25:51.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 1 -- 172.21.15.132:0/2044211822 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7778004300 con 0x7f778809ae90 2024-01-24T23:25:51.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:51.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:25:51.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 10 monclient(hunting): dump: 2024-01-24T23:25:51.194 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:51.194 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:51.194 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:51.194 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:51.194 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:51.194 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:51.195 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:51.195 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:51.195 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:51.195 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:51.195 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:51.195 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:51.195 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:51.195 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:51.195 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:51.195 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:51.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:51.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 1 -- 172.21.15.132:0/2044211822 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2153181375 0 0) 0x7f77881afc20 con 0x7f7788072690 2024-01-24T23:25:51.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:51.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:51.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:51.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 10 monclient(hunting): my global_id is 6270 2024-01-24T23:25:51.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 1 -- 172.21.15.132:0/2044211822 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f776c003530 con 0x7f7788072690 2024-01-24T23:25:51.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 1 -- 172.21.15.132:0/2044211822 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 539855350 0 0) 0x7f77881b1340 con 0x7f778809ae90 2024-01-24T23:25:51.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 1 -- 172.21.15.132:0/2044211822 >> v1:172.21.15.132:6792/0 conn(0x7f778809bdc0 legacy=0x7f77881d2450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:51.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 1 -- 172.21.15.132:0/2044211822 >> v1:172.21.15.115:6792/0 conn(0x7f7788072690 legacy=0x7f77881c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:51.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:25:51.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 1 monclient: found mon.g 2024-01-24T23:25:51.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:51.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:51.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 1 -- 172.21.15.132:0/2044211822 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77881b1060 con 0x7f778809ae90 2024-01-24T23:25:51.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 10 monclient: _finish_auth 0 2024-01-24T23:25:51.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:25:51.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:51.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f77900d4640 5 monclient: authenticate success, global_id 6105 2024-01-24T23:25:51.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f77900d4640 10 monclient: _renew_subs 2024-01-24T23:25:51.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f77900d4640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:51.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f77900d4640 1 -- 172.21.15.132:0/2044211822 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77881af4f0 con 0x7f778809ae90 2024-01-24T23:25:51.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f77900d4640 10 monclient: _renew_subs 2024-01-24T23:25:51.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f77900d4640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:51.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f77900d4640 1 -- 172.21.15.132:0/2044211822 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f77881b1760 con 0x7f778809ae90 2024-01-24T23:25:51.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 1 -- 172.21.15.132:0/2044211822 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f77780045b0 con 0x7f778809ae90 2024-01-24T23:25:51.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:51.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 1 -- 172.21.15.132:0/2044211822 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7778004ae0 con 0x7f778809ae90 2024-01-24T23:25:51.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:51.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777ffff640 4 set_mon_vals no callback set 2024-01-24T23:25:51.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:51.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 10 monclient: dump: 2024-01-24T23:25:51.198 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:51.198 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:51.198 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:51.198 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:51.198 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:51.198 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:51.198 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:51.198 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:51.198 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:51.198 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:51.198 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:51.198 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:51.198 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:51.199 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:51.199 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:51.199 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:51.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f77900d4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:51.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f77900d4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:51.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f77900d4640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:51.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f77900d4640 1 -- 172.21.15.132:0/2044211822 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7770005090 con 0x7f778809ae90 2024-01-24T23:25:51.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 1 -- 172.21.15.132:0/2044211822 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f7778004c40 con 0x7f778809ae90 2024-01-24T23:25:51.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f777e7fc640 1 -- 172.21.15.132:0/2044211822 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(135..135 src has 1..135) v4 ==== 7063+0+0 (unknown 2391774947 0 0) 0x7f7778044500 con 0x7f778809ae90 2024-01-24T23:25:51.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.164+0000 7f778de70640 10 monclient: get_auth_request con 0x7f776c036ee0 auth_method 0 2024-01-24T23:25:51.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.168+0000 7f777e7fc640 1 -- 172.21.15.132:0/2044211822 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f7778014a70 con 0x7f778809ae90 2024-01-24T23:25:51.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.168+0000 7f777e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:51.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.168+0000 7f777e7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:51.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.328+0000 7f77900d4640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-01-24T23:25:51.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.328+0000 7f77900d4640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-01-24T23:25:51.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.328+0000 7f77900d4640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:51.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.328+0000 7f77900d4640 1 -- 172.21.15.132:0/2044211822 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"} v 0) v1 -- 0x7f77700058c0 con 0x7f778809ae90 2024-01-24T23:25:51.932 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.932+0000 7f777e7fc640 1 -- 172.21.15.132:0/2044211822 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}]=0 set pool 12 size to 2 v136) v1 ==== 124+0+0 (unknown 942697343 0 0) 0x7f7778012ee0 con 0x7f778809ae90 2024-01-24T23:25:51.932 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.932+0000 7f777e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-01-24T23:25:51.932 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:51.932+0000 7f777e7fc640 10 monclient: _finish_command 2 = system:0 set pool 12 size to 2 2024-01-24T23:25:52.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:52.028+0000 7f77900d4640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-01-24T23:25:52.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:52.028+0000 7f77900d4640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-01-24T23:25:52.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:52.028+0000 7f77900d4640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:52.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:52.028+0000 7f77900d4640 1 -- 172.21.15.132:0/2044211822 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"} v 0) v1 -- 0x7f7770000bc0 con 0x7f778809ae90 2024-01-24T23:25:52.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:52.940+0000 7f777e7fc640 1 -- 172.21.15.132:0/2044211822 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}]=0 set pool 12 size to 2 v137) v1 ==== 124+0+0 (unknown 1277656640 0 0) 0x7f7778014840 con 0x7f778809ae90 2024-01-24T23:25:52.943 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:52.940+0000 7f777e7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-01-24T23:25:52.943 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:52.940+0000 7f777e7fc640 10 monclient: _finish_command 3 = system:0 set pool 12 size to 2 2024-01-24T23:25:52.943 INFO:tasks.workunit.client.0.smithi132.stderr:set pool 12 size to 2 2024-01-24T23:25:52.951 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:52.952+0000 7f77900d4640 1 -- 172.21.15.132:0/2044211822 >> v1:172.21.15.132:6812/1835401922 conn(0x7f776c036ee0 legacy=0x7f776c0393a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:52.951 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:52.952+0000 7f77900d4640 10 monclient: shutdown 2024-01-24T23:25:52.951 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:52.952+0000 7f77900d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:52.952 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:52.952+0000 7f77900d4640 1 -- 172.21.15.132:0/2044211822 >> v1:172.21.15.132:6790/0 conn(0x7f778809ae90 legacy=0x7f77881ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:52.952 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:52.952+0000 7f77900d4640 1 -- 172.21.15.132:0/2044211822 shutdown_connections 2024-01-24T23:25:52.952 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:52.952+0000 7f77900d4640 1 -- 172.21.15.132:0/2044211822 >> 172.21.15.132:0/2044211822 conn(0x7f77880a1bc0 msgr2=0x7f77880a3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:52.952 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:52.952+0000 7f77900d4640 1 -- 172.21.15.132:0/2044211822 shutdown_connections 2024-01-24T23:25:52.952 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:52.952+0000 7f77900d4640 1 -- 172.21.15.132:0/2044211822 wait complete. 2024-01-24T23:25:52.974 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set create-compat 2024-01-24T23:25:53.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.092+0000 7fb997b15640 10 monclient: get_monmap_and_config 2024-01-24T23:25:53.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.092+0000 7fb997b15640 10 monclient: build_initial_monmap 2024-01-24T23:25:53.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.092+0000 7fb997b15640 10 monclient: monmap: 2024-01-24T23:25:53.162 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:53.096104+0000 2024-01-24T23:25:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:53.096104+0000 2024-01-24T23:25:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:53.163 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:53.164 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:53.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 1 Processor -- start 2024-01-24T23:25:53.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 1 -- start start 2024-01-24T23:25:53.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 10 monclient: init 2024-01-24T23:25:53.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:53.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 10 monclient: _add_conns ranks=[0,7,5,3,6,4,1,2,8] 2024-01-24T23:25:53.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 10 monclient(hunting): picked mon.noname-a con 0x7fb990133b50 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:53.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 10 monclient(hunting): picked mon.noname-i con 0x7fb99009ec20 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:53.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 10 monclient(hunting): picked mon.noname-h con 0x7fb99013a1d0 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:53.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb990064700 con 0x7fb990133b50 2024-01-24T23:25:53.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb99009d9f0 con 0x7fb99013a1d0 2024-01-24T23:25:53.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb99009dc60 con 0x7fb99009ec20 2024-01-24T23:25:53.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:53.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 10 monclient(hunting): authenticate will time out at 1240.420990s 2024-01-24T23:25:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9950b0640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fb99009ec20 legacy=0x7fb990138ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:25:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9960b2640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fb99013a1d0 0x7fb99013c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:58638/0 (socket says 172.21.15.132:58638) 2024-01-24T23:25:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9960b2640 1 -- 172.21.15.132:0/1187159985 learned_addr learned my addr 172.21.15.132:0/1187159985 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 1 -- 172.21.15.132:0/1187159985 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3937215309 0 0) 0x7fb99009d9f0 con 0x7fb99013a1d0 2024-01-24T23:25:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 10 monclient(hunting): _negotiate 2024-01-24T23:25:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 10 monclient(hunting): my global_id is 6017 2024-01-24T23:25:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 1 -- 172.21.15.132:0/1187159985 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb97c003530 con 0x7fb99013a1d0 2024-01-24T23:25:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 1 -- 172.21.15.132:0/1187159985 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 63313347 0 0) 0x7fb97c003530 con 0x7fb99013a1d0 2024-01-24T23:25:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 1 -- 172.21.15.132:0/1187159985 >> v1:172.21.15.132:6792/0 conn(0x7fb99009ec20 legacy=0x7fb990138ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 1 -- 172.21.15.132:0/1187159985 >> v1:172.21.15.115:6789/0 conn(0x7fb990133b50 legacy=0x7fb990133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 10 monclient: _finish_hunting 0 2024-01-24T23:25:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 1 monclient: found mon.noname-h 2024-01-24T23:25:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:53.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:25:53.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 1 -- 172.21.15.132:0/1187159985 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99009df40 con 0x7fb99013a1d0 2024-01-24T23:25:53.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 10 monclient: _finish_auth 0 2024-01-24T23:25:53.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 10 monclient: _check_auth_tickets 2024-01-24T23:25:53.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:53.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 1 -- 172.21.15.132:0/1187159985 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb988001b40 con 0x7fb99013a1d0 2024-01-24T23:25:53.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:53.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:25:53.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 10 monclient: dump: 2024-01-24T23:25:53.172 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:53.172 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:53.172 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:53.172 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:53.172 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:53.173 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:53.173 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:53.173 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:53.173 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:53.173 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:53.173 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:53.173 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:53.173 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:53.173 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:53.173 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:53.173 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:53.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 5 monclient: authenticate success, global_id 6017 2024-01-24T23:25:53.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:53.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 1 -- 172.21.15.132:0/1187159985 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb988001e60 con 0x7fb99013a1d0 2024-01-24T23:25:53.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:53.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 1 -- 172.21.15.132:0/1187159985 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb988004f70 con 0x7fb99013a1d0 2024-01-24T23:25:53.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:53.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:53.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9948af640 10 monclient: dump: 2024-01-24T23:25:53.174 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:53.174 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:53.174 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:53.174 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:53.174 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:53.174 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:53.174 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:53.174 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:53.174 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:53.174 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:53.174 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:53.174 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:53.174 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:53.174 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:53.175 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:53.175 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:53.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:53.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 4 set_mon_vals no callback set 2024-01-24T23:25:53.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 10 monclient: shutdown 2024-01-24T23:25:53.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:53.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 1 -- 172.21.15.132:0/1187159985 >> v1:172.21.15.132:6791/0 conn(0x7fb99013a1d0 legacy=0x7fb99013c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:53.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 1 -- 172.21.15.132:0/1187159985 shutdown_connections 2024-01-24T23:25:53.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 1 -- 172.21.15.132:0/1187159985 >> 172.21.15.132:0/1187159985 conn(0x7fb9900a5a10 msgr2=0x7fb9900a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:53.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 1 -- 172.21.15.132:0/1187159985 shutdown_connections 2024-01-24T23:25:53.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 1 -- 172.21.15.132:0/1187159985 wait complete. 2024-01-24T23:25:53.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 10 monclient: build_initial_monmap 2024-01-24T23:25:53.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 10 monclient: monmap: 2024-01-24T23:25:53.175 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:53.175 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:53.175 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:53.176 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:53.176 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:53.176 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:53.176 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:53.176 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:53.176 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:53.176 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:53.176 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:53.176 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:53.176 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:53.176 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:53.176 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:53.176 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:53.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 1 Processor -- start 2024-01-24T23:25:53.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 1 -- start start 2024-01-24T23:25:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 10 monclient: init 2024-01-24T23:25:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 10 monclient: _add_conns ranks=[3,7,8,6,2,0,4,1,5] 2024-01-24T23:25:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 10 monclient(hunting): picked mon.noname-g con 0x7fb99009ec20 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 10 monclient(hunting): picked mon.noname-i con 0x7fb990133b50 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 10 monclient(hunting): picked mon.noname-e con 0x7fb99013a1d0 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb99011d160 con 0x7fb99009ec20 2024-01-24T23:25:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb990133920 con 0x7fb990133b50 2024-01-24T23:25:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9900a13b0 con 0x7fb99013a1d0 2024-01-24T23:25:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb997b15640 10 monclient(hunting): authenticate will time out at 1240.423793s 2024-01-24T23:25:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9950b0640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fb990133b50 legacy=0x7fb9901c9a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:25:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9958b1640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fb99009ec20 0x7fb9901c9360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:53232/0 (socket says 172.21.15.132:53232) 2024-01-24T23:25:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.096+0000 7fb9958b1640 1 -- 172.21.15.132:0/1350543838 learned_addr learned my addr 172.21.15.132:0/1350543838 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 1 -- 172.21.15.132:0/1350543838 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307939078 0 0) 0x7fb99011d160 con 0x7fb99009ec20 2024-01-24T23:25:53.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:53.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:53.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:53.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 10 monclient(hunting): my global_id is 6114 2024-01-24T23:25:53.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 1 -- 172.21.15.132:0/1350543838 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb974003530 con 0x7fb99009ec20 2024-01-24T23:25:53.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 1 -- 172.21.15.132:0/1350543838 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4227843319 0 0) 0x7fb974003530 con 0x7fb99009ec20 2024-01-24T23:25:53.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 1 -- 172.21.15.132:0/1350543838 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb99011d160 con 0x7fb99009ec20 2024-01-24T23:25:53.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 1 -- 172.21.15.132:0/1350543838 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb98c001a80 con 0x7fb99009ec20 2024-01-24T23:25:53.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:53.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:25:53.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 10 monclient(hunting): dump: 2024-01-24T23:25:53.178 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:53.178 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:53.178 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:53.178 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:53.178 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:53.179 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:53.179 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:53.179 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:53.179 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:53.179 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:53.179 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:53.179 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:53.179 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:53.179 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:53.179 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:53.179 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:53.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:53.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 1 -- 172.21.15.132:0/1350543838 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4015087606 0 0) 0x7fb9900a13b0 con 0x7fb99013a1d0 2024-01-24T23:25:53.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:53.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:53.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:53.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 10 monclient(hunting): my global_id is 85328 2024-01-24T23:25:53.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 1 -- 172.21.15.132:0/1350543838 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb974003530 con 0x7fb99013a1d0 2024-01-24T23:25:53.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 1 -- 172.21.15.132:0/1350543838 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 732709869 0 0) 0x7fb99011d160 con 0x7fb99009ec20 2024-01-24T23:25:53.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 1 -- 172.21.15.132:0/1350543838 >> v1:172.21.15.115:6793/0 conn(0x7fb99013a1d0 legacy=0x7fb9901d15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:53.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 1 -- 172.21.15.132:0/1350543838 >> v1:172.21.15.132:6792/0 conn(0x7fb990133b50 legacy=0x7fb9901c9a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:53.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 10 monclient: _finish_hunting 0 2024-01-24T23:25:53.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 1 monclient: found mon.g 2024-01-24T23:25:53.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:53.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:53.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 1 -- 172.21.15.132:0/1350543838 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99011eb30 con 0x7fb99009ec20 2024-01-24T23:25:53.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 10 monclient: _finish_auth 0 2024-01-24T23:25:53.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 10 monclient: _check_auth_tickets 2024-01-24T23:25:53.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:53.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb997b15640 5 monclient: authenticate success, global_id 6114 2024-01-24T23:25:53.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb997b15640 10 monclient: _renew_subs 2024-01-24T23:25:53.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb997b15640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:53.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb997b15640 1 -- 172.21.15.132:0/1350543838 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9901ca180 con 0x7fb99009ec20 2024-01-24T23:25:53.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb997b15640 10 monclient: _renew_subs 2024-01-24T23:25:53.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb997b15640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:53.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb997b15640 1 -- 172.21.15.132:0/1350543838 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb99011e850 con 0x7fb99009ec20 2024-01-24T23:25:53.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 1 -- 172.21.15.132:0/1350543838 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb98c003950 con 0x7fb99009ec20 2024-01-24T23:25:53.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:53.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 1 -- 172.21.15.132:0/1350543838 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb98c003e80 con 0x7fb99009ec20 2024-01-24T23:25:53.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:53.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb987fff640 4 set_mon_vals no callback set 2024-01-24T23:25:53.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:53.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 10 monclient: dump: 2024-01-24T23:25:53.181 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:53.181 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:53.181 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:53.181 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:53.181 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:53.182 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:53.182 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:53.182 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:53.182 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:53.182 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:53.182 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:53.182 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:53.182 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:53.182 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:53.182 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:53.182 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:53.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb997b15640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:53.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb997b15640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:53.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb997b15640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:53.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb997b15640 1 -- 172.21.15.132:0/1350543838 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb978005090 con 0x7fb99009ec20 2024-01-24T23:25:53.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 1 -- 172.21.15.132:0/1350543838 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fb98c004060 con 0x7fb99009ec20 2024-01-24T23:25:53.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9867fc640 1 -- 172.21.15.132:0/1350543838 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(137..137 src has 1..137) v4 ==== 7063+0+0 (unknown 1944652883 0 0) 0x7fb98c044630 con 0x7fb99009ec20 2024-01-24T23:25:53.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.100+0000 7fb9950b0640 10 monclient: get_auth_request con 0x7fb974036f70 auth_method 0 2024-01-24T23:25:53.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.104+0000 7fb9867fc640 1 -- 172.21.15.132:0/1350543838 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fb98c014bc0 con 0x7fb99009ec20 2024-01-24T23:25:53.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.104+0000 7fb9867fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:53.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.104+0000 7fb9867fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:53.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.268+0000 7fb997b15640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-01-24T23:25:53.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.268+0000 7fb997b15640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-01-24T23:25:53.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.268+0000 7fb997b15640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:53.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.268+0000 7fb997b15640 1 -- 172.21.15.132:0/1350543838 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) v1 -- 0x7fb978005340 con 0x7fb99009ec20 2024-01-24T23:25:53.945 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.944+0000 7fb9867fc640 1 -- 172.21.15.132:0/1350543838 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 v138) v1 ==== 82+0+0 (unknown 2061534354 0 0) 0x7fb98c013030 con 0x7fb99009ec20 2024-01-24T23:25:53.946 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.944+0000 7fb9867fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-01-24T23:25:53.946 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:53.944+0000 7fb9867fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:54.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.048+0000 7fb997b15640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-01-24T23:25:54.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.048+0000 7fb997b15640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-01-24T23:25:54.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.048+0000 7fb997b15640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:54.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.048+0000 7fb997b15640 1 -- 172.21.15.132:0/1350543838 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) v1 -- 0x7fb978002220 con 0x7fb99009ec20 2024-01-24T23:25:54.051 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.048+0000 7fb9867fc640 1 -- 172.21.15.132:0/1350543838 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 compat weight-set already created v138) v1 ==== 115+0+0 (unknown 2508283433 0 0) 0x7fb98c013030 con 0x7fb99009ec20 2024-01-24T23:25:54.051 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.048+0000 7fb9867fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-01-24T23:25:54.051 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.048+0000 7fb9867fc640 10 monclient: _finish_command 3 = system:0 compat weight-set already created 2024-01-24T23:25:54.051 INFO:tasks.workunit.client.0.smithi132.stderr:compat weight-set already created 2024-01-24T23:25:54.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.052+0000 7fb997b15640 1 -- 172.21.15.132:0/1350543838 >> v1:172.21.15.132:6812/1835401922 conn(0x7fb974036f70 legacy=0x7fb974039430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:54.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.052+0000 7fb997b15640 10 monclient: shutdown 2024-01-24T23:25:54.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.052+0000 7fb997b15640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:54.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.052+0000 7fb997b15640 1 -- 172.21.15.132:0/1350543838 >> v1:172.21.15.132:6790/0 conn(0x7fb99009ec20 legacy=0x7fb9901c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:54.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.052+0000 7fb997b15640 1 -- 172.21.15.132:0/1350543838 shutdown_connections 2024-01-24T23:25:54.054 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.052+0000 7fb997b15640 1 -- 172.21.15.132:0/1350543838 >> 172.21.15.132:0/1350543838 conn(0x7fb9900a5a10 msgr2=0x7fb99009cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:54.054 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.052+0000 7fb997b15640 1 -- 172.21.15.132:0/1350543838 shutdown_connections 2024-01-24T23:25:54.054 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.052+0000 7fb997b15640 1 -- 172.21.15.132:0/1350543838 wait complete. 2024-01-24T23:25:54.067 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set create cool flat 2024-01-24T23:25:54.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 10 monclient: get_monmap_and_config 2024-01-24T23:25:54.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 10 monclient: build_initial_monmap 2024-01-24T23:25:54.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 10 monclient: monmap: 2024-01-24T23:25:54.187 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:54.188 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:54.188 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:54.188884+0000 2024-01-24T23:25:54.188 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:54.188884+0000 2024-01-24T23:25:54.188 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:54.188 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:54.188 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:54.188 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:54.188 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:54.188 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:54.188 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:54.188 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:54.188 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:54.188 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:54.188 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:54.189 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:54.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 1 Processor -- start 2024-01-24T23:25:54.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 1 -- start start 2024-01-24T23:25:54.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 10 monclient: init 2024-01-24T23:25:54.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:54.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 10 monclient: _add_conns ranks=[8,3,0,1,4,2,6,5,7] 2024-01-24T23:25:54.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 10 monclient(hunting): picked mon.noname-e con 0x7f5378133ed0 addr v1:172.21.15.115:6793/0 2024-01-24T23:25:54.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 10 monclient(hunting): picked mon.noname-g con 0x7f53781342b0 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:54.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 10 monclient(hunting): picked mon.noname-a con 0x7f5378134690 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:54.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5378064700 con 0x7f5378134690 2024-01-24T23:25:54.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f537809b3d0 con 0x7f53781342b0 2024-01-24T23:25:54.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53780a58c0 con 0x7f5378133ed0 2024-01-24T23:25:54.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:54.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 10 monclient(hunting): authenticate will time out at 1241.513542s 2024-01-24T23:25:54.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f5377fff640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f53781342b0 0x7f537813c040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:46826/0 (socket says 172.21.15.132:46826) 2024-01-24T23:25:54.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f5377fff640 1 -- 172.21.15.132:0/3421547335 learned_addr learned my addr 172.21.15.132:0/3421547335 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:54.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 1 -- 172.21.15.132:0/3421547335 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2099725123 0 0) 0x7f537809b3d0 con 0x7f53781342b0 2024-01-24T23:25:54.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 10 monclient(hunting): _negotiate 2024-01-24T23:25:54.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:54.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:54.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 10 monclient(hunting): my global_id is 6123 2024-01-24T23:25:54.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 1 -- 172.21.15.132:0/3421547335 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5364003530 con 0x7f53781342b0 2024-01-24T23:25:54.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 1 -- 172.21.15.132:0/3421547335 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4183847707 0 0) 0x7f5364003530 con 0x7f53781342b0 2024-01-24T23:25:54.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 1 -- 172.21.15.132:0/3421547335 >> v1:172.21.15.115:6793/0 conn(0x7f5378133ed0 legacy=0x7f537809bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:54.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 1 -- 172.21.15.132:0/3421547335 >> v1:172.21.15.115:6789/0 conn(0x7f5378134690 legacy=0x7f537813f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:54.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 10 monclient: _finish_hunting 0 2024-01-24T23:25:54.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 1 monclient: found mon.noname-g 2024-01-24T23:25:54.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:54.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:25:54.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 1 -- 172.21.15.132:0/3421547335 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537809b0f0 con 0x7f53781342b0 2024-01-24T23:25:54.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 10 monclient: _finish_auth 0 2024-01-24T23:25:54.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 10 monclient: _check_auth_tickets 2024-01-24T23:25:54.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:54.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 1 -- 172.21.15.132:0/3421547335 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5368001b40 con 0x7f53781342b0 2024-01-24T23:25:54.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:54.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:25:54.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 10 monclient: dump: 2024-01-24T23:25:54.197 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:54.197 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:54.197 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:54.197 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:54.198 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:54.198 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:54.198 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:54.198 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:54.198 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:54.198 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:54.198 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:54.198 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:54.198 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:54.198 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:54.198 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:54.198 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:54.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 5 monclient: authenticate success, global_id 6123 2024-01-24T23:25:54.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:54.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 1 -- 172.21.15.132:0/3421547335 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5368001df0 con 0x7f53781342b0 2024-01-24T23:25:54.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:54.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:54.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 4 set_mon_vals no callback set 2024-01-24T23:25:54.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 1 -- 172.21.15.132:0/3421547335 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5368004f70 con 0x7f53781342b0 2024-01-24T23:25:54.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:54.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:54.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f53777fe640 10 monclient: dump: 2024-01-24T23:25:54.199 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:54.199 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:54.199 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:54.199 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:54.199 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:54.199 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:54.199 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:54.200 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:54.200 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:54.200 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:54.200 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:54.200 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:54.200 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:54.200 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:54.200 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:54.200 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:54.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 10 monclient: shutdown 2024-01-24T23:25:54.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:54.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 1 -- 172.21.15.132:0/3421547335 >> v1:172.21.15.132:6790/0 conn(0x7f53781342b0 legacy=0x7f537813c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:54.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 1 -- 172.21.15.132:0/3421547335 shutdown_connections 2024-01-24T23:25:54.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 1 -- 172.21.15.132:0/3421547335 >> 172.21.15.132:0/3421547335 conn(0x7f53780a5ef0 msgr2=0x7f53780a6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:54.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 1 -- 172.21.15.132:0/3421547335 shutdown_connections 2024-01-24T23:25:54.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 1 -- 172.21.15.132:0/3421547335 wait complete. 2024-01-24T23:25:54.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 10 monclient: build_initial_monmap 2024-01-24T23:25:54.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.188+0000 7f537f240640 10 monclient: monmap: 2024-01-24T23:25:54.201 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:54.201 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:54.201 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:54.201 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:54.201 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:54.201 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:54.201 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:54.201 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:54.201 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:54.201 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:54.201 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:54.202 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:54.202 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:54.202 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:54.202 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:54.202 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:54.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537f240640 1 Processor -- start 2024-01-24T23:25:54.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537f240640 1 -- start start 2024-01-24T23:25:54.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537f240640 10 monclient: init 2024-01-24T23:25:54.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537f240640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:54.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537f240640 10 monclient: _add_conns ranks=[7,1,3,4,0,8,2,6,5] 2024-01-24T23:25:54.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537f240640 10 monclient(hunting): picked mon.noname-i con 0x7f5378133ed0 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:54.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537f240640 10 monclient(hunting): picked mon.noname-f con 0x7f53781342b0 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:54.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537f240640 10 monclient(hunting): picked mon.noname-g con 0x7f5378134690 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:54.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537f240640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f537811fff0 con 0x7f53781342b0 2024-01-24T23:25:54.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537f240640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f537809ca90 con 0x7f5378134690 2024-01-24T23:25:54.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537cfdc640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f5378133ed0 legacy=0x7f53781330e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:25:54.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537f240640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53780a5af0 con 0x7f5378133ed0 2024-01-24T23:25:54.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537f240640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:54.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f5377fff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f53781342b0 legacy=0x7f53781d04a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:54.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537f240640 10 monclient(hunting): authenticate will time out at 1241.516616s 2024-01-24T23:25:54.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537d7dd640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f5378134690 0x7f53781d3bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:46840/0 (socket says 172.21.15.132:46840) 2024-01-24T23:25:54.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537d7dd640 1 -- 172.21.15.132:0/2055188345 learned_addr learned my addr 172.21.15.132:0/2055188345 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:54.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 1 -- 172.21.15.132:0/2055188345 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3555349770 0 0) 0x7f537809ca90 con 0x7f5378134690 2024-01-24T23:25:54.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 10 monclient(hunting): _negotiate 2024-01-24T23:25:54.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:54.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:54.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 10 monclient(hunting): my global_id is 6132 2024-01-24T23:25:54.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 1 -- 172.21.15.132:0/2055188345 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5358003530 con 0x7f5378134690 2024-01-24T23:25:54.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 1 -- 172.21.15.132:0/2055188345 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2543427975 0 0) 0x7f5358003530 con 0x7f5378134690 2024-01-24T23:25:54.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 1 -- 172.21.15.132:0/2055188345 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f537809ca90 con 0x7f5378134690 2024-01-24T23:25:54.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 1 -- 172.21.15.132:0/2055188345 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f53700019a0 con 0x7f5378134690 2024-01-24T23:25:54.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:54.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:25:54.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 10 monclient(hunting): dump: 2024-01-24T23:25:54.204 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:54.204 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:54.204 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:54.204 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:54.204 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:54.204 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:54.204 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:54.204 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:54.204 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:54.205 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:54.205 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:54.205 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:54.205 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:54.205 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:54.205 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:54.205 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:54.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:54.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 1 -- 172.21.15.132:0/2055188345 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4104679612 0 0) 0x7f537809ca90 con 0x7f5378134690 2024-01-24T23:25:54.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 1 -- 172.21.15.132:0/2055188345 >> v1:172.21.15.132:6792/0 conn(0x7f5378133ed0 legacy=0x7f53781330e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:54.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 1 -- 172.21.15.132:0/2055188345 >> v1:172.21.15.132:6789/0 conn(0x7f53781342b0 legacy=0x7f53781d04a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:54.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 10 monclient: _finish_hunting 0 2024-01-24T23:25:54.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 1 monclient: found mon.g 2024-01-24T23:25:54.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:54.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:54.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 1 -- 172.21.15.132:0/2055188345 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53781cb880 con 0x7f5378134690 2024-01-24T23:25:54.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 10 monclient: _finish_auth 0 2024-01-24T23:25:54.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 10 monclient: _check_auth_tickets 2024-01-24T23:25:54.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:54.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537f240640 5 monclient: authenticate success, global_id 6132 2024-01-24T23:25:54.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537f240640 10 monclient: _renew_subs 2024-01-24T23:25:54.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537f240640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:54.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537f240640 1 -- 172.21.15.132:0/2055188345 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5378120590 con 0x7f5378134690 2024-01-24T23:25:54.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537f240640 10 monclient: _renew_subs 2024-01-24T23:25:54.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537f240640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:54.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537f240640 1 -- 172.21.15.132:0/2055188345 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f537809f840 con 0x7f5378134690 2024-01-24T23:25:54.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 1 -- 172.21.15.132:0/2055188345 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5370001d70 con 0x7f5378134690 2024-01-24T23:25:54.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:54.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 1 -- 172.21.15.132:0/2055188345 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5370003c10 con 0x7f5378134690 2024-01-24T23:25:54.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:54.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:54.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f5376ffd640 4 set_mon_vals no callback set 2024-01-24T23:25:54.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 10 monclient: dump: 2024-01-24T23:25:54.207 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:54.207 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:54.207 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:54.207 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:54.207 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:54.207 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:54.207 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:54.207 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:54.207 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:54.208 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:54.208 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:54.208 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:54.208 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:54.208 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:54.208 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:54.208 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:54.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537f240640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:54.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537f240640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:54.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537f240640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:54.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537f240640 1 -- 172.21.15.132:0/2055188345 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535c005090 con 0x7f5378134690 2024-01-24T23:25:54.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 1 -- 172.21.15.132:0/2055188345 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f5370003d70 con 0x7f5378134690 2024-01-24T23:25:54.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f53757fa640 1 -- 172.21.15.132:0/2055188345 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(138..138 src has 1..138) v4 ==== 7395+0+0 (unknown 1566217064 0 0) 0x7f5370044570 con 0x7f5378134690 2024-01-24T23:25:54.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.192+0000 7f537cfdc640 10 monclient: get_auth_request con 0x7f5358036f40 auth_method 0 2024-01-24T23:25:54.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.196+0000 7f53757fa640 1 -- 172.21.15.132:0/2055188345 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f5370014780 con 0x7f5378134690 2024-01-24T23:25:54.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.196+0000 7f53757fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:54.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.196+0000 7f53757fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:54.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.356+0000 7f537f240640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}] 2024-01-24T23:25:54.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.356+0000 7f537f240640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}] 2024-01-24T23:25:54.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.356+0000 7f537f240640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:54.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.356+0000 7f537f240640 1 -- 172.21.15.132:0/2055188345 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"} v 0) v1 -- 0x7f535c0058c0 con 0x7f5378134690 2024-01-24T23:25:54.977 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.976+0000 7f53757fa640 1 -- 172.21.15.132:0/2055188345 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}]=0 v139) v1 ==== 107+0+0 (unknown 1352318241 0 0) 0x7f5370012e20 con 0x7f5378134690 2024-01-24T23:25:54.978 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.976+0000 7f53757fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}] 2024-01-24T23:25:54.978 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:54.976+0000 7f53757fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:55.079 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.076+0000 7f537f240640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}] 2024-01-24T23:25:55.079 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.076+0000 7f537f240640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}] 2024-01-24T23:25:55.079 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.076+0000 7f537f240640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:25:55.079 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.076+0000 7f537f240640 1 -- 172.21.15.132:0/2055188345 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"} v 0) v1 -- 0x7f535c002220 con 0x7f5378134690 2024-01-24T23:25:55.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.080+0000 7f53757fa640 1 -- 172.21.15.132:0/2055188345 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}]=0 weight-set for pool 'cool' already created v139) v1 ==== 149+0+0 (unknown 377729685 0 0) 0x7f53700145a0 con 0x7f5378134690 2024-01-24T23:25:55.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.080+0000 7f53757fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}] 2024-01-24T23:25:55.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.080+0000 7f53757fa640 10 monclient: _finish_command 3 = system:0 weight-set for pool 'cool' already created 2024-01-24T23:25:55.081 INFO:tasks.workunit.client.0.smithi132.stderr:weight-set for pool 'cool' already created 2024-01-24T23:25:55.082 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.080+0000 7f537f240640 1 -- 172.21.15.132:0/2055188345 >> v1:172.21.15.132:6812/1835401922 conn(0x7f5358036f40 legacy=0x7f5358039400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:55.082 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.080+0000 7f537f240640 10 monclient: shutdown 2024-01-24T23:25:55.082 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.080+0000 7f537f240640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:55.083 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.080+0000 7f537f240640 1 -- 172.21.15.132:0/2055188345 >> v1:172.21.15.132:6790/0 conn(0x7f5378134690 legacy=0x7f53781d3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:55.083 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.080+0000 7f537f240640 1 -- 172.21.15.132:0/2055188345 shutdown_connections 2024-01-24T23:25:55.083 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.080+0000 7f537f240640 1 -- 172.21.15.132:0/2055188345 >> 172.21.15.132:0/2055188345 conn(0x7f53780a5ef0 msgr2=0x7f53781bea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:55.083 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.080+0000 7f537f240640 1 -- 172.21.15.132:0/2055188345 shutdown_connections 2024-01-24T23:25:55.083 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.080+0000 7f537f240640 1 -- 172.21.15.132:0/2055188345 wait complete. 2024-01-24T23:25:55.098 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set create cold positional 2024-01-24T23:25:55.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.212+0000 7f36b3389640 10 monclient: get_monmap_and_config 2024-01-24T23:25:55.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.212+0000 7f36b3389640 10 monclient: build_initial_monmap 2024-01-24T23:25:55.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.212+0000 7f36b3389640 10 monclient: monmap: 2024-01-24T23:25:55.215 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:55.215 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:55.215 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:55.216602+0000 2024-01-24T23:25:55.215 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:55.216602+0000 2024-01-24T23:25:55.215 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:55.215 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:55.215 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:55.216 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:55.216 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:55.216 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:55.216 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:55.216 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:55.216 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:55.216 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:55.216 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:55.216 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:55.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 1 Processor -- start 2024-01-24T23:25:55.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 1 -- start start 2024-01-24T23:25:55.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 10 monclient: init 2024-01-24T23:25:55.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:55.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 10 monclient: _add_conns ranks=[6,4,7,0,3,2,1,5,8] 2024-01-24T23:25:55.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 10 monclient(hunting): picked mon.noname-d con 0x7f36ac0a3730 addr v1:172.21.15.115:6792/0 2024-01-24T23:25:55.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 10 monclient(hunting): picked mon.noname-c con 0x7f36ac137cf0 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:55.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 10 monclient(hunting): picked mon.noname-i con 0x7f36ac13b800 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:55.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ac064700 con 0x7f36ac137cf0 2024-01-24T23:25:55.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ac0a3b10 con 0x7f36ac0a3730 2024-01-24T23:25:55.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ac134ef0 con 0x7f36ac13b800 2024-01-24T23:25:55.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:55.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 10 monclient(hunting): authenticate will time out at 1242.541447s 2024-01-24T23:25:55.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b1926640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f36ac13b800 legacy=0x7f36ac13dbf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:25:55.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b1125640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f36ac0a3730 0x7f36ac09c540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:50274/0 (socket says 172.21.15.132:50274) 2024-01-24T23:25:55.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b0924640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f36ac137cf0 0x7f36ac13a0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:37770/0 (socket says 172.21.15.132:37770) 2024-01-24T23:25:55.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b1125640 1 -- 172.21.15.132:0/1972837013 learned_addr learned my addr 172.21.15.132:0/1972837013 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:55.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 1 -- 172.21.15.132:0/1972837013 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019393869 0 0) 0x7f36ac0a3b10 con 0x7f36ac0a3730 2024-01-24T23:25:55.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:55.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:55.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:55.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 10 monclient(hunting): my global_id is 6279 2024-01-24T23:25:55.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 1 -- 172.21.15.132:0/1972837013 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3698003530 con 0x7f36ac0a3730 2024-01-24T23:25:55.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 1 -- 172.21.15.132:0/1972837013 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4124374339 0 0) 0x7f36ac064700 con 0x7f36ac137cf0 2024-01-24T23:25:55.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:25:55.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:55.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:55.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 10 monclient(hunting): my global_id is 94387 2024-01-24T23:25:55.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 1 -- 172.21.15.132:0/1972837013 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36ac0a3b10 con 0x7f36ac137cf0 2024-01-24T23:25:55.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 1 -- 172.21.15.132:0/1972837013 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1617760720 0 0) 0x7f3698003530 con 0x7f36ac0a3730 2024-01-24T23:25:55.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 1 -- 172.21.15.132:0/1972837013 >> v1:172.21.15.132:6792/0 conn(0x7f36ac13b800 legacy=0x7f36ac13dbf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:55.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 1 -- 172.21.15.132:0/1972837013 >> v1:172.21.15.115:6791/0 conn(0x7f36ac137cf0 legacy=0x7f36ac13a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:55.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 10 monclient: _finish_hunting 0 2024-01-24T23:25:55.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 1 monclient: found mon.noname-d 2024-01-24T23:25:55.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:55.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:25:55.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 1 -- 172.21.15.132:0/1972837013 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac135190 con 0x7f36ac0a3730 2024-01-24T23:25:55.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 10 monclient: _finish_auth 0 2024-01-24T23:25:55.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 10 monclient: _check_auth_tickets 2024-01-24T23:25:55.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:55.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 1 -- 172.21.15.132:0/1972837013 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f36a80017c0 con 0x7f36ac0a3730 2024-01-24T23:25:55.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 5 monclient: authenticate success, global_id 6279 2024-01-24T23:25:55.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:55.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:55.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:25:55.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 10 monclient: dump: 2024-01-24T23:25:55.222 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:55.222 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:55.222 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:55.222 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:55.222 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:55.222 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:55.222 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:55.222 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:55.222 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:55.222 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:55.222 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:55.223 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:55.223 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:55.223 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:55.223 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:55.223 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:55.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:55.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 1 -- 172.21.15.132:0/1972837013 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f36a80019e0 con 0x7f36ac0a3730 2024-01-24T23:25:55.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:55.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 1 -- 172.21.15.132:0/1972837013 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f36a80048b0 con 0x7f36ac0a3730 2024-01-24T23:25:55.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:55.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:25:55.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36a3fff640 10 monclient: dump: 2024-01-24T23:25:55.223 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:55.223 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:55.223 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:55.224 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:55.224 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:55.224 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:55.224 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:55.224 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:55.224 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:55.224 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:55.224 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:55.224 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:55.224 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:55.224 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:55.224 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:55.224 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:55.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:55.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 4 set_mon_vals no callback set 2024-01-24T23:25:55.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 10 monclient: shutdown 2024-01-24T23:25:55.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:55.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 1 -- 172.21.15.132:0/1972837013 >> v1:172.21.15.115:6792/0 conn(0x7f36ac0a3730 legacy=0x7f36ac09c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:55.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 1 -- 172.21.15.132:0/1972837013 shutdown_connections 2024-01-24T23:25:55.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 1 -- 172.21.15.132:0/1972837013 >> 172.21.15.132:0/1972837013 conn(0x7f36ac0a2c40 msgr2=0x7f36ac0a3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:55.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 1 -- 172.21.15.132:0/1972837013 shutdown_connections 2024-01-24T23:25:55.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.216+0000 7f36b3389640 1 -- 172.21.15.132:0/1972837013 wait complete. 2024-01-24T23:25:55.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 10 monclient: build_initial_monmap 2024-01-24T23:25:55.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 10 monclient: monmap: 2024-01-24T23:25:55.229 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:55.229 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:55.229 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:55.229 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:55.230 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:55.230 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:55.230 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:55.230 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:55.230 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:55.230 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:55.230 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:55.230 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:55.230 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:55.230 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:55.230 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:55.230 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:55.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 1 Processor -- start 2024-01-24T23:25:55.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 1 -- start start 2024-01-24T23:25:55.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 10 monclient: init 2024-01-24T23:25:55.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:55.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 10 monclient: _add_conns ranks=[1,0,5,6,3,7,4,2,8] 2024-01-24T23:25:55.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 10 monclient(hunting): picked mon.noname-f con 0x7f36ac0a3730 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:55.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 10 monclient(hunting): picked mon.noname-a con 0x7f36ac137cf0 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:55.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 10 monclient(hunting): picked mon.noname-h con 0x7f36ac13b800 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:55.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ac09e460 con 0x7f36ac137cf0 2024-01-24T23:25:55.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ac09e7e0 con 0x7f36ac0a3730 2024-01-24T23:25:55.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ac09ea20 con 0x7f36ac13b800 2024-01-24T23:25:55.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:55.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 10 monclient(hunting): authenticate will time out at 1242.545414s 2024-01-24T23:25:55.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b1125640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f36ac0a3730 legacy=0x7f36ac09d640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:55.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b1926640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f36ac13b800 0x7f36ac1d39f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:52988/0 (socket says 172.21.15.132:52988) 2024-01-24T23:25:55.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b1926640 1 -- 172.21.15.132:0/3564548013 learned_addr learned my addr 172.21.15.132:0/3564548013 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:55.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 1 -- 172.21.15.132:0/3564548013 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568862747 0 0) 0x7f36ac09ea20 con 0x7f36ac13b800 2024-01-24T23:25:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 10 monclient(hunting): my global_id is 6026 2024-01-24T23:25:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 1 -- 172.21.15.132:0/3564548013 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3690003530 con 0x7f36ac13b800 2024-01-24T23:25:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 1 -- 172.21.15.132:0/3564548013 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1819409163 0 0) 0x7f3690003530 con 0x7f36ac13b800 2024-01-24T23:25:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 1 -- 172.21.15.132:0/3564548013 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f36ac09ea20 con 0x7f36ac13b800 2024-01-24T23:25:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 1 -- 172.21.15.132:0/3564548013 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f36a40045a0 con 0x7f36ac13b800 2024-01-24T23:25:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:25:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 10 monclient(hunting): dump: 2024-01-24T23:25:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:55.232 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:55.233 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:55.233 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:55.233 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:55.233 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:55.233 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:55.233 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:55.233 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:55.233 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:55.233 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:55.233 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:55.233 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:55.233 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:55.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:55.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 1 -- 172.21.15.132:0/3564548013 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1137958666 0 0) 0x7f36ac09e460 con 0x7f36ac137cf0 2024-01-24T23:25:55.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:25:55.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:55.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:55.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 10 monclient(hunting): my global_id is 94428 2024-01-24T23:25:55.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 1 -- 172.21.15.132:0/3564548013 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3690003530 con 0x7f36ac137cf0 2024-01-24T23:25:55.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 1 -- 172.21.15.132:0/3564548013 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 167429764 0 0) 0x7f36ac09ea20 con 0x7f36ac13b800 2024-01-24T23:25:55.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 1 -- 172.21.15.132:0/3564548013 >> v1:172.21.15.132:6789/0 conn(0x7f36ac0a3730 legacy=0x7f36ac09d640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:55.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 1 -- 172.21.15.132:0/3564548013 >> v1:172.21.15.115:6789/0 conn(0x7f36ac137cf0 legacy=0x7f36ac09dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:55.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:25:55.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 1 monclient: found mon.h 2024-01-24T23:25:55.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:55.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:55.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 1 -- 172.21.15.132:0/3564548013 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac0a00f0 con 0x7f36ac13b800 2024-01-24T23:25:55.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 10 monclient: _finish_auth 0 2024-01-24T23:25:55.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:25:55.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:55.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 5 monclient: authenticate success, global_id 6026 2024-01-24T23:25:55.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 10 monclient: _renew_subs 2024-01-24T23:25:55.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:55.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 1 -- 172.21.15.132:0/3564548013 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ac09b220 con 0x7f36ac13b800 2024-01-24T23:25:55.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 10 monclient: _renew_subs 2024-01-24T23:25:55.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:55.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 1 -- 172.21.15.132:0/3564548013 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ac096cc0 con 0x7f36ac13b800 2024-01-24T23:25:55.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 1 -- 172.21.15.132:0/3564548013 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f36a4004800 con 0x7f36ac13b800 2024-01-24T23:25:55.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:55.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 1 -- 172.21.15.132:0/3564548013 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f36a4004d30 con 0x7f36ac13b800 2024-01-24T23:25:55.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:55.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a37fe640 4 set_mon_vals no callback set 2024-01-24T23:25:55.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:25:55.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 10 monclient: dump: 2024-01-24T23:25:55.236 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:55.236 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:55.236 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:55.236 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:55.236 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:55.236 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:55.236 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:55.236 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:55.236 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:55.236 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:55.236 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:55.236 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:55.236 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:55.236 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:55.237 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:55.237 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:55.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 1 -- 172.21.15.132:0/3564548013 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f36a4004f10 con 0x7f36ac13b800 2024-01-24T23:25:55.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:55.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:55.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:55.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b3389640 1 -- 172.21.15.132:0/3564548013 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f368c005090 con 0x7f36ac13b800 2024-01-24T23:25:55.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36a1ffb640 1 -- 172.21.15.132:0/3564548013 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(139..139 src has 1..139) v4 ==== 7727+0+0 (unknown 2351214261 0 0) 0x7f36a40449b0 con 0x7f36ac13b800 2024-01-24T23:25:55.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.220+0000 7f36b1125640 10 monclient: get_auth_request con 0x7f3690036f70 auth_method 0 2024-01-24T23:25:55.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.224+0000 7f36a1ffb640 1 -- 172.21.15.132:0/3564548013 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f36a4046e60 con 0x7f36ac13b800 2024-01-24T23:25:55.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.224+0000 7f36a1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:55.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.224+0000 7f36a1ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:55.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.376+0000 7f36b3389640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}] 2024-01-24T23:25:55.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.376+0000 7f36b3389640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}] 2024-01-24T23:25:55.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.376+0000 7f36b3389640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:55.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.376+0000 7f36b3389640 1 -- 172.21.15.132:0/3564548013 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"} v 0) v1 -- 0x7f368c0058c0 con 0x7f36ac13b800 2024-01-24T23:25:55.983 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.980+0000 7f36a1ffb640 1 -- 172.21.15.132:0/3564548013 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}]=0 v140) v1 ==== 113+0+0 (unknown 3463350675 0 0) 0x7f36a4014d90 con 0x7f36ac13b800 2024-01-24T23:25:55.983 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.980+0000 7f36a1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}] 2024-01-24T23:25:55.983 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:55.980+0000 7f36a1ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:56.088 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.088+0000 7f36b3389640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}] 2024-01-24T23:25:56.088 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.088+0000 7f36b3389640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}] 2024-01-24T23:25:56.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.088+0000 7f36b3389640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:25:56.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.088+0000 7f36b3389640 1 -- 172.21.15.132:0/3564548013 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"} v 0) v1 -- 0x7f368c002220 con 0x7f36ac13b800 2024-01-24T23:25:56.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.088+0000 7f36a1ffb640 1 -- 172.21.15.132:0/3564548013 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}]=0 weight-set for pool 'cold' already created v140) v1 ==== 155+0+0 (unknown 602197741 0 0) 0x7f36a4013200 con 0x7f36ac13b800 2024-01-24T23:25:56.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.088+0000 7f36a1ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}] 2024-01-24T23:25:56.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.088+0000 7f36a1ffb640 10 monclient: _finish_command 3 = system:0 weight-set for pool 'cold' already created 2024-01-24T23:25:56.089 INFO:tasks.workunit.client.0.smithi132.stderr:weight-set for pool 'cold' already created 2024-01-24T23:25:56.091 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.088+0000 7f36b3389640 1 -- 172.21.15.132:0/3564548013 >> v1:172.21.15.132:6812/1835401922 conn(0x7f3690036f70 legacy=0x7f3690039430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:56.091 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.092+0000 7f36b3389640 10 monclient: shutdown 2024-01-24T23:25:56.091 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.092+0000 7f36b3389640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:56.091 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.092+0000 7f36b3389640 1 -- 172.21.15.132:0/3564548013 >> v1:172.21.15.132:6791/0 conn(0x7f36ac13b800 legacy=0x7f36ac1d39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:56.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.092+0000 7f36b3389640 1 -- 172.21.15.132:0/3564548013 shutdown_connections 2024-01-24T23:25:56.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.092+0000 7f36b3389640 1 -- 172.21.15.132:0/3564548013 >> 172.21.15.132:0/3564548013 conn(0x7f36ac0a2c40 msgr2=0x7f36ac1325d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:56.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.092+0000 7f36b3389640 1 -- 172.21.15.132:0/3564548013 shutdown_connections 2024-01-24T23:25:56.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.092+0000 7f36b3389640 1 -- 172.21.15.132:0/3564548013 wait complete. 2024-01-24T23:25:56.106 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm-device-class osd.0 2024-01-24T23:25:56.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.220+0000 7f39f58b9640 10 monclient: get_monmap_and_config 2024-01-24T23:25:56.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.220+0000 7f39f58b9640 10 monclient: build_initial_monmap 2024-01-24T23:25:56.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.220+0000 7f39f58b9640 10 monclient: monmap: 2024-01-24T23:25:56.222 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:56.222 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:56.222 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:56.223263+0000 2024-01-24T23:25:56.222 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:56.223263+0000 2024-01-24T23:25:56.222 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:56.222 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:56.222 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:56.222 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:56.222 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:56.222 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:56.223 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:56.223 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:56.223 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:56.223 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:56.223 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:56.223 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:56.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.220+0000 7f39f58b9640 1 Processor -- start 2024-01-24T23:25:56.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.220+0000 7f39f58b9640 1 -- start start 2024-01-24T23:25:56.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.220+0000 7f39f58b9640 10 monclient: init 2024-01-24T23:25:56.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:56.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 10 monclient: _add_conns ranks=[0,1,4,7,8,5,2,6,3] 2024-01-24T23:25:56.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 10 monclient(hunting): picked mon.noname-a con 0x7f39f01319b0 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:56.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 10 monclient(hunting): picked mon.noname-f con 0x7f39f0131330 addr v1:172.21.15.132:6789/0 2024-01-24T23:25:56.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 10 monclient(hunting): picked mon.noname-c con 0x7f39f0071c00 addr v1:172.21.15.115:6791/0 2024-01-24T23:25:56.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f0064700 con 0x7f39f01319b0 2024-01-24T23:25:56.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f0070b70 con 0x7f39f0131330 2024-01-24T23:25:56.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f00a1fd0 con 0x7f39f0071c00 2024-01-24T23:25:56.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:56.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 10 monclient(hunting): authenticate will time out at 1243.548663s 2024-01-24T23:25:56.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39e7fff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f39f0131330 legacy=0x7f39f00a18c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:25:56.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ef7fe640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f39f0071c00 0x7f39f013dc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:37776/0 (socket says 172.21.15.132:37776) 2024-01-24T23:25:56.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ef7fe640 1 -- 172.21.15.132:0/2699660483 learned_addr learned my addr 172.21.15.132:0/2699660483 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 1 -- 172.21.15.132:0/2699660483 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 85192611 0 0) 0x7f39f00a1fd0 con 0x7f39f0071c00 2024-01-24T23:25:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 10 monclient(hunting): my global_id is 94396 2024-01-24T23:25:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 1 -- 172.21.15.132:0/2699660483 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39dc003530 con 0x7f39f0071c00 2024-01-24T23:25:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 1 -- 172.21.15.132:0/2699660483 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3671635257 0 0) 0x7f39f0064700 con 0x7f39f01319b0 2024-01-24T23:25:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:25:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 10 monclient(hunting): my global_id is 94437 2024-01-24T23:25:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 1 -- 172.21.15.132:0/2699660483 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39f00a1fd0 con 0x7f39f01319b0 2024-01-24T23:25:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 1 -- 172.21.15.132:0/2699660483 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4197894156 0 0) 0x7f39dc003530 con 0x7f39f0071c00 2024-01-24T23:25:56.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 1 -- 172.21.15.132:0/2699660483 >> v1:172.21.15.132:6789/0 conn(0x7f39f0131330 legacy=0x7f39f00a18c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:56.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 1 -- 172.21.15.132:0/2699660483 >> v1:172.21.15.115:6789/0 conn(0x7f39f01319b0 legacy=0x7f39f00a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:56.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:25:56.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 1 monclient: found mon.noname-c 2024-01-24T23:25:56.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:56.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:25:56.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 1 -- 172.21.15.132:0/2699660483 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f00717c0 con 0x7f39f0071c00 2024-01-24T23:25:56.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 10 monclient: _finish_auth 0 2024-01-24T23:25:56.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:25:56.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:56.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 5 monclient: authenticate success, global_id 94396 2024-01-24T23:25:56.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 1 -- 172.21.15.132:0/2699660483 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f39e8001b40 con 0x7f39f0071c00 2024-01-24T23:25:56.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:56.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:56.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:25:56.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 10 monclient: dump: 2024-01-24T23:25:56.230 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:56.230 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:56.230 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:56.230 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:56.230 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:56.230 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:56.230 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:56.230 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:56.230 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:56.230 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:56.230 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:56.230 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:56.231 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:56.231 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:56.231 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:56.231 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:56.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:56.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 1 -- 172.21.15.132:0/2699660483 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f39e8001d60 con 0x7f39f0071c00 2024-01-24T23:25:56.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:56.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:56.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 4 set_mon_vals no callback set 2024-01-24T23:25:56.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 1 -- 172.21.15.132:0/2699660483 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f39e8004f70 con 0x7f39f0071c00 2024-01-24T23:25:56.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:56.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:25:56.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39ee7fc640 10 monclient: dump: 2024-01-24T23:25:56.231 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:56.231 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:56.232 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:56.232 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:56.232 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:56.232 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:56.232 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:56.232 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:56.232 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:56.232 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:56.232 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:56.232 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:56.232 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:56.232 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:56.232 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:56.232 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:56.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 10 monclient: shutdown 2024-01-24T23:25:56.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:56.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 1 -- 172.21.15.132:0/2699660483 >> v1:172.21.15.115:6791/0 conn(0x7f39f0071c00 legacy=0x7f39f013dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:56.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 1 -- 172.21.15.132:0/2699660483 shutdown_connections 2024-01-24T23:25:56.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 1 -- 172.21.15.132:0/2699660483 >> 172.21.15.132:0/2699660483 conn(0x7f39f00a22a0 msgr2=0x7f39f00a2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:56.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 1 -- 172.21.15.132:0/2699660483 shutdown_connections 2024-01-24T23:25:56.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 1 -- 172.21.15.132:0/2699660483 wait complete. 2024-01-24T23:25:56.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 10 monclient: build_initial_monmap 2024-01-24T23:25:56.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 10 monclient: monmap: 2024-01-24T23:25:56.233 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:56.233 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:56.233 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:56.233 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:56.233 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:56.233 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:56.234 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:56.234 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:56.234 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:56.234 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:56.234 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:56.234 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:56.234 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:56.234 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:56.234 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:56.234 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:56.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 1 Processor -- start 2024-01-24T23:25:56.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 1 -- start start 2024-01-24T23:25:56.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.224+0000 7f39f58b9640 10 monclient: init 2024-01-24T23:25:56.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39f58b9640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39f58b9640 10 monclient: _add_conns ranks=[0,7,2,8,3,4,5,6,1] 2024-01-24T23:25:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39f58b9640 10 monclient(hunting): picked mon.noname-a con 0x7f39f0071c00 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39f58b9640 10 monclient(hunting): picked mon.noname-i con 0x7f39f0131330 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39f58b9640 10 monclient(hunting): picked mon.noname-b con 0x7f39f01319b0 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39f58b9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f009d3e0 con 0x7f39f0071c00 2024-01-24T23:25:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39f58b9640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f009d910 con 0x7f39f01319b0 2024-01-24T23:25:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39f58b9640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f006f500 con 0x7f39f0131330 2024-01-24T23:25:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39f58b9640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39f58b9640 10 monclient(hunting): authenticate will time out at 1243.552372s 2024-01-24T23:25:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39e7fff640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f39f0131330 legacy=0x7f39f006edf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:25:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ef7fe640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f39f01319b0 legacy=0x7f39f01d3a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:25:56.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39eeffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f39f0071c00 0x7f39f006e6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:52014/0 (socket says 172.21.15.132:52014) 2024-01-24T23:25:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39eeffd640 1 -- 172.21.15.132:0/3260655000 learned_addr learned my addr 172.21.15.132:0/3260655000 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 1 -- 172.21.15.132:0/3260655000 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309311701 0 0) 0x7f39f009d3e0 con 0x7f39f0071c00 2024-01-24T23:25:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 10 monclient(hunting): _negotiate 2024-01-24T23:25:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 10 monclient(hunting): my global_id is 94446 2024-01-24T23:25:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 1 -- 172.21.15.132:0/3260655000 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39d0003530 con 0x7f39f0071c00 2024-01-24T23:25:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 1 -- 172.21.15.132:0/3260655000 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3797964448 0 0) 0x7f39d0003530 con 0x7f39f0071c00 2024-01-24T23:25:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 1 -- 172.21.15.132:0/3260655000 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f39f009d3e0 con 0x7f39f0071c00 2024-01-24T23:25:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 1 -- 172.21.15.132:0/3260655000 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f39e00019a0 con 0x7f39f0071c00 2024-01-24T23:25:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:25:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 10 monclient(hunting): dump: 2024-01-24T23:25:56.240 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:56.241 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:56.242 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 1 -- 172.21.15.132:0/3260655000 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1690884196 0 0) 0x7f39f009d3e0 con 0x7f39f0071c00 2024-01-24T23:25:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 1 -- 172.21.15.132:0/3260655000 >> v1:172.21.15.132:6792/0 conn(0x7f39f0131330 legacy=0x7f39f006edf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 1 -- 172.21.15.132:0/3260655000 >> v1:172.21.15.115:6790/0 conn(0x7f39f01319b0 legacy=0x7f39f01d3a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 10 monclient: _finish_hunting 0 2024-01-24T23:25:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 1 monclient: found mon.a 2024-01-24T23:25:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 1 -- 172.21.15.132:0/3260655000 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f00701d0 con 0x7f39f0071c00 2024-01-24T23:25:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 10 monclient: _finish_auth 0 2024-01-24T23:25:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 10 monclient: _check_auth_tickets 2024-01-24T23:25:56.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39f58b9640 5 monclient: authenticate success, global_id 94446 2024-01-24T23:25:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39f58b9640 10 monclient: _renew_subs 2024-01-24T23:25:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39f58b9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39f58b9640 1 -- 172.21.15.132:0/3260655000 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f009f550 con 0x7f39f0071c00 2024-01-24T23:25:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39f58b9640 10 monclient: _renew_subs 2024-01-24T23:25:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39f58b9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39f58b9640 1 -- 172.21.15.132:0/3260655000 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f009c110 con 0x7f39f0071c00 2024-01-24T23:25:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 1 -- 172.21.15.132:0/3260655000 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f39e0001c80 con 0x7f39f0071c00 2024-01-24T23:25:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39edffb640 4 set_mon_vals no callback set 2024-01-24T23:25:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 1 -- 172.21.15.132:0/3260655000 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f39e0003c10 con 0x7f39f0071c00 2024-01-24T23:25:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:56.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 10 monclient: dump: 2024-01-24T23:25:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:56.244 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:56.245 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:56.245 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:56.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39f58b9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:56.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39f58b9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:56.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39f58b9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:56.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39f58b9640 1 -- 172.21.15.132:0/3260655000 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d4005090 con 0x7f39f0071c00 2024-01-24T23:25:56.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 1 -- 172.21.15.132:0/3260655000 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f39e0003e30 con 0x7f39f0071c00 2024-01-24T23:25:56.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.228+0000 7f39ecff9640 1 -- 172.21.15.132:0/3260655000 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(140..140 src has 1..140) v4 ==== 8199+0+0 (unknown 4179166854 0 0) 0x7f39e0044800 con 0x7f39f0071c00 2024-01-24T23:25:56.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.232+0000 7f39e7fff640 10 monclient: get_auth_request con 0x7f39d0037030 auth_method 0 2024-01-24T23:25:56.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.244+0000 7f39ecff9640 1 -- 172.21.15.132:0/3260655000 <== mon.0 v1:172.21.15.115:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f39e00146f0 con 0x7f39f0071c00 2024-01-24T23:25:56.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.244+0000 7f39ecff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:56.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.244+0000 7f39ecff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:56.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.400+0000 7f39f58b9640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-01-24T23:25:56.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.400+0000 7f39f58b9640 10 monclient: _send_command 2 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-01-24T23:25:56.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.400+0000 7f39f58b9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:56.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.400+0000 7f39f58b9640 1 -- 172.21.15.132:0/3260655000 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["osd.0"]} v 0) v1 -- 0x7f39d4005b50 con 0x7f39f0071c00 2024-01-24T23:25:56.893 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.892+0000 7f39ecff9640 1 -- 172.21.15.132:0/3260655000 <== mon.0 v1:172.21.15.115:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}]=0 done removing class of osd(s): 0 v141) v1 ==== 123+0+0 (unknown 94469651 0 0) 0x7f39e0012d90 con 0x7f39f0071c00 2024-01-24T23:25:56.893 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.892+0000 7f39ecff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-01-24T23:25:56.894 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.892+0000 7f39ecff9640 10 monclient: _finish_command 2 = system:0 done removing class of osd(s): 0 2024-01-24T23:25:56.994 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.992+0000 7f39f58b9640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-01-24T23:25:56.994 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.992+0000 7f39f58b9640 10 monclient: _send_command 3 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-01-24T23:25:56.994 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.992+0000 7f39f58b9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:56.994 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:56.992+0000 7f39f58b9640 1 -- 172.21.15.132:0/3260655000 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["osd.0"]} v 0) v1 -- 0x7f39d40055d0 con 0x7f39f0071c00 2024-01-24T23:25:58.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.008+0000 7f39ecff9640 1 -- 172.21.15.132:0/3260655000 <== mon.0 v1:172.21.15.115:6789/0 11 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}]=0 osd.0 belongs to no class, done removing class of osd(s): v142) v1 ==== 149+0+0 (unknown 2352630098 0 0) 0x7f39e0014510 con 0x7f39f0071c00 2024-01-24T23:25:58.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.008+0000 7f39ecff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-01-24T23:25:58.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.008+0000 7f39ecff9640 10 monclient: _finish_command 3 = system:0 osd.0 belongs to no class, done removing class of osd(s): 2024-01-24T23:25:58.009 INFO:tasks.workunit.client.0.smithi132.stderr:osd.0 belongs to no class, done removing class of osd(s): 2024-01-24T23:25:58.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.012+0000 7f39f58b9640 1 -- 172.21.15.132:0/3260655000 >> v1:172.21.15.132:6812/1835401922 conn(0x7f39d0037030 legacy=0x7f39d00394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:58.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.012+0000 7f39f58b9640 10 monclient: shutdown 2024-01-24T23:25:58.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.012+0000 7f39f58b9640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:58.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.012+0000 7f39f58b9640 1 -- 172.21.15.132:0/3260655000 >> v1:172.21.15.115:6789/0 conn(0x7f39f0071c00 legacy=0x7f39f006e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:58.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.012+0000 7f39f58b9640 1 -- 172.21.15.132:0/3260655000 shutdown_connections 2024-01-24T23:25:58.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.012+0000 7f39f58b9640 1 -- 172.21.15.132:0/3260655000 >> 172.21.15.132:0/3260655000 conn(0x7f39f00a22a0 msgr2=0x7f39f012f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:58.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.012+0000 7f39f58b9640 1 -- 172.21.15.132:0/3260655000 shutdown_connections 2024-01-24T23:25:58.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.012+0000 7f39f58b9640 1 -- 172.21.15.132:0/3260655000 wait complete. 2024-01-24T23:25:58.031 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set reweight-compat osd.0 10.5 2024-01-24T23:25:58.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb4624bc640 10 monclient: get_monmap_and_config 2024-01-24T23:25:58.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb4624bc640 10 monclient: build_initial_monmap 2024-01-24T23:25:58.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb4624bc640 10 monclient: monmap: 2024-01-24T23:25:58.149 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:58.149 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:58.149 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:25:58.150344+0000 2024-01-24T23:25:58.149 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:25:58.150344+0000 2024-01-24T23:25:58.149 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:58.149 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:58.149 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:58.150 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:58.150 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:58.150 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:58.150 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:58.150 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:58.150 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:58.150 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:58.150 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:58.150 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:58.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb4624bc640 1 Processor -- start 2024-01-24T23:25:58.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb4624bc640 1 -- start start 2024-01-24T23:25:58.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb4624bc640 10 monclient: init 2024-01-24T23:25:58.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb4624bc640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:58.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb4624bc640 10 monclient: _add_conns ranks=[3,5,2,4,8,0,1,7,6] 2024-01-24T23:25:58.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb4624bc640 10 monclient(hunting): picked mon.noname-g con 0x7fb45c0a0ad0 addr v1:172.21.15.132:6790/0 2024-01-24T23:25:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb4624bc640 10 monclient(hunting): picked mon.noname-h con 0x7fb45c09ee80 addr v1:172.21.15.132:6791/0 2024-01-24T23:25:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb4624bc640 10 monclient(hunting): picked mon.noname-b con 0x7fb45c09b060 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb4624bc640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45c064700 con 0x7fb45c09b060 2024-01-24T23:25:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb4624bc640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45c09cb80 con 0x7fb45c0a0ad0 2024-01-24T23:25:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb4624bc640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45c0a0eb0 con 0x7fb45c09ee80 2024-01-24T23:25:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb4624bc640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb4624bc640 10 monclient(hunting): authenticate will time out at 1245.475045s 2024-01-24T23:25:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45bfff640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fb45c0a0ad0 0x7fb45c09a240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:46844/0 (socket says 172.21.15.132:46844) 2024-01-24T23:25:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45bfff640 1 -- 172.21.15.132:0/867377057 learned_addr learned my addr 172.21.15.132:0/867377057 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb460a59640 1 -- 172.21.15.132:0/867377057 >> v1:172.21.15.115:6790/0 conn(0x7fb45c09b060 legacy=0x7fb45c13c270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:25:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 1 -- 172.21.15.132:0/867377057 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1161184715 0 0) 0x7fb45c09cb80 con 0x7fb45c0a0ad0 2024-01-24T23:25:58.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 10 monclient(hunting): my global_id is 6141 2024-01-24T23:25:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 1 -- 172.21.15.132:0/867377057 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb444003530 con 0x7fb45c0a0ad0 2024-01-24T23:25:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 1 -- 172.21.15.132:0/867377057 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1471199266 0 0) 0x7fb45c0a0eb0 con 0x7fb45c09ee80 2024-01-24T23:25:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 10 monclient(hunting): _negotiate 2024-01-24T23:25:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 10 monclient(hunting): my global_id is 6035 2024-01-24T23:25:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 1 -- 172.21.15.132:0/867377057 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb45c09cb80 con 0x7fb45c09ee80 2024-01-24T23:25:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 1 -- 172.21.15.132:0/867377057 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3041920143 0 0) 0x7fb444003530 con 0x7fb45c0a0ad0 2024-01-24T23:25:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 1 -- 172.21.15.132:0/867377057 >> v1:172.21.15.132:6791/0 conn(0x7fb45c09ee80 legacy=0x7fb45c09a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:58.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 1 -- 172.21.15.132:0/867377057 >> v1:172.21.15.115:6790/0 conn(0x7fb45c09b060 legacy=0x7fb45c13c270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:25:58.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 10 monclient: _finish_hunting 0 2024-01-24T23:25:58.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 1 monclient: found mon.noname-g 2024-01-24T23:25:58.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:58.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:25:58.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 1 -- 172.21.15.132:0/867377057 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45c09f280 con 0x7fb45c0a0ad0 2024-01-24T23:25:58.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 10 monclient: _finish_auth 0 2024-01-24T23:25:58.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 10 monclient: _check_auth_tickets 2024-01-24T23:25:58.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:58.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb4624bc640 5 monclient: authenticate success, global_id 6141 2024-01-24T23:25:58.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 1 -- 172.21.15.132:0/867377057 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb44c0017c0 con 0x7fb45c0a0ad0 2024-01-24T23:25:58.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb4624bc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:58.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:58.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.148+0000 7fb45affd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:25:58.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb45affd640 10 monclient: dump: 2024-01-24T23:25:58.162 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:58.162 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:58.162 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:58.162 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:58.162 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:58.162 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:58.162 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:58.162 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:58.162 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:58.162 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:58.162 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:58.162 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:58.162 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:58.162 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:58.163 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:58.163 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:58.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:25:58.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb45affd640 1 -- 172.21.15.132:0/867377057 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb44c0019b0 con 0x7fb45c0a0ad0 2024-01-24T23:25:58.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb45affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:58.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 10 monclient: get_monmap_and_config success 2024-01-24T23:25:58.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb45affd640 1 -- 172.21.15.132:0/867377057 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb44c001be0 con 0x7fb45c0a0ad0 2024-01-24T23:25:58.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 4 set_mon_vals no callback set 2024-01-24T23:25:58.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb45affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:58.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb45affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:25:58.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb45affd640 10 monclient: dump: 2024-01-24T23:25:58.163 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:58.163 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:58.163 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:58.164 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:58.164 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:58.164 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:58.164 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:58.164 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:58.164 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:58.164 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:58.164 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:58.164 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:58.164 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:58.164 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:58.164 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:58.164 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:58.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 10 monclient: shutdown 2024-01-24T23:25:58.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:25:58.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 1 -- 172.21.15.132:0/867377057 >> v1:172.21.15.132:6790/0 conn(0x7fb45c0a0ad0 legacy=0x7fb45c09a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:25:58.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 1 -- 172.21.15.132:0/867377057 shutdown_connections 2024-01-24T23:25:58.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 1 -- 172.21.15.132:0/867377057 >> 172.21.15.132:0/867377057 conn(0x7fb45c0a1bc0 msgr2=0x7fb45c0a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:25:58.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 1 -- 172.21.15.132:0/867377057 shutdown_connections 2024-01-24T23:25:58.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 1 -- 172.21.15.132:0/867377057 wait complete. 2024-01-24T23:25:58.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 10 monclient: build_initial_monmap 2024-01-24T23:25:58.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 10 monclient: monmap: 2024-01-24T23:25:58.165 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:25:58.165 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:25:58.165 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:25:58.165 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:25:58.165 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:25:58.165 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:25:58.165 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:25:58.165 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:25:58.165 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:25:58.165 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:25:58.166 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:25:58.166 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:25:58.166 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:25:58.166 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:25:58.166 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:25:58.166 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:58.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 1 Processor -- start 2024-01-24T23:25:58.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 1 -- start start 2024-01-24T23:25:58.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 10 monclient: init 2024-01-24T23:25:58.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 10 monclient: _reopen_session rank -1 2024-01-24T23:25:58.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 10 monclient: _add_conns ranks=[7,2,0,8,5,1,6,4,3] 2024-01-24T23:25:58.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 10 monclient(hunting): picked mon.noname-i con 0x7fb45c09b060 addr v1:172.21.15.132:6792/0 2024-01-24T23:25:58.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 10 monclient(hunting): picked mon.noname-b con 0x7fb45c09ee80 addr v1:172.21.15.115:6790/0 2024-01-24T23:25:58.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 10 monclient(hunting): picked mon.noname-a con 0x7fb45c0a0ad0 addr v1:172.21.15.115:6789/0 2024-01-24T23:25:58.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45c070ec0 con 0x7fb45c0a0ad0 2024-01-24T23:25:58.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb45bfff640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fb45c09b060 legacy=0x7fb45c1b32b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:25:58.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45c0701b0 con 0x7fb45c09ee80 2024-01-24T23:25:58.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45c09e640 con 0x7fb45c09b060 2024-01-24T23:25:58.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 10 monclient(hunting): _renew_subs 2024-01-24T23:25:58.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb4624bc640 10 monclient(hunting): authenticate will time out at 1245.478379s 2024-01-24T23:25:58.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb45b7fe640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fb45c09ee80 legacy=0x7fb45c1ce570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:25:58.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb460a59640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb45c0a0ad0 0x7fb45c1d1ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:52016/0 (socket says 172.21.15.132:52016) 2024-01-24T23:25:58.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb460a59640 1 -- 172.21.15.132:0/2282799774 learned_addr learned my addr 172.21.15.132:0/2282799774 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:25:58.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb458ff9640 1 -- 172.21.15.132:0/2282799774 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3573920369 0 0) 0x7fb45c070ec0 con 0x7fb45c0a0ad0 2024-01-24T23:25:58.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb458ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:25:58.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb458ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:25:58.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb458ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:25:58.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb458ff9640 10 monclient(hunting): my global_id is 94455 2024-01-24T23:25:58.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb458ff9640 1 -- 172.21.15.132:0/2282799774 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb448003530 con 0x7fb45c0a0ad0 2024-01-24T23:25:58.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb458ff9640 1 -- 172.21.15.132:0/2282799774 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3416589349 0 0) 0x7fb448003530 con 0x7fb45c0a0ad0 2024-01-24T23:25:58.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb458ff9640 1 -- 172.21.15.132:0/2282799774 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb45c070ec0 con 0x7fb45c0a0ad0 2024-01-24T23:25:58.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb458ff9640 1 -- 172.21.15.132:0/2282799774 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb4540045a0 con 0x7fb45c0a0ad0 2024-01-24T23:25:58.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb458ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:58.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb458ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:25:58.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb458ff9640 10 monclient(hunting): dump: 2024-01-24T23:25:58.168 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:58.168 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:58.168 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:58.168 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:58.168 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:58.169 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:58.169 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:58.169 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:58.169 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:58.169 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:58.169 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:58.169 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:58.169 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:58.169 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:58.169 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:58.169 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:58.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.152+0000 7fb458ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:25:58.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb458ff9640 1 -- 172.21.15.132:0/2282799774 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3726708802 0 0) 0x7fb45c070ec0 con 0x7fb45c0a0ad0 2024-01-24T23:25:58.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb458ff9640 1 -- 172.21.15.132:0/2282799774 >> v1:172.21.15.132:6792/0 conn(0x7fb45c09b060 legacy=0x7fb45c1b32b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:58.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb458ff9640 1 -- 172.21.15.132:0/2282799774 >> v1:172.21.15.115:6790/0 conn(0x7fb45c09ee80 legacy=0x7fb45c1ce570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:25:58.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb458ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:25:58.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb458ff9640 1 monclient: found mon.a 2024-01-24T23:25:58.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb458ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:25:58.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb458ff9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:58.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb458ff9640 1 -- 172.21.15.132:0/2282799774 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45c09b4c0 con 0x7fb45c0a0ad0 2024-01-24T23:25:58.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb458ff9640 10 monclient: _finish_auth 0 2024-01-24T23:25:58.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb458ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:25:58.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb458ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:25:58.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb4624bc640 5 monclient: authenticate success, global_id 94455 2024-01-24T23:25:58.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb4624bc640 10 monclient: _renew_subs 2024-01-24T23:25:58.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb4624bc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:58.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb4624bc640 1 -- 172.21.15.132:0/2282799774 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb45c071a40 con 0x7fb45c0a0ad0 2024-01-24T23:25:58.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb4624bc640 10 monclient: _renew_subs 2024-01-24T23:25:58.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb4624bc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:58.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb4624bc640 1 -- 172.21.15.132:0/2282799774 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb45c1b6420 con 0x7fb45c0a0ad0 2024-01-24T23:25:58.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb458ff9640 1 -- 172.21.15.132:0/2282799774 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb454004850 con 0x7fb45c0a0ad0 2024-01-24T23:25:58.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb458ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:25:58.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb45a7fc640 4 set_mon_vals no callback set 2024-01-24T23:25:58.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb458ff9640 1 -- 172.21.15.132:0/2282799774 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb454004d80 con 0x7fb45c0a0ad0 2024-01-24T23:25:58.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb458ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:25:58.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb458ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:25:58.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb458ff9640 10 monclient: dump: 2024-01-24T23:25:58.171 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:25:58.171 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:25:58.171 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:58.171 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:25:58.171 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:25:58.172 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:25:58.172 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:25:58.172 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:25:58.172 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:25:58.172 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:25:58.172 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:25:58.172 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:25:58.172 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:25:58.172 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:25:58.172 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:25:58.172 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:25:58.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb4624bc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:58.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb4624bc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:58.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb4624bc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:58.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb4624bc640 1 -- 172.21.15.132:0/2282799774 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb43c005090 con 0x7fb45c0a0ad0 2024-01-24T23:25:58.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb458ff9640 1 -- 172.21.15.132:0/2282799774 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fb454004f60 con 0x7fb45c0a0ad0 2024-01-24T23:25:58.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb458ff9640 1 -- 172.21.15.132:0/2282799774 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(142..142 src has 1..142) v4 ==== 8167+0+0 (unknown 2753046764 0 0) 0x7fb454044be0 con 0x7fb45c0a0ad0 2024-01-24T23:25:58.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.156+0000 7fb45bfff640 10 monclient: get_auth_request con 0x7fb448036ed0 auth_method 0 2024-01-24T23:25:58.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.168+0000 7fb458ff9640 1 -- 172.21.15.132:0/2282799774 <== mon.0 v1:172.21.15.115:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fb454014cd0 con 0x7fb45c0a0ad0 2024-01-24T23:25:58.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.168+0000 7fb458ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:25:58.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.168+0000 7fb458ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:25:58.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.332+0000 7fb4624bc640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}] 2024-01-24T23:25:58.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.332+0000 7fb4624bc640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}] 2024-01-24T23:25:58.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.332+0000 7fb4624bc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:58.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:58.332+0000 7fb4624bc640 1 -- 172.21.15.132:0/2282799774 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]} v 0) v1 -- 0x7fb43c005340 con 0x7fb45c0a0ad0 2024-01-24T23:25:59.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:59.016+0000 7fb458ff9640 1 -- 172.21.15.132:0/2282799774 <== mon.0 v1:172.21.15.115:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}]=0 v143) v1 ==== 119+0+0 (unknown 3021120098 0 0) 0x7fb4540148c0 con 0x7fb45c0a0ad0 2024-01-24T23:25:59.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:59.016+0000 7fb458ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}] 2024-01-24T23:25:59.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:59.016+0000 7fb458ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:25:59.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:59.124+0000 7fb4624bc640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}] 2024-01-24T23:25:59.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:59.124+0000 7fb4624bc640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}] 2024-01-24T23:25:59.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:59.124+0000 7fb4624bc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:25:59.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:25:59.124+0000 7fb4624bc640 1 -- 172.21.15.132:0/2282799774 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]} v 0) v1 -- 0x7fb43c005a80 con 0x7fb45c0a0ad0 2024-01-24T23:26:00.043 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.040+0000 7fb458ff9640 1 -- 172.21.15.132:0/2282799774 <== mon.0 v1:172.21.15.115:6789/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}]=0 v144) v1 ==== 119+0+0 (unknown 4036482921 0 0) 0x7fb454014aa0 con 0x7fb45c0a0ad0 2024-01-24T23:26:00.043 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.040+0000 7fb458ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}] 2024-01-24T23:26:00.043 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.040+0000 7fb458ff9640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:26:00.046 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.044+0000 7fb4624bc640 1 -- 172.21.15.132:0/2282799774 >> v1:172.21.15.132:6812/1835401922 conn(0x7fb448036ed0 legacy=0x7fb448039390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:00.046 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.044+0000 7fb4624bc640 10 monclient: shutdown 2024-01-24T23:26:00.047 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.044+0000 7fb4624bc640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:00.047 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.044+0000 7fb4624bc640 1 -- 172.21.15.132:0/2282799774 >> v1:172.21.15.115:6789/0 conn(0x7fb45c0a0ad0 legacy=0x7fb45c1d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:00.047 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.044+0000 7fb4624bc640 1 -- 172.21.15.132:0/2282799774 shutdown_connections 2024-01-24T23:26:00.047 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.044+0000 7fb4624bc640 1 -- 172.21.15.132:0/2282799774 >> 172.21.15.132:0/2282799774 conn(0x7fb45c0a1bc0 msgr2=0x7fb45c1bca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:00.047 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.044+0000 7fb4624bc640 1 -- 172.21.15.132:0/2282799774 shutdown_connections 2024-01-24T23:26:00.047 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.044+0000 7fb4624bc640 1 -- 172.21.15.132:0/2282799774 wait complete. 2024-01-24T23:26:00.074 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set reweight cool osd.0 11.5 2024-01-24T23:26:00.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6d669d640 10 monclient: get_monmap_and_config 2024-01-24T23:26:00.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6d669d640 10 monclient: build_initial_monmap 2024-01-24T23:26:00.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6d669d640 10 monclient: monmap: 2024-01-24T23:26:00.201 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:00.201 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:00.201 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:00.202121+0000 2024-01-24T23:26:00.201 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:00.202121+0000 2024-01-24T23:26:00.201 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:00.201 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:00.201 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:00.201 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:00.201 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:00.201 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:00.201 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:00.201 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:00.201 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:00.201 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:00.202 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:00.202 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:00.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6d669d640 1 Processor -- start 2024-01-24T23:26:00.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6d669d640 1 -- start start 2024-01-24T23:26:00.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6d669d640 10 monclient: init 2024-01-24T23:26:00.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6d669d640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:00.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6d669d640 10 monclient: _add_conns ranks=[8,7,3,5,0,4,1,6,2] 2024-01-24T23:26:00.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6d669d640 10 monclient(hunting): picked mon.noname-e con 0x7fe6d009b810 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:00.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6d669d640 10 monclient(hunting): picked mon.noname-i con 0x7fe6d009bbf0 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:00.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6d669d640 10 monclient(hunting): picked mon.noname-g con 0x7fe6d01370a0 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:00.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6d669d640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6d0064700 con 0x7fe6d01370a0 2024-01-24T23:26:00.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6d669d640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6d0099af0 con 0x7fe6d009bbf0 2024-01-24T23:26:00.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6d669d640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6d0072110 con 0x7fe6d009b810 2024-01-24T23:26:00.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6d669d640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:00.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6d669d640 10 monclient(hunting): authenticate will time out at 1247.526682s 2024-01-24T23:26:00.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6cf7fe640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fe6d009bbf0 legacy=0x7fe6d0071a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:00.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6d4c3a640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fe6d01370a0 0x7fe6d013ff30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:46846/0 (socket says 172.21.15.132:46846) 2024-01-24T23:26:00.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6d4c3a640 1 -- 172.21.15.132:0/3313657513 learned_addr learned my addr 172.21.15.132:0/3313657513 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:00.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 1 -- 172.21.15.132:0/3313657513 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2820458753 0 0) 0x7fe6d0064700 con 0x7fe6d01370a0 2024-01-24T23:26:00.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:00.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:00.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:00.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 10 monclient(hunting): my global_id is 6150 2024-01-24T23:26:00.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 1 -- 172.21.15.132:0/3313657513 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6b8003530 con 0x7fe6d01370a0 2024-01-24T23:26:00.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 1 -- 172.21.15.132:0/3313657513 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2507530368 0 0) 0x7fe6b8003530 con 0x7fe6d01370a0 2024-01-24T23:26:00.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 1 -- 172.21.15.132:0/3313657513 >> v1:172.21.15.115:6793/0 conn(0x7fe6d009b810 legacy=0x7fe6d00712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:00.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 1 -- 172.21.15.132:0/3313657513 >> v1:172.21.15.132:6792/0 conn(0x7fe6d009bbf0 legacy=0x7fe6d0071a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:00.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 10 monclient: _finish_hunting 0 2024-01-24T23:26:00.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 1 monclient: found mon.noname-g 2024-01-24T23:26:00.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:00.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:26:00.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 1 -- 172.21.15.132:0/3313657513 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d009c050 con 0x7fe6d01370a0 2024-01-24T23:26:00.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 10 monclient: _finish_auth 0 2024-01-24T23:26:00.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 10 monclient: _check_auth_tickets 2024-01-24T23:26:00.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:00.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6d669d640 5 monclient: authenticate success, global_id 6150 2024-01-24T23:26:00.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6d669d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:00.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 1 -- 172.21.15.132:0/3313657513 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe6c80017c0 con 0x7fe6d01370a0 2024-01-24T23:26:00.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:00.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:00.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 10 monclient: dump: 2024-01-24T23:26:00.210 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:00.210 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:00.210 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:00.210 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:00.210 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:00.210 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:00.210 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:00.211 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:00.211 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:00.211 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:00.211 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:00.211 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:00.211 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:00.211 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:00.211 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:00.211 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:00.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6d669d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:00.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 1 -- 172.21.15.132:0/3313657513 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe6c8001a70 con 0x7fe6d01370a0 2024-01-24T23:26:00.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:00.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6d669d640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:00.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 1 -- 172.21.15.132:0/3313657513 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe6c8004bf0 con 0x7fe6d01370a0 2024-01-24T23:26:00.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6d669d640 4 set_mon_vals no callback set 2024-01-24T23:26:00.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:00.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:00.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.200+0000 7fe6ceffd640 10 monclient: dump: 2024-01-24T23:26:00.212 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:00.212 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:00.212 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:00.212 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:00.212 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:00.212 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:00.212 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:00.212 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:00.212 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:00.212 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:00.212 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:00.212 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:00.213 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:00.213 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:00.213 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:00.213 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:00.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 10 monclient: shutdown 2024-01-24T23:26:00.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:00.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 1 -- 172.21.15.132:0/3313657513 >> v1:172.21.15.132:6790/0 conn(0x7fe6d01370a0 legacy=0x7fe6d013ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:00.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 1 -- 172.21.15.132:0/3313657513 shutdown_connections 2024-01-24T23:26:00.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 1 -- 172.21.15.132:0/3313657513 >> 172.21.15.132:0/3313657513 conn(0x7fe6d00a1bc0 msgr2=0x7fe6d00a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:00.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 1 -- 172.21.15.132:0/3313657513 shutdown_connections 2024-01-24T23:26:00.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 1 -- 172.21.15.132:0/3313657513 wait complete. 2024-01-24T23:26:00.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 10 monclient: build_initial_monmap 2024-01-24T23:26:00.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 10 monclient: monmap: 2024-01-24T23:26:00.213 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:00.214 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:00.214 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:00.214 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:00.214 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:00.214 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:00.214 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:00.214 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:00.214 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:00.214 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:00.214 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:00.214 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:00.214 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:00.214 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:00.214 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:00.214 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:00.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 1 Processor -- start 2024-01-24T23:26:00.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 1 -- start start 2024-01-24T23:26:00.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 10 monclient: init 2024-01-24T23:26:00.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:00.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 10 monclient: _add_conns ranks=[0,3,6,2,1,5,7,8,4] 2024-01-24T23:26:00.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 10 monclient(hunting): picked mon.noname-a con 0x7fe6d009b810 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:00.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 10 monclient(hunting): picked mon.noname-g con 0x7fe6d009bbf0 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:00.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 10 monclient(hunting): picked mon.noname-d con 0x7fe6d01370a0 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:00.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6d0132f00 con 0x7fe6d009b810 2024-01-24T23:26:00.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6d012abe0 con 0x7fe6d009bbf0 2024-01-24T23:26:00.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6d009b5e0 con 0x7fe6d01370a0 2024-01-24T23:26:00.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:00.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 10 monclient(hunting): authenticate will time out at 1247.529814s 2024-01-24T23:26:00.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6cf7fe640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fe6d009bbf0 0x7fe6d01d05e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:46862/0 (socket says 172.21.15.132:46862) 2024-01-24T23:26:00.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6cf7fe640 1 -- 172.21.15.132:0/998838130 learned_addr learned my addr 172.21.15.132:0/998838130 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:00.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 1 -- 172.21.15.132:0/998838130 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4198230093 0 0) 0x7fe6d012abe0 con 0x7fe6d009bbf0 2024-01-24T23:26:00.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 10 monclient(hunting): _negotiate 2024-01-24T23:26:00.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:00.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:00.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 10 monclient(hunting): my global_id is 6159 2024-01-24T23:26:00.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 1 -- 172.21.15.132:0/998838130 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6bc003530 con 0x7fe6d009bbf0 2024-01-24T23:26:00.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 1 -- 172.21.15.132:0/998838130 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3252061879 0 0) 0x7fe6bc003530 con 0x7fe6d009bbf0 2024-01-24T23:26:00.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 1 -- 172.21.15.132:0/998838130 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe6d012abe0 con 0x7fe6d009bbf0 2024-01-24T23:26:00.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 1 -- 172.21.15.132:0/998838130 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe6c0004680 con 0x7fe6d009bbf0 2024-01-24T23:26:00.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:00.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:00.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 10 monclient(hunting): dump: 2024-01-24T23:26:00.217 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:00.217 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:00.217 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:00.217 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:00.217 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:00.217 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:00.217 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:00.217 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:00.217 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:00.217 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:00.217 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:00.217 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:00.217 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:00.217 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:00.217 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:00.218 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:00.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:00.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 1 -- 172.21.15.132:0/998838130 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3319077470 0 0) 0x7fe6d012abe0 con 0x7fe6d009bbf0 2024-01-24T23:26:00.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 1 -- 172.21.15.132:0/998838130 >> v1:172.21.15.115:6792/0 conn(0x7fe6d01370a0 legacy=0x7fe6d01d3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:00.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 1 -- 172.21.15.132:0/998838130 >> v1:172.21.15.115:6789/0 conn(0x7fe6d009b810 legacy=0x7fe6d0130680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:00.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 10 monclient: _finish_hunting 0 2024-01-24T23:26:00.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 1 monclient: found mon.g 2024-01-24T23:26:00.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:00.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:00.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 1 -- 172.21.15.132:0/998838130 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d0134120 con 0x7fe6d009bbf0 2024-01-24T23:26:00.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 10 monclient: _finish_auth 0 2024-01-24T23:26:00.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 10 monclient: _check_auth_tickets 2024-01-24T23:26:00.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:00.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 5 monclient: authenticate success, global_id 6159 2024-01-24T23:26:00.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 10 monclient: _renew_subs 2024-01-24T23:26:00.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:00.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 1 -- 172.21.15.132:0/998838130 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d012b110 con 0x7fe6d009bbf0 2024-01-24T23:26:00.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 10 monclient: _renew_subs 2024-01-24T23:26:00.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:00.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 1 -- 172.21.15.132:0/998838130 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d01cb520 con 0x7fe6d009bbf0 2024-01-24T23:26:00.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 1 -- 172.21.15.132:0/998838130 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe6c0004930 con 0x7fe6d009bbf0 2024-01-24T23:26:00.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:00.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 1 -- 172.21.15.132:0/998838130 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe6c0004e60 con 0x7fe6d009bbf0 2024-01-24T23:26:00.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:00.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6d669d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:00.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:00.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 10 monclient: dump: 2024-01-24T23:26:00.219 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:00.219 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:00.219 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:00.220 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:00.220 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:00.220 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:00.220 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:00.220 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:00.220 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:00.220 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:00.220 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:00.220 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:00.220 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:00.220 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:00.220 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:00.220 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:00.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ce7fc640 4 set_mon_vals no callback set 2024-01-24T23:26:00.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 1 -- 172.21.15.132:0/998838130 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fe6c0005040 con 0x7fe6d009bbf0 2024-01-24T23:26:00.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.204+0000 7fe6ccff9640 1 -- 172.21.15.132:0/998838130 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(144..144 src has 1..144) v4 ==== 8378+0+0 (unknown 389647491 0 0) 0x7fe6c0044d70 con 0x7fe6d009bbf0 2024-01-24T23:26:00.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.208+0000 7fe6d669d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:00.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.208+0000 7fe6d669d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:00.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.208+0000 7fe6d669d640 1 -- 172.21.15.132:0/998838130 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6d00750f0 con 0x7fe6d009bbf0 2024-01-24T23:26:00.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.208+0000 7fe6cffff640 10 monclient: get_auth_request con 0x7fe6bc036f60 auth_method 0 2024-01-24T23:26:00.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.212+0000 7fe6ccff9640 1 -- 172.21.15.132:0/998838130 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fe6c0014e90 con 0x7fe6d009bbf0 2024-01-24T23:26:00.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.212+0000 7fe6ccff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:00.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.212+0000 7fe6ccff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:00.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.376+0000 7fe6d669d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}] 2024-01-24T23:26:00.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.376+0000 7fe6d669d640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}] 2024-01-24T23:26:00.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.376+0000 7fe6d669d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:00.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:00.376+0000 7fe6d669d640 1 -- 172.21.15.132:0/998838130 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]} v 0) v1 -- 0x7fe6d0071820 con 0x7fe6d009bbf0 2024-01-24T23:26:01.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:01.068+0000 7fe6ccff9640 1 -- 172.21.15.132:0/998838130 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}]=0 v145) v1 ==== 128+0+0 (unknown 22163192 0 0) 0x7fe6c0013300 con 0x7fe6d009bbf0 2024-01-24T23:26:01.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:01.068+0000 7fe6ccff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}] 2024-01-24T23:26:01.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:01.068+0000 7fe6ccff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:26:01.105 INFO:tasks.ceph.osd.2.smithi115.stderr:terminate called after throwing an instance of 'ceph::buffer::v15_2_0::end_of_buffer' 2024-01-24T23:26:01.105 INFO:tasks.ceph.osd.2.smithi115.stderr: what(): End of buffer [buffer:2] 2024-01-24T23:26:01.105 INFO:tasks.ceph.osd.2.smithi115.stderr:*** Caught signal (Aborted) ** 2024-01-24T23:26:01.105 INFO:tasks.ceph.osd.2.smithi115.stderr: in thread 7f42d3c86640 thread_name:tp_osd_tp 2024-01-24T23:26:01.114 INFO:tasks.ceph.osd.2.smithi115.stderr: ceph version 19.0.0-431-g66387355 (663873555d45def955d1cc09730bfece2847b821) squid (dev) 2024-01-24T23:26:01.114 INFO:tasks.ceph.osd.2.smithi115.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f42ee297520] 2024-01-24T23:26:01.114 INFO:tasks.ceph.osd.2.smithi115.stderr: 2: pthread_kill() 2024-01-24T23:26:01.114 INFO:tasks.ceph.osd.2.smithi115.stderr: 3: raise() 2024-01-24T23:26:01.114 INFO:tasks.ceph.osd.2.smithi115.stderr: 4: abort() 2024-01-24T23:26:01.114 INFO:tasks.ceph.osd.2.smithi115.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xa2bbe) [0x7f42ee628bbe] 2024-01-24T23:26:01.114 INFO:tasks.ceph.osd.2.smithi115.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae24c) [0x7f42ee63424c] 2024-01-24T23:26:01.114 INFO:tasks.ceph.osd.2.smithi115.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae2b7) [0x7f42ee6342b7] 2024-01-24T23:26:01.114 INFO:tasks.ceph.osd.2.smithi115.stderr: 8: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae518) [0x7f42ee634518] 2024-01-24T23:26:01.114 INFO:tasks.ceph.osd.2.smithi115.stderr: 9: ceph-osd(+0x410fe9) [0x55b1ca99ffe9] 2024-01-24T23:26:01.114 INFO:tasks.ceph.osd.2.smithi115.stderr: 10: (BlueStore::_generate_read_result_bl(boost::intrusive_ptr&, unsigned long, unsigned long, std::map, std::allocator > >&, std::vector >&, std::map, std::__cxx11::list >, std::less >, std::allocator const, std::__cxx11::list > > > >&, bool, bool*, ceph::buffer::v15_2_0::list&)+0x3d0) [0x55b1caf0ee80] 2024-01-24T23:26:01.115 INFO:tasks.ceph.osd.2.smithi115.stderr: 11: (BlueStore::_do_readv(BlueStore::Collection*, boost::intrusive_ptr&, interval_set const&, ceph::buffer::v15_2_0::list&, unsigned int, unsigned long)+0x7dc) [0x55b1caf1163c] 2024-01-24T23:26:01.115 INFO:tasks.ceph.osd.2.smithi115.stderr: 12: (BlueStore::readv(boost::intrusive_ptr&, ghobject_t const&, interval_set&, ceph::buffer::v15_2_0::list&, unsigned int)+0x361) [0x55b1caf12641] 2024-01-24T23:26:01.115 INFO:tasks.ceph.osd.2.smithi115.stderr: 13: (ReplicatedBackend::build_push_op(ObjectRecoveryInfo const&, ObjectRecoveryProgress const&, ObjectRecoveryProgress*, PushOp*, object_stat_sum_t*, bool)+0x255) [0x55b1cae1e655] 2024-01-24T23:26:01.115 INFO:tasks.ceph.osd.2.smithi115.stderr: 14: (ReplicatedBackend::prep_push(std::shared_ptr, hobject_t const&, pg_shard_t, eversion_t, interval_set&, std::map, std::less, std::allocator > > >&, PushOp*, bool, ObcLockManager&&)+0x5bd) [0x55b1cae202ed] 2024-01-24T23:26:01.115 INFO:tasks.ceph.osd.2.smithi115.stderr: 15: (ReplicatedBackend::prep_push_to_replica(std::shared_ptr, hobject_t const&, pg_shard_t, PushOp*, bool)+0x324) [0x55b1cae20a64] 2024-01-24T23:26:01.115 INFO:tasks.ceph.osd.2.smithi115.stderr: 16: (ReplicatedBackend::start_pushes(hobject_t const&, std::shared_ptr, ReplicatedBackend::RPGHandle*)+0x2ef) [0x55b1cae2161f] 2024-01-24T23:26:01.115 INFO:tasks.ceph.osd.2.smithi115.stderr: 17: (ReplicatedBackend::recover_object(hobject_t const&, eversion_t, std::shared_ptr, std::shared_ptr, PGBackend::RecoveryHandle*)+0x185) [0x55b1cae03715] 2024-01-24T23:26:01.115 INFO:tasks.ceph.osd.2.smithi115.stderr: 18: (PrimaryLogPG::prep_object_replica_pushes(hobject_t const&, eversion_t, PGBackend::RecoveryHandle*, bool*)+0x238) [0x55b1cac146e8] 2024-01-24T23:26:01.115 INFO:tasks.ceph.osd.2.smithi115.stderr: 19: (PrimaryLogPG::recover_replicas(unsigned long, ThreadPool::TPHandle&, bool*)+0x144a) [0x55b1cac160ea] 2024-01-24T23:26:01.115 INFO:tasks.ceph.osd.2.smithi115.stderr: 20: (PrimaryLogPG::start_recovery_ops(unsigned long, ThreadPool::TPHandle&, unsigned long*)+0x25b) [0x55b1cac219eb] 2024-01-24T23:26:01.115 INFO:tasks.ceph.osd.2.smithi115.stderr: 21: (OSD::do_recovery(PG*, unsigned int, unsigned long, int, ThreadPool::TPHandle&)+0x245) [0x55b1caaee4a5] 2024-01-24T23:26:01.115 INFO:tasks.ceph.osd.2.smithi115.stderr: 22: (ceph::osd::scheduler::PGRecovery::run(OSD*, OSDShard*, boost::intrusive_ptr&, ThreadPool::TPHandle&)+0xca) [0x55b1cad16c3a] 2024-01-24T23:26:01.115 INFO:tasks.ceph.osd.2.smithi115.stderr: 23: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xab3) [0x55b1cab0f933] 2024-01-24T23:26:01.115 INFO:tasks.ceph.osd.2.smithi115.stderr: 24: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x293) [0x55b1caffcc33] 2024-01-24T23:26:01.116 INFO:tasks.ceph.osd.2.smithi115.stderr: 25: ceph-osd(+0xa6e194) [0x55b1caffd194] 2024-01-24T23:26:01.116 INFO:tasks.ceph.osd.2.smithi115.stderr: 26: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f42ee2e9b43] 2024-01-24T23:26:01.116 INFO:tasks.ceph.osd.2.smithi115.stderr: 27: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f42ee37ba00] 2024-01-24T23:26:01.116 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-01-24T23:26:01.113+0000 7f42d3c86640 -1 *** Caught signal (Aborted) ** 2024-01-24T23:26:01.116 INFO:tasks.ceph.osd.2.smithi115.stderr: in thread 7f42d3c86640 thread_name:tp_osd_tp 2024-01-24T23:26:01.116 INFO:tasks.ceph.osd.2.smithi115.stderr: 2024-01-24T23:26:01.116 INFO:tasks.ceph.osd.2.smithi115.stderr: ceph version 19.0.0-431-g66387355 (663873555d45def955d1cc09730bfece2847b821) squid (dev) 2024-01-24T23:26:01.116 INFO:tasks.ceph.osd.2.smithi115.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f42ee297520] 2024-01-24T23:26:01.116 INFO:tasks.ceph.osd.2.smithi115.stderr: 2: pthread_kill() 2024-01-24T23:26:01.117 INFO:tasks.ceph.osd.2.smithi115.stderr: 3: raise() 2024-01-24T23:26:01.117 INFO:tasks.ceph.osd.2.smithi115.stderr: 4: abort() 2024-01-24T23:26:01.117 INFO:tasks.ceph.osd.2.smithi115.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xa2bbe) [0x7f42ee628bbe] 2024-01-24T23:26:01.117 INFO:tasks.ceph.osd.2.smithi115.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae24c) [0x7f42ee63424c] 2024-01-24T23:26:01.117 INFO:tasks.ceph.osd.2.smithi115.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae2b7) [0x7f42ee6342b7] 2024-01-24T23:26:01.117 INFO:tasks.ceph.osd.2.smithi115.stderr: 8: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae518) [0x7f42ee634518] 2024-01-24T23:26:01.117 INFO:tasks.ceph.osd.2.smithi115.stderr: 9: ceph-osd(+0x410fe9) [0x55b1ca99ffe9] 2024-01-24T23:26:01.117 INFO:tasks.ceph.osd.2.smithi115.stderr: 10: (BlueStore::_generate_read_result_bl(boost::intrusive_ptr&, unsigned long, unsigned long, std::map, std::allocator > >&, std::vector >&, std::map, std::__cxx11::list >, std::less >, std::allocator const, std::__cxx11::list > > > >&, bool, bool*, ceph::buffer::v15_2_0::list&)+0x3d0) [0x55b1caf0ee80] 2024-01-24T23:26:01.117 INFO:tasks.ceph.osd.2.smithi115.stderr: 11: (BlueStore::_do_readv(BlueStore::Collection*, boost::intrusive_ptr&, interval_set const&, ceph::buffer::v15_2_0::list&, unsigned int, unsigned long)+0x7dc) [0x55b1caf1163c] 2024-01-24T23:26:01.117 INFO:tasks.ceph.osd.2.smithi115.stderr: 12: (BlueStore::readv(boost::intrusive_ptr&, ghobject_t const&, interval_set&, ceph::buffer::v15_2_0::list&, unsigned int)+0x361) [0x55b1caf12641] 2024-01-24T23:26:01.117 INFO:tasks.ceph.osd.2.smithi115.stderr: 13: (ReplicatedBackend::build_push_op(ObjectRecoveryInfo const&, ObjectRecoveryProgress const&, ObjectRecoveryProgress*, PushOp*, object_stat_sum_t*, bool)+0x255) [0x55b1cae1e655] 2024-01-24T23:26:01.117 INFO:tasks.ceph.osd.2.smithi115.stderr: 14: (ReplicatedBackend::prep_push(std::shared_ptr, hobject_t const&, pg_shard_t, eversion_t, interval_set&, std::map, std::less, std::allocator > > >&, PushOp*, bool, ObcLockManager&&)+0x5bd) [0x55b1cae202ed] 2024-01-24T23:26:01.117 INFO:tasks.ceph.osd.2.smithi115.stderr: 15: (ReplicatedBackend::prep_push_to_replica(std::shared_ptr, hobject_t const&, pg_shard_t, PushOp*, bool)+0x324) [0x55b1cae20a64] 2024-01-24T23:26:01.117 INFO:tasks.ceph.osd.2.smithi115.stderr: 16: (ReplicatedBackend::start_pushes(hobject_t const&, std::shared_ptr, ReplicatedBackend::RPGHandle*)+0x2ef) [0x55b1cae2161f] 2024-01-24T23:26:01.117 INFO:tasks.ceph.osd.2.smithi115.stderr: 17: (ReplicatedBackend::recover_object(hobject_t const&, eversion_t, std::shared_ptr, std::shared_ptr, PGBackend::RecoveryHandle*)+0x185) [0x55b1cae03715] 2024-01-24T23:26:01.118 INFO:tasks.ceph.osd.2.smithi115.stderr: 18: (PrimaryLogPG::prep_object_replica_pushes(hobject_t const&, eversion_t, PGBackend::RecoveryHandle*, bool*)+0x238) [0x55b1cac146e8] 2024-01-24T23:26:01.118 INFO:tasks.ceph.osd.2.smithi115.stderr: 19: (PrimaryLogPG::recover_replicas(unsigned long, ThreadPool::TPHandle&, bool*)+0x144a) [0x55b1cac160ea] 2024-01-24T23:26:01.118 INFO:tasks.ceph.osd.2.smithi115.stderr: 20: (PrimaryLogPG::start_recovery_ops(unsigned long, ThreadPool::TPHandle&, unsigned long*)+0x25b) [0x55b1cac219eb] 2024-01-24T23:26:01.118 INFO:tasks.ceph.osd.2.smithi115.stderr: 21: (OSD::do_recovery(PG*, unsigned int, unsigned long, int, ThreadPool::TPHandle&)+0x245) [0x55b1caaee4a5] 2024-01-24T23:26:01.118 INFO:tasks.ceph.osd.2.smithi115.stderr: 22: (ceph::osd::scheduler::PGRecovery::run(OSD*, OSDShard*, boost::intrusive_ptr&, ThreadPool::TPHandle&)+0xca) [0x55b1cad16c3a] 2024-01-24T23:26:01.118 INFO:tasks.ceph.osd.2.smithi115.stderr: 23: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xab3) [0x55b1cab0f933] 2024-01-24T23:26:01.118 INFO:tasks.ceph.osd.2.smithi115.stderr: 24: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x293) [0x55b1caffcc33] 2024-01-24T23:26:01.118 INFO:tasks.ceph.osd.2.smithi115.stderr: 25: ceph-osd(+0xa6e194) [0x55b1caffd194] 2024-01-24T23:26:01.118 INFO:tasks.ceph.osd.2.smithi115.stderr: 26: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f42ee2e9b43] 2024-01-24T23:26:01.118 INFO:tasks.ceph.osd.2.smithi115.stderr: 27: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f42ee37ba00] 2024-01-24T23:26:01.118 INFO:tasks.ceph.osd.2.smithi115.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-01-24T23:26:01.118 INFO:tasks.ceph.osd.2.smithi115.stderr: 2024-01-24T23:26:01.148 INFO:tasks.ceph.osd.2.smithi115.stderr: 0> 2024-01-24T23:26:01.113+0000 7f42d3c86640 -1 *** Caught signal (Aborted) ** 2024-01-24T23:26:01.148 INFO:tasks.ceph.osd.2.smithi115.stderr: in thread 7f42d3c86640 thread_name:tp_osd_tp 2024-01-24T23:26:01.148 INFO:tasks.ceph.osd.2.smithi115.stderr: 2024-01-24T23:26:01.148 INFO:tasks.ceph.osd.2.smithi115.stderr: ceph version 19.0.0-431-g66387355 (663873555d45def955d1cc09730bfece2847b821) squid (dev) 2024-01-24T23:26:01.148 INFO:tasks.ceph.osd.2.smithi115.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f42ee297520] 2024-01-24T23:26:01.148 INFO:tasks.ceph.osd.2.smithi115.stderr: 2: pthread_kill() 2024-01-24T23:26:01.148 INFO:tasks.ceph.osd.2.smithi115.stderr: 3: raise() 2024-01-24T23:26:01.148 INFO:tasks.ceph.osd.2.smithi115.stderr: 4: abort() 2024-01-24T23:26:01.148 INFO:tasks.ceph.osd.2.smithi115.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xa2bbe) [0x7f42ee628bbe] 2024-01-24T23:26:01.148 INFO:tasks.ceph.osd.2.smithi115.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae24c) [0x7f42ee63424c] 2024-01-24T23:26:01.148 INFO:tasks.ceph.osd.2.smithi115.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae2b7) [0x7f42ee6342b7] 2024-01-24T23:26:01.148 INFO:tasks.ceph.osd.2.smithi115.stderr: 8: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae518) [0x7f42ee634518] 2024-01-24T23:26:01.149 INFO:tasks.ceph.osd.2.smithi115.stderr: 9: ceph-osd(+0x410fe9) [0x55b1ca99ffe9] 2024-01-24T23:26:01.149 INFO:tasks.ceph.osd.2.smithi115.stderr: 10: (BlueStore::_generate_read_result_bl(boost::intrusive_ptr&, unsigned long, unsigned long, std::map, std::allocator > >&, std::vector >&, std::map, std::__cxx11::list >, std::less >, std::allocator const, std::__cxx11::list > > > >&, bool, bool*, ceph::buffer::v15_2_0::list&)+0x3d0) [0x55b1caf0ee80] 2024-01-24T23:26:01.149 INFO:tasks.ceph.osd.2.smithi115.stderr: 11: (BlueStore::_do_readv(BlueStore::Collection*, boost::intrusive_ptr&, interval_set const&, ceph::buffer::v15_2_0::list&, unsigned int, unsigned long)+0x7dc) [0x55b1caf1163c] 2024-01-24T23:26:01.149 INFO:tasks.ceph.osd.2.smithi115.stderr: 12: (BlueStore::readv(boost::intrusive_ptr&, ghobject_t const&, interval_set&, ceph::buffer::v15_2_0::list&, unsigned int)+0x361) [0x55b1caf12641] 2024-01-24T23:26:01.149 INFO:tasks.ceph.osd.2.smithi115.stderr: 13: (ReplicatedBackend::build_push_op(ObjectRecoveryInfo const&, ObjectRecoveryProgress const&, ObjectRecoveryProgress*, PushOp*, object_stat_sum_t*, bool)+0x255) [0x55b1cae1e655] 2024-01-24T23:26:01.149 INFO:tasks.ceph.osd.2.smithi115.stderr: 14: (ReplicatedBackend::prep_push(std::shared_ptr, hobject_t const&, pg_shard_t, eversion_t, interval_set&, std::map, std::less, std::allocator > > >&, PushOp*, bool, ObcLockManager&&)+0x5bd) [0x55b1cae202ed] 2024-01-24T23:26:01.149 INFO:tasks.ceph.osd.2.smithi115.stderr: 15: (ReplicatedBackend::prep_push_to_replica(std::shared_ptr, hobject_t const&, pg_shard_t, PushOp*, bool)+0x324) [0x55b1cae20a64] 2024-01-24T23:26:01.149 INFO:tasks.ceph.osd.2.smithi115.stderr: 16: (ReplicatedBackend::start_pushes(hobject_t const&, std::shared_ptr, ReplicatedBackend::RPGHandle*)+0x2ef) [0x55b1cae2161f] 2024-01-24T23:26:01.149 INFO:tasks.ceph.osd.2.smithi115.stderr: 17: (ReplicatedBackend::recover_object(hobject_t const&, eversion_t, std::shared_ptr, std::shared_ptr, PGBackend::RecoveryHandle*)+0x185) [0x55b1cae03715] 2024-01-24T23:26:01.149 INFO:tasks.ceph.osd.2.smithi115.stderr: 18: (PrimaryLogPG::prep_object_replica_pushes(hobject_t const&, eversion_t, PGBackend::RecoveryHandle*, bool*)+0x238) [0x55b1cac146e8] 2024-01-24T23:26:01.149 INFO:tasks.ceph.osd.2.smithi115.stderr: 19: (PrimaryLogPG::recover_replicas(unsigned long, ThreadPool::TPHandle&, bool*)+0x144a) [0x55b1cac160ea] 2024-01-24T23:26:01.149 INFO:tasks.ceph.osd.2.smithi115.stderr: 20: (PrimaryLogPG::start_recovery_ops(unsigned long, ThreadPool::TPHandle&, unsigned long*)+0x25b) [0x55b1cac219eb] 2024-01-24T23:26:01.149 INFO:tasks.ceph.osd.2.smithi115.stderr: 21: (OSD::do_recovery(PG*, unsigned int, unsigned long, int, ThreadPool::TPHandle&)+0x245) [0x55b1caaee4a5] 2024-01-24T23:26:01.149 INFO:tasks.ceph.osd.2.smithi115.stderr: 22: (ceph::osd::scheduler::PGRecovery::run(OSD*, OSDShard*, boost::intrusive_ptr&, ThreadPool::TPHandle&)+0xca) [0x55b1cad16c3a] 2024-01-24T23:26:01.150 INFO:tasks.ceph.osd.2.smithi115.stderr: 23: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xab3) [0x55b1cab0f933] 2024-01-24T23:26:01.150 INFO:tasks.ceph.osd.2.smithi115.stderr: 24: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x293) [0x55b1caffcc33] 2024-01-24T23:26:01.150 INFO:tasks.ceph.osd.2.smithi115.stderr: 25: ceph-osd(+0xa6e194) [0x55b1caffd194] 2024-01-24T23:26:01.150 INFO:tasks.ceph.osd.2.smithi115.stderr: 26: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f42ee2e9b43] 2024-01-24T23:26:01.150 INFO:tasks.ceph.osd.2.smithi115.stderr: 27: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f42ee37ba00] 2024-01-24T23:26:01.150 INFO:tasks.ceph.osd.2.smithi115.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-01-24T23:26:01.150 INFO:tasks.ceph.osd.2.smithi115.stderr: 2024-01-24T23:26:01.165 INFO:tasks.ceph.osd.2.smithi115.stderr: -13> 2024-01-24T23:26:01.113+0000 7f42d3c86640 -1 *** Caught signal (Aborted) ** 2024-01-24T23:26:01.165 INFO:tasks.ceph.osd.2.smithi115.stderr: in thread 7f42d3c86640 thread_name:tp_osd_tp 2024-01-24T23:26:01.165 INFO:tasks.ceph.osd.2.smithi115.stderr: 2024-01-24T23:26:01.165 INFO:tasks.ceph.osd.2.smithi115.stderr: ceph version 19.0.0-431-g66387355 (663873555d45def955d1cc09730bfece2847b821) squid (dev) 2024-01-24T23:26:01.165 INFO:tasks.ceph.osd.2.smithi115.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f42ee297520] 2024-01-24T23:26:01.166 INFO:tasks.ceph.osd.2.smithi115.stderr: 2: pthread_kill() 2024-01-24T23:26:01.166 INFO:tasks.ceph.osd.2.smithi115.stderr: 3: raise() 2024-01-24T23:26:01.166 INFO:tasks.ceph.osd.2.smithi115.stderr: 4: abort() 2024-01-24T23:26:01.166 INFO:tasks.ceph.osd.2.smithi115.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xa2bbe) [0x7f42ee628bbe] 2024-01-24T23:26:01.166 INFO:tasks.ceph.osd.2.smithi115.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae24c) [0x7f42ee63424c] 2024-01-24T23:26:01.166 INFO:tasks.ceph.osd.2.smithi115.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae2b7) [0x7f42ee6342b7] 2024-01-24T23:26:01.166 INFO:tasks.ceph.osd.2.smithi115.stderr: 8: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae518) [0x7f42ee634518] 2024-01-24T23:26:01.166 INFO:tasks.ceph.osd.2.smithi115.stderr: 9: ceph-osd(+0x410fe9) [0x55b1ca99ffe9] 2024-01-24T23:26:01.166 INFO:tasks.ceph.osd.2.smithi115.stderr: 10: (BlueStore::_generate_read_result_bl(boost::intrusive_ptr&, unsigned long, unsigned long, std::map, std::allocator > >&, std::vector >&, std::map, std::__cxx11::list >, std::less >, std::allocator const, std::__cxx11::list > > > >&, bool, bool*, ceph::buffer::v15_2_0::list&)+0x3d0) [0x55b1caf0ee80] 2024-01-24T23:26:01.166 INFO:tasks.ceph.osd.2.smithi115.stderr: 11: (BlueStore::_do_readv(BlueStore::Collection*, boost::intrusive_ptr&, interval_set const&, ceph::buffer::v15_2_0::list&, unsigned int, unsigned long)+0x7dc) [0x55b1caf1163c] 2024-01-24T23:26:01.166 INFO:tasks.ceph.osd.2.smithi115.stderr: 12: (BlueStore::readv(boost::intrusive_ptr&, ghobject_t const&, interval_set&, ceph::buffer::v15_2_0::list&, unsigned int)+0x361) [0x55b1caf12641] 2024-01-24T23:26:01.166 INFO:tasks.ceph.osd.2.smithi115.stderr: 13: (ReplicatedBackend::build_push_op(ObjectRecoveryInfo const&, ObjectRecoveryProgress const&, ObjectRecoveryProgress*, PushOp*, object_stat_sum_t*, bool)+0x255) [0x55b1cae1e655] 2024-01-24T23:26:01.166 INFO:tasks.ceph.osd.2.smithi115.stderr: 14: (ReplicatedBackend::prep_push(std::shared_ptr, hobject_t const&, pg_shard_t, eversion_t, interval_set&, std::map, std::less, std::allocator > > >&, PushOp*, bool, ObcLockManager&&)+0x5bd) [0x55b1cae202ed] 2024-01-24T23:26:01.167 INFO:tasks.ceph.osd.2.smithi115.stderr: 15: (ReplicatedBackend::prep_push_to_replica(std::shared_ptr, hobject_t const&, pg_shard_t, PushOp*, bool)+0x324) [0x55b1cae20a64] 2024-01-24T23:26:01.167 INFO:tasks.ceph.osd.2.smithi115.stderr: 16: (ReplicatedBackend::start_pushes(hobject_t const&, std::shared_ptr, ReplicatedBackend::RPGHandle*)+0x2ef) [0x55b1cae2161f] 2024-01-24T23:26:01.167 INFO:tasks.ceph.osd.2.smithi115.stderr: 17: (ReplicatedBackend::recover_object(hobject_t const&, eversion_t, std::shared_ptr, std::shared_ptr, PGBackend::RecoveryHandle*)+0x185) [0x55b1cae03715] 2024-01-24T23:26:01.167 INFO:tasks.ceph.osd.2.smithi115.stderr: 18: (PrimaryLogPG::prep_object_replica_pushes(hobject_t const&, eversion_t, PGBackend::RecoveryHandle*, bool*)+0x238) [0x55b1cac146e8] 2024-01-24T23:26:01.167 INFO:tasks.ceph.osd.2.smithi115.stderr: 19: (PrimaryLogPG::recover_replicas(unsigned long, ThreadPool::TPHandle&, bool*)+0x144a) [0x55b1cac160ea] 2024-01-24T23:26:01.167 INFO:tasks.ceph.osd.2.smithi115.stderr: 20: (PrimaryLogPG::start_recovery_ops(unsigned long, ThreadPool::TPHandle&, unsigned long*)+0x25b) [0x55b1cac219eb] 2024-01-24T23:26:01.167 INFO:tasks.ceph.osd.2.smithi115.stderr: 21: (OSD::do_recovery(PG*, unsigned int, unsigned long, int, ThreadPool::TPHandle&)+0x245) [0x55b1caaee4a5] 2024-01-24T23:26:01.167 INFO:tasks.ceph.osd.2.smithi115.stderr: 22: (ceph::osd::scheduler::PGRecovery::run(OSD*, OSDShard*, boost::intrusive_ptr&, ThreadPool::TPHandle&)+0xca) [0x55b1cad16c3a] 2024-01-24T23:26:01.167 INFO:tasks.ceph.osd.2.smithi115.stderr: 23: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xab3) [0x55b1cab0f933] 2024-01-24T23:26:01.167 INFO:tasks.ceph.osd.2.smithi115.stderr: 24: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x293) [0x55b1caffcc33] 2024-01-24T23:26:01.167 INFO:tasks.ceph.osd.2.smithi115.stderr: 25: ceph-osd(+0xa6e194) [0x55b1caffd194] 2024-01-24T23:26:01.167 INFO:tasks.ceph.osd.2.smithi115.stderr: 26: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f42ee2e9b43] 2024-01-24T23:26:01.167 INFO:tasks.ceph.osd.2.smithi115.stderr: 27: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f42ee37ba00] 2024-01-24T23:26:01.167 INFO:tasks.ceph.osd.2.smithi115.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-01-24T23:26:01.167 INFO:tasks.ceph.osd.2.smithi115.stderr: 2024-01-24T23:26:01.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:01.180+0000 7fe6d669d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}] 2024-01-24T23:26:01.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:01.180+0000 7fe6d669d640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}] 2024-01-24T23:26:01.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:01.180+0000 7fe6d669d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:01.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:01.180+0000 7fe6d669d640 1 -- 172.21.15.132:0/998838130 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]} v 0) v1 -- 0x7fe6d0073840 con 0x7fe6d009bbf0 2024-01-24T23:26:01.806 INFO:tasks.ceph.osd.2.smithi115.stderr:daemon-helper: command crashed with signal 6 2024-01-24T23:26:02.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.076+0000 7fe6ccff9640 1 -- 172.21.15.132:0/998838130 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}]=0 v146) v1 ==== 128+0+0 (unknown 3152379166 0 0) 0x7fe6c0014a80 con 0x7fe6d009bbf0 2024-01-24T23:26:02.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.076+0000 7fe6ccff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}] 2024-01-24T23:26:02.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.076+0000 7fe6ccff9640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:26:02.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.080+0000 7fe6d669d640 1 -- 172.21.15.132:0/998838130 >> v1:172.21.15.132:6812/1835401922 conn(0x7fe6bc036f60 legacy=0x7fe6bc039420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:02.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.080+0000 7fe6d669d640 10 monclient: shutdown 2024-01-24T23:26:02.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.080+0000 7fe6d669d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:02.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.080+0000 7fe6d669d640 1 -- 172.21.15.132:0/998838130 >> v1:172.21.15.132:6790/0 conn(0x7fe6d009bbf0 legacy=0x7fe6d01d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:02.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.080+0000 7fe6d669d640 1 -- 172.21.15.132:0/998838130 shutdown_connections 2024-01-24T23:26:02.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.080+0000 7fe6d669d640 1 -- 172.21.15.132:0/998838130 >> 172.21.15.132:0/998838130 conn(0x7fe6d00a1bc0 msgr2=0x7fe6d00a3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:02.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.080+0000 7fe6d669d640 1 -- 172.21.15.132:0/998838130 shutdown_connections 2024-01-24T23:26:02.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.080+0000 7fe6d669d640 1 -- 172.21.15.132:0/998838130 wait complete. 2024-01-24T23:26:02.105 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set reweight cold osd.0 12.5 12.4 2024-01-24T23:26:02.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.232+0000 7f4527ee6640 10 monclient: get_monmap_and_config 2024-01-24T23:26:02.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.232+0000 7f4527ee6640 10 monclient: build_initial_monmap 2024-01-24T23:26:02.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.232+0000 7f4527ee6640 10 monclient: monmap: 2024-01-24T23:26:02.234 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:02.234 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:02.234 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:02.235418+0000 2024-01-24T23:26:02.234 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:02.235418+0000 2024-01-24T23:26:02.234 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:02.234 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:02.234 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:02.234 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:02.234 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:02.235 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:02.235 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:02.235 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:02.235 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:02.235 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:02.235 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:02.235 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:02.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.232+0000 7f4527ee6640 1 Processor -- start 2024-01-24T23:26:02.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.232+0000 7f4527ee6640 1 -- start start 2024-01-24T23:26:02.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.232+0000 7f4527ee6640 10 monclient: init 2024-01-24T23:26:02.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.232+0000 7f4527ee6640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:02.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.232+0000 7f4527ee6640 10 monclient: _add_conns ranks=[2,4,7,6,1,5,8,0,3] 2024-01-24T23:26:02.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.232+0000 7f4527ee6640 10 monclient(hunting): picked mon.noname-b con 0x7f4520135e10 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:02.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.232+0000 7f4527ee6640 10 monclient(hunting): picked mon.noname-c con 0x7f45201361f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:02.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 10 monclient(hunting): picked mon.noname-i con 0x7f45201365f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:02.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4520064700 con 0x7f4520135e10 2024-01-24T23:26:02.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452009c3e0 con 0x7f45201361f0 2024-01-24T23:26:02.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45200a4f90 con 0x7f45201365f0 2024-01-24T23:26:02.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:02.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 10 monclient(hunting): authenticate will time out at 1249.560055s 2024-01-24T23:26:02.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4526483640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f45201365f0 legacy=0x7f452013f600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:02.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4525c82640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f4520135e10 legacy=0x7f452009bcd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:02.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4525481640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f45201361f0 0x7f452013bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:37786/0 (socket says 172.21.15.132:37786) 2024-01-24T23:26:02.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4525481640 1 -- 172.21.15.132:0/862636674 learned_addr learned my addr 172.21.15.132:0/862636674 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:02.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 1 -- 172.21.15.132:0/862636674 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 916371655 0 0) 0x7f452009c3e0 con 0x7f45201361f0 2024-01-24T23:26:02.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 10 monclient(hunting): _negotiate 2024-01-24T23:26:02.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:02.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:02.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 10 monclient(hunting): my global_id is 94405 2024-01-24T23:26:02.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 1 -- 172.21.15.132:0/862636674 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f450c003530 con 0x7f45201361f0 2024-01-24T23:26:02.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 1 -- 172.21.15.132:0/862636674 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2854919880 0 0) 0x7f450c003530 con 0x7f45201361f0 2024-01-24T23:26:02.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 1 -- 172.21.15.132:0/862636674 >> v1:172.21.15.132:6792/0 conn(0x7f45201365f0 legacy=0x7f452013f600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:02.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 1 -- 172.21.15.132:0/862636674 >> v1:172.21.15.115:6790/0 conn(0x7f4520135e10 legacy=0x7f452009bcd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:02.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 10 monclient: _finish_hunting 0 2024-01-24T23:26:02.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 1 monclient: found mon.noname-c 2024-01-24T23:26:02.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:02.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:26:02.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 1 -- 172.21.15.132:0/862636674 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452009b010 con 0x7f45201361f0 2024-01-24T23:26:02.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 10 monclient: _finish_auth 0 2024-01-24T23:26:02.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 10 monclient: _check_auth_tickets 2024-01-24T23:26:02.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:02.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 1 -- 172.21.15.132:0/862636674 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f45140017c0 con 0x7f45201361f0 2024-01-24T23:26:02.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:02.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:26:02.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 10 monclient: dump: 2024-01-24T23:26:02.244 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:02.245 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:02.245 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:02.245 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:02.245 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:02.245 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:02.245 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:02.245 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:02.245 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:02.245 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:02.245 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:02.245 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:02.245 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:02.245 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:02.245 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:02.245 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:02.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 5 monclient: authenticate success, global_id 94405 2024-01-24T23:26:02.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:02.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 1 -- 172.21.15.132:0/862636674 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4514001ae0 con 0x7f45201361f0 2024-01-24T23:26:02.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:02.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 1 -- 172.21.15.132:0/862636674 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4514004bf0 con 0x7f45201361f0 2024-01-24T23:26:02.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:02.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 4 set_mon_vals no callback set 2024-01-24T23:26:02.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:02.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:26:02.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4524c80640 10 monclient: dump: 2024-01-24T23:26:02.246 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:02.246 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:02.246 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:02.247 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:02.247 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:02.247 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:02.247 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:02.247 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:02.247 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:02.247 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:02.247 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:02.247 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:02.247 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:02.247 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:02.247 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:02.247 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:02.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 10 monclient: shutdown 2024-01-24T23:26:02.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:02.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 1 -- 172.21.15.132:0/862636674 >> v1:172.21.15.115:6791/0 conn(0x7f45201361f0 legacy=0x7f452013bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:02.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 1 -- 172.21.15.132:0/862636674 shutdown_connections 2024-01-24T23:26:02.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 1 -- 172.21.15.132:0/862636674 >> 172.21.15.132:0/862636674 conn(0x7f45200a6790 msgr2=0x7f4520071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:02.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 1 -- 172.21.15.132:0/862636674 shutdown_connections 2024-01-24T23:26:02.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 1 -- 172.21.15.132:0/862636674 wait complete. 2024-01-24T23:26:02.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 10 monclient: build_initial_monmap 2024-01-24T23:26:02.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 10 monclient: monmap: 2024-01-24T23:26:02.248 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:02.248 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:02.248 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:02.248 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:02.248 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:02.248 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:02.248 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:02.249 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:02.249 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:02.249 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:02.249 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:02.249 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:02.249 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:02.249 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:02.249 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:02.249 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:02.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 1 Processor -- start 2024-01-24T23:26:02.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 1 -- start start 2024-01-24T23:26:02.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 10 monclient: init 2024-01-24T23:26:02.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:02.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 10 monclient: _add_conns ranks=[7,3,2,0,1,4,6,8,5] 2024-01-24T23:26:02.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 10 monclient(hunting): picked mon.noname-i con 0x7f4520135e10 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:02.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 10 monclient(hunting): picked mon.noname-g con 0x7f45201361f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:02.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 10 monclient(hunting): picked mon.noname-b con 0x7f45201365f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:02.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45201cbfc0 con 0x7f45201365f0 2024-01-24T23:26:02.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452009c6b0 con 0x7f45201361f0 2024-01-24T23:26:02.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4525c82640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f4520135e10 legacy=0x7f4520134e40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:02.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4520122800 con 0x7f4520135e10 2024-01-24T23:26:02.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:02.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4527ee6640 10 monclient(hunting): authenticate will time out at 1249.563797s 2024-01-24T23:26:02.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4525481640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f45201361f0 0x7f45201d0380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:46874/0 (socket says 172.21.15.132:46874) 2024-01-24T23:26:02.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4525481640 1 -- 172.21.15.132:0/417844369 learned_addr learned my addr 172.21.15.132:0/417844369 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:02.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.236+0000 7f4526483640 1 -- 172.21.15.132:0/417844369 >> v1:172.21.15.115:6790/0 conn(0x7f45201365f0 legacy=0x7f45201d3ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:02.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 1 -- 172.21.15.132:0/417844369 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 304908126 0 0) 0x7f452009c6b0 con 0x7f45201361f0 2024-01-24T23:26:02.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 10 monclient(hunting): _negotiate 2024-01-24T23:26:02.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:02.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:02.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 10 monclient(hunting): my global_id is 6168 2024-01-24T23:26:02.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 1 -- 172.21.15.132:0/417844369 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4504003530 con 0x7f45201361f0 2024-01-24T23:26:02.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 1 -- 172.21.15.132:0/417844369 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3642576959 0 0) 0x7f4504003530 con 0x7f45201361f0 2024-01-24T23:26:02.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 1 -- 172.21.15.132:0/417844369 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f452009c6b0 con 0x7f45201361f0 2024-01-24T23:26:02.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 1 -- 172.21.15.132:0/417844369 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4514001780 con 0x7f45201361f0 2024-01-24T23:26:02.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:02.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:02.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 10 monclient(hunting): dump: 2024-01-24T23:26:02.251 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:02.251 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:02.251 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:02.251 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:02.252 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:02.252 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:02.252 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:02.252 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:02.252 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:02.252 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:02.252 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:02.252 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:02.252 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:02.252 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:02.252 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:02.252 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:02.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:02.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 1 -- 172.21.15.132:0/417844369 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2983053891 0 0) 0x7f452009c6b0 con 0x7f45201361f0 2024-01-24T23:26:02.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 1 -- 172.21.15.132:0/417844369 >> v1:172.21.15.132:6792/0 conn(0x7f4520135e10 legacy=0x7f4520134e40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:02.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 1 -- 172.21.15.132:0/417844369 >> v1:172.21.15.115:6790/0 conn(0x7f45201365f0 legacy=0x7f45201d3ab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:02.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 10 monclient: _finish_hunting 0 2024-01-24T23:26:02.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 1 monclient: found mon.g 2024-01-24T23:26:02.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:02.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:02.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 1 -- 172.21.15.132:0/417844369 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45201bf3d0 con 0x7f45201361f0 2024-01-24T23:26:02.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 10 monclient: _finish_auth 0 2024-01-24T23:26:02.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 10 monclient: _check_auth_tickets 2024-01-24T23:26:02.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:02.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f4527ee6640 5 monclient: authenticate success, global_id 6168 2024-01-24T23:26:02.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f4527ee6640 10 monclient: _renew_subs 2024-01-24T23:26:02.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f4527ee6640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:02.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f4527ee6640 1 -- 172.21.15.132:0/417844369 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4520121f60 con 0x7f45201361f0 2024-01-24T23:26:02.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f4527ee6640 10 monclient: _renew_subs 2024-01-24T23:26:02.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f4527ee6640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:02.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f4527ee6640 1 -- 172.21.15.132:0/417844369 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4520135550 con 0x7f45201361f0 2024-01-24T23:26:02.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 1 -- 172.21.15.132:0/417844369 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4514001c50 con 0x7f45201361f0 2024-01-24T23:26:02.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:02.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f4513fff640 4 set_mon_vals no callback set 2024-01-24T23:26:02.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 1 -- 172.21.15.132:0/417844369 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f45140054b0 con 0x7f45201361f0 2024-01-24T23:26:02.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:02.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:02.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 10 monclient: dump: 2024-01-24T23:26:02.254 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:02.254 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:02.254 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:02.254 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:02.254 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:02.254 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:02.255 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:02.255 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:02.255 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:02.255 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:02.255 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:02.255 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:02.255 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:02.255 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:02.255 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:02.255 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:02.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f4527ee6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:02.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f4527ee6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:02.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f4527ee6640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:02.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f4527ee6640 1 -- 172.21.15.132:0/417844369 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4508005090 con 0x7f45201361f0 2024-01-24T23:26:02.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 1 -- 172.21.15.132:0/417844369 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f451400ff30 con 0x7f45201361f0 2024-01-24T23:26:02.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f45127fc640 1 -- 172.21.15.132:0/417844369 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(146..146 src has 1..146) v4 ==== 8349+0+0 (unknown 2138194451 0 0) 0x7f4514044cf0 con 0x7f45201361f0 2024-01-24T23:26:02.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.240+0000 7f4525c82640 10 monclient: get_auth_request con 0x7f4504036f40 auth_method 0 2024-01-24T23:26:02.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.244+0000 7f45127fc640 1 -- 172.21.15.132:0/417844369 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f4514014c00 con 0x7f45201361f0 2024-01-24T23:26:02.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.244+0000 7f45127fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:02.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.244+0000 7f45127fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:02.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.408+0000 7f4527ee6640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}] 2024-01-24T23:26:02.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.408+0000 7f4527ee6640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}] 2024-01-24T23:26:02.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.408+0000 7f4527ee6640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:02.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:02.408+0000 7f4527ee6640 1 -- 172.21.15.132:0/417844369 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]} v 0) v1 -- 0x7f4508005340 con 0x7f45201361f0 2024-01-24T23:26:03.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:03.112+0000 7f45127fc640 1 -- 172.21.15.132:0/417844369 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}]=0 v147) v1 ==== 134+0+0 (unknown 350275470 0 0) 0x7f45140132a0 con 0x7f45201361f0 2024-01-24T23:26:03.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:03.112+0000 7f45127fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}] 2024-01-24T23:26:03.112 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:03.112+0000 7f45127fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:26:03.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:03.216+0000 7f4527ee6640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}] 2024-01-24T23:26:03.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:03.216+0000 7f4527ee6640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}] 2024-01-24T23:26:03.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:03.216+0000 7f4527ee6640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:03.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:03.216+0000 7f4527ee6640 1 -- 172.21.15.132:0/417844369 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]} v 0) v1 -- 0x7f4508005a80 con 0x7f45201361f0 2024-01-24T23:26:04.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.120+0000 7f45127fc640 1 -- 172.21.15.132:0/417844369 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}]=0 v148) v1 ==== 134+0+0 (unknown 3360596190 0 0) 0x7f4514014a20 con 0x7f45201361f0 2024-01-24T23:26:04.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.120+0000 7f45127fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}] 2024-01-24T23:26:04.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.120+0000 7f45127fc640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:26:04.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.124+0000 7f4527ee6640 1 -- 172.21.15.132:0/417844369 >> v1:172.21.15.132:6812/1835401922 conn(0x7f4504036f40 legacy=0x7f4504039400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:04.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.124+0000 7f4527ee6640 10 monclient: shutdown 2024-01-24T23:26:04.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.124+0000 7f4527ee6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:04.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.124+0000 7f4527ee6640 1 -- 172.21.15.132:0/417844369 >> v1:172.21.15.132:6790/0 conn(0x7f45201361f0 legacy=0x7f45201d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:04.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.124+0000 7f4527ee6640 1 -- 172.21.15.132:0/417844369 shutdown_connections 2024-01-24T23:26:04.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.124+0000 7f4527ee6640 1 -- 172.21.15.132:0/417844369 >> 172.21.15.132:0/417844369 conn(0x7f45200a6790 msgr2=0x7f452006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:04.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.124+0000 7f4527ee6640 1 -- 172.21.15.132:0/417844369 shutdown_connections 2024-01-24T23:26:04.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.124+0000 7f4527ee6640 1 -- 172.21.15.132:0/417844369 wait complete. 2024-01-24T23:26:04.145 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush set-device-class fish osd.0 2024-01-24T23:26:04.222 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-01-24T23:26:04.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f027d641640 10 monclient: get_monmap_and_config 2024-01-24T23:26:04.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f027d641640 10 monclient: build_initial_monmap 2024-01-24T23:26:04.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f027d641640 10 monclient: monmap: 2024-01-24T23:26:04.264 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:04.264 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:04.264 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:04.265141+0000 2024-01-24T23:26:04.264 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:04.265141+0000 2024-01-24T23:26:04.264 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:04.264 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:04.264 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:04.264 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:04.264 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:04.265 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:04.265 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:04.265 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:04.265 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:04.265 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:04.265 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:04.265 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:04.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f027d641640 1 Processor -- start 2024-01-24T23:26:04.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f027d641640 1 -- start start 2024-01-24T23:26:04.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f027d641640 10 monclient: init 2024-01-24T23:26:04.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f027d641640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:04.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f027d641640 10 monclient: _add_conns ranks=[6,7,0,1,3,8,2,4,5] 2024-01-24T23:26:04.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f027d641640 10 monclient(hunting): picked mon.noname-d con 0x7f027809ff20 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:04.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f027d641640 10 monclient(hunting): picked mon.noname-i con 0x7f027809f9c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:04.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f027d641640 10 monclient(hunting): picked mon.noname-a con 0x7f027813b2c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:04.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f027d641640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0278064700 con 0x7f027813b2c0 2024-01-24T23:26:04.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f027d641640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f027809e760 con 0x7f027809ff20 2024-01-24T23:26:04.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f027d641640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02780a1160 con 0x7f027809f9c0 2024-01-24T23:26:04.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f027d641640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:04.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f027d641640 10 monclient(hunting): authenticate will time out at 1251.589754s 2024-01-24T23:26:04.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f026ffff640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f027809f9c0 legacy=0x7f0278139ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:04.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f0276ffd640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f027809ff20 0x7f02780a0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:59368/0 (socket says 172.21.15.132:59368) 2024-01-24T23:26:04.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f0276ffd640 1 -- 172.21.15.132:0/81535967 learned_addr learned my addr 172.21.15.132:0/81535967 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:04.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02777fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f027813b2c0 0x7f027813d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:44990/0 (socket says 172.21.15.132:44990) 2024-01-24T23:26:04.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 1 -- 172.21.15.132:0/81535967 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1089791734 0 0) 0x7f027809e760 con 0x7f027809ff20 2024-01-24T23:26:04.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 10 monclient(hunting): _negotiate 2024-01-24T23:26:04.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:04.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:04.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 10 monclient(hunting): my global_id is 6297 2024-01-24T23:26:04.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 1 -- 172.21.15.132:0/81535967 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0264003530 con 0x7f027809ff20 2024-01-24T23:26:04.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 1 -- 172.21.15.132:0/81535967 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2305081344 0 0) 0x7f0278064700 con 0x7f027813b2c0 2024-01-24T23:26:04.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 10 monclient(hunting): _negotiate 2024-01-24T23:26:04.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:04.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:04.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 10 monclient(hunting): my global_id is 94473 2024-01-24T23:26:04.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 1 -- 172.21.15.132:0/81535967 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f027809e760 con 0x7f027813b2c0 2024-01-24T23:26:04.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 1 -- 172.21.15.132:0/81535967 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 736556403 0 0) 0x7f027809e760 con 0x7f027813b2c0 2024-01-24T23:26:04.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 1 -- 172.21.15.132:0/81535967 >> v1:172.21.15.132:6792/0 conn(0x7f027809f9c0 legacy=0x7f0278139ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:04.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 1 -- 172.21.15.132:0/81535967 >> v1:172.21.15.115:6792/0 conn(0x7f027809ff20 legacy=0x7f02780a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:04.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 10 monclient: _finish_hunting 0 2024-01-24T23:26:04.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 1 monclient: found mon.noname-a 2024-01-24T23:26:04.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:04.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:26:04.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 1 -- 172.21.15.132:0/81535967 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02780a0a10 con 0x7f027813b2c0 2024-01-24T23:26:04.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 10 monclient: _finish_auth 0 2024-01-24T23:26:04.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 10 monclient: _check_auth_tickets 2024-01-24T23:26:04.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:04.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 1 -- 172.21.15.132:0/81535967 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0270001aa0 con 0x7f027813b2c0 2024-01-24T23:26:04.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f027d641640 5 monclient: authenticate success, global_id 94473 2024-01-24T23:26:04.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f027d641640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:04.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:04.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:26:04.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 10 monclient: dump: 2024-01-24T23:26:04.275 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:04.275 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:04.275 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:04.275 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:04.276 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:04.277 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:04.277 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:04.277 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:04.277 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:04.277 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:04.278 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:04.278 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:04.278 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:04.278 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:04.278 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:04.278 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:04.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f027d641640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:04.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 1 -- 172.21.15.132:0/81535967 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0270001cc0 con 0x7f027813b2c0 2024-01-24T23:26:04.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:04.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 1 -- 172.21.15.132:0/81535967 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0270004b70 con 0x7f027813b2c0 2024-01-24T23:26:04.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f027d641640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:04.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f027d641640 4 set_mon_vals no callback set 2024-01-24T23:26:04.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:04.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:26:04.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.264+0000 7f02767fc640 10 monclient: dump: 2024-01-24T23:26:04.279 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:04.279 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:04.279 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:04.279 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:04.279 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:04.279 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:04.279 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:04.279 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:04.279 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:04.279 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:04.279 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:04.279 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:04.279 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:04.279 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:04.280 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:04.280 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:04.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 10 monclient: shutdown 2024-01-24T23:26:04.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:04.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 1 -- 172.21.15.132:0/81535967 >> v1:172.21.15.115:6789/0 conn(0x7f027813b2c0 legacy=0x7f027813d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:04.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 1 -- 172.21.15.132:0/81535967 shutdown_connections 2024-01-24T23:26:04.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 1 -- 172.21.15.132:0/81535967 >> 172.21.15.132:0/81535967 conn(0x7f02780a5a10 msgr2=0x7f02780a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:04.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 1 -- 172.21.15.132:0/81535967 shutdown_connections 2024-01-24T23:26:04.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 1 -- 172.21.15.132:0/81535967 wait complete. 2024-01-24T23:26:04.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 10 monclient: build_initial_monmap 2024-01-24T23:26:04.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 10 monclient: monmap: 2024-01-24T23:26:04.281 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:04.281 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:04.281 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:04.281 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:04.281 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:04.281 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:04.281 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:04.281 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:04.281 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:04.281 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:04.281 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:04.281 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:04.281 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:04.281 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:04.281 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:04.282 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:04.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 1 Processor -- start 2024-01-24T23:26:04.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 1 -- start start 2024-01-24T23:26:04.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 10 monclient: init 2024-01-24T23:26:04.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:04.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 10 monclient: _add_conns ranks=[6,7,3,4,5,8,2,1,0] 2024-01-24T23:26:04.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 10 monclient(hunting): picked mon.noname-d con 0x7f027809f9c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:04.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 10 monclient(hunting): picked mon.noname-i con 0x7f027809ff20 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:04.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 10 monclient(hunting): picked mon.noname-g con 0x7f027813b2c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:04.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0278071680 con 0x7f027813b2c0 2024-01-24T23:26:04.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0278125f10 con 0x7f027809f9c0 2024-01-24T23:26:04.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0278071960 con 0x7f027809ff20 2024-01-24T23:26:04.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:04.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 10 monclient(hunting): authenticate will time out at 1251.593770s 2024-01-24T23:26:04.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026ffff640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f027809ff20 legacy=0x7f02781d0710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:04.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f02777fe640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f027813b2c0 0x7f02781d3e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:48474/0 (socket says 172.21.15.132:48474) 2024-01-24T23:26:04.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f02777fe640 1 -- 172.21.15.132:0/127700922 learned_addr learned my addr 172.21.15.132:0/127700922 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:04.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 1 -- 172.21.15.132:0/127700922 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094862894 0 0) 0x7f0278071680 con 0x7f027813b2c0 2024-01-24T23:26:04.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:26:04.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:04.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:04.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 10 monclient(hunting): my global_id is 6177 2024-01-24T23:26:04.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 1 -- 172.21.15.132:0/127700922 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0258003530 con 0x7f027813b2c0 2024-01-24T23:26:04.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 1 -- 172.21.15.132:0/127700922 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2941273725 0 0) 0x7f0258003530 con 0x7f027813b2c0 2024-01-24T23:26:04.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 1 -- 172.21.15.132:0/127700922 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0278071680 con 0x7f027813b2c0 2024-01-24T23:26:04.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 1 -- 172.21.15.132:0/127700922 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513924463 0 0) 0x7f0278125f10 con 0x7f027809f9c0 2024-01-24T23:26:04.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:26:04.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:04.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:04.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 10 monclient(hunting): my global_id is 6306 2024-01-24T23:26:04.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 1 -- 172.21.15.132:0/127700922 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0258003530 con 0x7f027809f9c0 2024-01-24T23:26:04.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 1 -- 172.21.15.132:0/127700922 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f02700019a0 con 0x7f027813b2c0 2024-01-24T23:26:04.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:04.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:04.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 10 monclient(hunting): dump: 2024-01-24T23:26:04.284 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:04.284 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:04.284 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:04.284 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:04.284 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:04.284 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:04.284 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:04.284 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:04.284 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:04.285 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:04.285 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:04.285 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:04.285 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:04.285 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:04.285 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:04.285 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:04.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:04.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 1 -- 172.21.15.132:0/127700922 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1588532286 0 0) 0x7f0278071680 con 0x7f027813b2c0 2024-01-24T23:26:04.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 1 -- 172.21.15.132:0/127700922 >> v1:172.21.15.132:6792/0 conn(0x7f027809ff20 legacy=0x7f02781d0710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:04.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 1 -- 172.21.15.132:0/127700922 >> v1:172.21.15.115:6792/0 conn(0x7f027809f9c0 legacy=0x7f02781cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:04.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:26:04.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 1 monclient: found mon.g 2024-01-24T23:26:04.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:04.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:04.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 1 -- 172.21.15.132:0/127700922 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0278125c30 con 0x7f027813b2c0 2024-01-24T23:26:04.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 10 monclient: _finish_auth 0 2024-01-24T23:26:04.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:26:04.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:04.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 5 monclient: authenticate success, global_id 6177 2024-01-24T23:26:04.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 10 monclient: _renew_subs 2024-01-24T23:26:04.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:04.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 1 -- 172.21.15.132:0/127700922 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02781399e0 con 0x7f027813b2c0 2024-01-24T23:26:04.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 10 monclient: _renew_subs 2024-01-24T23:26:04.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:04.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 1 -- 172.21.15.132:0/127700922 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f02781244d0 con 0x7f027813b2c0 2024-01-24T23:26:04.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 1 -- 172.21.15.132:0/127700922 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0270005050 con 0x7f027813b2c0 2024-01-24T23:26:04.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:04.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 1 -- 172.21.15.132:0/127700922 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f02700013a0 con 0x7f027813b2c0 2024-01-24T23:26:04.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:04.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f0275ffb640 4 set_mon_vals no callback set 2024-01-24T23:26:04.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:04.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 10 monclient: dump: 2024-01-24T23:26:04.287 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:04.287 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:04.287 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:04.287 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:04.287 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:04.287 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:04.287 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:04.287 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:04.287 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:04.287 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:04.287 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:04.287 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:04.287 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:04.287 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:04.288 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:04.288 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:04.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:04.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:04.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:04.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f027d641640 1 -- 172.21.15.132:0/127700922 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025c005090 con 0x7f027813b2c0 2024-01-24T23:26:04.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 1 -- 172.21.15.132:0/127700922 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f027000fea0 con 0x7f027813b2c0 2024-01-24T23:26:04.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.268+0000 7f026f7fe640 1 -- 172.21.15.132:0/127700922 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(148..148 src has 1..148) v4 ==== 8349+0+0 (unknown 316280510 0 0) 0x7f0270044ef0 con 0x7f027813b2c0 2024-01-24T23:26:04.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.272+0000 7f0276ffd640 10 monclient: get_auth_request con 0x7f0258037050 auth_method 0 2024-01-24T23:26:04.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.276+0000 7f026f7fe640 1 -- 172.21.15.132:0/127700922 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f0270011440 con 0x7f027813b2c0 2024-01-24T23:26:04.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.276+0000 7f026f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:04.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.276+0000 7f026f7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:04.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.432+0000 7f027d641640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}] 2024-01-24T23:26:04.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.432+0000 7f027d641640 10 monclient: _send_command 2 [{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}] 2024-01-24T23:26:04.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.432+0000 7f027d641640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:04.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:04.432+0000 7f027d641640 1 -- 172.21.15.132:0/127700922 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]} v 0) v1 -- 0x7f025c005340 con 0x7f027813b2c0 2024-01-24T23:26:05.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:05.128+0000 7f026f7fe640 1 -- 172.21.15.132:0/127700922 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}]=0 set osd(s) 0 to class 'fish' v149) v1 ==== 137+0+0 (unknown 2246827698 0 0) 0x7f02700134a0 con 0x7f027813b2c0 2024-01-24T23:26:05.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:05.128+0000 7f026f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}] 2024-01-24T23:26:05.130 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:05.128+0000 7f026f7fe640 10 monclient: _finish_command 2 = system:0 set osd(s) 0 to class 'fish' 2024-01-24T23:26:05.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:05.228+0000 7f027d641640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}] 2024-01-24T23:26:05.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:05.228+0000 7f027d641640 10 monclient: _send_command 3 [{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}] 2024-01-24T23:26:05.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:05.228+0000 7f027d641640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:05.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:05.228+0000 7f027d641640 1 -- 172.21.15.132:0/127700922 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]} v 0) v1 -- 0x7f025c000c00 con 0x7f027813b2c0 2024-01-24T23:26:06.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.152+0000 7f026f7fe640 1 -- 172.21.15.132:0/127700922 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}]=0 osd.0 already set to class fish. set-device-class item id 0 name 'osd.0' device_class 'fish': no change. set osd(s) to class 'fish' v150) v1 ==== 241+0+0 (unknown 3166397269 0 0) 0x7f0270014c20 con 0x7f027813b2c0 2024-01-24T23:26:06.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.152+0000 7f026f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}] 2024-01-24T23:26:06.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.152+0000 7f026f7fe640 10 monclient: _finish_command 3 = system:0 osd.0 already set to class fish. set-device-class item id 0 name 'osd.0' device_class 'fish': no change. set osd(s) to class 'fish' 2024-01-24T23:26:06.155 INFO:tasks.workunit.client.0.smithi132.stderr:osd.0 already set to class fish. set-device-class item id 0 name 'osd.0' device_class 'fish': no change. set osd(s) to class 'fish' 2024-01-24T23:26:06.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.160+0000 7f027d641640 1 -- 172.21.15.132:0/127700922 >> v1:172.21.15.132:6812/1835401922 conn(0x7f0258037050 legacy=0x7f0258039510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:06.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.160+0000 7f027d641640 10 monclient: shutdown 2024-01-24T23:26:06.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.160+0000 7f027d641640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:06.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.160+0000 7f027d641640 1 -- 172.21.15.132:0/127700922 >> v1:172.21.15.132:6790/0 conn(0x7f027813b2c0 legacy=0x7f02781d3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:06.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.160+0000 7f027d641640 1 -- 172.21.15.132:0/127700922 shutdown_connections 2024-01-24T23:26:06.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.160+0000 7f027d641640 1 -- 172.21.15.132:0/127700922 >> 172.21.15.132:0/127700922 conn(0x7f02780a5a10 msgr2=0x7f0278139e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:06.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.160+0000 7f027d641640 1 -- 172.21.15.132:0/127700922 shutdown_connections 2024-01-24T23:26:06.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.160+0000 7f027d641640 1 -- 172.21.15.132:0/127700922 wait complete. 2024-01-24T23:26:06.181 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush tree --show-shadow 2024-01-24T23:26:06.181 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep 'osd\.0' 2024-01-24T23:26:06.181 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep fish 2024-01-24T23:26:06.181 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep '10\.' 2024-01-24T23:26:06.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.296+0000 7f58e1571640 10 monclient: get_monmap_and_config 2024-01-24T23:26:06.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.296+0000 7f58e1571640 10 monclient: build_initial_monmap 2024-01-24T23:26:06.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.296+0000 7f58e1571640 10 monclient: monmap: 2024-01-24T23:26:06.298 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:06.298 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:06.298 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:06.299538+0000 2024-01-24T23:26:06.299 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:06.299538+0000 2024-01-24T23:26:06.299 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:06.299 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:06.299 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:06.299 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:06.299 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:06.299 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:06.299 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:06.299 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:06.299 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:06.299 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:06.299 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:06.299 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:06.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 1 Processor -- start 2024-01-24T23:26:06.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 1 -- start start 2024-01-24T23:26:06.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 10 monclient: init 2024-01-24T23:26:06.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:06.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 10 monclient: _add_conns ranks=[2,3,5,0,6,1,4,7,8] 2024-01-24T23:26:06.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 10 monclient(hunting): picked mon.noname-b con 0x7f58dc135e10 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:06.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 10 monclient(hunting): picked mon.noname-g con 0x7f58dc1361f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:06.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 10 monclient(hunting): picked mon.noname-h con 0x7f58dc1365f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:06.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58dc064700 con 0x7f58dc135e10 2024-01-24T23:26:06.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58dc09c3e0 con 0x7f58dc1361f0 2024-01-24T23:26:06.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58dc0a4f90 con 0x7f58dc1365f0 2024-01-24T23:26:06.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:06.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 10 monclient(hunting): authenticate will time out at 1253.624801s 2024-01-24T23:26:06.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58daffd640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f58dc135e10 legacy=0x7f58dc09bcd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:06.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58db7fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f58dc1365f0 0x7f58dc13f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:35242/0 (socket says 172.21.15.132:35242) 2024-01-24T23:26:06.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58da7fc640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f58dc1361f0 0x7f58dc13bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:48476/0 (socket says 172.21.15.132:48476) 2024-01-24T23:26:06.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58db7fe640 1 -- 172.21.15.132:0/939116826 learned_addr learned my addr 172.21.15.132:0/939116826 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:06.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 1 -- 172.21.15.132:0/939116826 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2670536358 0 0) 0x7f58dc09c3e0 con 0x7f58dc1361f0 2024-01-24T23:26:06.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:26:06.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:06.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:06.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 10 monclient(hunting): my global_id is 6186 2024-01-24T23:26:06.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 1 -- 172.21.15.132:0/939116826 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58c4003530 con 0x7f58dc1361f0 2024-01-24T23:26:06.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 1 -- 172.21.15.132:0/939116826 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1976480956 0 0) 0x7f58dc0a4f90 con 0x7f58dc1365f0 2024-01-24T23:26:06.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:26:06.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:06.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:06.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 10 monclient(hunting): my global_id is 6044 2024-01-24T23:26:06.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 1 -- 172.21.15.132:0/939116826 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58dc09c3e0 con 0x7f58dc1365f0 2024-01-24T23:26:06.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 1 -- 172.21.15.132:0/939116826 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 471793270 0 0) 0x7f58dc09c3e0 con 0x7f58dc1365f0 2024-01-24T23:26:06.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 1 -- 172.21.15.132:0/939116826 >> v1:172.21.15.132:6790/0 conn(0x7f58dc1361f0 legacy=0x7f58dc13bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:06.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 1 -- 172.21.15.132:0/939116826 >> v1:172.21.15.115:6790/0 conn(0x7f58dc135e10 legacy=0x7f58dc09bcd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:06.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:26:06.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 1 monclient: found mon.noname-h 2024-01-24T23:26:06.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:06.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:26:06.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 1 -- 172.21.15.132:0/939116826 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc09b010 con 0x7f58dc1365f0 2024-01-24T23:26:06.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 10 monclient: _finish_auth 0 2024-01-24T23:26:06.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:26:06.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:06.309 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 1 -- 172.21.15.132:0/939116826 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f58d4001b20 con 0x7f58dc1365f0 2024-01-24T23:26:06.309 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 5 monclient: authenticate success, global_id 6044 2024-01-24T23:26:06.309 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:06.309 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:06.309 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:06.309 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 10 monclient: dump: 2024-01-24T23:26:06.309 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:06.309 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:06.309 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:06.309 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:06.309 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:06.309 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:06.309 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:06.309 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:06.310 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:06.310 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:06.310 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:06.310 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:06.310 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:06.310 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:06.310 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:06.310 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:06.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:06.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 1 -- 172.21.15.132:0/939116826 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f58d4001dc0 con 0x7f58dc1365f0 2024-01-24T23:26:06.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:06.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:06.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 4 set_mon_vals no callback set 2024-01-24T23:26:06.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 1 -- 172.21.15.132:0/939116826 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f58d4004f70 con 0x7f58dc1365f0 2024-01-24T23:26:06.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:06.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:06.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58d9ffb640 10 monclient: dump: 2024-01-24T23:26:06.311 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:06.311 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:06.311 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:06.311 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:06.311 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:06.311 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:06.311 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:06.311 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:06.311 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:06.311 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:06.311 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:06.311 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:06.312 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:06.312 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:06.312 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:06.312 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:06.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 10 monclient: shutdown 2024-01-24T23:26:06.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:06.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 1 -- 172.21.15.132:0/939116826 >> v1:172.21.15.132:6791/0 conn(0x7f58dc1365f0 legacy=0x7f58dc13f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:06.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 1 -- 172.21.15.132:0/939116826 shutdown_connections 2024-01-24T23:26:06.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 1 -- 172.21.15.132:0/939116826 >> 172.21.15.132:0/939116826 conn(0x7f58dc0a6790 msgr2=0x7f58dc071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:06.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 1 -- 172.21.15.132:0/939116826 shutdown_connections 2024-01-24T23:26:06.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 1 -- 172.21.15.132:0/939116826 wait complete. 2024-01-24T23:26:06.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 10 monclient: build_initial_monmap 2024-01-24T23:26:06.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 10 monclient: monmap: 2024-01-24T23:26:06.313 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:06.313 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:06.313 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:06.313 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:06.313 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:06.313 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:06.313 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:06.313 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:06.313 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:06.313 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:06.313 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:06.313 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:06.313 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:06.313 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:06.313 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:06.314 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:06.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 1 Processor -- start 2024-01-24T23:26:06.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 1 -- start start 2024-01-24T23:26:06.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 10 monclient: init 2024-01-24T23:26:06.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:06.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 10 monclient: _add_conns ranks=[1,8,2,0,3,5,6,4,7] 2024-01-24T23:26:06.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 10 monclient(hunting): picked mon.noname-f con 0x7f58dc135e10 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:06.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 10 monclient(hunting): picked mon.noname-e con 0x7f58dc1361f0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:06.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 10 monclient(hunting): picked mon.noname-b con 0x7f58dc1365f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:06.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58dc1222b0 con 0x7f58dc135e10 2024-01-24T23:26:06.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58dc09c6b0 con 0x7f58dc1365f0 2024-01-24T23:26:06.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58dc120780 con 0x7f58dc1361f0 2024-01-24T23:26:06.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:06.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58e1571640 10 monclient(hunting): authenticate will time out at 1253.627735s 2024-01-24T23:26:06.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58daffd640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f58dc135e10 legacy=0x7f58dc11f960 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:06.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.300+0000 7f58db7fe640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f58dc1365f0 legacy=0x7f58dc1d3590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:06.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58da7fc640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f58dc1361f0 0x7f58dc120070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:36090/0 (socket says 172.21.15.132:36090) 2024-01-24T23:26:06.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58da7fc640 1 -- 172.21.15.132:0/2577838675 learned_addr learned my addr 172.21.15.132:0/2577838675 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:06.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 1 -- 172.21.15.132:0/2577838675 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2917548797 0 0) 0x7f58dc120780 con 0x7f58dc1361f0 2024-01-24T23:26:06.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 10 monclient(hunting): _negotiate 2024-01-24T23:26:06.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:06.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:06.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 10 monclient(hunting): my global_id is 85355 2024-01-24T23:26:06.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 1 -- 172.21.15.132:0/2577838675 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58c8003530 con 0x7f58dc1361f0 2024-01-24T23:26:06.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 1 -- 172.21.15.132:0/2577838675 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3290545141 0 0) 0x7f58c8003530 con 0x7f58dc1361f0 2024-01-24T23:26:06.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 1 -- 172.21.15.132:0/2577838675 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f58dc120780 con 0x7f58dc1361f0 2024-01-24T23:26:06.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 1 -- 172.21.15.132:0/2577838675 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f58d0001490 con 0x7f58dc1361f0 2024-01-24T23:26:06.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:06.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:26:06.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 10 monclient(hunting): dump: 2024-01-24T23:26:06.316 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:06.316 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:06.316 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:06.316 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:06.316 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:06.316 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:06.316 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:06.316 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:06.316 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:06.316 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:06.316 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:06.316 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:06.316 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:06.317 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:06.317 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:06.317 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:06.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:06.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 1 -- 172.21.15.132:0/2577838675 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3551926537 0 0) 0x7f58dc120780 con 0x7f58dc1361f0 2024-01-24T23:26:06.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 1 -- 172.21.15.132:0/2577838675 >> v1:172.21.15.115:6790/0 conn(0x7f58dc1365f0 legacy=0x7f58dc1d3590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:06.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 1 -- 172.21.15.132:0/2577838675 >> v1:172.21.15.132:6789/0 conn(0x7f58dc135e10 legacy=0x7f58dc11f960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:06.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 10 monclient: _finish_hunting 0 2024-01-24T23:26:06.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 1 monclient: found mon.e 2024-01-24T23:26:06.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:06.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:06.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 1 -- 172.21.15.132:0/2577838675 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc1bf370 con 0x7f58dc1361f0 2024-01-24T23:26:06.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 10 monclient: _finish_auth 0 2024-01-24T23:26:06.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 10 monclient: _check_auth_tickets 2024-01-24T23:26:06.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:06.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58e1571640 5 monclient: authenticate success, global_id 85355 2024-01-24T23:26:06.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58e1571640 10 monclient: _renew_subs 2024-01-24T23:26:06.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58e1571640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:06.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58e1571640 1 -- 172.21.15.132:0/2577838675 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58dc121fd0 con 0x7f58dc1361f0 2024-01-24T23:26:06.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58e1571640 10 monclient: _renew_subs 2024-01-24T23:26:06.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58e1571640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:06.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58e1571640 1 -- 172.21.15.132:0/2577838675 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f58dc135820 con 0x7f58dc1361f0 2024-01-24T23:26:06.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 1 -- 172.21.15.132:0/2577838675 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f58d0004aa0 con 0x7f58dc1361f0 2024-01-24T23:26:06.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:06.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 1 -- 172.21.15.132:0/2577838675 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f58d0004f50 con 0x7f58dc1361f0 2024-01-24T23:26:06.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:06.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:26:06.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58d97fa640 4 set_mon_vals no callback set 2024-01-24T23:26:06.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 10 monclient: dump: 2024-01-24T23:26:06.319 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:06.319 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:06.319 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:06.319 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:06.319 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:06.319 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:06.319 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:06.319 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:06.319 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:06.319 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:06.319 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:06.319 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:06.320 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:06.320 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:06.320 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:06.320 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:06.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58e1571640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:06.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58e1571640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:06.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58e1571640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:06.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58e1571640 1 -- 172.21.15.132:0/2577838675 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58bc005090 con 0x7f58dc1361f0 2024-01-24T23:26:06.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 1 -- 172.21.15.132:0/2577838675 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f58d000fa50 con 0x7f58dc1361f0 2024-01-24T23:26:06.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58c37fe640 1 -- 172.21.15.132:0/2577838675 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(150..150 src has 1..150) v4 ==== 9014+0+0 (unknown 1497245371 0 0) 0x7f58d0044d50 con 0x7f58dc1361f0 2024-01-24T23:26:06.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.304+0000 7f58daffd640 10 monclient: get_auth_request con 0x7f58c8036fe0 auth_method 0 2024-01-24T23:26:06.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.316+0000 7f58c37fe640 1 -- 172.21.15.132:0/2577838675 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f58d0014910 con 0x7f58dc1361f0 2024-01-24T23:26:06.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.316+0000 7f58c37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:06.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.316+0000 7f58c37fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:06.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.480+0000 7f58e1571640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:06.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.480+0000 7f58e1571640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:06.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.480+0000 7f58e1571640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:06.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.480+0000 7f58e1571640 1 -- 172.21.15.132:0/2577838675 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f58bc005340 con 0x7f58dc1361f0 2024-01-24T23:26:06.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.484+0000 7f58c37fe640 1 -- 172.21.15.132:0/2577838675 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v150) v1 ==== 83+0+2175 (unknown 330556735 0 984707583) 0x7f58d0014730 con 0x7f58dc1361f0 2024-01-24T23:26:06.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.484+0000 7f58c37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:06.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.484+0000 7f58c37fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:26:06.584 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.584+0000 7f58e1571640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:06.584 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.584+0000 7f58e1571640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:06.584 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.584+0000 7f58e1571640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:06.584 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.584+0000 7f58e1571640 1 -- 172.21.15.132:0/2577838675 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f58bc005a80 con 0x7f58dc1361f0 2024-01-24T23:26:06.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.584+0000 7f58c37fe640 1 -- 172.21.15.132:0/2577838675 <== mon.8 v1:172.21.15.115:6793/0 11 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v150) v1 ==== 83+0+2175 (unknown 330556735 0 984707583) 0x7f58d0014730 con 0x7f58dc1361f0 2024-01-24T23:26:06.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.584+0000 7f58c37fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:06.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.584+0000 7f58c37fe640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:26:06.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.584+0000 7f58e1571640 1 -- 172.21.15.132:0/2577838675 >> v1:172.21.15.132:6812/1835401922 conn(0x7f58c8036fe0 legacy=0x7f58c80394a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:06.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.584+0000 7f58e1571640 10 monclient: shutdown 2024-01-24T23:26:06.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.584+0000 7f58e1571640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:06.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.584+0000 7f58e1571640 1 -- 172.21.15.132:0/2577838675 >> v1:172.21.15.115:6793/0 conn(0x7f58dc1361f0 legacy=0x7f58dc120070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:06.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.584+0000 7f58e1571640 1 -- 172.21.15.132:0/2577838675 shutdown_connections 2024-01-24T23:26:06.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.584+0000 7f58e1571640 1 -- 172.21.15.132:0/2577838675 >> 172.21.15.132:0/2577838675 conn(0x7f58dc0a6790 msgr2=0x7f58dc0a7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:06.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.588+0000 7f58e1571640 1 -- 172.21.15.132:0/2577838675 shutdown_connections 2024-01-24T23:26:06.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.588+0000 7f58e1571640 1 -- 172.21.15.132:0/2577838675 wait complete. 2024-01-24T23:26:06.601 INFO:tasks.workunit.client.0.smithi132.stdout: 0 fish 0.08789 10.50000 11.50000 12.50000 osd.0 2024-01-24T23:26:06.601 INFO:tasks.workunit.client.0.smithi132.stdout: 0 fish 0.08789 10.50000 11.50000 12.50000 osd.0 2024-01-24T23:26:06.602 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep 'osd\.0' 2024-01-24T23:26:06.602 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush tree --show-shadow 2024-01-24T23:26:06.602 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep fish 2024-01-24T23:26:06.602 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep '11\.' 2024-01-24T23:26:06.717 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.716+0000 7fef71519640 10 monclient: get_monmap_and_config 2024-01-24T23:26:06.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.716+0000 7fef71519640 10 monclient: build_initial_monmap 2024-01-24T23:26:06.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.716+0000 7fef71519640 10 monclient: monmap: 2024-01-24T23:26:06.718 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:06.718 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:06.718 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:06.719316+0000 2024-01-24T23:26:06.718 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:06.719316+0000 2024-01-24T23:26:06.718 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:06.718 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:06.718 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:06.718 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:06.718 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:06.718 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:06.718 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:06.718 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:06.719 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:06.719 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:06.719 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:06.719 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:06.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.716+0000 7fef71519640 1 Processor -- start 2024-01-24T23:26:06.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.716+0000 7fef71519640 1 -- start start 2024-01-24T23:26:06.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.716+0000 7fef71519640 10 monclient: init 2024-01-24T23:26:06.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.716+0000 7fef71519640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:06.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.716+0000 7fef71519640 10 monclient: _add_conns ranks=[8,0,5,6,7,3,4,1,2] 2024-01-24T23:26:06.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.716+0000 7fef71519640 10 monclient(hunting): picked mon.noname-e con 0x7fef6c136cb0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:06.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.716+0000 7fef71519640 10 monclient(hunting): picked mon.noname-a con 0x7fef6c1377a0 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:06.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 10 monclient(hunting): picked mon.noname-h con 0x7fef6c09bcc0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:06.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6c064700 con 0x7fef6c1377a0 2024-01-24T23:26:06.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6c09b650 con 0x7fef6c09bcc0 2024-01-24T23:26:06.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6c09c110 con 0x7fef6c136cb0 2024-01-24T23:26:06.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:06.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 10 monclient(hunting): authenticate will time out at 1254.044107s 2024-01-24T23:26:06.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef6b7fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fef6c09bcc0 0x7fef6c13f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:35252/0 (socket says 172.21.15.132:35252) 2024-01-24T23:26:06.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef6b7fe640 1 -- 172.21.15.132:0/4020872449 learned_addr learned my addr 172.21.15.132:0/4020872449 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:06.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 1 -- 172.21.15.132:0/4020872449 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1831950429 0 0) 0x7fef6c09b650 con 0x7fef6c09bcc0 2024-01-24T23:26:06.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:26:06.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:06.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:06.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 10 monclient(hunting): my global_id is 6053 2024-01-24T23:26:06.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 1 -- 172.21.15.132:0/4020872449 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef54003530 con 0x7fef6c09bcc0 2024-01-24T23:26:06.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 1 -- 172.21.15.132:0/4020872449 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 24313814 0 0) 0x7fef54003530 con 0x7fef6c09bcc0 2024-01-24T23:26:06.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 1 -- 172.21.15.132:0/4020872449 >> v1:172.21.15.115:6793/0 conn(0x7fef6c136cb0 legacy=0x7fef6c137090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:06.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 1 -- 172.21.15.132:0/4020872449 >> v1:172.21.15.115:6789/0 conn(0x7fef6c1377a0 legacy=0x7fef6c13c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:06.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:26:06.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 1 monclient: found mon.noname-h 2024-01-24T23:26:06.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:06.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:26:06.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 1 -- 172.21.15.132:0/4020872449 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c09b000 con 0x7fef6c09bcc0 2024-01-24T23:26:06.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 10 monclient: _finish_auth 0 2024-01-24T23:26:06.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:26:06.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:06.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 1 -- 172.21.15.132:0/4020872449 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fef640017c0 con 0x7fef6c09bcc0 2024-01-24T23:26:06.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 5 monclient: authenticate success, global_id 6053 2024-01-24T23:26:06.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:06.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:06.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:06.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 10 monclient: dump: 2024-01-24T23:26:06.724 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:06.724 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:06.724 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:06.724 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:06.724 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:06.725 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:06.725 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:06.725 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:06.725 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:06.725 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:06.725 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:06.725 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:06.725 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:06.725 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:06.725 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:06.725 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:06.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:06.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 1 -- 172.21.15.132:0/4020872449 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fef64001a60 con 0x7fef6c09bcc0 2024-01-24T23:26:06.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:06.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:06.725 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 4 set_mon_vals no callback set 2024-01-24T23:26:06.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 1 -- 172.21.15.132:0/4020872449 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fef64004bf0 con 0x7fef6c09bcc0 2024-01-24T23:26:06.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:06.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:06.726 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef69ffb640 10 monclient: dump: 2024-01-24T23:26:06.726 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:06.726 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:06.726 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:06.726 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:06.726 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:06.726 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:06.726 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:06.726 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:06.726 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:06.726 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:06.726 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:06.726 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:06.727 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:06.727 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:06.727 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:06.727 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:06.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 10 monclient: shutdown 2024-01-24T23:26:06.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:06.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 1 -- 172.21.15.132:0/4020872449 >> v1:172.21.15.132:6791/0 conn(0x7fef6c09bcc0 legacy=0x7fef6c13f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:06.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 1 -- 172.21.15.132:0/4020872449 shutdown_connections 2024-01-24T23:26:06.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 1 -- 172.21.15.132:0/4020872449 >> 172.21.15.132:0/4020872449 conn(0x7fef6c0a9c10 msgr2=0x7fef6c0a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:06.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 1 -- 172.21.15.132:0/4020872449 shutdown_connections 2024-01-24T23:26:06.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 1 -- 172.21.15.132:0/4020872449 wait complete. 2024-01-24T23:26:06.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 10 monclient: build_initial_monmap 2024-01-24T23:26:06.727 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 10 monclient: monmap: 2024-01-24T23:26:06.727 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:06.727 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:06.727 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:06.727 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:06.728 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:06.728 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:06.728 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:06.728 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:06.728 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:06.728 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:06.728 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:06.728 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:06.728 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:06.728 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:06.728 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:06.728 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:06.728 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 1 Processor -- start 2024-01-24T23:26:06.728 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 1 -- start start 2024-01-24T23:26:06.728 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 10 monclient: init 2024-01-24T23:26:06.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:06.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 10 monclient: _add_conns ranks=[4,7,2,0,1,6,5,3,8] 2024-01-24T23:26:06.731 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 10 monclient(hunting): picked mon.noname-c con 0x7fef6c09bcc0 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:06.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 10 monclient(hunting): picked mon.noname-i con 0x7fef6c136cb0 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:06.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 10 monclient(hunting): picked mon.noname-b con 0x7fef6c1377a0 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:06.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6c120450 con 0x7fef6c1377a0 2024-01-24T23:26:06.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6c09c6a0 con 0x7fef6c09bcc0 2024-01-24T23:26:06.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef6a7fc640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fef6c136cb0 legacy=0x7fef6c1d0650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:06.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6c137fb0 con 0x7fef6c136cb0 2024-01-24T23:26:06.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:06.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef71519640 10 monclient(hunting): authenticate will time out at 1254.047593s 2024-01-24T23:26:06.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef6b7fe640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fef6c1377a0 legacy=0x7fef6c1d3d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:06.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef6affd640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fef6c09bcc0 0x7fef6c130a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:56702/0 (socket says 172.21.15.132:56702) 2024-01-24T23:26:06.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.720+0000 7fef6affd640 1 -- 172.21.15.132:0/2142539027 learned_addr learned my addr 172.21.15.132:0/2142539027 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:06.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 1 -- 172.21.15.132:0/2142539027 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1104462619 0 0) 0x7fef6c09c6a0 con 0x7fef6c09bcc0 2024-01-24T23:26:06.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 10 monclient(hunting): _negotiate 2024-01-24T23:26:06.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:06.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:06.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 10 monclient(hunting): my global_id is 94414 2024-01-24T23:26:06.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 1 -- 172.21.15.132:0/2142539027 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef58003530 con 0x7fef6c09bcc0 2024-01-24T23:26:06.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 1 -- 172.21.15.132:0/2142539027 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2368534032 0 0) 0x7fef58003530 con 0x7fef6c09bcc0 2024-01-24T23:26:06.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 1 -- 172.21.15.132:0/2142539027 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fef6c09c6a0 con 0x7fef6c09bcc0 2024-01-24T23:26:06.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 1 -- 172.21.15.132:0/2142539027 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fef5c001dd0 con 0x7fef6c09bcc0 2024-01-24T23:26:06.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:06.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:26:06.733 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 10 monclient(hunting): dump: 2024-01-24T23:26:06.733 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:06.733 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:06.733 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:06.733 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:06.733 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:06.733 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:06.733 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:06.733 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:06.733 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:06.733 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:06.734 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:06.734 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:06.734 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:06.734 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:06.734 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:06.734 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:06.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:06.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 1 -- 172.21.15.132:0/2142539027 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1698968251 0 0) 0x7fef6c09c6a0 con 0x7fef6c09bcc0 2024-01-24T23:26:06.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 1 -- 172.21.15.132:0/2142539027 >> v1:172.21.15.132:6792/0 conn(0x7fef6c136cb0 legacy=0x7fef6c1d0650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:06.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 1 -- 172.21.15.132:0/2142539027 >> v1:172.21.15.115:6790/0 conn(0x7fef6c1377a0 legacy=0x7fef6c1d3d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:06.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 10 monclient: _finish_hunting 0 2024-01-24T23:26:06.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 1 monclient: found mon.c 2024-01-24T23:26:06.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:06.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:06.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 1 -- 172.21.15.132:0/2142539027 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c09fa70 con 0x7fef6c09bcc0 2024-01-24T23:26:06.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 10 monclient: _finish_auth 0 2024-01-24T23:26:06.734 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 10 monclient: _check_auth_tickets 2024-01-24T23:26:06.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:06.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef71519640 5 monclient: authenticate success, global_id 94414 2024-01-24T23:26:06.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef71519640 10 monclient: _renew_subs 2024-01-24T23:26:06.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef71519640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:06.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef71519640 1 -- 172.21.15.132:0/2142539027 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6c11fe90 con 0x7fef6c09bcc0 2024-01-24T23:26:06.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef71519640 10 monclient: _renew_subs 2024-01-24T23:26:06.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef71519640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:06.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef71519640 1 -- 172.21.15.132:0/2142539027 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6c09f5d0 con 0x7fef6c09bcc0 2024-01-24T23:26:06.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 1 -- 172.21.15.132:0/2142539027 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fef5c003c50 con 0x7fef6c09bcc0 2024-01-24T23:26:06.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:06.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 1 -- 172.21.15.132:0/2142539027 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fef5c004180 con 0x7fef6c09bcc0 2024-01-24T23:26:06.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef697fa640 4 set_mon_vals no callback set 2024-01-24T23:26:06.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:06.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:26:06.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 10 monclient: dump: 2024-01-24T23:26:06.735 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:06.735 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:06.736 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:06.736 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:06.736 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:06.736 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:06.736 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:06.736 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:06.736 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:06.736 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:06.736 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:06.736 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:06.736 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:06.736 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:06.736 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:06.736 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:06.736 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef71519640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:06.736 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef71519640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:06.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef71519640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:06.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef71519640 1 -- 172.21.15.132:0/2142539027 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef4c005090 con 0x7fef6c09bcc0 2024-01-24T23:26:06.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 1 -- 172.21.15.132:0/2142539027 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fef5c004360 con 0x7fef6c09bcc0 2024-01-24T23:26:06.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef537fe640 1 -- 172.21.15.132:0/2142539027 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(150..150 src has 1..150) v4 ==== 9014+0+0 (unknown 1497245371 0 0) 0x7fef5c0450f0 con 0x7fef6c09bcc0 2024-01-24T23:26:06.737 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.724+0000 7fef6a7fc640 10 monclient: get_auth_request con 0x7fef58036ef0 auth_method 0 2024-01-24T23:26:06.738 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.736+0000 7fef537fe640 1 -- 172.21.15.132:0/2142539027 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fef5c014f00 con 0x7fef6c09bcc0 2024-01-24T23:26:06.739 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.736+0000 7fef537fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:06.739 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.736+0000 7fef537fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:06.903 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.904+0000 7fef71519640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:06.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.904+0000 7fef71519640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:06.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.904+0000 7fef71519640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:06.904 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.904+0000 7fef71519640 1 -- 172.21.15.132:0/2142539027 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7fef4c005340 con 0x7fef6c09bcc0 2024-01-24T23:26:06.910 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.908+0000 7fef537fe640 1 -- 172.21.15.132:0/2142539027 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v151) v1 ==== 83+0+2175 (unknown 930915683 0 984707583) 0x7fef5c013370 con 0x7fef6c09bcc0 2024-01-24T23:26:06.910 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.908+0000 7fef537fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:06.910 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:06.908+0000 7fef537fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:26:07.003 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.004+0000 7fef71519640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:07.003 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.004+0000 7fef71519640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:07.003 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.004+0000 7fef71519640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:07.004 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.004+0000 7fef71519640 1 -- 172.21.15.132:0/2142539027 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7fef4c005a80 con 0x7fef6c09bcc0 2024-01-24T23:26:07.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.004+0000 7fef537fe640 1 -- 172.21.15.132:0/2142539027 <== mon.4 v1:172.21.15.115:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v151) v1 ==== 83+0+2175 (unknown 930915683 0 984707583) 0x7fef5c014af0 con 0x7fef6c09bcc0 2024-01-24T23:26:07.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.004+0000 7fef537fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:07.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.004+0000 7fef537fe640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:26:07.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.004+0000 7fef71519640 1 -- 172.21.15.132:0/2142539027 >> v1:172.21.15.132:6812/1835401922 conn(0x7fef58036ef0 legacy=0x7fef580393b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:07.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.004+0000 7fef71519640 10 monclient: shutdown 2024-01-24T23:26:07.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.004+0000 7fef71519640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:07.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.004+0000 7fef71519640 1 -- 172.21.15.132:0/2142539027 >> v1:172.21.15.115:6791/0 conn(0x7fef6c09bcc0 legacy=0x7fef6c130a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:07.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.008+0000 7fef71519640 1 -- 172.21.15.132:0/2142539027 shutdown_connections 2024-01-24T23:26:07.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.008+0000 7fef71519640 1 -- 172.21.15.132:0/2142539027 >> 172.21.15.132:0/2142539027 conn(0x7fef6c0a9c10 msgr2=0x7fef6c0a7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:07.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.008+0000 7fef71519640 1 -- 172.21.15.132:0/2142539027 shutdown_connections 2024-01-24T23:26:07.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.008+0000 7fef71519640 1 -- 172.21.15.132:0/2142539027 wait complete. 2024-01-24T23:26:07.021 INFO:tasks.workunit.client.0.smithi132.stdout: 0 fish 0.08789 10.50000 11.50000 12.50000 osd.0 2024-01-24T23:26:07.021 INFO:tasks.workunit.client.0.smithi132.stdout: 0 fish 0.08789 10.50000 11.50000 12.50000 osd.0 2024-01-24T23:26:07.022 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush tree --show-shadow 2024-01-24T23:26:07.022 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep 'osd\.0' 2024-01-24T23:26:07.022 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep fish 2024-01-24T23:26:07.022 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep '12\.' 2024-01-24T23:26:07.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f279e89c640 10 monclient: get_monmap_and_config 2024-01-24T23:26:07.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f279e89c640 10 monclient: build_initial_monmap 2024-01-24T23:26:07.133 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f279e89c640 10 monclient: monmap: 2024-01-24T23:26:07.133 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:07.133 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:07.134 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:07.134787+0000 2024-01-24T23:26:07.134 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:07.134787+0000 2024-01-24T23:26:07.134 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:07.134 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:07.134 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:07.134 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:07.134 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:07.134 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:07.134 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:07.134 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:07.134 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:07.134 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:07.134 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:07.134 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:07.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f279e89c640 1 Processor -- start 2024-01-24T23:26:07.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f279e89c640 1 -- start start 2024-01-24T23:26:07.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f279e89c640 10 monclient: init 2024-01-24T23:26:07.138 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f279e89c640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:07.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f279e89c640 10 monclient: _add_conns ranks=[2,4,3,5,1,6,8,0,7] 2024-01-24T23:26:07.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f279e89c640 10 monclient(hunting): picked mon.noname-b con 0x7f2798133b50 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:07.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f279e89c640 10 monclient(hunting): picked mon.noname-c con 0x7f279809ec20 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:07.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f279e89c640 10 monclient(hunting): picked mon.noname-g con 0x7f279813a1d0 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:07.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f279e89c640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2798064700 con 0x7f2798133b50 2024-01-24T23:26:07.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f279e89c640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f279809d9f0 con 0x7f279813a1d0 2024-01-24T23:26:07.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f279e89c640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f279809dc60 con 0x7f279809ec20 2024-01-24T23:26:07.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f279e89c640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:07.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f279e89c640 10 monclient(hunting): authenticate will time out at 1254.459375s 2024-01-24T23:26:07.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f279ce39640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f279813a1d0 0x7f279813c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:48486/0 (socket says 172.21.15.132:48486) 2024-01-24T23:26:07.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f279ce39640 1 -- 172.21.15.132:0/3156514520 learned_addr learned my addr 172.21.15.132:0/3156514520 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:07.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f2797fff640 1 -- 172.21.15.132:0/3156514520 >> v1:172.21.15.115:6790/0 conn(0x7f2798133b50 legacy=0x7f2798133f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:07.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f2796ffd640 1 -- 172.21.15.132:0/3156514520 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3584822309 0 0) 0x7f279809d9f0 con 0x7f279813a1d0 2024-01-24T23:26:07.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f2796ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:07.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f2796ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:07.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f2796ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:07.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f2796ffd640 10 monclient(hunting): my global_id is 6195 2024-01-24T23:26:07.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f2796ffd640 1 -- 172.21.15.132:0/3156514520 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2780003530 con 0x7f279813a1d0 2024-01-24T23:26:07.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.132+0000 7f2796ffd640 1 -- 172.21.15.132:0/3156514520 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3177042395 0 0) 0x7f2780003530 con 0x7f279813a1d0 2024-01-24T23:26:07.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2796ffd640 1 -- 172.21.15.132:0/3156514520 >> v1:172.21.15.115:6791/0 conn(0x7f279809ec20 legacy=0x7f2798138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:07.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2796ffd640 1 -- 172.21.15.132:0/3156514520 >> v1:172.21.15.115:6790/0 conn(0x7f2798133b50 legacy=0x7f2798133f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:07.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2796ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:26:07.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2796ffd640 1 monclient: found mon.noname-g 2024-01-24T23:26:07.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2796ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:07.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2796ffd640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:26:07.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2796ffd640 1 -- 172.21.15.132:0/3156514520 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f279809df40 con 0x7f279813a1d0 2024-01-24T23:26:07.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2796ffd640 10 monclient: _finish_auth 0 2024-01-24T23:26:07.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2796ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:26:07.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2796ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:07.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2796ffd640 1 -- 172.21.15.132:0/3156514520 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2790001b40 con 0x7f279813a1d0 2024-01-24T23:26:07.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 5 monclient: authenticate success, global_id 6195 2024-01-24T23:26:07.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:07.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2796ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:07.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2796ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:07.141 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2796ffd640 10 monclient: dump: 2024-01-24T23:26:07.141 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:07.141 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:07.141 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:07.141 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:07.141 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:07.141 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:07.141 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:07.141 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:07.141 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:07.141 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:07.141 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:07.141 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:07.141 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:07.142 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:07.142 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:07.142 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:07.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:07.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2796ffd640 1 -- 172.21.15.132:0/3156514520 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2790001e60 con 0x7f279813a1d0 2024-01-24T23:26:07.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2796ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:07.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:07.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2796ffd640 1 -- 172.21.15.132:0/3156514520 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2790004f70 con 0x7f279813a1d0 2024-01-24T23:26:07.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 4 set_mon_vals no callback set 2024-01-24T23:26:07.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2796ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:07.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2796ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:07.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2796ffd640 10 monclient: dump: 2024-01-24T23:26:07.142 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:07.142 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:07.142 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:07.142 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:07.142 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:07.143 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:07.143 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:07.143 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:07.143 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:07.143 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:07.143 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:07.143 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:07.143 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:07.143 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:07.143 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:07.143 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:07.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 10 monclient: shutdown 2024-01-24T23:26:07.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:07.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 1 -- 172.21.15.132:0/3156514520 >> v1:172.21.15.132:6790/0 conn(0x7f279813a1d0 legacy=0x7f279813c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:07.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 1 -- 172.21.15.132:0/3156514520 shutdown_connections 2024-01-24T23:26:07.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 1 -- 172.21.15.132:0/3156514520 >> 172.21.15.132:0/3156514520 conn(0x7f27980a5a10 msgr2=0x7f27980a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:07.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 1 -- 172.21.15.132:0/3156514520 shutdown_connections 2024-01-24T23:26:07.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 1 -- 172.21.15.132:0/3156514520 wait complete. 2024-01-24T23:26:07.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 10 monclient: build_initial_monmap 2024-01-24T23:26:07.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 10 monclient: monmap: 2024-01-24T23:26:07.144 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:07.144 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:07.144 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:07.144 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:07.144 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:07.144 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:07.144 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:07.144 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:07.144 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:07.144 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:07.144 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:07.144 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:07.144 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:07.145 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:07.145 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:07.145 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:07.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 1 Processor -- start 2024-01-24T23:26:07.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 1 -- start start 2024-01-24T23:26:07.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 10 monclient: init 2024-01-24T23:26:07.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:07.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 10 monclient: _add_conns ranks=[1,2,8,3,7,0,5,4,6] 2024-01-24T23:26:07.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 10 monclient(hunting): picked mon.noname-f con 0x7f279809ec20 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:07.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 10 monclient(hunting): picked mon.noname-b con 0x7f2798133b50 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:07.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 10 monclient(hunting): picked mon.noname-e con 0x7f279813a1d0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:07.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f279806edd0 con 0x7f279809ec20 2024-01-24T23:26:07.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f279806de10 con 0x7f2798133b50 2024-01-24T23:26:07.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2798133920 con 0x7f279813a1d0 2024-01-24T23:26:07.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:07.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 10 monclient(hunting): authenticate will time out at 1254.462083s 2024-01-24T23:26:07.145 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2797fff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f279809ec20 legacy=0x7f27981cb380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:07.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f27977fe640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f2798133b50 legacy=0x7f27981cba90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:07.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279ce39640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f279813a1d0 0x7f27981d35e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:36118/0 (socket says 172.21.15.132:36118) 2024-01-24T23:26:07.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279ce39640 1 -- 172.21.15.132:0/1258945346 learned_addr learned my addr 172.21.15.132:0/1258945346 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:07.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2794ff9640 1 -- 172.21.15.132:0/1258945346 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3715176164 0 0) 0x7f2798133920 con 0x7f279813a1d0 2024-01-24T23:26:07.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2794ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:26:07.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2794ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:07.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2794ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:07.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2794ff9640 10 monclient(hunting): my global_id is 85373 2024-01-24T23:26:07.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2794ff9640 1 -- 172.21.15.132:0/1258945346 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2784003560 con 0x7f279813a1d0 2024-01-24T23:26:07.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2794ff9640 1 -- 172.21.15.132:0/1258945346 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2314823093 0 0) 0x7f2784003560 con 0x7f279813a1d0 2024-01-24T23:26:07.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2794ff9640 1 -- 172.21.15.132:0/1258945346 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2798133920 con 0x7f279813a1d0 2024-01-24T23:26:07.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2794ff9640 1 -- 172.21.15.132:0/1258945346 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2790001b00 con 0x7f279813a1d0 2024-01-24T23:26:07.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2794ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:07.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2794ff9640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:26:07.146 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2794ff9640 10 monclient(hunting): dump: 2024-01-24T23:26:07.146 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:07.147 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:07.147 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:07.147 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:07.147 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:07.147 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:07.147 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:07.147 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:07.147 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:07.147 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:07.147 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:07.147 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:07.147 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:07.147 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:07.147 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:07.147 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:07.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2794ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:07.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2794ff9640 1 -- 172.21.15.132:0/1258945346 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2343828056 0 0) 0x7f2798133920 con 0x7f279813a1d0 2024-01-24T23:26:07.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2794ff9640 1 -- 172.21.15.132:0/1258945346 >> v1:172.21.15.115:6790/0 conn(0x7f2798133b50 legacy=0x7f27981cba90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:07.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2794ff9640 1 -- 172.21.15.132:0/1258945346 >> v1:172.21.15.132:6789/0 conn(0x7f279809ec20 legacy=0x7f27981cb380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:07.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2794ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:26:07.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2794ff9640 1 monclient: found mon.e 2024-01-24T23:26:07.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2794ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:07.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2794ff9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:07.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2794ff9640 1 -- 172.21.15.132:0/1258945346 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27980a13b0 con 0x7f279813a1d0 2024-01-24T23:26:07.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2794ff9640 10 monclient: _finish_auth 0 2024-01-24T23:26:07.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2794ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:26:07.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f2794ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:07.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 5 monclient: authenticate success, global_id 85373 2024-01-24T23:26:07.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.136+0000 7f279e89c640 10 monclient: _renew_subs 2024-01-24T23:26:07.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.140+0000 7f279e89c640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:07.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.140+0000 7f279e89c640 1 -- 172.21.15.132:0/1258945346 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27981cc1a0 con 0x7f279813a1d0 2024-01-24T23:26:07.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.140+0000 7f279e89c640 10 monclient: _renew_subs 2024-01-24T23:26:07.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.140+0000 7f279e89c640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:07.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.140+0000 7f279e89c640 1 -- 172.21.15.132:0/1258945346 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2798070e10 con 0x7f279813a1d0 2024-01-24T23:26:07.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.140+0000 7f2794ff9640 1 -- 172.21.15.132:0/1258945346 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2790004fd0 con 0x7f279813a1d0 2024-01-24T23:26:07.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.140+0000 7f2794ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:07.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.140+0000 7f27967fc640 4 set_mon_vals no callback set 2024-01-24T23:26:07.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.140+0000 7f2794ff9640 1 -- 172.21.15.132:0/1258945346 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2790001fd0 con 0x7f279813a1d0 2024-01-24T23:26:07.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.140+0000 7f2794ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:07.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.140+0000 7f2794ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:26:07.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.140+0000 7f2794ff9640 10 monclient: dump: 2024-01-24T23:26:07.151 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:07.151 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:07.151 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:07.151 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:07.151 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:07.151 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:07.151 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:07.151 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:07.151 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:07.151 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:07.151 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:07.151 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:07.151 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:07.151 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:07.151 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:07.151 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:07.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.140+0000 7f279e89c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:07.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.140+0000 7f279e89c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:07.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.140+0000 7f279e89c640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:07.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.140+0000 7f279e89c640 1 -- 172.21.15.132:0/1258945346 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2778005090 con 0x7f279813a1d0 2024-01-24T23:26:07.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.140+0000 7f2794ff9640 1 -- 172.21.15.132:0/1258945346 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f27900100b0 con 0x7f279813a1d0 2024-01-24T23:26:07.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.140+0000 7f2794ff9640 1 -- 172.21.15.132:0/1258945346 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(151..151 src has 1..151) v4 ==== 9014+0+0 (unknown 1297592744 0 0) 0x7f2790045130 con 0x7f279813a1d0 2024-01-24T23:26:07.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.140+0000 7f2797fff640 10 monclient: get_auth_request con 0x7f2784036f20 auth_method 0 2024-01-24T23:26:07.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.152+0000 7f2794ff9640 1 -- 172.21.15.132:0/1258945346 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f2790014d50 con 0x7f279813a1d0 2024-01-24T23:26:07.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.152+0000 7f2794ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:07.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.152+0000 7f2794ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:07.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.300+0000 7f279e89c640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:07.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.300+0000 7f279e89c640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:07.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.300+0000 7f279e89c640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:07.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.300+0000 7f279e89c640 1 -- 172.21.15.132:0/1258945346 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f2778005340 con 0x7f279813a1d0 2024-01-24T23:26:07.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.304+0000 7f2794ff9640 1 -- 172.21.15.132:0/1258945346 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v151) v1 ==== 83+0+2175 (unknown 930915683 0 984707583) 0x7f2790014b70 con 0x7f279813a1d0 2024-01-24T23:26:07.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.304+0000 7f2794ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:07.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.304+0000 7f2794ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:26:07.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.392+0000 7f279e89c640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:07.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.392+0000 7f279e89c640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:07.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.392+0000 7f279e89c640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:07.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.392+0000 7f279e89c640 1 -- 172.21.15.132:0/1258945346 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f2778005a80 con 0x7f279813a1d0 2024-01-24T23:26:07.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.396+0000 7f2794ff9640 1 -- 172.21.15.132:0/1258945346 <== mon.8 v1:172.21.15.115:6793/0 11 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v151) v1 ==== 83+0+2175 (unknown 930915683 0 984707583) 0x7f2790014d50 con 0x7f279813a1d0 2024-01-24T23:26:07.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.396+0000 7f2794ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:07.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.396+0000 7f2794ff9640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:26:07.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.396+0000 7f279e89c640 1 -- 172.21.15.132:0/1258945346 >> v1:172.21.15.132:6812/1835401922 conn(0x7f2784036f20 legacy=0x7f27840393e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:07.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.396+0000 7f279e89c640 10 monclient: shutdown 2024-01-24T23:26:07.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.396+0000 7f279e89c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:07.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.396+0000 7f279e89c640 1 -- 172.21.15.132:0/1258945346 >> v1:172.21.15.115:6793/0 conn(0x7f279813a1d0 legacy=0x7f27981d35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:07.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.396+0000 7f279e89c640 1 -- 172.21.15.132:0/1258945346 shutdown_connections 2024-01-24T23:26:07.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.396+0000 7f279e89c640 1 -- 172.21.15.132:0/1258945346 >> 172.21.15.132:0/1258945346 conn(0x7f27980a5a10 msgr2=0x7f27980a0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:07.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.396+0000 7f279e89c640 1 -- 172.21.15.132:0/1258945346 shutdown_connections 2024-01-24T23:26:07.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.396+0000 7f279e89c640 1 -- 172.21.15.132:0/1258945346 wait complete. 2024-01-24T23:26:07.410 INFO:tasks.workunit.client.0.smithi132.stdout: 0 fish 0.08789 10.50000 11.50000 12.50000 osd.0 2024-01-24T23:26:07.410 INFO:tasks.workunit.client.0.smithi132.stdout: 0 fish 0.08789 10.50000 11.50000 12.50000 osd.0 2024-01-24T23:26:07.411 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm-device-class osd.0 2024-01-24T23:26:07.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.519+0000 7f3774ad7640 10 monclient: get_monmap_and_config 2024-01-24T23:26:07.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.519+0000 7f3774ad7640 10 monclient: build_initial_monmap 2024-01-24T23:26:07.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.519+0000 7f3774ad7640 10 monclient: monmap: 2024-01-24T23:26:07.522 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:07.522 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:07.522 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:07.523418+0000 2024-01-24T23:26:07.522 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:07.523418+0000 2024-01-24T23:26:07.522 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:07.522 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:07.522 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:07.522 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:07.523 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:07.523 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:07.523 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:07.523 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:07.523 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:07.523 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:07.523 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:07.523 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:07.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.519+0000 7f3774ad7640 1 Processor -- start 2024-01-24T23:26:07.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.519+0000 7f3774ad7640 1 -- start start 2024-01-24T23:26:07.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.519+0000 7f3774ad7640 10 monclient: init 2024-01-24T23:26:07.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.519+0000 7f3774ad7640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:07.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.519+0000 7f3774ad7640 10 monclient: _add_conns ranks=[7,0,8,4,6,1,5,2,3] 2024-01-24T23:26:07.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.519+0000 7f3774ad7640 10 monclient(hunting): picked mon.noname-i con 0x7f377009ff20 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:07.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.519+0000 7f3774ad7640 10 monclient(hunting): picked mon.noname-a con 0x7f377009f9c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:07.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.519+0000 7f3774ad7640 10 monclient(hunting): picked mon.noname-e con 0x7f377013b2c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:07.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.519+0000 7f3774ad7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3770064700 con 0x7f377009f9c0 2024-01-24T23:26:07.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.519+0000 7f3774ad7640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f377009e760 con 0x7f377009ff20 2024-01-24T23:26:07.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.519+0000 7f3774ad7640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37700a1160 con 0x7f377013b2c0 2024-01-24T23:26:07.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.519+0000 7f3774ad7640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:07.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.519+0000 7f3774ad7640 10 monclient(hunting): authenticate will time out at 1254.847997s 2024-01-24T23:26:07.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376e59c640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f377009ff20 legacy=0x7f37700a0300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:07.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376ed9d640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f377013b2c0 0x7f377013d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:36130/0 (socket says 172.21.15.132:36130) 2024-01-24T23:26:07.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376dd9b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f377009f9c0 0x7f3770139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:45016/0 (socket says 172.21.15.132:45016) 2024-01-24T23:26:07.526 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376ed9d640 1 -- 172.21.15.132:0/518855065 learned_addr learned my addr 172.21.15.132:0/518855065 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:07.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 1 -- 172.21.15.132:0/518855065 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4001434593 0 0) 0x7f37700a1160 con 0x7f377013b2c0 2024-01-24T23:26:07.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 10 monclient(hunting): _negotiate 2024-01-24T23:26:07.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:07.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:07.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 10 monclient(hunting): my global_id is 85382 2024-01-24T23:26:07.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 1 -- 172.21.15.132:0/518855065 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f375c003530 con 0x7f377013b2c0 2024-01-24T23:26:07.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 1 -- 172.21.15.132:0/518855065 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 352466314 0 0) 0x7f3770064700 con 0x7f377009f9c0 2024-01-24T23:26:07.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 10 monclient(hunting): _negotiate 2024-01-24T23:26:07.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:07.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:07.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 10 monclient(hunting): my global_id is 94491 2024-01-24T23:26:07.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 1 -- 172.21.15.132:0/518855065 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37700a1160 con 0x7f377009f9c0 2024-01-24T23:26:07.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 1 -- 172.21.15.132:0/518855065 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3473163436 0 0) 0x7f375c003530 con 0x7f377013b2c0 2024-01-24T23:26:07.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 1 -- 172.21.15.132:0/518855065 >> v1:172.21.15.132:6792/0 conn(0x7f377009ff20 legacy=0x7f37700a0300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:07.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 1 -- 172.21.15.132:0/518855065 >> v1:172.21.15.115:6789/0 conn(0x7f377009f9c0 legacy=0x7f3770139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:07.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 10 monclient: _finish_hunting 0 2024-01-24T23:26:07.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 1 monclient: found mon.noname-e 2024-01-24T23:26:07.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:07.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:26:07.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 1 -- 172.21.15.132:0/518855065 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37700a0a10 con 0x7f377013b2c0 2024-01-24T23:26:07.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 10 monclient: _finish_auth 0 2024-01-24T23:26:07.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 10 monclient: _check_auth_tickets 2024-01-24T23:26:07.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:07.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 1 -- 172.21.15.132:0/518855065 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3758001730 con 0x7f377013b2c0 2024-01-24T23:26:07.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:07.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:26:07.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 10 monclient: dump: 2024-01-24T23:26:07.529 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:07.529 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:07.529 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:07.529 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:07.529 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:07.529 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:07.529 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:07.529 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:07.529 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:07.529 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:07.529 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:07.529 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:07.530 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:07.530 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:07.530 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:07.530 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:07.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f3774ad7640 5 monclient: authenticate success, global_id 85382 2024-01-24T23:26:07.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f3774ad7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:07.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 1 -- 172.21.15.132:0/518855065 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3758001940 con 0x7f377013b2c0 2024-01-24T23:26:07.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:07.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f3774ad7640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:07.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f3774ad7640 4 set_mon_vals no callback set 2024-01-24T23:26:07.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 1 -- 172.21.15.132:0/518855065 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f37580048b0 con 0x7f377013b2c0 2024-01-24T23:26:07.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:07.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:26:07.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f376d59a640 10 monclient: dump: 2024-01-24T23:26:07.530 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:07.530 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:07.531 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:07.531 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:07.531 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:07.531 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:07.531 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:07.531 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:07.531 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:07.531 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:07.531 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:07.531 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:07.531 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:07.531 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:07.531 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:07.531 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:07.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f3774ad7640 10 monclient: shutdown 2024-01-24T23:26:07.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f3774ad7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:07.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f3774ad7640 1 -- 172.21.15.132:0/518855065 >> v1:172.21.15.115:6793/0 conn(0x7f377013b2c0 legacy=0x7f377013d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:07.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f3774ad7640 1 -- 172.21.15.132:0/518855065 shutdown_connections 2024-01-24T23:26:07.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f3774ad7640 1 -- 172.21.15.132:0/518855065 >> 172.21.15.132:0/518855065 conn(0x7f37700a5a10 msgr2=0x7f37700a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:07.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f3774ad7640 1 -- 172.21.15.132:0/518855065 shutdown_connections 2024-01-24T23:26:07.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f3774ad7640 1 -- 172.21.15.132:0/518855065 wait complete. 2024-01-24T23:26:07.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f3774ad7640 10 monclient: build_initial_monmap 2024-01-24T23:26:07.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f3774ad7640 10 monclient: monmap: 2024-01-24T23:26:07.537 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:07.537 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:07.538 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:07.538 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:07.538 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:07.538 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:07.538 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:07.538 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:07.538 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:07.538 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:07.538 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:07.538 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:07.538 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:07.538 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:07.538 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:07.538 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:07.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f3774ad7640 1 Processor -- start 2024-01-24T23:26:07.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f3774ad7640 1 -- start start 2024-01-24T23:26:07.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.523+0000 7f3774ad7640 10 monclient: init 2024-01-24T23:26:07.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3774ad7640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:07.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3774ad7640 10 monclient: _add_conns ranks=[0,8,1,2,7,5,6,3,4] 2024-01-24T23:26:07.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3774ad7640 10 monclient(hunting): picked mon.noname-a con 0x7f377009f9c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:07.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3774ad7640 10 monclient(hunting): picked mon.noname-e con 0x7f377009ff20 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:07.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3774ad7640 10 monclient(hunting): picked mon.noname-f con 0x7f377013b2c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:07.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3774ad7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3770071680 con 0x7f377009f9c0 2024-01-24T23:26:07.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3774ad7640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f377011ee00 con 0x7f377013b2c0 2024-01-24T23:26:07.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3774ad7640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f377011e9e0 con 0x7f377009ff20 2024-01-24T23:26:07.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f376ed9d640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f377013b2c0 legacy=0x7f37701d1490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:07.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3774ad7640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:07.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3774ad7640 10 monclient(hunting): authenticate will time out at 1254.852301s 2024-01-24T23:26:07.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f376dd9b640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f377009ff20 0x7f377011bd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:36142/0 (socket says 172.21.15.132:36142) 2024-01-24T23:26:07.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f376e59c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f377009f9c0 0x7f377011b610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:45030/0 (socket says 172.21.15.132:45030) 2024-01-24T23:26:07.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f376dd9b640 1 -- 172.21.15.132:0/3830731366 learned_addr learned my addr 172.21.15.132:0/3830731366 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:07.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 1 -- 172.21.15.132:0/3830731366 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4195240574 0 0) 0x7f377011e9e0 con 0x7f377009ff20 2024-01-24T23:26:07.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:07.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:07.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:07.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 10 monclient(hunting): my global_id is 85391 2024-01-24T23:26:07.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 1 -- 172.21.15.132:0/3830731366 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3750003530 con 0x7f377009ff20 2024-01-24T23:26:07.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 1 -- 172.21.15.132:0/3830731366 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609732584 0 0) 0x7f3770071680 con 0x7f377009f9c0 2024-01-24T23:26:07.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:07.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:07.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:07.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 10 monclient(hunting): my global_id is 94500 2024-01-24T23:26:07.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 1 -- 172.21.15.132:0/3830731366 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f377011e9e0 con 0x7f377009f9c0 2024-01-24T23:26:07.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 1 -- 172.21.15.132:0/3830731366 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3707979885 0 0) 0x7f3750003530 con 0x7f377009ff20 2024-01-24T23:26:07.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 1 -- 172.21.15.132:0/3830731366 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3770071680 con 0x7f377009ff20 2024-01-24T23:26:07.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 1 -- 172.21.15.132:0/3830731366 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3760001ca0 con 0x7f377009ff20 2024-01-24T23:26:07.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:07.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:26:07.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 10 monclient(hunting): dump: 2024-01-24T23:26:07.541 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:07.541 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:07.541 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:07.541 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:07.541 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:07.541 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:07.541 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:07.541 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:07.541 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:07.541 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:07.542 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:07.542 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:07.542 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:07.542 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:07.542 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:07.542 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:07.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:07.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 1 -- 172.21.15.132:0/3830731366 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2939806917 0 0) 0x7f377011e9e0 con 0x7f377009f9c0 2024-01-24T23:26:07.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 1 -- 172.21.15.132:0/3830731366 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3750003530 con 0x7f377009f9c0 2024-01-24T23:26:07.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 1 -- 172.21.15.132:0/3830731366 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3768004520 con 0x7f377009f9c0 2024-01-24T23:26:07.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:07.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:26:07.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 10 monclient(hunting): dump: 2024-01-24T23:26:07.542 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:07.542 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:07.542 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:07.543 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:07.543 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:07.543 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:07.543 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:07.543 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:07.543 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:07.543 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:07.543 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:07.543 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:07.543 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:07.543 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:07.543 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:07.543 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:07.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 1 -- 172.21.15.132:0/3830731366 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2465557520 0 0) 0x7f3770071680 con 0x7f377009ff20 2024-01-24T23:26:07.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 1 -- 172.21.15.132:0/3830731366 >> v1:172.21.15.132:6789/0 conn(0x7f377013b2c0 legacy=0x7f37701d1490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:07.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 1 -- 172.21.15.132:0/3830731366 >> v1:172.21.15.115:6789/0 conn(0x7f377009f9c0 legacy=0x7f377011b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:07.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:26:07.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 1 monclient: found mon.e 2024-01-24T23:26:07.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:07.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:07.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 1 -- 172.21.15.132:0/3830731366 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377011c430 con 0x7f377009ff20 2024-01-24T23:26:07.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 10 monclient: _finish_auth 0 2024-01-24T23:26:07.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:26:07.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:07.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3774ad7640 5 monclient: authenticate success, global_id 85391 2024-01-24T23:26:07.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3774ad7640 10 monclient: _renew_subs 2024-01-24T23:26:07.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3774ad7640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:07.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3774ad7640 1 -- 172.21.15.132:0/3830731366 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f377011d250 con 0x7f377009ff20 2024-01-24T23:26:07.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3774ad7640 10 monclient: _renew_subs 2024-01-24T23:26:07.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3774ad7640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:07.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3774ad7640 1 -- 172.21.15.132:0/3830731366 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f377011e700 con 0x7f377009ff20 2024-01-24T23:26:07.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 1 -- 172.21.15.132:0/3830731366 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3760004aa0 con 0x7f377009ff20 2024-01-24T23:26:07.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:07.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 1 -- 172.21.15.132:0/3830731366 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3760004f50 con 0x7f377009ff20 2024-01-24T23:26:07.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f376cd99640 4 set_mon_vals no callback set 2024-01-24T23:26:07.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:07.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:26:07.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 10 monclient: dump: 2024-01-24T23:26:07.545 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:07.545 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:07.545 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:07.545 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:07.545 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:07.545 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:07.545 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:07.545 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:07.546 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:07.546 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:07.546 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:07.546 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:07.546 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:07.546 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:07.546 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:07.546 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:07.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3774ad7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:07.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3774ad7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:07.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3774ad7640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:07.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3774ad7640 1 -- 172.21.15.132:0/3830731366 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3754005090 con 0x7f377009ff20 2024-01-24T23:26:07.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 1 -- 172.21.15.132:0/3830731366 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f376000f9d0 con 0x7f377009ff20 2024-01-24T23:26:07.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f3766ffd640 1 -- 172.21.15.132:0/3830731366 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(151..151 src has 1..151) v4 ==== 9014+0+0 (unknown 1297592744 0 0) 0x7f3760044c50 con 0x7f377009ff20 2024-01-24T23:26:07.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.527+0000 7f376e59c640 10 monclient: get_auth_request con 0x7f3750037700 auth_method 0 2024-01-24T23:26:07.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.539+0000 7f3766ffd640 1 -- 172.21.15.132:0/3830731366 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f3760014830 con 0x7f377009ff20 2024-01-24T23:26:07.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.539+0000 7f3766ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:07.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.539+0000 7f3766ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:07.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.683+0000 7f3774ad7640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-01-24T23:26:07.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.683+0000 7f3774ad7640 10 monclient: _send_command 2 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-01-24T23:26:07.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.683+0000 7f3774ad7640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:07.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:07.683+0000 7f3774ad7640 1 -- 172.21.15.132:0/3830731366 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["osd.0"]} v 0) v1 -- 0x7f3754005b50 con 0x7f377009ff20 2024-01-24T23:26:08.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:08.179+0000 7f3766ffd640 1 -- 172.21.15.132:0/3830731366 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}]=0 done removing class of osd(s): 0 v152) v1 ==== 123+0+0 (unknown 475088009 0 0) 0x7f3760014650 con 0x7f377009ff20 2024-01-24T23:26:08.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:08.179+0000 7f3766ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-01-24T23:26:08.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:08.179+0000 7f3766ffd640 10 monclient: _finish_command 2 = system:0 done removing class of osd(s): 0 2024-01-24T23:26:08.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:08.275+0000 7f3774ad7640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-01-24T23:26:08.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:08.275+0000 7f3774ad7640 10 monclient: _send_command 3 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-01-24T23:26:08.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:08.275+0000 7f3774ad7640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:08.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:08.275+0000 7f3774ad7640 1 -- 172.21.15.132:0/3830731366 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["osd.0"]} v 0) v1 -- 0x7f37540055d0 con 0x7f377009ff20 2024-01-24T23:26:09.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.179+0000 7f3766ffd640 1 -- 172.21.15.132:0/3830731366 <== mon.8 v1:172.21.15.115:6793/0 11 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}]=0 osd.0 belongs to no class, done removing class of osd(s): v153) v1 ==== 149+0+0 (unknown 4080913855 0 0) 0x7f3760015390 con 0x7f377009ff20 2024-01-24T23:26:09.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.179+0000 7f3766ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-01-24T23:26:09.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.179+0000 7f3766ffd640 10 monclient: _finish_command 3 = system:0 osd.0 belongs to no class, done removing class of osd(s): 2024-01-24T23:26:09.181 INFO:tasks.workunit.client.0.smithi132.stderr:osd.0 belongs to no class, done removing class of osd(s): 2024-01-24T23:26:09.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.183+0000 7f3774ad7640 1 -- 172.21.15.132:0/3830731366 >> v1:172.21.15.132:6812/1835401922 conn(0x7f3750037700 legacy=0x7f3750039bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:09.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.183+0000 7f3774ad7640 10 monclient: shutdown 2024-01-24T23:26:09.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.183+0000 7f3774ad7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:09.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.183+0000 7f3774ad7640 1 -- 172.21.15.132:0/3830731366 >> v1:172.21.15.115:6793/0 conn(0x7f377009ff20 legacy=0x7f377011bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:09.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.183+0000 7f3774ad7640 1 -- 172.21.15.132:0/3830731366 shutdown_connections 2024-01-24T23:26:09.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.183+0000 7f3774ad7640 1 -- 172.21.15.132:0/3830731366 >> 172.21.15.132:0/3830731366 conn(0x7f37700a5a10 msgr2=0x7f3770066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:09.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.183+0000 7f3774ad7640 1 -- 172.21.15.132:0/3830731366 shutdown_connections 2024-01-24T23:26:09.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.183+0000 7f3774ad7640 1 -- 172.21.15.132:0/3830731366 wait complete. 2024-01-24T23:26:09.203 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush set-device-class globster osd.0 2024-01-24T23:26:09.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.323+0000 7ff599c3e640 10 monclient: get_monmap_and_config 2024-01-24T23:26:09.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.323+0000 7ff599c3e640 10 monclient: build_initial_monmap 2024-01-24T23:26:09.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.323+0000 7ff599c3e640 10 monclient: monmap: 2024-01-24T23:26:09.326 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:09.326 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:09.326 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:09.327500+0000 2024-01-24T23:26:09.326 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:09.327500+0000 2024-01-24T23:26:09.326 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:09.327 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:09.327 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:09.327 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:09.327 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:09.327 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:09.327 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:09.327 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:09.327 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:09.327 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:09.327 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:09.327 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:09.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.323+0000 7ff599c3e640 1 Processor -- start 2024-01-24T23:26:09.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.323+0000 7ff599c3e640 1 -- start start 2024-01-24T23:26:09.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.323+0000 7ff599c3e640 10 monclient: init 2024-01-24T23:26:09.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:09.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 10 monclient: _add_conns ranks=[8,7,3,6,1,4,2,5,0] 2024-01-24T23:26:09.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 10 monclient(hunting): picked mon.noname-e con 0x7ff59409ff20 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:09.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 10 monclient(hunting): picked mon.noname-i con 0x7ff59409f9c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:09.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 10 monclient(hunting): picked mon.noname-g con 0x7ff59413b2c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:09.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff594064700 con 0x7ff59413b2c0 2024-01-24T23:26:09.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff59409e760 con 0x7ff59409f9c0 2024-01-24T23:26:09.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5940a1160 con 0x7ff59409ff20 2024-01-24T23:26:09.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:09.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 10 monclient(hunting): authenticate will time out at 1256.652202s 2024-01-24T23:26:09.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff58bfff640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7ff59409f9c0 legacy=0x7ff594139ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:09.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff593fff640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7ff59413b2c0 0x7ff59413d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:48488/0 (socket says 172.21.15.132:48488) 2024-01-24T23:26:09.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff593fff640 1 -- 172.21.15.132:0/1093853265 learned_addr learned my addr 172.21.15.132:0/1093853265 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:09.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 1 -- 172.21.15.132:0/1093853265 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1208798333 0 0) 0x7ff594064700 con 0x7ff59413b2c0 2024-01-24T23:26:09.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:09.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:09.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:09.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 10 monclient(hunting): my global_id is 6204 2024-01-24T23:26:09.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 1 -- 172.21.15.132:0/1093853265 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff580003530 con 0x7ff59413b2c0 2024-01-24T23:26:09.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 1 -- 172.21.15.132:0/1093853265 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1318969657 0 0) 0x7ff580003530 con 0x7ff59413b2c0 2024-01-24T23:26:09.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 1 -- 172.21.15.132:0/1093853265 >> v1:172.21.15.115:6793/0 conn(0x7ff59409ff20 legacy=0x7ff5940a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:09.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 1 -- 172.21.15.132:0/1093853265 >> v1:172.21.15.132:6792/0 conn(0x7ff59409f9c0 legacy=0x7ff594139ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:09.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:26:09.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 1 monclient: found mon.noname-g 2024-01-24T23:26:09.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:09.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:26:09.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 1 -- 172.21.15.132:0/1093853265 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5940a0a10 con 0x7ff59413b2c0 2024-01-24T23:26:09.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 10 monclient: _finish_auth 0 2024-01-24T23:26:09.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:26:09.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:09.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 5 monclient: authenticate success, global_id 6204 2024-01-24T23:26:09.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:09.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 1 -- 172.21.15.132:0/1093853265 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff58c001b40 con 0x7ff59413b2c0 2024-01-24T23:26:09.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:09.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:09.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 10 monclient: dump: 2024-01-24T23:26:09.336 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:09.336 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:09.336 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:09.336 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:09.337 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:09.337 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:09.337 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:09.337 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:09.337 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:09.337 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:09.337 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:09.337 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:09.337 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:09.337 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:09.337 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:09.337 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:09.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:09.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 1 -- 172.21.15.132:0/1093853265 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff58c001e60 con 0x7ff59413b2c0 2024-01-24T23:26:09.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:09.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:09.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 4 set_mon_vals no callback set 2024-01-24T23:26:09.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 1 -- 172.21.15.132:0/1093853265 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff58c004f70 con 0x7ff59413b2c0 2024-01-24T23:26:09.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:09.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:09.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff592ffd640 10 monclient: dump: 2024-01-24T23:26:09.338 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:09.338 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:09.338 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:09.338 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:09.338 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:09.338 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:09.338 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:09.338 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:09.338 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:09.338 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:09.339 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:09.339 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:09.339 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:09.339 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:09.339 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:09.339 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:09.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 10 monclient: shutdown 2024-01-24T23:26:09.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:09.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 1 -- 172.21.15.132:0/1093853265 >> v1:172.21.15.132:6790/0 conn(0x7ff59413b2c0 legacy=0x7ff59413d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:09.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 1 -- 172.21.15.132:0/1093853265 shutdown_connections 2024-01-24T23:26:09.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 1 -- 172.21.15.132:0/1093853265 >> 172.21.15.132:0/1093853265 conn(0x7ff5940a5a10 msgr2=0x7ff5940a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:09.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 1 -- 172.21.15.132:0/1093853265 shutdown_connections 2024-01-24T23:26:09.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 1 -- 172.21.15.132:0/1093853265 wait complete. 2024-01-24T23:26:09.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 10 monclient: build_initial_monmap 2024-01-24T23:26:09.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 10 monclient: monmap: 2024-01-24T23:26:09.339 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:09.339 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:09.340 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:09.340 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:09.340 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:09.340 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:09.340 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:09.340 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:09.340 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:09.340 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:09.340 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:09.340 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:09.340 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:09.340 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:09.340 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:09.340 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:09.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 1 Processor -- start 2024-01-24T23:26:09.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 1 -- start start 2024-01-24T23:26:09.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 10 monclient: init 2024-01-24T23:26:09.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:09.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 10 monclient: _add_conns ranks=[5,0,1,7,8,3,6,2,4] 2024-01-24T23:26:09.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 10 monclient(hunting): picked mon.noname-h con 0x7ff59409f9c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:09.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 10 monclient(hunting): picked mon.noname-a con 0x7ff59409ff20 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:09.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 10 monclient(hunting): picked mon.noname-f con 0x7ff59413b2c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:09.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff594071960 con 0x7ff59409ff20 2024-01-24T23:26:09.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff594120e20 con 0x7ff59413b2c0 2024-01-24T23:26:09.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff59409d2e0 con 0x7ff59409f9c0 2024-01-24T23:26:09.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:09.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff599c3e640 10 monclient(hunting): authenticate will time out at 1256.655282s 2024-01-24T23:26:09.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff593fff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7ff59413b2c0 legacy=0x7ff5941d3940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:09.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff5937fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7ff59409f9c0 0x7ff59409c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:35266/0 (socket says 172.21.15.132:35266) 2024-01-24T23:26:09.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff5937fe640 1 -- 172.21.15.132:0/4249459417 learned_addr learned my addr 172.21.15.132:0/4249459417 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:09.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff590ff9640 1 -- 172.21.15.132:0/4249459417 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901582414 0 0) 0x7ff59409d2e0 con 0x7ff59409f9c0 2024-01-24T23:26:09.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff590ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:26:09.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff590ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:09.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff590ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:09.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff590ff9640 10 monclient(hunting): my global_id is 6062 2024-01-24T23:26:09.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff590ff9640 1 -- 172.21.15.132:0/4249459417 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff574003530 con 0x7ff59409f9c0 2024-01-24T23:26:09.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff590ff9640 1 -- 172.21.15.132:0/4249459417 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1081794733 0 0) 0x7ff574003530 con 0x7ff59409f9c0 2024-01-24T23:26:09.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff590ff9640 1 -- 172.21.15.132:0/4249459417 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff59409d2e0 con 0x7ff59409f9c0 2024-01-24T23:26:09.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff590ff9640 1 -- 172.21.15.132:0/4249459417 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff584001b60 con 0x7ff59409f9c0 2024-01-24T23:26:09.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff590ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:09.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff590ff9640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:09.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff590ff9640 10 monclient(hunting): dump: 2024-01-24T23:26:09.342 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:09.342 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:09.343 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:09.343 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:09.343 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:09.343 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:09.343 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:09.343 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:09.343 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:09.343 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:09.343 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:09.343 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:09.343 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:09.343 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:09.343 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:09.343 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:09.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.327+0000 7ff590ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:09.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 1 -- 172.21.15.132:0/4249459417 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455621235 0 0) 0x7ff594071960 con 0x7ff59409ff20 2024-01-24T23:26:09.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:26:09.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:09.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:09.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 10 monclient(hunting): my global_id is 94509 2024-01-24T23:26:09.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 1 -- 172.21.15.132:0/4249459417 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff574003530 con 0x7ff59409ff20 2024-01-24T23:26:09.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 1 -- 172.21.15.132:0/4249459417 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2995586280 0 0) 0x7ff59409d2e0 con 0x7ff59409f9c0 2024-01-24T23:26:09.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 1 -- 172.21.15.132:0/4249459417 >> v1:172.21.15.132:6789/0 conn(0x7ff59413b2c0 legacy=0x7ff5941d3940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:09.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 1 -- 172.21.15.132:0/4249459417 >> v1:172.21.15.115:6789/0 conn(0x7ff59409ff20 legacy=0x7ff59409cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:09.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:26:09.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 1 monclient: found mon.h 2024-01-24T23:26:09.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:09.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:09.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 1 -- 172.21.15.132:0/4249459417 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff594071680 con 0x7ff59409f9c0 2024-01-24T23:26:09.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 10 monclient: _finish_auth 0 2024-01-24T23:26:09.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:26:09.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:09.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff599c3e640 5 monclient: authenticate success, global_id 6062 2024-01-24T23:26:09.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff599c3e640 10 monclient: _renew_subs 2024-01-24T23:26:09.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff599c3e640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:09.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff599c3e640 1 -- 172.21.15.132:0/4249459417 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff594120b40 con 0x7ff59409f9c0 2024-01-24T23:26:09.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff599c3e640 10 monclient: _renew_subs 2024-01-24T23:26:09.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff599c3e640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:09.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff599c3e640 1 -- 172.21.15.132:0/4249459417 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff59409df20 con 0x7ff59409f9c0 2024-01-24T23:26:09.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 1 -- 172.21.15.132:0/4249459417 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff584003950 con 0x7ff59409f9c0 2024-01-24T23:26:09.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:09.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 1 -- 172.21.15.132:0/4249459417 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff584003e80 con 0x7ff59409f9c0 2024-01-24T23:26:09.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:09.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff5927fc640 4 set_mon_vals no callback set 2024-01-24T23:26:09.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:09.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 10 monclient: dump: 2024-01-24T23:26:09.345 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:09.346 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:09.346 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:09.346 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:09.346 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:09.346 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:09.346 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:09.346 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:09.346 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:09.346 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:09.346 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:09.346 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:09.346 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:09.346 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:09.346 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:09.346 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:09.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff599c3e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:09.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff599c3e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:09.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff599c3e640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:09.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 1 -- 172.21.15.132:0/4249459417 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7ff584003fe0 con 0x7ff59409f9c0 2024-01-24T23:26:09.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff599c3e640 1 -- 172.21.15.132:0/4249459417 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff578005090 con 0x7ff59409f9c0 2024-01-24T23:26:09.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff590ff9640 1 -- 172.21.15.132:0/4249459417 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(153..153 src has 1..153) v4 ==== 8349+0+0 (unknown 2777980158 0 0) 0x7ff584044ab0 con 0x7ff59409f9c0 2024-01-24T23:26:09.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.331+0000 7ff58bfff640 10 monclient: get_auth_request con 0x7ff574037000 auth_method 0 2024-01-24T23:26:09.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.335+0000 7ff590ff9640 1 -- 172.21.15.132:0/4249459417 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7ff584014900 con 0x7ff59409f9c0 2024-01-24T23:26:09.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.335+0000 7ff590ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:09.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.335+0000 7ff590ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:09.494 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.491+0000 7ff599c3e640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}] 2024-01-24T23:26:09.494 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.491+0000 7ff599c3e640 10 monclient: _send_command 2 [{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}] 2024-01-24T23:26:09.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.491+0000 7ff599c3e640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:09.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:09.491+0000 7ff599c3e640 1 -- 172.21.15.132:0/4249459417 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]} v 0) v1 -- 0x7ff578005340 con 0x7ff59409f9c0 2024-01-24T23:26:10.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:10.195+0000 7ff590ff9640 1 -- 172.21.15.132:0/4249459417 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}]=0 set osd(s) 0 to class 'globster' v154) v1 ==== 145+0+0 (unknown 3247958502 0 0) 0x7ff584012fa0 con 0x7ff59409f9c0 2024-01-24T23:26:10.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:10.195+0000 7ff590ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}] 2024-01-24T23:26:10.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:10.195+0000 7ff590ff9640 10 monclient: _finish_command 2 = system:0 set osd(s) 0 to class 'globster' 2024-01-24T23:26:10.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:10.295+0000 7ff599c3e640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}] 2024-01-24T23:26:10.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:10.295+0000 7ff599c3e640 10 monclient: _send_command 3 [{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}] 2024-01-24T23:26:10.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:10.295+0000 7ff599c3e640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:10.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:10.295+0000 7ff599c3e640 1 -- 172.21.15.132:0/4249459417 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]} v 0) v1 -- 0x7ff578000c00 con 0x7ff59409f9c0 2024-01-24T23:26:10.432 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~6s 2024-01-24T23:26:11.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.207+0000 7ff590ff9640 1 -- 172.21.15.132:0/4249459417 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}]=0 osd.0 already set to class globster. set-device-class item id 0 name 'osd.0' device_class 'globster': no change. set osd(s) to class 'globster' v155) v1 ==== 257+0+0 (unknown 1627845294 0 0) 0x7ff584014720 con 0x7ff59409f9c0 2024-01-24T23:26:11.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.207+0000 7ff590ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}] 2024-01-24T23:26:11.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.207+0000 7ff590ff9640 10 monclient: _finish_command 3 = system:0 osd.0 already set to class globster. set-device-class item id 0 name 'osd.0' device_class 'globster': no change. set osd(s) to class 'globster' 2024-01-24T23:26:11.208 INFO:tasks.workunit.client.0.smithi132.stderr:osd.0 already set to class globster. set-device-class item id 0 name 'osd.0' device_class 'globster': no change. set osd(s) to class 'globster' 2024-01-24T23:26:11.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.211+0000 7ff599c3e640 1 -- 172.21.15.132:0/4249459417 >> v1:172.21.15.132:6812/1835401922 conn(0x7ff574037000 legacy=0x7ff5740394c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:11.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.211+0000 7ff599c3e640 10 monclient: shutdown 2024-01-24T23:26:11.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.211+0000 7ff599c3e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:11.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.211+0000 7ff599c3e640 1 -- 172.21.15.132:0/4249459417 >> v1:172.21.15.132:6791/0 conn(0x7ff59409f9c0 legacy=0x7ff59409c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:11.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.211+0000 7ff599c3e640 1 -- 172.21.15.132:0/4249459417 shutdown_connections 2024-01-24T23:26:11.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.211+0000 7ff599c3e640 1 -- 172.21.15.132:0/4249459417 >> 172.21.15.132:0/4249459417 conn(0x7ff5940a5a10 msgr2=0x7ff594066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:11.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.211+0000 7ff599c3e640 1 -- 172.21.15.132:0/4249459417 shutdown_connections 2024-01-24T23:26:11.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.211+0000 7ff599c3e640 1 -- 172.21.15.132:0/4249459417 wait complete. 2024-01-24T23:26:11.231 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush tree --show-shadow 2024-01-24T23:26:11.231 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep 'osd\.0' 2024-01-24T23:26:11.231 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep globster 2024-01-24T23:26:11.232 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep '10\.' 2024-01-24T23:26:11.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.347+0000 7f2cbe34a640 10 monclient: get_monmap_and_config 2024-01-24T23:26:11.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.347+0000 7f2cbe34a640 10 monclient: build_initial_monmap 2024-01-24T23:26:11.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.347+0000 7f2cbe34a640 10 monclient: monmap: 2024-01-24T23:26:11.350 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:11.350 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:11.350 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:11.351297+0000 2024-01-24T23:26:11.350 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:11.351297+0000 2024-01-24T23:26:11.350 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:11.350 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:11.350 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:11.350 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:11.350 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:11.350 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:11.351 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:11.351 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:11.351 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:11.351 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:11.351 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:11.351 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:11.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.347+0000 7f2cbe34a640 1 Processor -- start 2024-01-24T23:26:11.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.347+0000 7f2cbe34a640 1 -- start start 2024-01-24T23:26:11.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.347+0000 7f2cbe34a640 10 monclient: init 2024-01-24T23:26:11.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.347+0000 7f2cbe34a640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:11.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.347+0000 7f2cbe34a640 10 monclient: _add_conns ranks=[5,1,0,6,3,8,4,2,7] 2024-01-24T23:26:11.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.347+0000 7f2cbe34a640 10 monclient(hunting): picked mon.noname-h con 0x7f2cb809c4c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:11.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.347+0000 7f2cbe34a640 10 monclient(hunting): picked mon.noname-f con 0x7f2cb809cfb0 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:11.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.347+0000 7f2cbe34a640 10 monclient(hunting): picked mon.noname-a con 0x7f2cb809ec50 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:11.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.347+0000 7f2cbe34a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb8064700 con 0x7f2cb809ec50 2024-01-24T23:26:11.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.347+0000 7f2cbe34a640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb809d9f0 con 0x7f2cb809cfb0 2024-01-24T23:26:11.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.347+0000 7f2cbe34a640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb809dc60 con 0x7f2cb809c4c0 2024-01-24T23:26:11.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.347+0000 7f2cbe34a640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:11.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.347+0000 7f2cbe34a640 10 monclient(hunting): authenticate will time out at 1258.675935s 2024-01-24T23:26:11.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.347+0000 7f2cb77fe640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f2cb809cfb0 legacy=0x7f2cb8138d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:11.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb7fff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f2cb809c4c0 0x7f2cb809c8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:35276/0 (socket says 172.21.15.132:35276) 2024-01-24T23:26:11.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb7fff640 1 -- 172.21.15.132:0/1222696918 learned_addr learned my addr 172.21.15.132:0/1222696918 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:11.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 1 -- 172.21.15.132:0/1222696918 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1802673117 0 0) 0x7f2cb809dc60 con 0x7f2cb809c4c0 2024-01-24T23:26:11.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:11.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:11.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:11.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 10 monclient(hunting): my global_id is 6071 2024-01-24T23:26:11.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 1 -- 172.21.15.132:0/1222696918 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ca0003530 con 0x7f2cb809c4c0 2024-01-24T23:26:11.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 1 -- 172.21.15.132:0/1222696918 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3522188218 0 0) 0x7f2ca0003530 con 0x7f2cb809c4c0 2024-01-24T23:26:11.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 1 -- 172.21.15.132:0/1222696918 >> v1:172.21.15.132:6789/0 conn(0x7f2cb809cfb0 legacy=0x7f2cb8138d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:11.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 1 -- 172.21.15.132:0/1222696918 >> v1:172.21.15.115:6789/0 conn(0x7f2cb809ec50 legacy=0x7f2cb813c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:11.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:26:11.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 1 monclient: found mon.noname-h 2024-01-24T23:26:11.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:11.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:26:11.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 1 -- 172.21.15.132:0/1222696918 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb809df40 con 0x7f2cb809c4c0 2024-01-24T23:26:11.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 10 monclient: _finish_auth 0 2024-01-24T23:26:11.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:26:11.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:11.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 1 -- 172.21.15.132:0/1222696918 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2cac0017c0 con 0x7f2cb809c4c0 2024-01-24T23:26:11.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:11.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:11.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 10 monclient: dump: 2024-01-24T23:26:11.359 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:11.359 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:11.360 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:11.360 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:11.360 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:11.360 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:11.360 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:11.360 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:11.360 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:11.360 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:11.360 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:11.360 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:11.360 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:11.360 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:11.360 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:11.360 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:11.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 5 monclient: authenticate success, global_id 6071 2024-01-24T23:26:11.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:11.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 1 -- 172.21.15.132:0/1222696918 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2cac001ab0 con 0x7f2cb809c4c0 2024-01-24T23:26:11.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:11.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 1 -- 172.21.15.132:0/1222696918 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2cac004bf0 con 0x7f2cb809c4c0 2024-01-24T23:26:11.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:11.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 4 set_mon_vals no callback set 2024-01-24T23:26:11.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:11.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:11.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb6ffd640 10 monclient: dump: 2024-01-24T23:26:11.361 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:11.361 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:11.361 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:11.361 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:11.361 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:11.362 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:11.362 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:11.362 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:11.362 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:11.362 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:11.362 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:11.362 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:11.362 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:11.362 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:11.362 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:11.362 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:11.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 10 monclient: shutdown 2024-01-24T23:26:11.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:11.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 1 -- 172.21.15.132:0/1222696918 >> v1:172.21.15.132:6791/0 conn(0x7f2cb809c4c0 legacy=0x7f2cb809c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:11.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 1 -- 172.21.15.132:0/1222696918 shutdown_connections 2024-01-24T23:26:11.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 1 -- 172.21.15.132:0/1222696918 >> 172.21.15.132:0/1222696918 conn(0x7f2cb80a5a10 msgr2=0x7f2cb80a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:11.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 1 -- 172.21.15.132:0/1222696918 shutdown_connections 2024-01-24T23:26:11.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 1 -- 172.21.15.132:0/1222696918 wait complete. 2024-01-24T23:26:11.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 10 monclient: build_initial_monmap 2024-01-24T23:26:11.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 10 monclient: monmap: 2024-01-24T23:26:11.363 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:11.363 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:11.363 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:11.363 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:11.363 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:11.363 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:11.363 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:11.363 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:11.363 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:11.363 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:11.363 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:11.363 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:11.364 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:11.364 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:11.364 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:11.364 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:11.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 1 Processor -- start 2024-01-24T23:26:11.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 1 -- start start 2024-01-24T23:26:11.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 10 monclient: init 2024-01-24T23:26:11.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:11.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 10 monclient: _add_conns ranks=[3,1,7,5,6,0,4,2,8] 2024-01-24T23:26:11.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 10 monclient(hunting): picked mon.noname-g con 0x7f2cb809c4c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:11.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 10 monclient(hunting): picked mon.noname-f con 0x7f2cb809cfb0 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:11.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 10 monclient(hunting): picked mon.noname-i con 0x7f2cb809ec50 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:11.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb809c020 con 0x7f2cb809cfb0 2024-01-24T23:26:11.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb77fe640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f2cb809cfb0 legacy=0x7f2cb81d05a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:11.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb80a13b0 con 0x7f2cb809c4c0 2024-01-24T23:26:11.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb80a08b0 con 0x7f2cb809ec50 2024-01-24T23:26:11.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbc8e7640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f2cb809ec50 legacy=0x7f2cb81d3cd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:11.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:11.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cbe34a640 10 monclient(hunting): authenticate will time out at 1258.679245s 2024-01-24T23:26:11.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb7fff640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f2cb809c4c0 0x7f2cb812ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:48498/0 (socket says 172.21.15.132:48498) 2024-01-24T23:26:11.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb7fff640 1 -- 172.21.15.132:0/1824061732 learned_addr learned my addr 172.21.15.132:0/1824061732 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:11.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb4ff9640 1 -- 172.21.15.132:0/1824061732 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336543804 0 0) 0x7f2cb80a13b0 con 0x7f2cb809c4c0 2024-01-24T23:26:11.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb4ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:26:11.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb4ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:11.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:11.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb4ff9640 10 monclient(hunting): my global_id is 6213 2024-01-24T23:26:11.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb4ff9640 1 -- 172.21.15.132:0/1824061732 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ca4003530 con 0x7f2cb809c4c0 2024-01-24T23:26:11.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb4ff9640 1 -- 172.21.15.132:0/1824061732 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 616528711 0 0) 0x7f2ca4003530 con 0x7f2cb809c4c0 2024-01-24T23:26:11.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb4ff9640 1 -- 172.21.15.132:0/1824061732 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2cb80a13b0 con 0x7f2cb809c4c0 2024-01-24T23:26:11.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb4ff9640 1 -- 172.21.15.132:0/1824061732 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2cac004b30 con 0x7f2cb809c4c0 2024-01-24T23:26:11.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:11.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb4ff9640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:11.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb4ff9640 10 monclient(hunting): dump: 2024-01-24T23:26:11.366 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:11.366 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:11.366 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:11.366 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:11.366 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:11.366 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:11.366 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:11.366 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:11.366 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:11.366 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:11.367 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:11.367 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:11.367 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:11.367 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:11.367 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:11.367 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:11.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb4ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:11.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.351+0000 7f2cb4ff9640 1 -- 172.21.15.132:0/1824061732 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 429935638 0 0) 0x7f2cb80a13b0 con 0x7f2cb809c4c0 2024-01-24T23:26:11.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cb4ff9640 1 -- 172.21.15.132:0/1824061732 >> v1:172.21.15.132:6792/0 conn(0x7f2cb809ec50 legacy=0x7f2cb81d3cd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:11.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cb4ff9640 1 -- 172.21.15.132:0/1824061732 >> v1:172.21.15.132:6789/0 conn(0x7f2cb809cfb0 legacy=0x7f2cb81d05a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:11.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cb4ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:26:11.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cb4ff9640 1 monclient: found mon.g 2024-01-24T23:26:11.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cb4ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:11.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cb4ff9640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:11.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cb4ff9640 1 -- 172.21.15.132:0/1824061732 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb8134b00 con 0x7f2cb809c4c0 2024-01-24T23:26:11.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cb4ff9640 10 monclient: _finish_auth 0 2024-01-24T23:26:11.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cb4ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:26:11.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cb4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:11.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cbe34a640 5 monclient: authenticate success, global_id 6213 2024-01-24T23:26:11.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cbe34a640 10 monclient: _renew_subs 2024-01-24T23:26:11.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cbe34a640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:11.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cbe34a640 1 -- 172.21.15.132:0/1824061732 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb811d000 con 0x7f2cb809c4c0 2024-01-24T23:26:11.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cbe34a640 10 monclient: _renew_subs 2024-01-24T23:26:11.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cbe34a640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:11.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cbe34a640 1 -- 172.21.15.132:0/1824061732 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb812f410 con 0x7f2cb809c4c0 2024-01-24T23:26:11.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cb4ff9640 1 -- 172.21.15.132:0/1824061732 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2cac001c60 con 0x7f2cb809c4c0 2024-01-24T23:26:11.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cb4ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:11.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cb4ff9640 1 -- 172.21.15.132:0/1824061732 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2cac0054a0 con 0x7f2cb809c4c0 2024-01-24T23:26:11.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cb4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:11.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cb67fc640 4 set_mon_vals no callback set 2024-01-24T23:26:11.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cb4ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:11.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cb4ff9640 10 monclient: dump: 2024-01-24T23:26:11.369 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:11.369 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:11.369 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:11.369 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:11.369 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:11.369 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:11.369 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:11.369 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:11.369 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:11.369 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:11.369 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:11.369 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:11.369 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:11.370 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:11.370 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:11.370 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:11.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cbe34a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:11.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cbe34a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:11.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cbe34a640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:11.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cbe34a640 1 -- 172.21.15.132:0/1824061732 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c98005090 con 0x7f2cb809c4c0 2024-01-24T23:26:11.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cb4ff9640 1 -- 172.21.15.132:0/1824061732 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f2cac00ff20 con 0x7f2cb809c4c0 2024-01-24T23:26:11.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cb4ff9640 1 -- 172.21.15.132:0/1824061732 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(155..155 src has 1..155) v4 ==== 9038+0+0 (unknown 1977377456 0 0) 0x7f2cac044f90 con 0x7f2cb809c4c0 2024-01-24T23:26:11.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.355+0000 7f2cb77fe640 10 monclient: get_auth_request con 0x7f2ca4036f40 auth_method 0 2024-01-24T23:26:11.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.359+0000 7f2cb4ff9640 1 -- 172.21.15.132:0/1824061732 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f2cac014bf0 con 0x7f2cb809c4c0 2024-01-24T23:26:11.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.359+0000 7f2cb4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:11.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.359+0000 7f2cb4ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:11.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.519+0000 7f2cbe34a640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:11.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.519+0000 7f2cbe34a640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:11.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.519+0000 7f2cbe34a640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:11.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.519+0000 7f2cbe34a640 1 -- 172.21.15.132:0/1824061732 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f2c98005340 con 0x7f2cb809c4c0 2024-01-24T23:26:11.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.519+0000 7f2cb4ff9640 1 -- 172.21.15.132:0/1824061732 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v155) v1 ==== 83+0+2378 (unknown 2279067650 0 2855864750) 0x7f2cac013290 con 0x7f2cb809c4c0 2024-01-24T23:26:11.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.519+0000 7f2cb4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:11.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.519+0000 7f2cb4ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:26:11.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.615+0000 7f2cbe34a640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:11.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.615+0000 7f2cbe34a640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:11.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.615+0000 7f2cbe34a640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:11.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.615+0000 7f2cbe34a640 1 -- 172.21.15.132:0/1824061732 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f2c98006590 con 0x7f2cb809c4c0 2024-01-24T23:26:11.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.615+0000 7f2cb4ff9640 1 -- 172.21.15.132:0/1824061732 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v155) v1 ==== 83+0+2378 (unknown 2279067650 0 2855864750) 0x7f2cac014a10 con 0x7f2cb809c4c0 2024-01-24T23:26:11.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.615+0000 7f2cb4ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:11.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.615+0000 7f2cb4ff9640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:26:11.620 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.619+0000 7f2cbe34a640 1 -- 172.21.15.132:0/1824061732 >> v1:172.21.15.132:6812/1835401922 conn(0x7f2ca4036f40 legacy=0x7f2ca4039400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:11.620 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.619+0000 7f2cbe34a640 10 monclient: shutdown 2024-01-24T23:26:11.620 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.619+0000 7f2cbe34a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:11.620 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.619+0000 7f2cbe34a640 1 -- 172.21.15.132:0/1824061732 >> v1:172.21.15.132:6790/0 conn(0x7f2cb809c4c0 legacy=0x7f2cb812ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:11.620 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.619+0000 7f2cbe34a640 1 -- 172.21.15.132:0/1824061732 shutdown_connections 2024-01-24T23:26:11.621 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.619+0000 7f2cbe34a640 1 -- 172.21.15.132:0/1824061732 >> 172.21.15.132:0/1824061732 conn(0x7f2cb80a5a10 msgr2=0x7f2cb80a3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:11.621 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.619+0000 7f2cbe34a640 1 -- 172.21.15.132:0/1824061732 shutdown_connections 2024-01-24T23:26:11.621 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.619+0000 7f2cbe34a640 1 -- 172.21.15.132:0/1824061732 wait complete. 2024-01-24T23:26:11.634 INFO:tasks.workunit.client.0.smithi132.stdout: 0 globster 0.08789 10.50000 11.50000 12.50000 osd.0 2024-01-24T23:26:11.634 INFO:tasks.workunit.client.0.smithi132.stdout: 0 globster 0.08789 10.50000 11.50000 12.50000 osd.0 2024-01-24T23:26:11.635 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep 'osd\.0' 2024-01-24T23:26:11.635 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep globster 2024-01-24T23:26:11.635 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush tree --show-shadow 2024-01-24T23:26:11.635 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep '11\.' 2024-01-24T23:26:11.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.747+0000 7f330ae9f640 10 monclient: get_monmap_and_config 2024-01-24T23:26:11.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.747+0000 7f330ae9f640 10 monclient: build_initial_monmap 2024-01-24T23:26:11.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 10 monclient: monmap: 2024-01-24T23:26:11.751 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:11.751 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:11.752 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:11.752802+0000 2024-01-24T23:26:11.752 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:11.752802+0000 2024-01-24T23:26:11.752 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:11.752 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:11.752 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:11.752 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:11.752 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:11.752 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:11.752 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:11.752 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:11.752 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:11.752 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:11.752 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:11.753 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:11.759 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 1 Processor -- start 2024-01-24T23:26:11.759 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 1 -- start start 2024-01-24T23:26:11.759 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 10 monclient: init 2024-01-24T23:26:11.759 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:11.759 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 10 monclient: _add_conns ranks=[8,1,0,7,5,3,4,2,6] 2024-01-24T23:26:11.759 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 10 monclient(hunting): picked mon.noname-e con 0x7f3304133e10 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:11.759 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 10 monclient(hunting): picked mon.noname-f con 0x7f330409c370 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:11.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 10 monclient(hunting): picked mon.noname-a con 0x7f330409a9d0 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:11.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3304064700 con 0x7f330409a9d0 2024-01-24T23:26:11.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f330406a560 con 0x7f330409c370 2024-01-24T23:26:11.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33040a2800 con 0x7f3304133e10 2024-01-24T23:26:11.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:11.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 10 monclient(hunting): authenticate will time out at 1259.077730s 2024-01-24T23:26:11.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f3303fff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f330409c370 legacy=0x7f33040a20f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:11.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330943c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f330409a9d0 0x7f330413fd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:45064/0 (socket says 172.21.15.132:45064) 2024-01-24T23:26:11.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330943c640 1 -- 172.21.15.132:0/4127240872 learned_addr learned my addr 172.21.15.132:0/4127240872 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:11.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 1 -- 172.21.15.132:0/4127240872 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3650828596 0 0) 0x7f3304064700 con 0x7f330409a9d0 2024-01-24T23:26:11.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 10 monclient(hunting): _negotiate 2024-01-24T23:26:11.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:11.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:11.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 10 monclient(hunting): my global_id is 94518 2024-01-24T23:26:11.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 1 -- 172.21.15.132:0/4127240872 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32ec003530 con 0x7f330409a9d0 2024-01-24T23:26:11.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 1 -- 172.21.15.132:0/4127240872 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 160898007 0 0) 0x7f33040a2800 con 0x7f3304133e10 2024-01-24T23:26:11.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 10 monclient(hunting): _negotiate 2024-01-24T23:26:11.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:11.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:11.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 10 monclient(hunting): my global_id is 85409 2024-01-24T23:26:11.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 1 -- 172.21.15.132:0/4127240872 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3304064700 con 0x7f3304133e10 2024-01-24T23:26:11.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 1 -- 172.21.15.132:0/4127240872 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4180502831 0 0) 0x7f32ec003530 con 0x7f330409a9d0 2024-01-24T23:26:11.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 1 -- 172.21.15.132:0/4127240872 >> v1:172.21.15.115:6793/0 conn(0x7f3304133e10 legacy=0x7f33040a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:11.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 1 -- 172.21.15.132:0/4127240872 >> v1:172.21.15.132:6789/0 conn(0x7f330409c370 legacy=0x7f33040a20f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:11.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 10 monclient: _finish_hunting 0 2024-01-24T23:26:11.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 1 monclient: found mon.noname-a 2024-01-24T23:26:11.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:11.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:26:11.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 1 -- 172.21.15.132:0/4127240872 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330409b0f0 con 0x7f330409a9d0 2024-01-24T23:26:11.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 10 monclient: _finish_auth 0 2024-01-24T23:26:11.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 10 monclient: _check_auth_tickets 2024-01-24T23:26:11.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:11.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 1 -- 172.21.15.132:0/4127240872 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f32fc001b40 con 0x7f330409a9d0 2024-01-24T23:26:11.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:11.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:26:11.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 10 monclient: dump: 2024-01-24T23:26:11.762 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:11.762 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:11.762 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:11.762 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:11.762 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:11.762 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:11.762 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:11.763 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:11.763 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:11.763 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:11.763 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:11.763 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:11.763 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:11.763 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:11.763 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:11.763 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:11.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 5 monclient: authenticate success, global_id 94518 2024-01-24T23:26:11.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:11.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 1 -- 172.21.15.132:0/4127240872 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f32fc001e20 con 0x7f330409a9d0 2024-01-24T23:26:11.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:11.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 1 -- 172.21.15.132:0/4127240872 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f32fc004f70 con 0x7f330409a9d0 2024-01-24T23:26:11.763 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:11.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:26:11.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f33037fe640 10 monclient: dump: 2024-01-24T23:26:11.764 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:11.764 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:11.764 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:11.764 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:11.764 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:11.764 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:11.764 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:11.764 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:11.764 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:11.764 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:11.764 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:11.764 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:11.765 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:11.765 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:11.765 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:11.765 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:11.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:11.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 4 set_mon_vals no callback set 2024-01-24T23:26:11.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 10 monclient: shutdown 2024-01-24T23:26:11.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:11.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 1 -- 172.21.15.132:0/4127240872 >> v1:172.21.15.115:6789/0 conn(0x7f330409a9d0 legacy=0x7f330413fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:11.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 1 -- 172.21.15.132:0/4127240872 shutdown_connections 2024-01-24T23:26:11.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 1 -- 172.21.15.132:0/4127240872 >> 172.21.15.132:0/4127240872 conn(0x7f33040a66f0 msgr2=0x7f33040a2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:11.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 1 -- 172.21.15.132:0/4127240872 shutdown_connections 2024-01-24T23:26:11.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.751+0000 7f330ae9f640 1 -- 172.21.15.132:0/4127240872 wait complete. 2024-01-24T23:26:11.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 10 monclient: build_initial_monmap 2024-01-24T23:26:11.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 10 monclient: monmap: 2024-01-24T23:26:11.765 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:11.766 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:11.766 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:11.766 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:11.766 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:11.766 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:11.766 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:11.766 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:11.766 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:11.766 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:11.766 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:11.766 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:11.766 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:11.766 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:11.766 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:11.766 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:11.766 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 1 Processor -- start 2024-01-24T23:26:11.767 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 1 -- start start 2024-01-24T23:26:11.767 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 10 monclient: init 2024-01-24T23:26:11.767 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:11.767 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 10 monclient: _add_conns ranks=[3,7,6,8,2,5,1,4,0] 2024-01-24T23:26:11.767 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 10 monclient(hunting): picked mon.noname-g con 0x7f330409a9d0 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:11.767 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 10 monclient(hunting): picked mon.noname-i con 0x7f330409c370 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:11.767 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 10 monclient(hunting): picked mon.noname-d con 0x7f3304133e10 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:11.767 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33041cc0c0 con 0x7f330409a9d0 2024-01-24T23:26:11.767 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33041205c0 con 0x7f3304133e10 2024-01-24T23:26:11.767 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f330409ca90 con 0x7f330409c370 2024-01-24T23:26:11.767 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:11.767 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 10 monclient(hunting): authenticate will time out at 1259.081358s 2024-01-24T23:26:11.767 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f3303fff640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f330409c370 legacy=0x7f33041d0480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:11.767 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f3308c3b640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f330409a9d0 0x7f3304132c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:48500/0 (socket says 172.21.15.132:48500) 2024-01-24T23:26:11.767 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f3308c3b640 1 -- 172.21.15.132:0/1113521290 learned_addr learned my addr 172.21.15.132:0/1113521290 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:11.767 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 1 -- 172.21.15.132:0/1113521290 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228054849 0 0) 0x7f33041cc0c0 con 0x7f330409a9d0 2024-01-24T23:26:11.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 10 monclient(hunting): _negotiate 2024-01-24T23:26:11.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:11.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:11.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 10 monclient(hunting): my global_id is 6222 2024-01-24T23:26:11.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 1 -- 172.21.15.132:0/1113521290 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32f0003530 con 0x7f330409a9d0 2024-01-24T23:26:11.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 1 -- 172.21.15.132:0/1113521290 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3011590244 0 0) 0x7f32f0003530 con 0x7f330409a9d0 2024-01-24T23:26:11.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 1 -- 172.21.15.132:0/1113521290 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f33041cc0c0 con 0x7f330409a9d0 2024-01-24T23:26:11.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 1 -- 172.21.15.132:0/1113521290 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f32f4001a00 con 0x7f330409a9d0 2024-01-24T23:26:11.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:11.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:11.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 10 monclient(hunting): dump: 2024-01-24T23:26:11.768 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:11.768 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:11.768 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:11.768 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:11.769 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:11.769 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:11.769 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:11.769 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:11.769 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:11.769 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:11.769 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:11.769 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:11.769 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:11.769 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:11.769 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:11.769 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:11.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:11.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 1 -- 172.21.15.132:0/1113521290 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2618555865 0 0) 0x7f33041205c0 con 0x7f3304133e10 2024-01-24T23:26:11.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 10 monclient(hunting): _negotiate 2024-01-24T23:26:11.769 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:11.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:11.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 10 monclient(hunting): my global_id is 6315 2024-01-24T23:26:11.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 1 -- 172.21.15.132:0/1113521290 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32f0003530 con 0x7f3304133e10 2024-01-24T23:26:11.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 1 -- 172.21.15.132:0/1113521290 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1895262680 0 0) 0x7f33041cc0c0 con 0x7f330409a9d0 2024-01-24T23:26:11.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 1 -- 172.21.15.132:0/1113521290 >> v1:172.21.15.132:6792/0 conn(0x7f330409c370 legacy=0x7f33041d0480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:11.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 1 -- 172.21.15.132:0/1113521290 >> v1:172.21.15.115:6792/0 conn(0x7f3304133e10 legacy=0x7f33041d3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:11.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 10 monclient: _finish_hunting 0 2024-01-24T23:26:11.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 1 monclient: found mon.g 2024-01-24T23:26:11.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:11.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:11.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 1 -- 172.21.15.132:0/1113521290 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330406af10 con 0x7f330409a9d0 2024-01-24T23:26:11.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 10 monclient: _finish_auth 0 2024-01-24T23:26:11.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 10 monclient: _check_auth_tickets 2024-01-24T23:26:11.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:11.770 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 5 monclient: authenticate success, global_id 6222 2024-01-24T23:26:11.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 10 monclient: _renew_subs 2024-01-24T23:26:11.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:11.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 1 -- 172.21.15.132:0/1113521290 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33041202e0 con 0x7f330409a9d0 2024-01-24T23:26:11.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 10 monclient: _renew_subs 2024-01-24T23:26:11.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:11.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 1 -- 172.21.15.132:0/1113521290 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3304133310 con 0x7f330409a9d0 2024-01-24T23:26:11.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 1 -- 172.21.15.132:0/1113521290 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f32f4003950 con 0x7f330409a9d0 2024-01-24T23:26:11.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:11.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 1 -- 172.21.15.132:0/1113521290 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f32f4003e80 con 0x7f330409a9d0 2024-01-24T23:26:11.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:11.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:11.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f3302ffd640 4 set_mon_vals no callback set 2024-01-24T23:26:11.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 10 monclient: dump: 2024-01-24T23:26:11.771 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:11.771 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:11.771 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:11.772 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:11.772 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:11.772 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:11.772 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:11.772 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:11.772 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:11.772 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:11.772 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:11.772 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:11.772 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:11.772 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:11.772 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:11.772 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:11.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:11.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:11.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:11.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f330ae9f640 1 -- 172.21.15.132:0/1113521290 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e4005090 con 0x7f330409a9d0 2024-01-24T23:26:11.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 1 -- 172.21.15.132:0/1113521290 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f32f4004060 con 0x7f330409a9d0 2024-01-24T23:26:11.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f33017fa640 1 -- 172.21.15.132:0/1113521290 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(155..155 src has 1..155) v4 ==== 9038+0+0 (unknown 1977377456 0 0) 0x7f32f4044d70 con 0x7f330409a9d0 2024-01-24T23:26:11.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.755+0000 7f3303fff640 10 monclient: get_auth_request con 0x7f32f0036f70 auth_method 0 2024-01-24T23:26:11.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.759+0000 7f33017fa640 1 -- 172.21.15.132:0/1113521290 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f32f4014ba0 con 0x7f330409a9d0 2024-01-24T23:26:11.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.759+0000 7f33017fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:11.773 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.759+0000 7f33017fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:11.934 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.931+0000 7f330ae9f640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:11.934 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.931+0000 7f330ae9f640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:11.934 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.931+0000 7f330ae9f640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:11.934 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.931+0000 7f330ae9f640 1 -- 172.21.15.132:0/1113521290 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f32e4005340 con 0x7f330409a9d0 2024-01-24T23:26:11.934 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.931+0000 7f33017fa640 1 -- 172.21.15.132:0/1113521290 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v155) v1 ==== 83+0+2378 (unknown 2279067650 0 2855864750) 0x7f32f4013010 con 0x7f330409a9d0 2024-01-24T23:26:11.935 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.931+0000 7f33017fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:11.935 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:11.931+0000 7f33017fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:26:12.033 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.031+0000 7f330ae9f640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:12.033 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.031+0000 7f330ae9f640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:12.033 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.031+0000 7f330ae9f640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:12.033 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.031+0000 7f330ae9f640 1 -- 172.21.15.132:0/1113521290 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f32e4006590 con 0x7f330409a9d0 2024-01-24T23:26:12.034 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.031+0000 7f33017fa640 1 -- 172.21.15.132:0/1113521290 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v155) v1 ==== 83+0+2378 (unknown 2279067650 0 2855864750) 0x7f32f4014790 con 0x7f330409a9d0 2024-01-24T23:26:12.034 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.031+0000 7f33017fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:12.034 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.031+0000 7f33017fa640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:26:12.036 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.035+0000 7f330ae9f640 1 -- 172.21.15.132:0/1113521290 >> v1:172.21.15.132:6812/1835401922 conn(0x7f32f0036f70 legacy=0x7f32f0039430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:12.036 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.035+0000 7f330ae9f640 10 monclient: shutdown 2024-01-24T23:26:12.036 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.035+0000 7f330ae9f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:12.036 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.035+0000 7f330ae9f640 1 -- 172.21.15.132:0/1113521290 >> v1:172.21.15.132:6790/0 conn(0x7f330409a9d0 legacy=0x7f3304132c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:12.037 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.035+0000 7f330ae9f640 1 -- 172.21.15.132:0/1113521290 shutdown_connections 2024-01-24T23:26:12.037 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.035+0000 7f330ae9f640 1 -- 172.21.15.132:0/1113521290 >> 172.21.15.132:0/1113521290 conn(0x7f33040a66f0 msgr2=0x7f33041341f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:12.037 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.035+0000 7f330ae9f640 1 -- 172.21.15.132:0/1113521290 shutdown_connections 2024-01-24T23:26:12.037 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.035+0000 7f330ae9f640 1 -- 172.21.15.132:0/1113521290 wait complete. 2024-01-24T23:26:12.050 INFO:tasks.workunit.client.0.smithi132.stdout: 0 globster 0.08789 10.50000 11.50000 12.50000 osd.0 2024-01-24T23:26:12.051 INFO:tasks.workunit.client.0.smithi132.stdout: 0 globster 0.08789 10.50000 11.50000 12.50000 osd.0 2024-01-24T23:26:12.051 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep 'osd\.0' 2024-01-24T23:26:12.051 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep globster 2024-01-24T23:26:12.051 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush tree --show-shadow 2024-01-24T23:26:12.051 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep '12\.' 2024-01-24T23:26:12.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 10 monclient: get_monmap_and_config 2024-01-24T23:26:12.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 10 monclient: build_initial_monmap 2024-01-24T23:26:12.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 10 monclient: monmap: 2024-01-24T23:26:12.156 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:12.156 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:12.157 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:12.157828+0000 2024-01-24T23:26:12.157 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:12.157828+0000 2024-01-24T23:26:12.157 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:12.157 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:12.157 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:12.157 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:12.157 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:12.157 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:12.157 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:12.157 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:12.157 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:12.157 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:12.157 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:12.158 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:12.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 1 Processor -- start 2024-01-24T23:26:12.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 1 -- start start 2024-01-24T23:26:12.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 10 monclient: init 2024-01-24T23:26:12.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:12.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 10 monclient: _add_conns ranks=[3,4,0,1,7,6,5,8,2] 2024-01-24T23:26:12.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 10 monclient(hunting): picked mon.noname-g con 0x7f42140a1b50 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:12.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 10 monclient(hunting): picked mon.noname-c con 0x7f42140a1f30 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:12.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 10 monclient(hunting): picked mon.noname-a con 0x7f421413b030 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:12.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4214064700 con 0x7f421413b030 2024-01-24T23:26:12.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4214070ff0 con 0x7f42140a1b50 2024-01-24T23:26:12.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f421406f7d0 con 0x7f42140a1f30 2024-01-24T23:26:12.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:12.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 10 monclient(hunting): authenticate will time out at 1259.482542s 2024-01-24T23:26:12.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421259c640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f42140a1b50 0x7f4214071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:48514/0 (socket says 172.21.15.132:48514) 2024-01-24T23:26:12.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421259c640 1 -- 172.21.15.132:0/828606348 learned_addr learned my addr 172.21.15.132:0/828606348 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:12.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 1 -- 172.21.15.132:0/828606348 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1004237872 0 0) 0x7f4214070ff0 con 0x7f42140a1b50 2024-01-24T23:26:12.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 10 monclient(hunting): _negotiate 2024-01-24T23:26:12.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:12.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:12.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 10 monclient(hunting): my global_id is 6231 2024-01-24T23:26:12.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 1 -- 172.21.15.132:0/828606348 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4200003530 con 0x7f42140a1b50 2024-01-24T23:26:12.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 1 -- 172.21.15.132:0/828606348 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2624640506 0 0) 0x7f4200003530 con 0x7f42140a1b50 2024-01-24T23:26:12.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 1 -- 172.21.15.132:0/828606348 >> v1:172.21.15.115:6791/0 conn(0x7f42140a1f30 legacy=0x7f42140a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:12.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 1 -- 172.21.15.132:0/828606348 >> v1:172.21.15.115:6789/0 conn(0x7f421413b030 legacy=0x7f421413d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:12.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 10 monclient: _finish_hunting 0 2024-01-24T23:26:12.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 1 monclient: found mon.noname-g 2024-01-24T23:26:12.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:12.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:26:12.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 1 -- 172.21.15.132:0/828606348 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42140a36b0 con 0x7f42140a1b50 2024-01-24T23:26:12.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 10 monclient: _finish_auth 0 2024-01-24T23:26:12.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 10 monclient: _check_auth_tickets 2024-01-24T23:26:12.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:12.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 1 -- 172.21.15.132:0/828606348 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f420c001b40 con 0x7f42140a1b50 2024-01-24T23:26:12.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 5 monclient: authenticate success, global_id 6231 2024-01-24T23:26:12.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:12.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:12.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:12.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 10 monclient: dump: 2024-01-24T23:26:12.167 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:12.167 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:12.167 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:12.168 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:12.168 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:12.168 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:12.168 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:12.168 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:12.168 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:12.168 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:12.168 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:12.168 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:12.168 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:12.168 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:12.168 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:12.168 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:12.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:12.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 1 -- 172.21.15.132:0/828606348 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f420c001e30 con 0x7f42140a1b50 2024-01-24T23:26:12.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:12.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:12.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 4 set_mon_vals no callback set 2024-01-24T23:26:12.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 1 -- 172.21.15.132:0/828606348 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f420c004f70 con 0x7f42140a1b50 2024-01-24T23:26:12.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:12.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:12.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f421159a640 10 monclient: dump: 2024-01-24T23:26:12.169 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:12.169 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:12.169 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:12.169 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:12.169 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:12.169 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:12.169 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:12.170 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:12.170 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:12.170 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:12.170 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:12.170 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:12.170 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:12.170 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:12.170 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:12.170 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:12.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 10 monclient: shutdown 2024-01-24T23:26:12.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:12.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 1 -- 172.21.15.132:0/828606348 >> v1:172.21.15.132:6790/0 conn(0x7f42140a1b50 legacy=0x7f4214071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:12.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 1 -- 172.21.15.132:0/828606348 shutdown_connections 2024-01-24T23:26:12.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 1 -- 172.21.15.132:0/828606348 >> 172.21.15.132:0/828606348 conn(0x7f42140a7150 msgr2=0x7f42140a7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:12.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 1 -- 172.21.15.132:0/828606348 shutdown_connections 2024-01-24T23:26:12.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.155+0000 7f42189ee640 1 -- 172.21.15.132:0/828606348 wait complete. 2024-01-24T23:26:12.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 10 monclient: build_initial_monmap 2024-01-24T23:26:12.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 10 monclient: monmap: 2024-01-24T23:26:12.171 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:12.171 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:12.171 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:12.171 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:12.171 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:12.171 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:12.171 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:12.171 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:12.171 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:12.171 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:12.171 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:12.172 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:12.172 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:12.172 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:12.172 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:12.172 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:12.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 1 Processor -- start 2024-01-24T23:26:12.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 1 -- start start 2024-01-24T23:26:12.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 10 monclient: init 2024-01-24T23:26:12.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:12.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 10 monclient: _add_conns ranks=[2,0,8,4,7,6,5,3,1] 2024-01-24T23:26:12.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 10 monclient(hunting): picked mon.noname-b con 0x7f42140a1b50 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:12.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 10 monclient(hunting): picked mon.noname-a con 0x7f42140a1f30 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:12.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 10 monclient(hunting): picked mon.noname-e con 0x7f421413b030 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:12.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4214120080 con 0x7f42140a1f30 2024-01-24T23:26:12.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f421411f7a0 con 0x7f42140a1b50 2024-01-24T23:26:12.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4214072110 con 0x7f421413b030 2024-01-24T23:26:12.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:12.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 10 monclient(hunting): authenticate will time out at 1259.485402s 2024-01-24T23:26:12.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f421259c640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f42140a1b50 legacy=0x7f4214120980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:12.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f4211d9b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f42140a1f30 0x7f421411f090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:45094/0 (socket says 172.21.15.132:45094) 2024-01-24T23:26:12.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f4211d9b640 1 -- 172.21.15.132:0/3716264776 learned_addr learned my addr 172.21.15.132:0/3716264776 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:12.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 1 -- 172.21.15.132:0/3716264776 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4013990954 0 0) 0x7f4214120080 con 0x7f42140a1f30 2024-01-24T23:26:12.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:12.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:12.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:12.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 10 monclient(hunting): my global_id is 94527 2024-01-24T23:26:12.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 1 -- 172.21.15.132:0/3716264776 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41f4003530 con 0x7f42140a1f30 2024-01-24T23:26:12.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 1 -- 172.21.15.132:0/3716264776 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3240522188 0 0) 0x7f4214072110 con 0x7f421413b030 2024-01-24T23:26:12.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:12.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:12.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:12.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 10 monclient(hunting): my global_id is 85418 2024-01-24T23:26:12.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 1 -- 172.21.15.132:0/3716264776 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4214120080 con 0x7f421413b030 2024-01-24T23:26:12.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 1 -- 172.21.15.132:0/3716264776 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1077590581 0 0) 0x7f4214120080 con 0x7f421413b030 2024-01-24T23:26:12.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 1 -- 172.21.15.132:0/3716264776 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4214072110 con 0x7f421413b030 2024-01-24T23:26:12.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 1 -- 172.21.15.132:0/3716264776 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4204001950 con 0x7f421413b030 2024-01-24T23:26:12.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:12.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:26:12.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 10 monclient(hunting): dump: 2024-01-24T23:26:12.174 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:12.174 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:12.174 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:12.174 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:12.175 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 1 -- 172.21.15.132:0/3716264776 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 976758103 0 0) 0x7f41f4003530 con 0x7f42140a1f30 2024-01-24T23:26:12.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 1 -- 172.21.15.132:0/3716264776 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4214120080 con 0x7f42140a1f30 2024-01-24T23:26:12.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 1 -- 172.21.15.132:0/3716264776 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f41fc001ae0 con 0x7f42140a1f30 2024-01-24T23:26:12.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:12.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:26:12.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 10 monclient(hunting): dump: 2024-01-24T23:26:12.176 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:12.176 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:12.176 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:12.176 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:12.176 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:12.176 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:12.176 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:12.176 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:12.176 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:12.177 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:12.177 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:12.177 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:12.177 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:12.177 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:12.177 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:12.177 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:12.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 1 -- 172.21.15.132:0/3716264776 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3492764536 0 0) 0x7f4214072110 con 0x7f421413b030 2024-01-24T23:26:12.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 1 -- 172.21.15.132:0/3716264776 >> v1:172.21.15.115:6790/0 conn(0x7f42140a1b50 legacy=0x7f4214120980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:12.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 1 -- 172.21.15.132:0/3716264776 >> v1:172.21.15.115:6789/0 conn(0x7f42140a1f30 legacy=0x7f421411f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:12.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 10 monclient: _finish_hunting 0 2024-01-24T23:26:12.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 1 monclient: found mon.e 2024-01-24T23:26:12.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:12.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:12.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 1 -- 172.21.15.132:0/3716264776 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4214070270 con 0x7f421413b030 2024-01-24T23:26:12.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 10 monclient: _finish_auth 0 2024-01-24T23:26:12.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 10 monclient: _check_auth_tickets 2024-01-24T23:26:12.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:12.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 5 monclient: authenticate success, global_id 85418 2024-01-24T23:26:12.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 10 monclient: _renew_subs 2024-01-24T23:26:12.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:12.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 1 -- 172.21.15.132:0/3716264776 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4214121ea0 con 0x7f421413b030 2024-01-24T23:26:12.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 10 monclient: _renew_subs 2024-01-24T23:26:12.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:12.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 1 -- 172.21.15.132:0/3716264776 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f421411e830 con 0x7f421413b030 2024-01-24T23:26:12.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 1 -- 172.21.15.132:0/3716264776 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4204001470 con 0x7f421413b030 2024-01-24T23:26:12.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:12.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f4210d99640 4 set_mon_vals no callback set 2024-01-24T23:26:12.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 1 -- 172.21.15.132:0/3716264776 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f42040039e0 con 0x7f421413b030 2024-01-24T23:26:12.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:12.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:26:12.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f420affd640 10 monclient: dump: 2024-01-24T23:26:12.179 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:12.179 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:12.179 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:12.179 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:12.179 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:12.179 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:12.179 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:12.179 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:12.179 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:12.179 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:12.179 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:12.179 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:12.179 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:12.180 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:12.180 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:12.180 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:12.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:12.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:12.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:12.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.159+0000 7f42189ee640 1 -- 172.21.15.132:0/3716264776 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f8005090 con 0x7f421413b030 2024-01-24T23:26:12.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.163+0000 7f420affd640 1 -- 172.21.15.132:0/3716264776 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f4204003bc0 con 0x7f421413b030 2024-01-24T23:26:12.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.163+0000 7f420affd640 1 -- 172.21.15.132:0/3716264776 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(155..155 src has 1..155) v4 ==== 9038+0+0 (unknown 1977377456 0 0) 0x7f4204043710 con 0x7f421413b030 2024-01-24T23:26:12.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.163+0000 7f421259c640 10 monclient: get_auth_request con 0x7f41f40376e0 auth_method 0 2024-01-24T23:26:12.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.175+0000 7f420affd640 1 -- 172.21.15.132:0/3716264776 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f4204013590 con 0x7f421413b030 2024-01-24T23:26:12.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.175+0000 7f420affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:12.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.175+0000 7f420affd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:12.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.335+0000 7f42189ee640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:12.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.335+0000 7f42189ee640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:12.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.335+0000 7f42189ee640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:12.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.335+0000 7f42189ee640 1 -- 172.21.15.132:0/3716264776 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f41f8005340 con 0x7f421413b030 2024-01-24T23:26:12.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.335+0000 7f420affd640 1 -- 172.21.15.132:0/3716264776 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v155) v1 ==== 83+0+2378 (unknown 2279067650 0 2855864750) 0x7f4204013180 con 0x7f421413b030 2024-01-24T23:26:12.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.335+0000 7f420affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:12.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.335+0000 7f420affd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:26:12.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.427+0000 7f42189ee640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:12.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.427+0000 7f42189ee640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:12.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.427+0000 7f42189ee640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:12.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.427+0000 7f42189ee640 1 -- 172.21.15.132:0/3716264776 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f41f8005a80 con 0x7f421413b030 2024-01-24T23:26:12.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.427+0000 7f420affd640 1 -- 172.21.15.132:0/3716264776 <== mon.8 v1:172.21.15.115:6793/0 11 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v155) v1 ==== 83+0+2378 (unknown 2279067650 0 2855864750) 0x7f4204013360 con 0x7f421413b030 2024-01-24T23:26:12.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.427+0000 7f420affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:12.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.427+0000 7f420affd640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:26:12.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.431+0000 7f42189ee640 1 -- 172.21.15.132:0/3716264776 >> v1:172.21.15.132:6812/1835401922 conn(0x7f41f40376e0 legacy=0x7f41f4039ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:12.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.431+0000 7f42189ee640 10 monclient: shutdown 2024-01-24T23:26:12.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.431+0000 7f42189ee640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:12.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.431+0000 7f42189ee640 1 -- 172.21.15.132:0/3716264776 >> v1:172.21.15.115:6793/0 conn(0x7f421413b030 legacy=0x7f42141d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:12.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.431+0000 7f42189ee640 1 -- 172.21.15.132:0/3716264776 shutdown_connections 2024-01-24T23:26:12.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.431+0000 7f42189ee640 1 -- 172.21.15.132:0/3716264776 >> 172.21.15.132:0/3716264776 conn(0x7f42140a7150 msgr2=0x7f421409f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:12.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.431+0000 7f42189ee640 1 -- 172.21.15.132:0/3716264776 shutdown_connections 2024-01-24T23:26:12.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.431+0000 7f42189ee640 1 -- 172.21.15.132:0/3716264776 wait complete. 2024-01-24T23:26:12.446 INFO:tasks.workunit.client.0.smithi132.stdout: 0 globster 0.08789 10.50000 11.50000 12.50000 osd.0 2024-01-24T23:26:12.446 INFO:tasks.workunit.client.0.smithi132.stdout: 0 globster 0.08789 10.50000 11.50000 12.50000 osd.0 2024-01-24T23:26:12.446 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set reweight-compat osd.0 7.5 2024-01-24T23:26:12.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.563+0000 7fceb2139640 10 monclient: get_monmap_and_config 2024-01-24T23:26:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.563+0000 7fceb2139640 10 monclient: build_initial_monmap 2024-01-24T23:26:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.563+0000 7fceb2139640 10 monclient: monmap: 2024-01-24T23:26:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:12.566194+0000 2024-01-24T23:26:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:12.566194+0000 2024-01-24T23:26:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:12.565 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:12.566 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:12.566 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:12.566 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:12.566 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:12.566 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:12.566 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:12.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.563+0000 7fceb2139640 1 Processor -- start 2024-01-24T23:26:12.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.563+0000 7fceb2139640 1 -- start start 2024-01-24T23:26:12.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.563+0000 7fceb2139640 10 monclient: init 2024-01-24T23:26:12.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.563+0000 7fceb2139640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:12.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.563+0000 7fceb2139640 10 monclient: _add_conns ranks=[5,6,8,7,3,2,0,1,4] 2024-01-24T23:26:12.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.563+0000 7fceb2139640 10 monclient(hunting): picked mon.noname-h con 0x7fceac09af90 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:12.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.563+0000 7fceb2139640 10 monclient(hunting): picked mon.noname-d con 0x7fceac098020 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:12.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.563+0000 7fceb2139640 10 monclient(hunting): picked mon.noname-e con 0x7fceac098b50 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:12.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.563+0000 7fceb2139640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceac064700 con 0x7fceac09af90 2024-01-24T23:26:12.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.563+0000 7fceb2139640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceac09a510 con 0x7fceac098020 2024-01-24T23:26:12.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.563+0000 7fceb2139640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceac09ba30 con 0x7fceac098b50 2024-01-24T23:26:12.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.563+0000 7fceb2139640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:12.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.563+0000 7fceb2139640 10 monclient(hunting): authenticate will time out at 1259.891432s 2024-01-24T23:26:12.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.563+0000 7fceab7fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fceac09af90 0x7fceac0a0ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:35286/0 (socket says 172.21.15.132:35286) 2024-01-24T23:26:12.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.563+0000 7fceab7fe640 1 -- 172.21.15.132:0/1039927360 learned_addr learned my addr 172.21.15.132:0/1039927360 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:12.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.563+0000 7fceaaffd640 1 -- 172.21.15.132:0/1039927360 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1201972081 0 0) 0x7fceac064700 con 0x7fceac09af90 2024-01-24T23:26:12.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.563+0000 7fceaaffd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:12.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.563+0000 7fceaaffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:12.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.563+0000 7fceaaffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:12.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.563+0000 7fceaaffd640 10 monclient(hunting): my global_id is 6080 2024-01-24T23:26:12.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceaaffd640 1 -- 172.21.15.132:0/1039927360 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce98003530 con 0x7fceac09af90 2024-01-24T23:26:12.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceaaffd640 1 -- 172.21.15.132:0/1039927360 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 918783017 0 0) 0x7fce98003530 con 0x7fceac09af90 2024-01-24T23:26:12.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceaaffd640 1 -- 172.21.15.132:0/1039927360 >> v1:172.21.15.115:6793/0 conn(0x7fceac098b50 legacy=0x7fceac13c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:12.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceaaffd640 1 -- 172.21.15.132:0/1039927360 >> v1:172.21.15.115:6792/0 conn(0x7fceac098020 legacy=0x7fceac098440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:12.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceaaffd640 10 monclient: _finish_hunting 0 2024-01-24T23:26:12.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceaaffd640 1 monclient: found mon.noname-h 2024-01-24T23:26:12.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceaaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:12.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceaaffd640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:26:12.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceaaffd640 1 -- 172.21.15.132:0/1039927360 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceac09bd10 con 0x7fceac09af90 2024-01-24T23:26:12.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceaaffd640 10 monclient: _finish_auth 0 2024-01-24T23:26:12.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceaaffd640 10 monclient: _check_auth_tickets 2024-01-24T23:26:12.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceaaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:12.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 5 monclient: authenticate success, global_id 6080 2024-01-24T23:26:12.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceaaffd640 1 -- 172.21.15.132:0/1039927360 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fce9c0017c0 con 0x7fceac09af90 2024-01-24T23:26:12.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:12.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceaaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:12.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceaaffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:12.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceaaffd640 10 monclient: dump: 2024-01-24T23:26:12.575 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:12.576 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:12.576 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:12.576 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:12.576 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:12.576 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:12.576 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:12.576 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:12.576 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:12.576 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:12.576 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:12.576 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:12.576 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:12.576 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:12.576 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:12.576 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:12.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:12.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceaaffd640 1 -- 172.21.15.132:0/1039927360 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fce9c001a70 con 0x7fceac09af90 2024-01-24T23:26:12.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceaaffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:12.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:12.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceaaffd640 1 -- 172.21.15.132:0/1039927360 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fce9c004bf0 con 0x7fceac09af90 2024-01-24T23:26:12.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 4 set_mon_vals no callback set 2024-01-24T23:26:12.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceaaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:12.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceaaffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:12.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceaaffd640 10 monclient: dump: 2024-01-24T23:26:12.577 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:12.577 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:12.577 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:12.577 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:12.577 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:12.578 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:12.578 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:12.578 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:12.578 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:12.578 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:12.578 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:12.578 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:12.578 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:12.578 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:12.578 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:12.578 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:12.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 10 monclient: shutdown 2024-01-24T23:26:12.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:12.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 1 -- 172.21.15.132:0/1039927360 >> v1:172.21.15.132:6791/0 conn(0x7fceac09af90 legacy=0x7fceac0a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:12.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 1 -- 172.21.15.132:0/1039927360 shutdown_connections 2024-01-24T23:26:12.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 1 -- 172.21.15.132:0/1039927360 >> 172.21.15.132:0/1039927360 conn(0x7fceac0a1bc0 msgr2=0x7fceac0a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:12.579 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 1 -- 172.21.15.132:0/1039927360 shutdown_connections 2024-01-24T23:26:12.579 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 1 -- 172.21.15.132:0/1039927360 wait complete. 2024-01-24T23:26:12.579 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 10 monclient: build_initial_monmap 2024-01-24T23:26:12.579 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 10 monclient: monmap: 2024-01-24T23:26:12.579 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:12.579 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:12.579 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:12.579 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:12.579 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:12.579 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:12.579 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:12.579 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:12.579 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:12.579 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:12.580 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:12.580 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:12.580 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:12.580 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:12.580 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:12.580 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:12.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 1 Processor -- start 2024-01-24T23:26:12.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 1 -- start start 2024-01-24T23:26:12.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 10 monclient: init 2024-01-24T23:26:12.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:12.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 10 monclient: _add_conns ranks=[2,0,1,7,8,4,6,3,5] 2024-01-24T23:26:12.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 10 monclient(hunting): picked mon.noname-b con 0x7fceac098020 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:12.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 10 monclient(hunting): picked mon.noname-a con 0x7fceac098b50 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:12.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 10 monclient(hunting): picked mon.noname-f con 0x7fceac09af90 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:12.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceac09b3e0 con 0x7fceac098b50 2024-01-24T23:26:12.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceac132de0 con 0x7fceac09af90 2024-01-24T23:26:12.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceac131f30 con 0x7fceac098020 2024-01-24T23:26:12.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:12.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceb2139640 10 monclient(hunting): authenticate will time out at 1259.894666s 2024-01-24T23:26:12.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceabfff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fceac09af90 legacy=0x7fceac1d3cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:12.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fceab7fe640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fceac098020 legacy=0x7fceac097360 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:12.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fcea3fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fceac098b50 0x7fceac1d0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:45108/0 (socket says 172.21.15.132:45108) 2024-01-24T23:26:12.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fcea3fff640 1 -- 172.21.15.132:0/2339385194 learned_addr learned my addr 172.21.15.132:0/2339385194 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:12.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fcea8ff9640 1 -- 172.21.15.132:0/2339385194 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 744444582 0 0) 0x7fceac09b3e0 con 0x7fceac098b50 2024-01-24T23:26:12.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fcea8ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:26:12.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fcea8ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:12.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fcea8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:12.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fcea8ff9640 10 monclient(hunting): my global_id is 94536 2024-01-24T23:26:12.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fcea8ff9640 1 -- 172.21.15.132:0/2339385194 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce8c003530 con 0x7fceac098b50 2024-01-24T23:26:12.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fcea8ff9640 1 -- 172.21.15.132:0/2339385194 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2740435959 0 0) 0x7fce8c003530 con 0x7fceac098b50 2024-01-24T23:26:12.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fcea8ff9640 1 -- 172.21.15.132:0/2339385194 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fceac09b3e0 con 0x7fceac098b50 2024-01-24T23:26:12.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fcea8ff9640 1 -- 172.21.15.132:0/2339385194 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fce94001b00 con 0x7fceac098b50 2024-01-24T23:26:12.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fcea8ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:12.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fcea8ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:26:12.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fcea8ff9640 10 monclient(hunting): dump: 2024-01-24T23:26:12.582 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:12.582 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:12.582 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:12.582 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:12.582 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:12.582 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:12.583 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:12.583 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:12.583 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:12.583 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:12.583 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:12.583 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:12.583 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:12.583 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:12.583 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:12.583 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:12.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.567+0000 7fcea8ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:12.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fcea8ff9640 1 -- 172.21.15.132:0/2339385194 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 822450804 0 0) 0x7fceac09b3e0 con 0x7fceac098b50 2024-01-24T23:26:12.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fcea8ff9640 1 -- 172.21.15.132:0/2339385194 >> v1:172.21.15.115:6790/0 conn(0x7fceac098020 legacy=0x7fceac097360 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:12.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fcea8ff9640 1 -- 172.21.15.132:0/2339385194 >> v1:172.21.15.132:6789/0 conn(0x7fceac09af90 legacy=0x7fceac1d3cb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:12.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fcea8ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:26:12.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fcea8ff9640 1 monclient: found mon.a 2024-01-24T23:26:12.584 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fcea8ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:12.584 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fcea8ff9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:26:12.584 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fcea8ff9640 1 -- 172.21.15.132:0/2339385194 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceac132820 con 0x7fceac098b50 2024-01-24T23:26:12.584 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fcea8ff9640 10 monclient: _finish_auth 0 2024-01-24T23:26:12.584 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fcea8ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:26:12.584 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fcea8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:12.584 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fceb2139640 5 monclient: authenticate success, global_id 94536 2024-01-24T23:26:12.584 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fceb2139640 10 monclient: _renew_subs 2024-01-24T23:26:12.584 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fceb2139640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:26:12.584 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fceb2139640 1 -- 172.21.15.132:0/2339385194 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceac130d60 con 0x7fceac098b50 2024-01-24T23:26:12.584 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fceb2139640 10 monclient: _renew_subs 2024-01-24T23:26:12.584 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fceb2139640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:26:12.584 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fceb2139640 1 -- 172.21.15.132:0/2339385194 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fceac1318c0 con 0x7fceac098b50 2024-01-24T23:26:12.584 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fcea8ff9640 1 -- 172.21.15.132:0/2339385194 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fce940038d0 con 0x7fceac098b50 2024-01-24T23:26:12.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fcea8ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:12.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fceaa7fc640 4 set_mon_vals no callback set 2024-01-24T23:26:12.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fcea8ff9640 1 -- 172.21.15.132:0/2339385194 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fce94003e00 con 0x7fceac098b50 2024-01-24T23:26:12.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fcea8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:12.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fcea8ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:26:12.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fcea8ff9640 10 monclient: dump: 2024-01-24T23:26:12.585 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:12.585 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:12.585 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:12.585 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:12.585 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:12.585 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:12.585 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:12.585 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:12.586 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:12.586 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:12.586 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:12.586 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:12.586 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:12.586 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:12.586 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:12.586 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:12.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fceb2139640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:12.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fceb2139640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:12.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fceb2139640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:26:12.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fceb2139640 1 -- 172.21.15.132:0/2339385194 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce90005090 con 0x7fceac098b50 2024-01-24T23:26:12.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fcea8ff9640 1 -- 172.21.15.132:0/2339385194 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fce94003fe0 con 0x7fceac098b50 2024-01-24T23:26:12.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fcea8ff9640 1 -- 172.21.15.132:0/2339385194 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(155..155 src has 1..155) v4 ==== 9038+0+0 (unknown 1977377456 0 0) 0x7fce94044e10 con 0x7fceac098b50 2024-01-24T23:26:12.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.571+0000 7fceab7fe640 10 monclient: get_auth_request con 0x7fce8c036f10 auth_method 0 2024-01-24T23:26:12.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.583+0000 7fcea8ff9640 1 -- 172.21.15.132:0/2339385194 <== mon.0 v1:172.21.15.115:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fce940115a0 con 0x7fceac098b50 2024-01-24T23:26:12.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.583+0000 7fcea8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:12.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.583+0000 7fcea8ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:12.747 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.747+0000 7fceb2139640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}] 2024-01-24T23:26:12.747 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.747+0000 7fceb2139640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}] 2024-01-24T23:26:12.748 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.747+0000 7fceb2139640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:26:12.748 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:12.747+0000 7fceb2139640 1 -- 172.21.15.132:0/2339385194 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]} v 0) v1 -- 0x7fce90005340 con 0x7fceac098b50 2024-01-24T23:26:13.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:13.223+0000 7fcea8ff9640 1 -- 172.21.15.132:0/2339385194 <== mon.0 v1:172.21.15.115:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}]=0 v156) v1 ==== 118+0+0 (unknown 919465323 0 0) 0x7fce94013050 con 0x7fceac098b50 2024-01-24T23:26:13.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:13.223+0000 7fcea8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}] 2024-01-24T23:26:13.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:13.223+0000 7fcea8ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:26:13.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:13.327+0000 7fceb2139640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}] 2024-01-24T23:26:13.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:13.327+0000 7fceb2139640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}] 2024-01-24T23:26:13.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:13.327+0000 7fceb2139640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:26:13.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:13.327+0000 7fceb2139640 1 -- 172.21.15.132:0/2339385194 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]} v 0) v1 -- 0x7fce90005a80 con 0x7fceac098b50 2024-01-24T23:26:14.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.243+0000 7fcea8ff9640 1 -- 172.21.15.132:0/2339385194 <== mon.0 v1:172.21.15.115:6789/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}]=0 v157) v1 ==== 118+0+0 (unknown 1949066829 0 0) 0x7fce940147d0 con 0x7fceac098b50 2024-01-24T23:26:14.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.243+0000 7fcea8ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}] 2024-01-24T23:26:14.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.243+0000 7fcea8ff9640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:26:14.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.247+0000 7fceb2139640 1 -- 172.21.15.132:0/2339385194 >> v1:172.21.15.132:6812/1835401922 conn(0x7fce8c036f10 legacy=0x7fce8c0393d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:14.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.247+0000 7fceb2139640 10 monclient: shutdown 2024-01-24T23:26:14.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.247+0000 7fceb2139640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:14.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.247+0000 7fceb2139640 1 -- 172.21.15.132:0/2339385194 >> v1:172.21.15.115:6789/0 conn(0x7fceac098b50 legacy=0x7fceac1d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:14.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.247+0000 7fceb2139640 1 -- 172.21.15.132:0/2339385194 shutdown_connections 2024-01-24T23:26:14.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.247+0000 7fceb2139640 1 -- 172.21.15.132:0/2339385194 >> 172.21.15.132:0/2339385194 conn(0x7fceac0a1bc0 msgr2=0x7fceac09d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:14.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.247+0000 7fceb2139640 1 -- 172.21.15.132:0/2339385194 shutdown_connections 2024-01-24T23:26:14.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.247+0000 7fceb2139640 1 -- 172.21.15.132:0/2339385194 wait complete. 2024-01-24T23:26:14.268 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set reweight cool osd.0 8.5 2024-01-24T23:26:14.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2eab71c640 10 monclient: get_monmap_and_config 2024-01-24T23:26:14.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2eab71c640 10 monclient: build_initial_monmap 2024-01-24T23:26:14.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2eab71c640 10 monclient: monmap: 2024-01-24T23:26:14.393 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:14.393 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:14.393 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:14.394554+0000 2024-01-24T23:26:14.393 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:14.394554+0000 2024-01-24T23:26:14.393 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:14.393 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:14.394 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:14.394 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:14.394 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:14.394 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:14.394 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:14.394 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:14.394 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:14.394 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:14.394 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:14.394 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:14.400 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2eab71c640 1 Processor -- start 2024-01-24T23:26:14.400 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2eab71c640 1 -- start start 2024-01-24T23:26:14.400 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2eab71c640 10 monclient: init 2024-01-24T23:26:14.400 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2eab71c640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:14.400 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2eab71c640 10 monclient: _add_conns ranks=[8,2,3,4,0,5,1,6,7] 2024-01-24T23:26:14.401 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2eab71c640 10 monclient(hunting): picked mon.noname-e con 0x7f2ea4135ea0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:14.401 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2eab71c640 10 monclient(hunting): picked mon.noname-b con 0x7f2ea409bcd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:14.401 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2eab71c640 10 monclient(hunting): picked mon.noname-g con 0x7f2ea409c0f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:14.401 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2eab71c640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea4064700 con 0x7f2ea409bcd0 2024-01-24T23:26:14.401 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2eab71c640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea4136990 con 0x7f2ea409c0f0 2024-01-24T23:26:14.401 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2eab71c640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea409b010 con 0x7f2ea4135ea0 2024-01-24T23:26:14.401 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2eab71c640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:14.401 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2eab71c640 10 monclient(hunting): authenticate will time out at 1261.719231s 2024-01-24T23:26:14.401 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2ea8cb7640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f2ea409bcd0 legacy=0x7f2ea413bed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:14.401 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2ea9cb9640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f2ea409c0f0 0x7f2ea413f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:38152/0 (socket says 172.21.15.132:38152) 2024-01-24T23:26:14.401 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2ea9cb9640 1 -- 172.21.15.132:0/3251054515 learned_addr learned my addr 172.21.15.132:0/3251054515 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:14.401 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2e9bfff640 1 -- 172.21.15.132:0/3251054515 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 873345191 0 0) 0x7f2ea4136990 con 0x7f2ea409c0f0 2024-01-24T23:26:14.401 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2e9bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:26:14.401 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2e9bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:14.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2e9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:14.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2e9bfff640 10 monclient(hunting): my global_id is 6240 2024-01-24T23:26:14.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2e9bfff640 1 -- 172.21.15.132:0/3251054515 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e90003530 con 0x7f2ea409c0f0 2024-01-24T23:26:14.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2e9bfff640 1 -- 172.21.15.132:0/3251054515 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4013846101 0 0) 0x7f2e90003530 con 0x7f2ea409c0f0 2024-01-24T23:26:14.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2e9bfff640 1 -- 172.21.15.132:0/3251054515 >> v1:172.21.15.115:6793/0 conn(0x7f2ea4135ea0 legacy=0x7f2ea4136280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:14.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2e9bfff640 1 -- 172.21.15.132:0/3251054515 >> v1:172.21.15.115:6790/0 conn(0x7f2ea409bcd0 legacy=0x7f2ea413bed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:14.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2e9bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:26:14.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2e9bfff640 1 monclient: found mon.noname-g 2024-01-24T23:26:14.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2e9bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:14.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2e9bfff640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:26:14.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2e9bfff640 1 -- 172.21.15.132:0/3251054515 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea40a4f50 con 0x7f2ea409c0f0 2024-01-24T23:26:14.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2e9bfff640 10 monclient: _finish_auth 0 2024-01-24T23:26:14.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2e9bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:26:14.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.391+0000 7f2e9bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:14.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e9bfff640 1 -- 172.21.15.132:0/3251054515 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2e9c001b40 con 0x7f2ea409c0f0 2024-01-24T23:26:14.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 5 monclient: authenticate success, global_id 6240 2024-01-24T23:26:14.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:14.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e9bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:14.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e9bfff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:14.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e9bfff640 10 monclient: dump: 2024-01-24T23:26:14.403 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:14.403 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:14.403 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:14.403 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:14.403 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:14.403 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:14.403 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:14.403 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:14.403 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:14.403 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:14.404 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:14.404 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:14.404 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:14.404 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:14.404 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:14.404 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:14.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:14.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e9bfff640 1 -- 172.21.15.132:0/3251054515 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2e9c001e60 con 0x7f2ea409c0f0 2024-01-24T23:26:14.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e9bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:14.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:14.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 4 set_mon_vals no callback set 2024-01-24T23:26:14.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e9bfff640 1 -- 172.21.15.132:0/3251054515 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2e9c004f70 con 0x7f2ea409c0f0 2024-01-24T23:26:14.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e9bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:14.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e9bfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:14.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e9bfff640 10 monclient: dump: 2024-01-24T23:26:14.405 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:14.405 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:14.405 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:14.405 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:14.405 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:14.405 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:14.405 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:14.405 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:14.405 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:14.405 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:14.405 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:14.405 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:14.405 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:14.405 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:14.405 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:14.406 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:14.406 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 10 monclient: shutdown 2024-01-24T23:26:14.406 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:14.406 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 1 -- 172.21.15.132:0/3251054515 >> v1:172.21.15.132:6790/0 conn(0x7f2ea409c0f0 legacy=0x7f2ea413f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:14.406 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 1 -- 172.21.15.132:0/3251054515 shutdown_connections 2024-01-24T23:26:14.406 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 1 -- 172.21.15.132:0/3251054515 >> 172.21.15.132:0/3251054515 conn(0x7f2ea40a9bf0 msgr2=0x7f2ea40a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:14.406 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 1 -- 172.21.15.132:0/3251054515 shutdown_connections 2024-01-24T23:26:14.406 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 1 -- 172.21.15.132:0/3251054515 wait complete. 2024-01-24T23:26:14.406 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 10 monclient: build_initial_monmap 2024-01-24T23:26:14.406 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 10 monclient: monmap: 2024-01-24T23:26:14.406 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:14.406 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:14.406 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:14.406 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:14.406 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:14.407 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:14.407 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:14.407 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:14.407 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:14.407 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:14.407 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:14.407 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:14.407 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:14.407 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:14.407 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:14.407 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:14.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 1 Processor -- start 2024-01-24T23:26:14.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 1 -- start start 2024-01-24T23:26:14.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 10 monclient: init 2024-01-24T23:26:14.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:14.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 10 monclient: _add_conns ranks=[3,6,4,5,8,2,0,1,7] 2024-01-24T23:26:14.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 10 monclient(hunting): picked mon.noname-g con 0x7f2ea409bcd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:14.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 10 monclient(hunting): picked mon.noname-d con 0x7f2ea409c0f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:14.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 10 monclient(hunting): picked mon.noname-c con 0x7f2ea4135ea0 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:14.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea41be1d0 con 0x7f2ea409bcd0 2024-01-24T23:26:14.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea409c6b0 con 0x7f2ea4135ea0 2024-01-24T23:26:14.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea4121e60 con 0x7f2ea409c0f0 2024-01-24T23:26:14.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:14.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 10 monclient(hunting): authenticate will time out at 1261.722368s 2024-01-24T23:26:14.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2ea94b8640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f2ea409bcd0 0x7f2ea41cb860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:38156/0 (socket says 172.21.15.132:38156) 2024-01-24T23:26:14.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2ea94b8640 1 -- 172.21.15.132:0/28725862 learned_addr learned my addr 172.21.15.132:0/28725862 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:14.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 1 -- 172.21.15.132:0/28725862 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1682989324 0 0) 0x7f2ea41be1d0 con 0x7f2ea409bcd0 2024-01-24T23:26:14.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:26:14.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:14.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:14.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 10 monclient(hunting): my global_id is 6249 2024-01-24T23:26:14.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 1 -- 172.21.15.132:0/28725862 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e88003560 con 0x7f2ea409bcd0 2024-01-24T23:26:14.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 1 -- 172.21.15.132:0/28725862 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4240807144 0 0) 0x7f2e88003560 con 0x7f2ea409bcd0 2024-01-24T23:26:14.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 1 -- 172.21.15.132:0/28725862 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2ea41be1d0 con 0x7f2ea409bcd0 2024-01-24T23:26:14.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 1 -- 172.21.15.132:0/28725862 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2e94001ac0 con 0x7f2ea409bcd0 2024-01-24T23:26:14.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:14.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:14.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 10 monclient(hunting): dump: 2024-01-24T23:26:14.409 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:14.409 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:14.409 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:14.410 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:14.410 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:14.410 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:14.410 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:14.410 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:14.410 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:14.410 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:14.410 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:14.410 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:14.410 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:14.410 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:14.410 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:14.410 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:14.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:14.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 1 -- 172.21.15.132:0/28725862 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3054099775 0 0) 0x7f2ea41be1d0 con 0x7f2ea409bcd0 2024-01-24T23:26:14.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 1 -- 172.21.15.132:0/28725862 >> v1:172.21.15.115:6792/0 conn(0x7f2ea409c0f0 legacy=0x7f2ea41d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:14.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 1 -- 172.21.15.132:0/28725862 >> v1:172.21.15.115:6791/0 conn(0x7f2ea4135ea0 legacy=0x7f2ea41d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:14.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:26:14.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 1 monclient: found mon.g 2024-01-24T23:26:14.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:14.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:14.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 1 -- 172.21.15.132:0/28725862 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea41209b0 con 0x7f2ea409bcd0 2024-01-24T23:26:14.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 10 monclient: _finish_auth 0 2024-01-24T23:26:14.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:26:14.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:14.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 5 monclient: authenticate success, global_id 6249 2024-01-24T23:26:14.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 10 monclient: _renew_subs 2024-01-24T23:26:14.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:14.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 1 -- 172.21.15.132:0/28725862 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea4122140 con 0x7f2ea409bcd0 2024-01-24T23:26:14.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 10 monclient: _renew_subs 2024-01-24T23:26:14.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:14.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2eab71c640 1 -- 172.21.15.132:0/28725862 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea4121b80 con 0x7f2ea409bcd0 2024-01-24T23:26:14.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 1 -- 172.21.15.132:0/28725862 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2e940038d0 con 0x7f2ea409bcd0 2024-01-24T23:26:14.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:14.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 1 -- 172.21.15.132:0/28725862 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2e94003e00 con 0x7f2ea409bcd0 2024-01-24T23:26:14.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:14.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:14.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e9b7fe640 4 set_mon_vals no callback set 2024-01-24T23:26:14.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.395+0000 7f2e99ffb640 10 monclient: dump: 2024-01-24T23:26:14.412 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:14.412 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:14.412 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:14.412 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:14.412 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:14.412 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:14.413 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:14.413 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:14.413 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:14.413 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:14.413 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:14.413 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:14.413 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:14.413 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:14.413 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:14.413 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:14.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.399+0000 7f2eab71c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:14.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.399+0000 7f2eab71c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:14.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.399+0000 7f2eab71c640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:14.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.399+0000 7f2eab71c640 1 -- 172.21.15.132:0/28725862 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e84005090 con 0x7f2ea409bcd0 2024-01-24T23:26:14.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.399+0000 7f2e99ffb640 1 -- 172.21.15.132:0/28725862 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f2e94003f60 con 0x7f2ea409bcd0 2024-01-24T23:26:14.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.399+0000 7f2e99ffb640 1 -- 172.21.15.132:0/28725862 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(157..157 src has 1..157) v4 ==== 9005+0+0 (unknown 1448385130 0 0) 0x7f2e94044c60 con 0x7f2ea409bcd0 2024-01-24T23:26:14.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.399+0000 7f2ea8cb7640 10 monclient: get_auth_request con 0x7f2e88036f70 auth_method 0 2024-01-24T23:26:14.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.403+0000 7f2e99ffb640 1 -- 172.21.15.132:0/28725862 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f2e940147c0 con 0x7f2ea409bcd0 2024-01-24T23:26:14.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.403+0000 7f2e99ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:14.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.403+0000 7f2e99ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:14.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.563+0000 7f2eab71c640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}] 2024-01-24T23:26:14.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.563+0000 7f2eab71c640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}] 2024-01-24T23:26:14.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.563+0000 7f2eab71c640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:14.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:14.563+0000 7f2eab71c640 1 -- 172.21.15.132:0/28725862 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]} v 0) v1 -- 0x7f2e84005340 con 0x7f2ea409bcd0 2024-01-24T23:26:15.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:15.267+0000 7f2e99ffb640 1 -- 172.21.15.132:0/28725862 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}]=0 v158) v1 ==== 127+0+0 (unknown 1532539261 0 0) 0x7f2e940145e0 con 0x7f2ea409bcd0 2024-01-24T23:26:15.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:15.267+0000 7f2e99ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}] 2024-01-24T23:26:15.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:15.267+0000 7f2e99ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:26:15.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:15.375+0000 7f2eab71c640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}] 2024-01-24T23:26:15.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:15.375+0000 7f2eab71c640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}] 2024-01-24T23:26:15.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:15.375+0000 7f2eab71c640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:15.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:15.375+0000 7f2eab71c640 1 -- 172.21.15.132:0/28725862 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]} v 0) v1 -- 0x7f2e84005c10 con 0x7f2ea409bcd0 2024-01-24T23:26:16.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.275+0000 7f2e99ffb640 1 -- 172.21.15.132:0/28725862 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}]=0 v159) v1 ==== 127+0+0 (unknown 543606421 0 0) 0x7f2e940147c0 con 0x7f2ea409bcd0 2024-01-24T23:26:16.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.275+0000 7f2e99ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}] 2024-01-24T23:26:16.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.275+0000 7f2e99ffb640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:26:16.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.279+0000 7f2eab71c640 1 -- 172.21.15.132:0/28725862 >> v1:172.21.15.132:6812/1835401922 conn(0x7f2e88036f70 legacy=0x7f2e88039430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:16.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.279+0000 7f2eab71c640 10 monclient: shutdown 2024-01-24T23:26:16.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.279+0000 7f2eab71c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:16.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.279+0000 7f2eab71c640 1 -- 172.21.15.132:0/28725862 >> v1:172.21.15.132:6790/0 conn(0x7f2ea409bcd0 legacy=0x7f2ea41cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:16.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.279+0000 7f2eab71c640 1 -- 172.21.15.132:0/28725862 shutdown_connections 2024-01-24T23:26:16.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.279+0000 7f2eab71c640 1 -- 172.21.15.132:0/28725862 >> 172.21.15.132:0/28725862 conn(0x7f2ea40a9bf0 msgr2=0x7f2ea4135330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:16.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.279+0000 7f2eab71c640 1 -- 172.21.15.132:0/28725862 shutdown_connections 2024-01-24T23:26:16.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.279+0000 7f2eab71c640 1 -- 172.21.15.132:0/28725862 wait complete. 2024-01-24T23:26:16.304 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set reweight cold osd.0 6.5 6.6 2024-01-24T23:26:16.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb641a35640 10 monclient: get_monmap_and_config 2024-01-24T23:26:16.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb641a35640 10 monclient: build_initial_monmap 2024-01-24T23:26:16.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb641a35640 10 monclient: monmap: 2024-01-24T23:26:16.428 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:16.428 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:16.428 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:16.429114+0000 2024-01-24T23:26:16.428 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:16.429114+0000 2024-01-24T23:26:16.428 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:16.428 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:16.428 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:16.428 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:16.428 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:16.428 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:16.429 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:16.429 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:16.429 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:16.429 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:16.429 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:16.429 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:16.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb641a35640 1 Processor -- start 2024-01-24T23:26:16.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb641a35640 1 -- start start 2024-01-24T23:26:16.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb641a35640 10 monclient: init 2024-01-24T23:26:16.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb641a35640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:16.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb641a35640 10 monclient: _add_conns ranks=[0,1,8,2,3,4,5,7,6] 2024-01-24T23:26:16.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb641a35640 10 monclient(hunting): picked mon.noname-a con 0x7fb63c13b350 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:16.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb641a35640 10 monclient(hunting): picked mon.noname-f con 0x7fb63c13b730 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:16.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb641a35640 10 monclient(hunting): picked mon.noname-e con 0x7fb63c13bb50 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:16.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb641a35640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63c064700 con 0x7fb63c13b350 2024-01-24T23:26:16.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb641a35640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63c0a2910 con 0x7fb63c13b730 2024-01-24T23:26:16.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb641a35640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63c09c8c0 con 0x7fb63c13bb50 2024-01-24T23:26:16.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb641a35640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:16.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb641a35640 10 monclient(hunting): authenticate will time out at 1263.753899s 2024-01-24T23:26:16.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb63a7fc640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fb63c13b730 legacy=0x7fb63c1401d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:16.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb63b7fe640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fb63c13bb50 0x7fb63c143900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:57052/0 (socket says 172.21.15.132:57052) 2024-01-24T23:26:16.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb63b7fe640 1 -- 172.21.15.132:0/2512699921 learned_addr learned my addr 172.21.15.132:0/2512699921 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:16.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 1 -- 172.21.15.132:0/2512699921 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1267536182 0 0) 0x7fb63c09c8c0 con 0x7fb63c13bb50 2024-01-24T23:26:16.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:26:16.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:16.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:16.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 10 monclient(hunting): my global_id is 85445 2024-01-24T23:26:16.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 1 -- 172.21.15.132:0/2512699921 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb628003530 con 0x7fb63c13bb50 2024-01-24T23:26:16.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 1 -- 172.21.15.132:0/2512699921 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3516029599 0 0) 0x7fb63c064700 con 0x7fb63c13b350 2024-01-24T23:26:16.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:26:16.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:16.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:16.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 10 monclient(hunting): my global_id is 94545 2024-01-24T23:26:16.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 1 -- 172.21.15.132:0/2512699921 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb63c09c8c0 con 0x7fb63c13b350 2024-01-24T23:26:16.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 1 -- 172.21.15.132:0/2512699921 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 343089322 0 0) 0x7fb628003530 con 0x7fb63c13bb50 2024-01-24T23:26:16.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 1 -- 172.21.15.132:0/2512699921 >> v1:172.21.15.132:6789/0 conn(0x7fb63c13b730 legacy=0x7fb63c1401d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:16.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 1 -- 172.21.15.132:0/2512699921 >> v1:172.21.15.115:6789/0 conn(0x7fb63c13b350 legacy=0x7fb63c09db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:16.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:26:16.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 1 monclient: found mon.noname-e 2024-01-24T23:26:16.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:16.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:26:16.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 1 -- 172.21.15.132:0/2512699921 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c09cba0 con 0x7fb63c13bb50 2024-01-24T23:26:16.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 10 monclient: _finish_auth 0 2024-01-24T23:26:16.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:26:16.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:16.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 1 -- 172.21.15.132:0/2512699921 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb6340017c0 con 0x7fb63c13bb50 2024-01-24T23:26:16.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:16.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:26:16.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 10 monclient: dump: 2024-01-24T23:26:16.434 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:16.435 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:16.435 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:16.435 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:16.435 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:16.435 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:16.435 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:16.435 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:16.435 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:16.435 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:16.435 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:16.435 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:16.435 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:16.435 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:16.435 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:16.435 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:16.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb641a35640 5 monclient: authenticate success, global_id 85445 2024-01-24T23:26:16.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb641a35640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:16.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 1 -- 172.21.15.132:0/2512699921 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb634001aa0 con 0x7fb63c13bb50 2024-01-24T23:26:16.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:16.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 1 -- 172.21.15.132:0/2512699921 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb634004bf0 con 0x7fb63c13bb50 2024-01-24T23:26:16.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:16.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:26:16.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb639ffb640 10 monclient: dump: 2024-01-24T23:26:16.436 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:16.436 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:16.436 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:16.436 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:16.436 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:16.436 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:16.436 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:16.437 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:16.437 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:16.437 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:16.437 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:16.437 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:16.437 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:16.437 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:16.437 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:16.437 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:16.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb641a35640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:16.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.427+0000 7fb641a35640 4 set_mon_vals no callback set 2024-01-24T23:26:16.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 10 monclient: shutdown 2024-01-24T23:26:16.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:16.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 1 -- 172.21.15.132:0/2512699921 >> v1:172.21.15.115:6793/0 conn(0x7fb63c13bb50 legacy=0x7fb63c143900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:16.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 1 -- 172.21.15.132:0/2512699921 shutdown_connections 2024-01-24T23:26:16.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 1 -- 172.21.15.132:0/2512699921 >> 172.21.15.132:0/2512699921 conn(0x7fb63c0a5a10 msgr2=0x7fb63c0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:16.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 1 -- 172.21.15.132:0/2512699921 shutdown_connections 2024-01-24T23:26:16.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 1 -- 172.21.15.132:0/2512699921 wait complete. 2024-01-24T23:26:16.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 10 monclient: build_initial_monmap 2024-01-24T23:26:16.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 10 monclient: monmap: 2024-01-24T23:26:16.442 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:16.442 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:16.442 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:16.442 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:16.442 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:16.442 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:16.442 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:16.442 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:16.443 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:16.443 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:16.443 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:16.443 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:16.443 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:16.443 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:16.443 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:16.443 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:16.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 1 Processor -- start 2024-01-24T23:26:16.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 1 -- start start 2024-01-24T23:26:16.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 10 monclient: init 2024-01-24T23:26:16.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:16.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 10 monclient: _add_conns ranks=[5,3,7,0,8,6,1,2,4] 2024-01-24T23:26:16.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 10 monclient(hunting): picked mon.noname-h con 0x7fb63c13b350 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:16.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 10 monclient(hunting): picked mon.noname-g con 0x7fb63c13b730 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:16.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 10 monclient(hunting): picked mon.noname-i con 0x7fb63c13bb50 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:16.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63c127d10 con 0x7fb63c13b730 2024-01-24T23:26:16.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63c0720f0 con 0x7fb63c13b350 2024-01-24T23:26:16.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63c13b120 con 0x7fb63c13bb50 2024-01-24T23:26:16.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:16.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 10 monclient(hunting): authenticate will time out at 1263.758032s 2024-01-24T23:26:16.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb63b7fe640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fb63c13bb50 legacy=0x7fb63c1d7f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:16.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb63a7fc640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fb63c13b730 0x7fb63c1d4840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:38172/0 (socket says 172.21.15.132:38172) 2024-01-24T23:26:16.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb63a7fc640 1 -- 172.21.15.132:0/4027452881 learned_addr learned my addr 172.21.15.132:0/4027452881 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:16.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 1 -- 172.21.15.132:0/4027452881 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986051666 0 0) 0x7fb63c127d10 con 0x7fb63c13b730 2024-01-24T23:26:16.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 10 monclient(hunting): _negotiate 2024-01-24T23:26:16.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:16.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:16.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 10 monclient(hunting): my global_id is 6258 2024-01-24T23:26:16.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 1 -- 172.21.15.132:0/4027452881 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb61c003530 con 0x7fb63c13b730 2024-01-24T23:26:16.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 1 -- 172.21.15.132:0/4027452881 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 322663497 0 0) 0x7fb63c0720f0 con 0x7fb63c13b350 2024-01-24T23:26:16.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 10 monclient(hunting): _negotiate 2024-01-24T23:26:16.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:16.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:16.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 10 monclient(hunting): my global_id is 6089 2024-01-24T23:26:16.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 1 -- 172.21.15.132:0/4027452881 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb63c127d10 con 0x7fb63c13b350 2024-01-24T23:26:16.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 1 -- 172.21.15.132:0/4027452881 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2456723509 0 0) 0x7fb61c003530 con 0x7fb63c13b730 2024-01-24T23:26:16.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 1 -- 172.21.15.132:0/4027452881 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb63c0720f0 con 0x7fb63c13b730 2024-01-24T23:26:16.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 1 -- 172.21.15.132:0/4027452881 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 147179279 0 0) 0x7fb63c127d10 con 0x7fb63c13b350 2024-01-24T23:26:16.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 1 -- 172.21.15.132:0/4027452881 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb61c003530 con 0x7fb63c13b350 2024-01-24T23:26:16.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 1 -- 172.21.15.132:0/4027452881 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb62c004680 con 0x7fb63c13b730 2024-01-24T23:26:16.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:16.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:16.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 10 monclient(hunting): dump: 2024-01-24T23:26:16.446 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:16.446 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:16.446 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:16.446 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:16.446 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:16.446 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:16.446 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:16.446 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:16.446 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:16.446 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:16.446 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:16.446 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:16.447 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:16.447 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:16.447 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:16.447 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:16.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:16.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 1 -- 172.21.15.132:0/4027452881 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb624001520 con 0x7fb63c13b350 2024-01-24T23:26:16.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:16.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:16.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 10 monclient(hunting): dump: 2024-01-24T23:26:16.447 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:16.447 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:16.447 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:16.447 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:16.447 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:16.447 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:16.447 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:16.448 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:16.448 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:16.448 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:16.448 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:16.448 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:16.448 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:16.448 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:16.448 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:16.448 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:16.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 1 -- 172.21.15.132:0/4027452881 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 475045712 0 0) 0x7fb63c0720f0 con 0x7fb63c13b730 2024-01-24T23:26:16.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 1 -- 172.21.15.132:0/4027452881 >> v1:172.21.15.132:6792/0 conn(0x7fb63c13bb50 legacy=0x7fb63c1d7f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:16.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 1 -- 172.21.15.132:0/4027452881 >> v1:172.21.15.132:6791/0 conn(0x7fb63c13b350 legacy=0x7fb63c13a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:16.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 10 monclient: _finish_hunting 0 2024-01-24T23:26:16.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 1 monclient: found mon.g 2024-01-24T23:26:16.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:16.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:16.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 1 -- 172.21.15.132:0/4027452881 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c1cfc00 con 0x7fb63c13b730 2024-01-24T23:26:16.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 10 monclient: _finish_auth 0 2024-01-24T23:26:16.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 10 monclient: _check_auth_tickets 2024-01-24T23:26:16.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:16.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 5 monclient: authenticate success, global_id 6258 2024-01-24T23:26:16.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 10 monclient: _renew_subs 2024-01-24T23:26:16.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:16.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 1 -- 172.21.15.132:0/4027452881 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb63c125a20 con 0x7fb63c13b730 2024-01-24T23:26:16.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 10 monclient: _renew_subs 2024-01-24T23:26:16.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:16.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 1 -- 172.21.15.132:0/4027452881 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63c1262d0 con 0x7fb63c13b730 2024-01-24T23:26:16.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 1 -- 172.21.15.132:0/4027452881 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb62c0048e0 con 0x7fb63c13b730 2024-01-24T23:26:16.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:16.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 1 -- 172.21.15.132:0/4027452881 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb62c004e10 con 0x7fb63c13b730 2024-01-24T23:26:16.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:16.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb6397fa640 4 set_mon_vals no callback set 2024-01-24T23:26:16.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:16.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb633fff640 10 monclient: dump: 2024-01-24T23:26:16.450 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:16.450 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:16.450 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:16.450 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:16.450 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:16.450 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:16.450 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:16.450 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:16.450 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:16.450 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:16.451 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:16.451 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:16.451 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:16.451 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:16.451 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:16.451 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:16.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:16.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:16.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:16.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.431+0000 7fb641a35640 1 -- 172.21.15.132:0/4027452881 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb620005090 con 0x7fb63c13b730 2024-01-24T23:26:16.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.435+0000 7fb633fff640 1 -- 172.21.15.132:0/4027452881 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fb62c004ff0 con 0x7fb63c13b730 2024-01-24T23:26:16.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.435+0000 7fb633fff640 1 -- 172.21.15.132:0/4027452881 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(159..159 src has 1..159) v4 ==== 9005+0+0 (unknown 2953925410 0 0) 0x7fb62c045030 con 0x7fb63c13b730 2024-01-24T23:26:16.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.435+0000 7fb63affd640 10 monclient: get_auth_request con 0x7fb61c037740 auth_method 0 2024-01-24T23:26:16.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.439+0000 7fb633fff640 1 -- 172.21.15.132:0/4027452881 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fb62c014e40 con 0x7fb63c13b730 2024-01-24T23:26:16.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.439+0000 7fb633fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:16.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.439+0000 7fb633fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:16.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.603+0000 7fb641a35640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}] 2024-01-24T23:26:16.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.603+0000 7fb641a35640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}] 2024-01-24T23:26:16.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.603+0000 7fb641a35640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:16.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:16.603+0000 7fb641a35640 1 -- 172.21.15.132:0/4027452881 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]} v 0) v1 -- 0x7fb620005340 con 0x7fb63c13b730 2024-01-24T23:26:16.656 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~12s 2024-01-24T23:26:17.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:17.291+0000 7fb633fff640 1 -- 172.21.15.132:0/4027452881 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}]=0 v160) v1 ==== 132+0+0 (unknown 1853824123 0 0) 0x7fb62c0132b0 con 0x7fb63c13b730 2024-01-24T23:26:17.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:17.291+0000 7fb633fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}] 2024-01-24T23:26:17.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:17.291+0000 7fb633fff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:26:17.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:17.395+0000 7fb641a35640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}] 2024-01-24T23:26:17.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:17.395+0000 7fb641a35640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}] 2024-01-24T23:26:17.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:17.395+0000 7fb641a35640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:17.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:17.395+0000 7fb641a35640 1 -- 172.21.15.132:0/4027452881 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]} v 0) v1 -- 0x7fb620005a80 con 0x7fb63c13b730 2024-01-24T23:26:18.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.295+0000 7fb633fff640 1 -- 172.21.15.132:0/4027452881 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}]=0 v161) v1 ==== 132+0+0 (unknown 185118270 0 0) 0x7fb62c014a30 con 0x7fb63c13b730 2024-01-24T23:26:18.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.295+0000 7fb633fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}] 2024-01-24T23:26:18.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.295+0000 7fb633fff640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:26:18.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.299+0000 7fb641a35640 1 -- 172.21.15.132:0/4027452881 >> v1:172.21.15.132:6812/1835401922 conn(0x7fb61c037740 legacy=0x7fb61c039c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:18.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.299+0000 7fb641a35640 10 monclient: shutdown 2024-01-24T23:26:18.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.299+0000 7fb641a35640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:18.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.299+0000 7fb641a35640 1 -- 172.21.15.132:0/4027452881 >> v1:172.21.15.132:6790/0 conn(0x7fb63c13b730 legacy=0x7fb63c1d4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:18.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.299+0000 7fb641a35640 1 -- 172.21.15.132:0/4027452881 shutdown_connections 2024-01-24T23:26:18.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.299+0000 7fb641a35640 1 -- 172.21.15.132:0/4027452881 >> 172.21.15.132:0/4027452881 conn(0x7fb63c0a5a10 msgr2=0x7fb63c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:18.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.299+0000 7fb641a35640 1 -- 172.21.15.132:0/4027452881 shutdown_connections 2024-01-24T23:26:18.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.299+0000 7fb641a35640 1 -- 172.21.15.132:0/4027452881 wait complete. 2024-01-24T23:26:18.320 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush tree --show-shadow 2024-01-24T23:26:18.321 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep 'osd\.0' 2024-01-24T23:26:18.321 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep globster 2024-01-24T23:26:18.321 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep '7\.' 2024-01-24T23:26:18.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.435+0000 7fbab5022640 10 monclient: get_monmap_and_config 2024-01-24T23:26:18.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.435+0000 7fbab5022640 10 monclient: build_initial_monmap 2024-01-24T23:26:18.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.435+0000 7fbab5022640 10 monclient: monmap: 2024-01-24T23:26:18.438 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:18.438 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:18.438 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:18.439430+0000 2024-01-24T23:26:18.438 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:18.439430+0000 2024-01-24T23:26:18.438 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:18.439 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:18.439 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:18.439 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:18.439 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:18.439 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:18.439 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:18.439 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:18.439 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:18.439 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:18.439 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:18.439 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:18.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.435+0000 7fbab5022640 1 Processor -- start 2024-01-24T23:26:18.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.435+0000 7fbab5022640 1 -- start start 2024-01-24T23:26:18.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.435+0000 7fbab5022640 10 monclient: init 2024-01-24T23:26:18.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.435+0000 7fbab5022640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:18.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.435+0000 7fbab5022640 10 monclient: _add_conns ranks=[7,8,2,5,1,4,6,3,0] 2024-01-24T23:26:18.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.435+0000 7fbab5022640 10 monclient(hunting): picked mon.noname-i con 0x7fbab0133b50 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:18.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbab5022640 10 monclient(hunting): picked mon.noname-e con 0x7fbab009ec20 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:18.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbab5022640 10 monclient(hunting): picked mon.noname-b con 0x7fbab013a1d0 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:18.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbab5022640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab0064700 con 0x7fbab013a1d0 2024-01-24T23:26:18.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbab5022640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab009d9f0 con 0x7fbab0133b50 2024-01-24T23:26:18.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbab5022640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab009dc60 con 0x7fbab009ec20 2024-01-24T23:26:18.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaaed9d640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fbab0133b50 legacy=0x7fbab0133f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:18.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbab5022640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:18.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbab5022640 10 monclient(hunting): authenticate will time out at 1265.764095s 2024-01-24T23:26:18.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaaf59e640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fbab013a1d0 legacy=0x7fbab013c5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:18.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaae59c640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fbab009ec20 0x7fbab0138ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:57066/0 (socket says 172.21.15.132:57066) 2024-01-24T23:26:18.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaae59c640 1 -- 172.21.15.132:0/3229906977 learned_addr learned my addr 172.21.15.132:0/3229906977 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:18.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 1 -- 172.21.15.132:0/3229906977 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3221699626 0 0) 0x7fbab009dc60 con 0x7fbab009ec20 2024-01-24T23:26:18.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 10 monclient(hunting): _negotiate 2024-01-24T23:26:18.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:18.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:18.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 10 monclient(hunting): my global_id is 85454 2024-01-24T23:26:18.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 1 -- 172.21.15.132:0/3229906977 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba98003530 con 0x7fbab009ec20 2024-01-24T23:26:18.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 1 -- 172.21.15.132:0/3229906977 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 290293709 0 0) 0x7fba98003530 con 0x7fbab009ec20 2024-01-24T23:26:18.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 1 -- 172.21.15.132:0/3229906977 >> v1:172.21.15.132:6792/0 conn(0x7fbab0133b50 legacy=0x7fbab0133f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:18.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 1 -- 172.21.15.132:0/3229906977 >> v1:172.21.15.115:6790/0 conn(0x7fbab013a1d0 legacy=0x7fbab013c5c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:18.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 10 monclient: _finish_hunting 0 2024-01-24T23:26:18.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 1 monclient: found mon.noname-e 2024-01-24T23:26:18.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:18.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:26:18.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 1 -- 172.21.15.132:0/3229906977 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab009df40 con 0x7fbab009ec20 2024-01-24T23:26:18.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 10 monclient: _finish_auth 0 2024-01-24T23:26:18.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 10 monclient: _check_auth_tickets 2024-01-24T23:26:18.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:18.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 1 -- 172.21.15.132:0/3229906977 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbaa40017c0 con 0x7fbab009ec20 2024-01-24T23:26:18.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbab5022640 5 monclient: authenticate success, global_id 85454 2024-01-24T23:26:18.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbab5022640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:18.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:18.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:26:18.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 10 monclient: dump: 2024-01-24T23:26:18.445 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:18.446 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:18.446 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:18.446 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:18.446 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:18.446 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:18.446 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:18.446 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:18.446 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:18.446 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:18.446 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:18.446 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:18.446 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:18.446 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:18.446 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:18.446 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:18.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbab5022640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:18.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 1 -- 172.21.15.132:0/3229906977 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbaa40019e0 con 0x7fbab009ec20 2024-01-24T23:26:18.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:18.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 1 -- 172.21.15.132:0/3229906977 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbaa4004bf0 con 0x7fbab009ec20 2024-01-24T23:26:18.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbab5022640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:18.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbab5022640 4 set_mon_vals no callback set 2024-01-24T23:26:18.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:18.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:26:18.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbaadd9b640 10 monclient: dump: 2024-01-24T23:26:18.447 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:18.447 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:18.447 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:18.447 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:18.447 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:18.448 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:18.448 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:18.448 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:18.448 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:18.448 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:18.448 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:18.448 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:18.448 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:18.448 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:18.448 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:18.448 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:18.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbab5022640 10 monclient: shutdown 2024-01-24T23:26:18.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbab5022640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:18.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbab5022640 1 -- 172.21.15.132:0/3229906977 >> v1:172.21.15.115:6793/0 conn(0x7fbab009ec20 legacy=0x7fbab0138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:18.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbab5022640 1 -- 172.21.15.132:0/3229906977 shutdown_connections 2024-01-24T23:26:18.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbab5022640 1 -- 172.21.15.132:0/3229906977 >> 172.21.15.132:0/3229906977 conn(0x7fbab00a5a10 msgr2=0x7fbab00a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:18.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbab5022640 1 -- 172.21.15.132:0/3229906977 shutdown_connections 2024-01-24T23:26:18.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbab5022640 1 -- 172.21.15.132:0/3229906977 wait complete. 2024-01-24T23:26:18.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbab5022640 10 monclient: build_initial_monmap 2024-01-24T23:26:18.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbab5022640 10 monclient: monmap: 2024-01-24T23:26:18.449 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:18.449 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:18.450 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:18.450 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:18.450 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:18.450 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:18.450 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:18.450 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:18.450 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:18.450 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:18.450 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:18.450 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:18.450 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:18.450 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:18.450 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:18.450 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:18.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbab5022640 1 Processor -- start 2024-01-24T23:26:18.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbab5022640 1 -- start start 2024-01-24T23:26:18.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.439+0000 7fbab5022640 10 monclient: init 2024-01-24T23:26:18.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fbab5022640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:18.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fbab5022640 10 monclient: _add_conns ranks=[8,0,4,7,6,2,5,1,3] 2024-01-24T23:26:18.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fbab5022640 10 monclient(hunting): picked mon.noname-e con 0x7fbab009ec20 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:18.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fbab5022640 10 monclient(hunting): picked mon.noname-a con 0x7fbab0133b50 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:18.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fbab5022640 10 monclient(hunting): picked mon.noname-c con 0x7fbab013a1d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:18.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fbab5022640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab00a13b0 con 0x7fbab0133b50 2024-01-24T23:26:18.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fbab5022640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab006df90 con 0x7fbab013a1d0 2024-01-24T23:26:18.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fbab5022640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab01c8160 con 0x7fbab009ec20 2024-01-24T23:26:18.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fbab5022640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:18.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fbab5022640 10 monclient(hunting): authenticate will time out at 1265.768520s 2024-01-24T23:26:18.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fbaaed9d640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fbab009ec20 0x7fbab01c7340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:57072/0 (socket says 172.21.15.132:57072) 2024-01-24T23:26:18.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fbaaed9d640 1 -- 172.21.15.132:0/1036029530 learned_addr learned my addr 172.21.15.132:0/1036029530 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:18.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1239346082 0 0) 0x7fbab00a13b0 con 0x7fbab0133b50 2024-01-24T23:26:18.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient(hunting): _negotiate 2024-01-24T23:26:18.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:18.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:18.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient(hunting): my global_id is 94554 2024-01-24T23:26:18.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba9c003530 con 0x7fbab0133b50 2024-01-24T23:26:18.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1620984540 0 0) 0x7fbab006df90 con 0x7fbab013a1d0 2024-01-24T23:26:18.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient(hunting): _negotiate 2024-01-24T23:26:18.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:18.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:18.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient(hunting): my global_id is 94441 2024-01-24T23:26:18.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbab00a13b0 con 0x7fbab013a1d0 2024-01-24T23:26:18.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4111924912 0 0) 0x7fbab01c8160 con 0x7fbab009ec20 2024-01-24T23:26:18.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient(hunting): _negotiate 2024-01-24T23:26:18.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:18.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:18.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient(hunting): my global_id is 85463 2024-01-24T23:26:18.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbab006df90 con 0x7fbab009ec20 2024-01-24T23:26:18.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3631664810 0 0) 0x7fbab00a13b0 con 0x7fbab013a1d0 2024-01-24T23:26:18.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbab01c8160 con 0x7fbab013a1d0 2024-01-24T23:26:18.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 170850108 0 0) 0x7fba9c003530 con 0x7fbab0133b50 2024-01-24T23:26:18.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbab00a13b0 con 0x7fbab0133b50 2024-01-24T23:26:18.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbaa8004560 con 0x7fbab013a1d0 2024-01-24T23:26:18.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:18.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:26:18.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient(hunting): dump: 2024-01-24T23:26:18.458 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:18.458 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:18.458 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:18.458 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:18.458 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:18.458 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:18.458 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:18.458 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:18.458 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:18.458 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:18.459 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:18.459 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:18.459 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:18.459 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:18.459 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:18.459 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:18.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:18.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 559158985 0 0) 0x7fbab006df90 con 0x7fbab009ec20 2024-01-24T23:26:18.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fba9c003530 con 0x7fbab009ec20 2024-01-24T23:26:18.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbaa4001780 con 0x7fbab0133b50 2024-01-24T23:26:18.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:18.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:26:18.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient(hunting): dump: 2024-01-24T23:26:18.459 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:18.459 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:18.460 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:18.460 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:18.460 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:18.460 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:18.460 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:18.460 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:18.460 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:18.460 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:18.460 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:18.460 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:18.460 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:18.460 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:18.460 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:18.460 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:18.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbaa0004280 con 0x7fbab009ec20 2024-01-24T23:26:18.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:18.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:26:18.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient(hunting): dump: 2024-01-24T23:26:18.461 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:18.461 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:18.461 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:18.461 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:18.461 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:18.461 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:18.461 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:18.461 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:18.461 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:18.461 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:18.461 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:18.461 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:18.461 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:18.461 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:18.462 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:18.462 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:18.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1886432849 0 0) 0x7fbab01c8160 con 0x7fbab013a1d0 2024-01-24T23:26:18.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 >> v1:172.21.15.115:6793/0 conn(0x7fbab009ec20 legacy=0x7fbab01c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:18.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 >> v1:172.21.15.115:6789/0 conn(0x7fbab0133b50 legacy=0x7fbab01c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:18.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient: _finish_hunting 0 2024-01-24T23:26:18.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 1 monclient: found mon.c 2024-01-24T23:26:18.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:18.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:18.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab0139050 con 0x7fbab013a1d0 2024-01-24T23:26:18.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient: _finish_auth 0 2024-01-24T23:26:18.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient: _check_auth_tickets 2024-01-24T23:26:18.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:18.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fbab5022640 5 monclient: authenticate success, global_id 94441 2024-01-24T23:26:18.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fbab5022640 10 monclient: _renew_subs 2024-01-24T23:26:18.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fbab5022640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:18.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fbab5022640 1 -- 172.21.15.132:0/1036029530 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab0133920 con 0x7fbab013a1d0 2024-01-24T23:26:18.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fbab5022640 10 monclient: _renew_subs 2024-01-24T23:26:18.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fbab5022640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:18.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fbab5022640 1 -- 172.21.15.132:0/1036029530 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab006e270 con 0x7fbab013a1d0 2024-01-24T23:26:18.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbaa8004840 con 0x7fbab013a1d0 2024-01-24T23:26:18.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:18.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fbaad59a640 4 set_mon_vals no callback set 2024-01-24T23:26:18.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbaa8004d70 con 0x7fbab013a1d0 2024-01-24T23:26:18.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:18.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:26:18.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fba977fe640 10 monclient: dump: 2024-01-24T23:26:18.463 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:18.463 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:18.463 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:18.464 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:18.464 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:18.464 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:18.464 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:18.464 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:18.464 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:18.464 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:18.464 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:18.464 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:18.464 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:18.464 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:18.464 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:18.464 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:18.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fbab5022640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:18.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fbab5022640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:18.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fbab5022640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:18.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.443+0000 7fbab5022640 1 -- 172.21.15.132:0/1036029530 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba90005090 con 0x7fbab013a1d0 2024-01-24T23:26:18.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.447+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fbaa8004ed0 con 0x7fbab013a1d0 2024-01-24T23:26:18.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.447+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(161..161 src has 1..161) v4 ==== 9005+0+0 (unknown 1961746305 0 0) 0x7fbaa8044e00 con 0x7fbab013a1d0 2024-01-24T23:26:18.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.447+0000 7fbaaed9d640 10 monclient: get_auth_request con 0x7fba9c037bb0 auth_method 0 2024-01-24T23:26:18.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.459+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fbaa8014bf0 con 0x7fbab013a1d0 2024-01-24T23:26:18.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.459+0000 7fba977fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:18.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.459+0000 7fba977fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:18.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.615+0000 7fbab5022640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:18.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.615+0000 7fbab5022640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:18.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.615+0000 7fbab5022640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:18.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.615+0000 7fbab5022640 1 -- 172.21.15.132:0/1036029530 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7fba90005340 con 0x7fbab013a1d0 2024-01-24T23:26:18.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.615+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v161) v1 ==== 83+0+2320 (unknown 615128269 0 2108941034) 0x7fbaa80147e0 con 0x7fbab013a1d0 2024-01-24T23:26:18.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.615+0000 7fba977fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:18.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.615+0000 7fba977fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:26:18.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.715+0000 7fbab5022640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:18.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.715+0000 7fbab5022640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:18.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.715+0000 7fbab5022640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:18.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.715+0000 7fbab5022640 1 -- 172.21.15.132:0/1036029530 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7fba90005a80 con 0x7fbab013a1d0 2024-01-24T23:26:18.719 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.715+0000 7fba977fe640 1 -- 172.21.15.132:0/1036029530 <== mon.4 v1:172.21.15.115:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v161) v1 ==== 83+0+2320 (unknown 615128269 0 2108941034) 0x7fbaa80149c0 con 0x7fbab013a1d0 2024-01-24T23:26:18.719 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.715+0000 7fba977fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:18.719 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.715+0000 7fba977fe640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:26:18.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.719+0000 7fbab5022640 1 -- 172.21.15.132:0/1036029530 >> v1:172.21.15.132:6812/1835401922 conn(0x7fba9c037bb0 legacy=0x7fba9c03a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:18.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.719+0000 7fbab5022640 10 monclient: shutdown 2024-01-24T23:26:18.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.719+0000 7fbab5022640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:18.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.719+0000 7fbab5022640 1 -- 172.21.15.132:0/1036029530 >> v1:172.21.15.115:6791/0 conn(0x7fbab013a1d0 legacy=0x7fbab01cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:18.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.719+0000 7fbab5022640 1 -- 172.21.15.132:0/1036029530 shutdown_connections 2024-01-24T23:26:18.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.719+0000 7fbab5022640 1 -- 172.21.15.132:0/1036029530 >> 172.21.15.132:0/1036029530 conn(0x7fbab00a5a10 msgr2=0x7fbab0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:18.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.719+0000 7fbab5022640 1 -- 172.21.15.132:0/1036029530 shutdown_connections 2024-01-24T23:26:18.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.719+0000 7fbab5022640 1 -- 172.21.15.132:0/1036029530 wait complete. 2024-01-24T23:26:18.736 INFO:tasks.workunit.client.0.smithi132.stdout: 0 globster 0.08789 7.50000 8.50000 6.50000 osd.0 2024-01-24T23:26:18.736 INFO:tasks.workunit.client.0.smithi132.stdout: 0 globster 0.08789 7.50000 8.50000 6.50000 osd.0 2024-01-24T23:26:18.737 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush tree --show-shadow 2024-01-24T23:26:18.737 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep 'osd\.0' 2024-01-24T23:26:18.737 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep globster 2024-01-24T23:26:18.737 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep '8\.' 2024-01-24T23:26:18.852 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcfcd2640 10 monclient: get_monmap_and_config 2024-01-24T23:26:18.852 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcfcd2640 10 monclient: build_initial_monmap 2024-01-24T23:26:18.852 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcfcd2640 10 monclient: monmap: 2024-01-24T23:26:18.852 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:18.852 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:18.852 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:18.853446+0000 2024-01-24T23:26:18.852 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:18.853446+0000 2024-01-24T23:26:18.852 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:18.853 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:18.853 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:18.853 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:18.853 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:18.853 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:18.853 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:18.853 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:18.853 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:18.853 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:18.853 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:18.853 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:18.857 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcfcd2640 1 Processor -- start 2024-01-24T23:26:18.857 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcfcd2640 1 -- start start 2024-01-24T23:26:18.857 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcfcd2640 10 monclient: init 2024-01-24T23:26:18.857 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcfcd2640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:18.857 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcfcd2640 10 monclient: _add_conns ranks=[6,5,3,2,7,1,4,8,0] 2024-01-24T23:26:18.858 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcfcd2640 10 monclient(hunting): picked mon.noname-d con 0x7f2bc809ff20 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:18.858 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcfcd2640 10 monclient(hunting): picked mon.noname-h con 0x7f2bc809f9c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:18.858 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcfcd2640 10 monclient(hunting): picked mon.noname-g con 0x7f2bc813b2c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:18.858 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcfcd2640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc8064700 con 0x7f2bc813b2c0 2024-01-24T23:26:18.858 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcfcd2640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc809e760 con 0x7f2bc809f9c0 2024-01-24T23:26:18.858 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcfcd2640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc80a1160 con 0x7f2bc809ff20 2024-01-24T23:26:18.858 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcfcd2640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:18.858 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcfcd2640 10 monclient(hunting): authenticate will time out at 1266.178530s 2024-01-24T23:26:18.858 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bce26f640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f2bc813b2c0 0x7f2bc813d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:38186/0 (socket says 172.21.15.132:38186) 2024-01-24T23:26:18.858 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bce26f640 1 -- 172.21.15.132:0/3342352999 learned_addr learned my addr 172.21.15.132:0/3342352999 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:18.858 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcd26d640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f2bc809f9c0 0x7f2bc8139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:41754/0 (socket says 172.21.15.132:41754) 2024-01-24T23:26:18.858 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 1 -- 172.21.15.132:0/3342352999 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2651103167 0 0) 0x7f2bc8064700 con 0x7f2bc813b2c0 2024-01-24T23:26:18.858 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 10 monclient(hunting): _negotiate 2024-01-24T23:26:18.858 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:18.858 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:18.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 10 monclient(hunting): my global_id is 6267 2024-01-24T23:26:18.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 1 -- 172.21.15.132:0/3342352999 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bb4003530 con 0x7f2bc813b2c0 2024-01-24T23:26:18.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 1 -- 172.21.15.132:0/3342352999 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2276873144 0 0) 0x7f2bc809e760 con 0x7f2bc809f9c0 2024-01-24T23:26:18.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 10 monclient(hunting): _negotiate 2024-01-24T23:26:18.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:18.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:18.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 10 monclient(hunting): my global_id is 6098 2024-01-24T23:26:18.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 1 -- 172.21.15.132:0/3342352999 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bc8064700 con 0x7f2bc809f9c0 2024-01-24T23:26:18.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 1 -- 172.21.15.132:0/3342352999 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2858740034 0 0) 0x7f2bc8064700 con 0x7f2bc809f9c0 2024-01-24T23:26:18.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 1 -- 172.21.15.132:0/3342352999 >> v1:172.21.15.115:6792/0 conn(0x7f2bc809ff20 legacy=0x7f2bc80a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:18.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 1 -- 172.21.15.132:0/3342352999 >> v1:172.21.15.132:6790/0 conn(0x7f2bc813b2c0 legacy=0x7f2bc813d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:18.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 10 monclient: _finish_hunting 0 2024-01-24T23:26:18.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 1 monclient: found mon.noname-h 2024-01-24T23:26:18.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:18.860 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:26:18.860 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 1 -- 172.21.15.132:0/3342352999 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc80a0a10 con 0x7f2bc809f9c0 2024-01-24T23:26:18.860 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 10 monclient: _finish_auth 0 2024-01-24T23:26:18.860 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 10 monclient: _check_auth_tickets 2024-01-24T23:26:18.860 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:18.860 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 1 -- 172.21.15.132:0/3342352999 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2bb80016f0 con 0x7f2bc809f9c0 2024-01-24T23:26:18.860 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:18.860 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:18.860 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 10 monclient: dump: 2024-01-24T23:26:18.860 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:18.860 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:18.860 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:18.860 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:18.860 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:18.861 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:18.861 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:18.861 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:18.861 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:18.861 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:18.861 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:18.861 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:18.861 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:18.861 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:18.861 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:18.861 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:18.861 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcfcd2640 5 monclient: authenticate success, global_id 6098 2024-01-24T23:26:18.861 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcfcd2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:18.861 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 1 -- 172.21.15.132:0/3342352999 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2bb8001920 con 0x7f2bc809f9c0 2024-01-24T23:26:18.861 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:18.861 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 1 -- 172.21.15.132:0/3342352999 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2bb80047f0 con 0x7f2bc809f9c0 2024-01-24T23:26:18.862 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcfcd2640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:18.862 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcfcd2640 4 set_mon_vals no callback set 2024-01-24T23:26:18.862 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:18.862 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:18.862 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcca6c640 10 monclient: dump: 2024-01-24T23:26:18.862 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:18.862 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:18.862 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:18.862 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:18.862 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:18.862 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:18.862 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:18.862 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:18.862 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:18.862 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:18.862 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:18.863 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:18.863 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:18.863 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:18.863 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:18.863 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:18.863 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcfcd2640 10 monclient: shutdown 2024-01-24T23:26:18.863 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcfcd2640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:18.863 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.851+0000 7f2bcfcd2640 1 -- 172.21.15.132:0/3342352999 >> v1:172.21.15.132:6791/0 conn(0x7f2bc809f9c0 legacy=0x7f2bc8139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:18.863 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 1 -- 172.21.15.132:0/3342352999 shutdown_connections 2024-01-24T23:26:18.863 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 1 -- 172.21.15.132:0/3342352999 >> 172.21.15.132:0/3342352999 conn(0x7f2bc80a5a10 msgr2=0x7f2bc80a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:18.863 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 1 -- 172.21.15.132:0/3342352999 shutdown_connections 2024-01-24T23:26:18.863 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 1 -- 172.21.15.132:0/3342352999 wait complete. 2024-01-24T23:26:18.863 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 10 monclient: build_initial_monmap 2024-01-24T23:26:18.863 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 10 monclient: monmap: 2024-01-24T23:26:18.863 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:18.864 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:18.864 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:18.864 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:18.864 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:18.864 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:18.864 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:18.864 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:18.864 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:18.864 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:18.864 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:18.864 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:18.864 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:18.864 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:18.864 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:18.864 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:18.865 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 1 Processor -- start 2024-01-24T23:26:18.865 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 1 -- start start 2024-01-24T23:26:18.865 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 10 monclient: init 2024-01-24T23:26:18.865 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:18.865 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 10 monclient: _add_conns ranks=[3,4,5,1,6,7,2,0,8] 2024-01-24T23:26:18.865 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 10 monclient(hunting): picked mon.noname-g con 0x7f2bc809f9c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:18.865 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 10 monclient(hunting): picked mon.noname-c con 0x7f2bc809ff20 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:18.865 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 10 monclient(hunting): picked mon.noname-h con 0x7f2bc813b2c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:18.865 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc811d620 con 0x7f2bc809f9c0 2024-01-24T23:26:18.865 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc811d470 con 0x7f2bc809ff20 2024-01-24T23:26:18.865 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc8071960 con 0x7f2bc813b2c0 2024-01-24T23:26:18.865 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:18.865 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 10 monclient(hunting): authenticate will time out at 1266.181551s 2024-01-24T23:26:18.865 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcda6e640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f2bc809f9c0 0x7f2bc811b610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:38198/0 (socket says 172.21.15.132:38198) 2024-01-24T23:26:18.865 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcda6e640 1 -- 172.21.15.132:0/1308344578 learned_addr learned my addr 172.21.15.132:0/1308344578 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:18.865 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 1 -- 172.21.15.132:0/1308344578 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4195960476 0 0) 0x7f2bc8071960 con 0x7f2bc813b2c0 2024-01-24T23:26:18.866 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:26:18.866 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:18.866 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:18.866 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 10 monclient(hunting): my global_id is 6107 2024-01-24T23:26:18.866 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 1 -- 172.21.15.132:0/1308344578 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bac003530 con 0x7f2bc813b2c0 2024-01-24T23:26:18.866 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 1 -- 172.21.15.132:0/1308344578 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1073721149 0 0) 0x7f2bc811d620 con 0x7f2bc809f9c0 2024-01-24T23:26:18.866 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:26:18.866 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:18.866 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:18.866 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 10 monclient(hunting): my global_id is 6276 2024-01-24T23:26:18.866 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 1 -- 172.21.15.132:0/1308344578 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bc8071960 con 0x7f2bc809f9c0 2024-01-24T23:26:18.866 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 1 -- 172.21.15.132:0/1308344578 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3254753129 0 0) 0x7f2bac003530 con 0x7f2bc813b2c0 2024-01-24T23:26:18.868 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 1 -- 172.21.15.132:0/1308344578 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2bc811d620 con 0x7f2bc813b2c0 2024-01-24T23:26:18.868 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 1 -- 172.21.15.132:0/1308344578 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2bc0001c00 con 0x7f2bc813b2c0 2024-01-24T23:26:18.868 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:18.869 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:18.869 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 10 monclient(hunting): dump: 2024-01-24T23:26:18.869 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:18.869 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:18.869 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:18.869 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:18.869 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:18.869 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:18.869 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:18.869 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:18.869 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:18.869 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:18.869 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:18.869 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:18.869 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:18.870 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:18.870 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:18.870 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:18.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:18.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 1 -- 172.21.15.132:0/1308344578 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2761393640 0 0) 0x7f2bc8071960 con 0x7f2bc809f9c0 2024-01-24T23:26:18.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 1 -- 172.21.15.132:0/1308344578 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2bac003530 con 0x7f2bc809f9c0 2024-01-24T23:26:18.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 1 -- 172.21.15.132:0/1308344578 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 885832895 0 0) 0x7f2bc811d620 con 0x7f2bc813b2c0 2024-01-24T23:26:18.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 1 -- 172.21.15.132:0/1308344578 >> v1:172.21.15.115:6791/0 conn(0x7f2bc809ff20 legacy=0x7f2bc811bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:18.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 1 -- 172.21.15.132:0/1308344578 >> v1:172.21.15.132:6790/0 conn(0x7f2bc809f9c0 legacy=0x7f2bc811b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:18.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:26:18.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 1 monclient: found mon.h 2024-01-24T23:26:18.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:18.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:18.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 1 -- 172.21.15.132:0/1308344578 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc811c430 con 0x7f2bc813b2c0 2024-01-24T23:26:18.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 10 monclient: _finish_auth 0 2024-01-24T23:26:18.870 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:26:18.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:18.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 5 monclient: authenticate success, global_id 6107 2024-01-24T23:26:18.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 10 monclient: _renew_subs 2024-01-24T23:26:18.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:18.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 1 -- 172.21.15.132:0/1308344578 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc811ec00 con 0x7f2bc813b2c0 2024-01-24T23:26:18.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 10 monclient: _renew_subs 2024-01-24T23:26:18.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:18.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 1 -- 172.21.15.132:0/1308344578 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc811e920 con 0x7f2bc813b2c0 2024-01-24T23:26:18.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 1 -- 172.21.15.132:0/1308344578 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2bc0004b70 con 0x7f2bc813b2c0 2024-01-24T23:26:18.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:18.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 1 -- 172.21.15.132:0/1308344578 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2bc0005020 con 0x7f2bc813b2c0 2024-01-24T23:26:18.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:18.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbffff640 4 set_mon_vals no callback set 2024-01-24T23:26:18.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:18.871 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 10 monclient: dump: 2024-01-24T23:26:18.872 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:18.872 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:18.872 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:18.872 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:18.872 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:18.872 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:18.872 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:18.872 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:18.872 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:18.872 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:18.872 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:18.872 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:18.872 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:18.872 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:18.872 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:18.873 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:18.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:18.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 1 -- 172.21.15.132:0/1308344578 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f2bc000fb20 con 0x7f2bc813b2c0 2024-01-24T23:26:18.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:18.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:18.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcfcd2640 1 -- 172.21.15.132:0/1308344578 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb0005090 con 0x7f2bc813b2c0 2024-01-24T23:26:18.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bbe7fc640 1 -- 172.21.15.132:0/1308344578 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(161..161 src has 1..161) v4 ==== 9005+0+0 (unknown 1961746305 0 0) 0x7f2bc0044f70 con 0x7f2bc813b2c0 2024-01-24T23:26:18.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.855+0000 7f2bcda6e640 10 monclient: get_auth_request con 0x7f2bac0371f0 auth_method 0 2024-01-24T23:26:18.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.859+0000 7f2bbe7fc640 1 -- 172.21.15.132:0/1308344578 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f2bc0014b30 con 0x7f2bc813b2c0 2024-01-24T23:26:18.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.859+0000 7f2bbe7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:18.873 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:18.859+0000 7f2bbe7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:19.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.023+0000 7f2bcfcd2640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:19.024 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.023+0000 7f2bcfcd2640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:19.024 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.023+0000 7f2bcfcd2640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:19.024 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.023+0000 7f2bcfcd2640 1 -- 172.21.15.132:0/1308344578 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f2bb0005340 con 0x7f2bc813b2c0 2024-01-24T23:26:19.024 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.023+0000 7f2bbe7fc640 1 -- 172.21.15.132:0/1308344578 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v161) v1 ==== 83+0+2320 (unknown 615128269 0 2108941034) 0x7f2bc00131d0 con 0x7f2bc813b2c0 2024-01-24T23:26:19.024 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.023+0000 7f2bbe7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:19.024 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.023+0000 7f2bbe7fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:26:19.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.119+0000 7f2bcfcd2640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:19.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.119+0000 7f2bcfcd2640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:19.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.119+0000 7f2bcfcd2640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:19.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.119+0000 7f2bcfcd2640 1 -- 172.21.15.132:0/1308344578 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f2bb0006590 con 0x7f2bc813b2c0 2024-01-24T23:26:19.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.123+0000 7f2bbe7fc640 1 -- 172.21.15.132:0/1308344578 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v161) v1 ==== 83+0+2320 (unknown 615128269 0 2108941034) 0x7f2bc0014950 con 0x7f2bc813b2c0 2024-01-24T23:26:19.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.123+0000 7f2bbe7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:19.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.123+0000 7f2bbe7fc640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:26:19.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.123+0000 7f2bcfcd2640 1 -- 172.21.15.132:0/1308344578 >> v1:172.21.15.132:6812/1835401922 conn(0x7f2bac0371f0 legacy=0x7f2bac0396b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:19.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.123+0000 7f2bcfcd2640 10 monclient: shutdown 2024-01-24T23:26:19.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.123+0000 7f2bcfcd2640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:19.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.123+0000 7f2bcfcd2640 1 -- 172.21.15.132:0/1308344578 >> v1:172.21.15.132:6791/0 conn(0x7f2bc813b2c0 legacy=0x7f2bc81d1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:19.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.123+0000 7f2bcfcd2640 1 -- 172.21.15.132:0/1308344578 shutdown_connections 2024-01-24T23:26:19.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.123+0000 7f2bcfcd2640 1 -- 172.21.15.132:0/1308344578 >> 172.21.15.132:0/1308344578 conn(0x7f2bc80a5a10 msgr2=0x7f2bc80a38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:19.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.123+0000 7f2bcfcd2640 1 -- 172.21.15.132:0/1308344578 shutdown_connections 2024-01-24T23:26:19.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.123+0000 7f2bcfcd2640 1 -- 172.21.15.132:0/1308344578 wait complete. 2024-01-24T23:26:19.141 INFO:tasks.workunit.client.0.smithi132.stdout: 0 globster 0.08789 7.50000 8.50000 6.50000 osd.0 2024-01-24T23:26:19.141 INFO:tasks.workunit.client.0.smithi132.stdout: 0 globster 0.08789 7.50000 8.50000 6.50000 osd.0 2024-01-24T23:26:19.142 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep 'osd\.0' 2024-01-24T23:26:19.142 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep globster 2024-01-24T23:26:19.142 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush tree --show-shadow 2024-01-24T23:26:19.142 INFO:tasks.workunit.client.0.smithi132.stderr:+ grep '6\.' 2024-01-24T23:26:19.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f7912e04640 10 monclient: get_monmap_and_config 2024-01-24T23:26:19.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f7912e04640 10 monclient: build_initial_monmap 2024-01-24T23:26:19.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f7912e04640 10 monclient: monmap: 2024-01-24T23:26:19.261 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:19.261 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:19.261 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:19.261919+0000 2024-01-24T23:26:19.261 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:19.261919+0000 2024-01-24T23:26:19.261 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:19.261 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:19.261 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:19.261 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:19.261 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:19.261 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:19.261 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:19.261 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:19.261 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:19.262 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:19.262 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:19.262 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:19.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f7912e04640 1 Processor -- start 2024-01-24T23:26:19.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f7912e04640 1 -- start start 2024-01-24T23:26:19.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f7912e04640 10 monclient: init 2024-01-24T23:26:19.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f7912e04640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:19.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f7912e04640 10 monclient: _add_conns ranks=[5,8,4,0,6,3,2,1,7] 2024-01-24T23:26:19.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f7912e04640 10 monclient(hunting): picked mon.noname-h con 0x7f790c136cb0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:19.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f7912e04640 10 monclient(hunting): picked mon.noname-e con 0x7f790c1377a0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:19.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f7912e04640 10 monclient(hunting): picked mon.noname-c con 0x7f790c09bcc0 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:19.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f7912e04640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f790c064700 con 0x7f790c09bcc0 2024-01-24T23:26:19.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f7912e04640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f790c09b650 con 0x7f790c136cb0 2024-01-24T23:26:19.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f7912e04640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f790c09c110 con 0x7f790c1377a0 2024-01-24T23:26:19.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f7912e04640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:19.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f7912e04640 10 monclient(hunting): authenticate will time out at 1266.586591s 2024-01-24T23:26:19.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f7910ba0640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f790c136cb0 0x7f790c137090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:41778/0 (socket says 172.21.15.132:41778) 2024-01-24T23:26:19.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f7910ba0640 1 -- 172.21.15.132:0/3493746797 learned_addr learned my addr 172.21.15.132:0/3493746797 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:19.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 1 -- 172.21.15.132:0/3493746797 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2690655511 0 0) 0x7f790c09b650 con 0x7f790c136cb0 2024-01-24T23:26:19.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:26:19.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:19.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:19.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 10 monclient(hunting): my global_id is 6116 2024-01-24T23:26:19.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 1 -- 172.21.15.132:0/3493746797 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78f4003530 con 0x7f790c136cb0 2024-01-24T23:26:19.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 1 -- 172.21.15.132:0/3493746797 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2567190149 0 0) 0x7f78f4003530 con 0x7f790c136cb0 2024-01-24T23:26:19.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 1 -- 172.21.15.132:0/3493746797 >> v1:172.21.15.115:6793/0 conn(0x7f790c1377a0 legacy=0x7f790c13c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:19.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 1 -- 172.21.15.132:0/3493746797 >> v1:172.21.15.115:6791/0 conn(0x7f790c09bcc0 legacy=0x7f790c13f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:19.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:26:19.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 1 monclient: found mon.noname-h 2024-01-24T23:26:19.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:19.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:26:19.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 1 -- 172.21.15.132:0/3493746797 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790c09b000 con 0x7f790c136cb0 2024-01-24T23:26:19.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 10 monclient: _finish_auth 0 2024-01-24T23:26:19.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:26:19.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:19.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 1 -- 172.21.15.132:0/3493746797 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f78fc0017c0 con 0x7f790c136cb0 2024-01-24T23:26:19.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:19.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:19.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 10 monclient: dump: 2024-01-24T23:26:19.272 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:19.273 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:19.273 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:19.273 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:19.273 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:19.273 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:19.273 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:19.273 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:19.273 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:19.273 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:19.273 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:19.273 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:19.273 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:19.273 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:19.274 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:19.274 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:19.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f7912e04640 5 monclient: authenticate success, global_id 6116 2024-01-24T23:26:19.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f7912e04640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:19.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 1 -- 172.21.15.132:0/3493746797 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f78fc001a60 con 0x7f790c136cb0 2024-01-24T23:26:19.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:19.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f7912e04640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:19.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f7912e04640 4 set_mon_vals no callback set 2024-01-24T23:26:19.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 1 -- 172.21.15.132:0/3493746797 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f78fc004bf0 con 0x7f790c136cb0 2024-01-24T23:26:19.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:19.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:19.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.259+0000 7f790b7fe640 10 monclient: dump: 2024-01-24T23:26:19.274 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:19.274 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:19.275 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:19.275 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:19.275 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:19.275 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:19.275 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:19.275 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:19.276 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:19.276 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:19.276 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:19.276 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:19.276 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:19.276 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:19.276 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:19.276 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:19.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 10 monclient: shutdown 2024-01-24T23:26:19.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:19.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 1 -- 172.21.15.132:0/3493746797 >> v1:172.21.15.132:6791/0 conn(0x7f790c136cb0 legacy=0x7f790c137090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:19.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 1 -- 172.21.15.132:0/3493746797 shutdown_connections 2024-01-24T23:26:19.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 1 -- 172.21.15.132:0/3493746797 >> 172.21.15.132:0/3493746797 conn(0x7f790c0a9c10 msgr2=0x7f790c0a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:19.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 1 -- 172.21.15.132:0/3493746797 shutdown_connections 2024-01-24T23:26:19.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 1 -- 172.21.15.132:0/3493746797 wait complete. 2024-01-24T23:26:19.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 10 monclient: build_initial_monmap 2024-01-24T23:26:19.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 10 monclient: monmap: 2024-01-24T23:26:19.277 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:19.277 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:19.277 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:19.277 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:19.277 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:19.277 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:19.277 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:19.278 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:19.278 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:19.278 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:19.278 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:19.278 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:19.278 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:19.279 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:19.279 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:19.279 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:19.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 1 Processor -- start 2024-01-24T23:26:19.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 1 -- start start 2024-01-24T23:26:19.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 10 monclient: init 2024-01-24T23:26:19.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:19.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 10 monclient: _add_conns ranks=[6,5,3,4,8,0,1,2,7] 2024-01-24T23:26:19.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 10 monclient(hunting): picked mon.noname-d con 0x7f790c09bcc0 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:19.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 10 monclient(hunting): picked mon.noname-h con 0x7f790c136cb0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:19.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 10 monclient(hunting): picked mon.noname-g con 0x7f790c1377a0 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:19.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f790c11ff50 con 0x7f790c1377a0 2024-01-24T23:26:19.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f790c09eec0 con 0x7f790c136cb0 2024-01-24T23:26:19.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f790c09f580 con 0x7f790c09bcc0 2024-01-24T23:26:19.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:19.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 10 monclient(hunting): authenticate will time out at 1266.589883s 2024-01-24T23:26:19.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79113a1640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f790c1377a0 0x7f790c1d3ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:38202/0 (socket says 172.21.15.132:38202) 2024-01-24T23:26:19.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79113a1640 1 -- 172.21.15.132:0/1891869778 learned_addr learned my addr 172.21.15.132:0/1891869778 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:19.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 1 -- 172.21.15.132:0/1891869778 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2410773636 0 0) 0x7f790c09eec0 con 0x7f790c136cb0 2024-01-24T23:26:19.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient(hunting): _negotiate 2024-01-24T23:26:19.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:19.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:19.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient(hunting): my global_id is 6125 2024-01-24T23:26:19.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 1 -- 172.21.15.132:0/1891869778 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78f8003530 con 0x7f790c136cb0 2024-01-24T23:26:19.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 1 -- 172.21.15.132:0/1891869778 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3160967771 0 0) 0x7f790c11ff50 con 0x7f790c1377a0 2024-01-24T23:26:19.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient(hunting): _negotiate 2024-01-24T23:26:19.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:19.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:19.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient(hunting): my global_id is 6285 2024-01-24T23:26:19.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 1 -- 172.21.15.132:0/1891869778 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f790c09eec0 con 0x7f790c1377a0 2024-01-24T23:26:19.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 1 -- 172.21.15.132:0/1891869778 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2717435310 0 0) 0x7f78f8003530 con 0x7f790c136cb0 2024-01-24T23:26:19.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 1 -- 172.21.15.132:0/1891869778 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f790c11ff50 con 0x7f790c136cb0 2024-01-24T23:26:19.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 1 -- 172.21.15.132:0/1891869778 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2177256763 0 0) 0x7f790c09eec0 con 0x7f790c1377a0 2024-01-24T23:26:19.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 1 -- 172.21.15.132:0/1891869778 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f78f8003530 con 0x7f790c1377a0 2024-01-24T23:26:19.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 1 -- 172.21.15.132:0/1891869778 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7904001d60 con 0x7f790c1377a0 2024-01-24T23:26:19.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:19.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:19.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient(hunting): dump: 2024-01-24T23:26:19.284 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:19.284 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:19.285 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:19.285 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:19.285 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:19.285 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:19.285 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:19.285 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:19.285 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:19.286 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:19.286 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:19.286 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:19.286 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:19.286 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:19.286 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:19.287 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:19.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:19.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 1 -- 172.21.15.132:0/1891869778 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7900001a30 con 0x7f790c136cb0 2024-01-24T23:26:19.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:19.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:19.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient(hunting): dump: 2024-01-24T23:26:19.287 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:19.288 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:19.288 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:19.288 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:19.288 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:19.288 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:19.289 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:19.289 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:19.289 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:19.289 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:19.289 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:19.289 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:19.289 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:19.290 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:19.290 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:19.290 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:19.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 1 -- 172.21.15.132:0/1891869778 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 180798014 0 0) 0x7f790c09f580 con 0x7f790c09bcc0 2024-01-24T23:26:19.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient(hunting): _negotiate 2024-01-24T23:26:19.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:19.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:19.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient(hunting): my global_id is 6351 2024-01-24T23:26:19.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 1 -- 172.21.15.132:0/1891869778 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f790c09eec0 con 0x7f790c09bcc0 2024-01-24T23:26:19.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 1 -- 172.21.15.132:0/1891869778 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1073166453 0 0) 0x7f78f8003530 con 0x7f790c1377a0 2024-01-24T23:26:19.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 1 -- 172.21.15.132:0/1891869778 >> v1:172.21.15.115:6792/0 conn(0x7f790c09bcc0 legacy=0x7f790c130a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:19.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 1 -- 172.21.15.132:0/1891869778 >> v1:172.21.15.132:6791/0 conn(0x7f790c136cb0 legacy=0x7f790c1d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:19.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient: _finish_hunting 0 2024-01-24T23:26:19.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 1 monclient: found mon.g 2024-01-24T23:26:19.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:19.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:19.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 1 -- 172.21.15.132:0/1891869778 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790c1be780 con 0x7f790c1377a0 2024-01-24T23:26:19.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient: _finish_auth 0 2024-01-24T23:26:19.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient: _check_auth_tickets 2024-01-24T23:26:19.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:19.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 5 monclient: authenticate success, global_id 6285 2024-01-24T23:26:19.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 10 monclient: _renew_subs 2024-01-24T23:26:19.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:19.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 1 -- 172.21.15.132:0/1891869778 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f790c11fc70 con 0x7f790c1377a0 2024-01-24T23:26:19.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 10 monclient: _renew_subs 2024-01-24T23:26:19.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:19.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 1 -- 172.21.15.132:0/1891869778 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f790c131190 con 0x7f790c1377a0 2024-01-24T23:26:19.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 1 -- 172.21.15.132:0/1891869778 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7904002000 con 0x7f790c1377a0 2024-01-24T23:26:19.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:19.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 1 -- 172.21.15.132:0/1891869778 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7904004010 con 0x7f790c1377a0 2024-01-24T23:26:19.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:19.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:19.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f790affd640 4 set_mon_vals no callback set 2024-01-24T23:26:19.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 10 monclient: dump: 2024-01-24T23:26:19.295 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:19.295 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:19.295 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:19.295 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:19.295 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:19.296 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:19.296 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:19.296 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:19.297 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:19.297 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:19.297 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:19.298 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:19.298 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:19.298 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:19.298 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:19.298 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:19.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 1 -- 172.21.15.132:0/1891869778 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f7904004170 con 0x7f790c1377a0 2024-01-24T23:26:19.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:19.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:19.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:19.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f7912e04640 1 -- 172.21.15.132:0/1891869778 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78ec005090 con 0x7f790c1377a0 2024-01-24T23:26:19.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.263+0000 7f79097fa640 1 -- 172.21.15.132:0/1891869778 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(161..161 src has 1..161) v4 ==== 9005+0+0 (unknown 1961746305 0 0) 0x7f7904044e50 con 0x7f790c1377a0 2024-01-24T23:26:19.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.267+0000 7f7910ba0640 10 monclient: get_auth_request con 0x7f78f80377d0 auth_method 0 2024-01-24T23:26:19.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.271+0000 7f79097fa640 1 -- 172.21.15.132:0/1891869778 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f7904014ad0 con 0x7f790c1377a0 2024-01-24T23:26:19.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.271+0000 7f79097fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:19.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.271+0000 7f79097fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:19.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.435+0000 7f7912e04640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:19.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.435+0000 7f7912e04640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:19.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.435+0000 7f7912e04640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:19.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.435+0000 7f7912e04640 1 -- 172.21.15.132:0/1891869778 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f78ec005340 con 0x7f790c1377a0 2024-01-24T23:26:19.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.435+0000 7f79097fa640 1 -- 172.21.15.132:0/1891869778 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v161) v1 ==== 83+0+2320 (unknown 615128269 0 2108941034) 0x7f7904013170 con 0x7f790c1377a0 2024-01-24T23:26:19.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.435+0000 7f79097fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:19.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.435+0000 7f79097fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:26:19.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.527+0000 7f7912e04640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:19.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.527+0000 7f7912e04640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:19.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.527+0000 7f7912e04640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:19.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.527+0000 7f7912e04640 1 -- 172.21.15.132:0/1891869778 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) v1 -- 0x7f78ec003e20 con 0x7f790c1377a0 2024-01-24T23:26:19.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.527+0000 7f79097fa640 1 -- 172.21.15.132:0/1891869778 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v161) v1 ==== 83+0+2320 (unknown 615128269 0 2108941034) 0x7f79040148f0 con 0x7f790c1377a0 2024-01-24T23:26:19.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.527+0000 7f79097fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-01-24T23:26:19.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.527+0000 7f79097fa640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:26:19.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.527+0000 7f7912e04640 1 -- 172.21.15.132:0/1891869778 >> v1:172.21.15.132:6812/1835401922 conn(0x7f78f80377d0 legacy=0x7f78f8039c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:19.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.527+0000 7f7912e04640 10 monclient: shutdown 2024-01-24T23:26:19.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.527+0000 7f7912e04640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:19.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.527+0000 7f7912e04640 1 -- 172.21.15.132:0/1891869778 >> v1:172.21.15.132:6790/0 conn(0x7f790c1377a0 legacy=0x7f790c1d3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:19.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.527+0000 7f7912e04640 1 -- 172.21.15.132:0/1891869778 shutdown_connections 2024-01-24T23:26:19.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.527+0000 7f7912e04640 1 -- 172.21.15.132:0/1891869778 >> 172.21.15.132:0/1891869778 conn(0x7f790c0a9c10 msgr2=0x7f790c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:19.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.527+0000 7f7912e04640 1 -- 172.21.15.132:0/1891869778 shutdown_connections 2024-01-24T23:26:19.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.527+0000 7f7912e04640 1 -- 172.21.15.132:0/1891869778 wait complete. 2024-01-24T23:26:19.544 INFO:tasks.workunit.client.0.smithi132.stdout: 0 globster 0.08789 7.50000 8.50000 6.50000 osd.0 2024-01-24T23:26:19.545 INFO:tasks.workunit.client.0.smithi132.stdout: 0 globster 0.08789 7.50000 8.50000 6.50000 osd.0 2024-01-24T23:26:19.545 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm-device-class osd.0 2024-01-24T23:26:19.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd763cd640 10 monclient: get_monmap_and_config 2024-01-24T23:26:19.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd763cd640 10 monclient: build_initial_monmap 2024-01-24T23:26:19.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd763cd640 10 monclient: monmap: 2024-01-24T23:26:19.661 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:19.661 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:19.661 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:19.661701+0000 2024-01-24T23:26:19.661 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:19.661701+0000 2024-01-24T23:26:19.661 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:19.661 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:19.662 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:19.662 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:19.662 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:19.662 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:19.662 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:19.662 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:19.662 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:19.662 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:19.662 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:19.662 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:19.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd763cd640 1 Processor -- start 2024-01-24T23:26:19.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd763cd640 1 -- start start 2024-01-24T23:26:19.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd763cd640 10 monclient: init 2024-01-24T23:26:19.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd763cd640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:19.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd763cd640 10 monclient: _add_conns ranks=[0,8,2,7,4,6,5,3,1] 2024-01-24T23:26:19.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd763cd640 10 monclient(hunting): picked mon.noname-a con 0x7efd7009c4c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:19.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd763cd640 10 monclient(hunting): picked mon.noname-e con 0x7efd7009cfb0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:19.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd763cd640 10 monclient(hunting): picked mon.noname-b con 0x7efd7009ec50 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:19.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd763cd640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd70064700 con 0x7efd7009c4c0 2024-01-24T23:26:19.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd763cd640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd7009d9f0 con 0x7efd7009ec50 2024-01-24T23:26:19.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd763cd640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd7009dc60 con 0x7efd7009cfb0 2024-01-24T23:26:19.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd763cd640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:19.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd763cd640 10 monclient(hunting): authenticate will time out at 1266.986487s 2024-01-24T23:26:19.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd7496a640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7efd7009ec50 legacy=0x7efd7013c460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:19.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6ffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7efd7009c4c0 0x7efd7009c8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:41248/0 (socket says 172.21.15.132:41248) 2024-01-24T23:26:19.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6ffff640 1 -- 172.21.15.132:0/4167100934 learned_addr learned my addr 172.21.15.132:0/4167100934 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:19.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 1 -- 172.21.15.132:0/4167100934 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 920239577 0 0) 0x7efd70064700 con 0x7efd7009c4c0 2024-01-24T23:26:19.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:19.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:19.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:19.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 10 monclient(hunting): my global_id is 94563 2024-01-24T23:26:19.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 1 -- 172.21.15.132:0/4167100934 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd58003530 con 0x7efd7009c4c0 2024-01-24T23:26:19.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 1 -- 172.21.15.132:0/4167100934 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162100318 0 0) 0x7efd7009dc60 con 0x7efd7009cfb0 2024-01-24T23:26:19.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:19.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:19.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:19.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 10 monclient(hunting): my global_id is 85481 2024-01-24T23:26:19.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 1 -- 172.21.15.132:0/4167100934 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd70064700 con 0x7efd7009cfb0 2024-01-24T23:26:19.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 1 -- 172.21.15.132:0/4167100934 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1408625224 0 0) 0x7efd58003530 con 0x7efd7009c4c0 2024-01-24T23:26:19.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 1 -- 172.21.15.132:0/4167100934 >> v1:172.21.15.115:6793/0 conn(0x7efd7009cfb0 legacy=0x7efd70138d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:19.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 1 -- 172.21.15.132:0/4167100934 >> v1:172.21.15.115:6790/0 conn(0x7efd7009ec50 legacy=0x7efd7013c460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:19.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 10 monclient: _finish_hunting 0 2024-01-24T23:26:19.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 1 monclient: found mon.noname-a 2024-01-24T23:26:19.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:19.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:26:19.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 1 -- 172.21.15.132:0/4167100934 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7009df40 con 0x7efd7009c4c0 2024-01-24T23:26:19.670 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 10 monclient: _finish_auth 0 2024-01-24T23:26:19.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 10 monclient: _check_auth_tickets 2024-01-24T23:26:19.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:19.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd763cd640 5 monclient: authenticate success, global_id 94563 2024-01-24T23:26:19.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 1 -- 172.21.15.132:0/4167100934 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7efd640017c0 con 0x7efd7009c4c0 2024-01-24T23:26:19.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd763cd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:19.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:19.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:26:19.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd6effd640 10 monclient: dump: 2024-01-24T23:26:19.671 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:19.671 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:19.671 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:19.671 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:19.671 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:19.671 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:19.671 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:19.672 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:19.672 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:19.672 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:19.672 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:19.672 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:19.672 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:19.672 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:19.672 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:19.672 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:19.672 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.659+0000 7efd763cd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:19.672 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6effd640 1 -- 172.21.15.132:0/4167100934 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efd64001aa0 con 0x7efd7009c4c0 2024-01-24T23:26:19.672 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6effd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:19.672 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:19.672 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 4 set_mon_vals no callback set 2024-01-24T23:26:19.672 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6effd640 1 -- 172.21.15.132:0/4167100934 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7efd64004bf0 con 0x7efd7009c4c0 2024-01-24T23:26:19.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:19.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:26:19.673 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6effd640 10 monclient: dump: 2024-01-24T23:26:19.673 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:19.673 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:19.673 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:19.673 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:19.673 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:19.673 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:19.673 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:19.673 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:19.673 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:19.673 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:19.673 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:19.673 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:19.673 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:19.674 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:19.674 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:19.674 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:19.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 10 monclient: shutdown 2024-01-24T23:26:19.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:19.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 1 -- 172.21.15.132:0/4167100934 >> v1:172.21.15.115:6789/0 conn(0x7efd7009c4c0 legacy=0x7efd7009c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:19.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 1 -- 172.21.15.132:0/4167100934 shutdown_connections 2024-01-24T23:26:19.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 1 -- 172.21.15.132:0/4167100934 >> 172.21.15.132:0/4167100934 conn(0x7efd700a5a10 msgr2=0x7efd700a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:19.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 1 -- 172.21.15.132:0/4167100934 shutdown_connections 2024-01-24T23:26:19.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 1 -- 172.21.15.132:0/4167100934 wait complete. 2024-01-24T23:26:19.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 10 monclient: build_initial_monmap 2024-01-24T23:26:19.674 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 10 monclient: monmap: 2024-01-24T23:26:19.674 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:19.674 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:19.674 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:19.675 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:19.675 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:19.675 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:19.675 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:19.675 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:19.675 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:19.675 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:19.675 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:19.675 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:19.675 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:19.675 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:19.675 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:19.675 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:19.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 1 Processor -- start 2024-01-24T23:26:19.675 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 1 -- start start 2024-01-24T23:26:19.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 10 monclient: init 2024-01-24T23:26:19.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:19.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 10 monclient: _add_conns ranks=[5,2,8,7,1,0,4,3,6] 2024-01-24T23:26:19.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 10 monclient(hunting): picked mon.noname-h con 0x7efd7009cfb0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:19.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 10 monclient(hunting): picked mon.noname-b con 0x7efd7009ec50 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:19.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 10 monclient(hunting): picked mon.noname-e con 0x7efd701c51b0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:19.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd7009c020 con 0x7efd7009ec50 2024-01-24T23:26:19.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd70134b00 con 0x7efd7009cfb0 2024-01-24T23:26:19.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd7013c7d0 con 0x7efd701c51b0 2024-01-24T23:26:19.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:19.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 10 monclient(hunting): authenticate will time out at 1266.990367s 2024-01-24T23:26:19.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6ffff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7efd7009cfb0 0x7efd70132ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:41798/0 (socket says 172.21.15.132:41798) 2024-01-24T23:26:19.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6ffff640 1 -- 172.21.15.132:0/1923027460 learned_addr learned my addr 172.21.15.132:0/1923027460 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:19.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6f7fe640 1 -- 172.21.15.132:0/1923027460 >> v1:172.21.15.115:6790/0 conn(0x7efd7009ec50 legacy=0x7efd701c4aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:19.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 1 -- 172.21.15.132:0/1923027460 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 139235259 0 0) 0x7efd70134b00 con 0x7efd7009cfb0 2024-01-24T23:26:19.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:26:19.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:19.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:19.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 10 monclient(hunting): my global_id is 6134 2024-01-24T23:26:19.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 1 -- 172.21.15.132:0/1923027460 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd5c003530 con 0x7efd7009cfb0 2024-01-24T23:26:19.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 1 -- 172.21.15.132:0/1923027460 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3352778251 0 0) 0x7efd5c003530 con 0x7efd7009cfb0 2024-01-24T23:26:19.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 1 -- 172.21.15.132:0/1923027460 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efd70134b00 con 0x7efd7009cfb0 2024-01-24T23:26:19.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 1 -- 172.21.15.132:0/1923027460 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7efd64004b30 con 0x7efd7009cfb0 2024-01-24T23:26:19.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:19.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:19.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 10 monclient(hunting): dump: 2024-01-24T23:26:19.677 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:19.678 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:19.678 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:19.678 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:19.678 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:19.678 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:19.678 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:19.678 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:19.678 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:19.678 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:19.678 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:19.678 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:19.678 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:19.678 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:19.678 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:19.678 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:19.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:19.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 1 -- 172.21.15.132:0/1923027460 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 757525912 0 0) 0x7efd68003dd0 con 0x7efd701c51b0 2024-01-24T23:26:19.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:26:19.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:19.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:19.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 10 monclient(hunting): my global_id is 85490 2024-01-24T23:26:19.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 1 -- 172.21.15.132:0/1923027460 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd5c003530 con 0x7efd701c51b0 2024-01-24T23:26:19.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 1 -- 172.21.15.132:0/1923027460 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2862458538 0 0) 0x7efd70134b00 con 0x7efd7009cfb0 2024-01-24T23:26:19.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 1 -- 172.21.15.132:0/1923027460 >> v1:172.21.15.115:6793/0 conn(0x7efd701c51b0 legacy=0x7efd701d3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:19.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 1 -- 172.21.15.132:0/1923027460 >> v1:172.21.15.115:6790/0 conn(0x7efd7009ec50 legacy=0x7efd701c4aa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:19.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 10 monclient: _finish_hunting 0 2024-01-24T23:26:19.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 1 monclient: found mon.h 2024-01-24T23:26:19.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:19.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:19.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 1 -- 172.21.15.132:0/1923027460 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd700a13b0 con 0x7efd7009cfb0 2024-01-24T23:26:19.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 10 monclient: _finish_auth 0 2024-01-24T23:26:19.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 10 monclient: _check_auth_tickets 2024-01-24T23:26:19.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:19.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 5 monclient: authenticate success, global_id 6134 2024-01-24T23:26:19.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 10 monclient: _renew_subs 2024-01-24T23:26:19.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:19.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 1 -- 172.21.15.132:0/1923027460 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7013dd30 con 0x7efd7009cfb0 2024-01-24T23:26:19.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 10 monclient: _renew_subs 2024-01-24T23:26:19.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:19.680 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd763cd640 1 -- 172.21.15.132:0/1923027460 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7efd7011b860 con 0x7efd7009cfb0 2024-01-24T23:26:19.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 1 -- 172.21.15.132:0/1923027460 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efd640011a0 con 0x7efd7009cfb0 2024-01-24T23:26:19.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:19.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 1 -- 172.21.15.132:0/1923027460 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7efd64005280 con 0x7efd7009cfb0 2024-01-24T23:26:19.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:19.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:19.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6dffb640 4 set_mon_vals no callback set 2024-01-24T23:26:19.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.663+0000 7efd6cff9640 10 monclient: dump: 2024-01-24T23:26:19.681 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:19.681 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:19.681 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:19.681 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:19.681 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:19.681 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:19.681 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:19.682 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:19.682 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:19.682 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:19.682 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:19.682 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:19.682 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:19.682 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:19.682 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:19.682 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:19.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.667+0000 7efd763cd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:19.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.667+0000 7efd763cd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:19.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.667+0000 7efd763cd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:19.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.667+0000 7efd763cd640 1 -- 172.21.15.132:0/1923027460 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd50005090 con 0x7efd7009cfb0 2024-01-24T23:26:19.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.667+0000 7efd6cff9640 1 -- 172.21.15.132:0/1923027460 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7efd6400fd00 con 0x7efd7009cfb0 2024-01-24T23:26:19.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.667+0000 7efd6cff9640 1 -- 172.21.15.132:0/1923027460 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(161..161 src has 1..161) v4 ==== 9005+0+0 (unknown 1961746305 0 0) 0x7efd64044e70 con 0x7efd7009cfb0 2024-01-24T23:26:19.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.667+0000 7efd6f7fe640 10 monclient: get_auth_request con 0x7efd5c036f00 auth_method 0 2024-01-24T23:26:19.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.671+0000 7efd6cff9640 1 -- 172.21.15.132:0/1923027460 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7efd64014b60 con 0x7efd7009cfb0 2024-01-24T23:26:19.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.671+0000 7efd6cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:19.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.671+0000 7efd6cff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:19.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.827+0000 7efd763cd640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-01-24T23:26:19.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.827+0000 7efd763cd640 10 monclient: _send_command 2 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-01-24T23:26:19.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.827+0000 7efd763cd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:19.830 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:19.827+0000 7efd763cd640 1 -- 172.21.15.132:0/1923027460 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["osd.0"]} v 0) v1 -- 0x7efd50005b50 con 0x7efd7009cfb0 2024-01-24T23:26:20.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:20.311+0000 7efd6cff9640 1 -- 172.21.15.132:0/1923027460 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}]=0 done removing class of osd(s): 0 v162) v1 ==== 123+0+0 (unknown 3572958356 0 0) 0x7efd64013200 con 0x7efd7009cfb0 2024-01-24T23:26:20.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:20.311+0000 7efd6cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-01-24T23:26:20.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:20.311+0000 7efd6cff9640 10 monclient: _finish_command 2 = system:0 done removing class of osd(s): 0 2024-01-24T23:26:20.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:20.407+0000 7efd763cd640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-01-24T23:26:20.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:20.407+0000 7efd763cd640 10 monclient: _send_command 3 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-01-24T23:26:20.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:20.407+0000 7efd763cd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:20.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:20.407+0000 7efd763cd640 1 -- 172.21.15.132:0/1923027460 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["osd.0"]} v 0) v1 -- 0x7efd500010a0 con 0x7efd7009cfb0 2024-01-24T23:26:21.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.315+0000 7efd6cff9640 1 -- 172.21.15.132:0/1923027460 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}]=0 osd.0 belongs to no class, done removing class of osd(s): v163) v1 ==== 149+0+0 (unknown 1132948932 0 0) 0x7efd64014980 con 0x7efd7009cfb0 2024-01-24T23:26:21.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.315+0000 7efd6cff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-01-24T23:26:21.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.315+0000 7efd6cff9640 10 monclient: _finish_command 3 = system:0 osd.0 belongs to no class, done removing class of osd(s): 2024-01-24T23:26:21.318 INFO:tasks.workunit.client.0.smithi132.stderr:osd.0 belongs to no class, done removing class of osd(s): 2024-01-24T23:26:21.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.319+0000 7efd763cd640 1 -- 172.21.15.132:0/1923027460 >> v1:172.21.15.132:6812/1835401922 conn(0x7efd5c036f00 legacy=0x7efd5c0393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:21.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.319+0000 7efd763cd640 10 monclient: shutdown 2024-01-24T23:26:21.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.319+0000 7efd763cd640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:21.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.319+0000 7efd763cd640 1 -- 172.21.15.132:0/1923027460 >> v1:172.21.15.132:6791/0 conn(0x7efd7009cfb0 legacy=0x7efd70132ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:21.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.319+0000 7efd763cd640 1 -- 172.21.15.132:0/1923027460 shutdown_connections 2024-01-24T23:26:21.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.319+0000 7efd763cd640 1 -- 172.21.15.132:0/1923027460 >> 172.21.15.132:0/1923027460 conn(0x7efd700a5a10 msgr2=0x7efd70066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:21.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.319+0000 7efd763cd640 1 -- 172.21.15.132:0/1923027460 shutdown_connections 2024-01-24T23:26:21.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.319+0000 7efd763cd640 1 -- 172.21.15.132:0/1923027460 wait complete. 2024-01-24T23:26:21.343 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool rm cool cool --yes-i-really-really-mean-it 2024-01-24T23:26:21.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b3e56640 10 monclient: get_monmap_and_config 2024-01-24T23:26:21.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b3e56640 10 monclient: build_initial_monmap 2024-01-24T23:26:21.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b3e56640 10 monclient: monmap: 2024-01-24T23:26:21.465 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:21.465 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:21.465 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:21.465923+0000 2024-01-24T23:26:21.465 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:21.465923+0000 2024-01-24T23:26:21.465 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:21.465 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:21.465 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:21.465 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:21.465 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:21.465 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:21.465 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:21.465 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:21.465 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:21.466 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:21.466 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:21.466 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:21.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b3e56640 1 Processor -- start 2024-01-24T23:26:21.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b3e56640 1 -- start start 2024-01-24T23:26:21.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b3e56640 10 monclient: init 2024-01-24T23:26:21.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b3e56640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:21.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b3e56640 10 monclient: _add_conns ranks=[2,5,6,3,0,7,4,8,1] 2024-01-24T23:26:21.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b3e56640 10 monclient(hunting): picked mon.noname-b con 0x7ff5ac133ed0 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:21.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b3e56640 10 monclient(hunting): picked mon.noname-h con 0x7ff5ac1342b0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:21.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b3e56640 10 monclient(hunting): picked mon.noname-d con 0x7ff5ac134690 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:21.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b3e56640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5ac064700 con 0x7ff5ac133ed0 2024-01-24T23:26:21.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b3e56640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5ac09b3d0 con 0x7ff5ac1342b0 2024-01-24T23:26:21.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b3e56640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5ac0a58c0 con 0x7ff5ac134690 2024-01-24T23:26:21.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b3e56640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:21.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b3e56640 10 monclient(hunting): authenticate will time out at 1268.791312s 2024-01-24T23:26:21.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b1bf2640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7ff5ac133ed0 legacy=0x7ff5ac09bdb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:21.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b13f1640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7ff5ac1342b0 0x7ff5ac13c040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:41808/0 (socket says 172.21.15.132:41808) 2024-01-24T23:26:21.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b13f1640 1 -- 172.21.15.132:0/1266493800 learned_addr learned my addr 172.21.15.132:0/1266493800 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:21.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b0bf0640 1 -- 172.21.15.132:0/1266493800 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1309207173 0 0) 0x7ff5ac09b3d0 con 0x7ff5ac1342b0 2024-01-24T23:26:21.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b0bf0640 10 monclient(hunting): _negotiate 2024-01-24T23:26:21.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b0bf0640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:21.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b0bf0640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:21.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b0bf0640 10 monclient(hunting): my global_id is 6143 2024-01-24T23:26:21.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.463+0000 7ff5b0bf0640 1 -- 172.21.15.132:0/1266493800 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff598003530 con 0x7ff5ac1342b0 2024-01-24T23:26:21.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b0bf0640 1 -- 172.21.15.132:0/1266493800 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3380376563 0 0) 0x7ff598003530 con 0x7ff5ac1342b0 2024-01-24T23:26:21.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b0bf0640 1 -- 172.21.15.132:0/1266493800 >> v1:172.21.15.115:6792/0 conn(0x7ff5ac134690 legacy=0x7ff5ac13f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:21.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b0bf0640 1 -- 172.21.15.132:0/1266493800 >> v1:172.21.15.115:6790/0 conn(0x7ff5ac133ed0 legacy=0x7ff5ac09bdb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:21.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b0bf0640 10 monclient: _finish_hunting 0 2024-01-24T23:26:21.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b0bf0640 1 monclient: found mon.noname-h 2024-01-24T23:26:21.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b0bf0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:21.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b0bf0640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:26:21.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b0bf0640 1 -- 172.21.15.132:0/1266493800 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ac09b0f0 con 0x7ff5ac1342b0 2024-01-24T23:26:21.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b0bf0640 10 monclient: _finish_auth 0 2024-01-24T23:26:21.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b0bf0640 10 monclient: _check_auth_tickets 2024-01-24T23:26:21.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b0bf0640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:21.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 5 monclient: authenticate success, global_id 6143 2024-01-24T23:26:21.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b0bf0640 1 -- 172.21.15.132:0/1266493800 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff59c0017c0 con 0x7ff5ac1342b0 2024-01-24T23:26:21.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:21.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b0bf0640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:21.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b0bf0640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:21.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b0bf0640 10 monclient: dump: 2024-01-24T23:26:21.475 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:21.475 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:21.475 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:21.475 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:21.475 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:21.475 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:21.475 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:21.475 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:21.475 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:21.475 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:21.476 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:21.476 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:21.476 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:21.476 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:21.476 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:21.476 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:21.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:21.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b0bf0640 1 -- 172.21.15.132:0/1266493800 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff59c001ae0 con 0x7ff5ac1342b0 2024-01-24T23:26:21.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b0bf0640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:21.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:21.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 4 set_mon_vals no callback set 2024-01-24T23:26:21.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b0bf0640 1 -- 172.21.15.132:0/1266493800 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff59c004bf0 con 0x7ff5ac1342b0 2024-01-24T23:26:21.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b0bf0640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:21.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b0bf0640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:21.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b0bf0640 10 monclient: dump: 2024-01-24T23:26:21.477 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:21.477 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:21.477 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:21.477 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:21.477 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:21.477 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:21.477 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:21.477 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:21.477 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:21.477 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:21.477 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:21.477 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:21.478 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:21.478 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:21.478 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:21.478 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:21.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 10 monclient: shutdown 2024-01-24T23:26:21.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:21.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 1 -- 172.21.15.132:0/1266493800 >> v1:172.21.15.132:6791/0 conn(0x7ff5ac1342b0 legacy=0x7ff5ac13c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:21.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 1 -- 172.21.15.132:0/1266493800 shutdown_connections 2024-01-24T23:26:21.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 1 -- 172.21.15.132:0/1266493800 >> 172.21.15.132:0/1266493800 conn(0x7ff5ac0a5ef0 msgr2=0x7ff5ac0a6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:21.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 1 -- 172.21.15.132:0/1266493800 shutdown_connections 2024-01-24T23:26:21.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 1 -- 172.21.15.132:0/1266493800 wait complete. 2024-01-24T23:26:21.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 10 monclient: build_initial_monmap 2024-01-24T23:26:21.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 10 monclient: monmap: 2024-01-24T23:26:21.478 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:21.478 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:21.479 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:21.479 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:21.479 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:21.479 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:21.479 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:21.479 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:21.479 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:21.479 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:21.479 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:21.479 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:21.479 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:21.479 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:21.479 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:21.479 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:21.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 1 Processor -- start 2024-01-24T23:26:21.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 1 -- start start 2024-01-24T23:26:21.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 10 monclient: init 2024-01-24T23:26:21.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:21.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 10 monclient: _add_conns ranks=[3,1,0,8,6,4,5,7,2] 2024-01-24T23:26:21.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 10 monclient(hunting): picked mon.noname-g con 0x7ff5ac133ed0 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:21.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 10 monclient(hunting): picked mon.noname-f con 0x7ff5ac1342b0 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:21.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 10 monclient(hunting): picked mon.noname-a con 0x7ff5ac134690 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:21.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5ac1338b0 con 0x7ff5ac134690 2024-01-24T23:26:21.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5ac0a5af0 con 0x7ff5ac1342b0 2024-01-24T23:26:21.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5ac1223c0 con 0x7ff5ac133ed0 2024-01-24T23:26:21.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:21.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b13f1640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7ff5ac1342b0 legacy=0x7ff5ac1d00b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:21.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 10 monclient(hunting): authenticate will time out at 1268.794628s 2024-01-24T23:26:21.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b1bf2640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7ff5ac133ed0 0x7ff5ac1331a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:38214/0 (socket says 172.21.15.132:38214) 2024-01-24T23:26:21.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b1bf2640 1 -- 172.21.15.132:0/3364890070 learned_addr learned my addr 172.21.15.132:0/3364890070 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:21.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 1 -- 172.21.15.132:0/3364890070 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1783078757 0 0) 0x7ff5ac1223c0 con 0x7ff5ac133ed0 2024-01-24T23:26:21.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 10 monclient(hunting): _negotiate 2024-01-24T23:26:21.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:21.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:21.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 10 monclient(hunting): my global_id is 6294 2024-01-24T23:26:21.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 1 -- 172.21.15.132:0/3364890070 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff590003530 con 0x7ff5ac133ed0 2024-01-24T23:26:21.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 1 -- 172.21.15.132:0/3364890070 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1398686852 0 0) 0x7ff590003530 con 0x7ff5ac133ed0 2024-01-24T23:26:21.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 1 -- 172.21.15.132:0/3364890070 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff5ac1223c0 con 0x7ff5ac133ed0 2024-01-24T23:26:21.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 1 -- 172.21.15.132:0/3364890070 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff5a80046b0 con 0x7ff5ac133ed0 2024-01-24T23:26:21.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:21.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:21.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 10 monclient(hunting): dump: 2024-01-24T23:26:21.482 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:21.482 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:21.482 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:21.482 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:21.482 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:21.482 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:21.482 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:21.482 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:21.482 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:21.482 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:21.482 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:21.482 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:21.482 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:21.482 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:21.482 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:21.483 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:21.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:21.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 1 -- 172.21.15.132:0/3364890070 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 886407538 0 0) 0x7ff5ac1223c0 con 0x7ff5ac133ed0 2024-01-24T23:26:21.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 1 -- 172.21.15.132:0/3364890070 >> v1:172.21.15.132:6789/0 conn(0x7ff5ac1342b0 legacy=0x7ff5ac1d00b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:21.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 1 -- 172.21.15.132:0/3364890070 >> v1:172.21.15.115:6789/0 conn(0x7ff5ac134690 legacy=0x7ff5ac1d37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:21.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 10 monclient: _finish_hunting 0 2024-01-24T23:26:21.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 1 monclient: found mon.g 2024-01-24T23:26:21.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:21.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:21.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 1 -- 172.21.15.132:0/3364890070 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ac123700 con 0x7ff5ac133ed0 2024-01-24T23:26:21.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 10 monclient: _finish_auth 0 2024-01-24T23:26:21.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 10 monclient: _check_auth_tickets 2024-01-24T23:26:21.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:21.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 5 monclient: authenticate success, global_id 6294 2024-01-24T23:26:21.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 10 monclient: _renew_subs 2024-01-24T23:26:21.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:21.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 1 -- 172.21.15.132:0/3364890070 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5ac122e60 con 0x7ff5ac133ed0 2024-01-24T23:26:21.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 10 monclient: _renew_subs 2024-01-24T23:26:21.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:21.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5b3e56640 1 -- 172.21.15.132:0/3364890070 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ac121f00 con 0x7ff5ac133ed0 2024-01-24T23:26:21.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 1 -- 172.21.15.132:0/3364890070 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff5a80049d0 con 0x7ff5ac133ed0 2024-01-24T23:26:21.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:21.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 1 -- 172.21.15.132:0/3364890070 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff5a8004f00 con 0x7ff5ac133ed0 2024-01-24T23:26:21.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:21.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a3fff640 4 set_mon_vals no callback set 2024-01-24T23:26:21.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:21.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.467+0000 7ff5a27fc640 10 monclient: dump: 2024-01-24T23:26:21.484 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:21.484 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:21.484 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:21.484 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:21.484 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:21.485 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:21.485 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:21.485 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:21.485 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:21.485 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:21.485 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:21.485 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:21.485 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:21.485 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:21.485 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:21.485 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:21.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.471+0000 7ff5b3e56640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:21.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.471+0000 7ff5b3e56640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:21.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.471+0000 7ff5b3e56640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:21.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.471+0000 7ff5b3e56640 1 -- 172.21.15.132:0/3364890070 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff594005090 con 0x7ff5ac133ed0 2024-01-24T23:26:21.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.471+0000 7ff5a27fc640 1 -- 172.21.15.132:0/3364890070 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7ff5a80050e0 con 0x7ff5ac133ed0 2024-01-24T23:26:21.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.471+0000 7ff5a27fc640 1 -- 172.21.15.132:0/3364890070 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(163..163 src has 1..163) v4 ==== 8316+0+0 (unknown 4207629837 0 0) 0x7ff5a8044e50 con 0x7ff5ac133ed0 2024-01-24T23:26:21.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.471+0000 7ff5b13f1640 10 monclient: get_auth_request con 0x7ff590036f00 auth_method 0 2024-01-24T23:26:21.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.475+0000 7ff5a27fc640 1 -- 172.21.15.132:0/3364890070 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7ff5a8014f10 con 0x7ff5ac133ed0 2024-01-24T23:26:21.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.475+0000 7ff5a27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:21.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.475+0000 7ff5a27fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:21.631 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.631+0000 7ff5b3e56640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}] 2024-01-24T23:26:21.632 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.631+0000 7ff5b3e56640 10 monclient: _send_command 2 [{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}] 2024-01-24T23:26:21.632 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.631+0000 7ff5b3e56640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:21.632 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:21.631+0000 7ff5b3e56640 1 -- 172.21.15.132:0/3364890070 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff594005340 con 0x7ff5ac133ed0 2024-01-24T23:26:22.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.347+0000 7ff5a27fc640 1 -- 172.21.15.132:0/3364890070 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}]=0 pool 'cool' removed v164) v1 ==== 148+0+0 (unknown 2958128655 0 0) 0x7ff5a8013380 con 0x7ff5ac133ed0 2024-01-24T23:26:22.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.347+0000 7ff5a27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}] 2024-01-24T23:26:22.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.347+0000 7ff5a27fc640 10 monclient: _finish_command 2 = system:0 pool 'cool' removed 2024-01-24T23:26:22.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.447+0000 7ff5b3e56640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}] 2024-01-24T23:26:22.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.447+0000 7ff5b3e56640 10 monclient: _send_command 3 [{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}] 2024-01-24T23:26:22.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.447+0000 7ff5b3e56640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:22.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.447+0000 7ff5b3e56640 1 -- 172.21.15.132:0/3364890070 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff594001150 con 0x7ff5ac133ed0 2024-01-24T23:26:22.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.447+0000 7ff5a27fc640 1 -- 172.21.15.132:0/3364890070 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}]=0 pool 'cool' does not exist v164) v1 ==== 155+0+0 (unknown 3005442268 0 0) 0x7ff5a8014b00 con 0x7ff5ac133ed0 2024-01-24T23:26:22.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.447+0000 7ff5a27fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}] 2024-01-24T23:26:22.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.447+0000 7ff5a27fc640 10 monclient: _finish_command 3 = system:0 pool 'cool' does not exist 2024-01-24T23:26:22.449 INFO:tasks.workunit.client.0.smithi132.stderr:pool 'cool' does not exist 2024-01-24T23:26:22.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.451+0000 7ff5b3e56640 1 -- 172.21.15.132:0/3364890070 >> v1:172.21.15.132:6812/1835401922 conn(0x7ff590036f00 legacy=0x7ff5900393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:22.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.451+0000 7ff5b3e56640 10 monclient: shutdown 2024-01-24T23:26:22.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.451+0000 7ff5b3e56640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:22.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.451+0000 7ff5b3e56640 1 -- 172.21.15.132:0/3364890070 >> v1:172.21.15.132:6790/0 conn(0x7ff5ac133ed0 legacy=0x7ff5ac1331a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:22.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.451+0000 7ff5b3e56640 1 -- 172.21.15.132:0/3364890070 shutdown_connections 2024-01-24T23:26:22.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.451+0000 7ff5b3e56640 1 -- 172.21.15.132:0/3364890070 >> 172.21.15.132:0/3364890070 conn(0x7ff5ac0a5ef0 msgr2=0x7ff5ac0a2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:22.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.451+0000 7ff5b3e56640 1 -- 172.21.15.132:0/3364890070 shutdown_connections 2024-01-24T23:26:22.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.451+0000 7ff5b3e56640 1 -- 172.21.15.132:0/3364890070 wait complete. 2024-01-24T23:26:22.467 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd pool rm cold cold --yes-i-really-really-mean-it 2024-01-24T23:26:22.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7c28ab640 10 monclient: get_monmap_and_config 2024-01-24T23:26:22.588 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7c28ab640 10 monclient: build_initial_monmap 2024-01-24T23:26:22.588 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7c28ab640 10 monclient: monmap: 2024-01-24T23:26:22.588 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:22.588 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:22.588 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:22.589080+0000 2024-01-24T23:26:22.588 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:22.589080+0000 2024-01-24T23:26:22.588 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:22.588 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:22.589 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:22.589 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:22.589 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:22.589 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:22.589 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:22.589 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:22.589 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:22.589 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:22.589 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:22.589 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:22.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7c28ab640 1 Processor -- start 2024-01-24T23:26:22.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7c28ab640 1 -- start start 2024-01-24T23:26:22.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7c28ab640 10 monclient: init 2024-01-24T23:26:22.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7c28ab640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:22.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7c28ab640 10 monclient: _add_conns ranks=[8,0,4,5,6,1,2,7,3] 2024-01-24T23:26:22.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7c28ab640 10 monclient(hunting): picked mon.noname-e con 0x7fe7bc133dc0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:22.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7c28ab640 10 monclient(hunting): picked mon.noname-a con 0x7fe7bc0a1b50 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:22.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7c28ab640 10 monclient(hunting): picked mon.noname-c con 0x7fe7bc13b430 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:22.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7c28ab640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7bc064700 con 0x7fe7bc0a1b50 2024-01-24T23:26:22.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7c28ab640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7bc0a3870 con 0x7fe7bc13b430 2024-01-24T23:26:22.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7c28ab640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7bc0a35c0 con 0x7fe7bc133dc0 2024-01-24T23:26:22.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7c28ab640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:22.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7c28ab640 10 monclient(hunting): authenticate will time out at 1269.913816s 2024-01-24T23:26:22.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7bb7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fe7bc0a1b50 0x7fe7bc139d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:41262/0 (socket says 172.21.15.132:41262) 2024-01-24T23:26:22.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7c0e48640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fe7bc13b430 0x7fe7bc13d820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:44228/0 (socket says 172.21.15.132:44228) 2024-01-24T23:26:22.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7bb7fe640 1 -- 172.21.15.132:0/493882874 learned_addr learned my addr 172.21.15.132:0/493882874 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:22.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 1 -- 172.21.15.132:0/493882874 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147431125 0 0) 0x7fe7bc064700 con 0x7fe7bc0a1b50 2024-01-24T23:26:22.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:22.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:22.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:22.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 10 monclient(hunting): my global_id is 94581 2024-01-24T23:26:22.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 1 -- 172.21.15.132:0/493882874 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7a4003530 con 0x7fe7bc0a1b50 2024-01-24T23:26:22.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 1 -- 172.21.15.132:0/493882874 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 913738464 0 0) 0x7fe7bc0a3870 con 0x7fe7bc13b430 2024-01-24T23:26:22.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:22.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:22.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:22.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 10 monclient(hunting): my global_id is 94468 2024-01-24T23:26:22.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 1 -- 172.21.15.132:0/493882874 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7bc064700 con 0x7fe7bc13b430 2024-01-24T23:26:22.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 1 -- 172.21.15.132:0/493882874 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 981677655 0 0) 0x7fe7bc0a35c0 con 0x7fe7bc133dc0 2024-01-24T23:26:22.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:22.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:22.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:22.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 10 monclient(hunting): my global_id is 85499 2024-01-24T23:26:22.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 1 -- 172.21.15.132:0/493882874 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7bc0a3870 con 0x7fe7bc133dc0 2024-01-24T23:26:22.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 1 -- 172.21.15.132:0/493882874 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3452417622 0 0) 0x7fe7a4003530 con 0x7fe7bc0a1b50 2024-01-24T23:26:22.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 1 -- 172.21.15.132:0/493882874 >> v1:172.21.15.115:6793/0 conn(0x7fe7bc133dc0 legacy=0x7fe7bc1341a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:22.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 1 -- 172.21.15.132:0/493882874 >> v1:172.21.15.115:6791/0 conn(0x7fe7bc13b430 legacy=0x7fe7bc13d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:22.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 10 monclient: _finish_hunting 0 2024-01-24T23:26:22.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 1 monclient: found mon.noname-a 2024-01-24T23:26:22.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:22.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:26:22.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 1 -- 172.21.15.132:0/493882874 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc070770 con 0x7fe7bc0a1b50 2024-01-24T23:26:22.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 10 monclient: _finish_auth 0 2024-01-24T23:26:22.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 10 monclient: _check_auth_tickets 2024-01-24T23:26:22.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:22.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 1 -- 172.21.15.132:0/493882874 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe7b00017c0 con 0x7fe7bc0a1b50 2024-01-24T23:26:22.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:22.602 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:26:22.602 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 10 monclient: dump: 2024-01-24T23:26:22.602 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:22.602 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:22.602 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:22.602 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:22.602 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:22.602 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:22.602 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:22.603 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:22.603 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:22.603 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:22.603 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:22.603 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:22.603 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:22.603 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:22.603 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:22.603 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:22.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7c28ab640 5 monclient: authenticate success, global_id 94581 2024-01-24T23:26:22.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7c28ab640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:22.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 1 -- 172.21.15.132:0/493882874 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe7b0001a20 con 0x7fe7bc0a1b50 2024-01-24T23:26:22.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:22.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7c28ab640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:22.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 1 -- 172.21.15.132:0/493882874 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe7b0004bf0 con 0x7fe7bc0a1b50 2024-01-24T23:26:22.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:22.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7c28ab640 4 set_mon_vals no callback set 2024-01-24T23:26:22.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:26:22.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.587+0000 7fe7baffd640 10 monclient: dump: 2024-01-24T23:26:22.604 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:22.604 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:22.604 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:22.604 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:22.604 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:22.604 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:22.605 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:22.605 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:22.605 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:22.605 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:22.605 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:22.605 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:22.605 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:22.605 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:22.605 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:22.605 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:22.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 10 monclient: shutdown 2024-01-24T23:26:22.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:22.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 1 -- 172.21.15.132:0/493882874 >> v1:172.21.15.115:6789/0 conn(0x7fe7bc0a1b50 legacy=0x7fe7bc139d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:22.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 1 -- 172.21.15.132:0/493882874 shutdown_connections 2024-01-24T23:26:22.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 1 -- 172.21.15.132:0/493882874 >> 172.21.15.132:0/493882874 conn(0x7fe7bc06d810 msgr2=0x7fe7bc0a7150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:22.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 1 -- 172.21.15.132:0/493882874 shutdown_connections 2024-01-24T23:26:22.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 1 -- 172.21.15.132:0/493882874 wait complete. 2024-01-24T23:26:22.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 10 monclient: build_initial_monmap 2024-01-24T23:26:22.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 10 monclient: monmap: 2024-01-24T23:26:22.606 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:22.606 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:22.606 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:22.606 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:22.607 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:22.607 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:22.607 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:22.607 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:22.607 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:22.607 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:22.607 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:22.607 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:22.607 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:22.607 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:22.607 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:22.607 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:22.607 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 1 Processor -- start 2024-01-24T23:26:22.607 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 1 -- start start 2024-01-24T23:26:22.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 10 monclient: init 2024-01-24T23:26:22.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:22.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 10 monclient: _add_conns ranks=[7,5,6,0,2,1,4,3,8] 2024-01-24T23:26:22.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 10 monclient(hunting): picked mon.noname-i con 0x7fe7bc0a1b50 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:22.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 10 monclient(hunting): picked mon.noname-h con 0x7fe7bc133dc0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:22.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 10 monclient(hunting): picked mon.noname-d con 0x7fe7bc13b430 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:22.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7bc06ffa0 con 0x7fe7bc133dc0 2024-01-24T23:26:22.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7bc09d700 con 0x7fe7bc13b430 2024-01-24T23:26:22.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7bc09be70 con 0x7fe7bc0a1b50 2024-01-24T23:26:22.609 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:22.609 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 10 monclient(hunting): authenticate will time out at 1269.917871s 2024-01-24T23:26:22.609 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7bbfff640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fe7bc0a1b50 legacy=0x7fe7bc09b050 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:22.609 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7bb7fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fe7bc133dc0 0x7fe7bc09b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:41812/0 (socket says 172.21.15.132:41812) 2024-01-24T23:26:22.609 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7bb7fe640 1 -- 172.21.15.132:0/3487874985 learned_addr learned my addr 172.21.15.132:0/3487874985 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:22.609 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 1 -- 172.21.15.132:0/3487874985 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3221178082 0 0) 0x7fe7bc06ffa0 con 0x7fe7bc133dc0 2024-01-24T23:26:22.609 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:26:22.609 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:22.609 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:22.609 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 10 monclient(hunting): my global_id is 6152 2024-01-24T23:26:22.609 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 1 -- 172.21.15.132:0/3487874985 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7a8003530 con 0x7fe7bc133dc0 2024-01-24T23:26:22.609 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 1 -- 172.21.15.132:0/3487874985 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3426416483 0 0) 0x7fe7a8003530 con 0x7fe7bc133dc0 2024-01-24T23:26:22.610 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 1 -- 172.21.15.132:0/3487874985 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe7bc06ffa0 con 0x7fe7bc133dc0 2024-01-24T23:26:22.610 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 1 -- 172.21.15.132:0/3487874985 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe7b00047f0 con 0x7fe7bc133dc0 2024-01-24T23:26:22.610 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:22.610 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:22.610 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 10 monclient(hunting): dump: 2024-01-24T23:26:22.610 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:22.610 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:22.610 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:22.610 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:22.610 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:22.610 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:22.610 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:22.610 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:22.611 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:22.611 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:22.611 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:22.611 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:22.611 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:22.611 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:22.611 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:22.611 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:22.611 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:22.611 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 1 -- 172.21.15.132:0/3487874985 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 883649716 0 0) 0x7fe7bc09d700 con 0x7fe7bc13b430 2024-01-24T23:26:22.611 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:26:22.611 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:22.611 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:22.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 10 monclient(hunting): my global_id is 6369 2024-01-24T23:26:22.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 1 -- 172.21.15.132:0/3487874985 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7a8003530 con 0x7fe7bc13b430 2024-01-24T23:26:22.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 1 -- 172.21.15.132:0/3487874985 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2265143365 0 0) 0x7fe7bc06ffa0 con 0x7fe7bc133dc0 2024-01-24T23:26:22.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 1 -- 172.21.15.132:0/3487874985 >> v1:172.21.15.132:6792/0 conn(0x7fe7bc0a1b50 legacy=0x7fe7bc09b050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:22.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 1 -- 172.21.15.132:0/3487874985 >> v1:172.21.15.115:6792/0 conn(0x7fe7bc13b430 legacy=0x7fe7bc1d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:22.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:26:22.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 1 monclient: found mon.h 2024-01-24T23:26:22.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:22.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:22.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 1 -- 172.21.15.132:0/3487874985 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc0720c0 con 0x7fe7bc133dc0 2024-01-24T23:26:22.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 10 monclient: _finish_auth 0 2024-01-24T23:26:22.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:26:22.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:22.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 5 monclient: authenticate success, global_id 6152 2024-01-24T23:26:22.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 10 monclient: _renew_subs 2024-01-24T23:26:22.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:22.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 1 -- 172.21.15.132:0/3487874985 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7bc070220 con 0x7fe7bc133dc0 2024-01-24T23:26:22.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 10 monclient: _renew_subs 2024-01-24T23:26:22.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:22.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 1 -- 172.21.15.132:0/3487874985 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7bc09c870 con 0x7fe7bc133dc0 2024-01-24T23:26:22.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:22.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:22.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:22.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7c28ab640 1 -- 172.21.15.132:0/3487874985 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7bc073400 con 0x7fe7bc133dc0 2024-01-24T23:26:22.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 1 -- 172.21.15.132:0/3487874985 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe7b00011a0 con 0x7fe7bc133dc0 2024-01-24T23:26:22.614 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:22.614 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 1 -- 172.21.15.132:0/3487874985 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe7b0005280 con 0x7fe7bc133dc0 2024-01-24T23:26:22.614 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:22.614 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:22.614 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b8ff9640 10 monclient: dump: 2024-01-24T23:26:22.614 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:22.614 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:22.614 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:22.614 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:22.615 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:22.615 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:22.615 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:22.615 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:22.615 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:22.616 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:22.616 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:22.616 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:22.616 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:22.616 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:22.616 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:22.616 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:22.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.591+0000 7fe7b9ffb640 4 set_mon_vals no callback set 2024-01-24T23:26:22.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.595+0000 7fe7b8ff9640 1 -- 172.21.15.132:0/3487874985 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fe7b000fd00 con 0x7fe7bc133dc0 2024-01-24T23:26:22.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.595+0000 7fe7b8ff9640 1 -- 172.21.15.132:0/3487874985 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(164..164 src has 1..164) v4 ==== 7645+0+0 (unknown 282477177 0 0) 0x7fe7b0044890 con 0x7fe7bc133dc0 2024-01-24T23:26:22.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.595+0000 7fe7bbfff640 10 monclient: get_auth_request con 0x7fe7a8036fd0 auth_method 0 2024-01-24T23:26:22.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.599+0000 7fe7b8ff9640 1 -- 172.21.15.132:0/3487874985 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fe7b0014ad0 con 0x7fe7bc133dc0 2024-01-24T23:26:22.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.599+0000 7fe7b8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:22.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.599+0000 7fe7b8ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:22.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.751+0000 7fe7c28ab640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}] 2024-01-24T23:26:22.755 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.751+0000 7fe7c28ab640 10 monclient: _send_command 2 [{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}] 2024-01-24T23:26:22.755 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.751+0000 7fe7c28ab640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:22.755 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:22.751+0000 7fe7c28ab640 1 -- 172.21.15.132:0/3487874985 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe7bc074c90 con 0x7fe7bc133dc0 2024-01-24T23:26:22.865 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~19s 2024-01-24T23:26:23.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.347+0000 7fe7b8ff9640 1 -- 172.21.15.132:0/3487874985 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}]=0 pool 'cold' removed v165) v1 ==== 148+0+0 (unknown 4220902896 0 0) 0x7fe7b00148f0 con 0x7fe7bc133dc0 2024-01-24T23:26:23.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.347+0000 7fe7b8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}] 2024-01-24T23:26:23.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.347+0000 7fe7b8ff9640 10 monclient: _finish_command 2 = system:0 pool 'cold' removed 2024-01-24T23:26:23.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.447+0000 7fe7c28ab640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}] 2024-01-24T23:26:23.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.447+0000 7fe7c28ab640 10 monclient: _send_command 3 [{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}] 2024-01-24T23:26:23.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.447+0000 7fe7c28ab640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:23.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.447+0000 7fe7c28ab640 1 -- 172.21.15.132:0/3487874985 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe7bc09de20 con 0x7fe7bc133dc0 2024-01-24T23:26:23.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.447+0000 7fe7b8ff9640 1 -- 172.21.15.132:0/3487874985 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}]=0 pool 'cold' does not exist v165) v1 ==== 155+0+0 (unknown 2049824506 0 0) 0x7fe7b0015630 con 0x7fe7bc133dc0 2024-01-24T23:26:23.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.447+0000 7fe7b8ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}] 2024-01-24T23:26:23.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.447+0000 7fe7b8ff9640 10 monclient: _finish_command 3 = system:0 pool 'cold' does not exist 2024-01-24T23:26:23.451 INFO:tasks.workunit.client.0.smithi132.stderr:pool 'cold' does not exist 2024-01-24T23:26:23.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.451+0000 7fe7c28ab640 1 -- 172.21.15.132:0/3487874985 >> v1:172.21.15.132:6812/1835401922 conn(0x7fe7a8036fd0 legacy=0x7fe7a8039490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:23.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.451+0000 7fe7c28ab640 10 monclient: shutdown 2024-01-24T23:26:23.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.451+0000 7fe7c28ab640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:23.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.451+0000 7fe7c28ab640 1 -- 172.21.15.132:0/3487874985 >> v1:172.21.15.132:6791/0 conn(0x7fe7bc133dc0 legacy=0x7fe7bc09b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:23.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.451+0000 7fe7c28ab640 1 -- 172.21.15.132:0/3487874985 shutdown_connections 2024-01-24T23:26:23.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.451+0000 7fe7c28ab640 1 -- 172.21.15.132:0/3487874985 >> 172.21.15.132:0/3487874985 conn(0x7fe7bc06d810 msgr2=0x7fe7bc134960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:23.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.451+0000 7fe7c28ab640 1 -- 172.21.15.132:0/3487874985 shutdown_connections 2024-01-24T23:26:23.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.451+0000 7fe7c28ab640 1 -- 172.21.15.132:0/3487874985 wait complete. 2024-01-24T23:26:23.468 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set rm-compat 2024-01-24T23:26:23.584 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faefbab8640 10 monclient: get_monmap_and_config 2024-01-24T23:26:23.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faefbab8640 10 monclient: build_initial_monmap 2024-01-24T23:26:23.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faefbab8640 10 monclient: monmap: 2024-01-24T23:26:23.585 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:23.585 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:23.585 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:23.586065+0000 2024-01-24T23:26:23.585 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:23.586065+0000 2024-01-24T23:26:23.585 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:23.585 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:23.585 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:23.585 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:23.585 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:23.585 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:23.585 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:23.586 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:23.586 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:23.586 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:23.586 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:23.586 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:23.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faefbab8640 1 Processor -- start 2024-01-24T23:26:23.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faefbab8640 1 -- start start 2024-01-24T23:26:23.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faefbab8640 10 monclient: init 2024-01-24T23:26:23.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faefbab8640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:23.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faefbab8640 10 monclient: _add_conns ranks=[5,1,8,2,6,0,3,7,4] 2024-01-24T23:26:23.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faefbab8640 10 monclient(hunting): picked mon.noname-h con 0x7faef4133b50 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:23.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faefbab8640 10 monclient(hunting): picked mon.noname-f con 0x7faef409ec20 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:23.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faefbab8640 10 monclient(hunting): picked mon.noname-e con 0x7faef413a1d0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:23.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faefbab8640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef4064700 con 0x7faef409ec20 2024-01-24T23:26:23.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faefbab8640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef409d9f0 con 0x7faef4133b50 2024-01-24T23:26:23.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faefbab8640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef409dc60 con 0x7faef413a1d0 2024-01-24T23:26:23.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faefbab8640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:23.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef9053640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7faef409ec20 legacy=0x7faef4138ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:23.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faefbab8640 10 monclient(hunting): authenticate will time out at 1270.910772s 2024-01-24T23:26:23.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef9854640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7faef4133b50 0x7faef4133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:41814/0 (socket says 172.21.15.132:41814) 2024-01-24T23:26:23.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef9854640 1 -- 172.21.15.132:0/403041800 learned_addr learned my addr 172.21.15.132:0/403041800 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:23.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 1 -- 172.21.15.132:0/403041800 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1797155569 0 0) 0x7faef409d9f0 con 0x7faef4133b50 2024-01-24T23:26:23.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 10 monclient(hunting): _negotiate 2024-01-24T23:26:23.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:23.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:23.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 10 monclient(hunting): my global_id is 6161 2024-01-24T23:26:23.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 1 -- 172.21.15.132:0/403041800 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faee0003530 con 0x7faef4133b50 2024-01-24T23:26:23.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 1 -- 172.21.15.132:0/403041800 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1084676483 0 0) 0x7faee0003530 con 0x7faef4133b50 2024-01-24T23:26:23.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 1 -- 172.21.15.132:0/403041800 >> v1:172.21.15.115:6793/0 conn(0x7faef413a1d0 legacy=0x7faef413c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:23.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 1 -- 172.21.15.132:0/403041800 >> v1:172.21.15.132:6789/0 conn(0x7faef409ec20 legacy=0x7faef4138ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:23.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 10 monclient: _finish_hunting 0 2024-01-24T23:26:23.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 1 monclient: found mon.noname-h 2024-01-24T23:26:23.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:23.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:26:23.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 1 -- 172.21.15.132:0/403041800 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef409df40 con 0x7faef4133b50 2024-01-24T23:26:23.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 10 monclient: _finish_auth 0 2024-01-24T23:26:23.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 10 monclient: _check_auth_tickets 2024-01-24T23:26:23.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:23.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 1 -- 172.21.15.132:0/403041800 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faef0001b40 con 0x7faef4133b50 2024-01-24T23:26:23.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faefbab8640 5 monclient: authenticate success, global_id 6161 2024-01-24T23:26:23.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faefbab8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:23.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:23.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:23.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 10 monclient: dump: 2024-01-24T23:26:23.595 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:23.595 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:23.595 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:23.595 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:23.595 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:23.595 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:23.595 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:23.595 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:23.595 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:23.595 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:23.595 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:23.595 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:23.596 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:23.596 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:23.596 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:23.596 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:23.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faefbab8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:23.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 1 -- 172.21.15.132:0/403041800 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faef0001e60 con 0x7faef4133b50 2024-01-24T23:26:23.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:23.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faefbab8640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:23.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 1 -- 172.21.15.132:0/403041800 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faef0004f70 con 0x7faef4133b50 2024-01-24T23:26:23.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faefbab8640 4 set_mon_vals no callback set 2024-01-24T23:26:23.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:23.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:23.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.583+0000 7faef8852640 10 monclient: dump: 2024-01-24T23:26:23.597 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:23.597 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:23.597 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:23.597 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:23.597 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:23.597 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:23.597 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:23.597 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:23.597 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:23.597 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:23.597 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:23.597 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:23.597 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:23.597 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:23.597 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:23.597 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:23.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 10 monclient: shutdown 2024-01-24T23:26:23.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:23.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 1 -- 172.21.15.132:0/403041800 >> v1:172.21.15.132:6791/0 conn(0x7faef4133b50 legacy=0x7faef4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:23.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 1 -- 172.21.15.132:0/403041800 shutdown_connections 2024-01-24T23:26:23.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 1 -- 172.21.15.132:0/403041800 >> 172.21.15.132:0/403041800 conn(0x7faef40a5a10 msgr2=0x7faef40a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:23.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 1 -- 172.21.15.132:0/403041800 shutdown_connections 2024-01-24T23:26:23.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 1 -- 172.21.15.132:0/403041800 wait complete. 2024-01-24T23:26:23.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 10 monclient: build_initial_monmap 2024-01-24T23:26:23.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 10 monclient: monmap: 2024-01-24T23:26:23.598 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:23.598 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:23.598 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:23.598 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:23.598 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:23.598 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:23.598 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:23.599 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:23.599 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:23.599 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:23.599 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:23.599 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:23.599 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:23.599 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:23.599 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:23.599 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:23.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 1 Processor -- start 2024-01-24T23:26:23.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 1 -- start start 2024-01-24T23:26:23.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 10 monclient: init 2024-01-24T23:26:23.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:23.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 10 monclient: _add_conns ranks=[0,5,1,3,7,4,8,2,6] 2024-01-24T23:26:23.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 10 monclient(hunting): picked mon.noname-a con 0x7faef409ec20 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:23.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 10 monclient(hunting): picked mon.noname-h con 0x7faef4133b50 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:23.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 10 monclient(hunting): picked mon.noname-f con 0x7faef413a1d0 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:23.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef4137680 con 0x7faef409ec20 2024-01-24T23:26:23.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef4133920 con 0x7faef413a1d0 2024-01-24T23:26:23.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef40a13b0 con 0x7faef4133b50 2024-01-24T23:26:23.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:23.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 10 monclient(hunting): authenticate will time out at 1270.913854s 2024-01-24T23:26:23.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefa055640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7faef413a1d0 legacy=0x7faef41d3f00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:23.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faef9053640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7faef4133b50 0x7faef41d07d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:41830/0 (socket says 172.21.15.132:41830) 2024-01-24T23:26:23.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faef9053640 1 -- 172.21.15.132:0/1317339943 learned_addr learned my addr 172.21.15.132:0/1317339943 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:23.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 1 -- 172.21.15.132:0/1317339943 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1225072423 0 0) 0x7faef40a13b0 con 0x7faef4133b50 2024-01-24T23:26:23.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:26:23.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:23.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:23.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 10 monclient(hunting): my global_id is 6170 2024-01-24T23:26:23.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 1 -- 172.21.15.132:0/1317339943 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faed8003530 con 0x7faef4133b50 2024-01-24T23:26:23.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 1 -- 172.21.15.132:0/1317339943 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2786375798 0 0) 0x7faed8003530 con 0x7faef4133b50 2024-01-24T23:26:23.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 1 -- 172.21.15.132:0/1317339943 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faef40a13b0 con 0x7faef4133b50 2024-01-24T23:26:23.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 1 -- 172.21.15.132:0/1317339943 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faee80040e0 con 0x7faef4133b50 2024-01-24T23:26:23.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:23.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:23.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 10 monclient(hunting): dump: 2024-01-24T23:26:23.601 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:23.601 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:23.601 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:23.601 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:23.601 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:23.601 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:23.601 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:23.601 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:23.602 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:23.602 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:23.602 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:23.602 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:23.602 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:23.602 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:23.602 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:23.602 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:23.602 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:23.602 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 1 -- 172.21.15.132:0/1317339943 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1050828154 0 0) 0x7faef4137680 con 0x7faef409ec20 2024-01-24T23:26:23.602 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:26:23.602 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:23.602 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:23.602 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 10 monclient(hunting): my global_id is 94590 2024-01-24T23:26:23.602 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 1 -- 172.21.15.132:0/1317339943 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faed8003530 con 0x7faef409ec20 2024-01-24T23:26:23.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 1 -- 172.21.15.132:0/1317339943 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1534973487 0 0) 0x7faef40a13b0 con 0x7faef4133b50 2024-01-24T23:26:23.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 1 -- 172.21.15.132:0/1317339943 >> v1:172.21.15.132:6789/0 conn(0x7faef413a1d0 legacy=0x7faef41d3f00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:23.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 1 -- 172.21.15.132:0/1317339943 >> v1:172.21.15.115:6789/0 conn(0x7faef409ec20 legacy=0x7faef412c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:23.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 10 monclient: _finish_hunting 0 2024-01-24T23:26:23.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 1 monclient: found mon.h 2024-01-24T23:26:23.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:23.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:23.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 1 -- 172.21.15.132:0/1317339943 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef4138d70 con 0x7faef4133b50 2024-01-24T23:26:23.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 10 monclient: _finish_auth 0 2024-01-24T23:26:23.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 10 monclient: _check_auth_tickets 2024-01-24T23:26:23.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:23.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 5 monclient: authenticate success, global_id 6170 2024-01-24T23:26:23.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 10 monclient: _renew_subs 2024-01-24T23:26:23.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:23.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 1 -- 172.21.15.132:0/1317339943 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef4136d50 con 0x7faef4133b50 2024-01-24T23:26:23.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 10 monclient: _renew_subs 2024-01-24T23:26:23.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:23.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 1 -- 172.21.15.132:0/1317339943 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7faef4138a90 con 0x7faef4133b50 2024-01-24T23:26:23.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 1 -- 172.21.15.132:0/1317339943 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faee8004890 con 0x7faef4133b50 2024-01-24T23:26:23.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:23.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 1 -- 172.21.15.132:0/1317339943 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faee8004dc0 con 0x7faef4133b50 2024-01-24T23:26:23.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:23.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:23.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee7fff640 4 set_mon_vals no callback set 2024-01-24T23:26:23.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 10 monclient: dump: 2024-01-24T23:26:23.604 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:23.604 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:23.604 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:23.604 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:23.604 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:23.605 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:23.605 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:23.605 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:23.605 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:23.605 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:23.605 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:23.605 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:23.605 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:23.605 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:23.605 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:23.605 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:23.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:23.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:23.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:23.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faefbab8640 1 -- 172.21.15.132:0/1317339943 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faedc005090 con 0x7faef4133b50 2024-01-24T23:26:23.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.587+0000 7faee67fc640 1 -- 172.21.15.132:0/1317339943 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7faee8004fa0 con 0x7faef4133b50 2024-01-24T23:26:23.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.591+0000 7faee67fc640 1 -- 172.21.15.132:0/1317339943 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(165..165 src has 1..165) v4 ==== 6838+0+0 (unknown 3624703773 0 0) 0x7faee80443f0 con 0x7faef4133b50 2024-01-24T23:26:23.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.591+0000 7faef9854640 10 monclient: get_auth_request con 0x7faed8036f70 auth_method 0 2024-01-24T23:26:23.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.595+0000 7faee67fc640 1 -- 172.21.15.132:0/1317339943 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7faee8014a60 con 0x7faef4133b50 2024-01-24T23:26:23.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.595+0000 7faee67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:23.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.595+0000 7faee67fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:23.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.759+0000 7faefbab8640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-01-24T23:26:23.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.759+0000 7faefbab8640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-01-24T23:26:23.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.759+0000 7faefbab8640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:23.762 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:23.759+0000 7faefbab8640 1 -- 172.21.15.132:0/1317339943 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) v1 -- 0x7faedc005340 con 0x7faef4133b50 2024-01-24T23:26:24.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:24.375+0000 7faee67fc640 1 -- 172.21.15.132:0/1317339943 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v166) v1 ==== 78+0+0 (unknown 1523177340 0 0) 0x7faee8012ed0 con 0x7faef4133b50 2024-01-24T23:26:24.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:24.375+0000 7faee67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-01-24T23:26:24.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:24.375+0000 7faee67fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:26:24.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:24.487+0000 7faefbab8640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-01-24T23:26:24.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:24.487+0000 7faefbab8640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-01-24T23:26:24.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:24.487+0000 7faefbab8640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:24.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:24.487+0000 7faefbab8640 1 -- 172.21.15.132:0/1317339943 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) v1 -- 0x7faedc002220 con 0x7faef4133b50 2024-01-24T23:26:25.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.387+0000 7faee67fc640 1 -- 172.21.15.132:0/1317339943 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v167) v1 ==== 78+0+0 (unknown 453549401 0 0) 0x7faee8014650 con 0x7faef4133b50 2024-01-24T23:26:25.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.387+0000 7faee67fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-01-24T23:26:25.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.387+0000 7faee67fc640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:26:25.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.391+0000 7faefbab8640 1 -- 172.21.15.132:0/1317339943 >> v1:172.21.15.132:6812/1835401922 conn(0x7faed8036f70 legacy=0x7faed8039430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:25.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.391+0000 7faefbab8640 10 monclient: shutdown 2024-01-24T23:26:25.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.391+0000 7faefbab8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:25.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.391+0000 7faefbab8640 1 -- 172.21.15.132:0/1317339943 >> v1:172.21.15.132:6791/0 conn(0x7faef4133b50 legacy=0x7faef41d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:25.399 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.399+0000 7faefbab8640 1 -- 172.21.15.132:0/1317339943 shutdown_connections 2024-01-24T23:26:25.400 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.399+0000 7faefbab8640 1 -- 172.21.15.132:0/1317339943 >> 172.21.15.132:0/1317339943 conn(0x7faef40a5a10 msgr2=0x7faef4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:25.400 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.399+0000 7faefbab8640 1 -- 172.21.15.132:0/1317339943 shutdown_connections 2024-01-24T23:26:25.402 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.399+0000 7faefbab8640 1 -- 172.21.15.132:0/1317339943 wait complete. 2024-01-24T23:26:25.427 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set create-compat 2024-01-24T23:26:25.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f855149c640 10 monclient: get_monmap_and_config 2024-01-24T23:26:25.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f855149c640 10 monclient: build_initial_monmap 2024-01-24T23:26:25.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f855149c640 10 monclient: monmap: 2024-01-24T23:26:25.544 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:25.544 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:25.545 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:25.545787+0000 2024-01-24T23:26:25.545 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:25.545787+0000 2024-01-24T23:26:25.545 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:25.545 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:25.545 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:25.545 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:25.545 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:25.546 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:25.546 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:25.546 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:25.546 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:25.546 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:25.546 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:25.546 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:25.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f855149c640 1 Processor -- start 2024-01-24T23:26:25.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f855149c640 1 -- start start 2024-01-24T23:26:25.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f855149c640 10 monclient: init 2024-01-24T23:26:25.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f855149c640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:25.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f855149c640 10 monclient: _add_conns ranks=[0,2,1,7,8,4,6,5,3] 2024-01-24T23:26:25.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f855149c640 10 monclient(hunting): picked mon.noname-a con 0x7f854c09ff20 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:25.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f855149c640 10 monclient(hunting): picked mon.noname-b con 0x7f854c09f9c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:25.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f855149c640 10 monclient(hunting): picked mon.noname-f con 0x7f854c13b2c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:25.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f855149c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f854c064700 con 0x7f854c09ff20 2024-01-24T23:26:25.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f855149c640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f854c09e760 con 0x7f854c13b2c0 2024-01-24T23:26:25.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f855149c640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f854c0a1160 con 0x7f854c09f9c0 2024-01-24T23:26:25.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f855149c640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:25.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f855149c640 10 monclient(hunting): authenticate will time out at 1272.870396s 2024-01-24T23:26:25.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f854b7fe640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f854c13b2c0 legacy=0x7f854c13d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:25.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f854a7fc640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f854c09f9c0 legacy=0x7f854c139ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:25.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f854affd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f854c09ff20 0x7f854c0a0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:40396/0 (socket says 172.21.15.132:40396) 2024-01-24T23:26:25.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f854affd640 1 -- 172.21.15.132:0/3019202788 learned_addr learned my addr 172.21.15.132:0/3019202788 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:25.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f8549ffb640 1 -- 172.21.15.132:0/3019202788 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 253025724 0 0) 0x7f854c064700 con 0x7f854c09ff20 2024-01-24T23:26:25.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f8549ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:26:25.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f8549ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:25.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f8549ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:25.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f8549ffb640 10 monclient(hunting): my global_id is 94599 2024-01-24T23:26:25.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f8549ffb640 1 -- 172.21.15.132:0/3019202788 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8534003530 con 0x7f854c09ff20 2024-01-24T23:26:25.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f8549ffb640 1 -- 172.21.15.132:0/3019202788 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 851990290 0 0) 0x7f8534003530 con 0x7f854c09ff20 2024-01-24T23:26:25.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f8549ffb640 1 -- 172.21.15.132:0/3019202788 >> v1:172.21.15.115:6790/0 conn(0x7f854c09f9c0 legacy=0x7f854c139ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:25.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f8549ffb640 1 -- 172.21.15.132:0/3019202788 >> v1:172.21.15.132:6789/0 conn(0x7f854c13b2c0 legacy=0x7f854c13d6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:25.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f8549ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:26:25.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f8549ffb640 1 monclient: found mon.noname-a 2024-01-24T23:26:25.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f8549ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:25.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f8549ffb640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:26:25.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f8549ffb640 1 -- 172.21.15.132:0/3019202788 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854c0a0a10 con 0x7f854c09ff20 2024-01-24T23:26:25.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f8549ffb640 10 monclient: _finish_auth 0 2024-01-24T23:26:25.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f8549ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:26:25.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f8549ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:25.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f8549ffb640 1 -- 172.21.15.132:0/3019202788 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f853c0017c0 con 0x7f854c09ff20 2024-01-24T23:26:25.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f8549ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:25.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f8549ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:26:25.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f8549ffb640 10 monclient: dump: 2024-01-24T23:26:25.556 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:25.556 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:25.556 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:25.556 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:25.556 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:25.556 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:25.557 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:25.557 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:25.557 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:25.557 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:25.557 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:25.557 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:25.557 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:25.557 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:25.557 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:25.557 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:25.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f855149c640 5 monclient: authenticate success, global_id 94599 2024-01-24T23:26:25.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.543+0000 7f855149c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:25.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f8549ffb640 1 -- 172.21.15.132:0/3019202788 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f853c001ae0 con 0x7f854c09ff20 2024-01-24T23:26:25.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f8549ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:25.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:25.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 4 set_mon_vals no callback set 2024-01-24T23:26:25.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f8549ffb640 1 -- 172.21.15.132:0/3019202788 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f853c004bf0 con 0x7f854c09ff20 2024-01-24T23:26:25.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f8549ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:25.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f8549ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:26:25.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f8549ffb640 10 monclient: dump: 2024-01-24T23:26:25.558 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:25.558 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:25.558 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:25.558 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:25.558 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:25.558 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:25.558 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:25.558 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:25.558 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:25.558 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:25.559 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:25.559 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:25.559 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:25.559 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:25.559 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:25.559 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:25.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 10 monclient: shutdown 2024-01-24T23:26:25.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:25.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 1 -- 172.21.15.132:0/3019202788 >> v1:172.21.15.115:6789/0 conn(0x7f854c09ff20 legacy=0x7f854c0a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:25.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 1 -- 172.21.15.132:0/3019202788 shutdown_connections 2024-01-24T23:26:25.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 1 -- 172.21.15.132:0/3019202788 >> 172.21.15.132:0/3019202788 conn(0x7f854c0a5a10 msgr2=0x7f854c0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:25.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 1 -- 172.21.15.132:0/3019202788 shutdown_connections 2024-01-24T23:26:25.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 1 -- 172.21.15.132:0/3019202788 wait complete. 2024-01-24T23:26:25.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 10 monclient: build_initial_monmap 2024-01-24T23:26:25.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 10 monclient: monmap: 2024-01-24T23:26:25.559 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:25.560 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:25.560 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:25.560 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:25.560 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:25.560 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:25.560 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:25.560 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:25.560 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:25.560 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:25.560 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:25.560 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:25.561 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:25.561 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:25.561 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:25.561 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:25.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 1 Processor -- start 2024-01-24T23:26:25.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 1 -- start start 2024-01-24T23:26:25.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 10 monclient: init 2024-01-24T23:26:25.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:25.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 10 monclient: _add_conns ranks=[5,2,4,0,1,6,8,7,3] 2024-01-24T23:26:25.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 10 monclient(hunting): picked mon.noname-h con 0x7f854c09f9c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:25.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 10 monclient(hunting): picked mon.noname-b con 0x7f854c09ff20 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:25.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 10 monclient(hunting): picked mon.noname-c con 0x7f854c13b2c0 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:25.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f854c121110 con 0x7f854c09ff20 2024-01-24T23:26:25.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f854c071680 con 0x7f854c13b2c0 2024-01-24T23:26:25.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f854c122f00 con 0x7f854c09f9c0 2024-01-24T23:26:25.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:25.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 10 monclient(hunting): authenticate will time out at 1272.874065s 2024-01-24T23:26:25.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f854affd640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f854c09f9c0 0x7f854c11eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:46492/0 (socket says 172.21.15.132:46492) 2024-01-24T23:26:25.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f854affd640 1 -- 172.21.15.132:0/2431094584 learned_addr learned my addr 172.21.15.132:0/2431094584 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:25.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f854a7fc640 1 -- 172.21.15.132:0/2431094584 >> v1:172.21.15.115:6790/0 conn(0x7f854c09ff20 legacy=0x7f854c11f5e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:25.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 1 -- 172.21.15.132:0/2431094584 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 561360008 0 0) 0x7f854c122f00 con 0x7f854c09f9c0 2024-01-24T23:26:25.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 10 monclient(hunting): _negotiate 2024-01-24T23:26:25.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:25.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:25.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 10 monclient(hunting): my global_id is 6179 2024-01-24T23:26:25.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 1 -- 172.21.15.132:0/2431094584 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8538003560 con 0x7f854c09f9c0 2024-01-24T23:26:25.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 1 -- 172.21.15.132:0/2431094584 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 633580915 0 0) 0x7f8538003560 con 0x7f854c09f9c0 2024-01-24T23:26:25.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 1 -- 172.21.15.132:0/2431094584 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f854c122f00 con 0x7f854c09f9c0 2024-01-24T23:26:25.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 1 -- 172.21.15.132:0/2431094584 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f853c001780 con 0x7f854c09f9c0 2024-01-24T23:26:25.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:25.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:25.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 10 monclient(hunting): dump: 2024-01-24T23:26:25.563 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:25.563 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:25.563 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:25.563 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:25.563 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:25.563 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:25.563 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:25.563 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:25.563 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:25.563 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:25.563 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:25.563 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:25.563 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:25.563 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:25.564 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:25.564 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:25.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:25.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 1 -- 172.21.15.132:0/2431094584 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666705413 0 0) 0x7f854c071680 con 0x7f854c13b2c0 2024-01-24T23:26:25.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 10 monclient(hunting): _negotiate 2024-01-24T23:26:25.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:25.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:25.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 10 monclient(hunting): my global_id is 94477 2024-01-24T23:26:25.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 1 -- 172.21.15.132:0/2431094584 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8538003560 con 0x7f854c13b2c0 2024-01-24T23:26:25.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 1 -- 172.21.15.132:0/2431094584 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1663437863 0 0) 0x7f854c122f00 con 0x7f854c09f9c0 2024-01-24T23:26:25.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 1 -- 172.21.15.132:0/2431094584 >> v1:172.21.15.115:6791/0 conn(0x7f854c13b2c0 legacy=0x7f854c1d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:25.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 1 -- 172.21.15.132:0/2431094584 >> v1:172.21.15.115:6790/0 conn(0x7f854c09ff20 legacy=0x7f854c11f5e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:25.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 10 monclient: _finish_hunting 0 2024-01-24T23:26:25.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 1 monclient: found mon.h 2024-01-24T23:26:25.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:25.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:25.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 1 -- 172.21.15.132:0/2431094584 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854c071960 con 0x7f854c09f9c0 2024-01-24T23:26:25.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 10 monclient: _finish_auth 0 2024-01-24T23:26:25.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 10 monclient: _check_auth_tickets 2024-01-24T23:26:25.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:25.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 5 monclient: authenticate success, global_id 6179 2024-01-24T23:26:25.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 10 monclient: _renew_subs 2024-01-24T23:26:25.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:25.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 1 -- 172.21.15.132:0/2431094584 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f854c122ae0 con 0x7f854c09f9c0 2024-01-24T23:26:25.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 10 monclient: _renew_subs 2024-01-24T23:26:25.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:25.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 1 -- 172.21.15.132:0/2431094584 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f854c136540 con 0x7f854c09f9c0 2024-01-24T23:26:25.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 1 -- 172.21.15.132:0/2431094584 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f853c001c50 con 0x7f854c09f9c0 2024-01-24T23:26:25.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:25.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 1 -- 172.21.15.132:0/2431094584 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f853c0054b0 con 0x7f854c09f9c0 2024-01-24T23:26:25.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:25.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85497fa640 4 set_mon_vals no callback set 2024-01-24T23:26:25.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:25.566 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 10 monclient: dump: 2024-01-24T23:26:25.566 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:25.566 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:25.566 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:25.566 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:25.566 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:25.567 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:25.567 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:25.567 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:25.567 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:25.567 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:25.567 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:25.567 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:25.567 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:25.567 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:25.567 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:25.567 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:25.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:25.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:25.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:25.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f85337fe640 1 -- 172.21.15.132:0/2431094584 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f853c00ff30 con 0x7f854c09f9c0 2024-01-24T23:26:25.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.547+0000 7f855149c640 1 -- 172.21.15.132:0/2431094584 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f852c005090 con 0x7f854c09f9c0 2024-01-24T23:26:25.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.551+0000 7f85337fe640 1 -- 172.21.15.132:0/2431094584 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(167..167 src has 1..167) v4 ==== 6667+0+0 (unknown 2306116782 0 0) 0x7f853c044610 con 0x7f854c09f9c0 2024-01-24T23:26:25.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.551+0000 7f854a7fc640 10 monclient: get_auth_request con 0x7f8538036fa0 auth_method 0 2024-01-24T23:26:25.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.555+0000 7f85337fe640 1 -- 172.21.15.132:0/2431094584 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f853c014c30 con 0x7f854c09f9c0 2024-01-24T23:26:25.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.555+0000 7f85337fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:25.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.555+0000 7f85337fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:25.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.715+0000 7f855149c640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-01-24T23:26:25.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.715+0000 7f855149c640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-01-24T23:26:25.719 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.715+0000 7f855149c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:25.719 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:25.715+0000 7f855149c640 1 -- 172.21.15.132:0/2431094584 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) v1 -- 0x7f852c005340 con 0x7f854c09f9c0 2024-01-24T23:26:26.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.387+0000 7f85337fe640 1 -- 172.21.15.132:0/2431094584 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 v168) v1 ==== 82+0+0 (unknown 656893846 0 0) 0x7f853c0132d0 con 0x7f854c09f9c0 2024-01-24T23:26:26.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.387+0000 7f85337fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-01-24T23:26:26.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.387+0000 7f85337fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:26:26.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.499+0000 7f855149c640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-01-24T23:26:26.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.499+0000 7f855149c640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-01-24T23:26:26.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.499+0000 7f855149c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:26.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.499+0000 7f855149c640 1 -- 172.21.15.132:0/2431094584 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) v1 -- 0x7f852c002220 con 0x7f854c09f9c0 2024-01-24T23:26:26.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.499+0000 7f85337fe640 1 -- 172.21.15.132:0/2431094584 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 compat weight-set already created v168) v1 ==== 115+0+0 (unknown 2620728590 0 0) 0x7f853c014a50 con 0x7f854c09f9c0 2024-01-24T23:26:26.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.499+0000 7f85337fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-01-24T23:26:26.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.499+0000 7f85337fe640 10 monclient: _finish_command 3 = system:0 compat weight-set already created 2024-01-24T23:26:26.503 INFO:tasks.workunit.client.0.smithi132.stderr:compat weight-set already created 2024-01-24T23:26:26.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.503+0000 7f855149c640 1 -- 172.21.15.132:0/2431094584 >> v1:172.21.15.132:6812/1835401922 conn(0x7f8538036fa0 legacy=0x7f8538039460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:26.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.503+0000 7f855149c640 10 monclient: shutdown 2024-01-24T23:26:26.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.503+0000 7f855149c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:26.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.503+0000 7f855149c640 1 -- 172.21.15.132:0/2431094584 >> v1:172.21.15.132:6791/0 conn(0x7f854c09f9c0 legacy=0x7f854c11eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:26.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.503+0000 7f855149c640 1 -- 172.21.15.132:0/2431094584 shutdown_connections 2024-01-24T23:26:26.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.503+0000 7f855149c640 1 -- 172.21.15.132:0/2431094584 >> 172.21.15.132:0/2431094584 conn(0x7f854c0a5a10 msgr2=0x7f854c139fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:26.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.503+0000 7f855149c640 1 -- 172.21.15.132:0/2431094584 shutdown_connections 2024-01-24T23:26:26.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.503+0000 7f855149c640 1 -- 172.21.15.132:0/2431094584 wait complete. 2024-01-24T23:26:26.521 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket fooo host 2024-01-24T23:26:26.640 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4bff2a2640 10 monclient: get_monmap_and_config 2024-01-24T23:26:26.640 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4bff2a2640 10 monclient: build_initial_monmap 2024-01-24T23:26:26.640 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4bff2a2640 10 monclient: monmap: 2024-01-24T23:26:26.640 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:26.640 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:26.640 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:26.641460+0000 2024-01-24T23:26:26.641 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:26.641460+0000 2024-01-24T23:26:26.641 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:26.641 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:26.641 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:26.641 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:26.641 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:26.641 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:26.641 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:26.641 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:26.641 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:26.641 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:26.641 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:26.641 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:26.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4bff2a2640 1 Processor -- start 2024-01-24T23:26:26.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4bff2a2640 1 -- start start 2024-01-24T23:26:26.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4bff2a2640 10 monclient: init 2024-01-24T23:26:26.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4bff2a2640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:26.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4bff2a2640 10 monclient: _add_conns ranks=[7,1,0,6,2,5,4,8,3] 2024-01-24T23:26:26.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4bff2a2640 10 monclient(hunting): picked mon.noname-i con 0x7f4bf80740e0 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:26.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4bff2a2640 10 monclient(hunting): picked mon.noname-f con 0x7f4bf8071830 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:26.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4bff2a2640 10 monclient(hunting): picked mon.noname-a con 0x7f4bf8071c90 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:26.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4bff2a2640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf8064700 con 0x7f4bf8071c90 2024-01-24T23:26:26.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4bff2a2640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf80744c0 con 0x7f4bf8071830 2024-01-24T23:26:26.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4bff2a2640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf8130e70 con 0x7f4bf80740e0 2024-01-24T23:26:26.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4bff2a2640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:26.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4bff2a2640 10 monclient(hunting): authenticate will time out at 1273.967064s 2024-01-24T23:26:26.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4bfd03e640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f4bf80740e0 legacy=0x7f4bf8130760 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:26.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4bfc83d640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f4bf8071830 legacy=0x7f4bf8138ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:26.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4bfd83f640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f4bf8071c90 0x7f4bf813c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:40398/0 (socket says 172.21.15.132:40398) 2024-01-24T23:26:26.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4bfd83f640 1 -- 172.21.15.132:0/1495126717 learned_addr learned my addr 172.21.15.132:0/1495126717 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:26.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4beffff640 1 -- 172.21.15.132:0/1495126717 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1150228076 0 0) 0x7f4bf8064700 con 0x7f4bf8071c90 2024-01-24T23:26:26.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4beffff640 10 monclient(hunting): _negotiate 2024-01-24T23:26:26.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4beffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:26.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4beffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:26.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4beffff640 10 monclient(hunting): my global_id is 94608 2024-01-24T23:26:26.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.639+0000 7f4beffff640 1 -- 172.21.15.132:0/1495126717 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4be4005740 con 0x7f4bf8071c90 2024-01-24T23:26:26.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4beffff640 1 -- 172.21.15.132:0/1495126717 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4235795923 0 0) 0x7f4be4005740 con 0x7f4bf8071c90 2024-01-24T23:26:26.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4beffff640 1 -- 172.21.15.132:0/1495126717 >> v1:172.21.15.132:6792/0 conn(0x7f4bf80740e0 legacy=0x7f4bf8130760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:26.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4beffff640 1 -- 172.21.15.132:0/1495126717 >> v1:172.21.15.132:6789/0 conn(0x7f4bf8071830 legacy=0x7f4bf8138ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:26.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4beffff640 10 monclient: _finish_hunting 0 2024-01-24T23:26:26.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4beffff640 1 monclient: found mon.noname-a 2024-01-24T23:26:26.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4beffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:26.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4beffff640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:26:26.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4beffff640 1 -- 172.21.15.132:0/1495126717 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf80720e0 con 0x7f4bf8071c90 2024-01-24T23:26:26.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4beffff640 10 monclient: _finish_auth 0 2024-01-24T23:26:26.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4beffff640 10 monclient: _check_auth_tickets 2024-01-24T23:26:26.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4beffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:26.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4beffff640 1 -- 172.21.15.132:0/1495126717 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4bf0001b40 con 0x7f4bf8071c90 2024-01-24T23:26:26.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4beffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:26.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4beffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:26:26.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4beffff640 10 monclient: dump: 2024-01-24T23:26:26.653 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:26.653 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:26.654 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:26.654 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:26.654 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:26.654 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:26.654 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:26.654 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:26.654 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:26.654 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:26.654 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:26.654 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:26.654 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:26.654 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:26.654 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:26.654 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:26.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 5 monclient: authenticate success, global_id 94608 2024-01-24T23:26:26.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:26.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4beffff640 1 -- 172.21.15.132:0/1495126717 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4bf0001d60 con 0x7f4bf8071c90 2024-01-24T23:26:26.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4beffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:26.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4beffff640 1 -- 172.21.15.132:0/1495126717 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4bf0004c30 con 0x7f4bf8071c90 2024-01-24T23:26:26.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:26.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 4 set_mon_vals no callback set 2024-01-24T23:26:26.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4beffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:26.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4beffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:26:26.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4beffff640 10 monclient: dump: 2024-01-24T23:26:26.655 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:26.655 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:26.655 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:26.655 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:26.655 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:26.655 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:26.656 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:26.656 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:26.656 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:26.656 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:26.656 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:26.656 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:26.656 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:26.656 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:26.656 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:26.656 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:26.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 10 monclient: shutdown 2024-01-24T23:26:26.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:26.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 1 -- 172.21.15.132:0/1495126717 >> v1:172.21.15.115:6789/0 conn(0x7f4bf8071c90 legacy=0x7f4bf813c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:26.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 1 -- 172.21.15.132:0/1495126717 shutdown_connections 2024-01-24T23:26:26.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 1 -- 172.21.15.132:0/1495126717 >> 172.21.15.132:0/1495126717 conn(0x7f4bf80a5f40 msgr2=0x7f4bf80a2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:26.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 1 -- 172.21.15.132:0/1495126717 shutdown_connections 2024-01-24T23:26:26.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 1 -- 172.21.15.132:0/1495126717 wait complete. 2024-01-24T23:26:26.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 10 monclient: build_initial_monmap 2024-01-24T23:26:26.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 10 monclient: monmap: 2024-01-24T23:26:26.657 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:26.657 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:26.657 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:26.657 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:26.657 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:26.657 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:26.657 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:26.657 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:26.657 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:26.657 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:26.657 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:26.658 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:26.658 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:26.658 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:26.658 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:26.658 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:26.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 1 Processor -- start 2024-01-24T23:26:26.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 1 -- start start 2024-01-24T23:26:26.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 10 monclient: init 2024-01-24T23:26:26.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:26.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 10 monclient: _add_conns ranks=[2,3,8,6,4,1,0,7,5] 2024-01-24T23:26:26.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 10 monclient(hunting): picked mon.noname-b con 0x7f4bf8071830 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:26.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 10 monclient(hunting): picked mon.noname-g con 0x7f4bf8071c90 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:26.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 10 monclient(hunting): picked mon.noname-e con 0x7f4bf80740e0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:26.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf806f4d0 con 0x7f4bf8071830 2024-01-24T23:26:26.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf8121100 con 0x7f4bf8071c90 2024-01-24T23:26:26.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf809ea80 con 0x7f4bf80740e0 2024-01-24T23:26:26.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:26.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 10 monclient(hunting): authenticate will time out at 1273.970671s 2024-01-24T23:26:26.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bfd03e640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f4bf8071830 legacy=0x7f4bf812d6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:26.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bfc83d640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f4bf8071c90 0x7f4bf81d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:45014/0 (socket says 172.21.15.132:45014) 2024-01-24T23:26:26.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bfc83d640 1 -- 172.21.15.132:0/2726324485 learned_addr learned my addr 172.21.15.132:0/2726324485 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:26.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 1 -- 172.21.15.132:0/2726324485 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3869893245 0 0) 0x7f4bf8121100 con 0x7f4bf8071c90 2024-01-24T23:26:26.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 10 monclient(hunting): _negotiate 2024-01-24T23:26:26.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:26.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:26.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 10 monclient(hunting): my global_id is 6303 2024-01-24T23:26:26.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 1 -- 172.21.15.132:0/2726324485 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bdc003530 con 0x7f4bf8071c90 2024-01-24T23:26:26.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 1 -- 172.21.15.132:0/2726324485 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 722647862 0 0) 0x7f4bdc003530 con 0x7f4bf8071c90 2024-01-24T23:26:26.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 1 -- 172.21.15.132:0/2726324485 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4bf8121100 con 0x7f4bf8071c90 2024-01-24T23:26:26.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 1 -- 172.21.15.132:0/2726324485 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4be80042f0 con 0x7f4bf8071c90 2024-01-24T23:26:26.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:26.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:26.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 10 monclient(hunting): dump: 2024-01-24T23:26:26.660 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:26.660 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:26.660 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:26.660 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:26.660 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:26.660 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:26.660 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:26.660 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:26.660 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:26.660 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:26.660 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:26.660 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:26.661 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:26.661 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:26.661 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:26.661 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:26.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:26.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 1 -- 172.21.15.132:0/2726324485 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1535985320 0 0) 0x7f4bf809ea80 con 0x7f4bf80740e0 2024-01-24T23:26:26.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 10 monclient(hunting): _negotiate 2024-01-24T23:26:26.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:26.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:26.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 10 monclient(hunting): my global_id is 85508 2024-01-24T23:26:26.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 1 -- 172.21.15.132:0/2726324485 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bdc003530 con 0x7f4bf80740e0 2024-01-24T23:26:26.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 1 -- 172.21.15.132:0/2726324485 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4268890919 0 0) 0x7f4bf8121100 con 0x7f4bf8071c90 2024-01-24T23:26:26.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 1 -- 172.21.15.132:0/2726324485 >> v1:172.21.15.115:6793/0 conn(0x7f4bf80740e0 legacy=0x7f4bf81d3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:26.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 1 -- 172.21.15.132:0/2726324485 >> v1:172.21.15.115:6790/0 conn(0x7f4bf8071830 legacy=0x7f4bf812d6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:26.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 10 monclient: _finish_hunting 0 2024-01-24T23:26:26.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 1 monclient: found mon.g 2024-01-24T23:26:26.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:26.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:26.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 1 -- 172.21.15.132:0/2726324485 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf806efa0 con 0x7f4bf8071c90 2024-01-24T23:26:26.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 10 monclient: _finish_auth 0 2024-01-24T23:26:26.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 10 monclient: _check_auth_tickets 2024-01-24T23:26:26.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:26.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 5 monclient: authenticate success, global_id 6303 2024-01-24T23:26:26.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 10 monclient: _renew_subs 2024-01-24T23:26:26.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:26.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 1 -- 172.21.15.132:0/2726324485 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf809e5d0 con 0x7f4bf8071c90 2024-01-24T23:26:26.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 10 monclient: _renew_subs 2024-01-24T23:26:26.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:26.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bff2a2640 1 -- 172.21.15.132:0/2726324485 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf812ddd0 con 0x7f4bf8071c90 2024-01-24T23:26:26.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 1 -- 172.21.15.132:0/2726324485 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4be80046e0 con 0x7f4bf8071c90 2024-01-24T23:26:26.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:26.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 1 -- 172.21.15.132:0/2726324485 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4be8004f50 con 0x7f4bf8071c90 2024-01-24T23:26:26.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:26.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4beeffd640 4 set_mon_vals no callback set 2024-01-24T23:26:26.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:26.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.643+0000 7f4bedffb640 10 monclient: dump: 2024-01-24T23:26:26.663 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:26.663 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:26.663 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:26.663 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:26.663 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:26.663 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:26.663 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:26.663 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:26.663 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:26.663 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:26.664 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:26.664 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:26.664 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:26.664 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:26.664 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:26.664 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:26.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.647+0000 7f4bff2a2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:26.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.647+0000 7f4bff2a2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:26.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.647+0000 7f4bff2a2640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:26.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.647+0000 7f4bff2a2640 1 -- 172.21.15.132:0/2726324485 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd8005090 con 0x7f4bf8071c90 2024-01-24T23:26:26.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.647+0000 7f4bedffb640 1 -- 172.21.15.132:0/2726324485 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f4be800f9d0 con 0x7f4bf8071c90 2024-01-24T23:26:26.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.647+0000 7f4bedffb640 1 -- 172.21.15.132:0/2726324485 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(168..168 src has 1..168) v4 ==== 6995+0+0 (unknown 2973133171 0 0) 0x7f4be8044600 con 0x7f4bf8071c90 2024-01-24T23:26:26.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.647+0000 7f4bfd03e640 10 monclient: get_auth_request con 0x7f4bdc036f90 auth_method 0 2024-01-24T23:26:26.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.651+0000 7f4bedffb640 1 -- 172.21.15.132:0/2726324485 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f4be80149a0 con 0x7f4bf8071c90 2024-01-24T23:26:26.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.651+0000 7f4bedffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:26.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.651+0000 7f4bedffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:26.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.807+0000 7f4bff2a2640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}] 2024-01-24T23:26:26.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.807+0000 7f4bff2a2640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}] 2024-01-24T23:26:26.810 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.807+0000 7f4bff2a2640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:26.811 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.807+0000 7f4bff2a2640 1 -- 172.21.15.132:0/2726324485 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"} v 0) v1 -- 0x7f4bd80058c0 con 0x7f4bf8071c90 2024-01-24T23:26:26.903 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.899+0000 7f4bedffb640 1 -- 172.21.15.132:0/2726324485 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}]=0 added bucket fooo type host to crush map v169) v1 ==== 140+0+0 (unknown 3988051509 0 0) 0x7f4be8013040 con 0x7f4bf8071c90 2024-01-24T23:26:26.903 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.899+0000 7f4bedffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}] 2024-01-24T23:26:26.903 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:26.899+0000 7f4bedffb640 10 monclient: _finish_command 2 = system:0 added bucket fooo type host to crush map 2024-01-24T23:26:27.004 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.003+0000 7f4bff2a2640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}] 2024-01-24T23:26:27.004 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.003+0000 7f4bff2a2640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}] 2024-01-24T23:26:27.004 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.003+0000 7f4bff2a2640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:27.004 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.003+0000 7f4bff2a2640 1 -- 172.21.15.132:0/2726324485 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"} v 0) v1 -- 0x7f4bd8000bc0 con 0x7f4bf8071c90 2024-01-24T23:26:27.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.003+0000 7f4bedffb640 1 -- 172.21.15.132:0/2726324485 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}]=0 bucket 'fooo' already exists v169) v1 ==== 128+0+0 (unknown 3075989263 0 0) 0x7f4be80147c0 con 0x7f4bf8071c90 2024-01-24T23:26:27.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.003+0000 7f4bedffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}] 2024-01-24T23:26:27.005 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.003+0000 7f4bedffb640 10 monclient: _finish_command 3 = system:0 bucket 'fooo' already exists 2024-01-24T23:26:27.005 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'fooo' already exists 2024-01-24T23:26:27.007 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.003+0000 7f4bff2a2640 1 -- 172.21.15.132:0/2726324485 >> v1:172.21.15.132:6812/1835401922 conn(0x7f4bdc036f90 legacy=0x7f4bdc039450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:27.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.007+0000 7f4bff2a2640 10 monclient: shutdown 2024-01-24T23:26:27.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.007+0000 7f4bff2a2640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:27.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.007+0000 7f4bff2a2640 1 -- 172.21.15.132:0/2726324485 >> v1:172.21.15.132:6790/0 conn(0x7f4bf8071c90 legacy=0x7f4bf81d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:27.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.007+0000 7f4bff2a2640 1 -- 172.21.15.132:0/2726324485 shutdown_connections 2024-01-24T23:26:27.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.007+0000 7f4bff2a2640 1 -- 172.21.15.132:0/2726324485 >> 172.21.15.132:0/2726324485 conn(0x7f4bf80a5f40 msgr2=0x7f4bf80a3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:27.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.007+0000 7f4bff2a2640 1 -- 172.21.15.132:0/2726324485 shutdown_connections 2024-01-24T23:26:27.008 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.007+0000 7f4bff2a2640 1 -- 172.21.15.132:0/2726324485 wait complete. 2024-01-24T23:26:27.022 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush move fooo root=default 2024-01-24T23:26:27.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 10 monclient: get_monmap_and_config 2024-01-24T23:26:27.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 10 monclient: build_initial_monmap 2024-01-24T23:26:27.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 10 monclient: monmap: 2024-01-24T23:26:27.140 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:27.140 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:27.140 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:27.141146+0000 2024-01-24T23:26:27.140 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:27.141146+0000 2024-01-24T23:26:27.140 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:27.140 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:27.140 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:27.140 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:27.140 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:27.140 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:27.140 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:27.141 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:27.141 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:27.141 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:27.141 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:27.141 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:27.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 1 Processor -- start 2024-01-24T23:26:27.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 1 -- start start 2024-01-24T23:26:27.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 10 monclient: init 2024-01-24T23:26:27.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:27.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 10 monclient: _add_conns ranks=[3,8,4,6,2,0,5,1,7] 2024-01-24T23:26:27.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 10 monclient(hunting): picked mon.noname-g con 0x7f6cdc136cb0 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:27.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 10 monclient(hunting): picked mon.noname-e con 0x7f6cdc1377a0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:27.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 10 monclient(hunting): picked mon.noname-c con 0x7f6cdc09bcc0 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:27.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cdc064700 con 0x7f6cdc136cb0 2024-01-24T23:26:27.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cdc09b650 con 0x7f6cdc09bcc0 2024-01-24T23:26:27.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cdc09c110 con 0x7f6cdc1377a0 2024-01-24T23:26:27.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:27.147 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 10 monclient(hunting): authenticate will time out at 1274.465860s 2024-01-24T23:26:27.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce122c640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f6cdc136cb0 0x7f6cdc137090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:45024/0 (socket says 172.21.15.132:45024) 2024-01-24T23:26:27.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce122c640 1 -- 172.21.15.132:0/620506883 learned_addr learned my addr 172.21.15.132:0/620506883 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:27.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 1 -- 172.21.15.132:0/620506883 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3025589086 0 0) 0x7f6cdc064700 con 0x7f6cdc136cb0 2024-01-24T23:26:27.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:26:27.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:27.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:27.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 10 monclient(hunting): my global_id is 6312 2024-01-24T23:26:27.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 1 -- 172.21.15.132:0/620506883 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cc8003530 con 0x7f6cdc136cb0 2024-01-24T23:26:27.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 1 -- 172.21.15.132:0/620506883 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 513390867 0 0) 0x7f6cc8003530 con 0x7f6cdc136cb0 2024-01-24T23:26:27.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 1 -- 172.21.15.132:0/620506883 >> v1:172.21.15.115:6793/0 conn(0x7f6cdc1377a0 legacy=0x7f6cdc13c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:27.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 1 -- 172.21.15.132:0/620506883 >> v1:172.21.15.115:6791/0 conn(0x7f6cdc09bcc0 legacy=0x7f6cdc13f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:27.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 10 monclient: _finish_hunting 0 2024-01-24T23:26:27.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 1 monclient: found mon.noname-g 2024-01-24T23:26:27.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:27.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:26:27.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 1 -- 172.21.15.132:0/620506883 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc09b000 con 0x7f6cdc136cb0 2024-01-24T23:26:27.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 10 monclient: _finish_auth 0 2024-01-24T23:26:27.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 10 monclient: _check_auth_tickets 2024-01-24T23:26:27.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:27.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 1 -- 172.21.15.132:0/620506883 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6cd80017e0 con 0x7f6cdc136cb0 2024-01-24T23:26:27.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:27.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:27.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 10 monclient: dump: 2024-01-24T23:26:27.149 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:27.149 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:27.149 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:27.149 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:27.149 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:27.149 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:27.149 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:27.150 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:27.150 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:27.150 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:27.150 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:27.150 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:27.150 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:27.150 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:27.150 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:27.150 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:27.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 5 monclient: authenticate success, global_id 6312 2024-01-24T23:26:27.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:27.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 1 -- 172.21.15.132:0/620506883 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6cd8001b00 con 0x7f6cdc136cb0 2024-01-24T23:26:27.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:27.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:27.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 1 -- 172.21.15.132:0/620506883 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6cd8004bf0 con 0x7f6cdc136cb0 2024-01-24T23:26:27.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 4 set_mon_vals no callback set 2024-01-24T23:26:27.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:27.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:27.151 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6cd3fff640 10 monclient: dump: 2024-01-24T23:26:27.151 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:27.151 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:27.151 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:27.151 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:27.151 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:27.151 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:27.151 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:27.151 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:27.151 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:27.152 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:27.152 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:27.152 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:27.152 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:27.152 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:27.152 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:27.152 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:27.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 10 monclient: shutdown 2024-01-24T23:26:27.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:27.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 1 -- 172.21.15.132:0/620506883 >> v1:172.21.15.132:6790/0 conn(0x7f6cdc136cb0 legacy=0x7f6cdc137090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:27.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 1 -- 172.21.15.132:0/620506883 shutdown_connections 2024-01-24T23:26:27.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 1 -- 172.21.15.132:0/620506883 >> 172.21.15.132:0/620506883 conn(0x7f6cdc0a9c10 msgr2=0x7f6cdc0a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:27.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 1 -- 172.21.15.132:0/620506883 shutdown_connections 2024-01-24T23:26:27.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 1 -- 172.21.15.132:0/620506883 wait complete. 2024-01-24T23:26:27.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 10 monclient: build_initial_monmap 2024-01-24T23:26:27.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.139+0000 7f6ce3490640 10 monclient: monmap: 2024-01-24T23:26:27.153 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:27.153 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:27.153 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:27.153 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:27.153 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:27.153 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:27.153 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:27.153 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:27.153 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:27.153 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:27.153 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:27.153 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:27.154 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:27.154 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:27.154 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:27.154 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:27.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce3490640 1 Processor -- start 2024-01-24T23:26:27.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce3490640 1 -- start start 2024-01-24T23:26:27.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce3490640 10 monclient: init 2024-01-24T23:26:27.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce3490640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:27.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce3490640 10 monclient: _add_conns ranks=[4,5,3,7,8,0,6,1,2] 2024-01-24T23:26:27.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce3490640 10 monclient(hunting): picked mon.noname-c con 0x7f6cdc09bcc0 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:27.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce3490640 10 monclient(hunting): picked mon.noname-h con 0x7f6cdc136cb0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:27.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce3490640 10 monclient(hunting): picked mon.noname-g con 0x7f6cdc1377a0 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:27.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce3490640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cdc124be0 con 0x7f6cdc1377a0 2024-01-24T23:26:27.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce3490640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cdc125d20 con 0x7f6cdc09bcc0 2024-01-24T23:26:27.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce3490640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cdc126580 con 0x7f6cdc136cb0 2024-01-24T23:26:27.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce3490640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:27.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce3490640 10 monclient(hunting): authenticate will time out at 1274.468741s 2024-01-24T23:26:27.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce1a2d640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f6cdc1377a0 0x7f6cdc1d3e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:45026/0 (socket says 172.21.15.132:45026) 2024-01-24T23:26:27.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce1a2d640 1 -- 172.21.15.132:0/1124151300 learned_addr learned my addr 172.21.15.132:0/1124151300 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:27.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 1 -- 172.21.15.132:0/1124151300 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1311172283 0 0) 0x7f6cdc124be0 con 0x7f6cdc1377a0 2024-01-24T23:26:27.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:26:27.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:27.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:27.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient(hunting): my global_id is 6321 2024-01-24T23:26:27.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 1 -- 172.21.15.132:0/1124151300 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cc0003530 con 0x7f6cdc1377a0 2024-01-24T23:26:27.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 1 -- 172.21.15.132:0/1124151300 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3314956129 0 0) 0x7f6cdc126580 con 0x7f6cdc136cb0 2024-01-24T23:26:27.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:26:27.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:27.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:27.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient(hunting): my global_id is 6188 2024-01-24T23:26:27.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 1 -- 172.21.15.132:0/1124151300 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cdc124be0 con 0x7f6cdc136cb0 2024-01-24T23:26:27.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 1 -- 172.21.15.132:0/1124151300 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3368859438 0 0) 0x7f6cc0003530 con 0x7f6cdc1377a0 2024-01-24T23:26:27.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 1 -- 172.21.15.132:0/1124151300 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6cdc126580 con 0x7f6cdc1377a0 2024-01-24T23:26:27.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 1 -- 172.21.15.132:0/1124151300 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6cd4001e20 con 0x7f6cdc1377a0 2024-01-24T23:26:27.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:27.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:27.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient(hunting): dump: 2024-01-24T23:26:27.156 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:27.156 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:27.157 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:27.157 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:27.157 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:27.157 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:27.157 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:27.157 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:27.157 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:27.157 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:27.157 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:27.157 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:27.157 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:27.157 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:27.157 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:27.157 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:27.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:27.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 1 -- 172.21.15.132:0/1124151300 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2168679997 0 0) 0x7f6cdc124be0 con 0x7f6cdc136cb0 2024-01-24T23:26:27.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 1 -- 172.21.15.132:0/1124151300 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6cc0003530 con 0x7f6cdc136cb0 2024-01-24T23:26:27.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 1 -- 172.21.15.132:0/1124151300 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2766875312 0 0) 0x7f6cdc125d20 con 0x7f6cdc09bcc0 2024-01-24T23:26:27.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:26:27.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:27.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:27.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient(hunting): my global_id is 94495 2024-01-24T23:26:27.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 1 -- 172.21.15.132:0/1124151300 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cdc124be0 con 0x7f6cdc09bcc0 2024-01-24T23:26:27.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 1 -- 172.21.15.132:0/1124151300 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6ccc0018f0 con 0x7f6cdc136cb0 2024-01-24T23:26:27.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:27.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:27.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient(hunting): dump: 2024-01-24T23:26:27.158 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:27.158 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:27.159 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:27.159 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:27.159 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:27.159 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:27.159 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:27.159 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:27.159 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:27.159 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:27.159 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:27.159 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:27.159 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:27.159 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:27.159 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:27.159 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:27.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 1 -- 172.21.15.132:0/1124151300 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2763535222 0 0) 0x7f6cdc126580 con 0x7f6cdc1377a0 2024-01-24T23:26:27.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 1 -- 172.21.15.132:0/1124151300 >> v1:172.21.15.132:6791/0 conn(0x7f6cdc136cb0 legacy=0x7f6cdc1d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:27.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 1 -- 172.21.15.132:0/1124151300 >> v1:172.21.15.115:6791/0 conn(0x7f6cdc09bcc0 legacy=0x7f6cdc135e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:27.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:26:27.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 1 monclient: found mon.g 2024-01-24T23:26:27.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:27.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:27.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 1 -- 172.21.15.132:0/1124151300 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc09c6a0 con 0x7f6cdc1377a0 2024-01-24T23:26:27.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient: _finish_auth 0 2024-01-24T23:26:27.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:26:27.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:27.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce3490640 5 monclient: authenticate success, global_id 6321 2024-01-24T23:26:27.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce3490640 10 monclient: _renew_subs 2024-01-24T23:26:27.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce3490640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:27.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce3490640 1 -- 172.21.15.132:0/1124151300 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cdc1262a0 con 0x7f6cdc1377a0 2024-01-24T23:26:27.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce3490640 10 monclient: _renew_subs 2024-01-24T23:26:27.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce3490640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:27.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce3490640 1 -- 172.21.15.132:0/1124151300 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cdc136750 con 0x7f6cdc1377a0 2024-01-24T23:26:27.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 1 -- 172.21.15.132:0/1124151300 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6cd4003c50 con 0x7f6cdc1377a0 2024-01-24T23:26:27.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:27.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 1 -- 172.21.15.132:0/1124151300 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6cd4004180 con 0x7f6cdc1377a0 2024-01-24T23:26:27.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:27.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd37fe640 4 set_mon_vals no callback set 2024-01-24T23:26:27.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:27.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 10 monclient: dump: 2024-01-24T23:26:27.161 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:27.161 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:27.161 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:27.161 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:27.162 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:27.162 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:27.162 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:27.162 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:27.162 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:27.162 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:27.162 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:27.162 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:27.162 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:27.162 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:27.162 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:27.162 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:27.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce3490640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:27.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce3490640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:27.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce3490640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:27.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce3490640 1 -- 172.21.15.132:0/1124151300 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cbc005090 con 0x7f6cdc1377a0 2024-01-24T23:26:27.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 1 -- 172.21.15.132:0/1124151300 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f6cd40042e0 con 0x7f6cdc1377a0 2024-01-24T23:26:27.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6cd1ffb640 1 -- 172.21.15.132:0/1124151300 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(169..169 src has 1..169) v4 ==== 7023+0+0 (unknown 608604921 0 0) 0x7f6cd4044760 con 0x7f6cdc1377a0 2024-01-24T23:26:27.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.143+0000 7f6ce122c640 10 monclient: get_auth_request con 0x7f6cc00378b0 auth_method 0 2024-01-24T23:26:27.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.147+0000 7f6cd1ffb640 1 -- 172.21.15.132:0/1124151300 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f6cd4014b70 con 0x7f6cdc1377a0 2024-01-24T23:26:27.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.147+0000 7f6cd1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:27.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.147+0000 7f6cd1ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:27.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.303+0000 7f6ce3490640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}] 2024-01-24T23:26:27.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.303+0000 7f6ce3490640 10 monclient: _send_command 2 [{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}] 2024-01-24T23:26:27.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.303+0000 7f6ce3490640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:27.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.303+0000 7f6ce3490640 1 -- 172.21.15.132:0/1124151300 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]} v 0) v1 -- 0x7f6cbc0058c0 con 0x7f6cdc1377a0 2024-01-24T23:26:27.917 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.915+0000 7f6cd1ffb640 1 -- 172.21.15.132:0/1124151300 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}]=0 moved item id -16 name 'fooo' to location {root=default} in crush map v170) v1 ==== 173+0+0 (unknown 699890692 0 0) 0x7f6cd4013210 con 0x7f6cdc1377a0 2024-01-24T23:26:27.918 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.915+0000 7f6cd1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}] 2024-01-24T23:26:27.918 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:27.915+0000 7f6cd1ffb640 10 monclient: _finish_command 2 = system:0 moved item id -16 name 'fooo' to location {root=default} in crush map 2024-01-24T23:26:28.011 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.011+0000 7f6ce3490640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}] 2024-01-24T23:26:28.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.011+0000 7f6ce3490640 10 monclient: _send_command 3 [{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}] 2024-01-24T23:26:28.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.011+0000 7f6ce3490640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:28.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.011+0000 7f6ce3490640 1 -- 172.21.15.132:0/1124151300 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]} v 0) v1 -- 0x7f6cbc000bc0 con 0x7f6cdc1377a0 2024-01-24T23:26:28.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.011+0000 7f6cd1ffb640 1 -- 172.21.15.132:0/1124151300 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}]=0 no need to move item id -16 name 'fooo' to location {root=default} in crush map v170) v1 ==== 183+0+0 (unknown 412290096 0 0) 0x7f6cd4014990 con 0x7f6cdc1377a0 2024-01-24T23:26:28.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.011+0000 7f6cd1ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}] 2024-01-24T23:26:28.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.011+0000 7f6cd1ffb640 10 monclient: _finish_command 3 = system:0 no need to move item id -16 name 'fooo' to location {root=default} in crush map 2024-01-24T23:26:28.013 INFO:tasks.workunit.client.0.smithi132.stderr:no need to move item id -16 name 'fooo' to location {root=default} in crush map 2024-01-24T23:26:28.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.015+0000 7f6ce3490640 1 -- 172.21.15.132:0/1124151300 >> v1:172.21.15.132:6812/1835401922 conn(0x7f6cc00378b0 legacy=0x7f6cc0039d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:28.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.015+0000 7f6ce3490640 10 monclient: shutdown 2024-01-24T23:26:28.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.015+0000 7f6ce3490640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:28.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.015+0000 7f6ce3490640 1 -- 172.21.15.132:0/1124151300 >> v1:172.21.15.132:6790/0 conn(0x7f6cdc1377a0 legacy=0x7f6cdc1d3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:28.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.015+0000 7f6ce3490640 1 -- 172.21.15.132:0/1124151300 shutdown_connections 2024-01-24T23:26:28.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.015+0000 7f6ce3490640 1 -- 172.21.15.132:0/1124151300 >> 172.21.15.132:0/1124151300 conn(0x7f6cdc0a9c10 msgr2=0x7f6cdc0a2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:28.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.015+0000 7f6ce3490640 1 -- 172.21.15.132:0/1124151300 shutdown_connections 2024-01-24T23:26:28.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.015+0000 7f6ce3490640 1 -- 172.21.15.132:0/1124151300 wait complete. 2024-01-24T23:26:28.030 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket barr rack 2024-01-24T23:26:28.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.147+0000 7fb5c409d640 10 monclient: get_monmap_and_config 2024-01-24T23:26:28.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.147+0000 7fb5c409d640 10 monclient: build_initial_monmap 2024-01-24T23:26:28.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.147+0000 7fb5c409d640 10 monclient: monmap: 2024-01-24T23:26:28.150 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:28.150 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:28.150 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:28.151489+0000 2024-01-24T23:26:28.150 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:28.151489+0000 2024-01-24T23:26:28.150 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:28.150 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:28.150 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:28.151 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:28.151 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:28.151 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:28.151 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:28.151 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:28.151 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:28.151 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:28.151 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:28.151 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:28.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.147+0000 7fb5c409d640 1 Processor -- start 2024-01-24T23:26:28.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.147+0000 7fb5c409d640 1 -- start start 2024-01-24T23:26:28.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.147+0000 7fb5c409d640 10 monclient: init 2024-01-24T23:26:28.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:28.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 10 monclient: _add_conns ranks=[6,8,2,3,7,4,0,5,1] 2024-01-24T23:26:28.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 10 monclient(hunting): picked mon.noname-d con 0x7fb5bc135e10 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:28.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 10 monclient(hunting): picked mon.noname-e con 0x7fb5bc1361f0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:28.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 10 monclient(hunting): picked mon.noname-b con 0x7fb5bc1365f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:28.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5bc064700 con 0x7fb5bc1365f0 2024-01-24T23:26:28.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5bc09c3e0 con 0x7fb5bc135e10 2024-01-24T23:26:28.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5bc0a4f90 con 0x7fb5bc1361f0 2024-01-24T23:26:28.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:28.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 10 monclient(hunting): authenticate will time out at 1275.476324s 2024-01-24T23:26:28.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c263a640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fb5bc1365f0 legacy=0x7fb5bc13f600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:28.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c1e39640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fb5bc135e10 0x7fb5bc09bcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:59300/0 (socket says 172.21.15.132:59300) 2024-01-24T23:26:28.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c1e39640 1 -- 172.21.15.132:0/629450989 learned_addr learned my addr 172.21.15.132:0/629450989 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:28.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 1 -- 172.21.15.132:0/629450989 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4215082133 0 0) 0x7fb5bc09c3e0 con 0x7fb5bc135e10 2024-01-24T23:26:28.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 10 monclient(hunting): _negotiate 2024-01-24T23:26:28.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:28.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:28.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 10 monclient(hunting): my global_id is 6378 2024-01-24T23:26:28.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 1 -- 172.21.15.132:0/629450989 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5a8003530 con 0x7fb5bc135e10 2024-01-24T23:26:28.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 1 -- 172.21.15.132:0/629450989 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4107604793 0 0) 0x7fb5bc0a4f90 con 0x7fb5bc1361f0 2024-01-24T23:26:28.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 10 monclient(hunting): _negotiate 2024-01-24T23:26:28.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:28.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:28.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 10 monclient(hunting): my global_id is 85526 2024-01-24T23:26:28.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 1 -- 172.21.15.132:0/629450989 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5bc09c3e0 con 0x7fb5bc1361f0 2024-01-24T23:26:28.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 1 -- 172.21.15.132:0/629450989 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3925595057 0 0) 0x7fb5bc09c3e0 con 0x7fb5bc1361f0 2024-01-24T23:26:28.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 1 -- 172.21.15.132:0/629450989 >> v1:172.21.15.115:6792/0 conn(0x7fb5bc135e10 legacy=0x7fb5bc09bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:28.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 1 -- 172.21.15.132:0/629450989 >> v1:172.21.15.115:6790/0 conn(0x7fb5bc1365f0 legacy=0x7fb5bc13f600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:28.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 10 monclient: _finish_hunting 0 2024-01-24T23:26:28.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 1 monclient: found mon.noname-e 2024-01-24T23:26:28.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:28.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:26:28.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 1 -- 172.21.15.132:0/629450989 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5bc09b010 con 0x7fb5bc1361f0 2024-01-24T23:26:28.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 10 monclient: _finish_auth 0 2024-01-24T23:26:28.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 10 monclient: _check_auth_tickets 2024-01-24T23:26:28.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:28.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 1 -- 172.21.15.132:0/629450989 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb5b00017a0 con 0x7fb5bc1361f0 2024-01-24T23:26:28.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 5 monclient: authenticate success, global_id 85526 2024-01-24T23:26:28.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:28.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:28.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:26:28.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 10 monclient: dump: 2024-01-24T23:26:28.162 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:28.162 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:28.162 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:28.162 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:28.162 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:28.162 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:28.162 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:28.162 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:28.162 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:28.162 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:28.162 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:28.162 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:28.162 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:28.162 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:28.162 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:28.163 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:28.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:28.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 1 -- 172.21.15.132:0/629450989 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb5b0001ac0 con 0x7fb5bc1361f0 2024-01-24T23:26:28.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:28.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 1 -- 172.21.15.132:0/629450989 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb5b0003960 con 0x7fb5bc1361f0 2024-01-24T23:26:28.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:28.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 4 set_mon_vals no callback set 2024-01-24T23:26:28.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:28.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:26:28.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c0e37640 10 monclient: dump: 2024-01-24T23:26:28.163 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:28.163 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:28.163 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:28.164 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:28.164 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:28.164 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:28.164 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:28.164 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:28.164 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:28.164 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:28.164 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:28.164 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:28.165 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:28.165 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:28.165 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:28.165 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:28.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 10 monclient: shutdown 2024-01-24T23:26:28.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:28.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 1 -- 172.21.15.132:0/629450989 >> v1:172.21.15.115:6793/0 conn(0x7fb5bc1361f0 legacy=0x7fb5bc13bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:28.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 1 -- 172.21.15.132:0/629450989 shutdown_connections 2024-01-24T23:26:28.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 1 -- 172.21.15.132:0/629450989 >> 172.21.15.132:0/629450989 conn(0x7fb5bc0a6790 msgr2=0x7fb5bc071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:28.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 1 -- 172.21.15.132:0/629450989 shutdown_connections 2024-01-24T23:26:28.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 1 -- 172.21.15.132:0/629450989 wait complete. 2024-01-24T23:26:28.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 10 monclient: build_initial_monmap 2024-01-24T23:26:28.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 10 monclient: monmap: 2024-01-24T23:26:28.165 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:28.165 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:28.166 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:28.166 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:28.166 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:28.166 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:28.166 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:28.166 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:28.166 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:28.166 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:28.166 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:28.166 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:28.166 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:28.166 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:28.166 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:28.166 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:28.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 1 Processor -- start 2024-01-24T23:26:28.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 1 -- start start 2024-01-24T23:26:28.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 10 monclient: init 2024-01-24T23:26:28.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:28.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 10 monclient: _add_conns ranks=[0,3,2,6,4,7,8,5,1] 2024-01-24T23:26:28.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 10 monclient(hunting): picked mon.noname-a con 0x7fb5bc135e10 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:28.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 10 monclient(hunting): picked mon.noname-g con 0x7fb5bc1361f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:28.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 10 monclient(hunting): picked mon.noname-b con 0x7fb5bc1365f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:28.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5bc121b20 con 0x7fb5bc135e10 2024-01-24T23:26:28.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5bc09c6b0 con 0x7fb5bc1365f0 2024-01-24T23:26:28.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5bc1bf180 con 0x7fb5bc1361f0 2024-01-24T23:26:28.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:28.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.151+0000 7fb5c409d640 10 monclient(hunting): authenticate will time out at 1275.479952s 2024-01-24T23:26:28.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5c263a640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fb5bc1365f0 legacy=0x7fb5bc1d3820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:28.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5c1638640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fb5bc1361f0 0x7fb5bc1d00f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:45030/0 (socket says 172.21.15.132:45030) 2024-01-24T23:26:28.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5c1638640 1 -- 172.21.15.132:0/129042431 learned_addr learned my addr 172.21.15.132:0/129042431 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:28.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 1 -- 172.21.15.132:0/129042431 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1489104145 0 0) 0x7fb5bc1bf180 con 0x7fb5bc1361f0 2024-01-24T23:26:28.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:26:28.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:28.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:28.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 10 monclient(hunting): my global_id is 6330 2024-01-24T23:26:28.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 1 -- 172.21.15.132:0/129042431 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5a0003530 con 0x7fb5bc1361f0 2024-01-24T23:26:28.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 1 -- 172.21.15.132:0/129042431 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3988352365 0 0) 0x7fb5a0003530 con 0x7fb5bc1361f0 2024-01-24T23:26:28.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 1 -- 172.21.15.132:0/129042431 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb5bc1bf180 con 0x7fb5bc1361f0 2024-01-24T23:26:28.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 1 -- 172.21.15.132:0/129042431 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb5b0001760 con 0x7fb5bc1361f0 2024-01-24T23:26:28.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:28.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:28.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 10 monclient(hunting): dump: 2024-01-24T23:26:28.169 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:28.169 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:28.169 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:28.169 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:28.169 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:28.169 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:28.169 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:28.169 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:28.169 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:28.169 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:28.169 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:28.169 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:28.169 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:28.169 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:28.169 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:28.169 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:28.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:28.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 1 -- 172.21.15.132:0/129042431 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 905949597 0 0) 0x7fb5bc121b20 con 0x7fb5bc135e10 2024-01-24T23:26:28.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:26:28.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:28.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:28.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 10 monclient(hunting): my global_id is 94617 2024-01-24T23:26:28.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 1 -- 172.21.15.132:0/129042431 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5a0003530 con 0x7fb5bc135e10 2024-01-24T23:26:28.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 1 -- 172.21.15.132:0/129042431 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3855920333 0 0) 0x7fb5bc1bf180 con 0x7fb5bc1361f0 2024-01-24T23:26:28.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 1 -- 172.21.15.132:0/129042431 >> v1:172.21.15.115:6790/0 conn(0x7fb5bc1365f0 legacy=0x7fb5bc1d3820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:28.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 1 -- 172.21.15.132:0/129042431 >> v1:172.21.15.115:6789/0 conn(0x7fb5bc135e10 legacy=0x7fb5bc134ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:28.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:26:28.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 1 monclient: found mon.g 2024-01-24T23:26:28.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:28.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:28.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 1 -- 172.21.15.132:0/129042431 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5bc121e00 con 0x7fb5bc1361f0 2024-01-24T23:26:28.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 10 monclient: _finish_auth 0 2024-01-24T23:26:28.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:26:28.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:28.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5c409d640 5 monclient: authenticate success, global_id 6330 2024-01-24T23:26:28.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5c409d640 10 monclient: _renew_subs 2024-01-24T23:26:28.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5c409d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:28.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5c409d640 1 -- 172.21.15.132:0/129042431 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5bc1220e0 con 0x7fb5bc1361f0 2024-01-24T23:26:28.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5c409d640 10 monclient: _renew_subs 2024-01-24T23:26:28.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5c409d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:28.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5c409d640 1 -- 172.21.15.132:0/129042431 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5bc1be860 con 0x7fb5bc1361f0 2024-01-24T23:26:28.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 1 -- 172.21.15.132:0/129042431 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb5b0001cf0 con 0x7fb5bc1361f0 2024-01-24T23:26:28.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:28.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 1 -- 172.21.15.132:0/129042431 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb5b00042a0 con 0x7fb5bc1361f0 2024-01-24T23:26:28.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:28.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:28.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5af7fe640 4 set_mon_vals no callback set 2024-01-24T23:26:28.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 10 monclient: dump: 2024-01-24T23:26:28.172 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:28.172 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:28.172 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:28.172 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:28.172 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:28.172 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:28.172 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:28.172 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:28.172 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:28.172 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:28.172 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:28.172 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:28.172 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:28.172 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:28.172 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:28.173 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:28.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5c409d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:28.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5c409d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:28.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5c409d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:28.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5c409d640 1 -- 172.21.15.132:0/129042431 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5a4005090 con 0x7fb5bc1361f0 2024-01-24T23:26:28.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 1 -- 172.21.15.132:0/129042431 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fb5b0004400 con 0x7fb5bc1361f0 2024-01-24T23:26:28.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5ae7fc640 1 -- 172.21.15.132:0/129042431 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(170..170 src has 1..170) v4 ==== 7211+0+0 (unknown 1073971303 0 0) 0x7fb5b0044800 con 0x7fb5bc1361f0 2024-01-24T23:26:28.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.155+0000 7fb5c1e39640 10 monclient: get_auth_request con 0x7fb5a00370a0 auth_method 0 2024-01-24T23:26:28.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.159+0000 7fb5ae7fc640 1 -- 172.21.15.132:0/129042431 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fb5b0014bf0 con 0x7fb5bc1361f0 2024-01-24T23:26:28.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.159+0000 7fb5ae7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:28.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.159+0000 7fb5ae7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:28.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.315+0000 7fb5c409d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}] 2024-01-24T23:26:28.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.315+0000 7fb5c409d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}] 2024-01-24T23:26:28.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.315+0000 7fb5c409d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:28.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.315+0000 7fb5c409d640 1 -- 172.21.15.132:0/129042431 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"} v 0) v1 -- 0x7fb5a40058c0 con 0x7fb5bc1361f0 2024-01-24T23:26:28.930 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.927+0000 7fb5ae7fc640 1 -- 172.21.15.132:0/129042431 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}]=0 added bucket barr type rack to crush map v171) v1 ==== 140+0+0 (unknown 1971299204 0 0) 0x7fb5b0014a10 con 0x7fb5bc1361f0 2024-01-24T23:26:28.930 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.927+0000 7fb5ae7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}] 2024-01-24T23:26:28.930 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:28.927+0000 7fb5ae7fc640 10 monclient: _finish_command 2 = system:0 added bucket barr type rack to crush map 2024-01-24T23:26:29.027 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.023+0000 7fb5c409d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}] 2024-01-24T23:26:29.027 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.027+0000 7fb5c409d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}] 2024-01-24T23:26:29.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.027+0000 7fb5c409d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:29.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.027+0000 7fb5c409d640 1 -- 172.21.15.132:0/129042431 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"} v 0) v1 -- 0x7fb5a4000bc0 con 0x7fb5bc1361f0 2024-01-24T23:26:29.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.027+0000 7fb5ae7fc640 1 -- 172.21.15.132:0/129042431 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}]=0 bucket 'barr' already exists v171) v1 ==== 128+0+0 (unknown 3965308306 0 0) 0x7fb5b0015750 con 0x7fb5bc1361f0 2024-01-24T23:26:29.029 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.027+0000 7fb5ae7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}] 2024-01-24T23:26:29.029 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.027+0000 7fb5ae7fc640 10 monclient: _finish_command 3 = system:0 bucket 'barr' already exists 2024-01-24T23:26:29.029 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'barr' already exists 2024-01-24T23:26:29.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.027+0000 7fb5c409d640 1 -- 172.21.15.132:0/129042431 >> v1:172.21.15.132:6812/1835401922 conn(0x7fb5a00370a0 legacy=0x7fb5a0039560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:29.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.027+0000 7fb5c409d640 10 monclient: shutdown 2024-01-24T23:26:29.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.027+0000 7fb5c409d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:29.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.027+0000 7fb5c409d640 1 -- 172.21.15.132:0/129042431 >> v1:172.21.15.132:6790/0 conn(0x7fb5bc1361f0 legacy=0x7fb5bc1d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:29.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.027+0000 7fb5c409d640 1 -- 172.21.15.132:0/129042431 shutdown_connections 2024-01-24T23:26:29.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.027+0000 7fb5c409d640 1 -- 172.21.15.132:0/129042431 >> 172.21.15.132:0/129042431 conn(0x7fb5bc0a6790 msgr2=0x7fb5bc06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:29.032 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.031+0000 7fb5c409d640 1 -- 172.21.15.132:0/129042431 shutdown_connections 2024-01-24T23:26:29.032 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.031+0000 7fb5c409d640 1 -- 172.21.15.132:0/129042431 wait complete. 2024-01-24T23:26:29.046 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush move barr root=default 2024-01-24T23:26:29.079 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~25s 2024-01-24T23:26:29.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f28f28640 10 monclient: get_monmap_and_config 2024-01-24T23:26:29.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f28f28640 10 monclient: build_initial_monmap 2024-01-24T23:26:29.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f28f28640 10 monclient: monmap: 2024-01-24T23:26:29.156 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:29.156 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:29.156 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:29.157533+0000 2024-01-24T23:26:29.157 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:29.157533+0000 2024-01-24T23:26:29.157 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:29.157 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:29.157 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:29.157 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:29.157 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:29.157 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:29.157 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:29.157 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:29.157 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:29.157 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:29.157 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:29.157 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:29.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f28f28640 1 Processor -- start 2024-01-24T23:26:29.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f28f28640 1 -- start start 2024-01-24T23:26:29.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f28f28640 10 monclient: init 2024-01-24T23:26:29.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f28f28640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:29.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f28f28640 10 monclient: _add_conns ranks=[0,2,4,7,6,3,1,8,5] 2024-01-24T23:26:29.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f28f28640 10 monclient(hunting): picked mon.noname-a con 0x7f5f2409ff20 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:29.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f28f28640 10 monclient(hunting): picked mon.noname-b con 0x7f5f2409f9c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:29.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f28f28640 10 monclient(hunting): picked mon.noname-c con 0x7f5f2413b2c0 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:29.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f28f28640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f24064700 con 0x7f5f2409ff20 2024-01-24T23:26:29.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f28f28640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f2409e760 con 0x7f5f2409f9c0 2024-01-24T23:26:29.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f28f28640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f240a1160 con 0x7f5f2413b2c0 2024-01-24T23:26:29.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f28f28640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:29.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f28f28640 10 monclient(hunting): authenticate will time out at 1276.482257s 2024-01-24T23:26:29.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f21d9b640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f5f2409f9c0 legacy=0x7f5f24139ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:29.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2259c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f5f2409ff20 0x7f5f240a0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:40416/0 (socket says 172.21.15.132:40416) 2024-01-24T23:26:29.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2259c640 1 -- 172.21.15.132:0/1827240325 learned_addr learned my addr 172.21.15.132:0/1827240325 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:29.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f22d9d640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f5f2413b2c0 0x7f5f2413d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:52206/0 (socket says 172.21.15.132:52206) 2024-01-24T23:26:29.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 1 -- 172.21.15.132:0/1827240325 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916140767 0 0) 0x7f5f240a1160 con 0x7f5f2413b2c0 2024-01-24T23:26:29.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 10 monclient(hunting): _negotiate 2024-01-24T23:26:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 10 monclient(hunting): my global_id is 94504 2024-01-24T23:26:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 1 -- 172.21.15.132:0/1827240325 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f10003530 con 0x7f5f2413b2c0 2024-01-24T23:26:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 1 -- 172.21.15.132:0/1827240325 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2063787335 0 0) 0x7f5f24064700 con 0x7f5f2409ff20 2024-01-24T23:26:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 10 monclient(hunting): _negotiate 2024-01-24T23:26:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 10 monclient(hunting): my global_id is 94626 2024-01-24T23:26:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 1 -- 172.21.15.132:0/1827240325 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f240a1160 con 0x7f5f2409ff20 2024-01-24T23:26:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 1 -- 172.21.15.132:0/1827240325 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1924415582 0 0) 0x7f5f10003530 con 0x7f5f2413b2c0 2024-01-24T23:26:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 1 -- 172.21.15.132:0/1827240325 >> v1:172.21.15.115:6790/0 conn(0x7f5f2409f9c0 legacy=0x7f5f24139ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 1 -- 172.21.15.132:0/1827240325 >> v1:172.21.15.115:6789/0 conn(0x7f5f2409ff20 legacy=0x7f5f240a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 10 monclient: _finish_hunting 0 2024-01-24T23:26:29.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 1 monclient: found mon.noname-c 2024-01-24T23:26:29.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:29.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:26:29.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 1 -- 172.21.15.132:0/1827240325 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f240a0a10 con 0x7f5f2413b2c0 2024-01-24T23:26:29.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 10 monclient: _finish_auth 0 2024-01-24T23:26:29.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 10 monclient: _check_auth_tickets 2024-01-24T23:26:29.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:29.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 1 -- 172.21.15.132:0/1827240325 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5f1c001800 con 0x7f5f2413b2c0 2024-01-24T23:26:29.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:29.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:26:29.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 10 monclient: dump: 2024-01-24T23:26:29.170 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:29.170 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:29.171 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:29.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f28f28640 5 monclient: authenticate success, global_id 94504 2024-01-24T23:26:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f28f28640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 1 -- 172.21.15.132:0/1827240325 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5f1c001a10 con 0x7f5f2413b2c0 2024-01-24T23:26:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f28f28640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f28f28640 4 set_mon_vals no callback set 2024-01-24T23:26:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 1 -- 172.21.15.132:0/1827240325 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5f1c004b30 con 0x7f5f2413b2c0 2024-01-24T23:26:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:26:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.155+0000 7f5f2159a640 10 monclient: dump: 2024-01-24T23:26:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:29.172 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:29.173 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 10 monclient: shutdown 2024-01-24T23:26:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:29.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 1 -- 172.21.15.132:0/1827240325 >> v1:172.21.15.115:6791/0 conn(0x7f5f2413b2c0 legacy=0x7f5f2413d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 1 -- 172.21.15.132:0/1827240325 shutdown_connections 2024-01-24T23:26:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 1 -- 172.21.15.132:0/1827240325 >> 172.21.15.132:0/1827240325 conn(0x7f5f240a5a10 msgr2=0x7f5f240a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 1 -- 172.21.15.132:0/1827240325 shutdown_connections 2024-01-24T23:26:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 1 -- 172.21.15.132:0/1827240325 wait complete. 2024-01-24T23:26:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 10 monclient: build_initial_monmap 2024-01-24T23:26:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 10 monclient: monmap: 2024-01-24T23:26:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:29.174 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:29.175 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 1 Processor -- start 2024-01-24T23:26:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 1 -- start start 2024-01-24T23:26:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 10 monclient: init 2024-01-24T23:26:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 10 monclient: _add_conns ranks=[0,1,4,8,6,3,7,2,5] 2024-01-24T23:26:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 10 monclient(hunting): picked mon.noname-a con 0x7f5f2409f9c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:29.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 10 monclient(hunting): picked mon.noname-f con 0x7f5f2409ff20 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 10 monclient(hunting): picked mon.noname-c con 0x7f5f2413b2c0 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f2409e380 con 0x7f5f2409f9c0 2024-01-24T23:26:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f24071960 con 0x7f5f2409ff20 2024-01-24T23:26:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f2409e160 con 0x7f5f2413b2c0 2024-01-24T23:26:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 10 monclient(hunting): authenticate will time out at 1276.486084s 2024-01-24T23:26:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f21d9b640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f5f2409ff20 legacy=0x7f5f2409cbd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f2259c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f5f2409f9c0 0x7f5f2409c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:40420/0 (socket says 172.21.15.132:40420) 2024-01-24T23:26:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f2259c640 1 -- 172.21.15.132:0/2089136034 learned_addr learned my addr 172.21.15.132:0/2089136034 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f22d9d640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f5f2413b2c0 0x7f5f241d3810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:52218/0 (socket says 172.21.15.132:52218) 2024-01-24T23:26:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 1 -- 172.21.15.132:0/2089136034 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3878313227 0 0) 0x7f5f2409e160 con 0x7f5f2413b2c0 2024-01-24T23:26:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 10 monclient(hunting): my global_id is 94513 2024-01-24T23:26:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 1 -- 172.21.15.132:0/2089136034 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f04003530 con 0x7f5f2413b2c0 2024-01-24T23:26:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 1 -- 172.21.15.132:0/2089136034 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3211052757 0 0) 0x7f5f2409e380 con 0x7f5f2409f9c0 2024-01-24T23:26:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 10 monclient(hunting): my global_id is 94635 2024-01-24T23:26:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 1 -- 172.21.15.132:0/2089136034 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f2409e160 con 0x7f5f2409f9c0 2024-01-24T23:26:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 1 -- 172.21.15.132:0/2089136034 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 279114545 0 0) 0x7f5f04003530 con 0x7f5f2413b2c0 2024-01-24T23:26:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 1 -- 172.21.15.132:0/2089136034 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5f2409e380 con 0x7f5f2413b2c0 2024-01-24T23:26:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 1 -- 172.21.15.132:0/2089136034 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5f1c004b30 con 0x7f5f2413b2c0 2024-01-24T23:26:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:26:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 10 monclient(hunting): dump: 2024-01-24T23:26:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:29.179 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 1 -- 172.21.15.132:0/2089136034 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2075300975 0 0) 0x7f5f2409e160 con 0x7f5f2409f9c0 2024-01-24T23:26:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 1 -- 172.21.15.132:0/2089136034 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5f04003530 con 0x7f5f2409f9c0 2024-01-24T23:26:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 1 -- 172.21.15.132:0/2089136034 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5f0c001bd0 con 0x7f5f2409f9c0 2024-01-24T23:26:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:26:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 10 monclient(hunting): dump: 2024-01-24T23:26:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:29.179 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:29.180 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 1 -- 172.21.15.132:0/2089136034 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2824109703 0 0) 0x7f5f2409e380 con 0x7f5f2413b2c0 2024-01-24T23:26:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 1 -- 172.21.15.132:0/2089136034 >> v1:172.21.15.132:6789/0 conn(0x7f5f2409ff20 legacy=0x7f5f2409cbd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 1 -- 172.21.15.132:0/2089136034 >> v1:172.21.15.115:6789/0 conn(0x7f5f2409f9c0 legacy=0x7f5f2409c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 10 monclient: _finish_hunting 0 2024-01-24T23:26:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 1 monclient: found mon.c 2024-01-24T23:26:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 1 -- 172.21.15.132:0/2089136034 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2409d2e0 con 0x7f5f2413b2c0 2024-01-24T23:26:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 10 monclient: _finish_auth 0 2024-01-24T23:26:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 10 monclient: _check_auth_tickets 2024-01-24T23:26:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f1affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 5 monclient: authenticate success, global_id 94513 2024-01-24T23:26:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 10 monclient: _renew_subs 2024-01-24T23:26:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.159+0000 7f5f28f28640 1 -- 172.21.15.132:0/2089136034 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f24120d00 con 0x7f5f2413b2c0 2024-01-24T23:26:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.163+0000 7f5f28f28640 10 monclient: _renew_subs 2024-01-24T23:26:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.163+0000 7f5f28f28640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.163+0000 7f5f28f28640 1 -- 172.21.15.132:0/2089136034 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f24120740 con 0x7f5f2413b2c0 2024-01-24T23:26:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.163+0000 7f5f1affd640 1 -- 172.21.15.132:0/2089136034 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5f1c001030 con 0x7f5f2413b2c0 2024-01-24T23:26:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.163+0000 7f5f1affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.163+0000 7f5f1affd640 1 -- 172.21.15.132:0/2089136034 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5f1c001800 con 0x7f5f2413b2c0 2024-01-24T23:26:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.163+0000 7f5f1affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.163+0000 7f5f20d99640 4 set_mon_vals no callback set 2024-01-24T23:26:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.163+0000 7f5f1affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:26:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.163+0000 7f5f1affd640 10 monclient: dump: 2024-01-24T23:26:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:29.182 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:29.183 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.163+0000 7f5f28f28640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.163+0000 7f5f28f28640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.163+0000 7f5f28f28640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.163+0000 7f5f28f28640 1 -- 172.21.15.132:0/2089136034 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f08005090 con 0x7f5f2413b2c0 2024-01-24T23:26:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.163+0000 7f5f1affd640 1 -- 172.21.15.132:0/2089136034 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f5f1c00f9c0 con 0x7f5f2413b2c0 2024-01-24T23:26:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.163+0000 7f5f1affd640 1 -- 172.21.15.132:0/2089136034 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(171..171 src has 1..171) v4 ==== 7239+0+0 (unknown 2443135444 0 0) 0x7f5f1c044440 con 0x7f5f2413b2c0 2024-01-24T23:26:29.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.163+0000 7f5f2259c640 10 monclient: get_auth_request con 0x7f5f04037780 auth_method 0 2024-01-24T23:26:29.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.175+0000 7f5f1affd640 1 -- 172.21.15.132:0/2089136034 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f5f1c014750 con 0x7f5f2413b2c0 2024-01-24T23:26:29.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.175+0000 7f5f1affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:29.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.175+0000 7f5f1affd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:29.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.335+0000 7f5f28f28640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}] 2024-01-24T23:26:29.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.335+0000 7f5f28f28640 10 monclient: _send_command 2 [{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}] 2024-01-24T23:26:29.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.335+0000 7f5f28f28640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:29.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.335+0000 7f5f28f28640 1 -- 172.21.15.132:0/2089136034 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush move", "name": "barr", "args": ["root=default"]} v 0) v1 -- 0x7f5f08005b50 con 0x7f5f2413b2c0 2024-01-24T23:26:29.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.939+0000 7f5f1affd640 1 -- 172.21.15.132:0/2089136034 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}]=0 moved item id -19 name 'barr' to location {root=default} in crush map v172) v1 ==== 173+0+0 (unknown 3657099575 0 0) 0x7f5f1c012df0 con 0x7f5f2413b2c0 2024-01-24T23:26:29.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.939+0000 7f5f1affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}] 2024-01-24T23:26:29.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:29.939+0000 7f5f1affd640 10 monclient: _finish_command 2 = system:0 moved item id -19 name 'barr' to location {root=default} in crush map 2024-01-24T23:26:30.043 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.039+0000 7f5f28f28640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}] 2024-01-24T23:26:30.043 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.039+0000 7f5f28f28640 10 monclient: _send_command 3 [{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}] 2024-01-24T23:26:30.043 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.039+0000 7f5f28f28640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:30.044 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.039+0000 7f5f28f28640 1 -- 172.21.15.132:0/2089136034 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush move", "name": "barr", "args": ["root=default"]} v 0) v1 -- 0x7f5f080055d0 con 0x7f5f2413b2c0 2024-01-24T23:26:30.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.043+0000 7f5f1affd640 1 -- 172.21.15.132:0/2089136034 <== mon.4 v1:172.21.15.115:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}]=0 no need to move item id -19 name 'barr' to location {root=default} in crush map v172) v1 ==== 183+0+0 (unknown 3169692518 0 0) 0x7f5f1c014570 con 0x7f5f2413b2c0 2024-01-24T23:26:30.046 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.043+0000 7f5f1affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}] 2024-01-24T23:26:30.046 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.043+0000 7f5f1affd640 10 monclient: _finish_command 3 = system:0 no need to move item id -19 name 'barr' to location {root=default} in crush map 2024-01-24T23:26:30.046 INFO:tasks.workunit.client.0.smithi132.stderr:no need to move item id -19 name 'barr' to location {root=default} in crush map 2024-01-24T23:26:30.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.047+0000 7f5f28f28640 1 -- 172.21.15.132:0/2089136034 >> v1:172.21.15.132:6812/1835401922 conn(0x7f5f04037780 legacy=0x7f5f04039c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:30.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.047+0000 7f5f28f28640 10 monclient: shutdown 2024-01-24T23:26:30.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.047+0000 7f5f28f28640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:30.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.047+0000 7f5f28f28640 1 -- 172.21.15.132:0/2089136034 >> v1:172.21.15.115:6791/0 conn(0x7f5f2413b2c0 legacy=0x7f5f241d3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:30.049 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.047+0000 7f5f28f28640 1 -- 172.21.15.132:0/2089136034 shutdown_connections 2024-01-24T23:26:30.049 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.047+0000 7f5f28f28640 1 -- 172.21.15.132:0/2089136034 >> 172.21.15.132:0/2089136034 conn(0x7f5f240a5a10 msgr2=0x7f5f24066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:30.049 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.047+0000 7f5f28f28640 1 -- 172.21.15.132:0/2089136034 shutdown_connections 2024-01-24T23:26:30.049 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.047+0000 7f5f28f28640 1 -- 172.21.15.132:0/2089136034 wait complete. 2024-01-24T23:26:30.063 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush move fooo rack=barr 2024-01-24T23:26:30.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f7296a29640 10 monclient: get_monmap_and_config 2024-01-24T23:26:30.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f7296a29640 10 monclient: build_initial_monmap 2024-01-24T23:26:30.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f7296a29640 10 monclient: monmap: 2024-01-24T23:26:30.184 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:30.184 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:30.184 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:30.185027+0000 2024-01-24T23:26:30.184 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:30.185027+0000 2024-01-24T23:26:30.184 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:30.184 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:30.184 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:30.184 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:30.184 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:30.184 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:30.184 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:30.185 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:30.185 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:30.185 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:30.185 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:30.185 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:30.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f7296a29640 1 Processor -- start 2024-01-24T23:26:30.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f7296a29640 1 -- start start 2024-01-24T23:26:30.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f7296a29640 10 monclient: init 2024-01-24T23:26:30.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f7296a29640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:30.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f7296a29640 10 monclient: _add_conns ranks=[4,6,2,7,8,5,3,1,0] 2024-01-24T23:26:30.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f7296a29640 10 monclient(hunting): picked mon.noname-c con 0x7f72901370e0 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:30.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f7296a29640 10 monclient(hunting): picked mon.noname-d con 0x7f729009bc20 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:30.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f7296a29640 10 monclient(hunting): picked mon.noname-b con 0x7f729013d640 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:30.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f7296a29640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7290064700 con 0x7f729013d640 2024-01-24T23:26:30.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f7296a29640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f729006a530 con 0x7f72901370e0 2024-01-24T23:26:30.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f7296a29640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f729009af60 con 0x7f729009bc20 2024-01-24T23:26:30.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f7296a29640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:30.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f7296a29640 10 monclient(hunting): authenticate will time out at 1277.509673s 2024-01-24T23:26:30.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f7294fc6640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f729013d640 legacy=0x7f729013fa30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:30.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728ffff640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f72901370e0 0x7f72901374c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:52222/0 (socket says 172.21.15.132:52222) 2024-01-24T23:26:30.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728ffff640 1 -- 172.21.15.132:0/1172328619 learned_addr learned my addr 172.21.15.132:0/1172328619 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:30.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728f7fe640 1 --1- 172.21.15.132:0/1172328619 >> v1:172.21.15.115:6792/0 conn(0x7f729009bc20 0x7f729013bf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:59310/0 (socket says 172.21.15.132:59310) 2024-01-24T23:26:30.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 1 -- 172.21.15.132:0/1172328619 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741713220 0 0) 0x7f729006a530 con 0x7f72901370e0 2024-01-24T23:26:30.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:30.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:30.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:30.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 10 monclient(hunting): my global_id is 94522 2024-01-24T23:26:30.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 1 -- 172.21.15.132:0/1172328619 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7278003530 con 0x7f72901370e0 2024-01-24T23:26:30.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 1 -- 172.21.15.132:0/1172328619 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3721248306 0 0) 0x7f729009af60 con 0x7f729009bc20 2024-01-24T23:26:30.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:30.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:30.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:30.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 10 monclient(hunting): my global_id is 6387 2024-01-24T23:26:30.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 1 -- 172.21.15.132:0/1172328619 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f729006a530 con 0x7f729009bc20 2024-01-24T23:26:30.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 1 -- 172.21.15.132:0/1172328619 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3964131733 0 0) 0x7f7278003530 con 0x7f72901370e0 2024-01-24T23:26:30.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 1 -- 172.21.15.132:0/1172328619 >> v1:172.21.15.115:6792/0 conn(0x7f729009bc20 legacy=0x7f729013bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:30.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 1 -- 172.21.15.132:0/1172328619 >> v1:172.21.15.115:6790/0 conn(0x7f729013d640 legacy=0x7f729013fa30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:30.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 10 monclient: _finish_hunting 0 2024-01-24T23:26:30.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 1 monclient: found mon.noname-c 2024-01-24T23:26:30.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:30.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:26:30.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 1 -- 172.21.15.132:0/1172328619 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729009b240 con 0x7f72901370e0 2024-01-24T23:26:30.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 10 monclient: _finish_auth 0 2024-01-24T23:26:30.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 10 monclient: _check_auth_tickets 2024-01-24T23:26:30.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:30.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 1 -- 172.21.15.132:0/1172328619 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f72800017c0 con 0x7f72901370e0 2024-01-24T23:26:30.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:30.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:26:30.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 10 monclient: dump: 2024-01-24T23:26:30.196 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:30.196 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:30.196 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:30.196 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:30.196 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:30.196 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:30.196 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:30.196 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:30.197 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:30.197 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:30.197 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:30.197 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:30.197 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:30.197 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:30.197 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:30.197 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:30.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f7296a29640 5 monclient: authenticate success, global_id 94522 2024-01-24T23:26:30.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f7296a29640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:30.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 1 -- 172.21.15.132:0/1172328619 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f72800019e0 con 0x7f72901370e0 2024-01-24T23:26:30.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:30.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f7296a29640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:30.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f7296a29640 4 set_mon_vals no callback set 2024-01-24T23:26:30.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 1 -- 172.21.15.132:0/1172328619 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7280004bf0 con 0x7f72901370e0 2024-01-24T23:26:30.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:30.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:26:30.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f728effd640 10 monclient: dump: 2024-01-24T23:26:30.198 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:30.198 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:30.198 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:30.198 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:30.198 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:30.198 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:30.198 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:30.198 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:30.198 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:30.198 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:30.198 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:30.198 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:30.199 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:30.199 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:30.199 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:30.199 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:30.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f7296a29640 10 monclient: shutdown 2024-01-24T23:26:30.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f7296a29640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:30.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.183+0000 7f7296a29640 1 -- 172.21.15.132:0/1172328619 >> v1:172.21.15.115:6791/0 conn(0x7f72901370e0 legacy=0x7f72901374c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:30.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 1 -- 172.21.15.132:0/1172328619 shutdown_connections 2024-01-24T23:26:30.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 1 -- 172.21.15.132:0/1172328619 >> 172.21.15.132:0/1172328619 conn(0x7f72900a2b40 msgr2=0x7f72900a2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:30.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 1 -- 172.21.15.132:0/1172328619 shutdown_connections 2024-01-24T23:26:30.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 1 -- 172.21.15.132:0/1172328619 wait complete. 2024-01-24T23:26:30.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 10 monclient: build_initial_monmap 2024-01-24T23:26:30.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 10 monclient: monmap: 2024-01-24T23:26:30.199 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:30.199 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:30.200 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:30.200 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:30.200 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:30.200 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:30.200 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:30.200 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:30.200 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:30.200 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:30.200 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:30.200 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:30.200 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:30.200 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:30.200 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:30.200 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:30.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 1 Processor -- start 2024-01-24T23:26:30.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 1 -- start start 2024-01-24T23:26:30.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 10 monclient: init 2024-01-24T23:26:30.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:30.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 10 monclient: _add_conns ranks=[8,4,2,7,3,0,1,6,5] 2024-01-24T23:26:30.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 10 monclient(hunting): picked mon.noname-e con 0x7f729009bc20 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:30.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 10 monclient(hunting): picked mon.noname-c con 0x7f72901370e0 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:30.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 10 monclient(hunting): picked mon.noname-b con 0x7f729013d640 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:30.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f729006e650 con 0x7f729013d640 2024-01-24T23:26:30.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f729009fd90 con 0x7f72901370e0 2024-01-24T23:26:30.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f729009f910 con 0x7f729009bc20 2024-01-24T23:26:30.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:30.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 10 monclient(hunting): authenticate will time out at 1277.513631s 2024-01-24T23:26:30.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7294fc6640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f729013d640 legacy=0x7f72901d3a20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:30.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728f7fe640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f72901370e0 0x7f729009eba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:52226/0 (socket says 172.21.15.132:52226) 2024-01-24T23:26:30.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728f7fe640 1 -- 172.21.15.132:0/2912376742 learned_addr learned my addr 172.21.15.132:0/2912376742 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:30.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 1 -- 172.21.15.132:0/2912376742 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1345963272 0 0) 0x7f729009fd90 con 0x7f72901370e0 2024-01-24T23:26:30.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:26:30.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:30.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:30.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 10 monclient(hunting): my global_id is 94531 2024-01-24T23:26:30.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 1 -- 172.21.15.132:0/2912376742 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f727c003530 con 0x7f72901370e0 2024-01-24T23:26:30.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 1 -- 172.21.15.132:0/2912376742 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4284070608 0 0) 0x7f729009f910 con 0x7f729009bc20 2024-01-24T23:26:30.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:26:30.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:30.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:30.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 10 monclient(hunting): my global_id is 85535 2024-01-24T23:26:30.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 1 -- 172.21.15.132:0/2912376742 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f729009fd90 con 0x7f729009bc20 2024-01-24T23:26:30.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 1 -- 172.21.15.132:0/2912376742 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3284582521 0 0) 0x7f727c003530 con 0x7f72901370e0 2024-01-24T23:26:30.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 1 -- 172.21.15.132:0/2912376742 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f729009f910 con 0x7f72901370e0 2024-01-24T23:26:30.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 1 -- 172.21.15.132:0/2912376742 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7284001c20 con 0x7f72901370e0 2024-01-24T23:26:30.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:30.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:26:30.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 10 monclient(hunting): dump: 2024-01-24T23:26:30.203 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:30.203 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:30.203 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:30.203 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:30.203 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:30.203 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:30.203 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:30.203 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:30.203 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:30.203 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:30.203 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:30.203 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:30.204 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:30.204 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:30.204 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:30.204 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:30.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:30.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 1 -- 172.21.15.132:0/2912376742 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3455486035 0 0) 0x7f729009fd90 con 0x7f729009bc20 2024-01-24T23:26:30.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 1 -- 172.21.15.132:0/2912376742 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f727c003530 con 0x7f729009bc20 2024-01-24T23:26:30.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 1 -- 172.21.15.132:0/2912376742 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f72800016c0 con 0x7f729009bc20 2024-01-24T23:26:30.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:30.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:26:30.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 10 monclient(hunting): dump: 2024-01-24T23:26:30.204 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:30.204 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:30.204 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:30.204 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:30.205 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:30.205 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:30.205 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:30.205 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:30.205 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:30.205 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:30.205 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:30.205 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:30.205 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:30.205 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:30.205 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:30.205 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:30.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 1 -- 172.21.15.132:0/2912376742 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2006519971 0 0) 0x7f729009f910 con 0x7f72901370e0 2024-01-24T23:26:30.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 1 -- 172.21.15.132:0/2912376742 >> v1:172.21.15.115:6793/0 conn(0x7f729009bc20 legacy=0x7f729009e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:30.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 1 -- 172.21.15.132:0/2912376742 >> v1:172.21.15.115:6790/0 conn(0x7f729013d640 legacy=0x7f72901d3a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:30.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 10 monclient: _finish_hunting 0 2024-01-24T23:26:30.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 1 monclient: found mon.c 2024-01-24T23:26:30.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:30.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:30.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 1 -- 172.21.15.132:0/2912376742 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729006f8c0 con 0x7f72901370e0 2024-01-24T23:26:30.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 10 monclient: _finish_auth 0 2024-01-24T23:26:30.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 10 monclient: _check_auth_tickets 2024-01-24T23:26:30.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:30.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 5 monclient: authenticate success, global_id 94531 2024-01-24T23:26:30.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 10 monclient: _renew_subs 2024-01-24T23:26:30.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:30.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 1 -- 172.21.15.132:0/2912376742 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f729009f2b0 con 0x7f72901370e0 2024-01-24T23:26:30.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 10 monclient: _renew_subs 2024-01-24T23:26:30.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:30.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f7296a29640 1 -- 172.21.15.132:0/2912376742 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f729006b490 con 0x7f72901370e0 2024-01-24T23:26:30.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 1 -- 172.21.15.132:0/2912376742 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7284004aa0 con 0x7f72901370e0 2024-01-24T23:26:30.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:30.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 1 -- 172.21.15.132:0/2912376742 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7284000fa0 con 0x7f72901370e0 2024-01-24T23:26:30.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:30.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:26:30.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728dffb640 4 set_mon_vals no callback set 2024-01-24T23:26:30.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.187+0000 7f728cff9640 10 monclient: dump: 2024-01-24T23:26:30.207 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:30.207 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:30.207 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:30.207 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:30.207 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:30.207 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:30.207 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:30.207 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:30.207 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:30.208 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:30.208 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:30.208 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:30.208 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:30.208 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:30.208 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:30.208 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:30.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.191+0000 7f7296a29640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:30.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.191+0000 7f7296a29640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:30.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.191+0000 7f7296a29640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:30.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.191+0000 7f7296a29640 1 -- 172.21.15.132:0/2912376742 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7270005090 con 0x7f72901370e0 2024-01-24T23:26:30.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.191+0000 7f728cff9640 1 -- 172.21.15.132:0/2912376742 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f728400f870 con 0x7f72901370e0 2024-01-24T23:26:30.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.191+0000 7f728cff9640 1 -- 172.21.15.132:0/2912376742 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(172..172 src has 1..172) v4 ==== 7427+0+0 (unknown 1093838147 0 0) 0x7f72840444e0 con 0x7f72901370e0 2024-01-24T23:26:30.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.191+0000 7f728ffff640 10 monclient: get_auth_request con 0x7f727c0376f0 auth_method 0 2024-01-24T23:26:30.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.203+0000 7f728cff9640 1 -- 172.21.15.132:0/2912376742 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f72840146d0 con 0x7f72901370e0 2024-01-24T23:26:30.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.203+0000 7f728cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:30.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.203+0000 7f728cff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:30.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.363+0000 7f7296a29640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}] 2024-01-24T23:26:30.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.363+0000 7f7296a29640 10 monclient: _send_command 2 [{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}] 2024-01-24T23:26:30.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.363+0000 7f7296a29640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:30.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.363+0000 7f7296a29640 1 -- 172.21.15.132:0/2912376742 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]} v 0) v1 -- 0x7f7270005b50 con 0x7f72901370e0 2024-01-24T23:26:30.964 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.963+0000 7f728cff9640 1 -- 172.21.15.132:0/2912376742 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}]=0 moved item id -16 name 'fooo' to location {rack=barr} in crush map v173) v1 ==== 167+0+0 (unknown 2975706860 0 0) 0x7f72840144f0 con 0x7f72901370e0 2024-01-24T23:26:30.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.963+0000 7f728cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}] 2024-01-24T23:26:30.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:30.963+0000 7f728cff9640 10 monclient: _finish_command 2 = system:0 moved item id -16 name 'fooo' to location {rack=barr} in crush map 2024-01-24T23:26:31.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.063+0000 7f7296a29640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}] 2024-01-24T23:26:31.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.063+0000 7f7296a29640 10 monclient: _send_command 3 [{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}] 2024-01-24T23:26:31.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.063+0000 7f7296a29640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:31.066 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.063+0000 7f7296a29640 1 -- 172.21.15.132:0/2912376742 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]} v 0) v1 -- 0x7f72700055d0 con 0x7f72901370e0 2024-01-24T23:26:31.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.067+0000 7f728cff9640 1 -- 172.21.15.132:0/2912376742 <== mon.4 v1:172.21.15.115:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}]=0 no need to move item id -16 name 'fooo' to location {rack=barr} in crush map v173) v1 ==== 177+0+0 (unknown 2930607156 0 0) 0x7f7284015230 con 0x7f72901370e0 2024-01-24T23:26:31.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.067+0000 7f728cff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}] 2024-01-24T23:26:31.068 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.067+0000 7f728cff9640 10 monclient: _finish_command 3 = system:0 no need to move item id -16 name 'fooo' to location {rack=barr} in crush map 2024-01-24T23:26:31.068 INFO:tasks.workunit.client.0.smithi132.stderr:no need to move item id -16 name 'fooo' to location {rack=barr} in crush map 2024-01-24T23:26:31.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.067+0000 7f7296a29640 1 -- 172.21.15.132:0/2912376742 >> v1:172.21.15.132:6812/1835401922 conn(0x7f727c0376f0 legacy=0x7f727c039bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:31.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.067+0000 7f7296a29640 10 monclient: shutdown 2024-01-24T23:26:31.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.067+0000 7f7296a29640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:31.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.067+0000 7f7296a29640 1 -- 172.21.15.132:0/2912376742 >> v1:172.21.15.115:6791/0 conn(0x7f72901370e0 legacy=0x7f729009eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:31.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.067+0000 7f7296a29640 1 -- 172.21.15.132:0/2912376742 shutdown_connections 2024-01-24T23:26:31.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.067+0000 7f7296a29640 1 -- 172.21.15.132:0/2912376742 >> 172.21.15.132:0/2912376742 conn(0x7f72900a2b40 msgr2=0x7f7290135b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:31.070 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.067+0000 7f7296a29640 1 -- 172.21.15.132:0/2912376742 shutdown_connections 2024-01-24T23:26:31.071 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.067+0000 7f7296a29640 1 -- 172.21.15.132:0/2912376742 wait complete. 2024-01-24T23:26:31.085 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm fooo 2024-01-24T23:26:31.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f2e69e640 10 monclient: get_monmap_and_config 2024-01-24T23:26:31.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f2e69e640 10 monclient: build_initial_monmap 2024-01-24T23:26:31.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f2e69e640 10 monclient: monmap: 2024-01-24T23:26:31.205 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:31.205 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:31.205 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:31.205785+0000 2024-01-24T23:26:31.205 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:31.205785+0000 2024-01-24T23:26:31.205 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:31.205 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:31.205 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:31.205 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:31.205 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:31.205 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:31.205 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:31.206 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:31.206 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:31.206 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:31.206 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:31.206 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:31.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f2e69e640 1 Processor -- start 2024-01-24T23:26:31.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f2e69e640 1 -- start start 2024-01-24T23:26:31.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f2e69e640 10 monclient: init 2024-01-24T23:26:31.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f2e69e640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:31.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f2e69e640 10 monclient: _add_conns ranks=[8,7,2,6,1,4,3,5,0] 2024-01-24T23:26:31.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f2e69e640 10 monclient(hunting): picked mon.noname-e con 0x7f9f2809ff20 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:31.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f2e69e640 10 monclient(hunting): picked mon.noname-i con 0x7f9f2809f9c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:31.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f2e69e640 10 monclient(hunting): picked mon.noname-b con 0x7f9f2813b2c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:31.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f2e69e640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f28064700 con 0x7f9f2813b2c0 2024-01-24T23:26:31.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f2e69e640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2809e760 con 0x7f9f2809f9c0 2024-01-24T23:26:31.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f2e69e640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f280a1160 con 0x7f9f2809ff20 2024-01-24T23:26:31.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f2e69e640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:31.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f2e69e640 10 monclient(hunting): authenticate will time out at 1278.530412s 2024-01-24T23:26:31.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f277fe640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f9f2809f9c0 legacy=0x7f9f28139ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:31.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f2cc3b640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f9f2813b2c0 legacy=0x7f9f2813d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:31.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f27fff640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f9f2809ff20 0x7f9f280a0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:36100/0 (socket says 172.21.15.132:36100) 2024-01-24T23:26:31.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f27fff640 1 -- 172.21.15.132:0/3571391386 learned_addr learned my addr 172.21.15.132:0/3571391386 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:31.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 1 -- 172.21.15.132:0/3571391386 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233346826 0 0) 0x7f9f280a1160 con 0x7f9f2809ff20 2024-01-24T23:26:31.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:31.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:31.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:31.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 10 monclient(hunting): my global_id is 85544 2024-01-24T23:26:31.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 1 -- 172.21.15.132:0/3571391386 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f10003530 con 0x7f9f2809ff20 2024-01-24T23:26:31.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 1 -- 172.21.15.132:0/3571391386 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1512836017 0 0) 0x7f9f10003530 con 0x7f9f2809ff20 2024-01-24T23:26:31.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 1 -- 172.21.15.132:0/3571391386 >> v1:172.21.15.132:6792/0 conn(0x7f9f2809f9c0 legacy=0x7f9f28139ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:31.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 1 -- 172.21.15.132:0/3571391386 >> v1:172.21.15.115:6790/0 conn(0x7f9f2813b2c0 legacy=0x7f9f2813d6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:31.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:26:31.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 1 monclient: found mon.noname-e 2024-01-24T23:26:31.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:31.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:26:31.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 1 -- 172.21.15.132:0/3571391386 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f280a0a10 con 0x7f9f2809ff20 2024-01-24T23:26:31.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 10 monclient: _finish_auth 0 2024-01-24T23:26:31.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:26:31.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:31.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 1 -- 172.21.15.132:0/3571391386 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9f18001800 con 0x7f9f2809ff20 2024-01-24T23:26:31.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:31.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:26:31.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 10 monclient: dump: 2024-01-24T23:26:31.215 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:31.215 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:31.215 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:31.215 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:31.215 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:31.215 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:31.216 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:31.216 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:31.216 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:31.216 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:31.216 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:31.216 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:31.216 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:31.216 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:31.216 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:31.216 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:31.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f2e69e640 5 monclient: authenticate success, global_id 85544 2024-01-24T23:26:31.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f2e69e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:31.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 1 -- 172.21.15.132:0/3571391386 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9f18001ad0 con 0x7f9f2809ff20 2024-01-24T23:26:31.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:31.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f2e69e640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:31.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f2e69e640 4 set_mon_vals no callback set 2024-01-24T23:26:31.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 1 -- 172.21.15.132:0/3571391386 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9f18004bf0 con 0x7f9f2809ff20 2024-01-24T23:26:31.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:31.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:26:31.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.203+0000 7f9f26ffd640 10 monclient: dump: 2024-01-24T23:26:31.217 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:31.217 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:31.217 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:31.218 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:31.218 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:31.218 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:31.218 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:31.218 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:31.218 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:31.218 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:31.218 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:31.218 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:31.218 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:31.218 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:31.218 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:31.218 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:31.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 10 monclient: shutdown 2024-01-24T23:26:31.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:31.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 1 -- 172.21.15.132:0/3571391386 >> v1:172.21.15.115:6793/0 conn(0x7f9f2809ff20 legacy=0x7f9f280a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:31.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 1 -- 172.21.15.132:0/3571391386 shutdown_connections 2024-01-24T23:26:31.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 1 -- 172.21.15.132:0/3571391386 >> 172.21.15.132:0/3571391386 conn(0x7f9f280a5a10 msgr2=0x7f9f280a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:31.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 1 -- 172.21.15.132:0/3571391386 shutdown_connections 2024-01-24T23:26:31.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 1 -- 172.21.15.132:0/3571391386 wait complete. 2024-01-24T23:26:31.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 10 monclient: build_initial_monmap 2024-01-24T23:26:31.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 10 monclient: monmap: 2024-01-24T23:26:31.219 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:31.219 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:31.219 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:31.219 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:31.219 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:31.219 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:31.219 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:31.220 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:31.220 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:31.220 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:31.220 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:31.220 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:31.220 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:31.220 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:31.220 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:31.220 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:31.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 1 Processor -- start 2024-01-24T23:26:31.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 1 -- start start 2024-01-24T23:26:31.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 10 monclient: init 2024-01-24T23:26:31.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:31.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 10 monclient: _add_conns ranks=[5,6,0,3,2,1,4,7,8] 2024-01-24T23:26:31.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 10 monclient(hunting): picked mon.noname-h con 0x7f9f2809f9c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:31.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 10 monclient(hunting): picked mon.noname-d con 0x7f9f2809ff20 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:31.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 10 monclient(hunting): picked mon.noname-a con 0x7f9f2813b2c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:31.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f28071680 con 0x7f9f2813b2c0 2024-01-24T23:26:31.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f28071960 con 0x7f9f2809f9c0 2024-01-24T23:26:31.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f28120700 con 0x7f9f2809ff20 2024-01-24T23:26:31.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:31.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 10 monclient(hunting): authenticate will time out at 1278.534085s 2024-01-24T23:26:31.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f27fff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f9f2809f9c0 0x7f9f2811f1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:46502/0 (socket says 172.21.15.132:46502) 2024-01-24T23:26:31.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f27fff640 1 -- 172.21.15.132:0/4038718795 learned_addr learned my addr 172.21.15.132:0/4038718795 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:31.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 1 -- 172.21.15.132:0/4038718795 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1065304168 0 0) 0x7f9f28071960 con 0x7f9f2809f9c0 2024-01-24T23:26:31.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:26:31.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:31.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:31.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 10 monclient(hunting): my global_id is 6197 2024-01-24T23:26:31.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 1 -- 172.21.15.132:0/4038718795 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f14003530 con 0x7f9f2809f9c0 2024-01-24T23:26:31.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 1 -- 172.21.15.132:0/4038718795 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3344585097 0 0) 0x7f9f14003530 con 0x7f9f2809f9c0 2024-01-24T23:26:31.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 1 -- 172.21.15.132:0/4038718795 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9f28071960 con 0x7f9f2809f9c0 2024-01-24T23:26:31.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 1 -- 172.21.15.132:0/4038718795 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9f180017c0 con 0x7f9f2809f9c0 2024-01-24T23:26:31.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:31.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:31.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 10 monclient(hunting): dump: 2024-01-24T23:26:31.222 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:31.222 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:31.222 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:31.222 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:31.222 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:31.223 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:31.223 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:31.223 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:31.223 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:31.223 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:31.223 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:31.223 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:31.223 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:31.223 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:31.223 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:31.223 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:31.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:31.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 1 -- 172.21.15.132:0/4038718795 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1464431223 0 0) 0x7f9f28071680 con 0x7f9f2813b2c0 2024-01-24T23:26:31.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:26:31.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:31.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:31.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 10 monclient(hunting): my global_id is 94644 2024-01-24T23:26:31.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 1 -- 172.21.15.132:0/4038718795 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f14003530 con 0x7f9f2813b2c0 2024-01-24T23:26:31.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 1 -- 172.21.15.132:0/4038718795 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2805660193 0 0) 0x7f9f28071960 con 0x7f9f2809f9c0 2024-01-24T23:26:31.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 1 -- 172.21.15.132:0/4038718795 >> v1:172.21.15.115:6792/0 conn(0x7f9f2809ff20 legacy=0x7f9f281cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:31.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 1 -- 172.21.15.132:0/4038718795 >> v1:172.21.15.115:6789/0 conn(0x7f9f2813b2c0 legacy=0x7f9f281d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:31.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:26:31.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 1 monclient: found mon.h 2024-01-24T23:26:31.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:31.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:31.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 1 -- 172.21.15.132:0/4038718795 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f281365d0 con 0x7f9f2809f9c0 2024-01-24T23:26:31.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 10 monclient: _finish_auth 0 2024-01-24T23:26:31.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:26:31.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:31.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 5 monclient: authenticate success, global_id 6197 2024-01-24T23:26:31.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 10 monclient: _renew_subs 2024-01-24T23:26:31.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:31.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 1 -- 172.21.15.132:0/4038718795 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2811e680 con 0x7f9f2809f9c0 2024-01-24T23:26:31.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 10 monclient: _renew_subs 2024-01-24T23:26:31.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:31.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 1 -- 172.21.15.132:0/4038718795 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f281d22b0 con 0x7f9f2809f9c0 2024-01-24T23:26:31.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 1 -- 172.21.15.132:0/4038718795 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9f18004ec0 con 0x7f9f2809f9c0 2024-01-24T23:26:31.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:31.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 1 -- 172.21.15.132:0/4038718795 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9f18005370 con 0x7f9f2809f9c0 2024-01-24T23:26:31.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:31.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f267fc640 4 set_mon_vals no callback set 2024-01-24T23:26:31.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:31.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 10 monclient: dump: 2024-01-24T23:26:31.225 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:31.226 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:31.226 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:31.226 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:31.226 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:31.226 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:31.226 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:31.226 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:31.226 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:31.226 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:31.226 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:31.226 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:31.226 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:31.226 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:31.226 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:31.227 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:31.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:31.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:31.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:31.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f2e69e640 1 -- 172.21.15.132:0/4038718795 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f08005090 con 0x7f9f2809f9c0 2024-01-24T23:26:31.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.207+0000 7f9f24ff9640 1 -- 172.21.15.132:0/4038718795 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f9f1800fdf0 con 0x7f9f2809f9c0 2024-01-24T23:26:31.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.211+0000 7f9f24ff9640 1 -- 172.21.15.132:0/4038718795 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(173..173 src has 1..173) v4 ==== 7427+0+0 (unknown 3705648412 0 0) 0x7f9f18044940 con 0x7f9f2809f9c0 2024-01-24T23:26:31.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.211+0000 7f9f277fe640 10 monclient: get_auth_request con 0x7f9f14036fb0 auth_method 0 2024-01-24T23:26:31.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.215+0000 7f9f24ff9640 1 -- 172.21.15.132:0/4038718795 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f9f18011250 con 0x7f9f2809f9c0 2024-01-24T23:26:31.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.215+0000 7f9f24ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:31.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.215+0000 7f9f24ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:31.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.375+0000 7f9f2e69e640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "fooo"}] 2024-01-24T23:26:31.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.375+0000 7f9f2e69e640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "fooo"}] 2024-01-24T23:26:31.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.375+0000 7f9f2e69e640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:31.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.375+0000 7f9f2e69e640 1 -- 172.21.15.132:0/4038718795 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "fooo"} v 0) v1 -- 0x7f9f08005340 con 0x7f9f2809f9c0 2024-01-24T23:26:31.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.971+0000 7f9f24ff9640 1 -- 172.21.15.132:0/4038718795 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "fooo"}]=0 removed item id -16 name 'fooo' from crush map v174) v1 ==== 122+0+0 (unknown 1475650920 0 0) 0x7f9f18013290 con 0x7f9f2809f9c0 2024-01-24T23:26:31.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.971+0000 7f9f24ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "fooo"}] 2024-01-24T23:26:31.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:31.971+0000 7f9f24ff9640 10 monclient: _finish_command 2 = system:0 removed item id -16 name 'fooo' from crush map 2024-01-24T23:26:32.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.071+0000 7f9f2e69e640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "fooo"}] 2024-01-24T23:26:32.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.071+0000 7f9f2e69e640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "fooo"}] 2024-01-24T23:26:32.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.071+0000 7f9f2e69e640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:32.072 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.071+0000 7f9f2e69e640 1 -- 172.21.15.132:0/4038718795 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "fooo"} v 0) v1 -- 0x7f9f08002220 con 0x7f9f2809f9c0 2024-01-24T23:26:32.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.071+0000 7f9f24ff9640 1 -- 172.21.15.132:0/4038718795 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "fooo"}]=0 device 'fooo' does not appear in the crush map v174) v1 ==== 122+0+0 (unknown 2136257465 0 0) 0x7f9f18014a10 con 0x7f9f2809f9c0 2024-01-24T23:26:32.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.071+0000 7f9f24ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "fooo"}] 2024-01-24T23:26:32.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.071+0000 7f9f24ff9640 10 monclient: _finish_command 3 = system:0 device 'fooo' does not appear in the crush map 2024-01-24T23:26:32.073 INFO:tasks.workunit.client.0.smithi132.stderr:device 'fooo' does not appear in the crush map 2024-01-24T23:26:32.075 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.071+0000 7f9f2e69e640 1 -- 172.21.15.132:0/4038718795 >> v1:172.21.15.132:6812/1835401922 conn(0x7f9f14036fb0 legacy=0x7f9f14039470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:32.075 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.071+0000 7f9f2e69e640 10 monclient: shutdown 2024-01-24T23:26:32.075 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.071+0000 7f9f2e69e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:32.075 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.071+0000 7f9f2e69e640 1 -- 172.21.15.132:0/4038718795 >> v1:172.21.15.132:6791/0 conn(0x7f9f2809f9c0 legacy=0x7f9f2811f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:32.075 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.071+0000 7f9f2e69e640 1 -- 172.21.15.132:0/4038718795 shutdown_connections 2024-01-24T23:26:32.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.071+0000 7f9f2e69e640 1 -- 172.21.15.132:0/4038718795 >> 172.21.15.132:0/4038718795 conn(0x7f9f280a5a10 msgr2=0x7f9f28066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:32.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.075+0000 7f9f2e69e640 1 -- 172.21.15.132:0/4038718795 shutdown_connections 2024-01-24T23:26:32.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.075+0000 7f9f2e69e640 1 -- 172.21.15.132:0/4038718795 wait complete. 2024-01-24T23:26:32.090 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm barr 2024-01-24T23:26:32.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa93453e640 10 monclient: get_monmap_and_config 2024-01-24T23:26:32.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa93453e640 10 monclient: build_initial_monmap 2024-01-24T23:26:32.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa93453e640 10 monclient: monmap: 2024-01-24T23:26:32.196 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:32.197395+0000 2024-01-24T23:26:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:32.197395+0000 2024-01-24T23:26:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:32.198 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:32.198 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:32.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa93453e640 1 Processor -- start 2024-01-24T23:26:32.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa93453e640 1 -- start start 2024-01-24T23:26:32.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa93453e640 10 monclient: init 2024-01-24T23:26:32.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa93453e640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:32.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa93453e640 10 monclient: _add_conns ranks=[2,7,6,0,1,5,3,4,8] 2024-01-24T23:26:32.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa93453e640 10 monclient(hunting): picked mon.noname-b con 0x7fa92c136e50 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:32.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa93453e640 10 monclient(hunting): picked mon.noname-i con 0x7fa92c0a0290 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:32.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa93453e640 10 monclient(hunting): picked mon.noname-d con 0x7fa92c0a1600 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:32.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa93453e640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa92c064700 con 0x7fa92c136e50 2024-01-24T23:26:32.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa93453e640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa92c137940 con 0x7fa92c0a1600 2024-01-24T23:26:32.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa93453e640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa92c09f9c0 con 0x7fa92c0a0290 2024-01-24T23:26:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa93453e640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa93453e640 10 monclient(hunting): authenticate will time out at 1279.521941s 2024-01-24T23:26:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa931ad9640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fa92c0a0290 legacy=0x7fa92c13bd60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9322da640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fa92c136e50 legacy=0x7fa92c137230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa932adb640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fa92c0a1600 0x7fa92c13f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:59330/0 (socket says 172.21.15.132:59330) 2024-01-24T23:26:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa932adb640 1 -- 172.21.15.132:0/1672514086 learned_addr learned my addr 172.21.15.132:0/1672514086 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 1 -- 172.21.15.132:0/1672514086 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2908134226 0 0) 0x7fa92c137940 con 0x7fa92c0a1600 2024-01-24T23:26:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 10 monclient(hunting): _negotiate 2024-01-24T23:26:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 10 monclient(hunting): my global_id is 6405 2024-01-24T23:26:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 1 -- 172.21.15.132:0/1672514086 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa914003530 con 0x7fa92c0a1600 2024-01-24T23:26:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 1 -- 172.21.15.132:0/1672514086 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4120582067 0 0) 0x7fa914003530 con 0x7fa92c0a1600 2024-01-24T23:26:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 1 -- 172.21.15.132:0/1672514086 >> v1:172.21.15.132:6792/0 conn(0x7fa92c0a0290 legacy=0x7fa92c13bd60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 1 -- 172.21.15.132:0/1672514086 >> v1:172.21.15.115:6790/0 conn(0x7fa92c136e50 legacy=0x7fa92c137230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 10 monclient: _finish_hunting 0 2024-01-24T23:26:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 1 monclient: found mon.noname-d 2024-01-24T23:26:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:26:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 1 -- 172.21.15.132:0/1672514086 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92c09eda0 con 0x7fa92c0a1600 2024-01-24T23:26:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 10 monclient: _finish_auth 0 2024-01-24T23:26:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 10 monclient: _check_auth_tickets 2024-01-24T23:26:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 1 -- 172.21.15.132:0/1672514086 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa924001b40 con 0x7fa92c0a1600 2024-01-24T23:26:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa93453e640 5 monclient: authenticate success, global_id 6405 2024-01-24T23:26:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa93453e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:26:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 10 monclient: dump: 2024-01-24T23:26:32.204 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:32.204 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:32.204 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:32.204 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:32.204 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:32.204 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:32.204 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:32.204 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:32.204 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:32.204 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:32.204 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:32.204 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:32.204 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:32.204 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:32.205 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:32.205 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:32.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa93453e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:32.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 1 -- 172.21.15.132:0/1672514086 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa924001e60 con 0x7fa92c0a1600 2024-01-24T23:26:32.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:32.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa93453e640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:32.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa93453e640 4 set_mon_vals no callback set 2024-01-24T23:26:32.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 1 -- 172.21.15.132:0/1672514086 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa924004f70 con 0x7fa92c0a1600 2024-01-24T23:26:32.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:32.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:26:32.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.195+0000 7fa9312d8640 10 monclient: dump: 2024-01-24T23:26:32.205 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:32.205 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:32.205 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:32.205 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:32.206 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:32.206 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:32.206 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:32.206 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:32.206 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:32.206 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:32.206 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:32.206 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:32.206 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:32.206 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:32.206 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:32.206 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:32.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 10 monclient: shutdown 2024-01-24T23:26:32.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:32.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 1 -- 172.21.15.132:0/1672514086 >> v1:172.21.15.115:6792/0 conn(0x7fa92c0a1600 legacy=0x7fa92c13f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:32.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 1 -- 172.21.15.132:0/1672514086 shutdown_connections 2024-01-24T23:26:32.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 1 -- 172.21.15.132:0/1672514086 >> 172.21.15.132:0/1672514086 conn(0x7fa92c0a5a10 msgr2=0x7fa92c0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:32.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 1 -- 172.21.15.132:0/1672514086 shutdown_connections 2024-01-24T23:26:32.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 1 -- 172.21.15.132:0/1672514086 wait complete. 2024-01-24T23:26:32.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 10 monclient: build_initial_monmap 2024-01-24T23:26:32.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 10 monclient: monmap: 2024-01-24T23:26:32.207 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:32.207 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:32.207 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:32.207 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:32.207 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:32.207 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:32.207 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:32.207 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:32.207 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:32.208 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:32.208 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:32.208 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:32.208 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:32.208 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:32.208 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:32.208 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:32.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 1 Processor -- start 2024-01-24T23:26:32.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 1 -- start start 2024-01-24T23:26:32.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 10 monclient: init 2024-01-24T23:26:32.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:32.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 10 monclient: _add_conns ranks=[4,6,0,7,3,1,2,8,5] 2024-01-24T23:26:32.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 10 monclient(hunting): picked mon.noname-c con 0x7fa92c0a0290 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:32.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 10 monclient(hunting): picked mon.noname-d con 0x7fa92c0a1600 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:32.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 10 monclient(hunting): picked mon.noname-a con 0x7fa92c136e50 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:32.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa92c11db40 con 0x7fa92c136e50 2024-01-24T23:26:32.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa92c136c20 con 0x7fa92c0a0290 2024-01-24T23:26:32.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa92c09ea70 con 0x7fa92c0a1600 2024-01-24T23:26:32.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:32.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 10 monclient(hunting): authenticate will time out at 1279.525509s 2024-01-24T23:26:32.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa9322da640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fa92c0a0290 0x7fa92c131ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:52236/0 (socket says 172.21.15.132:52236) 2024-01-24T23:26:32.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa9322da640 1 -- 172.21.15.132:0/3302706209 learned_addr learned my addr 172.21.15.132:0/3302706209 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:32.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1105715678 0 0) 0x7fa92c11db40 con 0x7fa92c136e50 2024-01-24T23:26:32.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:32.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:32.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:32.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient(hunting): my global_id is 94653 2024-01-24T23:26:32.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa918003530 con 0x7fa92c136e50 2024-01-24T23:26:32.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 561756971 0 0) 0x7fa92c136c20 con 0x7fa92c0a0290 2024-01-24T23:26:32.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:32.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:32.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:32.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient(hunting): my global_id is 94540 2024-01-24T23:26:32.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa92c11db40 con 0x7fa92c0a0290 2024-01-24T23:26:32.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252607244 0 0) 0x7fa92c09ea70 con 0x7fa92c0a1600 2024-01-24T23:26:32.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:32.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:32.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:32.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient(hunting): my global_id is 6414 2024-01-24T23:26:32.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa92c136c20 con 0x7fa92c0a1600 2024-01-24T23:26:32.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 111777113 0 0) 0x7fa92c136c20 con 0x7fa92c0a1600 2024-01-24T23:26:32.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa92c09ea70 con 0x7fa92c0a1600 2024-01-24T23:26:32.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa920004120 con 0x7fa92c0a1600 2024-01-24T23:26:32.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:32.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:26:32.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient(hunting): dump: 2024-01-24T23:26:32.216 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:32.216 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:32.216 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:32.217 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:32.217 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:32.217 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:32.217 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:32.217 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:32.217 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:32.217 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:32.217 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:32.217 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:32.217 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:32.217 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:32.217 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:32.217 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:32.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:32.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3938837675 0 0) 0x7fa92c11db40 con 0x7fa92c0a0290 2024-01-24T23:26:32.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa92c136c20 con 0x7fa92c0a0290 2024-01-24T23:26:32.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2593048649 0 0) 0x7fa918003530 con 0x7fa92c136e50 2024-01-24T23:26:32.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa92c11db40 con 0x7fa92c136e50 2024-01-24T23:26:32.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa91c004300 con 0x7fa92c0a0290 2024-01-24T23:26:32.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:32.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:26:32.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient(hunting): dump: 2024-01-24T23:26:32.218 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:32.218 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:32.218 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:32.218 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:32.218 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:32.218 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:32.219 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:32.219 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:32.219 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:32.219 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:32.219 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:32.219 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:32.219 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:32.219 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:32.219 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:32.219 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:32.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa924001b00 con 0x7fa92c136e50 2024-01-24T23:26:32.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:32.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:26:32.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient(hunting): dump: 2024-01-24T23:26:32.220 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:32.220 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:32.220 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:32.220 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:32.220 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:32.220 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:32.220 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:32.220 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:32.220 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:32.220 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:32.220 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:32.220 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:32.220 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:32.221 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:32.221 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:32.221 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:32.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3009880108 0 0) 0x7fa92c09ea70 con 0x7fa92c0a1600 2024-01-24T23:26:32.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 >> v1:172.21.15.115:6791/0 conn(0x7fa92c0a0290 legacy=0x7fa92c131ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:32.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 >> v1:172.21.15.115:6789/0 conn(0x7fa92c136e50 legacy=0x7fa92c1d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:32.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient: _finish_hunting 0 2024-01-24T23:26:32.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 1 monclient: found mon.d 2024-01-24T23:26:32.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:32.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:32.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92c11f260 con 0x7fa92c0a1600 2024-01-24T23:26:32.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient: _finish_auth 0 2024-01-24T23:26:32.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient: _check_auth_tickets 2024-01-24T23:26:32.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:32.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 5 monclient: authenticate success, global_id 6414 2024-01-24T23:26:32.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 10 monclient: _renew_subs 2024-01-24T23:26:32.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:32.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 1 -- 172.21.15.132:0/3302706209 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa92c1cb540 con 0x7fa92c0a1600 2024-01-24T23:26:32.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 10 monclient: _renew_subs 2024-01-24T23:26:32.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:32.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa93453e640 1 -- 172.21.15.132:0/3302706209 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92c11d7f0 con 0x7fa92c0a1600 2024-01-24T23:26:32.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa9200046b0 con 0x7fa92c0a1600 2024-01-24T23:26:32.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:32.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa930ad7640 4 set_mon_vals no callback set 2024-01-24T23:26:32.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa920004b60 con 0x7fa92c0a1600 2024-01-24T23:26:32.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:32.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:26:32.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.199+0000 7fa92affd640 10 monclient: dump: 2024-01-24T23:26:32.222 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:32.223 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:32.223 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:32.223 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:32.223 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:32.223 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:32.223 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:32.223 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:32.223 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:32.223 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:32.223 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:32.223 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:32.223 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:32.223 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:32.224 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:32.224 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:32.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.203+0000 7fa93453e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:32.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.203+0000 7fa93453e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:32.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.203+0000 7fa93453e640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:32.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.203+0000 7fa93453e640 1 -- 172.21.15.132:0/3302706209 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa90c005090 con 0x7fa92c0a1600 2024-01-24T23:26:32.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.203+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fa920004d40 con 0x7fa92c0a1600 2024-01-24T23:26:32.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.203+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(174..174 src has 1..174) v4 ==== 7211+0+0 (unknown 3018107422 0 0) 0x7fa920044310 con 0x7fa92c0a1600 2024-01-24T23:26:32.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.203+0000 7fa9322da640 10 monclient: get_auth_request con 0x7fa918037e20 auth_method 0 2024-01-24T23:26:32.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.215+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fa920014810 con 0x7fa92c0a1600 2024-01-24T23:26:32.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.215+0000 7fa92affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:32.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.215+0000 7fa92affd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:32.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.375+0000 7fa93453e640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "barr"}] 2024-01-24T23:26:32.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.375+0000 7fa93453e640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "barr"}] 2024-01-24T23:26:32.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.375+0000 7fa93453e640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:32.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.375+0000 7fa93453e640 1 -- 172.21.15.132:0/3302706209 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "barr"} v 0) v1 -- 0x7fa90c005340 con 0x7fa92c0a1600 2024-01-24T23:26:32.984 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.983+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "barr"}]=0 removed item id -19 name 'barr' from crush map v175) v1 ==== 122+0+0 (unknown 1957613055 0 0) 0x7fa920012c80 con 0x7fa92c0a1600 2024-01-24T23:26:32.984 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.983+0000 7fa92affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "barr"}] 2024-01-24T23:26:32.984 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:32.983+0000 7fa92affd640 10 monclient: _finish_command 2 = system:0 removed item id -19 name 'barr' from crush map 2024-01-24T23:26:33.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.087+0000 7fa93453e640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "barr"}] 2024-01-24T23:26:33.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.087+0000 7fa93453e640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "barr"}] 2024-01-24T23:26:33.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.087+0000 7fa93453e640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:33.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.087+0000 7fa93453e640 1 -- 172.21.15.132:0/3302706209 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "barr"} v 0) v1 -- 0x7fa90c0010a0 con 0x7fa92c0a1600 2024-01-24T23:26:33.091 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.091+0000 7fa92affd640 1 -- 172.21.15.132:0/3302706209 <== mon.6 v1:172.21.15.115:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "barr"}]=0 device 'barr' does not appear in the crush map v175) v1 ==== 122+0+0 (unknown 1064413280 0 0) 0x7fa920014400 con 0x7fa92c0a1600 2024-01-24T23:26:33.091 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.091+0000 7fa92affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "barr"}] 2024-01-24T23:26:33.091 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.091+0000 7fa92affd640 10 monclient: _finish_command 3 = system:0 device 'barr' does not appear in the crush map 2024-01-24T23:26:33.091 INFO:tasks.workunit.client.0.smithi132.stderr:device 'barr' does not appear in the crush map 2024-01-24T23:26:33.093 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.091+0000 7fa93453e640 1 -- 172.21.15.132:0/3302706209 >> v1:172.21.15.132:6812/1835401922 conn(0x7fa918037e20 legacy=0x7fa91803a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:33.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.091+0000 7fa93453e640 10 monclient: shutdown 2024-01-24T23:26:33.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.091+0000 7fa93453e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:33.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.091+0000 7fa93453e640 1 -- 172.21.15.132:0/3302706209 >> v1:172.21.15.115:6792/0 conn(0x7fa92c0a1600 legacy=0x7fa92c1d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:33.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.091+0000 7fa93453e640 1 -- 172.21.15.132:0/3302706209 shutdown_connections 2024-01-24T23:26:33.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.091+0000 7fa93453e640 1 -- 172.21.15.132:0/3302706209 >> 172.21.15.132:0/3302706209 conn(0x7fa92c0a5a10 msgr2=0x7fa92c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:33.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.091+0000 7fa93453e640 1 -- 172.21.15.132:0/3302706209 shutdown_connections 2024-01-24T23:26:33.094 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.091+0000 7fa93453e640 1 -- 172.21.15.132:0/3302706209 wait complete. 2024-01-24T23:26:33.108 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set rm-compat 2024-01-24T23:26:33.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3331981640 10 monclient: get_monmap_and_config 2024-01-24T23:26:33.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3331981640 10 monclient: build_initial_monmap 2024-01-24T23:26:33.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3331981640 10 monclient: monmap: 2024-01-24T23:26:33.225 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:33.225 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:33.225 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:33.226103+0000 2024-01-24T23:26:33.225 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:33.226103+0000 2024-01-24T23:26:33.225 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:33.225 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:33.225 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:33.225 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:33.226 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:33.226 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:33.226 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:33.226 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:33.226 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:33.226 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:33.226 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:33.226 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:33.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3331981640 1 Processor -- start 2024-01-24T23:26:33.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3331981640 1 -- start start 2024-01-24T23:26:33.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3331981640 10 monclient: init 2024-01-24T23:26:33.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3331981640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:33.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3331981640 10 monclient: _add_conns ranks=[5,0,7,2,8,3,1,4,6] 2024-01-24T23:26:33.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3331981640 10 monclient(hunting): picked mon.noname-h con 0x7f332c133b50 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:33.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3331981640 10 monclient(hunting): picked mon.noname-a con 0x7f332c09ec20 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:33.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3331981640 10 monclient(hunting): picked mon.noname-i con 0x7f332c13a1d0 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:33.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3331981640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f332c064700 con 0x7f332c09ec20 2024-01-24T23:26:33.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3331981640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f332c09d9f0 con 0x7f332c133b50 2024-01-24T23:26:33.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3331981640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f332c09dc60 con 0x7f332c13a1d0 2024-01-24T23:26:33.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3331981640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:33.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3331981640 10 monclient(hunting): authenticate will time out at 1280.550738s 2024-01-24T23:26:33.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f332b7fe640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f332c13a1d0 legacy=0x7f332c13c5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:33.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f332affd640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f332c133b50 0x7f332c133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:46516/0 (socket says 172.21.15.132:46516) 2024-01-24T23:26:33.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f332affd640 1 -- 172.21.15.132:0/1439934067 learned_addr learned my addr 172.21.15.132:0/1439934067 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:33.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 1 -- 172.21.15.132:0/1439934067 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1835773720 0 0) 0x7f332c09d9f0 con 0x7f332c133b50 2024-01-24T23:26:33.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:26:33.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:33.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:33.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 10 monclient(hunting): my global_id is 6206 2024-01-24T23:26:33.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 1 -- 172.21.15.132:0/1439934067 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3314003530 con 0x7f332c133b50 2024-01-24T23:26:33.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 1 -- 172.21.15.132:0/1439934067 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1711749152 0 0) 0x7f3314003530 con 0x7f332c133b50 2024-01-24T23:26:33.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 1 -- 172.21.15.132:0/1439934067 >> v1:172.21.15.132:6792/0 conn(0x7f332c13a1d0 legacy=0x7f332c13c5c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:33.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 1 -- 172.21.15.132:0/1439934067 >> v1:172.21.15.115:6789/0 conn(0x7f332c09ec20 legacy=0x7f332c138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:33.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:26:33.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 1 monclient: found mon.noname-h 2024-01-24T23:26:33.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:33.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:26:33.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 1 -- 172.21.15.132:0/1439934067 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332c09df40 con 0x7f332c133b50 2024-01-24T23:26:33.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 10 monclient: _finish_auth 0 2024-01-24T23:26:33.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:26:33.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:33.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 1 -- 172.21.15.132:0/1439934067 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f33200017c0 con 0x7f332c133b50 2024-01-24T23:26:33.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3331981640 5 monclient: authenticate success, global_id 6206 2024-01-24T23:26:33.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3331981640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:33.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:33.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:33.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 10 monclient: dump: 2024-01-24T23:26:33.238 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:33.238 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:33.238 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:33.238 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:33.238 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:33.238 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:33.238 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:33.238 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:33.238 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:33.238 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:33.239 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:33.239 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:33.239 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:33.239 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:33.239 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:33.239 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:33.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3331981640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:33.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 1 -- 172.21.15.132:0/1439934067 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3320001ae0 con 0x7f332c133b50 2024-01-24T23:26:33.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:33.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3331981640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:33.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 1 -- 172.21.15.132:0/1439934067 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3320004bf0 con 0x7f332c133b50 2024-01-24T23:26:33.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3331981640 4 set_mon_vals no callback set 2024-01-24T23:26:33.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:33.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:33.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.223+0000 7f3329ffb640 10 monclient: dump: 2024-01-24T23:26:33.240 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:33.240 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:33.240 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:33.240 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:33.240 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:33.240 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:33.240 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:33.240 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:33.240 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:33.240 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:33.240 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:33.240 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:33.240 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:33.240 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:33.241 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:33.241 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:33.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 10 monclient: shutdown 2024-01-24T23:26:33.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:33.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 1 -- 172.21.15.132:0/1439934067 >> v1:172.21.15.132:6791/0 conn(0x7f332c133b50 legacy=0x7f332c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:33.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 1 -- 172.21.15.132:0/1439934067 shutdown_connections 2024-01-24T23:26:33.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 1 -- 172.21.15.132:0/1439934067 >> 172.21.15.132:0/1439934067 conn(0x7f332c0a5a10 msgr2=0x7f332c0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:33.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 1 -- 172.21.15.132:0/1439934067 shutdown_connections 2024-01-24T23:26:33.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 1 -- 172.21.15.132:0/1439934067 wait complete. 2024-01-24T23:26:33.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 10 monclient: build_initial_monmap 2024-01-24T23:26:33.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 10 monclient: monmap: 2024-01-24T23:26:33.241 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:33.241 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:33.241 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:33.241 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:33.241 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:33.242 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:33.242 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:33.242 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:33.242 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:33.242 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:33.242 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:33.242 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:33.242 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:33.242 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:33.242 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:33.242 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:33.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 1 Processor -- start 2024-01-24T23:26:33.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 1 -- start start 2024-01-24T23:26:33.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 10 monclient: init 2024-01-24T23:26:33.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:33.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 10 monclient: _add_conns ranks=[0,4,8,6,7,1,5,3,2] 2024-01-24T23:26:33.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 10 monclient(hunting): picked mon.noname-a con 0x7f332c09ec20 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:33.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 10 monclient(hunting): picked mon.noname-c con 0x7f332c133b50 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:33.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 10 monclient(hunting): picked mon.noname-e con 0x7f332c13a1d0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:33.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f332c1bf2c0 con 0x7f332c09ec20 2024-01-24T23:26:33.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f332c11d610 con 0x7f332c133b50 2024-01-24T23:26:33.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f332c0a13b0 con 0x7f332c13a1d0 2024-01-24T23:26:33.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:33.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 10 monclient(hunting): authenticate will time out at 1280.553916s 2024-01-24T23:26:33.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f332a7fc640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f332c133b50 0x7f332c1ce5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:52246/0 (socket says 172.21.15.132:52246) 2024-01-24T23:26:33.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f332affd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f332c09ec20 0x7f332c1c9a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:40460/0 (socket says 172.21.15.132:40460) 2024-01-24T23:26:33.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f332a7fc640 1 -- 172.21.15.132:0/169836628 learned_addr learned my addr 172.21.15.132:0/169836628 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:33.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515066038 0 0) 0x7f332c11d610 con 0x7f332c133b50 2024-01-24T23:26:33.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient(hunting): _negotiate 2024-01-24T23:26:33.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:33.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:33.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient(hunting): my global_id is 94549 2024-01-24T23:26:33.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3318003530 con 0x7f332c133b50 2024-01-24T23:26:33.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2232121340 0 0) 0x7f332c1bf2c0 con 0x7f332c09ec20 2024-01-24T23:26:33.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient(hunting): _negotiate 2024-01-24T23:26:33.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:33.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:33.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient(hunting): my global_id is 94671 2024-01-24T23:26:33.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f332c11d610 con 0x7f332c09ec20 2024-01-24T23:26:33.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4241288401 0 0) 0x7f332c0a13b0 con 0x7f332c13a1d0 2024-01-24T23:26:33.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient(hunting): _negotiate 2024-01-24T23:26:33.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:33.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:33.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient(hunting): my global_id is 85553 2024-01-24T23:26:33.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f332c1bf2c0 con 0x7f332c13a1d0 2024-01-24T23:26:33.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1349197966 0 0) 0x7f332c1bf2c0 con 0x7f332c13a1d0 2024-01-24T23:26:33.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f332c0a13b0 con 0x7f332c13a1d0 2024-01-24T23:26:33.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f33240044e0 con 0x7f332c13a1d0 2024-01-24T23:26:33.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:33.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:26:33.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient(hunting): dump: 2024-01-24T23:26:33.245 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:33.245 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:33.246 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:33.246 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:33.246 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:33.246 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:33.246 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:33.246 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:33.246 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:33.246 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:33.246 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:33.246 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:33.246 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:33.246 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:33.246 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:33.246 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:33.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:33.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 292729925 0 0) 0x7f3318003530 con 0x7f332c133b50 2024-01-24T23:26:33.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f332c1bf2c0 con 0x7f332c133b50 2024-01-24T23:26:33.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f331c001ae0 con 0x7f332c133b50 2024-01-24T23:26:33.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:33.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:26:33.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient(hunting): dump: 2024-01-24T23:26:33.247 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:33.247 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:33.247 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:33.247 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:33.247 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:33.247 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:33.247 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:33.248 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:33.248 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:33.248 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:33.248 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:33.248 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:33.248 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:33.248 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:33.248 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:33.248 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:33.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1068297089 0 0) 0x7f332c11d610 con 0x7f332c09ec20 2024-01-24T23:26:33.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3318003530 con 0x7f332c09ec20 2024-01-24T23:26:33.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3320004b30 con 0x7f332c09ec20 2024-01-24T23:26:33.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:33.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:26:33.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient(hunting): dump: 2024-01-24T23:26:33.249 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:33.249 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:33.249 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:33.249 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:33.249 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:33.249 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:33.249 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:33.249 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:33.249 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:33.249 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:33.249 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:33.249 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:33.249 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:33.249 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:33.250 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:33.250 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:33.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2180110934 0 0) 0x7f332c0a13b0 con 0x7f332c13a1d0 2024-01-24T23:26:33.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 >> v1:172.21.15.115:6791/0 conn(0x7f332c133b50 legacy=0x7f332c1ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:33.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 >> v1:172.21.15.115:6789/0 conn(0x7f332c09ec20 legacy=0x7f332c1c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:33.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient: _finish_hunting 0 2024-01-24T23:26:33.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 1 monclient: found mon.e 2024-01-24T23:26:33.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:33.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:33.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332c133920 con 0x7f332c13a1d0 2024-01-24T23:26:33.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient: _finish_auth 0 2024-01-24T23:26:33.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 10 monclient: _check_auth_tickets 2024-01-24T23:26:33.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3313fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:33.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 5 monclient: authenticate success, global_id 85553 2024-01-24T23:26:33.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 10 monclient: _renew_subs 2024-01-24T23:26:33.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:33.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.227+0000 7f3331981640 1 -- 172.21.15.132:0/169836628 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f332c1ca150 con 0x7f332c13a1d0 2024-01-24T23:26:33.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.231+0000 7f3331981640 10 monclient: _renew_subs 2024-01-24T23:26:33.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.231+0000 7f3331981640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:33.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.231+0000 7f3331981640 1 -- 172.21.15.132:0/169836628 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f332c11eac0 con 0x7f332c13a1d0 2024-01-24T23:26:33.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.231+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3324004a40 con 0x7f332c13a1d0 2024-01-24T23:26:33.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.231+0000 7f3313fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:33.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.231+0000 7f3328ff9640 4 set_mon_vals no callback set 2024-01-24T23:26:33.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.231+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3324004f70 con 0x7f332c13a1d0 2024-01-24T23:26:33.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.231+0000 7f3313fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:33.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.231+0000 7f3313fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:26:33.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.231+0000 7f3313fff640 10 monclient: dump: 2024-01-24T23:26:33.251 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:33.251 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:33.251 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:33.251 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:33.252 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:33.252 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:33.252 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:33.252 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:33.252 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:33.252 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:33.252 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:33.252 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:33.252 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:33.252 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:33.252 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:33.252 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:33.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.231+0000 7f3331981640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:33.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.231+0000 7f3331981640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:33.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.231+0000 7f3331981640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:33.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.231+0000 7f3331981640 1 -- 172.21.15.132:0/169836628 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f330c005090 con 0x7f332c13a1d0 2024-01-24T23:26:33.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.231+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f33240050d0 con 0x7f332c13a1d0 2024-01-24T23:26:33.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.231+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(175..175 src has 1..175) v4 ==== 6995+0+0 (unknown 3278518893 0 0) 0x7f3324044520 con 0x7f332c13a1d0 2024-01-24T23:26:33.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.231+0000 7f332affd640 10 monclient: get_auth_request con 0x7f3318037b90 auth_method 0 2024-01-24T23:26:33.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.243+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f3324014af0 con 0x7f332c13a1d0 2024-01-24T23:26:33.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.243+0000 7f3313fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:33.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.243+0000 7f3313fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:33.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.391+0000 7f3331981640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-01-24T23:26:33.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.391+0000 7f3331981640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-01-24T23:26:33.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.391+0000 7f3331981640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:33.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:33.391+0000 7f3331981640 1 -- 172.21.15.132:0/169836628 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) v1 -- 0x7f330c005340 con 0x7f332c13a1d0 2024-01-24T23:26:34.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:34.007+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v176) v1 ==== 78+0+0 (unknown 3490145479 0 0) 0x7f33240148c0 con 0x7f332c13a1d0 2024-01-24T23:26:34.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:34.007+0000 7f3313fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-01-24T23:26:34.009 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:34.007+0000 7f3313fff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:26:34.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:34.111+0000 7f3331981640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-01-24T23:26:34.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:34.111+0000 7f3331981640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-01-24T23:26:34.113 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:34.111+0000 7f3331981640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:34.114 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:34.111+0000 7f3331981640 1 -- 172.21.15.132:0/169836628 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) v1 -- 0x7f330c005a80 con 0x7f332c13a1d0 2024-01-24T23:26:35.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.011+0000 7f3313fff640 1 -- 172.21.15.132:0/169836628 <== mon.8 v1:172.21.15.115:6793/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v177) v1 ==== 78+0+0 (unknown 2445684450 0 0) 0x7f33240148c0 con 0x7f332c13a1d0 2024-01-24T23:26:35.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.011+0000 7f3313fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-01-24T23:26:35.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.011+0000 7f3313fff640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:26:35.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.015+0000 7f3331981640 1 -- 172.21.15.132:0/169836628 >> v1:172.21.15.132:6812/1835401922 conn(0x7f3318037b90 legacy=0x7f331803a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:35.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.015+0000 7f3331981640 10 monclient: shutdown 2024-01-24T23:26:35.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.015+0000 7f3331981640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:35.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.015+0000 7f3331981640 1 -- 172.21.15.132:0/169836628 >> v1:172.21.15.115:6793/0 conn(0x7f332c13a1d0 legacy=0x7f332c1d1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:35.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.015+0000 7f3331981640 1 -- 172.21.15.132:0/169836628 shutdown_connections 2024-01-24T23:26:35.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.015+0000 7f3331981640 1 -- 172.21.15.132:0/169836628 >> 172.21.15.132:0/169836628 conn(0x7f332c0a5a10 msgr2=0x7f332c071700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:35.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.015+0000 7f3331981640 1 -- 172.21.15.132:0/169836628 shutdown_connections 2024-01-24T23:26:35.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.015+0000 7f3331981640 1 -- 172.21.15.132:0/169836628 wait complete. 2024-01-24T23:26:35.036 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set create-compat 2024-01-24T23:26:35.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 10 monclient: get_monmap_and_config 2024-01-24T23:26:35.148 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 10 monclient: build_initial_monmap 2024-01-24T23:26:35.149 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 10 monclient: monmap: 2024-01-24T23:26:35.149 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:35.149 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:35.149 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:35.149868+0000 2024-01-24T23:26:35.149 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:35.149868+0000 2024-01-24T23:26:35.149 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:35.149 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:35.149 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:35.149 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:35.149 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:35.149 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:35.149 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:35.150 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:35.150 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:35.150 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:35.150 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:35.150 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:35.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 1 Processor -- start 2024-01-24T23:26:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 1 -- start start 2024-01-24T23:26:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 10 monclient: init 2024-01-24T23:26:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 10 monclient: _add_conns ranks=[7,5,2,4,1,8,6,3,0] 2024-01-24T23:26:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 10 monclient(hunting): picked mon.noname-i con 0x7fbdc409ff20 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 10 monclient(hunting): picked mon.noname-h con 0x7fbdc409f9c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 10 monclient(hunting): picked mon.noname-b con 0x7fbdc413b2c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc4064700 con 0x7fbdc413b2c0 2024-01-24T23:26:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc9711640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fbdc409ff20 legacy=0x7fbdc40a0300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc409e760 con 0x7fbdc409f9c0 2024-01-24T23:26:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc40a1160 con 0x7fbdc409ff20 2024-01-24T23:26:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 10 monclient(hunting): authenticate will time out at 1282.474626s 2024-01-24T23:26:35.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc8f10640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fbdc409f9c0 0x7fbdc4139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:54844/0 (socket says 172.21.15.132:54844) 2024-01-24T23:26:35.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc8f10640 1 -- 172.21.15.132:0/1350029752 learned_addr learned my addr 172.21.15.132:0/1350029752 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:35.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc9f12640 1 -- 172.21.15.132:0/1350029752 >> v1:172.21.15.115:6790/0 conn(0x7fbdc413b2c0 legacy=0x7fbdc413d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:35.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc3fff640 1 -- 172.21.15.132:0/1350029752 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2229190139 0 0) 0x7fbdc409e760 con 0x7fbdc409f9c0 2024-01-24T23:26:35.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:26:35.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:35.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:35.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc3fff640 10 monclient(hunting): my global_id is 6215 2024-01-24T23:26:35.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc3fff640 1 -- 172.21.15.132:0/1350029752 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdac003530 con 0x7fbdc409f9c0 2024-01-24T23:26:35.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc3fff640 1 -- 172.21.15.132:0/1350029752 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3150690551 0 0) 0x7fbdac003530 con 0x7fbdc409f9c0 2024-01-24T23:26:35.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc3fff640 1 -- 172.21.15.132:0/1350029752 >> v1:172.21.15.132:6792/0 conn(0x7fbdc409ff20 legacy=0x7fbdc40a0300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:35.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc3fff640 1 -- 172.21.15.132:0/1350029752 >> v1:172.21.15.115:6790/0 conn(0x7fbdc413b2c0 legacy=0x7fbdc413d6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:35.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc3fff640 10 monclient: _finish_hunting 0 2024-01-24T23:26:35.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc3fff640 1 monclient: found mon.noname-h 2024-01-24T23:26:35.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:35.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc3fff640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:26:35.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc3fff640 1 -- 172.21.15.132:0/1350029752 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc40a0a10 con 0x7fbdc409f9c0 2024-01-24T23:26:35.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc3fff640 10 monclient: _finish_auth 0 2024-01-24T23:26:35.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc3fff640 10 monclient: _check_auth_tickets 2024-01-24T23:26:35.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:35.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc3fff640 1 -- 172.21.15.132:0/1350029752 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbdb80017c0 con 0x7fbdc409f9c0 2024-01-24T23:26:35.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 5 monclient: authenticate success, global_id 6215 2024-01-24T23:26:35.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:35.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:35.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc3fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:35.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc3fff640 10 monclient: dump: 2024-01-24T23:26:35.160 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:35.160 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:35.160 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:35.160 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:35.160 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:35.160 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:35.160 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:35.160 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:35.160 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:35.160 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:35.160 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:35.160 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:35.160 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:35.161 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:35.161 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:35.161 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:35.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:35.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc3fff640 1 -- 172.21.15.132:0/1350029752 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbdb8001ae0 con 0x7fbdc409f9c0 2024-01-24T23:26:35.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdc3fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:35.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:35.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 4 set_mon_vals no callback set 2024-01-24T23:26:35.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 10 monclient: shutdown 2024-01-24T23:26:35.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:35.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 1 -- 172.21.15.132:0/1350029752 >> v1:172.21.15.132:6791/0 conn(0x7fbdc409f9c0 legacy=0x7fbdc4139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:35.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 1 -- 172.21.15.132:0/1350029752 shutdown_connections 2024-01-24T23:26:35.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 1 -- 172.21.15.132:0/1350029752 >> 172.21.15.132:0/1350029752 conn(0x7fbdc40a5a10 msgr2=0x7fbdc40a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:35.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 1 -- 172.21.15.132:0/1350029752 shutdown_connections 2024-01-24T23:26:35.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.147+0000 7fbdcb975640 1 -- 172.21.15.132:0/1350029752 wait complete. 2024-01-24T23:26:35.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 10 monclient: build_initial_monmap 2024-01-24T23:26:35.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 10 monclient: monmap: 2024-01-24T23:26:35.162 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:35.162 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:35.162 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:35.162 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:35.162 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:35.162 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:35.162 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:35.162 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:35.162 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:35.162 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:35.162 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:35.163 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:35.163 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:35.163 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:35.163 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:35.163 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:35.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 1 Processor -- start 2024-01-24T23:26:35.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 1 -- start start 2024-01-24T23:26:35.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 10 monclient: init 2024-01-24T23:26:35.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:35.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 10 monclient: _add_conns ranks=[2,3,8,6,7,4,0,1,5] 2024-01-24T23:26:35.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 10 monclient(hunting): picked mon.noname-b con 0x7fbdc409f9c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:35.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 10 monclient(hunting): picked mon.noname-g con 0x7fbdc409ff20 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:35.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 10 monclient(hunting): picked mon.noname-e con 0x7fbdc413b2c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:35.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc4071680 con 0x7fbdc409f9c0 2024-01-24T23:26:35.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc411e970 con 0x7fbdc409ff20 2024-01-24T23:26:35.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc411c430 con 0x7fbdc413b2c0 2024-01-24T23:26:35.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:35.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 10 monclient(hunting): authenticate will time out at 1282.477405s 2024-01-24T23:26:35.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc8f10640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fbdc409ff20 0x7fbdc411bd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:48680/0 (socket says 172.21.15.132:48680) 2024-01-24T23:26:35.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc8f10640 1 -- 172.21.15.132:0/4149782176 learned_addr learned my addr 172.21.15.132:0/4149782176 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:35.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc9711640 1 -- 172.21.15.132:0/4149782176 >> v1:172.21.15.115:6790/0 conn(0x7fbdc409f9c0 legacy=0x7fbdc411b610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:35.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 1 -- 172.21.15.132:0/4149782176 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3798406550 0 0) 0x7fbdc411e970 con 0x7fbdc409ff20 2024-01-24T23:26:35.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:26:35.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:35.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:35.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 10 monclient(hunting): my global_id is 6339 2024-01-24T23:26:35.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 1 -- 172.21.15.132:0/4149782176 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdb0003530 con 0x7fbdc409ff20 2024-01-24T23:26:35.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 1 -- 172.21.15.132:0/4149782176 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1988000780 0 0) 0x7fbdb0003530 con 0x7fbdc409ff20 2024-01-24T23:26:35.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 1 -- 172.21.15.132:0/4149782176 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbdc411e970 con 0x7fbdc409ff20 2024-01-24T23:26:35.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 1 -- 172.21.15.132:0/4149782176 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbdb8001d70 con 0x7fbdc409ff20 2024-01-24T23:26:35.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:35.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:35.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 10 monclient(hunting): dump: 2024-01-24T23:26:35.166 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:35.166 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:35.166 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:35.166 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:35.166 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:35.166 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:35.166 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:35.166 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:35.166 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:35.166 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:35.166 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:35.166 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:35.167 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 1 -- 172.21.15.132:0/4149782176 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1229365916 0 0) 0x7fbdc411e970 con 0x7fbdc409ff20 2024-01-24T23:26:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 1 -- 172.21.15.132:0/4149782176 >> v1:172.21.15.115:6793/0 conn(0x7fbdc413b2c0 legacy=0x7fbdc41d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 1 -- 172.21.15.132:0/4149782176 >> v1:172.21.15.115:6790/0 conn(0x7fbdc409f9c0 legacy=0x7fbdc411b610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:26:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 1 monclient: found mon.g 2024-01-24T23:26:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 1 -- 172.21.15.132:0/4149782176 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc4071960 con 0x7fbdc409ff20 2024-01-24T23:26:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 10 monclient: _finish_auth 0 2024-01-24T23:26:35.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:26:35.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:35.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 5 monclient: authenticate success, global_id 6339 2024-01-24T23:26:35.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 10 monclient: _renew_subs 2024-01-24T23:26:35.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:35.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 1 -- 172.21.15.132:0/4149782176 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc411c6f0 con 0x7fbdc409ff20 2024-01-24T23:26:35.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 10 monclient: _renew_subs 2024-01-24T23:26:35.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:35.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 1 -- 172.21.15.132:0/4149782176 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc411cf30 con 0x7fbdc409ff20 2024-01-24T23:26:35.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 1 -- 172.21.15.132:0/4149782176 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbdb8004de0 con 0x7fbdc409ff20 2024-01-24T23:26:35.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:35.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 1 -- 172.21.15.132:0/4149782176 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbdb8005310 con 0x7fbdc409ff20 2024-01-24T23:26:35.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:35.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc37fe640 4 set_mon_vals no callback set 2024-01-24T23:26:35.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:35.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 10 monclient: dump: 2024-01-24T23:26:35.169 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:35.169 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:35.169 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:35.169 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:35.169 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:35.169 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:35.169 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:35.169 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:35.169 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:35.169 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:35.169 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:35.169 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:35.170 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:35.170 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:35.170 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:35.170 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:35.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:35.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:35.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:35.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdcb975640 1 -- 172.21.15.132:0/4149782176 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda4005090 con 0x7fbdc409ff20 2024-01-24T23:26:35.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 1 -- 172.21.15.132:0/4149782176 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fbdb8005470 con 0x7fbdc409ff20 2024-01-24T23:26:35.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc1ffb640 1 -- 172.21.15.132:0/4149782176 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(177..177 src has 1..177) v4 ==== 6667+0+0 (unknown 3291334250 0 0) 0x7fbdb8044650 con 0x7fbdc409ff20 2024-01-24T23:26:35.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.151+0000 7fbdc9711640 10 monclient: get_auth_request con 0x7fbdb0036f30 auth_method 0 2024-01-24T23:26:35.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.155+0000 7fbdc1ffb640 1 -- 172.21.15.132:0/4149782176 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fbdb8014dd0 con 0x7fbdc409ff20 2024-01-24T23:26:35.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.155+0000 7fbdc1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:35.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.155+0000 7fbdc1ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:35.288 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~31s 2024-01-24T23:26:35.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.319+0000 7fbdcb975640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-01-24T23:26:35.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.319+0000 7fbdcb975640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-01-24T23:26:35.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.319+0000 7fbdcb975640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:35.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:35.319+0000 7fbdcb975640 1 -- 172.21.15.132:0/4149782176 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) v1 -- 0x7fbda4005340 con 0x7fbdc409ff20 2024-01-24T23:26:36.034 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.031+0000 7fbdc1ffb640 1 -- 172.21.15.132:0/4149782176 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 v178) v1 ==== 82+0+0 (unknown 2206935051 0 0) 0x7fbdb8013240 con 0x7fbdc409ff20 2024-01-24T23:26:36.035 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.031+0000 7fbdc1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-01-24T23:26:36.035 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.031+0000 7fbdc1ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:26:36.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.135+0000 7fbdcb975640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-01-24T23:26:36.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.135+0000 7fbdcb975640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-01-24T23:26:36.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.135+0000 7fbdcb975640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:36.139 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.135+0000 7fbdcb975640 1 -- 172.21.15.132:0/4149782176 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) v1 -- 0x7fbda4002220 con 0x7fbdc409ff20 2024-01-24T23:26:36.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.139+0000 7fbdc1ffb640 1 -- 172.21.15.132:0/4149782176 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 compat weight-set already created v178) v1 ==== 115+0+0 (unknown 3458083800 0 0) 0x7fbdb8014ba0 con 0x7fbdc409ff20 2024-01-24T23:26:36.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.139+0000 7fbdc1ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-01-24T23:26:36.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.139+0000 7fbdc1ffb640 10 monclient: _finish_command 3 = system:0 compat weight-set already created 2024-01-24T23:26:36.140 INFO:tasks.workunit.client.0.smithi132.stderr:compat weight-set already created 2024-01-24T23:26:36.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.139+0000 7fbdcb975640 1 -- 172.21.15.132:0/4149782176 >> v1:172.21.15.132:6812/1835401922 conn(0x7fbdb0036f30 legacy=0x7fbdb00393f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:36.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.139+0000 7fbdcb975640 10 monclient: shutdown 2024-01-24T23:26:36.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.139+0000 7fbdcb975640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:36.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.139+0000 7fbdcb975640 1 -- 172.21.15.132:0/4149782176 >> v1:172.21.15.132:6790/0 conn(0x7fbdc409ff20 legacy=0x7fbdc411bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:36.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.139+0000 7fbdcb975640 1 -- 172.21.15.132:0/4149782176 shutdown_connections 2024-01-24T23:26:36.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.139+0000 7fbdcb975640 1 -- 172.21.15.132:0/4149782176 >> 172.21.15.132:0/4149782176 conn(0x7fbdc40a5a10 msgr2=0x7fbdc4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:36.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.139+0000 7fbdcb975640 1 -- 172.21.15.132:0/4149782176 shutdown_connections 2024-01-24T23:26:36.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.139+0000 7fbdcb975640 1 -- 172.21.15.132:0/4149782176 wait complete. 2024-01-24T23:26:36.158 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket r1 rack root=default 2024-01-24T23:26:36.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.275+0000 7f4b2db0f640 10 monclient: get_monmap_and_config 2024-01-24T23:26:36.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.275+0000 7f4b2db0f640 10 monclient: build_initial_monmap 2024-01-24T23:26:36.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.275+0000 7f4b2db0f640 10 monclient: monmap: 2024-01-24T23:26:36.279 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:36.279 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:36.280 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:36.280693+0000 2024-01-24T23:26:36.280 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:36.280693+0000 2024-01-24T23:26:36.280 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:36.280 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:36.280 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:36.280 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:36.280 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:36.280 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:36.280 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:36.280 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:36.280 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:36.280 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:36.280 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:36.280 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:36.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b2db0f640 1 Processor -- start 2024-01-24T23:26:36.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b2db0f640 1 -- start start 2024-01-24T23:26:36.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b2db0f640 10 monclient: init 2024-01-24T23:26:36.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b2db0f640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:36.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b2db0f640 10 monclient: _add_conns ranks=[8,0,6,4,7,5,1,3,2] 2024-01-24T23:26:36.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b2db0f640 10 monclient(hunting): picked mon.noname-e con 0x7f4b2809b6a0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:36.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b2db0f640 10 monclient(hunting): picked mon.noname-a con 0x7f4b2809ba80 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:36.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b2db0f640 10 monclient(hunting): picked mon.noname-d con 0x7f4b2809be60 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:36.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b2db0f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b28064700 con 0x7f4b2809ba80 2024-01-24T23:26:36.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b2db0f640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b280a16f0 con 0x7f4b2809be60 2024-01-24T23:26:36.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b2db0f640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b28099980 con 0x7f4b2809b6a0 2024-01-24T23:26:36.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b2db0f640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:36.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b2db0f640 10 monclient(hunting): authenticate will time out at 1283.605577s 2024-01-24T23:26:36.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b26ffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f4b2809ba80 0x7f4b28139d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:47040/0 (socket says 172.21.15.132:47040) 2024-01-24T23:26:36.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b26ffd640 1 -- 172.21.15.132:0/192839699 learned_addr learned my addr 172.21.15.132:0/192839699 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:36.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 1 -- 172.21.15.132:0/192839699 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3102480463 0 0) 0x7f4b280a16f0 con 0x7f4b2809be60 2024-01-24T23:26:36.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 10 monclient(hunting): _negotiate 2024-01-24T23:26:36.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:36.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:36.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 10 monclient(hunting): my global_id is 6423 2024-01-24T23:26:36.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 1 -- 172.21.15.132:0/192839699 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b10003530 con 0x7f4b2809be60 2024-01-24T23:26:36.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 1 -- 172.21.15.132:0/192839699 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812035254 0 0) 0x7f4b28099980 con 0x7f4b2809b6a0 2024-01-24T23:26:36.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 10 monclient(hunting): _negotiate 2024-01-24T23:26:36.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:36.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:36.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 10 monclient(hunting): my global_id is 85571 2024-01-24T23:26:36.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 1 -- 172.21.15.132:0/192839699 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b280a16f0 con 0x7f4b2809b6a0 2024-01-24T23:26:36.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 1 -- 172.21.15.132:0/192839699 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3226983810 0 0) 0x7f4b28064700 con 0x7f4b2809ba80 2024-01-24T23:26:36.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 10 monclient(hunting): _negotiate 2024-01-24T23:26:36.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:36.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:36.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 10 monclient(hunting): my global_id is 94680 2024-01-24T23:26:36.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 1 -- 172.21.15.132:0/192839699 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b28099980 con 0x7f4b2809ba80 2024-01-24T23:26:36.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 1 -- 172.21.15.132:0/192839699 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2094542343 0 0) 0x7f4b10003530 con 0x7f4b2809be60 2024-01-24T23:26:36.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 1 -- 172.21.15.132:0/192839699 >> v1:172.21.15.115:6793/0 conn(0x7f4b2809b6a0 legacy=0x7f4b280a02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:36.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 1 -- 172.21.15.132:0/192839699 >> v1:172.21.15.115:6789/0 conn(0x7f4b2809ba80 legacy=0x7f4b28139d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:36.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 10 monclient: _finish_hunting 0 2024-01-24T23:26:36.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 1 monclient: found mon.noname-d 2024-01-24T23:26:36.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:36.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:26:36.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 1 -- 172.21.15.132:0/192839699 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b280a09f0 con 0x7f4b2809be60 2024-01-24T23:26:36.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 10 monclient: _finish_auth 0 2024-01-24T23:26:36.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 10 monclient: _check_auth_tickets 2024-01-24T23:26:36.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:36.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 1 -- 172.21.15.132:0/192839699 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4b20001b40 con 0x7f4b2809be60 2024-01-24T23:26:36.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:36.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:26:36.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 10 monclient: dump: 2024-01-24T23:26:36.287 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:36.287 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:36.287 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:36.287 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:36.287 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:36.287 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:36.287 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:36.288 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:36.288 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:36.288 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:36.288 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:36.288 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:36.288 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:36.288 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:36.288 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:36.288 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:36.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b2db0f640 5 monclient: authenticate success, global_id 6423 2024-01-24T23:26:36.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b2db0f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:36.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 1 -- 172.21.15.132:0/192839699 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4b20001e20 con 0x7f4b2809be60 2024-01-24T23:26:36.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:36.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 1 -- 172.21.15.132:0/192839699 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4b20003ce0 con 0x7f4b2809be60 2024-01-24T23:26:36.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:36.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:26:36.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b267fc640 10 monclient: dump: 2024-01-24T23:26:36.289 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:36.289 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:36.289 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:36.289 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:36.289 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:36.289 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:36.289 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:36.289 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:36.289 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:36.289 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:36.289 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:36.289 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:36.289 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:36.289 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:36.290 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:36.290 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:36.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b2db0f640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:36.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b2db0f640 4 set_mon_vals no callback set 2024-01-24T23:26:36.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b2db0f640 10 monclient: shutdown 2024-01-24T23:26:36.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b2db0f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:36.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b2db0f640 1 -- 172.21.15.132:0/192839699 >> v1:172.21.15.115:6792/0 conn(0x7f4b2809be60 legacy=0x7f4b2813d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:36.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b2db0f640 1 -- 172.21.15.132:0/192839699 shutdown_connections 2024-01-24T23:26:36.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b2db0f640 1 -- 172.21.15.132:0/192839699 >> 172.21.15.132:0/192839699 conn(0x7f4b280a5660 msgr2=0x7f4b280a5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:36.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b2db0f640 1 -- 172.21.15.132:0/192839699 shutdown_connections 2024-01-24T23:26:36.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.279+0000 7f4b2db0f640 1 -- 172.21.15.132:0/192839699 wait complete. 2024-01-24T23:26:36.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b2db0f640 10 monclient: build_initial_monmap 2024-01-24T23:26:36.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b2db0f640 10 monclient: monmap: 2024-01-24T23:26:36.290 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:36.290 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:36.290 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:36.291 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:36.291 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:36.291 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:36.291 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:36.291 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:36.291 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:36.291 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:36.291 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:36.291 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:36.291 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:36.291 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:36.291 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:36.291 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:36.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b2db0f640 1 Processor -- start 2024-01-24T23:26:36.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b2db0f640 1 -- start start 2024-01-24T23:26:36.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b2db0f640 10 monclient: init 2024-01-24T23:26:36.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b2db0f640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:36.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b2db0f640 10 monclient: _add_conns ranks=[2,0,7,6,8,5,3,4,1] 2024-01-24T23:26:36.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b2db0f640 10 monclient(hunting): picked mon.noname-b con 0x7f4b2809b6a0 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:36.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b2db0f640 10 monclient(hunting): picked mon.noname-a con 0x7f4b2809ba80 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:36.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b2db0f640 10 monclient(hunting): picked mon.noname-i con 0x7f4b2809be60 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:36.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b2db0f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b2812c8c0 con 0x7f4b2809ba80 2024-01-24T23:26:36.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b2db0f640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b28098e40 con 0x7f4b2809b6a0 2024-01-24T23:26:36.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b2db0f640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b280727b0 con 0x7f4b2809be60 2024-01-24T23:26:36.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b2db0f640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:36.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b2db0f640 10 monclient(hunting): authenticate will time out at 1283.609461s 2024-01-24T23:26:36.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b27fff640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f4b2809be60 legacy=0x7f4b281d1f90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:36.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b277fe640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f4b2809b6a0 legacy=0x7f4b28098020 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:36.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b26ffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f4b2809ba80 0x7f4b28098730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:47044/0 (socket says 172.21.15.132:47044) 2024-01-24T23:26:36.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b26ffd640 1 -- 172.21.15.132:0/1661401088 learned_addr learned my addr 172.21.15.132:0/1661401088 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:36.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 1 -- 172.21.15.132:0/1661401088 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2380023424 0 0) 0x7f4b2812c8c0 con 0x7f4b2809ba80 2024-01-24T23:26:36.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:26:36.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:36.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:36.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 10 monclient(hunting): my global_id is 94689 2024-01-24T23:26:36.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 1 -- 172.21.15.132:0/1661401088 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b14003530 con 0x7f4b2809ba80 2024-01-24T23:26:36.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 1 -- 172.21.15.132:0/1661401088 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2778921851 0 0) 0x7f4b14003530 con 0x7f4b2809ba80 2024-01-24T23:26:36.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 1 -- 172.21.15.132:0/1661401088 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4b2812c8c0 con 0x7f4b2809ba80 2024-01-24T23:26:36.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 1 -- 172.21.15.132:0/1661401088 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4b1c001aa0 con 0x7f4b2809ba80 2024-01-24T23:26:36.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:36.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:26:36.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 10 monclient(hunting): dump: 2024-01-24T23:26:36.297 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:36.297 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:36.297 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:36.297 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:36.297 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:36.297 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:36.297 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:36.297 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:36.297 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:36.297 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:36.297 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:36.298 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:36.298 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:36.298 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:36.298 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:36.298 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:36.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:36.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 1 -- 172.21.15.132:0/1661401088 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 230981583 0 0) 0x7f4b2812c8c0 con 0x7f4b2809ba80 2024-01-24T23:26:36.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 1 -- 172.21.15.132:0/1661401088 >> v1:172.21.15.132:6792/0 conn(0x7f4b2809be60 legacy=0x7f4b281d1f90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:36.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 1 -- 172.21.15.132:0/1661401088 >> v1:172.21.15.115:6790/0 conn(0x7f4b2809b6a0 legacy=0x7f4b28098020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:36.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 10 monclient: _finish_hunting 0 2024-01-24T23:26:36.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 1 monclient: found mon.a 2024-01-24T23:26:36.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:36.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:26:36.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 1 -- 172.21.15.132:0/1661401088 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2812cba0 con 0x7f4b2809ba80 2024-01-24T23:26:36.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 10 monclient: _finish_auth 0 2024-01-24T23:26:36.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 10 monclient: _check_auth_tickets 2024-01-24T23:26:36.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:36.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b2db0f640 5 monclient: authenticate success, global_id 94689 2024-01-24T23:26:36.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b2db0f640 10 monclient: _renew_subs 2024-01-24T23:26:36.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b2db0f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:26:36.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b2db0f640 1 -- 172.21.15.132:0/1661401088 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b280a18a0 con 0x7f4b2809ba80 2024-01-24T23:26:36.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b2db0f640 10 monclient: _renew_subs 2024-01-24T23:26:36.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b2db0f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:26:36.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b2db0f640 1 -- 172.21.15.132:0/1661401088 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b280729f0 con 0x7f4b2809ba80 2024-01-24T23:26:36.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 1 -- 172.21.15.132:0/1661401088 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4b1c001d50 con 0x7f4b2809ba80 2024-01-24T23:26:36.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:36.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b257fa640 4 set_mon_vals no callback set 2024-01-24T23:26:36.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 1 -- 172.21.15.132:0/1661401088 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4b1c004e60 con 0x7f4b2809ba80 2024-01-24T23:26:36.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:36.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:26:36.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b0ffff640 10 monclient: dump: 2024-01-24T23:26:36.300 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:36.300 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:36.300 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:36.300 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:36.300 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:36.301 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:36.301 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:36.301 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:36.301 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:36.301 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:36.301 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:36.301 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:36.301 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:36.301 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:36.301 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:36.301 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:36.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.283+0000 7f4b2db0f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:36.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.287+0000 7f4b2db0f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:36.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.287+0000 7f4b2db0f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:26:36.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.287+0000 7f4b2db0f640 1 -- 172.21.15.132:0/1661401088 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b08005090 con 0x7f4b2809ba80 2024-01-24T23:26:36.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.287+0000 7f4b0ffff640 1 -- 172.21.15.132:0/1661401088 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f4b1c005040 con 0x7f4b2809ba80 2024-01-24T23:26:36.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.287+0000 7f4b0ffff640 1 -- 172.21.15.132:0/1661401088 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(178..178 src has 1..178) v4 ==== 6995+0+0 (unknown 1937531165 0 0) 0x7f4b1c044550 con 0x7f4b2809ba80 2024-01-24T23:26:36.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.287+0000 7f4b277fe640 10 monclient: get_auth_request con 0x7f4b14036ed0 auth_method 0 2024-01-24T23:26:36.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.299+0000 7f4b0ffff640 1 -- 172.21.15.132:0/1661401088 <== mon.0 v1:172.21.15.115:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f4b1c0148a0 con 0x7f4b2809ba80 2024-01-24T23:26:36.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.299+0000 7f4b0ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:36.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.299+0000 7f4b0ffff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:36.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.455+0000 7f4b2db0f640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}] 2024-01-24T23:26:36.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.455+0000 7f4b2db0f640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}] 2024-01-24T23:26:36.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.455+0000 7f4b2db0f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:26:36.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.455+0000 7f4b2db0f640 1 -- 172.21.15.132:0/1661401088 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]} v 0) v1 -- 0x7f4b08005340 con 0x7f4b2809ba80 2024-01-24T23:26:36.914 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.911+0000 7f4b0ffff640 1 -- 172.21.15.132:0/1661401088 <== mon.0 v1:172.21.15.115:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}]=0 added bucket r1 type rack to location {root=default} v179) v1 ==== 176+0+0 (unknown 1711268037 0 0) 0x7f4b1c0146c0 con 0x7f4b2809ba80 2024-01-24T23:26:36.914 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.911+0000 7f4b0ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}] 2024-01-24T23:26:36.914 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:36.911+0000 7f4b0ffff640 10 monclient: _finish_command 2 = system:0 added bucket r1 type rack to location {root=default} 2024-01-24T23:26:37.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.011+0000 7f4b2db0f640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}] 2024-01-24T23:26:37.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.011+0000 7f4b2db0f640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}] 2024-01-24T23:26:37.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.011+0000 7f4b2db0f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:26:37.014 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.011+0000 7f4b2db0f640 1 -- 172.21.15.132:0/1661401088 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]} v 0) v1 -- 0x7f4b08005a80 con 0x7f4b2809ba80 2024-01-24T23:26:37.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.015+0000 7f4b0ffff640 1 -- 172.21.15.132:0/1661401088 <== mon.0 v1:172.21.15.115:6789/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}]=0 bucket 'r1' already exists v179) v1 ==== 150+0+0 (unknown 3306357860 0 0) 0x7f4b1c0146c0 con 0x7f4b2809ba80 2024-01-24T23:26:37.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.015+0000 7f4b0ffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}] 2024-01-24T23:26:37.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.015+0000 7f4b0ffff640 10 monclient: _finish_command 3 = system:0 bucket 'r1' already exists 2024-01-24T23:26:37.016 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'r1' already exists 2024-01-24T23:26:37.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.015+0000 7f4b2db0f640 1 -- 172.21.15.132:0/1661401088 >> v1:172.21.15.132:6812/1835401922 conn(0x7f4b14036ed0 legacy=0x7f4b14039390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:37.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.015+0000 7f4b2db0f640 10 monclient: shutdown 2024-01-24T23:26:37.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.015+0000 7f4b2db0f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:37.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.015+0000 7f4b2db0f640 1 -- 172.21.15.132:0/1661401088 >> v1:172.21.15.115:6789/0 conn(0x7f4b2809ba80 legacy=0x7f4b28098730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:37.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.015+0000 7f4b2db0f640 1 -- 172.21.15.132:0/1661401088 shutdown_connections 2024-01-24T23:26:37.018 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.015+0000 7f4b2db0f640 1 -- 172.21.15.132:0/1661401088 >> 172.21.15.132:0/1661401088 conn(0x7f4b280a5660 msgr2=0x7f4b281bc970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:37.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.015+0000 7f4b2db0f640 1 -- 172.21.15.132:0/1661401088 shutdown_connections 2024-01-24T23:26:37.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.015+0000 7f4b2db0f640 1 -- 172.21.15.132:0/1661401088 wait complete. 2024-01-24T23:26:37.034 INFO:tasks.workunit.client.0.smithi132.stderr:++ seq 1 32 2024-01-24T23:26:37.120 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:26:37.120 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h1 host rack=r1 2024-01-24T23:26:37.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.239+0000 7ff622983640 10 monclient: get_monmap_and_config 2024-01-24T23:26:37.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.239+0000 7ff622983640 10 monclient: build_initial_monmap 2024-01-24T23:26:37.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.239+0000 7ff622983640 10 monclient: monmap: 2024-01-24T23:26:37.243 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:37.243 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:37.243 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:37.244431+0000 2024-01-24T23:26:37.243 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:37.244431+0000 2024-01-24T23:26:37.243 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:37.243 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:37.244 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:37.244 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:37.244 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:37.244 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:37.244 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:37.244 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:37.244 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:37.244 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:37.244 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:37.244 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:37.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff622983640 1 Processor -- start 2024-01-24T23:26:37.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff622983640 1 -- start start 2024-01-24T23:26:37.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff622983640 10 monclient: init 2024-01-24T23:26:37.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff622983640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:37.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff622983640 10 monclient: _add_conns ranks=[6,7,0,8,4,5,2,1,3] 2024-01-24T23:26:37.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff622983640 10 monclient(hunting): picked mon.noname-d con 0x7ff61c133ed0 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:37.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff622983640 10 monclient(hunting): picked mon.noname-i con 0x7ff61c1342b0 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:37.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff622983640 10 monclient(hunting): picked mon.noname-a con 0x7ff61c134690 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:37.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff622983640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61c064700 con 0x7ff61c134690 2024-01-24T23:26:37.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff622983640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61c09b3d0 con 0x7ff61c133ed0 2024-01-24T23:26:37.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff622983640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61c0a58c0 con 0x7ff61c1342b0 2024-01-24T23:26:37.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff622983640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:37.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff622983640 10 monclient(hunting): authenticate will time out at 1284.569342s 2024-01-24T23:26:37.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61b7fe640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7ff61c1342b0 legacy=0x7ff61c13c040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:37.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff620f20640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff61c134690 0x7ff61c13f770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:47058/0 (socket says 172.21.15.132:47058) 2024-01-24T23:26:37.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61bfff640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7ff61c133ed0 0x7ff61c09bdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:36882/0 (socket says 172.21.15.132:36882) 2024-01-24T23:26:37.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff620f20640 1 -- 172.21.15.132:0/1720611034 learned_addr learned my addr 172.21.15.132:0/1720611034 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:37.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 1 -- 172.21.15.132:0/1720611034 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2522137183 0 0) 0x7ff61c09b3d0 con 0x7ff61c133ed0 2024-01-24T23:26:37.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:37.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:37.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:37.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 10 monclient(hunting): my global_id is 6432 2024-01-24T23:26:37.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 1 -- 172.21.15.132:0/1720611034 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff604003530 con 0x7ff61c133ed0 2024-01-24T23:26:37.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 1 -- 172.21.15.132:0/1720611034 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 124015725 0 0) 0x7ff61c064700 con 0x7ff61c134690 2024-01-24T23:26:37.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:37.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:37.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:37.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 10 monclient(hunting): my global_id is 94698 2024-01-24T23:26:37.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 1 -- 172.21.15.132:0/1720611034 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff61c09b3d0 con 0x7ff61c134690 2024-01-24T23:26:37.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 1 -- 172.21.15.132:0/1720611034 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2056505847 0 0) 0x7ff604003530 con 0x7ff61c133ed0 2024-01-24T23:26:37.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 1 -- 172.21.15.132:0/1720611034 >> v1:172.21.15.132:6792/0 conn(0x7ff61c1342b0 legacy=0x7ff61c13c040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:37.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 1 -- 172.21.15.132:0/1720611034 >> v1:172.21.15.115:6789/0 conn(0x7ff61c134690 legacy=0x7ff61c13f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:37.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 10 monclient: _finish_hunting 0 2024-01-24T23:26:37.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 1 monclient: found mon.noname-d 2024-01-24T23:26:37.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:37.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:26:37.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 1 -- 172.21.15.132:0/1720611034 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c09b0f0 con 0x7ff61c133ed0 2024-01-24T23:26:37.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 10 monclient: _finish_auth 0 2024-01-24T23:26:37.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 10 monclient: _check_auth_tickets 2024-01-24T23:26:37.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:37.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff622983640 5 monclient: authenticate success, global_id 6432 2024-01-24T23:26:37.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 1 -- 172.21.15.132:0/1720611034 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff60c0016f0 con 0x7ff61c133ed0 2024-01-24T23:26:37.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff622983640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:37.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:37.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:26:37.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 10 monclient: dump: 2024-01-24T23:26:37.251 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:37.251 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:37.251 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:37.251 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:37.251 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:37.251 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:37.251 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:37.251 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:37.252 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:37.252 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:37.252 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:37.252 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:37.252 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:37.252 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:37.252 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:37.252 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:37.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff622983640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:37.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 1 -- 172.21.15.132:0/1720611034 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff60c001910 con 0x7ff61c133ed0 2024-01-24T23:26:37.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:37.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff622983640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:37.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff622983640 4 set_mon_vals no callback set 2024-01-24T23:26:37.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 1 -- 172.21.15.132:0/1720611034 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff60c0047f0 con 0x7ff61c133ed0 2024-01-24T23:26:37.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:37.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:26:37.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff61affd640 10 monclient: dump: 2024-01-24T23:26:37.253 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:37.253 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:37.253 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:37.253 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:37.253 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:37.253 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:37.253 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:37.253 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:37.253 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:37.253 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:37.253 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:37.253 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:37.254 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:37.254 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:37.254 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:37.254 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:37.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.243+0000 7ff622983640 10 monclient: shutdown 2024-01-24T23:26:37.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:37.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 1 -- 172.21.15.132:0/1720611034 >> v1:172.21.15.115:6792/0 conn(0x7ff61c133ed0 legacy=0x7ff61c09bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:37.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 1 -- 172.21.15.132:0/1720611034 shutdown_connections 2024-01-24T23:26:37.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 1 -- 172.21.15.132:0/1720611034 >> 172.21.15.132:0/1720611034 conn(0x7ff61c0a5ef0 msgr2=0x7ff61c0a6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:37.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 1 -- 172.21.15.132:0/1720611034 shutdown_connections 2024-01-24T23:26:37.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 1 -- 172.21.15.132:0/1720611034 wait complete. 2024-01-24T23:26:37.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 10 monclient: build_initial_monmap 2024-01-24T23:26:37.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 10 monclient: monmap: 2024-01-24T23:26:37.254 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:37.254 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:37.255 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:37.255 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:37.255 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:37.255 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:37.255 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:37.255 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:37.255 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:37.255 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:37.255 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:37.255 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:37.255 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:37.255 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:37.255 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:37.255 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:37.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 1 Processor -- start 2024-01-24T23:26:37.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 1 -- start start 2024-01-24T23:26:37.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 10 monclient: init 2024-01-24T23:26:37.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:37.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 10 monclient: _add_conns ranks=[0,7,1,5,8,6,2,4,3] 2024-01-24T23:26:37.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 10 monclient(hunting): picked mon.noname-a con 0x7ff61c133ed0 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:37.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 10 monclient(hunting): picked mon.noname-i con 0x7ff61c1342b0 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:37.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 10 monclient(hunting): picked mon.noname-f con 0x7ff61c134690 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:37.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61c1239f0 con 0x7ff61c133ed0 2024-01-24T23:26:37.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61c0a5af0 con 0x7ff61c134690 2024-01-24T23:26:37.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61c09ca90 con 0x7ff61c1342b0 2024-01-24T23:26:37.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:37.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 10 monclient(hunting): authenticate will time out at 1284.573914s 2024-01-24T23:26:37.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff620f20640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7ff61c134690 legacy=0x7ff61c1d3b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:37.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff61b7fe640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7ff61c1342b0 legacy=0x7ff61c1d0440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:37.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff61bfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff61c133ed0 0x7ff61c133000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:47060/0 (socket says 172.21.15.132:47060) 2024-01-24T23:26:37.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff61bfff640 1 -- 172.21.15.132:0/3019878967 learned_addr learned my addr 172.21.15.132:0/3019878967 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:37.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 1 -- 172.21.15.132:0/3019878967 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 732556615 0 0) 0x7ff61c1239f0 con 0x7ff61c133ed0 2024-01-24T23:26:37.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:26:37.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:37.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:37.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 10 monclient(hunting): my global_id is 94707 2024-01-24T23:26:37.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 1 -- 172.21.15.132:0/3019878967 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff608003530 con 0x7ff61c133ed0 2024-01-24T23:26:37.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 1 -- 172.21.15.132:0/3019878967 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 926716616 0 0) 0x7ff608003530 con 0x7ff61c133ed0 2024-01-24T23:26:37.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 1 -- 172.21.15.132:0/3019878967 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff61c1239f0 con 0x7ff61c133ed0 2024-01-24T23:26:37.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 1 -- 172.21.15.132:0/3019878967 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff60c001650 con 0x7ff61c133ed0 2024-01-24T23:26:37.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:37.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:26:37.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 10 monclient(hunting): dump: 2024-01-24T23:26:37.261 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:37.261 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:37.261 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:37.261 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:37.261 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:37.261 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:37.262 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:37.262 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:37.262 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:37.262 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:37.262 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:37.262 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:37.262 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:37.262 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:37.262 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:37.262 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:37.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:37.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 1 -- 172.21.15.132:0/3019878967 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1103564461 0 0) 0x7ff61c1239f0 con 0x7ff61c133ed0 2024-01-24T23:26:37.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 1 -- 172.21.15.132:0/3019878967 >> v1:172.21.15.132:6792/0 conn(0x7ff61c1342b0 legacy=0x7ff61c1d0440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:37.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 1 -- 172.21.15.132:0/3019878967 >> v1:172.21.15.132:6789/0 conn(0x7ff61c134690 legacy=0x7ff61c1d3b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:37.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:26:37.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 1 monclient: found mon.a 2024-01-24T23:26:37.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:37.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:26:37.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 1 -- 172.21.15.132:0/3019878967 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c1c87d0 con 0x7ff61c133ed0 2024-01-24T23:26:37.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 10 monclient: _finish_auth 0 2024-01-24T23:26:37.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:26:37.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:37.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 5 monclient: authenticate success, global_id 94707 2024-01-24T23:26:37.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 10 monclient: _renew_subs 2024-01-24T23:26:37.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:26:37.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 1 -- 172.21.15.132:0/3019878967 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61c121700 con 0x7ff61c133ed0 2024-01-24T23:26:37.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 10 monclient: _renew_subs 2024-01-24T23:26:37.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:26:37.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff622983640 1 -- 172.21.15.132:0/3019878967 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61c123710 con 0x7ff61c133ed0 2024-01-24T23:26:37.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 1 -- 172.21.15.132:0/3019878967 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff60c004d70 con 0x7ff61c133ed0 2024-01-24T23:26:37.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:37.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 1 -- 172.21.15.132:0/3019878967 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff60c005220 con 0x7ff61c133ed0 2024-01-24T23:26:37.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:37.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff61a7fc640 4 set_mon_vals no callback set 2024-01-24T23:26:37.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:26:37.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.247+0000 7ff618ff9640 10 monclient: dump: 2024-01-24T23:26:37.264 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:37.264 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:37.264 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:37.264 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:37.264 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:37.264 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:37.264 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:37.264 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:37.265 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:37.265 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:37.265 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:37.265 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:37.265 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:37.265 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:37.265 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:37.265 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:37.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.251+0000 7ff622983640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:37.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.251+0000 7ff622983640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:37.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.251+0000 7ff622983640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:26:37.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.251+0000 7ff622983640 1 -- 172.21.15.132:0/3019878967 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5fc005090 con 0x7ff61c133ed0 2024-01-24T23:26:37.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.251+0000 7ff618ff9640 1 -- 172.21.15.132:0/3019878967 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7ff60c00fca0 con 0x7ff61c133ed0 2024-01-24T23:26:37.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.251+0000 7ff618ff9640 1 -- 172.21.15.132:0/3019878967 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(179..179 src has 1..179) v4 ==== 7205+0+0 (unknown 401715343 0 0) 0x7ff60c0447f0 con 0x7ff61c133ed0 2024-01-24T23:26:37.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.251+0000 7ff61b7fe640 10 monclient: get_auth_request con 0x7ff608036f80 auth_method 0 2024-01-24T23:26:37.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.263+0000 7ff618ff9640 1 -- 172.21.15.132:0/3019878967 <== mon.0 v1:172.21.15.115:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7ff60c0111a0 con 0x7ff61c133ed0 2024-01-24T23:26:37.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.263+0000 7ff618ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:37.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.263+0000 7ff618ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:37.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.423+0000 7ff622983640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:37.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.423+0000 7ff622983640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:37.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.423+0000 7ff622983640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:26:37.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:37.423+0000 7ff622983640 1 -- 172.21.15.132:0/3019878967 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7ff5fc0058c0 con 0x7ff61c133ed0 2024-01-24T23:26:38.064 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.063+0000 7ff618ff9640 1 -- 172.21.15.132:0/3019878967 <== mon.0 v1:172.21.15.115:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}]=0 added bucket h1 type host to location {rack=r1} v180) v1 ==== 166+0+0 (unknown 164631995 0 0) 0x7ff60c013280 con 0x7ff61c133ed0 2024-01-24T23:26:38.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.063+0000 7ff618ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:38.065 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.063+0000 7ff618ff9640 10 monclient: _finish_command 2 = system:0 added bucket h1 type host to location {rack=r1} 2024-01-24T23:26:38.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.163+0000 7ff622983640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:38.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.163+0000 7ff622983640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:38.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.163+0000 7ff622983640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:26:38.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.163+0000 7ff622983640 1 -- 172.21.15.132:0/3019878967 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7ff5fc002220 con 0x7ff61c133ed0 2024-01-24T23:26:38.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.163+0000 7ff618ff9640 1 -- 172.21.15.132:0/3019878967 <== mon.0 v1:172.21.15.115:6789/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h1' already exists v180) v1 ==== 145+0+0 (unknown 3893609659 0 0) 0x7ff60c014a00 con 0x7ff61c133ed0 2024-01-24T23:26:38.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.163+0000 7ff618ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:38.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.163+0000 7ff618ff9640 10 monclient: _finish_command 3 = system:0 bucket 'h1' already exists 2024-01-24T23:26:38.167 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h1' already exists 2024-01-24T23:26:38.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.167+0000 7ff622983640 1 -- 172.21.15.132:0/3019878967 >> v1:172.21.15.132:6812/1835401922 conn(0x7ff608036f80 legacy=0x7ff608039440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:38.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.167+0000 7ff622983640 10 monclient: shutdown 2024-01-24T23:26:38.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.167+0000 7ff622983640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:38.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.167+0000 7ff622983640 1 -- 172.21.15.132:0/3019878967 >> v1:172.21.15.115:6789/0 conn(0x7ff61c133ed0 legacy=0x7ff61c133000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:38.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.167+0000 7ff622983640 1 -- 172.21.15.132:0/3019878967 shutdown_connections 2024-01-24T23:26:38.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.167+0000 7ff622983640 1 -- 172.21.15.132:0/3019878967 >> 172.21.15.132:0/3019878967 conn(0x7ff61c0a5ef0 msgr2=0x7ff61c0a72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:38.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.167+0000 7ff622983640 1 -- 172.21.15.132:0/3019878967 shutdown_connections 2024-01-24T23:26:38.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.167+0000 7ff622983640 1 -- 172.21.15.132:0/3019878967 wait complete. 2024-01-24T23:26:38.184 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:26:38.184 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h2 host rack=r1 2024-01-24T23:26:38.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa08f09d640 10 monclient: get_monmap_and_config 2024-01-24T23:26:38.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa08f09d640 10 monclient: build_initial_monmap 2024-01-24T23:26:38.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa08f09d640 10 monclient: monmap: 2024-01-24T23:26:38.305 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:38.305 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:38.305 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:38.306375+0000 2024-01-24T23:26:38.305 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:38.306375+0000 2024-01-24T23:26:38.305 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:38.305 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:38.306 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:38.306 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:38.306 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:38.306 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:38.306 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:38.306 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:38.306 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:38.306 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:38.306 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:38.306 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:38.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa08f09d640 1 Processor -- start 2024-01-24T23:26:38.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa08f09d640 1 -- start start 2024-01-24T23:26:38.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa08f09d640 10 monclient: init 2024-01-24T23:26:38.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa08f09d640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:38.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa08f09d640 10 monclient: _add_conns ranks=[7,2,6,8,3,0,4,5,1] 2024-01-24T23:26:38.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa08f09d640 10 monclient(hunting): picked mon.noname-i con 0x7fa08809b810 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:38.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa08f09d640 10 monclient(hunting): picked mon.noname-b con 0x7fa08809bbf0 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:38.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa08f09d640 10 monclient(hunting): picked mon.noname-d con 0x7fa0881370a0 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:38.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa08f09d640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa088064700 con 0x7fa08809bbf0 2024-01-24T23:26:38.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa08f09d640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa088099af0 con 0x7fa0881370a0 2024-01-24T23:26:38.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa08f09d640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa088072110 con 0x7fa08809b810 2024-01-24T23:26:38.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa08f09d640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:38.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa08f09d640 10 monclient(hunting): authenticate will time out at 1285.630951s 2024-01-24T23:26:38.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa08ce39640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fa08809b810 legacy=0x7fa0880712f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:38.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa087fff640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fa08809bbf0 legacy=0x7fa088071a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:38.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa08d63a640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fa0881370a0 0x7fa08813ff30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:36890/0 (socket says 172.21.15.132:36890) 2024-01-24T23:26:38.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa08d63a640 1 -- 172.21.15.132:0/3101183911 learned_addr learned my addr 172.21.15.132:0/3101183911 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:38.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa0877fe640 1 -- 172.21.15.132:0/3101183911 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2577506808 0 0) 0x7fa088099af0 con 0x7fa0881370a0 2024-01-24T23:26:38.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa0877fe640 10 monclient(hunting): _negotiate 2024-01-24T23:26:38.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa0877fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:38.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa0877fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:38.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa0877fe640 10 monclient(hunting): my global_id is 6441 2024-01-24T23:26:38.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.303+0000 7fa0877fe640 1 -- 172.21.15.132:0/3101183911 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa078003530 con 0x7fa0881370a0 2024-01-24T23:26:38.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0877fe640 1 -- 172.21.15.132:0/3101183911 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 791579911 0 0) 0x7fa078003530 con 0x7fa0881370a0 2024-01-24T23:26:38.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0877fe640 1 -- 172.21.15.132:0/3101183911 >> v1:172.21.15.132:6792/0 conn(0x7fa08809b810 legacy=0x7fa0880712f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:38.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0877fe640 1 -- 172.21.15.132:0/3101183911 >> v1:172.21.15.115:6790/0 conn(0x7fa08809bbf0 legacy=0x7fa088071a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:38.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0877fe640 10 monclient: _finish_hunting 0 2024-01-24T23:26:38.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0877fe640 1 monclient: found mon.noname-d 2024-01-24T23:26:38.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0877fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:38.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0877fe640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:26:38.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0877fe640 1 -- 172.21.15.132:0/3101183911 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08809c050 con 0x7fa0881370a0 2024-01-24T23:26:38.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0877fe640 10 monclient: _finish_auth 0 2024-01-24T23:26:38.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0877fe640 10 monclient: _check_auth_tickets 2024-01-24T23:26:38.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0877fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:38.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0877fe640 1 -- 172.21.15.132:0/3101183911 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa080001b40 con 0x7fa0881370a0 2024-01-24T23:26:38.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0877fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:38.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0877fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:26:38.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0877fe640 10 monclient: dump: 2024-01-24T23:26:38.316 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:38.316 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:38.316 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:38.316 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:38.316 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:38.316 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:38.316 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:38.317 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:38.317 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:38.317 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:38.317 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:38.317 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:38.317 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:38.317 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:38.317 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:38.317 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:38.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 5 monclient: authenticate success, global_id 6441 2024-01-24T23:26:38.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:38.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0877fe640 1 -- 172.21.15.132:0/3101183911 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa080001df0 con 0x7fa0881370a0 2024-01-24T23:26:38.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0877fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:38.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:38.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 4 set_mon_vals no callback set 2024-01-24T23:26:38.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0877fe640 1 -- 172.21.15.132:0/3101183911 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa080004f70 con 0x7fa0881370a0 2024-01-24T23:26:38.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0877fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:38.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0877fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:26:38.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0877fe640 10 monclient: dump: 2024-01-24T23:26:38.318 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:38.318 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:38.318 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:38.318 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:38.318 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:38.319 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:38.319 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:38.319 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:38.319 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:38.319 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:38.319 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:38.319 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:38.319 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:38.319 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:38.319 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:38.319 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:38.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 10 monclient: shutdown 2024-01-24T23:26:38.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:38.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 1 -- 172.21.15.132:0/3101183911 >> v1:172.21.15.115:6792/0 conn(0x7fa0881370a0 legacy=0x7fa08813ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:38.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 1 -- 172.21.15.132:0/3101183911 shutdown_connections 2024-01-24T23:26:38.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 1 -- 172.21.15.132:0/3101183911 >> 172.21.15.132:0/3101183911 conn(0x7fa0880a1bc0 msgr2=0x7fa0880a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:38.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 1 -- 172.21.15.132:0/3101183911 shutdown_connections 2024-01-24T23:26:38.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 1 -- 172.21.15.132:0/3101183911 wait complete. 2024-01-24T23:26:38.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 10 monclient: build_initial_monmap 2024-01-24T23:26:38.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 10 monclient: monmap: 2024-01-24T23:26:38.320 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:38.320 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:38.320 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:38.320 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:38.320 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:38.321 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:38.321 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:38.321 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:38.321 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:38.321 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:38.321 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:38.321 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:38.321 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:38.321 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:38.321 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:38.321 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:38.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 1 Processor -- start 2024-01-24T23:26:38.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 1 -- start start 2024-01-24T23:26:38.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 10 monclient: init 2024-01-24T23:26:38.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:38.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 10 monclient: _add_conns ranks=[6,3,5,4,8,0,7,2,1] 2024-01-24T23:26:38.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 10 monclient(hunting): picked mon.noname-d con 0x7fa08809b810 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:38.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 10 monclient(hunting): picked mon.noname-g con 0x7fa08809bbf0 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:38.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 10 monclient(hunting): picked mon.noname-h con 0x7fa0881370a0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:38.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa08809e870 con 0x7fa08809bbf0 2024-01-24T23:26:38.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0880a1620 con 0x7fa0881370a0 2024-01-24T23:26:38.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa08809b5e0 con 0x7fa08809b810 2024-01-24T23:26:38.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:38.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 10 monclient(hunting): authenticate will time out at 1285.634569s 2024-01-24T23:26:38.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08d63a640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fa0881370a0 0x7fa0881d3d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:54856/0 (socket says 172.21.15.132:54856) 2024-01-24T23:26:38.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08d63a640 1 -- 172.21.15.132:0/4129886831 learned_addr learned my addr 172.21.15.132:0/4129886831 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:38.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa087fff640 1 --1- 172.21.15.132:0/4129886831 >> v1:172.21.15.132:6790/0 conn(0x7fa08809bbf0 0x7fa0881d0610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:48694/0 (socket says 172.21.15.132:48694) 2024-01-24T23:26:38.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 1 -- 172.21.15.132:0/4129886831 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1056418880 0 0) 0x7fa0880a1620 con 0x7fa0881370a0 2024-01-24T23:26:38.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient(hunting): _negotiate 2024-01-24T23:26:38.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:38.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:38.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient(hunting): my global_id is 6224 2024-01-24T23:26:38.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 1 -- 172.21.15.132:0/4129886831 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa06c003530 con 0x7fa0881370a0 2024-01-24T23:26:38.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 1 -- 172.21.15.132:0/4129886831 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1210573891 0 0) 0x7fa08809e870 con 0x7fa08809bbf0 2024-01-24T23:26:38.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient(hunting): _negotiate 2024-01-24T23:26:38.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:38.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:38.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient(hunting): my global_id is 6348 2024-01-24T23:26:38.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 1 -- 172.21.15.132:0/4129886831 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0880a1620 con 0x7fa08809bbf0 2024-01-24T23:26:38.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 1 -- 172.21.15.132:0/4129886831 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1812376024 0 0) 0x7fa0880a1620 con 0x7fa08809bbf0 2024-01-24T23:26:38.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 1 -- 172.21.15.132:0/4129886831 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa08809e870 con 0x7fa08809bbf0 2024-01-24T23:26:38.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 1 -- 172.21.15.132:0/4129886831 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1089726320 0 0) 0x7fa06c003530 con 0x7fa0881370a0 2024-01-24T23:26:38.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 1 -- 172.21.15.132:0/4129886831 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa0880a1620 con 0x7fa0881370a0 2024-01-24T23:26:38.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 1 -- 172.21.15.132:0/4129886831 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa0740042a0 con 0x7fa08809bbf0 2024-01-24T23:26:38.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:38.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:38.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient(hunting): dump: 2024-01-24T23:26:38.324 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:38.324 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:38.324 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:38.324 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:38.324 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:38.324 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:38.325 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:38.325 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:38.325 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:38.325 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:38.325 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:38.325 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:38.325 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:38.325 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:38.325 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:38.325 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:38.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:38.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 1 -- 172.21.15.132:0/4129886831 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa080001b00 con 0x7fa0881370a0 2024-01-24T23:26:38.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:38.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:38.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient(hunting): dump: 2024-01-24T23:26:38.326 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:38.326 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:38.326 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:38.326 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:38.326 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:38.326 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:38.326 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:38.326 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:38.326 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:38.326 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:38.326 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:38.326 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:38.326 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:38.326 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:38.326 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:38.327 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:38.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 1 -- 172.21.15.132:0/4129886831 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3591857202 0 0) 0x7fa08809b5e0 con 0x7fa08809b810 2024-01-24T23:26:38.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient(hunting): _negotiate 2024-01-24T23:26:38.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:38.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:38.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient(hunting): my global_id is 6450 2024-01-24T23:26:38.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 1 -- 172.21.15.132:0/4129886831 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa06c003530 con 0x7fa08809b810 2024-01-24T23:26:38.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 1 -- 172.21.15.132:0/4129886831 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3766857007 0 0) 0x7fa08809e870 con 0x7fa08809bbf0 2024-01-24T23:26:38.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 1 -- 172.21.15.132:0/4129886831 >> v1:172.21.15.115:6792/0 conn(0x7fa08809b810 legacy=0x7fa0881306f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:38.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 1 -- 172.21.15.132:0/4129886831 >> v1:172.21.15.132:6791/0 conn(0x7fa0881370a0 legacy=0x7fa0881d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:38.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient: _finish_hunting 0 2024-01-24T23:26:38.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 1 monclient: found mon.g 2024-01-24T23:26:38.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:38.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:38.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 1 -- 172.21.15.132:0/4129886831 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0881c87f0 con 0x7fa08809bbf0 2024-01-24T23:26:38.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient: _finish_auth 0 2024-01-24T23:26:38.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient: _check_auth_tickets 2024-01-24T23:26:38.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:38.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 5 monclient: authenticate success, global_id 6348 2024-01-24T23:26:38.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 10 monclient: _renew_subs 2024-01-24T23:26:38.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:38.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 1 -- 172.21.15.132:0/4129886831 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08812aea0 con 0x7fa08809bbf0 2024-01-24T23:26:38.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 10 monclient: _renew_subs 2024-01-24T23:26:38.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:38.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa08f09d640 1 -- 172.21.15.132:0/4129886831 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa088130fd0 con 0x7fa08809bbf0 2024-01-24T23:26:38.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 1 -- 172.21.15.132:0/4129886831 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa0740044c0 con 0x7fa08809bbf0 2024-01-24T23:26:38.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:38.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 1 -- 172.21.15.132:0/4129886831 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa0740049f0 con 0x7fa08809bbf0 2024-01-24T23:26:38.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:38.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa086ffd640 4 set_mon_vals no callback set 2024-01-24T23:26:38.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:38.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.307+0000 7fa0857fa640 10 monclient: dump: 2024-01-24T23:26:38.329 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:38.329 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:38.329 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:38.329 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:38.329 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:38.329 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:38.329 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:38.329 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:38.329 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:38.329 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:38.329 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:38.329 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:38.329 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:38.330 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:38.330 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:38.330 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:38.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.311+0000 7fa08f09d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:38.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.311+0000 7fa08f09d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:38.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.311+0000 7fa08f09d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:38.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.311+0000 7fa08f09d640 1 -- 172.21.15.132:0/4129886831 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa068005090 con 0x7fa08809bbf0 2024-01-24T23:26:38.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.311+0000 7fa0857fa640 1 -- 172.21.15.132:0/4129886831 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fa074004c10 con 0x7fa08809bbf0 2024-01-24T23:26:38.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.311+0000 7fa0857fa640 1 -- 172.21.15.132:0/4129886831 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(180..180 src has 1..180) v4 ==== 7415+0+0 (unknown 2788788406 0 0) 0x7fa0740445d0 con 0x7fa08809bbf0 2024-01-24T23:26:38.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.311+0000 7fa08ce39640 10 monclient: get_auth_request con 0x7fa06c0377f0 auth_method 0 2024-01-24T23:26:38.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.315+0000 7fa0857fa640 1 -- 172.21.15.132:0/4129886831 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fa074014a40 con 0x7fa08809bbf0 2024-01-24T23:26:38.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.315+0000 7fa0857fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:38.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.315+0000 7fa0857fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:38.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.471+0000 7fa08f09d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:38.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.471+0000 7fa08f09d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:38.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.471+0000 7fa08f09d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:38.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:38.471+0000 7fa08f09d640 1 -- 172.21.15.132:0/4129886831 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fa0680058c0 con 0x7fa08809bbf0 2024-01-24T23:26:39.075 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.071+0000 7fa0857fa640 1 -- 172.21.15.132:0/4129886831 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}]=0 added bucket h2 type host to location {rack=r1} v181) v1 ==== 166+0+0 (unknown 771195085 0 0) 0x7fa074012eb0 con 0x7fa08809bbf0 2024-01-24T23:26:39.075 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.071+0000 7fa0857fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:39.075 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.071+0000 7fa0857fa640 10 monclient: _finish_command 2 = system:0 added bucket h2 type host to location {rack=r1} 2024-01-24T23:26:39.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.175+0000 7fa08f09d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:39.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.175+0000 7fa08f09d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:39.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.175+0000 7fa08f09d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:39.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.175+0000 7fa08f09d640 1 -- 172.21.15.132:0/4129886831 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fa068002220 con 0x7fa08809bbf0 2024-01-24T23:26:39.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.175+0000 7fa0857fa640 1 -- 172.21.15.132:0/4129886831 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h2' already exists v181) v1 ==== 145+0+0 (unknown 3115178263 0 0) 0x7fa074014630 con 0x7fa08809bbf0 2024-01-24T23:26:39.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.175+0000 7fa0857fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:39.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.175+0000 7fa0857fa640 10 monclient: _finish_command 3 = system:0 bucket 'h2' already exists 2024-01-24T23:26:39.226 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h2' already exists 2024-01-24T23:26:39.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.175+0000 7fa08f09d640 1 -- 172.21.15.132:0/4129886831 >> v1:172.21.15.132:6812/1835401922 conn(0x7fa06c0377f0 legacy=0x7fa06c039cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:39.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.175+0000 7fa08f09d640 10 monclient: shutdown 2024-01-24T23:26:39.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.175+0000 7fa08f09d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:39.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.175+0000 7fa08f09d640 1 -- 172.21.15.132:0/4129886831 >> v1:172.21.15.132:6790/0 conn(0x7fa08809bbf0 legacy=0x7fa0881d0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:39.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.175+0000 7fa08f09d640 1 -- 172.21.15.132:0/4129886831 shutdown_connections 2024-01-24T23:26:39.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.175+0000 7fa08f09d640 1 -- 172.21.15.132:0/4129886831 >> 172.21.15.132:0/4129886831 conn(0x7fa0880a1bc0 msgr2=0x7fa0880a3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:39.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.175+0000 7fa08f09d640 1 -- 172.21.15.132:0/4129886831 shutdown_connections 2024-01-24T23:26:39.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.175+0000 7fa08f09d640 1 -- 172.21.15.132:0/4129886831 wait complete. 2024-01-24T23:26:39.226 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:26:39.226 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h3 host rack=r1 2024-01-24T23:26:39.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.311+0000 7f33b0ab6640 10 monclient: get_monmap_and_config 2024-01-24T23:26:39.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.311+0000 7f33b0ab6640 10 monclient: build_initial_monmap 2024-01-24T23:26:39.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.311+0000 7f33b0ab6640 10 monclient: monmap: 2024-01-24T23:26:39.315 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:39.315 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:39.315 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:39.316689+0000 2024-01-24T23:26:39.316 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:39.316689+0000 2024-01-24T23:26:39.316 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:39.316 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:39.316 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:39.316 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:39.316 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:39.316 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:39.316 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:39.316 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:39.316 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:39.316 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:39.316 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:39.316 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:39.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 1 Processor -- start 2024-01-24T23:26:39.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 1 -- start start 2024-01-24T23:26:39.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 10 monclient: init 2024-01-24T23:26:39.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:39.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 10 monclient: _add_conns ranks=[7,5,1,8,3,6,0,2,4] 2024-01-24T23:26:39.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 10 monclient(hunting): picked mon.noname-i con 0x7f33ac135ea0 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:39.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 10 monclient(hunting): picked mon.noname-h con 0x7f33ac09bcd0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:39.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 10 monclient(hunting): picked mon.noname-f con 0x7f33ac09c0f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:39.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33ac064700 con 0x7f33ac09c0f0 2024-01-24T23:26:39.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33ac136990 con 0x7f33ac09bcd0 2024-01-24T23:26:39.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33ac09b010 con 0x7f33ac135ea0 2024-01-24T23:26:39.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:39.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 10 monclient(hunting): authenticate will time out at 1286.641279s 2024-01-24T23:26:39.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33aad9d640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f33ac09c0f0 legacy=0x7f33ac13f600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:39.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33aa59c640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f33ac135ea0 legacy=0x7f33ac136280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:39.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a9d9b640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f33ac09bcd0 0x7f33ac13bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:54870/0 (socket says 172.21.15.132:54870) 2024-01-24T23:26:39.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a9d9b640 1 -- 172.21.15.132:0/3348417374 learned_addr learned my addr 172.21.15.132:0/3348417374 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:39.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 1 -- 172.21.15.132:0/3348417374 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3573100069 0 0) 0x7f33ac136990 con 0x7f33ac09bcd0 2024-01-24T23:26:39.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 10 monclient(hunting): _negotiate 2024-01-24T23:26:39.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:39.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:39.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 10 monclient(hunting): my global_id is 6233 2024-01-24T23:26:39.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 1 -- 172.21.15.132:0/3348417374 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3394003530 con 0x7f33ac09bcd0 2024-01-24T23:26:39.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 1 -- 172.21.15.132:0/3348417374 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2633412491 0 0) 0x7f3394003530 con 0x7f33ac09bcd0 2024-01-24T23:26:39.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 1 -- 172.21.15.132:0/3348417374 >> v1:172.21.15.132:6792/0 conn(0x7f33ac135ea0 legacy=0x7f33ac136280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:39.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 1 -- 172.21.15.132:0/3348417374 >> v1:172.21.15.132:6789/0 conn(0x7f33ac09c0f0 legacy=0x7f33ac13f600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:39.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 10 monclient: _finish_hunting 0 2024-01-24T23:26:39.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 1 monclient: found mon.noname-h 2024-01-24T23:26:39.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:39.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:26:39.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 1 -- 172.21.15.132:0/3348417374 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ac0a4f50 con 0x7f33ac09bcd0 2024-01-24T23:26:39.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 10 monclient: _finish_auth 0 2024-01-24T23:26:39.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 10 monclient: _check_auth_tickets 2024-01-24T23:26:39.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:39.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 1 -- 172.21.15.132:0/3348417374 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f33a00017c0 con 0x7f33ac09bcd0 2024-01-24T23:26:39.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 5 monclient: authenticate success, global_id 6233 2024-01-24T23:26:39.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:39.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:39.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:39.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 10 monclient: dump: 2024-01-24T23:26:39.325 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:39.325 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:39.325 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:39.325 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:39.325 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:39.326 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:39.326 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:39.326 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:39.326 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:39.326 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:39.326 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:39.326 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:39.326 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:39.326 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:39.326 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:39.326 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:39.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:39.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 1 -- 172.21.15.132:0/3348417374 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f33a0001ae0 con 0x7f33ac09bcd0 2024-01-24T23:26:39.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:39.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 1 -- 172.21.15.132:0/3348417374 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f33a0004bf0 con 0x7f33ac09bcd0 2024-01-24T23:26:39.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:39.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 4 set_mon_vals no callback set 2024-01-24T23:26:39.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:39.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:39.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33a959a640 10 monclient: dump: 2024-01-24T23:26:39.327 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:39.327 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:39.327 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:39.327 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:39.327 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:39.327 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:39.327 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:39.327 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:39.327 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:39.328 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:39.328 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:39.328 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:39.328 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:39.328 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:39.328 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:39.328 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:39.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 10 monclient: shutdown 2024-01-24T23:26:39.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:39.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 1 -- 172.21.15.132:0/3348417374 >> v1:172.21.15.132:6791/0 conn(0x7f33ac09bcd0 legacy=0x7f33ac13bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:39.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 1 -- 172.21.15.132:0/3348417374 shutdown_connections 2024-01-24T23:26:39.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 1 -- 172.21.15.132:0/3348417374 >> 172.21.15.132:0/3348417374 conn(0x7f33ac0a9bf0 msgr2=0x7f33ac0a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:39.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 1 -- 172.21.15.132:0/3348417374 shutdown_connections 2024-01-24T23:26:39.328 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 1 -- 172.21.15.132:0/3348417374 wait complete. 2024-01-24T23:26:39.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 10 monclient: build_initial_monmap 2024-01-24T23:26:39.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 10 monclient: monmap: 2024-01-24T23:26:39.329 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:39.329 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:39.329 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:39.329 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:39.329 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:39.329 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:39.329 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:39.329 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:39.329 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:39.329 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:39.329 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:39.329 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:39.330 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:39.330 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:39.330 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:39.330 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:39.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 1 Processor -- start 2024-01-24T23:26:39.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 1 -- start start 2024-01-24T23:26:39.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.315+0000 7f33b0ab6640 10 monclient: init 2024-01-24T23:26:39.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33b0ab6640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:39.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33b0ab6640 10 monclient: _add_conns ranks=[3,8,0,5,2,1,7,4,6] 2024-01-24T23:26:39.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33b0ab6640 10 monclient(hunting): picked mon.noname-g con 0x7f33ac09bcd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:39.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33b0ab6640 10 monclient(hunting): picked mon.noname-e con 0x7f33ac09c0f0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:39.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33b0ab6640 10 monclient(hunting): picked mon.noname-a con 0x7f33ac135ea0 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:39.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33b0ab6640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33ac09c6b0 con 0x7f33ac135ea0 2024-01-24T23:26:39.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33b0ab6640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33ac125320 con 0x7f33ac09bcd0 2024-01-24T23:26:39.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33b0ab6640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33ac1234d0 con 0x7f33ac09c0f0 2024-01-24T23:26:39.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33b0ab6640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:39.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33b0ab6640 10 monclient(hunting): authenticate will time out at 1286.644105s 2024-01-24T23:26:39.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33aa59c640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f33ac09bcd0 0x7f33ac1cb740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:48704/0 (socket says 172.21.15.132:48704) 2024-01-24T23:26:39.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33aa59c640 1 -- 172.21.15.132:0/1363720669 learned_addr learned my addr 172.21.15.132:0/1363720669 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:39.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 1 -- 172.21.15.132:0/1363720669 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106774012 0 0) 0x7f33ac125320 con 0x7f33ac09bcd0 2024-01-24T23:26:39.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:39.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:39.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:39.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 10 monclient(hunting): my global_id is 6357 2024-01-24T23:26:39.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 1 -- 172.21.15.132:0/1363720669 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f338c003530 con 0x7f33ac09bcd0 2024-01-24T23:26:39.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 1 -- 172.21.15.132:0/1363720669 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 766180250 0 0) 0x7f338c003530 con 0x7f33ac09bcd0 2024-01-24T23:26:39.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 1 -- 172.21.15.132:0/1363720669 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f33ac125320 con 0x7f33ac09bcd0 2024-01-24T23:26:39.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 1 -- 172.21.15.132:0/1363720669 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f339c0042f0 con 0x7f33ac09bcd0 2024-01-24T23:26:39.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:39.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:39.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 10 monclient(hunting): dump: 2024-01-24T23:26:39.332 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:39.332 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:39.332 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:39.332 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:39.332 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:39.332 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:39.332 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:39.332 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:39.332 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:39.332 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:39.332 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:39.332 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:39.332 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:39.333 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:39.333 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:39.333 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:39.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:39.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 1 -- 172.21.15.132:0/1363720669 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3630039720 0 0) 0x7f33ac125320 con 0x7f33ac09bcd0 2024-01-24T23:26:39.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 1 -- 172.21.15.132:0/1363720669 >> v1:172.21.15.115:6793/0 conn(0x7f33ac09c0f0 legacy=0x7f33ac1d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:39.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 1 -- 172.21.15.132:0/1363720669 >> v1:172.21.15.115:6789/0 conn(0x7f33ac135ea0 legacy=0x7f33ac1d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:39.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 10 monclient: _finish_hunting 0 2024-01-24T23:26:39.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 1 monclient: found mon.g 2024-01-24T23:26:39.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:39.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:39.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 1 -- 172.21.15.132:0/1363720669 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ac135c70 con 0x7f33ac09bcd0 2024-01-24T23:26:39.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 10 monclient: _finish_auth 0 2024-01-24T23:26:39.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 10 monclient: _check_auth_tickets 2024-01-24T23:26:39.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:39.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33b0ab6640 5 monclient: authenticate success, global_id 6357 2024-01-24T23:26:39.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33b0ab6640 10 monclient: _renew_subs 2024-01-24T23:26:39.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33b0ab6640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:39.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33b0ab6640 1 -- 172.21.15.132:0/1363720669 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33ac125040 con 0x7f33ac09bcd0 2024-01-24T23:26:39.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33b0ab6640 10 monclient: _renew_subs 2024-01-24T23:26:39.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33b0ab6640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:39.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33b0ab6640 1 -- 172.21.15.132:0/1363720669 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ac125d50 con 0x7f33ac09bcd0 2024-01-24T23:26:39.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 1 -- 172.21.15.132:0/1363720669 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f339c0046e0 con 0x7f33ac09bcd0 2024-01-24T23:26:39.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:39.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 1 -- 172.21.15.132:0/1363720669 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f339c004f50 con 0x7f33ac09bcd0 2024-01-24T23:26:39.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:39.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:39.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 10 monclient: dump: 2024-01-24T23:26:39.334 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:39.334 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:39.335 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:39.335 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:39.335 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:39.335 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:39.335 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:39.335 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:39.335 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:39.335 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:39.335 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:39.335 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:39.335 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:39.335 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:39.335 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:39.335 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:39.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33a8d99640 4 set_mon_vals no callback set 2024-01-24T23:26:39.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33b0ab6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:39.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33b0ab6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:39.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33b0ab6640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:39.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 1 -- 172.21.15.132:0/1363720669 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f339c00f9d0 con 0x7f33ac09bcd0 2024-01-24T23:26:39.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33b0ab6640 1 -- 172.21.15.132:0/1363720669 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3390005090 con 0x7f33ac09bcd0 2024-01-24T23:26:39.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f339affd640 1 -- 172.21.15.132:0/1363720669 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(181..181 src has 1..181) v4 ==== 7625+0+0 (unknown 3525078538 0 0) 0x7f339c044820 con 0x7f33ac09bcd0 2024-01-24T23:26:39.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.319+0000 7f33a9d9b640 10 monclient: get_auth_request con 0x7f338c036f60 auth_method 0 2024-01-24T23:26:39.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.323+0000 7f339affd640 1 -- 172.21.15.132:0/1363720669 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f339c0114c0 con 0x7f33ac09bcd0 2024-01-24T23:26:39.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.323+0000 7f339affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:39.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.323+0000 7f339affd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:39.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.479+0000 7f33b0ab6640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:39.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.479+0000 7f33b0ab6640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:39.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.479+0000 7f33b0ab6640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:39.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:39.479+0000 7f33b0ab6640 1 -- 172.21.15.132:0/1363720669 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f33900058c0 con 0x7f33ac09bcd0 2024-01-24T23:26:40.088 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.087+0000 7f339affd640 1 -- 172.21.15.132:0/1363720669 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}]=0 added bucket h3 type host to location {rack=r1} v182) v1 ==== 166+0+0 (unknown 957473122 0 0) 0x7f339c012f90 con 0x7f33ac09bcd0 2024-01-24T23:26:40.088 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.087+0000 7f339affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:40.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.087+0000 7f339affd640 10 monclient: _finish_command 2 = system:0 added bucket h3 type host to location {rack=r1} 2024-01-24T23:26:40.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.187+0000 7f33b0ab6640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:40.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.187+0000 7f33b0ab6640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:40.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.187+0000 7f33b0ab6640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:40.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.187+0000 7f33b0ab6640 1 -- 172.21.15.132:0/1363720669 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f3390002220 con 0x7f33ac09bcd0 2024-01-24T23:26:40.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.191+0000 7f339affd640 1 -- 172.21.15.132:0/1363720669 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h3' already exists v182) v1 ==== 145+0+0 (unknown 1788953705 0 0) 0x7f339c014710 con 0x7f33ac09bcd0 2024-01-24T23:26:40.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.191+0000 7f339affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:40.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.191+0000 7f339affd640 10 monclient: _finish_command 3 = system:0 bucket 'h3' already exists 2024-01-24T23:26:40.193 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h3' already exists 2024-01-24T23:26:40.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.191+0000 7f33b0ab6640 1 -- 172.21.15.132:0/1363720669 >> v1:172.21.15.132:6812/1835401922 conn(0x7f338c036f60 legacy=0x7f338c039420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:40.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.191+0000 7f33b0ab6640 10 monclient: shutdown 2024-01-24T23:26:40.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.191+0000 7f33b0ab6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:40.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.191+0000 7f33b0ab6640 1 -- 172.21.15.132:0/1363720669 >> v1:172.21.15.132:6790/0 conn(0x7f33ac09bcd0 legacy=0x7f33ac1cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:40.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.191+0000 7f33b0ab6640 1 -- 172.21.15.132:0/1363720669 shutdown_connections 2024-01-24T23:26:40.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.191+0000 7f33b0ab6640 1 -- 172.21.15.132:0/1363720669 >> 172.21.15.132:0/1363720669 conn(0x7f33ac0a9bf0 msgr2=0x7f33ac0a2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:40.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.191+0000 7f33b0ab6640 1 -- 172.21.15.132:0/1363720669 shutdown_connections 2024-01-24T23:26:40.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.191+0000 7f33b0ab6640 1 -- 172.21.15.132:0/1363720669 wait complete. 2024-01-24T23:26:40.210 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:26:40.210 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h4 host rack=r1 2024-01-24T23:26:40.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.319+0000 7f203e3a6640 10 monclient: get_monmap_and_config 2024-01-24T23:26:40.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.319+0000 7f203e3a6640 10 monclient: build_initial_monmap 2024-01-24T23:26:40.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.319+0000 7f203e3a6640 10 monclient: monmap: 2024-01-24T23:26:40.323 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:40.323 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:40.324 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:40.324683+0000 2024-01-24T23:26:40.324 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:40.324683+0000 2024-01-24T23:26:40.324 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:40.324 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:40.324 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:40.324 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:40.325 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:40.325 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:40.325 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:40.325 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:40.325 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:40.325 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:40.326 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:40.326 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:40.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 1 Processor -- start 2024-01-24T23:26:40.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 1 -- start start 2024-01-24T23:26:40.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 10 monclient: init 2024-01-24T23:26:40.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:40.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 10 monclient: _add_conns ranks=[4,2,5,1,7,6,3,0,8] 2024-01-24T23:26:40.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 10 monclient(hunting): picked mon.noname-c con 0x7f20380a0ad0 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:40.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 10 monclient(hunting): picked mon.noname-b con 0x7f203809ee80 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:40.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 10 monclient(hunting): picked mon.noname-h con 0x7f203809b060 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:40.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2038064700 con 0x7f203809ee80 2024-01-24T23:26:40.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203809cb80 con 0x7f20380a0ad0 2024-01-24T23:26:40.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20380a0eb0 con 0x7f203809b060 2024-01-24T23:26:40.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:40.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 10 monclient(hunting): authenticate will time out at 1287.649259s 2024-01-24T23:26:40.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f20377fe640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f203809ee80 legacy=0x7f203809a950 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:40.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203c943640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f203809b060 0x7f203813c270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:54876/0 (socket says 172.21.15.132:54876) 2024-01-24T23:26:40.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203c943640 1 -- 172.21.15.132:0/2956898445 learned_addr learned my addr 172.21.15.132:0/2956898445 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:40.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 1 -- 172.21.15.132:0/2956898445 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1686535399 0 0) 0x7f20380a0eb0 con 0x7f203809b060 2024-01-24T23:26:40.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:40.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:40.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:40.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 10 monclient(hunting): my global_id is 6242 2024-01-24T23:26:40.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 1 -- 172.21.15.132:0/2956898445 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2024003530 con 0x7f203809b060 2024-01-24T23:26:40.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 1 -- 172.21.15.132:0/2956898445 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 980187887 0 0) 0x7f2024003530 con 0x7f203809b060 2024-01-24T23:26:40.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 1 -- 172.21.15.132:0/2956898445 >> v1:172.21.15.115:6791/0 conn(0x7f20380a0ad0 legacy=0x7f203809a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:40.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 1 -- 172.21.15.132:0/2956898445 >> v1:172.21.15.115:6790/0 conn(0x7f203809ee80 legacy=0x7f203809a950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:40.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:26:40.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 1 monclient: found mon.noname-h 2024-01-24T23:26:40.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:40.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:26:40.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 1 -- 172.21.15.132:0/2956898445 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203809f280 con 0x7f203809b060 2024-01-24T23:26:40.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 10 monclient: _finish_auth 0 2024-01-24T23:26:40.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:26:40.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:40.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 1 -- 172.21.15.132:0/2956898445 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2028001b40 con 0x7f203809b060 2024-01-24T23:26:40.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 5 monclient: authenticate success, global_id 6242 2024-01-24T23:26:40.335 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:40.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:40.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:40.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 10 monclient: dump: 2024-01-24T23:26:40.336 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:40.336 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:40.336 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:40.336 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:40.336 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:40.336 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:40.336 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:40.336 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:40.336 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:40.336 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:40.337 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:40.337 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:40.337 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:40.337 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:40.337 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:40.337 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:40.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:40.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 1 -- 172.21.15.132:0/2956898445 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2028001df0 con 0x7f203809b060 2024-01-24T23:26:40.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:40.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:40.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 4 set_mon_vals no callback set 2024-01-24T23:26:40.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 1 -- 172.21.15.132:0/2956898445 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2028004f70 con 0x7f203809b060 2024-01-24T23:26:40.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:40.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:40.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f2036ffd640 10 monclient: dump: 2024-01-24T23:26:40.338 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:40.338 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:40.338 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:40.338 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:40.338 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:40.338 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:40.338 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:40.338 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:40.338 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:40.338 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:40.338 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:40.338 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:40.338 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:40.338 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:40.339 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:40.339 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:40.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 10 monclient: shutdown 2024-01-24T23:26:40.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:40.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 1 -- 172.21.15.132:0/2956898445 >> v1:172.21.15.132:6791/0 conn(0x7f203809b060 legacy=0x7f203813c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:40.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 1 -- 172.21.15.132:0/2956898445 shutdown_connections 2024-01-24T23:26:40.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 1 -- 172.21.15.132:0/2956898445 >> 172.21.15.132:0/2956898445 conn(0x7f20380a1bc0 msgr2=0x7f20380a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:40.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 1 -- 172.21.15.132:0/2956898445 shutdown_connections 2024-01-24T23:26:40.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 1 -- 172.21.15.132:0/2956898445 wait complete. 2024-01-24T23:26:40.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 10 monclient: build_initial_monmap 2024-01-24T23:26:40.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 10 monclient: monmap: 2024-01-24T23:26:40.339 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:40.339 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:40.339 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:40.340 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:40.340 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:40.340 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:40.340 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:40.340 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:40.340 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:40.340 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:40.340 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:40.340 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:40.340 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:40.340 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:40.340 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:40.341 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:40.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 1 Processor -- start 2024-01-24T23:26:40.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 1 -- start start 2024-01-24T23:26:40.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.323+0000 7f203e3a6640 10 monclient: init 2024-01-24T23:26:40.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f203e3a6640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:40.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f203e3a6640 10 monclient: _add_conns ranks=[8,2,6,0,1,7,4,5,3] 2024-01-24T23:26:40.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f203e3a6640 10 monclient(hunting): picked mon.noname-e con 0x7f203809b060 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:40.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f203e3a6640 10 monclient(hunting): picked mon.noname-b con 0x7f203809ee80 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:40.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f203e3a6640 10 monclient(hunting): picked mon.noname-d con 0x7f20380a0ad0 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:40.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f203e3a6640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2038095910 con 0x7f203809ee80 2024-01-24T23:26:40.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f203e3a6640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203809b4c0 con 0x7f20380a0ad0 2024-01-24T23:26:40.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f203e3a6640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203809e640 con 0x7f203809b060 2024-01-24T23:26:40.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f203e3a6640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:40.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f203e3a6640 10 monclient(hunting): authenticate will time out at 1287.652302s 2024-01-24T23:26:40.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f20377fe640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f203809ee80 legacy=0x7f20381ce6d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:40.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2037fff640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f203809b060 0x7f2038099390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:41114/0 (socket says 172.21.15.132:41114) 2024-01-24T23:26:40.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2037fff640 1 -- 172.21.15.132:0/4076220110 learned_addr learned my addr 172.21.15.132:0/4076220110 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:40.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 1 -- 172.21.15.132:0/4076220110 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2135462730 0 0) 0x7f203809b4c0 con 0x7f20380a0ad0 2024-01-24T23:26:40.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:26:40.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:40.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:40.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 10 monclient(hunting): my global_id is 6459 2024-01-24T23:26:40.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 1 -- 172.21.15.132:0/4076220110 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2018003530 con 0x7f20380a0ad0 2024-01-24T23:26:40.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 1 -- 172.21.15.132:0/4076220110 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 761974337 0 0) 0x7f203809e640 con 0x7f203809b060 2024-01-24T23:26:40.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:26:40.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:40.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:40.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 10 monclient(hunting): my global_id is 85589 2024-01-24T23:26:40.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 1 -- 172.21.15.132:0/4076220110 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f203809b4c0 con 0x7f203809b060 2024-01-24T23:26:40.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 1 -- 172.21.15.132:0/4076220110 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3543207295 0 0) 0x7f203809b4c0 con 0x7f203809b060 2024-01-24T23:26:40.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 1 -- 172.21.15.132:0/4076220110 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f203809e640 con 0x7f203809b060 2024-01-24T23:26:40.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 1 -- 172.21.15.132:0/4076220110 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2030001a40 con 0x7f203809b060 2024-01-24T23:26:40.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:40.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:26:40.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 10 monclient(hunting): dump: 2024-01-24T23:26:40.343 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:40.343 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:40.343 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:40.343 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:40.343 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:40.344 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:40.344 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:40.344 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:40.344 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:40.344 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:40.344 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:40.344 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:40.344 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:40.344 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:40.344 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:40.344 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:40.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:40.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 1 -- 172.21.15.132:0/4076220110 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 341642511 0 0) 0x7f2018003530 con 0x7f20380a0ad0 2024-01-24T23:26:40.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 1 -- 172.21.15.132:0/4076220110 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f203809b4c0 con 0x7f20380a0ad0 2024-01-24T23:26:40.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 1 -- 172.21.15.132:0/4076220110 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2028004eb0 con 0x7f20380a0ad0 2024-01-24T23:26:40.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:40.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:26:40.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 10 monclient(hunting): dump: 2024-01-24T23:26:40.345 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:40.345 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:40.345 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:40.345 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:40.345 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:40.345 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:40.346 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:40.346 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:40.346 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:40.346 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:40.346 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:40.346 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:40.346 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:40.346 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:40.346 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:40.346 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:40.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 1 -- 172.21.15.132:0/4076220110 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1790163650 0 0) 0x7f203809e640 con 0x7f203809b060 2024-01-24T23:26:40.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 1 -- 172.21.15.132:0/4076220110 >> v1:172.21.15.115:6792/0 conn(0x7f20380a0ad0 legacy=0x7f20381d1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:40.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 1 -- 172.21.15.132:0/4076220110 >> v1:172.21.15.115:6790/0 conn(0x7f203809ee80 legacy=0x7f20381ce6d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:40.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:26:40.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 1 monclient: found mon.e 2024-01-24T23:26:40.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:40.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:40.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 1 -- 172.21.15.132:0/4076220110 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20380955a0 con 0x7f203809b060 2024-01-24T23:26:40.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 10 monclient: _finish_auth 0 2024-01-24T23:26:40.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:26:40.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:40.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f203e3a6640 5 monclient: authenticate success, global_id 85589 2024-01-24T23:26:40.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f203e3a6640 10 monclient: _renew_subs 2024-01-24T23:26:40.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f203e3a6640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:40.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f203e3a6640 1 -- 172.21.15.132:0/4076220110 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2038094640 con 0x7f203809b060 2024-01-24T23:26:40.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f203e3a6640 10 monclient: _renew_subs 2024-01-24T23:26:40.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f203e3a6640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:40.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f203e3a6640 1 -- 172.21.15.132:0/4076220110 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2038093d10 con 0x7f203809b060 2024-01-24T23:26:40.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 1 -- 172.21.15.132:0/4076220110 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2030001d60 con 0x7f203809b060 2024-01-24T23:26:40.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:40.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 1 -- 172.21.15.132:0/4076220110 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2030003c90 con 0x7f203809b060 2024-01-24T23:26:40.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:40.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f20367fc640 4 set_mon_vals no callback set 2024-01-24T23:26:40.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:26:40.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 10 monclient: dump: 2024-01-24T23:26:40.348 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:40.348 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:40.348 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:40.348 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:40.348 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:40.348 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:40.349 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:40.349 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:40.349 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:40.349 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:40.349 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:40.349 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:40.349 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:40.349 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:40.349 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:40.349 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:40.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f203e3a6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:40.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f203e3a6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:40.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f203e3a6640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:40.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f203e3a6640 1 -- 172.21.15.132:0/4076220110 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f201c005090 con 0x7f203809b060 2024-01-24T23:26:40.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 1 -- 172.21.15.132:0/4076220110 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f2030003df0 con 0x7f203809b060 2024-01-24T23:26:40.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f2034ff9640 1 -- 172.21.15.132:0/4076220110 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(182..182 src has 1..182) v4 ==== 7835+0+0 (unknown 1355675547 0 0) 0x7f2030044690 con 0x7f203809b060 2024-01-24T23:26:40.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.327+0000 7f20377fe640 10 monclient: get_auth_request con 0x7f2018037680 auth_method 0 2024-01-24T23:26:40.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.343+0000 7f2034ff9640 1 -- 172.21.15.132:0/4076220110 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f2030014730 con 0x7f203809b060 2024-01-24T23:26:40.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.343+0000 7f2034ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:40.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.343+0000 7f2034ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:40.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.491+0000 7f203e3a6640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:40.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.491+0000 7f203e3a6640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:40.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.491+0000 7f203e3a6640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:40.494 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:40.491+0000 7f203e3a6640 1 -- 172.21.15.132:0/4076220110 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f201c005340 con 0x7f203809b060 2024-01-24T23:26:41.091 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.087+0000 7f2034ff9640 1 -- 172.21.15.132:0/4076220110 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}]=0 added bucket h4 type host to location {rack=r1} v183) v1 ==== 166+0+0 (unknown 1706600993 0 0) 0x7f2030014730 con 0x7f203809b060 2024-01-24T23:26:41.091 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.091+0000 7f2034ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:41.091 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.091+0000 7f2034ff9640 10 monclient: _finish_command 2 = system:0 added bucket h4 type host to location {rack=r1} 2024-01-24T23:26:41.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.191+0000 7f203e3a6640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:41.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.191+0000 7f203e3a6640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:41.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.191+0000 7f203e3a6640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:41.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.191+0000 7f203e3a6640 1 -- 172.21.15.132:0/4076220110 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f201c005a80 con 0x7f203809b060 2024-01-24T23:26:41.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.195+0000 7f2034ff9640 1 -- 172.21.15.132:0/4076220110 <== mon.8 v1:172.21.15.115:6793/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h4' already exists v183) v1 ==== 145+0+0 (unknown 449962575 0 0) 0x7f2030014730 con 0x7f203809b060 2024-01-24T23:26:41.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.195+0000 7f2034ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:41.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.195+0000 7f2034ff9640 10 monclient: _finish_command 3 = system:0 bucket 'h4' already exists 2024-01-24T23:26:41.195 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h4' already exists 2024-01-24T23:26:41.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.195+0000 7f203e3a6640 1 -- 172.21.15.132:0/4076220110 >> v1:172.21.15.132:6812/1835401922 conn(0x7f2018037680 legacy=0x7f2018039b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:41.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.195+0000 7f203e3a6640 10 monclient: shutdown 2024-01-24T23:26:41.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.195+0000 7f203e3a6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:41.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.195+0000 7f203e3a6640 1 -- 172.21.15.132:0/4076220110 >> v1:172.21.15.115:6793/0 conn(0x7f203809b060 legacy=0x7f2038099390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:41.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.195+0000 7f203e3a6640 1 -- 172.21.15.132:0/4076220110 shutdown_connections 2024-01-24T23:26:41.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.195+0000 7f203e3a6640 1 -- 172.21.15.132:0/4076220110 >> 172.21.15.132:0/4076220110 conn(0x7f20380a1bc0 msgr2=0x7f203808bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:41.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.195+0000 7f203e3a6640 1 -- 172.21.15.132:0/4076220110 shutdown_connections 2024-01-24T23:26:41.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.195+0000 7f203e3a6640 1 -- 172.21.15.132:0/4076220110 wait complete. 2024-01-24T23:26:41.212 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:26:41.212 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h5 host rack=r1 2024-01-24T23:26:41.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 10 monclient: get_monmap_and_config 2024-01-24T23:26:41.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 10 monclient: build_initial_monmap 2024-01-24T23:26:41.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 10 monclient: monmap: 2024-01-24T23:26:41.331 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:41.332 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:41.332 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:41.333019+0000 2024-01-24T23:26:41.332 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:41.333019+0000 2024-01-24T23:26:41.332 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:41.332 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:41.332 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:41.332 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:41.332 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:41.332 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:41.332 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:41.332 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:41.333 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:41.333 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:41.333 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:41.333 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:41.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 1 Processor -- start 2024-01-24T23:26:41.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 1 -- start start 2024-01-24T23:26:41.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 10 monclient: init 2024-01-24T23:26:41.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:41.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 10 monclient: _add_conns ranks=[8,5,7,4,3,0,2,1,6] 2024-01-24T23:26:41.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 10 monclient(hunting): picked mon.noname-e con 0x7f0384135ea0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:41.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 10 monclient(hunting): picked mon.noname-h con 0x7f038409bcd0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:41.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 10 monclient(hunting): picked mon.noname-i con 0x7f038409c0f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:41.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0384064700 con 0x7f038409bcd0 2024-01-24T23:26:41.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0384136990 con 0x7f038409c0f0 2024-01-24T23:26:41.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038409b010 con 0x7f0384135ea0 2024-01-24T23:26:41.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:41.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 10 monclient(hunting): authenticate will time out at 1288.657712s 2024-01-24T23:26:41.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0383fff640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f038409c0f0 legacy=0x7f038413f600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:41.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0382ffd640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f038409bcd0 0x7f038413bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:54886/0 (socket says 172.21.15.132:54886) 2024-01-24T23:26:41.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0382ffd640 1 -- 172.21.15.132:0/804183978 learned_addr learned my addr 172.21.15.132:0/804183978 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:41.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 1 -- 172.21.15.132:0/804183978 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2944803236 0 0) 0x7f0384064700 con 0x7f038409bcd0 2024-01-24T23:26:41.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 10 monclient(hunting): _negotiate 2024-01-24T23:26:41.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:41.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:41.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 10 monclient(hunting): my global_id is 6251 2024-01-24T23:26:41.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 1 -- 172.21.15.132:0/804183978 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0370003530 con 0x7f038409bcd0 2024-01-24T23:26:41.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 1 -- 172.21.15.132:0/804183978 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3123800743 0 0) 0x7f0370003530 con 0x7f038409bcd0 2024-01-24T23:26:41.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 1 -- 172.21.15.132:0/804183978 >> v1:172.21.15.115:6793/0 conn(0x7f0384135ea0 legacy=0x7f0384136280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:41.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 1 -- 172.21.15.132:0/804183978 >> v1:172.21.15.132:6792/0 conn(0x7f038409c0f0 legacy=0x7f038413f600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:41.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 10 monclient: _finish_hunting 0 2024-01-24T23:26:41.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 1 monclient: found mon.noname-h 2024-01-24T23:26:41.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:41.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:26:41.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 1 -- 172.21.15.132:0/804183978 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03840a4f50 con 0x7f038409bcd0 2024-01-24T23:26:41.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 10 monclient: _finish_auth 0 2024-01-24T23:26:41.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 10 monclient: _check_auth_tickets 2024-01-24T23:26:41.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:41.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 1 -- 172.21.15.132:0/804183978 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f036c0017c0 con 0x7f038409bcd0 2024-01-24T23:26:41.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 5 monclient: authenticate success, global_id 6251 2024-01-24T23:26:41.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:41.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:41.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:41.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 10 monclient: dump: 2024-01-24T23:26:41.341 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:41.341 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:41.341 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:41.341 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:41.341 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:41.342 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:41.342 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:41.342 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:41.342 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:41.342 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:41.342 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:41.342 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:41.342 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:41.342 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:41.342 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:41.342 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:41.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:41.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 1 -- 172.21.15.132:0/804183978 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f036c0019e0 con 0x7f038409bcd0 2024-01-24T23:26:41.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:41.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:41.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 4 set_mon_vals no callback set 2024-01-24T23:26:41.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 1 -- 172.21.15.132:0/804183978 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f036c004bf0 con 0x7f038409bcd0 2024-01-24T23:26:41.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:41.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:41.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f03827fc640 10 monclient: dump: 2024-01-24T23:26:41.343 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:41.343 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:41.343 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:41.343 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:41.343 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:41.343 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:41.343 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:41.343 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:41.343 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:41.343 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:41.343 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:41.343 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:41.343 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:41.344 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:41.344 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:41.344 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:41.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 10 monclient: shutdown 2024-01-24T23:26:41.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:41.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 1 -- 172.21.15.132:0/804183978 >> v1:172.21.15.132:6791/0 conn(0x7f038409bcd0 legacy=0x7f038413bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:41.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 1 -- 172.21.15.132:0/804183978 shutdown_connections 2024-01-24T23:26:41.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 1 -- 172.21.15.132:0/804183978 >> 172.21.15.132:0/804183978 conn(0x7f03840a9bf0 msgr2=0x7f03840a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:41.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 1 -- 172.21.15.132:0/804183978 shutdown_connections 2024-01-24T23:26:41.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 1 -- 172.21.15.132:0/804183978 wait complete. 2024-01-24T23:26:41.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 10 monclient: build_initial_monmap 2024-01-24T23:26:41.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.331+0000 7f0389e04640 10 monclient: monmap: 2024-01-24T23:26:41.344 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:41.344 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:41.344 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:41.345 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:41.345 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:41.345 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:41.345 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:41.345 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:41.345 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:41.345 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:41.345 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:41.345 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:41.345 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:41.345 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:41.345 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:41.345 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:41.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0389e04640 1 Processor -- start 2024-01-24T23:26:41.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0389e04640 1 -- start start 2024-01-24T23:26:41.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0389e04640 10 monclient: init 2024-01-24T23:26:41.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0389e04640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:41.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0389e04640 10 monclient: _add_conns ranks=[2,5,8,3,4,0,7,1,6] 2024-01-24T23:26:41.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0389e04640 10 monclient(hunting): picked mon.noname-b con 0x7f038409bcd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:41.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0389e04640 10 monclient(hunting): picked mon.noname-h con 0x7f038409c0f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:41.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0389e04640 10 monclient(hunting): picked mon.noname-e con 0x7f0384135ea0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:41.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0389e04640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0384122ac0 con 0x7f038409bcd0 2024-01-24T23:26:41.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0389e04640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038409c6b0 con 0x7f038409c0f0 2024-01-24T23:26:41.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0389e04640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0384135c70 con 0x7f0384135ea0 2024-01-24T23:26:41.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0389e04640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:41.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0389e04640 10 monclient(hunting): authenticate will time out at 1288.660737s 2024-01-24T23:26:41.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f03837fe640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f038409bcd0 legacy=0x7f0384134c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:41.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0382ffd640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f038409c0f0 0x7f03841d04e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:54896/0 (socket says 172.21.15.132:54896) 2024-01-24T23:26:41.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0382ffd640 1 -- 172.21.15.132:0/48848612 learned_addr learned my addr 172.21.15.132:0/48848612 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:41.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 1 -- 172.21.15.132:0/48848612 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 480992665 0 0) 0x7f038409c6b0 con 0x7f038409c0f0 2024-01-24T23:26:41.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:26:41.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:41.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:41.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 10 monclient(hunting): my global_id is 6260 2024-01-24T23:26:41.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 1 -- 172.21.15.132:0/48848612 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0364003530 con 0x7f038409c0f0 2024-01-24T23:26:41.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 1 -- 172.21.15.132:0/48848612 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3362792618 0 0) 0x7f0364003530 con 0x7f038409c0f0 2024-01-24T23:26:41.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 1 -- 172.21.15.132:0/48848612 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f038409c6b0 con 0x7f038409c0f0 2024-01-24T23:26:41.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 1 -- 172.21.15.132:0/48848612 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f036c001740 con 0x7f038409c0f0 2024-01-24T23:26:41.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:41.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:41.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 10 monclient(hunting): dump: 2024-01-24T23:26:41.348 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:41.348 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:41.348 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:41.348 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:41.348 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:41.348 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:41.348 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:41.348 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:41.348 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:41.348 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:41.348 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:41.348 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:41.348 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:41.348 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:41.348 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:41.349 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:41.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:41.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 1 -- 172.21.15.132:0/48848612 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3162087388 0 0) 0x7f0384135c70 con 0x7f0384135ea0 2024-01-24T23:26:41.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:26:41.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:41.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:41.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 10 monclient(hunting): my global_id is 85607 2024-01-24T23:26:41.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 1 -- 172.21.15.132:0/48848612 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0364003530 con 0x7f0384135ea0 2024-01-24T23:26:41.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 1 -- 172.21.15.132:0/48848612 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2381552672 0 0) 0x7f038409c6b0 con 0x7f038409c0f0 2024-01-24T23:26:41.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 1 -- 172.21.15.132:0/48848612 >> v1:172.21.15.115:6793/0 conn(0x7f0384135ea0 legacy=0x7f03841d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:41.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 1 -- 172.21.15.132:0/48848612 >> v1:172.21.15.115:6790/0 conn(0x7f038409bcd0 legacy=0x7f0384134c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:41.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:26:41.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 1 monclient: found mon.h 2024-01-24T23:26:41.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:41.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:41.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 1 -- 172.21.15.132:0/48848612 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03841353a0 con 0x7f038409c0f0 2024-01-24T23:26:41.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 10 monclient: _finish_auth 0 2024-01-24T23:26:41.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:26:41.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:41.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0389e04640 5 monclient: authenticate success, global_id 6260 2024-01-24T23:26:41.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0389e04640 10 monclient: _renew_subs 2024-01-24T23:26:41.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0389e04640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:41.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0389e04640 1 -- 172.21.15.132:0/48848612 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0384123f00 con 0x7f038409c0f0 2024-01-24T23:26:41.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0389e04640 10 monclient: _renew_subs 2024-01-24T23:26:41.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0389e04640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:41.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0389e04640 1 -- 172.21.15.132:0/48848612 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f03841241e0 con 0x7f038409c0f0 2024-01-24T23:26:41.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 1 -- 172.21.15.132:0/48848612 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f036c004ec0 con 0x7f038409c0f0 2024-01-24T23:26:41.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:41.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 1 -- 172.21.15.132:0/48848612 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f036c005370 con 0x7f038409c0f0 2024-01-24T23:26:41.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:41.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:41.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0381ffb640 4 set_mon_vals no callback set 2024-01-24T23:26:41.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 10 monclient: dump: 2024-01-24T23:26:41.351 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:41.351 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:41.351 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:41.351 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:41.351 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:41.351 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:41.351 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:41.351 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:41.351 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:41.351 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:41.351 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:41.352 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:41.352 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:41.352 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:41.352 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:41.352 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:41.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0389e04640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:41.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0389e04640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:41.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0389e04640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:41.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f0389e04640 1 -- 172.21.15.132:0/48848612 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0368005090 con 0x7f038409c0f0 2024-01-24T23:26:41.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 1 -- 172.21.15.132:0/48848612 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f036c00fdf0 con 0x7f038409c0f0 2024-01-24T23:26:41.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f037bfff640 1 -- 172.21.15.132:0/48848612 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(183..183 src has 1..183) v4 ==== 8045+0+0 (unknown 3819998759 0 0) 0x7f036c044b60 con 0x7f038409c0f0 2024-01-24T23:26:41.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.335+0000 7f03837fe640 10 monclient: get_auth_request con 0x7f0364036f20 auth_method 0 2024-01-24T23:26:41.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.339+0000 7f037bfff640 1 -- 172.21.15.132:0/48848612 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f036c014e50 con 0x7f038409c0f0 2024-01-24T23:26:41.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.339+0000 7f037bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:41.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.339+0000 7f037bfff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:41.496 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~37s 2024-01-24T23:26:41.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.495+0000 7f0389e04640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:41.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.495+0000 7f0389e04640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:41.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.495+0000 7f0389e04640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:41.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:41.495+0000 7f0389e04640 1 -- 172.21.15.132:0/48848612 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f03680058c0 con 0x7f038409c0f0 2024-01-24T23:26:42.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.103+0000 7f037bfff640 1 -- 172.21.15.132:0/48848612 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}]=0 added bucket h5 type host to location {rack=r1} v184) v1 ==== 166+0+0 (unknown 1514771207 0 0) 0x7f036c0132c0 con 0x7f038409c0f0 2024-01-24T23:26:42.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.103+0000 7f037bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:42.104 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.103+0000 7f037bfff640 10 monclient: _finish_command 2 = system:0 added bucket h5 type host to location {rack=r1} 2024-01-24T23:26:42.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.199+0000 7f0389e04640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:42.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.199+0000 7f0389e04640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:42.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.199+0000 7f0389e04640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:42.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.199+0000 7f0389e04640 1 -- 172.21.15.132:0/48848612 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f0368002220 con 0x7f038409c0f0 2024-01-24T23:26:42.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.199+0000 7f037bfff640 1 -- 172.21.15.132:0/48848612 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h5' already exists v184) v1 ==== 145+0+0 (unknown 2853513808 0 0) 0x7f036c014a40 con 0x7f038409c0f0 2024-01-24T23:26:42.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.199+0000 7f037bfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:42.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.199+0000 7f037bfff640 10 monclient: _finish_command 3 = system:0 bucket 'h5' already exists 2024-01-24T23:26:42.201 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h5' already exists 2024-01-24T23:26:42.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.199+0000 7f0389e04640 1 -- 172.21.15.132:0/48848612 >> v1:172.21.15.132:6812/1835401922 conn(0x7f0364036f20 legacy=0x7f03640393e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:42.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.203+0000 7f0389e04640 10 monclient: shutdown 2024-01-24T23:26:42.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.203+0000 7f0389e04640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:42.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.203+0000 7f0389e04640 1 -- 172.21.15.132:0/48848612 >> v1:172.21.15.132:6791/0 conn(0x7f038409c0f0 legacy=0x7f03841d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:42.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.203+0000 7f0389e04640 1 -- 172.21.15.132:0/48848612 shutdown_connections 2024-01-24T23:26:42.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.203+0000 7f0389e04640 1 -- 172.21.15.132:0/48848612 >> 172.21.15.132:0/48848612 conn(0x7f03840a9bf0 msgr2=0x7f03840a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:42.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.203+0000 7f0389e04640 1 -- 172.21.15.132:0/48848612 shutdown_connections 2024-01-24T23:26:42.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.203+0000 7f0389e04640 1 -- 172.21.15.132:0/48848612 wait complete. 2024-01-24T23:26:42.218 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:26:42.218 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h6 host rack=r1 2024-01-24T23:26:42.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.335+0000 7fbbd2cc7640 10 monclient: get_monmap_and_config 2024-01-24T23:26:42.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.335+0000 7fbbd2cc7640 10 monclient: build_initial_monmap 2024-01-24T23:26:42.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.335+0000 7fbbd2cc7640 10 monclient: monmap: 2024-01-24T23:26:42.337 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:42.337 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:42.337 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:42.338553+0000 2024-01-24T23:26:42.337 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:42.338553+0000 2024-01-24T23:26:42.337 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:42.338 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:42.338 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:42.338 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:42.338 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:42.338 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:42.338 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:42.338 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:42.338 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:42.338 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:42.338 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:42.338 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:42.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.335+0000 7fbbd2cc7640 1 Processor -- start 2024-01-24T23:26:42.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.335+0000 7fbbd2cc7640 1 -- start start 2024-01-24T23:26:42.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.335+0000 7fbbd2cc7640 10 monclient: init 2024-01-24T23:26:42.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.335+0000 7fbbd2cc7640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:42.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.335+0000 7fbbd2cc7640 10 monclient: _add_conns ranks=[0,8,7,2,6,1,3,5,4] 2024-01-24T23:26:42.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.335+0000 7fbbd2cc7640 10 monclient(hunting): picked mon.noname-a con 0x7fbbcc072690 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:42.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.335+0000 7fbbd2cc7640 10 monclient(hunting): picked mon.noname-e con 0x7fbbcc09ae90 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:42.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.335+0000 7fbbd2cc7640 10 monclient(hunting): picked mon.noname-i con 0x7fbbcc09bdc0 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:42.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.335+0000 7fbbd2cc7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbcc064700 con 0x7fbbcc072690 2024-01-24T23:26:42.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.335+0000 7fbbd2cc7640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbcc09a790 con 0x7fbbcc09bdc0 2024-01-24T23:26:42.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.335+0000 7fbbd2cc7640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbcc0a18f0 con 0x7fbbcc09ae90 2024-01-24T23:26:42.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.335+0000 7fbbd2cc7640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:42.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.335+0000 7fbbd2cc7640 10 monclient(hunting): authenticate will time out at 1289.663604s 2024-01-24T23:26:42.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.335+0000 7fbbd1264640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fbbcc09bdc0 legacy=0x7fbbcc13d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:42.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcbfff640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fbbcc09ae90 0x7fbbcc0a11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:41124/0 (socket says 172.21.15.132:41124) 2024-01-24T23:26:42.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcbfff640 1 -- 172.21.15.132:0/2109368962 learned_addr learned my addr 172.21.15.132:0/2109368962 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:42.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbd0a63640 1 --1- 172.21.15.132:0/2109368962 >> v1:172.21.15.115:6789/0 conn(0x7fbbcc072690 0x7fbbcc0a0ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:47086/0 (socket says 172.21.15.132:47086) 2024-01-24T23:26:42.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 1 -- 172.21.15.132:0/2109368962 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4189693366 0 0) 0x7fbbcc064700 con 0x7fbbcc072690 2024-01-24T23:26:42.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:26:42.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:42.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:42.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 10 monclient(hunting): my global_id is 94725 2024-01-24T23:26:42.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 1 -- 172.21.15.132:0/2109368962 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbb4003530 con 0x7fbbcc072690 2024-01-24T23:26:42.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 1 -- 172.21.15.132:0/2109368962 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 113237830 0 0) 0x7fbbcc0a18f0 con 0x7fbbcc09ae90 2024-01-24T23:26:42.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:26:42.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:42.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:42.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 10 monclient(hunting): my global_id is 85616 2024-01-24T23:26:42.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 1 -- 172.21.15.132:0/2109368962 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbcc064700 con 0x7fbbcc09ae90 2024-01-24T23:26:42.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 1 -- 172.21.15.132:0/2109368962 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1231620698 0 0) 0x7fbbcc064700 con 0x7fbbcc09ae90 2024-01-24T23:26:42.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 1 -- 172.21.15.132:0/2109368962 >> v1:172.21.15.132:6792/0 conn(0x7fbbcc09bdc0 legacy=0x7fbbcc13d280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:42.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 1 -- 172.21.15.132:0/2109368962 >> v1:172.21.15.115:6789/0 conn(0x7fbbcc072690 legacy=0x7fbbcc0a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:42.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:26:42.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 1 monclient: found mon.noname-e 2024-01-24T23:26:42.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:42.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:26:42.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 1 -- 172.21.15.132:0/2109368962 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbcc0997a0 con 0x7fbbcc09ae90 2024-01-24T23:26:42.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 10 monclient: _finish_auth 0 2024-01-24T23:26:42.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:26:42.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:42.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 1 -- 172.21.15.132:0/2109368962 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbbc00017a0 con 0x7fbbcc09ae90 2024-01-24T23:26:42.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbd2cc7640 5 monclient: authenticate success, global_id 85616 2024-01-24T23:26:42.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbd2cc7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:42.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:42.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:26:42.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 10 monclient: dump: 2024-01-24T23:26:42.346 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:42.346 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:42.347 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:42.347 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:42.347 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:42.347 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:42.347 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:42.347 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:42.347 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:42.347 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:42.347 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:42.347 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:42.347 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:42.347 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:42.347 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:42.347 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:42.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbd2cc7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:42.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 1 -- 172.21.15.132:0/2109368962 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbbc0001a50 con 0x7fbbcc09ae90 2024-01-24T23:26:42.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:42.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbd2cc7640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:42.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 1 -- 172.21.15.132:0/2109368962 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbbc0004bf0 con 0x7fbbcc09ae90 2024-01-24T23:26:42.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbd2cc7640 4 set_mon_vals no callback set 2024-01-24T23:26:42.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:42.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:26:42.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbcb7fe640 10 monclient: dump: 2024-01-24T23:26:42.348 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:42.348 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:42.348 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:42.348 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:42.348 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:42.348 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:42.349 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:42.349 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:42.349 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:42.349 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:42.349 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:42.349 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:42.349 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:42.349 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:42.349 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:42.349 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:42.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbd2cc7640 10 monclient: shutdown 2024-01-24T23:26:42.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbd2cc7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:42.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbd2cc7640 1 -- 172.21.15.132:0/2109368962 >> v1:172.21.15.115:6793/0 conn(0x7fbbcc09ae90 legacy=0x7fbbcc0a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:42.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbd2cc7640 1 -- 172.21.15.132:0/2109368962 shutdown_connections 2024-01-24T23:26:42.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbd2cc7640 1 -- 172.21.15.132:0/2109368962 >> 172.21.15.132:0/2109368962 conn(0x7fbbcc0a1bc0 msgr2=0x7fbbcc0a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:42.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbd2cc7640 1 -- 172.21.15.132:0/2109368962 shutdown_connections 2024-01-24T23:26:42.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbd2cc7640 1 -- 172.21.15.132:0/2109368962 wait complete. 2024-01-24T23:26:42.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbd2cc7640 10 monclient: build_initial_monmap 2024-01-24T23:26:42.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbd2cc7640 10 monclient: monmap: 2024-01-24T23:26:42.350 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:42.350 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:42.350 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:42.350 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:42.350 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:42.350 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:42.350 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:42.350 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:42.350 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:42.350 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:42.350 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:42.351 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:42.351 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:42.351 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:42.351 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:42.351 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:42.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbd2cc7640 1 Processor -- start 2024-01-24T23:26:42.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbd2cc7640 1 -- start start 2024-01-24T23:26:42.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbd2cc7640 10 monclient: init 2024-01-24T23:26:42.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbd2cc7640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:42.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbd2cc7640 10 monclient: _add_conns ranks=[4,2,6,1,0,8,3,7,5] 2024-01-24T23:26:42.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbd2cc7640 10 monclient(hunting): picked mon.noname-c con 0x7fbbcc072690 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:42.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.339+0000 7fbbd2cc7640 10 monclient(hunting): picked mon.noname-b con 0x7fbbcc09ae90 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:42.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbd2cc7640 10 monclient(hunting): picked mon.noname-d con 0x7fbbcc09bdc0 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:42.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbd2cc7640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbcc08fe20 con 0x7fbbcc09ae90 2024-01-24T23:26:42.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbd2cc7640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbcc091990 con 0x7fbbcc072690 2024-01-24T23:26:42.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbd2cc7640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbcc0909a0 con 0x7fbbcc09bdc0 2024-01-24T23:26:42.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbd2cc7640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:42.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbd2cc7640 10 monclient(hunting): authenticate will time out at 1289.668092s 2024-01-24T23:26:42.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbcbfff640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fbbcc09ae90 legacy=0x7fbbcc1ce850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:42.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbd1264640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fbbcc09bdc0 0x7fbbcc1d1f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:36916/0 (socket says 172.21.15.132:36916) 2024-01-24T23:26:42.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbd1264640 1 -- 172.21.15.132:0/1822256398 learned_addr learned my addr 172.21.15.132:0/1822256398 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:42.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbd0a63640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fbbcc072690 0x7fbbcc0951c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:53978/0 (socket says 172.21.15.132:53978) 2024-01-24T23:26:42.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 1 -- 172.21.15.132:0/1822256398 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1884435608 0 0) 0x7fbbcc091990 con 0x7fbbcc072690 2024-01-24T23:26:42.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 10 monclient(hunting): _negotiate 2024-01-24T23:26:42.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:42.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:42.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 10 monclient(hunting): my global_id is 94567 2024-01-24T23:26:42.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 1 -- 172.21.15.132:0/1822256398 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbb8003530 con 0x7fbbcc072690 2024-01-24T23:26:42.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 1 -- 172.21.15.132:0/1822256398 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 496259170 0 0) 0x7fbbcc0909a0 con 0x7fbbcc09bdc0 2024-01-24T23:26:42.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 10 monclient(hunting): _negotiate 2024-01-24T23:26:42.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:42.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:42.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 10 monclient(hunting): my global_id is 6468 2024-01-24T23:26:42.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 1 -- 172.21.15.132:0/1822256398 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbcc091990 con 0x7fbbcc09bdc0 2024-01-24T23:26:42.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 1 -- 172.21.15.132:0/1822256398 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2317257993 0 0) 0x7fbbb8003530 con 0x7fbbcc072690 2024-01-24T23:26:42.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 1 -- 172.21.15.132:0/1822256398 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbbcc0909a0 con 0x7fbbcc072690 2024-01-24T23:26:42.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 1 -- 172.21.15.132:0/1822256398 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbbbc0018d0 con 0x7fbbcc072690 2024-01-24T23:26:42.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:42.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:26:42.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 10 monclient(hunting): dump: 2024-01-24T23:26:42.356 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:42.356 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:42.356 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:42.356 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:42.356 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:42.356 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:42.356 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:42.356 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:42.357 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:42.357 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:42.357 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:42.357 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:42.357 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:42.357 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:42.357 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:42.357 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:42.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:42.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 1 -- 172.21.15.132:0/1822256398 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4030077848 0 0) 0x7fbbcc091990 con 0x7fbbcc09bdc0 2024-01-24T23:26:42.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 1 -- 172.21.15.132:0/1822256398 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbbb8003530 con 0x7fbbcc09bdc0 2024-01-24T23:26:42.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 1 -- 172.21.15.132:0/1822256398 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbbc4004530 con 0x7fbbcc09bdc0 2024-01-24T23:26:42.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:42.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:26:42.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 10 monclient(hunting): dump: 2024-01-24T23:26:42.358 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:42.358 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:42.358 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:42.358 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:42.358 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:42.358 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:42.358 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:42.358 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:42.358 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:42.358 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:42.358 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:42.358 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:42.358 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:42.358 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:42.359 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:42.359 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:42.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 1 -- 172.21.15.132:0/1822256398 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2440638146 0 0) 0x7fbbcc0909a0 con 0x7fbbcc072690 2024-01-24T23:26:42.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 1 -- 172.21.15.132:0/1822256398 >> v1:172.21.15.115:6792/0 conn(0x7fbbcc09bdc0 legacy=0x7fbbcc1d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:42.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 1 -- 172.21.15.132:0/1822256398 >> v1:172.21.15.115:6790/0 conn(0x7fbbcc09ae90 legacy=0x7fbbcc1ce850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:42.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 10 monclient: _finish_hunting 0 2024-01-24T23:26:42.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 1 monclient: found mon.c 2024-01-24T23:26:42.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:42.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:42.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 1 -- 172.21.15.132:0/1822256398 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbcc099470 con 0x7fbbcc072690 2024-01-24T23:26:42.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 10 monclient: _finish_auth 0 2024-01-24T23:26:42.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 10 monclient: _check_auth_tickets 2024-01-24T23:26:42.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:42.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbd2cc7640 5 monclient: authenticate success, global_id 94567 2024-01-24T23:26:42.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbd2cc7640 10 monclient: _renew_subs 2024-01-24T23:26:42.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbd2cc7640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:42.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbd2cc7640 1 -- 172.21.15.132:0/1822256398 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbcc09b930 con 0x7fbbcc072690 2024-01-24T23:26:42.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbd2cc7640 10 monclient: _renew_subs 2024-01-24T23:26:42.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbd2cc7640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:42.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbd2cc7640 1 -- 172.21.15.132:0/1822256398 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbcc0958d0 con 0x7fbbcc072690 2024-01-24T23:26:42.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 1 -- 172.21.15.132:0/1822256398 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbbbc001ce0 con 0x7fbbcc072690 2024-01-24T23:26:42.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:42.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 1 -- 172.21.15.132:0/1822256398 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbbbc004de0 con 0x7fbbcc072690 2024-01-24T23:26:42.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:42.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbcaffd640 4 set_mon_vals no callback set 2024-01-24T23:26:42.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:26:42.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 10 monclient: dump: 2024-01-24T23:26:42.360 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:42.361 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:42.361 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:42.361 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:42.361 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:42.361 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:42.361 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:42.361 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:42.361 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:42.361 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:42.361 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:42.361 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:42.361 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:42.361 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:42.361 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:42.361 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:42.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbd2cc7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:42.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbd2cc7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:42.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbd2cc7640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:42.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbd2cc7640 1 -- 172.21.15.132:0/1822256398 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbac005090 con 0x7fbbcc072690 2024-01-24T23:26:42.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 1 -- 172.21.15.132:0/1822256398 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fbbbc00f860 con 0x7fbbcc072690 2024-01-24T23:26:42.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbc97fa640 1 -- 172.21.15.132:0/1822256398 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(184..184 src has 1..184) v4 ==== 8383+0+0 (unknown 907894897 0 0) 0x7fbbbc0448a0 con 0x7fbbcc072690 2024-01-24T23:26:42.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.343+0000 7fbbcbfff640 10 monclient: get_auth_request con 0x7fbbb80376a0 auth_method 0 2024-01-24T23:26:42.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.359+0000 7fbbc97fa640 1 -- 172.21.15.132:0/1822256398 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fbbbc010d10 con 0x7fbbcc072690 2024-01-24T23:26:42.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.359+0000 7fbbc97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:42.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.359+0000 7fbbc97fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:42.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.511+0000 7fbbd2cc7640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:42.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.511+0000 7fbbd2cc7640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:42.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.511+0000 7fbbd2cc7640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:42.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:42.511+0000 7fbbd2cc7640 1 -- 172.21.15.132:0/1822256398 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fbbac0058c0 con 0x7fbbcc072690 2024-01-24T23:26:43.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.123+0000 7fbbc97fa640 1 -- 172.21.15.132:0/1822256398 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}]=0 added bucket h6 type host to location {rack=r1} v185) v1 ==== 166+0+0 (unknown 2121139825 0 0) 0x7fbbbc012d90 con 0x7fbbcc072690 2024-01-24T23:26:43.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.123+0000 7fbbc97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:43.126 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.123+0000 7fbbc97fa640 10 monclient: _finish_command 2 = system:0 added bucket h6 type host to location {rack=r1} 2024-01-24T23:26:43.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.219+0000 7fbbd2cc7640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:43.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.219+0000 7fbbd2cc7640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:43.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.219+0000 7fbbd2cc7640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:43.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.219+0000 7fbbd2cc7640 1 -- 172.21.15.132:0/1822256398 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fbbac002220 con 0x7fbbcc072690 2024-01-24T23:26:43.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.223+0000 7fbbc97fa640 1 -- 172.21.15.132:0/1822256398 <== mon.4 v1:172.21.15.115:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h6' already exists v185) v1 ==== 145+0+0 (unknown 4222302204 0 0) 0x7fbbbc014510 con 0x7fbbcc072690 2024-01-24T23:26:43.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.223+0000 7fbbc97fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:43.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.223+0000 7fbbc97fa640 10 monclient: _finish_command 3 = system:0 bucket 'h6' already exists 2024-01-24T23:26:43.224 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h6' already exists 2024-01-24T23:26:43.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.223+0000 7fbbd2cc7640 1 -- 172.21.15.132:0/1822256398 >> v1:172.21.15.132:6812/1835401922 conn(0x7fbbb80376a0 legacy=0x7fbbb8039b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:43.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.223+0000 7fbbd2cc7640 10 monclient: shutdown 2024-01-24T23:26:43.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.223+0000 7fbbd2cc7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:43.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.223+0000 7fbbd2cc7640 1 -- 172.21.15.132:0/1822256398 >> v1:172.21.15.115:6791/0 conn(0x7fbbcc072690 legacy=0x7fbbcc0951c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:43.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.223+0000 7fbbd2cc7640 1 -- 172.21.15.132:0/1822256398 shutdown_connections 2024-01-24T23:26:43.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.223+0000 7fbbd2cc7640 1 -- 172.21.15.132:0/1822256398 >> 172.21.15.132:0/1822256398 conn(0x7fbbcc0a1bc0 msgr2=0x7fbbcc0989a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:43.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.223+0000 7fbbd2cc7640 1 -- 172.21.15.132:0/1822256398 shutdown_connections 2024-01-24T23:26:43.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.223+0000 7fbbd2cc7640 1 -- 172.21.15.132:0/1822256398 wait complete. 2024-01-24T23:26:43.240 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:26:43.240 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h7 host rack=r1 2024-01-24T23:26:43.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 10 monclient: get_monmap_and_config 2024-01-24T23:26:43.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 10 monclient: build_initial_monmap 2024-01-24T23:26:43.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 10 monclient: monmap: 2024-01-24T23:26:43.359 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:43.360 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:43.360 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:43.361040+0000 2024-01-24T23:26:43.360 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:43.361040+0000 2024-01-24T23:26:43.360 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:43.360 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:43.360 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:43.360 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:43.360 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:43.360 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:43.360 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:43.360 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:43.360 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:43.360 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:43.361 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:43.361 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:43.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 1 Processor -- start 2024-01-24T23:26:43.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 1 -- start start 2024-01-24T23:26:43.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 10 monclient: init 2024-01-24T23:26:43.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:43.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 10 monclient: _add_conns ranks=[3,7,2,6,4,5,0,8,1] 2024-01-24T23:26:43.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 10 monclient(hunting): picked mon.noname-g con 0x7fbecc135ea0 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:43.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 10 monclient(hunting): picked mon.noname-i con 0x7fbecc09bcd0 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:43.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 10 monclient(hunting): picked mon.noname-b con 0x7fbecc09c0f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:43.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbecc064700 con 0x7fbecc09c0f0 2024-01-24T23:26:43.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbecc136990 con 0x7fbecc135ea0 2024-01-24T23:26:43.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbecc09b010 con 0x7fbecc09bcd0 2024-01-24T23:26:43.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:43.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 10 monclient(hunting): authenticate will time out at 1290.685806s 2024-01-24T23:26:43.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbeca7fc640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fbecc09bcd0 legacy=0x7fbecc13bed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:43.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbecb7fe640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fbecc09c0f0 legacy=0x7fbecc13f600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:43.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbecaffd640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fbecc135ea0 0x7fbecc136280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:48706/0 (socket says 172.21.15.132:48706) 2024-01-24T23:26:43.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbecaffd640 1 -- 172.21.15.132:0/2469179503 learned_addr learned my addr 172.21.15.132:0/2469179503 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:43.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 1 -- 172.21.15.132:0/2469179503 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 456963244 0 0) 0x7fbecc136990 con 0x7fbecc135ea0 2024-01-24T23:26:43.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:26:43.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:43.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:43.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 10 monclient(hunting): my global_id is 6366 2024-01-24T23:26:43.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 1 -- 172.21.15.132:0/2469179503 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbeb8003530 con 0x7fbecc135ea0 2024-01-24T23:26:43.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 1 -- 172.21.15.132:0/2469179503 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1102007 0 0) 0x7fbeb8003530 con 0x7fbecc135ea0 2024-01-24T23:26:43.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 1 -- 172.21.15.132:0/2469179503 >> v1:172.21.15.132:6792/0 conn(0x7fbecc09bcd0 legacy=0x7fbecc13bed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:43.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 1 -- 172.21.15.132:0/2469179503 >> v1:172.21.15.115:6790/0 conn(0x7fbecc09c0f0 legacy=0x7fbecc13f600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:43.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:26:43.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 1 monclient: found mon.noname-g 2024-01-24T23:26:43.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:43.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:26:43.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 1 -- 172.21.15.132:0/2469179503 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc0a4f50 con 0x7fbecc135ea0 2024-01-24T23:26:43.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 10 monclient: _finish_auth 0 2024-01-24T23:26:43.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:26:43.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:43.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 1 -- 172.21.15.132:0/2469179503 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbeb40017c0 con 0x7fbecc135ea0 2024-01-24T23:26:43.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 5 monclient: authenticate success, global_id 6366 2024-01-24T23:26:43.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:43.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:43.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:43.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 10 monclient: dump: 2024-01-24T23:26:43.371 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:43.371 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:43.371 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:43.371 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:43.371 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:43.372 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:43.372 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:43.372 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:43.372 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:43.372 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:43.372 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:43.372 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:43.372 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:43.372 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:43.372 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:43.372 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:43.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:43.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 1 -- 172.21.15.132:0/2469179503 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbeb4001ae0 con 0x7fbecc135ea0 2024-01-24T23:26:43.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:43.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:43.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 4 set_mon_vals no callback set 2024-01-24T23:26:43.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 1 -- 172.21.15.132:0/2469179503 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbeb4004bf0 con 0x7fbecc135ea0 2024-01-24T23:26:43.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:43.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:43.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbec9ffb640 10 monclient: dump: 2024-01-24T23:26:43.373 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:43.373 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:43.373 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:43.373 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:43.373 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:43.373 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:43.373 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:43.373 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:43.373 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:43.374 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:43.374 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:43.374 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:43.374 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:43.374 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:43.374 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:43.374 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:43.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 10 monclient: shutdown 2024-01-24T23:26:43.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:43.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 1 -- 172.21.15.132:0/2469179503 >> v1:172.21.15.132:6790/0 conn(0x7fbecc135ea0 legacy=0x7fbecc136280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:43.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 1 -- 172.21.15.132:0/2469179503 shutdown_connections 2024-01-24T23:26:43.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 1 -- 172.21.15.132:0/2469179503 >> 172.21.15.132:0/2469179503 conn(0x7fbecc0a9bf0 msgr2=0x7fbecc0a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:43.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 1 -- 172.21.15.132:0/2469179503 shutdown_connections 2024-01-24T23:26:43.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 1 -- 172.21.15.132:0/2469179503 wait complete. 2024-01-24T23:26:43.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 10 monclient: build_initial_monmap 2024-01-24T23:26:43.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.359+0000 7fbed1810640 10 monclient: monmap: 2024-01-24T23:26:43.375 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:43.375 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:43.375 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:43.375 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:43.375 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:43.375 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:43.375 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:43.375 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:43.375 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:43.375 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:43.375 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:43.375 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:43.375 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:43.375 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:43.376 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:43.376 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:43.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbed1810640 1 Processor -- start 2024-01-24T23:26:43.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbed1810640 1 -- start start 2024-01-24T23:26:43.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbed1810640 10 monclient: init 2024-01-24T23:26:43.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbed1810640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:43.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbed1810640 10 monclient: _add_conns ranks=[4,8,6,7,0,5,2,1,3] 2024-01-24T23:26:43.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbed1810640 10 monclient(hunting): picked mon.noname-c con 0x7fbecc09bcd0 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:43.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbed1810640 10 monclient(hunting): picked mon.noname-e con 0x7fbecc09c0f0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:43.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbed1810640 10 monclient(hunting): picked mon.noname-d con 0x7fbecc135ea0 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:43.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbed1810640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbecc09c6b0 con 0x7fbecc09bcd0 2024-01-24T23:26:43.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbed1810640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbecc1251a0 con 0x7fbecc135ea0 2024-01-24T23:26:43.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbed1810640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbecc127500 con 0x7fbecc09c0f0 2024-01-24T23:26:43.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbed1810640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:43.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbed1810640 10 monclient(hunting): authenticate will time out at 1290.688791s 2024-01-24T23:26:43.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbecaffd640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fbecc09bcd0 0x7fbecc134af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:53986/0 (socket says 172.21.15.132:53986) 2024-01-24T23:26:43.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbecaffd640 1 -- 172.21.15.132:0/833341953 learned_addr learned my addr 172.21.15.132:0/833341953 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:43.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 929839686 0 0) 0x7fbecc09c6b0 con 0x7fbecc09bcd0 2024-01-24T23:26:43.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:26:43.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:43.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:43.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient(hunting): my global_id is 94576 2024-01-24T23:26:43.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbeac003530 con 0x7fbecc09bcd0 2024-01-24T23:26:43.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2301153595 0 0) 0x7fbecc127500 con 0x7fbecc09c0f0 2024-01-24T23:26:43.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:26:43.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:43.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:43.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient(hunting): my global_id is 85625 2024-01-24T23:26:43.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbecc09c6b0 con 0x7fbecc09c0f0 2024-01-24T23:26:43.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593710386 0 0) 0x7fbecc1251a0 con 0x7fbecc135ea0 2024-01-24T23:26:43.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:26:43.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:43.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:43.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient(hunting): my global_id is 6477 2024-01-24T23:26:43.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbecc127500 con 0x7fbecc135ea0 2024-01-24T23:26:43.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3595170774 0 0) 0x7fbeac003530 con 0x7fbecc09bcd0 2024-01-24T23:26:43.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbecc1251a0 con 0x7fbecc09bcd0 2024-01-24T23:26:43.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1467598144 0 0) 0x7fbecc09c6b0 con 0x7fbecc09c0f0 2024-01-24T23:26:43.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbeac003530 con 0x7fbecc09c0f0 2024-01-24T23:26:43.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbeb4001780 con 0x7fbecc09bcd0 2024-01-24T23:26:43.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:43.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:26:43.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient(hunting): dump: 2024-01-24T23:26:43.378 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:43.379 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:43.379 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:43.379 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:43.379 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:43.379 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:43.379 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:43.379 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:43.379 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:43.379 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:43.379 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:43.379 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:43.379 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:43.379 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:43.379 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:43.379 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:43.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:43.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1231210318 0 0) 0x7fbecc127500 con 0x7fbecc135ea0 2024-01-24T23:26:43.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbecc09c6b0 con 0x7fbecc135ea0 2024-01-24T23:26:43.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbebc0052d0 con 0x7fbecc09c0f0 2024-01-24T23:26:43.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:43.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:26:43.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient(hunting): dump: 2024-01-24T23:26:43.380 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:43.380 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:43.380 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:43.380 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:43.380 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:43.380 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:43.381 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:43.381 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:43.381 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:43.381 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:43.381 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:43.381 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:43.381 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:43.381 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:43.381 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:43.381 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:43.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbec4004560 con 0x7fbecc135ea0 2024-01-24T23:26:43.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:43.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:26:43.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient(hunting): dump: 2024-01-24T23:26:43.381 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:43.382 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:43.382 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:43.382 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:43.382 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:43.382 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:43.382 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:43.382 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:43.382 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:43.382 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:43.382 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:43.382 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:43.382 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:43.382 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:43.382 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:43.383 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:43.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1744168533 0 0) 0x7fbecc1251a0 con 0x7fbecc09bcd0 2024-01-24T23:26:43.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 >> v1:172.21.15.115:6793/0 conn(0x7fbecc09c0f0 legacy=0x7fbecc1d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:43.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 >> v1:172.21.15.115:6792/0 conn(0x7fbecc135ea0 legacy=0x7fbecc1d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:43.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient: _finish_hunting 0 2024-01-24T23:26:43.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 1 monclient: found mon.c 2024-01-24T23:26:43.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:43.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:43.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc135c70 con 0x7fbecc09bcd0 2024-01-24T23:26:43.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient: _finish_auth 0 2024-01-24T23:26:43.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient: _check_auth_tickets 2024-01-24T23:26:43.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:43.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbed1810640 5 monclient: authenticate success, global_id 94576 2024-01-24T23:26:43.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbed1810640 10 monclient: _renew_subs 2024-01-24T23:26:43.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbed1810640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:43.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbed1810640 1 -- 172.21.15.132:0/833341953 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbecc127220 con 0x7fbecc09bcd0 2024-01-24T23:26:43.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbed1810640 10 monclient: _renew_subs 2024-01-24T23:26:43.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbed1810640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:43.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbed1810640 1 -- 172.21.15.132:0/833341953 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbecc135200 con 0x7fbecc09bcd0 2024-01-24T23:26:43.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbeb4001c50 con 0x7fbecc09bcd0 2024-01-24T23:26:43.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:43.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbeb4005410 con 0x7fbecc09bcd0 2024-01-24T23:26:43.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:43.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec97fa640 4 set_mon_vals no callback set 2024-01-24T23:26:43.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:26:43.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbec3fff640 10 monclient: dump: 2024-01-24T23:26:43.384 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:43.384 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:43.384 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:43.385 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:43.385 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:43.385 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:43.385 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:43.385 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:43.385 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:43.385 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:43.385 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:43.385 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:43.385 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:43.385 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:43.385 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:43.385 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:43.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbed1810640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:43.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbed1810640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:43.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbed1810640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:43.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.363+0000 7fbed1810640 1 -- 172.21.15.132:0/833341953 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb0005090 con 0x7fbecc09bcd0 2024-01-24T23:26:43.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.367+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fbeb400fe90 con 0x7fbecc09bcd0 2024-01-24T23:26:43.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.367+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(185..185 src has 1..185) v4 ==== 8593+0+0 (unknown 3721864458 0 0) 0x7fbeb4044cd0 con 0x7fbecc09bcd0 2024-01-24T23:26:43.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.367+0000 7fbeca7fc640 10 monclient: get_auth_request con 0x7fbeac037b50 auth_method 0 2024-01-24T23:26:43.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.379+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fbeb4014ab0 con 0x7fbecc09bcd0 2024-01-24T23:26:43.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.379+0000 7fbec3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:43.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.379+0000 7fbec3fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:43.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.535+0000 7fbed1810640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:43.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.535+0000 7fbed1810640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:43.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.535+0000 7fbed1810640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:43.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:43.535+0000 7fbed1810640 1 -- 172.21.15.132:0/833341953 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fbeb0005340 con 0x7fbecc09bcd0 2024-01-24T23:26:44.142 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.139+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}]=0 added bucket h7 type host to location {rack=r1} v186) v1 ==== 166+0+0 (unknown 1787324382 0 0) 0x7fbeb40148d0 con 0x7fbecc09bcd0 2024-01-24T23:26:44.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.139+0000 7fbec3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:44.143 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.139+0000 7fbec3fff640 10 monclient: _finish_command 2 = system:0 added bucket h7 type host to location {rack=r1} 2024-01-24T23:26:44.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.239+0000 7fbed1810640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:44.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.239+0000 7fbed1810640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:44.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.239+0000 7fbed1810640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:26:44.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.239+0000 7fbed1810640 1 -- 172.21.15.132:0/833341953 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fbeb0005a80 con 0x7fbecc09bcd0 2024-01-24T23:26:44.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.243+0000 7fbec3fff640 1 -- 172.21.15.132:0/833341953 <== mon.4 v1:172.21.15.115:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h7' already exists v186) v1 ==== 145+0+0 (unknown 682076802 0 0) 0x7fbeb4014f10 con 0x7fbecc09bcd0 2024-01-24T23:26:44.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.243+0000 7fbec3fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:44.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.243+0000 7fbec3fff640 10 monclient: _finish_command 3 = system:0 bucket 'h7' already exists 2024-01-24T23:26:44.244 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h7' already exists 2024-01-24T23:26:44.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.243+0000 7fbed1810640 1 -- 172.21.15.132:0/833341953 >> v1:172.21.15.132:6812/1835401922 conn(0x7fbeac037b50 legacy=0x7fbeac03a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:44.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.243+0000 7fbed1810640 10 monclient: shutdown 2024-01-24T23:26:44.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.243+0000 7fbed1810640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:44.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.243+0000 7fbed1810640 1 -- 172.21.15.132:0/833341953 >> v1:172.21.15.115:6791/0 conn(0x7fbecc09bcd0 legacy=0x7fbecc134af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:44.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.243+0000 7fbed1810640 1 -- 172.21.15.132:0/833341953 shutdown_connections 2024-01-24T23:26:44.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.243+0000 7fbed1810640 1 -- 172.21.15.132:0/833341953 >> 172.21.15.132:0/833341953 conn(0x7fbecc0a9bf0 msgr2=0x7fbecc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:44.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.243+0000 7fbed1810640 1 -- 172.21.15.132:0/833341953 shutdown_connections 2024-01-24T23:26:44.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.243+0000 7fbed1810640 1 -- 172.21.15.132:0/833341953 wait complete. 2024-01-24T23:26:44.261 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:26:44.261 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h8 host rack=r1 2024-01-24T23:26:44.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd711b88640 10 monclient: get_monmap_and_config 2024-01-24T23:26:44.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd711b88640 10 monclient: build_initial_monmap 2024-01-24T23:26:44.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd711b88640 10 monclient: monmap: 2024-01-24T23:26:44.384 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:44.384 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:44.384 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:44.385124+0000 2024-01-24T23:26:44.384 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:44.385124+0000 2024-01-24T23:26:44.384 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:44.384 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:44.384 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:44.384 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:44.385 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:44.385 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:44.385 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:44.385 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:44.385 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:44.385 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:44.385 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:44.385 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:44.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd711b88640 1 Processor -- start 2024-01-24T23:26:44.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd711b88640 1 -- start start 2024-01-24T23:26:44.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd711b88640 10 monclient: init 2024-01-24T23:26:44.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd711b88640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:44.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd711b88640 10 monclient: _add_conns ranks=[0,7,6,2,4,1,8,5,3] 2024-01-24T23:26:44.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd711b88640 10 monclient(hunting): picked mon.noname-a con 0x7fd70c134af0 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:44.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd711b88640 10 monclient(hunting): picked mon.noname-i con 0x7fd70c071e00 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:44.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd711b88640 10 monclient(hunting): picked mon.noname-d con 0x7fd70c13b2d0 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:44.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd711b88640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd70c064700 con 0x7fd70c134af0 2024-01-24T23:26:44.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd711b88640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd70c06fef0 con 0x7fd70c13b2d0 2024-01-24T23:26:44.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd711b88640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd70c070b30 con 0x7fd70c071e00 2024-01-24T23:26:44.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd711b88640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:44.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd711b88640 10 monclient(hunting): authenticate will time out at 1291.709986s 2024-01-24T23:26:44.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70affd640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fd70c071e00 legacy=0x7fd70c139bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:44.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70bfff640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fd70c13b2d0 0x7fd70c13d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:47650/0 (socket says 172.21.15.132:47650) 2024-01-24T23:26:44.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70bfff640 1 -- 172.21.15.132:0/1342366503 learned_addr learned my addr 172.21.15.132:0/1342366503 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:44.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 1 -- 172.21.15.132:0/1342366503 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 45695183 0 0) 0x7fd70c064700 con 0x7fd70c134af0 2024-01-24T23:26:44.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:26:44.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:44.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:44.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 10 monclient(hunting): my global_id is 94734 2024-01-24T23:26:44.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 1 -- 172.21.15.132:0/1342366503 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6f4003530 con 0x7fd70c134af0 2024-01-24T23:26:44.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 1 -- 172.21.15.132:0/1342366503 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3864767731 0 0) 0x7fd70c06fef0 con 0x7fd70c13b2d0 2024-01-24T23:26:44.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:26:44.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:44.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:44.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 10 monclient(hunting): my global_id is 6486 2024-01-24T23:26:44.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 1 -- 172.21.15.132:0/1342366503 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd70c064700 con 0x7fd70c13b2d0 2024-01-24T23:26:44.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 1 -- 172.21.15.132:0/1342366503 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2371083727 0 0) 0x7fd70c064700 con 0x7fd70c13b2d0 2024-01-24T23:26:44.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 1 -- 172.21.15.132:0/1342366503 >> v1:172.21.15.132:6792/0 conn(0x7fd70c071e00 legacy=0x7fd70c139bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:44.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 1 -- 172.21.15.132:0/1342366503 >> v1:172.21.15.115:6789/0 conn(0x7fd70c134af0 legacy=0x7fd70c134ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:44.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:26:44.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 1 monclient: found mon.noname-d 2024-01-24T23:26:44.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:44.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:26:44.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 1 -- 172.21.15.132:0/1342366503 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70c0710f0 con 0x7fd70c13b2d0 2024-01-24T23:26:44.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 10 monclient: _finish_auth 0 2024-01-24T23:26:44.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:26:44.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:44.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 1 -- 172.21.15.132:0/1342366503 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd6fc001650 con 0x7fd70c13b2d0 2024-01-24T23:26:44.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:44.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:26:44.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 10 monclient: dump: 2024-01-24T23:26:44.392 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:44.392 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:44.393 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:44.393 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:44.393 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:44.393 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:44.393 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:44.393 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:44.393 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:44.393 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:44.393 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:44.393 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:44.393 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:44.393 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:44.394 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:44.394 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:44.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd711b88640 5 monclient: authenticate success, global_id 6486 2024-01-24T23:26:44.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd711b88640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:44.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 1 -- 172.21.15.132:0/1342366503 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd6fc0018f0 con 0x7fd70c13b2d0 2024-01-24T23:26:44.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:44.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 1 -- 172.21.15.132:0/1342366503 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd6fc0048b0 con 0x7fd70c13b2d0 2024-01-24T23:26:44.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd711b88640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:44.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd711b88640 4 set_mon_vals no callback set 2024-01-24T23:26:44.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:44.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:26:44.395 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.383+0000 7fd70a7fc640 10 monclient: dump: 2024-01-24T23:26:44.395 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:44.395 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:44.395 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:44.395 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:44.395 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:44.395 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:44.395 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:44.395 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:44.395 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:44.396 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:44.396 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:44.396 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:44.396 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:44.396 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:44.396 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:44.396 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:44.396 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 10 monclient: shutdown 2024-01-24T23:26:44.396 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:44.396 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 1 -- 172.21.15.132:0/1342366503 >> v1:172.21.15.115:6792/0 conn(0x7fd70c13b2d0 legacy=0x7fd70c13d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:44.396 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 1 -- 172.21.15.132:0/1342366503 shutdown_connections 2024-01-24T23:26:44.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 1 -- 172.21.15.132:0/1342366503 >> 172.21.15.132:0/1342366503 conn(0x7fd70c0a5940 msgr2=0x7fd70c0a5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:44.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 1 -- 172.21.15.132:0/1342366503 shutdown_connections 2024-01-24T23:26:44.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 1 -- 172.21.15.132:0/1342366503 wait complete. 2024-01-24T23:26:44.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 10 monclient: build_initial_monmap 2024-01-24T23:26:44.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 10 monclient: monmap: 2024-01-24T23:26:44.397 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:44.397 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:44.397 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:44.397 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:44.397 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:44.397 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:44.397 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:44.398 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:44.398 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:44.398 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:44.398 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:44.398 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:44.398 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:44.398 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:44.398 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:44.398 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:44.398 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 1 Processor -- start 2024-01-24T23:26:44.398 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 1 -- start start 2024-01-24T23:26:44.398 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 10 monclient: init 2024-01-24T23:26:44.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:44.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 10 monclient: _add_conns ranks=[3,5,0,6,1,4,8,2,7] 2024-01-24T23:26:44.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 10 monclient(hunting): picked mon.noname-g con 0x7fd70c134af0 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:44.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 10 monclient(hunting): picked mon.noname-h con 0x7fd70c13b2d0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:44.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 10 monclient(hunting): picked mon.noname-a con 0x7fd70c12aad0 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:44.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd70c12af10 con 0x7fd70c12aad0 2024-01-24T23:26:44.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd70c09e400 con 0x7fd70c134af0 2024-01-24T23:26:44.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd70c09e6e0 con 0x7fd70c13b2d0 2024-01-24T23:26:44.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:44.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 10 monclient(hunting): authenticate will time out at 1291.714329s 2024-01-24T23:26:44.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd70b7fe640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fd70c134af0 0x7fd70c12a3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:56186/0 (socket says 172.21.15.132:56186) 2024-01-24T23:26:44.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd70b7fe640 1 -- 172.21.15.132:0/1056399592 learned_addr learned my addr 172.21.15.132:0/1056399592 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:44.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 1 -- 172.21.15.132:0/1056399592 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3297340766 0 0) 0x7fd70c09e6e0 con 0x7fd70c13b2d0 2024-01-24T23:26:44.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:26:44.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:44.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:44.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient(hunting): my global_id is 6269 2024-01-24T23:26:44.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 1 -- 172.21.15.132:0/1056399592 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6f8003530 con 0x7fd70c13b2d0 2024-01-24T23:26:44.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 1 -- 172.21.15.132:0/1056399592 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 466420076 0 0) 0x7fd70c09e400 con 0x7fd70c134af0 2024-01-24T23:26:44.404 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:26:44.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:44.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:44.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient(hunting): my global_id is 6375 2024-01-24T23:26:44.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 1 -- 172.21.15.132:0/1056399592 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd70c09e6e0 con 0x7fd70c134af0 2024-01-24T23:26:44.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 1 -- 172.21.15.132:0/1056399592 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 147418895 0 0) 0x7fd6f8003530 con 0x7fd70c13b2d0 2024-01-24T23:26:44.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 1 -- 172.21.15.132:0/1056399592 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd70c09e400 con 0x7fd70c13b2d0 2024-01-24T23:26:44.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 1 -- 172.21.15.132:0/1056399592 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd704004270 con 0x7fd70c13b2d0 2024-01-24T23:26:44.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:44.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:44.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient(hunting): dump: 2024-01-24T23:26:44.405 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:44.406 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:44.406 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:44.406 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:44.406 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:44.406 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:44.406 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:44.406 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:44.406 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:44.406 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:44.406 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:44.406 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:44.406 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:44.407 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:44.407 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:44.407 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:44.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:44.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 1 -- 172.21.15.132:0/1056399592 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 960469691 0 0) 0x7fd70c09e6e0 con 0x7fd70c134af0 2024-01-24T23:26:44.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 1 -- 172.21.15.132:0/1056399592 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd6f8003530 con 0x7fd70c134af0 2024-01-24T23:26:44.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 1 -- 172.21.15.132:0/1056399592 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd700001980 con 0x7fd70c134af0 2024-01-24T23:26:44.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:44.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:44.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient(hunting): dump: 2024-01-24T23:26:44.407 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:44.407 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:44.408 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:44.408 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:44.408 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:44.408 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:44.408 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:44.408 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:44.408 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:44.408 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:44.408 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:44.408 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:44.409 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:44.409 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:44.409 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:44.409 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:44.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 1 -- 172.21.15.132:0/1056399592 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2164030475 0 0) 0x7fd70c12af10 con 0x7fd70c12aad0 2024-01-24T23:26:44.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:26:44.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:44.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:44.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient(hunting): my global_id is 94743 2024-01-24T23:26:44.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 1 -- 172.21.15.132:0/1056399592 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd70c09e6e0 con 0x7fd70c12aad0 2024-01-24T23:26:44.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 1 -- 172.21.15.132:0/1056399592 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1763639038 0 0) 0x7fd70c09e400 con 0x7fd70c13b2d0 2024-01-24T23:26:44.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 1 -- 172.21.15.132:0/1056399592 >> v1:172.21.15.132:6790/0 conn(0x7fd70c134af0 legacy=0x7fd70c12a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:44.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 1 -- 172.21.15.132:0/1056399592 >> v1:172.21.15.115:6789/0 conn(0x7fd70c12aad0 legacy=0x7fd70c1d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:44.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient: _finish_hunting 0 2024-01-24T23:26:44.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 1 monclient: found mon.h 2024-01-24T23:26:44.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:44.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:44.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 1 -- 172.21.15.132:0/1056399592 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70c09e9c0 con 0x7fd70c13b2d0 2024-01-24T23:26:44.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient: _finish_auth 0 2024-01-24T23:26:44.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient: _check_auth_tickets 2024-01-24T23:26:44.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:44.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 5 monclient: authenticate success, global_id 6269 2024-01-24T23:26:44.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 10 monclient: _renew_subs 2024-01-24T23:26:44.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:44.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 1 -- 172.21.15.132:0/1056399592 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd70c09eca0 con 0x7fd70c13b2d0 2024-01-24T23:26:44.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 10 monclient: _renew_subs 2024-01-24T23:26:44.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:44.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd711b88640 1 -- 172.21.15.132:0/1056399592 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd70c09d510 con 0x7fd70c13b2d0 2024-01-24T23:26:44.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 1 -- 172.21.15.132:0/1056399592 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd704004510 con 0x7fd70c13b2d0 2024-01-24T23:26:44.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:44.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 1 -- 172.21.15.132:0/1056399592 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd704004a40 con 0x7fd70c13b2d0 2024-01-24T23:26:44.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:44.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd7097fa640 4 set_mon_vals no callback set 2024-01-24T23:26:44.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:44.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.387+0000 7fd6f3fff640 10 monclient: dump: 2024-01-24T23:26:44.411 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:44.411 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:44.411 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:44.411 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:44.411 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:44.411 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:44.412 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:44.412 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:44.412 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:44.412 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:44.412 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:44.412 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:44.412 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:44.412 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:44.412 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:44.412 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:44.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.391+0000 7fd711b88640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:44.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.391+0000 7fd711b88640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:44.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.391+0000 7fd711b88640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:44.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.391+0000 7fd711b88640 1 -- 172.21.15.132:0/1056399592 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ec005090 con 0x7fd70c13b2d0 2024-01-24T23:26:44.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.391+0000 7fd6f3fff640 1 -- 172.21.15.132:0/1056399592 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fd704004ba0 con 0x7fd70c13b2d0 2024-01-24T23:26:44.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.391+0000 7fd6f3fff640 1 -- 172.21.15.132:0/1056399592 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(186..186 src has 1..186) v4 ==== 8803+0+0 (unknown 3599912204 0 0) 0x7fd704044b70 con 0x7fd70c13b2d0 2024-01-24T23:26:44.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.391+0000 7fd70b7fe640 10 monclient: get_auth_request con 0x7fd6f8037770 auth_method 0 2024-01-24T23:26:44.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.395+0000 7fd6f3fff640 1 -- 172.21.15.132:0/1056399592 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fd704014a30 con 0x7fd70c13b2d0 2024-01-24T23:26:44.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.395+0000 7fd6f3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:44.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.395+0000 7fd6f3fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:44.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.551+0000 7fd711b88640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:44.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.551+0000 7fd711b88640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:44.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.551+0000 7fd711b88640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:44.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:44.551+0000 7fd711b88640 1 -- 172.21.15.132:0/1056399592 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fd6ec0058c0 con 0x7fd70c13b2d0 2024-01-24T23:26:45.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.135+0000 7fd6f3fff640 1 -- 172.21.15.132:0/1056399592 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}]=0 added bucket h8 type host to location {rack=r1} v187) v1 ==== 166+0+0 (unknown 3342635255 0 0) 0x7fd704012ea0 con 0x7fd70c13b2d0 2024-01-24T23:26:45.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.135+0000 7fd6f3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:45.135 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.135+0000 7fd6f3fff640 10 monclient: _finish_command 2 = system:0 added bucket h8 type host to location {rack=r1} 2024-01-24T23:26:45.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.235+0000 7fd711b88640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:45.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.235+0000 7fd711b88640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:45.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.235+0000 7fd711b88640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:45.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.235+0000 7fd711b88640 1 -- 172.21.15.132:0/1056399592 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fd6ec002220 con 0x7fd70c13b2d0 2024-01-24T23:26:45.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.235+0000 7fd6f3fff640 1 -- 172.21.15.132:0/1056399592 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h8' already exists v187) v1 ==== 145+0+0 (unknown 457866160 0 0) 0x7fd704014620 con 0x7fd70c13b2d0 2024-01-24T23:26:45.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.235+0000 7fd6f3fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:45.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.235+0000 7fd6f3fff640 10 monclient: _finish_command 3 = system:0 bucket 'h8' already exists 2024-01-24T23:26:45.239 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h8' already exists 2024-01-24T23:26:45.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.239+0000 7fd711b88640 1 -- 172.21.15.132:0/1056399592 >> v1:172.21.15.132:6812/1835401922 conn(0x7fd6f8037770 legacy=0x7fd6f8039c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:45.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.239+0000 7fd711b88640 10 monclient: shutdown 2024-01-24T23:26:45.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.239+0000 7fd711b88640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:45.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.239+0000 7fd711b88640 1 -- 172.21.15.132:0/1056399592 >> v1:172.21.15.132:6791/0 conn(0x7fd70c13b2d0 legacy=0x7fd70c1cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:45.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.239+0000 7fd711b88640 1 -- 172.21.15.132:0/1056399592 shutdown_connections 2024-01-24T23:26:45.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.239+0000 7fd711b88640 1 -- 172.21.15.132:0/1056399592 >> 172.21.15.132:0/1056399592 conn(0x7fd70c0a5940 msgr2=0x7fd70c12e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:45.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.239+0000 7fd711b88640 1 -- 172.21.15.132:0/1056399592 shutdown_connections 2024-01-24T23:26:45.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.239+0000 7fd711b88640 1 -- 172.21.15.132:0/1056399592 wait complete. 2024-01-24T23:26:45.255 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:26:45.255 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h9 host rack=r1 2024-01-24T23:26:45.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.367+0000 7f230ed12640 10 monclient: get_monmap_and_config 2024-01-24T23:26:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.367+0000 7f230ed12640 10 monclient: build_initial_monmap 2024-01-24T23:26:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.367+0000 7f230ed12640 10 monclient: monmap: 2024-01-24T23:26:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:45.372424+0000 2024-01-24T23:26:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:45.372424+0000 2024-01-24T23:26:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:45.371 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:45.372 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:45.372 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:45.372 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:45.372 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:45.372 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:45.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 1 Processor -- start 2024-01-24T23:26:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 1 -- start start 2024-01-24T23:26:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 10 monclient: init 2024-01-24T23:26:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 10 monclient: _add_conns ranks=[3,5,6,8,7,1,0,2,4] 2024-01-24T23:26:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 10 monclient(hunting): picked mon.noname-g con 0x7f230809b810 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 10 monclient(hunting): picked mon.noname-h con 0x7f230809bbf0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 10 monclient(hunting): picked mon.noname-d con 0x7f23081370a0 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2308064700 con 0x7f230809b810 2024-01-24T23:26:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2308099af0 con 0x7f230809bbf0 2024-01-24T23:26:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2308072110 con 0x7f23081370a0 2024-01-24T23:26:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:45.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 10 monclient(hunting): authenticate will time out at 1292.697355s 2024-01-24T23:26:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230caae640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f230809b810 0x7f23080712f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:56198/0 (socket says 172.21.15.132:56198) 2024-01-24T23:26:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230caae640 1 -- 172.21.15.132:0/1221954648 learned_addr learned my addr 172.21.15.132:0/1221954648 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f2307fff640 1 --1- 172.21.15.132:0/1221954648 >> v1:172.21.15.132:6791/0 conn(0x7f230809bbf0 0x7f2308071a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:55788/0 (socket says 172.21.15.132:55788) 2024-01-24T23:26:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 1 -- 172.21.15.132:0/1221954648 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4203570916 0 0) 0x7f2308064700 con 0x7f230809b810 2024-01-24T23:26:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 10 monclient(hunting): _negotiate 2024-01-24T23:26:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 10 monclient(hunting): my global_id is 6384 2024-01-24T23:26:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 1 -- 172.21.15.132:0/1221954648 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22f0003530 con 0x7f230809b810 2024-01-24T23:26:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 1 -- 172.21.15.132:0/1221954648 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3754475 0 0) 0x7f2308099af0 con 0x7f230809bbf0 2024-01-24T23:26:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 10 monclient(hunting): _negotiate 2024-01-24T23:26:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:45.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 10 monclient(hunting): my global_id is 6278 2024-01-24T23:26:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 1 -- 172.21.15.132:0/1221954648 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2308064700 con 0x7f230809bbf0 2024-01-24T23:26:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 1 -- 172.21.15.132:0/1221954648 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 770105227 0 0) 0x7f22f0003530 con 0x7f230809b810 2024-01-24T23:26:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 1 -- 172.21.15.132:0/1221954648 >> v1:172.21.15.115:6792/0 conn(0x7f23081370a0 legacy=0x7f230813ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 1 -- 172.21.15.132:0/1221954648 >> v1:172.21.15.132:6791/0 conn(0x7f230809bbf0 legacy=0x7f2308071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 10 monclient: _finish_hunting 0 2024-01-24T23:26:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 1 monclient: found mon.noname-g 2024-01-24T23:26:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:26:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 1 -- 172.21.15.132:0/1221954648 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230809c050 con 0x7f230809b810 2024-01-24T23:26:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 10 monclient: _finish_auth 0 2024-01-24T23:26:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 10 monclient: _check_auth_tickets 2024-01-24T23:26:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 1 -- 172.21.15.132:0/1221954648 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f22f80017c0 con 0x7f230809b810 2024-01-24T23:26:45.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 10 monclient: dump: 2024-01-24T23:26:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:45.378 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:45.379 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 5 monclient: authenticate success, global_id 6384 2024-01-24T23:26:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 1 -- 172.21.15.132:0/1221954648 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f22f8001970 con 0x7f230809b810 2024-01-24T23:26:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 1 -- 172.21.15.132:0/1221954648 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f22f80048b0 con 0x7f230809b810 2024-01-24T23:26:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f23077fe640 10 monclient: dump: 2024-01-24T23:26:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:45.379 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:45.380 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:45.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 4 set_mon_vals no callback set 2024-01-24T23:26:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 10 monclient: shutdown 2024-01-24T23:26:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 1 -- 172.21.15.132:0/1221954648 >> v1:172.21.15.132:6790/0 conn(0x7f230809b810 legacy=0x7f23080712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 1 -- 172.21.15.132:0/1221954648 shutdown_connections 2024-01-24T23:26:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 1 -- 172.21.15.132:0/1221954648 >> 172.21.15.132:0/1221954648 conn(0x7f23080a1bc0 msgr2=0x7f23080a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 1 -- 172.21.15.132:0/1221954648 shutdown_connections 2024-01-24T23:26:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 1 -- 172.21.15.132:0/1221954648 wait complete. 2024-01-24T23:26:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 10 monclient: build_initial_monmap 2024-01-24T23:26:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.371+0000 7f230ed12640 10 monclient: monmap: 2024-01-24T23:26:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:45.381 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:45.382 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:45.382 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:45.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230ed12640 1 Processor -- start 2024-01-24T23:26:45.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230ed12640 1 -- start start 2024-01-24T23:26:45.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230ed12640 10 monclient: init 2024-01-24T23:26:45.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230ed12640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:45.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230ed12640 10 monclient: _add_conns ranks=[2,1,5,8,3,0,6,7,4] 2024-01-24T23:26:45.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230ed12640 10 monclient(hunting): picked mon.noname-b con 0x7f230809b810 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:45.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230ed12640 10 monclient(hunting): picked mon.noname-f con 0x7f230809bbf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:45.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230ed12640 10 monclient(hunting): picked mon.noname-h con 0x7f23081370a0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:45.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230ed12640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f230812ae30 con 0x7f230809bbf0 2024-01-24T23:26:45.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230ed12640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f230809b650 con 0x7f230809b810 2024-01-24T23:26:45.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f2307fff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f230809bbf0 legacy=0x7f2308132370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:45.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230ed12640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2308134230 con 0x7f23081370a0 2024-01-24T23:26:45.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230ed12640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:45.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230ed12640 10 monclient(hunting): authenticate will time out at 1292.701177s 2024-01-24T23:26:45.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230d2af640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f23081370a0 0x7f2308132a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:55798/0 (socket says 172.21.15.132:55798) 2024-01-24T23:26:45.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230d2af640 1 -- 172.21.15.132:0/3997146213 learned_addr learned my addr 172.21.15.132:0/3997146213 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:45.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230caae640 1 -- 172.21.15.132:0/3997146213 >> v1:172.21.15.115:6790/0 conn(0x7f230809b810 legacy=0x7f2308131c60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:45.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 1 -- 172.21.15.132:0/3997146213 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1861593500 0 0) 0x7f2308134230 con 0x7f23081370a0 2024-01-24T23:26:45.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 10 monclient(hunting): _negotiate 2024-01-24T23:26:45.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:45.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:45.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 10 monclient(hunting): my global_id is 6287 2024-01-24T23:26:45.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 1 -- 172.21.15.132:0/3997146213 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22f4003530 con 0x7f23081370a0 2024-01-24T23:26:45.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 1 -- 172.21.15.132:0/3997146213 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1240366735 0 0) 0x7f22f4003530 con 0x7f23081370a0 2024-01-24T23:26:45.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 1 -- 172.21.15.132:0/3997146213 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2308134230 con 0x7f23081370a0 2024-01-24T23:26:45.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 1 -- 172.21.15.132:0/3997146213 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2300001e20 con 0x7f23081370a0 2024-01-24T23:26:45.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:45.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:45.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 10 monclient(hunting): dump: 2024-01-24T23:26:45.388 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:45.388 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:45.388 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:45.388 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:45.388 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:45.388 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:45.388 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:45.388 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:45.389 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:45.389 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:45.389 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:45.389 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:45.389 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:45.389 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:45.389 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:45.389 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:45.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:45.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 1 -- 172.21.15.132:0/3997146213 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1728011569 0 0) 0x7f2308134230 con 0x7f23081370a0 2024-01-24T23:26:45.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 1 -- 172.21.15.132:0/3997146213 >> v1:172.21.15.115:6790/0 conn(0x7f230809b810 legacy=0x7f2308131c60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:45.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 1 -- 172.21.15.132:0/3997146213 >> v1:172.21.15.132:6789/0 conn(0x7f230809bbf0 legacy=0x7f2308132370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:45.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 10 monclient: _finish_hunting 0 2024-01-24T23:26:45.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 1 monclient: found mon.h 2024-01-24T23:26:45.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:45.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:45.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 1 -- 172.21.15.132:0/3997146213 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230812ab50 con 0x7f23081370a0 2024-01-24T23:26:45.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 10 monclient: _finish_auth 0 2024-01-24T23:26:45.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 10 monclient: _check_auth_tickets 2024-01-24T23:26:45.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:45.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230ed12640 5 monclient: authenticate success, global_id 6287 2024-01-24T23:26:45.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230ed12640 10 monclient: _renew_subs 2024-01-24T23:26:45.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230ed12640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:45.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230ed12640 1 -- 172.21.15.132:0/3997146213 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2308134470 con 0x7f23081370a0 2024-01-24T23:26:45.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230ed12640 10 monclient: _renew_subs 2024-01-24T23:26:45.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230ed12640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:45.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230ed12640 1 -- 172.21.15.132:0/3997146213 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2308133960 con 0x7f23081370a0 2024-01-24T23:26:45.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 1 -- 172.21.15.132:0/3997146213 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2300003c50 con 0x7f23081370a0 2024-01-24T23:26:45.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:45.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 1 -- 172.21.15.132:0/3997146213 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2300004180 con 0x7f23081370a0 2024-01-24T23:26:45.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:45.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f2306ffd640 4 set_mon_vals no callback set 2024-01-24T23:26:45.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:45.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 10 monclient: dump: 2024-01-24T23:26:45.391 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:45.391 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:45.391 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:45.391 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:45.391 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:45.391 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:45.391 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:45.391 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:45.391 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:45.391 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:45.392 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:45.392 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:45.392 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:45.392 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:45.392 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:45.392 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:45.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230ed12640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:45.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 1 -- 172.21.15.132:0/3997146213 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f23000042e0 con 0x7f23081370a0 2024-01-24T23:26:45.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230ed12640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:45.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230ed12640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:45.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230ed12640 1 -- 172.21.15.132:0/3997146213 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e8005090 con 0x7f23081370a0 2024-01-24T23:26:45.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f23057fa640 1 -- 172.21.15.132:0/3997146213 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(187..187 src has 1..187) v4 ==== 9013+0+0 (unknown 4045893709 0 0) 0x7f2300045090 con 0x7f23081370a0 2024-01-24T23:26:45.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.375+0000 7f230caae640 10 monclient: get_auth_request con 0x7f22f4036da0 auth_method 0 2024-01-24T23:26:45.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.379+0000 7f23057fa640 1 -- 172.21.15.132:0/3997146213 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f2300014c50 con 0x7f23081370a0 2024-01-24T23:26:45.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.379+0000 7f23057fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:45.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.379+0000 7f23057fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:45.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.539+0000 7f230ed12640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:45.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.539+0000 7f230ed12640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:45.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.539+0000 7f230ed12640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:45.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:45.539+0000 7f230ed12640 1 -- 172.21.15.132:0/3997146213 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f22e80058c0 con 0x7f23081370a0 2024-01-24T23:26:46.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.159+0000 7f23057fa640 1 -- 172.21.15.132:0/3997146213 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}]=0 added bucket h9 type host to location {rack=r1} v188) v1 ==== 166+0+0 (unknown 3403138783 0 0) 0x7f23000132f0 con 0x7f23081370a0 2024-01-24T23:26:46.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.159+0000 7f23057fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:46.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.159+0000 7f23057fa640 10 monclient: _finish_command 2 = system:0 added bucket h9 type host to location {rack=r1} 2024-01-24T23:26:46.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.263+0000 7f230ed12640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:46.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.263+0000 7f230ed12640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:46.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.263+0000 7f230ed12640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:46.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.263+0000 7f230ed12640 1 -- 172.21.15.132:0/3997146213 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f22e8002220 con 0x7f23081370a0 2024-01-24T23:26:46.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.263+0000 7f23057fa640 1 -- 172.21.15.132:0/3997146213 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h9' already exists v188) v1 ==== 145+0+0 (unknown 3909167633 0 0) 0x7f2300014a70 con 0x7f23081370a0 2024-01-24T23:26:46.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.263+0000 7f23057fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:46.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.263+0000 7f23057fa640 10 monclient: _finish_command 3 = system:0 bucket 'h9' already exists 2024-01-24T23:26:46.265 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h9' already exists 2024-01-24T23:26:46.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.263+0000 7f230ed12640 1 -- 172.21.15.132:0/3997146213 >> v1:172.21.15.132:6812/1835401922 conn(0x7f22f4036da0 legacy=0x7f22f4039260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:46.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.263+0000 7f230ed12640 10 monclient: shutdown 2024-01-24T23:26:46.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.263+0000 7f230ed12640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:46.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.263+0000 7f230ed12640 1 -- 172.21.15.132:0/3997146213 >> v1:172.21.15.132:6791/0 conn(0x7f23081370a0 legacy=0x7f2308132a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:46.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.263+0000 7f230ed12640 1 -- 172.21.15.132:0/3997146213 shutdown_connections 2024-01-24T23:26:46.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.263+0000 7f230ed12640 1 -- 172.21.15.132:0/3997146213 >> 172.21.15.132:0/3997146213 conn(0x7f23080a1bc0 msgr2=0x7f23080a3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:46.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.267+0000 7f230ed12640 1 -- 172.21.15.132:0/3997146213 shutdown_connections 2024-01-24T23:26:46.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.267+0000 7f230ed12640 1 -- 172.21.15.132:0/3997146213 wait complete. 2024-01-24T23:26:46.282 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:26:46.282 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h10 host rack=r1 2024-01-24T23:26:46.400 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcafe0ea640 10 monclient: get_monmap_and_config 2024-01-24T23:26:46.400 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcafe0ea640 10 monclient: build_initial_monmap 2024-01-24T23:26:46.400 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcafe0ea640 10 monclient: monmap: 2024-01-24T23:26:46.401 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:46.401 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:46.401 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:46.402140+0000 2024-01-24T23:26:46.401 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:46.402140+0000 2024-01-24T23:26:46.401 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:46.401 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:46.401 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:46.401 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:46.401 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:46.401 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:46.401 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:46.401 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:46.401 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:46.401 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:46.401 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:46.402 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:46.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcafe0ea640 1 Processor -- start 2024-01-24T23:26:46.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcafe0ea640 1 -- start start 2024-01-24T23:26:46.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcafe0ea640 10 monclient: init 2024-01-24T23:26:46.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcafe0ea640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:46.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcafe0ea640 10 monclient: _add_conns ranks=[3,6,8,5,4,0,2,1,7] 2024-01-24T23:26:46.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcafe0ea640 10 monclient(hunting): picked mon.noname-g con 0x7fcaf8072690 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:46.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcafe0ea640 10 monclient(hunting): picked mon.noname-d con 0x7fcaf809ae90 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:46.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcafe0ea640 10 monclient(hunting): picked mon.noname-e con 0x7fcaf809bdc0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:46.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcafe0ea640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf8064700 con 0x7fcaf8072690 2024-01-24T23:26:46.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcafe0ea640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf809a790 con 0x7fcaf809ae90 2024-01-24T23:26:46.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcafe0ea640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf80a18f0 con 0x7fcaf809bdc0 2024-01-24T23:26:46.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcafe0ea640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:46.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcafe0ea640 10 monclient(hunting): authenticate will time out at 1293.726717s 2024-01-24T23:26:46.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf77fe640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fcaf8072690 0x7fcaf80a0ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:56206/0 (socket says 172.21.15.132:56206) 2024-01-24T23:26:46.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf77fe640 1 -- 172.21.15.132:0/3171647057 learned_addr learned my addr 172.21.15.132:0/3171647057 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:46.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 1 -- 172.21.15.132:0/3171647057 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3634724808 0 0) 0x7fcaf8064700 con 0x7fcaf8072690 2024-01-24T23:26:46.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:46.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:46.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:46.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 10 monclient(hunting): my global_id is 6393 2024-01-24T23:26:46.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 1 -- 172.21.15.132:0/3171647057 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcae4003530 con 0x7fcaf8072690 2024-01-24T23:26:46.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf7fff640 1 -- 172.21.15.132:0/3171647057 >> v1:172.21.15.115:6793/0 conn(0x7fcaf809bdc0 legacy=0x7fcaf813d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:26:46.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf7fff640 1 -- 172.21.15.132:0/3171647057 >> v1:172.21.15.115:6793/0 conn(0x7fcaf809bdc0 legacy=0x7fcaf813d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:26:46.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf7fff640 1 --1- 172.21.15.132:0/3171647057 >> v1:172.21.15.115:6793/0 conn(0x7fcaf809bdc0 0x7fcaf813d280 :-1 s=OPENED pgs=314 cs=1 l=1).handle_message read tag failed 2024-01-24T23:26:46.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf7fff640 1 --1- 172.21.15.132:0/3171647057 >> v1:172.21.15.115:6793/0 conn(0x7fcaf809bdc0 0x7fcaf813d280 :-1 s=OPENED pgs=314 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:26:46.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6793/0 2024-01-24T23:26:46.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 1 -- 172.21.15.132:0/3171647057 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3285991739 0 0) 0x7fcae4003530 con 0x7fcaf8072690 2024-01-24T23:26:46.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 1 -- 172.21.15.132:0/3171647057 >> v1:172.21.15.115:6793/0 conn(0x7fcaf809bdc0 legacy=0x7fcaf813d280 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:26:46.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 1 -- 172.21.15.132:0/3171647057 >> v1:172.21.15.115:6792/0 conn(0x7fcaf809ae90 legacy=0x7fcaf80a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:46.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:26:46.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 1 monclient: found mon.noname-g 2024-01-24T23:26:46.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:46.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:26:46.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 1 -- 172.21.15.132:0/3171647057 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf80997a0 con 0x7fcaf8072690 2024-01-24T23:26:46.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 10 monclient: _finish_auth 0 2024-01-24T23:26:46.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:26:46.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:46.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 1 -- 172.21.15.132:0/3171647057 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fcae80017c0 con 0x7fcaf8072690 2024-01-24T23:26:46.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcafe0ea640 5 monclient: authenticate success, global_id 6393 2024-01-24T23:26:46.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcafe0ea640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:46.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:46.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:46.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 10 monclient: dump: 2024-01-24T23:26:46.411 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:46.411 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:46.411 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:46.411 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:46.411 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:46.411 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:46.411 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:46.411 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:46.411 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:46.411 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:46.411 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:46.411 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:46.411 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:46.412 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:46.412 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:46.412 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:46.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcafe0ea640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:46.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 1 -- 172.21.15.132:0/3171647057 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fcae8001a30 con 0x7fcaf8072690 2024-01-24T23:26:46.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:46.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 1 -- 172.21.15.132:0/3171647057 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fcae8004bf0 con 0x7fcaf8072690 2024-01-24T23:26:46.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:46.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:46.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcaf6ffd640 10 monclient: dump: 2024-01-24T23:26:46.412 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:46.412 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:46.412 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:46.413 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:46.413 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:46.413 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:46.413 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:46.413 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:46.413 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:46.413 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:46.413 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:46.413 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:46.413 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:46.413 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:46.413 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:46.413 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:46.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcafe0ea640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:46.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.399+0000 7fcafe0ea640 4 set_mon_vals no callback set 2024-01-24T23:26:46.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 10 monclient: shutdown 2024-01-24T23:26:46.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:46.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 1 -- 172.21.15.132:0/3171647057 >> v1:172.21.15.132:6790/0 conn(0x7fcaf8072690 legacy=0x7fcaf80a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:46.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 1 -- 172.21.15.132:0/3171647057 shutdown_connections 2024-01-24T23:26:46.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 1 -- 172.21.15.132:0/3171647057 >> 172.21.15.132:0/3171647057 conn(0x7fcaf80a1bc0 msgr2=0x7fcaf80a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:46.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 1 -- 172.21.15.132:0/3171647057 shutdown_connections 2024-01-24T23:26:46.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 1 -- 172.21.15.132:0/3171647057 wait complete. 2024-01-24T23:26:46.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 10 monclient: build_initial_monmap 2024-01-24T23:26:46.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 10 monclient: monmap: 2024-01-24T23:26:46.414 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:46.414 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:46.414 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:46.414 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:46.415 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:46.415 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:46.415 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:46.415 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:46.415 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:46.415 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:46.415 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:46.415 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:46.415 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:46.415 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:46.415 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:46.415 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:46.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 1 Processor -- start 2024-01-24T23:26:46.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 1 -- start start 2024-01-24T23:26:46.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 10 monclient: init 2024-01-24T23:26:46.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:46.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 10 monclient: _add_conns ranks=[5,3,1,7,6,8,0,2,4] 2024-01-24T23:26:46.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 10 monclient(hunting): picked mon.noname-h con 0x7fcaf8072690 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:46.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 10 monclient(hunting): picked mon.noname-g con 0x7fcaf809ae90 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:46.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 10 monclient(hunting): picked mon.noname-f con 0x7fcaf809bdc0 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:46.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf81b9130 con 0x7fcaf809bdc0 2024-01-24T23:26:46.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf81b9380 con 0x7fcaf809ae90 2024-01-24T23:26:46.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf81c84a0 con 0x7fcaf8072690 2024-01-24T23:26:46.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:46.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 10 monclient(hunting): authenticate will time out at 1293.729828s 2024-01-24T23:26:46.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf7fff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fcaf809bdc0 legacy=0x7fcaf81d4250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:46.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf77fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fcaf8072690 0x7fcaf81ba8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:55806/0 (socket says 172.21.15.132:55806) 2024-01-24T23:26:46.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf77fe640 1 -- 172.21.15.132:0/175568329 learned_addr learned my addr 172.21.15.132:0/175568329 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:46.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaeeffd640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fcaf809ae90 0x7fcaf81d0b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:56212/0 (socket says 172.21.15.132:56212) 2024-01-24T23:26:46.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 1 -- 172.21.15.132:0/175568329 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 931896185 0 0) 0x7fcaf81b9380 con 0x7fcaf809ae90 2024-01-24T23:26:46.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:26:46.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:46.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:46.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 10 monclient(hunting): my global_id is 6402 2024-01-24T23:26:46.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 1 -- 172.21.15.132:0/175568329 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcad8003530 con 0x7fcaf809ae90 2024-01-24T23:26:46.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 1 -- 172.21.15.132:0/175568329 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3340086081 0 0) 0x7fcaf81c84a0 con 0x7fcaf8072690 2024-01-24T23:26:46.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:26:46.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:46.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:46.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 10 monclient(hunting): my global_id is 6296 2024-01-24T23:26:46.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 1 -- 172.21.15.132:0/175568329 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaf81b9380 con 0x7fcaf8072690 2024-01-24T23:26:46.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 1 -- 172.21.15.132:0/175568329 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4234101921 0 0) 0x7fcaf81b9380 con 0x7fcaf8072690 2024-01-24T23:26:46.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 1 -- 172.21.15.132:0/175568329 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcaf81c84a0 con 0x7fcaf8072690 2024-01-24T23:26:46.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 1 -- 172.21.15.132:0/175568329 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3806384410 0 0) 0x7fcad8003530 con 0x7fcaf809ae90 2024-01-24T23:26:46.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 1 -- 172.21.15.132:0/175568329 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcaf81b9380 con 0x7fcaf809ae90 2024-01-24T23:26:46.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 1 -- 172.21.15.132:0/175568329 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fcae80047f0 con 0x7fcaf8072690 2024-01-24T23:26:46.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:46.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:46.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 10 monclient(hunting): dump: 2024-01-24T23:26:46.418 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:46.418 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:46.418 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:46.418 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:46.418 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:46.418 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:46.419 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:46.419 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:46.419 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:46.419 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:46.419 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:46.419 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:46.419 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:46.419 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:46.419 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:46.419 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:46.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:46.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 1 -- 172.21.15.132:0/175568329 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fcae00019d0 con 0x7fcaf809ae90 2024-01-24T23:26:46.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:46.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:46.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 10 monclient(hunting): dump: 2024-01-24T23:26:46.420 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:46.420 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:46.420 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:46.420 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:46.420 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:46.420 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:46.420 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:46.420 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:46.420 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:46.420 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:46.420 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:46.420 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:46.420 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:46.420 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:46.420 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:46.421 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:46.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 1 -- 172.21.15.132:0/175568329 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2473839792 0 0) 0x7fcaf81c84a0 con 0x7fcaf8072690 2024-01-24T23:26:46.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 1 -- 172.21.15.132:0/175568329 >> v1:172.21.15.132:6790/0 conn(0x7fcaf809ae90 legacy=0x7fcaf81d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:46.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 1 -- 172.21.15.132:0/175568329 >> v1:172.21.15.132:6789/0 conn(0x7fcaf809bdc0 legacy=0x7fcaf81d4250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:46.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:26:46.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 1 monclient: found mon.h 2024-01-24T23:26:46.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:46.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:46.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 1 -- 172.21.15.132:0/175568329 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf81c8630 con 0x7fcaf8072690 2024-01-24T23:26:46.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 10 monclient: _finish_auth 0 2024-01-24T23:26:46.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:26:46.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:46.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 5 monclient: authenticate success, global_id 6296 2024-01-24T23:26:46.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 10 monclient: _renew_subs 2024-01-24T23:26:46.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:46.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 1 -- 172.21.15.132:0/175568329 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf81c8d50 con 0x7fcaf8072690 2024-01-24T23:26:46.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 10 monclient: _renew_subs 2024-01-24T23:26:46.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:46.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 1 -- 172.21.15.132:0/175568329 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf81bb190 con 0x7fcaf8072690 2024-01-24T23:26:46.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 1 -- 172.21.15.132:0/175568329 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fcae8001860 con 0x7fcaf8072690 2024-01-24T23:26:46.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:46.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 1 -- 172.21.15.132:0/175568329 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fcae8001180 con 0x7fcaf8072690 2024-01-24T23:26:46.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:46.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:46.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf67fc640 4 set_mon_vals no callback set 2024-01-24T23:26:46.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 10 monclient: dump: 2024-01-24T23:26:46.422 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:46.422 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:46.423 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:46.423 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:46.423 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:46.423 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:46.423 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:46.423 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:46.423 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:46.423 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:46.423 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:46.423 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:46.423 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:46.423 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:46.423 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:46.423 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:46.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 1 -- 172.21.15.132:0/175568329 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fcae800fa80 con 0x7fcaf8072690 2024-01-24T23:26:46.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:46.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:46.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:46.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcafe0ea640 1 -- 172.21.15.132:0/175568329 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcadc005090 con 0x7fcaf8072690 2024-01-24T23:26:46.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.403+0000 7fcaf4ff9640 1 -- 172.21.15.132:0/175568329 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(188..188 src has 1..188) v4 ==== 9223+0+0 (unknown 2485752126 0 0) 0x7fcae8044e90 con 0x7fcaf8072690 2024-01-24T23:26:46.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.407+0000 7fcaeeffd640 10 monclient: get_auth_request con 0x7fcad8037670 auth_method 0 2024-01-24T23:26:46.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.411+0000 7fcaf4ff9640 1 -- 172.21.15.132:0/175568329 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fcae8010f80 con 0x7fcaf8072690 2024-01-24T23:26:46.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.411+0000 7fcaf4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:46.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.411+0000 7fcaf4ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:46.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.567+0000 7fcafe0ea640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:46.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.567+0000 7fcafe0ea640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:46.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.567+0000 7fcafe0ea640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:46.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.567+0000 7fcafe0ea640 1 -- 172.21.15.132:0/175568329 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fcadc0058c0 con 0x7fcaf8072690 2024-01-24T23:26:46.910 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.907+0000 7fcaf4ff9640 1 -- 172.21.15.132:0/175568329 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}]=0 added bucket h10 type host to location {rack=r1} v189) v1 ==== 168+0+0 (unknown 2585372734 0 0) 0x7fcae8013080 con 0x7fcaf8072690 2024-01-24T23:26:46.910 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.907+0000 7fcaf4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:46.910 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:46.907+0000 7fcaf4ff9640 10 monclient: _finish_command 2 = system:0 added bucket h10 type host to location {rack=r1} 2024-01-24T23:26:47.012 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.011+0000 7fcafe0ea640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:47.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.011+0000 7fcafe0ea640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:47.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.011+0000 7fcafe0ea640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:47.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.011+0000 7fcafe0ea640 1 -- 172.21.15.132:0/175568329 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fcadc005a80 con 0x7fcaf8072690 2024-01-24T23:26:47.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.011+0000 7fcaf4ff9640 1 -- 172.21.15.132:0/175568329 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h10' already exists v189) v1 ==== 147+0+0 (unknown 1341916185 0 0) 0x7fcae8014800 con 0x7fcaf8072690 2024-01-24T23:26:47.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.011+0000 7fcaf4ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:47.013 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.011+0000 7fcaf4ff9640 10 monclient: _finish_command 3 = system:0 bucket 'h10' already exists 2024-01-24T23:26:47.013 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h10' already exists 2024-01-24T23:26:47.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.015+0000 7fcafe0ea640 1 -- 172.21.15.132:0/175568329 >> v1:172.21.15.132:6812/1835401922 conn(0x7fcad8037670 legacy=0x7fcad8039b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:47.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.015+0000 7fcafe0ea640 10 monclient: shutdown 2024-01-24T23:26:47.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.015+0000 7fcafe0ea640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:47.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.015+0000 7fcafe0ea640 1 -- 172.21.15.132:0/175568329 >> v1:172.21.15.132:6791/0 conn(0x7fcaf8072690 legacy=0x7fcaf81ba8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:47.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.015+0000 7fcafe0ea640 1 -- 172.21.15.132:0/175568329 shutdown_connections 2024-01-24T23:26:47.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.015+0000 7fcafe0ea640 1 -- 172.21.15.132:0/175568329 >> 172.21.15.132:0/175568329 conn(0x7fcaf80a1bc0 msgr2=0x7fcaf809d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:47.015 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.015+0000 7fcafe0ea640 1 -- 172.21.15.132:0/175568329 shutdown_connections 2024-01-24T23:26:47.016 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.015+0000 7fcafe0ea640 1 -- 172.21.15.132:0/175568329 wait complete. 2024-01-24T23:26:47.030 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:26:47.030 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h11 host rack=r1 2024-01-24T23:26:47.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 10 monclient: get_monmap_and_config 2024-01-24T23:26:47.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 10 monclient: build_initial_monmap 2024-01-24T23:26:47.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 10 monclient: monmap: 2024-01-24T23:26:47.153 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:47.153 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:47.153 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:47.153859+0000 2024-01-24T23:26:47.153 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:47.153859+0000 2024-01-24T23:26:47.153 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:47.153 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:47.153 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:47.153 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:47.153 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:47.153 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:47.154 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:47.154 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:47.154 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:47.154 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:47.154 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:47.154 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:47.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 1 Processor -- start 2024-01-24T23:26:47.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 1 -- start start 2024-01-24T23:26:47.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 10 monclient: init 2024-01-24T23:26:47.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:47.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 10 monclient: _add_conns ranks=[7,5,8,4,2,1,0,6,3] 2024-01-24T23:26:47.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 10 monclient(hunting): picked mon.noname-i con 0x7f4968072690 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:47.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 10 monclient(hunting): picked mon.noname-h con 0x7f496809ae90 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:47.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 10 monclient(hunting): picked mon.noname-e con 0x7f496809bdc0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:47.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4968064700 con 0x7f496809ae90 2024-01-24T23:26:47.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f496809a790 con 0x7f4968072690 2024-01-24T23:26:47.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49680a18f0 con 0x7f496809bdc0 2024-01-24T23:26:47.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:47.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 10 monclient(hunting): authenticate will time out at 1294.478512s 2024-01-24T23:26:47.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496d704640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f4968072690 legacy=0x7f49680a0ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:47.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496cf03640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f496809ae90 0x7f49680a11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:55812/0 (socket says 172.21.15.132:55812) 2024-01-24T23:26:47.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496cf03640 1 -- 172.21.15.132:0/241068743 learned_addr learned my addr 172.21.15.132:0/241068743 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:47.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 1 -- 172.21.15.132:0/241068743 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1736374153 0 0) 0x7f4968064700 con 0x7f496809ae90 2024-01-24T23:26:47.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:26:47.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:47.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:47.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 10 monclient(hunting): my global_id is 6305 2024-01-24T23:26:47.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 1 -- 172.21.15.132:0/241068743 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4954003530 con 0x7f496809ae90 2024-01-24T23:26:47.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 1 -- 172.21.15.132:0/241068743 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2417447948 0 0) 0x7f4954003530 con 0x7f496809ae90 2024-01-24T23:26:47.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 1 -- 172.21.15.132:0/241068743 >> v1:172.21.15.115:6793/0 conn(0x7f496809bdc0 legacy=0x7f496813d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:47.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 1 -- 172.21.15.132:0/241068743 >> v1:172.21.15.132:6792/0 conn(0x7f4968072690 legacy=0x7f49680a0ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:47.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 10 monclient: _finish_hunting 0 2024-01-24T23:26:47.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 1 monclient: found mon.noname-h 2024-01-24T23:26:47.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:47.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:26:47.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 1 -- 172.21.15.132:0/241068743 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49680997a0 con 0x7f496809ae90 2024-01-24T23:26:47.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 10 monclient: _finish_auth 0 2024-01-24T23:26:47.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 10 monclient: _check_auth_tickets 2024-01-24T23:26:47.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:47.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 5 monclient: authenticate success, global_id 6305 2024-01-24T23:26:47.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 1 -- 172.21.15.132:0/241068743 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f49580017c0 con 0x7f496809ae90 2024-01-24T23:26:47.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:47.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:47.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 10 monclient: dump: 2024-01-24T23:26:47.164 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:47.164 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:47.165 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:47.165 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:47.165 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:47.165 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:47.165 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:47.165 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:47.165 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:47.165 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:47.165 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:47.165 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:47.165 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:47.165 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:47.165 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:47.165 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:47.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:47.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 1 -- 172.21.15.132:0/241068743 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4958001a70 con 0x7f496809ae90 2024-01-24T23:26:47.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:47.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 1 -- 172.21.15.132:0/241068743 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4958004bf0 con 0x7f496809ae90 2024-01-24T23:26:47.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:47.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 4 set_mon_vals no callback set 2024-01-24T23:26:47.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:47.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:47.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f495ffff640 10 monclient: dump: 2024-01-24T23:26:47.166 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:47.166 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:47.166 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:47.166 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:47.166 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:47.166 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:47.167 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:47.167 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:47.167 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:47.167 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:47.167 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:47.167 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:47.167 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:47.167 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:47.167 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:47.167 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:47.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 10 monclient: shutdown 2024-01-24T23:26:47.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:47.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 1 -- 172.21.15.132:0/241068743 >> v1:172.21.15.132:6791/0 conn(0x7f496809ae90 legacy=0x7f49680a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:47.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 1 -- 172.21.15.132:0/241068743 shutdown_connections 2024-01-24T23:26:47.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 1 -- 172.21.15.132:0/241068743 >> 172.21.15.132:0/241068743 conn(0x7f49680a1bc0 msgr2=0x7f49680a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:47.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 1 -- 172.21.15.132:0/241068743 shutdown_connections 2024-01-24T23:26:47.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.151+0000 7f496f968640 1 -- 172.21.15.132:0/241068743 wait complete. 2024-01-24T23:26:47.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496f968640 10 monclient: build_initial_monmap 2024-01-24T23:26:47.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496f968640 10 monclient: monmap: 2024-01-24T23:26:47.168 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:47.168 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:47.168 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:47.168 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:47.168 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:47.168 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:47.168 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:47.168 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:47.168 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:47.168 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:47.169 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:47.169 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:47.169 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:47.169 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:47.169 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:47.169 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:47.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496f968640 1 Processor -- start 2024-01-24T23:26:47.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496f968640 1 -- start start 2024-01-24T23:26:47.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496f968640 10 monclient: init 2024-01-24T23:26:47.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496f968640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:47.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496f968640 10 monclient: _add_conns ranks=[0,1,6,4,5,2,7,8,3] 2024-01-24T23:26:47.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496f968640 10 monclient(hunting): picked mon.noname-a con 0x7f4968072690 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:47.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496f968640 10 monclient(hunting): picked mon.noname-f con 0x7f496809ae90 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:47.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496f968640 10 monclient(hunting): picked mon.noname-d con 0x7f496809bdc0 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:47.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496f968640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4968099470 con 0x7f4968072690 2024-01-24T23:26:47.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496f968640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f496809b930 con 0x7f496809ae90 2024-01-24T23:26:47.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496f968640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4968071840 con 0x7f496809bdc0 2024-01-24T23:26:47.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496f968640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:47.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496cf03640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f496809ae90 legacy=0x7f49681b64c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:47.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496f968640 10 monclient(hunting): authenticate will time out at 1294.481533s 2024-01-24T23:26:47.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496d704640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f4968072690 0x7f49681c6190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:51786/0 (socket says 172.21.15.132:51786) 2024-01-24T23:26:47.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496df05640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f496809bdc0 0x7f49681d14c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:47678/0 (socket says 172.21.15.132:47678) 2024-01-24T23:26:47.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496d704640 1 -- 172.21.15.132:0/3681766274 learned_addr learned my addr 172.21.15.132:0/3681766274 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:47.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 1 -- 172.21.15.132:0/3681766274 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3087368522 0 0) 0x7f4968071840 con 0x7f496809bdc0 2024-01-24T23:26:47.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:26:47.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:47.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:47.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 10 monclient(hunting): my global_id is 6513 2024-01-24T23:26:47.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 1 -- 172.21.15.132:0/3681766274 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f494c003530 con 0x7f496809bdc0 2024-01-24T23:26:47.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 1 -- 172.21.15.132:0/3681766274 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3975296088 0 0) 0x7f4968099470 con 0x7f4968072690 2024-01-24T23:26:47.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:26:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 10 monclient(hunting): my global_id is 94752 2024-01-24T23:26:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 1 -- 172.21.15.132:0/3681766274 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4968071840 con 0x7f4968072690 2024-01-24T23:26:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 1 -- 172.21.15.132:0/3681766274 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 773699052 0 0) 0x7f494c003530 con 0x7f496809bdc0 2024-01-24T23:26:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 1 -- 172.21.15.132:0/3681766274 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4968099470 con 0x7f496809bdc0 2024-01-24T23:26:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 1 -- 172.21.15.132:0/3681766274 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4960001c40 con 0x7f496809bdc0 2024-01-24T23:26:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:26:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 10 monclient(hunting): dump: 2024-01-24T23:26:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:47.172 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:47.172 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:47.172 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:47.172 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:47.172 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:47.172 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:47.172 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:47.172 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:47.172 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:47.172 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:47.172 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:47.172 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:47.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:47.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 1 -- 172.21.15.132:0/3681766274 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2291463874 0 0) 0x7f4968071840 con 0x7f4968072690 2024-01-24T23:26:47.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 1 -- 172.21.15.132:0/3681766274 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f494c003530 con 0x7f4968072690 2024-01-24T23:26:47.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 1 -- 172.21.15.132:0/3681766274 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f49640042c0 con 0x7f4968072690 2024-01-24T23:26:47.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:47.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:26:47.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 10 monclient(hunting): dump: 2024-01-24T23:26:47.173 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:47.173 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:47.173 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:47.173 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:47.173 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:47.173 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:47.173 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:47.173 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:47.173 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:47.173 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:47.174 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:47.174 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:47.174 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:47.174 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:47.174 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:47.174 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:47.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 1 -- 172.21.15.132:0/3681766274 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1983882732 0 0) 0x7f4968099470 con 0x7f496809bdc0 2024-01-24T23:26:47.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 1 -- 172.21.15.132:0/3681766274 >> v1:172.21.15.132:6789/0 conn(0x7f496809ae90 legacy=0x7f49681b64c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:47.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 1 -- 172.21.15.132:0/3681766274 >> v1:172.21.15.115:6789/0 conn(0x7f4968072690 legacy=0x7f49681c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:47.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 10 monclient: _finish_hunting 0 2024-01-24T23:26:47.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 1 monclient: found mon.d 2024-01-24T23:26:47.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:47.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:47.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 1 -- 172.21.15.132:0/3681766274 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49681b06b0 con 0x7f496809bdc0 2024-01-24T23:26:47.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 10 monclient: _finish_auth 0 2024-01-24T23:26:47.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 10 monclient: _check_auth_tickets 2024-01-24T23:26:47.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:47.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496f968640 5 monclient: authenticate success, global_id 6513 2024-01-24T23:26:47.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496f968640 10 monclient: _renew_subs 2024-01-24T23:26:47.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496f968640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:47.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496f968640 1 -- 172.21.15.132:0/3681766274 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49681b6bd0 con 0x7f496809bdc0 2024-01-24T23:26:47.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496f968640 10 monclient: _renew_subs 2024-01-24T23:26:47.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496f968640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:47.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f496f968640 1 -- 172.21.15.132:0/3681766274 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f49681af920 con 0x7f496809bdc0 2024-01-24T23:26:47.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 1 -- 172.21.15.132:0/3681766274 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4960001e20 con 0x7f496809bdc0 2024-01-24T23:26:47.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:47.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 1 -- 172.21.15.132:0/3681766274 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4960003f90 con 0x7f496809bdc0 2024-01-24T23:26:47.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:47.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495f7fe640 4 set_mon_vals no callback set 2024-01-24T23:26:47.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:26:47.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.155+0000 7f495dffb640 10 monclient: dump: 2024-01-24T23:26:47.176 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:47.176 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:47.176 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:47.176 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:47.176 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:47.176 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:47.176 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:47.176 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:47.176 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:47.176 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:47.177 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:47.177 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:47.177 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:47.177 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:47.177 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:47.177 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:47.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.159+0000 7f496f968640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:47.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.159+0000 7f496f968640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:47.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.159+0000 7f496f968640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:47.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.159+0000 7f496f968640 1 -- 172.21.15.132:0/3681766274 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4948005090 con 0x7f496809bdc0 2024-01-24T23:26:47.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.159+0000 7f495dffb640 1 -- 172.21.15.132:0/3681766274 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f4960004170 con 0x7f496809bdc0 2024-01-24T23:26:47.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.159+0000 7f495dffb640 1 -- 172.21.15.132:0/3681766274 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(189..189 src has 1..189) v4 ==== 9436+0+0 (unknown 1380596053 0 0) 0x7f4960044f00 con 0x7f496809bdc0 2024-01-24T23:26:47.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.159+0000 7f496d704640 10 monclient: get_auth_request con 0x7f494c0376d0 auth_method 0 2024-01-24T23:26:47.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.171+0000 7f495dffb640 1 -- 172.21.15.132:0/3681766274 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f4960002010 con 0x7f496809bdc0 2024-01-24T23:26:47.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.171+0000 7f495dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:47.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.171+0000 7f495dffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:47.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.331+0000 7f496f968640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:47.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.331+0000 7f496f968640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:47.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.331+0000 7f496f968640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:47.334 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:47.331+0000 7f496f968640 1 -- 172.21.15.132:0/3681766274 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f4948005340 con 0x7f496809bdc0 2024-01-24T23:26:47.705 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~43s 2024-01-24T23:26:48.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.191+0000 7f495dffb640 1 -- 172.21.15.132:0/3681766274 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}]=0 added bucket h11 type host to location {rack=r1} v190) v1 ==== 168+0+0 (unknown 1446942030 0 0) 0x7f4960014750 con 0x7f496809bdc0 2024-01-24T23:26:48.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.191+0000 7f495dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:48.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.191+0000 7f495dffb640 10 monclient: _finish_command 2 = system:0 added bucket h11 type host to location {rack=r1} 2024-01-24T23:26:48.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.291+0000 7f496f968640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:48.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.291+0000 7f496f968640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:48.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.291+0000 7f496f968640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:48.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.291+0000 7f496f968640 1 -- 172.21.15.132:0/3681766274 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f4948005a80 con 0x7f496809bdc0 2024-01-24T23:26:48.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.291+0000 7f495dffb640 1 -- 172.21.15.132:0/3681766274 <== mon.6 v1:172.21.15.115:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h11' already exists v190) v1 ==== 147+0+0 (unknown 1830361629 0 0) 0x7f4960014b40 con 0x7f496809bdc0 2024-01-24T23:26:48.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.291+0000 7f495dffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:48.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.291+0000 7f495dffb640 10 monclient: _finish_command 3 = system:0 bucket 'h11' already exists 2024-01-24T23:26:48.295 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h11' already exists 2024-01-24T23:26:48.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.295+0000 7f496f968640 1 -- 172.21.15.132:0/3681766274 >> v1:172.21.15.132:6812/1835401922 conn(0x7f494c0376d0 legacy=0x7f494c039b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:48.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.295+0000 7f496f968640 10 monclient: shutdown 2024-01-24T23:26:48.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.295+0000 7f496f968640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:48.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.295+0000 7f496f968640 1 -- 172.21.15.132:0/3681766274 >> v1:172.21.15.115:6792/0 conn(0x7f496809bdc0 legacy=0x7f49681d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:48.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.295+0000 7f496f968640 1 -- 172.21.15.132:0/3681766274 shutdown_connections 2024-01-24T23:26:48.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.295+0000 7f496f968640 1 -- 172.21.15.132:0/3681766274 >> 172.21.15.132:0/3681766274 conn(0x7f49680a1bc0 msgr2=0x7f49681bc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:48.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.295+0000 7f496f968640 1 -- 172.21.15.132:0/3681766274 shutdown_connections 2024-01-24T23:26:48.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.295+0000 7f496f968640 1 -- 172.21.15.132:0/3681766274 wait complete. 2024-01-24T23:26:48.312 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:26:48.312 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h12 host rack=r1 2024-01-24T23:26:48.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc530f24640 10 monclient: get_monmap_and_config 2024-01-24T23:26:48.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc530f24640 10 monclient: build_initial_monmap 2024-01-24T23:26:48.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc530f24640 10 monclient: monmap: 2024-01-24T23:26:48.427 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:48.427 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:48.427 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:48.428975+0000 2024-01-24T23:26:48.427 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:48.428975+0000 2024-01-24T23:26:48.428 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:48.428 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:48.428 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:48.428 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:48.428 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:48.428 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:48.428 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:48.428 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:48.428 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:48.428 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:48.428 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:48.428 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:48.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc530f24640 1 Processor -- start 2024-01-24T23:26:48.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc530f24640 1 -- start start 2024-01-24T23:26:48.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc530f24640 10 monclient: init 2024-01-24T23:26:48.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc530f24640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:48.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc530f24640 10 monclient: _add_conns ranks=[2,8,7,5,0,3,4,1,6] 2024-01-24T23:26:48.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc530f24640 10 monclient(hunting): picked mon.noname-b con 0x7fc52c099480 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:48.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc530f24640 10 monclient(hunting): picked mon.noname-e con 0x7fc52c09fdb0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:48.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc530f24640 10 monclient(hunting): picked mon.noname-i con 0x7fc52c13a270 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:48.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc530f24640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52c064700 con 0x7fc52c099480 2024-01-24T23:26:48.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc530f24640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52c09eb20 con 0x7fc52c13a270 2024-01-24T23:26:48.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc530f24640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52c09ee00 con 0x7fc52c09fdb0 2024-01-24T23:26:48.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc530f24640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:48.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc530f24640 10 monclient(hunting): authenticate will time out at 1295.753752s 2024-01-24T23:26:48.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52ad9d640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fc52c13a270 legacy=0x7fc52c13c660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:48.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52a59c640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fc52c099480 legacy=0x7fc52c099860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:48.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc529d9b640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fc52c09fdb0 0x7fc52c138b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:34854/0 (socket says 172.21.15.132:34854) 2024-01-24T23:26:48.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc529d9b640 1 -- 172.21.15.132:0/398531742 learned_addr learned my addr 172.21.15.132:0/398531742 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:48.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 1 -- 172.21.15.132:0/398531742 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283363513 0 0) 0x7fc52c09ee00 con 0x7fc52c09fdb0 2024-01-24T23:26:48.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 10 monclient(hunting): _negotiate 2024-01-24T23:26:48.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:48.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:48.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 10 monclient(hunting): my global_id is 85643 2024-01-24T23:26:48.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 1 -- 172.21.15.132:0/398531742 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc518003530 con 0x7fc52c09fdb0 2024-01-24T23:26:48.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 1 -- 172.21.15.132:0/398531742 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4067609731 0 0) 0x7fc518003530 con 0x7fc52c09fdb0 2024-01-24T23:26:48.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 1 -- 172.21.15.132:0/398531742 >> v1:172.21.15.132:6792/0 conn(0x7fc52c13a270 legacy=0x7fc52c13c660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:48.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 1 -- 172.21.15.132:0/398531742 >> v1:172.21.15.115:6790/0 conn(0x7fc52c099480 legacy=0x7fc52c099860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:48.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 10 monclient: _finish_hunting 0 2024-01-24T23:26:48.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 1 monclient: found mon.noname-e 2024-01-24T23:26:48.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:48.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:26:48.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 1 -- 172.21.15.132:0/398531742 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c09f0e0 con 0x7fc52c09fdb0 2024-01-24T23:26:48.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 10 monclient: _finish_auth 0 2024-01-24T23:26:48.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 10 monclient: _check_auth_tickets 2024-01-24T23:26:48.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:48.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 1 -- 172.21.15.132:0/398531742 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc5140017c0 con 0x7fc52c09fdb0 2024-01-24T23:26:48.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc530f24640 5 monclient: authenticate success, global_id 85643 2024-01-24T23:26:48.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc530f24640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:48.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:48.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:26:48.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 10 monclient: dump: 2024-01-24T23:26:48.436 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:48.436 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:48.436 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:48.436 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:48.436 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:48.436 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:48.436 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:48.436 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:48.436 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:48.436 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:48.437 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:48.437 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:48.437 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:48.437 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:48.437 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:48.437 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:48.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc530f24640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:48.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 1 -- 172.21.15.132:0/398531742 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc5140019e0 con 0x7fc52c09fdb0 2024-01-24T23:26:48.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:48.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc530f24640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:48.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc530f24640 4 set_mon_vals no callback set 2024-01-24T23:26:48.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 1 -- 172.21.15.132:0/398531742 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc5140048b0 con 0x7fc52c09fdb0 2024-01-24T23:26:48.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:48.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:26:48.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.427+0000 7fc52959a640 10 monclient: dump: 2024-01-24T23:26:48.437 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:48.437 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:48.438 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:48.438 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:48.438 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:48.438 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:48.438 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:48.438 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:48.438 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:48.438 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:48.438 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:48.438 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:48.438 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:48.438 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:48.438 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:48.438 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:48.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 10 monclient: shutdown 2024-01-24T23:26:48.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:48.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 1 -- 172.21.15.132:0/398531742 >> v1:172.21.15.115:6793/0 conn(0x7fc52c09fdb0 legacy=0x7fc52c138b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:48.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 1 -- 172.21.15.132:0/398531742 shutdown_connections 2024-01-24T23:26:48.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 1 -- 172.21.15.132:0/398531742 >> 172.21.15.132:0/398531742 conn(0x7fc52c0a2c40 msgr2=0x7fc52c0a3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:48.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 1 -- 172.21.15.132:0/398531742 shutdown_connections 2024-01-24T23:26:48.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 1 -- 172.21.15.132:0/398531742 wait complete. 2024-01-24T23:26:48.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 10 monclient: build_initial_monmap 2024-01-24T23:26:48.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 10 monclient: monmap: 2024-01-24T23:26:48.439 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:48.439 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:48.439 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:48.439 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:48.439 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:48.439 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:48.439 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:48.439 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:48.439 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:48.439 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:48.440 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:48.440 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:48.440 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:48.440 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:48.440 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:48.440 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:48.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 1 Processor -- start 2024-01-24T23:26:48.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 1 -- start start 2024-01-24T23:26:48.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 10 monclient: init 2024-01-24T23:26:48.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:48.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 10 monclient: _add_conns ranks=[8,6,1,3,0,4,5,7,2] 2024-01-24T23:26:48.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 10 monclient(hunting): picked mon.noname-e con 0x7fc52c099480 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:48.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 10 monclient(hunting): picked mon.noname-d con 0x7fc52c09fdb0 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:48.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 10 monclient(hunting): picked mon.noname-f con 0x7fc52c13a270 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:48.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52c0a3b40 con 0x7fc52c13a270 2024-01-24T23:26:48.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52c06ea50 con 0x7fc52c09fdb0 2024-01-24T23:26:48.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52c098d50 con 0x7fc52c099480 2024-01-24T23:26:48.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:48.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 10 monclient(hunting): authenticate will time out at 1295.757711s 2024-01-24T23:26:48.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc52ad9d640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fc52c13a270 legacy=0x7fc52c1cf640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:48.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc529d9b640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fc52c09fdb0 0x7fc52c1c7af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:47690/0 (socket says 172.21.15.132:47690) 2024-01-24T23:26:48.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc529d9b640 1 -- 172.21.15.132:0/3881892838 learned_addr learned my addr 172.21.15.132:0/3881892838 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:48.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc52a59c640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fc52c099480 0x7fc52c1c73e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:34868/0 (socket says 172.21.15.132:34868) 2024-01-24T23:26:48.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 1 -- 172.21.15.132:0/3881892838 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1156846325 0 0) 0x7fc52c06ea50 con 0x7fc52c09fdb0 2024-01-24T23:26:48.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:48.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:48.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:48.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 10 monclient(hunting): my global_id is 6522 2024-01-24T23:26:48.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 1 -- 172.21.15.132:0/3881892838 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc50c003530 con 0x7fc52c09fdb0 2024-01-24T23:26:48.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 1 -- 172.21.15.132:0/3881892838 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1411088247 0 0) 0x7fc52c098d50 con 0x7fc52c099480 2024-01-24T23:26:48.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:48.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:48.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:48.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 10 monclient(hunting): my global_id is 85652 2024-01-24T23:26:48.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 1 -- 172.21.15.132:0/3881892838 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc52c06ea50 con 0x7fc52c099480 2024-01-24T23:26:48.444 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 1 -- 172.21.15.132:0/3881892838 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2686661250 0 0) 0x7fc52c06ea50 con 0x7fc52c099480 2024-01-24T23:26:48.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 1 -- 172.21.15.132:0/3881892838 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc52c098d50 con 0x7fc52c099480 2024-01-24T23:26:48.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 1 -- 172.21.15.132:0/3881892838 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2995939658 0 0) 0x7fc50c003530 con 0x7fc52c09fdb0 2024-01-24T23:26:48.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 1 -- 172.21.15.132:0/3881892838 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc52c06ea50 con 0x7fc52c09fdb0 2024-01-24T23:26:48.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 1 -- 172.21.15.132:0/3881892838 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc51c0041a0 con 0x7fc52c099480 2024-01-24T23:26:48.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:48.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:26:48.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 10 monclient(hunting): dump: 2024-01-24T23:26:48.445 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:48.445 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:48.445 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:48.445 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:48.445 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:48.445 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:48.445 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:48.445 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:48.445 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:48.446 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:48.446 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:48.446 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:48.446 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:48.446 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:48.446 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:48.446 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:48.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:48.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 1 -- 172.21.15.132:0/3881892838 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc5140016c0 con 0x7fc52c09fdb0 2024-01-24T23:26:48.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:48.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:26:48.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 10 monclient(hunting): dump: 2024-01-24T23:26:48.446 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:48.446 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:48.447 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:48.447 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:48.447 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:48.447 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:48.447 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:48.447 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:48.447 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:48.447 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:48.447 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:48.447 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:48.447 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:48.447 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:48.447 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:48.447 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:48.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 1 -- 172.21.15.132:0/3881892838 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1658459871 0 0) 0x7fc52c098d50 con 0x7fc52c099480 2024-01-24T23:26:48.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 1 -- 172.21.15.132:0/3881892838 >> v1:172.21.15.115:6792/0 conn(0x7fc52c09fdb0 legacy=0x7fc52c1c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:48.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 1 -- 172.21.15.132:0/3881892838 >> v1:172.21.15.132:6789/0 conn(0x7fc52c13a270 legacy=0x7fc52c1cf640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:48.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:26:48.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 1 monclient: found mon.e 2024-01-24T23:26:48.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:48.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:48.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 1 -- 172.21.15.132:0/3881892838 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c06fba0 con 0x7fc52c099480 2024-01-24T23:26:48.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 10 monclient: _finish_auth 0 2024-01-24T23:26:48.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:26:48.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc522ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:48.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 5 monclient: authenticate success, global_id 85652 2024-01-24T23:26:48.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 10 monclient: _renew_subs 2024-01-24T23:26:48.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:48.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 1 -- 172.21.15.132:0/3881892838 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52c098f50 con 0x7fc52c099480 2024-01-24T23:26:48.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 10 monclient: _renew_subs 2024-01-24T23:26:48.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:48.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.431+0000 7fc530f24640 1 -- 172.21.15.132:0/3881892838 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52c09e540 con 0x7fc52c099480 2024-01-24T23:26:48.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.435+0000 7fc522ffd640 1 -- 172.21.15.132:0/3881892838 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc51c004680 con 0x7fc52c099480 2024-01-24T23:26:48.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.435+0000 7fc522ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:48.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.435+0000 7fc522ffd640 1 -- 172.21.15.132:0/3881892838 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc51c004bb0 con 0x7fc52c099480 2024-01-24T23:26:48.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.435+0000 7fc522ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:48.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.435+0000 7fc528d99640 4 set_mon_vals no callback set 2024-01-24T23:26:48.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.435+0000 7fc522ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:26:48.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.435+0000 7fc522ffd640 10 monclient: dump: 2024-01-24T23:26:48.449 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:48.449 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:48.449 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:48.449 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:48.449 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:48.449 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:48.450 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:48.450 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:48.450 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:48.450 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:48.450 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:48.450 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:48.450 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:48.450 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:48.450 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:48.450 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:48.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.435+0000 7fc530f24640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:48.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.435+0000 7fc530f24640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:48.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.435+0000 7fc530f24640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:48.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.435+0000 7fc530f24640 1 -- 172.21.15.132:0/3881892838 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc510005090 con 0x7fc52c099480 2024-01-24T23:26:48.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.435+0000 7fc522ffd640 1 -- 172.21.15.132:0/3881892838 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fc51c004d90 con 0x7fc52c099480 2024-01-24T23:26:48.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.435+0000 7fc522ffd640 1 -- 172.21.15.132:0/3881892838 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(190..190 src has 1..190) v4 ==== 9649+0+0 (unknown 1136186039 0 0) 0x7fc51c044ce0 con 0x7fc52c099480 2024-01-24T23:26:48.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.435+0000 7fc529d9b640 10 monclient: get_auth_request con 0x7fc50c037670 auth_method 0 2024-01-24T23:26:48.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.447+0000 7fc522ffd640 1 -- 172.21.15.132:0/3881892838 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fc51c0147f0 con 0x7fc52c099480 2024-01-24T23:26:48.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.447+0000 7fc522ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:48.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.447+0000 7fc522ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:48.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.607+0000 7fc530f24640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:48.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.607+0000 7fc530f24640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:48.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.607+0000 7fc530f24640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:48.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:48.607+0000 7fc530f24640 1 -- 172.21.15.132:0/3881892838 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fc510005340 con 0x7fc52c099480 2024-01-24T23:26:49.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.203+0000 7fc522ffd640 1 -- 172.21.15.132:0/3881892838 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}]=0 added bucket h12 type host to location {rack=r1} v191) v1 ==== 168+0+0 (unknown 1648141252 0 0) 0x7fc51c0143e0 con 0x7fc52c099480 2024-01-24T23:26:49.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.203+0000 7fc522ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:49.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.203+0000 7fc522ffd640 10 monclient: _finish_command 2 = system:0 added bucket h12 type host to location {rack=r1} 2024-01-24T23:26:49.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.307+0000 7fc530f24640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:49.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.307+0000 7fc530f24640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:49.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.307+0000 7fc530f24640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:49.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.307+0000 7fc530f24640 1 -- 172.21.15.132:0/3881892838 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fc510005a80 con 0x7fc52c099480 2024-01-24T23:26:49.309 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.307+0000 7fc522ffd640 1 -- 172.21.15.132:0/3881892838 <== mon.8 v1:172.21.15.115:6793/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h12' already exists v191) v1 ==== 147+0+0 (unknown 1485846467 0 0) 0x7fc51c0145c0 con 0x7fc52c099480 2024-01-24T23:26:49.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.307+0000 7fc522ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:49.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.307+0000 7fc522ffd640 10 monclient: _finish_command 3 = system:0 bucket 'h12' already exists 2024-01-24T23:26:49.310 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h12' already exists 2024-01-24T23:26:49.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.311+0000 7fc530f24640 1 -- 172.21.15.132:0/3881892838 >> v1:172.21.15.132:6812/1835401922 conn(0x7fc50c037670 legacy=0x7fc50c039b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:49.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.311+0000 7fc530f24640 10 monclient: shutdown 2024-01-24T23:26:49.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.311+0000 7fc530f24640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:49.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.311+0000 7fc530f24640 1 -- 172.21.15.132:0/3881892838 >> v1:172.21.15.115:6793/0 conn(0x7fc52c099480 legacy=0x7fc52c1c73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:49.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.311+0000 7fc530f24640 1 -- 172.21.15.132:0/3881892838 shutdown_connections 2024-01-24T23:26:49.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.311+0000 7fc530f24640 1 -- 172.21.15.132:0/3881892838 >> 172.21.15.132:0/3881892838 conn(0x7fc52c0a2c40 msgr2=0x7fc52c0a5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:49.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.311+0000 7fc530f24640 1 -- 172.21.15.132:0/3881892838 shutdown_connections 2024-01-24T23:26:49.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.311+0000 7fc530f24640 1 -- 172.21.15.132:0/3881892838 wait complete. 2024-01-24T23:26:49.327 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:26:49.327 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h13 host rack=r1 2024-01-24T23:26:49.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 10 monclient: get_monmap_and_config 2024-01-24T23:26:49.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 10 monclient: build_initial_monmap 2024-01-24T23:26:49.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 10 monclient: monmap: 2024-01-24T23:26:49.447 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:49.449117+0000 2024-01-24T23:26:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:49.449117+0000 2024-01-24T23:26:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:49.448 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:49.448 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:49.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 1 Processor -- start 2024-01-24T23:26:49.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 1 -- start start 2024-01-24T23:26:49.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 10 monclient: init 2024-01-24T23:26:49.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:49.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 10 monclient: _add_conns ranks=[7,5,6,8,4,0,1,2,3] 2024-01-24T23:26:49.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 10 monclient(hunting): picked mon.noname-i con 0x7f6494135ea0 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:49.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 10 monclient(hunting): picked mon.noname-h con 0x7f649409bcd0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:49.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 10 monclient(hunting): picked mon.noname-d con 0x7f649409c0f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:49.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6494064700 con 0x7f649409bcd0 2024-01-24T23:26:49.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6494136990 con 0x7f649409c0f0 2024-01-24T23:26:49.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f649409b010 con 0x7f6494135ea0 2024-01-24T23:26:49.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:49.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 10 monclient(hunting): authenticate will time out at 1296.773721s 2024-01-24T23:26:49.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6492ffd640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f6494135ea0 legacy=0x7f6494136280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:49.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f648a7fc640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f649409bcd0 0x7f649413bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:55828/0 (socket says 172.21.15.132:55828) 2024-01-24T23:26:49.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f648a7fc640 1 -- 172.21.15.132:0/1927472423 learned_addr learned my addr 172.21.15.132:0/1927472423 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:49.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 1 -- 172.21.15.132:0/1927472423 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019905284 0 0) 0x7f6494064700 con 0x7f649409bcd0 2024-01-24T23:26:49.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 10 monclient(hunting): _negotiate 2024-01-24T23:26:49.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:49.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:49.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 10 monclient(hunting): my global_id is 6314 2024-01-24T23:26:49.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 1 -- 172.21.15.132:0/1927472423 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6474003530 con 0x7f649409bcd0 2024-01-24T23:26:49.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 1 -- 172.21.15.132:0/1927472423 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3558691021 0 0) 0x7f6474003530 con 0x7f649409bcd0 2024-01-24T23:26:49.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 1 -- 172.21.15.132:0/1927472423 >> v1:172.21.15.132:6792/0 conn(0x7f6494135ea0 legacy=0x7f6494136280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:49.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 1 -- 172.21.15.132:0/1927472423 >> v1:172.21.15.115:6792/0 conn(0x7f649409c0f0 legacy=0x7f649413f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:49.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 10 monclient: _finish_hunting 0 2024-01-24T23:26:49.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 1 monclient: found mon.noname-h 2024-01-24T23:26:49.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:49.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:26:49.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 1 -- 172.21.15.132:0/1927472423 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64940a4f50 con 0x7f649409bcd0 2024-01-24T23:26:49.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 10 monclient: _finish_auth 0 2024-01-24T23:26:49.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 10 monclient: _check_auth_tickets 2024-01-24T23:26:49.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:49.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 1 -- 172.21.15.132:0/1927472423 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f647c0017c0 con 0x7f649409bcd0 2024-01-24T23:26:49.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 5 monclient: authenticate success, global_id 6314 2024-01-24T23:26:49.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:49.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:49.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:49.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 10 monclient: dump: 2024-01-24T23:26:49.456 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:49.456 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:49.456 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:49.456 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:49.456 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:49.456 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:49.456 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:49.456 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:49.456 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:49.457 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:49.457 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:49.457 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:49.457 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:49.457 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:49.457 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:49.457 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:49.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:49.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 1 -- 172.21.15.132:0/1927472423 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f647c001ae0 con 0x7f649409bcd0 2024-01-24T23:26:49.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:49.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:49.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 1 -- 172.21.15.132:0/1927472423 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f647c004bf0 con 0x7f649409bcd0 2024-01-24T23:26:49.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 4 set_mon_vals no callback set 2024-01-24T23:26:49.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:49.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:49.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f64927fc640 10 monclient: dump: 2024-01-24T23:26:49.458 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:49.458 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:49.458 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:49.458 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:49.458 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:49.458 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:49.458 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:49.458 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:49.458 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:49.458 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:49.458 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:49.458 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:49.458 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:49.458 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:49.459 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:49.459 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:49.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 10 monclient: shutdown 2024-01-24T23:26:49.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:49.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 1 -- 172.21.15.132:0/1927472423 >> v1:172.21.15.132:6791/0 conn(0x7f649409bcd0 legacy=0x7f649413bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:49.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 1 -- 172.21.15.132:0/1927472423 shutdown_connections 2024-01-24T23:26:49.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 1 -- 172.21.15.132:0/1927472423 >> 172.21.15.132:0/1927472423 conn(0x7f64940a9bf0 msgr2=0x7f64940a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:49.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 1 -- 172.21.15.132:0/1927472423 shutdown_connections 2024-01-24T23:26:49.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 1 -- 172.21.15.132:0/1927472423 wait complete. 2024-01-24T23:26:49.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 10 monclient: build_initial_monmap 2024-01-24T23:26:49.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.447+0000 7f6499477640 10 monclient: monmap: 2024-01-24T23:26:49.459 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:49.460 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:49.460 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:49.460 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:49.460 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:49.460 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:49.460 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:49.460 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:49.460 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:49.460 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:49.460 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:49.460 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:49.461 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:49.461 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:49.461 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:49.461 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:49.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6499477640 1 Processor -- start 2024-01-24T23:26:49.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6499477640 1 -- start start 2024-01-24T23:26:49.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6499477640 10 monclient: init 2024-01-24T23:26:49.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6499477640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:49.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6499477640 10 monclient: _add_conns ranks=[0,1,7,8,2,3,4,5,6] 2024-01-24T23:26:49.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6499477640 10 monclient(hunting): picked mon.noname-a con 0x7f649409bcd0 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:49.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6499477640 10 monclient(hunting): picked mon.noname-f con 0x7f649409c0f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:49.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6499477640 10 monclient(hunting): picked mon.noname-i con 0x7f6494135ea0 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:49.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648a7fc640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f649409c0f0 legacy=0x7f64941d03b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:49.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6499477640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f649409c6b0 con 0x7f649409bcd0 2024-01-24T23:26:49.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6499477640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6494135c70 con 0x7f649409c0f0 2024-01-24T23:26:49.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f64937fe640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f6494135ea0 legacy=0x7f64941d3ae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:49.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6499477640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f649409fe10 con 0x7f6494135ea0 2024-01-24T23:26:49.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6499477640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:49.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6499477640 10 monclient(hunting): authenticate will time out at 1296.777203s 2024-01-24T23:26:49.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6492ffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f649409bcd0 0x7f6494134d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:51798/0 (socket says 172.21.15.132:51798) 2024-01-24T23:26:49.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6492ffd640 1 -- 172.21.15.132:0/1865898168 learned_addr learned my addr 172.21.15.132:0/1865898168 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:49.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 1 -- 172.21.15.132:0/1865898168 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3142065560 0 0) 0x7f649409c6b0 con 0x7f649409bcd0 2024-01-24T23:26:49.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:26:49.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:49.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:49.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 10 monclient(hunting): my global_id is 94761 2024-01-24T23:26:49.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 1 -- 172.21.15.132:0/1865898168 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6478003530 con 0x7f649409bcd0 2024-01-24T23:26:49.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 1 -- 172.21.15.132:0/1865898168 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3579551659 0 0) 0x7f6478003530 con 0x7f649409bcd0 2024-01-24T23:26:49.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 1 -- 172.21.15.132:0/1865898168 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f649409c6b0 con 0x7f649409bcd0 2024-01-24T23:26:49.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 1 -- 172.21.15.132:0/1865898168 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6484004300 con 0x7f649409bcd0 2024-01-24T23:26:49.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:49.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:26:49.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 10 monclient(hunting): dump: 2024-01-24T23:26:49.465 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:49.465 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:49.465 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:49.465 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:49.465 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:49.465 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:49.465 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:49.466 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:49.466 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:49.466 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:49.466 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:49.466 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:49.466 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:49.466 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:49.466 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:49.466 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:49.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:49.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 1 -- 172.21.15.132:0/1865898168 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 257355309 0 0) 0x7f649409c6b0 con 0x7f649409bcd0 2024-01-24T23:26:49.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 1 -- 172.21.15.132:0/1865898168 >> v1:172.21.15.132:6792/0 conn(0x7f6494135ea0 legacy=0x7f64941d3ae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:49.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 1 -- 172.21.15.132:0/1865898168 >> v1:172.21.15.132:6789/0 conn(0x7f649409c0f0 legacy=0x7f64941d03b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:49.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:26:49.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 1 monclient: found mon.a 2024-01-24T23:26:49.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:49.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:26:49.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 1 -- 172.21.15.132:0/1865898168 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6494120f80 con 0x7f649409bcd0 2024-01-24T23:26:49.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 10 monclient: _finish_auth 0 2024-01-24T23:26:49.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:26:49.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:49.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6499477640 5 monclient: authenticate success, global_id 94761 2024-01-24T23:26:49.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6499477640 10 monclient: _renew_subs 2024-01-24T23:26:49.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6499477640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:26:49.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6499477640 1 -- 172.21.15.132:0/1865898168 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6494121e70 con 0x7f649409bcd0 2024-01-24T23:26:49.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6499477640 10 monclient: _renew_subs 2024-01-24T23:26:49.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6499477640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:26:49.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6499477640 1 -- 172.21.15.132:0/1865898168 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6494120a20 con 0x7f649409bcd0 2024-01-24T23:26:49.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 1 -- 172.21.15.132:0/1865898168 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6484004520 con 0x7f649409bcd0 2024-01-24T23:26:49.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:49.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 1 -- 172.21.15.132:0/1865898168 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f64840049d0 con 0x7f649409bcd0 2024-01-24T23:26:49.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:49.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6491ffb640 4 set_mon_vals no callback set 2024-01-24T23:26:49.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:26:49.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f648bfff640 10 monclient: dump: 2024-01-24T23:26:49.468 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:49.468 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:49.468 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:49.468 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:49.468 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:49.468 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:49.468 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:49.469 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:49.469 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:49.469 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:49.469 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:49.469 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:49.469 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:49.469 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:49.469 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:49.469 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:49.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6499477640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:49.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6499477640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:49.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6499477640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:26:49.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.451+0000 7f6499477640 1 -- 172.21.15.132:0/1865898168 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f646c005090 con 0x7f649409bcd0 2024-01-24T23:26:49.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.455+0000 7f648bfff640 1 -- 172.21.15.132:0/1865898168 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f6484004bb0 con 0x7f649409bcd0 2024-01-24T23:26:49.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.455+0000 7f648bfff640 1 -- 172.21.15.132:0/1865898168 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(191..191 src has 1..191) v4 ==== 9862+0+0 (unknown 2960090235 0 0) 0x7f6484044f20 con 0x7f649409bcd0 2024-01-24T23:26:49.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.455+0000 7f648a7fc640 10 monclient: get_auth_request con 0x7f6478036f60 auth_method 0 2024-01-24T23:26:49.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.467+0000 7f648bfff640 1 -- 172.21.15.132:0/1865898168 <== mon.0 v1:172.21.15.115:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f6484014960 con 0x7f649409bcd0 2024-01-24T23:26:49.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.467+0000 7f648bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:49.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.467+0000 7f648bfff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:49.625 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.623+0000 7f6499477640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:49.625 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.623+0000 7f6499477640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:49.625 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.623+0000 7f6499477640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:26:49.625 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:49.623+0000 7f6499477640 1 -- 172.21.15.132:0/1865898168 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f646c005340 con 0x7f649409bcd0 2024-01-24T23:26:50.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.215+0000 7f648bfff640 1 -- 172.21.15.132:0/1865898168 <== mon.0 v1:172.21.15.115:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}]=0 added bucket h13 type host to location {rack=r1} v192) v1 ==== 168+0+0 (unknown 4118221577 0 0) 0x7f6484014550 con 0x7f649409bcd0 2024-01-24T23:26:50.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.215+0000 7f648bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:50.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.215+0000 7f648bfff640 10 monclient: _finish_command 2 = system:0 added bucket h13 type host to location {rack=r1} 2024-01-24T23:26:50.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.315+0000 7f6499477640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:50.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.315+0000 7f6499477640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:50.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.315+0000 7f6499477640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:26:50.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.315+0000 7f6499477640 1 -- 172.21.15.132:0/1865898168 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f646c005a80 con 0x7f649409bcd0 2024-01-24T23:26:50.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.319+0000 7f648bfff640 1 -- 172.21.15.132:0/1865898168 <== mon.0 v1:172.21.15.115:6789/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h13' already exists v192) v1 ==== 147+0+0 (unknown 1681830767 0 0) 0x7f6484014730 con 0x7f649409bcd0 2024-01-24T23:26:50.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.319+0000 7f648bfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:50.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.319+0000 7f648bfff640 10 monclient: _finish_command 3 = system:0 bucket 'h13' already exists 2024-01-24T23:26:50.319 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h13' already exists 2024-01-24T23:26:50.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.319+0000 7f6499477640 1 -- 172.21.15.132:0/1865898168 >> v1:172.21.15.132:6812/1835401922 conn(0x7f6478036f60 legacy=0x7f6478039420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:50.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.319+0000 7f6499477640 10 monclient: shutdown 2024-01-24T23:26:50.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.319+0000 7f6499477640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:50.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.319+0000 7f6499477640 1 -- 172.21.15.132:0/1865898168 >> v1:172.21.15.115:6789/0 conn(0x7f649409bcd0 legacy=0x7f6494134d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:50.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.319+0000 7f6499477640 1 -- 172.21.15.132:0/1865898168 shutdown_connections 2024-01-24T23:26:50.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.319+0000 7f6499477640 1 -- 172.21.15.132:0/1865898168 >> 172.21.15.132:0/1865898168 conn(0x7f64940a9bf0 msgr2=0x7f64940a2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:50.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.319+0000 7f6499477640 1 -- 172.21.15.132:0/1865898168 shutdown_connections 2024-01-24T23:26:50.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.319+0000 7f6499477640 1 -- 172.21.15.132:0/1865898168 wait complete. 2024-01-24T23:26:50.336 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:26:50.336 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h14 host rack=r1 2024-01-24T23:26:50.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.455+0000 7f8b0694d640 10 monclient: get_monmap_and_config 2024-01-24T23:26:50.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.455+0000 7f8b0694d640 10 monclient: build_initial_monmap 2024-01-24T23:26:50.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.455+0000 7f8b0694d640 10 monclient: monmap: 2024-01-24T23:26:50.459 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:50.459 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:50.459 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:50.460732+0000 2024-01-24T23:26:50.459 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:50.460732+0000 2024-01-24T23:26:50.459 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:50.460 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:50.460 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:50.460 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:50.460 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:50.460 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:50.460 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:50.460 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:50.460 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:50.460 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:50.460 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:50.460 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:50.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 1 Processor -- start 2024-01-24T23:26:50.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 1 -- start start 2024-01-24T23:26:50.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 10 monclient: init 2024-01-24T23:26:50.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:50.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 10 monclient: _add_conns ranks=[5,1,6,0,3,7,8,2,4] 2024-01-24T23:26:50.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 10 monclient(hunting): picked mon.noname-h con 0x7f8b000a0ad0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:50.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 10 monclient(hunting): picked mon.noname-f con 0x7f8b0009ee80 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:50.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 10 monclient(hunting): picked mon.noname-d con 0x7f8b0009b060 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:50.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b00064700 con 0x7f8b0009ee80 2024-01-24T23:26:50.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b0009cb80 con 0x7f8b000a0ad0 2024-01-24T23:26:50.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b000a0eb0 con 0x7f8b0009b060 2024-01-24T23:26:50.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:50.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 10 monclient(hunting): authenticate will time out at 1297.785382s 2024-01-24T23:26:50.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8aff7fe640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f8b0009ee80 legacy=0x7f8b0009a950 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:50.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afffff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f8b000a0ad0 0x7f8b0009a240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:55832/0 (socket says 172.21.15.132:55832) 2024-01-24T23:26:50.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afffff640 1 -- 172.21.15.132:0/2891132926 learned_addr learned my addr 172.21.15.132:0/2891132926 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:50.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 1 -- 172.21.15.132:0/2891132926 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1719122495 0 0) 0x7f8b0009cb80 con 0x7f8b000a0ad0 2024-01-24T23:26:50.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:50.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:50.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:50.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 10 monclient(hunting): my global_id is 6323 2024-01-24T23:26:50.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 1 -- 172.21.15.132:0/2891132926 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ae8003530 con 0x7f8b000a0ad0 2024-01-24T23:26:50.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 1 -- 172.21.15.132:0/2891132926 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1712329059 0 0) 0x7f8ae8003530 con 0x7f8b000a0ad0 2024-01-24T23:26:50.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 1 -- 172.21.15.132:0/2891132926 >> v1:172.21.15.115:6792/0 conn(0x7f8b0009b060 legacy=0x7f8b0013c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:50.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 1 -- 172.21.15.132:0/2891132926 >> v1:172.21.15.132:6789/0 conn(0x7f8b0009ee80 legacy=0x7f8b0009a950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:50.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 10 monclient: _finish_hunting 0 2024-01-24T23:26:50.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 1 monclient: found mon.noname-h 2024-01-24T23:26:50.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:50.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:26:50.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 1 -- 172.21.15.132:0/2891132926 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0009f280 con 0x7f8b000a0ad0 2024-01-24T23:26:50.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 10 monclient: _finish_auth 0 2024-01-24T23:26:50.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 10 monclient: _check_auth_tickets 2024-01-24T23:26:50.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:50.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 1 -- 172.21.15.132:0/2891132926 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8af00017c0 con 0x7f8b000a0ad0 2024-01-24T23:26:50.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 5 monclient: authenticate success, global_id 6323 2024-01-24T23:26:50.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:50.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:50.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:50.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 10 monclient: dump: 2024-01-24T23:26:50.469 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:50.469 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:50.469 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:50.470 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:50.470 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:50.470 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:50.470 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:50.470 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:50.470 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:50.470 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:50.470 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:50.470 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:50.470 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:50.470 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:50.470 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:50.470 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:50.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:50.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 1 -- 172.21.15.132:0/2891132926 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8af0001a70 con 0x7f8b000a0ad0 2024-01-24T23:26:50.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:50.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:50.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 1 -- 172.21.15.132:0/2891132926 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8af0004bf0 con 0x7f8b000a0ad0 2024-01-24T23:26:50.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:50.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 4 set_mon_vals no callback set 2024-01-24T23:26:50.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:50.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8afeffd640 10 monclient: dump: 2024-01-24T23:26:50.471 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:50.471 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:50.471 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:50.471 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:50.471 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:50.471 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:50.471 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:50.471 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:50.472 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:50.472 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:50.472 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:50.472 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:50.472 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:50.472 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:50.472 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:50.472 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:50.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 10 monclient: shutdown 2024-01-24T23:26:50.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:50.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 1 -- 172.21.15.132:0/2891132926 >> v1:172.21.15.132:6791/0 conn(0x7f8b000a0ad0 legacy=0x7f8b0009a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:50.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 1 -- 172.21.15.132:0/2891132926 shutdown_connections 2024-01-24T23:26:50.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 1 -- 172.21.15.132:0/2891132926 >> 172.21.15.132:0/2891132926 conn(0x7f8b000a1bc0 msgr2=0x7f8b000a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:50.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 1 -- 172.21.15.132:0/2891132926 shutdown_connections 2024-01-24T23:26:50.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 1 -- 172.21.15.132:0/2891132926 wait complete. 2024-01-24T23:26:50.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 10 monclient: build_initial_monmap 2024-01-24T23:26:50.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 10 monclient: monmap: 2024-01-24T23:26:50.473 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:50.473 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:50.473 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:50.473 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:50.473 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:50.473 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:50.473 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:50.473 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:50.473 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:50.473 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:50.473 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:50.473 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:50.473 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:50.474 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:50.474 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:50.474 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:50.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 1 Processor -- start 2024-01-24T23:26:50.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 1 -- start start 2024-01-24T23:26:50.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.459+0000 7f8b0694d640 10 monclient: init 2024-01-24T23:26:50.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8b0694d640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:50.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8b0694d640 10 monclient: _add_conns ranks=[7,3,6,4,1,2,5,0,8] 2024-01-24T23:26:50.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8b0694d640 10 monclient(hunting): picked mon.noname-i con 0x7f8b0009b060 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:50.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8b0694d640 10 monclient(hunting): picked mon.noname-g con 0x7f8b0009ee80 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:50.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8b0694d640 10 monclient(hunting): picked mon.noname-d con 0x7f8b000a0ad0 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:50.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8b0694d640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b00071d90 con 0x7f8b0009ee80 2024-01-24T23:26:50.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8b0694d640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b00070a90 con 0x7f8b000a0ad0 2024-01-24T23:26:50.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8b0694d640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b0009b4c0 con 0x7f8b0009b060 2024-01-24T23:26:50.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8b0694d640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:50.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8b0694d640 10 monclient(hunting): authenticate will time out at 1297.788458s 2024-01-24T23:26:50.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afffff640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f8b0009b060 legacy=0x7f8b001b3300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:50.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8aff7fe640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f8b0009ee80 0x7f8b001ce530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:56224/0 (socket says 172.21.15.132:56224) 2024-01-24T23:26:50.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8aff7fe640 1 -- 172.21.15.132:0/1700985699 learned_addr learned my addr 172.21.15.132:0/1700985699 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:50.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 1 -- 172.21.15.132:0/1700985699 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 105376636 0 0) 0x7f8b00071d90 con 0x7f8b0009ee80 2024-01-24T23:26:50.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 10 monclient(hunting): _negotiate 2024-01-24T23:26:50.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:50.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:50.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 10 monclient(hunting): my global_id is 6411 2024-01-24T23:26:50.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 1 -- 172.21.15.132:0/1700985699 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8aec003530 con 0x7f8b0009ee80 2024-01-24T23:26:50.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8aff7fe640 1 -- 172.21.15.132:0/1700985699 >> v1:172.21.15.132:6790/0 conn(0x7f8b0009ee80 legacy=0x7f8b001ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-01-24T23:26:50.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8aff7fe640 1 -- 172.21.15.132:0/1700985699 >> v1:172.21.15.132:6790/0 conn(0x7f8b0009ee80 legacy=0x7f8b001ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:26:50.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8aff7fe640 1 --1- 172.21.15.132:0/1700985699 >> v1:172.21.15.132:6790/0 conn(0x7f8b0009ee80 0x7f8b001ce530 :-1 s=OPENED pgs=449 cs=1 l=1).handle_message read tag failed 2024-01-24T23:26:50.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8aff7fe640 1 --1- 172.21.15.132:0/1700985699 >> v1:172.21.15.132:6790/0 conn(0x7f8b0009ee80 0x7f8b001ce530 :-1 s=OPENED pgs=449 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:26:50.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6790/0 2024-01-24T23:26:50.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 1 -- 172.21.15.132:0/1700985699 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153356606 0 0) 0x7f8b00070a90 con 0x7f8b000a0ad0 2024-01-24T23:26:50.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 10 monclient(hunting): _negotiate 2024-01-24T23:26:50.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:50.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:50.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 10 monclient(hunting): my global_id is 6549 2024-01-24T23:26:50.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 1 -- 172.21.15.132:0/1700985699 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b00071d90 con 0x7f8b000a0ad0 2024-01-24T23:26:50.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 1 -- 172.21.15.132:0/1700985699 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 290137664 0 0) 0x7f8b00071d90 con 0x7f8b000a0ad0 2024-01-24T23:26:50.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 1 -- 172.21.15.132:0/1700985699 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8b00070a90 con 0x7f8b000a0ad0 2024-01-24T23:26:50.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 1 -- 172.21.15.132:0/1700985699 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8af8001d20 con 0x7f8b000a0ad0 2024-01-24T23:26:50.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:50.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:26:50.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 10 monclient(hunting): dump: 2024-01-24T23:26:50.476 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:50.476 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:50.477 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:50.477 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:50.477 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:50.477 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:50.477 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:50.477 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:50.477 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:50.477 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:50.477 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:50.477 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:50.477 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:50.477 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:50.477 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:50.477 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:50.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:50.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 1 -- 172.21.15.132:0/1700985699 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4115699790 0 0) 0x7f8b00070a90 con 0x7f8b000a0ad0 2024-01-24T23:26:50.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 1 -- 172.21.15.132:0/1700985699 >> v1:172.21.15.132:6792/0 conn(0x7f8b0009b060 legacy=0x7f8b001b3300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:50.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 1 -- 172.21.15.132:0/1700985699 >> v1:172.21.15.132:6790/0 conn(0x7f8b0009ee80 legacy=0x7f8b001ce530 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:26:50.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 10 monclient: _finish_hunting 0 2024-01-24T23:26:50.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 1 monclient: found mon.d 2024-01-24T23:26:50.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:50.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:50.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 1 -- 172.21.15.132:0/1700985699 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b00071ab0 con 0x7f8b000a0ad0 2024-01-24T23:26:50.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 10 monclient: _finish_auth 0 2024-01-24T23:26:50.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 10 monclient: _check_auth_tickets 2024-01-24T23:26:50.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:50.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8b0694d640 5 monclient: authenticate success, global_id 6549 2024-01-24T23:26:50.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8b0694d640 10 monclient: _renew_subs 2024-01-24T23:26:50.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8b0694d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:50.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8b0694d640 1 -- 172.21.15.132:0/1700985699 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b000703f0 con 0x7f8b000a0ad0 2024-01-24T23:26:50.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8b0694d640 10 monclient: _renew_subs 2024-01-24T23:26:50.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8b0694d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:50.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8b0694d640 1 -- 172.21.15.132:0/1700985699 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b001c9690 con 0x7f8b000a0ad0 2024-01-24T23:26:50.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 1 -- 172.21.15.132:0/1700985699 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8af8002070 con 0x7f8b000a0ad0 2024-01-24T23:26:50.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:50.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 1 -- 172.21.15.132:0/1700985699 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8af8003f90 con 0x7f8b000a0ad0 2024-01-24T23:26:50.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:50.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:26:50.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afe7fc640 4 set_mon_vals no callback set 2024-01-24T23:26:50.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 10 monclient: dump: 2024-01-24T23:26:50.479 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:50.479 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:50.479 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:50.479 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:50.480 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:50.480 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:50.480 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:50.480 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:50.480 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:50.480 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:50.480 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:50.480 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:50.480 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:50.480 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:50.480 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:50.480 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:50.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8b0694d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:50.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8b0694d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:50.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8b0694d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:50.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8b0694d640 1 -- 172.21.15.132:0/1700985699 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae0005090 con 0x7f8b000a0ad0 2024-01-24T23:26:50.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 1 -- 172.21.15.132:0/1700985699 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f8af80040f0 con 0x7f8b000a0ad0 2024-01-24T23:26:50.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afcff9640 1 -- 172.21.15.132:0/1700985699 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(192..192 src has 1..192) v4 ==== 10075+0+0 (unknown 1863578430 0 0) 0x7f8af8044040 con 0x7f8b000a0ad0 2024-01-24T23:26:50.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.463+0000 7f8afffff640 10 monclient: get_auth_request con 0x7f8aec036ff0 auth_method 0 2024-01-24T23:26:50.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.475+0000 7f8afcff9640 1 -- 172.21.15.132:0/1700985699 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f8af8013770 con 0x7f8b000a0ad0 2024-01-24T23:26:50.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.475+0000 7f8afcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:50.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.475+0000 7f8afcff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:50.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.635+0000 7f8b0694d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:50.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.635+0000 7f8b0694d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:50.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.635+0000 7f8b0694d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:50.638 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:50.635+0000 7f8b0694d640 1 -- 172.21.15.132:0/1700985699 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f8ae0005340 con 0x7f8b000a0ad0 2024-01-24T23:26:51.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.227+0000 7f8afcff9640 1 -- 172.21.15.132:0/1700985699 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}]=0 added bucket h14 type host to location {rack=r1} v193) v1 ==== 168+0+0 (unknown 1364476781 0 0) 0x7f8af8013590 con 0x7f8b000a0ad0 2024-01-24T23:26:51.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.227+0000 7f8afcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:51.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.227+0000 7f8afcff9640 10 monclient: _finish_command 2 = system:0 added bucket h14 type host to location {rack=r1} 2024-01-24T23:26:51.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.335+0000 7f8b0694d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:51.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.335+0000 7f8b0694d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:51.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.335+0000 7f8b0694d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:51.336 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.335+0000 7f8b0694d640 1 -- 172.21.15.132:0/1700985699 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f8ae0005a80 con 0x7f8b000a0ad0 2024-01-24T23:26:51.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.335+0000 7f8afcff9640 1 -- 172.21.15.132:0/1700985699 <== mon.6 v1:172.21.15.115:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h14' already exists v193) v1 ==== 147+0+0 (unknown 768210647 0 0) 0x7f8af8013590 con 0x7f8b000a0ad0 2024-01-24T23:26:51.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.335+0000 7f8afcff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:51.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.335+0000 7f8afcff9640 10 monclient: _finish_command 3 = system:0 bucket 'h14' already exists 2024-01-24T23:26:51.338 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h14' already exists 2024-01-24T23:26:51.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.339+0000 7f8b0694d640 1 -- 172.21.15.132:0/1700985699 >> v1:172.21.15.132:6812/1835401922 conn(0x7f8aec036ff0 legacy=0x7f8aec0394b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:51.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.339+0000 7f8b0694d640 10 monclient: shutdown 2024-01-24T23:26:51.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.339+0000 7f8b0694d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:51.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.339+0000 7f8b0694d640 1 -- 172.21.15.132:0/1700985699 >> v1:172.21.15.115:6792/0 conn(0x7f8b000a0ad0 legacy=0x7f8b001d1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:51.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.339+0000 7f8b0694d640 1 -- 172.21.15.132:0/1700985699 shutdown_connections 2024-01-24T23:26:51.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.339+0000 7f8b0694d640 1 -- 172.21.15.132:0/1700985699 >> 172.21.15.132:0/1700985699 conn(0x7f8b000a1bc0 msgr2=0x7f8b001bc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:51.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.339+0000 7f8b0694d640 1 -- 172.21.15.132:0/1700985699 shutdown_connections 2024-01-24T23:26:51.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.339+0000 7f8b0694d640 1 -- 172.21.15.132:0/1700985699 wait complete. 2024-01-24T23:26:51.355 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:26:51.355 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h15 host rack=r1 2024-01-24T23:26:51.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.471+0000 7febc12a8640 10 monclient: get_monmap_and_config 2024-01-24T23:26:51.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.471+0000 7febc12a8640 10 monclient: build_initial_monmap 2024-01-24T23:26:51.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.471+0000 7febc12a8640 10 monclient: monmap: 2024-01-24T23:26:51.475 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:51.475 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:51.475 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:51.476638+0000 2024-01-24T23:26:51.475 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:51.476638+0000 2024-01-24T23:26:51.475 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:51.475 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:51.476 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:51.476 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:51.476 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:51.476 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:51.476 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:51.476 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:51.476 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:51.476 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:51.476 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:51.476 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:51.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 1 Processor -- start 2024-01-24T23:26:51.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 1 -- start start 2024-01-24T23:26:51.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 10 monclient: init 2024-01-24T23:26:51.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:51.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 10 monclient: _add_conns ranks=[2,0,5,3,8,4,7,1,6] 2024-01-24T23:26:51.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 10 monclient(hunting): picked mon.noname-b con 0x7febbc133ed0 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:51.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 10 monclient(hunting): picked mon.noname-a con 0x7febbc1342b0 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:51.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 10 monclient(hunting): picked mon.noname-h con 0x7febbc134690 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:51.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febbc064700 con 0x7febbc1342b0 2024-01-24T23:26:51.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febbc09b3d0 con 0x7febbc133ed0 2024-01-24T23:26:51.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febbc0a58c0 con 0x7febbc134690 2024-01-24T23:26:51.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:51.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 10 monclient(hunting): authenticate will time out at 1298.801624s 2024-01-24T23:26:51.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febbb7fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7febbc134690 0x7febbc13f770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:55834/0 (socket says 172.21.15.132:55834) 2024-01-24T23:26:51.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febbb7fe640 1 -- 172.21.15.132:0/889475358 learned_addr learned my addr 172.21.15.132:0/889475358 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:51.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febbaffd640 1 -- 172.21.15.132:0/889475358 >> v1:172.21.15.115:6790/0 conn(0x7febbc133ed0 legacy=0x7febbc09bdb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:51.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 1 -- 172.21.15.132:0/889475358 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1623262218 0 0) 0x7febbc0a58c0 con 0x7febbc134690 2024-01-24T23:26:51.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:26:51.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:51.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:51.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 10 monclient(hunting): my global_id is 6332 2024-01-24T23:26:51.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 1 -- 172.21.15.132:0/889475358 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feba4003530 con 0x7febbc134690 2024-01-24T23:26:51.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 1 -- 172.21.15.132:0/889475358 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1730540929 0 0) 0x7feba4003530 con 0x7febbc134690 2024-01-24T23:26:51.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 1 -- 172.21.15.132:0/889475358 >> v1:172.21.15.115:6790/0 conn(0x7febbc133ed0 legacy=0x7febbc09bdb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:51.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 1 -- 172.21.15.132:0/889475358 >> v1:172.21.15.115:6789/0 conn(0x7febbc1342b0 legacy=0x7febbc13c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:51.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:26:51.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 1 monclient: found mon.noname-h 2024-01-24T23:26:51.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:51.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:26:51.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 1 -- 172.21.15.132:0/889475358 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febbc09b0f0 con 0x7febbc134690 2024-01-24T23:26:51.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 10 monclient: _finish_auth 0 2024-01-24T23:26:51.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:26:51.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:51.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 1 -- 172.21.15.132:0/889475358 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7febac0017c0 con 0x7febbc134690 2024-01-24T23:26:51.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 5 monclient: authenticate success, global_id 6332 2024-01-24T23:26:51.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:51.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:51.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:51.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 10 monclient: dump: 2024-01-24T23:26:51.485 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:51.485 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:51.485 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:51.485 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:51.485 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:51.485 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:51.485 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:51.485 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:51.485 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:51.485 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:51.485 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:51.485 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:51.485 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:51.486 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:51.486 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:51.486 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:51.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:51.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 1 -- 172.21.15.132:0/889475358 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7febac001ae0 con 0x7febbc134690 2024-01-24T23:26:51.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:51.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:51.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 4 set_mon_vals no callback set 2024-01-24T23:26:51.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 1 -- 172.21.15.132:0/889475358 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7febac004bf0 con 0x7febbc134690 2024-01-24T23:26:51.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:51.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:51.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febb9ffb640 10 monclient: dump: 2024-01-24T23:26:51.486 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:51.487 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:51.487 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:51.487 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:51.487 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:51.487 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:51.487 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:51.487 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:51.487 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:51.487 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:51.487 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:51.487 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:51.487 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:51.487 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:51.487 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:51.487 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:51.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 10 monclient: shutdown 2024-01-24T23:26:51.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:51.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 1 -- 172.21.15.132:0/889475358 >> v1:172.21.15.132:6791/0 conn(0x7febbc134690 legacy=0x7febbc13f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:51.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 1 -- 172.21.15.132:0/889475358 shutdown_connections 2024-01-24T23:26:51.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 1 -- 172.21.15.132:0/889475358 >> 172.21.15.132:0/889475358 conn(0x7febbc0a5ef0 msgr2=0x7febbc0a6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:51.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 1 -- 172.21.15.132:0/889475358 shutdown_connections 2024-01-24T23:26:51.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 1 -- 172.21.15.132:0/889475358 wait complete. 2024-01-24T23:26:51.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 10 monclient: build_initial_monmap 2024-01-24T23:26:51.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.475+0000 7febc12a8640 10 monclient: monmap: 2024-01-24T23:26:51.488 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:51.488 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:51.488 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:51.488 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:51.489 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:51.489 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:51.489 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:51.489 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:51.489 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:51.489 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:51.489 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:51.489 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:51.489 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:51.489 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:51.489 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:51.489 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:51.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febc12a8640 1 Processor -- start 2024-01-24T23:26:51.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febc12a8640 1 -- start start 2024-01-24T23:26:51.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febc12a8640 10 monclient: init 2024-01-24T23:26:51.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febc12a8640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:51.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febc12a8640 10 monclient: _add_conns ranks=[1,5,6,7,3,2,4,0,8] 2024-01-24T23:26:51.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febc12a8640 10 monclient(hunting): picked mon.noname-f con 0x7febbc133ed0 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:51.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febc12a8640 10 monclient(hunting): picked mon.noname-h con 0x7febbc1342b0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:51.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febc12a8640 10 monclient(hunting): picked mon.noname-d con 0x7febbc134690 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:51.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febc12a8640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febbc09ca90 con 0x7febbc133ed0 2024-01-24T23:26:51.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febc12a8640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febbc1206f0 con 0x7febbc1342b0 2024-01-24T23:26:51.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febc12a8640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febbc0a5af0 con 0x7febbc134690 2024-01-24T23:26:51.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febc12a8640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:51.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febc12a8640 10 monclient(hunting): authenticate will time out at 1298.804814s 2024-01-24T23:26:51.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febbaffd640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7febbc133ed0 legacy=0x7febbc132d30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:51.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febba7fc640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7febbc1342b0 0x7febbc1d0310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:55850/0 (socket says 172.21.15.132:55850) 2024-01-24T23:26:51.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febba7fc640 1 -- 172.21.15.132:0/4150013427 learned_addr learned my addr 172.21.15.132:0/4150013427 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:51.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 1 -- 172.21.15.132:0/4150013427 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3732685269 0 0) 0x7febbc1206f0 con 0x7febbc1342b0 2024-01-24T23:26:51.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 10 monclient(hunting): _negotiate 2024-01-24T23:26:51.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:51.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:51.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 10 monclient(hunting): my global_id is 6341 2024-01-24T23:26:51.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 1 -- 172.21.15.132:0/4150013427 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feba8003530 con 0x7febbc1342b0 2024-01-24T23:26:51.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 1 -- 172.21.15.132:0/4150013427 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 103487186 0 0) 0x7feba8003530 con 0x7febbc1342b0 2024-01-24T23:26:51.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 1 -- 172.21.15.132:0/4150013427 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7febbc1206f0 con 0x7febbc1342b0 2024-01-24T23:26:51.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 1 -- 172.21.15.132:0/4150013427 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7febb0001e60 con 0x7febbc1342b0 2024-01-24T23:26:51.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:51.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:51.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 10 monclient(hunting): dump: 2024-01-24T23:26:51.491 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:51.492 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:51.492 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:51.492 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:51.492 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:51.492 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:51.492 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:51.492 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:51.492 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:51.492 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:51.492 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:51.492 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:51.492 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:51.492 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:51.492 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:51.492 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:51.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:51.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 1 -- 172.21.15.132:0/4150013427 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3414737873 0 0) 0x7febbc1206f0 con 0x7febbc1342b0 2024-01-24T23:26:51.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 1 -- 172.21.15.132:0/4150013427 >> v1:172.21.15.115:6792/0 conn(0x7febbc134690 legacy=0x7febbc1d3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:51.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 1 -- 172.21.15.132:0/4150013427 >> v1:172.21.15.132:6789/0 conn(0x7febbc133ed0 legacy=0x7febbc132d30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:51.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 10 monclient: _finish_hunting 0 2024-01-24T23:26:51.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 1 monclient: found mon.h 2024-01-24T23:26:51.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:51.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:51.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 1 -- 172.21.15.132:0/4150013427 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febbc0a2450 con 0x7febbc1342b0 2024-01-24T23:26:51.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 10 monclient: _finish_auth 0 2024-01-24T23:26:51.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 10 monclient: _check_auth_tickets 2024-01-24T23:26:51.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:51.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febc12a8640 5 monclient: authenticate success, global_id 6341 2024-01-24T23:26:51.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febc12a8640 10 monclient: _renew_subs 2024-01-24T23:26:51.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febc12a8640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:51.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febc12a8640 1 -- 172.21.15.132:0/4150013427 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febbc120410 con 0x7febbc1342b0 2024-01-24T23:26:51.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febc12a8640 10 monclient: _renew_subs 2024-01-24T23:26:51.494 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febc12a8640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:51.494 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febc12a8640 1 -- 172.21.15.132:0/4150013427 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7febbc11fa40 con 0x7febbc1342b0 2024-01-24T23:26:51.494 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 1 -- 172.21.15.132:0/4150013427 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7febb0003c50 con 0x7febbc1342b0 2024-01-24T23:26:51.494 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:51.494 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 1 -- 172.21.15.132:0/4150013427 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7febb0004180 con 0x7febbc1342b0 2024-01-24T23:26:51.494 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:51.494 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febb97fa640 4 set_mon_vals no callback set 2024-01-24T23:26:51.494 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:51.494 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 10 monclient: dump: 2024-01-24T23:26:51.494 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:51.494 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:51.494 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:51.494 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:51.494 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:51.494 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:51.494 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:51.495 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:51.495 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:51.495 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:51.495 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:51.495 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:51.495 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:51.495 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:51.495 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:51.495 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:51.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febc12a8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:51.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febc12a8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:51.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febc12a8640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:51.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febc12a8640 1 -- 172.21.15.132:0/4150013427 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb9c005090 con 0x7febbc1342b0 2024-01-24T23:26:51.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 1 -- 172.21.15.132:0/4150013427 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7febb00042e0 con 0x7febbc1342b0 2024-01-24T23:26:51.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7feba37fe640 1 -- 172.21.15.132:0/4150013427 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(193..193 src has 1..193) v4 ==== 10288+0+0 (unknown 2591931990 0 0) 0x7febb0045580 con 0x7febbc1342b0 2024-01-24T23:26:51.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.479+0000 7febbaffd640 10 monclient: get_auth_request con 0x7feba8036e80 auth_method 0 2024-01-24T23:26:51.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.483+0000 7feba37fe640 1 -- 172.21.15.132:0/4150013427 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7febb0014c60 con 0x7febbc1342b0 2024-01-24T23:26:51.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.483+0000 7feba37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:51.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.483+0000 7feba37fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:51.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.643+0000 7febc12a8640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:51.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.643+0000 7febc12a8640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:51.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.643+0000 7febc12a8640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:51.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:51.643+0000 7febc12a8640 1 -- 172.21.15.132:0/4150013427 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7feb9c005340 con 0x7febbc1342b0 2024-01-24T23:26:52.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.223+0000 7feba37fe640 1 -- 172.21.15.132:0/4150013427 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}]=0 added bucket h15 type host to location {rack=r1} v194) v1 ==== 168+0+0 (unknown 2641591837 0 0) 0x7febb0013300 con 0x7febbc1342b0 2024-01-24T23:26:52.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.223+0000 7feba37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:52.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.223+0000 7feba37fe640 10 monclient: _finish_command 2 = system:0 added bucket h15 type host to location {rack=r1} 2024-01-24T23:26:52.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.327+0000 7febc12a8640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:52.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.327+0000 7febc12a8640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:52.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.327+0000 7febc12a8640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:52.329 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.327+0000 7febc12a8640 1 -- 172.21.15.132:0/4150013427 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7feb9c005a80 con 0x7febbc1342b0 2024-01-24T23:26:52.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.327+0000 7feba37fe640 1 -- 172.21.15.132:0/4150013427 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h15' already exists v194) v1 ==== 147+0+0 (unknown 254601427 0 0) 0x7febb0014a80 con 0x7febbc1342b0 2024-01-24T23:26:52.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.327+0000 7feba37fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:52.330 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.327+0000 7feba37fe640 10 monclient: _finish_command 3 = system:0 bucket 'h15' already exists 2024-01-24T23:26:52.331 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h15' already exists 2024-01-24T23:26:52.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.331+0000 7febc12a8640 1 -- 172.21.15.132:0/4150013427 >> v1:172.21.15.132:6812/1835401922 conn(0x7feba8036e80 legacy=0x7feba8039340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:52.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.331+0000 7febc12a8640 10 monclient: shutdown 2024-01-24T23:26:52.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.331+0000 7febc12a8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:52.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.331+0000 7febc12a8640 1 -- 172.21.15.132:0/4150013427 >> v1:172.21.15.132:6791/0 conn(0x7febbc1342b0 legacy=0x7febbc1d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:52.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.331+0000 7febc12a8640 1 -- 172.21.15.132:0/4150013427 shutdown_connections 2024-01-24T23:26:52.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.331+0000 7febc12a8640 1 -- 172.21.15.132:0/4150013427 >> 172.21.15.132:0/4150013427 conn(0x7febbc0a5ef0 msgr2=0x7febbc1bea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:52.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.331+0000 7febc12a8640 1 -- 172.21.15.132:0/4150013427 shutdown_connections 2024-01-24T23:26:52.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.331+0000 7febc12a8640 1 -- 172.21.15.132:0/4150013427 wait complete. 2024-01-24T23:26:52.347 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:26:52.347 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h16 host rack=r1 2024-01-24T23:26:52.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.459+0000 7f949a9d5640 10 monclient: get_monmap_and_config 2024-01-24T23:26:52.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.459+0000 7f949a9d5640 10 monclient: build_initial_monmap 2024-01-24T23:26:52.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.459+0000 7f949a9d5640 10 monclient: monmap: 2024-01-24T23:26:52.461 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:52.461 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:52.461 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:52.462795+0000 2024-01-24T23:26:52.461 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:52.462795+0000 2024-01-24T23:26:52.461 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:52.461 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:52.462 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:52.462 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:52.462 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:52.462 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:52.462 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:52.462 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:52.462 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:52.462 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:52.462 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:52.462 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:52.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.459+0000 7f949a9d5640 1 Processor -- start 2024-01-24T23:26:52.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.459+0000 7f949a9d5640 1 -- start start 2024-01-24T23:26:52.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.459+0000 7f949a9d5640 10 monclient: init 2024-01-24T23:26:52.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.459+0000 7f949a9d5640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:52.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.459+0000 7f949a9d5640 10 monclient: _add_conns ranks=[8,4,3,5,0,7,6,1,2] 2024-01-24T23:26:52.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.459+0000 7f949a9d5640 10 monclient(hunting): picked mon.noname-e con 0x7f9494133ed0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:52.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.459+0000 7f949a9d5640 10 monclient(hunting): picked mon.noname-c con 0x7f94941342b0 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:52.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.459+0000 7f949a9d5640 10 monclient(hunting): picked mon.noname-g con 0x7f9494134690 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:52.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.459+0000 7f949a9d5640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9494064700 con 0x7f9494134690 2024-01-24T23:26:52.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.459+0000 7f949a9d5640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f949409b3d0 con 0x7f94941342b0 2024-01-24T23:26:52.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.459+0000 7f949a9d5640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94940a58c0 con 0x7f9494133ed0 2024-01-24T23:26:52.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.459+0000 7f949a9d5640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:52.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.459+0000 7f949a9d5640 10 monclient(hunting): authenticate will time out at 1299.787773s 2024-01-24T23:26:52.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.459+0000 7f9498f72640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f9494134690 0x7f949413f770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:56234/0 (socket says 172.21.15.132:56234) 2024-01-24T23:26:52.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.459+0000 7f9498f72640 1 -- 172.21.15.132:0/3318137277 learned_addr learned my addr 172.21.15.132:0/3318137277 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:52.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 1 -- 172.21.15.132:0/3318137277 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 581599977 0 0) 0x7f9494064700 con 0x7f9494134690 2024-01-24T23:26:52.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:52.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:52.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:52.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 10 monclient(hunting): my global_id is 6420 2024-01-24T23:26:52.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 1 -- 172.21.15.132:0/3318137277 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9480003530 con 0x7f9494134690 2024-01-24T23:26:52.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 1 -- 172.21.15.132:0/3318137277 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3728035714 0 0) 0x7f9480003530 con 0x7f9494134690 2024-01-24T23:26:52.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 1 -- 172.21.15.132:0/3318137277 >> v1:172.21.15.115:6793/0 conn(0x7f9494133ed0 legacy=0x7f949409bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:52.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 1 -- 172.21.15.132:0/3318137277 >> v1:172.21.15.115:6791/0 conn(0x7f94941342b0 legacy=0x7f949413c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:52.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:26:52.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 1 monclient: found mon.noname-g 2024-01-24T23:26:52.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:52.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:26:52.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 1 -- 172.21.15.132:0/3318137277 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949409b0f0 con 0x7f9494134690 2024-01-24T23:26:52.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 10 monclient: _finish_auth 0 2024-01-24T23:26:52.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:26:52.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:52.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 1 -- 172.21.15.132:0/3318137277 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f948c0017c0 con 0x7f9494134690 2024-01-24T23:26:52.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 5 monclient: authenticate success, global_id 6420 2024-01-24T23:26:52.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:52.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:52.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:52.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 10 monclient: dump: 2024-01-24T23:26:52.472 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:52.472 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:52.472 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:52.472 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:52.473 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:52.473 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:52.473 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:52.473 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:52.473 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:52.473 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:52.474 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:52.474 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:52.474 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:52.474 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:52.474 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:52.474 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:52.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:52.474 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 1 -- 172.21.15.132:0/3318137277 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f948c001a60 con 0x7f9494134690 2024-01-24T23:26:52.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:52.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 1 -- 172.21.15.132:0/3318137277 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f948c004bf0 con 0x7f9494134690 2024-01-24T23:26:52.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:52.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 4 set_mon_vals no callback set 2024-01-24T23:26:52.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:52.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:52.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9492ffd640 10 monclient: dump: 2024-01-24T23:26:52.476 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:52.476 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:52.476 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:52.476 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:52.476 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:52.476 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:52.476 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:52.476 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:52.476 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:52.476 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:52.476 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:52.476 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:52.476 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:52.476 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:52.477 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:52.477 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:52.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 10 monclient: shutdown 2024-01-24T23:26:52.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:52.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 1 -- 172.21.15.132:0/3318137277 >> v1:172.21.15.132:6790/0 conn(0x7f9494134690 legacy=0x7f949413f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:52.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 1 -- 172.21.15.132:0/3318137277 shutdown_connections 2024-01-24T23:26:52.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 1 -- 172.21.15.132:0/3318137277 >> 172.21.15.132:0/3318137277 conn(0x7f94940a5ef0 msgr2=0x7f94940a6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:52.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 1 -- 172.21.15.132:0/3318137277 shutdown_connections 2024-01-24T23:26:52.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 1 -- 172.21.15.132:0/3318137277 wait complete. 2024-01-24T23:26:52.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 10 monclient: build_initial_monmap 2024-01-24T23:26:52.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 10 monclient: monmap: 2024-01-24T23:26:52.481 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:52.481 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:52.481 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:52.482 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:52.482 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:52.482 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:52.482 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:52.482 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:52.482 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:52.483 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:52.483 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:52.483 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:52.483 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:52.483 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:52.483 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:52.483 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:52.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 1 Processor -- start 2024-01-24T23:26:52.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 1 -- start start 2024-01-24T23:26:52.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 10 monclient: init 2024-01-24T23:26:52.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:52.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 10 monclient: _add_conns ranks=[2,1,3,7,0,5,8,6,4] 2024-01-24T23:26:52.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 10 monclient(hunting): picked mon.noname-b con 0x7f9494133ed0 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:52.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 10 monclient(hunting): picked mon.noname-f con 0x7f94941342b0 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:52.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 10 monclient(hunting): picked mon.noname-g con 0x7f9494134690 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:52.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9494123880 con 0x7f94941342b0 2024-01-24T23:26:52.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9494121a30 con 0x7f9494133ed0 2024-01-24T23:26:52.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f949409ca90 con 0x7f9494134690 2024-01-24T23:26:52.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:52.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 10 monclient(hunting): authenticate will time out at 1299.791076s 2024-01-24T23:26:52.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f94937fe640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f94941342b0 legacy=0x7f94941d02d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:52.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9493fff640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f9494133ed0 legacy=0x7f94941330a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:52.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9498f72640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f9494134690 0x7f94941d3a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:56236/0 (socket says 172.21.15.132:56236) 2024-01-24T23:26:52.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9498f72640 1 -- 172.21.15.132:0/445970829 learned_addr learned my addr 172.21.15.132:0/445970829 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:52.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9490ff9640 1 -- 172.21.15.132:0/445970829 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2963543435 0 0) 0x7f949409ca90 con 0x7f9494134690 2024-01-24T23:26:52.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9490ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:26:52.487 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9490ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:52.487 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9490ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:52.487 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9490ff9640 10 monclient(hunting): my global_id is 6429 2024-01-24T23:26:52.487 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9490ff9640 1 -- 172.21.15.132:0/445970829 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9474003530 con 0x7f9494134690 2024-01-24T23:26:52.487 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9490ff9640 1 -- 172.21.15.132:0/445970829 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 540773256 0 0) 0x7f9474003530 con 0x7f9494134690 2024-01-24T23:26:52.487 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9490ff9640 1 -- 172.21.15.132:0/445970829 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f949409ca90 con 0x7f9494134690 2024-01-24T23:26:52.487 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9490ff9640 1 -- 172.21.15.132:0/445970829 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f948c0016c0 con 0x7f9494134690 2024-01-24T23:26:52.487 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9490ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:52.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9490ff9640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:52.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9490ff9640 10 monclient(hunting): dump: 2024-01-24T23:26:52.488 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:52.488 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:52.488 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:52.488 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:52.489 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:52.489 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:52.489 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:52.489 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:52.489 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:52.489 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:52.489 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:52.490 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:52.490 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:52.490 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:52.490 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:52.490 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:52.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9490ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:52.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9490ff9640 1 -- 172.21.15.132:0/445970829 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 531110456 0 0) 0x7f949409ca90 con 0x7f9494134690 2024-01-24T23:26:52.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9490ff9640 1 -- 172.21.15.132:0/445970829 >> v1:172.21.15.115:6790/0 conn(0x7f9494133ed0 legacy=0x7f94941330a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:52.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9490ff9640 1 -- 172.21.15.132:0/445970829 >> v1:172.21.15.132:6789/0 conn(0x7f94941342b0 legacy=0x7f94941d02d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:52.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9490ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:26:52.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9490ff9640 1 monclient: found mon.g 2024-01-24T23:26:52.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9490ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:52.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9490ff9640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:52.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9490ff9640 1 -- 172.21.15.132:0/445970829 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94940a5af0 con 0x7f9494134690 2024-01-24T23:26:52.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9490ff9640 10 monclient: _finish_auth 0 2024-01-24T23:26:52.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9490ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:26:52.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f9490ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:52.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 5 monclient: authenticate success, global_id 6429 2024-01-24T23:26:52.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 10 monclient: _renew_subs 2024-01-24T23:26:52.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:52.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 1 -- 172.21.15.132:0/445970829 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9494121590 con 0x7f9494134690 2024-01-24T23:26:52.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.463+0000 7f949a9d5640 10 monclient: _renew_subs 2024-01-24T23:26:52.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.467+0000 7f949a9d5640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:52.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.467+0000 7f949a9d5640 1 -- 172.21.15.132:0/445970829 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f94941337b0 con 0x7f9494134690 2024-01-24T23:26:52.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.467+0000 7f9490ff9640 1 -- 172.21.15.132:0/445970829 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f948c001100 con 0x7f9494134690 2024-01-24T23:26:52.494 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.467+0000 7f9490ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:52.494 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.467+0000 7f9490ff9640 1 -- 172.21.15.132:0/445970829 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f948c005160 con 0x7f9494134690 2024-01-24T23:26:52.494 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.467+0000 7f9490ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:52.494 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.467+0000 7f94927fc640 4 set_mon_vals no callback set 2024-01-24T23:26:52.494 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.467+0000 7f9490ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:52.494 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.467+0000 7f9490ff9640 10 monclient: dump: 2024-01-24T23:26:52.494 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:52.495 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:52.495 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:52.495 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:52.495 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:52.495 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:52.495 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:52.496 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:52.496 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:52.496 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:52.496 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:52.496 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:52.496 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:52.497 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:52.497 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:52.497 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:52.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.467+0000 7f949a9d5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:52.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.467+0000 7f949a9d5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:52.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.467+0000 7f949a9d5640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:52.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.467+0000 7f949a9d5640 1 -- 172.21.15.132:0/445970829 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9478005090 con 0x7f9494134690 2024-01-24T23:26:52.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.467+0000 7f9490ff9640 1 -- 172.21.15.132:0/445970829 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f948c00fbe0 con 0x7f9494134690 2024-01-24T23:26:52.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.467+0000 7f9490ff9640 1 -- 172.21.15.132:0/445970829 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(194..194 src has 1..194) v4 ==== 10501+0+0 (unknown 2255757555 0 0) 0x7f948c045340 con 0x7f9494134690 2024-01-24T23:26:52.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.467+0000 7f9493fff640 10 monclient: get_auth_request con 0x7f9474036f10 auth_method 0 2024-01-24T23:26:52.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.471+0000 7f9490ff9640 1 -- 172.21.15.132:0/445970829 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f948c014a70 con 0x7f9494134690 2024-01-24T23:26:52.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.471+0000 7f9490ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:52.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.471+0000 7f9490ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:52.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.627+0000 7f949a9d5640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:52.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.627+0000 7f949a9d5640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:52.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.627+0000 7f949a9d5640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:52.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:52.627+0000 7f949a9d5640 1 -- 172.21.15.132:0/445970829 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f94780058c0 con 0x7f9494134690 2024-01-24T23:26:53.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.251+0000 7f9490ff9640 1 -- 172.21.15.132:0/445970829 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}]=0 added bucket h16 type host to location {rack=r1} v195) v1 ==== 168+0+0 (unknown 2842773655 0 0) 0x7f948c013110 con 0x7f9494134690 2024-01-24T23:26:53.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.251+0000 7f9490ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:53.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.251+0000 7f9490ff9640 10 monclient: _finish_command 2 = system:0 added bucket h16 type host to location {rack=r1} 2024-01-24T23:26:53.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.355+0000 7f949a9d5640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:53.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.355+0000 7f949a9d5640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:53.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.355+0000 7f949a9d5640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:53.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.355+0000 7f949a9d5640 1 -- 172.21.15.132:0/445970829 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f9478002220 con 0x7f9494134690 2024-01-24T23:26:53.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.355+0000 7f9490ff9640 1 -- 172.21.15.132:0/445970829 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h16' already exists v195) v1 ==== 147+0+0 (unknown 983942413 0 0) 0x7f948c014890 con 0x7f9494134690 2024-01-24T23:26:53.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.355+0000 7f9490ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:53.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.355+0000 7f9490ff9640 10 monclient: _finish_command 3 = system:0 bucket 'h16' already exists 2024-01-24T23:26:53.358 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h16' already exists 2024-01-24T23:26:53.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.359+0000 7f949a9d5640 1 -- 172.21.15.132:0/445970829 >> v1:172.21.15.132:6812/1835401922 conn(0x7f9474036f10 legacy=0x7f94740393d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:53.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.359+0000 7f949a9d5640 10 monclient: shutdown 2024-01-24T23:26:53.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.359+0000 7f949a9d5640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:53.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.359+0000 7f949a9d5640 1 -- 172.21.15.132:0/445970829 >> v1:172.21.15.132:6790/0 conn(0x7f9494134690 legacy=0x7f94941d3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:53.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.359+0000 7f949a9d5640 1 -- 172.21.15.132:0/445970829 shutdown_connections 2024-01-24T23:26:53.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.359+0000 7f949a9d5640 1 -- 172.21.15.132:0/445970829 >> 172.21.15.132:0/445970829 conn(0x7f94940a5ef0 msgr2=0x7f94940a72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:53.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.359+0000 7f949a9d5640 1 -- 172.21.15.132:0/445970829 shutdown_connections 2024-01-24T23:26:53.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.359+0000 7f949a9d5640 1 -- 172.21.15.132:0/445970829 wait complete. 2024-01-24T23:26:53.374 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:26:53.374 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h17 host rack=r1 2024-01-24T23:26:53.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3f0e46640 10 monclient: get_monmap_and_config 2024-01-24T23:26:53.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3f0e46640 10 monclient: build_initial_monmap 2024-01-24T23:26:53.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3f0e46640 10 monclient: monmap: 2024-01-24T23:26:53.493 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:53.493 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:53.493 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:53.494551+0000 2024-01-24T23:26:53.493 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:53.494551+0000 2024-01-24T23:26:53.493 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:53.493 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:53.493 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:53.493 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:53.493 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:53.493 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:53.493 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:53.493 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:53.493 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:53.494 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:53.494 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:53.494 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:53.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3f0e46640 1 Processor -- start 2024-01-24T23:26:53.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3f0e46640 1 -- start start 2024-01-24T23:26:53.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3f0e46640 10 monclient: init 2024-01-24T23:26:53.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3f0e46640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:53.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3f0e46640 10 monclient: _add_conns ranks=[5,2,8,0,1,7,4,6,3] 2024-01-24T23:26:53.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3f0e46640 10 monclient(hunting): picked mon.noname-h con 0x7fa3ec09b810 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:53.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3f0e46640 10 monclient(hunting): picked mon.noname-b con 0x7fa3ec09bbf0 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:53.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3f0e46640 10 monclient(hunting): picked mon.noname-e con 0x7fa3ec1370a0 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:53.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3f0e46640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3ec064700 con 0x7fa3ec09bbf0 2024-01-24T23:26:53.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3f0e46640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3ec099af0 con 0x7fa3ec09b810 2024-01-24T23:26:53.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3f0e46640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3ec072110 con 0x7fa3ec1370a0 2024-01-24T23:26:53.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3f0e46640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:53.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3f0e46640 10 monclient(hunting): authenticate will time out at 1300.819149s 2024-01-24T23:26:53.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3ea59c640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fa3ec09b810 0x7fa3ec0712f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:55874/0 (socket says 172.21.15.132:55874) 2024-01-24T23:26:53.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3ea59c640 1 -- 172.21.15.132:0/3094631537 learned_addr learned my addr 172.21.15.132:0/3094631537 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:53.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3e9d9b640 1 -- 172.21.15.132:0/3094631537 >> v1:172.21.15.115:6790/0 conn(0x7fa3ec09bbf0 legacy=0x7fa3ec071a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:53.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3e959a640 1 -- 172.21.15.132:0/3094631537 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1101787799 0 0) 0x7fa3ec099af0 con 0x7fa3ec09b810 2024-01-24T23:26:53.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3e959a640 10 monclient(hunting): _negotiate 2024-01-24T23:26:53.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3e959a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:53.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3e959a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:53.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3e959a640 10 monclient(hunting): my global_id is 6350 2024-01-24T23:26:53.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3e959a640 1 -- 172.21.15.132:0/3094631537 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3d4003530 con 0x7fa3ec09b810 2024-01-24T23:26:53.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3e959a640 1 -- 172.21.15.132:0/3094631537 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 739536393 0 0) 0x7fa3d4003530 con 0x7fa3ec09b810 2024-01-24T23:26:53.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3e959a640 1 -- 172.21.15.132:0/3094631537 >> v1:172.21.15.115:6793/0 conn(0x7fa3ec1370a0 legacy=0x7fa3ec13ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:53.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3e959a640 1 -- 172.21.15.132:0/3094631537 >> v1:172.21.15.115:6790/0 conn(0x7fa3ec09bbf0 legacy=0x7fa3ec071a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:53.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3e959a640 10 monclient: _finish_hunting 0 2024-01-24T23:26:53.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3e959a640 1 monclient: found mon.noname-h 2024-01-24T23:26:53.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3e959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:53.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3e959a640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:26:53.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3e959a640 1 -- 172.21.15.132:0/3094631537 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ec09c050 con 0x7fa3ec09b810 2024-01-24T23:26:53.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3e959a640 10 monclient: _finish_auth 0 2024-01-24T23:26:53.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3e959a640 10 monclient: _check_auth_tickets 2024-01-24T23:26:53.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3e959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:53.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3e959a640 1 -- 172.21.15.132:0/3094631537 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa3dc001b40 con 0x7fa3ec09b810 2024-01-24T23:26:53.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3f0e46640 5 monclient: authenticate success, global_id 6350 2024-01-24T23:26:53.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3e959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:53.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3e959a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:53.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3e959a640 10 monclient: dump: 2024-01-24T23:26:53.504 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:53.504 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:53.504 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:53.504 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:53.504 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:53.504 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:53.504 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:53.505 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:53.505 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:53.505 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:53.505 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:53.505 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:53.505 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:53.505 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:53.505 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:53.505 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:53.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.491+0000 7fa3f0e46640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:53.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3e959a640 1 -- 172.21.15.132:0/3094631537 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa3dc001df0 con 0x7fa3ec09b810 2024-01-24T23:26:53.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3e959a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:53.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:53.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 4 set_mon_vals no callback set 2024-01-24T23:26:53.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3e959a640 1 -- 172.21.15.132:0/3094631537 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa3dc004f70 con 0x7fa3ec09b810 2024-01-24T23:26:53.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3e959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:53.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3e959a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:53.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3e959a640 10 monclient: dump: 2024-01-24T23:26:53.506 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:53.506 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:53.506 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:53.506 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:53.506 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:53.506 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:53.506 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:53.506 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:53.506 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:53.506 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:53.506 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:53.506 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:53.506 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:53.507 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:53.507 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:53.507 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:53.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 10 monclient: shutdown 2024-01-24T23:26:53.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:53.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 1 -- 172.21.15.132:0/3094631537 >> v1:172.21.15.132:6791/0 conn(0x7fa3ec09b810 legacy=0x7fa3ec0712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:53.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 1 -- 172.21.15.132:0/3094631537 shutdown_connections 2024-01-24T23:26:53.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 1 -- 172.21.15.132:0/3094631537 >> 172.21.15.132:0/3094631537 conn(0x7fa3ec0a1bc0 msgr2=0x7fa3ec0a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:53.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 1 -- 172.21.15.132:0/3094631537 shutdown_connections 2024-01-24T23:26:53.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 1 -- 172.21.15.132:0/3094631537 wait complete. 2024-01-24T23:26:53.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 10 monclient: build_initial_monmap 2024-01-24T23:26:53.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 10 monclient: monmap: 2024-01-24T23:26:53.507 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:53.507 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:53.507 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:53.508 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:53.508 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:53.508 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:53.508 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:53.508 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:53.508 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:53.508 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:53.508 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:53.508 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:53.508 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:53.508 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:53.508 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:53.508 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:53.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 1 Processor -- start 2024-01-24T23:26:53.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 1 -- start start 2024-01-24T23:26:53.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 10 monclient: init 2024-01-24T23:26:53.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:53.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 10 monclient: _add_conns ranks=[6,5,7,3,8,4,1,2,0] 2024-01-24T23:26:53.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 10 monclient(hunting): picked mon.noname-d con 0x7fa3ec09bbf0 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:53.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 10 monclient(hunting): picked mon.noname-h con 0x7fa3ec1370a0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:53.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 10 monclient(hunting): picked mon.noname-i con 0x7fa3ec130c50 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:53.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3ec1cba60 con 0x7fa3ec1370a0 2024-01-24T23:26:53.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3ec09b5e0 con 0x7fa3ec09bbf0 2024-01-24T23:26:53.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3ec1332d0 con 0x7fa3ec130c50 2024-01-24T23:26:53.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:53.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 10 monclient(hunting): authenticate will time out at 1300.822408s 2024-01-24T23:26:53.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3ead9d640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fa3ec130c50 legacy=0x7fa3ec1d36b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:53.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3e9d9b640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fa3ec1370a0 0x7fa3ec130540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:55884/0 (socket says 172.21.15.132:55884) 2024-01-24T23:26:53.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3e9d9b640 1 -- 172.21.15.132:0/1073979626 learned_addr learned my addr 172.21.15.132:0/1073979626 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:53.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 1 -- 172.21.15.132:0/1073979626 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3315868062 0 0) 0x7fa3ec1cba60 con 0x7fa3ec1370a0 2024-01-24T23:26:53.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:53.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:53.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:53.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 10 monclient(hunting): my global_id is 6359 2024-01-24T23:26:53.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 1 -- 172.21.15.132:0/1073979626 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3cc003530 con 0x7fa3ec1370a0 2024-01-24T23:26:53.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 1 -- 172.21.15.132:0/1073979626 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1404263409 0 0) 0x7fa3cc003530 con 0x7fa3ec1370a0 2024-01-24T23:26:53.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 1 -- 172.21.15.132:0/1073979626 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa3ec1cba60 con 0x7fa3ec1370a0 2024-01-24T23:26:53.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 1 -- 172.21.15.132:0/1073979626 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa3e0004330 con 0x7fa3ec1370a0 2024-01-24T23:26:53.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:53.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:53.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 10 monclient(hunting): dump: 2024-01-24T23:26:53.510 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:53.510 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:53.511 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:53.511 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:53.511 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:53.511 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:53.511 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:53.511 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:53.511 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:53.511 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:53.511 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:53.511 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:53.511 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:53.511 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:53.511 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:53.511 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:53.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:53.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 1 -- 172.21.15.132:0/1073979626 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1934685322 0 0) 0x7fa3ec09b5e0 con 0x7fa3ec09bbf0 2024-01-24T23:26:53.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:53.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:53.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:53.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 10 monclient(hunting): my global_id is 6567 2024-01-24T23:26:53.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 1 -- 172.21.15.132:0/1073979626 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3cc003530 con 0x7fa3ec09bbf0 2024-01-24T23:26:53.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 1 -- 172.21.15.132:0/1073979626 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3290702127 0 0) 0x7fa3ec1cba60 con 0x7fa3ec1370a0 2024-01-24T23:26:53.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 1 -- 172.21.15.132:0/1073979626 >> v1:172.21.15.132:6792/0 conn(0x7fa3ec130c50 legacy=0x7fa3ec1d36b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:53.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 1 -- 172.21.15.132:0/1073979626 >> v1:172.21.15.115:6792/0 conn(0x7fa3ec09bbf0 legacy=0x7fa3ec1cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:53.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 10 monclient: _finish_hunting 0 2024-01-24T23:26:53.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 1 monclient: found mon.h 2024-01-24T23:26:53.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:53.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:53.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 1 -- 172.21.15.132:0/1073979626 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ec12afd0 con 0x7fa3ec1370a0 2024-01-24T23:26:53.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 10 monclient: _finish_auth 0 2024-01-24T23:26:53.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 10 monclient: _check_auth_tickets 2024-01-24T23:26:53.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:53.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 5 monclient: authenticate success, global_id 6359 2024-01-24T23:26:53.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 10 monclient: _renew_subs 2024-01-24T23:26:53.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:53.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 1 -- 172.21.15.132:0/1073979626 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ec12a730 con 0x7fa3ec1370a0 2024-01-24T23:26:53.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 10 monclient: _renew_subs 2024-01-24T23:26:53.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:53.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3f0e46640 1 -- 172.21.15.132:0/1073979626 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ec134000 con 0x7fa3ec1370a0 2024-01-24T23:26:53.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 1 -- 172.21.15.132:0/1073979626 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa3e0004a50 con 0x7fa3ec1370a0 2024-01-24T23:26:53.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:53.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 1 -- 172.21.15.132:0/1073979626 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa3e0004f00 con 0x7fa3ec1370a0 2024-01-24T23:26:53.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:53.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3e8d99640 4 set_mon_vals no callback set 2024-01-24T23:26:53.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:53.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.495+0000 7fa3daffd640 10 monclient: dump: 2024-01-24T23:26:53.514 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:53.514 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:53.514 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:53.514 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:53.514 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:53.514 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:53.514 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:53.514 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:53.514 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:53.514 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:53.514 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:53.514 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:53.515 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:53.515 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:53.515 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:53.515 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:53.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.499+0000 7fa3daffd640 1 -- 172.21.15.132:0/1073979626 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fa3e000f980 con 0x7fa3ec1370a0 2024-01-24T23:26:53.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.499+0000 7fa3f0e46640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:53.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.499+0000 7fa3f0e46640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:53.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.499+0000 7fa3f0e46640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:53.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.499+0000 7fa3f0e46640 1 -- 172.21.15.132:0/1073979626 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d0005090 con 0x7fa3ec1370a0 2024-01-24T23:26:53.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.499+0000 7fa3daffd640 1 -- 172.21.15.132:0/1073979626 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(195..195 src has 1..195) v4 ==== 10970+0+0 (unknown 2302370765 0 0) 0x7fa3e0045480 con 0x7fa3ec1370a0 2024-01-24T23:26:53.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.499+0000 7fa3ea59c640 10 monclient: get_auth_request con 0x7fa3cc036fb0 auth_method 0 2024-01-24T23:26:53.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.503+0000 7fa3daffd640 1 -- 172.21.15.132:0/1073979626 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fa3e00148b0 con 0x7fa3ec1370a0 2024-01-24T23:26:53.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.503+0000 7fa3daffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:53.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.503+0000 7fa3daffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:53.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.659+0000 7fa3f0e46640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:53.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.659+0000 7fa3f0e46640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:53.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.659+0000 7fa3f0e46640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:53.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:53.659+0000 7fa3f0e46640 1 -- 172.21.15.132:0/1073979626 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fa3d00058c0 con 0x7fa3ec1370a0 2024-01-24T23:26:53.919 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~50s 2024-01-24T23:26:54.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.259+0000 7fa3daffd640 1 -- 172.21.15.132:0/1073979626 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}]=0 added bucket h17 type host to location {rack=r1} v196) v1 ==== 168+0+0 (unknown 13726220 0 0) 0x7fa3e0012f50 con 0x7fa3ec1370a0 2024-01-24T23:26:54.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.259+0000 7fa3daffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:54.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.259+0000 7fa3daffd640 10 monclient: _finish_command 2 = system:0 added bucket h17 type host to location {rack=r1} 2024-01-24T23:26:54.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.363+0000 7fa3f0e46640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:54.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.363+0000 7fa3f0e46640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:54.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.363+0000 7fa3f0e46640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:54.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.363+0000 7fa3f0e46640 1 -- 172.21.15.132:0/1073979626 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fa3d0002220 con 0x7fa3ec1370a0 2024-01-24T23:26:54.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.363+0000 7fa3daffd640 1 -- 172.21.15.132:0/1073979626 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h17' already exists v196) v1 ==== 147+0+0 (unknown 1256641755 0 0) 0x7fa3e00146d0 con 0x7fa3ec1370a0 2024-01-24T23:26:54.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.363+0000 7fa3daffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:54.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.363+0000 7fa3daffd640 10 monclient: _finish_command 3 = system:0 bucket 'h17' already exists 2024-01-24T23:26:54.365 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h17' already exists 2024-01-24T23:26:54.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.367+0000 7fa3f0e46640 1 -- 172.21.15.132:0/1073979626 >> v1:172.21.15.132:6812/1835401922 conn(0x7fa3cc036fb0 legacy=0x7fa3cc039470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:54.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.367+0000 7fa3f0e46640 10 monclient: shutdown 2024-01-24T23:26:54.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.367+0000 7fa3f0e46640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:54.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.367+0000 7fa3f0e46640 1 -- 172.21.15.132:0/1073979626 >> v1:172.21.15.132:6791/0 conn(0x7fa3ec1370a0 legacy=0x7fa3ec130540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:54.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.367+0000 7fa3f0e46640 1 -- 172.21.15.132:0/1073979626 shutdown_connections 2024-01-24T23:26:54.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.367+0000 7fa3f0e46640 1 -- 172.21.15.132:0/1073979626 >> 172.21.15.132:0/1073979626 conn(0x7fa3ec0a1bc0 msgr2=0x7fa3ec0a0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:54.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.367+0000 7fa3f0e46640 1 -- 172.21.15.132:0/1073979626 shutdown_connections 2024-01-24T23:26:54.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.367+0000 7fa3f0e46640 1 -- 172.21.15.132:0/1073979626 wait complete. 2024-01-24T23:26:54.383 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:26:54.383 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h18 host rack=r1 2024-01-24T23:26:54.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d60df8640 10 monclient: get_monmap_and_config 2024-01-24T23:26:54.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d60df8640 10 monclient: build_initial_monmap 2024-01-24T23:26:54.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d60df8640 10 monclient: monmap: 2024-01-24T23:26:54.503 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:54.503 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:54.503 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:54.504915+0000 2024-01-24T23:26:54.503 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:54.504915+0000 2024-01-24T23:26:54.503 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:54.504 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:54.504 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:54.504 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:54.504 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:54.504 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:54.504 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:54.504 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:54.504 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:54.504 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:54.504 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:54.504 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:54.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d60df8640 1 Processor -- start 2024-01-24T23:26:54.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d60df8640 1 -- start start 2024-01-24T23:26:54.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d60df8640 10 monclient: init 2024-01-24T23:26:54.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d60df8640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:54.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d60df8640 10 monclient: _add_conns ranks=[8,4,0,5,2,7,1,3,6] 2024-01-24T23:26:54.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d60df8640 10 monclient(hunting): picked mon.noname-e con 0x7f8d5c0a2a60 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:54.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d60df8640 10 monclient(hunting): picked mon.noname-c con 0x7f8d5c09f170 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:54.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d60df8640 10 monclient(hunting): picked mon.noname-a con 0x7f8d5c0a1130 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:54.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d60df8640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d5c064700 con 0x7f8d5c0a1130 2024-01-24T23:26:54.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d60df8640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d5c09f550 con 0x7f8d5c09f170 2024-01-24T23:26:54.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d60df8640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d5c0a0750 con 0x7f8d5c0a2a60 2024-01-24T23:26:54.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d60df8640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:54.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d60df8640 10 monclient(hunting): authenticate will time out at 1301.829671s 2024-01-24T23:26:54.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d59d9b640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f8d5c09f170 0x7f8d5c139d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:55280/0 (socket says 172.21.15.132:55280) 2024-01-24T23:26:54.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d59d9b640 1 -- 172.21.15.132:0/1996370309 learned_addr learned my addr 172.21.15.132:0/1996370309 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:54.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5ad9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f8d5c0a1130 0x7f8d5c13d440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:41410/0 (socket says 172.21.15.132:41410) 2024-01-24T23:26:54.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5a59c640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f8d5c0a2a60 0x7f8d5c09ea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:51602/0 (socket says 172.21.15.132:51602) 2024-01-24T23:26:54.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 1 -- 172.21.15.132:0/1996370309 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695420466 0 0) 0x7f8d5c0a0750 con 0x7f8d5c0a2a60 2024-01-24T23:26:54.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 10 monclient(hunting): _negotiate 2024-01-24T23:26:54.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:54.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:54.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 10 monclient(hunting): my global_id is 85679 2024-01-24T23:26:54.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 1 -- 172.21.15.132:0/1996370309 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d3c003530 con 0x7f8d5c0a2a60 2024-01-24T23:26:54.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 1 -- 172.21.15.132:0/1996370309 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 800504239 0 0) 0x7f8d44001120 con 0x7f8d5c09f170 2024-01-24T23:26:54.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 10 monclient(hunting): _negotiate 2024-01-24T23:26:54.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:54.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:54.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 10 monclient(hunting): my global_id is 94594 2024-01-24T23:26:54.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 1 -- 172.21.15.132:0/1996370309 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d5c0a0750 con 0x7f8d5c09f170 2024-01-24T23:26:54.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 1 -- 172.21.15.132:0/1996370309 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1699356340 0 0) 0x7f8d5c064700 con 0x7f8d5c0a1130 2024-01-24T23:26:54.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 10 monclient(hunting): _negotiate 2024-01-24T23:26:54.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:54.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:54.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 10 monclient(hunting): my global_id is 94779 2024-01-24T23:26:54.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 1 -- 172.21.15.132:0/1996370309 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d44001120 con 0x7f8d5c0a1130 2024-01-24T23:26:54.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 1 -- 172.21.15.132:0/1996370309 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1635470799 0 0) 0x7f8d3c003530 con 0x7f8d5c0a2a60 2024-01-24T23:26:54.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 1 -- 172.21.15.132:0/1996370309 >> v1:172.21.15.115:6791/0 conn(0x7f8d5c09f170 legacy=0x7f8d5c139d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:54.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 1 -- 172.21.15.132:0/1996370309 >> v1:172.21.15.115:6789/0 conn(0x7f8d5c0a1130 legacy=0x7f8d5c13d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:54.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 10 monclient: _finish_hunting 0 2024-01-24T23:26:54.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 1 monclient: found mon.noname-e 2024-01-24T23:26:54.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:54.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:26:54.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 1 -- 172.21.15.132:0/1996370309 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5c0a2e40 con 0x7f8d5c0a2a60 2024-01-24T23:26:54.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 10 monclient: _finish_auth 0 2024-01-24T23:26:54.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 10 monclient: _check_auth_tickets 2024-01-24T23:26:54.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:54.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 1 -- 172.21.15.132:0/1996370309 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8d54001b40 con 0x7f8d5c0a2a60 2024-01-24T23:26:54.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:54.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:26:54.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 10 monclient: dump: 2024-01-24T23:26:54.515 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:54.515 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:54.515 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:54.515 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:54.515 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:54.515 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:54.515 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:54.515 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:54.516 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:54.516 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:54.516 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:54.516 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:54.516 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:54.516 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:54.516 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:54.516 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:54.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d60df8640 5 monclient: authenticate success, global_id 85679 2024-01-24T23:26:54.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d60df8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:54.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 1 -- 172.21.15.132:0/1996370309 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8d54001e20 con 0x7f8d5c0a2a60 2024-01-24T23:26:54.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:54.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d60df8640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:54.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d60df8640 4 set_mon_vals no callback set 2024-01-24T23:26:54.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 1 -- 172.21.15.132:0/1996370309 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8d54004eb0 con 0x7f8d5c0a2a60 2024-01-24T23:26:54.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:54.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:26:54.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.503+0000 7f8d5959a640 10 monclient: dump: 2024-01-24T23:26:54.517 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:54.517 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:54.517 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:54.517 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:54.517 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:54.517 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:54.517 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:54.518 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:54.518 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:54.518 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:54.518 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:54.518 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:54.518 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:54.518 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:54.518 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:54.518 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:54.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 10 monclient: shutdown 2024-01-24T23:26:54.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:54.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 1 -- 172.21.15.132:0/1996370309 >> v1:172.21.15.115:6793/0 conn(0x7f8d5c0a2a60 legacy=0x7f8d5c09ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:54.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 1 -- 172.21.15.132:0/1996370309 shutdown_connections 2024-01-24T23:26:54.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 1 -- 172.21.15.132:0/1996370309 >> 172.21.15.132:0/1996370309 conn(0x7f8d5c0a7990 msgr2=0x7f8d5c0a3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:54.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 1 -- 172.21.15.132:0/1996370309 shutdown_connections 2024-01-24T23:26:54.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 1 -- 172.21.15.132:0/1996370309 wait complete. 2024-01-24T23:26:54.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 10 monclient: build_initial_monmap 2024-01-24T23:26:54.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 10 monclient: monmap: 2024-01-24T23:26:54.519 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:54.519 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:54.519 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:54.519 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:54.519 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:54.519 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:54.519 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:54.519 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:54.519 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:54.520 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:54.520 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:54.520 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:54.520 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:54.520 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:54.520 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:54.520 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:54.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 1 Processor -- start 2024-01-24T23:26:54.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 1 -- start start 2024-01-24T23:26:54.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 10 monclient: init 2024-01-24T23:26:54.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:54.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 10 monclient: _add_conns ranks=[8,3,7,5,4,6,1,0,2] 2024-01-24T23:26:54.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 10 monclient(hunting): picked mon.noname-e con 0x7f8d5c09f170 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:54.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 10 monclient(hunting): picked mon.noname-g con 0x7f8d5c0a1130 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:54.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 10 monclient(hunting): picked mon.noname-i con 0x7f8d5c0a2a60 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:54.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d5c1a5360 con 0x7f8d5c0a1130 2024-01-24T23:26:54.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d5c1ca8a0 con 0x7f8d5c0a2a60 2024-01-24T23:26:54.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d5c09f9f0 con 0x7f8d5c09f170 2024-01-24T23:26:54.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:54.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 10 monclient(hunting): authenticate will time out at 1301.834010s 2024-01-24T23:26:54.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d5ad9d640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f8d5c0a2a60 legacy=0x7f8d5c1d1ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:54.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d59d9b640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f8d5c0a1130 0x7f8d5c1ca190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:32774/0 (socket says 172.21.15.132:32774) 2024-01-24T23:26:54.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d59d9b640 1 -- 172.21.15.132:0/613594224 learned_addr learned my addr 172.21.15.132:0/613594224 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:54.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 1 -- 172.21.15.132:0/613594224 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2102987345 0 0) 0x7f8d5c1a5360 con 0x7f8d5c0a1130 2024-01-24T23:26:54.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:54.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:54.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:54.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 10 monclient(hunting): my global_id is 6438 2024-01-24T23:26:54.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 1 -- 172.21.15.132:0/613594224 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d40003530 con 0x7f8d5c0a1130 2024-01-24T23:26:54.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 1 -- 172.21.15.132:0/613594224 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045516251 0 0) 0x7f8d5c09f9f0 con 0x7f8d5c09f170 2024-01-24T23:26:54.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:54.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:54.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:54.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 10 monclient(hunting): my global_id is 85688 2024-01-24T23:26:54.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 1 -- 172.21.15.132:0/613594224 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d5c1a5360 con 0x7f8d5c09f170 2024-01-24T23:26:54.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 1 -- 172.21.15.132:0/613594224 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2045067182 0 0) 0x7f8d40003530 con 0x7f8d5c0a1130 2024-01-24T23:26:54.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 1 -- 172.21.15.132:0/613594224 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8d5c09f9f0 con 0x7f8d5c0a1130 2024-01-24T23:26:54.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 1 -- 172.21.15.132:0/613594224 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8d44001cb0 con 0x7f8d5c0a1130 2024-01-24T23:26:54.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:54.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:54.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 10 monclient(hunting): dump: 2024-01-24T23:26:54.522 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:54.522 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:54.523 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:54.523 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:54.523 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:54.523 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:54.523 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:54.523 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:54.523 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:54.523 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:54.523 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:54.523 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:54.523 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:54.523 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:54.523 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:54.523 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:54.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:54.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 1 -- 172.21.15.132:0/613594224 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 861544832 0 0) 0x7f8d5c09f9f0 con 0x7f8d5c0a1130 2024-01-24T23:26:54.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 1 -- 172.21.15.132:0/613594224 >> v1:172.21.15.115:6793/0 conn(0x7f8d5c09f170 legacy=0x7f8d5c1c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:54.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 1 -- 172.21.15.132:0/613594224 >> v1:172.21.15.132:6792/0 conn(0x7f8d5c0a2a60 legacy=0x7f8d5c1d1ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:54.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:26:54.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 1 monclient: found mon.g 2024-01-24T23:26:54.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:54.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:54.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 1 -- 172.21.15.132:0/613594224 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5c0726f0 con 0x7f8d5c0a1130 2024-01-24T23:26:54.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 10 monclient: _finish_auth 0 2024-01-24T23:26:54.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:26:54.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:54.524 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 5 monclient: authenticate success, global_id 6438 2024-01-24T23:26:54.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 10 monclient: _renew_subs 2024-01-24T23:26:54.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:54.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 1 -- 172.21.15.132:0/613594224 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5c1a7650 con 0x7f8d5c0a1130 2024-01-24T23:26:54.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 10 monclient: _renew_subs 2024-01-24T23:26:54.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:54.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 1 -- 172.21.15.132:0/613594224 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5c1a60e0 con 0x7f8d5c0a1130 2024-01-24T23:26:54.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 1 -- 172.21.15.132:0/613594224 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8d44004b60 con 0x7f8d5c0a1130 2024-01-24T23:26:54.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:54.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 1 -- 172.21.15.132:0/613594224 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8d44005090 con 0x7f8d5c0a1130 2024-01-24T23:26:54.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:54.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d58d99640 4 set_mon_vals no callback set 2024-01-24T23:26:54.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:54.525 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d52ffd640 10 monclient: dump: 2024-01-24T23:26:54.526 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:54.526 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:54.526 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:54.526 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:54.526 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:54.526 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:54.526 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:54.526 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:54.526 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:54.526 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:54.526 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:54.526 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:54.526 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:54.526 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:54.527 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:54.527 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:54.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:54.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:54.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:54.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.507+0000 7f8d60df8640 1 -- 172.21.15.132:0/613594224 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d34005090 con 0x7f8d5c0a1130 2024-01-24T23:26:54.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.511+0000 7f8d52ffd640 1 -- 172.21.15.132:0/613594224 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f8d440051f0 con 0x7f8d5c0a1130 2024-01-24T23:26:54.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.511+0000 7f8d52ffd640 1 -- 172.21.15.132:0/613594224 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(196..196 src has 1..196) v4 ==== 11183+0+0 (unknown 1904457828 0 0) 0x7f8d44045570 con 0x7f8d5c0a1130 2024-01-24T23:26:54.527 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.511+0000 7f8d5a59c640 10 monclient: get_auth_request con 0x7f8d40036fb0 auth_method 0 2024-01-24T23:26:54.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.515+0000 7f8d52ffd640 1 -- 172.21.15.132:0/613594224 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f8d44014b20 con 0x7f8d5c0a1130 2024-01-24T23:26:54.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.515+0000 7f8d52ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:54.528 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.515+0000 7f8d52ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:54.671 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.671+0000 7f8d60df8640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:54.672 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.671+0000 7f8d60df8640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:54.672 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.671+0000 7f8d60df8640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:54.672 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:54.671+0000 7f8d60df8640 1 -- 172.21.15.132:0/613594224 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f8d34005340 con 0x7f8d5c0a1130 2024-01-24T23:26:55.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.275+0000 7f8d52ffd640 1 -- 172.21.15.132:0/613594224 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}]=0 added bucket h18 type host to location {rack=r1} v197) v1 ==== 168+0+0 (unknown 2170329413 0 0) 0x7f8d44012f90 con 0x7f8d5c0a1130 2024-01-24T23:26:55.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.275+0000 7f8d52ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:55.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.275+0000 7f8d52ffd640 10 monclient: _finish_command 2 = system:0 added bucket h18 type host to location {rack=r1} 2024-01-24T23:26:55.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.375+0000 7f8d60df8640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:55.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.375+0000 7f8d60df8640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:55.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.375+0000 7f8d60df8640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:55.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.375+0000 7f8d60df8640 1 -- 172.21.15.132:0/613594224 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f8d34005a80 con 0x7f8d5c0a1130 2024-01-24T23:26:55.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.379+0000 7f8d52ffd640 1 -- 172.21.15.132:0/613594224 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h18' already exists v197) v1 ==== 147+0+0 (unknown 4226630063 0 0) 0x7f8d44014710 con 0x7f8d5c0a1130 2024-01-24T23:26:55.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.379+0000 7f8d52ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:55.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.379+0000 7f8d52ffd640 10 monclient: _finish_command 3 = system:0 bucket 'h18' already exists 2024-01-24T23:26:55.379 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h18' already exists 2024-01-24T23:26:55.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.379+0000 7f8d60df8640 1 -- 172.21.15.132:0/613594224 >> v1:172.21.15.132:6812/1835401922 conn(0x7f8d40036fb0 legacy=0x7f8d40039470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:55.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.379+0000 7f8d60df8640 10 monclient: shutdown 2024-01-24T23:26:55.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.379+0000 7f8d60df8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:55.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.379+0000 7f8d60df8640 1 -- 172.21.15.132:0/613594224 >> v1:172.21.15.132:6790/0 conn(0x7f8d5c0a1130 legacy=0x7f8d5c1ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:55.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.379+0000 7f8d60df8640 1 -- 172.21.15.132:0/613594224 shutdown_connections 2024-01-24T23:26:55.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.379+0000 7f8d60df8640 1 -- 172.21.15.132:0/613594224 >> 172.21.15.132:0/613594224 conn(0x7f8d5c0a7990 msgr2=0x7f8d5c0a5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:55.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.379+0000 7f8d60df8640 1 -- 172.21.15.132:0/613594224 shutdown_connections 2024-01-24T23:26:55.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.379+0000 7f8d60df8640 1 -- 172.21.15.132:0/613594224 wait complete. 2024-01-24T23:26:55.396 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:26:55.396 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h19 host rack=r1 2024-01-24T23:26:55.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb818365640 10 monclient: get_monmap_and_config 2024-01-24T23:26:55.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb818365640 10 monclient: build_initial_monmap 2024-01-24T23:26:55.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb818365640 10 monclient: monmap: 2024-01-24T23:26:55.517 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:55.517 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:55.517 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:55.518695+0000 2024-01-24T23:26:55.517 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:55.518695+0000 2024-01-24T23:26:55.517 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:55.517 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:55.517 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:55.517 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:55.517 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:55.518 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:55.518 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:55.518 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:55.518 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:55.518 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:55.518 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:55.518 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:55.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb818365640 1 Processor -- start 2024-01-24T23:26:55.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb818365640 1 -- start start 2024-01-24T23:26:55.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb818365640 10 monclient: init 2024-01-24T23:26:55.529 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb818365640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:55.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb818365640 10 monclient: _add_conns ranks=[7,0,3,5,6,8,4,1,2] 2024-01-24T23:26:55.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb818365640 10 monclient(hunting): picked mon.noname-i con 0x7fb8100a1b50 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:55.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb818365640 10 monclient(hunting): picked mon.noname-a con 0x7fb8100a2640 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:55.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb818365640 10 monclient(hunting): picked mon.noname-g con 0x7fb81006d240 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:55.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb818365640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb810064700 con 0x7fb8100a2640 2024-01-24T23:26:55.530 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb818365640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb81006f860 con 0x7fb81006d240 2024-01-24T23:26:55.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb818365640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb81006b600 con 0x7fb8100a1b50 2024-01-24T23:26:55.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb818365640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:55.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb818365640 10 monclient(hunting): authenticate will time out at 1302.843334s 2024-01-24T23:26:55.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb816101640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fb8100a1b50 legacy=0x7fb8100a1f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:55.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb816902640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fb81006d240 0x7fb81013f960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:32786/0 (socket says 172.21.15.132:32786) 2024-01-24T23:26:55.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb816902640 1 -- 172.21.15.132:0/2980451079 learned_addr learned my addr 172.21.15.132:0/2980451079 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:55.531 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb8150ff640 1 -- 172.21.15.132:0/2980451079 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3611616897 0 0) 0x7fb81006f860 con 0x7fb81006d240 2024-01-24T23:26:55.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb8150ff640 10 monclient(hunting): _negotiate 2024-01-24T23:26:55.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb8150ff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:55.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb8150ff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:55.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb8150ff640 10 monclient(hunting): my global_id is 6447 2024-01-24T23:26:55.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb8150ff640 1 -- 172.21.15.132:0/2980451079 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7fc003530 con 0x7fb81006d240 2024-01-24T23:26:55.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.515+0000 7fb8150ff640 1 -- 172.21.15.132:0/2980451079 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2508698552 0 0) 0x7fb7fc003530 con 0x7fb81006d240 2024-01-24T23:26:55.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb8150ff640 1 -- 172.21.15.132:0/2980451079 >> v1:172.21.15.132:6792/0 conn(0x7fb8100a1b50 legacy=0x7fb8100a1f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:55.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb8150ff640 1 -- 172.21.15.132:0/2980451079 >> v1:172.21.15.115:6789/0 conn(0x7fb8100a2640 legacy=0x7fb81013c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:55.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb8150ff640 10 monclient: _finish_hunting 0 2024-01-24T23:26:55.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb8150ff640 1 monclient: found mon.noname-g 2024-01-24T23:26:55.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb8150ff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:55.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb8150ff640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:26:55.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb8150ff640 1 -- 172.21.15.132:0/2980451079 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81006ce20 con 0x7fb81006d240 2024-01-24T23:26:55.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb8150ff640 10 monclient: _finish_auth 0 2024-01-24T23:26:55.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb8150ff640 10 monclient: _check_auth_tickets 2024-01-24T23:26:55.533 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb8150ff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb8150ff640 1 -- 172.21.15.132:0/2980451079 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb808001b40 con 0x7fb81006d240 2024-01-24T23:26:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb8150ff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb8150ff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb8150ff640 10 monclient: dump: 2024-01-24T23:26:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:55.534 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:55.535 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 5 monclient: authenticate success, global_id 6447 2024-01-24T23:26:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb8150ff640 1 -- 172.21.15.132:0/2980451079 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb808001e60 con 0x7fb81006d240 2024-01-24T23:26:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb8150ff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb8150ff640 1 -- 172.21.15.132:0/2980451079 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb808004f70 con 0x7fb81006d240 2024-01-24T23:26:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb8150ff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb8150ff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb8150ff640 10 monclient: dump: 2024-01-24T23:26:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:55.535 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:55.536 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:55.536 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 4 set_mon_vals no callback set 2024-01-24T23:26:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 10 monclient: shutdown 2024-01-24T23:26:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 1 -- 172.21.15.132:0/2980451079 >> v1:172.21.15.132:6790/0 conn(0x7fb81006d240 legacy=0x7fb81013f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 1 -- 172.21.15.132:0/2980451079 shutdown_connections 2024-01-24T23:26:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 1 -- 172.21.15.132:0/2980451079 >> 172.21.15.132:0/2980451079 conn(0x7fb81006d9e0 msgr2=0x7fb8100a7150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 1 -- 172.21.15.132:0/2980451079 shutdown_connections 2024-01-24T23:26:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 1 -- 172.21.15.132:0/2980451079 wait complete. 2024-01-24T23:26:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 10 monclient: build_initial_monmap 2024-01-24T23:26:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 10 monclient: monmap: 2024-01-24T23:26:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:55.537 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:55.538 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 1 Processor -- start 2024-01-24T23:26:55.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 1 -- start start 2024-01-24T23:26:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 10 monclient: init 2024-01-24T23:26:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 10 monclient: _add_conns ranks=[2,1,6,7,3,5,8,0,4] 2024-01-24T23:26:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 10 monclient(hunting): picked mon.noname-b con 0x7fb81006d240 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 10 monclient(hunting): picked mon.noname-f con 0x7fb8100a1b50 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 10 monclient(hunting): picked mon.noname-d con 0x7fb8100a2640 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8101272f0 con 0x7fb8100a1b50 2024-01-24T23:26:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb81006c0a0 con 0x7fb81006d240 2024-01-24T23:26:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb810127820 con 0x7fb8100a2640 2024-01-24T23:26:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb815900640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fb8100a1b50 legacy=0x7fb8101d0520 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb818365640 10 monclient(hunting): authenticate will time out at 1302.846636s 2024-01-24T23:26:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb816101640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fb81006d240 legacy=0x7fb810136e30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:55.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb816902640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fb8100a2640 0x7fb8101d3c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:40474/0 (socket says 172.21.15.132:40474) 2024-01-24T23:26:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb816902640 1 -- 172.21.15.132:0/645742969 learned_addr learned my addr 172.21.15.132:0/645742969 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb802ffd640 1 -- 172.21.15.132:0/645742969 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 299325784 0 0) 0x7fb810127820 con 0x7fb8100a2640 2024-01-24T23:26:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb802ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb802ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb802ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb802ffd640 10 monclient(hunting): my global_id is 6576 2024-01-24T23:26:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.519+0000 7fb802ffd640 1 -- 172.21.15.132:0/645742969 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7f4003530 con 0x7fb8100a2640 2024-01-24T23:26:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 1 -- 172.21.15.132:0/645742969 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3448072367 0 0) 0x7fb7f4003530 con 0x7fb8100a2640 2024-01-24T23:26:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 1 -- 172.21.15.132:0/645742969 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb810127820 con 0x7fb8100a2640 2024-01-24T23:26:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 1 -- 172.21.15.132:0/645742969 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb808001b00 con 0x7fb8100a2640 2024-01-24T23:26:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:26:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 10 monclient(hunting): dump: 2024-01-24T23:26:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:55.540 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:55.541 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:55.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 1 -- 172.21.15.132:0/645742969 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2566049638 0 0) 0x7fb810127820 con 0x7fb8100a2640 2024-01-24T23:26:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 1 -- 172.21.15.132:0/645742969 >> v1:172.21.15.115:6790/0 conn(0x7fb81006d240 legacy=0x7fb810136e30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 1 -- 172.21.15.132:0/645742969 >> v1:172.21.15.132:6789/0 conn(0x7fb8100a1b50 legacy=0x7fb8101d0520 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:26:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 1 monclient: found mon.d 2024-01-24T23:26:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 1 -- 172.21.15.132:0/645742969 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb810138000 con 0x7fb8100a2640 2024-01-24T23:26:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 10 monclient: _finish_auth 0 2024-01-24T23:26:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:26:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb818365640 5 monclient: authenticate success, global_id 6576 2024-01-24T23:26:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb818365640 10 monclient: _renew_subs 2024-01-24T23:26:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb818365640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb818365640 1 -- 172.21.15.132:0/645742969 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb810126f80 con 0x7fb8100a2640 2024-01-24T23:26:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb818365640 10 monclient: _renew_subs 2024-01-24T23:26:55.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb818365640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb818365640 1 -- 172.21.15.132:0/645742969 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8101256d0 con 0x7fb8100a2640 2024-01-24T23:26:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 1 -- 172.21.15.132:0/645742969 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb808001fd0 con 0x7fb8100a2640 2024-01-24T23:26:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb8148fe640 4 set_mon_vals no callback set 2024-01-24T23:26:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 1 -- 172.21.15.132:0/645742969 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb808005830 con 0x7fb8100a2640 2024-01-24T23:26:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:26:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 10 monclient: dump: 2024-01-24T23:26:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:55.543 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:55.544 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb818365640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb818365640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb818365640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb818365640 1 -- 172.21.15.132:0/645742969 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f8005090 con 0x7fb8100a2640 2024-01-24T23:26:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 1 -- 172.21.15.132:0/645742969 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fb8080102b0 con 0x7fb8100a2640 2024-01-24T23:26:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb802ffd640 1 -- 172.21.15.132:0/645742969 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(197..197 src has 1..197) v4 ==== 11396+0+0 (unknown 3878609289 0 0) 0x7fb808045b00 con 0x7fb8100a2640 2024-01-24T23:26:55.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.523+0000 7fb816101640 10 monclient: get_auth_request con 0x7fb7f4036ef0 auth_method 0 2024-01-24T23:26:55.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.535+0000 7fb802ffd640 1 -- 172.21.15.132:0/645742969 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fb808014e30 con 0x7fb8100a2640 2024-01-24T23:26:55.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.535+0000 7fb802ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:55.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.535+0000 7fb802ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:55.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.691+0000 7fb818365640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:55.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.691+0000 7fb818365640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:55.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.691+0000 7fb818365640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:55.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:55.691+0000 7fb818365640 1 -- 172.21.15.132:0/645742969 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fb7f8005340 con 0x7fb8100a2640 2024-01-24T23:26:56.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.299+0000 7fb802ffd640 1 -- 172.21.15.132:0/645742969 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}]=0 added bucket h19 type host to location {rack=r1} v198) v1 ==== 168+0+0 (unknown 1299948597 0 0) 0x7fb8080134d0 con 0x7fb8100a2640 2024-01-24T23:26:56.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.299+0000 7fb802ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:56.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.299+0000 7fb802ffd640 10 monclient: _finish_command 2 = system:0 added bucket h19 type host to location {rack=r1} 2024-01-24T23:26:56.400 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.399+0000 7fb818365640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:56.401 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.399+0000 7fb818365640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:56.401 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.399+0000 7fb818365640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:56.401 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.399+0000 7fb818365640 1 -- 172.21.15.132:0/645742969 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fb7f8005a80 con 0x7fb8100a2640 2024-01-24T23:26:56.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.399+0000 7fb802ffd640 1 -- 172.21.15.132:0/645742969 <== mon.6 v1:172.21.15.115:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h19' already exists v198) v1 ==== 147+0+0 (unknown 3641204651 0 0) 0x7fb8080134d0 con 0x7fb8100a2640 2024-01-24T23:26:56.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.403+0000 7fb802ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:56.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.403+0000 7fb802ffd640 10 monclient: _finish_command 3 = system:0 bucket 'h19' already exists 2024-01-24T23:26:56.403 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h19' already exists 2024-01-24T23:26:56.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.403+0000 7fb818365640 1 -- 172.21.15.132:0/645742969 >> v1:172.21.15.132:6812/1835401922 conn(0x7fb7f4036ef0 legacy=0x7fb7f40393b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:56.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.403+0000 7fb818365640 10 monclient: shutdown 2024-01-24T23:26:56.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.403+0000 7fb818365640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:56.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.403+0000 7fb818365640 1 -- 172.21.15.132:0/645742969 >> v1:172.21.15.115:6792/0 conn(0x7fb8100a2640 legacy=0x7fb8101d3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:56.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.403+0000 7fb818365640 1 -- 172.21.15.132:0/645742969 shutdown_connections 2024-01-24T23:26:56.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.403+0000 7fb818365640 1 -- 172.21.15.132:0/645742969 >> 172.21.15.132:0/645742969 conn(0x7fb81006d9e0 msgr2=0x7fb8100a5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:56.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.403+0000 7fb818365640 1 -- 172.21.15.132:0/645742969 shutdown_connections 2024-01-24T23:26:56.405 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.403+0000 7fb818365640 1 -- 172.21.15.132:0/645742969 wait complete. 2024-01-24T23:26:56.420 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:26:56.420 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h20 host rack=r1 2024-01-24T23:26:56.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 10 monclient: get_monmap_and_config 2024-01-24T23:26:56.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 10 monclient: build_initial_monmap 2024-01-24T23:26:56.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 10 monclient: monmap: 2024-01-24T23:26:56.539 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:56.539 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:56.539 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:56.540890+0000 2024-01-24T23:26:56.539 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:56.540890+0000 2024-01-24T23:26:56.539 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:56.539 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:56.539 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:56.540 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:56.540 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:56.540 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:56.540 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:56.540 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:56.540 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:56.540 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:56.540 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:56.540 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:56.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 1 Processor -- start 2024-01-24T23:26:56.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 1 -- start start 2024-01-24T23:26:56.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 10 monclient: init 2024-01-24T23:26:56.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:56.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 10 monclient: _add_conns ranks=[7,8,3,5,0,6,1,4,2] 2024-01-24T23:26:56.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 10 monclient(hunting): picked mon.noname-i con 0x7fca540a2a60 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:56.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 10 monclient(hunting): picked mon.noname-e con 0x7fca5409f170 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:56.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 10 monclient(hunting): picked mon.noname-g con 0x7fca540a1130 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:56.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca54064700 con 0x7fca540a1130 2024-01-24T23:26:56.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca5409f550 con 0x7fca540a2a60 2024-01-24T23:26:56.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca540a0750 con 0x7fca5409f170 2024-01-24T23:26:56.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:56.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 10 monclient(hunting): authenticate will time out at 1303.865485s 2024-01-24T23:26:56.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5966b640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fca540a2a60 legacy=0x7fca5409ea60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:56.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca59e6c640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fca540a1130 0x7fca5413d440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:32792/0 (socket says 172.21.15.132:32792) 2024-01-24T23:26:56.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca59e6c640 1 -- 172.21.15.132:0/2682162154 learned_addr learned my addr 172.21.15.132:0/2682162154 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:56.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 1 -- 172.21.15.132:0/2682162154 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2369911785 0 0) 0x7fca54064700 con 0x7fca540a1130 2024-01-24T23:26:56.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:26:56.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:56.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:56.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 10 monclient(hunting): my global_id is 6456 2024-01-24T23:26:56.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 1 -- 172.21.15.132:0/2682162154 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca40003530 con 0x7fca540a1130 2024-01-24T23:26:56.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 1 -- 172.21.15.132:0/2682162154 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 637890321 0 0) 0x7fca40003530 con 0x7fca540a1130 2024-01-24T23:26:56.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 1 -- 172.21.15.132:0/2682162154 >> v1:172.21.15.115:6793/0 conn(0x7fca5409f170 legacy=0x7fca54139d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:56.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 1 -- 172.21.15.132:0/2682162154 >> v1:172.21.15.132:6792/0 conn(0x7fca540a2a60 legacy=0x7fca5409ea60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:56.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:26:56.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 1 monclient: found mon.noname-g 2024-01-24T23:26:56.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:56.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:26:56.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 1 -- 172.21.15.132:0/2682162154 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca540a2e40 con 0x7fca540a1130 2024-01-24T23:26:56.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 10 monclient: _finish_auth 0 2024-01-24T23:26:56.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:26:56.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:56.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 1 -- 172.21.15.132:0/2682162154 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fca500017c0 con 0x7fca540a1130 2024-01-24T23:26:56.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 5 monclient: authenticate success, global_id 6456 2024-01-24T23:26:56.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:56.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:56.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:56.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 10 monclient: dump: 2024-01-24T23:26:56.550 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:56.550 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:56.550 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:56.550 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:56.550 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:56.550 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:56.550 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:56.550 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:56.550 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:56.550 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:56.550 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:56.550 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:56.550 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:56.550 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:56.550 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:56.550 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:56.551 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:56.551 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 1 -- 172.21.15.132:0/2682162154 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fca50001ae0 con 0x7fca540a1130 2024-01-24T23:26:56.551 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:56.551 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 1 -- 172.21.15.132:0/2682162154 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fca50004bf0 con 0x7fca540a1130 2024-01-24T23:26:56.551 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:56.551 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:56.551 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca4bfff640 10 monclient: dump: 2024-01-24T23:26:56.551 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:56.551 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:56.551 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:56.551 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:56.551 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:56.551 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:56.551 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:56.551 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:56.552 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:56.552 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:56.552 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:56.552 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:56.552 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:56.552 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:56.552 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:56.552 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:56.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:56.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 4 set_mon_vals no callback set 2024-01-24T23:26:56.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 10 monclient: shutdown 2024-01-24T23:26:56.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:56.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 1 -- 172.21.15.132:0/2682162154 >> v1:172.21.15.132:6790/0 conn(0x7fca540a1130 legacy=0x7fca5413d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:56.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 1 -- 172.21.15.132:0/2682162154 shutdown_connections 2024-01-24T23:26:56.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 1 -- 172.21.15.132:0/2682162154 >> 172.21.15.132:0/2682162154 conn(0x7fca540a7990 msgr2=0x7fca540a3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:56.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 1 -- 172.21.15.132:0/2682162154 shutdown_connections 2024-01-24T23:26:56.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 1 -- 172.21.15.132:0/2682162154 wait complete. 2024-01-24T23:26:56.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 10 monclient: build_initial_monmap 2024-01-24T23:26:56.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.539+0000 7fca5b8cf640 10 monclient: monmap: 2024-01-24T23:26:56.553 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:56.553 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:56.553 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:56.553 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:56.553 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:56.553 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:56.553 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:56.553 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:56.553 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:56.553 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:56.553 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:56.554 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:56.554 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:56.554 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:56.554 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:56.554 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:56.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5b8cf640 1 Processor -- start 2024-01-24T23:26:56.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5b8cf640 1 -- start start 2024-01-24T23:26:56.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5b8cf640 10 monclient: init 2024-01-24T23:26:56.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5b8cf640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:56.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5b8cf640 10 monclient: _add_conns ranks=[8,6,7,2,0,3,4,5,1] 2024-01-24T23:26:56.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5b8cf640 10 monclient(hunting): picked mon.noname-e con 0x7fca5409f170 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:56.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5b8cf640 10 monclient(hunting): picked mon.noname-d con 0x7fca540a1130 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:56.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5b8cf640 10 monclient(hunting): picked mon.noname-i con 0x7fca540a2a60 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:56.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5b8cf640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca540726f0 con 0x7fca540a1130 2024-01-24T23:26:56.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5b8cf640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca541ac320 con 0x7fca540a2a60 2024-01-24T23:26:56.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5b8cf640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca5409f9f0 con 0x7fca5409f170 2024-01-24T23:26:56.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5b8cf640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:56.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5b8cf640 10 monclient(hunting): authenticate will time out at 1303.868669s 2024-01-24T23:26:56.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca59e6c640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fca540a2a60 legacy=0x7fca541d37c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:56.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca58e6a640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fca540a1130 0x7fca541d0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:40490/0 (socket says 172.21.15.132:40490) 2024-01-24T23:26:56.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5966b640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fca5409f170 0x7fca541bc940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:51636/0 (socket says 172.21.15.132:51636) 2024-01-24T23:26:56.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca58e6a640 1 -- 172.21.15.132:0/88656038 learned_addr learned my addr 172.21.15.132:0/88656038 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:56.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 1 -- 172.21.15.132:0/88656038 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144768045 0 0) 0x7fca5409f9f0 con 0x7fca5409f170 2024-01-24T23:26:56.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:26:56.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:56.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:56.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 10 monclient(hunting): my global_id is 85706 2024-01-24T23:26:56.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 1 -- 172.21.15.132:0/88656038 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca38003530 con 0x7fca5409f170 2024-01-24T23:26:56.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 1 -- 172.21.15.132:0/88656038 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3770529139 0 0) 0x7fca540726f0 con 0x7fca540a1130 2024-01-24T23:26:56.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:26:56.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:56.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:56.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 10 monclient(hunting): my global_id is 6585 2024-01-24T23:26:56.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 1 -- 172.21.15.132:0/88656038 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca5409f9f0 con 0x7fca540a1130 2024-01-24T23:26:56.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 1 -- 172.21.15.132:0/88656038 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1612757498 0 0) 0x7fca5409f9f0 con 0x7fca540a1130 2024-01-24T23:26:56.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 1 -- 172.21.15.132:0/88656038 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fca540726f0 con 0x7fca540a1130 2024-01-24T23:26:56.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 1 -- 172.21.15.132:0/88656038 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fca44001aa0 con 0x7fca540a1130 2024-01-24T23:26:56.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:56.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:26:56.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 10 monclient(hunting): dump: 2024-01-24T23:26:56.556 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:56.556 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:56.557 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:56.557 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:56.557 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:56.557 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:56.557 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:56.557 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:56.557 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:56.557 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:56.557 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:56.557 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:56.557 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:56.557 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:56.557 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:56.557 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:56.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:56.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 1 -- 172.21.15.132:0/88656038 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2779718292 0 0) 0x7fca38003530 con 0x7fca5409f170 2024-01-24T23:26:56.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 1 -- 172.21.15.132:0/88656038 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fca5409f9f0 con 0x7fca5409f170 2024-01-24T23:26:56.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 1 -- 172.21.15.132:0/88656038 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fca4c0045f0 con 0x7fca5409f170 2024-01-24T23:26:56.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:56.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:26:56.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 10 monclient(hunting): dump: 2024-01-24T23:26:56.558 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:56.558 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:56.558 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:56.558 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:56.558 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:56.558 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:56.558 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:56.558 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:56.558 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:56.558 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:56.559 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:56.559 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:56.559 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:56.559 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:56.559 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:56.559 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:56.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 1 -- 172.21.15.132:0/88656038 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3252536180 0 0) 0x7fca540726f0 con 0x7fca540a1130 2024-01-24T23:26:56.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 1 -- 172.21.15.132:0/88656038 >> v1:172.21.15.115:6793/0 conn(0x7fca5409f170 legacy=0x7fca541bc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:56.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 1 -- 172.21.15.132:0/88656038 >> v1:172.21.15.132:6792/0 conn(0x7fca540a2a60 legacy=0x7fca541d37c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:56.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:26:56.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 1 monclient: found mon.d 2024-01-24T23:26:56.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:56.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:56.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 1 -- 172.21.15.132:0/88656038 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca541ac040 con 0x7fca540a1130 2024-01-24T23:26:56.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 10 monclient: _finish_auth 0 2024-01-24T23:26:56.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:26:56.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:56.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5b8cf640 5 monclient: authenticate success, global_id 6585 2024-01-24T23:26:56.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5b8cf640 10 monclient: _renew_subs 2024-01-24T23:26:56.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5b8cf640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:56.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5b8cf640 1 -- 172.21.15.132:0/88656038 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca541aa860 con 0x7fca540a1130 2024-01-24T23:26:56.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5b8cf640 10 monclient: _renew_subs 2024-01-24T23:26:56.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5b8cf640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:56.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5b8cf640 1 -- 172.21.15.132:0/88656038 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fca541bd220 con 0x7fca540a1130 2024-01-24T23:26:56.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 1 -- 172.21.15.132:0/88656038 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fca440038d0 con 0x7fca540a1130 2024-01-24T23:26:56.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:56.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 1 -- 172.21.15.132:0/88656038 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fca44003e00 con 0x7fca540a1130 2024-01-24T23:26:56.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:56.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:26:56.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca4b7fe640 4 set_mon_vals no callback set 2024-01-24T23:26:56.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 10 monclient: dump: 2024-01-24T23:26:56.561 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:56.561 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:56.561 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:56.561 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:56.561 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:56.561 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:56.561 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:56.561 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:56.561 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:56.561 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:56.561 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:56.561 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:56.561 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:56.561 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:56.561 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:56.561 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:56.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5b8cf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:56.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5b8cf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:56.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5b8cf640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:56.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca5b8cf640 1 -- 172.21.15.132:0/88656038 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca34005090 con 0x7fca540a1130 2024-01-24T23:26:56.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.543+0000 7fca49ffb640 1 -- 172.21.15.132:0/88656038 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fca44003fe0 con 0x7fca540a1130 2024-01-24T23:26:56.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.547+0000 7fca49ffb640 1 -- 172.21.15.132:0/88656038 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(198..198 src has 1..198) v4 ==== 11609+0+0 (unknown 4221900858 0 0) 0x7fca44045650 con 0x7fca540a1130 2024-01-24T23:26:56.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.547+0000 7fca5966b640 10 monclient: get_auth_request con 0x7fca38037730 auth_method 0 2024-01-24T23:26:56.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.559+0000 7fca49ffb640 1 -- 172.21.15.132:0/88656038 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fca44014840 con 0x7fca540a1130 2024-01-24T23:26:56.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.559+0000 7fca49ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:56.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.559+0000 7fca49ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:56.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.715+0000 7fca5b8cf640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:56.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.715+0000 7fca5b8cf640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:56.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.715+0000 7fca5b8cf640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:56.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.715+0000 7fca5b8cf640 1 -- 172.21.15.132:0/88656038 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fca34005340 con 0x7fca540a1130 2024-01-24T23:26:56.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.915+0000 7fca49ffb640 1 -- 172.21.15.132:0/88656038 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}]=0 added bucket h20 type host to location {rack=r1} v199) v1 ==== 168+0+0 (unknown 1638306274 0 0) 0x7fca44012ee0 con 0x7fca540a1130 2024-01-24T23:26:56.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.919+0000 7fca49ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:56.919 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:56.919+0000 7fca49ffb640 10 monclient: _finish_command 2 = system:0 added bucket h20 type host to location {rack=r1} 2024-01-24T23:26:57.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.019+0000 7fca5b8cf640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:57.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.019+0000 7fca5b8cf640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:57.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.019+0000 7fca5b8cf640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:26:57.021 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.019+0000 7fca5b8cf640 1 -- 172.21.15.132:0/88656038 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fca34005a80 con 0x7fca540a1130 2024-01-24T23:26:57.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.023+0000 7fca49ffb640 1 -- 172.21.15.132:0/88656038 <== mon.6 v1:172.21.15.115:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h20' already exists v199) v1 ==== 147+0+0 (unknown 2600462907 0 0) 0x7fca44012ee0 con 0x7fca540a1130 2024-01-24T23:26:57.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.023+0000 7fca49ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:57.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.023+0000 7fca49ffb640 10 monclient: _finish_command 3 = system:0 bucket 'h20' already exists 2024-01-24T23:26:57.023 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h20' already exists 2024-01-24T23:26:57.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.023+0000 7fca5b8cf640 1 -- 172.21.15.132:0/88656038 >> v1:172.21.15.132:6812/1835401922 conn(0x7fca38037730 legacy=0x7fca38039bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:57.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.023+0000 7fca5b8cf640 10 monclient: shutdown 2024-01-24T23:26:57.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.023+0000 7fca5b8cf640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:57.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.023+0000 7fca5b8cf640 1 -- 172.21.15.132:0/88656038 >> v1:172.21.15.115:6792/0 conn(0x7fca540a1130 legacy=0x7fca541d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:57.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.023+0000 7fca5b8cf640 1 -- 172.21.15.132:0/88656038 shutdown_connections 2024-01-24T23:26:57.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.023+0000 7fca5b8cf640 1 -- 172.21.15.132:0/88656038 >> 172.21.15.132:0/88656038 conn(0x7fca540a7990 msgr2=0x7fca5406fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:57.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.023+0000 7fca5b8cf640 1 -- 172.21.15.132:0/88656038 shutdown_connections 2024-01-24T23:26:57.025 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.023+0000 7fca5b8cf640 1 -- 172.21.15.132:0/88656038 wait complete. 2024-01-24T23:26:57.040 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:26:57.040 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h21 host rack=r1 2024-01-24T23:26:57.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f334fa20640 10 monclient: get_monmap_and_config 2024-01-24T23:26:57.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f334fa20640 10 monclient: build_initial_monmap 2024-01-24T23:26:57.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f334fa20640 10 monclient: monmap: 2024-01-24T23:26:57.161 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:57.161 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:57.161 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:57.162398+0000 2024-01-24T23:26:57.161 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:57.162398+0000 2024-01-24T23:26:57.161 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:57.162 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:57.162 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:57.162 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:57.162 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:57.162 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:57.162 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:57.162 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:57.162 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:57.162 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:57.162 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:57.162 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:57.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f334fa20640 1 Processor -- start 2024-01-24T23:26:57.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f334fa20640 1 -- start start 2024-01-24T23:26:57.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f334fa20640 10 monclient: init 2024-01-24T23:26:57.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f334fa20640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:57.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f334fa20640 10 monclient: _add_conns ranks=[4,0,5,2,8,1,7,6,3] 2024-01-24T23:26:57.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f334fa20640 10 monclient(hunting): picked mon.noname-c con 0x7f3348072690 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:57.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f334fa20640 10 monclient(hunting): picked mon.noname-a con 0x7f334809ae90 addr v1:172.21.15.115:6789/0 2024-01-24T23:26:57.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f334fa20640 10 monclient(hunting): picked mon.noname-h con 0x7f334809bdc0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:57.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f334fa20640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3348064700 con 0x7f334809ae90 2024-01-24T23:26:57.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f334fa20640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334809a790 con 0x7f3348072690 2024-01-24T23:26:57.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f334fa20640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33480a18f0 con 0x7f334809bdc0 2024-01-24T23:26:57.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f334fa20640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:57.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f334fa20640 10 monclient(hunting): authenticate will time out at 1304.486948s 2024-01-24T23:26:57.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f334dfbd640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f334809bdc0 0x7f334813d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:36858/0 (socket says 172.21.15.132:36858) 2024-01-24T23:26:57.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f334dfbd640 1 -- 172.21.15.132:0/2402641314 learned_addr learned my addr 172.21.15.132:0/2402641314 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:57.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 1 -- 172.21.15.132:0/2402641314 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740218403 0 0) 0x7f33480a18f0 con 0x7f334809bdc0 2024-01-24T23:26:57.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:26:57.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:57.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:57.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 10 monclient(hunting): my global_id is 6368 2024-01-24T23:26:57.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 1 -- 172.21.15.132:0/2402641314 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3334003530 con 0x7f334809bdc0 2024-01-24T23:26:57.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 1 -- 172.21.15.132:0/2402641314 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1288348581 0 0) 0x7f3334003530 con 0x7f334809bdc0 2024-01-24T23:26:57.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 1 -- 172.21.15.132:0/2402641314 >> v1:172.21.15.115:6791/0 conn(0x7f3348072690 legacy=0x7f33480a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:57.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 1 -- 172.21.15.132:0/2402641314 >> v1:172.21.15.115:6789/0 conn(0x7f334809ae90 legacy=0x7f33480a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:57.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 10 monclient: _finish_hunting 0 2024-01-24T23:26:57.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 1 monclient: found mon.noname-h 2024-01-24T23:26:57.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:57.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:26:57.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 1 -- 172.21.15.132:0/2402641314 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33480997a0 con 0x7f334809bdc0 2024-01-24T23:26:57.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 10 monclient: _finish_auth 0 2024-01-24T23:26:57.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 10 monclient: _check_auth_tickets 2024-01-24T23:26:57.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:57.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 1 -- 172.21.15.132:0/2402641314 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3340001b40 con 0x7f334809bdc0 2024-01-24T23:26:57.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:57.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:57.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 10 monclient: dump: 2024-01-24T23:26:57.171 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:57.171 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:57.171 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:57.171 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:57.171 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:57.171 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:57.171 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:57.171 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:57.171 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:57.171 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:57.171 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:57.171 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:57.171 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:57.172 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:57.172 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:57.172 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:57.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f334fa20640 5 monclient: authenticate success, global_id 6368 2024-01-24T23:26:57.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f334fa20640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:57.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 1 -- 172.21.15.132:0/2402641314 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3340001df0 con 0x7f334809bdc0 2024-01-24T23:26:57.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:57.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 1 -- 172.21.15.132:0/2402641314 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3340004f70 con 0x7f334809bdc0 2024-01-24T23:26:57.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:57.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:57.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f333ffff640 10 monclient: dump: 2024-01-24T23:26:57.172 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:57.172 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:57.172 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:57.173 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f334fa20640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.159+0000 7f334fa20640 4 set_mon_vals no callback set 2024-01-24T23:26:57.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 10 monclient: shutdown 2024-01-24T23:26:57.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:57.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 1 -- 172.21.15.132:0/2402641314 >> v1:172.21.15.132:6791/0 conn(0x7f334809bdc0 legacy=0x7f334813d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:57.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 1 -- 172.21.15.132:0/2402641314 shutdown_connections 2024-01-24T23:26:57.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 1 -- 172.21.15.132:0/2402641314 >> 172.21.15.132:0/2402641314 conn(0x7f33480a1bc0 msgr2=0x7f33480a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:57.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 1 -- 172.21.15.132:0/2402641314 shutdown_connections 2024-01-24T23:26:57.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 1 -- 172.21.15.132:0/2402641314 wait complete. 2024-01-24T23:26:57.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 10 monclient: build_initial_monmap 2024-01-24T23:26:57.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 10 monclient: monmap: 2024-01-24T23:26:57.174 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:57.174 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:57.174 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:57.174 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:57.174 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:57.174 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:57.175 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:57.175 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:57.175 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:57.175 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:57.175 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:57.175 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:57.175 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:57.175 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:57.175 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:57.175 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:57.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 1 Processor -- start 2024-01-24T23:26:57.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 1 -- start start 2024-01-24T23:26:57.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 10 monclient: init 2024-01-24T23:26:57.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:57.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 10 monclient: _add_conns ranks=[4,3,6,7,0,5,1,8,2] 2024-01-24T23:26:57.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 10 monclient(hunting): picked mon.noname-c con 0x7f3348072690 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:57.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 10 monclient(hunting): picked mon.noname-g con 0x7f334809ae90 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:57.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 10 monclient(hunting): picked mon.noname-d con 0x7f334809bdc0 addr v1:172.21.15.115:6792/0 2024-01-24T23:26:57.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3348096e40 con 0x7f334809ae90 2024-01-24T23:26:57.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3348099470 con 0x7f3348072690 2024-01-24T23:26:57.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334809b930 con 0x7f334809bdc0 2024-01-24T23:26:57.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:57.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 10 monclient(hunting): authenticate will time out at 1304.490127s 2024-01-24T23:26:57.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334cfbb640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f334809ae90 0x7f33481d07d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:32806/0 (socket says 172.21.15.132:32806) 2024-01-24T23:26:57.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334cfbb640 1 -- 172.21.15.132:0/3592433821 learned_addr learned my addr 172.21.15.132:0/3592433821 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:57.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 1 -- 172.21.15.132:0/3592433821 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 581010361 0 0) 0x7f3348096e40 con 0x7f334809ae90 2024-01-24T23:26:57.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:26:57.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:57.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:57.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 10 monclient(hunting): my global_id is 6465 2024-01-24T23:26:57.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 1 -- 172.21.15.132:0/3592433821 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f332c003530 con 0x7f334809ae90 2024-01-24T23:26:57.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 1 -- 172.21.15.132:0/3592433821 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 797446124 0 0) 0x7f332c003530 con 0x7f334809ae90 2024-01-24T23:26:57.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 1 -- 172.21.15.132:0/3592433821 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3348096e40 con 0x7f334809ae90 2024-01-24T23:26:57.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 1 -- 172.21.15.132:0/3592433821 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3338001a60 con 0x7f334809ae90 2024-01-24T23:26:57.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:57.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:26:57.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 10 monclient(hunting): dump: 2024-01-24T23:26:57.177 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:57.177 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:57.177 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:57.177 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:57.177 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:57.177 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:57.177 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:57.177 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:57.178 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:57.178 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:57.178 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:57.178 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:57.178 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:57.178 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:57.178 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:57.178 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:57.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:57.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 1 -- 172.21.15.132:0/3592433821 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3278101783 0 0) 0x7f3348099470 con 0x7f3348072690 2024-01-24T23:26:57.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:26:57.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:57.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:57.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 10 monclient(hunting): my global_id is 94612 2024-01-24T23:26:57.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 1 -- 172.21.15.132:0/3592433821 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f332c003530 con 0x7f3348072690 2024-01-24T23:26:57.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 1 -- 172.21.15.132:0/3592433821 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3259712599 0 0) 0x7f3348096e40 con 0x7f334809ae90 2024-01-24T23:26:57.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 1 -- 172.21.15.132:0/3592433821 >> v1:172.21.15.115:6792/0 conn(0x7f334809bdc0 legacy=0x7f33481d3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:57.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 1 -- 172.21.15.132:0/3592433821 >> v1:172.21.15.115:6791/0 conn(0x7f3348072690 legacy=0x7f3348098370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:57.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 10 monclient: _finish_hunting 0 2024-01-24T23:26:57.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 1 monclient: found mon.g 2024-01-24T23:26:57.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:57.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:57.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 1 -- 172.21.15.132:0/3592433821 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3348094b50 con 0x7f334809ae90 2024-01-24T23:26:57.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 10 monclient: _finish_auth 0 2024-01-24T23:26:57.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 10 monclient: _check_auth_tickets 2024-01-24T23:26:57.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:57.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 5 monclient: authenticate success, global_id 6465 2024-01-24T23:26:57.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 10 monclient: _renew_subs 2024-01-24T23:26:57.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:57.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 1 -- 172.21.15.132:0/3592433821 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3348096b60 con 0x7f334809ae90 2024-01-24T23:26:57.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 10 monclient: _renew_subs 2024-01-24T23:26:57.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:57.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 1 -- 172.21.15.132:0/3592433821 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3348098a80 con 0x7f334809ae90 2024-01-24T23:26:57.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 1 -- 172.21.15.132:0/3592433821 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3338003950 con 0x7f334809ae90 2024-01-24T23:26:57.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:57.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 1 -- 172.21.15.132:0/3592433821 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3338003e80 con 0x7f334809ae90 2024-01-24T23:26:57.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:57.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333f7fe640 4 set_mon_vals no callback set 2024-01-24T23:26:57.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:26:57.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f333dffb640 10 monclient: dump: 2024-01-24T23:26:57.184 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:57.184 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:57.184 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:57.184 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:57.185 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:57.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:57.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.163+0000 7f334fa20640 1 -- 172.21.15.132:0/3592433821 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3328005090 con 0x7f334809ae90 2024-01-24T23:26:57.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.167+0000 7f333dffb640 1 -- 172.21.15.132:0/3592433821 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f3338003fe0 con 0x7f334809ae90 2024-01-24T23:26:57.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.167+0000 7f333dffb640 1 -- 172.21.15.132:0/3592433821 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(199..199 src has 1..199) v4 ==== 11822+0+0 (unknown 1968646360 0 0) 0x7f33380457c0 con 0x7f334809ae90 2024-01-24T23:26:57.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.167+0000 7f334d7bc640 10 monclient: get_auth_request con 0x7f332c036ec0 auth_method 0 2024-01-24T23:26:57.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.171+0000 7f333dffb640 1 -- 172.21.15.132:0/3592433821 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f3338014840 con 0x7f334809ae90 2024-01-24T23:26:57.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.171+0000 7f333dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:57.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.171+0000 7f333dffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:57.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.331+0000 7f334fa20640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:57.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.331+0000 7f334fa20640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:57.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.331+0000 7f334fa20640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:57.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:57.331+0000 7f334fa20640 1 -- 172.21.15.132:0/3592433821 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f3328005340 con 0x7f334809ae90 2024-01-24T23:26:58.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.311+0000 7f333dffb640 1 -- 172.21.15.132:0/3592433821 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}]=0 added bucket h21 type host to location {rack=r1} v200) v1 ==== 168+0+0 (unknown 50898095 0 0) 0x7f3338012ee0 con 0x7f334809ae90 2024-01-24T23:26:58.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.311+0000 7f333dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:58.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.311+0000 7f333dffb640 10 monclient: _finish_command 2 = system:0 added bucket h21 type host to location {rack=r1} 2024-01-24T23:26:58.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.411+0000 7f334fa20640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:58.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.411+0000 7f334fa20640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:58.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.411+0000 7f334fa20640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:26:58.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.411+0000 7f334fa20640 1 -- 172.21.15.132:0/3592433821 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f3328005a80 con 0x7f334809ae90 2024-01-24T23:26:58.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.415+0000 7f333dffb640 1 -- 172.21.15.132:0/3592433821 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h21' already exists v200) v1 ==== 147+0+0 (unknown 1341187145 0 0) 0x7f3338012ee0 con 0x7f334809ae90 2024-01-24T23:26:58.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.415+0000 7f333dffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:58.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.415+0000 7f333dffb640 10 monclient: _finish_command 3 = system:0 bucket 'h21' already exists 2024-01-24T23:26:58.416 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h21' already exists 2024-01-24T23:26:58.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.415+0000 7f334fa20640 1 -- 172.21.15.132:0/3592433821 >> v1:172.21.15.132:6812/1835401922 conn(0x7f332c036ec0 legacy=0x7f332c039380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:58.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.415+0000 7f334fa20640 10 monclient: shutdown 2024-01-24T23:26:58.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.415+0000 7f334fa20640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:58.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.415+0000 7f334fa20640 1 -- 172.21.15.132:0/3592433821 >> v1:172.21.15.132:6790/0 conn(0x7f334809ae90 legacy=0x7f33481d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:58.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.415+0000 7f334fa20640 1 -- 172.21.15.132:0/3592433821 shutdown_connections 2024-01-24T23:26:58.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.415+0000 7f334fa20640 1 -- 172.21.15.132:0/3592433821 >> 172.21.15.132:0/3592433821 conn(0x7f33480a1bc0 msgr2=0x7f33480a2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:58.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.415+0000 7f334fa20640 1 -- 172.21.15.132:0/3592433821 shutdown_connections 2024-01-24T23:26:58.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.415+0000 7f334fa20640 1 -- 172.21.15.132:0/3592433821 wait complete. 2024-01-24T23:26:58.433 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:26:58.433 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h22 host rack=r1 2024-01-24T23:26:58.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.551+0000 7f051bd8f640 10 monclient: get_monmap_and_config 2024-01-24T23:26:58.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.551+0000 7f051bd8f640 10 monclient: build_initial_monmap 2024-01-24T23:26:58.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.551+0000 7f051bd8f640 10 monclient: monmap: 2024-01-24T23:26:58.553 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:58.553 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:58.553 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:58.554548+0000 2024-01-24T23:26:58.553 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:58.554548+0000 2024-01-24T23:26:58.553 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:58.553 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:58.553 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:58.553 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:58.553 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:58.553 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:58.553 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:58.553 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:58.553 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:58.553 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:58.554 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:58.554 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:58.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.551+0000 7f051bd8f640 1 Processor -- start 2024-01-24T23:26:58.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.551+0000 7f051bd8f640 1 -- start start 2024-01-24T23:26:58.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.551+0000 7f051bd8f640 10 monclient: init 2024-01-24T23:26:58.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.551+0000 7f051bd8f640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:58.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.551+0000 7f051bd8f640 10 monclient: _add_conns ranks=[5,2,3,1,4,6,0,8,7] 2024-01-24T23:26:58.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.551+0000 7f051bd8f640 10 monclient(hunting): picked mon.noname-h con 0x7f0514134c70 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:58.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.551+0000 7f051bd8f640 10 monclient(hunting): picked mon.noname-b con 0x7f0514071d70 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:58.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.551+0000 7f051bd8f640 10 monclient(hunting): picked mon.noname-g con 0x7f051413b380 addr v1:172.21.15.132:6790/0 2024-01-24T23:26:58.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.551+0000 7f051bd8f640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0514064700 con 0x7f0514071d70 2024-01-24T23:26:58.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.551+0000 7f051bd8f640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0514070bc0 con 0x7f051413b380 2024-01-24T23:26:58.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0514070ea0 con 0x7f0514134c70 2024-01-24T23:26:58.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:58.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 10 monclient(hunting): authenticate will time out at 1305.880033s 2024-01-24T23:26:58.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0519b2b640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f0514134c70 0x7f0514135050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:36868/0 (socket says 172.21.15.132:36868) 2024-01-24T23:26:58.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0519b2b640 1 -- 172.21.15.132:0/886180826 learned_addr learned my addr 172.21.15.132:0/886180826 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:58.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051932a640 1 -- 172.21.15.132:0/886180826 >> v1:172.21.15.115:6790/0 conn(0x7f0514071d70 legacy=0x7f0514139c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:58.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 1 -- 172.21.15.132:0/886180826 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3408953407 0 0) 0x7f0514070ea0 con 0x7f0514134c70 2024-01-24T23:26:58.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 10 monclient(hunting): _negotiate 2024-01-24T23:26:58.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:58.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:58.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 10 monclient(hunting): my global_id is 6377 2024-01-24T23:26:58.558 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 1 -- 172.21.15.132:0/886180826 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0500003530 con 0x7f0514134c70 2024-01-24T23:26:58.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 1 -- 172.21.15.132:0/886180826 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 698014807 0 0) 0x7f0514070bc0 con 0x7f051413b380 2024-01-24T23:26:58.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 10 monclient(hunting): _negotiate 2024-01-24T23:26:58.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:58.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:58.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 10 monclient(hunting): my global_id is 6474 2024-01-24T23:26:58.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 1 -- 172.21.15.132:0/886180826 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0514070ea0 con 0x7f051413b380 2024-01-24T23:26:58.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 1 -- 172.21.15.132:0/886180826 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 111574149 0 0) 0x7f0500003530 con 0x7f0514134c70 2024-01-24T23:26:58.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 1 -- 172.21.15.132:0/886180826 >> v1:172.21.15.132:6790/0 conn(0x7f051413b380 legacy=0x7f051413d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:58.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 1 -- 172.21.15.132:0/886180826 >> v1:172.21.15.115:6790/0 conn(0x7f0514071d70 legacy=0x7f0514139c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:58.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 10 monclient: _finish_hunting 0 2024-01-24T23:26:58.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 1 monclient: found mon.noname-h 2024-01-24T23:26:58.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:58.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:26:58.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 1 -- 172.21.15.132:0/886180826 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0514071180 con 0x7f0514134c70 2024-01-24T23:26:58.559 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 10 monclient: _finish_auth 0 2024-01-24T23:26:58.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 10 monclient: _check_auth_tickets 2024-01-24T23:26:58.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:58.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 1 -- 172.21.15.132:0/886180826 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f05100017c0 con 0x7f0514134c70 2024-01-24T23:26:58.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:58.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:58.560 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 10 monclient: dump: 2024-01-24T23:26:58.560 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:58.560 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:58.560 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:58.560 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:58.560 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:58.560 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:58.560 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:58.560 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:58.560 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:58.561 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:58.561 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:58.561 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:58.561 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:58.561 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:58.561 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:58.561 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:58.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 5 monclient: authenticate success, global_id 6377 2024-01-24T23:26:58.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:58.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 1 -- 172.21.15.132:0/886180826 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f05100019b0 con 0x7f0514134c70 2024-01-24T23:26:58.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:58.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 1 -- 172.21.15.132:0/886180826 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0510004bf0 con 0x7f0514134c70 2024-01-24T23:26:58.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:58.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:58.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 4 set_mon_vals no callback set 2024-01-24T23:26:58.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:58.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0518b29640 10 monclient: dump: 2024-01-24T23:26:58.562 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:58.562 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:58.562 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:58.562 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:58.562 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:58.562 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:58.562 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:58.562 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:58.562 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:58.562 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:58.562 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:58.563 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:58.563 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:58.563 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:58.563 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:58.563 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:58.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 10 monclient: shutdown 2024-01-24T23:26:58.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:58.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 1 -- 172.21.15.132:0/886180826 >> v1:172.21.15.132:6791/0 conn(0x7f0514134c70 legacy=0x7f0514135050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:58.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 1 -- 172.21.15.132:0/886180826 shutdown_connections 2024-01-24T23:26:58.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 1 -- 172.21.15.132:0/886180826 >> 172.21.15.132:0/886180826 conn(0x7f051406aaf0 msgr2=0x7f05140a24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:58.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 1 -- 172.21.15.132:0/886180826 shutdown_connections 2024-01-24T23:26:58.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 1 -- 172.21.15.132:0/886180826 wait complete. 2024-01-24T23:26:58.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 10 monclient: build_initial_monmap 2024-01-24T23:26:58.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 10 monclient: monmap: 2024-01-24T23:26:58.564 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:58.564 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:58.564 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:58.564 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:58.564 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:58.564 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:58.564 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:58.564 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:58.564 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:58.564 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:58.564 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:58.564 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:58.565 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:58.565 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:58.565 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:58.565 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:58.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 1 Processor -- start 2024-01-24T23:26:58.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 1 -- start start 2024-01-24T23:26:58.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 10 monclient: init 2024-01-24T23:26:58.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:58.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 10 monclient: _add_conns ranks=[5,2,4,8,3,6,7,1,0] 2024-01-24T23:26:58.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 10 monclient(hunting): picked mon.noname-h con 0x7f0514071d70 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:58.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 10 monclient(hunting): picked mon.noname-b con 0x7f0514134c70 addr v1:172.21.15.115:6790/0 2024-01-24T23:26:58.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 10 monclient(hunting): picked mon.noname-c con 0x7f051413b380 addr v1:172.21.15.115:6791/0 2024-01-24T23:26:58.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f051413a280 con 0x7f0514134c70 2024-01-24T23:26:58.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f051412f130 con 0x7f051413b380 2024-01-24T23:26:58.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f051406e050 con 0x7f0514071d70 2024-01-24T23:26:58.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:58.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 10 monclient(hunting): authenticate will time out at 1305.883190s 2024-01-24T23:26:58.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0519b2b640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f0514071d70 0x7f0514139460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:36882/0 (socket says 172.21.15.132:36882) 2024-01-24T23:26:58.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f0519b2b640 1 -- 172.21.15.132:0/41143097 learned_addr learned my addr 172.21.15.132:0/41143097 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:58.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051932a640 1 -- 172.21.15.132:0/41143097 >> v1:172.21.15.115:6790/0 conn(0x7f0514134c70 legacy=0x7f0514139b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:26:58.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f050a7fc640 1 -- 172.21.15.132:0/41143097 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 879994855 0 0) 0x7f051406e050 con 0x7f0514071d70 2024-01-24T23:26:58.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f050a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:26:58.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f050a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:58.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f050a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:58.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f050a7fc640 10 monclient(hunting): my global_id is 6386 2024-01-24T23:26:58.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f050a7fc640 1 -- 172.21.15.132:0/41143097 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04f8003530 con 0x7f0514071d70 2024-01-24T23:26:58.570 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f050a7fc640 1 -- 172.21.15.132:0/41143097 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 470664042 0 0) 0x7f04f8003530 con 0x7f0514071d70 2024-01-24T23:26:58.570 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f050a7fc640 1 -- 172.21.15.132:0/41143097 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f051406e050 con 0x7f0514071d70 2024-01-24T23:26:58.570 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f050a7fc640 1 -- 172.21.15.132:0/41143097 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f05100047f0 con 0x7f0514071d70 2024-01-24T23:26:58.570 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f050a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:58.570 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f050a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:58.570 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f050a7fc640 10 monclient(hunting): dump: 2024-01-24T23:26:58.570 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:58.570 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:58.570 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:58.570 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:58.570 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:58.570 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:58.570 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:58.570 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:58.571 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:58.571 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:58.571 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:58.571 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:58.571 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:58.571 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:58.571 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:58.571 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:58.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f050a7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:58.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f050a7fc640 1 -- 172.21.15.132:0/41143097 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 794540312 0 0) 0x7f051406e050 con 0x7f0514071d70 2024-01-24T23:26:58.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f050a7fc640 1 -- 172.21.15.132:0/41143097 >> v1:172.21.15.115:6791/0 conn(0x7f051413b380 legacy=0x7f05141d3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:58.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f050a7fc640 1 -- 172.21.15.132:0/41143097 >> v1:172.21.15.115:6790/0 conn(0x7f0514134c70 legacy=0x7f0514139b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:58.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f050a7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:26:58.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f050a7fc640 1 monclient: found mon.h 2024-01-24T23:26:58.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f050a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:58.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f050a7fc640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:58.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f050a7fc640 1 -- 172.21.15.132:0/41143097 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05140701a0 con 0x7f0514071d70 2024-01-24T23:26:58.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f050a7fc640 10 monclient: _finish_auth 0 2024-01-24T23:26:58.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f050a7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:26:58.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f050a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:58.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 5 monclient: authenticate success, global_id 6386 2024-01-24T23:26:58.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 10 monclient: _renew_subs 2024-01-24T23:26:58.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:58.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.555+0000 7f051bd8f640 1 -- 172.21.15.132:0/41143097 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f051409bdf0 con 0x7f0514071d70 2024-01-24T23:26:58.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.559+0000 7f051bd8f640 10 monclient: _renew_subs 2024-01-24T23:26:58.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.559+0000 7f051bd8f640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:58.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.559+0000 7f051bd8f640 1 -- 172.21.15.132:0/41143097 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f051406ef50 con 0x7f0514071d70 2024-01-24T23:26:58.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.559+0000 7f050a7fc640 1 -- 172.21.15.132:0/41143097 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0510004ec0 con 0x7f0514071d70 2024-01-24T23:26:58.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.559+0000 7f050a7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:58.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.559+0000 7f050a7fc640 1 -- 172.21.15.132:0/41143097 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0510005370 con 0x7f0514071d70 2024-01-24T23:26:58.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.559+0000 7f050a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:58.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.559+0000 7f050bfff640 4 set_mon_vals no callback set 2024-01-24T23:26:58.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.559+0000 7f050a7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:58.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.559+0000 7f050a7fc640 10 monclient: dump: 2024-01-24T23:26:58.574 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:58.574 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:58.574 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:58.574 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:58.574 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:58.574 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:58.574 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:58.574 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:58.574 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:58.574 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:58.574 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:58.575 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:58.575 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:58.575 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:58.575 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:58.575 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:58.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.559+0000 7f051bd8f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:58.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.559+0000 7f051bd8f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:58.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.559+0000 7f051bd8f640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:58.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.559+0000 7f051bd8f640 1 -- 172.21.15.132:0/41143097 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04fc005090 con 0x7f0514071d70 2024-01-24T23:26:58.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.559+0000 7f050a7fc640 1 -- 172.21.15.132:0/41143097 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f051000fdf0 con 0x7f0514071d70 2024-01-24T23:26:58.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.559+0000 7f050a7fc640 1 -- 172.21.15.132:0/41143097 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(200..200 src has 1..200) v4 ==== 12035+0+0 (unknown 319148842 0 0) 0x7f0510045c80 con 0x7f0514071d70 2024-01-24T23:26:58.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.559+0000 7f051932a640 10 monclient: get_auth_request con 0x7f04f8037030 auth_method 0 2024-01-24T23:26:58.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.563+0000 7f050a7fc640 1 -- 172.21.15.132:0/41143097 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f05100112f0 con 0x7f0514071d70 2024-01-24T23:26:58.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.563+0000 7f050a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:58.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.563+0000 7f050a7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:58.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.719+0000 7f051bd8f640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:58.721 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.719+0000 7f051bd8f640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:58.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.719+0000 7f051bd8f640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:58.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:58.719+0000 7f051bd8f640 1 -- 172.21.15.132:0/41143097 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f04fc0058c0 con 0x7f0514071d70 2024-01-24T23:26:59.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.311+0000 7f050a7fc640 1 -- 172.21.15.132:0/41143097 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}]=0 added bucket h22 type host to location {rack=r1} v201) v1 ==== 168+0+0 (unknown 923439653 0 0) 0x7f0510014af0 con 0x7f0514071d70 2024-01-24T23:26:59.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.311+0000 7f050a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:59.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.311+0000 7f050a7fc640 10 monclient: _finish_command 2 = system:0 added bucket h22 type host to location {rack=r1} 2024-01-24T23:26:59.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.415+0000 7f051bd8f640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:59.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.415+0000 7f051bd8f640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:59.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.415+0000 7f051bd8f640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:26:59.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.415+0000 7f051bd8f640 1 -- 172.21.15.132:0/41143097 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f04fc002220 con 0x7f0514071d70 2024-01-24T23:26:59.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.415+0000 7f050a7fc640 1 -- 172.21.15.132:0/41143097 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h22' already exists v201) v1 ==== 147+0+0 (unknown 2054801815 0 0) 0x7f0510014cd0 con 0x7f0514071d70 2024-01-24T23:26:59.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.415+0000 7f050a7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:59.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.415+0000 7f050a7fc640 10 monclient: _finish_command 3 = system:0 bucket 'h22' already exists 2024-01-24T23:26:59.417 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h22' already exists 2024-01-24T23:26:59.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.415+0000 7f051bd8f640 1 -- 172.21.15.132:0/41143097 >> v1:172.21.15.132:6812/1835401922 conn(0x7f04f8037030 legacy=0x7f04f80394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:59.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.415+0000 7f051bd8f640 10 monclient: shutdown 2024-01-24T23:26:59.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.415+0000 7f051bd8f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:59.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.415+0000 7f051bd8f640 1 -- 172.21.15.132:0/41143097 >> v1:172.21.15.132:6791/0 conn(0x7f0514071d70 legacy=0x7f0514139460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:59.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.419+0000 7f051bd8f640 1 -- 172.21.15.132:0/41143097 shutdown_connections 2024-01-24T23:26:59.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.419+0000 7f051bd8f640 1 -- 172.21.15.132:0/41143097 >> 172.21.15.132:0/41143097 conn(0x7f051406aaf0 msgr2=0x7f05140a0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:59.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.419+0000 7f051bd8f640 1 -- 172.21.15.132:0/41143097 shutdown_connections 2024-01-24T23:26:59.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.419+0000 7f051bd8f640 1 -- 172.21.15.132:0/41143097 wait complete. 2024-01-24T23:26:59.433 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:26:59.433 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h23 host rack=r1 2024-01-24T23:26:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.551+0000 7f441619c640 10 monclient: get_monmap_and_config 2024-01-24T23:26:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.551+0000 7f441619c640 10 monclient: build_initial_monmap 2024-01-24T23:26:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.551+0000 7f441619c640 10 monclient: monmap: 2024-01-24T23:26:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:26:59.556691+0000 2024-01-24T23:26:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:26:59.556691+0000 2024-01-24T23:26:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:59.555 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:59.556 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:59.556 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:59.556 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:59.556 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:59.556 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:59.556 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:59.556 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:59.556 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:59.556 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:59.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 1 Processor -- start 2024-01-24T23:26:59.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 1 -- start start 2024-01-24T23:26:59.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 10 monclient: init 2024-01-24T23:26:59.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:59.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 10 monclient: _add_conns ranks=[7,5,1,8,3,2,6,0,4] 2024-01-24T23:26:59.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 10 monclient(hunting): picked mon.noname-i con 0x7f4410133ed0 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:59.561 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 10 monclient(hunting): picked mon.noname-h con 0x7f44101342b0 addr v1:172.21.15.132:6791/0 2024-01-24T23:26:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 10 monclient(hunting): picked mon.noname-f con 0x7f4410134690 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4410064700 con 0x7f4410134690 2024-01-24T23:26:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f441009b3d0 con 0x7f44101342b0 2024-01-24T23:26:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44100a58c0 con 0x7f4410133ed0 2024-01-24T23:26:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 10 monclient(hunting): authenticate will time out at 1306.881493s 2024-01-24T23:26:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440ffff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f4410134690 legacy=0x7f441013f770 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440f7fe640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f4410133ed0 legacy=0x7f441009bdb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f4407fff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f44101342b0 0x7f441013c040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:36896/0 (socket says 172.21.15.132:36896) 2024-01-24T23:26:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f4407fff640 1 -- 172.21.15.132:0/2498613833 learned_addr learned my addr 172.21.15.132:0/2498613833 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 1 -- 172.21.15.132:0/2498613833 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 921633146 0 0) 0x7f441009b3d0 con 0x7f44101342b0 2024-01-24T23:26:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 10 monclient(hunting): _negotiate 2024-01-24T23:26:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:59.562 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:59.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 10 monclient(hunting): my global_id is 6395 2024-01-24T23:26:59.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 1 -- 172.21.15.132:0/2498613833 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43fc003530 con 0x7f44101342b0 2024-01-24T23:26:59.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 1 -- 172.21.15.132:0/2498613833 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 180395531 0 0) 0x7f43fc003530 con 0x7f44101342b0 2024-01-24T23:26:59.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 1 -- 172.21.15.132:0/2498613833 >> v1:172.21.15.132:6792/0 conn(0x7f4410133ed0 legacy=0x7f441009bdb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:59.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 1 -- 172.21.15.132:0/2498613833 >> v1:172.21.15.132:6789/0 conn(0x7f4410134690 legacy=0x7f441013f770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:26:59.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 10 monclient: _finish_hunting 0 2024-01-24T23:26:59.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 1 monclient: found mon.noname-h 2024-01-24T23:26:59.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:59.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:26:59.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 1 -- 172.21.15.132:0/2498613833 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441009b0f0 con 0x7f44101342b0 2024-01-24T23:26:59.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 10 monclient: _finish_auth 0 2024-01-24T23:26:59.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 10 monclient: _check_auth_tickets 2024-01-24T23:26:59.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:59.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 1 -- 172.21.15.132:0/2498613833 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f43f80017c0 con 0x7f44101342b0 2024-01-24T23:26:59.563 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:59.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:26:59.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 10 monclient: dump: 2024-01-24T23:26:59.564 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:59.564 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:59.564 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:59.564 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:59.564 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:59.564 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:59.564 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:59.564 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:59.564 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:59.564 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:59.564 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:59.564 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:59.565 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 5 monclient: authenticate success, global_id 6395 2024-01-24T23:26:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:26:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 1 -- 172.21.15.132:0/2498613833 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f43f8001ae0 con 0x7f44101342b0 2024-01-24T23:26:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 1 -- 172.21.15.132:0/2498613833 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f43f8004bf0 con 0x7f44101342b0 2024-01-24T23:26:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 10 monclient: get_monmap_and_config success 2024-01-24T23:26:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 4 set_mon_vals no callback set 2024-01-24T23:26:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:26:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f440effd640 10 monclient: dump: 2024-01-24T23:26:59.565 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:59.566 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:59.567 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:59.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 10 monclient: shutdown 2024-01-24T23:26:59.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:26:59.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 1 -- 172.21.15.132:0/2498613833 >> v1:172.21.15.132:6791/0 conn(0x7f44101342b0 legacy=0x7f441013c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:26:59.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 1 -- 172.21.15.132:0/2498613833 shutdown_connections 2024-01-24T23:26:59.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 1 -- 172.21.15.132:0/2498613833 >> 172.21.15.132:0/2498613833 conn(0x7f44100a5ef0 msgr2=0x7f44100a6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:26:59.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 1 -- 172.21.15.132:0/2498613833 shutdown_connections 2024-01-24T23:26:59.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 1 -- 172.21.15.132:0/2498613833 wait complete. 2024-01-24T23:26:59.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 10 monclient: build_initial_monmap 2024-01-24T23:26:59.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 10 monclient: monmap: 2024-01-24T23:26:59.567 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:26:59.567 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:26:59.567 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:26:59.567 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:26:59.568 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:26:59.568 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:26:59.568 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:26:59.568 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:26:59.568 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:26:59.568 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:26:59.568 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:26:59.568 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:26:59.568 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:26:59.568 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:26:59.568 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:26:59.568 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:59.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 1 Processor -- start 2024-01-24T23:26:59.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 1 -- start start 2024-01-24T23:26:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.555+0000 7f441619c640 10 monclient: init 2024-01-24T23:26:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f441619c640 10 monclient: _reopen_session rank -1 2024-01-24T23:26:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f441619c640 10 monclient: _add_conns ranks=[7,1,8,4,6,2,5,0,3] 2024-01-24T23:26:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f441619c640 10 monclient(hunting): picked mon.noname-i con 0x7f4410133ed0 addr v1:172.21.15.132:6792/0 2024-01-24T23:26:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f441619c640 10 monclient(hunting): picked mon.noname-f con 0x7f44101342b0 addr v1:172.21.15.132:6789/0 2024-01-24T23:26:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f441619c640 10 monclient(hunting): picked mon.noname-e con 0x7f4410134690 addr v1:172.21.15.115:6793/0 2024-01-24T23:26:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f441619c640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44101201e0 con 0x7f44101342b0 2024-01-24T23:26:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f441619c640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44100a2240 con 0x7f4410133ed0 2024-01-24T23:26:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f441619c640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f441009ca90 con 0x7f4410134690 2024-01-24T23:26:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f441619c640 10 monclient(hunting): _renew_subs 2024-01-24T23:26:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440f7fe640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f4410133ed0 legacy=0x7f4410132b00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:26:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f441619c640 10 monclient(hunting): authenticate will time out at 1306.884399s 2024-01-24T23:26:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f4407fff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f44101342b0 legacy=0x7f44101d0170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:26:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440ffff640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f4410134690 0x7f44101d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:51646/0 (socket says 172.21.15.132:51646) 2024-01-24T23:26:59.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440ffff640 1 -- 172.21.15.132:0/4234726745 learned_addr learned my addr 172.21.15.132:0/4234726745 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:26:59.570 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 1 -- 172.21.15.132:0/4234726745 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2274040407 0 0) 0x7f441009ca90 con 0x7f4410134690 2024-01-24T23:26:59.570 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:26:59.570 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:26:59.570 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:26:59.570 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 10 monclient(hunting): my global_id is 85715 2024-01-24T23:26:59.570 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 1 -- 172.21.15.132:0/4234726745 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43f0003560 con 0x7f4410134690 2024-01-24T23:26:59.570 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 1 -- 172.21.15.132:0/4234726745 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 151723150 0 0) 0x7f43f0003560 con 0x7f4410134690 2024-01-24T23:26:59.570 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 1 -- 172.21.15.132:0/4234726745 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f441009ca90 con 0x7f4410134690 2024-01-24T23:26:59.570 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 1 -- 172.21.15.132:0/4234726745 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4408004560 con 0x7f4410134690 2024-01-24T23:26:59.570 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:59.570 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:26:59.570 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 10 monclient(hunting): dump: 2024-01-24T23:26:59.570 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:59.570 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:59.571 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:59.571 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:59.571 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:59.571 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:59.571 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:59.571 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:59.571 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:59.571 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:59.571 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:59.571 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:59.571 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:59.571 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:59.571 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:59.571 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:59.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:26:59.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 1 -- 172.21.15.132:0/4234726745 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 866366438 0 0) 0x7f441009ca90 con 0x7f4410134690 2024-01-24T23:26:59.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 1 -- 172.21.15.132:0/4234726745 >> v1:172.21.15.132:6792/0 conn(0x7f4410133ed0 legacy=0x7f4410132b00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:59.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 1 -- 172.21.15.132:0/4234726745 >> v1:172.21.15.132:6789/0 conn(0x7f44101342b0 legacy=0x7f44101d0170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:26:59.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 10 monclient: _finish_hunting 0 2024-01-24T23:26:59.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 1 monclient: found mon.e 2024-01-24T23:26:59.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:26:59.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:59.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 1 -- 172.21.15.132:0/4234726745 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44100a5af0 con 0x7f4410134690 2024-01-24T23:26:59.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 10 monclient: _finish_auth 0 2024-01-24T23:26:59.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 10 monclient: _check_auth_tickets 2024-01-24T23:26:59.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:26:59.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f441619c640 5 monclient: authenticate success, global_id 85715 2024-01-24T23:26:59.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f441619c640 10 monclient: _renew_subs 2024-01-24T23:26:59.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f441619c640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:59.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f441619c640 1 -- 172.21.15.132:0/4234726745 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f441011ff00 con 0x7f4410134690 2024-01-24T23:26:59.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f441619c640 10 monclient: _renew_subs 2024-01-24T23:26:59.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f441619c640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:59.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f441619c640 1 -- 172.21.15.132:0/4234726745 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4410133210 con 0x7f4410134690 2024-01-24T23:26:59.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 1 -- 172.21.15.132:0/4234726745 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4408004780 con 0x7f4410134690 2024-01-24T23:26:59.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:26:59.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 1 -- 172.21.15.132:0/4234726745 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4408004cb0 con 0x7f4410134690 2024-01-24T23:26:59.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:26:59.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:26:59.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440e7fc640 4 set_mon_vals no callback set 2024-01-24T23:26:59.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 10 monclient: dump: 2024-01-24T23:26:59.573 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:26:59.573 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:26:59.573 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:59.573 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:26:59.574 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:26:59.574 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:26:59.574 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:26:59.574 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:26:59.574 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:26:59.574 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:26:59.574 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:26:59.574 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:26:59.574 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:26:59.574 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:26:59.574 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:26:59.574 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:26:59.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f441619c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:59.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f441619c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:59.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f441619c640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:59.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f441619c640 1 -- 172.21.15.132:0/4234726745 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f4005090 con 0x7f4410134690 2024-01-24T23:26:59.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 1 -- 172.21.15.132:0/4234726745 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f4408004e90 con 0x7f4410134690 2024-01-24T23:26:59.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440cff9640 1 -- 172.21.15.132:0/4234726745 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(201..201 src has 1..201) v4 ==== 12248+0+0 (unknown 4113686896 0 0) 0x7f4408045af0 con 0x7f4410134690 2024-01-24T23:26:59.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.559+0000 7f440f7fe640 10 monclient: get_auth_request con 0x7f43f0036f70 auth_method 0 2024-01-24T23:26:59.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.571+0000 7f440cff9640 1 -- 172.21.15.132:0/4234726745 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f4408014c40 con 0x7f4410134690 2024-01-24T23:26:59.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.571+0000 7f440cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:26:59.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.571+0000 7f440cff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:26:59.732 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.731+0000 7f441619c640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:59.750 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.731+0000 7f441619c640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:26:59.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.731+0000 7f441619c640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:26:59.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:26:59.731+0000 7f441619c640 1 -- 172.21.15.132:0/4234726745 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f43f4005340 con 0x7f4410134690 2024-01-24T23:27:00.129 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~56s 2024-01-24T23:27:00.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.339+0000 7f440cff9640 1 -- 172.21.15.132:0/4234726745 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}]=0 added bucket h23 type host to location {rack=r1} v202) v1 ==== 168+0+0 (unknown 4214075221 0 0) 0x7f4408014830 con 0x7f4410134690 2024-01-24T23:27:00.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.339+0000 7f440cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:00.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.339+0000 7f440cff9640 10 monclient: _finish_command 2 = system:0 added bucket h23 type host to location {rack=r1} 2024-01-24T23:27:00.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.439+0000 7f441619c640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:00.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.439+0000 7f441619c640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:00.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.439+0000 7f441619c640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:27:00.443 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.439+0000 7f441619c640 1 -- 172.21.15.132:0/4234726745 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f43f4005a80 con 0x7f4410134690 2024-01-24T23:27:00.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.443+0000 7f440cff9640 1 -- 172.21.15.132:0/4234726745 <== mon.8 v1:172.21.15.115:6793/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h23' already exists v202) v1 ==== 147+0+0 (unknown 1486673811 0 0) 0x7f4408014a10 con 0x7f4410134690 2024-01-24T23:27:00.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.443+0000 7f440cff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:00.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.443+0000 7f440cff9640 10 monclient: _finish_command 3 = system:0 bucket 'h23' already exists 2024-01-24T23:27:00.445 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h23' already exists 2024-01-24T23:27:00.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.447+0000 7f441619c640 1 -- 172.21.15.132:0/4234726745 >> v1:172.21.15.132:6812/1835401922 conn(0x7f43f0036f70 legacy=0x7f43f0039430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:00.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.447+0000 7f441619c640 10 monclient: shutdown 2024-01-24T23:27:00.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.447+0000 7f441619c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:00.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.447+0000 7f441619c640 1 -- 172.21.15.132:0/4234726745 >> v1:172.21.15.115:6793/0 conn(0x7f4410134690 legacy=0x7f44101d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:00.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.447+0000 7f441619c640 1 -- 172.21.15.132:0/4234726745 shutdown_connections 2024-01-24T23:27:00.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.447+0000 7f441619c640 1 -- 172.21.15.132:0/4234726745 >> 172.21.15.132:0/4234726745 conn(0x7f44100a5ef0 msgr2=0x7f44101bea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:00.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.447+0000 7f441619c640 1 -- 172.21.15.132:0/4234726745 shutdown_connections 2024-01-24T23:27:00.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.447+0000 7f441619c640 1 -- 172.21.15.132:0/4234726745 wait complete. 2024-01-24T23:27:00.463 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:00.463 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h24 host rack=r1 2024-01-24T23:27:00.584 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5fab840640 10 monclient: get_monmap_and_config 2024-01-24T23:27:00.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5fab840640 10 monclient: build_initial_monmap 2024-01-24T23:27:00.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5fab840640 10 monclient: monmap: 2024-01-24T23:27:00.585 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:00.585 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:00.585 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:00.586343+0000 2024-01-24T23:27:00.585 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:00.586343+0000 2024-01-24T23:27:00.585 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:00.585 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:00.585 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:00.585 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:00.585 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:00.585 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:00.585 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:00.585 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:00.585 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:00.586 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:00.586 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:00.586 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:00.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5fab840640 1 Processor -- start 2024-01-24T23:27:00.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5fab840640 1 -- start start 2024-01-24T23:27:00.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5fab840640 10 monclient: init 2024-01-24T23:27:00.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5fab840640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:00.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5fab840640 10 monclient: _add_conns ranks=[0,1,5,3,7,6,8,2,4] 2024-01-24T23:27:00.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5fab840640 10 monclient(hunting): picked mon.noname-a con 0x7f5fa40a0ad0 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:00.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5fab840640 10 monclient(hunting): picked mon.noname-f con 0x7f5fa409ee80 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:00.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5fab840640 10 monclient(hunting): picked mon.noname-h con 0x7f5fa409b060 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:00.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5fab840640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa4064700 con 0x7f5fa40a0ad0 2024-01-24T23:27:00.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5fab840640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa409cb80 con 0x7f5fa409ee80 2024-01-24T23:27:00.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5fab840640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa40a0eb0 con 0x7f5fa409b060 2024-01-24T23:27:00.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5fab840640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:00.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5fab840640 10 monclient(hunting): authenticate will time out at 1307.910955s 2024-01-24T23:27:00.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5fa8ddb640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f5fa409ee80 legacy=0x7f5fa409a950 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:27:00.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5fa9ddd640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f5fa409b060 0x7f5fa413c270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:36902/0 (socket says 172.21.15.132:36902) 2024-01-24T23:27:00.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5fa9ddd640 1 -- 172.21.15.132:0/3190789785 learned_addr learned my addr 172.21.15.132:0/3190789785 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:00.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 1 -- 172.21.15.132:0/3190789785 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1363850450 0 0) 0x7f5fa40a0eb0 con 0x7f5fa409b060 2024-01-24T23:27:00.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:00.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:00.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:00.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 10 monclient(hunting): my global_id is 6404 2024-01-24T23:27:00.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 1 -- 172.21.15.132:0/3190789785 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f90003530 con 0x7f5fa409b060 2024-01-24T23:27:00.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 1 -- 172.21.15.132:0/3190789785 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3288471889 0 0) 0x7f5f90003530 con 0x7f5fa409b060 2024-01-24T23:27:00.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 1 -- 172.21.15.132:0/3190789785 >> v1:172.21.15.132:6789/0 conn(0x7f5fa409ee80 legacy=0x7f5fa409a950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:00.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 1 -- 172.21.15.132:0/3190789785 >> v1:172.21.15.115:6789/0 conn(0x7f5fa40a0ad0 legacy=0x7f5fa409a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:00.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:27:00.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 1 monclient: found mon.noname-h 2024-01-24T23:27:00.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:00.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:27:00.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 1 -- 172.21.15.132:0/3190789785 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa409f280 con 0x7f5fa409b060 2024-01-24T23:27:00.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 10 monclient: _finish_auth 0 2024-01-24T23:27:00.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:27:00.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:00.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5fab840640 5 monclient: authenticate success, global_id 6404 2024-01-24T23:27:00.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 1 -- 172.21.15.132:0/3190789785 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5f9c001b40 con 0x7f5fa409b060 2024-01-24T23:27:00.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5fab840640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:00.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:00.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:27:00.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 10 monclient: dump: 2024-01-24T23:27:00.595 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:00.595 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:00.595 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:00.595 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:00.595 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:00.595 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:00.595 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:00.595 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:00.595 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:00.595 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:00.595 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:00.595 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:00.596 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:00.596 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:00.596 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:00.596 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:00.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5fab840640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:00.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 1 -- 172.21.15.132:0/3190789785 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5f9c001df0 con 0x7f5fa409b060 2024-01-24T23:27:00.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:00.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5fab840640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:00.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 1 -- 172.21.15.132:0/3190789785 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5f9c004f70 con 0x7f5fa409b060 2024-01-24T23:27:00.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5fab840640 4 set_mon_vals no callback set 2024-01-24T23:27:00.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:00.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:00.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.582+0000 7f5f9bfff640 10 monclient: dump: 2024-01-24T23:27:00.596 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:00.597 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:00.597 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:00.597 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:00.597 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:00.597 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:00.597 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:00.597 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:00.597 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:00.597 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:00.597 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:00.597 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:00.597 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:00.598 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:00.598 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:00.598 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:00.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fab840640 10 monclient: shutdown 2024-01-24T23:27:00.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fab840640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:00.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fab840640 1 -- 172.21.15.132:0/3190789785 >> v1:172.21.15.132:6791/0 conn(0x7f5fa409b060 legacy=0x7f5fa413c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:00.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fab840640 1 -- 172.21.15.132:0/3190789785 shutdown_connections 2024-01-24T23:27:00.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fab840640 1 -- 172.21.15.132:0/3190789785 >> 172.21.15.132:0/3190789785 conn(0x7f5fa40a1bc0 msgr2=0x7f5fa40a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:00.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fab840640 1 -- 172.21.15.132:0/3190789785 shutdown_connections 2024-01-24T23:27:00.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fab840640 1 -- 172.21.15.132:0/3190789785 wait complete. 2024-01-24T23:27:00.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fab840640 10 monclient: build_initial_monmap 2024-01-24T23:27:00.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fab840640 10 monclient: monmap: 2024-01-24T23:27:00.598 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:00.598 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:00.598 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:00.598 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:00.599 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:00.599 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:00.599 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:00.599 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:00.599 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:00.599 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:00.599 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:00.599 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:00.599 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:00.599 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:00.599 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:00.599 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:00.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fab840640 1 Processor -- start 2024-01-24T23:27:00.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fab840640 1 -- start start 2024-01-24T23:27:00.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fab840640 10 monclient: init 2024-01-24T23:27:00.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fab840640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:00.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fab840640 10 monclient: _add_conns ranks=[2,1,7,5,0,8,4,6,3] 2024-01-24T23:27:00.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fab840640 10 monclient(hunting): picked mon.noname-b con 0x7f5fa409b060 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:00.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fab840640 10 monclient(hunting): picked mon.noname-f con 0x7f5fa409ee80 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:00.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fab840640 10 monclient(hunting): picked mon.noname-i con 0x7f5fa40a0ad0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:00.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fab840640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa4093490 con 0x7f5fa409ee80 2024-01-24T23:27:00.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fab840640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa409b4c0 con 0x7f5fa409b060 2024-01-24T23:27:00.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fa8ddb640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f5fa409ee80 legacy=0x7f5fa41c8940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:27:00.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fab840640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa408e490 con 0x7f5fa40a0ad0 2024-01-24T23:27:00.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fab840640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:00.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fa9ddd640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f5fa40a0ad0 legacy=0x7f5fa41d06a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:00.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fab840640 10 monclient(hunting): authenticate will time out at 1307.914207s 2024-01-24T23:27:00.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.586+0000 7f5fa95dc640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f5fa409b060 legacy=0x7f5fa41c8230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:00.789 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.790+0000 7f5fa8ddb640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f5fa409ee80 legacy=0x7f5fa41c8940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:27:00.789 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.790+0000 7f5fa9ddd640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f5fa40a0ad0 legacy=0x7f5fa41d06a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:00.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:00.790+0000 7f5fa95dc640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f5fa409b060 legacy=0x7f5fa41c8230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:01.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:01.194+0000 7f5fa8ddb640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f5fa409ee80 legacy=0x7f5fa41c8940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:27:01.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:01.194+0000 7f5fa9ddd640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f5fa40a0ad0 legacy=0x7f5fa41d06a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:01.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:01.194+0000 7f5fa95dc640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f5fa409b060 legacy=0x7f5fa41c8230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:01.999 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:01.998+0000 7f5fa8ddb640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f5fa409ee80 legacy=0x7f5fa41c8940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:27:01.999 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:01.998+0000 7f5fa9ddd640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f5fa40a0ad0 legacy=0x7f5fa41d06a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:01.999 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:01.998+0000 7f5fa95dc640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f5fa409b060 legacy=0x7f5fa41c8230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:03.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f98ff9640 10 monclient(hunting): tick 2024-01-24T23:27:03.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f98ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:27:03.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f98ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:27:03.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f98ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:27:03.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f98ff9640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f5fa40a0ad0 legacy=0x7f5fa41d06a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:03.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f98ff9640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f5fa409b060 legacy=0x7f5fa41c8230 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:03.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f98ff9640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f5fa409ee80 legacy=0x7f5fa41c8940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:03.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f98ff9640 10 monclient: _add_conns ranks=[4,5,0,6,8,3] 2024-01-24T23:27:03.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f98ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f5f8c0059a0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:03.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f98ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f5f8c009630 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:03.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f98ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f5f8c00d3c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:03.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f98ff9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa409e640 con 0x7f5f8c00d3c0 2024-01-24T23:27:03.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f98ff9640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa4093490 con 0x7f5f8c0059a0 2024-01-24T23:27:03.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f98ff9640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa409b4c0 con 0x7f5f8c009630 2024-01-24T23:27:03.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f98ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:03.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5fa8ddb640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f5f8c009630 0x7f5f8c00baf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:36914/0 (socket says 172.21.15.132:36914) 2024-01-24T23:27:03.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5fa8ddb640 1 -- 172.21.15.132:0/611452661 learned_addr learned my addr 172.21.15.132:0/611452661 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:03.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1411634117 0 0) 0x7f5fa409b4c0 con 0x7f5f8c009630 2024-01-24T23:27:03.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f99ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:03.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f99ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:03.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f99ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:03.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f99ffb640 10 monclient(hunting): my global_id is 6413 2024-01-24T23:27:03.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f88003530 con 0x7f5f8c009630 2024-01-24T23:27:03.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1125670893 0 0) 0x7f5fa4093490 con 0x7f5f8c0059a0 2024-01-24T23:27:03.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f99ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:03.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f99ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:03.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f99ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:03.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f99ffb640 10 monclient(hunting): my global_id is 94630 2024-01-24T23:27:03.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fa409b4c0 con 0x7f5f8c0059a0 2024-01-24T23:27:03.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1913436904 0 0) 0x7f5fa409e640 con 0x7f5f8c00d3c0 2024-01-24T23:27:03.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f99ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:03.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f99ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:03.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f99ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:03.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f99ffb640 10 monclient(hunting): my global_id is 94815 2024-01-24T23:27:03.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.586+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fa4093490 con 0x7f5f8c00d3c0 2024-01-24T23:27:03.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1547051517 0 0) 0x7f5f88003530 con 0x7f5f8c009630 2024-01-24T23:27:03.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5fa409e640 con 0x7f5f8c009630 2024-01-24T23:27:03.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5f94004450 con 0x7f5f8c009630 2024-01-24T23:27:03.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:03.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:27:03.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 10 monclient(hunting): dump: 2024-01-24T23:27:03.596 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:03.596 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:03.596 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:03.596 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:03.596 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:03.596 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:03.596 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:03.596 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:03.596 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:03.596 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:03.596 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:03.596 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:03.596 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:03.596 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:03.596 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:03.597 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:03.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:03.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1924440153 0 0) 0x7f5fa4093490 con 0x7f5f8c00d3c0 2024-01-24T23:27:03.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5f88003530 con 0x7f5f8c00d3c0 2024-01-24T23:27:03.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5f9c004eb0 con 0x7f5f8c00d3c0 2024-01-24T23:27:03.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:03.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:27:03.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 10 monclient(hunting): dump: 2024-01-24T23:27:03.597 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:03.597 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:03.597 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:03.597 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:03.597 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:03.597 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:03.597 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:03.598 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:03.598 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:03.598 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:03.598 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:03.598 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:03.598 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:03.598 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:03.598 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:03.598 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:03.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2566006388 0 0) 0x7f5fa409b4c0 con 0x7f5f8c0059a0 2024-01-24T23:27:03.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5fa4093490 con 0x7f5f8c0059a0 2024-01-24T23:27:03.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5fa0001a30 con 0x7f5f8c0059a0 2024-01-24T23:27:03.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:03.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:03.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 10 monclient(hunting): dump: 2024-01-24T23:27:03.598 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:03.599 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:03.599 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:03.599 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:03.599 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:03.599 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:03.599 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:03.599 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:03.599 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:03.599 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:03.599 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:03.599 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:03.599 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:03.599 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:03.599 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:03.599 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:03.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2418713011 0 0) 0x7f5fa409e640 con 0x7f5f8c009630 2024-01-24T23:27:03.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 >> v1:172.21.15.115:6791/0 conn(0x7f5f8c0059a0 legacy=0x7f5f8c007df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:03.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 >> v1:172.21.15.115:6789/0 conn(0x7f5f8c00d3c0 legacy=0x7f5f8c00f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:03.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:27:03.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 1 monclient: found mon.h 2024-01-24T23:27:03.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:03.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5fa9ddd640 1 -- 172.21.15.132:0/611452661 reap_dead start 2024-01-24T23:27:03.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:03.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa408e490 con 0x7f5f8c009630 2024-01-24T23:27:03.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 10 monclient: _finish_auth 0 2024-01-24T23:27:03.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:27:03.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:03.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5fab840640 5 monclient: authenticate success, global_id 6413 2024-01-24T23:27:03.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5fab840640 10 monclient: _renew_subs 2024-01-24T23:27:03.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5fab840640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:03.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5fab840640 1 -- 172.21.15.132:0/611452661 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa40715b0 con 0x7f5f8c009630 2024-01-24T23:27:03.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5fab840640 10 monclient: _renew_subs 2024-01-24T23:27:03.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5fab840640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:03.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5fab840640 1 -- 172.21.15.132:0/611452661 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa4071c90 con 0x7f5f8c009630 2024-01-24T23:27:03.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5f94004a30 con 0x7f5f8c009630 2024-01-24T23:27:03.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:03.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5f94004f60 con 0x7f5f8c009630 2024-01-24T23:27:03.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:03.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:03.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f9affd640 4 set_mon_vals no callback set 2024-01-24T23:27:03.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 10 monclient: dump: 2024-01-24T23:27:03.601 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:03.601 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:03.601 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:03.601 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:03.602 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:03.602 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:03.602 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:03.602 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:03.602 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:03.602 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:03.602 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:03.602 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:03.602 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:03.602 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:03.602 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:03.602 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:03.602 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5fab840640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:03.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5fab840640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:03.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5fab840640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:03.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5fab840640 1 -- 172.21.15.132:0/611452661 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f80005090 con 0x7f5f8c009630 2024-01-24T23:27:03.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f5f9400fa60 con 0x7f5f8c009630 2024-01-24T23:27:03.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.590+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(203..203 src has 1..203) v4 ==== 12461+0+0 (unknown 1111790764 0 0) 0x7f5f94045b30 con 0x7f5f8c009630 2024-01-24T23:27:03.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.594+0000 7f5fa95dc640 10 monclient: get_auth_request con 0x7f5f88037b30 auth_method 0 2024-01-24T23:27:03.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.598+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f5f940149e0 con 0x7f5f8c009630 2024-01-24T23:27:03.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.598+0000 7f5f99ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:03.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.598+0000 7f5f99ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:03.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.762+0000 7f5fab840640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:03.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.762+0000 7f5fab840640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:03.766 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.762+0000 7f5fab840640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:03.766 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.762+0000 7f5fab840640 1 -- 172.21.15.132:0/611452661 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f5f800058c0 con 0x7f5f8c009630 2024-01-24T23:27:03.941 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.938+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}]=0 added bucket h24 type host to location {rack=r1} v204) v1 ==== 168+0+0 (unknown 1347463017 0 0) 0x7f5f94013080 con 0x7f5f8c009630 2024-01-24T23:27:03.941 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.938+0000 7f5f99ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:03.941 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:03.938+0000 7f5f99ffb640 10 monclient: _finish_command 2 = system:0 added bucket h24 type host to location {rack=r1} 2024-01-24T23:27:04.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.042+0000 7f5fab840640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:04.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.042+0000 7f5fab840640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:04.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.042+0000 7f5fab840640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:04.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.042+0000 7f5fab840640 1 -- 172.21.15.132:0/611452661 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f5f80002220 con 0x7f5f8c009630 2024-01-24T23:27:04.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.042+0000 7f5f99ffb640 1 -- 172.21.15.132:0/611452661 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h24' already exists v204) v1 ==== 147+0+0 (unknown 4240515740 0 0) 0x7f5f94014800 con 0x7f5f8c009630 2024-01-24T23:27:04.046 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.042+0000 7f5f99ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:04.046 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.042+0000 7f5f99ffb640 10 monclient: _finish_command 3 = system:0 bucket 'h24' already exists 2024-01-24T23:27:04.046 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h24' already exists 2024-01-24T23:27:04.047 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.046+0000 7f5fab840640 1 -- 172.21.15.132:0/611452661 >> v1:172.21.15.132:6812/1835401922 conn(0x7f5f88037b30 legacy=0x7f5f88039ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:04.047 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.046+0000 7f5fab840640 10 monclient: shutdown 2024-01-24T23:27:04.047 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.046+0000 7f5fab840640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:04.047 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.046+0000 7f5fab840640 1 -- 172.21.15.132:0/611452661 >> v1:172.21.15.132:6791/0 conn(0x7f5f8c009630 legacy=0x7f5f8c00baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:04.047 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.046+0000 7f5fab840640 1 -- 172.21.15.132:0/611452661 shutdown_connections 2024-01-24T23:27:04.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.046+0000 7f5fab840640 1 -- 172.21.15.132:0/611452661 >> 172.21.15.132:0/611452661 conn(0x7f5fa40a1bc0 msgr2=0x7f5fa40a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:04.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.046+0000 7f5fab840640 1 -- 172.21.15.132:0/611452661 shutdown_connections 2024-01-24T23:27:04.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.046+0000 7f5fab840640 1 -- 172.21.15.132:0/611452661 wait complete. 2024-01-24T23:27:04.062 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:04.062 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h25 host rack=r1 2024-01-24T23:27:04.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 10 monclient: get_monmap_and_config 2024-01-24T23:27:04.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 10 monclient: build_initial_monmap 2024-01-24T23:27:04.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 10 monclient: monmap: 2024-01-24T23:27:04.183 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:04.183 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:04.183 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:04.184899+0000 2024-01-24T23:27:04.183 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:04.184899+0000 2024-01-24T23:27:04.183 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:04.183 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:04.183 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:04.183 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:04.183 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:04.184 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:04.184 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:04.184 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:04.184 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:04.184 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:04.184 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:04.184 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:04.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 1 Processor -- start 2024-01-24T23:27:04.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 1 -- start start 2024-01-24T23:27:04.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 10 monclient: init 2024-01-24T23:27:04.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:04.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 10 monclient: _add_conns ranks=[2,5,1,3,8,0,4,7,6] 2024-01-24T23:27:04.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 10 monclient(hunting): picked mon.noname-b con 0x7f88900a3730 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:04.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 10 monclient(hunting): picked mon.noname-h con 0x7f8890137cf0 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:04.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 10 monclient(hunting): picked mon.noname-f con 0x7f889013b800 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:04.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8890064700 con 0x7f889013b800 2024-01-24T23:27:04.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88900a3b10 con 0x7f88900a3730 2024-01-24T23:27:04.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8890134ef0 con 0x7f8890137cf0 2024-01-24T23:27:04.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:04.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8896730640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f889013b800 legacy=0x7f889013dbf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:27:04.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 10 monclient(hunting): authenticate will time out at 1311.509633s 2024-01-24T23:27:04.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f889572e640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f8890137cf0 0x7f889013a0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:59658/0 (socket says 172.21.15.132:59658) 2024-01-24T23:27:04.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f889572e640 1 -- 172.21.15.132:0/3272199492 learned_addr learned my addr 172.21.15.132:0/3272199492 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:04.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8895f2f640 1 -- 172.21.15.132:0/3272199492 >> v1:172.21.15.115:6790/0 conn(0x7f88900a3730 legacy=0x7f889009c540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:04.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 1 -- 172.21.15.132:0/3272199492 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1185189164 0 0) 0x7f8890134ef0 con 0x7f8890137cf0 2024-01-24T23:27:04.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 10 monclient(hunting): _negotiate 2024-01-24T23:27:04.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:04.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:04.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 10 monclient(hunting): my global_id is 6422 2024-01-24T23:27:04.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 1 -- 172.21.15.132:0/3272199492 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f887c003530 con 0x7f8890137cf0 2024-01-24T23:27:04.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 1 -- 172.21.15.132:0/3272199492 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2967465919 0 0) 0x7f887c003530 con 0x7f8890137cf0 2024-01-24T23:27:04.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 1 -- 172.21.15.132:0/3272199492 >> v1:172.21.15.115:6790/0 conn(0x7f88900a3730 legacy=0x7f889009c540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:04.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 1 -- 172.21.15.132:0/3272199492 >> v1:172.21.15.132:6789/0 conn(0x7f889013b800 legacy=0x7f889013dbf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:04.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 10 monclient: _finish_hunting 0 2024-01-24T23:27:04.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 1 monclient: found mon.noname-h 2024-01-24T23:27:04.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:04.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:27:04.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 1 -- 172.21.15.132:0/3272199492 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8890135190 con 0x7f8890137cf0 2024-01-24T23:27:04.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 10 monclient: _finish_auth 0 2024-01-24T23:27:04.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 10 monclient: _check_auth_tickets 2024-01-24T23:27:04.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:04.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 5 monclient: authenticate success, global_id 6422 2024-01-24T23:27:04.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:04.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 1 -- 172.21.15.132:0/3272199492 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f888c001b40 con 0x7f8890137cf0 2024-01-24T23:27:04.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:04.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:27:04.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 10 monclient: dump: 2024-01-24T23:27:04.193 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:04.193 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:04.193 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:04.193 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:04.193 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:04.194 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:04.194 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:04.194 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:04.194 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:04.194 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:04.194 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:04.194 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:04.194 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:04.194 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:04.194 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:04.194 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:04.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:04.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 1 -- 172.21.15.132:0/3272199492 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f888c001df0 con 0x7f8890137cf0 2024-01-24T23:27:04.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:04.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 1 -- 172.21.15.132:0/3272199492 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f888c004f70 con 0x7f8890137cf0 2024-01-24T23:27:04.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:04.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 4 set_mon_vals no callback set 2024-01-24T23:27:04.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:04.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:04.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8894f2d640 10 monclient: dump: 2024-01-24T23:27:04.195 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:04.195 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:04.195 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:04.195 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:04.195 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:04.195 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:04.195 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:04.195 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:04.195 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:04.195 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:04.196 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:04.196 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:04.196 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:04.196 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:04.196 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:04.196 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:04.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 10 monclient: shutdown 2024-01-24T23:27:04.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:04.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 1 -- 172.21.15.132:0/3272199492 >> v1:172.21.15.132:6791/0 conn(0x7f8890137cf0 legacy=0x7f889013a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:04.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 1 -- 172.21.15.132:0/3272199492 shutdown_connections 2024-01-24T23:27:04.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 1 -- 172.21.15.132:0/3272199492 >> 172.21.15.132:0/3272199492 conn(0x7f88900a2c40 msgr2=0x7f88900a3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:04.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 1 -- 172.21.15.132:0/3272199492 shutdown_connections 2024-01-24T23:27:04.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 1 -- 172.21.15.132:0/3272199492 wait complete. 2024-01-24T23:27:04.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 10 monclient: build_initial_monmap 2024-01-24T23:27:04.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.182+0000 7f8898193640 10 monclient: monmap: 2024-01-24T23:27:04.197 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:04.197 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:04.197 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:04.197 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:04.197 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:04.197 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:04.197 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:04.197 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:04.197 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:04.197 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:04.197 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:04.197 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:04.197 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:04.197 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:04.198 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:04.198 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:04.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8898193640 1 Processor -- start 2024-01-24T23:27:04.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8898193640 1 -- start start 2024-01-24T23:27:04.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8898193640 10 monclient: init 2024-01-24T23:27:04.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8898193640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:04.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8898193640 10 monclient: _add_conns ranks=[1,3,6,8,7,5,2,0,4] 2024-01-24T23:27:04.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8898193640 10 monclient(hunting): picked mon.noname-f con 0x7f88900a3730 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:04.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8898193640 10 monclient(hunting): picked mon.noname-g con 0x7f8890137cf0 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:04.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8898193640 10 monclient(hunting): picked mon.noname-d con 0x7f889013b800 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:04.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8898193640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f889009afd0 con 0x7f88900a3730 2024-01-24T23:27:04.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8898193640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f889012f860 con 0x7f8890137cf0 2024-01-24T23:27:04.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8898193640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f889012fc80 con 0x7f889013b800 2024-01-24T23:27:04.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8898193640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:04.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8898193640 10 monclient(hunting): authenticate will time out at 1311.512542s 2024-01-24T23:27:04.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8895f2f640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f88900a3730 legacy=0x7f889009a1b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:27:04.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f889572e640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f8890137cf0 0x7f889009a8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:58324/0 (socket says 172.21.15.132:58324) 2024-01-24T23:27:04.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f889572e640 1 -- 172.21.15.132:0/2372978499 learned_addr learned my addr 172.21.15.132:0/2372978499 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:04.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 1 -- 172.21.15.132:0/2372978499 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2830362590 0 0) 0x7f889012f860 con 0x7f8890137cf0 2024-01-24T23:27:04.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:04.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:04.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:04.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 10 monclient(hunting): my global_id is 6483 2024-01-24T23:27:04.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 1 -- 172.21.15.132:0/2372978499 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8874003530 con 0x7f8890137cf0 2024-01-24T23:27:04.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 1 -- 172.21.15.132:0/2372978499 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3732576245 0 0) 0x7f8874003530 con 0x7f8890137cf0 2024-01-24T23:27:04.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 1 -- 172.21.15.132:0/2372978499 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f889012f860 con 0x7f8890137cf0 2024-01-24T23:27:04.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 1 -- 172.21.15.132:0/2372978499 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f888c004eb0 con 0x7f8890137cf0 2024-01-24T23:27:04.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:04.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:27:04.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 10 monclient(hunting): dump: 2024-01-24T23:27:04.200 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:04.200 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:04.200 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:04.200 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:04.200 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:04.200 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:04.200 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:04.200 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:04.200 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:04.200 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:04.200 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:04.200 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:04.201 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:04.201 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:04.201 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:04.201 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:04.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:04.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 1 -- 172.21.15.132:0/2372978499 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3764672573 0 0) 0x7f889012f860 con 0x7f8890137cf0 2024-01-24T23:27:04.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 1 -- 172.21.15.132:0/2372978499 >> v1:172.21.15.115:6792/0 conn(0x7f889013b800 legacy=0x7f88901d19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:04.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 1 -- 172.21.15.132:0/2372978499 >> v1:172.21.15.132:6789/0 conn(0x7f88900a3730 legacy=0x7f889009a1b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:04.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 10 monclient: _finish_hunting 0 2024-01-24T23:27:04.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 1 monclient: found mon.g 2024-01-24T23:27:04.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:04.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:04.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 1 -- 172.21.15.132:0/2372978499 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889009b440 con 0x7f8890137cf0 2024-01-24T23:27:04.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 10 monclient: _finish_auth 0 2024-01-24T23:27:04.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 10 monclient: _check_auth_tickets 2024-01-24T23:27:04.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:04.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8898193640 5 monclient: authenticate success, global_id 6483 2024-01-24T23:27:04.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8898193640 10 monclient: _renew_subs 2024-01-24T23:27:04.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8898193640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:04.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8898193640 1 -- 172.21.15.132:0/2372978499 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f889009b800 con 0x7f8890137cf0 2024-01-24T23:27:04.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8898193640 10 monclient: _renew_subs 2024-01-24T23:27:04.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8898193640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:04.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8898193640 1 -- 172.21.15.132:0/2372978499 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f889009bc80 con 0x7f8890137cf0 2024-01-24T23:27:04.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 1 -- 172.21.15.132:0/2372978499 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f888c005230 con 0x7f8890137cf0 2024-01-24T23:27:04.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:04.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 1 -- 172.21.15.132:0/2372978499 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f888c0056e0 con 0x7f8890137cf0 2024-01-24T23:27:04.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:04.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8883fff640 4 set_mon_vals no callback set 2024-01-24T23:27:04.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:04.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 10 monclient: dump: 2024-01-24T23:27:04.203 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:04.203 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:04.203 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:04.203 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:04.203 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:04.203 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:04.203 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:04.203 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:04.203 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:04.203 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:04.203 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:04.203 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:04.203 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:04.203 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:04.203 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:04.204 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:04.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8898193640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:04.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8898193640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:04.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8898193640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:04.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8898193640 1 -- 172.21.15.132:0/2372978499 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8878005090 con 0x7f8890137cf0 2024-01-24T23:27:04.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 1 -- 172.21.15.132:0/2372978499 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f888c010160 con 0x7f8890137cf0 2024-01-24T23:27:04.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f88827fc640 1 -- 172.21.15.132:0/2372978499 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(204..204 src has 1..204) v4 ==== 12674+0+0 (unknown 720066964 0 0) 0x7f888c046170 con 0x7f8890137cf0 2024-01-24T23:27:04.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.186+0000 7f8895f2f640 10 monclient: get_auth_request con 0x7f8874036f60 auth_method 0 2024-01-24T23:27:04.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.190+0000 7f88827fc640 1 -- 172.21.15.132:0/2372978499 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f888c014ee0 con 0x7f8890137cf0 2024-01-24T23:27:04.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.190+0000 7f88827fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:04.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.190+0000 7f88827fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:04.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.350+0000 7f8898193640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:04.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.350+0000 7f8898193640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:04.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.350+0000 7f8898193640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:04.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.350+0000 7f8898193640 1 -- 172.21.15.132:0/2372978499 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f88780058c0 con 0x7f8890137cf0 2024-01-24T23:27:04.963 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.962+0000 7f88827fc640 1 -- 172.21.15.132:0/2372978499 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}]=0 added bucket h25 type host to location {rack=r1} v205) v1 ==== 168+0+0 (unknown 742531732 0 0) 0x7f888c013580 con 0x7f8890137cf0 2024-01-24T23:27:04.963 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.962+0000 7f88827fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:04.963 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:04.962+0000 7f88827fc640 10 monclient: _finish_command 2 = system:0 added bucket h25 type host to location {rack=r1} 2024-01-24T23:27:05.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.058+0000 7f8898193640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:05.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.058+0000 7f8898193640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:05.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.058+0000 7f8898193640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:05.060 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.058+0000 7f8898193640 1 -- 172.21.15.132:0/2372978499 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f8878002220 con 0x7f8890137cf0 2024-01-24T23:27:05.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.058+0000 7f88827fc640 1 -- 172.21.15.132:0/2372978499 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h25' already exists v205) v1 ==== 147+0+0 (unknown 4151786353 0 0) 0x7f888c014d00 con 0x7f8890137cf0 2024-01-24T23:27:05.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.058+0000 7f88827fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:05.061 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.058+0000 7f88827fc640 10 monclient: _finish_command 3 = system:0 bucket 'h25' already exists 2024-01-24T23:27:05.061 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h25' already exists 2024-01-24T23:27:05.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.062+0000 7f8898193640 1 -- 172.21.15.132:0/2372978499 >> v1:172.21.15.132:6812/1835401922 conn(0x7f8874036f60 legacy=0x7f8874039420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:05.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.062+0000 7f8898193640 10 monclient: shutdown 2024-01-24T23:27:05.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.062+0000 7f8898193640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:05.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.062+0000 7f8898193640 1 -- 172.21.15.132:0/2372978499 >> v1:172.21.15.132:6790/0 conn(0x7f8890137cf0 legacy=0x7f889009a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:05.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.062+0000 7f8898193640 1 -- 172.21.15.132:0/2372978499 shutdown_connections 2024-01-24T23:27:05.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.062+0000 7f8898193640 1 -- 172.21.15.132:0/2372978499 >> 172.21.15.132:0/2372978499 conn(0x7f88900a2c40 msgr2=0x7f88900a4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:05.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.062+0000 7f8898193640 1 -- 172.21.15.132:0/2372978499 shutdown_connections 2024-01-24T23:27:05.063 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.062+0000 7f8898193640 1 -- 172.21.15.132:0/2372978499 wait complete. 2024-01-24T23:27:05.078 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:05.078 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h26 host rack=r1 2024-01-24T23:27:05.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 10 monclient: get_monmap_and_config 2024-01-24T23:27:05.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 10 monclient: build_initial_monmap 2024-01-24T23:27:05.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 10 monclient: monmap: 2024-01-24T23:27:05.200 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:05.200 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:05.200 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:05.201615+0000 2024-01-24T23:27:05.200 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:05.201615+0000 2024-01-24T23:27:05.200 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:05.200 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:05.201 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:05.201 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:05.201 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:05.201 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:05.201 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:05.201 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:05.201 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:05.201 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:05.201 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:05.201 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:05.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 1 Processor -- start 2024-01-24T23:27:05.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 1 -- start start 2024-01-24T23:27:05.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 10 monclient: init 2024-01-24T23:27:05.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:05.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 10 monclient: _add_conns ranks=[1,3,0,8,4,5,6,2,7] 2024-01-24T23:27:05.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 10 monclient(hunting): picked mon.noname-f con 0x7fbab409b810 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:05.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 10 monclient(hunting): picked mon.noname-g con 0x7fbab409bbf0 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:05.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 10 monclient(hunting): picked mon.noname-a con 0x7fbab41370a0 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:05.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab4064700 con 0x7fbab41370a0 2024-01-24T23:27:05.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab4099af0 con 0x7fbab409b810 2024-01-24T23:27:05.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab4072110 con 0x7fbab409bbf0 2024-01-24T23:27:05.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:05.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 10 monclient(hunting): authenticate will time out at 1312.526186s 2024-01-24T23:27:05.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab3fff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fbab409b810 legacy=0x7fbab40712f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:27:05.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaab7fe640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fbab409bbf0 0x7fbab4071a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:58340/0 (socket says 172.21.15.132:58340) 2024-01-24T23:27:05.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaab7fe640 1 -- 172.21.15.132:0/1116847365 learned_addr learned my addr 172.21.15.132:0/1116847365 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:05.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 1 -- 172.21.15.132:0/1116847365 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 847777300 0 0) 0x7fbab4072110 con 0x7fbab409bbf0 2024-01-24T23:27:05.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:05.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:05.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:05.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 10 monclient(hunting): my global_id is 6492 2024-01-24T23:27:05.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 1 -- 172.21.15.132:0/1116847365 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba94003530 con 0x7fbab409bbf0 2024-01-24T23:27:05.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 1 -- 172.21.15.132:0/1116847365 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2928905860 0 0) 0x7fba94003530 con 0x7fbab409bbf0 2024-01-24T23:27:05.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 1 -- 172.21.15.132:0/1116847365 >> v1:172.21.15.132:6789/0 conn(0x7fbab409b810 legacy=0x7fbab40712f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:05.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 1 -- 172.21.15.132:0/1116847365 >> v1:172.21.15.115:6789/0 conn(0x7fbab41370a0 legacy=0x7fbab413ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:05.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 10 monclient: _finish_hunting 0 2024-01-24T23:27:05.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 1 monclient: found mon.noname-g 2024-01-24T23:27:05.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:05.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:27:05.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 1 -- 172.21.15.132:0/1116847365 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab409c050 con 0x7fbab409bbf0 2024-01-24T23:27:05.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 10 monclient: _finish_auth 0 2024-01-24T23:27:05.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:05.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:05.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 1 -- 172.21.15.132:0/1116847365 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fba9c0017c0 con 0x7fbab409bbf0 2024-01-24T23:27:05.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:05.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:27:05.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 10 monclient: dump: 2024-01-24T23:27:05.209 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:05.209 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:05.209 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:05.209 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:05.209 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:05.209 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:05.209 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:05.209 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:05.209 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:05.209 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:05.210 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:05.210 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:05.210 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:05.210 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:05.210 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:05.210 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:05.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 5 monclient: authenticate success, global_id 6492 2024-01-24T23:27:05.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:05.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 1 -- 172.21.15.132:0/1116847365 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fba9c001a70 con 0x7fbab409bbf0 2024-01-24T23:27:05.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:05.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 1 -- 172.21.15.132:0/1116847365 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fba9c004bf0 con 0x7fbab409bbf0 2024-01-24T23:27:05.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:05.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 4 set_mon_vals no callback set 2024-01-24T23:27:05.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:05.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:05.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbab37fe640 10 monclient: dump: 2024-01-24T23:27:05.211 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:05.211 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:05.211 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:05.211 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:05.211 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:05.211 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:05.211 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:05.211 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:05.211 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:05.211 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:05.211 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:05.211 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:05.211 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:05.211 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:05.211 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:05.211 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:05.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 10 monclient: shutdown 2024-01-24T23:27:05.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:05.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 1 -- 172.21.15.132:0/1116847365 >> v1:172.21.15.132:6790/0 conn(0x7fbab409bbf0 legacy=0x7fbab4071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:05.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 1 -- 172.21.15.132:0/1116847365 shutdown_connections 2024-01-24T23:27:05.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 1 -- 172.21.15.132:0/1116847365 >> 172.21.15.132:0/1116847365 conn(0x7fbab40a1bc0 msgr2=0x7fbab40a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:05.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 1 -- 172.21.15.132:0/1116847365 shutdown_connections 2024-01-24T23:27:05.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.198+0000 7fbaba315640 1 -- 172.21.15.132:0/1116847365 wait complete. 2024-01-24T23:27:05.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 10 monclient: build_initial_monmap 2024-01-24T23:27:05.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 10 monclient: monmap: 2024-01-24T23:27:05.212 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:05.212 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:05.212 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:05.212 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:05.212 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:05.212 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:05.213 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:05.213 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:05.213 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:05.213 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:05.213 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:05.213 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:05.213 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:05.213 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:05.213 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:05.213 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:05.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 1 Processor -- start 2024-01-24T23:27:05.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 1 -- start start 2024-01-24T23:27:05.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 10 monclient: init 2024-01-24T23:27:05.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:05.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 10 monclient: _add_conns ranks=[2,7,4,1,6,0,3,5,8] 2024-01-24T23:27:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 10 monclient(hunting): picked mon.noname-b con 0x7fbab409b810 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 10 monclient(hunting): picked mon.noname-i con 0x7fbab409bbf0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 10 monclient(hunting): picked mon.noname-c con 0x7fbab41370a0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab412a280 con 0x7fbab409b810 2024-01-24T23:27:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab412a950 con 0x7fbab41370a0 2024-01-24T23:27:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaab7fe640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fbab409bbf0 legacy=0x7fbab41d0670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab412ac30 con 0x7fbab409bbf0 2024-01-24T23:27:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 10 monclient(hunting): authenticate will time out at 1312.529320s 2024-01-24T23:27:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab3fff640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fbab409b810 legacy=0x7fbab4130760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab88b2640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fbab41370a0 0x7fbab41d3da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:33722/0 (socket says 172.21.15.132:33722) 2024-01-24T23:27:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab88b2640 1 -- 172.21.15.132:0/3258689729 learned_addr learned my addr 172.21.15.132:0/3258689729 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 1 -- 172.21.15.132:0/3258689729 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2544034840 0 0) 0x7fbab412a950 con 0x7fbab41370a0 2024-01-24T23:27:05.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 10 monclient(hunting): _negotiate 2024-01-24T23:27:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 10 monclient(hunting): my global_id is 94639 2024-01-24T23:27:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 1 -- 172.21.15.132:0/3258689729 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba98003530 con 0x7fbab41370a0 2024-01-24T23:27:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 1 -- 172.21.15.132:0/3258689729 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4053697663 0 0) 0x7fba98003530 con 0x7fbab41370a0 2024-01-24T23:27:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 1 -- 172.21.15.132:0/3258689729 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbab412a950 con 0x7fbab41370a0 2024-01-24T23:27:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 1 -- 172.21.15.132:0/3258689729 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbaac001de0 con 0x7fbab41370a0 2024-01-24T23:27:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:27:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 10 monclient(hunting): dump: 2024-01-24T23:27:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:05.215 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:05.216 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:05.216 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:05.216 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:05.216 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:05.216 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:05.216 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:05.216 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:05.216 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:05.216 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:05.216 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:05.216 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:05.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:05.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 1 -- 172.21.15.132:0/3258689729 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3830599840 0 0) 0x7fbab412a950 con 0x7fbab41370a0 2024-01-24T23:27:05.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 1 -- 172.21.15.132:0/3258689729 >> v1:172.21.15.132:6792/0 conn(0x7fbab409bbf0 legacy=0x7fbab41d0670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:05.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 1 -- 172.21.15.132:0/3258689729 >> v1:172.21.15.115:6790/0 conn(0x7fbab409b810 legacy=0x7fbab4130760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 10 monclient: _finish_hunting 0 2024-01-24T23:27:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 1 monclient: found mon.c 2024-01-24T23:27:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 1 -- 172.21.15.132:0/3258689729 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab412af10 con 0x7fbab41370a0 2024-01-24T23:27:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 10 monclient: _finish_auth 0 2024-01-24T23:27:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 10 monclient: _check_auth_tickets 2024-01-24T23:27:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 5 monclient: authenticate success, global_id 94639 2024-01-24T23:27:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 10 monclient: _renew_subs 2024-01-24T23:27:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 1 -- 172.21.15.132:0/3258689729 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab4134250 con 0x7fbab41370a0 2024-01-24T23:27:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 10 monclient: _renew_subs 2024-01-24T23:27:05.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 1 -- 172.21.15.132:0/3258689729 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab4130e70 con 0x7fbab41370a0 2024-01-24T23:27:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 1 -- 172.21.15.132:0/3258689729 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbaac003c50 con 0x7fbab41370a0 2024-01-24T23:27:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab2ffd640 4 set_mon_vals no callback set 2024-01-24T23:27:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 1 -- 172.21.15.132:0/3258689729 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbaac004180 con 0x7fbab41370a0 2024-01-24T23:27:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbab17fa640 10 monclient: dump: 2024-01-24T23:27:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:05.218 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:05.219 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.202+0000 7fbaba315640 1 -- 172.21.15.132:0/3258689729 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba8c005090 con 0x7fbab41370a0 2024-01-24T23:27:05.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.206+0000 7fbab17fa640 1 -- 172.21.15.132:0/3258689729 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fbaac004360 con 0x7fbab41370a0 2024-01-24T23:27:05.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.206+0000 7fbab17fa640 1 -- 172.21.15.132:0/3258689729 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(205..205 src has 1..205) v4 ==== 12887+0+0 (unknown 1592522397 0 0) 0x7fbaac045ec0 con 0x7fbab41370a0 2024-01-24T23:27:05.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.206+0000 7fbab3fff640 10 monclient: get_auth_request con 0x7fba98036f10 auth_method 0 2024-01-24T23:27:05.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.218+0000 7fbab17fa640 1 -- 172.21.15.132:0/3258689729 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fbaac014b80 con 0x7fbab41370a0 2024-01-24T23:27:05.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.218+0000 7fbab17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:05.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.218+0000 7fbab17fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:05.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.378+0000 7fbaba315640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:05.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.378+0000 7fbaba315640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:05.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.378+0000 7fbaba315640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:05.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.378+0000 7fbaba315640 1 -- 172.21.15.132:0/3258689729 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fba8c005340 con 0x7fbab41370a0 2024-01-24T23:27:05.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.978+0000 7fbab17fa640 1 -- 172.21.15.132:0/3258689729 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}]=0 added bucket h26 type host to location {rack=r1} v206) v1 ==== 168+0+0 (unknown 2826284179 0 0) 0x7fbaac0149a0 con 0x7fbab41370a0 2024-01-24T23:27:05.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.978+0000 7fbab17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:05.981 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:05.978+0000 7fbab17fa640 10 monclient: _finish_command 2 = system:0 added bucket h26 type host to location {rack=r1} 2024-01-24T23:27:06.079 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.078+0000 7fbaba315640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:06.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.078+0000 7fbaba315640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:06.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.078+0000 7fbaba315640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:06.080 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.078+0000 7fbaba315640 1 -- 172.21.15.132:0/3258689729 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fba8c005a80 con 0x7fbab41370a0 2024-01-24T23:27:06.082 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.078+0000 7fbab17fa640 1 -- 172.21.15.132:0/3258689729 <== mon.4 v1:172.21.15.115:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h26' already exists v206) v1 ==== 147+0+0 (unknown 3953987910 0 0) 0x7fbaac0156e0 con 0x7fbab41370a0 2024-01-24T23:27:06.082 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.078+0000 7fbab17fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:06.082 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.078+0000 7fbab17fa640 10 monclient: _finish_command 3 = system:0 bucket 'h26' already exists 2024-01-24T23:27:06.082 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h26' already exists 2024-01-24T23:27:06.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.082+0000 7fbaba315640 1 -- 172.21.15.132:0/3258689729 >> v1:172.21.15.132:6812/1835401922 conn(0x7fba98036f10 legacy=0x7fba980393d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:06.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.082+0000 7fbaba315640 10 monclient: shutdown 2024-01-24T23:27:06.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.082+0000 7fbaba315640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:06.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.082+0000 7fbaba315640 1 -- 172.21.15.132:0/3258689729 >> v1:172.21.15.115:6791/0 conn(0x7fbab41370a0 legacy=0x7fbab41d3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:06.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.082+0000 7fbaba315640 1 -- 172.21.15.132:0/3258689729 shutdown_connections 2024-01-24T23:27:06.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.082+0000 7fbaba315640 1 -- 172.21.15.132:0/3258689729 >> 172.21.15.132:0/3258689729 conn(0x7fbab40a1bc0 msgr2=0x7fbab409c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:06.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.082+0000 7fbaba315640 1 -- 172.21.15.132:0/3258689729 shutdown_connections 2024-01-24T23:27:06.084 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.082+0000 7fbaba315640 1 -- 172.21.15.132:0/3258689729 wait complete. 2024-01-24T23:27:06.099 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:06.099 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h27 host rack=r1 2024-01-24T23:27:06.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.218+0000 7f89b0039640 10 monclient: get_monmap_and_config 2024-01-24T23:27:06.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.218+0000 7f89b0039640 10 monclient: build_initial_monmap 2024-01-24T23:27:06.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.218+0000 7f89b0039640 10 monclient: monmap: 2024-01-24T23:27:06.221 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:06.221 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:06.221 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:06.222437+0000 2024-01-24T23:27:06.221 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:06.222437+0000 2024-01-24T23:27:06.221 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:06.221 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:06.221 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:06.222 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:06.222 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:06.222 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:06.222 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:06.222 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:06.222 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:06.222 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:06.222 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:06.222 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:06.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.218+0000 7f89b0039640 1 Processor -- start 2024-01-24T23:27:06.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.218+0000 7f89b0039640 1 -- start start 2024-01-24T23:27:06.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.218+0000 7f89b0039640 10 monclient: init 2024-01-24T23:27:06.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.218+0000 7f89b0039640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:06.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.218+0000 7f89b0039640 10 monclient: _add_conns ranks=[7,4,0,3,1,5,2,8,6] 2024-01-24T23:27:06.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.218+0000 7f89b0039640 10 monclient(hunting): picked mon.noname-i con 0x7f89a8072690 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:06.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.218+0000 7f89b0039640 10 monclient(hunting): picked mon.noname-c con 0x7f89a809ae90 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:06.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.218+0000 7f89b0039640 10 monclient(hunting): picked mon.noname-a con 0x7f89a809bdc0 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:06.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.218+0000 7f89b0039640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a8064700 con 0x7f89a809bdc0 2024-01-24T23:27:06.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.218+0000 7f89b0039640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a809a790 con 0x7f89a809ae90 2024-01-24T23:27:06.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.218+0000 7f89b0039640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a80a18f0 con 0x7f89a8072690 2024-01-24T23:27:06.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.218+0000 7f89addd5640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f89a8072690 legacy=0x7f89a80a0ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:06.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.218+0000 7f89b0039640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:06.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.218+0000 7f89b0039640 10 monclient(hunting): authenticate will time out at 1313.547154s 2024-01-24T23:27:06.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.218+0000 7f89ad5d4640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f89a809ae90 0x7f89a80a11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:33732/0 (socket says 172.21.15.132:33732) 2024-01-24T23:27:06.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.218+0000 7f89ae5d6640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f89a809bdc0 0x7f89a813d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:59450/0 (socket says 172.21.15.132:59450) 2024-01-24T23:27:06.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.218+0000 7f89ad5d4640 1 -- 172.21.15.132:0/1735200099 learned_addr learned my addr 172.21.15.132:0/1735200099 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:06.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 1 -- 172.21.15.132:0/1735200099 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 552710291 0 0) 0x7f89a8064700 con 0x7f89a809bdc0 2024-01-24T23:27:06.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 10 monclient(hunting): _negotiate 2024-01-24T23:27:06.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:06.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:06.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 10 monclient(hunting): my global_id is 94833 2024-01-24T23:27:06.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 1 -- 172.21.15.132:0/1735200099 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8994003530 con 0x7f89a809bdc0 2024-01-24T23:27:06.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 1 -- 172.21.15.132:0/1735200099 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1921362600 0 0) 0x7f89a809a790 con 0x7f89a809ae90 2024-01-24T23:27:06.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 10 monclient(hunting): _negotiate 2024-01-24T23:27:06.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:06.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:06.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 10 monclient(hunting): my global_id is 94648 2024-01-24T23:27:06.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 1 -- 172.21.15.132:0/1735200099 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89a8064700 con 0x7f89a809ae90 2024-01-24T23:27:06.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 1 -- 172.21.15.132:0/1735200099 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2979781529 0 0) 0x7f8994003530 con 0x7f89a809bdc0 2024-01-24T23:27:06.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 1 -- 172.21.15.132:0/1735200099 >> v1:172.21.15.132:6792/0 conn(0x7f89a8072690 legacy=0x7f89a80a0ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:06.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 1 -- 172.21.15.132:0/1735200099 >> v1:172.21.15.115:6791/0 conn(0x7f89a809ae90 legacy=0x7f89a80a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:06.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 10 monclient: _finish_hunting 0 2024-01-24T23:27:06.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 1 monclient: found mon.noname-a 2024-01-24T23:27:06.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:06.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:27:06.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 1 -- 172.21.15.132:0/1735200099 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a80997a0 con 0x7f89a809bdc0 2024-01-24T23:27:06.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 10 monclient: _finish_auth 0 2024-01-24T23:27:06.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 10 monclient: _check_auth_tickets 2024-01-24T23:27:06.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:06.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 1 -- 172.21.15.132:0/1735200099 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f89a0001a70 con 0x7f89a809bdc0 2024-01-24T23:27:06.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:06.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:27:06.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 10 monclient: dump: 2024-01-24T23:27:06.232 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:06.232 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:06.232 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:06.232 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:06.232 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:06.232 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:06.232 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:06.232 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:06.232 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:06.232 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:06.232 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:06.232 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:06.233 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:06.233 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:06.233 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:06.233 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:06.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 5 monclient: authenticate success, global_id 94833 2024-01-24T23:27:06.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:06.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 1 -- 172.21.15.132:0/1735200099 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f89a0001c90 con 0x7f89a809bdc0 2024-01-24T23:27:06.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:06.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 1 -- 172.21.15.132:0/1735200099 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f89a0004b70 con 0x7f89a809bdc0 2024-01-24T23:27:06.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:06.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:27:06.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89acdd3640 10 monclient: dump: 2024-01-24T23:27:06.233 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:06.233 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:06.233 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:06.234 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:06.234 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:06.234 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:06.234 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:06.234 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:06.234 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:06.234 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:06.234 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:06.234 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:06.234 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:06.234 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:06.234 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:06.234 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:06.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:06.234 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 4 set_mon_vals no callback set 2024-01-24T23:27:06.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 10 monclient: shutdown 2024-01-24T23:27:06.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:06.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 1 -- 172.21.15.132:0/1735200099 >> v1:172.21.15.115:6789/0 conn(0x7f89a809bdc0 legacy=0x7f89a813d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:06.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 1 -- 172.21.15.132:0/1735200099 shutdown_connections 2024-01-24T23:27:06.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 1 -- 172.21.15.132:0/1735200099 >> 172.21.15.132:0/1735200099 conn(0x7f89a80a1bc0 msgr2=0x7f89a80a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:06.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 1 -- 172.21.15.132:0/1735200099 shutdown_connections 2024-01-24T23:27:06.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 1 -- 172.21.15.132:0/1735200099 wait complete. 2024-01-24T23:27:06.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 10 monclient: build_initial_monmap 2024-01-24T23:27:06.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 10 monclient: monmap: 2024-01-24T23:27:06.235 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:06.235 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:06.235 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:06.236 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:06.236 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:06.236 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:06.236 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:06.236 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:06.236 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:06.236 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:06.236 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:06.236 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:06.236 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:06.236 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:06.236 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:06.236 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:06.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 1 Processor -- start 2024-01-24T23:27:06.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 1 -- start start 2024-01-24T23:27:06.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 10 monclient: init 2024-01-24T23:27:06.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:06.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 10 monclient: _add_conns ranks=[0,1,3,6,2,7,4,8,5] 2024-01-24T23:27:06.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 10 monclient(hunting): picked mon.noname-a con 0x7f89a8072690 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:06.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 10 monclient(hunting): picked mon.noname-f con 0x7f89a809ae90 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:06.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 10 monclient(hunting): picked mon.noname-g con 0x7f89a809bdc0 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:06.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a81b4790 con 0x7f89a8072690 2024-01-24T23:27:06.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a8099470 con 0x7f89a809ae90 2024-01-24T23:27:06.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89ad5d4640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f89a809ae90 legacy=0x7f89a81d06b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:27:06.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a809b930 con 0x7f89a809bdc0 2024-01-24T23:27:06.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:06.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89b0039640 10 monclient(hunting): authenticate will time out at 1313.551141s 2024-01-24T23:27:06.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89ae5d6640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f89a809bdc0 0x7f89a81d3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:58352/0 (socket says 172.21.15.132:58352) 2024-01-24T23:27:06.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f89ae5d6640 1 -- 172.21.15.132:0/1167519693 learned_addr learned my addr 172.21.15.132:0/1167519693 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:06.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f899e7fc640 1 -- 172.21.15.132:0/1167519693 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1393112314 0 0) 0x7f89a809b930 con 0x7f89a809bdc0 2024-01-24T23:27:06.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f899e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:06.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f899e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:06.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f899e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:06.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f899e7fc640 10 monclient(hunting): my global_id is 6501 2024-01-24T23:27:06.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f899e7fc640 1 -- 172.21.15.132:0/1167519693 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f898c003530 con 0x7f89a809bdc0 2024-01-24T23:27:06.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f899e7fc640 1 -- 172.21.15.132:0/1167519693 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3700664234 0 0) 0x7f898c003530 con 0x7f89a809bdc0 2024-01-24T23:27:06.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f899e7fc640 1 -- 172.21.15.132:0/1167519693 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f89a809b930 con 0x7f89a809bdc0 2024-01-24T23:27:06.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f899e7fc640 1 -- 172.21.15.132:0/1167519693 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f89a0004c30 con 0x7f89a809bdc0 2024-01-24T23:27:06.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f899e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:06.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f899e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:27:06.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f899e7fc640 10 monclient(hunting): dump: 2024-01-24T23:27:06.238 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:06.238 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:06.238 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:06.238 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:06.239 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:06.239 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:06.239 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:06.239 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:06.239 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:06.239 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:06.239 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:06.239 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:06.239 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:06.239 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:06.239 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:06.239 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:06.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f899e7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:06.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f899e7fc640 1 -- 172.21.15.132:0/1167519693 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596518162 0 0) 0x7f89a81b4790 con 0x7f89a8072690 2024-01-24T23:27:06.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f899e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:06.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f899e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:06.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f899e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:06.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f899e7fc640 10 monclient(hunting): my global_id is 94842 2024-01-24T23:27:06.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.222+0000 7f899e7fc640 1 -- 172.21.15.132:0/1167519693 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f898c003530 con 0x7f89a8072690 2024-01-24T23:27:06.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f899e7fc640 1 -- 172.21.15.132:0/1167519693 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 146674722 0 0) 0x7f89a809b930 con 0x7f89a809bdc0 2024-01-24T23:27:06.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f899e7fc640 1 -- 172.21.15.132:0/1167519693 >> v1:172.21.15.132:6789/0 conn(0x7f89a809ae90 legacy=0x7f89a81d06b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:06.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f899e7fc640 1 -- 172.21.15.132:0/1167519693 >> v1:172.21.15.115:6789/0 conn(0x7f89a8072690 legacy=0x7f89a81cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:06.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f899e7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:27:06.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f899e7fc640 1 monclient: found mon.g 2024-01-24T23:27:06.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f899e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:06.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f899e7fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:06.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f899e7fc640 1 -- 172.21.15.132:0/1167519693 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a81b44b0 con 0x7f89a809bdc0 2024-01-24T23:27:06.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f899e7fc640 10 monclient: _finish_auth 0 2024-01-24T23:27:06.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f899e7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:27:06.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f899e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:06.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f89b0039640 5 monclient: authenticate success, global_id 6501 2024-01-24T23:27:06.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f89b0039640 10 monclient: _renew_subs 2024-01-24T23:27:06.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f89b0039640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:06.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f89b0039640 1 -- 172.21.15.132:0/1167519693 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a81b2b10 con 0x7f89a809bdc0 2024-01-24T23:27:06.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f89b0039640 10 monclient: _renew_subs 2024-01-24T23:27:06.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f89b0039640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:06.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f89b0039640 1 -- 172.21.15.132:0/1167519693 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a81b52d0 con 0x7f89a809bdc0 2024-01-24T23:27:06.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f899e7fc640 1 -- 172.21.15.132:0/1167519693 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f89a00050e0 con 0x7f89a809bdc0 2024-01-24T23:27:06.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f899e7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:06.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f899e7fc640 1 -- 172.21.15.132:0/1167519693 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f89a0005590 con 0x7f89a809bdc0 2024-01-24T23:27:06.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f899e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:06.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f899ffff640 4 set_mon_vals no callback set 2024-01-24T23:27:06.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f899e7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:06.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f899e7fc640 10 monclient: dump: 2024-01-24T23:27:06.241 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:06.242 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:06.242 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:06.242 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:06.242 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:06.242 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:06.242 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:06.242 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:06.242 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:06.242 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:06.242 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:06.242 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:06.242 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:06.242 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:06.242 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:06.243 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:06.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f89b0039640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:06.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f89b0039640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:06.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f89b0039640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:06.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f89b0039640 1 -- 172.21.15.132:0/1167519693 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8990005090 con 0x7f89a809bdc0 2024-01-24T23:27:06.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f899e7fc640 1 -- 172.21.15.132:0/1167519693 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f89a0010010 con 0x7f89a809bdc0 2024-01-24T23:27:06.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f899e7fc640 1 -- 172.21.15.132:0/1167519693 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(206..206 src has 1..206) v4 ==== 13100+0+0 (unknown 1289259630 0 0) 0x7f89a0046200 con 0x7f89a809bdc0 2024-01-24T23:27:06.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.226+0000 7f89addd5640 10 monclient: get_auth_request con 0x7f898c036f20 auth_method 0 2024-01-24T23:27:06.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.230+0000 7f899e7fc640 1 -- 172.21.15.132:0/1167519693 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f89a0011510 con 0x7f89a809bdc0 2024-01-24T23:27:06.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.230+0000 7f899e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:06.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.230+0000 7f899e7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:06.344 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~62s 2024-01-24T23:27:06.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.390+0000 7f89b0039640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:06.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.390+0000 7f89b0039640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:06.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.390+0000 7f89b0039640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:06.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.390+0000 7f89b0039640 1 -- 172.21.15.132:0/1167519693 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f8990005340 con 0x7f89a809bdc0 2024-01-24T23:27:06.926 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.922+0000 7f899e7fc640 1 -- 172.21.15.132:0/1167519693 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}]=0 added bucket h27 type host to location {rack=r1} v207) v1 ==== 168+0+0 (unknown 3563530606 0 0) 0x7f89a0013550 con 0x7f89a809bdc0 2024-01-24T23:27:06.926 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.922+0000 7f899e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:06.926 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:06.922+0000 7f899e7fc640 10 monclient: _finish_command 2 = system:0 added bucket h27 type host to location {rack=r1} 2024-01-24T23:27:07.027 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.026+0000 7f89b0039640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:07.027 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.026+0000 7f89b0039640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:07.027 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.026+0000 7f89b0039640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:07.027 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.026+0000 7f89b0039640 1 -- 172.21.15.132:0/1167519693 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f8990005a80 con 0x7f89a809bdc0 2024-01-24T23:27:07.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.026+0000 7f899e7fc640 1 -- 172.21.15.132:0/1167519693 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h27' already exists v207) v1 ==== 147+0+0 (unknown 3759868075 0 0) 0x7f89a0014cd0 con 0x7f89a809bdc0 2024-01-24T23:27:07.028 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.026+0000 7f899e7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:07.029 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.026+0000 7f899e7fc640 10 monclient: _finish_command 3 = system:0 bucket 'h27' already exists 2024-01-24T23:27:07.029 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h27' already exists 2024-01-24T23:27:07.030 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.026+0000 7f89b0039640 1 -- 172.21.15.132:0/1167519693 >> v1:172.21.15.132:6812/1835401922 conn(0x7f898c036f20 legacy=0x7f898c0393e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:07.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.026+0000 7f89b0039640 10 monclient: shutdown 2024-01-24T23:27:07.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.026+0000 7f89b0039640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:07.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.026+0000 7f89b0039640 1 -- 172.21.15.132:0/1167519693 >> v1:172.21.15.132:6790/0 conn(0x7f89a809bdc0 legacy=0x7f89a81d3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:07.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.030+0000 7f89b0039640 1 -- 172.21.15.132:0/1167519693 shutdown_connections 2024-01-24T23:27:07.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.030+0000 7f89b0039640 1 -- 172.21.15.132:0/1167519693 >> 172.21.15.132:0/1167519693 conn(0x7f89a80a1bc0 msgr2=0x7f89a81bc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:07.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.030+0000 7f89b0039640 1 -- 172.21.15.132:0/1167519693 shutdown_connections 2024-01-24T23:27:07.031 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.030+0000 7f89b0039640 1 -- 172.21.15.132:0/1167519693 wait complete. 2024-01-24T23:27:07.046 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:07.046 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h28 host rack=r1 2024-01-24T23:27:07.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 10 monclient: get_monmap_and_config 2024-01-24T23:27:07.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 10 monclient: build_initial_monmap 2024-01-24T23:27:07.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 10 monclient: monmap: 2024-01-24T23:27:07.163 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:07.163 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:07.163 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:07.165052+0000 2024-01-24T23:27:07.163 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:07.165052+0000 2024-01-24T23:27:07.163 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:07.163 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:07.163 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:07.164 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:07.164 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:07.164 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:07.164 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:07.164 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:07.164 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:07.164 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:07.164 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:07.164 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:07.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 1 Processor -- start 2024-01-24T23:27:07.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 1 -- start start 2024-01-24T23:27:07.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 10 monclient: init 2024-01-24T23:27:07.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:07.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 10 monclient: _add_conns ranks=[5,3,4,7,6,1,8,0,2] 2024-01-24T23:27:07.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 10 monclient(hunting): picked mon.noname-h con 0x7f4308135ea0 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:07.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 10 monclient(hunting): picked mon.noname-g con 0x7f430809bcd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:07.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 10 monclient(hunting): picked mon.noname-c con 0x7f430809c0f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:07.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4308064700 con 0x7f430809bcd0 2024-01-24T23:27:07.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4308136990 con 0x7f430809c0f0 2024-01-24T23:27:07.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f430809b010 con 0x7f4308135ea0 2024-01-24T23:27:07.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:07.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 10 monclient(hunting): authenticate will time out at 1314.489730s 2024-01-24T23:27:07.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430cb9b640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f430809bcd0 0x7f430813bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:58362/0 (socket says 172.21.15.132:58362) 2024-01-24T23:27:07.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430d39c640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f4308135ea0 0x7f4308136280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:59674/0 (socket says 172.21.15.132:59674) 2024-01-24T23:27:07.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430cb9b640 1 -- 172.21.15.132:0/184658239 learned_addr learned my addr 172.21.15.132:0/184658239 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:07.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 1 -- 172.21.15.132:0/184658239 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684780161 0 0) 0x7f4308064700 con 0x7f430809bcd0 2024-01-24T23:27:07.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:07.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:07.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:07.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 10 monclient(hunting): my global_id is 6510 2024-01-24T23:27:07.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 1 -- 172.21.15.132:0/184658239 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42f4003530 con 0x7f430809bcd0 2024-01-24T23:27:07.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 1 -- 172.21.15.132:0/184658239 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 815295509 0 0) 0x7f430809b010 con 0x7f4308135ea0 2024-01-24T23:27:07.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:07.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:07.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:07.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 10 monclient(hunting): my global_id is 6431 2024-01-24T23:27:07.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 1 -- 172.21.15.132:0/184658239 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4308064700 con 0x7f4308135ea0 2024-01-24T23:27:07.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 1 -- 172.21.15.132:0/184658239 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3099154662 0 0) 0x7f42f4003530 con 0x7f430809bcd0 2024-01-24T23:27:07.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 1 -- 172.21.15.132:0/184658239 >> v1:172.21.15.132:6791/0 conn(0x7f4308135ea0 legacy=0x7f4308136280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:07.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 1 -- 172.21.15.132:0/184658239 >> v1:172.21.15.115:6791/0 conn(0x7f430809c0f0 legacy=0x7f430813f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:07.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 10 monclient: _finish_hunting 0 2024-01-24T23:27:07.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 1 monclient: found mon.noname-g 2024-01-24T23:27:07.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:07.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:27:07.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 1 -- 172.21.15.132:0/184658239 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43080a4f50 con 0x7f430809bcd0 2024-01-24T23:27:07.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 10 monclient: _finish_auth 0 2024-01-24T23:27:07.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 10 monclient: _check_auth_tickets 2024-01-24T23:27:07.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:07.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 1 -- 172.21.15.132:0/184658239 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f42f80017c0 con 0x7f430809bcd0 2024-01-24T23:27:07.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:07.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:27:07.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 10 monclient: dump: 2024-01-24T23:27:07.173 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:07.173 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:07.173 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:07.173 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:07.173 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:07.173 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:07.173 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:07.173 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:07.173 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:07.173 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:07.173 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:07.173 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:07.173 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:07.174 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:07.174 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:07.174 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:07.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 5 monclient: authenticate success, global_id 6510 2024-01-24T23:27:07.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:07.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 1 -- 172.21.15.132:0/184658239 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f42f8001a60 con 0x7f430809bcd0 2024-01-24T23:27:07.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:07.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 1 -- 172.21.15.132:0/184658239 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f42f8004bf0 con 0x7f430809bcd0 2024-01-24T23:27:07.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:07.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:07.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f42fffff640 10 monclient: dump: 2024-01-24T23:27:07.174 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:07.174 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:07.174 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:07.174 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:07.174 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:07.175 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:07.175 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:07.175 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:07.175 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:07.175 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:07.175 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:07.175 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:07.175 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:07.175 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:07.175 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:07.175 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:07.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:07.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 4 set_mon_vals no callback set 2024-01-24T23:27:07.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 10 monclient: shutdown 2024-01-24T23:27:07.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:07.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 1 -- 172.21.15.132:0/184658239 >> v1:172.21.15.132:6790/0 conn(0x7f430809bcd0 legacy=0x7f430813bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:07.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 1 -- 172.21.15.132:0/184658239 shutdown_connections 2024-01-24T23:27:07.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 1 -- 172.21.15.132:0/184658239 >> 172.21.15.132:0/184658239 conn(0x7f43080a9bf0 msgr2=0x7f43080a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:07.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 1 -- 172.21.15.132:0/184658239 shutdown_connections 2024-01-24T23:27:07.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 1 -- 172.21.15.132:0/184658239 wait complete. 2024-01-24T23:27:07.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 10 monclient: build_initial_monmap 2024-01-24T23:27:07.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.162+0000 7f430f600640 10 monclient: monmap: 2024-01-24T23:27:07.176 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:07.176 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:07.176 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:07.176 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:07.176 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:07.176 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:07.176 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:07.176 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:07.176 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:07.177 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:07.177 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:07.177 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:07.177 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:07.177 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:07.177 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:07.177 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:07.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430f600640 1 Processor -- start 2024-01-24T23:27:07.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430f600640 1 -- start start 2024-01-24T23:27:07.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430f600640 10 monclient: init 2024-01-24T23:27:07.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430f600640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:07.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430f600640 10 monclient: _add_conns ranks=[7,6,1,0,4,2,3,5,8] 2024-01-24T23:27:07.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430f600640 10 monclient(hunting): picked mon.noname-i con 0x7f430809bcd0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:07.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430f600640 10 monclient(hunting): picked mon.noname-d con 0x7f430809c0f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:07.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430f600640 10 monclient(hunting): picked mon.noname-f con 0x7f4308135ea0 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:07.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430f600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43081cc1b0 con 0x7f4308135ea0 2024-01-24T23:27:07.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430f600640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f430809c6b0 con 0x7f430809c0f0 2024-01-24T23:27:07.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430d39c640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f430809bcd0 legacy=0x7f4308135050 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:07.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430f600640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4308135c70 con 0x7f430809bcd0 2024-01-24T23:27:07.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430f600640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:07.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430db9d640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f4308135ea0 legacy=0x7f43081d3cd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:27:07.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430f600640 10 monclient(hunting): authenticate will time out at 1314.492875s 2024-01-24T23:27:07.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430cb9b640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f430809c0f0 0x7f43081d05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:49600/0 (socket says 172.21.15.132:49600) 2024-01-24T23:27:07.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430cb9b640 1 -- 172.21.15.132:0/1330385946 learned_addr learned my addr 172.21.15.132:0/1330385946 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:07.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 1 -- 172.21.15.132:0/1330385946 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 707713629 0 0) 0x7f430809c6b0 con 0x7f430809c0f0 2024-01-24T23:27:07.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:07.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:07.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:07.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 10 monclient(hunting): my global_id is 6612 2024-01-24T23:27:07.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 1 -- 172.21.15.132:0/1330385946 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42ec003530 con 0x7f430809c0f0 2024-01-24T23:27:07.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 1 -- 172.21.15.132:0/1330385946 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1910216812 0 0) 0x7f42ec003530 con 0x7f430809c0f0 2024-01-24T23:27:07.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 1 -- 172.21.15.132:0/1330385946 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f430809c6b0 con 0x7f430809c0f0 2024-01-24T23:27:07.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 1 -- 172.21.15.132:0/1330385946 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f42f8004b30 con 0x7f430809c0f0 2024-01-24T23:27:07.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:07.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:27:07.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 10 monclient(hunting): dump: 2024-01-24T23:27:07.179 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:07.179 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:07.179 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:07.179 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:07.179 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:07.179 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:07.179 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:07.179 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:07.179 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:07.179 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:07.180 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:07.180 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:07.180 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:07.180 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:07.180 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:07.180 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:07.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:07.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 1 -- 172.21.15.132:0/1330385946 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 307416365 0 0) 0x7f430809c6b0 con 0x7f430809c0f0 2024-01-24T23:27:07.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 1 -- 172.21.15.132:0/1330385946 >> v1:172.21.15.132:6792/0 conn(0x7f430809bcd0 legacy=0x7f4308135050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:07.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 1 -- 172.21.15.132:0/1330385946 >> v1:172.21.15.132:6789/0 conn(0x7f4308135ea0 legacy=0x7f43081d3cd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:07.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 10 monclient: _finish_hunting 0 2024-01-24T23:27:07.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 1 monclient: found mon.d 2024-01-24T23:27:07.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:07.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:07.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 1 -- 172.21.15.132:0/1330385946 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430809fe80 con 0x7f430809c0f0 2024-01-24T23:27:07.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 10 monclient: _finish_auth 0 2024-01-24T23:27:07.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 10 monclient: _check_auth_tickets 2024-01-24T23:27:07.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:07.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430f600640 5 monclient: authenticate success, global_id 6612 2024-01-24T23:27:07.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430f600640 10 monclient: _renew_subs 2024-01-24T23:27:07.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430f600640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:07.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430f600640 1 -- 172.21.15.132:0/1330385946 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4308122500 con 0x7f430809c0f0 2024-01-24T23:27:07.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430f600640 10 monclient: _renew_subs 2024-01-24T23:27:07.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430f600640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:07.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430f600640 1 -- 172.21.15.132:0/1330385946 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4308120da0 con 0x7f430809c0f0 2024-01-24T23:27:07.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 1 -- 172.21.15.132:0/1330385946 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f42f8004f00 con 0x7f430809c0f0 2024-01-24T23:27:07.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:07.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 1 -- 172.21.15.132:0/1330385946 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f42f80053b0 con 0x7f430809c0f0 2024-01-24T23:27:07.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:07.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42ff7fe640 4 set_mon_vals no callback set 2024-01-24T23:27:07.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:07.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 10 monclient: dump: 2024-01-24T23:27:07.182 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:07.182 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:07.182 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:07.182 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:07.182 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:07.182 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:07.182 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:07.182 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:07.182 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:07.182 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:07.182 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:07.182 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:07.182 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:07.182 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:07.182 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:07.183 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:07.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430f600640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:07.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430f600640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:07.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430f600640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:07.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f430f600640 1 -- 172.21.15.132:0/1330385946 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e8005090 con 0x7f430809c0f0 2024-01-24T23:27:07.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.166+0000 7f42fdffb640 1 -- 172.21.15.132:0/1330385946 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f42f800fe30 con 0x7f430809c0f0 2024-01-24T23:27:07.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.170+0000 7f42fdffb640 1 -- 172.21.15.132:0/1330385946 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(207..207 src has 1..207) v4 ==== 13313+0+0 (unknown 3366947691 0 0) 0x7f42f8045fd0 con 0x7f430809c0f0 2024-01-24T23:27:07.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.170+0000 7f430d39c640 10 monclient: get_auth_request con 0x7f42ec036e50 auth_method 0 2024-01-24T23:27:07.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.182+0000 7f42fdffb640 1 -- 172.21.15.132:0/1330385946 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f42f8014b80 con 0x7f430809c0f0 2024-01-24T23:27:07.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.182+0000 7f42fdffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:07.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.182+0000 7f42fdffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:07.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.342+0000 7f430f600640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:07.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.342+0000 7f430f600640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:07.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.342+0000 7f430f600640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:07.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.342+0000 7f430f600640 1 -- 172.21.15.132:0/1330385946 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f42e8005340 con 0x7f430809c0f0 2024-01-24T23:27:07.998 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.994+0000 7f42fdffb640 1 -- 172.21.15.132:0/1330385946 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}]=0 added bucket h28 type host to location {rack=r1} v208) v1 ==== 168+0+0 (unknown 3634764746 0 0) 0x7f42f8013220 con 0x7f430809c0f0 2024-01-24T23:27:07.998 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.994+0000 7f42fdffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:07.998 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:07.994+0000 7f42fdffb640 10 monclient: _finish_command 2 = system:0 added bucket h28 type host to location {rack=r1} 2024-01-24T23:27:08.097 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.094+0000 7f430f600640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:08.097 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.094+0000 7f430f600640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:08.098 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.094+0000 7f430f600640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:08.098 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.094+0000 7f430f600640 1 -- 172.21.15.132:0/1330385946 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f42e8005a80 con 0x7f430809c0f0 2024-01-24T23:27:08.099 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.098+0000 7f42fdffb640 1 -- 172.21.15.132:0/1330385946 <== mon.6 v1:172.21.15.115:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h28' already exists v208) v1 ==== 147+0+0 (unknown 3235759097 0 0) 0x7f42f8013220 con 0x7f430809c0f0 2024-01-24T23:27:08.099 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.098+0000 7f42fdffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:08.099 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.098+0000 7f42fdffb640 10 monclient: _finish_command 3 = system:0 bucket 'h28' already exists 2024-01-24T23:27:08.099 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h28' already exists 2024-01-24T23:27:08.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.098+0000 7f430f600640 1 -- 172.21.15.132:0/1330385946 >> v1:172.21.15.132:6812/1835401922 conn(0x7f42ec036e50 legacy=0x7f42ec039310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:08.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.098+0000 7f430f600640 10 monclient: shutdown 2024-01-24T23:27:08.101 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.098+0000 7f430f600640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:08.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.098+0000 7f430f600640 1 -- 172.21.15.132:0/1330385946 >> v1:172.21.15.115:6792/0 conn(0x7f430809c0f0 legacy=0x7f43081d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:08.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.098+0000 7f430f600640 1 -- 172.21.15.132:0/1330385946 shutdown_connections 2024-01-24T23:27:08.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.098+0000 7f430f600640 1 -- 172.21.15.132:0/1330385946 >> 172.21.15.132:0/1330385946 conn(0x7f43080a9bf0 msgr2=0x7f43081bea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:08.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.098+0000 7f430f600640 1 -- 172.21.15.132:0/1330385946 shutdown_connections 2024-01-24T23:27:08.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.098+0000 7f430f600640 1 -- 172.21.15.132:0/1330385946 wait complete. 2024-01-24T23:27:08.117 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:08.117 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h29 host rack=r1 2024-01-24T23:27:08.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.234+0000 7f12369a2640 10 monclient: get_monmap_and_config 2024-01-24T23:27:08.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.234+0000 7f12369a2640 10 monclient: build_initial_monmap 2024-01-24T23:27:08.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.234+0000 7f12369a2640 10 monclient: monmap: 2024-01-24T23:27:08.237 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:08.237 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:08.237 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:08.238554+0000 2024-01-24T23:27:08.237 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:08.238554+0000 2024-01-24T23:27:08.237 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:08.237 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:08.237 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:08.237 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:08.237 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:08.237 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:08.237 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:08.238 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:08.238 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:08.238 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:08.238 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:08.238 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:08.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.234+0000 7f12369a2640 1 Processor -- start 2024-01-24T23:27:08.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.234+0000 7f12369a2640 1 -- start start 2024-01-24T23:27:08.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.234+0000 7f12369a2640 10 monclient: init 2024-01-24T23:27:08.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.234+0000 7f12369a2640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:08.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.234+0000 7f12369a2640 10 monclient: _add_conns ranks=[1,8,3,6,5,2,0,7,4] 2024-01-24T23:27:08.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.234+0000 7f12369a2640 10 monclient(hunting): picked mon.noname-f con 0x7f1230072690 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:08.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.234+0000 7f12369a2640 10 monclient(hunting): picked mon.noname-e con 0x7f123009ae90 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:08.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.234+0000 7f12369a2640 10 monclient(hunting): picked mon.noname-g con 0x7f123009bdc0 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:08.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.234+0000 7f12369a2640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1230064700 con 0x7f1230072690 2024-01-24T23:27:08.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.234+0000 7f12369a2640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f123009a790 con 0x7f123009bdc0 2024-01-24T23:27:08.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.234+0000 7f12369a2640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12300a18f0 con 0x7f123009ae90 2024-01-24T23:27:08.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.234+0000 7f12369a2640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:08.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.234+0000 7f12369a2640 10 monclient(hunting): authenticate will time out at 1315.563728s 2024-01-24T23:27:08.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.234+0000 7f122ffff640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f1230072690 legacy=0x7f12300a0ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:27:08.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.234+0000 7f1234f3f640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f123009bdc0 0x7f123013d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:58366/0 (socket says 172.21.15.132:58366) 2024-01-24T23:27:08.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.234+0000 7f1234f3f640 1 -- 172.21.15.132:0/3874363424 learned_addr learned my addr 172.21.15.132:0/3874363424 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 1 -- 172.21.15.132:0/3874363424 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 293245305 0 0) 0x7f123009a790 con 0x7f123009bdc0 2024-01-24T23:27:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 10 monclient(hunting): my global_id is 6519 2024-01-24T23:27:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 1 -- 172.21.15.132:0/3874363424 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1218003530 con 0x7f123009bdc0 2024-01-24T23:27:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 1 -- 172.21.15.132:0/3874363424 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2877249103 0 0) 0x7f1218003530 con 0x7f123009bdc0 2024-01-24T23:27:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 1 -- 172.21.15.132:0/3874363424 >> v1:172.21.15.115:6793/0 conn(0x7f123009ae90 legacy=0x7f12300a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 1 -- 172.21.15.132:0/3874363424 >> v1:172.21.15.132:6789/0 conn(0x7f1230072690 legacy=0x7f12300a0ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 10 monclient: _finish_hunting 0 2024-01-24T23:27:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 1 monclient: found mon.noname-g 2024-01-24T23:27:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:27:08.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 1 -- 172.21.15.132:0/3874363424 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12300997a0 con 0x7f123009bdc0 2024-01-24T23:27:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 10 monclient: _finish_auth 0 2024-01-24T23:27:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 10 monclient: _check_auth_tickets 2024-01-24T23:27:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 1 -- 172.21.15.132:0/3874363424 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1228001b40 con 0x7f123009bdc0 2024-01-24T23:27:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:27:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 10 monclient: dump: 2024-01-24T23:27:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:08.247 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:08.248 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:08.248 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:08.248 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:08.248 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:08.248 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:08.248 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:08.248 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:08.248 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:08.248 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:08.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 5 monclient: authenticate success, global_id 6519 2024-01-24T23:27:08.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:08.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 1 -- 172.21.15.132:0/3874363424 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1228001df0 con 0x7f123009bdc0 2024-01-24T23:27:08.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:08.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 4 set_mon_vals no callback set 2024-01-24T23:27:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 1 -- 172.21.15.132:0/3874363424 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1228004f70 con 0x7f123009bdc0 2024-01-24T23:27:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122effd640 10 monclient: dump: 2024-01-24T23:27:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:08.249 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:08.250 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 10 monclient: shutdown 2024-01-24T23:27:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 1 -- 172.21.15.132:0/3874363424 >> v1:172.21.15.132:6790/0 conn(0x7f123009bdc0 legacy=0x7f123013d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 1 -- 172.21.15.132:0/3874363424 shutdown_connections 2024-01-24T23:27:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 1 -- 172.21.15.132:0/3874363424 >> 172.21.15.132:0/3874363424 conn(0x7f12300a1bc0 msgr2=0x7f12300a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 1 -- 172.21.15.132:0/3874363424 shutdown_connections 2024-01-24T23:27:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 1 -- 172.21.15.132:0/3874363424 wait complete. 2024-01-24T23:27:08.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 10 monclient: build_initial_monmap 2024-01-24T23:27:08.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 10 monclient: monmap: 2024-01-24T23:27:08.251 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:08.251 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:08.251 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:08.251 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:08.251 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:08.251 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:08.251 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:08.251 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:08.251 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:08.251 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:08.251 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:08.251 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:08.251 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:08.251 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:08.251 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:08.252 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:08.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 1 Processor -- start 2024-01-24T23:27:08.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 1 -- start start 2024-01-24T23:27:08.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 10 monclient: init 2024-01-24T23:27:08.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:08.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 10 monclient: _add_conns ranks=[0,4,7,2,6,8,5,3,1] 2024-01-24T23:27:08.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 10 monclient(hunting): picked mon.noname-a con 0x7f1230072690 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:08.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 10 monclient(hunting): picked mon.noname-c con 0x7f123009ae90 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:08.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 10 monclient(hunting): picked mon.noname-i con 0x7f123009bdc0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:08.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12301ca240 con 0x7f1230072690 2024-01-24T23:27:08.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12301b2530 con 0x7f123009ae90 2024-01-24T23:27:08.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12301b2810 con 0x7f123009bdc0 2024-01-24T23:27:08.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:08.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f12369a2640 10 monclient(hunting): authenticate will time out at 1315.566755s 2024-01-24T23:27:08.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f1234f3f640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f123009bdc0 legacy=0x7f12301d1760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:08.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122f7fe640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f123009ae90 0x7f12301c9b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:33754/0 (socket says 172.21.15.132:33754) 2024-01-24T23:27:08.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122f7fe640 1 -- 172.21.15.132:0/2726769178 learned_addr learned my addr 172.21.15.132:0/2726769178 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:08.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122cff9640 1 -- 172.21.15.132:0/2726769178 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889438295 0 0) 0x7f12301b2530 con 0x7f123009ae90 2024-01-24T23:27:08.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:08.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:08.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:08.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122cff9640 10 monclient(hunting): my global_id is 94666 2024-01-24T23:27:08.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122cff9640 1 -- 172.21.15.132:0/2726769178 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f121c003530 con 0x7f123009ae90 2024-01-24T23:27:08.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122cff9640 1 -- 172.21.15.132:0/2726769178 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1612116130 0 0) 0x7f12301ca240 con 0x7f1230072690 2024-01-24T23:27:08.253 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:08.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:08.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:08.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122cff9640 10 monclient(hunting): my global_id is 94851 2024-01-24T23:27:08.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122cff9640 1 -- 172.21.15.132:0/2726769178 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12301b2530 con 0x7f1230072690 2024-01-24T23:27:08.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.238+0000 7f122cff9640 1 -- 172.21.15.132:0/2726769178 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3191226307 0 0) 0x7f121c003530 con 0x7f123009ae90 2024-01-24T23:27:08.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 1 -- 172.21.15.132:0/2726769178 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f12301ca240 con 0x7f123009ae90 2024-01-24T23:27:08.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 1 -- 172.21.15.132:0/2726769178 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1224001a60 con 0x7f123009ae90 2024-01-24T23:27:08.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:08.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:27:08.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 10 monclient(hunting): dump: 2024-01-24T23:27:08.254 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:08.254 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:08.254 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:08.254 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:08.254 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:08.255 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:08.255 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:08.255 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:08.255 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:08.255 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:08.255 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:08.255 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:08.255 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:08.255 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:08.255 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:08.255 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:08.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:08.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 1 -- 172.21.15.132:0/2726769178 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2708611896 0 0) 0x7f12301b2530 con 0x7f1230072690 2024-01-24T23:27:08.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 1 -- 172.21.15.132:0/2726769178 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f121c003530 con 0x7f1230072690 2024-01-24T23:27:08.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 1 -- 172.21.15.132:0/2726769178 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f12200041f0 con 0x7f1230072690 2024-01-24T23:27:08.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:08.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:27:08.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 10 monclient(hunting): dump: 2024-01-24T23:27:08.256 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:08.256 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:08.256 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:08.256 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:08.256 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:08.256 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:08.256 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:08.256 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:08.256 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:08.256 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:08.257 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:08.257 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:08.257 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:08.257 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:08.257 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:08.257 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:08.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 1 -- 172.21.15.132:0/2726769178 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2392383891 0 0) 0x7f12301ca240 con 0x7f123009ae90 2024-01-24T23:27:08.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 1 -- 172.21.15.132:0/2726769178 >> v1:172.21.15.132:6792/0 conn(0x7f123009bdc0 legacy=0x7f12301d1760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:08.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 1 -- 172.21.15.132:0/2726769178 >> v1:172.21.15.115:6789/0 conn(0x7f1230072690 legacy=0x7f12301c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:08.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 10 monclient: _finish_hunting 0 2024-01-24T23:27:08.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 1 monclient: found mon.c 2024-01-24T23:27:08.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:08.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:08.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 1 -- 172.21.15.132:0/2726769178 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12301b12f0 con 0x7f123009ae90 2024-01-24T23:27:08.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 10 monclient: _finish_auth 0 2024-01-24T23:27:08.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 10 monclient: _check_auth_tickets 2024-01-24T23:27:08.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:08.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f12369a2640 5 monclient: authenticate success, global_id 94666 2024-01-24T23:27:08.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f12369a2640 10 monclient: _renew_subs 2024-01-24T23:27:08.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f12369a2640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:08.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f12369a2640 1 -- 172.21.15.132:0/2726769178 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12301b6a40 con 0x7f123009ae90 2024-01-24T23:27:08.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f12369a2640 10 monclient: _renew_subs 2024-01-24T23:27:08.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f12369a2640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:08.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f12369a2640 1 -- 172.21.15.132:0/2726769178 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f12301b0e80 con 0x7f123009ae90 2024-01-24T23:27:08.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 1 -- 172.21.15.132:0/2726769178 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1224001d40 con 0x7f123009ae90 2024-01-24T23:27:08.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:08.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122e7fc640 4 set_mon_vals no callback set 2024-01-24T23:27:08.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 1 -- 172.21.15.132:0/2726769178 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1224003c90 con 0x7f123009ae90 2024-01-24T23:27:08.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:08.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:08.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 10 monclient: dump: 2024-01-24T23:27:08.259 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:08.259 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:08.259 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:08.259 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:08.259 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:08.259 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:08.259 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:08.259 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:08.259 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:08.259 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:08.259 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:08.259 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:08.259 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:08.260 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:08.260 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:08.260 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:08.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f12369a2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:08.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f12369a2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:08.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f12369a2640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:08.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f12369a2640 1 -- 172.21.15.132:0/2726769178 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1210005090 con 0x7f123009ae90 2024-01-24T23:27:08.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 1 -- 172.21.15.132:0/2726769178 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f1224003e70 con 0x7f123009ae90 2024-01-24T23:27:08.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122cff9640 1 -- 172.21.15.132:0/2726769178 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(208..208 src has 1..208) v4 ==== 13526+0+0 (unknown 944022840 0 0) 0x7f1224045de0 con 0x7f123009ae90 2024-01-24T23:27:08.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.242+0000 7f122ffff640 10 monclient: get_auth_request con 0x7f121c037610 auth_method 0 2024-01-24T23:27:08.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.254+0000 7f122cff9640 1 -- 172.21.15.132:0/2726769178 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f1224014800 con 0x7f123009ae90 2024-01-24T23:27:08.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.254+0000 7f122cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:08.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.254+0000 7f122cff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:08.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.414+0000 7f12369a2640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:08.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.414+0000 7f12369a2640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:08.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.414+0000 7f12369a2640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:08.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:08.414+0000 7f12369a2640 1 -- 172.21.15.132:0/2726769178 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f1210005340 con 0x7f123009ae90 2024-01-24T23:27:09.006 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.002+0000 7f122cff9640 1 -- 172.21.15.132:0/2726769178 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}]=0 added bucket h29 type host to location {rack=r1} v209) v1 ==== 168+0+0 (unknown 2763300407 0 0) 0x7f1224015360 con 0x7f123009ae90 2024-01-24T23:27:09.006 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.002+0000 7f122cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:09.006 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.002+0000 7f122cff9640 10 monclient: _finish_command 2 = system:0 added bucket h29 type host to location {rack=r1} 2024-01-24T23:27:09.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.102+0000 7f12369a2640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:09.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.102+0000 7f12369a2640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:09.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.102+0000 7f12369a2640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:09.107 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.102+0000 7f12369a2640 1 -- 172.21.15.132:0/2726769178 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f1210005a80 con 0x7f123009ae90 2024-01-24T23:27:09.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.106+0000 7f122cff9640 1 -- 172.21.15.132:0/2726769178 <== mon.4 v1:172.21.15.115:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h29' already exists v209) v1 ==== 147+0+0 (unknown 3412843028 0 0) 0x7f122401cf30 con 0x7f123009ae90 2024-01-24T23:27:09.108 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h29' already exists 2024-01-24T23:27:09.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.106+0000 7f122cff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:09.108 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.106+0000 7f122cff9640 10 monclient: _finish_command 3 = system:0 bucket 'h29' already exists 2024-01-24T23:27:09.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.106+0000 7f12369a2640 1 -- 172.21.15.132:0/2726769178 >> v1:172.21.15.132:6812/1835401922 conn(0x7f121c037610 legacy=0x7f121c039ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:09.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.106+0000 7f12369a2640 10 monclient: shutdown 2024-01-24T23:27:09.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.106+0000 7f12369a2640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:09.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.106+0000 7f12369a2640 1 -- 172.21.15.132:0/2726769178 >> v1:172.21.15.115:6791/0 conn(0x7f123009ae90 legacy=0x7f12301c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:09.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.106+0000 7f12369a2640 1 -- 172.21.15.132:0/2726769178 shutdown_connections 2024-01-24T23:27:09.110 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.106+0000 7f12369a2640 1 -- 172.21.15.132:0/2726769178 >> 172.21.15.132:0/2726769178 conn(0x7f12300a1bc0 msgr2=0x7f12301ba850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:09.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.106+0000 7f12369a2640 1 -- 172.21.15.132:0/2726769178 shutdown_connections 2024-01-24T23:27:09.111 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.106+0000 7f12369a2640 1 -- 172.21.15.132:0/2726769178 wait complete. 2024-01-24T23:27:09.126 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:09.126 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h30 host rack=r1 2024-01-24T23:27:09.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.246+0000 7ff8c5031640 10 monclient: get_monmap_and_config 2024-01-24T23:27:09.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.246+0000 7ff8c5031640 10 monclient: build_initial_monmap 2024-01-24T23:27:09.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.246+0000 7ff8c5031640 10 monclient: monmap: 2024-01-24T23:27:09.248 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:09.249 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:09.249 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:09.250345+0000 2024-01-24T23:27:09.249 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:09.250345+0000 2024-01-24T23:27:09.249 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:09.249 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:09.249 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:09.249 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:09.249 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:09.249 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:09.249 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:09.249 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:09.249 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:09.249 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:09.249 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:09.249 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:09.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.246+0000 7ff8c5031640 1 Processor -- start 2024-01-24T23:27:09.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.246+0000 7ff8c5031640 1 -- start start 2024-01-24T23:27:09.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.246+0000 7ff8c5031640 10 monclient: init 2024-01-24T23:27:09.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.246+0000 7ff8c5031640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:09.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.246+0000 7ff8c5031640 10 monclient: _add_conns ranks=[8,7,6,3,1,2,4,5,0] 2024-01-24T23:27:09.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.246+0000 7ff8c5031640 10 monclient(hunting): picked mon.noname-e con 0x7ff8c00a0ad0 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:09.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.246+0000 7ff8c5031640 10 monclient(hunting): picked mon.noname-i con 0x7ff8c009b7a0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:09.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.246+0000 7ff8c5031640 10 monclient(hunting): picked mon.noname-d con 0x7ff8c00a15c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:09.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.246+0000 7ff8c5031640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c0064700 con 0x7ff8c00a15c0 2024-01-24T23:27:09.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.246+0000 7ff8c5031640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c009a4d0 con 0x7ff8c009b7a0 2024-01-24T23:27:09.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.246+0000 7ff8c5031640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c009aa90 con 0x7ff8c00a0ad0 2024-01-24T23:27:09.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.246+0000 7ff8c5031640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:09.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.246+0000 7ff8c5031640 10 monclient(hunting): authenticate will time out at 1316.575791s 2024-01-24T23:27:09.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.246+0000 7ff8be59c640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7ff8c009b7a0 legacy=0x7ff8c013c280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:09.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bf59e640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7ff8c00a15c0 0x7ff8c013f9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:49608/0 (socket says 172.21.15.132:49608) 2024-01-24T23:27:09.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bf59e640 1 -- 172.21.15.132:0/224673901 learned_addr learned my addr 172.21.15.132:0/224673901 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:09.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 1 -- 172.21.15.132:0/224673901 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3975715543 0 0) 0x7ff8c0064700 con 0x7ff8c00a15c0 2024-01-24T23:27:09.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 10 monclient(hunting): _negotiate 2024-01-24T23:27:09.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:09.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:09.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 10 monclient(hunting): my global_id is 6621 2024-01-24T23:27:09.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 1 -- 172.21.15.132:0/224673901 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8a8003530 con 0x7ff8c00a15c0 2024-01-24T23:27:09.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 1 -- 172.21.15.132:0/224673901 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2628553484 0 0) 0x7ff8c009aa90 con 0x7ff8c00a0ad0 2024-01-24T23:27:09.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 10 monclient(hunting): _negotiate 2024-01-24T23:27:09.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:09.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:09.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 10 monclient(hunting): my global_id is 85733 2024-01-24T23:27:09.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 1 -- 172.21.15.132:0/224673901 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8c0064700 con 0x7ff8c00a0ad0 2024-01-24T23:27:09.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 1 -- 172.21.15.132:0/224673901 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2964126273 0 0) 0x7ff8c0064700 con 0x7ff8c00a0ad0 2024-01-24T23:27:09.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 1 -- 172.21.15.132:0/224673901 >> v1:172.21.15.132:6792/0 conn(0x7ff8c009b7a0 legacy=0x7ff8c013c280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:09.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 1 -- 172.21.15.132:0/224673901 >> v1:172.21.15.115:6792/0 conn(0x7ff8c00a15c0 legacy=0x7ff8c013f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:09.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 10 monclient: _finish_hunting 0 2024-01-24T23:27:09.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 1 monclient: found mon.noname-e 2024-01-24T23:27:09.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:09.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:27:09.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 1 -- 172.21.15.132:0/224673901 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c0099df0 con 0x7ff8c00a0ad0 2024-01-24T23:27:09.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 10 monclient: _finish_auth 0 2024-01-24T23:27:09.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 10 monclient: _check_auth_tickets 2024-01-24T23:27:09.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:09.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 1 -- 172.21.15.132:0/224673901 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff8b00017a0 con 0x7ff8c00a0ad0 2024-01-24T23:27:09.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:09.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:27:09.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 10 monclient: dump: 2024-01-24T23:27:09.256 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:09.257 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:09.257 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:09.257 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:09.257 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:09.257 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:09.257 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:09.257 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:09.257 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:09.257 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:09.257 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:09.257 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:09.257 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:09.257 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:09.257 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:09.257 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:09.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 5 monclient: authenticate success, global_id 85733 2024-01-24T23:27:09.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:09.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 1 -- 172.21.15.132:0/224673901 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff8b0001a50 con 0x7ff8c00a0ad0 2024-01-24T23:27:09.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:09.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 1 -- 172.21.15.132:0/224673901 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff8b0003960 con 0x7ff8c00a0ad0 2024-01-24T23:27:09.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:09.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:27:09.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bdd9b640 10 monclient: dump: 2024-01-24T23:27:09.258 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:09.258 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:09.258 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:09.258 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:09.258 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:09.258 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:09.258 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:09.258 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:09.259 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:09.259 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:09.259 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:09.259 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:09.259 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:09.259 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:09.259 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:09.259 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:09.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:09.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 4 set_mon_vals no callback set 2024-01-24T23:27:09.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 10 monclient: shutdown 2024-01-24T23:27:09.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:09.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 1 -- 172.21.15.132:0/224673901 >> v1:172.21.15.115:6793/0 conn(0x7ff8c00a0ad0 legacy=0x7ff8c00a0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:09.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 1 -- 172.21.15.132:0/224673901 shutdown_connections 2024-01-24T23:27:09.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 1 -- 172.21.15.132:0/224673901 >> 172.21.15.132:0/224673901 conn(0x7ff8c00a60d0 msgr2=0x7ff8c00a64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:09.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 1 -- 172.21.15.132:0/224673901 shutdown_connections 2024-01-24T23:27:09.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 1 -- 172.21.15.132:0/224673901 wait complete. 2024-01-24T23:27:09.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 10 monclient: build_initial_monmap 2024-01-24T23:27:09.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 10 monclient: monmap: 2024-01-24T23:27:09.264 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:09.264 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:09.264 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:09.264 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:09.264 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:09.264 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:09.264 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:09.264 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:09.265 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:09.265 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:09.265 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:09.265 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:09.265 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:09.265 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:09.265 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:09.265 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:09.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 1 Processor -- start 2024-01-24T23:27:09.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 1 -- start start 2024-01-24T23:27:09.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 10 monclient: init 2024-01-24T23:27:09.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:09.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 10 monclient: _add_conns ranks=[7,5,4,6,2,1,3,0,8] 2024-01-24T23:27:09.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 10 monclient(hunting): picked mon.noname-i con 0x7ff8c009b7a0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:09.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 10 monclient(hunting): picked mon.noname-h con 0x7ff8c00a15c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:09.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 10 monclient(hunting): picked mon.noname-c con 0x7ff8c01d1e50 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:09.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c0070100 con 0x7ff8c01d1e50 2024-01-24T23:27:09.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c00a27d0 con 0x7ff8c00a15c0 2024-01-24T23:27:09.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c0071f20 con 0x7ff8c009b7a0 2024-01-24T23:27:09.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:09.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8bed9d640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7ff8c009b7a0 legacy=0x7ff8c006e7d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:09.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8c5031640 10 monclient(hunting): authenticate will time out at 1316.579697s 2024-01-24T23:27:09.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8be59c640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7ff8c00a15c0 0x7ff8c006eee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:59690/0 (socket says 172.21.15.132:59690) 2024-01-24T23:27:09.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.250+0000 7ff8be59c640 1 -- 172.21.15.132:0/3432354631 learned_addr learned my addr 172.21.15.132:0/3432354631 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:09.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 1 -- 172.21.15.132:0/3432354631 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454276744 0 0) 0x7ff8c00a27d0 con 0x7ff8c00a15c0 2024-01-24T23:27:09.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:09.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:09.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:09.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 10 monclient(hunting): my global_id is 6440 2024-01-24T23:27:09.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 1 -- 172.21.15.132:0/3432354631 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8ac003530 con 0x7ff8c00a15c0 2024-01-24T23:27:09.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 1 -- 172.21.15.132:0/3432354631 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2306161002 0 0) 0x7ff8ac003530 con 0x7ff8c00a15c0 2024-01-24T23:27:09.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 1 -- 172.21.15.132:0/3432354631 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff8c00a27d0 con 0x7ff8c00a15c0 2024-01-24T23:27:09.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 1 -- 172.21.15.132:0/3432354631 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff8b4004300 con 0x7ff8c00a15c0 2024-01-24T23:27:09.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:09.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:27:09.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 10 monclient(hunting): dump: 2024-01-24T23:27:09.267 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:09.267 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:09.267 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:09.267 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:09.267 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:09.267 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:09.267 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:09.267 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:09.267 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:09.268 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:09.268 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:09.268 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:09.268 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:09.268 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:09.268 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:09.268 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:09.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:09.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 1 -- 172.21.15.132:0/3432354631 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983632993 0 0) 0x7ff8c0070100 con 0x7ff8c01d1e50 2024-01-24T23:27:09.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:09.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:09.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:09.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 10 monclient(hunting): my global_id is 94675 2024-01-24T23:27:09.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 1 -- 172.21.15.132:0/3432354631 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8ac003530 con 0x7ff8c01d1e50 2024-01-24T23:27:09.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 1 -- 172.21.15.132:0/3432354631 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 935380542 0 0) 0x7ff8c00a27d0 con 0x7ff8c00a15c0 2024-01-24T23:27:09.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 1 -- 172.21.15.132:0/3432354631 >> v1:172.21.15.132:6792/0 conn(0x7ff8c009b7a0 legacy=0x7ff8c006e7d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:09.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 1 -- 172.21.15.132:0/3432354631 >> v1:172.21.15.115:6791/0 conn(0x7ff8c01d1e50 legacy=0x7ff8c01d4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:09.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 10 monclient: _finish_hunting 0 2024-01-24T23:27:09.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 1 monclient: found mon.h 2024-01-24T23:27:09.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:09.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:09.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 1 -- 172.21.15.132:0/3432354631 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c006f5f0 con 0x7ff8c00a15c0 2024-01-24T23:27:09.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 10 monclient: _finish_auth 0 2024-01-24T23:27:09.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:09.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:09.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8c5031640 5 monclient: authenticate success, global_id 6440 2024-01-24T23:27:09.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8c5031640 10 monclient: _renew_subs 2024-01-24T23:27:09.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8c5031640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:09.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8c5031640 1 -- 172.21.15.132:0/3432354631 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c006fc50 con 0x7ff8c00a15c0 2024-01-24T23:27:09.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8c5031640 10 monclient: _renew_subs 2024-01-24T23:27:09.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8c5031640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:09.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8c5031640 1 -- 172.21.15.132:0/3432354631 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c00992a0 con 0x7ff8c00a15c0 2024-01-24T23:27:09.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 1 -- 172.21.15.132:0/3432354631 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff8b40045b0 con 0x7ff8c00a15c0 2024-01-24T23:27:09.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:09.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 1 -- 172.21.15.132:0/3432354631 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff8b4004ae0 con 0x7ff8c00a15c0 2024-01-24T23:27:09.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:09.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8bd59a640 4 set_mon_vals no callback set 2024-01-24T23:27:09.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:09.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 10 monclient: dump: 2024-01-24T23:27:09.270 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:09.270 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:09.270 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:09.270 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:09.270 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:09.270 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:09.270 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:09.271 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:09.271 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:09.271 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:09.271 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:09.271 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:09.271 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:09.271 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:09.271 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:09.271 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:09.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 1 -- 172.21.15.132:0/3432354631 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7ff8b4004c40 con 0x7ff8c00a15c0 2024-01-24T23:27:09.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8c5031640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:09.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8c5031640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:09.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8c5031640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:09.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8c5031640 1 -- 172.21.15.132:0/3432354631 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a0005090 con 0x7ff8c00a15c0 2024-01-24T23:27:09.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8a77fe640 1 -- 172.21.15.132:0/3432354631 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(209..209 src has 1..209) v4 ==== 13739+0+0 (unknown 543472313 0 0) 0x7ff8b4045f70 con 0x7ff8c00a15c0 2024-01-24T23:27:09.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.254+0000 7ff8bed9d640 10 monclient: get_auth_request con 0x7ff8ac0370a0 auth_method 0 2024-01-24T23:27:09.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.258+0000 7ff8a77fe640 1 -- 172.21.15.132:0/3432354631 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7ff8b4014aa0 con 0x7ff8c00a15c0 2024-01-24T23:27:09.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.258+0000 7ff8a77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:09.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.258+0000 7ff8a77fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:09.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.422+0000 7ff8c5031640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:09.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.422+0000 7ff8c5031640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:09.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.422+0000 7ff8c5031640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:09.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:09.422+0000 7ff8c5031640 1 -- 172.21.15.132:0/3432354631 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7ff8a0005340 con 0x7ff8c00a15c0 2024-01-24T23:27:10.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.018+0000 7ff8a77fe640 1 -- 172.21.15.132:0/3432354631 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}]=0 added bucket h30 type host to location {rack=r1} v210) v1 ==== 168+0+0 (unknown 2638623544 0 0) 0x7ff8b4012f10 con 0x7ff8c00a15c0 2024-01-24T23:27:10.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.018+0000 7ff8a77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:10.019 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.018+0000 7ff8a77fe640 10 monclient: _finish_command 2 = system:0 added bucket h30 type host to location {rack=r1} 2024-01-24T23:27:10.124 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.122+0000 7ff8c5031640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:10.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.122+0000 7ff8c5031640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:10.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.122+0000 7ff8c5031640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:10.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.122+0000 7ff8c5031640 1 -- 172.21.15.132:0/3432354631 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7ff8a0005a80 con 0x7ff8c00a15c0 2024-01-24T23:27:10.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.122+0000 7ff8a77fe640 1 -- 172.21.15.132:0/3432354631 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h30' already exists v210) v1 ==== 147+0+0 (unknown 2185424156 0 0) 0x7ff8b4014690 con 0x7ff8c00a15c0 2024-01-24T23:27:10.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.122+0000 7ff8a77fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:10.125 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.122+0000 7ff8a77fe640 10 monclient: _finish_command 3 = system:0 bucket 'h30' already exists 2024-01-24T23:27:10.125 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h30' already exists 2024-01-24T23:27:10.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.126+0000 7ff8c5031640 1 -- 172.21.15.132:0/3432354631 >> v1:172.21.15.132:6812/1835401922 conn(0x7ff8ac0370a0 legacy=0x7ff8ac039560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:10.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.126+0000 7ff8c5031640 10 monclient: shutdown 2024-01-24T23:27:10.127 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.126+0000 7ff8c5031640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:10.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.126+0000 7ff8c5031640 1 -- 172.21.15.132:0/3432354631 >> v1:172.21.15.132:6791/0 conn(0x7ff8c00a15c0 legacy=0x7ff8c006eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:10.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.126+0000 7ff8c5031640 1 -- 172.21.15.132:0/3432354631 shutdown_connections 2024-01-24T23:27:10.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.126+0000 7ff8c5031640 1 -- 172.21.15.132:0/3432354631 >> 172.21.15.132:0/3432354631 conn(0x7ff8c00a60d0 msgr2=0x7ff8c01be5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:10.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.126+0000 7ff8c5031640 1 -- 172.21.15.132:0/3432354631 shutdown_connections 2024-01-24T23:27:10.128 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.126+0000 7ff8c5031640 1 -- 172.21.15.132:0/3432354631 wait complete. 2024-01-24T23:27:10.142 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:10.142 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h31 host rack=r1 2024-01-24T23:27:10.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe3f5fb640 10 monclient: get_monmap_and_config 2024-01-24T23:27:10.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe3f5fb640 10 monclient: build_initial_monmap 2024-01-24T23:27:10.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe3f5fb640 10 monclient: monmap: 2024-01-24T23:27:10.259 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:10.259 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:10.260 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:10.261314+0000 2024-01-24T23:27:10.260 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:10.261314+0000 2024-01-24T23:27:10.260 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:10.260 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:10.260 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:10.260 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:10.260 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:10.260 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:10.260 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:10.260 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:10.260 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:10.260 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:10.260 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:10.261 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:10.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe3f5fb640 1 Processor -- start 2024-01-24T23:27:10.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe3f5fb640 1 -- start start 2024-01-24T23:27:10.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe3f5fb640 10 monclient: init 2024-01-24T23:27:10.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe3f5fb640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:10.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe3f5fb640 10 monclient: _add_conns ranks=[8,5,0,1,6,7,4,3,2] 2024-01-24T23:27:10.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe3f5fb640 10 monclient(hunting): picked mon.noname-e con 0x7fbe38072690 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:10.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe3f5fb640 10 monclient(hunting): picked mon.noname-h con 0x7fbe3809ae90 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:10.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe3f5fb640 10 monclient(hunting): picked mon.noname-a con 0x7fbe3809bdc0 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:10.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe3f5fb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe38064700 con 0x7fbe3809bdc0 2024-01-24T23:27:10.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe3f5fb640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3809a790 con 0x7fbe3809ae90 2024-01-24T23:27:10.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe3f5fb640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe380a18f0 con 0x7fbe38072690 2024-01-24T23:27:10.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe3f5fb640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:10.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe3f5fb640 10 monclient(hunting): authenticate will time out at 1317.586552s 2024-01-24T23:27:10.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe3cb96640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fbe3809ae90 0x7fbe380a11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:59702/0 (socket says 172.21.15.132:59702) 2024-01-24T23:27:10.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe3cb96640 1 -- 172.21.15.132:0/2891663409 learned_addr learned my addr 172.21.15.132:0/2891663409 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:10.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 1 -- 172.21.15.132:0/2891663409 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3724984433 0 0) 0x7fbe3809a790 con 0x7fbe3809ae90 2024-01-24T23:27:10.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:10.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:10.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:10.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 10 monclient(hunting): my global_id is 6449 2024-01-24T23:27:10.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 1 -- 172.21.15.132:0/2891663409 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe24003530 con 0x7fbe3809ae90 2024-01-24T23:27:10.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 1 -- 172.21.15.132:0/2891663409 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2767772350 0 0) 0x7fbe24003530 con 0x7fbe3809ae90 2024-01-24T23:27:10.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 1 -- 172.21.15.132:0/2891663409 >> v1:172.21.15.115:6793/0 conn(0x7fbe38072690 legacy=0x7fbe380a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:10.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 1 -- 172.21.15.132:0/2891663409 >> v1:172.21.15.115:6789/0 conn(0x7fbe3809bdc0 legacy=0x7fbe3813d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:10.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:27:10.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 1 monclient: found mon.noname-h 2024-01-24T23:27:10.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:10.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:27:10.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 1 -- 172.21.15.132:0/2891663409 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe380997a0 con 0x7fbe3809ae90 2024-01-24T23:27:10.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 10 monclient: _finish_auth 0 2024-01-24T23:27:10.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:27:10.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:10.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 1 -- 172.21.15.132:0/2891663409 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbe2c0017c0 con 0x7fbe3809ae90 2024-01-24T23:27:10.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:10.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:27:10.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 10 monclient: dump: 2024-01-24T23:27:10.270 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:10.270 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:10.270 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:10.270 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:10.270 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:10.270 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:10.270 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:10.270 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:10.270 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:10.270 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:10.271 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:10.271 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:10.271 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:10.271 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:10.271 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:10.271 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:10.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe3f5fb640 5 monclient: authenticate success, global_id 6449 2024-01-24T23:27:10.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe3f5fb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:10.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 1 -- 172.21.15.132:0/2891663409 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbe2c001a70 con 0x7fbe3809ae90 2024-01-24T23:27:10.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:10.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 1 -- 172.21.15.132:0/2891663409 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbe2c004bf0 con 0x7fbe3809ae90 2024-01-24T23:27:10.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:10.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:10.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe2bfff640 10 monclient: dump: 2024-01-24T23:27:10.272 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:10.272 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:10.272 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:10.272 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:10.272 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:10.272 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:10.272 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:10.272 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:10.272 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:10.272 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:10.272 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:10.272 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:10.272 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:10.272 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:10.272 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:10.273 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:10.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe3f5fb640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:10.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.258+0000 7fbe3f5fb640 4 set_mon_vals no callback set 2024-01-24T23:27:10.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 10 monclient: shutdown 2024-01-24T23:27:10.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:10.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 1 -- 172.21.15.132:0/2891663409 >> v1:172.21.15.132:6791/0 conn(0x7fbe3809ae90 legacy=0x7fbe380a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:10.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 1 -- 172.21.15.132:0/2891663409 shutdown_connections 2024-01-24T23:27:10.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 1 -- 172.21.15.132:0/2891663409 >> 172.21.15.132:0/2891663409 conn(0x7fbe380a1bc0 msgr2=0x7fbe380a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:10.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 1 -- 172.21.15.132:0/2891663409 shutdown_connections 2024-01-24T23:27:10.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 1 -- 172.21.15.132:0/2891663409 wait complete. 2024-01-24T23:27:10.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 10 monclient: build_initial_monmap 2024-01-24T23:27:10.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 10 monclient: monmap: 2024-01-24T23:27:10.276 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:10.276 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:10.276 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:10.276 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:10.276 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:10.276 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:10.276 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:10.276 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:10.276 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:10.276 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:10.276 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:10.276 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:10.276 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:10.276 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:10.277 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:10.277 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:10.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 1 Processor -- start 2024-01-24T23:27:10.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 1 -- start start 2024-01-24T23:27:10.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 10 monclient: init 2024-01-24T23:27:10.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:10.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 10 monclient: _add_conns ranks=[0,1,5,2,8,6,4,3,7] 2024-01-24T23:27:10.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 10 monclient(hunting): picked mon.noname-a con 0x7fbe38072690 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:10.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 10 monclient(hunting): picked mon.noname-f con 0x7fbe3809ae90 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:10.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 10 monclient(hunting): picked mon.noname-h con 0x7fbe3809bdc0 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:10.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe38099470 con 0x7fbe38072690 2024-01-24T23:27:10.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3809de60 con 0x7fbe3809ae90 2024-01-24T23:27:10.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe381b6960 con 0x7fbe3809bdc0 2024-01-24T23:27:10.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:10.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 10 monclient(hunting): authenticate will time out at 1317.589713s 2024-01-24T23:27:10.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3cb96640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7fbe3809ae90 legacy=0x7fbe3809ccd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:27:10.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3db98640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fbe3809bdc0 0x7fbe381d3a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:59718/0 (socket says 172.21.15.132:59718) 2024-01-24T23:27:10.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3db98640 1 -- 172.21.15.132:0/1224532080 learned_addr learned my addr 172.21.15.132:0/1224532080 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:10.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 1 -- 172.21.15.132:0/1224532080 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2679686289 0 0) 0x7fbe381b6960 con 0x7fbe3809bdc0 2024-01-24T23:27:10.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:10.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:10.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:10.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 10 monclient(hunting): my global_id is 6458 2024-01-24T23:27:10.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 1 -- 172.21.15.132:0/1224532080 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe1c003530 con 0x7fbe3809bdc0 2024-01-24T23:27:10.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 1 -- 172.21.15.132:0/1224532080 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1553762966 0 0) 0x7fbe1c003530 con 0x7fbe3809bdc0 2024-01-24T23:27:10.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 1 -- 172.21.15.132:0/1224532080 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbe381b6960 con 0x7fbe3809bdc0 2024-01-24T23:27:10.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 1 -- 172.21.15.132:0/1224532080 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbe30001da0 con 0x7fbe3809bdc0 2024-01-24T23:27:10.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:10.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:27:10.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 10 monclient(hunting): dump: 2024-01-24T23:27:10.279 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:10.279 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:10.279 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:10.279 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:10.279 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:10.279 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:10.279 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:10.279 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:10.280 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:10.280 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:10.280 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:10.280 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:10.280 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:10.280 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:10.280 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:10.280 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:10.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:10.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 1 -- 172.21.15.132:0/1224532080 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 965938356 0 0) 0x7fbe38099470 con 0x7fbe38072690 2024-01-24T23:27:10.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:10.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:10.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:10.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 10 monclient(hunting): my global_id is 94869 2024-01-24T23:27:10.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 1 -- 172.21.15.132:0/1224532080 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe1c003530 con 0x7fbe38072690 2024-01-24T23:27:10.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 1 -- 172.21.15.132:0/1224532080 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2933433198 0 0) 0x7fbe381b6960 con 0x7fbe3809bdc0 2024-01-24T23:27:10.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 1 -- 172.21.15.132:0/1224532080 >> v1:172.21.15.132:6789/0 conn(0x7fbe3809ae90 legacy=0x7fbe3809ccd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:10.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 1 -- 172.21.15.132:0/1224532080 >> v1:172.21.15.115:6789/0 conn(0x7fbe38072690 legacy=0x7fbe3809c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:10.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:27:10.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 1 monclient: found mon.h 2024-01-24T23:27:10.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:10.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:10.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 1 -- 172.21.15.132:0/1224532080 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3809d3e0 con 0x7fbe3809bdc0 2024-01-24T23:27:10.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 10 monclient: _finish_auth 0 2024-01-24T23:27:10.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:27:10.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:10.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 5 monclient: authenticate success, global_id 6458 2024-01-24T23:27:10.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 10 monclient: _renew_subs 2024-01-24T23:27:10.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:10.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 1 -- 172.21.15.132:0/1224532080 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3809b930 con 0x7fbe3809bdc0 2024-01-24T23:27:10.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 10 monclient: _renew_subs 2024-01-24T23:27:10.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:10.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 1 -- 172.21.15.132:0/1224532080 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe381b6b60 con 0x7fbe3809bdc0 2024-01-24T23:27:10.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 1 -- 172.21.15.132:0/1224532080 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbe30003cd0 con 0x7fbe3809bdc0 2024-01-24T23:27:10.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:10.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 1 -- 172.21.15.132:0/1224532080 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbe30004200 con 0x7fbe3809bdc0 2024-01-24T23:27:10.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:10.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:10.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe2affd640 4 set_mon_vals no callback set 2024-01-24T23:27:10.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 10 monclient: dump: 2024-01-24T23:27:10.282 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:10.283 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:10.283 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:10.283 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:10.283 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:10.283 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:10.283 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:10.283 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:10.283 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:10.283 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:10.283 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:10.283 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:10.283 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:10.283 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:10.283 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:10.283 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:10.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:10.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:10.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:10.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3f5fb640 1 -- 172.21.15.132:0/1224532080 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe18005090 con 0x7fbe3809bdc0 2024-01-24T23:27:10.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 1 -- 172.21.15.132:0/1224532080 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fbe300043e0 con 0x7fbe3809bdc0 2024-01-24T23:27:10.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe29ffb640 1 -- 172.21.15.132:0/1224532080 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(210..210 src has 1..210) v4 ==== 13952+0+0 (unknown 4159688877 0 0) 0x7fbe30046420 con 0x7fbe3809bdc0 2024-01-24T23:27:10.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.262+0000 7fbe3d397640 10 monclient: get_auth_request con 0x7fbe1c036f70 auth_method 0 2024-01-24T23:27:10.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.270+0000 7fbe29ffb640 1 -- 172.21.15.132:0/1224532080 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fbe30014ec0 con 0x7fbe3809bdc0 2024-01-24T23:27:10.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.270+0000 7fbe29ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:10.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.270+0000 7fbe29ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:10.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.414+0000 7fbe3f5fb640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:10.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.414+0000 7fbe3f5fb640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:10.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.414+0000 7fbe3f5fb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:10.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:10.414+0000 7fbe3f5fb640 1 -- 172.21.15.132:0/1224532080 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fbe180058c0 con 0x7fbe3809bdc0 2024-01-24T23:27:11.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.022+0000 7fbe29ffb640 1 -- 172.21.15.132:0/1224532080 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}]=0 added bucket h31 type host to location {rack=r1} v211) v1 ==== 168+0+0 (unknown 3780425413 0 0) 0x7fbe30013330 con 0x7fbe3809bdc0 2024-01-24T23:27:11.023 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.022+0000 7fbe29ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:11.024 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.022+0000 7fbe29ffb640 10 monclient: _finish_command 2 = system:0 added bucket h31 type host to location {rack=r1} 2024-01-24T23:27:11.136 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.134+0000 7fbe3f5fb640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:11.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.134+0000 7fbe3f5fb640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:11.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.134+0000 7fbe3f5fb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:11.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.134+0000 7fbe3f5fb640 1 -- 172.21.15.132:0/1224532080 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7fbe18002220 con 0x7fbe3809bdc0 2024-01-24T23:27:11.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.134+0000 7fbe29ffb640 1 -- 172.21.15.132:0/1224532080 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h31' already exists v211) v1 ==== 147+0+0 (unknown 2314532081 0 0) 0x7fbe30014ab0 con 0x7fbe3809bdc0 2024-01-24T23:27:11.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.134+0000 7fbe29ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:11.137 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.134+0000 7fbe29ffb640 10 monclient: _finish_command 3 = system:0 bucket 'h31' already exists 2024-01-24T23:27:11.137 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h31' already exists 2024-01-24T23:27:11.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.138+0000 7fbe3f5fb640 1 -- 172.21.15.132:0/1224532080 >> v1:172.21.15.132:6812/1835401922 conn(0x7fbe1c036f70 legacy=0x7fbe1c039430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:11.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.138+0000 7fbe3f5fb640 10 monclient: shutdown 2024-01-24T23:27:11.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.138+0000 7fbe3f5fb640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:11.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.138+0000 7fbe3f5fb640 1 -- 172.21.15.132:0/1224532080 >> v1:172.21.15.132:6791/0 conn(0x7fbe3809bdc0 legacy=0x7fbe381d3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:11.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.138+0000 7fbe3f5fb640 1 -- 172.21.15.132:0/1224532080 shutdown_connections 2024-01-24T23:27:11.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.138+0000 7fbe3f5fb640 1 -- 172.21.15.132:0/1224532080 >> 172.21.15.132:0/1224532080 conn(0x7fbe380a1bc0 msgr2=0x7fbe381ba6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:11.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.138+0000 7fbe3f5fb640 1 -- 172.21.15.132:0/1224532080 shutdown_connections 2024-01-24T23:27:11.140 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.138+0000 7fbe3f5fb640 1 -- 172.21.15.132:0/1224532080 wait complete. 2024-01-24T23:27:11.154 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:11.154 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush add-bucket h32 host rack=r1 2024-01-24T23:27:11.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.270+0000 7f997dc84640 10 monclient: get_monmap_and_config 2024-01-24T23:27:11.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.270+0000 7f997dc84640 10 monclient: build_initial_monmap 2024-01-24T23:27:11.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 10 monclient: monmap: 2024-01-24T23:27:11.275 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:11.275 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:11.275 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:11.276804+0000 2024-01-24T23:27:11.275 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:11.276804+0000 2024-01-24T23:27:11.275 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:11.275 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:11.275 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:11.275 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:11.275 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:11.275 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:11.276 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:11.276 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:11.276 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:11.276 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:11.276 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:11.276 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:11.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 1 Processor -- start 2024-01-24T23:27:11.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 1 -- start start 2024-01-24T23:27:11.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 10 monclient: init 2024-01-24T23:27:11.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:11.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 10 monclient: _add_conns ranks=[2,1,0,4,7,6,8,5,3] 2024-01-24T23:27:11.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 10 monclient(hunting): picked mon.noname-b con 0x7f99780a0ad0 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:11.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 10 monclient(hunting): picked mon.noname-f con 0x7f997809ee80 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:11.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 10 monclient(hunting): picked mon.noname-a con 0x7f997809b060 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:11.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9978064700 con 0x7f997809b060 2024-01-24T23:27:11.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f997809cb80 con 0x7f997809ee80 2024-01-24T23:27:11.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99780a0eb0 con 0x7f99780a0ad0 2024-01-24T23:27:11.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:11.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 10 monclient(hunting): authenticate will time out at 1318.601523s 2024-01-24T23:27:11.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f9976ffd640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f997809ee80 legacy=0x7f997809a950 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:27:11.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99777fe640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f99780a0ad0 legacy=0x7f997809a240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:11.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f9977fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f997809b060 0x7f997813c270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.132:59492/0 (socket says 172.21.15.132:59492) 2024-01-24T23:27:11.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f9977fff640 1 -- 172.21.15.132:0/4262647736 learned_addr learned my addr 172.21.15.132:0/4262647736 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:11.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 1 -- 172.21.15.132:0/4262647736 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2651377772 0 0) 0x7f9978064700 con 0x7f997809b060 2024-01-24T23:27:11.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:11.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:11.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:11.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 10 monclient(hunting): my global_id is 94878 2024-01-24T23:27:11.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 1 -- 172.21.15.132:0/4262647736 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9960003530 con 0x7f997809b060 2024-01-24T23:27:11.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 1 -- 172.21.15.132:0/4262647736 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3655142549 0 0) 0x7f9960003530 con 0x7f997809b060 2024-01-24T23:27:11.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 1 -- 172.21.15.132:0/4262647736 >> v1:172.21.15.115:6790/0 conn(0x7f99780a0ad0 legacy=0x7f997809a240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:11.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 1 -- 172.21.15.132:0/4262647736 >> v1:172.21.15.132:6789/0 conn(0x7f997809ee80 legacy=0x7f997809a950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:11.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 10 monclient: _finish_hunting 0 2024-01-24T23:27:11.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 1 monclient: found mon.noname-a 2024-01-24T23:27:11.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:11.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:27:11.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 1 -- 172.21.15.132:0/4262647736 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997809f280 con 0x7f997809b060 2024-01-24T23:27:11.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 10 monclient: _finish_auth 0 2024-01-24T23:27:11.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 10 monclient: _check_auth_tickets 2024-01-24T23:27:11.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:11.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 1 -- 172.21.15.132:0/4262647736 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9970001b40 con 0x7f997809b060 2024-01-24T23:27:11.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:11.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:27:11.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 10 monclient: dump: 2024-01-24T23:27:11.286 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:11.286 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:11.286 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:11.286 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:11.287 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:11.287 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:11.287 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:11.287 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:11.287 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:11.287 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:11.287 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:11.287 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:11.287 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:11.287 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:11.287 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:11.287 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:11.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 5 monclient: authenticate success, global_id 94878 2024-01-24T23:27:11.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:11.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 1 -- 172.21.15.132:0/4262647736 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9970001df0 con 0x7f997809b060 2024-01-24T23:27:11.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:11.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:11.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 4 set_mon_vals no callback set 2024-01-24T23:27:11.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 1 -- 172.21.15.132:0/4262647736 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9970004f70 con 0x7f997809b060 2024-01-24T23:27:11.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:11.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:27:11.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f99767fc640 10 monclient: dump: 2024-01-24T23:27:11.288 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:11.288 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:11.288 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:11.288 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:11.288 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:11.288 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:11.289 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:11.289 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:11.289 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:11.289 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:11.289 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:11.289 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:11.289 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:11.289 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:11.289 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:11.289 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:11.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 10 monclient: shutdown 2024-01-24T23:27:11.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:11.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 1 -- 172.21.15.132:0/4262647736 >> v1:172.21.15.115:6789/0 conn(0x7f997809b060 legacy=0x7f997813c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:11.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 1 -- 172.21.15.132:0/4262647736 shutdown_connections 2024-01-24T23:27:11.289 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 1 -- 172.21.15.132:0/4262647736 >> 172.21.15.132:0/4262647736 conn(0x7f99780a1bc0 msgr2=0x7f99780a1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:11.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 1 -- 172.21.15.132:0/4262647736 shutdown_connections 2024-01-24T23:27:11.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.274+0000 7f997dc84640 1 -- 172.21.15.132:0/4262647736 wait complete. 2024-01-24T23:27:11.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 10 monclient: build_initial_monmap 2024-01-24T23:27:11.290 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 10 monclient: monmap: 2024-01-24T23:27:11.290 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:11.290 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:11.290 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:11.290 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:11.290 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:11.290 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:11.290 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:11.290 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:11.290 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:11.290 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:11.291 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:11.291 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:11.291 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:11.291 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:11.291 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:11.291 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:11.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 1 Processor -- start 2024-01-24T23:27:11.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 1 -- start start 2024-01-24T23:27:11.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 10 monclient: init 2024-01-24T23:27:11.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:11.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 10 monclient: _add_conns ranks=[6,8,7,4,0,2,5,1,3] 2024-01-24T23:27:11.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 10 monclient(hunting): picked mon.noname-d con 0x7f997809b060 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:11.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 10 monclient(hunting): picked mon.noname-e con 0x7f997809ee80 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:11.291 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 10 monclient(hunting): picked mon.noname-i con 0x7f99780a0ad0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:11.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99781b3050 con 0x7f997809b060 2024-01-24T23:27:11.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99781b18c0 con 0x7f99780a0ad0 2024-01-24T23:27:11.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99781b22f0 con 0x7f997809ee80 2024-01-24T23:27:11.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:11.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 10 monclient(hunting): authenticate will time out at 1318.605288s 2024-01-24T23:27:11.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f9977fff640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f99780a0ad0 legacy=0x7f99781cfad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:11.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f99777fe640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f997809b060 0x7f99781b65a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:49618/0 (socket says 172.21.15.132:49618) 2024-01-24T23:27:11.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f99777fe640 1 -- 172.21.15.132:0/1878025520 learned_addr learned my addr 172.21.15.132:0/1878025520 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:11.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 1 -- 172.21.15.132:0/1878025520 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 372497214 0 0) 0x7f99781b3050 con 0x7f997809b060 2024-01-24T23:27:11.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:11.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:11.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:11.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 10 monclient(hunting): my global_id is 6630 2024-01-24T23:27:11.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 1 -- 172.21.15.132:0/1878025520 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9964003530 con 0x7f997809b060 2024-01-24T23:27:11.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 1 -- 172.21.15.132:0/1878025520 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2223629927 0 0) 0x7f99781b22f0 con 0x7f997809ee80 2024-01-24T23:27:11.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:11.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:11.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:11.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 10 monclient(hunting): my global_id is 85751 2024-01-24T23:27:11.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 1 -- 172.21.15.132:0/1878025520 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99781b3050 con 0x7f997809ee80 2024-01-24T23:27:11.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 1 -- 172.21.15.132:0/1878025520 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2546338679 0 0) 0x7f9964003530 con 0x7f997809b060 2024-01-24T23:27:11.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 1 -- 172.21.15.132:0/1878025520 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f99781b22f0 con 0x7f997809b060 2024-01-24T23:27:11.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 1 -- 172.21.15.132:0/1878025520 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2104960769 0 0) 0x7f99781b3050 con 0x7f997809ee80 2024-01-24T23:27:11.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 1 -- 172.21.15.132:0/1878025520 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9964003530 con 0x7f997809ee80 2024-01-24T23:27:11.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 1 -- 172.21.15.132:0/1878025520 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f996c004270 con 0x7f997809ee80 2024-01-24T23:27:11.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:11.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:27:11.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 10 monclient(hunting): dump: 2024-01-24T23:27:11.294 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:11.294 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:11.294 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:11.294 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:11.294 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:11.294 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:11.294 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:11.294 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:11.294 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:11.294 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:11.294 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:11.294 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:11.294 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:11.295 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:11.295 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:11.295 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:11.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:11.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 1 -- 172.21.15.132:0/1878025520 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f99680054b0 con 0x7f997809b060 2024-01-24T23:27:11.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:11.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:11.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 10 monclient(hunting): dump: 2024-01-24T23:27:11.295 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:11.295 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:11.295 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:11.295 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:11.295 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:11.295 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:11.295 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:11.296 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:11.296 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:11.296 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:11.296 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:11.296 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:11.296 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:11.296 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:11.296 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:11.296 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:11.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 1 -- 172.21.15.132:0/1878025520 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3314678632 0 0) 0x7f99781b22f0 con 0x7f997809b060 2024-01-24T23:27:11.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 1 -- 172.21.15.132:0/1878025520 >> v1:172.21.15.115:6793/0 conn(0x7f997809ee80 legacy=0x7f99781cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:11.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 1 -- 172.21.15.132:0/1878025520 >> v1:172.21.15.132:6792/0 conn(0x7f99780a0ad0 legacy=0x7f99781cfad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:11.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 10 monclient: _finish_hunting 0 2024-01-24T23:27:11.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 1 monclient: found mon.d 2024-01-24T23:27:11.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:11.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:11.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 1 -- 172.21.15.132:0/1878025520 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99781b27b0 con 0x7f997809b060 2024-01-24T23:27:11.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 10 monclient: _finish_auth 0 2024-01-24T23:27:11.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 10 monclient: _check_auth_tickets 2024-01-24T23:27:11.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:11.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 5 monclient: authenticate success, global_id 6630 2024-01-24T23:27:11.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 10 monclient: _renew_subs 2024-01-24T23:27:11.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:11.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 1 -- 172.21.15.132:0/1878025520 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99781b2a90 con 0x7f997809b060 2024-01-24T23:27:11.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 10 monclient: _renew_subs 2024-01-24T23:27:11.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:11.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 1 -- 172.21.15.132:0/1878025520 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f99781b6cb0 con 0x7f997809b060 2024-01-24T23:27:11.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 1 -- 172.21.15.132:0/1878025520 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9968004000 con 0x7f997809b060 2024-01-24T23:27:11.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:11.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 1 -- 172.21.15.132:0/1878025520 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9968004530 con 0x7f997809b060 2024-01-24T23:27:11.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:11.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:11.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f9975ffb640 4 set_mon_vals no callback set 2024-01-24T23:27:11.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f995ffff640 10 monclient: dump: 2024-01-24T23:27:11.298 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:11.298 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:11.298 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:11.298 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:11.298 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:11.298 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:11.298 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:11.298 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:11.298 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:11.298 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:11.299 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:11.299 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:11.299 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:11.299 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:11.299 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:11.299 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:11.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:11.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:11.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:11.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.278+0000 7f997dc84640 1 -- 172.21.15.132:0/1878025520 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9958005090 con 0x7f997809b060 2024-01-24T23:27:11.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.282+0000 7f995ffff640 1 -- 172.21.15.132:0/1878025520 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f9968004690 con 0x7f997809b060 2024-01-24T23:27:11.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.282+0000 7f995ffff640 1 -- 172.21.15.132:0/1878025520 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(211..211 src has 1..211) v4 ==== 14165+0+0 (unknown 1729985917 0 0) 0x7f9968045fa0 con 0x7f997809b060 2024-01-24T23:27:11.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.282+0000 7f9976ffd640 10 monclient: get_auth_request con 0x7f9964037670 auth_method 0 2024-01-24T23:27:11.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.294+0000 7f995ffff640 1 -- 172.21.15.132:0/1878025520 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f9968014970 con 0x7f997809b060 2024-01-24T23:27:11.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.294+0000 7f995ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:11.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.294+0000 7f995ffff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:11.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.454+0000 7f997dc84640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:11.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.454+0000 7f997dc84640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:11.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.454+0000 7f997dc84640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:11.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:11.454+0000 7f997dc84640 1 -- 172.21.15.132:0/1878025520 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f9958005340 con 0x7f997809b060 2024-01-24T23:27:12.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.050+0000 7f995ffff640 1 -- 172.21.15.132:0/1878025520 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}]=0 added bucket h32 type host to location {rack=r1} v212) v1 ==== 168+0+0 (unknown 14958377 0 0) 0x7f9968014560 con 0x7f997809b060 2024-01-24T23:27:12.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.050+0000 7f995ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:12.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.050+0000 7f995ffff640 10 monclient: _finish_command 2 = system:0 added bucket h32 type host to location {rack=r1} 2024-01-24T23:27:12.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.154+0000 7f997dc84640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:12.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.154+0000 7f997dc84640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:12.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.154+0000 7f997dc84640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:12.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.154+0000 7f997dc84640 1 -- 172.21.15.132:0/1878025520 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]} v 0) v1 -- 0x7f9958005a80 con 0x7f997809b060 2024-01-24T23:27:12.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.154+0000 7f995ffff640 1 -- 172.21.15.132:0/1878025520 <== mon.6 v1:172.21.15.115:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h32' already exists v212) v1 ==== 147+0+0 (unknown 3347632404 0 0) 0x7f9968014740 con 0x7f997809b060 2024-01-24T23:27:12.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.154+0000 7f995ffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}] 2024-01-24T23:27:12.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.154+0000 7f995ffff640 10 monclient: _finish_command 3 = system:0 bucket 'h32' already exists 2024-01-24T23:27:12.158 INFO:tasks.workunit.client.0.smithi132.stderr:bucket 'h32' already exists 2024-01-24T23:27:12.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.158+0000 7f997dc84640 1 -- 172.21.15.132:0/1878025520 >> v1:172.21.15.132:6812/1835401922 conn(0x7f9964037670 legacy=0x7f9964039b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:12.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.158+0000 7f997dc84640 10 monclient: shutdown 2024-01-24T23:27:12.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.158+0000 7f997dc84640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:12.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.158+0000 7f997dc84640 1 -- 172.21.15.132:0/1878025520 >> v1:172.21.15.115:6792/0 conn(0x7f997809b060 legacy=0x7f99781b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:12.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.158+0000 7f997dc84640 1 -- 172.21.15.132:0/1878025520 shutdown_connections 2024-01-24T23:27:12.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.158+0000 7f997dc84640 1 -- 172.21.15.132:0/1878025520 >> 172.21.15.132:0/1878025520 conn(0x7f99780a1bc0 msgr2=0x7f99781ba4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:12.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.158+0000 7f997dc84640 1 -- 172.21.15.132:0/1878025520 shutdown_connections 2024-01-24T23:27:12.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.158+0000 7f997dc84640 1 -- 172.21.15.132:0/1878025520 wait complete. 2024-01-24T23:27:12.175 INFO:tasks.workunit.client.0.smithi132.stderr:++ seq 1 32 2024-01-24T23:27:12.176 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:12.176 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h1 2024-01-24T23:27:12.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a83c4d640 10 monclient: get_monmap_and_config 2024-01-24T23:27:12.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a83c4d640 10 monclient: build_initial_monmap 2024-01-24T23:27:12.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a83c4d640 10 monclient: monmap: 2024-01-24T23:27:12.292 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:12.292 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:12.292 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:12.293966+0000 2024-01-24T23:27:12.292 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:12.293966+0000 2024-01-24T23:27:12.292 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:12.292 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:12.292 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:12.292 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:12.292 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:12.293 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:12.293 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:12.293 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:12.293 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:12.293 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:12.293 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:12.293 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:12.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a83c4d640 1 Processor -- start 2024-01-24T23:27:12.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a83c4d640 1 -- start start 2024-01-24T23:27:12.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a83c4d640 10 monclient: init 2024-01-24T23:27:12.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a83c4d640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:12.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a83c4d640 10 monclient: _add_conns ranks=[6,1,5,3,4,8,7,2,0] 2024-01-24T23:27:12.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a83c4d640 10 monclient(hunting): picked mon.noname-d con 0x7f0a7c133b50 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:12.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a83c4d640 10 monclient(hunting): picked mon.noname-f con 0x7f0a7c09ec20 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:12.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a83c4d640 10 monclient(hunting): picked mon.noname-h con 0x7f0a7c13a1d0 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:12.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a83c4d640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a7c064700 con 0x7f0a7c09ec20 2024-01-24T23:27:12.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a83c4d640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a7c09d9f0 con 0x7f0a7c13a1d0 2024-01-24T23:27:12.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a83c4d640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a7c09dc60 con 0x7f0a7c133b50 2024-01-24T23:27:12.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a83c4d640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:12.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a83c4d640 10 monclient(hunting): authenticate will time out at 1319.618859s 2024-01-24T23:27:12.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a811e8640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f0a7c09ec20 legacy=0x7f0a7c138ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:27:12.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a821ea640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f0a7c13a1d0 0x7f0a7c13c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:59722/0 (socket says 172.21.15.132:59722) 2024-01-24T23:27:12.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a821ea640 1 -- 172.21.15.132:0/2613751760 learned_addr learned my addr 172.21.15.132:0/2613751760 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:12.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 1 -- 172.21.15.132:0/2613751760 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140321223 0 0) 0x7f0a7c09d9f0 con 0x7f0a7c13a1d0 2024-01-24T23:27:12.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 10 monclient(hunting): _negotiate 2024-01-24T23:27:12.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:12.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:12.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 10 monclient(hunting): my global_id is 6467 2024-01-24T23:27:12.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 1 -- 172.21.15.132:0/2613751760 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a68003530 con 0x7f0a7c13a1d0 2024-01-24T23:27:12.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 1 -- 172.21.15.132:0/2613751760 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3245778469 0 0) 0x7f0a7c09dc60 con 0x7f0a7c133b50 2024-01-24T23:27:12.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 10 monclient(hunting): _negotiate 2024-01-24T23:27:12.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:12.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:12.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 10 monclient(hunting): my global_id is 6639 2024-01-24T23:27:12.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 1 -- 172.21.15.132:0/2613751760 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a7c09d9f0 con 0x7f0a7c133b50 2024-01-24T23:27:12.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 1 -- 172.21.15.132:0/2613751760 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3745449466 0 0) 0x7f0a68003530 con 0x7f0a7c13a1d0 2024-01-24T23:27:12.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 1 -- 172.21.15.132:0/2613751760 >> v1:172.21.15.115:6792/0 conn(0x7f0a7c133b50 legacy=0x7f0a7c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:12.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 1 -- 172.21.15.132:0/2613751760 >> v1:172.21.15.132:6789/0 conn(0x7f0a7c09ec20 legacy=0x7f0a7c138ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:12.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 10 monclient: _finish_hunting 0 2024-01-24T23:27:12.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 1 monclient: found mon.noname-h 2024-01-24T23:27:12.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:12.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:27:12.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 1 -- 172.21.15.132:0/2613751760 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c09df40 con 0x7f0a7c13a1d0 2024-01-24T23:27:12.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 10 monclient: _finish_auth 0 2024-01-24T23:27:12.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 10 monclient: _check_auth_tickets 2024-01-24T23:27:12.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:12.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 1 -- 172.21.15.132:0/2613751760 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0a74001b40 con 0x7f0a7c13a1d0 2024-01-24T23:27:12.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a83c4d640 5 monclient: authenticate success, global_id 6467 2024-01-24T23:27:12.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a83c4d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:12.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:12.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:27:12.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 10 monclient: dump: 2024-01-24T23:27:12.299 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:12.299 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:12.299 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:12.299 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:12.299 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:12.299 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:12.299 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:12.299 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:12.299 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:12.299 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:12.300 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:12.300 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:12.300 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:12.300 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:12.300 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:12.300 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:12.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a83c4d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:12.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 1 -- 172.21.15.132:0/2613751760 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0a74001d60 con 0x7f0a7c13a1d0 2024-01-24T23:27:12.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:12.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a83c4d640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:12.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a83c4d640 4 set_mon_vals no callback set 2024-01-24T23:27:12.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 1 -- 172.21.15.132:0/2613751760 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0a74004f70 con 0x7f0a7c13a1d0 2024-01-24T23:27:12.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:12.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:12.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.290+0000 7f0a809e7640 10 monclient: dump: 2024-01-24T23:27:12.301 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:12.301 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:12.301 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:12.301 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:12.301 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:12.301 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:12.301 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:12.301 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:12.301 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:12.301 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:12.301 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:12.301 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:12.301 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:12.301 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:12.301 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:12.302 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:12.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 10 monclient: shutdown 2024-01-24T23:27:12.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:12.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 1 -- 172.21.15.132:0/2613751760 >> v1:172.21.15.132:6791/0 conn(0x7f0a7c13a1d0 legacy=0x7f0a7c13c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:12.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 1 -- 172.21.15.132:0/2613751760 shutdown_connections 2024-01-24T23:27:12.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 1 -- 172.21.15.132:0/2613751760 >> 172.21.15.132:0/2613751760 conn(0x7f0a7c0a5a10 msgr2=0x7f0a7c0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:12.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 1 -- 172.21.15.132:0/2613751760 shutdown_connections 2024-01-24T23:27:12.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 1 -- 172.21.15.132:0/2613751760 wait complete. 2024-01-24T23:27:12.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 10 monclient: build_initial_monmap 2024-01-24T23:27:12.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 10 monclient: monmap: 2024-01-24T23:27:12.302 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:12.302 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:12.302 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:12.302 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:12.302 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:12.303 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:12.303 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:12.303 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:12.303 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:12.303 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:12.303 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:12.303 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:12.303 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:12.303 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:12.303 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:12.303 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:12.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 1 Processor -- start 2024-01-24T23:27:12.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 1 -- start start 2024-01-24T23:27:12.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 10 monclient: init 2024-01-24T23:27:12.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:12.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 10 monclient: _add_conns ranks=[5,4,6,1,3,2,8,0,7] 2024-01-24T23:27:12.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 10 monclient(hunting): picked mon.noname-h con 0x7f0a7c09ec20 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:12.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 10 monclient(hunting): picked mon.noname-c con 0x7f0a7c133b50 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:12.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 10 monclient(hunting): picked mon.noname-d con 0x7f0a7c13a1d0 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:12.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a7c06f740 con 0x7f0a7c133b50 2024-01-24T23:27:12.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a7c133920 con 0x7f0a7c09ec20 2024-01-24T23:27:12.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a7c0a13b0 con 0x7f0a7c13a1d0 2024-01-24T23:27:12.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:12.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 10 monclient(hunting): authenticate will time out at 1319.622117s 2024-01-24T23:27:12.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a819e9640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f0a7c09ec20 0x7f0a7c12c720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:59732/0 (socket says 172.21.15.132:59732) 2024-01-24T23:27:12.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a819e9640 1 -- 172.21.15.132:0/1043288788 learned_addr learned my addr 172.21.15.132:0/1043288788 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:12.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 1 -- 172.21.15.132:0/1043288788 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1081842862 0 0) 0x7f0a7c133920 con 0x7f0a7c09ec20 2024-01-24T23:27:12.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:12.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:12.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:12.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 10 monclient(hunting): my global_id is 6476 2024-01-24T23:27:12.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 1 -- 172.21.15.132:0/1043288788 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a60003530 con 0x7f0a7c09ec20 2024-01-24T23:27:12.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 1 -- 172.21.15.132:0/1043288788 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1360807659 0 0) 0x7f0a60003530 con 0x7f0a7c09ec20 2024-01-24T23:27:12.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 1 -- 172.21.15.132:0/1043288788 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0a7c133920 con 0x7f0a7c09ec20 2024-01-24T23:27:12.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 1 -- 172.21.15.132:0/1043288788 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0a78001b20 con 0x7f0a7c09ec20 2024-01-24T23:27:12.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:12.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:27:12.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 10 monclient(hunting): dump: 2024-01-24T23:27:12.308 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:12.309 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:12.309 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:12.309 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:12.309 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:12.309 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:12.309 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:12.309 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:12.309 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:12.309 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:12.309 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:12.309 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:12.309 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:12.309 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:12.309 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:12.309 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:12.309 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:12.309 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 1 -- 172.21.15.132:0/1043288788 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 417941397 0 0) 0x7f0a7c133920 con 0x7f0a7c09ec20 2024-01-24T23:27:12.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 1 -- 172.21.15.132:0/1043288788 >> v1:172.21.15.115:6792/0 conn(0x7f0a7c13a1d0 legacy=0x7f0a7c1d3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:12.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 1 -- 172.21.15.132:0/1043288788 >> v1:172.21.15.115:6791/0 conn(0x7f0a7c133b50 legacy=0x7f0a7c1d0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:12.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 10 monclient: _finish_hunting 0 2024-01-24T23:27:12.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 1 monclient: found mon.h 2024-01-24T23:27:12.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:12.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:12.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 1 -- 172.21.15.132:0/1043288788 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c071aa0 con 0x7f0a7c09ec20 2024-01-24T23:27:12.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 10 monclient: _finish_auth 0 2024-01-24T23:27:12.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 10 monclient: _check_auth_tickets 2024-01-24T23:27:12.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:12.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 5 monclient: authenticate success, global_id 6476 2024-01-24T23:27:12.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 10 monclient: _renew_subs 2024-01-24T23:27:12.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:12.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 1 -- 172.21.15.132:0/1043288788 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a7c0704c0 con 0x7f0a7c09ec20 2024-01-24T23:27:12.310 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 10 monclient: _renew_subs 2024-01-24T23:27:12.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:12.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 1 -- 172.21.15.132:0/1043288788 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a7c12d000 con 0x7f0a7c09ec20 2024-01-24T23:27:12.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 1 -- 172.21.15.132:0/1043288788 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0a780038d0 con 0x7f0a7c09ec20 2024-01-24T23:27:12.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:12.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 1 -- 172.21.15.132:0/1043288788 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0a78003e00 con 0x7f0a7c09ec20 2024-01-24T23:27:12.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:12.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a73fff640 4 set_mon_vals no callback set 2024-01-24T23:27:12.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:12.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 10 monclient: dump: 2024-01-24T23:27:12.311 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:12.311 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:12.311 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:12.311 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:12.311 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:12.311 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:12.311 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:12.312 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:12.312 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:12.312 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:12.312 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:12.312 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:12.312 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:12.312 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:12.312 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:12.312 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:12.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:12.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:12.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:12.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a83c4d640 1 -- 172.21.15.132:0/1043288788 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a64005090 con 0x7f0a7c09ec20 2024-01-24T23:27:12.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.294+0000 7f0a727fc640 1 -- 172.21.15.132:0/1043288788 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f0a78003f60 con 0x7f0a7c09ec20 2024-01-24T23:27:12.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.298+0000 7f0a727fc640 1 -- 172.21.15.132:0/1043288788 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(212..212 src has 1..212) v4 ==== 14378+0+0 (unknown 3386625206 0 0) 0x7f0a78046150 con 0x7f0a7c09ec20 2024-01-24T23:27:12.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.298+0000 7f0a811e8640 10 monclient: get_auth_request con 0x7f0a60036f20 auth_method 0 2024-01-24T23:27:12.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.302+0000 7f0a727fc640 1 -- 172.21.15.132:0/1043288788 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f0a78014810 con 0x7f0a7c09ec20 2024-01-24T23:27:12.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.302+0000 7f0a727fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:12.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.302+0000 7f0a727fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:12.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.462+0000 7f0a83c4d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h1"}] 2024-01-24T23:27:12.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.462+0000 7f0a83c4d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h1"}] 2024-01-24T23:27:12.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.462+0000 7f0a83c4d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:12.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:12.462+0000 7f0a83c4d640 1 -- 172.21.15.132:0/1043288788 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h1"} v 0) v1 -- 0x7f0a64005340 con 0x7f0a7c09ec20 2024-01-24T23:27:12.553 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~68s 2024-01-24T23:27:13.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.046+0000 7f0a727fc640 1 -- 172.21.15.132:0/1043288788 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h1"}]=0 removed item id -19 name 'h1' from crush map v213) v1 ==== 118+0+0 (unknown 711994448 0 0) 0x7f0a78012eb0 con 0x7f0a7c09ec20 2024-01-24T23:27:13.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.046+0000 7f0a727fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h1"}] 2024-01-24T23:27:13.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.046+0000 7f0a727fc640 10 monclient: _finish_command 2 = system:0 removed item id -19 name 'h1' from crush map 2024-01-24T23:27:13.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.158+0000 7f0a83c4d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h1"}] 2024-01-24T23:27:13.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.158+0000 7f0a83c4d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h1"}] 2024-01-24T23:27:13.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.158+0000 7f0a83c4d640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:13.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.158+0000 7f0a83c4d640 1 -- 172.21.15.132:0/1043288788 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h1"} v 0) v1 -- 0x7f0a64002220 con 0x7f0a7c09ec20 2024-01-24T23:27:13.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.158+0000 7f0a727fc640 1 -- 172.21.15.132:0/1043288788 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h1"}]=0 device 'h1' does not appear in the crush map v213) v1 ==== 118+0+0 (unknown 381910933 0 0) 0x7f0a78014630 con 0x7f0a7c09ec20 2024-01-24T23:27:13.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.158+0000 7f0a727fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h1"}] 2024-01-24T23:27:13.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.158+0000 7f0a727fc640 10 monclient: _finish_command 3 = system:0 device 'h1' does not appear in the crush map 2024-01-24T23:27:13.160 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h1' does not appear in the crush map 2024-01-24T23:27:13.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.158+0000 7f0a83c4d640 1 -- 172.21.15.132:0/1043288788 >> v1:172.21.15.132:6812/1835401922 conn(0x7f0a60036f20 legacy=0x7f0a600393e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:13.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.158+0000 7f0a83c4d640 10 monclient: shutdown 2024-01-24T23:27:13.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.158+0000 7f0a83c4d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:13.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.158+0000 7f0a83c4d640 1 -- 172.21.15.132:0/1043288788 >> v1:172.21.15.132:6791/0 conn(0x7f0a7c09ec20 legacy=0x7f0a7c12c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:13.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.158+0000 7f0a83c4d640 1 -- 172.21.15.132:0/1043288788 shutdown_connections 2024-01-24T23:27:13.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.158+0000 7f0a83c4d640 1 -- 172.21.15.132:0/1043288788 >> 172.21.15.132:0/1043288788 conn(0x7f0a7c0a5a10 msgr2=0x7f0a7c09cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:13.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.158+0000 7f0a83c4d640 1 -- 172.21.15.132:0/1043288788 shutdown_connections 2024-01-24T23:27:13.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.162+0000 7f0a83c4d640 1 -- 172.21.15.132:0/1043288788 wait complete. 2024-01-24T23:27:13.177 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:13.177 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h2 2024-01-24T23:27:13.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc905f640 10 monclient: get_monmap_and_config 2024-01-24T23:27:13.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc905f640 10 monclient: build_initial_monmap 2024-01-24T23:27:13.288 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc905f640 10 monclient: monmap: 2024-01-24T23:27:13.288 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:13.288 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:13.288 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:13.290218+0000 2024-01-24T23:27:13.288 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:13.290218+0000 2024-01-24T23:27:13.288 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:13.288 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:13.289 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:13.289 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:13.289 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:13.289 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:13.289 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:13.289 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:13.289 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:13.289 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:13.289 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:13.289 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:13.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc905f640 1 Processor -- start 2024-01-24T23:27:13.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc905f640 1 -- start start 2024-01-24T23:27:13.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc905f640 10 monclient: init 2024-01-24T23:27:13.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc905f640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:13.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc905f640 10 monclient: _add_conns ranks=[7,2,6,8,4,0,5,3,1] 2024-01-24T23:27:13.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc905f640 10 monclient(hunting): picked mon.noname-i con 0x7f7dc4135e10 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:13.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc905f640 10 monclient(hunting): picked mon.noname-b con 0x7f7dc41361f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:13.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc905f640 10 monclient(hunting): picked mon.noname-d con 0x7f7dc41365f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:13.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc905f640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc4064700 con 0x7f7dc41361f0 2024-01-24T23:27:13.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc905f640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc409c3e0 con 0x7f7dc41365f0 2024-01-24T23:27:13.292 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc905f640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc40a4f90 con 0x7f7dc4135e10 2024-01-24T23:27:13.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc905f640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:13.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc905f640 10 monclient(hunting): authenticate will time out at 1320.614861s 2024-01-24T23:27:13.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc2d9d640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f7dc4135e10 legacy=0x7f7dc409bcd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:13.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc259c640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f7dc41361f0 legacy=0x7f7dc413bed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:13.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc359e640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f7dc41365f0 0x7f7dc413f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:49640/0 (socket says 172.21.15.132:49640) 2024-01-24T23:27:13.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc359e640 1 -- 172.21.15.132:0/575972106 learned_addr learned my addr 172.21.15.132:0/575972106 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:13.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc1d9b640 1 -- 172.21.15.132:0/575972106 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2965150566 0 0) 0x7f7dc409c3e0 con 0x7f7dc41365f0 2024-01-24T23:27:13.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc1d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:27:13.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc1d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:13.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:13.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc1d9b640 10 monclient(hunting): my global_id is 6657 2024-01-24T23:27:13.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.286+0000 7f7dc1d9b640 1 -- 172.21.15.132:0/575972106 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7db0003530 con 0x7f7dc41365f0 2024-01-24T23:27:13.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc1d9b640 1 -- 172.21.15.132:0/575972106 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1174309799 0 0) 0x7f7db0003530 con 0x7f7dc41365f0 2024-01-24T23:27:13.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc1d9b640 1 -- 172.21.15.132:0/575972106 >> v1:172.21.15.132:6792/0 conn(0x7f7dc4135e10 legacy=0x7f7dc409bcd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:13.293 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc1d9b640 1 -- 172.21.15.132:0/575972106 >> v1:172.21.15.115:6790/0 conn(0x7f7dc41361f0 legacy=0x7f7dc413bed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:13.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc1d9b640 10 monclient: _finish_hunting 0 2024-01-24T23:27:13.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc1d9b640 1 monclient: found mon.noname-d 2024-01-24T23:27:13.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:13.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc1d9b640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:27:13.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc1d9b640 1 -- 172.21.15.132:0/575972106 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc409b010 con 0x7f7dc41365f0 2024-01-24T23:27:13.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc1d9b640 10 monclient: _finish_auth 0 2024-01-24T23:27:13.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc1d9b640 10 monclient: _check_auth_tickets 2024-01-24T23:27:13.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc1d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:13.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc1d9b640 1 -- 172.21.15.132:0/575972106 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7db40017c0 con 0x7f7dc41365f0 2024-01-24T23:27:13.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 5 monclient: authenticate success, global_id 6657 2024-01-24T23:27:13.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:13.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc1d9b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:27:13.294 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc1d9b640 10 monclient: dump: 2024-01-24T23:27:13.294 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:13.294 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:13.294 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:13.295 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:13.295 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:13.295 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:13.295 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:13.295 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:13.295 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:13.295 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:13.295 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:13.295 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:13.295 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:13.295 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:13.295 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:13.295 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:13.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:13.295 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc1d9b640 1 -- 172.21.15.132:0/575972106 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7db4001ae0 con 0x7f7dc41365f0 2024-01-24T23:27:13.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc1d9b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:13.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:13.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc1d9b640 1 -- 172.21.15.132:0/575972106 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7db4004bf0 con 0x7f7dc41365f0 2024-01-24T23:27:13.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 4 set_mon_vals no callback set 2024-01-24T23:27:13.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:13.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc1d9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:13.296 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc1d9b640 10 monclient: dump: 2024-01-24T23:27:13.296 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:13.296 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:13.296 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:13.296 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:13.296 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:13.296 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:13.296 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:13.297 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:13.297 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:13.297 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:13.297 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:13.297 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:13.297 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:13.297 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:13.297 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:13.297 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:13.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 10 monclient: shutdown 2024-01-24T23:27:13.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:13.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 1 -- 172.21.15.132:0/575972106 >> v1:172.21.15.115:6792/0 conn(0x7f7dc41365f0 legacy=0x7f7dc413f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:13.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 1 -- 172.21.15.132:0/575972106 shutdown_connections 2024-01-24T23:27:13.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 1 -- 172.21.15.132:0/575972106 >> 172.21.15.132:0/575972106 conn(0x7f7dc40a6790 msgr2=0x7f7dc4071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:13.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 1 -- 172.21.15.132:0/575972106 shutdown_connections 2024-01-24T23:27:13.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 1 -- 172.21.15.132:0/575972106 wait complete. 2024-01-24T23:27:13.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 10 monclient: build_initial_monmap 2024-01-24T23:27:13.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 10 monclient: monmap: 2024-01-24T23:27:13.298 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:13.298 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:13.298 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:13.298 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:13.298 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:13.298 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:13.298 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:13.298 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:13.298 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:13.298 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:13.298 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:13.298 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:13.299 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:13.299 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:13.299 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:13.299 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:13.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 1 Processor -- start 2024-01-24T23:27:13.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 1 -- start start 2024-01-24T23:27:13.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 10 monclient: init 2024-01-24T23:27:13.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:13.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 10 monclient: _add_conns ranks=[6,7,1,2,0,3,8,5,4] 2024-01-24T23:27:13.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 10 monclient(hunting): picked mon.noname-d con 0x7f7dc4135e10 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:13.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 10 monclient(hunting): picked mon.noname-i con 0x7f7dc41361f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:13.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 10 monclient(hunting): picked mon.noname-f con 0x7f7dc41365f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:13.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc4122780 con 0x7f7dc41365f0 2024-01-24T23:27:13.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc409c6b0 con 0x7f7dc4135e10 2024-01-24T23:27:13.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc41bf2c0 con 0x7f7dc41361f0 2024-01-24T23:27:13.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:13.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc905f640 10 monclient(hunting): authenticate will time out at 1320.618488s 2024-01-24T23:27:13.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc259c640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f7dc41361f0 legacy=0x7f7dc41d01c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:13.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc359e640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f7dc41365f0 legacy=0x7f7dc41d38f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:27:13.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc2d9d640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f7dc4135e10 0x7f7dc4134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:49642/0 (socket says 172.21.15.132:49642) 2024-01-24T23:27:13.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dc2d9d640 1 -- 172.21.15.132:0/1448595025 learned_addr learned my addr 172.21.15.132:0/1448595025 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:13.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dbb7fe640 1 -- 172.21.15.132:0/1448595025 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1224866476 0 0) 0x7f7dc409c6b0 con 0x7f7dc4135e10 2024-01-24T23:27:13.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dbb7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:13.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dbb7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:13.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dbb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:13.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dbb7fe640 10 monclient(hunting): my global_id is 6666 2024-01-24T23:27:13.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dbb7fe640 1 -- 172.21.15.132:0/1448595025 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7da4003530 con 0x7f7dc4135e10 2024-01-24T23:27:13.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dbb7fe640 1 -- 172.21.15.132:0/1448595025 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3522245931 0 0) 0x7f7da4003530 con 0x7f7dc4135e10 2024-01-24T23:27:13.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dbb7fe640 1 -- 172.21.15.132:0/1448595025 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7dc409c6b0 con 0x7f7dc4135e10 2024-01-24T23:27:13.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dbb7fe640 1 -- 172.21.15.132:0/1448595025 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7dbc004680 con 0x7f7dc4135e10 2024-01-24T23:27:13.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dbb7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:13.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dbb7fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:27:13.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dbb7fe640 10 monclient(hunting): dump: 2024-01-24T23:27:13.304 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:13.304 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:13.304 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:13.304 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:13.304 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:13.304 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:13.304 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:13.305 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:13.305 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:13.305 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:13.305 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:13.305 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:13.305 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:13.305 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:13.305 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:13.305 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:13.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.290+0000 7f7dbb7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:13.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dbb7fe640 1 -- 172.21.15.132:0/1448595025 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3168582609 0 0) 0x7f7dc409c6b0 con 0x7f7dc4135e10 2024-01-24T23:27:13.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dbb7fe640 1 -- 172.21.15.132:0/1448595025 >> v1:172.21.15.132:6792/0 conn(0x7f7dc41361f0 legacy=0x7f7dc41d01c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:13.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dbb7fe640 1 -- 172.21.15.132:0/1448595025 >> v1:172.21.15.132:6789/0 conn(0x7f7dc41365f0 legacy=0x7f7dc41d38f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:13.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dbb7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:27:13.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dbb7fe640 1 monclient: found mon.d 2024-01-24T23:27:13.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dbb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:13.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dbb7fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:13.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dbb7fe640 1 -- 172.21.15.132:0/1448595025 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc4121ee0 con 0x7f7dc4135e10 2024-01-24T23:27:13.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dbb7fe640 10 monclient: _finish_auth 0 2024-01-24T23:27:13.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dbb7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:13.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dbb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:13.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dc905f640 5 monclient: authenticate success, global_id 6666 2024-01-24T23:27:13.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dc905f640 10 monclient: _renew_subs 2024-01-24T23:27:13.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dc905f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:13.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dc905f640 1 -- 172.21.15.132:0/1448595025 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc41221c0 con 0x7f7dc4135e10 2024-01-24T23:27:13.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dc905f640 10 monclient: _renew_subs 2024-01-24T23:27:13.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dc905f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:13.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dc905f640 1 -- 172.21.15.132:0/1448595025 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc41be980 con 0x7f7dc4135e10 2024-01-24T23:27:13.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dbb7fe640 1 -- 172.21.15.132:0/1448595025 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7dbc0048a0 con 0x7f7dc4135e10 2024-01-24T23:27:13.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dbb7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:13.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dbb7fe640 1 -- 172.21.15.132:0/1448595025 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7dbc004dd0 con 0x7f7dc4135e10 2024-01-24T23:27:13.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dbb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:13.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dbb7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:13.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dbb7fe640 10 monclient: dump: 2024-01-24T23:27:13.307 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:13.307 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:13.307 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:13.307 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:13.307 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:13.307 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:13.307 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:13.307 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:13.307 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:13.307 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:13.308 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:13.308 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:13.308 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:13.308 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:13.308 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:13.308 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:13.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dc159a640 4 set_mon_vals no callback set 2024-01-24T23:27:13.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dc905f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:13.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dc905f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:13.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dc905f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:13.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dc905f640 1 -- 172.21.15.132:0/1448595025 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7da8005090 con 0x7f7dc4135e10 2024-01-24T23:27:13.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dbb7fe640 1 -- 172.21.15.132:0/1448595025 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f7dbc004fb0 con 0x7f7dc4135e10 2024-01-24T23:27:13.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dbb7fe640 1 -- 172.21.15.132:0/1448595025 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(213..213 src has 1..213) v4 ==== 14168+0+0 (unknown 3819835050 0 0) 0x7f7dbc046390 con 0x7f7dc4135e10 2024-01-24T23:27:13.308 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.294+0000 7f7dc259c640 10 monclient: get_auth_request con 0x7f7da4036fb0 auth_method 0 2024-01-24T23:27:13.309 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.306+0000 7f7dbb7fe640 1 -- 172.21.15.132:0/1448595025 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f7dbc014d60 con 0x7f7dc4135e10 2024-01-24T23:27:13.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.306+0000 7f7dbb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:13.311 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.306+0000 7f7dbb7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:13.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.470+0000 7f7dc905f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h2"}] 2024-01-24T23:27:13.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.470+0000 7f7dc905f640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h2"}] 2024-01-24T23:27:13.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.470+0000 7f7dc905f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:13.471 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:13.470+0000 7f7dc905f640 1 -- 172.21.15.132:0/1448595025 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "h2"} v 0) v1 -- 0x7f7da8005340 con 0x7f7dc4135e10 2024-01-24T23:27:14.088 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.086+0000 7f7dbb7fe640 1 -- 172.21.15.132:0/1448595025 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h2"}]=0 removed item id -22 name 'h2' from crush map v214) v1 ==== 118+0+0 (unknown 1404782099 0 0) 0x7f7dbc014950 con 0x7f7dc4135e10 2024-01-24T23:27:14.088 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.086+0000 7f7dbb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h2"}] 2024-01-24T23:27:14.088 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.086+0000 7f7dbb7fe640 10 monclient: _finish_command 2 = system:0 removed item id -22 name 'h2' from crush map 2024-01-24T23:27:14.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.190+0000 7f7dc905f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h2"}] 2024-01-24T23:27:14.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.190+0000 7f7dc905f640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h2"}] 2024-01-24T23:27:14.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.190+0000 7f7dc905f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:14.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.190+0000 7f7dc905f640 1 -- 172.21.15.132:0/1448595025 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "h2"} v 0) v1 -- 0x7f7da80010a0 con 0x7f7dc4135e10 2024-01-24T23:27:14.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.190+0000 7f7dbb7fe640 1 -- 172.21.15.132:0/1448595025 <== mon.6 v1:172.21.15.115:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h2"}]=0 device 'h2' does not appear in the crush map v214) v1 ==== 118+0+0 (unknown 1453253224 0 0) 0x7f7dbc014b30 con 0x7f7dc4135e10 2024-01-24T23:27:14.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.190+0000 7f7dbb7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h2"}] 2024-01-24T23:27:14.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.190+0000 7f7dbb7fe640 10 monclient: _finish_command 3 = system:0 device 'h2' does not appear in the crush map 2024-01-24T23:27:14.194 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h2' does not appear in the crush map 2024-01-24T23:27:14.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.194+0000 7f7dc905f640 1 -- 172.21.15.132:0/1448595025 >> v1:172.21.15.132:6812/1835401922 conn(0x7f7da4036fb0 legacy=0x7f7da4039470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:14.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.194+0000 7f7dc905f640 10 monclient: shutdown 2024-01-24T23:27:14.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.194+0000 7f7dc905f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:14.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.194+0000 7f7dc905f640 1 -- 172.21.15.132:0/1448595025 >> v1:172.21.15.115:6792/0 conn(0x7f7dc4135e10 legacy=0x7f7dc4134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:14.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.194+0000 7f7dc905f640 1 -- 172.21.15.132:0/1448595025 shutdown_connections 2024-01-24T23:27:14.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.194+0000 7f7dc905f640 1 -- 172.21.15.132:0/1448595025 >> 172.21.15.132:0/1448595025 conn(0x7f7dc40a6790 msgr2=0x7f7dc406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:14.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.194+0000 7f7dc905f640 1 -- 172.21.15.132:0/1448595025 shutdown_connections 2024-01-24T23:27:14.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.194+0000 7f7dc905f640 1 -- 172.21.15.132:0/1448595025 wait complete. 2024-01-24T23:27:14.211 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:14.211 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h3 2024-01-24T23:27:14.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.330+0000 7f2f3666c640 10 monclient: get_monmap_and_config 2024-01-24T23:27:14.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.330+0000 7f2f3666c640 10 monclient: build_initial_monmap 2024-01-24T23:27:14.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.330+0000 7f2f3666c640 10 monclient: monmap: 2024-01-24T23:27:14.333 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:14.333 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:14.333 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:14.335205+0000 2024-01-24T23:27:14.333 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:14.335205+0000 2024-01-24T23:27:14.333 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:14.333 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:14.334 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:14.334 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:14.334 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:14.334 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:14.334 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:14.334 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:14.334 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:14.334 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:14.334 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:14.334 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:14.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.330+0000 7f2f3666c640 1 Processor -- start 2024-01-24T23:27:14.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.330+0000 7f2f3666c640 1 -- start start 2024-01-24T23:27:14.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.330+0000 7f2f3666c640 10 monclient: init 2024-01-24T23:27:14.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 10 monclient: _add_conns ranks=[2,5,4,6,1,8,0,3,7] 2024-01-24T23:27:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 10 monclient(hunting): picked mon.noname-b con 0x7f2f300a2e80 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 10 monclient(hunting): picked mon.noname-h con 0x7f2f300a4150 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 10 monclient(hunting): picked mon.noname-c con 0x7f2f3013d4f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f30064700 con 0x7f2f300a2e80 2024-01-24T23:27:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f300a4f40 con 0x7f2f3013d4f0 2024-01-24T23:27:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f300a0e80 con 0x7f2f300a4150 2024-01-24T23:27:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 10 monclient(hunting): authenticate will time out at 1321.660636s 2024-01-24T23:27:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f277fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f2f300a4150 0x7f2f3013bdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:48464/0 (socket says 172.21.15.132:48464) 2024-01-24T23:27:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2ffff640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f2f300a2e80 legacy=0x7f2f300a3260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f277fe640 1 -- 172.21.15.132:0/2222271989 learned_addr learned my addr 172.21.15.132:0/2222271989 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 1 -- 172.21.15.132:0/2222271989 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1393134656 0 0) 0x7f2f300a0e80 con 0x7f2f300a4150 2024-01-24T23:27:14.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 10 monclient(hunting): my global_id is 6485 2024-01-24T23:27:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 1 -- 172.21.15.132:0/2222271989 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f1c003530 con 0x7f2f300a4150 2024-01-24T23:27:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 1 -- 172.21.15.132:0/2222271989 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1496897379 0 0) 0x7f2f1c003530 con 0x7f2f300a4150 2024-01-24T23:27:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 1 -- 172.21.15.132:0/2222271989 >> v1:172.21.15.115:6791/0 conn(0x7f2f3013d4f0 legacy=0x7f2f3013f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 1 -- 172.21.15.132:0/2222271989 >> v1:172.21.15.115:6790/0 conn(0x7f2f300a2e80 legacy=0x7f2f300a3260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:27:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 1 monclient: found mon.noname-h 2024-01-24T23:27:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:27:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 1 -- 172.21.15.132:0/2222271989 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f300a1440 con 0x7f2f300a4150 2024-01-24T23:27:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 10 monclient: _finish_auth 0 2024-01-24T23:27:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:14.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 1 -- 172.21.15.132:0/2222271989 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2f200017c0 con 0x7f2f300a4150 2024-01-24T23:27:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 5 monclient: authenticate success, global_id 6485 2024-01-24T23:27:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:27:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 10 monclient: dump: 2024-01-24T23:27:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:14.344 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:14.345 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 1 -- 172.21.15.132:0/2222271989 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2f20001ae0 con 0x7f2f300a4150 2024-01-24T23:27:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 1 -- 172.21.15.132:0/2222271989 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2f20004bf0 con 0x7f2f300a4150 2024-01-24T23:27:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 4 set_mon_vals no callback set 2024-01-24T23:27:14.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2f7fe640 10 monclient: dump: 2024-01-24T23:27:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:14.346 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:14.347 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 10 monclient: shutdown 2024-01-24T23:27:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 1 -- 172.21.15.132:0/2222271989 >> v1:172.21.15.132:6791/0 conn(0x7f2f300a4150 legacy=0x7f2f3013bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 1 -- 172.21.15.132:0/2222271989 shutdown_connections 2024-01-24T23:27:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 1 -- 172.21.15.132:0/2222271989 >> 172.21.15.132:0/2222271989 conn(0x7f2f30071c20 msgr2=0x7f2f30072030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 1 -- 172.21.15.132:0/2222271989 shutdown_connections 2024-01-24T23:27:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 1 -- 172.21.15.132:0/2222271989 wait complete. 2024-01-24T23:27:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 10 monclient: build_initial_monmap 2024-01-24T23:27:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 10 monclient: monmap: 2024-01-24T23:27:14.347 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:14.348 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:14.348 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:14.348 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:14.348 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:14.348 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:14.348 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:14.348 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:14.348 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:14.348 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:14.348 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:14.348 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:14.348 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:14.349 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:14.349 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:14.349 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:14.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 1 Processor -- start 2024-01-24T23:27:14.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 1 -- start start 2024-01-24T23:27:14.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 10 monclient: init 2024-01-24T23:27:14.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:14.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 10 monclient: _add_conns ranks=[6,5,3,1,7,2,4,0,8] 2024-01-24T23:27:14.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 10 monclient(hunting): picked mon.noname-d con 0x7f2f300a2e80 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:14.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 10 monclient(hunting): picked mon.noname-h con 0x7f2f300a4150 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:14.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 10 monclient(hunting): picked mon.noname-g con 0x7f2f3013d4f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:14.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f301359e0 con 0x7f2f3013d4f0 2024-01-24T23:27:14.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f300a2c50 con 0x7f2f300a4150 2024-01-24T23:27:14.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f30137340 con 0x7f2f300a2e80 2024-01-24T23:27:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f3666c640 10 monclient(hunting): authenticate will time out at 1321.663580s 2024-01-24T23:27:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f277fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f2f300a4150 0x7f2f30133c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:48474/0 (socket says 172.21.15.132:48474) 2024-01-24T23:27:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f277fe640 1 -- 172.21.15.132:0/1593715682 learned_addr learned my addr 172.21.15.132:0/1593715682 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2d7fa640 1 -- 172.21.15.132:0/1593715682 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355909877 0 0) 0x7f2f300a2c50 con 0x7f2f300a4150 2024-01-24T23:27:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2d7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:27:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2d7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2d7fa640 10 monclient(hunting): my global_id is 6494 2024-01-24T23:27:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2d7fa640 1 -- 172.21.15.132:0/1593715682 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f10003530 con 0x7f2f300a4150 2024-01-24T23:27:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2d7fa640 1 -- 172.21.15.132:0/1593715682 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2602752087 0 0) 0x7f2f301359e0 con 0x7f2f3013d4f0 2024-01-24T23:27:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2d7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:27:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2d7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:14.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2d7fa640 10 monclient(hunting): my global_id is 6528 2024-01-24T23:27:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.334+0000 7f2f2d7fa640 1 -- 172.21.15.132:0/1593715682 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f300a2c50 con 0x7f2f3013d4f0 2024-01-24T23:27:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 1 -- 172.21.15.132:0/1593715682 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3303087181 0 0) 0x7f2f10003530 con 0x7f2f300a4150 2024-01-24T23:27:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 1 -- 172.21.15.132:0/1593715682 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2f301359e0 con 0x7f2f300a4150 2024-01-24T23:27:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 1 -- 172.21.15.132:0/1593715682 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2f20001780 con 0x7f2f300a4150 2024-01-24T23:27:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:27:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 10 monclient(hunting): dump: 2024-01-24T23:27:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:14.351 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:14.352 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 1 -- 172.21.15.132:0/1593715682 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1519666315 0 0) 0x7f2f300a2c50 con 0x7f2f3013d4f0 2024-01-24T23:27:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 1 -- 172.21.15.132:0/1593715682 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2f10003530 con 0x7f2f3013d4f0 2024-01-24T23:27:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 1 -- 172.21.15.132:0/1593715682 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2f28001c80 con 0x7f2f3013d4f0 2024-01-24T23:27:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:14.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 10 monclient(hunting): dump: 2024-01-24T23:27:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:14.353 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:14.354 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 1 -- 172.21.15.132:0/1593715682 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 274524909 0 0) 0x7f2f30137340 con 0x7f2f300a2e80 2024-01-24T23:27:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 10 monclient(hunting): _negotiate 2024-01-24T23:27:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 10 monclient(hunting): my global_id is 6675 2024-01-24T23:27:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 1 -- 172.21.15.132:0/1593715682 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f300a2c50 con 0x7f2f300a2e80 2024-01-24T23:27:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 1 -- 172.21.15.132:0/1593715682 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2953045150 0 0) 0x7f2f301359e0 con 0x7f2f300a4150 2024-01-24T23:27:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 1 -- 172.21.15.132:0/1593715682 >> v1:172.21.15.115:6792/0 conn(0x7f2f300a2e80 legacy=0x7f2f301334f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 1 -- 172.21.15.132:0/1593715682 >> v1:172.21.15.132:6790/0 conn(0x7f2f3013d4f0 legacy=0x7f2f301d36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 10 monclient: _finish_hunting 0 2024-01-24T23:27:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 1 monclient: found mon.h 2024-01-24T23:27:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:14.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 1 -- 172.21.15.132:0/1593715682 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f300a5820 con 0x7f2f300a4150 2024-01-24T23:27:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 10 monclient: _finish_auth 0 2024-01-24T23:27:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:27:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f3666c640 5 monclient: authenticate success, global_id 6494 2024-01-24T23:27:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f3666c640 10 monclient: _renew_subs 2024-01-24T23:27:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f3666c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f3666c640 1 -- 172.21.15.132:0/1593715682 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f30135050 con 0x7f2f300a4150 2024-01-24T23:27:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f3666c640 10 monclient: _renew_subs 2024-01-24T23:27:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f3666c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f3666c640 1 -- 172.21.15.132:0/1593715682 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3009f8d0 con 0x7f2f300a4150 2024-01-24T23:27:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 1 -- 172.21.15.132:0/1593715682 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2f20001180 con 0x7f2f300a4150 2024-01-24T23:27:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 1 -- 172.21.15.132:0/1593715682 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2f20001c50 con 0x7f2f300a4150 2024-01-24T23:27:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:14.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2effd640 4 set_mon_vals no callback set 2024-01-24T23:27:14.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:14.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 10 monclient: dump: 2024-01-24T23:27:14.356 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:14.356 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:14.356 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:14.356 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:14.356 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:14.356 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:14.356 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:14.356 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:14.356 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:14.356 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:14.356 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:14.356 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:14.357 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:14.357 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:14.357 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:14.357 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:14.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f3666c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:14.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f3666c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:14.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f3666c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:14.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f3666c640 1 -- 172.21.15.132:0/1593715682 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f14005090 con 0x7f2f300a4150 2024-01-24T23:27:14.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 1 -- 172.21.15.132:0/1593715682 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f2f2000fdd0 con 0x7f2f300a4150 2024-01-24T23:27:14.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2d7fa640 1 -- 172.21.15.132:0/1593715682 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(214..214 src has 1..214) v4 ==== 13958+0+0 (unknown 1660764193 0 0) 0x7f2f20046280 con 0x7f2f300a4150 2024-01-24T23:27:14.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.338+0000 7f2f2ffff640 10 monclient: get_auth_request con 0x7f2f10037760 auth_method 0 2024-01-24T23:27:14.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.342+0000 7f2f2d7fa640 1 -- 172.21.15.132:0/1593715682 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f2f20014bb0 con 0x7f2f300a4150 2024-01-24T23:27:14.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.342+0000 7f2f2d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:14.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.342+0000 7f2f2d7fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:14.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.502+0000 7f2f3666c640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h3"}] 2024-01-24T23:27:14.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.502+0000 7f2f3666c640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h3"}] 2024-01-24T23:27:14.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.502+0000 7f2f3666c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:14.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:14.502+0000 7f2f3666c640 1 -- 172.21.15.132:0/1593715682 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h3"} v 0) v1 -- 0x7f2f14005340 con 0x7f2f300a4150 2024-01-24T23:27:15.091 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.090+0000 7f2f2d7fa640 1 -- 172.21.15.132:0/1593715682 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h3"}]=0 removed item id -25 name 'h3' from crush map v215) v1 ==== 118+0+0 (unknown 2788747460 0 0) 0x7f2f20013250 con 0x7f2f300a4150 2024-01-24T23:27:15.091 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.090+0000 7f2f2d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h3"}] 2024-01-24T23:27:15.091 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.090+0000 7f2f2d7fa640 10 monclient: _finish_command 2 = system:0 removed item id -25 name 'h3' from crush map 2024-01-24T23:27:15.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.194+0000 7f2f3666c640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h3"}] 2024-01-24T23:27:15.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.194+0000 7f2f3666c640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h3"}] 2024-01-24T23:27:15.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.194+0000 7f2f3666c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:15.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.194+0000 7f2f3666c640 1 -- 172.21.15.132:0/1593715682 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h3"} v 0) v1 -- 0x7f2f14002220 con 0x7f2f300a4150 2024-01-24T23:27:15.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.198+0000 7f2f2d7fa640 1 -- 172.21.15.132:0/1593715682 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h3"}]=0 device 'h3' does not appear in the crush map v215) v1 ==== 118+0+0 (unknown 2515647340 0 0) 0x7f2f200149d0 con 0x7f2f300a4150 2024-01-24T23:27:15.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.198+0000 7f2f2d7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h3"}] 2024-01-24T23:27:15.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.198+0000 7f2f2d7fa640 10 monclient: _finish_command 3 = system:0 device 'h3' does not appear in the crush map 2024-01-24T23:27:15.199 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h3' does not appear in the crush map 2024-01-24T23:27:15.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.198+0000 7f2f3666c640 1 -- 172.21.15.132:0/1593715682 >> v1:172.21.15.132:6812/1835401922 conn(0x7f2f10037760 legacy=0x7f2f10039c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:15.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.198+0000 7f2f3666c640 10 monclient: shutdown 2024-01-24T23:27:15.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.198+0000 7f2f3666c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:15.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.198+0000 7f2f3666c640 1 -- 172.21.15.132:0/1593715682 >> v1:172.21.15.132:6791/0 conn(0x7f2f300a4150 legacy=0x7f2f30133c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:15.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.198+0000 7f2f3666c640 1 -- 172.21.15.132:0/1593715682 shutdown_connections 2024-01-24T23:27:15.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.198+0000 7f2f3666c640 1 -- 172.21.15.132:0/1593715682 >> 172.21.15.132:0/1593715682 conn(0x7f2f30071c20 msgr2=0x7f2f30138280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:15.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.198+0000 7f2f3666c640 1 -- 172.21.15.132:0/1593715682 shutdown_connections 2024-01-24T23:27:15.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.198+0000 7f2f3666c640 1 -- 172.21.15.132:0/1593715682 wait complete. 2024-01-24T23:27:15.216 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:15.216 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h4 2024-01-24T23:27:15.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.330+0000 7f70501da640 10 monclient: get_monmap_and_config 2024-01-24T23:27:15.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.330+0000 7f70501da640 10 monclient: build_initial_monmap 2024-01-24T23:27:15.333 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.330+0000 7f70501da640 10 monclient: monmap: 2024-01-24T23:27:15.333 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:15.334 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:15.334 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:15.335722+0000 2024-01-24T23:27:15.334 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:15.335722+0000 2024-01-24T23:27:15.334 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:15.334 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:15.334 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:15.334 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:15.334 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:15.334 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:15.334 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:15.334 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:15.334 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:15.334 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:15.334 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:15.335 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:15.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 1 Processor -- start 2024-01-24T23:27:15.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 1 -- start start 2024-01-24T23:27:15.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 10 monclient: init 2024-01-24T23:27:15.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:15.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 10 monclient: _add_conns ranks=[5,1,7,3,6,2,4,8,0] 2024-01-24T23:27:15.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 10 monclient(hunting): picked mon.noname-h con 0x7f7048135e10 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:15.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 10 monclient(hunting): picked mon.noname-f con 0x7f70481361f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:15.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 10 monclient(hunting): picked mon.noname-i con 0x7f70481365f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:15.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7048064700 con 0x7f70481361f0 2024-01-24T23:27:15.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f704809c3e0 con 0x7f7048135e10 2024-01-24T23:27:15.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70480a4f90 con 0x7f70481365f0 2024-01-24T23:27:15.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:15.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704d775640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f70481361f0 legacy=0x7f704813bed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:27:15.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 10 monclient(hunting): authenticate will time out at 1322.661149s 2024-01-24T23:27:15.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704e777640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f70481365f0 legacy=0x7f704813f600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:15.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704df76640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f7048135e10 0x7f704809bcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:48484/0 (socket says 172.21.15.132:48484) 2024-01-24T23:27:15.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704df76640 1 -- 172.21.15.132:0/2238938621 learned_addr learned my addr 172.21.15.132:0/2238938621 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:15.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 1 -- 172.21.15.132:0/2238938621 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1789839371 0 0) 0x7f704809c3e0 con 0x7f7048135e10 2024-01-24T23:27:15.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 10 monclient(hunting): _negotiate 2024-01-24T23:27:15.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:15.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:15.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 10 monclient(hunting): my global_id is 6503 2024-01-24T23:27:15.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 1 -- 172.21.15.132:0/2238938621 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7034003530 con 0x7f7048135e10 2024-01-24T23:27:15.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 1 -- 172.21.15.132:0/2238938621 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4049895470 0 0) 0x7f7034003530 con 0x7f7048135e10 2024-01-24T23:27:15.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 1 -- 172.21.15.132:0/2238938621 >> v1:172.21.15.132:6792/0 conn(0x7f70481365f0 legacy=0x7f704813f600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:15.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 1 -- 172.21.15.132:0/2238938621 >> v1:172.21.15.132:6789/0 conn(0x7f70481361f0 legacy=0x7f704813bed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:15.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 10 monclient: _finish_hunting 0 2024-01-24T23:27:15.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 1 monclient: found mon.noname-h 2024-01-24T23:27:15.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:15.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:27:15.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 1 -- 172.21.15.132:0/2238938621 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704809b010 con 0x7f7048135e10 2024-01-24T23:27:15.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 10 monclient: _finish_auth 0 2024-01-24T23:27:15.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 10 monclient: _check_auth_tickets 2024-01-24T23:27:15.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:15.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 1 -- 172.21.15.132:0/2238938621 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f70440017c0 con 0x7f7048135e10 2024-01-24T23:27:15.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:15.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:27:15.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 10 monclient: dump: 2024-01-24T23:27:15.340 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:15.340 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:15.340 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:15.340 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:15.340 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:15.340 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:15.340 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:15.340 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:15.340 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:15.341 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:15.341 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:15.341 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:15.341 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:15.341 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:15.341 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:15.341 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:15.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 5 monclient: authenticate success, global_id 6503 2024-01-24T23:27:15.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:15.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 1 -- 172.21.15.132:0/2238938621 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7044001ae0 con 0x7f7048135e10 2024-01-24T23:27:15.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:15.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 1 -- 172.21.15.132:0/2238938621 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7044004bf0 con 0x7f7048135e10 2024-01-24T23:27:15.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:15.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:15.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f704cf74640 10 monclient: dump: 2024-01-24T23:27:15.342 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:15.342 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:15.342 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:15.342 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:15.342 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:15.342 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:15.342 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:15.342 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:15.342 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:15.342 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:15.342 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:15.342 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:15.342 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:15.342 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:15.342 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:15.342 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:15.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:15.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 4 set_mon_vals no callback set 2024-01-24T23:27:15.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 10 monclient: shutdown 2024-01-24T23:27:15.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:15.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 1 -- 172.21.15.132:0/2238938621 >> v1:172.21.15.132:6791/0 conn(0x7f7048135e10 legacy=0x7f704809bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:15.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 1 -- 172.21.15.132:0/2238938621 shutdown_connections 2024-01-24T23:27:15.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 1 -- 172.21.15.132:0/2238938621 >> 172.21.15.132:0/2238938621 conn(0x7f70480a6790 msgr2=0x7f7048071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:15.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 1 -- 172.21.15.132:0/2238938621 shutdown_connections 2024-01-24T23:27:15.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 1 -- 172.21.15.132:0/2238938621 wait complete. 2024-01-24T23:27:15.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 10 monclient: build_initial_monmap 2024-01-24T23:27:15.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 10 monclient: monmap: 2024-01-24T23:27:15.343 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:15.343 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:15.343 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:15.343 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:15.344 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:15.344 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:15.344 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:15.344 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:15.344 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:15.344 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:15.344 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:15.344 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:15.344 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:15.344 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:15.344 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:15.344 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:15.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 1 Processor -- start 2024-01-24T23:27:15.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 1 -- start start 2024-01-24T23:27:15.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.334+0000 7f70501da640 10 monclient: init 2024-01-24T23:27:15.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f70501da640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:15.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f70501da640 10 monclient: _add_conns ranks=[2,7,3,6,5,1,4,8,0] 2024-01-24T23:27:15.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f70501da640 10 monclient(hunting): picked mon.noname-b con 0x7f7048135e10 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:15.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f70501da640 10 monclient(hunting): picked mon.noname-i con 0x7f70481361f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:15.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f70501da640 10 monclient(hunting): picked mon.noname-g con 0x7f70481365f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:15.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f70501da640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f704809ca00 con 0x7f7048135e10 2024-01-24T23:27:15.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f70501da640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f704809c6b0 con 0x7f70481365f0 2024-01-24T23:27:15.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f704d775640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f70481361f0 legacy=0x7f70481d0330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:15.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f70501da640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70481bf3b0 con 0x7f70481361f0 2024-01-24T23:27:15.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f70501da640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:15.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f70501da640 10 monclient(hunting): authenticate will time out at 1322.664471s 2024-01-24T23:27:15.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f704df76640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f7048135e10 legacy=0x7f7048134d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:15.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f704e777640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f70481365f0 0x7f70481d3a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:48082/0 (socket says 172.21.15.132:48082) 2024-01-24T23:27:15.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f704e777640 1 -- 172.21.15.132:0/1834108242 learned_addr learned my addr 172.21.15.132:0/1834108242 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:15.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 1 -- 172.21.15.132:0/1834108242 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1615258175 0 0) 0x7f704809c6b0 con 0x7f70481365f0 2024-01-24T23:27:15.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:15.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:15.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:15.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 10 monclient(hunting): my global_id is 6537 2024-01-24T23:27:15.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 1 -- 172.21.15.132:0/1834108242 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f702c003530 con 0x7f70481365f0 2024-01-24T23:27:15.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 1 -- 172.21.15.132:0/1834108242 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3402418738 0 0) 0x7f702c003530 con 0x7f70481365f0 2024-01-24T23:27:15.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 1 -- 172.21.15.132:0/1834108242 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f704809c6b0 con 0x7f70481365f0 2024-01-24T23:27:15.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 1 -- 172.21.15.132:0/1834108242 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7040004560 con 0x7f70481365f0 2024-01-24T23:27:15.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:15.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:27:15.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 10 monclient(hunting): dump: 2024-01-24T23:27:15.350 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:15.350 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:15.350 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:15.350 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:15.350 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:15.350 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:15.350 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:15.350 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:15.350 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:15.350 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:15.351 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:15.351 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:15.351 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:15.351 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:15.351 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:15.351 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:15.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:15.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 1 -- 172.21.15.132:0/1834108242 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2494229387 0 0) 0x7f704809c6b0 con 0x7f70481365f0 2024-01-24T23:27:15.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 1 -- 172.21.15.132:0/1834108242 >> v1:172.21.15.132:6792/0 conn(0x7f70481361f0 legacy=0x7f70481d0330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:15.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 1 -- 172.21.15.132:0/1834108242 >> v1:172.21.15.115:6790/0 conn(0x7f7048135e10 legacy=0x7f7048134d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:15.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:27:15.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 1 monclient: found mon.g 2024-01-24T23:27:15.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:15.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:15.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 1 -- 172.21.15.132:0/1834108242 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7048135490 con 0x7f70481365f0 2024-01-24T23:27:15.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 10 monclient: _finish_auth 0 2024-01-24T23:27:15.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:27:15.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:15.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f70501da640 5 monclient: authenticate success, global_id 6537 2024-01-24T23:27:15.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f70501da640 10 monclient: _renew_subs 2024-01-24T23:27:15.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f70501da640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:15.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f70501da640 1 -- 172.21.15.132:0/1834108242 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7048120fb0 con 0x7f70481365f0 2024-01-24T23:27:15.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f70501da640 10 monclient: _renew_subs 2024-01-24T23:27:15.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f70501da640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:15.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f70501da640 1 -- 172.21.15.132:0/1834108242 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f70481358c0 con 0x7f70481365f0 2024-01-24T23:27:15.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 1 -- 172.21.15.132:0/1834108242 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7040004880 con 0x7f70481365f0 2024-01-24T23:27:15.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:15.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 1 -- 172.21.15.132:0/1834108242 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7040004db0 con 0x7f70481365f0 2024-01-24T23:27:15.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:15.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703ffff640 4 set_mon_vals no callback set 2024-01-24T23:27:15.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:15.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 10 monclient: dump: 2024-01-24T23:27:15.353 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:15.353 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:15.353 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:15.353 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:15.353 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:15.353 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:15.353 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:15.353 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:15.353 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:15.353 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:15.353 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:15.353 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:15.354 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:15.354 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:15.354 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:15.354 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:15.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 1 -- 172.21.15.132:0/1834108242 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f7040004f10 con 0x7f70481365f0 2024-01-24T23:27:15.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f70501da640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:15.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f70501da640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:15.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f70501da640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:15.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f70501da640 1 -- 172.21.15.132:0/1834108242 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7030005090 con 0x7f70481365f0 2024-01-24T23:27:15.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f703e7fc640 1 -- 172.21.15.132:0/1834108242 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(215..215 src has 1..215) v4 ==== 13748+0+0 (unknown 2317821301 0 0) 0x7f70400461a0 con 0x7f70481365f0 2024-01-24T23:27:15.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.338+0000 7f704df76640 10 monclient: get_auth_request con 0x7f702c036e80 auth_method 0 2024-01-24T23:27:15.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.342+0000 7f703e7fc640 1 -- 172.21.15.132:0/1834108242 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f7040014d10 con 0x7f70481365f0 2024-01-24T23:27:15.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.342+0000 7f703e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:15.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.342+0000 7f703e7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:15.498 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-01-24T23:27:15.499 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-01-24T23:27:15.499 INFO:tasks.ceph.mon.b:Restarting daemon 2024-01-24T23:27:15.499 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-01-24T23:27:15.502 INFO:tasks.ceph.mon.b:Started 2024-01-24T23:27:15.502 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-01-24T23:27:15.502 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.i 2024-01-24T23:27:15.502 INFO:tasks.ceph.mon.i:Restarting daemon 2024-01-24T23:27:15.502 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-01-24T23:27:15.504 INFO:tasks.ceph.mon.i:Started 2024-01-24T23:27:15.504 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-01-24T23:27:15.505 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.f 2024-01-24T23:27:15.505 INFO:tasks.ceph.mon.f:Restarting daemon 2024-01-24T23:27:15.505 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-01-24T23:27:15.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.502+0000 7f70501da640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h4"}] 2024-01-24T23:27:15.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.502+0000 7f70501da640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h4"}] 2024-01-24T23:27:15.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.502+0000 7f70501da640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:15.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:15.502+0000 7f70501da640 1 -- 172.21.15.132:0/1834108242 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rm", "name": "h4"} v 0) v1 -- 0x7f7030005340 con 0x7f70481365f0 2024-01-24T23:27:15.507 INFO:tasks.ceph.mon.f:Started 2024-01-24T23:27:15.507 INFO:tasks.mon_thrash:Sending STOP to mon c 2024-01-24T23:27:15.508 INFO:tasks.ceph.mon.c:Sent signal 19 2024-01-24T23:27:15.508 INFO:tasks.mon_thrash:Sending STOP to mon d 2024-01-24T23:27:15.508 INFO:tasks.ceph.mon.d:Sent signal 19 2024-01-24T23:27:15.508 INFO:tasks.mon_thrash:Sending STOP to mon g 2024-01-24T23:27:15.508 INFO:tasks.ceph.mon.g:Sent signal 19 2024-01-24T23:27:15.508 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-01-24T23:27:18.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:18.338+0000 7f703d7fa640 10 monclient: tick 2024-01-24T23:27:18.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:18.338+0000 7f703d7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:27:18.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:18.338+0000 7f703d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:18.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:18.338+0000 7f703d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:19.066 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~75s 2024-01-24T23:27:19.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:19.338+0000 7f703d7fa640 10 monclient: tick 2024-01-24T23:27:19.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:19.338+0000 7f703d7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:27:19.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:19.338+0000 7f703d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:20.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:20.338+0000 7f703d7fa640 10 monclient: tick 2024-01-24T23:27:20.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:20.338+0000 7f703d7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:27:20.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:20.338+0000 7f703d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:21.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:21.338+0000 7f703d7fa640 10 monclient: tick 2024-01-24T23:27:21.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:21.338+0000 7f703d7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:27:21.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:21.338+0000 7f703d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:22.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:22.338+0000 7f703d7fa640 10 monclient: tick 2024-01-24T23:27:22.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:22.338+0000 7f703d7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:27:22.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:22.338+0000 7f703d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:23.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:23.338+0000 7f703d7fa640 10 monclient: tick 2024-01-24T23:27:23.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:23.338+0000 7f703d7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:27:23.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:23.338+0000 7f703d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:24.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:24.338+0000 7f703d7fa640 10 monclient: tick 2024-01-24T23:27:24.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:24.338+0000 7f703d7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:27:24.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:24.338+0000 7f703d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:25.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:25.338+0000 7f703d7fa640 10 monclient: tick 2024-01-24T23:27:25.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:25.338+0000 7f703d7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:27:25.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:25.338+0000 7f703d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:25.578 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~81s 2024-01-24T23:27:26.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:26.338+0000 7f703d7fa640 10 monclient: tick 2024-01-24T23:27:26.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:26.338+0000 7f703d7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:27:26.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:26.338+0000 7f703d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:27.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:27.338+0000 7f703d7fa640 10 monclient: tick 2024-01-24T23:27:27.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:27.338+0000 7f703d7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:27:27.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:27.338+0000 7f703d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:28.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:28.338+0000 7f703d7fa640 10 monclient: tick 2024-01-24T23:27:28.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:28.338+0000 7f703d7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:27:28.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:28.338+0000 7f703d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:28.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:28.338+0000 7f703d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:29.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:29.338+0000 7f703d7fa640 10 monclient: tick 2024-01-24T23:27:29.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:29.338+0000 7f703d7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:27:29.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:29.338+0000 7f703d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:30.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:30.338+0000 7f703d7fa640 10 monclient: tick 2024-01-24T23:27:30.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:30.338+0000 7f703d7fa640 10 monclient: _check_auth_tickets 2024-01-24T23:27:30.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:30.338+0000 7f703d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:30.509 INFO:tasks.mon_thrash:Sending CONT to mon c 2024-01-24T23:27:30.510 INFO:tasks.ceph.mon.c:Sent signal 18 2024-01-24T23:27:30.510 INFO:tasks.mon_thrash:Sending CONT to mon d 2024-01-24T23:27:30.510 INFO:tasks.ceph.mon.d:Sent signal 18 2024-01-24T23:27:30.510 INFO:tasks.mon_thrash:Sending CONT to mon g 2024-01-24T23:27:30.510 INFO:tasks.ceph.mon.g:Sent signal 18 2024-01-24T23:27:30.511 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-01-24T23:27:30.511 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-01-24T23:27:30.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.656+0000 7f5e958d1640 10 monclient: get_monmap_and_config 2024-01-24T23:27:30.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.656+0000 7f5e958d1640 10 monclient: build_initial_monmap 2024-01-24T23:27:30.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.656+0000 7f5e958d1640 10 monclient: monmap: 2024-01-24T23:27:30.660 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:30.660 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:30.660 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:30.661193+0000 2024-01-24T23:27:30.660 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:30.661193+0000 2024-01-24T23:27:30.660 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:30.661 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:30.661 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:30.661 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:30.661 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:30.661 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:30.661 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:30.661 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:30.661 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:30.661 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:30.661 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:30.661 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:30.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.656+0000 7f5e958d1640 1 Processor -- start 2024-01-24T23:27:30.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.656+0000 7f5e958d1640 1 -- start start 2024-01-24T23:27:30.661 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.656+0000 7f5e958d1640 10 monclient: init 2024-01-24T23:27:30.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.656+0000 7f5e958d1640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:30.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.656+0000 7f5e958d1640 10 monclient: _add_conns ranks=[1,4,7,2,6,8,0,3,5] 2024-01-24T23:27:30.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.656+0000 7f5e958d1640 10 monclient(hunting): picked mon.noname-f con 0x7f5e90133bf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:30.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.656+0000 7f5e958d1640 10 monclient(hunting): picked mon.noname-c con 0x7f5e90133fd0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:30.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.656+0000 7f5e958d1640 10 monclient(hunting): picked mon.noname-i con 0x7f5e901343b0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:30.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.656+0000 7f5e958d1640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e9004ed70 con 0x7f5e90133bf0 2024-01-24T23:27:30.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.656+0000 7f5e958d1640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e900a4b60 con 0x7f5e90133fd0 2024-01-24T23:27:30.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.656+0000 7f5e958d1640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e9009ae50 con 0x7f5e901343b0 2024-01-24T23:27:30.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.656+0000 7f5e958d1640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:30.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.656+0000 7f5e958d1640 10 monclient(hunting): authenticate will time out at 1329.583628s 2024-01-24T23:27:30.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.656+0000 7f5e867fc640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f5e90133fd0 0x7f5e9013bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:43608/0 (socket says 172.21.15.115:43608) 2024-01-24T23:27:30.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.656+0000 7f5e867fc640 1 -- 172.21.15.115:0/392535508 learned_addr learned my addr 172.21.15.115:0/392535508 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:30.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 1 -- 172.21.15.115:0/392535508 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1324038917 0 0) 0x7f5e900a4b60 con 0x7f5e90133fd0 2024-01-24T23:27:30.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:30.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:30.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:30.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 10 monclient(hunting): my global_id is 94702 2024-01-24T23:27:30.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 1 -- 172.21.15.115:0/392535508 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e7c003530 con 0x7f5e90133fd0 2024-01-24T23:27:30.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 1 -- 172.21.15.115:0/392535508 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 204425988 0 0) 0x7f5e7c003530 con 0x7f5e90133fd0 2024-01-24T23:27:30.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 1 -- 172.21.15.115:0/392535508 >> v1:172.21.15.132:6792/0 conn(0x7f5e901343b0 legacy=0x7f5e9013f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:30.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 1 -- 172.21.15.115:0/392535508 >> v1:172.21.15.132:6789/0 conn(0x7f5e90133bf0 legacy=0x7f5e9009bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:30.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:27:30.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 1 monclient: found mon.noname-c 2024-01-24T23:27:30.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:30.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:27:30.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 1 -- 172.21.15.115:0/392535508 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9009b130 con 0x7f5e90133fd0 2024-01-24T23:27:30.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 10 monclient: _finish_auth 0 2024-01-24T23:27:30.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:27:30.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:30.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 1 -- 172.21.15.115:0/392535508 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5e780017c0 con 0x7f5e90133fd0 2024-01-24T23:27:30.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 5 monclient: authenticate success, global_id 94702 2024-01-24T23:27:30.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:30.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:30.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:27:30.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 10 monclient: dump: 2024-01-24T23:27:30.671 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:30.671 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:30.671 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:30.671 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:30.671 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:30.671 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:30.671 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:30.671 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:30.671 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:30.671 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:30.672 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:30.672 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:30.672 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:30.672 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:30.672 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:30.672 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:30.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:30.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 1 -- 172.21.15.115:0/392535508 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5e78001ae0 con 0x7f5e90133fd0 2024-01-24T23:27:30.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:30.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 1 -- 172.21.15.115:0/392535508 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5e78004bf0 con 0x7f5e90133fd0 2024-01-24T23:27:30.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:30.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:30.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8e7fc640 10 monclient: dump: 2024-01-24T23:27:30.672 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:30.672 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:30.673 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:30.673 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:30.673 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:30.673 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:30.673 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:30.673 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:30.673 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:30.673 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:30.673 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:30.673 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:30.673 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:30.673 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:30.673 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:30.673 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:30.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:30.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 4 set_mon_vals no callback set 2024-01-24T23:27:30.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 10 monclient: shutdown 2024-01-24T23:27:30.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:30.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 1 -- 172.21.15.115:0/392535508 >> v1:172.21.15.115:6791/0 conn(0x7f5e90133fd0 legacy=0x7f5e9013bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:30.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 1 -- 172.21.15.115:0/392535508 shutdown_connections 2024-01-24T23:27:30.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 1 -- 172.21.15.115:0/392535508 >> 172.21.15.115:0/392535508 conn(0x7f5e900662f0 msgr2=0x7f5e90066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:30.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 1 -- 172.21.15.115:0/392535508 shutdown_connections 2024-01-24T23:27:30.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 1 -- 172.21.15.115:0/392535508 wait complete. 2024-01-24T23:27:30.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 10 monclient: build_initial_monmap 2024-01-24T23:27:30.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 10 monclient: monmap: 2024-01-24T23:27:30.674 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:30.674 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:30.674 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:30.674 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:30.674 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:30.674 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:30.674 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:30.675 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:30.675 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:30.675 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:30.675 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:30.675 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:30.675 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:30.675 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:30.675 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:30.675 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:30.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 1 Processor -- start 2024-01-24T23:27:30.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 1 -- start start 2024-01-24T23:27:30.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 10 monclient: init 2024-01-24T23:27:30.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:30.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 10 monclient: _add_conns ranks=[4,2,6,0,7,5,3,1,8] 2024-01-24T23:27:30.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 10 monclient(hunting): picked mon.noname-c con 0x7f5e90133bf0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:30.675 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 10 monclient(hunting): picked mon.noname-b con 0x7f5e90133fd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:30.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 10 monclient(hunting): picked mon.noname-d con 0x7f5e901343b0 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:30.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e901cc110 con 0x7f5e90133fd0 2024-01-24T23:27:30.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e900a5990 con 0x7f5e90133bf0 2024-01-24T23:27:30.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e90120840 con 0x7f5e901343b0 2024-01-24T23:27:30.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:30.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 10 monclient(hunting): authenticate will time out at 1329.586748s 2024-01-24T23:27:30.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8effd640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f5e90133bf0 0x7f5e90132e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:43620/0 (socket says 172.21.15.115:43620) 2024-01-24T23:27:30.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8effd640 1 -- 172.21.15.115:0/3138974938 learned_addr learned my addr 172.21.15.115:0/3138974938 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:30.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 56178185 0 0) 0x7f5e900a5990 con 0x7f5e90133bf0 2024-01-24T23:27:30.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:30.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:30.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:30.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient(hunting): my global_id is 94711 2024-01-24T23:27:30.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e70003530 con 0x7f5e90133bf0 2024-01-24T23:27:30.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2342534676 0 0) 0x7f5e90120840 con 0x7f5e901343b0 2024-01-24T23:27:30.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:30.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:30.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:30.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient(hunting): my global_id is 6684 2024-01-24T23:27:30.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e900a5990 con 0x7f5e901343b0 2024-01-24T23:27:30.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1553423592 0 0) 0x7f5e901cc110 con 0x7f5e90133fd0 2024-01-24T23:27:30.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:30.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:30.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:30.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient(hunting): my global_id is 124103 2024-01-24T23:27:30.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e90120840 con 0x7f5e90133fd0 2024-01-24T23:27:30.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 383231074 0 0) 0x7f5e70003530 con 0x7f5e90133bf0 2024-01-24T23:27:30.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5e901cc110 con 0x7f5e90133bf0 2024-01-24T23:27:30.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5e800019c0 con 0x7f5e90133bf0 2024-01-24T23:27:30.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:30.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:27:30.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient(hunting): dump: 2024-01-24T23:27:30.678 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:30.678 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:30.678 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:30.678 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:30.678 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:30.678 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:30.678 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:30.678 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:30.678 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:30.678 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:30.678 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:30.678 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:30.679 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:30.679 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:30.679 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:30.679 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:30.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:30.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2172335904 0 0) 0x7f5e90120840 con 0x7f5e90133fd0 2024-01-24T23:27:30.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5e70003530 con 0x7f5e90133fd0 2024-01-24T23:27:30.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 178958877 0 0) 0x7f5e900a5990 con 0x7f5e901343b0 2024-01-24T23:27:30.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5e90120840 con 0x7f5e901343b0 2024-01-24T23:27:30.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5e78001780 con 0x7f5e90133fd0 2024-01-24T23:27:30.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:30.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:27:30.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient(hunting): dump: 2024-01-24T23:27:30.679 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:30.679 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:30.680 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:30.680 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:30.680 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:30.680 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:30.680 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:30.680 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:30.680 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:30.680 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:30.680 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:30.680 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:30.680 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:30.680 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:30.680 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:30.680 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:30.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5e88001ce0 con 0x7f5e901343b0 2024-01-24T23:27:30.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:30.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:30.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient(hunting): dump: 2024-01-24T23:27:30.681 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:30.681 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:30.681 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:30.681 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:30.681 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:30.681 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:30.681 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:30.681 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:30.681 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:30.681 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:30.681 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:30.681 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:30.681 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:30.681 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:30.682 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1737335013 0 0) 0x7f5e901cc110 con 0x7f5e90133bf0 2024-01-24T23:27:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 >> v1:172.21.15.115:6792/0 conn(0x7f5e901343b0 legacy=0x7f5e901d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 >> v1:172.21.15.115:6790/0 conn(0x7f5e90133fd0 legacy=0x7f5e901d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient: _finish_hunting 0 2024-01-24T23:27:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 1 monclient: found mon.c 2024-01-24T23:27:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e900a25f0 con 0x7f5e90133bf0 2024-01-24T23:27:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient: _finish_auth 0 2024-01-24T23:27:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient: _check_auth_tickets 2024-01-24T23:27:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:30.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 5 monclient: authenticate success, global_id 94711 2024-01-24T23:27:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 10 monclient: _renew_subs 2024-01-24T23:27:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 1 -- 172.21.15.115:0/3138974938 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e9011ffa0 con 0x7f5e90133bf0 2024-01-24T23:27:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 10 monclient: _renew_subs 2024-01-24T23:27:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e958d1640 1 -- 172.21.15.115:0/3138974938 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e90133740 con 0x7f5e90133bf0 2024-01-24T23:27:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5e80001d70 con 0x7f5e90133bf0 2024-01-24T23:27:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5e80003c10 con 0x7f5e90133bf0 2024-01-24T23:27:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8dffb640 4 set_mon_vals no callback set 2024-01-24T23:27:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.660+0000 7f5e8cff9640 10 monclient: dump: 2024-01-24T23:27:30.683 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:30.684 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:30.684 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:30.684 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:30.684 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:30.684 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:30.684 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:30.684 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:30.684 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:30.684 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:30.684 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:30.684 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:30.684 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:30.684 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:30.684 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:30.684 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:30.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.664+0000 7f5e958d1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:30.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.664+0000 7f5e958d1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:30.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.664+0000 7f5e958d1640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:30.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.664+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f5e80003d70 con 0x7f5e90133bf0 2024-01-24T23:27:30.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.664+0000 7f5e958d1640 1 -- 172.21.15.115:0/3138974938 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e901202f0 con 0x7f5e90133bf0 2024-01-24T23:27:30.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.664+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(215..215 src has 1..215) v4 ==== 13748+0+0 (unknown 2317821301 0 0) 0x7f5e80045e50 con 0x7f5e90133bf0 2024-01-24T23:27:30.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.664+0000 7f5e867fc640 10 monclient: get_auth_request con 0x7f5e70037b10 auth_method 0 2024-01-24T23:27:30.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.668+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f5e80014790 con 0x7f5e90133bf0 2024-01-24T23:27:30.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.668+0000 7f5e8cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:30.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.668+0000 7f5e8cff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:30.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.884+0000 7f5e958d1640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-01-24T23:27:30.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.884+0000 7f5e958d1640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-01-24T23:27:30.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.884+0000 7f5e958d1640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:30.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.884+0000 7f5e958d1640 1 -- 172.21.15.115:0/3138974938 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f5e9006a0d0 con 0x7f5e90133bf0 2024-01-24T23:27:30.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.884+0000 7f5e8cff9640 1 -- 172.21.15.115:0/3138974938 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2744 (unknown 254746657 0 2042493446) 0x7f5e80014790 con 0x7f5e90133bf0 2024-01-24T23:27:30.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.884+0000 7f5e8cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-01-24T23:27:30.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.884+0000 7f5e8cff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:30.887 INFO:teuthology.orchestra.run.smithi115.stdout:{"election_epoch":40,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":0,"features":{"quorum_con":"4540701272860131327","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","modified":"2024-01-24T23:20:20.557246Z","created":"2024-01-24T23:20:20.557246Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6790","nonce":0}]},"addr":"172.21.15.115:6790/0","public_addr":"172.21.15.115:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6790","nonce":0}]},"addr":"172.21.15.132:6790/0","public_addr":"172.21.15.132:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6791","nonce":0}]},"addr":"172.21.15.115:6791/0","public_addr":"172.21.15.115:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6791","nonce":0}]},"addr":"172.21.15.132:6791/0","public_addr":"172.21.15.132:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6792","nonce":0}]},"addr":"172.21.15.115:6792/0","public_addr":"172.21.15.115:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6792","nonce":0}]},"addr":"172.21.15.132:6792/0","public_addr":"172.21.15.132:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6793","nonce":0}]},"addr":"172.21.15.115:6793/0","public_addr":"172.21.15.115:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-01-24T23:27:30.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.884+0000 7f5e958d1640 1 -- 172.21.15.115:0/3138974938 >> v1:172.21.15.132:6812/1835401922 conn(0x7f5e70037b10 legacy=0x7f5e70039fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:30.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.884+0000 7f5e958d1640 10 monclient: shutdown 2024-01-24T23:27:30.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.884+0000 7f5e958d1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:30.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.884+0000 7f5e958d1640 1 -- 172.21.15.115:0/3138974938 >> v1:172.21.15.115:6791/0 conn(0x7f5e90133bf0 legacy=0x7f5e90132e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:30.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.884+0000 7f5e958d1640 1 -- 172.21.15.115:0/3138974938 shutdown_connections 2024-01-24T23:27:30.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.884+0000 7f5e958d1640 1 -- 172.21.15.115:0/3138974938 >> 172.21.15.115:0/3138974938 conn(0x7f5e900662f0 msgr2=0x7f5e901bec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:30.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.884+0000 7f5e958d1640 1 -- 172.21.15.115:0/3138974938 shutdown_connections 2024-01-24T23:27:30.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:30.884+0000 7f5e958d1640 1 -- 172.21.15.115:0/3138974938 wait complete. 2024-01-24T23:27:30.905 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-01-24T23:27:30.905 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-01-24T23:27:30.905 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-01-24T23:27:31.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.014+0000 7f703e7fc640 1 -- 172.21.15.132:0/1834108242 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h4"}]=0 removed item id -28 name 'h4' from crush map v216) v1 ==== 118+0+0 (unknown 1334860049 0 0) 0x7f7040013180 con 0x7f70481365f0 2024-01-24T23:27:31.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.014+0000 7f703e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h4"}] 2024-01-24T23:27:31.017 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.014+0000 7f703e7fc640 10 monclient: _finish_command 2 = system:0 removed item id -28 name 'h4' from crush map 2024-01-24T23:27:31.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.020+0000 7f490811e640 10 monclient: get_monmap_and_config 2024-01-24T23:27:31.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.020+0000 7f490811e640 10 monclient: build_initial_monmap 2024-01-24T23:27:31.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.020+0000 7f490811e640 10 monclient: monmap: 2024-01-24T23:27:31.024 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:31.024 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.024 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:31.025375+0000 2024-01-24T23:27:31.024 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:31.025375+0000 2024-01-24T23:27:31.024 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:31.024 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.024 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:31.024 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:31.024 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:31.024 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:31.024 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:31.024 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:31.025 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:31.025 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:31.025 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:31.025 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.020+0000 7f490811e640 1 Processor -- start 2024-01-24T23:27:31.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.020+0000 7f490811e640 1 -- start start 2024-01-24T23:27:31.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.020+0000 7f490811e640 10 monclient: init 2024-01-24T23:27:31.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f490811e640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:31.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f490811e640 10 monclient: _add_conns ranks=[3,5,6,8,1,4,0,2,7] 2024-01-24T23:27:31.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f490811e640 10 monclient(hunting): picked mon.noname-g con 0x7f490012fc60 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:31.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f490811e640 10 monclient(hunting): picked mon.noname-h con 0x7f490009f450 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:31.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f490811e640 10 monclient(hunting): picked mon.noname-d con 0x7f49001398a0 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:31.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f490811e640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490004ed70 con 0x7f490012fc60 2024-01-24T23:27:31.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f490811e640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490009e860 con 0x7f490009f450 2024-01-24T23:27:31.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f490811e640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490009eb40 con 0x7f49001398a0 2024-01-24T23:27:31.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f490811e640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:31.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f490811e640 10 monclient(hunting): authenticate will time out at 1329.948355s 2024-01-24T23:27:31.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f49066bb640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f49001398a0 0x7f490013bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:40390/0 (socket says 172.21.15.115:40390) 2024-01-24T23:27:31.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f49066bb640 1 -- 172.21.15.115:0/425080230 learned_addr learned my addr 172.21.15.115:0/425080230 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:31.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 1 -- 172.21.15.115:0/425080230 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936153600 0 0) 0x7f490009eb40 con 0x7f49001398a0 2024-01-24T23:27:31.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 10 monclient(hunting): my global_id is 6693 2024-01-24T23:27:31.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 1 -- 172.21.15.115:0/425080230 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48ec003530 con 0x7f49001398a0 2024-01-24T23:27:31.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 1 -- 172.21.15.115:0/425080230 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684145535 0 0) 0x7f490004ed70 con 0x7f490012fc60 2024-01-24T23:27:31.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 10 monclient(hunting): my global_id is 6546 2024-01-24T23:27:31.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 1 -- 172.21.15.115:0/425080230 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f490009eb40 con 0x7f490012fc60 2024-01-24T23:27:31.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 1 -- 172.21.15.115:0/425080230 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3398737204 0 0) 0x7f48ec003530 con 0x7f49001398a0 2024-01-24T23:27:31.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 1 -- 172.21.15.115:0/425080230 >> v1:172.21.15.132:6791/0 conn(0x7f490009f450 legacy=0x7f4900138180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 1 -- 172.21.15.115:0/425080230 >> v1:172.21.15.132:6790/0 conn(0x7f490012fc60 legacy=0x7f4900130040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 10 monclient: _finish_hunting 0 2024-01-24T23:27:31.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 1 monclient: found mon.noname-d 2024-01-24T23:27:31.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:31.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:27:31.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 1 -- 172.21.15.115:0/425080230 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490009eff0 con 0x7f49001398a0 2024-01-24T23:27:31.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 10 monclient: _finish_auth 0 2024-01-24T23:27:31.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 10 monclient: _check_auth_tickets 2024-01-24T23:27:31.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:31.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 1 -- 172.21.15.115:0/425080230 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f48f8001ac0 con 0x7f49001398a0 2024-01-24T23:27:31.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f490811e640 5 monclient: authenticate success, global_id 6693 2024-01-24T23:27:31.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f490811e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:31.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:27:31.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 10 monclient: dump: 2024-01-24T23:27:31.031 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.031 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.032 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.032 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.032 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.032 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.032 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.032 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.032 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.032 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.032 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.032 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.032 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.032 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.033 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f490811e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 1 -- 172.21.15.115:0/425080230 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f48f8001ce0 con 0x7f49001398a0 2024-01-24T23:27:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 1 -- 172.21.15.115:0/425080230 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f48f8004c30 con 0x7f49001398a0 2024-01-24T23:27:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f4904eb8640 10 monclient: dump: 2024-01-24T23:27:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.033 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.034 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.034 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.034 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.034 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.034 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.034 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.034 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.034 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.034 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.034 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.034 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f490811e640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:31.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f490811e640 4 set_mon_vals no callback set 2024-01-24T23:27:31.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f490811e640 10 monclient: shutdown 2024-01-24T23:27:31.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f490811e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:31.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f490811e640 1 -- 172.21.15.115:0/425080230 >> v1:172.21.15.115:6792/0 conn(0x7f49001398a0 legacy=0x7f490013bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f490811e640 1 -- 172.21.15.115:0/425080230 shutdown_connections 2024-01-24T23:27:31.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f490811e640 1 -- 172.21.15.115:0/425080230 >> 172.21.15.115:0/425080230 conn(0x7f49000a2170 msgr2=0x7f49000a2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:31.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f490811e640 1 -- 172.21.15.115:0/425080230 shutdown_connections 2024-01-24T23:27:31.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.024+0000 7f490811e640 1 -- 172.21.15.115:0/425080230 wait complete. 2024-01-24T23:27:31.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f490811e640 10 monclient: build_initial_monmap 2024-01-24T23:27:31.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f490811e640 10 monclient: monmap: 2024-01-24T23:27:31.035 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:31.035 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:31.035 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:31.035 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:31.035 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:31.036 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:31.036 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:31.036 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:31.036 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:31.036 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:31.036 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:31.036 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:31.036 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:31.036 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:31.036 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:31.036 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f490811e640 1 Processor -- start 2024-01-24T23:27:31.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f490811e640 1 -- start start 2024-01-24T23:27:31.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f490811e640 10 monclient: init 2024-01-24T23:27:31.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f490811e640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:31.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f490811e640 10 monclient: _add_conns ranks=[5,2,1,3,4,0,8,6,7] 2024-01-24T23:27:31.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f490811e640 10 monclient(hunting): picked mon.noname-h con 0x7f490009f450 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:31.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f490811e640 10 monclient(hunting): picked mon.noname-b con 0x7f490012fc60 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:31.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f490811e640 10 monclient(hunting): picked mon.noname-f con 0x7f49001398a0 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:31.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f490811e640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4900131cf0 con 0x7f49001398a0 2024-01-24T23:27:31.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f490811e640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490009da20 con 0x7f490012fc60 2024-01-24T23:27:31.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f490811e640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490012ec90 con 0x7f490009f450 2024-01-24T23:27:31.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f490811e640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:31.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f490811e640 10 monclient(hunting): authenticate will time out at 1329.953064s 2024-01-24T23:27:31.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f49056b9640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f490012fc60 0x7f490012dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:36650/0 (socket says 172.21.15.115:36650) 2024-01-24T23:27:31.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f49056b9640 1 -- 172.21.15.115:0/3514739644 learned_addr learned my addr 172.21.15.115:0/3514739644 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:31.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441236063 0 0) 0x7f490009da20 con 0x7f490012fc60 2024-01-24T23:27:31.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient(hunting): my global_id is 124112 2024-01-24T23:27:31.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48e4003530 con 0x7f490012fc60 2024-01-24T23:27:31.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3836570212 0 0) 0x7f48e4003530 con 0x7f490012fc60 2024-01-24T23:27:31.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f490009da20 con 0x7f490012fc60 2024-01-24T23:27:31.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f48f0001b20 con 0x7f490012fc60 2024-01-24T23:27:31.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:27:31.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient(hunting): dump: 2024-01-24T23:27:31.044 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.044 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.044 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.044 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.044 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.044 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.044 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.044 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.044 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.044 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.044 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.044 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.044 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.044 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.045 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.045 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:31.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362668781 0 0) 0x7f490012ec90 con 0x7f490009f450 2024-01-24T23:27:31.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient(hunting): my global_id is 6512 2024-01-24T23:27:31.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48e4003530 con 0x7f490009f450 2024-01-24T23:27:31.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3466077013 0 0) 0x7f490009da20 con 0x7f490012fc60 2024-01-24T23:27:31.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 >> v1:172.21.15.132:6791/0 conn(0x7f490009f450 legacy=0x7f490012d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 >> v1:172.21.15.132:6789/0 conn(0x7f49001398a0 legacy=0x7f49001d3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient: _finish_hunting 0 2024-01-24T23:27:31.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 monclient: found mon.b 2024-01-24T23:27:31.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:31.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:27:31.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4900131fd0 con 0x7f490012fc60 2024-01-24T23:27:31.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient: _finish_auth 0 2024-01-24T23:27:31.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient: _check_auth_tickets 2024-01-24T23:27:31.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:31.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f490811e640 5 monclient: authenticate success, global_id 124112 2024-01-24T23:27:31.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f490811e640 10 monclient: _renew_subs 2024-01-24T23:27:31.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f490811e640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:27:31.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f490811e640 1 -- 172.21.15.115:0/3514739644 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490012f310 con 0x7f490012fc60 2024-01-24T23:27:31.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f490811e640 10 monclient: _renew_subs 2024-01-24T23:27:31.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f490811e640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:27:31.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f490811e640 1 -- 172.21.15.115:0/3514739644 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f49000729a0 con 0x7f490012fc60 2024-01-24T23:27:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f48f0003950 con 0x7f490012fc60 2024-01-24T23:27:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f48f0003e80 con 0x7f490012fc60 2024-01-24T23:27:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:27:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f7fff640 4 set_mon_vals no callback set 2024-01-24T23:27:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient: dump: 2024-01-24T23:27:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.048 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.048 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.048 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.048 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.048 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.048 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.048 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.048 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.048 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.049 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f490811e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:31.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f490811e640 1 -- 172.21.15.115:0/3514739644 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f48e80065d0 con 0x7f48e8002870 2024-01-24T23:27:31.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f48f0003fe0 con 0x7f490012fc60 2024-01-24T23:27:31.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(216..216 src has 1..216) v4 ==== 13538+0+0 (unknown 289362477 0 0) 0x7f48f0045ed0 con 0x7f490012fc60 2024-01-24T23:27:31.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f49066bb640 10 monclient: get_auth_request con 0x7f48e4037040 auth_method 0 2024-01-24T23:27:31.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3971470454 0 0) 0x7f48e80065d0 con 0x7f48e8002870 2024-01-24T23:27:31.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient(hunting): my global_id is 94887 2024-01-24T23:27:31.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f490009da20 con 0x7f48e8002870 2024-01-24T23:27:31.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 943404863 0 0) 0x7f490009da20 con 0x7f48e8002870 2024-01-24T23:27:31.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f48e80065d0 con 0x7f48e8002870 2024-01-24T23:27:31.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f48fc0018b0 con 0x7f48e8002870 2024-01-24T23:27:31.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:27:31.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient: dump: 2024-01-24T23:27:31.050 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.050 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.050 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.050 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.050 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.050 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.050 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.050 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.050 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.050 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.051 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.051 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.051 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.051 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.051 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.051 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2003833761 0 0) 0x7f48e80065d0 con 0x7f48e8002870 2024-01-24T23:27:31.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 --> v1:172.21.15.115:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f48e800a120 con 0x7f48e8002870 2024-01-24T23:27:31.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 <== mon.0 v1:172.21.15.115:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f48fc003600 con 0x7f48e8002870 2024-01-24T23:27:31.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:31.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:31.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 >> v1:172.21.15.115:6789/0 conn(0x7f48e8002870 legacy=0x7f48e8004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.028+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 >> v1:172.21.15.115:6789/0 conn(0x7f48e8002870 legacy=0x7f48e8004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:31.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f490811e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:27:31.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f490811e640 1 -- 172.21.15.115:0/3514739644 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f48e8009df0 con 0x7f48e80090b0 2024-01-24T23:27:31.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2884129090 0 0) 0x7f48e8009df0 con 0x7f48e80090b0 2024-01-24T23:27:31.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f48f67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f48f67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f48f67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f48f67fc640 10 monclient(hunting): my global_id is 94896 2024-01-24T23:27:31.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48e80065d0 con 0x7f48e80090b0 2024-01-24T23:27:31.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3961744506 0 0) 0x7f48e80065d0 con 0x7f48e80090b0 2024-01-24T23:27:31.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f48e8009df0 con 0x7f48e80090b0 2024-01-24T23:27:31.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f48fc003d60 con 0x7f48e80090b0 2024-01-24T23:27:31.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f48f67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f48f67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:27:31.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f48f67fc640 10 monclient: dump: 2024-01-24T23:27:31.052 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.052 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.053 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1155559209 0 0) 0x7f48e8009df0 con 0x7f48e80090b0 2024-01-24T23:27:31.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 --> v1:172.21.15.115:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f48e800c310 con 0x7f48e80090b0 2024-01-24T23:27:31.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 <== mon.0 v1:172.21.15.115:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 2919041874) 0x7f48fc004140 con 0x7f48e80090b0 2024-01-24T23:27:31.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f48f67fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:27:31.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f48f67fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:31.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 >> v1:172.21.15.115:6789/0 conn(0x7f48e80090b0 legacy=0x7f48e8009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f48f67fc640 1 -- 172.21.15.115:0/3514739644 >> v1:172.21.15.115:6789/0 conn(0x7f48e80090b0 legacy=0x7f48e8009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:31.054 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:27:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:27:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 40, 2024-01-24T23:27:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:27:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:27:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:27:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:27:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:27:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:27:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:27:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:27:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:27:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:27:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:27:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:27:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:27:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:27:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:27:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:27:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:27:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:27:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:27:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:27:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:27:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:27:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:27:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:27:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:27:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:27:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:27:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:27:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:27:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:27:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:27:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:27:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:27:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:27:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:27:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:27:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:27:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:27:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:27:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:27:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:27:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:27:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:27:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:27:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:27:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:27:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:27:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:27:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:27:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:27:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:27:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:27:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:27:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:27:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:27:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:27:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:27:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:27:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:27:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:27:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:27:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:27:31.072 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:27:31.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f490811e640 1 -- 172.21.15.115:0/3514739644 >> v1:172.21.15.132:6812/1835401922 conn(0x7f48e4037040 legacy=0x7f48e4039500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f490811e640 10 monclient: shutdown 2024-01-24T23:27:31.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f490811e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:31.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f490811e640 1 -- 172.21.15.115:0/3514739644 >> v1:172.21.15.115:6790/0 conn(0x7f490012fc60 legacy=0x7f490012dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f49066bb640 1 -- 172.21.15.115:0/3514739644 reap_dead start 2024-01-24T23:27:31.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f49066bb640 1 -- 172.21.15.115:0/3514739644 reap_dead start 2024-01-24T23:27:31.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f490811e640 1 -- 172.21.15.115:0/3514739644 shutdown_connections 2024-01-24T23:27:31.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f490811e640 1 -- 172.21.15.115:0/3514739644 >> 172.21.15.115:0/3514739644 conn(0x7f49000a2170 msgr2=0x7f4900098460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:31.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f490811e640 1 -- 172.21.15.115:0/3514739644 shutdown_connections 2024-01-24T23:27:31.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.036+0000 7f490811e640 1 -- 172.21.15.115:0/3514739644 wait complete. 2024-01-24T23:27:31.074 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:27:31.074 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-01-24T23:27:31.075 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-01-24T23:27:31.118 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.114+0000 7f70501da640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h4"}] 2024-01-24T23:27:31.118 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.114+0000 7f70501da640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h4"}] 2024-01-24T23:27:31.118 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.114+0000 7f70501da640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:31.118 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.114+0000 7f70501da640 1 -- 172.21.15.132:0/1834108242 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rm", "name": "h4"} v 0) v1 -- 0x7f7030002220 con 0x7f70481365f0 2024-01-24T23:27:31.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.118+0000 7f703e7fc640 1 -- 172.21.15.132:0/1834108242 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h4"}]=0 device 'h4' does not appear in the crush map v216) v1 ==== 118+0+0 (unknown 3404819776 0 0) 0x7f7040014900 con 0x7f70481365f0 2024-01-24T23:27:31.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.118+0000 7f703e7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h4"}] 2024-01-24T23:27:31.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.118+0000 7f703e7fc640 10 monclient: _finish_command 3 = system:0 device 'h4' does not appear in the crush map 2024-01-24T23:27:31.120 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h4' does not appear in the crush map 2024-01-24T23:27:31.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.118+0000 7f70501da640 1 -- 172.21.15.132:0/1834108242 >> v1:172.21.15.132:6812/1835401922 conn(0x7f702c036e80 legacy=0x7f702c039340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.118+0000 7f70501da640 10 monclient: shutdown 2024-01-24T23:27:31.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.118+0000 7f70501da640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:31.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.118+0000 7f70501da640 1 -- 172.21.15.132:0/1834108242 >> v1:172.21.15.132:6790/0 conn(0x7f70481365f0 legacy=0x7f70481d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.118+0000 7f70501da640 1 -- 172.21.15.132:0/1834108242 shutdown_connections 2024-01-24T23:27:31.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.118+0000 7f70501da640 1 -- 172.21.15.132:0/1834108242 >> 172.21.15.132:0/1834108242 conn(0x7f70480a6790 msgr2=0x7f704806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:31.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.118+0000 7f70501da640 1 -- 172.21.15.132:0/1834108242 shutdown_connections 2024-01-24T23:27:31.123 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.122+0000 7f70501da640 1 -- 172.21.15.132:0/1834108242 wait complete. 2024-01-24T23:27:31.137 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:31.137 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h5 2024-01-24T23:27:31.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5f0520640 10 monclient: get_monmap_and_config 2024-01-24T23:27:31.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5f0520640 10 monclient: build_initial_monmap 2024-01-24T23:27:31.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5f0520640 10 monclient: monmap: 2024-01-24T23:27:31.190 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:31.190 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.190 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:31.191755+0000 2024-01-24T23:27:31.190 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:31.191755+0000 2024-01-24T23:27:31.190 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:31.191 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.191 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:31.191 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:31.191 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:31.191 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:31.191 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:31.191 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:31.191 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:31.191 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:31.191 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:31.191 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5f0520640 1 Processor -- start 2024-01-24T23:27:31.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5f0520640 1 -- start start 2024-01-24T23:27:31.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5f0520640 10 monclient: init 2024-01-24T23:27:31.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5f0520640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:31.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5f0520640 10 monclient: _add_conns ranks=[0,1,2,3,6,5,8,4,7] 2024-01-24T23:27:31.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5f0520640 10 monclient(hunting): picked mon.noname-a con 0x7ff5e8133bf0 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:31.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5f0520640 10 monclient(hunting): picked mon.noname-f con 0x7ff5e8133fd0 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:31.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5f0520640 10 monclient(hunting): picked mon.noname-b con 0x7ff5e8134430 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:31.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5f0520640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e804ed70 con 0x7ff5e8133bf0 2024-01-24T23:27:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5f0520640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e809af30 con 0x7ff5e8133fd0 2024-01-24T23:27:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5f0520640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e809b210 con 0x7ff5e8134430 2024-01-24T23:27:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5f0520640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5f0520640 10 monclient(hunting): authenticate will time out at 1330.115113s 2024-01-24T23:27:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5ee2bc640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff5e8133bf0 0x7ff5e809bd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:60586/0 (socket says 172.21.15.115:60586) 2024-01-24T23:27:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5ee2bc640 1 -- 172.21.15.115:0/1207618730 learned_addr learned my addr 172.21.15.115:0/1207618730 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5ed2ba640 1 -- 172.21.15.115:0/1207618730 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 903118088 0 0) 0x7ff5e804ed70 con 0x7ff5e8133bf0 2024-01-24T23:27:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5ed2ba640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5ed2ba640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5ed2ba640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5ed2ba640 10 monclient(hunting): my global_id is 94905 2024-01-24T23:27:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5ed2ba640 1 -- 172.21.15.115:0/1207618730 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5d4003530 con 0x7ff5e8133bf0 2024-01-24T23:27:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5ed2ba640 1 -- 172.21.15.115:0/1207618730 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317959463 0 0) 0x7ff5e809b210 con 0x7ff5e8134430 2024-01-24T23:27:31.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5ed2ba640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5ed2ba640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5ed2ba640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5ed2ba640 10 monclient(hunting): my global_id is 124121 2024-01-24T23:27:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5ed2ba640 1 -- 172.21.15.115:0/1207618730 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5e804ed70 con 0x7ff5e8134430 2024-01-24T23:27:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5ed2ba640 1 -- 172.21.15.115:0/1207618730 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2616447486 0 0) 0x7ff5d4003530 con 0x7ff5e8133bf0 2024-01-24T23:27:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5ed2ba640 1 -- 172.21.15.115:0/1207618730 >> v1:172.21.15.115:6790/0 conn(0x7ff5e8134430 legacy=0x7ff5e813f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5ed2ba640 1 -- 172.21.15.115:0/1207618730 >> v1:172.21.15.132:6789/0 conn(0x7ff5e8133fd0 legacy=0x7ff5e813bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5ed2ba640 10 monclient: _finish_hunting 0 2024-01-24T23:27:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5ed2ba640 1 monclient: found mon.noname-a 2024-01-24T23:27:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5ed2ba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5ed2ba640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:27:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.188+0000 7ff5ed2ba640 1 -- 172.21.15.115:0/1207618730 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e809b4f0 con 0x7ff5e8133bf0 2024-01-24T23:27:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5ed2ba640 10 monclient: _finish_auth 0 2024-01-24T23:27:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5ed2ba640 10 monclient: _check_auth_tickets 2024-01-24T23:27:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5ed2ba640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:31.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5ed2ba640 1 -- 172.21.15.115:0/1207618730 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff5e40017c0 con 0x7ff5e8133bf0 2024-01-24T23:27:31.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 5 monclient: authenticate success, global_id 94905 2024-01-24T23:27:31.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:31.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5ed2ba640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5ed2ba640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:27:31.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5ed2ba640 10 monclient: dump: 2024-01-24T23:27:31.202 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.202 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.202 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.202 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.202 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.202 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.202 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.202 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.203 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5ed2ba640 1 -- 172.21.15.115:0/1207618730 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff5e4001aa0 con 0x7ff5e8133bf0 2024-01-24T23:27:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5ed2ba640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5ed2ba640 1 -- 172.21.15.115:0/1207618730 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff5e4004bf0 con 0x7ff5e8133bf0 2024-01-24T23:27:31.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5ed2ba640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5ed2ba640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:27:31.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5ed2ba640 10 monclient: dump: 2024-01-24T23:27:31.204 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.204 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.204 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.204 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.204 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.204 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.204 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.204 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.204 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.204 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.204 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.204 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.205 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.205 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.205 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.205 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:31.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 4 set_mon_vals no callback set 2024-01-24T23:27:31.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 10 monclient: shutdown 2024-01-24T23:27:31.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:31.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 1 -- 172.21.15.115:0/1207618730 >> v1:172.21.15.115:6789/0 conn(0x7ff5e8133bf0 legacy=0x7ff5e809bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 1 -- 172.21.15.115:0/1207618730 shutdown_connections 2024-01-24T23:27:31.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 1 -- 172.21.15.115:0/1207618730 >> 172.21.15.115:0/1207618730 conn(0x7ff5e80a9910 msgr2=0x7ff5e80a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:31.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 1 -- 172.21.15.115:0/1207618730 shutdown_connections 2024-01-24T23:27:31.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 1 -- 172.21.15.115:0/1207618730 wait complete. 2024-01-24T23:27:31.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 10 monclient: build_initial_monmap 2024-01-24T23:27:31.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 10 monclient: monmap: 2024-01-24T23:27:31.206 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:31.206 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:31.206 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:31.206 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:31.206 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:31.206 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:31.206 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:31.206 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:31.206 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:31.206 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:31.207 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:31.207 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:31.207 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:31.207 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:31.207 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:31.207 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 1 Processor -- start 2024-01-24T23:27:31.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 1 -- start start 2024-01-24T23:27:31.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 10 monclient: init 2024-01-24T23:27:31.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:31.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 10 monclient: _add_conns ranks=[5,7,8,2,6,1,3,4,0] 2024-01-24T23:27:31.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 10 monclient(hunting): picked mon.noname-h con 0x7ff5e8133bf0 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:31.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 10 monclient(hunting): picked mon.noname-i con 0x7ff5e8133fd0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:31.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 10 monclient(hunting): picked mon.noname-e con 0x7ff5e8134430 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:31.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e80a5960 con 0x7ff5e8133bf0 2024-01-24T23:27:31.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e813a510 con 0x7ff5e8133fd0 2024-01-24T23:27:31.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e813a7f0 con 0x7ff5e8134430 2024-01-24T23:27:31.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:31.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 10 monclient(hunting): authenticate will time out at 1330.118409s 2024-01-24T23:27:31.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5eeabd640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7ff5e8134430 0x7ff5e81d3d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:48992/0 (socket says 172.21.15.115:48992) 2024-01-24T23:27:31.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5eeabd640 1 -- 172.21.15.115:0/127122860 learned_addr learned my addr 172.21.15.115:0/127122860 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:31.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3232003299 0 0) 0x7ff5e813a7f0 con 0x7ff5e8134430 2024-01-24T23:27:31.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 10 monclient(hunting): my global_id is 85760 2024-01-24T23:27:31.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5cc003530 con 0x7ff5e8134430 2024-01-24T23:27:31.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3402478557 0 0) 0x7ff5e80a5960 con 0x7ff5e8133bf0 2024-01-24T23:27:31.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 10 monclient(hunting): my global_id is 6521 2024-01-24T23:27:31.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5e813a7f0 con 0x7ff5e8133bf0 2024-01-24T23:27:31.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 48468726 0 0) 0x7ff5cc003530 con 0x7ff5e8134430 2024-01-24T23:27:31.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff5e80a5960 con 0x7ff5e8134430 2024-01-24T23:27:31.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964082093 0 0) 0x7ff5e813a510 con 0x7ff5e8133fd0 2024-01-24T23:27:31.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 10 monclient(hunting): my global_id is 124117 2024-01-24T23:27:31.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5cc003530 con 0x7ff5e8133fd0 2024-01-24T23:27:31.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff5e0001d20 con 0x7ff5e8134430 2024-01-24T23:27:31.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:27:31.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 10 monclient(hunting): dump: 2024-01-24T23:27:31.210 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.210 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.210 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.210 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.210 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.210 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.210 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.210 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.210 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.210 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.211 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.211 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.211 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.211 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.211 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.211 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:31.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 327481938 0 0) 0x7ff5e80a5960 con 0x7ff5e8134430 2024-01-24T23:27:31.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 >> v1:172.21.15.132:6792/0 conn(0x7ff5e8133fd0 legacy=0x7ff5e81d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 >> v1:172.21.15.132:6791/0 conn(0x7ff5e8133bf0 legacy=0x7ff5e81cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 10 monclient: _finish_hunting 0 2024-01-24T23:27:31.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 1 monclient: found mon.e 2024-01-24T23:27:31.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:31.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:27:31.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e813aad0 con 0x7ff5e8134430 2024-01-24T23:27:31.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 10 monclient: _finish_auth 0 2024-01-24T23:27:31.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 10 monclient: _check_auth_tickets 2024-01-24T23:27:31.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:31.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 5 monclient: authenticate success, global_id 85760 2024-01-24T23:27:31.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 10 monclient: _renew_subs 2024-01-24T23:27:31.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:27:31.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 1 -- 172.21.15.115:0/127122860 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e8139620 con 0x7ff5e8134430 2024-01-24T23:27:31.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 10 monclient: _renew_subs 2024-01-24T23:27:31.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:27:31.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5f0520640 1 -- 172.21.15.115:0/127122860 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e8139b30 con 0x7ff5e8134430 2024-01-24T23:27:31.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff5e00020b0 con 0x7ff5e8134430 2024-01-24T23:27:31.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:31.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff5e0003f90 con 0x7ff5e8134430 2024-01-24T23:27:31.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5ecab9640 4 set_mon_vals no callback set 2024-01-24T23:27:31.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:27:31.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.192+0000 7ff5deffd640 10 monclient: dump: 2024-01-24T23:27:31.213 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.213 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.213 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.213 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.213 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.213 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.213 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.213 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.213 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.213 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.214 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.214 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.214 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.214 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.214 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.214 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.196+0000 7ff5f0520640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:31.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.196+0000 7ff5f0520640 1 -- 172.21.15.115:0/127122860 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff5d00065d0 con 0x7ff5d0002870 2024-01-24T23:27:31.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.196+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7ff5e00040f0 con 0x7ff5e8134430 2024-01-24T23:27:31.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.196+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(216..216 src has 1..216) v4 ==== 13538+0+0 (unknown 289362477 0 0) 0x7ff5e0045fc0 con 0x7ff5e8134430 2024-01-24T23:27:31.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.196+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 680178489 0 0) 0x7ff5d00065d0 con 0x7ff5d0002870 2024-01-24T23:27:31.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.196+0000 7ff5deffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.196+0000 7ff5deffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.196+0000 7ff5deffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.196+0000 7ff5deffd640 10 monclient(hunting): my global_id is 124130 2024-01-24T23:27:31.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.196+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5e80a5960 con 0x7ff5d0002870 2024-01-24T23:27:31.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.196+0000 7ff5edabb640 10 monclient: get_auth_request con 0x7ff5cc0371e0 auth_method 0 2024-01-24T23:27:31.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.196+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1635549936 0 0) 0x7ff5e80a5960 con 0x7ff5d0002870 2024-01-24T23:27:31.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.196+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff5d00065d0 con 0x7ff5d0002870 2024-01-24T23:27:31.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.196+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff5e4001780 con 0x7ff5d0002870 2024-01-24T23:27:31.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.196+0000 7ff5deffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.196+0000 7ff5deffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:27:31.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.196+0000 7ff5deffd640 10 monclient: dump: 2024-01-24T23:27:31.215 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.215 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.215 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.215 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.215 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.215 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.215 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.216 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.216 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.216 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.216 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.216 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.216 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.216 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.216 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.216 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.196+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1826194267 0 0) 0x7ff5d00065d0 con 0x7ff5d0002870 2024-01-24T23:27:31.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.196+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 --> v1:172.21.15.115:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5d000a120 con 0x7ff5d0002870 2024-01-24T23:27:31.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.196+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 <== mon.2 v1:172.21.15.115:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7ff5e4004a20 con 0x7ff5d0002870 2024-01-24T23:27:31.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.196+0000 7ff5deffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:31.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.196+0000 7ff5deffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:31.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.196+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 >> v1:172.21.15.115:6790/0 conn(0x7ff5d0002870 legacy=0x7ff5d0004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.196+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 >> v1:172.21.15.115:6790/0 conn(0x7ff5d0002870 legacy=0x7ff5d0004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:31.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.200+0000 7ff5f0520640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:27:31.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.200+0000 7ff5f0520640 1 -- 172.21.15.115:0/127122860 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff5d0009e10 con 0x7ff5d00090d0 2024-01-24T23:27:31.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2851890780 0 0) 0x7ff5d0009e10 con 0x7ff5d00090d0 2024-01-24T23:27:31.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5deffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5deffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5deffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5deffd640 10 monclient(hunting): my global_id is 124139 2024-01-24T23:27:31.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5d00065d0 con 0x7ff5d00090d0 2024-01-24T23:27:31.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2375799656 0 0) 0x7ff5d00065d0 con 0x7ff5d00090d0 2024-01-24T23:27:31.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff5d0009e10 con 0x7ff5d00090d0 2024-01-24T23:27:31.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff5e4005340 con 0x7ff5d00090d0 2024-01-24T23:27:31.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5deffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5deffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:27:31.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5deffd640 10 monclient: dump: 2024-01-24T23:27:31.219 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.219 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.219 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.219 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.219 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.219 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.219 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.219 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.220 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.220 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.220 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.220 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.220 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.220 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.220 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.220 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1708047253 0 0) 0x7ff5d0009e10 con 0x7ff5d00090d0 2024-01-24T23:27:31.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 --> v1:172.21.15.115:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff5d000c310 con 0x7ff5d00090d0 2024-01-24T23:27:31.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 <== mon.2 v1:172.21.15.115:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 4000107716) 0x7ff5e4007680 con 0x7ff5d00090d0 2024-01-24T23:27:31.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5deffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:27:31.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5deffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:31.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 >> v1:172.21.15.115:6790/0 conn(0x7ff5d00090d0 legacy=0x7ff5d0009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5deffd640 1 -- 172.21.15.115:0/127122860 >> v1:172.21.15.115:6790/0 conn(0x7ff5d00090d0 legacy=0x7ff5d0009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:31.221 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:27:31.221 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:31.221 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:31.221 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:27:31.221 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 40, 2024-01-24T23:27:31.221 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:27:31.221 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:27:31.221 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:27:31.221 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:27:31.222 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:27:31.222 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:27:31.222 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:27:31.222 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:27:31.222 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:27:31.222 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:27:31.222 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.222 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:27:31.222 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:31.222 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:27:31.222 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:27:31.222 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:31.222 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:31.222 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:31.223 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:31.223 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:31.223 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:31.223 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:31.223 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:31.223 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:31.223 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:31.223 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:31.223 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.223 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:27:31.223 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:27:31.223 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:31.223 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:31.223 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:31.223 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:31.224 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:31.224 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:31.224 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:31.224 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:31.224 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:31.224 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:31.224 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:31.224 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.224 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:27:31.224 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:27:31.224 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:27:31.224 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:27:31.225 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:27:31.225 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:27:31.225 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:31.225 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:31.225 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:27:31.225 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:27:31.225 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:27:31.225 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:27:31.225 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:27:31.225 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:27:31.225 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:27:31.225 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:31.225 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:27:31.225 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:31.225 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:31.226 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:31.226 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:31.226 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:31.226 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:31.226 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:31.226 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:31.226 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:31.226 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:31.226 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:31.226 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.226 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:27:31.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.226 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:27:31.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.226 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:31.227 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:31.227 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.227 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.227 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.227 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:27:31.227 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.227 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.227 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.227 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:27:31.227 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:27:31.227 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.227 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.228 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.228 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:31.228 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:31.228 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.228 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.228 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.228 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:27:31.228 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.228 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.228 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.229 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:27:31.229 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:27:31.229 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.229 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.229 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.229 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:31.229 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:31.229 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.229 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.229 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.229 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:27:31.229 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.229 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.230 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.230 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:27:31.230 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:27:31.230 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.230 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.230 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.230 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:31.230 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:31.230 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.230 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.230 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.230 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:27:31.231 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.231 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.231 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.231 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:27:31.231 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:27:31.231 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.231 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.231 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.231 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:31.231 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:31.231 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.231 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.232 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.232 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:27:31.232 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.232 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.232 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.232 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:27:31.232 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:27:31.232 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.232 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.232 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.232 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:31.232 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:31.233 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.233 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.233 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.233 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:27:31.233 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.233 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.233 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.233 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:27:31.233 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:27:31.233 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.233 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.233 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.234 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:31.234 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:31.234 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.234 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.234 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.234 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:27:31.234 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.234 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.234 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.234 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:27:31.234 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:27:31.234 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.234 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.234 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.235 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:31.235 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:31.235 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.235 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.235 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.235 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:27:31.235 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.235 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.235 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.236 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:27:31.236 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:27:31.236 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.236 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.236 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.236 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:31.236 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:31.236 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.236 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.236 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.237 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:27:31.237 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.237 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.237 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.237 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:27:31.237 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:27:31.237 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.237 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.237 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.237 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.237 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.237 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:27:31.238 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:27:31.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.238 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:31.238 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:31.238 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:31.238 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.238 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.238 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:27:31.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.238 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:31.238 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:31.238 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:31.238 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.238 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.239 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:27:31.239 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:27:31.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5f0520640 1 -- 172.21.15.115:0/127122860 >> v1:172.21.15.132:6812/1835401922 conn(0x7ff5cc0371e0 legacy=0x7ff5cc0396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5f0520640 10 monclient: shutdown 2024-01-24T23:27:31.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5f0520640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:31.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5f0520640 1 -- 172.21.15.115:0/127122860 >> v1:172.21.15.115:6793/0 conn(0x7ff5e8134430 legacy=0x7ff5e81d3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5eeabd640 1 -- 172.21.15.115:0/127122860 reap_dead start 2024-01-24T23:27:31.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5eeabd640 1 -- 172.21.15.115:0/127122860 reap_dead start 2024-01-24T23:27:31.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5f0520640 1 -- 172.21.15.115:0/127122860 shutdown_connections 2024-01-24T23:27:31.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5f0520640 1 -- 172.21.15.115:0/127122860 >> 172.21.15.115:0/127122860 conn(0x7ff5e80a9910 msgr2=0x7ff5e80a7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:31.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5f0520640 1 -- 172.21.15.115:0/127122860 shutdown_connections 2024-01-24T23:27:31.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.204+0000 7ff5f0520640 1 -- 172.21.15.115:0/127122860 wait complete. 2024-01-24T23:27:31.241 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:27:31.241 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-01-24T23:27:31.241 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-01-24T23:27:31.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa25a0dc640 10 monclient: get_monmap_and_config 2024-01-24T23:27:31.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa25a0dc640 10 monclient: build_initial_monmap 2024-01-24T23:27:31.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa25a0dc640 10 monclient: monmap: 2024-01-24T23:27:31.248 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:31.248 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.248 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:31.250201+0000 2024-01-24T23:27:31.248 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:31.250201+0000 2024-01-24T23:27:31.248 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:31.248 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:31.248 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:31.248 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:31.248 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:31.248 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:31.249 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:31.249 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:31.249 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:31.249 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:31.249 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:31.249 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:31.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa25a0dc640 1 Processor -- start 2024-01-24T23:27:31.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa25a0dc640 1 -- start start 2024-01-24T23:27:31.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa25a0dc640 10 monclient: init 2024-01-24T23:27:31.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa25a0dc640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:31.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa25a0dc640 10 monclient: _add_conns ranks=[5,7,8,3,2,6,1,4,0] 2024-01-24T23:27:31.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa25a0dc640 10 monclient(hunting): picked mon.noname-h con 0x7fa25409ff20 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:31.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa25a0dc640 10 monclient(hunting): picked mon.noname-i con 0x7fa25409f9c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:31.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa25a0dc640 10 monclient(hunting): picked mon.noname-e con 0x7fa25413b2c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:31.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa25a0dc640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa254064700 con 0x7fa25409ff20 2024-01-24T23:27:31.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa25a0dc640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa25409e760 con 0x7fa25409f9c0 2024-01-24T23:27:31.256 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa25a0dc640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2540a1160 con 0x7fa25413b2c0 2024-01-24T23:27:31.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa25a0dc640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:31.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa25a0dc640 10 monclient(hunting): authenticate will time out at 1338.574894s 2024-01-24T23:27:31.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa252ffd640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fa25409f9c0 0x7fa254139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:35594/0 (socket says 172.21.15.132:35594) 2024-01-24T23:27:31.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2537fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fa25409ff20 0x7fa2540a0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:32824/0 (socket says 172.21.15.132:32824) 2024-01-24T23:27:31.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa252ffd640 1 -- 172.21.15.132:0/4263811853 learned_addr learned my addr 172.21.15.132:0/4263811853 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:31.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 1 -- 172.21.15.132:0/4263811853 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 297304788 0 0) 0x7fa25409e760 con 0x7fa25409f9c0 2024-01-24T23:27:31.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 10 monclient(hunting): my global_id is 124126 2024-01-24T23:27:31.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 1 -- 172.21.15.132:0/4263811853 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa240003530 con 0x7fa25409f9c0 2024-01-24T23:27:31.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 1 -- 172.21.15.132:0/4263811853 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1748596863 0 0) 0x7fa254064700 con 0x7fa25409ff20 2024-01-24T23:27:31.257 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 10 monclient(hunting): my global_id is 6530 2024-01-24T23:27:31.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 1 -- 172.21.15.132:0/4263811853 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa25409e760 con 0x7fa25409ff20 2024-01-24T23:27:31.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 1 -- 172.21.15.132:0/4263811853 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528716587 0 0) 0x7fa2540a1160 con 0x7fa25413b2c0 2024-01-24T23:27:31.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 10 monclient(hunting): my global_id is 85769 2024-01-24T23:27:31.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 1 -- 172.21.15.132:0/4263811853 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa254064700 con 0x7fa25413b2c0 2024-01-24T23:27:31.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 1 -- 172.21.15.132:0/4263811853 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3588785535 0 0) 0x7fa240003530 con 0x7fa25409f9c0 2024-01-24T23:27:31.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 1 -- 172.21.15.132:0/4263811853 >> v1:172.21.15.115:6793/0 conn(0x7fa25413b2c0 legacy=0x7fa25413d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 1 -- 172.21.15.132:0/4263811853 >> v1:172.21.15.132:6791/0 conn(0x7fa25409ff20 legacy=0x7fa2540a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 10 monclient: _finish_hunting 0 2024-01-24T23:27:31.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 1 monclient: found mon.noname-i 2024-01-24T23:27:31.258 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:31.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:27:31.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 1 -- 172.21.15.132:0/4263811853 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2540a0a10 con 0x7fa25409f9c0 2024-01-24T23:27:31.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 10 monclient: _finish_auth 0 2024-01-24T23:27:31.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 10 monclient: _check_auth_tickets 2024-01-24T23:27:31.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:31.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 1 -- 172.21.15.132:0/4263811853 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa23c0017c0 con 0x7fa25409f9c0 2024-01-24T23:27:31.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:27:31.259 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 10 monclient: dump: 2024-01-24T23:27:31.259 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:31.259 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.259 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.259 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.259 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.259 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:31.259 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.259 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.260 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.260 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.260 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.260 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.260 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.260 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.260 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.260 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:31.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa25a0dc640 5 monclient: authenticate success, global_id 124126 2024-01-24T23:27:31.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa25a0dc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:31.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 1 -- 172.21.15.132:0/4263811853 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa23c001a20 con 0x7fa25409f9c0 2024-01-24T23:27:31.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:31.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 1 -- 172.21.15.132:0/4263811853 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa23c004bf0 con 0x7fa25409f9c0 2024-01-24T23:27:31.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa25a0dc640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:31.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa25a0dc640 4 set_mon_vals no callback set 2024-01-24T23:27:31.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:27:31.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.246+0000 7fa2527fc640 10 monclient: dump: 2024-01-24T23:27:31.261 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:31.261 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.261 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.261 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.261 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.261 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:31.261 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.261 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.261 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.261 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.261 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.261 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.262 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.262 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.262 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.262 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:31.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 10 monclient: shutdown 2024-01-24T23:27:31.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:31.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 1 -- 172.21.15.132:0/4263811853 >> v1:172.21.15.132:6792/0 conn(0x7fa25409f9c0 legacy=0x7fa254139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 1 -- 172.21.15.132:0/4263811853 shutdown_connections 2024-01-24T23:27:31.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 1 -- 172.21.15.132:0/4263811853 >> 172.21.15.132:0/4263811853 conn(0x7fa2540a5a10 msgr2=0x7fa2540a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:31.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 1 -- 172.21.15.132:0/4263811853 shutdown_connections 2024-01-24T23:27:31.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 1 -- 172.21.15.132:0/4263811853 wait complete. 2024-01-24T23:27:31.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 10 monclient: build_initial_monmap 2024-01-24T23:27:31.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 10 monclient: monmap: 2024-01-24T23:27:31.262 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:31.262 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:31.262 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:31.263 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:31.263 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:31.263 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:31.263 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:31.263 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:31.263 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:31.263 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:31.263 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:31.263 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:31.263 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:31.263 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:31.263 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:31.263 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:31.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 1 Processor -- start 2024-01-24T23:27:31.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 1 -- start start 2024-01-24T23:27:31.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 10 monclient: init 2024-01-24T23:27:31.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:31.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 10 monclient: _add_conns ranks=[3,5,4,6,1,2,0,8,7] 2024-01-24T23:27:31.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 10 monclient(hunting): picked mon.noname-g con 0x7fa25409f9c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:31.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 10 monclient(hunting): picked mon.noname-h con 0x7fa25409ff20 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:31.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 10 monclient(hunting): picked mon.noname-c con 0x7fa25413b2c0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:31.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2541203f0 con 0x7fa25409f9c0 2024-01-24T23:27:31.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2541206d0 con 0x7fa25413b2c0 2024-01-24T23:27:31.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa25409d2e0 con 0x7fa25409ff20 2024-01-24T23:27:31.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:31.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 10 monclient(hunting): authenticate will time out at 1338.577996s 2024-01-24T23:27:31.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa252ffd640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fa25409ff20 0x7fa25409cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:32828/0 (socket says 172.21.15.132:32828) 2024-01-24T23:27:31.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa252ffd640 1 -- 172.21.15.132:0/1451332584 learned_addr learned my addr 172.21.15.132:0/1451332584 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:31.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1738476139 0 0) 0x7fa25409d2e0 con 0x7fa25409ff20 2024-01-24T23:27:31.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient(hunting): my global_id is 6539 2024-01-24T23:27:31.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa234003530 con 0x7fa25409ff20 2024-01-24T23:27:31.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2629670211 0 0) 0x7fa2541203f0 con 0x7fa25409f9c0 2024-01-24T23:27:31.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient(hunting): my global_id is 6555 2024-01-24T23:27:31.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa25409d2e0 con 0x7fa25409f9c0 2024-01-24T23:27:31.265 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1663799029 0 0) 0x7fa2541206d0 con 0x7fa25413b2c0 2024-01-24T23:27:31.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient(hunting): my global_id is 94720 2024-01-24T23:27:31.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2541203f0 con 0x7fa25413b2c0 2024-01-24T23:27:31.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 848530254 0 0) 0x7fa234003530 con 0x7fa25409ff20 2024-01-24T23:27:31.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa2541206d0 con 0x7fa25409ff20 2024-01-24T23:27:31.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa23c0047f0 con 0x7fa25409ff20 2024-01-24T23:27:31.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:27:31.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient(hunting): dump: 2024-01-24T23:27:31.266 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:31.266 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.267 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.267 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.267 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.267 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:31.267 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.267 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.267 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.267 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.267 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.267 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.267 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.267 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.267 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.267 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1108283838 0 0) 0x7fa25409d2e0 con 0x7fa25409f9c0 2024-01-24T23:27:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa234003530 con 0x7fa25409f9c0 2024-01-24T23:27:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa24c0016f0 con 0x7fa25409f9c0 2024-01-24T23:27:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient(hunting): dump: 2024-01-24T23:27:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.268 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:31.269 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.269 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.269 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.269 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.269 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.269 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.269 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.269 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.269 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.269 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:31.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1470006318 0 0) 0x7fa2541203f0 con 0x7fa25413b2c0 2024-01-24T23:27:31.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa25409d2e0 con 0x7fa25413b2c0 2024-01-24T23:27:31.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa244001a00 con 0x7fa25413b2c0 2024-01-24T23:27:31.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:31.270 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient(hunting): dump: 2024-01-24T23:27:31.270 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:31.270 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.270 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.270 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.270 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.270 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:31.270 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.270 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.270 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.270 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.270 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.270 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.271 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.271 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.271 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.271 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:31.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 997746442 0 0) 0x7fa2541206d0 con 0x7fa25409ff20 2024-01-24T23:27:31.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 >> v1:172.21.15.115:6791/0 conn(0x7fa25413b2c0 legacy=0x7fa2541d38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 >> v1:172.21.15.132:6790/0 conn(0x7fa25409f9c0 legacy=0x7fa25409c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:27:31.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 1 monclient: found mon.h 2024-01-24T23:27:31.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:31.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:31.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2541209b0 con 0x7fa25409ff20 2024-01-24T23:27:31.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient: _finish_auth 0 2024-01-24T23:27:31.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:27:31.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:31.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 5 monclient: authenticate success, global_id 6539 2024-01-24T23:27:31.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 10 monclient: _renew_subs 2024-01-24T23:27:31.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:31.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 1 -- 172.21.15.132:0/1451332584 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa254071960 con 0x7fa25409ff20 2024-01-24T23:27:31.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 10 monclient: _renew_subs 2024-01-24T23:27:31.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:31.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 1 -- 172.21.15.132:0/1451332584 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25409dce0 con 0x7fa25409ff20 2024-01-24T23:27:31.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa23c0011a0 con 0x7fa25409ff20 2024-01-24T23:27:31.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:31.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa23c005200 con 0x7fa25409ff20 2024-01-24T23:27:31.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa2517fa640 4 set_mon_vals no callback set 2024-01-24T23:27:31.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:31.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa24bfff640 10 monclient: dump: 2024-01-24T23:27:31.272 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:31.272 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.272 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.273 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.273 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.273 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:31.273 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.273 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.273 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.273 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.273 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.273 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.273 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.273 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.273 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.273 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:31.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:31.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:31.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:31.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.250+0000 7fa25a0dc640 1 -- 172.21.15.132:0/1451332584 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa238005090 con 0x7fa25409ff20 2024-01-24T23:27:31.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.254+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fa23c00fc80 con 0x7fa25409ff20 2024-01-24T23:27:31.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.254+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(216..216 src has 1..216) v4 ==== 13538+0+0 (unknown 289362477 0 0) 0x7fa23c045fd0 con 0x7fa25409ff20 2024-01-24T23:27:31.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.254+0000 7fa2537fe640 10 monclient: get_auth_request con 0x7fa234037b50 auth_method 0 2024-01-24T23:27:31.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.258+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fa23c0116d0 con 0x7fa25409ff20 2024-01-24T23:27:31.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.258+0000 7fa24bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:31.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.258+0000 7fa24bfff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:31.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 10 monclient: get_monmap_and_config 2024-01-24T23:27:31.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 10 monclient: build_initial_monmap 2024-01-24T23:27:31.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 10 monclient: monmap: 2024-01-24T23:27:31.372 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:31.372 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.372 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:31.363198+0000 2024-01-24T23:27:31.372 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:31.363198+0000 2024-01-24T23:27:31.372 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:31.372 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.372 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:31.372 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:31.372 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:31.372 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:31.372 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:31.372 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:31.373 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:31.373 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:31.373 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:31.373 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 1 Processor -- start 2024-01-24T23:27:31.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 1 -- start start 2024-01-24T23:27:31.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 10 monclient: init 2024-01-24T23:27:31.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:31.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 10 monclient: _add_conns ranks=[6,2,5,7,4,3,0,1,8] 2024-01-24T23:27:31.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 10 monclient(hunting): picked mon.noname-d con 0x7f9890133bf0 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:31.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 10 monclient(hunting): picked mon.noname-b con 0x7f9890133fd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:31.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 10 monclient(hunting): picked mon.noname-h con 0x7f9890134430 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:31.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989004ed70 con 0x7f9890133fd0 2024-01-24T23:27:31.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989009af30 con 0x7f9890134430 2024-01-24T23:27:31.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989009b210 con 0x7f9890133bf0 2024-01-24T23:27:31.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:31.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 10 monclient(hunting): authenticate will time out at 1330.285804s 2024-01-24T23:27:31.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988ffff640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f9890133bf0 0x7f989009bd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:40400/0 (socket says 172.21.15.115:40400) 2024-01-24T23:27:31.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988f7fe640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f9890133fd0 0x7f989013bdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:36680/0 (socket says 172.21.15.115:36680) 2024-01-24T23:27:31.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988ffff640 1 -- 172.21.15.115:0/2113141296 learned_addr learned my addr 172.21.15.115:0/2113141296 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:31.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 1 -- 172.21.15.115:0/2113141296 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3010096889 0 0) 0x7f989004ed70 con 0x7f9890133fd0 2024-01-24T23:27:31.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 10 monclient(hunting): my global_id is 124148 2024-01-24T23:27:31.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 1 -- 172.21.15.115:0/2113141296 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f987c003530 con 0x7f9890133fd0 2024-01-24T23:27:31.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 1 -- 172.21.15.115:0/2113141296 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 737713588 0 0) 0x7f989009b210 con 0x7f9890133bf0 2024-01-24T23:27:31.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 10 monclient(hunting): my global_id is 6702 2024-01-24T23:27:31.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 1 -- 172.21.15.115:0/2113141296 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f989004ed70 con 0x7f9890133bf0 2024-01-24T23:27:31.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 1 -- 172.21.15.115:0/2113141296 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513900566 0 0) 0x7f989009af30 con 0x7f9890134430 2024-01-24T23:27:31.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 10 monclient(hunting): my global_id is 6548 2024-01-24T23:27:31.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 1 -- 172.21.15.115:0/2113141296 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f989009b210 con 0x7f9890134430 2024-01-24T23:27:31.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 1 -- 172.21.15.115:0/2113141296 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 314893468 0 0) 0x7f987c003530 con 0x7f9890133fd0 2024-01-24T23:27:31.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 1 -- 172.21.15.115:0/2113141296 >> v1:172.21.15.115:6792/0 conn(0x7f9890133bf0 legacy=0x7f989009bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 1 -- 172.21.15.115:0/2113141296 >> v1:172.21.15.132:6791/0 conn(0x7f9890134430 legacy=0x7f989013f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 10 monclient: _finish_hunting 0 2024-01-24T23:27:31.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 1 monclient: found mon.noname-b 2024-01-24T23:27:31.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:31.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:27:31.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 1 -- 172.21.15.115:0/2113141296 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989009b4f0 con 0x7f9890133fd0 2024-01-24T23:27:31.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 10 monclient: _finish_auth 0 2024-01-24T23:27:31.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 10 monclient: _check_auth_tickets 2024-01-24T23:27:31.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:31.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 1 -- 172.21.15.115:0/2113141296 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f98780017c0 con 0x7f9890133fd0 2024-01-24T23:27:31.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 5 monclient: authenticate success, global_id 124148 2024-01-24T23:27:31.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:31.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:27:31.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 10 monclient: dump: 2024-01-24T23:27:31.377 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.377 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.377 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.377 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.377 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.377 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.377 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.377 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.377 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.377 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.377 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.377 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.377 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.377 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.378 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.378 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:31.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 1 -- 172.21.15.115:0/2113141296 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f98780019e0 con 0x7f9890133fd0 2024-01-24T23:27:31.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:31.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:31.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 4 set_mon_vals no callback set 2024-01-24T23:27:31.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 1 -- 172.21.15.115:0/2113141296 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9878004b30 con 0x7f9890133fd0 2024-01-24T23:27:31.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:27:31.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f988effd640 10 monclient: dump: 2024-01-24T23:27:31.378 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.378 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.378 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.379 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.379 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.379 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.379 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.379 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.379 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.379 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.379 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.379 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.379 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.379 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.379 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.379 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 10 monclient: shutdown 2024-01-24T23:27:31.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:31.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 1 -- 172.21.15.115:0/2113141296 >> v1:172.21.15.115:6790/0 conn(0x7f9890133fd0 legacy=0x7f989013bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 1 -- 172.21.15.115:0/2113141296 shutdown_connections 2024-01-24T23:27:31.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 1 -- 172.21.15.115:0/2113141296 >> 172.21.15.115:0/2113141296 conn(0x7f98900a9910 msgr2=0x7f98900a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:31.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 1 -- 172.21.15.115:0/2113141296 shutdown_connections 2024-01-24T23:27:31.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.360+0000 7f9896a0c640 1 -- 172.21.15.115:0/2113141296 wait complete. 2024-01-24T23:27:31.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f9896a0c640 10 monclient: build_initial_monmap 2024-01-24T23:27:31.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f9896a0c640 10 monclient: monmap: 2024-01-24T23:27:31.380 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:31.380 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:31.380 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:31.380 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:31.380 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:31.380 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:31.380 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:31.380 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:31.381 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:31.381 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:31.381 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:31.381 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:31.381 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:31.381 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:31.381 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:31.381 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f9896a0c640 1 Processor -- start 2024-01-24T23:27:31.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f9896a0c640 1 -- start start 2024-01-24T23:27:31.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f9896a0c640 10 monclient: init 2024-01-24T23:27:31.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f9896a0c640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:31.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f9896a0c640 10 monclient: _add_conns ranks=[2,3,6,8,1,7,5,4,0] 2024-01-24T23:27:31.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f9896a0c640 10 monclient(hunting): picked mon.noname-b con 0x7f9890133bf0 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:31.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f9896a0c640 10 monclient(hunting): picked mon.noname-g con 0x7f9890133fd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:31.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f9896a0c640 10 monclient(hunting): picked mon.noname-d con 0x7f9890134430 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:31.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f9896a0c640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989009c8f0 con 0x7f9890133bf0 2024-01-24T23:27:31.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f9896a0c640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98900a5980 con 0x7f9890133fd0 2024-01-24T23:27:31.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f9896a0c640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989013a530 con 0x7f9890134430 2024-01-24T23:27:31.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f9896a0c640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:31.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f9896a0c640 10 monclient(hunting): authenticate will time out at 1330.289104s 2024-01-24T23:27:31.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988ffff640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f9890133bf0 0x7f98901cb790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:36688/0 (socket says 172.21.15.115:36688) 2024-01-24T23:27:31.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988ffff640 1 -- 172.21.15.115:0/2784211345 learned_addr learned my addr 172.21.15.115:0/2784211345 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:31.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 110886912 0 0) 0x7f989013a530 con 0x7f9890134430 2024-01-24T23:27:31.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient(hunting): my global_id is 6711 2024-01-24T23:27:31.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9870003530 con 0x7f9890134430 2024-01-24T23:27:31.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 214961753 0 0) 0x7f989009c8f0 con 0x7f9890133bf0 2024-01-24T23:27:31.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient(hunting): my global_id is 124157 2024-01-24T23:27:31.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f989013a530 con 0x7f9890133bf0 2024-01-24T23:27:31.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1644534261 0 0) 0x7f9870003530 con 0x7f9890134430 2024-01-24T23:27:31.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f989009c8f0 con 0x7f9890134430 2024-01-24T23:27:31.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4069628768 0 0) 0x7f98900a5980 con 0x7f9890133fd0 2024-01-24T23:27:31.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient(hunting): my global_id is 6564 2024-01-24T23:27:31.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9870003530 con 0x7f9890133fd0 2024-01-24T23:27:31.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f98880019c0 con 0x7f9890134430 2024-01-24T23:27:31.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:27:31.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient(hunting): dump: 2024-01-24T23:27:31.384 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.384 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.384 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.384 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.384 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.384 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.384 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.384 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.384 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.384 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.385 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.385 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.385 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.385 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.385 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.385 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:31.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2309796056 0 0) 0x7f989013a530 con 0x7f9890133bf0 2024-01-24T23:27:31.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f98900a5980 con 0x7f9890133bf0 2024-01-24T23:27:31.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f98800018a0 con 0x7f9890133bf0 2024-01-24T23:27:31.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:27:31.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient(hunting): dump: 2024-01-24T23:27:31.385 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.386 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.386 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.386 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.386 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.386 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.386 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.386 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.386 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.386 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.386 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.386 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.386 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.386 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.386 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.387 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2528711884 0 0) 0x7f989009c8f0 con 0x7f9890134430 2024-01-24T23:27:31.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 >> v1:172.21.15.132:6790/0 conn(0x7f9890133fd0 legacy=0x7f98901d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 >> v1:172.21.15.115:6790/0 conn(0x7f9890133bf0 legacy=0x7f98901cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient: _finish_hunting 0 2024-01-24T23:27:31.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 monclient: found mon.d 2024-01-24T23:27:31.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:31.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:31.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989013a810 con 0x7f9890134430 2024-01-24T23:27:31.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient: _finish_auth 0 2024-01-24T23:27:31.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient: _check_auth_tickets 2024-01-24T23:27:31.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:31.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f9896a0c640 5 monclient: authenticate success, global_id 6711 2024-01-24T23:27:31.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f9896a0c640 10 monclient: _renew_subs 2024-01-24T23:27:31.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f9896a0c640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:31.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f9896a0c640 1 -- 172.21.15.115:0/2784211345 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9890139640 con 0x7f9890134430 2024-01-24T23:27:31.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f9896a0c640 10 monclient: _renew_subs 2024-01-24T23:27:31.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f9896a0c640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:31.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f9896a0c640 1 -- 172.21.15.115:0/2784211345 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9890139b50 con 0x7f9890134430 2024-01-24T23:27:31.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9888001cd0 con 0x7f9890134430 2024-01-24T23:27:31.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:31.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9888003c10 con 0x7f9890134430 2024-01-24T23:27:31.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988e7fc640 4 set_mon_vals no callback set 2024-01-24T23:27:31.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:31.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient: dump: 2024-01-24T23:27:31.388 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.388 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.388 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.389 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.389 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.389 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.389 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.389 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.389 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.389 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.389 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.389 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.389 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.389 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.389 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.389 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f9896a0c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:31.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f9896a0c640 1 -- 172.21.15.115:0/2784211345 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f98740065d0 con 0x7f9874002870 2024-01-24T23:27:31.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f988800e690 con 0x7f9890134430 2024-01-24T23:27:31.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(216..216 src has 1..216) v4 ==== 13538+0+0 (unknown 289362477 0 0) 0x7f9888044a50 con 0x7f9890134430 2024-01-24T23:27:31.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3663722015 0 0) 0x7f98740065d0 con 0x7f9874002870 2024-01-24T23:27:31.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient(hunting): my global_id is 94729 2024-01-24T23:27:31.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f989009c8f0 con 0x7f9874002870 2024-01-24T23:27:31.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988f7fe640 10 monclient: get_auth_request con 0x7f98700378c0 auth_method 0 2024-01-24T23:27:31.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3487408335 0 0) 0x7f989009c8f0 con 0x7f9874002870 2024-01-24T23:27:31.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f98740065d0 con 0x7f9874002870 2024-01-24T23:27:31.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9880004eb0 con 0x7f9874002870 2024-01-24T23:27:31.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:31.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 10 monclient: dump: 2024-01-24T23:27:31.391 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.391 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.391 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.391 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.391 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.391 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.391 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.391 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.391 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.391 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.391 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.391 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.391 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.391 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.392 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.392 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1738152283 0 0) 0x7f98740065d0 con 0x7f9874002870 2024-01-24T23:27:31.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.364+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 --> v1:172.21.15.115:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f987400a120 con 0x7f9874002870 2024-01-24T23:27:31.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.368+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 <== mon.4 v1:172.21.15.115:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f9880007180 con 0x7f9874002870 2024-01-24T23:27:31.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.368+0000 7f988cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:31.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.368+0000 7f988cff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:31.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.368+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 >> v1:172.21.15.115:6791/0 conn(0x7f9874002870 legacy=0x7f9874004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.368+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 >> v1:172.21.15.115:6791/0 conn(0x7f9874002870 legacy=0x7f9874004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:31.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f9896a0c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:27:31.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f9896a0c640 1 -- 172.21.15.115:0/2784211345 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9874009e10 con 0x7f98740090d0 2024-01-24T23:27:31.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3375821688 0 0) 0x7f9874009e10 con 0x7f98740090d0 2024-01-24T23:27:31.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f988cff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f988cff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f988cff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f988cff9640 10 monclient(hunting): my global_id is 94738 2024-01-24T23:27:31.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98740065d0 con 0x7f98740090d0 2024-01-24T23:27:31.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2290491997 0 0) 0x7f98740065d0 con 0x7f98740090d0 2024-01-24T23:27:31.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9874009e10 con 0x7f98740090d0 2024-01-24T23:27:31.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f98800059e0 con 0x7f98740090d0 2024-01-24T23:27:31.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f988cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f988cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:31.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f988cff9640 10 monclient: dump: 2024-01-24T23:27:31.395 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.395 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.395 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.395 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.395 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.395 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.395 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.395 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.395 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.395 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.396 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.396 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.396 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.396 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.396 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.396 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3739189703 0 0) 0x7f9874009e10 con 0x7f98740090d0 2024-01-24T23:27:31.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 --> v1:172.21.15.115:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f987400c310 con 0x7f98740090d0 2024-01-24T23:27:31.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 <== mon.4 v1:172.21.15.115:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 1541473042) 0x7f9880006f60 con 0x7f98740090d0 2024-01-24T23:27:31.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f988cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:27:31.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f988cff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:31.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 >> v1:172.21.15.115:6791/0 conn(0x7f98740090d0 legacy=0x7f9874009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f988cff9640 1 -- 172.21.15.115:0/2784211345 >> v1:172.21.15.115:6791/0 conn(0x7f98740090d0 legacy=0x7f9874009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:31.397 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:27:31.397 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:31.397 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:31.397 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:27:31.397 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 40, 2024-01-24T23:27:31.397 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:27:31.397 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:27:31.397 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:27:31.397 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:27:31.397 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:27:31.397 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:27:31.398 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:27:31.398 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:27:31.398 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:27:31.398 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:27:31.398 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.398 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:27:31.398 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:31.398 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:27:31.398 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:27:31.398 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:31.398 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:31.398 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:31.398 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:31.399 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:31.399 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:31.399 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:31.399 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:31.399 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:31.399 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:31.399 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:31.399 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.399 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:27:31.399 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:27:31.399 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:31.399 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:31.399 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:31.400 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:31.400 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:31.400 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:31.400 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:31.400 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:31.400 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:31.400 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:31.400 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:31.400 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.400 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:27:31.400 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:27:31.401 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:27:31.401 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:27:31.401 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:27:31.401 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:27:31.401 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:31.401 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:31.401 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:27:31.401 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:27:31.401 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:27:31.401 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:27:31.401 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:27:31.401 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:27:31.401 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:27:31.401 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:31.402 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:27:31.402 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:31.402 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:31.402 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:31.402 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:31.402 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:31.402 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:31.402 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:31.402 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:31.402 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:31.402 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:31.402 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:31.402 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.402 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:27:31.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.403 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:27:31.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.403 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:31.403 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:31.403 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.403 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.403 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.403 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:27:31.403 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.403 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.403 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.404 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:27:31.404 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:27:31.404 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.404 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.404 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.404 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:31.404 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:31.404 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.404 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.404 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.404 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:27:31.405 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.405 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.405 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.405 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:27:31.405 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:27:31.405 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.405 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.405 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.405 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:31.405 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:31.405 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.406 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.406 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.406 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:27:31.406 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.406 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.406 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.406 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:27:31.406 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:27:31.406 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.406 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.406 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.407 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:31.407 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:31.407 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.407 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.407 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.407 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:27:31.407 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.407 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.407 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.407 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:27:31.407 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:27:31.407 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.408 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.408 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.408 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:31.408 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:31.408 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.408 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.408 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.408 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:27:31.408 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.408 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.408 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.409 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:27:31.409 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:27:31.409 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.409 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.409 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.409 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:31.409 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:31.409 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.409 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.409 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.409 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:27:31.410 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.410 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.410 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.410 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:27:31.410 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:27:31.410 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.410 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.410 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.410 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:31.410 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:31.410 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.410 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.411 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.411 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:27:31.411 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.411 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.411 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.411 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:27:31.411 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:27:31.411 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.411 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.411 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.411 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:31.412 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:31.412 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.412 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.412 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.412 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:27:31.412 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.412 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.412 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.412 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:27:31.412 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:27:31.412 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.412 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.413 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.413 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:31.413 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:31.413 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.413 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.413 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.413 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:27:31.413 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.413 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.413 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.413 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:27:31.414 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:27:31.414 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.414 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.414 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.414 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.414 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.414 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:27:31.414 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:27:31.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.414 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:31.414 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:31.414 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:31.414 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.414 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.414 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:27:31.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.415 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:31.415 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:31.415 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:27:31.415 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.415 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.415 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:27:31.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.415 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:31.415 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:31.415 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:31.415 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.415 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.416 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:27:31.416 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:27:31.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f9896a0c640 1 -- 172.21.15.115:0/2784211345 >> v1:172.21.15.132:6812/1835401922 conn(0x7f98700378c0 legacy=0x7f9870039d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f9896a0c640 10 monclient: shutdown 2024-01-24T23:27:31.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f9896a0c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:31.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f9896a0c640 1 -- 172.21.15.115:0/2784211345 >> v1:172.21.15.115:6792/0 conn(0x7f9890134430 legacy=0x7f98901d3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f9894fa9640 1 -- 172.21.15.115:0/2784211345 reap_dead start 2024-01-24T23:27:31.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f9894fa9640 1 -- 172.21.15.115:0/2784211345 reap_dead start 2024-01-24T23:27:31.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f9896a0c640 1 -- 172.21.15.115:0/2784211345 shutdown_connections 2024-01-24T23:27:31.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f9896a0c640 1 -- 172.21.15.115:0/2784211345 >> 172.21.15.115:0/2784211345 conn(0x7f98900a9910 msgr2=0x7f9890073ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:31.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f9896a0c640 1 -- 172.21.15.115:0/2784211345 shutdown_connections 2024-01-24T23:27:31.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.372+0000 7f9896a0c640 1 -- 172.21.15.115:0/2784211345 wait complete. 2024-01-24T23:27:31.418 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:27:31.418 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-01-24T23:27:31.418 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-01-24T23:27:31.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.418+0000 7fa25a0dc640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h5"}] 2024-01-24T23:27:31.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.418+0000 7fa25a0dc640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h5"}] 2024-01-24T23:27:31.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.418+0000 7fa25a0dc640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:31.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.418+0000 7fa25a0dc640 1 -- 172.21.15.132:0/1451332584 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h5"} v 0) v1 -- 0x7fa238005340 con 0x7fa25409ff20 2024-01-24T23:27:31.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 10 monclient: get_monmap_and_config 2024-01-24T23:27:31.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 10 monclient: build_initial_monmap 2024-01-24T23:27:31.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 10 monclient: monmap: 2024-01-24T23:27:31.534 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:31.534 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.534 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:31.535149+0000 2024-01-24T23:27:31.534 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:31.535149+0000 2024-01-24T23:27:31.534 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:31.534 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.534 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:31.534 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:31.534 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:31.535 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:31.535 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:31.535 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:31.535 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:31.535 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:31.535 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:31.535 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 1 Processor -- start 2024-01-24T23:27:31.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 1 -- start start 2024-01-24T23:27:31.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 10 monclient: init 2024-01-24T23:27:31.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:31.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 10 monclient: _add_conns ranks=[7,2,3,0,8,4,1,6,5] 2024-01-24T23:27:31.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 10 monclient(hunting): picked mon.noname-i con 0x7f5dcc133bf0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:31.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 10 monclient(hunting): picked mon.noname-b con 0x7f5dcc133fd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:31.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 10 monclient(hunting): picked mon.noname-g con 0x7f5dcc1343b0 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:31.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dcc04ed70 con 0x7f5dcc133fd0 2024-01-24T23:27:31.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dcc0a4b60 con 0x7f5dcc1343b0 2024-01-24T23:27:31.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dcc09ae50 con 0x7f5dcc133bf0 2024-01-24T23:27:31.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:31.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 10 monclient(hunting): authenticate will time out at 1330.457908s 2024-01-24T23:27:31.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcbfff640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f5dcc133fd0 0x7f5dcc13bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:36694/0 (socket says 172.21.15.115:36694) 2024-01-24T23:27:31.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcbfff640 1 -- 172.21.15.115:0/4114026213 learned_addr learned my addr 172.21.15.115:0/4114026213 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:31.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 1 -- 172.21.15.115:0/4114026213 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 666683129 0 0) 0x7f5dcc04ed70 con 0x7f5dcc133fd0 2024-01-24T23:27:31.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 10 monclient(hunting): my global_id is 124166 2024-01-24T23:27:31.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 1 -- 172.21.15.115:0/4114026213 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5db4003530 con 0x7f5dcc133fd0 2024-01-24T23:27:31.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 1 -- 172.21.15.115:0/4114026213 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3275487291 0 0) 0x7f5db4003530 con 0x7f5dcc133fd0 2024-01-24T23:27:31.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 1 -- 172.21.15.115:0/4114026213 >> v1:172.21.15.132:6792/0 conn(0x7f5dcc133bf0 legacy=0x7f5dcc09bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 1 -- 172.21.15.115:0/4114026213 >> v1:172.21.15.132:6790/0 conn(0x7f5dcc1343b0 legacy=0x7f5dcc13f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:27:31.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 1 monclient: found mon.noname-b 2024-01-24T23:27:31.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:31.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:27:31.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 1 -- 172.21.15.115:0/4114026213 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dcc09b130 con 0x7f5dcc133fd0 2024-01-24T23:27:31.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 10 monclient: _finish_auth 0 2024-01-24T23:27:31.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:31.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:31.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 1 -- 172.21.15.115:0/4114026213 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5dbc0017c0 con 0x7f5dcc133fd0 2024-01-24T23:27:31.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 5 monclient: authenticate success, global_id 124166 2024-01-24T23:27:31.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:27:31.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 10 monclient: dump: 2024-01-24T23:27:31.546 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.546 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.547 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.547 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.547 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.547 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.547 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.547 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.547 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.547 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.547 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.547 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.547 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.547 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.548 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.548 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:31.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 1 -- 172.21.15.115:0/4114026213 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5dbc001a70 con 0x7f5dcc133fd0 2024-01-24T23:27:31.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:31.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 1 -- 172.21.15.115:0/4114026213 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5dbc004bf0 con 0x7f5dcc133fd0 2024-01-24T23:27:31.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:27:31.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dcb7fe640 10 monclient: dump: 2024-01-24T23:27:31.549 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.549 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.549 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.549 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.549 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.549 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.549 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.549 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.549 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.549 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.549 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.549 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.549 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.549 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.550 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.550 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:31.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 4 set_mon_vals no callback set 2024-01-24T23:27:31.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 10 monclient: shutdown 2024-01-24T23:27:31.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:31.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 1 -- 172.21.15.115:0/4114026213 >> v1:172.21.15.115:6790/0 conn(0x7f5dcc133fd0 legacy=0x7f5dcc13bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 1 -- 172.21.15.115:0/4114026213 shutdown_connections 2024-01-24T23:27:31.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 1 -- 172.21.15.115:0/4114026213 >> 172.21.15.115:0/4114026213 conn(0x7f5dcc0662f0 msgr2=0x7f5dcc066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:31.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 1 -- 172.21.15.115:0/4114026213 shutdown_connections 2024-01-24T23:27:31.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.532+0000 7f5dd2b6f640 1 -- 172.21.15.115:0/4114026213 wait complete. 2024-01-24T23:27:31.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd2b6f640 10 monclient: build_initial_monmap 2024-01-24T23:27:31.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd2b6f640 10 monclient: monmap: 2024-01-24T23:27:31.550 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:31.551 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:31.551 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:31.551 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:31.551 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:31.551 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:31.551 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:31.551 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:31.551 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:31.551 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:31.551 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:31.551 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:31.551 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:31.551 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:31.551 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:31.552 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd2b6f640 1 Processor -- start 2024-01-24T23:27:31.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd2b6f640 1 -- start start 2024-01-24T23:27:31.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd2b6f640 10 monclient: init 2024-01-24T23:27:31.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd2b6f640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:31.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd2b6f640 10 monclient: _add_conns ranks=[5,4,2,6,7,3,1,8,0] 2024-01-24T23:27:31.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd2b6f640 10 monclient(hunting): picked mon.noname-h con 0x7f5dcc133bf0 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:31.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd2b6f640 10 monclient(hunting): picked mon.noname-c con 0x7f5dcc133fd0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:31.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd2b6f640 10 monclient(hunting): picked mon.noname-b con 0x7f5dcc1343b0 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:31.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd2b6f640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dcc0a2490 con 0x7f5dcc1343b0 2024-01-24T23:27:31.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd2b6f640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dcc11fb20 con 0x7f5dcc133fd0 2024-01-24T23:27:31.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd2b6f640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dcc1200c0 con 0x7f5dcc133bf0 2024-01-24T23:27:31.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd2b6f640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:31.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd2b6f640 10 monclient(hunting): authenticate will time out at 1330.461191s 2024-01-24T23:27:31.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd110c640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f5dcc1343b0 0x7f5dcc1d3840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:36704/0 (socket says 172.21.15.115:36704) 2024-01-24T23:27:31.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd110c640 1 -- 172.21.15.115:0/269323512 learned_addr learned my addr 172.21.15.115:0/269323512 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:31.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 943246359 0 0) 0x7f5dcc0a2490 con 0x7f5dcc1343b0 2024-01-24T23:27:31.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient(hunting): my global_id is 124175 2024-01-24T23:27:31.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5db8003530 con 0x7f5dcc1343b0 2024-01-24T23:27:31.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2810741529 0 0) 0x7f5dcc11fb20 con 0x7f5dcc133fd0 2024-01-24T23:27:31.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient(hunting): my global_id is 94747 2024-01-24T23:27:31.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dcc0a2490 con 0x7f5dcc133fd0 2024-01-24T23:27:31.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3566449764 0 0) 0x7f5db8003530 con 0x7f5dcc1343b0 2024-01-24T23:27:31.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5dcc11fb20 con 0x7f5dcc1343b0 2024-01-24T23:27:31.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5dc4001dc0 con 0x7f5dcc1343b0 2024-01-24T23:27:31.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:27:31.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient(hunting): dump: 2024-01-24T23:27:31.554 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.554 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.554 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.554 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.554 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.554 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.554 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.554 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.554 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.555 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.555 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.555 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.555 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.555 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.555 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.555 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:31.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1846206988 0 0) 0x7f5dcc1200c0 con 0x7f5dcc133bf0 2024-01-24T23:27:31.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient(hunting): my global_id is 6557 2024-01-24T23:27:31.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5db8003530 con 0x7f5dcc133bf0 2024-01-24T23:27:31.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4084997868 0 0) 0x7f5dcc0a2490 con 0x7f5dcc133fd0 2024-01-24T23:27:31.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5dcc1200c0 con 0x7f5dcc133fd0 2024-01-24T23:27:31.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5dbc004b30 con 0x7f5dcc133fd0 2024-01-24T23:27:31.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:31.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient(hunting): dump: 2024-01-24T23:27:31.556 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.556 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.556 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.556 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.557 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.557 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.557 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.557 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.557 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.557 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.557 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.557 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.557 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.557 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.557 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.557 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2676544646 0 0) 0x7f5dcc11fb20 con 0x7f5dcc1343b0 2024-01-24T23:27:31.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 >> v1:172.21.15.132:6791/0 conn(0x7f5dcc133bf0 legacy=0x7f5dcc1cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 >> v1:172.21.15.115:6791/0 conn(0x7f5dcc133fd0 legacy=0x7f5dcc132fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient: _finish_hunting 0 2024-01-24T23:27:31.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 monclient: found mon.b 2024-01-24T23:27:31.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:31.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:27:31.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dcc1203a0 con 0x7f5dcc1343b0 2024-01-24T23:27:31.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient: _finish_auth 0 2024-01-24T23:27:31.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient: _check_auth_tickets 2024-01-24T23:27:31.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:31.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd2b6f640 5 monclient: authenticate success, global_id 124175 2024-01-24T23:27:31.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd2b6f640 10 monclient: _renew_subs 2024-01-24T23:27:31.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd2b6f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:27:31.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd2b6f640 1 -- 172.21.15.115:0/269323512 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dcc1336b0 con 0x7f5dcc1343b0 2024-01-24T23:27:31.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd2b6f640 10 monclient: _renew_subs 2024-01-24T23:27:31.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd2b6f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:27:31.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd2b6f640 1 -- 172.21.15.115:0/269323512 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dcc0a31c0 con 0x7f5dcc1343b0 2024-01-24T23:27:31.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5dc4002080 con 0x7f5dcc1343b0 2024-01-24T23:27:31.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:31.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5dc4003f90 con 0x7f5dcc1343b0 2024-01-24T23:27:31.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dcaffd640 4 set_mon_vals no callback set 2024-01-24T23:27:31.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:27:31.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient: dump: 2024-01-24T23:27:31.559 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.559 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.559 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.559 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.559 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.559 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.559 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.560 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.560 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.560 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.560 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.560 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.560 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.560 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.560 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.560 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd2b6f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:31.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dd2b6f640 1 -- 172.21.15.115:0/269323512 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5dac0065d0 con 0x7f5dac002870 2024-01-24T23:27:31.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f5dc40040f0 con 0x7f5dcc1343b0 2024-01-24T23:27:31.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(216..216 src has 1..216) v4 ==== 13538+0+0 (unknown 289362477 0 0) 0x7f5dc4046100 con 0x7f5dcc1343b0 2024-01-24T23:27:31.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3299693345 0 0) 0x7f5dac0065d0 con 0x7f5dac002870 2024-01-24T23:27:31.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient(hunting): my global_id is 6720 2024-01-24T23:27:31.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dcc11fb20 con 0x7f5dac002870 2024-01-24T23:27:31.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dcbfff640 10 monclient: get_auth_request con 0x7f5db80378c0 auth_method 0 2024-01-24T23:27:31.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2140822660 0 0) 0x7f5dcc11fb20 con 0x7f5dac002870 2024-01-24T23:27:31.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5dac0065d0 con 0x7f5dac002870 2024-01-24T23:27:31.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5dc00017e0 con 0x7f5dac002870 2024-01-24T23:27:31.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:31.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.536+0000 7f5dc97fa640 10 monclient: dump: 2024-01-24T23:27:31.561 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.561 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.561 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.562 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.562 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.562 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.562 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.562 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.562 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.562 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.562 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.562 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.562 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.562 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.562 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.562 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.540+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 875605469 0 0) 0x7f5dac0065d0 con 0x7f5dac002870 2024-01-24T23:27:31.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.540+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 --> v1:172.21.15.115:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5dac00a120 con 0x7f5dac002870 2024-01-24T23:27:31.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.540+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f5dc0004630 con 0x7f5dac002870 2024-01-24T23:27:31.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.540+0000 7f5dc97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:31.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.540+0000 7f5dc97fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:31.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.540+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 >> v1:172.21.15.115:6792/0 conn(0x7f5dac002870 legacy=0x7f5dac004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.540+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 >> v1:172.21.15.115:6792/0 conn(0x7f5dac002870 legacy=0x7f5dac004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:31.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dd2b6f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:27:31.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dd2b6f640 1 -- 172.21.15.115:0/269323512 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5dac009df0 con 0x7f5dac0090b0 2024-01-24T23:27:31.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 984635238 0 0) 0x7f5dac009df0 con 0x7f5dac0090b0 2024-01-24T23:27:31.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dc97fa640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dc97fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dc97fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dc97fa640 10 monclient(hunting): my global_id is 6729 2024-01-24T23:27:31.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dac0065d0 con 0x7f5dac0090b0 2024-01-24T23:27:31.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3891142911 0 0) 0x7f5dac0065d0 con 0x7f5dac0090b0 2024-01-24T23:27:31.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5dac009df0 con 0x7f5dac0090b0 2024-01-24T23:27:31.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5dc00019c0 con 0x7f5dac0090b0 2024-01-24T23:27:31.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dc97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dc97fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:31.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dc97fa640 10 monclient: dump: 2024-01-24T23:27:31.565 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.565 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.565 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.565 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.566 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.566 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.566 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.566 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.566 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.566 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.566 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.566 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.566 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.566 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.566 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.566 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3827045276 0 0) 0x7f5dac009df0 con 0x7f5dac0090b0 2024-01-24T23:27:31.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 --> v1:172.21.15.115:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5dac00c310 con 0x7f5dac0090b0 2024-01-24T23:27:31.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 2829137918) 0x7f5dc0003ee0 con 0x7f5dac0090b0 2024-01-24T23:27:31.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dc97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:27:31.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dc97fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:31.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 >> v1:172.21.15.115:6792/0 conn(0x7f5dac0090b0 legacy=0x7f5dac009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dc97fa640 1 -- 172.21.15.115:0/269323512 >> v1:172.21.15.115:6792/0 conn(0x7f5dac0090b0 legacy=0x7f5dac009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:31.567 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:27:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:27:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 40, 2024-01-24T23:27:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:27:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:27:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:27:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:27:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:27:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:27:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:27:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:27:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:27:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:27:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 1, 2024-01-24T23:27:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:27:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:27:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:27:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:27:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:27:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:27:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:27:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:27:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:27:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:27:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:27:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:27:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:27:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:27:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:27:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:27:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:27:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:27:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:27:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:27:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:27:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:27:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:27:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:27:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:27:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:27:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:27:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:27:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:27:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:27:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:27:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:27:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:27:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:27:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:27:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:27:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:27:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:27:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:27:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:27:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:27:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:27:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:27:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:27:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:27:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:27:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:27:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:27:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:27:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:27:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:27:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:27:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:27:31.587 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:27:31.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dd2b6f640 1 -- 172.21.15.115:0/269323512 >> v1:172.21.15.132:6812/1835401922 conn(0x7f5db80378c0 legacy=0x7f5db8039d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dd2b6f640 10 monclient: shutdown 2024-01-24T23:27:31.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dd2b6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:31.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dd2b6f640 1 -- 172.21.15.115:0/269323512 >> v1:172.21.15.115:6790/0 conn(0x7f5dcc1343b0 legacy=0x7f5dcc1d3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.588 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dd110c640 1 -- 172.21.15.115:0/269323512 reap_dead start 2024-01-24T23:27:31.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dd110c640 1 -- 172.21.15.115:0/269323512 reap_dead start 2024-01-24T23:27:31.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dd2b6f640 1 -- 172.21.15.115:0/269323512 shutdown_connections 2024-01-24T23:27:31.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dd2b6f640 1 -- 172.21.15.115:0/269323512 >> 172.21.15.115:0/269323512 conn(0x7f5dcc0662f0 msgr2=0x7f5dcc1bea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:31.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.544+0000 7f5dd2b6f640 1 -- 172.21.15.115:0/269323512 shutdown_connections 2024-01-24T23:27:31.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.548+0000 7f5dd2b6f640 1 -- 172.21.15.115:0/269323512 wait complete. 2024-01-24T23:27:31.589 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:27:31.589 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-01-24T23:27:31.589 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-01-24T23:27:31.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb650fc640 10 monclient: get_monmap_and_config 2024-01-24T23:27:31.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb650fc640 10 monclient: build_initial_monmap 2024-01-24T23:27:31.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb650fc640 10 monclient: monmap: 2024-01-24T23:27:31.703 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:31.703 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.703 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:31.703929+0000 2024-01-24T23:27:31.703 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:31.703929+0000 2024-01-24T23:27:31.703 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:31.703 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.703 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:31.703 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:31.703 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:31.703 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:31.703 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:31.703 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:31.703 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:31.704 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:31.704 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:31.704 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb650fc640 1 Processor -- start 2024-01-24T23:27:31.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb650fc640 1 -- start start 2024-01-24T23:27:31.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb650fc640 10 monclient: init 2024-01-24T23:27:31.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb650fc640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:31.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb650fc640 10 monclient: _add_conns ranks=[6,8,0,2,1,7,3,5,4] 2024-01-24T23:27:31.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb650fc640 10 monclient(hunting): picked mon.noname-d con 0x7fbb60136f20 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:31.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb650fc640 10 monclient(hunting): picked mon.noname-e con 0x7fbb60139ca0 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:31.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb650fc640 10 monclient(hunting): picked mon.noname-a con 0x7fbb6013d7b0 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:31.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb650fc640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb6004ed70 con 0x7fbb6013d7b0 2024-01-24T23:27:31.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb650fc640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb600a23f0 con 0x7fbb60136f20 2024-01-24T23:27:31.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb650fc640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb600a16c0 con 0x7fbb60139ca0 2024-01-24T23:27:31.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb650fc640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:31.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb650fc640 10 monclient(hunting): authenticate will time out at 1330.626774s 2024-01-24T23:27:31.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5ed9d640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fbb60136f20 0x7fbb60137300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:40446/0 (socket says 172.21.15.115:40446) 2024-01-24T23:27:31.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5ed9d640 1 -- 172.21.15.115:0/1965545580 learned_addr learned my addr 172.21.15.115:0/1965545580 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:31.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5e59c640 1 --1- 172.21.15.115:0/1965545580 >> v1:172.21.15.115:6793/0 conn(0x7fbb60139ca0 0x7fbb6013c090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:48996/0 (socket says 172.21.15.115:48996) 2024-01-24T23:27:31.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 1 -- 172.21.15.115:0/1965545580 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 451039298 0 0) 0x7fbb600a23f0 con 0x7fbb60136f20 2024-01-24T23:27:31.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 10 monclient(hunting): my global_id is 6738 2024-01-24T23:27:31.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 1 -- 172.21.15.115:0/1965545580 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb4c003530 con 0x7fbb60136f20 2024-01-24T23:27:31.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 1 -- 172.21.15.115:0/1965545580 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 796333880 0 0) 0x7fbb6004ed70 con 0x7fbb6013d7b0 2024-01-24T23:27:31.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 10 monclient(hunting): my global_id is 94914 2024-01-24T23:27:31.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 1 -- 172.21.15.115:0/1965545580 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb600a23f0 con 0x7fbb6013d7b0 2024-01-24T23:27:31.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 1 -- 172.21.15.115:0/1965545580 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1870978661 0 0) 0x7fbb600a16c0 con 0x7fbb60139ca0 2024-01-24T23:27:31.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 10 monclient(hunting): my global_id is 85778 2024-01-24T23:27:31.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 1 -- 172.21.15.115:0/1965545580 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb6004ed70 con 0x7fbb60139ca0 2024-01-24T23:27:31.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 1 -- 172.21.15.115:0/1965545580 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2099415683 0 0) 0x7fbb4c003530 con 0x7fbb60136f20 2024-01-24T23:27:31.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 1 -- 172.21.15.115:0/1965545580 >> v1:172.21.15.115:6793/0 conn(0x7fbb60139ca0 legacy=0x7fbb6013c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 1 -- 172.21.15.115:0/1965545580 >> v1:172.21.15.115:6789/0 conn(0x7fbb6013d7b0 legacy=0x7fbb6013fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 10 monclient: _finish_hunting 0 2024-01-24T23:27:31.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 1 monclient: found mon.noname-d 2024-01-24T23:27:31.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:31.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:27:31.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 1 -- 172.21.15.115:0/1965545580 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb600a0800 con 0x7fbb60136f20 2024-01-24T23:27:31.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 10 monclient: _finish_auth 0 2024-01-24T23:27:31.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 10 monclient: _check_auth_tickets 2024-01-24T23:27:31.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:31.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 1 -- 172.21.15.115:0/1965545580 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbb500017c0 con 0x7fbb60136f20 2024-01-24T23:27:31.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb650fc640 5 monclient: authenticate success, global_id 6738 2024-01-24T23:27:31.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb650fc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:31.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:27:31.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 10 monclient: dump: 2024-01-24T23:27:31.715 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.715 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.716 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb650fc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 1 -- 172.21.15.115:0/1965545580 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbb50001aa0 con 0x7fbb60136f20 2024-01-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 1 -- 172.21.15.115:0/1965545580 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbb50004bf0 con 0x7fbb60136f20 2024-01-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb5dd9b640 10 monclient: dump: 2024-01-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.717 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.718 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.718 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.718 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.718 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.718 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.718 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.718 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.718 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.718 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb650fc640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:31.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.700+0000 7fbb650fc640 4 set_mon_vals no callback set 2024-01-24T23:27:31.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 10 monclient: shutdown 2024-01-24T23:27:31.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:31.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 1 -- 172.21.15.115:0/1965545580 >> v1:172.21.15.115:6792/0 conn(0x7fbb60136f20 legacy=0x7fbb60137300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 1 -- 172.21.15.115:0/1965545580 shutdown_connections 2024-01-24T23:27:31.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 1 -- 172.21.15.115:0/1965545580 >> 172.21.15.115:0/1965545580 conn(0x7fbb600a5730 msgr2=0x7fbb600a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:31.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 1 -- 172.21.15.115:0/1965545580 shutdown_connections 2024-01-24T23:27:31.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 1 -- 172.21.15.115:0/1965545580 wait complete. 2024-01-24T23:27:31.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 10 monclient: build_initial_monmap 2024-01-24T23:27:31.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 10 monclient: monmap: 2024-01-24T23:27:31.719 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:31.719 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:31.719 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:31.719 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:31.719 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:31.719 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:31.719 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:31.719 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:31.719 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:31.719 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:31.720 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:31.720 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:31.720 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:31.720 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:31.720 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:31.720 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 1 Processor -- start 2024-01-24T23:27:31.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 1 -- start start 2024-01-24T23:27:31.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 10 monclient: init 2024-01-24T23:27:31.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:31.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 10 monclient: _add_conns ranks=[7,4,5,1,0,6,3,2,8] 2024-01-24T23:27:31.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 10 monclient(hunting): picked mon.noname-i con 0x7fbb60136f20 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:31.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 10 monclient(hunting): picked mon.noname-c con 0x7fbb60139ca0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:31.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 10 monclient(hunting): picked mon.noname-h con 0x7fbb6013d7b0 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:31.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb6009be00 con 0x7fbb60139ca0 2024-01-24T23:27:31.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb6013a5b0 con 0x7fbb6013d7b0 2024-01-24T23:27:31.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb60132c20 con 0x7fbb60136f20 2024-01-24T23:27:31.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:31.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 10 monclient(hunting): authenticate will time out at 1330.630179s 2024-01-24T23:27:31.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb5e59c640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fbb60139ca0 0x7fbb60132510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:43656/0 (socket says 172.21.15.115:43656) 2024-01-24T23:27:31.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb5e59c640 1 -- 172.21.15.115:0/1444808301 learned_addr learned my addr 172.21.15.115:0/1444808301 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:31.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967718400 0 0) 0x7fbb6009be00 con 0x7fbb60139ca0 2024-01-24T23:27:31.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 10 monclient(hunting): my global_id is 94756 2024-01-24T23:27:31.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb40003530 con 0x7fbb60139ca0 2024-01-24T23:27:31.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 69482348 0 0) 0x7fbb40003530 con 0x7fbb60139ca0 2024-01-24T23:27:31.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbb6009be00 con 0x7fbb60139ca0 2024-01-24T23:27:31.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbb48001a20 con 0x7fbb60139ca0 2024-01-24T23:27:31.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:27:31.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 10 monclient(hunting): dump: 2024-01-24T23:27:31.722 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.722 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.722 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.722 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.722 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.722 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.722 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.722 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.722 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.722 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.723 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.723 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.723 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.723 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.723 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.723 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:31.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1283081329 0 0) 0x7fbb60132c20 con 0x7fbb60136f20 2024-01-24T23:27:31.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 10 monclient(hunting): my global_id is 124144 2024-01-24T23:27:31.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb40003530 con 0x7fbb60136f20 2024-01-24T23:27:31.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3820287268 0 0) 0x7fbb6013a5b0 con 0x7fbb6013d7b0 2024-01-24T23:27:31.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 10 monclient(hunting): my global_id is 6566 2024-01-24T23:27:31.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb60132c20 con 0x7fbb6013d7b0 2024-01-24T23:27:31.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2277912187 0 0) 0x7fbb6009be00 con 0x7fbb60139ca0 2024-01-24T23:27:31.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 >> v1:172.21.15.132:6792/0 conn(0x7fbb60136f20 legacy=0x7fbb60131e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 >> v1:172.21.15.132:6791/0 conn(0x7fbb6013d7b0 legacy=0x7fbb601d3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 10 monclient: _finish_hunting 0 2024-01-24T23:27:31.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 1 monclient: found mon.c 2024-01-24T23:27:31.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:31.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:31.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb60132ec0 con 0x7fbb60139ca0 2024-01-24T23:27:31.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 10 monclient: _finish_auth 0 2024-01-24T23:27:31.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:31.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:31.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 5 monclient: authenticate success, global_id 94756 2024-01-24T23:27:31.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 10 monclient: _renew_subs 2024-01-24T23:27:31.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:31.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 1 -- 172.21.15.115:0/1444808301 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb601337c0 con 0x7fbb60139ca0 2024-01-24T23:27:31.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 10 monclient: _renew_subs 2024-01-24T23:27:31.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:31.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 1 -- 172.21.15.115:0/1444808301 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6013abe0 con 0x7fbb60139ca0 2024-01-24T23:27:31.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbb48004b20 con 0x7fbb60139ca0 2024-01-24T23:27:31.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:31.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb5cd99640 4 set_mon_vals no callback set 2024-01-24T23:27:31.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbb48005050 con 0x7fbb60139ca0 2024-01-24T23:27:31.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb577fe640 10 monclient: dump: 2024-01-24T23:27:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.726 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.727 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.727 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.727 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:31.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.704+0000 7fbb650fc640 1 -- 172.21.15.115:0/1444808301 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbb440065d0 con 0x7fbb44002870 2024-01-24T23:27:31.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.708+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fbb48005230 con 0x7fbb60139ca0 2024-01-24T23:27:31.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.708+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(216..216 src has 1..216) v4 ==== 13538+0+0 (unknown 289362477 0 0) 0x7fbb48045e70 con 0x7fbb60139ca0 2024-01-24T23:27:31.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.708+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1358536098 0 0) 0x7fbb440065d0 con 0x7fbb44002870 2024-01-24T23:27:31.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.708+0000 7fbb577fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.708+0000 7fbb577fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.708+0000 7fbb577fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.708+0000 7fbb577fe640 10 monclient(hunting): my global_id is 85787 2024-01-24T23:27:31.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.708+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb6009be00 con 0x7fbb44002870 2024-01-24T23:27:31.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.708+0000 7fbb5f59e640 10 monclient: get_auth_request con 0x7fbb40037140 auth_method 0 2024-01-24T23:27:31.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.708+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 883557734 0 0) 0x7fbb6009be00 con 0x7fbb44002870 2024-01-24T23:27:31.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.708+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbb440065d0 con 0x7fbb44002870 2024-01-24T23:27:31.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.708+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbb50001780 con 0x7fbb44002870 2024-01-24T23:27:31.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.708+0000 7fbb577fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.708+0000 7fbb577fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:27:31.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.708+0000 7fbb577fe640 10 monclient: dump: 2024-01-24T23:27:31.728 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.728 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.728 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.728 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.728 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.728 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.728 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.728 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.729 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.729 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.729 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.729 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.729 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.729 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.729 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.729 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.708+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 212687591 0 0) 0x7fbb440065d0 con 0x7fbb44002870 2024-01-24T23:27:31.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.708+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 --> v1:172.21.15.115:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb4400a120 con 0x7fbb44002870 2024-01-24T23:27:31.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.708+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fbb500049b0 con 0x7fbb44002870 2024-01-24T23:27:31.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.708+0000 7fbb577fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:31.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.708+0000 7fbb577fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:31.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.708+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 >> v1:172.21.15.115:6793/0 conn(0x7fbb44002870 legacy=0x7fbb44004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.708+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 >> v1:172.21.15.115:6793/0 conn(0x7fbb44002870 legacy=0x7fbb44004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:31.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.712+0000 7fbb650fc640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:27:31.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.712+0000 7fbb650fc640 1 -- 172.21.15.115:0/1444808301 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbb44009df0 con 0x7fbb440090b0 2024-01-24T23:27:31.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.712+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1133665388 0 0) 0x7fbb44009df0 con 0x7fbb440090b0 2024-01-24T23:27:31.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.712+0000 7fbb577fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.712+0000 7fbb577fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.712+0000 7fbb577fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.712+0000 7fbb577fe640 10 monclient(hunting): my global_id is 85796 2024-01-24T23:27:31.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.712+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb440065d0 con 0x7fbb440090b0 2024-01-24T23:27:31.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.712+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2656669576 0 0) 0x7fbb440065d0 con 0x7fbb440090b0 2024-01-24T23:27:31.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.712+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbb44009df0 con 0x7fbb440090b0 2024-01-24T23:27:31.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.712+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fbb50005340 con 0x7fbb440090b0 2024-01-24T23:27:31.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.712+0000 7fbb577fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.716+0000 7fbb577fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:27:31.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.716+0000 7fbb577fe640 10 monclient: dump: 2024-01-24T23:27:31.732 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.732 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.732 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.732 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.732 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.732 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.732 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.732 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.732 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.733 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.733 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.733 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.733 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.733 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.733 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.733 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.716+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1565167825 0 0) 0x7fbb44009df0 con 0x7fbb440090b0 2024-01-24T23:27:31.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.716+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 --> v1:172.21.15.115:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbb4400c310 con 0x7fbb440090b0 2024-01-24T23:27:31.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.716+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 4165052659) 0x7fbb500075e0 con 0x7fbb440090b0 2024-01-24T23:27:31.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.716+0000 7fbb577fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:27:31.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.716+0000 7fbb577fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:31.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.716+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 >> v1:172.21.15.115:6793/0 conn(0x7fbb440090b0 legacy=0x7fbb44009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.716+0000 7fbb577fe640 1 -- 172.21.15.115:0/1444808301 >> v1:172.21.15.115:6793/0 conn(0x7fbb440090b0 legacy=0x7fbb44009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:31.734 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:27:31.734 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:31.734 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:31.734 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:27:31.734 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 40, 2024-01-24T23:27:31.734 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:27:31.734 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:27:31.734 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:27:31.734 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:27:31.734 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:27:31.734 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:27:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:27:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:27:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:27:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:27:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 1, 2024-01-24T23:27:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:27:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:27:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:27:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:27:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:27:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:27:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:27:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:27:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:27:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:27:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:27:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:27:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:27:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:27:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:27:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:27:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:27:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:27:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:27:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:27:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:27:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:27:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:27:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:27:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:27:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:27:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:27:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:27:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:27:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:27:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:27:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:27:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:27:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.745 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.745 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.745 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:27:31.745 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:27:31.745 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.745 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.745 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.745 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:31.745 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:31.745 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.745 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:27:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:27:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:27:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:27:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:27:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:27:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:27:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:27:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:27:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:27:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:27:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:27:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:27:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:27:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:27:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": [ 2024-01-24T23:27:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.752 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:31.752 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:31.752 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:31.752 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.752 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.752 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:27:31.752 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:27:31.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.716+0000 7fbb650fc640 1 -- 172.21.15.115:0/1444808301 >> v1:172.21.15.132:6812/1835401922 conn(0x7fbb40037140 legacy=0x7fbb40039600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.716+0000 7fbb650fc640 10 monclient: shutdown 2024-01-24T23:27:31.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.716+0000 7fbb650fc640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:31.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.716+0000 7fbb650fc640 1 -- 172.21.15.115:0/1444808301 >> v1:172.21.15.115:6791/0 conn(0x7fbb60139ca0 legacy=0x7fbb60132510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.716+0000 7fbb5f59e640 1 -- 172.21.15.115:0/1444808301 reap_dead start 2024-01-24T23:27:31.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.716+0000 7fbb5f59e640 1 -- 172.21.15.115:0/1444808301 reap_dead start 2024-01-24T23:27:31.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.716+0000 7fbb650fc640 1 -- 172.21.15.115:0/1444808301 shutdown_connections 2024-01-24T23:27:31.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.716+0000 7fbb650fc640 1 -- 172.21.15.115:0/1444808301 >> 172.21.15.115:0/1444808301 conn(0x7fbb600a5730 msgr2=0x7fbb60070ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:31.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.716+0000 7fbb650fc640 1 -- 172.21.15.115:0/1444808301 shutdown_connections 2024-01-24T23:27:31.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.716+0000 7fbb650fc640 1 -- 172.21.15.115:0/1444808301 wait complete. 2024-01-24T23:27:31.754 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:27:31.754 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-01-24T23:27:31.754 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-01-24T23:27:31.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.868+0000 7face5166640 10 monclient: get_monmap_and_config 2024-01-24T23:27:31.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.868+0000 7face5166640 10 monclient: build_initial_monmap 2024-01-24T23:27:31.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.868+0000 7face5166640 10 monclient: monmap: 2024-01-24T23:27:31.872 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:31.872 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.872 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:31.873579+0000 2024-01-24T23:27:31.872 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:31.873579+0000 2024-01-24T23:27:31.872 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:31.873 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.873 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:31.873 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:31.873 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:31.873 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:31.873 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:31.873 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:31.873 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:31.873 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:31.873 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:31.873 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.868+0000 7face5166640 1 Processor -- start 2024-01-24T23:27:31.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.868+0000 7face5166640 1 -- start start 2024-01-24T23:27:31.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.868+0000 7face5166640 10 monclient: init 2024-01-24T23:27:31.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:31.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 10 monclient: _add_conns ranks=[7,6,2,3,1,4,8,5,0] 2024-01-24T23:27:31.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 10 monclient(hunting): picked mon.noname-i con 0x7face0133bf0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:31.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 10 monclient(hunting): picked mon.noname-d con 0x7face0133fd0 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:31.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 10 monclient(hunting): picked mon.noname-b con 0x7face0134430 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:31.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face004ed70 con 0x7face0134430 2024-01-24T23:27:31.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face009af30 con 0x7face0133fd0 2024-01-24T23:27:31.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face009b210 con 0x7face0133bf0 2024-01-24T23:27:31.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:31.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 10 monclient(hunting): authenticate will time out at 1330.796600s 2024-01-24T23:27:31.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facdf59e640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7face0134430 0x7face013f500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:36706/0 (socket says 172.21.15.115:36706) 2024-01-24T23:27:31.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facde59c640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7face0133fd0 0x7face013bdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:40450/0 (socket says 172.21.15.115:40450) 2024-01-24T23:27:31.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facdf59e640 1 -- 172.21.15.115:0/981190948 learned_addr learned my addr 172.21.15.115:0/981190948 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:31.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 1 -- 172.21.15.115:0/981190948 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516720417 0 0) 0x7face009af30 con 0x7face0133fd0 2024-01-24T23:27:31.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 10 monclient(hunting): my global_id is 6747 2024-01-24T23:27:31.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 1 -- 172.21.15.115:0/981190948 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facc8003530 con 0x7face0133fd0 2024-01-24T23:27:31.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 1 -- 172.21.15.115:0/981190948 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458554459 0 0) 0x7face004ed70 con 0x7face0134430 2024-01-24T23:27:31.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 10 monclient(hunting): my global_id is 124184 2024-01-24T23:27:31.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 1 -- 172.21.15.115:0/981190948 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7face009af30 con 0x7face0134430 2024-01-24T23:27:31.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 1 -- 172.21.15.115:0/981190948 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 395294617 0 0) 0x7facc8003530 con 0x7face0133fd0 2024-01-24T23:27:31.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 1 -- 172.21.15.115:0/981190948 >> v1:172.21.15.132:6792/0 conn(0x7face0133bf0 legacy=0x7face009bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 1 -- 172.21.15.115:0/981190948 >> v1:172.21.15.115:6790/0 conn(0x7face0134430 legacy=0x7face013f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 10 monclient: _finish_hunting 0 2024-01-24T23:27:31.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 1 monclient: found mon.noname-d 2024-01-24T23:27:31.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:31.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:27:31.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 1 -- 172.21.15.115:0/981190948 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face009b4f0 con 0x7face0133fd0 2024-01-24T23:27:31.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 10 monclient: _finish_auth 0 2024-01-24T23:27:31.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 10 monclient: _check_auth_tickets 2024-01-24T23:27:31.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:31.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 1 -- 172.21.15.115:0/981190948 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7facd40017c0 con 0x7face0133fd0 2024-01-24T23:27:31.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 5 monclient: authenticate success, global_id 6747 2024-01-24T23:27:31.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:31.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:27:31.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 10 monclient: dump: 2024-01-24T23:27:31.885 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.885 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.885 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.885 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.885 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.885 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.885 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.885 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.885 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.885 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.885 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.885 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.885 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.885 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.886 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.886 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:31.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 1 -- 172.21.15.115:0/981190948 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7facd40019e0 con 0x7face0133fd0 2024-01-24T23:27:31.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:31.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 1 -- 172.21.15.115:0/981190948 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7facd4004b30 con 0x7face0133fd0 2024-01-24T23:27:31.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:31.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7facddd9b640 10 monclient: dump: 2024-01-24T23:27:31.886 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.886 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.887 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.887 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.887 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.887 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.887 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.887 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.887 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.887 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.887 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.887 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.887 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.887 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.887 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.887 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:31.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 4 set_mon_vals no callback set 2024-01-24T23:27:31.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 10 monclient: shutdown 2024-01-24T23:27:31.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:31.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 1 -- 172.21.15.115:0/981190948 >> v1:172.21.15.115:6792/0 conn(0x7face0133fd0 legacy=0x7face013bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 1 -- 172.21.15.115:0/981190948 shutdown_connections 2024-01-24T23:27:31.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 1 -- 172.21.15.115:0/981190948 >> 172.21.15.115:0/981190948 conn(0x7face00a9910 msgr2=0x7face00a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:31.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 1 -- 172.21.15.115:0/981190948 shutdown_connections 2024-01-24T23:27:31.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 1 -- 172.21.15.115:0/981190948 wait complete. 2024-01-24T23:27:31.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 10 monclient: build_initial_monmap 2024-01-24T23:27:31.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 10 monclient: monmap: 2024-01-24T23:27:31.889 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:31.889 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:31.889 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:31.889 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:31.889 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:31.889 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:31.889 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:31.889 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:31.889 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:31.889 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:31.889 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:31.889 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:31.889 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:31.889 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:31.890 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:31.890 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 1 Processor -- start 2024-01-24T23:27:31.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 1 -- start start 2024-01-24T23:27:31.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 10 monclient: init 2024-01-24T23:27:31.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:31.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 10 monclient: _add_conns ranks=[5,3,1,6,4,8,2,0,7] 2024-01-24T23:27:31.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 10 monclient(hunting): picked mon.noname-h con 0x7face0133bf0 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:31.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 10 monclient(hunting): picked mon.noname-g con 0x7face0133fd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:31.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 10 monclient(hunting): picked mon.noname-f con 0x7face0134430 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:31.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face0138c40 con 0x7face0134430 2024-01-24T23:27:31.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face013af30 con 0x7face0133fd0 2024-01-24T23:27:31.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face01389c0 con 0x7face0133bf0 2024-01-24T23:27:31.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:31.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.872+0000 7face5166640 10 monclient(hunting): authenticate will time out at 1330.799760s 2024-01-24T23:27:31.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facded9d640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7face0133bf0 0x7face0137ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.115:59678/0 (socket says 172.21.15.115:59678) 2024-01-24T23:27:31.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facded9d640 1 -- 172.21.15.115:0/1857054022 learned_addr learned my addr 172.21.15.115:0/1857054022 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:31.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2986348173 0 0) 0x7face01389c0 con 0x7face0133bf0 2024-01-24T23:27:31.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): my global_id is 6575 2024-01-24T23:27:31.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faccc003530 con 0x7face0133bf0 2024-01-24T23:27:31.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2027850213 0 0) 0x7face013af30 con 0x7face0133fd0 2024-01-24T23:27:31.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): my global_id is 6582 2024-01-24T23:27:31.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7face01389c0 con 0x7face0133fd0 2024-01-24T23:27:31.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875504956 0 0) 0x7face0138c40 con 0x7face0134430 2024-01-24T23:27:31.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): my global_id is 124129 2024-01-24T23:27:31.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7face013af30 con 0x7face0134430 2024-01-24T23:27:31.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2048169096 0 0) 0x7faccc003530 con 0x7face0133bf0 2024-01-24T23:27:31.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7face0138c40 con 0x7face0133bf0 2024-01-24T23:27:31.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3916675055 0 0) 0x7face013af30 con 0x7face0134430 2024-01-24T23:27:31.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faccc003530 con 0x7face0134430 2024-01-24T23:27:31.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7facd0001b20 con 0x7face0133bf0 2024-01-24T23:27:31.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:27:31.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): dump: 2024-01-24T23:27:31.893 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.893 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.893 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.893 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.893 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.893 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.893 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.893 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.893 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.893 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.893 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.893 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.893 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.893 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.894 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.894 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:31.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7facd8001b40 con 0x7face0134430 2024-01-24T23:27:31.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:27:31.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): dump: 2024-01-24T23:27:31.894 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.894 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.894 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.894 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.894 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.894 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.894 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.894 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.894 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.895 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.895 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.895 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.895 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.895 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.895 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.895 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 728354288 0 0) 0x7face01389c0 con 0x7face0133fd0 2024-01-24T23:27:31.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7face013af30 con 0x7face0133fd0 2024-01-24T23:27:31.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7facd4004b30 con 0x7face0133fd0 2024-01-24T23:27:31.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:31.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): dump: 2024-01-24T23:27:31.895 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.895 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.895 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.895 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.896 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.896 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.896 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.896 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.896 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.896 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.896 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.896 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.896 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.896 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.896 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.896 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4060709514 0 0) 0x7face0138c40 con 0x7face0133bf0 2024-01-24T23:27:31.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 >> v1:172.21.15.132:6790/0 conn(0x7face0133fd0 legacy=0x7face01382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 >> v1:172.21.15.132:6789/0 conn(0x7face0134430 legacy=0x7face01d3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient: _finish_hunting 0 2024-01-24T23:27:31.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 monclient: found mon.h 2024-01-24T23:27:31.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:31.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:31.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face009c8f0 con 0x7face0133bf0 2024-01-24T23:27:31.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient: _finish_auth 0 2024-01-24T23:27:31.897 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:31.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:31.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7face5166640 5 monclient: authenticate success, global_id 6575 2024-01-24T23:27:31.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7face5166640 10 monclient: _renew_subs 2024-01-24T23:27:31.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7face5166640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:31.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7face5166640 1 -- 172.21.15.115:0/1857054022 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face00a5960 con 0x7face0133bf0 2024-01-24T23:27:31.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7face5166640 10 monclient: _renew_subs 2024-01-24T23:27:31.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7face5166640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:31.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7face5166640 1 -- 172.21.15.115:0/1857054022 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7face0139520 con 0x7face0133bf0 2024-01-24T23:27:31.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7facd00038d0 con 0x7face0133bf0 2024-01-24T23:27:31.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:31.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7facd0003e00 con 0x7face0133bf0 2024-01-24T23:27:31.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facdd59a640 4 set_mon_vals no callback set 2024-01-24T23:27:31.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:31.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient: dump: 2024-01-24T23:27:31.899 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.899 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.899 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.899 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.899 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.899 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.899 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.899 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.899 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.899 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.899 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.899 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.899 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.900 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.900 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.900 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7face5166640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:31.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7face5166640 1 -- 172.21.15.115:0/1857054022 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7facc00065d0 con 0x7facc0002870 2024-01-24T23:27:31.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7facd0003fe0 con 0x7face0133bf0 2024-01-24T23:27:31.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(216..216 src has 1..216) v4 ==== 13538+0+0 (unknown 289362477 0 0) 0x7facd0045e70 con 0x7face0133bf0 2024-01-24T23:27:31.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2333397060 0 0) 0x7facc00065d0 con 0x7facc0002870 2024-01-24T23:27:31.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facdf59e640 10 monclient: get_auth_request con 0x7faccc037b00 auth_method 0 2024-01-24T23:27:31.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 10 monclient(hunting): my global_id is 124138 2024-01-24T23:27:31.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.876+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7face0138c40 con 0x7facc0002870 2024-01-24T23:27:31.900 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.880+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2768436479 0 0) 0x7face0138c40 con 0x7facc0002870 2024-01-24T23:27:31.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.880+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7facc00065d0 con 0x7facc0002870 2024-01-24T23:27:31.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.880+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7facd4004e40 con 0x7facc0002870 2024-01-24T23:27:31.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.880+0000 7facc77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.880+0000 7facc77fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:27:31.901 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.880+0000 7facc77fe640 10 monclient: dump: 2024-01-24T23:27:31.901 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.901 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.901 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.901 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.901 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.901 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.901 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.901 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.901 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.901 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.901 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.902 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.902 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.902 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.902 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.902 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.880+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1094349603 0 0) 0x7facc00065d0 con 0x7facc0002870 2024-01-24T23:27:31.902 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.880+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 --> v1:172.21.15.132:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7facc000a120 con 0x7facc0002870 2024-01-24T23:27:31.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.880+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 <== mon.1 v1:172.21.15.132:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7facd4007040 con 0x7facc0002870 2024-01-24T23:27:31.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.880+0000 7facc77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:31.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.880+0000 7facc77fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:31.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.880+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 >> v1:172.21.15.132:6789/0 conn(0x7facc0002870 legacy=0x7facc0004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.880+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 >> v1:172.21.15.132:6789/0 conn(0x7facc0002870 legacy=0x7facc0004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:31.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.884+0000 7face5166640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:27:31.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.884+0000 7face5166640 1 -- 172.21.15.115:0/1857054022 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7facc0009e10 con 0x7facc00090d0 2024-01-24T23:27:31.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799403802 0 0) 0x7facc0009e10 con 0x7facc00090d0 2024-01-24T23:27:31.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7facc77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:31.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7facc77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:31.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7facc77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:31.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7facc77fe640 10 monclient(hunting): my global_id is 124147 2024-01-24T23:27:31.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facc00065d0 con 0x7facc00090d0 2024-01-24T23:27:31.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2893886807 0 0) 0x7facc00065d0 con 0x7facc00090d0 2024-01-24T23:27:31.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7facc0009e10 con 0x7facc00090d0 2024-01-24T23:27:31.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7facd40058e0 con 0x7facc00090d0 2024-01-24T23:27:31.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7facc77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:31.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7facc77fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:27:31.905 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7facc77fe640 10 monclient: dump: 2024-01-24T23:27:31.905 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:31.905 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:31.905 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.905 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:31.905 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:31.906 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:31.906 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:31.906 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:31.906 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:31.906 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:31.906 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:31.906 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:31.906 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:31.906 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:31.906 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:31.906 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:31.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2633320564 0 0) 0x7facc0009e10 con 0x7facc00090d0 2024-01-24T23:27:31.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 --> v1:172.21.15.132:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7facc000c310 con 0x7facc00090d0 2024-01-24T23:27:31.906 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 <== mon.1 v1:172.21.15.132:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 846041716) 0x7facd4006d00 con 0x7facc00090d0 2024-01-24T23:27:31.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7facc77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:27:31.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7facc77fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:31.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 >> v1:172.21.15.132:6789/0 conn(0x7facc00090d0 legacy=0x7facc0009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.907 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7facc77fe640 1 -- 172.21.15.115:0/1857054022 >> v1:172.21.15.132:6789/0 conn(0x7facc00090d0 legacy=0x7facc0009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:31.907 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:27:31.907 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:31.907 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:31.907 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:27:31.907 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 40, 2024-01-24T23:27:31.907 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:27:31.908 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:27:31.908 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:27:31.908 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:27:31.908 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:27:31.908 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:27:31.908 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:27:31.908 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:27:31.908 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:27:31.908 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:27:31.908 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.908 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 1, 2024-01-24T23:27:31.908 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:31.908 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:27:31.909 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:27:31.909 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:31.909 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:31.909 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:31.909 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:31.909 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:31.909 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:31.909 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:31.909 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:31.909 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:31.909 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:31.909 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:31.909 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.909 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:27:31.909 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:27:31.910 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:31.910 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:31.910 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:31.910 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:31.910 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:31.910 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:31.910 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:31.910 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:31.910 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:31.910 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:31.910 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:31.910 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.911 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:27:31.911 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:27:31.911 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:27:31.911 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:27:31.911 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:27:31.911 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:27:31.911 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:31.911 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:31.911 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:27:31.911 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:27:31.911 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:27:31.911 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:27:31.911 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:27:31.911 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:27:31.911 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:27:31.912 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:31.912 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:27:31.912 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:31.912 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:31.912 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:31.912 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:31.912 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:31.912 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:31.912 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:31.912 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:31.912 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:31.912 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:31.912 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:31.912 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.913 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:27:31.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.913 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:27:31.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.913 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:31.913 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:31.913 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.913 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.913 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.913 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:27:31.913 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.914 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.914 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:27:31.914 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:27:31.914 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.914 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.914 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.914 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:31.914 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:31.914 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.914 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.915 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.915 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:27:31.915 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.915 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.915 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.915 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:27:31.915 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:27:31.915 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.915 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.915 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.915 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:31.916 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:31.916 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.916 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.916 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.916 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:27:31.916 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.916 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.916 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.916 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:27:31.916 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:27:31.916 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.916 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.917 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.917 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:31.917 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:31.917 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.917 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.917 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.917 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:27:31.917 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.917 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.917 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.918 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:27:31.918 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:27:31.918 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.918 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.918 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.918 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:31.918 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:31.918 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.918 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.918 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.919 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:27:31.919 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.919 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.919 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.919 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:27:31.919 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:27:31.919 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.919 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.919 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.919 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:31.919 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:31.919 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.920 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.920 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.920 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:27:31.920 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.920 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.920 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:27:31.920 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:27:31.920 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.920 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.920 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.921 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:31.921 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:31.921 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.921 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.921 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.921 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:27:31.921 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.921 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.921 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.921 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:27:31.921 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:27:31.921 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.922 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.922 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.922 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:31.922 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:31.922 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.922 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.922 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.922 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:27:31.922 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.922 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.922 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.923 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:27:31.923 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:27:31.923 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.923 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.923 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.923 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:31.923 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:31.923 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:31.923 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:31.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.923 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:31.923 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:27:31.923 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:31.924 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.924 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.924 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:27:31.924 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:27:31.924 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:31.924 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:31.924 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:31.924 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.924 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.924 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:27:31.924 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:27:31.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.925 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:31.925 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:31.925 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:31.925 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.925 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:31.925 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:27:31.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:31.925 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:31.925 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:31.925 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:31.925 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:31.925 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:31.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:31.926 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:27:31.926 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:27:31.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7face5166640 1 -- 172.21.15.115:0/1857054022 >> v1:172.21.15.132:6812/1835401922 conn(0x7faccc037b00 legacy=0x7faccc039fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7face5166640 10 monclient: shutdown 2024-01-24T23:27:31.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7face5166640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:31.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7face5166640 1 -- 172.21.15.115:0/1857054022 >> v1:172.21.15.132:6791/0 conn(0x7face0133bf0 legacy=0x7face0137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:31.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7facdf59e640 1 -- 172.21.15.115:0/1857054022 reap_dead start 2024-01-24T23:27:31.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7facdf59e640 1 -- 172.21.15.115:0/1857054022 reap_dead start 2024-01-24T23:27:31.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7face5166640 1 -- 172.21.15.115:0/1857054022 shutdown_connections 2024-01-24T23:27:31.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7face5166640 1 -- 172.21.15.115:0/1857054022 >> 172.21.15.115:0/1857054022 conn(0x7face00a9910 msgr2=0x7face013ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:31.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7face5166640 1 -- 172.21.15.115:0/1857054022 shutdown_connections 2024-01-24T23:27:31.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:31.888+0000 7face5166640 1 -- 172.21.15.115:0/1857054022 wait complete. 2024-01-24T23:27:31.928 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:27:31.928 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-01-24T23:27:31.928 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-01-24T23:27:31.929 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.926+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h5"}]=0 removed item id -31 name 'h5' from crush map v217) v1 ==== 118+0+0 (unknown 794371125 0 0) 0x7fa23c013160 con 0x7fa25409ff20 2024-01-24T23:27:31.929 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.926+0000 7fa24bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h5"}] 2024-01-24T23:27:31.929 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:31.926+0000 7fa24bfff640 10 monclient: _finish_command 2 = system:0 removed item id -31 name 'h5' from crush map 2024-01-24T23:27:32.044 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.042+0000 7fa25a0dc640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h5"}] 2024-01-24T23:27:32.044 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.042+0000 7fa25a0dc640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h5"}] 2024-01-24T23:27:32.044 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.042+0000 7fa25a0dc640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:32.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.042+0000 7fa25a0dc640 1 -- 172.21.15.132:0/1451332584 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h5"} v 0) v1 -- 0x7fa2380010a0 con 0x7fa25409ff20 2024-01-24T23:27:32.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.042+0000 7fa24bfff640 1 -- 172.21.15.132:0/1451332584 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h5"}]=0 device 'h5' does not appear in the crush map v217) v1 ==== 118+0+0 (unknown 161363012 0 0) 0x7fa23c0148e0 con 0x7fa25409ff20 2024-01-24T23:27:32.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.042+0000 7fa24bfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h5"}] 2024-01-24T23:27:32.045 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.042+0000 7fa24bfff640 10 monclient: _finish_command 3 = system:0 device 'h5' does not appear in the crush map 2024-01-24T23:27:32.045 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h5' does not appear in the crush map 2024-01-24T23:27:32.047 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.046+0000 7fa25a0dc640 1 -- 172.21.15.132:0/1451332584 >> v1:172.21.15.132:6812/1835401922 conn(0x7fa234037b50 legacy=0x7fa23403a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.046+0000 7fa25a0dc640 10 monclient: shutdown 2024-01-24T23:27:32.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.046+0000 7fa25a0dc640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:32.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.046+0000 7fa25a0dc640 1 -- 172.21.15.132:0/1451332584 >> v1:172.21.15.132:6791/0 conn(0x7fa25409ff20 legacy=0x7fa25409cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.046+0000 7fa25a0dc640 1 -- 172.21.15.132:0/1451332584 shutdown_connections 2024-01-24T23:27:32.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.046+0000 7fa25a0dc640 1 -- 172.21.15.132:0/1451332584 >> 172.21.15.132:0/1451332584 conn(0x7fa2540a5a10 msgr2=0x7fa254066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:32.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.046+0000 7fa25a0dc640 1 -- 172.21.15.132:0/1451332584 shutdown_connections 2024-01-24T23:27:32.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.046+0000 7fa25a0dc640 1 -- 172.21.15.132:0/1451332584 wait complete. 2024-01-24T23:27:32.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.052+0000 7f0938e1d640 10 monclient: get_monmap_and_config 2024-01-24T23:27:32.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.052+0000 7f0938e1d640 10 monclient: build_initial_monmap 2024-01-24T23:27:32.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.052+0000 7f0938e1d640 10 monclient: monmap: 2024-01-24T23:27:32.057 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:32.057 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.057 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:32.058780+0000 2024-01-24T23:27:32.057 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:32.058780+0000 2024-01-24T23:27:32.057 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:32.058 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.058 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:32.058 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:32.058 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:32.058 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:32.058 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:32.058 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:32.058 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:32.058 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:32.058 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:32.058 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.064 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:32.064 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h6 2024-01-24T23:27:32.066 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 1 Processor -- start 2024-01-24T23:27:32.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 1 -- start start 2024-01-24T23:27:32.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 10 monclient: init 2024-01-24T23:27:32.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:32.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 10 monclient: _add_conns ranks=[2,6,5,8,3,1,0,7,4] 2024-01-24T23:27:32.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 10 monclient(hunting): picked mon.noname-b con 0x7f093409e740 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:32.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 10 monclient(hunting): picked mon.noname-d con 0x7f093409f230 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:32.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 10 monclient(hunting): picked mon.noname-h con 0x7f0934071e90 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:32.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f093404ed70 con 0x7f093409e740 2024-01-24T23:27:32.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09340a1400 con 0x7f0934071e90 2024-01-24T23:27:32.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09340a37d0 con 0x7f093409f230 2024-01-24T23:27:32.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:32.067 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 10 monclient(hunting): authenticate will time out at 1330.981506s 2024-01-24T23:27:32.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093259c640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f093409e740 0x7f093409eb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:36708/0 (socket says 172.21.15.115:36708) 2024-01-24T23:27:32.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093259c640 1 -- 172.21.15.115:0/2625082685 learned_addr learned my addr 172.21.15.115:0/2625082685 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:32.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0931d9b640 1 --1- 172.21.15.115:0/2625082685 >> v1:172.21.15.115:6792/0 conn(0x7f093409f230 0x7f0934139b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:40466/0 (socket says 172.21.15.115:40466) 2024-01-24T23:27:32.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 1 -- 172.21.15.115:0/2625082685 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1628683692 0 0) 0x7f093404ed70 con 0x7f093409e740 2024-01-24T23:27:32.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 10 monclient(hunting): my global_id is 124193 2024-01-24T23:27:32.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 1 -- 172.21.15.115:0/2625082685 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f091c003530 con 0x7f093409e740 2024-01-24T23:27:32.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 1 -- 172.21.15.115:0/2625082685 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 456669846 0 0) 0x7f09340a37d0 con 0x7f093409f230 2024-01-24T23:27:32.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.068 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 10 monclient(hunting): my global_id is 6756 2024-01-24T23:27:32.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 1 -- 172.21.15.115:0/2625082685 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f093404ed70 con 0x7f093409f230 2024-01-24T23:27:32.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 1 -- 172.21.15.115:0/2625082685 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2267603682 0 0) 0x7f091c003530 con 0x7f093409e740 2024-01-24T23:27:32.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 1 -- 172.21.15.115:0/2625082685 >> v1:172.21.15.115:6792/0 conn(0x7f093409f230 legacy=0x7f0934139b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 1 -- 172.21.15.115:0/2625082685 >> v1:172.21.15.132:6791/0 conn(0x7f0934071e90 legacy=0x7f093413d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 10 monclient: _finish_hunting 0 2024-01-24T23:27:32.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 1 monclient: found mon.noname-b 2024-01-24T23:27:32.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:32.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:27:32.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 1 -- 172.21.15.115:0/2625082685 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09340a3120 con 0x7f093409e740 2024-01-24T23:27:32.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 10 monclient: _finish_auth 0 2024-01-24T23:27:32.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 10 monclient: _check_auth_tickets 2024-01-24T23:27:32.069 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:32.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 1 -- 172.21.15.115:0/2625082685 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f09240017c0 con 0x7f093409e740 2024-01-24T23:27:32.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:27:32.070 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 10 monclient: dump: 2024-01-24T23:27:32.070 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:32.070 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.070 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.070 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.070 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.070 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.070 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.070 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.070 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.070 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.071 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.071 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.071 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.071 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.071 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.071 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 5 monclient: authenticate success, global_id 124193 2024-01-24T23:27:32.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:32.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 1 -- 172.21.15.115:0/2625082685 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0924001a20 con 0x7f093409e740 2024-01-24T23:27:32.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:32.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:32.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 4 set_mon_vals no callback set 2024-01-24T23:27:32.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 1 -- 172.21.15.115:0/2625082685 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0924004bf0 con 0x7f093409e740 2024-01-24T23:27:32.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:27:32.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f093159a640 10 monclient: dump: 2024-01-24T23:27:32.072 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:32.072 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.072 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.072 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.072 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.072 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.072 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.073 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.073 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.073 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.073 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.073 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.073 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.073 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.073 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.073 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 10 monclient: shutdown 2024-01-24T23:27:32.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:32.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 1 -- 172.21.15.115:0/2625082685 >> v1:172.21.15.115:6790/0 conn(0x7f093409e740 legacy=0x7f093409eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 1 -- 172.21.15.115:0/2625082685 shutdown_connections 2024-01-24T23:27:32.073 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 1 -- 172.21.15.115:0/2625082685 >> 172.21.15.115:0/2625082685 conn(0x7f09340666f0 msgr2=0x7f0934066b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:32.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 1 -- 172.21.15.115:0/2625082685 shutdown_connections 2024-01-24T23:27:32.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 1 -- 172.21.15.115:0/2625082685 wait complete. 2024-01-24T23:27:32.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 10 monclient: build_initial_monmap 2024-01-24T23:27:32.074 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 10 monclient: monmap: 2024-01-24T23:27:32.074 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:32.074 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:32.074 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:32.074 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:32.074 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:32.074 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:32.074 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:32.074 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:32.074 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:32.074 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:32.075 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:32.075 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:32.075 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:32.075 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:32.075 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:32.075 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 1 Processor -- start 2024-01-24T23:27:32.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 1 -- start start 2024-01-24T23:27:32.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.056+0000 7f0938e1d640 10 monclient: init 2024-01-24T23:27:32.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0938e1d640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:32.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0938e1d640 10 monclient: _add_conns ranks=[5,0,7,8,4,2,6,3,1] 2024-01-24T23:27:32.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0938e1d640 10 monclient(hunting): picked mon.noname-h con 0x7f0934071e90 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:32.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0938e1d640 10 monclient(hunting): picked mon.noname-a con 0x7f093409e740 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:32.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0938e1d640 10 monclient(hunting): picked mon.noname-i con 0x7f09341b9d70 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:32.075 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0938e1d640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09340722e0 con 0x7f093409e740 2024-01-24T23:27:32.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0938e1d640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09341a7020 con 0x7f0934071e90 2024-01-24T23:27:32.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0938e1d640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f093409f640 con 0x7f09341b9d70 2024-01-24T23:27:32.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0938e1d640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:32.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0938e1d640 10 monclient(hunting): authenticate will time out at 1330.984449s 2024-01-24T23:27:32.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0931d9b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f093409e740 0x7f09341ce120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:60612/0 (socket says 172.21.15.115:60612) 2024-01-24T23:27:32.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0931d9b640 1 -- 172.21.15.115:0/3551826473 learned_addr learned my addr 172.21.15.115:0/3551826473 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:32.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2295163120 0 0) 0x7f09340722e0 con 0x7f093409e740 2024-01-24T23:27:32.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient(hunting): my global_id is 94923 2024-01-24T23:27:32.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0914003530 con 0x7f093409e740 2024-01-24T23:27:32.076 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718116528 0 0) 0x7f09341a7020 con 0x7f0934071e90 2024-01-24T23:27:32.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient(hunting): my global_id is 6593 2024-01-24T23:27:32.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09340722e0 con 0x7f0934071e90 2024-01-24T23:27:32.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4216610594 0 0) 0x7f0914003530 con 0x7f093409e740 2024-01-24T23:27:32.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f09341a7020 con 0x7f093409e740 2024-01-24T23:27:32.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f09280015e0 con 0x7f093409e740 2024-01-24T23:27:32.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:27:32.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient(hunting): dump: 2024-01-24T23:27:32.077 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:32.077 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.077 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.078 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.078 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.078 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.078 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.078 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.078 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.078 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.078 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.078 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.078 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.078 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.078 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.078 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:32.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4189894870 0 0) 0x7f093409f640 con 0x7f09341b9d70 2024-01-24T23:27:32.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient(hunting): my global_id is 124162 2024-01-24T23:27:32.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0914003530 con 0x7f09341b9d70 2024-01-24T23:27:32.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1173014046 0 0) 0x7f09341a7020 con 0x7f093409e740 2024-01-24T23:27:32.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 >> v1:172.21.15.132:6792/0 conn(0x7f09341b9d70 legacy=0x7f09341d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 >> v1:172.21.15.132:6791/0 conn(0x7f0934071e90 legacy=0x7f09341b9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:27:32.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 monclient: found mon.a 2024-01-24T23:27:32.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:32.079 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:27:32.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09341ba1e0 con 0x7f093409e740 2024-01-24T23:27:32.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient: _finish_auth 0 2024-01-24T23:27:32.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:27:32.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:32.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0938e1d640 5 monclient: authenticate success, global_id 94923 2024-01-24T23:27:32.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0938e1d640 10 monclient: _renew_subs 2024-01-24T23:27:32.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0938e1d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:27:32.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0938e1d640 1 -- 172.21.15.115:0/3551826473 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09341a6d40 con 0x7f093409e740 2024-01-24T23:27:32.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0938e1d640 10 monclient: _renew_subs 2024-01-24T23:27:32.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0938e1d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:27:32.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0938e1d640 1 -- 172.21.15.115:0/3551826473 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f09341c9020 con 0x7f093409e740 2024-01-24T23:27:32.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0928004aa0 con 0x7f093409e740 2024-01-24T23:27:32.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:32.080 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0928004f50 con 0x7f093409e740 2024-01-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0930d99640 4 set_mon_vals no callback set 2024-01-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient: dump: 2024-01-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.081 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0938e1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0938e1d640 1 -- 172.21.15.115:0/3551826473 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f09180065d0 con 0x7f0918002870 2024-01-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f092800f9d0 con 0x7f093409e740 2024-01-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(217..217 src has 1..217) v4 ==== 13328+0+0 (unknown 244530927 0 0) 0x7f0928045db0 con 0x7f093409e740 2024-01-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2921805577 0 0) 0x7f09180065d0 con 0x7f0918002870 2024-01-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.082 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient(hunting): my global_id is 6591 2024-01-24T23:27:32.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09341a7020 con 0x7f0918002870 2024-01-24T23:27:32.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0932d9d640 10 monclient: get_auth_request con 0x7f09140371c0 auth_method 0 2024-01-24T23:27:32.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 680520861 0 0) 0x7f09341a7020 con 0x7f0918002870 2024-01-24T23:27:32.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f09180065d0 con 0x7f0918002870 2024-01-24T23:27:32.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0924001780 con 0x7f0918002870 2024-01-24T23:27:32.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:32.083 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient: dump: 2024-01-24T23:27:32.083 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:32.083 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.083 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.083 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.083 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.083 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.083 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.084 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.084 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.084 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.084 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.084 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.084 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.084 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.084 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.084 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 603723107 0 0) 0x7f09180065d0 con 0x7f0918002870 2024-01-24T23:27:32.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 --> v1:172.21.15.132:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f091800a120 con 0x7f0918002870 2024-01-24T23:27:32.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f0924004a40 con 0x7f0918002870 2024-01-24T23:27:32.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:32.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:32.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 >> v1:172.21.15.132:6790/0 conn(0x7f0918002870 legacy=0x7f0918004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.085 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.060+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 >> v1:172.21.15.132:6790/0 conn(0x7f0918002870 legacy=0x7f0918004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:32.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0938e1d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:27:32.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0938e1d640 1 -- 172.21.15.115:0/3551826473 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0918009e10 con 0x7f09180090d0 2024-01-24T23:27:32.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1000747660 0 0) 0x7f0918009e10 con 0x7f09180090d0 2024-01-24T23:27:32.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0922ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0922ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0922ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0922ffd640 10 monclient(hunting): my global_id is 6600 2024-01-24T23:27:32.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09180065d0 con 0x7f09180090d0 2024-01-24T23:27:32.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2804653365 0 0) 0x7f09180065d0 con 0x7f09180090d0 2024-01-24T23:27:32.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0918009e10 con 0x7f09180090d0 2024-01-24T23:27:32.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0924005530 con 0x7f09180090d0 2024-01-24T23:27:32.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0922ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0922ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:32.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0922ffd640 10 monclient: dump: 2024-01-24T23:27:32.087 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:32.087 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.087 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.087 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.087 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.087 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.087 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.087 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.087 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.087 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.088 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.088 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.088 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.088 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.088 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.088 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1900238153 0 0) 0x7f0918009e10 con 0x7f09180090d0 2024-01-24T23:27:32.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 --> v1:172.21.15.132:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f091800c310 con 0x7f09180090d0 2024-01-24T23:27:32.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 1559839490) 0x7f09240077b0 con 0x7f09180090d0 2024-01-24T23:27:32.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0922ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:27:32.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0922ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:32.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 >> v1:172.21.15.132:6790/0 conn(0x7f09180090d0 legacy=0x7f0918009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.088 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0922ffd640 1 -- 172.21.15.115:0/3551826473 >> v1:172.21.15.132:6790/0 conn(0x7f09180090d0 legacy=0x7f0918009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:32.089 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:27:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:27:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 40, 2024-01-24T23:27:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:27:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:27:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:27:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:27:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:27:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:27:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:27:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:27:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:27:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:27:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 1, 2024-01-24T23:27:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:27:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:27:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:27:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:27:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:32.091 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:27:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:27:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:27:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:27:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:27:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:27:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:32.092 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:27:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:27:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:27:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:27:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:27:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:27:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:27:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:27:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:32.093 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:27:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:27:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:32.094 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:27:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:27:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:27:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:27:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:27:32.096 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:27:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:27:32.097 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:27:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:27:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.098 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:27:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:27:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:27:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.099 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.100 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:27:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:27:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:27:32.102 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:27:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:27:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:27:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:27:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:27:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.105 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.106 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:27:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:27:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 4 2024-01-24T23:27:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.107 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:27:32.107 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:27:32.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0938e1d640 1 -- 172.21.15.115:0/3551826473 >> v1:172.21.15.132:6812/1835401922 conn(0x7f09140371c0 legacy=0x7f0914039680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0938e1d640 10 monclient: shutdown 2024-01-24T23:27:32.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0938e1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:32.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0938e1d640 1 -- 172.21.15.115:0/3551826473 >> v1:172.21.15.115:6789/0 conn(0x7f093409e740 legacy=0x7f09341ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0932d9d640 1 -- 172.21.15.115:0/3551826473 reap_dead start 2024-01-24T23:27:32.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0932d9d640 1 -- 172.21.15.115:0/3551826473 reap_dead start 2024-01-24T23:27:32.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0938e1d640 1 -- 172.21.15.115:0/3551826473 shutdown_connections 2024-01-24T23:27:32.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0938e1d640 1 -- 172.21.15.115:0/3551826473 >> 172.21.15.115:0/3551826473 conn(0x7f09340666f0 msgr2=0x7f09340a6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:32.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.068+0000 7f0938e1d640 1 -- 172.21.15.115:0/3551826473 shutdown_connections 2024-01-24T23:27:32.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.072+0000 7f0938e1d640 1 -- 172.21.15.115:0/3551826473 wait complete. 2024-01-24T23:27:32.110 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:27:32.110 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-01-24T23:27:32.110 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-01-24T23:27:32.110 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~88s 2024-01-24T23:27:32.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd9a0165640 10 monclient: get_monmap_and_config 2024-01-24T23:27:32.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd9a0165640 10 monclient: build_initial_monmap 2024-01-24T23:27:32.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd9a0165640 10 monclient: monmap: 2024-01-24T23:27:32.180 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:32.180 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.180 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:32.181956+0000 2024-01-24T23:27:32.180 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:32.181956+0000 2024-01-24T23:27:32.180 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:32.180 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:32.180 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:32.180 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:32.180 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:32.180 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:32.180 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:32.181 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:32.181 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:32.181 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:32.181 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:32.181 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:32.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd9a0165640 1 Processor -- start 2024-01-24T23:27:32.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd9a0165640 1 -- start start 2024-01-24T23:27:32.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd9a0165640 10 monclient: init 2024-01-24T23:27:32.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd9a0165640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:32.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd9a0165640 10 monclient: _add_conns ranks=[1,5,3,8,4,7,0,6,2] 2024-01-24T23:27:32.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd9a0165640 10 monclient(hunting): picked mon.noname-f con 0x7fd998136cb0 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:32.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd9a0165640 10 monclient(hunting): picked mon.noname-h con 0x7fd9981377a0 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:32.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd9a0165640 10 monclient(hunting): picked mon.noname-g con 0x7fd99809bcc0 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:32.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd9a0165640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd998064700 con 0x7fd998136cb0 2024-01-24T23:27:32.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd9a0165640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd99809b650 con 0x7fd99809bcc0 2024-01-24T23:27:32.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd9a0165640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd99809c110 con 0x7fd9981377a0 2024-01-24T23:27:32.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd9a0165640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:32.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd9a0165640 10 monclient(hunting): authenticate will time out at 1339.507379s 2024-01-24T23:27:32.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd99d700640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fd9981377a0 0x7fd99813c1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:32836/0 (socket says 172.21.15.132:32836) 2024-01-24T23:27:32.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd99d700640 1 -- 172.21.15.132:0/799328850 learned_addr learned my addr 172.21.15.132:0/799328850 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:32.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd99ceff640 1 -- 172.21.15.132:0/799328850 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2234812438 0 0) 0x7fd99809c110 con 0x7fd9981377a0 2024-01-24T23:27:32.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd99ceff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd99ceff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd99ceff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd99ceff640 10 monclient(hunting): my global_id is 6602 2024-01-24T23:27:32.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd99ceff640 1 -- 172.21.15.132:0/799328850 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd984003530 con 0x7fd9981377a0 2024-01-24T23:27:32.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd99ceff640 1 -- 172.21.15.132:0/799328850 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302928407 0 0) 0x7fd99809b650 con 0x7fd99809bcc0 2024-01-24T23:27:32.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd99ceff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd99ceff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd99ceff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd99ceff640 10 monclient(hunting): my global_id is 6609 2024-01-24T23:27:32.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.178+0000 7fd99ceff640 1 -- 172.21.15.132:0/799328850 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd99809c110 con 0x7fd99809bcc0 2024-01-24T23:27:32.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 1 -- 172.21.15.132:0/799328850 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029818359 0 0) 0x7fd998064700 con 0x7fd998136cb0 2024-01-24T23:27:32.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 10 monclient(hunting): my global_id is 124156 2024-01-24T23:27:32.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 1 -- 172.21.15.132:0/799328850 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd99809b650 con 0x7fd998136cb0 2024-01-24T23:27:32.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 1 -- 172.21.15.132:0/799328850 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3469626086 0 0) 0x7fd99809c110 con 0x7fd99809bcc0 2024-01-24T23:27:32.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 1 -- 172.21.15.132:0/799328850 >> v1:172.21.15.132:6791/0 conn(0x7fd9981377a0 legacy=0x7fd99813c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 1 -- 172.21.15.132:0/799328850 >> v1:172.21.15.132:6789/0 conn(0x7fd998136cb0 legacy=0x7fd998137090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 10 monclient: _finish_hunting 0 2024-01-24T23:27:32.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 1 monclient: found mon.noname-g 2024-01-24T23:27:32.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:32.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:27:32.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 1 -- 172.21.15.132:0/799328850 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99809b000 con 0x7fd99809bcc0 2024-01-24T23:27:32.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 10 monclient: _finish_auth 0 2024-01-24T23:27:32.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 10 monclient: _check_auth_tickets 2024-01-24T23:27:32.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:32.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 1 -- 172.21.15.132:0/799328850 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd9900017a0 con 0x7fd99809bcc0 2024-01-24T23:27:32.190 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:27:32.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 10 monclient: dump: 2024-01-24T23:27:32.191 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:32.191 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.191 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.191 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.191 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.191 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:32.191 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.191 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.191 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.191 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.191 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.191 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.191 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.192 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.192 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.192 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:32.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 5 monclient: authenticate success, global_id 6609 2024-01-24T23:27:32.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:32.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 1 -- 172.21.15.132:0/799328850 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd990001ac0 con 0x7fd99809bcc0 2024-01-24T23:27:32.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:32.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 1 -- 172.21.15.132:0/799328850 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd990003960 con 0x7fd99809bcc0 2024-01-24T23:27:32.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:32.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 4 set_mon_vals no callback set 2024-01-24T23:27:32.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:32.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99ceff640 10 monclient: dump: 2024-01-24T23:27:32.193 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:32.193 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.193 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.193 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.193 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.193 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:32.193 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.193 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.193 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.193 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.194 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.194 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.194 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.194 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.194 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.194 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:32.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 10 monclient: shutdown 2024-01-24T23:27:32.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:32.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 1 -- 172.21.15.132:0/799328850 >> v1:172.21.15.132:6790/0 conn(0x7fd99809bcc0 legacy=0x7fd99813f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 1 -- 172.21.15.132:0/799328850 shutdown_connections 2024-01-24T23:27:32.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 1 -- 172.21.15.132:0/799328850 >> 172.21.15.132:0/799328850 conn(0x7fd9980a9c10 msgr2=0x7fd9980a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:32.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 1 -- 172.21.15.132:0/799328850 shutdown_connections 2024-01-24T23:27:32.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 1 -- 172.21.15.132:0/799328850 wait complete. 2024-01-24T23:27:32.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 10 monclient: build_initial_monmap 2024-01-24T23:27:32.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 10 monclient: monmap: 2024-01-24T23:27:32.194 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:32.195 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:32.195 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:32.195 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:32.195 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:32.195 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:32.195 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:32.195 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:32.195 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:32.195 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:32.195 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:32.195 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:32.195 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:32.195 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:32.195 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:32.195 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:32.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 1 Processor -- start 2024-01-24T23:27:32.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 1 -- start start 2024-01-24T23:27:32.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 10 monclient: init 2024-01-24T23:27:32.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:32.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 10 monclient: _add_conns ranks=[3,1,4,8,7,6,2,0,5] 2024-01-24T23:27:32.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 10 monclient(hunting): picked mon.noname-g con 0x7fd99809bcc0 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:32.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 10 monclient(hunting): picked mon.noname-f con 0x7fd998136cb0 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:32.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 10 monclient(hunting): picked mon.noname-c con 0x7fd9981377a0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:32.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd99809c6a0 con 0x7fd998136cb0 2024-01-24T23:27:32.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd998137fb0 con 0x7fd99809bcc0 2024-01-24T23:27:32.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd99809f440 con 0x7fd9981377a0 2024-01-24T23:27:32.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:32.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 10 monclient(hunting): authenticate will time out at 1339.510609s 2024-01-24T23:27:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99d700640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fd998136cb0 0x7fd9981d0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:56354/0 (socket says 172.21.15.132:56354) 2024-01-24T23:27:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99d700640 1 -- 172.21.15.132:0/3673063451 learned_addr learned my addr 172.21.15.132:0/3673063451 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd99df01640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fd99809bcc0 0x7fd9981309e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:46596/0 (socket says 172.21.15.132:46596) 2024-01-24T23:27:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 1 -- 172.21.15.132:0/3673063451 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1731192709 0 0) 0x7fd998137fb0 con 0x7fd99809bcc0 2024-01-24T23:27:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient(hunting): my global_id is 6618 2024-01-24T23:27:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 1 -- 172.21.15.132:0/3673063451 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd97c003530 con 0x7fd99809bcc0 2024-01-24T23:27:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 1 -- 172.21.15.132:0/3673063451 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1124199143 0 0) 0x7fd99809c6a0 con 0x7fd998136cb0 2024-01-24T23:27:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient(hunting): my global_id is 124165 2024-01-24T23:27:32.197 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 1 -- 172.21.15.132:0/3673063451 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd998137fb0 con 0x7fd998136cb0 2024-01-24T23:27:32.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 1 -- 172.21.15.132:0/3673063451 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1116377817 0 0) 0x7fd998137fb0 con 0x7fd998136cb0 2024-01-24T23:27:32.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 1 -- 172.21.15.132:0/3673063451 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd99809c6a0 con 0x7fd998136cb0 2024-01-24T23:27:32.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 1 -- 172.21.15.132:0/3673063451 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1308286091 0 0) 0x7fd97c003530 con 0x7fd99809bcc0 2024-01-24T23:27:32.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 1 -- 172.21.15.132:0/3673063451 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd998137fb0 con 0x7fd99809bcc0 2024-01-24T23:27:32.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 1 -- 172.21.15.132:0/3673063451 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd9880019c0 con 0x7fd998136cb0 2024-01-24T23:27:32.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:27:32.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient(hunting): dump: 2024-01-24T23:27:32.198 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:32.198 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.198 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.198 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.198 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.198 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:32.198 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.199 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.199 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.199 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.199 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.199 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.199 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.199 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.199 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.199 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:32.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:32.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 1 -- 172.21.15.132:0/3673063451 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1052462017 0 0) 0x7fd99809f440 con 0x7fd9981377a0 2024-01-24T23:27:32.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient(hunting): my global_id is 94765 2024-01-24T23:27:32.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 1 -- 172.21.15.132:0/3673063451 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd97c003530 con 0x7fd9981377a0 2024-01-24T23:27:32.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 1 -- 172.21.15.132:0/3673063451 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd9940018a0 con 0x7fd99809bcc0 2024-01-24T23:27:32.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:32.200 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient(hunting): dump: 2024-01-24T23:27:32.200 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:32.200 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.200 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.200 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.200 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.200 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:32.200 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.201 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.201 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.201 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.201 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.201 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.201 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.201 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.201 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.201 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:32.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 1 -- 172.21.15.132:0/3673063451 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1734018803 0 0) 0x7fd99809c6a0 con 0x7fd998136cb0 2024-01-24T23:27:32.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 1 -- 172.21.15.132:0/3673063451 >> v1:172.21.15.115:6791/0 conn(0x7fd9981377a0 legacy=0x7fd9981d3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 1 -- 172.21.15.132:0/3673063451 >> v1:172.21.15.132:6790/0 conn(0x7fd99809bcc0 legacy=0x7fd9981309e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:27:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 1 monclient: found mon.f 2024-01-24T23:27:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:27:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 1 -- 172.21.15.132:0/3673063451 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99809f6f0 con 0x7fd998136cb0 2024-01-24T23:27:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient: _finish_auth 0 2024-01-24T23:27:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:27:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 5 monclient: authenticate success, global_id 124165 2024-01-24T23:27:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 10 monclient: _renew_subs 2024-01-24T23:27:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:27:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 1 -- 172.21.15.132:0/3673063451 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd99809f940 con 0x7fd998136cb0 2024-01-24T23:27:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 10 monclient: _renew_subs 2024-01-24T23:27:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:27:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd9a0165640 1 -- 172.21.15.132:0/3673063451 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9981cb760 con 0x7fd998136cb0 2024-01-24T23:27:32.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 1 -- 172.21.15.132:0/3673063451 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd988001d30 con 0x7fd998136cb0 2024-01-24T23:27:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 1 -- 172.21.15.132:0/3673063451 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd988004de0 con 0x7fd998136cb0 2024-01-24T23:27:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98ffff640 4 set_mon_vals no callback set 2024-01-24T23:27:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:27:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.182+0000 7fd98e7fc640 10 monclient: dump: 2024-01-24T23:27:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.203 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.204 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.204 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.204 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.204 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.204 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.204 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.204 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:32.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.186+0000 7fd9a0165640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:32.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.186+0000 7fd9a0165640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:32.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.186+0000 7fd9a0165640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:27:32.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.186+0000 7fd98e7fc640 1 -- 172.21.15.132:0/3673063451 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fd98800f860 con 0x7fd998136cb0 2024-01-24T23:27:32.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.186+0000 7fd9a0165640 1 -- 172.21.15.132:0/3673063451 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd980005090 con 0x7fd998136cb0 2024-01-24T23:27:32.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.186+0000 7fd98e7fc640 1 -- 172.21.15.132:0/3673063451 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(217..217 src has 1..217) v4 ==== 13328+0+0 (unknown 244530927 0 0) 0x7fd988045d70 con 0x7fd998136cb0 2024-01-24T23:27:32.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.186+0000 7fd99df01640 10 monclient: get_auth_request con 0x7fd97c037820 auth_method 0 2024-01-24T23:27:32.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.190+0000 7fd98e7fc640 1 -- 172.21.15.132:0/3673063451 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fd988010e50 con 0x7fd998136cb0 2024-01-24T23:27:32.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.190+0000 7fd98e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:32.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.190+0000 7fd98e7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:32.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.216+0000 7f499d00f640 10 monclient: get_monmap_and_config 2024-01-24T23:27:32.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.216+0000 7f499d00f640 10 monclient: build_initial_monmap 2024-01-24T23:27:32.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.216+0000 7f499d00f640 10 monclient: monmap: 2024-01-24T23:27:32.220 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:32.220 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.220 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:32.221566+0000 2024-01-24T23:27:32.220 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:32.221566+0000 2024-01-24T23:27:32.220 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:32.220 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.220 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:32.220 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:32.220 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:32.221 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:32.221 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:32.221 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:32.221 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:32.221 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:32.221 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:32.221 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.216+0000 7f499d00f640 1 Processor -- start 2024-01-24T23:27:32.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.216+0000 7f499d00f640 1 -- start start 2024-01-24T23:27:32.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.216+0000 7f499d00f640 10 monclient: init 2024-01-24T23:27:32.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.216+0000 7f499d00f640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:32.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 10 monclient: _add_conns ranks=[1,6,7,4,0,5,2,8,3] 2024-01-24T23:27:32.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 10 monclient(hunting): picked mon.noname-f con 0x7f49980a0a10 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:32.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 10 monclient(hunting): picked mon.noname-d con 0x7f499809f1e0 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:32.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 10 monclient(hunting): picked mon.noname-i con 0x7f499813d360 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:32.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499804ed70 con 0x7f49980a0a10 2024-01-24T23:27:32.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49980a1500 con 0x7f499809f1e0 2024-01-24T23:27:32.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499806aaf0 con 0x7f499813d360 2024-01-24T23:27:32.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:32.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 10 monclient(hunting): authenticate will time out at 1331.144138s 2024-01-24T23:27:32.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499659c640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f499809f1e0 0x7f499813bc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:54716/0 (socket says 172.21.15.115:54716) 2024-01-24T23:27:32.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499659c640 1 -- 172.21.15.115:0/1314104926 learned_addr learned my addr 172.21.15.115:0/1314104926 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:32.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 1 -- 172.21.15.115:0/1314104926 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3322791587 0 0) 0x7f49980a1500 con 0x7f499809f1e0 2024-01-24T23:27:32.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.230 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 10 monclient(hunting): my global_id is 6765 2024-01-24T23:27:32.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 1 -- 172.21.15.115:0/1314104926 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4984003530 con 0x7f499809f1e0 2024-01-24T23:27:32.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 1 -- 172.21.15.115:0/1314104926 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3567741427 0 0) 0x7f499804ed70 con 0x7f49980a0a10 2024-01-24T23:27:32.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 10 monclient(hunting): my global_id is 124174 2024-01-24T23:27:32.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 1 -- 172.21.15.115:0/1314104926 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49980a1500 con 0x7f49980a0a10 2024-01-24T23:27:32.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 1 -- 172.21.15.115:0/1314104926 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 452871297 0 0) 0x7f499806aaf0 con 0x7f499813d360 2024-01-24T23:27:32.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 10 monclient(hunting): my global_id is 124171 2024-01-24T23:27:32.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 1 -- 172.21.15.115:0/1314104926 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f499804ed70 con 0x7f499813d360 2024-01-24T23:27:32.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 1 -- 172.21.15.115:0/1314104926 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 309064704 0 0) 0x7f4984003530 con 0x7f499809f1e0 2024-01-24T23:27:32.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 1 -- 172.21.15.115:0/1314104926 >> v1:172.21.15.132:6792/0 conn(0x7f499813d360 legacy=0x7f499813f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 1 -- 172.21.15.115:0/1314104926 >> v1:172.21.15.132:6789/0 conn(0x7f49980a0a10 legacy=0x7f49980a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 10 monclient: _finish_hunting 0 2024-01-24T23:27:32.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 1 monclient: found mon.noname-d 2024-01-24T23:27:32.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:32.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:27:32.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 1 -- 172.21.15.115:0/1314104926 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499809e560 con 0x7f499809f1e0 2024-01-24T23:27:32.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 10 monclient: _finish_auth 0 2024-01-24T23:27:32.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 10 monclient: _check_auth_tickets 2024-01-24T23:27:32.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:32.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 5 monclient: authenticate success, global_id 6765 2024-01-24T23:27:32.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 1 -- 172.21.15.115:0/1314104926 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f49880017c0 con 0x7f499809f1e0 2024-01-24T23:27:32.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:32.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:27:32.233 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 10 monclient: dump: 2024-01-24T23:27:32.233 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:32.233 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.233 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.233 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.233 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.233 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.233 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.233 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.233 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.234 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.234 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.234 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.234 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.234 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.234 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.234 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:32.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 1 -- 172.21.15.115:0/1314104926 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4988001aa0 con 0x7f499809f1e0 2024-01-24T23:27:32.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:32.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:32.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 1 -- 172.21.15.115:0/1314104926 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4988004bf0 con 0x7f499809f1e0 2024-01-24T23:27:32.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 4 set_mon_vals no callback set 2024-01-24T23:27:32.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:32.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f4995d9b640 10 monclient: dump: 2024-01-24T23:27:32.235 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:32.235 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.235 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.235 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.235 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.235 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.235 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.235 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.235 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.235 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.235 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.235 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.235 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.235 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.236 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.236 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 10 monclient: shutdown 2024-01-24T23:27:32.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:32.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 1 -- 172.21.15.115:0/1314104926 >> v1:172.21.15.115:6792/0 conn(0x7f499809f1e0 legacy=0x7f499813bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 1 -- 172.21.15.115:0/1314104926 shutdown_connections 2024-01-24T23:27:32.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 1 -- 172.21.15.115:0/1314104926 >> 172.21.15.115:0/1314104926 conn(0x7f49980a7660 msgr2=0x7f49980a3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:32.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 1 -- 172.21.15.115:0/1314104926 shutdown_connections 2024-01-24T23:27:32.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 1 -- 172.21.15.115:0/1314104926 wait complete. 2024-01-24T23:27:32.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 10 monclient: build_initial_monmap 2024-01-24T23:27:32.236 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 10 monclient: monmap: 2024-01-24T23:27:32.236 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:32.237 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:32.237 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:32.237 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:32.237 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:32.237 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:32.237 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:32.237 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:32.237 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:32.237 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:32.237 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:32.237 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:32.237 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:32.237 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:32.237 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:32.237 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 1 Processor -- start 2024-01-24T23:27:32.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 1 -- start start 2024-01-24T23:27:32.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 10 monclient: init 2024-01-24T23:27:32.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:32.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 10 monclient: _add_conns ranks=[5,1,4,8,2,6,7,0,3] 2024-01-24T23:27:32.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 10 monclient(hunting): picked mon.noname-h con 0x7f499809f1e0 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:32.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 10 monclient(hunting): picked mon.noname-f con 0x7f49980a0a10 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:32.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 10 monclient(hunting): picked mon.noname-c con 0x7f499813d360 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:32.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4998071eb0 con 0x7f49980a0a10 2024-01-24T23:27:32.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49981cbcd0 con 0x7f499813d360 2024-01-24T23:27:32.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4998139b40 con 0x7f499809f1e0 2024-01-24T23:27:32.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:32.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499d00f640 10 monclient(hunting): authenticate will time out at 1331.147317s 2024-01-24T23:27:32.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499759e640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f499813d360 0x7f49981d3900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:57588/0 (socket says 172.21.15.115:57588) 2024-01-24T23:27:32.238 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f499759e640 1 -- 172.21.15.115:0/1496048260 learned_addr learned my addr 172.21.15.115:0/1496048260 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:32.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1986041594 0 0) 0x7f49981cbcd0 con 0x7f499813d360 2024-01-24T23:27:32.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f498f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f498f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f498f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f498f7fe640 10 monclient(hunting): my global_id is 94774 2024-01-24T23:27:32.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4978003530 con 0x7f499813d360 2024-01-24T23:27:32.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3191481171 0 0) 0x7f4978003530 con 0x7f499813d360 2024-01-24T23:27:32.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f49981cbcd0 con 0x7f499813d360 2024-01-24T23:27:32.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4990001cc0 con 0x7f499813d360 2024-01-24T23:27:32.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.220+0000 7f498f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:27:32.239 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient(hunting): dump: 2024-01-24T23:27:32.239 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:32.239 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.239 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.240 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:32.240 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888719155 0 0) 0x7f4998139b40 con 0x7f499809f1e0 2024-01-24T23:27:32.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient(hunting): my global_id is 6611 2024-01-24T23:27:32.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4978003530 con 0x7f499809f1e0 2024-01-24T23:27:32.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1209879499 0 0) 0x7f4998071eb0 con 0x7f49980a0a10 2024-01-24T23:27:32.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient(hunting): my global_id is 124183 2024-01-24T23:27:32.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4998139b40 con 0x7f49980a0a10 2024-01-24T23:27:32.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 281132562 0 0) 0x7f49981cbcd0 con 0x7f499813d360 2024-01-24T23:27:32.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 >> v1:172.21.15.132:6791/0 conn(0x7f499809f1e0 legacy=0x7f49981cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.241 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 >> v1:172.21.15.132:6789/0 conn(0x7f49980a0a10 legacy=0x7f499813c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:27:32.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 1 monclient: found mon.c 2024-01-24T23:27:32.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:32.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:32.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4998072400 con 0x7f499813d360 2024-01-24T23:27:32.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient: _finish_auth 0 2024-01-24T23:27:32.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:32.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:32.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f499d00f640 5 monclient: authenticate success, global_id 94774 2024-01-24T23:27:32.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f499d00f640 10 monclient: _renew_subs 2024-01-24T23:27:32.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f499d00f640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:32.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f499d00f640 1 -- 172.21.15.115:0/1496048260 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499809d890 con 0x7f499813d360 2024-01-24T23:27:32.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f499d00f640 10 monclient: _renew_subs 2024-01-24T23:27:32.242 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f499d00f640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:32.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f499d00f640 1 -- 172.21.15.115:0/1496048260 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f499813a810 con 0x7f499813d360 2024-01-24T23:27:32.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4990002090 con 0x7f499813d360 2024-01-24T23:27:32.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:32.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4990004010 con 0x7f499813d360 2024-01-24T23:27:32.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f499559a640 4 set_mon_vals no callback set 2024-01-24T23:27:32.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:32.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient: dump: 2024-01-24T23:27:32.243 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:32.243 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.243 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.243 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.243 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.243 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.244 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.244 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.244 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.244 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.244 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.244 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.244 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.244 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.244 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.244 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f499d00f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:32.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f499d00f640 1 -- 172.21.15.115:0/1496048260 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f497c0065d0 con 0x7f497c002870 2024-01-24T23:27:32.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f49900041f0 con 0x7f499813d360 2024-01-24T23:27:32.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(217..217 src has 1..217) v4 ==== 13328+0+0 (unknown 244530927 0 0) 0x7f49900460b0 con 0x7f499813d360 2024-01-24T23:27:32.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1550366917 0 0) 0x7f497c0065d0 con 0x7f497c002870 2024-01-24T23:27:32.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient(hunting): my global_id is 6620 2024-01-24T23:27:32.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49981cbcd0 con 0x7f497c002870 2024-01-24T23:27:32.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f499659c640 10 monclient: get_auth_request con 0x7f49780371e0 auth_method 0 2024-01-24T23:27:32.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4191294510 0 0) 0x7f49981cbcd0 con 0x7f497c002870 2024-01-24T23:27:32.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f497c0065d0 con 0x7f497c002870 2024-01-24T23:27:32.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4980001970 con 0x7f497c002870 2024-01-24T23:27:32.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:32.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient: dump: 2024-01-24T23:27:32.245 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:32.246 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.246 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.246 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.246 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.246 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.246 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.246 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.246 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.246 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.246 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.246 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.246 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.246 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.247 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.247 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 646222077 0 0) 0x7f497c0065d0 con 0x7f497c002870 2024-01-24T23:27:32.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 --> v1:172.21.15.132:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f497c00a120 con 0x7f497c002870 2024-01-24T23:27:32.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f4980003660 con 0x7f497c002870 2024-01-24T23:27:32.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:32.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:32.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 >> v1:172.21.15.132:6791/0 conn(0x7f497c002870 legacy=0x7f497c004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.224+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 >> v1:172.21.15.132:6791/0 conn(0x7f497c002870 legacy=0x7f497c004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:32.248 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f499d00f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:27:32.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f499d00f640 1 -- 172.21.15.115:0/1496048260 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f497c009e10 con 0x7f497c0090d0 2024-01-24T23:27:32.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1890676773 0 0) 0x7f497c009e10 con 0x7f497c0090d0 2024-01-24T23:27:32.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f498f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f498f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f498f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f498f7fe640 10 monclient(hunting): my global_id is 6629 2024-01-24T23:27:32.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f497c0065d0 con 0x7f497c0090d0 2024-01-24T23:27:32.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3096900947 0 0) 0x7f497c0065d0 con 0x7f497c0090d0 2024-01-24T23:27:32.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f497c009e10 con 0x7f497c0090d0 2024-01-24T23:27:32.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4980003d00 con 0x7f497c0090d0 2024-01-24T23:27:32.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f498f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f498f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:32.249 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f498f7fe640 10 monclient: dump: 2024-01-24T23:27:32.250 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:32.250 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.250 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.250 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.250 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.250 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.250 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.250 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.250 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.250 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.250 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.250 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.250 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.250 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.250 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.250 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3475503183 0 0) 0x7f497c009e10 con 0x7f497c0090d0 2024-01-24T23:27:32.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 --> v1:172.21.15.132:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f497c00c310 con 0x7f497c0090d0 2024-01-24T23:27:32.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7545 (unknown 0 0 481404289) 0x7f49800060a0 con 0x7f497c0090d0 2024-01-24T23:27:32.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f498f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:27:32.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f498f7fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:32.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 >> v1:172.21.15.132:6791/0 conn(0x7f497c0090d0 legacy=0x7f497c009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.251 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f498f7fe640 1 -- 172.21.15.115:0/1496048260 >> v1:172.21.15.132:6791/0 conn(0x7f497c0090d0 legacy=0x7f497c009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:32.251 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:27:32.252 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:32.252 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:32.252 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "leader", 2024-01-24T23:27:32.252 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 40, 2024-01-24T23:27:32.252 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:27:32.252 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:27:32.252 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:27:32.252 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:27:32.252 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:27:32.252 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:27:32.252 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:27:32.252 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:27:32.253 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:27:32.253 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:27:32.253 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:32.253 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 1, 2024-01-24T23:27:32.253 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:32.253 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:27:32.253 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:27:32.253 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:32.253 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:32.253 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:32.253 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:32.253 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:32.253 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:32.253 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:32.253 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:32.254 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:32.254 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:32.254 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:32.254 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:32.254 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:27:32.254 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:27:32.254 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:32.254 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:32.254 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:32.254 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:32.254 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:32.254 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:32.254 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:32.254 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:32.254 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:32.255 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:32.255 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:32.255 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.255 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:27:32.255 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:27:32.255 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:27:32.255 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:27:32.255 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:27:32.255 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:27:32.255 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:32.255 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:32.255 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:27:32.255 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:27:32.255 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:27:32.256 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:27:32.256 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:27:32.256 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:27:32.256 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:27:32.256 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:32.256 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:27:32.256 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:32.256 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:32.256 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:32.256 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:32.256 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:32.256 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:32.256 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:32.257 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:32.257 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:32.257 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:32.257 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:32.257 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:32.257 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:27:32.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.257 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:27:32.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.257 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:32.257 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:32.257 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.257 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.258 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.258 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:27:32.258 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.258 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.258 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.258 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:27:32.258 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:27:32.258 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.258 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.258 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.258 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:32.259 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:32.259 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.259 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.259 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.259 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:27:32.259 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.259 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.259 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.259 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:27:32.259 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:27:32.259 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.259 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.259 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.260 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:32.260 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:32.260 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.260 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.260 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.260 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:27:32.260 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.260 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.260 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.260 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:27:32.260 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:27:32.261 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.261 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.261 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.261 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:32.261 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:32.261 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.261 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.261 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.261 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:27:32.261 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.262 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.262 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.262 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:27:32.262 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:27:32.262 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.262 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.262 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.262 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:32.262 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:32.262 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.262 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.263 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.263 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:27:32.263 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.263 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.263 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.263 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:27:32.263 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:27:32.263 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.263 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.263 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.264 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:32.264 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:32.264 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.264 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.264 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.264 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:27:32.264 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.265 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.265 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.265 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:27:32.265 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:27:32.265 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.265 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.265 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.265 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:32.265 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:32.265 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.266 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.266 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.266 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:27:32.266 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.266 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.266 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.266 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:27:32.266 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:27:32.266 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.266 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.266 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.267 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:32.267 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:32.267 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.267 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.267 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.267 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:27:32.267 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.267 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.267 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.267 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:27:32.267 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:27:32.268 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.268 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.268 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.268 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:32.268 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:32.268 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.268 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.269 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.269 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:27:32.269 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.269 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.269 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.269 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:27:32.269 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:27:32.269 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.269 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.269 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.269 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.269 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.270 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:27:32.270 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:27:32.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.270 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:32.270 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:32.270 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:32.270 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.270 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:32.270 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:27:32.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.270 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:32.270 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:32.270 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:32.271 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.271 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.271 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:27:32.271 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:27:32.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f499d00f640 1 -- 172.21.15.115:0/1496048260 >> v1:172.21.15.132:6812/1835401922 conn(0x7f49780371e0 legacy=0x7f49780396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f499d00f640 10 monclient: shutdown 2024-01-24T23:27:32.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f499d00f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:32.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f499d00f640 1 -- 172.21.15.115:0/1496048260 >> v1:172.21.15.115:6791/0 conn(0x7f499813d360 legacy=0x7f49981d3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f499759e640 1 -- 172.21.15.115:0/1496048260 reap_dead start 2024-01-24T23:27:32.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f499759e640 1 -- 172.21.15.115:0/1496048260 reap_dead start 2024-01-24T23:27:32.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f499d00f640 1 -- 172.21.15.115:0/1496048260 shutdown_connections 2024-01-24T23:27:32.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f499d00f640 1 -- 172.21.15.115:0/1496048260 >> 172.21.15.115:0/1496048260 conn(0x7f49980a7660 msgr2=0x7f499806b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:32.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f499d00f640 1 -- 172.21.15.115:0/1496048260 shutdown_connections 2024-01-24T23:27:32.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.232+0000 7f499d00f640 1 -- 172.21.15.115:0/1496048260 wait complete. 2024-01-24T23:27:32.273 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:27:32.273 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-01-24T23:27:32.273 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-01-24T23:27:32.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.350+0000 7fd9a0165640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h6"}] 2024-01-24T23:27:32.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.350+0000 7fd9a0165640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h6"}] 2024-01-24T23:27:32.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.350+0000 7fd9a0165640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:27:32.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:32.350+0000 7fd9a0165640 1 -- 172.21.15.132:0/3673063451 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd crush rm", "name": "h6"} v 0) v1 -- 0x7fd980005340 con 0x7fd998136cb0 2024-01-24T23:27:32.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.384+0000 7f07749cc640 10 monclient: get_monmap_and_config 2024-01-24T23:27:32.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.384+0000 7f07749cc640 10 monclient: build_initial_monmap 2024-01-24T23:27:32.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.384+0000 7f07749cc640 10 monclient: monmap: 2024-01-24T23:27:32.391 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:32.391 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.391 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:32.389312+0000 2024-01-24T23:27:32.391 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:32.389312+0000 2024-01-24T23:27:32.392 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:32.392 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.392 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:32.392 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:32.392 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:32.392 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:32.392 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:32.392 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:32.392 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:32.392 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:32.392 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:32.392 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.384+0000 7f07749cc640 1 Processor -- start 2024-01-24T23:27:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.384+0000 7f07749cc640 1 -- start start 2024-01-24T23:27:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.384+0000 7f07749cc640 10 monclient: init 2024-01-24T23:27:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.384+0000 7f07749cc640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.384+0000 7f07749cc640 10 monclient: _add_conns ranks=[4,0,5,3,6,7,8,1,2] 2024-01-24T23:27:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.384+0000 7f07749cc640 10 monclient(hunting): picked mon.noname-c con 0x7f077009ff10 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.384+0000 7f07749cc640 10 monclient(hunting): picked mon.noname-a con 0x7f077006ae80 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.384+0000 7f07749cc640 10 monclient(hunting): picked mon.noname-h con 0x7f07700a02f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.384+0000 7f07749cc640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077004ed70 con 0x7f077006ae80 2024-01-24T23:27:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.384+0000 7f07749cc640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07700a3890 con 0x7f077009ff10 2024-01-24T23:27:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.384+0000 7f07749cc640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077006ac40 con 0x7f07700a02f0 2024-01-24T23:27:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.384+0000 7f07749cc640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.384+0000 7f07749cc640 10 monclient(hunting): authenticate will time out at 1331.311897s 2024-01-24T23:27:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076dd9b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f077006ae80 0x7f07700a3180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:41180/0 (socket says 172.21.15.115:41180) 2024-01-24T23:27:32.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076dd9b640 1 -- 172.21.15.115:0/1597823186 learned_addr learned my addr 172.21.15.115:0/1597823186 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076e59c640 1 --1- 172.21.15.115:0/1597823186 >> v1:172.21.15.115:6791/0 conn(0x7f077009ff10 0x7f07700a2a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:57600/0 (socket says 172.21.15.115:57600) 2024-01-24T23:27:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 1 -- 172.21.15.115:0/1597823186 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935558828 0 0) 0x7f077004ed70 con 0x7f077006ae80 2024-01-24T23:27:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 10 monclient(hunting): my global_id is 94932 2024-01-24T23:27:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 1 -- 172.21.15.115:0/1597823186 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f075c003530 con 0x7f077006ae80 2024-01-24T23:27:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 1 -- 172.21.15.115:0/1597823186 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1198540110 0 0) 0x7f07700a3890 con 0x7f077009ff10 2024-01-24T23:27:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 10 monclient(hunting): my global_id is 94783 2024-01-24T23:27:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 1 -- 172.21.15.115:0/1597823186 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f077004ed70 con 0x7f077009ff10 2024-01-24T23:27:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 1 -- 172.21.15.115:0/1597823186 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3419259080 0 0) 0x7f075c003530 con 0x7f077006ae80 2024-01-24T23:27:32.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 1 -- 172.21.15.115:0/1597823186 >> v1:172.21.15.132:6791/0 conn(0x7f07700a02f0 legacy=0x7f077013fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 1 -- 172.21.15.115:0/1597823186 >> v1:172.21.15.115:6791/0 conn(0x7f077009ff10 legacy=0x7f07700a2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 10 monclient: _finish_hunting 0 2024-01-24T23:27:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 1 monclient: found mon.noname-a 2024-01-24T23:27:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:27:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 1 -- 172.21.15.115:0/1597823186 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077009e9e0 con 0x7f077006ae80 2024-01-24T23:27:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 10 monclient: _finish_auth 0 2024-01-24T23:27:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 10 monclient: _check_auth_tickets 2024-01-24T23:27:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 1 -- 172.21.15.115:0/1597823186 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f07580017c0 con 0x7f077006ae80 2024-01-24T23:27:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 5 monclient: authenticate success, global_id 94932 2024-01-24T23:27:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:27:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 10 monclient: dump: 2024-01-24T23:27:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.396 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.397 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.397 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.397 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.397 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:32.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 1 -- 172.21.15.115:0/1597823186 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0758001a60 con 0x7f077006ae80 2024-01-24T23:27:32.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:32.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:32.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 1 -- 172.21.15.115:0/1597823186 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0758004bf0 con 0x7f077006ae80 2024-01-24T23:27:32.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 4 set_mon_vals no callback set 2024-01-24T23:27:32.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:27:32.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076d59a640 10 monclient: dump: 2024-01-24T23:27:32.397 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:32.398 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.398 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.398 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.398 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.398 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.398 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.398 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.398 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.398 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.398 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.398 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.398 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.398 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.398 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.398 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 10 monclient: shutdown 2024-01-24T23:27:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 1 -- 172.21.15.115:0/1597823186 >> v1:172.21.15.115:6789/0 conn(0x7f077006ae80 legacy=0x7f07700a3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 1 -- 172.21.15.115:0/1597823186 shutdown_connections 2024-01-24T23:27:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 1 -- 172.21.15.115:0/1597823186 >> 172.21.15.115:0/1597823186 conn(0x7f07700a7820 msgr2=0x7f07700a3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 1 -- 172.21.15.115:0/1597823186 shutdown_connections 2024-01-24T23:27:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 1 -- 172.21.15.115:0/1597823186 wait complete. 2024-01-24T23:27:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 10 monclient: build_initial_monmap 2024-01-24T23:27:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 10 monclient: monmap: 2024-01-24T23:27:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:32.400 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 1 Processor -- start 2024-01-24T23:27:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 1 -- start start 2024-01-24T23:27:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 10 monclient: init 2024-01-24T23:27:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 10 monclient: _add_conns ranks=[3,0,4,2,7,1,5,6,8] 2024-01-24T23:27:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 10 monclient(hunting): picked mon.noname-g con 0x7f077006ae80 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 10 monclient(hunting): picked mon.noname-a con 0x7f077009ff10 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 10 monclient(hunting): picked mon.noname-c con 0x7f07700a02f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07701ab080 con 0x7f077009ff10 2024-01-24T23:27:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07701ab5b0 con 0x7f077006ae80 2024-01-24T23:27:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07700720e0 con 0x7f07700a02f0 2024-01-24T23:27:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f07749cc640 10 monclient(hunting): authenticate will time out at 1331.315116s 2024-01-24T23:27:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076dd9b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f077009ff10 0x7f07701d04e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:41192/0 (socket says 172.21.15.115:41192) 2024-01-24T23:27:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f076dd9b640 1 -- 172.21.15.115:0/2054209760 learned_addr learned my addr 172.21.15.115:0/2054209760 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1561194740 0 0) 0x7f07701ab080 con 0x7f077009ff10 2024-01-24T23:27:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 10 monclient(hunting): my global_id is 94941 2024-01-24T23:27:32.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0750003530 con 0x7f077009ff10 2024-01-24T23:27:32.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 499948086 0 0) 0x7f07700720e0 con 0x7f07700a02f0 2024-01-24T23:27:32.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 10 monclient(hunting): my global_id is 94792 2024-01-24T23:27:32.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07701ab080 con 0x7f07700a02f0 2024-01-24T23:27:32.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 662932882 0 0) 0x7f0750003530 con 0x7f077009ff10 2024-01-24T23:27:32.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f07700720e0 con 0x7f077009ff10 2024-01-24T23:27:32.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0758004b30 con 0x7f077009ff10 2024-01-24T23:27:32.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:27:32.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 10 monclient(hunting): dump: 2024-01-24T23:27:32.408 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.409 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:32.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2762679232 0 0) 0x7f07701ab5b0 con 0x7f077006ae80 2024-01-24T23:27:32.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 10 monclient(hunting): my global_id is 6627 2024-01-24T23:27:32.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0750003530 con 0x7f077006ae80 2024-01-24T23:27:32.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1027111315 0 0) 0x7f07701ab080 con 0x7f07700a02f0 2024-01-24T23:27:32.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f07701ab5b0 con 0x7f07700a02f0 2024-01-24T23:27:32.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0760001880 con 0x7f07700a02f0 2024-01-24T23:27:32.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:32.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 10 monclient(hunting): dump: 2024-01-24T23:27:32.410 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:32.410 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.411 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.411 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.411 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.411 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.411 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.411 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.411 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.411 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.411 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.411 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.411 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.411 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.411 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.411 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 381912727 0 0) 0x7f07700720e0 con 0x7f077009ff10 2024-01-24T23:27:32.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 >> v1:172.21.15.115:6791/0 conn(0x7f07700a02f0 legacy=0x7f07701d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 >> v1:172.21.15.132:6790/0 conn(0x7f077006ae80 legacy=0x7f07701bdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:27:32.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 1 monclient: found mon.a 2024-01-24T23:27:32.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.388+0000 7f0766ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:32.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:27:32.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07701a92c0 con 0x7f077009ff10 2024-01-24T23:27:32.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 10 monclient: _finish_auth 0 2024-01-24T23:27:32.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:27:32.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:32.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f07749cc640 5 monclient: authenticate success, global_id 94941 2024-01-24T23:27:32.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f07749cc640 10 monclient: _renew_subs 2024-01-24T23:27:32.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f07749cc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:27:32.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f07749cc640 1 -- 172.21.15.115:0/2054209760 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07701aad10 con 0x7f077009ff10 2024-01-24T23:27:32.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f07749cc640 10 monclient: _renew_subs 2024-01-24T23:27:32.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f07749cc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:27:32.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f07749cc640 1 -- 172.21.15.115:0/2054209760 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f07701a9e20 con 0x7f077009ff10 2024-01-24T23:27:32.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0758001180 con 0x7f077009ff10 2024-01-24T23:27:32.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:32.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0758005240 con 0x7f077009ff10 2024-01-24T23:27:32.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f076cd99640 4 set_mon_vals no callback set 2024-01-24T23:27:32.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:27:32.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 10 monclient: dump: 2024-01-24T23:27:32.413 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:32.413 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.414 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.414 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.414 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.414 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.414 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.414 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.414 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.414 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.414 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.414 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.414 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.414 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.414 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.414 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f07749cc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:32.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f07749cc640 1 -- 172.21.15.115:0/2054209760 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f07540065d0 con 0x7f0754002870 2024-01-24T23:27:32.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f075800fcc0 con 0x7f077009ff10 2024-01-24T23:27:32.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(217..217 src has 1..217) v4 ==== 13328+0+0 (unknown 244530927 0 0) 0x7f0758045fe0 con 0x7f077009ff10 2024-01-24T23:27:32.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 124980845 0 0) 0x7f07540065d0 con 0x7f0754002870 2024-01-24T23:27:32.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 10 monclient(hunting): my global_id is 124180 2024-01-24T23:27:32.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07700720e0 con 0x7f0754002870 2024-01-24T23:27:32.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f076ed9d640 10 monclient: get_auth_request con 0x7f07500378d0 auth_method 0 2024-01-24T23:27:32.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 178228760 0 0) 0x7f07700720e0 con 0x7f0754002870 2024-01-24T23:27:32.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f07540065d0 con 0x7f0754002870 2024-01-24T23:27:32.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0768001680 con 0x7f0754002870 2024-01-24T23:27:32.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:27:32.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 10 monclient: dump: 2024-01-24T23:27:32.416 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:32.416 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.416 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.416 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.416 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.416 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.417 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.417 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.417 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.417 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.417 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.417 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.417 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.417 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.417 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.417 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1725591706 0 0) 0x7f07540065d0 con 0x7f0754002870 2024-01-24T23:27:32.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 --> v1:172.21.15.132:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f075400a120 con 0x7f0754002870 2024-01-24T23:27:32.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 <== mon.7 v1:172.21.15.132:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f0768004b70 con 0x7f0754002870 2024-01-24T23:27:32.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:32.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:32.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 >> v1:172.21.15.132:6792/0 conn(0x7f0754002870 legacy=0x7f0754004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.392+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 >> v1:172.21.15.132:6792/0 conn(0x7f0754002870 legacy=0x7f0754004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:32.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f07749cc640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:27:32.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f07749cc640 1 -- 172.21.15.115:0/2054209760 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0754009e10 con 0x7f07540090d0 2024-01-24T23:27:32.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593752976 0 0) 0x7f0754009e10 con 0x7f07540090d0 2024-01-24T23:27:32.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f0766ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f0766ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f0766ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f0766ffd640 10 monclient(hunting): my global_id is 124189 2024-01-24T23:27:32.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07540065d0 con 0x7f07540090d0 2024-01-24T23:27:32.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3879005625 0 0) 0x7f07540065d0 con 0x7f07540090d0 2024-01-24T23:27:32.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0754009e10 con 0x7f07540090d0 2024-01-24T23:27:32.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0768005210 con 0x7f07540090d0 2024-01-24T23:27:32.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f0766ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f0766ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:27:32.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f0766ffd640 10 monclient: dump: 2024-01-24T23:27:32.419 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:32.419 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.420 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.420 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.420 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.420 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.420 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.420 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.420 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.420 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.420 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.420 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.420 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.420 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.420 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.420 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2385270140 0 0) 0x7f0754009e10 con 0x7f07540090d0 2024-01-24T23:27:32.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 --> v1:172.21.15.132:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f075400c310 con 0x7f07540090d0 2024-01-24T23:27:32.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 <== mon.7 v1:172.21.15.132:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 4028440416) 0x7f07680074d0 con 0x7f07540090d0 2024-01-24T23:27:32.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f0766ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:27:32.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f0766ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:32.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 >> v1:172.21.15.132:6792/0 conn(0x7f07540090d0 legacy=0x7f0754009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f0766ffd640 1 -- 172.21.15.115:0/2054209760 >> v1:172.21.15.132:6792/0 conn(0x7f07540090d0 legacy=0x7f0754009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:32.421 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:27:32.421 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:32.421 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:32.422 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:27:32.422 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 40, 2024-01-24T23:27:32.422 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:27:32.422 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:27:32.422 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:27:32.422 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:27:32.422 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:27:32.422 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:27:32.422 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:27:32.422 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:27:32.422 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:27:32.422 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:27:32.422 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:32.422 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 1, 2024-01-24T23:27:32.423 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:32.423 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:27:32.423 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:27:32.423 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:32.423 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:32.423 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:32.423 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:32.423 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:32.423 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:32.423 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:32.423 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:32.423 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:32.423 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:32.423 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:32.424 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:32.424 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:27:32.424 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:27:32.424 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:32.424 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:32.424 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:32.424 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:32.424 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:32.424 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:32.424 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:32.424 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:32.424 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:32.424 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:32.424 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:32.425 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.425 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:27:32.425 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:27:32.425 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:27:32.425 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:27:32.425 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:27:32.425 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:27:32.425 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:32.425 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:32.425 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:27:32.425 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:27:32.425 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:27:32.426 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:27:32.426 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:27:32.426 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:27:32.426 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:27:32.426 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:32.426 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:27:32.426 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:32.426 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:32.426 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:32.426 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:32.426 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:32.426 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:32.427 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:32.427 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:32.427 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:32.428 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:32.428 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:32.428 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:32.428 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:27:32.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.428 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:27:32.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.428 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:32.428 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:32.428 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.428 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.428 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.429 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:27:32.429 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.429 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.429 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.429 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:27:32.429 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:27:32.429 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.429 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.429 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.429 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:32.429 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:32.430 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.430 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.430 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.430 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:27:32.430 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.430 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.430 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.430 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:27:32.430 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:27:32.430 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.430 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.430 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.431 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:32.431 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:32.431 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.431 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.431 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.431 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:27:32.431 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.431 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.431 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.431 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:27:32.431 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:27:32.432 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.432 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.432 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.432 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:32.432 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:32.432 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.432 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.432 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.432 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:27:32.432 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.432 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.433 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.433 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:27:32.433 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:27:32.433 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.433 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.433 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.433 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:32.433 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:32.433 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.433 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.434 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.434 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:27:32.434 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.434 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.434 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.434 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:27:32.434 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:27:32.434 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.434 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.434 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.435 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:32.435 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:32.435 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.435 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.435 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.435 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:27:32.435 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.435 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.435 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.435 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:27:32.435 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:27:32.435 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.436 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.436 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.436 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:32.436 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:32.436 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.436 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.436 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.436 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:27:32.436 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.436 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.436 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.437 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:27:32.437 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:27:32.437 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.437 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.437 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.437 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:32.437 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:32.437 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.437 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.437 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.438 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:27:32.438 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.438 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.438 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.438 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:27:32.438 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:27:32.438 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.438 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.438 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.438 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:32.438 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:32.438 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:32.438 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:32.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.439 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:32.439 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:27:32.439 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:32.439 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.439 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.439 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:27:32.439 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:27:32.439 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:32.439 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:32.439 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:32.439 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.440 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.440 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:27:32.440 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:27:32.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.440 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:32.440 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:32.440 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:32.440 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.440 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:32.440 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:27:32.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:32.440 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:32.441 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:32.441 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:32.441 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:32.441 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:32.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:32.441 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:27:32.441 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:27:32.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f07749cc640 1 -- 172.21.15.115:0/2054209760 >> v1:172.21.15.132:6812/1835401922 conn(0x7f07500378d0 legacy=0x7f0750039d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f07749cc640 10 monclient: shutdown 2024-01-24T23:27:32.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f07749cc640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:32.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f07749cc640 1 -- 172.21.15.115:0/2054209760 >> v1:172.21.15.115:6789/0 conn(0x7f077009ff10 legacy=0x7f07701d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f076ed9d640 1 -- 172.21.15.115:0/2054209760 reap_dead start 2024-01-24T23:27:32.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f076ed9d640 1 -- 172.21.15.115:0/2054209760 reap_dead start 2024-01-24T23:27:32.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f07749cc640 1 -- 172.21.15.115:0/2054209760 shutdown_connections 2024-01-24T23:27:32.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f07749cc640 1 -- 172.21.15.115:0/2054209760 >> 172.21.15.115:0/2054209760 conn(0x7f07700a7820 msgr2=0x7f0770070e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:32.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f07749cc640 1 -- 172.21.15.115:0/2054209760 shutdown_connections 2024-01-24T23:27:32.443 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.400+0000 7f07749cc640 1 -- 172.21.15.115:0/2054209760 wait complete. 2024-01-24T23:27:32.443 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:27:32.443 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-01-24T23:27:32.444 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-01-24T23:27:32.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.556+0000 7f29d9201640 10 monclient: get_monmap_and_config 2024-01-24T23:27:32.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.556+0000 7f29d9201640 10 monclient: build_initial_monmap 2024-01-24T23:27:32.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.556+0000 7f29d9201640 10 monclient: monmap: 2024-01-24T23:27:32.560 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:32.560 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.560 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:32.561721+0000 2024-01-24T23:27:32.560 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:32.561721+0000 2024-01-24T23:27:32.560 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:32.560 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.560 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:32.561 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:32.561 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:32.561 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:32.561 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:32.561 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:32.561 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:32.561 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:32.561 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:32.561 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 1 Processor -- start 2024-01-24T23:27:32.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 1 -- start start 2024-01-24T23:27:32.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 10 monclient: init 2024-01-24T23:27:32.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:32.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 10 monclient: _add_conns ranks=[5,4,6,8,0,1,7,3,2] 2024-01-24T23:27:32.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 10 monclient(hunting): picked mon.noname-h con 0x7f29d409ec20 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:32.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 10 monclient(hunting): picked mon.noname-c con 0x7f29d4134e20 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:32.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 10 monclient(hunting): picked mon.noname-d con 0x7f29d413b3a0 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:32.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d404ed70 con 0x7f29d4134e20 2024-01-24T23:27:32.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d409ccb0 con 0x7f29d409ec20 2024-01-24T23:27:32.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d409cf00 con 0x7f29d413b3a0 2024-01-24T23:27:32.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:32.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 10 monclient(hunting): authenticate will time out at 1331.485006s 2024-01-24T23:27:32.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d259c640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f29d4134e20 0x7f29d4139c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:57624/0 (socket says 172.21.15.115:57624) 2024-01-24T23:27:32.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d259c640 1 -- 172.21.15.115:0/3327267852 learned_addr learned my addr 172.21.15.115:0/3327267852 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:32.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d359e640 1 --1- 172.21.15.115:0/3327267852 >> v1:172.21.15.115:6792/0 conn(0x7f29d413b3a0 0x7f29d413d790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:54732/0 (socket says 172.21.15.115:54732) 2024-01-24T23:27:32.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 1 -- 172.21.15.115:0/3327267852 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3134185515 0 0) 0x7f29d409cf00 con 0x7f29d413b3a0 2024-01-24T23:27:32.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 10 monclient(hunting): my global_id is 6774 2024-01-24T23:27:32.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 1 -- 172.21.15.115:0/3327267852 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29bc003530 con 0x7f29d413b3a0 2024-01-24T23:27:32.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 1 -- 172.21.15.115:0/3327267852 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1412979743 0 0) 0x7f29d404ed70 con 0x7f29d4134e20 2024-01-24T23:27:32.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 10 monclient(hunting): my global_id is 94801 2024-01-24T23:27:32.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 1 -- 172.21.15.115:0/3327267852 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29d409cf00 con 0x7f29d4134e20 2024-01-24T23:27:32.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 1 -- 172.21.15.115:0/3327267852 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1782693007 0 0) 0x7f29bc003530 con 0x7f29d413b3a0 2024-01-24T23:27:32.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 1 -- 172.21.15.115:0/3327267852 >> v1:172.21.15.132:6791/0 conn(0x7f29d409ec20 legacy=0x7f29d409f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 1 -- 172.21.15.115:0/3327267852 >> v1:172.21.15.115:6791/0 conn(0x7f29d4134e20 legacy=0x7f29d4139c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 10 monclient: _finish_hunting 0 2024-01-24T23:27:32.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 1 monclient: found mon.noname-d 2024-01-24T23:27:32.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:32.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:27:32.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 1 -- 172.21.15.115:0/3327267852 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d4135240 con 0x7f29d413b3a0 2024-01-24T23:27:32.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 10 monclient: _finish_auth 0 2024-01-24T23:27:32.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 10 monclient: _check_auth_tickets 2024-01-24T23:27:32.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:32.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 1 -- 172.21.15.115:0/3327267852 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f29c40017c0 con 0x7f29d413b3a0 2024-01-24T23:27:32.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 5 monclient: authenticate success, global_id 6774 2024-01-24T23:27:32.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:32.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:27:32.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 10 monclient: dump: 2024-01-24T23:27:32.571 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:32.571 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.571 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.571 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.571 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.571 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.571 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.571 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.571 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.571 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.571 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.571 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.571 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 1 -- 172.21.15.115:0/3327267852 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f29c40019b0 con 0x7f29d413b3a0 2024-01-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 1 -- 172.21.15.115:0/3327267852 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f29c4004b30 con 0x7f29d413b3a0 2024-01-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d1d9b640 10 monclient: dump: 2024-01-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.572 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.573 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.573 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.573 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.573 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.573 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.573 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.573 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.573 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.573 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.573 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.573 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.573 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:32.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 4 set_mon_vals no callback set 2024-01-24T23:27:32.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 10 monclient: shutdown 2024-01-24T23:27:32.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:32.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 1 -- 172.21.15.115:0/3327267852 >> v1:172.21.15.115:6792/0 conn(0x7f29d413b3a0 legacy=0x7f29d413d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 1 -- 172.21.15.115:0/3327267852 shutdown_connections 2024-01-24T23:27:32.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 1 -- 172.21.15.115:0/3327267852 >> 172.21.15.115:0/3327267852 conn(0x7f29d40a2010 msgr2=0x7f29d40a23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:32.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 1 -- 172.21.15.115:0/3327267852 shutdown_connections 2024-01-24T23:27:32.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 1 -- 172.21.15.115:0/3327267852 wait complete. 2024-01-24T23:27:32.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 10 monclient: build_initial_monmap 2024-01-24T23:27:32.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 10 monclient: monmap: 2024-01-24T23:27:32.575 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:32.575 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:32.575 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:32.575 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:32.575 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:32.575 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:32.575 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:32.575 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:32.575 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:32.575 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:32.575 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:32.576 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 1 Processor -- start 2024-01-24T23:27:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 1 -- start start 2024-01-24T23:27:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.560+0000 7f29d9201640 10 monclient: init 2024-01-24T23:27:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d9201640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d9201640 10 monclient: _add_conns ranks=[7,4,5,8,0,6,2,3,1] 2024-01-24T23:27:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d9201640 10 monclient(hunting): picked mon.noname-i con 0x7f29d409ec20 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d9201640 10 monclient(hunting): picked mon.noname-c con 0x7f29d4134e20 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d9201640 10 monclient(hunting): picked mon.noname-h con 0x7f29d413b3a0 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d9201640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d409fa40 con 0x7f29d4134e20 2024-01-24T23:27:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d9201640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d4139af0 con 0x7f29d413b3a0 2024-01-24T23:27:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d9201640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d409bfb0 con 0x7f29d409ec20 2024-01-24T23:27:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d9201640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d9201640 10 monclient(hunting): authenticate will time out at 1331.488116s 2024-01-24T23:27:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d259c640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f29d4134e20 0x7f29d41c9c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:57632/0 (socket says 172.21.15.115:57632) 2024-01-24T23:27:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d259c640 1 -- 172.21.15.115:0/156380118 learned_addr learned my addr 172.21.15.115:0/156380118 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 1 -- 172.21.15.115:0/156380118 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3904462656 0 0) 0x7f29d409fa40 con 0x7f29d4134e20 2024-01-24T23:27:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 10 monclient(hunting): my global_id is 94810 2024-01-24T23:27:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 1 -- 172.21.15.115:0/156380118 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29c0003530 con 0x7f29d4134e20 2024-01-24T23:27:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 1 -- 172.21.15.115:0/156380118 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1800215555 0 0) 0x7f29c0003530 con 0x7f29d4134e20 2024-01-24T23:27:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 1 -- 172.21.15.115:0/156380118 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f29d409fa40 con 0x7f29d4134e20 2024-01-24T23:27:32.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 1 -- 172.21.15.115:0/156380118 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f29c80015e0 con 0x7f29d4134e20 2024-01-24T23:27:32.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:27:32.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 10 monclient(hunting): dump: 2024-01-24T23:27:32.578 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:32.578 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.578 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.578 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.578 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.578 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.578 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.578 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.578 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.578 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.579 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.579 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.579 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.579 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.579 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.579 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:32.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 1 -- 172.21.15.115:0/156380118 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 123012728 0 0) 0x7f29d409fa40 con 0x7f29d4134e20 2024-01-24T23:27:32.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 1 -- 172.21.15.115:0/156380118 >> v1:172.21.15.132:6792/0 conn(0x7f29d409ec20 legacy=0x7f29d41c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 1 -- 172.21.15.115:0/156380118 >> v1:172.21.15.132:6791/0 conn(0x7f29d413b3a0 legacy=0x7f29d41d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:27:32.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 1 monclient: found mon.c 2024-01-24T23:27:32.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:32.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:32.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 1 -- 172.21.15.115:0/156380118 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d4072af0 con 0x7f29d4134e20 2024-01-24T23:27:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 10 monclient: _finish_auth 0 2024-01-24T23:27:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d9201640 5 monclient: authenticate success, global_id 94810 2024-01-24T23:27:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d9201640 10 monclient: _renew_subs 2024-01-24T23:27:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d9201640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d9201640 1 -- 172.21.15.115:0/156380118 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d41ca340 con 0x7f29d4134e20 2024-01-24T23:27:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d9201640 10 monclient: _renew_subs 2024-01-24T23:27:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d9201640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d9201640 1 -- 172.21.15.115:0/156380118 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d406dcf0 con 0x7f29d4134e20 2024-01-24T23:27:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 1 -- 172.21.15.115:0/156380118 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f29c8004aa0 con 0x7f29d4134e20 2024-01-24T23:27:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 1 -- 172.21.15.115:0/156380118 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f29c8004f50 con 0x7f29d4134e20 2024-01-24T23:27:32.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:32.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d159a640 4 set_mon_vals no callback set 2024-01-24T23:27:32.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:32.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 10 monclient: dump: 2024-01-24T23:27:32.581 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:32.581 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:32.581 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.581 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:32.581 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:32.581 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:32.581 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:32.581 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:32.581 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:32.581 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:32.582 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:32.582 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:32.582 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:32.582 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:32.582 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:32.582 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:32.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d9201640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:32.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d9201640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:32.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d9201640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:32.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d9201640 1 -- 172.21.15.115:0/156380118 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b4005090 con 0x7f29d4134e20 2024-01-24T23:27:32.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 1 -- 172.21.15.115:0/156380118 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f29c800fa50 con 0x7f29d4134e20 2024-01-24T23:27:32.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29bb7fe640 1 -- 172.21.15.115:0/156380118 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(217..217 src has 1..217) v4 ==== 13328+0+0 (unknown 244530927 0 0) 0x7f29c8045ed0 con 0x7f29d4134e20 2024-01-24T23:27:32.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.564+0000 7f29d2d9d640 10 monclient: get_auth_request con 0x7f29c0036e30 auth_method 0 2024-01-24T23:27:32.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.568+0000 7f29bb7fe640 1 -- 172.21.15.115:0/156380118 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f29c80149d0 con 0x7f29d4134e20 2024-01-24T23:27:32.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.568+0000 7f29bb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:32.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.568+0000 7f29bb7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:32.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.784+0000 7f29d9201640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-01-24T23:27:32.786 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.784+0000 7f29d9201640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-01-24T23:27:32.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.784+0000 7f29d9201640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:32.787 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.784+0000 7f29d9201640 1 -- 172.21.15.115:0/156380118 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f29b4005b50 con 0x7f29d4134e20 2024-01-24T23:27:32.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.788+0000 7f29bb7fe640 1 -- 172.21.15.115:0/156380118 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (unknown 246732539 0 0) 0x7f29c8013070 con 0x7f29d4134e20 2024-01-24T23:27:32.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.788+0000 7f29bb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-01-24T23:27:32.789 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.788+0000 7f29bb7fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:32.791 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.790+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.791 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.790+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=20,config=2,health=11,kv=9,logm=58} crc {auth=420394691,config=1618278320,health=3833386041,kv=3447624886,logm=3014776329}) 2024-01-24T23:27:32.791 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.790+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=19,config=1,health=11,kv=9,logm=60} crc {auth=1241689987,config=1228700967,health=3346653304,kv=3447624886,logm=2142136436}) 2024-01-24T23:27:32.791 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.790+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.791 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.790+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=20,config=2,health=11,kv=9,logm=58} crc {auth=420394691,config=1618278320,health=3833386041,kv=3447624886,logm=3014776329}) 2024-01-24T23:27:32.791 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.790+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=19,config=2,health=12,kv=9,logm=58} crc {auth=35235175,config=1618278320,health=3087514651,kv=3447624886,logm=2596242601}) 2024-01-24T23:27:32.791 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.790+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.791 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.790+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=20,config=2,health=11,kv=9,logm=58} crc {auth=420394691,config=1618278320,health=3833386041,kv=3447624886,logm=3014776329}) 2024-01-24T23:27:32.792 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.790+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=21,config=2,health=12,kv=9,logm=56} crc {auth=1327458978,config=1618278320,health=3087514651,kv=3447624886,logm=4047160226}) 2024-01-24T23:27:32.792 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.790+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.792 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.790+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=20,config=2,health=11,kv=9,logm=58} crc {auth=420394691,config=1618278320,health=3833386041,kv=3447624886,logm=3014776329}) 2024-01-24T23:27:32.792 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.790+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=21,config=2,health=12,kv=9,logm=56} crc {auth=1327458978,config=1618278320,health=3087514651,kv=3447624886,logm=2928597716}) 2024-01-24T23:27:32.792 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.790+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.792 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.790+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=20,config=2,health=11,kv=9,logm=58} crc {auth=420394691,config=1618278320,health=3833386041,kv=3447624886,logm=3014776329}) 2024-01-24T23:27:32.792 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.790+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=18,config=2,health=11,kv=8,logm=61} crc {auth=3927383162,config=1618278320,health=2448717686,kv=2560653513,logm=959085129}) 2024-01-24T23:27:32.792 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.790+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.792 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.790+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=20,config=2,health=11,kv=9,logm=58} crc {auth=420394691,config=1618278320,health=3833386041,kv=3447624886,logm=3014776329}) 2024-01-24T23:27:32.792 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.790+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=20,config=2,health=12,kv=7,logm=59} crc {auth=4080629337,config=1618278320,health=3087514651,kv=3644668441,logm=1053894867}) 2024-01-24T23:27:32.792 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.790+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.792 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.790+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=20,config=2,health=11,kv=9,logm=58} crc {auth=420394691,config=1618278320,health=3833386041,kv=3447624886,logm=3014776329}) 2024-01-24T23:27:32.792 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.790+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=20,config=1,health=12,kv=9,logm=58} crc {auth=650892174,config=1228700967,health=3087514651,kv=3447624886,logm=720461143}) 2024-01-24T23:27:32.793 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.790+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.793 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.790+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=20,config=2,health=11,kv=9,logm=58} crc {auth=420394691,config=1618278320,health=3833386041,kv=3447624886,logm=3014776329}) 2024-01-24T23:27:32.793 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.790+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=18,health=11,kv=8,logm=63} crc {auth=3617204078,health=2643264880,kv=4100078458,logm=3731705342}) 2024-01-24T23:27:32.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.788+0000 7f29d9201640 1 -- 172.21.15.115:0/156380118 >> v1:172.21.15.132:6812/1835401922 conn(0x7f29c0036e30 legacy=0x7f29c00392f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.788+0000 7f29d9201640 10 monclient: shutdown 2024-01-24T23:27:32.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.788+0000 7f29d9201640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:32.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.788+0000 7f29d9201640 1 -- 172.21.15.115:0/156380118 >> v1:172.21.15.115:6791/0 conn(0x7f29d4134e20 legacy=0x7f29d41c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:32.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.788+0000 7f29d9201640 1 -- 172.21.15.115:0/156380118 shutdown_connections 2024-01-24T23:27:32.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.788+0000 7f29d9201640 1 -- 172.21.15.115:0/156380118 >> 172.21.15.115:0/156380118 conn(0x7f29d40a2010 msgr2=0x7f29d40953a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:32.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.788+0000 7f29d9201640 1 -- 172.21.15.115:0/156380118 shutdown_connections 2024-01-24T23:27:32.794 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:32.788+0000 7f29d9201640 1 -- 172.21.15.115:0/156380118 wait complete. 2024-01-24T23:27:32.796 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.794+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.796 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.794+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1598174643}) 2024-01-24T23:27:32.796 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.794+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3842173506}) 2024-01-24T23:27:32.796 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.794+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.796 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.794+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1598174643}) 2024-01-24T23:27:32.796 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.794+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2368229266}) 2024-01-24T23:27:32.796 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.794+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.796 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.794+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1598174643}) 2024-01-24T23:27:32.796 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.794+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=794590895}) 2024-01-24T23:27:32.796 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.794+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.796 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.794+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1598174643}) 2024-01-24T23:27:32.796 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.794+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=901937740}) 2024-01-24T23:27:32.796 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.794+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.797 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.794+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1598174643}) 2024-01-24T23:27:32.797 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.794+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2200055092}) 2024-01-24T23:27:32.797 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.794+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.797 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.794+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1598174643}) 2024-01-24T23:27:32.797 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.794+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3454946527}) 2024-01-24T23:27:32.797 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.794+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.797 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.794+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1598174643}) 2024-01-24T23:27:32.797 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.794+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3133661089}) 2024-01-24T23:27:32.797 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.794+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.797 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.794+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1598174643}) 2024-01-24T23:27:32.797 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.794+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=185573948}) 2024-01-24T23:27:32.801 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.798+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.801 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.798+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3374210802}) 2024-01-24T23:27:32.801 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.798+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=533405162}) 2024-01-24T23:27:32.801 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.798+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.801 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.798+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3374210802}) 2024-01-24T23:27:32.801 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.798+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2013551428}) 2024-01-24T23:27:32.801 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.798+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.801 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.798+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3374210802}) 2024-01-24T23:27:32.801 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.798+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1729654419}) 2024-01-24T23:27:32.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.798+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.798+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3374210802}) 2024-01-24T23:27:32.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.798+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3997236331}) 2024-01-24T23:27:32.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.798+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.798+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3374210802}) 2024-01-24T23:27:32.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.798+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=13375538}) 2024-01-24T23:27:32.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.798+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.798+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3374210802}) 2024-01-24T23:27:32.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.798+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=913563531}) 2024-01-24T23:27:32.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.798+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.798+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3374210802}) 2024-01-24T23:27:32.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.798+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3593159459}) 2024-01-24T23:27:32.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.798+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.798+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3374210802}) 2024-01-24T23:27:32.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.798+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=59969309}) 2024-01-24T23:27:32.804 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.802+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.805 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.802+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1212578568}) 2024-01-24T23:27:32.805 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.802+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=466706315}) 2024-01-24T23:27:32.805 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.802+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.805 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.802+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1212578568}) 2024-01-24T23:27:32.805 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.802+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=625235009}) 2024-01-24T23:27:32.805 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.802+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.805 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.802+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1212578568}) 2024-01-24T23:27:32.805 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.802+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=482849147}) 2024-01-24T23:27:32.805 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.802+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.805 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.802+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1212578568}) 2024-01-24T23:27:32.805 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.802+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=985680031}) 2024-01-24T23:27:32.805 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.802+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.805 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.802+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1212578568}) 2024-01-24T23:27:32.805 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.802+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2814327120}) 2024-01-24T23:27:32.806 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.802+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.806 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.802+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1212578568}) 2024-01-24T23:27:32.806 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.802+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3948253733}) 2024-01-24T23:27:32.806 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.802+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.806 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.802+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1212578568}) 2024-01-24T23:27:32.806 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.802+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1507337683}) 2024-01-24T23:27:32.806 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.802+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.806 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.802+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1212578568}) 2024-01-24T23:27:32.806 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.802+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3506392859}) 2024-01-24T23:27:32.808 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.806+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.808 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.806+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1424565365}) 2024-01-24T23:27:32.808 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.806+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1082117503}) 2024-01-24T23:27:32.808 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.806+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.808 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.806+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1424565365}) 2024-01-24T23:27:32.809 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.806+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4131763262}) 2024-01-24T23:27:32.809 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.806+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.809 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.806+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1424565365}) 2024-01-24T23:27:32.809 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.806+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2580607714}) 2024-01-24T23:27:32.809 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.806+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.809 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.806+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1424565365}) 2024-01-24T23:27:32.809 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.806+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2877719101}) 2024-01-24T23:27:32.809 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.806+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.809 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.806+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1424565365}) 2024-01-24T23:27:32.809 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.806+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3217647497}) 2024-01-24T23:27:32.809 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.806+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.809 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.806+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1424565365}) 2024-01-24T23:27:32.809 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.806+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4259845986}) 2024-01-24T23:27:32.809 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.806+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.810 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.806+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1424565365}) 2024-01-24T23:27:32.810 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.806+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=696947855}) 2024-01-24T23:27:32.810 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.806+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.810 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.806+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1424565365}) 2024-01-24T23:27:32.810 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.806+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1544526527}) 2024-01-24T23:27:32.811 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-01-24T23:27:32.812 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.810+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.812 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.810+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1960160687}) 2024-01-24T23:27:32.812 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.810+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2899754465}) 2024-01-24T23:27:32.813 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.810+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.813 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.810+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1960160687}) 2024-01-24T23:27:32.813 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.810+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4177579097}) 2024-01-24T23:27:32.813 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.810+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.813 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.810+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1960160687}) 2024-01-24T23:27:32.813 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.810+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2113740194}) 2024-01-24T23:27:32.813 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.810+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.813 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.810+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1960160687}) 2024-01-24T23:27:32.813 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.810+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=448044417}) 2024-01-24T23:27:32.813 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.810+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.813 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.810+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1960160687}) 2024-01-24T23:27:32.813 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.810+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=908488831}) 2024-01-24T23:27:32.813 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.810+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.813 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.810+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1960160687}) 2024-01-24T23:27:32.813 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.810+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3098494435}) 2024-01-24T23:27:32.814 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.810+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.814 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.810+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1960160687}) 2024-01-24T23:27:32.814 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.810+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4067086989}) 2024-01-24T23:27:32.814 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.810+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.814 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.810+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1960160687}) 2024-01-24T23:27:32.814 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.810+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=760819306}) 2024-01-24T23:27:32.815 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.815 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=355329233}) 2024-01-24T23:27:32.815 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4175388882}) 2024-01-24T23:27:32.815 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.815 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=355329233}) 2024-01-24T23:27:32.815 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2724663342}) 2024-01-24T23:27:32.815 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.816 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=355329233}) 2024-01-24T23:27:32.816 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3628543371}) 2024-01-24T23:27:32.816 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.816 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=355329233}) 2024-01-24T23:27:32.816 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2445978829}) 2024-01-24T23:27:32.816 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.816 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=355329233}) 2024-01-24T23:27:32.816 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4079099336}) 2024-01-24T23:27:32.816 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.816 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=355329233}) 2024-01-24T23:27:32.816 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3993540723}) 2024-01-24T23:27:32.816 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.816 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=355329233}) 2024-01-24T23:27:32.816 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1583551370}) 2024-01-24T23:27:32.817 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.817 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=355329233}) 2024-01-24T23:27:32.817 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=352130505}) 2024-01-24T23:27:32.818 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.818 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2699324317}) 2024-01-24T23:27:32.819 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1208736230}) 2024-01-24T23:27:32.819 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.819 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2699324317}) 2024-01-24T23:27:32.819 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1141249950}) 2024-01-24T23:27:32.819 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.819 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2699324317}) 2024-01-24T23:27:32.819 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3998360337}) 2024-01-24T23:27:32.819 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.819 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2699324317}) 2024-01-24T23:27:32.819 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=73240713}) 2024-01-24T23:27:32.819 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.819 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2699324317}) 2024-01-24T23:27:32.819 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=358538125}) 2024-01-24T23:27:32.819 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.820 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2699324317}) 2024-01-24T23:27:32.820 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3399908818}) 2024-01-24T23:27:32.820 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.820 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2699324317}) 2024-01-24T23:27:32.820 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.814+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2699592461}) 2024-01-24T23:27:32.820 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.820 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2699324317}) 2024-01-24T23:27:32.820 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2238311235}) 2024-01-24T23:27:32.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2836840419}) 2024-01-24T23:27:32.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2011268323}) 2024-01-24T23:27:32.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2836840419}) 2024-01-24T23:27:32.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2987630573}) 2024-01-24T23:27:32.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2836840419}) 2024-01-24T23:27:32.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2743431203}) 2024-01-24T23:27:32.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2836840419}) 2024-01-24T23:27:32.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2179220532}) 2024-01-24T23:27:32.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2836840419}) 2024-01-24T23:27:32.823 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3189024699}) 2024-01-24T23:27:32.823 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.823 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2836840419}) 2024-01-24T23:27:32.823 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1333792485}) 2024-01-24T23:27:32.823 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.823 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2836840419}) 2024-01-24T23:27:32.823 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=809639465}) 2024-01-24T23:27:32.823 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.823 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2836840419}) 2024-01-24T23:27:32.823 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.818+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=935610306}) 2024-01-24T23:27:32.825 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.822+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.825 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.822+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2763624821}) 2024-01-24T23:27:32.825 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.822+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2294037798}) 2024-01-24T23:27:32.825 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.822+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.826 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.822+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2763624821}) 2024-01-24T23:27:32.826 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.822+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1683846307}) 2024-01-24T23:27:32.826 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.822+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.826 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.822+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2763624821}) 2024-01-24T23:27:32.826 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.822+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1294227257}) 2024-01-24T23:27:32.826 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.822+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.826 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.822+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2763624821}) 2024-01-24T23:27:32.826 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.822+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4069595731}) 2024-01-24T23:27:32.826 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.822+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.826 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.822+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2763624821}) 2024-01-24T23:27:32.826 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.822+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4079084072}) 2024-01-24T23:27:32.826 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.822+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.826 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.822+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2763624821}) 2024-01-24T23:27:32.826 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.822+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3390127778}) 2024-01-24T23:27:32.826 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.822+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.827 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.822+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2763624821}) 2024-01-24T23:27:32.827 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.822+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=910618377}) 2024-01-24T23:27:32.827 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.822+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.827 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.822+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2763624821}) 2024-01-24T23:27:32.827 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.822+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4271522145}) 2024-01-24T23:27:32.830 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.826+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.830 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.826+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1372384314}) 2024-01-24T23:27:32.830 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.826+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=483470339}) 2024-01-24T23:27:32.830 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.826+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.830 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.826+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1372384314}) 2024-01-24T23:27:32.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.826+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1572067421}) 2024-01-24T23:27:32.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.826+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.826+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1372384314}) 2024-01-24T23:27:32.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.826+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2318248711}) 2024-01-24T23:27:32.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.826+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.826+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1372384314}) 2024-01-24T23:27:32.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.826+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3678987335}) 2024-01-24T23:27:32.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.826+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.826+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1372384314}) 2024-01-24T23:27:32.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.826+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3960249678}) 2024-01-24T23:27:32.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.826+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.826+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1372384314}) 2024-01-24T23:27:32.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.826+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=572476088}) 2024-01-24T23:27:32.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.826+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.826+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1372384314}) 2024-01-24T23:27:32.832 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.826+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=921268360}) 2024-01-24T23:27:32.832 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.826+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.832 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.826+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1372384314}) 2024-01-24T23:27:32.832 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.826+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3544015315}) 2024-01-24T23:27:32.833 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.830+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.833 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.830+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3659794178}) 2024-01-24T23:27:32.833 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.830+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3999344320}) 2024-01-24T23:27:32.834 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.830+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.834 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.830+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3659794178}) 2024-01-24T23:27:32.834 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.830+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4102264272}) 2024-01-24T23:27:32.834 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.830+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.834 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.830+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3659794178}) 2024-01-24T23:27:32.834 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.830+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2913920775}) 2024-01-24T23:27:32.834 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.830+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.834 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.830+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3659794178}) 2024-01-24T23:27:32.834 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.830+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=598495936}) 2024-01-24T23:27:32.834 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.830+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.834 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.830+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3659794178}) 2024-01-24T23:27:32.834 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.830+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=705591781}) 2024-01-24T23:27:32.834 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.830+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.834 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.830+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3659794178}) 2024-01-24T23:27:32.835 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.830+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4011051196}) 2024-01-24T23:27:32.835 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.830+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.835 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.830+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3659794178}) 2024-01-24T23:27:32.835 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.830+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=559312963}) 2024-01-24T23:27:32.835 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.830+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.835 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.830+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3659794178}) 2024-01-24T23:27:32.835 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.830+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2765488732}) 2024-01-24T23:27:32.836 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.834+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.837 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.834+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3977228155}) 2024-01-24T23:27:32.837 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.834+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1942390475}) 2024-01-24T23:27:32.837 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.834+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.837 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.834+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3977228155}) 2024-01-24T23:27:32.837 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.834+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1190280400}) 2024-01-24T23:27:32.837 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.834+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.837 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.834+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3977228155}) 2024-01-24T23:27:32.837 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.834+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=780011505}) 2024-01-24T23:27:32.837 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.834+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.837 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.834+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3977228155}) 2024-01-24T23:27:32.837 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.834+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3393331296}) 2024-01-24T23:27:32.837 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.834+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.838 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.834+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3977228155}) 2024-01-24T23:27:32.838 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.834+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2782387631}) 2024-01-24T23:27:32.838 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.834+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.838 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.834+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3977228155}) 2024-01-24T23:27:32.838 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.834+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2989296606}) 2024-01-24T23:27:32.838 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.834+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.838 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.834+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3977228155}) 2024-01-24T23:27:32.838 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.834+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1280076548}) 2024-01-24T23:27:32.838 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.834+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.838 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.834+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3977228155}) 2024-01-24T23:27:32.838 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.834+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2460417458}) 2024-01-24T23:27:32.839 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.839 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2625762421}) 2024-01-24T23:27:32.839 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1858281534}) 2024-01-24T23:27:32.840 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.840 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2625762421}) 2024-01-24T23:27:32.840 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=375934855}) 2024-01-24T23:27:32.840 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.840 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2625762421}) 2024-01-24T23:27:32.840 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3550979251}) 2024-01-24T23:27:32.840 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.840 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2625762421}) 2024-01-24T23:27:32.840 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2574080672}) 2024-01-24T23:27:32.840 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.840 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2625762421}) 2024-01-24T23:27:32.840 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1042290437}) 2024-01-24T23:27:32.840 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.841 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2625762421}) 2024-01-24T23:27:32.841 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2399388334}) 2024-01-24T23:27:32.841 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.841 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2625762421}) 2024-01-24T23:27:32.841 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2934527715}) 2024-01-24T23:27:32.841 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.841 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2625762421}) 2024-01-24T23:27:32.841 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1213877981}) 2024-01-24T23:27:32.842 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.842 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=40180436}) 2024-01-24T23:27:32.842 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=629211334}) 2024-01-24T23:27:32.842 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.842 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=40180436}) 2024-01-24T23:27:32.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=102980406}) 2024-01-24T23:27:32.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=40180436}) 2024-01-24T23:27:32.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2638605018}) 2024-01-24T23:27:32.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=40180436}) 2024-01-24T23:27:32.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1223856451}) 2024-01-24T23:27:32.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=40180436}) 2024-01-24T23:27:32.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2773925462}) 2024-01-24T23:27:32.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=40180436}) 2024-01-24T23:27:32.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1591109378}) 2024-01-24T23:27:32.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.844 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=40180436}) 2024-01-24T23:27:32.844 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=930594409}) 2024-01-24T23:27:32.844 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.844 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=40180436}) 2024-01-24T23:27:32.844 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.838+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2797581532}) 2024-01-24T23:27:32.846 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.842+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.846 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.842+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1697544763}) 2024-01-24T23:27:32.846 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.842+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1819904052}) 2024-01-24T23:27:32.846 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.842+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.846 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.842+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1697544763}) 2024-01-24T23:27:32.847 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.842+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4254097393}) 2024-01-24T23:27:32.847 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.842+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.847 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.842+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1697544763}) 2024-01-24T23:27:32.847 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.842+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1750938937}) 2024-01-24T23:27:32.847 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.842+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.847 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.842+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1697544763}) 2024-01-24T23:27:32.847 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.842+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1231367893}) 2024-01-24T23:27:32.847 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.842+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.847 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.842+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1697544763}) 2024-01-24T23:27:32.847 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.842+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2885824076}) 2024-01-24T23:27:32.847 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.842+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.847 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.842+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1697544763}) 2024-01-24T23:27:32.847 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.842+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3640968140}) 2024-01-24T23:27:32.847 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.842+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.848 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.842+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1697544763}) 2024-01-24T23:27:32.848 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.842+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1959327124}) 2024-01-24T23:27:32.848 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.842+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.848 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.842+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1697544763}) 2024-01-24T23:27:32.848 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.842+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=220154044}) 2024-01-24T23:27:32.849 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.846+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.849 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.846+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3984726506}) 2024-01-24T23:27:32.849 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.846+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1933342940}) 2024-01-24T23:27:32.849 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.846+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.849 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.846+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3984726506}) 2024-01-24T23:27:32.850 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.846+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3979549782}) 2024-01-24T23:27:32.850 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.846+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.850 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.846+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3984726506}) 2024-01-24T23:27:32.850 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.846+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2435682269}) 2024-01-24T23:27:32.850 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.846+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.850 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.846+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3984726506}) 2024-01-24T23:27:32.850 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.846+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3160708579}) 2024-01-24T23:27:32.850 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.846+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.850 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.846+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3984726506}) 2024-01-24T23:27:32.850 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.846+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=102209269}) 2024-01-24T23:27:32.850 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.846+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.850 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.846+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3984726506}) 2024-01-24T23:27:32.850 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.846+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4030005378}) 2024-01-24T23:27:32.850 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.846+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.850 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.846+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3984726506}) 2024-01-24T23:27:32.851 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.846+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1206306188}) 2024-01-24T23:27:32.851 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.846+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.851 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.846+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3984726506}) 2024-01-24T23:27:32.851 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.846+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=695685586}) 2024-01-24T23:27:32.852 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.850+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.852 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.850+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=893418493}) 2024-01-24T23:27:32.852 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.850+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3520673306}) 2024-01-24T23:27:32.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.850+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.850+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=893418493}) 2024-01-24T23:27:32.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.850+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1388870489}) 2024-01-24T23:27:32.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.850+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.850+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=893418493}) 2024-01-24T23:27:32.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.850+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=125264555}) 2024-01-24T23:27:32.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.850+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.850+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=893418493}) 2024-01-24T23:27:32.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.850+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=50985240}) 2024-01-24T23:27:32.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.850+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.850+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=893418493}) 2024-01-24T23:27:32.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.850+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1671198563}) 2024-01-24T23:27:32.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.850+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.850+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=893418493}) 2024-01-24T23:27:32.854 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.850+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4283821356}) 2024-01-24T23:27:32.854 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.850+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.854 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.850+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=893418493}) 2024-01-24T23:27:32.854 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.850+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3512168690}) 2024-01-24T23:27:32.854 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.850+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.854 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.850+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=893418493}) 2024-01-24T23:27:32.854 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.850+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3367706631}) 2024-01-24T23:27:32.857 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.854+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.857 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.854+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=413526493}) 2024-01-24T23:27:32.857 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.854+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=809430710}) 2024-01-24T23:27:32.857 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.854+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.857 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.854+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=413526493}) 2024-01-24T23:27:32.857 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.854+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3037850414}) 2024-01-24T23:27:32.857 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.854+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.857 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.854+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=413526493}) 2024-01-24T23:27:32.857 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.854+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1922544891}) 2024-01-24T23:27:32.857 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.854+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.857 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.854+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=413526493}) 2024-01-24T23:27:32.857 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.854+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2458741906}) 2024-01-24T23:27:32.857 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.854+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.858 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.854+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=413526493}) 2024-01-24T23:27:32.858 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.854+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1124093961}) 2024-01-24T23:27:32.858 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.854+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.858 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.854+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=413526493}) 2024-01-24T23:27:32.858 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.854+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4211145972}) 2024-01-24T23:27:32.858 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.854+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.858 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.854+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=413526493}) 2024-01-24T23:27:32.858 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.854+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1782095436}) 2024-01-24T23:27:32.858 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.854+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.858 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.854+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=413526493}) 2024-01-24T23:27:32.858 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.854+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3905702554}) 2024-01-24T23:27:32.859 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.859 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1250546550}) 2024-01-24T23:27:32.859 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=277337491}) 2024-01-24T23:27:32.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1250546550}) 2024-01-24T23:27:32.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3862208231}) 2024-01-24T23:27:32.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1250546550}) 2024-01-24T23:27:32.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=117777948}) 2024-01-24T23:27:32.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1250546550}) 2024-01-24T23:27:32.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3865108324}) 2024-01-24T23:27:32.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1250546550}) 2024-01-24T23:27:32.861 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3015504488}) 2024-01-24T23:27:32.861 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.861 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1250546550}) 2024-01-24T23:27:32.861 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3466007626}) 2024-01-24T23:27:32.861 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.861 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1250546550}) 2024-01-24T23:27:32.861 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3770093571}) 2024-01-24T23:27:32.861 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.861 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1250546550}) 2024-01-24T23:27:32.861 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=640723579}) 2024-01-24T23:27:32.863 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.858+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.863 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1242694465}) 2024-01-24T23:27:32.863 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3667594564}) 2024-01-24T23:27:32.863 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.863 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1242694465}) 2024-01-24T23:27:32.863 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2893041840}) 2024-01-24T23:27:32.863 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.863 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1242694465}) 2024-01-24T23:27:32.863 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2593110375}) 2024-01-24T23:27:32.863 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.863 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1242694465}) 2024-01-24T23:27:32.864 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=60215424}) 2024-01-24T23:27:32.864 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.864 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1242694465}) 2024-01-24T23:27:32.864 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2330514581}) 2024-01-24T23:27:32.864 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.864 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1242694465}) 2024-01-24T23:27:32.864 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1593068434}) 2024-01-24T23:27:32.864 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.864 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1242694465}) 2024-01-24T23:27:32.864 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2515552440}) 2024-01-24T23:27:32.864 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.864 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1242694465}) 2024-01-24T23:27:32.864 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2780363565}) 2024-01-24T23:27:32.866 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.866 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4250295742}) 2024-01-24T23:27:32.866 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2656536323}) 2024-01-24T23:27:32.866 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.867 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4250295742}) 2024-01-24T23:27:32.867 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2418632111}) 2024-01-24T23:27:32.867 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.867 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4250295742}) 2024-01-24T23:27:32.867 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4175970142}) 2024-01-24T23:27:32.867 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.867 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4250295742}) 2024-01-24T23:27:32.867 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4272021669}) 2024-01-24T23:27:32.867 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.867 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4250295742}) 2024-01-24T23:27:32.867 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1476574882}) 2024-01-24T23:27:32.867 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.867 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4250295742}) 2024-01-24T23:27:32.868 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1377787733}) 2024-01-24T23:27:32.868 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.868 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4250295742}) 2024-01-24T23:27:32.868 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2418632111}) 2024-01-24T23:27:32.868 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.868 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4250295742}) 2024-01-24T23:27:32.868 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.862+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3852783244}) 2024-01-24T23:27:32.869 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.866+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.869 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.866+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3019435639}) 2024-01-24T23:27:32.869 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.866+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=822170614}) 2024-01-24T23:27:32.869 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.866+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.866+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3019435639}) 2024-01-24T23:27:32.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.866+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1244566331}) 2024-01-24T23:27:32.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.866+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.866+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3019435639}) 2024-01-24T23:27:32.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.866+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1794668844}) 2024-01-24T23:27:32.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.866+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.866+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3019435639}) 2024-01-24T23:27:32.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.866+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=88811697}) 2024-01-24T23:27:32.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.866+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.866+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3019435639}) 2024-01-24T23:27:32.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.866+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2778966375}) 2024-01-24T23:27:32.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.866+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.866+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3019435639}) 2024-01-24T23:27:32.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.866+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2389150958}) 2024-01-24T23:27:32.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.866+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.871 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.866+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3019435639}) 2024-01-24T23:27:32.871 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.866+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1244566331}) 2024-01-24T23:27:32.871 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.866+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.871 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.866+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3019435639}) 2024-01-24T23:27:32.871 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.866+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2131329620}) 2024-01-24T23:27:32.872 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.870+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.872 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.870+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3329733600}) 2024-01-24T23:27:32.872 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.870+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2155581245}) 2024-01-24T23:27:32.872 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.870+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.873 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.870+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3329733600}) 2024-01-24T23:27:32.873 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.870+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=298020569}) 2024-01-24T23:27:32.873 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.870+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.873 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.870+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3329733600}) 2024-01-24T23:27:32.873 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.870+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1477574824}) 2024-01-24T23:27:32.873 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.870+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.873 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.870+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3329733600}) 2024-01-24T23:27:32.873 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.870+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3996783524}) 2024-01-24T23:27:32.873 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.870+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.873 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.870+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3329733600}) 2024-01-24T23:27:32.873 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.870+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1263311633}) 2024-01-24T23:27:32.873 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.870+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.873 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.870+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3329733600}) 2024-01-24T23:27:32.873 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.870+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1833038869}) 2024-01-24T23:27:32.874 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.870+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.874 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.870+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3329733600}) 2024-01-24T23:27:32.874 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.870+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=298020569}) 2024-01-24T23:27:32.874 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.870+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.874 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.870+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3329733600}) 2024-01-24T23:27:32.874 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.870+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2870608383}) 2024-01-24T23:27:32.875 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=781775563}) 2024-01-24T23:27:32.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2360621009}) 2024-01-24T23:27:32.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=781775563}) 2024-01-24T23:27:32.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=615046149}) 2024-01-24T23:27:32.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=781775563}) 2024-01-24T23:27:32.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=7610478}) 2024-01-24T23:27:32.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=781775563}) 2024-01-24T23:27:32.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1614877546}) 2024-01-24T23:27:32.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.877 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=781775563}) 2024-01-24T23:27:32.877 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1550885525}) 2024-01-24T23:27:32.877 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.877 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=781775563}) 2024-01-24T23:27:32.877 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=267754277}) 2024-01-24T23:27:32.877 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.877 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=781775563}) 2024-01-24T23:27:32.877 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=615046149}) 2024-01-24T23:27:32.877 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.877 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=781775563}) 2024-01-24T23:27:32.877 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4247195575}) 2024-01-24T23:27:32.879 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.879 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1168784174}) 2024-01-24T23:27:32.879 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3488373459}) 2024-01-24T23:27:32.879 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.879 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1168784174}) 2024-01-24T23:27:32.879 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3980757538}) 2024-01-24T23:27:32.879 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.879 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.874+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1168784174}) 2024-01-24T23:27:32.879 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=68997244}) 2024-01-24T23:27:32.879 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.879 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1168784174}) 2024-01-24T23:27:32.879 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2099205940}) 2024-01-24T23:27:32.880 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.880 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1168784174}) 2024-01-24T23:27:32.880 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2702640913}) 2024-01-24T23:27:32.880 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.880 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1168784174}) 2024-01-24T23:27:32.880 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2409595661}) 2024-01-24T23:27:32.880 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.880 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1168784174}) 2024-01-24T23:27:32.880 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3980757538}) 2024-01-24T23:27:32.880 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.880 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1168784174}) 2024-01-24T23:27:32.880 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1323563046}) 2024-01-24T23:27:32.881 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3217198422}) 2024-01-24T23:27:32.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1965482782}) 2024-01-24T23:27:32.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3217198422}) 2024-01-24T23:27:32.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=955700837}) 2024-01-24T23:27:32.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3217198422}) 2024-01-24T23:27:32.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3773193857}) 2024-01-24T23:27:32.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3217198422}) 2024-01-24T23:27:32.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=155346359}) 2024-01-24T23:27:32.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3217198422}) 2024-01-24T23:27:32.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1408828276}) 2024-01-24T23:27:32.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3217198422}) 2024-01-24T23:27:32.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=155278536}) 2024-01-24T23:27:32.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3217198422}) 2024-01-24T23:27:32.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=955700837}) 2024-01-24T23:27:32.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3217198422}) 2024-01-24T23:27:32.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.878+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2948912547}) 2024-01-24T23:27:32.885 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.882+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.885 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.882+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=811248387}) 2024-01-24T23:27:32.885 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.882+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3415598567}) 2024-01-24T23:27:32.885 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.882+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.885 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.882+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=811248387}) 2024-01-24T23:27:32.886 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.882+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2842872502}) 2024-01-24T23:27:32.886 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.882+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.886 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.882+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=811248387}) 2024-01-24T23:27:32.886 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.882+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3163872598}) 2024-01-24T23:27:32.886 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.882+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.886 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.882+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=811248387}) 2024-01-24T23:27:32.886 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.882+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3156682550}) 2024-01-24T23:27:32.886 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.882+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.886 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.882+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=811248387}) 2024-01-24T23:27:32.886 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.882+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1177094994}) 2024-01-24T23:27:32.886 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.882+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.886 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.882+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=811248387}) 2024-01-24T23:27:32.886 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.882+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1407501324}) 2024-01-24T23:27:32.886 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.882+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.887 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.882+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=811248387}) 2024-01-24T23:27:32.887 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.882+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2842872502}) 2024-01-24T23:27:32.887 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.882+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.887 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.882+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=811248387}) 2024-01-24T23:27:32.887 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.882+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3389340044}) 2024-01-24T23:27:32.888 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.886+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.886+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3598029461}) 2024-01-24T23:27:32.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.886+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3106897766}) 2024-01-24T23:27:32.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.886+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.886+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3598029461}) 2024-01-24T23:27:32.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.886+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3013901115}) 2024-01-24T23:27:32.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.886+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.886+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3598029461}) 2024-01-24T23:27:32.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.886+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=930651192}) 2024-01-24T23:27:32.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.886+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.886+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3598029461}) 2024-01-24T23:27:32.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.886+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=670681679}) 2024-01-24T23:27:32.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.886+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.886+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3598029461}) 2024-01-24T23:27:32.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.886+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3237571716}) 2024-01-24T23:27:32.890 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.886+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.890 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.886+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3598029461}) 2024-01-24T23:27:32.890 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.886+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4007012885}) 2024-01-24T23:27:32.890 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.886+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.890 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.886+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3598029461}) 2024-01-24T23:27:32.890 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.886+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3013901115}) 2024-01-24T23:27:32.890 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.886+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.890 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.886+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3598029461}) 2024-01-24T23:27:32.890 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.886+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2971875729}) 2024-01-24T23:27:32.891 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.891 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2266994930}) 2024-01-24T23:27:32.891 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4265335731}) 2024-01-24T23:27:32.891 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.891 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2266994930}) 2024-01-24T23:27:32.891 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1043009529}) 2024-01-24T23:27:32.891 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.892 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2266994930}) 2024-01-24T23:27:32.892 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3322376363}) 2024-01-24T23:27:32.892 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.892 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2266994930}) 2024-01-24T23:27:32.892 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4191541161}) 2024-01-24T23:27:32.892 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.892 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2266994930}) 2024-01-24T23:27:32.892 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2864489833}) 2024-01-24T23:27:32.892 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.892 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2266994930}) 2024-01-24T23:27:32.892 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2951523200}) 2024-01-24T23:27:32.892 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.892 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2266994930}) 2024-01-24T23:27:32.892 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1043009529}) 2024-01-24T23:27:32.893 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.893 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2266994930}) 2024-01-24T23:27:32.893 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=493080818}) 2024-01-24T23:27:32.893 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.894 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3875710541}) 2024-01-24T23:27:32.894 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2989165487}) 2024-01-24T23:27:32.894 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.894 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3875710541}) 2024-01-24T23:27:32.894 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4122537945}) 2024-01-24T23:27:32.894 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.894 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3875710541}) 2024-01-24T23:27:32.894 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=84833272}) 2024-01-24T23:27:32.894 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.894 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3875710541}) 2024-01-24T23:27:32.894 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1040758969}) 2024-01-24T23:27:32.894 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.894 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3875710541}) 2024-01-24T23:27:32.895 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3832683192}) 2024-01-24T23:27:32.895 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.895 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3875710541}) 2024-01-24T23:27:32.895 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2579959510}) 2024-01-24T23:27:32.895 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.895 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3875710541}) 2024-01-24T23:27:32.895 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4122537945}) 2024-01-24T23:27:32.895 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.895 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3875710541}) 2024-01-24T23:27:32.895 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.890+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2686406573}) 2024-01-24T23:27:32.898 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.894+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.898 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.894+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1294209573}) 2024-01-24T23:27:32.898 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.894+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3070435957}) 2024-01-24T23:27:32.898 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.894+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.898 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.894+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1294209573}) 2024-01-24T23:27:32.898 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.894+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3696073679}) 2024-01-24T23:27:32.898 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.894+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.898 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.894+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1294209573}) 2024-01-24T23:27:32.898 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.894+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3699186900}) 2024-01-24T23:27:32.898 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.894+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.898 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.894+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1294209573}) 2024-01-24T23:27:32.898 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.894+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=482605593}) 2024-01-24T23:27:32.898 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.894+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.899 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.894+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1294209573}) 2024-01-24T23:27:32.899 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.894+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1710800634}) 2024-01-24T23:27:32.899 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.894+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.899 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.894+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1294209573}) 2024-01-24T23:27:32.899 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.894+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3219864589}) 2024-01-24T23:27:32.899 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.894+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.899 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.894+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1294209573}) 2024-01-24T23:27:32.899 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.894+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3696073679}) 2024-01-24T23:27:32.899 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.894+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.899 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.894+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1294209573}) 2024-01-24T23:27:32.899 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.894+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=307667925}) 2024-01-24T23:27:32.900 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.900 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1377245812}) 2024-01-24T23:27:32.901 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3124613068}) 2024-01-24T23:27:32.901 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.901 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1377245812}) 2024-01-24T23:27:32.901 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1313110198}) 2024-01-24T23:27:32.901 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.901 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1377245812}) 2024-01-24T23:27:32.901 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3379465470}) 2024-01-24T23:27:32.901 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.901 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1377245812}) 2024-01-24T23:27:32.901 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=163439257}) 2024-01-24T23:27:32.901 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.901 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1377245812}) 2024-01-24T23:27:32.901 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3689768827}) 2024-01-24T23:27:32.901 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.902 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1377245812}) 2024-01-24T23:27:32.902 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=885998074}) 2024-01-24T23:27:32.902 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.902 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1377245812}) 2024-01-24T23:27:32.902 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1313110198}) 2024-01-24T23:27:32.902 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.902 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1377245812}) 2024-01-24T23:27:32.902 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1710239497}) 2024-01-24T23:27:32.903 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.903 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3973297619}) 2024-01-24T23:27:32.903 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=673506137}) 2024-01-24T23:27:32.903 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.903 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3973297619}) 2024-01-24T23:27:32.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=459055168}) 2024-01-24T23:27:32.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3973297619}) 2024-01-24T23:27:32.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1443091294}) 2024-01-24T23:27:32.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3973297619}) 2024-01-24T23:27:32.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3866114313}) 2024-01-24T23:27:32.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3973297619}) 2024-01-24T23:27:32.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2881948196}) 2024-01-24T23:27:32.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3973297619}) 2024-01-24T23:27:32.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=331010453}) 2024-01-24T23:27:32.905 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.905 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3973297619}) 2024-01-24T23:27:32.905 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=459055168}) 2024-01-24T23:27:32.905 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.905 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3973297619}) 2024-01-24T23:27:32.905 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.898+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3138965974}) 2024-01-24T23:27:32.909 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.902+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.909 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.902+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2073299633}) 2024-01-24T23:27:32.909 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.902+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1535571163}) 2024-01-24T23:27:32.910 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.902+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.910 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.902+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2073299633}) 2024-01-24T23:27:32.910 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.902+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=487131021}) 2024-01-24T23:27:32.910 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.902+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.910 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.902+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2073299633}) 2024-01-24T23:27:32.910 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.902+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2732000267}) 2024-01-24T23:27:32.910 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.902+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.910 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.902+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2073299633}) 2024-01-24T23:27:32.910 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.902+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3029328174}) 2024-01-24T23:27:32.910 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.902+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.910 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.902+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2073299633}) 2024-01-24T23:27:32.910 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.902+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=252452592}) 2024-01-24T23:27:32.910 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.902+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.910 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.902+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2073299633}) 2024-01-24T23:27:32.911 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.902+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3745926671}) 2024-01-24T23:27:32.911 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.902+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.911 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.902+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2073299633}) 2024-01-24T23:27:32.911 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.902+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=487131021}) 2024-01-24T23:27:32.911 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.902+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.911 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.902+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2073299633}) 2024-01-24T23:27:32.911 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.902+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2286786834}) 2024-01-24T23:27:32.911 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.911 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2305314582}) 2024-01-24T23:27:32.911 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1500930949}) 2024-01-24T23:27:32.911 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.911 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2305314582}) 2024-01-24T23:27:32.911 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1386051876}) 2024-01-24T23:27:32.911 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2305314582}) 2024-01-24T23:27:32.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2172070481}) 2024-01-24T23:27:32.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2305314582}) 2024-01-24T23:27:32.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2756558136}) 2024-01-24T23:27:32.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2305314582}) 2024-01-24T23:27:32.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2427270763}) 2024-01-24T23:27:32.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2305314582}) 2024-01-24T23:27:32.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1544907742}) 2024-01-24T23:27:32.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2305314582}) 2024-01-24T23:27:32.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1386051876}) 2024-01-24T23:27:32.913 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.913 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2305314582}) 2024-01-24T23:27:32.913 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=217157257}) 2024-01-24T23:27:32.919 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.920 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=192083889}) 2024-01-24T23:27:32.920 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3251039194}) 2024-01-24T23:27:32.920 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.906+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.920 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=192083889}) 2024-01-24T23:27:32.920 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1281680056}) 2024-01-24T23:27:32.920 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.920 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=192083889}) 2024-01-24T23:27:32.920 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2201651583}) 2024-01-24T23:27:32.920 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.920 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=192083889}) 2024-01-24T23:27:32.920 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1311941431}) 2024-01-24T23:27:32.920 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.920 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=192083889}) 2024-01-24T23:27:32.921 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1800760022}) 2024-01-24T23:27:32.921 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.921 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=192083889}) 2024-01-24T23:27:32.921 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=151471630}) 2024-01-24T23:27:32.921 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.921 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=192083889}) 2024-01-24T23:27:32.921 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1281680056}) 2024-01-24T23:27:32.921 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.921 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=192083889}) 2024-01-24T23:27:32.921 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3416070285}) 2024-01-24T23:27:32.921 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.921 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3663071374}) 2024-01-24T23:27:32.921 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3722029271}) 2024-01-24T23:27:32.921 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.922 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3663071374}) 2024-01-24T23:27:32.922 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3669396874}) 2024-01-24T23:27:32.922 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.922 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3663071374}) 2024-01-24T23:27:32.922 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2472543252}) 2024-01-24T23:27:32.922 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.922 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3663071374}) 2024-01-24T23:27:32.922 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1284913717}) 2024-01-24T23:27:32.922 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.922 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3663071374}) 2024-01-24T23:27:32.922 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2773292366}) 2024-01-24T23:27:32.922 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.922 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3663071374}) 2024-01-24T23:27:32.922 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2988852751}) 2024-01-24T23:27:32.923 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.923 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3663071374}) 2024-01-24T23:27:32.923 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3669396874}) 2024-01-24T23:27:32.923 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.923 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3663071374}) 2024-01-24T23:27:32.923 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.910+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=27801250}) 2024-01-24T23:27:32.923 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.914+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.923 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.914+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=13,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=48} crc {logm=909421545,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=1656278728,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=230294024,osdmap=2962251394}) 2024-01-24T23:27:32.923 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.914+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=13,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=48} crc {logm=921077318,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=1645637050,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=230294024,osdmap=23492789}) 2024-01-24T23:27:32.923 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.914+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.923 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.914+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=13,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=48} crc {logm=909421545,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=1656278728,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=230294024,osdmap=2962251394}) 2024-01-24T23:27:32.923 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.914+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=13,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=49} crc {logm=627022630,mdsmap=2441318933,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=1645637051,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=230294024,osdmap=2353074219}) 2024-01-24T23:27:32.923 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.914+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.914+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=13,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=48} crc {logm=909421545,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=1656278728,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=230294024,osdmap=2962251394}) 2024-01-24T23:27:32.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.914+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=14,mdsmap=3,mgr=7,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=50} crc {logm=1729080216,mdsmap=3984425268,mgr=1613570594,mgr_metadata=2479342215,mgrstat=1656278728,mon_config_key=4121867429,monmap=701157023,osd_metadata=3543209436,osd_pg_creating=230294024,osdmap=436845197}) 2024-01-24T23:27:32.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.914+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.914+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=13,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=48} crc {logm=909421545,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=1656278728,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=230294024,osdmap=2962251394}) 2024-01-24T23:27:32.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.914+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=14,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=7,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=49} crc {logm=128658910,mdsmap=3984425268,mgr=3539714768,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=1656278728,mon_config_key=4207563763,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=230294024,osdmap=2659341671}) 2024-01-24T23:27:32.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.914+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.914+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=13,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=48} crc {logm=909421545,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=1656278728,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=230294024,osdmap=2962251394}) 2024-01-24T23:27:32.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.914+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=14,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=48} crc {logm=4121370656,mdsmap=3984425268,mgr=1117054149,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=1656278728,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=230294024,osdmap=434300429}) 2024-01-24T23:27:32.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.914+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.914+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=13,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=48} crc {logm=909421545,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=1656278728,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=230294024,osdmap=2962251394}) 2024-01-24T23:27:32.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.914+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=13,mdsmap=2,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=8,monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=52} crc {logm=4146203794,mdsmap=3140514009,mgr=2746627711,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=4087002730,mon_config_key=4121867429,monmap=701157023,osd_metadata=2031414162,osd_pg_creating=230294024,osdmap=2379605403}) 2024-01-24T23:27:32.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.914+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.914+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=13,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=48} crc {logm=909421545,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=1656278728,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=230294024,osdmap=2962251394}) 2024-01-24T23:27:32.925 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.914+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=13,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=49} crc {logm=627022630,mdsmap=2441318933,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=1645637051,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=230294024,osdmap=2353074219}) 2024-01-24T23:27:32.925 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.914+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.925 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.914+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=13,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=48} crc {logm=909421545,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=1656278728,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=230294024,osdmap=2962251394}) 2024-01-24T23:27:32.925 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.914+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=13,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=7,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=49} crc {logm=319614729,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=1656278728,mon_config_key=3302197801,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=230294024,osdmap=845656969}) 2024-01-24T23:27:32.929 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.918+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.930 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.918+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=931682065}) 2024-01-24T23:27:32.930 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.918+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3706212198}) 2024-01-24T23:27:32.930 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.918+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.930 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.918+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=931682065}) 2024-01-24T23:27:32.930 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.918+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1997695690}) 2024-01-24T23:27:32.930 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.918+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.930 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.918+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=931682065}) 2024-01-24T23:27:32.930 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.918+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2041913627}) 2024-01-24T23:27:32.930 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.918+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.930 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.918+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=931682065}) 2024-01-24T23:27:32.930 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.918+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1763404246}) 2024-01-24T23:27:32.930 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.918+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.930 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.918+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=931682065}) 2024-01-24T23:27:32.931 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.918+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3962970512}) 2024-01-24T23:27:32.931 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.918+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.931 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.918+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=931682065}) 2024-01-24T23:27:32.931 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.918+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1051093820}) 2024-01-24T23:27:32.931 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.918+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.931 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.918+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=931682065}) 2024-01-24T23:27:32.931 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.918+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1997695690}) 2024-01-24T23:27:32.931 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.918+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.931 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.918+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=931682065}) 2024-01-24T23:27:32.931 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.918+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=158195016}) 2024-01-24T23:27:32.931 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.922+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.931 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.922+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1191992900}) 2024-01-24T23:27:32.931 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.922+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4279107934}) 2024-01-24T23:27:32.932 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.922+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.932 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.922+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1191992900}) 2024-01-24T23:27:32.932 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.922+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=825306112}) 2024-01-24T23:27:32.932 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.922+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.932 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.922+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1191992900}) 2024-01-24T23:27:32.932 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.922+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=893379361}) 2024-01-24T23:27:32.932 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.922+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.932 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.922+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1191992900}) 2024-01-24T23:27:32.932 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.922+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=4133914016}) 2024-01-24T23:27:32.932 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.922+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.932 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.922+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1191992900}) 2024-01-24T23:27:32.932 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.922+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2954058473}) 2024-01-24T23:27:32.932 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.922+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.933 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.922+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1191992900}) 2024-01-24T23:27:32.933 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.922+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2701164464}) 2024-01-24T23:27:32.933 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.922+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.933 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.922+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1191992900}) 2024-01-24T23:27:32.933 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.922+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=825306112}) 2024-01-24T23:27:32.933 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.922+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.933 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.922+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1191992900}) 2024-01-24T23:27:32.933 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.922+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1679697603}) 2024-01-24T23:27:32.933 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.926+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.933 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.926+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=79} crc {osdmap=982022145}) 2024-01-24T23:27:32.933 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.926+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=76} crc {osdmap=3907806813}) 2024-01-24T23:27:32.933 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.926+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.933 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.926+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=79} crc {osdmap=982022145}) 2024-01-24T23:27:32.933 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.926+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=76} crc {osdmap=2392692014}) 2024-01-24T23:27:32.934 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.926+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.934 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.926+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=79} crc {osdmap=982022145}) 2024-01-24T23:27:32.934 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.926+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=77} crc {osdmap=4210527849}) 2024-01-24T23:27:32.934 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.926+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.934 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.926+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=79} crc {osdmap=982022145}) 2024-01-24T23:27:32.934 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.926+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=81} crc {osdmap=1209143915}) 2024-01-24T23:27:32.934 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.926+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.934 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.926+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=79} crc {osdmap=982022145}) 2024-01-24T23:27:32.934 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.926+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=80} crc {osdmap=1783011676}) 2024-01-24T23:27:32.934 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.926+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.934 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.926+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=79} crc {osdmap=982022145}) 2024-01-24T23:27:32.934 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.926+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=79} crc {osdmap=2236277110}) 2024-01-24T23:27:32.934 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.926+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.934 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.926+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=79} crc {osdmap=982022145}) 2024-01-24T23:27:32.935 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.926+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=76} crc {osdmap=2392692014}) 2024-01-24T23:27:32.935 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.926+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:27:32.935 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.926+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=79} crc {osdmap=982022145}) 2024-01-24T23:27:32.935 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:27:32.926+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=79} crc {osdmap=2007280475}) 2024-01-24T23:27:33.040 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.038+0000 7fd98e7fc640 1 -- 172.21.15.132:0/3673063451 <== mon.1 v1:172.21.15.132:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h6"}]=0 removed item id -34 name 'h6' from crush map v218) v1 ==== 118+0+0 (unknown 791081437 0 0) 0x7fd988012f10 con 0x7fd998136cb0 2024-01-24T23:27:33.040 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.038+0000 7fd98e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h6"}] 2024-01-24T23:27:33.040 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.038+0000 7fd98e7fc640 10 monclient: _finish_command 2 = system:0 removed item id -34 name 'h6' from crush map 2024-01-24T23:27:33.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.166+0000 7fd9a0165640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h6"}] 2024-01-24T23:27:33.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.166+0000 7fd9a0165640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h6"}] 2024-01-24T23:27:33.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.166+0000 7fd9a0165640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:27:33.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.166+0000 7fd9a0165640 1 -- 172.21.15.132:0/3673063451 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd crush rm", "name": "h6"} v 0) v1 -- 0x7fd9800010a0 con 0x7fd998136cb0 2024-01-24T23:27:33.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.170+0000 7fd98e7fc640 1 -- 172.21.15.132:0/3673063451 <== mon.1 v1:172.21.15.132:6789/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h6"}]=0 device 'h6' does not appear in the crush map v218) v1 ==== 118+0+0 (unknown 1237559737 0 0) 0x7fd988014690 con 0x7fd998136cb0 2024-01-24T23:27:33.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.170+0000 7fd98e7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h6"}] 2024-01-24T23:27:33.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.170+0000 7fd98e7fc640 10 monclient: _finish_command 3 = system:0 device 'h6' does not appear in the crush map 2024-01-24T23:27:33.170 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h6' does not appear in the crush map 2024-01-24T23:27:33.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.170+0000 7fd9a0165640 1 -- 172.21.15.132:0/3673063451 >> v1:172.21.15.132:6812/1835401922 conn(0x7fd97c037820 legacy=0x7fd97c039ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:33.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.170+0000 7fd9a0165640 10 monclient: shutdown 2024-01-24T23:27:33.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.170+0000 7fd9a0165640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:33.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.170+0000 7fd9a0165640 1 -- 172.21.15.132:0/3673063451 >> v1:172.21.15.132:6789/0 conn(0x7fd998136cb0 legacy=0x7fd9981d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:33.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.170+0000 7fd9a0165640 1 -- 172.21.15.132:0/3673063451 shutdown_connections 2024-01-24T23:27:33.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.170+0000 7fd9a0165640 1 -- 172.21.15.132:0/3673063451 >> 172.21.15.132:0/3673063451 conn(0x7fd9980a9c10 msgr2=0x7fd998066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:33.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.170+0000 7fd9a0165640 1 -- 172.21.15.132:0/3673063451 shutdown_connections 2024-01-24T23:27:33.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.170+0000 7fd9a0165640 1 -- 172.21.15.132:0/3673063451 wait complete. 2024-01-24T23:27:33.188 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:33.188 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h7 2024-01-24T23:27:33.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.302+0000 7effe4d2b640 10 monclient: get_monmap_and_config 2024-01-24T23:27:33.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.302+0000 7effe4d2b640 10 monclient: build_initial_monmap 2024-01-24T23:27:33.305 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.302+0000 7effe4d2b640 10 monclient: monmap: 2024-01-24T23:27:33.305 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:33.305 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:33.305 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:33.306862+0000 2024-01-24T23:27:33.305 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:33.306862+0000 2024-01-24T23:27:33.305 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:33.305 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:33.305 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:33.306 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:33.306 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:33.306 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:33.306 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:33.306 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:33.306 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:33.306 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:33.306 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:33.306 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:33.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.302+0000 7effe4d2b640 1 Processor -- start 2024-01-24T23:27:33.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.302+0000 7effe4d2b640 1 -- start start 2024-01-24T23:27:33.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.302+0000 7effe4d2b640 10 monclient: init 2024-01-24T23:27:33.312 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.302+0000 7effe4d2b640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:33.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.302+0000 7effe4d2b640 10 monclient: _add_conns ranks=[6,1,8,0,4,7,5,3,2] 2024-01-24T23:27:33.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.302+0000 7effe4d2b640 10 monclient(hunting): picked mon.noname-d con 0x7effe009ff20 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:33.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.302+0000 7effe4d2b640 10 monclient(hunting): picked mon.noname-f con 0x7effe009f9c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:33.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.302+0000 7effe4d2b640 10 monclient(hunting): picked mon.noname-e con 0x7effe013b2c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:33.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.302+0000 7effe4d2b640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe0064700 con 0x7effe009f9c0 2024-01-24T23:27:33.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.302+0000 7effe4d2b640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe009e760 con 0x7effe009ff20 2024-01-24T23:27:33.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.302+0000 7effe4d2b640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe00a1160 con 0x7effe013b2c0 2024-01-24T23:27:33.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.302+0000 7effe4d2b640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:33.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.302+0000 7effe4d2b640 10 monclient(hunting): authenticate will time out at 1340.631494s 2024-01-24T23:27:33.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.302+0000 7effddd9b640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7effe009f9c0 0x7effe0139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:56368/0 (socket says 172.21.15.132:56368) 2024-01-24T23:27:33.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.302+0000 7effddd9b640 1 -- 172.21.15.132:0/1000769970 learned_addr learned my addr 172.21.15.132:0/1000769970 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:33.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 1 -- 172.21.15.132:0/1000769970 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941381015 0 0) 0x7effe0064700 con 0x7effe009f9c0 2024-01-24T23:27:33.313 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 10 monclient(hunting): _negotiate 2024-01-24T23:27:33.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:33.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:33.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 10 monclient(hunting): my global_id is 124192 2024-01-24T23:27:33.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 1 -- 172.21.15.132:0/1000769970 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effcc003530 con 0x7effe009f9c0 2024-01-24T23:27:33.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 1 -- 172.21.15.132:0/1000769970 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2902853885 0 0) 0x7effcc003530 con 0x7effe009f9c0 2024-01-24T23:27:33.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 1 -- 172.21.15.132:0/1000769970 >> v1:172.21.15.115:6793/0 conn(0x7effe013b2c0 legacy=0x7effe013d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:33.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 1 -- 172.21.15.132:0/1000769970 >> v1:172.21.15.115:6792/0 conn(0x7effe009ff20 legacy=0x7effe00a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:33.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 10 monclient: _finish_hunting 0 2024-01-24T23:27:33.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 1 monclient: found mon.noname-f 2024-01-24T23:27:33.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:33.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:27:33.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 1 -- 172.21.15.132:0/1000769970 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe00a0a10 con 0x7effe009f9c0 2024-01-24T23:27:33.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 10 monclient: _finish_auth 0 2024-01-24T23:27:33.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 10 monclient: _check_auth_tickets 2024-01-24T23:27:33.314 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:33.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 5 monclient: authenticate success, global_id 124192 2024-01-24T23:27:33.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 1 -- 172.21.15.132:0/1000769970 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7effd00017c0 con 0x7effe009f9c0 2024-01-24T23:27:33.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:33.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:33.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:27:33.315 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 10 monclient: dump: 2024-01-24T23:27:33.315 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:33.315 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:33.315 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:33.315 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:33.315 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:33.315 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:33.315 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:33.315 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:33.316 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:33.316 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:33.316 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:33.316 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:33.316 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:33.316 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:33.316 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:33.316 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:33.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:33.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 1 -- 172.21.15.132:0/1000769970 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7effd0001ae0 con 0x7effe009f9c0 2024-01-24T23:27:33.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:33.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:33.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 4 set_mon_vals no callback set 2024-01-24T23:27:33.316 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 1 -- 172.21.15.132:0/1000769970 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7effd0004bf0 con 0x7effe009f9c0 2024-01-24T23:27:33.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:33.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:27:33.317 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effdd59a640 10 monclient: dump: 2024-01-24T23:27:33.317 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:33.317 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:33.317 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:33.317 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:33.317 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:33.317 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:33.317 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:33.317 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:33.317 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:33.317 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:33.317 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:33.317 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:33.318 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:33.318 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:33.318 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:33.318 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:33.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 10 monclient: shutdown 2024-01-24T23:27:33.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:33.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 1 -- 172.21.15.132:0/1000769970 >> v1:172.21.15.132:6789/0 conn(0x7effe009f9c0 legacy=0x7effe0139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:33.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 1 -- 172.21.15.132:0/1000769970 shutdown_connections 2024-01-24T23:27:33.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 1 -- 172.21.15.132:0/1000769970 >> 172.21.15.132:0/1000769970 conn(0x7effe00a5a10 msgr2=0x7effe00a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:33.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 1 -- 172.21.15.132:0/1000769970 shutdown_connections 2024-01-24T23:27:33.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 1 -- 172.21.15.132:0/1000769970 wait complete. 2024-01-24T23:27:33.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 10 monclient: build_initial_monmap 2024-01-24T23:27:33.318 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 10 monclient: monmap: 2024-01-24T23:27:33.318 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:33.318 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:33.318 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:33.319 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:33.319 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:33.319 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:33.319 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:33.319 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:33.319 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:33.319 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:33.319 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:33.319 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:33.319 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:33.319 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:33.319 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:33.319 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:33.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 1 Processor -- start 2024-01-24T23:27:33.319 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 1 -- start start 2024-01-24T23:27:33.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 10 monclient: init 2024-01-24T23:27:33.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:33.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 10 monclient: _add_conns ranks=[8,5,6,0,3,1,4,7,2] 2024-01-24T23:27:33.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 10 monclient(hunting): picked mon.noname-e con 0x7effe009f9c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:33.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 10 monclient(hunting): picked mon.noname-h con 0x7effe009ff20 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:33.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 10 monclient(hunting): picked mon.noname-d con 0x7effe013b2c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:33.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe0071680 con 0x7effe009ff20 2024-01-24T23:27:33.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe011c710 con 0x7effe013b2c0 2024-01-24T23:27:33.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe011ee90 con 0x7effe009f9c0 2024-01-24T23:27:33.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:33.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 10 monclient(hunting): authenticate will time out at 1340.634835s 2024-01-24T23:27:33.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effddd9b640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7effe009ff20 0x7effe011bd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:32848/0 (socket says 172.21.15.132:32848) 2024-01-24T23:27:33.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effddd9b640 1 -- 172.21.15.132:0/624668441 learned_addr learned my addr 172.21.15.132:0/624668441 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:33.320 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 1 -- 172.21.15.132:0/624668441 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 394514399 0 0) 0x7effe0071680 con 0x7effe009ff20 2024-01-24T23:27:33.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:33.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:33.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:33.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 10 monclient(hunting): my global_id is 6665 2024-01-24T23:27:33.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 1 -- 172.21.15.132:0/624668441 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effc0003530 con 0x7effe009ff20 2024-01-24T23:27:33.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 1 -- 172.21.15.132:0/624668441 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2928449981 0 0) 0x7effc0003530 con 0x7effe009ff20 2024-01-24T23:27:33.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 1 -- 172.21.15.132:0/624668441 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7effe0071680 con 0x7effe009ff20 2024-01-24T23:27:33.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 1 -- 172.21.15.132:0/624668441 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7effd0001780 con 0x7effe009ff20 2024-01-24T23:27:33.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:33.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:27:33.321 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 10 monclient(hunting): dump: 2024-01-24T23:27:33.321 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:33.321 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:33.321 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:33.321 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:33.321 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:33.322 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:33.322 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:33.322 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:33.322 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:33.322 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:33.322 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:33.322 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:33.322 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:33.322 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:33.322 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:33.322 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:33.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:33.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 1 -- 172.21.15.132:0/624668441 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 221006328 0 0) 0x7effe011ee90 con 0x7effe009f9c0 2024-01-24T23:27:33.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:33.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:33.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:33.322 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 10 monclient(hunting): my global_id is 85814 2024-01-24T23:27:33.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 1 -- 172.21.15.132:0/624668441 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effc0003530 con 0x7effe009f9c0 2024-01-24T23:27:33.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 1 -- 172.21.15.132:0/624668441 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850561603 0 0) 0x7effe011c710 con 0x7effe013b2c0 2024-01-24T23:27:33.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:33.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:33.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:33.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 10 monclient(hunting): my global_id is 6792 2024-01-24T23:27:33.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 1 -- 172.21.15.132:0/624668441 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effe011ee90 con 0x7effe013b2c0 2024-01-24T23:27:33.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 1 -- 172.21.15.132:0/624668441 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 462213188 0 0) 0x7effe0071680 con 0x7effe009ff20 2024-01-24T23:27:33.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 1 -- 172.21.15.132:0/624668441 >> v1:172.21.15.115:6793/0 conn(0x7effe009f9c0 legacy=0x7effe011b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:33.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 1 -- 172.21.15.132:0/624668441 >> v1:172.21.15.115:6792/0 conn(0x7effe013b2c0 legacy=0x7effe01d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:33.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:27:33.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 1 monclient: found mon.h 2024-01-24T23:27:33.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:33.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:33.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 1 -- 172.21.15.132:0/624668441 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe011c430 con 0x7effe009ff20 2024-01-24T23:27:33.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 10 monclient: _finish_auth 0 2024-01-24T23:27:33.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:27:33.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:33.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 5 monclient: authenticate success, global_id 6665 2024-01-24T23:27:33.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 10 monclient: _renew_subs 2024-01-24T23:27:33.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:33.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 1 -- 172.21.15.132:0/624668441 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe0071960 con 0x7effe009ff20 2024-01-24T23:27:33.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 10 monclient: _renew_subs 2024-01-24T23:27:33.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:33.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effe4d2b640 1 -- 172.21.15.132:0/624668441 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7effe011ea70 con 0x7effe009ff20 2024-01-24T23:27:33.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 1 -- 172.21.15.132:0/624668441 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7effd0004c50 con 0x7effe009ff20 2024-01-24T23:27:33.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.306+0000 7effd6ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:33.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.310+0000 7effd6ffd640 1 -- 172.21.15.132:0/624668441 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7effd0001c50 con 0x7effe009ff20 2024-01-24T23:27:33.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.310+0000 7effd6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:33.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.310+0000 7effd6ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:33.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.310+0000 7effdcd99640 4 set_mon_vals no callback set 2024-01-24T23:27:33.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.310+0000 7effd6ffd640 10 monclient: dump: 2024-01-24T23:27:33.325 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:33.325 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:33.325 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:33.325 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:33.325 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:33.325 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:33.325 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:33.326 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:33.326 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:33.326 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:33.326 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:33.326 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:33.326 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:33.326 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:33.326 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:33.326 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:33.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.310+0000 7effd6ffd640 1 -- 172.21.15.132:0/624668441 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7effd000fdd0 con 0x7effe009ff20 2024-01-24T23:27:33.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.310+0000 7effe4d2b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:33.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.310+0000 7effe4d2b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:33.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.310+0000 7effe4d2b640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:33.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.310+0000 7effe4d2b640 1 -- 172.21.15.132:0/624668441 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc4005090 con 0x7effe009ff20 2024-01-24T23:27:33.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.310+0000 7effd6ffd640 1 -- 172.21.15.132:0/624668441 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(218..218 src has 1..218) v4 ==== 13118+0+0 (unknown 2016787575 0 0) 0x7effd0045fe0 con 0x7effe009ff20 2024-01-24T23:27:33.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.310+0000 7effde59c640 10 monclient: get_auth_request con 0x7effc00371e0 auth_method 0 2024-01-24T23:27:33.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.314+0000 7effd6ffd640 1 -- 172.21.15.132:0/624668441 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7effd0014c70 con 0x7effe009ff20 2024-01-24T23:27:33.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.314+0000 7effd6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:33.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.314+0000 7effd6ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:33.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.474+0000 7effe4d2b640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h7"}] 2024-01-24T23:27:33.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.474+0000 7effe4d2b640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h7"}] 2024-01-24T23:27:33.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.474+0000 7effe4d2b640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:33.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:33.474+0000 7effe4d2b640 1 -- 172.21.15.132:0/624668441 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h7"} v 0) v1 -- 0x7effc4005340 con 0x7effe009ff20 2024-01-24T23:27:33.811 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-01-24T23:27:33.812 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-01-24T23:27:33.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f47f7640 10 monclient: get_monmap_and_config 2024-01-24T23:27:33.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f47f7640 10 monclient: build_initial_monmap 2024-01-24T23:27:33.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f47f7640 10 monclient: monmap: 2024-01-24T23:27:33.939 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:33.939 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:33.939 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:33.940684+0000 2024-01-24T23:27:33.940 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:33.940684+0000 2024-01-24T23:27:33.940 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:33.940 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:33.940 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:33.940 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:33.940 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:33.940 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:33.940 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:33.940 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:33.941 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:33.941 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:33.941 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:33.941 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:33.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f47f7640 1 Processor -- start 2024-01-24T23:27:33.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f47f7640 1 -- start start 2024-01-24T23:27:33.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f47f7640 10 monclient: init 2024-01-24T23:27:33.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f47f7640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:33.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f47f7640 10 monclient: _add_conns ranks=[7,0,8,5,1,6,2,4,3] 2024-01-24T23:27:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f47f7640 10 monclient(hunting): picked mon.noname-i con 0x7f81ec133bf0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f47f7640 10 monclient(hunting): picked mon.noname-a con 0x7f81ec133fd0 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f47f7640 10 monclient(hunting): picked mon.noname-e con 0x7f81ec1343b0 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f47f7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ec04ed70 con 0x7f81ec133fd0 2024-01-24T23:27:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f47f7640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ec0a4b60 con 0x7f81ec133bf0 2024-01-24T23:27:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f47f7640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ec09ae50 con 0x7f81ec1343b0 2024-01-24T23:27:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f47f7640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f47f7640 10 monclient(hunting): authenticate will time out at 1332.863389s 2024-01-24T23:27:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f1d92640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f81ec133fd0 0x7f81ec13bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:41204/0 (socket says 172.21.15.115:41204) 2024-01-24T23:27:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f2d94640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f81ec1343b0 0x7f81ec13f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:46332/0 (socket says 172.21.15.115:46332) 2024-01-24T23:27:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f1d92640 1 -- 172.21.15.115:0/1830354675 learned_addr learned my addr 172.21.15.115:0/1830354675 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f1591640 1 -- 172.21.15.115:0/1830354675 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812058437 0 0) 0x7f81ec09ae50 con 0x7f81ec1343b0 2024-01-24T23:27:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f1591640 10 monclient(hunting): _negotiate 2024-01-24T23:27:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f1591640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f1591640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f1591640 10 monclient(hunting): my global_id is 85823 2024-01-24T23:27:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f1591640 1 -- 172.21.15.115:0/1830354675 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81d4003530 con 0x7f81ec1343b0 2024-01-24T23:27:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f1591640 1 -- 172.21.15.115:0/1830354675 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3857093352 0 0) 0x7f81ec04ed70 con 0x7f81ec133fd0 2024-01-24T23:27:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f1591640 10 monclient(hunting): _negotiate 2024-01-24T23:27:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f1591640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f1591640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f1591640 10 monclient(hunting): my global_id is 94950 2024-01-24T23:27:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.936+0000 7f81f1591640 1 -- 172.21.15.115:0/1830354675 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81ec09ae50 con 0x7f81ec133fd0 2024-01-24T23:27:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f1591640 1 -- 172.21.15.115:0/1830354675 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2398829950 0 0) 0x7f81ec09ae50 con 0x7f81ec133fd0 2024-01-24T23:27:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f1591640 1 -- 172.21.15.115:0/1830354675 >> v1:172.21.15.115:6793/0 conn(0x7f81ec1343b0 legacy=0x7f81ec13f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f1591640 1 -- 172.21.15.115:0/1830354675 >> v1:172.21.15.132:6792/0 conn(0x7f81ec133bf0 legacy=0x7f81ec09bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f1591640 10 monclient: _finish_hunting 0 2024-01-24T23:27:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f1591640 1 monclient: found mon.noname-a 2024-01-24T23:27:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f1591640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f1591640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:27:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f1591640 1 -- 172.21.15.115:0/1830354675 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ec09b130 con 0x7f81ec133fd0 2024-01-24T23:27:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f1591640 10 monclient: _finish_auth 0 2024-01-24T23:27:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f1591640 10 monclient: _check_auth_tickets 2024-01-24T23:27:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f1591640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f1591640 1 -- 172.21.15.115:0/1830354675 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f81e00017a0 con 0x7f81ec133fd0 2024-01-24T23:27:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 5 monclient: authenticate success, global_id 94950 2024-01-24T23:27:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f1591640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f1591640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:27:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f1591640 10 monclient: dump: 2024-01-24T23:27:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:33.953 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:33.953 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:33.953 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:33.953 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:33.953 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:33.953 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:33.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:33.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f1591640 1 -- 172.21.15.115:0/1830354675 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f81e0001a40 con 0x7f81ec133fd0 2024-01-24T23:27:33.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f1591640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:33.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:33.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f1591640 1 -- 172.21.15.115:0/1830354675 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f81e0004bf0 con 0x7f81ec133fd0 2024-01-24T23:27:33.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 4 set_mon_vals no callback set 2024-01-24T23:27:33.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f1591640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:33.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f1591640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:27:33.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f1591640 10 monclient: dump: 2024-01-24T23:27:33.954 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:33.954 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:33.954 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:33.954 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:33.954 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:33.954 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:33.954 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:33.954 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:33.955 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 10 monclient: shutdown 2024-01-24T23:27:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 1 -- 172.21.15.115:0/1830354675 >> v1:172.21.15.115:6789/0 conn(0x7f81ec133fd0 legacy=0x7f81ec13bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 1 -- 172.21.15.115:0/1830354675 shutdown_connections 2024-01-24T23:27:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 1 -- 172.21.15.115:0/1830354675 >> 172.21.15.115:0/1830354675 conn(0x7f81ec0662f0 msgr2=0x7f81ec066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 1 -- 172.21.15.115:0/1830354675 shutdown_connections 2024-01-24T23:27:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 1 -- 172.21.15.115:0/1830354675 wait complete. 2024-01-24T23:27:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 10 monclient: build_initial_monmap 2024-01-24T23:27:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 10 monclient: monmap: 2024-01-24T23:27:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:33.958 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:33.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 1 Processor -- start 2024-01-24T23:27:33.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 1 -- start start 2024-01-24T23:27:33.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 10 monclient: init 2024-01-24T23:27:33.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:33.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 10 monclient: _add_conns ranks=[2,7,1,5,4,8,0,3,6] 2024-01-24T23:27:33.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 10 monclient(hunting): picked mon.noname-b con 0x7f81ec133bf0 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:33.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 10 monclient(hunting): picked mon.noname-i con 0x7f81ec133fd0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:33.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 10 monclient(hunting): picked mon.noname-f con 0x7f81ec1343b0 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:33.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ec11ffe0 con 0x7f81ec1343b0 2024-01-24T23:27:33.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ec0a5990 con 0x7f81ec133bf0 2024-01-24T23:27:33.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ec11f970 con 0x7f81ec133fd0 2024-01-24T23:27:33.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:33.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 10 monclient(hunting): authenticate will time out at 1332.866989s 2024-01-24T23:27:33.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f2593640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f81ec133bf0 0x7f81ec132ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:40320/0 (socket says 172.21.15.115:40320) 2024-01-24T23:27:33.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f2593640 1 -- 172.21.15.115:0/2960321477 learned_addr learned my addr 172.21.15.115:0/2960321477 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:33.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 1 -- 172.21.15.115:0/2960321477 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3108962361 0 0) 0x7f81ec0a5990 con 0x7f81ec133bf0 2024-01-24T23:27:33.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:33.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:33.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:33.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 10 monclient(hunting): my global_id is 124202 2024-01-24T23:27:33.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 1 -- 172.21.15.115:0/2960321477 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81d8003530 con 0x7f81ec133bf0 2024-01-24T23:27:33.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 1 -- 172.21.15.115:0/2960321477 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3135914304 0 0) 0x7f81d8003530 con 0x7f81ec133bf0 2024-01-24T23:27:33.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 1 -- 172.21.15.115:0/2960321477 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f81ec0a5990 con 0x7f81ec133bf0 2024-01-24T23:27:33.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 1 -- 172.21.15.115:0/2960321477 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f81e4001e40 con 0x7f81ec133bf0 2024-01-24T23:27:33.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:33.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:27:33.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 10 monclient(hunting): dump: 2024-01-24T23:27:33.960 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:33.960 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:33.960 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:33.960 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:33.960 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:33.960 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:33.960 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:33.960 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:33.961 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:33.961 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:33.961 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:33.961 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:33.961 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:33.961 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:33.961 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:33.961 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:33.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:33.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 1 -- 172.21.15.115:0/2960321477 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894607155 0 0) 0x7f81ec11f970 con 0x7f81ec133fd0 2024-01-24T23:27:33.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:33.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:33.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:33.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 10 monclient(hunting): my global_id is 124216 2024-01-24T23:27:33.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 1 -- 172.21.15.115:0/2960321477 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81d8003530 con 0x7f81ec133fd0 2024-01-24T23:27:33.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 1 -- 172.21.15.115:0/2960321477 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1092494480 0 0) 0x7f81ec11ffe0 con 0x7f81ec1343b0 2024-01-24T23:27:33.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:33.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:33.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:33.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 10 monclient(hunting): my global_id is 124201 2024-01-24T23:27:33.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 1 -- 172.21.15.115:0/2960321477 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81ec11f970 con 0x7f81ec1343b0 2024-01-24T23:27:33.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 1 -- 172.21.15.115:0/2960321477 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 986291356 0 0) 0x7f81ec0a5990 con 0x7f81ec133bf0 2024-01-24T23:27:33.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 1 -- 172.21.15.115:0/2960321477 >> v1:172.21.15.132:6792/0 conn(0x7f81ec133fd0 legacy=0x7f81ec1d0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:33.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 1 -- 172.21.15.115:0/2960321477 >> v1:172.21.15.132:6789/0 conn(0x7f81ec1343b0 legacy=0x7f81ec1d3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:33.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 10 monclient: _finish_hunting 0 2024-01-24T23:27:33.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 1 monclient: found mon.b 2024-01-24T23:27:33.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:33.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:27:33.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 1 -- 172.21.15.115:0/2960321477 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ec0a2410 con 0x7f81ec133bf0 2024-01-24T23:27:33.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 10 monclient: _finish_auth 0 2024-01-24T23:27:33.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 10 monclient: _check_auth_tickets 2024-01-24T23:27:33.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81eaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:33.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 5 monclient: authenticate success, global_id 124202 2024-01-24T23:27:33.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 10 monclient: _renew_subs 2024-01-24T23:27:33.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.940+0000 7f81f47f7640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:27:33.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.944+0000 7f81f47f7640 1 -- 172.21.15.115:0/2960321477 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81ec120880 con 0x7f81ec133bf0 2024-01-24T23:27:33.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.944+0000 7f81f47f7640 10 monclient: _renew_subs 2024-01-24T23:27:33.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.944+0000 7f81f47f7640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:27:33.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.944+0000 7f81f47f7640 1 -- 172.21.15.115:0/2960321477 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ec1cbbd0 con 0x7f81ec133bf0 2024-01-24T23:27:33.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.944+0000 7f81eaffd640 1 -- 172.21.15.115:0/2960321477 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f81e4003cd0 con 0x7f81ec133bf0 2024-01-24T23:27:33.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.944+0000 7f81eaffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:33.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.944+0000 7f81eaffd640 1 -- 172.21.15.115:0/2960321477 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f81e4004180 con 0x7f81ec133bf0 2024-01-24T23:27:33.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.944+0000 7f81eaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:33.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.944+0000 7f81eaffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:27:33.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.944+0000 7f81f0d90640 4 set_mon_vals no callback set 2024-01-24T23:27:33.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.944+0000 7f81eaffd640 10 monclient: dump: 2024-01-24T23:27:33.964 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:33.964 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:33.964 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:33.964 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:33.964 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:33.965 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:33.965 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:33.965 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:33.965 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:33.965 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:33.965 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:33.965 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:33.965 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:33.965 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:33.965 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:33.965 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:33.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.944+0000 7f81eaffd640 1 -- 172.21.15.115:0/2960321477 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f81e4004360 con 0x7f81ec133bf0 2024-01-24T23:27:33.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.944+0000 7f81f47f7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:33.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.944+0000 7f81f47f7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:33.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.944+0000 7f81f47f7640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:27:33.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.944+0000 7f81f47f7640 1 -- 172.21.15.115:0/2960321477 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81cc005090 con 0x7f81ec133bf0 2024-01-24T23:27:33.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.944+0000 7f81eaffd640 1 -- 172.21.15.115:0/2960321477 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(218..218 src has 1..218) v4 ==== 13118+0+0 (unknown 2016787575 0 0) 0x7f81e4046050 con 0x7f81ec133bf0 2024-01-24T23:27:33.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.944+0000 7f81f1d92640 10 monclient: get_auth_request con 0x7f81d8037230 auth_method 0 2024-01-24T23:27:33.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.948+0000 7f81eaffd640 1 -- 172.21.15.115:0/2960321477 <== mon.2 v1:172.21.15.115:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f81e4014e90 con 0x7f81ec133bf0 2024-01-24T23:27:33.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.948+0000 7f81eaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:33.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:33.948+0000 7f81eaffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:34.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.074+0000 7effd6ffd640 1 -- 172.21.15.132:0/624668441 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h7"}]=0 removed item id -37 name 'h7' from crush map v219) v1 ==== 118+0+0 (unknown 789332613 0 0) 0x7effd0013310 con 0x7effe009ff20 2024-01-24T23:27:34.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.074+0000 7effd6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h7"}] 2024-01-24T23:27:34.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.074+0000 7effd6ffd640 10 monclient: _finish_command 2 = system:0 removed item id -37 name 'h7' from crush map 2024-01-24T23:27:34.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.168+0000 7f81f47f7640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-01-24T23:27:34.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.168+0000 7f81f47f7640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-01-24T23:27:34.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.168+0000 7f81f47f7640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:27:34.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.168+0000 7f81f47f7640 1 -- 172.21.15.115:0/2960321477 --> v1:172.21.15.115:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f81cc0058c0 con 0x7f81ec133bf0 2024-01-24T23:27:34.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.168+0000 7f81eaffd640 1 -- 172.21.15.115:0/2960321477 <== mon.2 v1:172.21.15.115:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2744 (unknown 254746657 0 553866294) 0x7f81e4013300 con 0x7f81ec133bf0 2024-01-24T23:27:34.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.168+0000 7f81eaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-01-24T23:27:34.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.168+0000 7f81eaffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:34.171 INFO:teuthology.orchestra.run.smithi115.stdout:{"election_epoch":40,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":3,"features":{"quorum_con":"4540701272860131327","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","modified":"2024-01-24T23:20:20.557246Z","created":"2024-01-24T23:20:20.557246Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6790","nonce":0}]},"addr":"172.21.15.115:6790/0","public_addr":"172.21.15.115:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6790","nonce":0}]},"addr":"172.21.15.132:6790/0","public_addr":"172.21.15.132:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6791","nonce":0}]},"addr":"172.21.15.115:6791/0","public_addr":"172.21.15.115:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6791","nonce":0}]},"addr":"172.21.15.132:6791/0","public_addr":"172.21.15.132:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6792","nonce":0}]},"addr":"172.21.15.115:6792/0","public_addr":"172.21.15.115:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6792","nonce":0}]},"addr":"172.21.15.132:6792/0","public_addr":"172.21.15.132:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6793","nonce":0}]},"addr":"172.21.15.115:6793/0","public_addr":"172.21.15.115:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-01-24T23:27:34.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.172+0000 7f81f47f7640 1 -- 172.21.15.115:0/2960321477 >> v1:172.21.15.132:6812/1835401922 conn(0x7f81d8037230 legacy=0x7f81d80396f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.172+0000 7f81f47f7640 10 monclient: shutdown 2024-01-24T23:27:34.173 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.172+0000 7f81f47f7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:34.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.172+0000 7f81f47f7640 1 -- 172.21.15.115:0/2960321477 >> v1:172.21.15.115:6790/0 conn(0x7f81ec133bf0 legacy=0x7f81ec132ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.172+0000 7f81f47f7640 1 -- 172.21.15.115:0/2960321477 shutdown_connections 2024-01-24T23:27:34.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.172+0000 7f81f47f7640 1 -- 172.21.15.115:0/2960321477 >> 172.21.15.115:0/2960321477 conn(0x7f81ec0662f0 msgr2=0x7f81ec1be760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:34.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.172+0000 7f81f47f7640 1 -- 172.21.15.115:0/2960321477 shutdown_connections 2024-01-24T23:27:34.174 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.172+0000 7f81f47f7640 1 -- 172.21.15.115:0/2960321477 wait complete. 2024-01-24T23:27:34.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.178+0000 7effe4d2b640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h7"}] 2024-01-24T23:27:34.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.178+0000 7effe4d2b640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h7"}] 2024-01-24T23:27:34.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.178+0000 7effe4d2b640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:34.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.178+0000 7effe4d2b640 1 -- 172.21.15.132:0/624668441 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h7"} v 0) v1 -- 0x7effc4002220 con 0x7effe009ff20 2024-01-24T23:27:34.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.178+0000 7effd6ffd640 1 -- 172.21.15.132:0/624668441 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h7"}]=0 device 'h7' does not appear in the crush map v219) v1 ==== 118+0+0 (unknown 2326589629 0 0) 0x7effd0014a90 con 0x7effe009ff20 2024-01-24T23:27:34.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.178+0000 7effd6ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h7"}] 2024-01-24T23:27:34.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.178+0000 7effd6ffd640 10 monclient: _finish_command 3 = system:0 device 'h7' does not appear in the crush map 2024-01-24T23:27:34.182 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h7' does not appear in the crush map 2024-01-24T23:27:34.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.182+0000 7effe4d2b640 1 -- 172.21.15.132:0/624668441 >> v1:172.21.15.132:6812/1835401922 conn(0x7effc00371e0 legacy=0x7effc00396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.182+0000 7effe4d2b640 10 monclient: shutdown 2024-01-24T23:27:34.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.182+0000 7effe4d2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:34.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.182+0000 7effe4d2b640 1 -- 172.21.15.132:0/624668441 >> v1:172.21.15.132:6791/0 conn(0x7effe009ff20 legacy=0x7effe011bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.182+0000 7effe4d2b640 1 -- 172.21.15.132:0/624668441 shutdown_connections 2024-01-24T23:27:34.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.182+0000 7effe4d2b640 1 -- 172.21.15.132:0/624668441 >> 172.21.15.132:0/624668441 conn(0x7effe00a5a10 msgr2=0x7effe00a2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:34.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.182+0000 7effe4d2b640 1 -- 172.21.15.132:0/624668441 shutdown_connections 2024-01-24T23:27:34.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.182+0000 7effe4d2b640 1 -- 172.21.15.132:0/624668441 wait complete. 2024-01-24T23:27:34.189 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-01-24T23:27:34.189 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-01-24T23:27:34.189 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-01-24T23:27:34.189 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-01-24T23:27:34.198 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:34.198 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h8 2024-01-24T23:27:34.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.304+0000 7f043e46a640 10 monclient: get_monmap_and_config 2024-01-24T23:27:34.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.304+0000 7f043e46a640 10 monclient: build_initial_monmap 2024-01-24T23:27:34.309 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 10 monclient: monmap: 2024-01-24T23:27:34.309 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:34.309 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.310 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:34.310920+0000 2024-01-24T23:27:34.310 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:34.310920+0000 2024-01-24T23:27:34.310 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:34.310 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.310 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:34.310 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:34.310 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:34.310 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:34.310 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:34.310 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:34.310 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:34.310 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:34.310 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:34.310 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 1 Processor -- start 2024-01-24T23:27:34.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 1 -- start start 2024-01-24T23:27:34.319 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 10 monclient: init 2024-01-24T23:27:34.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:34.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 10 monclient: _add_conns ranks=[4,8,6,3,5,7,1,2,0] 2024-01-24T23:27:34.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 10 monclient(hunting): picked mon.noname-c con 0x7f04380662f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:34.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 10 monclient(hunting): picked mon.noname-e con 0x7f04381305c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:34.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 10 monclient(hunting): picked mon.noname-d con 0x7f043813a030 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:34.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f043804ed70 con 0x7f04380662f0 2024-01-24T23:27:34.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04380a2ff0 con 0x7f043813a030 2024-01-24T23:27:34.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04380666d0 con 0x7f04381305c0 2024-01-24T23:27:34.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:34.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 10 monclient(hunting): authenticate will time out at 1333.233582s 2024-01-24T23:27:34.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0437fff640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f04380662f0 0x7f0438071660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:57644/0 (socket says 172.21.15.115:57644) 2024-01-24T23:27:34.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0437fff640 1 -- 172.21.15.115:0/3828769818 learned_addr learned my addr 172.21.15.115:0/3828769818 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:34.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 1 -- 172.21.15.115:0/3828769818 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889420685 0 0) 0x7f04380666d0 con 0x7f04381305c0 2024-01-24T23:27:34.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 10 monclient(hunting): my global_id is 85832 2024-01-24T23:27:34.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 1 -- 172.21.15.115:0/3828769818 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0420003530 con 0x7f04381305c0 2024-01-24T23:27:34.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 1 -- 172.21.15.115:0/3828769818 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4052774007 0 0) 0x7f04380a2ff0 con 0x7f043813a030 2024-01-24T23:27:34.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 10 monclient(hunting): my global_id is 6801 2024-01-24T23:27:34.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 1 -- 172.21.15.115:0/3828769818 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04380666d0 con 0x7f043813a030 2024-01-24T23:27:34.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 1 -- 172.21.15.115:0/3828769818 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1108387194 0 0) 0x7f043804ed70 con 0x7f04380662f0 2024-01-24T23:27:34.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 10 monclient(hunting): my global_id is 94819 2024-01-24T23:27:34.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 1 -- 172.21.15.115:0/3828769818 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04380a2ff0 con 0x7f04380662f0 2024-01-24T23:27:34.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 1 -- 172.21.15.115:0/3828769818 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 977195286 0 0) 0x7f0420003530 con 0x7f04381305c0 2024-01-24T23:27:34.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 1 -- 172.21.15.115:0/3828769818 >> v1:172.21.15.115:6792/0 conn(0x7f043813a030 legacy=0x7f043813c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 1 -- 172.21.15.115:0/3828769818 >> v1:172.21.15.115:6791/0 conn(0x7f04380662f0 legacy=0x7f0438071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:27:34.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 1 monclient: found mon.noname-e 2024-01-24T23:27:34.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:34.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:27:34.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 1 -- 172.21.15.115:0/3828769818 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043806ebe0 con 0x7f04381305c0 2024-01-24T23:27:34.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 10 monclient: _finish_auth 0 2024-01-24T23:27:34.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:27:34.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:34.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 1 -- 172.21.15.115:0/3828769818 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f042c0017c0 con 0x7f04381305c0 2024-01-24T23:27:34.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 5 monclient: authenticate success, global_id 85832 2024-01-24T23:27:34.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:34.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:27:34.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 10 monclient: dump: 2024-01-24T23:27:34.323 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.323 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.323 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.323 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.323 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.323 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.323 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.323 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.323 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.323 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.323 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.323 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.324 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.324 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.324 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.324 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:34.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 1 -- 172.21.15.115:0/3828769818 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f042c0019e0 con 0x7f04381305c0 2024-01-24T23:27:34.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:34.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:34.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 4 set_mon_vals no callback set 2024-01-24T23:27:34.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 1 -- 172.21.15.115:0/3828769818 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f042c003960 con 0x7f04381305c0 2024-01-24T23:27:34.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:27:34.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f0436ffd640 10 monclient: dump: 2024-01-24T23:27:34.324 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.325 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.325 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.325 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.325 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.325 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.325 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.325 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.325 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.325 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.325 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.325 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.325 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.325 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.325 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.326 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 10 monclient: shutdown 2024-01-24T23:27:34.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:34.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 1 -- 172.21.15.115:0/3828769818 >> v1:172.21.15.115:6793/0 conn(0x7f04381305c0 legacy=0x7f0438138910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 1 -- 172.21.15.115:0/3828769818 shutdown_connections 2024-01-24T23:27:34.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 1 -- 172.21.15.115:0/3828769818 >> 172.21.15.115:0/3828769818 conn(0x7f04380a5d40 msgr2=0x7f04380a2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:34.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 1 -- 172.21.15.115:0/3828769818 shutdown_connections 2024-01-24T23:27:34.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 1 -- 172.21.15.115:0/3828769818 wait complete. 2024-01-24T23:27:34.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 10 monclient: build_initial_monmap 2024-01-24T23:27:34.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.308+0000 7f043e46a640 10 monclient: monmap: 2024-01-24T23:27:34.326 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:34.326 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:34.326 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:34.326 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:34.327 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:34.327 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:34.327 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:34.327 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:34.327 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:34.327 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:34.327 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:34.327 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:34.327 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:34.327 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:34.327 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:34.327 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f043e46a640 1 Processor -- start 2024-01-24T23:27:34.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f043e46a640 1 -- start start 2024-01-24T23:27:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f043e46a640 10 monclient: init 2024-01-24T23:27:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f043e46a640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f043e46a640 10 monclient: _add_conns ranks=[7,2,1,8,0,4,6,3,5] 2024-01-24T23:27:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f043e46a640 10 monclient(hunting): picked mon.noname-i con 0x7f04380662f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f043e46a640 10 monclient(hunting): picked mon.noname-b con 0x7f04381305c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f043e46a640 10 monclient(hunting): picked mon.noname-f con 0x7f043813a030 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f043e46a640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f043806f390 con 0x7f043813a030 2024-01-24T23:27:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f043e46a640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0438131fc0 con 0x7f04381305c0 2024-01-24T23:27:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f043e46a640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f043806ee40 con 0x7f04380662f0 2024-01-24T23:27:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f043e46a640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f043e46a640 10 monclient(hunting): authenticate will time out at 1333.236719s 2024-01-24T23:27:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f04377fe640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f04381305c0 0x7f04381318b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:40328/0 (socket says 172.21.15.115:40328) 2024-01-24T23:27:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f04377fe640 1 -- 172.21.15.115:0/1350445504 learned_addr learned my addr 172.21.15.115:0/1350445504 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351375105 0 0) 0x7f0438131fc0 con 0x7f04381305c0 2024-01-24T23:27:34.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient(hunting): my global_id is 124211 2024-01-24T23:27:34.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0424003530 con 0x7f04381305c0 2024-01-24T23:27:34.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2370140328 0 0) 0x7f043806f390 con 0x7f043813a030 2024-01-24T23:27:34.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient(hunting): my global_id is 124210 2024-01-24T23:27:34.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0438131fc0 con 0x7f043813a030 2024-01-24T23:27:34.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4462003 0 0) 0x7f0424003530 con 0x7f04381305c0 2024-01-24T23:27:34.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f043806f390 con 0x7f04381305c0 2024-01-24T23:27:34.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f042c001380 con 0x7f04381305c0 2024-01-24T23:27:34.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:27:34.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient(hunting): dump: 2024-01-24T23:27:34.330 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.330 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.330 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.330 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.330 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.330 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.330 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.330 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.330 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.330 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.330 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.330 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.330 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.330 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.331 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.331 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:34.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3751239711 0 0) 0x7f043806ee40 con 0x7f04380662f0 2024-01-24T23:27:34.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient(hunting): my global_id is 124225 2024-01-24T23:27:34.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0424003530 con 0x7f04380662f0 2024-01-24T23:27:34.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4226758265 0 0) 0x7f043806f390 con 0x7f04381305c0 2024-01-24T23:27:34.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 >> v1:172.21.15.132:6792/0 conn(0x7f04380662f0 legacy=0x7f04381311a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 >> v1:172.21.15.132:6789/0 conn(0x7f043813a030 legacy=0x7f04381cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:27:34.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 monclient: found mon.b 2024-01-24T23:27:34.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:34.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:27:34.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0438134a40 con 0x7f04381305c0 2024-01-24T23:27:34.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient: _finish_auth 0 2024-01-24T23:27:34.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:27:34.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:34.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f043e46a640 5 monclient: authenticate success, global_id 124211 2024-01-24T23:27:34.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f043e46a640 10 monclient: _renew_subs 2024-01-24T23:27:34.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f043e46a640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:27:34.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f043e46a640 1 -- 172.21.15.115:0/1350445504 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0438134760 con 0x7f04381305c0 2024-01-24T23:27:34.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f043e46a640 10 monclient: _renew_subs 2024-01-24T23:27:34.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f043e46a640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:27:34.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f043e46a640 1 -- 172.21.15.115:0/1350445504 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f043809f6d0 con 0x7f04381305c0 2024-01-24T23:27:34.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f042c003c30 con 0x7f04381305c0 2024-01-24T23:27:34.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:34.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f042c004160 con 0x7f04381305c0 2024-01-24T23:27:34.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f04367fc640 4 set_mon_vals no callback set 2024-01-24T23:27:34.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:27:34.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient: dump: 2024-01-24T23:27:34.333 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.333 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.333 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.333 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.333 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.333 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.333 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.334 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.334 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.334 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.334 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.334 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.334 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.334 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.334 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.334 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f043e46a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:34.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f043e46a640 1 -- 172.21.15.115:0/1350445504 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f04180065d0 con 0x7f0418002870 2024-01-24T23:27:34.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f042c004340 con 0x7f04381305c0 2024-01-24T23:27:34.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(219..219 src has 1..219) v4 ==== 12908+0+0 (unknown 3846234139 0 0) 0x7f042c045da0 con 0x7f04381305c0 2024-01-24T23:27:34.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2990255126 0 0) 0x7f04180065d0 con 0x7f0418002870 2024-01-24T23:27:34.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient(hunting): my global_id is 94959 2024-01-24T23:27:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f043806f390 con 0x7f0418002870 2024-01-24T23:27:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f043ca07640 10 monclient: get_auth_request con 0x7f04240371b0 auth_method 0 2024-01-24T23:27:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4161748115 0 0) 0x7f043806f390 con 0x7f0418002870 2024-01-24T23:27:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f04180065d0 con 0x7f0418002870 2024-01-24T23:27:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f04300013c0 con 0x7f0418002870 2024-01-24T23:27:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:27:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient: dump: 2024-01-24T23:27:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.336 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1153345481 0 0) 0x7f04180065d0 con 0x7f0418002870 2024-01-24T23:27:34.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 --> v1:172.21.15.115:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f041800a120 con 0x7f0418002870 2024-01-24T23:27:34.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 <== mon.0 v1:172.21.15.115:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f04300047f0 con 0x7f0418002870 2024-01-24T23:27:34.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:34.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:34.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 >> v1:172.21.15.115:6789/0 conn(0x7f0418002870 legacy=0x7f0418004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.312+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 >> v1:172.21.15.115:6789/0 conn(0x7f0418002870 legacy=0x7f0418004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:34.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.320+0000 7f043e46a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:27:34.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.320+0000 7f043e46a640 1 -- 172.21.15.115:0/1350445504 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0418009e10 con 0x7f04180090d0 2024-01-24T23:27:34.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.320+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2642039023 0 0) 0x7f0418009e10 con 0x7f04180090d0 2024-01-24T23:27:34.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.320+0000 7f0434ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.320+0000 7f0434ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.320+0000 7f0434ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.320+0000 7f0434ff9640 10 monclient(hunting): my global_id is 94968 2024-01-24T23:27:34.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.320+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04180065d0 con 0x7f04180090d0 2024-01-24T23:27:34.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.320+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2714421757 0 0) 0x7f04180065d0 con 0x7f04180090d0 2024-01-24T23:27:34.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.320+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0418009e10 con 0x7f04180090d0 2024-01-24T23:27:34.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.320+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0430004e90 con 0x7f04180090d0 2024-01-24T23:27:34.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.320+0000 7f0434ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.320+0000 7f0434ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:27:34.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.320+0000 7f0434ff9640 10 monclient: dump: 2024-01-24T23:27:34.342 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.342 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.342 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.342 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.342 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.342 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.342 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.342 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.342 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.343 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.343 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.343 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.343 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.343 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.343 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.343 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.320+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2714837624 0 0) 0x7f0418009e10 con 0x7f04180090d0 2024-01-24T23:27:34.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.320+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 --> v1:172.21.15.115:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f041800c310 con 0x7f04180090d0 2024-01-24T23:27:34.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.320+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 <== mon.0 v1:172.21.15.115:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 3999097700) 0x7f0430007150 con 0x7f04180090d0 2024-01-24T23:27:34.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.320+0000 7f0434ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:27:34.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.320+0000 7f0434ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:34.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.320+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 >> v1:172.21.15.115:6789/0 conn(0x7f04180090d0 legacy=0x7f0418009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.320+0000 7f0434ff9640 1 -- 172.21.15.115:0/1350445504 >> v1:172.21.15.115:6789/0 conn(0x7f04180090d0 legacy=0x7f0418009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:34.344 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:27:34.344 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:34.344 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:34.344 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:27:34.344 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 40, 2024-01-24T23:27:34.344 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:27:34.344 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:27:34.344 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:27:34.344 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:27:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:27:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:27:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:27:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:27:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:27:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:27:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 3, 2024-01-24T23:27:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:27:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:27:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:27:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:27:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:27:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:27:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:27:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:27:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:27:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:27:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:27:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:27:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:27:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:27:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:27:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:27:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:27:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:27:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:27:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:27:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:27:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:27:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:27:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:27:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:27:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:27:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:27:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:27:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:27:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:27:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:27:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:27:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:27:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:27:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:27:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:27:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:27:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:27:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:27:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:27:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:27:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:27:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:27:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:27:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:27:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:27:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:27:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:27:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:27:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:27:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:27:34.362 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:27:34.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.320+0000 7f043e46a640 1 -- 172.21.15.115:0/1350445504 >> v1:172.21.15.132:6812/1835401922 conn(0x7f04240371b0 legacy=0x7f0424039670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.324+0000 7f043e46a640 10 monclient: shutdown 2024-01-24T23:27:34.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.324+0000 7f043e46a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:34.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.324+0000 7f043e46a640 1 -- 172.21.15.115:0/1350445504 >> v1:172.21.15.115:6790/0 conn(0x7f04381305c0 legacy=0x7f04381318b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.324+0000 7f043ca07640 1 -- 172.21.15.115:0/1350445504 reap_dead start 2024-01-24T23:27:34.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.324+0000 7f043ca07640 1 -- 172.21.15.115:0/1350445504 reap_dead start 2024-01-24T23:27:34.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.324+0000 7f043e46a640 1 -- 172.21.15.115:0/1350445504 shutdown_connections 2024-01-24T23:27:34.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.324+0000 7f043e46a640 1 -- 172.21.15.115:0/1350445504 >> 172.21.15.115:0/1350445504 conn(0x7f04380a5d40 msgr2=0x7f04380a38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:34.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.324+0000 7f043e46a640 1 -- 172.21.15.115:0/1350445504 shutdown_connections 2024-01-24T23:27:34.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.324+0000 7f043e46a640 1 -- 172.21.15.115:0/1350445504 wait complete. 2024-01-24T23:27:34.366 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:27:34.366 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-01-24T23:27:34.366 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-01-24T23:27:34.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdadecbd640 10 monclient: get_monmap_and_config 2024-01-24T23:27:34.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdadecbd640 10 monclient: build_initial_monmap 2024-01-24T23:27:34.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdadecbd640 10 monclient: monmap: 2024-01-24T23:27:34.369 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:34.369 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.369 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:34.318126+0000 2024-01-24T23:27:34.369 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:34.318126+0000 2024-01-24T23:27:34.369 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:34.369 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:34.369 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:34.370 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:34.370 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:34.370 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:34.370 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:34.370 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:34.370 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:34.370 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:34.370 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:34.370 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:34.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdadecbd640 1 Processor -- start 2024-01-24T23:27:34.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdadecbd640 1 -- start start 2024-01-24T23:27:34.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdadecbd640 10 monclient: init 2024-01-24T23:27:34.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdadecbd640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:34.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdadecbd640 10 monclient: _add_conns ranks=[2,3,4,1,8,0,7,5,6] 2024-01-24T23:27:34.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdadecbd640 10 monclient(hunting): picked mon.noname-b con 0x7fdad8135e10 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:34.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdadecbd640 10 monclient(hunting): picked mon.noname-g con 0x7fdad81361f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:34.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdadecbd640 10 monclient(hunting): picked mon.noname-c con 0x7fdad81365f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:34.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdadecbd640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad8064700 con 0x7fdad8135e10 2024-01-24T23:27:34.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdadecbd640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad809c3e0 con 0x7fdad81361f0 2024-01-24T23:27:34.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdadecbd640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad80a4f90 con 0x7fdad81365f0 2024-01-24T23:27:34.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdadecbd640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:34.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdadecbd640 10 monclient(hunting): authenticate will time out at 1341.642766s 2024-01-24T23:27:34.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdadca59640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fdad8135e10 0x7fdad809bcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.132:59620/0 (socket says 172.21.15.132:59620) 2024-01-24T23:27:34.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad7fff640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fdad81361f0 0x7fdad813bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:37818/0 (socket says 172.21.15.132:37818) 2024-01-24T23:27:34.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdadca59640 1 -- 172.21.15.132:0/1413815003 learned_addr learned my addr 172.21.15.132:0/1413815003 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:34.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 1 -- 172.21.15.132:0/1413815003 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3621223013 0 0) 0x7fdad80a4f90 con 0x7fdad81365f0 2024-01-24T23:27:34.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 10 monclient(hunting): my global_id is 94828 2024-01-24T23:27:34.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 1 -- 172.21.15.132:0/1413815003 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdac8003530 con 0x7fdad81365f0 2024-01-24T23:27:34.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 1 -- 172.21.15.132:0/1413815003 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1473442606 0 0) 0x7fdad8064700 con 0x7fdad8135e10 2024-01-24T23:27:34.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 10 monclient(hunting): my global_id is 124220 2024-01-24T23:27:34.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 1 -- 172.21.15.132:0/1413815003 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdad80a4f90 con 0x7fdad8135e10 2024-01-24T23:27:34.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 1 -- 172.21.15.132:0/1413815003 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1884062561 0 0) 0x7fdad809c3e0 con 0x7fdad81361f0 2024-01-24T23:27:34.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 10 monclient(hunting): my global_id is 6636 2024-01-24T23:27:34.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 1 -- 172.21.15.132:0/1413815003 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdad8064700 con 0x7fdad81361f0 2024-01-24T23:27:34.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 1 -- 172.21.15.132:0/1413815003 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 958161960 0 0) 0x7fdac8003530 con 0x7fdad81365f0 2024-01-24T23:27:34.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 1 -- 172.21.15.132:0/1413815003 >> v1:172.21.15.132:6790/0 conn(0x7fdad81361f0 legacy=0x7fdad813bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 1 -- 172.21.15.132:0/1413815003 >> v1:172.21.15.115:6790/0 conn(0x7fdad8135e10 legacy=0x7fdad809bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 10 monclient: _finish_hunting 0 2024-01-24T23:27:34.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 1 monclient: found mon.noname-c 2024-01-24T23:27:34.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:34.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:27:34.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 1 -- 172.21.15.132:0/1413815003 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad809b010 con 0x7fdad81365f0 2024-01-24T23:27:34.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 10 monclient: _finish_auth 0 2024-01-24T23:27:34.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:34.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:34.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 1 -- 172.21.15.132:0/1413815003 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdad0001a70 con 0x7fdad81365f0 2024-01-24T23:27:34.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:27:34.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 10 monclient: dump: 2024-01-24T23:27:34.374 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:34.374 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.374 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.374 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.374 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.374 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:34.374 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.374 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.374 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.374 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.374 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.375 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.375 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.375 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.375 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.375 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:34.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdadecbd640 5 monclient: authenticate success, global_id 94828 2024-01-24T23:27:34.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdadecbd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:34.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 1 -- 172.21.15.132:0/1413815003 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdad0001cd0 con 0x7fdad81365f0 2024-01-24T23:27:34.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:34.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 1 -- 172.21.15.132:0/1413815003 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdad00039a0 con 0x7fdad81365f0 2024-01-24T23:27:34.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdadecbd640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:34.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdadecbd640 4 set_mon_vals no callback set 2024-01-24T23:27:34.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:34.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.314+0000 7fdad77fe640 10 monclient: dump: 2024-01-24T23:27:34.376 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:34.376 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.376 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.376 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.376 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.376 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:34.376 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.376 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.376 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.376 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.376 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.376 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.376 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.376 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.377 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.377 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:34.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 10 monclient: shutdown 2024-01-24T23:27:34.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:34.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 1 -- 172.21.15.132:0/1413815003 >> v1:172.21.15.115:6791/0 conn(0x7fdad81365f0 legacy=0x7fdad813f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 1 -- 172.21.15.132:0/1413815003 shutdown_connections 2024-01-24T23:27:34.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 1 -- 172.21.15.132:0/1413815003 >> 172.21.15.132:0/1413815003 conn(0x7fdad80a6790 msgr2=0x7fdad8071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:34.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 1 -- 172.21.15.132:0/1413815003 shutdown_connections 2024-01-24T23:27:34.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 1 -- 172.21.15.132:0/1413815003 wait complete. 2024-01-24T23:27:34.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 10 monclient: build_initial_monmap 2024-01-24T23:27:34.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 10 monclient: monmap: 2024-01-24T23:27:34.377 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:34.377 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:34.377 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:34.377 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:34.378 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:34.378 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:34.378 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:34.378 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:34.378 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:34.378 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:34.378 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:34.378 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:34.378 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:34.378 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:34.378 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:34.378 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:34.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 1 Processor -- start 2024-01-24T23:27:34.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 1 -- start start 2024-01-24T23:27:34.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 10 monclient: init 2024-01-24T23:27:34.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:34.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 10 monclient: _add_conns ranks=[5,4,7,8,0,6,1,2,3] 2024-01-24T23:27:34.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 10 monclient(hunting): picked mon.noname-h con 0x7fdad8135e10 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:34.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 10 monclient(hunting): picked mon.noname-c con 0x7fdad81361f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:34.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 10 monclient(hunting): picked mon.noname-i con 0x7fdad81365f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:34.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad8120f20 con 0x7fdad81361f0 2024-01-24T23:27:34.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad8122570 con 0x7fdad8135e10 2024-01-24T23:27:34.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad8121cd0 con 0x7fdad81365f0 2024-01-24T23:27:34.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:34.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 10 monclient(hunting): authenticate will time out at 1341.645959s 2024-01-24T23:27:34.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadca59640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fdad8135e10 0x7fdad811fe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:45756/0 (socket says 172.21.15.132:45756) 2024-01-24T23:27:34.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadca59640 1 -- 172.21.15.132:0/1286709720 learned_addr learned my addr 172.21.15.132:0/1286709720 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:34.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadd25a640 1 --1- 172.21.15.132:0/1286709720 >> v1:172.21.15.132:6792/0 conn(0x7fdad81365f0 0x7fdad81d33e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.132:45678/0 (socket says 172.21.15.132:45678) 2024-01-24T23:27:34.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2963688657 0 0) 0x7fdad8121cd0 con 0x7fdad81365f0 2024-01-24T23:27:34.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient(hunting): my global_id is 124234 2024-01-24T23:27:34.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdabc003530 con 0x7fdad81365f0 2024-01-24T23:27:34.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538532991 0 0) 0x7fdad8122570 con 0x7fdad8135e10 2024-01-24T23:27:34.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient(hunting): my global_id is 6674 2024-01-24T23:27:34.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdad8121cd0 con 0x7fdad8135e10 2024-01-24T23:27:34.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2017485126 0 0) 0x7fdad8120f20 con 0x7fdad81361f0 2024-01-24T23:27:34.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient(hunting): my global_id is 94837 2024-01-24T23:27:34.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdad8122570 con 0x7fdad81361f0 2024-01-24T23:27:34.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3437792713 0 0) 0x7fdabc003530 con 0x7fdad81365f0 2024-01-24T23:27:34.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdad8120f20 con 0x7fdad81365f0 2024-01-24T23:27:34.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdad0001a30 con 0x7fdad81365f0 2024-01-24T23:27:34.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:27:34.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient(hunting): dump: 2024-01-24T23:27:34.381 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:34.381 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.381 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.381 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.381 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.381 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:34.381 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.381 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.381 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.382 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.382 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.382 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.382 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.382 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.382 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.382 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:34.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:34.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 842134364 0 0) 0x7fdad8121cd0 con 0x7fdad8135e10 2024-01-24T23:27:34.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdabc003530 con 0x7fdad8135e10 2024-01-24T23:27:34.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdacc001440 con 0x7fdad8135e10 2024-01-24T23:27:34.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:34.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient(hunting): dump: 2024-01-24T23:27:34.382 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:34.382 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.383 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.383 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.383 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.383 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:34.383 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.383 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.383 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.383 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.383 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.383 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.383 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.383 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.383 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.383 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:34.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3882733150 0 0) 0x7fdad8122570 con 0x7fdad81361f0 2024-01-24T23:27:34.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdad8121cd0 con 0x7fdad81361f0 2024-01-24T23:27:34.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdac4001920 con 0x7fdad81361f0 2024-01-24T23:27:34.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:34.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient(hunting): dump: 2024-01-24T23:27:34.384 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:34.384 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.384 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.384 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.384 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.384 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:34.384 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.384 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.384 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.384 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.384 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.384 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.385 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.385 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.385 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.385 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:34.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 314625501 0 0) 0x7fdad8120f20 con 0x7fdad81365f0 2024-01-24T23:27:34.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 >> v1:172.21.15.132:6791/0 conn(0x7fdad8135e10 legacy=0x7fdad811fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 >> v1:172.21.15.115:6791/0 conn(0x7fdad81361f0 legacy=0x7fdad8134de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient: _finish_hunting 0 2024-01-24T23:27:34.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 1 monclient: found mon.i 2024-01-24T23:27:34.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:34.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:27:34.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad8121fb0 con 0x7fdad81365f0 2024-01-24T23:27:34.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient: _finish_auth 0 2024-01-24T23:27:34.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient: _check_auth_tickets 2024-01-24T23:27:34.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:34.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 5 monclient: authenticate success, global_id 124234 2024-01-24T23:27:34.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 10 monclient: _renew_subs 2024-01-24T23:27:34.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:27:34.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 1 -- 172.21.15.132:0/1286709720 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad8122290 con 0x7fdad81365f0 2024-01-24T23:27:34.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 10 monclient: _renew_subs 2024-01-24T23:27:34.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:27:34.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 1 -- 172.21.15.132:0/1286709720 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad81354f0 con 0x7fdad81365f0 2024-01-24T23:27:34.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdad0003e90 con 0x7fdad81365f0 2024-01-24T23:27:34.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:34.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdad00043c0 con 0x7fdad81365f0 2024-01-24T23:27:34.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad6ffd640 4 set_mon_vals no callback set 2024-01-24T23:27:34.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:27:34.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 10 monclient: dump: 2024-01-24T23:27:34.386 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:34.387 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.387 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.387 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.387 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.387 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:34.387 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.387 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.387 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.387 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.387 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.387 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.387 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.387 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.388 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.388 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:34.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:34.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:34.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:27:34.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fdad00045a0 con 0x7fdad81365f0 2024-01-24T23:27:34.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.318+0000 7fdadecbd640 1 -- 172.21.15.132:0/1286709720 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab8005090 con 0x7fdad81365f0 2024-01-24T23:27:34.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.322+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(219..219 src has 1..219) v4 ==== 12908+0+0 (unknown 3846234139 0 0) 0x7fdad0046030 con 0x7fdad81365f0 2024-01-24T23:27:34.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.322+0000 7fdadca59640 10 monclient: get_auth_request con 0x7fdabc037b90 auth_method 0 2024-01-24T23:27:34.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.326+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 <== mon.7 v1:172.21.15.132:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fdad0014e60 con 0x7fdad81365f0 2024-01-24T23:27:34.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.326+0000 7fdad57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:34.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.326+0000 7fdad57fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:34.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.482+0000 7fdadecbd640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h8"}] 2024-01-24T23:27:34.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.482+0000 7fdadecbd640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h8"}] 2024-01-24T23:27:34.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.482+0000 7fdadecbd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:27:34.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:34.482+0000 7fdadecbd640 1 -- 172.21.15.132:0/1286709720 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "h8"} v 0) v1 -- 0x7fdab8005340 con 0x7fdad81365f0 2024-01-24T23:27:34.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 10 monclient: get_monmap_and_config 2024-01-24T23:27:34.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 10 monclient: build_initial_monmap 2024-01-24T23:27:34.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 10 monclient: monmap: 2024-01-24T23:27:34.513 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:34.513 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.513 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:34.515001+0000 2024-01-24T23:27:34.514 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:34.515001+0000 2024-01-24T23:27:34.514 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:34.514 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.514 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:34.514 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:34.514 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:34.514 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:34.514 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:34.514 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:34.514 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:34.514 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:34.514 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:34.514 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 1 Processor -- start 2024-01-24T23:27:34.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 1 -- start start 2024-01-24T23:27:34.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 10 monclient: init 2024-01-24T23:27:34.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:34.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 10 monclient: _add_conns ranks=[7,2,6,8,0,3,4,1,5] 2024-01-24T23:27:34.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 10 monclient(hunting): picked mon.noname-i con 0x7f4a30133bf0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:34.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 10 monclient(hunting): picked mon.noname-b con 0x7f4a30133fd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:34.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 10 monclient(hunting): picked mon.noname-d con 0x7f4a301343b0 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:34.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a3004ed70 con 0x7f4a30133fd0 2024-01-24T23:27:34.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a300a4b60 con 0x7f4a301343b0 2024-01-24T23:27:34.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a3009ae50 con 0x7f4a30133bf0 2024-01-24T23:27:34.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:34.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 10 monclient(hunting): authenticate will time out at 1333.437509s 2024-01-24T23:27:34.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2e7fc640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f4a30133fd0 0x7f4a3013bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:40332/0 (socket says 172.21.15.115:40332) 2024-01-24T23:27:34.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2e7fc640 1 -- 172.21.15.115:0/1383137153 learned_addr learned my addr 172.21.15.115:0/1383137153 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:34.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 1 -- 172.21.15.115:0/1383137153 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800211562 0 0) 0x7f4a3004ed70 con 0x7f4a30133fd0 2024-01-24T23:27:34.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 10 monclient(hunting): my global_id is 124229 2024-01-24T23:27:34.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 1 -- 172.21.15.115:0/1383137153 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a10003530 con 0x7f4a30133fd0 2024-01-24T23:27:34.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 1 -- 172.21.15.115:0/1383137153 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2064261426 0 0) 0x7f4a300a4b60 con 0x7f4a301343b0 2024-01-24T23:27:34.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 10 monclient(hunting): my global_id is 6810 2024-01-24T23:27:34.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 1 -- 172.21.15.115:0/1383137153 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a3004ed70 con 0x7f4a301343b0 2024-01-24T23:27:34.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 1 -- 172.21.15.115:0/1383137153 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2247778018 0 0) 0x7f4a10003530 con 0x7f4a30133fd0 2024-01-24T23:27:34.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 1 -- 172.21.15.115:0/1383137153 >> v1:172.21.15.132:6792/0 conn(0x7f4a30133bf0 legacy=0x7f4a3009bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 1 -- 172.21.15.115:0/1383137153 >> v1:172.21.15.115:6792/0 conn(0x7f4a301343b0 legacy=0x7f4a3013f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 10 monclient: _finish_hunting 0 2024-01-24T23:27:34.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 1 monclient: found mon.noname-b 2024-01-24T23:27:34.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:34.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:27:34.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 1 -- 172.21.15.115:0/1383137153 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a3009b130 con 0x7f4a30133fd0 2024-01-24T23:27:34.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 10 monclient: _finish_auth 0 2024-01-24T23:27:34.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 10 monclient: _check_auth_tickets 2024-01-24T23:27:34.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:34.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 1 -- 172.21.15.115:0/1383137153 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4a180017c0 con 0x7f4a30133fd0 2024-01-24T23:27:34.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 5 monclient: authenticate success, global_id 124229 2024-01-24T23:27:34.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:34.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:27:34.520 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 10 monclient: dump: 2024-01-24T23:27:34.520 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.521 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.521 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.521 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.521 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.521 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.521 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.521 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.521 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.521 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.521 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.521 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.521 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.521 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.521 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.521 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:34.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 1 -- 172.21.15.115:0/1383137153 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4a18001a60 con 0x7f4a30133fd0 2024-01-24T23:27:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 1 -- 172.21.15.115:0/1383137153 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4a18004bf0 con 0x7f4a30133fd0 2024-01-24T23:27:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 4 set_mon_vals no callback set 2024-01-24T23:27:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:27:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a2dffb640 10 monclient: dump: 2024-01-24T23:27:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.523 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.523 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.523 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.523 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.523 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.523 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.523 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.523 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 10 monclient: shutdown 2024-01-24T23:27:34.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:34.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 1 -- 172.21.15.115:0/1383137153 >> v1:172.21.15.115:6790/0 conn(0x7f4a30133fd0 legacy=0x7f4a3013bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 1 -- 172.21.15.115:0/1383137153 shutdown_connections 2024-01-24T23:27:34.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 1 -- 172.21.15.115:0/1383137153 >> 172.21.15.115:0/1383137153 conn(0x7f4a300662f0 msgr2=0x7f4a30066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:34.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 1 -- 172.21.15.115:0/1383137153 shutdown_connections 2024-01-24T23:27:34.523 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.512+0000 7f4a35667640 1 -- 172.21.15.115:0/1383137153 wait complete. 2024-01-24T23:27:34.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a35667640 10 monclient: build_initial_monmap 2024-01-24T23:27:34.524 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a35667640 10 monclient: monmap: 2024-01-24T23:27:34.524 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:34.524 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:34.524 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:34.524 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:34.524 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:34.524 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:34.524 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:34.524 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:34.524 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:34.524 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:34.524 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:34.524 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:34.524 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:34.525 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:34.525 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:34.525 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a35667640 1 Processor -- start 2024-01-24T23:27:34.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a35667640 1 -- start start 2024-01-24T23:27:34.525 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a35667640 10 monclient: init 2024-01-24T23:27:34.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a35667640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:34.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a35667640 10 monclient: _add_conns ranks=[6,4,0,3,2,1,8,5,7] 2024-01-24T23:27:34.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a35667640 10 monclient(hunting): picked mon.noname-d con 0x7f4a30133bf0 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:34.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a35667640 10 monclient(hunting): picked mon.noname-c con 0x7f4a301343b0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:34.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a35667640 10 monclient(hunting): picked mon.noname-a con 0x7f4a300a3410 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:34.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a35667640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a301c8b60 con 0x7f4a300a3410 2024-01-24T23:27:34.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a35667640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a3009c810 con 0x7f4a301343b0 2024-01-24T23:27:34.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a35667640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a300a5990 con 0x7f4a30133bf0 2024-01-24T23:27:34.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a35667640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:34.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a35667640 10 monclient(hunting): authenticate will time out at 1333.441326s 2024-01-24T23:27:34.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a2e7fc640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f4a301343b0 0x7f4a301d0980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:57654/0 (socket says 172.21.15.115:57654) 2024-01-24T23:27:34.531 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a2e7fc640 1 -- 172.21.15.115:0/1089206940 learned_addr learned my addr 172.21.15.115:0/1089206940 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:34.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a2f7fe640 1 --1- 172.21.15.115:0/1089206940 >> v1:172.21.15.115:6789/0 conn(0x7f4a300a3410 0x7f4a301d40b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:41236/0 (socket says 172.21.15.115:41236) 2024-01-24T23:27:34.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181311447 0 0) 0x7f4a3009c810 con 0x7f4a301343b0 2024-01-24T23:27:34.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): my global_id is 94846 2024-01-24T23:27:34.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a14003530 con 0x7f4a301343b0 2024-01-24T23:27:34.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1778424295 0 0) 0x7f4a300a5990 con 0x7f4a30133bf0 2024-01-24T23:27:34.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): my global_id is 6819 2024-01-24T23:27:34.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a3009c810 con 0x7f4a30133bf0 2024-01-24T23:27:34.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006243571 0 0) 0x7f4a301c8b60 con 0x7f4a300a3410 2024-01-24T23:27:34.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): my global_id is 94977 2024-01-24T23:27:34.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a300a5990 con 0x7f4a300a3410 2024-01-24T23:27:34.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2838752717 0 0) 0x7f4a14003530 con 0x7f4a301343b0 2024-01-24T23:27:34.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4a301c8b60 con 0x7f4a301343b0 2024-01-24T23:27:34.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4a18004b30 con 0x7f4a301343b0 2024-01-24T23:27:34.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:27:34.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): dump: 2024-01-24T23:27:34.533 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.534 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.534 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.534 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.534 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.534 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.534 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.534 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.534 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.534 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.534 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.534 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.534 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.534 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.534 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.534 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:34.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2151652742 0 0) 0x7f4a3009c810 con 0x7f4a30133bf0 2024-01-24T23:27:34.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4a14003530 con 0x7f4a30133bf0 2024-01-24T23:27:34.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4a28001d20 con 0x7f4a30133bf0 2024-01-24T23:27:34.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:34.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): dump: 2024-01-24T23:27:34.535 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.535 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.535 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.535 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.536 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.536 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.536 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.536 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.536 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.536 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.536 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.536 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.536 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.536 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.536 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.536 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2007500130 0 0) 0x7f4a300a5990 con 0x7f4a300a3410 2024-01-24T23:27:34.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4a3009c810 con 0x7f4a300a3410 2024-01-24T23:27:34.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4a20001a00 con 0x7f4a300a3410 2024-01-24T23:27:34.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:27:34.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): dump: 2024-01-24T23:27:34.537 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.537 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.537 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.537 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.537 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.537 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.537 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.538 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.538 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.538 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.538 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.538 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.538 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.538 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.538 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.538 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 429501660 0 0) 0x7f4a301c8b60 con 0x7f4a301343b0 2024-01-24T23:27:34.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 >> v1:172.21.15.115:6792/0 conn(0x7f4a30133bf0 legacy=0x7f4a30132f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 >> v1:172.21.15.115:6789/0 conn(0x7f4a300a3410 legacy=0x7f4a301d40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient: _finish_hunting 0 2024-01-24T23:27:34.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 monclient: found mon.c 2024-01-24T23:27:34.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:34.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:34.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a300a2690 con 0x7f4a301343b0 2024-01-24T23:27:34.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient: _finish_auth 0 2024-01-24T23:27:34.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:34.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:34.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a35667640 5 monclient: authenticate success, global_id 94846 2024-01-24T23:27:34.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a35667640 10 monclient: _renew_subs 2024-01-24T23:27:34.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a35667640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:34.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a35667640 1 -- 172.21.15.115:0/1089206940 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a30120610 con 0x7f4a301343b0 2024-01-24T23:27:34.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a35667640 10 monclient: _renew_subs 2024-01-24T23:27:34.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a35667640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:34.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a35667640 1 -- 172.21.15.115:0/1089206940 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a30133640 con 0x7f4a301343b0 2024-01-24T23:27:34.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4a18001090 con 0x7f4a301343b0 2024-01-24T23:27:34.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:34.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4a18001200 con 0x7f4a301343b0 2024-01-24T23:27:34.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a2d7fa640 4 set_mon_vals no callback set 2024-01-24T23:27:34.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:34.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient: dump: 2024-01-24T23:27:34.540 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.540 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.540 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.540 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.540 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.540 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.541 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.541 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.541 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.541 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.541 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.541 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.541 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.541 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.541 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.541 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a35667640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:34.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a35667640 1 -- 172.21.15.115:0/1089206940 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4a080065d0 con 0x7f4a08002870 2024-01-24T23:27:34.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f4a1800fb10 con 0x7f4a301343b0 2024-01-24T23:27:34.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(219..219 src has 1..219) v4 ==== 12908+0+0 (unknown 3846234139 0 0) 0x7f4a18045b50 con 0x7f4a301343b0 2024-01-24T23:27:34.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3634190292 0 0) 0x7f4a080065d0 con 0x7f4a08002870 2024-01-24T23:27:34.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 10 monclient(hunting): my global_id is 124238 2024-01-24T23:27:34.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a301c8b60 con 0x7f4a08002870 2024-01-24T23:27:34.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.516+0000 7f4a2f7fe640 10 monclient: get_auth_request con 0x7f4a14037b50 auth_method 0 2024-01-24T23:27:34.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.520+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4276195172 0 0) 0x7f4a301c8b60 con 0x7f4a08002870 2024-01-24T23:27:34.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.520+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4a080065d0 con 0x7f4a08002870 2024-01-24T23:27:34.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.520+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4a28004000 con 0x7f4a08002870 2024-01-24T23:27:34.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.520+0000 7f4a277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.520+0000 7f4a277fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:27:34.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.520+0000 7f4a277fe640 10 monclient: dump: 2024-01-24T23:27:34.542 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.542 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.542 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.542 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.543 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.543 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.543 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.543 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.543 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.543 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.543 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.543 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.543 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.543 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.543 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.543 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.520+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2099788605 0 0) 0x7f4a080065d0 con 0x7f4a08002870 2024-01-24T23:27:34.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.520+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 --> v1:172.21.15.115:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4a0800a120 con 0x7f4a08002870 2024-01-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.520+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 <== mon.2 v1:172.21.15.115:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f4a280044e0 con 0x7f4a08002870 2024-01-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.520+0000 7f4a277fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.520+0000 7f4a277fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.520+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 >> v1:172.21.15.115:6790/0 conn(0x7f4a08002870 legacy=0x7f4a08004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.520+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 >> v1:172.21.15.115:6790/0 conn(0x7f4a08002870 legacy=0x7f4a08004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.524+0000 7f4a35667640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.524+0000 7f4a35667640 1 -- 172.21.15.115:0/1089206940 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4a08009df0 con 0x7f4a080090b0 2024-01-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.524+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3910887543 0 0) 0x7f4a08009df0 con 0x7f4a080090b0 2024-01-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.524+0000 7f4a277fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.524+0000 7f4a277fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.524+0000 7f4a277fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.524+0000 7f4a277fe640 10 monclient(hunting): my global_id is 124247 2024-01-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.524+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a080065d0 con 0x7f4a080090b0 2024-01-24T23:27:34.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.524+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 585258333 0 0) 0x7f4a080065d0 con 0x7f4a080090b0 2024-01-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.524+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4a08009df0 con 0x7f4a080090b0 2024-01-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.524+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4a28004d10 con 0x7f4a080090b0 2024-01-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.524+0000 7f4a277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.524+0000 7f4a277fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.524+0000 7f4a277fe640 10 monclient: dump: 2024-01-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.545 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.546 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.546 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.546 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.546 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.546 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.546 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.546 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.546 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.524+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 756681423 0 0) 0x7f4a08009df0 con 0x7f4a080090b0 2024-01-24T23:27:34.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.524+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 --> v1:172.21.15.115:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4a0800c310 con 0x7f4a080090b0 2024-01-24T23:27:34.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.524+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 <== mon.2 v1:172.21.15.115:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 2915869938) 0x7f4a28006ff0 con 0x7f4a080090b0 2024-01-24T23:27:34.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.524+0000 7f4a277fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:27:34.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.524+0000 7f4a277fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:34.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.524+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 >> v1:172.21.15.115:6790/0 conn(0x7f4a080090b0 legacy=0x7f4a08009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.524+0000 7f4a277fe640 1 -- 172.21.15.115:0/1089206940 >> v1:172.21.15.115:6790/0 conn(0x7f4a080090b0 legacy=0x7f4a08009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:34.547 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:27:34.547 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:34.547 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:34.547 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:27:34.547 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 40, 2024-01-24T23:27:34.547 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:27:34.547 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:27:34.547 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:27:34.547 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:27:34.548 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:27:34.548 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:27:34.548 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:27:34.548 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:27:34.548 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:27:34.548 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:27:34.548 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:34.548 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 3, 2024-01-24T23:27:34.548 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:34.548 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:27:34.548 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:27:34.548 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:34.548 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:34.549 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:34.549 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:34.549 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:34.549 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:34.549 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:34.549 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:34.549 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:34.549 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:34.549 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:34.549 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:34.549 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:27:34.549 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:27:34.549 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:34.549 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:34.550 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:34.550 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:34.550 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:34.550 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:34.550 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:34.550 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:34.550 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:34.550 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:34.550 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:34.550 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.550 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:27:34.550 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:27:34.550 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:27:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:27:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:27:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:27:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:27:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:27:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:27:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:27:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:27:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:27:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:27:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:27:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:34.552 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:34.552 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:34.552 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:34.552 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:34.552 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:34.552 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:34.552 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:34.552 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:34.552 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:34.552 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:34.552 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:34.552 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:27:34.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.552 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:27:34.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.553 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:34.553 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:34.553 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.553 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.553 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.553 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:27:34.553 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.553 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.553 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.553 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:27:34.553 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:27:34.553 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.553 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.554 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.554 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:34.554 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:34.554 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.554 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.554 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.554 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:27:34.554 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.554 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.554 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.555 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:27:34.555 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:27:34.555 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.555 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.555 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.555 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:34.555 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:34.555 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.555 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.556 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.556 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:27:34.556 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.556 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.556 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.556 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:27:34.556 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:27:34.556 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.556 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.556 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.557 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:34.557 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:34.557 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.557 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.557 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.557 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:27:34.557 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.557 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.557 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.557 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:27:34.557 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:27:34.557 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.558 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.558 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.558 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:34.558 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:34.558 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.558 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.558 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.558 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:27:34.558 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.558 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.558 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.559 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:27:34.559 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:27:34.559 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.559 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.559 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.559 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:34.559 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:34.559 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.559 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.560 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.560 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:27:34.560 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.560 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.560 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.560 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:27:34.560 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:27:34.560 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.560 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.561 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.561 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:34.561 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:34.561 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.561 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.561 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.561 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:27:34.561 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.561 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.561 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.562 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:27:34.562 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:27:34.562 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.562 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.562 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.562 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:34.562 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:34.562 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.562 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.562 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.563 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:27:34.563 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.563 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.563 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.563 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:27:34.563 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:27:34.563 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.563 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.563 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.563 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:34.563 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:34.563 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.564 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.564 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.564 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:27:34.564 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.564 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.564 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.564 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:27:34.564 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:27:34.564 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.564 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.564 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.564 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.565 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.565 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:27:34.565 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:27:34.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.565 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:34.565 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:34.565 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:34.565 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.565 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:34.565 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:27:34.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.565 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:34.565 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:34.566 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:34.566 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.566 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.566 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:27:34.566 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:27:34.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.528+0000 7f4a35667640 1 -- 172.21.15.115:0/1089206940 >> v1:172.21.15.132:6812/1835401922 conn(0x7f4a14037b50 legacy=0x7f4a1403a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.528+0000 7f4a35667640 10 monclient: shutdown 2024-01-24T23:27:34.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.528+0000 7f4a35667640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:34.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.528+0000 7f4a35667640 1 -- 172.21.15.115:0/1089206940 >> v1:172.21.15.115:6791/0 conn(0x7f4a301343b0 legacy=0x7f4a301d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.528+0000 7f4a2f7fe640 1 -- 172.21.15.115:0/1089206940 reap_dead start 2024-01-24T23:27:34.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.528+0000 7f4a2f7fe640 1 -- 172.21.15.115:0/1089206940 reap_dead start 2024-01-24T23:27:34.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.528+0000 7f4a35667640 1 -- 172.21.15.115:0/1089206940 shutdown_connections 2024-01-24T23:27:34.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.528+0000 7f4a35667640 1 -- 172.21.15.115:0/1089206940 >> 172.21.15.115:0/1089206940 conn(0x7f4a300662f0 msgr2=0x7f4a300a8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:34.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.528+0000 7f4a35667640 1 -- 172.21.15.115:0/1089206940 shutdown_connections 2024-01-24T23:27:34.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.528+0000 7f4a35667640 1 -- 172.21.15.115:0/1089206940 wait complete. 2024-01-24T23:27:34.568 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:27:34.568 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-01-24T23:27:34.568 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-01-24T23:27:34.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 10 monclient: get_monmap_and_config 2024-01-24T23:27:34.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 10 monclient: build_initial_monmap 2024-01-24T23:27:34.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 10 monclient: monmap: 2024-01-24T23:27:34.682 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:34.682 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.682 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:34.683495+0000 2024-01-24T23:27:34.682 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:34.683495+0000 2024-01-24T23:27:34.682 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:34.682 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.682 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:34.682 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:34.683 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:34.683 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:34.683 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:34.683 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:34.683 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:34.683 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:34.683 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:34.683 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 1 Processor -- start 2024-01-24T23:27:34.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 1 -- start start 2024-01-24T23:27:34.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 10 monclient: init 2024-01-24T23:27:34.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:34.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 10 monclient: _add_conns ranks=[3,7,0,4,1,6,5,2,8] 2024-01-24T23:27:34.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 10 monclient(hunting): picked mon.noname-g con 0x7fad64133d90 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:34.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 10 monclient(hunting): picked mon.noname-i con 0x7fad64134170 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:34.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 10 monclient(hunting): picked mon.noname-a con 0x7fad6413dcf0 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:34.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad6404ed70 con 0x7fad6413dcf0 2024-01-24T23:27:34.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad6409b250 con 0x7fad64133d90 2024-01-24T23:27:34.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad641345d0 con 0x7fad64134170 2024-01-24T23:27:34.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:34.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 10 monclient(hunting): authenticate will time out at 1333.606066s 2024-01-24T23:27:34.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad688ee640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fad6413dcf0 0x7fad641400e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:41240/0 (socket says 172.21.15.115:41240) 2024-01-24T23:27:34.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad688ee640 1 -- 172.21.15.115:0/3304318704 learned_addr learned my addr 172.21.15.115:0/3304318704 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:34.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 1 -- 172.21.15.115:0/3304318704 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2673713841 0 0) 0x7fad6404ed70 con 0x7fad6413dcf0 2024-01-24T23:27:34.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 10 monclient(hunting): my global_id is 94986 2024-01-24T23:27:34.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 1 -- 172.21.15.115:0/3304318704 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad4c003530 con 0x7fad6413dcf0 2024-01-24T23:27:34.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 1 -- 172.21.15.115:0/3304318704 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2808433214 0 0) 0x7fad4c003530 con 0x7fad6413dcf0 2024-01-24T23:27:34.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 1 -- 172.21.15.115:0/3304318704 >> v1:172.21.15.132:6792/0 conn(0x7fad64134170 legacy=0x7fad640a2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 1 -- 172.21.15.115:0/3304318704 >> v1:172.21.15.132:6790/0 conn(0x7fad64133d90 legacy=0x7fad640a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:27:34.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 1 monclient: found mon.noname-a 2024-01-24T23:27:34.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:34.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:27:34.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 1 -- 172.21.15.115:0/3304318704 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad640a2810 con 0x7fad6413dcf0 2024-01-24T23:27:34.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 10 monclient: _finish_auth 0 2024-01-24T23:27:34.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:27:34.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:34.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 1 -- 172.21.15.115:0/3304318704 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fad5c001b40 con 0x7fad6413dcf0 2024-01-24T23:27:34.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 5 monclient: authenticate success, global_id 94986 2024-01-24T23:27:34.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:34.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:27:34.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 10 monclient: dump: 2024-01-24T23:27:34.688 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.689 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.689 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.689 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.689 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.689 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.689 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.689 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.689 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.689 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.689 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.689 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.689 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.689 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.689 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.689 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:34.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 1 -- 172.21.15.115:0/3304318704 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fad5c001df0 con 0x7fad6413dcf0 2024-01-24T23:27:34.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:34.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:34.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 1 -- 172.21.15.115:0/3304318704 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fad5c004f70 con 0x7fad6413dcf0 2024-01-24T23:27:34.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 4 set_mon_vals no callback set 2024-01-24T23:27:34.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:27:34.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad62ffd640 10 monclient: dump: 2024-01-24T23:27:34.690 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.690 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.690 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.690 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.690 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.690 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.691 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.691 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.691 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.691 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.691 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.691 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.691 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.691 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.691 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.691 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 10 monclient: shutdown 2024-01-24T23:27:34.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:34.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 1 -- 172.21.15.115:0/3304318704 >> v1:172.21.15.115:6789/0 conn(0x7fad6413dcf0 legacy=0x7fad641400e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 1 -- 172.21.15.115:0/3304318704 shutdown_connections 2024-01-24T23:27:34.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 1 -- 172.21.15.115:0/3304318704 >> 172.21.15.115:0/3304318704 conn(0x7fad640a65e0 msgr2=0x7fad640a2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 1 -- 172.21.15.115:0/3304318704 shutdown_connections 2024-01-24T23:27:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.680+0000 7fad6a351640 1 -- 172.21.15.115:0/3304318704 wait complete. 2024-01-24T23:27:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad6a351640 10 monclient: build_initial_monmap 2024-01-24T23:27:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad6a351640 10 monclient: monmap: 2024-01-24T23:27:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:34.693 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:34.693 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:34.693 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:34.693 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:34.693 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:34.693 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:34.693 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad6a351640 1 Processor -- start 2024-01-24T23:27:34.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad6a351640 1 -- start start 2024-01-24T23:27:34.693 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad6a351640 10 monclient: init 2024-01-24T23:27:34.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad6a351640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:34.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad6a351640 10 monclient: _add_conns ranks=[8,5,4,3,2,1,7,6,0] 2024-01-24T23:27:34.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad6a351640 10 monclient(hunting): picked mon.noname-e con 0x7fad64133d90 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:34.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad6a351640 10 monclient(hunting): picked mon.noname-h con 0x7fad64134170 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:34.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad6a351640 10 monclient(hunting): picked mon.noname-c con 0x7fad6413dcf0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:34.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad6a351640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad640676c0 con 0x7fad6413dcf0 2024-01-24T23:27:34.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad6a351640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad64139ab0 con 0x7fad64134170 2024-01-24T23:27:34.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad6a351640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad641393a0 con 0x7fad64133d90 2024-01-24T23:27:34.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad6a351640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:34.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad6a351640 10 monclient(hunting): authenticate will time out at 1333.609866s 2024-01-24T23:27:34.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad63fff640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fad64133d90 0x7fad6413d340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:46348/0 (socket says 172.21.15.115:46348) 2024-01-24T23:27:34.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad63fff640 1 -- 172.21.15.115:0/3086712893 learned_addr learned my addr 172.21.15.115:0/3086712893 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:34.699 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3168462061 0 0) 0x7fad640676c0 con 0x7fad6413dcf0 2024-01-24T23:27:34.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient(hunting): my global_id is 94855 2024-01-24T23:27:34.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad50003530 con 0x7fad6413dcf0 2024-01-24T23:27:34.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 387963128 0 0) 0x7fad641393a0 con 0x7fad64133d90 2024-01-24T23:27:34.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient(hunting): my global_id is 85841 2024-01-24T23:27:34.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad640676c0 con 0x7fad64133d90 2024-01-24T23:27:34.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4031626733 0 0) 0x7fad50003530 con 0x7fad6413dcf0 2024-01-24T23:27:34.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fad641393a0 con 0x7fad6413dcf0 2024-01-24T23:27:34.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fad5c001b00 con 0x7fad6413dcf0 2024-01-24T23:27:34.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:27:34.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient(hunting): dump: 2024-01-24T23:27:34.701 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.701 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.701 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.701 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.701 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.701 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.701 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.701 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.701 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.701 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.701 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.701 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.702 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.702 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.702 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.702 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:34.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 332226691 0 0) 0x7fad640676c0 con 0x7fad64133d90 2024-01-24T23:27:34.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fad50003530 con 0x7fad64133d90 2024-01-24T23:27:34.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 261270852 0 0) 0x7fad64139ab0 con 0x7fad64134170 2024-01-24T23:27:34.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient(hunting): my global_id is 6683 2024-01-24T23:27:34.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad640676c0 con 0x7fad64134170 2024-01-24T23:27:34.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fad540019c0 con 0x7fad64133d90 2024-01-24T23:27:34.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:27:34.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient(hunting): dump: 2024-01-24T23:27:34.703 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.703 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.703 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.703 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.703 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.703 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.703 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.703 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.703 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.703 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.703 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.704 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.704 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.704 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.704 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.704 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1240201467 0 0) 0x7fad50003530 con 0x7fad64133d90 2024-01-24T23:27:34.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 >> v1:172.21.15.132:6791/0 conn(0x7fad64134170 legacy=0x7fad641d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 >> v1:172.21.15.115:6791/0 conn(0x7fad6413dcf0 legacy=0x7fad641d3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:27:34.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 1 monclient: found mon.e 2024-01-24T23:27:34.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:34.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:27:34.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6413b570 con 0x7fad64133d90 2024-01-24T23:27:34.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient: _finish_auth 0 2024-01-24T23:27:34.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:27:34.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:34.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad6a351640 5 monclient: authenticate success, global_id 85841 2024-01-24T23:27:34.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad6a351640 10 monclient: _renew_subs 2024-01-24T23:27:34.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad6a351640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:27:34.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad6a351640 1 -- 172.21.15.115:0/3086712893 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad6413da50 con 0x7fad64133d90 2024-01-24T23:27:34.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad6a351640 10 monclient: _renew_subs 2024-01-24T23:27:34.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad6a351640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:27:34.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad6a351640 1 -- 172.21.15.115:0/3086712893 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fad641cc0d0 con 0x7fad64133d90 2024-01-24T23:27:34.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fad54001cc0 con 0x7fad64133d90 2024-01-24T23:27:34.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:34.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fad54003c10 con 0x7fad64133d90 2024-01-24T23:27:34.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad627fc640 4 set_mon_vals no callback set 2024-01-24T23:27:34.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:27:34.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 10 monclient: dump: 2024-01-24T23:27:34.706 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.706 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.706 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.706 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.706 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.706 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.706 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.706 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.706 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.706 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.706 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.706 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.706 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.706 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.707 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.707 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad6a351640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:34.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad6a351640 1 -- 172.21.15.115:0/3086712893 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fad440065d0 con 0x7fad44002870 2024-01-24T23:27:34.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.684+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fad54003df0 con 0x7fad64133d90 2024-01-24T23:27:34.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.688+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(219..219 src has 1..219) v4 ==== 12908+0+0 (unknown 3846234139 0 0) 0x7fad54045ae0 con 0x7fad64133d90 2024-01-24T23:27:34.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.688+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1540468422 0 0) 0x7fad440065d0 con 0x7fad44002870 2024-01-24T23:27:34.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.688+0000 7fad60ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.688+0000 7fad60ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.688+0000 7fad60ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.688+0000 7fad60ff9640 10 monclient(hunting): my global_id is 94864 2024-01-24T23:27:34.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.688+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad50003530 con 0x7fad44002870 2024-01-24T23:27:34.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.688+0000 7fad688ee640 10 monclient: get_auth_request con 0x7fad500376d0 auth_method 0 2024-01-24T23:27:34.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.688+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3782712939 0 0) 0x7fad50003530 con 0x7fad44002870 2024-01-24T23:27:34.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.688+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fad440065d0 con 0x7fad44002870 2024-01-24T23:27:34.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.688+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fad58001830 con 0x7fad44002870 2024-01-24T23:27:34.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.688+0000 7fad60ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.688+0000 7fad60ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:34.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.688+0000 7fad60ff9640 10 monclient: dump: 2024-01-24T23:27:34.708 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.708 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.708 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.708 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.708 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.708 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.708 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.708 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.708 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.709 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.709 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.709 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.709 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.709 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.709 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.709 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.688+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 478899908 0 0) 0x7fad440065d0 con 0x7fad44002870 2024-01-24T23:27:34.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.688+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 --> v1:172.21.15.115:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad4400a120 con 0x7fad44002870 2024-01-24T23:27:34.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.688+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 <== mon.4 v1:172.21.15.115:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fad58004650 con 0x7fad44002870 2024-01-24T23:27:34.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.688+0000 7fad60ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:34.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.688+0000 7fad60ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:34.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.688+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 >> v1:172.21.15.115:6791/0 conn(0x7fad44002870 legacy=0x7fad44004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.688+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 >> v1:172.21.15.115:6791/0 conn(0x7fad44002870 legacy=0x7fad44004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:34.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.692+0000 7fad6a351640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:27:34.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.692+0000 7fad6a351640 1 -- 172.21.15.115:0/3086712893 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fad44009e10 con 0x7fad440090d0 2024-01-24T23:27:34.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.692+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3034939382 0 0) 0x7fad44009e10 con 0x7fad440090d0 2024-01-24T23:27:34.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.692+0000 7fad60ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.692+0000 7fad60ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.692+0000 7fad60ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.692+0000 7fad60ff9640 10 monclient(hunting): my global_id is 94873 2024-01-24T23:27:34.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.692+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad440065d0 con 0x7fad440090d0 2024-01-24T23:27:34.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.692+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2176303747 0 0) 0x7fad440065d0 con 0x7fad440090d0 2024-01-24T23:27:34.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.692+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fad44009e10 con 0x7fad440090d0 2024-01-24T23:27:34.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.692+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fad58003b50 con 0x7fad440090d0 2024-01-24T23:27:34.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.692+0000 7fad60ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.692+0000 7fad60ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:34.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.692+0000 7fad60ff9640 10 monclient: dump: 2024-01-24T23:27:34.710 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.710 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.711 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.711 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.711 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.711 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.711 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.711 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.711 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.711 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.711 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.711 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.711 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.711 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.711 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.711 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.692+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 734160331 0 0) 0x7fad44009e10 con 0x7fad440090d0 2024-01-24T23:27:34.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.692+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 --> v1:172.21.15.115:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fad4400c310 con 0x7fad440090d0 2024-01-24T23:27:34.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.696+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 <== mon.4 v1:172.21.15.115:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 2628098282) 0x7fad58003fc0 con 0x7fad440090d0 2024-01-24T23:27:34.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.696+0000 7fad60ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:27:34.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.696+0000 7fad60ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:34.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.696+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 >> v1:172.21.15.115:6791/0 conn(0x7fad440090d0 legacy=0x7fad44009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.696+0000 7fad60ff9640 1 -- 172.21.15.115:0/3086712893 >> v1:172.21.15.115:6791/0 conn(0x7fad440090d0 legacy=0x7fad44009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:34.712 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:27:34.712 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:34.712 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:27:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 40, 2024-01-24T23:27:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:27:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:27:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:27:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:27:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:27:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:27:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:27:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:27:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:27:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:27:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 4, 2024-01-24T23:27:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:27:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:27:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:27:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:27:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:27:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:27:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:27:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:27:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:27:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:27:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:27:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:27:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:27:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:27:34.717 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:27:34.717 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:27:34.717 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:27:34.717 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:34.717 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:27:34.717 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:34.717 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:34.717 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:34.717 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:34.717 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:34.717 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:34.717 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:34.717 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:34.717 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:34.718 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:34.718 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:34.718 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:34.718 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:27:34.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.718 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:27:34.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.718 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:34.718 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:34.718 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.718 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.718 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.718 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:27:34.719 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.719 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.719 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:27:34.719 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:27:34.719 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.719 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.719 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.719 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:34.719 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:34.719 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.720 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.720 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.720 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:27:34.720 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.720 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.720 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:27:34.720 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:27:34.720 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.720 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.720 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.721 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:34.721 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:34.721 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.721 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.721 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.721 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:27:34.721 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.721 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.721 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.721 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:27:34.721 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:27:34.721 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.722 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.722 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.722 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:34.722 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:34.722 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.722 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.722 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.722 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:27:34.722 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.722 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.722 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.723 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:27:34.723 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:27:34.723 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.723 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.723 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.723 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:34.723 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:34.723 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.723 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.723 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.723 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:27:34.723 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.723 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:27:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:27:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:27:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:27:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:27:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:27:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:27:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:27:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:27:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:27:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:27:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:27:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:27:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:27:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:27:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:27:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:27:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:27:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:27:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.731 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:34.731 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:34.731 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:34.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.731 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.731 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:27:34.731 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:27:34.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.696+0000 7fad6a351640 1 -- 172.21.15.115:0/3086712893 >> v1:172.21.15.132:6812/1835401922 conn(0x7fad500376d0 legacy=0x7fad50039b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.696+0000 7fad6a351640 10 monclient: shutdown 2024-01-24T23:27:34.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.696+0000 7fad6a351640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:34.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.696+0000 7fad6a351640 1 -- 172.21.15.115:0/3086712893 >> v1:172.21.15.115:6793/0 conn(0x7fad64133d90 legacy=0x7fad6413d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.696+0000 7fad688ee640 1 -- 172.21.15.115:0/3086712893 reap_dead start 2024-01-24T23:27:34.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.696+0000 7fad688ee640 1 -- 172.21.15.115:0/3086712893 reap_dead start 2024-01-24T23:27:34.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.696+0000 7fad6a351640 1 -- 172.21.15.115:0/3086712893 shutdown_connections 2024-01-24T23:27:34.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.696+0000 7fad6a351640 1 -- 172.21.15.115:0/3086712893 >> 172.21.15.115:0/3086712893 conn(0x7fad640a65e0 msgr2=0x7fad64132ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:34.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.696+0000 7fad6a351640 1 -- 172.21.15.115:0/3086712893 shutdown_connections 2024-01-24T23:27:34.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.696+0000 7fad6a351640 1 -- 172.21.15.115:0/3086712893 wait complete. 2024-01-24T23:27:34.733 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:27:34.733 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-01-24T23:27:34.733 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-01-24T23:27:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.848+0000 7f6f05427640 10 monclient: get_monmap_and_config 2024-01-24T23:27:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.848+0000 7f6f05427640 10 monclient: build_initial_monmap 2024-01-24T23:27:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.848+0000 7f6f05427640 10 monclient: monmap: 2024-01-24T23:27:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:34.854848+0000 2024-01-24T23:27:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:34.854848+0000 2024-01-24T23:27:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:34.854 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 1 Processor -- start 2024-01-24T23:27:34.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 1 -- start start 2024-01-24T23:27:34.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 10 monclient: init 2024-01-24T23:27:34.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:34.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 10 monclient: _add_conns ranks=[8,1,2,4,0,7,6,3,5] 2024-01-24T23:27:34.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 10 monclient(hunting): picked mon.noname-e con 0x7f6f00133bf0 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:34.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 10 monclient(hunting): picked mon.noname-f con 0x7f6f00133fd0 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:34.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 10 monclient(hunting): picked mon.noname-b con 0x7f6f001343b0 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:34.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f0004ed70 con 0x7f6f00133fd0 2024-01-24T23:27:34.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f000a4b60 con 0x7f6f001343b0 2024-01-24T23:27:34.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f0009ae50 con 0x7f6f00133bf0 2024-01-24T23:27:34.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:34.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 10 monclient(hunting): authenticate will time out at 1333.777462s 2024-01-24T23:27:34.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6eff7fe640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f6f001343b0 0x7f6f0013f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:40356/0 (socket says 172.21.15.115:40356) 2024-01-24T23:27:34.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6eff7fe640 1 -- 172.21.15.115:0/1431408342 learned_addr learned my addr 172.21.15.115:0/1431408342 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:34.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 1 -- 172.21.15.115:0/1431408342 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840364651 0 0) 0x7f6f000a4b60 con 0x7f6f001343b0 2024-01-24T23:27:34.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 10 monclient(hunting): my global_id is 124256 2024-01-24T23:27:34.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 1 -- 172.21.15.115:0/1431408342 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ee8003530 con 0x7f6f001343b0 2024-01-24T23:27:34.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 1 -- 172.21.15.115:0/1431408342 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1020266460 0 0) 0x7f6f0009ae50 con 0x7f6f00133bf0 2024-01-24T23:27:34.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 10 monclient(hunting): my global_id is 85850 2024-01-24T23:27:34.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 1 -- 172.21.15.115:0/1431408342 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f000a4b60 con 0x7f6f00133bf0 2024-01-24T23:27:34.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 1 -- 172.21.15.115:0/1431408342 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3704492936 0 0) 0x7f6ee8003530 con 0x7f6f001343b0 2024-01-24T23:27:34.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 1 -- 172.21.15.115:0/1431408342 >> v1:172.21.15.115:6793/0 conn(0x7f6f00133bf0 legacy=0x7f6f0009bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 1 -- 172.21.15.115:0/1431408342 >> v1:172.21.15.132:6789/0 conn(0x7f6f00133fd0 legacy=0x7f6f0013bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.864 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 10 monclient: _finish_hunting 0 2024-01-24T23:27:34.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 1 monclient: found mon.noname-b 2024-01-24T23:27:34.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:34.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:27:34.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 1 -- 172.21.15.115:0/1431408342 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0009b130 con 0x7f6f001343b0 2024-01-24T23:27:34.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 10 monclient: _finish_auth 0 2024-01-24T23:27:34.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 10 monclient: _check_auth_tickets 2024-01-24T23:27:34.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:34.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 1 -- 172.21.15.115:0/1431408342 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6ef8001a70 con 0x7f6f001343b0 2024-01-24T23:27:34.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:27:34.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 10 monclient: dump: 2024-01-24T23:27:34.865 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.865 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.865 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.865 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.866 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.866 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.866 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.866 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.866 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.866 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.866 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.866 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.866 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.866 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.866 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.866 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 5 monclient: authenticate success, global_id 124256 2024-01-24T23:27:34.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:34.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 1 -- 172.21.15.115:0/1431408342 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6ef8001d10 con 0x7f6f001343b0 2024-01-24T23:27:34.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:34.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:34.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 1 -- 172.21.15.115:0/1431408342 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6ef8004f70 con 0x7f6f001343b0 2024-01-24T23:27:34.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 4 set_mon_vals no callback set 2024-01-24T23:27:34.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:27:34.867 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6efdffb640 10 monclient: dump: 2024-01-24T23:27:34.867 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.867 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.867 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.867 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.867 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.867 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.867 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.867 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.867 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.868 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.868 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.868 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.868 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.868 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.868 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.868 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 10 monclient: shutdown 2024-01-24T23:27:34.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:34.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 1 -- 172.21.15.115:0/1431408342 >> v1:172.21.15.115:6790/0 conn(0x7f6f001343b0 legacy=0x7f6f0013f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 1 -- 172.21.15.115:0/1431408342 shutdown_connections 2024-01-24T23:27:34.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 1 -- 172.21.15.115:0/1431408342 >> 172.21.15.115:0/1431408342 conn(0x7f6f000662f0 msgr2=0x7f6f00066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:34.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 1 -- 172.21.15.115:0/1431408342 shutdown_connections 2024-01-24T23:27:34.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 1 -- 172.21.15.115:0/1431408342 wait complete. 2024-01-24T23:27:34.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 10 monclient: build_initial_monmap 2024-01-24T23:27:34.869 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.852+0000 7f6f05427640 10 monclient: monmap: 2024-01-24T23:27:34.869 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:34.869 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:34.869 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:34.869 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:34.869 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:34.869 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:34.869 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:34.869 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:34.869 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:34.869 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:34.869 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:34.869 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:34.869 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:34.869 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:34.870 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:34.870 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6f05427640 1 Processor -- start 2024-01-24T23:27:34.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6f05427640 1 -- start start 2024-01-24T23:27:34.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6f05427640 10 monclient: init 2024-01-24T23:27:34.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6f05427640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:34.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6f05427640 10 monclient: _add_conns ranks=[1,6,7,3,2,0,4,5,8] 2024-01-24T23:27:34.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6f05427640 10 monclient(hunting): picked mon.noname-f con 0x7f6f00133bf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:34.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6f05427640 10 monclient(hunting): picked mon.noname-d con 0x7f6f00133fd0 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:34.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6f05427640 10 monclient(hunting): picked mon.noname-i con 0x7f6f001343b0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:34.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6f05427640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f0009c810 con 0x7f6f00133bf0 2024-01-24T23:27:34.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6f05427640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f000a5990 con 0x7f6f00133fd0 2024-01-24T23:27:34.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6f05427640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f001208d0 con 0x7f6f001343b0 2024-01-24T23:27:34.870 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6f05427640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:34.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6f05427640 10 monclient(hunting): authenticate will time out at 1333.780845s 2024-01-24T23:27:34.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6efe7fc640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f6f00133fd0 0x7f6f001d05e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:54782/0 (socket says 172.21.15.115:54782) 2024-01-24T23:27:34.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6efe7fc640 1 -- 172.21.15.115:0/2539941323 learned_addr learned my addr 172.21.15.115:0/2539941323 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:34.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 207658695 0 0) 0x7f6f000a5990 con 0x7f6f00133fd0 2024-01-24T23:27:34.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient(hunting): my global_id is 6828 2024-01-24T23:27:34.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6eec003530 con 0x7f6f00133fd0 2024-01-24T23:27:34.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1617651886 0 0) 0x7f6eec003530 con 0x7f6f00133fd0 2024-01-24T23:27:34.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6f000a5990 con 0x7f6f00133fd0 2024-01-24T23:27:34.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6ef4001ae0 con 0x7f6f00133fd0 2024-01-24T23:27:34.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:27:34.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient(hunting): dump: 2024-01-24T23:27:34.872 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.872 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.872 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.872 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.872 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.872 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.872 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.872 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.872 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.872 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.872 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.872 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.872 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.872 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.872 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.873 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:34.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 931169992 0 0) 0x7f6f001208d0 con 0x7f6f001343b0 2024-01-24T23:27:34.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient(hunting): my global_id is 124252 2024-01-24T23:27:34.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6eec003530 con 0x7f6f001343b0 2024-01-24T23:27:34.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1713413173 0 0) 0x7f6f0009c810 con 0x7f6f00133bf0 2024-01-24T23:27:34.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient(hunting): my global_id is 124228 2024-01-24T23:27:34.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f001208d0 con 0x7f6f00133bf0 2024-01-24T23:27:34.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2330479744 0 0) 0x7f6f000a5990 con 0x7f6f00133fd0 2024-01-24T23:27:34.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 >> v1:172.21.15.132:6792/0 conn(0x7f6f001343b0 legacy=0x7f6f001d3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 >> v1:172.21.15.132:6789/0 conn(0x7f6f00133bf0 legacy=0x7f6f00132ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient: _finish_hunting 0 2024-01-24T23:27:34.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 monclient: found mon.d 2024-01-24T23:27:34.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:34.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:34.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f000a2610 con 0x7f6f00133fd0 2024-01-24T23:27:34.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient: _finish_auth 0 2024-01-24T23:27:34.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:34.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:34.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6f05427640 5 monclient: authenticate success, global_id 6828 2024-01-24T23:27:34.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6f05427640 10 monclient: _renew_subs 2024-01-24T23:27:34.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6f05427640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:34.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6f05427640 1 -- 172.21.15.115:0/2539941323 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f001205f0 con 0x7f6f00133fd0 2024-01-24T23:27:34.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6f05427640 10 monclient: _renew_subs 2024-01-24T23:27:34.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6f05427640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:34.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6f05427640 1 -- 172.21.15.115:0/2539941323 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f00133600 con 0x7f6f00133fd0 2024-01-24T23:27:34.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6ef4003910 con 0x7f6f00133fd0 2024-01-24T23:27:34.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:34.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6ef4003e40 con 0x7f6f00133fd0 2024-01-24T23:27:34.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6efd7fa640 4 set_mon_vals no callback set 2024-01-24T23:27:34.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:34.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient: dump: 2024-01-24T23:27:34.875 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.875 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.875 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.875 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.875 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.875 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.876 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.876 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.876 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.876 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.876 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.876 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.876 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.876 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.876 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.876 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6f05427640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:34.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6f05427640 1 -- 172.21.15.115:0/2539941323 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6ee00065d0 con 0x7f6ee0002870 2024-01-24T23:27:34.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f6ef4004020 con 0x7f6f00133fd0 2024-01-24T23:27:34.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(219..219 src has 1..219) v4 ==== 12908+0+0 (unknown 3846234139 0 0) 0x7f6ef4045c60 con 0x7f6f00133fd0 2024-01-24T23:27:34.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283806941 0 0) 0x7f6ee00065d0 con 0x7f6ee0002870 2024-01-24T23:27:34.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient(hunting): my global_id is 6837 2024-01-24T23:27:34.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f000a5990 con 0x7f6ee0002870 2024-01-24T23:27:34.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6eff7fe640 10 monclient: get_auth_request con 0x7f6eec0370b0 auth_method 0 2024-01-24T23:27:34.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2066886749 0 0) 0x7f6f000a5990 con 0x7f6ee0002870 2024-01-24T23:27:34.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6ee00065d0 con 0x7f6ee0002870 2024-01-24T23:27:34.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6ef0001820 con 0x7f6ee0002870 2024-01-24T23:27:34.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:34.877 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient: dump: 2024-01-24T23:27:34.877 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.877 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.877 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.878 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.878 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.878 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.878 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.878 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.878 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.878 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.878 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.878 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.878 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.878 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.878 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.878 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3272975738 0 0) 0x7f6ee00065d0 con 0x7f6ee0002870 2024-01-24T23:27:34.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 --> v1:172.21.15.115:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ee000a120 con 0x7f6ee0002870 2024-01-24T23:27:34.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f6ef0001d10 con 0x7f6ee0002870 2024-01-24T23:27:34.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:34.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:34.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 >> v1:172.21.15.115:6792/0 conn(0x7f6ee0002870 legacy=0x7f6ee0004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.856+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 >> v1:172.21.15.115:6792/0 conn(0x7f6ee0002870 legacy=0x7f6ee0004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:34.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6f05427640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:27:34.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6f05427640 1 -- 172.21.15.115:0/2539941323 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6ee0009e10 con 0x7f6ee00090d0 2024-01-24T23:27:34.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2248494886 0 0) 0x7f6ee0009e10 con 0x7f6ee00090d0 2024-01-24T23:27:34.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6ee77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:34.880 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6ee77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:34.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6ee77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:34.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6ee77fe640 10 monclient(hunting): my global_id is 6846 2024-01-24T23:27:34.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ee00065d0 con 0x7f6ee00090d0 2024-01-24T23:27:34.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1760596736 0 0) 0x7f6ee00065d0 con 0x7f6ee00090d0 2024-01-24T23:27:34.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6ee0009e10 con 0x7f6ee00090d0 2024-01-24T23:27:34.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6ef0003b30 con 0x7f6ee00090d0 2024-01-24T23:27:34.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6ee77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:34.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6ee77fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:34.881 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6ee77fe640 10 monclient: dump: 2024-01-24T23:27:34.881 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:34.881 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:34.881 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.881 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:34.881 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:34.881 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:34.882 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:34.882 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:34.882 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:34.882 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:34.882 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:34.882 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:34.882 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:34.882 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:34.882 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:34.882 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:34.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1972224420 0 0) 0x7f6ee0009e10 con 0x7f6ee00090d0 2024-01-24T23:27:34.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 --> v1:172.21.15.115:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6ee000c310 con 0x7f6ee00090d0 2024-01-24T23:27:34.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 2532130996) 0x7f6ef0004040 con 0x7f6ee00090d0 2024-01-24T23:27:34.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6ee77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:27:34.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6ee77fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:34.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 >> v1:172.21.15.115:6792/0 conn(0x7f6ee00090d0 legacy=0x7f6ee0009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6ee77fe640 1 -- 172.21.15.115:0/2539941323 >> v1:172.21.15.115:6792/0 conn(0x7f6ee00090d0 legacy=0x7f6ee0009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:34.883 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:27:34.883 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:34.883 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:34.883 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:27:34.883 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 40, 2024-01-24T23:27:34.883 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:27:34.883 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:27:34.883 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:27:34.884 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:27:34.884 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:27:34.884 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:27:34.884 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:27:34.884 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:27:34.884 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:27:34.884 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:27:34.884 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:34.884 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 4, 2024-01-24T23:27:34.884 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:34.884 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:27:34.884 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:27:34.884 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:34.884 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:34.885 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:34.885 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:34.885 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:34.885 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:34.885 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:34.885 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:34.885 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:34.885 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:34.885 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:34.885 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:34.885 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:27:34.885 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:27:34.885 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:34.885 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:34.886 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:34.886 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:34.886 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:34.886 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:34.886 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:34.886 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:34.886 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:34.886 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:34.886 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:34.886 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.886 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:27:34.886 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:27:34.886 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:27:34.887 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:27:34.887 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:27:34.887 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:27:34.887 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:34.887 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:34.887 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:27:34.887 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:27:34.887 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:27:34.887 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:27:34.887 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:27:34.887 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:27:34.887 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:27:34.887 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:34.887 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:27:34.888 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:34.888 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:34.888 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:34.888 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:34.888 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:34.888 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:34.888 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:34.888 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:34.888 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:34.888 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:34.888 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:34.888 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:34.889 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:27:34.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.889 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:27:34.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.889 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:34.889 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:34.889 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.889 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.889 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.889 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:27:34.889 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.889 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.889 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:27:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:27:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:27:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:27:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:27:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:27:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:27:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:27:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:27:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:27:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:27:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:27:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:27:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:27:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:27:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:27:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:27:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:27:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:27:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:27:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:27:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:27:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:27:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:27:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:27:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:27:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:27:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:27:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:27:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:34.902 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:27:34.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:34.902 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:34.902 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:34.902 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 3 2024-01-24T23:27:34.902 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:34.902 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:34.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:34.902 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:27:34.902 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:27:34.903 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6f05427640 1 -- 172.21.15.115:0/2539941323 >> v1:172.21.15.132:6812/1835401922 conn(0x7f6eec0370b0 legacy=0x7f6eec039570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6f05427640 10 monclient: shutdown 2024-01-24T23:27:34.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6f05427640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:34.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6f05427640 1 -- 172.21.15.115:0/2539941323 >> v1:172.21.15.115:6792/0 conn(0x7f6f00133fd0 legacy=0x7f6f001d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:34.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6eff7fe640 1 -- 172.21.15.115:0/2539941323 reap_dead start 2024-01-24T23:27:34.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6eff7fe640 1 -- 172.21.15.115:0/2539941323 reap_dead start 2024-01-24T23:27:34.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6f05427640 1 -- 172.21.15.115:0/2539941323 shutdown_connections 2024-01-24T23:27:34.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6f05427640 1 -- 172.21.15.115:0/2539941323 >> 172.21.15.115:0/2539941323 conn(0x7f6f000662f0 msgr2=0x7f6f000a78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:34.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6f05427640 1 -- 172.21.15.115:0/2539941323 shutdown_connections 2024-01-24T23:27:34.904 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:34.864+0000 7f6f05427640 1 -- 172.21.15.115:0/2539941323 wait complete. 2024-01-24T23:27:34.904 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:27:34.905 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-01-24T23:27:34.905 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-01-24T23:27:35.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.012+0000 7f49895d3640 10 monclient: get_monmap_and_config 2024-01-24T23:27:35.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.012+0000 7f49895d3640 10 monclient: build_initial_monmap 2024-01-24T23:27:35.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.012+0000 7f49895d3640 10 monclient: monmap: 2024-01-24T23:27:35.017 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:35.017 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.017 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:35.018801+0000 2024-01-24T23:27:35.017 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:35.018801+0000 2024-01-24T23:27:35.017 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:35.018 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.018 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:35.018 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:35.018 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:35.018 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:35.018 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:35.018 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:35.018 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:35.018 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:35.018 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:35.018 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 1 Processor -- start 2024-01-24T23:27:35.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 1 -- start start 2024-01-24T23:27:35.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 10 monclient: init 2024-01-24T23:27:35.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:35.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 10 monclient: _add_conns ranks=[2,1,7,3,4,6,5,0,8] 2024-01-24T23:27:35.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 10 monclient(hunting): picked mon.noname-b con 0x7f4984133bf0 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:35.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 10 monclient(hunting): picked mon.noname-f con 0x7f4984133fd0 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:35.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 10 monclient(hunting): picked mon.noname-i con 0x7f49841343b0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:35.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f498404ed70 con 0x7f4984133fd0 2024-01-24T23:27:35.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49840a4b60 con 0x7f4984133bf0 2024-01-24T23:27:35.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f498409ae50 con 0x7f49841343b0 2024-01-24T23:27:35.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:35.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 10 monclient(hunting): authenticate will time out at 1333.941118s 2024-01-24T23:27:35.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4982ffd640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f4984133bf0 0x7f498409bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:40366/0 (socket says 172.21.15.115:40366) 2024-01-24T23:27:35.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4982ffd640 1 -- 172.21.15.115:0/2534646047 learned_addr learned my addr 172.21.15.115:0/2534646047 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:35.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 1 -- 172.21.15.115:0/2534646047 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808325636 0 0) 0x7f49840a4b60 con 0x7f4984133bf0 2024-01-24T23:27:35.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 10 monclient(hunting): my global_id is 124265 2024-01-24T23:27:35.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 1 -- 172.21.15.115:0/2534646047 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f496c003530 con 0x7f4984133bf0 2024-01-24T23:27:35.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 1 -- 172.21.15.115:0/2534646047 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1152249712 0 0) 0x7f496c003530 con 0x7f4984133bf0 2024-01-24T23:27:35.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 1 -- 172.21.15.115:0/2534646047 >> v1:172.21.15.132:6792/0 conn(0x7f49841343b0 legacy=0x7f498413f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 1 -- 172.21.15.115:0/2534646047 >> v1:172.21.15.132:6789/0 conn(0x7f4984133fd0 legacy=0x7f498413bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:27:35.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 1 monclient: found mon.noname-b 2024-01-24T23:27:35.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:35.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:27:35.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 1 -- 172.21.15.115:0/2534646047 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498409b130 con 0x7f4984133bf0 2024-01-24T23:27:35.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 10 monclient: _finish_auth 0 2024-01-24T23:27:35.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:27:35.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:35.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 5 monclient: authenticate success, global_id 124265 2024-01-24T23:27:35.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:35.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 1 -- 172.21.15.115:0/2534646047 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f497c001b40 con 0x7f4984133bf0 2024-01-24T23:27:35.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:27:35.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 10 monclient: dump: 2024-01-24T23:27:35.028 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.028 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.029 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.029 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.029 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.029 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.029 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.029 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.029 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.029 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.029 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.029 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.029 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.029 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.029 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.029 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:35.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 1 -- 172.21.15.115:0/2534646047 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f497c001e60 con 0x7f4984133bf0 2024-01-24T23:27:35.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:35.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 1 -- 172.21.15.115:0/2534646047 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f497c004f70 con 0x7f4984133bf0 2024-01-24T23:27:35.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:27:35.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f4981ffb640 10 monclient: dump: 2024-01-24T23:27:35.030 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.030 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.030 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.030 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.030 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.030 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.030 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.030 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.030 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.030 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.031 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.031 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.031 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.031 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.031 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.031 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:35.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 4 set_mon_vals no callback set 2024-01-24T23:27:35.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 10 monclient: shutdown 2024-01-24T23:27:35.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:35.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 1 -- 172.21.15.115:0/2534646047 >> v1:172.21.15.115:6790/0 conn(0x7f4984133bf0 legacy=0x7f498409bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 1 -- 172.21.15.115:0/2534646047 shutdown_connections 2024-01-24T23:27:35.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 1 -- 172.21.15.115:0/2534646047 >> 172.21.15.115:0/2534646047 conn(0x7f49840662f0 msgr2=0x7f4984066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:35.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 1 -- 172.21.15.115:0/2534646047 shutdown_connections 2024-01-24T23:27:35.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 1 -- 172.21.15.115:0/2534646047 wait complete. 2024-01-24T23:27:35.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 10 monclient: build_initial_monmap 2024-01-24T23:27:35.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 10 monclient: monmap: 2024-01-24T23:27:35.032 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:35.032 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:35.032 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:35.032 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:35.032 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:35.032 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:35.032 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:35.032 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:35.032 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:35.032 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:35.032 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:35.032 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:35.032 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:35.033 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:35.033 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:35.033 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 1 Processor -- start 2024-01-24T23:27:35.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 1 -- start start 2024-01-24T23:27:35.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.016+0000 7f49895d3640 10 monclient: init 2024-01-24T23:27:35.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f49895d3640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:35.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f49895d3640 10 monclient: _add_conns ranks=[8,2,6,0,1,4,3,5,7] 2024-01-24T23:27:35.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f49895d3640 10 monclient(hunting): picked mon.noname-e con 0x7f4984133bf0 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:35.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f49895d3640 10 monclient(hunting): picked mon.noname-b con 0x7f4984133fd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:35.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f49895d3640 10 monclient(hunting): picked mon.noname-d con 0x7f49841343b0 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:35.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f49895d3640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f498409c810 con 0x7f4984133fd0 2024-01-24T23:27:35.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f49895d3640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4984120920 con 0x7f49841343b0 2024-01-24T23:27:35.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f49895d3640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49840a5990 con 0x7f4984133bf0 2024-01-24T23:27:35.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f49895d3640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:35.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f49895d3640 10 monclient(hunting): authenticate will time out at 1333.944185s 2024-01-24T23:27:35.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f49827fc640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f4984133fd0 0x7f49841d08d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:40372/0 (socket says 172.21.15.115:40372) 2024-01-24T23:27:35.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f49827fc640 1 -- 172.21.15.115:0/1416743280 learned_addr learned my addr 172.21.15.115:0/1416743280 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:35.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 223186807 0 0) 0x7f498409c810 con 0x7f4984133fd0 2024-01-24T23:27:35.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): my global_id is 124274 2024-01-24T23:27:35.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4970003530 con 0x7f4984133fd0 2024-01-24T23:27:35.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4141145297 0 0) 0x7f49840a5990 con 0x7f4984133bf0 2024-01-24T23:27:35.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): my global_id is 85859 2024-01-24T23:27:35.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f498409c810 con 0x7f4984133bf0 2024-01-24T23:27:35.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 70835561 0 0) 0x7f4984120920 con 0x7f49841343b0 2024-01-24T23:27:35.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): my global_id is 6855 2024-01-24T23:27:35.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49840a5990 con 0x7f49841343b0 2024-01-24T23:27:35.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 973964616 0 0) 0x7f498409c810 con 0x7f4984133bf0 2024-01-24T23:27:35.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4984120920 con 0x7f4984133bf0 2024-01-24T23:27:35.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3695490838 0 0) 0x7f4970003530 con 0x7f4984133fd0 2024-01-24T23:27:35.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f498409c810 con 0x7f4984133fd0 2024-01-24T23:27:35.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 63817226 0 0) 0x7f49840a5990 con 0x7f49841343b0 2024-01-24T23:27:35.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4970003530 con 0x7f49841343b0 2024-01-24T23:27:35.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f497c004eb0 con 0x7f4984133bf0 2024-01-24T23:27:35.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:27:35.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): dump: 2024-01-24T23:27:35.036 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.036 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.036 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.036 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.036 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.036 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.036 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.036 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.036 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.036 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.036 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.036 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.036 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.037 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.037 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.037 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:35.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4978001ae0 con 0x7f4984133fd0 2024-01-24T23:27:35.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:27:35.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): dump: 2024-01-24T23:27:35.037 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.037 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.037 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.037 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.037 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.037 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.038 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.038 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.038 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.038 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.038 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.038 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.038 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.038 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.038 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.038 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4974001890 con 0x7f49841343b0 2024-01-24T23:27:35.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:35.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): dump: 2024-01-24T23:27:35.039 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.039 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.039 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.039 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.039 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.039 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.039 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.039 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.039 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.039 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.039 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.039 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.040 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.040 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.040 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.040 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2155292470 0 0) 0x7f4984120920 con 0x7f4984133bf0 2024-01-24T23:27:35.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 >> v1:172.21.15.115:6792/0 conn(0x7f49841343b0 legacy=0x7f49841d4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 >> v1:172.21.15.115:6790/0 conn(0x7f4984133fd0 legacy=0x7f49841d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:27:35.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 monclient: found mon.e 2024-01-24T23:27:35.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:35.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:27:35.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49840a25d0 con 0x7f4984133bf0 2024-01-24T23:27:35.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient: _finish_auth 0 2024-01-24T23:27:35.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:35.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:35.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f49895d3640 5 monclient: authenticate success, global_id 85859 2024-01-24T23:27:35.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f49895d3640 10 monclient: _renew_subs 2024-01-24T23:27:35.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f49895d3640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:27:35.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f49895d3640 1 -- 172.21.15.115:0/1416743280 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4984120640 con 0x7f4984133bf0 2024-01-24T23:27:35.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f49895d3640 10 monclient: _renew_subs 2024-01-24T23:27:35.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f49895d3640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:27:35.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f49895d3640 1 -- 172.21.15.115:0/1416743280 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4984133740 con 0x7f4984133bf0 2024-01-24T23:27:35.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f497c001bb0 con 0x7f4984133bf0 2024-01-24T23:27:35.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:35.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f497c0054e0 con 0x7f4984133bf0 2024-01-24T23:27:35.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:27:35.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f49817fa640 4 set_mon_vals no callback set 2024-01-24T23:27:35.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient: dump: 2024-01-24T23:27:35.042 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.042 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.042 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.042 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.042 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.042 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.042 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.042 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.042 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.042 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.042 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.042 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.042 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.042 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.042 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.043 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f49895d3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:35.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f497c00ff60 con 0x7f4984133bf0 2024-01-24T23:27:35.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f49895d3640 1 -- 172.21.15.115:0/1416743280 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f49640065d0 con 0x7f4964002870 2024-01-24T23:27:35.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(219..219 src has 1..219) v4 ==== 12908+0+0 (unknown 3846234139 0 0) 0x7f497c046040 con 0x7f4984133bf0 2024-01-24T23:27:35.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2327835110 0 0) 0x7f49640065d0 con 0x7f4964002870 2024-01-24T23:27:35.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient(hunting): my global_id is 85868 2024-01-24T23:27:35.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4984120920 con 0x7f4964002870 2024-01-24T23:27:35.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f49837fe640 10 monclient: get_auth_request con 0x7f4970037b20 auth_method 0 2024-01-24T23:27:35.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1955040424 0 0) 0x7f4984120920 con 0x7f4964002870 2024-01-24T23:27:35.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f49640065d0 con 0x7f4964002870 2024-01-24T23:27:35.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4978003e80 con 0x7f4964002870 2024-01-24T23:27:35.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:27:35.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient: dump: 2024-01-24T23:27:35.044 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.044 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.044 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.044 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.044 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.044 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.044 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.044 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.044 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.044 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.044 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.044 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.044 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.045 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.045 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.045 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2510498363 0 0) 0x7f49640065d0 con 0x7f4964002870 2024-01-24T23:27:35.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 --> v1:172.21.15.115:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f496400a120 con 0x7f4964002870 2024-01-24T23:27:35.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f4978004290 con 0x7f4964002870 2024-01-24T23:27:35.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:35.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:35.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 >> v1:172.21.15.115:6793/0 conn(0x7f4964002870 legacy=0x7f4964004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.020+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 >> v1:172.21.15.115:6793/0 conn(0x7f4964002870 legacy=0x7f4964004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:35.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f49895d3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:27:35.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f49895d3640 1 -- 172.21.15.115:0/1416743280 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4964009df0 con 0x7f49640090b0 2024-01-24T23:27:35.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 107506162 0 0) 0x7f4964009df0 con 0x7f49640090b0 2024-01-24T23:27:35.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f496b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f496b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f496b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f496b7fe640 10 monclient(hunting): my global_id is 85877 2024-01-24T23:27:35.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49640065d0 con 0x7f49640090b0 2024-01-24T23:27:35.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2183825862 0 0) 0x7f49640065d0 con 0x7f49640090b0 2024-01-24T23:27:35.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4964009df0 con 0x7f49640090b0 2024-01-24T23:27:35.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4978005b10 con 0x7f49640090b0 2024-01-24T23:27:35.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f496b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f496b7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:27:35.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f496b7fe640 10 monclient: dump: 2024-01-24T23:27:35.048 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.048 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.048 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.048 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.048 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.048 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.048 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.049 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.049 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.049 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.049 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.049 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.049 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.049 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.049 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.049 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3070754209 0 0) 0x7f4964009df0 con 0x7f49640090b0 2024-01-24T23:27:35.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 --> v1:172.21.15.115:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f496400c310 con 0x7f49640090b0 2024-01-24T23:27:35.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 522144542) 0x7f4978005f60 con 0x7f49640090b0 2024-01-24T23:27:35.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f496b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:27:35.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f496b7fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:35.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 >> v1:172.21.15.115:6793/0 conn(0x7f49640090b0 legacy=0x7f4964009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f496b7fe640 1 -- 172.21.15.115:0/1416743280 >> v1:172.21.15.115:6793/0 conn(0x7f49640090b0 legacy=0x7f4964009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:35.050 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:27:35.050 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:35.050 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:35.050 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:27:35.050 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 40, 2024-01-24T23:27:35.050 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:27:35.050 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:27:35.050 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:27:35.051 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:27:35.051 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:27:35.051 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:27:35.051 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:27:35.051 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:27:35.051 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:27:35.051 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:27:35.051 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:35.051 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 4, 2024-01-24T23:27:35.051 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:35.051 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:27:35.051 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:27:35.051 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:35.051 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:35.052 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:35.052 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:35.052 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:35.052 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:35.052 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:35.052 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:35.052 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:35.052 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:35.052 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:35.052 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:35.052 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:27:35.052 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:27:35.052 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:35.052 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:35.053 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:35.053 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:35.053 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:35.053 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:35.053 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:35.053 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:35.053 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:35.053 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:35.053 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:35.053 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.053 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:27:35.053 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:27:35.053 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:27:35.053 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:27:35.054 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:27:35.054 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:27:35.054 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:35.054 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:35.054 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:27:35.054 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:27:35.054 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:27:35.054 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:27:35.054 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:27:35.054 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:27:35.054 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:27:35.054 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:35.054 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:27:35.055 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:35.055 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:35.055 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:35.055 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:35.055 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:35.055 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:35.055 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:35.055 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:35.055 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:35.055 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:35.055 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:35.055 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:35.055 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:27:35.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.056 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:27:35.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.056 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:35.056 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:35.056 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.056 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.056 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.056 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:27:35.056 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.056 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.056 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.056 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:27:35.057 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:27:35.057 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.057 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.057 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.057 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:35.057 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:35.057 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.057 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.057 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.057 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:27:35.057 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.057 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.058 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.058 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:27:35.058 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:27:35.058 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.058 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.058 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.058 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:35.058 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:35.058 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.058 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.059 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.059 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:27:35.059 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.059 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.059 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.059 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:27:35.059 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:27:35.059 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.059 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.059 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.060 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:35.060 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:35.060 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.060 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.060 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.060 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:27:35.060 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.060 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.060 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.060 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:27:35.060 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:27:35.061 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.061 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.061 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.061 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:35.061 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:35.061 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.061 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.061 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.061 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:27:35.061 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.061 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.062 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.062 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:27:35.062 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:27:35.062 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.062 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.062 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.062 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:35.062 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:35.062 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.062 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.062 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.063 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:27:35.063 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.063 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.063 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.063 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:27:35.063 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:27:35.063 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.063 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.063 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.063 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:35.063 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:35.063 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.064 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.064 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.064 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:27:35.064 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.064 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.064 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.064 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:27:35.064 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:27:35.064 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.064 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.064 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.065 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:35.065 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:35.065 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.065 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.065 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.065 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:27:35.065 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.065 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.065 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.065 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:27:35.065 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:27:35.065 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.065 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.066 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.066 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:35.066 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:35.066 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.066 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.066 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.066 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:27:35.066 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.066 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.066 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.066 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:27:35.067 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:27:35.067 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.067 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.067 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.067 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.067 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.067 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:27:35.067 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:27:35.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.067 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:35.067 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:35.067 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:35.067 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.067 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:35.068 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:27:35.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.068 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:35.068 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:35.068 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:27:35.068 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.068 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:35.068 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": [ 2024-01-24T23:27:35.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.068 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:35.068 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:35.068 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:35.068 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.068 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.069 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:27:35.069 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:27:35.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f49895d3640 1 -- 172.21.15.115:0/1416743280 >> v1:172.21.15.132:6812/1835401922 conn(0x7f4970037b20 legacy=0x7f4970039fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f49895d3640 10 monclient: shutdown 2024-01-24T23:27:35.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f49895d3640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:35.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f49895d3640 1 -- 172.21.15.115:0/1416743280 >> v1:172.21.15.115:6793/0 conn(0x7f4984133bf0 legacy=0x7f4984132e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f49837fe640 1 -- 172.21.15.115:0/1416743280 reap_dead start 2024-01-24T23:27:35.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f49837fe640 1 -- 172.21.15.115:0/1416743280 reap_dead start 2024-01-24T23:27:35.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f49895d3640 1 -- 172.21.15.115:0/1416743280 shutdown_connections 2024-01-24T23:27:35.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f49895d3640 1 -- 172.21.15.115:0/1416743280 >> 172.21.15.115:0/1416743280 conn(0x7f49840662f0 msgr2=0x7f49841bea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:35.071 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f49895d3640 1 -- 172.21.15.115:0/1416743280 shutdown_connections 2024-01-24T23:27:35.072 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.028+0000 7f49895d3640 1 -- 172.21.15.115:0/1416743280 wait complete. 2024-01-24T23:27:35.072 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:27:35.072 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-01-24T23:27:35.073 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-01-24T23:27:35.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.086+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 <== mon.7 v1:172.21.15.132:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h8"}]=0 removed item id -40 name 'h8' from crush map v220) v1 ==== 118+0+0 (unknown 2714944821 0 0) 0x7fdad0013500 con 0x7fdad81365f0 2024-01-24T23:27:35.089 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.086+0000 7fdad57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h8"}] 2024-01-24T23:27:35.090 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.086+0000 7fdad57fa640 10 monclient: _finish_command 2 = system:0 removed item id -40 name 'h8' from crush map 2024-01-24T23:27:35.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f58a3730640 10 monclient: get_monmap_and_config 2024-01-24T23:27:35.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f58a3730640 10 monclient: build_initial_monmap 2024-01-24T23:27:35.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f58a3730640 10 monclient: monmap: 2024-01-24T23:27:35.190 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:35.190 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.190 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:35.191815+0000 2024-01-24T23:27:35.190 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:35.191815+0000 2024-01-24T23:27:35.191 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:35.191 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.191 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:35.191 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:35.191 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:35.191 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:35.191 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:35.191 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:35.191 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:35.191 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:35.191 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:35.191 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.194+0000 7fdadecbd640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h8"}] 2024-01-24T23:27:35.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.194+0000 7fdadecbd640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h8"}] 2024-01-24T23:27:35.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.194+0000 7fdadecbd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:27:35.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.194+0000 7fdadecbd640 1 -- 172.21.15.132:0/1286709720 --> v1:172.21.15.132:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "h8"} v 0) v1 -- 0x7fdab80010a0 con 0x7fdad81365f0 2024-01-24T23:27:35.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.194+0000 7fdad57fa640 1 -- 172.21.15.132:0/1286709720 <== mon.7 v1:172.21.15.132:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h8"}]=0 device 'h8' does not appear in the crush map v220) v1 ==== 118+0+0 (unknown 3535788183 0 0) 0x7fdad0014c80 con 0x7fdad81365f0 2024-01-24T23:27:35.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.194+0000 7fdad57fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h8"}] 2024-01-24T23:27:35.198 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.194+0000 7fdad57fa640 10 monclient: _finish_command 3 = system:0 device 'h8' does not appear in the crush map 2024-01-24T23:27:35.198 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h8' does not appear in the crush map 2024-01-24T23:27:35.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f58a3730640 1 Processor -- start 2024-01-24T23:27:35.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f58a3730640 1 -- start start 2024-01-24T23:27:35.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f58a3730640 10 monclient: init 2024-01-24T23:27:35.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f58a3730640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:35.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f58a3730640 10 monclient: _add_conns ranks=[2,1,3,8,7,4,5,0,6] 2024-01-24T23:27:35.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f58a3730640 10 monclient(hunting): picked mon.noname-b con 0x7f589c0672f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:35.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f58a3730640 10 monclient(hunting): picked mon.noname-f con 0x7f589c0a0e30 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:35.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f58a3730640 10 monclient(hunting): picked mon.noname-g con 0x7f589c13d2f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:35.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f58a3730640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f589c04ed70 con 0x7f589c0a0e30 2024-01-24T23:27:35.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f58a3730640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f589c0a0540 con 0x7f589c0672f0 2024-01-24T23:27:35.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f58a3730640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f589c09e900 con 0x7f589c13d2f0 2024-01-24T23:27:35.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f58a3730640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:35.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f58a3730640 10 monclient(hunting): authenticate will time out at 1334.114452s 2024-01-24T23:27:35.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f58a14cc640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f589c0672f0 0x7f589c09fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:40378/0 (socket says 172.21.15.115:40378) 2024-01-24T23:27:35.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f58a14cc640 1 -- 172.21.15.115:0/509573210 learned_addr learned my addr 172.21.15.115:0/509573210 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:35.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 1 -- 172.21.15.115:0/509573210 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2938296926 0 0) 0x7f589c0a0540 con 0x7f589c0672f0 2024-01-24T23:27:35.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 10 monclient(hunting): my global_id is 124283 2024-01-24T23:27:35.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 1 -- 172.21.15.115:0/509573210 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5888003530 con 0x7f589c0672f0 2024-01-24T23:27:35.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 1 -- 172.21.15.115:0/509573210 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3920239105 0 0) 0x7f589c04ed70 con 0x7f589c0a0e30 2024-01-24T23:27:35.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 10 monclient(hunting): my global_id is 124246 2024-01-24T23:27:35.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 1 -- 172.21.15.115:0/509573210 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f589c0a0540 con 0x7f589c0a0e30 2024-01-24T23:27:35.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 1 -- 172.21.15.115:0/509573210 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1260935093 0 0) 0x7f589c09e900 con 0x7f589c13d2f0 2024-01-24T23:27:35.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 10 monclient(hunting): my global_id is 6654 2024-01-24T23:27:35.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 1 -- 172.21.15.115:0/509573210 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f589c04ed70 con 0x7f589c13d2f0 2024-01-24T23:27:35.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 1 -- 172.21.15.115:0/509573210 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3368558331 0 0) 0x7f5888003530 con 0x7f589c0672f0 2024-01-24T23:27:35.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 1 -- 172.21.15.115:0/509573210 >> v1:172.21.15.132:6790/0 conn(0x7f589c13d2f0 legacy=0x7f589c13f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 1 -- 172.21.15.115:0/509573210 >> v1:172.21.15.132:6789/0 conn(0x7f589c0a0e30 legacy=0x7f589c13bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 10 monclient: _finish_hunting 0 2024-01-24T23:27:35.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 1 monclient: found mon.noname-b 2024-01-24T23:27:35.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:35.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:27:35.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 1 -- 172.21.15.115:0/509573210 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589c0724a0 con 0x7f589c0672f0 2024-01-24T23:27:35.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 10 monclient: _finish_auth 0 2024-01-24T23:27:35.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 10 monclient: _check_auth_tickets 2024-01-24T23:27:35.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:35.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 1 -- 172.21.15.115:0/509573210 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5898001740 con 0x7f589c0672f0 2024-01-24T23:27:35.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f58a3730640 5 monclient: authenticate success, global_id 124283 2024-01-24T23:27:35.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f58a3730640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:35.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:27:35.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 10 monclient: dump: 2024-01-24T23:27:35.205 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.205 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.205 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.205 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.205 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.205 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.205 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.205 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.205 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.205 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.205 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.206 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.206 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.206 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.206 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.206 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f58a3730640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:35.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 1 -- 172.21.15.115:0/509573210 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5898001960 con 0x7f589c0672f0 2024-01-24T23:27:35.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:35.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 1 -- 172.21.15.115:0/509573210 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f58980048b0 con 0x7f589c0672f0 2024-01-24T23:27:35.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:27:35.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f588ffff640 10 monclient: dump: 2024-01-24T23:27:35.206 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.206 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.206 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.206 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.207 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.207 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.207 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.207 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.207 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.207 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.207 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.207 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.207 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.207 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.207 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.207 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f58a3730640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:35.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.188+0000 7f58a3730640 4 set_mon_vals no callback set 2024-01-24T23:27:35.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 10 monclient: shutdown 2024-01-24T23:27:35.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:35.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 1 -- 172.21.15.115:0/509573210 >> v1:172.21.15.115:6790/0 conn(0x7f589c0672f0 legacy=0x7f589c09fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 1 -- 172.21.15.115:0/509573210 shutdown_connections 2024-01-24T23:27:35.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 1 -- 172.21.15.115:0/509573210 >> 172.21.15.115:0/509573210 conn(0x7f589c0662f0 msgr2=0x7f589c066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:35.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 1 -- 172.21.15.115:0/509573210 shutdown_connections 2024-01-24T23:27:35.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 1 -- 172.21.15.115:0/509573210 wait complete. 2024-01-24T23:27:35.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 10 monclient: build_initial_monmap 2024-01-24T23:27:35.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 10 monclient: monmap: 2024-01-24T23:27:35.208 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:35.208 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:35.208 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:35.208 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:35.208 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:35.208 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:35.209 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:35.209 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:35.209 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:35.209 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:35.209 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:35.209 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:35.209 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:35.209 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:35.209 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:35.209 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 1 Processor -- start 2024-01-24T23:27:35.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 1 -- start start 2024-01-24T23:27:35.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 10 monclient: init 2024-01-24T23:27:35.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:35.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 10 monclient: _add_conns ranks=[7,6,0,1,4,3,8,5,2] 2024-01-24T23:27:35.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 10 monclient(hunting): picked mon.noname-i con 0x7f589c0672f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:35.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 10 monclient(hunting): picked mon.noname-d con 0x7f589c13d2f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:35.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 10 monclient(hunting): picked mon.noname-a con 0x7f589c13a9d0 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:35.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f589c06b5d0 con 0x7f589c13a9d0 2024-01-24T23:27:35.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f589c072170 con 0x7f589c13d2f0 2024-01-24T23:27:35.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f589c1394b0 con 0x7f589c0672f0 2024-01-24T23:27:35.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:35.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 10 monclient(hunting): authenticate will time out at 1334.117880s 2024-01-24T23:27:35.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a1ccd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f589c13a9d0 0x7f589c1d3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:41252/0 (socket says 172.21.15.115:41252) 2024-01-24T23:27:35.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a1ccd640 1 -- 172.21.15.115:0/2699797892 learned_addr learned my addr 172.21.15.115:0/2699797892 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:35.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1865618510 0 0) 0x7f589c072170 con 0x7f589c13d2f0 2024-01-24T23:27:35.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient(hunting): my global_id is 6864 2024-01-24T23:27:35.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5880003530 con 0x7f589c13d2f0 2024-01-24T23:27:35.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2665386329 0 0) 0x7f589c06b5d0 con 0x7f589c13a9d0 2024-01-24T23:27:35.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient(hunting): my global_id is 94995 2024-01-24T23:27:35.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f589c072170 con 0x7f589c13a9d0 2024-01-24T23:27:35.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3490564149 0 0) 0x7f5880003530 con 0x7f589c13d2f0 2024-01-24T23:27:35.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f589c06b5d0 con 0x7f589c13d2f0 2024-01-24T23:27:35.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5890001a60 con 0x7f589c13d2f0 2024-01-24T23:27:35.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:27:35.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient(hunting): dump: 2024-01-24T23:27:35.211 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.211 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.211 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.212 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.212 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.212 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.212 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.212 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.212 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.212 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.212 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.212 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.212 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.212 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.212 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.212 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:35.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3575553632 0 0) 0x7f589c072170 con 0x7f589c13a9d0 2024-01-24T23:27:35.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5880003530 con 0x7f589c13a9d0 2024-01-24T23:27:35.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2395973204 0 0) 0x7f589c1394b0 con 0x7f589c0672f0 2024-01-24T23:27:35.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient(hunting): my global_id is 124270 2024-01-24T23:27:35.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f589c072170 con 0x7f589c0672f0 2024-01-24T23:27:35.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5894001d60 con 0x7f589c13a9d0 2024-01-24T23:27:35.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:27:35.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient(hunting): dump: 2024-01-24T23:27:35.213 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.213 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.213 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.214 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.214 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.214 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.214 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.214 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.214 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.214 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.214 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.214 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.214 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.214 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.214 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.214 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2691634173 0 0) 0x7f589c06b5d0 con 0x7f589c13d2f0 2024-01-24T23:27:35.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 >> v1:172.21.15.132:6792/0 conn(0x7f589c0672f0 legacy=0x7f589c12dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 >> v1:172.21.15.115:6789/0 conn(0x7f589c13a9d0 legacy=0x7f589c1d3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient: _finish_hunting 0 2024-01-24T23:27:35.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 1 monclient: found mon.d 2024-01-24T23:27:35.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:35.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:35.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589c139750 con 0x7f589c13d2f0 2024-01-24T23:27:35.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient: _finish_auth 0 2024-01-24T23:27:35.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient: _check_auth_tickets 2024-01-24T23:27:35.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:35.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 5 monclient: authenticate success, global_id 6864 2024-01-24T23:27:35.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 10 monclient: _renew_subs 2024-01-24T23:27:35.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:35.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 1 -- 172.21.15.115:0/2699797892 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f589c071790 con 0x7f589c13d2f0 2024-01-24T23:27:35.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 10 monclient: _renew_subs 2024-01-24T23:27:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 1 -- 172.21.15.115:0/2699797892 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f589c0702a0 con 0x7f589c13d2f0 2024-01-24T23:27:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f58900038d0 con 0x7f589c13d2f0 2024-01-24T23:27:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5890003d80 con 0x7f589c13d2f0 2024-01-24T23:27:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588f7fe640 4 set_mon_vals no callback set 2024-01-24T23:27:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 10 monclient: dump: 2024-01-24T23:27:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.217 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f58a3730640 1 -- 172.21.15.115:0/2699797892 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f587c0065d0 con 0x7f587c002870 2024-01-24T23:27:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.192+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f589000e800 con 0x7f589c13d2f0 2024-01-24T23:27:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.196+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(220..220 src has 1..220) v4 ==== 12698+0+0 (unknown 1413254373 0 0) 0x7f5890044900 con 0x7f589c13d2f0 2024-01-24T23:27:35.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.196+0000 7f58a1ccd640 10 monclient: get_auth_request con 0x7f58800378f0 auth_method 0 2024-01-24T23:27:35.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.196+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172802709 0 0) 0x7f587c0065d0 con 0x7f587c002870 2024-01-24T23:27:35.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.196+0000 7f588dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.196+0000 7f588dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.196+0000 7f588dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.196+0000 7f588dffb640 10 monclient(hunting): my global_id is 124255 2024-01-24T23:27:35.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.196+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f589c06b5d0 con 0x7f587c002870 2024-01-24T23:27:35.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.196+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3180949332 0 0) 0x7f589c06b5d0 con 0x7f587c002870 2024-01-24T23:27:35.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.196+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f587c0065d0 con 0x7f587c002870 2024-01-24T23:27:35.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.196+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5898001240 con 0x7f587c002870 2024-01-24T23:27:35.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.196+0000 7f588dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.196+0000 7f588dffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:27:35.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.196+0000 7f588dffb640 10 monclient: dump: 2024-01-24T23:27:35.218 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.218 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.218 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.218 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.219 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.196+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1655360923 0 0) 0x7f587c0065d0 con 0x7f587c002870 2024-01-24T23:27:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.196+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 --> v1:172.21.15.132:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f587c00a120 con 0x7f587c002870 2024-01-24T23:27:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.196+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 <== mon.1 v1:172.21.15.132:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f5898001760 con 0x7f587c002870 2024-01-24T23:27:35.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.196+0000 7f588dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:35.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.196+0000 7f588dffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:35.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.196+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 >> v1:172.21.15.132:6789/0 conn(0x7f587c002870 legacy=0x7f587c004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.196+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 >> v1:172.21.15.132:6789/0 conn(0x7f587c002870 legacy=0x7f587c004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:35.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.198+0000 7fdadecbd640 1 -- 172.21.15.132:0/1286709720 >> v1:172.21.15.132:6812/1835401922 conn(0x7fdabc037b90 legacy=0x7fdabc03a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.198+0000 7fdadecbd640 10 monclient: shutdown 2024-01-24T23:27:35.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.198+0000 7fdadecbd640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:35.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.198+0000 7fdadecbd640 1 -- 172.21.15.132:0/1286709720 >> v1:172.21.15.132:6792/0 conn(0x7fdad81365f0 legacy=0x7fdad81d33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.198+0000 7fdadecbd640 1 -- 172.21.15.132:0/1286709720 shutdown_connections 2024-01-24T23:27:35.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.198+0000 7fdadecbd640 1 -- 172.21.15.132:0/1286709720 >> 172.21.15.132:0/1286709720 conn(0x7fdad80a6790 msgr2=0x7fdad81bb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:35.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.198+0000 7fdadecbd640 1 -- 172.21.15.132:0/1286709720 shutdown_connections 2024-01-24T23:27:35.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.198+0000 7fdadecbd640 1 -- 172.21.15.132:0/1286709720 wait complete. 2024-01-24T23:27:35.222 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:35.222 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h9 2024-01-24T23:27:35.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.200+0000 7f58a3730640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:27:35.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.200+0000 7f58a3730640 1 -- 172.21.15.115:0/2699797892 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f587c009df0 con 0x7f587c0090b0 2024-01-24T23:27:35.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406941767 0 0) 0x7f587c009df0 con 0x7f587c0090b0 2024-01-24T23:27:35.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f588dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f588dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f588dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f588dffb640 10 monclient(hunting): my global_id is 124264 2024-01-24T23:27:35.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f587c0065d0 con 0x7f587c0090b0 2024-01-24T23:27:35.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3685757057 0 0) 0x7f587c0065d0 con 0x7f587c0090b0 2024-01-24T23:27:35.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f587c009df0 con 0x7f587c0090b0 2024-01-24T23:27:35.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5898005010 con 0x7f587c0090b0 2024-01-24T23:27:35.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f588dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f588dffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:27:35.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f588dffb640 10 monclient: dump: 2024-01-24T23:27:35.223 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.223 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.223 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.224 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.224 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.224 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.224 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.224 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.224 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.224 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.224 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.224 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.224 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.224 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.224 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.224 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1720805649 0 0) 0x7f587c009df0 con 0x7f587c0090b0 2024-01-24T23:27:35.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 --> v1:172.21.15.132:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f587c00c310 con 0x7f587c0090b0 2024-01-24T23:27:35.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 <== mon.1 v1:172.21.15.132:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 4136446510) 0x7f5898007310 con 0x7f587c0090b0 2024-01-24T23:27:35.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f588dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:27:35.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f588dffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:35.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 >> v1:172.21.15.132:6789/0 conn(0x7f587c0090b0 legacy=0x7f587c009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f588dffb640 1 -- 172.21.15.115:0/2699797892 >> v1:172.21.15.132:6789/0 conn(0x7f587c0090b0 legacy=0x7f587c009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:35.225 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:27:35.225 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:35.225 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:27:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 40, 2024-01-24T23:27:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:27:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:27:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:27:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:27:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:27:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:27:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:27:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:27:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:27:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:27:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 4, 2024-01-24T23:27:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:27:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:27:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:27:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:27:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:27:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:27:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:27:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:27:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:27:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:27:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:27:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:27:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:27:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:27:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:27:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:27:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:27:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:27:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:27:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:27:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:27:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:27:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:27:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:27:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:27:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:27:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:27:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:27:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:27:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:27:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:27:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:27:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:27:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:27:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:27:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:27:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:27:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:27:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:27:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:27:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:27:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:27:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:27:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:27:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:27:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:27:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:27:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:27:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:27:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:27:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:35.244 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.244 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.244 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:27:35.244 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:27:35.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f58a3730640 1 -- 172.21.15.115:0/2699797892 >> v1:172.21.15.132:6812/1835401922 conn(0x7f58800378f0 legacy=0x7f5880039db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f58a3730640 10 monclient: shutdown 2024-01-24T23:27:35.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f58a3730640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:35.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f58a3730640 1 -- 172.21.15.115:0/2699797892 >> v1:172.21.15.115:6792/0 conn(0x7f589c13d2f0 legacy=0x7f589c1d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.245 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f58a1ccd640 1 -- 172.21.15.115:0/2699797892 reap_dead start 2024-01-24T23:27:35.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f58a1ccd640 1 -- 172.21.15.115:0/2699797892 reap_dead start 2024-01-24T23:27:35.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f58a3730640 1 -- 172.21.15.115:0/2699797892 shutdown_connections 2024-01-24T23:27:35.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f58a3730640 1 -- 172.21.15.115:0/2699797892 >> 172.21.15.115:0/2699797892 conn(0x7f589c0662f0 msgr2=0x7f589c1320d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:35.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f58a3730640 1 -- 172.21.15.115:0/2699797892 shutdown_connections 2024-01-24T23:27:35.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.204+0000 7f58a3730640 1 -- 172.21.15.115:0/2699797892 wait complete. 2024-01-24T23:27:35.246 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:27:35.246 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-01-24T23:27:35.246 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-01-24T23:27:35.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f917a70f640 10 monclient: get_monmap_and_config 2024-01-24T23:27:35.331 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f917a70f640 10 monclient: build_initial_monmap 2024-01-24T23:27:35.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f917a70f640 10 monclient: monmap: 2024-01-24T23:27:35.332 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:35.332 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.332 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:35.334128+0000 2024-01-24T23:27:35.332 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:35.334128+0000 2024-01-24T23:27:35.332 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:35.332 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:35.332 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:35.332 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:35.332 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:35.332 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:35.332 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:35.332 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:35.333 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:35.333 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:35.333 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:35.333 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:35.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f917a70f640 1 Processor -- start 2024-01-24T23:27:35.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f917a70f640 1 -- start start 2024-01-24T23:27:35.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f917a70f640 10 monclient: init 2024-01-24T23:27:35.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f917a70f640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:35.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f917a70f640 10 monclient: _add_conns ranks=[1,5,0,3,6,4,8,2,7] 2024-01-24T23:27:35.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f917a70f640 10 monclient(hunting): picked mon.noname-f con 0x7f917409ff20 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:35.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f917a70f640 10 monclient(hunting): picked mon.noname-h con 0x7f917409f9c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:35.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f917a70f640 10 monclient(hunting): picked mon.noname-a con 0x7f917413b2c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:35.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f917a70f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9174064700 con 0x7f917413b2c0 2024-01-24T23:27:35.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f917a70f640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f917409e760 con 0x7f917409ff20 2024-01-24T23:27:35.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f917a70f640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91740a1160 con 0x7f917409f9c0 2024-01-24T23:27:35.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f917a70f640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:35.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f917a70f640 10 monclient(hunting): authenticate will time out at 1342.659204s 2024-01-24T23:27:35.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f9173fff640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f917409ff20 0x7f91740a0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:46270/0 (socket says 172.21.15.132:46270) 2024-01-24T23:27:35.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f91737fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f917409f9c0 0x7f9174139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:45770/0 (socket says 172.21.15.132:45770) 2024-01-24T23:27:35.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f9173fff640 1 -- 172.21.15.132:0/2743839079 learned_addr learned my addr 172.21.15.132:0/2743839079 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:35.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f9172ffd640 1 -- 172.21.15.132:0/2743839079 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3973247127 0 0) 0x7f9174064700 con 0x7f917413b2c0 2024-01-24T23:27:35.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f9172ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f9172ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f9172ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f9172ffd640 10 monclient(hunting): my global_id is 95004 2024-01-24T23:27:35.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f9172ffd640 1 -- 172.21.15.132:0/2743839079 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f915c003530 con 0x7f917413b2c0 2024-01-24T23:27:35.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f9172ffd640 1 -- 172.21.15.132:0/2743839079 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1374355375 0 0) 0x7f91740a1160 con 0x7f917409f9c0 2024-01-24T23:27:35.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f9172ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f9172ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f9172ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f9172ffd640 10 monclient(hunting): my global_id is 6692 2024-01-24T23:27:35.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f9172ffd640 1 -- 172.21.15.132:0/2743839079 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9174064700 con 0x7f917409f9c0 2024-01-24T23:27:35.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f9172ffd640 1 -- 172.21.15.132:0/2743839079 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 772861283 0 0) 0x7f917409e760 con 0x7f917409ff20 2024-01-24T23:27:35.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f9172ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f9172ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f9172ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f9172ffd640 10 monclient(hunting): my global_id is 124273 2024-01-24T23:27:35.342 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.330+0000 7f9172ffd640 1 -- 172.21.15.132:0/2743839079 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91740a1160 con 0x7f917409ff20 2024-01-24T23:27:35.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9172ffd640 1 -- 172.21.15.132:0/2743839079 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 529667973 0 0) 0x7f915c003530 con 0x7f917413b2c0 2024-01-24T23:27:35.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9172ffd640 1 -- 172.21.15.132:0/2743839079 >> v1:172.21.15.132:6791/0 conn(0x7f917409f9c0 legacy=0x7f9174139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9172ffd640 1 -- 172.21.15.132:0/2743839079 >> v1:172.21.15.132:6789/0 conn(0x7f917409ff20 legacy=0x7f91740a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9172ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:27:35.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9172ffd640 1 monclient: found mon.noname-a 2024-01-24T23:27:35.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9172ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:35.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9172ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:27:35.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9172ffd640 1 -- 172.21.15.132:0/2743839079 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91740a0a10 con 0x7f917413b2c0 2024-01-24T23:27:35.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9172ffd640 10 monclient: _finish_auth 0 2024-01-24T23:27:35.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9172ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:27:35.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9172ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:35.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9172ffd640 1 -- 172.21.15.132:0/2743839079 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f916c0017c0 con 0x7f917413b2c0 2024-01-24T23:27:35.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9172ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9172ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:27:35.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9172ffd640 10 monclient: dump: 2024-01-24T23:27:35.344 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:35.344 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.344 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.344 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.344 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.344 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:35.344 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.344 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.344 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.344 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.344 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.344 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.344 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.344 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.345 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.345 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:35.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 5 monclient: authenticate success, global_id 95004 2024-01-24T23:27:35.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:35.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9172ffd640 1 -- 172.21.15.132:0/2743839079 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f916c001a20 con 0x7f917413b2c0 2024-01-24T23:27:35.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9172ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:35.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9172ffd640 1 -- 172.21.15.132:0/2743839079 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f916c003960 con 0x7f917413b2c0 2024-01-24T23:27:35.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:35.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 4 set_mon_vals no callback set 2024-01-24T23:27:35.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9172ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9172ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:27:35.345 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9172ffd640 10 monclient: dump: 2024-01-24T23:27:35.345 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:35.345 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.345 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.345 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.346 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.346 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:35.346 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.346 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.346 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.346 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.346 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.346 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.346 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.346 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.346 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.346 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:35.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 10 monclient: shutdown 2024-01-24T23:27:35.346 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:35.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 1 -- 172.21.15.132:0/2743839079 >> v1:172.21.15.115:6789/0 conn(0x7f917413b2c0 legacy=0x7f917413d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 1 -- 172.21.15.132:0/2743839079 shutdown_connections 2024-01-24T23:27:35.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 1 -- 172.21.15.132:0/2743839079 >> 172.21.15.132:0/2743839079 conn(0x7f91740a5a10 msgr2=0x7f91740a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:35.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 1 -- 172.21.15.132:0/2743839079 shutdown_connections 2024-01-24T23:27:35.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 1 -- 172.21.15.132:0/2743839079 wait complete. 2024-01-24T23:27:35.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 10 monclient: build_initial_monmap 2024-01-24T23:27:35.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 10 monclient: monmap: 2024-01-24T23:27:35.347 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:35.347 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:35.347 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:35.347 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:35.347 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:35.347 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:35.347 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:35.348 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:35.348 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:35.348 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:35.348 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:35.348 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:35.348 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:35.348 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:35.348 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:35.348 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:35.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 1 Processor -- start 2024-01-24T23:27:35.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 1 -- start start 2024-01-24T23:27:35.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 10 monclient: init 2024-01-24T23:27:35.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:35.348 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 10 monclient: _add_conns ranks=[6,8,2,1,0,7,4,5,3] 2024-01-24T23:27:35.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 10 monclient(hunting): picked mon.noname-d con 0x7f917409f9c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:35.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 10 monclient(hunting): picked mon.noname-e con 0x7f917409ff20 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:35.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 10 monclient(hunting): picked mon.noname-b con 0x7f917413b2c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:35.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9174071680 con 0x7f917413b2c0 2024-01-24T23:27:35.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f917411e910 con 0x7f917409f9c0 2024-01-24T23:27:35.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91741ca260 con 0x7f917409ff20 2024-01-24T23:27:35.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:35.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f917a70f640 10 monclient(hunting): authenticate will time out at 1342.662678s 2024-01-24T23:27:35.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9173fff640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f917409f9c0 0x7f91741c9440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:41234/0 (socket says 172.21.15.132:41234) 2024-01-24T23:27:35.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9173fff640 1 -- 172.21.15.132:0/58969232 learned_addr learned my addr 172.21.15.132:0/58969232 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:35.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4275334789 0 0) 0x7f91741ca260 con 0x7f917409ff20 2024-01-24T23:27:35.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.349 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 10 monclient(hunting): my global_id is 85886 2024-01-24T23:27:35.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9160003530 con 0x7f917409ff20 2024-01-24T23:27:35.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1934894730 0 0) 0x7f917411e910 con 0x7f917409f9c0 2024-01-24T23:27:35.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 10 monclient(hunting): my global_id is 6873 2024-01-24T23:27:35.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91741ca260 con 0x7f917409f9c0 2024-01-24T23:27:35.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2622181478 0 0) 0x7f9174071680 con 0x7f917413b2c0 2024-01-24T23:27:35.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 10 monclient(hunting): my global_id is 124292 2024-01-24T23:27:35.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f917411e910 con 0x7f917413b2c0 2024-01-24T23:27:35.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 799567831 0 0) 0x7f9160003530 con 0x7f917409ff20 2024-01-24T23:27:35.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9174071680 con 0x7f917409ff20 2024-01-24T23:27:35.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f91680018e0 con 0x7f917409ff20 2024-01-24T23:27:35.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:27:35.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 10 monclient(hunting): dump: 2024-01-24T23:27:35.351 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:35.351 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.351 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.351 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.351 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.351 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:35.351 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.351 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.351 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.351 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.352 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.352 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.352 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.352 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.352 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.352 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:35.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:35.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 807683184 0 0) 0x7f91741ca260 con 0x7f917409f9c0 2024-01-24T23:27:35.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9160003530 con 0x7f917409f9c0 2024-01-24T23:27:35.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f91640012e0 con 0x7f917409f9c0 2024-01-24T23:27:35.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:35.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 10 monclient(hunting): dump: 2024-01-24T23:27:35.352 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:35.352 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.352 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.353 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.353 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.353 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:35.353 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.353 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.353 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.353 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.353 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.353 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.353 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.353 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.353 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.353 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:35.353 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3961027463 0 0) 0x7f917411e910 con 0x7f917413b2c0 2024-01-24T23:27:35.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f91741ca260 con 0x7f917413b2c0 2024-01-24T23:27:35.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f916c001780 con 0x7f917413b2c0 2024-01-24T23:27:35.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:27:35.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 10 monclient(hunting): dump: 2024-01-24T23:27:35.354 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:35.354 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.354 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.354 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.354 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.354 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:35.354 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.354 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.354 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.354 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.355 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.355 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.355 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.355 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.355 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.355 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:35.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.334+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 496062264 0 0) 0x7f9174071680 con 0x7f917409ff20 2024-01-24T23:27:35.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 >> v1:172.21.15.115:6792/0 conn(0x7f917409f9c0 legacy=0x7f91741c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 >> v1:172.21.15.115:6790/0 conn(0x7f917413b2c0 legacy=0x7f91741d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f9170ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:27:35.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f9170ff9640 1 monclient: found mon.e 2024-01-24T23:27:35.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f9170ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:35.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f9170ff9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:27:35.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9174071960 con 0x7f917409ff20 2024-01-24T23:27:35.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f9170ff9640 10 monclient: _finish_auth 0 2024-01-24T23:27:35.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f9170ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:27:35.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f9170ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:35.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f917a70f640 5 monclient: authenticate success, global_id 85886 2024-01-24T23:27:35.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f917a70f640 10 monclient: _renew_subs 2024-01-24T23:27:35.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f917a70f640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:27:35.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f917a70f640 1 -- 172.21.15.132:0/58969232 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f917411c620 con 0x7f917409ff20 2024-01-24T23:27:35.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f917a70f640 10 monclient: _renew_subs 2024-01-24T23:27:35.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f917a70f640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:27:35.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f917a70f640 1 -- 172.21.15.132:0/58969232 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f917411ced0 con 0x7f917409ff20 2024-01-24T23:27:35.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9168004e20 con 0x7f917409ff20 2024-01-24T23:27:35.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f9170ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:35.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f91680052d0 con 0x7f917409ff20 2024-01-24T23:27:35.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f9170ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f91727fc640 4 set_mon_vals no callback set 2024-01-24T23:27:35.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f9170ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:27:35.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f9170ff9640 10 monclient: dump: 2024-01-24T23:27:35.357 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:35.357 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.357 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.357 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.357 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.357 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:35.357 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.357 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.357 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.358 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.358 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.358 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.358 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.358 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.358 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.358 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:35.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f917a70f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:35.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f917a70f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:35.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f917a70f640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:27:35.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f917a70f640 1 -- 172.21.15.132:0/58969232 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9154005090 con 0x7f917409ff20 2024-01-24T23:27:35.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f916800fd50 con 0x7f917409ff20 2024-01-24T23:27:35.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(220..220 src has 1..220) v4 ==== 12698+0+0 (unknown 1413254373 0 0) 0x7f9168045d50 con 0x7f917409ff20 2024-01-24T23:27:35.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.338+0000 7f9173fff640 10 monclient: get_auth_request con 0x7f9160037c40 auth_method 0 2024-01-24T23:27:35.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.342+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f9168014aa0 con 0x7f917409ff20 2024-01-24T23:27:35.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.342+0000 7f9170ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:35.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.342+0000 7f9170ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:35.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f1054273640 10 monclient: get_monmap_and_config 2024-01-24T23:27:35.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f1054273640 10 monclient: build_initial_monmap 2024-01-24T23:27:35.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f1054273640 10 monclient: monmap: 2024-01-24T23:27:35.362 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:35.362 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.362 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:35.364196+0000 2024-01-24T23:27:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:35.364196+0000 2024-01-24T23:27:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:35.363 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f1054273640 1 Processor -- start 2024-01-24T23:27:35.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f1054273640 1 -- start start 2024-01-24T23:27:35.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f1054273640 10 monclient: init 2024-01-24T23:27:35.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f1054273640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:35.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f1054273640 10 monclient: _add_conns ranks=[6,2,7,4,3,1,5,0,8] 2024-01-24T23:27:35.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f1054273640 10 monclient(hunting): picked mon.noname-d con 0x7f104c0a1910 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:35.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f1054273640 10 monclient(hunting): picked mon.noname-b con 0x7f104c0a2400 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:35.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f1054273640 10 monclient(hunting): picked mon.noname-i con 0x7f104c09bca0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:35.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f1054273640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f104c04ed70 con 0x7f104c0a2400 2024-01-24T23:27:35.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f1054273640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f104c09b630 con 0x7f104c0a1910 2024-01-24T23:27:35.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f1054273640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f104c09c110 con 0x7f104c09bca0 2024-01-24T23:27:35.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f1054273640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:35.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f1054273640 10 monclient(hunting): authenticate will time out at 1334.286765s 2024-01-24T23:27:35.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105200f640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f104c0a1910 0x7f104c0a1cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:54818/0 (socket says 172.21.15.115:54818) 2024-01-24T23:27:35.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105200f640 1 -- 172.21.15.115:0/189975414 learned_addr learned my addr 172.21.15.115:0/189975414 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:35.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105180e640 1 --1- 172.21.15.115:0/189975414 >> v1:172.21.15.115:6790/0 conn(0x7f104c0a2400 0x7f104c13bf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:40384/0 (socket says 172.21.15.115:40384) 2024-01-24T23:27:35.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 1 -- 172.21.15.115:0/189975414 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912874084 0 0) 0x7f104c04ed70 con 0x7f104c0a2400 2024-01-24T23:27:35.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 10 monclient(hunting): my global_id is 124301 2024-01-24T23:27:35.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 1 -- 172.21.15.115:0/189975414 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1038003530 con 0x7f104c0a2400 2024-01-24T23:27:35.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 1 -- 172.21.15.115:0/189975414 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024315895 0 0) 0x7f104c09b630 con 0x7f104c0a1910 2024-01-24T23:27:35.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 10 monclient(hunting): my global_id is 6882 2024-01-24T23:27:35.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 1 -- 172.21.15.115:0/189975414 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f104c04ed70 con 0x7f104c0a1910 2024-01-24T23:27:35.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 1 -- 172.21.15.115:0/189975414 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2391368237 0 0) 0x7f104c09c110 con 0x7f104c09bca0 2024-01-24T23:27:35.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 10 monclient(hunting): my global_id is 124279 2024-01-24T23:27:35.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 1 -- 172.21.15.115:0/189975414 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f104c09b630 con 0x7f104c09bca0 2024-01-24T23:27:35.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 1 -- 172.21.15.115:0/189975414 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3157176353 0 0) 0x7f1038003530 con 0x7f104c0a2400 2024-01-24T23:27:35.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 1 -- 172.21.15.115:0/189975414 >> v1:172.21.15.132:6792/0 conn(0x7f104c09bca0 legacy=0x7f104c13f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 1 -- 172.21.15.115:0/189975414 >> v1:172.21.15.115:6792/0 conn(0x7f104c0a1910 legacy=0x7f104c0a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 10 monclient: _finish_hunting 0 2024-01-24T23:27:35.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 1 monclient: found mon.noname-b 2024-01-24T23:27:35.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:35.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:27:35.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 1 -- 172.21.15.115:0/189975414 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104c09afe0 con 0x7f104c0a2400 2024-01-24T23:27:35.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 10 monclient: _finish_auth 0 2024-01-24T23:27:35.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 10 monclient: _check_auth_tickets 2024-01-24T23:27:35.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:35.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f1054273640 5 monclient: authenticate success, global_id 124301 2024-01-24T23:27:35.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f1054273640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:35.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 1 -- 172.21.15.115:0/189975414 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f103c0017c0 con 0x7f104c0a2400 2024-01-24T23:27:35.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:27:35.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 10 monclient: dump: 2024-01-24T23:27:35.375 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.375 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.375 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.375 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.375 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.375 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.375 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.375 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.375 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.375 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.375 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.375 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.376 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.376 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.376 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.376 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f1054273640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:35.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 1 -- 172.21.15.115:0/189975414 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f103c001aa0 con 0x7f104c0a2400 2024-01-24T23:27:35.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:35.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f1054273640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:35.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f1054273640 4 set_mon_vals no callback set 2024-01-24T23:27:35.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 1 -- 172.21.15.115:0/189975414 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f103c004b30 con 0x7f104c0a2400 2024-01-24T23:27:35.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:27:35.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.360+0000 7f105100d640 10 monclient: dump: 2024-01-24T23:27:35.377 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.377 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.377 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.377 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.377 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.377 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.377 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.377 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.377 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.377 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.377 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.377 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.377 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.377 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.378 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.378 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 10 monclient: shutdown 2024-01-24T23:27:35.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:35.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 1 -- 172.21.15.115:0/189975414 >> v1:172.21.15.115:6790/0 conn(0x7f104c0a2400 legacy=0x7f104c13bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 1 -- 172.21.15.115:0/189975414 shutdown_connections 2024-01-24T23:27:35.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 1 -- 172.21.15.115:0/189975414 >> 172.21.15.115:0/189975414 conn(0x7f104c0a6500 msgr2=0x7f104c0a2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:35.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 1 -- 172.21.15.115:0/189975414 shutdown_connections 2024-01-24T23:27:35.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 1 -- 172.21.15.115:0/189975414 wait complete. 2024-01-24T23:27:35.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 10 monclient: build_initial_monmap 2024-01-24T23:27:35.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 10 monclient: monmap: 2024-01-24T23:27:35.378 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:35.378 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:35.378 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:35.378 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:35.378 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:35.379 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:35.379 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:35.379 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:35.379 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:35.379 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:35.379 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:35.379 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:35.379 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:35.379 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:35.379 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:35.379 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 1 Processor -- start 2024-01-24T23:27:35.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 1 -- start start 2024-01-24T23:27:35.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 10 monclient: init 2024-01-24T23:27:35.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:35.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 10 monclient: _add_conns ranks=[4,1,0,7,8,6,2,3,5] 2024-01-24T23:27:35.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 10 monclient(hunting): picked mon.noname-c con 0x7f104c09bca0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:35.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 10 monclient(hunting): picked mon.noname-f con 0x7f104c0a1910 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:35.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 10 monclient(hunting): picked mon.noname-a con 0x7f104c0a2400 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:35.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f104c09ca40 con 0x7f104c0a2400 2024-01-24T23:27:35.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f104c09c680 con 0x7f104c0a1910 2024-01-24T23:27:35.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f104c121540 con 0x7f104c09bca0 2024-01-24T23:27:35.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:35.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 10 monclient(hunting): authenticate will time out at 1334.289820s 2024-01-24T23:27:35.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f105200f640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f104c09bca0 0x7f104c133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:57684/0 (socket says 172.21.15.115:57684) 2024-01-24T23:27:35.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f105200f640 1 -- 172.21.15.115:0/2944141871 learned_addr learned my addr 172.21.15.115:0/2944141871 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:35.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 581781580 0 0) 0x7f104c121540 con 0x7f104c09bca0 2024-01-24T23:27:35.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient(hunting): my global_id is 94882 2024-01-24T23:27:35.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1030003530 con 0x7f104c09bca0 2024-01-24T23:27:35.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2214839015 0 0) 0x7f104c09ca40 con 0x7f104c0a2400 2024-01-24T23:27:35.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient(hunting): my global_id is 95013 2024-01-24T23:27:35.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f104c121540 con 0x7f104c0a2400 2024-01-24T23:27:35.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2419678885 0 0) 0x7f104c09c680 con 0x7f104c0a1910 2024-01-24T23:27:35.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient(hunting): my global_id is 124282 2024-01-24T23:27:35.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f104c09ca40 con 0x7f104c0a1910 2024-01-24T23:27:35.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3196331347 0 0) 0x7f1030003530 con 0x7f104c09bca0 2024-01-24T23:27:35.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f104c09c680 con 0x7f104c09bca0 2024-01-24T23:27:35.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3980677971 0 0) 0x7f104c121540 con 0x7f104c0a2400 2024-01-24T23:27:35.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1030003530 con 0x7f104c0a2400 2024-01-24T23:27:35.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f10480013d0 con 0x7f104c09bca0 2024-01-24T23:27:35.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:27:35.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient(hunting): dump: 2024-01-24T23:27:35.382 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.382 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.382 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.382 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.382 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.382 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.383 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.383 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.383 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.383 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.383 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.383 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.383 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.383 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.383 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.383 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:35.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1044001d60 con 0x7f104c0a2400 2024-01-24T23:27:35.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:27:35.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient(hunting): dump: 2024-01-24T23:27:35.384 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.384 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.384 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.384 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.384 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.384 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.384 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.384 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.384 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.384 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.384 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.384 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.384 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.384 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.385 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.385 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2996040199 0 0) 0x7f104c09ca40 con 0x7f104c0a1910 2024-01-24T23:27:35.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f104c121540 con 0x7f104c0a1910 2024-01-24T23:27:35.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f103c001780 con 0x7f104c0a1910 2024-01-24T23:27:35.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:27:35.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient(hunting): dump: 2024-01-24T23:27:35.385 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.385 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.385 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.385 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.385 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.385 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.386 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.386 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.386 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.386 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.386 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.386 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.386 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.386 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.386 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.386 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3502488806 0 0) 0x7f104c09c680 con 0x7f104c09bca0 2024-01-24T23:27:35.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 >> v1:172.21.15.132:6789/0 conn(0x7f104c0a1910 legacy=0x7f104c1d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 >> v1:172.21.15.115:6789/0 conn(0x7f104c0a2400 legacy=0x7f104c1d3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:27:35.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 monclient: found mon.c 2024-01-24T23:27:35.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:35.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:35.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104c11f250 con 0x7f104c09bca0 2024-01-24T23:27:35.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient: _finish_auth 0 2024-01-24T23:27:35.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:27:35.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:35.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 5 monclient: authenticate success, global_id 94882 2024-01-24T23:27:35.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 10 monclient: _renew_subs 2024-01-24T23:27:35.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:35.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 1 -- 172.21.15.115:0/2944141871 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f104c121260 con 0x7f104c09bca0 2024-01-24T23:27:35.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 10 monclient: _renew_subs 2024-01-24T23:27:35.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:35.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 1 -- 172.21.15.115:0/2944141871 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f104c1344a0 con 0x7f104c09bca0 2024-01-24T23:27:35.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f10480015c0 con 0x7f104c09bca0 2024-01-24T23:27:35.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:35.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1048004de0 con 0x7f104c09bca0 2024-01-24T23:27:35.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:35.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f105080c640 4 set_mon_vals no callback set 2024-01-24T23:27:35.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 10 monclient: dump: 2024-01-24T23:27:35.388 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.388 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.388 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.388 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.388 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.388 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.388 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.388 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.388 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.389 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.389 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.389 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.389 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.389 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.389 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.389 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:35.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1054273640 1 -- 172.21.15.115:0/2944141871 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f10340065d0 con 0x7f1034002870 2024-01-24T23:27:35.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f104800f860 con 0x7f104c09bca0 2024-01-24T23:27:35.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(220..220 src has 1..220) v4 ==== 12698+0+0 (unknown 1413254373 0 0) 0x7f1048045ad0 con 0x7f104c09bca0 2024-01-24T23:27:35.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1052810640 10 monclient: get_auth_request con 0x7f1030037b40 auth_method 0 2024-01-24T23:27:35.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.364+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516195475 0 0) 0x7f10340065d0 con 0x7f1034002870 2024-01-24T23:27:35.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.368+0000 7f1042ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.368+0000 7f1042ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.368+0000 7f1042ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.368+0000 7f1042ffd640 10 monclient(hunting): my global_id is 6663 2024-01-24T23:27:35.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.368+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f104c09c680 con 0x7f1034002870 2024-01-24T23:27:35.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.368+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4157667957 0 0) 0x7f104c09c680 con 0x7f1034002870 2024-01-24T23:27:35.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.368+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f10340065d0 con 0x7f1034002870 2024-01-24T23:27:35.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.368+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f103c001090 con 0x7f1034002870 2024-01-24T23:27:35.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.368+0000 7f1042ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.368+0000 7f1042ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:35.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.368+0000 7f1042ffd640 10 monclient: dump: 2024-01-24T23:27:35.390 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.390 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.390 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.390 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.391 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.391 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.391 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.391 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.391 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.391 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.391 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.391 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.391 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.391 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.391 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.391 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.368+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 543522646 0 0) 0x7f10340065d0 con 0x7f1034002870 2024-01-24T23:27:35.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.368+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 --> v1:172.21.15.132:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f103400a120 con 0x7f1034002870 2024-01-24T23:27:35.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.368+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f103c0052d0 con 0x7f1034002870 2024-01-24T23:27:35.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.368+0000 7f1042ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:35.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.368+0000 7f1042ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:35.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.368+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 >> v1:172.21.15.132:6790/0 conn(0x7f1034002870 legacy=0x7f1034004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.368+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 >> v1:172.21.15.132:6790/0 conn(0x7f1034002870 legacy=0x7f1034004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:35.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.372+0000 7f1054273640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:27:35.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.372+0000 7f1054273640 1 -- 172.21.15.115:0/2944141871 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1034009e10 con 0x7f10340090d0 2024-01-24T23:27:35.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.372+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 246190314 0 0) 0x7f1034009e10 con 0x7f10340090d0 2024-01-24T23:27:35.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.372+0000 7f1042ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.372+0000 7f1042ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.372+0000 7f1042ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.372+0000 7f1042ffd640 10 monclient(hunting): my global_id is 6672 2024-01-24T23:27:35.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.372+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10340065d0 con 0x7f10340090d0 2024-01-24T23:27:35.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.372+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3350570409 0 0) 0x7f10340065d0 con 0x7f10340090d0 2024-01-24T23:27:35.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.372+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1034009e10 con 0x7f10340090d0 2024-01-24T23:27:35.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.372+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f103c005a40 con 0x7f10340090d0 2024-01-24T23:27:35.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.372+0000 7f1042ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.372+0000 7f1042ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:35.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.372+0000 7f1042ffd640 10 monclient: dump: 2024-01-24T23:27:35.394 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.394 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.394 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.394 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.394 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.394 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.394 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.394 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.394 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.395 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.395 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.395 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.395 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.395 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.395 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.395 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.372+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2834438995 0 0) 0x7f1034009e10 con 0x7f10340090d0 2024-01-24T23:27:35.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.372+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 --> v1:172.21.15.132:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f103400c310 con 0x7f10340090d0 2024-01-24T23:27:35.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.372+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 2859269636) 0x7f103c007fb0 con 0x7f10340090d0 2024-01-24T23:27:35.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.372+0000 7f1042ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:27:35.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.372+0000 7f1042ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:35.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.372+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 >> v1:172.21.15.132:6790/0 conn(0x7f10340090d0 legacy=0x7f1034009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.372+0000 7f1042ffd640 1 -- 172.21.15.115:0/2944141871 >> v1:172.21.15.132:6790/0 conn(0x7f10340090d0 legacy=0x7f1034009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:35.396 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:27:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:27:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 40, 2024-01-24T23:27:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:27:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:27:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:27:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:27:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:27:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:27:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:27:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:27:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:27:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:27:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 4, 2024-01-24T23:27:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:27:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:27:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:27:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:27:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:27:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:27:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:27:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:27:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:27:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:27:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:27:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:27:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:27:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:27:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:27:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:27:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:27:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:27:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:27:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:27:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:27:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:27:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:27:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:27:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:27:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:27:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:27:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:27:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:27:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:27:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:27:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:27:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:27:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:27:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:27:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:27:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:27:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:27:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:27:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:27:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:27:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:27:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:27:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:27:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:27:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:27:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:27:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:27:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:27:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:27:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:27:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:27:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.415 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:35.415 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:35.415 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 4 2024-01-24T23:27:35.415 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.415 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.415 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:27:35.415 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:27:35.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.376+0000 7f1054273640 1 -- 172.21.15.115:0/2944141871 >> v1:172.21.15.132:6812/1835401922 conn(0x7f1030037b40 legacy=0x7f103003a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.376+0000 7f1054273640 10 monclient: shutdown 2024-01-24T23:27:35.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.376+0000 7f1054273640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:35.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.376+0000 7f1054273640 1 -- 172.21.15.115:0/2944141871 >> v1:172.21.15.115:6791/0 conn(0x7f104c09bca0 legacy=0x7f104c133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.376+0000 7f1052810640 1 -- 172.21.15.115:0/2944141871 reap_dead start 2024-01-24T23:27:35.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.376+0000 7f1052810640 1 -- 172.21.15.115:0/2944141871 reap_dead start 2024-01-24T23:27:35.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.376+0000 7f1054273640 1 -- 172.21.15.115:0/2944141871 shutdown_connections 2024-01-24T23:27:35.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.376+0000 7f1054273640 1 -- 172.21.15.115:0/2944141871 >> 172.21.15.115:0/2944141871 conn(0x7f104c0a6500 msgr2=0x7f104c1be7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:35.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.376+0000 7f1054273640 1 -- 172.21.15.115:0/2944141871 shutdown_connections 2024-01-24T23:27:35.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.376+0000 7f1054273640 1 -- 172.21.15.115:0/2944141871 wait complete. 2024-01-24T23:27:35.417 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:27:35.417 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-01-24T23:27:35.417 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-01-24T23:27:35.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.502+0000 7f917a70f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h9"}] 2024-01-24T23:27:35.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.502+0000 7f917a70f640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h9"}] 2024-01-24T23:27:35.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.502+0000 7f917a70f640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:27:35.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:35.502+0000 7f917a70f640 1 -- 172.21.15.132:0/58969232 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush rm", "name": "h9"} v 0) v1 -- 0x7f9154005340 con 0x7f917409ff20 2024-01-24T23:27:35.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f7199c0f640 10 monclient: get_monmap_and_config 2024-01-24T23:27:35.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f7199c0f640 10 monclient: build_initial_monmap 2024-01-24T23:27:35.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f7199c0f640 10 monclient: monmap: 2024-01-24T23:27:35.538 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:35.538 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.538 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:35.539815+0000 2024-01-24T23:27:35.538 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:35.539815+0000 2024-01-24T23:27:35.538 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:35.538 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.538 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:35.538 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:35.539 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:35.539 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:35.539 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:35.539 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:35.539 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:35.539 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:35.539 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:35.539 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f7199c0f640 1 Processor -- start 2024-01-24T23:27:35.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f7199c0f640 1 -- start start 2024-01-24T23:27:35.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f7199c0f640 10 monclient: init 2024-01-24T23:27:35.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f7199c0f640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:35.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f7199c0f640 10 monclient: _add_conns ranks=[6,5,7,1,4,3,0,2,8] 2024-01-24T23:27:35.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f7199c0f640 10 monclient(hunting): picked mon.noname-d con 0x7f7194134d50 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:35.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f7199c0f640 10 monclient(hunting): picked mon.noname-h con 0x7f71940a20b0 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:35.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f7199c0f640 10 monclient(hunting): picked mon.noname-i con 0x7f7194134890 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:35.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f7199c0f640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f719404f7e0 con 0x7f71940a20b0 2024-01-24T23:27:35.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f7199c0f640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f719409d6f0 con 0x7f7194134d50 2024-01-24T23:27:35.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f7199c0f640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71940a1230 con 0x7f7194134890 2024-01-24T23:27:35.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f7199c0f640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:35.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f7199c0f640 10 monclient(hunting): authenticate will time out at 1334.463368s 2024-01-24T23:27:35.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f71937fe640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f7194134d50 0x7f7194135130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:54832/0 (socket says 172.21.15.115:54832) 2024-01-24T23:27:35.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f71937fe640 1 -- 172.21.15.115:0/2757716975 learned_addr learned my addr 172.21.15.115:0/2757716975 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:35.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f71927fc640 1 -- 172.21.15.115:0/2757716975 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996711375 0 0) 0x7f719409d6f0 con 0x7f7194134d50 2024-01-24T23:27:35.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f71927fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f71927fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f71927fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f71927fc640 10 monclient(hunting): my global_id is 6891 2024-01-24T23:27:35.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f71927fc640 1 -- 172.21.15.115:0/2757716975 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f717c003530 con 0x7f7194134d50 2024-01-24T23:27:35.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f71927fc640 1 -- 172.21.15.115:0/2757716975 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3012419252 0 0) 0x7f71940a1230 con 0x7f7194134890 2024-01-24T23:27:35.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f71927fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f71927fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f71927fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f71927fc640 10 monclient(hunting): my global_id is 124288 2024-01-24T23:27:35.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f71927fc640 1 -- 172.21.15.115:0/2757716975 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f719409d6f0 con 0x7f7194134890 2024-01-24T23:27:35.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f71927fc640 1 -- 172.21.15.115:0/2757716975 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2704732730 0 0) 0x7f717c003530 con 0x7f7194134d50 2024-01-24T23:27:35.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f71927fc640 1 -- 172.21.15.115:0/2757716975 >> v1:172.21.15.132:6792/0 conn(0x7f7194134890 legacy=0x7f719413d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.536+0000 7f71927fc640 1 -- 172.21.15.115:0/2757716975 >> v1:172.21.15.132:6791/0 conn(0x7f71940a20b0 legacy=0x7f7194139bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f71927fc640 10 monclient: _finish_hunting 0 2024-01-24T23:27:35.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f71927fc640 1 monclient: found mon.noname-d 2024-01-24T23:27:35.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f71927fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:35.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f71927fc640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:27:35.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f71927fc640 1 -- 172.21.15.115:0/2757716975 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719409d990 con 0x7f7194134d50 2024-01-24T23:27:35.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f71927fc640 10 monclient: _finish_auth 0 2024-01-24T23:27:35.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f71927fc640 10 monclient: _check_auth_tickets 2024-01-24T23:27:35.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f71927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:35.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f71927fc640 1 -- 172.21.15.115:0/2757716975 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f71840017c0 con 0x7f7194134d50 2024-01-24T23:27:35.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 5 monclient: authenticate success, global_id 6891 2024-01-24T23:27:35.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:35.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f71927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f71927fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:27:35.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f71927fc640 10 monclient: dump: 2024-01-24T23:27:35.550 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.550 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.550 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.550 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.550 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.551 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.551 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.551 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.551 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.551 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.551 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.551 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.551 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.551 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.551 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.551 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:35.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f71927fc640 1 -- 172.21.15.115:0/2757716975 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7184001aa0 con 0x7f7194134d50 2024-01-24T23:27:35.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f71927fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:35.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f71927fc640 1 -- 172.21.15.115:0/2757716975 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7184004bf0 con 0x7f7194134d50 2024-01-24T23:27:35.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f71927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f71927fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:35.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f71927fc640 10 monclient: dump: 2024-01-24T23:27:35.552 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.552 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.552 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.552 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.552 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.552 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.552 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.552 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.552 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.552 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.552 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.552 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.553 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 4 set_mon_vals no callback set 2024-01-24T23:27:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 10 monclient: shutdown 2024-01-24T23:27:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 1 -- 172.21.15.115:0/2757716975 >> v1:172.21.15.115:6792/0 conn(0x7f7194134d50 legacy=0x7f7194135130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 1 -- 172.21.15.115:0/2757716975 shutdown_connections 2024-01-24T23:27:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 1 -- 172.21.15.115:0/2757716975 >> 172.21.15.115:0/2757716975 conn(0x7f71940a6220 msgr2=0x7f71940a6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 1 -- 172.21.15.115:0/2757716975 shutdown_connections 2024-01-24T23:27:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 1 -- 172.21.15.115:0/2757716975 wait complete. 2024-01-24T23:27:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 10 monclient: build_initial_monmap 2024-01-24T23:27:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 10 monclient: monmap: 2024-01-24T23:27:35.554 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:35.554 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:35.554 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:35.554 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:35.554 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:35.554 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:35.554 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:35.554 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:35.554 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:35.554 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:35.554 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:35.554 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:35.554 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:35.554 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:35.554 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:35.555 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 1 Processor -- start 2024-01-24T23:27:35.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 1 -- start start 2024-01-24T23:27:35.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 10 monclient: init 2024-01-24T23:27:35.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:35.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 10 monclient: _add_conns ranks=[1,6,0,8,5,2,3,7,4] 2024-01-24T23:27:35.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 10 monclient(hunting): picked mon.noname-f con 0x7f71940a20b0 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:35.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 10 monclient(hunting): picked mon.noname-d con 0x7f7194134890 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:35.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 10 monclient(hunting): picked mon.noname-a con 0x7f7194134d50 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:35.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f719409fe40 con 0x7f7194134d50 2024-01-24T23:27:35.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f719409fa80 con 0x7f71940a20b0 2024-01-24T23:27:35.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7194120f70 con 0x7f7194134890 2024-01-24T23:27:35.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:35.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 10 monclient(hunting): authenticate will time out at 1334.466222s 2024-01-24T23:27:35.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7192ffd640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f7194134890 0x7f7194133da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:54838/0 (socket says 172.21.15.115:54838) 2024-01-24T23:27:35.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7192ffd640 1 -- 172.21.15.115:0/2326814770 learned_addr learned my addr 172.21.15.115:0/2326814770 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:35.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7193fff640 1 --1- 172.21.15.115:0/2326814770 >> v1:172.21.15.115:6789/0 conn(0x7f7194134d50 0x7f71941d1240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:41278/0 (socket says 172.21.15.115:41278) 2024-01-24T23:27:35.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2933350815 0 0) 0x7f719409fe40 con 0x7f7194134d50 2024-01-24T23:27:35.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient(hunting): my global_id is 95022 2024-01-24T23:27:35.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7180003560 con 0x7f7194134d50 2024-01-24T23:27:35.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3444110784 0 0) 0x7f7194120f70 con 0x7f7194134890 2024-01-24T23:27:35.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient(hunting): my global_id is 6900 2024-01-24T23:27:35.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f719409fe40 con 0x7f7194134890 2024-01-24T23:27:35.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2321756625 0 0) 0x7f7180003560 con 0x7f7194134d50 2024-01-24T23:27:35.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7194120f70 con 0x7f7194134d50 2024-01-24T23:27:35.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f718c001de0 con 0x7f7194134d50 2024-01-24T23:27:35.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:27:35.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient(hunting): dump: 2024-01-24T23:27:35.557 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.557 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.557 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.557 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.557 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.558 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.558 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.558 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.558 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.558 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.558 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.558 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.558 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.558 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.558 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.558 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:35.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2988134914 0 0) 0x7f719409fa80 con 0x7f71940a20b0 2024-01-24T23:27:35.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient(hunting): my global_id is 124291 2024-01-24T23:27:35.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7180003560 con 0x7f71940a20b0 2024-01-24T23:27:35.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2612155064 0 0) 0x7f719409fe40 con 0x7f7194134890 2024-01-24T23:27:35.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f719409fa80 con 0x7f7194134890 2024-01-24T23:27:35.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7188001a40 con 0x7f7194134890 2024-01-24T23:27:35.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:35.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient(hunting): dump: 2024-01-24T23:27:35.559 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.559 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.559 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.559 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.560 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.560 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.560 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.560 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.560 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.560 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.560 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.560 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.560 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.560 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.560 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.560 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2243393438 0 0) 0x7f7194120f70 con 0x7f7194134d50 2024-01-24T23:27:35.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 >> v1:172.21.15.115:6792/0 conn(0x7f7194134890 legacy=0x7f7194133da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 >> v1:172.21.15.132:6789/0 conn(0x7f71940a20b0 legacy=0x7f71941c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:27:35.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 1 monclient: found mon.a 2024-01-24T23:27:35.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:35.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:27:35.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7194121250 con 0x7f7194134d50 2024-01-24T23:27:35.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient: _finish_auth 0 2024-01-24T23:27:35.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:27:35.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:35.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 5 monclient: authenticate success, global_id 95022 2024-01-24T23:27:35.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 10 monclient: _renew_subs 2024-01-24T23:27:35.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:27:35.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 1 -- 172.21.15.115:0/2326814770 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719411fda0 con 0x7f7194134d50 2024-01-24T23:27:35.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 10 monclient: _renew_subs 2024-01-24T23:27:35.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:27:35.561 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 1 -- 172.21.15.115:0/2326814770 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f71941344b0 con 0x7f7194134d50 2024-01-24T23:27:35.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f718c003c50 con 0x7f7194134d50 2024-01-24T23:27:35.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:35.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f718c002110 con 0x7f7194134d50 2024-01-24T23:27:35.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7191ffb640 4 set_mon_vals no callback set 2024-01-24T23:27:35.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:27:35.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 10 monclient: dump: 2024-01-24T23:27:35.562 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.562 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.562 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.562 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.562 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.562 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.562 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.562 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.563 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.563 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.563 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.563 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.563 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.563 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.563 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.563 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:35.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f7199c0f640 1 -- 172.21.15.115:0/2326814770 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f71740065d0 con 0x7f7174002870 2024-01-24T23:27:35.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.540+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f718c004180 con 0x7f7194134d50 2024-01-24T23:27:35.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.544+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(220..220 src has 1..220) v4 ==== 12698+0+0 (unknown 1413254373 0 0) 0x7f718c045d70 con 0x7f7194134d50 2024-01-24T23:27:35.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.544+0000 7f7192ffd640 10 monclient: get_auth_request con 0x7f71800378d0 auth_method 0 2024-01-24T23:27:35.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.544+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540055316 0 0) 0x7f71740065d0 con 0x7f7174002870 2024-01-24T23:27:35.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.544+0000 7f717bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.544+0000 7f717bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.544+0000 7f717bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.544+0000 7f717bfff640 10 monclient(hunting): my global_id is 6710 2024-01-24T23:27:35.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.544+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7194120f70 con 0x7f7174002870 2024-01-24T23:27:35.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.544+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2631538269 0 0) 0x7f7194120f70 con 0x7f7174002870 2024-01-24T23:27:35.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.544+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f71740065d0 con 0x7f7174002870 2024-01-24T23:27:35.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.544+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7184001780 con 0x7f7174002870 2024-01-24T23:27:35.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.544+0000 7f717bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.544+0000 7f717bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:35.564 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.544+0000 7f717bfff640 10 monclient: dump: 2024-01-24T23:27:35.564 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.564 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.564 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.564 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.564 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.565 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.565 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.565 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.565 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.565 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.565 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.565 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.565 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.565 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.565 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.565 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.544+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2408997823 0 0) 0x7f71740065d0 con 0x7f7174002870 2024-01-24T23:27:35.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.544+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 --> v1:172.21.15.132:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f717400a120 con 0x7f7174002870 2024-01-24T23:27:35.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.544+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f7184004a20 con 0x7f7174002870 2024-01-24T23:27:35.565 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.544+0000 7f717bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:35.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.544+0000 7f717bfff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:35.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.544+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 >> v1:172.21.15.132:6791/0 conn(0x7f7174002870 legacy=0x7f7174004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.544+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 >> v1:172.21.15.132:6791/0 conn(0x7f7174002870 legacy=0x7f7174004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:35.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.548+0000 7f7199c0f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:27:35.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.548+0000 7f7199c0f640 1 -- 172.21.15.115:0/2326814770 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7174009e10 con 0x7f71740090d0 2024-01-24T23:27:35.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1295488001 0 0) 0x7f7174009e10 con 0x7f71740090d0 2024-01-24T23:27:35.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f717bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f717bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f717bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f717bfff640 10 monclient(hunting): my global_id is 6719 2024-01-24T23:27:35.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71740065d0 con 0x7f71740090d0 2024-01-24T23:27:35.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 429266941 0 0) 0x7f71740065d0 con 0x7f71740090d0 2024-01-24T23:27:35.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7174009e10 con 0x7f71740090d0 2024-01-24T23:27:35.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7184005340 con 0x7f71740090d0 2024-01-24T23:27:35.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f717bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f717bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:35.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f717bfff640 10 monclient: dump: 2024-01-24T23:27:35.568 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.568 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.568 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.568 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.569 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.569 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.569 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.569 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.569 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.569 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.569 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.569 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.569 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.569 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.569 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.569 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1823304763 0 0) 0x7f7174009e10 con 0x7f71740090d0 2024-01-24T23:27:35.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 --> v1:172.21.15.132:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f717400c310 con 0x7f71740090d0 2024-01-24T23:27:35.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7545 (unknown 0 0 3872310729) 0x7f71840075e0 con 0x7f71740090d0 2024-01-24T23:27:35.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f717bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:27:35.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f717bfff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:35.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 >> v1:172.21.15.132:6791/0 conn(0x7f71740090d0 legacy=0x7f7174009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f717bfff640 1 -- 172.21.15.115:0/2326814770 >> v1:172.21.15.132:6791/0 conn(0x7f71740090d0 legacy=0x7f7174009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:35.570 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:27:35.570 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:35.570 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:35.570 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "leader", 2024-01-24T23:27:35.570 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 40, 2024-01-24T23:27:35.570 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:27:35.570 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:27:35.570 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:27:35.571 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:27:35.571 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:27:35.571 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:27:35.571 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:27:35.571 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:27:35.571 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:27:35.571 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:27:35.571 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:35.571 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 5, 2024-01-24T23:27:35.571 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:35.571 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:27:35.571 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:27:35.571 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:35.571 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:35.571 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:35.572 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:35.572 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:35.572 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:35.572 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:35.572 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:35.572 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:35.572 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:35.572 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:35.572 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:35.572 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:27:35.572 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:27:35.572 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:35.572 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:35.572 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:35.573 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:35.573 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:35.573 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:35.573 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:35.573 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:35.573 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:35.573 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:35.573 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:35.573 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.573 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:27:35.573 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:27:35.573 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:27:35.573 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:27:35.574 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:27:35.574 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:27:35.574 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:35.574 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:35.574 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:27:35.574 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:27:35.574 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:27:35.574 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:27:35.574 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:27:35.574 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:27:35.574 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:27:35.574 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:35.574 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:27:35.574 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:35.575 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:35.575 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:35.575 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:35.575 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:35.575 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:35.575 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:35.575 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:35.575 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:35.575 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:35.575 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:35.575 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:35.575 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:27:35.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.576 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:27:35.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.576 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:35.576 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:35.576 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.576 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.576 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.576 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:27:35.576 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.576 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.576 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.576 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:27:35.577 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:27:35.577 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.577 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.577 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.577 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:35.577 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:35.577 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.577 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.577 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.577 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:27:35.577 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.578 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.578 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.578 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:27:35.578 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:27:35.578 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.578 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.578 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.578 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:35.578 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:35.578 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.579 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.579 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.579 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:27:35.579 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.579 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.579 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.579 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:27:35.579 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:27:35.579 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.579 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.579 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.580 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:35.580 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:35.580 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.580 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.580 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.580 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:27:35.580 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.580 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.580 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.580 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:27:35.580 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:27:35.580 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.581 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.581 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.581 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:35.581 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:35.581 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.581 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.581 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.581 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:27:35.581 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.581 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.582 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.582 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:27:35.582 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:27:35.582 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.582 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.582 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.582 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:35.582 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:35.582 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.582 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.583 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.583 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:27:35.583 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.583 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.583 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.583 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:27:35.583 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:27:35.583 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.583 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.583 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.583 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:35.584 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:35.584 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.584 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.584 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.584 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:27:35.584 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.584 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.584 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.584 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:27:35.584 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:27:35.584 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.585 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.585 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.585 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:35.585 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:35.585 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.585 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.585 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.585 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:27:35.585 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.585 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.585 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.586 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:27:35.586 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:27:35.586 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.586 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.586 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.586 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:35.586 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:35.586 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.586 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.587 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.587 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:27:35.587 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.587 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.587 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.587 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:27:35.587 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:27:35.587 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.587 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.587 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.587 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.587 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.587 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:27:35.588 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:27:35.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.588 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:35.588 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:35.588 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:35.588 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.588 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:35.588 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:27:35.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.588 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:35.588 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:35.588 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:35.588 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.588 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.589 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:27:35.589 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:27:35.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f7199c0f640 1 -- 172.21.15.115:0/2326814770 >> v1:172.21.15.132:6812/1835401922 conn(0x7f71800378d0 legacy=0x7f7180039d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f7199c0f640 10 monclient: shutdown 2024-01-24T23:27:35.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f7199c0f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:35.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f7199c0f640 1 -- 172.21.15.115:0/2326814770 >> v1:172.21.15.115:6789/0 conn(0x7f7194134d50 legacy=0x7f71941d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f7193fff640 1 -- 172.21.15.115:0/2326814770 reap_dead start 2024-01-24T23:27:35.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f7193fff640 1 -- 172.21.15.115:0/2326814770 reap_dead start 2024-01-24T23:27:35.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f7199c0f640 1 -- 172.21.15.115:0/2326814770 shutdown_connections 2024-01-24T23:27:35.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f7199c0f640 1 -- 172.21.15.115:0/2326814770 >> 172.21.15.115:0/2326814770 conn(0x7f71940a6220 msgr2=0x7f719409e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:35.590 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f7199c0f640 1 -- 172.21.15.115:0/2326814770 shutdown_connections 2024-01-24T23:27:35.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.552+0000 7f7199c0f640 1 -- 172.21.15.115:0/2326814770 wait complete. 2024-01-24T23:27:35.591 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:27:35.591 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-01-24T23:27:35.591 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-01-24T23:27:35.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 10 monclient: get_monmap_and_config 2024-01-24T23:27:35.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 10 monclient: build_initial_monmap 2024-01-24T23:27:35.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 10 monclient: monmap: 2024-01-24T23:27:35.709 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:35.709 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.709 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:35.710999+0000 2024-01-24T23:27:35.710 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:35.710999+0000 2024-01-24T23:27:35.710 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:35.710 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.710 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:35.710 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:35.710 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:35.710 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:35.710 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:35.710 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:35.710 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:35.710 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:35.710 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:35.710 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 1 Processor -- start 2024-01-24T23:27:35.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 1 -- start start 2024-01-24T23:27:35.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 10 monclient: init 2024-01-24T23:27:35.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:35.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 10 monclient: _add_conns ranks=[5,6,4,3,7,0,2,1,8] 2024-01-24T23:27:35.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 10 monclient(hunting): picked mon.noname-h con 0x7f218009f790 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:35.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 10 monclient(hunting): picked mon.noname-d con 0x7f2180137b50 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:35.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 10 monclient(hunting): picked mon.noname-c con 0x7f218013b660 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:35.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f218004ed70 con 0x7f218013b660 2024-01-24T23:27:35.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2180134d50 con 0x7f218009f790 2024-01-24T23:27:35.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2180066bf0 con 0x7f2180137b50 2024-01-24T23:27:35.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:35.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 10 monclient(hunting): authenticate will time out at 1334.633688s 2024-01-24T23:27:35.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217dd9b640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f2180137b50 0x7f2180139f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:54844/0 (socket says 172.21.15.115:54844) 2024-01-24T23:27:35.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217dd9b640 1 -- 172.21.15.115:0/2147938581 learned_addr learned my addr 172.21.15.115:0/2147938581 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:35.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 1 -- 172.21.15.115:0/2147938581 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1130187238 0 0) 0x7f2180066bf0 con 0x7f2180137b50 2024-01-24T23:27:35.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 10 monclient(hunting): my global_id is 6909 2024-01-24T23:27:35.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 1 -- 172.21.15.115:0/2147938581 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2160003530 con 0x7f2180137b50 2024-01-24T23:27:35.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 1 -- 172.21.15.115:0/2147938581 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437351524 0 0) 0x7f218004ed70 con 0x7f218013b660 2024-01-24T23:27:35.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 10 monclient(hunting): my global_id is 94891 2024-01-24T23:27:35.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 1 -- 172.21.15.115:0/2147938581 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2180066bf0 con 0x7f218013b660 2024-01-24T23:27:35.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 1 -- 172.21.15.115:0/2147938581 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 531539691 0 0) 0x7f2160003530 con 0x7f2180137b50 2024-01-24T23:27:35.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 1 -- 172.21.15.115:0/2147938581 >> v1:172.21.15.132:6791/0 conn(0x7f218009f790 legacy=0x7f218009fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 1 -- 172.21.15.115:0/2147938581 >> v1:172.21.15.115:6791/0 conn(0x7f218013b660 legacy=0x7f218013da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 10 monclient: _finish_hunting 0 2024-01-24T23:27:35.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 1 monclient: found mon.noname-d 2024-01-24T23:27:35.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:35.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:27:35.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 1 -- 172.21.15.115:0/2147938581 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21800a0280 con 0x7f2180137b50 2024-01-24T23:27:35.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 10 monclient: _finish_auth 0 2024-01-24T23:27:35.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 10 monclient: _check_auth_tickets 2024-01-24T23:27:35.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:35.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 5 monclient: authenticate success, global_id 6909 2024-01-24T23:27:35.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 1 -- 172.21.15.115:0/2147938581 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2168001b40 con 0x7f2180137b50 2024-01-24T23:27:35.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:35.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:27:35.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 10 monclient: dump: 2024-01-24T23:27:35.721 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.721 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.721 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.721 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.721 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.721 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.721 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.722 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.722 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.722 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.722 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.722 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.722 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.722 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.722 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.722 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:35.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 1 -- 172.21.15.115:0/2147938581 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2168001d60 con 0x7f2180137b50 2024-01-24T23:27:35.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:35.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:35.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 1 -- 172.21.15.115:0/2147938581 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2168004f70 con 0x7f2180137b50 2024-01-24T23:27:35.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 4 set_mon_vals no callback set 2024-01-24T23:27:35.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:35.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f217d59a640 10 monclient: dump: 2024-01-24T23:27:35.723 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.723 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.723 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.723 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.723 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.723 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.723 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.723 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.723 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.723 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.723 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.723 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.724 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.724 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.724 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.724 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 10 monclient: shutdown 2024-01-24T23:27:35.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:35.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 1 -- 172.21.15.115:0/2147938581 >> v1:172.21.15.115:6792/0 conn(0x7f2180137b50 legacy=0x7f2180139f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 1 -- 172.21.15.115:0/2147938581 shutdown_connections 2024-01-24T23:27:35.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 1 -- 172.21.15.115:0/2147938581 >> 172.21.15.115:0/2147938581 conn(0x7f21800a6500 msgr2=0x7f21800a2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:35.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 1 -- 172.21.15.115:0/2147938581 shutdown_connections 2024-01-24T23:27:35.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 1 -- 172.21.15.115:0/2147938581 wait complete. 2024-01-24T23:27:35.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 10 monclient: build_initial_monmap 2024-01-24T23:27:35.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.708+0000 7f2184bd4640 10 monclient: monmap: 2024-01-24T23:27:35.724 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:35.725 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:35.725 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:35.725 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:35.725 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:35.725 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:35.725 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:35.725 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:35.725 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:35.725 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:35.725 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:35.725 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:35.725 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:35.725 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:35.725 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:35.726 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2184bd4640 1 Processor -- start 2024-01-24T23:27:35.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2184bd4640 1 -- start start 2024-01-24T23:27:35.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2184bd4640 10 monclient: init 2024-01-24T23:27:35.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2184bd4640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:35.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2184bd4640 10 monclient: _add_conns ranks=[1,0,3,2,4,8,5,6,7] 2024-01-24T23:27:35.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2184bd4640 10 monclient(hunting): picked mon.noname-f con 0x7f218009f790 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:35.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2184bd4640 10 monclient(hunting): picked mon.noname-a con 0x7f2180137b50 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:35.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2184bd4640 10 monclient(hunting): picked mon.noname-g con 0x7f218013b660 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:35.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2184bd4640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21800677b0 con 0x7f2180137b50 2024-01-24T23:27:35.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2184bd4640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f218013ab40 con 0x7f218009f790 2024-01-24T23:27:35.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2184bd4640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f218006f530 con 0x7f218013b660 2024-01-24T23:27:35.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2184bd4640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:35.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2184bd4640 10 monclient(hunting): authenticate will time out at 1334.636679s 2024-01-24T23:27:35.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f217dd9b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f2180137b50 0x7f2180130a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:41294/0 (socket says 172.21.15.115:41294) 2024-01-24T23:27:35.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f217dd9b640 1 -- 172.21.15.115:0/38688091 learned_addr learned my addr 172.21.15.115:0/38688091 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:35.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1488830027 0 0) 0x7f21800677b0 con 0x7f2180137b50 2024-01-24T23:27:35.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient(hunting): my global_id is 95031 2024-01-24T23:27:35.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2164003530 con 0x7f2180137b50 2024-01-24T23:27:35.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2090991130 0 0) 0x7f2164003530 con 0x7f2180137b50 2024-01-24T23:27:35.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f21800677b0 con 0x7f2180137b50 2024-01-24T23:27:35.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2168001b00 con 0x7f2180137b50 2024-01-24T23:27:35.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:27:35.728 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient(hunting): dump: 2024-01-24T23:27:35.728 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.728 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.728 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.728 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.728 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.728 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.728 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.728 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.728 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.728 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.728 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.728 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.728 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.729 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.729 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.729 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:35.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2324484541 0 0) 0x7f218006f530 con 0x7f218013b660 2024-01-24T23:27:35.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient(hunting): my global_id is 6681 2024-01-24T23:27:35.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2164003530 con 0x7f218013b660 2024-01-24T23:27:35.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2265003419 0 0) 0x7f21800677b0 con 0x7f2180137b50 2024-01-24T23:27:35.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 >> v1:172.21.15.132:6790/0 conn(0x7f218013b660 legacy=0x7f21801d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 >> v1:172.21.15.132:6789/0 conn(0x7f218009f790 legacy=0x7f2180070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:27:35.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 1 monclient: found mon.a 2024-01-24T23:27:35.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:35.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:27:35.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2180070320 con 0x7f2180137b50 2024-01-24T23:27:35.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient: _finish_auth 0 2024-01-24T23:27:35.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:27:35.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:35.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2184bd4640 5 monclient: authenticate success, global_id 95031 2024-01-24T23:27:35.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2184bd4640 10 monclient: _renew_subs 2024-01-24T23:27:35.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2184bd4640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:27:35.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2184bd4640 1 -- 172.21.15.115:0/38688091 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2180070600 con 0x7f2180137b50 2024-01-24T23:27:35.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2184bd4640 10 monclient: _renew_subs 2024-01-24T23:27:35.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2184bd4640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:27:35.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2184bd4640 1 -- 172.21.15.115:0/38688091 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2180131160 con 0x7f2180137b50 2024-01-24T23:27:35.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2168005230 con 0x7f2180137b50 2024-01-24T23:27:35.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:35.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f21680056e0 con 0x7f2180137b50 2024-01-24T23:27:35.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f217cd99640 4 set_mon_vals no callback set 2024-01-24T23:27:35.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:27:35.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient: dump: 2024-01-24T23:27:35.731 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.731 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.731 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.731 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.731 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.731 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.731 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.732 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.732 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.732 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.732 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.732 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.732 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.732 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.732 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.732 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2184bd4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:35.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2184bd4640 1 -- 172.21.15.115:0/38688091 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f21580065d0 con 0x7f2158002870 2024-01-24T23:27:35.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f2168010160 con 0x7f2180137b50 2024-01-24T23:27:35.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(220..220 src has 1..220) v4 ==== 12698+0+0 (unknown 1413254373 0 0) 0x7f2168045ff0 con 0x7f2180137b50 2024-01-24T23:27:35.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f217ed9d640 10 monclient: get_auth_request con 0x7f2164037090 auth_method 0 2024-01-24T23:27:35.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2246802816 0 0) 0x7f21580065d0 con 0x7f2158002870 2024-01-24T23:27:35.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient(hunting): my global_id is 124297 2024-01-24T23:27:35.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21800677b0 con 0x7f2158002870 2024-01-24T23:27:35.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4265143694 0 0) 0x7f21800677b0 con 0x7f2158002870 2024-01-24T23:27:35.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f21580065d0 con 0x7f2158002870 2024-01-24T23:27:35.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f21780018a0 con 0x7f2158002870 2024-01-24T23:27:35.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:27:35.733 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 10 monclient: dump: 2024-01-24T23:27:35.733 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.733 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.733 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.733 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.734 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.734 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.734 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.734 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.734 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.734 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.734 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.734 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.734 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.734 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.734 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.734 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1162227767 0 0) 0x7f21580065d0 con 0x7f2158002870 2024-01-24T23:27:35.734 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.712+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 --> v1:172.21.15.132:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f215800a120 con 0x7f2158002870 2024-01-24T23:27:35.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.716+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 <== mon.7 v1:172.21.15.132:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f2178001ad0 con 0x7f2158002870 2024-01-24T23:27:35.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.716+0000 7f2176ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:35.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.716+0000 7f2176ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:35.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.716+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 >> v1:172.21.15.132:6792/0 conn(0x7f2158002870 legacy=0x7f2158004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.735 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.716+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 >> v1:172.21.15.132:6792/0 conn(0x7f2158002870 legacy=0x7f2158004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:35.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.720+0000 7f2184bd4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:27:35.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.720+0000 7f2184bd4640 1 -- 172.21.15.115:0/38688091 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2158009df0 con 0x7f21580090b0 2024-01-24T23:27:35.736 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.720+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 988838872 0 0) 0x7f2158009df0 con 0x7f21580090b0 2024-01-24T23:27:35.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.720+0000 7f2176ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.720+0000 7f2176ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.720+0000 7f2176ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.720+0000 7f2176ffd640 10 monclient(hunting): my global_id is 124306 2024-01-24T23:27:35.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.720+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21580065d0 con 0x7f21580090b0 2024-01-24T23:27:35.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.720+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1332394621 0 0) 0x7f21580065d0 con 0x7f21580090b0 2024-01-24T23:27:35.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.720+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2158009df0 con 0x7f21580090b0 2024-01-24T23:27:35.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.720+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2178003bb0 con 0x7f21580090b0 2024-01-24T23:27:35.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.720+0000 7f2176ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.720+0000 7f2176ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:27:35.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.720+0000 7f2176ffd640 10 monclient: dump: 2024-01-24T23:27:35.737 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.737 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.737 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.738 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.738 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.738 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.738 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.738 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.738 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.738 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.738 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.738 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.738 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.738 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.738 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.738 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.724+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3692505523 0 0) 0x7f2158009df0 con 0x7f21580090b0 2024-01-24T23:27:35.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.724+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 --> v1:172.21.15.132:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f215800c310 con 0x7f21580090b0 2024-01-24T23:27:35.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.724+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 <== mon.7 v1:172.21.15.132:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 174227240) 0x7f21780040a0 con 0x7f21580090b0 2024-01-24T23:27:35.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.724+0000 7f2176ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:27:35.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.724+0000 7f2176ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:35.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.724+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 >> v1:172.21.15.132:6792/0 conn(0x7f21580090b0 legacy=0x7f2158009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.739 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.724+0000 7f2176ffd640 1 -- 172.21.15.115:0/38688091 >> v1:172.21.15.132:6792/0 conn(0x7f21580090b0 legacy=0x7f2158009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:35.739 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:27:35.739 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:35.739 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:35.739 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:27:35.739 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 40, 2024-01-24T23:27:35.739 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:27:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:27:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:27:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:27:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:27:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:27:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:27:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:27:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:27:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:27:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 5, 2024-01-24T23:27:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:27:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:27:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:27:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:27:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:27:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:27:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:27:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:27:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:27:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:27:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:27:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:27:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:27:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:27:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:27:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:27:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:27:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:27:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:27:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:27:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:27:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.746 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.746 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.746 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:27:35.746 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:27:35.746 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.746 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.746 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.746 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:35.746 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:35.746 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.747 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.747 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.747 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:27:35.747 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.747 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.747 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.747 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:27:35.747 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:27:35.747 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.747 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.747 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:27:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:27:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:27:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:27:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:27:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:27:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:27:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:27:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:27:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:27:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:27:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:27:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:27:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:27:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:27:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:27:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:27:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:27:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:27:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:27:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:27:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:27:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:27:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:27:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:35.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:35.758 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:27:35.758 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:27:35.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.724+0000 7f2184bd4640 1 -- 172.21.15.115:0/38688091 >> v1:172.21.15.132:6812/1835401922 conn(0x7f2164037090 legacy=0x7f2164039550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.724+0000 7f2184bd4640 10 monclient: shutdown 2024-01-24T23:27:35.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.724+0000 7f2184bd4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:35.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.724+0000 7f2184bd4640 1 -- 172.21.15.115:0/38688091 >> v1:172.21.15.115:6789/0 conn(0x7f2180137b50 legacy=0x7f2180130a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.724+0000 7f217ed9d640 1 -- 172.21.15.115:0/38688091 reap_dead start 2024-01-24T23:27:35.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.724+0000 7f217ed9d640 1 -- 172.21.15.115:0/38688091 reap_dead start 2024-01-24T23:27:35.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.724+0000 7f2184bd4640 1 -- 172.21.15.115:0/38688091 shutdown_connections 2024-01-24T23:27:35.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.724+0000 7f2184bd4640 1 -- 172.21.15.115:0/38688091 >> 172.21.15.115:0/38688091 conn(0x7f21800a6500 msgr2=0x7f2180136360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:35.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.724+0000 7f2184bd4640 1 -- 172.21.15.115:0/38688091 shutdown_connections 2024-01-24T23:27:35.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.724+0000 7f2184bd4640 1 -- 172.21.15.115:0/38688091 wait complete. 2024-01-24T23:27:35.760 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:27:35.760 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a', 'b', 'i'] 2024-01-24T23:27:35.761 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-01-24T23:27:35.761 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-01-24T23:27:35.761 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-01-24T23:27:35.761 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-01-24T23:27:35.761 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:27:35.772 INFO:tasks.ceph.mon.a:Stopped 2024-01-24T23:27:35.772 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-01-24T23:27:35.772 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-01-24T23:27:35.772 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-01-24T23:27:35.772 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:27:35.823 INFO:tasks.ceph.mon.b:Stopped 2024-01-24T23:27:35.823 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-01-24T23:27:35.823 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-01-24T23:27:35.823 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-01-24T23:27:35.823 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:27:35.845 INFO:tasks.ceph.mon.i:Stopped 2024-01-24T23:27:35.845 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 6 2024-01-24T23:27:35.845 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-01-24T23:27:35.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 10 monclient: get_monmap_and_config 2024-01-24T23:27:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 10 monclient: build_initial_monmap 2024-01-24T23:27:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 10 monclient: monmap: 2024-01-24T23:27:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:35.972223+0000 2024-01-24T23:27:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:35.972223+0000 2024-01-24T23:27:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:35.972 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:35.972 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:35.972 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:35.972 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 1 Processor -- start 2024-01-24T23:27:35.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 1 -- start start 2024-01-24T23:27:35.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 10 monclient: init 2024-01-24T23:27:35.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:35.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 10 monclient: _add_conns ranks=[2,6,0,1,5,4,8,7,3] 2024-01-24T23:27:35.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 10 monclient(hunting): picked mon.noname-b con 0x7fd6580662f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:35.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 10 monclient(hunting): picked mon.noname-d con 0x7fd65809b690 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:35.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 10 monclient(hunting): picked mon.noname-a con 0x7fd65809bab0 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:35.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65804ed70 con 0x7fd65809bab0 2024-01-24T23:27:35.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd658099e80 con 0x7fd6580662f0 2024-01-24T23:27:35.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6580666d0 con 0x7fd65809b690 2024-01-24T23:27:35.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:35.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 10 monclient(hunting): authenticate will time out at 1334.894774s 2024-01-24T23:27:35.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd657fff640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fd6580662f0 legacy=0x7fd658136b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:35.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65cc4e640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fd65809bab0 legacy=0x7fd65813f560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:27:35.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd6577fe640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fd65809b690 0x7fd65813be30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:54846/0 (socket says 172.21.15.115:54846) 2024-01-24T23:27:35.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd6577fe640 1 -- 172.21.15.115:0/1601053594 learned_addr learned my addr 172.21.15.115:0/1601053594 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:35.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 1 -- 172.21.15.115:0/1601053594 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2696173569 0 0) 0x7fd6580666d0 con 0x7fd65809b690 2024-01-24T23:27:35.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 10 monclient(hunting): my global_id is 6918 2024-01-24T23:27:35.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 1 -- 172.21.15.115:0/1601053594 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd640003530 con 0x7fd65809b690 2024-01-24T23:27:35.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 1 -- 172.21.15.115:0/1601053594 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 652350229 0 0) 0x7fd640003530 con 0x7fd65809b690 2024-01-24T23:27:35.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 1 -- 172.21.15.115:0/1601053594 >> v1:172.21.15.115:6790/0 conn(0x7fd6580662f0 legacy=0x7fd658136b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:35.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 1 -- 172.21.15.115:0/1601053594 >> v1:172.21.15.115:6789/0 conn(0x7fd65809bab0 legacy=0x7fd65813f560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:35.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:27:35.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 1 monclient: found mon.noname-d 2024-01-24T23:27:35.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:35.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:27:35.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 1 -- 172.21.15.115:0/1601053594 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6580998c0 con 0x7fd65809b690 2024-01-24T23:27:35.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 10 monclient: _finish_auth 0 2024-01-24T23:27:35.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:27:35.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:35.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 1 -- 172.21.15.115:0/1601053594 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd64c0017c0 con 0x7fd65809b690 2024-01-24T23:27:35.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:27:35.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 10 monclient: dump: 2024-01-24T23:27:35.981 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.981 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.981 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.981 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.981 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.981 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.981 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.981 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.981 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.981 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.981 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.981 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.981 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.981 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.981 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.982 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 5 monclient: authenticate success, global_id 6918 2024-01-24T23:27:35.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:35.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 1 -- 172.21.15.115:0/1601053594 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd64c001ae0 con 0x7fd65809b690 2024-01-24T23:27:35.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:35.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 1 -- 172.21.15.115:0/1601053594 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd64c004bf0 con 0x7fd65809b690 2024-01-24T23:27:35.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:35.982 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd656ffd640 10 monclient: dump: 2024-01-24T23:27:35.982 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.982 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.982 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.982 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.982 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.982 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.983 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.983 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.983 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.983 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.983 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.983 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.983 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.983 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.983 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.983 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:35.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 4 set_mon_vals no callback set 2024-01-24T23:27:35.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 10 monclient: shutdown 2024-01-24T23:27:35.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:35.983 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 1 -- 172.21.15.115:0/1601053594 >> v1:172.21.15.115:6792/0 conn(0x7fd65809b690 legacy=0x7fd65813be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 1 -- 172.21.15.115:0/1601053594 shutdown_connections 2024-01-24T23:27:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.968+0000 7fd65e6b1640 1 -- 172.21.15.115:0/1601053594 >> 172.21.15.115:0/1601053594 conn(0x7fd6580a2110 msgr2=0x7fd6580a24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 1 -- 172.21.15.115:0/1601053594 shutdown_connections 2024-01-24T23:27:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 1 -- 172.21.15.115:0/1601053594 wait complete. 2024-01-24T23:27:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 10 monclient: build_initial_monmap 2024-01-24T23:27:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 10 monclient: monmap: 2024-01-24T23:27:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:35.985 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:35.985 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:35.985 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:35.985 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:35.985 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:35.985 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:35.985 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:35.985 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 1 Processor -- start 2024-01-24T23:27:35.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 1 -- start start 2024-01-24T23:27:35.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 10 monclient: init 2024-01-24T23:27:35.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:35.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 10 monclient: _add_conns ranks=[7,2,4,8,5,0,6,3,1] 2024-01-24T23:27:35.985 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 10 monclient(hunting): picked mon.noname-i con 0x7fd6580662f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:35.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 10 monclient(hunting): picked mon.noname-b con 0x7fd65809b690 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:35.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 10 monclient(hunting): picked mon.noname-c con 0x7fd65809bab0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:35.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd658072750 con 0x7fd65809b690 2024-01-24T23:27:35.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6580956b0 con 0x7fd65809bab0 2024-01-24T23:27:35.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd658072200 con 0x7fd6580662f0 2024-01-24T23:27:35.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:35.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 10 monclient(hunting): authenticate will time out at 1334.897481s 2024-01-24T23:27:35.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd6577fe640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fd65809b690 legacy=0x7fd6581cb6d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:35.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd657fff640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fd6580662f0 legacy=0x7fd6581cafc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:35.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65cc4e640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fd65809bab0 0x7fd6581d3340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:57696/0 (socket says 172.21.15.115:57696) 2024-01-24T23:27:35.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65cc4e640 1 -- 172.21.15.115:0/359513260 learned_addr learned my addr 172.21.15.115:0/359513260 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:35.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 1 -- 172.21.15.115:0/359513260 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2601607583 0 0) 0x7fd6580956b0 con 0x7fd65809bab0 2024-01-24T23:27:35.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:35.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 10 monclient(hunting): my global_id is 94900 2024-01-24T23:27:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 1 -- 172.21.15.115:0/359513260 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd644003530 con 0x7fd65809bab0 2024-01-24T23:27:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 1 -- 172.21.15.115:0/359513260 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 573899880 0 0) 0x7fd644003530 con 0x7fd65809bab0 2024-01-24T23:27:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 1 -- 172.21.15.115:0/359513260 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd6580956b0 con 0x7fd65809bab0 2024-01-24T23:27:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 1 -- 172.21.15.115:0/359513260 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd6480041e0 con 0x7fd65809bab0 2024-01-24T23:27:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:27:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 10 monclient(hunting): dump: 2024-01-24T23:27:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.988 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 1 -- 172.21.15.115:0/359513260 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2385864787 0 0) 0x7fd6580956b0 con 0x7fd65809bab0 2024-01-24T23:27:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 1 -- 172.21.15.115:0/359513260 >> v1:172.21.15.132:6792/0 conn(0x7fd6580662f0 legacy=0x7fd6581cafc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 1 -- 172.21.15.115:0/359513260 >> v1:172.21.15.115:6790/0 conn(0x7fd65809b690 legacy=0x7fd6581cb6d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:35.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:27:35.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 1 monclient: found mon.c 2024-01-24T23:27:35.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:35.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:35.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 1 -- 172.21.15.115:0/359513260 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65808f5c0 con 0x7fd65809bab0 2024-01-24T23:27:35.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 10 monclient: _finish_auth 0 2024-01-24T23:27:35.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:27:35.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:35.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 5 monclient: authenticate success, global_id 94900 2024-01-24T23:27:35.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 10 monclient: _renew_subs 2024-01-24T23:27:35.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:35.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 1 -- 172.21.15.115:0/359513260 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd658093a30 con 0x7fd65809bab0 2024-01-24T23:27:35.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 10 monclient: _renew_subs 2024-01-24T23:27:35.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:35.989 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 1 -- 172.21.15.115:0/359513260 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65808f1e0 con 0x7fd65809bab0 2024-01-24T23:27:35.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 1 -- 172.21.15.115:0/359513260 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd648004500 con 0x7fd65809bab0 2024-01-24T23:27:35.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:35.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 1 -- 172.21.15.115:0/359513260 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd648004a30 con 0x7fd65809bab0 2024-01-24T23:27:35.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:35.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd6567fc640 4 set_mon_vals no callback set 2024-01-24T23:27:35.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:35.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 10 monclient: dump: 2024-01-24T23:27:35.990 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:35.990 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:35.990 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.990 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:35.990 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:35.990 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:35.990 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:35.990 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:35.991 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:35.991 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:35.991 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:35.991 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:35.991 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:35.991 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:35.991 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:35.991 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:35.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:35.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:35.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:35.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 1 -- 172.21.15.115:0/359513260 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fd648004b90 con 0x7fd65809bab0 2024-01-24T23:27:35.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd65e6b1640 1 -- 172.21.15.115:0/359513260 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd638005090 con 0x7fd65809bab0 2024-01-24T23:27:35.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd654ff9640 1 -- 172.21.15.115:0/359513260 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(220..220 src has 1..220) v4 ==== 12698+0+0 (unknown 1413254373 0 0) 0x7fd6480459f0 con 0x7fd65809bab0 2024-01-24T23:27:35.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.972+0000 7fd657fff640 10 monclient: get_auth_request con 0x7fd644036e80 auth_method 0 2024-01-24T23:27:35.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.976+0000 7fd654ff9640 1 -- 172.21.15.115:0/359513260 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fd648014990 con 0x7fd65809bab0 2024-01-24T23:27:35.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.976+0000 7fd654ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:35.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:35.976+0000 7fd654ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:36.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:36.172+0000 7fd65e6b1640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-01-24T23:27:36.176 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:36.172+0000 7fd65e6b1640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-01-24T23:27:36.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:36.172+0000 7fd65e6b1640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:36.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:36.172+0000 7fd65e6b1640 1 -- 172.21.15.115:0/359513260 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd6380058c0 con 0x7fd65809bab0 2024-01-24T23:27:36.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:36.176+0000 7fd654ff9640 1 -- 172.21.15.115:0/359513260 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2744 (unknown 254746657 0 2424178774) 0x7fd648012e00 con 0x7fd65809bab0 2024-01-24T23:27:36.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:36.176+0000 7fd654ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-01-24T23:27:36.177 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:36.176+0000 7fd654ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:36.177 INFO:teuthology.orchestra.run.smithi115.stdout:{"election_epoch":40,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":5,"features":{"quorum_con":"4540701272860131327","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","modified":"2024-01-24T23:20:20.557246Z","created":"2024-01-24T23:20:20.557246Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6790","nonce":0}]},"addr":"172.21.15.115:6790/0","public_addr":"172.21.15.115:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6790","nonce":0}]},"addr":"172.21.15.132:6790/0","public_addr":"172.21.15.132:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6791","nonce":0}]},"addr":"172.21.15.115:6791/0","public_addr":"172.21.15.115:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6791","nonce":0}]},"addr":"172.21.15.132:6791/0","public_addr":"172.21.15.132:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6792","nonce":0}]},"addr":"172.21.15.115:6792/0","public_addr":"172.21.15.115:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6792","nonce":0}]},"addr":"172.21.15.132:6792/0","public_addr":"172.21.15.132:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6793","nonce":0}]},"addr":"172.21.15.115:6793/0","public_addr":"172.21.15.115:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-01-24T23:27:36.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:36.176+0000 7fd65e6b1640 1 -- 172.21.15.115:0/359513260 >> v1:172.21.15.132:6812/1835401922 conn(0x7fd644036e80 legacy=0x7fd644039340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:36.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:36.176+0000 7fd65e6b1640 10 monclient: shutdown 2024-01-24T23:27:36.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:36.176+0000 7fd65e6b1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:36.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:36.176+0000 7fd65e6b1640 1 -- 172.21.15.115:0/359513260 >> v1:172.21.15.115:6791/0 conn(0x7fd65809bab0 legacy=0x7fd6581d3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:36.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:36.176+0000 7fd65e6b1640 1 -- 172.21.15.115:0/359513260 shutdown_connections 2024-01-24T23:27:36.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:36.176+0000 7fd65e6b1640 1 -- 172.21.15.115:0/359513260 >> 172.21.15.115:0/359513260 conn(0x7fd6580a2110 msgr2=0x7fd65809ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:36.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:36.176+0000 7fd65e6b1640 1 -- 172.21.15.115:0/359513260 shutdown_connections 2024-01-24T23:27:36.180 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:36.176+0000 7fd65e6b1640 1 -- 172.21.15.115:0/359513260 wait complete. 2024-01-24T23:27:38.320 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~94s 2024-01-24T23:27:38.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:38.338+0000 7f915b7fe640 10 monclient: tick 2024-01-24T23:27:38.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:38.338+0000 7f915b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:38.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:38.338+0000 7f915b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:38.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:38.338+0000 7f915b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:39.195 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-01-24T23:27:39.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.324+0000 7f5de3817640 10 monclient: get_monmap_and_config 2024-01-24T23:27:39.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.324+0000 7f5de3817640 10 monclient: build_initial_monmap 2024-01-24T23:27:39.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.324+0000 7f5de3817640 10 monclient: monmap: 2024-01-24T23:27:39.326 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:39.327 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:39.327 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:39.328079+0000 2024-01-24T23:27:39.327 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:39.328079+0000 2024-01-24T23:27:39.327 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:39.327 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:39.327 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:39.327 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:39.327 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:39.327 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:39.327 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:39.327 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:39.327 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:39.327 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:39.327 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:39.328 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:39.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.324+0000 7f5de3817640 1 Processor -- start 2024-01-24T23:27:39.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.324+0000 7f5de3817640 1 -- start start 2024-01-24T23:27:39.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.324+0000 7f5de3817640 10 monclient: init 2024-01-24T23:27:39.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.324+0000 7f5de3817640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:39.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.324+0000 7f5de3817640 10 monclient: _add_conns ranks=[5,3,6,8,7,4,0,2,1] 2024-01-24T23:27:39.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.324+0000 7f5de3817640 10 monclient(hunting): picked mon.noname-h con 0x7f5ddc134740 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:39.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.324+0000 7f5de3817640 10 monclient(hunting): picked mon.noname-g con 0x7f5ddc09ec80 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:39.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.324+0000 7f5de3817640 10 monclient(hunting): picked mon.noname-d con 0x7f5ddc135230 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:39.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.324+0000 7f5de3817640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ddc04ed70 con 0x7f5ddc09ec80 2024-01-24T23:27:39.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.324+0000 7f5de3817640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ddc0a0d90 con 0x7f5ddc134740 2024-01-24T23:27:39.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.324+0000 7f5de3817640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ddc09e7c0 con 0x7f5ddc135230 2024-01-24T23:27:39.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.324+0000 7f5de3817640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:39.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.324+0000 7f5de3817640 10 monclient(hunting): authenticate will time out at 1338.251257s 2024-01-24T23:27:39.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.324+0000 7f5de1db4640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f5ddc135230 0x7f5ddc13d450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:54854/0 (socket says 172.21.15.115:54854) 2024-01-24T23:27:39.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.324+0000 7f5de1db4640 1 -- 172.21.15.115:0/731851486 learned_addr learned my addr 172.21.15.115:0/731851486 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:39.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 1 -- 172.21.15.115:0/731851486 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2372867309 0 0) 0x7f5ddc0a0d90 con 0x7f5ddc134740 2024-01-24T23:27:39.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:39.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:39.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:39.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 10 monclient(hunting): my global_id is 6737 2024-01-24T23:27:39.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 1 -- 172.21.15.115:0/731851486 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dc8003530 con 0x7f5ddc134740 2024-01-24T23:27:39.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 1 -- 172.21.15.115:0/731851486 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1663402057 0 0) 0x7f5dc8003530 con 0x7f5ddc134740 2024-01-24T23:27:39.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 1 -- 172.21.15.115:0/731851486 >> v1:172.21.15.115:6792/0 conn(0x7f5ddc135230 legacy=0x7f5ddc13d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:39.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 1 -- 172.21.15.115:0/731851486 >> v1:172.21.15.132:6790/0 conn(0x7f5ddc09ec80 legacy=0x7f5ddc139d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:39.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 10 monclient: _finish_hunting 0 2024-01-24T23:27:39.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 1 monclient: found mon.noname-h 2024-01-24T23:27:39.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:39.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:27:39.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 1 -- 172.21.15.115:0/731851486 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ddc0a1070 con 0x7f5ddc134740 2024-01-24T23:27:39.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 10 monclient: _finish_auth 0 2024-01-24T23:27:39.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 10 monclient: _check_auth_tickets 2024-01-24T23:27:39.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:39.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 1 -- 172.21.15.115:0/731851486 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5dcc0017e0 con 0x7f5ddc134740 2024-01-24T23:27:39.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:39.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:27:39.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 10 monclient: dump: 2024-01-24T23:27:39.336 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:39.336 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:39.336 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:39.336 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:39.336 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:39.336 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:39.336 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:39.336 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:39.336 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:39.336 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:39.336 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:39.337 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:39.337 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:39.337 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:39.337 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:39.337 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:39.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 5 monclient: authenticate success, global_id 6737 2024-01-24T23:27:39.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:39.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 1 -- 172.21.15.115:0/731851486 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5dcc001a00 con 0x7f5ddc134740 2024-01-24T23:27:39.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:39.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:39.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 4 set_mon_vals no callback set 2024-01-24T23:27:39.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 1 -- 172.21.15.115:0/731851486 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5dcc003960 con 0x7f5ddc134740 2024-01-24T23:27:39.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:39.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:39.338 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd3fff640 10 monclient: dump: 2024-01-24T23:27:39.338 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:39.338 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:39.338 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:39.338 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:39.338 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:39.338 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:39.338 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:39.338 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:39.338 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:39.338 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:39.338 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:39.338 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:39.338 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:39.338 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:39.339 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:39.339 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:39.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 10 monclient: shutdown 2024-01-24T23:27:39.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:39.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 1 -- 172.21.15.115:0/731851486 >> v1:172.21.15.132:6791/0 conn(0x7f5ddc134740 legacy=0x7f5ddc134b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:39.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 1 -- 172.21.15.115:0/731851486 shutdown_connections 2024-01-24T23:27:39.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 1 -- 172.21.15.115:0/731851486 >> 172.21.15.115:0/731851486 conn(0x7f5ddc0a5730 msgr2=0x7f5ddc0a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:39.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 1 -- 172.21.15.115:0/731851486 shutdown_connections 2024-01-24T23:27:39.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 1 -- 172.21.15.115:0/731851486 wait complete. 2024-01-24T23:27:39.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 10 monclient: build_initial_monmap 2024-01-24T23:27:39.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 10 monclient: monmap: 2024-01-24T23:27:39.339 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:39.339 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:39.339 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:39.339 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:39.340 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:39.340 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:39.340 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:39.340 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:39.340 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:39.340 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:39.340 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:39.340 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:39.340 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:39.340 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:39.340 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:39.340 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:39.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 1 Processor -- start 2024-01-24T23:27:39.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 1 -- start start 2024-01-24T23:27:39.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 10 monclient: init 2024-01-24T23:27:39.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:39.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 10 monclient: _add_conns ranks=[5,6,8,0,1,4,7,2,3] 2024-01-24T23:27:39.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 10 monclient(hunting): picked mon.noname-h con 0x7f5ddc09ec80 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:39.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 10 monclient(hunting): picked mon.noname-d con 0x7f5ddc134740 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:39.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 10 monclient(hunting): picked mon.noname-e con 0x7f5ddc135230 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:39.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ddc1369d0 con 0x7f5ddc09ec80 2024-01-24T23:27:39.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ddc071c00 con 0x7f5ddc134740 2024-01-24T23:27:39.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ddc0716b0 con 0x7f5ddc135230 2024-01-24T23:27:39.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:39.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de3817640 10 monclient(hunting): authenticate will time out at 1338.254834s 2024-01-24T23:27:39.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de1db4640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f5ddc135230 0x7f5ddc1d1c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:46374/0 (socket says 172.21.15.115:46374) 2024-01-24T23:27:39.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de0db2640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f5ddc134740 0x7f5ddc1ce500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:54862/0 (socket says 172.21.15.115:54862) 2024-01-24T23:27:39.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5de1db4640 1 -- 172.21.15.115:0/166421298 learned_addr learned my addr 172.21.15.115:0/166421298 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:39.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd1ffb640 1 -- 172.21.15.115:0/166421298 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3925222831 0 0) 0x7f5ddc1369d0 con 0x7f5ddc09ec80 2024-01-24T23:27:39.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd1ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:39.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd1ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:39.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:39.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd1ffb640 10 monclient(hunting): my global_id is 6746 2024-01-24T23:27:39.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.328+0000 7f5dd1ffb640 1 -- 172.21.15.115:0/166421298 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dc0003530 con 0x7f5ddc09ec80 2024-01-24T23:27:39.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 1 -- 172.21.15.115:0/166421298 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 130171128 0 0) 0x7f5dc0003530 con 0x7f5ddc09ec80 2024-01-24T23:27:39.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 1 -- 172.21.15.115:0/166421298 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5ddc1369d0 con 0x7f5ddc09ec80 2024-01-24T23:27:39.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 1 -- 172.21.15.115:0/166421298 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5dcc0013a0 con 0x7f5ddc09ec80 2024-01-24T23:27:39.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:39.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:27:39.342 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 10 monclient(hunting): dump: 2024-01-24T23:27:39.342 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:39.342 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:39.342 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:39.342 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:39.343 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:39.343 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:39.343 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:39.343 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:39.343 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:39.343 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:39.343 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:39.343 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:39.343 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:39.343 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:39.343 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:39.343 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:39.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:39.343 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 1 -- 172.21.15.115:0/166421298 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1310356295 0 0) 0x7f5ddc1369d0 con 0x7f5ddc09ec80 2024-01-24T23:27:39.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 1 -- 172.21.15.115:0/166421298 >> v1:172.21.15.115:6793/0 conn(0x7f5ddc135230 legacy=0x7f5ddc1d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:39.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 1 -- 172.21.15.115:0/166421298 >> v1:172.21.15.115:6792/0 conn(0x7f5ddc134740 legacy=0x7f5ddc1ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:39.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:27:39.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 1 monclient: found mon.h 2024-01-24T23:27:39.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:39.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:39.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 1 -- 172.21.15.115:0/166421298 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ddc138780 con 0x7f5ddc09ec80 2024-01-24T23:27:39.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 10 monclient: _finish_auth 0 2024-01-24T23:27:39.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:27:39.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:39.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5de3817640 5 monclient: authenticate success, global_id 6746 2024-01-24T23:27:39.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5de3817640 10 monclient: _renew_subs 2024-01-24T23:27:39.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5de3817640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:39.344 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5de3817640 1 -- 172.21.15.115:0/166421298 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ddc136fa0 con 0x7f5ddc09ec80 2024-01-24T23:27:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5de3817640 10 monclient: _renew_subs 2024-01-24T23:27:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5de3817640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5de3817640 1 -- 172.21.15.115:0/166421298 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ddc1376e0 con 0x7f5ddc09ec80 2024-01-24T23:27:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 1 -- 172.21.15.115:0/166421298 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5dcc003c30 con 0x7f5ddc09ec80 2024-01-24T23:27:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 1 -- 172.21.15.115:0/166421298 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f5dcc0040e0 con 0x7f5ddc09ec80 2024-01-24T23:27:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd37fe640 4 set_mon_vals no callback set 2024-01-24T23:27:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 10 monclient: dump: 2024-01-24T23:27:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:39.345 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:39.346 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:39.346 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:39.346 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:39.346 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:39.346 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:39.346 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:39.346 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:39.346 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:39.346 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:39.346 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:39.346 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:39.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5de3817640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:39.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5de3817640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:39.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5de3817640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:39.346 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5de3817640 1 -- 172.21.15.115:0/166421298 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dbc005090 con 0x7f5ddc09ec80 2024-01-24T23:27:39.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:39.334+0000 7f915b7fe640 10 monclient: tick 2024-01-24T23:27:39.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:39.334+0000 7f915b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:39.347 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:39.334+0000 7f915b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:39.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 1 -- 172.21.15.115:0/166421298 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f5dcc00eb60 con 0x7f5ddc09ec80 2024-01-24T23:27:39.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5dd1ffb640 1 -- 172.21.15.115:0/166421298 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(220..220 src has 1..220) v4 ==== 12698+0+0 (unknown 1413254373 0 0) 0x7f5dcc044a50 con 0x7f5ddc09ec80 2024-01-24T23:27:39.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.332+0000 7f5de0db2640 10 monclient: get_auth_request con 0x7f5dc0036f10 auth_method 0 2024-01-24T23:27:39.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.344+0000 7f5dd1ffb640 1 -- 172.21.15.115:0/166421298 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f5dcc0136c0 con 0x7f5ddc09ec80 2024-01-24T23:27:39.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.344+0000 7f5dd1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:39.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.344+0000 7f5dd1ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.556+0000 7f5de3817640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-01-24T23:27:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.556+0000 7f5de3817640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-01-24T23:27:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.556+0000 7f5de3817640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.556+0000 7f5de3817640 1 -- 172.21.15.115:0/166421298 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f5dbc005340 con 0x7f5ddc09ec80 2024-01-24T23:27:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.556+0000 7f5dd1ffb640 1 -- 172.21.15.115:0/166421298 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2744 (unknown 254746657 0 4150838887) 0x7f5dcc0136c0 con 0x7f5ddc09ec80 2024-01-24T23:27:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.556+0000 7f5dd1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-01-24T23:27:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.556+0000 7f5dd1ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:39.561 INFO:teuthology.orchestra.run.smithi115.stdout:{"election_epoch":40,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":9,"features":{"quorum_con":"4540701272860131327","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","modified":"2024-01-24T23:20:20.557246Z","created":"2024-01-24T23:20:20.557246Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6790","nonce":0}]},"addr":"172.21.15.115:6790/0","public_addr":"172.21.15.115:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6790","nonce":0}]},"addr":"172.21.15.132:6790/0","public_addr":"172.21.15.132:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6791","nonce":0}]},"addr":"172.21.15.115:6791/0","public_addr":"172.21.15.115:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6791","nonce":0}]},"addr":"172.21.15.132:6791/0","public_addr":"172.21.15.132:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6792","nonce":0}]},"addr":"172.21.15.115:6792/0","public_addr":"172.21.15.115:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6792","nonce":0}]},"addr":"172.21.15.132:6792/0","public_addr":"172.21.15.132:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6793","nonce":0}]},"addr":"172.21.15.115:6793/0","public_addr":"172.21.15.115:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-01-24T23:27:39.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.560+0000 7f5de3817640 1 -- 172.21.15.115:0/166421298 >> v1:172.21.15.132:6812/1835401922 conn(0x7f5dc0036f10 legacy=0x7f5dc00393d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:39.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.560+0000 7f5de3817640 10 monclient: shutdown 2024-01-24T23:27:39.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.560+0000 7f5de3817640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:39.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.560+0000 7f5de3817640 1 -- 172.21.15.115:0/166421298 >> v1:172.21.15.132:6791/0 conn(0x7f5ddc09ec80 legacy=0x7f5ddc1c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:39.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.560+0000 7f5de3817640 1 -- 172.21.15.115:0/166421298 shutdown_connections 2024-01-24T23:27:39.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.560+0000 7f5de3817640 1 -- 172.21.15.115:0/166421298 >> 172.21.15.115:0/166421298 conn(0x7f5ddc0a5730 msgr2=0x7f5ddc0a3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:39.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.560+0000 7f5de3817640 1 -- 172.21.15.115:0/166421298 shutdown_connections 2024-01-24T23:27:39.563 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:39.560+0000 7f5de3817640 1 -- 172.21.15.115:0/166421298 wait complete. 2024-01-24T23:27:40.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:40.334+0000 7f915b7fe640 10 monclient: tick 2024-01-24T23:27:40.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:40.334+0000 7f915b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:40.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:40.334+0000 7f915b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:41.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:41.338+0000 7f915b7fe640 10 monclient: tick 2024-01-24T23:27:41.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:41.338+0000 7f915b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:41.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:41.338+0000 7f915b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:42.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:42.334+0000 7f915b7fe640 10 monclient: tick 2024-01-24T23:27:42.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:42.334+0000 7f915b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:42.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:42.334+0000 7f915b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:42.580 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-01-24T23:27:42.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:42.704+0000 7f52d4a4e640 10 monclient: get_monmap_and_config 2024-01-24T23:27:42.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:42.704+0000 7f52d4a4e640 10 monclient: build_initial_monmap 2024-01-24T23:27:42.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:42.704+0000 7f52d4a4e640 10 monclient: monmap: 2024-01-24T23:27:42.708 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:42.708 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:42.708 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:42.709461+0000 2024-01-24T23:27:42.708 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:42.709461+0000 2024-01-24T23:27:42.708 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:42.708 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:42.708 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:42.708 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:42.708 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:42.709 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:42.709 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:42.709 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:42.709 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:42.709 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:42.709 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:42.709 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:42.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:42.704+0000 7f52d4a4e640 1 Processor -- start 2024-01-24T23:27:42.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:42.704+0000 7f52d4a4e640 1 -- start start 2024-01-24T23:27:42.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:42.704+0000 7f52d4a4e640 10 monclient: init 2024-01-24T23:27:42.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:42.704+0000 7f52d4a4e640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:42.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:42.704+0000 7f52d4a4e640 10 monclient: _add_conns ranks=[3,4,1,6,8,5,7,0,2] 2024-01-24T23:27:42.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:42.704+0000 7f52d4a4e640 10 monclient(hunting): picked mon.noname-g con 0x7f52d0133a10 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:42.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:42.704+0000 7f52d4a4e640 10 monclient(hunting): picked mon.noname-c con 0x7f52d0133df0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:42.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:42.704+0000 7f52d4a4e640 10 monclient(hunting): picked mon.noname-f con 0x7f52d0134250 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:42.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:42.704+0000 7f52d4a4e640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d004ed70 con 0x7f52d0134250 2024-01-24T23:27:42.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:42.704+0000 7f52d4a4e640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d009aae0 con 0x7f52d0133a10 2024-01-24T23:27:42.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:42.704+0000 7f52d4a4e640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d00678e0 con 0x7f52d0133df0 2024-01-24T23:27:42.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:42.704+0000 7f52d4a4e640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:42.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:42.704+0000 7f52d4a4e640 10 monclient(hunting): authenticate will time out at 1341.632000s 2024-01-24T23:27:42.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:42.708+0000 7f52cdd9b640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f52d0133df0 0x7f52d013bc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:48652/0 (socket says 172.21.15.115:48652) 2024-01-24T23:27:42.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:42.708+0000 7f52cdd9b640 1 -- 172.21.15.115:0/523198606 learned_addr learned my addr 172.21.15.115:0/523198606 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:43.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:43.334+0000 7f915b7fe640 10 monclient: tick 2024-01-24T23:27:43.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:43.334+0000 7f915b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:43.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:43.334+0000 7f915b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:44.337 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:44.334+0000 7f915b7fe640 10 monclient: tick 2024-01-24T23:27:44.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:44.334+0000 7f915b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:44.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:44.334+0000 7f915b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:44.529 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~100s 2024-01-24T23:27:45.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:45.334+0000 7f915b7fe640 10 monclient: tick 2024-01-24T23:27:45.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:45.334+0000 7f915b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:45.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:45.334+0000 7f915b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:45.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:45.708+0000 7f52bffff640 10 monclient(hunting): tick 2024-01-24T23:27:45.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:45.708+0000 7f52bffff640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:27:45.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:45.708+0000 7f52bffff640 1 monclient(hunting): continuing hunt 2024-01-24T23:27:45.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:45.708+0000 7f52bffff640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:27:45.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:45.708+0000 7f52bffff640 1 -- 172.21.15.115:0/523198606 >> v1:172.21.15.115:6791/0 conn(0x7f52d0133df0 legacy=0x7f52d013bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:45.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:45.708+0000 7f52bffff640 1 -- 172.21.15.115:0/523198606 >> v1:172.21.15.132:6790/0 conn(0x7f52d0133a10 legacy=0x7f52d009bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:45.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:45.708+0000 7f52bffff640 1 -- 172.21.15.115:0/523198606 >> v1:172.21.15.132:6789/0 conn(0x7f52d0134250 legacy=0x7f52d013f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:45.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:45.708+0000 7f52bffff640 10 monclient: _add_conns ranks=[0,2,6,5,8,7] 2024-01-24T23:27:45.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:45.708+0000 7f52bffff640 10 monclient(hunting): picked mon.noname-a con 0x7f52b00037f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:45.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:45.708+0000 7f52bffff640 10 monclient(hunting): picked mon.noname-b con 0x7f52b0007480 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:45.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:45.708+0000 7f52bffff640 10 monclient(hunting): picked mon.noname-d con 0x7f52b000b1d0 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:45.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:45.708+0000 7f52bffff640 1 -- 172.21.15.115:0/523198606 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d0066d10 con 0x7f52b00037f0 2024-01-24T23:27:45.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:45.708+0000 7f52bffff640 1 -- 172.21.15.115:0/523198606 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b000ffe0 con 0x7f52b0007480 2024-01-24T23:27:45.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:45.708+0000 7f52bffff640 1 -- 172.21.15.115:0/523198606 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b0011230 con 0x7f52b000b1d0 2024-01-24T23:27:45.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:45.708+0000 7f52bffff640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:45.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:45.708+0000 7f52cdd9b640 1 -- 172.21.15.115:0/523198606 >> v1:172.21.15.115:6790/0 conn(0x7f52b0007480 legacy=0x7f52b0009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:45.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:45.708+0000 7f52ce59c640 1 -- 172.21.15.115:0/523198606 >> v1:172.21.15.115:6789/0 conn(0x7f52b00037f0 legacy=0x7f52b0005c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:27:45.913 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:45.912+0000 7f52ce59c640 1 -- 172.21.15.115:0/523198606 >> v1:172.21.15.115:6789/0 conn(0x7f52b00037f0 legacy=0x7f52b0005c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:27:45.913 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:45.912+0000 7f52cdd9b640 1 -- 172.21.15.115:0/523198606 >> v1:172.21.15.115:6790/0 conn(0x7f52b0007480 legacy=0x7f52b0009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:46.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:46.316+0000 7f52ce59c640 1 -- 172.21.15.115:0/523198606 >> v1:172.21.15.115:6789/0 conn(0x7f52b00037f0 legacy=0x7f52b0005c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:27:46.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:46.316+0000 7f52cdd9b640 1 -- 172.21.15.115:0/523198606 >> v1:172.21.15.115:6790/0 conn(0x7f52b0007480 legacy=0x7f52b0009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:46.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:46.334+0000 7f915b7fe640 10 monclient: tick 2024-01-24T23:27:46.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:46.334+0000 7f915b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:46.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:46.334+0000 7f915b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:47.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:47.120+0000 7f52ce59c640 1 -- 172.21.15.115:0/523198606 >> v1:172.21.15.115:6789/0 conn(0x7f52b00037f0 legacy=0x7f52b0005c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:27:47.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:47.120+0000 7f52cdd9b640 1 -- 172.21.15.115:0/523198606 >> v1:172.21.15.115:6790/0 conn(0x7f52b0007480 legacy=0x7f52b0009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:47.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:47.338+0000 7f915b7fe640 10 monclient: tick 2024-01-24T23:27:47.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:47.338+0000 7f915b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:47.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:47.338+0000 7f915b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:48.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:48.338+0000 7f915b7fe640 10 monclient: tick 2024-01-24T23:27:48.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:48.338+0000 7f915b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:48.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:48.338+0000 7f915b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:48.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:48.338+0000 7f915b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:48.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:48.708+0000 7f52bffff640 10 monclient(hunting): tick 2024-01-24T23:27:48.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:48.708+0000 7f52bffff640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:27:48.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:48.708+0000 7f52bffff640 1 monclient(hunting): continuing hunt 2024-01-24T23:27:48.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:48.708+0000 7f52bffff640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:27:48.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:48.708+0000 7f52bffff640 1 -- 172.21.15.115:0/523198606 >> v1:172.21.15.115:6792/0 conn(0x7f52b000b1d0 legacy=0x7f52b000d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:48.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:48.708+0000 7f52bffff640 1 -- 172.21.15.115:0/523198606 >> v1:172.21.15.115:6790/0 conn(0x7f52b0007480 legacy=0x7f52b0009920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:48.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:48.708+0000 7f52bffff640 1 -- 172.21.15.115:0/523198606 >> v1:172.21.15.115:6789/0 conn(0x7f52b00037f0 legacy=0x7f52b0005c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:48.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:48.708+0000 7f52bffff640 10 monclient: _add_conns ranks=[7,8,5] 2024-01-24T23:27:48.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:48.708+0000 7f52bffff640 10 monclient(hunting): picked mon.noname-i con 0x7f52b00115c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:48.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:48.708+0000 7f52bffff640 10 monclient(hunting): picked mon.noname-e con 0x7f52b0010170 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:48.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:48.708+0000 7f52bffff640 10 monclient(hunting): picked mon.noname-h con 0x7f52b0010d30 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:48.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:48.708+0000 7f52ced9d640 1 -- 172.21.15.115:0/523198606 reap_dead start 2024-01-24T23:27:48.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:48.708+0000 7f52bffff640 1 -- 172.21.15.115:0/523198606 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b0012480 con 0x7f52b0010d30 2024-01-24T23:27:48.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:48.708+0000 7f52bffff640 1 -- 172.21.15.115:0/523198606 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d0066d10 con 0x7f52b00115c0 2024-01-24T23:27:48.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:48.708+0000 7f52bffff640 1 -- 172.21.15.115:0/523198606 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b000ffe0 con 0x7f52b0010170 2024-01-24T23:27:48.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:48.708+0000 7f52bffff640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:48.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:48.708+0000 7f52ce59c640 1 -- 172.21.15.115:0/523198606 >> v1:172.21.15.132:6792/0 conn(0x7f52b00115c0 legacy=0x7f52b00119a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:48.913 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:48.912+0000 7f52ce59c640 1 -- 172.21.15.115:0/523198606 >> v1:172.21.15.132:6792/0 conn(0x7f52b00115c0 legacy=0x7f52b00119a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:49.317 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:49.316+0000 7f52ce59c640 1 -- 172.21.15.115:0/523198606 >> v1:172.21.15.132:6792/0 conn(0x7f52b00115c0 legacy=0x7f52b00119a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:49.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:49.338+0000 7f915b7fe640 10 monclient: tick 2024-01-24T23:27:49.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:49.338+0000 7f915b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:49.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:49.338+0000 7f915b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:50.121 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52ce59c640 1 -- 172.21.15.115:0/523198606 >> v1:172.21.15.132:6792/0 conn(0x7f52b00115c0 legacy=0x7f52b00119a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:50.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 1 -- 172.21.15.115:0/523198606 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2548292822 0 0) 0x7f52b000ffe0 con 0x7f52b0010170 2024-01-24T23:27:50.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 10 monclient(hunting): _negotiate 2024-01-24T23:27:50.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:50.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:50.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 10 monclient(hunting): my global_id is 85895 2024-01-24T23:27:50.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 1 -- 172.21.15.115:0/523198606 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52b8003530 con 0x7f52b0010170 2024-01-24T23:27:50.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 1 -- 172.21.15.115:0/523198606 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 962461762 0 0) 0x7f52b8003530 con 0x7f52b0010170 2024-01-24T23:27:50.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 1 -- 172.21.15.115:0/523198606 >> v1:172.21.15.132:6792/0 conn(0x7f52b00115c0 legacy=0x7f52b00119a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:50.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 1 -- 172.21.15.115:0/523198606 >> v1:172.21.15.132:6791/0 conn(0x7f52b0010d30 legacy=0x7f52b00197b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 10 monclient: _finish_hunting 0 2024-01-24T23:27:50.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 1 monclient: found mon.noname-e 2024-01-24T23:27:50.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:50.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:27:50.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 1 -- 172.21.15.115:0/523198606 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b000a0d0 con 0x7f52b0010170 2024-01-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 10 monclient: _finish_auth 0 2024-01-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 10 monclient: _check_auth_tickets 2024-01-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52d4a4e640 5 monclient: authenticate success, global_id 85895 2024-01-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 1 -- 172.21.15.115:0/523198606 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f52c40017c0 con 0x7f52b0010170 2024-01-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52d4a4e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 10 monclient: dump: 2024-01-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.126 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52d4a4e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 1 -- 172.21.15.115:0/523198606 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f52c40019e0 con 0x7f52b0010170 2024-01-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:50.127 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52d4a4e640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52d4a4e640 4 set_mon_vals no callback set 2024-01-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 1 -- 172.21.15.115:0/523198606 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f52c4004bf0 con 0x7f52b0010170 2024-01-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.120+0000 7f52cd59a640 10 monclient: dump: 2024-01-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:50.128 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 10 monclient: shutdown 2024-01-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 1 -- 172.21.15.115:0/523198606 >> v1:172.21.15.115:6793/0 conn(0x7f52b0010170 legacy=0x7f52b0010620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 1 -- 172.21.15.115:0/523198606 shutdown_connections 2024-01-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 1 -- 172.21.15.115:0/523198606 >> 172.21.15.115:0/523198606 conn(0x7f52d00a6490 msgr2=0x7f52d00a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 1 -- 172.21.15.115:0/523198606 shutdown_connections 2024-01-24T23:27:50.129 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 1 -- 172.21.15.115:0/523198606 wait complete. 2024-01-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 10 monclient: build_initial_monmap 2024-01-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 10 monclient: monmap: 2024-01-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:50.134 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:50.135 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:50.135 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:50.135 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:50.135 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:50.135 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:50.135 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:50.135 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 1 Processor -- start 2024-01-24T23:27:50.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 1 -- start start 2024-01-24T23:27:50.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 10 monclient: init 2024-01-24T23:27:50.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:50.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 10 monclient: _add_conns ranks=[6,3,0,5,8,7,1,2,4] 2024-01-24T23:27:50.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 10 monclient(hunting): picked mon.noname-d con 0x7f52b0010170 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:50.135 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 10 monclient(hunting): picked mon.noname-g con 0x7f52b0010d30 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:50.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 10 monclient(hunting): picked mon.noname-a con 0x7f52b00115c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:50.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d009c610 con 0x7f52b00115c0 2024-01-24T23:27:50.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d011fdd0 con 0x7f52b0010d30 2024-01-24T23:27:50.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d01200b0 con 0x7f52b0010170 2024-01-24T23:27:50.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:50.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 10 monclient(hunting): authenticate will time out at 1349.050429s 2024-01-24T23:27:50.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52ced9d640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f52b00115c0 legacy=0x7f52d01d3ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:27:50.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52ce59c640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f52b0010170 0x7f52d01329d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:50514/0 (socket says 172.21.15.115:50514) 2024-01-24T23:27:50.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52ce59c640 1 -- 172.21.15.115:0/1556904849 learned_addr learned my addr 172.21.15.115:0/1556904849 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:50.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 1 -- 172.21.15.115:0/1556904849 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1885225683 0 0) 0x7f52d01200b0 con 0x7f52b0010170 2024-01-24T23:27:50.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:50.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:50.136 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:50.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 10 monclient(hunting): my global_id is 6927 2024-01-24T23:27:50.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 1 -- 172.21.15.115:0/1556904849 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52b000ffa0 con 0x7f52b0010170 2024-01-24T23:27:50.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 1 -- 172.21.15.115:0/1556904849 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3278502104 0 0) 0x7f52b000ffa0 con 0x7f52b0010170 2024-01-24T23:27:50.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 1 -- 172.21.15.115:0/1556904849 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f52d01200b0 con 0x7f52b0010170 2024-01-24T23:27:50.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 1 -- 172.21.15.115:0/1556904849 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f52c0001bf0 con 0x7f52b0010170 2024-01-24T23:27:50.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:50.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:27:50.137 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 10 monclient(hunting): dump: 2024-01-24T23:27:50.137 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:50.137 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.137 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.137 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.137 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:50.137 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.138 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:50.138 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:50.138 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:50.138 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:50.138 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:50.138 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:50.138 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:50.138 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:50.138 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:50.138 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:50.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 1 -- 172.21.15.115:0/1556904849 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3186695026 0 0) 0x7f52d011fdd0 con 0x7f52b0010d30 2024-01-24T23:27:50.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:50.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:50.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:50.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 10 monclient(hunting): my global_id is 6690 2024-01-24T23:27:50.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 1 -- 172.21.15.115:0/1556904849 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52b000ffa0 con 0x7f52b0010d30 2024-01-24T23:27:50.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 1 -- 172.21.15.115:0/1556904849 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1593961067 0 0) 0x7f52d01200b0 con 0x7f52b0010170 2024-01-24T23:27:50.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 1 -- 172.21.15.115:0/1556904849 >> v1:172.21.15.132:6790/0 conn(0x7f52b0010d30 legacy=0x7f52d01d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 1 -- 172.21.15.115:0/1556904849 >> v1:172.21.15.115:6789/0 conn(0x7f52b00115c0 legacy=0x7f52d01d3ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:50.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 10 monclient: _finish_hunting 0 2024-01-24T23:27:50.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 1 monclient: found mon.d 2024-01-24T23:27:50.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:50.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:50.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 1 -- 172.21.15.115:0/1556904849 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d009c9d0 con 0x7f52b0010170 2024-01-24T23:27:50.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 10 monclient: _finish_auth 0 2024-01-24T23:27:50.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 10 monclient: _check_auth_tickets 2024-01-24T23:27:50.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:50.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 5 monclient: authenticate success, global_id 6927 2024-01-24T23:27:50.139 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 10 monclient: _renew_subs 2024-01-24T23:27:50.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:50.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 1 -- 172.21.15.115:0/1556904849 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d011e030 con 0x7f52b0010170 2024-01-24T23:27:50.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 10 monclient: _renew_subs 2024-01-24T23:27:50.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:50.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52d4a4e640 1 -- 172.21.15.115:0/1556904849 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d0120390 con 0x7f52b0010170 2024-01-24T23:27:50.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 1 -- 172.21.15.115:0/1556904849 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f52c0003950 con 0x7f52b0010170 2024-01-24T23:27:50.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:50.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 1 -- 172.21.15.115:0/1556904849 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f52c0003e80 con 0x7f52b0010170 2024-01-24T23:27:50.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:50.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52bffff640 4 set_mon_vals no callback set 2024-01-24T23:27:50.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:50.140 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.124+0000 7f52beffd640 10 monclient: dump: 2024-01-24T23:27:50.140 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:50.141 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.141 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.141 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.141 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:50.141 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.141 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:50.141 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:50.141 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:50.141 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:50.141 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:50.141 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:50.141 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:50.141 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:50.141 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:50.142 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.128+0000 7f52beffd640 1 -- 172.21.15.115:0/1556904849 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f52c00040a0 con 0x7f52b0010170 2024-01-24T23:27:50.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.128+0000 7f52d4a4e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:50.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.128+0000 7f52d4a4e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:50.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.128+0000 7f52d4a4e640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:50.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.128+0000 7f52d4a4e640 1 -- 172.21.15.115:0/1556904849 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b4005090 con 0x7f52b0010170 2024-01-24T23:27:50.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.128+0000 7f52beffd640 1 -- 172.21.15.115:0/1556904849 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(220..220 src has 1..220) v4 ==== 12698+0+0 (unknown 1413254373 0 0) 0x7f52c0045bf0 con 0x7f52b0010170 2024-01-24T23:27:50.142 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.128+0000 7f52cdd9b640 10 monclient: get_auth_request con 0x7f52b0039230 auth_method 0 2024-01-24T23:27:50.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.132+0000 7f52beffd640 1 -- 172.21.15.115:0/1556904849 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f52c0014bd0 con 0x7f52b0010170 2024-01-24T23:27:50.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.132+0000 7f52beffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:50.143 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.132+0000 7f52beffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:50.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:50.338+0000 7f915b7fe640 10 monclient: tick 2024-01-24T23:27:50.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:50.338+0000 7f915b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:50.339 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:50.338+0000 7f915b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:50.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.344+0000 7f52d4a4e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-01-24T23:27:50.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.344+0000 7f52d4a4e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-01-24T23:27:50.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.344+0000 7f52d4a4e640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:50.348 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.344+0000 7f52d4a4e640 1 -- 172.21.15.115:0/1556904849 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f52b40058c0 con 0x7f52b0010170 2024-01-24T23:27:50.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.348+0000 7f52beffd640 1 -- 172.21.15.115:0/1556904849 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2726 (unknown 254746657 0 1373386634) 0x7f52c0013040 con 0x7f52b0010170 2024-01-24T23:27:50.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.348+0000 7f52beffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-01-24T23:27:50.349 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.348+0000 7f52beffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:50.349 INFO:teuthology.orchestra.run.smithi115.stdout:{"election_epoch":44,"quorum":[1,3,4,5,6,8],"quorum_names":["f","g","c","h","d","e"],"quorum_leader_name":"h","quorum_age":0,"features":{"quorum_con":"4540701272860131327","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","modified":"2024-01-24T23:20:20.557246Z","created":"2024-01-24T23:20:20.557246Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6790","nonce":0}]},"addr":"172.21.15.115:6790/0","public_addr":"172.21.15.115:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6790","nonce":0}]},"addr":"172.21.15.132:6790/0","public_addr":"172.21.15.132:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6791","nonce":0}]},"addr":"172.21.15.115:6791/0","public_addr":"172.21.15.115:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6791","nonce":0}]},"addr":"172.21.15.132:6791/0","public_addr":"172.21.15.132:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6792","nonce":0}]},"addr":"172.21.15.115:6792/0","public_addr":"172.21.15.115:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6792","nonce":0}]},"addr":"172.21.15.132:6792/0","public_addr":"172.21.15.132:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6793","nonce":0}]},"addr":"172.21.15.115:6793/0","public_addr":"172.21.15.115:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-01-24T23:27:50.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.348+0000 7f52d4a4e640 1 -- 172.21.15.115:0/1556904849 >> v1:172.21.15.132:6812/1835401922 conn(0x7f52b0039230 legacy=0x7f52b003b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.348+0000 7f52d4a4e640 10 monclient: shutdown 2024-01-24T23:27:50.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.348+0000 7f52d4a4e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:50.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.348+0000 7f52d4a4e640 1 -- 172.21.15.115:0/1556904849 >> v1:172.21.15.115:6792/0 conn(0x7f52b0010170 legacy=0x7f52d01329d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.348+0000 7f52d4a4e640 1 -- 172.21.15.115:0/1556904849 shutdown_connections 2024-01-24T23:27:50.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.348+0000 7f52d4a4e640 1 -- 172.21.15.115:0/1556904849 >> 172.21.15.115:0/1556904849 conn(0x7f52d00a6490 msgr2=0x7f52d00a53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:50.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.348+0000 7f52d4a4e640 1 -- 172.21.15.115:0/1556904849 shutdown_connections 2024-01-24T23:27:50.352 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.348+0000 7f52d4a4e640 1 -- 172.21.15.115:0/1556904849 wait complete. 2024-01-24T23:27:50.368 INFO:tasks.mon_thrash.ceph_manager:quorum is size 6 2024-01-24T23:27:50.368 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon c 2024-01-24T23:27:50.368 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-01-24T23:27:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.480+0000 7fdf991eb640 10 monclient: get_monmap_and_config 2024-01-24T23:27:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.480+0000 7fdf991eb640 10 monclient: build_initial_monmap 2024-01-24T23:27:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.480+0000 7fdf991eb640 10 monclient: monmap: 2024-01-24T23:27:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:50.486786+0000 2024-01-24T23:27:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:50.486786+0000 2024-01-24T23:27:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:50.486 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf991eb640 1 Processor -- start 2024-01-24T23:27:50.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf991eb640 1 -- start start 2024-01-24T23:27:50.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf991eb640 10 monclient: init 2024-01-24T23:27:50.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf991eb640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:50.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf991eb640 10 monclient: _add_conns ranks=[1,7,3,2,5,0,6,8,4] 2024-01-24T23:27:50.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf991eb640 10 monclient(hunting): picked mon.noname-f con 0x7fdf94133c10 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:50.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf991eb640 10 monclient(hunting): picked mon.noname-i con 0x7fdf94136960 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:50.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf991eb640 10 monclient(hunting): picked mon.noname-g con 0x7fdf9413a470 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:50.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf991eb640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf9404ed70 con 0x7fdf94133c10 2024-01-24T23:27:50.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf991eb640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf9409dee0 con 0x7fdf9413a470 2024-01-24T23:27:50.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf991eb640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf9409d400 con 0x7fdf94136960 2024-01-24T23:27:50.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf991eb640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:50.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf991eb640 10 monclient(hunting): authenticate will time out at 1349.409376s 2024-01-24T23:27:50.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf9259c640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fdf94136960 legacy=0x7fdf94138d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:50.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf92d9d640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fdf94133c10 0x7fdf94133ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.115:52932/0 (socket says 172.21.15.115:52932) 2024-01-24T23:27:50.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf92d9d640 1 -- 172.21.15.115:0/582641137 learned_addr learned my addr 172.21.15.115:0/582641137 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:50.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 1 -- 172.21.15.115:0/582641137 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 451414620 0 0) 0x7fdf9404ed70 con 0x7fdf94133c10 2024-01-24T23:27:50.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:27:50.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:50.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:50.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 10 monclient(hunting): my global_id is 124309 2024-01-24T23:27:50.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 1 -- 172.21.15.115:0/582641137 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf7c003530 con 0x7fdf94133c10 2024-01-24T23:27:50.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 1 -- 172.21.15.115:0/582641137 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3791826454 0 0) 0x7fdf9409dee0 con 0x7fdf9413a470 2024-01-24T23:27:50.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:27:50.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:50.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:50.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 10 monclient(hunting): my global_id is 6699 2024-01-24T23:27:50.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 1 -- 172.21.15.115:0/582641137 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf9404ed70 con 0x7fdf9413a470 2024-01-24T23:27:50.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 1 -- 172.21.15.115:0/582641137 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 87402173 0 0) 0x7fdf9404ed70 con 0x7fdf9413a470 2024-01-24T23:27:50.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 1 -- 172.21.15.115:0/582641137 >> v1:172.21.15.132:6792/0 conn(0x7fdf94136960 legacy=0x7fdf94138d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:50.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 1 -- 172.21.15.115:0/582641137 >> v1:172.21.15.132:6789/0 conn(0x7fdf94133c10 legacy=0x7fdf94133ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 10 monclient: _finish_hunting 0 2024-01-24T23:27:50.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 1 monclient: found mon.noname-g 2024-01-24T23:27:50.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:50.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:27:50.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 1 -- 172.21.15.115:0/582641137 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9409f380 con 0x7fdf9413a470 2024-01-24T23:27:50.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 10 monclient: _finish_auth 0 2024-01-24T23:27:50.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 10 monclient: _check_auth_tickets 2024-01-24T23:27:50.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:50.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 1 -- 172.21.15.115:0/582641137 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdf8c001b20 con 0x7fdf9413a470 2024-01-24T23:27:50.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:50.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:27:50.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 10 monclient: dump: 2024-01-24T23:27:50.500 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:50.500 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.500 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.500 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.500 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:50.500 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.500 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:50.500 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:50.501 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:50.501 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:50.501 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:50.501 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:50.501 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:50.501 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:50.501 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:50.501 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf991eb640 5 monclient: authenticate success, global_id 6699 2024-01-24T23:27:50.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf991eb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:50.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 1 -- 172.21.15.115:0/582641137 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdf8c001e40 con 0x7fdf9413a470 2024-01-24T23:27:50.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:50.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 1 -- 172.21.15.115:0/582641137 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdf8c003ce0 con 0x7fdf9413a470 2024-01-24T23:27:50.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:50.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:50.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf91d9b640 10 monclient: dump: 2024-01-24T23:27:50.502 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:50.502 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.502 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.502 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.502 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:50.502 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.502 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:50.502 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:50.502 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:50.502 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:50.502 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:50.502 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:50.502 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:50.502 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:50.503 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:50.503 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf991eb640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:50.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf991eb640 4 set_mon_vals no callback set 2024-01-24T23:27:50.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf991eb640 10 monclient: shutdown 2024-01-24T23:27:50.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf991eb640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:50.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf991eb640 1 -- 172.21.15.115:0/582641137 >> v1:172.21.15.132:6790/0 conn(0x7fdf9413a470 legacy=0x7fdf9413c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf991eb640 1 -- 172.21.15.115:0/582641137 shutdown_connections 2024-01-24T23:27:50.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf991eb640 1 -- 172.21.15.115:0/582641137 >> 172.21.15.115:0/582641137 conn(0x7fdf940a5730 msgr2=0x7fdf940a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:50.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf991eb640 1 -- 172.21.15.115:0/582641137 shutdown_connections 2024-01-24T23:27:50.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.484+0000 7fdf991eb640 1 -- 172.21.15.115:0/582641137 wait complete. 2024-01-24T23:27:50.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf991eb640 10 monclient: build_initial_monmap 2024-01-24T23:27:50.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf991eb640 10 monclient: monmap: 2024-01-24T23:27:50.503 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:50.504 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:50.504 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:50.504 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:50.504 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:50.504 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:50.504 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:50.504 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:50.504 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:50.504 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:50.504 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:50.504 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:50.504 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:50.504 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:50.504 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:50.505 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf991eb640 1 Processor -- start 2024-01-24T23:27:50.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf991eb640 1 -- start start 2024-01-24T23:27:50.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf991eb640 10 monclient: init 2024-01-24T23:27:50.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf991eb640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:50.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf991eb640 10 monclient: _add_conns ranks=[8,7,6,5,3,0,1,4,2] 2024-01-24T23:27:50.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf991eb640 10 monclient(hunting): picked mon.noname-e con 0x7fdf94133c10 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:50.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf991eb640 10 monclient(hunting): picked mon.noname-i con 0x7fdf94136960 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:50.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf991eb640 10 monclient(hunting): picked mon.noname-d con 0x7fdf9413a470 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:50.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf991eb640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf9411ca20 con 0x7fdf9413a470 2024-01-24T23:27:50.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf991eb640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf94133630 con 0x7fdf94136960 2024-01-24T23:27:50.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf991eb640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf94071c30 con 0x7fdf94133c10 2024-01-24T23:27:50.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf991eb640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:50.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf991eb640 10 monclient(hunting): authenticate will time out at 1349.413088s 2024-01-24T23:27:50.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf9259c640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fdf94136960 legacy=0x7fdf94132140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:50.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf92d9d640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fdf94133c10 0x7fdf94131a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:43358/0 (socket says 172.21.15.115:43358) 2024-01-24T23:27:50.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf92d9d640 1 -- 172.21.15.115:0/3726543303 learned_addr learned my addr 172.21.15.115:0/3726543303 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:50.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf9359e640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fdf9413a470 0x7fdf941d3670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:50518/0 (socket says 172.21.15.115:50518) 2024-01-24T23:27:50.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2095351504 0 0) 0x7fdf9411ca20 con 0x7fdf9413a470 2024-01-24T23:27:50.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:50.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:50.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:50.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient(hunting): my global_id is 6936 2024-01-24T23:27:50.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf80003530 con 0x7fdf9413a470 2024-01-24T23:27:50.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199393097 0 0) 0x7fdf94071c30 con 0x7fdf94133c10 2024-01-24T23:27:50.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:50.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:50.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:50.506 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient(hunting): my global_id is 85904 2024-01-24T23:27:50.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf9411ca20 con 0x7fdf94133c10 2024-01-24T23:27:50.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2066284892 0 0) 0x7fdf80003530 con 0x7fdf9413a470 2024-01-24T23:27:50.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdf94071c30 con 0x7fdf9413a470 2024-01-24T23:27:50.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdf8c001ae0 con 0x7fdf9413a470 2024-01-24T23:27:50.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:50.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:27:50.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient(hunting): dump: 2024-01-24T23:27:50.507 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:50.507 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.507 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.507 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.507 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:50.507 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.507 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:50.507 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:50.508 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:50.508 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:50.508 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:50.508 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:50.508 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:50.508 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:50.508 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:50.508 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:50.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 804804626 0 0) 0x7fdf9411ca20 con 0x7fdf94133c10 2024-01-24T23:27:50.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdf80003530 con 0x7fdf94133c10 2024-01-24T23:27:50.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdf84001a00 con 0x7fdf94133c10 2024-01-24T23:27:50.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:50.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:27:50.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient(hunting): dump: 2024-01-24T23:27:50.509 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:50.509 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.509 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.509 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.509 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:50.509 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.509 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:50.509 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:50.509 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:50.509 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:50.509 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:50.509 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:50.509 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:50.509 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:50.510 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:50.510 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1091478653 0 0) 0x7fdf94071c30 con 0x7fdf9413a470 2024-01-24T23:27:50.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 >> v1:172.21.15.115:6793/0 conn(0x7fdf94133c10 legacy=0x7fdf94131a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 >> v1:172.21.15.132:6792/0 conn(0x7fdf94136960 legacy=0x7fdf94132140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:50.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:27:50.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 monclient: found mon.d 2024-01-24T23:27:50.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:50.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:50.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9411c740 con 0x7fdf9413a470 2024-01-24T23:27:50.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient: _finish_auth 0 2024-01-24T23:27:50.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:50.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:50.510 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf991eb640 5 monclient: authenticate success, global_id 6936 2024-01-24T23:27:50.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf991eb640 10 monclient: _renew_subs 2024-01-24T23:27:50.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf991eb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:50.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf991eb640 1 -- 172.21.15.115:0/3726543303 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9412f4e0 con 0x7fdf9413a470 2024-01-24T23:27:50.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf991eb640 10 monclient: _renew_subs 2024-01-24T23:27:50.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf991eb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:50.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf991eb640 1 -- 172.21.15.115:0/3726543303 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf941332b0 con 0x7fdf9413a470 2024-01-24T23:27:50.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdf8c003f10 con 0x7fdf9413a470 2024-01-24T23:27:50.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:50.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdf8c004440 con 0x7fdf9413a470 2024-01-24T23:27:50.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:50.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:50.511 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient: dump: 2024-01-24T23:27:50.511 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:50.511 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.511 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.512 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.512 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:50.512 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.512 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:50.512 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:50.512 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:50.512 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:50.512 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:50.512 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:50.512 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:50.512 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:50.513 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:50.513 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf9159a640 4 set_mon_vals no callback set 2024-01-24T23:27:50.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf991eb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:50.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf991eb640 1 -- 172.21.15.115:0/3726543303 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdf740065d0 con 0x7fdf74002870 2024-01-24T23:27:50.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fdf8c004620 con 0x7fdf9413a470 2024-01-24T23:27:50.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(220..220 src has 1..220) v4 ==== 12698+0+0 (unknown 1413254373 0 0) 0x7fdf8c045fb0 con 0x7fdf9413a470 2024-01-24T23:27:50.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 374437324 0 0) 0x7fdf740065d0 con 0x7fdf74002870 2024-01-24T23:27:50.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:50.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:50.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:50.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient(hunting): my global_id is 94909 2024-01-24T23:27:50.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf94071c30 con 0x7fdf74002870 2024-01-24T23:27:50.513 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 52960983 0 0) 0x7fdf94071c30 con 0x7fdf74002870 2024-01-24T23:27:50.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf9259c640 10 monclient: get_auth_request con 0x7fdf800376a0 auth_method 0 2024-01-24T23:27:50.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdf740065d0 con 0x7fdf74002870 2024-01-24T23:27:50.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdf84004b30 con 0x7fdf74002870 2024-01-24T23:27:50.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:50.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:50.514 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 10 monclient: dump: 2024-01-24T23:27:50.514 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:50.514 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.514 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.514 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.514 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:50.514 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.514 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:50.514 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:50.514 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:50.514 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:50.515 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:50.515 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:50.515 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:50.515 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:50.515 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:50.515 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 833123923 0 0) 0x7fdf740065d0 con 0x7fdf74002870 2024-01-24T23:27:50.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.488+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 --> v1:172.21.15.115:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf7400a120 con 0x7fdf74002870 2024-01-24T23:27:50.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.492+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 <== mon.4 v1:172.21.15.115:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fdf84005010 con 0x7fdf74002870 2024-01-24T23:27:50.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.492+0000 7fdf7b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:50.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.492+0000 7fdf7b7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:50.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.492+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 >> v1:172.21.15.115:6791/0 conn(0x7fdf74002870 legacy=0x7fdf74004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.515 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.492+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 >> v1:172.21.15.115:6791/0 conn(0x7fdf74002870 legacy=0x7fdf74004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:50.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf991eb640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:27:50.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf991eb640 1 -- 172.21.15.115:0/3726543303 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdf74009e10 con 0x7fdf740090d0 2024-01-24T23:27:50.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1240257117 0 0) 0x7fdf74009e10 con 0x7fdf740090d0 2024-01-24T23:27:50.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf7b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:50.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf7b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:50.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf7b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:50.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf7b7fe640 10 monclient(hunting): my global_id is 94918 2024-01-24T23:27:50.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf740065d0 con 0x7fdf740090d0 2024-01-24T23:27:50.517 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2952976530 0 0) 0x7fdf740065d0 con 0x7fdf740090d0 2024-01-24T23:27:50.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdf74009e10 con 0x7fdf740090d0 2024-01-24T23:27:50.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdf84005810 con 0x7fdf740090d0 2024-01-24T23:27:50.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf7b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:50.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf7b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:50.518 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf7b7fe640 10 monclient: dump: 2024-01-24T23:27:50.518 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:50.518 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.518 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.518 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.518 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:50.518 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.518 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:50.518 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:50.518 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:50.519 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:50.519 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:50.519 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:50.519 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:50.519 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:50.519 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:50.519 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1725992831 0 0) 0x7fdf74009e10 con 0x7fdf740090d0 2024-01-24T23:27:50.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 --> v1:172.21.15.115:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdf7400c310 con 0x7fdf740090d0 2024-01-24T23:27:50.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 <== mon.4 v1:172.21.15.115:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7680 (unknown 0 0 1291884900) 0x7fdf84005a90 con 0x7fdf740090d0 2024-01-24T23:27:50.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf7b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:27:50.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf7b7fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:50.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 >> v1:172.21.15.115:6791/0 conn(0x7fdf740090d0 legacy=0x7fdf74009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.519 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf7b7fe640 1 -- 172.21.15.115:0/3726543303 >> v1:172.21.15.115:6791/0 conn(0x7fdf740090d0 legacy=0x7fdf74009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:50.520 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:27:50.520 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:50.520 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:50.520 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:27:50.520 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 44, 2024-01-24T23:27:50.520 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:27:50.520 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:27:50.520 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:27:50.520 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:27:50.520 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:27:50.520 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:27:50.520 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:27:50.521 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:50.521 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:27:50.521 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:50.521 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:27:50.521 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:27:50.521 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:50.521 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:50.521 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:50.521 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:50.521 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:50.521 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:50.521 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:50.521 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:50.521 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:50.521 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:50.522 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:50.522 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:50.522 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:27:50.522 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:27:50.522 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:50.522 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:50.522 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:50.522 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:50.522 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:50.523 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:50.523 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:50.523 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:50.523 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:50.523 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:50.523 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:50.523 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.523 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:27:50.523 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:27:50.523 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:27:50.523 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:27:50.524 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:27:50.524 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:27:50.524 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:50.524 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:50.524 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:27:50.524 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:27:50.524 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:27:50.524 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:27:50.524 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:27:50.524 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:27:50.524 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:27:50.524 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:50.524 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:27:50.525 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:50.525 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:50.525 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:50.525 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:50.525 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:50.525 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:50.525 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:50.525 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:50.525 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:50.525 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:50.525 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:50.525 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:50.525 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:27:50.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.525 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:27:50.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.526 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:50.526 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:50.526 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.526 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.526 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.526 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:27:50.526 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.526 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.526 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.527 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:27:50.527 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:27:50.527 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.527 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.527 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.527 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:50.527 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:50.527 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.527 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.527 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.528 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:27:50.528 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.528 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.528 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.528 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:27:50.528 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:27:50.528 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.528 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.528 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.528 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:50.528 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:50.529 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.529 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.529 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.529 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:27:50.529 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.529 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.529 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.529 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:27:50.529 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:27:50.529 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.529 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.529 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.530 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:50.530 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:50.530 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.530 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.530 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.530 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:27:50.530 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.530 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.530 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.531 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:27:50.531 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:27:50.531 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.531 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.531 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.531 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:50.531 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:50.531 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.531 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.531 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.532 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:27:50.532 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.532 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.532 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.532 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:27:50.532 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:27:50.532 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.532 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.532 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.532 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:50.532 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:50.533 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.533 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.533 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.533 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:27:50.533 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.533 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.533 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.533 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:27:50.533 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:27:50.533 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.533 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.533 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.534 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:50.534 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:50.534 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.534 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.534 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.534 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:27:50.534 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.534 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.534 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.534 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:27:50.534 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:27:50.535 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.535 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.535 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.535 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:50.535 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:50.535 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.535 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.535 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.535 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:27:50.535 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.535 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.536 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.536 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:27:50.536 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:27:50.536 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.536 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.536 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.536 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:50.536 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:50.536 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.536 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.536 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.537 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:27:50.537 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.537 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.537 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.537 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:27:50.537 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:27:50.537 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.537 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.537 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.537 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.538 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.538 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:27:50.538 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:27:50.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.539 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:50.539 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:50.539 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:50.539 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.539 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:50.539 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:27:50.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.539 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:50.539 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:50.539 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:27:50.539 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.539 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:50.539 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:27:50.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.540 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:50.540 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:50.540 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:50.540 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.540 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.540 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:27:50.540 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:27:50.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf991eb640 1 -- 172.21.15.115:0/3726543303 >> v1:172.21.15.132:6812/1835401922 conn(0x7fdf800376a0 legacy=0x7fdf80039b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf991eb640 10 monclient: shutdown 2024-01-24T23:27:50.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf991eb640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:50.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf991eb640 1 -- 172.21.15.115:0/3726543303 >> v1:172.21.15.115:6792/0 conn(0x7fdf9413a470 legacy=0x7fdf941d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf9359e640 1 -- 172.21.15.115:0/3726543303 reap_dead start 2024-01-24T23:27:50.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf9359e640 1 -- 172.21.15.115:0/3726543303 reap_dead start 2024-01-24T23:27:50.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf991eb640 1 -- 172.21.15.115:0/3726543303 shutdown_connections 2024-01-24T23:27:50.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf991eb640 1 -- 172.21.15.115:0/3726543303 >> 172.21.15.115:0/3726543303 conn(0x7fdf940a5730 msgr2=0x7fdf940a4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:50.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf991eb640 1 -- 172.21.15.115:0/3726543303 shutdown_connections 2024-01-24T23:27:50.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.496+0000 7fdf991eb640 1 -- 172.21.15.115:0/3726543303 wait complete. 2024-01-24T23:27:50.542 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-01-24T23:27:50.542 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon d 2024-01-24T23:27:50.543 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-01-24T23:27:50.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.656+0000 7f8325522640 10 monclient: get_monmap_and_config 2024-01-24T23:27:50.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.656+0000 7f8325522640 10 monclient: build_initial_monmap 2024-01-24T23:27:50.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.656+0000 7f8325522640 10 monclient: monmap: 2024-01-24T23:27:50.660 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:50.660 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.660 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:50.661452+0000 2024-01-24T23:27:50.660 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:50.661452+0000 2024-01-24T23:27:50.660 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:50.660 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.660 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:50.660 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:50.660 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:50.660 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:50.660 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:50.661 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:50.661 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:50.661 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:50.661 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:50.661 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.656+0000 7f8325522640 1 Processor -- start 2024-01-24T23:27:50.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.656+0000 7f8325522640 1 -- start start 2024-01-24T23:27:50.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.656+0000 7f8325522640 10 monclient: init 2024-01-24T23:27:50.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.656+0000 7f8325522640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:50.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.656+0000 7f8325522640 10 monclient: _add_conns ranks=[0,2,4,8,6,5,3,7,1] 2024-01-24T23:27:50.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.656+0000 7f8325522640 10 monclient(hunting): picked mon.noname-a con 0x7f8320133d90 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:50.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.656+0000 7f8325522640 10 monclient(hunting): picked mon.noname-b con 0x7f8320134170 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:50.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.656+0000 7f8325522640 10 monclient(hunting): picked mon.noname-c con 0x7f832013dcf0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:50.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.656+0000 7f8325522640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832004ed70 con 0x7f8320133d90 2024-01-24T23:27:50.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.656+0000 7f8325522640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832009b250 con 0x7f8320134170 2024-01-24T23:27:50.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.656+0000 7f8325522640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83201345d0 con 0x7f832013dcf0 2024-01-24T23:27:50.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.656+0000 7f8325522640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:50.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.656+0000 7f8325522640 10 monclient(hunting): authenticate will time out at 1349.583897s 2024-01-24T23:27:50.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.656+0000 7f831effd640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f8320133d90 legacy=0x7f83200a19f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:27:50.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.656+0000 7f831e7fc640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f8320134170 legacy=0x7f83200a2100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:50.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831f7fe640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f832013dcf0 0x7f83201400e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:48664/0 (socket says 172.21.15.115:48664) 2024-01-24T23:27:50.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831f7fe640 1 -- 172.21.15.115:0/342581208 learned_addr learned my addr 172.21.15.115:0/342581208 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:50.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 1 -- 172.21.15.115:0/342581208 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 222062339 0 0) 0x7f83201345d0 con 0x7f832013dcf0 2024-01-24T23:27:50.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:50.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:50.669 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:50.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 10 monclient(hunting): my global_id is 94927 2024-01-24T23:27:50.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 1 -- 172.21.15.115:0/342581208 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8308003530 con 0x7f832013dcf0 2024-01-24T23:27:50.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 1 -- 172.21.15.115:0/342581208 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1528908657 0 0) 0x7f8308003530 con 0x7f832013dcf0 2024-01-24T23:27:50.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 1 -- 172.21.15.115:0/342581208 >> v1:172.21.15.115:6790/0 conn(0x7f8320134170 legacy=0x7f83200a2100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:50.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 1 -- 172.21.15.115:0/342581208 >> v1:172.21.15.115:6789/0 conn(0x7f8320133d90 legacy=0x7f83200a19f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:50.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 10 monclient: _finish_hunting 0 2024-01-24T23:27:50.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 1 monclient: found mon.noname-c 2024-01-24T23:27:50.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:50.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:27:50.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 1 -- 172.21.15.115:0/342581208 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83200a2810 con 0x7f832013dcf0 2024-01-24T23:27:50.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 10 monclient: _finish_auth 0 2024-01-24T23:27:50.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 10 monclient: _check_auth_tickets 2024-01-24T23:27:50.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:50.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 1 -- 172.21.15.115:0/342581208 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8318001b80 con 0x7f832013dcf0 2024-01-24T23:27:50.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 5 monclient: authenticate success, global_id 94927 2024-01-24T23:27:50.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:50.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:50.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:27:50.671 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 10 monclient: dump: 2024-01-24T23:27:50.671 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:50.671 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.671 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.671 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.671 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:50.671 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.671 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:50.671 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:50.671 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:50.671 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:50.672 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:50.672 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:50.672 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:50.672 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:50.672 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:50.672 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:50.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 1 -- 172.21.15.115:0/342581208 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8318001e50 con 0x7f832013dcf0 2024-01-24T23:27:50.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:50.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:50.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 4 set_mon_vals no callback set 2024-01-24T23:27:50.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 1 -- 172.21.15.115:0/342581208 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8318004f70 con 0x7f832013dcf0 2024-01-24T23:27:50.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:50.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:50.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831dffb640 10 monclient: dump: 2024-01-24T23:27:50.673 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:50.673 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.673 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.673 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.673 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:50.673 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.673 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:50.673 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:50.673 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:50.673 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:50.673 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:50.673 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:50.673 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:50.673 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:50.674 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:50.674 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 10 monclient: shutdown 2024-01-24T23:27:50.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:50.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 1 -- 172.21.15.115:0/342581208 >> v1:172.21.15.115:6791/0 conn(0x7f832013dcf0 legacy=0x7f83201400e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 1 -- 172.21.15.115:0/342581208 shutdown_connections 2024-01-24T23:27:50.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 1 -- 172.21.15.115:0/342581208 >> 172.21.15.115:0/342581208 conn(0x7f83200a65e0 msgr2=0x7f83200a2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:50.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 1 -- 172.21.15.115:0/342581208 shutdown_connections 2024-01-24T23:27:50.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 1 -- 172.21.15.115:0/342581208 wait complete. 2024-01-24T23:27:50.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 10 monclient: build_initial_monmap 2024-01-24T23:27:50.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 10 monclient: monmap: 2024-01-24T23:27:50.674 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:50.674 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:50.674 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:50.674 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:50.675 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:50.675 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:50.675 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:50.675 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:50.675 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:50.675 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:50.675 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:50.675 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:50.675 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:50.675 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:50.676 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:50.676 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 1 Processor -- start 2024-01-24T23:27:50.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 1 -- start start 2024-01-24T23:27:50.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 10 monclient: init 2024-01-24T23:27:50.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:50.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 10 monclient: _add_conns ranks=[2,6,3,8,1,7,5,0,4] 2024-01-24T23:27:50.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 10 monclient(hunting): picked mon.noname-b con 0x7f8320133d90 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:50.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 10 monclient(hunting): picked mon.noname-d con 0x7f8320134170 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:50.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 10 monclient(hunting): picked mon.noname-g con 0x7f832013dcf0 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:50.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83201396e0 con 0x7f8320133d90 2024-01-24T23:27:50.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832013b130 con 0x7f832013dcf0 2024-01-24T23:27:50.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83200676e0 con 0x7f8320134170 2024-01-24T23:27:50.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:50.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 10 monclient(hunting): authenticate will time out at 1349.586703s 2024-01-24T23:27:50.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831effd640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f8320133d90 legacy=0x7f83201cb3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:50.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831e7fc640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f8320134170 0x7f832013cf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:50532/0 (socket says 172.21.15.115:50532) 2024-01-24T23:27:50.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831e7fc640 1 -- 172.21.15.115:0/485190670 learned_addr learned my addr 172.21.15.115:0/485190670 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:50.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3638782845 0 0) 0x7f83200676e0 con 0x7f8320134170 2024-01-24T23:27:50.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:50.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:50.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:50.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 10 monclient(hunting): my global_id is 6945 2024-01-24T23:27:50.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f830c003530 con 0x7f8320134170 2024-01-24T23:27:50.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3422188052 0 0) 0x7f830c003530 con 0x7f8320134170 2024-01-24T23:27:50.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f83200676e0 con 0x7f8320134170 2024-01-24T23:27:50.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f83100042f0 con 0x7f8320134170 2024-01-24T23:27:50.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:50.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:27:50.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 10 monclient(hunting): dump: 2024-01-24T23:27:50.678 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:50.678 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.678 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.678 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.678 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:50.678 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.679 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:50.679 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:50.679 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:50.679 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:50.679 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:50.679 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:50.679 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:50.679 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:50.679 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:50.679 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:50.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 207912165 0 0) 0x7f832013b130 con 0x7f832013dcf0 2024-01-24T23:27:50.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:50.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:50.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:50.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 10 monclient(hunting): my global_id is 6708 2024-01-24T23:27:50.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f830c003530 con 0x7f832013dcf0 2024-01-24T23:27:50.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4050799664 0 0) 0x7f83200676e0 con 0x7f8320134170 2024-01-24T23:27:50.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 >> v1:172.21.15.132:6790/0 conn(0x7f832013dcf0 legacy=0x7f83201d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 >> v1:172.21.15.115:6790/0 conn(0x7f8320133d90 legacy=0x7f83201cb3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:50.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 10 monclient: _finish_hunting 0 2024-01-24T23:27:50.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 1 monclient: found mon.d 2024-01-24T23:27:50.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:50.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:50.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832009c650 con 0x7f8320134170 2024-01-24T23:27:50.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 10 monclient: _finish_auth 0 2024-01-24T23:27:50.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:50.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:50.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 5 monclient: authenticate success, global_id 6945 2024-01-24T23:27:50.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 10 monclient: _renew_subs 2024-01-24T23:27:50.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:50.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 1 -- 172.21.15.115:0/485190670 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832013b9d0 con 0x7f8320134170 2024-01-24T23:27:50.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 10 monclient: _renew_subs 2024-01-24T23:27:50.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:50.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f8325522640 1 -- 172.21.15.115:0/485190670 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f832013d610 con 0x7f8320134170 2024-01-24T23:27:50.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f83100046e0 con 0x7f8320134170 2024-01-24T23:27:50.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:50.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8310004f50 con 0x7f8320134170 2024-01-24T23:27:50.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:50.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f831d7fa640 4 set_mon_vals no callback set 2024-01-24T23:27:50.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:50.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.660+0000 7f83077fe640 10 monclient: dump: 2024-01-24T23:27:50.682 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:50.682 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.682 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.682 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.682 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:50.682 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.682 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:50.682 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:50.682 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:50.682 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:50.682 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:50.682 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:50.682 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:50.682 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:50.683 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:50.683 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.664+0000 7f8325522640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:50.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.664+0000 7f8325522640 1 -- 172.21.15.115:0/485190670 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f83000065d0 con 0x7f8300002870 2024-01-24T23:27:50.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.664+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f831000f9d0 con 0x7f8320134170 2024-01-24T23:27:50.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.664+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(220..220 src has 1..220) v4 ==== 12698+0+0 (unknown 1413254373 0 0) 0x7f8310045bc0 con 0x7f8320134170 2024-01-24T23:27:50.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.664+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960631553 0 0) 0x7f83000065d0 con 0x7f8300002870 2024-01-24T23:27:50.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.664+0000 7f83077fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:50.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.664+0000 7f83077fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:50.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.664+0000 7f83077fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:50.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.664+0000 7f83077fe640 10 monclient(hunting): my global_id is 6954 2024-01-24T23:27:50.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.664+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83200676e0 con 0x7f8300002870 2024-01-24T23:27:50.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.664+0000 7f831f7fe640 10 monclient: get_auth_request con 0x7f830c036f20 auth_method 0 2024-01-24T23:27:50.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.664+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3244729233 0 0) 0x7f83200676e0 con 0x7f8300002870 2024-01-24T23:27:50.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.664+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f83000065d0 con 0x7f8300002870 2024-01-24T23:27:50.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.664+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f83140040d0 con 0x7f8300002870 2024-01-24T23:27:50.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.664+0000 7f83077fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:50.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.664+0000 7f83077fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:50.684 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.664+0000 7f83077fe640 10 monclient: dump: 2024-01-24T23:27:50.684 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:50.684 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.684 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.684 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.684 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:50.684 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.684 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:50.684 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:50.684 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:50.684 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:50.685 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:50.685 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:50.685 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:50.685 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:50.685 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:50.685 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.664+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1252638489 0 0) 0x7f83000065d0 con 0x7f8300002870 2024-01-24T23:27:50.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.664+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 --> v1:172.21.15.115:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f830000a120 con 0x7f8300002870 2024-01-24T23:27:50.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.664+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f83140064a0 con 0x7f8300002870 2024-01-24T23:27:50.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.664+0000 7f83077fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:50.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.664+0000 7f83077fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:50.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.664+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 >> v1:172.21.15.115:6792/0 conn(0x7f8300002870 legacy=0x7f8300004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.664+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 >> v1:172.21.15.115:6792/0 conn(0x7f8300002870 legacy=0x7f8300004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:50.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.668+0000 7f8325522640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:27:50.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f8325522640 1 -- 172.21.15.115:0/485190670 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8300009e10 con 0x7f83000090d0 2024-01-24T23:27:50.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351636486 0 0) 0x7f8300009e10 con 0x7f83000090d0 2024-01-24T23:27:50.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f83077fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:50.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f83077fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:50.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f83077fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:50.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f83077fe640 10 monclient(hunting): my global_id is 6963 2024-01-24T23:27:50.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83000065d0 con 0x7f83000090d0 2024-01-24T23:27:50.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 416356672 0 0) 0x7f83000065d0 con 0x7f83000090d0 2024-01-24T23:27:50.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8300009e10 con 0x7f83000090d0 2024-01-24T23:27:50.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8314004c70 con 0x7f83000090d0 2024-01-24T23:27:50.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f83077fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:50.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f83077fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:50.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f83077fe640 10 monclient: dump: 2024-01-24T23:27:50.688 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:50.688 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.688 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.688 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.688 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:50.688 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.688 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:50.688 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:50.688 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:50.689 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:50.689 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:50.689 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:50.689 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:50.689 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:50.689 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:50.689 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 546917170 0 0) 0x7f8300009e10 con 0x7f83000090d0 2024-01-24T23:27:50.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 --> v1:172.21.15.115:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f830000c310 con 0x7f83000090d0 2024-01-24T23:27:50.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7680 (unknown 0 0 837286103) 0x7f83140062b0 con 0x7f83000090d0 2024-01-24T23:27:50.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f83077fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:27:50.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f83077fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:50.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 >> v1:172.21.15.115:6792/0 conn(0x7f83000090d0 legacy=0x7f8300009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f83077fe640 1 -- 172.21.15.115:0/485190670 >> v1:172.21.15.115:6792/0 conn(0x7f83000090d0 legacy=0x7f8300009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:50.690 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:27:50.690 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:50.690 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:50.690 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:27:50.690 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 44, 2024-01-24T23:27:50.690 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:27:50.690 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:27:50.690 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:27:50.691 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:27:50.691 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:27:50.692 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:27:50.692 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:27:50.692 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:50.692 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:27:50.692 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:50.692 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:27:50.692 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:27:50.692 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:50.692 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:50.692 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:50.693 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:50.693 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:50.693 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:50.693 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:50.693 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:50.693 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:50.693 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:50.693 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:50.693 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:50.693 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:27:50.693 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:27:50.693 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:50.693 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:50.694 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:50.694 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:50.694 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:50.694 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:50.694 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:50.694 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:50.694 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:50.694 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:50.694 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:50.694 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.694 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:27:50.694 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:27:50.694 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:27:50.695 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:27:50.695 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:27:50.695 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:27:50.695 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:50.695 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:50.695 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:27:50.695 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:27:50.695 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:27:50.695 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:27:50.695 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:27:50.695 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:27:50.695 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:27:50.695 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:50.695 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:27:50.696 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:50.696 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:50.696 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:50.696 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:50.696 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:50.696 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:50.696 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:50.696 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:50.696 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:50.696 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:50.696 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:50.696 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:50.696 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:27:50.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.697 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:27:50.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.697 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:50.697 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:50.697 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.697 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.697 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.697 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:27:50.697 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.697 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.697 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.698 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:27:50.698 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:27:50.698 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.698 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.698 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.698 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:50.698 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:50.698 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.698 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.698 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.698 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:27:50.699 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.699 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.699 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.699 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:27:50.699 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:27:50.699 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.699 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.699 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.699 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:50.699 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:50.699 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.700 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.700 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.700 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:27:50.700 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.701 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.701 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.701 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:27:50.701 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:27:50.701 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.701 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.701 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.701 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:50.701 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:50.701 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.701 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.702 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.702 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:27:50.702 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.702 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.702 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.702 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:27:50.702 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:27:50.702 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.702 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.702 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.703 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:50.703 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:50.703 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.703 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.703 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.703 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:27:50.703 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.703 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.703 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.703 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:27:50.703 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:27:50.704 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.704 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.704 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.704 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:50.704 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:50.704 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.704 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.704 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.704 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:27:50.704 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.704 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.705 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.705 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:27:50.705 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:27:50.705 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.705 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.705 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.705 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:50.705 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:50.705 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.705 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.706 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.706 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:27:50.706 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.706 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.706 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.706 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:27:50.706 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:27:50.706 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.706 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.706 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.706 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:50.707 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:50.707 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.707 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.707 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.707 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:27:50.707 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.707 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.707 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.707 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:27:50.707 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:27:50.707 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.707 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.707 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.708 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:50.708 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:50.708 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.708 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.708 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.708 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:27:50.708 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.708 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.708 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.708 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:27:50.709 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:27:50.709 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.709 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.709 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.709 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.709 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.709 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:27:50.709 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:27:50.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.709 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:50.709 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:50.709 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:50.709 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.710 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:50.710 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:27:50.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.710 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:50.710 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:50.710 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:50.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.710 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:50.710 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:27:50.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.710 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:50.710 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:50.710 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 3 2024-01-24T23:27:50.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.710 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.711 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:27:50.711 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:27:50.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f8325522640 1 -- 172.21.15.115:0/485190670 >> v1:172.21.15.132:6812/1835401922 conn(0x7f830c036f20 legacy=0x7f830c0393e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f8325522640 10 monclient: shutdown 2024-01-24T23:27:50.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f8325522640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:50.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f8325522640 1 -- 172.21.15.115:0/485190670 >> v1:172.21.15.115:6792/0 conn(0x7f8320134170 legacy=0x7f832013cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f831f7fe640 1 -- 172.21.15.115:0/485190670 reap_dead start 2024-01-24T23:27:50.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f831f7fe640 1 -- 172.21.15.115:0/485190670 reap_dead start 2024-01-24T23:27:50.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f8325522640 1 -- 172.21.15.115:0/485190670 shutdown_connections 2024-01-24T23:27:50.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f8325522640 1 -- 172.21.15.115:0/485190670 >> 172.21.15.115:0/485190670 conn(0x7f83200a65e0 msgr2=0x7f832009f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:50.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f8325522640 1 -- 172.21.15.115:0/485190670 shutdown_connections 2024-01-24T23:27:50.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.672+0000 7f8325522640 1 -- 172.21.15.115:0/485190670 wait complete. 2024-01-24T23:27:50.713 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-01-24T23:27:50.713 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon e 2024-01-24T23:27:50.713 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-01-24T23:27:50.747 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~107s 2024-01-24T23:27:50.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.832+0000 7fe615915640 10 monclient: get_monmap_and_config 2024-01-24T23:27:50.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.832+0000 7fe615915640 10 monclient: build_initial_monmap 2024-01-24T23:27:50.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.832+0000 7fe615915640 10 monclient: monmap: 2024-01-24T23:27:50.836 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:50.836 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.836 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:50.837669+0000 2024-01-24T23:27:50.836 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:50.837669+0000 2024-01-24T23:27:50.836 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:50.836 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.836 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:50.837 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:50.837 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:50.837 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:50.837 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:50.837 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:50.837 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:50.837 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:50.837 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:50.837 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.832+0000 7fe615915640 1 Processor -- start 2024-01-24T23:27:50.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.832+0000 7fe615915640 1 -- start start 2024-01-24T23:27:50.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.832+0000 7fe615915640 10 monclient: init 2024-01-24T23:27:50.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:50.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 10 monclient: _add_conns ranks=[5,3,6,8,2,7,0,1,4] 2024-01-24T23:27:50.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 10 monclient(hunting): picked mon.noname-h con 0x7fe610134740 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:50.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 10 monclient(hunting): picked mon.noname-g con 0x7fe61009ec80 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:50.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 10 monclient(hunting): picked mon.noname-d con 0x7fe610135230 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:50.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe61004ed70 con 0x7fe61009ec80 2024-01-24T23:27:50.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6100a0d90 con 0x7fe610134740 2024-01-24T23:27:50.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe61009e7c0 con 0x7fe610135230 2024-01-24T23:27:50.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:50.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 10 monclient(hunting): authenticate will time out at 1349.760398s 2024-01-24T23:27:50.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60f7fe640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fe610135230 0x7fe61013d450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:50556/0 (socket says 172.21.15.115:50556) 2024-01-24T23:27:50.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60f7fe640 1 -- 172.21.15.115:0/2142444863 learned_addr learned my addr 172.21.15.115:0/2142444863 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:50.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 1 -- 172.21.15.115:0/2142444863 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189294553 0 0) 0x7fe61009e7c0 con 0x7fe610135230 2024-01-24T23:27:50.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:50.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:50.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:50.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 10 monclient(hunting): my global_id is 6972 2024-01-24T23:27:50.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 1 -- 172.21.15.115:0/2142444863 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5f8003530 con 0x7fe610135230 2024-01-24T23:27:50.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 1 -- 172.21.15.115:0/2142444863 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2238228084 0 0) 0x7fe5f8003530 con 0x7fe610135230 2024-01-24T23:27:50.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 1 -- 172.21.15.115:0/2142444863 >> v1:172.21.15.132:6791/0 conn(0x7fe610134740 legacy=0x7fe610134b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 1 -- 172.21.15.115:0/2142444863 >> v1:172.21.15.132:6790/0 conn(0x7fe61009ec80 legacy=0x7fe610139d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 10 monclient: _finish_hunting 0 2024-01-24T23:27:50.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 1 monclient: found mon.noname-d 2024-01-24T23:27:50.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:50.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:27:50.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 1 -- 172.21.15.115:0/2142444863 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6100a1070 con 0x7fe610135230 2024-01-24T23:27:50.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 10 monclient: _finish_auth 0 2024-01-24T23:27:50.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 10 monclient: _check_auth_tickets 2024-01-24T23:27:50.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:50.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 1 -- 172.21.15.115:0/2142444863 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe6080017c0 con 0x7fe610135230 2024-01-24T23:27:50.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 5 monclient: authenticate success, global_id 6972 2024-01-24T23:27:50.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:50.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:50.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:27:50.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 10 monclient: dump: 2024-01-24T23:27:50.848 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:50.848 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.848 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.848 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.848 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:50.848 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.848 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:50.848 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:50.848 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:50.848 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:50.848 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:50.848 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:50.848 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:50.848 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:50.849 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:50.849 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:50.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 1 -- 172.21.15.115:0/2142444863 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe608001ae0 con 0x7fe610135230 2024-01-24T23:27:50.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:50.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:50.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 4 set_mon_vals no callback set 2024-01-24T23:27:50.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 1 -- 172.21.15.115:0/2142444863 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe608004bf0 con 0x7fe610135230 2024-01-24T23:27:50.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:50.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:50.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60dffb640 10 monclient: dump: 2024-01-24T23:27:50.849 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:50.849 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.849 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.850 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.850 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:50.850 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.850 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:50.850 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:50.850 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:50.850 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:50.850 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:50.850 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:50.850 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:50.850 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:50.850 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:50.850 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 10 monclient: shutdown 2024-01-24T23:27:50.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:50.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 1 -- 172.21.15.115:0/2142444863 >> v1:172.21.15.115:6792/0 conn(0x7fe610135230 legacy=0x7fe61013d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 1 -- 172.21.15.115:0/2142444863 shutdown_connections 2024-01-24T23:27:50.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 1 -- 172.21.15.115:0/2142444863 >> 172.21.15.115:0/2142444863 conn(0x7fe6100a5730 msgr2=0x7fe6100a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:50.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 1 -- 172.21.15.115:0/2142444863 shutdown_connections 2024-01-24T23:27:50.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 1 -- 172.21.15.115:0/2142444863 wait complete. 2024-01-24T23:27:50.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 10 monclient: build_initial_monmap 2024-01-24T23:27:50.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 10 monclient: monmap: 2024-01-24T23:27:50.851 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:50.851 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:50.851 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:50.851 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:50.851 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:50.851 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:50.851 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:50.851 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:50.852 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:50.852 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:50.852 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:50.852 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:50.852 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:50.852 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:50.852 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:50.852 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 1 Processor -- start 2024-01-24T23:27:50.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 1 -- start start 2024-01-24T23:27:50.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 10 monclient: init 2024-01-24T23:27:50.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:50.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 10 monclient: _add_conns ranks=[2,1,6,3,8,4,7,0,5] 2024-01-24T23:27:50.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 10 monclient(hunting): picked mon.noname-b con 0x7fe61009ec80 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:50.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 10 monclient(hunting): picked mon.noname-f con 0x7fe610134740 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:50.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 10 monclient(hunting): picked mon.noname-d con 0x7fe610135230 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:50.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6100716b0 con 0x7fe610134740 2024-01-24T23:27:50.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe610071c00 con 0x7fe61009ec80 2024-01-24T23:27:50.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60effd640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fe61009ec80 legacy=0x7fe61013df60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:50.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6101379e0 con 0x7fe610135230 2024-01-24T23:27:50.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:50.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe615915640 10 monclient(hunting): authenticate will time out at 1349.763246s 2024-01-24T23:27:50.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60f7fe640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fe610135230 0x7fe6101d1c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:50558/0 (socket says 172.21.15.115:50558) 2024-01-24T23:27:50.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe60f7fe640 1 -- 172.21.15.115:0/1509619946 learned_addr learned my addr 172.21.15.115:0/1509619946 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:50.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1220870976 0 0) 0x7fe6101379e0 con 0x7fe610135230 2024-01-24T23:27:50.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe5f7fff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:50.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe5f7fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:50.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe5f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:50.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe5f7fff640 10 monclient(hunting): my global_id is 6981 2024-01-24T23:27:50.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5fc003530 con 0x7fe610135230 2024-01-24T23:27:50.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3681059010 0 0) 0x7fe5fc003530 con 0x7fe610135230 2024-01-24T23:27:50.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe6101379e0 con 0x7fe610135230 2024-01-24T23:27:50.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe608001780 con 0x7fe610135230 2024-01-24T23:27:50.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe5f7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:50.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe5f7fff640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:27:50.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe5f7fff640 10 monclient(hunting): dump: 2024-01-24T23:27:50.854 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:50.854 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.854 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.854 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.854 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:50.854 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.855 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:50.855 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:50.855 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:50.855 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:50.855 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:50.855 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:50.855 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:50.855 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:50.855 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:50.855 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe5f7fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:50.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71339762 0 0) 0x7fe6100716b0 con 0x7fe610134740 2024-01-24T23:27:50.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe5f7fff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:50.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe5f7fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:50.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe5f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:50.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe5f7fff640 10 monclient(hunting): my global_id is 124318 2024-01-24T23:27:50.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5fc003530 con 0x7fe610134740 2024-01-24T23:27:50.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.836+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2840600015 0 0) 0x7fe6101379e0 con 0x7fe610135230 2024-01-24T23:27:50.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 >> v1:172.21.15.115:6790/0 conn(0x7fe61009ec80 legacy=0x7fe61013df60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:50.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 >> v1:172.21.15.132:6789/0 conn(0x7fe610134740 legacy=0x7fe6101ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 10 monclient: _finish_hunting 0 2024-01-24T23:27:50.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 1 monclient: found mon.d 2024-01-24T23:27:50.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:50.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:50.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe610137140 con 0x7fe610135230 2024-01-24T23:27:50.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 10 monclient: _finish_auth 0 2024-01-24T23:27:50.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 10 monclient: _check_auth_tickets 2024-01-24T23:27:50.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:50.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe615915640 5 monclient: authenticate success, global_id 6981 2024-01-24T23:27:50.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe615915640 10 monclient: _renew_subs 2024-01-24T23:27:50.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe615915640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:50.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe615915640 1 -- 172.21.15.115:0/1509619946 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe610137420 con 0x7fe610135230 2024-01-24T23:27:50.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe615915640 10 monclient: _renew_subs 2024-01-24T23:27:50.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe615915640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:50.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe615915640 1 -- 172.21.15.115:0/1509619946 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe610136250 con 0x7fe610135230 2024-01-24T23:27:50.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe608001c50 con 0x7fe610135230 2024-01-24T23:27:50.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:50.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe608005410 con 0x7fe610135230 2024-01-24T23:27:50.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:50.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe60d7fa640 4 set_mon_vals no callback set 2024-01-24T23:27:50.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:50.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 10 monclient: dump: 2024-01-24T23:27:50.857 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:50.857 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.857 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.858 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.858 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:50.858 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.858 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:50.858 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:50.858 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:50.858 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:50.858 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:50.858 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:50.858 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:50.858 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:50.858 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:50.858 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe615915640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:50.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe615915640 1 -- 172.21.15.115:0/1509619946 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe5f00065d0 con 0x7fe5f0002870 2024-01-24T23:27:50.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fe60800fe90 con 0x7fe610135230 2024-01-24T23:27:50.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(220..220 src has 1..220) v4 ==== 12698+0+0 (unknown 1413254373 0 0) 0x7fe608045c50 con 0x7fe610135230 2024-01-24T23:27:50.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213025274 0 0) 0x7fe5f00065d0 con 0x7fe5f0002870 2024-01-24T23:27:50.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:50.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:50.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:50.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 10 monclient(hunting): my global_id is 85913 2024-01-24T23:27:50.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6101379e0 con 0x7fe5f0002870 2024-01-24T23:27:50.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe60e7fc640 10 monclient: get_auth_request con 0x7fe5fc036f70 auth_method 0 2024-01-24T23:27:50.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2377888552 0 0) 0x7fe6101379e0 con 0x7fe5f0002870 2024-01-24T23:27:50.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe5f00065d0 con 0x7fe5f0002870 2024-01-24T23:27:50.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe600001bf0 con 0x7fe5f0002870 2024-01-24T23:27:50.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:50.859 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:27:50.860 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 10 monclient: dump: 2024-01-24T23:27:50.860 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:50.860 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.860 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.860 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.860 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:50.860 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.860 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:50.860 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:50.860 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:50.860 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:50.860 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:50.860 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:50.860 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:50.860 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:50.861 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:50.861 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 895379803 0 0) 0x7fe5f00065d0 con 0x7fe5f0002870 2024-01-24T23:27:50.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 --> v1:172.21.15.115:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe5f000a120 con 0x7fe5f0002870 2024-01-24T23:27:50.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fe600005b50 con 0x7fe5f0002870 2024-01-24T23:27:50.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:50.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:50.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 >> v1:172.21.15.115:6793/0 conn(0x7fe5f0002870 legacy=0x7fe5f0004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.861 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.840+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 >> v1:172.21.15.115:6793/0 conn(0x7fe5f0002870 legacy=0x7fe5f0004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:50.862 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe615915640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:27:50.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe615915640 1 -- 172.21.15.115:0/1509619946 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe5f0009e10 con 0x7fe5f00090d0 2024-01-24T23:27:50.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4196729094 0 0) 0x7fe5f0009e10 con 0x7fe5f00090d0 2024-01-24T23:27:50.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe5f7fff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:50.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe5f7fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:50.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe5f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:50.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe5f7fff640 10 monclient(hunting): my global_id is 85922 2024-01-24T23:27:50.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5f00065d0 con 0x7fe5f00090d0 2024-01-24T23:27:50.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2631310231 0 0) 0x7fe5f00065d0 con 0x7fe5f00090d0 2024-01-24T23:27:50.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe5f0009e10 con 0x7fe5f00090d0 2024-01-24T23:27:50.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe600003f80 con 0x7fe5f00090d0 2024-01-24T23:27:50.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe5f7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:50.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe5f7fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:27:50.863 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe5f7fff640 10 monclient: dump: 2024-01-24T23:27:50.863 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:50.864 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:50.864 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.864 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:50.864 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:50.864 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:50.864 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:50.864 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:50.864 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:50.864 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:50.864 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:50.864 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:50.864 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:50.864 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:50.864 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:50.864 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:50.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2474841331 0 0) 0x7fe5f0009e10 con 0x7fe5f00090d0 2024-01-24T23:27:50.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 --> v1:172.21.15.115:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe5f000c310 con 0x7fe5f00090d0 2024-01-24T23:27:50.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7680 (unknown 0 0 885300643) 0x7fe600004460 con 0x7fe5f00090d0 2024-01-24T23:27:50.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe5f7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:27:50.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe5f7fff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:50.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 >> v1:172.21.15.115:6793/0 conn(0x7fe5f00090d0 legacy=0x7fe5f0009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.865 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe5f7fff640 1 -- 172.21.15.115:0/1509619946 >> v1:172.21.15.115:6793/0 conn(0x7fe5f00090d0 legacy=0x7fe5f0009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:50.865 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:27:50.865 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:50.865 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:50.865 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:27:50.866 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 44, 2024-01-24T23:27:50.866 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:27:50.866 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:27:50.866 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:27:50.866 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:27:50.866 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:27:50.866 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:27:50.866 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:27:50.866 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:50.866 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:27:50.866 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:50.866 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:27:50.866 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:27:50.866 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:50.867 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:50.867 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:50.867 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:50.867 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:50.867 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:50.867 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:50.867 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:50.867 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:50.867 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:50.867 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:50.867 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:50.867 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:27:50.867 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:27:50.867 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:50.868 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:50.868 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:50.868 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:50.868 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:50.868 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:50.868 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:50.868 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:50.868 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:50.868 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:50.868 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:50.868 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.868 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:27:50.868 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:27:50.868 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:27:50.869 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:27:50.869 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:27:50.869 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:27:50.869 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:50.869 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:50.869 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:27:50.869 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:27:50.869 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:27:50.869 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:27:50.869 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:27:50.869 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:27:50.869 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:27:50.869 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:50.869 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:27:50.869 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:50.870 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:50.870 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:50.870 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:50.870 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:50.870 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:50.870 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:50.870 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:50.870 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:50.870 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:50.870 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:50.870 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:50.870 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:27:50.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.870 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:27:50.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.871 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:50.871 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:50.871 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.871 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.871 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.871 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:27:50.871 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.871 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.871 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.871 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:27:50.871 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:27:50.871 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.871 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.872 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.872 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:50.872 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:50.872 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.872 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.872 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.872 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:27:50.872 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.872 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.872 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.873 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:27:50.873 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:27:50.873 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.873 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.873 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.873 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:50.873 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:50.873 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.873 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.873 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.873 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:27:50.873 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.874 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.874 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.874 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:27:50.874 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:27:50.874 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.874 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.874 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.874 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:50.874 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:50.874 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.874 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.875 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.875 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:27:50.875 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.875 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.875 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:27:50.875 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:27:50.875 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.875 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.875 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.875 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:50.876 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:50.876 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.876 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.876 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.876 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:27:50.876 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.876 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.876 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.877 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:27:50.877 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:27:50.877 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.877 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.877 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.877 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:50.877 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:50.877 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.877 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.877 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.877 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:27:50.878 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.878 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.878 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.878 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:27:50.878 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:27:50.878 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.878 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.878 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.878 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:50.878 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:50.878 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.879 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.879 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.879 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:27:50.879 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.879 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.879 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.879 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:27:50.879 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:27:50.879 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.879 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.879 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.880 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:50.880 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:50.880 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.880 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.880 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.880 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:27:50.880 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.880 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.880 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.880 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:27:50.880 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:27:50.881 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.881 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.881 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.881 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:50.881 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:50.881 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:50.881 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:50.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.881 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:50.881 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:27:50.881 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:50.881 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.882 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.882 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:27:50.882 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:27:50.882 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:50.882 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:50.882 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:50.882 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.882 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.882 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:27:50.882 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:27:50.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.882 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:50.883 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:50.883 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:50.883 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.883 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:50.883 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:27:50.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.883 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:50.883 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:50.883 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:27:50.883 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.883 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:50.883 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": [ 2024-01-24T23:27:50.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:50.883 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:50.884 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:50.884 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:50.884 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:50.884 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:50.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:50.884 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:27:50.884 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:27:50.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe615915640 1 -- 172.21.15.115:0/1509619946 >> v1:172.21.15.132:6812/1835401922 conn(0x7fe5fc036f70 legacy=0x7fe5fc039430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe615915640 10 monclient: shutdown 2024-01-24T23:27:50.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe615915640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:50.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe615915640 1 -- 172.21.15.115:0/1509619946 >> v1:172.21.15.115:6792/0 conn(0x7fe610135230 legacy=0x7fe6101d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:50.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe60f7fe640 1 -- 172.21.15.115:0/1509619946 reap_dead start 2024-01-24T23:27:50.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe60f7fe640 1 -- 172.21.15.115:0/1509619946 reap_dead start 2024-01-24T23:27:50.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe615915640 1 -- 172.21.15.115:0/1509619946 shutdown_connections 2024-01-24T23:27:50.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe615915640 1 -- 172.21.15.115:0/1509619946 >> 172.21.15.115:0/1509619946 conn(0x7fe6100a5730 msgr2=0x7fe6100a0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:50.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe615915640 1 -- 172.21.15.115:0/1509619946 shutdown_connections 2024-01-24T23:27:50.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:50.848+0000 7fe615915640 1 -- 172.21.15.115:0/1509619946 wait complete. 2024-01-24T23:27:50.886 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-01-24T23:27:50.886 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon f 2024-01-24T23:27:50.887 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-01-24T23:27:51.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.004+0000 7ff40d5c7640 10 monclient: get_monmap_and_config 2024-01-24T23:27:51.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.004+0000 7ff40d5c7640 10 monclient: build_initial_monmap 2024-01-24T23:27:51.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.004+0000 7ff40d5c7640 10 monclient: monmap: 2024-01-24T23:27:51.010 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:51.009786+0000 2024-01-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:51.009786+0000 2024-01-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:51.011 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:51.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.004+0000 7ff40d5c7640 1 Processor -- start 2024-01-24T23:27:51.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.004+0000 7ff40d5c7640 1 -- start start 2024-01-24T23:27:51.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.004+0000 7ff40d5c7640 10 monclient: init 2024-01-24T23:27:51.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:51.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 10 monclient: _add_conns ranks=[1,0,2,6,4,3,7,8,5] 2024-01-24T23:27:51.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 10 monclient(hunting): picked mon.noname-f con 0x7ff408133bf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:51.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 10 monclient(hunting): picked mon.noname-a con 0x7ff408133fd0 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:51.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 10 monclient(hunting): picked mon.noname-b con 0x7ff4081343b0 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:51.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff40804ed70 con 0x7ff408133fd0 2024-01-24T23:27:51.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff4067fc640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7ff408133fd0 legacy=0x7ff40813bd60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:27:51.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4080a4b60 con 0x7ff408133bf0 2024-01-24T23:27:51.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff4077fe640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7ff4081343b0 legacy=0x7ff40813f490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:51.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff40809ae50 con 0x7ff4081343b0 2024-01-24T23:27:51.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:51.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 10 monclient(hunting): authenticate will time out at 1349.932452s 2024-01-24T23:27:51.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff406ffd640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7ff408133bf0 0x7ff40809bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.115:52960/0 (socket says 172.21.15.115:52960) 2024-01-24T23:27:51.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff406ffd640 1 -- 172.21.15.115:0/2259021196 learned_addr learned my addr 172.21.15.115:0/2259021196 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:51.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 1 -- 172.21.15.115:0/2259021196 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2103196773 0 0) 0x7ff4080a4b60 con 0x7ff408133bf0 2024-01-24T23:27:51.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:51.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:51.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:51.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 10 monclient(hunting): my global_id is 124327 2024-01-24T23:27:51.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 1 -- 172.21.15.115:0/2259021196 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3f4003530 con 0x7ff408133bf0 2024-01-24T23:27:51.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 1 -- 172.21.15.115:0/2259021196 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2554452405 0 0) 0x7ff3f4003530 con 0x7ff408133bf0 2024-01-24T23:27:51.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 1 -- 172.21.15.115:0/2259021196 >> v1:172.21.15.115:6790/0 conn(0x7ff4081343b0 legacy=0x7ff40813f490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:51.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 1 -- 172.21.15.115:0/2259021196 >> v1:172.21.15.115:6789/0 conn(0x7ff408133fd0 legacy=0x7ff40813bd60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:51.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:27:51.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 1 monclient: found mon.noname-f 2024-01-24T23:27:51.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:51.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:27:51.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 1 -- 172.21.15.115:0/2259021196 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40809b130 con 0x7ff408133bf0 2024-01-24T23:27:51.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 10 monclient: _finish_auth 0 2024-01-24T23:27:51.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:27:51.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:51.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 1 -- 172.21.15.115:0/2259021196 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff3f00017c0 con 0x7ff408133bf0 2024-01-24T23:27:51.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:51.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:27:51.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 10 monclient: dump: 2024-01-24T23:27:51.021 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:51.021 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:51.021 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.021 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.021 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:51.021 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:51.021 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:51.021 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 5 monclient: authenticate success, global_id 124327 2024-01-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 1 -- 172.21.15.115:0/2259021196 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff3f00019e0 con 0x7ff408133bf0 2024-01-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:51.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 4 set_mon_vals no callback set 2024-01-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 1 -- 172.21.15.115:0/2259021196 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff3f0004bf0 con 0x7ff408133bf0 2024-01-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff405ffb640 10 monclient: dump: 2024-01-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:51.023 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 10 monclient: shutdown 2024-01-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 1 -- 172.21.15.115:0/2259021196 >> v1:172.21.15.132:6789/0 conn(0x7ff408133bf0 legacy=0x7ff40809bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 1 -- 172.21.15.115:0/2259021196 shutdown_connections 2024-01-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 1 -- 172.21.15.115:0/2259021196 >> 172.21.15.115:0/2259021196 conn(0x7ff4080662f0 msgr2=0x7ff408066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 1 -- 172.21.15.115:0/2259021196 shutdown_connections 2024-01-24T23:27:51.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 1 -- 172.21.15.115:0/2259021196 wait complete. 2024-01-24T23:27:51.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 10 monclient: build_initial_monmap 2024-01-24T23:27:51.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 10 monclient: monmap: 2024-01-24T23:27:51.025 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:51.025 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:51.025 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:51.025 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:51.025 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:51.025 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:51.025 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:51.025 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:51.025 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:51.025 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:51.025 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:51.025 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:51.025 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:51.026 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:51.026 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:51.026 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:51.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 1 Processor -- start 2024-01-24T23:27:51.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 1 -- start start 2024-01-24T23:27:51.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.008+0000 7ff40d5c7640 10 monclient: init 2024-01-24T23:27:51.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff40d5c7640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:51.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff40d5c7640 10 monclient: _add_conns ranks=[5,3,4,1,6,0,7,2,8] 2024-01-24T23:27:51.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff40d5c7640 10 monclient(hunting): picked mon.noname-h con 0x7ff408133bf0 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:51.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff40d5c7640 10 monclient(hunting): picked mon.noname-g con 0x7ff408133fd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:51.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff40d5c7640 10 monclient(hunting): picked mon.noname-c con 0x7ff4081343b0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:51.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff40d5c7640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff40809c810 con 0x7ff408133fd0 2024-01-24T23:27:51.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff40d5c7640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff408120940 con 0x7ff4081343b0 2024-01-24T23:27:51.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff40d5c7640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4080a1f10 con 0x7ff408133bf0 2024-01-24T23:27:51.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff40d5c7640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:51.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff40d5c7640 10 monclient(hunting): authenticate will time out at 1349.936710s 2024-01-24T23:27:51.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff4077fe640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7ff4081343b0 0x7ff4081d3c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:48680/0 (socket says 172.21.15.115:48680) 2024-01-24T23:27:51.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff4077fe640 1 -- 172.21.15.115:0/2920045299 learned_addr learned my addr 172.21.15.115:0/2920045299 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:51.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2797256101 0 0) 0x7ff408120940 con 0x7ff4081343b0 2024-01-24T23:27:51.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:51.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:51.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:51.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient(hunting): my global_id is 94936 2024-01-24T23:27:51.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3e8003530 con 0x7ff4081343b0 2024-01-24T23:27:51.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1205535942 0 0) 0x7ff3e8003530 con 0x7ff4081343b0 2024-01-24T23:27:51.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff408120940 con 0x7ff4081343b0 2024-01-24T23:27:51.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff3f80041e0 con 0x7ff4081343b0 2024-01-24T23:27:51.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:51.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:27:51.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient(hunting): dump: 2024-01-24T23:27:51.028 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:51.028 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:51.028 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.028 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.028 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:51.028 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:51.028 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:51.028 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:51.028 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:51.028 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:51.028 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:51.028 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:51.028 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:51.029 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:51.029 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:51.029 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:51.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:51.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1844159262 0 0) 0x7ff40809c810 con 0x7ff408133fd0 2024-01-24T23:27:51.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:51.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:51.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:51.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient(hunting): my global_id is 6726 2024-01-24T23:27:51.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3e8003530 con 0x7ff408133fd0 2024-01-24T23:27:51.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799861124 0 0) 0x7ff4080a1f10 con 0x7ff408133bf0 2024-01-24T23:27:51.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:51.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:51.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:51.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient(hunting): my global_id is 6764 2024-01-24T23:27:51.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff40809c810 con 0x7ff408133bf0 2024-01-24T23:27:51.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2804774324 0 0) 0x7ff408120940 con 0x7ff4081343b0 2024-01-24T23:27:51.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 >> v1:172.21.15.132:6791/0 conn(0x7ff408133bf0 legacy=0x7ff408132f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:51.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 >> v1:172.21.15.132:6790/0 conn(0x7ff408133fd0 legacy=0x7ff4081d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:51.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:27:51.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 monclient: found mon.c 2024-01-24T23:27:51.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:51.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:51.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4080a5990 con 0x7ff4081343b0 2024-01-24T23:27:51.030 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient: _finish_auth 0 2024-01-24T23:27:51.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:51.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:51.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff40d5c7640 5 monclient: authenticate success, global_id 94936 2024-01-24T23:27:51.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff40d5c7640 10 monclient: _renew_subs 2024-01-24T23:27:51.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff40d5c7640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:51.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff40d5c7640 1 -- 172.21.15.115:0/2920045299 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff408120660 con 0x7ff4081343b0 2024-01-24T23:27:51.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff40d5c7640 10 monclient: _renew_subs 2024-01-24T23:27:51.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff40d5c7640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:51.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff40d5c7640 1 -- 172.21.15.115:0/2920045299 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4080a2ee0 con 0x7ff4081343b0 2024-01-24T23:27:51.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff3f8004490 con 0x7ff4081343b0 2024-01-24T23:27:51.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:51.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff3f80049c0 con 0x7ff4081343b0 2024-01-24T23:27:51.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:51.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff4057fa640 4 set_mon_vals no callback set 2024-01-24T23:27:51.031 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:51.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient: dump: 2024-01-24T23:27:51.032 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:51.032 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:51.032 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.032 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.032 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:51.032 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:51.032 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:51.032 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:51.032 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:51.032 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:51.032 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:51.032 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:51.032 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:51.033 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:51.033 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:51.033 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:51.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff40d5c7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:51.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff40d5c7640 1 -- 172.21.15.115:0/2920045299 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff3ec0065d0 con 0x7ff3ec002870 2024-01-24T23:27:51.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7ff3f8004ba0 con 0x7ff4081343b0 2024-01-24T23:27:51.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(220..220 src has 1..220) v4 ==== 12698+0+0 (unknown 1413254373 0 0) 0x7ff3f8045b00 con 0x7ff4081343b0 2024-01-24T23:27:51.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 488238992 0 0) 0x7ff3ec0065d0 con 0x7ff3ec002870 2024-01-24T23:27:51.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:51.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:51.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:51.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient(hunting): my global_id is 124336 2024-01-24T23:27:51.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff408120940 con 0x7ff3ec002870 2024-01-24T23:27:51.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff4067fc640 10 monclient: get_auth_request con 0x7ff3e80370b0 auth_method 0 2024-01-24T23:27:51.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1206439758 0 0) 0x7ff408120940 con 0x7ff3ec002870 2024-01-24T23:27:51.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff3ec0065d0 con 0x7ff3ec002870 2024-01-24T23:27:51.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff3f0004bf0 con 0x7ff3ec002870 2024-01-24T23:27:51.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:51.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:27:51.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 10 monclient: dump: 2024-01-24T23:27:51.034 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:51.034 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:51.034 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.034 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.034 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:51.034 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:51.034 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:51.034 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:51.034 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:51.035 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:51.035 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:51.035 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:51.035 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:51.035 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:51.035 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:51.035 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:51.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3244622058 0 0) 0x7ff3ec0065d0 con 0x7ff3ec002870 2024-01-24T23:27:51.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.012+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 --> v1:172.21.15.132:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff3ec00a120 con 0x7ff3ec002870 2024-01-24T23:27:51.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.016+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 <== mon.1 v1:172.21.15.132:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7ff3f00015c0 con 0x7ff3ec002870 2024-01-24T23:27:51.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.016+0000 7ff3ff7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:51.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.016+0000 7ff3ff7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:51.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.016+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 >> v1:172.21.15.132:6789/0 conn(0x7ff3ec002870 legacy=0x7ff3ec004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:51.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.016+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 >> v1:172.21.15.132:6789/0 conn(0x7ff3ec002870 legacy=0x7ff3ec004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:51.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.020+0000 7ff40d5c7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:27:51.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.020+0000 7ff40d5c7640 1 -- 172.21.15.115:0/2920045299 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff3ec009e10 con 0x7ff3ec0090d0 2024-01-24T23:27:51.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.020+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2542603753 0 0) 0x7ff3ec009e10 con 0x7ff3ec0090d0 2024-01-24T23:27:51.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.020+0000 7ff3ff7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:51.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.020+0000 7ff3ff7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:51.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.020+0000 7ff3ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:51.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.020+0000 7ff3ff7fe640 10 monclient(hunting): my global_id is 124345 2024-01-24T23:27:51.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.020+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3ec0065d0 con 0x7ff3ec0090d0 2024-01-24T23:27:51.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.024+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 423702104 0 0) 0x7ff3ec0065d0 con 0x7ff3ec0090d0 2024-01-24T23:27:51.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.024+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff3ec009e10 con 0x7ff3ec0090d0 2024-01-24T23:27:51.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.024+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff3f0005530 con 0x7ff3ec0090d0 2024-01-24T23:27:51.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.024+0000 7ff3ff7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:51.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.024+0000 7ff3ff7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:27:51.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.024+0000 7ff3ff7fe640 10 monclient: dump: 2024-01-24T23:27:51.038 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:51.038 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:51.038 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.038 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.038 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:51.038 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:51.038 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:51.038 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:51.038 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:51.038 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:51.038 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:51.039 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:51.039 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:51.039 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:51.039 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:51.039 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:51.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.024+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3704418679 0 0) 0x7ff3ec009e10 con 0x7ff3ec0090d0 2024-01-24T23:27:51.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.024+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 --> v1:172.21.15.132:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff3ec00c310 con 0x7ff3ec0090d0 2024-01-24T23:27:51.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.024+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 <== mon.1 v1:172.21.15.132:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7510 (unknown 0 0 338477710) 0x7ff3f00076f0 con 0x7ff3ec0090d0 2024-01-24T23:27:51.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.024+0000 7ff3ff7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:27:51.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.024+0000 7ff3ff7fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:51.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.024+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 >> v1:172.21.15.132:6789/0 conn(0x7ff3ec0090d0 legacy=0x7ff3ec009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:51.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.024+0000 7ff3ff7fe640 1 -- 172.21.15.115:0/2920045299 >> v1:172.21.15.132:6789/0 conn(0x7ff3ec0090d0 legacy=0x7ff3ec009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:51.040 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:27:51.040 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:51.040 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:51.040 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:27:51.040 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 44, 2024-01-24T23:27:51.040 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:27:51.040 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:27:51.040 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:27:51.040 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:27:51.040 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:27:51.040 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:27:51.040 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:27:51.040 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:51.041 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:27:51.041 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:51.041 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:27:51.041 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:27:51.041 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:51.041 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:51.041 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:51.041 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:51.041 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:51.041 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:51.041 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:51.041 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:51.041 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:51.041 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:51.042 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:51.042 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:51.042 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:27:51.042 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:27:51.042 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:51.042 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:51.042 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:51.042 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:51.042 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:51.042 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:51.042 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:51.042 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:51.042 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:51.042 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:27:51.043 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:51.044 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:51.045 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:51.046 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:51.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.047 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:51.047 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:27:51.047 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:51.047 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.047 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:51.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.047 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:27:51.047 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:27:51.047 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:51.047 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:51.047 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:51.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.048 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:51.048 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:51.048 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:51.048 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:51.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.048 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:51.048 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:27:51.048 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:51.048 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.048 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:51.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.048 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:27:51.048 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:27:51.048 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:51.049 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:51.049 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:51.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.049 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:51.049 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:51.049 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:51.049 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:51.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.049 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:51.049 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:27:51.049 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:51.049 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.049 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:51.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.050 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:27:51.050 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:27:51.050 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:51.050 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:51.050 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:51.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.050 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:51.050 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:51.050 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:51.050 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:51.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.050 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:51.050 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:27:51.051 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:51.051 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.051 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:51.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.051 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:27:51.051 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:27:51.051 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:51.051 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:51.051 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:51.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.051 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:51.051 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:51.052 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:51.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.053 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:51.053 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:51.053 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:51.053 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:51.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.053 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:51.053 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:27:51.053 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:51.053 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.053 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:51.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.053 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:27:51.053 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:51.054 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.055 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:51.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.055 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:27:51.055 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:27:51.055 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:51.055 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:51.055 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:51.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.055 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:51.055 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:51.055 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:51.055 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:51.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.055 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:51.056 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:27:51.056 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:51.056 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.056 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:51.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.056 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:27:51.056 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:27:51.056 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:51.056 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:51.056 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:51.056 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.056 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:51.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.056 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:27:51.057 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:27:51.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.057 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:51.057 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:51.057 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:51.057 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.057 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:51.057 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:27:51.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.057 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:51.057 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:51.057 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:51.057 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.057 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:51.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.058 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:27:51.058 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:27:51.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.024+0000 7ff40d5c7640 1 -- 172.21.15.115:0/2920045299 >> v1:172.21.15.132:6812/1835401922 conn(0x7ff3e80370b0 legacy=0x7ff3e8039570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:51.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.024+0000 7ff40d5c7640 10 monclient: shutdown 2024-01-24T23:27:51.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.024+0000 7ff40d5c7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:51.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.024+0000 7ff40d5c7640 1 -- 172.21.15.115:0/2920045299 >> v1:172.21.15.115:6791/0 conn(0x7ff4081343b0 legacy=0x7ff4081d3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:51.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.024+0000 7ff4077fe640 1 -- 172.21.15.115:0/2920045299 reap_dead start 2024-01-24T23:27:51.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.024+0000 7ff4077fe640 1 -- 172.21.15.115:0/2920045299 reap_dead start 2024-01-24T23:27:51.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.024+0000 7ff40d5c7640 1 -- 172.21.15.115:0/2920045299 shutdown_connections 2024-01-24T23:27:51.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.024+0000 7ff40d5c7640 1 -- 172.21.15.115:0/2920045299 >> 172.21.15.115:0/2920045299 conn(0x7ff4080662f0 msgr2=0x7ff4081be660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:51.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.024+0000 7ff40d5c7640 1 -- 172.21.15.115:0/2920045299 shutdown_connections 2024-01-24T23:27:51.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.024+0000 7ff40d5c7640 1 -- 172.21.15.115:0/2920045299 wait complete. 2024-01-24T23:27:51.060 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-01-24T23:27:51.060 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon g 2024-01-24T23:27:51.060 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-01-24T23:27:51.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.130+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h9"}]=0 removed item id -43 name 'h9' from crush map v221) v1 ==== 118+0+0 (unknown 2717415021 0 0) 0x7f9168013140 con 0x7f917409ff20 2024-01-24T23:27:51.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.130+0000 7f9170ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h9"}] 2024-01-24T23:27:51.132 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.130+0000 7f9170ff9640 10 monclient: _finish_command 2 = system:0 removed item id -43 name 'h9' from crush map 2024-01-24T23:27:51.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.176+0000 7f3e5bc93640 10 monclient: get_monmap_and_config 2024-01-24T23:27:51.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.176+0000 7f3e5bc93640 10 monclient: build_initial_monmap 2024-01-24T23:27:51.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.176+0000 7f3e5bc93640 10 monclient: monmap: 2024-01-24T23:27:51.181 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:51.181 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:51.181 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:51.182806+0000 2024-01-24T23:27:51.181 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:51.182806+0000 2024-01-24T23:27:51.181 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:51.181 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:51.182 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:51.182 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:51.182 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:51.182 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:51.182 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:51.182 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:51.182 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:51.182 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:51.182 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:51.182 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:51.189 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 1 Processor -- start 2024-01-24T23:27:51.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 1 -- start start 2024-01-24T23:27:51.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 10 monclient: init 2024-01-24T23:27:51.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:51.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 10 monclient: _add_conns ranks=[0,8,1,4,5,3,2,6,7] 2024-01-24T23:27:51.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 10 monclient(hunting): picked mon.noname-a con 0x7f3e540662f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:51.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 10 monclient(hunting): picked mon.noname-e con 0x7f3e5409b690 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:51.190 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 10 monclient(hunting): picked mon.noname-f con 0x7f3e5409bab0 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:51.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e5404ed70 con 0x7f3e540662f0 2024-01-24T23:27:51.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e54099e80 con 0x7f3e5409bab0 2024-01-24T23:27:51.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e540666d0 con 0x7f3e5409b690 2024-01-24T23:27:51.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:51.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 10 monclient(hunting): authenticate will time out at 1350.105364s 2024-01-24T23:27:51.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e59a2f640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f3e540662f0 legacy=0x7f3e54136b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:27:51.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5922e640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f3e5409b690 0x7f3e5413be30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:43386/0 (socket says 172.21.15.115:43386) 2024-01-24T23:27:51.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5922e640 1 -- 172.21.15.115:0/1846321215 learned_addr learned my addr 172.21.15.115:0/1846321215 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:51.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 1 -- 172.21.15.115:0/1846321215 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2674440896 0 0) 0x7f3e540666d0 con 0x7f3e5409b690 2024-01-24T23:27:51.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 10 monclient(hunting): _negotiate 2024-01-24T23:27:51.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:51.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:51.191 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 10 monclient(hunting): my global_id is 85931 2024-01-24T23:27:51.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 1 -- 172.21.15.115:0/1846321215 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e3c003530 con 0x7f3e5409b690 2024-01-24T23:27:51.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 1 -- 172.21.15.115:0/1846321215 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2376989092 0 0) 0x7f3e3c003530 con 0x7f3e5409b690 2024-01-24T23:27:51.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 1 -- 172.21.15.115:0/1846321215 >> v1:172.21.15.132:6789/0 conn(0x7f3e5409bab0 legacy=0x7f3e5413f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:51.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 1 -- 172.21.15.115:0/1846321215 >> v1:172.21.15.115:6789/0 conn(0x7f3e540662f0 legacy=0x7f3e54136b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:51.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 10 monclient: _finish_hunting 0 2024-01-24T23:27:51.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 1 monclient: found mon.noname-e 2024-01-24T23:27:51.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:51.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:27:51.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 1 -- 172.21.15.115:0/1846321215 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e540998c0 con 0x7f3e5409b690 2024-01-24T23:27:51.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 10 monclient: _finish_auth 0 2024-01-24T23:27:51.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 10 monclient: _check_auth_tickets 2024-01-24T23:27:51.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:51.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 5 monclient: authenticate success, global_id 85931 2024-01-24T23:27:51.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:51.192 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 1 -- 172.21.15.115:0/1846321215 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3e480017c0 con 0x7f3e5409b690 2024-01-24T23:27:51.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:51.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:27:51.193 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 10 monclient: dump: 2024-01-24T23:27:51.193 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:51.193 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:51.193 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.193 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.193 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:51.193 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:51.193 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:51.193 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:51.193 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:51.193 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:51.193 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:51.193 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:51.193 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:51.194 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:51.194 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:51.194 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:51.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:51.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 1 -- 172.21.15.115:0/1846321215 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3e48001ae0 con 0x7f3e5409b690 2024-01-24T23:27:51.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:51.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:51.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 1 -- 172.21.15.115:0/1846321215 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3e48004bf0 con 0x7f3e5409b690 2024-01-24T23:27:51.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 4 set_mon_vals no callback set 2024-01-24T23:27:51.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:51.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:27:51.194 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e58a2d640 10 monclient: dump: 2024-01-24T23:27:51.194 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:51.194 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:51.194 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.195 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.195 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:51.195 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:51.195 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:51.195 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:51.195 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:51.195 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:51.195 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:51.195 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:51.195 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:51.195 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:51.195 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:51.195 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:51.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 10 monclient: shutdown 2024-01-24T23:27:51.195 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:51.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 1 -- 172.21.15.115:0/1846321215 >> v1:172.21.15.115:6793/0 conn(0x7f3e5409b690 legacy=0x7f3e5413be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:51.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 1 -- 172.21.15.115:0/1846321215 shutdown_connections 2024-01-24T23:27:51.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 1 -- 172.21.15.115:0/1846321215 >> 172.21.15.115:0/1846321215 conn(0x7f3e540a2110 msgr2=0x7f3e540a24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:51.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 1 -- 172.21.15.115:0/1846321215 shutdown_connections 2024-01-24T23:27:51.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 1 -- 172.21.15.115:0/1846321215 wait complete. 2024-01-24T23:27:51.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 10 monclient: build_initial_monmap 2024-01-24T23:27:51.196 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 10 monclient: monmap: 2024-01-24T23:27:51.196 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:51.196 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:51.196 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:51.196 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:51.196 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:51.196 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:51.196 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:51.197 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:51.197 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:51.197 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:51.197 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:51.197 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:51.197 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:51.197 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:51.197 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:51.197 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:51.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.180+0000 7f3e5bc93640 1 Processor -- start 2024-01-24T23:27:51.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e5bc93640 1 -- start start 2024-01-24T23:27:51.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e5bc93640 10 monclient: init 2024-01-24T23:27:51.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e5bc93640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:51.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e5bc93640 10 monclient: _add_conns ranks=[2,3,8,5,0,1,4,7,6] 2024-01-24T23:27:51.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e5bc93640 10 monclient(hunting): picked mon.noname-b con 0x7f3e540662f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:51.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e5bc93640 10 monclient(hunting): picked mon.noname-g con 0x7f3e5409b690 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:51.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e5bc93640 10 monclient(hunting): picked mon.noname-e con 0x7f3e5409bab0 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:51.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e5bc93640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e54093c10 con 0x7f3e540662f0 2024-01-24T23:27:51.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e5bc93640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e54072750 con 0x7f3e5409b690 2024-01-24T23:27:51.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e5bc93640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e54098610 con 0x7f3e5409bab0 2024-01-24T23:27:51.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e5bc93640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:51.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e5bc93640 10 monclient(hunting): authenticate will time out at 1350.108538s 2024-01-24T23:27:51.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e59a2f640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f3e540662f0 legacy=0x7f3e5408e480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:51.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e5a230640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f3e5409bab0 0x7f3e541d3bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:43392/0 (socket says 172.21.15.115:43392) 2024-01-24T23:27:51.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e5a230640 1 -- 172.21.15.115:0/2151710514 learned_addr learned my addr 172.21.15.115:0/2151710514 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:51.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3032295263 0 0) 0x7f3e54098610 con 0x7f3e5409bab0 2024-01-24T23:27:51.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:51.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:51.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:51.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 10 monclient(hunting): my global_id is 85940 2024-01-24T23:27:51.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e40003530 con 0x7f3e5409bab0 2024-01-24T23:27:51.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 189175297 0 0) 0x7f3e40003530 con 0x7f3e5409bab0 2024-01-24T23:27:51.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3e54098610 con 0x7f3e5409bab0 2024-01-24T23:27:51.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3e4c001a00 con 0x7f3e5409bab0 2024-01-24T23:27:51.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:51.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:27:51.199 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 10 monclient(hunting): dump: 2024-01-24T23:27:51.199 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:51.199 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:51.199 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.199 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.199 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:51.199 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:51.199 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:51.199 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:51.199 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:51.200 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:51.200 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:51.200 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:51.200 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:51.200 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:51.200 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:51.200 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:51.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:51.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3400450640 0 0) 0x7f3e54098610 con 0x7f3e5409bab0 2024-01-24T23:27:51.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 >> v1:172.21.15.132:6790/0 conn(0x7f3e5409b690 legacy=0x7f3e5408eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:51.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 >> v1:172.21.15.115:6790/0 conn(0x7f3e540662f0 legacy=0x7f3e5408e480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:51.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 10 monclient: _finish_hunting 0 2024-01-24T23:27:51.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 1 monclient: found mon.e 2024-01-24T23:27:51.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:51.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:27:51.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e54072200 con 0x7f3e5409bab0 2024-01-24T23:27:51.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 10 monclient: _finish_auth 0 2024-01-24T23:27:51.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 10 monclient: _check_auth_tickets 2024-01-24T23:27:51.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:51.201 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e5bc93640 5 monclient: authenticate success, global_id 85940 2024-01-24T23:27:51.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e5bc93640 10 monclient: _renew_subs 2024-01-24T23:27:51.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e5bc93640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:27:51.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e5bc93640 1 -- 172.21.15.115:0/2151710514 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e54095150 con 0x7f3e5409bab0 2024-01-24T23:27:51.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e5bc93640 10 monclient: _renew_subs 2024-01-24T23:27:51.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e5bc93640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:27:51.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e5bc93640 1 -- 172.21.15.115:0/2151710514 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e54093740 con 0x7f3e5409bab0 2024-01-24T23:27:51.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3e4c0038d0 con 0x7f3e5409bab0 2024-01-24T23:27:51.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:51.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3e4c003e00 con 0x7f3e5409bab0 2024-01-24T23:27:51.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:51.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:27:51.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e53fff640 4 set_mon_vals no callback set 2024-01-24T23:27:51.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 10 monclient: dump: 2024-01-24T23:27:51.203 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:51.203 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:51.203 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.203 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.203 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:51.203 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:51.203 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:51.203 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:51.203 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:51.203 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:51.203 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:51.203 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:51.203 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:51.203 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:51.203 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:51.204 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:51.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e5bc93640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:51.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e5bc93640 1 -- 172.21.15.115:0/2151710514 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3e340065d0 con 0x7f3e34002870 2024-01-24T23:27:51.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f3e4c003f60 con 0x7f3e5409bab0 2024-01-24T23:27:51.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(221..221 src has 1..221) v4 ==== 12488+0+0 (unknown 227041204 0 0) 0x7f3e4c0459f0 con 0x7f3e5409bab0 2024-01-24T23:27:51.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e5922e640 10 monclient: get_auth_request con 0x7f3e40036ef0 auth_method 0 2024-01-24T23:27:51.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695768182 0 0) 0x7f3e340065d0 con 0x7f3e34002870 2024-01-24T23:27:51.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:51.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:51.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:51.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 10 monclient(hunting): my global_id is 6744 2024-01-24T23:27:51.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e54098610 con 0x7f3e34002870 2024-01-24T23:27:51.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2999941134 0 0) 0x7f3e54098610 con 0x7f3e34002870 2024-01-24T23:27:51.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3e340065d0 con 0x7f3e34002870 2024-01-24T23:27:51.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3e44004490 con 0x7f3e34002870 2024-01-24T23:27:51.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:51.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:51.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 10 monclient: dump: 2024-01-24T23:27:51.208 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:51.208 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:51.208 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.209 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.209 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:51.209 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:51.209 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:51.209 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:51.209 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:51.209 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:51.209 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:51.209 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:51.209 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:51.209 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:51.209 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:51.209 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:51.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3371075533 0 0) 0x7f3e340065d0 con 0x7f3e34002870 2024-01-24T23:27:51.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.184+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 --> v1:172.21.15.132:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3e3400a120 con 0x7f3e34002870 2024-01-24T23:27:51.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.188+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f3e44006860 con 0x7f3e34002870 2024-01-24T23:27:51.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.188+0000 7f3e527fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:51.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.188+0000 7f3e527fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:51.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.188+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 >> v1:172.21.15.132:6790/0 conn(0x7f3e34002870 legacy=0x7f3e34004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:51.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.188+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 >> v1:172.21.15.132:6790/0 conn(0x7f3e34002870 legacy=0x7f3e34004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:51.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.192+0000 7f3e5bc93640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:27:51.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.192+0000 7f3e5bc93640 1 -- 172.21.15.115:0/2151710514 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3e34009e10 con 0x7f3e340090d0 2024-01-24T23:27:51.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.192+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 776424457 0 0) 0x7f3e34009e10 con 0x7f3e340090d0 2024-01-24T23:27:51.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.192+0000 7f3e527fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:51.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.192+0000 7f3e527fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:51.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.192+0000 7f3e527fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:51.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.192+0000 7f3e527fc640 10 monclient(hunting): my global_id is 6753 2024-01-24T23:27:51.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.192+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e340065d0 con 0x7f3e340090d0 2024-01-24T23:27:51.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.192+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2349498560 0 0) 0x7f3e340065d0 con 0x7f3e340090d0 2024-01-24T23:27:51.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.192+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3e34009e10 con 0x7f3e340090d0 2024-01-24T23:27:51.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.192+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3e44005070 con 0x7f3e340090d0 2024-01-24T23:27:51.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.192+0000 7f3e527fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:51.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.192+0000 7f3e527fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:51.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.192+0000 7f3e527fc640 10 monclient: dump: 2024-01-24T23:27:51.213 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:51.213 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:51.213 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.213 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.213 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:51.213 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:51.213 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:51.213 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:51.213 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:51.213 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:51.213 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:51.213 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:51.213 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:51.213 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:51.213 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:51.214 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:51.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.192+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1153895610 0 0) 0x7f3e34009e10 con 0x7f3e340090d0 2024-01-24T23:27:51.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.192+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 --> v1:172.21.15.132:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3e3400c310 con 0x7f3e340090d0 2024-01-24T23:27:51.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.196+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7680 (unknown 0 0 3026238052) 0x7f3e44006630 con 0x7f3e340090d0 2024-01-24T23:27:51.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.196+0000 7f3e527fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:27:51.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.196+0000 7f3e527fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:51.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.196+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 >> v1:172.21.15.132:6790/0 conn(0x7f3e340090d0 legacy=0x7f3e34009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:51.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.196+0000 7f3e527fc640 1 -- 172.21.15.115:0/2151710514 >> v1:172.21.15.132:6790/0 conn(0x7f3e340090d0 legacy=0x7f3e34009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:51.214 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:27:51.214 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:51.214 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:27:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 44, 2024-01-24T23:27:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:27:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:27:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:27:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:27:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:27:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:27:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:27:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 1, 2024-01-24T23:27:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:27:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:27:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:27:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:27:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:27:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:27:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:27:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:27:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:27:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:27:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:27:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:27:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:27:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:27:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:27:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:27:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:27:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:27:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:27:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:27:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:27:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:27:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:27:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:27:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:27:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:27:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:27:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:27:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:27:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:27:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:27:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:27:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:27:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:27:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:27:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:27:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:27:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:27:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:51.228 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:51.228 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:51.228 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:51.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.228 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:51.228 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:27:51.228 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:51.228 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.228 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:51.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.228 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:27:51.228 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:27:51.228 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:51.228 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:27:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:27:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:27:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:27:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:27:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:27:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:27:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:27:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:27:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:27:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:27:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 4 2024-01-24T23:27:51.233 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:51.233 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:51.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:51.233 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:27:51.233 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:27:51.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.196+0000 7f3e5bc93640 1 -- 172.21.15.115:0/2151710514 >> v1:172.21.15.132:6812/1835401922 conn(0x7f3e40036ef0 legacy=0x7f3e400393b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:51.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.196+0000 7f3e5bc93640 10 monclient: shutdown 2024-01-24T23:27:51.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.196+0000 7f3e5bc93640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:51.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.196+0000 7f3e5bc93640 1 -- 172.21.15.115:0/2151710514 >> v1:172.21.15.115:6793/0 conn(0x7f3e5409bab0 legacy=0x7f3e541d3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:51.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.196+0000 7f3e5a230640 1 -- 172.21.15.115:0/2151710514 reap_dead start 2024-01-24T23:27:51.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.196+0000 7f3e5a230640 1 -- 172.21.15.115:0/2151710514 reap_dead start 2024-01-24T23:27:51.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.196+0000 7f3e5bc93640 1 -- 172.21.15.115:0/2151710514 shutdown_connections 2024-01-24T23:27:51.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.196+0000 7f3e5bc93640 1 -- 172.21.15.115:0/2151710514 >> 172.21.15.115:0/2151710514 conn(0x7f3e540a2110 msgr2=0x7f3e54092370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:51.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.196+0000 7f3e5bc93640 1 -- 172.21.15.115:0/2151710514 shutdown_connections 2024-01-24T23:27:51.235 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.196+0000 7f3e5bc93640 1 -- 172.21.15.115:0/2151710514 wait complete. 2024-01-24T23:27:51.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.234+0000 7f917a70f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h9"}] 2024-01-24T23:27:51.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.234+0000 7f917a70f640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h9"}] 2024-01-24T23:27:51.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.234+0000 7f917a70f640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:27:51.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.234+0000 7f917a70f640 1 -- 172.21.15.132:0/58969232 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush rm", "name": "h9"} v 0) v1 -- 0x7f9154002220 con 0x7f917409ff20 2024-01-24T23:27:51.236 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-01-24T23:27:51.236 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon h 2024-01-24T23:27:51.236 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-01-24T23:27:51.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.234+0000 7f9170ff9640 1 -- 172.21.15.132:0/58969232 <== mon.8 v1:172.21.15.115:6793/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h9"}]=0 device 'h9' does not appear in the crush map v221) v1 ==== 118+0+0 (unknown 298877331 0 0) 0x7f91680148c0 con 0x7f917409ff20 2024-01-24T23:27:51.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.234+0000 7f9170ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h9"}] 2024-01-24T23:27:51.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.234+0000 7f9170ff9640 10 monclient: _finish_command 3 = system:0 device 'h9' does not appear in the crush map 2024-01-24T23:27:51.237 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h9' does not appear in the crush map 2024-01-24T23:27:51.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.238+0000 7f917a70f640 1 -- 172.21.15.132:0/58969232 >> v1:172.21.15.132:6812/1835401922 conn(0x7f9160037c40 legacy=0x7f916003a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:51.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.238+0000 7f917a70f640 10 monclient: shutdown 2024-01-24T23:27:51.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.238+0000 7f917a70f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:51.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.238+0000 7f917a70f640 1 -- 172.21.15.132:0/58969232 >> v1:172.21.15.115:6793/0 conn(0x7f917409ff20 legacy=0x7f91741c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:51.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.238+0000 7f917a70f640 1 -- 172.21.15.132:0/58969232 shutdown_connections 2024-01-24T23:27:51.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.238+0000 7f917a70f640 1 -- 172.21.15.132:0/58969232 >> 172.21.15.132:0/58969232 conn(0x7f91740a5a10 msgr2=0x7f9174066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:51.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.238+0000 7f917a70f640 1 -- 172.21.15.132:0/58969232 shutdown_connections 2024-01-24T23:27:51.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.238+0000 7f917a70f640 1 -- 172.21.15.132:0/58969232 wait complete. 2024-01-24T23:27:51.253 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:51.253 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h10 2024-01-24T23:27:51.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.352+0000 7f9e17885640 10 monclient: get_monmap_and_config 2024-01-24T23:27:51.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.352+0000 7f9e17885640 10 monclient: build_initial_monmap 2024-01-24T23:27:51.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.352+0000 7f9e17885640 10 monclient: monmap: 2024-01-24T23:27:51.353 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:51.353 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:51.354 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:27:51.355105+0000 2024-01-24T23:27:51.354 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:27:51.355105+0000 2024-01-24T23:27:51.354 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:51.354 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:51.354 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:51.354 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:51.354 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:51.354 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:51.354 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:51.354 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:51.354 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:51.354 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:51.354 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:51.354 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.352+0000 7f9e17885640 1 Processor -- start 2024-01-24T23:27:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.352+0000 7f9e17885640 1 -- start start 2024-01-24T23:27:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.352+0000 7f9e17885640 10 monclient: init 2024-01-24T23:27:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.352+0000 7f9e17885640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.352+0000 7f9e17885640 10 monclient: _add_conns ranks=[0,2,7,1,3,4,8,6,5] 2024-01-24T23:27:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.352+0000 7f9e17885640 10 monclient(hunting): picked mon.noname-a con 0x7f9e10136b50 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.352+0000 7f9e17885640 10 monclient(hunting): picked mon.noname-b con 0x7f9e10136f30 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.352+0000 7f9e17885640 10 monclient(hunting): picked mon.noname-i con 0x7f9e1013d090 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.352+0000 7f9e17885640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1004ed70 con 0x7f9e10136b50 2024-01-24T23:27:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.352+0000 7f9e17885640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1006bf60 con 0x7f9e10136f30 2024-01-24T23:27:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.352+0000 7f9e17885640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1009b1e0 con 0x7f9e1013d090 2024-01-24T23:27:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.352+0000 7f9e17885640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.352+0000 7f9e17885640 10 monclient(hunting): authenticate will time out at 1350.277660s 2024-01-24T23:27:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.352+0000 7f9e15621640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f9e10136b50 legacy=0x7f9e1009bd30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:27:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.352+0000 7f9e14e20640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f9e10136f30 legacy=0x7f9e10137370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.352+0000 7f9e15e22640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f9e1013d090 legacy=0x7f9e1013f480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:51.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f9682c89640 10 monclient: get_monmap_and_config 2024-01-24T23:27:51.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f9682c89640 10 monclient: build_initial_monmap 2024-01-24T23:27:51.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f9682c89640 10 monclient: monmap: 2024-01-24T23:27:51.371 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:51.371 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:51.371 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:51.374011+0000 2024-01-24T23:27:51.371 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:51.374011+0000 2024-01-24T23:27:51.371 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:51.371 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:51.372 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:51.372 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:51.372 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:51.372 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:51.372 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:51.372 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:51.372 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:51.372 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:51.372 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:51.372 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:51.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f9682c89640 1 Processor -- start 2024-01-24T23:27:51.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f9682c89640 1 -- start start 2024-01-24T23:27:51.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f9682c89640 10 monclient: init 2024-01-24T23:27:51.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f9682c89640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:51.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f9682c89640 10 monclient: _add_conns ranks=[2,8,0,7,1,6,3,4,5] 2024-01-24T23:27:51.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f9682c89640 10 monclient(hunting): picked mon.noname-b con 0x7f967c09ff20 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:51.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f9682c89640 10 monclient(hunting): picked mon.noname-e con 0x7f967c09f9c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:51.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f9682c89640 10 monclient(hunting): picked mon.noname-a con 0x7f967c13b2c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:51.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f9682c89640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f967c064700 con 0x7f967c13b2c0 2024-01-24T23:27:51.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f9682c89640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f967c09e760 con 0x7f967c09ff20 2024-01-24T23:27:51.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f9682c89640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f967c0a1160 con 0x7f967c09f9c0 2024-01-24T23:27:51.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f9682c89640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:51.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f9682c89640 10 monclient(hunting): authenticate will time out at 1358.698643s 2024-01-24T23:27:51.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f9680a25640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f967c09ff20 legacy=0x7f967c0a0300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:51.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f9681226640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f967c13b2c0 legacy=0x7f967c13d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:27:51.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967bfff640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f967c09f9c0 0x7f967c139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:43584/0 (socket says 172.21.15.132:43584) 2024-01-24T23:27:51.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967bfff640 1 -- 172.21.15.132:0/1836337854 learned_addr learned my addr 172.21.15.132:0/1836337854 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:51.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 1 -- 172.21.15.132:0/1836337854 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2033675048 0 0) 0x7f967c0a1160 con 0x7f967c09f9c0 2024-01-24T23:27:51.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:51.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:51.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:51.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 10 monclient(hunting): my global_id is 85949 2024-01-24T23:27:51.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 1 -- 172.21.15.132:0/1836337854 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9664003530 con 0x7f967c09f9c0 2024-01-24T23:27:51.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 1 -- 172.21.15.132:0/1836337854 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 915781965 0 0) 0x7f9664003530 con 0x7f967c09f9c0 2024-01-24T23:27:51.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 1 -- 172.21.15.132:0/1836337854 >> v1:172.21.15.115:6790/0 conn(0x7f967c09ff20 legacy=0x7f967c0a0300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:51.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 1 -- 172.21.15.132:0/1836337854 >> v1:172.21.15.115:6789/0 conn(0x7f967c13b2c0 legacy=0x7f967c13d6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:51.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:27:51.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 1 monclient: found mon.noname-e 2024-01-24T23:27:51.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:51.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:27:51.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 1 -- 172.21.15.132:0/1836337854 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967c0a0a10 con 0x7f967c09f9c0 2024-01-24T23:27:51.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 10 monclient: _finish_auth 0 2024-01-24T23:27:51.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:51.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:51.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 1 -- 172.21.15.132:0/1836337854 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9670001800 con 0x7f967c09f9c0 2024-01-24T23:27:51.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f9682c89640 5 monclient: authenticate success, global_id 85949 2024-01-24T23:27:51.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f9682c89640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:51.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:51.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:27:51.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 10 monclient: dump: 2024-01-24T23:27:51.381 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:51.381 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:51.381 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.381 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.381 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:51.382 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:51.382 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:51.382 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:51.382 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:51.382 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:51.382 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:51.382 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:51.382 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:51.382 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:51.382 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:51.382 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:51.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f9682c89640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:51.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 1 -- 172.21.15.132:0/1836337854 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9670001ad0 con 0x7f967c09f9c0 2024-01-24T23:27:51.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:51.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f9682c89640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:51.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f9682c89640 4 set_mon_vals no callback set 2024-01-24T23:27:51.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 1 -- 172.21.15.132:0/1836337854 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9670004bf0 con 0x7f967c09f9c0 2024-01-24T23:27:51.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:51.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:27:51.383 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.370+0000 7f967b7fe640 10 monclient: dump: 2024-01-24T23:27:51.383 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:51.383 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:51.383 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.383 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.383 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:51.383 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:51.383 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:51.383 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:51.383 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:51.383 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:51.383 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:51.384 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:51.384 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:51.384 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:51.384 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:51.384 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:51.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 10 monclient: shutdown 2024-01-24T23:27:51.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:51.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 1 -- 172.21.15.132:0/1836337854 >> v1:172.21.15.115:6793/0 conn(0x7f967c09f9c0 legacy=0x7f967c139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:51.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 1 -- 172.21.15.132:0/1836337854 shutdown_connections 2024-01-24T23:27:51.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 1 -- 172.21.15.132:0/1836337854 >> 172.21.15.132:0/1836337854 conn(0x7f967c0a5a10 msgr2=0x7f967c0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:51.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 1 -- 172.21.15.132:0/1836337854 shutdown_connections 2024-01-24T23:27:51.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 1 -- 172.21.15.132:0/1836337854 wait complete. 2024-01-24T23:27:51.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 10 monclient: build_initial_monmap 2024-01-24T23:27:51.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 10 monclient: monmap: 2024-01-24T23:27:51.385 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:51.385 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:51.385 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:51.385 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:51.385 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:51.385 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:51.385 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:51.385 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:51.385 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:51.385 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:51.385 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:51.385 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:51.385 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:51.385 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:51.386 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:51.386 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:51.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 1 Processor -- start 2024-01-24T23:27:51.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 1 -- start start 2024-01-24T23:27:51.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 10 monclient: init 2024-01-24T23:27:51.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:51.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 10 monclient: _add_conns ranks=[4,7,6,3,1,2,0,5,8] 2024-01-24T23:27:51.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 10 monclient(hunting): picked mon.noname-c con 0x7f967c09f9c0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:51.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 10 monclient(hunting): picked mon.noname-i con 0x7f967c09ff20 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:51.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 10 monclient(hunting): picked mon.noname-d con 0x7f967c13b2c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:51.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f967c06e410 con 0x7f967c09f9c0 2024-01-24T23:27:51.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f967c071680 con 0x7f967c13b2c0 2024-01-24T23:27:51.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f967c071960 con 0x7f967c09ff20 2024-01-24T23:27:51.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:51.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 10 monclient(hunting): authenticate will time out at 1358.701727s 2024-01-24T23:27:51.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f967bfff640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f967c09ff20 legacy=0x7f967c1cbc50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:51.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9680a25640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f967c09f9c0 0x7f967c1cb540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:59184/0 (socket says 172.21.15.132:59184) 2024-01-24T23:27:51.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9680a25640 1 -- 172.21.15.132:0/1506675989 learned_addr learned my addr 172.21.15.132:0/1506675989 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:51.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 1 -- 172.21.15.132:0/1506675989 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 280126970 0 0) 0x7f967c06e410 con 0x7f967c09f9c0 2024-01-24T23:27:51.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 10 monclient(hunting): _negotiate 2024-01-24T23:27:51.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:51.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:51.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 10 monclient(hunting): my global_id is 94945 2024-01-24T23:27:51.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 1 -- 172.21.15.132:0/1506675989 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9668003560 con 0x7f967c09f9c0 2024-01-24T23:27:51.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 1 -- 172.21.15.132:0/1506675989 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57140108 0 0) 0x7f967c071680 con 0x7f967c13b2c0 2024-01-24T23:27:51.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 10 monclient(hunting): _negotiate 2024-01-24T23:27:51.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:51.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:51.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 10 monclient(hunting): my global_id is 6990 2024-01-24T23:27:51.387 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 1 -- 172.21.15.132:0/1506675989 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f967c06e410 con 0x7f967c13b2c0 2024-01-24T23:27:51.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 1 -- 172.21.15.132:0/1506675989 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3814603977 0 0) 0x7f9668003560 con 0x7f967c09f9c0 2024-01-24T23:27:51.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 1 -- 172.21.15.132:0/1506675989 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f967c071680 con 0x7f967c09f9c0 2024-01-24T23:27:51.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 1 -- 172.21.15.132:0/1506675989 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f966c004330 con 0x7f967c09f9c0 2024-01-24T23:27:51.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:51.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:27:51.388 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 10 monclient(hunting): dump: 2024-01-24T23:27:51.388 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:51.388 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:51.388 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.388 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.388 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:51.388 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:51.388 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:51.388 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:51.388 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:51.388 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:51.389 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:51.389 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:51.389 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:51.389 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:51.389 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:51.389 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:51.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:51.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 1 -- 172.21.15.132:0/1506675989 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3019089334 0 0) 0x7f967c06e410 con 0x7f967c13b2c0 2024-01-24T23:27:51.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 1 -- 172.21.15.132:0/1506675989 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9668003560 con 0x7f967c13b2c0 2024-01-24T23:27:51.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 1 -- 172.21.15.132:0/1506675989 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f96740043e0 con 0x7f967c13b2c0 2024-01-24T23:27:51.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:51.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:51.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 10 monclient(hunting): dump: 2024-01-24T23:27:51.389 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:51.389 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:51.389 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.390 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.390 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:51.390 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:51.390 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:51.390 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:51.390 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:51.390 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:51.390 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:51.390 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:51.390 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:51.390 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:51.390 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:51.390 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:51.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 1 -- 172.21.15.132:0/1506675989 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 278892631 0 0) 0x7f967c071680 con 0x7f967c09f9c0 2024-01-24T23:27:51.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 1 -- 172.21.15.132:0/1506675989 >> v1:172.21.15.132:6792/0 conn(0x7f967c09ff20 legacy=0x7f967c1cbc50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:51.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 1 -- 172.21.15.132:0/1506675989 >> v1:172.21.15.115:6792/0 conn(0x7f967c13b2c0 legacy=0x7f967c1d3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:51.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 10 monclient: _finish_hunting 0 2024-01-24T23:27:51.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 1 monclient: found mon.c 2024-01-24T23:27:51.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:51.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:51.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 1 -- 172.21.15.132:0/1506675989 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967c06f010 con 0x7f967c09f9c0 2024-01-24T23:27:51.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 10 monclient: _finish_auth 0 2024-01-24T23:27:51.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 10 monclient: _check_auth_tickets 2024-01-24T23:27:51.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:51.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 5 monclient: authenticate success, global_id 94945 2024-01-24T23:27:51.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 10 monclient: _renew_subs 2024-01-24T23:27:51.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:51.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 1 -- 172.21.15.132:0/1506675989 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f967c09db10 con 0x7f967c09f9c0 2024-01-24T23:27:51.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 10 monclient: _renew_subs 2024-01-24T23:27:51.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:51.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 1 -- 172.21.15.132:0/1506675989 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f967c09dd90 con 0x7f967c09f9c0 2024-01-24T23:27:51.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 1 -- 172.21.15.132:0/1506675989 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f966c004610 con 0x7f967c09f9c0 2024-01-24T23:27:51.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:51.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 1 -- 172.21.15.132:0/1506675989 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f966c004b40 con 0x7f967c09f9c0 2024-01-24T23:27:51.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:51.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f967affd640 4 set_mon_vals no callback set 2024-01-24T23:27:51.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:27:51.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 10 monclient: dump: 2024-01-24T23:27:51.392 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:51.392 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:51.392 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.392 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:51.392 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:51.393 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:51.393 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:51.393 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:51.393 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:51.393 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:51.393 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:51.393 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:51.393 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:51.393 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:51.393 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:51.393 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:51.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:51.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:51.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:51.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f9682c89640 1 -- 172.21.15.132:0/1506675989 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f965c005090 con 0x7f967c09f9c0 2024-01-24T23:27:51.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 1 -- 172.21.15.132:0/1506675989 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f966c004d20 con 0x7f967c09f9c0 2024-01-24T23:27:51.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.374+0000 7f96797fa640 1 -- 172.21.15.132:0/1506675989 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(221..221 src has 1..221) v4 ==== 12488+0+0 (unknown 227041204 0 0) 0x7f966c045af0 con 0x7f967c09f9c0 2024-01-24T23:27:51.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.378+0000 7f967bfff640 10 monclient: get_auth_request con 0x7f9668037630 auth_method 0 2024-01-24T23:27:51.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.382+0000 7f96797fa640 1 -- 172.21.15.132:0/1506675989 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f966c047e60 con 0x7f967c09f9c0 2024-01-24T23:27:51.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.382+0000 7f96797fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:51.394 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.382+0000 7f96797fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:51.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.542+0000 7f9682c89640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h10"}] 2024-01-24T23:27:51.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.542+0000 7f9682c89640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h10"}] 2024-01-24T23:27:51.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.542+0000 7f9682c89640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:51.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:51.542+0000 7f9682c89640 1 -- 172.21.15.132:0/1506675989 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h10"} v 0) v1 -- 0x7f965c005340 con 0x7f967c09f9c0 2024-01-24T23:27:51.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.556+0000 7f9e15621640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f9e10136b50 legacy=0x7f9e1009bd30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:27:51.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.556+0000 7f9e14e20640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f9e10136f30 legacy=0x7f9e10137370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:51.557 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.556+0000 7f9e15e22640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f9e1013d090 legacy=0x7f9e1013f480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:51.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.960+0000 7f9e15621640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f9e10136b50 legacy=0x7f9e1009bd30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:27:51.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.960+0000 7f9e14e20640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f9e10136f30 legacy=0x7f9e10137370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:51.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:51.960+0000 7f9e15e22640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f9e1013d090 legacy=0x7f9e1013f480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.150+0000 7f96797fa640 1 -- 172.21.15.132:0/1506675989 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h10"}]=0 removed item id -46 name 'h10' from crush map v222) v1 ==== 120+0+0 (unknown 476927875 0 0) 0x7f966c014b50 con 0x7f967c09f9c0 2024-01-24T23:27:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.150+0000 7f96797fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h10"}] 2024-01-24T23:27:52.152 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.150+0000 7f96797fa640 10 monclient: _finish_command 2 = system:0 removed item id -46 name 'h10' from crush map 2024-01-24T23:27:52.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.258+0000 7f9682c89640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h10"}] 2024-01-24T23:27:52.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.258+0000 7f9682c89640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h10"}] 2024-01-24T23:27:52.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.262+0000 7f9682c89640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:27:52.262 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.262+0000 7f9682c89640 1 -- 172.21.15.132:0/1506675989 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h10"} v 0) v1 -- 0x7f965c002220 con 0x7f967c09f9c0 2024-01-24T23:27:52.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.262+0000 7f96797fa640 1 -- 172.21.15.132:0/1506675989 <== mon.4 v1:172.21.15.115:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h10"}]=0 device 'h10' does not appear in the crush map v222) v1 ==== 120+0+0 (unknown 3861204250 0 0) 0x7f966c012fc0 con 0x7f967c09f9c0 2024-01-24T23:27:52.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.262+0000 7f96797fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h10"}] 2024-01-24T23:27:52.264 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.262+0000 7f96797fa640 10 monclient: _finish_command 3 = system:0 device 'h10' does not appear in the crush map 2024-01-24T23:27:52.264 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h10' does not appear in the crush map 2024-01-24T23:27:52.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.266+0000 7f9682c89640 1 -- 172.21.15.132:0/1506675989 >> v1:172.21.15.132:6812/1835401922 conn(0x7f9668037630 legacy=0x7f9668039af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:52.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.266+0000 7f9682c89640 10 monclient: shutdown 2024-01-24T23:27:52.266 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.266+0000 7f9682c89640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:52.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.266+0000 7f9682c89640 1 -- 172.21.15.132:0/1506675989 >> v1:172.21.15.115:6791/0 conn(0x7f967c09f9c0 legacy=0x7f967c1cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:52.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.266+0000 7f9682c89640 1 -- 172.21.15.132:0/1506675989 shutdown_connections 2024-01-24T23:27:52.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.266+0000 7f9682c89640 1 -- 172.21.15.132:0/1506675989 >> 172.21.15.132:0/1506675989 conn(0x7f967c0a5a10 msgr2=0x7f967c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:52.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.266+0000 7f9682c89640 1 -- 172.21.15.132:0/1506675989 shutdown_connections 2024-01-24T23:27:52.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.266+0000 7f9682c89640 1 -- 172.21.15.132:0/1506675989 wait complete. 2024-01-24T23:27:52.281 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:52.281 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h11 2024-01-24T23:27:52.399 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.398+0000 7f26435d0640 10 monclient: get_monmap_and_config 2024-01-24T23:27:52.399 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.398+0000 7f26435d0640 10 monclient: build_initial_monmap 2024-01-24T23:27:52.399 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.398+0000 7f26435d0640 10 monclient: monmap: 2024-01-24T23:27:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:52.402228+0000 2024-01-24T23:27:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:52.402228+0000 2024-01-24T23:27:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:52.400 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:52.401 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:52.401 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.398+0000 7f26435d0640 1 Processor -- start 2024-01-24T23:27:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.398+0000 7f26435d0640 1 -- start start 2024-01-24T23:27:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.398+0000 7f26435d0640 10 monclient: init 2024-01-24T23:27:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.398+0000 7f26435d0640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.398+0000 7f26435d0640 10 monclient: _add_conns ranks=[7,1,3,6,5,4,0,8,2] 2024-01-24T23:27:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.398+0000 7f26435d0640 10 monclient(hunting): picked mon.noname-i con 0x7f263c135e10 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.398+0000 7f26435d0640 10 monclient(hunting): picked mon.noname-f con 0x7f263c1361f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.398+0000 7f26435d0640 10 monclient(hunting): picked mon.noname-g con 0x7f263c1365f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.398+0000 7f26435d0640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f263c064700 con 0x7f263c1361f0 2024-01-24T23:27:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.398+0000 7f26435d0640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f263c09c3e0 con 0x7f263c1365f0 2024-01-24T23:27:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.398+0000 7f26435d0640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f263c0a4f90 con 0x7f263c135e10 2024-01-24T23:27:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.398+0000 7f26435d0640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:52.407 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.398+0000 7f26435d0640 10 monclient(hunting): authenticate will time out at 1359.727640s 2024-01-24T23:27:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.398+0000 7f264136c640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f263c135e10 legacy=0x7f263c09bcd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.398+0000 7f2641b6d640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f263c1365f0 0x7f263c13f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:43994/0 (socket says 172.21.15.132:43994) 2024-01-24T23:27:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.398+0000 7f2641b6d640 1 -- 172.21.15.132:0/3345260400 learned_addr learned my addr 172.21.15.132:0/3345260400 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 1 -- 172.21.15.132:0/3345260400 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 701631382 0 0) 0x7f263c064700 con 0x7f263c1361f0 2024-01-24T23:27:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 10 monclient(hunting): my global_id is 124363 2024-01-24T23:27:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 1 -- 172.21.15.132:0/3345260400 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2628003530 con 0x7f263c1361f0 2024-01-24T23:27:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 1 -- 172.21.15.132:0/3345260400 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4150689672 0 0) 0x7f263c09c3e0 con 0x7f263c1365f0 2024-01-24T23:27:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:52.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 10 monclient(hunting): my global_id is 6762 2024-01-24T23:27:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 1 -- 172.21.15.132:0/3345260400 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f263c064700 con 0x7f263c1365f0 2024-01-24T23:27:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 1 -- 172.21.15.132:0/3345260400 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 709751637 0 0) 0x7f2628003530 con 0x7f263c1361f0 2024-01-24T23:27:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 1 -- 172.21.15.132:0/3345260400 >> v1:172.21.15.132:6792/0 conn(0x7f263c135e10 legacy=0x7f263c09bcd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 1 -- 172.21.15.132:0/3345260400 >> v1:172.21.15.132:6790/0 conn(0x7f263c1365f0 legacy=0x7f263c13f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 10 monclient: _finish_hunting 0 2024-01-24T23:27:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 1 monclient: found mon.noname-f 2024-01-24T23:27:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:27:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 1 -- 172.21.15.132:0/3345260400 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263c09b010 con 0x7f263c1361f0 2024-01-24T23:27:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 10 monclient: _finish_auth 0 2024-01-24T23:27:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 10 monclient: _check_auth_tickets 2024-01-24T23:27:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 1 -- 172.21.15.132:0/3345260400 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f26380017c0 con 0x7f263c1361f0 2024-01-24T23:27:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 5 monclient: authenticate success, global_id 124363 2024-01-24T23:27:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:52.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:27:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 10 monclient: dump: 2024-01-24T23:27:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:52.410 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:52.411 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 1 -- 172.21.15.132:0/3345260400 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f26380019e0 con 0x7f263c1361f0 2024-01-24T23:27:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 4 set_mon_vals no callback set 2024-01-24T23:27:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 1 -- 172.21.15.132:0/3345260400 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2638003960 con 0x7f263c1361f0 2024-01-24T23:27:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:27:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2633fff640 10 monclient: dump: 2024-01-24T23:27:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:52.411 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:52.412 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 10 monclient: shutdown 2024-01-24T23:27:52.412 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 1 -- 172.21.15.132:0/3345260400 >> v1:172.21.15.132:6789/0 conn(0x7f263c1361f0 legacy=0x7f263c13bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 1 -- 172.21.15.132:0/3345260400 shutdown_connections 2024-01-24T23:27:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 1 -- 172.21.15.132:0/3345260400 >> 172.21.15.132:0/3345260400 conn(0x7f263c0a6790 msgr2=0x7f263c071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 1 -- 172.21.15.132:0/3345260400 shutdown_connections 2024-01-24T23:27:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 1 -- 172.21.15.132:0/3345260400 wait complete. 2024-01-24T23:27:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 10 monclient: build_initial_monmap 2024-01-24T23:27:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 10 monclient: monmap: 2024-01-24T23:27:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:52.413 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:52.414 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 1 Processor -- start 2024-01-24T23:27:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 1 -- start start 2024-01-24T23:27:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 10 monclient: init 2024-01-24T23:27:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 10 monclient: _add_conns ranks=[3,6,4,8,5,7,1,2,0] 2024-01-24T23:27:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 10 monclient(hunting): picked mon.noname-g con 0x7f263c135e10 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 10 monclient(hunting): picked mon.noname-d con 0x7f263c1361f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:52.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 10 monclient(hunting): picked mon.noname-c con 0x7f263c1365f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f263c1221c0 con 0x7f263c135e10 2024-01-24T23:27:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f264136c640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f263c135e10 0x7f263c134dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:44002/0 (socket says 172.21.15.132:44002) 2024-01-24T23:27:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f264136c640 1 -- 172.21.15.132:0/287819632 learned_addr learned my addr 172.21.15.132:0/287819632 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 1 -- 172.21.15.132:0/287819632 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f263c09c720 con 0x7f263c1365f0 2024-01-24T23:27:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 1 -- 172.21.15.132:0/287819632 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f263c1bf3c0 con 0x7f263c1361f0 2024-01-24T23:27:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 10 monclient(hunting): authenticate will time out at 1359.731081s 2024-01-24T23:27:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 1 -- 172.21.15.132:0/287819632 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2074323153 0 0) 0x7f263c1221c0 con 0x7f263c135e10 2024-01-24T23:27:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 10 monclient(hunting): my global_id is 6771 2024-01-24T23:27:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 1 -- 172.21.15.132:0/287819632 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2620003530 con 0x7f263c135e10 2024-01-24T23:27:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 1 -- 172.21.15.132:0/287819632 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2312772610 0 0) 0x7f2620003530 con 0x7f263c135e10 2024-01-24T23:27:52.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 1 -- 172.21.15.132:0/287819632 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f263c1221c0 con 0x7f263c135e10 2024-01-24T23:27:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 1 -- 172.21.15.132:0/287819632 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f262c004300 con 0x7f263c135e10 2024-01-24T23:27:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:27:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 10 monclient(hunting): dump: 2024-01-24T23:27:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:52.416 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:52.417 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 1 -- 172.21.15.132:0/287819632 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1821730077 0 0) 0x7f263c1221c0 con 0x7f263c135e10 2024-01-24T23:27:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 1 -- 172.21.15.132:0/287819632 >> v1:172.21.15.115:6792/0 conn(0x7f263c1361f0 legacy=0x7f263c1d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 1 -- 172.21.15.132:0/287819632 >> v1:172.21.15.115:6791/0 conn(0x7f263c1365f0 legacy=0x7f263c1d3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:27:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 1 monclient: found mon.g 2024-01-24T23:27:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:52.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 1 -- 172.21.15.132:0/287819632 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263c121ee0 con 0x7f263c135e10 2024-01-24T23:27:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 10 monclient: _finish_auth 0 2024-01-24T23:27:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:27:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 5 monclient: authenticate success, global_id 6771 2024-01-24T23:27:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 10 monclient: _renew_subs 2024-01-24T23:27:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 1 -- 172.21.15.132:0/287819632 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f263c120f80 con 0x7f263c135e10 2024-01-24T23:27:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 10 monclient: _renew_subs 2024-01-24T23:27:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26435d0640 1 -- 172.21.15.132:0/287819632 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f263c11fe70 con 0x7f263c135e10 2024-01-24T23:27:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 1 -- 172.21.15.132:0/287819632 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f262c004660 con 0x7f263c135e10 2024-01-24T23:27:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 1 -- 172.21.15.132:0/287819632 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f262c004b90 con 0x7f263c135e10 2024-01-24T23:27:52.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f26337fe640 4 set_mon_vals no callback set 2024-01-24T23:27:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.402+0000 7f2631ffb640 10 monclient: dump: 2024-01-24T23:27:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:52.419 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:52.420 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.406+0000 7f26435d0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.406+0000 7f26435d0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.406+0000 7f26435d0640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.406+0000 7f26435d0640 1 -- 172.21.15.132:0/287819632 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f261c005090 con 0x7f263c135e10 2024-01-24T23:27:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.406+0000 7f2631ffb640 1 -- 172.21.15.132:0/287819632 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f262c004d70 con 0x7f263c135e10 2024-01-24T23:27:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.406+0000 7f2631ffb640 1 -- 172.21.15.132:0/287819632 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(222..222 src has 1..222) v4 ==== 12275+0+0 (unknown 3964722052 0 0) 0x7f262c045ad0 con 0x7f263c135e10 2024-01-24T23:27:52.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.406+0000 7f2640b6b640 10 monclient: get_auth_request con 0x7f2620036fd0 auth_method 0 2024-01-24T23:27:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.410+0000 7f2631ffb640 1 -- 172.21.15.132:0/287819632 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f262c0055c0 con 0x7f263c135e10 2024-01-24T23:27:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.410+0000 7f2631ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:52.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.410+0000 7f2631ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:52.571 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.570+0000 7f26435d0640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h11"}] 2024-01-24T23:27:52.571 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.570+0000 7f26435d0640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h11"}] 2024-01-24T23:27:52.571 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.570+0000 7f26435d0640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:52.571 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:52.570+0000 7f26435d0640 1 -- 172.21.15.132:0/287819632 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rm", "name": "h11"} v 0) v1 -- 0x7f261c005340 con 0x7f263c135e10 2024-01-24T23:27:52.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:52.764+0000 7f9e15621640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f9e10136b50 legacy=0x7f9e1009bd30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:27:52.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:52.764+0000 7f9e14e20640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f9e10136f30 legacy=0x7f9e10137370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:52.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:52.764+0000 7f9e15e22640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f9e1013d090 legacy=0x7f9e1013f480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:53.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.166+0000 7f2631ffb640 1 -- 172.21.15.132:0/287819632 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h11"}]=0 removed item id -49 name 'h11' from crush map v223) v1 ==== 120+0+0 (unknown 1230033723 0 0) 0x7f262c013070 con 0x7f263c135e10 2024-01-24T23:27:53.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.166+0000 7f2631ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h11"}] 2024-01-24T23:27:53.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.166+0000 7f2631ffb640 10 monclient: _finish_command 2 = system:0 removed item id -49 name 'h11' from crush map 2024-01-24T23:27:53.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.266+0000 7f26435d0640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h11"}] 2024-01-24T23:27:53.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.266+0000 7f26435d0640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h11"}] 2024-01-24T23:27:53.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.266+0000 7f26435d0640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:53.269 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.266+0000 7f26435d0640 1 -- 172.21.15.132:0/287819632 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rm", "name": "h11"} v 0) v1 -- 0x7f261c002220 con 0x7f263c135e10 2024-01-24T23:27:53.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.270+0000 7f2631ffb640 1 -- 172.21.15.132:0/287819632 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h11"}]=0 device 'h11' does not appear in the crush map v223) v1 ==== 120+0+0 (unknown 1377658293 0 0) 0x7f262c0147f0 con 0x7f263c135e10 2024-01-24T23:27:53.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.270+0000 7f2631ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h11"}] 2024-01-24T23:27:53.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.270+0000 7f2631ffb640 10 monclient: _finish_command 3 = system:0 device 'h11' does not appear in the crush map 2024-01-24T23:27:53.272 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h11' does not appear in the crush map 2024-01-24T23:27:53.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.270+0000 7f26435d0640 1 -- 172.21.15.132:0/287819632 >> v1:172.21.15.132:6812/1835401922 conn(0x7f2620036fd0 legacy=0x7f2620039490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:53.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.270+0000 7f26435d0640 10 monclient: shutdown 2024-01-24T23:27:53.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.270+0000 7f26435d0640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:53.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.270+0000 7f26435d0640 1 -- 172.21.15.132:0/287819632 >> v1:172.21.15.132:6790/0 conn(0x7f263c135e10 legacy=0x7f263c134dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:53.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.270+0000 7f26435d0640 1 -- 172.21.15.132:0/287819632 shutdown_connections 2024-01-24T23:27:53.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.270+0000 7f26435d0640 1 -- 172.21.15.132:0/287819632 >> 172.21.15.132:0/287819632 conn(0x7f263c0a6790 msgr2=0x7f263c06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:53.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.270+0000 7f26435d0640 1 -- 172.21.15.132:0/287819632 shutdown_connections 2024-01-24T23:27:53.274 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.270+0000 7f26435d0640 1 -- 172.21.15.132:0/287819632 wait complete. 2024-01-24T23:27:53.288 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:53.288 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h12 2024-01-24T23:27:53.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.406+0000 7f9fa0178640 10 monclient: get_monmap_and_config 2024-01-24T23:27:53.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.406+0000 7f9fa0178640 10 monclient: build_initial_monmap 2024-01-24T23:27:53.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.406+0000 7f9fa0178640 10 monclient: monmap: 2024-01-24T23:27:53.409 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:53.409 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:53.409 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:53.411838+0000 2024-01-24T23:27:53.409 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:53.411838+0000 2024-01-24T23:27:53.409 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:53.409 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:53.409 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:53.410 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:53.410 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:53.410 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:53.410 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:53.410 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:53.410 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:53.410 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:53.410 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:53.410 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:53.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 1 Processor -- start 2024-01-24T23:27:53.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 1 -- start start 2024-01-24T23:27:53.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 10 monclient: init 2024-01-24T23:27:53.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:53.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 10 monclient: _add_conns ranks=[8,4,2,7,6,0,1,3,5] 2024-01-24T23:27:53.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 10 monclient(hunting): picked mon.noname-e con 0x7f9f980714f0 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:53.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 10 monclient(hunting): picked mon.noname-c con 0x7f9f981339d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:53.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 10 monclient(hunting): picked mon.noname-b con 0x7f9f9813d4f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:53.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f98064700 con 0x7f9f9813d4f0 2024-01-24T23:27:53.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f9806dcc0 con 0x7f9f981339d0 2024-01-24T23:27:53.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f9806d9e0 con 0x7f9f980714f0 2024-01-24T23:27:53.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:53.414 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 10 monclient(hunting): authenticate will time out at 1360.737270s 2024-01-24T23:27:53.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9e715640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f9f9813d4f0 legacy=0x7f9f9813f8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:53.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9d713640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f9f981339d0 0x7f9f9813bdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:59204/0 (socket says 172.21.15.132:59204) 2024-01-24T23:27:53.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9df14640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f9f980714f0 0x7f9f980718d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:43600/0 (socket says 172.21.15.132:43600) 2024-01-24T23:27:53.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9d713640 1 -- 172.21.15.132:0/3388842507 learned_addr learned my addr 172.21.15.132:0/3388842507 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:53.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 1 -- 172.21.15.132:0/3388842507 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3286305423 0 0) 0x7f9f9806d9e0 con 0x7f9f980714f0 2024-01-24T23:27:53.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 10 monclient(hunting): _negotiate 2024-01-24T23:27:53.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:53.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:53.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 10 monclient(hunting): my global_id is 85958 2024-01-24T23:27:53.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 1 -- 172.21.15.132:0/3388842507 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f84003530 con 0x7f9f980714f0 2024-01-24T23:27:53.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 1 -- 172.21.15.132:0/3388842507 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1003541614 0 0) 0x7f9f9806dcc0 con 0x7f9f981339d0 2024-01-24T23:27:53.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 10 monclient(hunting): _negotiate 2024-01-24T23:27:53.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:53.415 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:53.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 10 monclient(hunting): my global_id is 94963 2024-01-24T23:27:53.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 1 -- 172.21.15.132:0/3388842507 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f9806d9e0 con 0x7f9f981339d0 2024-01-24T23:27:53.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 1 -- 172.21.15.132:0/3388842507 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2272714955 0 0) 0x7f9f84003530 con 0x7f9f980714f0 2024-01-24T23:27:53.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 1 -- 172.21.15.132:0/3388842507 >> v1:172.21.15.115:6791/0 conn(0x7f9f981339d0 legacy=0x7f9f9813bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:53.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 1 -- 172.21.15.132:0/3388842507 >> v1:172.21.15.115:6790/0 conn(0x7f9f9813d4f0 legacy=0x7f9f9813f8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:53.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 10 monclient: _finish_hunting 0 2024-01-24T23:27:53.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 1 monclient: found mon.noname-e 2024-01-24T23:27:53.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:53.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:27:53.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 1 -- 172.21.15.132:0/3388842507 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9806dfa0 con 0x7f9f980714f0 2024-01-24T23:27:53.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 10 monclient: _finish_auth 0 2024-01-24T23:27:53.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 10 monclient: _check_auth_tickets 2024-01-24T23:27:53.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:53.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 5 monclient: authenticate success, global_id 85958 2024-01-24T23:27:53.416 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 1 -- 172.21.15.132:0/3388842507 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9f940016f0 con 0x7f9f980714f0 2024-01-24T23:27:53.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:53.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:53.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:27:53.417 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 10 monclient: dump: 2024-01-24T23:27:53.417 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:53.417 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:53.417 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:53.417 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:53.417 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:53.417 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:53.417 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:53.417 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:53.417 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:53.417 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:53.417 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:53.418 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:53.418 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:53.418 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:53.418 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:53.418 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:53.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:53.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 1 -- 172.21.15.132:0/3388842507 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9f940019d0 con 0x7f9f980714f0 2024-01-24T23:27:53.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:53.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:53.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 1 -- 172.21.15.132:0/3388842507 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9f94004b30 con 0x7f9f980714f0 2024-01-24T23:27:53.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 4 set_mon_vals no callback set 2024-01-24T23:27:53.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:53.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:27:53.418 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9f9cf12640 10 monclient: dump: 2024-01-24T23:27:53.418 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:53.418 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:53.419 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:53.419 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:53.419 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:53.419 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:53.419 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:53.419 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:53.419 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:53.419 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:53.419 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:53.419 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:53.419 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:53.419 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:53.419 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:53.419 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:53.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 10 monclient: shutdown 2024-01-24T23:27:53.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:53.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 1 -- 172.21.15.132:0/3388842507 >> v1:172.21.15.115:6793/0 conn(0x7f9f980714f0 legacy=0x7f9f980718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:53.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 1 -- 172.21.15.132:0/3388842507 shutdown_connections 2024-01-24T23:27:53.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 1 -- 172.21.15.132:0/3388842507 >> 172.21.15.132:0/3388842507 conn(0x7f9f980a22a0 msgr2=0x7f9f980a2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:53.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 1 -- 172.21.15.132:0/3388842507 shutdown_connections 2024-01-24T23:27:53.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.410+0000 7f9fa0178640 1 -- 172.21.15.132:0/3388842507 wait complete. 2024-01-24T23:27:53.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 10 monclient: build_initial_monmap 2024-01-24T23:27:53.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 10 monclient: monmap: 2024-01-24T23:27:53.420 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:53.420 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:53.420 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:53.420 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:53.420 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:53.421 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:53.421 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:53.421 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:53.421 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:53.421 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:53.421 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:53.421 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:53.421 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:53.421 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:53.421 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:53.421 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:53.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 1 Processor -- start 2024-01-24T23:27:53.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 1 -- start start 2024-01-24T23:27:53.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 10 monclient: init 2024-01-24T23:27:53.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:53.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 10 monclient: _add_conns ranks=[3,1,7,6,2,8,5,0,4] 2024-01-24T23:27:53.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 10 monclient(hunting): picked mon.noname-g con 0x7f9f980714f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:53.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 10 monclient(hunting): picked mon.noname-f con 0x7f9f981339d0 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:53.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 10 monclient(hunting): picked mon.noname-i con 0x7f9f9813d4f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:53.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f98135300 con 0x7f9f981339d0 2024-01-24T23:27:53.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f981355e0 con 0x7f9f980714f0 2024-01-24T23:27:53.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f9813c610 con 0x7f9f9813d4f0 2024-01-24T23:27:53.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:53.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 10 monclient(hunting): authenticate will time out at 1360.741383s 2024-01-24T23:27:53.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f9e715640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f9f9813d4f0 legacy=0x7f9f981d36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:53.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f9df14640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f9f980714f0 0x7f9f9806ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:44010/0 (socket says 172.21.15.132:44010) 2024-01-24T23:27:53.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f9df14640 1 -- 172.21.15.132:0/2224533885 learned_addr learned my addr 172.21.15.132:0/2224533885 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:53.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 1 -- 172.21.15.132:0/2224533885 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3755018507 0 0) 0x7f9f981355e0 con 0x7f9f980714f0 2024-01-24T23:27:53.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:53.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:53.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:53.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 10 monclient(hunting): my global_id is 6780 2024-01-24T23:27:53.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 1 -- 172.21.15.132:0/2224533885 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f7c003560 con 0x7f9f980714f0 2024-01-24T23:27:53.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 1 -- 172.21.15.132:0/2224533885 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2043547836 0 0) 0x7f9f98135300 con 0x7f9f981339d0 2024-01-24T23:27:53.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:53.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:53.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:53.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 10 monclient(hunting): my global_id is 124372 2024-01-24T23:27:53.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 1 -- 172.21.15.132:0/2224533885 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f981355e0 con 0x7f9f981339d0 2024-01-24T23:27:53.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 1 -- 172.21.15.132:0/2224533885 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2365177420 0 0) 0x7f9f981355e0 con 0x7f9f981339d0 2024-01-24T23:27:53.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 1 -- 172.21.15.132:0/2224533885 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9f98135300 con 0x7f9f981339d0 2024-01-24T23:27:53.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 1 -- 172.21.15.132:0/2224533885 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4207649055 0 0) 0x7f9f7c003560 con 0x7f9f980714f0 2024-01-24T23:27:53.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 1 -- 172.21.15.132:0/2224533885 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9f981355e0 con 0x7f9f980714f0 2024-01-24T23:27:53.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 1 -- 172.21.15.132:0/2224533885 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9f88001a60 con 0x7f9f981339d0 2024-01-24T23:27:53.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:53.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:27:53.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 10 monclient(hunting): dump: 2024-01-24T23:27:53.428 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:53.428 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:53.428 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:53.428 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:53.428 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:53.428 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:53.428 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:53.428 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:53.428 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:53.428 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:53.429 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:53.429 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:53.429 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:53.429 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:53.429 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:53.429 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:53.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:53.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 1 -- 172.21.15.132:0/2224533885 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9f94001370 con 0x7f9f980714f0 2024-01-24T23:27:53.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:53.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:53.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 10 monclient(hunting): dump: 2024-01-24T23:27:53.429 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:53.429 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:53.429 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:53.430 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:53.430 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:53.430 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:53.430 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:53.430 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:53.430 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:53.430 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:53.430 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:53.430 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:53.430 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:53.430 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:53.430 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:53.430 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:53.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 1 -- 172.21.15.132:0/2224533885 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1321045349 0 0) 0x7f9f98135300 con 0x7f9f981339d0 2024-01-24T23:27:53.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 1 -- 172.21.15.132:0/2224533885 >> v1:172.21.15.132:6792/0 conn(0x7f9f9813d4f0 legacy=0x7f9f981d36c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:53.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 1 -- 172.21.15.132:0/2224533885 >> v1:172.21.15.132:6790/0 conn(0x7f9f980714f0 legacy=0x7f9f9806ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:53.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:27:53.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 1 monclient: found mon.f 2024-01-24T23:27:53.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:53.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:27:53.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 1 -- 172.21.15.132:0/2224533885 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f98131100 con 0x7f9f981339d0 2024-01-24T23:27:53.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 10 monclient: _finish_auth 0 2024-01-24T23:27:53.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:27:53.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:53.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 5 monclient: authenticate success, global_id 124372 2024-01-24T23:27:53.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 10 monclient: _renew_subs 2024-01-24T23:27:53.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:27:53.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 1 -- 172.21.15.132:0/2224533885 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9806fd70 con 0x7f9f981339d0 2024-01-24T23:27:53.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 10 monclient: _renew_subs 2024-01-24T23:27:53.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:27:53.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 1 -- 172.21.15.132:0/2224533885 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f981d4de0 con 0x7f9f981339d0 2024-01-24T23:27:53.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 1 -- 172.21.15.132:0/2224533885 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9f88001ce0 con 0x7f9f981339d0 2024-01-24T23:27:53.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:53.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 1 -- 172.21.15.132:0/2224533885 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9f88004de0 con 0x7f9f981339d0 2024-01-24T23:27:53.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:53.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8ffff640 4 set_mon_vals no callback set 2024-01-24T23:27:53.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:27:53.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 10 monclient: dump: 2024-01-24T23:27:53.432 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:53.432 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:53.432 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:53.432 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:53.432 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:53.433 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:53.433 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:53.433 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:53.433 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:53.433 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:53.433 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:53.433 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:53.433 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:53.433 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:53.433 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:53.433 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:53.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 1 -- 172.21.15.132:0/2224533885 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f9f8800f860 con 0x7f9f981339d0 2024-01-24T23:27:53.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:53.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:53.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:27:53.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9fa0178640 1 -- 172.21.15.132:0/2224533885 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f80005090 con 0x7f9f981339d0 2024-01-24T23:27:53.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f8e7fc640 1 -- 172.21.15.132:0/2224533885 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(223..223 src has 1..223) v4 ==== 12062+0+0 (unknown 875684530 0 0) 0x7f9f88045720 con 0x7f9f981339d0 2024-01-24T23:27:53.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.414+0000 7f9f9df14640 10 monclient: get_auth_request con 0x7f9f7c037800 auth_method 0 2024-01-24T23:27:53.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.422+0000 7f9f8e7fc640 1 -- 172.21.15.132:0/2224533885 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f9f880146f0 con 0x7f9f981339d0 2024-01-24T23:27:53.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.422+0000 7f9f8e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:53.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.422+0000 7f9f8e7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:53.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.582+0000 7f9fa0178640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h12"}] 2024-01-24T23:27:53.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.582+0000 7f9fa0178640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h12"}] 2024-01-24T23:27:53.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.582+0000 7f9fa0178640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:27:53.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:53.582+0000 7f9fa0178640 1 -- 172.21.15.132:0/2224533885 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd crush rm", "name": "h12"} v 0) v1 -- 0x7f9f80005340 con 0x7f9f981339d0 2024-01-24T23:27:54.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.178+0000 7f9f8e7fc640 1 -- 172.21.15.132:0/2224533885 <== mon.1 v1:172.21.15.132:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h12"}]=0 removed item id -52 name 'h12' from crush map v224) v1 ==== 120+0+0 (unknown 1573842943 0 0) 0x7f9f88012d90 con 0x7f9f981339d0 2024-01-24T23:27:54.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.178+0000 7f9f8e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h12"}] 2024-01-24T23:27:54.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.178+0000 7f9f8e7fc640 10 monclient: _finish_command 2 = system:0 removed item id -52 name 'h12' from crush map 2024-01-24T23:27:54.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.278+0000 7f9fa0178640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h12"}] 2024-01-24T23:27:54.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.278+0000 7f9fa0178640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h12"}] 2024-01-24T23:27:54.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.278+0000 7f9fa0178640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:27:54.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.278+0000 7f9fa0178640 1 -- 172.21.15.132:0/2224533885 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd crush rm", "name": "h12"} v 0) v1 -- 0x7f9f80002220 con 0x7f9f981339d0 2024-01-24T23:27:54.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.278+0000 7f9f8e7fc640 1 -- 172.21.15.132:0/2224533885 <== mon.1 v1:172.21.15.132:6789/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h12"}]=0 device 'h12' does not appear in the crush map v224) v1 ==== 120+0+0 (unknown 2813952508 0 0) 0x7f9f88014510 con 0x7f9f981339d0 2024-01-24T23:27:54.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.278+0000 7f9f8e7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h12"}] 2024-01-24T23:27:54.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.278+0000 7f9f8e7fc640 10 monclient: _finish_command 3 = system:0 device 'h12' does not appear in the crush map 2024-01-24T23:27:54.285 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h12' does not appear in the crush map 2024-01-24T23:27:54.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.282+0000 7f9fa0178640 1 -- 172.21.15.132:0/2224533885 >> v1:172.21.15.132:6812/1835401922 conn(0x7f9f7c037800 legacy=0x7f9f7c039cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:54.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.282+0000 7f9fa0178640 10 monclient: shutdown 2024-01-24T23:27:54.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.282+0000 7f9fa0178640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:54.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.282+0000 7f9fa0178640 1 -- 172.21.15.132:0/2224533885 >> v1:172.21.15.132:6789/0 conn(0x7f9f981339d0 legacy=0x7f9f9806f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:54.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.282+0000 7f9fa0178640 1 -- 172.21.15.132:0/2224533885 shutdown_connections 2024-01-24T23:27:54.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.282+0000 7f9fa0178640 1 -- 172.21.15.132:0/2224533885 >> 172.21.15.132:0/2224533885 conn(0x7f9f980a22a0 msgr2=0x7f9f980a1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:54.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.282+0000 7f9fa0178640 1 -- 172.21.15.132:0/2224533885 shutdown_connections 2024-01-24T23:27:54.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.282+0000 7f9fa0178640 1 -- 172.21.15.132:0/2224533885 wait complete. 2024-01-24T23:27:54.297 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:54.297 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h13 2024-01-24T23:27:54.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e02ffd640 10 monclient(hunting): tick 2024-01-24T23:27:54.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e02ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:27:54.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e02ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:27:54.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e02ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:27:54.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e02ffd640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f9e1013d090 legacy=0x7f9e1013f480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:54.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e02ffd640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f9e10136f30 legacy=0x7f9e10137370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:54.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e02ffd640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f9e10136b50 legacy=0x7f9e1009bd30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:54.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e02ffd640 10 monclient: _add_conns ranks=[3,1,4,8,6,5] 2024-01-24T23:27:54.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e02ffd640 10 monclient(hunting): picked mon.noname-g con 0x7f9df4003790 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:54.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e02ffd640 10 monclient(hunting): picked mon.noname-f con 0x7f9df4007420 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:54.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e02ffd640 10 monclient(hunting): picked mon.noname-c con 0x7f9df400b1b0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:54.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e02ffd640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1009af00 con 0x7f9df4007420 2024-01-24T23:27:54.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e02ffd640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1004ed70 con 0x7f9df4003790 2024-01-24T23:27:54.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e02ffd640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1006bf60 con 0x7f9df400b1b0 2024-01-24T23:27:54.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e02ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:54.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e15e22640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f9df400b1b0 0x7f9df400d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:53966/0 (socket says 172.21.15.115:53966) 2024-01-24T23:27:54.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e15e22640 1 -- 172.21.15.115:0/221602374 learned_addr learned my addr 172.21.15.115:0/221602374 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:54.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 1 -- 172.21.15.115:0/221602374 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3553601796 0 0) 0x7f9e1009af00 con 0x7f9df4007420 2024-01-24T23:27:54.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:54.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:54.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:54.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 10 monclient(hunting): my global_id is 124381 2024-01-24T23:27:54.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 1 -- 172.21.15.115:0/221602374 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dfc003530 con 0x7f9df4007420 2024-01-24T23:27:54.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 1 -- 172.21.15.115:0/221602374 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 912755146 0 0) 0x7f9e1006bf60 con 0x7f9df400b1b0 2024-01-24T23:27:54.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:54.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:54.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:54.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 10 monclient(hunting): my global_id is 94972 2024-01-24T23:27:54.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 1 -- 172.21.15.115:0/221602374 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e1009af00 con 0x7f9df400b1b0 2024-01-24T23:27:54.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 1 -- 172.21.15.115:0/221602374 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2733666607 0 0) 0x7f9e1004ed70 con 0x7f9df4003790 2024-01-24T23:27:54.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:54.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:54.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:54.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 10 monclient(hunting): my global_id is 6789 2024-01-24T23:27:54.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 1 -- 172.21.15.115:0/221602374 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e1006bf60 con 0x7f9df4003790 2024-01-24T23:27:54.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 1 -- 172.21.15.115:0/221602374 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3564425720 0 0) 0x7f9dfc003530 con 0x7f9df4007420 2024-01-24T23:27:54.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 1 -- 172.21.15.115:0/221602374 >> v1:172.21.15.115:6791/0 conn(0x7f9df400b1b0 legacy=0x7f9df400d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:54.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 1 -- 172.21.15.115:0/221602374 >> v1:172.21.15.132:6790/0 conn(0x7f9df4003790 legacy=0x7f9df4005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:54.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 10 monclient: _finish_hunting 0 2024-01-24T23:27:54.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 1 monclient: found mon.noname-f 2024-01-24T23:27:54.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:54.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:27:54.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 1 -- 172.21.15.115:0/221602374 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1009b1e0 con 0x7f9df4007420 2024-01-24T23:27:54.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 10 monclient: _finish_auth 0 2024-01-24T23:27:54.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 10 monclient: _check_auth_tickets 2024-01-24T23:27:54.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:54.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e17885640 5 monclient: authenticate success, global_id 124381 2024-01-24T23:27:54.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e15e22640 1 -- 172.21.15.115:0/221602374 reap_dead start 2024-01-24T23:27:54.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 1 -- 172.21.15.115:0/221602374 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9e04003f40 con 0x7f9df4007420 2024-01-24T23:27:54.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e17885640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:54.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:54.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:27:54.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 10 monclient: dump: 2024-01-24T23:27:54.376 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:54.376 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:54.376 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:54.377 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:54.377 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:54.377 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:54.377 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:54.377 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:54.377 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:54.377 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:54.377 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:54.377 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:54.377 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:54.377 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:54.377 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:54.377 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:54.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e17885640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:54.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 1 -- 172.21.15.115:0/221602374 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9e04004160 con 0x7f9df4007420 2024-01-24T23:27:54.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:54.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e17885640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:54.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e17885640 4 set_mon_vals no callback set 2024-01-24T23:27:54.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 1 -- 172.21.15.115:0/221602374 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9e04004710 con 0x7f9df4007420 2024-01-24T23:27:54.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:54.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:27:54.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.352+0000 7f9e03fff640 10 monclient: dump: 2024-01-24T23:27:54.378 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:54.378 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:54.378 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:54.378 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:54.378 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:54.378 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:54.378 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:54.379 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:54.379 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:54.379 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:54.379 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:54.379 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:54.379 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:54.379 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:54.379 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:54.379 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:54.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.356+0000 7f9e17885640 10 monclient: shutdown 2024-01-24T23:27:54.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.356+0000 7f9e17885640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:54.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.356+0000 7f9e17885640 1 -- 172.21.15.115:0/221602374 >> v1:172.21.15.132:6789/0 conn(0x7f9df4007420 legacy=0x7f9df40098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:54.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.356+0000 7f9e17885640 1 -- 172.21.15.115:0/221602374 shutdown_connections 2024-01-24T23:27:54.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.356+0000 7f9e17885640 1 -- 172.21.15.115:0/221602374 >> 172.21.15.115:0/221602374 conn(0x7f9e100a9850 msgr2=0x7f9e100a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:54.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.356+0000 7f9e17885640 1 -- 172.21.15.115:0/221602374 shutdown_connections 2024-01-24T23:27:54.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.356+0000 7f9e17885640 1 -- 172.21.15.115:0/221602374 wait complete. 2024-01-24T23:27:54.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.356+0000 7f9e17885640 10 monclient: build_initial_monmap 2024-01-24T23:27:54.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.356+0000 7f9e17885640 10 monclient: monmap: 2024-01-24T23:27:54.380 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:27:54.380 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:54.380 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:27:54.380 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:27:54.380 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:54.380 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:27:54.380 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:54.380 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:54.380 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:54.380 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:54.380 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:54.380 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:54.381 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:54.381 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:54.381 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:54.381 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:54.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.356+0000 7f9e17885640 1 Processor -- start 2024-01-24T23:27:54.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.356+0000 7f9e17885640 1 -- start start 2024-01-24T23:27:54.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.356+0000 7f9e17885640 10 monclient: init 2024-01-24T23:27:54.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.356+0000 7f9e17885640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:54.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.356+0000 7f9e17885640 10 monclient: _add_conns ranks=[5,7,8,0,6,4,1,2,3] 2024-01-24T23:27:54.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.356+0000 7f9e17885640 10 monclient(hunting): picked mon.noname-h con 0x7f9e101cb690 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:54.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.356+0000 7f9e17885640 10 monclient(hunting): picked mon.noname-i con 0x7f9e10132cc0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:54.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.356+0000 7f9e17885640 10 monclient(hunting): picked mon.noname-e con 0x7f9e101cc180 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:54.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.356+0000 7f9e17885640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e100a56d0 con 0x7f9e101cb690 2024-01-24T23:27:54.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.356+0000 7f9e17885640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1009c5a0 con 0x7f9e10132cc0 2024-01-24T23:27:54.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.356+0000 7f9e17885640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1009c960 con 0x7f9e101cc180 2024-01-24T23:27:54.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.356+0000 7f9e17885640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:54.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.356+0000 7f9e14e20640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f9e10132cc0 legacy=0x7f9e101d0c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:54.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.356+0000 7f9e17885640 10 monclient(hunting): authenticate will time out at 1353.283864s 2024-01-24T23:27:54.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.356+0000 7f9e15621640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f9e101cb690 0x7f9e101cba70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.115:35792/0 (socket says 172.21.15.115:35792) 2024-01-24T23:27:54.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e15621640 1 -- 172.21.15.115:0/3638360458 learned_addr learned my addr 172.21.15.115:0/3638360458 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:27:54.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936541051 0 0) 0x7f9e100a56d0 con 0x7f9e101cb690 2024-01-24T23:27:54.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:54.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:54.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:54.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient(hunting): my global_id is 6773 2024-01-24T23:27:54.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9df400efc0 con 0x7f9e101cb690 2024-01-24T23:27:54.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4014012005 0 0) 0x7f9e1009c960 con 0x7f9e101cc180 2024-01-24T23:27:54.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:54.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:54.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:54.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient(hunting): my global_id is 85967 2024-01-24T23:27:54.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e100a56d0 con 0x7f9e101cc180 2024-01-24T23:27:54.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 979164826 0 0) 0x7f9df400efc0 con 0x7f9e101cb690 2024-01-24T23:27:54.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9e1009c960 con 0x7f9e101cb690 2024-01-24T23:27:54.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9e0c0044f0 con 0x7f9e101cb690 2024-01-24T23:27:54.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:54.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:27:54.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient(hunting): dump: 2024-01-24T23:27:54.383 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:54.383 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:54.383 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:54.383 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:54.383 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:54.383 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1172745311 0 0) 0x7f9e1009c960 con 0x7f9e101cb690 2024-01-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 >> v1:172.21.15.115:6793/0 conn(0x7f9e101cc180 legacy=0x7f9e101d4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:54.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 >> v1:172.21.15.132:6792/0 conn(0x7f9e10132cc0 legacy=0x7f9e101d0c20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 monclient: found mon.h 2024-01-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1009d160 con 0x7f9e101cb690 2024-01-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient: _finish_auth 0 2024-01-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e17885640 5 monclient: authenticate success, global_id 6773 2024-01-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e17885640 10 monclient: _renew_subs 2024-01-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e17885640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e17885640 1 -- 172.21.15.115:0/3638360458 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e1009e710 con 0x7f9e101cb690 2024-01-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e17885640 10 monclient: _renew_subs 2024-01-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e17885640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:54.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e17885640 1 -- 172.21.15.115:0/3638360458 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e1009f730 con 0x7f9e101cb690 2024-01-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9e0c0049c0 con 0x7f9e101cb690 2024-01-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9e0c004e70 con 0x7f9e101cb690 2024-01-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e037fe640 4 set_mon_vals no callback set 2024-01-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient: dump: 2024-01-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:54.386 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:54.387 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:54.387 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:54.387 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:54.387 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:54.387 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:54.387 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:54.387 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:54.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e17885640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:54.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f9e0c004fd0 con 0x7f9e101cb690 2024-01-24T23:27:54.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e17885640 1 -- 172.21.15.115:0/3638360458 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9df00065d0 con 0x7f9df0002870 2024-01-24T23:27:54.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(224..224 src has 1..224) v4 ==== 11849+0+0 (unknown 3096591530 0 0) 0x7f9e0c045830 con 0x7f9e101cb690 2024-01-24T23:27:54.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e15e22640 10 monclient: get_auth_request con 0x7f9df4038110 auth_method 0 2024-01-24T23:27:54.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 59676029 0 0) 0x7f9df00065d0 con 0x7f9df0002870 2024-01-24T23:27:54.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:54.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:54.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:54.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient(hunting): my global_id is 6782 2024-01-24T23:27:54.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e1009c960 con 0x7f9df0002870 2024-01-24T23:27:54.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1926133450 0 0) 0x7f9e1009c960 con 0x7f9df0002870 2024-01-24T23:27:54.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9df00065d0 con 0x7f9df0002870 2024-01-24T23:27:54.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9e04004650 con 0x7f9df0002870 2024-01-24T23:27:54.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:54.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:54.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient: dump: 2024-01-24T23:27:54.388 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:54.388 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:54.388 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:54.388 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:54.388 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:54.388 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:54.389 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:54.389 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:54.389 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:54.389 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:54.389 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:54.389 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:54.389 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:54.389 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:54.389 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:54.389 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:54.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1428316806 0 0) 0x7f9df00065d0 con 0x7f9df0002870 2024-01-24T23:27:54.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 --> v1:172.21.15.132:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9df000a120 con 0x7f9df0002870 2024-01-24T23:27:54.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f9e04004830 con 0x7f9df0002870 2024-01-24T23:27:54.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:54.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:54.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 >> v1:172.21.15.132:6791/0 conn(0x7f9df0002870 legacy=0x7f9df0004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:54.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.360+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 >> v1:172.21.15.132:6791/0 conn(0x7f9df0002870 legacy=0x7f9df0004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:54.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.368+0000 7f9e17885640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:27:54.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.368+0000 7f9e17885640 1 -- 172.21.15.115:0/3638360458 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9df0009e10 con 0x7f9df00090d0 2024-01-24T23:27:54.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 886879100 0 0) 0x7f9df0009e10 con 0x7f9df00090d0 2024-01-24T23:27:54.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e01ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:54.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e01ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:54.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e01ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:54.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e01ffb640 10 monclient(hunting): my global_id is 6791 2024-01-24T23:27:54.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9df00065d0 con 0x7f9df00090d0 2024-01-24T23:27:54.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3325216834 0 0) 0x7f9df00065d0 con 0x7f9df00090d0 2024-01-24T23:27:54.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9df0009e10 con 0x7f9df00090d0 2024-01-24T23:27:54.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9e04004ff0 con 0x7f9df00090d0 2024-01-24T23:27:54.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e01ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:54.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e01ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:54.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e01ffb640 10 monclient: dump: 2024-01-24T23:27:54.392 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:27:54.392 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:54.392 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:54.392 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:54.392 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:54.392 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:27:54.392 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:54.392 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:54.393 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:54.393 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:54.393 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:54.393 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:54.393 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:54.393 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:54.393 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:54.393 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:27:54.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1621158729 0 0) 0x7f9df0009e10 con 0x7f9df00090d0 2024-01-24T23:27:54.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 --> v1:172.21.15.132:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9df000c310 con 0x7f9df00090d0 2024-01-24T23:27:54.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7512 (unknown 0 0 2183046639) 0x7f9e040072d0 con 0x7f9df00090d0 2024-01-24T23:27:54.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e01ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:27:54.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e01ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:27:54.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 >> v1:172.21.15.132:6791/0 conn(0x7f9df00090d0 legacy=0x7f9df0009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:54.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e01ffb640 1 -- 172.21.15.115:0/3638360458 >> v1:172.21.15.132:6791/0 conn(0x7f9df00090d0 legacy=0x7f9df0009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:27:54.394 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:27:54.394 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:54.394 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:54.394 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "leader", 2024-01-24T23:27:54.394 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 44, 2024-01-24T23:27:54.394 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:27:54.394 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:27:54.394 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:27:54.394 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:27:54.395 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:27:54.395 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:27:54.395 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:27:54.395 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:54.395 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 4, 2024-01-24T23:27:54.395 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:54.395 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:27:54.395 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:27:54.395 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:54.395 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:54.395 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:54.395 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:54.395 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:54.395 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:54.396 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:54.396 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:54.396 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:54.396 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:54.396 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:54.396 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:54.396 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:27:54.396 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:27:54.396 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:54.396 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:54.396 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:54.396 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:54.396 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:54.396 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:54.397 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:54.397 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:54.397 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:54.397 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:54.397 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:54.397 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:54.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:54.397 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:27:54.397 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:27:54.397 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:27:54.397 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:27:54.397 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:27:54.397 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:27:54.397 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:54.397 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:27:54.398 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:27:54.398 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:27:54.398 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:27:54.398 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:27:54.398 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:27:54.398 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:27:54.398 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:27:54.398 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:27:54.398 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:27:54.398 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:27:54.398 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:27:54.398 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:27:54.398 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:27:54.398 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:27:54.398 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:27:54.399 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:27:54.399 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:27:54.399 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:27:54.399 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:27:54.399 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:27:54.399 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:54.399 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:27:54.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:54.399 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:27:54.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:54.399 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:27:54.399 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:27:54.399 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:54.399 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:54.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:54.400 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:54.400 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:27:54.400 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:54.400 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:54.400 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:54.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:54.400 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:27:54.400 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:27:54.400 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:54.400 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:54.400 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:54.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:54.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:54.401 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:27:54.401 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:27:54.401 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:54.401 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:54.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:54.401 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:54.401 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:27:54.401 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:54.401 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:54.401 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:54.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:54.401 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:27:54.401 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:27:54.401 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:54.402 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:54.402 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:54.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:54.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:54.402 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:27:54.402 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:27:54.402 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:54.402 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:54.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:54.402 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:54.402 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:27:54.402 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:54.402 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:54.402 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:54.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:54.403 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:27:54.403 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:27:54.403 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:54.403 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:54.403 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:54.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:54.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:54.403 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:27:54.403 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:27:54.403 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:54.403 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:54.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:54.403 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:54.403 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:27:54.403 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:54.404 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:54.404 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:54.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:54.404 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:27:54.404 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:27:54.404 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:54.404 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:54.404 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:54.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:54.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:54.404 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:27:54.404 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:27:54.404 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:54.405 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:54.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:54.405 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:54.405 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:27:54.405 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:54.405 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:54.405 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:54.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:54.405 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:27:54.405 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:27:54.405 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:54.405 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:54.405 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:54.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:54.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:54.406 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:27:54.406 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:27:54.406 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:54.406 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:54.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:54.406 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:54.406 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:27:54.406 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:54.406 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:54.406 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:54.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:54.406 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:27:54.406 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:27:54.407 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:54.407 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:54.407 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:54.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:54.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:54.407 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:27:54.407 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:27:54.407 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:54.407 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:54.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:54.407 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:54.407 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:27:54.407 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:54.407 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:54.408 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:54.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:54.408 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:27:54.408 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:27:54.408 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:54.408 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:54.408 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:54.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:54.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:54.408 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:27:54.408 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:27:54.408 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:54.408 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:54.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:54.409 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:54.409 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:27:54.409 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:54.409 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:54.409 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:54.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:54.409 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:27:54.409 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:27:54.409 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:54.409 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:54.409 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:54.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:54.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:54.409 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:27:54.410 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:27:54.410 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:27:54.410 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:27:54.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:54.410 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:27:54.410 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:27:54.410 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:27:54.410 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:54.410 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:54.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:54.410 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:27:54.410 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:27:54.410 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:27:54.410 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:27:54.411 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:27:54.411 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:54.411 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:54.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:54.411 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:27:54.411 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:27:54.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:54.411 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:54.411 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:54.411 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:27:54.411 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:54.411 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:27:54.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:27:54.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:27:54.411 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:27:54.412 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:27:54.412 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:27:54.412 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:27:54.412 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:27:54.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:27:54.412 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:27:54.412 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:27:54.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e17885640 1 -- 172.21.15.115:0/3638360458 >> v1:172.21.15.132:6812/1835401922 conn(0x7f9df4038110 legacy=0x7f9df403a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:54.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e17885640 10 monclient: shutdown 2024-01-24T23:27:54.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e17885640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:54.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e17885640 1 -- 172.21.15.115:0/3638360458 >> v1:172.21.15.132:6791/0 conn(0x7f9e101cb690 legacy=0x7f9e101cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:54.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e15e22640 1 -- 172.21.15.115:0/3638360458 reap_dead start 2024-01-24T23:27:54.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e15e22640 1 -- 172.21.15.115:0/3638360458 reap_dead start 2024-01-24T23:27:54.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e17885640 1 -- 172.21.15.115:0/3638360458 shutdown_connections 2024-01-24T23:27:54.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e17885640 1 -- 172.21.15.115:0/3638360458 >> 172.21.15.115:0/3638360458 conn(0x7f9e100a9850 msgr2=0x7f9e10131180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:54.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e17885640 1 -- 172.21.15.115:0/3638360458 shutdown_connections 2024-01-24T23:27:54.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:27:54.372+0000 7f9e17885640 1 -- 172.21.15.115:0/3638360458 wait complete. 2024-01-24T23:27:54.414 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-01-24T23:27:54.414 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-01-24T23:27:54.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.402+0000 7f1c88512640 10 monclient: get_monmap_and_config 2024-01-24T23:27:54.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.402+0000 7f1c88512640 10 monclient: build_initial_monmap 2024-01-24T23:27:54.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.402+0000 7f1c88512640 10 monclient: monmap: 2024-01-24T23:27:54.422 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:54.422 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:54.422 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:54.407789+0000 2024-01-24T23:27:54.422 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:54.407789+0000 2024-01-24T23:27:54.422 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:54.423 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:54.423 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:54.423 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:54.423 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:54.423 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:54.423 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:54.423 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:54.423 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:54.423 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:54.423 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:54.423 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:54.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 1 Processor -- start 2024-01-24T23:27:54.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 1 -- start start 2024-01-24T23:27:54.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 10 monclient: init 2024-01-24T23:27:54.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:54.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 10 monclient: _add_conns ranks=[3,4,0,2,8,6,5,7,1] 2024-01-24T23:27:54.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 10 monclient(hunting): picked mon.noname-g con 0x7f1c801319b0 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:54.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 10 monclient(hunting): picked mon.noname-c con 0x7f1c80131330 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:54.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 10 monclient(hunting): picked mon.noname-a con 0x7f1c80071c00 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:54.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c80064700 con 0x7f1c80071c00 2024-01-24T23:27:54.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c80070b70 con 0x7f1c801319b0 2024-01-24T23:27:54.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c800a1fd0 con 0x7f1c80131330 2024-01-24T23:27:54.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:54.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 10 monclient(hunting): authenticate will time out at 1361.733162s 2024-01-24T23:27:54.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c86aaf640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f1c80071c00 legacy=0x7f1c8013dc00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:27:54.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c862ae640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f1c801319b0 0x7f1c800a11b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:41894/0 (socket says 172.21.15.132:41894) 2024-01-24T23:27:54.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c862ae640 1 -- 172.21.15.132:0/3383635827 learned_addr learned my addr 172.21.15.132:0/3383635827 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:54.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 1 -- 172.21.15.132:0/3383635827 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2397342287 0 0) 0x7f1c80070b70 con 0x7f1c801319b0 2024-01-24T23:27:54.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 10 monclient(hunting): _negotiate 2024-01-24T23:27:54.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:54.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:54.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 10 monclient(hunting): my global_id is 6798 2024-01-24T23:27:54.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 1 -- 172.21.15.132:0/3383635827 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c6c003530 con 0x7f1c801319b0 2024-01-24T23:27:54.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 1 -- 172.21.15.132:0/3383635827 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2078468758 0 0) 0x7f1c6c003530 con 0x7f1c801319b0 2024-01-24T23:27:54.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 1 -- 172.21.15.132:0/3383635827 >> v1:172.21.15.115:6791/0 conn(0x7f1c80131330 legacy=0x7f1c800a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:54.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 1 -- 172.21.15.132:0/3383635827 >> v1:172.21.15.115:6789/0 conn(0x7f1c80071c00 legacy=0x7f1c8013dc00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:54.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 10 monclient: _finish_hunting 0 2024-01-24T23:27:54.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 1 monclient: found mon.noname-g 2024-01-24T23:27:54.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:54.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:27:54.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 1 -- 172.21.15.132:0/3383635827 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c800717c0 con 0x7f1c801319b0 2024-01-24T23:27:54.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 10 monclient: _finish_auth 0 2024-01-24T23:27:54.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 10 monclient: _check_auth_tickets 2024-01-24T23:27:54.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:54.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 1 -- 172.21.15.132:0/3383635827 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1c7c0017c0 con 0x7f1c801319b0 2024-01-24T23:27:54.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:54.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:27:54.426 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 10 monclient: dump: 2024-01-24T23:27:54.426 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:54.426 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:54.426 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:54.426 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:54.426 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:54.426 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:54.426 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:54.426 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:54.426 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:54.426 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:54.426 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:54.427 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:54.427 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:54.427 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:54.427 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:54.427 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:54.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 5 monclient: authenticate success, global_id 6798 2024-01-24T23:27:54.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:54.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 1 -- 172.21.15.132:0/3383635827 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1c7c0019e0 con 0x7f1c801319b0 2024-01-24T23:27:54.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:54.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 1 -- 172.21.15.132:0/3383635827 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1c7c004bf0 con 0x7f1c801319b0 2024-01-24T23:27:54.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:54.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 4 set_mon_vals no callback set 2024-01-24T23:27:54.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:54.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:54.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c852ac640 10 monclient: dump: 2024-01-24T23:27:54.427 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:54.428 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:54.428 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:54.428 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:54.428 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:54.428 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:54.428 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:54.428 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:54.428 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:54.428 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:54.428 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:54.428 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:54.428 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:54.428 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:54.428 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:54.428 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:54.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 10 monclient: shutdown 2024-01-24T23:27:54.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:54.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 1 -- 172.21.15.132:0/3383635827 >> v1:172.21.15.132:6790/0 conn(0x7f1c801319b0 legacy=0x7f1c800a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:54.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 1 -- 172.21.15.132:0/3383635827 shutdown_connections 2024-01-24T23:27:54.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 1 -- 172.21.15.132:0/3383635827 >> 172.21.15.132:0/3383635827 conn(0x7f1c800a22a0 msgr2=0x7f1c800a2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:54.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 1 -- 172.21.15.132:0/3383635827 shutdown_connections 2024-01-24T23:27:54.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 1 -- 172.21.15.132:0/3383635827 wait complete. 2024-01-24T23:27:54.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 10 monclient: build_initial_monmap 2024-01-24T23:27:54.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 10 monclient: monmap: 2024-01-24T23:27:54.429 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:54.429 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:54.429 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:54.429 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:54.429 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:54.429 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:54.430 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:54.430 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:54.430 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:54.430 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:54.430 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:54.430 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:54.430 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:54.430 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:54.430 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:54.430 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:54.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 1 Processor -- start 2024-01-24T23:27:54.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 1 -- start start 2024-01-24T23:27:54.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 10 monclient: init 2024-01-24T23:27:54.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:54.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 10 monclient: _add_conns ranks=[8,2,6,3,7,5,0,4,1] 2024-01-24T23:27:54.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 10 monclient(hunting): picked mon.noname-e con 0x7f1c80071c00 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:54.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 10 monclient(hunting): picked mon.noname-b con 0x7f1c80131330 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:54.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 10 monclient(hunting): picked mon.noname-d con 0x7f1c801319b0 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:54.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c8009d6a0 con 0x7f1c80131330 2024-01-24T23:27:54.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c80070360 con 0x7f1c801319b0 2024-01-24T23:27:54.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c8009d050 con 0x7f1c80071c00 2024-01-24T23:27:54.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:54.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.406+0000 7f1c88512640 10 monclient(hunting): authenticate will time out at 1361.735979s 2024-01-24T23:27:54.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c85aad640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f1c80131330 legacy=0x7f1c801cf9f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:54.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c862ae640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f1c80071c00 0x7f1c8012ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:43270/0 (socket says 172.21.15.132:43270) 2024-01-24T23:27:54.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c862ae640 1 -- 172.21.15.132:0/1444221843 learned_addr learned my addr 172.21.15.132:0/1444221843 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:54.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 1 -- 172.21.15.132:0/1444221843 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 272288480 0 0) 0x7f1c8009d050 con 0x7f1c80071c00 2024-01-24T23:27:54.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:54.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:54.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:54.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 10 monclient(hunting): my global_id is 85976 2024-01-24T23:27:54.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 1 -- 172.21.15.132:0/1444221843 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c64003530 con 0x7f1c80071c00 2024-01-24T23:27:54.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 1 -- 172.21.15.132:0/1444221843 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 120659973 0 0) 0x7f1c80070360 con 0x7f1c801319b0 2024-01-24T23:27:54.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:54.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:54.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:54.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 10 monclient(hunting): my global_id is 7008 2024-01-24T23:27:54.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 1 -- 172.21.15.132:0/1444221843 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c8009d050 con 0x7f1c801319b0 2024-01-24T23:27:54.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 1 -- 172.21.15.132:0/1444221843 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1077164676 0 0) 0x7f1c8009d050 con 0x7f1c801319b0 2024-01-24T23:27:54.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 1 -- 172.21.15.132:0/1444221843 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1c80070360 con 0x7f1c801319b0 2024-01-24T23:27:54.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 1 -- 172.21.15.132:0/1444221843 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 848750019 0 0) 0x7f1c64003530 con 0x7f1c80071c00 2024-01-24T23:27:54.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 1 -- 172.21.15.132:0/1444221843 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1c8009d050 con 0x7f1c80071c00 2024-01-24T23:27:54.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 1 -- 172.21.15.132:0/1444221843 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1c78004560 con 0x7f1c801319b0 2024-01-24T23:27:54.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:54.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:27:54.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 10 monclient(hunting): dump: 2024-01-24T23:27:54.433 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:54.433 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:54.433 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:54.433 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:54.433 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:54.433 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:54.433 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:54.433 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:54.433 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:54.433 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:54.433 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:54.433 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:54.433 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:54.433 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:54.433 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:54.434 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:54.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:54.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 1 -- 172.21.15.132:0/1444221843 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1c7c001740 con 0x7f1c80071c00 2024-01-24T23:27:54.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:54.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:27:54.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 10 monclient(hunting): dump: 2024-01-24T23:27:54.434 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:54.434 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:54.434 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:54.434 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:54.434 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:54.434 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:54.434 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:54.434 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:54.434 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:54.435 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:54.435 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:54.435 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:54.435 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:54.435 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:54.435 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:54.435 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:54.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 1 -- 172.21.15.132:0/1444221843 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1915811332 0 0) 0x7f1c80070360 con 0x7f1c801319b0 2024-01-24T23:27:54.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 1 -- 172.21.15.132:0/1444221843 >> v1:172.21.15.115:6793/0 conn(0x7f1c80071c00 legacy=0x7f1c8012ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:54.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 1 -- 172.21.15.132:0/1444221843 >> v1:172.21.15.115:6790/0 conn(0x7f1c80131330 legacy=0x7f1c801cf9f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:54.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:27:54.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 1 monclient: found mon.d 2024-01-24T23:27:54.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:54.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:54.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 1 -- 172.21.15.132:0/1444221843 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c8009d330 con 0x7f1c801319b0 2024-01-24T23:27:54.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 10 monclient: _finish_auth 0 2024-01-24T23:27:54.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:27:54.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:54.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c88512640 5 monclient: authenticate success, global_id 7008 2024-01-24T23:27:54.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c88512640 10 monclient: _renew_subs 2024-01-24T23:27:54.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c88512640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:54.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c88512640 1 -- 172.21.15.132:0/1444221843 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c8009bb70 con 0x7f1c801319b0 2024-01-24T23:27:54.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c88512640 10 monclient: _renew_subs 2024-01-24T23:27:54.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c88512640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:54.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c88512640 1 -- 172.21.15.132:0/1444221843 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c8009dee0 con 0x7f1c801319b0 2024-01-24T23:27:54.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 1 -- 172.21.15.132:0/1444221843 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1c78004750 con 0x7f1c801319b0 2024-01-24T23:27:54.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:54.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 1 -- 172.21.15.132:0/1444221843 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1c78004c80 con 0x7f1c801319b0 2024-01-24T23:27:54.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:54.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c84aab640 4 set_mon_vals no callback set 2024-01-24T23:27:54.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:27:54.437 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 10 monclient: dump: 2024-01-24T23:27:54.437 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:54.437 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:54.437 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:54.437 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:54.437 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:54.437 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:54.437 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:54.437 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:54.437 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:54.437 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:54.437 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:54.437 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:54.437 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:54.438 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:54.438 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:54.438 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:54.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c88512640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:54.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c88512640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:54.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c88512640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:54.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c88512640 1 -- 172.21.15.132:0/1444221843 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c68005090 con 0x7f1c801319b0 2024-01-24T23:27:54.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 1 -- 172.21.15.132:0/1444221843 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f1c78004e60 con 0x7f1c801319b0 2024-01-24T23:27:54.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c76ffd640 1 -- 172.21.15.132:0/1444221843 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(224..224 src has 1..224) v4 ==== 11849+0+0 (unknown 3096591530 0 0) 0x7f1c780459f0 con 0x7f1c801319b0 2024-01-24T23:27:54.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.410+0000 7f1c862ae640 10 monclient: get_auth_request con 0x7f1c640376b0 auth_method 0 2024-01-24T23:27:54.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.414+0000 7f1c76ffd640 1 -- 172.21.15.132:0/1444221843 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f1c78014c60 con 0x7f1c801319b0 2024-01-24T23:27:54.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.414+0000 7f1c76ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:54.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.414+0000 7f1c76ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:54.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.574+0000 7f1c88512640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h13"}] 2024-01-24T23:27:54.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.574+0000 7f1c88512640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h13"}] 2024-01-24T23:27:54.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.574+0000 7f1c88512640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:54.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:54.574+0000 7f1c88512640 1 -- 172.21.15.132:0/1444221843 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "h13"} v 0) v1 -- 0x7f1c68005340 con 0x7f1c801319b0 2024-01-24T23:27:55.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.178+0000 7f1c76ffd640 1 -- 172.21.15.132:0/1444221843 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h13"}]=0 removed item id -55 name 'h13' from crush map v225) v1 ==== 120+0+0 (unknown 1787167124 0 0) 0x7f1c7801d1c0 con 0x7f1c801319b0 2024-01-24T23:27:55.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.178+0000 7f1c76ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h13"}] 2024-01-24T23:27:55.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.178+0000 7f1c76ffd640 10 monclient: _finish_command 2 = system:0 removed item id -55 name 'h13' from crush map 2024-01-24T23:27:55.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.278+0000 7f1c88512640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h13"}] 2024-01-24T23:27:55.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.278+0000 7f1c88512640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h13"}] 2024-01-24T23:27:55.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.278+0000 7f1c88512640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:27:55.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.278+0000 7f1c88512640 1 -- 172.21.15.132:0/1444221843 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "h13"} v 0) v1 -- 0x7f1c68002220 con 0x7f1c801319b0 2024-01-24T23:27:55.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.282+0000 7f1c76ffd640 1 -- 172.21.15.132:0/1444221843 <== mon.6 v1:172.21.15.115:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h13"}]=0 device 'h13' does not appear in the crush map v225) v1 ==== 120+0+0 (unknown 327264595 0 0) 0x7f1c780130d0 con 0x7f1c801319b0 2024-01-24T23:27:55.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.282+0000 7f1c76ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h13"}] 2024-01-24T23:27:55.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.282+0000 7f1c76ffd640 10 monclient: _finish_command 3 = system:0 device 'h13' does not appear in the crush map 2024-01-24T23:27:55.282 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h13' does not appear in the crush map 2024-01-24T23:27:55.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.282+0000 7f1c88512640 1 -- 172.21.15.132:0/1444221843 >> v1:172.21.15.132:6812/1835401922 conn(0x7f1c640376b0 legacy=0x7f1c64039b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:55.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.282+0000 7f1c88512640 10 monclient: shutdown 2024-01-24T23:27:55.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.282+0000 7f1c88512640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:55.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.282+0000 7f1c88512640 1 -- 172.21.15.132:0/1444221843 >> v1:172.21.15.115:6792/0 conn(0x7f1c801319b0 legacy=0x7f1c801d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:55.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.282+0000 7f1c88512640 1 -- 172.21.15.132:0/1444221843 shutdown_connections 2024-01-24T23:27:55.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.282+0000 7f1c88512640 1 -- 172.21.15.132:0/1444221843 >> 172.21.15.132:0/1444221843 conn(0x7f1c800a22a0 msgr2=0x7f1c8012f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:55.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.282+0000 7f1c88512640 1 -- 172.21.15.132:0/1444221843 shutdown_connections 2024-01-24T23:27:55.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.282+0000 7f1c88512640 1 -- 172.21.15.132:0/1444221843 wait complete. 2024-01-24T23:27:55.298 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:55.298 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h14 2024-01-24T23:27:55.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.410+0000 7f444e7b3640 10 monclient: get_monmap_and_config 2024-01-24T23:27:55.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.410+0000 7f444e7b3640 10 monclient: build_initial_monmap 2024-01-24T23:27:55.413 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.410+0000 7f444e7b3640 10 monclient: monmap: 2024-01-24T23:27:55.413 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:55.413 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:55.413 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:55.415635+0000 2024-01-24T23:27:55.413 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:55.415635+0000 2024-01-24T23:27:55.413 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:55.413 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:55.413 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:55.413 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:55.413 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:55.414 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:55.414 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:55.414 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:55.414 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:55.414 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:55.414 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:55.414 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:55.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.410+0000 7f444e7b3640 1 Processor -- start 2024-01-24T23:27:55.420 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.410+0000 7f444e7b3640 1 -- start start 2024-01-24T23:27:55.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.410+0000 7f444e7b3640 10 monclient: init 2024-01-24T23:27:55.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:55.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 10 monclient: _add_conns ranks=[0,4,8,3,6,1,7,2,5] 2024-01-24T23:27:55.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 10 monclient(hunting): picked mon.noname-a con 0x7f444809ff20 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:55.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 10 monclient(hunting): picked mon.noname-c con 0x7f444809f9c0 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:55.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 10 monclient(hunting): picked mon.noname-e con 0x7f444813b2c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:55.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4448064700 con 0x7f444809ff20 2024-01-24T23:27:55.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f444809e760 con 0x7f444809f9c0 2024-01-24T23:27:55.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44480a1160 con 0x7f444813b2c0 2024-01-24T23:27:55.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:55.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 10 monclient(hunting): authenticate will time out at 1362.740479s 2024-01-24T23:27:55.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4447fff640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f444809ff20 legacy=0x7f44480a0300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:27:55.421 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444cd50640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f444813b2c0 0x7f444813d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:43272/0 (socket says 172.21.15.132:43272) 2024-01-24T23:27:55.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444cd50640 1 -- 172.21.15.132:0/2006107501 learned_addr learned my addr 172.21.15.132:0/2006107501 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:55.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 1 -- 172.21.15.132:0/2006107501 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2623722320 0 0) 0x7f44480a1160 con 0x7f444813b2c0 2024-01-24T23:27:55.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:55.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:55.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:55.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 10 monclient(hunting): my global_id is 85985 2024-01-24T23:27:55.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 1 -- 172.21.15.132:0/2006107501 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4430003530 con 0x7f444813b2c0 2024-01-24T23:27:55.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 1 -- 172.21.15.132:0/2006107501 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3364236886 0 0) 0x7f444809e760 con 0x7f444809f9c0 2024-01-24T23:27:55.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:55.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:55.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:55.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 10 monclient(hunting): my global_id is 94990 2024-01-24T23:27:55.422 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 1 -- 172.21.15.132:0/2006107501 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44480a1160 con 0x7f444809f9c0 2024-01-24T23:27:55.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 1 -- 172.21.15.132:0/2006107501 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1815434339 0 0) 0x7f4430003530 con 0x7f444813b2c0 2024-01-24T23:27:55.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 1 -- 172.21.15.132:0/2006107501 >> v1:172.21.15.115:6791/0 conn(0x7f444809f9c0 legacy=0x7f4448139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:55.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 1 -- 172.21.15.132:0/2006107501 >> v1:172.21.15.115:6789/0 conn(0x7f444809ff20 legacy=0x7f44480a0300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:55.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:27:55.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 1 monclient: found mon.noname-e 2024-01-24T23:27:55.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:55.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:27:55.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 1 -- 172.21.15.132:0/2006107501 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44480a0a10 con 0x7f444813b2c0 2024-01-24T23:27:55.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 10 monclient: _finish_auth 0 2024-01-24T23:27:55.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:27:55.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:55.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 5 monclient: authenticate success, global_id 85985 2024-01-24T23:27:55.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:55.423 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 1 -- 172.21.15.132:0/2006107501 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4440001800 con 0x7f444813b2c0 2024-01-24T23:27:55.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:55.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:27:55.424 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 10 monclient: dump: 2024-01-24T23:27:55.424 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:55.424 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:55.424 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:55.424 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:55.424 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:55.424 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:55.424 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:55.424 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:55.424 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:55.424 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:55.424 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:55.425 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:55.425 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:55.425 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:55.425 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:55.425 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:55.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:55.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 1 -- 172.21.15.132:0/2006107501 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4440001a90 con 0x7f444813b2c0 2024-01-24T23:27:55.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:55.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:55.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 1 -- 172.21.15.132:0/2006107501 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4440004bf0 con 0x7f444813b2c0 2024-01-24T23:27:55.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 4 set_mon_vals no callback set 2024-01-24T23:27:55.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:55.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:27:55.425 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f4446ffd640 10 monclient: dump: 2024-01-24T23:27:55.426 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:55.426 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:55.426 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:55.426 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:55.426 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:55.426 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:55.426 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:55.426 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:55.426 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:55.426 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:55.426 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:55.426 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:55.426 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:55.426 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:55.426 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:55.427 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:55.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 10 monclient: shutdown 2024-01-24T23:27:55.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:55.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 1 -- 172.21.15.132:0/2006107501 >> v1:172.21.15.115:6793/0 conn(0x7f444813b2c0 legacy=0x7f444813d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:55.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 1 -- 172.21.15.132:0/2006107501 shutdown_connections 2024-01-24T23:27:55.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 1 -- 172.21.15.132:0/2006107501 >> 172.21.15.132:0/2006107501 conn(0x7f44480a5a10 msgr2=0x7f44480a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:55.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 1 -- 172.21.15.132:0/2006107501 shutdown_connections 2024-01-24T23:27:55.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 1 -- 172.21.15.132:0/2006107501 wait complete. 2024-01-24T23:27:55.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 10 monclient: build_initial_monmap 2024-01-24T23:27:55.427 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 10 monclient: monmap: 2024-01-24T23:27:55.427 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:55.427 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:55.427 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:55.428 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:55.428 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:55.428 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:55.428 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:55.428 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:55.428 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:55.428 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:55.428 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:55.428 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:55.428 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:55.428 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:55.428 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:55.428 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:55.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 1 Processor -- start 2024-01-24T23:27:55.428 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 1 -- start start 2024-01-24T23:27:55.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.414+0000 7f444e7b3640 10 monclient: init 2024-01-24T23:27:55.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f444e7b3640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:55.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f444e7b3640 10 monclient: _add_conns ranks=[8,7,3,0,6,5,1,2,4] 2024-01-24T23:27:55.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f444e7b3640 10 monclient(hunting): picked mon.noname-e con 0x7f444809f9c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:55.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f444e7b3640 10 monclient(hunting): picked mon.noname-i con 0x7f444809ff20 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:55.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f444e7b3640 10 monclient(hunting): picked mon.noname-g con 0x7f444813b2c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:55.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f444e7b3640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f444811ee40 con 0x7f444813b2c0 2024-01-24T23:27:55.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f444e7b3640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f444811ea20 con 0x7f444809ff20 2024-01-24T23:27:55.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f444e7b3640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f444811c430 con 0x7f444809f9c0 2024-01-24T23:27:55.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f444e7b3640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:55.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f444e7b3640 10 monclient(hunting): authenticate will time out at 1362.744090s 2024-01-24T23:27:55.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f44477fe640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f444809ff20 legacy=0x7f444811bd20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:55.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f444cd50640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f444813b2c0 0x7f44481d1490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:41910/0 (socket says 172.21.15.132:41910) 2024-01-24T23:27:55.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f444cd50640 1 -- 172.21.15.132:0/2351332934 learned_addr learned my addr 172.21.15.132:0/2351332934 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:55.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 1 -- 172.21.15.132:0/2351332934 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186623735 0 0) 0x7f444811ee40 con 0x7f444813b2c0 2024-01-24T23:27:55.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:55.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:55.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:55.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 10 monclient(hunting): my global_id is 6807 2024-01-24T23:27:55.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 1 -- 172.21.15.132:0/2351332934 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4434003530 con 0x7f444813b2c0 2024-01-24T23:27:55.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 1 -- 172.21.15.132:0/2351332934 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 157460437 0 0) 0x7f4434003530 con 0x7f444813b2c0 2024-01-24T23:27:55.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 1 -- 172.21.15.132:0/2351332934 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f444811ee40 con 0x7f444813b2c0 2024-01-24T23:27:55.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 1 -- 172.21.15.132:0/2351332934 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f44400017c0 con 0x7f444813b2c0 2024-01-24T23:27:55.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:55.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:27:55.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 10 monclient(hunting): dump: 2024-01-24T23:27:55.430 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:55.431 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:55.431 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:55.431 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:55.431 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:55.431 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:55.431 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:55.431 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:55.431 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:55.431 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:55.431 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:55.431 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:55.431 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:55.431 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:55.431 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:55.431 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:55.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:55.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 1 -- 172.21.15.132:0/2351332934 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4241529803 0 0) 0x7f444811c430 con 0x7f444809f9c0 2024-01-24T23:27:55.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:55.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:55.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:55.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 10 monclient(hunting): my global_id is 85994 2024-01-24T23:27:55.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 1 -- 172.21.15.132:0/2351332934 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4434003530 con 0x7f444809f9c0 2024-01-24T23:27:55.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 1 -- 172.21.15.132:0/2351332934 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2986901885 0 0) 0x7f444811ee40 con 0x7f444813b2c0 2024-01-24T23:27:55.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 1 -- 172.21.15.132:0/2351332934 >> v1:172.21.15.115:6793/0 conn(0x7f444809f9c0 legacy=0x7f444811b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:55.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 1 -- 172.21.15.132:0/2351332934 >> v1:172.21.15.132:6792/0 conn(0x7f444809ff20 legacy=0x7f444811bd20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:55.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:27:55.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 1 monclient: found mon.g 2024-01-24T23:27:55.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:55.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:55.432 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 1 -- 172.21.15.132:0/2351332934 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444811d290 con 0x7f444813b2c0 2024-01-24T23:27:55.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 10 monclient: _finish_auth 0 2024-01-24T23:27:55.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:27:55.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:55.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f444e7b3640 5 monclient: authenticate success, global_id 6807 2024-01-24T23:27:55.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f444e7b3640 10 monclient: _renew_subs 2024-01-24T23:27:55.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f444e7b3640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:55.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f444e7b3640 1 -- 172.21.15.132:0/2351332934 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4448071960 con 0x7f444813b2c0 2024-01-24T23:27:55.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f444e7b3640 10 monclient: _renew_subs 2024-01-24T23:27:55.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f444e7b3640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:55.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f444e7b3640 1 -- 172.21.15.132:0/2351332934 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f444811e740 con 0x7f444813b2c0 2024-01-24T23:27:55.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 1 -- 172.21.15.132:0/2351332934 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4440004c50 con 0x7f444813b2c0 2024-01-24T23:27:55.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:55.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f44467fc640 4 set_mon_vals no callback set 2024-01-24T23:27:55.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 1 -- 172.21.15.132:0/2351332934 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4440001100 con 0x7f444813b2c0 2024-01-24T23:27:55.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:55.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:55.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 10 monclient: dump: 2024-01-24T23:27:55.434 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:55.434 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:55.434 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:55.434 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:55.434 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:55.434 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:55.434 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:55.434 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:55.434 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:55.434 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:55.434 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:55.434 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:55.434 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:55.435 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:55.435 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:55.435 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:55.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f444e7b3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:55.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f444e7b3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:55.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f444e7b3640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:55.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f444e7b3640 1 -- 172.21.15.132:0/2351332934 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4428005090 con 0x7f444813b2c0 2024-01-24T23:27:55.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 1 -- 172.21.15.132:0/2351332934 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f444000fa80 con 0x7f444813b2c0 2024-01-24T23:27:55.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4444ff9640 1 -- 172.21.15.132:0/2351332934 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(225..225 src has 1..225) v4 ==== 11636+0+0 (unknown 3945068259 0 0) 0x7f44400457c0 con 0x7f444813b2c0 2024-01-24T23:27:55.435 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.418+0000 7f4447fff640 10 monclient: get_auth_request con 0x7f4434037050 auth_method 0 2024-01-24T23:27:55.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.422+0000 7f4444ff9640 1 -- 172.21.15.132:0/2351332934 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f4440011020 con 0x7f444813b2c0 2024-01-24T23:27:55.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.422+0000 7f4444ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:55.436 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.422+0000 7f4444ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:55.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.570+0000 7f444e7b3640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h14"}] 2024-01-24T23:27:55.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.570+0000 7f444e7b3640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h14"}] 2024-01-24T23:27:55.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.570+0000 7f444e7b3640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:55.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:55.570+0000 7f444e7b3640 1 -- 172.21.15.132:0/2351332934 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rm", "name": "h14"} v 0) v1 -- 0x7f4428005340 con 0x7f444813b2c0 2024-01-24T23:27:56.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.206+0000 7f4444ff9640 1 -- 172.21.15.132:0/2351332934 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h14"}]=0 removed item id -58 name 'h14' from crush map v226) v1 ==== 120+0+0 (unknown 2081339320 0 0) 0x7f4440013060 con 0x7f444813b2c0 2024-01-24T23:27:56.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.206+0000 7f4444ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h14"}] 2024-01-24T23:27:56.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.206+0000 7f4444ff9640 10 monclient: _finish_command 2 = system:0 removed item id -58 name 'h14' from crush map 2024-01-24T23:27:56.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.302+0000 7f444e7b3640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h14"}] 2024-01-24T23:27:56.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.302+0000 7f444e7b3640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h14"}] 2024-01-24T23:27:56.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.302+0000 7f444e7b3640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:56.302 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.302+0000 7f444e7b3640 1 -- 172.21.15.132:0/2351332934 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rm", "name": "h14"} v 0) v1 -- 0x7f4428002220 con 0x7f444813b2c0 2024-01-24T23:27:56.303 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.302+0000 7f4444ff9640 1 -- 172.21.15.132:0/2351332934 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h14"}]=0 device 'h14' does not appear in the crush map v226) v1 ==== 120+0+0 (unknown 636513093 0 0) 0x7f44400147e0 con 0x7f444813b2c0 2024-01-24T23:27:56.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.302+0000 7f4444ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h14"}] 2024-01-24T23:27:56.304 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.302+0000 7f4444ff9640 10 monclient: _finish_command 3 = system:0 device 'h14' does not appear in the crush map 2024-01-24T23:27:56.304 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h14' does not appear in the crush map 2024-01-24T23:27:56.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.306+0000 7f444e7b3640 1 -- 172.21.15.132:0/2351332934 >> v1:172.21.15.132:6812/1835401922 conn(0x7f4434037050 legacy=0x7f4434039510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:56.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.306+0000 7f444e7b3640 10 monclient: shutdown 2024-01-24T23:27:56.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.306+0000 7f444e7b3640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:56.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.306+0000 7f444e7b3640 1 -- 172.21.15.132:0/2351332934 >> v1:172.21.15.132:6790/0 conn(0x7f444813b2c0 legacy=0x7f44481d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:56.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.306+0000 7f444e7b3640 1 -- 172.21.15.132:0/2351332934 shutdown_connections 2024-01-24T23:27:56.306 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.306+0000 7f444e7b3640 1 -- 172.21.15.132:0/2351332934 >> 172.21.15.132:0/2351332934 conn(0x7f44480a5a10 msgr2=0x7f4448066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:56.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.306+0000 7f444e7b3640 1 -- 172.21.15.132:0/2351332934 shutdown_connections 2024-01-24T23:27:56.307 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.306+0000 7f444e7b3640 1 -- 172.21.15.132:0/2351332934 wait complete. 2024-01-24T23:27:56.321 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:56.321 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h15 2024-01-24T23:27:56.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 10 monclient: get_monmap_and_config 2024-01-24T23:27:56.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 10 monclient: build_initial_monmap 2024-01-24T23:27:56.439 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 10 monclient: monmap: 2024-01-24T23:27:56.439 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:56.439 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:56.439 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:56.441690+0000 2024-01-24T23:27:56.439 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:56.441690+0000 2024-01-24T23:27:56.439 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:56.439 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:56.439 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:56.439 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:56.440 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:56.440 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:56.440 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:56.440 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:56.440 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:56.440 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:56.440 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:56.440 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:56.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 1 Processor -- start 2024-01-24T23:27:56.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 1 -- start start 2024-01-24T23:27:56.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 10 monclient: init 2024-01-24T23:27:56.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:56.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 10 monclient: _add_conns ranks=[1,5,3,2,6,7,8,4,0] 2024-01-24T23:27:56.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 10 monclient(hunting): picked mon.noname-f con 0x7fa09809ff20 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:56.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 10 monclient(hunting): picked mon.noname-h con 0x7fa09809f9c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:56.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 10 monclient(hunting): picked mon.noname-g con 0x7fa09813b2c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:56.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa098064700 con 0x7fa09809ff20 2024-01-24T23:27:56.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa09809e760 con 0x7fa09813b2c0 2024-01-24T23:27:56.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0980a1160 con 0x7fa09809f9c0 2024-01-24T23:27:56.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:56.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 10 monclient(hunting): authenticate will time out at 1363.766303s 2024-01-24T23:27:56.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa096ffd640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fa09809f9c0 0x7fa098139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:56438/0 (socket says 172.21.15.132:56438) 2024-01-24T23:27:56.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa096ffd640 1 -- 172.21.15.132:0/3137026421 learned_addr learned my addr 172.21.15.132:0/3137026421 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:56.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 1 -- 172.21.15.132:0/3137026421 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3691084275 0 0) 0x7fa09809e760 con 0x7fa09813b2c0 2024-01-24T23:27:56.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:56.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:56.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:56.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 10 monclient(hunting): my global_id is 6816 2024-01-24T23:27:56.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 1 -- 172.21.15.132:0/3137026421 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa080003530 con 0x7fa09813b2c0 2024-01-24T23:27:56.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 1 -- 172.21.15.132:0/3137026421 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4018302388 0 0) 0x7fa0980a1160 con 0x7fa09809f9c0 2024-01-24T23:27:56.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:56.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:56.447 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:56.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 10 monclient(hunting): my global_id is 6800 2024-01-24T23:27:56.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 1 -- 172.21.15.132:0/3137026421 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa09809e760 con 0x7fa09809f9c0 2024-01-24T23:27:56.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 1 -- 172.21.15.132:0/3137026421 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3141958329 0 0) 0x7fa098064700 con 0x7fa09809ff20 2024-01-24T23:27:56.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 10 monclient(hunting): _negotiate 2024-01-24T23:27:56.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:56.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:56.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 10 monclient(hunting): my global_id is 124390 2024-01-24T23:27:56.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 1 -- 172.21.15.132:0/3137026421 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0980a1160 con 0x7fa09809ff20 2024-01-24T23:27:56.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 1 -- 172.21.15.132:0/3137026421 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4079173491 0 0) 0x7fa080003530 con 0x7fa09813b2c0 2024-01-24T23:27:56.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 1 -- 172.21.15.132:0/3137026421 >> v1:172.21.15.132:6791/0 conn(0x7fa09809f9c0 legacy=0x7fa098139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:56.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 1 -- 172.21.15.132:0/3137026421 >> v1:172.21.15.132:6789/0 conn(0x7fa09809ff20 legacy=0x7fa0980a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:56.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 10 monclient: _finish_hunting 0 2024-01-24T23:27:56.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 1 monclient: found mon.noname-g 2024-01-24T23:27:56.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:56.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:27:56.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 1 -- 172.21.15.132:0/3137026421 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0980a0a10 con 0x7fa09813b2c0 2024-01-24T23:27:56.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 10 monclient: _finish_auth 0 2024-01-24T23:27:56.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 10 monclient: _check_auth_tickets 2024-01-24T23:27:56.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:56.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 1 -- 172.21.15.132:0/3137026421 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa0900017c0 con 0x7fa09813b2c0 2024-01-24T23:27:56.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:56.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:27:56.449 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 10 monclient: dump: 2024-01-24T23:27:56.449 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:56.449 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:56.449 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:56.449 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:56.450 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:56.450 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:56.450 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:56.450 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:56.450 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:56.450 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:56.450 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:56.450 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:56.450 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:56.450 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:56.450 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:56.450 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:56.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 5 monclient: authenticate success, global_id 6816 2024-01-24T23:27:56.450 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:56.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 1 -- 172.21.15.132:0/3137026421 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa090001aa0 con 0x7fa09813b2c0 2024-01-24T23:27:56.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:56.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:56.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 1 -- 172.21.15.132:0/3137026421 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa090003960 con 0x7fa09813b2c0 2024-01-24T23:27:56.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:56.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 4 set_mon_vals no callback set 2024-01-24T23:27:56.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:56.451 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa0967fc640 10 monclient: dump: 2024-01-24T23:27:56.451 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:56.451 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:56.451 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:56.451 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:56.451 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:56.451 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:56.451 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:56.452 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:56.452 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:56.452 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:56.452 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:56.452 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:56.452 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:56.452 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:56.452 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:56.452 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:56.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 10 monclient: shutdown 2024-01-24T23:27:56.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:56.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 1 -- 172.21.15.132:0/3137026421 >> v1:172.21.15.132:6790/0 conn(0x7fa09813b2c0 legacy=0x7fa09813d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:56.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 1 -- 172.21.15.132:0/3137026421 shutdown_connections 2024-01-24T23:27:56.452 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 1 -- 172.21.15.132:0/3137026421 >> 172.21.15.132:0/3137026421 conn(0x7fa0980a5a10 msgr2=0x7fa0980a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:56.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.438+0000 7fa09e17c640 1 -- 172.21.15.132:0/3137026421 shutdown_connections 2024-01-24T23:27:56.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 1 -- 172.21.15.132:0/3137026421 wait complete. 2024-01-24T23:27:56.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 10 monclient: build_initial_monmap 2024-01-24T23:27:56.453 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 10 monclient: monmap: 2024-01-24T23:27:56.453 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:56.453 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:56.453 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:56.453 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:56.453 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:56.453 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:56.453 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:56.453 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:56.453 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:56.453 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:56.453 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:56.454 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:56.454 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:56.454 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:56.454 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:56.454 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:56.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 1 Processor -- start 2024-01-24T23:27:56.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 1 -- start start 2024-01-24T23:27:56.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 10 monclient: init 2024-01-24T23:27:56.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:56.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 10 monclient: _add_conns ranks=[2,0,5,7,4,3,8,1,6] 2024-01-24T23:27:56.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 10 monclient(hunting): picked mon.noname-b con 0x7fa09809f9c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:27:56.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 10 monclient(hunting): picked mon.noname-a con 0x7fa09809ff20 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:56.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 10 monclient(hunting): picked mon.noname-h con 0x7fa09813b2c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:56.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa09811fe10 con 0x7fa09809ff20 2024-01-24T23:27:56.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa098132b60 con 0x7fa09809f9c0 2024-01-24T23:27:56.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa09809ca30 con 0x7fa09813b2c0 2024-01-24T23:27:56.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:56.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 10 monclient(hunting): authenticate will time out at 1363.769431s 2024-01-24T23:27:56.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa097fff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fa09813b2c0 0x7fa0981d3dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:56440/0 (socket says 172.21.15.132:56440) 2024-01-24T23:27:56.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa097fff640 1 -- 172.21.15.132:0/1607134429 learned_addr learned my addr 172.21.15.132:0/1607134429 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:56.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa0977fe640 1 -- 172.21.15.132:0/1607134429 >> v1:172.21.15.115:6790/0 conn(0x7fa09809f9c0 legacy=0x7fa098132450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:27:56.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa096ffd640 1 -- 172.21.15.132:0/1607134429 >> v1:172.21.15.115:6789/0 conn(0x7fa09809ff20 legacy=0x7fa0981d06a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:27:56.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 1 -- 172.21.15.132:0/1607134429 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 563098915 0 0) 0x7fa09809ca30 con 0x7fa09813b2c0 2024-01-24T23:27:56.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:27:56.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:56.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:56.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 10 monclient(hunting): my global_id is 6809 2024-01-24T23:27:56.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 1 -- 172.21.15.132:0/1607134429 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa084003530 con 0x7fa09813b2c0 2024-01-24T23:27:56.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 1 -- 172.21.15.132:0/1607134429 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 848479975 0 0) 0x7fa084003530 con 0x7fa09813b2c0 2024-01-24T23:27:56.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 1 -- 172.21.15.132:0/1607134429 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa09809ca30 con 0x7fa09813b2c0 2024-01-24T23:27:56.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 1 -- 172.21.15.132:0/1607134429 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa090001780 con 0x7fa09813b2c0 2024-01-24T23:27:56.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:56.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:27:56.456 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 10 monclient(hunting): dump: 2024-01-24T23:27:56.456 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:56.456 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:56.456 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:56.456 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:56.456 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:56.456 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:56.456 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:56.456 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:56.457 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:56.457 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:56.457 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:56.457 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:56.457 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:56.457 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:56.457 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:56.457 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:56.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:56.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 1 -- 172.21.15.132:0/1607134429 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3804479404 0 0) 0x7fa09809ca30 con 0x7fa09813b2c0 2024-01-24T23:27:56.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 1 -- 172.21.15.132:0/1607134429 >> v1:172.21.15.115:6790/0 conn(0x7fa09809f9c0 legacy=0x7fa098132450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:56.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 1 -- 172.21.15.132:0/1607134429 >> v1:172.21.15.115:6789/0 conn(0x7fa09809ff20 legacy=0x7fa0981d06a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:56.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 10 monclient: _finish_hunting 0 2024-01-24T23:27:56.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 1 monclient: found mon.h 2024-01-24T23:27:56.457 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:56.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:56.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 1 -- 172.21.15.132:0/1607134429 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa098071960 con 0x7fa09813b2c0 2024-01-24T23:27:56.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 10 monclient: _finish_auth 0 2024-01-24T23:27:56.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 10 monclient: _check_auth_tickets 2024-01-24T23:27:56.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:56.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 5 monclient: authenticate success, global_id 6809 2024-01-24T23:27:56.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 10 monclient: _renew_subs 2024-01-24T23:27:56.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:56.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 1 -- 172.21.15.132:0/1607134429 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa098071680 con 0x7fa09813b2c0 2024-01-24T23:27:56.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 10 monclient: _renew_subs 2024-01-24T23:27:56.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:56.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 1 -- 172.21.15.132:0/1607134429 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09809d7f0 con 0x7fa09813b2c0 2024-01-24T23:27:56.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 1 -- 172.21.15.132:0/1607134429 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa090001cb0 con 0x7fa09813b2c0 2024-01-24T23:27:56.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:56.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 1 -- 172.21.15.132:0/1607134429 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa090004180 con 0x7fa09813b2c0 2024-01-24T23:27:56.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:56.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:56.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa095ffb640 4 set_mon_vals no callback set 2024-01-24T23:27:56.459 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 10 monclient: dump: 2024-01-24T23:27:56.459 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:56.459 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:56.459 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:56.459 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:56.459 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:56.459 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:56.459 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:56.459 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:56.459 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:56.459 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:56.460 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:56.460 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:56.460 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:56.460 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:56.460 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:56.460 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:56.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:56.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 1 -- 172.21.15.132:0/1607134429 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fa0900042e0 con 0x7fa09813b2c0 2024-01-24T23:27:56.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:56.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:56.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa09e17c640 1 -- 172.21.15.132:0/1607134429 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa078005090 con 0x7fa09813b2c0 2024-01-24T23:27:56.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa07ffff640 1 -- 172.21.15.132:0/1607134429 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(226..226 src has 1..226) v4 ==== 11423+0+0 (unknown 1131679035 0 0) 0x7fa0900457d0 con 0x7fa09813b2c0 2024-01-24T23:27:56.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.442+0000 7fa0977fe640 10 monclient: get_auth_request con 0x7fa084036e70 auth_method 0 2024-01-24T23:27:56.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.446+0000 7fa07ffff640 1 -- 172.21.15.132:0/1607134429 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fa090014aa0 con 0x7fa09813b2c0 2024-01-24T23:27:56.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.446+0000 7fa07ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:56.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.446+0000 7fa07ffff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:56.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.598+0000 7fa09e17c640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h15"}] 2024-01-24T23:27:56.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.598+0000 7fa09e17c640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h15"}] 2024-01-24T23:27:56.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.598+0000 7fa09e17c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:56.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.598+0000 7fa09e17c640 1 -- 172.21.15.132:0/1607134429 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h15"} v 0) v1 -- 0x7fa078005340 con 0x7fa09813b2c0 2024-01-24T23:27:56.931 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.930+0000 7fa07ffff640 1 -- 172.21.15.132:0/1607134429 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h15"}]=0 removed item id -61 name 'h15' from crush map v227) v1 ==== 120+0+0 (unknown 2331712592 0 0) 0x7fa09001d170 con 0x7fa09813b2c0 2024-01-24T23:27:56.931 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.930+0000 7fa07ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h15"}] 2024-01-24T23:27:56.931 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:56.930+0000 7fa07ffff640 10 monclient: _finish_command 2 = system:0 removed item id -61 name 'h15' from crush map 2024-01-24T23:27:56.965 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~113s 2024-01-24T23:27:57.038 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.038+0000 7fa09e17c640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h15"}] 2024-01-24T23:27:57.038 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.038+0000 7fa09e17c640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h15"}] 2024-01-24T23:27:57.038 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.038+0000 7fa09e17c640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:27:57.038 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.038+0000 7fa09e17c640 1 -- 172.21.15.132:0/1607134429 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h15"} v 0) v1 -- 0x7fa0780010a0 con 0x7fa09813b2c0 2024-01-24T23:27:57.039 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.038+0000 7fa07ffff640 1 -- 172.21.15.132:0/1607134429 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h15"}]=0 device 'h15' does not appear in the crush map v227) v1 ==== 120+0+0 (unknown 2445820906 0 0) 0x7fa090013140 con 0x7fa09813b2c0 2024-01-24T23:27:57.039 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.038+0000 7fa07ffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h15"}] 2024-01-24T23:27:57.039 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.038+0000 7fa07ffff640 10 monclient: _finish_command 3 = system:0 device 'h15' does not appear in the crush map 2024-01-24T23:27:57.039 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h15' does not appear in the crush map 2024-01-24T23:27:57.041 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.038+0000 7fa09e17c640 1 -- 172.21.15.132:0/1607134429 >> v1:172.21.15.132:6812/1835401922 conn(0x7fa084036e70 legacy=0x7fa084039330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:57.041 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.038+0000 7fa09e17c640 10 monclient: shutdown 2024-01-24T23:27:57.041 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.038+0000 7fa09e17c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:57.041 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.038+0000 7fa09e17c640 1 -- 172.21.15.132:0/1607134429 >> v1:172.21.15.132:6791/0 conn(0x7fa09813b2c0 legacy=0x7fa0981d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:57.041 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.038+0000 7fa09e17c640 1 -- 172.21.15.132:0/1607134429 shutdown_connections 2024-01-24T23:27:57.041 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.038+0000 7fa09e17c640 1 -- 172.21.15.132:0/1607134429 >> 172.21.15.132:0/1607134429 conn(0x7fa0980a5a10 msgr2=0x7fa0980a2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:57.041 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.038+0000 7fa09e17c640 1 -- 172.21.15.132:0/1607134429 shutdown_connections 2024-01-24T23:27:57.041 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.038+0000 7fa09e17c640 1 -- 172.21.15.132:0/1607134429 wait complete. 2024-01-24T23:27:57.056 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:57.056 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h16 2024-01-24T23:27:57.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.170+0000 7f8714f61640 10 monclient: get_monmap_and_config 2024-01-24T23:27:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.170+0000 7f8714f61640 10 monclient: build_initial_monmap 2024-01-24T23:27:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.170+0000 7f8714f61640 10 monclient: monmap: 2024-01-24T23:27:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:57.175415+0000 2024-01-24T23:27:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:57.175415+0000 2024-01-24T23:27:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:57.173 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:57.174 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:57.174 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:57.174 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:57.174 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:57.174 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:57.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.170+0000 7f8714f61640 1 Processor -- start 2024-01-24T23:27:57.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 1 -- start start 2024-01-24T23:27:57.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 10 monclient: init 2024-01-24T23:27:57.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:57.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 10 monclient: _add_conns ranks=[5,0,8,3,6,2,7,1,4] 2024-01-24T23:27:57.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 10 monclient(hunting): picked mon.noname-h con 0x7f871006a4f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:27:57.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 10 monclient(hunting): picked mon.noname-a con 0x7f87100a11e0 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:57.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 10 monclient(hunting): picked mon.noname-e con 0x7f87100a15e0 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:57.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8710064700 con 0x7f87100a11e0 2024-01-24T23:27:57.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871006a8d0 con 0x7f871006a4f0 2024-01-24T23:27:57.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871009bae0 con 0x7f87100a15e0 2024-01-24T23:27:57.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:57.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 10 monclient(hunting): authenticate will time out at 1364.500813s 2024-01-24T23:27:57.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870e59c640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f871006a4f0 0x7f87100a0ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:56444/0 (socket says 172.21.15.132:56444) 2024-01-24T23:27:57.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870e59c640 1 -- 172.21.15.132:0/1058016150 learned_addr learned my addr 172.21.15.132:0/1058016150 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:57.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870dd9b640 1 -- 172.21.15.132:0/1058016150 >> v1:172.21.15.115:6789/0 conn(0x7f87100a11e0 legacy=0x7f871013c280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:27:57.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 1 -- 172.21.15.132:0/1058016150 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1883430671 0 0) 0x7f871006a8d0 con 0x7f871006a4f0 2024-01-24T23:27:57.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 10 monclient(hunting): _negotiate 2024-01-24T23:27:57.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:57.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:57.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 10 monclient(hunting): my global_id is 6818 2024-01-24T23:27:57.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 1 -- 172.21.15.132:0/1058016150 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86f8003530 con 0x7f871006a4f0 2024-01-24T23:27:57.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 1 -- 172.21.15.132:0/1058016150 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2666517331 0 0) 0x7f86f8003530 con 0x7f871006a4f0 2024-01-24T23:27:57.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 1 -- 172.21.15.132:0/1058016150 >> v1:172.21.15.115:6793/0 conn(0x7f87100a15e0 legacy=0x7f871013f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:57.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 1 -- 172.21.15.132:0/1058016150 >> v1:172.21.15.115:6789/0 conn(0x7f87100a11e0 legacy=0x7f871013c280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:57.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 10 monclient: _finish_hunting 0 2024-01-24T23:27:57.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 1 monclient: found mon.noname-h 2024-01-24T23:27:57.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:57.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:27:57.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 1 -- 172.21.15.132:0/1058016150 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871009a3f0 con 0x7f871006a4f0 2024-01-24T23:27:57.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 10 monclient: _finish_auth 0 2024-01-24T23:27:57.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 10 monclient: _check_auth_tickets 2024-01-24T23:27:57.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:57.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 1 -- 172.21.15.132:0/1058016150 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8700001a70 con 0x7f871006a4f0 2024-01-24T23:27:57.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 5 monclient: authenticate success, global_id 6818 2024-01-24T23:27:57.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:57.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:57.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:27:57.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 10 monclient: dump: 2024-01-24T23:27:57.183 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:57.183 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:57.183 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:57.183 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:57.183 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:57.183 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:57.183 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:57.183 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:57.184 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:57.184 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:57.184 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:57.184 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:57.184 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:57.184 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:57.184 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:57.184 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:57.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:57.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 1 -- 172.21.15.132:0/1058016150 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8700001d90 con 0x7f871006a4f0 2024-01-24T23:27:57.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:57.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 1 -- 172.21.15.132:0/1058016150 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8700004f70 con 0x7f871006a4f0 2024-01-24T23:27:57.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:57.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 4 set_mon_vals no callback set 2024-01-24T23:27:57.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:27:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870d59a640 10 monclient: dump: 2024-01-24T23:27:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:57.185 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:57.186 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:57.186 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:57.186 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:57.186 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:57.186 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:57.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 10 monclient: shutdown 2024-01-24T23:27:57.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:57.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 1 -- 172.21.15.132:0/1058016150 >> v1:172.21.15.132:6791/0 conn(0x7f871006a4f0 legacy=0x7f87100a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:57.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 1 -- 172.21.15.132:0/1058016150 shutdown_connections 2024-01-24T23:27:57.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 1 -- 172.21.15.132:0/1058016150 >> 172.21.15.132:0/1058016150 conn(0x7f87100a60d0 msgr2=0x7f87100a64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:57.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 1 -- 172.21.15.132:0/1058016150 shutdown_connections 2024-01-24T23:27:57.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 1 -- 172.21.15.132:0/1058016150 wait complete. 2024-01-24T23:27:57.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 10 monclient: build_initial_monmap 2024-01-24T23:27:57.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 10 monclient: monmap: 2024-01-24T23:27:57.186 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:57.187 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:57.187 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:57.187 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:57.187 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:57.187 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:57.187 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:57.187 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:57.187 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:57.187 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:57.187 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:57.187 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:57.187 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:57.187 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:57.187 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:57.188 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:57.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 1 Processor -- start 2024-01-24T23:27:57.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 1 -- start start 2024-01-24T23:27:57.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 10 monclient: init 2024-01-24T23:27:57.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:57.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 10 monclient: _add_conns ranks=[6,7,3,8,5,1,2,0,4] 2024-01-24T23:27:57.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 10 monclient(hunting): picked mon.noname-d con 0x7f871006a4f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:57.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 10 monclient(hunting): picked mon.noname-i con 0x7f87100a11e0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:57.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 10 monclient(hunting): picked mon.noname-g con 0x7f87100a15e0 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:57.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8710070b80 con 0x7f87100a15e0 2024-01-24T23:27:57.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8710071e30 con 0x7f871006a4f0 2024-01-24T23:27:57.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87101b07d0 con 0x7f87100a11e0 2024-01-24T23:27:57.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:57.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f8714f61640 10 monclient(hunting): authenticate will time out at 1364.503797s 2024-01-24T23:27:57.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870dd9b640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f87100a11e0 legacy=0x7f87101b8040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:57.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870ed9d640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f87100a15e0 0x7f87101d3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:41930/0 (socket says 172.21.15.132:41930) 2024-01-24T23:27:57.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.174+0000 7f870ed9d640 1 -- 172.21.15.132:0/2190023028 learned_addr learned my addr 172.21.15.132:0/2190023028 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:57.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 1 -- 172.21.15.132:0/2190023028 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456895668 0 0) 0x7f8710070b80 con 0x7f87100a15e0 2024-01-24T23:27:57.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:57.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:57.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:57.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 10 monclient(hunting): my global_id is 6825 2024-01-24T23:27:57.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 1 -- 172.21.15.132:0/2190023028 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86f0003530 con 0x7f87100a15e0 2024-01-24T23:27:57.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 1 -- 172.21.15.132:0/2190023028 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4237799099 0 0) 0x7f86f0003530 con 0x7f87100a15e0 2024-01-24T23:27:57.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 1 -- 172.21.15.132:0/2190023028 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8710070b80 con 0x7f87100a15e0 2024-01-24T23:27:57.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 1 -- 172.21.15.132:0/2190023028 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8708001aa0 con 0x7f87100a15e0 2024-01-24T23:27:57.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:57.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:27:57.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 10 monclient(hunting): dump: 2024-01-24T23:27:57.190 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:57.190 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:57.190 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:57.190 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:57.190 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:57.190 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:57.190 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:57.190 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:57.190 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:57.190 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:57.190 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:57.190 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:57.190 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:57.190 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:57.190 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:57.190 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:57.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:57.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 1 -- 172.21.15.132:0/2190023028 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909984175 0 0) 0x7f8710071e30 con 0x7f871006a4f0 2024-01-24T23:27:57.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:57.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:57.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:57.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 10 monclient(hunting): my global_id is 7017 2024-01-24T23:27:57.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 1 -- 172.21.15.132:0/2190023028 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86f0003530 con 0x7f871006a4f0 2024-01-24T23:27:57.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 1 -- 172.21.15.132:0/2190023028 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3666692564 0 0) 0x7f8710070b80 con 0x7f87100a15e0 2024-01-24T23:27:57.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 1 -- 172.21.15.132:0/2190023028 >> v1:172.21.15.132:6792/0 conn(0x7f87100a11e0 legacy=0x7f87101b8040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:57.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 1 -- 172.21.15.132:0/2190023028 >> v1:172.21.15.115:6792/0 conn(0x7f871006a4f0 legacy=0x7f871006fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:57.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 10 monclient: _finish_hunting 0 2024-01-24T23:27:57.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 1 monclient: found mon.g 2024-01-24T23:27:57.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:57.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:57.191 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 1 -- 172.21.15.132:0/2190023028 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87101b0ab0 con 0x7f87100a15e0 2024-01-24T23:27:57.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 10 monclient: _finish_auth 0 2024-01-24T23:27:57.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 10 monclient: _check_auth_tickets 2024-01-24T23:27:57.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:57.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f8714f61640 5 monclient: authenticate success, global_id 6825 2024-01-24T23:27:57.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f8714f61640 10 monclient: _renew_subs 2024-01-24T23:27:57.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f8714f61640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:57.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f8714f61640 1 -- 172.21.15.132:0/2190023028 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87100717b0 con 0x7f87100a15e0 2024-01-24T23:27:57.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f8714f61640 10 monclient: _renew_subs 2024-01-24T23:27:57.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f8714f61640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:57.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f8714f61640 1 -- 172.21.15.132:0/2190023028 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f87100712f0 con 0x7f87100a15e0 2024-01-24T23:27:57.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 1 -- 172.21.15.132:0/2190023028 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8708003950 con 0x7f87100a15e0 2024-01-24T23:27:57.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:57.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 1 -- 172.21.15.132:0/2190023028 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8708003e80 con 0x7f87100a15e0 2024-01-24T23:27:57.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:57.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:57.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f870cd99640 4 set_mon_vals no callback set 2024-01-24T23:27:57.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 10 monclient: dump: 2024-01-24T23:27:57.193 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:57.193 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:57.193 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:57.193 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:57.193 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:57.193 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:57.193 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:57.193 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:57.193 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:57.193 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:57.194 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:57.194 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:57.194 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:57.194 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:57.194 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:57.194 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:57.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f8714f61640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:57.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f8714f61640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:57.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f8714f61640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:57.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f8714f61640 1 -- 172.21.15.132:0/2190023028 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f4005090 con 0x7f87100a15e0 2024-01-24T23:27:57.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 1 -- 172.21.15.132:0/2190023028 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f8708004060 con 0x7f87100a15e0 2024-01-24T23:27:57.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f86feffd640 1 -- 172.21.15.132:0/2190023028 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(227..227 src has 1..227) v4 ==== 11210+0+0 (unknown 3748986923 0 0) 0x7f87080456a0 con 0x7f87100a15e0 2024-01-24T23:27:57.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.178+0000 7f870e59c640 10 monclient: get_auth_request con 0x7f86f0036f70 auth_method 0 2024-01-24T23:27:57.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.182+0000 7f86feffd640 1 -- 172.21.15.132:0/2190023028 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f8708014c10 con 0x7f87100a15e0 2024-01-24T23:27:57.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.182+0000 7f86feffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:57.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.182+0000 7f86feffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:57.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.338+0000 7f8714f61640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h16"}] 2024-01-24T23:27:57.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.338+0000 7f8714f61640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h16"}] 2024-01-24T23:27:57.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.338+0000 7f8714f61640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:57.338 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:57.338+0000 7f8714f61640 1 -- 172.21.15.132:0/2190023028 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rm", "name": "h16"} v 0) v1 -- 0x7f86f4005340 con 0x7f87100a15e0 2024-01-24T23:27:58.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.226+0000 7f86feffd640 1 -- 172.21.15.132:0/2190023028 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h16"}]=0 removed item id -64 name 'h16' from crush map v228) v1 ==== 120+0+0 (unknown 913721447 0 0) 0x7f8708013080 con 0x7f87100a15e0 2024-01-24T23:27:58.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.226+0000 7f86feffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h16"}] 2024-01-24T23:27:58.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.226+0000 7f86feffd640 10 monclient: _finish_command 2 = system:0 removed item id -64 name 'h16' from crush map 2024-01-24T23:27:58.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.322+0000 7f8714f61640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h16"}] 2024-01-24T23:27:58.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.322+0000 7f8714f61640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h16"}] 2024-01-24T23:27:58.323 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.322+0000 7f8714f61640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:58.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.322+0000 7f8714f61640 1 -- 172.21.15.132:0/2190023028 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rm", "name": "h16"} v 0) v1 -- 0x7f86f40010a0 con 0x7f87100a15e0 2024-01-24T23:27:58.324 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.322+0000 7f86feffd640 1 -- 172.21.15.132:0/2190023028 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h16"}]=0 device 'h16' does not appear in the crush map v228) v1 ==== 120+0+0 (unknown 2088881826 0 0) 0x7f8708014800 con 0x7f87100a15e0 2024-01-24T23:27:58.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.322+0000 7f86feffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h16"}] 2024-01-24T23:27:58.325 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.322+0000 7f86feffd640 10 monclient: _finish_command 3 = system:0 device 'h16' does not appear in the crush map 2024-01-24T23:27:58.325 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h16' does not appear in the crush map 2024-01-24T23:27:58.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.326+0000 7f8714f61640 1 -- 172.21.15.132:0/2190023028 >> v1:172.21.15.132:6812/1835401922 conn(0x7f86f0036f70 legacy=0x7f86f0039430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:58.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.326+0000 7f8714f61640 10 monclient: shutdown 2024-01-24T23:27:58.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.326+0000 7f8714f61640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:58.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.326+0000 7f8714f61640 1 -- 172.21.15.132:0/2190023028 >> v1:172.21.15.132:6790/0 conn(0x7f87100a15e0 legacy=0x7f87101d3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:58.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.326+0000 7f8714f61640 1 -- 172.21.15.132:0/2190023028 shutdown_connections 2024-01-24T23:27:58.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.326+0000 7f8714f61640 1 -- 172.21.15.132:0/2190023028 >> 172.21.15.132:0/2190023028 conn(0x7f87100a60d0 msgr2=0x7f87101bc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:58.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.326+0000 7f8714f61640 1 -- 172.21.15.132:0/2190023028 shutdown_connections 2024-01-24T23:27:58.327 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.326+0000 7f8714f61640 1 -- 172.21.15.132:0/2190023028 wait complete. 2024-01-24T23:27:58.341 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:58.341 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h17 2024-01-24T23:27:58.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.458+0000 7f236d355640 10 monclient: get_monmap_and_config 2024-01-24T23:27:58.460 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.458+0000 7f236d355640 10 monclient: build_initial_monmap 2024-01-24T23:27:58.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.458+0000 7f236d355640 10 monclient: monmap: 2024-01-24T23:27:58.461 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:58.461 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:58.461 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:58.462943+0000 2024-01-24T23:27:58.461 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:58.462943+0000 2024-01-24T23:27:58.461 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:58.461 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:58.461 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:58.461 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:58.461 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:58.461 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:58.461 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:58.461 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:58.461 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:58.462 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:58.462 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:58.462 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:58.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.458+0000 7f236d355640 1 Processor -- start 2024-01-24T23:27:58.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.458+0000 7f236d355640 1 -- start start 2024-01-24T23:27:58.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.458+0000 7f236d355640 10 monclient: init 2024-01-24T23:27:58.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:58.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 10 monclient: _add_conns ranks=[3,6,0,5,7,1,8,2,4] 2024-01-24T23:27:58.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 10 monclient(hunting): picked mon.noname-g con 0x7f2368135e10 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:58.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 10 monclient(hunting): picked mon.noname-d con 0x7f23681361f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:58.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 10 monclient(hunting): picked mon.noname-a con 0x7f23681365f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:58.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2368064700 con 0x7f23681365f0 2024-01-24T23:27:58.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236809c3e0 con 0x7f2368135e10 2024-01-24T23:27:58.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23680a4f90 con 0x7f23681361f0 2024-01-24T23:27:58.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:58.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 10 monclient(hunting): authenticate will time out at 1365.788445s 2024-01-24T23:27:58.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2366ffd640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f2368135e10 0x7f236809bcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:41936/0 (socket says 172.21.15.132:41936) 2024-01-24T23:27:58.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2366ffd640 1 -- 172.21.15.132:0/213984103 learned_addr learned my addr 172.21.15.132:0/213984103 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:58.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f23677fe640 1 -- 172.21.15.132:0/213984103 >> v1:172.21.15.115:6789/0 conn(0x7f23681365f0 legacy=0x7f236813f600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:27:58.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 1 -- 172.21.15.132:0/213984103 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708403076 0 0) 0x7f236809c3e0 con 0x7f2368135e10 2024-01-24T23:27:58.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:27:58.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:58.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:58.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 10 monclient(hunting): my global_id is 6834 2024-01-24T23:27:58.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 1 -- 172.21.15.132:0/213984103 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2350003530 con 0x7f2368135e10 2024-01-24T23:27:58.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 1 -- 172.21.15.132:0/213984103 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 729556669 0 0) 0x7f2350003530 con 0x7f2368135e10 2024-01-24T23:27:58.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 1 -- 172.21.15.132:0/213984103 >> v1:172.21.15.115:6792/0 conn(0x7f23681361f0 legacy=0x7f236813bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:58.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 1 -- 172.21.15.132:0/213984103 >> v1:172.21.15.115:6789/0 conn(0x7f23681365f0 legacy=0x7f236813f600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:58.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:27:58.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 1 monclient: found mon.noname-g 2024-01-24T23:27:58.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:58.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:27:58.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 1 -- 172.21.15.132:0/213984103 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236809b010 con 0x7f2368135e10 2024-01-24T23:27:58.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 10 monclient: _finish_auth 0 2024-01-24T23:27:58.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:27:58.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:58.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 1 -- 172.21.15.132:0/213984103 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f23580017c0 con 0x7f2368135e10 2024-01-24T23:27:58.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 5 monclient: authenticate success, global_id 6834 2024-01-24T23:27:58.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:58.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:58.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:27:58.467 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 10 monclient: dump: 2024-01-24T23:27:58.467 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:58.467 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:58.467 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:58.467 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:58.467 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:58.468 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:58.468 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:58.468 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:58.468 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:58.468 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:58.468 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:58.468 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:58.468 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:58.468 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:58.468 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:58.468 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:58.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:58.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 1 -- 172.21.15.132:0/213984103 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2358001ae0 con 0x7f2368135e10 2024-01-24T23:27:58.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:58.468 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:58.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 4 set_mon_vals no callback set 2024-01-24T23:27:58.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 1 -- 172.21.15.132:0/213984103 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2358004bf0 con 0x7f2368135e10 2024-01-24T23:27:58.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:58.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:58.469 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f2365ffb640 10 monclient: dump: 2024-01-24T23:27:58.469 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:58.469 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:58.469 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:58.469 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:58.469 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:58.469 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:58.469 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:58.469 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:58.469 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:58.469 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:58.470 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:58.470 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:58.470 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:58.470 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:58.470 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:58.470 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:58.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 10 monclient: shutdown 2024-01-24T23:27:58.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:58.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 1 -- 172.21.15.132:0/213984103 >> v1:172.21.15.132:6790/0 conn(0x7f2368135e10 legacy=0x7f236809bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:58.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 1 -- 172.21.15.132:0/213984103 shutdown_connections 2024-01-24T23:27:58.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 1 -- 172.21.15.132:0/213984103 >> 172.21.15.132:0/213984103 conn(0x7f23680a6790 msgr2=0x7f2368071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:58.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 1 -- 172.21.15.132:0/213984103 shutdown_connections 2024-01-24T23:27:58.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 1 -- 172.21.15.132:0/213984103 wait complete. 2024-01-24T23:27:58.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 10 monclient: build_initial_monmap 2024-01-24T23:27:58.470 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 10 monclient: monmap: 2024-01-24T23:27:58.471 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:58.471 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:58.471 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:58.471 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:58.471 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:58.471 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:58.471 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:58.471 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:58.471 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:58.471 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:58.471 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:58.471 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:58.471 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:58.471 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:58.471 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:58.471 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:58.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 1 Processor -- start 2024-01-24T23:27:58.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 1 -- start start 2024-01-24T23:27:58.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 10 monclient: init 2024-01-24T23:27:58.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:58.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 10 monclient: _add_conns ranks=[6,0,3,5,7,2,8,1,4] 2024-01-24T23:27:58.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 10 monclient(hunting): picked mon.noname-d con 0x7f2368135e10 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:58.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 10 monclient(hunting): picked mon.noname-a con 0x7f23681361f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:58.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 10 monclient(hunting): picked mon.noname-g con 0x7f23681365f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:27:58.475 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2368122510 con 0x7f23681361f0 2024-01-24T23:27:58.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23681bf430 con 0x7f23681365f0 2024-01-24T23:27:58.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2368121f50 con 0x7f2368135e10 2024-01-24T23:27:58.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:58.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f236d355640 10 monclient(hunting): authenticate will time out at 1365.791613s 2024-01-24T23:27:58.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f23677fe640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f23681365f0 0x7f23681d3b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:41948/0 (socket says 172.21.15.132:41948) 2024-01-24T23:27:58.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f23667fc640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f23681361f0 legacy=0x7f23681d03d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:27:58.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f23677fe640 1 -- 172.21.15.132:0/2314469194 learned_addr learned my addr 172.21.15.132:0/2314469194 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:58.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f234f7fe640 1 -- 172.21.15.132:0/2314469194 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 949890744 0 0) 0x7f23681bf430 con 0x7f23681365f0 2024-01-24T23:27:58.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f234f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:58.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f234f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:58.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f234f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:58.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f234f7fe640 10 monclient(hunting): my global_id is 6843 2024-01-24T23:27:58.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.462+0000 7f234f7fe640 1 -- 172.21.15.132:0/2314469194 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2354003530 con 0x7f23681365f0 2024-01-24T23:27:58.476 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 1 -- 172.21.15.132:0/2314469194 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2147899622 0 0) 0x7f2354003530 con 0x7f23681365f0 2024-01-24T23:27:58.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 1 -- 172.21.15.132:0/2314469194 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f23681bf430 con 0x7f23681365f0 2024-01-24T23:27:58.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 1 -- 172.21.15.132:0/2314469194 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2360004570 con 0x7f23681365f0 2024-01-24T23:27:58.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:58.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:27:58.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 10 monclient(hunting): dump: 2024-01-24T23:27:58.477 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:58.477 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:58.477 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:58.477 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:58.477 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:58.477 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:58.477 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:58.477 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:58.477 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:58.477 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:58.478 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:58.478 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:58.478 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:58.478 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:58.478 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:58.478 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:58.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:58.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 1 -- 172.21.15.132:0/2314469194 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 842970835 0 0) 0x7f2368121f50 con 0x7f2368135e10 2024-01-24T23:27:58.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:27:58.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:58.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:58.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 10 monclient(hunting): my global_id is 7035 2024-01-24T23:27:58.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 1 -- 172.21.15.132:0/2314469194 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2354003530 con 0x7f2368135e10 2024-01-24T23:27:58.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 1 -- 172.21.15.132:0/2314469194 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1158916412 0 0) 0x7f23681bf430 con 0x7f23681365f0 2024-01-24T23:27:58.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 1 -- 172.21.15.132:0/2314469194 >> v1:172.21.15.115:6792/0 conn(0x7f2368135e10 legacy=0x7f2368134e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:58.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 1 -- 172.21.15.132:0/2314469194 >> v1:172.21.15.115:6789/0 conn(0x7f23681361f0 legacy=0x7f23681d03d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:58.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:27:58.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 1 monclient: found mon.g 2024-01-24T23:27:58.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:58.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:58.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 1 -- 172.21.15.132:0/2314469194 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2368122230 con 0x7f23681365f0 2024-01-24T23:27:58.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 10 monclient: _finish_auth 0 2024-01-24T23:27:58.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:27:58.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:58.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f236d355640 5 monclient: authenticate success, global_id 6843 2024-01-24T23:27:58.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f236d355640 10 monclient: _renew_subs 2024-01-24T23:27:58.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f236d355640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:58.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f236d355640 1 -- 172.21.15.132:0/2314469194 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23681beb30 con 0x7f23681365f0 2024-01-24T23:27:58.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f236d355640 10 monclient: _renew_subs 2024-01-24T23:27:58.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f236d355640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:58.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f236d355640 1 -- 172.21.15.132:0/2314469194 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f23681211c0 con 0x7f23681365f0 2024-01-24T23:27:58.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 1 -- 172.21.15.132:0/2314469194 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2360004860 con 0x7f23681365f0 2024-01-24T23:27:58.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:58.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 1 -- 172.21.15.132:0/2314469194 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2360004d90 con 0x7f23681365f0 2024-01-24T23:27:58.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:58.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f23657fa640 4 set_mon_vals no callback set 2024-01-24T23:27:58.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:27:58.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 10 monclient: dump: 2024-01-24T23:27:58.480 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:58.480 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:58.480 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:58.480 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:58.480 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:58.480 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:58.480 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:58.480 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:58.481 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:58.481 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:58.481 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:58.481 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:58.481 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:58.481 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:58.481 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:58.481 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:58.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f236d355640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:58.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f236d355640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:58.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f236d355640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:58.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f236d355640 1 -- 172.21.15.132:0/2314469194 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2348005090 con 0x7f23681365f0 2024-01-24T23:27:58.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 1 -- 172.21.15.132:0/2314469194 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f2360004ef0 con 0x7f23681365f0 2024-01-24T23:27:58.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f234f7fe640 1 -- 172.21.15.132:0/2314469194 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(228..228 src has 1..228) v4 ==== 10997+0+0 (unknown 650970243 0 0) 0x7f23600457b0 con 0x7f23681365f0 2024-01-24T23:27:58.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.466+0000 7f2366ffd640 10 monclient: get_auth_request con 0x7f2354036f70 auth_method 0 2024-01-24T23:27:58.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.470+0000 7f234f7fe640 1 -- 172.21.15.132:0/2314469194 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f2360014e00 con 0x7f23681365f0 2024-01-24T23:27:58.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.470+0000 7f234f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:58.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.470+0000 7f234f7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:58.621 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.618+0000 7f236d355640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h17"}] 2024-01-24T23:27:58.621 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.618+0000 7f236d355640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h17"}] 2024-01-24T23:27:58.621 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.618+0000 7f236d355640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:58.621 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:58.618+0000 7f236d355640 1 -- 172.21.15.132:0/2314469194 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rm", "name": "h17"} v 0) v1 -- 0x7f2348005340 con 0x7f23681365f0 2024-01-24T23:27:59.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.234+0000 7f234f7fe640 1 -- 172.21.15.132:0/2314469194 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h17"}]=0 removed item id -67 name 'h17' from crush map v229) v1 ==== 120+0+0 (unknown 3002214429 0 0) 0x7f2360013270 con 0x7f23681365f0 2024-01-24T23:27:59.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.234+0000 7f234f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h17"}] 2024-01-24T23:27:59.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.234+0000 7f234f7fe640 10 monclient: _finish_command 2 = system:0 removed item id -67 name 'h17' from crush map 2024-01-24T23:27:59.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.338+0000 7f236d355640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h17"}] 2024-01-24T23:27:59.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.338+0000 7f236d355640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h17"}] 2024-01-24T23:27:59.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.338+0000 7f236d355640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:27:59.340 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.338+0000 7f236d355640 1 -- 172.21.15.132:0/2314469194 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rm", "name": "h17"} v 0) v1 -- 0x7f23480010a0 con 0x7f23681365f0 2024-01-24T23:27:59.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.338+0000 7f234f7fe640 1 -- 172.21.15.132:0/2314469194 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h17"}]=0 device 'h17' does not appear in the crush map v229) v1 ==== 120+0+0 (unknown 3367626253 0 0) 0x7f23600149f0 con 0x7f23681365f0 2024-01-24T23:27:59.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.338+0000 7f234f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h17"}] 2024-01-24T23:27:59.341 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.338+0000 7f234f7fe640 10 monclient: _finish_command 3 = system:0 device 'h17' does not appear in the crush map 2024-01-24T23:27:59.341 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h17' does not appear in the crush map 2024-01-24T23:27:59.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.342+0000 7f236d355640 1 -- 172.21.15.132:0/2314469194 >> v1:172.21.15.132:6812/1835401922 conn(0x7f2354036f70 legacy=0x7f2354039430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:59.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.342+0000 7f236d355640 10 monclient: shutdown 2024-01-24T23:27:59.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.342+0000 7f236d355640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:59.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.342+0000 7f236d355640 1 -- 172.21.15.132:0/2314469194 >> v1:172.21.15.132:6790/0 conn(0x7f23681365f0 legacy=0x7f23681d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:59.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.342+0000 7f236d355640 1 -- 172.21.15.132:0/2314469194 shutdown_connections 2024-01-24T23:27:59.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.342+0000 7f236d355640 1 -- 172.21.15.132:0/2314469194 >> 172.21.15.132:0/2314469194 conn(0x7f23680a6790 msgr2=0x7f23680a31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:59.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.342+0000 7f236d355640 1 -- 172.21.15.132:0/2314469194 shutdown_connections 2024-01-24T23:27:59.344 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.342+0000 7f236d355640 1 -- 172.21.15.132:0/2314469194 wait complete. 2024-01-24T23:27:59.357 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:27:59.357 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h18 2024-01-24T23:27:59.472 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.470+0000 7f449e943640 10 monclient: get_monmap_and_config 2024-01-24T23:27:59.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.470+0000 7f449e943640 10 monclient: build_initial_monmap 2024-01-24T23:27:59.473 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.470+0000 7f449e943640 10 monclient: monmap: 2024-01-24T23:27:59.473 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:59.473 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:59.473 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:27:59.475386+0000 2024-01-24T23:27:59.473 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:27:59.475386+0000 2024-01-24T23:27:59.473 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:59.473 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:59.473 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:59.473 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:59.473 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:59.473 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:59.474 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:59.474 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:59.474 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:59.474 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:59.474 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:59.474 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:59.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.470+0000 7f449e943640 1 Processor -- start 2024-01-24T23:27:59.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.470+0000 7f449e943640 1 -- start start 2024-01-24T23:27:59.477 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.470+0000 7f449e943640 10 monclient: init 2024-01-24T23:27:59.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:59.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 10 monclient: _add_conns ranks=[4,7,8,3,5,1,6,2,0] 2024-01-24T23:27:59.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 10 monclient(hunting): picked mon.noname-c con 0x7f4498135e10 addr v1:172.21.15.115:6791/0 2024-01-24T23:27:59.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 10 monclient(hunting): picked mon.noname-i con 0x7f44981361f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:27:59.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 10 monclient(hunting): picked mon.noname-e con 0x7f44981365f0 addr v1:172.21.15.115:6793/0 2024-01-24T23:27:59.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4498064700 con 0x7f4498135e10 2024-01-24T23:27:59.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f449809c3e0 con 0x7f44981361f0 2024-01-24T23:27:59.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44980a4f90 con 0x7f44981365f0 2024-01-24T23:27:59.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:59.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f44977fe640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f44981361f0 legacy=0x7f449813bed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:27:59.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 10 monclient(hunting): authenticate will time out at 1366.800424s 2024-01-24T23:27:59.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449cee0640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f44981365f0 0x7f449813f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:43304/0 (socket says 172.21.15.132:43304) 2024-01-24T23:27:59.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449cee0640 1 -- 172.21.15.132:0/4115620518 learned_addr learned my addr 172.21.15.132:0/4115620518 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:59.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 1 -- 172.21.15.132:0/4115620518 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2096985832 0 0) 0x7f44980a4f90 con 0x7f44981365f0 2024-01-24T23:27:59.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:59.478 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:59.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:59.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 10 monclient(hunting): my global_id is 86003 2024-01-24T23:27:59.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 1 -- 172.21.15.132:0/4115620518 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4480003530 con 0x7f44981365f0 2024-01-24T23:27:59.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 1 -- 172.21.15.132:0/4115620518 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3730680335 0 0) 0x7f4498064700 con 0x7f4498135e10 2024-01-24T23:27:59.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:27:59.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:59.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:59.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 10 monclient(hunting): my global_id is 94999 2024-01-24T23:27:59.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 1 -- 172.21.15.132:0/4115620518 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44980a4f90 con 0x7f4498135e10 2024-01-24T23:27:59.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 1 -- 172.21.15.132:0/4115620518 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 984919938 0 0) 0x7f4480003530 con 0x7f44981365f0 2024-01-24T23:27:59.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 1 -- 172.21.15.132:0/4115620518 >> v1:172.21.15.132:6792/0 conn(0x7f44981361f0 legacy=0x7f449813bed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:27:59.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 1 -- 172.21.15.132:0/4115620518 >> v1:172.21.15.115:6791/0 conn(0x7f4498135e10 legacy=0x7f449809bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:59.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:27:59.479 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 1 monclient: found mon.noname-e 2024-01-24T23:27:59.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:59.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:27:59.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 1 -- 172.21.15.132:0/4115620518 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449809b010 con 0x7f44981365f0 2024-01-24T23:27:59.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 10 monclient: _finish_auth 0 2024-01-24T23:27:59.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:27:59.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:59.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 5 monclient: authenticate success, global_id 86003 2024-01-24T23:27:59.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:59.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 1 -- 172.21.15.132:0/4115620518 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4490001b40 con 0x7f44981365f0 2024-01-24T23:27:59.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:59.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:27:59.480 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 10 monclient: dump: 2024-01-24T23:27:59.480 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:59.480 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:59.481 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:59.481 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:59.481 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:59.481 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:59.481 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:59.481 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:59.481 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:59.481 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:59.481 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:59.481 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:59.481 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:59.481 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:59.481 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:59.481 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:59.481 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:27:59.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 1 -- 172.21.15.132:0/4115620518 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4490001e20 con 0x7f44981365f0 2024-01-24T23:27:59.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:59.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 1 -- 172.21.15.132:0/4115620518 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f4490004f70 con 0x7f44981365f0 2024-01-24T23:27:59.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 10 monclient: get_monmap_and_config success 2024-01-24T23:27:59.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 4 set_mon_vals no callback set 2024-01-24T23:27:59.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:59.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:27:59.482 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f4496ffd640 10 monclient: dump: 2024-01-24T23:27:59.482 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:59.482 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:59.482 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:59.482 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:59.482 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:59.482 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:59.482 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:59.482 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:59.482 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:59.483 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:59.483 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:59.483 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:59.483 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:59.483 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:59.483 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:59.483 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:59.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 10 monclient: shutdown 2024-01-24T23:27:59.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:27:59.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 1 -- 172.21.15.132:0/4115620518 >> v1:172.21.15.115:6793/0 conn(0x7f44981365f0 legacy=0x7f449813f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:59.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 1 -- 172.21.15.132:0/4115620518 shutdown_connections 2024-01-24T23:27:59.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 1 -- 172.21.15.132:0/4115620518 >> 172.21.15.132:0/4115620518 conn(0x7f44980a6790 msgr2=0x7f4498071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:27:59.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 1 -- 172.21.15.132:0/4115620518 shutdown_connections 2024-01-24T23:27:59.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 1 -- 172.21.15.132:0/4115620518 wait complete. 2024-01-24T23:27:59.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 10 monclient: build_initial_monmap 2024-01-24T23:27:59.483 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 10 monclient: monmap: 2024-01-24T23:27:59.484 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:27:59.484 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:27:59.484 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:27:59.484 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:27:59.484 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:27:59.484 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:27:59.484 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:27:59.484 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:27:59.484 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:27:59.484 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:27:59.484 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:27:59.484 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:27:59.484 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:27:59.484 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:27:59.485 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:27:59.485 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:59.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 1 Processor -- start 2024-01-24T23:27:59.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 1 -- start start 2024-01-24T23:27:59.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.474+0000 7f449e943640 10 monclient: init 2024-01-24T23:27:59.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f449e943640 10 monclient: _reopen_session rank -1 2024-01-24T23:27:59.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f449e943640 10 monclient: _add_conns ranks=[6,1,0,5,7,8,4,3,2] 2024-01-24T23:27:59.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f449e943640 10 monclient(hunting): picked mon.noname-d con 0x7f4498135e10 addr v1:172.21.15.115:6792/0 2024-01-24T23:27:59.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f449e943640 10 monclient(hunting): picked mon.noname-f con 0x7f44981361f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:27:59.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f449e943640 10 monclient(hunting): picked mon.noname-a con 0x7f44981365f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:27:59.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f449e943640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4498120cd0 con 0x7f44981365f0 2024-01-24T23:27:59.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f449e943640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f449809ca00 con 0x7f44981361f0 2024-01-24T23:27:59.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f449e943640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4498122630 con 0x7f4498135e10 2024-01-24T23:27:59.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f449e943640 10 monclient(hunting): _renew_subs 2024-01-24T23:27:59.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f449e943640 10 monclient(hunting): authenticate will time out at 1366.804658s 2024-01-24T23:27:59.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f449cee0640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f44981365f0 legacy=0x7f44981d3be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:27:59.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f44977fe640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f44981361f0 0x7f44981d04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:46012/0 (socket says 172.21.15.132:46012) 2024-01-24T23:27:59.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f44977fe640 1 -- 172.21.15.132:0/1894789702 learned_addr learned my addr 172.21.15.132:0/1894789702 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:27:59.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 1 -- 172.21.15.132:0/1894789702 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1704458817 0 0) 0x7f449809ca00 con 0x7f44981361f0 2024-01-24T23:27:59.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:59.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:59.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:59.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 10 monclient(hunting): my global_id is 124399 2024-01-24T23:27:59.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 1 -- 172.21.15.132:0/1894789702 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4484003530 con 0x7f44981361f0 2024-01-24T23:27:59.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 1 -- 172.21.15.132:0/1894789702 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4127739047 0 0) 0x7f4484003530 con 0x7f44981361f0 2024-01-24T23:27:59.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 1 -- 172.21.15.132:0/1894789702 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f449809ca00 con 0x7f44981361f0 2024-01-24T23:27:59.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 1 -- 172.21.15.132:0/1894789702 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f448c004300 con 0x7f44981361f0 2024-01-24T23:27:59.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:59.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:27:59.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 10 monclient(hunting): dump: 2024-01-24T23:27:59.490 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:59.490 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:59.490 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:59.490 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:59.491 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:59.491 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:59.491 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:59.491 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:59.491 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:59.491 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:59.491 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:59.491 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:59.491 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:59.491 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:59.491 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:59.491 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:59.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:27:59.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 1 -- 172.21.15.132:0/1894789702 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4041767011 0 0) 0x7f4498122630 con 0x7f4498135e10 2024-01-24T23:27:59.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:27:59.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:27:59.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:27:59.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 10 monclient(hunting): my global_id is 7044 2024-01-24T23:27:59.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 1 -- 172.21.15.132:0/1894789702 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4484003530 con 0x7f4498135e10 2024-01-24T23:27:59.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 1 -- 172.21.15.132:0/1894789702 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1301405111 0 0) 0x7f449809ca00 con 0x7f44981361f0 2024-01-24T23:27:59.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 1 -- 172.21.15.132:0/1894789702 >> v1:172.21.15.115:6792/0 conn(0x7f4498135e10 legacy=0x7f4498134c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:27:59.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 1 -- 172.21.15.132:0/1894789702 >> v1:172.21.15.115:6789/0 conn(0x7f44981365f0 legacy=0x7f44981d3be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:27:59.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:27:59.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 1 monclient: found mon.f 2024-01-24T23:27:59.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:27:59.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:27:59.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 1 -- 172.21.15.132:0/1894789702 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449809c6b0 con 0x7f44981361f0 2024-01-24T23:27:59.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 10 monclient: _finish_auth 0 2024-01-24T23:27:59.492 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:27:59.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:27:59.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f449e943640 5 monclient: authenticate success, global_id 124399 2024-01-24T23:27:59.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f449e943640 10 monclient: _renew_subs 2024-01-24T23:27:59.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f449e943640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:27:59.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f449e943640 1 -- 172.21.15.132:0/1894789702 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4498122350 con 0x7f44981361f0 2024-01-24T23:27:59.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f449e943640 10 monclient: _renew_subs 2024-01-24T23:27:59.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f449e943640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:27:59.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f449e943640 1 -- 172.21.15.132:0/1894789702 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f44981cb650 con 0x7f44981361f0 2024-01-24T23:27:59.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 1 -- 172.21.15.132:0/1894789702 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f448c004620 con 0x7f44981361f0 2024-01-24T23:27:59.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:27:59.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 1 -- 172.21.15.132:0/1894789702 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f448c004b50 con 0x7f44981361f0 2024-01-24T23:27:59.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:27:59.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:27:59.493 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 10 monclient: dump: 2024-01-24T23:27:59.494 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:27:59.494 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:27:59.494 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:59.494 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:27:59.494 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:27:59.494 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:27:59.494 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:27:59.494 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:27:59.494 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:27:59.494 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:27:59.494 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:27:59.494 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:27:59.494 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:27:59.494 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:27:59.494 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:27:59.495 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:27:59.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f44967fc640 4 set_mon_vals no callback set 2024-01-24T23:27:59.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f449e943640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:59.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 1 -- 172.21.15.132:0/1894789702 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f448c004cb0 con 0x7f44981361f0 2024-01-24T23:27:59.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f449e943640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:59.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f449e943640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:27:59.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f449e943640 1 -- 172.21.15.132:0/1894789702 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4478005090 con 0x7f44981361f0 2024-01-24T23:27:59.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4494ff9640 1 -- 172.21.15.132:0/1894789702 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(229..229 src has 1..229) v4 ==== 10784+0+0 (unknown 1397060557 0 0) 0x7f448c0453c0 con 0x7f44981361f0 2024-01-24T23:27:59.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.478+0000 7f4497fff640 10 monclient: get_auth_request con 0x7f4484036fd0 auth_method 0 2024-01-24T23:27:59.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.482+0000 7f4494ff9640 1 -- 172.21.15.132:0/1894789702 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f448c014b30 con 0x7f44981361f0 2024-01-24T23:27:59.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.482+0000 7f4494ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:27:59.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.482+0000 7f4494ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:27:59.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.642+0000 7f449e943640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h18"}] 2024-01-24T23:27:59.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.642+0000 7f449e943640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h18"}] 2024-01-24T23:27:59.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.642+0000 7f449e943640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:27:59.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:27:59.642+0000 7f449e943640 1 -- 172.21.15.132:0/1894789702 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd crush rm", "name": "h18"} v 0) v1 -- 0x7f4478005340 con 0x7f44981361f0 2024-01-24T23:28:00.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.250+0000 7f4494ff9640 1 -- 172.21.15.132:0/1894789702 <== mon.1 v1:172.21.15.132:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h18"}]=0 removed item id -70 name 'h18' from crush map v230) v1 ==== 120+0+0 (unknown 1537110563 0 0) 0x7f448c01d030 con 0x7f44981361f0 2024-01-24T23:28:00.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.250+0000 7f4494ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h18"}] 2024-01-24T23:28:00.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.250+0000 7f4494ff9640 10 monclient: _finish_command 2 = system:0 removed item id -70 name 'h18' from crush map 2024-01-24T23:28:00.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.350+0000 7f449e943640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h18"}] 2024-01-24T23:28:00.350 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.350+0000 7f449e943640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h18"}] 2024-01-24T23:28:00.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.350+0000 7f449e943640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:00.351 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.350+0000 7f449e943640 1 -- 172.21.15.132:0/1894789702 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd crush rm", "name": "h18"} v 0) v1 -- 0x7f4478002220 con 0x7f44981361f0 2024-01-24T23:28:00.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.350+0000 7f4494ff9640 1 -- 172.21.15.132:0/1894789702 <== mon.1 v1:172.21.15.132:6789/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h18"}]=0 device 'h18' does not appear in the crush map v230) v1 ==== 120+0+0 (unknown 613402822 0 0) 0x7f448c012fa0 con 0x7f44981361f0 2024-01-24T23:28:00.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.350+0000 7f4494ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h18"}] 2024-01-24T23:28:00.352 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.350+0000 7f4494ff9640 10 monclient: _finish_command 3 = system:0 device 'h18' does not appear in the crush map 2024-01-24T23:28:00.352 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h18' does not appear in the crush map 2024-01-24T23:28:00.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.354+0000 7f449e943640 1 -- 172.21.15.132:0/1894789702 >> v1:172.21.15.132:6812/1835401922 conn(0x7f4484036fd0 legacy=0x7f4484039490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:00.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.354+0000 7f449e943640 10 monclient: shutdown 2024-01-24T23:28:00.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.354+0000 7f449e943640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:00.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.354+0000 7f449e943640 1 -- 172.21.15.132:0/1894789702 >> v1:172.21.15.132:6789/0 conn(0x7f44981361f0 legacy=0x7f44981d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:00.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.354+0000 7f449e943640 1 -- 172.21.15.132:0/1894789702 shutdown_connections 2024-01-24T23:28:00.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.354+0000 7f449e943640 1 -- 172.21.15.132:0/1894789702 >> 172.21.15.132:0/1894789702 conn(0x7f44980a6790 msgr2=0x7f44981bb850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:00.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.354+0000 7f449e943640 1 -- 172.21.15.132:0/1894789702 shutdown_connections 2024-01-24T23:28:00.354 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.354+0000 7f449e943640 1 -- 172.21.15.132:0/1894789702 wait complete. 2024-01-24T23:28:00.369 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:28:00.369 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h19 2024-01-24T23:28:00.487 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f875be45640 10 monclient: get_monmap_and_config 2024-01-24T23:28:00.487 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f875be45640 10 monclient: build_initial_monmap 2024-01-24T23:28:00.487 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f875be45640 10 monclient: monmap: 2024-01-24T23:28:00.487 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:00.487 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:00.487 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:00.489655+0000 2024-01-24T23:28:00.487 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:00.489655+0000 2024-01-24T23:28:00.487 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:00.487 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:00.487 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:00.488 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:00.488 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:00.488 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:00.488 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:00.488 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:00.488 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:00.488 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:00.488 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:00.488 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:00.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f875be45640 1 Processor -- start 2024-01-24T23:28:00.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f875be45640 1 -- start start 2024-01-24T23:28:00.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f875be45640 10 monclient: init 2024-01-24T23:28:00.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f875be45640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:00.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f875be45640 10 monclient: _add_conns ranks=[1,6,3,8,5,0,2,4,7] 2024-01-24T23:28:00.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f875be45640 10 monclient(hunting): picked mon.noname-f con 0x7f875409ff20 addr v1:172.21.15.132:6789/0 2024-01-24T23:28:00.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f875be45640 10 monclient(hunting): picked mon.noname-d con 0x7f875409f9c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:28:00.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f875be45640 10 monclient(hunting): picked mon.noname-g con 0x7f875413b2c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:28:00.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f875be45640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8754064700 con 0x7f875409ff20 2024-01-24T23:28:00.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f875be45640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f875409e760 con 0x7f875413b2c0 2024-01-24T23:28:00.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f875be45640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87540a1160 con 0x7f875409f9c0 2024-01-24T23:28:00.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f875be45640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:00.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f875be45640 10 monclient(hunting): authenticate will time out at 1367.814836s 2024-01-24T23:28:00.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f875a3e2640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f875413b2c0 0x7f875413d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:41964/0 (socket says 172.21.15.132:41964) 2024-01-24T23:28:00.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f875a3e2640 1 -- 172.21.15.132:0/3402126367 learned_addr learned my addr 172.21.15.132:0/3402126367 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:00.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 1 -- 172.21.15.132:0/3402126367 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1663528080 0 0) 0x7f8754064700 con 0x7f875409ff20 2024-01-24T23:28:00.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 10 monclient(hunting): _negotiate 2024-01-24T23:28:00.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:00.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:00.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 10 monclient(hunting): my global_id is 124408 2024-01-24T23:28:00.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 1 -- 172.21.15.132:0/3402126367 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8740003530 con 0x7f875409ff20 2024-01-24T23:28:00.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 1 -- 172.21.15.132:0/3402126367 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193935420 0 0) 0x7f875409e760 con 0x7f875413b2c0 2024-01-24T23:28:00.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 10 monclient(hunting): _negotiate 2024-01-24T23:28:00.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:00.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:00.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 10 monclient(hunting): my global_id is 6852 2024-01-24T23:28:00.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 1 -- 172.21.15.132:0/3402126367 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8754064700 con 0x7f875413b2c0 2024-01-24T23:28:00.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 1 -- 172.21.15.132:0/3402126367 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3172861022 0 0) 0x7f8740003530 con 0x7f875409ff20 2024-01-24T23:28:00.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 1 -- 172.21.15.132:0/3402126367 >> v1:172.21.15.115:6792/0 conn(0x7f875409f9c0 legacy=0x7f8754139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:00.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 1 -- 172.21.15.132:0/3402126367 >> v1:172.21.15.132:6790/0 conn(0x7f875413b2c0 legacy=0x7f875413d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:00.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 10 monclient: _finish_hunting 0 2024-01-24T23:28:00.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 1 monclient: found mon.noname-f 2024-01-24T23:28:00.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:00.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:28:00.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 1 -- 172.21.15.132:0/3402126367 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87540a0a10 con 0x7f875409ff20 2024-01-24T23:28:00.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 10 monclient: _finish_auth 0 2024-01-24T23:28:00.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 10 monclient: _check_auth_tickets 2024-01-24T23:28:00.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:00.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 1 -- 172.21.15.132:0/3402126367 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f87500016f0 con 0x7f875409ff20 2024-01-24T23:28:00.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:00.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:28:00.498 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 10 monclient: dump: 2024-01-24T23:28:00.498 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:00.498 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:00.498 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:00.498 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:00.498 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:00.498 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:00.499 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:00.499 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:00.499 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:00.499 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:00.499 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:00.499 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:00.499 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:00.499 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:00.499 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:00.499 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:00.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f875be45640 5 monclient: authenticate success, global_id 124408 2024-01-24T23:28:00.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f875be45640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:00.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 1 -- 172.21.15.132:0/3402126367 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8750001910 con 0x7f875409ff20 2024-01-24T23:28:00.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:00.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f875be45640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:00.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f875be45640 4 set_mon_vals no callback set 2024-01-24T23:28:00.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 1 -- 172.21.15.132:0/3402126367 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8750003620 con 0x7f875409ff20 2024-01-24T23:28:00.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:00.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:28:00.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.486+0000 7f8758bdf640 10 monclient: dump: 2024-01-24T23:28:00.500 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:00.500 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:00.500 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:00.500 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:00.500 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:00.500 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:00.500 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:00.500 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:00.501 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:00.501 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:00.501 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:00.501 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:00.501 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:00.501 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:00.501 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:00.501 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:00.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 10 monclient: shutdown 2024-01-24T23:28:00.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:00.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 1 -- 172.21.15.132:0/3402126367 >> v1:172.21.15.132:6789/0 conn(0x7f875409ff20 legacy=0x7f87540a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:00.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 1 -- 172.21.15.132:0/3402126367 shutdown_connections 2024-01-24T23:28:00.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 1 -- 172.21.15.132:0/3402126367 >> 172.21.15.132:0/3402126367 conn(0x7f87540a5a10 msgr2=0x7f87540a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:00.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 1 -- 172.21.15.132:0/3402126367 shutdown_connections 2024-01-24T23:28:00.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 1 -- 172.21.15.132:0/3402126367 wait complete. 2024-01-24T23:28:00.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 10 monclient: build_initial_monmap 2024-01-24T23:28:00.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 10 monclient: monmap: 2024-01-24T23:28:00.502 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:00.502 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:00.502 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:00.502 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:00.502 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:00.502 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:00.502 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:00.502 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:00.502 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:00.502 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:00.502 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:00.503 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:00.503 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:00.503 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:00.503 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:00.503 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:00.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 1 Processor -- start 2024-01-24T23:28:00.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 1 -- start start 2024-01-24T23:28:00.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 10 monclient: init 2024-01-24T23:28:00.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:00.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 10 monclient: _add_conns ranks=[1,8,7,3,0,2,5,4,6] 2024-01-24T23:28:00.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 10 monclient(hunting): picked mon.noname-f con 0x7f875409f9c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:28:00.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 10 monclient(hunting): picked mon.noname-e con 0x7f875409ff20 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:00.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 10 monclient(hunting): picked mon.noname-i con 0x7f875413b2c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:28:00.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f875409d780 con 0x7f875409f9c0 2024-01-24T23:28:00.503 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f875409d2e0 con 0x7f875413b2c0 2024-01-24T23:28:00.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87541201a0 con 0x7f875409ff20 2024-01-24T23:28:00.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:00.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 10 monclient(hunting): authenticate will time out at 1367.818046s 2024-01-24T23:28:00.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875a3e2640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f875413b2c0 legacy=0x7f87541d38d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:28:00.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f8759be1640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f875409f9c0 0x7f875409c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:46040/0 (socket says 172.21.15.132:46040) 2024-01-24T23:28:00.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f8759be1640 1 -- 172.21.15.132:0/2521236458 learned_addr learned my addr 172.21.15.132:0/2521236458 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:00.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 1 -- 172.21.15.132:0/2521236458 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3251138874 0 0) 0x7f875409d780 con 0x7f875409f9c0 2024-01-24T23:28:00.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:28:00.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:00.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:00.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 10 monclient(hunting): my global_id is 124417 2024-01-24T23:28:00.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 1 -- 172.21.15.132:0/2521236458 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8738003530 con 0x7f875409f9c0 2024-01-24T23:28:00.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 1 -- 172.21.15.132:0/2521236458 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2665937234 0 0) 0x7f8738003530 con 0x7f875409f9c0 2024-01-24T23:28:00.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 1 -- 172.21.15.132:0/2521236458 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f875409d780 con 0x7f875409f9c0 2024-01-24T23:28:00.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 1 -- 172.21.15.132:0/2521236458 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f87500016b0 con 0x7f875409f9c0 2024-01-24T23:28:00.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:00.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:28:00.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 10 monclient(hunting): dump: 2024-01-24T23:28:00.505 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:00.505 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:00.505 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:00.505 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:00.505 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:00.505 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:00.505 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:00.505 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:00.505 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:00.505 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:00.505 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:00.505 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:00.505 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:00.506 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:00.506 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:00.506 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:00.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:00.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 1 -- 172.21.15.132:0/2521236458 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3517523454 0 0) 0x7f87541201a0 con 0x7f875409ff20 2024-01-24T23:28:00.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:28:00.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:00.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:00.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 10 monclient(hunting): my global_id is 86012 2024-01-24T23:28:00.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 1 -- 172.21.15.132:0/2521236458 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8738003530 con 0x7f875409ff20 2024-01-24T23:28:00.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 1 -- 172.21.15.132:0/2521236458 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1822039983 0 0) 0x7f875409d780 con 0x7f875409f9c0 2024-01-24T23:28:00.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 1 -- 172.21.15.132:0/2521236458 >> v1:172.21.15.115:6793/0 conn(0x7f875409ff20 legacy=0x7f875409cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:00.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 1 -- 172.21.15.132:0/2521236458 >> v1:172.21.15.132:6792/0 conn(0x7f875413b2c0 legacy=0x7f87541d38d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:28:00.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:28:00.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 1 monclient: found mon.f 2024-01-24T23:28:00.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:00.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:00.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 1 -- 172.21.15.132:0/2521236458 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8754071960 con 0x7f875409f9c0 2024-01-24T23:28:00.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 10 monclient: _finish_auth 0 2024-01-24T23:28:00.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:28:00.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:00.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 5 monclient: authenticate success, global_id 124417 2024-01-24T23:28:00.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 10 monclient: _renew_subs 2024-01-24T23:28:00.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:00.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 1 -- 172.21.15.132:0/2521236458 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8754071680 con 0x7f875409f9c0 2024-01-24T23:28:00.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 10 monclient: _renew_subs 2024-01-24T23:28:00.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:00.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 1 -- 172.21.15.132:0/2521236458 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f87540707b0 con 0x7f875409f9c0 2024-01-24T23:28:00.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 1 -- 172.21.15.132:0/2521236458 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8750003c60 con 0x7f875409f9c0 2024-01-24T23:28:00.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:00.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 1 -- 172.21.15.132:0/2521236458 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8750004110 con 0x7f875409f9c0 2024-01-24T23:28:00.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:00.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874bfff640 4 set_mon_vals no callback set 2024-01-24T23:28:00.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:28:00.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 10 monclient: dump: 2024-01-24T23:28:00.508 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:00.508 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:00.508 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:00.508 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:00.508 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:00.508 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:00.508 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:00.508 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:00.508 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:00.508 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:00.509 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:00.509 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:00.509 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:00.509 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:00.509 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:00.509 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:00.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:00.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:00.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f874a7fc640 1 -- 172.21.15.132:0/2521236458 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f87500042f0 con 0x7f875409f9c0 2024-01-24T23:28:00.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:00.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.490+0000 7f875be45640 1 -- 172.21.15.132:0/2521236458 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f873c005090 con 0x7f875409f9c0 2024-01-24T23:28:00.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.494+0000 7f874a7fc640 1 -- 172.21.15.132:0/2521236458 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(230..230 src has 1..230) v4 ==== 10571+0+0 (unknown 2206199821 0 0) 0x7f87500455f0 con 0x7f875409f9c0 2024-01-24T23:28:00.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.494+0000 7f87593e0640 10 monclient: get_auth_request con 0x7f8738036f30 auth_method 0 2024-01-24T23:28:00.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.498+0000 7f874a7fc640 1 -- 172.21.15.132:0/2521236458 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f8750014e20 con 0x7f875409f9c0 2024-01-24T23:28:00.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.498+0000 7f874a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:00.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.498+0000 7f874a7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:00.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.658+0000 7f875be45640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h19"}] 2024-01-24T23:28:00.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.658+0000 7f875be45640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h19"}] 2024-01-24T23:28:00.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.658+0000 7f875be45640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:00.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:00.658+0000 7f875be45640 1 -- 172.21.15.132:0/2521236458 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd crush rm", "name": "h19"} v 0) v1 -- 0x7f873c005340 con 0x7f875409f9c0 2024-01-24T23:28:01.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.258+0000 7f874a7fc640 1 -- 172.21.15.132:0/2521236458 <== mon.1 v1:172.21.15.132:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h19"}]=0 removed item id -73 name 'h19' from crush map v231) v1 ==== 120+0+0 (unknown 3743027801 0 0) 0x7f8750013290 con 0x7f875409f9c0 2024-01-24T23:28:01.260 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.258+0000 7f874a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h19"}] 2024-01-24T23:28:01.261 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.258+0000 7f874a7fc640 10 monclient: _finish_command 2 = system:0 removed item id -73 name 'h19' from crush map 2024-01-24T23:28:01.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.358+0000 7f875be45640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h19"}] 2024-01-24T23:28:01.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.358+0000 7f875be45640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h19"}] 2024-01-24T23:28:01.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.362+0000 7f875be45640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:01.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.362+0000 7f875be45640 1 -- 172.21.15.132:0/2521236458 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd crush rm", "name": "h19"} v 0) v1 -- 0x7f873c002220 con 0x7f875409f9c0 2024-01-24T23:28:01.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.362+0000 7f874a7fc640 1 -- 172.21.15.132:0/2521236458 <== mon.1 v1:172.21.15.132:6789/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h19"}]=0 device 'h19' does not appear in the crush map v231) v1 ==== 120+0+0 (unknown 2427969641 0 0) 0x7f8750014a10 con 0x7f875409f9c0 2024-01-24T23:28:01.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.362+0000 7f874a7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h19"}] 2024-01-24T23:28:01.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.362+0000 7f874a7fc640 10 monclient: _finish_command 3 = system:0 device 'h19' does not appear in the crush map 2024-01-24T23:28:01.363 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h19' does not appear in the crush map 2024-01-24T23:28:01.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.366+0000 7f875be45640 1 -- 172.21.15.132:0/2521236458 >> v1:172.21.15.132:6812/1835401922 conn(0x7f8738036f30 legacy=0x7f87380393f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:01.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.366+0000 7f875be45640 10 monclient: shutdown 2024-01-24T23:28:01.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.366+0000 7f875be45640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:01.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.366+0000 7f875be45640 1 -- 172.21.15.132:0/2521236458 >> v1:172.21.15.132:6789/0 conn(0x7f875409f9c0 legacy=0x7f875409c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:01.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.366+0000 7f875be45640 1 -- 172.21.15.132:0/2521236458 shutdown_connections 2024-01-24T23:28:01.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.366+0000 7f875be45640 1 -- 172.21.15.132:0/2521236458 >> 172.21.15.132:0/2521236458 conn(0x7f87540a5a10 msgr2=0x7f8754136c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:01.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.366+0000 7f875be45640 1 -- 172.21.15.132:0/2521236458 shutdown_connections 2024-01-24T23:28:01.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.366+0000 7f875be45640 1 -- 172.21.15.132:0/2521236458 wait complete. 2024-01-24T23:28:01.381 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:28:01.381 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h20 2024-01-24T23:28:01.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f2705974640 10 monclient: get_monmap_and_config 2024-01-24T23:28:01.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f2705974640 10 monclient: build_initial_monmap 2024-01-24T23:28:01.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f2705974640 10 monclient: monmap: 2024-01-24T23:28:01.499 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:01.499 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:01.499 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:01.501930+0000 2024-01-24T23:28:01.499 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:01.501930+0000 2024-01-24T23:28:01.500 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:01.500 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:01.500 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:01.500 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:01.500 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:01.500 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:01.500 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:01.500 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:01.500 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:01.500 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:01.500 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:01.500 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:01.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f2705974640 1 Processor -- start 2024-01-24T23:28:01.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f2705974640 1 -- start start 2024-01-24T23:28:01.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f2705974640 10 monclient: init 2024-01-24T23:28:01.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f2705974640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:01.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f2705974640 10 monclient: _add_conns ranks=[7,1,2,6,8,4,3,5,0] 2024-01-24T23:28:01.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f2705974640 10 monclient(hunting): picked mon.noname-i con 0x7f270009ff90 addr v1:172.21.15.132:6792/0 2024-01-24T23:28:01.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f2705974640 10 monclient(hunting): picked mon.noname-f con 0x7f270009f9c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:28:01.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f2705974640 10 monclient(hunting): picked mon.noname-b con 0x7f270013b2c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:28:01.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f2705974640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2700064700 con 0x7f270009f9c0 2024-01-24T23:28:01.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f2705974640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f270009fe00 con 0x7f270013b2c0 2024-01-24T23:28:01.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26feffd640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f270009ff90 legacy=0x7f27000a0370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:28:01.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f2705974640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27000a1160 con 0x7f270009ff90 2024-01-24T23:28:01.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f2705974640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:01.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f2705974640 10 monclient(hunting): authenticate will time out at 1368.826790s 2024-01-24T23:28:01.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fe7fc640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f270009f9c0 0x7f2700139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:46046/0 (socket says 172.21.15.132:46046) 2024-01-24T23:28:01.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fe7fc640 1 -- 172.21.15.132:0/658164772 learned_addr learned my addr 172.21.15.132:0/658164772 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:01.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26ff7fe640 1 -- 172.21.15.132:0/658164772 >> v1:172.21.15.115:6790/0 conn(0x7f270013b2c0 legacy=0x7f270013d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:28:01.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 1 -- 172.21.15.132:0/658164772 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1739691215 0 0) 0x7f2700064700 con 0x7f270009f9c0 2024-01-24T23:28:01.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 10 monclient(hunting): _negotiate 2024-01-24T23:28:01.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:01.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:01.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 10 monclient(hunting): my global_id is 124426 2024-01-24T23:28:01.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 1 -- 172.21.15.132:0/658164772 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26e8003530 con 0x7f270009f9c0 2024-01-24T23:28:01.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 1 -- 172.21.15.132:0/658164772 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1891120915 0 0) 0x7f26e8003530 con 0x7f270009f9c0 2024-01-24T23:28:01.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 1 -- 172.21.15.132:0/658164772 >> v1:172.21.15.132:6792/0 conn(0x7f270009ff90 legacy=0x7f27000a0370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:28:01.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 1 -- 172.21.15.132:0/658164772 >> v1:172.21.15.115:6790/0 conn(0x7f270013b2c0 legacy=0x7f270013d6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:28:01.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 10 monclient: _finish_hunting 0 2024-01-24T23:28:01.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 1 monclient: found mon.noname-f 2024-01-24T23:28:01.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:01.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:28:01.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 1 -- 172.21.15.132:0/658164772 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27000a1440 con 0x7f270009f9c0 2024-01-24T23:28:01.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 10 monclient: _finish_auth 0 2024-01-24T23:28:01.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 10 monclient: _check_auth_tickets 2024-01-24T23:28:01.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:01.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 1 -- 172.21.15.132:0/658164772 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f26f40017c0 con 0x7f270009f9c0 2024-01-24T23:28:01.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:01.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:28:01.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 10 monclient: dump: 2024-01-24T23:28:01.510 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:01.510 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:01.510 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:01.511 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:01.511 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:01.511 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:01.511 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:01.511 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:01.511 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:01.511 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:01.511 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:01.511 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:01.511 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:01.511 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:01.511 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:01.511 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:01.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f2705974640 5 monclient: authenticate success, global_id 124426 2024-01-24T23:28:01.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f2705974640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:01.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 1 -- 172.21.15.132:0/658164772 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f26f4001ae0 con 0x7f270009f9c0 2024-01-24T23:28:01.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:01.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 1 -- 172.21.15.132:0/658164772 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f26f4004bf0 con 0x7f270009f9c0 2024-01-24T23:28:01.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:01.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:28:01.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f26fdffb640 10 monclient: dump: 2024-01-24T23:28:01.512 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:01.512 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:01.512 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:01.512 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:01.512 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:01.512 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:01.512 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:01.513 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:01.513 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:01.513 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:01.513 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:01.513 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:01.513 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:01.513 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:01.513 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:01.513 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:01.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f2705974640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:01.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.498+0000 7f2705974640 4 set_mon_vals no callback set 2024-01-24T23:28:01.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 10 monclient: shutdown 2024-01-24T23:28:01.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:01.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 1 -- 172.21.15.132:0/658164772 >> v1:172.21.15.132:6789/0 conn(0x7f270009f9c0 legacy=0x7f2700139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:01.513 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 1 -- 172.21.15.132:0/658164772 shutdown_connections 2024-01-24T23:28:01.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 1 -- 172.21.15.132:0/658164772 >> 172.21.15.132:0/658164772 conn(0x7f27000a5a10 msgr2=0x7f27000a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:01.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 1 -- 172.21.15.132:0/658164772 shutdown_connections 2024-01-24T23:28:01.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 1 -- 172.21.15.132:0/658164772 wait complete. 2024-01-24T23:28:01.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 10 monclient: build_initial_monmap 2024-01-24T23:28:01.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 10 monclient: monmap: 2024-01-24T23:28:01.514 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:01.514 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:01.514 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:01.514 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:01.514 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:01.514 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:01.514 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:01.514 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:01.514 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:01.515 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:01.515 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:01.515 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:01.515 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:01.515 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:01.515 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:01.515 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:01.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 1 Processor -- start 2024-01-24T23:28:01.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 1 -- start start 2024-01-24T23:28:01.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 10 monclient: init 2024-01-24T23:28:01.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:01.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 10 monclient: _add_conns ranks=[3,6,8,5,7,0,2,4,1] 2024-01-24T23:28:01.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 10 monclient(hunting): picked mon.noname-g con 0x7f270009f9c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:28:01.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 10 monclient(hunting): picked mon.noname-d con 0x7f270009ff90 addr v1:172.21.15.115:6792/0 2024-01-24T23:28:01.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 10 monclient(hunting): picked mon.noname-e con 0x7f270013b2c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:01.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f270009f450 con 0x7f270009f9c0 2024-01-24T23:28:01.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27001382f0 con 0x7f270009ff90 2024-01-24T23:28:01.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2700138b00 con 0x7f270013b2c0 2024-01-24T23:28:01.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:01.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 10 monclient(hunting): authenticate will time out at 1368.829827s 2024-01-24T23:28:01.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26feffd640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f270009f9c0 0x7f270009c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:41972/0 (socket says 172.21.15.132:41972) 2024-01-24T23:28:01.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26feffd640 1 -- 172.21.15.132:0/55847631 learned_addr learned my addr 172.21.15.132:0/55847631 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:01.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 1 -- 172.21.15.132:0/55847631 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1952946711 0 0) 0x7f270009f450 con 0x7f270009f9c0 2024-01-24T23:28:01.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 10 monclient(hunting): _negotiate 2024-01-24T23:28:01.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:01.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:01.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 10 monclient(hunting): my global_id is 6861 2024-01-24T23:28:01.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 1 -- 172.21.15.132:0/55847631 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26ec003530 con 0x7f270009f9c0 2024-01-24T23:28:01.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 1 -- 172.21.15.132:0/55847631 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3960136071 0 0) 0x7f26ec003530 con 0x7f270009f9c0 2024-01-24T23:28:01.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 1 -- 172.21.15.132:0/55847631 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f270009f450 con 0x7f270009f9c0 2024-01-24T23:28:01.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 1 -- 172.21.15.132:0/55847631 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f26f0004300 con 0x7f270009f9c0 2024-01-24T23:28:01.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:01.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:28:01.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 10 monclient(hunting): dump: 2024-01-24T23:28:01.517 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:01.517 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:01.517 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:01.517 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:01.517 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:01.517 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:01.517 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:01.518 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 1 -- 172.21.15.132:0/55847631 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2219827707 0 0) 0x7f2700138b00 con 0x7f270013b2c0 2024-01-24T23:28:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 10 monclient(hunting): _negotiate 2024-01-24T23:28:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 10 monclient(hunting): my global_id is 86021 2024-01-24T23:28:01.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 1 -- 172.21.15.132:0/55847631 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26ec003530 con 0x7f270013b2c0 2024-01-24T23:28:01.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 1 -- 172.21.15.132:0/55847631 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1665417775 0 0) 0x7f270009f450 con 0x7f270009f9c0 2024-01-24T23:28:01.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 1 -- 172.21.15.132:0/55847631 >> v1:172.21.15.115:6793/0 conn(0x7f270013b2c0 legacy=0x7f27001d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:01.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 1 -- 172.21.15.132:0/55847631 >> v1:172.21.15.115:6792/0 conn(0x7f270009ff90 legacy=0x7f270009cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:01.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 10 monclient: _finish_hunting 0 2024-01-24T23:28:01.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 1 monclient: found mon.g 2024-01-24T23:28:01.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:01.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:01.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 1 -- 172.21.15.132:0/55847631 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2700136df0 con 0x7f270009f9c0 2024-01-24T23:28:01.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 10 monclient: _finish_auth 0 2024-01-24T23:28:01.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 10 monclient: _check_auth_tickets 2024-01-24T23:28:01.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:01.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 5 monclient: authenticate success, global_id 6861 2024-01-24T23:28:01.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 10 monclient: _renew_subs 2024-01-24T23:28:01.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:01.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 1 -- 172.21.15.132:0/55847631 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2700137da0 con 0x7f270009f9c0 2024-01-24T23:28:01.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 10 monclient: _renew_subs 2024-01-24T23:28:01.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:01.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 1 -- 172.21.15.132:0/55847631 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f270009d2e0 con 0x7f270009f9c0 2024-01-24T23:28:01.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 1 -- 172.21.15.132:0/55847631 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f26f00045f0 con 0x7f270009f9c0 2024-01-24T23:28:01.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:01.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 1 -- 172.21.15.132:0/55847631 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f26f0004aa0 con 0x7f270009f9c0 2024-01-24T23:28:01.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:01.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26fcff9640 4 set_mon_vals no callback set 2024-01-24T23:28:01.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:28:01.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 10 monclient: dump: 2024-01-24T23:28:01.520 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:01.520 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:01.520 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:01.521 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:01.521 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:01.521 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:01.521 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:01.521 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:01.521 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:01.521 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:01.521 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:01.521 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:01.521 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:01.521 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:01.521 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:01.521 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:01.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:01.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:01.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:01.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f2705974640 1 -- 172.21.15.132:0/55847631 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e0005090 con 0x7f270009f9c0 2024-01-24T23:28:01.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 1 -- 172.21.15.132:0/55847631 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f26f0004c00 con 0x7f270009f9c0 2024-01-24T23:28:01.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.502+0000 7f26e7fff640 1 -- 172.21.15.132:0/55847631 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(231..231 src has 1..231) v4 ==== 10358+0+0 (unknown 1606538321 0 0) 0x7f26f0045270 con 0x7f270009f9c0 2024-01-24T23:28:01.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.506+0000 7f26fe7fc640 10 monclient: get_auth_request con 0x7f26ec037000 auth_method 0 2024-01-24T23:28:01.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.510+0000 7f26e7fff640 1 -- 172.21.15.132:0/55847631 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f26f0014ac0 con 0x7f270009f9c0 2024-01-24T23:28:01.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.510+0000 7f26e7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:01.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.510+0000 7f26e7fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:01.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.666+0000 7f2705974640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h20"}] 2024-01-24T23:28:01.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.666+0000 7f2705974640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h20"}] 2024-01-24T23:28:01.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.666+0000 7f2705974640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:01.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:01.666+0000 7f2705974640 1 -- 172.21.15.132:0/55847631 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rm", "name": "h20"} v 0) v1 -- 0x7f26e0005340 con 0x7f270009f9c0 2024-01-24T23:28:02.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.266+0000 7f26e7fff640 1 -- 172.21.15.132:0/55847631 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h20"}]=0 removed item id -76 name 'h20' from crush map v232) v1 ==== 120+0+0 (unknown 1806001554 0 0) 0x7f26f0012f30 con 0x7f270009f9c0 2024-01-24T23:28:02.267 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.266+0000 7f26e7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h20"}] 2024-01-24T23:28:02.268 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.266+0000 7f26e7fff640 10 monclient: _finish_command 2 = system:0 removed item id -76 name 'h20' from crush map 2024-01-24T23:28:02.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.362+0000 7f2705974640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h20"}] 2024-01-24T23:28:02.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.362+0000 7f2705974640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h20"}] 2024-01-24T23:28:02.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.362+0000 7f2705974640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:02.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.362+0000 7f2705974640 1 -- 172.21.15.132:0/55847631 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rm", "name": "h20"} v 0) v1 -- 0x7f26e0002220 con 0x7f270009f9c0 2024-01-24T23:28:02.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.362+0000 7f26e7fff640 1 -- 172.21.15.132:0/55847631 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h20"}]=0 device 'h20' does not appear in the crush map v232) v1 ==== 120+0+0 (unknown 770842472 0 0) 0x7f26f00146b0 con 0x7f270009f9c0 2024-01-24T23:28:02.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.366+0000 7f26e7fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h20"}] 2024-01-24T23:28:02.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.366+0000 7f26e7fff640 10 monclient: _finish_command 3 = system:0 device 'h20' does not appear in the crush map 2024-01-24T23:28:02.366 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h20' does not appear in the crush map 2024-01-24T23:28:02.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.366+0000 7f2705974640 1 -- 172.21.15.132:0/55847631 >> v1:172.21.15.132:6812/1835401922 conn(0x7f26ec037000 legacy=0x7f26ec0394c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:02.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.366+0000 7f2705974640 10 monclient: shutdown 2024-01-24T23:28:02.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.366+0000 7f2705974640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:02.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.366+0000 7f2705974640 1 -- 172.21.15.132:0/55847631 >> v1:172.21.15.132:6790/0 conn(0x7f270009f9c0 legacy=0x7f270009c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:02.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.366+0000 7f2705974640 1 -- 172.21.15.132:0/55847631 shutdown_connections 2024-01-24T23:28:02.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.366+0000 7f2705974640 1 -- 172.21.15.132:0/55847631 >> 172.21.15.132:0/55847631 conn(0x7f27000a5a10 msgr2=0x7f2700134bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:02.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.366+0000 7f2705974640 1 -- 172.21.15.132:0/55847631 shutdown_connections 2024-01-24T23:28:02.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.366+0000 7f2705974640 1 -- 172.21.15.132:0/55847631 wait complete. 2024-01-24T23:28:02.382 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:28:02.383 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h21 2024-01-24T23:28:02.490 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc5683b6640 10 monclient: get_monmap_and_config 2024-01-24T23:28:02.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc5683b6640 10 monclient: build_initial_monmap 2024-01-24T23:28:02.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc5683b6640 10 monclient: monmap: 2024-01-24T23:28:02.491 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:02.491 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:02.491 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:02.493492+0000 2024-01-24T23:28:02.491 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:02.493492+0000 2024-01-24T23:28:02.491 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:02.491 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:02.491 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:02.491 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:02.491 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:02.491 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:02.492 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:02.492 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:02.492 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:02.492 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:02.492 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:02.492 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:02.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc5683b6640 1 Processor -- start 2024-01-24T23:28:02.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc5683b6640 1 -- start start 2024-01-24T23:28:02.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc5683b6640 10 monclient: init 2024-01-24T23:28:02.499 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc5683b6640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:02.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc5683b6640 10 monclient: _add_conns ranks=[5,6,8,1,7,4,0,2,3] 2024-01-24T23:28:02.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc5683b6640 10 monclient(hunting): picked mon.noname-h con 0x7fc56009bc10 addr v1:172.21.15.132:6791/0 2024-01-24T23:28:02.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc5683b6640 10 monclient(hunting): picked mon.noname-d con 0x7fc5600a00d0 addr v1:172.21.15.115:6792/0 2024-01-24T23:28:02.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc5683b6640 10 monclient(hunting): picked mon.noname-e con 0x7fc5600a0560 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:02.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc5683b6640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc560064700 con 0x7fc56009bc10 2024-01-24T23:28:02.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc5683b6640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc56009bff0 con 0x7fc5600a00d0 2024-01-24T23:28:02.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc5683b6640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5600a1870 con 0x7fc5600a0560 2024-01-24T23:28:02.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc5683b6640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:02.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc5683b6640 10 monclient(hunting): authenticate will time out at 1369.818654s 2024-01-24T23:28:02.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc566152640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fc56009bc10 0x7fc56009f9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:56450/0 (socket says 172.21.15.132:56450) 2024-01-24T23:28:02.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc566152640 1 -- 172.21.15.132:0/3425056448 learned_addr learned my addr 172.21.15.132:0/3425056448 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:02.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 1 -- 172.21.15.132:0/3425056448 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 911031057 0 0) 0x7fc560064700 con 0x7fc56009bc10 2024-01-24T23:28:02.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 10 monclient(hunting): _negotiate 2024-01-24T23:28:02.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:02.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:02.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 10 monclient(hunting): my global_id is 6827 2024-01-24T23:28:02.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 1 -- 172.21.15.132:0/3425056448 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc54c003450 con 0x7fc56009bc10 2024-01-24T23:28:02.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 1 -- 172.21.15.132:0/3425056448 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3893627713 0 0) 0x7fc54c003450 con 0x7fc56009bc10 2024-01-24T23:28:02.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 1 -- 172.21.15.132:0/3425056448 >> v1:172.21.15.115:6793/0 conn(0x7fc5600a0560 legacy=0x7fc56013bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:02.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 1 -- 172.21.15.132:0/3425056448 >> v1:172.21.15.115:6792/0 conn(0x7fc5600a00d0 legacy=0x7fc5601387c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:02.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 10 monclient: _finish_hunting 0 2024-01-24T23:28:02.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 1 monclient: found mon.noname-h 2024-01-24T23:28:02.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:02.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:28:02.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 1 -- 172.21.15.132:0/3425056448 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5600a0fa0 con 0x7fc56009bc10 2024-01-24T23:28:02.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 10 monclient: _finish_auth 0 2024-01-24T23:28:02.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 10 monclient: _check_auth_tickets 2024-01-24T23:28:02.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:02.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 1 -- 172.21.15.132:0/3425056448 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc55c0017c0 con 0x7fc56009bc10 2024-01-24T23:28:02.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:02.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:28:02.502 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 10 monclient: dump: 2024-01-24T23:28:02.502 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:02.503 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:02.503 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:02.503 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:02.503 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:02.503 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:02.503 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:02.503 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:02.503 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:02.503 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:02.503 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:02.503 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:02.503 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:02.504 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:02.504 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:02.504 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:02.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc5683b6640 5 monclient: authenticate success, global_id 6827 2024-01-24T23:28:02.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc5683b6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:02.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 1 -- 172.21.15.132:0/3425056448 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc55c001a70 con 0x7fc56009bc10 2024-01-24T23:28:02.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:02.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc5683b6640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:02.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc5683b6640 4 set_mon_vals no callback set 2024-01-24T23:28:02.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 1 -- 172.21.15.132:0/3425056448 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc55c004bf0 con 0x7fc56009bc10 2024-01-24T23:28:02.504 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:02.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:28:02.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.490+0000 7fc565150640 10 monclient: dump: 2024-01-24T23:28:02.505 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:02.505 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:02.505 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:02.505 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:02.505 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:02.505 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:02.505 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:02.505 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:02.506 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:02.506 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:02.506 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:02.506 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:02.506 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:02.506 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:02.506 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:02.506 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:02.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 10 monclient: shutdown 2024-01-24T23:28:02.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:02.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 1 -- 172.21.15.132:0/3425056448 >> v1:172.21.15.132:6791/0 conn(0x7fc56009bc10 legacy=0x7fc56009f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:02.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 1 -- 172.21.15.132:0/3425056448 shutdown_connections 2024-01-24T23:28:02.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 1 -- 172.21.15.132:0/3425056448 >> 172.21.15.132:0/3425056448 conn(0x7fc5600a5a10 msgr2=0x7fc5600a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:02.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 1 -- 172.21.15.132:0/3425056448 shutdown_connections 2024-01-24T23:28:02.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 1 -- 172.21.15.132:0/3425056448 wait complete. 2024-01-24T23:28:02.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 10 monclient: build_initial_monmap 2024-01-24T23:28:02.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 10 monclient: monmap: 2024-01-24T23:28:02.507 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:02.507 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:02.508 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:02.508 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:02.508 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:02.508 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:02.508 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:02.508 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:02.508 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:02.508 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:02.508 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:02.508 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:02.508 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:02.509 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:02.509 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:02.509 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:02.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 1 Processor -- start 2024-01-24T23:28:02.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 1 -- start start 2024-01-24T23:28:02.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 10 monclient: init 2024-01-24T23:28:02.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:02.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 10 monclient: _add_conns ranks=[0,5,3,4,8,1,6,7,2] 2024-01-24T23:28:02.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 10 monclient(hunting): picked mon.noname-a con 0x7fc56009bc10 addr v1:172.21.15.115:6789/0 2024-01-24T23:28:02.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 10 monclient(hunting): picked mon.noname-h con 0x7fc5600a00d0 addr v1:172.21.15.132:6791/0 2024-01-24T23:28:02.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 10 monclient(hunting): picked mon.noname-g con 0x7fc5600a0560 addr v1:172.21.15.132:6790/0 2024-01-24T23:28:02.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5601c7eb0 con 0x7fc56009bc10 2024-01-24T23:28:02.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5601a2ed0 con 0x7fc5600a0560 2024-01-24T23:28:02.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc56009c650 con 0x7fc5600a00d0 2024-01-24T23:28:02.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:02.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 10 monclient(hunting): authenticate will time out at 1369.821652s 2024-01-24T23:28:02.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc566152640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fc56009bc10 legacy=0x7fc5601b7a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:28:02.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc566953640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fc5600a0560 0x7fc5601cfa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:41974/0 (socket says 172.21.15.132:41974) 2024-01-24T23:28:02.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc566953640 1 -- 172.21.15.132:0/3544696796 learned_addr learned my addr 172.21.15.132:0/3544696796 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:02.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 1 -- 172.21.15.132:0/3544696796 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2458632723 0 0) 0x7fc56009c650 con 0x7fc5600a00d0 2024-01-24T23:28:02.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:28:02.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:02.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:02.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 10 monclient(hunting): my global_id is 6836 2024-01-24T23:28:02.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 1 -- 172.21.15.132:0/3544696796 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc544003530 con 0x7fc5600a00d0 2024-01-24T23:28:02.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 1 -- 172.21.15.132:0/3544696796 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 211543450 0 0) 0x7fc5601a2ed0 con 0x7fc5600a0560 2024-01-24T23:28:02.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:28:02.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:02.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:02.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 10 monclient(hunting): my global_id is 6870 2024-01-24T23:28:02.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 1 -- 172.21.15.132:0/3544696796 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc56009c650 con 0x7fc5600a0560 2024-01-24T23:28:02.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 1 -- 172.21.15.132:0/3544696796 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1055700915 0 0) 0x7fc544003530 con 0x7fc5600a00d0 2024-01-24T23:28:02.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 1 -- 172.21.15.132:0/3544696796 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc5601a2ed0 con 0x7fc5600a00d0 2024-01-24T23:28:02.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 1 -- 172.21.15.132:0/3544696796 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2977421044 0 0) 0x7fc56009c650 con 0x7fc5600a0560 2024-01-24T23:28:02.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 1 -- 172.21.15.132:0/3544696796 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc544003530 con 0x7fc5600a0560 2024-01-24T23:28:02.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 1 -- 172.21.15.132:0/3544696796 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc558001c60 con 0x7fc5600a0560 2024-01-24T23:28:02.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:02.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:28:02.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 10 monclient(hunting): dump: 2024-01-24T23:28:02.513 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:02.513 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:02.513 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:02.513 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:02.513 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:02.513 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:02.513 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:02.513 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:02.513 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:02.513 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:02.513 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:02.513 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:02.514 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:02.514 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:02.514 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:02.514 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:02.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:02.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 1 -- 172.21.15.132:0/3544696796 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc550001a60 con 0x7fc5600a00d0 2024-01-24T23:28:02.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:02.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:28:02.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 10 monclient(hunting): dump: 2024-01-24T23:28:02.514 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:02.514 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:02.515 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:02.515 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:02.515 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:02.515 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:02.515 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:02.515 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:02.515 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:02.515 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:02.515 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:02.516 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:02.516 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:02.516 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:02.516 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:02.516 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:02.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 1 -- 172.21.15.132:0/3544696796 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 774475264 0 0) 0x7fc5601a2ed0 con 0x7fc5600a00d0 2024-01-24T23:28:02.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 1 -- 172.21.15.132:0/3544696796 >> v1:172.21.15.132:6790/0 conn(0x7fc5600a0560 legacy=0x7fc5601cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:02.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 1 -- 172.21.15.132:0/3544696796 >> v1:172.21.15.115:6789/0 conn(0x7fc56009bc10 legacy=0x7fc5601b7a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:28:02.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:28:02.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 1 monclient: found mon.h 2024-01-24T23:28:02.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:02.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:02.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 1 -- 172.21.15.132:0/3544696796 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc560071fe0 con 0x7fc5600a00d0 2024-01-24T23:28:02.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 10 monclient: _finish_auth 0 2024-01-24T23:28:02.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:28:02.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:02.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 5 monclient: authenticate success, global_id 6836 2024-01-24T23:28:02.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 10 monclient: _renew_subs 2024-01-24T23:28:02.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:02.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 1 -- 172.21.15.132:0/3544696796 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5601a4ee0 con 0x7fc5600a00d0 2024-01-24T23:28:02.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 10 monclient: _renew_subs 2024-01-24T23:28:02.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:02.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 1 -- 172.21.15.132:0/3544696796 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5601a3780 con 0x7fc5600a00d0 2024-01-24T23:28:02.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 1 -- 172.21.15.132:0/3544696796 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc550003600 con 0x7fc5600a00d0 2024-01-24T23:28:02.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:02.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 1 -- 172.21.15.132:0/3544696796 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc550003ab0 con 0x7fc5600a00d0 2024-01-24T23:28:02.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:02.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:28:02.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc557fff640 4 set_mon_vals no callback set 2024-01-24T23:28:02.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 10 monclient: dump: 2024-01-24T23:28:02.518 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:02.518 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:02.518 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:02.518 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:02.518 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:02.518 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:02.519 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:02.519 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:02.519 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:02.519 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:02.519 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:02.519 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:02.519 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:02.519 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:02.519 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:02.519 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:02.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:02.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:02.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:02.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc5683b6640 1 -- 172.21.15.132:0/3544696796 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc548005090 con 0x7fc5600a00d0 2024-01-24T23:28:02.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 1 -- 172.21.15.132:0/3544696796 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fc55000e530 con 0x7fc5600a00d0 2024-01-24T23:28:02.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc556ffd640 1 -- 172.21.15.132:0/3544696796 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(232..232 src has 1..232) v4 ==== 10145+0+0 (unknown 2230408521 0 0) 0x7fc550043cc0 con 0x7fc5600a00d0 2024-01-24T23:28:02.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.494+0000 7fc566152640 10 monclient: get_auth_request con 0x7fc5440374c0 auth_method 0 2024-01-24T23:28:02.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.498+0000 7fc556ffd640 1 -- 172.21.15.132:0/3544696796 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fc550013430 con 0x7fc5600a00d0 2024-01-24T23:28:02.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.498+0000 7fc556ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:02.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.498+0000 7fc556ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:02.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.658+0000 7fc5683b6640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h21"}] 2024-01-24T23:28:02.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.658+0000 7fc5683b6640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h21"}] 2024-01-24T23:28:02.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.658+0000 7fc5683b6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:02.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:02.658+0000 7fc5683b6640 1 -- 172.21.15.132:0/3544696796 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h21"} v 0) v1 -- 0x7fc548005340 con 0x7fc5600a00d0 2024-01-24T23:28:03.174 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~119s 2024-01-24T23:28:03.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.274+0000 7fc556ffd640 1 -- 172.21.15.132:0/3544696796 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h21"}]=0 removed item id -79 name 'h21' from crush map v233) v1 ==== 120+0+0 (unknown 1050206506 0 0) 0x7fc550011ad0 con 0x7fc5600a00d0 2024-01-24T23:28:03.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.274+0000 7fc556ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h21"}] 2024-01-24T23:28:03.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.274+0000 7fc556ffd640 10 monclient: _finish_command 2 = system:0 removed item id -79 name 'h21' from crush map 2024-01-24T23:28:03.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.374+0000 7fc5683b6640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h21"}] 2024-01-24T23:28:03.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.374+0000 7fc5683b6640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h21"}] 2024-01-24T23:28:03.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.374+0000 7fc5683b6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:03.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.374+0000 7fc5683b6640 1 -- 172.21.15.132:0/3544696796 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h21"} v 0) v1 -- 0x7fc5480010a0 con 0x7fc5600a00d0 2024-01-24T23:28:03.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.374+0000 7fc556ffd640 1 -- 172.21.15.132:0/3544696796 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h21"}]=0 device 'h21' does not appear in the crush map v233) v1 ==== 120+0+0 (unknown 2580166599 0 0) 0x7fc550013250 con 0x7fc5600a00d0 2024-01-24T23:28:03.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.374+0000 7fc556ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h21"}] 2024-01-24T23:28:03.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.374+0000 7fc556ffd640 10 monclient: _finish_command 3 = system:0 device 'h21' does not appear in the crush map 2024-01-24T23:28:03.377 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h21' does not appear in the crush map 2024-01-24T23:28:03.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.378+0000 7fc5683b6640 1 -- 172.21.15.132:0/3544696796 >> v1:172.21.15.132:6812/1835401922 conn(0x7fc5440374c0 legacy=0x7fc544039980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:03.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.378+0000 7fc5683b6640 10 monclient: shutdown 2024-01-24T23:28:03.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.378+0000 7fc5683b6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:03.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.378+0000 7fc5683b6640 1 -- 172.21.15.132:0/3544696796 >> v1:172.21.15.132:6791/0 conn(0x7fc5600a00d0 legacy=0x7fc5601cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:03.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.378+0000 7fc5683b6640 1 -- 172.21.15.132:0/3544696796 shutdown_connections 2024-01-24T23:28:03.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.378+0000 7fc5683b6640 1 -- 172.21.15.132:0/3544696796 >> 172.21.15.132:0/3544696796 conn(0x7fc5600a5a10 msgr2=0x7fc5601ba740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:03.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.378+0000 7fc5683b6640 1 -- 172.21.15.132:0/3544696796 shutdown_connections 2024-01-24T23:28:03.380 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.378+0000 7fc5683b6640 1 -- 172.21.15.132:0/3544696796 wait complete. 2024-01-24T23:28:03.394 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:28:03.394 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h22 2024-01-24T23:28:03.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.494+0000 7f252b593640 10 monclient: get_monmap_and_config 2024-01-24T23:28:03.496 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.494+0000 7f252b593640 10 monclient: build_initial_monmap 2024-01-24T23:28:03.497 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.494+0000 7f252b593640 10 monclient: monmap: 2024-01-24T23:28:03.497 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:03.497 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:03.497 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:03.499577+0000 2024-01-24T23:28:03.497 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:03.499577+0000 2024-01-24T23:28:03.497 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:03.497 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:03.497 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:03.497 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:03.497 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:03.497 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:03.497 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:03.497 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:03.497 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:03.498 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:03.498 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:03.498 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:03.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.494+0000 7f252b593640 1 Processor -- start 2024-01-24T23:28:03.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.494+0000 7f252b593640 1 -- start start 2024-01-24T23:28:03.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.494+0000 7f252b593640 10 monclient: init 2024-01-24T23:28:03.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:03.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 10 monclient: _add_conns ranks=[1,5,2,7,8,4,6,3,0] 2024-01-24T23:28:03.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 10 monclient(hunting): picked mon.noname-f con 0x7f252409c4c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:28:03.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 10 monclient(hunting): picked mon.noname-h con 0x7f252409cfb0 addr v1:172.21.15.132:6791/0 2024-01-24T23:28:03.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 10 monclient(hunting): picked mon.noname-b con 0x7f252409ec50 addr v1:172.21.15.115:6790/0 2024-01-24T23:28:03.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2524064700 con 0x7f252409c4c0 2024-01-24T23:28:03.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f252409d9f0 con 0x7f252409ec50 2024-01-24T23:28:03.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f252409dc60 con 0x7f252409cfb0 2024-01-24T23:28:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 10 monclient(hunting): authenticate will time out at 1370.824386s 2024-01-24T23:28:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f2528b2e640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f252409cfb0 0x7f2524138d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:56498/0 (socket says 172.21.15.132:56498) 2024-01-24T23:28:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f2528b2e640 1 -- 172.21.15.132:0/3035563992 learned_addr learned my addr 172.21.15.132:0/3035563992 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252932f640 1 --1- 172.21.15.132:0/3035563992 >> v1:172.21.15.132:6789/0 conn(0x7f252409c4c0 0x7f252409c8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:46048/0 (socket says 172.21.15.132:46048) 2024-01-24T23:28:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f2529b30640 1 -- 172.21.15.132:0/3035563992 >> v1:172.21.15.115:6790/0 conn(0x7f252409ec50 legacy=0x7f252413c460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:28:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 1 -- 172.21.15.132:0/3035563992 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4000281912 0 0) 0x7f252409dc60 con 0x7f252409cfb0 2024-01-24T23:28:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:28:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 10 monclient(hunting): my global_id is 6845 2024-01-24T23:28:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 1 -- 172.21.15.132:0/3035563992 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2510003530 con 0x7f252409cfb0 2024-01-24T23:28:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 1 -- 172.21.15.132:0/3035563992 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3089767613 0 0) 0x7f2524064700 con 0x7f252409c4c0 2024-01-24T23:28:03.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:28:03.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:03.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:03.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 10 monclient(hunting): my global_id is 124435 2024-01-24T23:28:03.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 1 -- 172.21.15.132:0/3035563992 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f252409dc60 con 0x7f252409c4c0 2024-01-24T23:28:03.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 1 -- 172.21.15.132:0/3035563992 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3115805985 0 0) 0x7f2510003530 con 0x7f252409cfb0 2024-01-24T23:28:03.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 1 -- 172.21.15.132:0/3035563992 >> v1:172.21.15.115:6790/0 conn(0x7f252409ec50 legacy=0x7f252413c460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:28:03.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 1 -- 172.21.15.132:0/3035563992 >> v1:172.21.15.132:6789/0 conn(0x7f252409c4c0 legacy=0x7f252409c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:03.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:28:03.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 1 monclient: found mon.noname-h 2024-01-24T23:28:03.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:03.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:28:03.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 1 -- 172.21.15.132:0/3035563992 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252409df40 con 0x7f252409cfb0 2024-01-24T23:28:03.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 10 monclient: _finish_auth 0 2024-01-24T23:28:03.507 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:28:03.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:03.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 5 monclient: authenticate success, global_id 6845 2024-01-24T23:28:03.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:03.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 1 -- 172.21.15.132:0/3035563992 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2520001b40 con 0x7f252409cfb0 2024-01-24T23:28:03.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:03.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:28:03.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 10 monclient: dump: 2024-01-24T23:28:03.508 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:03.508 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:03.508 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:03.508 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:03.508 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:03.508 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:03.508 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:03.508 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:03.509 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:03.509 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:03.509 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:03.509 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:03.509 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:03.509 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:03.509 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:03.509 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:03.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:03.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 1 -- 172.21.15.132:0/3035563992 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2520001e20 con 0x7f252409cfb0 2024-01-24T23:28:03.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:03.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:03.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 1 -- 172.21.15.132:0/3035563992 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2520004f70 con 0x7f252409cfb0 2024-01-24T23:28:03.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 4 set_mon_vals no callback set 2024-01-24T23:28:03.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:03.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:28:03.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f251bfff640 10 monclient: dump: 2024-01-24T23:28:03.510 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:03.510 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:03.510 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:03.510 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:03.510 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:03.510 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:03.510 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:03.510 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:03.510 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:03.510 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:03.510 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:03.510 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:03.510 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:03.510 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:03.510 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:03.510 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:03.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 10 monclient: shutdown 2024-01-24T23:28:03.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:03.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 1 -- 172.21.15.132:0/3035563992 >> v1:172.21.15.132:6791/0 conn(0x7f252409cfb0 legacy=0x7f2524138d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:03.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 1 -- 172.21.15.132:0/3035563992 shutdown_connections 2024-01-24T23:28:03.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 1 -- 172.21.15.132:0/3035563992 >> 172.21.15.132:0/3035563992 conn(0x7f25240a5a10 msgr2=0x7f25240a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:03.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 1 -- 172.21.15.132:0/3035563992 shutdown_connections 2024-01-24T23:28:03.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 1 -- 172.21.15.132:0/3035563992 wait complete. 2024-01-24T23:28:03.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 10 monclient: build_initial_monmap 2024-01-24T23:28:03.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 10 monclient: monmap: 2024-01-24T23:28:03.511 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:03.511 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:03.511 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:03.511 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:03.511 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:03.511 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:03.511 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:03.512 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:03.512 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:03.512 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:03.512 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:03.512 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:03.512 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:03.512 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:03.512 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:03.512 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:03.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 1 Processor -- start 2024-01-24T23:28:03.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 1 -- start start 2024-01-24T23:28:03.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 10 monclient: init 2024-01-24T23:28:03.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:03.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 10 monclient: _add_conns ranks=[4,7,2,5,0,6,3,1,8] 2024-01-24T23:28:03.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 10 monclient(hunting): picked mon.noname-c con 0x7f252409c4c0 addr v1:172.21.15.115:6791/0 2024-01-24T23:28:03.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 10 monclient(hunting): picked mon.noname-i con 0x7f252409cfb0 addr v1:172.21.15.132:6792/0 2024-01-24T23:28:03.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 10 monclient(hunting): picked mon.noname-b con 0x7f252409ec50 addr v1:172.21.15.115:6790/0 2024-01-24T23:28:03.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f252409c020 con 0x7f252409ec50 2024-01-24T23:28:03.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f252411ca70 con 0x7f252409c4c0 2024-01-24T23:28:03.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25240716a0 con 0x7f252409cfb0 2024-01-24T23:28:03.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:03.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252b593640 10 monclient(hunting): authenticate will time out at 1370.827151s 2024-01-24T23:28:03.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f2528b2e640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f252409cfb0 legacy=0x7f25241cc100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:28:03.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f2529b30640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f252409ec50 legacy=0x7f25241cf830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:28:03.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252932f640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f252409c4c0 0x7f252412d5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:35092/0 (socket says 172.21.15.132:35092) 2024-01-24T23:28:03.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f252932f640 1 -- 172.21.15.132:0/593404473 learned_addr learned my addr 172.21.15.132:0/593404473 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:03.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f2519ffb640 1 -- 172.21.15.132:0/593404473 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720310130 0 0) 0x7f252411ca70 con 0x7f252409c4c0 2024-01-24T23:28:03.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f2519ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:28:03.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f2519ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:03.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f2519ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:03.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f2519ffb640 10 monclient(hunting): my global_id is 95008 2024-01-24T23:28:03.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.498+0000 7f2519ffb640 1 -- 172.21.15.132:0/593404473 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2508003530 con 0x7f252409c4c0 2024-01-24T23:28:03.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 1 -- 172.21.15.132:0/593404473 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 208327032 0 0) 0x7f2508003530 con 0x7f252409c4c0 2024-01-24T23:28:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 1 -- 172.21.15.132:0/593404473 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f252411ca70 con 0x7f252409c4c0 2024-01-24T23:28:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 1 -- 172.21.15.132:0/593404473 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f251c001c20 con 0x7f252409c4c0 2024-01-24T23:28:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:28:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 10 monclient(hunting): dump: 2024-01-24T23:28:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:03.517 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:03.518 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 1 -- 172.21.15.132:0/593404473 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1009384880 0 0) 0x7f252411ca70 con 0x7f252409c4c0 2024-01-24T23:28:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 1 -- 172.21.15.132:0/593404473 >> v1:172.21.15.132:6792/0 conn(0x7f252409cfb0 legacy=0x7f25241cc100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:28:03.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 1 -- 172.21.15.132:0/593404473 >> v1:172.21.15.115:6790/0 conn(0x7f252409ec50 legacy=0x7f25241cf830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:28:03.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:28:03.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 1 monclient: found mon.c 2024-01-24T23:28:03.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:03.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:28:03.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 1 -- 172.21.15.132:0/593404473 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252411c700 con 0x7f252409c4c0 2024-01-24T23:28:03.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 10 monclient: _finish_auth 0 2024-01-24T23:28:03.519 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:28:03.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:03.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f252b593640 5 monclient: authenticate success, global_id 95008 2024-01-24T23:28:03.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f252b593640 10 monclient: _renew_subs 2024-01-24T23:28:03.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f252b593640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:28:03.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f252b593640 1 -- 172.21.15.132:0/593404473 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f252412dcd0 con 0x7f252409c4c0 2024-01-24T23:28:03.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f252b593640 10 monclient: _renew_subs 2024-01-24T23:28:03.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f252b593640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:28:03.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f252b593640 1 -- 172.21.15.132:0/593404473 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f252412dfc0 con 0x7f252409c4c0 2024-01-24T23:28:03.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 1 -- 172.21.15.132:0/593404473 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f251c004aa0 con 0x7f252409c4c0 2024-01-24T23:28:03.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:03.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 1 -- 172.21.15.132:0/593404473 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f251c004f50 con 0x7f252409c4c0 2024-01-24T23:28:03.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:03.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:28:03.520 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f251b7fe640 4 set_mon_vals no callback set 2024-01-24T23:28:03.521 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 10 monclient: dump: 2024-01-24T23:28:03.521 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:03.521 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:03.521 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:03.521 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:03.521 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:03.521 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:03.521 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:03.521 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:03.521 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:03.521 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:03.521 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:03.521 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:03.521 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:03.521 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:03.522 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:03.522 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:03.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f252b593640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:03.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f252b593640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:03.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f252b593640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:28:03.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f252b593640 1 -- 172.21.15.132:0/593404473 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2504005090 con 0x7f252409c4c0 2024-01-24T23:28:03.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 1 -- 172.21.15.132:0/593404473 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f251c00fa50 con 0x7f252409c4c0 2024-01-24T23:28:03.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2519ffb640 1 -- 172.21.15.132:0/593404473 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(233..233 src has 1..233) v4 ==== 9932+0+0 (unknown 2779522759 0 0) 0x7f251c045050 con 0x7f252409c4c0 2024-01-24T23:28:03.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.502+0000 7f2528b2e640 10 monclient: get_auth_request con 0x7f2508036fe0 auth_method 0 2024-01-24T23:28:03.522 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.514+0000 7f2519ffb640 1 -- 172.21.15.132:0/593404473 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f251c0146d0 con 0x7f252409c4c0 2024-01-24T23:28:03.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.514+0000 7f2519ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:03.523 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.514+0000 7f2519ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:03.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.662+0000 7f252b593640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h22"}] 2024-01-24T23:28:03.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.662+0000 7f252b593640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h22"}] 2024-01-24T23:28:03.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.662+0000 7f252b593640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:28:03.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:03.662+0000 7f252b593640 1 -- 172.21.15.132:0/593404473 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h22"} v 0) v1 -- 0x7f2504005340 con 0x7f252409c4c0 2024-01-24T23:28:04.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.298+0000 7f2519ffb640 1 -- 172.21.15.132:0/593404473 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h22"}]=0 removed item id -82 name 'h22' from crush map v234) v1 ==== 120+0+0 (unknown 1428331777 0 0) 0x7f251c0146d0 con 0x7f252409c4c0 2024-01-24T23:28:04.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.298+0000 7f2519ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h22"}] 2024-01-24T23:28:04.300 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.298+0000 7f2519ffb640 10 monclient: _finish_command 2 = system:0 removed item id -82 name 'h22' from crush map 2024-01-24T23:28:04.398 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.398+0000 7f252b593640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h22"}] 2024-01-24T23:28:04.398 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.398+0000 7f252b593640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h22"}] 2024-01-24T23:28:04.398 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.398+0000 7f252b593640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:28:04.398 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.398+0000 7f252b593640 1 -- 172.21.15.132:0/593404473 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h22"} v 0) v1 -- 0x7f25040010a0 con 0x7f252409c4c0 2024-01-24T23:28:04.400 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.398+0000 7f2519ffb640 1 -- 172.21.15.132:0/593404473 <== mon.4 v1:172.21.15.115:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h22"}]=0 device 'h22' does not appear in the crush map v234) v1 ==== 120+0+0 (unknown 1080965319 0 0) 0x7f251c0148b0 con 0x7f252409c4c0 2024-01-24T23:28:04.401 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.398+0000 7f2519ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h22"}] 2024-01-24T23:28:04.401 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.398+0000 7f2519ffb640 10 monclient: _finish_command 3 = system:0 device 'h22' does not appear in the crush map 2024-01-24T23:28:04.401 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h22' does not appear in the crush map 2024-01-24T23:28:04.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.402+0000 7f252b593640 1 -- 172.21.15.132:0/593404473 >> v1:172.21.15.132:6812/1835401922 conn(0x7f2508036fe0 legacy=0x7f25080394a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:04.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.402+0000 7f252b593640 10 monclient: shutdown 2024-01-24T23:28:04.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.402+0000 7f252b593640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:04.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.402+0000 7f252b593640 1 -- 172.21.15.132:0/593404473 >> v1:172.21.15.115:6791/0 conn(0x7f252409c4c0 legacy=0x7f252412d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:04.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.402+0000 7f252b593640 1 -- 172.21.15.132:0/593404473 shutdown_connections 2024-01-24T23:28:04.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.402+0000 7f252b593640 1 -- 172.21.15.132:0/593404473 >> 172.21.15.132:0/593404473 conn(0x7f25240a5a10 msgr2=0x7f25240a3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:04.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.402+0000 7f252b593640 1 -- 172.21.15.132:0/593404473 shutdown_connections 2024-01-24T23:28:04.403 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.402+0000 7f252b593640 1 -- 172.21.15.132:0/593404473 wait complete. 2024-01-24T23:28:04.417 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:28:04.418 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h23 2024-01-24T23:28:04.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.530+0000 7f2f5946e640 10 monclient: get_monmap_and_config 2024-01-24T23:28:04.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.530+0000 7f2f5946e640 10 monclient: build_initial_monmap 2024-01-24T23:28:04.532 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.530+0000 7f2f5946e640 10 monclient: monmap: 2024-01-24T23:28:04.532 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:04.532 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:04.532 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:04.535207+0000 2024-01-24T23:28:04.533 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:04.535207+0000 2024-01-24T23:28:04.533 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:04.533 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:04.533 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:04.533 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:04.533 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:04.533 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:04.533 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:04.533 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:04.533 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:04.533 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:04.533 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:04.533 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:04.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.530+0000 7f2f5946e640 1 Processor -- start 2024-01-24T23:28:04.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 1 -- start start 2024-01-24T23:28:04.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 10 monclient: init 2024-01-24T23:28:04.538 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:04.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 10 monclient: _add_conns ranks=[7,5,6,8,1,2,4,0,3] 2024-01-24T23:28:04.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 10 monclient(hunting): picked mon.noname-i con 0x7f2f54131ca0 addr v1:172.21.15.132:6792/0 2024-01-24T23:28:04.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 10 monclient(hunting): picked mon.noname-h con 0x7f2f54132080 addr v1:172.21.15.132:6791/0 2024-01-24T23:28:04.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 10 monclient(hunting): picked mon.noname-d con 0x7f2f54132500 addr v1:172.21.15.115:6792/0 2024-01-24T23:28:04.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f54064700 con 0x7f2f54132080 2024-01-24T23:28:04.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f54072680 con 0x7f2f54132500 2024-01-24T23:28:04.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f52ffd640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f2f54131ca0 legacy=0x7f2f54071480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:28:04.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f54071060 con 0x7f2f54131ca0 2024-01-24T23:28:04.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:04.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 10 monclient(hunting): authenticate will time out at 1371.860973s 2024-01-24T23:28:04.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f4bfff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f2f54132080 0x7f2f54139da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:48692/0 (socket says 172.21.15.132:48692) 2024-01-24T23:28:04.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f4bfff640 1 -- 172.21.15.132:0/298591154 learned_addr learned my addr 172.21.15.132:0/298591154 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:04.539 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 1 -- 172.21.15.132:0/298591154 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3451993312 0 0) 0x7f2f54064700 con 0x7f2f54132080 2024-01-24T23:28:04.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 10 monclient(hunting): _negotiate 2024-01-24T23:28:04.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:04.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:04.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 10 monclient(hunting): my global_id is 6854 2024-01-24T23:28:04.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 1 -- 172.21.15.132:0/298591154 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f40003530 con 0x7f2f54132080 2024-01-24T23:28:04.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 1 -- 172.21.15.132:0/298591154 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 65337909 0 0) 0x7f2f40003530 con 0x7f2f54132080 2024-01-24T23:28:04.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 1 -- 172.21.15.132:0/298591154 >> v1:172.21.15.132:6792/0 conn(0x7f2f54131ca0 legacy=0x7f2f54071480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:28:04.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 1 -- 172.21.15.132:0/298591154 >> v1:172.21.15.115:6792/0 conn(0x7f2f54132500 legacy=0x7f2f5413d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-24T23:28:04.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 10 monclient: _finish_hunting 0 2024-01-24T23:28:04.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 1 monclient: found mon.noname-h 2024-01-24T23:28:04.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:04.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:28:04.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 1 -- 172.21.15.132:0/298591154 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f54070aa0 con 0x7f2f54132080 2024-01-24T23:28:04.540 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 10 monclient: _finish_auth 0 2024-01-24T23:28:04.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 10 monclient: _check_auth_tickets 2024-01-24T23:28:04.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:04.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 5 monclient: authenticate success, global_id 6854 2024-01-24T23:28:04.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 1 -- 172.21.15.132:0/298591154 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2f440017c0 con 0x7f2f54132080 2024-01-24T23:28:04.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:04.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:04.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:28:04.541 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 10 monclient: dump: 2024-01-24T23:28:04.541 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:04.541 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:04.541 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:04.541 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:04.541 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:04.541 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:04.541 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:04.542 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:04.542 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:04.542 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:04.542 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:04.542 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:04.542 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:04.542 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:04.542 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:04.542 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:04.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:04.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 1 -- 172.21.15.132:0/298591154 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2f44001ab0 con 0x7f2f54132080 2024-01-24T23:28:04.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:04.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:04.542 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 4 set_mon_vals no callback set 2024-01-24T23:28:04.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 1 -- 172.21.15.132:0/298591154 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2f44004bf0 con 0x7f2f54132080 2024-01-24T23:28:04.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:04.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:28:04.543 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f527fc640 10 monclient: dump: 2024-01-24T23:28:04.543 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:04.543 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:04.543 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:04.543 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:04.543 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:04.543 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:04.543 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:04.543 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:04.543 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:04.543 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:04.544 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:04.544 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:04.544 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:04.544 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:04.544 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:04.544 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:04.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 10 monclient: shutdown 2024-01-24T23:28:04.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:04.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 1 -- 172.21.15.132:0/298591154 >> v1:172.21.15.132:6791/0 conn(0x7f2f54132080 legacy=0x7f2f54139da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:04.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 1 -- 172.21.15.132:0/298591154 shutdown_connections 2024-01-24T23:28:04.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 1 -- 172.21.15.132:0/298591154 >> 172.21.15.132:0/298591154 conn(0x7f2f540a7150 msgr2=0x7f2f540a7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:04.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 1 -- 172.21.15.132:0/298591154 shutdown_connections 2024-01-24T23:28:04.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 1 -- 172.21.15.132:0/298591154 wait complete. 2024-01-24T23:28:04.544 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 10 monclient: build_initial_monmap 2024-01-24T23:28:04.545 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 10 monclient: monmap: 2024-01-24T23:28:04.545 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:04.545 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:04.545 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:04.545 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:04.545 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:04.545 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:04.545 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:04.545 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:04.545 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:04.545 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:04.545 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:04.545 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:04.545 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:04.545 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:04.546 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:04.546 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:04.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 1 Processor -- start 2024-01-24T23:28:04.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 1 -- start start 2024-01-24T23:28:04.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 10 monclient: init 2024-01-24T23:28:04.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:04.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 10 monclient: _add_conns ranks=[6,7,8,1,5,0,2,4,3] 2024-01-24T23:28:04.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 10 monclient(hunting): picked mon.noname-d con 0x7f2f54131ca0 addr v1:172.21.15.115:6792/0 2024-01-24T23:28:04.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 10 monclient(hunting): picked mon.noname-i con 0x7f2f54132080 addr v1:172.21.15.132:6792/0 2024-01-24T23:28:04.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 10 monclient(hunting): picked mon.noname-e con 0x7f2f54131270 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:04.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f5406fd00 con 0x7f2f54131ca0 2024-01-24T23:28:04.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f540a0140 con 0x7f2f54132080 2024-01-24T23:28:04.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f5406c420 con 0x7f2f54131270 2024-01-24T23:28:04.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:04.546 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f5946e640 10 monclient(hunting): authenticate will time out at 1371.863784s 2024-01-24T23:28:04.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.534+0000 7f2f4bfff640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f2f54132080 legacy=0x7f2f5406bd10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:28:04.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f52ffd640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f2f54131ca0 0x7f2f5406b600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:57820/0 (socket says 172.21.15.132:57820) 2024-01-24T23:28:04.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f52ffd640 1 -- 172.21.15.132:0/1313285348 learned_addr learned my addr 172.21.15.132:0/1313285348 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:04.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f537fe640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f2f54131270 0x7f2f541d3bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:56678/0 (socket says 172.21.15.132:56678) 2024-01-24T23:28:04.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 1 -- 172.21.15.132:0/1313285348 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1286642018 0 0) 0x7f2f5406fd00 con 0x7f2f54131ca0 2024-01-24T23:28:04.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:28:04.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:04.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:04.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 10 monclient(hunting): my global_id is 7071 2024-01-24T23:28:04.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 1 -- 172.21.15.132:0/1313285348 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f34003530 con 0x7f2f54131ca0 2024-01-24T23:28:04.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 1 -- 172.21.15.132:0/1313285348 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1738400067 0 0) 0x7f2f5406c420 con 0x7f2f54131270 2024-01-24T23:28:04.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:28:04.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:04.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:04.547 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 10 monclient(hunting): my global_id is 86030 2024-01-24T23:28:04.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 1 -- 172.21.15.132:0/1313285348 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f5406fd00 con 0x7f2f54131270 2024-01-24T23:28:04.550 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 1 -- 172.21.15.132:0/1313285348 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 675844704 0 0) 0x7f2f34003530 con 0x7f2f54131ca0 2024-01-24T23:28:04.550 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 1 -- 172.21.15.132:0/1313285348 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2f5406c420 con 0x7f2f54131ca0 2024-01-24T23:28:04.550 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 1 -- 172.21.15.132:0/1313285348 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1102106024 0 0) 0x7f2f5406fd00 con 0x7f2f54131270 2024-01-24T23:28:04.550 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 1 -- 172.21.15.132:0/1313285348 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2f34003530 con 0x7f2f54131270 2024-01-24T23:28:04.550 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 1 -- 172.21.15.132:0/1313285348 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2f3c004230 con 0x7f2f54131ca0 2024-01-24T23:28:04.550 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:04.550 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:28:04.550 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 10 monclient(hunting): dump: 2024-01-24T23:28:04.550 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:04.550 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:04.551 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:04.551 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:04.551 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:04.551 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:04.551 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:04.551 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:04.551 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:04.551 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:04.551 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:04.551 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:04.551 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:04.551 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:04.551 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:04.551 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:04.551 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:04.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 1 -- 172.21.15.132:0/1313285348 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2f4c001bb0 con 0x7f2f54131270 2024-01-24T23:28:04.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:04.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:28:04.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 10 monclient(hunting): dump: 2024-01-24T23:28:04.552 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:04.552 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:04.552 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:04.552 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:04.552 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:04.552 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:04.552 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:04.552 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:04.552 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:04.552 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:04.553 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:04.553 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:04.553 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:04.553 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:04.553 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:04.553 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:04.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 1 -- 172.21.15.132:0/1313285348 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2762310614 0 0) 0x7f2f5406c420 con 0x7f2f54131ca0 2024-01-24T23:28:04.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 1 -- 172.21.15.132:0/1313285348 >> v1:172.21.15.115:6793/0 conn(0x7f2f54131270 legacy=0x7f2f541d3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:04.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 1 -- 172.21.15.132:0/1313285348 >> v1:172.21.15.132:6792/0 conn(0x7f2f54132080 legacy=0x7f2f5406bd10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:28:04.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:28:04.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 1 monclient: found mon.d 2024-01-24T23:28:04.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:04.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:28:04.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 1 -- 172.21.15.132:0/1313285348 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5406d100 con 0x7f2f54131ca0 2024-01-24T23:28:04.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 10 monclient: _finish_auth 0 2024-01-24T23:28:04.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:28:04.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:04.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f5946e640 5 monclient: authenticate success, global_id 7071 2024-01-24T23:28:04.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f5946e640 10 monclient: _renew_subs 2024-01-24T23:28:04.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f5946e640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:28:04.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f5946e640 1 -- 172.21.15.132:0/1313285348 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f5406fa40 con 0x7f2f54131ca0 2024-01-24T23:28:04.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f5946e640 10 monclient: _renew_subs 2024-01-24T23:28:04.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f5946e640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:28:04.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f5946e640 1 -- 172.21.15.132:0/1313285348 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f540a0d80 con 0x7f2f54131ca0 2024-01-24T23:28:04.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 1 -- 172.21.15.132:0/1313285348 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2f3c004490 con 0x7f2f54131ca0 2024-01-24T23:28:04.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:04.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 1 -- 172.21.15.132:0/1313285348 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2f3c0049c0 con 0x7f2f54131ca0 2024-01-24T23:28:04.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:04.554 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:28:04.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f51ffb640 4 set_mon_vals no callback set 2024-01-24T23:28:04.555 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 10 monclient: dump: 2024-01-24T23:28:04.555 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:04.555 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:04.555 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:04.555 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:04.555 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:04.555 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:04.555 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:04.555 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:04.555 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:04.555 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:04.555 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:04.555 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:04.555 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:04.556 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:04.556 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:04.556 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:04.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f5946e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:04.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f5946e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:04.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f5946e640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:28:04.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f5946e640 1 -- 172.21.15.132:0/1313285348 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f38005090 con 0x7f2f54131ca0 2024-01-24T23:28:04.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 1 -- 172.21.15.132:0/1313285348 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f2f3c004ba0 con 0x7f2f54131ca0 2024-01-24T23:28:04.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4b7fe640 1 -- 172.21.15.132:0/1313285348 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(234..234 src has 1..234) v4 ==== 9719+0+0 (unknown 2952397306 0 0) 0x7f2f3c044e50 con 0x7f2f54131ca0 2024-01-24T23:28:04.556 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.538+0000 7f2f4bfff640 10 monclient: get_auth_request con 0x7f2f34037670 auth_method 0 2024-01-24T23:28:04.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.554+0000 7f2f4b7fe640 1 -- 172.21.15.132:0/1313285348 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f2f3c014980 con 0x7f2f54131ca0 2024-01-24T23:28:04.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.554+0000 7f2f4b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:04.557 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.554+0000 7f2f4b7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:04.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.710+0000 7f2f5946e640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h23"}] 2024-01-24T23:28:04.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.710+0000 7f2f5946e640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h23"}] 2024-01-24T23:28:04.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.710+0000 7f2f5946e640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:28:04.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:04.710+0000 7f2f5946e640 1 -- 172.21.15.132:0/1313285348 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "h23"} v 0) v1 -- 0x7f2f38005340 con 0x7f2f54131ca0 2024-01-24T23:28:05.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.326+0000 7f2f4b7fe640 1 -- 172.21.15.132:0/1313285348 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h23"}]=0 removed item id -85 name 'h23' from crush map v235) v1 ==== 120+0+0 (unknown 1651143530 0 0) 0x7f2f3c012df0 con 0x7f2f54131ca0 2024-01-24T23:28:05.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.326+0000 7f2f4b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h23"}] 2024-01-24T23:28:05.326 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.326+0000 7f2f4b7fe640 10 monclient: _finish_command 2 = system:0 removed item id -85 name 'h23' from crush map 2024-01-24T23:28:05.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.426+0000 7f2f5946e640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h23"}] 2024-01-24T23:28:05.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.426+0000 7f2f5946e640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h23"}] 2024-01-24T23:28:05.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.426+0000 7f2f5946e640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:28:05.429 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.426+0000 7f2f5946e640 1 -- 172.21.15.132:0/1313285348 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "h23"} v 0) v1 -- 0x7f2f38002220 con 0x7f2f54131ca0 2024-01-24T23:28:05.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.430+0000 7f2f4b7fe640 1 -- 172.21.15.132:0/1313285348 <== mon.6 v1:172.21.15.115:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h23"}]=0 device 'h23' does not appear in the crush map v235) v1 ==== 120+0+0 (unknown 4099281000 0 0) 0x7f2f3c014570 con 0x7f2f54131ca0 2024-01-24T23:28:05.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.430+0000 7f2f4b7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h23"}] 2024-01-24T23:28:05.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.430+0000 7f2f4b7fe640 10 monclient: _finish_command 3 = system:0 device 'h23' does not appear in the crush map 2024-01-24T23:28:05.431 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h23' does not appear in the crush map 2024-01-24T23:28:05.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.430+0000 7f2f5946e640 1 -- 172.21.15.132:0/1313285348 >> v1:172.21.15.132:6812/1835401922 conn(0x7f2f34037670 legacy=0x7f2f34039b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:05.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.430+0000 7f2f5946e640 10 monclient: shutdown 2024-01-24T23:28:05.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.430+0000 7f2f5946e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:05.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.430+0000 7f2f5946e640 1 -- 172.21.15.132:0/1313285348 >> v1:172.21.15.115:6792/0 conn(0x7f2f54131ca0 legacy=0x7f2f5406b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:05.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.430+0000 7f2f5946e640 1 -- 172.21.15.132:0/1313285348 shutdown_connections 2024-01-24T23:28:05.433 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.430+0000 7f2f5946e640 1 -- 172.21.15.132:0/1313285348 >> 172.21.15.132:0/1313285348 conn(0x7f2f540a7150 msgr2=0x7f2f540a3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:05.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.430+0000 7f2f5946e640 1 -- 172.21.15.132:0/1313285348 shutdown_connections 2024-01-24T23:28:05.434 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.430+0000 7f2f5946e640 1 -- 172.21.15.132:0/1313285348 wait complete. 2024-01-24T23:28:05.448 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:28:05.449 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h24 2024-01-24T23:28:05.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.562+0000 7f21ec321640 10 monclient: get_monmap_and_config 2024-01-24T23:28:05.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.562+0000 7f21ec321640 10 monclient: build_initial_monmap 2024-01-24T23:28:05.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.562+0000 7f21ec321640 10 monclient: monmap: 2024-01-24T23:28:05.565 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:05.565 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:05.565 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:05.567776+0000 2024-01-24T23:28:05.565 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:05.567776+0000 2024-01-24T23:28:05.565 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:05.565 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:05.565 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:05.565 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:05.565 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:05.565 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:05.566 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:05.566 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:05.566 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:05.566 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:05.566 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:05.566 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:05.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 1 Processor -- start 2024-01-24T23:28:05.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 1 -- start start 2024-01-24T23:28:05.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 10 monclient: init 2024-01-24T23:28:05.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:05.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 10 monclient: _add_conns ranks=[4,5,3,7,8,6,1,0,2] 2024-01-24T23:28:05.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 10 monclient(hunting): picked mon.noname-c con 0x7f21e41370e0 addr v1:172.21.15.115:6791/0 2024-01-24T23:28:05.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 10 monclient(hunting): picked mon.noname-h con 0x7f21e406d240 addr v1:172.21.15.132:6791/0 2024-01-24T23:28:05.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 10 monclient(hunting): picked mon.noname-g con 0x7f21e413d740 addr v1:172.21.15.132:6790/0 2024-01-24T23:28:05.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e4064700 con 0x7f21e413d740 2024-01-24T23:28:05.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e406c8f0 con 0x7f21e41370e0 2024-01-24T23:28:05.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e406d640 con 0x7f21e406d240 2024-01-24T23:28:05.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:05.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 10 monclient(hunting): authenticate will time out at 1372.893064s 2024-01-24T23:28:05.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ea8be640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f21e413d740 0x7f21e413fb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:43978/0 (socket says 172.21.15.132:43978) 2024-01-24T23:28:05.574 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ea8be640 1 -- 172.21.15.132:0/2478998764 learned_addr learned my addr 172.21.15.132:0/2478998764 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:05.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e98bc640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f21e406d240 0x7f21e413c020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:48702/0 (socket says 172.21.15.132:48702) 2024-01-24T23:28:05.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 1 -- 172.21.15.132:0/2478998764 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3705245444 0 0) 0x7f21e4064700 con 0x7f21e413d740 2024-01-24T23:28:05.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 10 monclient(hunting): _negotiate 2024-01-24T23:28:05.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:05.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:05.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 10 monclient(hunting): my global_id is 6879 2024-01-24T23:28:05.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 1 -- 172.21.15.132:0/2478998764 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21d0003530 con 0x7f21e413d740 2024-01-24T23:28:05.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 1 -- 172.21.15.132:0/2478998764 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 414918240 0 0) 0x7f21e406d640 con 0x7f21e406d240 2024-01-24T23:28:05.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 10 monclient(hunting): _negotiate 2024-01-24T23:28:05.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:05.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:05.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 10 monclient(hunting): my global_id is 6863 2024-01-24T23:28:05.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 1 -- 172.21.15.132:0/2478998764 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21e4064700 con 0x7f21e406d240 2024-01-24T23:28:05.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 1 -- 172.21.15.132:0/2478998764 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2016106997 0 0) 0x7f21d0003530 con 0x7f21e413d740 2024-01-24T23:28:05.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 1 -- 172.21.15.132:0/2478998764 >> v1:172.21.15.132:6791/0 conn(0x7f21e406d240 legacy=0x7f21e413c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:05.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 1 -- 172.21.15.132:0/2478998764 >> v1:172.21.15.115:6791/0 conn(0x7f21e41370e0 legacy=0x7f21e41374c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:05.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 10 monclient: _finish_hunting 0 2024-01-24T23:28:05.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 1 monclient: found mon.noname-g 2024-01-24T23:28:05.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:05.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:28:05.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 1 -- 172.21.15.132:0/2478998764 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e406b600 con 0x7f21e413d740 2024-01-24T23:28:05.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 10 monclient: _finish_auth 0 2024-01-24T23:28:05.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 10 monclient: _check_auth_tickets 2024-01-24T23:28:05.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:05.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 1 -- 172.21.15.132:0/2478998764 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f21dc001b40 con 0x7f21e413d740 2024-01-24T23:28:05.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:05.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:28:05.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 10 monclient: dump: 2024-01-24T23:28:05.578 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:05.578 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:05.578 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:05.578 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:05.578 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:05.578 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:05.578 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:05.579 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:05.579 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:05.579 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:05.579 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:05.579 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:05.579 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:05.579 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:05.579 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:05.579 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:05.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 5 monclient: authenticate success, global_id 6879 2024-01-24T23:28:05.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:05.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 1 -- 172.21.15.132:0/2478998764 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f21dc001d70 con 0x7f21e413d740 2024-01-24T23:28:05.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:05.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 1 -- 172.21.15.132:0/2478998764 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f21dc004f70 con 0x7f21e413d740 2024-01-24T23:28:05.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:05.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:28:05.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21e90bb640 10 monclient: dump: 2024-01-24T23:28:05.580 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:05.581 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:05.581 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:05.581 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:05.581 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:05.581 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:05.581 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:05.581 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:05.581 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:05.581 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:05.582 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:05.582 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:05.582 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:05.582 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:05.582 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:05.582 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:05.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:05.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 4 set_mon_vals no callback set 2024-01-24T23:28:05.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 10 monclient: shutdown 2024-01-24T23:28:05.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:05.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 1 -- 172.21.15.132:0/2478998764 >> v1:172.21.15.132:6790/0 conn(0x7f21e413d740 legacy=0x7f21e413fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:05.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 1 -- 172.21.15.132:0/2478998764 shutdown_connections 2024-01-24T23:28:05.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 1 -- 172.21.15.132:0/2478998764 >> 172.21.15.132:0/2478998764 conn(0x7f21e40a7150 msgr2=0x7f21e40a7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:05.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 1 -- 172.21.15.132:0/2478998764 shutdown_connections 2024-01-24T23:28:05.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 1 -- 172.21.15.132:0/2478998764 wait complete. 2024-01-24T23:28:05.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 10 monclient: build_initial_monmap 2024-01-24T23:28:05.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 10 monclient: monmap: 2024-01-24T23:28:05.583 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:05.584 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:05.584 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:05.584 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:05.584 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:05.584 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:05.584 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:05.584 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:05.584 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:05.584 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:05.585 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:05.585 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:05.585 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:05.585 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:05.585 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:05.585 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:05.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 1 Processor -- start 2024-01-24T23:28:05.585 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 1 -- start start 2024-01-24T23:28:05.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 10 monclient: init 2024-01-24T23:28:05.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:05.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 10 monclient: _add_conns ranks=[5,0,7,2,1,8,6,4,3] 2024-01-24T23:28:05.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 10 monclient(hunting): picked mon.noname-h con 0x7f21e406d240 addr v1:172.21.15.132:6791/0 2024-01-24T23:28:05.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 10 monclient(hunting): picked mon.noname-a con 0x7f21e41370e0 addr v1:172.21.15.115:6789/0 2024-01-24T23:28:05.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 10 monclient(hunting): picked mon.noname-i con 0x7f21e413d740 addr v1:172.21.15.132:6792/0 2024-01-24T23:28:05.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e409da40 con 0x7f21e41370e0 2024-01-24T23:28:05.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e40a0d30 con 0x7f21e406d240 2024-01-24T23:28:05.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e4130ab0 con 0x7f21e413d740 2024-01-24T23:28:05.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:05.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ec321640 10 monclient(hunting): authenticate will time out at 1372.895933s 2024-01-24T23:28:05.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ea8be640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f21e413d740 legacy=0x7f21e41d3d30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:28:05.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.566+0000 7f21ea0bd640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f21e406d240 0x7f21e412fc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:48716/0 (socket says 172.21.15.132:48716) 2024-01-24T23:28:05.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21ea0bd640 1 -- 172.21.15.132:0/3706654007 learned_addr learned my addr 172.21.15.132:0/3706654007 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:05.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21e98bc640 1 -- 172.21.15.132:0/3706654007 >> v1:172.21.15.115:6789/0 conn(0x7f21e41370e0 legacy=0x7f21e41303a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:28:05.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 1 -- 172.21.15.132:0/3706654007 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3455905822 0 0) 0x7f21e40a0d30 con 0x7f21e406d240 2024-01-24T23:28:05.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 10 monclient(hunting): _negotiate 2024-01-24T23:28:05.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:05.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:05.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 10 monclient(hunting): my global_id is 6872 2024-01-24T23:28:05.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 1 -- 172.21.15.132:0/3706654007 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21c8003530 con 0x7f21e406d240 2024-01-24T23:28:05.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 1 -- 172.21.15.132:0/3706654007 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 632852937 0 0) 0x7f21c8003530 con 0x7f21e406d240 2024-01-24T23:28:05.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 1 -- 172.21.15.132:0/3706654007 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f21e40a0d30 con 0x7f21e406d240 2024-01-24T23:28:05.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 1 -- 172.21.15.132:0/3706654007 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f21e0001a40 con 0x7f21e406d240 2024-01-24T23:28:05.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:05.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:28:05.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 10 monclient(hunting): dump: 2024-01-24T23:28:05.587 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:05.587 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:05.587 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:05.588 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:05.588 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:05.588 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:05.588 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:05.588 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:05.588 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:05.588 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:05.588 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:05.588 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:05.588 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:05.588 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:05.588 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:05.588 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:05.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:05.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 1 -- 172.21.15.132:0/3706654007 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4127211306 0 0) 0x7f21e40a0d30 con 0x7f21e406d240 2024-01-24T23:28:05.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 1 -- 172.21.15.132:0/3706654007 >> v1:172.21.15.132:6792/0 conn(0x7f21e413d740 legacy=0x7f21e41d3d30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:28:05.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 1 -- 172.21.15.132:0/3706654007 >> v1:172.21.15.115:6789/0 conn(0x7f21e41370e0 legacy=0x7f21e41303a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:28:05.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 10 monclient: _finish_hunting 0 2024-01-24T23:28:05.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 1 monclient: found mon.h 2024-01-24T23:28:05.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:05.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:05.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 1 -- 172.21.15.132:0/3706654007 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e4071f20 con 0x7f21e406d240 2024-01-24T23:28:05.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 10 monclient: _finish_auth 0 2024-01-24T23:28:05.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 10 monclient: _check_auth_tickets 2024-01-24T23:28:05.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:05.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21ec321640 5 monclient: authenticate success, global_id 6872 2024-01-24T23:28:05.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21ec321640 10 monclient: _renew_subs 2024-01-24T23:28:05.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21ec321640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:05.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21ec321640 1 -- 172.21.15.132:0/3706654007 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e406c0a0 con 0x7f21e406d240 2024-01-24T23:28:05.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21ec321640 10 monclient: _renew_subs 2024-01-24T23:28:05.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21ec321640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:05.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21ec321640 1 -- 172.21.15.132:0/3706654007 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e409c1b0 con 0x7f21e406d240 2024-01-24T23:28:05.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 1 -- 172.21.15.132:0/3706654007 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f21e00038d0 con 0x7f21e406d240 2024-01-24T23:28:05.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:05.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 1 -- 172.21.15.132:0/3706654007 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f21e0003e00 con 0x7f21e406d240 2024-01-24T23:28:05.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:05.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:28:05.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21e88ba640 4 set_mon_vals no callback set 2024-01-24T23:28:05.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 10 monclient: dump: 2024-01-24T23:28:05.590 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:05.591 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:05.591 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:05.591 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:05.591 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:05.591 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:05.591 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:05.591 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:05.591 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:05.591 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:05.591 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:05.591 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:05.591 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:05.591 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:05.591 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:05.591 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:05.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21ec321640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:05.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21ec321640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:05.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21ec321640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:05.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21ec321640 1 -- 172.21.15.132:0/3706654007 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21cc005090 con 0x7f21e406d240 2024-01-24T23:28:05.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 1 -- 172.21.15.132:0/3706654007 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f21e0003fe0 con 0x7f21e406d240 2024-01-24T23:28:05.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21daffd640 1 -- 172.21.15.132:0/3706654007 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(235..235 src has 1..235) v4 ==== 9506+0+0 (unknown 1768865826 0 0) 0x7f21e0044ef0 con 0x7f21e406d240 2024-01-24T23:28:05.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.570+0000 7f21e98bc640 10 monclient: get_auth_request con 0x7f21c8036f10 auth_method 0 2024-01-24T23:28:05.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.574+0000 7f21daffd640 1 -- 172.21.15.132:0/3706654007 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f21e0014b50 con 0x7f21e406d240 2024-01-24T23:28:05.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.574+0000 7f21daffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:05.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.574+0000 7f21daffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:05.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.734+0000 7f21ec321640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h24"}] 2024-01-24T23:28:05.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.734+0000 7f21ec321640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h24"}] 2024-01-24T23:28:05.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.734+0000 7f21ec321640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:05.735 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:05.734+0000 7f21ec321640 1 -- 172.21.15.132:0/3706654007 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h24"} v 0) v1 -- 0x7f21cc005340 con 0x7f21e406d240 2024-01-24T23:28:06.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.330+0000 7f21daffd640 1 -- 172.21.15.132:0/3706654007 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h24"}]=0 removed item id -88 name 'h24' from crush map v236) v1 ==== 120+0+0 (unknown 1074743054 0 0) 0x7f21e0012fc0 con 0x7f21e406d240 2024-01-24T23:28:06.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.330+0000 7f21daffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h24"}] 2024-01-24T23:28:06.332 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.330+0000 7f21daffd640 10 monclient: _finish_command 2 = system:0 removed item id -88 name 'h24' from crush map 2024-01-24T23:28:06.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.438+0000 7f21ec321640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h24"}] 2024-01-24T23:28:06.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.438+0000 7f21ec321640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h24"}] 2024-01-24T23:28:06.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.438+0000 7f21ec321640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:06.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.438+0000 7f21ec321640 1 -- 172.21.15.132:0/3706654007 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h24"} v 0) v1 -- 0x7f21cc002220 con 0x7f21e406d240 2024-01-24T23:28:06.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.438+0000 7f21daffd640 1 -- 172.21.15.132:0/3706654007 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h24"}]=0 device 'h24' does not appear in the crush map v236) v1 ==== 120+0+0 (unknown 4140453942 0 0) 0x7f21e0014740 con 0x7f21e406d240 2024-01-24T23:28:06.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.438+0000 7f21daffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h24"}] 2024-01-24T23:28:06.438 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.438+0000 7f21daffd640 10 monclient: _finish_command 3 = system:0 device 'h24' does not appear in the crush map 2024-01-24T23:28:06.439 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h24' does not appear in the crush map 2024-01-24T23:28:06.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.438+0000 7f21ec321640 1 -- 172.21.15.132:0/3706654007 >> v1:172.21.15.132:6812/1835401922 conn(0x7f21c8036f10 legacy=0x7f21c80393d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:06.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.438+0000 7f21ec321640 10 monclient: shutdown 2024-01-24T23:28:06.440 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.438+0000 7f21ec321640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:06.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.438+0000 7f21ec321640 1 -- 172.21.15.132:0/3706654007 >> v1:172.21.15.132:6791/0 conn(0x7f21e406d240 legacy=0x7f21e412fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:06.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.438+0000 7f21ec321640 1 -- 172.21.15.132:0/3706654007 shutdown_connections 2024-01-24T23:28:06.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.438+0000 7f21ec321640 1 -- 172.21.15.132:0/3706654007 >> 172.21.15.132:0/3706654007 conn(0x7f21e40a7150 msgr2=0x7f21e4133210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:06.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.438+0000 7f21ec321640 1 -- 172.21.15.132:0/3706654007 shutdown_connections 2024-01-24T23:28:06.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.438+0000 7f21ec321640 1 -- 172.21.15.132:0/3706654007 wait complete. 2024-01-24T23:28:06.454 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:28:06.454 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h25 2024-01-24T23:28:06.571 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabe0509640 10 monclient: get_monmap_and_config 2024-01-24T23:28:06.571 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabe0509640 10 monclient: build_initial_monmap 2024-01-24T23:28:06.571 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabe0509640 10 monclient: monmap: 2024-01-24T23:28:06.571 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:06.571 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:06.571 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:06.573951+0000 2024-01-24T23:28:06.571 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:06.573951+0000 2024-01-24T23:28:06.571 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:06.571 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:06.571 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:06.572 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:06.572 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:06.572 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:06.572 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:06.572 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:06.572 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:06.572 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:06.572 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:06.572 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:06.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabe0509640 1 Processor -- start 2024-01-24T23:28:06.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabe0509640 1 -- start start 2024-01-24T23:28:06.575 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabe0509640 10 monclient: init 2024-01-24T23:28:06.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabe0509640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:06.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabe0509640 10 monclient: _add_conns ranks=[8,4,6,5,0,3,7,1,2] 2024-01-24T23:28:06.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabe0509640 10 monclient(hunting): picked mon.noname-e con 0x7fabd809bc10 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:06.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabe0509640 10 monclient(hunting): picked mon.noname-c con 0x7fabd81368b0 addr v1:172.21.15.115:6791/0 2024-01-24T23:28:06.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabe0509640 10 monclient(hunting): picked mon.noname-d con 0x7fabd813a3c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:28:06.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabe0509640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd8064700 con 0x7fabd81368b0 2024-01-24T23:28:06.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabe0509640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd809bff0 con 0x7fabd813a3c0 2024-01-24T23:28:06.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabe0509640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd80a1870 con 0x7fabd809bc10 2024-01-24T23:28:06.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabe0509640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:06.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabe0509640 10 monclient(hunting): authenticate will time out at 1373.898694s 2024-01-24T23:28:06.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabdeaa6640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fabd813a3c0 0x7fabd813c7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:57836/0 (socket says 172.21.15.132:57836) 2024-01-24T23:28:06.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabde2a5640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fabd809bc10 0x7fabd80a21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:56684/0 (socket says 172.21.15.132:56684) 2024-01-24T23:28:06.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabdeaa6640 1 -- 172.21.15.132:0/920668009 learned_addr learned my addr 172.21.15.132:0/920668009 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:06.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabddaa4640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fabd81368b0 0x7fabd8138ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:34336/0 (socket says 172.21.15.132:34336) 2024-01-24T23:28:06.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabdd2a3640 1 -- 172.21.15.132:0/920668009 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1295893579 0 0) 0x7fabd8064700 con 0x7fabd81368b0 2024-01-24T23:28:06.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabdd2a3640 10 monclient(hunting): _negotiate 2024-01-24T23:28:06.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabdd2a3640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:06.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabdd2a3640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:06.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabdd2a3640 10 monclient(hunting): my global_id is 95026 2024-01-24T23:28:06.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabdd2a3640 1 -- 172.21.15.132:0/920668009 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabc4003530 con 0x7fabd81368b0 2024-01-24T23:28:06.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabdd2a3640 1 -- 172.21.15.132:0/920668009 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378739033 0 0) 0x7fabd809bff0 con 0x7fabd813a3c0 2024-01-24T23:28:06.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabdd2a3640 10 monclient(hunting): _negotiate 2024-01-24T23:28:06.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabdd2a3640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:06.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabdd2a3640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:06.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabdd2a3640 10 monclient(hunting): my global_id is 7080 2024-01-24T23:28:06.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabdd2a3640 1 -- 172.21.15.132:0/920668009 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabd8064700 con 0x7fabd813a3c0 2024-01-24T23:28:06.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabdd2a3640 1 -- 172.21.15.132:0/920668009 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1232385048 0 0) 0x7fabd80a1870 con 0x7fabd809bc10 2024-01-24T23:28:06.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabdd2a3640 10 monclient(hunting): _negotiate 2024-01-24T23:28:06.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabdd2a3640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:06.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabdd2a3640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:06.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabdd2a3640 10 monclient(hunting): my global_id is 86039 2024-01-24T23:28:06.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.570+0000 7fabdd2a3640 1 -- 172.21.15.132:0/920668009 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabd809bff0 con 0x7fabd809bc10 2024-01-24T23:28:06.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabdd2a3640 1 -- 172.21.15.132:0/920668009 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 348825813 0 0) 0x7fabc4003530 con 0x7fabd81368b0 2024-01-24T23:28:06.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabdd2a3640 1 -- 172.21.15.132:0/920668009 >> v1:172.21.15.115:6793/0 conn(0x7fabd809bc10 legacy=0x7fabd80a21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:06.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabdd2a3640 1 -- 172.21.15.132:0/920668009 >> v1:172.21.15.115:6792/0 conn(0x7fabd813a3c0 legacy=0x7fabd813c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:06.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabdd2a3640 10 monclient: _finish_hunting 0 2024-01-24T23:28:06.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabdd2a3640 1 monclient: found mon.noname-c 2024-01-24T23:28:06.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabdd2a3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:06.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabdd2a3640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:28:06.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabdd2a3640 1 -- 172.21.15.132:0/920668009 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd80a0fa0 con 0x7fabd81368b0 2024-01-24T23:28:06.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabdd2a3640 10 monclient: _finish_auth 0 2024-01-24T23:28:06.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabdd2a3640 10 monclient: _check_auth_tickets 2024-01-24T23:28:06.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabdd2a3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:06.578 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabdd2a3640 1 -- 172.21.15.132:0/920668009 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fabd40017c0 con 0x7fabd81368b0 2024-01-24T23:28:06.579 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 5 monclient: authenticate success, global_id 95026 2024-01-24T23:28:06.579 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabdd2a3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:06.579 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabdd2a3640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:28:06.579 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabdd2a3640 10 monclient: dump: 2024-01-24T23:28:06.579 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:06.579 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:06.579 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:06.579 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:06.579 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:06.579 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:06.579 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:06.579 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:06.579 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:06.579 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:06.579 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:06.580 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:06.580 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:06.580 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:06.580 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:06.580 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:06.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:06.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabdd2a3640 1 -- 172.21.15.132:0/920668009 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fabd4001a30 con 0x7fabd81368b0 2024-01-24T23:28:06.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabdd2a3640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:06.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:06.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 4 set_mon_vals no callback set 2024-01-24T23:28:06.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabdd2a3640 1 -- 172.21.15.132:0/920668009 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fabd4004b30 con 0x7fabd81368b0 2024-01-24T23:28:06.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabdd2a3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:06.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabdd2a3640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:28:06.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabdd2a3640 10 monclient: dump: 2024-01-24T23:28:06.580 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:06.581 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:06.581 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:06.581 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:06.581 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:06.581 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:06.581 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:06.581 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:06.581 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:06.581 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:06.581 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:06.581 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:06.581 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:06.581 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:06.581 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:06.581 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:06.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 10 monclient: shutdown 2024-01-24T23:28:06.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:06.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 1 -- 172.21.15.132:0/920668009 >> v1:172.21.15.115:6791/0 conn(0x7fabd81368b0 legacy=0x7fabd8138ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:06.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 1 -- 172.21.15.132:0/920668009 shutdown_connections 2024-01-24T23:28:06.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 1 -- 172.21.15.132:0/920668009 >> 172.21.15.132:0/920668009 conn(0x7fabd80a5a10 msgr2=0x7fabd80a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:06.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 1 -- 172.21.15.132:0/920668009 shutdown_connections 2024-01-24T23:28:06.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 1 -- 172.21.15.132:0/920668009 wait complete. 2024-01-24T23:28:06.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 10 monclient: build_initial_monmap 2024-01-24T23:28:06.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 10 monclient: monmap: 2024-01-24T23:28:06.582 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:06.582 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:06.582 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:06.582 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:06.582 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:06.582 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:06.583 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:06.583 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:06.583 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:06.583 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:06.583 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:06.583 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:06.583 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:06.583 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:06.583 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:06.583 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:06.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 1 Processor -- start 2024-01-24T23:28:06.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 1 -- start start 2024-01-24T23:28:06.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 10 monclient: init 2024-01-24T23:28:06.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:06.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 10 monclient: _add_conns ranks=[6,8,2,4,3,5,0,7,1] 2024-01-24T23:28:06.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 10 monclient(hunting): picked mon.noname-d con 0x7fabd809bc10 addr v1:172.21.15.115:6792/0 2024-01-24T23:28:06.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 10 monclient(hunting): picked mon.noname-e con 0x7fabd81368b0 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:06.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 10 monclient(hunting): picked mon.noname-b con 0x7fabd813a3c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:28:06.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd806ef40 con 0x7fabd813a3c0 2024-01-24T23:28:06.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd806f220 con 0x7fabd809bc10 2024-01-24T23:28:06.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd8134560 con 0x7fabd81368b0 2024-01-24T23:28:06.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:06.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabe0509640 10 monclient(hunting): authenticate will time out at 1373.902492s 2024-01-24T23:28:06.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabdeaa6640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fabd813a3c0 legacy=0x7fabd81d17b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:28:06.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabddaa4640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fabd81368b0 0x7fabd8133e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:56692/0 (socket says 172.21.15.132:56692) 2024-01-24T23:28:06.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabddaa4640 1 -- 172.21.15.132:0/137211558 learned_addr learned my addr 172.21.15.132:0/137211558 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:06.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 1 -- 172.21.15.132:0/137211558 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 58537952 0 0) 0x7fabd8134560 con 0x7fabd81368b0 2024-01-24T23:28:06.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 10 monclient(hunting): _negotiate 2024-01-24T23:28:06.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:06.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:06.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 10 monclient(hunting): my global_id is 86048 2024-01-24T23:28:06.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 1 -- 172.21.15.132:0/137211558 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabbc003530 con 0x7fabd81368b0 2024-01-24T23:28:06.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 1 -- 172.21.15.132:0/137211558 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 945908212 0 0) 0x7fabd806f220 con 0x7fabd809bc10 2024-01-24T23:28:06.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 10 monclient(hunting): _negotiate 2024-01-24T23:28:06.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:06.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:06.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 10 monclient(hunting): my global_id is 7089 2024-01-24T23:28:06.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 1 -- 172.21.15.132:0/137211558 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabd8134560 con 0x7fabd809bc10 2024-01-24T23:28:06.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 1 -- 172.21.15.132:0/137211558 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4074360869 0 0) 0x7fabbc003530 con 0x7fabd81368b0 2024-01-24T23:28:06.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 1 -- 172.21.15.132:0/137211558 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fabd806f220 con 0x7fabd81368b0 2024-01-24T23:28:06.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 1 -- 172.21.15.132:0/137211558 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fabd4001780 con 0x7fabd81368b0 2024-01-24T23:28:06.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:06.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:28:06.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 10 monclient(hunting): dump: 2024-01-24T23:28:06.591 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:06.591 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:06.591 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:06.591 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:06.591 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:06.591 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:06.591 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:06.591 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:06.591 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:06.592 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:06.592 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:06.592 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:06.592 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:06.592 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:06.592 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:06.592 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:06.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:06.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 1 -- 172.21.15.132:0/137211558 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4114413180 0 0) 0x7fabd8134560 con 0x7fabd809bc10 2024-01-24T23:28:06.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 1 -- 172.21.15.132:0/137211558 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fabbc003530 con 0x7fabd809bc10 2024-01-24T23:28:06.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 1 -- 172.21.15.132:0/137211558 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fabc8001870 con 0x7fabd809bc10 2024-01-24T23:28:06.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:06.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:28:06.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 10 monclient(hunting): dump: 2024-01-24T23:28:06.593 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:06.593 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:06.593 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:06.593 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:06.593 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:06.593 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:06.593 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:06.593 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:06.593 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:06.593 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:06.593 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:06.593 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:06.593 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:06.593 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:06.593 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:06.594 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:06.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 1 -- 172.21.15.132:0/137211558 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3740683913 0 0) 0x7fabd806f220 con 0x7fabd81368b0 2024-01-24T23:28:06.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.574+0000 7fabceffd640 1 -- 172.21.15.132:0/137211558 >> v1:172.21.15.115:6792/0 conn(0x7fabd809bc10 legacy=0x7fabd8133740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:06.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabceffd640 1 -- 172.21.15.132:0/137211558 >> v1:172.21.15.115:6790/0 conn(0x7fabd813a3c0 legacy=0x7fabd81d17b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:28:06.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabceffd640 10 monclient: _finish_hunting 0 2024-01-24T23:28:06.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabceffd640 1 monclient: found mon.e 2024-01-24T23:28:06.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabceffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:06.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabceffd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:28:06.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabceffd640 1 -- 172.21.15.132:0/137211558 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd809c650 con 0x7fabd81368b0 2024-01-24T23:28:06.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabceffd640 10 monclient: _finish_auth 0 2024-01-24T23:28:06.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabceffd640 10 monclient: _check_auth_tickets 2024-01-24T23:28:06.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:06.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabe0509640 5 monclient: authenticate success, global_id 86048 2024-01-24T23:28:06.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabe0509640 10 monclient: _renew_subs 2024-01-24T23:28:06.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabe0509640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:28:06.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabe0509640 1 -- 172.21.15.132:0/137211558 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd806d1a0 con 0x7fabd81368b0 2024-01-24T23:28:06.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabe0509640 10 monclient: _renew_subs 2024-01-24T23:28:06.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabe0509640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:28:06.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabe0509640 1 -- 172.21.15.132:0/137211558 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd806da80 con 0x7fabd81368b0 2024-01-24T23:28:06.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabceffd640 1 -- 172.21.15.132:0/137211558 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fabd4004b50 con 0x7fabd81368b0 2024-01-24T23:28:06.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabceffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:06.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabceffd640 1 -- 172.21.15.132:0/137211558 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fabd4001c60 con 0x7fabd81368b0 2024-01-24T23:28:06.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:06.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabdcaa2640 4 set_mon_vals no callback set 2024-01-24T23:28:06.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabceffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:28:06.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabceffd640 10 monclient: dump: 2024-01-24T23:28:06.595 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:06.595 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:06.595 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:06.596 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:06.596 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:06.596 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:06.596 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:06.596 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:06.596 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:06.596 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:06.596 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:06.596 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:06.596 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:06.596 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:06.596 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:06.596 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:06.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabe0509640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:06.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabe0509640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:06.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabe0509640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:28:06.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabe0509640 1 -- 172.21.15.132:0/137211558 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc0005090 con 0x7fabd81368b0 2024-01-24T23:28:06.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabceffd640 1 -- 172.21.15.132:0/137211558 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fabd400fc70 con 0x7fabd81368b0 2024-01-24T23:28:06.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabceffd640 1 -- 172.21.15.132:0/137211558 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(236..236 src has 1..236) v4 ==== 9293+0+0 (unknown 530472544 0 0) 0x7fabd4045000 con 0x7fabd81368b0 2024-01-24T23:28:06.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.578+0000 7fabde2a5640 10 monclient: get_auth_request con 0x7fabbc0376a0 auth_method 0 2024-01-24T23:28:06.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.590+0000 7fabceffd640 1 -- 172.21.15.132:0/137211558 <== mon.8 v1:172.21.15.115:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fabd40111c0 con 0x7fabd81368b0 2024-01-24T23:28:06.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.590+0000 7fabceffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:06.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.590+0000 7fabceffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:06.750 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.750+0000 7fabe0509640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h25"}] 2024-01-24T23:28:06.750 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.750+0000 7fabe0509640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h25"}] 2024-01-24T23:28:06.750 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.750+0000 7fabe0509640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:28:06.750 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.750+0000 7fabe0509640 1 -- 172.21.15.132:0/137211558 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush rm", "name": "h25"} v 0) v1 -- 0x7fabc0005340 con 0x7fabd81368b0 2024-01-24T23:28:06.943 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.942+0000 7fabceffd640 1 -- 172.21.15.132:0/137211558 <== mon.8 v1:172.21.15.115:6793/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h25"}]=0 removed item id -91 name 'h25' from crush map v237) v1 ==== 120+0+0 (unknown 3350833303 0 0) 0x7fabd4014960 con 0x7fabd81368b0 2024-01-24T23:28:06.943 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.942+0000 7fabceffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h25"}] 2024-01-24T23:28:06.943 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:06.942+0000 7fabceffd640 10 monclient: _finish_command 2 = system:0 removed item id -91 name 'h25' from crush map 2024-01-24T23:28:07.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.046+0000 7fabe0509640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h25"}] 2024-01-24T23:28:07.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.046+0000 7fabe0509640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h25"}] 2024-01-24T23:28:07.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.046+0000 7fabe0509640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:28:07.048 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.046+0000 7fabe0509640 1 -- 172.21.15.132:0/137211558 --> v1:172.21.15.115:6793/0 -- mon_command({"prefix": "osd crush rm", "name": "h25"} v 0) v1 -- 0x7fabc00010a0 con 0x7fabd81368b0 2024-01-24T23:28:07.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.050+0000 7fabceffd640 1 -- 172.21.15.132:0/137211558 <== mon.8 v1:172.21.15.115:6793/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h25"}]=0 device 'h25' does not appear in the crush map v237) v1 ==== 120+0+0 (unknown 1123186841 0 0) 0x7fabd4014d50 con 0x7fabd81368b0 2024-01-24T23:28:07.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.050+0000 7fabceffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h25"}] 2024-01-24T23:28:07.050 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.050+0000 7fabceffd640 10 monclient: _finish_command 3 = system:0 device 'h25' does not appear in the crush map 2024-01-24T23:28:07.050 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h25' does not appear in the crush map 2024-01-24T23:28:07.052 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.050+0000 7fabe0509640 1 -- 172.21.15.132:0/137211558 >> v1:172.21.15.132:6812/1835401922 conn(0x7fabbc0376a0 legacy=0x7fabbc039b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:07.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.050+0000 7fabe0509640 10 monclient: shutdown 2024-01-24T23:28:07.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.050+0000 7fabe0509640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:07.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.050+0000 7fabe0509640 1 -- 172.21.15.132:0/137211558 >> v1:172.21.15.115:6793/0 conn(0x7fabd81368b0 legacy=0x7fabd8133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:07.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.050+0000 7fabe0509640 1 -- 172.21.15.132:0/137211558 shutdown_connections 2024-01-24T23:28:07.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.050+0000 7fabe0509640 1 -- 172.21.15.132:0/137211558 >> 172.21.15.132:0/137211558 conn(0x7fabd80a5a10 msgr2=0x7fabd8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:07.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.050+0000 7fabe0509640 1 -- 172.21.15.132:0/137211558 shutdown_connections 2024-01-24T23:28:07.053 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.050+0000 7fabe0509640 1 -- 172.21.15.132:0/137211558 wait complete. 2024-01-24T23:28:07.067 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:28:07.067 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h26 2024-01-24T23:28:07.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.182+0000 7fa2c61fa640 10 monclient: get_monmap_and_config 2024-01-24T23:28:07.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.182+0000 7fa2c61fa640 10 monclient: build_initial_monmap 2024-01-24T23:28:07.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.182+0000 7fa2c61fa640 10 monclient: monmap: 2024-01-24T23:28:07.185 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:07.185 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:07.185 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:07.187140+0000 2024-01-24T23:28:07.185 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:07.187140+0000 2024-01-24T23:28:07.185 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:07.185 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:07.185 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:07.186 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:07.186 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:07.186 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:07.186 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:07.186 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:07.186 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:07.186 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:07.186 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:07.186 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:07.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.182+0000 7fa2c61fa640 1 Processor -- start 2024-01-24T23:28:07.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.182+0000 7fa2c61fa640 1 -- start start 2024-01-24T23:28:07.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.182+0000 7fa2c61fa640 10 monclient: init 2024-01-24T23:28:07.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.182+0000 7fa2c61fa640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:07.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.182+0000 7fa2c61fa640 10 monclient: _add_conns ranks=[3,8,7,0,5,4,2,6,1] 2024-01-24T23:28:07.192 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.182+0000 7fa2c61fa640 10 monclient(hunting): picked mon.noname-g con 0x7fa2c0135e10 addr v1:172.21.15.132:6790/0 2024-01-24T23:28:07.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.182+0000 7fa2c61fa640 10 monclient(hunting): picked mon.noname-e con 0x7fa2c01361f0 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:07.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.182+0000 7fa2c61fa640 10 monclient(hunting): picked mon.noname-i con 0x7fa2c01365f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:28:07.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.182+0000 7fa2c61fa640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c0064700 con 0x7fa2c0135e10 2024-01-24T23:28:07.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.182+0000 7fa2c61fa640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c009c3e0 con 0x7fa2c01365f0 2024-01-24T23:28:07.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.182+0000 7fa2c61fa640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c00a4f90 con 0x7fa2c01361f0 2024-01-24T23:28:07.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.182+0000 7fa2c61fa640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:07.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.182+0000 7fa2c61fa640 10 monclient(hunting): authenticate will time out at 1374.511895s 2024-01-24T23:28:07.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.182+0000 7fa2bffff640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fa2c01365f0 legacy=0x7fa2c013f600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:28:07.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.182+0000 7fa2bf7fe640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fa2c0135e10 0x7fa2c009bcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:43984/0 (socket says 172.21.15.132:43984) 2024-01-24T23:28:07.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.182+0000 7fa2bf7fe640 1 -- 172.21.15.132:0/3974927788 learned_addr learned my addr 172.21.15.132:0/3974927788 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:07.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 1 -- 172.21.15.132:0/3974927788 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2448431989 0 0) 0x7fa2c0064700 con 0x7fa2c0135e10 2024-01-24T23:28:07.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:28:07.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:07.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:07.193 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 10 monclient(hunting): my global_id is 6888 2024-01-24T23:28:07.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 1 -- 172.21.15.132:0/3974927788 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2a8003530 con 0x7fa2c0135e10 2024-01-24T23:28:07.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 1 -- 172.21.15.132:0/3974927788 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 936544143 0 0) 0x7fa2a8003530 con 0x7fa2c0135e10 2024-01-24T23:28:07.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 1 -- 172.21.15.132:0/3974927788 >> v1:172.21.15.115:6793/0 conn(0x7fa2c01361f0 legacy=0x7fa2c013bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:07.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 1 -- 172.21.15.132:0/3974927788 >> v1:172.21.15.132:6792/0 conn(0x7fa2c01365f0 legacy=0x7fa2c013f600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:28:07.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:28:07.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 1 monclient: found mon.noname-g 2024-01-24T23:28:07.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:07.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:28:07.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 1 -- 172.21.15.132:0/3974927788 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c009b010 con 0x7fa2c0135e10 2024-01-24T23:28:07.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 10 monclient: _finish_auth 0 2024-01-24T23:28:07.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:28:07.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:07.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 1 -- 172.21.15.132:0/3974927788 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa2b00017c0 con 0x7fa2c0135e10 2024-01-24T23:28:07.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:07.194 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:28:07.195 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 10 monclient: dump: 2024-01-24T23:28:07.195 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:07.195 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:07.195 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:07.195 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:07.195 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:07.195 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:07.195 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:07.195 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:07.195 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:07.195 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:07.195 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:07.195 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:07.195 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:07.195 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:07.196 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:07.196 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:07.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 5 monclient: authenticate success, global_id 6888 2024-01-24T23:28:07.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:07.196 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 1 -- 172.21.15.132:0/3974927788 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa2b0001ae0 con 0x7fa2c0135e10 2024-01-24T23:28:07.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:07.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 1 -- 172.21.15.132:0/3974927788 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa2b0004bf0 con 0x7fa2c0135e10 2024-01-24T23:28:07.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:07.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 4 set_mon_vals no callback set 2024-01-24T23:28:07.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:07.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:28:07.199 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2be7fc640 10 monclient: dump: 2024-01-24T23:28:07.199 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:07.200 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:07.200 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:07.200 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:07.200 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:07.200 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:07.200 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:07.200 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:07.200 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:07.200 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:07.200 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:07.200 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:07.200 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:07.200 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:07.200 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:07.200 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:07.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 10 monclient: shutdown 2024-01-24T23:28:07.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:07.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 1 -- 172.21.15.132:0/3974927788 >> v1:172.21.15.132:6790/0 conn(0x7fa2c0135e10 legacy=0x7fa2c009bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:07.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 1 -- 172.21.15.132:0/3974927788 shutdown_connections 2024-01-24T23:28:07.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 1 -- 172.21.15.132:0/3974927788 >> 172.21.15.132:0/3974927788 conn(0x7fa2c00a6790 msgr2=0x7fa2c0071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:07.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 1 -- 172.21.15.132:0/3974927788 shutdown_connections 2024-01-24T23:28:07.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 1 -- 172.21.15.132:0/3974927788 wait complete. 2024-01-24T23:28:07.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 10 monclient: build_initial_monmap 2024-01-24T23:28:07.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 10 monclient: monmap: 2024-01-24T23:28:07.201 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:07.201 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:07.201 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:07.201 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:07.201 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:07.201 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:07.202 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:07.202 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:07.202 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:07.202 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:07.202 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:07.202 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:07.202 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:07.202 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:07.202 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:07.202 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:07.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 1 Processor -- start 2024-01-24T23:28:07.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 1 -- start start 2024-01-24T23:28:07.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 10 monclient: init 2024-01-24T23:28:07.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:07.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 10 monclient: _add_conns ranks=[5,7,6,2,3,4,0,1,8] 2024-01-24T23:28:07.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 10 monclient(hunting): picked mon.noname-h con 0x7fa2c0135e10 addr v1:172.21.15.132:6791/0 2024-01-24T23:28:07.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 10 monclient(hunting): picked mon.noname-i con 0x7fa2c01361f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:28:07.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 10 monclient(hunting): picked mon.noname-d con 0x7fa2c01365f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:28:07.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c009ca00 con 0x7fa2c0135e10 2024-01-24T23:28:07.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c009c6b0 con 0x7fa2c01365f0 2024-01-24T23:28:07.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c01c8cb0 con 0x7fa2c01361f0 2024-01-24T23:28:07.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:07.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2beffd640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fa2c01361f0 legacy=0x7fa2c01d0640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:28:07.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 10 monclient(hunting): authenticate will time out at 1374.514740s 2024-01-24T23:28:07.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2bf7fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fa2c0135e10 0x7fa2c0134eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:48718/0 (socket says 172.21.15.132:48718) 2024-01-24T23:28:07.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2bf7fe640 1 -- 172.21.15.132:0/1991410839 learned_addr learned my addr 172.21.15.132:0/1991410839 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:07.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 1 -- 172.21.15.132:0/1991410839 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2513583346 0 0) 0x7fa2c009ca00 con 0x7fa2c0135e10 2024-01-24T23:28:07.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 10 monclient(hunting): _negotiate 2024-01-24T23:28:07.203 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:07.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:07.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 10 monclient(hunting): my global_id is 6881 2024-01-24T23:28:07.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 1 -- 172.21.15.132:0/1991410839 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2ac003530 con 0x7fa2c0135e10 2024-01-24T23:28:07.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 1 -- 172.21.15.132:0/1991410839 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1037617731 0 0) 0x7fa2ac003530 con 0x7fa2c0135e10 2024-01-24T23:28:07.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 1 -- 172.21.15.132:0/1991410839 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa2c009ca00 con 0x7fa2c0135e10 2024-01-24T23:28:07.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 1 -- 172.21.15.132:0/1991410839 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa2b0004b30 con 0x7fa2c0135e10 2024-01-24T23:28:07.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:07.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:28:07.204 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 10 monclient(hunting): dump: 2024-01-24T23:28:07.204 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:07.204 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:07.204 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:07.204 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:07.204 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:07.205 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:07.205 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:07.205 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:07.205 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:07.205 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:07.205 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:07.205 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:07.205 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:07.205 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:07.205 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:07.205 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:07.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:07.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 1 -- 172.21.15.132:0/1991410839 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2612182127 0 0) 0x7fa2c009c6b0 con 0x7fa2c01365f0 2024-01-24T23:28:07.205 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 10 monclient(hunting): _negotiate 2024-01-24T23:28:07.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:07.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:07.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 10 monclient(hunting): my global_id is 7098 2024-01-24T23:28:07.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 1 -- 172.21.15.132:0/1991410839 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2ac003530 con 0x7fa2c01365f0 2024-01-24T23:28:07.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 1 -- 172.21.15.132:0/1991410839 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1798366915 0 0) 0x7fa2c009ca00 con 0x7fa2c0135e10 2024-01-24T23:28:07.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 1 -- 172.21.15.132:0/1991410839 >> v1:172.21.15.132:6792/0 conn(0x7fa2c01361f0 legacy=0x7fa2c01d0640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:28:07.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 1 -- 172.21.15.132:0/1991410839 >> v1:172.21.15.115:6792/0 conn(0x7fa2c01365f0 legacy=0x7fa2c01d3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:07.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 10 monclient: _finish_hunting 0 2024-01-24T23:28:07.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 1 monclient: found mon.h 2024-01-24T23:28:07.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:07.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:07.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 1 -- 172.21.15.132:0/1991410839 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c009fd70 con 0x7fa2c0135e10 2024-01-24T23:28:07.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 10 monclient: _finish_auth 0 2024-01-24T23:28:07.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 10 monclient: _check_auth_tickets 2024-01-24T23:28:07.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:07.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 5 monclient: authenticate success, global_id 6881 2024-01-24T23:28:07.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 10 monclient: _renew_subs 2024-01-24T23:28:07.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:07.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 1 -- 172.21.15.132:0/1991410839 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c0121fd0 con 0x7fa2c0135e10 2024-01-24T23:28:07.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 10 monclient: _renew_subs 2024-01-24T23:28:07.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:07.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2c61fa640 1 -- 172.21.15.132:0/1991410839 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c0135820 con 0x7fa2c0135e10 2024-01-24T23:28:07.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 1 -- 172.21.15.132:0/1991410839 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa2b0001180 con 0x7fa2c0135e10 2024-01-24T23:28:07.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:07.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 1 -- 172.21.15.132:0/1991410839 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa2b0001c50 con 0x7fa2c0135e10 2024-01-24T23:28:07.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:07.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2bdffb640 4 set_mon_vals no callback set 2024-01-24T23:28:07.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:28:07.208 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.186+0000 7fa2a7fff640 10 monclient: dump: 2024-01-24T23:28:07.208 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:07.208 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:07.208 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:07.208 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:07.208 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:07.208 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:07.208 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:07.208 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:07.208 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:07.208 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:07.208 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:07.208 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:07.209 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:07.209 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:07.209 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:07.209 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:07.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.190+0000 7fa2c61fa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:07.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.190+0000 7fa2c61fa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:07.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.190+0000 7fa2c61fa640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:07.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.190+0000 7fa2a7fff640 1 -- 172.21.15.132:0/1991410839 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fa2b000fdc0 con 0x7fa2c0135e10 2024-01-24T23:28:07.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.190+0000 7fa2c61fa640 1 -- 172.21.15.132:0/1991410839 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a0005090 con 0x7fa2c0135e10 2024-01-24T23:28:07.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.190+0000 7fa2a7fff640 1 -- 172.21.15.132:0/1991410839 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(237..237 src has 1..237) v4 ==== 9080+0+0 (unknown 3084093187 0 0) 0x7fa2b0044fe0 con 0x7fa2c0135e10 2024-01-24T23:28:07.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.190+0000 7fa2beffd640 10 monclient: get_auth_request con 0x7fa2ac036e80 auth_method 0 2024-01-24T23:28:07.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.194+0000 7fa2a7fff640 1 -- 172.21.15.132:0/1991410839 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fa2b0014c40 con 0x7fa2c0135e10 2024-01-24T23:28:07.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.194+0000 7fa2a7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:07.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.194+0000 7fa2a7fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:07.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.354+0000 7fa2c61fa640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h26"}] 2024-01-24T23:28:07.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.354+0000 7fa2c61fa640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h26"}] 2024-01-24T23:28:07.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.354+0000 7fa2c61fa640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:07.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:07.354+0000 7fa2c61fa640 1 -- 172.21.15.132:0/1991410839 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h26"} v 0) v1 -- 0x7fa2a0005340 con 0x7fa2c0135e10 2024-01-24T23:28:08.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.342+0000 7fa2a7fff640 1 -- 172.21.15.132:0/1991410839 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h26"}]=0 removed item id -94 name 'h26' from crush map v238) v1 ==== 120+0+0 (unknown 1339653864 0 0) 0x7fa2b00132e0 con 0x7fa2c0135e10 2024-01-24T23:28:08.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.342+0000 7fa2a7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h26"}] 2024-01-24T23:28:08.343 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.342+0000 7fa2a7fff640 10 monclient: _finish_command 2 = system:0 removed item id -94 name 'h26' from crush map 2024-01-24T23:28:08.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.442+0000 7fa2c61fa640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h26"}] 2024-01-24T23:28:08.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.442+0000 7fa2c61fa640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h26"}] 2024-01-24T23:28:08.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.442+0000 7fa2c61fa640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:08.445 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.442+0000 7fa2c61fa640 1 -- 172.21.15.132:0/1991410839 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h26"} v 0) v1 -- 0x7fa2a00010a0 con 0x7fa2c0135e10 2024-01-24T23:28:08.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.446+0000 7fa2a7fff640 1 -- 172.21.15.132:0/1991410839 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h26"}]=0 device 'h26' does not appear in the crush map v238) v1 ==== 120+0+0 (unknown 2606135193 0 0) 0x7fa2b0014a60 con 0x7fa2c0135e10 2024-01-24T23:28:08.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.446+0000 7fa2a7fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h26"}] 2024-01-24T23:28:08.446 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.446+0000 7fa2a7fff640 10 monclient: _finish_command 3 = system:0 device 'h26' does not appear in the crush map 2024-01-24T23:28:08.446 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h26' does not appear in the crush map 2024-01-24T23:28:08.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.446+0000 7fa2c61fa640 1 -- 172.21.15.132:0/1991410839 >> v1:172.21.15.132:6812/1835401922 conn(0x7fa2ac036e80 legacy=0x7fa2ac039340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:08.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.446+0000 7fa2c61fa640 10 monclient: shutdown 2024-01-24T23:28:08.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.446+0000 7fa2c61fa640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:08.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.446+0000 7fa2c61fa640 1 -- 172.21.15.132:0/1991410839 >> v1:172.21.15.132:6791/0 conn(0x7fa2c0135e10 legacy=0x7fa2c0134eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:08.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.446+0000 7fa2c61fa640 1 -- 172.21.15.132:0/1991410839 shutdown_connections 2024-01-24T23:28:08.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.446+0000 7fa2c61fa640 1 -- 172.21.15.132:0/1991410839 >> 172.21.15.132:0/1991410839 conn(0x7fa2c00a6790 msgr2=0x7fa2c00a82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:08.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.446+0000 7fa2c61fa640 1 -- 172.21.15.132:0/1991410839 shutdown_connections 2024-01-24T23:28:08.448 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.446+0000 7fa2c61fa640 1 -- 172.21.15.132:0/1991410839 wait complete. 2024-01-24T23:28:08.462 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:28:08.462 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h27 2024-01-24T23:28:08.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.578+0000 7f99067a9640 10 monclient: get_monmap_and_config 2024-01-24T23:28:08.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.578+0000 7f99067a9640 10 monclient: build_initial_monmap 2024-01-24T23:28:08.581 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.578+0000 7f99067a9640 10 monclient: monmap: 2024-01-24T23:28:08.581 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:08.581 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:08.581 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:08.583880+0000 2024-01-24T23:28:08.581 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:08.583880+0000 2024-01-24T23:28:08.581 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:08.581 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:08.581 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:08.581 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:08.582 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:08.582 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:08.582 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:08.582 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:08.582 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:08.582 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:08.582 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:08.582 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:08.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 1 Processor -- start 2024-01-24T23:28:08.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 1 -- start start 2024-01-24T23:28:08.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 10 monclient: init 2024-01-24T23:28:08.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:08.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 10 monclient: _add_conns ranks=[8,4,0,3,6,5,1,7,2] 2024-01-24T23:28:08.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 10 monclient(hunting): picked mon.noname-e con 0x7f9900135e10 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:08.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 10 monclient(hunting): picked mon.noname-c con 0x7f99001361f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:28:08.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 10 monclient(hunting): picked mon.noname-a con 0x7f99001365f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:28:08.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9900064700 con 0x7f99001365f0 2024-01-24T23:28:08.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990009c3e0 con 0x7f99001361f0 2024-01-24T23:28:08.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99000a4f90 con 0x7f9900135e10 2024-01-24T23:28:08.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:08.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 10 monclient(hunting): authenticate will time out at 1375.908787s 2024-01-24T23:28:08.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f9904d46640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f99001365f0 legacy=0x7f990013f600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:28:08.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98fffff640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f9900135e10 0x7f990009bcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:56704/0 (socket says 172.21.15.132:56704) 2024-01-24T23:28:08.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98ff7fe640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f99001361f0 0x7f990013bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:34338/0 (socket says 172.21.15.132:34338) 2024-01-24T23:28:08.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98fffff640 1 -- 172.21.15.132:0/371071460 learned_addr learned my addr 172.21.15.132:0/371071460 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:08.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 1 -- 172.21.15.132:0/371071460 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142183793 0 0) 0x7f99000a4f90 con 0x7f9900135e10 2024-01-24T23:28:08.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 10 monclient(hunting): _negotiate 2024-01-24T23:28:08.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:08.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:08.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 10 monclient(hunting): my global_id is 86066 2024-01-24T23:28:08.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 1 -- 172.21.15.132:0/371071460 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98ec003530 con 0x7f9900135e10 2024-01-24T23:28:08.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 1 -- 172.21.15.132:0/371071460 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3170998085 0 0) 0x7f990009c3e0 con 0x7f99001361f0 2024-01-24T23:28:08.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 10 monclient(hunting): _negotiate 2024-01-24T23:28:08.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:08.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:08.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 10 monclient(hunting): my global_id is 95035 2024-01-24T23:28:08.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 1 -- 172.21.15.132:0/371071460 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99000a4f90 con 0x7f99001361f0 2024-01-24T23:28:08.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 1 -- 172.21.15.132:0/371071460 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3044447818 0 0) 0x7f98ec003530 con 0x7f9900135e10 2024-01-24T23:28:08.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 1 -- 172.21.15.132:0/371071460 >> v1:172.21.15.115:6791/0 conn(0x7f99001361f0 legacy=0x7f990013bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:08.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 1 -- 172.21.15.132:0/371071460 >> v1:172.21.15.115:6789/0 conn(0x7f99001365f0 legacy=0x7f990013f600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:28:08.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 10 monclient: _finish_hunting 0 2024-01-24T23:28:08.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 1 monclient: found mon.noname-e 2024-01-24T23:28:08.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:08.591 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:28:08.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 1 -- 172.21.15.132:0/371071460 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990009b010 con 0x7f9900135e10 2024-01-24T23:28:08.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 10 monclient: _finish_auth 0 2024-01-24T23:28:08.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 10 monclient: _check_auth_tickets 2024-01-24T23:28:08.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:08.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 5 monclient: authenticate success, global_id 86066 2024-01-24T23:28:08.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:08.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 1 -- 172.21.15.132:0/371071460 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f98f0001b40 con 0x7f9900135e10 2024-01-24T23:28:08.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:08.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:28:08.592 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 10 monclient: dump: 2024-01-24T23:28:08.592 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:08.592 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:08.592 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:08.592 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:08.593 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:08.593 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:08.593 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:08.593 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:08.593 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:08.593 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:08.593 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:08.593 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:08.593 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:08.593 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:08.593 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:08.593 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:08.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:08.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 1 -- 172.21.15.132:0/371071460 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f98f0001d60 con 0x7f9900135e10 2024-01-24T23:28:08.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:08.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:08.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 1 -- 172.21.15.132:0/371071460 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f98f0004f70 con 0x7f9900135e10 2024-01-24T23:28:08.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:08.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 4 set_mon_vals no callback set 2024-01-24T23:28:08.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:28:08.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f98feffd640 10 monclient: dump: 2024-01-24T23:28:08.594 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:08.594 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:08.594 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:08.594 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:08.594 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:08.594 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:08.594 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:08.594 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:08.594 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:08.595 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:08.595 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:08.595 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:08.595 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:08.595 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:08.595 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:08.595 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:08.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 10 monclient: shutdown 2024-01-24T23:28:08.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:08.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 1 -- 172.21.15.132:0/371071460 >> v1:172.21.15.115:6793/0 conn(0x7f9900135e10 legacy=0x7f990009bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:08.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 1 -- 172.21.15.132:0/371071460 shutdown_connections 2024-01-24T23:28:08.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 1 -- 172.21.15.132:0/371071460 >> 172.21.15.132:0/371071460 conn(0x7f99000a6790 msgr2=0x7f9900071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:08.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 1 -- 172.21.15.132:0/371071460 shutdown_connections 2024-01-24T23:28:08.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 1 -- 172.21.15.132:0/371071460 wait complete. 2024-01-24T23:28:08.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 10 monclient: build_initial_monmap 2024-01-24T23:28:08.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 10 monclient: monmap: 2024-01-24T23:28:08.596 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:08.596 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:08.596 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:08.596 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:08.596 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:08.596 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:08.596 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:08.596 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:08.596 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:08.596 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:08.596 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:08.596 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:08.597 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:08.597 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:08.597 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:08.597 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:08.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.582+0000 7f99067a9640 1 Processor -- start 2024-01-24T23:28:08.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f99067a9640 1 -- start start 2024-01-24T23:28:08.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f99067a9640 10 monclient: init 2024-01-24T23:28:08.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f99067a9640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:08.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f99067a9640 10 monclient: _add_conns ranks=[2,5,0,7,8,3,6,4,1] 2024-01-24T23:28:08.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f99067a9640 10 monclient(hunting): picked mon.noname-b con 0x7f9900135e10 addr v1:172.21.15.115:6790/0 2024-01-24T23:28:08.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f99067a9640 10 monclient(hunting): picked mon.noname-h con 0x7f99001361f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:28:08.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f99067a9640 10 monclient(hunting): picked mon.noname-a con 0x7f99001365f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:28:08.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f99067a9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9900120780 con 0x7f99001365f0 2024-01-24T23:28:08.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f99067a9640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990009ca00 con 0x7f9900135e10 2024-01-24T23:28:08.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f99067a9640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990009c6b0 con 0x7f99001361f0 2024-01-24T23:28:08.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f99067a9640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:08.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f99067a9640 10 monclient(hunting): authenticate will time out at 1375.912530s 2024-01-24T23:28:08.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98ff7fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f99001361f0 0x7f9900120070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:48732/0 (socket says 172.21.15.132:48732) 2024-01-24T23:28:08.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98ff7fe640 1 -- 172.21.15.132:0/1148105242 learned_addr learned my addr 172.21.15.132:0/1148105242 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:08.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fffff640 1 -- 172.21.15.132:0/1148105242 >> v1:172.21.15.115:6790/0 conn(0x7f9900135e10 legacy=0x7f990011f960 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:28:08.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f9904d46640 1 -- 172.21.15.132:0/1148105242 >> v1:172.21.15.115:6789/0 conn(0x7f99001365f0 legacy=0x7f99001d3390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:28:08.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 1 -- 172.21.15.132:0/1148105242 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 457681782 0 0) 0x7f990009c6b0 con 0x7f99001361f0 2024-01-24T23:28:08.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 10 monclient(hunting): _negotiate 2024-01-24T23:28:08.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:08.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:08.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 10 monclient(hunting): my global_id is 6890 2024-01-24T23:28:08.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 1 -- 172.21.15.132:0/1148105242 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98e0003530 con 0x7f99001361f0 2024-01-24T23:28:08.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 1 -- 172.21.15.132:0/1148105242 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2787423210 0 0) 0x7f98e0003530 con 0x7f99001361f0 2024-01-24T23:28:08.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 1 -- 172.21.15.132:0/1148105242 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f990009c6b0 con 0x7f99001361f0 2024-01-24T23:28:08.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 1 -- 172.21.15.132:0/1148105242 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f98e8001b50 con 0x7f99001361f0 2024-01-24T23:28:08.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:08.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:28:08.599 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 10 monclient(hunting): dump: 2024-01-24T23:28:08.599 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:08.599 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:08.599 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:08.599 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:08.599 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:08.599 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:08.599 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:08.599 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:08.599 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:08.599 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:08.599 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:08.600 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:08.600 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:08.600 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:08.600 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:08.600 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:08.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:08.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 1 -- 172.21.15.132:0/1148105242 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4157413347 0 0) 0x7f990009c6b0 con 0x7f99001361f0 2024-01-24T23:28:08.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 1 -- 172.21.15.132:0/1148105242 >> v1:172.21.15.115:6790/0 conn(0x7f9900135e10 legacy=0x7f990011f960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:28:08.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 1 -- 172.21.15.132:0/1148105242 >> v1:172.21.15.115:6789/0 conn(0x7f99001365f0 legacy=0x7f99001d3390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:28:08.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 10 monclient: _finish_hunting 0 2024-01-24T23:28:08.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 1 monclient: found mon.h 2024-01-24T23:28:08.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:08.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:08.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 1 -- 172.21.15.132:0/1148105242 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99001bf1a0 con 0x7f99001361f0 2024-01-24T23:28:08.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 10 monclient: _finish_auth 0 2024-01-24T23:28:08.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 10 monclient: _check_auth_tickets 2024-01-24T23:28:08.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:08.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f99067a9640 5 monclient: authenticate success, global_id 6890 2024-01-24T23:28:08.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f99067a9640 10 monclient: _renew_subs 2024-01-24T23:28:08.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f99067a9640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:08.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f99067a9640 1 -- 172.21.15.132:0/1148105242 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9900120c90 con 0x7f99001361f0 2024-01-24T23:28:08.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f99067a9640 10 monclient: _renew_subs 2024-01-24T23:28:08.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f99067a9640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:08.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f99067a9640 1 -- 172.21.15.132:0/1148105242 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9900120f10 con 0x7f99001361f0 2024-01-24T23:28:08.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 1 -- 172.21.15.132:0/1148105242 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f98e8004aa0 con 0x7f99001361f0 2024-01-24T23:28:08.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:08.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 1 -- 172.21.15.132:0/1148105242 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f98e8004f50 con 0x7f99001361f0 2024-01-24T23:28:08.601 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:08.602 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fe7fc640 4 set_mon_vals no callback set 2024-01-24T23:28:08.602 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:28:08.602 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 10 monclient: dump: 2024-01-24T23:28:08.602 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:08.602 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:08.602 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:08.602 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:08.602 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:08.602 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:08.602 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:08.602 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:08.602 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:08.602 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:08.602 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:08.602 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:08.603 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:08.603 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:08.603 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:08.603 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:08.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f99067a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:08.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f99067a9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:08.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f99067a9640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:08.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f99067a9640 1 -- 172.21.15.132:0/1148105242 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e4005090 con 0x7f99001361f0 2024-01-24T23:28:08.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 1 -- 172.21.15.132:0/1148105242 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f98e800fa50 con 0x7f99001361f0 2024-01-24T23:28:08.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fcff9640 1 -- 172.21.15.132:0/1148105242 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(238..238 src has 1..238) v4 ==== 8867+0+0 (unknown 660432594 0 0) 0x7f98e8044d60 con 0x7f99001361f0 2024-01-24T23:28:08.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.586+0000 7f98fffff640 10 monclient: get_auth_request con 0x7f98e0037010 auth_method 0 2024-01-24T23:28:08.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.590+0000 7f98fcff9640 1 -- 172.21.15.132:0/1148105242 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f98e8072310 con 0x7f99001361f0 2024-01-24T23:28:08.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.590+0000 7f98fcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:08.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.590+0000 7f98fcff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:08.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.750+0000 7f99067a9640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h27"}] 2024-01-24T23:28:08.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.750+0000 7f99067a9640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h27"}] 2024-01-24T23:28:08.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.750+0000 7f99067a9640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:08.751 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:08.750+0000 7f99067a9640 1 -- 172.21.15.132:0/1148105242 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h27"} v 0) v1 -- 0x7f98e4005340 con 0x7f99001361f0 2024-01-24T23:28:09.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.358+0000 7f98fcff9640 1 -- 172.21.15.132:0/1148105242 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h27"}]=0 removed item id -97 name 'h27' from crush map v239) v1 ==== 120+0+0 (unknown 3411877522 0 0) 0x7f98e80149d0 con 0x7f99001361f0 2024-01-24T23:28:09.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.358+0000 7f98fcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h27"}] 2024-01-24T23:28:09.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.358+0000 7f98fcff9640 10 monclient: _finish_command 2 = system:0 removed item id -97 name 'h27' from crush map 2024-01-24T23:28:09.384 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~125s 2024-01-24T23:28:09.461 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.458+0000 7f99067a9640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h27"}] 2024-01-24T23:28:09.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.458+0000 7f99067a9640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h27"}] 2024-01-24T23:28:09.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.458+0000 7f99067a9640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:09.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.458+0000 7f99067a9640 1 -- 172.21.15.132:0/1148105242 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h27"} v 0) v1 -- 0x7f98e4002220 con 0x7f99001361f0 2024-01-24T23:28:09.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.462+0000 7f98fcff9640 1 -- 172.21.15.132:0/1148105242 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h27"}]=0 device 'h27' does not appear in the crush map v239) v1 ==== 120+0+0 (unknown 795762486 0 0) 0x7f98e8013070 con 0x7f99001361f0 2024-01-24T23:28:09.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.462+0000 7f98fcff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h27"}] 2024-01-24T23:28:09.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.462+0000 7f98fcff9640 10 monclient: _finish_command 3 = system:0 device 'h27' does not appear in the crush map 2024-01-24T23:28:09.463 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h27' does not appear in the crush map 2024-01-24T23:28:09.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.462+0000 7f99067a9640 1 -- 172.21.15.132:0/1148105242 >> v1:172.21.15.132:6812/1835401922 conn(0x7f98e0037010 legacy=0x7f98e00394d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:09.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.462+0000 7f99067a9640 10 monclient: shutdown 2024-01-24T23:28:09.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.462+0000 7f99067a9640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:09.465 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.462+0000 7f99067a9640 1 -- 172.21.15.132:0/1148105242 >> v1:172.21.15.132:6791/0 conn(0x7f99001361f0 legacy=0x7f9900120070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:09.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.462+0000 7f99067a9640 1 -- 172.21.15.132:0/1148105242 shutdown_connections 2024-01-24T23:28:09.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.462+0000 7f99067a9640 1 -- 172.21.15.132:0/1148105242 >> 172.21.15.132:0/1148105242 conn(0x7f99000a6790 msgr2=0x7f990006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:09.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.462+0000 7f99067a9640 1 -- 172.21.15.132:0/1148105242 shutdown_connections 2024-01-24T23:28:09.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.462+0000 7f99067a9640 1 -- 172.21.15.132:0/1148105242 wait complete. 2024-01-24T23:28:09.479 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:28:09.479 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h28 2024-01-24T23:28:09.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.594+0000 7fae602d6640 10 monclient: get_monmap_and_config 2024-01-24T23:28:09.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.594+0000 7fae602d6640 10 monclient: build_initial_monmap 2024-01-24T23:28:09.596 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.594+0000 7fae602d6640 10 monclient: monmap: 2024-01-24T23:28:09.596 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:09.596 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:09.596 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:09.599221+0000 2024-01-24T23:28:09.596 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:09.599221+0000 2024-01-24T23:28:09.596 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:09.597 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:09.597 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:09.597 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:09.597 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:09.597 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:09.597 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:09.597 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:09.597 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:09.597 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:09.597 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:09.597 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:09.603 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.594+0000 7fae602d6640 1 Processor -- start 2024-01-24T23:28:09.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.594+0000 7fae602d6640 1 -- start start 2024-01-24T23:28:09.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.594+0000 7fae602d6640 10 monclient: init 2024-01-24T23:28:09.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.594+0000 7fae602d6640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:09.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.594+0000 7fae602d6640 10 monclient: _add_conns ranks=[7,8,6,4,0,5,2,1,3] 2024-01-24T23:28:09.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.594+0000 7fae602d6640 10 monclient(hunting): picked mon.noname-i con 0x7fae5809ff20 addr v1:172.21.15.132:6792/0 2024-01-24T23:28:09.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.594+0000 7fae602d6640 10 monclient(hunting): picked mon.noname-e con 0x7fae5809f9c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:09.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.594+0000 7fae602d6640 10 monclient(hunting): picked mon.noname-d con 0x7fae5813b2c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:28:09.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.594+0000 7fae602d6640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae58064700 con 0x7fae5813b2c0 2024-01-24T23:28:09.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.594+0000 7fae602d6640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae5809e760 con 0x7fae5809ff20 2024-01-24T23:28:09.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.594+0000 7fae602d6640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae580a1160 con 0x7fae5809f9c0 2024-01-24T23:28:09.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.594+0000 7fae602d6640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:09.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.594+0000 7fae602d6640 10 monclient(hunting): authenticate will time out at 1376.923830s 2024-01-24T23:28:09.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.594+0000 7fae5e072640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fae5809ff20 legacy=0x7fae580a0300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:28:09.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d871640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fae5809f9c0 0x7fae58139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:56708/0 (socket says 172.21.15.132:56708) 2024-01-24T23:28:09.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5e873640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fae5813b2c0 0x7fae5813d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:57876/0 (socket says 172.21.15.132:57876) 2024-01-24T23:28:09.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d871640 1 -- 172.21.15.132:0/2562662219 learned_addr learned my addr 172.21.15.132:0/2562662219 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:09.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 1 -- 172.21.15.132:0/2562662219 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3480785125 0 0) 0x7fae58064700 con 0x7fae5813b2c0 2024-01-24T23:28:09.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 10 monclient(hunting): _negotiate 2024-01-24T23:28:09.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:09.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:09.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 10 monclient(hunting): my global_id is 7107 2024-01-24T23:28:09.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 1 -- 172.21.15.132:0/2562662219 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae44003530 con 0x7fae5813b2c0 2024-01-24T23:28:09.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 1 -- 172.21.15.132:0/2562662219 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 237032199 0 0) 0x7fae580a1160 con 0x7fae5809f9c0 2024-01-24T23:28:09.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 10 monclient(hunting): _negotiate 2024-01-24T23:28:09.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:09.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:09.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 10 monclient(hunting): my global_id is 86075 2024-01-24T23:28:09.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 1 -- 172.21.15.132:0/2562662219 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae58064700 con 0x7fae5809f9c0 2024-01-24T23:28:09.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 1 -- 172.21.15.132:0/2562662219 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1037947086 0 0) 0x7fae44003530 con 0x7fae5813b2c0 2024-01-24T23:28:09.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 1 -- 172.21.15.132:0/2562662219 >> v1:172.21.15.115:6793/0 conn(0x7fae5809f9c0 legacy=0x7fae58139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:09.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 1 -- 172.21.15.132:0/2562662219 >> v1:172.21.15.132:6792/0 conn(0x7fae5809ff20 legacy=0x7fae580a0300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:28:09.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 10 monclient: _finish_hunting 0 2024-01-24T23:28:09.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 1 monclient: found mon.noname-d 2024-01-24T23:28:09.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:09.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:28:09.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 1 -- 172.21.15.132:0/2562662219 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae580a0a10 con 0x7fae5813b2c0 2024-01-24T23:28:09.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 10 monclient: _finish_auth 0 2024-01-24T23:28:09.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 10 monclient: _check_auth_tickets 2024-01-24T23:28:09.607 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:09.607 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 5 monclient: authenticate success, global_id 7107 2024-01-24T23:28:09.607 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 1 -- 172.21.15.132:0/2562662219 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fae50001a70 con 0x7fae5813b2c0 2024-01-24T23:28:09.607 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:09.607 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:09.607 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:28:09.607 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 10 monclient: dump: 2024-01-24T23:28:09.607 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:09.607 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:09.607 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:09.607 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:09.607 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:09.607 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:09.607 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:09.608 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:09.608 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:09.608 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:09.608 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:09.608 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:09.608 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:09.608 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:09.608 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:09.608 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:09.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:09.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 1 -- 172.21.15.132:0/2562662219 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fae50001d50 con 0x7fae5813b2c0 2024-01-24T23:28:09.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:09.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:09.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 4 set_mon_vals no callback set 2024-01-24T23:28:09.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 1 -- 172.21.15.132:0/2562662219 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fae50004eb0 con 0x7fae5813b2c0 2024-01-24T23:28:09.609 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:09.609 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:28:09.609 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5d070640 10 monclient: dump: 2024-01-24T23:28:09.609 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:09.609 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:09.609 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:09.609 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:09.609 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:09.609 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:09.609 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:09.609 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:09.609 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:09.610 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:09.610 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:09.610 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:09.610 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:09.610 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:09.610 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:09.610 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:09.610 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 10 monclient: shutdown 2024-01-24T23:28:09.610 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:09.610 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 1 -- 172.21.15.132:0/2562662219 >> v1:172.21.15.115:6792/0 conn(0x7fae5813b2c0 legacy=0x7fae5813d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:09.610 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 1 -- 172.21.15.132:0/2562662219 shutdown_connections 2024-01-24T23:28:09.610 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 1 -- 172.21.15.132:0/2562662219 >> 172.21.15.132:0/2562662219 conn(0x7fae580a5a10 msgr2=0x7fae580a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:09.610 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 1 -- 172.21.15.132:0/2562662219 shutdown_connections 2024-01-24T23:28:09.610 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 1 -- 172.21.15.132:0/2562662219 wait complete. 2024-01-24T23:28:09.611 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 10 monclient: build_initial_monmap 2024-01-24T23:28:09.611 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 10 monclient: monmap: 2024-01-24T23:28:09.611 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:09.611 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:09.611 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:09.611 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:09.611 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:09.611 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:09.611 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:09.611 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:09.611 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:09.611 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:09.611 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:09.611 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:09.612 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:09.612 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:09.612 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:09.612 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:09.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 1 Processor -- start 2024-01-24T23:28:09.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 1 -- start start 2024-01-24T23:28:09.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 10 monclient: init 2024-01-24T23:28:09.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:09.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 10 monclient: _add_conns ranks=[1,8,4,5,0,2,7,3,6] 2024-01-24T23:28:09.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 10 monclient(hunting): picked mon.noname-f con 0x7fae5809f9c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:28:09.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 10 monclient(hunting): picked mon.noname-e con 0x7fae5809ff20 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:09.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 10 monclient(hunting): picked mon.noname-c con 0x7fae5813b2c0 addr v1:172.21.15.115:6791/0 2024-01-24T23:28:09.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae5811cbc0 con 0x7fae5809f9c0 2024-01-24T23:28:09.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae5811e970 con 0x7fae5813b2c0 2024-01-24T23:28:09.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae58071680 con 0x7fae5809ff20 2024-01-24T23:28:09.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:09.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae602d6640 10 monclient(hunting): authenticate will time out at 1376.927761s 2024-01-24T23:28:09.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5e072640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fae5809f9c0 0x7fae581c9440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:35132/0 (socket says 172.21.15.132:35132) 2024-01-24T23:28:09.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.598+0000 7fae5e072640 1 -- 172.21.15.132:0/2380443918 learned_addr learned my addr 172.21.15.132:0/2380443918 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:09.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 1 -- 172.21.15.132:0/2380443918 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 668829919 0 0) 0x7fae5811cbc0 con 0x7fae5809f9c0 2024-01-24T23:28:09.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 10 monclient(hunting): _negotiate 2024-01-24T23:28:09.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:09.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:09.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 10 monclient(hunting): my global_id is 124444 2024-01-24T23:28:09.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 1 -- 172.21.15.132:0/2380443918 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae3c003530 con 0x7fae5809f9c0 2024-01-24T23:28:09.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 1 -- 172.21.15.132:0/2380443918 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2395499408 0 0) 0x7fae3c003530 con 0x7fae5809f9c0 2024-01-24T23:28:09.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 1 -- 172.21.15.132:0/2380443918 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fae5811cbc0 con 0x7fae5809f9c0 2024-01-24T23:28:09.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 1 -- 172.21.15.132:0/2380443918 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fae54004300 con 0x7fae5809f9c0 2024-01-24T23:28:09.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:09.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:28:09.614 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 10 monclient(hunting): dump: 2024-01-24T23:28:09.614 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:09.614 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:09.614 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:09.614 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:09.614 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:09.614 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:09.614 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:09.614 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:09.614 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:09.614 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:09.614 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:09.614 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:09.614 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:09.614 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:09.615 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:09.615 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:09.615 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:09.615 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 1 -- 172.21.15.132:0/2380443918 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1227027662 0 0) 0x7fae58071680 con 0x7fae5809ff20 2024-01-24T23:28:09.615 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 10 monclient(hunting): _negotiate 2024-01-24T23:28:09.615 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:09.615 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:09.615 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 10 monclient(hunting): my global_id is 86084 2024-01-24T23:28:09.615 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 1 -- 172.21.15.132:0/2380443918 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae3c003530 con 0x7fae5809ff20 2024-01-24T23:28:09.615 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 1 -- 172.21.15.132:0/2380443918 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2504015681 0 0) 0x7fae5811cbc0 con 0x7fae5809f9c0 2024-01-24T23:28:09.615 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 1 -- 172.21.15.132:0/2380443918 >> v1:172.21.15.115:6793/0 conn(0x7fae5809ff20 legacy=0x7fae581c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:09.615 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 1 -- 172.21.15.132:0/2380443918 >> v1:172.21.15.115:6791/0 conn(0x7fae5813b2c0 legacy=0x7fae581d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:09.615 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 10 monclient: _finish_hunting 0 2024-01-24T23:28:09.615 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 1 monclient: found mon.f 2024-01-24T23:28:09.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:09.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:09.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 1 -- 172.21.15.132:0/2380443918 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae581ca260 con 0x7fae5809f9c0 2024-01-24T23:28:09.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 10 monclient: _finish_auth 0 2024-01-24T23:28:09.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 10 monclient: _check_auth_tickets 2024-01-24T23:28:09.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:09.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae602d6640 5 monclient: authenticate success, global_id 124444 2024-01-24T23:28:09.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae602d6640 10 monclient: _renew_subs 2024-01-24T23:28:09.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae602d6640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:09.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae602d6640 1 -- 172.21.15.132:0/2380443918 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae58071960 con 0x7fae5809f9c0 2024-01-24T23:28:09.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae602d6640 10 monclient: _renew_subs 2024-01-24T23:28:09.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae602d6640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:09.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae602d6640 1 -- 172.21.15.132:0/2380443918 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fae5811d5e0 con 0x7fae5809f9c0 2024-01-24T23:28:09.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 1 -- 172.21.15.132:0/2380443918 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fae54004620 con 0x7fae5809f9c0 2024-01-24T23:28:09.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:09.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 1 -- 172.21.15.132:0/2380443918 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fae54004b50 con 0x7fae5809f9c0 2024-01-24T23:28:09.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:09.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae5c86f640 4 set_mon_vals no callback set 2024-01-24T23:28:09.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:28:09.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 10 monclient: dump: 2024-01-24T23:28:09.617 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:09.617 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:09.617 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:09.617 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:09.617 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:09.617 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:09.617 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:09.617 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:09.617 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:09.618 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:09.618 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:09.618 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:09.618 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:09.618 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:09.618 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:09.618 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:09.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae602d6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:09.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 1 -- 172.21.15.132:0/2380443918 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fae54004d30 con 0x7fae5809f9c0 2024-01-24T23:28:09.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae602d6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:09.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae602d6640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:09.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae602d6640 1 -- 172.21.15.132:0/2380443918 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae40005090 con 0x7fae5809f9c0 2024-01-24T23:28:09.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae4effd640 1 -- 172.21.15.132:0/2380443918 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(239..239 src has 1..239) v4 ==== 8654+0+0 (unknown 2267018297 0 0) 0x7fae54044c80 con 0x7fae5809f9c0 2024-01-24T23:28:09.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.602+0000 7fae5d871640 10 monclient: get_auth_request con 0x7fae3c036fd0 auth_method 0 2024-01-24T23:28:09.619 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.606+0000 7fae4effd640 1 -- 172.21.15.132:0/2380443918 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fae540149a0 con 0x7fae5809f9c0 2024-01-24T23:28:09.619 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.606+0000 7fae4effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:09.619 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.606+0000 7fae4effd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:09.753 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.750+0000 7fae602d6640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h28"}] 2024-01-24T23:28:09.753 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.750+0000 7fae602d6640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h28"}] 2024-01-24T23:28:09.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.750+0000 7fae602d6640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:09.754 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:09.750+0000 7fae602d6640 1 -- 172.21.15.132:0/2380443918 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd crush rm", "name": "h28"} v 0) v1 -- 0x7fae40005340 con 0x7fae5809f9c0 2024-01-24T23:28:10.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.366+0000 7fae4effd640 1 -- 172.21.15.132:0/2380443918 <== mon.1 v1:172.21.15.132:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h28"}]=0 removed item id -100 name 'h28' from crush map v240) v1 ==== 121+0+0 (unknown 3990925037 0 0) 0x7fae54013040 con 0x7fae5809f9c0 2024-01-24T23:28:10.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.366+0000 7fae4effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h28"}] 2024-01-24T23:28:10.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.366+0000 7fae4effd640 10 monclient: _finish_command 2 = system:0 removed item id -100 name 'h28' from crush map 2024-01-24T23:28:10.462 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.462+0000 7fae602d6640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h28"}] 2024-01-24T23:28:10.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.462+0000 7fae602d6640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h28"}] 2024-01-24T23:28:10.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.462+0000 7fae602d6640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:10.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.462+0000 7fae602d6640 1 -- 172.21.15.132:0/2380443918 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd crush rm", "name": "h28"} v 0) v1 -- 0x7fae40002220 con 0x7fae5809f9c0 2024-01-24T23:28:10.463 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.462+0000 7fae4effd640 1 -- 172.21.15.132:0/2380443918 <== mon.1 v1:172.21.15.132:6789/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h28"}]=0 device 'h28' does not appear in the crush map v240) v1 ==== 120+0+0 (unknown 1306066437 0 0) 0x7fae540147c0 con 0x7fae5809f9c0 2024-01-24T23:28:10.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.462+0000 7fae4effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h28"}] 2024-01-24T23:28:10.464 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.462+0000 7fae4effd640 10 monclient: _finish_command 3 = system:0 device 'h28' does not appear in the crush map 2024-01-24T23:28:10.464 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h28' does not appear in the crush map 2024-01-24T23:28:10.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.466+0000 7fae602d6640 1 -- 172.21.15.132:0/2380443918 >> v1:172.21.15.132:6812/1835401922 conn(0x7fae3c036fd0 legacy=0x7fae3c039490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:10.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.466+0000 7fae602d6640 10 monclient: shutdown 2024-01-24T23:28:10.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.466+0000 7fae602d6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:10.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.466+0000 7fae602d6640 1 -- 172.21.15.132:0/2380443918 >> v1:172.21.15.132:6789/0 conn(0x7fae5809f9c0 legacy=0x7fae581c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:10.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.466+0000 7fae602d6640 1 -- 172.21.15.132:0/2380443918 shutdown_connections 2024-01-24T23:28:10.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.466+0000 7fae602d6640 1 -- 172.21.15.132:0/2380443918 >> 172.21.15.132:0/2380443918 conn(0x7fae580a5a10 msgr2=0x7fae580a2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:10.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.466+0000 7fae602d6640 1 -- 172.21.15.132:0/2380443918 shutdown_connections 2024-01-24T23:28:10.466 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.466+0000 7fae602d6640 1 -- 172.21.15.132:0/2380443918 wait complete. 2024-01-24T23:28:10.480 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:28:10.480 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h29 2024-01-24T23:28:10.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f607b0ad640 10 monclient: get_monmap_and_config 2024-01-24T23:28:10.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f607b0ad640 10 monclient: build_initial_monmap 2024-01-24T23:28:10.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f607b0ad640 10 monclient: monmap: 2024-01-24T23:28:10.595 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:10.595 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:10.595 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:10.598375+0000 2024-01-24T23:28:10.596 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:10.598375+0000 2024-01-24T23:28:10.596 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:10.596 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:10.596 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:10.596 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:10.596 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:10.596 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:10.596 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:10.596 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:10.596 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:10.596 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:10.596 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:10.596 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:10.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f607b0ad640 1 Processor -- start 2024-01-24T23:28:10.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f607b0ad640 1 -- start start 2024-01-24T23:28:10.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f607b0ad640 10 monclient: init 2024-01-24T23:28:10.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f607b0ad640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:10.604 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f607b0ad640 10 monclient: _add_conns ranks=[2,4,7,0,8,5,6,3,1] 2024-01-24T23:28:10.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f607b0ad640 10 monclient(hunting): picked mon.noname-b con 0x7f6074136c20 addr v1:172.21.15.115:6790/0 2024-01-24T23:28:10.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f607b0ad640 10 monclient(hunting): picked mon.noname-c con 0x7f6074137000 addr v1:172.21.15.115:6791/0 2024-01-24T23:28:10.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f607b0ad640 10 monclient(hunting): picked mon.noname-i con 0x7f607413d1d0 addr v1:172.21.15.132:6792/0 2024-01-24T23:28:10.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f607b0ad640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6074064700 con 0x7f6074136c20 2024-01-24T23:28:10.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f607b0ad640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607409b2f0 con 0x7f6074137000 2024-01-24T23:28:10.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f607b0ad640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607409c3e0 con 0x7f607413d1d0 2024-01-24T23:28:10.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f607b0ad640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:10.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f607b0ad640 10 monclient(hunting): authenticate will time out at 1377.922905s 2024-01-24T23:28:10.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f607964a640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f607413d1d0 legacy=0x7f607413f5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:28:10.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f6078e49640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f6074136c20 legacy=0x7f607409bcd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:28:10.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f6073fff640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f6074137000 0x7f6074137420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:34342/0 (socket says 172.21.15.132:34342) 2024-01-24T23:28:10.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f6073fff640 1 -- 172.21.15.132:0/1585274220 learned_addr learned my addr 172.21.15.132:0/1585274220 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:10.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f60737fe640 1 -- 172.21.15.132:0/1585274220 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2379810121 0 0) 0x7f607409b2f0 con 0x7f6074137000 2024-01-24T23:28:10.605 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f60737fe640 10 monclient(hunting): _negotiate 2024-01-24T23:28:10.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f60737fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:10.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f60737fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:10.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f60737fe640 10 monclient(hunting): my global_id is 95053 2024-01-24T23:28:10.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.594+0000 7f60737fe640 1 -- 172.21.15.132:0/1585274220 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f605c003530 con 0x7f6074137000 2024-01-24T23:28:10.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60737fe640 1 -- 172.21.15.132:0/1585274220 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1849492189 0 0) 0x7f605c003530 con 0x7f6074137000 2024-01-24T23:28:10.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60737fe640 1 -- 172.21.15.132:0/1585274220 >> v1:172.21.15.132:6792/0 conn(0x7f607413d1d0 legacy=0x7f607413f5c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:28:10.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60737fe640 1 -- 172.21.15.132:0/1585274220 >> v1:172.21.15.115:6790/0 conn(0x7f6074136c20 legacy=0x7f607409bcd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:28:10.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60737fe640 10 monclient: _finish_hunting 0 2024-01-24T23:28:10.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60737fe640 1 monclient: found mon.noname-c 2024-01-24T23:28:10.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:10.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60737fe640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:28:10.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60737fe640 1 -- 172.21.15.132:0/1585274220 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607409b5d0 con 0x7f6074137000 2024-01-24T23:28:10.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60737fe640 10 monclient: _finish_auth 0 2024-01-24T23:28:10.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60737fe640 10 monclient: _check_auth_tickets 2024-01-24T23:28:10.606 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:10.607 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60737fe640 1 -- 172.21.15.132:0/1585274220 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f60680017c0 con 0x7f6074137000 2024-01-24T23:28:10.607 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60737fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:10.607 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60737fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:28:10.607 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60737fe640 10 monclient: dump: 2024-01-24T23:28:10.607 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:10.607 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:10.607 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:10.607 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:10.607 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:10.607 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:10.607 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:10.607 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:10.607 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:10.607 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:10.607 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:10.608 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:10.608 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:10.608 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:10.608 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:10.608 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:10.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 5 monclient: authenticate success, global_id 95053 2024-01-24T23:28:10.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:10.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60737fe640 1 -- 172.21.15.132:0/1585274220 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6068001ae0 con 0x7f6074137000 2024-01-24T23:28:10.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60737fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:10.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:10.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 4 set_mon_vals no callback set 2024-01-24T23:28:10.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60737fe640 1 -- 172.21.15.132:0/1585274220 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6068004bf0 con 0x7f6074137000 2024-01-24T23:28:10.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60737fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:10.608 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60737fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:28:10.609 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60737fe640 10 monclient: dump: 2024-01-24T23:28:10.609 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:10.609 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:10.609 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:10.609 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:10.609 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:10.609 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:10.609 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:10.609 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:10.609 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:10.609 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:10.609 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:10.609 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:10.609 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:10.609 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:10.609 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:10.610 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:10.610 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 10 monclient: shutdown 2024-01-24T23:28:10.610 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:10.610 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 1 -- 172.21.15.132:0/1585274220 >> v1:172.21.15.115:6791/0 conn(0x7f6074137000 legacy=0x7f6074137420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:10.610 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 1 -- 172.21.15.132:0/1585274220 shutdown_connections 2024-01-24T23:28:10.610 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 1 -- 172.21.15.132:0/1585274220 >> 172.21.15.132:0/1585274220 conn(0x7f60740a5c70 msgr2=0x7f60740a6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:10.610 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 1 -- 172.21.15.132:0/1585274220 shutdown_connections 2024-01-24T23:28:10.610 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 1 -- 172.21.15.132:0/1585274220 wait complete. 2024-01-24T23:28:10.610 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 10 monclient: build_initial_monmap 2024-01-24T23:28:10.610 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 10 monclient: monmap: 2024-01-24T23:28:10.610 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:10.610 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:10.610 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:10.610 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:10.610 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:10.611 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:10.611 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:10.611 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:10.611 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:10.611 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:10.611 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:10.611 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:10.611 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:10.611 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:10.611 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:10.611 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:10.611 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 1 Processor -- start 2024-01-24T23:28:10.611 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 1 -- start start 2024-01-24T23:28:10.611 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 10 monclient: init 2024-01-24T23:28:10.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:10.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 10 monclient: _add_conns ranks=[5,8,3,6,1,7,0,2,4] 2024-01-24T23:28:10.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 10 monclient(hunting): picked mon.noname-h con 0x7f6074136c20 addr v1:172.21.15.132:6791/0 2024-01-24T23:28:10.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 10 monclient(hunting): picked mon.noname-e con 0x7f6074137000 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:10.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 10 monclient(hunting): picked mon.noname-g con 0x7f607413d1d0 addr v1:172.21.15.132:6790/0 2024-01-24T23:28:10.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607406cf10 con 0x7f607413d1d0 2024-01-24T23:28:10.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607409c6b0 con 0x7f6074136c20 2024-01-24T23:28:10.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6074137f90 con 0x7f6074137000 2024-01-24T23:28:10.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:10.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 10 monclient(hunting): authenticate will time out at 1377.926298s 2024-01-24T23:28:10.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f6078e49640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f6074136c20 0x7f60741cb610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:48748/0 (socket says 172.21.15.132:48748) 2024-01-24T23:28:10.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f6078e49640 1 -- 172.21.15.132:0/1081825003 learned_addr learned my addr 172.21.15.132:0/1081825003 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:10.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 1 -- 172.21.15.132:0/1081825003 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1646431562 0 0) 0x7f607409c6b0 con 0x7f6074136c20 2024-01-24T23:28:10.612 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 10 monclient(hunting): _negotiate 2024-01-24T23:28:10.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:10.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:10.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 10 monclient(hunting): my global_id is 6899 2024-01-24T23:28:10.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 1 -- 172.21.15.132:0/1081825003 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6060003530 con 0x7f6074136c20 2024-01-24T23:28:10.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 1 -- 172.21.15.132:0/1081825003 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 89015222 0 0) 0x7f607406cf10 con 0x7f607413d1d0 2024-01-24T23:28:10.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 10 monclient(hunting): _negotiate 2024-01-24T23:28:10.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:10.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:10.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 10 monclient(hunting): my global_id is 6897 2024-01-24T23:28:10.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 1 -- 172.21.15.132:0/1081825003 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f607409c6b0 con 0x7f607413d1d0 2024-01-24T23:28:10.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 1 -- 172.21.15.132:0/1081825003 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2980182570 0 0) 0x7f607409c6b0 con 0x7f607413d1d0 2024-01-24T23:28:10.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 1 -- 172.21.15.132:0/1081825003 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f607406cf10 con 0x7f607413d1d0 2024-01-24T23:28:10.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 1 -- 172.21.15.132:0/1081825003 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f606c004490 con 0x7f607413d1d0 2024-01-24T23:28:10.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:10.613 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:28:10.614 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 10 monclient(hunting): dump: 2024-01-24T23:28:10.614 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:10.614 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:10.614 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:10.614 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:10.614 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:10.614 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:10.614 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:10.614 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:10.614 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:10.614 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:10.614 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:10.614 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:10.614 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:10.615 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:10.615 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:10.615 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:10.615 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:10.615 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 1 -- 172.21.15.132:0/1081825003 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3009614540 0 0) 0x7f6060003530 con 0x7f6074136c20 2024-01-24T23:28:10.615 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 1 -- 172.21.15.132:0/1081825003 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f607409c6b0 con 0x7f6074136c20 2024-01-24T23:28:10.615 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 1 -- 172.21.15.132:0/1081825003 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6064004330 con 0x7f6074136c20 2024-01-24T23:28:10.615 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:10.615 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:28:10.615 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 10 monclient(hunting): dump: 2024-01-24T23:28:10.615 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:10.615 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:10.615 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:10.615 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:10.615 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:10.616 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:10.616 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:10.616 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:10.616 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:10.616 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:10.616 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:10.616 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:10.616 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:10.616 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:10.616 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:10.616 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:10.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 1 -- 172.21.15.132:0/1081825003 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 632168375 0 0) 0x7f6074137f90 con 0x7f6074137000 2024-01-24T23:28:10.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 10 monclient(hunting): _negotiate 2024-01-24T23:28:10.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:10.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:10.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 10 monclient(hunting): my global_id is 86093 2024-01-24T23:28:10.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 1 -- 172.21.15.132:0/1081825003 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6060003530 con 0x7f6074137000 2024-01-24T23:28:10.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 1 -- 172.21.15.132:0/1081825003 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3526771441 0 0) 0x7f607406cf10 con 0x7f607413d1d0 2024-01-24T23:28:10.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 1 -- 172.21.15.132:0/1081825003 >> v1:172.21.15.115:6793/0 conn(0x7f6074137000 legacy=0x7f60741d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:10.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 1 -- 172.21.15.132:0/1081825003 >> v1:172.21.15.132:6791/0 conn(0x7f6074136c20 legacy=0x7f60741cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:10.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 10 monclient: _finish_hunting 0 2024-01-24T23:28:10.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 1 monclient: found mon.g 2024-01-24T23:28:10.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:10.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:10.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 1 -- 172.21.15.132:0/1081825003 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607409fe10 con 0x7f607413d1d0 2024-01-24T23:28:10.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 10 monclient: _finish_auth 0 2024-01-24T23:28:10.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 10 monclient: _check_auth_tickets 2024-01-24T23:28:10.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f60717fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:10.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 5 monclient: authenticate success, global_id 6897 2024-01-24T23:28:10.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 10 monclient: _renew_subs 2024-01-24T23:28:10.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:10.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 1 -- 172.21.15.132:0/1081825003 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6074138240 con 0x7f607413d1d0 2024-01-24T23:28:10.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 10 monclient: _renew_subs 2024-01-24T23:28:10.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:10.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.598+0000 7f607b0ad640 1 -- 172.21.15.132:0/1081825003 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f607409eb80 con 0x7f607413d1d0 2024-01-24T23:28:10.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.602+0000 7f607b0ad640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:10.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.602+0000 7f607b0ad640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:10.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.602+0000 7f607b0ad640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:10.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.602+0000 7f607b0ad640 1 -- 172.21.15.132:0/1081825003 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f607409edd0 con 0x7f607413d1d0 2024-01-24T23:28:10.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.602+0000 7f60717fa640 1 -- 172.21.15.132:0/1081825003 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f606c004b70 con 0x7f607413d1d0 2024-01-24T23:28:10.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.602+0000 7f60717fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:10.619 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.602+0000 7f60717fa640 1 -- 172.21.15.132:0/1081825003 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f606c005020 con 0x7f607413d1d0 2024-01-24T23:28:10.619 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.602+0000 7f60717fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:10.619 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.602+0000 7f60717fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:28:10.619 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.602+0000 7f60717fa640 10 monclient: dump: 2024-01-24T23:28:10.619 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:10.619 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:10.619 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:10.619 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:10.619 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:10.619 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:10.619 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:10.619 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:10.619 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:10.619 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:10.619 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:10.620 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:10.620 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:10.620 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:10.620 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:10.620 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:10.620 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.602+0000 7f6072ffd640 4 set_mon_vals no callback set 2024-01-24T23:28:10.620 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.602+0000 7f60717fa640 1 -- 172.21.15.132:0/1081825003 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f606c00faa0 con 0x7f607413d1d0 2024-01-24T23:28:10.620 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.602+0000 7f60717fa640 1 -- 172.21.15.132:0/1081825003 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(240..240 src has 1..240) v4 ==== 8441+0+0 (unknown 3693550791 0 0) 0x7f606c044b20 con 0x7f607413d1d0 2024-01-24T23:28:10.620 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.602+0000 7f6078e49640 10 monclient: get_auth_request con 0x7f6060037770 auth_method 0 2024-01-24T23:28:10.620 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.606+0000 7f60717fa640 1 -- 172.21.15.132:0/1081825003 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f606c0149e0 con 0x7f607413d1d0 2024-01-24T23:28:10.621 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.606+0000 7f60717fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:10.621 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.606+0000 7f60717fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:10.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.766+0000 7f607b0ad640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h29"}] 2024-01-24T23:28:10.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.766+0000 7f607b0ad640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h29"}] 2024-01-24T23:28:10.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.766+0000 7f607b0ad640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:10.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:10.766+0000 7f607b0ad640 1 -- 172.21.15.132:0/1081825003 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rm", "name": "h29"} v 0) v1 -- 0x7f6074074770 con 0x7f607413d1d0 2024-01-24T23:28:11.392 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.390+0000 7f60717fa640 1 -- 172.21.15.132:0/1081825003 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h29"}]=0 removed item id -103 name 'h29' from crush map v241) v1 ==== 121+0+0 (unknown 398530949 0 0) 0x7f606c013080 con 0x7f607413d1d0 2024-01-24T23:28:11.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.390+0000 7f60717fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h29"}] 2024-01-24T23:28:11.393 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.390+0000 7f60717fa640 10 monclient: _finish_command 2 = system:0 removed item id -103 name 'h29' from crush map 2024-01-24T23:28:11.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.494+0000 7f607b0ad640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h29"}] 2024-01-24T23:28:11.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.494+0000 7f607b0ad640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h29"}] 2024-01-24T23:28:11.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.494+0000 7f607b0ad640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:11.495 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.494+0000 7f607b0ad640 1 -- 172.21.15.132:0/1081825003 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rm", "name": "h29"} v 0) v1 -- 0x7f60740aa260 con 0x7f607413d1d0 2024-01-24T23:28:11.500 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.494+0000 7f60717fa640 1 -- 172.21.15.132:0/1081825003 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h29"}]=0 device 'h29' does not appear in the crush map v241) v1 ==== 120+0+0 (unknown 4192261802 0 0) 0x7f606c014800 con 0x7f607413d1d0 2024-01-24T23:28:11.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.494+0000 7f60717fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h29"}] 2024-01-24T23:28:11.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.494+0000 7f60717fa640 10 monclient: _finish_command 3 = system:0 device 'h29' does not appear in the crush map 2024-01-24T23:28:11.501 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h29' does not appear in the crush map 2024-01-24T23:28:11.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.498+0000 7f607b0ad640 1 -- 172.21.15.132:0/1081825003 >> v1:172.21.15.132:6812/1835401922 conn(0x7f6060037770 legacy=0x7f6060039c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:11.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.498+0000 7f607b0ad640 10 monclient: shutdown 2024-01-24T23:28:11.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.498+0000 7f607b0ad640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:11.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.498+0000 7f607b0ad640 1 -- 172.21.15.132:0/1081825003 >> v1:172.21.15.132:6790/0 conn(0x7f607413d1d0 legacy=0x7f60741d3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:11.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.498+0000 7f607b0ad640 1 -- 172.21.15.132:0/1081825003 shutdown_connections 2024-01-24T23:28:11.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.498+0000 7f607b0ad640 1 -- 172.21.15.132:0/1081825003 >> 172.21.15.132:0/1081825003 conn(0x7f60740a5c70 msgr2=0x7f6074066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:11.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.498+0000 7f607b0ad640 1 -- 172.21.15.132:0/1081825003 shutdown_connections 2024-01-24T23:28:11.501 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.498+0000 7f607b0ad640 1 -- 172.21.15.132:0/1081825003 wait complete. 2024-01-24T23:28:11.513 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:28:11.513 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h30 2024-01-24T23:28:11.624 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.622+0000 7f6fe3e6e640 10 monclient: get_monmap_and_config 2024-01-24T23:28:11.624 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.622+0000 7f6fe3e6e640 10 monclient: build_initial_monmap 2024-01-24T23:28:11.625 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.622+0000 7f6fe3e6e640 10 monclient: monmap: 2024-01-24T23:28:11.625 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:11.625 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:11.625 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:11.627483+0000 2024-01-24T23:28:11.625 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:11.627483+0000 2024-01-24T23:28:11.625 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:11.625 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:11.625 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:11.625 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:11.625 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:11.625 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:11.625 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:11.626 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:11.626 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:11.626 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:11.626 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:11.626 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:11.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.622+0000 7f6fe3e6e640 1 Processor -- start 2024-01-24T23:28:11.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.622+0000 7f6fe3e6e640 1 -- start start 2024-01-24T23:28:11.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.622+0000 7f6fe3e6e640 10 monclient: init 2024-01-24T23:28:11.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:11.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 10 monclient: _add_conns ranks=[0,4,6,2,5,3,7,1,8] 2024-01-24T23:28:11.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 10 monclient(hunting): picked mon.noname-a con 0x7f6fdc133b50 addr v1:172.21.15.115:6789/0 2024-01-24T23:28:11.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 10 monclient(hunting): picked mon.noname-c con 0x7f6fdc09ec20 addr v1:172.21.15.115:6791/0 2024-01-24T23:28:11.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 10 monclient(hunting): picked mon.noname-d con 0x7f6fdc13a1d0 addr v1:172.21.15.115:6792/0 2024-01-24T23:28:11.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fdc064700 con 0x7f6fdc133b50 2024-01-24T23:28:11.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fdc09d9f0 con 0x7f6fdc09ec20 2024-01-24T23:28:11.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fdc09dc60 con 0x7f6fdc13a1d0 2024-01-24T23:28:11.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:11.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 10 monclient(hunting): authenticate will time out at 1378.952271s 2024-01-24T23:28:11.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe1c0a640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f6fdc133b50 legacy=0x7f6fdc133f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:28:11.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe1409640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f6fdc09ec20 0x7f6fdc138ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:34344/0 (socket says 172.21.15.132:34344) 2024-01-24T23:28:11.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe1409640 1 -- 172.21.15.132:0/4259795846 learned_addr learned my addr 172.21.15.132:0/4259795846 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:11.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 1 -- 172.21.15.132:0/4259795846 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3888734510 0 0) 0x7f6fdc09d9f0 con 0x7f6fdc09ec20 2024-01-24T23:28:11.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 10 monclient(hunting): _negotiate 2024-01-24T23:28:11.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:11.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:11.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 10 monclient(hunting): my global_id is 95062 2024-01-24T23:28:11.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 1 -- 172.21.15.132:0/4259795846 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fc8003530 con 0x7f6fdc09ec20 2024-01-24T23:28:11.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 1 -- 172.21.15.132:0/4259795846 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 850643880 0 0) 0x7f6fdc09dc60 con 0x7f6fdc13a1d0 2024-01-24T23:28:11.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 10 monclient(hunting): _negotiate 2024-01-24T23:28:11.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:11.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:11.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 10 monclient(hunting): my global_id is 7116 2024-01-24T23:28:11.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 1 -- 172.21.15.132:0/4259795846 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fdc09d9f0 con 0x7f6fdc13a1d0 2024-01-24T23:28:11.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 1 -- 172.21.15.132:0/4259795846 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 977621862 0 0) 0x7f6fc8003530 con 0x7f6fdc09ec20 2024-01-24T23:28:11.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 1 -- 172.21.15.132:0/4259795846 >> v1:172.21.15.115:6792/0 conn(0x7f6fdc13a1d0 legacy=0x7f6fdc13c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:11.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 1 -- 172.21.15.132:0/4259795846 >> v1:172.21.15.115:6789/0 conn(0x7f6fdc133b50 legacy=0x7f6fdc133f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:28:11.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 10 monclient: _finish_hunting 0 2024-01-24T23:28:11.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 1 monclient: found mon.noname-c 2024-01-24T23:28:11.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:11.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:28:11.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 1 -- 172.21.15.132:0/4259795846 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fdc09df40 con 0x7f6fdc09ec20 2024-01-24T23:28:11.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 10 monclient: _finish_auth 0 2024-01-24T23:28:11.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 10 monclient: _check_auth_tickets 2024-01-24T23:28:11.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:11.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 1 -- 172.21.15.132:0/4259795846 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6fcc0017c0 con 0x7f6fdc09ec20 2024-01-24T23:28:11.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:11.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:28:11.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 10 monclient: dump: 2024-01-24T23:28:11.637 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:11.637 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:11.637 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:11.637 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:11.637 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:11.637 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:11.637 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:11.637 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:11.638 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:11.638 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:11.638 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:11.638 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:11.638 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:11.638 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:11.638 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:11.638 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:11.638 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 5 monclient: authenticate success, global_id 95062 2024-01-24T23:28:11.638 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:11.638 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 1 -- 172.21.15.132:0/4259795846 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6fcc001aa0 con 0x7f6fdc09ec20 2024-01-24T23:28:11.638 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:11.638 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:11.638 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 4 set_mon_vals no callback set 2024-01-24T23:28:11.639 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 1 -- 172.21.15.132:0/4259795846 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6fcc004bf0 con 0x7f6fdc09ec20 2024-01-24T23:28:11.639 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:11.639 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:28:11.639 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe0c08640 10 monclient: dump: 2024-01-24T23:28:11.639 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:11.639 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:11.639 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:11.639 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:11.639 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:11.639 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:11.639 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:11.639 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:11.639 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:11.639 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:11.639 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:11.640 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:11.640 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:11.640 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:11.640 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:11.640 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:11.640 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 10 monclient: shutdown 2024-01-24T23:28:11.640 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:11.640 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 1 -- 172.21.15.132:0/4259795846 >> v1:172.21.15.115:6791/0 conn(0x7f6fdc09ec20 legacy=0x7f6fdc138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:11.640 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 1 -- 172.21.15.132:0/4259795846 shutdown_connections 2024-01-24T23:28:11.640 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 1 -- 172.21.15.132:0/4259795846 >> 172.21.15.132:0/4259795846 conn(0x7f6fdc0a5a10 msgr2=0x7f6fdc0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:11.640 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 1 -- 172.21.15.132:0/4259795846 shutdown_connections 2024-01-24T23:28:11.640 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 1 -- 172.21.15.132:0/4259795846 wait complete. 2024-01-24T23:28:11.640 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 10 monclient: build_initial_monmap 2024-01-24T23:28:11.641 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 10 monclient: monmap: 2024-01-24T23:28:11.641 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:11.641 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:11.641 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:11.641 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:11.641 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:11.641 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:11.641 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:11.641 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:11.641 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:11.641 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:11.641 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:11.641 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:11.641 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:11.641 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:11.642 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:11.642 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:11.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 1 Processor -- start 2024-01-24T23:28:11.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 1 -- start start 2024-01-24T23:28:11.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 10 monclient: init 2024-01-24T23:28:11.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:11.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 10 monclient: _add_conns ranks=[8,3,6,5,0,1,4,7,2] 2024-01-24T23:28:11.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.626+0000 7f6fe3e6e640 10 monclient(hunting): picked mon.noname-e con 0x7f6fdc09ec20 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:11.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fe3e6e640 10 monclient(hunting): picked mon.noname-g con 0x7f6fdc133b50 addr v1:172.21.15.132:6790/0 2024-01-24T23:28:11.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fe3e6e640 10 monclient(hunting): picked mon.noname-d con 0x7f6fdc13a1d0 addr v1:172.21.15.115:6792/0 2024-01-24T23:28:11.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fe3e6e640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fdc0a07e0 con 0x7f6fdc133b50 2024-01-24T23:28:11.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fe3e6e640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fdc0a13b0 con 0x7f6fdc13a1d0 2024-01-24T23:28:11.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fe3e6e640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fdc133920 con 0x7f6fdc09ec20 2024-01-24T23:28:11.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fe3e6e640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:11.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fe3e6e640 10 monclient(hunting): authenticate will time out at 1378.956082s 2024-01-24T23:28:11.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fe1409640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f6fdc133b50 0x7f6fdc0a00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:44020/0 (socket says 172.21.15.132:44020) 2024-01-24T23:28:11.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fe1409640 1 -- 172.21.15.132:0/2937358874 learned_addr learned my addr 172.21.15.132:0/2937358874 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:11.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 1 -- 172.21.15.132:0/2937358874 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807335256 0 0) 0x7f6fdc0a07e0 con 0x7f6fdc133b50 2024-01-24T23:28:11.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 10 monclient(hunting): _negotiate 2024-01-24T23:28:11.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:11.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:11.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 10 monclient(hunting): my global_id is 6906 2024-01-24T23:28:11.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 1 -- 172.21.15.132:0/2937358874 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fc0003530 con 0x7f6fdc133b50 2024-01-24T23:28:11.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 1 -- 172.21.15.132:0/2937358874 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4170873996 0 0) 0x7f6fc0003530 con 0x7f6fdc133b50 2024-01-24T23:28:11.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 1 -- 172.21.15.132:0/2937358874 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6fdc0a07e0 con 0x7f6fdc133b50 2024-01-24T23:28:11.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 1 -- 172.21.15.132:0/2937358874 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6fcc004b30 con 0x7f6fdc133b50 2024-01-24T23:28:11.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:11.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:28:11.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 10 monclient(hunting): dump: 2024-01-24T23:28:11.643 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:11.644 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:11.644 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:11.644 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:11.644 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:11.644 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:11.644 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:11.644 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:11.644 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:11.644 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:11.644 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:11.644 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:11.644 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:11.644 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:11.644 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:11.645 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:11.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:11.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 1 -- 172.21.15.132:0/2937358874 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773320187 0 0) 0x7f6fdc0a13b0 con 0x7f6fdc13a1d0 2024-01-24T23:28:11.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 10 monclient(hunting): _negotiate 2024-01-24T23:28:11.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:11.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:11.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 10 monclient(hunting): my global_id is 7125 2024-01-24T23:28:11.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 1 -- 172.21.15.132:0/2937358874 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fc0003530 con 0x7f6fdc13a1d0 2024-01-24T23:28:11.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 1 -- 172.21.15.132:0/2937358874 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 308008209 0 0) 0x7f6fdc133920 con 0x7f6fdc09ec20 2024-01-24T23:28:11.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 10 monclient(hunting): _negotiate 2024-01-24T23:28:11.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:11.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:11.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 10 monclient(hunting): my global_id is 86102 2024-01-24T23:28:11.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 1 -- 172.21.15.132:0/2937358874 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fdc0a13b0 con 0x7f6fdc09ec20 2024-01-24T23:28:11.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 1 -- 172.21.15.132:0/2937358874 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1072759232 0 0) 0x7f6fdc0a07e0 con 0x7f6fdc133b50 2024-01-24T23:28:11.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 1 -- 172.21.15.132:0/2937358874 >> v1:172.21.15.115:6793/0 conn(0x7f6fdc09ec20 legacy=0x7f6fdc09f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:11.645 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 1 -- 172.21.15.132:0/2937358874 >> v1:172.21.15.115:6792/0 conn(0x7f6fdc13a1d0 legacy=0x7f6fdc1d37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:11.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 10 monclient: _finish_hunting 0 2024-01-24T23:28:11.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 1 monclient: found mon.g 2024-01-24T23:28:11.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:11.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:11.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 1 -- 172.21.15.132:0/2937358874 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fdc070f40 con 0x7f6fdc133b50 2024-01-24T23:28:11.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 10 monclient: _finish_auth 0 2024-01-24T23:28:11.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 10 monclient: _check_auth_tickets 2024-01-24T23:28:11.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:11.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fe3e6e640 5 monclient: authenticate success, global_id 6906 2024-01-24T23:28:11.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fe3e6e640 10 monclient: _renew_subs 2024-01-24T23:28:11.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fe3e6e640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:11.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fe3e6e640 1 -- 172.21.15.132:0/2937358874 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fdc06f7a0 con 0x7f6fdc133b50 2024-01-24T23:28:11.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fe3e6e640 10 monclient: _renew_subs 2024-01-24T23:28:11.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fe3e6e640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:11.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fe3e6e640 1 -- 172.21.15.132:0/2937358874 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fdc0705e0 con 0x7f6fdc133b50 2024-01-24T23:28:11.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 1 -- 172.21.15.132:0/2937358874 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6fcc004ec0 con 0x7f6fdc133b50 2024-01-24T23:28:11.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:11.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd3fff640 4 set_mon_vals no callback set 2024-01-24T23:28:11.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 1 -- 172.21.15.132:0/2937358874 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6fcc005370 con 0x7f6fdc133b50 2024-01-24T23:28:11.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:11.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:28:11.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 10 monclient: dump: 2024-01-24T23:28:11.647 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:11.647 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:11.648 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:11.648 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:11.648 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:11.648 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:11.648 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:11.648 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:11.648 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:11.648 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:11.648 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:11.648 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:11.648 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:11.648 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:11.648 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:11.648 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:11.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fe3e6e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:11.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fe3e6e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:11.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fe3e6e640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:11.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fe3e6e640 1 -- 172.21.15.132:0/2937358874 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc4005090 con 0x7f6fdc133b50 2024-01-24T23:28:11.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 1 -- 172.21.15.132:0/2937358874 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f6fcc00fdf0 con 0x7f6fdc133b50 2024-01-24T23:28:11.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fd27fc640 1 -- 172.21.15.132:0/2937358874 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(241..241 src has 1..241) v4 ==== 8228+0+0 (unknown 2885508406 0 0) 0x7f6fcc044c30 con 0x7f6fdc133b50 2024-01-24T23:28:11.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.630+0000 7f6fe1c0a640 10 monclient: get_auth_request con 0x7f6fc0037150 auth_method 0 2024-01-24T23:28:11.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.634+0000 7f6fd27fc640 1 -- 172.21.15.132:0/2937358874 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f6fcc014bc0 con 0x7f6fdc133b50 2024-01-24T23:28:11.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.634+0000 7f6fd27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:11.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.634+0000 7f6fd27fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:11.795 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.794+0000 7f6fe3e6e640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h30"}] 2024-01-24T23:28:11.795 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.794+0000 7f6fe3e6e640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h30"}] 2024-01-24T23:28:11.795 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.794+0000 7f6fe3e6e640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:11.795 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:11.794+0000 7f6fe3e6e640 1 -- 172.21.15.132:0/2937358874 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rm", "name": "h30"} v 0) v1 -- 0x7f6fc4005340 con 0x7f6fdc133b50 2024-01-24T23:28:12.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.410+0000 7f6fd27fc640 1 -- 172.21.15.132:0/2937358874 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h30"}]=0 removed item id -106 name 'h30' from crush map v242) v1 ==== 121+0+0 (unknown 1017365145 0 0) 0x7f6fcc013260 con 0x7f6fdc133b50 2024-01-24T23:28:12.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.410+0000 7f6fd27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h30"}] 2024-01-24T23:28:12.411 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.410+0000 7f6fd27fc640 10 monclient: _finish_command 2 = system:0 removed item id -106 name 'h30' from crush map 2024-01-24T23:28:12.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.510+0000 7f6fe3e6e640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h30"}] 2024-01-24T23:28:12.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.510+0000 7f6fe3e6e640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h30"}] 2024-01-24T23:28:12.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.510+0000 7f6fe3e6e640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:12.511 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.510+0000 7f6fe3e6e640 1 -- 172.21.15.132:0/2937358874 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd crush rm", "name": "h30"} v 0) v1 -- 0x7f6fc4002220 con 0x7f6fdc133b50 2024-01-24T23:28:12.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.510+0000 7f6fd27fc640 1 -- 172.21.15.132:0/2937358874 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h30"}]=0 device 'h30' does not appear in the crush map v242) v1 ==== 120+0+0 (unknown 43767254 0 0) 0x7f6fcc0149e0 con 0x7f6fdc133b50 2024-01-24T23:28:12.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.510+0000 7f6fd27fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h30"}] 2024-01-24T23:28:12.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.510+0000 7f6fd27fc640 10 monclient: _finish_command 3 = system:0 device 'h30' does not appear in the crush map 2024-01-24T23:28:12.512 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h30' does not appear in the crush map 2024-01-24T23:28:12.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.514+0000 7f6fe3e6e640 1 -- 172.21.15.132:0/2937358874 >> v1:172.21.15.132:6812/1835401922 conn(0x7f6fc0037150 legacy=0x7f6fc0039610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:12.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.514+0000 7f6fe3e6e640 10 monclient: shutdown 2024-01-24T23:28:12.514 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.514+0000 7f6fe3e6e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:12.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.514+0000 7f6fe3e6e640 1 -- 172.21.15.132:0/2937358874 >> v1:172.21.15.132:6790/0 conn(0x7f6fdc133b50 legacy=0x7f6fdc0a00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:12.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.514+0000 7f6fe3e6e640 1 -- 172.21.15.132:0/2937358874 shutdown_connections 2024-01-24T23:28:12.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.514+0000 7f6fe3e6e640 1 -- 172.21.15.132:0/2937358874 >> 172.21.15.132:0/2937358874 conn(0x7f6fdc0a5a10 msgr2=0x7f6fdc09c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:12.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.514+0000 7f6fe3e6e640 1 -- 172.21.15.132:0/2937358874 shutdown_connections 2024-01-24T23:28:12.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.514+0000 7f6fe3e6e640 1 -- 172.21.15.132:0/2937358874 wait complete. 2024-01-24T23:28:12.529 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:28:12.529 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h31 2024-01-24T23:28:12.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f99108e4640 10 monclient: get_monmap_and_config 2024-01-24T23:28:12.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f99108e4640 10 monclient: build_initial_monmap 2024-01-24T23:28:12.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f99108e4640 10 monclient: monmap: 2024-01-24T23:28:12.647 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:12.647 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:12.647 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:12.649840+0000 2024-01-24T23:28:12.647 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:12.649840+0000 2024-01-24T23:28:12.647 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:12.647 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:12.647 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:12.647 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:12.647 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:12.647 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:12.647 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:12.648 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:12.648 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:12.648 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:12.648 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:12.648 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:12.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f99108e4640 1 Processor -- start 2024-01-24T23:28:12.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f99108e4640 1 -- start start 2024-01-24T23:28:12.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f99108e4640 10 monclient: init 2024-01-24T23:28:12.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f99108e4640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:12.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f99108e4640 10 monclient: _add_conns ranks=[6,8,4,7,2,3,0,5,1] 2024-01-24T23:28:12.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f99108e4640 10 monclient(hunting): picked mon.noname-d con 0x7f990c133b50 addr v1:172.21.15.115:6792/0 2024-01-24T23:28:12.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f99108e4640 10 monclient(hunting): picked mon.noname-e con 0x7f990c09ec20 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:12.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f99108e4640 10 monclient(hunting): picked mon.noname-c con 0x7f990c13a1d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:28:12.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f99108e4640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990c064700 con 0x7f990c13a1d0 2024-01-24T23:28:12.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f99108e4640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990c09d9f0 con 0x7f990c133b50 2024-01-24T23:28:12.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f99108e4640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990c09dc60 con 0x7f990c09ec20 2024-01-24T23:28:12.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f99108e4640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:12.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f99108e4640 10 monclient(hunting): authenticate will time out at 1379.974481s 2024-01-24T23:28:12.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990ad9d640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f990c13a1d0 0x7f990c13c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:34350/0 (socket says 172.21.15.132:34350) 2024-01-24T23:28:12.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990ad9d640 1 -- 172.21.15.132:0/1132497628 learned_addr learned my addr 172.21.15.132:0/1132497628 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:12.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990a59c640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f990c133b50 0x7f990c133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:57900/0 (socket says 172.21.15.132:57900) 2024-01-24T23:28:12.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f9909d9b640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f990c09ec20 0x7f990c138ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:56740/0 (socket says 172.21.15.132:56740) 2024-01-24T23:28:12.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 1 -- 172.21.15.132:0/1132497628 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1779353056 0 0) 0x7f990c09dc60 con 0x7f990c09ec20 2024-01-24T23:28:12.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 10 monclient(hunting): _negotiate 2024-01-24T23:28:12.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:12.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:12.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 10 monclient(hunting): my global_id is 86111 2024-01-24T23:28:12.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 1 -- 172.21.15.132:0/1132497628 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98f4003530 con 0x7f990c09ec20 2024-01-24T23:28:12.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 1 -- 172.21.15.132:0/1132497628 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2180936975 0 0) 0x7f990c09d9f0 con 0x7f990c133b50 2024-01-24T23:28:12.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 10 monclient(hunting): _negotiate 2024-01-24T23:28:12.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:12.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:12.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 10 monclient(hunting): my global_id is 7134 2024-01-24T23:28:12.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 1 -- 172.21.15.132:0/1132497628 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f990c09dc60 con 0x7f990c133b50 2024-01-24T23:28:12.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 1 -- 172.21.15.132:0/1132497628 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 955390357 0 0) 0x7f990c064700 con 0x7f990c13a1d0 2024-01-24T23:28:12.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 10 monclient(hunting): _negotiate 2024-01-24T23:28:12.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:12.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:12.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 10 monclient(hunting): my global_id is 95071 2024-01-24T23:28:12.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 1 -- 172.21.15.132:0/1132497628 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f990c09d9f0 con 0x7f990c13a1d0 2024-01-24T23:28:12.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 1 -- 172.21.15.132:0/1132497628 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3805889335 0 0) 0x7f98f4003530 con 0x7f990c09ec20 2024-01-24T23:28:12.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 1 -- 172.21.15.132:0/1132497628 >> v1:172.21.15.115:6792/0 conn(0x7f990c133b50 legacy=0x7f990c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:12.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 1 -- 172.21.15.132:0/1132497628 >> v1:172.21.15.115:6791/0 conn(0x7f990c13a1d0 legacy=0x7f990c13c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:12.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 10 monclient: _finish_hunting 0 2024-01-24T23:28:12.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 1 monclient: found mon.noname-e 2024-01-24T23:28:12.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:12.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:28:12.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 1 -- 172.21.15.132:0/1132497628 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990c09df40 con 0x7f990c09ec20 2024-01-24T23:28:12.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 10 monclient: _finish_auth 0 2024-01-24T23:28:12.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 10 monclient: _check_auth_tickets 2024-01-24T23:28:12.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:12.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 5 monclient: authenticate success, global_id 86111 2024-01-24T23:28:12.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:12.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.646+0000 7f990959a640 1 -- 172.21.15.132:0/1132497628 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f99000017c0 con 0x7f990c09ec20 2024-01-24T23:28:12.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f990959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:12.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f990959a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:28:12.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f990959a640 10 monclient: dump: 2024-01-24T23:28:12.654 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:12.654 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:12.654 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:12.654 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:12.654 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:12.654 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:12.654 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:12.654 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:12.654 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:12.654 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:12.654 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:12.654 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:12.655 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:12.655 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:12.655 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:12.655 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:12.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:12.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f990959a640 1 -- 172.21.15.132:0/1132497628 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9900001aa0 con 0x7f990c09ec20 2024-01-24T23:28:12.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f990959a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:12.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:12.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 4 set_mon_vals no callback set 2024-01-24T23:28:12.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f990959a640 1 -- 172.21.15.132:0/1132497628 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9900004b30 con 0x7f990c09ec20 2024-01-24T23:28:12.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f990959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:12.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f990959a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:28:12.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f990959a640 10 monclient: dump: 2024-01-24T23:28:12.655 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:12.656 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:12.656 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:12.656 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:12.656 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:12.656 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:12.656 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:12.656 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:12.656 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:12.656 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:12.656 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:12.656 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:12.656 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:12.656 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:12.656 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:12.656 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:12.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 10 monclient: shutdown 2024-01-24T23:28:12.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:12.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 1 -- 172.21.15.132:0/1132497628 >> v1:172.21.15.115:6793/0 conn(0x7f990c09ec20 legacy=0x7f990c138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:12.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 1 -- 172.21.15.132:0/1132497628 shutdown_connections 2024-01-24T23:28:12.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 1 -- 172.21.15.132:0/1132497628 >> 172.21.15.132:0/1132497628 conn(0x7f990c0a5a10 msgr2=0x7f990c0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:12.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 1 -- 172.21.15.132:0/1132497628 shutdown_connections 2024-01-24T23:28:12.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 1 -- 172.21.15.132:0/1132497628 wait complete. 2024-01-24T23:28:12.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 10 monclient: build_initial_monmap 2024-01-24T23:28:12.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 10 monclient: monmap: 2024-01-24T23:28:12.661 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:12.662 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:12.662 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:12.662 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:12.662 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:12.662 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:12.662 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:12.662 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:12.662 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:12.662 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:12.662 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:12.662 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:12.662 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:12.662 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:12.662 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:12.663 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:12.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 1 Processor -- start 2024-01-24T23:28:12.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 1 -- start start 2024-01-24T23:28:12.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 10 monclient: init 2024-01-24T23:28:12.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:12.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 10 monclient: _add_conns ranks=[2,5,0,4,1,6,3,7,8] 2024-01-24T23:28:12.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 10 monclient(hunting): picked mon.noname-b con 0x7f990c09ec20 addr v1:172.21.15.115:6790/0 2024-01-24T23:28:12.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 10 monclient(hunting): picked mon.noname-h con 0x7f990c133b50 addr v1:172.21.15.132:6791/0 2024-01-24T23:28:12.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 10 monclient(hunting): picked mon.noname-a con 0x7f990c13a1d0 addr v1:172.21.15.115:6789/0 2024-01-24T23:28:12.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990c0a13b0 con 0x7f990c13a1d0 2024-01-24T23:28:12.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990c137800 con 0x7f990c09ec20 2024-01-24T23:28:12.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990c139000 con 0x7f990c133b50 2024-01-24T23:28:12.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:12.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 10 monclient(hunting): authenticate will time out at 1379.978485s 2024-01-24T23:28:12.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f9909d9b640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f990c133b50 0x7f990c1cc580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:48754/0 (socket says 172.21.15.132:48754) 2024-01-24T23:28:12.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f9909d9b640 1 -- 172.21.15.132:0/1194900598 learned_addr learned my addr 172.21.15.132:0/1194900598 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:12.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f990a59c640 1 -- 172.21.15.132:0/1194900598 >> v1:172.21.15.115:6790/0 conn(0x7f990c09ec20 legacy=0x7f990c1c75d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:28:12.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f990ad9d640 1 -- 172.21.15.132:0/1194900598 >> v1:172.21.15.115:6789/0 conn(0x7f990c13a1d0 legacy=0x7f990c1cfcb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:28:12.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 1 -- 172.21.15.132:0/1194900598 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2148380462 0 0) 0x7f990c139000 con 0x7f990c133b50 2024-01-24T23:28:12.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 10 monclient(hunting): _negotiate 2024-01-24T23:28:12.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:12.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:12.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 10 monclient(hunting): my global_id is 6908 2024-01-24T23:28:12.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 1 -- 172.21.15.132:0/1194900598 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98ec003530 con 0x7f990c133b50 2024-01-24T23:28:12.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 1 -- 172.21.15.132:0/1194900598 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1539750427 0 0) 0x7f98ec003530 con 0x7f990c133b50 2024-01-24T23:28:12.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 1 -- 172.21.15.132:0/1194900598 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f990c139000 con 0x7f990c133b50 2024-01-24T23:28:12.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 1 -- 172.21.15.132:0/1194900598 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9900001780 con 0x7f990c133b50 2024-01-24T23:28:12.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:12.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:28:12.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 10 monclient(hunting): dump: 2024-01-24T23:28:12.665 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:12.665 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:12.665 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:12.665 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:12.665 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:12.665 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:12.665 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:12.665 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:12.665 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:12.665 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:12.665 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:12.665 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:12.665 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:12.665 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:12.666 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:12.666 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:12.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:12.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 1 -- 172.21.15.132:0/1194900598 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2106893529 0 0) 0x7f990c139000 con 0x7f990c133b50 2024-01-24T23:28:12.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 1 -- 172.21.15.132:0/1194900598 >> v1:172.21.15.115:6790/0 conn(0x7f990c09ec20 legacy=0x7f990c1c75d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:28:12.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 1 -- 172.21.15.132:0/1194900598 >> v1:172.21.15.115:6789/0 conn(0x7f990c13a1d0 legacy=0x7f990c1cfcb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:28:12.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 10 monclient: _finish_hunting 0 2024-01-24T23:28:12.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 1 monclient: found mon.h 2024-01-24T23:28:12.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:12.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:12.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 1 -- 172.21.15.132:0/1194900598 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990c138a40 con 0x7f990c133b50 2024-01-24T23:28:12.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 10 monclient: _finish_auth 0 2024-01-24T23:28:12.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 10 monclient: _check_auth_tickets 2024-01-24T23:28:12.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:12.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 5 monclient: authenticate success, global_id 6908 2024-01-24T23:28:12.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 10 monclient: _renew_subs 2024-01-24T23:28:12.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:12.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 1 -- 172.21.15.132:0/1194900598 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990c138d20 con 0x7f990c133b50 2024-01-24T23:28:12.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 10 monclient: _renew_subs 2024-01-24T23:28:12.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:12.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f99108e4640 1 -- 172.21.15.132:0/1194900598 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f990c137500 con 0x7f990c133b50 2024-01-24T23:28:12.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 1 -- 172.21.15.132:0/1194900598 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9900001d30 con 0x7f990c133b50 2024-01-24T23:28:12.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:12.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 1 -- 172.21.15.132:0/1194900598 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f99000053f0 con 0x7f990c133b50 2024-01-24T23:28:12.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:12.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98fbfff640 4 set_mon_vals no callback set 2024-01-24T23:28:12.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:28:12.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.650+0000 7f98faffd640 10 monclient: dump: 2024-01-24T23:28:12.667 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:12.667 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:12.668 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:12.668 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:12.668 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:12.668 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:12.668 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:12.668 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:12.668 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:12.668 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:12.668 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:12.668 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:12.668 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:12.668 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:12.668 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:12.668 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:12.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.654+0000 7f99108e4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:12.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.654+0000 7f99108e4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:12.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.654+0000 7f99108e4640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:12.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.654+0000 7f99108e4640 1 -- 172.21.15.132:0/1194900598 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f0005090 con 0x7f990c133b50 2024-01-24T23:28:12.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.654+0000 7f98faffd640 1 -- 172.21.15.132:0/1194900598 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f990000fe70 con 0x7f990c133b50 2024-01-24T23:28:12.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.654+0000 7f98faffd640 1 -- 172.21.15.132:0/1194900598 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(242..242 src has 1..242) v4 ==== 8015+0+0 (unknown 664511147 0 0) 0x7f9900044b50 con 0x7f990c133b50 2024-01-24T23:28:12.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.654+0000 7f990a59c640 10 monclient: get_auth_request con 0x7f98ec036f50 auth_method 0 2024-01-24T23:28:12.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.658+0000 7f98faffd640 1 -- 172.21.15.132:0/1194900598 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f9900014bc0 con 0x7f990c133b50 2024-01-24T23:28:12.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.658+0000 7f98faffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:12.669 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.658+0000 7f98faffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:12.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.802+0000 7f99108e4640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h31"}] 2024-01-24T23:28:12.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.802+0000 7f99108e4640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h31"}] 2024-01-24T23:28:12.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.802+0000 7f99108e4640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:12.804 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:12.802+0000 7f99108e4640 1 -- 172.21.15.132:0/1194900598 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h31"} v 0) v1 -- 0x7f98f0005340 con 0x7f990c133b50 2024-01-24T23:28:13.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.406+0000 7f98faffd640 1 -- 172.21.15.132:0/1194900598 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h31"}]=0 removed item id -109 name 'h31' from crush map v243) v1 ==== 121+0+0 (unknown 389779763 0 0) 0x7f990001d290 con 0x7f990c133b50 2024-01-24T23:28:13.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.406+0000 7f98faffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h31"}] 2024-01-24T23:28:13.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.406+0000 7f98faffd640 10 monclient: _finish_command 2 = system:0 removed item id -109 name 'h31' from crush map 2024-01-24T23:28:13.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.514+0000 7f99108e4640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h31"}] 2024-01-24T23:28:13.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.514+0000 7f99108e4640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h31"}] 2024-01-24T23:28:13.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.514+0000 7f99108e4640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:13.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.514+0000 7f99108e4640 1 -- 172.21.15.132:0/1194900598 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h31"} v 0) v1 -- 0x7f98f0002220 con 0x7f990c133b50 2024-01-24T23:28:13.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.514+0000 7f98faffd640 1 -- 172.21.15.132:0/1194900598 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h31"}]=0 device 'h31' does not appear in the crush map v243) v1 ==== 120+0+0 (unknown 3064200569 0 0) 0x7f9900013260 con 0x7f990c133b50 2024-01-24T23:28:13.515 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.514+0000 7f98faffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h31"}] 2024-01-24T23:28:13.516 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.514+0000 7f98faffd640 10 monclient: _finish_command 3 = system:0 device 'h31' does not appear in the crush map 2024-01-24T23:28:13.516 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h31' does not appear in the crush map 2024-01-24T23:28:13.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.514+0000 7f99108e4640 1 -- 172.21.15.132:0/1194900598 >> v1:172.21.15.132:6812/1835401922 conn(0x7f98ec036f50 legacy=0x7f98ec039410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:13.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.514+0000 7f99108e4640 10 monclient: shutdown 2024-01-24T23:28:13.517 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.514+0000 7f99108e4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:13.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.514+0000 7f99108e4640 1 -- 172.21.15.132:0/1194900598 >> v1:172.21.15.132:6791/0 conn(0x7f990c133b50 legacy=0x7f990c1cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:13.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.518+0000 7f99108e4640 1 -- 172.21.15.132:0/1194900598 shutdown_connections 2024-01-24T23:28:13.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.518+0000 7f99108e4640 1 -- 172.21.15.132:0/1194900598 >> 172.21.15.132:0/1194900598 conn(0x7f990c0a5a10 msgr2=0x7f990c12eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:13.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.518+0000 7f99108e4640 1 -- 172.21.15.132:0/1194900598 shutdown_connections 2024-01-24T23:28:13.518 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.518+0000 7f99108e4640 1 -- 172.21.15.132:0/1194900598 wait complete. 2024-01-24T23:28:13.532 INFO:tasks.workunit.client.0.smithi132.stderr:+ for f in `seq 1 32` 2024-01-24T23:28:13.532 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm h32 2024-01-24T23:28:13.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.642+0000 7f109d3e9640 10 monclient: get_monmap_and_config 2024-01-24T23:28:13.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.642+0000 7f109d3e9640 10 monclient: build_initial_monmap 2024-01-24T23:28:13.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.642+0000 7f109d3e9640 10 monclient: monmap: 2024-01-24T23:28:13.647 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:13.647 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:13.647 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:13.647179+0000 2024-01-24T23:28:13.647 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:13.647179+0000 2024-01-24T23:28:13.647 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:13.647 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:13.647 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:13.647 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:13.647 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:13.648 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:13.648 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:13.648 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:13.648 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:13.648 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:13.648 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:13.648 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:13.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.642+0000 7f109d3e9640 1 Processor -- start 2024-01-24T23:28:13.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.642+0000 7f109d3e9640 1 -- start start 2024-01-24T23:28:13.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.642+0000 7f109d3e9640 10 monclient: init 2024-01-24T23:28:13.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.642+0000 7f109d3e9640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:13.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.642+0000 7f109d3e9640 10 monclient: _add_conns ranks=[5,4,2,3,1,7,6,0,8] 2024-01-24T23:28:13.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.642+0000 7f109d3e9640 10 monclient(hunting): picked mon.noname-h con 0x7f1098135e10 addr v1:172.21.15.132:6791/0 2024-01-24T23:28:13.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.642+0000 7f109d3e9640 10 monclient(hunting): picked mon.noname-c con 0x7f10981361f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:28:13.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.642+0000 7f109d3e9640 10 monclient(hunting): picked mon.noname-b con 0x7f10981365f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:28:13.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.642+0000 7f109d3e9640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1098064700 con 0x7f10981365f0 2024-01-24T23:28:13.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.642+0000 7f109d3e9640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f109809c3e0 con 0x7f10981361f0 2024-01-24T23:28:13.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.642+0000 7f109d3e9640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10980a4f90 con 0x7f1098135e10 2024-01-24T23:28:13.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.642+0000 7f109d3e9640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:13.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.642+0000 7f109d3e9640 10 monclient(hunting): authenticate will time out at 1380.971610s 2024-01-24T23:28:13.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.642+0000 7f1096ffd640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f1098135e10 0x7f109809bcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:48764/0 (socket says 172.21.15.132:48764) 2024-01-24T23:28:13.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.642+0000 7f1096ffd640 1 -- 172.21.15.132:0/3425266422 learned_addr learned my addr 172.21.15.132:0/3425266422 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:13.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.642+0000 7f10977fe640 1 -- 172.21.15.132:0/3425266422 >> v1:172.21.15.115:6790/0 conn(0x7f10981365f0 legacy=0x7f109813f600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:28:13.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.642+0000 7f1095ffb640 1 -- 172.21.15.132:0/3425266422 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2716623221 0 0) 0x7f10980a4f90 con 0x7f1098135e10 2024-01-24T23:28:13.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:28:13.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:13.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:13.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 10 monclient(hunting): my global_id is 6917 2024-01-24T23:28:13.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 1 -- 172.21.15.132:0/3425266422 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1080003530 con 0x7f1098135e10 2024-01-24T23:28:13.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 1 -- 172.21.15.132:0/3425266422 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3575287440 0 0) 0x7f1080003530 con 0x7f1098135e10 2024-01-24T23:28:13.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 1 -- 172.21.15.132:0/3425266422 >> v1:172.21.15.115:6791/0 conn(0x7f10981361f0 legacy=0x7f109813bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:13.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 1 -- 172.21.15.132:0/3425266422 >> v1:172.21.15.115:6790/0 conn(0x7f10981365f0 legacy=0x7f109813f600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:28:13.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:28:13.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 1 monclient: found mon.noname-h 2024-01-24T23:28:13.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:13.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:28:13.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 1 -- 172.21.15.132:0/3425266422 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109809b010 con 0x7f1098135e10 2024-01-24T23:28:13.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 10 monclient: _finish_auth 0 2024-01-24T23:28:13.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:28:13.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:13.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 1 -- 172.21.15.132:0/3425266422 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f10880017c0 con 0x7f1098135e10 2024-01-24T23:28:13.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 5 monclient: authenticate success, global_id 6917 2024-01-24T23:28:13.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:13.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:13.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:28:13.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 10 monclient: dump: 2024-01-24T23:28:13.652 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:13.652 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:13.652 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:13.652 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:13.652 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:13.653 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:13.653 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:13.653 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:13.653 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:13.653 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:13.653 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:13.653 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:13.653 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:13.653 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:13.653 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:13.653 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:13.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:13.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 1 -- 172.21.15.132:0/3425266422 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1088001ae0 con 0x7f1098135e10 2024-01-24T23:28:13.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:13.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 1 -- 172.21.15.132:0/3425266422 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1088004bf0 con 0x7f1098135e10 2024-01-24T23:28:13.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:13.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 4 set_mon_vals no callback set 2024-01-24T23:28:13.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:13.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:28:13.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f1095ffb640 10 monclient: dump: 2024-01-24T23:28:13.654 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:13.654 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:13.654 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:13.654 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:13.654 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:13.654 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:13.654 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:13.654 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:13.654 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:13.655 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:13.655 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:13.655 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:13.655 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:13.655 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:13.655 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:13.655 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:13.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 10 monclient: shutdown 2024-01-24T23:28:13.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:13.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 1 -- 172.21.15.132:0/3425266422 >> v1:172.21.15.132:6791/0 conn(0x7f1098135e10 legacy=0x7f109809bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:13.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 1 -- 172.21.15.132:0/3425266422 shutdown_connections 2024-01-24T23:28:13.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 1 -- 172.21.15.132:0/3425266422 >> 172.21.15.132:0/3425266422 conn(0x7f10980a6790 msgr2=0x7f1098071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:13.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 1 -- 172.21.15.132:0/3425266422 shutdown_connections 2024-01-24T23:28:13.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 1 -- 172.21.15.132:0/3425266422 wait complete. 2024-01-24T23:28:13.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 10 monclient: build_initial_monmap 2024-01-24T23:28:13.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 10 monclient: monmap: 2024-01-24T23:28:13.656 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:13.656 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:13.656 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:13.656 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:13.656 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:13.656 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:13.656 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:13.656 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:13.656 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:13.656 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:13.656 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:13.656 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:13.657 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:13.657 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:13.657 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:13.657 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:13.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 1 Processor -- start 2024-01-24T23:28:13.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 1 -- start start 2024-01-24T23:28:13.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 10 monclient: init 2024-01-24T23:28:13.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:13.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 10 monclient: _add_conns ranks=[8,5,6,7,4,3,2,1,0] 2024-01-24T23:28:13.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 10 monclient(hunting): picked mon.noname-e con 0x7f1098135e10 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:13.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 10 monclient(hunting): picked mon.noname-h con 0x7f10981361f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:28:13.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 10 monclient(hunting): picked mon.noname-d con 0x7f10981365f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:28:13.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f109809fd10 con 0x7f10981361f0 2024-01-24T23:28:13.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f109809c6b0 con 0x7f10981365f0 2024-01-24T23:28:13.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1098121e00 con 0x7f1098135e10 2024-01-24T23:28:13.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:13.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 10 monclient(hunting): authenticate will time out at 1380.974439s 2024-01-24T23:28:13.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f10967fc640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f10981361f0 0x7f10981d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:48766/0 (socket says 172.21.15.132:48766) 2024-01-24T23:28:13.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f10967fc640 1 -- 172.21.15.132:0/2431644445 learned_addr learned my addr 172.21.15.132:0/2431644445 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:13.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 1 -- 172.21.15.132:0/2431644445 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2103468843 0 0) 0x7f109809fd10 con 0x7f10981361f0 2024-01-24T23:28:13.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:28:13.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:13.661 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:13.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 10 monclient(hunting): my global_id is 6926 2024-01-24T23:28:13.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 1 -- 172.21.15.132:0/2431644445 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1084003530 con 0x7f10981361f0 2024-01-24T23:28:13.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 1 -- 172.21.15.132:0/2431644445 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 951454744 0 0) 0x7f1084003530 con 0x7f10981361f0 2024-01-24T23:28:13.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 1 -- 172.21.15.132:0/2431644445 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f109809fd10 con 0x7f10981361f0 2024-01-24T23:28:13.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 1 -- 172.21.15.132:0/2431644445 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f108c001ae0 con 0x7f10981361f0 2024-01-24T23:28:13.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:13.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:28:13.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 10 monclient(hunting): dump: 2024-01-24T23:28:13.662 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:13.663 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:13.663 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:13.663 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:13.663 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:13.663 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:13.663 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:13.663 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:13.663 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:13.663 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:13.663 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:13.663 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:13.663 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:13.663 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:13.663 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:13.663 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:13.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:13.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 1 -- 172.21.15.132:0/2431644445 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3384987323 0 0) 0x7f109809c6b0 con 0x7f10981365f0 2024-01-24T23:28:13.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:28:13.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:13.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:13.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 10 monclient(hunting): my global_id is 7143 2024-01-24T23:28:13.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 1 -- 172.21.15.132:0/2431644445 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1084003530 con 0x7f10981365f0 2024-01-24T23:28:13.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 1 -- 172.21.15.132:0/2431644445 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1410052003 0 0) 0x7f109809fd10 con 0x7f10981361f0 2024-01-24T23:28:13.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 1 -- 172.21.15.132:0/2431644445 >> v1:172.21.15.115:6793/0 conn(0x7f1098135e10 legacy=0x7f1098134ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:13.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 1 -- 172.21.15.132:0/2431644445 >> v1:172.21.15.115:6792/0 conn(0x7f10981365f0 legacy=0x7f10981d3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:13.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:28:13.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 1 monclient: found mon.h 2024-01-24T23:28:13.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:13.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:13.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 1 -- 172.21.15.132:0/2431644445 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10981220e0 con 0x7f10981361f0 2024-01-24T23:28:13.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 10 monclient: _finish_auth 0 2024-01-24T23:28:13.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:28:13.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:13.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 5 monclient: authenticate success, global_id 6926 2024-01-24T23:28:13.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 10 monclient: _renew_subs 2024-01-24T23:28:13.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:13.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 1 -- 172.21.15.132:0/2431644445 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1098120f10 con 0x7f10981361f0 2024-01-24T23:28:13.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 10 monclient: _renew_subs 2024-01-24T23:28:13.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:13.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f109d3e9640 1 -- 172.21.15.132:0/2431644445 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f10981cb530 con 0x7f10981361f0 2024-01-24T23:28:13.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 1 -- 172.21.15.132:0/2431644445 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f108c003950 con 0x7f10981361f0 2024-01-24T23:28:13.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:13.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 1 -- 172.21.15.132:0/2431644445 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f108c003e00 con 0x7f10981361f0 2024-01-24T23:28:13.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:13.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:28:13.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f107f7fe640 10 monclient: dump: 2024-01-24T23:28:13.666 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:13.666 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:13.666 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:13.666 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:13.666 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:13.666 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:13.666 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:13.666 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:13.666 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:13.666 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:13.666 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:13.666 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:13.667 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:13.667 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:13.667 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:13.667 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:13.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.646+0000 7f10957fa640 4 set_mon_vals no callback set 2024-01-24T23:28:13.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.650+0000 7f109d3e9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:13.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.650+0000 7f109d3e9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:13.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.650+0000 7f109d3e9640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:13.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.650+0000 7f109d3e9640 1 -- 172.21.15.132:0/2431644445 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1078005090 con 0x7f10981361f0 2024-01-24T23:28:13.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.650+0000 7f107f7fe640 1 -- 172.21.15.132:0/2431644445 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f108c003f60 con 0x7f10981361f0 2024-01-24T23:28:13.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.650+0000 7f107f7fe640 1 -- 172.21.15.132:0/2431644445 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(243..243 src has 1..243) v4 ==== 7802+0+0 (unknown 3735165332 0 0) 0x7f108c0447e0 con 0x7f10981361f0 2024-01-24T23:28:13.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.650+0000 7f1096ffd640 10 monclient: get_auth_request con 0x7f1084037000 auth_method 0 2024-01-24T23:28:13.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.654+0000 7f107f7fe640 1 -- 172.21.15.132:0/2431644445 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f108c014640 con 0x7f10981361f0 2024-01-24T23:28:13.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.654+0000 7f107f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:13.668 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.654+0000 7f107f7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:13.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.814+0000 7f109d3e9640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h32"}] 2024-01-24T23:28:13.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.814+0000 7f109d3e9640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h32"}] 2024-01-24T23:28:13.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.814+0000 7f109d3e9640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:13.815 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:13.814+0000 7f109d3e9640 1 -- 172.21.15.132:0/2431644445 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h32"} v 0) v1 -- 0x7f1078005340 con 0x7f10981361f0 2024-01-24T23:28:14.430 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.430+0000 7f107f7fe640 1 -- 172.21.15.132:0/2431644445 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h32"}]=0 removed item id -112 name 'h32' from crush map v244) v1 ==== 121+0+0 (unknown 1742575481 0 0) 0x7f108c014640 con 0x7f10981361f0 2024-01-24T23:28:14.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.430+0000 7f107f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h32"}] 2024-01-24T23:28:14.431 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.430+0000 7f107f7fe640 10 monclient: _finish_command 2 = system:0 removed item id -112 name 'h32' from crush map 2024-01-24T23:28:14.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.534+0000 7f109d3e9640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h32"}] 2024-01-24T23:28:14.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.534+0000 7f109d3e9640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h32"}] 2024-01-24T23:28:14.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.534+0000 7f109d3e9640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:14.534 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.534+0000 7f109d3e9640 1 -- 172.21.15.132:0/2431644445 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush rm", "name": "h32"} v 0) v1 -- 0x7f1078002220 con 0x7f10981361f0 2024-01-24T23:28:14.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.534+0000 7f107f7fe640 1 -- 172.21.15.132:0/2431644445 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h32"}]=0 device 'h32' does not appear in the crush map v244) v1 ==== 120+0+0 (unknown 1542090801 0 0) 0x7f108c014820 con 0x7f10981361f0 2024-01-24T23:28:14.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.534+0000 7f107f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h32"}] 2024-01-24T23:28:14.535 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.534+0000 7f107f7fe640 10 monclient: _finish_command 3 = system:0 device 'h32' does not appear in the crush map 2024-01-24T23:28:14.535 INFO:tasks.workunit.client.0.smithi132.stderr:device 'h32' does not appear in the crush map 2024-01-24T23:28:14.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.534+0000 7f109d3e9640 1 -- 172.21.15.132:0/2431644445 >> v1:172.21.15.132:6812/1835401922 conn(0x7f1084037000 legacy=0x7f10840394c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:14.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.534+0000 7f109d3e9640 10 monclient: shutdown 2024-01-24T23:28:14.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.534+0000 7f109d3e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:14.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.534+0000 7f109d3e9640 1 -- 172.21.15.132:0/2431644445 >> v1:172.21.15.132:6791/0 conn(0x7f10981361f0 legacy=0x7f10981d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:14.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.534+0000 7f109d3e9640 1 -- 172.21.15.132:0/2431644445 shutdown_connections 2024-01-24T23:28:14.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.534+0000 7f109d3e9640 1 -- 172.21.15.132:0/2431644445 >> 172.21.15.132:0/2431644445 conn(0x7f10980a6790 msgr2=0x7f10980a7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:14.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.534+0000 7f109d3e9640 1 -- 172.21.15.132:0/2431644445 shutdown_connections 2024-01-24T23:28:14.537 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.534+0000 7f109d3e9640 1 -- 172.21.15.132:0/2431644445 wait complete. 2024-01-24T23:28:14.552 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush rm r1 2024-01-24T23:28:14.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.666+0000 7fb01042d640 10 monclient: get_monmap_and_config 2024-01-24T23:28:14.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.666+0000 7fb01042d640 10 monclient: build_initial_monmap 2024-01-24T23:28:14.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.666+0000 7fb01042d640 10 monclient: monmap: 2024-01-24T23:28:14.667 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:14.668 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:14.668 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:14.670474+0000 2024-01-24T23:28:14.668 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:14.670474+0000 2024-01-24T23:28:14.668 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:14.668 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:14.668 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:14.668 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:14.668 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:14.668 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:14.668 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:14.668 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:14.668 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:14.668 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:14.669 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:14.669 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:14.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.666+0000 7fb01042d640 1 Processor -- start 2024-01-24T23:28:14.676 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.666+0000 7fb01042d640 1 -- start start 2024-01-24T23:28:14.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.666+0000 7fb01042d640 10 monclient: init 2024-01-24T23:28:14.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.666+0000 7fb01042d640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:14.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.666+0000 7fb01042d640 10 monclient: _add_conns ranks=[3,8,6,7,0,5,4,2,1] 2024-01-24T23:28:14.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.666+0000 7fb01042d640 10 monclient(hunting): picked mon.noname-g con 0x7fb008135e10 addr v1:172.21.15.132:6790/0 2024-01-24T23:28:14.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.666+0000 7fb01042d640 10 monclient(hunting): picked mon.noname-e con 0x7fb0081361f0 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:14.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.666+0000 7fb01042d640 10 monclient(hunting): picked mon.noname-d con 0x7fb0081365f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:28:14.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.666+0000 7fb01042d640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb008064700 con 0x7fb008135e10 2024-01-24T23:28:14.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb00809c3e0 con 0x7fb0081365f0 2024-01-24T23:28:14.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0080a4f90 con 0x7fb0081361f0 2024-01-24T23:28:14.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:14.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 10 monclient(hunting): authenticate will time out at 1381.996047s 2024-01-24T23:28:14.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00e1c9640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fb008135e10 0x7fb00809bcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:49558/0 (socket says 172.21.15.132:49558) 2024-01-24T23:28:14.677 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00e1c9640 1 -- 172.21.15.132:0/492258488 learned_addr learned my addr 172.21.15.132:0/492258488 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:14.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 1 -- 172.21.15.132:0/492258488 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 334296285 0 0) 0x7fb008064700 con 0x7fb008135e10 2024-01-24T23:28:14.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 10 monclient(hunting): _negotiate 2024-01-24T23:28:14.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:14.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:14.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 10 monclient(hunting): my global_id is 6915 2024-01-24T23:28:14.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 1 -- 172.21.15.132:0/492258488 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faff4003530 con 0x7fb008135e10 2024-01-24T23:28:14.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 1 -- 172.21.15.132:0/492258488 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4091580372 0 0) 0x7faff4003530 con 0x7fb008135e10 2024-01-24T23:28:14.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 1 -- 172.21.15.132:0/492258488 >> v1:172.21.15.115:6793/0 conn(0x7fb0081361f0 legacy=0x7fb00813bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:14.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 1 -- 172.21.15.132:0/492258488 >> v1:172.21.15.115:6792/0 conn(0x7fb0081365f0 legacy=0x7fb00813f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:14.678 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 10 monclient: _finish_hunting 0 2024-01-24T23:28:14.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 1 monclient: found mon.noname-g 2024-01-24T23:28:14.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:14.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:28:14.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 1 -- 172.21.15.132:0/492258488 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00809b010 con 0x7fb008135e10 2024-01-24T23:28:14.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 10 monclient: _finish_auth 0 2024-01-24T23:28:14.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 10 monclient: _check_auth_tickets 2024-01-24T23:28:14.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:14.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 5 monclient: authenticate success, global_id 6915 2024-01-24T23:28:14.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 1 -- 172.21.15.132:0/492258488 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb0040017c0 con 0x7fb008135e10 2024-01-24T23:28:14.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:14.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:14.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:28:14.679 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 10 monclient: dump: 2024-01-24T23:28:14.679 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:14.679 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:14.679 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:14.680 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:14.680 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:14.680 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:14.680 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:14.680 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:14.680 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:14.681 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:14.681 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:14.681 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:14.681 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:14.681 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:14.681 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:14.681 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:14.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:14.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 1 -- 172.21.15.132:0/492258488 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb004001ae0 con 0x7fb008135e10 2024-01-24T23:28:14.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:14.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 1 -- 172.21.15.132:0/492258488 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fb004004bf0 con 0x7fb008135e10 2024-01-24T23:28:14.681 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:14.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 4 set_mon_vals no callback set 2024-01-24T23:28:14.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:14.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:28:14.682 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d1c7640 10 monclient: dump: 2024-01-24T23:28:14.682 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:14.682 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:14.682 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:14.682 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:14.682 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:14.682 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:14.683 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:14.683 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:14.683 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:14.683 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:14.683 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:14.683 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:14.683 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:14.683 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:14.683 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:14.683 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:14.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 10 monclient: shutdown 2024-01-24T23:28:14.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:14.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 1 -- 172.21.15.132:0/492258488 >> v1:172.21.15.132:6790/0 conn(0x7fb008135e10 legacy=0x7fb00809bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:14.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 1 -- 172.21.15.132:0/492258488 shutdown_connections 2024-01-24T23:28:14.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 1 -- 172.21.15.132:0/492258488 >> 172.21.15.132:0/492258488 conn(0x7fb0080a6790 msgr2=0x7fb008071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:14.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 1 -- 172.21.15.132:0/492258488 shutdown_connections 2024-01-24T23:28:14.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 1 -- 172.21.15.132:0/492258488 wait complete. 2024-01-24T23:28:14.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 10 monclient: build_initial_monmap 2024-01-24T23:28:14.684 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 10 monclient: monmap: 2024-01-24T23:28:14.684 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:14.684 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:14.685 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:14.685 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:14.685 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:14.685 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:14.685 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:14.685 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:14.685 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:14.685 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:14.686 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:14.686 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:14.686 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:14.686 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:14.686 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:14.686 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:14.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 1 Processor -- start 2024-01-24T23:28:14.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 1 -- start start 2024-01-24T23:28:14.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 10 monclient: init 2024-01-24T23:28:14.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:14.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 10 monclient: _add_conns ranks=[7,6,0,4,3,2,5,8,1] 2024-01-24T23:28:14.686 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 10 monclient(hunting): picked mon.noname-i con 0x7fb008135e10 addr v1:172.21.15.132:6792/0 2024-01-24T23:28:14.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 10 monclient(hunting): picked mon.noname-d con 0x7fb0081361f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:28:14.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 10 monclient(hunting): picked mon.noname-a con 0x7fb0081365f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:28:14.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb008120780 con 0x7fb0081365f0 2024-01-24T23:28:14.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb008121490 con 0x7fb0081361f0 2024-01-24T23:28:14.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb00809c6b0 con 0x7fb008135e10 2024-01-24T23:28:14.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00e1c9640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fb008135e10 legacy=0x7fb00811f960 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:28:14.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:14.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb01042d640 10 monclient(hunting): authenticate will time out at 1381.998986s 2024-01-24T23:28:14.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00e9ca640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fb0081365f0 legacy=0x7fb0081d3bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:28:14.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d9c8640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fb0081361f0 0x7fb008120070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:44948/0 (socket says 172.21.15.132:44948) 2024-01-24T23:28:14.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7fb00d9c8640 1 -- 172.21.15.132:0/1921671372 learned_addr learned my addr 172.21.15.132:0/1921671372 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:14.687 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7faffeffd640 1 -- 172.21.15.132:0/1921671372 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922027417 0 0) 0x7fb008121490 con 0x7fb0081361f0 2024-01-24T23:28:14.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7faffeffd640 10 monclient(hunting): _negotiate 2024-01-24T23:28:14.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7faffeffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:14.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7faffeffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:14.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7faffeffd640 10 monclient(hunting): my global_id is 7152 2024-01-24T23:28:14.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.670+0000 7faffeffd640 1 -- 172.21.15.132:0/1921671372 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafec003530 con 0x7fb0081361f0 2024-01-24T23:28:14.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 1 -- 172.21.15.132:0/1921671372 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3005397202 0 0) 0x7fafec003530 con 0x7fb0081361f0 2024-01-24T23:28:14.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 1 -- 172.21.15.132:0/1921671372 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb008121490 con 0x7fb0081361f0 2024-01-24T23:28:14.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 1 -- 172.21.15.132:0/1921671372 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faff8001ac0 con 0x7fb0081361f0 2024-01-24T23:28:14.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:14.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:28:14.688 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 10 monclient(hunting): dump: 2024-01-24T23:28:14.688 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:14.688 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:14.688 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:14.689 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:14.689 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:14.689 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:14.689 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:14.689 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:14.689 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:14.689 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:14.689 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:14.689 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:14.689 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:14.689 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:14.690 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:14.690 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:14.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:14.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 1 -- 172.21.15.132:0/1921671372 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3389150396 0 0) 0x7fb008121490 con 0x7fb0081361f0 2024-01-24T23:28:14.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 1 -- 172.21.15.132:0/1921671372 >> v1:172.21.15.132:6792/0 conn(0x7fb008135e10 legacy=0x7fb00811f960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:28:14.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 1 -- 172.21.15.132:0/1921671372 >> v1:172.21.15.115:6789/0 conn(0x7fb0081365f0 legacy=0x7fb0081d3bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:28:14.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 10 monclient: _finish_hunting 0 2024-01-24T23:28:14.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 1 monclient: found mon.d 2024-01-24T23:28:14.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:14.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:28:14.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 1 -- 172.21.15.132:0/1921671372 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00809ca00 con 0x7fb0081361f0 2024-01-24T23:28:14.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 10 monclient: _finish_auth 0 2024-01-24T23:28:14.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 10 monclient: _check_auth_tickets 2024-01-24T23:28:14.690 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:14.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7fb01042d640 5 monclient: authenticate success, global_id 7152 2024-01-24T23:28:14.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7fb01042d640 10 monclient: _renew_subs 2024-01-24T23:28:14.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7fb01042d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:28:14.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7fb01042d640 1 -- 172.21.15.132:0/1921671372 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb008122940 con 0x7fb0081361f0 2024-01-24T23:28:14.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7fb01042d640 10 monclient: _renew_subs 2024-01-24T23:28:14.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7fb01042d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:28:14.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7fb01042d640 1 -- 172.21.15.132:0/1921671372 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0081211e0 con 0x7fb0081361f0 2024-01-24T23:28:14.691 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 1 -- 172.21.15.132:0/1921671372 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faff80038d0 con 0x7fb0081361f0 2024-01-24T23:28:14.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:14.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7fb00c9c6640 4 set_mon_vals no callback set 2024-01-24T23:28:14.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 1 -- 172.21.15.132:0/1921671372 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faff8003e00 con 0x7fb0081361f0 2024-01-24T23:28:14.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:14.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:28:14.692 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 10 monclient: dump: 2024-01-24T23:28:14.692 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:14.692 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:14.692 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:14.692 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:14.692 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:14.693 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:14.693 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:14.693 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:14.693 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:14.693 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:14.693 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:14.693 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:14.693 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:14.693 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:14.693 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:14.693 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:14.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7fb01042d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:14.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7fb01042d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:14.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7fb01042d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:28:14.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7fb01042d640 1 -- 172.21.15.132:0/1921671372 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff0005090 con 0x7fb0081361f0 2024-01-24T23:28:14.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 1 -- 172.21.15.132:0/1921671372 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7faff8003fe0 con 0x7fb0081361f0 2024-01-24T23:28:14.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7faffeffd640 1 -- 172.21.15.132:0/1921671372 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(244..244 src has 1..244) v4 ==== 7589+0+0 (unknown 2611333594 0 0) 0x7faff8044700 con 0x7fb0081361f0 2024-01-24T23:28:14.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.674+0000 7fb00e1c9640 10 monclient: get_auth_request con 0x7fafec036ef0 auth_method 0 2024-01-24T23:28:14.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.686+0000 7faffeffd640 1 -- 172.21.15.132:0/1921671372 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7faff8014850 con 0x7fb0081361f0 2024-01-24T23:28:14.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.686+0000 7faffeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:14.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.686+0000 7faffeffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:14.850 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.850+0000 7fb01042d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "r1"}] 2024-01-24T23:28:14.850 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.850+0000 7fb01042d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "r1"}] 2024-01-24T23:28:14.851 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.850+0000 7fb01042d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:28:14.851 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:14.850+0000 7fb01042d640 1 -- 172.21.15.132:0/1921671372 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "r1"} v 0) v1 -- 0x7faff0005340 con 0x7fb0081361f0 2024-01-24T23:28:15.441 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.438+0000 7faffeffd640 1 -- 172.21.15.132:0/1921671372 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "r1"}]=0 removed item id -16 name 'r1' from crush map v245) v1 ==== 118+0+0 (unknown 2650000602 0 0) 0x7faff8012ef0 con 0x7fb0081361f0 2024-01-24T23:28:15.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.438+0000 7faffeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "r1"}] 2024-01-24T23:28:15.442 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.438+0000 7faffeffd640 10 monclient: _finish_command 2 = system:0 removed item id -16 name 'r1' from crush map 2024-01-24T23:28:15.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.546+0000 7fb01042d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "r1"}] 2024-01-24T23:28:15.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.546+0000 7fb01042d640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "r1"}] 2024-01-24T23:28:15.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.546+0000 7fb01042d640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:28:15.548 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.546+0000 7fb01042d640 1 -- 172.21.15.132:0/1921671372 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd crush rm", "name": "r1"} v 0) v1 -- 0x7faff00057e0 con 0x7fb0081361f0 2024-01-24T23:28:15.549 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.550+0000 7faffeffd640 1 -- 172.21.15.132:0/1921671372 <== mon.6 v1:172.21.15.115:6792/0 11 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "r1"}]=0 device 'r1' does not appear in the crush map v245) v1 ==== 118+0+0 (unknown 3733252568 0 0) 0x7faff8014670 con 0x7fb0081361f0 2024-01-24T23:28:15.550 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.550+0000 7faffeffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "r1"}] 2024-01-24T23:28:15.550 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.550+0000 7faffeffd640 10 monclient: _finish_command 3 = system:0 device 'r1' does not appear in the crush map 2024-01-24T23:28:15.550 INFO:tasks.workunit.client.0.smithi132.stderr:device 'r1' does not appear in the crush map 2024-01-24T23:28:15.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.550+0000 7fb01042d640 1 -- 172.21.15.132:0/1921671372 >> v1:172.21.15.132:6812/1835401922 conn(0x7fafec036ef0 legacy=0x7fafec0393b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:15.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.550+0000 7fb01042d640 10 monclient: shutdown 2024-01-24T23:28:15.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.550+0000 7fb01042d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:15.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.550+0000 7fb01042d640 1 -- 172.21.15.132:0/1921671372 >> v1:172.21.15.115:6792/0 conn(0x7fb0081361f0 legacy=0x7fb008120070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:15.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.550+0000 7fb01042d640 1 -- 172.21.15.132:0/1921671372 shutdown_connections 2024-01-24T23:28:15.552 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.550+0000 7fb01042d640 1 -- 172.21.15.132:0/1921671372 >> 172.21.15.132:0/1921671372 conn(0x7fb0080a6790 msgr2=0x7fb0080a79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:15.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.550+0000 7fb01042d640 1 -- 172.21.15.132:0/1921671372 shutdown_connections 2024-01-24T23:28:15.553 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.550+0000 7fb01042d640 1 -- 172.21.15.132:0/1921671372 wait complete. 2024-01-24T23:28:15.567 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd crush weight-set rm-compat 2024-01-24T23:28:15.602 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~131s 2024-01-24T23:28:15.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0cfd8e640 10 monclient: get_monmap_and_config 2024-01-24T23:28:15.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0cfd8e640 10 monclient: build_initial_monmap 2024-01-24T23:28:15.683 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0cfd8e640 10 monclient: monmap: 2024-01-24T23:28:15.684 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:15.684 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:15.684 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:15.686159+0000 2024-01-24T23:28:15.684 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:15.686159+0000 2024-01-24T23:28:15.684 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:15.684 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:15.684 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:15.685 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:15.685 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:15.685 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:15.685 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:15.685 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:15.685 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:15.685 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:15.685 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:15.685 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:15.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0cfd8e640 1 Processor -- start 2024-01-24T23:28:15.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0cfd8e640 1 -- start start 2024-01-24T23:28:15.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0cfd8e640 10 monclient: init 2024-01-24T23:28:15.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0cfd8e640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:15.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0cfd8e640 10 monclient: _add_conns ranks=[2,7,4,3,5,0,1,6,8] 2024-01-24T23:28:15.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0cfd8e640 10 monclient(hunting): picked mon.noname-b con 0x7fc0c8133b50 addr v1:172.21.15.115:6790/0 2024-01-24T23:28:15.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0cfd8e640 10 monclient(hunting): picked mon.noname-i con 0x7fc0c809ec20 addr v1:172.21.15.132:6792/0 2024-01-24T23:28:15.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0cfd8e640 10 monclient(hunting): picked mon.noname-c con 0x7fc0c813a1d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:28:15.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0cfd8e640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c8064700 con 0x7fc0c8133b50 2024-01-24T23:28:15.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0cfd8e640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c809d9f0 con 0x7fc0c813a1d0 2024-01-24T23:28:15.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0cfd8e640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c809dc60 con 0x7fc0c809ec20 2024-01-24T23:28:15.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0cfd8e640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:15.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0cfd8e640 10 monclient(hunting): authenticate will time out at 1383.010959s 2024-01-24T23:28:15.693 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0cd329640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fc0c809ec20 legacy=0x7fc0c8138ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:28:15.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0cdb2a640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7fc0c8133b50 legacy=0x7fc0c8133f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:28:15.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0ce32b640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7fc0c813a1d0 0x7fc0c813c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:46242/0 (socket says 172.21.15.132:46242) 2024-01-24T23:28:15.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0ce32b640 1 -- 172.21.15.132:0/1728439044 learned_addr learned my addr 172.21.15.132:0/1728439044 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:15.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0ccb28640 1 -- 172.21.15.132:0/1728439044 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2482319488 0 0) 0x7fc0c809d9f0 con 0x7fc0c813a1d0 2024-01-24T23:28:15.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0ccb28640 10 monclient(hunting): _negotiate 2024-01-24T23:28:15.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0ccb28640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:15.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0ccb28640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:15.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0ccb28640 10 monclient(hunting): my global_id is 95089 2024-01-24T23:28:15.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.682+0000 7fc0ccb28640 1 -- 172.21.15.132:0/1728439044 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0b4003530 con 0x7fc0c813a1d0 2024-01-24T23:28:15.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0ccb28640 1 -- 172.21.15.132:0/1728439044 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2177319952 0 0) 0x7fc0b4003530 con 0x7fc0c813a1d0 2024-01-24T23:28:15.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0ccb28640 1 -- 172.21.15.132:0/1728439044 >> v1:172.21.15.132:6792/0 conn(0x7fc0c809ec20 legacy=0x7fc0c8138ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:28:15.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0ccb28640 1 -- 172.21.15.132:0/1728439044 >> v1:172.21.15.115:6790/0 conn(0x7fc0c8133b50 legacy=0x7fc0c8133f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:28:15.694 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0ccb28640 10 monclient: _finish_hunting 0 2024-01-24T23:28:15.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0ccb28640 1 monclient: found mon.noname-c 2024-01-24T23:28:15.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0ccb28640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:15.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0ccb28640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:28:15.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0ccb28640 1 -- 172.21.15.132:0/1728439044 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c809df40 con 0x7fc0c813a1d0 2024-01-24T23:28:15.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0ccb28640 10 monclient: _finish_auth 0 2024-01-24T23:28:15.695 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0ccb28640 10 monclient: _check_auth_tickets 2024-01-24T23:28:15.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0ccb28640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:15.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 5 monclient: authenticate success, global_id 95089 2024-01-24T23:28:15.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0ccb28640 1 -- 172.21.15.132:0/1728439044 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc0c0001b40 con 0x7fc0c813a1d0 2024-01-24T23:28:15.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0ccb28640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:15.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0ccb28640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:28:15.696 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0ccb28640 10 monclient: dump: 2024-01-24T23:28:15.697 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:15.697 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:15.697 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:15.697 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:15.697 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:15.697 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:15.698 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:15.698 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:15.698 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:15.698 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:15.698 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:15.698 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:15.699 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:15.699 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:15.699 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:15.699 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:15.699 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:15.699 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0ccb28640 1 -- 172.21.15.132:0/1728439044 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc0c0001e60 con 0x7fc0c813a1d0 2024-01-24T23:28:15.700 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0ccb28640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:15.700 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0ccb28640 1 -- 172.21.15.132:0/1728439044 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc0c0004f70 con 0x7fc0c813a1d0 2024-01-24T23:28:15.700 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0ccb28640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:15.700 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0ccb28640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:28:15.700 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0ccb28640 10 monclient: dump: 2024-01-24T23:28:15.700 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:15.701 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:15.701 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:15.701 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:15.701 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:15.701 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:15.701 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:15.702 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:15.702 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:15.702 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:15.702 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:15.702 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:15.702 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:15.703 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:15.703 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:15.703 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:15.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:15.703 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 4 set_mon_vals no callback set 2024-01-24T23:28:15.704 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 10 monclient: shutdown 2024-01-24T23:28:15.704 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:15.704 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 1 -- 172.21.15.132:0/1728439044 >> v1:172.21.15.115:6791/0 conn(0x7fc0c813a1d0 legacy=0x7fc0c813c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:15.704 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 1 -- 172.21.15.132:0/1728439044 shutdown_connections 2024-01-24T23:28:15.704 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 1 -- 172.21.15.132:0/1728439044 >> 172.21.15.132:0/1728439044 conn(0x7fc0c80a5a10 msgr2=0x7fc0c80a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:15.704 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 1 -- 172.21.15.132:0/1728439044 shutdown_connections 2024-01-24T23:28:15.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 1 -- 172.21.15.132:0/1728439044 wait complete. 2024-01-24T23:28:15.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 10 monclient: build_initial_monmap 2024-01-24T23:28:15.705 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 10 monclient: monmap: 2024-01-24T23:28:15.705 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:15.705 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:15.705 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:15.706 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:15.706 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:15.706 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:15.706 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:15.706 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:15.706 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:15.707 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:15.707 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:15.707 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:15.707 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:15.707 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:15.707 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:15.707 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:15.708 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 1 Processor -- start 2024-01-24T23:28:15.708 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 1 -- start start 2024-01-24T23:28:15.708 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 10 monclient: init 2024-01-24T23:28:15.708 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:15.708 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 10 monclient: _add_conns ranks=[5,6,1,8,7,0,4,2,3] 2024-01-24T23:28:15.708 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 10 monclient(hunting): picked mon.noname-h con 0x7fc0c809ec20 addr v1:172.21.15.132:6791/0 2024-01-24T23:28:15.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 10 monclient(hunting): picked mon.noname-d con 0x7fc0c8133b50 addr v1:172.21.15.115:6792/0 2024-01-24T23:28:15.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 10 monclient(hunting): picked mon.noname-f con 0x7fc0c813a1d0 addr v1:172.21.15.132:6789/0 2024-01-24T23:28:15.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c8070860 con 0x7fc0c813a1d0 2024-01-24T23:28:15.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c8133920 con 0x7fc0c809ec20 2024-01-24T23:28:15.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c80a07e0 con 0x7fc0c8133b50 2024-01-24T23:28:15.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:15.709 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 10 monclient(hunting): authenticate will time out at 1383.014526s 2024-01-24T23:28:15.710 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0ce32b640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fc0c813a1d0 0x7fc0c81d35e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:58676/0 (socket says 172.21.15.132:58676) 2024-01-24T23:28:15.710 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0ce32b640 1 -- 172.21.15.132:0/3107103813 learned_addr learned my addr 172.21.15.132:0/3107103813 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:15.710 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 1 -- 172.21.15.132:0/3107103813 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844245320 0 0) 0x7fc0c8133920 con 0x7fc0c809ec20 2024-01-24T23:28:15.710 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:28:15.710 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:15.710 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:15.710 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient(hunting): my global_id is 6935 2024-01-24T23:28:15.710 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 1 -- 172.21.15.132:0/3107103813 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0ac003530 con 0x7fc0c809ec20 2024-01-24T23:28:15.710 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 1 -- 172.21.15.132:0/3107103813 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3386535142 0 0) 0x7fc0c8070860 con 0x7fc0c813a1d0 2024-01-24T23:28:15.710 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:28:15.710 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:15.710 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:15.710 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient(hunting): my global_id is 124453 2024-01-24T23:28:15.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 1 -- 172.21.15.132:0/3107103813 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0c8133920 con 0x7fc0c813a1d0 2024-01-24T23:28:15.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 1 -- 172.21.15.132:0/3107103813 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3197314643 0 0) 0x7fc0ac003530 con 0x7fc0c809ec20 2024-01-24T23:28:15.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 1 -- 172.21.15.132:0/3107103813 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc0c8070860 con 0x7fc0c809ec20 2024-01-24T23:28:15.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 1 -- 172.21.15.132:0/3107103813 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1270590690 0 0) 0x7fc0c8133920 con 0x7fc0c813a1d0 2024-01-24T23:28:15.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 1 -- 172.21.15.132:0/3107103813 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc0ac003530 con 0x7fc0c813a1d0 2024-01-24T23:28:15.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 1 -- 172.21.15.132:0/3107103813 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc0c40042f0 con 0x7fc0c809ec20 2024-01-24T23:28:15.711 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:15.712 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:28:15.712 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient(hunting): dump: 2024-01-24T23:28:15.712 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:15.712 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:15.712 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:15.712 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:15.712 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:15.712 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:15.712 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:15.712 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:15.712 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:15.712 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:15.713 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:15.713 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:15.713 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:15.713 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:15.713 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:15.713 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:15.713 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:15.713 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 1 -- 172.21.15.132:0/3107103813 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283126212 0 0) 0x7fc0c80a07e0 con 0x7fc0c8133b50 2024-01-24T23:28:15.713 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:28:15.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:15.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:15.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient(hunting): my global_id is 7161 2024-01-24T23:28:15.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 1 -- 172.21.15.132:0/3107103813 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0c8133920 con 0x7fc0c8133b50 2024-01-24T23:28:15.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 1 -- 172.21.15.132:0/3107103813 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc0c0001b00 con 0x7fc0c813a1d0 2024-01-24T23:28:15.714 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:15.715 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:28:15.715 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient(hunting): dump: 2024-01-24T23:28:15.715 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:15.715 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:15.715 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:15.715 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:15.716 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:15.716 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:15.716 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:15.716 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:15.716 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:15.716 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:15.716 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:15.717 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:15.717 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:15.717 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:15.717 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:15.717 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:15.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 1 -- 172.21.15.132:0/3107103813 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1865609911 0 0) 0x7fc0c8070860 con 0x7fc0c809ec20 2024-01-24T23:28:15.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 1 -- 172.21.15.132:0/3107103813 >> v1:172.21.15.115:6792/0 conn(0x7fc0c8133b50 legacy=0x7fc0c80a00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:15.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 1 -- 172.21.15.132:0/3107103813 >> v1:172.21.15.132:6789/0 conn(0x7fc0c813a1d0 legacy=0x7fc0c81d35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:15.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:28:15.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 1 monclient: found mon.h 2024-01-24T23:28:15.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:15.718 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:15.719 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 1 -- 172.21.15.132:0/3107103813 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c806fb00 con 0x7fc0c809ec20 2024-01-24T23:28:15.719 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient: _finish_auth 0 2024-01-24T23:28:15.719 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:28:15.719 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:15.719 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 5 monclient: authenticate success, global_id 6935 2024-01-24T23:28:15.719 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 10 monclient: _renew_subs 2024-01-24T23:28:15.719 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:15.719 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 1 -- 172.21.15.132:0/3107103813 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c81394a0 con 0x7fc0c809ec20 2024-01-24T23:28:15.719 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 10 monclient: _renew_subs 2024-01-24T23:28:15.719 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:15.719 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0cfd8e640 1 -- 172.21.15.132:0/3107103813 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c812d5c0 con 0x7fc0c809ec20 2024-01-24T23:28:15.719 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 1 -- 172.21.15.132:0/3107103813 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc0c4004550 con 0x7fc0c809ec20 2024-01-24T23:28:15.719 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:15.719 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 1 -- 172.21.15.132:0/3107103813 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc0c4004a80 con 0x7fc0c809ec20 2024-01-24T23:28:15.720 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:15.720 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:28:15.720 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0be7fc640 10 monclient: dump: 2024-01-24T23:28:15.720 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:15.720 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:15.720 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:15.720 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:15.720 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:15.720 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:15.720 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:15.721 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:15.721 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:15.721 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:15.721 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:15.721 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:15.721 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:15.722 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:15.722 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:15.722 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:15.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.686+0000 7fc0bffff640 4 set_mon_vals no callback set 2024-01-24T23:28:15.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.690+0000 7fc0cfd8e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:15.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.690+0000 7fc0cfd8e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:15.722 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.690+0000 7fc0cfd8e640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:15.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.690+0000 7fc0cfd8e640 1 -- 172.21.15.132:0/3107103813 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0b0005090 con 0x7fc0c809ec20 2024-01-24T23:28:15.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.690+0000 7fc0be7fc640 1 -- 172.21.15.132:0/3107103813 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fc0c4004be0 con 0x7fc0c809ec20 2024-01-24T23:28:15.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.690+0000 7fc0be7fc640 1 -- 172.21.15.132:0/3107103813 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(245..245 src has 1..245) v4 ==== 7379+0+0 (unknown 2852747781 0 0) 0x7fc0c40444c0 con 0x7fc0c809ec20 2024-01-24T23:28:15.723 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.690+0000 7fc0cd329640 10 monclient: get_auth_request con 0x7fc0ac037960 auth_method 0 2024-01-24T23:28:15.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.694+0000 7fc0be7fc640 1 -- 172.21.15.132:0/3107103813 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fc0c4014980 con 0x7fc0c809ec20 2024-01-24T23:28:15.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.694+0000 7fc0be7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:15.724 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.694+0000 7fc0be7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:15.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.858+0000 7fc0cfd8e640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-01-24T23:28:15.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.858+0000 7fc0cfd8e640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-01-24T23:28:15.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.858+0000 7fc0cfd8e640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:15.859 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:15.858+0000 7fc0cfd8e640 1 -- 172.21.15.132:0/3107103813 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) v1 -- 0x7fc0b0005340 con 0x7fc0c809ec20 2024-01-24T23:28:16.458 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:16.458+0000 7fc0be7fc640 1 -- 172.21.15.132:0/3107103813 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v246) v1 ==== 78+0+0 (unknown 2147043512 0 0) 0x7fc0c4012df0 con 0x7fc0c809ec20 2024-01-24T23:28:16.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:16.458+0000 7fc0be7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-01-24T23:28:16.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:16.458+0000 7fc0be7fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:28:16.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:16.566+0000 7fc0cfd8e640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-01-24T23:28:16.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:16.566+0000 7fc0cfd8e640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-01-24T23:28:16.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:16.566+0000 7fc0cfd8e640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:16.567 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:16.566+0000 7fc0cfd8e640 1 -- 172.21.15.132:0/3107103813 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) v1 -- 0x7fc0b0002220 con 0x7fc0c809ec20 2024-01-24T23:28:16.941 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:16.938+0000 7fc0be7fc640 1 -- 172.21.15.132:0/3107103813 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v247) v1 ==== 78+0+0 (unknown 1043858077 0 0) 0x7fc0c4014570 con 0x7fc0c809ec20 2024-01-24T23:28:16.941 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:16.938+0000 7fc0be7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-01-24T23:28:16.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:16.938+0000 7fc0be7fc640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:28:16.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:16.946+0000 7fc0cfd8e640 1 -- 172.21.15.132:0/3107103813 >> v1:172.21.15.132:6812/1835401922 conn(0x7fc0ac037960 legacy=0x7fc0ac039e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:16.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:16.946+0000 7fc0cfd8e640 10 monclient: shutdown 2024-01-24T23:28:16.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:16.946+0000 7fc0cfd8e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:16.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:16.946+0000 7fc0cfd8e640 1 -- 172.21.15.132:0/3107103813 >> v1:172.21.15.132:6791/0 conn(0x7fc0c809ec20 legacy=0x7fc0c809f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:16.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:16.946+0000 7fc0cfd8e640 1 -- 172.21.15.132:0/3107103813 shutdown_connections 2024-01-24T23:28:16.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:16.946+0000 7fc0cfd8e640 1 -- 172.21.15.132:0/3107103813 >> 172.21.15.132:0/3107103813 conn(0x7fc0c80a5a10 msgr2=0x7fc0c8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:16.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:16.946+0000 7fc0cfd8e640 1 -- 172.21.15.132:0/3107103813 shutdown_connections 2024-01-24T23:28:16.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:16.946+0000 7fc0cfd8e640 1 -- 172.21.15.132:0/3107103813 wait complete. 2024-01-24T23:28:16.971 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo OK 2024-01-24T23:28:16.972 INFO:tasks.workunit.client.0.smithi132.stdout:OK 2024-01-24T23:28:16.973 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-01-24T23:28:16.973 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-01-24T23:28:16.983 INFO:tasks.workunit:Running workunits matching mon/osd.sh on client.0... 2024-01-24T23:28:16.985 INFO:tasks.workunit:Running workunit mon/osd.sh... 2024-01-24T23:28:16.985 DEBUG:teuthology.orchestra.run.smithi132:workunit test mon/osd.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=663873555d45def955d1cc09730bfece2847b821 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/osd.sh 2024-01-24T23:28:17.140 INFO:tasks.workunit.client.0.smithi132.stderr:+ set -e 2024-01-24T23:28:17.141 INFO:tasks.workunit.client.0.smithi132.stderr:+ uuidgen 2024-01-24T23:28:17.143 INFO:tasks.workunit.client.0.smithi132.stderr:+ ua=f78c88ca-c3ba-4894-b52e-23b6472a955a 2024-01-24T23:28:17.143 INFO:tasks.workunit.client.0.smithi132.stderr:+ uuidgen 2024-01-24T23:28:17.143 INFO:tasks.workunit.client.0.smithi132.stderr:+ ub=1fd09acd-2e41-4645-8e46-2b2acbb8d380 2024-01-24T23:28:17.143 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd create f78c88ca-c3ba-4894-b52e-23b6472a955a 2024-01-24T23:28:17.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc18d38f640 10 monclient: get_monmap_and_config 2024-01-24T23:28:17.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc18d38f640 10 monclient: build_initial_monmap 2024-01-24T23:28:17.263 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc18d38f640 10 monclient: monmap: 2024-01-24T23:28:17.263 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:17.263 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:17.263 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:17.266211+0000 2024-01-24T23:28:17.263 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:17.266211+0000 2024-01-24T23:28:17.263 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:17.263 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:17.263 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:17.264 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:17.264 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:17.264 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:17.264 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:17.264 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:17.264 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:17.264 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:17.264 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:17.264 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:17.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc18d38f640 1 Processor -- start 2024-01-24T23:28:17.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc18d38f640 1 -- start start 2024-01-24T23:28:17.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc18d38f640 10 monclient: init 2024-01-24T23:28:17.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc18d38f640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:17.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc18d38f640 10 monclient: _add_conns ranks=[4,3,0,1,2,5,8,7,6] 2024-01-24T23:28:17.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc18d38f640 10 monclient(hunting): picked mon.noname-c con 0x7fc18809ff20 addr v1:172.21.15.115:6791/0 2024-01-24T23:28:17.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc18d38f640 10 monclient(hunting): picked mon.noname-g con 0x7fc18809f9c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:28:17.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc18d38f640 10 monclient(hunting): picked mon.noname-a con 0x7fc18813b2c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:28:17.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc18d38f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc188064700 con 0x7fc18813b2c0 2024-01-24T23:28:17.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc18d38f640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc18809e760 con 0x7fc18809f9c0 2024-01-24T23:28:17.271 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc18d38f640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1880a1160 con 0x7fc18809ff20 2024-01-24T23:28:17.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc18d38f640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:17.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc18d38f640 10 monclient(hunting): authenticate will time out at 1384.590980s 2024-01-24T23:28:17.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc1867fc640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fc18809f9c0 0x7fc188139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:49568/0 (socket says 172.21.15.132:49568) 2024-01-24T23:28:17.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc1867fc640 1 -- 172.21.15.132:0/2218442133 learned_addr learned my addr 172.21.15.132:0/2218442133 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:17.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc1877fe640 1 -- 172.21.15.132:0/2218442133 >> v1:172.21.15.115:6789/0 conn(0x7fc18813b2c0 legacy=0x7fc18813d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:28:17.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 1 -- 172.21.15.132:0/2218442133 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3840511659 0 0) 0x7fc18809e760 con 0x7fc18809f9c0 2024-01-24T23:28:17.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:28:17.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:17.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:17.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 10 monclient(hunting): my global_id is 6924 2024-01-24T23:28:17.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 1 -- 172.21.15.132:0/2218442133 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc170003530 con 0x7fc18809f9c0 2024-01-24T23:28:17.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 1 -- 172.21.15.132:0/2218442133 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3177465821 0 0) 0x7fc170003530 con 0x7fc18809f9c0 2024-01-24T23:28:17.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 1 -- 172.21.15.132:0/2218442133 >> v1:172.21.15.115:6791/0 conn(0x7fc18809ff20 legacy=0x7fc1880a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:17.272 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 1 -- 172.21.15.132:0/2218442133 >> v1:172.21.15.115:6789/0 conn(0x7fc18813b2c0 legacy=0x7fc18813d6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:28:17.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:28:17.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 1 monclient: found mon.noname-g 2024-01-24T23:28:17.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:17.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:28:17.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 1 -- 172.21.15.132:0/2218442133 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1880a0a10 con 0x7fc18809f9c0 2024-01-24T23:28:17.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 10 monclient: _finish_auth 0 2024-01-24T23:28:17.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:28:17.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:17.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 1 -- 172.21.15.132:0/2218442133 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc17c0017c0 con 0x7fc18809f9c0 2024-01-24T23:28:17.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc18d38f640 5 monclient: authenticate success, global_id 6924 2024-01-24T23:28:17.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc18d38f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:17.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:17.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:28:17.273 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 10 monclient: dump: 2024-01-24T23:28:17.273 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:17.274 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:17.274 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:17.274 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:17.274 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:17.274 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:17.274 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:17.274 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:17.274 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:17.274 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:17.274 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:17.274 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:17.274 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:17.274 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:17.274 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:17.274 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:17.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc18d38f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:17.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 1 -- 172.21.15.132:0/2218442133 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc17c001ae0 con 0x7fc18809f9c0 2024-01-24T23:28:17.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:17.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc18d38f640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:17.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc18d38f640 4 set_mon_vals no callback set 2024-01-24T23:28:17.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 1 -- 172.21.15.132:0/2218442133 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc17c004bf0 con 0x7fc18809f9c0 2024-01-24T23:28:17.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:17.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:28:17.275 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.262+0000 7fc185ffb640 10 monclient: dump: 2024-01-24T23:28:17.275 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:17.275 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:17.275 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:17.275 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:17.275 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:17.275 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:17.276 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:17.276 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:17.276 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:17.276 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:17.276 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:17.276 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:17.276 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:17.276 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:17.276 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:17.276 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:17.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 10 monclient: shutdown 2024-01-24T23:28:17.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:17.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 1 -- 172.21.15.132:0/2218442133 >> v1:172.21.15.132:6790/0 conn(0x7fc18809f9c0 legacy=0x7fc188139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:17.276 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 1 -- 172.21.15.132:0/2218442133 shutdown_connections 2024-01-24T23:28:17.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 1 -- 172.21.15.132:0/2218442133 >> 172.21.15.132:0/2218442133 conn(0x7fc1880a5a10 msgr2=0x7fc1880a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:17.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 1 -- 172.21.15.132:0/2218442133 shutdown_connections 2024-01-24T23:28:17.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 1 -- 172.21.15.132:0/2218442133 wait complete. 2024-01-24T23:28:17.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 10 monclient: build_initial_monmap 2024-01-24T23:28:17.277 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 10 monclient: monmap: 2024-01-24T23:28:17.277 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:17.277 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:17.277 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:17.277 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:17.277 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:17.277 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:17.277 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:17.277 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:17.277 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:17.277 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:17.278 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:17.278 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:17.278 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:17.278 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:17.278 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:17.278 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:17.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 1 Processor -- start 2024-01-24T23:28:17.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 1 -- start start 2024-01-24T23:28:17.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 10 monclient: init 2024-01-24T23:28:17.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:17.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 10 monclient: _add_conns ranks=[6,8,7,2,0,1,5,4,3] 2024-01-24T23:28:17.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 10 monclient(hunting): picked mon.noname-d con 0x7fc18809f9c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:28:17.278 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 10 monclient(hunting): picked mon.noname-e con 0x7fc18809ff20 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:17.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 10 monclient(hunting): picked mon.noname-i con 0x7fc18813b2c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:28:17.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc18809d2e0 con 0x7fc18809f9c0 2024-01-24T23:28:17.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc18809da40 con 0x7fc18813b2c0 2024-01-24T23:28:17.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc188071960 con 0x7fc18809ff20 2024-01-24T23:28:17.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:17.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 10 monclient(hunting): authenticate will time out at 1384.593925s 2024-01-24T23:28:17.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc1877fe640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fc18813b2c0 legacy=0x7fc1881d36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:28:17.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc1867fc640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fc18809ff20 0x7fc18809cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:39746/0 (socket says 172.21.15.132:39746) 2024-01-24T23:28:17.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc1867fc640 1 -- 172.21.15.132:0/258661255 learned_addr learned my addr 172.21.15.132:0/258661255 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:17.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc186ffd640 1 --1- 172.21.15.132:0/258661255 >> v1:172.21.15.115:6792/0 conn(0x7fc18809f9c0 0x7fc18809c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.132:44970/0 (socket says 172.21.15.132:44970) 2024-01-24T23:28:17.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 1 -- 172.21.15.132:0/258661255 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2648816866 0 0) 0x7fc188071960 con 0x7fc18809ff20 2024-01-24T23:28:17.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:28:17.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:17.279 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:17.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 10 monclient(hunting): my global_id is 86138 2024-01-24T23:28:17.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 1 -- 172.21.15.132:0/258661255 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc174003530 con 0x7fc18809ff20 2024-01-24T23:28:17.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 1 -- 172.21.15.132:0/258661255 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3965262714 0 0) 0x7fc18809d2e0 con 0x7fc18809f9c0 2024-01-24T23:28:17.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:28:17.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:17.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:17.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 10 monclient(hunting): my global_id is 7170 2024-01-24T23:28:17.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 1 -- 172.21.15.132:0/258661255 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc188071960 con 0x7fc18809f9c0 2024-01-24T23:28:17.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 1 -- 172.21.15.132:0/258661255 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2245308451 0 0) 0x7fc188071960 con 0x7fc18809f9c0 2024-01-24T23:28:17.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 1 -- 172.21.15.132:0/258661255 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc18809d2e0 con 0x7fc18809f9c0 2024-01-24T23:28:17.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 1 -- 172.21.15.132:0/258661255 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 652923054 0 0) 0x7fc174003530 con 0x7fc18809ff20 2024-01-24T23:28:17.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 1 -- 172.21.15.132:0/258661255 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc188071960 con 0x7fc18809ff20 2024-01-24T23:28:17.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 1 -- 172.21.15.132:0/258661255 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc17c001780 con 0x7fc18809ff20 2024-01-24T23:28:17.280 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:17.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:28:17.281 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 10 monclient(hunting): dump: 2024-01-24T23:28:17.281 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:17.281 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:17.281 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:17.281 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:17.281 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:17.281 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:17.281 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:17.281 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:17.281 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:17.281 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:17.281 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:17.281 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:17.281 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:17.282 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:17.282 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:17.282 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:17.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:17.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 1 -- 172.21.15.132:0/258661255 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc178001b80 con 0x7fc18809f9c0 2024-01-24T23:28:17.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:17.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:28:17.282 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 10 monclient(hunting): dump: 2024-01-24T23:28:17.282 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:17.282 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:17.282 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:17.282 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:17.282 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:17.282 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:17.283 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:17.283 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:17.283 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:17.283 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:17.283 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:17.283 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:17.283 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:17.283 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:17.283 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:17.283 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:17.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 1 -- 172.21.15.132:0/258661255 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 895632059 0 0) 0x7fc18809d2e0 con 0x7fc18809f9c0 2024-01-24T23:28:17.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 1 -- 172.21.15.132:0/258661255 >> v1:172.21.15.115:6793/0 conn(0x7fc18809ff20 legacy=0x7fc18809cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:17.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 1 -- 172.21.15.132:0/258661255 >> v1:172.21.15.132:6792/0 conn(0x7fc18813b2c0 legacy=0x7fc1881d36c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:28:17.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:28:17.283 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 1 monclient: found mon.d 2024-01-24T23:28:17.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:17.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:28:17.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 1 -- 172.21.15.132:0/258661255 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc188071680 con 0x7fc18809f9c0 2024-01-24T23:28:17.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 10 monclient: _finish_auth 0 2024-01-24T23:28:17.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:28:17.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:17.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 5 monclient: authenticate success, global_id 7170 2024-01-24T23:28:17.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 10 monclient: _renew_subs 2024-01-24T23:28:17.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:28:17.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 1 -- 172.21.15.132:0/258661255 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc188120970 con 0x7fc18809f9c0 2024-01-24T23:28:17.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 10 monclient: _renew_subs 2024-01-24T23:28:17.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:28:17.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc18d38f640 1 -- 172.21.15.132:0/258661255 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc188134610 con 0x7fc18809f9c0 2024-01-24T23:28:17.284 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 1 -- 172.21.15.132:0/258661255 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc178001ce0 con 0x7fc18809f9c0 2024-01-24T23:28:17.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc16f7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:17.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.266+0000 7fc1857fa640 4 set_mon_vals no callback set 2024-01-24T23:28:17.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.270+0000 7fc16f7fe640 1 -- 172.21.15.132:0/258661255 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc178003c10 con 0x7fc18809f9c0 2024-01-24T23:28:17.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.270+0000 7fc16f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:17.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.270+0000 7fc16f7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:28:17.285 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.270+0000 7fc16f7fe640 10 monclient: dump: 2024-01-24T23:28:17.285 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:17.285 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:17.285 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:17.285 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:17.285 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:17.285 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:17.285 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:17.285 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:17.285 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:17.285 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:17.286 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:17.286 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:17.286 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:17.286 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:17.286 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:17.286 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:17.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.270+0000 7fc18d38f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:17.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.270+0000 7fc18d38f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:17.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.270+0000 7fc18d38f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:28:17.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.270+0000 7fc18d38f640 1 -- 172.21.15.132:0/258661255 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc168005090 con 0x7fc18809f9c0 2024-01-24T23:28:17.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.270+0000 7fc16f7fe640 1 -- 172.21.15.132:0/258661255 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fc178003df0 con 0x7fc18809f9c0 2024-01-24T23:28:17.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.270+0000 7fc16f7fe640 1 -- 172.21.15.132:0/258661255 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(247..247 src has 1..247) v4 ==== 7051+0+0 (unknown 232943445 0 0) 0x7fc1780443a0 con 0x7fc18809f9c0 2024-01-24T23:28:17.286 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.270+0000 7fc1867fc640 10 monclient: get_auth_request con 0x7fc174037670 auth_method 0 2024-01-24T23:28:17.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.282+0000 7fc16f7fe640 1 -- 172.21.15.132:0/258661255 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fc1780146b0 con 0x7fc18809f9c0 2024-01-24T23:28:17.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.282+0000 7fc16f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:17.287 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.282+0000 7fc16f7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:17.454 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.454+0000 7fc18d38f640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"}] 2024-01-24T23:28:17.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.454+0000 7fc18d38f640 10 monclient: _send_command 2 [{"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"}] 2024-01-24T23:28:17.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.454+0000 7fc18d38f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:28:17.455 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.454+0000 7fc18d38f640 1 -- 172.21.15.132:0/258661255 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"} v 0) v1 -- 0x7fc168005340 con 0x7fc18809f9c0 2024-01-24T23:28:17.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.962+0000 7fc16f7fe640 1 -- 172.21.15.132:0/258661255 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"}]=0 v248) v1 ==== 106+0+3 (unknown 2759744126 0 944643765) 0x7fc1780144d0 con 0x7fc18809f9c0 2024-01-24T23:28:17.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.962+0000 7fc16f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"}] 2024-01-24T23:28:17.965 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:17.962+0000 7fc16f7fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:28:18.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.074+0000 7fc18d38f640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"}] 2024-01-24T23:28:18.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.074+0000 7fc18d38f640 10 monclient: _send_command 3 [{"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"}] 2024-01-24T23:28:18.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.074+0000 7fc18d38f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:28:18.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.074+0000 7fc18d38f640 1 -- 172.21.15.132:0/258661255 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"} v 0) v1 -- 0x7fc1680010a0 con 0x7fc18809f9c0 2024-01-24T23:28:18.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.074+0000 7fc16f7fe640 1 -- 172.21.15.132:0/258661255 <== mon.6 v1:172.21.15.115:6792/0 11 ==== mon_command_ack([{"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"}]=0 v248) v1 ==== 106+0+3 (unknown 2759744126 0 944643765) 0x7fc1780144d0 con 0x7fc18809f9c0 2024-01-24T23:28:18.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.074+0000 7fc16f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"}] 2024-01-24T23:28:18.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.074+0000 7fc16f7fe640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:28:18.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.078+0000 7fc18d38f640 1 -- 172.21.15.132:0/258661255 >> v1:172.21.15.132:6812/1835401922 conn(0x7fc174037670 legacy=0x7fc174039b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:18.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.078+0000 7fc18d38f640 10 monclient: shutdown 2024-01-24T23:28:18.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.078+0000 7fc18d38f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:18.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.078+0000 7fc18d38f640 1 -- 172.21.15.132:0/258661255 >> v1:172.21.15.115:6792/0 conn(0x7fc18809f9c0 legacy=0x7fc18809c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:18.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.078+0000 7fc18d38f640 1 -- 172.21.15.132:0/258661255 shutdown_connections 2024-01-24T23:28:18.078 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.078+0000 7fc18d38f640 1 -- 172.21.15.132:0/258661255 >> 172.21.15.132:0/258661255 conn(0x7fc1880a5a10 msgr2=0x7fc188066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:18.079 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.078+0000 7fc18d38f640 1 -- 172.21.15.132:0/258661255 shutdown_connections 2024-01-24T23:28:18.079 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.078+0000 7fc18d38f640 1 -- 172.21.15.132:0/258661255 wait complete. 2024-01-24T23:28:18.092 INFO:tasks.workunit.client.0.smithi132.stderr:+ na=12 2024-01-24T23:28:18.092 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd create f78c88ca-c3ba-4894-b52e-23b6472a955a 2024-01-24T23:28:18.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.206+0000 7f68a1ea1640 10 monclient: get_monmap_and_config 2024-01-24T23:28:18.206 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.206+0000 7f68a1ea1640 10 monclient: build_initial_monmap 2024-01-24T23:28:18.207 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.206+0000 7f68a1ea1640 10 monclient: monmap: 2024-01-24T23:28:18.207 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:18.207 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:18.207 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:18.209796+0000 2024-01-24T23:28:18.207 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:18.209796+0000 2024-01-24T23:28:18.207 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:18.207 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:18.207 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:18.207 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:18.207 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:18.207 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:18.207 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:18.207 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:18.207 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:18.207 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:18.208 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:18.208 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:18.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.206+0000 7f68a1ea1640 1 Processor -- start 2024-01-24T23:28:18.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.206+0000 7f68a1ea1640 1 -- start start 2024-01-24T23:28:18.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.206+0000 7f68a1ea1640 10 monclient: init 2024-01-24T23:28:18.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.206+0000 7f68a1ea1640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:18.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.206+0000 7f68a1ea1640 10 monclient: _add_conns ranks=[0,6,8,5,3,2,4,7,1] 2024-01-24T23:28:18.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.206+0000 7f68a1ea1640 10 monclient(hunting): picked mon.noname-a con 0x7f689c134cb0 addr v1:172.21.15.115:6789/0 2024-01-24T23:28:18.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.206+0000 7f68a1ea1640 10 monclient(hunting): picked mon.noname-d con 0x7f689c137b70 addr v1:172.21.15.115:6792/0 2024-01-24T23:28:18.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.206+0000 7f68a1ea1640 10 monclient(hunting): picked mon.noname-e con 0x7f689c13b680 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:18.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.206+0000 7f68a1ea1640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689c064700 con 0x7f689c134cb0 2024-01-24T23:28:18.209 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.206+0000 7f68a1ea1640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689c06d770 con 0x7f689c137b70 2024-01-24T23:28:18.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.206+0000 7f68a1ea1640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689c1357a0 con 0x7f689c13b680 2024-01-24T23:28:18.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.206+0000 7f68a1ea1640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:18.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.206+0000 7f68a1ea1640 10 monclient(hunting): authenticate will time out at 1385.535283s 2024-01-24T23:28:18.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.206+0000 7f689b7fe640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f689c134cb0 legacy=0x7f689c135090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:28:18.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.206+0000 7f689bfff640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f689c13b680 0x7f689c13da70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:39758/0 (socket says 172.21.15.132:39758) 2024-01-24T23:28:18.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.206+0000 7f689bfff640 1 -- 172.21.15.132:0/1529736878 learned_addr learned my addr 172.21.15.132:0/1529736878 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:18.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 1 -- 172.21.15.132:0/1529736878 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130357902 0 0) 0x7f689c06d770 con 0x7f689c137b70 2024-01-24T23:28:18.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 10 monclient(hunting): _negotiate 2024-01-24T23:28:18.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:18.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:18.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 10 monclient(hunting): my global_id is 7179 2024-01-24T23:28:18.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 1 -- 172.21.15.132:0/1529736878 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6888003530 con 0x7f689c137b70 2024-01-24T23:28:18.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 1 -- 172.21.15.132:0/1529736878 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3378009419 0 0) 0x7f689c1357a0 con 0x7f689c13b680 2024-01-24T23:28:18.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 10 monclient(hunting): _negotiate 2024-01-24T23:28:18.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:18.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:18.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 10 monclient(hunting): my global_id is 86147 2024-01-24T23:28:18.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 1 -- 172.21.15.132:0/1529736878 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f689c06d770 con 0x7f689c13b680 2024-01-24T23:28:18.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 1 -- 172.21.15.132:0/1529736878 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3445748105 0 0) 0x7f6888003530 con 0x7f689c137b70 2024-01-24T23:28:18.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 1 -- 172.21.15.132:0/1529736878 >> v1:172.21.15.115:6793/0 conn(0x7f689c13b680 legacy=0x7f689c13da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:18.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 1 -- 172.21.15.132:0/1529736878 >> v1:172.21.15.115:6789/0 conn(0x7f689c134cb0 legacy=0x7f689c135090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:28:18.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 10 monclient: _finish_hunting 0 2024-01-24T23:28:18.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 1 monclient: found mon.noname-d 2024-01-24T23:28:18.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:18.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:28:18.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 1 -- 172.21.15.132:0/1529736878 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689c134860 con 0x7f689c137b70 2024-01-24T23:28:18.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 10 monclient: _finish_auth 0 2024-01-24T23:28:18.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 10 monclient: _check_auth_tickets 2024-01-24T23:28:18.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:18.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 1 -- 172.21.15.132:0/1529736878 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f68840016f0 con 0x7f689c137b70 2024-01-24T23:28:18.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:18.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:28:18.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 10 monclient: dump: 2024-01-24T23:28:18.217 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:18.217 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:18.217 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:18.217 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:18.217 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:18.217 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:18.218 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:18.218 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:18.218 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:18.218 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:18.218 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:18.218 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:18.218 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:18.218 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:18.218 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:18.218 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:18.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 5 monclient: authenticate success, global_id 7179 2024-01-24T23:28:18.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:18.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 1 -- 172.21.15.132:0/1529736878 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6884001910 con 0x7f689c137b70 2024-01-24T23:28:18.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:18.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 1 -- 172.21.15.132:0/1529736878 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6884003620 con 0x7f689c137b70 2024-01-24T23:28:18.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:18.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 4 set_mon_vals no callback set 2024-01-24T23:28:18.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:18.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:28:18.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689affd640 10 monclient: dump: 2024-01-24T23:28:18.219 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:18.219 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:18.219 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:18.219 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:18.219 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:18.219 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:18.219 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:18.219 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:18.219 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:18.220 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:18.220 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:18.220 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:18.220 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:18.220 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:18.220 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:18.220 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:18.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 10 monclient: shutdown 2024-01-24T23:28:18.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:18.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 1 -- 172.21.15.132:0/1529736878 >> v1:172.21.15.115:6792/0 conn(0x7f689c137b70 legacy=0x7f689c139f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:18.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 1 -- 172.21.15.132:0/1529736878 shutdown_connections 2024-01-24T23:28:18.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 1 -- 172.21.15.132:0/1529736878 >> 172.21.15.132:0/1529736878 conn(0x7f689c0a2340 msgr2=0x7f689c0a2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:18.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 1 -- 172.21.15.132:0/1529736878 shutdown_connections 2024-01-24T23:28:18.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 1 -- 172.21.15.132:0/1529736878 wait complete. 2024-01-24T23:28:18.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 10 monclient: build_initial_monmap 2024-01-24T23:28:18.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 10 monclient: monmap: 2024-01-24T23:28:18.221 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:18.221 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:18.221 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:18.221 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:18.221 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:18.221 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:18.221 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:18.221 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:18.221 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:18.221 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:18.221 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:18.221 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:18.221 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:18.222 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:18.222 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:18.222 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:18.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 1 Processor -- start 2024-01-24T23:28:18.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 1 -- start start 2024-01-24T23:28:18.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 10 monclient: init 2024-01-24T23:28:18.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:18.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 10 monclient: _add_conns ranks=[3,0,8,7,5,1,4,2,6] 2024-01-24T23:28:18.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 10 monclient(hunting): picked mon.noname-g con 0x7f689c134cb0 addr v1:172.21.15.132:6790/0 2024-01-24T23:28:18.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 10 monclient(hunting): picked mon.noname-a con 0x7f689c137b70 addr v1:172.21.15.115:6789/0 2024-01-24T23:28:18.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 10 monclient(hunting): picked mon.noname-e con 0x7f689c13b680 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:18.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689c13ac60 con 0x7f689c137b70 2024-01-24T23:28:18.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689c139010 con 0x7f689c134cb0 2024-01-24T23:28:18.222 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689c1c9ae0 con 0x7f689c13b680 2024-01-24T23:28:18.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:18.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f68a1ea1640 10 monclient(hunting): authenticate will time out at 1385.539085s 2024-01-24T23:28:18.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689b7fe640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f689c134cb0 0x7f689c1c8cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:49574/0 (socket says 172.21.15.132:49574) 2024-01-24T23:28:18.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f689b7fe640 1 -- 172.21.15.132:0/2187772620 learned_addr learned my addr 172.21.15.132:0/2187772620 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:18.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6893fff640 1 -- 172.21.15.132:0/2187772620 >> v1:172.21.15.115:6789/0 conn(0x7f689c137b70 legacy=0x7f689c1c93d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:28:18.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 1 -- 172.21.15.132:0/2187772620 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4185079386 0 0) 0x7f689c139010 con 0x7f689c134cb0 2024-01-24T23:28:18.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:28:18.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:18.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:18.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 10 monclient(hunting): my global_id is 6933 2024-01-24T23:28:18.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 1 -- 172.21.15.132:0/2187772620 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f687c003530 con 0x7f689c134cb0 2024-01-24T23:28:18.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 1 -- 172.21.15.132:0/2187772620 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3903899169 0 0) 0x7f687c003530 con 0x7f689c134cb0 2024-01-24T23:28:18.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 1 -- 172.21.15.132:0/2187772620 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f689c139010 con 0x7f689c134cb0 2024-01-24T23:28:18.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 1 -- 172.21.15.132:0/2187772620 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f688c004330 con 0x7f689c134cb0 2024-01-24T23:28:18.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:18.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:28:18.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 10 monclient(hunting): dump: 2024-01-24T23:28:18.224 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:18.224 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:18.224 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:18.224 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:18.224 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:18.224 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:18.224 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:18.224 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:18.224 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:18.224 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:18.224 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:18.224 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:18.225 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:18.225 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:18.225 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:18.225 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:18.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:18.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 1 -- 172.21.15.132:0/2187772620 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1876287049 0 0) 0x7f689c1c9ae0 con 0x7f689c13b680 2024-01-24T23:28:18.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:28:18.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:18.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:18.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 10 monclient(hunting): my global_id is 86156 2024-01-24T23:28:18.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 1 -- 172.21.15.132:0/2187772620 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f687c003530 con 0x7f689c13b680 2024-01-24T23:28:18.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 1 -- 172.21.15.132:0/2187772620 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 614527159 0 0) 0x7f689c139010 con 0x7f689c134cb0 2024-01-24T23:28:18.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 1 -- 172.21.15.132:0/2187772620 >> v1:172.21.15.115:6793/0 conn(0x7f689c13b680 legacy=0x7f689c1d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:18.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 1 -- 172.21.15.132:0/2187772620 >> v1:172.21.15.115:6789/0 conn(0x7f689c137b70 legacy=0x7f689c1c93d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:28:18.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:28:18.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 1 monclient: found mon.g 2024-01-24T23:28:18.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:18.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:18.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 1 -- 172.21.15.132:0/2187772620 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689c130a10 con 0x7f689c134cb0 2024-01-24T23:28:18.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 10 monclient: _finish_auth 0 2024-01-24T23:28:18.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:28:18.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.210+0000 7f6898ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:18.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.214+0000 7f68a1ea1640 5 monclient: authenticate success, global_id 6933 2024-01-24T23:28:18.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.214+0000 7f68a1ea1640 10 monclient: _renew_subs 2024-01-24T23:28:18.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.214+0000 7f68a1ea1640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:18.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.214+0000 7f68a1ea1640 1 -- 172.21.15.132:0/2187772620 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f689c130cf0 con 0x7f689c134cb0 2024-01-24T23:28:18.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.214+0000 7f68a1ea1640 10 monclient: _renew_subs 2024-01-24T23:28:18.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.214+0000 7f68a1ea1640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:18.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.214+0000 7f68a1ea1640 1 -- 172.21.15.132:0/2187772620 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f689c131a60 con 0x7f689c134cb0 2024-01-24T23:28:18.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.214+0000 7f6898ff9640 1 -- 172.21.15.132:0/2187772620 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f688c004650 con 0x7f689c134cb0 2024-01-24T23:28:18.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.214+0000 7f6898ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:18.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.214+0000 7f6898ff9640 1 -- 172.21.15.132:0/2187772620 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f688c004b80 con 0x7f689c134cb0 2024-01-24T23:28:18.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.214+0000 7f6898ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:18.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.214+0000 7f6899ffb640 4 set_mon_vals no callback set 2024-01-24T23:28:18.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.214+0000 7f6898ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:28:18.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.214+0000 7f6898ff9640 10 monclient: dump: 2024-01-24T23:28:18.227 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:18.227 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:18.227 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:18.227 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:18.227 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:18.227 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:18.227 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:18.227 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:18.228 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:18.228 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:18.228 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:18.228 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:18.228 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:18.228 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:18.228 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:18.228 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:18.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.214+0000 7f68a1ea1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:18.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.214+0000 7f68a1ea1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:18.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.214+0000 7f68a1ea1640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:18.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.214+0000 7f6898ff9640 1 -- 172.21.15.132:0/2187772620 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f688c004ce0 con 0x7f689c134cb0 2024-01-24T23:28:18.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.214+0000 7f68a1ea1640 1 -- 172.21.15.132:0/2187772620 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6880005090 con 0x7f689c134cb0 2024-01-24T23:28:18.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.214+0000 7f6898ff9640 1 -- 172.21.15.132:0/2187772620 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(248..248 src has 1..248) v4 ==== 7051+0+0 (unknown 1773304109 0 0) 0x7f688c044580 con 0x7f689c134cb0 2024-01-24T23:28:18.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.214+0000 7f6893fff640 10 monclient: get_auth_request con 0x7f687c036ee0 auth_method 0 2024-01-24T23:28:18.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.218+0000 7f6898ff9640 1 -- 172.21.15.132:0/2187772620 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f688c014b20 con 0x7f689c134cb0 2024-01-24T23:28:18.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.218+0000 7f6898ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:18.229 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.218+0000 7f6898ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:18.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.382+0000 7f68a1ea1640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"}] 2024-01-24T23:28:18.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.382+0000 7f68a1ea1640 10 monclient: _send_command 2 [{"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"}] 2024-01-24T23:28:18.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.382+0000 7f68a1ea1640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:18.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.382+0000 7f68a1ea1640 1 -- 172.21.15.132:0/2187772620 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"} v 0) v1 -- 0x7f68800058c0 con 0x7f689c134cb0 2024-01-24T23:28:18.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.386+0000 7f6898ff9640 1 -- 172.21.15.132:0/2187772620 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"}]=0 v248) v1 ==== 106+0+3 (unknown 2759744126 0 944643765) 0x7f688c012f90 con 0x7f689c134cb0 2024-01-24T23:28:18.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.386+0000 7f6898ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"}] 2024-01-24T23:28:18.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.386+0000 7f6898ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:28:18.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.482+0000 7f68a1ea1640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"}] 2024-01-24T23:28:18.484 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.482+0000 7f68a1ea1640 10 monclient: _send_command 3 [{"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"}] 2024-01-24T23:28:18.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.482+0000 7f68a1ea1640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:18.485 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.482+0000 7f68a1ea1640 1 -- 172.21.15.132:0/2187772620 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"} v 0) v1 -- 0x7f6880002220 con 0x7f689c134cb0 2024-01-24T23:28:18.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.486+0000 7f6898ff9640 1 -- 172.21.15.132:0/2187772620 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"}]=0 v248) v1 ==== 106+0+3 (unknown 2759744126 0 944643765) 0x7f688c014710 con 0x7f689c134cb0 2024-01-24T23:28:18.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.486+0000 7f6898ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"}] 2024-01-24T23:28:18.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.486+0000 7f6898ff9640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:28:18.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.486+0000 7f68a1ea1640 1 -- 172.21.15.132:0/2187772620 >> v1:172.21.15.132:6812/1835401922 conn(0x7f687c036ee0 legacy=0x7f687c0393a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:18.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.486+0000 7f68a1ea1640 10 monclient: shutdown 2024-01-24T23:28:18.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.486+0000 7f68a1ea1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:18.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.486+0000 7f68a1ea1640 1 -- 172.21.15.132:0/2187772620 >> v1:172.21.15.132:6790/0 conn(0x7f689c134cb0 legacy=0x7f689c1c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:18.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.486+0000 7f68a1ea1640 1 -- 172.21.15.132:0/2187772620 shutdown_connections 2024-01-24T23:28:18.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.486+0000 7f68a1ea1640 1 -- 172.21.15.132:0/2187772620 >> 172.21.15.132:0/2187772620 conn(0x7f689c0a2340 msgr2=0x7f689c0a4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:18.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.486+0000 7f68a1ea1640 1 -- 172.21.15.132:0/2187772620 shutdown_connections 2024-01-24T23:28:18.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.486+0000 7f68a1ea1640 1 -- 172.21.15.132:0/2187772620 wait complete. 2024-01-24T23:28:18.501 INFO:tasks.workunit.client.0.smithi132.stderr:+ test 12 -eq 12 2024-01-24T23:28:18.501 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd create 1fd09acd-2e41-4645-8e46-2b2acbb8d380 2024-01-24T23:28:18.611 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.610+0000 7f353f16d640 10 monclient: get_monmap_and_config 2024-01-24T23:28:18.611 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.610+0000 7f353f16d640 10 monclient: build_initial_monmap 2024-01-24T23:28:18.611 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.610+0000 7f353f16d640 10 monclient: monmap: 2024-01-24T23:28:18.611 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:18.611 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:18.611 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:18.614333+0000 2024-01-24T23:28:18.611 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:18.614333+0000 2024-01-24T23:28:18.611 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:18.611 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:18.612 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:18.612 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:18.612 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:18.612 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:18.612 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:18.612 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:18.612 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:18.612 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:18.612 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:18.612 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:18.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.610+0000 7f353f16d640 1 Processor -- start 2024-01-24T23:28:18.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.610+0000 7f353f16d640 1 -- start start 2024-01-24T23:28:18.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.610+0000 7f353f16d640 10 monclient: init 2024-01-24T23:28:18.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.610+0000 7f353f16d640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:18.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.610+0000 7f353f16d640 10 monclient: _add_conns ranks=[7,8,4,1,3,0,5,6,2] 2024-01-24T23:28:18.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.610+0000 7f353f16d640 10 monclient(hunting): picked mon.noname-i con 0x7f3538135e10 addr v1:172.21.15.132:6792/0 2024-01-24T23:28:18.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.610+0000 7f353f16d640 10 monclient(hunting): picked mon.noname-e con 0x7f35381361f0 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:18.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.610+0000 7f353f16d640 10 monclient(hunting): picked mon.noname-c con 0x7f35381365f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:28:18.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.610+0000 7f353f16d640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3538064700 con 0x7f35381365f0 2024-01-24T23:28:18.616 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.610+0000 7f353f16d640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f353809c3e0 con 0x7f3538135e10 2024-01-24T23:28:18.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.610+0000 7f353f16d640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35380a4f90 con 0x7f35381361f0 2024-01-24T23:28:18.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.610+0000 7f353f16d640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:18.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.610+0000 7f353f16d640 10 monclient(hunting): authenticate will time out at 1385.939745s 2024-01-24T23:28:18.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.610+0000 7f353cf09640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f3538135e10 legacy=0x7f353809bcd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:28:18.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f3537fff640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f35381361f0 0x7f353813bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.132:39772/0 (socket says 172.21.15.132:39772) 2024-01-24T23:28:18.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f3537fff640 1 -- 172.21.15.132:0/410099465 learned_addr learned my addr 172.21.15.132:0/410099465 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:18.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353d70a640 1 --1- 172.21.15.132:0/410099465 >> v1:172.21.15.115:6791/0 conn(0x7f35381365f0 0x7f353813f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:46252/0 (socket says 172.21.15.132:46252) 2024-01-24T23:28:18.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 1 -- 172.21.15.132:0/410099465 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 214711223 0 0) 0x7f3538064700 con 0x7f35381365f0 2024-01-24T23:28:18.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 10 monclient(hunting): _negotiate 2024-01-24T23:28:18.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:18.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:18.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 10 monclient(hunting): my global_id is 95107 2024-01-24T23:28:18.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 1 -- 172.21.15.132:0/410099465 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3520003530 con 0x7f35381365f0 2024-01-24T23:28:18.617 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 1 -- 172.21.15.132:0/410099465 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2739211522 0 0) 0x7f35380a4f90 con 0x7f35381361f0 2024-01-24T23:28:18.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 10 monclient(hunting): _negotiate 2024-01-24T23:28:18.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:18.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:18.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 10 monclient(hunting): my global_id is 86165 2024-01-24T23:28:18.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 1 -- 172.21.15.132:0/410099465 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3538064700 con 0x7f35381361f0 2024-01-24T23:28:18.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 1 -- 172.21.15.132:0/410099465 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1155193520 0 0) 0x7f3538064700 con 0x7f35381361f0 2024-01-24T23:28:18.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 1 -- 172.21.15.132:0/410099465 >> v1:172.21.15.132:6792/0 conn(0x7f3538135e10 legacy=0x7f353809bcd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:28:18.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 1 -- 172.21.15.132:0/410099465 >> v1:172.21.15.115:6791/0 conn(0x7f35381365f0 legacy=0x7f353813f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:18.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 10 monclient: _finish_hunting 0 2024-01-24T23:28:18.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 1 monclient: found mon.noname-e 2024-01-24T23:28:18.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:18.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:28:18.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 1 -- 172.21.15.132:0/410099465 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f353809b010 con 0x7f35381361f0 2024-01-24T23:28:18.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 10 monclient: _finish_auth 0 2024-01-24T23:28:18.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 10 monclient: _check_auth_tickets 2024-01-24T23:28:18.619 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:18.619 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 1 -- 172.21.15.132:0/410099465 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f352c0017a0 con 0x7f35381361f0 2024-01-24T23:28:18.619 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:18.619 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:28:18.619 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 10 monclient: dump: 2024-01-24T23:28:18.619 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:18.619 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:18.619 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:18.619 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:18.619 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:18.619 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:18.619 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:18.619 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:18.619 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:18.619 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:18.620 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:18.620 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:18.620 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:18.620 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:18.620 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:18.620 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:18.620 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 5 monclient: authenticate success, global_id 86165 2024-01-24T23:28:18.620 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:18.620 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 1 -- 172.21.15.132:0/410099465 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f352c001ac0 con 0x7f35381361f0 2024-01-24T23:28:18.620 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:18.620 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:18.620 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 4 set_mon_vals no callback set 2024-01-24T23:28:18.620 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 1 -- 172.21.15.132:0/410099465 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f352c004bf0 con 0x7f35381361f0 2024-01-24T23:28:18.620 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:18.621 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:28:18.621 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35377fe640 10 monclient: dump: 2024-01-24T23:28:18.621 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:18.621 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:18.621 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:18.621 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:18.621 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:18.621 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:18.621 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:18.621 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:18.621 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:18.621 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:18.621 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:18.621 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:18.621 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:18.621 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:18.622 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:18.622 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:18.622 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 10 monclient: shutdown 2024-01-24T23:28:18.622 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:18.622 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 1 -- 172.21.15.132:0/410099465 >> v1:172.21.15.115:6793/0 conn(0x7f35381361f0 legacy=0x7f353813bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:18.622 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 1 -- 172.21.15.132:0/410099465 shutdown_connections 2024-01-24T23:28:18.622 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 1 -- 172.21.15.132:0/410099465 >> 172.21.15.132:0/410099465 conn(0x7f35380a6790 msgr2=0x7f3538071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:18.622 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 1 -- 172.21.15.132:0/410099465 shutdown_connections 2024-01-24T23:28:18.622 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 1 -- 172.21.15.132:0/410099465 wait complete. 2024-01-24T23:28:18.622 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 10 monclient: build_initial_monmap 2024-01-24T23:28:18.622 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 10 monclient: monmap: 2024-01-24T23:28:18.622 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:18.622 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:18.622 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:18.623 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:18.623 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:18.623 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:18.623 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:18.623 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:18.623 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:18.623 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:18.623 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:18.623 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:18.623 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:18.623 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:18.623 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:18.623 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:18.623 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 1 Processor -- start 2024-01-24T23:28:18.623 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 1 -- start start 2024-01-24T23:28:18.624 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 10 monclient: init 2024-01-24T23:28:18.627 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:18.627 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 10 monclient: _add_conns ranks=[3,1,8,7,4,0,5,6,2] 2024-01-24T23:28:18.627 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 10 monclient(hunting): picked mon.noname-g con 0x7f3538135e10 addr v1:172.21.15.132:6790/0 2024-01-24T23:28:18.627 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 10 monclient(hunting): picked mon.noname-f con 0x7f35381361f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:28:18.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 10 monclient(hunting): picked mon.noname-e con 0x7f35381365f0 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:18.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f353809ca00 con 0x7f35381361f0 2024-01-24T23:28:18.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3538122520 con 0x7f3538135e10 2024-01-24T23:28:18.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3538122800 con 0x7f35381365f0 2024-01-24T23:28:18.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:18.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353f16d640 10 monclient(hunting): authenticate will time out at 1385.943325s 2024-01-24T23:28:18.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353cf09640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f3538135e10 0x7f3538135070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:49590/0 (socket says 172.21.15.132:49590) 2024-01-24T23:28:18.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f353cf09640 1 -- 172.21.15.132:0/1824725070 learned_addr learned my addr 172.21.15.132:0/1824725070 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:18.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 1 -- 172.21.15.132:0/1824725070 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2140924691 0 0) 0x7f3538122520 con 0x7f3538135e10 2024-01-24T23:28:18.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 10 monclient(hunting): _negotiate 2024-01-24T23:28:18.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:18.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:18.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 10 monclient(hunting): my global_id is 6942 2024-01-24T23:28:18.629 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 1 -- 172.21.15.132:0/1824725070 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3524003530 con 0x7f3538135e10 2024-01-24T23:28:18.629 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 1 -- 172.21.15.132:0/1824725070 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1355583935 0 0) 0x7f353809ca00 con 0x7f35381361f0 2024-01-24T23:28:18.629 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 10 monclient(hunting): _negotiate 2024-01-24T23:28:18.629 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:18.629 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:18.629 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 10 monclient(hunting): my global_id is 124462 2024-01-24T23:28:18.629 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 1 -- 172.21.15.132:0/1824725070 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3538122520 con 0x7f35381361f0 2024-01-24T23:28:18.629 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 1 -- 172.21.15.132:0/1824725070 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3443577368 0 0) 0x7f3524003530 con 0x7f3538135e10 2024-01-24T23:28:18.629 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 1 -- 172.21.15.132:0/1824725070 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f353809ca00 con 0x7f3538135e10 2024-01-24T23:28:18.629 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 1 -- 172.21.15.132:0/1824725070 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3528004680 con 0x7f3538135e10 2024-01-24T23:28:18.629 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:18.629 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:28:18.629 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 10 monclient(hunting): dump: 2024-01-24T23:28:18.629 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:18.630 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:18.630 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:18.630 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:18.630 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:18.630 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:18.630 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:18.630 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:18.630 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:18.630 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:18.630 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:18.630 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:18.630 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:18.630 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:18.630 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:18.630 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:18.631 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:18.631 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 1 -- 172.21.15.132:0/1824725070 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 957251419 0 0) 0x7f3538122520 con 0x7f35381361f0 2024-01-24T23:28:18.631 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 1 -- 172.21.15.132:0/1824725070 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3524003530 con 0x7f35381361f0 2024-01-24T23:28:18.631 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 1 -- 172.21.15.132:0/1824725070 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2629101902 0 0) 0x7f3538122800 con 0x7f35381365f0 2024-01-24T23:28:18.631 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 10 monclient(hunting): _negotiate 2024-01-24T23:28:18.631 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:18.631 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:18.631 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 10 monclient(hunting): my global_id is 86174 2024-01-24T23:28:18.631 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 1 -- 172.21.15.132:0/1824725070 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3538122520 con 0x7f35381365f0 2024-01-24T23:28:18.631 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 1 -- 172.21.15.132:0/1824725070 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f352c001760 con 0x7f35381361f0 2024-01-24T23:28:18.631 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:18.631 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:28:18.631 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.614+0000 7f35357fa640 10 monclient(hunting): dump: 2024-01-24T23:28:18.631 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:18.631 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:18.631 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:18.632 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:18.632 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:18.632 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:18.632 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:18.632 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:18.632 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:18.632 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:18.632 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:18.632 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:18.632 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:18.632 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:18.632 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:18.632 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:18.632 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f35357fa640 1 -- 172.21.15.132:0/1824725070 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 423807121 0 0) 0x7f353809ca00 con 0x7f3538135e10 2024-01-24T23:28:18.632 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f35357fa640 1 -- 172.21.15.132:0/1824725070 >> v1:172.21.15.115:6793/0 conn(0x7f35381365f0 legacy=0x7f35381d38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:18.633 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f35357fa640 1 -- 172.21.15.132:0/1824725070 >> v1:172.21.15.132:6789/0 conn(0x7f35381361f0 legacy=0x7f35381d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:18.633 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f35357fa640 10 monclient: _finish_hunting 0 2024-01-24T23:28:18.633 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f35357fa640 1 monclient: found mon.g 2024-01-24T23:28:18.633 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f35357fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:18.633 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f35357fa640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:18.633 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f35357fa640 1 -- 172.21.15.132:0/1824725070 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f353809c6b0 con 0x7f3538135e10 2024-01-24T23:28:18.633 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f35357fa640 10 monclient: _finish_auth 0 2024-01-24T23:28:18.633 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f35357fa640 10 monclient: _check_auth_tickets 2024-01-24T23:28:18.633 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f35357fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:18.633 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f353f16d640 5 monclient: authenticate success, global_id 6942 2024-01-24T23:28:18.633 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f353f16d640 10 monclient: _renew_subs 2024-01-24T23:28:18.633 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f353f16d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:18.633 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f353f16d640 1 -- 172.21.15.132:0/1824725070 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35381cb610 con 0x7f3538135e10 2024-01-24T23:28:18.633 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f353f16d640 10 monclient: _renew_subs 2024-01-24T23:28:18.633 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f353f16d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:18.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f353f16d640 1 -- 172.21.15.132:0/1824725070 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f35381cb340 con 0x7f3538135e10 2024-01-24T23:28:18.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f35357fa640 1 -- 172.21.15.132:0/1824725070 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f35280048b0 con 0x7f3538135e10 2024-01-24T23:28:18.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f35357fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:18.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f35357fa640 1 -- 172.21.15.132:0/1824725070 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3528004de0 con 0x7f3538135e10 2024-01-24T23:28:18.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f35357fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:18.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f3536ffd640 4 set_mon_vals no callback set 2024-01-24T23:28:18.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f35357fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:28:18.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f35357fa640 10 monclient: dump: 2024-01-24T23:28:18.634 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:18.634 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:18.634 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:18.634 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:18.634 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:18.634 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:18.634 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:18.635 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:18.635 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:18.635 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:18.635 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:18.635 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:18.635 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:18.635 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:18.635 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:18.635 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:18.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f353f16d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:18.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f353f16d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:18.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f353f16d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:18.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f353f16d640 1 -- 172.21.15.132:0/1824725070 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3518005090 con 0x7f3538135e10 2024-01-24T23:28:18.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f35357fa640 1 -- 172.21.15.132:0/1824725070 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f3528004f40 con 0x7f3538135e10 2024-01-24T23:28:18.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f35357fa640 1 -- 172.21.15.132:0/1824725070 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(248..248 src has 1..248) v4 ==== 7051+0+0 (unknown 1773304109 0 0) 0x7f3528044780 con 0x7f3538135e10 2024-01-24T23:28:18.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.618+0000 7f3537fff640 10 monclient: get_auth_request con 0x7f3524037890 auth_method 0 2024-01-24T23:28:18.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.622+0000 7f35357fa640 1 -- 172.21.15.132:0/1824725070 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f3528014d90 con 0x7f3538135e10 2024-01-24T23:28:18.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.622+0000 7f35357fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:18.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.622+0000 7f35357fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:18.774 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.774+0000 7f353f16d640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "1fd09acd-2e41-4645-8e46-2b2acbb8d380"}] 2024-01-24T23:28:18.774 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.774+0000 7f353f16d640 10 monclient: _send_command 2 [{"prefix": "osd create", "uuid": "1fd09acd-2e41-4645-8e46-2b2acbb8d380"}] 2024-01-24T23:28:18.774 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.774+0000 7f353f16d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:18.774 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.774+0000 7f353f16d640 1 -- 172.21.15.132:0/1824725070 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd create", "uuid": "1fd09acd-2e41-4645-8e46-2b2acbb8d380"} v 0) v1 -- 0x7f35180058c0 con 0x7f3538135e10 2024-01-24T23:28:18.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.974+0000 7f35357fa640 1 -- 172.21.15.132:0/1824725070 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd create", "uuid": "1fd09acd-2e41-4645-8e46-2b2acbb8d380"}]=0 v249) v1 ==== 106+0+3 (unknown 768028177 0 1367428999) 0x7f3528013200 con 0x7f3538135e10 2024-01-24T23:28:18.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.974+0000 7f35357fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create", "uuid": "1fd09acd-2e41-4645-8e46-2b2acbb8d380"}] 2024-01-24T23:28:18.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:18.974+0000 7f35357fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:28:19.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.070+0000 7f353f16d640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "1fd09acd-2e41-4645-8e46-2b2acbb8d380"}] 2024-01-24T23:28:19.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.070+0000 7f353f16d640 10 monclient: _send_command 3 [{"prefix": "osd create", "uuid": "1fd09acd-2e41-4645-8e46-2b2acbb8d380"}] 2024-01-24T23:28:19.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.070+0000 7f353f16d640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:19.073 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.070+0000 7f353f16d640 1 -- 172.21.15.132:0/1824725070 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd create", "uuid": "1fd09acd-2e41-4645-8e46-2b2acbb8d380"} v 0) v1 -- 0x7f3518002220 con 0x7f3538135e10 2024-01-24T23:28:19.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.074+0000 7f35357fa640 1 -- 172.21.15.132:0/1824725070 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd create", "uuid": "1fd09acd-2e41-4645-8e46-2b2acbb8d380"}]=0 v249) v1 ==== 106+0+3 (unknown 768028177 0 1367428999) 0x7f3528014980 con 0x7f3538135e10 2024-01-24T23:28:19.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.074+0000 7f35357fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create", "uuid": "1fd09acd-2e41-4645-8e46-2b2acbb8d380"}] 2024-01-24T23:28:19.074 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.074+0000 7f35357fa640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:28:19.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.074+0000 7f353f16d640 1 -- 172.21.15.132:0/1824725070 >> v1:172.21.15.132:6812/1835401922 conn(0x7f3524037890 legacy=0x7f3524039d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:19.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.074+0000 7f353f16d640 10 monclient: shutdown 2024-01-24T23:28:19.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.074+0000 7f353f16d640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:19.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.074+0000 7f353f16d640 1 -- 172.21.15.132:0/1824725070 >> v1:172.21.15.132:6790/0 conn(0x7f3538135e10 legacy=0x7f3538135070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:19.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.074+0000 7f353f16d640 1 -- 172.21.15.132:0/1824725070 shutdown_connections 2024-01-24T23:28:19.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.074+0000 7f353f16d640 1 -- 172.21.15.132:0/1824725070 >> 172.21.15.132:0/1824725070 conn(0x7f35380a6790 msgr2=0x7f353806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:19.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.074+0000 7f353f16d640 1 -- 172.21.15.132:0/1824725070 shutdown_connections 2024-01-24T23:28:19.076 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.074+0000 7f353f16d640 1 -- 172.21.15.132:0/1824725070 wait complete. 2024-01-24T23:28:19.089 INFO:tasks.workunit.client.0.smithi132.stderr:+ nb=14 2024-01-24T23:28:19.089 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd create 1fd09acd-2e41-4645-8e46-2b2acbb8d380 2024-01-24T23:28:19.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 10 monclient: get_monmap_and_config 2024-01-24T23:28:19.201 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 10 monclient: build_initial_monmap 2024-01-24T23:28:19.202 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 10 monclient: monmap: 2024-01-24T23:28:19.202 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:19.202 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:19.202 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:19.204889+0000 2024-01-24T23:28:19.202 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:19.204889+0000 2024-01-24T23:28:19.202 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:19.202 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:19.202 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:19.202 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:19.202 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:19.202 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:19.202 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:19.202 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:19.202 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:19.202 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:19.202 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:19.203 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:19.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 1 Processor -- start 2024-01-24T23:28:19.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 1 -- start start 2024-01-24T23:28:19.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 10 monclient: init 2024-01-24T23:28:19.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:19.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 10 monclient: _add_conns ranks=[3,7,5,4,1,6,2,0,8] 2024-01-24T23:28:19.210 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 10 monclient(hunting): picked mon.noname-g con 0x7f7350135e10 addr v1:172.21.15.132:6790/0 2024-01-24T23:28:19.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 10 monclient(hunting): picked mon.noname-i con 0x7f73501361f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:28:19.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 10 monclient(hunting): picked mon.noname-h con 0x7f73501365f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:28:19.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7350064700 con 0x7f7350135e10 2024-01-24T23:28:19.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f735009c3e0 con 0x7f73501365f0 2024-01-24T23:28:19.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73500a4f90 con 0x7f73501361f0 2024-01-24T23:28:19.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:19.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 10 monclient(hunting): authenticate will time out at 1386.529503s 2024-01-24T23:28:19.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734e7fc640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f73501361f0 legacy=0x7f735013bed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:28:19.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734effd640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f7350135e10 0x7f735009bcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:49606/0 (socket says 172.21.15.132:49606) 2024-01-24T23:28:19.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734f7fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f73501365f0 0x7f735013f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:49306/0 (socket says 172.21.15.132:49306) 2024-01-24T23:28:19.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734effd640 1 -- 172.21.15.132:0/2014514739 learned_addr learned my addr 172.21.15.132:0/2014514739 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:19.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 1 -- 172.21.15.132:0/2014514739 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966799564 0 0) 0x7f7350064700 con 0x7f7350135e10 2024-01-24T23:28:19.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:28:19.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:19.211 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:19.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 10 monclient(hunting): my global_id is 6951 2024-01-24T23:28:19.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 1 -- 172.21.15.132:0/2014514739 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7330003530 con 0x7f7350135e10 2024-01-24T23:28:19.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 1 -- 172.21.15.132:0/2014514739 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940630107 0 0) 0x7f735009c3e0 con 0x7f73501365f0 2024-01-24T23:28:19.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:28:19.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:19.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:19.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 10 monclient(hunting): my global_id is 6944 2024-01-24T23:28:19.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 1 -- 172.21.15.132:0/2014514739 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7350064700 con 0x7f73501365f0 2024-01-24T23:28:19.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 1 -- 172.21.15.132:0/2014514739 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2997515978 0 0) 0x7f7330003530 con 0x7f7350135e10 2024-01-24T23:28:19.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 1 -- 172.21.15.132:0/2014514739 >> v1:172.21.15.132:6792/0 conn(0x7f73501361f0 legacy=0x7f735013bed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:28:19.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 1 -- 172.21.15.132:0/2014514739 >> v1:172.21.15.132:6791/0 conn(0x7f73501365f0 legacy=0x7f735013f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:19.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 10 monclient: _finish_hunting 0 2024-01-24T23:28:19.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 1 monclient: found mon.noname-g 2024-01-24T23:28:19.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:19.212 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:28:19.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 1 -- 172.21.15.132:0/2014514739 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735009b010 con 0x7f7350135e10 2024-01-24T23:28:19.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 10 monclient: _finish_auth 0 2024-01-24T23:28:19.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 10 monclient: _check_auth_tickets 2024-01-24T23:28:19.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:19.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 1 -- 172.21.15.132:0/2014514739 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7348001b40 con 0x7f7350135e10 2024-01-24T23:28:19.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:19.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:28:19.213 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 10 monclient: dump: 2024-01-24T23:28:19.213 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:19.213 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:19.213 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:19.213 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:19.213 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:19.213 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:19.213 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:19.213 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:19.214 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:19.214 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:19.214 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:19.214 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:19.214 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:19.214 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:19.214 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:19.214 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:19.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 5 monclient: authenticate success, global_id 6951 2024-01-24T23:28:19.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:19.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 1 -- 172.21.15.132:0/2014514739 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7348001e20 con 0x7f7350135e10 2024-01-24T23:28:19.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:19.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 1 -- 172.21.15.132:0/2014514739 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7348004f70 con 0x7f7350135e10 2024-01-24T23:28:19.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:19.214 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:28:19.215 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f734dffb640 10 monclient: dump: 2024-01-24T23:28:19.215 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:19.215 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:19.215 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:19.215 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:19.215 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:19.215 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:19.215 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:19.215 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:19.215 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:19.215 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:19.215 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:19.215 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:19.215 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:19.215 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:19.215 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:19.216 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:19.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:19.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 4 set_mon_vals no callback set 2024-01-24T23:28:19.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 10 monclient: shutdown 2024-01-24T23:28:19.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:19.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 1 -- 172.21.15.132:0/2014514739 >> v1:172.21.15.132:6790/0 conn(0x7f7350135e10 legacy=0x7f735009bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:19.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 1 -- 172.21.15.132:0/2014514739 shutdown_connections 2024-01-24T23:28:19.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 1 -- 172.21.15.132:0/2014514739 >> 172.21.15.132:0/2014514739 conn(0x7f73500a6790 msgr2=0x7f7350071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:19.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 1 -- 172.21.15.132:0/2014514739 shutdown_connections 2024-01-24T23:28:19.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 1 -- 172.21.15.132:0/2014514739 wait complete. 2024-01-24T23:28:19.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 10 monclient: build_initial_monmap 2024-01-24T23:28:19.216 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 10 monclient: monmap: 2024-01-24T23:28:19.216 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:19.216 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:19.217 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:19.217 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:19.217 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:19.217 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:19.217 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:19.217 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:19.217 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:19.217 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:19.217 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:19.217 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:19.217 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:19.217 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:19.217 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:19.217 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:19.217 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 1 Processor -- start 2024-01-24T23:28:19.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 1 -- start start 2024-01-24T23:28:19.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.202+0000 7f7355717640 10 monclient: init 2024-01-24T23:28:19.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f7355717640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:19.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f7355717640 10 monclient: _add_conns ranks=[4,8,6,1,5,3,2,0,7] 2024-01-24T23:28:19.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f7355717640 10 monclient(hunting): picked mon.noname-c con 0x7f7350135e10 addr v1:172.21.15.115:6791/0 2024-01-24T23:28:19.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f7355717640 10 monclient(hunting): picked mon.noname-e con 0x7f73501361f0 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:19.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f7355717640 10 monclient(hunting): picked mon.noname-d con 0x7f73501365f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:28:19.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f7355717640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73501224d0 con 0x7f7350135e10 2024-01-24T23:28:19.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f7355717640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f735009ca70 con 0x7f73501365f0 2024-01-24T23:28:19.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f7355717640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73501c80a0 con 0x7f73501361f0 2024-01-24T23:28:19.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f7355717640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:19.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f7355717640 10 monclient(hunting): authenticate will time out at 1386.532511s 2024-01-24T23:28:19.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f734effd640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f7350135e10 0x7f73501350d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.132:46260/0 (socket says 172.21.15.132:46260) 2024-01-24T23:28:19.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f734effd640 1 -- 172.21.15.132:0/1970375575 learned_addr learned my addr 172.21.15.132:0/1970375575 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:19.218 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828359673 0 0) 0x7f73501224d0 con 0x7f7350135e10 2024-01-24T23:28:19.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient(hunting): _negotiate 2024-01-24T23:28:19.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:19.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:19.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient(hunting): my global_id is 95116 2024-01-24T23:28:19.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7334003530 con 0x7f7350135e10 2024-01-24T23:28:19.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4276672904 0 0) 0x7f73501c80a0 con 0x7f73501361f0 2024-01-24T23:28:19.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient(hunting): _negotiate 2024-01-24T23:28:19.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:19.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:19.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient(hunting): my global_id is 86183 2024-01-24T23:28:19.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73501224d0 con 0x7f73501361f0 2024-01-24T23:28:19.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1745818531 0 0) 0x7f7340001140 con 0x7f73501365f0 2024-01-24T23:28:19.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient(hunting): _negotiate 2024-01-24T23:28:19.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:19.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:19.219 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient(hunting): my global_id is 7188 2024-01-24T23:28:19.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73501c80a0 con 0x7f73501365f0 2024-01-24T23:28:19.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 287676809 0 0) 0x7f73501c80a0 con 0x7f73501365f0 2024-01-24T23:28:19.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7340001140 con 0x7f73501365f0 2024-01-24T23:28:19.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 431242593 0 0) 0x7f7334003530 con 0x7f7350135e10 2024-01-24T23:28:19.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f73501c80a0 con 0x7f7350135e10 2024-01-24T23:28:19.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7340001af0 con 0x7f73501365f0 2024-01-24T23:28:19.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:19.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:28:19.220 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient(hunting): dump: 2024-01-24T23:28:19.220 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:19.220 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:19.220 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:19.220 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:19.220 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:19.220 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:19.221 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:19.221 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:19.221 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:19.221 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:19.221 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:19.221 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:19.221 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:19.221 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:19.221 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:19.221 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:19.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:19.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7348004eb0 con 0x7f7350135e10 2024-01-24T23:28:19.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:19.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:28:19.221 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient(hunting): dump: 2024-01-24T23:28:19.222 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:19.222 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:19.222 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:19.222 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:19.222 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:19.222 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:19.222 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:19.222 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:19.222 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:19.222 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:19.222 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:19.222 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:19.222 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:19.222 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:19.222 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:19.223 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:19.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3794393806 0 0) 0x7f73501224d0 con 0x7f73501361f0 2024-01-24T23:28:19.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7334003530 con 0x7f73501361f0 2024-01-24T23:28:19.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7338004270 con 0x7f73501361f0 2024-01-24T23:28:19.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:19.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:28:19.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient(hunting): dump: 2024-01-24T23:28:19.223 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:19.223 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:19.223 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:19.223 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:19.223 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:19.223 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:19.223 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:19.224 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:19.224 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:19.224 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:19.224 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:19.224 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:19.224 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:19.224 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:19.224 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:19.224 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:19.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3037248771 0 0) 0x7f7340001140 con 0x7f73501365f0 2024-01-24T23:28:19.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 >> v1:172.21.15.115:6793/0 conn(0x7f73501361f0 legacy=0x7f73501d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:19.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 >> v1:172.21.15.115:6791/0 conn(0x7f7350135e10 legacy=0x7f73501350d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:19.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient: _finish_hunting 0 2024-01-24T23:28:19.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 1 monclient: found mon.d 2024-01-24T23:28:19.224 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:19.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:28:19.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735009c6b0 con 0x7f73501365f0 2024-01-24T23:28:19.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient: _finish_auth 0 2024-01-24T23:28:19.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient: _check_auth_tickets 2024-01-24T23:28:19.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:19.225 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f7355717640 5 monclient: authenticate success, global_id 7188 2024-01-24T23:28:19.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f7355717640 10 monclient: _renew_subs 2024-01-24T23:28:19.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f7355717640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:28:19.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f7355717640 1 -- 172.21.15.132:0/1970375575 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73501bf530 con 0x7f73501365f0 2024-01-24T23:28:19.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f7355717640 10 monclient: _renew_subs 2024-01-24T23:28:19.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f7355717640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:28:19.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f7355717640 1 -- 172.21.15.132:0/1970375575 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f73501357e0 con 0x7f73501365f0 2024-01-24T23:28:19.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7340000fc0 con 0x7f73501365f0 2024-01-24T23:28:19.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:19.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f734d7fa640 4 set_mon_vals no callback set 2024-01-24T23:28:19.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7340004de0 con 0x7f73501365f0 2024-01-24T23:28:19.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:19.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:28:19.226 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 10 monclient: dump: 2024-01-24T23:28:19.226 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:19.226 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:19.227 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:19.227 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:19.227 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:19.227 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:19.227 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:19.227 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:19.227 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:19.227 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:19.227 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:19.227 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:19.227 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:19.227 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:19.227 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:19.227 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:19.227 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f7355717640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:19.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f7355717640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:19.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f7355717640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:28:19.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f7355717640 1 -- 172.21.15.132:0/1970375575 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7328005090 con 0x7f73501365f0 2024-01-24T23:28:19.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.206+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f734000f860 con 0x7f73501365f0 2024-01-24T23:28:19.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.210+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(249..249 src has 1..249) v4 ==== 7051+0+0 (unknown 2675953290 0 0) 0x7f73400441d0 con 0x7f73501365f0 2024-01-24T23:28:19.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.210+0000 7f734effd640 10 monclient: get_auth_request con 0x7f7334037c40 auth_method 0 2024-01-24T23:28:19.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.218+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f73400144e0 con 0x7f73501365f0 2024-01-24T23:28:19.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.218+0000 7f73477fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:19.228 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.218+0000 7f73477fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:19.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.386+0000 7f7355717640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "1fd09acd-2e41-4645-8e46-2b2acbb8d380"}] 2024-01-24T23:28:19.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.386+0000 7f7355717640 10 monclient: _send_command 2 [{"prefix": "osd create", "uuid": "1fd09acd-2e41-4645-8e46-2b2acbb8d380"}] 2024-01-24T23:28:19.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.386+0000 7f7355717640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:28:19.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.386+0000 7f7355717640 1 -- 172.21.15.132:0/1970375575 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd create", "uuid": "1fd09acd-2e41-4645-8e46-2b2acbb8d380"} v 0) v1 -- 0x7f7328005340 con 0x7f73501365f0 2024-01-24T23:28:19.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.390+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "osd create", "uuid": "1fd09acd-2e41-4645-8e46-2b2acbb8d380"}]=0 v249) v1 ==== 106+0+3 (unknown 768028177 0 1367428999) 0x7f7340014300 con 0x7f73501365f0 2024-01-24T23:28:19.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.390+0000 7f73477fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create", "uuid": "1fd09acd-2e41-4645-8e46-2b2acbb8d380"}] 2024-01-24T23:28:19.391 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.390+0000 7f73477fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:28:19.486 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.486+0000 7f7355717640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "1fd09acd-2e41-4645-8e46-2b2acbb8d380"}] 2024-01-24T23:28:19.487 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.486+0000 7f7355717640 10 monclient: _send_command 3 [{"prefix": "osd create", "uuid": "1fd09acd-2e41-4645-8e46-2b2acbb8d380"}] 2024-01-24T23:28:19.487 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.486+0000 7f7355717640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:28:19.487 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.486+0000 7f7355717640 1 -- 172.21.15.132:0/1970375575 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "osd create", "uuid": "1fd09acd-2e41-4645-8e46-2b2acbb8d380"} v 0) v1 -- 0x7f73280010a0 con 0x7f73501365f0 2024-01-24T23:28:19.488 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.486+0000 7f73477fe640 1 -- 172.21.15.132:0/1970375575 <== mon.6 v1:172.21.15.115:6792/0 11 ==== mon_command_ack([{"prefix": "osd create", "uuid": "1fd09acd-2e41-4645-8e46-2b2acbb8d380"}]=0 v249) v1 ==== 106+0+3 (unknown 768028177 0 1367428999) 0x7f7340014300 con 0x7f73501365f0 2024-01-24T23:28:19.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.486+0000 7f73477fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create", "uuid": "1fd09acd-2e41-4645-8e46-2b2acbb8d380"}] 2024-01-24T23:28:19.489 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.486+0000 7f73477fe640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:28:19.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.490+0000 7f7355717640 1 -- 172.21.15.132:0/1970375575 >> v1:172.21.15.132:6812/1835401922 conn(0x7f7334037c40 legacy=0x7f733403a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:19.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.490+0000 7f7355717640 10 monclient: shutdown 2024-01-24T23:28:19.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.490+0000 7f7355717640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:19.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.490+0000 7f7355717640 1 -- 172.21.15.132:0/1970375575 >> v1:172.21.15.115:6792/0 conn(0x7f73501365f0 legacy=0x7f73501d3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:19.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.490+0000 7f7355717640 1 -- 172.21.15.132:0/1970375575 shutdown_connections 2024-01-24T23:28:19.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.490+0000 7f7355717640 1 -- 172.21.15.132:0/1970375575 >> 172.21.15.132:0/1970375575 conn(0x7f73500a6790 msgr2=0x7f735006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:19.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.490+0000 7f7355717640 1 -- 172.21.15.132:0/1970375575 shutdown_connections 2024-01-24T23:28:19.491 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.490+0000 7f7355717640 1 -- 172.21.15.132:0/1970375575 wait complete. 2024-01-24T23:28:19.505 INFO:tasks.workunit.client.0.smithi132.stderr:+ test 14 -eq 14 2024-01-24T23:28:19.505 INFO:tasks.workunit.client.0.smithi132.stderr:+ test 14 -ne 12 2024-01-24T23:28:19.505 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd rm 12 2024-01-24T23:28:19.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 10 monclient: get_monmap_and_config 2024-01-24T23:28:19.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 10 monclient: build_initial_monmap 2024-01-24T23:28:19.618 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 10 monclient: monmap: 2024-01-24T23:28:19.618 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:19.619 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:19.619 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:19.621684+0000 2024-01-24T23:28:19.619 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:19.621684+0000 2024-01-24T23:28:19.619 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:19.619 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:19.619 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:19.619 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:19.619 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:19.619 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:19.619 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:19.619 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:19.619 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:19.619 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:19.619 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:19.620 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:19.626 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 1 Processor -- start 2024-01-24T23:28:19.626 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 1 -- start start 2024-01-24T23:28:19.626 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 10 monclient: init 2024-01-24T23:28:19.626 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:19.626 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 10 monclient: _add_conns ranks=[8,1,7,3,6,2,0,5,4] 2024-01-24T23:28:19.626 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 10 monclient(hunting): picked mon.noname-e con 0x7f285c09ff20 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:19.626 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 10 monclient(hunting): picked mon.noname-f con 0x7f285c09f9c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:28:19.626 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 10 monclient(hunting): picked mon.noname-i con 0x7f285c13b2c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:28:19.626 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f285c064700 con 0x7f285c09f9c0 2024-01-24T23:28:19.626 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f285c09e760 con 0x7f285c13b2c0 2024-01-24T23:28:19.627 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f285c0a1160 con 0x7f285c09ff20 2024-01-24T23:28:19.627 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:19.627 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 10 monclient(hunting): authenticate will time out at 1386.946397s 2024-01-24T23:28:19.627 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f28613af640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f285c13b2c0 legacy=0x7f285c13d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:28:19.627 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285bfff640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f285c09f9c0 0x7f285c139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:58696/0 (socket says 172.21.15.132:58696) 2024-01-24T23:28:19.627 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285bfff640 1 -- 172.21.15.132:0/4221116917 learned_addr learned my addr 172.21.15.132:0/4221116917 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:19.627 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 1 -- 172.21.15.132:0/4221116917 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3728502681 0 0) 0x7f285c064700 con 0x7f285c09f9c0 2024-01-24T23:28:19.627 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:28:19.627 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:19.627 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:19.627 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 10 monclient(hunting): my global_id is 124471 2024-01-24T23:28:19.627 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 1 -- 172.21.15.132:0/4221116917 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2844003530 con 0x7f285c09f9c0 2024-01-24T23:28:19.627 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 1 -- 172.21.15.132:0/4221116917 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3201598368 0 0) 0x7f2844003530 con 0x7f285c09f9c0 2024-01-24T23:28:19.627 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 1 -- 172.21.15.132:0/4221116917 >> v1:172.21.15.115:6793/0 conn(0x7f285c09ff20 legacy=0x7f285c0a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:19.627 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 1 -- 172.21.15.132:0/4221116917 >> v1:172.21.15.132:6792/0 conn(0x7f285c13b2c0 legacy=0x7f285c13d6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:28:19.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:28:19.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 1 monclient: found mon.noname-f 2024-01-24T23:28:19.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:19.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:28:19.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 1 -- 172.21.15.132:0/4221116917 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285c0a0a10 con 0x7f285c09f9c0 2024-01-24T23:28:19.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 10 monclient: _finish_auth 0 2024-01-24T23:28:19.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:28:19.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:19.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 1 -- 172.21.15.132:0/4221116917 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f284c0017c0 con 0x7f285c09f9c0 2024-01-24T23:28:19.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 5 monclient: authenticate success, global_id 124471 2024-01-24T23:28:19.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:19.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:19.628 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:28:19.629 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 10 monclient: dump: 2024-01-24T23:28:19.629 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:19.629 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:19.629 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:19.629 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:19.629 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:19.629 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:19.629 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:19.629 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:19.629 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:19.629 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:19.629 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:19.629 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:19.629 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:19.629 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:19.630 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:19.630 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:19.630 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:19.630 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 1 -- 172.21.15.132:0/4221116917 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f284c001ae0 con 0x7f285c09f9c0 2024-01-24T23:28:19.630 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:19.630 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:19.630 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 4 set_mon_vals no callback set 2024-01-24T23:28:19.630 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 1 -- 172.21.15.132:0/4221116917 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f284c004bf0 con 0x7f285c09f9c0 2024-01-24T23:28:19.630 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:19.630 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:28:19.630 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f285b7fe640 10 monclient: dump: 2024-01-24T23:28:19.630 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:19.630 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:19.630 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:19.630 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:19.631 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:19.631 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:19.631 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:19.631 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:19.631 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:19.631 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:19.631 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:19.631 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:19.631 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:19.631 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:19.631 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:19.631 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:19.631 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 10 monclient: shutdown 2024-01-24T23:28:19.631 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:19.631 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 1 -- 172.21.15.132:0/4221116917 >> v1:172.21.15.132:6789/0 conn(0x7f285c09f9c0 legacy=0x7f285c139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:19.631 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 1 -- 172.21.15.132:0/4221116917 shutdown_connections 2024-01-24T23:28:19.632 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 1 -- 172.21.15.132:0/4221116917 >> 172.21.15.132:0/4221116917 conn(0x7f285c0a5a10 msgr2=0x7f285c0a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:19.632 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 1 -- 172.21.15.132:0/4221116917 shutdown_connections 2024-01-24T23:28:19.632 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.618+0000 7f2862e12640 1 -- 172.21.15.132:0/4221116917 wait complete. 2024-01-24T23:28:19.632 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 10 monclient: build_initial_monmap 2024-01-24T23:28:19.632 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 10 monclient: monmap: 2024-01-24T23:28:19.632 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:19.632 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:19.632 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:19.632 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:19.632 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:19.632 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:19.632 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:19.632 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:19.632 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:19.632 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:19.633 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:19.633 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:19.633 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:19.633 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:19.633 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:19.633 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:19.633 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 1 Processor -- start 2024-01-24T23:28:19.633 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 1 -- start start 2024-01-24T23:28:19.633 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 10 monclient: init 2024-01-24T23:28:19.633 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:19.633 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 10 monclient: _add_conns ranks=[8,0,5,3,2,7,6,4,1] 2024-01-24T23:28:19.633 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 10 monclient(hunting): picked mon.noname-e con 0x7f285c09f9c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:19.633 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 10 monclient(hunting): picked mon.noname-a con 0x7f285c09ff20 addr v1:172.21.15.115:6789/0 2024-01-24T23:28:19.633 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 10 monclient(hunting): picked mon.noname-h con 0x7f285c13b2c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:28:19.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f285c11d120 con 0x7f285c09ff20 2024-01-24T23:28:19.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f285c071960 con 0x7f285c13b2c0 2024-01-24T23:28:19.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f285c134600 con 0x7f285c09f9c0 2024-01-24T23:28:19.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:19.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 10 monclient(hunting): authenticate will time out at 1386.949407s 2024-01-24T23:28:19.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f285bfff640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f285c09ff20 legacy=0x7f285c11bd20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:28:19.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28613af640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f285c13b2c0 0x7f285c1d1490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:49316/0 (socket says 172.21.15.132:49316) 2024-01-24T23:28:19.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28613af640 1 -- 172.21.15.132:0/1725402339 learned_addr learned my addr 172.21.15.132:0/1725402339 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:19.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 1 -- 172.21.15.132:0/1725402339 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3744971494 0 0) 0x7f285c071960 con 0x7f285c13b2c0 2024-01-24T23:28:19.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 10 monclient(hunting): _negotiate 2024-01-24T23:28:19.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:19.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:19.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 10 monclient(hunting): my global_id is 6953 2024-01-24T23:28:19.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 1 -- 172.21.15.132:0/1725402339 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2848003560 con 0x7f285c13b2c0 2024-01-24T23:28:19.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 1 -- 172.21.15.132:0/1725402339 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4177320951 0 0) 0x7f2848003560 con 0x7f285c13b2c0 2024-01-24T23:28:19.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 1 -- 172.21.15.132:0/1725402339 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f285c071960 con 0x7f285c13b2c0 2024-01-24T23:28:19.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 1 -- 172.21.15.132:0/1725402339 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2854004530 con 0x7f285c13b2c0 2024-01-24T23:28:19.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:19.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:28:19.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 10 monclient(hunting): dump: 2024-01-24T23:28:19.635 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:19.635 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:19.635 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:19.635 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:19.635 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:19.635 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:19.635 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:19.635 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:19.635 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:19.635 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:19.635 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:19.636 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:19.636 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:19.636 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:19.636 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:19.636 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:19.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:19.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 1 -- 172.21.15.132:0/1725402339 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3223232582 0 0) 0x7f285c134600 con 0x7f285c09f9c0 2024-01-24T23:28:19.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 10 monclient(hunting): _negotiate 2024-01-24T23:28:19.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:19.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:19.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 10 monclient(hunting): my global_id is 86201 2024-01-24T23:28:19.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 1 -- 172.21.15.132:0/1725402339 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2848003560 con 0x7f285c09f9c0 2024-01-24T23:28:19.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 1 -- 172.21.15.132:0/1725402339 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3426907016 0 0) 0x7f285c071960 con 0x7f285c13b2c0 2024-01-24T23:28:19.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 1 -- 172.21.15.132:0/1725402339 >> v1:172.21.15.115:6793/0 conn(0x7f285c09f9c0 legacy=0x7f285c11b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:19.636 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 1 -- 172.21.15.132:0/1725402339 >> v1:172.21.15.115:6789/0 conn(0x7f285c09ff20 legacy=0x7f285c11bd20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:28:19.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 10 monclient: _finish_hunting 0 2024-01-24T23:28:19.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 1 monclient: found mon.h 2024-01-24T23:28:19.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:19.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:19.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 1 -- 172.21.15.132:0/1725402339 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285c071680 con 0x7f285c13b2c0 2024-01-24T23:28:19.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 10 monclient: _finish_auth 0 2024-01-24T23:28:19.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 10 monclient: _check_auth_tickets 2024-01-24T23:28:19.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:19.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 5 monclient: authenticate success, global_id 6953 2024-01-24T23:28:19.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 10 monclient: _renew_subs 2024-01-24T23:28:19.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:19.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 1 -- 172.21.15.132:0/1725402339 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f285c11e7a0 con 0x7f285c13b2c0 2024-01-24T23:28:19.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 10 monclient: _renew_subs 2024-01-24T23:28:19.637 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:19.638 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 1 -- 172.21.15.132:0/1725402339 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f285c11c430 con 0x7f285c13b2c0 2024-01-24T23:28:19.638 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 1 -- 172.21.15.132:0/1725402339 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2854004850 con 0x7f285c13b2c0 2024-01-24T23:28:19.638 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:19.638 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 1 -- 172.21.15.132:0/1725402339 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f2854004d80 con 0x7f285c13b2c0 2024-01-24T23:28:19.638 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:19.638 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f285affd640 4 set_mon_vals no callback set 2024-01-24T23:28:19.638 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:28:19.638 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 10 monclient: dump: 2024-01-24T23:28:19.638 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:19.638 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:19.638 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:19.638 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:19.638 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:19.638 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:19.639 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:19.639 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:19.639 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:19.639 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:19.639 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:19.639 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:19.639 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:19.639 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:19.639 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:19.639 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:19.639 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:19.639 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:19.639 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:19.639 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 1 -- 172.21.15.132:0/1725402339 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f2854004f60 con 0x7f285c13b2c0 2024-01-24T23:28:19.639 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2862e12640 1 -- 172.21.15.132:0/1725402339 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f283c005090 con 0x7f285c13b2c0 2024-01-24T23:28:19.640 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f28597fa640 1 -- 172.21.15.132:0/1725402339 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(249..249 src has 1..249) v4 ==== 7051+0+0 (unknown 2675953290 0 0) 0x7f2854044870 con 0x7f285c13b2c0 2024-01-24T23:28:19.640 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.622+0000 7f2860bae640 10 monclient: get_auth_request con 0x7f2848036fa0 auth_method 0 2024-01-24T23:28:19.640 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.626+0000 7f28597fa640 1 -- 172.21.15.132:0/1725402339 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f2854014be0 con 0x7f285c13b2c0 2024-01-24T23:28:19.640 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.626+0000 7f28597fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:19.640 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.626+0000 7f28597fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:19.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.790+0000 7f2862e12640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["12"]}] 2024-01-24T23:28:19.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.790+0000 7f2862e12640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["12"]}] 2024-01-24T23:28:19.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.790+0000 7f2862e12640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:19.791 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.790+0000 7f2862e12640 1 -- 172.21.15.132:0/1725402339 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd rm", "ids": ["12"]} v 0) v1 -- 0x7f283c0058c0 con 0x7f285c13b2c0 2024-01-24T23:28:19.792 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:28:19.790+0000 7f48d6072640 -1 mon.h@5(leader).osd e249 definitely_dead 0 2024-01-24T23:28:19.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.974+0000 7f28597fa640 1 -- 172.21.15.132:0/1725402339 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["12"]}]=0 removed osd.12 v250) v1 ==== 83+0+0 (unknown 1285384214 0 0) 0x7f285401d310 con 0x7f285c13b2c0 2024-01-24T23:28:19.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.974+0000 7f28597fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["12"]}] 2024-01-24T23:28:19.974 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:19.974+0000 7f28597fa640 10 monclient: _finish_command 2 = system:0 removed osd.12 2024-01-24T23:28:20.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.090+0000 7f2862e12640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["12"]}] 2024-01-24T23:28:20.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.090+0000 7f2862e12640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["12"]}] 2024-01-24T23:28:20.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.090+0000 7f2862e12640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:28:20.092 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.090+0000 7f2862e12640 1 -- 172.21.15.132:0/1725402339 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "osd rm", "ids": ["12"]} v 0) v1 -- 0x7f283c000bd0 con 0x7f285c13b2c0 2024-01-24T23:28:20.093 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:28:20.090+0000 7f48d6072640 -1 mon.h@5(leader).osd e250 definitely_dead 0 2024-01-24T23:28:20.095 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.090+0000 7f28597fa640 1 -- 172.21.15.132:0/1725402339 <== mon.5 v1:172.21.15.132:6791/0 11 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["12"]}]=0 osd.12 does not exist. v250) v1 ==== 92+0+0 (unknown 189017566 0 0) 0x7f2854013280 con 0x7f285c13b2c0 2024-01-24T23:28:20.095 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.090+0000 7f28597fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["12"]}] 2024-01-24T23:28:20.095 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.090+0000 7f28597fa640 10 monclient: _finish_command 3 = system:0 osd.12 does not exist. 2024-01-24T23:28:20.095 INFO:tasks.workunit.client.0.smithi132.stderr:osd.12 does not exist. 2024-01-24T23:28:20.096 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.094+0000 7f2862e12640 1 -- 172.21.15.132:0/1725402339 >> v1:172.21.15.132:6812/1835401922 conn(0x7f2848036fa0 legacy=0x7f2848039460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:20.096 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.094+0000 7f2862e12640 10 monclient: shutdown 2024-01-24T23:28:20.096 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.094+0000 7f2862e12640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:20.096 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.094+0000 7f2862e12640 1 -- 172.21.15.132:0/1725402339 >> v1:172.21.15.132:6791/0 conn(0x7f285c13b2c0 legacy=0x7f285c1d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:20.096 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.094+0000 7f2862e12640 1 -- 172.21.15.132:0/1725402339 shutdown_connections 2024-01-24T23:28:20.096 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.094+0000 7f2862e12640 1 -- 172.21.15.132:0/1725402339 >> 172.21.15.132:0/1725402339 conn(0x7f285c0a5a10 msgr2=0x7f285c136c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:20.096 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.094+0000 7f2862e12640 1 -- 172.21.15.132:0/1725402339 shutdown_connections 2024-01-24T23:28:20.096 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.094+0000 7f2862e12640 1 -- 172.21.15.132:0/1725402339 wait complete. 2024-01-24T23:28:20.109 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd rm 12 2024-01-24T23:28:20.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8bcec3640 10 monclient: get_monmap_and_config 2024-01-24T23:28:20.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8bcec3640 10 monclient: build_initial_monmap 2024-01-24T23:28:20.223 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8bcec3640 10 monclient: monmap: 2024-01-24T23:28:20.223 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:20.223 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:20.223 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:20.226194+0000 2024-01-24T23:28:20.223 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:20.226194+0000 2024-01-24T23:28:20.223 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:20.223 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:20.224 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:20.224 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:20.224 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:20.224 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:20.224 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:20.224 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:20.224 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:20.224 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:20.224 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:20.224 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:20.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8bcec3640 1 Processor -- start 2024-01-24T23:28:20.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8bcec3640 1 -- start start 2024-01-24T23:28:20.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8bcec3640 10 monclient: init 2024-01-24T23:28:20.230 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8bcec3640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:20.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8bcec3640 10 monclient: _add_conns ranks=[1,0,3,4,7,5,2,8,6] 2024-01-24T23:28:20.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8bcec3640 10 monclient(hunting): picked mon.noname-f con 0x7fd8b8133b50 addr v1:172.21.15.132:6789/0 2024-01-24T23:28:20.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8bcec3640 10 monclient(hunting): picked mon.noname-a con 0x7fd8b809ec20 addr v1:172.21.15.115:6789/0 2024-01-24T23:28:20.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8bcec3640 10 monclient(hunting): picked mon.noname-g con 0x7fd8b813a1d0 addr v1:172.21.15.132:6790/0 2024-01-24T23:28:20.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8bcec3640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b8064700 con 0x7fd8b809ec20 2024-01-24T23:28:20.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8bcec3640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b809d9f0 con 0x7fd8b8133b50 2024-01-24T23:28:20.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8bcec3640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b809dc60 con 0x7fd8b813a1d0 2024-01-24T23:28:20.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8bcec3640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:20.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8bcec3640 10 monclient(hunting): authenticate will time out at 1387.550920s 2024-01-24T23:28:20.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b5d9b640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fd8b809ec20 legacy=0x7fd8b8138ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:28:20.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b659c640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fd8b8133b50 0x7fd8b8133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:58710/0 (socket says 172.21.15.132:58710) 2024-01-24T23:28:20.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b6d9d640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7fd8b813a1d0 0x7fd8b813c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:49618/0 (socket says 172.21.15.132:49618) 2024-01-24T23:28:20.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b659c640 1 -- 172.21.15.132:0/3280896687 learned_addr learned my addr 172.21.15.132:0/3280896687 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:20.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 1 -- 172.21.15.132:0/3280896687 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1525830026 0 0) 0x7fd8b809d9f0 con 0x7fd8b8133b50 2024-01-24T23:28:20.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 10 monclient(hunting): _negotiate 2024-01-24T23:28:20.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:20.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:20.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 10 monclient(hunting): my global_id is 124480 2024-01-24T23:28:20.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 1 -- 172.21.15.132:0/3280896687 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8a0003530 con 0x7fd8b8133b50 2024-01-24T23:28:20.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 1 -- 172.21.15.132:0/3280896687 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443879577 0 0) 0x7fd8b809dc60 con 0x7fd8b813a1d0 2024-01-24T23:28:20.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 10 monclient(hunting): _negotiate 2024-01-24T23:28:20.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:20.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:20.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 10 monclient(hunting): my global_id is 6960 2024-01-24T23:28:20.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 1 -- 172.21.15.132:0/3280896687 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8b809d9f0 con 0x7fd8b813a1d0 2024-01-24T23:28:20.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 1 -- 172.21.15.132:0/3280896687 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 655896178 0 0) 0x7fd8a0003530 con 0x7fd8b8133b50 2024-01-24T23:28:20.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 1 -- 172.21.15.132:0/3280896687 >> v1:172.21.15.132:6790/0 conn(0x7fd8b813a1d0 legacy=0x7fd8b813c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:20.232 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 1 -- 172.21.15.132:0/3280896687 >> v1:172.21.15.115:6789/0 conn(0x7fd8b809ec20 legacy=0x7fd8b8138ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:28:20.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 10 monclient: _finish_hunting 0 2024-01-24T23:28:20.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 1 monclient: found mon.noname-f 2024-01-24T23:28:20.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:20.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:28:20.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 1 -- 172.21.15.132:0/3280896687 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b809df40 con 0x7fd8b8133b50 2024-01-24T23:28:20.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 10 monclient: _finish_auth 0 2024-01-24T23:28:20.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 10 monclient: _check_auth_tickets 2024-01-24T23:28:20.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:20.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8bcec3640 5 monclient: authenticate success, global_id 124480 2024-01-24T23:28:20.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8bcec3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:20.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 1 -- 172.21.15.132:0/3280896687 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd8a80017c0 con 0x7fd8b8133b50 2024-01-24T23:28:20.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:20.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:28:20.233 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 10 monclient: dump: 2024-01-24T23:28:20.234 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:20.234 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:20.234 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:20.234 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:20.234 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:20.234 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:20.234 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:20.234 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:20.234 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:20.234 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:20.234 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:20.234 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:20.234 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:20.234 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:20.234 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:20.235 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:20.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8bcec3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:20.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 1 -- 172.21.15.132:0/3280896687 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd8a8001a20 con 0x7fd8b8133b50 2024-01-24T23:28:20.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:20.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8bcec3640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:20.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8bcec3640 4 set_mon_vals no callback set 2024-01-24T23:28:20.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 1 -- 172.21.15.132:0/3280896687 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd8a8004bf0 con 0x7fd8b8133b50 2024-01-24T23:28:20.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:20.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:28:20.235 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.222+0000 7fd8b559a640 10 monclient: dump: 2024-01-24T23:28:20.235 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:20.235 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:20.235 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:20.235 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:20.235 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:20.235 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:20.236 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:20.236 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:20.236 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:20.236 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:20.236 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:20.236 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:20.236 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:20.236 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:20.236 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:20.236 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:20.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 10 monclient: shutdown 2024-01-24T23:28:20.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:20.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 1 -- 172.21.15.132:0/3280896687 >> v1:172.21.15.132:6789/0 conn(0x7fd8b8133b50 legacy=0x7fd8b8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:20.236 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 1 -- 172.21.15.132:0/3280896687 shutdown_connections 2024-01-24T23:28:20.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 1 -- 172.21.15.132:0/3280896687 >> 172.21.15.132:0/3280896687 conn(0x7fd8b80a5a10 msgr2=0x7fd8b80a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:20.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 1 -- 172.21.15.132:0/3280896687 shutdown_connections 2024-01-24T23:28:20.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 1 -- 172.21.15.132:0/3280896687 wait complete. 2024-01-24T23:28:20.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 10 monclient: build_initial_monmap 2024-01-24T23:28:20.237 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 10 monclient: monmap: 2024-01-24T23:28:20.237 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:20.237 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:20.237 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:20.237 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:20.237 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:20.237 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:20.237 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:20.237 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:20.237 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:20.237 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:20.238 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:20.238 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:20.238 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:20.238 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:20.238 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:20.238 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:20.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 1 Processor -- start 2024-01-24T23:28:20.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 1 -- start start 2024-01-24T23:28:20.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 10 monclient: init 2024-01-24T23:28:20.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:20.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 10 monclient: _add_conns ranks=[2,8,1,4,0,7,6,3,5] 2024-01-24T23:28:20.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 10 monclient(hunting): picked mon.noname-b con 0x7fd8b809ec20 addr v1:172.21.15.115:6790/0 2024-01-24T23:28:20.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 10 monclient(hunting): picked mon.noname-e con 0x7fd8b8133b50 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:20.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 10 monclient(hunting): picked mon.noname-f con 0x7fd8b813a1d0 addr v1:172.21.15.132:6789/0 2024-01-24T23:28:20.238 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b8133920 con 0x7fd8b813a1d0 2024-01-24T23:28:20.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b806f520 con 0x7fd8b809ec20 2024-01-24T23:28:20.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b8071880 con 0x7fd8b8133b50 2024-01-24T23:28:20.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:20.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 10 monclient(hunting): authenticate will time out at 1387.554176s 2024-01-24T23:28:20.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8b6d9d640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fd8b813a1d0 0x7fd8b81d3fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:58712/0 (socket says 172.21.15.132:58712) 2024-01-24T23:28:20.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8b6d9d640 1 -- 172.21.15.132:0/219459903 learned_addr learned my addr 172.21.15.132:0/219459903 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:20.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8b659c640 1 -- 172.21.15.132:0/219459903 >> v1:172.21.15.115:6790/0 conn(0x7fd8b809ec20 legacy=0x7fd8b812bf60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:28:20.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 1 -- 172.21.15.132:0/219459903 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3891741479 0 0) 0x7fd8b8133920 con 0x7fd8b813a1d0 2024-01-24T23:28:20.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:28:20.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:20.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:20.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 10 monclient(hunting): my global_id is 124489 2024-01-24T23:28:20.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 1 -- 172.21.15.132:0/219459903 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd898003530 con 0x7fd8b813a1d0 2024-01-24T23:28:20.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 1 -- 172.21.15.132:0/219459903 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 699030620 0 0) 0x7fd898003530 con 0x7fd8b813a1d0 2024-01-24T23:28:20.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 1 -- 172.21.15.132:0/219459903 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd8b8133920 con 0x7fd8b813a1d0 2024-01-24T23:28:20.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 1 -- 172.21.15.132:0/219459903 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd8b00018e0 con 0x7fd8b813a1d0 2024-01-24T23:28:20.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:20.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:28:20.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 10 monclient(hunting): dump: 2024-01-24T23:28:20.240 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:20.240 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:20.240 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:20.240 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:20.240 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:20.240 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:20.240 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:20.240 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:20.240 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:20.240 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:20.241 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:20.241 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:20.241 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:20.241 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:20.241 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:20.241 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:20.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:20.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 1 -- 172.21.15.132:0/219459903 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 74290137 0 0) 0x7fd8b8133920 con 0x7fd8b813a1d0 2024-01-24T23:28:20.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 1 -- 172.21.15.132:0/219459903 >> v1:172.21.15.115:6793/0 conn(0x7fd8b8133b50 legacy=0x7fd8b81d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:20.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 1 -- 172.21.15.132:0/219459903 >> v1:172.21.15.115:6790/0 conn(0x7fd8b809ec20 legacy=0x7fd8b812bf60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:28:20.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:28:20.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 1 monclient: found mon.f 2024-01-24T23:28:20.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:20.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:20.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 1 -- 172.21.15.132:0/219459903 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b8070fe0 con 0x7fd8b813a1d0 2024-01-24T23:28:20.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 10 monclient: _finish_auth 0 2024-01-24T23:28:20.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:28:20.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:20.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 5 monclient: authenticate success, global_id 124489 2024-01-24T23:28:20.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 10 monclient: _renew_subs 2024-01-24T23:28:20.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:20.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 1 -- 172.21.15.132:0/219459903 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b80712c0 con 0x7fd8b813a1d0 2024-01-24T23:28:20.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 10 monclient: _renew_subs 2024-01-24T23:28:20.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:20.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 1 -- 172.21.15.132:0/219459903 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b8071e70 con 0x7fd8b813a1d0 2024-01-24T23:28:20.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 1 -- 172.21.15.132:0/219459903 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd8b0004e20 con 0x7fd8b813a1d0 2024-01-24T23:28:20.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:20.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 1 -- 172.21.15.132:0/219459903 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd8b00052d0 con 0x7fd8b813a1d0 2024-01-24T23:28:20.242 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:20.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8b4d99640 4 set_mon_vals no callback set 2024-01-24T23:28:20.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:28:20.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 10 monclient: dump: 2024-01-24T23:28:20.243 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:20.243 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:20.243 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:20.243 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:20.243 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:20.243 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:20.243 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:20.243 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:20.243 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:20.243 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:20.243 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:20.243 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:20.243 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:20.244 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:20.244 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:20.244 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:20.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:20.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:20.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:20.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8bcec3640 1 -- 172.21.15.132:0/219459903 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd89c005090 con 0x7fd8b813a1d0 2024-01-24T23:28:20.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.226+0000 7fd8a6ffd640 1 -- 172.21.15.132:0/219459903 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fd8b000fd50 con 0x7fd8b813a1d0 2024-01-24T23:28:20.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.230+0000 7fd8a6ffd640 1 -- 172.21.15.132:0/219459903 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(250..250 src has 1..250) v4 ==== 7051+0+0 (unknown 2177196178 0 0) 0x7fd8b0044880 con 0x7fd8b813a1d0 2024-01-24T23:28:20.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.230+0000 7fd8b659c640 10 monclient: get_auth_request con 0x7fd898036fb0 auth_method 0 2024-01-24T23:28:20.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.234+0000 7fd8a6ffd640 1 -- 172.21.15.132:0/219459903 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fd8b00117a0 con 0x7fd8b813a1d0 2024-01-24T23:28:20.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.234+0000 7fd8a6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:20.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.234+0000 7fd8a6ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:20.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.394+0000 7fd8bcec3640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["12"]}] 2024-01-24T23:28:20.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.394+0000 7fd8bcec3640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["12"]}] 2024-01-24T23:28:20.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.394+0000 7fd8bcec3640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:20.397 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.394+0000 7fd8bcec3640 1 -- 172.21.15.132:0/219459903 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd rm", "ids": ["12"]} v 0) v1 -- 0x7fd89c0058c0 con 0x7fd8b813a1d0 2024-01-24T23:28:20.398 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:28:20.398+0000 7f48d6072640 -1 mon.h@5(leader).osd e250 definitely_dead 0 2024-01-24T23:28:20.398 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.398+0000 7fd8a6ffd640 1 -- 172.21.15.132:0/219459903 <== mon.1 v1:172.21.15.132:6789/0 10 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["12"]}]=0 osd.12 does not exist. v250) v1 ==== 92+0+0 (unknown 189017566 0 0) 0x7fd8b0014a10 con 0x7fd8b813a1d0 2024-01-24T23:28:20.398 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.398+0000 7fd8a6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["12"]}] 2024-01-24T23:28:20.398 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.398+0000 7fd8a6ffd640 10 monclient: _finish_command 2 = system:0 osd.12 does not exist. 2024-01-24T23:28:20.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.502+0000 7fd8bcec3640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["12"]}] 2024-01-24T23:28:20.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.502+0000 7fd8bcec3640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["12"]}] 2024-01-24T23:28:20.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.502+0000 7fd8bcec3640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:20.505 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.502+0000 7fd8bcec3640 1 -- 172.21.15.132:0/219459903 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd rm", "ids": ["12"]} v 0) v1 -- 0x7fd89c000bd0 con 0x7fd8b813a1d0 2024-01-24T23:28:20.505 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:28:20.506+0000 7f48d6072640 -1 mon.h@5(leader).osd e250 definitely_dead 0 2024-01-24T23:28:20.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.506+0000 7fd8a6ffd640 1 -- 172.21.15.132:0/219459903 <== mon.1 v1:172.21.15.132:6789/0 11 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["12"]}]=0 osd.12 does not exist. v250) v1 ==== 92+0+0 (unknown 189017566 0 0) 0x7fd8b0014bf0 con 0x7fd8b813a1d0 2024-01-24T23:28:20.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.506+0000 7fd8a6ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["12"]}] 2024-01-24T23:28:20.506 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.506+0000 7fd8a6ffd640 10 monclient: _finish_command 3 = system:0 osd.12 does not exist. 2024-01-24T23:28:20.506 INFO:tasks.workunit.client.0.smithi132.stderr:osd.12 does not exist. 2024-01-24T23:28:20.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.506+0000 7fd8bcec3640 1 -- 172.21.15.132:0/219459903 >> v1:172.21.15.132:6812/1835401922 conn(0x7fd898036fb0 legacy=0x7fd898039470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:20.508 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.506+0000 7fd8bcec3640 10 monclient: shutdown 2024-01-24T23:28:20.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.506+0000 7fd8bcec3640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:20.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.506+0000 7fd8bcec3640 1 -- 172.21.15.132:0/219459903 >> v1:172.21.15.132:6789/0 conn(0x7fd8b813a1d0 legacy=0x7fd8b81d3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:20.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.506+0000 7fd8bcec3640 1 -- 172.21.15.132:0/219459903 shutdown_connections 2024-01-24T23:28:20.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.506+0000 7fd8bcec3640 1 -- 172.21.15.132:0/219459903 >> 172.21.15.132:0/219459903 conn(0x7fd8b80a5a10 msgr2=0x7fd8b809f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:20.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.506+0000 7fd8bcec3640 1 -- 172.21.15.132:0/219459903 shutdown_connections 2024-01-24T23:28:20.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.506+0000 7fd8bcec3640 1 -- 172.21.15.132:0/219459903 wait complete. 2024-01-24T23:28:20.523 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd rm 14 2024-01-24T23:28:20.634 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.634+0000 7f58c4096640 10 monclient: get_monmap_and_config 2024-01-24T23:28:20.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.634+0000 7f58c4096640 10 monclient: build_initial_monmap 2024-01-24T23:28:20.635 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.634+0000 7f58c4096640 10 monclient: monmap: 2024-01-24T23:28:20.635 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:20.635 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:20.635 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:20.638000+0000 2024-01-24T23:28:20.635 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:20.638000+0000 2024-01-24T23:28:20.635 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:20.635 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:20.635 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:20.635 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:20.635 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:20.635 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:20.635 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:20.636 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:20.636 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:20.636 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:20.636 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:20.636 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:20.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.634+0000 7f58c4096640 1 Processor -- start 2024-01-24T23:28:20.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.634+0000 7f58c4096640 1 -- start start 2024-01-24T23:28:20.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.634+0000 7f58c4096640 10 monclient: init 2024-01-24T23:28:20.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.634+0000 7f58c4096640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:20.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.634+0000 7f58c4096640 10 monclient: _add_conns ranks=[8,3,4,2,1,6,7,0,5] 2024-01-24T23:28:20.642 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.634+0000 7f58c4096640 10 monclient(hunting): picked mon.noname-e con 0x7f58bc071400 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:20.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.634+0000 7f58c4096640 10 monclient(hunting): picked mon.noname-g con 0x7f58bc09f640 addr v1:172.21.15.132:6790/0 2024-01-24T23:28:20.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.634+0000 7f58c4096640 10 monclient(hunting): picked mon.noname-c con 0x7f58bc136f00 addr v1:172.21.15.115:6791/0 2024-01-24T23:28:20.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.634+0000 7f58c4096640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58bc064700 con 0x7f58bc09f640 2024-01-24T23:28:20.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.634+0000 7f58c4096640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58bc071ef0 con 0x7f58bc136f00 2024-01-24T23:28:20.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.634+0000 7f58c4096640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58bc06d910 con 0x7f58bc071400 2024-01-24T23:28:20.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.634+0000 7f58c4096640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:20.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.634+0000 7f58c4096640 10 monclient(hunting): authenticate will time out at 1387.963489s 2024-01-24T23:28:20.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.634+0000 7f58c1631640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f58bc09f640 0x7f58bc13bdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:49620/0 (socket says 172.21.15.132:49620) 2024-01-24T23:28:20.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.634+0000 7f58c1631640 1 -- 172.21.15.132:0/1596077220 learned_addr learned my addr 172.21.15.132:0/1596077220 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:20.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.634+0000 7f58c0e30640 1 -- 172.21.15.132:0/1596077220 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1446752484 0 0) 0x7f58bc064700 con 0x7f58bc09f640 2024-01-24T23:28:20.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.634+0000 7f58c0e30640 10 monclient(hunting): _negotiate 2024-01-24T23:28:20.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.634+0000 7f58c0e30640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:20.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.634+0000 7f58c0e30640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:20.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.634+0000 7f58c0e30640 10 monclient(hunting): my global_id is 6969 2024-01-24T23:28:20.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.634+0000 7f58c0e30640 1 -- 172.21.15.132:0/1596077220 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58a8003530 con 0x7f58bc09f640 2024-01-24T23:28:20.643 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c0e30640 1 -- 172.21.15.132:0/1596077220 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2520407744 0 0) 0x7f58a8003530 con 0x7f58bc09f640 2024-01-24T23:28:20.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c0e30640 1 -- 172.21.15.132:0/1596077220 >> v1:172.21.15.115:6793/0 conn(0x7f58bc071400 legacy=0x7f58bc0717e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:20.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c0e30640 1 -- 172.21.15.132:0/1596077220 >> v1:172.21.15.115:6791/0 conn(0x7f58bc136f00 legacy=0x7f58bc13f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:20.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c0e30640 10 monclient: _finish_hunting 0 2024-01-24T23:28:20.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c0e30640 1 monclient: found mon.noname-g 2024-01-24T23:28:20.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c0e30640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:20.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c0e30640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:28:20.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c0e30640 1 -- 172.21.15.132:0/1596077220 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc06dbf0 con 0x7f58bc09f640 2024-01-24T23:28:20.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c0e30640 10 monclient: _finish_auth 0 2024-01-24T23:28:20.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c0e30640 10 monclient: _check_auth_tickets 2024-01-24T23:28:20.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c0e30640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:20.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c0e30640 1 -- 172.21.15.132:0/1596077220 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f58ac0017c0 con 0x7f58bc09f640 2024-01-24T23:28:20.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c0e30640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:20.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c0e30640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:28:20.644 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c0e30640 10 monclient: dump: 2024-01-24T23:28:20.645 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:20.645 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:20.645 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:20.645 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:20.645 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:20.645 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:20.645 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:20.645 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:20.645 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:20.645 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:20.645 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:20.645 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:20.645 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:20.645 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:20.645 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:20.646 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:20.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 5 monclient: authenticate success, global_id 6969 2024-01-24T23:28:20.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:20.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c0e30640 1 -- 172.21.15.132:0/1596077220 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f58ac001ae0 con 0x7f58bc09f640 2024-01-24T23:28:20.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c0e30640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:20.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:20.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c0e30640 1 -- 172.21.15.132:0/1596077220 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f58ac004bf0 con 0x7f58bc09f640 2024-01-24T23:28:20.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c0e30640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:20.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 4 set_mon_vals no callback set 2024-01-24T23:28:20.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c0e30640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:28:20.646 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c0e30640 10 monclient: dump: 2024-01-24T23:28:20.646 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:20.646 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:20.646 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:20.646 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:20.646 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:20.647 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:20.647 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:20.647 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:20.647 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:20.647 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:20.647 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:20.647 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:20.647 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:20.647 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:20.647 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:20.647 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:20.647 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 10 monclient: shutdown 2024-01-24T23:28:20.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:20.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 1 -- 172.21.15.132:0/1596077220 >> v1:172.21.15.132:6790/0 conn(0x7f58bc09f640 legacy=0x7f58bc13bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:20.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 1 -- 172.21.15.132:0/1596077220 shutdown_connections 2024-01-24T23:28:20.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 1 -- 172.21.15.132:0/1596077220 >> 172.21.15.132:0/1596077220 conn(0x7f58bc0a22a0 msgr2=0x7f58bc0a2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:20.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 1 -- 172.21.15.132:0/1596077220 shutdown_connections 2024-01-24T23:28:20.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 1 -- 172.21.15.132:0/1596077220 wait complete. 2024-01-24T23:28:20.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 10 monclient: build_initial_monmap 2024-01-24T23:28:20.648 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 10 monclient: monmap: 2024-01-24T23:28:20.648 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:20.648 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:20.648 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:20.648 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:20.648 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:20.648 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:20.648 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:20.649 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:20.649 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:20.649 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:20.649 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:20.649 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:20.649 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:20.649 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:20.649 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:20.649 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:20.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 1 Processor -- start 2024-01-24T23:28:20.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 1 -- start start 2024-01-24T23:28:20.649 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 10 monclient: init 2024-01-24T23:28:20.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:20.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 10 monclient: _add_conns ranks=[2,6,3,7,5,4,8,0,1] 2024-01-24T23:28:20.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 10 monclient(hunting): picked mon.noname-b con 0x7f58bc071400 addr v1:172.21.15.115:6790/0 2024-01-24T23:28:20.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 10 monclient(hunting): picked mon.noname-d con 0x7f58bc09f640 addr v1:172.21.15.115:6792/0 2024-01-24T23:28:20.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 10 monclient(hunting): picked mon.noname-g con 0x7f58bc136f00 addr v1:172.21.15.132:6790/0 2024-01-24T23:28:20.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58bc0a31a0 con 0x7f58bc071400 2024-01-24T23:28:20.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58bc12e9c0 con 0x7f58bc136f00 2024-01-24T23:28:20.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58bc131190 con 0x7f58bc09f640 2024-01-24T23:28:20.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:20.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 10 monclient(hunting): authenticate will time out at 1387.966281s 2024-01-24T23:28:20.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c1e32640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7f58bc071400 legacy=0x7f58bc130740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:28:20.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c2633640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f58bc136f00 0x7f58bc1d34e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:49636/0 (socket says 172.21.15.132:49636) 2024-01-24T23:28:20.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c2633640 1 -- 172.21.15.132:0/1668628464 learned_addr learned my addr 172.21.15.132:0/1668628464 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:20.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 1 -- 172.21.15.132:0/1668628464 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1671268179 0 0) 0x7f58bc12e9c0 con 0x7f58bc136f00 2024-01-24T23:28:20.650 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 10 monclient(hunting): _negotiate 2024-01-24T23:28:20.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:20.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:20.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 10 monclient(hunting): my global_id is 6978 2024-01-24T23:28:20.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 1 -- 172.21.15.132:0/1668628464 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58a0003530 con 0x7f58bc136f00 2024-01-24T23:28:20.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 1 -- 172.21.15.132:0/1668628464 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 603141127 0 0) 0x7f58a0003530 con 0x7f58bc136f00 2024-01-24T23:28:20.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 1 -- 172.21.15.132:0/1668628464 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f58bc12e9c0 con 0x7f58bc136f00 2024-01-24T23:28:20.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 1 -- 172.21.15.132:0/1668628464 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f58b4001cc0 con 0x7f58bc136f00 2024-01-24T23:28:20.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:20.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:28:20.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 10 monclient(hunting): dump: 2024-01-24T23:28:20.651 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:20.651 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:20.651 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:20.651 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:20.651 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:20.651 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:20.652 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:20.652 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:20.652 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:20.652 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:20.652 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:20.652 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:20.652 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:20.652 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:20.652 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:20.652 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:20.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:20.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 1 -- 172.21.15.132:0/1668628464 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 814619371 0 0) 0x7f58bc131190 con 0x7f58bc09f640 2024-01-24T23:28:20.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 10 monclient(hunting): _negotiate 2024-01-24T23:28:20.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:20.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:20.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 10 monclient(hunting): my global_id is 7197 2024-01-24T23:28:20.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 1 -- 172.21.15.132:0/1668628464 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58a0003530 con 0x7f58bc09f640 2024-01-24T23:28:20.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 1 -- 172.21.15.132:0/1668628464 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2093712538 0 0) 0x7f58bc12e9c0 con 0x7f58bc136f00 2024-01-24T23:28:20.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 1 -- 172.21.15.132:0/1668628464 >> v1:172.21.15.115:6792/0 conn(0x7f58bc09f640 legacy=0x7f58bc12a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:20.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 1 -- 172.21.15.132:0/1668628464 >> v1:172.21.15.115:6790/0 conn(0x7f58bc071400 legacy=0x7f58bc130740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:28:20.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 10 monclient: _finish_hunting 0 2024-01-24T23:28:20.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 1 monclient: found mon.g 2024-01-24T23:28:20.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:20.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:20.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 1 -- 172.21.15.132:0/1668628464 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc09fa90 con 0x7f58bc136f00 2024-01-24T23:28:20.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 10 monclient: _finish_auth 0 2024-01-24T23:28:20.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 10 monclient: _check_auth_tickets 2024-01-24T23:28:20.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:20.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 5 monclient: authenticate success, global_id 6978 2024-01-24T23:28:20.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 10 monclient: _renew_subs 2024-01-24T23:28:20.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:20.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 1 -- 172.21.15.132:0/1668628464 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58bc12dcd0 con 0x7f58bc136f00 2024-01-24T23:28:20.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 10 monclient: _renew_subs 2024-01-24T23:28:20.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:20.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 1 -- 172.21.15.132:0/1668628464 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f58bc12fa00 con 0x7f58bc136f00 2024-01-24T23:28:20.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 1 -- 172.21.15.132:0/1668628464 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f58b4002090 con 0x7f58bc136f00 2024-01-24T23:28:20.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:20.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 1 -- 172.21.15.132:0/1668628464 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f58b4004010 con 0x7f58bc136f00 2024-01-24T23:28:20.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:20.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b3fff640 4 set_mon_vals no callback set 2024-01-24T23:28:20.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:28:20.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 10 monclient: dump: 2024-01-24T23:28:20.654 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:20.654 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:20.654 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:20.654 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:20.655 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:20.655 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:20.655 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:20.655 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:20.655 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:20.655 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:20.655 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:20.655 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:20.655 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:20.655 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:20.655 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:20.655 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:20.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58b27fc640 1 -- 172.21.15.132:0/1668628464 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f58b4004170 con 0x7f58bc136f00 2024-01-24T23:28:20.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:20.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:20.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:20.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.638+0000 7f58c4096640 1 -- 172.21.15.132:0/1668628464 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a4005090 con 0x7f58bc136f00 2024-01-24T23:28:20.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.642+0000 7f58b27fc640 1 -- 172.21.15.132:0/1668628464 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(250..250 src has 1..250) v4 ==== 7051+0+0 (unknown 2177196178 0 0) 0x7f58b40447b0 con 0x7f58bc136f00 2024-01-24T23:28:20.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.642+0000 7f58c1e32640 10 monclient: get_auth_request con 0x7f58a0036fb0 auth_method 0 2024-01-24T23:28:20.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.646+0000 7f58b27fc640 1 -- 172.21.15.132:0/1668628464 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f58b4014b20 con 0x7f58bc136f00 2024-01-24T23:28:20.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.646+0000 7f58b27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:20.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.646+0000 7f58b27fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:20.796 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.794+0000 7f58c4096640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["14"]}] 2024-01-24T23:28:20.797 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.794+0000 7f58c4096640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["14"]}] 2024-01-24T23:28:20.797 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.794+0000 7f58c4096640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:20.797 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.794+0000 7f58c4096640 1 -- 172.21.15.132:0/1668628464 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd rm", "ids": ["14"]} v 0) v1 -- 0x7f58a40058c0 con 0x7f58bc136f00 2024-01-24T23:28:20.797 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:28:20.798+0000 7f48d6072640 -1 mon.h@5(leader).osd e250 definitely_dead 0 2024-01-24T23:28:20.997 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.994+0000 7f58b27fc640 1 -- 172.21.15.132:0/1668628464 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["14"]}]=0 removed osd.14 v251) v1 ==== 83+0+0 (unknown 3894252619 0 0) 0x7f58b40131c0 con 0x7f58bc136f00 2024-01-24T23:28:20.997 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.994+0000 7f58b27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["14"]}] 2024-01-24T23:28:20.997 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:20.994+0000 7f58b27fc640 10 monclient: _finish_command 2 = system:0 removed osd.14 2024-01-24T23:28:21.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.102+0000 7f58c4096640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["14"]}] 2024-01-24T23:28:21.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.102+0000 7f58c4096640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["14"]}] 2024-01-24T23:28:21.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.102+0000 7f58c4096640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:21.102 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.102+0000 7f58c4096640 1 -- 172.21.15.132:0/1668628464 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd rm", "ids": ["14"]} v 0) v1 -- 0x7f58a4000bd0 con 0x7f58bc136f00 2024-01-24T23:28:21.103 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:28:21.102+0000 7f48d6072640 -1 mon.h@5(leader).osd e251 definitely_dead 0 2024-01-24T23:28:21.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.102+0000 7f58b27fc640 1 -- 172.21.15.132:0/1668628464 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["14"]}]=0 osd.14 does not exist. v251) v1 ==== 92+0+0 (unknown 1164467084 0 0) 0x7f58b4014940 con 0x7f58bc136f00 2024-01-24T23:28:21.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.102+0000 7f58b27fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["14"]}] 2024-01-24T23:28:21.103 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.102+0000 7f58b27fc640 10 monclient: _finish_command 3 = system:0 osd.14 does not exist. 2024-01-24T23:28:21.104 INFO:tasks.workunit.client.0.smithi132.stderr:osd.14 does not exist. 2024-01-24T23:28:21.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.106+0000 7f58c4096640 1 -- 172.21.15.132:0/1668628464 >> v1:172.21.15.132:6812/1835401922 conn(0x7f58a0036fb0 legacy=0x7f58a0039470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:21.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.106+0000 7f58c4096640 10 monclient: shutdown 2024-01-24T23:28:21.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.106+0000 7f58c4096640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:21.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.106+0000 7f58c4096640 1 -- 172.21.15.132:0/1668628464 >> v1:172.21.15.132:6790/0 conn(0x7f58bc136f00 legacy=0x7f58bc1d34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:21.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.106+0000 7f58c4096640 1 -- 172.21.15.132:0/1668628464 shutdown_connections 2024-01-24T23:28:21.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.106+0000 7f58c4096640 1 -- 172.21.15.132:0/1668628464 >> 172.21.15.132:0/1668628464 conn(0x7f58bc0a22a0 msgr2=0x7f58bc070390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:21.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.106+0000 7f58c4096640 1 -- 172.21.15.132:0/1668628464 shutdown_connections 2024-01-24T23:28:21.106 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.106+0000 7f58c4096640 1 -- 172.21.15.132:0/1668628464 wait complete. 2024-01-24T23:28:21.120 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd rm 1000 2024-01-24T23:28:21.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd309d54640 10 monclient: get_monmap_and_config 2024-01-24T23:28:21.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd309d54640 10 monclient: build_initial_monmap 2024-01-24T23:28:21.231 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd309d54640 10 monclient: monmap: 2024-01-24T23:28:21.231 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:21.231 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:21.231 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:21.234188+0000 2024-01-24T23:28:21.231 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:21.234188+0000 2024-01-24T23:28:21.231 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:21.231 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:21.231 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:21.232 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:21.232 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:21.232 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:21.232 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:21.232 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:21.232 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:21.232 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:21.232 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:21.232 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:21.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd309d54640 1 Processor -- start 2024-01-24T23:28:21.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd309d54640 1 -- start start 2024-01-24T23:28:21.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd309d54640 10 monclient: init 2024-01-24T23:28:21.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd309d54640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:21.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd309d54640 10 monclient: _add_conns ranks=[4,7,1,8,3,6,2,5,0] 2024-01-24T23:28:21.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd309d54640 10 monclient(hunting): picked mon.noname-c con 0x7fd304135e10 addr v1:172.21.15.115:6791/0 2024-01-24T23:28:21.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd309d54640 10 monclient(hunting): picked mon.noname-i con 0x7fd3041361f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:28:21.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd309d54640 10 monclient(hunting): picked mon.noname-f con 0x7fd3041365f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:28:21.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd309d54640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd304064700 con 0x7fd3041365f0 2024-01-24T23:28:21.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd309d54640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30409c3e0 con 0x7fd304135e10 2024-01-24T23:28:21.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd309d54640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3040a4f90 con 0x7fd3041361f0 2024-01-24T23:28:21.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd309d54640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:21.239 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd309d54640 10 monclient(hunting): authenticate will time out at 1388.558851s 2024-01-24T23:28:21.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd302ffd640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7fd3041361f0 legacy=0x7fd30413bed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:28:21.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd303fff640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fd3041365f0 0x7fd30413f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:58718/0 (socket says 172.21.15.132:58718) 2024-01-24T23:28:21.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd303fff640 1 -- 172.21.15.132:0/1897253549 learned_addr learned my addr 172.21.15.132:0/1897253549 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:21.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 1 -- 172.21.15.132:0/1897253549 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3318317985 0 0) 0x7fd304064700 con 0x7fd3041365f0 2024-01-24T23:28:21.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 10 monclient(hunting): _negotiate 2024-01-24T23:28:21.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:21.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:21.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 10 monclient(hunting): my global_id is 124498 2024-01-24T23:28:21.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 1 -- 172.21.15.132:0/1897253549 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2f0003530 con 0x7fd3041365f0 2024-01-24T23:28:21.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 1 -- 172.21.15.132:0/1897253549 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3642840724 0 0) 0x7fd2f0003530 con 0x7fd3041365f0 2024-01-24T23:28:21.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 1 -- 172.21.15.132:0/1897253549 >> v1:172.21.15.132:6792/0 conn(0x7fd3041361f0 legacy=0x7fd30413bed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:28:21.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 1 -- 172.21.15.132:0/1897253549 >> v1:172.21.15.115:6791/0 conn(0x7fd304135e10 legacy=0x7fd30409bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:21.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 10 monclient: _finish_hunting 0 2024-01-24T23:28:21.240 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 1 monclient: found mon.noname-f 2024-01-24T23:28:21.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:21.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:28:21.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 1 -- 172.21.15.132:0/1897253549 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30409b010 con 0x7fd3041365f0 2024-01-24T23:28:21.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 10 monclient: _finish_auth 0 2024-01-24T23:28:21.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 10 monclient: _check_auth_tickets 2024-01-24T23:28:21.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:21.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 1 -- 172.21.15.132:0/1897253549 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd2fc001b40 con 0x7fd3041365f0 2024-01-24T23:28:21.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd309d54640 5 monclient: authenticate success, global_id 124498 2024-01-24T23:28:21.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd309d54640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:21.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:21.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:28:21.241 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 10 monclient: dump: 2024-01-24T23:28:21.241 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:21.241 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:21.241 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:21.242 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:21.242 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:21.242 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:21.242 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:21.242 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:21.242 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:21.242 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:21.242 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:21.242 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:21.242 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:21.242 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:21.242 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:21.242 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:21.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd309d54640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:21.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 1 -- 172.21.15.132:0/1897253549 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd2fc001e60 con 0x7fd3041365f0 2024-01-24T23:28:21.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:21.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd309d54640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:21.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd309d54640 4 set_mon_vals no callback set 2024-01-24T23:28:21.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 1 -- 172.21.15.132:0/1897253549 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd2fc004f70 con 0x7fd3041365f0 2024-01-24T23:28:21.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:21.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:28:21.243 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.230+0000 7fd3027fc640 10 monclient: dump: 2024-01-24T23:28:21.243 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:21.243 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:21.243 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:21.243 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:21.243 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:21.244 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:21.244 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:21.244 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:21.244 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:21.244 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:21.244 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:21.244 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:21.244 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:21.244 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:21.244 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:21.244 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:21.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 10 monclient: shutdown 2024-01-24T23:28:21.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:21.244 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 1 -- 172.21.15.132:0/1897253549 >> v1:172.21.15.132:6789/0 conn(0x7fd3041365f0 legacy=0x7fd30413f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:21.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 1 -- 172.21.15.132:0/1897253549 shutdown_connections 2024-01-24T23:28:21.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 1 -- 172.21.15.132:0/1897253549 >> 172.21.15.132:0/1897253549 conn(0x7fd3040a6790 msgr2=0x7fd304071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:21.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 1 -- 172.21.15.132:0/1897253549 shutdown_connections 2024-01-24T23:28:21.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 1 -- 172.21.15.132:0/1897253549 wait complete. 2024-01-24T23:28:21.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 10 monclient: build_initial_monmap 2024-01-24T23:28:21.245 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 10 monclient: monmap: 2024-01-24T23:28:21.245 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:21.245 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:21.245 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:21.245 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:21.245 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:21.245 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:21.245 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:21.245 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:21.245 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:21.245 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:21.245 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:21.246 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:21.246 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:21.246 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:21.246 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:21.246 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:21.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 1 Processor -- start 2024-01-24T23:28:21.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 1 -- start start 2024-01-24T23:28:21.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 10 monclient: init 2024-01-24T23:28:21.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:21.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 10 monclient: _add_conns ranks=[5,3,1,8,4,7,2,0,6] 2024-01-24T23:28:21.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 10 monclient(hunting): picked mon.noname-h con 0x7fd304135e10 addr v1:172.21.15.132:6791/0 2024-01-24T23:28:21.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 10 monclient(hunting): picked mon.noname-g con 0x7fd3041361f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:28:21.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 10 monclient(hunting): picked mon.noname-f con 0x7fd3041365f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:28:21.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30409c6b0 con 0x7fd3041365f0 2024-01-24T23:28:21.246 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30409fd60 con 0x7fd3041361f0 2024-01-24T23:28:21.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd304121e00 con 0x7fd304135e10 2024-01-24T23:28:21.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:21.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 10 monclient(hunting): authenticate will time out at 1388.561863s 2024-01-24T23:28:21.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd3037fe640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7fd304135e10 0x7fd304134ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.132:49338/0 (socket says 172.21.15.132:49338) 2024-01-24T23:28:21.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd3037fe640 1 -- 172.21.15.132:0/1383373167 learned_addr learned my addr 172.21.15.132:0/1383373167 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:21.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126886840 0 0) 0x7fd30409c6b0 con 0x7fd3041365f0 2024-01-24T23:28:21.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient(hunting): _negotiate 2024-01-24T23:28:21.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:21.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:21.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient(hunting): my global_id is 124507 2024-01-24T23:28:21.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2e4003530 con 0x7fd3041365f0 2024-01-24T23:28:21.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2692946786 0 0) 0x7fd304121e00 con 0x7fd304135e10 2024-01-24T23:28:21.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient(hunting): _negotiate 2024-01-24T23:28:21.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:21.247 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:21.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient(hunting): my global_id is 6962 2024-01-24T23:28:21.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd30409c6b0 con 0x7fd304135e10 2024-01-24T23:28:21.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1451206313 0 0) 0x7fd30409fd60 con 0x7fd3041361f0 2024-01-24T23:28:21.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient(hunting): _negotiate 2024-01-24T23:28:21.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:21.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:21.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient(hunting): my global_id is 6987 2024-01-24T23:28:21.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd304121e00 con 0x7fd3041361f0 2024-01-24T23:28:21.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1192981344 0 0) 0x7fd304121e00 con 0x7fd3041361f0 2024-01-24T23:28:21.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd30409fd60 con 0x7fd3041361f0 2024-01-24T23:28:21.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3960736141 0 0) 0x7fd2e4003530 con 0x7fd3041365f0 2024-01-24T23:28:21.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd304121e00 con 0x7fd3041365f0 2024-01-24T23:28:21.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2080686322 0 0) 0x7fd30409c6b0 con 0x7fd304135e10 2024-01-24T23:28:21.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd2e4003530 con 0x7fd304135e10 2024-01-24T23:28:21.248 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd2f4004270 con 0x7fd3041361f0 2024-01-24T23:28:21.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:21.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:28:21.249 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient(hunting): dump: 2024-01-24T23:28:21.249 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:21.249 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:21.249 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:21.249 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:21.249 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:21.249 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:21.249 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:21.249 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:21.249 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:21.249 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:21.249 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:21.249 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:21.249 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:21.250 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:21.250 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:21.250 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:21.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:21.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd2fc001b00 con 0x7fd3041365f0 2024-01-24T23:28:21.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:21.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:28:21.250 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient(hunting): dump: 2024-01-24T23:28:21.250 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:21.250 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:21.250 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:21.250 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:21.250 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:21.250 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:21.250 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:21.250 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:21.250 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:21.250 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:21.251 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:21.251 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:21.251 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:21.251 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:21.251 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:21.251 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:21.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3811888429 0 0) 0x7fd304121e00 con 0x7fd3041365f0 2024-01-24T23:28:21.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 >> v1:172.21.15.132:6791/0 conn(0x7fd304135e10 legacy=0x7fd304134ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:21.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 >> v1:172.21.15.132:6790/0 conn(0x7fd3041361f0 legacy=0x7fd3041d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:21.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient: _finish_hunting 0 2024-01-24T23:28:21.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 1 monclient: found mon.f 2024-01-24T23:28:21.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:21.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:21.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3041220e0 con 0x7fd3041365f0 2024-01-24T23:28:21.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient: _finish_auth 0 2024-01-24T23:28:21.251 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient: _check_auth_tickets 2024-01-24T23:28:21.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:21.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 5 monclient: authenticate success, global_id 124507 2024-01-24T23:28:21.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 10 monclient: _renew_subs 2024-01-24T23:28:21.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:21.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 1 -- 172.21.15.132:0/1383373167 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd304120f10 con 0x7fd3041365f0 2024-01-24T23:28:21.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 10 monclient: _renew_subs 2024-01-24T23:28:21.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:21.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 1 -- 172.21.15.132:0/1383373167 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3041cb5a0 con 0x7fd3041365f0 2024-01-24T23:28:21.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd2fc001fd0 con 0x7fd3041365f0 2024-01-24T23:28:21.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:21.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd2fc005450 con 0x7fd3041365f0 2024-01-24T23:28:21.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:21.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd301ffb640 4 set_mon_vals no callback set 2024-01-24T23:28:21.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:28:21.252 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 10 monclient: dump: 2024-01-24T23:28:21.253 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:21.253 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:21.253 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:21.253 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:21.253 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:21.253 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:21.253 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:21.253 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:21.253 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:21.253 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:21.253 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:21.253 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:21.253 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:21.253 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:21.254 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:21.254 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:21.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:21.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:21.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:21.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd309d54640 1 -- 172.21.15.132:0/1383373167 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e8005090 con 0x7fd3041365f0 2024-01-24T23:28:21.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.234+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fd2fc00fed0 con 0x7fd3041365f0 2024-01-24T23:28:21.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.238+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(251..251 src has 1..251) v4 ==== 7051+0+0 (unknown 1997068085 0 0) 0x7fd2fc044850 con 0x7fd3041365f0 2024-01-24T23:28:21.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.238+0000 7fd3037fe640 10 monclient: get_auth_request con 0x7fd2e4037a90 auth_method 0 2024-01-24T23:28:21.254 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.242+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fd2fc014c80 con 0x7fd3041365f0 2024-01-24T23:28:21.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.242+0000 7fd2fbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:21.255 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.242+0000 7fd2fbfff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:21.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.406+0000 7fd309d54640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["1000"]}] 2024-01-24T23:28:21.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.406+0000 7fd309d54640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["1000"]}] 2024-01-24T23:28:21.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.406+0000 7fd309d54640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:21.408 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.406+0000 7fd309d54640 1 -- 172.21.15.132:0/1383373167 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd rm", "ids": ["1000"]} v 0) v1 -- 0x7fd2e80058c0 con 0x7fd3041365f0 2024-01-24T23:28:21.409 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:28:21.406+0000 7f48d6072640 -1 mon.h@5(leader).osd e251 definitely_dead 0 2024-01-24T23:28:21.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.406+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 <== mon.1 v1:172.21.15.132:6789/0 10 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["1000"]}]=0 osd.1000 does not exist. v251) v1 ==== 96+0+0 (unknown 1707474571 0 0) 0x7fd2fc013320 con 0x7fd3041365f0 2024-01-24T23:28:21.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.406+0000 7fd2fbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["1000"]}] 2024-01-24T23:28:21.409 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.406+0000 7fd2fbfff640 10 monclient: _finish_command 2 = system:0 osd.1000 does not exist. 2024-01-24T23:28:21.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.506+0000 7fd309d54640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["1000"]}] 2024-01-24T23:28:21.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.506+0000 7fd309d54640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["1000"]}] 2024-01-24T23:28:21.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.506+0000 7fd309d54640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:21.509 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.506+0000 7fd309d54640 1 -- 172.21.15.132:0/1383373167 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "osd rm", "ids": ["1000"]} v 0) v1 -- 0x7fd2e8000bd0 con 0x7fd3041365f0 2024-01-24T23:28:21.509 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:28:21.510+0000 7f48d6072640 -1 mon.h@5(leader).osd e251 definitely_dead 0 2024-01-24T23:28:21.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.510+0000 7fd2fbfff640 1 -- 172.21.15.132:0/1383373167 <== mon.1 v1:172.21.15.132:6789/0 11 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["1000"]}]=0 osd.1000 does not exist. v251) v1 ==== 96+0+0 (unknown 1707474571 0 0) 0x7fd2fc014aa0 con 0x7fd3041365f0 2024-01-24T23:28:21.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.510+0000 7fd2fbfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["1000"]}] 2024-01-24T23:28:21.510 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.510+0000 7fd2fbfff640 10 monclient: _finish_command 3 = system:0 osd.1000 does not exist. 2024-01-24T23:28:21.510 INFO:tasks.workunit.client.0.smithi132.stderr:osd.1000 does not exist. 2024-01-24T23:28:21.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.510+0000 7fd309d54640 1 -- 172.21.15.132:0/1383373167 >> v1:172.21.15.132:6812/1835401922 conn(0x7fd2e4037a90 legacy=0x7fd2e4039f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:21.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.510+0000 7fd309d54640 10 monclient: shutdown 2024-01-24T23:28:21.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.510+0000 7fd309d54640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:21.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.510+0000 7fd309d54640 1 -- 172.21.15.132:0/1383373167 >> v1:172.21.15.132:6789/0 conn(0x7fd3041365f0 legacy=0x7fd3041d3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:21.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.510+0000 7fd309d54640 1 -- 172.21.15.132:0/1383373167 shutdown_connections 2024-01-24T23:28:21.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.510+0000 7fd309d54640 1 -- 172.21.15.132:0/1383373167 >> 172.21.15.132:0/1383373167 conn(0x7fd3040a6790 msgr2=0x7fd3040a7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:21.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.510+0000 7fd309d54640 1 -- 172.21.15.132:0/1383373167 shutdown_connections 2024-01-24T23:28:21.512 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.510+0000 7fd309d54640 1 -- 172.21.15.132:0/1383373167 wait complete. 2024-01-24T23:28:21.528 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph osd create f78c88ca-c3ba-4894-b52e-23b6472a955a 2024-01-24T23:28:21.641 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.638+0000 7f44ed52e640 10 monclient: get_monmap_and_config 2024-01-24T23:28:21.641 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.638+0000 7f44ed52e640 10 monclient: build_initial_monmap 2024-01-24T23:28:21.641 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.638+0000 7f44ed52e640 10 monclient: monmap: 2024-01-24T23:28:21.641 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:21.642 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:21.642 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:21.644652+0000 2024-01-24T23:28:21.642 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:21.644652+0000 2024-01-24T23:28:21.642 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:21.642 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:21.642 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:21.642 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:21.642 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:21.642 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:21.642 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:21.642 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:21.642 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:21.642 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:21.642 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:21.642 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:21.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 1 Processor -- start 2024-01-24T23:28:21.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 1 -- start start 2024-01-24T23:28:21.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 10 monclient: init 2024-01-24T23:28:21.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:21.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 10 monclient: _add_conns ranks=[4,7,3,6,1,8,0,5,2] 2024-01-24T23:28:21.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 10 monclient(hunting): picked mon.noname-c con 0x7f44e809ff20 addr v1:172.21.15.115:6791/0 2024-01-24T23:28:21.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 10 monclient(hunting): picked mon.noname-i con 0x7f44e809f9c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:28:21.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 10 monclient(hunting): picked mon.noname-g con 0x7f44e813b2c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:28:21.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e8064700 con 0x7f44e813b2c0 2024-01-24T23:28:21.651 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e809e760 con 0x7f44e809ff20 2024-01-24T23:28:21.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e80a1160 con 0x7f44e809f9c0 2024-01-24T23:28:21.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:21.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 10 monclient(hunting): authenticate will time out at 1388.969274s 2024-01-24T23:28:21.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44dffff640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f44e809f9c0 legacy=0x7f44e8139ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:28:21.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e77fe640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f44e813b2c0 0x7f44e813d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:49656/0 (socket says 172.21.15.132:49656) 2024-01-24T23:28:21.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e77fe640 1 -- 172.21.15.132:0/1449438951 learned_addr learned my addr 172.21.15.132:0/1449438951 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:21.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 1 -- 172.21.15.132:0/1449438951 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3957634650 0 0) 0x7f44e8064700 con 0x7f44e813b2c0 2024-01-24T23:28:21.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:28:21.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:21.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:21.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 10 monclient(hunting): my global_id is 6996 2024-01-24T23:28:21.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 1 -- 172.21.15.132:0/1449438951 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44d4003530 con 0x7f44e813b2c0 2024-01-24T23:28:21.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 1 -- 172.21.15.132:0/1449438951 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 916399746 0 0) 0x7f44d4003530 con 0x7f44e813b2c0 2024-01-24T23:28:21.652 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 1 -- 172.21.15.132:0/1449438951 >> v1:172.21.15.132:6792/0 conn(0x7f44e809f9c0 legacy=0x7f44e8139ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:28:21.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 1 -- 172.21.15.132:0/1449438951 >> v1:172.21.15.115:6791/0 conn(0x7f44e809ff20 legacy=0x7f44e80a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:21.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 10 monclient: _finish_hunting 0 2024-01-24T23:28:21.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 1 monclient: found mon.noname-g 2024-01-24T23:28:21.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:21.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:28:21.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 1 -- 172.21.15.132:0/1449438951 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e80a0a10 con 0x7f44e813b2c0 2024-01-24T23:28:21.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 10 monclient: _finish_auth 0 2024-01-24T23:28:21.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 10 monclient: _check_auth_tickets 2024-01-24T23:28:21.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:21.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 5 monclient: authenticate success, global_id 6996 2024-01-24T23:28:21.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:21.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 1 -- 172.21.15.132:0/1449438951 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f44e0001b40 con 0x7f44e813b2c0 2024-01-24T23:28:21.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:21.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:28:21.653 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 10 monclient: dump: 2024-01-24T23:28:21.653 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:21.654 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:21.654 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:21.654 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:21.654 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:21.654 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:21.654 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:21.654 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:21.654 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:21.654 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:21.654 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:21.654 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:21.654 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:21.654 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:21.654 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:21.654 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:21.654 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:21.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 1 -- 172.21.15.132:0/1449438951 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f44e0001e60 con 0x7f44e813b2c0 2024-01-24T23:28:21.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:21.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 1 -- 172.21.15.132:0/1449438951 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f44e0004f70 con 0x7f44e813b2c0 2024-01-24T23:28:21.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:21.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:28:21.655 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44e67fc640 10 monclient: dump: 2024-01-24T23:28:21.655 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:21.655 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:21.655 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:21.655 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:21.655 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:21.655 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:21.655 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:21.655 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:21.656 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:21.656 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:21.656 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:21.656 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:21.656 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:21.656 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:21.656 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:21.656 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:21.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:21.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 4 set_mon_vals no callback set 2024-01-24T23:28:21.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 10 monclient: shutdown 2024-01-24T23:28:21.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:21.656 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 1 -- 172.21.15.132:0/1449438951 >> v1:172.21.15.132:6790/0 conn(0x7f44e813b2c0 legacy=0x7f44e813d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:21.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 1 -- 172.21.15.132:0/1449438951 shutdown_connections 2024-01-24T23:28:21.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 1 -- 172.21.15.132:0/1449438951 >> 172.21.15.132:0/1449438951 conn(0x7f44e80a5a10 msgr2=0x7f44e80a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:21.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 1 -- 172.21.15.132:0/1449438951 shutdown_connections 2024-01-24T23:28:21.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 1 -- 172.21.15.132:0/1449438951 wait complete. 2024-01-24T23:28:21.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 10 monclient: build_initial_monmap 2024-01-24T23:28:21.657 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 10 monclient: monmap: 2024-01-24T23:28:21.657 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:21.657 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:21.657 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:21.657 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:21.657 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:21.657 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:21.657 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:21.657 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:21.657 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:21.657 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:21.658 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:21.658 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:21.658 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:21.658 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:21.658 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:21.658 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:21.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 1 Processor -- start 2024-01-24T23:28:21.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 1 -- start start 2024-01-24T23:28:21.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 10 monclient: init 2024-01-24T23:28:21.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.642+0000 7f44ed52e640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:21.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44ed52e640 10 monclient: _add_conns ranks=[1,3,6,7,4,0,5,8,2] 2024-01-24T23:28:21.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44ed52e640 10 monclient(hunting): picked mon.noname-f con 0x7f44e809f9c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:28:21.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44ed52e640 10 monclient(hunting): picked mon.noname-g con 0x7f44e809ff20 addr v1:172.21.15.132:6790/0 2024-01-24T23:28:21.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44ed52e640 10 monclient(hunting): picked mon.noname-d con 0x7f44e813b2c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:28:21.658 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44ed52e640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e8071960 con 0x7f44e809f9c0 2024-01-24T23:28:21.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44ed52e640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e8071680 con 0x7f44e809ff20 2024-01-24T23:28:21.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44ed52e640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e811fda0 con 0x7f44e813b2c0 2024-01-24T23:28:21.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44ed52e640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:21.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44ed52e640 10 monclient(hunting): authenticate will time out at 1388.972111s 2024-01-24T23:28:21.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44dffff640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f44e809ff20 0x7f44e81d06a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:49658/0 (socket says 172.21.15.132:49658) 2024-01-24T23:28:21.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44e6ffd640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f44e809f9c0 0x7f44e81323c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:58734/0 (socket says 172.21.15.132:58734) 2024-01-24T23:28:21.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44dffff640 1 -- 172.21.15.132:0/83223691 learned_addr learned my addr 172.21.15.132:0/83223691 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:21.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 1 -- 172.21.15.132:0/83223691 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3538164488 0 0) 0x7f44e8071680 con 0x7f44e809ff20 2024-01-24T23:28:21.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:28:21.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:21.659 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:21.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient(hunting): my global_id is 7005 2024-01-24T23:28:21.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 1 -- 172.21.15.132:0/83223691 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44c8003530 con 0x7f44e809ff20 2024-01-24T23:28:21.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 1 -- 172.21.15.132:0/83223691 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 488833378 0 0) 0x7f44e8071960 con 0x7f44e809f9c0 2024-01-24T23:28:21.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:28:21.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:21.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:21.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient(hunting): my global_id is 124516 2024-01-24T23:28:21.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 1 -- 172.21.15.132:0/83223691 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44e8071680 con 0x7f44e809f9c0 2024-01-24T23:28:21.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 1 -- 172.21.15.132:0/83223691 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1254777626 0 0) 0x7f44c8003530 con 0x7f44e809ff20 2024-01-24T23:28:21.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 1 -- 172.21.15.132:0/83223691 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f44e8071960 con 0x7f44e809ff20 2024-01-24T23:28:21.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 1 -- 172.21.15.132:0/83223691 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f44d0004300 con 0x7f44e809ff20 2024-01-24T23:28:21.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:21.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:28:21.660 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient(hunting): dump: 2024-01-24T23:28:21.661 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:21.661 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:21.661 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:21.661 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:21.661 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:21.661 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:21.661 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:21.661 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:21.661 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:21.661 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:21.661 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:21.661 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:21.661 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:21.661 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:21.661 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:21.662 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:21.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:21.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 1 -- 172.21.15.132:0/83223691 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2774928808 0 0) 0x7f44e8071680 con 0x7f44e809f9c0 2024-01-24T23:28:21.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 1 -- 172.21.15.132:0/83223691 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f44c8003530 con 0x7f44e809f9c0 2024-01-24T23:28:21.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 1 -- 172.21.15.132:0/83223691 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f44d8001960 con 0x7f44e809f9c0 2024-01-24T23:28:21.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:21.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-01-24T23:28:21.662 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient(hunting): dump: 2024-01-24T23:28:21.662 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:21.662 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:21.662 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:21.662 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:21.662 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:21.662 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:21.662 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:21.663 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:21.663 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:21.663 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:21.663 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:21.663 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:21.663 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:21.663 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:21.663 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:21.663 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:21.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 1 -- 172.21.15.132:0/83223691 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132058659 0 0) 0x7f44e811fda0 con 0x7f44e813b2c0 2024-01-24T23:28:21.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:28:21.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:21.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:21.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient(hunting): my global_id is 7206 2024-01-24T23:28:21.663 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 1 -- 172.21.15.132:0/83223691 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44e8071680 con 0x7f44e813b2c0 2024-01-24T23:28:21.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 1 -- 172.21.15.132:0/83223691 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1874139932 0 0) 0x7f44e8071960 con 0x7f44e809ff20 2024-01-24T23:28:21.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 1 -- 172.21.15.132:0/83223691 >> v1:172.21.15.115:6792/0 conn(0x7f44e813b2c0 legacy=0x7f44e81d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:21.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 1 -- 172.21.15.132:0/83223691 >> v1:172.21.15.132:6789/0 conn(0x7f44e809f9c0 legacy=0x7f44e81323c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:21.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:28:21.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 1 monclient: found mon.g 2024-01-24T23:28:21.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:21.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:21.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 1 -- 172.21.15.132:0/83223691 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e809d2d0 con 0x7f44e809ff20 2024-01-24T23:28:21.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient: _finish_auth 0 2024-01-24T23:28:21.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:28:21.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:21.664 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44ed52e640 5 monclient: authenticate success, global_id 7005 2024-01-24T23:28:21.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44ed52e640 10 monclient: _renew_subs 2024-01-24T23:28:21.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44ed52e640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:21.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44ed52e640 1 -- 172.21.15.132:0/83223691 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e81c0a40 con 0x7f44e809ff20 2024-01-24T23:28:21.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44ed52e640 10 monclient: _renew_subs 2024-01-24T23:28:21.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44ed52e640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:21.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44ed52e640 1 -- 172.21.15.132:0/83223691 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e809cf30 con 0x7f44e809ff20 2024-01-24T23:28:21.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 1 -- 172.21.15.132:0/83223691 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f44d0004520 con 0x7f44e809ff20 2024-01-24T23:28:21.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:21.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 1 -- 172.21.15.132:0/83223691 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f44d00049d0 con 0x7f44e809ff20 2024-01-24T23:28:21.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:21.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44e5ffb640 4 set_mon_vals no callback set 2024-01-24T23:28:21.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:28:21.665 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 10 monclient: dump: 2024-01-24T23:28:21.665 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:21.665 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:21.666 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:21.666 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:21.666 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:21.666 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:21.666 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:21.666 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:21.666 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:21.666 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:21.666 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:21.666 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:21.666 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:21.666 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:21.666 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:21.666 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:21.666 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44ed52e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:21.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44ed52e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:21.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44ed52e640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:21.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 1 -- 172.21.15.132:0/83223691 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f44d0004bb0 con 0x7f44e809ff20 2024-01-24T23:28:21.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44ed52e640 1 -- 172.21.15.132:0/83223691 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44cc005090 con 0x7f44e809ff20 2024-01-24T23:28:21.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44df7fe640 1 -- 172.21.15.132:0/83223691 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(251..251 src has 1..251) v4 ==== 7051+0+0 (unknown 1997068085 0 0) 0x7f44d0044520 con 0x7f44e809ff20 2024-01-24T23:28:21.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.646+0000 7f44e6ffd640 10 monclient: get_auth_request con 0x7f44c8037810 auth_method 0 2024-01-24T23:28:21.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.650+0000 7f44df7fe640 1 -- 172.21.15.132:0/83223691 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f44d0014830 con 0x7f44e809ff20 2024-01-24T23:28:21.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.650+0000 7f44df7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:21.667 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.650+0000 7f44df7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:21.813 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~138s 2024-01-24T23:28:21.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.814+0000 7f44ed52e640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"}] 2024-01-24T23:28:21.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.814+0000 7f44ed52e640 10 monclient: _send_command 2 [{"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"}] 2024-01-24T23:28:21.816 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.814+0000 7f44ed52e640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:21.817 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:21.814+0000 7f44ed52e640 1 -- 172.21.15.132:0/83223691 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"} v 0) v1 -- 0x7f44cc0058c0 con 0x7f44e809ff20 2024-01-24T23:28:22.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:22.010+0000 7f44df7fe640 1 -- 172.21.15.132:0/83223691 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"}]=0 v252) v1 ==== 106+0+3 (unknown 4201652001 0 944643765) 0x7f44d0012ed0 con 0x7f44e809ff20 2024-01-24T23:28:22.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:22.010+0000 7f44df7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"}] 2024-01-24T23:28:22.010 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:22.010+0000 7f44df7fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:28:22.118 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:22.118+0000 7f44ed52e640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"}] 2024-01-24T23:28:22.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:22.118+0000 7f44ed52e640 10 monclient: _send_command 3 [{"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"}] 2024-01-24T23:28:22.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:22.118+0000 7f44ed52e640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:28:22.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:22.118+0000 7f44ed52e640 1 -- 172.21.15.132:0/83223691 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"} v 0) v1 -- 0x7f44cc002220 con 0x7f44e809ff20 2024-01-24T23:28:22.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:22.118+0000 7f44df7fe640 1 -- 172.21.15.132:0/83223691 <== mon.3 v1:172.21.15.132:6790/0 11 ==== mon_command_ack([{"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"}]=0 v252) v1 ==== 106+0+3 (unknown 4201652001 0 944643765) 0x7f44d0014650 con 0x7f44e809ff20 2024-01-24T23:28:22.119 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:22.118+0000 7f44df7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create", "uuid": "f78c88ca-c3ba-4894-b52e-23b6472a955a"}] 2024-01-24T23:28:22.120 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:22.118+0000 7f44df7fe640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:28:22.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:22.122+0000 7f44ed52e640 1 -- 172.21.15.132:0/83223691 >> v1:172.21.15.132:6812/1835401922 conn(0x7f44c8037810 legacy=0x7f44c8039cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:22.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:22.122+0000 7f44ed52e640 10 monclient: shutdown 2024-01-24T23:28:22.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:22.122+0000 7f44ed52e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:22.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:22.122+0000 7f44ed52e640 1 -- 172.21.15.132:0/83223691 >> v1:172.21.15.132:6790/0 conn(0x7f44e809ff20 legacy=0x7f44e81d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:22.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:22.122+0000 7f44ed52e640 1 -- 172.21.15.132:0/83223691 shutdown_connections 2024-01-24T23:28:22.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:22.122+0000 7f44ed52e640 1 -- 172.21.15.132:0/83223691 >> 172.21.15.132:0/83223691 conn(0x7f44e80a5a10 msgr2=0x7f44e8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:22.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:22.122+0000 7f44ed52e640 1 -- 172.21.15.132:0/83223691 shutdown_connections 2024-01-24T23:28:22.122 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:22.122+0000 7f44ed52e640 1 -- 172.21.15.132:0/83223691 wait complete. 2024-01-24T23:28:22.136 INFO:tasks.workunit.client.0.smithi132.stdout:OK 2024-01-24T23:28:22.137 INFO:tasks.workunit.client.0.smithi132.stderr:+ na2=12 2024-01-24T23:28:22.137 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo OK 2024-01-24T23:28:22.137 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-01-24T23:28:22.138 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-01-24T23:28:22.150 INFO:tasks.workunit:Running workunits matching mon/caps.sh on client.0... 2024-01-24T23:28:22.152 INFO:tasks.workunit:Running workunit mon/caps.sh... 2024-01-24T23:28:22.153 DEBUG:teuthology.orchestra.run.smithi132:workunit test mon/caps.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=663873555d45def955d1cc09730bfece2847b821 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/caps.sh 2024-01-24T23:28:22.201 INFO:tasks.workunit.client.0.smithi132.stderr:+ tmp=/tmp/cephtest-mon-caps-madness 2024-01-24T23:28:22.201 INFO:tasks.workunit.client.0.smithi132.stderr:+ exit_on_error=1 2024-01-24T23:28:22.201 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ ! -z '' ]] 2024-01-24T23:28:22.202 INFO:tasks.workunit.client.0.smithi132.stderr:++ uname 2024-01-24T23:28:22.202 INFO:tasks.workunit.client.0.smithi132.stderr:+ '[' Linux = FreeBSD ']' 2024-01-24T23:28:22.202 INFO:tasks.workunit.client.0.smithi132.stderr:+ ETIMEDOUT=110 2024-01-24T23:28:22.202 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect 'ceph auth get-or-create client.bazar > /tmp/cephtest-mon-caps-madness.bazar.keyring' 0 2024-01-24T23:28:22.202 INFO:tasks.workunit.client.0.smithi132.stderr:+ cmd='ceph auth get-or-create client.bazar > /tmp/cephtest-mon-caps-madness.bazar.keyring' 2024-01-24T23:28:22.202 INFO:tasks.workunit.client.0.smithi132.stderr:+ expected_ret=0 2024-01-24T23:28:22.202 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo ceph auth get-or-create client.bazar '>' /tmp/cephtest-mon-caps-madness.bazar.keyring 2024-01-24T23:28:22.203 INFO:tasks.workunit.client.0.smithi132.stdout:ceph auth get-or-create client.bazar > /tmp/cephtest-mon-caps-madness.bazar.keyring 2024-01-24T23:28:22.203 INFO:tasks.workunit.client.0.smithi132.stderr:+ eval ceph auth get-or-create client.bazar '>' /tmp/cephtest-mon-caps-madness.bazar.keyring 2024-01-24T23:28:22.688 INFO:tasks.workunit.client.0.smithi132.stderr:+ ret=0 2024-01-24T23:28:22.688 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ 0 -ne 0 ]] 2024-01-24T23:28:22.688 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:28:22.688 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bazar.keyring --user bazar quorum_status' 13 2024-01-24T23:28:22.688 INFO:tasks.workunit.client.0.smithi132.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bazar.keyring --user bazar quorum_status 2024-01-24T23:28:22.689 INFO:tasks.workunit.client.0.smithi132.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bazar.keyring --user bazar quorum_status' 2024-01-24T23:28:22.689 INFO:tasks.workunit.client.0.smithi132.stderr:+ expected_ret=13 2024-01-24T23:28:22.689 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bazar.keyring --user bazar quorum_status 2024-01-24T23:28:22.689 INFO:tasks.workunit.client.0.smithi132.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bazar.keyring --user bazar quorum_status 2024-01-24T23:28:28.022 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~144s 2024-01-24T23:28:28.807 INFO:tasks.workunit.client.0.smithi132.stderr:+ ret=13 2024-01-24T23:28:28.807 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ 13 -ne 13 ]] 2024-01-24T23:28:28.807 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:28:28.807 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph auth del client.bazar 2024-01-24T23:28:28.929 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.926+0000 7f7b71492640 10 monclient: get_monmap_and_config 2024-01-24T23:28:28.929 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.926+0000 7f7b71492640 10 monclient: build_initial_monmap 2024-01-24T23:28:28.929 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.926+0000 7f7b71492640 10 monclient: monmap: 2024-01-24T23:28:28.929 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:28.929 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:28.929 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:28:28.932378+0000 2024-01-24T23:28:28.929 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:28:28.932378+0000 2024-01-24T23:28:28.929 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:28.929 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:28.929 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:28.929 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:28.930 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:28.930 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:28.930 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:28.930 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:28.930 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:28.930 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:28.930 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:28.930 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:28.936 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 1 Processor -- start 2024-01-24T23:28:28.936 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 1 -- start start 2024-01-24T23:28:28.936 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 10 monclient: init 2024-01-24T23:28:28.936 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:28.936 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 10 monclient: _add_conns ranks=[3,4,1,0,7,6,5,2,8] 2024-01-24T23:28:28.936 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 10 monclient(hunting): picked mon.noname-g con 0x7f7b6c0a1b50 addr v1:172.21.15.132:6790/0 2024-01-24T23:28:28.936 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 10 monclient(hunting): picked mon.noname-c con 0x7f7b6c0a1f30 addr v1:172.21.15.115:6791/0 2024-01-24T23:28:28.937 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 10 monclient(hunting): picked mon.noname-f con 0x7f7b6c13b030 addr v1:172.21.15.132:6789/0 2024-01-24T23:28:28.937 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b6c064700 con 0x7f7b6c13b030 2024-01-24T23:28:28.937 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b6c070ff0 con 0x7f7b6c0a1b50 2024-01-24T23:28:28.937 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b6c06f7d0 con 0x7f7b6c0a1f30 2024-01-24T23:28:28.937 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:28.937 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 10 monclient(hunting): authenticate will time out at 1396.257457s 2024-01-24T23:28:28.937 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b6b7fe640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f7b6c13b030 0x7f7b6c13d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:38040/0 (socket says 172.21.15.132:38040) 2024-01-24T23:28:28.937 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b6affd640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f7b6c0a1b50 0x7f7b6c071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.132:44524/0 (socket says 172.21.15.132:44524) 2024-01-24T23:28:28.937 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b6b7fe640 1 -- 172.21.15.132:0/3689315871 learned_addr learned my addr 172.21.15.132:0/3689315871 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:28.937 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 1 -- 172.21.15.132:0/3689315871 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3442918140 0 0) 0x7f7b6c064700 con 0x7f7b6c13b030 2024-01-24T23:28:28.937 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:28:28.937 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:28.937 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:28.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 10 monclient(hunting): my global_id is 124534 2024-01-24T23:28:28.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 1 -- 172.21.15.132:0/3689315871 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b54003530 con 0x7f7b6c13b030 2024-01-24T23:28:28.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 1 -- 172.21.15.132:0/3689315871 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3211671957 0 0) 0x7f7b6c070ff0 con 0x7f7b6c0a1b50 2024-01-24T23:28:28.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:28:28.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:28.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:28.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 10 monclient(hunting): my global_id is 7032 2024-01-24T23:28:28.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 1 -- 172.21.15.132:0/3689315871 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b6c064700 con 0x7f7b6c0a1b50 2024-01-24T23:28:28.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 1 -- 172.21.15.132:0/3689315871 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2416278095 0 0) 0x7f7b6c064700 con 0x7f7b6c0a1b50 2024-01-24T23:28:28.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 1 -- 172.21.15.132:0/3689315871 >> v1:172.21.15.115:6791/0 conn(0x7f7b6c0a1f30 legacy=0x7f7b6c0a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:28.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 1 -- 172.21.15.132:0/3689315871 >> v1:172.21.15.132:6789/0 conn(0x7f7b6c13b030 legacy=0x7f7b6c13d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:28.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:28:28.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 1 monclient: found mon.noname-g 2024-01-24T23:28:28.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:28.938 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:28:28.939 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 1 -- 172.21.15.132:0/3689315871 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6c0a36b0 con 0x7f7b6c0a1b50 2024-01-24T23:28:28.939 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 10 monclient: _finish_auth 0 2024-01-24T23:28:28.939 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:28:28.939 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:28.939 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 1 -- 172.21.15.132:0/3689315871 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7b5c001670 con 0x7f7b6c0a1b50 2024-01-24T23:28:28.939 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 5 monclient: authenticate success, global_id 7032 2024-01-24T23:28:28.939 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:28.939 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:28.939 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:28:28.939 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 10 monclient: dump: 2024-01-24T23:28:28.939 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:28.939 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:28.939 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:28.939 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:28.940 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:28.940 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:28.940 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:28.940 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:28.940 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:28.940 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:28.940 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:28.940 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:28.940 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:28.940 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:28.940 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:28.940 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:28.940 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:28:28.940 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 1 -- 172.21.15.132:0/3689315871 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7b5c001820 con 0x7f7b6c0a1b50 2024-01-24T23:28:28.941 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:28.941 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 10 monclient: get_monmap_and_config success 2024-01-24T23:28:28.941 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 4 set_mon_vals no callback set 2024-01-24T23:28:28.941 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 1 -- 172.21.15.132:0/3689315871 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7b5c001d90 con 0x7f7b6c0a1b50 2024-01-24T23:28:28.941 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:28.941 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:28:28.941 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b69ffb640 10 monclient: dump: 2024-01-24T23:28:28.941 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:28.941 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:28.941 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:28.941 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:28.941 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:28.941 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:28.941 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:28.941 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:28.941 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:28.942 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:28.942 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:28.942 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:28.942 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:28.942 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:28.942 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:28.942 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:28.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 10 monclient: shutdown 2024-01-24T23:28:28.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:28.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 1 -- 172.21.15.132:0/3689315871 >> v1:172.21.15.132:6790/0 conn(0x7f7b6c0a1b50 legacy=0x7f7b6c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:28.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 1 -- 172.21.15.132:0/3689315871 shutdown_connections 2024-01-24T23:28:28.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 1 -- 172.21.15.132:0/3689315871 >> 172.21.15.132:0/3689315871 conn(0x7f7b6c0a7150 msgr2=0x7f7b6c0a7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:28.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 1 -- 172.21.15.132:0/3689315871 shutdown_connections 2024-01-24T23:28:28.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 1 -- 172.21.15.132:0/3689315871 wait complete. 2024-01-24T23:28:28.942 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 10 monclient: build_initial_monmap 2024-01-24T23:28:28.943 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.930+0000 7f7b71492640 10 monclient: monmap: 2024-01-24T23:28:28.943 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:28:28.943 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:28:28.943 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 0.000000 2024-01-24T23:28:28.943 INFO:tasks.workunit.client.0.smithi132.stderr:created 0.000000 2024-01-24T23:28:28.943 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:28:28.943 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 1 2024-01-24T23:28:28.943 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:28:28.943 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:28:28.943 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:28:28.943 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:28:28.943 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:28:28.943 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:28:28.943 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:28:28.943 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:28:28.944 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:28:28.944 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:28.944 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b71492640 1 Processor -- start 2024-01-24T23:28:28.944 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b71492640 1 -- start start 2024-01-24T23:28:28.944 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b71492640 10 monclient: init 2024-01-24T23:28:28.944 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b71492640 10 monclient: _reopen_session rank -1 2024-01-24T23:28:28.944 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b71492640 10 monclient: _add_conns ranks=[7,8,1,2,3,6,0,5,4] 2024-01-24T23:28:28.944 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b71492640 10 monclient(hunting): picked mon.noname-i con 0x7f7b6c0a1b50 addr v1:172.21.15.132:6792/0 2024-01-24T23:28:28.944 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b71492640 10 monclient(hunting): picked mon.noname-e con 0x7f7b6c0a1f30 addr v1:172.21.15.115:6793/0 2024-01-24T23:28:28.944 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b71492640 10 monclient(hunting): picked mon.noname-f con 0x7f7b6c13b030 addr v1:172.21.15.132:6789/0 2024-01-24T23:28:28.944 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b71492640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b6c1389e0 con 0x7f7b6c13b030 2024-01-24T23:28:28.944 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b71492640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b6c1393f0 con 0x7f7b6c0a1b50 2024-01-24T23:28:28.944 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b71492640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b6c072110 con 0x7f7b6c0a1f30 2024-01-24T23:28:28.944 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b71492640 10 monclient(hunting): _renew_subs 2024-01-24T23:28:28.945 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b71492640 10 monclient(hunting): authenticate will time out at 1396.260573s 2024-01-24T23:28:28.945 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b6affd640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f7b6c0a1b50 legacy=0x7f7b6c133100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:28:28.945 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b6b7fe640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f7b6c13b030 0x7f7b6c1d3930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:38050/0 (socket says 172.21.15.132:38050) 2024-01-24T23:28:28.945 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b6b7fe640 1 -- 172.21.15.132:0/19896611 learned_addr learned my addr 172.21.15.132:0/19896611 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:28:28.945 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 1 -- 172.21.15.132:0/19896611 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3868886582 0 0) 0x7f7b6c1389e0 con 0x7f7b6c13b030 2024-01-24T23:28:28.945 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 10 monclient(hunting): _negotiate 2024-01-24T23:28:28.945 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:28.945 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:28.945 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 10 monclient(hunting): my global_id is 124543 2024-01-24T23:28:28.945 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 1 -- 172.21.15.132:0/19896611 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b58003530 con 0x7f7b6c13b030 2024-01-24T23:28:28.945 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 1 -- 172.21.15.132:0/19896611 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3378393179 0 0) 0x7f7b58003530 con 0x7f7b6c13b030 2024-01-24T23:28:28.945 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 1 -- 172.21.15.132:0/19896611 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7b6c1389e0 con 0x7f7b6c13b030 2024-01-24T23:28:28.945 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 1 -- 172.21.15.132:0/19896611 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7b64001cc0 con 0x7f7b6c13b030 2024-01-24T23:28:28.945 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:28.946 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:28:28.946 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 10 monclient(hunting): dump: 2024-01-24T23:28:28.946 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:28.946 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:28.946 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:28.946 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:28.946 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:28.946 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:28.946 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:28.946 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:28.946 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:28.946 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:28.946 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:28.946 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:28.947 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:28.947 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:28.947 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:28.947 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:28.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:28:28.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 1 -- 172.21.15.132:0/19896611 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2988208718 0 0) 0x7f7b6c072110 con 0x7f7b6c0a1f30 2024-01-24T23:28:28.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 10 monclient(hunting): _negotiate 2024-01-24T23:28:28.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:28:28.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:28:28.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 10 monclient(hunting): my global_id is 86237 2024-01-24T23:28:28.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 1 -- 172.21.15.132:0/19896611 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b58003530 con 0x7f7b6c0a1f30 2024-01-24T23:28:28.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 1 -- 172.21.15.132:0/19896611 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 735244777 0 0) 0x7f7b6c1389e0 con 0x7f7b6c13b030 2024-01-24T23:28:28.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 1 -- 172.21.15.132:0/19896611 >> v1:172.21.15.115:6793/0 conn(0x7f7b6c0a1f30 legacy=0x7f7b6c1d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:28.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 1 -- 172.21.15.132:0/19896611 >> v1:172.21.15.132:6792/0 conn(0x7f7b6c0a1b50 legacy=0x7f7b6c133100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:28:28.947 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 10 monclient: _finish_hunting 0 2024-01-24T23:28:28.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 1 monclient: found mon.f 2024-01-24T23:28:28.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:28:28.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:28.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 1 -- 172.21.15.132:0/19896611 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6c139110 con 0x7f7b6c13b030 2024-01-24T23:28:28.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 10 monclient: _finish_auth 0 2024-01-24T23:28:28.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 10 monclient: _check_auth_tickets 2024-01-24T23:28:28.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:28:28.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b71492640 5 monclient: authenticate success, global_id 124543 2024-01-24T23:28:28.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b71492640 10 monclient: _renew_subs 2024-01-24T23:28:28.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b71492640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:28.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b71492640 1 -- 172.21.15.132:0/19896611 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6c133810 con 0x7f7b6c13b030 2024-01-24T23:28:28.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b71492640 10 monclient: _renew_subs 2024-01-24T23:28:28.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b71492640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:28.948 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b71492640 1 -- 172.21.15.132:0/19896611 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6c137c10 con 0x7f7b6c13b030 2024-01-24T23:28:28.949 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 1 -- 172.21.15.132:0/19896611 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7b640020b0 con 0x7f7b6c13b030 2024-01-24T23:28:28.949 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:28:28.949 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 1 -- 172.21.15.132:0/19896611 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7b64005160 con 0x7f7b6c13b030 2024-01-24T23:28:28.949 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:28:28.949 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b697fa640 4 set_mon_vals no callback set 2024-01-24T23:28:28.949 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:28:28.949 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 10 monclient: dump: 2024-01-24T23:28:28.949 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 1 2024-01-24T23:28:28.949 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:28:28.949 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:28.949 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:28:28.949 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 19 (squid) 2024-01-24T23:28:28.949 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:28:28.949 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:28:28.950 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:28:28.950 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:28:28.950 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:28:28.950 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:28:28.950 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:28:28.950 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:28:28.950 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:28:28.950 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:28:28.950 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:28:28.950 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b71492640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:28.950 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 1 -- 172.21.15.132:0/19896611 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f7b6400fc60 con 0x7f7b6c13b030 2024-01-24T23:28:28.950 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b71492640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:28.950 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b71492640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:28.950 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b71492640 1 -- 172.21.15.132:0/19896611 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b4c005090 con 0x7f7b6c13b030 2024-01-24T23:28:28.950 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b537fe640 1 -- 172.21.15.132:0/19896611 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7f7b640447f0 con 0x7f7b6c13b030 2024-01-24T23:28:28.950 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.934+0000 7f7b6affd640 10 monclient: get_auth_request con 0x7f7b58036f70 auth_method 0 2024-01-24T23:28:28.951 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.938+0000 7f7b537fe640 1 -- 172.21.15.132:0/19896611 <== mon.1 v1:172.21.15.132:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f7b64011250 con 0x7f7b6c13b030 2024-01-24T23:28:28.951 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.938+0000 7f7b537fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:28:28.951 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:28.938+0000 7f7b537fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:28:29.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:29.142+0000 7f7b71492640 10 start_mon_command cmd=[{"prefix": "auth del", "entity": "client.bazar"}] 2024-01-24T23:28:29.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:29.142+0000 7f7b71492640 10 monclient: _send_command 2 [{"prefix": "auth del", "entity": "client.bazar"}] 2024-01-24T23:28:29.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:29.142+0000 7f7b71492640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:29.144 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:29.142+0000 7f7b71492640 1 -- 172.21.15.132:0/19896611 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "auth del", "entity": "client.bazar"} v 0) v1 -- 0x7f7b4c005340 con 0x7f7b6c13b030 2024-01-24T23:28:29.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:29.150+0000 7f7b537fe640 1 -- 172.21.15.132:0/19896611 <== mon.1 v1:172.21.15.132:6789/0 10 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bazar"}]=0 v18) v1 ==== 82+0+0 (unknown 1432643774 0 0) 0x7f7b640132b0 con 0x7f7b6c13b030 2024-01-24T23:28:29.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:29.150+0000 7f7b537fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "auth del", "entity": "client.bazar"}] 2024-01-24T23:28:29.150 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:29.150+0000 7f7b537fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:28:29.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:29.298+0000 7f7b71492640 10 start_mon_command cmd=[{"prefix": "auth del", "entity": "client.bazar"}] 2024-01-24T23:28:29.297 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:29.298+0000 7f7b71492640 10 monclient: _send_command 3 [{"prefix": "auth del", "entity": "client.bazar"}] 2024-01-24T23:28:29.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:29.298+0000 7f7b71492640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:28:29.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:29.298+0000 7f7b71492640 1 -- 172.21.15.132:0/19896611 --> v1:172.21.15.132:6789/0 -- mon_command({"prefix": "auth del", "entity": "client.bazar"} v 0) v1 -- 0x7f7b4c002220 con 0x7f7b6c13b030 2024-01-24T23:28:29.298 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:29.298+0000 7f7b537fe640 1 -- 172.21.15.132:0/19896611 <== mon.1 v1:172.21.15.132:6789/0 11 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bazar"}]=0 v18) v1 ==== 82+0+0 (unknown 1432643774 0 0) 0x7f7b64014a30 con 0x7f7b6c13b030 2024-01-24T23:28:29.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:29.298+0000 7f7b537fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "auth del", "entity": "client.bazar"}] 2024-01-24T23:28:29.299 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:29.298+0000 7f7b537fe640 10 monclient: _finish_command 3 = system:0 2024-01-24T23:28:29.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:29.298+0000 7f7b71492640 1 -- 172.21.15.132:0/19896611 >> v1:172.21.15.132:6812/1835401922 conn(0x7f7b58036f70 legacy=0x7f7b58039430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:29.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:29.298+0000 7f7b71492640 10 monclient: shutdown 2024-01-24T23:28:29.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:29.298+0000 7f7b71492640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:28:29.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:29.298+0000 7f7b71492640 1 -- 172.21.15.132:0/19896611 >> v1:172.21.15.132:6789/0 conn(0x7f7b6c13b030 legacy=0x7f7b6c1d3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:28:29.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:29.298+0000 7f7b71492640 1 -- 172.21.15.132:0/19896611 shutdown_connections 2024-01-24T23:28:29.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:29.298+0000 7f7b71492640 1 -- 172.21.15.132:0/19896611 >> 172.21.15.132:0/19896611 conn(0x7f7b6c0a7150 msgr2=0x7f7b6c135ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:28:29.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:29.298+0000 7f7b71492640 1 -- 172.21.15.132:0/19896611 shutdown_connections 2024-01-24T23:28:29.301 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:28:29.302+0000 7f7b71492640 1 -- 172.21.15.132:0/19896611 wait complete. 2024-01-24T23:28:29.315 INFO:tasks.workunit.client.0.smithi132.stderr:+ c=''\''allow command "auth ls", allow command quorum_status'\''' 2024-01-24T23:28:29.315 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect 'ceph auth get-or-create client.foo mon '\''allow command "auth ls", allow command quorum_status'\'' > /tmp/cephtest-mon-caps-madness.foo.keyring' 0 2024-01-24T23:28:29.315 INFO:tasks.workunit.client.0.smithi132.stderr:+ cmd='ceph auth get-or-create client.foo mon '\''allow command "auth ls", allow command quorum_status'\'' > /tmp/cephtest-mon-caps-madness.foo.keyring' 2024-01-24T23:28:29.315 INFO:tasks.workunit.client.0.smithi132.stderr:+ expected_ret=0 2024-01-24T23:28:29.316 INFO:tasks.workunit.client.0.smithi132.stdout:ceph auth get-or-create client.foo mon 'allow command "auth ls", allow command quorum_status' > /tmp/cephtest-mon-caps-madness.foo.keyring 2024-01-24T23:28:29.316 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo ceph auth get-or-create client.foo mon ''\''allow' command '"auth' 'ls",' allow command 'quorum_status'\''' '>' /tmp/cephtest-mon-caps-madness.foo.keyring 2024-01-24T23:28:29.316 INFO:tasks.workunit.client.0.smithi132.stderr:+ eval ceph auth get-or-create client.foo mon ''\''allow' command '"auth' 'ls",' allow command 'quorum_status'\''' '>' /tmp/cephtest-mon-caps-madness.foo.keyring 2024-01-24T23:28:29.828 INFO:tasks.workunit.client.0.smithi132.stderr:+ ret=0 2024-01-24T23:28:29.828 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ 0 -ne 0 ]] 2024-01-24T23:28:29.828 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:28:29.828 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status' 0 2024-01-24T23:28:29.829 INFO:tasks.workunit.client.0.smithi132.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status 2024-01-24T23:28:29.829 INFO:tasks.workunit.client.0.smithi132.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status' 2024-01-24T23:28:29.829 INFO:tasks.workunit.client.0.smithi132.stderr:+ expected_ret=0 2024-01-24T23:28:29.829 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status 2024-01-24T23:28:29.829 INFO:tasks.workunit.client.0.smithi132.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status 2024-01-24T23:28:33.332 INFO:tasks.workunit.client.0.smithi132.stderr:+ ret=0 2024-01-24T23:28:33.333 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ 0 -ne 0 ]] 2024-01-24T23:28:33.333 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:28:33.333 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth ls' 0 2024-01-24T23:28:33.333 INFO:tasks.workunit.client.0.smithi132.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth ls 2024-01-24T23:28:33.333 INFO:tasks.workunit.client.0.smithi132.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth ls' 2024-01-24T23:28:33.333 INFO:tasks.workunit.client.0.smithi132.stderr:+ expected_ret=0 2024-01-24T23:28:33.334 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth ls 2024-01-24T23:28:33.334 INFO:tasks.workunit.client.0.smithi132.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth ls 2024-01-24T23:28:33.828 INFO:tasks.workunit.client.0.smithi132.stderr:+ ret=0 2024-01-24T23:28:33.829 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ 0 -ne 0 ]] 2024-01-24T23:28:33.829 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:28:33.829 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth export' 13 2024-01-24T23:28:33.829 INFO:tasks.workunit.client.0.smithi132.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth export' 2024-01-24T23:28:33.829 INFO:tasks.workunit.client.0.smithi132.stderr:+ expected_ret=13 2024-01-24T23:28:33.829 INFO:tasks.workunit.client.0.smithi132.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth export 2024-01-24T23:28:33.830 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth export 2024-01-24T23:28:33.830 INFO:tasks.workunit.client.0.smithi132.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth export 2024-01-24T23:28:34.177 INFO:tasks.workunit.client.0.smithi132.stderr:+ ret=13 2024-01-24T23:28:34.177 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ 13 -ne 13 ]] 2024-01-24T23:28:34.177 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:28:34.177 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth del client.bazar' 13 2024-01-24T23:28:34.177 INFO:tasks.workunit.client.0.smithi132.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth del client.bazar' 2024-01-24T23:28:34.177 INFO:tasks.workunit.client.0.smithi132.stderr:+ expected_ret=13 2024-01-24T23:28:34.178 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth del client.bazar 2024-01-24T23:28:34.178 INFO:tasks.workunit.client.0.smithi132.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth del client.bazar 2024-01-24T23:28:34.178 INFO:tasks.workunit.client.0.smithi132.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth del client.bazar 2024-01-24T23:28:34.230 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~150s 2024-01-24T23:28:34.498 INFO:tasks.workunit.client.0.smithi132.stderr:+ ret=13 2024-01-24T23:28:34.498 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ 13 -ne 13 ]] 2024-01-24T23:28:34.498 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:28:34.498 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo osd dump' 13 2024-01-24T23:28:34.498 INFO:tasks.workunit.client.0.smithi132.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo osd dump' 2024-01-24T23:28:34.498 INFO:tasks.workunit.client.0.smithi132.stderr:+ expected_ret=13 2024-01-24T23:28:34.498 INFO:tasks.workunit.client.0.smithi132.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo osd dump 2024-01-24T23:28:34.499 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo osd dump 2024-01-24T23:28:34.499 INFO:tasks.workunit.client.0.smithi132.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo osd dump 2024-01-24T23:28:34.778 INFO:tasks.workunit.client.0.smithi132.stderr:+ ret=13 2024-01-24T23:28:34.779 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ 13 -ne 13 ]] 2024-01-24T23:28:34.779 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:28:34.779 INFO:tasks.workunit.client.0.smithi132.stderr:+ export CEPH_ARGS=--rados-mon-op-timeout=300 2024-01-24T23:28:34.779 INFO:tasks.workunit.client.0.smithi132.stderr:+ CEPH_ARGS=--rados-mon-op-timeout=300 2024-01-24T23:28:34.779 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo pg dump' 110 2024-01-24T23:28:34.779 INFO:tasks.workunit.client.0.smithi132.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo pg dump' 2024-01-24T23:28:34.779 INFO:tasks.workunit.client.0.smithi132.stderr:+ expected_ret=110 2024-01-24T23:28:34.779 INFO:tasks.workunit.client.0.smithi132.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo pg dump 2024-01-24T23:28:34.780 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo pg dump 2024-01-24T23:28:34.780 INFO:tasks.workunit.client.0.smithi132.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo pg dump 2024-01-24T23:28:40.441 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~156s 2024-01-24T23:28:46.650 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~162s 2024-01-24T23:28:52.861 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~169s 2024-01-24T23:28:59.071 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~175s 2024-01-24T23:29:05.282 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~181s 2024-01-24T23:29:11.493 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~187s 2024-01-24T23:29:17.703 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~193s 2024-01-24T23:29:23.915 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~200s 2024-01-24T23:29:24.415 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-01-24T23:29:24.416 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-01-24T23:29:24.416 INFO:tasks.ceph.mon.a:Restarting daemon 2024-01-24T23:29:24.416 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-01-24T23:29:24.420 INFO:tasks.ceph.mon.a:Started 2024-01-24T23:29:24.420 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-01-24T23:29:24.420 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-01-24T23:29:24.420 INFO:tasks.ceph.mon.b:Restarting daemon 2024-01-24T23:29:24.420 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-01-24T23:29:24.423 INFO:tasks.ceph.mon.b:Started 2024-01-24T23:29:24.423 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-01-24T23:29:24.423 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.i 2024-01-24T23:29:24.423 INFO:tasks.ceph.mon.i:Restarting daemon 2024-01-24T23:29:24.424 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-01-24T23:29:24.426 INFO:tasks.ceph.mon.i:Started 2024-01-24T23:29:24.426 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-01-24T23:29:24.426 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-01-24T23:29:24.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 10 monclient: get_monmap_and_config 2024-01-24T23:29:24.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 10 monclient: build_initial_monmap 2024-01-24T23:29:24.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 10 monclient: monmap: 2024-01-24T23:29:24.560 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:24.560 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:24.561 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:24.562996+0000 2024-01-24T23:29:24.561 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:24.562996+0000 2024-01-24T23:29:24.561 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:24.561 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:24.561 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:24.561 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:24.561 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:24.561 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:24.561 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:24.561 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:24.561 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:24.561 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:24.561 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:24.561 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:24.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 1 Processor -- start 2024-01-24T23:29:24.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 1 -- start start 2024-01-24T23:29:24.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 10 monclient: init 2024-01-24T23:29:24.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:24.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 10 monclient: _add_conns ranks=[7,1,3,4,0,5,2,8,6] 2024-01-24T23:29:24.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 10 monclient(hunting): picked mon.noname-i con 0x7f9438133bf0 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:24.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 10 monclient(hunting): picked mon.noname-f con 0x7f9438133fd0 addr v1:172.21.15.132:6789/0 2024-01-24T23:29:24.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 10 monclient(hunting): picked mon.noname-g con 0x7f94381343b0 addr v1:172.21.15.132:6790/0 2024-01-24T23:29:24.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943804ed70 con 0x7f9438133fd0 2024-01-24T23:29:24.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94380a4b60 con 0x7f94381343b0 2024-01-24T23:29:24.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943809ae50 con 0x7f9438133bf0 2024-01-24T23:29:24.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:24.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 10 monclient(hunting): authenticate will time out at 1443.485374s 2024-01-24T23:29:24.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943d3b0640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f9438133bf0 0x7f943809bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.115:45638/0 (socket says 172.21.15.115:45638) 2024-01-24T23:29:24.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943d3b0640 1 -- 172.21.15.115:0/247765954 learned_addr learned my addr 172.21.15.115:0/247765954 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:24.569 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 1 -- 172.21.15.115:0/247765954 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2182453416 0 0) 0x7f94380a4b60 con 0x7f94381343b0 2024-01-24T23:29:24.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:24.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:24.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:24.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 10 monclient(hunting): my global_id is 7077 2024-01-24T23:29:24.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 1 -- 172.21.15.115:0/247765954 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9424003530 con 0x7f94381343b0 2024-01-24T23:29:24.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 1 -- 172.21.15.115:0/247765954 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3191060694 0 0) 0x7f943804ed70 con 0x7f9438133fd0 2024-01-24T23:29:24.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:24.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:24.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:24.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 10 monclient(hunting): my global_id is 124588 2024-01-24T23:29:24.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 1 -- 172.21.15.115:0/247765954 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94380a4b60 con 0x7f9438133fd0 2024-01-24T23:29:24.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 1 -- 172.21.15.115:0/247765954 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2851318683 0 0) 0x7f9424003530 con 0x7f94381343b0 2024-01-24T23:29:24.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 1 -- 172.21.15.115:0/247765954 >> v1:172.21.15.132:6792/0 conn(0x7f9438133bf0 legacy=0x7f943809bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:24.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 1 -- 172.21.15.115:0/247765954 >> v1:172.21.15.132:6789/0 conn(0x7f9438133fd0 legacy=0x7f943813bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:24.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 10 monclient: _finish_hunting 0 2024-01-24T23:29:24.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 1 monclient: found mon.noname-g 2024-01-24T23:29:24.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:24.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:29:24.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 1 -- 172.21.15.115:0/247765954 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943809b130 con 0x7f94381343b0 2024-01-24T23:29:24.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 10 monclient: _finish_auth 0 2024-01-24T23:29:24.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 10 monclient: _check_auth_tickets 2024-01-24T23:29:24.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:24.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 1 -- 172.21.15.115:0/247765954 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9430001b40 con 0x7f94381343b0 2024-01-24T23:29:24.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 5 monclient: authenticate success, global_id 7077 2024-01-24T23:29:24.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:24.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:24.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:29:24.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 10 monclient: dump: 2024-01-24T23:29:24.572 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:24.572 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:24.572 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:24.572 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:24.572 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:24.572 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:24.572 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:24.572 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:24.572 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:24.572 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:24.572 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:24.572 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:24.572 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:24.573 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:24.573 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:24.573 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:24.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:24.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 1 -- 172.21.15.115:0/247765954 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9430001e20 con 0x7f94381343b0 2024-01-24T23:29:24.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:24.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 1 -- 172.21.15.115:0/247765954 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9430003ce0 con 0x7f94381343b0 2024-01-24T23:29:24.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:24.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:29:24.573 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f942ffff640 10 monclient: dump: 2024-01-24T23:29:24.573 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:24.573 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:24.573 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:24.573 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:24.573 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:24.574 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:24.574 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:24.574 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:24.574 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:24.574 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:24.574 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:24.574 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:24.574 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:24.574 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:24.574 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:24.574 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:24.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:24.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 4 set_mon_vals no callback set 2024-01-24T23:29:24.574 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 10 monclient: shutdown 2024-01-24T23:29:24.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:24.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 1 -- 172.21.15.115:0/247765954 >> v1:172.21.15.132:6790/0 conn(0x7f94381343b0 legacy=0x7f943813f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:24.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 1 -- 172.21.15.115:0/247765954 shutdown_connections 2024-01-24T23:29:24.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 1 -- 172.21.15.115:0/247765954 >> 172.21.15.115:0/247765954 conn(0x7f94380662f0 msgr2=0x7f9438066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:24.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 1 -- 172.21.15.115:0/247765954 shutdown_connections 2024-01-24T23:29:24.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.559+0000 7f943f614640 1 -- 172.21.15.115:0/247765954 wait complete. 2024-01-24T23:29:24.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f943f614640 10 monclient: build_initial_monmap 2024-01-24T23:29:24.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f943f614640 10 monclient: monmap: 2024-01-24T23:29:24.575 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:24.575 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:24.575 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:24.575 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:24.575 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:24.575 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:24.575 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:24.576 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:24.576 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:24.576 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:24.576 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:24.576 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:24.576 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:24.576 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:24.576 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:24.576 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:24.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f943f614640 1 Processor -- start 2024-01-24T23:29:24.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f943f614640 1 -- start start 2024-01-24T23:29:24.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f943f614640 10 monclient: init 2024-01-24T23:29:24.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f943f614640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:24.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f943f614640 10 monclient: _add_conns ranks=[2,3,5,1,0,6,4,8,7] 2024-01-24T23:29:24.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f943f614640 10 monclient(hunting): picked mon.noname-b con 0x7f9438133bf0 addr v1:172.21.15.115:6790/0 2024-01-24T23:29:24.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f943f614640 10 monclient(hunting): picked mon.noname-g con 0x7f9438133fd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:29:24.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f943f614640 10 monclient(hunting): picked mon.noname-h con 0x7f94381343b0 addr v1:172.21.15.132:6791/0 2024-01-24T23:29:24.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f943f614640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94380a25f0 con 0x7f9438133bf0 2024-01-24T23:29:24.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f943f614640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94380a5990 con 0x7f9438133fd0 2024-01-24T23:29:24.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f943f614640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94381c8ce0 con 0x7f94381343b0 2024-01-24T23:29:24.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f943f614640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:24.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f943f614640 10 monclient(hunting): authenticate will time out at 1443.489985s 2024-01-24T23:29:24.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f943d3b0640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f9438133bf0 0x7f9438132ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:35080/0 (socket says 172.21.15.115:35080) 2024-01-24T23:29:24.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f943d3b0640 1 -- 172.21.15.115:0/4177020108 learned_addr learned my addr 172.21.15.115:0/4177020108 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:24.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 1 -- 172.21.15.115:0/4177020108 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 439443865 0 0) 0x7f94380a5990 con 0x7f9438133fd0 2024-01-24T23:29:24.577 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 10 monclient(hunting): _negotiate 2024-01-24T23:29:24.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:24.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:24.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 10 monclient(hunting): my global_id is 7086 2024-01-24T23:29:24.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 1 -- 172.21.15.115:0/4177020108 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f941c003530 con 0x7f9438133fd0 2024-01-24T23:29:24.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 1 -- 172.21.15.115:0/4177020108 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3790101340 0 0) 0x7f941c003530 con 0x7f9438133fd0 2024-01-24T23:29:24.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 1 -- 172.21.15.115:0/4177020108 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f94380a5990 con 0x7f9438133fd0 2024-01-24T23:29:24.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 1 -- 172.21.15.115:0/4177020108 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9428001a60 con 0x7f9438133fd0 2024-01-24T23:29:24.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:24.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:29:24.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 10 monclient(hunting): dump: 2024-01-24T23:29:24.578 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:24.578 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:24.578 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:24.578 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:24.578 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:24.579 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:24.579 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:24.579 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:24.579 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:24.579 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:24.579 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:24.579 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:24.579 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:24.579 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:24.579 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:24.579 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:24.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:24.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 1 -- 172.21.15.115:0/4177020108 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 552142978 0 0) 0x7f94380a5990 con 0x7f9438133fd0 2024-01-24T23:29:24.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 1 -- 172.21.15.115:0/4177020108 >> v1:172.21.15.132:6791/0 conn(0x7f94381343b0 legacy=0x7f94381d40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:24.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 1 -- 172.21.15.115:0/4177020108 >> v1:172.21.15.115:6790/0 conn(0x7f9438133bf0 legacy=0x7f9438132ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:24.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 10 monclient: _finish_hunting 0 2024-01-24T23:29:24.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 1 monclient: found mon.g 2024-01-24T23:29:24.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:24.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:29:24.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 1 -- 172.21.15.115:0/4177020108 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943811ffe0 con 0x7f9438133fd0 2024-01-24T23:29:24.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 10 monclient: _finish_auth 0 2024-01-24T23:29:24.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 10 monclient: _check_auth_tickets 2024-01-24T23:29:24.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:24.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f943f614640 5 monclient: authenticate success, global_id 7086 2024-01-24T23:29:24.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f943f614640 10 monclient: _renew_subs 2024-01-24T23:29:24.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f943f614640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:29:24.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f943f614640 1 -- 172.21.15.115:0/4177020108 --> v1:172.21.15.132:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94381335d0 con 0x7f9438133fd0 2024-01-24T23:29:24.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f943f614640 10 monclient: _renew_subs 2024-01-24T23:29:24.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f943f614640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:29:24.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f943f614640 1 -- 172.21.15.115:0/4177020108 --> v1:172.21.15.132:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f94380a3540 con 0x7f9438133fd0 2024-01-24T23:29:24.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 1 -- 172.21.15.115:0/4177020108 <== mon.3 v1:172.21.15.132:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f94280038d0 con 0x7f9438133fd0 2024-01-24T23:29:24.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:24.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 1 -- 172.21.15.115:0/4177020108 <== mon.3 v1:172.21.15.132:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f9428003e00 con 0x7f9438133fd0 2024-01-24T23:29:24.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:24.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942f7fe640 4 set_mon_vals no callback set 2024-01-24T23:29:24.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:29:24.581 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.563+0000 7f942dffb640 10 monclient: dump: 2024-01-24T23:29:24.581 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:24.581 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:24.581 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:24.581 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:24.581 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:24.581 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:24.582 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.567+0000 7f943f614640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.567+0000 7f943f614640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.567+0000 7f943f614640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:29:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.567+0000 7f943f614640 1 -- 172.21.15.115:0/4177020108 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9418005090 con 0x7f9438133fd0 2024-01-24T23:29:24.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.567+0000 7f942dffb640 1 -- 172.21.15.115:0/4177020108 <== mon.3 v1:172.21.15.132:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f9428003f60 con 0x7f9438133fd0 2024-01-24T23:29:24.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.567+0000 7f942dffb640 1 -- 172.21.15.115:0/4177020108 <== mon.3 v1:172.21.15.132:6790/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7f9428043320 con 0x7f9438133fd0 2024-01-24T23:29:24.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.567+0000 7f943d3b0640 10 monclient: get_auth_request con 0x7f941c037070 auth_method 0 2024-01-24T23:29:24.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.571+0000 7f942dffb640 1 -- 172.21.15.115:0/4177020108 <== mon.3 v1:172.21.15.132:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f9428011d30 con 0x7f9438133fd0 2024-01-24T23:29:24.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.571+0000 7f942dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:24.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.571+0000 7f942dffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:24.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.779+0000 7f943f614640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-01-24T23:29:24.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.779+0000 7f943f614640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-01-24T23:29:24.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.779+0000 7f943f614640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.132:6790/0 2024-01-24T23:29:24.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:24.779+0000 7f943f614640 1 -- 172.21.15.115:0/4177020108 --> v1:172.21.15.132:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f94180058c0 con 0x7f9438133fd0 2024-01-24T23:29:27.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:27.563+0000 7f942cff9640 10 monclient: tick 2024-01-24T23:29:27.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:27.563+0000 7f942cff9640 10 monclient: _check_auth_tickets 2024-01-24T23:29:27.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:27.563+0000 7f942cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:27.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:27.563+0000 7f942cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:28.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:28.563+0000 7f942cff9640 10 monclient: tick 2024-01-24T23:29:28.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:28.563+0000 7f942cff9640 10 monclient: _check_auth_tickets 2024-01-24T23:29:28.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:28.563+0000 7f942cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:29.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:29.563+0000 7f942cff9640 10 monclient: tick 2024-01-24T23:29:29.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:29.563+0000 7f942cff9640 10 monclient: _check_auth_tickets 2024-01-24T23:29:29.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:29.563+0000 7f942cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:29.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:29.583+0000 7f942dffb640 1 -- 172.21.15.115:0/4177020108 <== mon.3 v1:172.21.15.132:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2732 (unknown 254746657 0 746725103) 0x7f942801bdc0 con 0x7f9438133fd0 2024-01-24T23:29:29.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:29.583+0000 7f942dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-01-24T23:29:29.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:29.583+0000 7f942dffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:29.588 INFO:teuthology.orchestra.run.smithi115.stdout:{"election_epoch":46,"quorum":[0,1,3,4,5,6,8],"quorum_names":["a","f","g","c","h","d","e"],"quorum_leader_name":"h","quorum_age":0,"features":{"quorum_con":"4540701272860131327","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","modified":"2024-01-24T23:20:20.557246Z","created":"2024-01-24T23:20:20.557246Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6790","nonce":0}]},"addr":"172.21.15.115:6790/0","public_addr":"172.21.15.115:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6790","nonce":0}]},"addr":"172.21.15.132:6790/0","public_addr":"172.21.15.132:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6791","nonce":0}]},"addr":"172.21.15.115:6791/0","public_addr":"172.21.15.115:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6791","nonce":0}]},"addr":"172.21.15.132:6791/0","public_addr":"172.21.15.132:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6792","nonce":0}]},"addr":"172.21.15.115:6792/0","public_addr":"172.21.15.115:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6792","nonce":0}]},"addr":"172.21.15.132:6792/0","public_addr":"172.21.15.132:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6793","nonce":0}]},"addr":"172.21.15.115:6793/0","public_addr":"172.21.15.115:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-01-24T23:29:29.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:29.587+0000 7f943f614640 1 -- 172.21.15.115:0/4177020108 >> v1:172.21.15.132:6812/1835401922 conn(0x7f941c037070 legacy=0x7f941c039530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:29.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:29.587+0000 7f943f614640 10 monclient: shutdown 2024-01-24T23:29:29.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:29.587+0000 7f943f614640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:29.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:29.587+0000 7f943f614640 1 -- 172.21.15.115:0/4177020108 >> v1:172.21.15.132:6790/0 conn(0x7f9438133fd0 legacy=0x7f94381d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:29.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:29.587+0000 7f943f614640 1 -- 172.21.15.115:0/4177020108 shutdown_connections 2024-01-24T23:29:29.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:29.587+0000 7f943f614640 1 -- 172.21.15.115:0/4177020108 >> 172.21.15.115:0/4177020108 conn(0x7f94380662f0 msgr2=0x7f94381beb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:29.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:29.587+0000 7f943f614640 1 -- 172.21.15.115:0/4177020108 shutdown_connections 2024-01-24T23:29:29.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:29.587+0000 7f943f614640 1 -- 172.21.15.115:0/4177020108 wait complete. 2024-01-24T23:29:30.427 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~206s 2024-01-24T23:29:32.612 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-01-24T23:29:32.737 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.735+0000 7fd6e1ab8640 10 monclient: get_monmap_and_config 2024-01-24T23:29:32.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.735+0000 7fd6e1ab8640 10 monclient: build_initial_monmap 2024-01-24T23:29:32.738 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.735+0000 7fd6e1ab8640 10 monclient: monmap: 2024-01-24T23:29:32.738 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:32.738 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:32.738 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:32.740416+0000 2024-01-24T23:29:32.738 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:32.740416+0000 2024-01-24T23:29:32.738 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:32.738 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:32.738 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:32.738 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:32.738 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:32.738 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:32.738 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:32.738 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:32.738 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:32.739 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:32.739 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:32.739 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:32.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.735+0000 7fd6e1ab8640 1 Processor -- start 2024-01-24T23:29:32.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.735+0000 7fd6e1ab8640 1 -- start start 2024-01-24T23:29:32.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.735+0000 7fd6e1ab8640 10 monclient: init 2024-01-24T23:29:32.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.735+0000 7fd6e1ab8640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:32.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.735+0000 7fd6e1ab8640 10 monclient: _add_conns ranks=[1,2,0,4,7,5,3,6,8] 2024-01-24T23:29:32.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.735+0000 7fd6e1ab8640 10 monclient(hunting): picked mon.noname-f con 0x7fd6dc133bf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:29:32.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.735+0000 7fd6e1ab8640 10 monclient(hunting): picked mon.noname-b con 0x7fd6dc133fd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:29:32.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.735+0000 7fd6e1ab8640 10 monclient(hunting): picked mon.noname-a con 0x7fd6dc134430 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:32.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.735+0000 7fd6e1ab8640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6dc04ed70 con 0x7fd6dc134430 2024-01-24T23:29:32.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.735+0000 7fd6e1ab8640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6dc09af30 con 0x7fd6dc133bf0 2024-01-24T23:29:32.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.735+0000 7fd6e1ab8640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6dc09b210 con 0x7fd6dc133fd0 2024-01-24T23:29:32.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.735+0000 7fd6e1ab8640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:32.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.735+0000 7fd6e1ab8640 10 monclient(hunting): authenticate will time out at 1451.663220s 2024-01-24T23:29:32.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.735+0000 7fd6dbfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fd6dc134430 0x7fd6dc13f500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39080/0 (socket says 172.21.15.115:39080) 2024-01-24T23:29:32.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.735+0000 7fd6daffd640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fd6dc133fd0 0x7fd6dc13bdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:43026/0 (socket says 172.21.15.115:43026) 2024-01-24T23:29:32.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.735+0000 7fd6dbfff640 1 -- 172.21.15.115:0/3065629651 learned_addr learned my addr 172.21.15.115:0/3065629651 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:32.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 1 -- 172.21.15.115:0/3065629651 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2672574748 0 0) 0x7fd6dc09af30 con 0x7fd6dc133bf0 2024-01-24T23:29:32.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:32.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:32.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:32.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 10 monclient(hunting): my global_id is 124597 2024-01-24T23:29:32.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 1 -- 172.21.15.115:0/3065629651 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6c4003530 con 0x7fd6dc133bf0 2024-01-24T23:29:32.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 1 -- 172.21.15.115:0/3065629651 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2610594263 0 0) 0x7fd6c4003530 con 0x7fd6dc133bf0 2024-01-24T23:29:32.743 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 1 -- 172.21.15.115:0/3065629651 >> v1:172.21.15.115:6790/0 conn(0x7fd6dc133fd0 legacy=0x7fd6dc13bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:32.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 1 -- 172.21.15.115:0/3065629651 >> v1:172.21.15.115:6789/0 conn(0x7fd6dc134430 legacy=0x7fd6dc13f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:32.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:29:32.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 1 monclient: found mon.noname-f 2024-01-24T23:29:32.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:32.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:29:32.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 1 -- 172.21.15.115:0/3065629651 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6dc09b4f0 con 0x7fd6dc133bf0 2024-01-24T23:29:32.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 10 monclient: _finish_auth 0 2024-01-24T23:29:32.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:29:32.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:32.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 5 monclient: authenticate success, global_id 124597 2024-01-24T23:29:32.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:32.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 1 -- 172.21.15.115:0/3065629651 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd6cc0017c0 con 0x7fd6dc133bf0 2024-01-24T23:29:32.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:32.744 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:29:32.745 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 10 monclient: dump: 2024-01-24T23:29:32.745 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:32.745 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:32.745 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:32.745 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:32.745 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:32.745 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:32.745 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:32.745 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:32.745 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:32.745 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:32.745 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:32.745 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:32.745 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:32.746 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:32.746 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:32.747 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:32.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:32.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 1 -- 172.21.15.115:0/3065629651 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd6cc001a70 con 0x7fd6dc133bf0 2024-01-24T23:29:32.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:32.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 1 -- 172.21.15.115:0/3065629651 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd6cc003960 con 0x7fd6dc133bf0 2024-01-24T23:29:32.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:32.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:29:32.747 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6da7fc640 10 monclient: dump: 2024-01-24T23:29:32.747 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:32.747 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:32.747 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:32.747 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:32.747 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:32.747 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:32.747 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:32.748 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:32.748 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:32.748 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:32.748 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:32.748 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:32.748 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:32.748 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:32.748 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:32.748 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:32.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:32.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 4 set_mon_vals no callback set 2024-01-24T23:29:32.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 10 monclient: shutdown 2024-01-24T23:29:32.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:32.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 1 -- 172.21.15.115:0/3065629651 >> v1:172.21.15.132:6789/0 conn(0x7fd6dc133bf0 legacy=0x7fd6dc09bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:32.748 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 1 -- 172.21.15.115:0/3065629651 shutdown_connections 2024-01-24T23:29:32.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 1 -- 172.21.15.115:0/3065629651 >> 172.21.15.115:0/3065629651 conn(0x7fd6dc0a9910 msgr2=0x7fd6dc0a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:32.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 1 -- 172.21.15.115:0/3065629651 shutdown_connections 2024-01-24T23:29:32.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 1 -- 172.21.15.115:0/3065629651 wait complete. 2024-01-24T23:29:32.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 10 monclient: build_initial_monmap 2024-01-24T23:29:32.749 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 10 monclient: monmap: 2024-01-24T23:29:32.749 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:32.749 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:32.749 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:32.749 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:32.749 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:32.749 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:32.749 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:32.749 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:32.749 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:32.749 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:32.750 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:32.750 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:32.750 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:32.750 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:32.750 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:32.750 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:32.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 1 Processor -- start 2024-01-24T23:29:32.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 1 -- start start 2024-01-24T23:29:32.750 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 10 monclient: init 2024-01-24T23:29:32.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:32.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 10 monclient: _add_conns ranks=[8,5,6,7,1,2,0,4,3] 2024-01-24T23:29:32.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 10 monclient(hunting): picked mon.noname-e con 0x7fd6dc133bf0 addr v1:172.21.15.115:6793/0 2024-01-24T23:29:32.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 10 monclient(hunting): picked mon.noname-h con 0x7fd6dc133fd0 addr v1:172.21.15.132:6791/0 2024-01-24T23:29:32.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 10 monclient(hunting): picked mon.noname-d con 0x7fd6dc134430 addr v1:172.21.15.115:6792/0 2024-01-24T23:29:32.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6dc13a690 con 0x7fd6dc133fd0 2024-01-24T23:29:32.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6dc09c8f0 con 0x7fd6dc134430 2024-01-24T23:29:32.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6dc0a5960 con 0x7fd6dc133bf0 2024-01-24T23:29:32.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:32.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6e1ab8640 10 monclient(hunting): authenticate will time out at 1451.667119s 2024-01-24T23:29:32.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6db7fe640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fd6dc133bf0 0x7fd6dc137ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:55878/0 (socket says 172.21.15.115:55878) 2024-01-24T23:29:32.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.739+0000 7fd6db7fe640 1 -- 172.21.15.115:0/182143364 learned_addr learned my addr 172.21.15.115:0/182143364 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:32.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 1 -- 172.21.15.115:0/182143364 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693273269 0 0) 0x7fd6dc09c8f0 con 0x7fd6dc134430 2024-01-24T23:29:32.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:32.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:32.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:32.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 10 monclient(hunting): my global_id is 7251 2024-01-24T23:29:32.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 1 -- 172.21.15.115:0/182143364 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6c8003530 con 0x7fd6dc134430 2024-01-24T23:29:32.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 1 -- 172.21.15.115:0/182143364 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1416209345 0 0) 0x7fd6dc0a5960 con 0x7fd6dc133bf0 2024-01-24T23:29:32.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:32.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:32.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:32.754 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 10 monclient(hunting): my global_id is 86300 2024-01-24T23:29:32.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 1 -- 172.21.15.115:0/182143364 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6dc09c8f0 con 0x7fd6dc133bf0 2024-01-24T23:29:32.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 1 -- 172.21.15.115:0/182143364 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3336071090 0 0) 0x7fd6c8003530 con 0x7fd6dc134430 2024-01-24T23:29:32.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 1 -- 172.21.15.115:0/182143364 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd6dc0a5960 con 0x7fd6dc134430 2024-01-24T23:29:32.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 1 -- 172.21.15.115:0/182143364 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd6d4001870 con 0x7fd6dc134430 2024-01-24T23:29:32.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:32.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:29:32.755 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 10 monclient(hunting): dump: 2024-01-24T23:29:32.755 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:32.755 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:32.755 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:32.755 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:32.755 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:32.755 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:32.755 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:32.755 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:32.755 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:32.756 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:32.756 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:32.756 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:32.756 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:32.756 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:32.756 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:32.756 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:32.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:32.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 1 -- 172.21.15.115:0/182143364 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2368055388 0 0) 0x7fd6dc09c8f0 con 0x7fd6dc133bf0 2024-01-24T23:29:32.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 1 -- 172.21.15.115:0/182143364 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd6c8003530 con 0x7fd6dc133bf0 2024-01-24T23:29:32.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 1 -- 172.21.15.115:0/182143364 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd6cc001780 con 0x7fd6dc133bf0 2024-01-24T23:29:32.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:32.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:29:32.756 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 10 monclient(hunting): dump: 2024-01-24T23:29:32.756 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:32.757 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:32.757 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:32.757 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:32.757 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:32.757 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:32.757 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:32.757 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:32.757 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:32.757 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:32.757 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:32.757 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:32.757 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:32.757 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:32.757 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:32.757 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:32.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 1 -- 172.21.15.115:0/182143364 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2224253844 0 0) 0x7fd6dc0a5960 con 0x7fd6dc134430 2024-01-24T23:29:32.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 1 -- 172.21.15.115:0/182143364 >> v1:172.21.15.115:6793/0 conn(0x7fd6dc133bf0 legacy=0x7fd6dc137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:32.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 1 -- 172.21.15.115:0/182143364 >> v1:172.21.15.132:6791/0 conn(0x7fd6dc133fd0 legacy=0x7fd6dc1382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:32.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 10 monclient: _finish_hunting 0 2024-01-24T23:29:32.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 1 monclient: found mon.d 2024-01-24T23:29:32.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:32.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:29:32.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 1 -- 172.21.15.115:0/182143364 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6dc13ac50 con 0x7fd6dc134430 2024-01-24T23:29:32.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 10 monclient: _finish_auth 0 2024-01-24T23:29:32.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 10 monclient: _check_auth_tickets 2024-01-24T23:29:32.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:32.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6e1ab8640 5 monclient: authenticate success, global_id 7251 2024-01-24T23:29:32.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6e1ab8640 10 monclient: _renew_subs 2024-01-24T23:29:32.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6e1ab8640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:29:32.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6e1ab8640 1 -- 172.21.15.115:0/182143364 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6dc13af30 con 0x7fd6dc134430 2024-01-24T23:29:32.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6e1ab8640 10 monclient: _renew_subs 2024-01-24T23:29:32.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6e1ab8640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:29:32.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6e1ab8640 1 -- 172.21.15.115:0/182143364 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6dc138c40 con 0x7fd6dc134430 2024-01-24T23:29:32.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 1 -- 172.21.15.115:0/182143364 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd6d40047f0 con 0x7fd6dc134430 2024-01-24T23:29:32.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:32.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 1 -- 172.21.15.115:0/182143364 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fd6d4004ce0 con 0x7fd6dc134430 2024-01-24T23:29:32.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:32.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:29:32.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6d9ffb640 4 set_mon_vals no callback set 2024-01-24T23:29:32.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 10 monclient: dump: 2024-01-24T23:29:32.759 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:32.759 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:32.759 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:32.759 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:32.759 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:32.760 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:32.760 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:32.760 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:32.760 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:32.760 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:32.760 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:32.760 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:32.760 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:32.760 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:32.760 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:32.760 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:32.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6e1ab8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:32.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 1 -- 172.21.15.115:0/182143364 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fd6d4004e40 con 0x7fd6dc134430 2024-01-24T23:29:32.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6e1ab8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:32.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6e1ab8640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:29:32.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6e1ab8640 1 -- 172.21.15.115:0/182143364 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6bc005090 con 0x7fd6dc134430 2024-01-24T23:29:32.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6c3fff640 1 -- 172.21.15.115:0/182143364 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7fd6d40441f0 con 0x7fd6dc134430 2024-01-24T23:29:32.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.747+0000 7fd6db7fe640 10 monclient: get_auth_request con 0x7fd6c8037630 auth_method 0 2024-01-24T23:29:32.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.751+0000 7fd6c3fff640 1 -- 172.21.15.115:0/182143364 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7fd6d40050f0 con 0x7fd6dc134430 2024-01-24T23:29:32.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.751+0000 7fd6c3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:32.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.751+0000 7fd6c3fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:32.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.963+0000 7fd6e1ab8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-01-24T23:29:32.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.963+0000 7fd6e1ab8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-01-24T23:29:32.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.963+0000 7fd6e1ab8640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:29:32.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.963+0000 7fd6e1ab8640 1 -- 172.21.15.115:0/182143364 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd6bc0058c0 con 0x7fd6dc134430 2024-01-24T23:29:32.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.963+0000 7fd6c3fff640 1 -- 172.21.15.115:0/182143364 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2744 (unknown 254746657 0 3681181965) 0x7fd6d40143e0 con 0x7fd6dc134430 2024-01-24T23:29:32.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.963+0000 7fd6c3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-01-24T23:29:32.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.963+0000 7fd6c3fff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:32.967 INFO:teuthology.orchestra.run.smithi115.stdout:{"election_epoch":48,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":3,"features":{"quorum_con":"4540701272860131327","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","modified":"2024-01-24T23:20:20.557246Z","created":"2024-01-24T23:20:20.557246Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6790","nonce":0}]},"addr":"172.21.15.115:6790/0","public_addr":"172.21.15.115:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6790","nonce":0}]},"addr":"172.21.15.132:6790/0","public_addr":"172.21.15.132:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6791","nonce":0}]},"addr":"172.21.15.115:6791/0","public_addr":"172.21.15.115:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6791","nonce":0}]},"addr":"172.21.15.132:6791/0","public_addr":"172.21.15.132:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6792","nonce":0}]},"addr":"172.21.15.115:6792/0","public_addr":"172.21.15.115:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6792","nonce":0}]},"addr":"172.21.15.132:6792/0","public_addr":"172.21.15.132:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6793","nonce":0}]},"addr":"172.21.15.115:6793/0","public_addr":"172.21.15.115:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-01-24T23:29:32.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.967+0000 7fd6e1ab8640 1 -- 172.21.15.115:0/182143364 >> v1:172.21.15.132:6812/1835401922 conn(0x7fd6c8037630 legacy=0x7fd6c8039af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:32.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.967+0000 7fd6e1ab8640 10 monclient: shutdown 2024-01-24T23:29:32.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.967+0000 7fd6e1ab8640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:32.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.967+0000 7fd6e1ab8640 1 -- 172.21.15.115:0/182143364 >> v1:172.21.15.115:6792/0 conn(0x7fd6dc134430 legacy=0x7fd6dc1d3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:32.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.967+0000 7fd6e1ab8640 1 -- 172.21.15.115:0/182143364 shutdown_connections 2024-01-24T23:29:32.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.967+0000 7fd6e1ab8640 1 -- 172.21.15.115:0/182143364 >> 172.21.15.115:0/182143364 conn(0x7fd6dc0a9910 msgr2=0x7fd6dc13ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:32.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.967+0000 7fd6e1ab8640 1 -- 172.21.15.115:0/182143364 shutdown_connections 2024-01-24T23:29:32.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:32.967+0000 7fd6e1ab8640 1 -- 172.21.15.115:0/182143364 wait complete. 2024-01-24T23:29:32.984 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-01-24T23:29:32.985 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-01-24T23:29:32.985 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-01-24T23:29:33.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 10 monclient: get_monmap_and_config 2024-01-24T23:29:33.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 10 monclient: build_initial_monmap 2024-01-24T23:29:33.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 10 monclient: monmap: 2024-01-24T23:29:33.096 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:33.096 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.096 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:33.099026+0000 2024-01-24T23:29:33.097 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:33.099026+0000 2024-01-24T23:29:33.097 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:33.097 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.097 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:33.097 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:33.097 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:33.097 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:33.097 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:33.097 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:33.097 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:33.097 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:33.097 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:33.097 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 1 Processor -- start 2024-01-24T23:29:33.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 1 -- start start 2024-01-24T23:29:33.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 10 monclient: init 2024-01-24T23:29:33.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:33.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 10 monclient: _add_conns ranks=[0,8,6,3,5,2,4,1,7] 2024-01-24T23:29:33.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 10 monclient(hunting): picked mon.noname-a con 0x7fe9200a19f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:33.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 10 monclient(hunting): picked mon.noname-e con 0x7fe9200a1dd0 addr v1:172.21.15.115:6793/0 2024-01-24T23:29:33.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 10 monclient(hunting): picked mon.noname-d con 0x7fe92013cef0 addr v1:172.21.15.115:6792/0 2024-01-24T23:29:33.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe92004ed70 con 0x7fe9200a19f0 2024-01-24T23:29:33.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe920066d80 con 0x7fe92013cef0 2024-01-24T23:29:33.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe92009afe0 con 0x7fe9200a1dd0 2024-01-24T23:29:33.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:33.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 10 monclient(hunting): authenticate will time out at 1452.021587s 2024-01-24T23:29:33.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91f7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fe9200a19f0 0x7fe92009bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39082/0 (socket says 172.21.15.115:39082) 2024-01-24T23:29:33.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91ffff640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fe92013cef0 0x7fe92013f2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:53670/0 (socket says 172.21.15.115:53670) 2024-01-24T23:29:33.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91effd640 1 --1- 172.21.15.115:0/162018120 >> v1:172.21.15.115:6793/0 conn(0x7fe9200a1dd0 0x7fe9200a21f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:55892/0 (socket says 172.21.15.115:55892) 2024-01-24T23:29:33.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91f7fe640 1 -- 172.21.15.115:0/162018120 learned_addr learned my addr 172.21.15.115:0/162018120 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:33.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 1 -- 172.21.15.115:0/162018120 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406771503 0 0) 0x7fe92004ed70 con 0x7fe9200a19f0 2024-01-24T23:29:33.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 10 monclient(hunting): my global_id is 154098 2024-01-24T23:29:33.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 1 -- 172.21.15.115:0/162018120 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe908003530 con 0x7fe9200a19f0 2024-01-24T23:29:33.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 1 -- 172.21.15.115:0/162018120 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 58489903 0 0) 0x7fe920066d80 con 0x7fe92013cef0 2024-01-24T23:29:33.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 10 monclient(hunting): my global_id is 7260 2024-01-24T23:29:33.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 1 -- 172.21.15.115:0/162018120 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe92004ed70 con 0x7fe92013cef0 2024-01-24T23:29:33.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 1 -- 172.21.15.115:0/162018120 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3532215497 0 0) 0x7fe92009afe0 con 0x7fe9200a1dd0 2024-01-24T23:29:33.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 10 monclient(hunting): my global_id is 86309 2024-01-24T23:29:33.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 1 -- 172.21.15.115:0/162018120 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe920066d80 con 0x7fe9200a1dd0 2024-01-24T23:29:33.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 1 -- 172.21.15.115:0/162018120 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2324293433 0 0) 0x7fe908003530 con 0x7fe9200a19f0 2024-01-24T23:29:33.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 1 -- 172.21.15.115:0/162018120 >> v1:172.21.15.115:6793/0 conn(0x7fe9200a1dd0 legacy=0x7fe9200a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 1 -- 172.21.15.115:0/162018120 >> v1:172.21.15.115:6792/0 conn(0x7fe92013cef0 legacy=0x7fe92013f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:29:33.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 1 monclient: found mon.noname-a 2024-01-24T23:29:33.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:33.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:29:33.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 1 -- 172.21.15.115:0/162018120 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92009b2c0 con 0x7fe9200a19f0 2024-01-24T23:29:33.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 10 monclient: _finish_auth 0 2024-01-24T23:29:33.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:29:33.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:33.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 1 -- 172.21.15.115:0/162018120 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe9140017c0 con 0x7fe9200a19f0 2024-01-24T23:29:33.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:29:33.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 10 monclient: dump: 2024-01-24T23:29:33.108 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.108 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.108 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.108 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.108 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.108 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.108 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.108 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.108 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.108 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.108 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.109 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.109 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.109 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.109 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.109 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 5 monclient: authenticate success, global_id 154098 2024-01-24T23:29:33.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:33.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 1 -- 172.21.15.115:0/162018120 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe914001a30 con 0x7fe9200a19f0 2024-01-24T23:29:33.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:33.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:33.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 4 set_mon_vals no callback set 2024-01-24T23:29:33.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 1 -- 172.21.15.115:0/162018120 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe914001c60 con 0x7fe9200a19f0 2024-01-24T23:29:33.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:29:33.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe91e7fc640 10 monclient: dump: 2024-01-24T23:29:33.110 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.110 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.110 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.110 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.110 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.110 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.110 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.110 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.110 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.110 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.110 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.110 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.110 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.110 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.110 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.111 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 10 monclient: shutdown 2024-01-24T23:29:33.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:33.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 1 -- 172.21.15.115:0/162018120 >> v1:172.21.15.115:6789/0 conn(0x7fe9200a19f0 legacy=0x7fe92009bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 1 -- 172.21.15.115:0/162018120 shutdown_connections 2024-01-24T23:29:33.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 1 -- 172.21.15.115:0/162018120 >> 172.21.15.115:0/162018120 conn(0x7fe9200a65e0 msgr2=0x7fe9200a2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:33.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 1 -- 172.21.15.115:0/162018120 shutdown_connections 2024-01-24T23:29:33.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 1 -- 172.21.15.115:0/162018120 wait complete. 2024-01-24T23:29:33.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 10 monclient: build_initial_monmap 2024-01-24T23:29:33.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.095+0000 7fe925b8c640 10 monclient: monmap: 2024-01-24T23:29:33.111 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:33.111 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:33.111 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:33.111 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:33.111 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:33.111 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:33.111 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:33.112 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:33.112 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:33.112 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:33.112 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:33.112 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:33.112 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:33.112 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:33.112 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:33.112 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe925b8c640 1 Processor -- start 2024-01-24T23:29:33.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe925b8c640 1 -- start start 2024-01-24T23:29:33.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe925b8c640 10 monclient: init 2024-01-24T23:29:33.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe925b8c640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:33.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe925b8c640 10 monclient: _add_conns ranks=[1,0,7,4,8,6,3,5,2] 2024-01-24T23:29:33.112 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe925b8c640 10 monclient(hunting): picked mon.noname-f con 0x7fe9200a19f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:29:33.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe925b8c640 10 monclient(hunting): picked mon.noname-a con 0x7fe9200a1dd0 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:33.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe925b8c640 10 monclient(hunting): picked mon.noname-i con 0x7fe92013cef0 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:33.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe925b8c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe920138520 con 0x7fe9200a1dd0 2024-01-24T23:29:33.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe925b8c640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe92009ca40 con 0x7fe9200a19f0 2024-01-24T23:29:33.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe925b8c640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe92009c680 con 0x7fe92013cef0 2024-01-24T23:29:33.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe925b8c640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:33.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe925b8c640 10 monclient(hunting): authenticate will time out at 1452.024574s 2024-01-24T23:29:33.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe91effd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fe9200a1dd0 0x7fe9201d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39094/0 (socket says 172.21.15.115:39094) 2024-01-24T23:29:33.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe91effd640 1 -- 172.21.15.115:0/398485031 learned_addr learned my addr 172.21.15.115:0/398485031 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:33.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1076778936 0 0) 0x7fe920138520 con 0x7fe9200a1dd0 2024-01-24T23:29:33.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient(hunting): my global_id is 154107 2024-01-24T23:29:33.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe90c003530 con 0x7fe9200a1dd0 2024-01-24T23:29:33.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 728786172 0 0) 0x7fe90c003530 con 0x7fe9200a1dd0 2024-01-24T23:29:33.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe920138520 con 0x7fe9200a1dd0 2024-01-24T23:29:33.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe910001aa0 con 0x7fe9200a1dd0 2024-01-24T23:29:33.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:29:33.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient(hunting): dump: 2024-01-24T23:29:33.114 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.114 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.114 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.114 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.114 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.114 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.114 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.114 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.114 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.115 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.115 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.115 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.115 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.115 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.115 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.115 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:33.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 102448799 0 0) 0x7fe920138520 con 0x7fe9200a1dd0 2024-01-24T23:29:33.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 >> v1:172.21.15.132:6792/0 conn(0x7fe92013cef0 legacy=0x7fe9201d39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 >> v1:172.21.15.132:6789/0 conn(0x7fe9200a19f0 legacy=0x7fe92013c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient: _finish_hunting 0 2024-01-24T23:29:33.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 1 monclient: found mon.a 2024-01-24T23:29:33.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:33.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:29:33.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe920138c30 con 0x7fe9200a1dd0 2024-01-24T23:29:33.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient: _finish_auth 0 2024-01-24T23:29:33.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient: _check_auth_tickets 2024-01-24T23:29:33.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:33.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe925b8c640 5 monclient: authenticate success, global_id 154107 2024-01-24T23:29:33.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe925b8c640 10 monclient: _renew_subs 2024-01-24T23:29:33.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe925b8c640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:29:33.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe925b8c640 1 -- 172.21.15.115:0/398485031 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92013a680 con 0x7fe9200a1dd0 2024-01-24T23:29:33.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe925b8c640 10 monclient: _renew_subs 2024-01-24T23:29:33.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe925b8c640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:29:33.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe925b8c640 1 -- 172.21.15.115:0/398485031 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9201cb980 con 0x7fe9200a1dd0 2024-01-24T23:29:33.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe910004aa0 con 0x7fe9200a1dd0 2024-01-24T23:29:33.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:33.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe910004fd0 con 0x7fe9200a1dd0 2024-01-24T23:29:33.116 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:29:33.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe91dffb640 4 set_mon_vals no callback set 2024-01-24T23:29:33.117 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient: dump: 2024-01-24T23:29:33.117 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.117 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.117 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.117 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.117 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.117 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.117 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.117 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.117 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.117 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.117 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.117 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.118 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.118 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.118 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.118 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe925b8c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:33.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe925b8c640 1 -- 172.21.15.115:0/398485031 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe9000065d0 con 0x7fe900002870 2024-01-24T23:29:33.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fe9100051b0 con 0x7fe9200a1dd0 2024-01-24T23:29:33.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7fe910044540 con 0x7fe9200a1dd0 2024-01-24T23:29:33.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473413023 0 0) 0x7fe9000065d0 con 0x7fe900002870 2024-01-24T23:29:33.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient(hunting): my global_id is 154116 2024-01-24T23:29:33.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe920138520 con 0x7fe900002870 2024-01-24T23:29:33.118 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 769078457 0 0) 0x7fe920138520 con 0x7fe900002870 2024-01-24T23:29:33.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe9000065d0 con 0x7fe900002870 2024-01-24T23:29:33.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe9140010d0 con 0x7fe900002870 2024-01-24T23:29:33.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:29:33.119 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient: dump: 2024-01-24T23:29:33.119 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.119 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.119 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.119 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.119 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.119 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.119 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.119 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.119 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.119 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.120 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.120 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.120 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.120 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.120 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.120 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe91ffff640 10 monclient: get_auth_request con 0x7fe90c036fb0 auth_method 0 2024-01-24T23:29:33.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2306120528 0 0) 0x7fe9000065d0 con 0x7fe900002870 2024-01-24T23:29:33.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 --> v1:172.21.15.115:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe90000a120 con 0x7fe900002870 2024-01-24T23:29:33.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 <== mon.0 v1:172.21.15.115:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fe9140016d0 con 0x7fe900002870 2024-01-24T23:29:33.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:33.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:33.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 >> v1:172.21.15.115:6789/0 conn(0x7fe900002870 legacy=0x7fe900004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.120 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.099+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 >> v1:172.21.15.115:6789/0 conn(0x7fe900002870 legacy=0x7fe900004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:33.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe925b8c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:29:33.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe925b8c640 1 -- 172.21.15.115:0/398485031 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe900009df0 con 0x7fe9000090b0 2024-01-24T23:29:33.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975485301 0 0) 0x7fe900009df0 con 0x7fe9000090b0 2024-01-24T23:29:33.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe907fff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe907fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe907fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe907fff640 10 monclient(hunting): my global_id is 154125 2024-01-24T23:29:33.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9000065d0 con 0x7fe9000090b0 2024-01-24T23:29:33.122 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1581589181 0 0) 0x7fe9000065d0 con 0x7fe9000090b0 2024-01-24T23:29:33.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe900009df0 con 0x7fe9000090b0 2024-01-24T23:29:33.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fe914005530 con 0x7fe9000090b0 2024-01-24T23:29:33.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe907fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe907fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:29:33.123 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe907fff640 10 monclient: dump: 2024-01-24T23:29:33.123 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.123 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.123 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.123 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.123 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.123 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.123 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.123 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.123 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.124 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.124 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.124 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.124 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.124 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.124 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.124 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2178675738 0 0) 0x7fe900009df0 con 0x7fe9000090b0 2024-01-24T23:29:33.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 --> v1:172.21.15.115:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe90000c310 con 0x7fe9000090b0 2024-01-24T23:29:33.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 <== mon.0 v1:172.21.15.115:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 2992124959) 0x7fe914007750 con 0x7fe9000090b0 2024-01-24T23:29:33.124 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe907fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:29:33.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe907fff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:33.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 >> v1:172.21.15.115:6789/0 conn(0x7fe9000090b0 legacy=0x7fe900009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.125 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe907fff640 1 -- 172.21.15.115:0/398485031 >> v1:172.21.15.115:6789/0 conn(0x7fe9000090b0 legacy=0x7fe900009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:33.125 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:29:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:29:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 48, 2024-01-24T23:29:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:29:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:29:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:29:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:29:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:29:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:29:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:29:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:29:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:29:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:29:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 3, 2024-01-24T23:29:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:29:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:29:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:33.128 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:33.128 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:33.128 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.128 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:29:33.128 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:29:33.128 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:33.128 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:33.128 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:33.128 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:33.128 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:33.128 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:33.128 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:33.128 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:33.128 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:33.129 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:33.129 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:33.129 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.129 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:29:33.129 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:29:33.129 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:29:33.129 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:29:33.129 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:29:33.129 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:29:33.129 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:33.129 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:33.129 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:29:33.129 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:29:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:29:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:29:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:29:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:29:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:29:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:29:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:33.131 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:33.131 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:33.131 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:33.131 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.131 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:29:33.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.131 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:29:33.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.131 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:33.131 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:33.131 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.131 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.131 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.132 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:29:33.132 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.132 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.132 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.132 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:29:33.132 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:29:33.132 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.132 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.132 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.132 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:33.132 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:29:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:29:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:29:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:29:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:29:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:29:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:29:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:29:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:29:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:29:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:29:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:29:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:29:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:29:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:29:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.139 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:33.139 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:33.139 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.139 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.139 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.139 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:29:33.139 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.139 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.139 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.139 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:29:33.139 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:29:33.139 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:29:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:29:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:29:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:29:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:29:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:29:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:29:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:29:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:29:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:29:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.144 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:29:33.144 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:29:33.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe925b8c640 1 -- 172.21.15.115:0/398485031 >> v1:172.21.15.132:6812/1835401922 conn(0x7fe90c036fb0 legacy=0x7fe90c039470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe925b8c640 10 monclient: shutdown 2024-01-24T23:29:33.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe925b8c640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:33.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe925b8c640 1 -- 172.21.15.115:0/398485031 >> v1:172.21.15.115:6789/0 conn(0x7fe9200a1dd0 legacy=0x7fe9201d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe91ffff640 1 -- 172.21.15.115:0/398485031 reap_dead start 2024-01-24T23:29:33.145 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe91ffff640 1 -- 172.21.15.115:0/398485031 reap_dead start 2024-01-24T23:29:33.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe925b8c640 1 -- 172.21.15.115:0/398485031 shutdown_connections 2024-01-24T23:29:33.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe925b8c640 1 -- 172.21.15.115:0/398485031 >> 172.21.15.115:0/398485031 conn(0x7fe9200a65e0 msgr2=0x7fe92009d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:33.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe925b8c640 1 -- 172.21.15.115:0/398485031 shutdown_connections 2024-01-24T23:29:33.146 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.107+0000 7fe925b8c640 1 -- 172.21.15.115:0/398485031 wait complete. 2024-01-24T23:29:33.146 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:29:33.146 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-01-24T23:29:33.146 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-01-24T23:29:33.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 10 monclient: get_monmap_and_config 2024-01-24T23:29:33.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 10 monclient: build_initial_monmap 2024-01-24T23:29:33.252 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 10 monclient: monmap: 2024-01-24T23:29:33.252 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:33.252 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.252 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:33.255044+0000 2024-01-24T23:29:33.253 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:33.255044+0000 2024-01-24T23:29:33.253 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:33.253 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.253 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:33.253 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:33.253 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:33.253 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:33.253 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:33.253 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:33.253 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:33.253 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:33.253 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:33.253 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 1 Processor -- start 2024-01-24T23:29:33.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 1 -- start start 2024-01-24T23:29:33.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 10 monclient: init 2024-01-24T23:29:33.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:33.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 10 monclient: _add_conns ranks=[5,3,7,4,2,8,0,6,1] 2024-01-24T23:29:33.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 10 monclient(hunting): picked mon.noname-h con 0x7f35f0136d80 addr v1:172.21.15.132:6791/0 2024-01-24T23:29:33.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 10 monclient(hunting): picked mon.noname-g con 0x7f35f01368c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:29:33.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 10 monclient(hunting): picked mon.noname-i con 0x7f35f013d360 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:33.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f004ed70 con 0x7f35f01368c0 2024-01-24T23:29:33.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f009f280 con 0x7f35f0136d80 2024-01-24T23:29:33.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f009edf0 con 0x7f35f013d360 2024-01-24T23:29:33.256 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 10 monclient(hunting): authenticate will time out at 1452.177639s 2024-01-24T23:29:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35ee7fc640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f35f01368c0 0x7f35f013bc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.115:57276/0 (socket says 172.21.15.115:57276) 2024-01-24T23:29:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35ee7fc640 1 -- 172.21.15.115:0/3720185096 learned_addr learned my addr 172.21.15.115:0/3720185096 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 1 -- 172.21.15.115:0/3720185096 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3041594480 0 0) 0x7f35f009f280 con 0x7f35f0136d80 2024-01-24T23:29:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 10 monclient(hunting): my global_id is 7043 2024-01-24T23:29:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 1 -- 172.21.15.115:0/3720185096 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35d8003530 con 0x7f35f0136d80 2024-01-24T23:29:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 1 -- 172.21.15.115:0/3720185096 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3662192426 0 0) 0x7f35f004ed70 con 0x7f35f01368c0 2024-01-24T23:29:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 10 monclient(hunting): my global_id is 7095 2024-01-24T23:29:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 1 -- 172.21.15.115:0/3720185096 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35f009f280 con 0x7f35f01368c0 2024-01-24T23:29:33.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 1 -- 172.21.15.115:0/3720185096 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 582546665 0 0) 0x7f35f009edf0 con 0x7f35f013d360 2024-01-24T23:29:33.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 10 monclient(hunting): my global_id is 154114 2024-01-24T23:29:33.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 1 -- 172.21.15.115:0/3720185096 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35f004ed70 con 0x7f35f013d360 2024-01-24T23:29:33.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 1 -- 172.21.15.115:0/3720185096 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 835433721 0 0) 0x7f35f009f280 con 0x7f35f01368c0 2024-01-24T23:29:33.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 1 -- 172.21.15.115:0/3720185096 >> v1:172.21.15.132:6792/0 conn(0x7f35f013d360 legacy=0x7f35f013f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 1 -- 172.21.15.115:0/3720185096 >> v1:172.21.15.132:6791/0 conn(0x7f35f0136d80 legacy=0x7f35f0137160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 10 monclient: _finish_hunting 0 2024-01-24T23:29:33.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 1 monclient: found mon.noname-g 2024-01-24T23:29:33.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:33.258 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:29:33.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 1 -- 172.21.15.115:0/3720185096 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f006d4d0 con 0x7f35f01368c0 2024-01-24T23:29:33.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 10 monclient: _finish_auth 0 2024-01-24T23:29:33.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 10 monclient: _check_auth_tickets 2024-01-24T23:29:33.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:33.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 1 -- 172.21.15.115:0/3720185096 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f35e8001b20 con 0x7f35f01368c0 2024-01-24T23:29:33.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:29:33.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 10 monclient: dump: 2024-01-24T23:29:33.259 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.259 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.259 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.259 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.259 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.259 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.259 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.259 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.260 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.260 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.260 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.260 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.260 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.260 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.260 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.260 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 5 monclient: authenticate success, global_id 7095 2024-01-24T23:29:33.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:33.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 1 -- 172.21.15.115:0/3720185096 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f35e8001d40 con 0x7f35f01368c0 2024-01-24T23:29:33.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:33.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:33.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 4 set_mon_vals no callback set 2024-01-24T23:29:33.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 1 -- 172.21.15.115:0/3720185096 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f35e8004f70 con 0x7f35f01368c0 2024-01-24T23:29:33.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:29:33.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35edffb640 10 monclient: dump: 2024-01-24T23:29:33.261 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.261 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.261 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.261 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.261 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.261 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.261 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.262 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.262 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.262 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.262 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.262 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.262 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.262 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.262 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.262 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 10 monclient: shutdown 2024-01-24T23:29:33.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:33.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 1 -- 172.21.15.115:0/3720185096 >> v1:172.21.15.132:6790/0 conn(0x7f35f01368c0 legacy=0x7f35f013bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 1 -- 172.21.15.115:0/3720185096 shutdown_connections 2024-01-24T23:29:33.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 1 -- 172.21.15.115:0/3720185096 >> 172.21.15.115:0/3720185096 conn(0x7f35f00a5d40 msgr2=0x7f35f00a2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:33.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 1 -- 172.21.15.115:0/3720185096 shutdown_connections 2024-01-24T23:29:33.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.251+0000 7f35f5701640 1 -- 172.21.15.115:0/3720185096 wait complete. 2024-01-24T23:29:33.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35f5701640 10 monclient: build_initial_monmap 2024-01-24T23:29:33.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35f5701640 10 monclient: monmap: 2024-01-24T23:29:33.268 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:33.268 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:33.268 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:33.268 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:33.268 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:33.268 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:33.268 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:33.269 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:33.269 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:33.269 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:33.269 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:33.269 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:33.269 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:33.269 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:33.269 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:33.269 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35f5701640 1 Processor -- start 2024-01-24T23:29:33.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35f5701640 1 -- start start 2024-01-24T23:29:33.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35f5701640 10 monclient: init 2024-01-24T23:29:33.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35f5701640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:33.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35f5701640 10 monclient: _add_conns ranks=[2,4,0,5,7,8,3,6,1] 2024-01-24T23:29:33.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35f5701640 10 monclient(hunting): picked mon.noname-b con 0x7f35f01368c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:29:33.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35f5701640 10 monclient(hunting): picked mon.noname-c con 0x7f35f0136d80 addr v1:172.21.15.115:6791/0 2024-01-24T23:29:33.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35f5701640 10 monclient(hunting): picked mon.noname-a con 0x7f35f013d360 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:33.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35f5701640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f009f920 con 0x7f35f013d360 2024-01-24T23:29:33.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35f5701640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f0131420 con 0x7f35f01368c0 2024-01-24T23:29:33.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35f5701640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f0131700 con 0x7f35f0136d80 2024-01-24T23:29:33.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35f5701640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:33.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35f5701640 10 monclient(hunting): authenticate will time out at 1452.181447s 2024-01-24T23:29:33.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35eeffd640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f35f01368c0 0x7f35f01290c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:43036/0 (socket says 172.21.15.115:43036) 2024-01-24T23:29:33.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35ee7fc640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f35f0136d80 0x7f35f01d0720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:58918/0 (socket says 172.21.15.115:58918) 2024-01-24T23:29:33.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35eeffd640 1 -- 172.21.15.115:0/3815180956 learned_addr learned my addr 172.21.15.115:0/3815180956 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:33.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3243540217 0 0) 0x7f35f009f920 con 0x7f35f013d360 2024-01-24T23:29:33.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): my global_id is 154134 2024-01-24T23:29:33.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35dc003530 con 0x7f35f013d360 2024-01-24T23:29:33.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2291762787 0 0) 0x7f35f0131420 con 0x7f35f01368c0 2024-01-24T23:29:33.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): my global_id is 154100 2024-01-24T23:29:33.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35f009f920 con 0x7f35f01368c0 2024-01-24T23:29:33.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1552258881 0 0) 0x7f35f0131700 con 0x7f35f0136d80 2024-01-24T23:29:33.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): my global_id is 95224 2024-01-24T23:29:33.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35f0131420 con 0x7f35f0136d80 2024-01-24T23:29:33.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3673669870 0 0) 0x7f35f009f920 con 0x7f35f01368c0 2024-01-24T23:29:33.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f35f0131700 con 0x7f35f01368c0 2024-01-24T23:29:33.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f35e0001a80 con 0x7f35f01368c0 2024-01-24T23:29:33.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:29:33.272 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): dump: 2024-01-24T23:29:33.272 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.272 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.272 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.272 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.272 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.272 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.272 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.272 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.272 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.272 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.272 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.272 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.273 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.273 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.273 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.273 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:33.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1969141983 0 0) 0x7f35dc003530 con 0x7f35f013d360 2024-01-24T23:29:33.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f35f009f920 con 0x7f35f013d360 2024-01-24T23:29:33.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 879662132 0 0) 0x7f35f0131420 con 0x7f35f0136d80 2024-01-24T23:29:33.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f35dc003530 con 0x7f35f0136d80 2024-01-24T23:29:33.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f35e4001940 con 0x7f35f013d360 2024-01-24T23:29:33.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:29:33.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): dump: 2024-01-24T23:29:33.273 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.273 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.274 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.274 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.274 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.274 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.274 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.274 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.274 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.274 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.274 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.274 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.274 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.274 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.274 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.274 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f35e8001a20 con 0x7f35f0136d80 2024-01-24T23:29:33.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:29:33.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): dump: 2024-01-24T23:29:33.275 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.275 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.275 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.275 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.275 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.275 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.275 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.275 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.275 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.275 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.275 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.276 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.276 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.276 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.276 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.276 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3805992482 0 0) 0x7f35f0131700 con 0x7f35f01368c0 2024-01-24T23:29:33.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 >> v1:172.21.15.115:6791/0 conn(0x7f35f0136d80 legacy=0x7f35f01d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 >> v1:172.21.15.115:6789/0 conn(0x7f35f013d360 legacy=0x7f35f01d3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient: _finish_hunting 0 2024-01-24T23:29:33.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 monclient: found mon.b 2024-01-24T23:29:33.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:33.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:29:33.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f01319e0 con 0x7f35f01368c0 2024-01-24T23:29:33.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient: _finish_auth 0 2024-01-24T23:29:33.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient: _check_auth_tickets 2024-01-24T23:29:33.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:33.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35f5701640 5 monclient: authenticate success, global_id 154100 2024-01-24T23:29:33.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35f5701640 10 monclient: _renew_subs 2024-01-24T23:29:33.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35f5701640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:29:33.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35f5701640 1 -- 172.21.15.115:0/3815180956 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f0070be0 con 0x7f35f01368c0 2024-01-24T23:29:33.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35f5701640 10 monclient: _renew_subs 2024-01-24T23:29:33.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35f5701640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:29:33.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35f5701640 1 -- 172.21.15.115:0/3815180956 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f01310b0 con 0x7f35f01368c0 2024-01-24T23:29:33.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f35e0001d70 con 0x7f35f01368c0 2024-01-24T23:29:33.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:33.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f35e0003c10 con 0x7f35f01368c0 2024-01-24T23:29:33.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35ed7fa640 4 set_mon_vals no callback set 2024-01-24T23:29:33.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:29:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient: dump: 2024-01-24T23:29:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.279 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35f5701640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:33.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35f5701640 1 -- 172.21.15.115:0/3815180956 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f35d00065d0 con 0x7f35d0002870 2024-01-24T23:29:33.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1733793039 0 0) 0x7f35d00065d0 con 0x7f35d0002870 2024-01-24T23:29:33.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient(hunting): my global_id is 154109 2024-01-24T23:29:33.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35f0131700 con 0x7f35d0002870 2024-01-24T23:29:33.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1936369836 0 0) 0x7f35f0131700 con 0x7f35d0002870 2024-01-24T23:29:33.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f35d00065d0 con 0x7f35d0002870 2024-01-24T23:29:33.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f35e8005240 con 0x7f35d0002870 2024-01-24T23:29:33.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:29:33.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 10 monclient: dump: 2024-01-24T23:29:33.280 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.280 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.280 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.280 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.280 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.280 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.280 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.280 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.280 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.280 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.280 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.280 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.280 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.280 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.280 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.280 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f35e0003e10 con 0x7f35f01368c0 2024-01-24T23:29:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.255+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7f35e0044430 con 0x7f35f01368c0 2024-01-24T23:29:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.259+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1163162094 0 0) 0x7f35d00065d0 con 0x7f35d0002870 2024-01-24T23:29:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.259+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 --> v1:172.21.15.115:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35d000a120 con 0x7f35d0002870 2024-01-24T23:29:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.259+0000 7f35ef7fe640 10 monclient: get_auth_request con 0x7f35dc037e80 auth_method 0 2024-01-24T23:29:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.259+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 <== mon.2 v1:172.21.15.115:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f35e8005430 con 0x7f35d0002870 2024-01-24T23:29:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.259+0000 7f35d77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.259+0000 7f35d77fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.259+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 >> v1:172.21.15.115:6790/0 conn(0x7f35d0002870 legacy=0x7f35d0004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.259+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 >> v1:172.21.15.115:6790/0 conn(0x7f35d0002870 legacy=0x7f35d0004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35f5701640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:29:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35f5701640 1 -- 172.21.15.115:0/3815180956 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f35d0009e00 con 0x7f35d00090e0 2024-01-24T23:29:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1668211048 0 0) 0x7f35d0009e00 con 0x7f35d00090e0 2024-01-24T23:29:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35d77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35d77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35d77fe640 10 monclient(hunting): my global_id is 154118 2024-01-24T23:29:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35d00065d0 con 0x7f35d00090e0 2024-01-24T23:29:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1172362147 0 0) 0x7f35d00065d0 con 0x7f35d00090e0 2024-01-24T23:29:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f35d0009e00 con 0x7f35d00090e0 2024-01-24T23:29:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f35e8005ca0 con 0x7f35d00090e0 2024-01-24T23:29:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35d77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35d77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:29:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35d77fe640 10 monclient: dump: 2024-01-24T23:29:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.283 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.283 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.283 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.283 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.283 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.283 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.283 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.283 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.283 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.283 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1812306676 0 0) 0x7f35d0009e00 con 0x7f35d00090e0 2024-01-24T23:29:33.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 --> v1:172.21.15.115:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f35d000c310 con 0x7f35d00090e0 2024-01-24T23:29:33.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 <== mon.2 v1:172.21.15.115:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 4056544137) 0x7f35e8008080 con 0x7f35d00090e0 2024-01-24T23:29:33.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35d77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:29:33.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35d77fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:33.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 >> v1:172.21.15.115:6790/0 conn(0x7f35d00090e0 legacy=0x7f35d0009550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35d77fe640 1 -- 172.21.15.115:0/3815180956 >> v1:172.21.15.115:6790/0 conn(0x7f35d00090e0 legacy=0x7f35d0009550 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:33.284 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:29:33.284 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:33.284 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:33.284 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:29:33.284 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 48, 2024-01-24T23:29:33.284 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:29:33.284 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:29:33.284 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:29:33.284 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:29:33.285 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:29:33.285 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:29:33.285 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:29:33.285 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:29:33.285 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:29:33.285 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:29:33.285 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.285 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 3, 2024-01-24T23:29:33.285 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:33.285 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:29:33.285 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:29:33.285 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:33.285 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:33.285 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:33.286 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:33.286 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:33.286 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:33.286 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:33.286 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:33.286 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:33.286 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:33.286 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:33.286 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.286 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:29:33.286 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:29:33.286 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:33.286 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:33.286 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:33.287 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:33.287 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:33.287 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:33.287 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:33.287 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:33.287 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:33.287 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:33.287 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:33.287 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.287 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:29:33.287 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:29:33.287 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:29:33.287 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:29:33.287 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:29:33.288 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:29:33.288 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:33.288 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:33.288 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:29:33.288 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:29:33.288 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:29:33.288 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:29:33.288 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:29:33.288 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:29:33.288 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:29:33.288 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:33.288 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:29:33.288 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:33.288 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:33.289 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:33.289 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:33.289 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:33.289 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:33.289 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:33.289 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:33.289 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:33.289 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:33.289 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:33.289 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.289 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:29:33.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.289 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:29:33.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.289 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:33.290 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:33.290 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.290 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.290 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.290 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:29:33.290 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.290 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.290 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.290 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:29:33.290 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:29:33.290 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.290 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.290 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.291 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:33.291 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:33.291 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.291 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.291 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.291 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:29:33.291 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.291 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.291 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.291 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:29:33.292 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:29:33.292 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.292 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.292 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.292 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:33.292 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:33.292 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.292 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.293 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.293 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:29:33.293 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.293 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.293 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.293 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:29:33.293 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:29:33.293 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.293 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.293 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.293 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:33.294 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:33.294 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.294 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.294 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.294 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:29:33.294 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.294 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.294 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.294 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:29:33.294 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:29:33.294 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.294 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.295 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.295 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:33.295 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:33.295 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.295 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.295 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.295 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:29:33.295 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.295 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.295 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.296 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:29:33.296 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:29:33.296 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.296 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.296 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.296 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:33.296 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:33.296 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.296 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.296 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.296 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:29:33.297 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.297 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.297 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.297 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:29:33.297 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:29:33.297 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.297 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.297 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.297 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:33.297 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:33.297 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.298 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.298 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.298 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:29:33.298 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.298 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.298 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.298 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:29:33.298 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:29:33.298 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.298 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.298 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.299 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:33.299 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:33.299 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.299 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.299 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.299 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:29:33.299 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.299 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.299 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.299 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:29:33.299 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:29:33.299 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.299 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.300 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.300 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:33.300 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:33.300 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.300 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.300 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.300 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:29:33.300 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.300 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.300 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.301 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:29:33.301 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:29:33.301 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.301 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.301 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.301 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.301 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.301 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:29:33.301 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:29:33.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.301 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:33.301 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:33.301 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:33.301 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.302 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.302 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:29:33.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.302 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:33.302 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:33.302 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:29:33.302 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.302 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.302 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:29:33.302 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:29:33.303 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35f5701640 1 -- 172.21.15.115:0/3815180956 >> v1:172.21.15.132:6812/1835401922 conn(0x7f35dc037e80 legacy=0x7f35dc03a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35f5701640 10 monclient: shutdown 2024-01-24T23:29:33.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35f5701640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:33.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35f5701640 1 -- 172.21.15.115:0/3815180956 >> v1:172.21.15.115:6790/0 conn(0x7f35f01368c0 legacy=0x7f35f01290c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35ef7fe640 1 -- 172.21.15.115:0/3815180956 reap_dead start 2024-01-24T23:29:33.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35ef7fe640 1 -- 172.21.15.115:0/3815180956 reap_dead start 2024-01-24T23:29:33.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35f5701640 1 -- 172.21.15.115:0/3815180956 shutdown_connections 2024-01-24T23:29:33.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35f5701640 1 -- 172.21.15.115:0/3815180956 >> 172.21.15.115:0/3815180956 conn(0x7f35f00a5d40 msgr2=0x7f35f0074830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:33.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35f5701640 1 -- 172.21.15.115:0/3815180956 shutdown_connections 2024-01-24T23:29:33.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.263+0000 7f35f5701640 1 -- 172.21.15.115:0/3815180956 wait complete. 2024-01-24T23:29:33.305 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:29:33.305 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-01-24T23:29:33.305 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-01-24T23:29:33.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.403+0000 7fdffdfd6640 10 monclient: get_monmap_and_config 2024-01-24T23:29:33.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.403+0000 7fdffdfd6640 10 monclient: build_initial_monmap 2024-01-24T23:29:33.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 10 monclient: monmap: 2024-01-24T23:29:33.408 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:33.408 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.408 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:33.410919+0000 2024-01-24T23:29:33.408 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:33.410919+0000 2024-01-24T23:29:33.408 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:33.408 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.408 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:33.409 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:33.409 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:33.409 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:33.409 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:33.409 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:33.409 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:33.409 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:33.409 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:33.409 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 1 Processor -- start 2024-01-24T23:29:33.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 1 -- start start 2024-01-24T23:29:33.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 10 monclient: init 2024-01-24T23:29:33.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:33.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 10 monclient: _add_conns ranks=[6,8,1,4,3,7,5,2,0] 2024-01-24T23:29:33.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 10 monclient(hunting): picked mon.noname-d con 0x7fdff8133bf0 addr v1:172.21.15.115:6792/0 2024-01-24T23:29:33.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 10 monclient(hunting): picked mon.noname-e con 0x7fdff8133fd0 addr v1:172.21.15.115:6793/0 2024-01-24T23:29:33.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 10 monclient(hunting): picked mon.noname-f con 0x7fdff81343b0 addr v1:172.21.15.132:6789/0 2024-01-24T23:29:33.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff804ed70 con 0x7fdff81343b0 2024-01-24T23:29:33.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff80a4b60 con 0x7fdff8133bf0 2024-01-24T23:29:33.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff809ae50 con 0x7fdff8133fd0 2024-01-24T23:29:33.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:33.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 10 monclient(hunting): authenticate will time out at 1452.333461s 2024-01-24T23:29:33.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff77fe640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7fdff8133bf0 0x7fdff809bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:53674/0 (socket says 172.21.15.115:53674) 2024-01-24T23:29:33.418 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff77fe640 1 -- 172.21.15.115:0/4084588583 learned_addr learned my addr 172.21.15.115:0/4084588583 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:33.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff6ffd640 1 --1- 172.21.15.115:0/4084588583 >> v1:172.21.15.115:6793/0 conn(0x7fdff8133fd0 0x7fdff813bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:55898/0 (socket says 172.21.15.115:55898) 2024-01-24T23:29:33.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 1 -- 172.21.15.115:0/4084588583 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1565448866 0 0) 0x7fdff80a4b60 con 0x7fdff8133bf0 2024-01-24T23:29:33.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 10 monclient(hunting): my global_id is 7269 2024-01-24T23:29:33.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 1 -- 172.21.15.115:0/4084588583 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfe0003530 con 0x7fdff8133bf0 2024-01-24T23:29:33.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 1 -- 172.21.15.115:0/4084588583 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1579166744 0 0) 0x7fdff809ae50 con 0x7fdff8133fd0 2024-01-24T23:29:33.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 10 monclient(hunting): my global_id is 86318 2024-01-24T23:29:33.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 1 -- 172.21.15.115:0/4084588583 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdff80a4b60 con 0x7fdff8133fd0 2024-01-24T23:29:33.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 1 -- 172.21.15.115:0/4084588583 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 808549695 0 0) 0x7fdff80a4b60 con 0x7fdff8133fd0 2024-01-24T23:29:33.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 1 -- 172.21.15.115:0/4084588583 >> v1:172.21.15.115:6792/0 conn(0x7fdff8133bf0 legacy=0x7fdff809bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 1 -- 172.21.15.115:0/4084588583 >> v1:172.21.15.132:6789/0 conn(0x7fdff81343b0 legacy=0x7fdff813f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 10 monclient: _finish_hunting 0 2024-01-24T23:29:33.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 1 monclient: found mon.noname-e 2024-01-24T23:29:33.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:33.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:29:33.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 1 -- 172.21.15.115:0/4084588583 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff809b130 con 0x7fdff8133fd0 2024-01-24T23:29:33.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 10 monclient: _finish_auth 0 2024-01-24T23:29:33.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 10 monclient: _check_auth_tickets 2024-01-24T23:29:33.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:33.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 1 -- 172.21.15.115:0/4084588583 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdfec0017c0 con 0x7fdff8133fd0 2024-01-24T23:29:33.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:29:33.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 10 monclient: dump: 2024-01-24T23:29:33.420 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.421 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.421 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.421 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.421 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.421 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.421 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.421 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.421 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.421 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.421 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.421 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.421 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.421 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.421 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.421 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 5 monclient: authenticate success, global_id 86318 2024-01-24T23:29:33.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:33.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 1 -- 172.21.15.115:0/4084588583 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdfec001a60 con 0x7fdff8133fd0 2024-01-24T23:29:33.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:33.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:33.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 1 -- 172.21.15.115:0/4084588583 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdfec004b30 con 0x7fdff8133fd0 2024-01-24T23:29:33.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 4 set_mon_vals no callback set 2024-01-24T23:29:33.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:29:33.422 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdff67fc640 10 monclient: dump: 2024-01-24T23:29:33.422 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.422 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.422 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.422 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.422 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.423 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.423 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.423 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.423 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.423 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.423 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.423 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.423 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.423 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.423 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.423 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 10 monclient: shutdown 2024-01-24T23:29:33.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:33.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 1 -- 172.21.15.115:0/4084588583 >> v1:172.21.15.115:6793/0 conn(0x7fdff8133fd0 legacy=0x7fdff813bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 1 -- 172.21.15.115:0/4084588583 shutdown_connections 2024-01-24T23:29:33.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 1 -- 172.21.15.115:0/4084588583 >> 172.21.15.115:0/4084588583 conn(0x7fdff80662f0 msgr2=0x7fdff8066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:33.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 1 -- 172.21.15.115:0/4084588583 shutdown_connections 2024-01-24T23:29:33.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 1 -- 172.21.15.115:0/4084588583 wait complete. 2024-01-24T23:29:33.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 10 monclient: build_initial_monmap 2024-01-24T23:29:33.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 10 monclient: monmap: 2024-01-24T23:29:33.424 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:33.424 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:33.424 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:33.424 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:33.424 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:33.424 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:33.424 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:33.424 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:33.424 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:33.424 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:33.425 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:33.425 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:33.425 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:33.425 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:33.425 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:33.425 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 1 Processor -- start 2024-01-24T23:29:33.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 1 -- start start 2024-01-24T23:29:33.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.407+0000 7fdffdfd6640 10 monclient: init 2024-01-24T23:29:33.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdffdfd6640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:33.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdffdfd6640 10 monclient: _add_conns ranks=[3,7,1,0,4,5,2,8,6] 2024-01-24T23:29:33.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdffdfd6640 10 monclient(hunting): picked mon.noname-g con 0x7fdff8133bf0 addr v1:172.21.15.132:6790/0 2024-01-24T23:29:33.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdffdfd6640 10 monclient(hunting): picked mon.noname-i con 0x7fdff8133fd0 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:33.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdffdfd6640 10 monclient(hunting): picked mon.noname-f con 0x7fdff81343b0 addr v1:172.21.15.132:6789/0 2024-01-24T23:29:33.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdffdfd6640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff80a5990 con 0x7fdff81343b0 2024-01-24T23:29:33.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdffdfd6640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff80a2520 con 0x7fdff8133bf0 2024-01-24T23:29:33.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdffdfd6640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff81c8930 con 0x7fdff8133fd0 2024-01-24T23:29:33.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdffdfd6640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:33.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdffdfd6640 10 monclient(hunting): authenticate will time out at 1452.336366s 2024-01-24T23:29:33.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdff7fff640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fdff81343b0 0x7fdff81d3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.115:37044/0 (socket says 172.21.15.115:37044) 2024-01-24T23:29:33.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdff7fff640 1 -- 172.21.15.115:0/53801464 learned_addr learned my addr 172.21.15.115:0/53801464 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:33.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdff77fe640 1 --1- 172.21.15.115:0/53801464 >> v1:172.21.15.132:6790/0 conn(0x7fdff8133bf0 0x7fdff8132f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.115:57280/0 (socket says 172.21.15.115:57280) 2024-01-24T23:29:33.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3294735582 0 0) 0x7fdff80a2520 con 0x7fdff8133bf0 2024-01-24T23:29:33.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): my global_id is 7104 2024-01-24T23:29:33.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfe4003530 con 0x7fdff8133bf0 2024-01-24T23:29:33.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1037736640 0 0) 0x7fdff80a5990 con 0x7fdff81343b0 2024-01-24T23:29:33.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): my global_id is 124624 2024-01-24T23:29:33.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdff80a2520 con 0x7fdff81343b0 2024-01-24T23:29:33.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3589945990 0 0) 0x7fdff81c8930 con 0x7fdff8133fd0 2024-01-24T23:29:33.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): my global_id is 154123 2024-01-24T23:29:33.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdff80a5990 con 0x7fdff8133fd0 2024-01-24T23:29:33.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4161571041 0 0) 0x7fdff80a2520 con 0x7fdff81343b0 2024-01-24T23:29:33.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdff81c8930 con 0x7fdff81343b0 2024-01-24T23:29:33.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdff0001d60 con 0x7fdff81343b0 2024-01-24T23:29:33.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:29:33.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): dump: 2024-01-24T23:29:33.428 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.428 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.428 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.428 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.428 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.428 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.428 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.428 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.428 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.428 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.429 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.429 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.429 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.429 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.429 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.429 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:33.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1306596299 0 0) 0x7fdfe4003530 con 0x7fdff8133bf0 2024-01-24T23:29:33.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdff80a2520 con 0x7fdff8133bf0 2024-01-24T23:29:33.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 439381860 0 0) 0x7fdff80a5990 con 0x7fdff8133fd0 2024-01-24T23:29:33.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdfe4003530 con 0x7fdff8133fd0 2024-01-24T23:29:33.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdfe8001820 con 0x7fdff8133bf0 2024-01-24T23:29:33.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:29:33.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): dump: 2024-01-24T23:29:33.430 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.430 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.430 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.430 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.430 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.430 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.430 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.430 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.430 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.430 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.430 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.430 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.430 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.430 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.430 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.431 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdfec004b30 con 0x7fdff8133fd0 2024-01-24T23:29:33.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-01-24T23:29:33.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): dump: 2024-01-24T23:29:33.431 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.431 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.431 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.431 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.431 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.431 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.431 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.431 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.431 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.431 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.432 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.432 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.432 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.432 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.432 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.432 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1992455764 0 0) 0x7fdff81c8930 con 0x7fdff81343b0 2024-01-24T23:29:33.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 >> v1:172.21.15.132:6792/0 conn(0x7fdff8133fd0 legacy=0x7fdff81d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 >> v1:172.21.15.132:6790/0 conn(0x7fdff8133bf0 legacy=0x7fdff8132f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient: _finish_hunting 0 2024-01-24T23:29:33.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 monclient: found mon.f 2024-01-24T23:29:33.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:33.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:29:33.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff809c810 con 0x7fdff81343b0 2024-01-24T23:29:33.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient: _finish_auth 0 2024-01-24T23:29:33.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient: _check_auth_tickets 2024-01-24T23:29:33.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:33.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdffdfd6640 5 monclient: authenticate success, global_id 124624 2024-01-24T23:29:33.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdffdfd6640 10 monclient: _renew_subs 2024-01-24T23:29:33.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdffdfd6640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:29:33.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdffdfd6640 1 -- 172.21.15.115:0/53801464 --> v1:172.21.15.132:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff8120960 con 0x7fdff81343b0 2024-01-24T23:29:33.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdffdfd6640 10 monclient: _renew_subs 2024-01-24T23:29:33.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdffdfd6640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.132:6789/0 2024-01-24T23:29:33.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdffdfd6640 1 -- 172.21.15.115:0/53801464 --> v1:172.21.15.132:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff8133690 con 0x7fdff81343b0 2024-01-24T23:29:33.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 <== mon.1 v1:172.21.15.132:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdff0003c50 con 0x7fdff81343b0 2024-01-24T23:29:33.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:33.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdff5ffb640 4 set_mon_vals no callback set 2024-01-24T23:29:33.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 <== mon.1 v1:172.21.15.132:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdff0004180 con 0x7fdff81343b0 2024-01-24T23:29:33.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:29:33.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient: dump: 2024-01-24T23:29:33.434 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.434 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.434 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.434 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.434 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.434 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.434 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.434 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.434 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.434 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.434 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.434 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.434 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.435 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.435 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.435 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdffdfd6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:33.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdffdfd6640 1 -- 172.21.15.115:0/53801464 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdfd80065d0 con 0x7fdfd8002870 2024-01-24T23:29:33.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792900590 0 0) 0x7fdfd80065d0 con 0x7fdfd8002870 2024-01-24T23:29:33.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient(hunting): my global_id is 95233 2024-01-24T23:29:33.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdff81c8930 con 0x7fdfd8002870 2024-01-24T23:29:33.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1690370659 0 0) 0x7fdff81c8930 con 0x7fdfd8002870 2024-01-24T23:29:33.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdfd80065d0 con 0x7fdfd8002870 2024-01-24T23:29:33.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdfe8001d40 con 0x7fdfd8002870 2024-01-24T23:29:33.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:29:33.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 10 monclient: dump: 2024-01-24T23:29:33.436 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.436 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.436 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.436 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.436 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.436 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.436 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.436 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.436 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.436 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.436 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.436 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.436 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.436 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.437 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.437 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4111044995 0 0) 0x7fdfd80065d0 con 0x7fdfd8002870 2024-01-24T23:29:33.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 --> v1:172.21.15.115:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdfd800a120 con 0x7fdfd8002870 2024-01-24T23:29:33.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.411+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 <== mon.1 v1:172.21.15.132:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fdff0004360 con 0x7fdff81343b0 2024-01-24T23:29:33.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.415+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 <== mon.1 v1:172.21.15.132:6789/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7fdff0044880 con 0x7fdff81343b0 2024-01-24T23:29:33.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.415+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 <== mon.4 v1:172.21.15.115:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fdfe8004f00 con 0x7fdfd8002870 2024-01-24T23:29:33.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.415+0000 7fdfdffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:33.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.415+0000 7fdfdffff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:33.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.415+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 >> v1:172.21.15.115:6791/0 conn(0x7fdfd8002870 legacy=0x7fdfd8004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.415+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 >> v1:172.21.15.115:6791/0 conn(0x7fdfd8002870 legacy=0x7fdfd8004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:33.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.415+0000 7fdff6ffd640 10 monclient: get_auth_request con 0x7fdfe4037fa0 auth_method 0 2024-01-24T23:29:33.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdffdfd6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:29:33.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdffdfd6640 1 -- 172.21.15.115:0/53801464 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdfd8009e50 con 0x7fdfd8009110 2024-01-24T23:29:33.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4214666535 0 0) 0x7fdfd8009e50 con 0x7fdfd8009110 2024-01-24T23:29:33.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdfdffff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdfdffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.439 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdfdffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdfdffff640 10 monclient(hunting): my global_id is 95242 2024-01-24T23:29:33.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfd80065d0 con 0x7fdfd8009110 2024-01-24T23:29:33.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3949732107 0 0) 0x7fdfd80065d0 con 0x7fdfd8009110 2024-01-24T23:29:33.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdfd8009e50 con 0x7fdfd8009110 2024-01-24T23:29:33.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdfe8005620 con 0x7fdfd8009110 2024-01-24T23:29:33.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdfdffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdfdffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:29:33.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdfdffff640 10 monclient: dump: 2024-01-24T23:29:33.440 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.440 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.440 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.440 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.440 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.440 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.440 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.441 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.441 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.441 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.441 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.441 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.441 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.441 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.441 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.441 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 223761347 0 0) 0x7fdfd8009e50 con 0x7fdfd8009110 2024-01-24T23:29:33.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 --> v1:172.21.15.115:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdfd800c2d0 con 0x7fdfd8009110 2024-01-24T23:29:33.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 <== mon.4 v1:172.21.15.115:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 827958685) 0x7fdfe8005920 con 0x7fdfd8009110 2024-01-24T23:29:33.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdfdffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:29:33.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdfdffff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:33.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 >> v1:172.21.15.115:6791/0 conn(0x7fdfd8009110 legacy=0x7fdfd80095a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdfdffff640 1 -- 172.21.15.115:0/53801464 >> v1:172.21.15.115:6791/0 conn(0x7fdfd8009110 legacy=0x7fdfd80095a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:33.442 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:29:33.442 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:33.442 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:33.442 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:29:33.442 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 48, 2024-01-24T23:29:33.442 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:29:33.442 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:29:33.442 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:29:33.442 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:29:33.443 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:29:33.443 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:29:33.443 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:29:33.443 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:29:33.443 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:29:33.443 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:29:33.443 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.443 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 3, 2024-01-24T23:29:33.443 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:33.443 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:29:33.443 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:29:33.443 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:33.444 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:33.444 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:33.444 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:33.444 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:33.444 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:33.444 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:33.444 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:33.445 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:33.445 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:33.445 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:33.445 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.445 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:29:33.445 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:29:33.445 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:33.445 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:33.445 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:33.445 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:33.445 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:33.445 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:33.445 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:33.445 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:33.446 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:33.446 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:33.446 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:33.446 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.446 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:29:33.446 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:29:33.446 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:29:33.446 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:29:33.446 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:29:33.446 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:29:33.446 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:33.446 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:33.446 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:29:33.446 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:29:33.447 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:29:33.447 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:29:33.447 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:29:33.447 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:29:33.447 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:29:33.447 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:33.447 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:29:33.447 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:33.447 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:33.447 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:33.447 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:33.447 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:33.447 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:33.447 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:33.448 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:33.448 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:33.448 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:33.448 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:33.448 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.448 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:29:33.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.448 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:29:33.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.448 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:33.448 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:33.448 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.448 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.449 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.449 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:29:33.449 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.449 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.449 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.449 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:29:33.449 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:29:33.449 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.449 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.449 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.449 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:33.449 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:33.450 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.450 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.450 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.450 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:29:33.450 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.450 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.450 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.450 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:29:33.450 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:29:33.450 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.450 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.450 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.451 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:33.451 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:33.451 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.451 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.451 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.451 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:29:33.451 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.451 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.451 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.451 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:29:33.451 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:29:33.452 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.452 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.452 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.452 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:33.452 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:33.452 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.452 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.452 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.452 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:29:33.452 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.453 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.453 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.453 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:29:33.453 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:29:33.453 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.453 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.453 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.453 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:33.453 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:33.454 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.454 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.454 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.454 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:29:33.454 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.454 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.454 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.454 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:29:33.454 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:29:33.454 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.454 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.454 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.455 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:33.455 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:33.455 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.455 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.455 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.455 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:29:33.455 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.455 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.455 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.455 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:29:33.455 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:29:33.455 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.456 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.456 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.456 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:33.456 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:33.456 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.456 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.456 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.456 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:29:33.456 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.456 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.456 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.457 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:29:33.457 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:29:33.457 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.457 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.457 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.457 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:33.457 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:33.457 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.457 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.457 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.458 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:29:33.458 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.458 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.458 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.458 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:29:33.458 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:29:33.458 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.458 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.458 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.458 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:33.458 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:33.458 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.458 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.459 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.459 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:29:33.459 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.459 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.459 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.459 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:29:33.459 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:29:33.459 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.459 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.459 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.459 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.459 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.460 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:29:33.460 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:29:33.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.460 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:33.460 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:33.460 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:33.460 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.460 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.460 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:29:33.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.460 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:33.460 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:33.460 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:29:33.460 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.460 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.461 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:29:33.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.461 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:33.461 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:33.461 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:33.461 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.461 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.461 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:29:33.461 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:29:33.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdffdfd6640 1 -- 172.21.15.115:0/53801464 >> v1:172.21.15.132:6812/1835401922 conn(0x7fdfe4037fa0 legacy=0x7fdfe403a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdffdfd6640 10 monclient: shutdown 2024-01-24T23:29:33.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdffdfd6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:33.462 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdffdfd6640 1 -- 172.21.15.115:0/53801464 >> v1:172.21.15.132:6789/0 conn(0x7fdff81343b0 legacy=0x7fdff81d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdff7fff640 1 -- 172.21.15.115:0/53801464 reap_dead start 2024-01-24T23:29:33.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdff7fff640 1 -- 172.21.15.115:0/53801464 reap_dead start 2024-01-24T23:29:33.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdffdfd6640 1 -- 172.21.15.115:0/53801464 shutdown_connections 2024-01-24T23:29:33.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdffdfd6640 1 -- 172.21.15.115:0/53801464 >> 172.21.15.115:0/53801464 conn(0x7fdff80662f0 msgr2=0x7fdff80a8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:33.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdffdfd6640 1 -- 172.21.15.115:0/53801464 shutdown_connections 2024-01-24T23:29:33.463 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.419+0000 7fdffdfd6640 1 -- 172.21.15.115:0/53801464 wait complete. 2024-01-24T23:29:33.463 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:29:33.463 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-01-24T23:29:33.463 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-01-24T23:29:33.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.579+0000 7f74f466a640 10 monclient: get_monmap_and_config 2024-01-24T23:29:33.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.579+0000 7f74f466a640 10 monclient: build_initial_monmap 2024-01-24T23:29:33.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 10 monclient: monmap: 2024-01-24T23:29:33.584 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:33.584 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.584 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:33.586969+0000 2024-01-24T23:29:33.585 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:33.586969+0000 2024-01-24T23:29:33.585 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:33.585 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.585 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:33.585 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:33.585 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:33.585 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:33.585 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:33.585 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:33.585 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:33.585 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:33.585 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:33.585 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 1 Processor -- start 2024-01-24T23:29:33.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 1 -- start start 2024-01-24T23:29:33.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 10 monclient: init 2024-01-24T23:29:33.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:33.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 10 monclient: _add_conns ranks=[6,2,5,7,0,1,3,4,8] 2024-01-24T23:29:33.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 10 monclient(hunting): picked mon.noname-d con 0x7f74ec098580 addr v1:172.21.15.115:6792/0 2024-01-24T23:29:33.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 10 monclient(hunting): picked mon.noname-b con 0x7f74ec136f40 addr v1:172.21.15.115:6790/0 2024-01-24T23:29:33.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 10 monclient(hunting): picked mon.noname-h con 0x7f74ec137360 addr v1:172.21.15.132:6791/0 2024-01-24T23:29:33.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74ec04ed70 con 0x7f74ec136f40 2024-01-24T23:29:33.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74ec072600 con 0x7f74ec137360 2024-01-24T23:29:33.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74ec099aa0 con 0x7f74ec098580 2024-01-24T23:29:33.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:33.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 10 monclient(hunting): authenticate will time out at 1452.509747s 2024-01-24T23:29:33.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f2406640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f74ec098580 0x7f74ec098960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:53676/0 (socket says 172.21.15.115:53676) 2024-01-24T23:29:33.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f2406640 1 -- 172.21.15.115:0/402743271 learned_addr learned my addr 172.21.15.115:0/402743271 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:33.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1c05640 1 --1- 172.21.15.115:0/402743271 >> v1:172.21.15.115:6790/0 conn(0x7f74ec136f40 0x7f74ec13bcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:43054/0 (socket says 172.21.15.115:43054) 2024-01-24T23:29:33.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 1 -- 172.21.15.115:0/402743271 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1533013047 0 0) 0x7f74ec099aa0 con 0x7f74ec098580 2024-01-24T23:29:33.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 10 monclient(hunting): my global_id is 7278 2024-01-24T23:29:33.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 1 -- 172.21.15.115:0/402743271 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74d8003530 con 0x7f74ec098580 2024-01-24T23:29:33.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 1 -- 172.21.15.115:0/402743271 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3688262642 0 0) 0x7f74ec04ed70 con 0x7f74ec136f40 2024-01-24T23:29:33.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 10 monclient(hunting): my global_id is 154127 2024-01-24T23:29:33.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 1 -- 172.21.15.115:0/402743271 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74ec099aa0 con 0x7f74ec136f40 2024-01-24T23:29:33.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 1 -- 172.21.15.115:0/402743271 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2712060127 0 0) 0x7f74d8003530 con 0x7f74ec098580 2024-01-24T23:29:33.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 1 -- 172.21.15.115:0/402743271 >> v1:172.21.15.132:6791/0 conn(0x7f74ec137360 legacy=0x7f74ec13f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 1 -- 172.21.15.115:0/402743271 >> v1:172.21.15.115:6790/0 conn(0x7f74ec136f40 legacy=0x7f74ec13bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 10 monclient: _finish_hunting 0 2024-01-24T23:29:33.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 1 monclient: found mon.noname-d 2024-01-24T23:29:33.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:33.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:29:33.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 1 -- 172.21.15.115:0/402743271 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ec0997c0 con 0x7f74ec098580 2024-01-24T23:29:33.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 10 monclient: _finish_auth 0 2024-01-24T23:29:33.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 10 monclient: _check_auth_tickets 2024-01-24T23:29:33.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:33.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 1 -- 172.21.15.115:0/402743271 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f74e8001800 con 0x7f74ec098580 2024-01-24T23:29:33.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:29:33.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 10 monclient: dump: 2024-01-24T23:29:33.595 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.595 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.596 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.596 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.596 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.596 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.596 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.596 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.596 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.596 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.596 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.596 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.596 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.596 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.596 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.596 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 5 monclient: authenticate success, global_id 7278 2024-01-24T23:29:33.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:33.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 1 -- 172.21.15.115:0/402743271 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f74e8001a20 con 0x7f74ec098580 2024-01-24T23:29:33.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:33.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 1 -- 172.21.15.115:0/402743271 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f74e8004bf0 con 0x7f74ec098580 2024-01-24T23:29:33.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:29:33.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f1404640 10 monclient: dump: 2024-01-24T23:29:33.597 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.597 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.597 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.597 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.597 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.597 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.597 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.597 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.597 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.598 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.598 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.598 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.598 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.598 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.598 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.598 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:33.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 4 set_mon_vals no callback set 2024-01-24T23:29:33.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 10 monclient: shutdown 2024-01-24T23:29:33.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:33.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 1 -- 172.21.15.115:0/402743271 >> v1:172.21.15.115:6792/0 conn(0x7f74ec098580 legacy=0x7f74ec098960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 1 -- 172.21.15.115:0/402743271 shutdown_connections 2024-01-24T23:29:33.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 1 -- 172.21.15.115:0/402743271 >> 172.21.15.115:0/402743271 conn(0x7f74ec0a1fa0 msgr2=0x7f74ec0a2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:33.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 1 -- 172.21.15.115:0/402743271 shutdown_connections 2024-01-24T23:29:33.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 1 -- 172.21.15.115:0/402743271 wait complete. 2024-01-24T23:29:33.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 10 monclient: build_initial_monmap 2024-01-24T23:29:33.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.583+0000 7f74f466a640 10 monclient: monmap: 2024-01-24T23:29:33.599 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:33.599 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:33.599 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:33.599 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:33.599 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:33.599 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:33.599 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:33.599 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:33.599 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:33.599 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:33.599 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:33.599 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:33.600 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:33.600 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:33.600 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:33.600 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f466a640 1 Processor -- start 2024-01-24T23:29:33.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f466a640 1 -- start start 2024-01-24T23:29:33.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f466a640 10 monclient: init 2024-01-24T23:29:33.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f466a640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:33.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f466a640 10 monclient: _add_conns ranks=[0,6,1,5,7,3,8,2,4] 2024-01-24T23:29:33.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f466a640 10 monclient(hunting): picked mon.noname-a con 0x7f74ec098580 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:33.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f466a640 10 monclient(hunting): picked mon.noname-d con 0x7f74ec136f40 addr v1:172.21.15.115:6792/0 2024-01-24T23:29:33.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f466a640 10 monclient(hunting): picked mon.noname-f con 0x7f74ec137360 addr v1:172.21.15.132:6789/0 2024-01-24T23:29:33.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f466a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74ec13cc90 con 0x7f74ec098580 2024-01-24T23:29:33.600 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f466a640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74ec0722f0 con 0x7f74ec137360 2024-01-24T23:29:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f466a640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74ec13c9b0 con 0x7f74ec136f40 2024-01-24T23:29:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f466a640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f466a640 10 monclient(hunting): authenticate will time out at 1452.512812s 2024-01-24T23:29:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f2406640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f74ec098580 0x7f74ec139840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39128/0 (socket says 172.21.15.115:39128) 2024-01-24T23:29:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f2406640 1 -- 172.21.15.115:0/3247434757 learned_addr learned my addr 172.21.15.115:0/3247434757 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2310262326 0 0) 0x7f74ec13cc90 con 0x7f74ec098580 2024-01-24T23:29:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient(hunting): my global_id is 154143 2024-01-24T23:29:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74d0003530 con 0x7f74ec098580 2024-01-24T23:29:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460143671 0 0) 0x7f74ec13c9b0 con 0x7f74ec136f40 2024-01-24T23:29:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient(hunting): my global_id is 7287 2024-01-24T23:29:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74ec13cc90 con 0x7f74ec136f40 2024-01-24T23:29:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 112152679 0 0) 0x7f74d0003530 con 0x7f74ec098580 2024-01-24T23:29:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f74ec13c9b0 con 0x7f74ec098580 2024-01-24T23:29:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f74e80017c0 con 0x7f74ec098580 2024-01-24T23:29:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:29:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient(hunting): dump: 2024-01-24T23:29:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.603 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2632326908 0 0) 0x7f74ec13cc90 con 0x7f74ec136f40 2024-01-24T23:29:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f74d0003530 con 0x7f74ec136f40 2024-01-24T23:29:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f74dc001560 con 0x7f74ec136f40 2024-01-24T23:29:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:29:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient(hunting): dump: 2024-01-24T23:29:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.605 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 521680952 0 0) 0x7f74ec072600 con 0x7f74ec137360 2024-01-24T23:29:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient(hunting): my global_id is 124633 2024-01-24T23:29:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74ec13cc90 con 0x7f74ec137360 2024-01-24T23:29:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1236166043 0 0) 0x7f74ec13c9b0 con 0x7f74ec098580 2024-01-24T23:29:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 >> v1:172.21.15.115:6792/0 conn(0x7f74ec136f40 legacy=0x7f74ec139f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 >> v1:172.21.15.132:6789/0 conn(0x7f74ec137360 legacy=0x7f74ec1d3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:29:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 monclient: found mon.a 2024-01-24T23:29:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:29:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ec13f7e0 con 0x7f74ec098580 2024-01-24T23:29:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient: _finish_auth 0 2024-01-24T23:29:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:29:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f466a640 5 monclient: authenticate success, global_id 154143 2024-01-24T23:29:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f466a640 10 monclient: _renew_subs 2024-01-24T23:29:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f466a640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:29:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f466a640 1 -- 172.21.15.115:0/3247434757 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ec13af70 con 0x7f74ec098580 2024-01-24T23:29:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f74e8004c10 con 0x7f74ec098580 2024-01-24T23:29:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f466a640 10 monclient: _renew_subs 2024-01-24T23:29:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f74e80051f0 con 0x7f74ec098580 2024-01-24T23:29:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f466a640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:29:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f466a640 1 -- 172.21.15.115:0/3247434757 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ec13c750 con 0x7f74ec098580 2024-01-24T23:29:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f0c03640 4 set_mon_vals no callback set 2024-01-24T23:29:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:29:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient: dump: 2024-01-24T23:29:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.608 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f466a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f466a640 1 -- 172.21.15.115:0/3247434757 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f74ec13aa40 con 0x7f74ec13a660 2024-01-24T23:29:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f74e800fc70 con 0x7f74ec098580 2024-01-24T23:29:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7f74e8044680 con 0x7f74ec098580 2024-01-24T23:29:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3209121986 0 0) 0x7f74ec13aa40 con 0x7f74ec13a660 2024-01-24T23:29:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient(hunting): my global_id is 7296 2024-01-24T23:29:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74ec13c9b0 con 0x7f74ec13a660 2024-01-24T23:29:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74f2c07640 10 monclient: get_auth_request con 0x7f74d00378e0 auth_method 0 2024-01-24T23:29:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4159995671 0 0) 0x7f74ec13c9b0 con 0x7f74ec13a660 2024-01-24T23:29:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f74ec13aa40 con 0x7f74ec13a660 2024-01-24T23:29:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f74dc004d20 con 0x7f74ec13a660 2024-01-24T23:29:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:29:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient: dump: 2024-01-24T23:29:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.610 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3575868612 0 0) 0x7f74ec13aa40 con 0x7f74ec13a660 2024-01-24T23:29:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 --> v1:172.21.15.115:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74ec140b10 con 0x7f74ec13a660 2024-01-24T23:29:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f74dc005010 con 0x7f74ec13a660 2024-01-24T23:29:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 >> v1:172.21.15.115:6792/0 conn(0x7f74ec13a660 legacy=0x7f74ec09c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.587+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 >> v1:172.21.15.115:6792/0 conn(0x7f74ec13a660 legacy=0x7f74ec09c8a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74f466a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:29:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74f466a640 1 -- 172.21.15.115:0/3247434757 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f74ec06a0d0 con 0x7f74ec02c7f0 2024-01-24T23:29:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1020448363 0 0) 0x7f74ec06a0d0 con 0x7f74ec02c7f0 2024-01-24T23:29:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74e2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74e2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74e2ffd640 10 monclient(hunting): my global_id is 7305 2024-01-24T23:29:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74ec13aa40 con 0x7f74ec02c7f0 2024-01-24T23:29:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2592902279 0 0) 0x7f74ec13aa40 con 0x7f74ec02c7f0 2024-01-24T23:29:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f74ec06a0d0 con 0x7f74ec02c7f0 2024-01-24T23:29:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f74dc005730 con 0x7f74ec02c7f0 2024-01-24T23:29:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74e2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74e2ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:29:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74e2ffd640 10 monclient: dump: 2024-01-24T23:29:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.614 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.614 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.614 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.614 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.614 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.614 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.614 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4103038788 0 0) 0x7f74ec06a0d0 con 0x7f74ec02c7f0 2024-01-24T23:29:33.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 --> v1:172.21.15.115:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f74ec067a30 con 0x7f74ec02c7f0 2024-01-24T23:29:33.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 4090017679) 0x7f74dc007af0 con 0x7f74ec02c7f0 2024-01-24T23:29:33.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74e2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:29:33.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74e2ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:33.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 >> v1:172.21.15.115:6792/0 conn(0x7f74ec02c7f0 legacy=0x7f74ec09cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74e2ffd640 1 -- 172.21.15.115:0/3247434757 >> v1:172.21.15.115:6792/0 conn(0x7f74ec02c7f0 legacy=0x7f74ec09cfb0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:33.615 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:29:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:29:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 48, 2024-01-24T23:29:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:29:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:29:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:29:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:29:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:29:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:29:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:29:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:29:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:29:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:29:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 3, 2024-01-24T23:29:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:29:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:29:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:29:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:29:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:29:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:29:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:29:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:29:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:29:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:29:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:29:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:29:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:29:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:29:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:29:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:29:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:29:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:29:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:33.619 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:29:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:29:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:29:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.620 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:29:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:29:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.621 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:29:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:29:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:29:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.622 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:29:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:29:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:29:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:33.623 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:29:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:29:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:29:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.624 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:29:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:29:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:29:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.625 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:29:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:29:33.626 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:29:33.627 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.627 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.627 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.627 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:33.627 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:33.627 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.627 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.627 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.627 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:29:33.627 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.627 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.627 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:29:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:29:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.628 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:29:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:29:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:29:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:33.629 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:29:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:29:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:29:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.630 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:29:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:29:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:29:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:33.631 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:33.632 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.632 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.632 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:29:33.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.632 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:33.632 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:33.632 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:29:33.632 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.632 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.632 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:29:33.632 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:29:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74f466a640 1 -- 172.21.15.115:0/3247434757 >> v1:172.21.15.132:6812/1835401922 conn(0x7f74d00378e0 legacy=0x7f74d0039da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74f466a640 10 monclient: shutdown 2024-01-24T23:29:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74f466a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74f466a640 1 -- 172.21.15.115:0/3247434757 >> v1:172.21.15.115:6789/0 conn(0x7f74ec098580 legacy=0x7f74ec139840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74f2c07640 1 -- 172.21.15.115:0/3247434757 reap_dead start 2024-01-24T23:29:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74f2c07640 1 -- 172.21.15.115:0/3247434757 reap_dead start 2024-01-24T23:29:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74f466a640 1 -- 172.21.15.115:0/3247434757 shutdown_connections 2024-01-24T23:29:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74f466a640 1 -- 172.21.15.115:0/3247434757 >> 172.21.15.115:0/3247434757 conn(0x7f74ec0a1fa0 msgr2=0x7f74ec074860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74f466a640 1 -- 172.21.15.115:0/3247434757 shutdown_connections 2024-01-24T23:29:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.595+0000 7f74f466a640 1 -- 172.21.15.115:0/3247434757 wait complete. 2024-01-24T23:29:33.635 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:29:33.635 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-01-24T23:29:33.635 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-01-24T23:29:33.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 10 monclient: get_monmap_and_config 2024-01-24T23:29:33.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 10 monclient: build_initial_monmap 2024-01-24T23:29:33.753 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 10 monclient: monmap: 2024-01-24T23:29:33.753 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:33.753 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.753 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:33.755775+0000 2024-01-24T23:29:33.753 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:33.755775+0000 2024-01-24T23:29:33.753 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:33.753 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.754 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:33.754 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:33.754 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:33.754 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:33.754 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:33.754 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:33.754 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:33.754 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:33.754 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:33.754 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 1 Processor -- start 2024-01-24T23:29:33.757 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 1 -- start start 2024-01-24T23:29:33.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 10 monclient: init 2024-01-24T23:29:33.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:33.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 10 monclient: _add_conns ranks=[0,1,7,3,5,2,4,8,6] 2024-01-24T23:29:33.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 10 monclient(hunting): picked mon.noname-a con 0x7f7ec0133bf0 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:33.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 10 monclient(hunting): picked mon.noname-f con 0x7f7ec0133fd0 addr v1:172.21.15.132:6789/0 2024-01-24T23:29:33.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 10 monclient(hunting): picked mon.noname-i con 0x7f7ec01343b0 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:33.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec004ed70 con 0x7f7ec0133bf0 2024-01-24T23:29:33.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec00a4b60 con 0x7f7ec0133fd0 2024-01-24T23:29:33.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec009ae50 con 0x7f7ec01343b0 2024-01-24T23:29:33.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:33.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 10 monclient(hunting): authenticate will time out at 1452.678370s 2024-01-24T23:29:33.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec64cc640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f7ec0133bf0 0x7f7ec009bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39136/0 (socket says 172.21.15.115:39136) 2024-01-24T23:29:33.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec64cc640 1 -- 172.21.15.115:0/2248735007 learned_addr learned my addr 172.21.15.115:0/2248735007 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:33.758 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 1 -- 172.21.15.115:0/2248735007 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1579583217 0 0) 0x7f7ec004ed70 con 0x7f7ec0133bf0 2024-01-24T23:29:33.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 10 monclient(hunting): my global_id is 154152 2024-01-24T23:29:33.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 1 -- 172.21.15.115:0/2248735007 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7eac003530 con 0x7f7ec0133bf0 2024-01-24T23:29:33.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 1 -- 172.21.15.115:0/2248735007 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3541636751 0 0) 0x7f7eac003530 con 0x7f7ec0133bf0 2024-01-24T23:29:33.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 1 -- 172.21.15.115:0/2248735007 >> v1:172.21.15.132:6792/0 conn(0x7f7ec01343b0 legacy=0x7f7ec013f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 1 -- 172.21.15.115:0/2248735007 >> v1:172.21.15.132:6789/0 conn(0x7f7ec0133fd0 legacy=0x7f7ec013bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 10 monclient: _finish_hunting 0 2024-01-24T23:29:33.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 1 monclient: found mon.noname-a 2024-01-24T23:29:33.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:33.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:29:33.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 1 -- 172.21.15.115:0/2248735007 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec009b130 con 0x7f7ec0133bf0 2024-01-24T23:29:33.759 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 10 monclient: _finish_auth 0 2024-01-24T23:29:33.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 10 monclient: _check_auth_tickets 2024-01-24T23:29:33.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:33.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 1 -- 172.21.15.115:0/2248735007 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7eb8001b40 con 0x7f7ec0133bf0 2024-01-24T23:29:33.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:29:33.760 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 10 monclient: dump: 2024-01-24T23:29:33.760 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.760 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.760 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.760 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.760 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.760 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.760 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.760 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.761 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.761 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.761 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.761 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.761 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.761 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.761 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.761 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 5 monclient: authenticate success, global_id 154152 2024-01-24T23:29:33.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:33.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 1 -- 172.21.15.115:0/2248735007 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7eb8001df0 con 0x7f7ec0133bf0 2024-01-24T23:29:33.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:33.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 1 -- 172.21.15.115:0/2248735007 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7eb8004f70 con 0x7f7ec0133bf0 2024-01-24T23:29:33.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.761 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:29:33.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec54ca640 10 monclient: dump: 2024-01-24T23:29:33.762 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.762 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.762 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.762 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.762 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.762 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.762 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.762 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.762 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.762 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.762 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.762 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.762 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.763 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.763 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.763 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:33.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 4 set_mon_vals no callback set 2024-01-24T23:29:33.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 10 monclient: shutdown 2024-01-24T23:29:33.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:33.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 1 -- 172.21.15.115:0/2248735007 >> v1:172.21.15.115:6789/0 conn(0x7f7ec0133bf0 legacy=0x7f7ec009bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 1 -- 172.21.15.115:0/2248735007 shutdown_connections 2024-01-24T23:29:33.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 1 -- 172.21.15.115:0/2248735007 >> 172.21.15.115:0/2248735007 conn(0x7f7ec00662f0 msgr2=0x7f7ec0066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:33.763 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 1 -- 172.21.15.115:0/2248735007 shutdown_connections 2024-01-24T23:29:33.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.751+0000 7f7ec8730640 1 -- 172.21.15.115:0/2248735007 wait complete. 2024-01-24T23:29:33.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.755+0000 7f7ec8730640 10 monclient: build_initial_monmap 2024-01-24T23:29:33.764 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.755+0000 7f7ec8730640 10 monclient: monmap: 2024-01-24T23:29:33.764 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:33.764 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:33.764 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:33.764 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:33.764 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:33.764 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:33.764 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:33.764 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:33.764 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:33.764 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:33.765 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:33.765 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:33.765 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:33.765 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:33.765 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:33.765 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.755+0000 7f7ec8730640 1 Processor -- start 2024-01-24T23:29:33.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.755+0000 7f7ec8730640 1 -- start start 2024-01-24T23:29:33.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.755+0000 7f7ec8730640 10 monclient: init 2024-01-24T23:29:33.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.755+0000 7f7ec8730640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:33.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.755+0000 7f7ec8730640 10 monclient: _add_conns ranks=[0,7,3,1,6,5,8,2,4] 2024-01-24T23:29:33.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.755+0000 7f7ec8730640 10 monclient(hunting): picked mon.noname-a con 0x7f7ec0133bf0 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:33.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.755+0000 7f7ec8730640 10 monclient(hunting): picked mon.noname-i con 0x7f7ec0133fd0 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:33.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.755+0000 7f7ec8730640 10 monclient(hunting): picked mon.noname-g con 0x7f7ec01343b0 addr v1:172.21.15.132:6790/0 2024-01-24T23:29:33.765 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.755+0000 7f7ec8730640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec009c810 con 0x7f7ec0133bf0 2024-01-24T23:29:33.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.755+0000 7f7ec8730640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec00a5990 con 0x7f7ec01343b0 2024-01-24T23:29:33.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.755+0000 7f7ec8730640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec00a2450 con 0x7f7ec0133fd0 2024-01-24T23:29:33.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.755+0000 7f7ec8730640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:33.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.755+0000 7f7ec8730640 10 monclient(hunting): authenticate will time out at 1452.681467s 2024-01-24T23:29:33.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.755+0000 7f7ec64cc640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f7ec0133bf0 0x7f7ec0132fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39146/0 (socket says 172.21.15.115:39146) 2024-01-24T23:29:33.766 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.755+0000 7f7ec64cc640 1 -- 172.21.15.115:0/2542248601 learned_addr learned my addr 172.21.15.115:0/2542248601 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:33.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3791834268 0 0) 0x7f7ec009c810 con 0x7f7ec0133bf0 2024-01-24T23:29:33.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.771 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 10 monclient(hunting): my global_id is 154161 2024-01-24T23:29:33.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ea4003530 con 0x7f7ec0133bf0 2024-01-24T23:29:33.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1869095789 0 0) 0x7f7ea4003530 con 0x7f7ec0133bf0 2024-01-24T23:29:33.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ec009c810 con 0x7f7ec0133bf0 2024-01-24T23:29:33.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7eb8001be0 con 0x7f7ec0133bf0 2024-01-24T23:29:33.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:29:33.772 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 10 monclient(hunting): dump: 2024-01-24T23:29:33.772 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.772 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.772 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.772 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.772 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.773 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.773 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.773 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.773 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.773 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.773 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.773 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.773 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.773 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.773 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.773 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:33.773 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3109234608 0 0) 0x7f7ec00a2450 con 0x7f7ec0133fd0 2024-01-24T23:29:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 10 monclient(hunting): my global_id is 154141 2024-01-24T23:29:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ea4003530 con 0x7f7ec0133fd0 2024-01-24T23:29:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775161652 0 0) 0x7f7ec00a5990 con 0x7f7ec01343b0 2024-01-24T23:29:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 10 monclient(hunting): my global_id is 7113 2024-01-24T23:29:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ec00a2450 con 0x7f7ec01343b0 2024-01-24T23:29:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1925769646 0 0) 0x7f7ec009c810 con 0x7f7ec0133bf0 2024-01-24T23:29:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 >> v1:172.21.15.132:6792/0 conn(0x7f7ec0133fd0 legacy=0x7f7ec01d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.774 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 >> v1:172.21.15.132:6790/0 conn(0x7f7ec01343b0 legacy=0x7f7ec01d4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:29:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 1 monclient: found mon.a 2024-01-24T23:29:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:29:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec0120960 con 0x7f7ec0133bf0 2024-01-24T23:29:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 10 monclient: _finish_auth 0 2024-01-24T23:29:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:29:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7eb6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7ec8730640 5 monclient: authenticate success, global_id 154161 2024-01-24T23:29:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7ec8730640 10 monclient: _renew_subs 2024-01-24T23:29:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7ec8730640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:29:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7ec8730640 1 -- 172.21.15.115:0/2542248601 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec01200c0 con 0x7f7ec0133bf0 2024-01-24T23:29:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7ec8730640 10 monclient: _renew_subs 2024-01-24T23:29:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7ec8730640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:29:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.763+0000 7f7ec8730640 1 -- 172.21.15.115:0/2542248601 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec00a31d0 con 0x7f7ec0133bf0 2024-01-24T23:29:33.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.767+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7eb8001580 con 0x7f7ec0133bf0 2024-01-24T23:29:33.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.767+0000 7f7eb6ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:33.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.767+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7eb80054a0 con 0x7f7ec0133bf0 2024-01-24T23:29:33.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.767+0000 7f7eb6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.767+0000 7f7ec4cc9640 4 set_mon_vals no callback set 2024-01-24T23:29:33.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.767+0000 7f7eb6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:29:33.776 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.767+0000 7f7eb6ffd640 10 monclient: dump: 2024-01-24T23:29:33.776 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.776 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.776 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.776 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.776 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.776 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.776 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.777 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.777 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.777 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.777 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.777 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.777 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.777 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.777 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.777 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.767+0000 7f7ec8730640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:33.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.767+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f7eb800ff20 con 0x7f7ec0133bf0 2024-01-24T23:29:33.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.767+0000 7f7ec8730640 1 -- 172.21.15.115:0/2542248601 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7ea80065d0 con 0x7f7ea8002870 2024-01-24T23:29:33.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.767+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7f7eb80449b0 con 0x7f7ec0133bf0 2024-01-24T23:29:33.777 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.767+0000 7f7ec6ccd640 10 monclient: get_auth_request con 0x7f7ea40371a0 auth_method 0 2024-01-24T23:29:33.782 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.779+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135560657 0 0) 0x7f7ea80065d0 con 0x7f7ea8002870 2024-01-24T23:29:33.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.779+0000 7f7eb6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.779+0000 7f7eb6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.779+0000 7f7eb6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.779+0000 7f7eb6ffd640 10 monclient(hunting): my global_id is 86327 2024-01-24T23:29:33.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.779+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ec009c810 con 0x7f7ea8002870 2024-01-24T23:29:33.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.779+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 347985771 0 0) 0x7f7ec009c810 con 0x7f7ea8002870 2024-01-24T23:29:33.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.779+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ea80065d0 con 0x7f7ea8002870 2024-01-24T23:29:33.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.779+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7eb0001880 con 0x7f7ea8002870 2024-01-24T23:29:33.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.779+0000 7f7eb6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.779+0000 7f7eb6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:29:33.783 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.779+0000 7f7eb6ffd640 10 monclient: dump: 2024-01-24T23:29:33.783 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.783 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.783 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.784 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.784 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.784 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.784 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.784 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.784 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.784 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.784 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.784 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.784 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.784 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.784 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.784 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.779+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2796300433 0 0) 0x7f7ea80065d0 con 0x7f7ea8002870 2024-01-24T23:29:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.779+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 --> v1:172.21.15.115:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7ea800a120 con 0x7f7ea8002870 2024-01-24T23:29:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.779+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f7eb0004650 con 0x7f7ea8002870 2024-01-24T23:29:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.779+0000 7f7eb6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.779+0000 7f7eb6ffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.779+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 >> v1:172.21.15.115:6793/0 conn(0x7f7ea8002870 legacy=0x7f7ea8004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.779+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 >> v1:172.21.15.115:6793/0 conn(0x7f7ea8002870 legacy=0x7f7ea8004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:33.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7ec8730640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:29:33.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7ec8730640 1 -- 172.21.15.115:0/2542248601 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7ea8009df0 con 0x7f7ea80090b0 2024-01-24T23:29:33.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 305304824 0 0) 0x7f7ea8009df0 con 0x7f7ea80090b0 2024-01-24T23:29:33.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7eb6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7eb6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7eb6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7eb6ffd640 10 monclient(hunting): my global_id is 86336 2024-01-24T23:29:33.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ea80065d0 con 0x7f7ea80090b0 2024-01-24T23:29:33.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1289817608 0 0) 0x7f7ea80065d0 con 0x7f7ea80090b0 2024-01-24T23:29:33.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ea8009df0 con 0x7f7ea80090b0 2024-01-24T23:29:33.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7eb0003cc0 con 0x7f7ea80090b0 2024-01-24T23:29:33.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7eb6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7eb6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:29:33.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7eb6ffd640 10 monclient: dump: 2024-01-24T23:29:33.792 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.792 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.792 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.792 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.792 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.792 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.792 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.792 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.793 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.793 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.793 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.793 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.793 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.793 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.793 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.793 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1797076858 0 0) 0x7f7ea8009df0 con 0x7f7ea80090b0 2024-01-24T23:29:33.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 --> v1:172.21.15.115:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7ea800c310 con 0x7f7ea80090b0 2024-01-24T23:29:33.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 1537748277) 0x7f7eb0003f00 con 0x7f7ea80090b0 2024-01-24T23:29:33.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7eb6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:29:33.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7eb6ffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:33.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 >> v1:172.21.15.115:6793/0 conn(0x7f7ea80090b0 legacy=0x7f7ea8009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.793 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7eb6ffd640 1 -- 172.21.15.115:0/2542248601 >> v1:172.21.15.115:6793/0 conn(0x7f7ea80090b0 legacy=0x7f7ea8009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:33.794 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:29:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:29:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 48, 2024-01-24T23:29:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:29:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:29:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:29:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:29:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:29:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:29:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:29:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:29:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:29:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:29:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 4, 2024-01-24T23:29:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:29:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:29:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:29:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:29:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:29:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:29:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:29:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:29:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:29:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:29:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:29:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:29:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:29:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:29:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:29:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:29:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:29:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:29:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:29:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:29:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:29:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:29:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:29:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:29:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:29:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:29:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:29:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:29:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:29:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:29:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:29:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:29:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:29:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:29:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:29:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:29:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:29:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:29:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:29:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:29:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:29:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:29:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:29:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:29:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:29:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:29:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:29:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:29:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:29:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:29:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:29:33.811 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:29:33.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7ec8730640 1 -- 172.21.15.115:0/2542248601 >> v1:172.21.15.132:6812/1835401922 conn(0x7f7ea40371a0 legacy=0x7f7ea4039660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7ec8730640 10 monclient: shutdown 2024-01-24T23:29:33.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7ec8730640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:33.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7ec8730640 1 -- 172.21.15.115:0/2542248601 >> v1:172.21.15.115:6789/0 conn(0x7f7ec0133bf0 legacy=0x7f7ec0132fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7ec6ccd640 1 -- 172.21.15.115:0/2542248601 reap_dead start 2024-01-24T23:29:33.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.787+0000 7f7ec6ccd640 1 -- 172.21.15.115:0/2542248601 reap_dead start 2024-01-24T23:29:33.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.791+0000 7f7ec8730640 1 -- 172.21.15.115:0/2542248601 shutdown_connections 2024-01-24T23:29:33.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.791+0000 7f7ec8730640 1 -- 172.21.15.115:0/2542248601 >> 172.21.15.115:0/2542248601 conn(0x7f7ec00662f0 msgr2=0x7f7ec01bea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:33.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.791+0000 7f7ec8730640 1 -- 172.21.15.115:0/2542248601 shutdown_connections 2024-01-24T23:29:33.813 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.791+0000 7f7ec8730640 1 -- 172.21.15.115:0/2542248601 wait complete. 2024-01-24T23:29:33.814 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:29:33.814 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-01-24T23:29:33.814 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-01-24T23:29:33.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.923+0000 7fdfe780a640 10 monclient: get_monmap_and_config 2024-01-24T23:29:33.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.923+0000 7fdfe780a640 10 monclient: build_initial_monmap 2024-01-24T23:29:33.927 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.923+0000 7fdfe780a640 10 monclient: monmap: 2024-01-24T23:29:33.927 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:33.927 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.927 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:33.929663+0000 2024-01-24T23:29:33.927 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:33.929663+0000 2024-01-24T23:29:33.927 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:33.927 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.927 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:33.928 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:33.928 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:33.928 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:33.928 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:33.928 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:33.928 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:33.928 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:33.928 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:33.928 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.923+0000 7fdfe780a640 1 Processor -- start 2024-01-24T23:29:33.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.923+0000 7fdfe780a640 1 -- start start 2024-01-24T23:29:33.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.923+0000 7fdfe780a640 10 monclient: init 2024-01-24T23:29:33.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:33.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 10 monclient: _add_conns ranks=[8,0,4,7,6,5,3,2,1] 2024-01-24T23:29:33.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 10 monclient(hunting): picked mon.noname-e con 0x7fdfe00a0ae0 addr v1:172.21.15.115:6793/0 2024-01-24T23:29:33.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 10 monclient(hunting): picked mon.noname-a con 0x7fdfe00a2de0 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:33.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 10 monclient(hunting): picked mon.noname-c con 0x7fdfe009f2d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:29:33.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe004ed70 con 0x7fdfe00a2de0 2024-01-24T23:29:33.936 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe00a3350 con 0x7fdfe009f2d0 2024-01-24T23:29:33.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe006c1f0 con 0x7fdfe00a0ae0 2024-01-24T23:29:33.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:33.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 10 monclient(hunting): authenticate will time out at 1452.852620s 2024-01-24T23:29:33.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe55a6640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fdfe00a0ae0 0x7fdfe00a0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:55922/0 (socket says 172.21.15.115:55922) 2024-01-24T23:29:33.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe55a6640 1 -- 172.21.15.115:0/1802342268 learned_addr learned my addr 172.21.15.115:0/1802342268 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:33.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 1 -- 172.21.15.115:0/1802342268 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3900217936 0 0) 0x7fdfe004ed70 con 0x7fdfe00a2de0 2024-01-24T23:29:33.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 10 monclient(hunting): my global_id is 154170 2024-01-24T23:29:33.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 1 -- 172.21.15.115:0/1802342268 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfcc003530 con 0x7fdfe00a2de0 2024-01-24T23:29:33.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 1 -- 172.21.15.115:0/1802342268 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536190335 0 0) 0x7fdfe00a3350 con 0x7fdfe009f2d0 2024-01-24T23:29:33.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.937 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 10 monclient(hunting): my global_id is 95251 2024-01-24T23:29:33.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 1 -- 172.21.15.115:0/1802342268 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfe004ed70 con 0x7fdfe009f2d0 2024-01-24T23:29:33.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 1 -- 172.21.15.115:0/1802342268 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2751628190 0 0) 0x7fdfe006c1f0 con 0x7fdfe00a0ae0 2024-01-24T23:29:33.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 10 monclient(hunting): my global_id is 86345 2024-01-24T23:29:33.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 1 -- 172.21.15.115:0/1802342268 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfe00a3350 con 0x7fdfe00a0ae0 2024-01-24T23:29:33.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 1 -- 172.21.15.115:0/1802342268 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 862519428 0 0) 0x7fdfcc003530 con 0x7fdfe00a2de0 2024-01-24T23:29:33.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 1 -- 172.21.15.115:0/1802342268 >> v1:172.21.15.115:6793/0 conn(0x7fdfe00a0ae0 legacy=0x7fdfe00a0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 1 -- 172.21.15.115:0/1802342268 >> v1:172.21.15.115:6791/0 conn(0x7fdfe009f2d0 legacy=0x7fdfe013f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 10 monclient: _finish_hunting 0 2024-01-24T23:29:33.938 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 1 monclient: found mon.noname-a 2024-01-24T23:29:33.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:33.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:29:33.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 1 -- 172.21.15.115:0/1802342268 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe006c4d0 con 0x7fdfe00a2de0 2024-01-24T23:29:33.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 10 monclient: _finish_auth 0 2024-01-24T23:29:33.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 10 monclient: _check_auth_tickets 2024-01-24T23:29:33.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:33.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 1 -- 172.21.15.115:0/1802342268 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdfdc0017c0 con 0x7fdfe00a2de0 2024-01-24T23:29:33.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:29:33.939 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 10 monclient: dump: 2024-01-24T23:29:33.939 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.939 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.939 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.939 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.940 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.940 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.940 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.940 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.940 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.940 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.940 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.940 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.940 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.940 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.940 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.940 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 5 monclient: authenticate success, global_id 154170 2024-01-24T23:29:33.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:33.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 1 -- 172.21.15.115:0/1802342268 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdfdc001aa0 con 0x7fdfe00a2de0 2024-01-24T23:29:33.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:33.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:33.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 1 -- 172.21.15.115:0/1802342268 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdfdc003960 con 0x7fdfe00a2de0 2024-01-24T23:29:33.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 4 set_mon_vals no callback set 2024-01-24T23:29:33.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:29:33.941 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd7fff640 10 monclient: dump: 2024-01-24T23:29:33.941 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.941 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.941 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.941 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.942 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.942 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.942 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.942 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.942 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.942 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.942 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.942 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.942 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.942 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.942 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.942 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 10 monclient: shutdown 2024-01-24T23:29:33.942 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:33.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 1 -- 172.21.15.115:0/1802342268 >> v1:172.21.15.115:6789/0 conn(0x7fdfe00a2de0 legacy=0x7fdfe013bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 1 -- 172.21.15.115:0/1802342268 shutdown_connections 2024-01-24T23:29:33.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 1 -- 172.21.15.115:0/1802342268 >> 172.21.15.115:0/1802342268 conn(0x7fdfe00666f0 msgr2=0x7fdfe0066b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:33.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 1 -- 172.21.15.115:0/1802342268 shutdown_connections 2024-01-24T23:29:33.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 1 -- 172.21.15.115:0/1802342268 wait complete. 2024-01-24T23:29:33.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 10 monclient: build_initial_monmap 2024-01-24T23:29:33.943 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 10 monclient: monmap: 2024-01-24T23:29:33.943 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:33.943 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:33.943 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:33.943 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:33.943 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:33.943 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:33.943 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:33.944 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:33.944 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:33.944 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:33.944 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:33.944 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:33.944 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:33.944 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:33.944 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:33.944 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 1 Processor -- start 2024-01-24T23:29:33.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 1 -- start start 2024-01-24T23:29:33.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 10 monclient: init 2024-01-24T23:29:33.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:33.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 10 monclient: _add_conns ranks=[7,0,1,3,5,8,4,2,6] 2024-01-24T23:29:33.944 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 10 monclient(hunting): picked mon.noname-i con 0x7fdfe009f2d0 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:33.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 10 monclient(hunting): picked mon.noname-a con 0x7fdfe00a0ae0 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:33.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 10 monclient(hunting): picked mon.noname-f con 0x7fdfe00a2de0 addr v1:172.21.15.132:6789/0 2024-01-24T23:29:33.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe013e7a0 con 0x7fdfe00a0ae0 2024-01-24T23:29:33.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe01cc220 con 0x7fdfe00a2de0 2024-01-24T23:29:33.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe006b5c0 con 0x7fdfe009f2d0 2024-01-24T23:29:33.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:33.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe780a640 10 monclient(hunting): authenticate will time out at 1452.855542s 2024-01-24T23:29:33.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe4da5640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fdfe00a0ae0 0x7fdfe01d0660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39160/0 (socket says 172.21.15.115:39160) 2024-01-24T23:29:33.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfe4da5640 1 -- 172.21.15.115:0/3926952318 learned_addr learned my addr 172.21.15.115:0/3926952318 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:33.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330291825 0 0) 0x7fdfe013e7a0 con 0x7fdfe00a0ae0 2024-01-24T23:29:33.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd5ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd5ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd5ffb640 10 monclient(hunting): my global_id is 154179 2024-01-24T23:29:33.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfc4003530 con 0x7fdfe00a0ae0 2024-01-24T23:29:33.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.927+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 673251243 0 0) 0x7fdfc4003530 con 0x7fdfe00a0ae0 2024-01-24T23:29:33.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdfe013e7a0 con 0x7fdfe00a0ae0 2024-01-24T23:29:33.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdfdc0039e0 con 0x7fdfe00a0ae0 2024-01-24T23:29:33.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:29:33.946 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 10 monclient(hunting): dump: 2024-01-24T23:29:33.946 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.946 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.946 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.946 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.946 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.946 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.946 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.946 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.946 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.947 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.947 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.947 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.947 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.947 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.947 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.947 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:33.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 665572281 0 0) 0x7fdfe013e7a0 con 0x7fdfe00a0ae0 2024-01-24T23:29:33.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 >> v1:172.21.15.132:6792/0 conn(0x7fdfe009f2d0 legacy=0x7fdfe01cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 >> v1:172.21.15.132:6789/0 conn(0x7fdfe00a2de0 legacy=0x7fdfe01d3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:29:33.947 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 1 monclient: found mon.a 2024-01-24T23:29:33.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:33.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:29:33.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe0071f70 con 0x7fdfe00a0ae0 2024-01-24T23:29:33.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 10 monclient: _finish_auth 0 2024-01-24T23:29:33.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:29:33.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:33.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfe780a640 5 monclient: authenticate success, global_id 154179 2024-01-24T23:29:33.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfe780a640 10 monclient: _renew_subs 2024-01-24T23:29:33.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfe780a640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:29:33.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfe780a640 1 -- 172.21.15.115:0/3926952318 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe01401f0 con 0x7fdfe00a0ae0 2024-01-24T23:29:33.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfe780a640 10 monclient: _renew_subs 2024-01-24T23:29:33.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfe780a640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:29:33.948 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfe780a640 1 -- 172.21.15.115:0/3926952318 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe013f300 con 0x7fdfe00a0ae0 2024-01-24T23:29:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdfdc001cf0 con 0x7fdfe00a0ae0 2024-01-24T23:29:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdfdc004180 con 0x7fdfe00a0ae0 2024-01-24T23:29:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd77fe640 4 set_mon_vals no callback set 2024-01-24T23:29:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:29:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 10 monclient: dump: 2024-01-24T23:29:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.950 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfe780a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfe780a640 1 -- 172.21.15.115:0/3926952318 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdfc00065d0 con 0x7fdfc0002870 2024-01-24T23:29:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fdfdc004360 con 0x7fdfe00a0ae0 2024-01-24T23:29:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7fdfdc044700 con 0x7fdfe00a0ae0 2024-01-24T23:29:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 651109413 0 0) 0x7fdfc00065d0 con 0x7fdfc0002870 2024-01-24T23:29:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 10 monclient(hunting): my global_id is 124660 2024-01-24T23:29:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfe013e7a0 con 0x7fdfc0002870 2024-01-24T23:29:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfe5da7640 10 monclient: get_auth_request con 0x7fdfc4036f80 auth_method 0 2024-01-24T23:29:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1859126710 0 0) 0x7fdfe013e7a0 con 0x7fdfc0002870 2024-01-24T23:29:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdfc00065d0 con 0x7fdfc0002870 2024-01-24T23:29:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdfd00018f0 con 0x7fdfc0002870 2024-01-24T23:29:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:29:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 10 monclient: dump: 2024-01-24T23:29:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.953 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.953 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.953 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.953 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.953 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1137392229 0 0) 0x7fdfc00065d0 con 0x7fdfc0002870 2024-01-24T23:29:33.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 --> v1:172.21.15.132:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdfc000a120 con 0x7fdfc0002870 2024-01-24T23:29:33.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 <== mon.1 v1:172.21.15.132:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fdfd00058e0 con 0x7fdfc0002870 2024-01-24T23:29:33.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:33.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:33.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 >> v1:172.21.15.132:6789/0 conn(0x7fdfc0002870 legacy=0x7fdfc0004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.931+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 >> v1:172.21.15.132:6789/0 conn(0x7fdfc0002870 legacy=0x7fdfc0004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfe780a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:29:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfe780a640 1 -- 172.21.15.115:0/3926952318 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdfc0009e10 con 0x7fdfc00090d0 2024-01-24T23:29:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2981660890 0 0) 0x7fdfc0009e10 con 0x7fdfc00090d0 2024-01-24T23:29:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfd5ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:29:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfd5ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfd5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfd5ffb640 10 monclient(hunting): my global_id is 124669 2024-01-24T23:29:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfc00065d0 con 0x7fdfc00090d0 2024-01-24T23:29:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4091862981 0 0) 0x7fdfc00065d0 con 0x7fdfc00090d0 2024-01-24T23:29:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdfc0009e10 con 0x7fdfc00090d0 2024-01-24T23:29:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fdfd0004de0 con 0x7fdfc00090d0 2024-01-24T23:29:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfd5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfd5ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:29:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfd5ffb640 10 monclient: dump: 2024-01-24T23:29:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:33.957 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2738194708 0 0) 0x7fdfc0009e10 con 0x7fdfc00090d0 2024-01-24T23:29:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 --> v1:172.21.15.132:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdfc000c310 con 0x7fdfc00090d0 2024-01-24T23:29:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 <== mon.1 v1:172.21.15.132:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 4128134512) 0x7fdfd0005a50 con 0x7fdfc00090d0 2024-01-24T23:29:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfd5ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:29:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfd5ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 >> v1:172.21.15.132:6789/0 conn(0x7fdfc00090d0 legacy=0x7fdfc0009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfd5ffb640 1 -- 172.21.15.115:0/3926952318 >> v1:172.21.15.132:6789/0 conn(0x7fdfc00090d0 legacy=0x7fdfc0009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:33.958 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:29:33.958 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:33.958 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:33.958 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:29:33.958 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 48, 2024-01-24T23:29:33.958 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:29:33.958 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:29:33.958 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:29:33.958 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:29:33.958 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:29:33.958 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:29:33.958 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:29:33.959 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:29:33.959 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:29:33.959 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:29:33.959 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.959 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 4, 2024-01-24T23:29:33.959 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:33.959 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:29:33.959 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:29:33.959 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:33.959 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:33.959 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:33.959 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:33.959 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:33.960 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:33.960 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:33.960 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:33.960 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:33.960 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:33.960 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:33.960 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.960 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:29:33.960 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:29:33.960 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:33.960 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:33.960 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:33.960 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:33.961 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:33.961 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:33.961 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:33.961 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:33.961 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:33.961 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:33.961 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:33.961 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.961 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:29:33.961 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:29:33.961 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:29:33.961 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:29:33.961 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:29:33.961 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:29:33.962 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:33.962 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:33.962 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:29:33.962 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:29:33.962 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:29:33.962 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:29:33.962 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:29:33.962 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:29:33.962 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:29:33.962 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:33.962 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:29:33.962 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:33.962 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:33.962 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:33.962 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:33.962 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:33.962 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:33.963 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:33.963 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:33.963 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:33.963 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:33.963 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:33.963 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.963 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:29:33.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.963 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:29:33.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.963 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:33.963 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:33.963 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.964 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.964 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.964 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:29:33.964 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.964 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.964 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.964 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:29:33.964 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:29:33.964 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.964 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.964 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.965 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:33.965 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:33.965 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.965 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.965 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.965 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:29:33.965 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.965 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.965 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.965 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:29:33.965 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:29:33.965 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.966 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.966 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.966 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:33.966 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:33.966 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.966 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.966 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.966 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:29:33.966 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.966 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:29:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:29:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.968 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:29:33.968 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.968 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.968 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.968 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:29:33.968 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:29:33.968 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.968 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:29:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:29:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:29:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:29:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:29:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:29:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:29:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:29:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:29:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:29:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.975 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.975 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:29:33.975 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:29:33.975 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.975 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.975 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:29:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:29:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:29:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:29:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:29:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:29:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:29:33.978 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:29:33.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfe780a640 1 -- 172.21.15.115:0/3926952318 >> v1:172.21.15.132:6812/1835401922 conn(0x7fdfc4036f80 legacy=0x7fdfc4039440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfe780a640 10 monclient: shutdown 2024-01-24T23:29:33.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfe780a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:33.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfe780a640 1 -- 172.21.15.115:0/3926952318 >> v1:172.21.15.115:6789/0 conn(0x7fdfe00a0ae0 legacy=0x7fdfe01d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:33.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfe5da7640 1 -- 172.21.15.115:0/3926952318 reap_dead start 2024-01-24T23:29:33.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfe5da7640 1 -- 172.21.15.115:0/3926952318 reap_dead start 2024-01-24T23:29:33.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfe780a640 1 -- 172.21.15.115:0/3926952318 shutdown_connections 2024-01-24T23:29:33.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.939+0000 7fdfe780a640 1 -- 172.21.15.115:0/3926952318 >> 172.21.15.115:0/3926952318 conn(0x7fdfe00666f0 msgr2=0x7fdfe009f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:33.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.943+0000 7fdfe780a640 1 -- 172.21.15.115:0/3926952318 shutdown_connections 2024-01-24T23:29:33.981 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:33.943+0000 7fdfe780a640 1 -- 172.21.15.115:0/3926952318 wait complete. 2024-01-24T23:29:33.981 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:29:33.981 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-01-24T23:29:33.981 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-01-24T23:29:34.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.079+0000 7f313bbe6640 10 monclient: get_monmap_and_config 2024-01-24T23:29:34.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.079+0000 7f313bbe6640 10 monclient: build_initial_monmap 2024-01-24T23:29:34.084 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.079+0000 7f313bbe6640 10 monclient: monmap: 2024-01-24T23:29:34.084 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:34.084 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.084 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:34.086681+0000 2024-01-24T23:29:34.084 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:34.086681+0000 2024-01-24T23:29:34.084 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:34.084 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.084 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:34.085 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:34.085 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:34.085 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:34.085 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:34.085 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:34.085 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:34.085 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:34.085 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:34.085 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 1 Processor -- start 2024-01-24T23:29:34.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 1 -- start start 2024-01-24T23:29:34.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 10 monclient: init 2024-01-24T23:29:34.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:34.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 10 monclient: _add_conns ranks=[8,1,0,3,6,5,4,2,7] 2024-01-24T23:29:34.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 10 monclient(hunting): picked mon.noname-e con 0x7f31340720b0 addr v1:172.21.15.115:6793/0 2024-01-24T23:29:34.093 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 10 monclient(hunting): picked mon.noname-f con 0x7f313409b9f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:29:34.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 10 monclient(hunting): picked mon.noname-a con 0x7f3134139f10 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:34.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f313404ed70 con 0x7f3134139f10 2024-01-24T23:29:34.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31340a0eb0 con 0x7f313409b9f0 2024-01-24T23:29:34.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f313409d020 con 0x7f31340720b0 2024-01-24T23:29:34.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:34.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 10 monclient(hunting): authenticate will time out at 1453.009297s 2024-01-24T23:29:34.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3139982640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f31340720b0 0x7f31340a2100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:55934/0 (socket says 172.21.15.115:55934) 2024-01-24T23:29:34.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3139982640 1 -- 172.21.15.115:0/814580999 learned_addr learned my addr 172.21.15.115:0/814580999 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:34.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 1 -- 172.21.15.115:0/814580999 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14596248 0 0) 0x7f313409d020 con 0x7f31340720b0 2024-01-24T23:29:34.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.094 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 10 monclient(hunting): my global_id is 86354 2024-01-24T23:29:34.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 1 -- 172.21.15.115:0/814580999 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3120003530 con 0x7f31340720b0 2024-01-24T23:29:34.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 1 -- 172.21.15.115:0/814580999 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3055218431 0 0) 0x7f313404ed70 con 0x7f3134139f10 2024-01-24T23:29:34.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 10 monclient(hunting): my global_id is 154188 2024-01-24T23:29:34.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 1 -- 172.21.15.115:0/814580999 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f313409d020 con 0x7f3134139f10 2024-01-24T23:29:34.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 1 -- 172.21.15.115:0/814580999 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 708716283 0 0) 0x7f313409d020 con 0x7f3134139f10 2024-01-24T23:29:34.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 1 -- 172.21.15.115:0/814580999 >> v1:172.21.15.115:6793/0 conn(0x7f31340720b0 legacy=0x7f31340a2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 1 -- 172.21.15.115:0/814580999 >> v1:172.21.15.132:6789/0 conn(0x7f313409b9f0 legacy=0x7f31341387f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 10 monclient: _finish_hunting 0 2024-01-24T23:29:34.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 1 monclient: found mon.noname-a 2024-01-24T23:29:34.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:34.095 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:29:34.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 1 -- 172.21.15.115:0/814580999 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313409d300 con 0x7f3134139f10 2024-01-24T23:29:34.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 10 monclient: _finish_auth 0 2024-01-24T23:29:34.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 10 monclient: _check_auth_tickets 2024-01-24T23:29:34.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:34.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 1 -- 172.21.15.115:0/814580999 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f312c001b40 con 0x7f3134139f10 2024-01-24T23:29:34.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:29:34.096 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 10 monclient: dump: 2024-01-24T23:29:34.096 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.096 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.096 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.096 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.096 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.096 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.097 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.097 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.097 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.097 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.097 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.097 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.097 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.097 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.097 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.097 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 5 monclient: authenticate success, global_id 154188 2024-01-24T23:29:34.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:34.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 1 -- 172.21.15.115:0/814580999 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f312c001de0 con 0x7f3134139f10 2024-01-24T23:29:34.097 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:34.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 1 -- 172.21.15.115:0/814580999 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f312c003ce0 con 0x7f3134139f10 2024-01-24T23:29:34.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:29:34.098 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f3138980640 10 monclient: dump: 2024-01-24T23:29:34.098 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.098 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.098 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.098 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.098 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.098 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.098 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.098 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.098 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.098 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.098 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.099 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.099 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.099 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.099 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.099 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:34.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 4 set_mon_vals no callback set 2024-01-24T23:29:34.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 10 monclient: shutdown 2024-01-24T23:29:34.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:34.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 1 -- 172.21.15.115:0/814580999 >> v1:172.21.15.115:6789/0 conn(0x7f3134139f10 legacy=0x7f313413c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 1 -- 172.21.15.115:0/814580999 shutdown_connections 2024-01-24T23:29:34.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 1 -- 172.21.15.115:0/814580999 >> 172.21.15.115:0/814580999 conn(0x7f31340a5730 msgr2=0x7f31340a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:34.099 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 1 -- 172.21.15.115:0/814580999 shutdown_connections 2024-01-24T23:29:34.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 1 -- 172.21.15.115:0/814580999 wait complete. 2024-01-24T23:29:34.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 10 monclient: build_initial_monmap 2024-01-24T23:29:34.100 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 10 monclient: monmap: 2024-01-24T23:29:34.100 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:34.100 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:34.100 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:34.100 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:34.100 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:34.100 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:34.100 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:34.100 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:34.100 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:34.101 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:34.101 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:34.101 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:34.101 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:34.101 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:34.101 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:34.101 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 1 Processor -- start 2024-01-24T23:29:34.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 1 -- start start 2024-01-24T23:29:34.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 10 monclient: init 2024-01-24T23:29:34.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:34.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.083+0000 7f313bbe6640 10 monclient: _add_conns ranks=[6,5,4,3,1,0,2,8,7] 2024-01-24T23:29:34.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f313bbe6640 10 monclient(hunting): picked mon.noname-d con 0x7f31340720b0 addr v1:172.21.15.115:6792/0 2024-01-24T23:29:34.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f313bbe6640 10 monclient(hunting): picked mon.noname-h con 0x7f313409b9f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:29:34.101 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f313bbe6640 10 monclient(hunting): picked mon.noname-c con 0x7f3134139f10 addr v1:172.21.15.115:6791/0 2024-01-24T23:29:34.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f313bbe6640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3134135260 con 0x7f3134139f10 2024-01-24T23:29:34.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f313bbe6640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f313409c480 con 0x7f313409b9f0 2024-01-24T23:29:34.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f313bbe6640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3134136130 con 0x7f31340720b0 2024-01-24T23:29:34.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f313bbe6640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:34.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f313bbe6640 10 monclient(hunting): authenticate will time out at 1453.012058s 2024-01-24T23:29:34.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f3139982640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f31340720b0 0x7f31341c7310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:53716/0 (socket says 172.21.15.115:53716) 2024-01-24T23:29:34.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f3139982640 1 -- 172.21.15.115:0/817069963 learned_addr learned my addr 172.21.15.115:0/817069963 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:34.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 850072810 0 0) 0x7f3134136130 con 0x7f31340720b0 2024-01-24T23:29:34.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient(hunting): my global_id is 7314 2024-01-24T23:29:34.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3118003530 con 0x7f31340720b0 2024-01-24T23:29:34.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3192325827 0 0) 0x7f3134135260 con 0x7f3134139f10 2024-01-24T23:29:34.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.102 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient(hunting): my global_id is 95260 2024-01-24T23:29:34.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3134136130 con 0x7f3134139f10 2024-01-24T23:29:34.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2997025198 0 0) 0x7f3118003530 con 0x7f31340720b0 2024-01-24T23:29:34.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3134135260 con 0x7f31340720b0 2024-01-24T23:29:34.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f31240014a0 con 0x7f31340720b0 2024-01-24T23:29:34.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:29:34.103 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient(hunting): dump: 2024-01-24T23:29:34.103 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.103 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.103 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.103 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.103 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.104 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.104 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.104 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.104 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.104 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.104 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.104 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.104 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.104 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.104 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.104 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:34.104 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3945887611 0 0) 0x7f3134136130 con 0x7f3134139f10 2024-01-24T23:29:34.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3118003530 con 0x7f3134139f10 2024-01-24T23:29:34.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f312c003cc0 con 0x7f3134139f10 2024-01-24T23:29:34.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:29:34.105 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient(hunting): dump: 2024-01-24T23:29:34.105 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.105 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.105 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.105 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.105 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.105 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.105 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.105 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.105 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.105 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.106 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.106 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.106 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.106 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.106 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.106 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2181084922 0 0) 0x7f3134135260 con 0x7f31340720b0 2024-01-24T23:29:34.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 >> v1:172.21.15.132:6791/0 conn(0x7f313409b9f0 legacy=0x7f31341cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 >> v1:172.21.15.115:6791/0 conn(0x7f3134139f10 legacy=0x7f31341cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:29:34.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 monclient: found mon.d 2024-01-24T23:29:34.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:34.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:29:34.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3134136410 con 0x7f31340720b0 2024-01-24T23:29:34.106 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient: _finish_auth 0 2024-01-24T23:29:34.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:29:34.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:34.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f313bbe6640 5 monclient: authenticate success, global_id 7314 2024-01-24T23:29:34.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f313bbe6640 10 monclient: _renew_subs 2024-01-24T23:29:34.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f313bbe6640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:29:34.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f313bbe6640 1 -- 172.21.15.115:0/817069963 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31341366f0 con 0x7f31340720b0 2024-01-24T23:29:34.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f313bbe6640 10 monclient: _renew_subs 2024-01-24T23:29:34.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f313bbe6640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:29:34.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f313bbe6640 1 -- 172.21.15.115:0/817069963 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3134071920 con 0x7f31340720b0 2024-01-24T23:29:34.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f31240017f0 con 0x7f31340720b0 2024-01-24T23:29:34.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:34.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3124004de0 con 0x7f31340720b0 2024-01-24T23:29:34.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.107 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312bfff640 4 set_mon_vals no callback set 2024-01-24T23:29:34.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:29:34.108 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient: dump: 2024-01-24T23:29:34.108 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.108 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.108 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.108 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.108 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.108 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.108 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.108 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.108 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.108 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.108 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.109 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.109 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.109 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.109 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.109 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f313bbe6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:34.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f313bbe6640 1 -- 172.21.15.115:0/817069963 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f311c0065d0 con 0x7f311c002870 2024-01-24T23:29:34.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f312400f860 con 0x7f31340720b0 2024-01-24T23:29:34.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7f31240444b0 con 0x7f31340720b0 2024-01-24T23:29:34.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f313a183640 10 monclient: get_auth_request con 0x7f3118037740 auth_method 0 2024-01-24T23:29:34.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3827230398 0 0) 0x7f311c0065d0 con 0x7f311c002870 2024-01-24T23:29:34.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.109 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient(hunting): my global_id is 7122 2024-01-24T23:29:34.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3134135260 con 0x7f311c002870 2024-01-24T23:29:34.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3152158439 0 0) 0x7f3134135260 con 0x7f311c002870 2024-01-24T23:29:34.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f311c0065d0 con 0x7f311c002870 2024-01-24T23:29:34.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3130001920 con 0x7f311c002870 2024-01-24T23:29:34.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:29:34.110 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient: dump: 2024-01-24T23:29:34.110 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.110 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.110 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.110 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.110 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.110 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.110 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.110 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.111 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.111 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.111 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.111 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.111 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.111 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.111 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.111 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1725400955 0 0) 0x7f311c0065d0 con 0x7f311c002870 2024-01-24T23:29:34.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 --> v1:172.21.15.132:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f311c00a120 con 0x7f311c002870 2024-01-24T23:29:34.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f3130003600 con 0x7f311c002870 2024-01-24T23:29:34.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:34.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:34.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 >> v1:172.21.15.132:6790/0 conn(0x7f311c002870 legacy=0x7f311c004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.111 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.087+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 >> v1:172.21.15.132:6790/0 conn(0x7f311c002870 legacy=0x7f311c004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:34.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.095+0000 7f313bbe6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:29:34.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.095+0000 7f313bbe6640 1 -- 172.21.15.115:0/817069963 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f311c009dd0 con 0x7f311c009090 2024-01-24T23:29:34.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.095+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 590653910 0 0) 0x7f311c009dd0 con 0x7f311c009090 2024-01-24T23:29:34.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.095+0000 7f312a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.095+0000 7f312a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.095+0000 7f312a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.095+0000 7f312a7fc640 10 monclient(hunting): my global_id is 7131 2024-01-24T23:29:34.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.095+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f311c0065d0 con 0x7f311c009090 2024-01-24T23:29:34.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.095+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 401848854 0 0) 0x7f311c0065d0 con 0x7f311c009090 2024-01-24T23:29:34.113 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.095+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f311c009dd0 con 0x7f311c009090 2024-01-24T23:29:34.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.095+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3130003da0 con 0x7f311c009090 2024-01-24T23:29:34.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.095+0000 7f312a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.095+0000 7f312a7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:29:34.114 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.095+0000 7f312a7fc640 10 monclient: dump: 2024-01-24T23:29:34.114 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.114 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.114 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.114 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.114 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.114 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.114 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.114 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.114 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.114 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.114 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.115 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.115 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.115 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.115 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.115 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.095+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2072212053 0 0) 0x7f311c009dd0 con 0x7f311c009090 2024-01-24T23:29:34.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.095+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 --> v1:172.21.15.132:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f311c00c2d0 con 0x7f311c009090 2024-01-24T23:29:34.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.099+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 2460431241) 0x7f3130004270 con 0x7f311c009090 2024-01-24T23:29:34.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.099+0000 7f312a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:29:34.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.099+0000 7f312a7fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:34.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.099+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 >> v1:172.21.15.132:6790/0 conn(0x7f311c009090 legacy=0x7f311c009520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.115 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.099+0000 7f312a7fc640 1 -- 172.21.15.115:0/817069963 >> v1:172.21.15.132:6790/0 conn(0x7f311c009090 legacy=0x7f311c009520 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:34.116 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:29:34.116 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:34.116 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:34.116 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:29:34.116 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 48, 2024-01-24T23:29:34.116 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:29:34.116 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:29:34.116 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:29:34.116 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:29:34.116 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:29:34.116 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:29:34.116 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:29:34.117 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:29:34.117 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:29:34.117 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:29:34.117 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:34.117 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 4, 2024-01-24T23:29:34.117 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:34.117 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:29:34.117 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:29:34.117 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:34.117 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:34.117 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:34.117 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:34.117 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:34.117 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:34.117 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:34.118 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:34.118 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:34.118 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:34.118 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:34.118 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:34.118 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:29:34.118 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:29:34.118 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:34.118 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:34.118 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:34.118 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:34.118 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:34.118 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:34.118 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:34.119 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:34.119 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:34.119 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:34.119 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:34.119 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.119 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:29:34.119 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:29:34.119 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:29:34.119 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:29:34.119 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:29:34.119 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:29:34.119 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:34.119 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:34.120 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:29:34.120 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:29:34.120 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:29:34.120 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:29:34.120 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:29:34.120 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:29:34.120 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:29:34.120 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:34.120 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:29:34.120 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:34.120 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:34.120 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:34.120 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:34.120 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:34.121 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:34.121 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:34.121 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:34.121 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:34.121 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:34.121 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:34.121 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:34.121 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:29:34.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.121 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:29:34.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.121 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:34.121 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:34.121 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.122 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.122 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.122 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:29:34.122 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.122 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.122 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.122 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:29:34.122 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:29:34.122 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.122 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.122 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.123 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:34.123 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:34.123 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.123 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.123 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.123 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:29:34.123 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.123 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.123 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.123 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:29:34.124 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:29:34.124 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.124 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.124 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.124 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:34.124 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:34.124 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.124 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.124 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.124 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:29:34.124 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.125 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.125 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.125 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:29:34.125 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:29:34.125 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.125 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.125 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.125 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:34.125 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:34.125 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.125 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.126 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.126 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:29:34.126 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.126 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.126 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.126 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:29:34.126 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:29:34.126 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.126 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.127 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.127 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:34.127 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:34.127 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.127 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.127 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.127 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:29:34.127 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.127 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.127 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.128 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:29:34.128 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:29:34.128 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.128 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.128 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.128 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:34.128 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:34.128 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.128 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.129 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.129 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:29:34.129 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.129 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.129 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.129 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:29:34.129 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:29:34.129 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.129 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.129 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.130 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:34.130 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:34.130 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.130 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.130 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.130 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:29:34.130 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.130 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.130 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.131 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:29:34.131 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:29:34.131 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.131 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.131 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.131 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:34.131 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:34.131 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.131 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.131 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.132 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:29:34.132 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.132 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.132 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.132 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:29:34.132 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:29:34.132 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.132 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.132 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.133 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:34.133 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:34.133 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.133 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.133 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.133 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:29:34.133 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.133 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.133 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.133 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:29:34.133 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:29:34.134 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.134 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.134 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.134 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.134 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.134 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:29:34.134 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:29:34.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.134 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:34.134 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:34.135 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:34.135 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.135 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:34.135 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:29:34.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.135 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:34.135 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:34.135 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:29:34.135 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.135 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:34.135 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:29:34.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.136 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:34.136 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:34.136 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 4 2024-01-24T23:29:34.136 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.136 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.136 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:29:34.136 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:29:34.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.099+0000 7f313bbe6640 1 -- 172.21.15.115:0/817069963 >> v1:172.21.15.132:6812/1835401922 conn(0x7f3118037740 legacy=0x7f3118039c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.099+0000 7f313bbe6640 10 monclient: shutdown 2024-01-24T23:29:34.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.099+0000 7f313bbe6640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:34.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.099+0000 7f313bbe6640 1 -- 172.21.15.115:0/817069963 >> v1:172.21.15.115:6792/0 conn(0x7f31340720b0 legacy=0x7f31341c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.099+0000 7f313a183640 1 -- 172.21.15.115:0/817069963 reap_dead start 2024-01-24T23:29:34.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.099+0000 7f313a183640 1 -- 172.21.15.115:0/817069963 reap_dead start 2024-01-24T23:29:34.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.099+0000 7f313bbe6640 1 -- 172.21.15.115:0/817069963 shutdown_connections 2024-01-24T23:29:34.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.099+0000 7f313bbe6640 1 -- 172.21.15.115:0/817069963 >> 172.21.15.115:0/817069963 conn(0x7f31340a5730 msgr2=0x7f3134129720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:34.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.099+0000 7f313bbe6640 1 -- 172.21.15.115:0/817069963 shutdown_connections 2024-01-24T23:29:34.138 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.099+0000 7f313bbe6640 1 -- 172.21.15.115:0/817069963 wait complete. 2024-01-24T23:29:34.139 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:29:34.139 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-01-24T23:29:34.139 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-01-24T23:29:34.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.247+0000 7f13dc43f640 10 monclient: get_monmap_and_config 2024-01-24T23:29:34.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.247+0000 7f13dc43f640 10 monclient: build_initial_monmap 2024-01-24T23:29:34.250 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.247+0000 7f13dc43f640 10 monclient: monmap: 2024-01-24T23:29:34.250 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:34.251 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.251 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:34.253382+0000 2024-01-24T23:29:34.251 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:34.253382+0000 2024-01-24T23:29:34.251 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:34.251 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.251 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:34.251 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:34.251 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:34.251 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:34.251 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:34.251 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:34.251 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:34.251 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:34.251 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:34.251 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.247+0000 7f13dc43f640 1 Processor -- start 2024-01-24T23:29:34.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.247+0000 7f13dc43f640 1 -- start start 2024-01-24T23:29:34.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.247+0000 7f13dc43f640 10 monclient: init 2024-01-24T23:29:34.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.247+0000 7f13dc43f640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:34.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.247+0000 7f13dc43f640 10 monclient: _add_conns ranks=[4,5,8,7,2,6,3,0,1] 2024-01-24T23:29:34.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.247+0000 7f13dc43f640 10 monclient(hunting): picked mon.noname-c con 0x7f13d4133bf0 addr v1:172.21.15.115:6791/0 2024-01-24T23:29:34.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.247+0000 7f13dc43f640 10 monclient(hunting): picked mon.noname-h con 0x7f13d4133fd0 addr v1:172.21.15.132:6791/0 2024-01-24T23:29:34.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.247+0000 7f13dc43f640 10 monclient(hunting): picked mon.noname-e con 0x7f13d41343b0 addr v1:172.21.15.115:6793/0 2024-01-24T23:29:34.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.247+0000 7f13dc43f640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d404ed70 con 0x7f13d4133bf0 2024-01-24T23:29:34.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.247+0000 7f13dc43f640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d40a4b60 con 0x7f13d4133fd0 2024-01-24T23:29:34.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.247+0000 7f13dc43f640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d409ae50 con 0x7f13d41343b0 2024-01-24T23:29:34.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.247+0000 7f13dc43f640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:34.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.247+0000 7f13dc43f640 10 monclient(hunting): authenticate will time out at 1453.175844s 2024-01-24T23:29:34.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.247+0000 7f13da1db640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f13d4133bf0 0x7f13d409bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:58966/0 (socket says 172.21.15.115:58966) 2024-01-24T23:29:34.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13da1db640 1 -- 172.21.15.115:0/1276391440 learned_addr learned my addr 172.21.15.115:0/1276391440 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:34.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.247+0000 7f13da9dc640 1 --1- 172.21.15.115:0/1276391440 >> v1:172.21.15.115:6793/0 conn(0x7f13d41343b0 0x7f13d413f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:55948/0 (socket says 172.21.15.115:55948) 2024-01-24T23:29:34.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 1 -- 172.21.15.115:0/1276391440 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4122252719 0 0) 0x7f13d404ed70 con 0x7f13d4133bf0 2024-01-24T23:29:34.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.261 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 10 monclient(hunting): my global_id is 95269 2024-01-24T23:29:34.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 1 -- 172.21.15.115:0/1276391440 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13c0003530 con 0x7f13d4133bf0 2024-01-24T23:29:34.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 1 -- 172.21.15.115:0/1276391440 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 819375648 0 0) 0x7f13d409ae50 con 0x7f13d41343b0 2024-01-24T23:29:34.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 10 monclient(hunting): my global_id is 86363 2024-01-24T23:29:34.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 1 -- 172.21.15.115:0/1276391440 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13d404ed70 con 0x7f13d41343b0 2024-01-24T23:29:34.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 1 -- 172.21.15.115:0/1276391440 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1377719518 0 0) 0x7f13c0003530 con 0x7f13d4133bf0 2024-01-24T23:29:34.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 1 -- 172.21.15.115:0/1276391440 >> v1:172.21.15.115:6793/0 conn(0x7f13d41343b0 legacy=0x7f13d413f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 1 -- 172.21.15.115:0/1276391440 >> v1:172.21.15.132:6791/0 conn(0x7f13d4133fd0 legacy=0x7f13d413bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 10 monclient: _finish_hunting 0 2024-01-24T23:29:34.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 1 monclient: found mon.noname-c 2024-01-24T23:29:34.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:34.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:29:34.262 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 1 -- 172.21.15.115:0/1276391440 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d409b130 con 0x7f13d4133bf0 2024-01-24T23:29:34.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 10 monclient: _finish_auth 0 2024-01-24T23:29:34.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 10 monclient: _check_auth_tickets 2024-01-24T23:29:34.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:34.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 1 -- 172.21.15.115:0/1276391440 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f13d00017c0 con 0x7f13d4133bf0 2024-01-24T23:29:34.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:29:34.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 10 monclient: dump: 2024-01-24T23:29:34.263 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.263 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.263 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.263 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.263 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.263 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.263 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.263 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.264 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.264 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.264 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.264 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.264 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.264 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.264 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.264 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 5 monclient: authenticate success, global_id 95269 2024-01-24T23:29:34.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:34.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 1 -- 172.21.15.115:0/1276391440 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f13d0001a20 con 0x7f13d4133bf0 2024-01-24T23:29:34.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:34.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 1 -- 172.21.15.115:0/1276391440 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f13d0004bf0 con 0x7f13d4133bf0 2024-01-24T23:29:34.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:29:34.265 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13d91d9640 10 monclient: dump: 2024-01-24T23:29:34.265 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.265 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.265 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.265 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.265 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.265 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.265 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.265 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.265 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.265 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.265 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.265 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.265 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.266 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.266 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.266 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:34.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 4 set_mon_vals no callback set 2024-01-24T23:29:34.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 10 monclient: shutdown 2024-01-24T23:29:34.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:34.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 1 -- 172.21.15.115:0/1276391440 >> v1:172.21.15.115:6791/0 conn(0x7f13d4133bf0 legacy=0x7f13d409bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 1 -- 172.21.15.115:0/1276391440 shutdown_connections 2024-01-24T23:29:34.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 1 -- 172.21.15.115:0/1276391440 >> 172.21.15.115:0/1276391440 conn(0x7f13d40662f0 msgr2=0x7f13d4066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:34.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 1 -- 172.21.15.115:0/1276391440 shutdown_connections 2024-01-24T23:29:34.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 1 -- 172.21.15.115:0/1276391440 wait complete. 2024-01-24T23:29:34.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 10 monclient: build_initial_monmap 2024-01-24T23:29:34.266 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 10 monclient: monmap: 2024-01-24T23:29:34.267 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:34.267 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:34.267 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:34.267 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:34.267 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:34.267 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:34.267 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:34.267 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:34.267 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:34.267 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:34.267 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:34.267 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:34.267 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:34.267 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:34.267 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:34.268 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 1 Processor -- start 2024-01-24T23:29:34.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 1 -- start start 2024-01-24T23:29:34.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 10 monclient: init 2024-01-24T23:29:34.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:34.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 10 monclient: _add_conns ranks=[8,2,4,6,5,3,1,0,7] 2024-01-24T23:29:34.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 10 monclient(hunting): picked mon.noname-e con 0x7f13d4133bf0 addr v1:172.21.15.115:6793/0 2024-01-24T23:29:34.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 10 monclient(hunting): picked mon.noname-b con 0x7f13d4133fd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:29:34.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 10 monclient(hunting): picked mon.noname-c con 0x7f13d41343b0 addr v1:172.21.15.115:6791/0 2024-01-24T23:29:34.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d409c810 con 0x7f13d4133fd0 2024-01-24T23:29:34.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d40a5990 con 0x7f13d41343b0 2024-01-24T23:29:34.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d40a1f00 con 0x7f13d4133bf0 2024-01-24T23:29:34.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:34.268 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 10 monclient(hunting): authenticate will time out at 1453.178867s 2024-01-24T23:29:34.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13da1db640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f13d4133bf0 0x7f13d4132ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:55962/0 (socket says 172.21.15.115:55962) 2024-01-24T23:29:34.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13da1db640 1 -- 172.21.15.115:0/1529285814 learned_addr learned my addr 172.21.15.115:0/1529285814 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:34.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3730396614 0 0) 0x7f13d40a1f00 con 0x7f13d4133bf0 2024-01-24T23:29:34.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient(hunting): my global_id is 86372 2024-01-24T23:29:34.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13b8003530 con 0x7f13d4133bf0 2024-01-24T23:29:34.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 873384386 0 0) 0x7f13d40a5990 con 0x7f13d41343b0 2024-01-24T23:29:34.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient(hunting): my global_id is 95278 2024-01-24T23:29:34.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13d40a1f00 con 0x7f13d41343b0 2024-01-24T23:29:34.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 569562269 0 0) 0x7f13d409c810 con 0x7f13d4133fd0 2024-01-24T23:29:34.269 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient(hunting): my global_id is 154136 2024-01-24T23:29:34.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13d40a5990 con 0x7f13d4133fd0 2024-01-24T23:29:34.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 742587749 0 0) 0x7f13b8003530 con 0x7f13d4133bf0 2024-01-24T23:29:34.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f13d409c810 con 0x7f13d4133bf0 2024-01-24T23:29:34.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f13d00017c0 con 0x7f13d4133bf0 2024-01-24T23:29:34.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:29:34.270 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient(hunting): dump: 2024-01-24T23:29:34.270 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.270 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.270 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.270 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.270 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.270 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.270 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.271 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.271 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.271 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.271 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.271 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.271 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.271 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.271 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.271 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:34.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4094098037 0 0) 0x7f13d40a1f00 con 0x7f13d41343b0 2024-01-24T23:29:34.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f13b8003530 con 0x7f13d41343b0 2024-01-24T23:29:34.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f13cc001c80 con 0x7f13d41343b0 2024-01-24T23:29:34.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:29:34.271 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient(hunting): dump: 2024-01-24T23:29:34.271 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.272 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.272 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.272 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.272 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.272 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.272 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.272 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.272 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.272 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.272 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.272 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.272 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.272 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.272 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.273 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3457463729 0 0) 0x7f13d40a5990 con 0x7f13d4133fd0 2024-01-24T23:29:34.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f13d40a1f00 con 0x7f13d4133fd0 2024-01-24T23:29:34.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f13c40019a0 con 0x7f13d4133fd0 2024-01-24T23:29:34.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:29:34.273 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient(hunting): dump: 2024-01-24T23:29:34.273 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.273 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.273 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.273 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.273 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.273 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.273 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.274 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.274 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.274 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.274 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.274 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.274 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.274 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.274 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.274 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1873426490 0 0) 0x7f13d409c810 con 0x7f13d4133bf0 2024-01-24T23:29:34.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 >> v1:172.21.15.115:6791/0 conn(0x7f13d41343b0 legacy=0x7f13d41d3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 >> v1:172.21.15.115:6790/0 conn(0x7f13d4133fd0 legacy=0x7f13d41d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient: _finish_hunting 0 2024-01-24T23:29:34.274 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 1 monclient: found mon.e 2024-01-24T23:29:34.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:34.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:29:34.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d40a2630 con 0x7f13d4133bf0 2024-01-24T23:29:34.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient: _finish_auth 0 2024-01-24T23:29:34.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient: _check_auth_tickets 2024-01-24T23:29:34.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:34.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 5 monclient: authenticate success, global_id 86372 2024-01-24T23:29:34.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 10 monclient: _renew_subs 2024-01-24T23:29:34.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:29:34.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 1 -- 172.21.15.115:0/1529285814 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d41208b0 con 0x7f13d4133bf0 2024-01-24T23:29:34.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 10 monclient: _renew_subs 2024-01-24T23:29:34.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:29:34.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13dc43f640 1 -- 172.21.15.115:0/1529285814 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d40a33b0 con 0x7f13d4133bf0 2024-01-24T23:29:34.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f13d0004c00 con 0x7f13d4133bf0 2024-01-24T23:29:34.275 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.251+0000 7f13caffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:34.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f13d00051f0 con 0x7f13d4133bf0 2024-01-24T23:29:34.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13d89d8640 4 set_mon_vals no callback set 2024-01-24T23:29:34.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:29:34.276 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 10 monclient: dump: 2024-01-24T23:29:34.276 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.276 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.276 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.276 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.276 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.276 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.276 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.276 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.276 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.277 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.277 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.277 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.277 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.277 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.277 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.277 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13dc43f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:34.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13dc43f640 1 -- 172.21.15.115:0/1529285814 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f13bc0065d0 con 0x7f13bc002870 2024-01-24T23:29:34.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f13d000fc70 con 0x7f13d4133bf0 2024-01-24T23:29:34.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7f13d0044720 con 0x7f13d4133bf0 2024-01-24T23:29:34.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13da9dc640 10 monclient: get_auth_request con 0x7f13b8037cf0 auth_method 0 2024-01-24T23:29:34.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2388802745 0 0) 0x7f13bc0065d0 con 0x7f13bc002870 2024-01-24T23:29:34.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.277 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 10 monclient(hunting): my global_id is 7079 2024-01-24T23:29:34.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13d409c810 con 0x7f13bc002870 2024-01-24T23:29:34.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 265452047 0 0) 0x7f13d409c810 con 0x7f13bc002870 2024-01-24T23:29:34.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f13bc0065d0 con 0x7f13bc002870 2024-01-24T23:29:34.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f13c4003940 con 0x7f13bc002870 2024-01-24T23:29:34.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:29:34.278 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 10 monclient: dump: 2024-01-24T23:29:34.278 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.278 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.278 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.278 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.278 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.278 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.279 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.279 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.279 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.279 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.279 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.279 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.279 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.279 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.279 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.279 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 553334431 0 0) 0x7f13bc0065d0 con 0x7f13bc002870 2024-01-24T23:29:34.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 --> v1:172.21.15.132:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f13bc00a120 con 0x7f13bc002870 2024-01-24T23:29:34.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f13c4003e40 con 0x7f13bc002870 2024-01-24T23:29:34.279 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:34.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:34.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 >> v1:172.21.15.132:6791/0 conn(0x7f13bc002870 legacy=0x7f13bc004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.280 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.255+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 >> v1:172.21.15.132:6791/0 conn(0x7f13bc002870 legacy=0x7f13bc004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:34.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13dc43f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:29:34.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13dc43f640 1 -- 172.21.15.115:0/1529285814 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f13bc009e10 con 0x7f13bc0090d0 2024-01-24T23:29:34.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1303687319 0 0) 0x7f13bc009e10 con 0x7f13bc0090d0 2024-01-24T23:29:34.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13caffd640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13caffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.281 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13caffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13caffd640 10 monclient(hunting): my global_id is 7088 2024-01-24T23:29:34.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13bc0065d0 con 0x7f13bc0090d0 2024-01-24T23:29:34.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3116580221 0 0) 0x7f13bc0065d0 con 0x7f13bc0090d0 2024-01-24T23:29:34.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f13bc009e10 con 0x7f13bc0090d0 2024-01-24T23:29:34.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f13c40045b0 con 0x7f13bc0090d0 2024-01-24T23:29:34.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13caffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13caffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:29:34.282 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13caffd640 10 monclient: dump: 2024-01-24T23:29:34.282 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.282 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.282 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.282 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.282 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.282 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.282 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.283 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.283 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.283 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.283 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.283 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.283 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.283 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.283 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.283 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1164934084 0 0) 0x7f13bc009e10 con 0x7f13bc0090d0 2024-01-24T23:29:34.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 --> v1:172.21.15.132:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f13bc00c310 con 0x7f13bc0090d0 2024-01-24T23:29:34.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7715 (unknown 0 0 201219326) 0x7f13c40047a0 con 0x7f13bc0090d0 2024-01-24T23:29:34.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13caffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:29:34.283 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13caffd640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:34.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 >> v1:172.21.15.132:6791/0 conn(0x7f13bc0090d0 legacy=0x7f13bc009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.284 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13caffd640 1 -- 172.21.15.115:0/1529285814 >> v1:172.21.15.132:6791/0 conn(0x7f13bc0090d0 legacy=0x7f13bc009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:34.284 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:29:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "leader", 2024-01-24T23:29:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 48, 2024-01-24T23:29:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:29:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:29:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:29:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:29:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:29:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:29:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:29:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:29:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:29:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:29:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 4, 2024-01-24T23:29:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:29:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:29:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:29:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:29:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:29:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:29:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:29:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:29:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:29:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:29:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:29:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:29:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:29:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:29:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:29:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:29:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:29:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:29:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:29:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:29:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:29:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:29:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:29:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:29:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:29:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:29:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:29:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:29:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:29:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:29:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:29:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:29:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:29:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:29:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:29:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:29:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:29:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:29:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:29:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:29:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:29:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:29:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:29:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:29:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:29:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:29:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:29:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:29:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:29:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": [ 2024-01-24T23:29:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:29:34.303 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:29:34.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13dc43f640 1 -- 172.21.15.115:0/1529285814 >> v1:172.21.15.132:6812/1835401922 conn(0x7f13b8037cf0 legacy=0x7f13b803a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13dc43f640 10 monclient: shutdown 2024-01-24T23:29:34.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13dc43f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:34.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13dc43f640 1 -- 172.21.15.115:0/1529285814 >> v1:172.21.15.115:6793/0 conn(0x7f13d4133bf0 legacy=0x7f13d4132ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13da9dc640 1 -- 172.21.15.115:0/1529285814 reap_dead start 2024-01-24T23:29:34.304 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13da9dc640 1 -- 172.21.15.115:0/1529285814 reap_dead start 2024-01-24T23:29:34.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13dc43f640 1 -- 172.21.15.115:0/1529285814 shutdown_connections 2024-01-24T23:29:34.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13dc43f640 1 -- 172.21.15.115:0/1529285814 >> 172.21.15.115:0/1529285814 conn(0x7f13d40662f0 msgr2=0x7f13d40a8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:34.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13dc43f640 1 -- 172.21.15.115:0/1529285814 shutdown_connections 2024-01-24T23:29:34.305 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.263+0000 7f13dc43f640 1 -- 172.21.15.115:0/1529285814 wait complete. 2024-01-24T23:29:34.305 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:29:34.305 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-01-24T23:29:34.305 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-01-24T23:29:34.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.411+0000 7f22ee3c0640 10 monclient: get_monmap_and_config 2024-01-24T23:29:34.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.411+0000 7f22ee3c0640 10 monclient: build_initial_monmap 2024-01-24T23:29:34.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.411+0000 7f22ee3c0640 10 monclient: monmap: 2024-01-24T23:29:34.416 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:34.416 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.416 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:34.418566+0000 2024-01-24T23:29:34.416 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:34.418566+0000 2024-01-24T23:29:34.416 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:34.416 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.416 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:34.416 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:34.416 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:34.416 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:34.417 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:34.417 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:34.417 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:34.417 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:34.417 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:34.417 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 1 Processor -- start 2024-01-24T23:29:34.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 1 -- start start 2024-01-24T23:29:34.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 10 monclient: init 2024-01-24T23:29:34.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:34.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 10 monclient: _add_conns ranks=[7,8,6,2,3,0,1,5,4] 2024-01-24T23:29:34.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 10 monclient(hunting): picked mon.noname-i con 0x7f22e80a0a10 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:34.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 10 monclient(hunting): picked mon.noname-e con 0x7f22e809f1e0 addr v1:172.21.15.115:6793/0 2024-01-24T23:29:34.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 10 monclient(hunting): picked mon.noname-d con 0x7f22e813d360 addr v1:172.21.15.115:6792/0 2024-01-24T23:29:34.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e804ed70 con 0x7f22e813d360 2024-01-24T23:29:34.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e80a1500 con 0x7f22e80a0a10 2024-01-24T23:29:34.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e806aaf0 con 0x7f22e809f1e0 2024-01-24T23:29:34.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:34.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 10 monclient(hunting): authenticate will time out at 1453.340975s 2024-01-24T23:29:34.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ec95d640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f22e813d360 0x7f22e813f750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:53722/0 (socket says 172.21.15.115:53722) 2024-01-24T23:29:34.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e77fe640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f22e809f1e0 0x7f22e813bc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:55976/0 (socket says 172.21.15.115:55976) 2024-01-24T23:29:34.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ec95d640 1 -- 172.21.15.115:0/398111501 learned_addr learned my addr 172.21.15.115:0/398111501 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:34.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 1 -- 172.21.15.115:0/398111501 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 996853038 0 0) 0x7f22e806aaf0 con 0x7f22e809f1e0 2024-01-24T23:29:34.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 10 monclient(hunting): my global_id is 86381 2024-01-24T23:29:34.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 1 -- 172.21.15.115:0/398111501 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22d0003530 con 0x7f22e809f1e0 2024-01-24T23:29:34.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 1 -- 172.21.15.115:0/398111501 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2954551475 0 0) 0x7f22e804ed70 con 0x7f22e813d360 2024-01-24T23:29:34.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 10 monclient(hunting): my global_id is 7323 2024-01-24T23:29:34.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 1 -- 172.21.15.115:0/398111501 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22e806aaf0 con 0x7f22e813d360 2024-01-24T23:29:34.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 1 -- 172.21.15.115:0/398111501 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1105257485 0 0) 0x7f22d0003530 con 0x7f22e809f1e0 2024-01-24T23:29:34.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 1 -- 172.21.15.115:0/398111501 >> v1:172.21.15.132:6792/0 conn(0x7f22e80a0a10 legacy=0x7f22e80a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 1 -- 172.21.15.115:0/398111501 >> v1:172.21.15.115:6792/0 conn(0x7f22e813d360 legacy=0x7f22e813f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.426 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:29:34.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 1 monclient: found mon.noname-e 2024-01-24T23:29:34.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:34.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:29:34.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 1 -- 172.21.15.115:0/398111501 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e809e560 con 0x7f22e809f1e0 2024-01-24T23:29:34.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 10 monclient: _finish_auth 0 2024-01-24T23:29:34.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:29:34.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:34.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 1 -- 172.21.15.115:0/398111501 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f22dc0017c0 con 0x7f22e809f1e0 2024-01-24T23:29:34.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:29:34.427 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 10 monclient: dump: 2024-01-24T23:29:34.427 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.427 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.427 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.428 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.428 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.428 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.428 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.428 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.428 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.428 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.428 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.428 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.428 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.428 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.428 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.428 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 5 monclient: authenticate success, global_id 86381 2024-01-24T23:29:34.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:34.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 1 -- 172.21.15.115:0/398111501 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f22dc001a60 con 0x7f22e809f1e0 2024-01-24T23:29:34.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:34.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:34.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 4 set_mon_vals no callback set 2024-01-24T23:29:34.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 1 -- 172.21.15.115:0/398111501 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f22dc004b30 con 0x7f22e809f1e0 2024-01-24T23:29:34.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:29:34.429 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e6ffd640 10 monclient: dump: 2024-01-24T23:29:34.429 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.429 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.429 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.429 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.429 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.429 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.429 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.430 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.430 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.430 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.430 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.430 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.430 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.430 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.430 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.430 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 10 monclient: shutdown 2024-01-24T23:29:34.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:34.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 1 -- 172.21.15.115:0/398111501 >> v1:172.21.15.115:6793/0 conn(0x7f22e809f1e0 legacy=0x7f22e813bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 1 -- 172.21.15.115:0/398111501 shutdown_connections 2024-01-24T23:29:34.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 1 -- 172.21.15.115:0/398111501 >> 172.21.15.115:0/398111501 conn(0x7f22e80a7660 msgr2=0x7f22e80a3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:34.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 1 -- 172.21.15.115:0/398111501 shutdown_connections 2024-01-24T23:29:34.430 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 1 -- 172.21.15.115:0/398111501 wait complete. 2024-01-24T23:29:34.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 10 monclient: build_initial_monmap 2024-01-24T23:29:34.431 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 10 monclient: monmap: 2024-01-24T23:29:34.431 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:34.431 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:34.431 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:34.431 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:34.431 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:34.431 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:34.431 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:34.431 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:34.431 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:34.431 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:34.431 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:34.431 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:34.431 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:34.432 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:34.432 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:34.432 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 1 Processor -- start 2024-01-24T23:29:34.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 1 -- start start 2024-01-24T23:29:34.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 10 monclient: init 2024-01-24T23:29:34.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:34.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 10 monclient: _add_conns ranks=[2,4,1,7,8,6,3,5,0] 2024-01-24T23:29:34.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 10 monclient(hunting): picked mon.noname-b con 0x7f22e809f1e0 addr v1:172.21.15.115:6790/0 2024-01-24T23:29:34.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 10 monclient(hunting): picked mon.noname-c con 0x7f22e80a0a10 addr v1:172.21.15.115:6791/0 2024-01-24T23:29:34.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 10 monclient(hunting): picked mon.noname-f con 0x7f22e813d360 addr v1:172.21.15.132:6789/0 2024-01-24T23:29:34.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e8071eb0 con 0x7f22e813d360 2024-01-24T23:29:34.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e8135e70 con 0x7f22e809f1e0 2024-01-24T23:29:34.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e8072400 con 0x7f22e80a0a10 2024-01-24T23:29:34.432 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22ee3c0640 10 monclient(hunting): authenticate will time out at 1453.343875s 2024-01-24T23:29:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e7fff640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f22e809f1e0 0x7f22e8137d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:43068/0 (socket says 172.21.15.115:43068) 2024-01-24T23:29:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.415+0000 7f22e7fff640 1 -- 172.21.15.115:0/487581806 learned_addr learned my addr 172.21.15.115:0/487581806 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172347651 0 0) 0x7f22e8135e70 con 0x7f22e809f1e0 2024-01-24T23:29:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient(hunting): my global_id is 154145 2024-01-24T23:29:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22d4003530 con 0x7f22e809f1e0 2024-01-24T23:29:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 786378235 0 0) 0x7f22e8072400 con 0x7f22e80a0a10 2024-01-24T23:29:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient(hunting): my global_id is 95287 2024-01-24T23:29:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22e8135e70 con 0x7f22e80a0a10 2024-01-24T23:29:34.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3379964150 0 0) 0x7f22e8135e70 con 0x7f22e80a0a10 2024-01-24T23:29:34.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f22e8072400 con 0x7f22e80a0a10 2024-01-24T23:29:34.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f22dc001780 con 0x7f22e80a0a10 2024-01-24T23:29:34.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:29:34.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient(hunting): dump: 2024-01-24T23:29:34.434 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.434 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.434 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.434 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.434 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.434 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.434 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.434 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.434 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.435 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.435 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.435 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.435 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.435 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.435 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.435 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:34.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2695820233 0 0) 0x7f22d4003530 con 0x7f22e809f1e0 2024-01-24T23:29:34.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f22e8135e70 con 0x7f22e809f1e0 2024-01-24T23:29:34.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f22d8001a60 con 0x7f22e809f1e0 2024-01-24T23:29:34.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:29:34.435 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient(hunting): dump: 2024-01-24T23:29:34.435 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.435 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.436 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.436 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.436 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.436 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.436 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.436 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.436 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.436 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.436 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.436 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.436 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.436 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.436 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.436 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3268615000 0 0) 0x7f22e8071eb0 con 0x7f22e813d360 2024-01-24T23:29:34.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient(hunting): my global_id is 124687 2024-01-24T23:29:34.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22d4003530 con 0x7f22e813d360 2024-01-24T23:29:34.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3812763593 0 0) 0x7f22e8072400 con 0x7f22e80a0a10 2024-01-24T23:29:34.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 >> v1:172.21.15.115:6790/0 conn(0x7f22e809f1e0 legacy=0x7f22e8137d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 >> v1:172.21.15.132:6789/0 conn(0x7f22e813d360 legacy=0x7f22e81d3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:29:34.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 monclient: found mon.c 2024-01-24T23:29:34.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:34.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:29:34.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e8137320 con 0x7f22e80a0a10 2024-01-24T23:29:34.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient: _finish_auth 0 2024-01-24T23:29:34.437 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:29:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22ee3c0640 5 monclient: authenticate success, global_id 95287 2024-01-24T23:29:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22ee3c0640 10 monclient: _renew_subs 2024-01-24T23:29:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22ee3c0640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:29:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22ee3c0640 1 -- 172.21.15.115:0/487581806 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e8137600 con 0x7f22e80a0a10 2024-01-24T23:29:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22ee3c0640 10 monclient: _renew_subs 2024-01-24T23:29:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22ee3c0640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:29:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22ee3c0640 1 -- 172.21.15.115:0/487581806 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e81317a0 con 0x7f22e80a0a10 2024-01-24T23:29:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f22dc001110 con 0x7f22e80a0a10 2024-01-24T23:29:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f22dc005180 con 0x7f22e80a0a10 2024-01-24T23:29:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e67fc640 4 set_mon_vals no callback set 2024-01-24T23:29:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:29:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient: dump: 2024-01-24T23:29:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.439 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22ee3c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22ee3c0640 1 -- 172.21.15.115:0/487581806 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f22c80065d0 con 0x7f22c8002870 2024-01-24T23:29:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f22dc00fc00 con 0x7f22e80a0a10 2024-01-24T23:29:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7f22dc0446a0 con 0x7f22e80a0a10 2024-01-24T23:29:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22ec95d640 10 monclient: get_auth_request con 0x7f22d40378f0 auth_method 0 2024-01-24T23:29:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1990066048 0 0) 0x7f22c80065d0 con 0x7f22c8002870 2024-01-24T23:29:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient(hunting): my global_id is 154168 2024-01-24T23:29:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22e8072400 con 0x7f22c8002870 2024-01-24T23:29:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2661129738 0 0) 0x7f22e8072400 con 0x7f22c8002870 2024-01-24T23:29:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f22c80065d0 con 0x7f22c8002870 2024-01-24T23:29:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f22d8003b30 con 0x7f22c8002870 2024-01-24T23:29:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:29:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient: dump: 2024-01-24T23:29:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.442 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.442 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.442 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.442 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1067627368 0 0) 0x7f22c80065d0 con 0x7f22c8002870 2024-01-24T23:29:34.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 --> v1:172.21.15.132:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22c800a120 con 0x7f22c8002870 2024-01-24T23:29:34.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 <== mon.7 v1:172.21.15.132:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f22d8003f50 con 0x7f22c8002870 2024-01-24T23:29:34.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:34.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:34.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 >> v1:172.21.15.132:6792/0 conn(0x7f22c8002870 legacy=0x7f22c8004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.442 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.419+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 >> v1:172.21.15.132:6792/0 conn(0x7f22c8002870 legacy=0x7f22c8004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:34.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.427+0000 7f22ee3c0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:29:34.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.427+0000 7f22ee3c0640 1 -- 172.21.15.115:0/487581806 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f22c8009df0 con 0x7f22c80090b0 2024-01-24T23:29:34.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.427+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2449969259 0 0) 0x7f22c8009df0 con 0x7f22c80090b0 2024-01-24T23:29:34.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.427+0000 7f22e4ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.427+0000 7f22e4ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.427+0000 7f22e4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.427+0000 7f22e4ff9640 10 monclient(hunting): my global_id is 154177 2024-01-24T23:29:34.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.427+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22c80065d0 con 0x7f22c80090b0 2024-01-24T23:29:34.444 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.427+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1975014131 0 0) 0x7f22c80065d0 con 0x7f22c80090b0 2024-01-24T23:29:34.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.427+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f22c8009df0 con 0x7f22c80090b0 2024-01-24T23:29:34.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.427+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f22d8001d70 con 0x7f22c80090b0 2024-01-24T23:29:34.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.427+0000 7f22e4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.427+0000 7f22e4ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:29:34.445 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.427+0000 7f22e4ff9640 10 monclient: dump: 2024-01-24T23:29:34.445 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.445 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.445 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.445 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.445 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.445 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.446 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.446 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.446 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.446 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.446 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.446 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.446 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.446 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.446 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.446 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.427+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1271489283 0 0) 0x7f22c8009df0 con 0x7f22c80090b0 2024-01-24T23:29:34.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.427+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 --> v1:172.21.15.132:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f22c800c310 con 0x7f22c80090b0 2024-01-24T23:29:34.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.431+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 <== mon.7 v1:172.21.15.132:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 880728164) 0x7f22d8007d80 con 0x7f22c80090b0 2024-01-24T23:29:34.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.431+0000 7f22e4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:29:34.446 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.431+0000 7f22e4ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:34.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.431+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 >> v1:172.21.15.132:6792/0 conn(0x7f22c80090b0 legacy=0x7f22c8009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.447 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.431+0000 7f22e4ff9640 1 -- 172.21.15.115:0/487581806 >> v1:172.21.15.132:6792/0 conn(0x7f22c80090b0 legacy=0x7f22c8009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:34.447 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:29:34.447 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:34.447 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:34.447 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:29:34.447 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 48, 2024-01-24T23:29:34.447 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:29:34.447 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:29:34.447 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:29:34.447 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:29:34.448 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:29:34.448 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:29:34.448 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:29:34.448 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:29:34.448 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:29:34.448 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:29:34.448 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:34.448 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 4, 2024-01-24T23:29:34.448 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:34.448 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:29:34.448 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:29:34.448 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:34.448 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:34.448 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:34.448 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:34.449 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:34.449 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:34.449 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:34.449 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:34.449 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:34.449 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:34.449 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:34.449 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:34.449 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:29:34.449 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:29:34.449 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:34.449 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:34.449 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:34.449 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:34.449 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:34.450 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:34.450 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:34.450 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:34.450 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:34.450 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:34.450 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:34.450 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.450 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:29:34.450 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:29:34.450 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:29:34.450 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:29:34.450 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:29:34.450 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:29:34.451 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:34.451 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:34.451 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:29:34.451 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:29:34.451 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:29:34.451 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:29:34.451 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:29:34.451 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:29:34.451 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:29:34.451 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:34.451 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:29:34.451 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:34.451 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:34.451 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:34.451 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:34.452 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:34.452 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:34.452 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:34.452 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:34.452 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:34.452 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:34.452 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:34.452 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:34.452 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:29:34.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.452 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:29:34.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.452 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:34.452 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:34.452 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.453 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.453 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.453 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:29:34.453 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.453 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.453 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.453 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:29:34.453 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:29:34.453 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.453 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.453 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.454 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:34.454 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:34.454 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.454 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.454 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.454 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:29:34.454 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.454 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.454 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.454 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:29:34.454 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:29:34.454 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.455 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.455 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.455 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:34.455 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:34.455 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.455 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.455 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.455 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:29:34.455 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.455 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.455 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.456 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:29:34.456 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:29:34.456 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.456 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.456 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.456 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:34.456 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:34.456 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.456 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.456 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.456 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:29:34.456 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.457 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.457 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.457 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:29:34.457 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:29:34.457 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.457 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.457 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.457 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:34.457 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:34.457 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.457 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.458 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.458 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:29:34.458 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.458 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.458 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.458 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:29:34.458 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:29:34.458 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.458 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.458 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.458 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:34.458 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:34.459 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.459 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.459 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.459 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:29:34.459 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.459 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.459 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.459 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:29:34.459 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:29:34.459 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.459 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.459 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.460 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:34.460 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:34.460 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.460 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.460 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.460 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:29:34.460 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.460 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.460 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.460 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:29:34.460 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:29:34.460 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.461 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.461 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.461 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:34.461 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:34.461 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.461 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.461 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.461 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:29:34.461 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.461 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.461 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.461 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:29:34.462 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:29:34.462 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.462 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.462 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.462 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:34.462 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:34.462 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:34.462 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:34.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.462 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:34.462 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:29:34.462 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:34.463 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.463 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.463 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:29:34.463 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:29:34.463 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:34.463 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:34.463 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:34.463 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.463 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.463 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:29:34.463 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:29:34.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.464 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:34.464 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:34.464 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:34.464 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.464 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:34.464 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:29:34.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:34.464 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:34.464 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:34.464 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:34.464 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:34.464 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:34.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:34.464 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:29:34.465 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:29:34.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.431+0000 7f22ee3c0640 1 -- 172.21.15.115:0/487581806 >> v1:172.21.15.132:6812/1835401922 conn(0x7f22d40378f0 legacy=0x7f22d4039db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.431+0000 7f22ee3c0640 10 monclient: shutdown 2024-01-24T23:29:34.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.431+0000 7f22ee3c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:34.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.431+0000 7f22ee3c0640 1 -- 172.21.15.115:0/487581806 >> v1:172.21.15.115:6791/0 conn(0x7f22e80a0a10 legacy=0x7f22e8131090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.431+0000 7f22ec95d640 1 -- 172.21.15.115:0/487581806 reap_dead start 2024-01-24T23:29:34.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.431+0000 7f22ec95d640 1 -- 172.21.15.115:0/487581806 reap_dead start 2024-01-24T23:29:34.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.431+0000 7f22ee3c0640 1 -- 172.21.15.115:0/487581806 shutdown_connections 2024-01-24T23:29:34.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.431+0000 7f22ee3c0640 1 -- 172.21.15.115:0/487581806 >> 172.21.15.115:0/487581806 conn(0x7f22e80a7660 msgr2=0x7f22e806b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:34.466 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.431+0000 7f22ee3c0640 1 -- 172.21.15.115:0/487581806 shutdown_connections 2024-01-24T23:29:34.467 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.431+0000 7f22ee3c0640 1 -- 172.21.15.115:0/487581806 wait complete. 2024-01-24T23:29:34.467 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:29:34.467 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-01-24T23:29:34.467 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-01-24T23:29:34.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.571+0000 7f6e81f26640 10 monclient: get_monmap_and_config 2024-01-24T23:29:34.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.571+0000 7f6e81f26640 10 monclient: build_initial_monmap 2024-01-24T23:29:34.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.571+0000 7f6e81f26640 10 monclient: monmap: 2024-01-24T23:29:34.575 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:34.575 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.575 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:34.577776+0000 2024-01-24T23:29:34.575 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:34.577776+0000 2024-01-24T23:29:34.575 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:34.575 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.575 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:34.575 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:34.575 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:34.576 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:34.576 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:34.576 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:34.576 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:34.576 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:34.576 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:34.576 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.582 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.571+0000 7f6e81f26640 1 Processor -- start 2024-01-24T23:29:34.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.571+0000 7f6e81f26640 1 -- start start 2024-01-24T23:29:34.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.571+0000 7f6e81f26640 10 monclient: init 2024-01-24T23:29:34.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.571+0000 7f6e81f26640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:34.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.571+0000 7f6e81f26640 10 monclient: _add_conns ranks=[6,5,0,7,8,2,3,4,1] 2024-01-24T23:29:34.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.571+0000 7f6e81f26640 10 monclient(hunting): picked mon.noname-d con 0x7f6e7c09e740 addr v1:172.21.15.115:6792/0 2024-01-24T23:29:34.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.571+0000 7f6e81f26640 10 monclient(hunting): picked mon.noname-h con 0x7f6e7c09f230 addr v1:172.21.15.132:6791/0 2024-01-24T23:29:34.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.571+0000 7f6e81f26640 10 monclient(hunting): picked mon.noname-a con 0x7f6e7c071e90 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:34.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.571+0000 7f6e81f26640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7c04ed70 con 0x7f6e7c071e90 2024-01-24T23:29:34.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.571+0000 7f6e81f26640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7c0a1400 con 0x7f6e7c09f230 2024-01-24T23:29:34.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7c0a37d0 con 0x7f6e7c09e740 2024-01-24T23:29:34.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:34.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 10 monclient(hunting): authenticate will time out at 1453.500015s 2024-01-24T23:29:34.583 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7b7fe640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f6e7c09e740 0x7f6e7c09eb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:53736/0 (socket says 172.21.15.115:53736) 2024-01-24T23:29:34.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7b7fe640 1 -- 172.21.15.115:0/3452207136 learned_addr learned my addr 172.21.15.115:0/3452207136 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:34.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 1 -- 172.21.15.115:0/3452207136 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3782146411 0 0) 0x7f6e7c0a37d0 con 0x7f6e7c09e740 2024-01-24T23:29:34.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 10 monclient(hunting): my global_id is 7332 2024-01-24T23:29:34.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 1 -- 172.21.15.115:0/3452207136 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e64003530 con 0x7f6e7c09e740 2024-01-24T23:29:34.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 1 -- 172.21.15.115:0/3452207136 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3558588210 0 0) 0x7f6e7c04ed70 con 0x7f6e7c071e90 2024-01-24T23:29:34.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 10 monclient(hunting): my global_id is 154197 2024-01-24T23:29:34.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 1 -- 172.21.15.115:0/3452207136 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e7c0a37d0 con 0x7f6e7c071e90 2024-01-24T23:29:34.584 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 1 -- 172.21.15.115:0/3452207136 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1986871639 0 0) 0x7f6e64003530 con 0x7f6e7c09e740 2024-01-24T23:29:34.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 1 -- 172.21.15.115:0/3452207136 >> v1:172.21.15.132:6791/0 conn(0x7f6e7c09f230 legacy=0x7f6e7c139b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 1 -- 172.21.15.115:0/3452207136 >> v1:172.21.15.115:6789/0 conn(0x7f6e7c071e90 legacy=0x7f6e7c13d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:29:34.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 1 monclient: found mon.noname-d 2024-01-24T23:29:34.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:34.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:29:34.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 1 -- 172.21.15.115:0/3452207136 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c0a3120 con 0x7f6e7c09e740 2024-01-24T23:29:34.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 10 monclient: _finish_auth 0 2024-01-24T23:29:34.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:29:34.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:34.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 1 -- 172.21.15.115:0/3452207136 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6e740017c0 con 0x7f6e7c09e740 2024-01-24T23:29:34.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 5 monclient: authenticate success, global_id 7332 2024-01-24T23:29:34.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:34.585 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:29:34.586 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 10 monclient: dump: 2024-01-24T23:29:34.586 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.586 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.586 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.586 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.586 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.586 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.586 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.586 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.586 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.586 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.586 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.586 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.586 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.587 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.587 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.587 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:34.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 1 -- 172.21.15.115:0/3452207136 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6e74001a60 con 0x7f6e7c09e740 2024-01-24T23:29:34.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:34.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:34.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 4 set_mon_vals no callback set 2024-01-24T23:29:34.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 1 -- 172.21.15.115:0/3452207136 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6e74004bf0 con 0x7f6e7c09e740 2024-01-24T23:29:34.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:29:34.587 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7a7fc640 10 monclient: dump: 2024-01-24T23:29:34.587 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.587 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.588 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.588 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.588 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.588 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.588 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.588 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.588 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.588 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.588 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.588 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.588 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.588 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.588 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.588 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 10 monclient: shutdown 2024-01-24T23:29:34.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:34.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 1 -- 172.21.15.115:0/3452207136 >> v1:172.21.15.115:6792/0 conn(0x7f6e7c09e740 legacy=0x7f6e7c09eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 1 -- 172.21.15.115:0/3452207136 shutdown_connections 2024-01-24T23:29:34.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 1 -- 172.21.15.115:0/3452207136 >> 172.21.15.115:0/3452207136 conn(0x7f6e7c0666f0 msgr2=0x7f6e7c066b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:34.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 1 -- 172.21.15.115:0/3452207136 shutdown_connections 2024-01-24T23:29:34.589 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 1 -- 172.21.15.115:0/3452207136 wait complete. 2024-01-24T23:29:34.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 10 monclient: build_initial_monmap 2024-01-24T23:29:34.591 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 10 monclient: monmap: 2024-01-24T23:29:34.591 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:34.591 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:34.591 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:34.591 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:34.591 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:34.592 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:34.592 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:34.592 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:34.592 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:34.592 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:34.592 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:34.592 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:34.592 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:34.592 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:34.592 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:34.592 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 1 Processor -- start 2024-01-24T23:29:34.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 1 -- start start 2024-01-24T23:29:34.592 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 10 monclient: init 2024-01-24T23:29:34.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:34.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 10 monclient: _add_conns ranks=[1,0,4,7,8,3,5,6,2] 2024-01-24T23:29:34.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 10 monclient(hunting): picked mon.noname-f con 0x7f6e7c071e90 addr v1:172.21.15.132:6789/0 2024-01-24T23:29:34.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 10 monclient(hunting): picked mon.noname-a con 0x7f6e7c09e740 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:34.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 10 monclient(hunting): picked mon.noname-c con 0x7f6e7c1ba110 addr v1:172.21.15.115:6791/0 2024-01-24T23:29:34.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7c09f640 con 0x7f6e7c09e740 2024-01-24T23:29:34.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7c06c230 con 0x7f6e7c071e90 2024-01-24T23:29:34.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7c0722e0 con 0x7f6e7c1ba110 2024-01-24T23:29:34.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:34.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 10 monclient(hunting): authenticate will time out at 1453.502825s 2024-01-24T23:29:34.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7affd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f6e7c09e740 0x7f6e7c1d0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39190/0 (socket says 172.21.15.115:39190) 2024-01-24T23:29:34.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e7affd640 1 -- 172.21.15.115:0/4181565264 learned_addr learned my addr 172.21.15.115:0/4181565264 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:34.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 1 -- 172.21.15.115:0/4181565264 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3729640183 0 0) 0x7f6e7c0722e0 con 0x7f6e7c1ba110 2024-01-24T23:29:34.593 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient(hunting): my global_id is 95296 2024-01-24T23:29:34.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 1 -- 172.21.15.115:0/4181565264 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e68003530 con 0x7f6e7c1ba110 2024-01-24T23:29:34.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 1 -- 172.21.15.115:0/4181565264 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2578445063 0 0) 0x7f6e7c09f640 con 0x7f6e7c09e740 2024-01-24T23:29:34.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient(hunting): my global_id is 154206 2024-01-24T23:29:34.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 1 -- 172.21.15.115:0/4181565264 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e7c0722e0 con 0x7f6e7c09e740 2024-01-24T23:29:34.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 1 -- 172.21.15.115:0/4181565264 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1917129689 0 0) 0x7f6e68003530 con 0x7f6e7c1ba110 2024-01-24T23:29:34.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 1 -- 172.21.15.115:0/4181565264 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6e7c09f640 con 0x7f6e7c1ba110 2024-01-24T23:29:34.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 1 -- 172.21.15.115:0/4181565264 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6e700013c0 con 0x7f6e7c1ba110 2024-01-24T23:29:34.594 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:29:34.595 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient(hunting): dump: 2024-01-24T23:29:34.595 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.595 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.595 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.595 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.595 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.595 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.595 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.595 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.595 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.595 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.595 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.595 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.595 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.596 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.596 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.596 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:34.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 1 -- 172.21.15.115:0/4181565264 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3772780081 0 0) 0x7f6e7c0722e0 con 0x7f6e7c09e740 2024-01-24T23:29:34.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 1 -- 172.21.15.115:0/4181565264 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6e68003530 con 0x7f6e7c09e740 2024-01-24T23:29:34.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 1 -- 172.21.15.115:0/4181565264 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6e6c001dc0 con 0x7f6e7c09e740 2024-01-24T23:29:34.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:29:34.596 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient(hunting): dump: 2024-01-24T23:29:34.596 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.596 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.596 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.596 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.597 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.597 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.597 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.597 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.597 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.597 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.597 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.597 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.597 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.597 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.597 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.597 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 1 -- 172.21.15.115:0/4181565264 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3707713440 0 0) 0x7f6e7c06c230 con 0x7f6e7c071e90 2024-01-24T23:29:34.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:34.597 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:34.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:34.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient(hunting): my global_id is 124696 2024-01-24T23:29:34.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 1 -- 172.21.15.115:0/4181565264 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e7c0722e0 con 0x7f6e7c071e90 2024-01-24T23:29:34.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 1 -- 172.21.15.115:0/4181565264 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 23763824 0 0) 0x7f6e7c09f640 con 0x7f6e7c1ba110 2024-01-24T23:29:34.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 1 -- 172.21.15.115:0/4181565264 >> v1:172.21.15.132:6789/0 conn(0x7f6e7c071e90 legacy=0x7f6e7c1b9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 1 -- 172.21.15.115:0/4181565264 >> v1:172.21.15.115:6789/0 conn(0x7f6e7c09e740 legacy=0x7f6e7c1d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient: _finish_hunting 0 2024-01-24T23:29:34.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 1 monclient: found mon.c 2024-01-24T23:29:34.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:34.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:29:34.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 1 -- 172.21.15.115:0/4181565264 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c1a8b40 con 0x7f6e7c1ba110 2024-01-24T23:29:34.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient: _finish_auth 0 2024-01-24T23:29:34.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient: _check_auth_tickets 2024-01-24T23:29:34.598 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:34.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 5 monclient: authenticate success, global_id 95296 2024-01-24T23:29:34.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 10 monclient: _renew_subs 2024-01-24T23:29:34.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:29:34.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 1 -- 172.21.15.115:0/4181565264 --> v1:172.21.15.115:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7c1c8600 con 0x7f6e7c1ba110 2024-01-24T23:29:34.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 10 monclient: _renew_subs 2024-01-24T23:29:34.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:29:34.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e81f26640 1 -- 172.21.15.115:0/4181565264 --> v1:172.21.15.115:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7c1a7ff0 con 0x7f6e7c1ba110 2024-01-24T23:29:34.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 1 -- 172.21.15.115:0/4181565264 <== mon.4 v1:172.21.15.115:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6e70001710 con 0x7f6e7c1ba110 2024-01-24T23:29:34.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:34.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e79ffb640 4 set_mon_vals no callback set 2024-01-24T23:29:34.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 1 -- 172.21.15.115:0/4181565264 <== mon.4 v1:172.21.15.115:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6e70003bf0 con 0x7f6e7c1ba110 2024-01-24T23:29:34.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:34.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:29:34.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.575+0000 7f6e63fff640 10 monclient: dump: 2024-01-24T23:29:34.599 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:34.600 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:34.600 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.600 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:34.600 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:34.600 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:34.600 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:34.600 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:34.600 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:34.600 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:34.600 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:34.600 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:34.600 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:34.600 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:34.600 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:34.600 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:34.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.579+0000 7f6e81f26640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:34.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.579+0000 7f6e81f26640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:34.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.579+0000 7f6e63fff640 1 -- 172.21.15.115:0/4181565264 <== mon.4 v1:172.21.15.115:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f6e7000e670 con 0x7f6e7c1ba110 2024-01-24T23:29:34.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.579+0000 7f6e81f26640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:29:34.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.579+0000 7f6e81f26640 1 -- 172.21.15.115:0/4181565264 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e5c005090 con 0x7f6e7c1ba110 2024-01-24T23:29:34.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.579+0000 7f6e63fff640 1 -- 172.21.15.115:0/4181565264 <== mon.4 v1:172.21.15.115:6791/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7f6e70043240 con 0x7f6e7c1ba110 2024-01-24T23:29:34.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.579+0000 7f6e7b7fe640 10 monclient: get_auth_request con 0x7f6e68037890 auth_method 0 2024-01-24T23:29:34.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.583+0000 7f6e63fff640 1 -- 172.21.15.115:0/4181565264 <== mon.4 v1:172.21.15.115:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f6e70013620 con 0x7f6e7c1ba110 2024-01-24T23:29:34.601 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.583+0000 7f6e63fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:34.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.583+0000 7f6e63fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:34.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.787+0000 7f6e81f26640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-01-24T23:29:34.791 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.787+0000 7f6e81f26640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-01-24T23:29:34.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.787+0000 7f6e81f26640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.115:6791/0 2024-01-24T23:29:34.792 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.787+0000 7f6e81f26640 1 -- 172.21.15.115:0/4181565264 --> v1:172.21.15.115:6791/0 -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f6e5c005b50 con 0x7f6e7c1ba110 2024-01-24T23:29:34.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.795+0000 7f6e63fff640 1 -- 172.21.15.115:0/4181565264 <== mon.4 v1:172.21.15.115:6791/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (unknown 246732539 0 0) 0x7f6e70011cc0 con 0x7f6e7c1ba110 2024-01-24T23:29:34.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.795+0000 7f6e63fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-01-24T23:29:34.799 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.795+0000 7f6e63fff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:34.801 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.797+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.797+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=26,config=2,health=10,kv=9,logm=53} crc {auth=113272874,config=1618278320,health=1528423856,kv=3733386309,logm=1154631693}) 2024-01-24T23:29:34.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.797+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=11,kv=9,logm=50} crc {auth=317302932,config=1618278320,health=466628169,kv=3447624886,logm=1673843791}) 2024-01-24T23:29:34.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.797+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.797+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=26,config=2,health=10,kv=9,logm=53} crc {auth=113272874,config=1618278320,health=1528423856,kv=3733386309,logm=1154631693}) 2024-01-24T23:29:34.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.797+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=26,config=2,health=10,kv=9,logm=53} crc {auth=2559382199,config=1618278320,health=725850947,kv=3447624886,logm=1085220508}) 2024-01-24T23:29:34.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.797+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.797+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=26,config=2,health=10,kv=9,logm=53} crc {auth=113272874,config=1618278320,health=1528423856,kv=3733386309,logm=1154631693}) 2024-01-24T23:29:34.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.797+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=26,config=2,health=12,kv=8,logm=52} crc {auth=3012703200,config=1618278320,health=3488661123,kv=3527524503,logm=1245911319}) 2024-01-24T23:29:34.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.797+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.797+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=26,config=2,health=10,kv=9,logm=53} crc {auth=113272874,config=1618278320,health=1528423856,kv=3733386309,logm=1154631693}) 2024-01-24T23:29:34.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.797+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=25,config=1,health=11,kv=9,logm=54} crc {auth=1229885790,config=2457401934,health=2717531994,kv=3447624886,logm=3679227149}) 2024-01-24T23:29:34.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.797+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.797+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=26,config=2,health=10,kv=9,logm=53} crc {auth=113272874,config=1618278320,health=1528423856,kv=3733386309,logm=1154631693}) 2024-01-24T23:29:34.802 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.797+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=26,config=2,health=11,kv=8,logm=53} crc {auth=742223978,config=1618278320,health=1713972557,kv=4100078458,logm=1729105190}) 2024-01-24T23:29:34.803 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.797+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.803 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.797+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=26,config=2,health=10,kv=9,logm=53} crc {auth=113272874,config=1618278320,health=1528423856,kv=3733386309,logm=1154631693}) 2024-01-24T23:29:34.803 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.797+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=28,config=2,health=11,kv=8,logm=51} crc {auth=317302932,config=1618278320,health=1198002227,kv=3618020637,logm=3826012005}) 2024-01-24T23:29:34.803 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.797+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.803 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.797+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=26,config=2,health=10,kv=9,logm=53} crc {auth=113272874,config=1618278320,health=1528423856,kv=3733386309,logm=1154631693}) 2024-01-24T23:29:34.803 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.797+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=27,config=2,health=11,kv=9,logm=51} crc {auth=4252821954,config=1618278320,health=466628169,kv=3447624886,logm=1230143656}) 2024-01-24T23:29:34.803 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.797+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.803 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.797+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=26,config=2,health=10,kv=9,logm=53} crc {auth=113272874,config=1618278320,health=1528423856,kv=3733386309,logm=1154631693}) 2024-01-24T23:29:34.803 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.797+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=28,config=2,health=11,kv=9,logm=50} crc {auth=317302932,config=1618278320,health=651691243,kv=3447624886,logm=2346107877}) 2024-01-24T23:29:34.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.799+0000 7f6e81f26640 1 -- 172.21.15.115:0/4181565264 >> v1:172.21.15.132:6812/1835401922 conn(0x7f6e68037890 legacy=0x7f6e68039d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.799+0000 7f6e81f26640 10 monclient: shutdown 2024-01-24T23:29:34.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.799+0000 7f6e81f26640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:34.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.799+0000 7f6e81f26640 1 -- 172.21.15.115:0/4181565264 >> v1:172.21.15.115:6791/0 conn(0x7f6e7c1ba110 legacy=0x7f6e7c1d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:34.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.799+0000 7f6e81f26640 1 -- 172.21.15.115:0/4181565264 shutdown_connections 2024-01-24T23:29:34.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.799+0000 7f6e81f26640 1 -- 172.21.15.115:0/4181565264 >> 172.21.15.115:0/4181565264 conn(0x7f6e7c0666f0 msgr2=0x7f6e7c0a6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:34.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.799+0000 7f6e81f26640 1 -- 172.21.15.115:0/4181565264 shutdown_connections 2024-01-24T23:29:34.804 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:34.799+0000 7f6e81f26640 1 -- 172.21.15.115:0/4181565264 wait complete. 2024-01-24T23:29:34.806 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.805+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.806 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.805+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4066938440}) 2024-01-24T23:29:34.806 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.805+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1624641323}) 2024-01-24T23:29:34.806 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.805+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.806 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.805+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4066938440}) 2024-01-24T23:29:34.806 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.805+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=638759078}) 2024-01-24T23:29:34.806 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.805+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.806 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.805+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4066938440}) 2024-01-24T23:29:34.807 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.805+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2253013756}) 2024-01-24T23:29:34.807 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.805+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.807 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.805+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4066938440}) 2024-01-24T23:29:34.807 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.805+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3490880351}) 2024-01-24T23:29:34.807 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.805+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.807 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.805+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4066938440}) 2024-01-24T23:29:34.807 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.805+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=733271903}) 2024-01-24T23:29:34.807 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.805+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.807 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.805+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4066938440}) 2024-01-24T23:29:34.807 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.805+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=731586699}) 2024-01-24T23:29:34.807 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.805+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.807 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.805+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4066938440}) 2024-01-24T23:29:34.807 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.805+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2118424739}) 2024-01-24T23:29:34.807 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.805+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.807 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.805+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4066938440}) 2024-01-24T23:29:34.808 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.805+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2678580860}) 2024-01-24T23:29:34.810 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.809+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.810 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.809+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2557386234}) 2024-01-24T23:29:34.810 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.809+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1957849829}) 2024-01-24T23:29:34.810 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.809+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.810 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.809+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2557386234}) 2024-01-24T23:29:34.811 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.809+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2768111235}) 2024-01-24T23:29:34.811 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.809+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.811 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.809+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2557386234}) 2024-01-24T23:29:34.811 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.809+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3640066788}) 2024-01-24T23:29:34.811 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.809+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.811 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.809+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2557386234}) 2024-01-24T23:29:34.811 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.809+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1918954481}) 2024-01-24T23:29:34.811 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.809+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.811 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.809+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2557386234}) 2024-01-24T23:29:34.811 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.809+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1842940631}) 2024-01-24T23:29:34.811 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.809+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.811 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.809+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2557386234}) 2024-01-24T23:29:34.811 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.809+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2154095997}) 2024-01-24T23:29:34.812 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.809+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.812 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.809+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2557386234}) 2024-01-24T23:29:34.812 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.809+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2308805636}) 2024-01-24T23:29:34.812 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.809+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.812 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.809+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2557386234}) 2024-01-24T23:29:34.812 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.809+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=799384417}) 2024-01-24T23:29:34.813 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.813+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.814 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.813+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1583479932}) 2024-01-24T23:29:34.814 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.813+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=990203307}) 2024-01-24T23:29:34.814 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.813+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.814 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.813+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1583479932}) 2024-01-24T23:29:34.814 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.813+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2799469130}) 2024-01-24T23:29:34.814 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.813+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.814 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.813+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1583479932}) 2024-01-24T23:29:34.814 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.813+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=218850432}) 2024-01-24T23:29:34.814 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.813+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.814 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.813+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1583479932}) 2024-01-24T23:29:34.814 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.813+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2457665214}) 2024-01-24T23:29:34.814 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.813+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.814 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.813+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1583479932}) 2024-01-24T23:29:34.814 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.813+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3948687337}) 2024-01-24T23:29:34.814 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.813+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.815 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.813+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1583479932}) 2024-01-24T23:29:34.815 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.813+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2992896216}) 2024-01-24T23:29:34.815 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.813+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.815 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.813+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1583479932}) 2024-01-24T23:29:34.815 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.813+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2995183077}) 2024-01-24T23:29:34.815 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.813+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.815 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.813+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1583479932}) 2024-01-24T23:29:34.815 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.813+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2076328722}) 2024-01-24T23:29:34.817 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.817 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2951368234}) 2024-01-24T23:29:34.817 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3464388557}) 2024-01-24T23:29:34.817 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.818 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2951368234}) 2024-01-24T23:29:34.818 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=618865087}) 2024-01-24T23:29:34.818 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.818 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2951368234}) 2024-01-24T23:29:34.818 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3770624012}) 2024-01-24T23:29:34.818 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.818 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2951368234}) 2024-01-24T23:29:34.818 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2642263156}) 2024-01-24T23:29:34.818 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.818 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2951368234}) 2024-01-24T23:29:34.818 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2555177553}) 2024-01-24T23:29:34.818 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.818 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2951368234}) 2024-01-24T23:29:34.818 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1279981835}) 2024-01-24T23:29:34.819 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.819 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2951368234}) 2024-01-24T23:29:34.819 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=264934189}) 2024-01-24T23:29:34.819 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.819 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2951368234}) 2024-01-24T23:29:34.819 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=20017620}) 2024-01-24T23:29:34.820 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-01-24T23:29:34.821 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.821 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=844905192}) 2024-01-24T23:29:34.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2365087206}) 2024-01-24T23:29:34.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=844905192}) 2024-01-24T23:29:34.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3449230392}) 2024-01-24T23:29:34.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=844905192}) 2024-01-24T23:29:34.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2490414195}) 2024-01-24T23:29:34.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=844905192}) 2024-01-24T23:29:34.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1510786308}) 2024-01-24T23:29:34.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.817+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=844905192}) 2024-01-24T23:29:34.822 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2814230082}) 2024-01-24T23:29:34.823 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.823 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=844905192}) 2024-01-24T23:29:34.823 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2741385981}) 2024-01-24T23:29:34.823 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.823 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=844905192}) 2024-01-24T23:29:34.823 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2246892482}) 2024-01-24T23:29:34.823 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.823 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=844905192}) 2024-01-24T23:29:34.823 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3096974262}) 2024-01-24T23:29:34.824 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.824 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3573985217}) 2024-01-24T23:29:34.824 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1974626979}) 2024-01-24T23:29:34.825 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.825 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3573985217}) 2024-01-24T23:29:34.825 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3501568179}) 2024-01-24T23:29:34.825 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.825 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3573985217}) 2024-01-24T23:29:34.825 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3607777383}) 2024-01-24T23:29:34.825 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.825 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3573985217}) 2024-01-24T23:29:34.825 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4150482780}) 2024-01-24T23:29:34.825 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.825 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3573985217}) 2024-01-24T23:29:34.825 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3245116574}) 2024-01-24T23:29:34.825 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.825 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3573985217}) 2024-01-24T23:29:34.826 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2374212598}) 2024-01-24T23:29:34.826 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.826 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3573985217}) 2024-01-24T23:29:34.826 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1942653504}) 2024-01-24T23:29:34.826 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.826 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3573985217}) 2024-01-24T23:29:34.826 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.821+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3868408002}) 2024-01-24T23:29:34.827 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.825+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.828 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.825+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3915264001}) 2024-01-24T23:29:34.828 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.825+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1419651990}) 2024-01-24T23:29:34.828 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.825+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.828 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.825+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3915264001}) 2024-01-24T23:29:34.828 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.825+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3696879778}) 2024-01-24T23:29:34.828 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.825+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.828 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.825+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3915264001}) 2024-01-24T23:29:34.828 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.825+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2075840597}) 2024-01-24T23:29:34.828 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.825+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.828 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.825+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3915264001}) 2024-01-24T23:29:34.828 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.825+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1898007469}) 2024-01-24T23:29:34.828 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.825+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.828 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.825+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3915264001}) 2024-01-24T23:29:34.829 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.825+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=24553462}) 2024-01-24T23:29:34.829 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.825+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.829 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.825+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3915264001}) 2024-01-24T23:29:34.829 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.825+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=772169225}) 2024-01-24T23:29:34.829 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.825+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.829 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.825+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3915264001}) 2024-01-24T23:29:34.829 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.825+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2468188892}) 2024-01-24T23:29:34.829 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.825+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.829 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.825+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3915264001}) 2024-01-24T23:29:34.829 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.825+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1501666329}) 2024-01-24T23:29:34.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.829+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.829+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=939329194}) 2024-01-24T23:29:34.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.829+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3910573742}) 2024-01-24T23:29:34.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.829+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.829+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=939329194}) 2024-01-24T23:29:34.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.829+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3333014107}) 2024-01-24T23:29:34.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.829+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.829+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=939329194}) 2024-01-24T23:29:34.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.829+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2252862038}) 2024-01-24T23:29:34.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.829+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.831 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.829+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=939329194}) 2024-01-24T23:29:34.832 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.829+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1087067539}) 2024-01-24T23:29:34.832 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.829+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.832 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.829+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=939329194}) 2024-01-24T23:29:34.832 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.829+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3070787182}) 2024-01-24T23:29:34.832 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.829+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.832 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.829+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=939329194}) 2024-01-24T23:29:34.832 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.829+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=214388625}) 2024-01-24T23:29:34.832 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.829+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.832 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.829+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=939329194}) 2024-01-24T23:29:34.832 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.829+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=937261328}) 2024-01-24T23:29:34.832 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.829+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.832 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.829+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=939329194}) 2024-01-24T23:29:34.832 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.829+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=253568017}) 2024-01-24T23:29:34.835 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.833+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.835 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.833+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3071993032}) 2024-01-24T23:29:34.835 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.833+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=380139473}) 2024-01-24T23:29:34.835 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.833+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.836 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.833+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3071993032}) 2024-01-24T23:29:34.836 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.833+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3284510107}) 2024-01-24T23:29:34.836 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.833+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.836 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.833+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3071993032}) 2024-01-24T23:29:34.836 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.833+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2439979203}) 2024-01-24T23:29:34.836 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.833+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.836 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.833+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3071993032}) 2024-01-24T23:29:34.836 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.833+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3266654261}) 2024-01-24T23:29:34.836 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.833+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.837 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.833+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3071993032}) 2024-01-24T23:29:34.837 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.833+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3918485378}) 2024-01-24T23:29:34.837 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.833+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.837 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.833+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3071993032}) 2024-01-24T23:29:34.837 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.833+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2745808956}) 2024-01-24T23:29:34.837 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.833+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.837 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.833+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3071993032}) 2024-01-24T23:29:34.837 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.833+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1069443933}) 2024-01-24T23:29:34.837 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.833+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.837 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.833+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3071993032}) 2024-01-24T23:29:34.837 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.833+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=896416435}) 2024-01-24T23:29:34.841 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.841 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1145961122}) 2024-01-24T23:29:34.841 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3001347838}) 2024-01-24T23:29:34.841 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.841 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1145961122}) 2024-01-24T23:29:34.841 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3940619131}) 2024-01-24T23:29:34.841 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.841 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1145961122}) 2024-01-24T23:29:34.841 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=177566442}) 2024-01-24T23:29:34.841 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.841 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1145961122}) 2024-01-24T23:29:34.841 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1430973752}) 2024-01-24T23:29:34.842 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.842 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1145961122}) 2024-01-24T23:29:34.842 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3766280834}) 2024-01-24T23:29:34.842 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.842 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1145961122}) 2024-01-24T23:29:34.842 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=355246823}) 2024-01-24T23:29:34.842 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.842 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1145961122}) 2024-01-24T23:29:34.842 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1961980303}) 2024-01-24T23:29:34.842 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.842 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1145961122}) 2024-01-24T23:29:34.842 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=727296982}) 2024-01-24T23:29:34.842 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.842 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3480157880}) 2024-01-24T23:29:34.842 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=258662518}) 2024-01-24T23:29:34.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3480157880}) 2024-01-24T23:29:34.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3187530183}) 2024-01-24T23:29:34.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3480157880}) 2024-01-24T23:29:34.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3937630858}) 2024-01-24T23:29:34.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3480157880}) 2024-01-24T23:29:34.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.837+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=706743444}) 2024-01-24T23:29:34.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3480157880}) 2024-01-24T23:29:34.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2169736792}) 2024-01-24T23:29:34.843 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.844 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3480157880}) 2024-01-24T23:29:34.844 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3530470206}) 2024-01-24T23:29:34.844 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.844 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3480157880}) 2024-01-24T23:29:34.844 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2209831449}) 2024-01-24T23:29:34.844 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.844 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3480157880}) 2024-01-24T23:29:34.844 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3787958334}) 2024-01-24T23:29:34.845 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.845 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2730901100}) 2024-01-24T23:29:34.845 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2644202492}) 2024-01-24T23:29:34.845 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.846 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2730901100}) 2024-01-24T23:29:34.846 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3672359889}) 2024-01-24T23:29:34.846 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.846 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2730901100}) 2024-01-24T23:29:34.846 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1959366778}) 2024-01-24T23:29:34.846 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.846 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2730901100}) 2024-01-24T23:29:34.846 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=766755097}) 2024-01-24T23:29:34.846 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.846 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2730901100}) 2024-01-24T23:29:34.846 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3585853324}) 2024-01-24T23:29:34.847 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.847 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2730901100}) 2024-01-24T23:29:34.847 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1397939998}) 2024-01-24T23:29:34.847 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.847 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2730901100}) 2024-01-24T23:29:34.847 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3956931823}) 2024-01-24T23:29:34.847 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.847 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2730901100}) 2024-01-24T23:29:34.847 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.841+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=232721414}) 2024-01-24T23:29:34.851 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.845+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.851 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.845+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=943915454}) 2024-01-24T23:29:34.851 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.845+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4016480374}) 2024-01-24T23:29:34.852 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.845+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.852 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.845+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=943915454}) 2024-01-24T23:29:34.852 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.845+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1377130160}) 2024-01-24T23:29:34.852 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.845+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.852 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.845+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=943915454}) 2024-01-24T23:29:34.852 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.845+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3325858862}) 2024-01-24T23:29:34.852 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.845+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.852 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.845+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=943915454}) 2024-01-24T23:29:34.852 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.845+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2193462184}) 2024-01-24T23:29:34.852 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.845+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.852 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.845+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=943915454}) 2024-01-24T23:29:34.852 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.845+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1050130239}) 2024-01-24T23:29:34.852 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.845+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.852 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.845+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=943915454}) 2024-01-24T23:29:34.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.845+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=393243149}) 2024-01-24T23:29:34.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.845+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.845+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=943915454}) 2024-01-24T23:29:34.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.845+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3301683856}) 2024-01-24T23:29:34.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.845+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.845+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=943915454}) 2024-01-24T23:29:34.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.845+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3090158825}) 2024-01-24T23:29:34.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.849+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.849+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2848372604}) 2024-01-24T23:29:34.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.849+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3941368746}) 2024-01-24T23:29:34.853 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.849+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.854 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.849+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2848372604}) 2024-01-24T23:29:34.854 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.849+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3744582322}) 2024-01-24T23:29:34.854 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.849+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.854 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.849+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2848372604}) 2024-01-24T23:29:34.854 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.849+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2750499255}) 2024-01-24T23:29:34.854 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.849+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.854 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.849+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2848372604}) 2024-01-24T23:29:34.854 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.849+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2356165357}) 2024-01-24T23:29:34.854 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.849+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.854 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.849+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2848372604}) 2024-01-24T23:29:34.854 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.849+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2619936430}) 2024-01-24T23:29:34.854 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.849+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.854 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.849+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2848372604}) 2024-01-24T23:29:34.854 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.849+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1014374856}) 2024-01-24T23:29:34.855 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.849+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.855 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.849+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2848372604}) 2024-01-24T23:29:34.855 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.849+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4160346385}) 2024-01-24T23:29:34.855 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.849+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.855 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.849+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2848372604}) 2024-01-24T23:29:34.855 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.849+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3677761578}) 2024-01-24T23:29:34.858 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.853+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.858 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.853+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3894531748}) 2024-01-24T23:29:34.858 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.853+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=59320912}) 2024-01-24T23:29:34.858 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.853+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.858 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.853+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3894531748}) 2024-01-24T23:29:34.858 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.853+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3762570727}) 2024-01-24T23:29:34.858 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.853+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.858 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.853+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3894531748}) 2024-01-24T23:29:34.858 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.853+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1278285231}) 2024-01-24T23:29:34.858 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.853+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.858 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.853+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3894531748}) 2024-01-24T23:29:34.858 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.853+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2348814431}) 2024-01-24T23:29:34.859 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.853+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.859 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.853+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3894531748}) 2024-01-24T23:29:34.859 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.853+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2349154310}) 2024-01-24T23:29:34.859 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.853+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.859 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.853+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3894531748}) 2024-01-24T23:29:34.859 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.853+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1871561685}) 2024-01-24T23:29:34.859 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.853+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.859 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.853+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3894531748}) 2024-01-24T23:29:34.859 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.853+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1297260429}) 2024-01-24T23:29:34.859 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.853+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.859 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.853+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3894531748}) 2024-01-24T23:29:34.859 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.853+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=345253062}) 2024-01-24T23:29:34.859 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.857+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.859 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.857+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2221977925}) 2024-01-24T23:29:34.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.857+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1725082919}) 2024-01-24T23:29:34.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.857+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.857+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2221977925}) 2024-01-24T23:29:34.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.857+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2095782629}) 2024-01-24T23:29:34.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.857+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.857+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2221977925}) 2024-01-24T23:29:34.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.857+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3692554741}) 2024-01-24T23:29:34.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.857+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.857+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2221977925}) 2024-01-24T23:29:34.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.857+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2138118460}) 2024-01-24T23:29:34.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.857+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.857+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2221977925}) 2024-01-24T23:29:34.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.857+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=617200523}) 2024-01-24T23:29:34.860 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.857+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.861 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.857+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2221977925}) 2024-01-24T23:29:34.861 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.857+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=739610423}) 2024-01-24T23:29:34.861 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.857+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.861 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.857+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2221977925}) 2024-01-24T23:29:34.861 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.857+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2394963999}) 2024-01-24T23:29:34.861 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.857+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.861 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.857+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2221977925}) 2024-01-24T23:29:34.861 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.857+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2034552043}) 2024-01-24T23:29:34.862 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.862 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=62142516}) 2024-01-24T23:29:34.862 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=407010011}) 2024-01-24T23:29:34.862 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.863 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=62142516}) 2024-01-24T23:29:34.863 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2337362981}) 2024-01-24T23:29:34.863 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.863 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=62142516}) 2024-01-24T23:29:34.863 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4187600608}) 2024-01-24T23:29:34.863 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.863 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=62142516}) 2024-01-24T23:29:34.863 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=270113024}) 2024-01-24T23:29:34.863 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.863 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=62142516}) 2024-01-24T23:29:34.863 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1660079783}) 2024-01-24T23:29:34.863 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.864 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=62142516}) 2024-01-24T23:29:34.864 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2947667222}) 2024-01-24T23:29:34.864 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.864 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=62142516}) 2024-01-24T23:29:34.864 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1619996348}) 2024-01-24T23:29:34.864 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.864 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=62142516}) 2024-01-24T23:29:34.864 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3316888598}) 2024-01-24T23:29:34.867 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.867 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1675555141}) 2024-01-24T23:29:34.867 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=585347501}) 2024-01-24T23:29:34.867 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.867 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1675555141}) 2024-01-24T23:29:34.867 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3244304879}) 2024-01-24T23:29:34.867 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.867 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1675555141}) 2024-01-24T23:29:34.867 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3058884772}) 2024-01-24T23:29:34.867 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.868 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1675555141}) 2024-01-24T23:29:34.868 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=223143524}) 2024-01-24T23:29:34.868 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.868 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1675555141}) 2024-01-24T23:29:34.868 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=508153961}) 2024-01-24T23:29:34.868 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.868 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1675555141}) 2024-01-24T23:29:34.868 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2120728709}) 2024-01-24T23:29:34.868 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.868 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1675555141}) 2024-01-24T23:29:34.868 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4292209170}) 2024-01-24T23:29:34.868 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.868 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1675555141}) 2024-01-24T23:29:34.869 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.861+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1217426093}) 2024-01-24T23:29:34.869 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.865+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.869 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.865+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2995486511}) 2024-01-24T23:29:34.869 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.865+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2888481402}) 2024-01-24T23:29:34.869 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.865+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.869 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.865+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2995486511}) 2024-01-24T23:29:34.869 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.865+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4177506473}) 2024-01-24T23:29:34.869 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.865+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.869 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.865+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2995486511}) 2024-01-24T23:29:34.869 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.865+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1078456194}) 2024-01-24T23:29:34.869 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.865+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.869 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.865+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2995486511}) 2024-01-24T23:29:34.869 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.865+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2099999464}) 2024-01-24T23:29:34.869 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.865+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.865+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2995486511}) 2024-01-24T23:29:34.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.865+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=95739000}) 2024-01-24T23:29:34.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.865+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.865+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2995486511}) 2024-01-24T23:29:34.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.865+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2222512997}) 2024-01-24T23:29:34.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.865+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.865+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2995486511}) 2024-01-24T23:29:34.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.865+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1689019268}) 2024-01-24T23:29:34.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.865+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.865+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2995486511}) 2024-01-24T23:29:34.870 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.865+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=197311072}) 2024-01-24T23:29:34.873 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.869+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.873 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.869+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2030787766}) 2024-01-24T23:29:34.873 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.869+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1497877131}) 2024-01-24T23:29:34.873 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.869+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.873 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.869+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2030787766}) 2024-01-24T23:29:34.873 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.869+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3599090950}) 2024-01-24T23:29:34.873 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.869+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.874 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.869+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2030787766}) 2024-01-24T23:29:34.874 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.869+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=254854051}) 2024-01-24T23:29:34.874 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.869+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.874 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.869+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2030787766}) 2024-01-24T23:29:34.874 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.869+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=617646938}) 2024-01-24T23:29:34.874 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.869+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.874 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.869+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2030787766}) 2024-01-24T23:29:34.874 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.869+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2494527231}) 2024-01-24T23:29:34.874 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.869+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.874 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.869+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2030787766}) 2024-01-24T23:29:34.874 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.869+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2393294254}) 2024-01-24T23:29:34.874 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.869+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.874 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.869+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2030787766}) 2024-01-24T23:29:34.875 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.869+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=211572250}) 2024-01-24T23:29:34.875 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.869+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.875 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.869+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2030787766}) 2024-01-24T23:29:34.875 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.869+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=244202864}) 2024-01-24T23:29:34.875 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.875 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1648877065}) 2024-01-24T23:29:34.875 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=692282223}) 2024-01-24T23:29:34.875 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.875 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1648877065}) 2024-01-24T23:29:34.875 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2481677166}) 2024-01-24T23:29:34.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1648877065}) 2024-01-24T23:29:34.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3895993711}) 2024-01-24T23:29:34.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1648877065}) 2024-01-24T23:29:34.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3570868476}) 2024-01-24T23:29:34.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1648877065}) 2024-01-24T23:29:34.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=456576540}) 2024-01-24T23:29:34.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1648877065}) 2024-01-24T23:29:34.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4163764680}) 2024-01-24T23:29:34.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.876 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1648877065}) 2024-01-24T23:29:34.877 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2499373155}) 2024-01-24T23:29:34.877 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.877 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1648877065}) 2024-01-24T23:29:34.877 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1329063951}) 2024-01-24T23:29:34.881 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.881 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1731454427}) 2024-01-24T23:29:34.881 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1340447658}) 2024-01-24T23:29:34.881 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.881 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1731454427}) 2024-01-24T23:29:34.881 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3668753858}) 2024-01-24T23:29:34.881 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1731454427}) 2024-01-24T23:29:34.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1226285300}) 2024-01-24T23:29:34.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1731454427}) 2024-01-24T23:29:34.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1880399946}) 2024-01-24T23:29:34.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1731454427}) 2024-01-24T23:29:34.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.873+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2780197}) 2024-01-24T23:29:34.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1731454427}) 2024-01-24T23:29:34.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1639678635}) 2024-01-24T23:29:34.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1731454427}) 2024-01-24T23:29:34.882 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4270342301}) 2024-01-24T23:29:34.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1731454427}) 2024-01-24T23:29:34.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=996593447}) 2024-01-24T23:29:34.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1782296355}) 2024-01-24T23:29:34.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3192583045}) 2024-01-24T23:29:34.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1782296355}) 2024-01-24T23:29:34.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=400412428}) 2024-01-24T23:29:34.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1782296355}) 2024-01-24T23:29:34.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=814873991}) 2024-01-24T23:29:34.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1782296355}) 2024-01-24T23:29:34.883 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=110349004}) 2024-01-24T23:29:34.884 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.884 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1782296355}) 2024-01-24T23:29:34.884 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3918521809}) 2024-01-24T23:29:34.884 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.884 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1782296355}) 2024-01-24T23:29:34.884 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2845041582}) 2024-01-24T23:29:34.884 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.884 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1782296355}) 2024-01-24T23:29:34.884 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4280891181}) 2024-01-24T23:29:34.884 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.884 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1782296355}) 2024-01-24T23:29:34.884 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.877+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2088276700}) 2024-01-24T23:29:34.888 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.881+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.888 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.881+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3608289076}) 2024-01-24T23:29:34.888 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.881+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2938281917}) 2024-01-24T23:29:34.888 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.881+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.888 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.881+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3608289076}) 2024-01-24T23:29:34.888 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.881+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2072044293}) 2024-01-24T23:29:34.888 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.881+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.888 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.881+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3608289076}) 2024-01-24T23:29:34.888 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.881+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=58382293}) 2024-01-24T23:29:34.888 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.881+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.888 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.881+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3608289076}) 2024-01-24T23:29:34.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.881+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2338960213}) 2024-01-24T23:29:34.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.881+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.881+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3608289076}) 2024-01-24T23:29:34.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.881+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1030606347}) 2024-01-24T23:29:34.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.881+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.881+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3608289076}) 2024-01-24T23:29:34.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.881+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3642616009}) 2024-01-24T23:29:34.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.881+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.881+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3608289076}) 2024-01-24T23:29:34.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.881+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3798762741}) 2024-01-24T23:29:34.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.881+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.881+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3608289076}) 2024-01-24T23:29:34.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.881+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=613899552}) 2024-01-24T23:29:34.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.885+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.889 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.885+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1804360018}) 2024-01-24T23:29:34.890 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.885+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=110092027}) 2024-01-24T23:29:34.890 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.885+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.890 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.885+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1804360018}) 2024-01-24T23:29:34.890 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.885+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3077703866}) 2024-01-24T23:29:34.890 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.885+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.890 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.885+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1804360018}) 2024-01-24T23:29:34.890 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.885+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3397382783}) 2024-01-24T23:29:34.890 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.885+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.890 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.885+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1804360018}) 2024-01-24T23:29:34.890 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.885+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3485274250}) 2024-01-24T23:29:34.890 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.885+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.890 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.885+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1804360018}) 2024-01-24T23:29:34.890 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.885+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=45310290}) 2024-01-24T23:29:34.890 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.885+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.890 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.885+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1804360018}) 2024-01-24T23:29:34.891 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.885+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1687966916}) 2024-01-24T23:29:34.891 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.885+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.891 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.885+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1804360018}) 2024-01-24T23:29:34.891 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.885+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=483755004}) 2024-01-24T23:29:34.891 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.885+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.891 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.885+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1804360018}) 2024-01-24T23:29:34.891 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.885+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1715514452}) 2024-01-24T23:29:34.894 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.889+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.894 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.889+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419346336}) 2024-01-24T23:29:34.894 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.889+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3038933315}) 2024-01-24T23:29:34.895 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.889+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.895 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.889+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419346336}) 2024-01-24T23:29:34.895 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.889+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=220114360}) 2024-01-24T23:29:34.895 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.889+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.895 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.889+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419346336}) 2024-01-24T23:29:34.895 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.889+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1261830636}) 2024-01-24T23:29:34.895 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.889+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.895 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.889+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419346336}) 2024-01-24T23:29:34.895 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.889+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1547763658}) 2024-01-24T23:29:34.895 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.889+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.895 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.889+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419346336}) 2024-01-24T23:29:34.895 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.889+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2542299968}) 2024-01-24T23:29:34.895 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.889+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.895 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.889+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419346336}) 2024-01-24T23:29:34.896 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.889+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1363273299}) 2024-01-24T23:29:34.896 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.889+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.896 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.889+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419346336}) 2024-01-24T23:29:34.896 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.889+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=903509293}) 2024-01-24T23:29:34.896 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.889+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.896 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.889+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419346336}) 2024-01-24T23:29:34.896 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.889+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2118472610}) 2024-01-24T23:29:34.896 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.896 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2967269937}) 2024-01-24T23:29:34.896 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1394367739}) 2024-01-24T23:29:34.896 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.896 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2967269937}) 2024-01-24T23:29:34.896 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=737716768}) 2024-01-24T23:29:34.896 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.897 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2967269937}) 2024-01-24T23:29:34.897 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2099141743}) 2024-01-24T23:29:34.897 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.897 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2967269937}) 2024-01-24T23:29:34.897 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3955122054}) 2024-01-24T23:29:34.897 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.897 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2967269937}) 2024-01-24T23:29:34.897 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1637374938}) 2024-01-24T23:29:34.897 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.897 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2967269937}) 2024-01-24T23:29:34.897 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=290327287}) 2024-01-24T23:29:34.897 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.897 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2967269937}) 2024-01-24T23:29:34.897 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2110903004}) 2024-01-24T23:29:34.898 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.898 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2967269937}) 2024-01-24T23:29:34.898 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1610456572}) 2024-01-24T23:29:34.901 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.901 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1592082067}) 2024-01-24T23:29:34.901 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2402945453}) 2024-01-24T23:29:34.901 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.901 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1592082067}) 2024-01-24T23:29:34.901 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.893+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1276716102}) 2024-01-24T23:29:34.901 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.902 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1592082067}) 2024-01-24T23:29:34.902 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3021620023}) 2024-01-24T23:29:34.902 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.902 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1592082067}) 2024-01-24T23:29:34.902 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2362248891}) 2024-01-24T23:29:34.902 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.902 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1592082067}) 2024-01-24T23:29:34.902 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4028202688}) 2024-01-24T23:29:34.902 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.902 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1592082067}) 2024-01-24T23:29:34.903 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=265912931}) 2024-01-24T23:29:34.903 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.903 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1592082067}) 2024-01-24T23:29:34.903 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1430533736}) 2024-01-24T23:29:34.903 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.903 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1592082067}) 2024-01-24T23:29:34.903 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2297556687}) 2024-01-24T23:29:34.903 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.903 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4063060186}) 2024-01-24T23:29:34.903 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2543042813}) 2024-01-24T23:29:34.903 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.903 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4063060186}) 2024-01-24T23:29:34.903 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4001709812}) 2024-01-24T23:29:34.903 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4063060186}) 2024-01-24T23:29:34.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2258584642}) 2024-01-24T23:29:34.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4063060186}) 2024-01-24T23:29:34.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3524959165}) 2024-01-24T23:29:34.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4063060186}) 2024-01-24T23:29:34.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4133693809}) 2024-01-24T23:29:34.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4063060186}) 2024-01-24T23:29:34.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=888791354}) 2024-01-24T23:29:34.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.904 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4063060186}) 2024-01-24T23:29:34.905 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=133283771}) 2024-01-24T23:29:34.905 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.905 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4063060186}) 2024-01-24T23:29:34.905 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.897+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2927465052}) 2024-01-24T23:29:34.911 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.901+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.901+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1796614800}) 2024-01-24T23:29:34.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.901+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2596468595}) 2024-01-24T23:29:34.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.901+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.901+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1796614800}) 2024-01-24T23:29:34.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.901+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2394325976}) 2024-01-24T23:29:34.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.901+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.901+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1796614800}) 2024-01-24T23:29:34.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.901+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=346273228}) 2024-01-24T23:29:34.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.901+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.901+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1796614800}) 2024-01-24T23:29:34.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.901+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2570828048}) 2024-01-24T23:29:34.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.901+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.912 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.901+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1796614800}) 2024-01-24T23:29:34.913 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.901+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2926055920}) 2024-01-24T23:29:34.913 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.901+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.913 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.901+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1796614800}) 2024-01-24T23:29:34.913 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.901+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4181592807}) 2024-01-24T23:29:34.913 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.901+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.913 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.901+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1796614800}) 2024-01-24T23:29:34.913 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.901+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3063512749}) 2024-01-24T23:29:34.913 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.901+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.913 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.901+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1796614800}) 2024-01-24T23:29:34.913 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.901+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2387435397}) 2024-01-24T23:29:34.913 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.905+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.913 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.905+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1380323526}) 2024-01-24T23:29:34.913 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.905+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1949585168}) 2024-01-24T23:29:34.913 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.905+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.913 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.905+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1380323526}) 2024-01-24T23:29:34.914 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.905+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3782570474}) 2024-01-24T23:29:34.914 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.905+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.914 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.905+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1380323526}) 2024-01-24T23:29:34.914 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.905+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3342994382}) 2024-01-24T23:29:34.914 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.905+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.914 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.905+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1380323526}) 2024-01-24T23:29:34.914 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.905+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3192346616}) 2024-01-24T23:29:34.914 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.905+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.914 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.905+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1380323526}) 2024-01-24T23:29:34.914 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.905+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2303484606}) 2024-01-24T23:29:34.914 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.905+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.914 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.905+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1380323526}) 2024-01-24T23:29:34.914 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.905+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3404810936}) 2024-01-24T23:29:34.914 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.905+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.914 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.905+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1380323526}) 2024-01-24T23:29:34.914 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.905+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2014568626}) 2024-01-24T23:29:34.915 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.905+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.915 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.905+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1380323526}) 2024-01-24T23:29:34.915 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.905+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1620348816}) 2024-01-24T23:29:34.915 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.909+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.915 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.909+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3259646665}) 2024-01-24T23:29:34.915 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.909+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=782347039}) 2024-01-24T23:29:34.915 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.909+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.915 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.909+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3259646665}) 2024-01-24T23:29:34.915 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.909+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2126363346}) 2024-01-24T23:29:34.915 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.909+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.915 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.909+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3259646665}) 2024-01-24T23:29:34.915 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.909+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2785976815}) 2024-01-24T23:29:34.915 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.909+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.915 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.909+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3259646665}) 2024-01-24T23:29:34.916 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.909+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1875687049}) 2024-01-24T23:29:34.916 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.909+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.916 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.909+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3259646665}) 2024-01-24T23:29:34.916 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.909+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1283250340}) 2024-01-24T23:29:34.916 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.909+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.916 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.909+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3259646665}) 2024-01-24T23:29:34.916 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.909+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4187205225}) 2024-01-24T23:29:34.916 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.909+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.916 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.909+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3259646665}) 2024-01-24T23:29:34.916 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.909+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1742634492}) 2024-01-24T23:29:34.916 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.909+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.916 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.909+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3259646665}) 2024-01-24T23:29:34.916 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.909+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2269539145}) 2024-01-24T23:29:34.922 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.923 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=60218461}) 2024-01-24T23:29:34.923 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3632490944}) 2024-01-24T23:29:34.923 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.923 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=60218461}) 2024-01-24T23:29:34.923 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2347612166}) 2024-01-24T23:29:34.923 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.923 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=60218461}) 2024-01-24T23:29:34.923 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1197358305}) 2024-01-24T23:29:34.923 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.923 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=60218461}) 2024-01-24T23:29:34.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1595429905}) 2024-01-24T23:29:34.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=60218461}) 2024-01-24T23:29:34.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1758420009}) 2024-01-24T23:29:34.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=60218461}) 2024-01-24T23:29:34.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2894003105}) 2024-01-24T23:29:34.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=60218461}) 2024-01-24T23:29:34.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1318962286}) 2024-01-24T23:29:34.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=60218461}) 2024-01-24T23:29:34.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=746996569}) 2024-01-24T23:29:34.924 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.925 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3346248347}) 2024-01-24T23:29:34.925 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4105905595}) 2024-01-24T23:29:34.925 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.925 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3346248347}) 2024-01-24T23:29:34.925 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3527682018}) 2024-01-24T23:29:34.925 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.925 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3346248347}) 2024-01-24T23:29:34.925 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=816970235}) 2024-01-24T23:29:34.925 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.925 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3346248347}) 2024-01-24T23:29:34.925 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3133886682}) 2024-01-24T23:29:34.925 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.926 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3346248347}) 2024-01-24T23:29:34.926 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=430710578}) 2024-01-24T23:29:34.926 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.926 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3346248347}) 2024-01-24T23:29:34.926 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=663108452}) 2024-01-24T23:29:34.926 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.926 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3346248347}) 2024-01-24T23:29:34.926 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1355937665}) 2024-01-24T23:29:34.926 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.926 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3346248347}) 2024-01-24T23:29:34.926 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.913+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3532752941}) 2024-01-24T23:29:34.926 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.917+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.927 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.917+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2566427970}) 2024-01-24T23:29:34.927 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.917+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1052560759}) 2024-01-24T23:29:34.927 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.917+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.927 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.917+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2566427970}) 2024-01-24T23:29:34.927 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.917+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1846175462}) 2024-01-24T23:29:34.927 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.917+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.927 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.917+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2566427970}) 2024-01-24T23:29:34.927 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.917+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2882533048}) 2024-01-24T23:29:34.927 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.917+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.927 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.917+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2566427970}) 2024-01-24T23:29:34.927 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.917+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2527360785}) 2024-01-24T23:29:34.927 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.917+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.927 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.917+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2566427970}) 2024-01-24T23:29:34.927 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.917+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1121698603}) 2024-01-24T23:29:34.928 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.917+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.928 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.917+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2566427970}) 2024-01-24T23:29:34.928 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.917+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1366628776}) 2024-01-24T23:29:34.928 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.917+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.928 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.917+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2566427970}) 2024-01-24T23:29:34.928 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.917+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1946471826}) 2024-01-24T23:29:34.928 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.917+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.928 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.917+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2566427970}) 2024-01-24T23:29:34.928 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.917+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3876593425}) 2024-01-24T23:29:34.928 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.928 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4225699830}) 2024-01-24T23:29:34.928 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1215012627}) 2024-01-24T23:29:34.928 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.929 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4225699830}) 2024-01-24T23:29:34.929 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2949782258}) 2024-01-24T23:29:34.929 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.929 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4225699830}) 2024-01-24T23:29:34.929 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=746385460}) 2024-01-24T23:29:34.929 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.929 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4225699830}) 2024-01-24T23:29:34.929 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1563928241}) 2024-01-24T23:29:34.929 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.929 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4225699830}) 2024-01-24T23:29:34.929 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1195896539}) 2024-01-24T23:29:34.929 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.929 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4225699830}) 2024-01-24T23:29:34.929 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1901321432}) 2024-01-24T23:29:34.930 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.930 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4225699830}) 2024-01-24T23:29:34.930 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2539105581}) 2024-01-24T23:29:34.930 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.930 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4225699830}) 2024-01-24T23:29:34.930 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=855804758}) 2024-01-24T23:29:34.940 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.940 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3736737700}) 2024-01-24T23:29:34.940 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2521278031}) 2024-01-24T23:29:34.940 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.940 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3736737700}) 2024-01-24T23:29:34.941 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2121793622}) 2024-01-24T23:29:34.941 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.941 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3736737700}) 2024-01-24T23:29:34.941 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2424245991}) 2024-01-24T23:29:34.941 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.941 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3736737700}) 2024-01-24T23:29:34.941 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1822233253}) 2024-01-24T23:29:34.941 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.941 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3736737700}) 2024-01-24T23:29:34.941 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1831204403}) 2024-01-24T23:29:34.941 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.941 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3736737700}) 2024-01-24T23:29:34.941 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1741509933}) 2024-01-24T23:29:34.941 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.942 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3736737700}) 2024-01-24T23:29:34.942 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1485991463}) 2024-01-24T23:29:34.942 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.942 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3736737700}) 2024-01-24T23:29:34.942 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.921+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3940435754}) 2024-01-24T23:29:34.942 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.925+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.942 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.925+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4004455673}) 2024-01-24T23:29:34.942 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.925+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3971212405}) 2024-01-24T23:29:34.942 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.925+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.942 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.925+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4004455673}) 2024-01-24T23:29:34.942 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.925+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2855238742}) 2024-01-24T23:29:34.942 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.925+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.942 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.925+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4004455673}) 2024-01-24T23:29:34.943 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.925+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1236879503}) 2024-01-24T23:29:34.943 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.925+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.943 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.925+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4004455673}) 2024-01-24T23:29:34.943 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.925+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2747617365}) 2024-01-24T23:29:34.943 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.925+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.943 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.925+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4004455673}) 2024-01-24T23:29:34.943 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.925+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=691352607}) 2024-01-24T23:29:34.943 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.925+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.943 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.925+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4004455673}) 2024-01-24T23:29:34.943 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.925+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4097008984}) 2024-01-24T23:29:34.943 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.925+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.943 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.925+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4004455673}) 2024-01-24T23:29:34.943 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.925+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2624028695}) 2024-01-24T23:29:34.944 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.925+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.944 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.925+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4004455673}) 2024-01-24T23:29:34.944 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.925+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4024429870}) 2024-01-24T23:29:34.944 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.929+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.944 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.929+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419006122}) 2024-01-24T23:29:34.944 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.929+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2440216098}) 2024-01-24T23:29:34.944 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.929+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.944 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.929+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419006122}) 2024-01-24T23:29:34.944 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.929+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=590434634}) 2024-01-24T23:29:34.944 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.929+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.944 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.929+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419006122}) 2024-01-24T23:29:34.944 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.929+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3517083459}) 2024-01-24T23:29:34.944 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.929+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.945 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.929+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419006122}) 2024-01-24T23:29:34.945 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.929+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=663067268}) 2024-01-24T23:29:34.945 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.929+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.945 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.929+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419006122}) 2024-01-24T23:29:34.945 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.929+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3073093991}) 2024-01-24T23:29:34.945 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.929+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.945 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.929+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419006122}) 2024-01-24T23:29:34.945 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.929+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1476605795}) 2024-01-24T23:29:34.945 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.929+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.945 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.929+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419006122}) 2024-01-24T23:29:34.945 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.929+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1655565822}) 2024-01-24T23:29:34.945 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.929+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.945 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.929+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419006122}) 2024-01-24T23:29:34.945 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.929+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4046101760}) 2024-01-24T23:29:34.946 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.946 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1066009174}) 2024-01-24T23:29:34.946 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1785602603}) 2024-01-24T23:29:34.946 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.946 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1066009174}) 2024-01-24T23:29:34.946 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3404299533}) 2024-01-24T23:29:34.946 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.946 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1066009174}) 2024-01-24T23:29:34.946 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1210368138}) 2024-01-24T23:29:34.946 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.946 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1066009174}) 2024-01-24T23:29:34.946 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=492441742}) 2024-01-24T23:29:34.946 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.946 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1066009174}) 2024-01-24T23:29:34.947 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4481512}) 2024-01-24T23:29:34.947 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.947 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1066009174}) 2024-01-24T23:29:34.947 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2048159243}) 2024-01-24T23:29:34.947 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.947 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1066009174}) 2024-01-24T23:29:34.947 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1929076368}) 2024-01-24T23:29:34.947 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.947 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1066009174}) 2024-01-24T23:29:34.947 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4212610168}) 2024-01-24T23:29:34.947 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.947 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4048833639}) 2024-01-24T23:29:34.948 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1183267154}) 2024-01-24T23:29:34.948 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.948 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4048833639}) 2024-01-24T23:29:34.948 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1056640603}) 2024-01-24T23:29:34.948 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.948 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4048833639}) 2024-01-24T23:29:34.948 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2492862644}) 2024-01-24T23:29:34.948 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.948 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4048833639}) 2024-01-24T23:29:34.948 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2919859896}) 2024-01-24T23:29:34.948 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.948 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4048833639}) 2024-01-24T23:29:34.948 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=457909463}) 2024-01-24T23:29:34.948 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.949 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4048833639}) 2024-01-24T23:29:34.949 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1888356349}) 2024-01-24T23:29:34.949 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.949 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4048833639}) 2024-01-24T23:29:34.949 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2542961397}) 2024-01-24T23:29:34.949 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.949 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4048833639}) 2024-01-24T23:29:34.949 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.933+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=704442366}) 2024-01-24T23:29:34.949 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.937+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.949 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.937+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=641720512}) 2024-01-24T23:29:34.949 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.937+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3703643035}) 2024-01-24T23:29:34.949 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.937+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.949 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.937+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=641720512}) 2024-01-24T23:29:34.949 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.937+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2673517344}) 2024-01-24T23:29:34.949 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.937+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.950 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.937+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=641720512}) 2024-01-24T23:29:34.950 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.937+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4145663115}) 2024-01-24T23:29:34.950 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.937+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.950 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.937+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=641720512}) 2024-01-24T23:29:34.950 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.937+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3037465694}) 2024-01-24T23:29:34.950 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.937+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.950 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.937+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=641720512}) 2024-01-24T23:29:34.950 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.937+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2791689850}) 2024-01-24T23:29:34.950 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.937+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.950 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.937+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=641720512}) 2024-01-24T23:29:34.950 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.937+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3828083790}) 2024-01-24T23:29:34.950 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.937+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.950 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.937+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=641720512}) 2024-01-24T23:29:34.950 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.937+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3655039031}) 2024-01-24T23:29:34.951 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.937+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.951 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.937+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=641720512}) 2024-01-24T23:29:34.951 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.937+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1938376047}) 2024-01-24T23:29:34.966 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.941+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.966 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.941+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2699363481}) 2024-01-24T23:29:34.966 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.941+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1074800852}) 2024-01-24T23:29:34.966 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.941+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.966 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.941+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2699363481}) 2024-01-24T23:29:34.966 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.941+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2945239745}) 2024-01-24T23:29:34.966 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.941+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.966 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.941+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2699363481}) 2024-01-24T23:29:34.967 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.941+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4157783628}) 2024-01-24T23:29:34.967 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.941+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.967 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.941+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2699363481}) 2024-01-24T23:29:34.967 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.941+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3798432276}) 2024-01-24T23:29:34.967 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.941+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.967 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.941+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2699363481}) 2024-01-24T23:29:34.967 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.941+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=937873618}) 2024-01-24T23:29:34.967 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.941+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.967 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.941+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2699363481}) 2024-01-24T23:29:34.967 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.941+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1058494332}) 2024-01-24T23:29:34.967 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.941+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.967 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.941+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2699363481}) 2024-01-24T23:29:34.967 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.941+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2040068564}) 2024-01-24T23:29:34.968 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.941+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.968 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.941+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2699363481}) 2024-01-24T23:29:34.968 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.941+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3833321058}) 2024-01-24T23:29:34.968 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.968 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=492703079}) 2024-01-24T23:29:34.968 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4257932041}) 2024-01-24T23:29:34.968 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.968 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=492703079}) 2024-01-24T23:29:34.968 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1467796003}) 2024-01-24T23:29:34.968 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.968 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=492703079}) 2024-01-24T23:29:34.968 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1559559305}) 2024-01-24T23:29:34.968 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.968 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=492703079}) 2024-01-24T23:29:34.969 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3196478168}) 2024-01-24T23:29:34.969 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.969 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=492703079}) 2024-01-24T23:29:34.969 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2364855986}) 2024-01-24T23:29:34.969 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.969 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=492703079}) 2024-01-24T23:29:34.969 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3732551263}) 2024-01-24T23:29:34.969 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.969 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=492703079}) 2024-01-24T23:29:34.969 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=28228486}) 2024-01-24T23:29:34.969 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.969 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=492703079}) 2024-01-24T23:29:34.969 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1303060410}) 2024-01-24T23:29:34.969 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.969 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1687144767}) 2024-01-24T23:29:34.970 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3872975574}) 2024-01-24T23:29:34.970 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.970 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1687144767}) 2024-01-24T23:29:34.970 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3657717596}) 2024-01-24T23:29:34.970 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.970 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1687144767}) 2024-01-24T23:29:34.970 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1411413375}) 2024-01-24T23:29:34.970 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.970 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1687144767}) 2024-01-24T23:29:34.970 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=32603506}) 2024-01-24T23:29:34.970 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.970 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.945+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1687144767}) 2024-01-24T23:29:34.970 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2173516623}) 2024-01-24T23:29:34.970 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.971 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1687144767}) 2024-01-24T23:29:34.971 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1555036238}) 2024-01-24T23:29:34.971 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.971 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1687144767}) 2024-01-24T23:29:34.971 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2896049904}) 2024-01-24T23:29:34.971 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.971 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1687144767}) 2024-01-24T23:29:34.971 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1652764976}) 2024-01-24T23:29:34.971 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.971 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1376985575}) 2024-01-24T23:29:34.971 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2145519112}) 2024-01-24T23:29:34.971 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.971 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1376985575}) 2024-01-24T23:29:34.971 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=223121120}) 2024-01-24T23:29:34.972 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.972 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1376985575}) 2024-01-24T23:29:34.972 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4003195809}) 2024-01-24T23:29:34.972 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.972 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1376985575}) 2024-01-24T23:29:34.972 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3814473738}) 2024-01-24T23:29:34.972 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.972 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1376985575}) 2024-01-24T23:29:34.972 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3270151025}) 2024-01-24T23:29:34.972 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.972 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1376985575}) 2024-01-24T23:29:34.972 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3331814025}) 2024-01-24T23:29:34.972 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.972 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1376985575}) 2024-01-24T23:29:34.973 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4262830748}) 2024-01-24T23:29:34.973 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.973 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1376985575}) 2024-01-24T23:29:34.973 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.949+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2251567041}) 2024-01-24T23:29:34.973 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.953+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.973 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.953+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=540848875}) 2024-01-24T23:29:34.973 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.953+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2036042387}) 2024-01-24T23:29:34.973 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.953+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.973 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.953+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=540848875}) 2024-01-24T23:29:34.973 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.953+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3465201647}) 2024-01-24T23:29:34.973 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.953+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.973 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.953+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=540848875}) 2024-01-24T23:29:34.973 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.953+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2861367220}) 2024-01-24T23:29:34.973 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.953+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.974 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.953+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=540848875}) 2024-01-24T23:29:34.974 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.953+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2516829277}) 2024-01-24T23:29:34.974 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.953+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.974 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.953+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=540848875}) 2024-01-24T23:29:34.974 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.953+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=794271293}) 2024-01-24T23:29:34.974 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.953+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.974 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.953+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=540848875}) 2024-01-24T23:29:34.974 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.953+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3742605037}) 2024-01-24T23:29:34.974 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.953+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.974 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.953+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=540848875}) 2024-01-24T23:29:34.974 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.953+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2629893937}) 2024-01-24T23:29:34.974 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.953+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.974 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.953+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=540848875}) 2024-01-24T23:29:34.974 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.953+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=359934021}) 2024-01-24T23:29:34.975 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.975 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=931589754}) 2024-01-24T23:29:34.975 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1175906282}) 2024-01-24T23:29:34.975 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.975 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=931589754}) 2024-01-24T23:29:34.975 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2848606145}) 2024-01-24T23:29:34.975 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.975 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=931589754}) 2024-01-24T23:29:34.975 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3422454351}) 2024-01-24T23:29:34.975 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.975 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=931589754}) 2024-01-24T23:29:34.975 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1474745777}) 2024-01-24T23:29:34.975 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.975 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=931589754}) 2024-01-24T23:29:34.976 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1277075220}) 2024-01-24T23:29:34.976 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.976 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=931589754}) 2024-01-24T23:29:34.976 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3051518422}) 2024-01-24T23:29:34.976 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.976 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=931589754}) 2024-01-24T23:29:34.976 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1311704620}) 2024-01-24T23:29:34.976 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.976 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=931589754}) 2024-01-24T23:29:34.976 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1155072956}) 2024-01-24T23:29:34.976 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.976 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1015884148}) 2024-01-24T23:29:34.976 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=520686351}) 2024-01-24T23:29:34.977 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.977 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1015884148}) 2024-01-24T23:29:34.977 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4194079060}) 2024-01-24T23:29:34.977 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.977 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1015884148}) 2024-01-24T23:29:34.977 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1585506493}) 2024-01-24T23:29:34.977 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.977 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1015884148}) 2024-01-24T23:29:34.977 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=428112719}) 2024-01-24T23:29:34.977 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.977 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1015884148}) 2024-01-24T23:29:34.977 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2512158988}) 2024-01-24T23:29:34.977 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.977 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1015884148}) 2024-01-24T23:29:34.978 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4286963977}) 2024-01-24T23:29:34.978 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.978 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1015884148}) 2024-01-24T23:29:34.978 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1741076892}) 2024-01-24T23:29:34.978 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.978 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1015884148}) 2024-01-24T23:29:34.978 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.957+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1741793784}) 2024-01-24T23:29:34.978 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.961+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.978 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.961+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2143991177}) 2024-01-24T23:29:34.978 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.961+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3239613217}) 2024-01-24T23:29:34.978 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.961+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.978 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.961+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2143991177}) 2024-01-24T23:29:34.978 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.961+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2074322131}) 2024-01-24T23:29:34.978 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.961+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.979 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.961+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2143991177}) 2024-01-24T23:29:34.979 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.961+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3670985563}) 2024-01-24T23:29:34.979 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.961+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.979 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.961+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2143991177}) 2024-01-24T23:29:34.979 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.961+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3908630449}) 2024-01-24T23:29:34.979 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.961+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.979 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.961+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2143991177}) 2024-01-24T23:29:34.979 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.961+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2326731258}) 2024-01-24T23:29:34.979 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.961+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.979 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.961+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2143991177}) 2024-01-24T23:29:34.979 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.961+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=191833770}) 2024-01-24T23:29:34.979 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.961+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.979 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.961+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2143991177}) 2024-01-24T23:29:34.980 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.961+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2334040405}) 2024-01-24T23:29:34.980 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.961+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.980 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.961+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2143991177}) 2024-01-24T23:29:34.980 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.961+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=61602984}) 2024-01-24T23:29:34.980 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.980 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1812091149}) 2024-01-24T23:29:34.980 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1926154432}) 2024-01-24T23:29:34.980 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.980 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1812091149}) 2024-01-24T23:29:34.980 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2884559648}) 2024-01-24T23:29:34.980 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.980 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1812091149}) 2024-01-24T23:29:34.980 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=183000354}) 2024-01-24T23:29:34.981 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.981 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1812091149}) 2024-01-24T23:29:34.981 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=915979456}) 2024-01-24T23:29:34.981 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.981 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1812091149}) 2024-01-24T23:29:34.981 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2932202850}) 2024-01-24T23:29:34.981 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.981 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1812091149}) 2024-01-24T23:29:34.981 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2933962877}) 2024-01-24T23:29:34.981 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.981 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1812091149}) 2024-01-24T23:29:34.981 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2605346982}) 2024-01-24T23:29:34.981 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.981 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1812091149}) 2024-01-24T23:29:34.982 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1964890093}) 2024-01-24T23:29:34.994 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.994 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1479526644}) 2024-01-24T23:29:34.995 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1506699427}) 2024-01-24T23:29:34.995 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.995 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1479526644}) 2024-01-24T23:29:34.995 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=151356775}) 2024-01-24T23:29:34.995 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.995 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1479526644}) 2024-01-24T23:29:34.995 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=177076164}) 2024-01-24T23:29:34.995 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.995 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1479526644}) 2024-01-24T23:29:34.995 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3355906987}) 2024-01-24T23:29:34.995 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.995 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.965+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1479526644}) 2024-01-24T23:29:34.996 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1417319703}) 2024-01-24T23:29:34.996 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.996 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1479526644}) 2024-01-24T23:29:34.996 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3376423121}) 2024-01-24T23:29:34.996 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.996 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1479526644}) 2024-01-24T23:29:34.996 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1748588607}) 2024-01-24T23:29:34.996 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.996 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1479526644}) 2024-01-24T23:29:34.996 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2161611666}) 2024-01-24T23:29:34.996 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.996 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3897399189}) 2024-01-24T23:29:34.996 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2371934979}) 2024-01-24T23:29:34.996 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.997 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3897399189}) 2024-01-24T23:29:34.997 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=189221781}) 2024-01-24T23:29:34.997 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.997 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3897399189}) 2024-01-24T23:29:34.997 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1017035004}) 2024-01-24T23:29:34.997 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.997 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3897399189}) 2024-01-24T23:29:34.997 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3442435168}) 2024-01-24T23:29:34.997 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.997 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3897399189}) 2024-01-24T23:29:34.997 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1105162393}) 2024-01-24T23:29:34.997 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.997 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3897399189}) 2024-01-24T23:29:34.997 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2213866547}) 2024-01-24T23:29:34.997 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.998 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3897399189}) 2024-01-24T23:29:34.998 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=630514025}) 2024-01-24T23:29:34.998 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.998 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3897399189}) 2024-01-24T23:29:34.998 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.969+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2257205831}) 2024-01-24T23:29:34.998 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.973+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.998 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.973+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=33,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=30} crc {logm=2006798523,mdsmap=3984425268,mgr=435811569,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2584768273,mon_config_key=4192202708,monmap=701157023,osd_metadata=2031414162,osd_pg_creating=4127804811,osdmap=2137859984}) 2024-01-24T23:29:34.998 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.973+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=33,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=7,monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=28} crc {logm=3388228410,mdsmap=3984425269,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2355767147,mon_config_key=2187968908,monmap=701157023,osd_metadata=3073132558,osd_pg_creating=4127804811,osdmap=1152080397}) 2024-01-24T23:29:34.998 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.973+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.998 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.973+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=33,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=30} crc {logm=2006798523,mdsmap=3984425268,mgr=435811569,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2584768273,mon_config_key=4192202708,monmap=701157023,osd_metadata=2031414162,osd_pg_creating=4127804811,osdmap=2137859984}) 2024-01-24T23:29:34.998 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.973+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=35,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=7,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=26} crc {logm=1344954900,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2328222014,mon_config_key=3549988868,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=4127804811,osdmap=1816760695}) 2024-01-24T23:29:34.998 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.973+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.998 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.973+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=33,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=30} crc {logm=2006798523,mdsmap=3984425268,mgr=435811569,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2584768273,mon_config_key=4192202708,monmap=701157023,osd_metadata=2031414162,osd_pg_creating=4127804811,osdmap=2137859984}) 2024-01-24T23:29:34.998 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.973+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=34,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=8,monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=26} crc {logm=3860395173,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2355767147,mon_config_key=4121867429,monmap=701157023,osd_metadata=3207875622,osd_pg_creating=4127804811,osdmap=2264117446}) 2024-01-24T23:29:34.999 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.973+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.999 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.973+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=33,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=30} crc {logm=2006798523,mdsmap=3984425268,mgr=435811569,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2584768273,mon_config_key=4192202708,monmap=701157023,osd_metadata=2031414162,osd_pg_creating=4127804811,osdmap=2137859984}) 2024-01-24T23:29:34.999 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.973+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=35,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=24} crc {logm=1344954900,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2361689113,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=4127804811,osdmap=2062779658}) 2024-01-24T23:29:34.999 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.973+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.999 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.973+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=33,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=30} crc {logm=2006798523,mdsmap=3984425268,mgr=435811569,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2584768273,mon_config_key=4192202708,monmap=701157023,osd_metadata=2031414162,osd_pg_creating=4127804811,osdmap=2137859984}) 2024-01-24T23:29:34.999 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.973+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=32,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=28} crc {logm=307344074,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=4151247847,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=4127804811,osdmap=1985090854}) 2024-01-24T23:29:34.999 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.973+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.999 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.973+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=33,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=30} crc {logm=2006798523,mdsmap=3984425268,mgr=435811569,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2584768273,mon_config_key=4192202708,monmap=701157023,osd_metadata=2031414162,osd_pg_creating=4127804811,osdmap=2137859984}) 2024-01-24T23:29:34.999 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.973+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=32,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=27} crc {logm=1912320539,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2355767147,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=4127804811,osdmap=213859840}) 2024-01-24T23:29:34.999 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.973+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.999 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.973+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=33,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=30} crc {logm=2006798523,mdsmap=3984425268,mgr=435811569,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2584768273,mon_config_key=4192202708,monmap=701157023,osd_metadata=2031414162,osd_pg_creating=4127804811,osdmap=2137859984}) 2024-01-24T23:29:34.999 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.973+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=34,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=26} crc {logm=895582695,mdsmap=2441318933,mgr=1221499742,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2355767147,mon_config_key=4121867429,monmap=701157023,osd_metadata=3193413316,osd_pg_creating=4127804811,osdmap=1014660581}) 2024-01-24T23:29:34.999 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.973+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:34.999 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.973+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=33,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8,monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=30} crc {logm=2006798523,mdsmap=3984425268,mgr=435811569,mgr_command_descs=3928247861,mgr_metadata=2479342215,mgrstat=2584768273,mon_config_key=4192202708,monmap=701157023,osd_metadata=2031414162,osd_pg_creating=4127804811,osdmap=2137859984}) 2024-01-24T23:29:35.000 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.973+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=35,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=10,mon_config_key=8,monmap=2,osd_metadata=6,osd_pg_creating=1,osdmap=26} crc {logm=1344954900,mdsmap=3984425268,mgr=1221499742,mgr_command_descs=3928247861,mgrstat=2355767147,mon_config_key=4121867429,monmap=1293597301,osd_metadata=3193413316,osd_pg_creating=4127804811,osdmap=4099141947}) 2024-01-24T23:29:35.000 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.977+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.000 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.977+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=615029968}) 2024-01-24T23:29:35.000 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.977+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2281121030}) 2024-01-24T23:29:35.000 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.977+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.000 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.977+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=615029968}) 2024-01-24T23:29:35.000 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.977+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4160198632}) 2024-01-24T23:29:35.000 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.977+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.000 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.977+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=615029968}) 2024-01-24T23:29:35.000 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.977+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=80812684}) 2024-01-24T23:29:35.000 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.977+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.000 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.977+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=615029968}) 2024-01-24T23:29:35.000 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.977+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3688040020}) 2024-01-24T23:29:35.000 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.977+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.000 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.977+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=615029968}) 2024-01-24T23:29:35.001 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.977+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2284981341}) 2024-01-24T23:29:35.001 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.977+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.001 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.977+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=615029968}) 2024-01-24T23:29:35.001 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.977+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3722454484}) 2024-01-24T23:29:35.001 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.977+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.001 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.977+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=615029968}) 2024-01-24T23:29:35.001 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.977+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1839852350}) 2024-01-24T23:29:35.001 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.977+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.001 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.977+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=615029968}) 2024-01-24T23:29:35.001 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.977+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3076889043}) 2024-01-24T23:29:35.001 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.981+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.001 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.981+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1377717148}) 2024-01-24T23:29:35.001 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.981+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2621633056}) 2024-01-24T23:29:35.002 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.981+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.002 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.981+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1377717148}) 2024-01-24T23:29:35.002 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.981+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=125582740}) 2024-01-24T23:29:35.002 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.981+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.002 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.981+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1377717148}) 2024-01-24T23:29:35.002 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.981+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3902424450}) 2024-01-24T23:29:35.002 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.981+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.002 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.981+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1377717148}) 2024-01-24T23:29:35.002 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.981+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1975094511}) 2024-01-24T23:29:35.002 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.981+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.002 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.981+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1377717148}) 2024-01-24T23:29:35.002 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.981+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1306316941}) 2024-01-24T23:29:35.002 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.981+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.002 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.981+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1377717148}) 2024-01-24T23:29:35.002 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.981+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1877753482}) 2024-01-24T23:29:35.003 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.981+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.003 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.981+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1377717148}) 2024-01-24T23:29:35.003 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.981+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=784881852}) 2024-01-24T23:29:35.003 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.981+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.003 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.981+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1377717148}) 2024-01-24T23:29:35.003 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.981+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3356717684}) 2024-01-24T23:29:35.003 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.003 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=667090043}) 2024-01-24T23:29:35.003 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1418056520}) 2024-01-24T23:29:35.003 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.003 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=667090043}) 2024-01-24T23:29:35.003 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2090400413}) 2024-01-24T23:29:35.003 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.003 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=667090043}) 2024-01-24T23:29:35.004 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=175919869}) 2024-01-24T23:29:35.004 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.004 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=667090043}) 2024-01-24T23:29:35.004 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=557735443}) 2024-01-24T23:29:35.004 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.004 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=667090043}) 2024-01-24T23:29:35.004 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=12402802}) 2024-01-24T23:29:35.004 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.004 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=667090043}) 2024-01-24T23:29:35.004 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=633036888}) 2024-01-24T23:29:35.004 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.004 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=667090043}) 2024-01-24T23:29:35.004 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2518098194}) 2024-01-24T23:29:35.004 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.004 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=667090043}) 2024-01-24T23:29:35.005 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3551119179}) 2024-01-24T23:29:35.005 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.005 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=41} crc {osdmap=1645203744}) 2024-01-24T23:29:35.005 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=42} crc {osdmap=3176959334}) 2024-01-24T23:29:35.005 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.005 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=41} crc {osdmap=1645203744}) 2024-01-24T23:29:35.005 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=39} crc {osdmap=3462174822}) 2024-01-24T23:29:35.005 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.005 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=41} crc {osdmap=1645203744}) 2024-01-24T23:29:35.005 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=45} crc {osdmap=2187742401}) 2024-01-24T23:29:35.005 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.005 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=41} crc {osdmap=1645203744}) 2024-01-24T23:29:35.005 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=42} crc {osdmap=3498702398}) 2024-01-24T23:29:35.005 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.006 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=41} crc {osdmap=1645203744}) 2024-01-24T23:29:35.006 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=43} crc {osdmap=4271976632}) 2024-01-24T23:29:35.006 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.006 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=41} crc {osdmap=1645203744}) 2024-01-24T23:29:35.006 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=43} crc {osdmap=3053783738}) 2024-01-24T23:29:35.006 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.006 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=41} crc {osdmap=1645203744}) 2024-01-24T23:29:35.006 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=45} crc {osdmap=3754573816}) 2024-01-24T23:29:35.006 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-01-24T23:29:35.006 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=41} crc {osdmap=1645203744}) 2024-01-24T23:29:35.006 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:29:34.989+0000 7f48d6072640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=43} crc {osdmap=2862641065}) 2024-01-24T23:29:35.821 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-01-24T23:29:35.821 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-01-24T23:29:35.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c8e012640 10 monclient: get_monmap_and_config 2024-01-24T23:29:35.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c8e012640 10 monclient: build_initial_monmap 2024-01-24T23:29:35.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c8e012640 10 monclient: monmap: 2024-01-24T23:29:35.953 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:35.953 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:35.954 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:35.956100+0000 2024-01-24T23:29:35.954 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:35.956100+0000 2024-01-24T23:29:35.954 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:35.954 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:35.954 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:35.954 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:35.954 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:35.954 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:35.954 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:35.954 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:35.954 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:35.954 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:35.954 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:35.955 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:35.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c8e012640 1 Processor -- start 2024-01-24T23:29:35.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c8e012640 1 -- start start 2024-01-24T23:29:35.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c8e012640 10 monclient: init 2024-01-24T23:29:35.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c8e012640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:35.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c8e012640 10 monclient: _add_conns ranks=[1,7,3,0,4,2,8,5,6] 2024-01-24T23:29:35.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c8e012640 10 monclient(hunting): picked mon.noname-f con 0x7f3c88133bf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:29:35.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c8e012640 10 monclient(hunting): picked mon.noname-i con 0x7f3c88133fd0 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:35.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c8e012640 10 monclient(hunting): picked mon.noname-g con 0x7f3c88134430 addr v1:172.21.15.132:6790/0 2024-01-24T23:29:35.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c8e012640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c8804ed70 con 0x7f3c88133bf0 2024-01-24T23:29:35.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c8e012640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c8809af30 con 0x7f3c88134430 2024-01-24T23:29:35.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c8e012640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c8809b210 con 0x7f3c88133fd0 2024-01-24T23:29:35.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c8e012640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:35.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c8e012640 10 monclient(hunting): authenticate will time out at 1454.878752s 2024-01-24T23:29:35.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c877fe640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f3c88133bf0 0x7f3c8809bd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.115:37142/0 (socket says 172.21.15.115:37142) 2024-01-24T23:29:35.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c877fe640 1 -- 172.21.15.115:0/1499324812 learned_addr learned my addr 172.21.15.115:0/1499324812 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:35.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c867fc640 1 -- 172.21.15.115:0/1499324812 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357694671 0 0) 0x7f3c8804ed70 con 0x7f3c88133bf0 2024-01-24T23:29:35.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c867fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:35.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c867fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:35.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c867fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:35.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c867fc640 10 monclient(hunting): my global_id is 124705 2024-01-24T23:29:35.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c867fc640 1 -- 172.21.15.115:0/1499324812 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c70003530 con 0x7f3c88133bf0 2024-01-24T23:29:35.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c867fc640 1 -- 172.21.15.115:0/1499324812 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3297148354 0 0) 0x7f3c8809af30 con 0x7f3c88134430 2024-01-24T23:29:35.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c867fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:35.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c867fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:35.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c867fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:35.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c867fc640 10 monclient(hunting): my global_id is 7140 2024-01-24T23:29:35.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c867fc640 1 -- 172.21.15.115:0/1499324812 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c8804ed70 con 0x7f3c88134430 2024-01-24T23:29:35.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c867fc640 1 -- 172.21.15.115:0/1499324812 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2469361772 0 0) 0x7f3c8809b210 con 0x7f3c88133fd0 2024-01-24T23:29:35.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c867fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:35.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c867fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:35.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c867fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:35.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c867fc640 10 monclient(hunting): my global_id is 154186 2024-01-24T23:29:35.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.951+0000 7f3c867fc640 1 -- 172.21.15.115:0/1499324812 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c8809af30 con 0x7f3c88133fd0 2024-01-24T23:29:35.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c867fc640 1 -- 172.21.15.115:0/1499324812 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 177251132 0 0) 0x7f3c8804ed70 con 0x7f3c88134430 2024-01-24T23:29:35.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c867fc640 1 -- 172.21.15.115:0/1499324812 >> v1:172.21.15.132:6792/0 conn(0x7f3c88133fd0 legacy=0x7f3c8813bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:35.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c867fc640 1 -- 172.21.15.115:0/1499324812 >> v1:172.21.15.132:6789/0 conn(0x7f3c88133bf0 legacy=0x7f3c8809bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:35.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c867fc640 10 monclient: _finish_hunting 0 2024-01-24T23:29:35.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c867fc640 1 monclient: found mon.noname-g 2024-01-24T23:29:35.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c867fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:35.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c867fc640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:29:35.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c867fc640 1 -- 172.21.15.115:0/1499324812 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8809b4f0 con 0x7f3c88134430 2024-01-24T23:29:35.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c867fc640 10 monclient: _finish_auth 0 2024-01-24T23:29:35.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c867fc640 10 monclient: _check_auth_tickets 2024-01-24T23:29:35.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:35.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c867fc640 1 -- 172.21.15.115:0/1499324812 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3c80001b20 con 0x7f3c88134430 2024-01-24T23:29:35.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:35.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c867fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:29:35.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c867fc640 10 monclient: dump: 2024-01-24T23:29:35.962 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:35.962 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:35.962 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:35.962 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:35.962 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:35.962 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:35.962 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:35.962 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:35.963 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:35.963 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:35.963 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:35.963 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:35.963 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:35.963 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:35.963 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:35.963 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:35.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 5 monclient: authenticate success, global_id 7140 2024-01-24T23:29:35.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:35.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c867fc640 1 -- 172.21.15.115:0/1499324812 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3c80001d40 con 0x7f3c88134430 2024-01-24T23:29:35.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c867fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:35.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c867fc640 1 -- 172.21.15.115:0/1499324812 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3c80003ce0 con 0x7f3c88134430 2024-01-24T23:29:35.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:35.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c867fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:29:35.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c867fc640 10 monclient: dump: 2024-01-24T23:29:35.964 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:35.964 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:35.964 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:35.964 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:35.964 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:35.964 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:35.964 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:35.964 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:35.964 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:35.964 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:35.964 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:35.965 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:35.965 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:35.965 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:35.965 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:35.965 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:35.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:35.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 4 set_mon_vals no callback set 2024-01-24T23:29:35.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 10 monclient: shutdown 2024-01-24T23:29:35.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:35.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 1 -- 172.21.15.115:0/1499324812 >> v1:172.21.15.132:6790/0 conn(0x7f3c88134430 legacy=0x7f3c8813f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:35.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 1 -- 172.21.15.115:0/1499324812 shutdown_connections 2024-01-24T23:29:35.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 1 -- 172.21.15.115:0/1499324812 >> 172.21.15.115:0/1499324812 conn(0x7f3c880a9910 msgr2=0x7f3c880a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:35.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 1 -- 172.21.15.115:0/1499324812 shutdown_connections 2024-01-24T23:29:35.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 1 -- 172.21.15.115:0/1499324812 wait complete. 2024-01-24T23:29:35.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 10 monclient: build_initial_monmap 2024-01-24T23:29:35.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 10 monclient: monmap: 2024-01-24T23:29:35.966 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:35.966 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:35.966 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:35.966 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:35.966 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:35.966 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:35.966 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:35.966 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:35.966 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:35.966 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:35.966 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:35.967 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:35.967 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:35.967 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:35.967 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:35.967 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:35.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 1 Processor -- start 2024-01-24T23:29:35.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 1 -- start start 2024-01-24T23:29:35.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 10 monclient: init 2024-01-24T23:29:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 10 monclient: _add_conns ranks=[0,8,7,4,2,5,3,1,6] 2024-01-24T23:29:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 10 monclient(hunting): picked mon.noname-a con 0x7f3c88133bf0 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 10 monclient(hunting): picked mon.noname-e con 0x7f3c88133fd0 addr v1:172.21.15.115:6793/0 2024-01-24T23:29:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 10 monclient(hunting): picked mon.noname-i con 0x7f3c88134430 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c8809c8f0 con 0x7f3c88133bf0 2024-01-24T23:29:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c880a5960 con 0x7f3c88134430 2024-01-24T23:29:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c8809ff60 con 0x7f3c88133fd0 2024-01-24T23:29:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c8e012640 10 monclient(hunting): authenticate will time out at 1454.883214s 2024-01-24T23:29:35.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c877fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f3c88133bf0 0x7f3c8813ef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39200/0 (socket says 172.21.15.115:39200) 2024-01-24T23:29:35.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c877fe640 1 -- 172.21.15.115:0/2866051174 learned_addr learned my addr 172.21.15.115:0/2866051174 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:35.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 1 -- 172.21.15.115:0/2866051174 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546906530 0 0) 0x7f3c8809c8f0 con 0x7f3c88133bf0 2024-01-24T23:29:35.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:35.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:35.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:35.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 10 monclient(hunting): my global_id is 154215 2024-01-24T23:29:35.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 1 -- 172.21.15.115:0/2866051174 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c74003530 con 0x7f3c88133bf0 2024-01-24T23:29:35.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 1 -- 172.21.15.115:0/2866051174 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2569370707 0 0) 0x7f3c8809ff60 con 0x7f3c88133fd0 2024-01-24T23:29:35.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:35.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:35.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:35.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 10 monclient(hunting): my global_id is 86390 2024-01-24T23:29:35.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 1 -- 172.21.15.115:0/2866051174 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c8809c8f0 con 0x7f3c88133fd0 2024-01-24T23:29:35.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 1 -- 172.21.15.115:0/2866051174 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1980094158 0 0) 0x7f3c74003530 con 0x7f3c88133bf0 2024-01-24T23:29:35.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 1 -- 172.21.15.115:0/2866051174 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3c8809ff60 con 0x7f3c88133bf0 2024-01-24T23:29:35.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 1 -- 172.21.15.115:0/2866051174 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3c780015a0 con 0x7f3c88133bf0 2024-01-24T23:29:35.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:35.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:29:35.973 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 10 monclient(hunting): dump: 2024-01-24T23:29:35.973 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:35.973 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:35.973 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:35.973 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:35.973 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:35.973 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:35.973 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:35.973 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:35.973 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:35.974 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:35.974 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:35.974 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:35.974 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:35.974 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:35.974 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:35.974 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:35.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:35.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 1 -- 172.21.15.115:0/2866051174 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3236341519 0 0) 0x7f3c880a5960 con 0x7f3c88134430 2024-01-24T23:29:35.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:35.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:35.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:35.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 10 monclient(hunting): my global_id is 154195 2024-01-24T23:29:35.974 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.955+0000 7f3c6ffff640 1 -- 172.21.15.115:0/2866051174 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c74003530 con 0x7f3c88134430 2024-01-24T23:29:35.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c6ffff640 1 -- 172.21.15.115:0/2866051174 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1721984298 0 0) 0x7f3c8809ff60 con 0x7f3c88133bf0 2024-01-24T23:29:35.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c6ffff640 1 -- 172.21.15.115:0/2866051174 >> v1:172.21.15.115:6793/0 conn(0x7f3c88133fd0 legacy=0x7f3c881d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:35.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c6ffff640 1 -- 172.21.15.115:0/2866051174 >> v1:172.21.15.132:6792/0 conn(0x7f3c88134430 legacy=0x7f3c881d3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:35.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c6ffff640 10 monclient: _finish_hunting 0 2024-01-24T23:29:35.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c6ffff640 1 monclient: found mon.a 2024-01-24T23:29:35.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c6ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:35.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c6ffff640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:29:35.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c6ffff640 1 -- 172.21.15.115:0/2866051174 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c881395b0 con 0x7f3c88133bf0 2024-01-24T23:29:35.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c6ffff640 10 monclient: _finish_auth 0 2024-01-24T23:29:35.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c6ffff640 10 monclient: _check_auth_tickets 2024-01-24T23:29:35.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c6ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:35.975 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c8e012640 5 monclient: authenticate success, global_id 154215 2024-01-24T23:29:35.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c8e012640 10 monclient: _renew_subs 2024-01-24T23:29:35.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c8e012640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:29:35.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c8e012640 1 -- 172.21.15.115:0/2866051174 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c8813ab20 con 0x7f3c88133bf0 2024-01-24T23:29:35.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c8e012640 10 monclient: _renew_subs 2024-01-24T23:29:35.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c8e012640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:29:35.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c8e012640 1 -- 172.21.15.115:0/2866051174 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c8809f620 con 0x7f3c88133bf0 2024-01-24T23:29:35.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c6ffff640 1 -- 172.21.15.115:0/2866051174 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3c78004aa0 con 0x7f3c88133bf0 2024-01-24T23:29:35.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c6ffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:35.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c6ffff640 1 -- 172.21.15.115:0/2866051174 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3c78004f50 con 0x7f3c88133bf0 2024-01-24T23:29:35.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c6ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:35.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c85ffb640 4 set_mon_vals no callback set 2024-01-24T23:29:35.976 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c6ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:29:35.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c6ffff640 10 monclient: dump: 2024-01-24T23:29:35.977 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:35.977 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:35.977 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:35.977 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:35.977 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:35.977 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:35.977 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:35.977 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:35.977 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:35.977 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:35.977 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:35.977 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:35.978 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:35.978 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:35.978 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:35.978 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:35.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c6ffff640 1 -- 172.21.15.115:0/2866051174 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f3c7800f9d0 con 0x7f3c88133bf0 2024-01-24T23:29:35.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c8e012640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:35.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c8e012640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:35.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c8e012640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:29:35.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c8e012640 1 -- 172.21.15.115:0/2866051174 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c68005090 con 0x7f3c88133bf0 2024-01-24T23:29:35.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c6ffff640 1 -- 172.21.15.115:0/2866051174 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7f3c780445a0 con 0x7f3c88133bf0 2024-01-24T23:29:35.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.959+0000 7f3c86ffd640 10 monclient: get_auth_request con 0x7f3c740370c0 auth_method 0 2024-01-24T23:29:35.978 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.963+0000 7f3c6ffff640 1 -- 172.21.15.115:0/2866051174 <== mon.0 v1:172.21.15.115:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f3c78014910 con 0x7f3c88133bf0 2024-01-24T23:29:35.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.963+0000 7f3c6ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:35.979 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:35.963+0000 7f3c6ffff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:36.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.175+0000 7f3c8e012640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-01-24T23:29:36.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.175+0000 7f3c8e012640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-01-24T23:29:36.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.175+0000 7f3c8e012640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:29:36.181 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.175+0000 7f3c8e012640 1 -- 172.21.15.115:0/2866051174 --> v1:172.21.15.115:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3c680058c0 con 0x7f3c88133bf0 2024-01-24T23:29:36.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.175+0000 7f3c6ffff640 1 -- 172.21.15.115:0/2866051174 <== mon.0 v1:172.21.15.115:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2744 (unknown 254746657 0 849974109) 0x7f3c78012fb0 con 0x7f3c88133bf0 2024-01-24T23:29:36.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.175+0000 7f3c6ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-01-24T23:29:36.182 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.175+0000 7f3c6ffff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:36.182 INFO:teuthology.orchestra.run.smithi115.stdout:{"election_epoch":48,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":6,"features":{"quorum_con":"4540701272860131327","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","modified":"2024-01-24T23:20:20.557246Z","created":"2024-01-24T23:20:20.557246Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6790","nonce":0}]},"addr":"172.21.15.115:6790/0","public_addr":"172.21.15.115:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6790","nonce":0}]},"addr":"172.21.15.132:6790/0","public_addr":"172.21.15.132:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6791","nonce":0}]},"addr":"172.21.15.115:6791/0","public_addr":"172.21.15.115:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6791","nonce":0}]},"addr":"172.21.15.132:6791/0","public_addr":"172.21.15.132:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6792","nonce":0}]},"addr":"172.21.15.115:6792/0","public_addr":"172.21.15.115:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6792","nonce":0}]},"addr":"172.21.15.132:6792/0","public_addr":"172.21.15.132:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6793","nonce":0}]},"addr":"172.21.15.115:6793/0","public_addr":"172.21.15.115:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-01-24T23:29:36.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.179+0000 7f3c8e012640 1 -- 172.21.15.115:0/2866051174 >> v1:172.21.15.132:6812/1835401922 conn(0x7f3c740370c0 legacy=0x7f3c74039580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.179+0000 7f3c8e012640 10 monclient: shutdown 2024-01-24T23:29:36.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.179+0000 7f3c8e012640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:36.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.179+0000 7f3c8e012640 1 -- 172.21.15.115:0/2866051174 >> v1:172.21.15.115:6789/0 conn(0x7f3c88133bf0 legacy=0x7f3c8813ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.179+0000 7f3c8e012640 1 -- 172.21.15.115:0/2866051174 shutdown_connections 2024-01-24T23:29:36.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.179+0000 7f3c8e012640 1 -- 172.21.15.115:0/2866051174 >> 172.21.15.115:0/2866051174 conn(0x7f3c880a9910 msgr2=0x7f3c8813fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:36.183 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.179+0000 7f3c8e012640 1 -- 172.21.15.115:0/2866051174 shutdown_connections 2024-01-24T23:29:36.184 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.179+0000 7f3c8e012640 1 -- 172.21.15.115:0/2866051174 wait complete. 2024-01-24T23:29:36.197 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-01-24T23:29:36.197 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-01-24T23:29:36.197 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-01-24T23:29:36.197 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-01-24T23:29:36.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 10 monclient: get_monmap_and_config 2024-01-24T23:29:36.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 10 monclient: build_initial_monmap 2024-01-24T23:29:36.312 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 10 monclient: monmap: 2024-01-24T23:29:36.312 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:36.312 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.312 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:36.315249+0000 2024-01-24T23:29:36.313 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:36.315249+0000 2024-01-24T23:29:36.313 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:36.313 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.313 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:36.313 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:36.313 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:36.313 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:36.313 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:36.313 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:36.313 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:36.313 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:36.313 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:36.313 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 1 Processor -- start 2024-01-24T23:29:36.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 1 -- start start 2024-01-24T23:29:36.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 10 monclient: init 2024-01-24T23:29:36.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:36.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 10 monclient: _add_conns ranks=[4,0,7,6,5,3,1,2,8] 2024-01-24T23:29:36.320 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 10 monclient(hunting): picked mon.noname-c con 0x7f7e54133bf0 addr v1:172.21.15.115:6791/0 2024-01-24T23:29:36.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 10 monclient(hunting): picked mon.noname-a con 0x7f7e54133fd0 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:36.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 10 monclient(hunting): picked mon.noname-i con 0x7f7e541343b0 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:36.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e5404ed70 con 0x7f7e54133fd0 2024-01-24T23:29:36.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e540a4b60 con 0x7f7e54133bf0 2024-01-24T23:29:36.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e5409ae50 con 0x7f7e541343b0 2024-01-24T23:29:36.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:36.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 10 monclient(hunting): authenticate will time out at 1455.237948s 2024-01-24T23:29:36.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e59dd1640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f7e54133bf0 0x7f7e5409bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:59000/0 (socket says 172.21.15.115:59000) 2024-01-24T23:29:36.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e595d0640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f7e54133fd0 0x7f7e5413bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39202/0 (socket says 172.21.15.115:39202) 2024-01-24T23:29:36.321 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e59dd1640 1 -- 172.21.15.115:0/1343398781 learned_addr learned my addr 172.21.15.115:0/1343398781 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:36.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 1 -- 172.21.15.115:0/1343398781 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3868815872 0 0) 0x7f7e5404ed70 con 0x7f7e54133fd0 2024-01-24T23:29:36.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 10 monclient(hunting): _negotiate 2024-01-24T23:29:36.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:36.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:36.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 10 monclient(hunting): my global_id is 154224 2024-01-24T23:29:36.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 1 -- 172.21.15.115:0/1343398781 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e40003530 con 0x7f7e54133fd0 2024-01-24T23:29:36.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 1 -- 172.21.15.115:0/1343398781 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330765846 0 0) 0x7f7e540a4b60 con 0x7f7e54133bf0 2024-01-24T23:29:36.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 10 monclient(hunting): _negotiate 2024-01-24T23:29:36.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:36.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:36.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 10 monclient(hunting): my global_id is 95305 2024-01-24T23:29:36.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 1 -- 172.21.15.115:0/1343398781 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e5404ed70 con 0x7f7e54133bf0 2024-01-24T23:29:36.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 1 -- 172.21.15.115:0/1343398781 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3462040856 0 0) 0x7f7e5404ed70 con 0x7f7e54133bf0 2024-01-24T23:29:36.322 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 1 -- 172.21.15.115:0/1343398781 >> v1:172.21.15.132:6792/0 conn(0x7f7e541343b0 legacy=0x7f7e5413f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 1 -- 172.21.15.115:0/1343398781 >> v1:172.21.15.115:6789/0 conn(0x7f7e54133fd0 legacy=0x7f7e5413bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 10 monclient: _finish_hunting 0 2024-01-24T23:29:36.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 1 monclient: found mon.noname-c 2024-01-24T23:29:36.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:36.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:29:36.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 1 -- 172.21.15.115:0/1343398781 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5409b130 con 0x7f7e54133bf0 2024-01-24T23:29:36.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 10 monclient: _finish_auth 0 2024-01-24T23:29:36.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 10 monclient: _check_auth_tickets 2024-01-24T23:29:36.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:36.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 1 -- 172.21.15.115:0/1343398781 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e500017c0 con 0x7f7e54133bf0 2024-01-24T23:29:36.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:29:36.323 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 10 monclient: dump: 2024-01-24T23:29:36.323 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.323 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.324 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.324 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.324 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.324 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.324 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.324 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.324 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.324 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.324 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.324 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.324 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.324 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.324 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.324 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.324 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 5 monclient: authenticate success, global_id 95305 2024-01-24T23:29:36.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:36.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 1 -- 172.21.15.115:0/1343398781 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7e50001970 con 0x7f7e54133bf0 2024-01-24T23:29:36.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:36.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 1 -- 172.21.15.115:0/1343398781 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e500048b0 con 0x7f7e54133bf0 2024-01-24T23:29:36.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:36.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 4 set_mon_vals no callback set 2024-01-24T23:29:36.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:29:36.325 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e58dcf640 10 monclient: dump: 2024-01-24T23:29:36.325 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.325 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.325 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.325 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.325 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.326 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.326 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.326 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.326 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.326 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.326 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.326 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.326 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.326 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.326 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.326 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 10 monclient: shutdown 2024-01-24T23:29:36.326 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 1 -- 172.21.15.115:0/1343398781 >> v1:172.21.15.115:6791/0 conn(0x7f7e54133bf0 legacy=0x7f7e5409bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 1 -- 172.21.15.115:0/1343398781 shutdown_connections 2024-01-24T23:29:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 1 -- 172.21.15.115:0/1343398781 >> 172.21.15.115:0/1343398781 conn(0x7f7e540662f0 msgr2=0x7f7e54066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 1 -- 172.21.15.115:0/1343398781 shutdown_connections 2024-01-24T23:29:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.311+0000 7f7e5c035640 1 -- 172.21.15.115:0/1343398781 wait complete. 2024-01-24T23:29:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5c035640 10 monclient: build_initial_monmap 2024-01-24T23:29:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5c035640 10 monclient: monmap: 2024-01-24T23:29:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:36.327 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:36.328 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:36.328 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:36.328 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:36.328 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:36.328 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:36.328 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:36.328 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:36.328 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:36.328 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5c035640 1 Processor -- start 2024-01-24T23:29:36.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5c035640 1 -- start start 2024-01-24T23:29:36.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5c035640 10 monclient: init 2024-01-24T23:29:36.328 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5c035640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:36.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5c035640 10 monclient: _add_conns ranks=[5,3,8,6,4,0,2,1,7] 2024-01-24T23:29:36.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5c035640 10 monclient(hunting): picked mon.noname-h con 0x7f7e54133bf0 addr v1:172.21.15.132:6791/0 2024-01-24T23:29:36.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5c035640 10 monclient(hunting): picked mon.noname-g con 0x7f7e54133fd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:29:36.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5c035640 10 monclient(hunting): picked mon.noname-e con 0x7f7e541343b0 addr v1:172.21.15.115:6793/0 2024-01-24T23:29:36.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5c035640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e5409c810 con 0x7f7e54133fd0 2024-01-24T23:29:36.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5c035640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e540a5990 con 0x7f7e54133bf0 2024-01-24T23:29:36.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5c035640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e54120760 con 0x7f7e541343b0 2024-01-24T23:29:36.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5c035640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:36.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5c035640 10 monclient(hunting): authenticate will time out at 1455.241151s 2024-01-24T23:29:36.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5a5d2640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f7e541343b0 0x7f7e541d3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:55990/0 (socket says 172.21.15.115:55990) 2024-01-24T23:29:36.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5a5d2640 1 -- 172.21.15.115:0/2572399075 learned_addr learned my addr 172.21.15.115:0/2572399075 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:36.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2534374699 0 0) 0x7f7e54120760 con 0x7f7e541343b0 2024-01-24T23:29:36.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:36.329 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:36.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:36.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient(hunting): my global_id is 86399 2024-01-24T23:29:36.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e38003530 con 0x7f7e541343b0 2024-01-24T23:29:36.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4221245244 0 0) 0x7f7e38003530 con 0x7f7e541343b0 2024-01-24T23:29:36.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7e54120760 con 0x7f7e541343b0 2024-01-24T23:29:36.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e4c001e20 con 0x7f7e541343b0 2024-01-24T23:29:36.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:29:36.330 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient(hunting): dump: 2024-01-24T23:29:36.330 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.330 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.330 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.330 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.330 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.331 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.331 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.331 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.331 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.331 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.331 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.331 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.331 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.331 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.331 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.331 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:36.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2797123291 0 0) 0x7f7e5409c810 con 0x7f7e54133fd0 2024-01-24T23:29:36.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:36.331 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:36.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:36.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient(hunting): my global_id is 7149 2024-01-24T23:29:36.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e38003530 con 0x7f7e54133fd0 2024-01-24T23:29:36.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 595490460 0 0) 0x7f7e54120760 con 0x7f7e541343b0 2024-01-24T23:29:36.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 >> v1:172.21.15.132:6791/0 conn(0x7f7e54133bf0 legacy=0x7f7e54132da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 >> v1:172.21.15.132:6790/0 conn(0x7f7e54133fd0 legacy=0x7f7e541d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:29:36.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 1 monclient: found mon.e 2024-01-24T23:29:36.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:36.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:29:36.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5411fec0 con 0x7f7e541343b0 2024-01-24T23:29:36.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient: _finish_auth 0 2024-01-24T23:29:36.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:29:36.332 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:36.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5c035640 5 monclient: authenticate success, global_id 86399 2024-01-24T23:29:36.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5c035640 10 monclient: _renew_subs 2024-01-24T23:29:36.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5c035640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:29:36.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5c035640 1 -- 172.21.15.115:0/2572399075 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e541201a0 con 0x7f7e541343b0 2024-01-24T23:29:36.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5c035640 10 monclient: _renew_subs 2024-01-24T23:29:36.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5c035640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:29:36.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5c035640 1 -- 172.21.15.115:0/2572399075 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e540a3150 con 0x7f7e541343b0 2024-01-24T23:29:36.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5c035640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:36.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e5c035640 1 -- 172.21.15.115:0/2572399075 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7e5406a0d0 con 0x7f7e541c8840 2024-01-24T23:29:36.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7e4c003cd0 con 0x7f7e541343b0 2024-01-24T23:29:36.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:36.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e4c004200 con 0x7f7e541343b0 2024-01-24T23:29:36.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.333 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:29:36.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4bfff640 4 set_mon_vals no callback set 2024-01-24T23:29:36.334 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient: dump: 2024-01-24T23:29:36.334 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.334 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.334 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.334 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.334 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.334 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.334 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.334 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.334 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.334 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.334 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.334 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.335 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.335 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.335 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.335 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f7e4c004360 con 0x7f7e541343b0 2024-01-24T23:29:36.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7f7e4c0447f0 con 0x7f7e541343b0 2024-01-24T23:29:36.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1185911006 0 0) 0x7f7e5406a0d0 con 0x7f7e541c8840 2024-01-24T23:29:36.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:36.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:36.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:36.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient(hunting): my global_id is 154233 2024-01-24T23:29:36.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e54120760 con 0x7f7e541c8840 2024-01-24T23:29:36.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e595d0640 10 monclient: get_auth_request con 0x7f7e38036f90 auth_method 0 2024-01-24T23:29:36.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4029875342 0 0) 0x7f7e54120760 con 0x7f7e541c8840 2024-01-24T23:29:36.335 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7e5406a0d0 con 0x7f7e541c8840 2024-01-24T23:29:36.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e50001560 con 0x7f7e541c8840 2024-01-24T23:29:36.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:29:36.336 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 10 monclient: dump: 2024-01-24T23:29:36.336 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.336 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.336 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.336 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.336 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.336 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.336 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.336 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.336 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.336 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.337 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.337 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.337 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.337 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.337 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.337 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2394888661 0 0) 0x7f7e5406a0d0 con 0x7f7e541c8840 2024-01-24T23:29:36.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.315+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 --> v1:172.21.15.115:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7e540a57c0 con 0x7f7e541c8840 2024-01-24T23:29:36.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.319+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 <== mon.0 v1:172.21.15.115:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f7e50004b20 con 0x7f7e541c8840 2024-01-24T23:29:36.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.319+0000 7f7e4a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:36.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.319+0000 7f7e4a7fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:36.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.319+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 >> v1:172.21.15.115:6789/0 conn(0x7f7e541c8840 legacy=0x7f7e54061790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.337 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.319+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 >> v1:172.21.15.115:6789/0 conn(0x7f7e541c8840 legacy=0x7f7e54061790 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:36.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e5c035640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:29:36.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e5c035640 1 -- 172.21.15.115:0/2572399075 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7e5402cd40 con 0x7f7e5402c960 2024-01-24T23:29:36.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1642243156 0 0) 0x7f7e5402cd40 con 0x7f7e5402c960 2024-01-24T23:29:36.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e4a7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:36.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e4a7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:36.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e4a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:36.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e4a7fc640 10 monclient(hunting): my global_id is 154242 2024-01-24T23:29:36.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e5406a0d0 con 0x7f7e5402c960 2024-01-24T23:29:36.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1583783055 0 0) 0x7f7e5406a0d0 con 0x7f7e5402c960 2024-01-24T23:29:36.339 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7e5402cd40 con 0x7f7e5402c960 2024-01-24T23:29:36.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f7e500051c0 con 0x7f7e5402c960 2024-01-24T23:29:36.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e4a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e4a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:29:36.340 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e4a7fc640 10 monclient: dump: 2024-01-24T23:29:36.340 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.340 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.340 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.340 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.340 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.340 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.340 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.340 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.340 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.341 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.341 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.341 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.341 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.341 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.341 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.341 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3823675802 0 0) 0x7f7e5402cd40 con 0x7f7e5402c960 2024-01-24T23:29:36.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 --> v1:172.21.15.115:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7e541336f0 con 0x7f7e5402c960 2024-01-24T23:29:36.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 <== mon.0 v1:172.21.15.115:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 708748384) 0x7f7e500073c0 con 0x7f7e5402c960 2024-01-24T23:29:36.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e4a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:29:36.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e4a7fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:36.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 >> v1:172.21.15.115:6789/0 conn(0x7f7e5402c960 legacy=0x7f7e5409e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.341 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e4a7fc640 1 -- 172.21.15.115:0/2572399075 >> v1:172.21.15.115:6789/0 conn(0x7f7e5402c960 legacy=0x7f7e5409e010 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:36.342 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:29:36.342 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:36.342 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:36.342 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:29:36.342 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 48, 2024-01-24T23:29:36.342 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:29:36.342 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:29:36.342 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:29:36.342 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:29:36.342 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:29:36.342 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:29:36.343 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:29:36.343 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:29:36.343 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:29:36.343 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:29:36.343 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:36.343 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 6, 2024-01-24T23:29:36.343 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:36.343 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:29:36.343 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:29:36.343 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:36.343 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:36.343 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:36.343 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:36.343 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:36.344 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:36.344 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:36.344 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:36.344 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:36.344 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:36.344 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:36.344 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:36.344 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:29:36.344 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:29:36.344 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:36.344 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:36.344 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:36.344 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:36.344 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:36.345 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:36.345 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:36.345 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:36.345 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:36.345 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:36.345 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:36.345 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.345 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:29:36.345 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:29:36.345 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:29:36.345 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:29:36.345 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:29:36.346 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:29:36.346 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:36.346 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:36.346 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:29:36.346 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:29:36.346 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:29:36.346 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:29:36.346 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:29:36.346 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:29:36.346 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:29:36.346 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:36.346 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:29:36.346 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:36.346 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:36.347 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:36.347 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:36.347 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:36.347 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:36.347 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:36.347 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:36.347 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:36.347 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:36.347 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:36.347 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:36.347 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:29:36.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.347 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:29:36.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.348 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:36.348 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:36.348 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.348 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.348 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.348 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:29:36.348 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.348 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.348 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:29:36.348 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:29:36.348 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.349 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.349 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.349 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:36.349 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:36.349 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.349 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.349 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.349 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:29:36.349 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.349 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.349 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.350 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:29:36.350 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:29:36.350 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.350 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.350 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.350 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:36.350 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:36.350 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.350 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.350 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.351 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:29:36.351 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.351 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.351 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.351 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:29:36.351 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:29:36.351 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.351 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.351 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.351 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:36.351 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:36.352 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.352 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.352 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.352 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:29:36.352 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.352 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.352 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.352 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:29:36.352 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:29:36.352 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.352 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.352 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.353 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:36.353 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:36.353 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.353 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.353 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.353 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:29:36.353 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.353 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.353 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.353 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:29:36.354 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:29:36.354 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.354 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.354 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.354 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:36.354 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:36.354 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.354 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.354 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.354 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:29:36.354 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.355 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.355 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:29:36.355 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:29:36.355 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.355 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.355 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.355 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:36.355 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:36.355 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.355 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.356 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.356 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:29:36.356 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.356 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.356 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.356 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:29:36.356 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:29:36.356 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.356 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.356 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.357 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:36.357 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:36.357 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.357 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.357 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.357 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:29:36.357 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.357 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.357 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.357 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:29:36.357 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:29:36.357 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.358 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.358 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.358 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:36.358 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:36.358 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.358 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.358 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.358 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:29:36.358 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.358 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.358 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.359 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:29:36.359 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:29:36.359 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.359 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.359 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.359 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.359 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:29:36.359 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:29:36.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.359 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:36.359 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:36.359 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:36.360 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.360 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:36.360 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:29:36.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.360 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:36.360 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:36.360 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:36.360 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.360 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.360 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:29:36.360 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:29:36.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e5c035640 1 -- 172.21.15.115:0/2572399075 >> v1:172.21.15.132:6812/1835401922 conn(0x7f7e38036f90 legacy=0x7f7e38039450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e5c035640 10 monclient: shutdown 2024-01-24T23:29:36.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e5c035640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:36.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e5c035640 1 -- 172.21.15.115:0/2572399075 >> v1:172.21.15.115:6793/0 conn(0x7f7e541343b0 legacy=0x7f7e541d3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e5a5d2640 1 -- 172.21.15.115:0/2572399075 reap_dead start 2024-01-24T23:29:36.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e5a5d2640 1 -- 172.21.15.115:0/2572399075 reap_dead start 2024-01-24T23:29:36.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e5c035640 1 -- 172.21.15.115:0/2572399075 shutdown_connections 2024-01-24T23:29:36.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e5c035640 1 -- 172.21.15.115:0/2572399075 >> 172.21.15.115:0/2572399075 conn(0x7f7e540662f0 msgr2=0x7f7e540a7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:36.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e5c035640 1 -- 172.21.15.115:0/2572399075 shutdown_connections 2024-01-24T23:29:36.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.323+0000 7f7e5c035640 1 -- 172.21.15.115:0/2572399075 wait complete. 2024-01-24T23:29:36.363 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:29:36.363 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-01-24T23:29:36.363 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-01-24T23:29:36.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.471+0000 7f8198179640 10 monclient: get_monmap_and_config 2024-01-24T23:29:36.475 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.471+0000 7f8198179640 10 monclient: build_initial_monmap 2024-01-24T23:29:36.476 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.471+0000 7f8198179640 10 monclient: monmap: 2024-01-24T23:29:36.476 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:36.476 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.476 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:36.478376+0000 2024-01-24T23:29:36.476 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:36.478376+0000 2024-01-24T23:29:36.476 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:36.476 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.476 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:36.476 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:36.477 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 1 Processor -- start 2024-01-24T23:29:36.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 1 -- start start 2024-01-24T23:29:36.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 10 monclient: init 2024-01-24T23:29:36.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:36.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 10 monclient: _add_conns ranks=[0,7,2,8,6,5,1,4,3] 2024-01-24T23:29:36.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 10 monclient(hunting): picked mon.noname-a con 0x7f8190133bf0 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:36.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 10 monclient(hunting): picked mon.noname-i con 0x7f8190133fd0 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:36.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 10 monclient(hunting): picked mon.noname-b con 0x7f8190134430 addr v1:172.21.15.115:6790/0 2024-01-24T23:29:36.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f819004ed70 con 0x7f8190133bf0 2024-01-24T23:29:36.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f819009af30 con 0x7f8190134430 2024-01-24T23:29:36.485 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f819009b210 con 0x7f8190133fd0 2024-01-24T23:29:36.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:36.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 10 monclient(hunting): authenticate will time out at 1455.400802s 2024-01-24T23:29:36.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8196716640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f8190134430 0x7f819013f500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:43078/0 (socket says 172.21.15.115:43078) 2024-01-24T23:29:36.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8196716640 1 -- 172.21.15.115:0/3135539677 learned_addr learned my addr 172.21.15.115:0/3135539677 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:36.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8195f15640 1 --1- 172.21.15.115:0/3135539677 >> v1:172.21.15.115:6789/0 conn(0x7f8190133bf0 0x7f819009bd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39236/0 (socket says 172.21.15.115:39236) 2024-01-24T23:29:36.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 1 -- 172.21.15.115:0/3135539677 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 343847243 0 0) 0x7f819009af30 con 0x7f8190134430 2024-01-24T23:29:36.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 10 monclient(hunting): _negotiate 2024-01-24T23:29:36.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:36.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:36.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 10 monclient(hunting): my global_id is 154154 2024-01-24T23:29:36.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 1 -- 172.21.15.115:0/3135539677 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f817c003530 con 0x7f8190134430 2024-01-24T23:29:36.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 1 -- 172.21.15.115:0/3135539677 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1281250749 0 0) 0x7f819004ed70 con 0x7f8190133bf0 2024-01-24T23:29:36.486 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 10 monclient(hunting): _negotiate 2024-01-24T23:29:36.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:36.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:36.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 10 monclient(hunting): my global_id is 154251 2024-01-24T23:29:36.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 1 -- 172.21.15.115:0/3135539677 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f819009af30 con 0x7f8190133bf0 2024-01-24T23:29:36.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 1 -- 172.21.15.115:0/3135539677 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2195456316 0 0) 0x7f819009af30 con 0x7f8190133bf0 2024-01-24T23:29:36.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 1 -- 172.21.15.115:0/3135539677 >> v1:172.21.15.132:6792/0 conn(0x7f8190133fd0 legacy=0x7f819013bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 1 -- 172.21.15.115:0/3135539677 >> v1:172.21.15.115:6790/0 conn(0x7f8190134430 legacy=0x7f819013f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 10 monclient: _finish_hunting 0 2024-01-24T23:29:36.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 1 monclient: found mon.noname-a 2024-01-24T23:29:36.487 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:36.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.115:6789/0 2024-01-24T23:29:36.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 1 -- 172.21.15.115:0/3135539677 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819009b4f0 con 0x7f8190133bf0 2024-01-24T23:29:36.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 10 monclient: _finish_auth 0 2024-01-24T23:29:36.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 10 monclient: _check_auth_tickets 2024-01-24T23:29:36.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:36.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 1 -- 172.21.15.115:0/3135539677 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f818c0017c0 con 0x7f8190133bf0 2024-01-24T23:29:36.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 5 monclient: authenticate success, global_id 154251 2024-01-24T23:29:36.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:36.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:29:36.488 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 10 monclient: dump: 2024-01-24T23:29:36.488 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.488 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.488 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.488 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.489 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.489 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.489 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.489 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.489 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.489 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.489 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.489 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.489 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.489 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.489 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.489 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:36.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 1 -- 172.21.15.115:0/3135539677 <== mon.0 v1:172.21.15.115:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f818c001970 con 0x7f8190133bf0 2024-01-24T23:29:36.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:36.489 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:36.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 4 set_mon_vals no callback set 2024-01-24T23:29:36.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 1 -- 172.21.15.115:0/3135539677 <== mon.0 v1:172.21.15.115:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f818c0047f0 con 0x7f8190133bf0 2024-01-24T23:29:36.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:29:36.490 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8194f13640 10 monclient: dump: 2024-01-24T23:29:36.490 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.490 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.490 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.490 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.490 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.490 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.490 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.490 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.490 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.490 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.491 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.491 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.491 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.491 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.491 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.491 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 10 monclient: shutdown 2024-01-24T23:29:36.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:36.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 1 -- 172.21.15.115:0/3135539677 >> v1:172.21.15.115:6789/0 conn(0x7f8190133bf0 legacy=0x7f819009bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 1 -- 172.21.15.115:0/3135539677 shutdown_connections 2024-01-24T23:29:36.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 1 -- 172.21.15.115:0/3135539677 >> 172.21.15.115:0/3135539677 conn(0x7f81900a9910 msgr2=0x7f81900a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:36.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 1 -- 172.21.15.115:0/3135539677 shutdown_connections 2024-01-24T23:29:36.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 1 -- 172.21.15.115:0/3135539677 wait complete. 2024-01-24T23:29:36.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 10 monclient: build_initial_monmap 2024-01-24T23:29:36.491 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 10 monclient: monmap: 2024-01-24T23:29:36.492 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:36.492 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:36.492 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:36.492 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:36.492 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:36.492 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:36.492 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:36.492 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:36.492 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:36.492 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:36.492 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:36.492 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:36.492 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:36.492 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:36.492 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:36.493 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 1 Processor -- start 2024-01-24T23:29:36.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 1 -- start start 2024-01-24T23:29:36.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 10 monclient: init 2024-01-24T23:29:36.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:36.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.475+0000 7f8198179640 10 monclient: _add_conns ranks=[4,6,8,0,2,5,1,3,7] 2024-01-24T23:29:36.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f8198179640 10 monclient(hunting): picked mon.noname-c con 0x7f8190133bf0 addr v1:172.21.15.115:6791/0 2024-01-24T23:29:36.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f8198179640 10 monclient(hunting): picked mon.noname-d con 0x7f8190133fd0 addr v1:172.21.15.115:6792/0 2024-01-24T23:29:36.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f8198179640 10 monclient(hunting): picked mon.noname-e con 0x7f8190134430 addr v1:172.21.15.115:6793/0 2024-01-24T23:29:36.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f8198179640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f819009c8f0 con 0x7f8190133bf0 2024-01-24T23:29:36.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f8198179640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81900a5960 con 0x7f8190133fd0 2024-01-24T23:29:36.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f8198179640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8190139560 con 0x7f8190134430 2024-01-24T23:29:36.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f8198179640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:36.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f8198179640 10 monclient(hunting): authenticate will time out at 1455.404096s 2024-01-24T23:29:36.493 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f8195f15640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f8190133bf0 0x7f81901cb4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:59006/0 (socket says 172.21.15.115:59006) 2024-01-24T23:29:36.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f8195f15640 1 -- 172.21.15.115:0/556869637 learned_addr learned my addr 172.21.15.115:0/556869637 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:36.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 530198746 0 0) 0x7f8190139560 con 0x7f8190134430 2024-01-24T23:29:36.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:36.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:36.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:36.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): my global_id is 86408 2024-01-24T23:29:36.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8174003530 con 0x7f8190134430 2024-01-24T23:29:36.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1790196688 0 0) 0x7f819009c8f0 con 0x7f8190133bf0 2024-01-24T23:29:36.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:36.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:36.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:36.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): my global_id is 95314 2024-01-24T23:29:36.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8190139560 con 0x7f8190133bf0 2024-01-24T23:29:36.494 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605447393 0 0) 0x7f81900a5960 con 0x7f8190133fd0 2024-01-24T23:29:36.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:36.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:36.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:36.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): my global_id is 7341 2024-01-24T23:29:36.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f819009c8f0 con 0x7f8190133fd0 2024-01-24T23:29:36.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 264496527 0 0) 0x7f8174003530 con 0x7f8190134430 2024-01-24T23:29:36.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f81900a5960 con 0x7f8190134430 2024-01-24T23:29:36.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8188001c40 con 0x7f8190134430 2024-01-24T23:29:36.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:29:36.495 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): dump: 2024-01-24T23:29:36.495 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.495 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.495 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.495 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.495 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.496 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.496 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.496 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.496 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.496 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.496 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.496 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.496 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.496 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.496 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.496 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:36.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3282037625 0 0) 0x7f819009c8f0 con 0x7f8190133fd0 2024-01-24T23:29:36.496 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8174003530 con 0x7f8190133fd0 2024-01-24T23:29:36.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 409273863 0 0) 0x7f8190139560 con 0x7f8190133bf0 2024-01-24T23:29:36.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f819009c8f0 con 0x7f8190133bf0 2024-01-24T23:29:36.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8180001960 con 0x7f8190133fd0 2024-01-24T23:29:36.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-01-24T23:29:36.497 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): dump: 2024-01-24T23:29:36.497 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.497 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.497 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.498 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.498 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.498 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.498 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.498 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.498 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.498 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.498 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.498 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.498 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.498 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.498 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.498 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f818c0016c0 con 0x7f8190133bf0 2024-01-24T23:29:36.498 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-01-24T23:29:36.499 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): dump: 2024-01-24T23:29:36.499 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.499 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.499 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.499 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.499 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.499 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.499 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.499 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.499 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.499 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.499 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.499 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.499 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.500 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1815324263 0 0) 0x7f81900a5960 con 0x7f8190134430 2024-01-24T23:29:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 >> v1:172.21.15.115:6792/0 conn(0x7f8190133fd0 legacy=0x7f81901d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 >> v1:172.21.15.115:6791/0 conn(0x7f8190133bf0 legacy=0x7f81901cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient: _finish_hunting 0 2024-01-24T23:29:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 monclient: found mon.e 2024-01-24T23:29:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:29:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819013aa80 con 0x7f8190134430 2024-01-24T23:29:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient: _finish_auth 0 2024-01-24T23:29:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient: _check_auth_tickets 2024-01-24T23:29:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:36.500 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f8198179640 5 monclient: authenticate success, global_id 86408 2024-01-24T23:29:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f8198179640 10 monclient: _renew_subs 2024-01-24T23:29:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f8198179640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:29:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f8198179640 1 -- 172.21.15.115:0/556869637 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81901392e0 con 0x7f8190134430 2024-01-24T23:29:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f8198179640 10 monclient: _renew_subs 2024-01-24T23:29:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f8198179640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:29:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f8198179640 1 -- 172.21.15.115:0/556869637 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f819009f3f0 con 0x7f8190134430 2024-01-24T23:29:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8188004b70 con 0x7f8190134430 2024-01-24T23:29:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8188005020 con 0x7f8190134430 2024-01-24T23:29:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f8187fff640 4 set_mon_vals no callback set 2024-01-24T23:29:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:29:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient: dump: 2024-01-24T23:29:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.501 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.502 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f8198179640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f8198179640 1 -- 172.21.15.115:0/556869637 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f81780065d0 con 0x7f8178002870 2024-01-24T23:29:36.502 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f818800faa0 con 0x7f8190134430 2024-01-24T23:29:36.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7f81880445d0 con 0x7f8190134430 2024-01-24T23:29:36.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3926550133 0 0) 0x7f81780065d0 con 0x7f8178002870 2024-01-24T23:29:36.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:36.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:36.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:36.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient(hunting): my global_id is 154163 2024-01-24T23:29:36.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81900a5960 con 0x7f8178002870 2024-01-24T23:29:36.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3139482904 0 0) 0x7f81900a5960 con 0x7f8178002870 2024-01-24T23:29:36.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f81780065d0 con 0x7f8178002870 2024-01-24T23:29:36.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f818c004880 con 0x7f8178002870 2024-01-24T23:29:36.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:29:36.503 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient: dump: 2024-01-24T23:29:36.503 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.504 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.504 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.504 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.504 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.504 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.504 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.504 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.504 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.504 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.504 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.504 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.504 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.504 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.504 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.504 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f8195714640 10 monclient: get_auth_request con 0x7f8174037db0 auth_method 0 2024-01-24T23:29:36.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 235114497 0 0) 0x7f81780065d0 con 0x7f8178002870 2024-01-24T23:29:36.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 --> v1:172.21.15.115:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f817800a120 con 0x7f8178002870 2024-01-24T23:29:36.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 <== mon.2 v1:172.21.15.115:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f818c0051f0 con 0x7f8178002870 2024-01-24T23:29:36.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:36.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:36.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 >> v1:172.21.15.115:6790/0 conn(0x7f8178002870 legacy=0x7f8178004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.505 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.479+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 >> v1:172.21.15.115:6790/0 conn(0x7f8178002870 legacy=0x7f8178004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:36.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f8198179640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:29:36.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f8198179640 1 -- 172.21.15.115:0/556869637 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8178009e10 con 0x7f81780090d0 2024-01-24T23:29:36.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1401640896 0 0) 0x7f8178009e10 con 0x7f81780090d0 2024-01-24T23:29:36.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f81867fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:36.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f81867fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:36.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f81867fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:36.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f81867fc640 10 monclient(hunting): my global_id is 154172 2024-01-24T23:29:36.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81780065d0 con 0x7f81780090d0 2024-01-24T23:29:36.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1583575943 0 0) 0x7f81780065d0 con 0x7f81780090d0 2024-01-24T23:29:36.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8178009e10 con 0x7f81780090d0 2024-01-24T23:29:36.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f818c0059f0 con 0x7f81780090d0 2024-01-24T23:29:36.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f81867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.507 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f81867fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:29:36.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f81867fc640 10 monclient: dump: 2024-01-24T23:29:36.508 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.508 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.508 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.508 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.508 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.508 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.508 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.508 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.508 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.508 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.508 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.508 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.508 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.509 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 75979940 0 0) 0x7f8178009e10 con 0x7f81780090d0 2024-01-24T23:29:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 --> v1:172.21.15.115:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f817800c310 con 0x7f81780090d0 2024-01-24T23:29:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 <== mon.2 v1:172.21.15.115:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 1773102070) 0x7f818c007eb0 con 0x7f81780090d0 2024-01-24T23:29:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f81867fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:29:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f81867fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 >> v1:172.21.15.115:6790/0 conn(0x7f81780090d0 legacy=0x7f8178009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f81867fc640 1 -- 172.21.15.115:0/556869637 >> v1:172.21.15.115:6790/0 conn(0x7f81780090d0 legacy=0x7f8178009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:36.510 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:29:36.510 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:36.510 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:36.510 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:29:36.510 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 48, 2024-01-24T23:29:36.510 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:29:36.510 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:29:36.510 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:29:36.510 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:29:36.510 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:29:36.510 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:29:36.510 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:29:36.511 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:29:36.511 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:29:36.511 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:29:36.511 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:36.511 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 6, 2024-01-24T23:29:36.511 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:36.511 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:29:36.511 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:29:36.511 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:36.511 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:36.511 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:36.511 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:36.511 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:36.511 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:29:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:29:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:29:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:29:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:29:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:29:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:29:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:29:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:29:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:29:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:29:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:29:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:29:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:29:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:29:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:29:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:29:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:29:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:29:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:29:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:29:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:29:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:29:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:29:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:29:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:29:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:29:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:29:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:29:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:29:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:29:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:29:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:29:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:29:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:29:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:29:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.524 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:29:36.524 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.524 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.524 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.524 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:29:36.524 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:29:36.524 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.524 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.524 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.524 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:29:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:29:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:29:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:29:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:29:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:29:36.527 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.527 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.527 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.527 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.527 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.527 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:29:36.527 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:29:36.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.527 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:36.527 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:36.527 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:36.527 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.527 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:36.527 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:29:36.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.528 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:36.528 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:36.528 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:36.528 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.528 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.528 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:29:36.528 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:29:36.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f8198179640 1 -- 172.21.15.115:0/556869637 >> v1:172.21.15.132:6812/1835401922 conn(0x7f8174037db0 legacy=0x7f817403a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f8198179640 10 monclient: shutdown 2024-01-24T23:29:36.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f8198179640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:36.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f8198179640 1 -- 172.21.15.115:0/556869637 >> v1:172.21.15.115:6793/0 conn(0x7f8190134430 legacy=0x7f81901d3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f8196716640 1 -- 172.21.15.115:0/556869637 reap_dead start 2024-01-24T23:29:36.529 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f8196716640 1 -- 172.21.15.115:0/556869637 reap_dead start 2024-01-24T23:29:36.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f8198179640 1 -- 172.21.15.115:0/556869637 shutdown_connections 2024-01-24T23:29:36.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f8198179640 1 -- 172.21.15.115:0/556869637 >> 172.21.15.115:0/556869637 conn(0x7f81900a9910 msgr2=0x7f819013fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:36.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f8198179640 1 -- 172.21.15.115:0/556869637 shutdown_connections 2024-01-24T23:29:36.530 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.487+0000 7f8198179640 1 -- 172.21.15.115:0/556869637 wait complete. 2024-01-24T23:29:36.530 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:29:36.530 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-01-24T23:29:36.530 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-01-24T23:29:36.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.643+0000 7fa637852640 10 monclient: get_monmap_and_config 2024-01-24T23:29:36.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.643+0000 7fa637852640 10 monclient: build_initial_monmap 2024-01-24T23:29:36.647 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.643+0000 7fa637852640 10 monclient: monmap: 2024-01-24T23:29:36.647 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:36.647 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.647 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:36.649375+0000 2024-01-24T23:29:36.647 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:36.649375+0000 2024-01-24T23:29:36.647 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:36.647 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.647 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:36.647 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:36.647 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:36.647 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:36.648 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:36.648 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:36.648 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:36.648 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:36.648 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:36.648 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.655 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.643+0000 7fa637852640 1 Processor -- start 2024-01-24T23:29:36.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.643+0000 7fa637852640 1 -- start start 2024-01-24T23:29:36.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.643+0000 7fa637852640 10 monclient: init 2024-01-24T23:29:36.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:36.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 10 monclient: _add_conns ranks=[2,7,3,5,4,8,0,6,1] 2024-01-24T23:29:36.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 10 monclient(hunting): picked mon.noname-b con 0x7fa630134740 addr v1:172.21.15.115:6790/0 2024-01-24T23:29:36.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 10 monclient(hunting): picked mon.noname-i con 0x7fa63009ec80 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:36.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 10 monclient(hunting): picked mon.noname-g con 0x7fa630135230 addr v1:172.21.15.132:6790/0 2024-01-24T23:29:36.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa63004ed70 con 0x7fa630134740 2024-01-24T23:29:36.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6300a0d90 con 0x7fa630135230 2024-01-24T23:29:36.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa63009e7c0 con 0x7fa63009ec80 2024-01-24T23:29:36.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:36.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 10 monclient(hunting): authenticate will time out at 1455.572512s 2024-01-24T23:29:36.656 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa6355ee640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fa630134740 0x7fa630134b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:43108/0 (socket says 172.21.15.115:43108) 2024-01-24T23:29:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa6355ee640 1 -- 172.21.15.115:0/205766681 learned_addr learned my addr 172.21.15.115:0/205766681 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 1 -- 172.21.15.115:0/205766681 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2075137316 0 0) 0x7fa63004ed70 con 0x7fa630134740 2024-01-24T23:29:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 10 monclient(hunting): my global_id is 154181 2024-01-24T23:29:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 1 -- 172.21.15.115:0/205766681 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa61c003530 con 0x7fa630134740 2024-01-24T23:29:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 1 -- 172.21.15.115:0/205766681 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3393456525 0 0) 0x7fa61c003530 con 0x7fa630134740 2024-01-24T23:29:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 1 -- 172.21.15.115:0/205766681 >> v1:172.21.15.132:6792/0 conn(0x7fa63009ec80 legacy=0x7fa630139d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 1 -- 172.21.15.115:0/205766681 >> v1:172.21.15.132:6790/0 conn(0x7fa630135230 legacy=0x7fa63013d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 10 monclient: _finish_hunting 0 2024-01-24T23:29:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 1 monclient: found mon.noname-b 2024-01-24T23:29:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:36.657 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:29:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 1 -- 172.21.15.115:0/205766681 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6300a1070 con 0x7fa630134740 2024-01-24T23:29:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 10 monclient: _finish_auth 0 2024-01-24T23:29:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 10 monclient: _check_auth_tickets 2024-01-24T23:29:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 1 -- 172.21.15.115:0/205766681 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa62c001b40 con 0x7fa630134740 2024-01-24T23:29:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 5 monclient: authenticate success, global_id 154181 2024-01-24T23:29:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:29:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 10 monclient: dump: 2024-01-24T23:29:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.658 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.659 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:36.659 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 1 -- 172.21.15.115:0/205766681 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa62c001df0 con 0x7fa630134740 2024-01-24T23:29:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 1 -- 172.21.15.115:0/205766681 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa62c004f70 con 0x7fa630134740 2024-01-24T23:29:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 4 set_mon_vals no callback set 2024-01-24T23:29:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:29:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa627fff640 10 monclient: dump: 2024-01-24T23:29:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.660 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.661 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.661 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.661 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.661 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.661 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.661 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.662 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.662 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.662 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.662 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.662 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.662 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.662 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 10 monclient: shutdown 2024-01-24T23:29:36.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:36.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 1 -- 172.21.15.115:0/205766681 >> v1:172.21.15.115:6790/0 conn(0x7fa630134740 legacy=0x7fa630134b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 1 -- 172.21.15.115:0/205766681 shutdown_connections 2024-01-24T23:29:36.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 1 -- 172.21.15.115:0/205766681 >> 172.21.15.115:0/205766681 conn(0x7fa6300a5730 msgr2=0x7fa6300a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:36.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 1 -- 172.21.15.115:0/205766681 shutdown_connections 2024-01-24T23:29:36.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 1 -- 172.21.15.115:0/205766681 wait complete. 2024-01-24T23:29:36.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 10 monclient: build_initial_monmap 2024-01-24T23:29:36.663 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 10 monclient: monmap: 2024-01-24T23:29:36.663 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:36.663 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:36.664 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:36.664 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:36.664 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:36.664 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:36.664 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:36.664 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:36.664 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:36.664 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:36.664 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:36.664 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:36.664 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:36.664 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:36.664 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:36.664 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 1 Processor -- start 2024-01-24T23:29:36.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 1 -- start start 2024-01-24T23:29:36.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 10 monclient: init 2024-01-24T23:29:36.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:36.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 10 monclient: _add_conns ranks=[0,1,2,5,6,4,3,7,8] 2024-01-24T23:29:36.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 10 monclient(hunting): picked mon.noname-a con 0x7fa63009ec80 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:36.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 10 monclient(hunting): picked mon.noname-f con 0x7fa630134740 addr v1:172.21.15.132:6789/0 2024-01-24T23:29:36.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 10 monclient(hunting): picked mon.noname-b con 0x7fa630135230 addr v1:172.21.15.115:6790/0 2024-01-24T23:29:36.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa630137480 con 0x7fa63009ec80 2024-01-24T23:29:36.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa630138c20 con 0x7fa630134740 2024-01-24T23:29:36.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa630136650 con 0x7fa630135230 2024-01-24T23:29:36.665 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:36.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa637852640 10 monclient(hunting): authenticate will time out at 1455.575443s 2024-01-24T23:29:36.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa6355ee640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fa63009ec80 0x7fa630135830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39240/0 (socket says 172.21.15.115:39240) 2024-01-24T23:29:36.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa6355ee640 1 -- 172.21.15.115:0/3994681901 learned_addr learned my addr 172.21.15.115:0/3994681901 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:36.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3736685181 0 0) 0x7fa630136650 con 0x7fa630135230 2024-01-24T23:29:36.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa625ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:29:36.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa625ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:36.666 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa625ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:36.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa625ffb640 10 monclient(hunting): my global_id is 154190 2024-01-24T23:29:36.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa614003530 con 0x7fa630135230 2024-01-24T23:29:36.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3508570678 0 0) 0x7fa6300a1070 con 0x7fa63009ec80 2024-01-24T23:29:36.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa625ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:29:36.667 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa625ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:36.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa625ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:36.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa625ffb640 10 monclient(hunting): my global_id is 154260 2024-01-24T23:29:36.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa630136650 con 0x7fa63009ec80 2024-01-24T23:29:36.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.647+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 307288603 0 0) 0x7fa614003530 con 0x7fa630135230 2024-01-24T23:29:36.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa6300a1070 con 0x7fa630135230 2024-01-24T23:29:36.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa628001900 con 0x7fa630135230 2024-01-24T23:29:36.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:29:36.668 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 10 monclient(hunting): dump: 2024-01-24T23:29:36.668 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.668 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.668 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.669 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.669 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.669 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.669 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.669 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.669 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.669 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.669 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.669 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.669 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.669 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.670 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.670 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:36.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2273987421 0 0) 0x7fa630136650 con 0x7fa63009ec80 2024-01-24T23:29:36.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa614003530 con 0x7fa63009ec80 2024-01-24T23:29:36.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa62c004eb0 con 0x7fa63009ec80 2024-01-24T23:29:36.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-01-24T23:29:36.670 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 10 monclient(hunting): dump: 2024-01-24T23:29:36.670 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.670 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.671 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.671 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.671 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.671 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.671 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.671 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.671 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.671 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.672 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.672 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.672 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.672 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.672 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.672 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3886383668 0 0) 0x7fa6300a1070 con 0x7fa630135230 2024-01-24T23:29:36.672 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 >> v1:172.21.15.132:6789/0 conn(0x7fa630134740 legacy=0x7fa630135f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 >> v1:172.21.15.115:6789/0 conn(0x7fa63009ec80 legacy=0x7fa630135830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:29:36.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 1 monclient: found mon.b 2024-01-24T23:29:36.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:36.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:29:36.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa630071c00 con 0x7fa630135230 2024-01-24T23:29:36.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 10 monclient: _finish_auth 0 2024-01-24T23:29:36.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:29:36.673 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:36.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa637852640 5 monclient: authenticate success, global_id 154190 2024-01-24T23:29:36.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa637852640 10 monclient: _renew_subs 2024-01-24T23:29:36.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa637852640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:29:36.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa637852640 1 -- 172.21.15.115:0/3994681901 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa630071710 con 0x7fa630135230 2024-01-24T23:29:36.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa637852640 10 monclient: _renew_subs 2024-01-24T23:29:36.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa637852640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:29:36.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa637852640 1 -- 172.21.15.115:0/3994681901 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa630137180 con 0x7fa630135230 2024-01-24T23:29:36.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa628001c50 con 0x7fa630135230 2024-01-24T23:29:36.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:36.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa628003bf0 con 0x7fa630135230 2024-01-24T23:29:36.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa6277fe640 4 set_mon_vals no callback set 2024-01-24T23:29:36.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:29:36.674 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 10 monclient: dump: 2024-01-24T23:29:36.674 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.675 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.675 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.675 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.675 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.675 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.675 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.675 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.675 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.675 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.675 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.675 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.675 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.675 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.675 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.675 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa637852640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:36.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa637852640 1 -- 172.21.15.115:0/3994681901 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa6100065d0 con 0x7fa610002870 2024-01-24T23:29:36.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fa62800e670 con 0x7fa630135230 2024-01-24T23:29:36.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7fa6280430d0 con 0x7fa630135230 2024-01-24T23:29:36.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 335172612 0 0) 0x7fa6100065d0 con 0x7fa610002870 2024-01-24T23:29:36.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:29:36.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:36.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:36.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 10 monclient(hunting): my global_id is 95323 2024-01-24T23:29:36.676 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6300a1070 con 0x7fa610002870 2024-01-24T23:29:36.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa634ded640 10 monclient: get_auth_request con 0x7fa6140376c0 auth_method 0 2024-01-24T23:29:36.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3968688762 0 0) 0x7fa6300a1070 con 0x7fa610002870 2024-01-24T23:29:36.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa6100065d0 con 0x7fa610002870 2024-01-24T23:29:36.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa62c001590 con 0x7fa610002870 2024-01-24T23:29:36.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:29:36.677 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 10 monclient: dump: 2024-01-24T23:29:36.677 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.677 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.677 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.677 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.677 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.677 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.677 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.678 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.678 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.678 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.678 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.678 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.678 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.678 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.678 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.678 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3558048776 0 0) 0x7fa6100065d0 con 0x7fa610002870 2024-01-24T23:29:36.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 --> v1:172.21.15.115:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa61000a120 con 0x7fa610002870 2024-01-24T23:29:36.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 <== mon.4 v1:172.21.15.115:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fa62c0077d0 con 0x7fa610002870 2024-01-24T23:29:36.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:36.678 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:36.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 >> v1:172.21.15.115:6791/0 conn(0x7fa610002870 legacy=0x7fa610004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.679 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.651+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 >> v1:172.21.15.115:6791/0 conn(0x7fa610002870 legacy=0x7fa610004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:36.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa637852640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:29:36.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa637852640 1 -- 172.21.15.115:0/3994681901 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa610009e10 con 0x7fa6100090d0 2024-01-24T23:29:36.680 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 105059539 0 0) 0x7fa610009e10 con 0x7fa6100090d0 2024-01-24T23:29:36.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa625ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:29:36.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa625ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:36.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa625ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:36.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa625ffb640 10 monclient(hunting): my global_id is 95332 2024-01-24T23:29:36.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6100065d0 con 0x7fa6100090d0 2024-01-24T23:29:36.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2739533204 0 0) 0x7fa6100065d0 con 0x7fa6100090d0 2024-01-24T23:29:36.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 --> v1:172.21.15.115:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa610009e10 con 0x7fa6100090d0 2024-01-24T23:29:36.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa62c005fc0 con 0x7fa6100090d0 2024-01-24T23:29:36.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa625ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa625ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:29:36.681 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa625ffb640 10 monclient: dump: 2024-01-24T23:29:36.681 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.681 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.682 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.682 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.682 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.682 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.682 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.682 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.682 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.682 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.682 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.682 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.682 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.682 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.682 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.682 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.682 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 <== mon.4 v1:172.21.15.115:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 464999382 0 0) 0x7fa610009e10 con 0x7fa6100090d0 2024-01-24T23:29:36.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 --> v1:172.21.15.115:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa61000c310 con 0x7fa6100090d0 2024-01-24T23:29:36.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 <== mon.4 v1:172.21.15.115:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 4129111653) 0x7fa62c0075b0 con 0x7fa6100090d0 2024-01-24T23:29:36.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa625ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:29:36.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa625ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:36.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 >> v1:172.21.15.115:6791/0 conn(0x7fa6100090d0 legacy=0x7fa610009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.683 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa625ffb640 1 -- 172.21.15.115:0/3994681901 >> v1:172.21.15.115:6791/0 conn(0x7fa6100090d0 legacy=0x7fa610009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:36.683 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:29:36.684 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:36.684 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:36.684 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:29:36.684 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 48, 2024-01-24T23:29:36.684 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:29:36.685 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:29:36.685 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:29:36.685 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:29:36.685 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:29:36.685 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:29:36.685 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:29:36.685 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:29:36.686 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:29:36.686 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:29:36.686 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:36.686 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 7, 2024-01-24T23:29:36.686 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:36.686 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:29:36.686 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:29:36.686 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:36.687 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:36.687 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:36.687 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:36.687 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:36.687 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:36.687 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:36.687 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:36.687 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:36.688 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:36.688 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:36.688 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:36.688 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:29:36.688 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:29:36.688 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:36.688 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:36.689 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:36.689 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:36.689 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:36.689 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:36.689 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:36.689 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:36.689 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:36.689 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:36.689 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:36.689 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.689 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:29:36.689 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:29:36.689 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:29:36.690 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:29:36.690 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:29:36.690 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:29:36.690 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:36.690 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:36.690 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:29:36.690 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:29:36.690 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:29:36.690 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:29:36.690 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:29:36.691 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:29:36.691 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:29:36.691 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:36.691 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:29:36.691 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:36.691 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:29:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:29:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:29:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:29:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:29:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:29:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:29:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:29:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:29:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:29:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:29:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:36.697 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:36.697 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.697 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.697 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.697 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:29:36.697 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.697 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.697 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.698 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:29:36.698 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:29:36.698 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.698 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.698 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.698 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:36.698 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:36.698 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.699 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.699 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.699 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:29:36.699 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.699 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.699 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.700 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:29:36.700 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:29:36.700 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.700 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.700 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.700 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:36.700 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:36.700 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.701 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.701 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.701 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:29:36.701 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.701 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.701 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.701 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:29:36.701 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:29:36.701 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.702 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.702 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.702 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:36.703 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:36.703 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.703 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.703 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.703 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:29:36.704 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.704 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.704 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.704 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:29:36.704 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:29:36.704 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.704 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:29:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.706 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.706 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:29:36.706 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:29:36.706 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.706 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.707 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.707 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:36.707 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:36.707 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.707 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.707 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.707 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:29:36.707 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.708 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.708 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.708 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:29:36.708 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:29:36.708 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.708 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.708 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.708 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:29:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:29:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:29:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:36.710 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:29:36.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.710 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:36.710 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:29:36.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.710 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:36.710 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:36.710 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:36.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.710 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.711 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:29:36.711 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:29:36.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa637852640 1 -- 172.21.15.115:0/3994681901 >> v1:172.21.15.132:6812/1835401922 conn(0x7fa6140376c0 legacy=0x7fa614039b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa637852640 10 monclient: shutdown 2024-01-24T23:29:36.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa637852640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:36.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa637852640 1 -- 172.21.15.115:0/3994681901 >> v1:172.21.15.115:6790/0 conn(0x7fa630135230 legacy=0x7fa6301d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa635def640 1 -- 172.21.15.115:0/3994681901 reap_dead start 2024-01-24T23:29:36.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa635def640 1 -- 172.21.15.115:0/3994681901 reap_dead start 2024-01-24T23:29:36.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa637852640 1 -- 172.21.15.115:0/3994681901 shutdown_connections 2024-01-24T23:29:36.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa637852640 1 -- 172.21.15.115:0/3994681901 >> 172.21.15.115:0/3994681901 conn(0x7fa6300a5730 msgr2=0x7fa6300a3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:36.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa637852640 1 -- 172.21.15.115:0/3994681901 shutdown_connections 2024-01-24T23:29:36.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.659+0000 7fa637852640 1 -- 172.21.15.115:0/3994681901 wait complete. 2024-01-24T23:29:36.713 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:29:36.713 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-01-24T23:29:36.714 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-01-24T23:29:36.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.827+0000 7fb007beb640 10 monclient: get_monmap_and_config 2024-01-24T23:29:36.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.827+0000 7fb007beb640 10 monclient: build_initial_monmap 2024-01-24T23:29:36.829 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.827+0000 7fb007beb640 10 monclient: monmap: 2024-01-24T23:29:36.829 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:36.829 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.829 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:36.831631+0000 2024-01-24T23:29:36.829 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:36.831631+0000 2024-01-24T23:29:36.829 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:36.829 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.829 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:36.829 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:36.829 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:36.830 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:36.830 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:36.830 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:36.830 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:36.830 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:36.830 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:36.830 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.827+0000 7fb007beb640 1 Processor -- start 2024-01-24T23:29:36.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.827+0000 7fb007beb640 1 -- start start 2024-01-24T23:29:36.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.827+0000 7fb007beb640 10 monclient: init 2024-01-24T23:29:36.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.827+0000 7fb007beb640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:36.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.827+0000 7fb007beb640 10 monclient: _add_conns ranks=[8,7,5,4,3,0,2,6,1] 2024-01-24T23:29:36.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.827+0000 7fb007beb640 10 monclient(hunting): picked mon.noname-e con 0x7fb000133bf0 addr v1:172.21.15.115:6793/0 2024-01-24T23:29:36.831 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.827+0000 7fb007beb640 10 monclient(hunting): picked mon.noname-i con 0x7fb000133fd0 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:36.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.827+0000 7fb007beb640 10 monclient(hunting): picked mon.noname-h con 0x7fb0001343b0 addr v1:172.21.15.132:6791/0 2024-01-24T23:29:36.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.827+0000 7fb007beb640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb00004ed70 con 0x7fb0001343b0 2024-01-24T23:29:36.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.827+0000 7fb007beb640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0000a4b60 con 0x7fb000133fd0 2024-01-24T23:29:36.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.827+0000 7fb007beb640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb00009ae50 con 0x7fb000133bf0 2024-01-24T23:29:36.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.827+0000 7fb007beb640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:36.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.827+0000 7fb007beb640 10 monclient(hunting): authenticate will time out at 1455.754325s 2024-01-24T23:29:36.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.827+0000 7fb005987640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fb000133bf0 0x7fb00009bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:56010/0 (socket says 172.21.15.115:56010) 2024-01-24T23:29:36.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.827+0000 7fb005987640 1 -- 172.21.15.115:0/112782672 learned_addr learned my addr 172.21.15.115:0/112782672 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:36.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 1 -- 172.21.15.115:0/112782672 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2951090768 0 0) 0x7fb00004ed70 con 0x7fb0001343b0 2024-01-24T23:29:36.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 10 monclient(hunting): _negotiate 2024-01-24T23:29:36.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:36.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:36.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 10 monclient(hunting): my global_id is 7115 2024-01-24T23:29:36.833 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 1 -- 172.21.15.115:0/112782672 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafec003530 con 0x7fb0001343b0 2024-01-24T23:29:36.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 1 -- 172.21.15.115:0/112782672 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2715793029 0 0) 0x7fafec003530 con 0x7fb0001343b0 2024-01-24T23:29:36.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 1 -- 172.21.15.115:0/112782672 >> v1:172.21.15.115:6793/0 conn(0x7fb000133bf0 legacy=0x7fb00009bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 1 -- 172.21.15.115:0/112782672 >> v1:172.21.15.132:6792/0 conn(0x7fb000133fd0 legacy=0x7fb00013bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 10 monclient: _finish_hunting 0 2024-01-24T23:29:36.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 1 monclient: found mon.noname-h 2024-01-24T23:29:36.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:36.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:29:36.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 1 -- 172.21.15.115:0/112782672 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00009b130 con 0x7fb0001343b0 2024-01-24T23:29:36.835 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 10 monclient: _finish_auth 0 2024-01-24T23:29:36.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 10 monclient: _check_auth_tickets 2024-01-24T23:29:36.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:36.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 1 -- 172.21.15.115:0/112782672 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faffc0017c0 con 0x7fb0001343b0 2024-01-24T23:29:36.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:29:36.836 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 10 monclient: dump: 2024-01-24T23:29:36.836 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.836 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.836 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.836 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.836 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.836 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.836 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.836 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.837 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.837 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.837 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.837 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.837 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.837 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.837 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.837 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb007beb640 5 monclient: authenticate success, global_id 7115 2024-01-24T23:29:36.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb007beb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:36.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 1 -- 172.21.15.115:0/112782672 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faffc0019e0 con 0x7fb0001343b0 2024-01-24T23:29:36.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:36.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb007beb640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:36.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb007beb640 4 set_mon_vals no callback set 2024-01-24T23:29:36.837 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 1 -- 172.21.15.115:0/112782672 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faffc003960 con 0x7fb0001343b0 2024-01-24T23:29:36.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:29:36.838 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb004985640 10 monclient: dump: 2024-01-24T23:29:36.838 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.838 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.838 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.838 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.838 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.838 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.838 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.838 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.838 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.838 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.838 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.838 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.839 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.839 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.839 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.839 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb007beb640 10 monclient: shutdown 2024-01-24T23:29:36.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb007beb640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:36.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb007beb640 1 -- 172.21.15.115:0/112782672 >> v1:172.21.15.132:6791/0 conn(0x7fb0001343b0 legacy=0x7fb00013f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb007beb640 1 -- 172.21.15.115:0/112782672 shutdown_connections 2024-01-24T23:29:36.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb007beb640 1 -- 172.21.15.115:0/112782672 >> 172.21.15.115:0/112782672 conn(0x7fb0000662f0 msgr2=0x7fb000066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:36.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb007beb640 1 -- 172.21.15.115:0/112782672 shutdown_connections 2024-01-24T23:29:36.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb007beb640 1 -- 172.21.15.115:0/112782672 wait complete. 2024-01-24T23:29:36.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb007beb640 10 monclient: build_initial_monmap 2024-01-24T23:29:36.839 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb007beb640 10 monclient: monmap: 2024-01-24T23:29:36.839 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:36.839 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:36.839 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:36.840 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:36.840 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:36.840 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:36.840 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:36.840 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:36.840 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:36.840 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:36.840 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:36.840 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:36.840 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:36.840 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:36.840 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:36.840 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb007beb640 1 Processor -- start 2024-01-24T23:29:36.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb007beb640 1 -- start start 2024-01-24T23:29:36.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.831+0000 7fb007beb640 10 monclient: init 2024-01-24T23:29:36.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.835+0000 7fb007beb640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:36.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.835+0000 7fb007beb640 10 monclient: _add_conns ranks=[8,0,4,2,5,1,7,3,6] 2024-01-24T23:29:36.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.835+0000 7fb007beb640 10 monclient(hunting): picked mon.noname-e con 0x7fb000133bf0 addr v1:172.21.15.115:6793/0 2024-01-24T23:29:36.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.835+0000 7fb007beb640 10 monclient(hunting): picked mon.noname-a con 0x7fb000133fd0 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:36.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.835+0000 7fb007beb640 10 monclient(hunting): picked mon.noname-c con 0x7fb0001343b0 addr v1:172.21.15.115:6791/0 2024-01-24T23:29:36.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.835+0000 7fb007beb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0001208f0 con 0x7fb000133fd0 2024-01-24T23:29:36.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.835+0000 7fb007beb640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0000a5990 con 0x7fb0001343b0 2024-01-24T23:29:36.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.835+0000 7fb007beb640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0000a2510 con 0x7fb000133bf0 2024-01-24T23:29:36.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.835+0000 7fb007beb640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:36.845 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.835+0000 7fb007beb640 10 monclient(hunting): authenticate will time out at 1455.760561s 2024-01-24T23:29:36.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.835+0000 7fb005987640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7fb000133bf0 0x7fb000132f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:56020/0 (socket says 172.21.15.115:56020) 2024-01-24T23:29:36.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.835+0000 7fb005987640 1 -- 172.21.15.115:0/3513502812 learned_addr learned my addr 172.21.15.115:0/3513502812 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:36.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.835+0000 7fb005186640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb000133fd0 0x7fb0001d0920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39252/0 (socket says 172.21.15.115:39252) 2024-01-24T23:29:36.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196930152 0 0) 0x7fb0001208f0 con 0x7fb000133fd0 2024-01-24T23:29:36.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:36.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:36.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:36.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 10 monclient(hunting): my global_id is 154269 2024-01-24T23:29:36.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafe4003530 con 0x7fb000133fd0 2024-01-24T23:29:36.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2750145024 0 0) 0x7fb0000a2510 con 0x7fb000133bf0 2024-01-24T23:29:36.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:36.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:36.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:36.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 10 monclient(hunting): my global_id is 86417 2024-01-24T23:29:36.846 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0001208f0 con 0x7fb000133bf0 2024-01-24T23:29:36.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3790559935 0 0) 0x7fafe4003530 con 0x7fb000133fd0 2024-01-24T23:29:36.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb0000a2510 con 0x7fb000133fd0 2024-01-24T23:29:36.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faff00019d0 con 0x7fb000133fd0 2024-01-24T23:29:36.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:29:36.847 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 10 monclient(hunting): dump: 2024-01-24T23:29:36.847 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.847 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.847 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.847 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.847 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.847 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.847 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.847 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.847 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.848 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.848 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.848 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.848 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.848 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.848 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.848 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:36.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 719770021 0 0) 0x7fb0000a5990 con 0x7fb0001343b0 2024-01-24T23:29:36.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:36.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:36.848 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:36.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 10 monclient(hunting): my global_id is 95341 2024-01-24T23:29:36.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafe4003530 con 0x7fb0001343b0 2024-01-24T23:29:36.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 645478343 0 0) 0x7fb0000a2510 con 0x7fb000133fd0 2024-01-24T23:29:36.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 >> v1:172.21.15.115:6793/0 conn(0x7fb000133bf0 legacy=0x7fb000132f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.849 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 >> v1:172.21.15.115:6791/0 conn(0x7fb0001343b0 legacy=0x7fb0001d4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 10 monclient: _finish_hunting 0 2024-01-24T23:29:36.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 1 monclient: found mon.a 2024-01-24T23:29:36.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:36.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:29:36.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb000120050 con 0x7fb000133fd0 2024-01-24T23:29:36.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 10 monclient: _finish_auth 0 2024-01-24T23:29:36.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 10 monclient: _check_auth_tickets 2024-01-24T23:29:36.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:36.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7fb007beb640 5 monclient: authenticate success, global_id 154269 2024-01-24T23:29:36.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7fb007beb640 10 monclient: _renew_subs 2024-01-24T23:29:36.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7fb007beb640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:29:36.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7fb007beb640 1 -- 172.21.15.115:0/3513502812 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0000a3410 con 0x7fb000133fd0 2024-01-24T23:29:36.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faff0001d10 con 0x7fb000133fd0 2024-01-24T23:29:36.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:36.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faff0003c90 con 0x7fb000133fd0 2024-01-24T23:29:36.850 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:29:36.851 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 10 monclient: dump: 2024-01-24T23:29:36.851 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.851 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.851 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.851 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.851 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.851 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.851 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.851 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.851 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.851 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.851 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.851 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.851 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.852 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.852 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.852 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff77fe640 4 set_mon_vals no callback set 2024-01-24T23:29:36.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7fb007beb640 10 monclient: _renew_subs 2024-01-24T23:29:36.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7fb007beb640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:29:36.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7fb007beb640 1 -- 172.21.15.115:0/3513502812 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb000133640 con 0x7fb000133fd0 2024-01-24T23:29:36.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7fb007beb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:36.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7fb007beb640 1 -- 172.21.15.115:0/3513502812 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb0001cb980 con 0x7fb0001cb5a0 2024-01-24T23:29:36.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7faff0003ec0 con 0x7fb000133fd0 2024-01-24T23:29:36.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7faff00443f0 con 0x7fb000133fd0 2024-01-24T23:29:36.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.839+0000 7fb006188640 10 monclient: get_auth_request con 0x7fafe4037110 auth_method 0 2024-01-24T23:29:36.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.843+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1967585727 0 0) 0x7fb0001cb980 con 0x7fb0001cb5a0 2024-01-24T23:29:36.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.843+0000 7faff67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:36.852 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.843+0000 7faff67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:36.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.843+0000 7faff67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:36.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.843+0000 7faff67fc640 10 monclient(hunting): my global_id is 7350 2024-01-24T23:29:36.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.843+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0000a2510 con 0x7fb0001cb5a0 2024-01-24T23:29:36.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.843+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3892051515 0 0) 0x7fb0000a2510 con 0x7fb0001cb5a0 2024-01-24T23:29:36.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.843+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb0001cb980 con 0x7fb0001cb5a0 2024-01-24T23:29:36.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.843+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faff8001bd0 con 0x7fb0001cb5a0 2024-01-24T23:29:36.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.843+0000 7faff67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.843+0000 7faff67fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:29:36.853 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.843+0000 7faff67fc640 10 monclient: dump: 2024-01-24T23:29:36.853 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.853 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.853 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.853 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.853 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.853 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.854 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.854 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.854 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.854 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.854 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.854 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.854 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.854 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.854 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.854 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.843+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1769122727 0 0) 0x7fb0001cb980 con 0x7fb0001cb5a0 2024-01-24T23:29:36.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.843+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 --> v1:172.21.15.115:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb0000a57c0 con 0x7fb0001cb5a0 2024-01-24T23:29:36.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.843+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7faff8004b70 con 0x7fb0001cb5a0 2024-01-24T23:29:36.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.843+0000 7faff67fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:36.854 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.843+0000 7faff67fc640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:36.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.843+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 >> v1:172.21.15.115:6792/0 conn(0x7fb0001cb5a0 legacy=0x7fb0001bca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.855 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.843+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 >> v1:172.21.15.115:6792/0 conn(0x7fb0001cb5a0 legacy=0x7fb0001bca50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:36.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7fb007beb640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:29:36.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7fb007beb640 1 -- 172.21.15.115:0/3513502812 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb00006a0d0 con 0x7fb00002c7f0 2024-01-24T23:29:36.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2439110164 0 0) 0x7fb00006a0d0 con 0x7fb00002c7f0 2024-01-24T23:29:36.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7faff67fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:36.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7faff67fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:36.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7faff67fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:36.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7faff67fc640 10 monclient(hunting): my global_id is 7359 2024-01-24T23:29:36.856 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0001cb980 con 0x7fb00002c7f0 2024-01-24T23:29:36.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 342229671 0 0) 0x7fb0001cb980 con 0x7fb00002c7f0 2024-01-24T23:29:36.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb00006a0d0 con 0x7fb00002c7f0 2024-01-24T23:29:36.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7faff80052d0 con 0x7fb00002c7f0 2024-01-24T23:29:36.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7faff67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:36.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7faff67fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:29:36.857 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7faff67fc640 10 monclient: dump: 2024-01-24T23:29:36.857 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:36.857 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.857 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.857 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:36.857 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:36.857 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.857 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:36.857 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:36.857 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:36.857 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:36.858 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:36.858 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:36.858 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:36.858 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:36.858 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:36.858 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:36.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2302960587 0 0) 0x7fb00006a0d0 con 0x7fb00002c7f0 2024-01-24T23:29:36.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 --> v1:172.21.15.115:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb0000711a0 con 0x7fb00002c7f0 2024-01-24T23:29:36.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 881601655) 0x7faff8005680 con 0x7fb00002c7f0 2024-01-24T23:29:36.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7faff67fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:29:36.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7faff67fc640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:36.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 >> v1:172.21.15.115:6792/0 conn(0x7fb00002c7f0 legacy=0x7fb0001bd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.858 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7faff67fc640 1 -- 172.21.15.115:0/3513502812 >> v1:172.21.15.115:6792/0 conn(0x7fb00002c7f0 legacy=0x7fb0001bd160 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:36.858 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:29:36.859 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:36.859 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:36.859 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:29:36.859 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 48, 2024-01-24T23:29:36.859 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:29:36.859 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:29:36.859 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:29:36.859 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:29:36.859 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:29:36.859 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:29:36.859 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:29:36.859 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:29:36.859 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:29:36.860 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:29:36.860 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:36.860 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 7, 2024-01-24T23:29:36.860 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:36.860 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:29:36.860 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:29:36.860 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:36.860 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:36.860 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:36.860 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:36.860 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:36.860 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:36.860 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:36.861 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:36.861 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:36.861 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:36.861 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:36.861 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:36.861 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:29:36.861 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:29:36.861 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:36.861 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:36.861 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:36.861 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:36.861 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:36.861 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:36.861 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:36.862 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:36.862 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:36.862 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:36.862 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:36.862 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.862 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:29:36.862 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:29:36.862 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:29:36.862 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:29:36.862 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:29:36.862 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:29:36.862 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:36.862 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:36.862 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:29:36.863 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:29:36.863 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:29:36.863 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:29:36.863 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:29:36.863 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:29:36.863 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:29:36.863 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:36.863 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:29:36.863 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:36.863 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:36.863 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:36.863 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:36.863 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:36.863 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:36.864 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:36.864 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:36.864 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:36.864 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:36.864 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:36.864 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:36.864 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:29:36.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.864 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:29:36.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.864 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:36.864 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:36.864 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.864 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.865 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.865 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:29:36.865 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.865 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.865 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.865 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:29:36.865 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:29:36.865 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.865 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.865 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.865 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:36.866 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:36.866 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.866 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.866 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.866 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:29:36.866 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.866 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.866 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.866 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:29:36.866 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:29:36.866 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.866 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.866 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.867 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:36.867 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:36.867 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.867 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.867 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.867 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:29:36.867 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.867 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.867 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.868 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:29:36.868 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:29:36.868 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.868 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.868 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.868 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:36.868 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:36.868 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.868 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.868 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.868 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:29:36.869 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.869 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.869 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.869 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:29:36.869 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:29:36.869 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.869 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.869 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.869 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:36.869 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:36.869 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.870 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.870 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.870 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:29:36.870 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.870 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.870 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.870 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:29:36.870 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:29:36.870 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.870 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.870 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.871 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:36.871 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:36.871 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.871 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.871 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.871 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:29:36.871 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.871 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.871 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.871 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:29:36.871 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:29:36.872 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.872 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.872 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.872 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:36.872 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:36.872 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.872 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.872 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.872 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:29:36.872 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.872 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.872 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.873 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:29:36.873 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:29:36.873 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.873 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.873 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.873 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:36.873 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:36.873 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.873 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.873 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.874 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:29:36.874 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.874 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.874 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.874 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:29:36.874 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:29:36.874 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.874 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.874 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.874 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:36.874 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:36.874 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.875 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:36.876 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:36.877 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.877 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:36.877 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:29:36.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:36.877 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:36.877 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:36.877 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:29:36.877 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:36.877 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:36.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:36.877 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:29:36.877 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:29:36.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7fb007beb640 1 -- 172.21.15.115:0/3513502812 >> v1:172.21.15.132:6812/1835401922 conn(0x7fafe4037110 legacy=0x7fafe40395d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7fb007beb640 10 monclient: shutdown 2024-01-24T23:29:36.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7fb007beb640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:36.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7fb007beb640 1 -- 172.21.15.115:0/3513502812 >> v1:172.21.15.115:6789/0 conn(0x7fb000133fd0 legacy=0x7fb0001d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:36.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7fb006188640 1 -- 172.21.15.115:0/3513502812 reap_dead start 2024-01-24T23:29:36.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7fb006188640 1 -- 172.21.15.115:0/3513502812 reap_dead start 2024-01-24T23:29:36.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7fb007beb640 1 -- 172.21.15.115:0/3513502812 shutdown_connections 2024-01-24T23:29:36.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7fb007beb640 1 -- 172.21.15.115:0/3513502812 >> 172.21.15.115:0/3513502812 conn(0x7fb0000662f0 msgr2=0x7fb0001beb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:36.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7fb007beb640 1 -- 172.21.15.115:0/3513502812 shutdown_connections 2024-01-24T23:29:36.879 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.847+0000 7fb007beb640 1 -- 172.21.15.115:0/3513502812 wait complete. 2024-01-24T23:29:36.879 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:29:36.880 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-01-24T23:29:36.880 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-01-24T23:29:36.955 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~212s 2024-01-24T23:29:36.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.991+0000 7ff545fe4640 10 monclient: get_monmap_and_config 2024-01-24T23:29:36.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.991+0000 7ff545fe4640 10 monclient: build_initial_monmap 2024-01-24T23:29:36.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.991+0000 7ff545fe4640 10 monclient: monmap: 2024-01-24T23:29:36.996 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:36.996 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:36.997 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:36.998752+0000 2024-01-24T23:29:36.997 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:36.998752+0000 2024-01-24T23:29:36.997 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:36.997 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:36.997 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:36.998 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:36.998 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:36.998 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:36.998 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:36.998 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:36.998 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:36.999 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:36.999 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:36.999 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff545fe4640 1 Processor -- start 2024-01-24T23:29:37.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff545fe4640 1 -- start start 2024-01-24T23:29:37.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff545fe4640 10 monclient: init 2024-01-24T23:29:37.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff545fe4640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:37.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff545fe4640 10 monclient: _add_conns ranks=[1,3,5,4,0,7,8,2,6] 2024-01-24T23:29:37.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff545fe4640 10 monclient(hunting): picked mon.noname-f con 0x7ff54009e740 addr v1:172.21.15.132:6789/0 2024-01-24T23:29:37.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff545fe4640 10 monclient(hunting): picked mon.noname-g con 0x7ff54009f230 addr v1:172.21.15.132:6790/0 2024-01-24T23:29:37.017 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff545fe4640 10 monclient(hunting): picked mon.noname-h con 0x7ff540071e90 addr v1:172.21.15.132:6791/0 2024-01-24T23:29:37.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff545fe4640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff54004ed70 con 0x7ff54009e740 2024-01-24T23:29:37.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff545fe4640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5400a1400 con 0x7ff54009f230 2024-01-24T23:29:37.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff545fe4640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5400a37d0 con 0x7ff540071e90 2024-01-24T23:29:37.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff545fe4640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:37.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff545fe4640 10 monclient(hunting): authenticate will time out at 1455.921297s 2024-01-24T23:29:37.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53effd640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7ff54009f230 0x7ff540139b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.115:57322/0 (socket says 172.21.15.115:57322) 2024-01-24T23:29:37.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53effd640 1 -- 172.21.15.115:0/561688317 learned_addr learned my addr 172.21.15.115:0/561688317 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:37.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 1 -- 172.21.15.115:0/561688317 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3349442183 0 0) 0x7ff5400a37d0 con 0x7ff540071e90 2024-01-24T23:29:37.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 10 monclient(hunting): my global_id is 7124 2024-01-24T23:29:37.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 1 -- 172.21.15.115:0/561688317 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff528003530 con 0x7ff540071e90 2024-01-24T23:29:37.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 1 -- 172.21.15.115:0/561688317 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597373820 0 0) 0x7ff5400a1400 con 0x7ff54009f230 2024-01-24T23:29:37.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 10 monclient(hunting): my global_id is 7167 2024-01-24T23:29:37.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 1 -- 172.21.15.115:0/561688317 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5400a37d0 con 0x7ff54009f230 2024-01-24T23:29:37.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 1 -- 172.21.15.115:0/561688317 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 453438687 0 0) 0x7ff54004ed70 con 0x7ff54009e740 2024-01-24T23:29:37.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 10 monclient(hunting): my global_id is 124723 2024-01-24T23:29:37.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 1 -- 172.21.15.115:0/561688317 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5400a1400 con 0x7ff54009e740 2024-01-24T23:29:37.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 1 -- 172.21.15.115:0/561688317 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3389415989 0 0) 0x7ff5400a1400 con 0x7ff54009e740 2024-01-24T23:29:37.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 1 -- 172.21.15.115:0/561688317 >> v1:172.21.15.132:6791/0 conn(0x7ff540071e90 legacy=0x7ff54013d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 1 -- 172.21.15.115:0/561688317 >> v1:172.21.15.132:6790/0 conn(0x7ff54009f230 legacy=0x7ff540139b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:29:37.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 1 monclient: found mon.noname-f 2024-01-24T23:29:37.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:37.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.132:6789/0 2024-01-24T23:29:37.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 1 -- 172.21.15.115:0/561688317 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5400a3120 con 0x7ff54009e740 2024-01-24T23:29:37.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 10 monclient: _finish_auth 0 2024-01-24T23:29:37.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:29:37.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:37.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 1 -- 172.21.15.115:0/561688317 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff5300016e0 con 0x7ff54009e740 2024-01-24T23:29:37.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-01-24T23:29:37.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 10 monclient: dump: 2024-01-24T23:29:37.024 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.025 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.025 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.025 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.025 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.025 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.026 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.026 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.026 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.026 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.026 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.026 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.027 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.027 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.027 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.027 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff545fe4640 5 monclient: authenticate success, global_id 124723 2024-01-24T23:29:37.027 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff545fe4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:37.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 1 -- 172.21.15.115:0/561688317 <== mon.1 v1:172.21.15.132:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff5300018d0 con 0x7ff54009e740 2024-01-24T23:29:37.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:37.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff545fe4640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:37.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 1 -- 172.21.15.115:0/561688317 <== mon.1 v1:172.21.15.132:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff530003600 con 0x7ff54009e740 2024-01-24T23:29:37.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff545fe4640 4 set_mon_vals no callback set 2024-01-24T23:29:37.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:29:37.029 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff53e7fc640 10 monclient: dump: 2024-01-24T23:29:37.029 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.029 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.029 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.030 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.030 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.030 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.030 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.030 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.030 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.031 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.031 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.031 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.031 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.031 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.031 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.032 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff545fe4640 10 monclient: shutdown 2024-01-24T23:29:37.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff545fe4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:37.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff545fe4640 1 -- 172.21.15.115:0/561688317 >> v1:172.21.15.132:6789/0 conn(0x7ff54009e740 legacy=0x7ff54009eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff545fe4640 1 -- 172.21.15.115:0/561688317 shutdown_connections 2024-01-24T23:29:37.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff545fe4640 1 -- 172.21.15.115:0/561688317 >> 172.21.15.115:0/561688317 conn(0x7ff5400666f0 msgr2=0x7ff540066b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:37.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff545fe4640 1 -- 172.21.15.115:0/561688317 shutdown_connections 2024-01-24T23:29:37.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.995+0000 7ff545fe4640 1 -- 172.21.15.115:0/561688317 wait complete. 2024-01-24T23:29:37.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff545fe4640 10 monclient: build_initial_monmap 2024-01-24T23:29:37.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff545fe4640 10 monclient: monmap: 2024-01-24T23:29:37.033 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:37.034 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:37.034 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:37.034 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:37.034 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:37.034 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:37.035 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:37.035 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:37.035 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:37.035 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:37.035 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:37.035 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:37.036 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:37.036 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:37.036 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:37.036 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff545fe4640 1 Processor -- start 2024-01-24T23:29:37.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff545fe4640 1 -- start start 2024-01-24T23:29:37.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff545fe4640 10 monclient: init 2024-01-24T23:29:37.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff545fe4640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:37.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff545fe4640 10 monclient: _add_conns ranks=[6,2,7,4,8,3,1,0,5] 2024-01-24T23:29:37.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff545fe4640 10 monclient(hunting): picked mon.noname-d con 0x7ff540071e90 addr v1:172.21.15.115:6792/0 2024-01-24T23:29:37.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff545fe4640 10 monclient(hunting): picked mon.noname-b con 0x7ff54009e740 addr v1:172.21.15.115:6790/0 2024-01-24T23:29:37.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff545fe4640 10 monclient(hunting): picked mon.noname-i con 0x7ff5401cba30 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:37.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff545fe4640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5401a8400 con 0x7ff54009e740 2024-01-24T23:29:37.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff545fe4640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5401a7d90 con 0x7ff540071e90 2024-01-24T23:29:37.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff545fe4640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff54006c1b0 con 0x7ff5401cba30 2024-01-24T23:29:37.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff545fe4640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:37.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff545fe4640 10 monclient(hunting): authenticate will time out at 1455.925198s 2024-01-24T23:29:37.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff53effd640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7ff54009e740 0x7ff5401d0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:43126/0 (socket says 172.21.15.115:43126) 2024-01-24T23:29:37.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff53f7fe640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7ff540071e90 0x7ff5401b9b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:53788/0 (socket says 172.21.15.115:53788) 2024-01-24T23:29:37.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff53effd640 1 -- 172.21.15.115:0/2166395859 learned_addr learned my addr 172.21.15.115:0/2166395859 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:37.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 888042880 0 0) 0x7ff5401a7d90 con 0x7ff540071e90 2024-01-24T23:29:37.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient(hunting): my global_id is 7368 2024-01-24T23:29:37.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff52c003530 con 0x7ff540071e90 2024-01-24T23:29:37.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3494633545 0 0) 0x7ff5401a8400 con 0x7ff54009e740 2024-01-24T23:29:37.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient(hunting): my global_id is 154199 2024-01-24T23:29:37.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5401a7d90 con 0x7ff54009e740 2024-01-24T23:29:37.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140028032 0 0) 0x7ff54006c1b0 con 0x7ff5401cba30 2024-01-24T23:29:37.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient(hunting): my global_id is 154231 2024-01-24T23:29:37.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5401a8400 con 0x7ff5401cba30 2024-01-24T23:29:37.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 142443174 0 0) 0x7ff52c003530 con 0x7ff540071e90 2024-01-24T23:29:37.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff54006c1b0 con 0x7ff540071e90 2024-01-24T23:29:37.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2943197589 0 0) 0x7ff5401a7d90 con 0x7ff54009e740 2024-01-24T23:29:37.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff52c003530 con 0x7ff54009e740 2024-01-24T23:29:37.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff5300016a0 con 0x7ff540071e90 2024-01-24T23:29:37.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:29:37.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient(hunting): dump: 2024-01-24T23:29:37.044 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.047 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.047 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.047 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.047 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.047 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.047 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.047 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.047 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.047 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.047 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.047 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.048 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.048 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.048 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.048 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:37.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff534001860 con 0x7ff54009e740 2024-01-24T23:29:37.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-01-24T23:29:37.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient(hunting): dump: 2024-01-24T23:29:37.048 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.048 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.048 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.048 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.048 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.049 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.049 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.049 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.049 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.049 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.049 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.049 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.049 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.049 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.049 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.049 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4024326561 0 0) 0x7ff52c003530 con 0x7ff54009e740 2024-01-24T23:29:37.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 >> v1:172.21.15.132:6792/0 conn(0x7ff5401cba30 legacy=0x7ff5401d3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 >> v1:172.21.15.115:6792/0 conn(0x7ff540071e90 legacy=0x7ff5401b9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient: _finish_hunting 0 2024-01-24T23:29:37.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 monclient: found mon.b 2024-01-24T23:29:37.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:37.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:29:37.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5401a8a40 con 0x7ff54009e740 2024-01-24T23:29:37.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient: _finish_auth 0 2024-01-24T23:29:37.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient: _check_auth_tickets 2024-01-24T23:29:37.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:37.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff545fe4640 5 monclient: authenticate success, global_id 154199 2024-01-24T23:29:37.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff545fe4640 10 monclient: _renew_subs 2024-01-24T23:29:37.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff545fe4640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:29:37.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff545fe4640 1 -- 172.21.15.115:0/2166395859 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5401a8d20 con 0x7ff54009e740 2024-01-24T23:29:37.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff545fe4640 10 monclient: _renew_subs 2024-01-24T23:29:37.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff545fe4640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:29:37.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff545fe4640 1 -- 172.21.15.115:0/2166395859 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5401d5150 con 0x7ff54009e740 2024-01-24T23:29:37.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff534001a00 con 0x7ff54009e740 2024-01-24T23:29:37.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:37.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff534004b30 con 0x7ff54009e740 2024-01-24T23:29:37.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff53dffb640 4 set_mon_vals no callback set 2024-01-24T23:29:37.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:29:37.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient: dump: 2024-01-24T23:29:37.051 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.051 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.051 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.051 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.051 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.051 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.051 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.051 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.051 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.052 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.052 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.052 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.052 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.052 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.052 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.052 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff545fe4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:37.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff545fe4640 1 -- 172.21.15.115:0/2166395859 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff5200065d0 con 0x7ff520002870 2024-01-24T23:29:37.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7ff53400f5b0 con 0x7ff54009e740 2024-01-24T23:29:37.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7ff5340441e0 con 0x7ff54009e740 2024-01-24T23:29:37.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2626654112 0 0) 0x7ff5200065d0 con 0x7ff520002870 2024-01-24T23:29:37.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.052 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient(hunting): my global_id is 86426 2024-01-24T23:29:37.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff52c003530 con 0x7ff520002870 2024-01-24T23:29:37.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3040967357 0 0) 0x7ff52c003530 con 0x7ff520002870 2024-01-24T23:29:37.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff5200065d0 con 0x7ff520002870 2024-01-24T23:29:37.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff530004070 con 0x7ff520002870 2024-01-24T23:29:37.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff53ffff640 10 monclient: get_auth_request con 0x7ff52c0378e0 auth_method 0 2024-01-24T23:29:37.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:29:37.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 10 monclient: dump: 2024-01-24T23:29:37.053 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.053 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.053 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.053 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.053 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.054 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.054 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.054 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.054 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.054 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.054 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.054 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.054 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.054 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.054 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.054 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1660371683 0 0) 0x7ff5200065d0 con 0x7ff520002870 2024-01-24T23:29:37.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:36.999+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 --> v1:172.21.15.115:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff52000a120 con 0x7ff520002870 2024-01-24T23:29:37.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.003+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7ff530004550 con 0x7ff520002870 2024-01-24T23:29:37.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.003+0000 7ff527fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:37.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.003+0000 7ff527fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:37.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.003+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 >> v1:172.21.15.115:6793/0 conn(0x7ff520002870 legacy=0x7ff520004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.003+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 >> v1:172.21.15.115:6793/0 conn(0x7ff520002870 legacy=0x7ff520004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:37.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff545fe4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:29:37.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff545fe4640 1 -- 172.21.15.115:0/2166395859 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff520009e10 con 0x7ff5200090d0 2024-01-24T23:29:37.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 360944716 0 0) 0x7ff520009e10 con 0x7ff5200090d0 2024-01-24T23:29:37.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff527fff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff527fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff527fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff527fff640 10 monclient(hunting): my global_id is 86435 2024-01-24T23:29:37.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5200065d0 con 0x7ff5200090d0 2024-01-24T23:29:37.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3830227303 0 0) 0x7ff5200065d0 con 0x7ff5200090d0 2024-01-24T23:29:37.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff520009e10 con 0x7ff5200090d0 2024-01-24T23:29:37.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7ff530005f10 con 0x7ff5200090d0 2024-01-24T23:29:37.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff527fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff527fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:29:37.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff527fff640 10 monclient: dump: 2024-01-24T23:29:37.056 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.056 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.056 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.056 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.056 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.056 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.056 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.056 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.056 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.056 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.056 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.056 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.056 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.057 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.057 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.057 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3581774952 0 0) 0x7ff520009e10 con 0x7ff5200090d0 2024-01-24T23:29:37.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 --> v1:172.21.15.115:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff52000c310 con 0x7ff5200090d0 2024-01-24T23:29:37.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 403195139) 0x7ff530008190 con 0x7ff5200090d0 2024-01-24T23:29:37.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff527fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:29:37.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff527fff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:37.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 >> v1:172.21.15.115:6793/0 conn(0x7ff5200090d0 legacy=0x7ff520009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff527fff640 1 -- 172.21.15.115:0/2166395859 >> v1:172.21.15.115:6793/0 conn(0x7ff5200090d0 legacy=0x7ff520009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:37.058 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:29:37.058 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:37.058 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:37.058 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:29:37.058 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 48, 2024-01-24T23:29:37.058 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:29:37.058 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:29:37.058 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:29:37.058 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:29:37.058 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:29:37.058 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:29:37.058 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:29:37.058 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:29:37.059 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:29:37.059 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:29:37.059 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:37.059 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 7, 2024-01-24T23:29:37.059 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:37.059 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:29:37.059 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:29:37.059 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:37.059 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:37.059 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:37.059 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:37.059 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:37.059 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:37.059 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:37.059 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:37.060 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:37.061 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:37.062 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.064 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.065 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.066 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:37.066 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:37.066 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.066 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.066 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.066 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:29:37.066 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.066 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.066 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.066 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:29:37.067 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:29:37.067 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.067 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.067 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.067 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:37.067 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:37.067 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.067 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.067 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.067 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:29:37.067 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.067 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.068 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.068 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:29:37.068 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:29:37.068 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.068 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.068 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.068 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:37.068 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:37.068 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.068 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.069 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.069 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:29:37.069 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.069 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.069 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.069 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:29:37.069 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:29:37.069 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.069 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.069 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.069 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:37.070 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:37.070 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.070 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.070 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.070 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:29:37.070 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.070 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.070 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.070 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:29:37.070 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:29:37.070 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.070 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.070 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.071 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:37.071 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:37.071 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.071 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.071 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.071 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:29:37.071 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.071 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.071 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.071 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:29:37.072 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:29:37.072 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.072 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.072 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.072 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:37.072 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:37.072 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.072 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.072 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.072 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:29:37.072 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.073 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.073 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.073 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:29:37.073 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:29:37.073 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.073 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.073 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.073 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:37.073 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:37.073 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.073 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.074 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.074 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:29:37.074 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.074 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.074 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.074 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:29:37.074 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:29:37.074 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.074 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.074 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.074 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.074 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.075 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:29:37.075 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:29:37.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.075 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:37.075 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:37.075 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:37.075 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.075 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:37.075 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:29:37.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.075 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:37.075 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:37.075 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:37.075 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.076 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.076 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:29:37.076 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:29:37.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff545fe4640 1 -- 172.21.15.115:0/2166395859 >> v1:172.21.15.132:6812/1835401922 conn(0x7ff52c0378e0 legacy=0x7ff52c039da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff545fe4640 10 monclient: shutdown 2024-01-24T23:29:37.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff545fe4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:37.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff545fe4640 1 -- 172.21.15.115:0/2166395859 >> v1:172.21.15.115:6790/0 conn(0x7ff54009e740 legacy=0x7ff5401d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.077 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff53ffff640 1 -- 172.21.15.115:0/2166395859 reap_dead start 2024-01-24T23:29:37.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff53ffff640 1 -- 172.21.15.115:0/2166395859 reap_dead start 2024-01-24T23:29:37.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff545fe4640 1 -- 172.21.15.115:0/2166395859 shutdown_connections 2024-01-24T23:29:37.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff545fe4640 1 -- 172.21.15.115:0/2166395859 >> 172.21.15.115:0/2166395859 conn(0x7ff5400666f0 msgr2=0x7ff54006fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:37.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff545fe4640 1 -- 172.21.15.115:0/2166395859 shutdown_connections 2024-01-24T23:29:37.078 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.007+0000 7ff545fe4640 1 -- 172.21.15.115:0/2166395859 wait complete. 2024-01-24T23:29:37.078 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:29:37.078 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-01-24T23:29:37.078 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-01-24T23:29:37.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 10 monclient: get_monmap_and_config 2024-01-24T23:29:37.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 10 monclient: build_initial_monmap 2024-01-24T23:29:37.200 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 10 monclient: monmap: 2024-01-24T23:29:37.200 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:37.200 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.200 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:37.203074+0000 2024-01-24T23:29:37.201 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:37.203074+0000 2024-01-24T23:29:37.201 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:37.201 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.201 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:37.201 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:37.201 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:37.201 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:37.201 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:37.201 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:37.201 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:37.201 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:37.201 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:37.201 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 1 Processor -- start 2024-01-24T23:29:37.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 1 -- start start 2024-01-24T23:29:37.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 10 monclient: init 2024-01-24T23:29:37.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:37.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 10 monclient: _add_conns ranks=[0,8,5,2,1,3,7,4,6] 2024-01-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 10 monclient(hunting): picked mon.noname-a con 0x7f22d4133bf0 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 10 monclient(hunting): picked mon.noname-e con 0x7f22d4133fd0 addr v1:172.21.15.115:6793/0 2024-01-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 10 monclient(hunting): picked mon.noname-h con 0x7f22d41343b0 addr v1:172.21.15.132:6791/0 2024-01-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d404ed70 con 0x7f22d4133bf0 2024-01-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d40a4b60 con 0x7f22d41343b0 2024-01-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d409ae50 con 0x7f22d4133fd0 2024-01-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 10 monclient(hunting): authenticate will time out at 1456.125658s 2024-01-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d37fe640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f22d4133fd0 0x7f22d413bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:56058/0 (socket says 172.21.15.115:56058) 2024-01-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d3fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f22d4133bf0 0x7f22d409bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39264/0 (socket says 172.21.15.115:39264) 2024-01-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d37fe640 1 -- 172.21.15.115:0/4056801021 learned_addr learned my addr 172.21.15.115:0/4056801021 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 1 -- 172.21.15.115:0/4056801021 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1803430116 0 0) 0x7f22d409ae50 con 0x7f22d4133fd0 2024-01-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 10 monclient(hunting): my global_id is 86444 2024-01-24T23:29:37.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 1 -- 172.21.15.115:0/4056801021 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22bc003530 con 0x7f22d4133fd0 2024-01-24T23:29:37.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 1 -- 172.21.15.115:0/4056801021 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4196952116 0 0) 0x7f22d404ed70 con 0x7f22d4133bf0 2024-01-24T23:29:37.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 10 monclient(hunting): my global_id is 154278 2024-01-24T23:29:37.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 1 -- 172.21.15.115:0/4056801021 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22d409ae50 con 0x7f22d4133bf0 2024-01-24T23:29:37.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 1 -- 172.21.15.115:0/4056801021 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 317061735 0 0) 0x7f22bc003530 con 0x7f22d4133fd0 2024-01-24T23:29:37.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 1 -- 172.21.15.115:0/4056801021 >> v1:172.21.15.132:6791/0 conn(0x7f22d41343b0 legacy=0x7f22d413f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 1 -- 172.21.15.115:0/4056801021 >> v1:172.21.15.115:6789/0 conn(0x7f22d4133bf0 legacy=0x7f22d409bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:29:37.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 1 monclient: found mon.noname-e 2024-01-24T23:29:37.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:37.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:29:37.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 1 -- 172.21.15.115:0/4056801021 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d409b130 con 0x7f22d4133fd0 2024-01-24T23:29:37.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 10 monclient: _finish_auth 0 2024-01-24T23:29:37.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:29:37.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:37.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 1 -- 172.21.15.115:0/4056801021 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f22c40017c0 con 0x7f22d4133fd0 2024-01-24T23:29:37.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 5 monclient: authenticate success, global_id 86444 2024-01-24T23:29:37.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:37.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:29:37.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 10 monclient: dump: 2024-01-24T23:29:37.211 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.211 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.211 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.211 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.211 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.211 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.212 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.212 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.212 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.212 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.212 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.212 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.212 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.212 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.212 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.212 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:37.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 1 -- 172.21.15.115:0/4056801021 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f22c4001a20 con 0x7f22d4133fd0 2024-01-24T23:29:37.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:37.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:37.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 4 set_mon_vals no callback set 2024-01-24T23:29:37.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 1 -- 172.21.15.115:0/4056801021 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f22c4004bf0 con 0x7f22d4133fd0 2024-01-24T23:29:37.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:29:37.213 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22d2ffd640 10 monclient: dump: 2024-01-24T23:29:37.213 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.213 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.213 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.213 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.213 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.213 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.213 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.213 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.213 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.213 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.213 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.213 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.214 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.214 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.214 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.214 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 10 monclient: shutdown 2024-01-24T23:29:37.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:37.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 1 -- 172.21.15.115:0/4056801021 >> v1:172.21.15.115:6793/0 conn(0x7f22d4133fd0 legacy=0x7f22d413bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 1 -- 172.21.15.115:0/4056801021 shutdown_connections 2024-01-24T23:29:37.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 1 -- 172.21.15.115:0/4056801021 >> 172.21.15.115:0/4056801021 conn(0x7f22d40662f0 msgr2=0x7f22d4066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:37.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 1 -- 172.21.15.115:0/4056801021 shutdown_connections 2024-01-24T23:29:37.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 1 -- 172.21.15.115:0/4056801021 wait complete. 2024-01-24T23:29:37.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 10 monclient: build_initial_monmap 2024-01-24T23:29:37.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.199+0000 7f22da3b1640 10 monclient: monmap: 2024-01-24T23:29:37.214 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:37.214 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:37.214 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:37.215 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:37.215 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:37.215 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:37.215 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:37.215 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:37.215 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:37.215 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:37.215 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:37.215 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:37.215 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:37.215 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:37.215 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:37.215 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22da3b1640 1 Processor -- start 2024-01-24T23:29:37.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22da3b1640 1 -- start start 2024-01-24T23:29:37.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22da3b1640 10 monclient: init 2024-01-24T23:29:37.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22da3b1640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:37.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22da3b1640 10 monclient: _add_conns ranks=[5,7,0,4,3,8,6,1,2] 2024-01-24T23:29:37.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22da3b1640 10 monclient(hunting): picked mon.noname-h con 0x7f22d4133bf0 addr v1:172.21.15.132:6791/0 2024-01-24T23:29:37.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22da3b1640 10 monclient(hunting): picked mon.noname-i con 0x7f22d4133fd0 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:37.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22da3b1640 10 monclient(hunting): picked mon.noname-a con 0x7f22d41343b0 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:37.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22da3b1640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d411fe50 con 0x7f22d41343b0 2024-01-24T23:29:37.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22da3b1640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d409c810 con 0x7f22d4133bf0 2024-01-24T23:29:37.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22da3b1640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d40a5990 con 0x7f22d4133fd0 2024-01-24T23:29:37.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22da3b1640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:37.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22da3b1640 10 monclient(hunting): authenticate will time out at 1456.128821s 2024-01-24T23:29:37.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d894e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f22d41343b0 0x7f22d41d3b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39270/0 (socket says 172.21.15.115:39270) 2024-01-24T23:29:37.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d894e640 1 -- 172.21.15.115:0/1093073864 learned_addr learned my addr 172.21.15.115:0/1093073864 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:37.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3212321744 0 0) 0x7f22d411fe50 con 0x7f22d41343b0 2024-01-24T23:29:37.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 10 monclient(hunting): my global_id is 154287 2024-01-24T23:29:37.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22c0003530 con 0x7f22d41343b0 2024-01-24T23:29:37.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2628395706 0 0) 0x7f22c0003530 con 0x7f22d41343b0 2024-01-24T23:29:37.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f22d411fe50 con 0x7f22d41343b0 2024-01-24T23:29:37.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f22cc001d20 con 0x7f22d41343b0 2024-01-24T23:29:37.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-01-24T23:29:37.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 10 monclient(hunting): dump: 2024-01-24T23:29:37.217 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.217 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.217 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.217 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.217 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.218 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.218 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.218 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.218 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.218 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.218 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.218 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.218 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.218 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.218 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.218 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:37.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1615158895 0 0) 0x7f22d411fe50 con 0x7f22d41343b0 2024-01-24T23:29:37.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 >> v1:172.21.15.132:6792/0 conn(0x7f22d4133fd0 legacy=0x7f22d41d0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 >> v1:172.21.15.132:6791/0 conn(0x7f22d4133bf0 legacy=0x7f22d4132ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:29:37.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 1 monclient: found mon.a 2024-01-24T23:29:37.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:37.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:29:37.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d4120380 con 0x7f22d41343b0 2024-01-24T23:29:37.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 10 monclient: _finish_auth 0 2024-01-24T23:29:37.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:29:37.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:37.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22da3b1640 5 monclient: authenticate success, global_id 154287 2024-01-24T23:29:37.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22da3b1640 10 monclient: _renew_subs 2024-01-24T23:29:37.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22da3b1640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:29:37.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22da3b1640 1 -- 172.21.15.115:0/1093073864 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d4120660 con 0x7f22d41343b0 2024-01-24T23:29:37.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22da3b1640 10 monclient: _renew_subs 2024-01-24T23:29:37.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22da3b1640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.115:6789/0 2024-01-24T23:29:37.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22da3b1640 1 -- 172.21.15.115:0/1093073864 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d40a2ae0 con 0x7f22d41343b0 2024-01-24T23:29:37.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f22cc003c50 con 0x7f22d41343b0 2024-01-24T23:29:37.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:37.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f22cc004180 con 0x7f22d41343b0 2024-01-24T23:29:37.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d27fc640 4 set_mon_vals no callback set 2024-01-24T23:29:37.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-01-24T23:29:37.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 10 monclient: dump: 2024-01-24T23:29:37.220 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.220 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.220 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.220 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.220 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.220 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.220 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.220 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.220 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.220 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.220 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.220 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.221 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.221 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.221 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.221 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22da3b1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:37.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22da3b1640 1 -- 172.21.15.115:0/1093073864 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f22b40065d0 con 0x7f22b4002870 2024-01-24T23:29:37.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 <== mon.0 v1:172.21.15.115:6789/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f22cc0042e0 con 0x7f22d41343b0 2024-01-24T23:29:37.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 <== mon.0 v1:172.21.15.115:6789/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7f22cc044730 con 0x7f22d41343b0 2024-01-24T23:29:37.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d37fe640 10 monclient: get_auth_request con 0x7f22c0037150 auth_method 0 2024-01-24T23:29:37.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 156352267 0 0) 0x7f22b40065d0 con 0x7f22b4002870 2024-01-24T23:29:37.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 10 monclient(hunting): my global_id is 124732 2024-01-24T23:29:37.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22d411fe50 con 0x7f22b4002870 2024-01-24T23:29:37.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3762263121 0 0) 0x7f22d411fe50 con 0x7f22b4002870 2024-01-24T23:29:37.221 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f22b40065d0 con 0x7f22b4002870 2024-01-24T23:29:37.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f22c8001520 con 0x7f22b4002870 2024-01-24T23:29:37.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:29:37.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 10 monclient: dump: 2024-01-24T23:29:37.222 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.222 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.222 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.222 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.222 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.222 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.222 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.222 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.222 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.222 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.222 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.222 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.223 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.223 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.223 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.223 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2892155437 0 0) 0x7f22b40065d0 con 0x7f22b4002870 2024-01-24T23:29:37.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.203+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 --> v1:172.21.15.132:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22b400a120 con 0x7f22b4002870 2024-01-24T23:29:37.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.207+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 <== mon.1 v1:172.21.15.132:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f22c80017f0 con 0x7f22b4002870 2024-01-24T23:29:37.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.207+0000 7f22d0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:37.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.207+0000 7f22d0ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:37.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.207+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 >> v1:172.21.15.132:6789/0 conn(0x7f22b4002870 legacy=0x7f22b4004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.207+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 >> v1:172.21.15.132:6789/0 conn(0x7f22b4002870 legacy=0x7f22b4004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:37.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.211+0000 7f22da3b1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:29:37.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.211+0000 7f22da3b1640 1 -- 172.21.15.115:0/1093073864 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f22b4009e10 con 0x7f22b40090d0 2024-01-24T23:29:37.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.211+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1496651274 0 0) 0x7f22b4009e10 con 0x7f22b40090d0 2024-01-24T23:29:37.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.211+0000 7f22d0ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.211+0000 7f22d0ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.211+0000 7f22d0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.211+0000 7f22d0ff9640 10 monclient(hunting): my global_id is 124741 2024-01-24T23:29:37.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.211+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22b40065d0 con 0x7f22b40090d0 2024-01-24T23:29:37.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.211+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3815512143 0 0) 0x7f22b40065d0 con 0x7f22b40090d0 2024-01-24T23:29:37.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.211+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 --> v1:172.21.15.132:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f22b4009e10 con 0x7f22b40090d0 2024-01-24T23:29:37.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.211+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f22c8004e30 con 0x7f22b40090d0 2024-01-24T23:29:37.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.211+0000 7f22d0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.211+0000 7f22d0ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-01-24T23:29:37.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.211+0000 7f22d0ff9640 10 monclient: dump: 2024-01-24T23:29:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.226 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.227 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.211+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 <== mon.1 v1:172.21.15.132:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2186599474 0 0) 0x7f22b4009e10 con 0x7f22b40090d0 2024-01-24T23:29:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.211+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 --> v1:172.21.15.132:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f22b400c310 con 0x7f22b40090d0 2024-01-24T23:29:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.215+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 <== mon.1 v1:172.21.15.132:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 3048095046) 0x7f22c8007180 con 0x7f22b40090d0 2024-01-24T23:29:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.215+0000 7f22d0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:29:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.215+0000 7f22d0ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.215+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 >> v1:172.21.15.132:6789/0 conn(0x7f22b40090d0 legacy=0x7f22b4009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.215+0000 7f22d0ff9640 1 -- 172.21.15.115:0/1093073864 >> v1:172.21.15.132:6789/0 conn(0x7f22b40090d0 legacy=0x7f22b4009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:37.227 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:29:37.227 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:29:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 48, 2024-01-24T23:29:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:29:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:29:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:29:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:29:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:29:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:29:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:29:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:29:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:29:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:29:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 7, 2024-01-24T23:29:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:29:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:29:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:29:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:29:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:29:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:29:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:29:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:29:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:29:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:29:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:29:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:29:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:29:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:29:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:29:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:29:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:29:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:29:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:29:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:29:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:29:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:29:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:29:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:29:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:29:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:29:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:29:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:29:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:29:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:29:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:29:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:29:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:29:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:29:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:29:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:29:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:29:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:29:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:29:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:29:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:29:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:29:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:29:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:29:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:29:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:29:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:29:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:29:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:29:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:29:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:29:37.245 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:29:37.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.215+0000 7f22da3b1640 1 -- 172.21.15.115:0/1093073864 >> v1:172.21.15.132:6812/1835401922 conn(0x7f22c0037150 legacy=0x7f22c0039610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.215+0000 7f22da3b1640 10 monclient: shutdown 2024-01-24T23:29:37.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.215+0000 7f22da3b1640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:37.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.215+0000 7f22da3b1640 1 -- 172.21.15.115:0/1093073864 >> v1:172.21.15.115:6789/0 conn(0x7f22d41343b0 legacy=0x7f22d41d3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.215+0000 7f22d894e640 1 -- 172.21.15.115:0/1093073864 reap_dead start 2024-01-24T23:29:37.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.215+0000 7f22d894e640 1 -- 172.21.15.115:0/1093073864 reap_dead start 2024-01-24T23:29:37.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.215+0000 7f22da3b1640 1 -- 172.21.15.115:0/1093073864 shutdown_connections 2024-01-24T23:29:37.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.215+0000 7f22da3b1640 1 -- 172.21.15.115:0/1093073864 >> 172.21.15.115:0/1093073864 conn(0x7f22d40662f0 msgr2=0x7f22d41be780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:37.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.215+0000 7f22da3b1640 1 -- 172.21.15.115:0/1093073864 shutdown_connections 2024-01-24T23:29:37.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.215+0000 7f22da3b1640 1 -- 172.21.15.115:0/1093073864 wait complete. 2024-01-24T23:29:37.247 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:29:37.248 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-01-24T23:29:37.248 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-01-24T23:29:37.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd8e2f640 10 monclient: get_monmap_and_config 2024-01-24T23:29:37.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd8e2f640 10 monclient: build_initial_monmap 2024-01-24T23:29:37.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd8e2f640 10 monclient: monmap: 2024-01-24T23:29:37.358 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:37.358 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.358 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:37.360283+0000 2024-01-24T23:29:37.358 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:37.360283+0000 2024-01-24T23:29:37.358 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:37.358 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.358 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:37.358 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:37.358 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:37.358 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:37.358 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:37.358 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:37.358 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:37.358 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:37.358 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:37.359 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd8e2f640 1 Processor -- start 2024-01-24T23:29:37.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd8e2f640 1 -- start start 2024-01-24T23:29:37.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd8e2f640 10 monclient: init 2024-01-24T23:29:37.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd8e2f640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:37.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd8e2f640 10 monclient: _add_conns ranks=[0,4,2,5,1,8,7,3,6] 2024-01-24T23:29:37.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd8e2f640 10 monclient(hunting): picked mon.noname-a con 0x7f0dd40a2c40 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:37.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd8e2f640 10 monclient(hunting): picked mon.noname-c con 0x7f0dd40a3fd0 addr v1:172.21.15.115:6791/0 2024-01-24T23:29:37.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd8e2f640 10 monclient(hunting): picked mon.noname-b con 0x7f0dd40a06b0 addr v1:172.21.15.115:6790/0 2024-01-24T23:29:37.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd8e2f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd404ed70 con 0x7f0dd40a2c40 2024-01-24T23:29:37.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd8e2f640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd40a0e70 con 0x7f0dd40a06b0 2024-01-24T23:29:37.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd8e2f640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd40a3730 con 0x7f0dd40a3fd0 2024-01-24T23:29:37.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd8e2f640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:37.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd8e2f640 10 monclient(hunting): authenticate will time out at 1456.282861s 2024-01-24T23:29:37.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd2d9d640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f0dd40a06b0 0x7f0dd413f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:43140/0 (socket says 172.21.15.115:43140) 2024-01-24T23:29:37.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd2d9d640 1 -- 172.21.15.115:0/1052607307 learned_addr learned my addr 172.21.15.115:0/1052607307 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:37.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd259c640 1 --1- 172.21.15.115:0/1052607307 >> v1:172.21.15.115:6789/0 conn(0x7f0dd40a2c40 0x7f0dd40a3020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39274/0 (socket says 172.21.15.115:39274) 2024-01-24T23:29:37.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 1 -- 172.21.15.115:0/1052607307 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 357537903 0 0) 0x7f0dd404ed70 con 0x7f0dd40a2c40 2024-01-24T23:29:37.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 10 monclient(hunting): my global_id is 154296 2024-01-24T23:29:37.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 1 -- 172.21.15.115:0/1052607307 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dc0003530 con 0x7f0dd40a2c40 2024-01-24T23:29:37.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 1 -- 172.21.15.115:0/1052607307 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2257920276 0 0) 0x7f0dd40a3730 con 0x7f0dd40a3fd0 2024-01-24T23:29:37.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 10 monclient(hunting): my global_id is 95350 2024-01-24T23:29:37.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 1 -- 172.21.15.115:0/1052607307 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dd404ed70 con 0x7f0dd40a3fd0 2024-01-24T23:29:37.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 1 -- 172.21.15.115:0/1052607307 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1165491525 0 0) 0x7f0dd40a0e70 con 0x7f0dd40a06b0 2024-01-24T23:29:37.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 10 monclient(hunting): my global_id is 154208 2024-01-24T23:29:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 1 -- 172.21.15.115:0/1052607307 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dd40a3730 con 0x7f0dd40a06b0 2024-01-24T23:29:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 1 -- 172.21.15.115:0/1052607307 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2206440738 0 0) 0x7f0dd40a3730 con 0x7f0dd40a06b0 2024-01-24T23:29:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 1 -- 172.21.15.115:0/1052607307 >> v1:172.21.15.115:6791/0 conn(0x7f0dd40a3fd0 legacy=0x7f0dd413be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 1 -- 172.21.15.115:0/1052607307 >> v1:172.21.15.115:6789/0 conn(0x7f0dd40a2c40 legacy=0x7f0dd40a3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 10 monclient: _finish_hunting 0 2024-01-24T23:29:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 1 monclient: found mon.noname-b 2024-01-24T23:29:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:29:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 1 -- 172.21.15.115:0/1052607307 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd40a1150 con 0x7f0dd40a06b0 2024-01-24T23:29:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 10 monclient: _finish_auth 0 2024-01-24T23:29:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 10 monclient: _check_auth_tickets 2024-01-24T23:29:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 1 -- 172.21.15.115:0/1052607307 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0dcc001a60 con 0x7f0dd40a06b0 2024-01-24T23:29:37.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd8e2f640 5 monclient: authenticate success, global_id 154208 2024-01-24T23:29:37.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd8e2f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:37.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:29:37.369 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 10 monclient: dump: 2024-01-24T23:29:37.369 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.369 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.369 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.369 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.369 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.369 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.369 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.369 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.369 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.369 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.369 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.370 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.370 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.370 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.370 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.370 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd8e2f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:37.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 1 -- 172.21.15.115:0/1052607307 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0dcc001cd0 con 0x7f0dd40a06b0 2024-01-24T23:29:37.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:37.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 1 -- 172.21.15.115:0/1052607307 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0dcc004c30 con 0x7f0dd40a06b0 2024-01-24T23:29:37.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd8e2f640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:37.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd8e2f640 4 set_mon_vals no callback set 2024-01-24T23:29:37.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.355+0000 7f0dd1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd1d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:29:37.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd1d9b640 10 monclient: dump: 2024-01-24T23:29:37.371 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.371 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.371 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.371 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.371 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.371 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.371 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.371 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.371 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.371 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.371 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.371 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.371 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.371 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.372 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.372 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 10 monclient: shutdown 2024-01-24T23:29:37.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:37.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 1 -- 172.21.15.115:0/1052607307 >> v1:172.21.15.115:6790/0 conn(0x7f0dd40a06b0 legacy=0x7f0dd413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 1 -- 172.21.15.115:0/1052607307 shutdown_connections 2024-01-24T23:29:37.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 1 -- 172.21.15.115:0/1052607307 >> 172.21.15.115:0/1052607307 conn(0x7f0dd40a9ba0 msgr2=0x7f0dd40a5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:37.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 1 -- 172.21.15.115:0/1052607307 shutdown_connections 2024-01-24T23:29:37.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 1 -- 172.21.15.115:0/1052607307 wait complete. 2024-01-24T23:29:37.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 10 monclient: build_initial_monmap 2024-01-24T23:29:37.372 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 10 monclient: monmap: 2024-01-24T23:29:37.372 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:37.372 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:37.372 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:37.373 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:37.373 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:37.373 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:37.373 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:37.373 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:37.373 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:37.373 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:37.373 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:37.373 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:37.373 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:37.373 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:37.373 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:37.373 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 1 Processor -- start 2024-01-24T23:29:37.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 1 -- start start 2024-01-24T23:29:37.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 10 monclient: init 2024-01-24T23:29:37.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:37.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 10 monclient: _add_conns ranks=[7,2,3,0,6,1,4,5,8] 2024-01-24T23:29:37.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 10 monclient(hunting): picked mon.noname-i con 0x7f0dd40a06b0 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:37.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 10 monclient(hunting): picked mon.noname-b con 0x7f0dd40a2c40 addr v1:172.21.15.115:6790/0 2024-01-24T23:29:37.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 10 monclient(hunting): picked mon.noname-g con 0x7f0dd40a3fd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:29:37.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd40a58a0 con 0x7f0dd40a2c40 2024-01-24T23:29:37.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd41a70e0 con 0x7f0dd40a3fd0 2024-01-24T23:29:37.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd41a9440 con 0x7f0dd40a06b0 2024-01-24T23:29:37.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:37.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 10 monclient(hunting): authenticate will time out at 1456.285969s 2024-01-24T23:29:37.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dc9d9b640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f0dd40a2c40 0x7f0dd41d0420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:43150/0 (socket says 172.21.15.115:43150) 2024-01-24T23:29:37.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dc9d9b640 1 -- 172.21.15.115:0/3853719607 learned_addr learned my addr 172.21.15.115:0/3853719607 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:37.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568763133 0 0) 0x7f0dd40a58a0 con 0x7f0dd40a2c40 2024-01-24T23:29:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 10 monclient(hunting): my global_id is 154217 2024-01-24T23:29:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0db4003530 con 0x7f0dd40a2c40 2024-01-24T23:29:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3181942174 0 0) 0x7f0db4003530 con 0x7f0dd40a2c40 2024-01-24T23:29:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0dd40a58a0 con 0x7f0dd40a2c40 2024-01-24T23:29:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0dbc001ac0 con 0x7f0dd40a2c40 2024-01-24T23:29:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:29:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 10 monclient(hunting): dump: 2024-01-24T23:29:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.376 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 929034296 0 0) 0x7f0dd40a58a0 con 0x7f0dd40a2c40 2024-01-24T23:29:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 >> v1:172.21.15.132:6792/0 conn(0x7f0dd40a06b0 legacy=0x7f0dd41bba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 >> v1:172.21.15.132:6790/0 conn(0x7f0dd40a3fd0 legacy=0x7f0dd41d3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:29:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 1 monclient: found mon.b 2024-01-24T23:29:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:29:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd41a8ba0 con 0x7f0dd40a2c40 2024-01-24T23:29:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 10 monclient: _finish_auth 0 2024-01-24T23:29:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:29:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 5 monclient: authenticate success, global_id 154217 2024-01-24T23:29:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 10 monclient: _renew_subs 2024-01-24T23:29:37.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:29:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 1 -- 172.21.15.115:0/3853719607 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd409fb00 con 0x7f0dd40a2c40 2024-01-24T23:29:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 10 monclient: _renew_subs 2024-01-24T23:29:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:29:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 1 -- 172.21.15.115:0/3853719607 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd41bc360 con 0x7f0dd40a2c40 2024-01-24T23:29:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0dbc0038b0 con 0x7f0dd40a2c40 2024-01-24T23:29:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0dbc003de0 con 0x7f0dd40a2c40 2024-01-24T23:29:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:29:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd159a640 4 set_mon_vals no callback set 2024-01-24T23:29:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 10 monclient: dump: 2024-01-24T23:29:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.378 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.379 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.379 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.379 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.379 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.379 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.379 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.379 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.379 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.379 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.379 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.379 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.379 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.379 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:37.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dd8e2f640 1 -- 172.21.15.115:0/3853719607 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0db80065d0 con 0x7f0db8002870 2024-01-24T23:29:37.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f0dbc003fc0 con 0x7f0dd40a2c40 2024-01-24T23:29:37.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.359+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7f0dbc044570 con 0x7f0dd40a2c40 2024-01-24T23:29:37.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.363+0000 7f0dd2d9d640 10 monclient: get_auth_request con 0x7f0db4036fd0 auth_method 0 2024-01-24T23:29:37.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.363+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1040707165 0 0) 0x7f0db80065d0 con 0x7f0db8002870 2024-01-24T23:29:37.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.363+0000 7f0dcb7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.363+0000 7f0dcb7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.363+0000 7f0dcb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.363+0000 7f0dcb7fe640 10 monclient(hunting): my global_id is 7185 2024-01-24T23:29:37.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.363+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dd40a58a0 con 0x7f0db8002870 2024-01-24T23:29:37.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.363+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2855510504 0 0) 0x7f0dd40a58a0 con 0x7f0db8002870 2024-01-24T23:29:37.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.363+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0db80065d0 con 0x7f0db8002870 2024-01-24T23:29:37.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.363+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0dc4001350 con 0x7f0db8002870 2024-01-24T23:29:37.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.363+0000 7f0dcb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.363+0000 7f0dcb7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:29:37.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.363+0000 7f0dcb7fe640 10 monclient: dump: 2024-01-24T23:29:37.380 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.381 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.381 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.381 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.381 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.381 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.381 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.381 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.381 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.381 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.381 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.381 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.381 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.381 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.381 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.381 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.363+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1374319863 0 0) 0x7f0db80065d0 con 0x7f0db8002870 2024-01-24T23:29:37.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.363+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 --> v1:172.21.15.132:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0db800a120 con 0x7f0db8002870 2024-01-24T23:29:37.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.363+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f0dc40017e0 con 0x7f0db8002870 2024-01-24T23:29:37.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.363+0000 7f0dcb7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:37.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.363+0000 7f0dcb7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:37.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.363+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 >> v1:172.21.15.132:6790/0 conn(0x7f0db8002870 legacy=0x7f0db8004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.363+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 >> v1:172.21.15.132:6790/0 conn(0x7f0db8002870 legacy=0x7f0db8004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:37.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.367+0000 7f0dd8e2f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:29:37.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.367+0000 7f0dd8e2f640 1 -- 172.21.15.115:0/3853719607 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0db8009e10 con 0x7f0db80090d0 2024-01-24T23:29:37.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 342737219 0 0) 0x7f0db8009e10 con 0x7f0db80090d0 2024-01-24T23:29:37.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dcb7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dcb7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dcb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dcb7fe640 10 monclient(hunting): my global_id is 7194 2024-01-24T23:29:37.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0db80065d0 con 0x7f0db80090d0 2024-01-24T23:29:37.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 244391420 0 0) 0x7f0db80065d0 con 0x7f0db80090d0 2024-01-24T23:29:37.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0db8009e10 con 0x7f0db80090d0 2024-01-24T23:29:37.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f0dc4004de0 con 0x7f0db80090d0 2024-01-24T23:29:37.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dcb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dcb7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:29:37.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dcb7fe640 10 monclient: dump: 2024-01-24T23:29:37.384 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.384 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.384 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.384 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.384 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.385 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.385 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.385 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.385 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.385 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.385 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.385 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.385 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.385 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.385 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.385 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3427385955 0 0) 0x7f0db8009e10 con 0x7f0db80090d0 2024-01-24T23:29:37.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 --> v1:172.21.15.132:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0db800c310 con 0x7f0db80090d0 2024-01-24T23:29:37.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7713 (unknown 0 0 3235141771) 0x7f0dc40070a0 con 0x7f0db80090d0 2024-01-24T23:29:37.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dcb7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:29:37.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dcb7fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:37.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 >> v1:172.21.15.132:6790/0 conn(0x7f0db80090d0 legacy=0x7f0db8009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dcb7fe640 1 -- 172.21.15.115:0/3853719607 >> v1:172.21.15.132:6790/0 conn(0x7f0db80090d0 legacy=0x7f0db8009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:37.386 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:29:37.386 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:37.386 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:37.386 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:29:37.386 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 48, 2024-01-24T23:29:37.386 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:29:37.386 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:29:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:29:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:29:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:29:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:29:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:29:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:29:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:29:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:29:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 7, 2024-01-24T23:29:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:29:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:29:37.388 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:37.388 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:37.388 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:37.388 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:37.388 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:37.388 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:37.388 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:37.388 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:37.388 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:37.388 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:37.388 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:37.388 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:37.388 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:29:37.389 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:29:37.389 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:37.389 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:37.389 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:37.389 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:37.389 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:37.389 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:37.389 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:37.389 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:37.389 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:37.390 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:37.390 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:37.390 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.390 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:29:37.390 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:29:37.390 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:29:37.390 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:29:37.390 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:29:37.390 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:29:37.390 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:37.390 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:37.390 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:29:37.390 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:29:37.391 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:29:37.391 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:29:37.391 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:29:37.391 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:29:37.391 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:29:37.391 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:37.391 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:29:37.391 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:37.391 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:37.391 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:37.391 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:37.391 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:37.391 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:37.392 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:37.392 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:37.392 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:37.392 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:37.392 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:37.392 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:37.392 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:29:37.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.392 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:29:37.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.392 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:37.392 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:37.392 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.392 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.393 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.393 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:29:37.393 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.393 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.393 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.393 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:29:37.393 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:29:37.393 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.393 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.393 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.394 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:37.394 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:37.394 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.394 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.394 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.394 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:29:37.394 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.394 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.394 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.394 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:29:37.394 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:29:37.394 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.394 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.395 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.395 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:37.395 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:37.395 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.395 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.395 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.395 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:29:37.395 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.395 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.395 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.395 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:29:37.396 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:29:37.396 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.396 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.396 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.396 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:37.396 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:37.396 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.396 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.396 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.396 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:29:37.397 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.397 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.397 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.397 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:29:37.397 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:29:37.397 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.397 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.397 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.397 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:37.397 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:37.397 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.397 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.398 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.398 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:29:37.398 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.398 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.398 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.398 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:29:37.398 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:29:37.398 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.398 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.398 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.399 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:37.399 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:37.399 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.399 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.399 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.399 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:29:37.399 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.399 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.399 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.399 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:29:37.399 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:29:37.399 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.400 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.400 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.400 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:37.400 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:37.400 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.400 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.400 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.400 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:29:37.400 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.400 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.400 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.401 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:29:37.401 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:29:37.401 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.401 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.401 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.401 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:37.401 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:37.401 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.401 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.401 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.401 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:29:37.402 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.402 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.402 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.402 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:29:37.402 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:29:37.402 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.402 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.402 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.402 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:37.402 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:37.402 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.402 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.403 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.403 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:29:37.403 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.403 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.403 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.403 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:29:37.403 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:29:37.403 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.403 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.403 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.403 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.403 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.404 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:29:37.404 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:29:37.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.404 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:37.404 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:37.404 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:37.404 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.404 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:37.404 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:29:37.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.404 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:37.404 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:37.404 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:29:37.404 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.405 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:37.405 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:29:37.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.405 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:37.405 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:37.405 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 4 2024-01-24T23:29:37.405 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.405 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.405 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:29:37.405 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:29:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dd8e2f640 1 -- 172.21.15.115:0/3853719607 >> v1:172.21.15.132:6812/1835401922 conn(0x7f0db4036fd0 legacy=0x7f0db4039490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dd8e2f640 10 monclient: shutdown 2024-01-24T23:29:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dd8e2f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dd8e2f640 1 -- 172.21.15.115:0/3853719607 >> v1:172.21.15.115:6790/0 conn(0x7f0dd40a2c40 legacy=0x7f0dd41d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dd2d9d640 1 -- 172.21.15.115:0/3853719607 reap_dead start 2024-01-24T23:29:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dd2d9d640 1 -- 172.21.15.115:0/3853719607 reap_dead start 2024-01-24T23:29:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dd8e2f640 1 -- 172.21.15.115:0/3853719607 shutdown_connections 2024-01-24T23:29:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dd8e2f640 1 -- 172.21.15.115:0/3853719607 >> 172.21.15.115:0/3853719607 conn(0x7f0dd40a9ba0 msgr2=0x7f0dd41be890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dd8e2f640 1 -- 172.21.15.115:0/3853719607 shutdown_connections 2024-01-24T23:29:37.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.371+0000 7f0dd8e2f640 1 -- 172.21.15.115:0/3853719607 wait complete. 2024-01-24T23:29:37.408 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:29:37.408 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-01-24T23:29:37.408 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-01-24T23:29:37.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f14697ee640 10 monclient: get_monmap_and_config 2024-01-24T23:29:37.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f14697ee640 10 monclient: build_initial_monmap 2024-01-24T23:29:37.521 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f14697ee640 10 monclient: monmap: 2024-01-24T23:29:37.521 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:37.521 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.522 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:37.524173+0000 2024-01-24T23:29:37.522 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:37.524173+0000 2024-01-24T23:29:37.522 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:37.522 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.522 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:37.522 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:37.522 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:37.522 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:37.522 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:37.522 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:37.522 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:37.522 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:37.522 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:37.522 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f14697ee640 1 Processor -- start 2024-01-24T23:29:37.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f14697ee640 1 -- start start 2024-01-24T23:29:37.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f14697ee640 10 monclient: init 2024-01-24T23:29:37.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f14697ee640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:37.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f14697ee640 10 monclient: _add_conns ranks=[1,4,3,0,5,2,8,6,7] 2024-01-24T23:29:37.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f14697ee640 10 monclient(hunting): picked mon.noname-f con 0x7f1464133c10 addr v1:172.21.15.132:6789/0 2024-01-24T23:29:37.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f14697ee640 10 monclient(hunting): picked mon.noname-c con 0x7f1464136960 addr v1:172.21.15.115:6791/0 2024-01-24T23:29:37.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f14697ee640 10 monclient(hunting): picked mon.noname-g con 0x7f146413a470 addr v1:172.21.15.132:6790/0 2024-01-24T23:29:37.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f14697ee640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f146404ed70 con 0x7f1464133c10 2024-01-24T23:29:37.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f14697ee640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f146409dee0 con 0x7f146413a470 2024-01-24T23:29:37.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f14697ee640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f146409d400 con 0x7f1464136960 2024-01-24T23:29:37.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f14697ee640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:37.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f14697ee640 10 monclient(hunting): authenticate will time out at 1456.446871s 2024-01-24T23:29:37.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f14627fc640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7f1464136960 0x7f1464138d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:59042/0 (socket says 172.21.15.115:59042) 2024-01-24T23:29:37.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f14627fc640 1 -- 172.21.15.115:0/700982500 learned_addr learned my addr 172.21.15.115:0/700982500 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:37.533 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 1 -- 172.21.15.115:0/700982500 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 295389633 0 0) 0x7f146409d400 con 0x7f1464136960 2024-01-24T23:29:37.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 10 monclient(hunting): my global_id is 95359 2024-01-24T23:29:37.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 1 -- 172.21.15.115:0/700982500 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f144c003530 con 0x7f1464136960 2024-01-24T23:29:37.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 1 -- 172.21.15.115:0/700982500 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 824405938 0 0) 0x7f144c003530 con 0x7f1464136960 2024-01-24T23:29:37.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 1 -- 172.21.15.115:0/700982500 >> v1:172.21.15.132:6790/0 conn(0x7f146413a470 legacy=0x7f146413c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 1 -- 172.21.15.115:0/700982500 >> v1:172.21.15.132:6789/0 conn(0x7f1464133c10 legacy=0x7f1464133ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:29:37.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 1 monclient: found mon.noname-c 2024-01-24T23:29:37.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:37.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:29:37.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 1 -- 172.21.15.115:0/700982500 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146409f380 con 0x7f1464136960 2024-01-24T23:29:37.534 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 10 monclient: _finish_auth 0 2024-01-24T23:29:37.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:29:37.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:37.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 1 -- 172.21.15.115:0/700982500 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f14580017c0 con 0x7f1464136960 2024-01-24T23:29:37.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:29:37.535 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 10 monclient: dump: 2024-01-24T23:29:37.535 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.535 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.535 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.535 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.535 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.535 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.535 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.535 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.535 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.536 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.536 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.536 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.536 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.536 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.536 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.536 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f14697ee640 5 monclient: authenticate success, global_id 95359 2024-01-24T23:29:37.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f14697ee640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:37.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 1 -- 172.21.15.115:0/700982500 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1458001a70 con 0x7f1464136960 2024-01-24T23:29:37.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:37.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f14697ee640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:37.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 1 -- 172.21.15.115:0/700982500 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1458004bf0 con 0x7f1464136960 2024-01-24T23:29:37.536 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f14697ee640 4 set_mon_vals no callback set 2024-01-24T23:29:37.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:29:37.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.519+0000 7f1461ffb640 10 monclient: dump: 2024-01-24T23:29:37.537 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.537 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.537 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.537 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.537 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.537 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.537 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.537 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.537 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.537 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.537 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.537 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.537 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.538 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.538 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.538 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 10 monclient: shutdown 2024-01-24T23:29:37.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:37.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 1 -- 172.21.15.115:0/700982500 >> v1:172.21.15.115:6791/0 conn(0x7f1464136960 legacy=0x7f1464138d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 1 -- 172.21.15.115:0/700982500 shutdown_connections 2024-01-24T23:29:37.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 1 -- 172.21.15.115:0/700982500 >> 172.21.15.115:0/700982500 conn(0x7f14640a5730 msgr2=0x7f14640a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:37.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 1 -- 172.21.15.115:0/700982500 shutdown_connections 2024-01-24T23:29:37.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 1 -- 172.21.15.115:0/700982500 wait complete. 2024-01-24T23:29:37.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 10 monclient: build_initial_monmap 2024-01-24T23:29:37.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 10 monclient: monmap: 2024-01-24T23:29:37.538 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:37.538 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:37.538 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:37.539 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:37.539 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:37.539 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:37.539 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:37.539 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:37.539 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:37.539 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:37.539 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:37.539 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:37.539 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:37.539 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:37.539 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:37.539 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 1 Processor -- start 2024-01-24T23:29:37.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 1 -- start start 2024-01-24T23:29:37.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 10 monclient: init 2024-01-24T23:29:37.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:37.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 10 monclient: _add_conns ranks=[6,7,8,5,1,3,0,4,2] 2024-01-24T23:29:37.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 10 monclient(hunting): picked mon.noname-d con 0x7f1464133c10 addr v1:172.21.15.115:6792/0 2024-01-24T23:29:37.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 10 monclient(hunting): picked mon.noname-i con 0x7f1464136960 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:37.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 10 monclient(hunting): picked mon.noname-e con 0x7f146413a470 addr v1:172.21.15.115:6793/0 2024-01-24T23:29:37.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14641310c0 con 0x7f1464133c10 2024-01-24T23:29:37.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1464133550 con 0x7f1464136960 2024-01-24T23:29:37.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f146412c460 con 0x7f146413a470 2024-01-24T23:29:37.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:37.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 10 monclient(hunting): authenticate will time out at 1456.449745s 2024-01-24T23:29:37.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f1462ffd640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f1464133c10 0x7f1464132e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:53794/0 (socket says 172.21.15.115:53794) 2024-01-24T23:29:37.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f1462ffd640 1 -- 172.21.15.115:0/401805729 learned_addr learned my addr 172.21.15.115:0/401805729 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:37.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1783428643 0 0) 0x7f14641310c0 con 0x7f1464133c10 2024-01-24T23:29:37.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient(hunting): my global_id is 7377 2024-01-24T23:29:37.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1450003530 con 0x7f1464133c10 2024-01-24T23:29:37.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2253220421 0 0) 0x7f146412c460 con 0x7f146413a470 2024-01-24T23:29:37.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient(hunting): my global_id is 86453 2024-01-24T23:29:37.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14641310c0 con 0x7f146413a470 2024-01-24T23:29:37.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 485008948 0 0) 0x7f1450003530 con 0x7f1464133c10 2024-01-24T23:29:37.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f146412c460 con 0x7f1464133c10 2024-01-24T23:29:37.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1644231194 0 0) 0x7f14641310c0 con 0x7f146413a470 2024-01-24T23:29:37.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1450003530 con 0x7f146413a470 2024-01-24T23:29:37.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1454001e40 con 0x7f1464133c10 2024-01-24T23:29:37.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:29:37.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient(hunting): dump: 2024-01-24T23:29:37.542 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.542 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.542 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.542 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.542 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.542 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.542 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.542 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.542 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.542 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.542 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.543 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.543 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.543 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.543 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.543 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:37.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f145c001880 con 0x7f146413a470 2024-01-24T23:29:37.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-01-24T23:29:37.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient(hunting): dump: 2024-01-24T23:29:37.543 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.543 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.543 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.543 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.543 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.544 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.544 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.544 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.544 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.544 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.544 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.544 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.544 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.544 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.544 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.544 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1536731277 0 0) 0x7f1464133550 con 0x7f1464136960 2024-01-24T23:29:37.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient(hunting): my global_id is 154258 2024-01-24T23:29:37.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14641310c0 con 0x7f1464136960 2024-01-24T23:29:37.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 394173506 0 0) 0x7f146412c460 con 0x7f1464133c10 2024-01-24T23:29:37.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 >> v1:172.21.15.115:6793/0 conn(0x7f146413a470 legacy=0x7f146412bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 >> v1:172.21.15.132:6792/0 conn(0x7f1464136960 legacy=0x7f146412b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient: _finish_hunting 0 2024-01-24T23:29:37.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 1 monclient: found mon.d 2024-01-24T23:29:37.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:37.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:29:37.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14640a1310 con 0x7f1464133c10 2024-01-24T23:29:37.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient: _finish_auth 0 2024-01-24T23:29:37.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient: _check_auth_tickets 2024-01-24T23:29:37.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:37.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 5 monclient: authenticate success, global_id 7377 2024-01-24T23:29:37.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 10 monclient: _renew_subs 2024-01-24T23:29:37.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:29:37.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 1 -- 172.21.15.115:0/401805729 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1464139820 con 0x7f1464133c10 2024-01-24T23:29:37.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 10 monclient: _renew_subs 2024-01-24T23:29:37.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:29:37.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 1 -- 172.21.15.115:0/401805729 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1464138f00 con 0x7f1464133c10 2024-01-24T23:29:37.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1454001ff0 con 0x7f1464133c10 2024-01-24T23:29:37.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:37.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1454004010 con 0x7f1464133c10 2024-01-24T23:29:37.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14617fa640 4 set_mon_vals no callback set 2024-01-24T23:29:37.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:29:37.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 10 monclient: dump: 2024-01-24T23:29:37.547 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.547 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.547 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.547 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.547 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.547 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.547 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.547 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.547 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.547 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.547 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.547 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.548 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.548 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.548 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.548 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:37.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f14697ee640 1 -- 172.21.15.115:0/401805729 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f14440065d0 con 0x7f1444002870 2024-01-24T23:29:37.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f1454004230 con 0x7f1464133c10 2024-01-24T23:29:37.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.523+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7f1454044810 con 0x7f1464133c10 2024-01-24T23:29:37.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.527+0000 7f14637fe640 10 monclient: get_auth_request con 0x7f14500377d0 auth_method 0 2024-01-24T23:29:37.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.527+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276607479 0 0) 0x7f14440065d0 con 0x7f1444002870 2024-01-24T23:29:37.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.527+0000 7f144b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.527+0000 7f144b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.527+0000 7f144b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.527+0000 7f144b7fe640 10 monclient(hunting): my global_id is 7151 2024-01-24T23:29:37.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.527+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f146412c460 con 0x7f1444002870 2024-01-24T23:29:37.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.527+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2079414302 0 0) 0x7f146412c460 con 0x7f1444002870 2024-01-24T23:29:37.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.527+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f14440065d0 con 0x7f1444002870 2024-01-24T23:29:37.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.527+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1458004b30 con 0x7f1444002870 2024-01-24T23:29:37.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.527+0000 7f144b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.527+0000 7f144b7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:29:37.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.527+0000 7f144b7fe640 10 monclient: dump: 2024-01-24T23:29:37.549 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.549 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.549 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.549 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.549 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.549 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.549 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.550 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.550 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.550 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.550 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.550 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.550 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.550 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.550 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.550 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.527+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3074025596 0 0) 0x7f14440065d0 con 0x7f1444002870 2024-01-24T23:29:37.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.527+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 --> v1:172.21.15.132:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f144400a120 con 0x7f1444002870 2024-01-24T23:29:37.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.527+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f1458001670 con 0x7f1444002870 2024-01-24T23:29:37.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.527+0000 7f144b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:37.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.527+0000 7f144b7fe640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:37.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.527+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 >> v1:172.21.15.132:6791/0 conn(0x7f1444002870 legacy=0x7f1444004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.527+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 >> v1:172.21.15.132:6791/0 conn(0x7f1444002870 legacy=0x7f1444004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:37.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f14697ee640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:29:37.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f14697ee640 1 -- 172.21.15.115:0/401805729 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1444009e10 con 0x7f14440090d0 2024-01-24T23:29:37.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3944284539 0 0) 0x7f1444009e10 con 0x7f14440090d0 2024-01-24T23:29:37.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f144b7fe640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f144b7fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f144b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f144b7fe640 10 monclient(hunting): my global_id is 7160 2024-01-24T23:29:37.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14440065d0 con 0x7f14440090d0 2024-01-24T23:29:37.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 709556195 0 0) 0x7f14440065d0 con 0x7f14440090d0 2024-01-24T23:29:37.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1444009e10 con 0x7f14440090d0 2024-01-24T23:29:37.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f14580053c0 con 0x7f14440090d0 2024-01-24T23:29:37.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f144b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f144b7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:29:37.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f144b7fe640 10 monclient: dump: 2024-01-24T23:29:37.553 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.554 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.554 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.554 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.554 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.554 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.554 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.554 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.554 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.554 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.554 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.554 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.554 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.554 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.554 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.555 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1048397884 0 0) 0x7f1444009e10 con 0x7f14440090d0 2024-01-24T23:29:37.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 --> v1:172.21.15.132:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f144400c310 con 0x7f14440090d0 2024-01-24T23:29:37.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7715 (unknown 0 0 1502427132) 0x7f14580076a0 con 0x7f14440090d0 2024-01-24T23:29:37.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f144b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:29:37.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f144b7fe640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:37.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 >> v1:172.21.15.132:6791/0 conn(0x7f14440090d0 legacy=0x7f1444009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f144b7fe640 1 -- 172.21.15.115:0/401805729 >> v1:172.21.15.132:6791/0 conn(0x7f14440090d0 legacy=0x7f1444009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:37.555 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:29:37.555 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:37.555 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:37.556 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "leader", 2024-01-24T23:29:37.556 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 48, 2024-01-24T23:29:37.556 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:29:37.556 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:29:37.556 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:29:37.556 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:29:37.556 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:29:37.556 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:29:37.556 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:29:37.556 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:29:37.556 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:29:37.556 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:29:37.556 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:37.557 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 7, 2024-01-24T23:29:37.557 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:37.557 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:29:37.557 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:29:37.557 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:37.557 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:37.557 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:37.557 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:37.557 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:37.557 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:37.557 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:37.557 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:37.557 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:37.557 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:37.558 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:37.558 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:37.558 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:29:37.558 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:29:37.558 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:37.558 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:37.558 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:37.558 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:37.558 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:37.558 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:37.558 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:37.558 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:37.558 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:37.559 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:37.559 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:37.559 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.559 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:29:37.559 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:29:37.559 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:29:37.559 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:29:37.559 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:29:37.559 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:29:37.559 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:37.559 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:37.559 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:29:37.559 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:29:37.559 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:29:37.560 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:29:37.560 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:29:37.560 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:29:37.560 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:29:37.560 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:37.560 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:29:37.560 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:37.560 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:37.560 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:37.560 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:37.560 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:37.560 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:37.560 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:37.561 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:37.561 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:37.561 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:37.561 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:37.561 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:37.561 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:29:37.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.561 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:29:37.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.561 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:37.561 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:37.561 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.561 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.561 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.562 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:29:37.562 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.562 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.562 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.562 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:29:37.562 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:29:37.562 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.562 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.562 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.562 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:37.562 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:37.563 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.563 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.563 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.563 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:29:37.563 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.563 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.563 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.563 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:29:37.563 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:29:37.563 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.563 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.563 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.564 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:37.564 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:37.564 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.564 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.564 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.564 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:29:37.564 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.564 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.564 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.565 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:29:37.565 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:29:37.565 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.565 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.565 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.565 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:37.565 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:37.565 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.565 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.565 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.565 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:29:37.565 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.566 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.566 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.566 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:29:37.566 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:29:37.566 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.566 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.566 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.566 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:37.566 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:37.566 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.567 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.567 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.567 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:29:37.567 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.567 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.567 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.567 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:29:37.567 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:29:37.567 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.567 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.567 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.568 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:37.568 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:37.568 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.568 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.568 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.568 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:29:37.568 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.568 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.568 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.568 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:29:37.568 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:29:37.569 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.569 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.569 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.569 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:37.569 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:37.569 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.569 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.569 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.569 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:29:37.569 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.569 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.569 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.570 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:29:37.570 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:29:37.570 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.570 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.570 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.570 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:37.570 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:37.570 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.570 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.570 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.571 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:29:37.571 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.571 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.571 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.571 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:29:37.571 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:29:37.571 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.571 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.571 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.571 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:37.571 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:37.571 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.572 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.572 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.572 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:29:37.572 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.572 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.572 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.572 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:29:37.572 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:29:37.572 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.572 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.572 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.572 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.573 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.573 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:29:37.573 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:29:37.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.573 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:37.573 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:37.573 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:37.573 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.573 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:37.573 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:29:37.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.573 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:37.573 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:37.573 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:29:37.574 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.574 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:37.574 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": [ 2024-01-24T23:29:37.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.574 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:37.574 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:37.574 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:37.574 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.574 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.574 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:29:37.574 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:29:37.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f14697ee640 1 -- 172.21.15.115:0/401805729 >> v1:172.21.15.132:6812/1835401922 conn(0x7f14500377d0 legacy=0x7f1450039c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f14697ee640 10 monclient: shutdown 2024-01-24T23:29:37.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f14697ee640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:37.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f14697ee640 1 -- 172.21.15.115:0/401805729 >> v1:172.21.15.115:6792/0 conn(0x7f1464133c10 legacy=0x7f1464132e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.535+0000 7f14637fe640 1 -- 172.21.15.115:0/401805729 reap_dead start 2024-01-24T23:29:37.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.539+0000 7f14637fe640 1 -- 172.21.15.115:0/401805729 reap_dead start 2024-01-24T23:29:37.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.539+0000 7f14697ee640 1 -- 172.21.15.115:0/401805729 shutdown_connections 2024-01-24T23:29:37.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.539+0000 7f14697ee640 1 -- 172.21.15.115:0/401805729 >> 172.21.15.115:0/401805729 conn(0x7f14640a5730 msgr2=0x7f146412e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:37.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.539+0000 7f14697ee640 1 -- 172.21.15.115:0/401805729 shutdown_connections 2024-01-24T23:29:37.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.539+0000 7f14697ee640 1 -- 172.21.15.115:0/401805729 wait complete. 2024-01-24T23:29:37.577 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:29:37.577 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-01-24T23:29:37.577 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-01-24T23:29:37.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.687+0000 7fc82b68e640 10 monclient: get_monmap_and_config 2024-01-24T23:29:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.687+0000 7fc82b68e640 10 monclient: build_initial_monmap 2024-01-24T23:29:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.687+0000 7fc82b68e640 10 monclient: monmap: 2024-01-24T23:29:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:37.694607+0000 2024-01-24T23:29:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:37.694607+0000 2024-01-24T23:29:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:37.692 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:37.693 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:37.693 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:37.693 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:37.693 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:37.693 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:37.693 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 1 Processor -- start 2024-01-24T23:29:37.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 1 -- start start 2024-01-24T23:29:37.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 10 monclient: init 2024-01-24T23:29:37.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:37.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 10 monclient: _add_conns ranks=[6,0,4,3,7,5,8,1,2] 2024-01-24T23:29:37.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 10 monclient(hunting): picked mon.noname-d con 0x7fc82409ebf0 addr v1:172.21.15.115:6792/0 2024-01-24T23:29:37.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 10 monclient(hunting): picked mon.noname-a con 0x7fc82409efd0 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:37.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 10 monclient(hunting): picked mon.noname-c con 0x7fc82413b220 addr v1:172.21.15.115:6791/0 2024-01-24T23:29:37.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc82404ed70 con 0x7fc82409efd0 2024-01-24T23:29:37.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc82409d400 con 0x7fc82413b220 2024-01-24T23:29:37.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc82409f800 con 0x7fc82409ebf0 2024-01-24T23:29:37.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:37.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 10 monclient(hunting): authenticate will time out at 1456.617159s 2024-01-24T23:29:37.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc828c29640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fc82409efd0 0x7fc824139b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39282/0 (socket says 172.21.15.115:39282) 2024-01-24T23:29:37.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc828c29640 1 -- 172.21.15.115:0/731696015 learned_addr learned my addr 172.21.15.115:0/731696015 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:37.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82942a640 1 --1- 172.21.15.115:0/731696015 >> v1:172.21.15.115:6792/0 conn(0x7fc82409ebf0 0x7fc824134cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:53796/0 (socket says 172.21.15.115:53796) 2024-01-24T23:29:37.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 1 -- 172.21.15.115:0/731696015 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1844546915 0 0) 0x7fc82404ed70 con 0x7fc82409efd0 2024-01-24T23:29:37.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 10 monclient(hunting): my global_id is 154305 2024-01-24T23:29:37.701 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 1 -- 172.21.15.115:0/731696015 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc810003530 con 0x7fc82409efd0 2024-01-24T23:29:37.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 1 -- 172.21.15.115:0/731696015 <== mon.4 v1:172.21.15.115:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4125779517 0 0) 0x7fc82409d400 con 0x7fc82413b220 2024-01-24T23:29:37.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 10 monclient(hunting): my global_id is 95368 2024-01-24T23:29:37.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 1 -- 172.21.15.115:0/731696015 --> v1:172.21.15.115:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc82404ed70 con 0x7fc82413b220 2024-01-24T23:29:37.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 1 -- 172.21.15.115:0/731696015 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3258539942 0 0) 0x7fc82409f800 con 0x7fc82409ebf0 2024-01-24T23:29:37.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 10 monclient(hunting): my global_id is 7386 2024-01-24T23:29:37.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 1 -- 172.21.15.115:0/731696015 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc82409d400 con 0x7fc82409ebf0 2024-01-24T23:29:37.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 1 -- 172.21.15.115:0/731696015 <== mon.4 v1:172.21.15.115:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2671786186 0 0) 0x7fc82404ed70 con 0x7fc82413b220 2024-01-24T23:29:37.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 1 -- 172.21.15.115:0/731696015 >> v1:172.21.15.115:6792/0 conn(0x7fc82409ebf0 legacy=0x7fc824134cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.702 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 1 -- 172.21.15.115:0/731696015 >> v1:172.21.15.115:6789/0 conn(0x7fc82409efd0 legacy=0x7fc824139b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:29:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 1 monclient: found mon.noname-c 2024-01-24T23:29:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.115:6791/0 2024-01-24T23:29:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 1 -- 172.21.15.115:0/731696015 --> v1:172.21.15.115:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82409cb10 con 0x7fc82413b220 2024-01-24T23:29:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 10 monclient: _finish_auth 0 2024-01-24T23:29:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:29:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 1 -- 172.21.15.115:0/731696015 <== mon.4 v1:172.21.15.115:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc81c001a50 con 0x7fc82413b220 2024-01-24T23:29:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-01-24T23:29:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 10 monclient: dump: 2024-01-24T23:29:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.703 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.704 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.704 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.704 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.704 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.704 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.704 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.704 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.704 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.704 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.704 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.704 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.704 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.704 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 5 monclient: authenticate success, global_id 95368 2024-01-24T23:29:37.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:37.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 1 -- 172.21.15.115:0/731696015 <== mon.4 v1:172.21.15.115:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc81c001cf0 con 0x7fc82413b220 2024-01-24T23:29:37.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:37.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:37.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 4 set_mon_vals no callback set 2024-01-24T23:29:37.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 1 -- 172.21.15.115:0/731696015 <== mon.4 v1:172.21.15.115:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc81c0039a0 con 0x7fc82413b220 2024-01-24T23:29:37.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-01-24T23:29:37.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc81bfff640 10 monclient: dump: 2024-01-24T23:29:37.705 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.705 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.705 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.705 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.705 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.705 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.705 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.706 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.706 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.706 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.706 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.706 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.706 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.706 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.706 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.706 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 10 monclient: shutdown 2024-01-24T23:29:37.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:37.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 1 -- 172.21.15.115:0/731696015 >> v1:172.21.15.115:6791/0 conn(0x7fc82413b220 legacy=0x7fc82413d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 1 -- 172.21.15.115:0/731696015 shutdown_connections 2024-01-24T23:29:37.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 1 -- 172.21.15.115:0/731696015 >> 172.21.15.115:0/731696015 conn(0x7fc8240a2110 msgr2=0x7fc8240a24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:37.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 1 -- 172.21.15.115:0/731696015 shutdown_connections 2024-01-24T23:29:37.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 1 -- 172.21.15.115:0/731696015 wait complete. 2024-01-24T23:29:37.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 10 monclient: build_initial_monmap 2024-01-24T23:29:37.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 10 monclient: monmap: 2024-01-24T23:29:37.707 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:37.707 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:37.707 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:37.707 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:37.707 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:37.707 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:37.707 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:37.707 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:37.707 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:37.707 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:37.707 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:37.708 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:37.708 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:37.708 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:37.708 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:37.708 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 1 Processor -- start 2024-01-24T23:29:37.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 1 -- start start 2024-01-24T23:29:37.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 10 monclient: init 2024-01-24T23:29:37.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:37.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 10 monclient: _add_conns ranks=[5,2,3,0,6,7,1,8,4] 2024-01-24T23:29:37.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 10 monclient(hunting): picked mon.noname-h con 0x7fc82409ebf0 addr v1:172.21.15.132:6791/0 2024-01-24T23:29:37.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 10 monclient(hunting): picked mon.noname-b con 0x7fc82409efd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:29:37.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 10 monclient(hunting): picked mon.noname-g con 0x7fc82413b220 addr v1:172.21.15.132:6790/0 2024-01-24T23:29:37.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc824066700 con 0x7fc82409efd0 2024-01-24T23:29:37.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc82409a980 con 0x7fc82413b220 2024-01-24T23:29:37.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc824070a20 con 0x7fc82409ebf0 2024-01-24T23:29:37.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:37.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.691+0000 7fc82b68e640 10 monclient(hunting): authenticate will time out at 1456.619949s 2024-01-24T23:29:37.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc828c29640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fc82409efd0 0x7fc824099d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:43156/0 (socket says 172.21.15.115:43156) 2024-01-24T23:29:37.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc828c29640 1 -- 172.21.15.115:0/1312260794 learned_addr learned my addr 172.21.15.115:0/1312260794 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:37.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4146194712 0 0) 0x7fc824066700 con 0x7fc82409efd0 2024-01-24T23:29:37.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient(hunting): my global_id is 154226 2024-01-24T23:29:37.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc808003530 con 0x7fc82409efd0 2024-01-24T23:29:37.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1792392641 0 0) 0x7fc808003530 con 0x7fc82409efd0 2024-01-24T23:29:37.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc824066700 con 0x7fc82409efd0 2024-01-24T23:29:37.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc820001560 con 0x7fc82409efd0 2024-01-24T23:29:37.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:29:37.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient(hunting): dump: 2024-01-24T23:29:37.710 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.710 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.710 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.710 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.710 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.710 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.710 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.710 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.710 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.710 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.710 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.710 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.711 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.711 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.711 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.711 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:37.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3515225780 0 0) 0x7fc824066700 con 0x7fc82409efd0 2024-01-24T23:29:37.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 >> v1:172.21.15.132:6791/0 conn(0x7fc82409ebf0 legacy=0x7fc824099610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 >> v1:172.21.15.132:6790/0 conn(0x7fc82413b220 legacy=0x7fc8241d3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:29:37.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 1 monclient: found mon.b 2024-01-24T23:29:37.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:37.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:29:37.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc824072ba0 con 0x7fc82409efd0 2024-01-24T23:29:37.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient: _finish_auth 0 2024-01-24T23:29:37.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:29:37.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:37.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc82b68e640 5 monclient: authenticate success, global_id 154226 2024-01-24T23:29:37.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc82b68e640 10 monclient: _renew_subs 2024-01-24T23:29:37.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc82b68e640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:29:37.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc82b68e640 1 -- 172.21.15.115:0/1312260794 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc82406ff60 con 0x7fc82409efd0 2024-01-24T23:29:37.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc82b68e640 10 monclient: _renew_subs 2024-01-24T23:29:37.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc82b68e640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:29:37.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc82b68e640 1 -- 172.21.15.115:0/1312260794 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc82406fc80 con 0x7fc82409efd0 2024-01-24T23:29:37.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc820004aa0 con 0x7fc82409efd0 2024-01-24T23:29:37.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:37.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc820004f50 con 0x7fc82409efd0 2024-01-24T23:29:37.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc81b7fe640 4 set_mon_vals no callback set 2024-01-24T23:29:37.712 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:29:37.713 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient: dump: 2024-01-24T23:29:37.713 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.713 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.713 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.713 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.713 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.713 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.713 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.713 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.713 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.713 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.713 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.713 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.713 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.713 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.713 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.714 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc82b68e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc82b68e640 1 -- 172.21.15.115:0/1312260794 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc8040065d0 con 0x7fc804002870 2024-01-24T23:29:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fc82000f9d0 con 0x7fc82409efd0 2024-01-24T23:29:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7fc8200445e0 con 0x7fc82409efd0 2024-01-24T23:29:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc829c2b640 10 monclient: get_auth_request con 0x7fc808036fd0 auth_method 0 2024-01-24T23:29:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 293370956 0 0) 0x7fc8040065d0 con 0x7fc804002870 2024-01-24T23:29:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient(hunting): my global_id is 154267 2024-01-24T23:29:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc824066700 con 0x7fc804002870 2024-01-24T23:29:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2448030686 0 0) 0x7fc824066700 con 0x7fc804002870 2024-01-24T23:29:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc8040065d0 con 0x7fc804002870 2024-01-24T23:29:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc814001220 con 0x7fc804002870 2024-01-24T23:29:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:29:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient: dump: 2024-01-24T23:29:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.716 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.716 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.716 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4292880351 0 0) 0x7fc8040065d0 con 0x7fc804002870 2024-01-24T23:29:37.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 --> v1:172.21.15.132:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc80400a120 con 0x7fc804002870 2024-01-24T23:29:37.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 <== mon.7 v1:172.21.15.132:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fc814001650 con 0x7fc804002870 2024-01-24T23:29:37.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:37.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:37.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 >> v1:172.21.15.132:6792/0 conn(0x7fc804002870 legacy=0x7fc804004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.716 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.695+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 >> v1:172.21.15.132:6792/0 conn(0x7fc804002870 legacy=0x7fc804004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:37.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.703+0000 7fc82b68e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:29:37.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.703+0000 7fc82b68e640 1 -- 172.21.15.115:0/1312260794 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc804009e10 con 0x7fc8040090d0 2024-01-24T23:29:37.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.703+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1272666870 0 0) 0x7fc804009e10 con 0x7fc8040090d0 2024-01-24T23:29:37.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.703+0000 7fc819ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.703+0000 7fc819ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.703+0000 7fc819ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.703+0000 7fc819ffb640 10 monclient(hunting): my global_id is 154276 2024-01-24T23:29:37.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.703+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8040065d0 con 0x7fc8040090d0 2024-01-24T23:29:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.703+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4249850074 0 0) 0x7fc8040065d0 con 0x7fc8040090d0 2024-01-24T23:29:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.703+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc804009e10 con 0x7fc8040090d0 2024-01-24T23:29:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.703+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fc814004da0 con 0x7fc8040090d0 2024-01-24T23:29:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.703+0000 7fc819ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.703+0000 7fc819ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:29:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.703+0000 7fc819ffb640 10 monclient: dump: 2024-01-24T23:29:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.720 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.720 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.720 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.720 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.720 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.720 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.720 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.703+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2880161370 0 0) 0x7fc804009e10 con 0x7fc8040090d0 2024-01-24T23:29:37.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.703+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 --> v1:172.21.15.132:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc80400c310 con 0x7fc8040090d0 2024-01-24T23:29:37.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.707+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 <== mon.7 v1:172.21.15.132:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7543 (unknown 0 0 1058213453) 0x7fc814005230 con 0x7fc8040090d0 2024-01-24T23:29:37.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.707+0000 7fc819ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:29:37.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.707+0000 7fc819ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:37.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.707+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 >> v1:172.21.15.132:6792/0 conn(0x7fc8040090d0 legacy=0x7fc804009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.707+0000 7fc819ffb640 1 -- 172.21.15.115:0/1312260794 >> v1:172.21.15.132:6792/0 conn(0x7fc8040090d0 legacy=0x7fc804009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:37.721 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:29:37.721 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:37.721 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:37.721 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:29:37.721 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 48, 2024-01-24T23:29:37.721 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:29:37.721 INFO:teuthology.orchestra.run.smithi115.stdout: 0, 2024-01-24T23:29:37.721 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-01-24T23:29:37.721 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:29:37.721 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:29:37.722 INFO:teuthology.orchestra.run.smithi115.stdout: 4, 2024-01-24T23:29:37.722 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:29:37.722 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:29:37.722 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:29:37.722 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:29:37.722 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:37.722 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 8, 2024-01-24T23:29:37.722 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:37.722 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:29:37.722 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:29:37.722 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:37.722 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:37.722 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:37.722 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:37.723 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:37.723 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:37.723 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:37.723 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:37.723 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:37.723 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:37.723 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:37.723 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:37.723 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:29:37.723 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:29:37.723 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:37.723 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:37.723 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:37.723 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:37.723 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:37.724 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:37.724 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:37.724 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:37.724 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:37.724 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:37.724 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:37.724 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.724 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:29:37.724 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:29:37.724 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:29:37.724 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:29:37.724 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:29:37.725 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:29:37.725 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:37.725 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:37.725 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:29:37.725 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:29:37.725 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:29:37.725 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:29:37.725 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:29:37.725 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:29:37.725 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:29:37.725 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:37.725 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:29:37.725 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:37.726 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:37.726 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:37.726 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:37.726 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:37.726 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:37.726 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:37.726 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:37.726 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:37.726 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:37.726 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:37.726 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:37.726 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:29:37.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.726 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:29:37.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.727 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:37.727 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:37.727 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.727 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.727 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.727 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:29:37.727 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.727 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.727 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.727 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:29:37.727 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:29:37.728 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.728 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.728 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.728 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:37.728 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:37.728 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.728 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.728 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.728 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:29:37.728 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.728 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.729 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.729 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:29:37.729 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:29:37.729 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.729 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.729 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.729 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:37.729 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:37.729 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.729 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.730 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.730 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:29:37.730 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.730 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.730 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:29:37.730 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:29:37.730 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.730 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.730 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.730 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:37.730 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:37.731 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.731 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.731 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.731 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:29:37.731 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.731 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.731 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:29:37.731 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:29:37.731 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.731 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.731 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.732 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:37.732 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:37.732 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.732 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.732 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.732 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:29:37.732 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.732 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.732 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.732 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:29:37.732 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:29:37.733 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.733 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.733 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.733 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:37.733 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:37.733 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.733 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.733 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.733 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:29:37.733 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.734 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.734 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:29:37.734 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:29:37.734 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.734 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.734 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.734 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:37.734 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:37.734 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.734 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.735 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.735 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:29:37.735 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.735 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.735 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.735 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:29:37.735 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:29:37.735 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.735 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.735 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.735 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:37.736 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:37.736 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.736 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.736 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.736 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:29:37.736 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.736 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.736 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.736 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:29:37.736 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:29:37.736 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.736 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.737 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.737 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:37.737 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:37.737 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:37.737 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:37.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.737 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:37.737 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:29:37.737 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:37.737 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.737 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.738 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:29:37.738 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:29:37.738 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:37.738 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:37.738 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:37.738 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.738 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.738 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:29:37.738 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:29:37.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.738 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:37.738 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:37.738 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:37.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.739 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:37.739 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:29:37.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:37.739 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:37.739 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:37.739 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:37.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:37.739 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:37.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:37.739 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:29:37.739 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:29:37.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.707+0000 7fc82b68e640 1 -- 172.21.15.115:0/1312260794 >> v1:172.21.15.132:6812/1835401922 conn(0x7fc808036fd0 legacy=0x7fc808039490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.707+0000 7fc82b68e640 10 monclient: shutdown 2024-01-24T23:29:37.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.707+0000 7fc82b68e640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:37.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.707+0000 7fc82b68e640 1 -- 172.21.15.115:0/1312260794 >> v1:172.21.15.115:6790/0 conn(0x7fc82409efd0 legacy=0x7fc824099d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.707+0000 7fc829c2b640 1 -- 172.21.15.115:0/1312260794 reap_dead start 2024-01-24T23:29:37.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.707+0000 7fc829c2b640 1 -- 172.21.15.115:0/1312260794 reap_dead start 2024-01-24T23:29:37.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.707+0000 7fc82b68e640 1 -- 172.21.15.115:0/1312260794 shutdown_connections 2024-01-24T23:29:37.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.707+0000 7fc82b68e640 1 -- 172.21.15.115:0/1312260794 >> 172.21.15.115:0/1312260794 conn(0x7fc8240a2110 msgr2=0x7fc824095480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:37.741 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.707+0000 7fc82b68e640 1 -- 172.21.15.115:0/1312260794 shutdown_connections 2024-01-24T23:29:37.742 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.707+0000 7fc82b68e640 1 -- 172.21.15.115:0/1312260794 wait complete. 2024-01-24T23:29:37.742 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-01-24T23:29:37.742 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a', 'f', 'c'] 2024-01-24T23:29:37.742 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-01-24T23:29:37.742 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-01-24T23:29:37.742 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-01-24T23:29:37.743 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-01-24T23:29:37.743 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:29:37.756 INFO:tasks.ceph.mon.a:Stopped 2024-01-24T23:29:37.756 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f 2024-01-24T23:29:37.756 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-01-24T23:29:37.756 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-01-24T23:29:37.756 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:29:37.771 INFO:tasks.ceph.mon.f:Stopped 2024-01-24T23:29:37.771 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-01-24T23:29:37.771 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-01-24T23:29:37.771 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-01-24T23:29:37.771 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:29:37.824 INFO:tasks.ceph.mon.c:Stopped 2024-01-24T23:29:37.824 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 6 2024-01-24T23:29:37.824 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-01-24T23:29:37.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.983+0000 7f1499f02640 10 monclient: get_monmap_and_config 2024-01-24T23:29:37.986 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.983+0000 7f1499f02640 10 monclient: build_initial_monmap 2024-01-24T23:29:37.987 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.983+0000 7f1499f02640 10 monclient: monmap: 2024-01-24T23:29:37.987 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:37.987 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.987 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:37.989303+0000 2024-01-24T23:29:37.987 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:37.989303+0000 2024-01-24T23:29:37.987 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:37.987 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.987 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:37.987 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:37.987 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:37.987 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:37.987 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:37.987 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:37.987 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:37.987 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:37.988 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:37.988 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.983+0000 7f1499f02640 1 Processor -- start 2024-01-24T23:29:37.990 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.983+0000 7f1499f02640 1 -- start start 2024-01-24T23:29:37.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.983+0000 7f1499f02640 10 monclient: init 2024-01-24T23:29:37.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.983+0000 7f1499f02640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:37.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.983+0000 7f1499f02640 10 monclient: _add_conns ranks=[5,6,8,1,2,3,0,4,7] 2024-01-24T23:29:37.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.983+0000 7f1499f02640 10 monclient(hunting): picked mon.noname-h con 0x7f1494098480 addr v1:172.21.15.132:6791/0 2024-01-24T23:29:37.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.983+0000 7f1499f02640 10 monclient(hunting): picked mon.noname-d con 0x7f149409f920 addr v1:172.21.15.115:6792/0 2024-01-24T23:29:37.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.983+0000 7f1499f02640 10 monclient(hunting): picked mon.noname-e con 0x7f1494066580 addr v1:172.21.15.115:6793/0 2024-01-24T23:29:37.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.983+0000 7f1499f02640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f149404ed70 con 0x7f1494098480 2024-01-24T23:29:37.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.983+0000 7f1499f02640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1494099610 con 0x7f149409f920 2024-01-24T23:29:37.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.983+0000 7f1499f02640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f149409dc20 con 0x7f1494066580 2024-01-24T23:29:37.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.983+0000 7f1499f02640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:37.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.983+0000 7f1499f02640 10 monclient(hunting): authenticate will time out at 1456.911847s 2024-01-24T23:29:37.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.983+0000 7f1492ffd640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f149409f920 0x7f1494138a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:53806/0 (socket says 172.21.15.115:53806) 2024-01-24T23:29:37.991 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.983+0000 7f1492ffd640 1 -- 172.21.15.115:0/1968402522 learned_addr learned my addr 172.21.15.115:0/1968402522 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:37.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 1 -- 172.21.15.115:0/1968402522 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2244272189 0 0) 0x7f149404ed70 con 0x7f1494098480 2024-01-24T23:29:37.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:37.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:37.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:37.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 10 monclient(hunting): my global_id is 7178 2024-01-24T23:29:37.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 1 -- 172.21.15.115:0/1968402522 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1480003530 con 0x7f1494098480 2024-01-24T23:29:37.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 1 -- 172.21.15.115:0/1968402522 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3791628225 0 0) 0x7f1480003530 con 0x7f1494098480 2024-01-24T23:29:37.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 1 -- 172.21.15.115:0/1968402522 >> v1:172.21.15.115:6793/0 conn(0x7f1494066580 legacy=0x7f149413c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 1 -- 172.21.15.115:0/1968402522 >> v1:172.21.15.115:6792/0 conn(0x7f149409f920 legacy=0x7f1494138a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 10 monclient: _finish_hunting 0 2024-01-24T23:29:37.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 1 monclient: found mon.noname-h 2024-01-24T23:29:37.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:37.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:29:37.992 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 1 -- 172.21.15.115:0/1968402522 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149409f4e0 con 0x7f1494098480 2024-01-24T23:29:37.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 10 monclient: _finish_auth 0 2024-01-24T23:29:37.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 10 monclient: _check_auth_tickets 2024-01-24T23:29:37.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:37.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 1 -- 172.21.15.115:0/1968402522 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f148c001b40 con 0x7f1494098480 2024-01-24T23:29:37.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 5 monclient: authenticate success, global_id 7178 2024-01-24T23:29:37.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:37.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:29:37.993 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 10 monclient: dump: 2024-01-24T23:29:37.993 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.993 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.993 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.993 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.993 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.993 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.994 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.994 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.994 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.994 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.994 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.994 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.994 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.994 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.994 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.994 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:37.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 1 -- 172.21.15.115:0/1968402522 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f148c001d60 con 0x7f1494098480 2024-01-24T23:29:37.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:37.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:37.994 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 4 set_mon_vals no callback set 2024-01-24T23:29:37.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 1 -- 172.21.15.115:0/1968402522 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f148c0039a0 con 0x7f1494098480 2024-01-24T23:29:37.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:37.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:29:37.995 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14927fc640 10 monclient: dump: 2024-01-24T23:29:37.995 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:37.995 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:37.995 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.995 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:37.995 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:37.995 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:37.995 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:37.995 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:37.995 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:37.995 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:37.995 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:37.995 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:37.996 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:37.996 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:37.996 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:37.996 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 10 monclient: shutdown 2024-01-24T23:29:37.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:37.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 1 -- 172.21.15.115:0/1968402522 >> v1:172.21.15.132:6791/0 conn(0x7f1494098480 legacy=0x7f1494098860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:37.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 1 -- 172.21.15.115:0/1968402522 shutdown_connections 2024-01-24T23:29:37.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 1 -- 172.21.15.115:0/1968402522 >> 172.21.15.115:0/1968402522 conn(0x7f14940a2110 msgr2=0x7f14940a24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:37.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 1 -- 172.21.15.115:0/1968402522 shutdown_connections 2024-01-24T23:29:37.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 1 -- 172.21.15.115:0/1968402522 wait complete. 2024-01-24T23:29:37.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 10 monclient: build_initial_monmap 2024-01-24T23:29:37.996 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 10 monclient: monmap: 2024-01-24T23:29:37.997 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:37.997 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:37.997 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:37.997 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:37.997 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:37.997 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:37.997 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:37.997 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:37.997 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:37.997 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:37.997 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:37.997 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:37.997 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:37.997 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:37.997 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:37.998 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:37.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 1 Processor -- start 2024-01-24T23:29:37.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 1 -- start start 2024-01-24T23:29:37.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 10 monclient: init 2024-01-24T23:29:38.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 10 monclient: _add_conns ranks=[0,1,5,7,2,6,3,8,4] 2024-01-24T23:29:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 10 monclient(hunting): picked mon.noname-a con 0x7f1494066580 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 10 monclient(hunting): picked mon.noname-f con 0x7f1494098480 addr v1:172.21.15.132:6789/0 2024-01-24T23:29:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 10 monclient(hunting): picked mon.noname-h con 0x7f149409f920 addr v1:172.21.15.132:6791/0 2024-01-24T23:29:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f149409be20 con 0x7f1494066580 2024-01-24T23:29:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1494099040 con 0x7f1494098480 2024-01-24T23:29:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f14937fe640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f1494066580 legacy=0x7f14941cba50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:29:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f149412d890 con 0x7f149409f920 2024-01-24T23:29:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1499f02640 10 monclient(hunting): authenticate will time out at 1456.915595s 2024-01-24T23:29:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1492ffd640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f1494098480 legacy=0x7f14941d08b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:29:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1493fff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f149409f920 0x7f14941d3fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.115:35642/0 (socket says 172.21.15.115:35642) 2024-01-24T23:29:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.987+0000 7f1493fff640 1 -- 172.21.15.115:0/1097892755 learned_addr learned my addr 172.21.15.115:0/1097892755 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 1 -- 172.21.15.115:0/1097892755 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1538077357 0 0) 0x7f149412d890 con 0x7f149409f920 2024-01-24T23:29:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:38.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:38.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:38.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 10 monclient(hunting): my global_id is 7187 2024-01-24T23:29:38.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 1 -- 172.21.15.115:0/1097892755 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1474003530 con 0x7f149409f920 2024-01-24T23:29:38.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 1 -- 172.21.15.115:0/1097892755 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3856987736 0 0) 0x7f1474003530 con 0x7f149409f920 2024-01-24T23:29:38.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 1 -- 172.21.15.115:0/1097892755 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f149412d890 con 0x7f149409f920 2024-01-24T23:29:38.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 1 -- 172.21.15.115:0/1097892755 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1484001a40 con 0x7f149409f920 2024-01-24T23:29:38.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:38.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:29:38.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 10 monclient(hunting): dump: 2024-01-24T23:29:38.002 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:38.002 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:38.002 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:38.002 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:38.003 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 1 -- 172.21.15.115:0/1097892755 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 225732359 0 0) 0x7f149412d890 con 0x7f149409f920 2024-01-24T23:29:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 1 -- 172.21.15.115:0/1097892755 >> v1:172.21.15.132:6789/0 conn(0x7f1494098480 legacy=0x7f14941d08b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:29:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 1 -- 172.21.15.115:0/1097892755 >> v1:172.21.15.115:6789/0 conn(0x7f1494066580 legacy=0x7f14941cba50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:29:38.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:29:38.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 1 monclient: found mon.h 2024-01-24T23:29:38.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:38.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:29:38.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 1 -- 172.21.15.115:0/1097892755 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149412bc10 con 0x7f149409f920 2024-01-24T23:29:38.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 10 monclient: _finish_auth 0 2024-01-24T23:29:38.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:29:38.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:38.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f1499f02640 5 monclient: authenticate success, global_id 7187 2024-01-24T23:29:38.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f1499f02640 10 monclient: _renew_subs 2024-01-24T23:29:38.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f1499f02640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:29:38.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f1499f02640 1 -- 172.21.15.115:0/1097892755 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f149412cff0 con 0x7f149409f920 2024-01-24T23:29:38.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f1499f02640 10 monclient: _renew_subs 2024-01-24T23:29:38.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f1499f02640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:29:38.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f1499f02640 1 -- 172.21.15.115:0/1097892755 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f149412c480 con 0x7f149409f920 2024-01-24T23:29:38.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 1 -- 172.21.15.115:0/1097892755 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1484001d40 con 0x7f149409f920 2024-01-24T23:29:38.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:38.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f1491ffb640 4 set_mon_vals no callback set 2024-01-24T23:29:38.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 1 -- 172.21.15.115:0/1097892755 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1484003c10 con 0x7f149409f920 2024-01-24T23:29:38.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:38.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:29:38.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 10 monclient: dump: 2024-01-24T23:29:38.005 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:38.005 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:38.005 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:38.005 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:38.005 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:38.005 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:38.005 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:38.006 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:38.006 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:38.006 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:38.006 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:38.006 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:38.006 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:38.006 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:38.006 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:38.006 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:38.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f1499f02640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:38.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f1499f02640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:38.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f1499f02640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:29:38.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f1499f02640 1 -- 172.21.15.115:0/1097892755 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1478005090 con 0x7f149409f920 2024-01-24T23:29:38.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 1 -- 172.21.15.115:0/1097892755 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f1484003df0 con 0x7f149409f920 2024-01-24T23:29:38.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f148bfff640 1 -- 172.21.15.115:0/1097892755 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7f14840443c0 con 0x7f149409f920 2024-01-24T23:29:38.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:37.991+0000 7f14937fe640 10 monclient: get_auth_request con 0x7f1474036f20 auth_method 0 2024-01-24T23:29:38.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:38.003+0000 7f148bfff640 1 -- 172.21.15.115:0/1097892755 <== mon.5 v1:172.21.15.132:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f14840146d0 con 0x7f149409f920 2024-01-24T23:29:38.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:38.003+0000 7f148bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:38.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:38.003+0000 7f148bfff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:38.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:38.219+0000 7f1499f02640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-01-24T23:29:38.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:38.219+0000 7f1499f02640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-01-24T23:29:38.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:38.219+0000 7f1499f02640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:29:38.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:38.219+0000 7f1499f02640 1 -- 172.21.15.115:0/1097892755 --> v1:172.21.15.132:6791/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1478005340 con 0x7f149409f920 2024-01-24T23:29:38.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:38.219+0000 7f148bfff640 1 -- 172.21.15.115:0/1097892755 <== mon.5 v1:172.21.15.132:6791/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2744 (unknown 254746657 0 3325052131) 0x7f14840144f0 con 0x7f149409f920 2024-01-24T23:29:38.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:38.219+0000 7f148bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-01-24T23:29:38.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:38.219+0000 7f148bfff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:38.224 INFO:teuthology.orchestra.run.smithi115.stdout:{"election_epoch":48,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":8,"features":{"quorum_con":"4540701272860131327","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","modified":"2024-01-24T23:20:20.557246Z","created":"2024-01-24T23:20:20.557246Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6790","nonce":0}]},"addr":"172.21.15.115:6790/0","public_addr":"172.21.15.115:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6790","nonce":0}]},"addr":"172.21.15.132:6790/0","public_addr":"172.21.15.132:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6791","nonce":0}]},"addr":"172.21.15.115:6791/0","public_addr":"172.21.15.115:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6791","nonce":0}]},"addr":"172.21.15.132:6791/0","public_addr":"172.21.15.132:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6792","nonce":0}]},"addr":"172.21.15.115:6792/0","public_addr":"172.21.15.115:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6792","nonce":0}]},"addr":"172.21.15.132:6792/0","public_addr":"172.21.15.132:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6793","nonce":0}]},"addr":"172.21.15.115:6793/0","public_addr":"172.21.15.115:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-01-24T23:29:38.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:38.223+0000 7f1499f02640 1 -- 172.21.15.115:0/1097892755 >> v1:172.21.15.132:6812/1835401922 conn(0x7f1474036f20 legacy=0x7f14740393e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:38.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:38.223+0000 7f1499f02640 10 monclient: shutdown 2024-01-24T23:29:38.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:38.223+0000 7f1499f02640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:38.226 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:38.223+0000 7f1499f02640 1 -- 172.21.15.115:0/1097892755 >> v1:172.21.15.132:6791/0 conn(0x7f149409f920 legacy=0x7f14941d3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:38.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:38.223+0000 7f1499f02640 1 -- 172.21.15.115:0/1097892755 shutdown_connections 2024-01-24T23:29:38.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:38.223+0000 7f1499f02640 1 -- 172.21.15.115:0/1097892755 >> 172.21.15.115:0/1097892755 conn(0x7f14940a2110 msgr2=0x7f14941beb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:38.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:38.223+0000 7f1499f02640 1 -- 172.21.15.115:0/1097892755 shutdown_connections 2024-01-24T23:29:38.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:38.223+0000 7f1499f02640 1 -- 172.21.15.115:0/1097892755 wait complete. 2024-01-24T23:29:41.244 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-01-24T23:29:41.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:41.371+0000 7f614b85a640 10 monclient: get_monmap_and_config 2024-01-24T23:29:41.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:41.371+0000 7f614b85a640 10 monclient: build_initial_monmap 2024-01-24T23:29:41.373 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:41.371+0000 7f614b85a640 10 monclient: monmap: 2024-01-24T23:29:41.373 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:41.373 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:41.374 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:41.376223+0000 2024-01-24T23:29:41.374 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:41.376223+0000 2024-01-24T23:29:41.374 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:41.374 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:41.374 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:41.374 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:41.374 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:41.374 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:41.374 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:41.374 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:41.374 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:41.374 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:41.374 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:41.374 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:41.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:41.371+0000 7f614b85a640 1 Processor -- start 2024-01-24T23:29:41.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:41.371+0000 7f614b85a640 1 -- start start 2024-01-24T23:29:41.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:41.371+0000 7f614b85a640 10 monclient: init 2024-01-24T23:29:41.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:41.371+0000 7f614b85a640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:41.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:41.371+0000 7f614b85a640 10 monclient: _add_conns ranks=[8,1,0,4,6,5,3,7,2] 2024-01-24T23:29:41.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:41.371+0000 7f614b85a640 10 monclient(hunting): picked mon.noname-e con 0x7f61440a2c40 addr v1:172.21.15.115:6793/0 2024-01-24T23:29:41.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:41.371+0000 7f614b85a640 10 monclient(hunting): picked mon.noname-f con 0x7f61440a3fd0 addr v1:172.21.15.132:6789/0 2024-01-24T23:29:41.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:41.371+0000 7f614b85a640 10 monclient(hunting): picked mon.noname-a con 0x7f61440a06b0 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:41.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:41.371+0000 7f614b85a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614404ed70 con 0x7f61440a06b0 2024-01-24T23:29:41.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:41.371+0000 7f614b85a640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61440a0e70 con 0x7f61440a3fd0 2024-01-24T23:29:41.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:41.371+0000 7f614b85a640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61440a3730 con 0x7f61440a2c40 2024-01-24T23:29:41.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:41.371+0000 7f614b85a640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:41.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:41.371+0000 7f614b85a640 10 monclient(hunting): authenticate will time out at 1460.299626s 2024-01-24T23:29:41.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:41.371+0000 7f6149df7640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f61440a06b0 legacy=0x7f614413f5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:29:41.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:41.371+0000 7f61495f6640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f61440a2c40 0x7f61440a3020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:56082/0 (socket says 172.21.15.115:56082) 2024-01-24T23:29:41.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:41.371+0000 7f61495f6640 1 -- 172.21.15.115:0/4272342764 learned_addr learned my addr 172.21.15.115:0/4272342764 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:41.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:41.371+0000 7f6148df5640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.132:6789/0 conn(0x7f61440a3fd0 legacy=0x7f614413be70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:29:41.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:41.575+0000 7f6149df7640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.115:6789/0 conn(0x7f61440a06b0 legacy=0x7f614413f5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:29:41.576 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:41.575+0000 7f6148df5640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.132:6789/0 conn(0x7f61440a3fd0 legacy=0x7f614413be70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:29:41.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:41.979+0000 7f6149df7640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.115:6789/0 conn(0x7f61440a06b0 legacy=0x7f614413f5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:29:41.977 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:41.979+0000 7f6148df5640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.132:6789/0 conn(0x7f61440a3fd0 legacy=0x7f614413be70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:29:42.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:42.783+0000 7f6149df7640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.115:6789/0 conn(0x7f61440a06b0 legacy=0x7f614413f5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:29:42.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:42.783+0000 7f6148df5640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.132:6789/0 conn(0x7f61440a3fd0 legacy=0x7f614413be70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:29:43.166 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~219s 2024-01-24T23:29:44.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:44.375+0000 7f613affd640 10 monclient(hunting): tick 2024-01-24T23:29:44.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:44.375+0000 7f613affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:29:44.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:44.375+0000 7f613affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:29:44.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:44.375+0000 7f613affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:29:44.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:44.375+0000 7f613affd640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.115:6793/0 conn(0x7f61440a2c40 legacy=0x7f61440a3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:44.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:44.375+0000 7f613affd640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.132:6789/0 conn(0x7f61440a3fd0 legacy=0x7f614413be70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:29:44.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:44.375+0000 7f613affd640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.115:6789/0 conn(0x7f61440a06b0 legacy=0x7f614413f5a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:29:44.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:44.375+0000 7f613affd640 10 monclient: _add_conns ranks=[3,6,7,2,4,5] 2024-01-24T23:29:44.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:44.375+0000 7f613affd640 10 monclient(hunting): picked mon.noname-g con 0x7f61280037b0 addr v1:172.21.15.132:6790/0 2024-01-24T23:29:44.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:44.375+0000 7f613affd640 10 monclient(hunting): picked mon.noname-d con 0x7f6128007440 addr v1:172.21.15.115:6792/0 2024-01-24T23:29:44.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:44.375+0000 7f613affd640 10 monclient(hunting): picked mon.noname-i con 0x7f612800b1d0 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:44.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:44.375+0000 7f613affd640 1 -- 172.21.15.115:0/4272342764 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61440a1150 con 0x7f61280037b0 2024-01-24T23:29:44.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:44.375+0000 7f613affd640 1 -- 172.21.15.115:0/4272342764 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614404ed70 con 0x7f6128007440 2024-01-24T23:29:44.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:44.375+0000 7f613affd640 1 -- 172.21.15.115:0/4272342764 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61440a0e70 con 0x7f612800b1d0 2024-01-24T23:29:44.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:44.375+0000 7f613affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:47.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:47.375+0000 7f613affd640 10 monclient(hunting): tick 2024-01-24T23:29:47.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:47.375+0000 7f613affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:29:47.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:47.375+0000 7f613affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:29:47.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:47.375+0000 7f613affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:29:47.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:47.375+0000 7f613affd640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.132:6792/0 conn(0x7f612800b1d0 legacy=0x7f612800d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:47.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:47.375+0000 7f613affd640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.115:6792/0 conn(0x7f6128007440 legacy=0x7f6128009900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:47.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:47.375+0000 7f613affd640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.132:6790/0 conn(0x7f61280037b0 legacy=0x7f6128005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:47.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:47.375+0000 7f613affd640 10 monclient: _add_conns ranks=[5,2,4] 2024-01-24T23:29:47.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:47.375+0000 7f613affd640 10 monclient(hunting): picked mon.noname-h con 0x7f6128011080 addr v1:172.21.15.132:6791/0 2024-01-24T23:29:47.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:47.375+0000 7f613affd640 10 monclient(hunting): picked mon.noname-b con 0x7f61280133a0 addr v1:172.21.15.115:6790/0 2024-01-24T23:29:47.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:47.375+0000 7f6149df7640 1 -- 172.21.15.115:0/4272342764 reap_dead start 2024-01-24T23:29:47.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:47.375+0000 7f613affd640 10 monclient(hunting): picked mon.noname-c con 0x7f6128016f40 addr v1:172.21.15.115:6791/0 2024-01-24T23:29:47.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:47.375+0000 7f613affd640 1 -- 172.21.15.115:0/4272342764 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61280120a0 con 0x7f61280133a0 2024-01-24T23:29:47.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:47.375+0000 7f613affd640 1 -- 172.21.15.115:0/4272342764 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f612801bc90 con 0x7f6128016f40 2024-01-24T23:29:47.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:47.375+0000 7f613affd640 1 -- 172.21.15.115:0/4272342764 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f612800a160 con 0x7f6128011080 2024-01-24T23:29:47.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:47.375+0000 7f613affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:47.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:47.375+0000 7f6149df7640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.115:6791/0 conn(0x7f6128016f40 legacy=0x7f6128019360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:29:47.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:47.579+0000 7f6149df7640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.115:6791/0 conn(0x7f6128016f40 legacy=0x7f6128019360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:29:47.980 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:47.983+0000 7f6149df7640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.115:6791/0 conn(0x7f6128016f40 legacy=0x7f6128019360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:29:48.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:48.787+0000 7f6149df7640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.115:6791/0 conn(0x7f6128016f40 legacy=0x7f6128019360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:29:49.377 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~225s 2024-01-24T23:29:50.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:50.375+0000 7f613affd640 10 monclient(hunting): tick 2024-01-24T23:29:50.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:50.375+0000 7f613affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:29:50.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:50.375+0000 7f613affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:29:50.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:50.375+0000 7f613affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:29:50.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:50.375+0000 7f613affd640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.132:6791/0 conn(0x7f6128011080 legacy=0x7f61280114c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:50.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:50.375+0000 7f613affd640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.115:6791/0 conn(0x7f6128016f40 legacy=0x7f6128019360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:29:50.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:50.375+0000 7f613affd640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.115:6790/0 conn(0x7f61280133a0 legacy=0x7f6128015790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:50.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:50.375+0000 7f613affd640 10 monclient: _add_conns ranks=[3,6,1,2,0,7,8,5,4] 2024-01-24T23:29:50.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:50.375+0000 7f613affd640 10 monclient(hunting): picked mon.noname-g con 0x7f612800a500 addr v1:172.21.15.132:6790/0 2024-01-24T23:29:50.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:50.375+0000 7f613affd640 10 monclient(hunting): picked mon.noname-d con 0x7f612800a980 addr v1:172.21.15.115:6792/0 2024-01-24T23:29:50.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:50.375+0000 7f613affd640 10 monclient(hunting): picked mon.noname-f con 0x7f612800de60 addr v1:172.21.15.132:6789/0 2024-01-24T23:29:50.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:50.375+0000 7f613affd640 1 -- 172.21.15.115:0/4272342764 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f612800a2f0 con 0x7f612800de60 2024-01-24T23:29:50.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:50.375+0000 7f613affd640 1 -- 172.21.15.115:0/4272342764 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f612801bc90 con 0x7f612800a500 2024-01-24T23:29:50.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:50.375+0000 7f613affd640 1 -- 172.21.15.115:0/4272342764 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f612800e2f0 con 0x7f612800a980 2024-01-24T23:29:50.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:50.375+0000 7f613affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:50.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:50.375+0000 7f6149df7640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.132:6789/0 conn(0x7f612800de60 legacy=0x7f6128005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:29:50.580 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:50.579+0000 7f6149df7640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.132:6789/0 conn(0x7f612800de60 legacy=0x7f6128005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:29:50.984 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:50.983+0000 7f6149df7640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.132:6789/0 conn(0x7f612800de60 legacy=0x7f6128005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:29:51.788 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:51.787+0000 7f6149df7640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.132:6789/0 conn(0x7f612800de60 legacy=0x7f6128005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:29:52.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 1 -- 172.21.15.115:0/4272342764 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1218518755 0 0) 0x7f612801bc90 con 0x7f612800a500 2024-01-24T23:29:52.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:52.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:52.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:52.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 10 monclient(hunting): my global_id is 7221 2024-01-24T23:29:52.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 1 -- 172.21.15.115:0/4272342764 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6130003530 con 0x7f612800a500 2024-01-24T23:29:52.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 1 -- 172.21.15.115:0/4272342764 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2648343783 0 0) 0x7f6130003530 con 0x7f612800a500 2024-01-24T23:29:52.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.115:6792/0 conn(0x7f612800a980 legacy=0x7f612800d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:52.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.132:6789/0 conn(0x7f612800de60 legacy=0x7f6128005ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:29:52.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 10 monclient: _finish_hunting 0 2024-01-24T23:29:52.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 1 monclient: found mon.noname-g 2024-01-24T23:29:52.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:52.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.132:6790/0 2024-01-24T23:29:52.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f6149df7640 1 -- 172.21.15.115:0/4272342764 reap_dead start 2024-01-24T23:29:52.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 1 -- 172.21.15.115:0/4272342764 --> v1:172.21.15.132:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612800e480 con 0x7f612800a500 2024-01-24T23:29:52.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 10 monclient: _finish_auth 0 2024-01-24T23:29:52.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 10 monclient: _check_auth_tickets 2024-01-24T23:29:52.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:52.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 1 -- 172.21.15.115:0/4272342764 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f61400016c0 con 0x7f612800a500 2024-01-24T23:29:52.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:52.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-01-24T23:29:52.025 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 10 monclient: dump: 2024-01-24T23:29:52.025 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:52.025 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:52.025 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:52.025 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:52.025 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:52.025 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:52.025 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:52.025 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:52.025 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:52.025 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:52.025 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:52.025 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:52.025 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:52.026 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:52.026 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:52.026 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:52.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f614b85a640 5 monclient: authenticate success, global_id 7221 2024-01-24T23:29:52.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f614b85a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:52.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 1 -- 172.21.15.115:0/4272342764 <== mon.3 v1:172.21.15.132:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f61400018e0 con 0x7f612800a500 2024-01-24T23:29:52.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:52.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 1 -- 172.21.15.115:0/4272342764 <== mon.3 v1:172.21.15.132:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f61400048b0 con 0x7f612800a500 2024-01-24T23:29:52.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f614b85a640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:52.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f614b85a640 4 set_mon_vals no callback set 2024-01-24T23:29:52.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:52.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:29:52.026 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f613bfff640 10 monclient: dump: 2024-01-24T23:29:52.026 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:52.027 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:52.027 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:52.027 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:52.027 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:52.027 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:52.027 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:52.027 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:52.027 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:52.027 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:52.027 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:52.027 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:52.027 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:52.027 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:52.027 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:52.027 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:52.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f614b85a640 10 monclient: shutdown 2024-01-24T23:29:52.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f614b85a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:52.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.019+0000 7f614b85a640 1 -- 172.21.15.115:0/4272342764 >> v1:172.21.15.132:6790/0 conn(0x7f612800a500 legacy=0x7f6128009900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:52.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 1 -- 172.21.15.115:0/4272342764 shutdown_connections 2024-01-24T23:29:52.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 1 -- 172.21.15.115:0/4272342764 >> 172.21.15.115:0/4272342764 conn(0x7f61440a9ba0 msgr2=0x7f61440a5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:52.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 1 -- 172.21.15.115:0/4272342764 shutdown_connections 2024-01-24T23:29:52.028 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 1 -- 172.21.15.115:0/4272342764 wait complete. 2024-01-24T23:29:52.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 10 monclient: build_initial_monmap 2024-01-24T23:29:52.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 10 monclient: monmap: 2024-01-24T23:29:52.032 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:52.032 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:52.032 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:52.032 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:52.032 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:52.032 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:52.033 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:52.033 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:52.033 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:52.033 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:52.033 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:52.033 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:52.033 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:52.033 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:52.033 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:52.033 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:52.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 1 Processor -- start 2024-01-24T23:29:52.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 1 -- start start 2024-01-24T23:29:52.033 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 10 monclient: init 2024-01-24T23:29:52.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:52.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 10 monclient: _add_conns ranks=[6,0,7,8,3,4,5,1,2] 2024-01-24T23:29:52.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 10 monclient(hunting): picked mon.noname-d con 0x7f612800a500 addr v1:172.21.15.115:6792/0 2024-01-24T23:29:52.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 10 monclient(hunting): picked mon.noname-a con 0x7f61441cb740 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:52.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 10 monclient(hunting): picked mon.noname-i con 0x7f61441d21e0 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:52.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614409fbc0 con 0x7f61441cb740 2024-01-24T23:29:52.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61441cbb40 con 0x7f612800a500 2024-01-24T23:29:52.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61441a7e20 con 0x7f61441d21e0 2024-01-24T23:29:52.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:52.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6148df5640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f61441cb740 legacy=0x7f61441d0ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:29:52.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 10 monclient(hunting): authenticate will time out at 1470.950779s 2024-01-24T23:29:52.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f61495f6640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f612800a500 0x7f61441bad00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:43934/0 (socket says 172.21.15.115:43934) 2024-01-24T23:29:52.034 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f61495f6640 1 -- 172.21.15.115:0/4133434851 learned_addr learned my addr 172.21.15.115:0/4133434851 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:52.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 1 -- 172.21.15.115:0/4133434851 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1590163108 0 0) 0x7f612800e480 con 0x7f612800a500 2024-01-24T23:29:52.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:29:52.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:52.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:52.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 10 monclient(hunting): my global_id is 7395 2024-01-24T23:29:52.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 1 -- 172.21.15.115:0/4133434851 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f612800a2f0 con 0x7f612800a500 2024-01-24T23:29:52.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 1 -- 172.21.15.115:0/4133434851 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3834112255 0 0) 0x7f612800a2f0 con 0x7f612800a500 2024-01-24T23:29:52.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 1 -- 172.21.15.115:0/4133434851 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f612800e480 con 0x7f612800a500 2024-01-24T23:29:52.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 1 -- 172.21.15.115:0/4133434851 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6140001680 con 0x7f612800a500 2024-01-24T23:29:52.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:52.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:29:52.035 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 10 monclient(hunting): dump: 2024-01-24T23:29:52.035 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:52.035 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:52.035 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:52.036 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:52.036 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:52.036 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:52.036 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:52.036 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:52.036 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:52.036 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:52.036 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:52.036 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:52.036 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:52.036 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:52.036 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:52.036 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:52.036 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:52.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 1 -- 172.21.15.115:0/4133434851 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1934732931 0 0) 0x7f61441a7e20 con 0x7f61441d21e0 2024-01-24T23:29:52.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:29:52.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:52.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:52.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 10 monclient(hunting): my global_id is 154285 2024-01-24T23:29:52.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 1 -- 172.21.15.115:0/4133434851 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f612800a2f0 con 0x7f61441d21e0 2024-01-24T23:29:52.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 1 -- 172.21.15.115:0/4133434851 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3905867415 0 0) 0x7f612800e480 con 0x7f612800a500 2024-01-24T23:29:52.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 1 -- 172.21.15.115:0/4133434851 >> v1:172.21.15.132:6792/0 conn(0x7f61441d21e0 legacy=0x7f61441d45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:52.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 1 -- 172.21.15.115:0/4133434851 >> v1:172.21.15.115:6789/0 conn(0x7f61441cb740 legacy=0x7f61441d0ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:29:52.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:29:52.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 1 monclient: found mon.d 2024-01-24T23:29:52.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:52.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:29:52.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 1 -- 172.21.15.115:0/4133434851 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61440a58a0 con 0x7f612800a500 2024-01-24T23:29:52.037 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 10 monclient: _finish_auth 0 2024-01-24T23:29:52.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:29:52.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f6139ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:52.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 5 monclient: authenticate success, global_id 7395 2024-01-24T23:29:52.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 10 monclient: _renew_subs 2024-01-24T23:29:52.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:29:52.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 1 -- 172.21.15.115:0/4133434851 --> v1:172.21.15.115:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61441a6360 con 0x7f612800a500 2024-01-24T23:29:52.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 10 monclient: _renew_subs 2024-01-24T23:29:52.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:29:52.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.023+0000 7f614b85a640 1 -- 172.21.15.115:0/4133434851 --> v1:172.21.15.115:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f61441cc1a0 con 0x7f612800a500 2024-01-24T23:29:52.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.027+0000 7f6139ffb640 1 -- 172.21.15.115:0/4133434851 <== mon.6 v1:172.21.15.115:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6140004c30 con 0x7f612800a500 2024-01-24T23:29:52.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.027+0000 7f6139ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:52.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.027+0000 7f6139ffb640 1 -- 172.21.15.115:0/4133434851 <== mon.6 v1:172.21.15.115:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f6140005160 con 0x7f612800a500 2024-01-24T23:29:52.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.027+0000 7f6139ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:52.038 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.027+0000 7f613affd640 4 set_mon_vals no callback set 2024-01-24T23:29:52.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.027+0000 7f6139ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:29:52.039 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.027+0000 7f6139ffb640 10 monclient: dump: 2024-01-24T23:29:52.039 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:52.039 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:52.039 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:52.039 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:52.039 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:52.039 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:52.039 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:52.039 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:52.039 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:52.039 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:52.039 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:52.039 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:52.039 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:52.039 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:52.039 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:52.040 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:52.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.027+0000 7f6139ffb640 1 -- 172.21.15.115:0/4133434851 <== mon.6 v1:172.21.15.115:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f61400052c0 con 0x7f612800a500 2024-01-24T23:29:52.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.027+0000 7f614b85a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:52.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.027+0000 7f614b85a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:52.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.027+0000 7f614b85a640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:29:52.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.027+0000 7f614b85a640 1 -- 172.21.15.115:0/4133434851 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6124005090 con 0x7f612800a500 2024-01-24T23:29:52.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.027+0000 7f6139ffb640 1 -- 172.21.15.115:0/4133434851 <== mon.6 v1:172.21.15.115:6792/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7f6140044510 con 0x7f612800a500 2024-01-24T23:29:52.040 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.027+0000 7f6148df5640 10 monclient: get_auth_request con 0x7f6128039300 auth_method 0 2024-01-24T23:29:52.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.031+0000 7f6139ffb640 1 -- 172.21.15.115:0/4133434851 <== mon.6 v1:172.21.15.115:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172797 (unknown 1092875540 0 1220221878) 0x7f6140014b70 con 0x7f612800a500 2024-01-24T23:29:52.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.031+0000 7f6139ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:52.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.031+0000 7f6139ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:52.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.239+0000 7f614b85a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-01-24T23:29:52.243 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.239+0000 7f614b85a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-01-24T23:29:52.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.239+0000 7f614b85a640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.115:6792/0 2024-01-24T23:29:52.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.239+0000 7f614b85a640 1 -- 172.21.15.115:0/4133434851 --> v1:172.21.15.115:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f61240058c0 con 0x7f612800a500 2024-01-24T23:29:52.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.243+0000 7f6139ffb640 1 -- 172.21.15.115:0/4133434851 <== mon.6 v1:172.21.15.115:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2726 (unknown 254746657 0 2135459250) 0x7f6140012fe0 con 0x7f612800a500 2024-01-24T23:29:52.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.243+0000 7f6139ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-01-24T23:29:52.244 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.243+0000 7f6139ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:52.244 INFO:teuthology.orchestra.run.smithi115.stdout:{"election_epoch":52,"quorum":[2,3,5,6,7,8],"quorum_names":["b","g","h","d","i","e"],"quorum_leader_name":"h","quorum_age":0,"features":{"quorum_con":"4540701272860131327","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4c8039f0-ed94-40df-8309-0a291ebf7f77","modified":"2024-01-24T23:20:20.557246Z","created":"2024-01-24T23:20:20.557246Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6790","nonce":0}]},"addr":"172.21.15.115:6790/0","public_addr":"172.21.15.115:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6790","nonce":0}]},"addr":"172.21.15.132:6790/0","public_addr":"172.21.15.132:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6791","nonce":0}]},"addr":"172.21.15.115:6791/0","public_addr":"172.21.15.115:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6791","nonce":0}]},"addr":"172.21.15.132:6791/0","public_addr":"172.21.15.132:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6792","nonce":0}]},"addr":"172.21.15.115:6792/0","public_addr":"172.21.15.115:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.132:6792","nonce":0}]},"addr":"172.21.15.132:6792/0","public_addr":"172.21.15.132:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6793","nonce":0}]},"addr":"172.21.15.115:6793/0","public_addr":"172.21.15.115:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-01-24T23:29:52.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.243+0000 7f614b85a640 1 -- 172.21.15.115:0/4133434851 >> v1:172.21.15.132:6812/1835401922 conn(0x7f6128039300 legacy=0x7f612803b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:52.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.243+0000 7f614b85a640 10 monclient: shutdown 2024-01-24T23:29:52.246 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.243+0000 7f614b85a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:52.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.243+0000 7f614b85a640 1 -- 172.21.15.115:0/4133434851 >> v1:172.21.15.115:6792/0 conn(0x7f612800a500 legacy=0x7f61441bad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:52.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.243+0000 7f614b85a640 1 -- 172.21.15.115:0/4133434851 shutdown_connections 2024-01-24T23:29:52.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.243+0000 7f614b85a640 1 -- 172.21.15.115:0/4133434851 >> 172.21.15.115:0/4133434851 conn(0x7f61440a9ba0 msgr2=0x7f61440a79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:52.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.243+0000 7f614b85a640 1 -- 172.21.15.115:0/4133434851 shutdown_connections 2024-01-24T23:29:52.247 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.243+0000 7f614b85a640 1 -- 172.21.15.115:0/4133434851 wait complete. 2024-01-24T23:29:52.261 INFO:tasks.mon_thrash.ceph_manager:quorum is size 6 2024-01-24T23:29:52.261 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon b 2024-01-24T23:29:52.261 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-01-24T23:29:52.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.363+0000 7f03fec61640 10 monclient: get_monmap_and_config 2024-01-24T23:29:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.363+0000 7f03fec61640 10 monclient: build_initial_monmap 2024-01-24T23:29:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.363+0000 7f03fec61640 10 monclient: monmap: 2024-01-24T23:29:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:52.369794+0000 2024-01-24T23:29:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:52.369794+0000 2024-01-24T23:29:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:52.368 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:52.368 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:52.368 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:52.368 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:52.368 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:52.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.363+0000 7f03fec61640 1 Processor -- start 2024-01-24T23:29:52.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.363+0000 7f03fec61640 1 -- start start 2024-01-24T23:29:52.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.363+0000 7f03fec61640 10 monclient: init 2024-01-24T23:29:52.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:52.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 10 monclient: _add_conns ranks=[2,0,4,5,6,1,3,7,8] 2024-01-24T23:29:52.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 10 monclient(hunting): picked mon.noname-b con 0x7f03f8133c10 addr v1:172.21.15.115:6790/0 2024-01-24T23:29:52.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 10 monclient(hunting): picked mon.noname-a con 0x7f03f8136960 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:52.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 10 monclient(hunting): picked mon.noname-c con 0x7f03f813a470 addr v1:172.21.15.115:6791/0 2024-01-24T23:29:52.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f804ed70 con 0x7f03f8136960 2024-01-24T23:29:52.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f809dee0 con 0x7f03f8133c10 2024-01-24T23:29:52.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f809d400 con 0x7f03f813a470 2024-01-24T23:29:52.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:52.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 10 monclient(hunting): authenticate will time out at 1471.292401s 2024-01-24T23:29:52.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fd1fe640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f03f813a470 legacy=0x7f03f813c860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:29:52.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f7fff640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f03f8136960 legacy=0x7f03f8138d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:29:52.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fc9fd640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f03f8133c10 0x7f03f8133ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:38092/0 (socket says 172.21.15.115:38092) 2024-01-24T23:29:52.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fc9fd640 1 -- 172.21.15.115:0/4242087674 learned_addr learned my addr 172.21.15.115:0/4242087674 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:52.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 1 -- 172.21.15.115:0/4242087674 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 17501706 0 0) 0x7f03f809dee0 con 0x7f03f8133c10 2024-01-24T23:29:52.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 10 monclient(hunting): _negotiate 2024-01-24T23:29:52.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:52.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:52.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 10 monclient(hunting): my global_id is 154235 2024-01-24T23:29:52.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 1 -- 172.21.15.115:0/4242087674 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03e4003530 con 0x7f03f8133c10 2024-01-24T23:29:52.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 1 -- 172.21.15.115:0/4242087674 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3880334409 0 0) 0x7f03e4003530 con 0x7f03f8133c10 2024-01-24T23:29:52.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 1 -- 172.21.15.115:0/4242087674 >> v1:172.21.15.115:6791/0 conn(0x7f03f813a470 legacy=0x7f03f813c860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:29:52.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 1 -- 172.21.15.115:0/4242087674 >> v1:172.21.15.115:6789/0 conn(0x7f03f8136960 legacy=0x7f03f8138d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:29:52.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 10 monclient: _finish_hunting 0 2024-01-24T23:29:52.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 1 monclient: found mon.noname-b 2024-01-24T23:29:52.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:52.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.115:6790/0 2024-01-24T23:29:52.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 1 -- 172.21.15.115:0/4242087674 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f809f380 con 0x7f03f8133c10 2024-01-24T23:29:52.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 10 monclient: _finish_auth 0 2024-01-24T23:29:52.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 10 monclient: _check_auth_tickets 2024-01-24T23:29:52.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:52.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 1 -- 172.21.15.115:0/4242087674 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f03f0001b40 con 0x7f03f8133c10 2024-01-24T23:29:52.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:52.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:29:52.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 10 monclient: dump: 2024-01-24T23:29:52.378 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:52.378 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:52.378 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:52.378 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:52.378 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:52.378 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:52.378 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:52.378 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:52.379 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:52.379 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:52.379 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:52.379 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:52.379 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:52.379 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:52.379 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:52.379 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:52.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 5 monclient: authenticate success, global_id 154235 2024-01-24T23:29:52.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:52.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 1 -- 172.21.15.115:0/4242087674 <== mon.2 v1:172.21.15.115:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f03f0001e60 con 0x7f03f8133c10 2024-01-24T23:29:52.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:52.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:52.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 4 set_mon_vals no callback set 2024-01-24T23:29:52.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 1 -- 172.21.15.115:0/4242087674 <== mon.2 v1:172.21.15.115:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f03f0004f70 con 0x7f03f8133c10 2024-01-24T23:29:52.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:52.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:29:52.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f77fe640 10 monclient: dump: 2024-01-24T23:29:52.380 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:52.380 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:52.380 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:52.380 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:52.380 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:52.380 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:52.380 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:52.380 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:52.380 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:52.380 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:52.380 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:52.380 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:52.381 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:52.381 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:52.381 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:52.381 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:52.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 10 monclient: shutdown 2024-01-24T23:29:52.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:52.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 1 -- 172.21.15.115:0/4242087674 >> v1:172.21.15.115:6790/0 conn(0x7f03f8133c10 legacy=0x7f03f8133ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:52.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 1 -- 172.21.15.115:0/4242087674 shutdown_connections 2024-01-24T23:29:52.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 1 -- 172.21.15.115:0/4242087674 >> 172.21.15.115:0/4242087674 conn(0x7f03f80a5730 msgr2=0x7f03f80a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:52.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 1 -- 172.21.15.115:0/4242087674 shutdown_connections 2024-01-24T23:29:52.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 1 -- 172.21.15.115:0/4242087674 wait complete. 2024-01-24T23:29:52.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 10 monclient: build_initial_monmap 2024-01-24T23:29:52.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 10 monclient: monmap: 2024-01-24T23:29:52.381 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:52.381 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:52.382 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:52.382 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:52.382 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:52.382 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:52.382 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:52.382 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:52.382 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:52.382 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:52.382 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:52.382 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:52.382 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:52.382 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:52.382 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:52.382 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:52.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 1 Processor -- start 2024-01-24T23:29:52.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 1 -- start start 2024-01-24T23:29:52.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 10 monclient: init 2024-01-24T23:29:52.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:52.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 10 monclient: _add_conns ranks=[3,1,8,2,6,0,5,4,7] 2024-01-24T23:29:52.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 10 monclient(hunting): picked mon.noname-g con 0x7f03f8133c10 addr v1:172.21.15.132:6790/0 2024-01-24T23:29:52.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 10 monclient(hunting): picked mon.noname-f con 0x7f03f8136960 addr v1:172.21.15.132:6789/0 2024-01-24T23:29:52.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 10 monclient(hunting): picked mon.noname-e con 0x7f03f813a470 addr v1:172.21.15.115:6793/0 2024-01-24T23:29:52.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f8138520 con 0x7f03f8136960 2024-01-24T23:29:52.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f80a1310 con 0x7f03f8133c10 2024-01-24T23:29:52.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f8071c30 con 0x7f03f813a470 2024-01-24T23:29:52.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:52.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 10 monclient(hunting): authenticate will time out at 1471.295052s 2024-01-24T23:29:52.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fd1fe640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f03f813a470 0x7f03f81d3670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:58354/0 (socket says 172.21.15.115:58354) 2024-01-24T23:29:52.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fd1fe640 1 -- 172.21.15.115:0/3586142633 learned_addr learned my addr 172.21.15.115:0/3586142633 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:52.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f7fff640 1 -- 172.21.15.115:0/3586142633 >> v1:172.21.15.132:6789/0 conn(0x7f03f8136960 legacy=0x7f03f8137450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:29:52.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129067882 0 0) 0x7f03f8071c30 con 0x7f03f813a470 2024-01-24T23:29:52.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 10 monclient(hunting): _negotiate 2024-01-24T23:29:52.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:52.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:52.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 10 monclient(hunting): my global_id is 86462 2024-01-24T23:29:52.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03d8003530 con 0x7f03f813a470 2024-01-24T23:29:52.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2638203742 0 0) 0x7f03d8003530 con 0x7f03f813a470 2024-01-24T23:29:52.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f03f8071c30 con 0x7f03f813a470 2024-01-24T23:29:52.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f03e80041e0 con 0x7f03f813a470 2024-01-24T23:29:52.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:52.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:29:52.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 10 monclient(hunting): dump: 2024-01-24T23:29:52.385 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:52.385 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:52.385 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:52.385 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:52.385 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:52.385 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:52.385 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:52.385 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:52.385 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:52.385 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:52.385 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:52.386 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:52.386 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:52.386 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:52.386 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:52.386 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:52.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:52.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3572326818 0 0) 0x7f03f80a1310 con 0x7f03f8133c10 2024-01-24T23:29:52.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 10 monclient(hunting): _negotiate 2024-01-24T23:29:52.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:52.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:52.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 10 monclient(hunting): my global_id is 7230 2024-01-24T23:29:52.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03d8003530 con 0x7f03f8133c10 2024-01-24T23:29:52.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1071443782 0 0) 0x7f03f8071c30 con 0x7f03f813a470 2024-01-24T23:29:52.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 >> v1:172.21.15.132:6790/0 conn(0x7f03f8133c10 legacy=0x7f03f8136d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:52.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 >> v1:172.21.15.132:6789/0 conn(0x7f03f8136960 legacy=0x7f03f8137450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:29:52.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 10 monclient: _finish_hunting 0 2024-01-24T23:29:52.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 1 monclient: found mon.e 2024-01-24T23:29:52.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:52.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:29:52.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f8135b10 con 0x7f03f813a470 2024-01-24T23:29:52.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 10 monclient: _finish_auth 0 2024-01-24T23:29:52.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 10 monclient: _check_auth_tickets 2024-01-24T23:29:52.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:52.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 5 monclient: authenticate success, global_id 86462 2024-01-24T23:29:52.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 10 monclient: _renew_subs 2024-01-24T23:29:52.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:29:52.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 1 -- 172.21.15.115:0/3586142633 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f8138ca0 con 0x7f03f813a470 2024-01-24T23:29:52.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 10 monclient: _renew_subs 2024-01-24T23:29:52.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:29:52.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03fec61640 1 -- 172.21.15.115:0/3586142633 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f812e310 con 0x7f03f813a470 2024-01-24T23:29:52.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f03e8004500 con 0x7f03f813a470 2024-01-24T23:29:52.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:52.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f03e8004a30 con 0x7f03f813a470 2024-01-24T23:29:52.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:52.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:29:52.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f6ffd640 4 set_mon_vals no callback set 2024-01-24T23:29:52.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.367+0000 7f03f57fa640 10 monclient: dump: 2024-01-24T23:29:52.388 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:52.388 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:52.389 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:52.389 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:52.389 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:52.389 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:52.389 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:52.389 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:52.389 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:52.389 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:52.389 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:52.389 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:52.390 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:52.390 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:52.390 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:52.390 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:52.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.371+0000 7f03fec61640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:52.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.371+0000 7f03fec61640 1 -- 172.21.15.115:0/3586142633 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f03f8067c70 con 0x7f03f809f7f0 2024-01-24T23:29:52.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.371+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f03e8004c10 con 0x7f03f813a470 2024-01-24T23:29:52.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.371+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7f03e80442a0 con 0x7f03f813a470 2024-01-24T23:29:52.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.371+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3253175290 0 0) 0x7f03f8067c70 con 0x7f03f809f7f0 2024-01-24T23:29:52.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.371+0000 7f03f57fa640 10 monclient(hunting): _negotiate 2024-01-24T23:29:52.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.371+0000 7f03f57fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:52.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.371+0000 7f03f57fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:52.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.371+0000 7f03f57fa640 10 monclient(hunting): my global_id is 154244 2024-01-24T23:29:52.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.371+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03f8071c30 con 0x7f03f809f7f0 2024-01-24T23:29:52.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.371+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4022222773 0 0) 0x7f03f8071c30 con 0x7f03f809f7f0 2024-01-24T23:29:52.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.371+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f03f8067c70 con 0x7f03f809f7f0 2024-01-24T23:29:52.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.371+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f03f0004eb0 con 0x7f03f809f7f0 2024-01-24T23:29:52.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.371+0000 7f03f57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:52.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.371+0000 7f03f57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:29:52.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.371+0000 7f03f57fa640 10 monclient: dump: 2024-01-24T23:29:52.391 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:52.391 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:52.391 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:52.392 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:52.392 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:52.392 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:52.392 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:52.392 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:52.392 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:52.392 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:52.392 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:52.392 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:52.392 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:52.393 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:52.393 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:52.393 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:52.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.371+0000 7f03f7fff640 10 monclient: get_auth_request con 0x7f03d8036f60 auth_method 0 2024-01-24T23:29:52.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.371+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2080807502 0 0) 0x7f03f8067c70 con 0x7f03f809f7f0 2024-01-24T23:29:52.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.371+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 --> v1:172.21.15.115:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f03f8135e90 con 0x7f03f809f7f0 2024-01-24T23:29:52.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.371+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 <== mon.2 v1:172.21.15.115:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f03f0001a50 con 0x7f03f809f7f0 2024-01-24T23:29:52.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.371+0000 7f03f57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:52.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.371+0000 7f03f57fa640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:52.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.371+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 >> v1:172.21.15.115:6790/0 conn(0x7f03f809f7f0 legacy=0x7f03f809fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:52.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.371+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 >> v1:172.21.15.115:6790/0 conn(0x7f03f809f7f0 legacy=0x7f03f809fbd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:52.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.375+0000 7f03fec61640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:29:52.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.375+0000 7f03fec61640 1 -- 172.21.15.115:0/3586142633 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f03f8069c80 con 0x7f03f80a02e0 2024-01-24T23:29:52.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.375+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2633746000 0 0) 0x7f03f8069c80 con 0x7f03f80a02e0 2024-01-24T23:29:52.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.375+0000 7f03f57fa640 10 monclient(hunting): _negotiate 2024-01-24T23:29:52.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.375+0000 7f03f57fa640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:52.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.375+0000 7f03f57fa640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:52.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.375+0000 7f03f57fa640 10 monclient(hunting): my global_id is 154253 2024-01-24T23:29:52.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.375+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03f8067c70 con 0x7f03f80a02e0 2024-01-24T23:29:52.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.375+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 315098216 0 0) 0x7f03f8067c70 con 0x7f03f80a02e0 2024-01-24T23:29:52.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.375+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f03f8069c80 con 0x7f03f80a02e0 2024-01-24T23:29:52.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.375+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f03f00057b0 con 0x7f03f80a02e0 2024-01-24T23:29:52.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.375+0000 7f03f57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:52.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.375+0000 7f03f57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:29:52.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.375+0000 7f03f57fa640 10 monclient: dump: 2024-01-24T23:29:52.400 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:52.400 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:52.400 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:52.400 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:52.400 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:52.400 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:52.400 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:52.400 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:52.400 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:52.401 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:52.401 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:52.401 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:52.401 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:52.401 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:52.401 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:52.401 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:52.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.379+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3116381256 0 0) 0x7f03f8069c80 con 0x7f03f80a02e0 2024-01-24T23:29:52.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.379+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 --> v1:172.21.15.115:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f03f80a1080 con 0x7f03f80a02e0 2024-01-24T23:29:52.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.379+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 <== mon.2 v1:172.21.15.115:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7680 (unknown 0 0 1155945418) 0x7f03f00086f0 con 0x7f03f80a02e0 2024-01-24T23:29:52.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.379+0000 7f03f57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:29:52.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.379+0000 7f03f57fa640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:52.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.379+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 >> v1:172.21.15.115:6790/0 conn(0x7f03f80a02e0 legacy=0x7f03f81d8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:52.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.379+0000 7f03f57fa640 1 -- 172.21.15.115:0/3586142633 >> v1:172.21.15.115:6790/0 conn(0x7f03f80a02e0 legacy=0x7f03f81d8db0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:52.402 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:29:52.402 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:52.402 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:52.402 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:29:52.403 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 52, 2024-01-24T23:29:52.403 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:29:52.403 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:29:52.403 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:29:52.403 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:29:52.403 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:29:52.403 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:29:52.403 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:29:52.403 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:52.403 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 0, 2024-01-24T23:29:52.403 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:52.403 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:29:52.403 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:29:52.404 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:52.404 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:52.404 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:52.404 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:52.404 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:52.404 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:52.404 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:52.404 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:52.404 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:52.404 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:52.405 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:52.405 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:52.405 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:29:52.405 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:29:52.405 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:52.405 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:52.405 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:52.405 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:52.405 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:52.405 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:52.405 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:52.405 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:52.405 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:52.406 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:52.406 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:52.406 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:52.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:52.406 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:29:52.406 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:29:52.406 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:29:52.406 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:29:52.406 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:29:52.406 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:29:52.406 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:52.406 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:52.406 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:29:52.406 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:29:52.407 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:29:52.407 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:29:52.407 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:29:52.407 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:29:52.407 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:29:52.407 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:52.407 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:29:52.407 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:52.407 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:52.407 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:52.407 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:52.408 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:52.408 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:52.408 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:52.408 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:52.408 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:52.408 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:52.408 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:52.408 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:52.408 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:29:52.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:52.408 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:29:52.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:52.408 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:52.408 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:52.409 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:52.409 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:52.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:52.409 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:52.409 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:29:52.409 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:52.409 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:52.409 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:52.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:52.409 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:29:52.409 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:29:52.409 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:52.409 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:52.410 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:52.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:52.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:52.410 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:52.410 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:52.410 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:52.410 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:52.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:52.410 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:52.410 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:29:52.410 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:52.410 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:52.410 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:52.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:52.410 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:29:52.411 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:29:52.411 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:52.411 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:52.411 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:52.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:52.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:52.411 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:52.411 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:52.411 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:52.411 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:52.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:52.411 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:52.411 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:29:52.411 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:52.412 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:52.412 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:52.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:52.412 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:29:52.412 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:29:52.412 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:52.412 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:52.412 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:52.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:52.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:52.412 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:52.412 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:52.412 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:52.413 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:52.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:52.413 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:52.413 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:29:52.413 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:52.413 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:52.413 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:52.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:52.413 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:29:52.413 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:29:52.413 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:52.413 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:52.413 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:52.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:52.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:52.414 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:52.414 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:52.414 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:52.414 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:52.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:52.414 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:52.414 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:29:52.414 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:52.414 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:52.414 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:52.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:52.414 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:29:52.415 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:29:52.415 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:52.415 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:52.415 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:52.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:52.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:52.415 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:52.415 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:52.415 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:52.415 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:52.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:52.415 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:52.415 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:29:52.415 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:52.416 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:52.416 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:52.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:52.416 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:29:52.416 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:29:52.416 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:52.416 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:52.416 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:52.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:52.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:52.416 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:52.416 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:52.416 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:52.416 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:52.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:52.417 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:52.417 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:29:52.417 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:52.417 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:52.417 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:52.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:52.417 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:29:52.417 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:29:52.417 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:52.417 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:52.417 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:52.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:52.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:52.418 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:52.418 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:52.418 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:52.418 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:52.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:52.418 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:52.418 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:29:52.418 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:52.418 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:52.418 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:52.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:52.419 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:29:52.419 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:29:52.419 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:52.419 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:52.419 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:52.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:52.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:52.419 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:52.419 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:52.419 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:52.420 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:52.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:52.420 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:52.420 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:29:52.420 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:52.420 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:52.420 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:52.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:52.420 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:29:52.420 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:29:52.420 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:52.420 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:52.420 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:52.421 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:52.421 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:52.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:52.421 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:29:52.421 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:29:52.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:52.421 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:52.421 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:52.421 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:52.421 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:52.421 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:52.421 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:29:52.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:52.422 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:52.422 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:52.422 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:29:52.422 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:52.422 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:52.422 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:29:52.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:52.422 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:52.422 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:52.422 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:52.422 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:52.422 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:52.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:52.422 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:29:52.423 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:29:52.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.379+0000 7f03fec61640 1 -- 172.21.15.115:0/3586142633 >> v1:172.21.15.132:6812/1835401922 conn(0x7f03d8036f60 legacy=0x7f03d8039420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:52.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.379+0000 7f03fec61640 10 monclient: shutdown 2024-01-24T23:29:52.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.379+0000 7f03fec61640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:52.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.379+0000 7f03fd1fe640 1 -- 172.21.15.115:0/3586142633 reap_dead start 2024-01-24T23:29:52.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.379+0000 7f03fec61640 1 -- 172.21.15.115:0/3586142633 >> v1:172.21.15.115:6793/0 conn(0x7f03f813a470 legacy=0x7f03f81d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:52.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.379+0000 7f03fec61640 1 -- 172.21.15.115:0/3586142633 shutdown_connections 2024-01-24T23:29:52.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.379+0000 7f03fec61640 1 -- 172.21.15.115:0/3586142633 >> 172.21.15.115:0/3586142633 conn(0x7f03f80a5730 msgr2=0x7f03f8138f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:52.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.379+0000 7f03fec61640 1 -- 172.21.15.115:0/3586142633 shutdown_connections 2024-01-24T23:29:52.425 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.379+0000 7f03fec61640 1 -- 172.21.15.115:0/3586142633 wait complete. 2024-01-24T23:29:52.425 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-01-24T23:29:52.425 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon d 2024-01-24T23:29:52.425 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-01-24T23:29:52.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 10 monclient: get_monmap_and_config 2024-01-24T23:29:52.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 10 monclient: build_initial_monmap 2024-01-24T23:29:52.532 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 10 monclient: monmap: 2024-01-24T23:29:52.532 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:52.532 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:52.532 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:52.535016+0000 2024-01-24T23:29:52.532 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:52.535016+0000 2024-01-24T23:29:52.532 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:52.532 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:52.533 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:52.533 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:52.533 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:52.533 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:52.533 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:52.533 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:52.533 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:52.533 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:52.533 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:52.533 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:52.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 1 Processor -- start 2024-01-24T23:29:52.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 1 -- start start 2024-01-24T23:29:52.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 10 monclient: init 2024-01-24T23:29:52.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:52.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 10 monclient: _add_conns ranks=[6,7,8,1,0,2,5,4,3] 2024-01-24T23:29:52.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 10 monclient(hunting): picked mon.noname-d con 0x7f1264136d80 addr v1:172.21.15.115:6792/0 2024-01-24T23:29:52.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 10 monclient(hunting): picked mon.noname-i con 0x7f12641368c0 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:52.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 10 monclient(hunting): picked mon.noname-e con 0x7f126413d360 addr v1:172.21.15.115:6793/0 2024-01-24T23:29:52.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f126404ed70 con 0x7f1264136d80 2024-01-24T23:29:52.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f126409f280 con 0x7f12641368c0 2024-01-24T23:29:52.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f126409edf0 con 0x7f126413d360 2024-01-24T23:29:52.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:52.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 10 monclient(hunting): authenticate will time out at 1471.457614s 2024-01-24T23:29:52.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1263fff640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f1264136d80 0x7f1264137160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:57714/0 (socket says 172.21.15.115:57714) 2024-01-24T23:29:52.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1268f41640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f126413d360 0x7f126413f750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:58366/0 (socket says 172.21.15.115:58366) 2024-01-24T23:29:52.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1263fff640 1 -- 172.21.15.115:0/826052040 learned_addr learned my addr 172.21.15.115:0/826052040 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:52.539 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 1 -- 172.21.15.115:0/826052040 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1129076208 0 0) 0x7f126404ed70 con 0x7f1264136d80 2024-01-24T23:29:52.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:29:52.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:52.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:52.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 10 monclient(hunting): my global_id is 7404 2024-01-24T23:29:52.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 1 -- 172.21.15.115:0/826052040 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1250003530 con 0x7f1264136d80 2024-01-24T23:29:52.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 1 -- 172.21.15.115:0/826052040 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991098587 0 0) 0x7f126409edf0 con 0x7f126413d360 2024-01-24T23:29:52.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:29:52.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:52.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:52.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 10 monclient(hunting): my global_id is 86471 2024-01-24T23:29:52.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 1 -- 172.21.15.115:0/826052040 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f126404ed70 con 0x7f126413d360 2024-01-24T23:29:52.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 1 -- 172.21.15.115:0/826052040 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4223831390 0 0) 0x7f1250003530 con 0x7f1264136d80 2024-01-24T23:29:52.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 1 -- 172.21.15.115:0/826052040 >> v1:172.21.15.115:6793/0 conn(0x7f126413d360 legacy=0x7f126413f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:52.540 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 1 -- 172.21.15.115:0/826052040 >> v1:172.21.15.132:6792/0 conn(0x7f12641368c0 legacy=0x7f126413bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:52.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:29:52.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 1 monclient: found mon.noname-d 2024-01-24T23:29:52.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:52.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.115:6792/0 2024-01-24T23:29:52.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 1 -- 172.21.15.115:0/826052040 --> v1:172.21.15.115:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f126406d4d0 con 0x7f1264136d80 2024-01-24T23:29:52.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 10 monclient: _finish_auth 0 2024-01-24T23:29:52.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:29:52.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:52.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 5 monclient: authenticate success, global_id 7404 2024-01-24T23:29:52.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 1 -- 172.21.15.115:0/826052040 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f125c001b40 con 0x7f1264136d80 2024-01-24T23:29:52.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:52.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-01-24T23:29:52.541 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 10 monclient: dump: 2024-01-24T23:29:52.541 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:52.542 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:52.542 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:52.542 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:52.542 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:52.542 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:52.542 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:52.542 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:52.542 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:52.542 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:52.542 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:52.542 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:52.542 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:52.542 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:52.542 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:52.542 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:52.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:52.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 1 -- 172.21.15.115:0/826052040 <== mon.6 v1:172.21.15.115:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f125c001e20 con 0x7f1264136d80 2024-01-24T23:29:52.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:52.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:52.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 1 -- 172.21.15.115:0/826052040 <== mon.6 v1:172.21.15.115:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f125c004f70 con 0x7f1264136d80 2024-01-24T23:29:52.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 4 set_mon_vals no callback set 2024-01-24T23:29:52.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:52.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:29:52.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f1262ffd640 10 monclient: dump: 2024-01-24T23:29:52.543 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:52.543 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:52.543 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:52.543 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:52.543 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:52.544 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:52.544 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:52.544 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:52.544 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:52.544 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:52.544 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:52.544 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:52.544 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:52.544 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:52.544 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:52.544 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:52.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 10 monclient: shutdown 2024-01-24T23:29:52.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:52.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 1 -- 172.21.15.115:0/826052040 >> v1:172.21.15.115:6792/0 conn(0x7f1264136d80 legacy=0x7f1264137160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:52.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 1 -- 172.21.15.115:0/826052040 shutdown_connections 2024-01-24T23:29:52.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 1 -- 172.21.15.115:0/826052040 >> 172.21.15.115:0/826052040 conn(0x7f12640a5d40 msgr2=0x7f12640a2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:52.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 1 -- 172.21.15.115:0/826052040 shutdown_connections 2024-01-24T23:29:52.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 1 -- 172.21.15.115:0/826052040 wait complete. 2024-01-24T23:29:52.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 10 monclient: build_initial_monmap 2024-01-24T23:29:52.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 10 monclient: monmap: 2024-01-24T23:29:52.545 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:52.545 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:52.545 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:52.545 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:52.545 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:52.545 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:52.545 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:52.545 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:52.545 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:52.546 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:52.546 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:52.546 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:52.546 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:52.546 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:52.546 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:52.546 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:52.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.531+0000 7f126a9a4640 1 Processor -- start 2024-01-24T23:29:52.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.535+0000 7f126a9a4640 1 -- start start 2024-01-24T23:29:52.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.535+0000 7f126a9a4640 10 monclient: init 2024-01-24T23:29:52.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.535+0000 7f126a9a4640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:52.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.535+0000 7f126a9a4640 10 monclient: _add_conns ranks=[0,4,6,2,1,7,3,5,8] 2024-01-24T23:29:52.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.535+0000 7f126a9a4640 10 monclient(hunting): picked mon.noname-a con 0x7f12641368c0 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:52.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.535+0000 7f126a9a4640 10 monclient(hunting): picked mon.noname-c con 0x7f1264136d80 addr v1:172.21.15.115:6791/0 2024-01-24T23:29:52.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.535+0000 7f126a9a4640 10 monclient(hunting): picked mon.noname-d con 0x7f126413d360 addr v1:172.21.15.115:6792/0 2024-01-24T23:29:52.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.535+0000 7f126a9a4640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1264132760 con 0x7f12641368c0 2024-01-24T23:29:52.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.535+0000 7f126a9a4640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1264071590 con 0x7f1264136d80 2024-01-24T23:29:52.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.535+0000 7f126a9a4640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f126409f920 con 0x7f126413d360 2024-01-24T23:29:52.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.535+0000 7f126a9a4640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:52.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.535+0000 7f126a9a4640 10 monclient(hunting): authenticate will time out at 1471.460485s 2024-01-24T23:29:52.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.535+0000 7f1263fff640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f12641368c0 legacy=0x7f126406f9a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:29:52.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.535+0000 7f12637fe640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f1264136d80 legacy=0x7f12640700b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:29:52.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.535+0000 7f1268f41640 1 --1- >> v1:172.21.15.115:6792/0 conn(0x7f126413d360 0x7f12641d3740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6792/0 says I am v1:172.21.15.115:57726/0 (socket says 172.21.15.115:57726) 2024-01-24T23:29:52.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.535+0000 7f1268f41640 1 -- 172.21.15.115:0/3815476017 learned_addr learned my addr 172.21.15.115:0/3815476017 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:52.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.535+0000 7f1268f41640 1 -- 172.21.15.115:0/3815476017 >> v1:172.21.15.115:6792/0 conn(0x7f126413d360 legacy=0x7f12641d3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-01-24T23:29:52.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.535+0000 7f1268f41640 1 -- 172.21.15.115:0/3815476017 >> v1:172.21.15.115:6792/0 conn(0x7f126413d360 legacy=0x7f12641d3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:29:52.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.535+0000 7f1268f41640 1 --1- 172.21.15.115:0/3815476017 >> v1:172.21.15.115:6792/0 conn(0x7f126413d360 0x7f12641d3740 :-1 s=OPENED pgs=632 cs=1 l=1).handle_message read tag failed 2024-01-24T23:29:52.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.535+0000 7f1268f41640 1 --1- 172.21.15.115:0/3815476017 >> v1:172.21.15.115:6792/0 conn(0x7f126413d360 0x7f12641d3740 :-1 s=OPENED pgs=632 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:29:52.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.535+0000 7f1260ff9640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6792/0 2024-01-24T23:29:52.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.739+0000 7f12637fe640 1 -- 172.21.15.115:0/3815476017 >> v1:172.21.15.115:6791/0 conn(0x7f1264136d80 legacy=0x7f12640700b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:29:52.740 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:52.739+0000 7f1263fff640 1 -- 172.21.15.115:0/3815476017 >> v1:172.21.15.115:6789/0 conn(0x7f12641368c0 legacy=0x7f126406f9a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:29:53.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:53.143+0000 7f12637fe640 1 -- 172.21.15.115:0/3815476017 >> v1:172.21.15.115:6791/0 conn(0x7f1264136d80 legacy=0x7f12640700b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:29:53.144 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:53.143+0000 7f1263fff640 1 -- 172.21.15.115:0/3815476017 >> v1:172.21.15.115:6789/0 conn(0x7f12641368c0 legacy=0x7f126406f9a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:29:53.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:53.947+0000 7f12637fe640 1 -- 172.21.15.115:0/3815476017 >> v1:172.21.15.115:6791/0 conn(0x7f1264136d80 legacy=0x7f12640700b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:29:53.945 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:53.947+0000 7f1263fff640 1 -- 172.21.15.115:0/3815476017 >> v1:172.21.15.115:6789/0 conn(0x7f12641368c0 legacy=0x7f126406f9a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:29:55.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f125b7fe640 10 monclient(hunting): tick 2024-01-24T23:29:55.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f125b7fe640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:29:55.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f125b7fe640 1 monclient(hunting): continuing hunt 2024-01-24T23:29:55.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f125b7fe640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:29:55.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f125b7fe640 1 -- 172.21.15.115:0/3815476017 >> v1:172.21.15.115:6792/0 conn(0x7f126413d360 legacy=0x7f12641d3740 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:55.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f125b7fe640 1 -- 172.21.15.115:0/3815476017 >> v1:172.21.15.115:6791/0 conn(0x7f1264136d80 legacy=0x7f12640700b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:29:55.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f125b7fe640 1 -- 172.21.15.115:0/3815476017 >> v1:172.21.15.115:6789/0 conn(0x7f12641368c0 legacy=0x7f126406f9a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:29:55.537 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f125b7fe640 10 monclient: _add_conns ranks=[8,7,5,3,1,2] 2024-01-24T23:29:55.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f125b7fe640 10 monclient(hunting): picked mon.noname-e con 0x7f12480036e0 addr v1:172.21.15.115:6793/0 2024-01-24T23:29:55.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f125b7fe640 10 monclient(hunting): picked mon.noname-i con 0x7f1248007370 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:55.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f125b7fe640 10 monclient(hunting): picked mon.noname-h con 0x7f124800b100 addr v1:172.21.15.132:6791/0 2024-01-24T23:29:55.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f125b7fe640 1 -- 172.21.15.115:0/3815476017 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12640707c0 con 0x7f124800b100 2024-01-24T23:29:55.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f125b7fe640 1 -- 172.21.15.115:0/3815476017 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1264132760 con 0x7f1248007370 2024-01-24T23:29:55.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f125b7fe640 1 -- 172.21.15.115:0/3815476017 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1264071590 con 0x7f12480036e0 2024-01-24T23:29:55.538 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f125b7fe640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:55.542 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 99209784 0 0) 0x7f1264071590 con 0x7f12480036e0 2024-01-24T23:29:55.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:29:55.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:55.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:55.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 10 monclient(hunting): my global_id is 86480 2024-01-24T23:29:55.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1244003560 con 0x7f12480036e0 2024-01-24T23:29:55.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3261664653 0 0) 0x7f1264132760 con 0x7f1248007370 2024-01-24T23:29:55.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:29:55.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:55.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:55.543 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 10 monclient(hunting): my global_id is 154303 2024-01-24T23:29:55.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1264071590 con 0x7f1248007370 2024-01-24T23:29:55.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 910455615 0 0) 0x7f12640707c0 con 0x7f124800b100 2024-01-24T23:29:55.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:29:55.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:55.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:55.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 10 monclient(hunting): my global_id is 7196 2024-01-24T23:29:55.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1264132760 con 0x7f124800b100 2024-01-24T23:29:55.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2561124965 0 0) 0x7f1244003560 con 0x7f12480036e0 2024-01-24T23:29:55.544 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f12640707c0 con 0x7f12480036e0 2024-01-24T23:29:55.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f125c004f70 con 0x7f12480036e0 2024-01-24T23:29:55.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:55.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:29:55.545 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 10 monclient(hunting): dump: 2024-01-24T23:29:55.545 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:55.545 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:55.545 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.545 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.545 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:55.545 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:55.545 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:55.545 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:55.545 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:55.545 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:55.546 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:55.546 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:55.546 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:55.546 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:55.546 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:55.546 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:55.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:55.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 764205705 0 0) 0x7f12640707c0 con 0x7f12480036e0 2024-01-24T23:29:55.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 >> v1:172.21.15.132:6792/0 conn(0x7f1248007370 legacy=0x7f1248009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:55.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 >> v1:172.21.15.132:6791/0 conn(0x7f124800b100 legacy=0x7f124800d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:55.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:29:55.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 1 monclient: found mon.e 2024-01-24T23:29:55.546 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1268f41640 1 -- 172.21.15.115:0/3815476017 reap_dead start 2024-01-24T23:29:55.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:55.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:29:55.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1248011f90 con 0x7f12480036e0 2024-01-24T23:29:55.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 10 monclient: _finish_auth 0 2024-01-24T23:29:55.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:29:55.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:55.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f126a9a4640 5 monclient: authenticate success, global_id 86480 2024-01-24T23:29:55.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f126a9a4640 10 monclient: _renew_subs 2024-01-24T23:29:55.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f126a9a4640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:29:55.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f126a9a4640 1 -- 172.21.15.115:0/3815476017 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1264132480 con 0x7f12480036e0 2024-01-24T23:29:55.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f126a9a4640 10 monclient: _renew_subs 2024-01-24T23:29:55.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f126a9a4640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:29:55.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f126a9a4640 1 -- 172.21.15.115:0/3815476017 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1264130f00 con 0x7f12480036e0 2024-01-24T23:29:55.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f125c0015c0 con 0x7f12480036e0 2024-01-24T23:29:55.547 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:55.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f125c005580 con 0x7f12480036e0 2024-01-24T23:29:55.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:55.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f12627fc640 4 set_mon_vals no callback set 2024-01-24T23:29:55.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:29:55.548 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f1260ff9640 10 monclient: dump: 2024-01-24T23:29:55.548 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:55.548 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:55.548 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.548 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.548 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:55.548 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:55.548 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:55.548 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:55.549 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:55.549 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:55.549 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:55.549 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:55.549 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:55.549 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:55.549 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:55.549 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:55.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.535+0000 7f126a9a4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:55.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.539+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f125c010000 con 0x7f12480036e0 2024-01-24T23:29:55.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.539+0000 7f126a9a4640 1 -- 172.21.15.115:0/3815476017 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f123c0065d0 con 0x7f123c002870 2024-01-24T23:29:55.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.539+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7f125c044980 con 0x7f12480036e0 2024-01-24T23:29:55.549 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.539+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392406466 0 0) 0x7f123c0065d0 con 0x7f123c002870 2024-01-24T23:29:55.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.539+0000 7f1260ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:29:55.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.539+0000 7f1260ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:55.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.539+0000 7f1260ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:55.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.539+0000 7f1260ff9640 10 monclient(hunting): my global_id is 7413 2024-01-24T23:29:55.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.539+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12640707c0 con 0x7f123c002870 2024-01-24T23:29:55.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.539+0000 7f1268f41640 10 monclient: get_auth_request con 0x7f12440370b0 auth_method 0 2024-01-24T23:29:55.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.539+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3390821985 0 0) 0x7f12640707c0 con 0x7f123c002870 2024-01-24T23:29:55.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.539+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f123c0065d0 con 0x7f123c002870 2024-01-24T23:29:55.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.539+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f124c004cd0 con 0x7f123c002870 2024-01-24T23:29:55.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.539+0000 7f1260ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:55.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.539+0000 7f1260ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:29:55.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.539+0000 7f1260ff9640 10 monclient: dump: 2024-01-24T23:29:55.551 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:55.551 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:55.551 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.551 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.551 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:55.551 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:55.551 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:55.551 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:55.551 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:55.551 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:55.551 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:55.551 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:55.551 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:55.552 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:55.552 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:55.552 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:55.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.539+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1902238900 0 0) 0x7f123c0065d0 con 0x7f123c002870 2024-01-24T23:29:55.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.539+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 --> v1:172.21.15.115:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f123c00a120 con 0x7f123c002870 2024-01-24T23:29:55.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.539+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f124c0051f0 con 0x7f123c002870 2024-01-24T23:29:55.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.539+0000 7f1260ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:55.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.539+0000 7f1260ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:55.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.539+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 >> v1:172.21.15.115:6792/0 conn(0x7f123c002870 legacy=0x7f123c004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:55.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.539+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 >> v1:172.21.15.115:6792/0 conn(0x7f123c002870 legacy=0x7f123c004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:55.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f126a9a4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:29:55.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f126a9a4640 1 -- 172.21.15.115:0/3815476017 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f123c009df0 con 0x7f123c0090b0 2024-01-24T23:29:55.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 <== mon.6 v1:172.21.15.115:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 270942188 0 0) 0x7f123c009df0 con 0x7f123c0090b0 2024-01-24T23:29:55.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f1260ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:29:55.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f1260ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:55.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f1260ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:55.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f1260ff9640 10 monclient(hunting): my global_id is 7422 2024-01-24T23:29:55.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 --> v1:172.21.15.115:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f123c0065d0 con 0x7f123c0090b0 2024-01-24T23:29:55.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 <== mon.6 v1:172.21.15.115:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 987921362 0 0) 0x7f123c0065d0 con 0x7f123c0090b0 2024-01-24T23:29:55.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 --> v1:172.21.15.115:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f123c009df0 con 0x7f123c0090b0 2024-01-24T23:29:55.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 <== mon.6 v1:172.21.15.115:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f124c0058d0 con 0x7f123c0090b0 2024-01-24T23:29:55.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f1260ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:55.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f1260ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-01-24T23:29:55.555 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f1260ff9640 10 monclient: dump: 2024-01-24T23:29:55.555 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:55.555 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:55.555 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.555 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.555 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:55.555 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:55.555 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:55.555 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:55.555 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:55.555 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:55.555 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:55.555 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:55.556 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:55.556 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:55.556 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:55.556 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:55.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 <== mon.6 v1:172.21.15.115:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3065645498 0 0) 0x7f123c009df0 con 0x7f123c0090b0 2024-01-24T23:29:55.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 --> v1:172.21.15.115:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f123c00c310 con 0x7f123c0090b0 2024-01-24T23:29:55.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 <== mon.6 v1:172.21.15.115:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7680 (unknown 0 0 1987979978) 0x7f124c005ce0 con 0x7f123c0090b0 2024-01-24T23:29:55.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f1260ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:29:55.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f1260ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:55.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 >> v1:172.21.15.115:6792/0 conn(0x7f123c0090b0 legacy=0x7f123c009540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:55.556 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f1260ff9640 1 -- 172.21.15.115:0/3815476017 >> v1:172.21.15.115:6792/0 conn(0x7f123c0090b0 legacy=0x7f123c009540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:55.557 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:29:55.557 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:55.557 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:55.557 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:29:55.557 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 52, 2024-01-24T23:29:55.557 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:29:55.557 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:29:55.557 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:29:55.557 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:29:55.558 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:29:55.558 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:29:55.558 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:29:55.558 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:55.558 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 3, 2024-01-24T23:29:55.558 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:55.558 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:29:55.558 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:29:55.558 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:55.558 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:55.558 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:55.558 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:55.558 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:55.559 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:55.559 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:55.559 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:55.559 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:55.559 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:55.559 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:55.559 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:55.559 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:29:55.559 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:29:55.559 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:55.559 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:55.559 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:55.559 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:55.559 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:55.560 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:55.560 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:55.560 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:55.560 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:55.560 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:55.560 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:55.560 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.560 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:29:55.560 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:29:55.560 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:29:55.560 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:29:55.560 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:29:55.560 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:29:55.561 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:55.561 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:55.561 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:29:55.561 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:29:55.561 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:29:55.561 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:29:55.561 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:29:55.561 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:29:55.561 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:29:55.561 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:55.561 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:29:55.561 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:55.561 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:55.561 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:55.562 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:55.562 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:55.562 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:55.562 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:55.562 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:55.562 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:55.562 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:55.562 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:55.562 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:55.562 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:29:55.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.562 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:29:55.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.562 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:55.563 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:55.563 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.563 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.563 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.563 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:29:55.563 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.563 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.563 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.563 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:29:55.563 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:29:55.563 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.563 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.564 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.564 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:55.564 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:55.564 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.564 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.564 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.564 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:29:55.564 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.564 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.564 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.565 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:29:55.565 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:29:55.565 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.565 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.565 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.565 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:55.566 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:55.566 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.566 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.566 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.566 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:29:55.566 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.566 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.566 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.566 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:29:55.566 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:29:55.566 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.566 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.567 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.567 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:55.567 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:55.567 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.567 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.567 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.567 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:29:55.567 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.567 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.567 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.568 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:29:55.568 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:29:55.568 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.568 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.568 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.568 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:55.568 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:55.568 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.568 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.568 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.569 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:29:55.569 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.569 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.569 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.569 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:29:55.569 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:29:55.569 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.569 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.569 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.569 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:55.569 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:55.570 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.570 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.570 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.570 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:29:55.570 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.570 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.570 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.570 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:29:55.570 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:29:55.570 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.571 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.571 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.571 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:55.571 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:55.571 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.571 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.571 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.571 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:29:55.571 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.571 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.571 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.572 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:29:55.572 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:29:55.572 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.572 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.572 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.572 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:55.572 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:55.572 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.572 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.572 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.572 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:29:55.573 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.573 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.573 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.573 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:29:55.573 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:29:55.573 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.573 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.573 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.573 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:55.573 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:55.573 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.573 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.574 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.574 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:29:55.574 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.574 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.574 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.574 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:29:55.574 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:29:55.574 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.574 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.574 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.574 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.575 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.575 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:29:55.575 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:29:55.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.575 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:55.575 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:55.575 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:55.575 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.575 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:55.576 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:29:55.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.576 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:55.576 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:55.576 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:55.576 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.576 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:55.576 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:29:55.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.576 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:55.576 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:55.576 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:29:55.576 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.577 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.577 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:29:55.577 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:29:55.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f126a9a4640 1 -- 172.21.15.115:0/3815476017 >> v1:172.21.15.132:6812/1835401922 conn(0x7f12440370b0 legacy=0x7f1244039570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:55.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f126a9a4640 10 monclient: shutdown 2024-01-24T23:29:55.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f126a9a4640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:55.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f126a9a4640 1 -- 172.21.15.115:0/3815476017 >> v1:172.21.15.115:6793/0 conn(0x7f12480036e0 legacy=0x7f1248005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:55.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f126a9a4640 1 -- 172.21.15.115:0/3815476017 shutdown_connections 2024-01-24T23:29:55.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f126a9a4640 1 -- 172.21.15.115:0/3815476017 >> 172.21.15.115:0/3815476017 conn(0x7f12640a5d40 msgr2=0x7f126412d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:55.578 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f126a9a4640 1 -- 172.21.15.115:0/3815476017 shutdown_connections 2024-01-24T23:29:55.579 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.547+0000 7f126a9a4640 1 -- 172.21.15.115:0/3815476017 wait complete. 2024-01-24T23:29:55.579 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-01-24T23:29:55.579 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon e 2024-01-24T23:29:55.579 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-01-24T23:29:55.587 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~231s 2024-01-24T23:29:55.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 10 monclient: get_monmap_and_config 2024-01-24T23:29:55.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 10 monclient: build_initial_monmap 2024-01-24T23:29:55.700 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 10 monclient: monmap: 2024-01-24T23:29:55.700 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:55.700 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:55.701 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:55.703224+0000 2024-01-24T23:29:55.701 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:55.703224+0000 2024-01-24T23:29:55.701 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:55.701 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:55.701 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:55.701 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:55.701 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:55.701 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:55.701 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:55.701 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:55.701 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:55.701 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:55.701 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:55.701 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:55.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 1 Processor -- start 2024-01-24T23:29:55.704 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 1 -- start start 2024-01-24T23:29:55.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 10 monclient: init 2024-01-24T23:29:55.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:55.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 10 monclient: _add_conns ranks=[7,4,1,2,6,0,8,5,3] 2024-01-24T23:29:55.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 10 monclient(hunting): picked mon.noname-i con 0x7f3764134e60 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:55.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 10 monclient(hunting): picked mon.noname-c con 0x7f3764134890 addr v1:172.21.15.115:6791/0 2024-01-24T23:29:55.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 10 monclient(hunting): picked mon.noname-f con 0x7f376413b2d0 addr v1:172.21.15.132:6789/0 2024-01-24T23:29:55.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f376404ed70 con 0x7f376413b2d0 2024-01-24T23:29:55.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3764134c90 con 0x7f3764134890 2024-01-24T23:29:55.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37640a0630 con 0x7f3764134e60 2024-01-24T23:29:55.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:55.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 10 monclient(hunting): authenticate will time out at 1474.625854s 2024-01-24T23:29:55.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3768b42640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f3764134890 legacy=0x7f3764139bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:29:55.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3769b44640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f376413b2d0 legacy=0x7f376413d6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:29:55.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3769343640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7f3764134e60 0x7f3764135240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.115:36788/0 (socket says 172.21.15.115:36788) 2024-01-24T23:29:55.705 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3769343640 1 -- 172.21.15.115:0/1028013975 learned_addr learned my addr 172.21.15.115:0/1028013975 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:55.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 1 -- 172.21.15.115:0/1028013975 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 521691560 0 0) 0x7f37640a0630 con 0x7f3764134e60 2024-01-24T23:29:55.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:55.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:55.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:55.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 10 monclient(hunting): my global_id is 154312 2024-01-24T23:29:55.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 1 -- 172.21.15.115:0/1028013975 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f374c003530 con 0x7f3764134e60 2024-01-24T23:29:55.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 1 -- 172.21.15.115:0/1028013975 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2800357141 0 0) 0x7f374c003530 con 0x7f3764134e60 2024-01-24T23:29:55.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 1 -- 172.21.15.115:0/1028013975 >> v1:172.21.15.115:6791/0 conn(0x7f3764134890 legacy=0x7f3764139bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:29:55.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 1 -- 172.21.15.115:0/1028013975 >> v1:172.21.15.132:6789/0 conn(0x7f376413b2d0 legacy=0x7f376413d6c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:29:55.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 10 monclient: _finish_hunting 0 2024-01-24T23:29:55.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 1 monclient: found mon.noname-i 2024-01-24T23:29:55.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:55.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:29:55.706 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 1 -- 172.21.15.115:0/1028013975 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37640a0350 con 0x7f3764134e60 2024-01-24T23:29:55.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 10 monclient: _finish_auth 0 2024-01-24T23:29:55.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 10 monclient: _check_auth_tickets 2024-01-24T23:29:55.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:55.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 1 -- 172.21.15.115:0/1028013975 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f375c001b40 con 0x7f3764134e60 2024-01-24T23:29:55.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:55.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:29:55.707 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 10 monclient: dump: 2024-01-24T23:29:55.707 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:55.707 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:55.707 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.707 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.707 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:55.707 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:55.707 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:55.707 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:55.708 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:55.708 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:55.708 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:55.708 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:55.708 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:55.708 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:55.708 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:55.708 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:55.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 5 monclient: authenticate success, global_id 154312 2024-01-24T23:29:55.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:55.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 1 -- 172.21.15.115:0/1028013975 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f375c001d60 con 0x7f3764134e60 2024-01-24T23:29:55.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:55.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:55.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 4 set_mon_vals no callback set 2024-01-24T23:29:55.708 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 1 -- 172.21.15.115:0/1028013975 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f375c004f70 con 0x7f3764134e60 2024-01-24T23:29:55.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:55.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:29:55.709 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f3763fff640 10 monclient: dump: 2024-01-24T23:29:55.709 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:55.709 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:55.709 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.709 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.709 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:55.709 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:55.709 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:55.709 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:55.709 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:55.709 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:55.709 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:55.709 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:55.709 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:55.710 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:55.710 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:55.710 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:55.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 10 monclient: shutdown 2024-01-24T23:29:55.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:55.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 1 -- 172.21.15.115:0/1028013975 >> v1:172.21.15.132:6792/0 conn(0x7f3764134e60 legacy=0x7f3764135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:55.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 1 -- 172.21.15.115:0/1028013975 shutdown_connections 2024-01-24T23:29:55.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.699+0000 7f376b5a7640 1 -- 172.21.15.115:0/1028013975 >> 172.21.15.115:0/1028013975 conn(0x7f37640a7820 msgr2=0x7f37640a3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:55.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f376b5a7640 1 -- 172.21.15.115:0/1028013975 shutdown_connections 2024-01-24T23:29:55.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f376b5a7640 1 -- 172.21.15.115:0/1028013975 wait complete. 2024-01-24T23:29:55.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f376b5a7640 10 monclient: build_initial_monmap 2024-01-24T23:29:55.710 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f376b5a7640 10 monclient: monmap: 2024-01-24T23:29:55.710 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:55.710 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:55.710 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:55.711 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:55.711 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:55.711 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:55.711 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:55.711 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:55.711 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:55.711 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:55.711 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:55.711 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:55.711 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:55.711 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:55.711 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:55.711 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:55.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f376b5a7640 1 Processor -- start 2024-01-24T23:29:55.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f376b5a7640 1 -- start start 2024-01-24T23:29:55.711 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f376b5a7640 10 monclient: init 2024-01-24T23:29:55.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f376b5a7640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:55.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f376b5a7640 10 monclient: _add_conns ranks=[5,3,7,4,8,6,0,1,2] 2024-01-24T23:29:55.717 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f376b5a7640 10 monclient(hunting): picked mon.noname-h con 0x7f3764134890 addr v1:172.21.15.132:6791/0 2024-01-24T23:29:55.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f376b5a7640 10 monclient(hunting): picked mon.noname-g con 0x7f3764134e60 addr v1:172.21.15.132:6790/0 2024-01-24T23:29:55.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f376b5a7640 10 monclient(hunting): picked mon.noname-i con 0x7f376413b2d0 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:55.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f376b5a7640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37641365d0 con 0x7f3764134e60 2024-01-24T23:29:55.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f376b5a7640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3764136010 con 0x7f3764134890 2024-01-24T23:29:55.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f376b5a7640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37641362f0 con 0x7f376413b2d0 2024-01-24T23:29:55.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f376b5a7640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:55.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f376b5a7640 10 monclient(hunting): authenticate will time out at 1474.629561s 2024-01-24T23:29:55.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3768b42640 1 --1- >> v1:172.21.15.132:6790/0 conn(0x7f3764134e60 0x7f376409b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6790/0 says I am v1:172.21.15.115:50402/0 (socket says 172.21.15.115:50402) 2024-01-24T23:29:55.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3768b42640 1 -- 172.21.15.115:0/525743966 learned_addr learned my addr 172.21.15.115:0/525743966 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:55.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1941252307 0 0) 0x7f37641365d0 con 0x7f3764134e60 2024-01-24T23:29:55.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:29:55.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:55.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:55.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient(hunting): my global_id is 7239 2024-01-24T23:29:55.718 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3750003530 con 0x7f3764134e60 2024-01-24T23:29:55.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2198760424 0 0) 0x7f37641362f0 con 0x7f376413b2d0 2024-01-24T23:29:55.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:29:55.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:55.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:55.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient(hunting): my global_id is 154321 2024-01-24T23:29:55.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37641365d0 con 0x7f376413b2d0 2024-01-24T23:29:55.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1891441986 0 0) 0x7f3764136010 con 0x7f3764134890 2024-01-24T23:29:55.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:29:55.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:55.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:55.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient(hunting): my global_id is 7205 2024-01-24T23:29:55.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37641362f0 con 0x7f3764134890 2024-01-24T23:29:55.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1727893631 0 0) 0x7f37641365d0 con 0x7f376413b2d0 2024-01-24T23:29:55.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3764136010 con 0x7f376413b2d0 2024-01-24T23:29:55.719 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f37540041e0 con 0x7f376413b2d0 2024-01-24T23:29:55.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:55.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:29:55.720 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient(hunting): dump: 2024-01-24T23:29:55.720 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:55.720 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:55.720 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.720 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.720 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:55.720 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:55.720 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:55.720 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:55.720 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:55.720 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:55.720 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:55.720 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:55.721 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:55.721 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:55.721 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:55.721 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:55.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:55.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 68054563 0 0) 0x7f3750003530 con 0x7f3764134e60 2024-01-24T23:29:55.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f37641365d0 con 0x7f3764134e60 2024-01-24T23:29:55.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f3758004300 con 0x7f3764134e60 2024-01-24T23:29:55.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:55.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:29:55.721 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient(hunting): dump: 2024-01-24T23:29:55.721 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:55.721 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:55.721 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.721 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.722 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:55.722 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:55.722 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:55.722 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:55.722 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:55.722 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:55.722 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:55.722 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:55.722 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:55.722 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:55.722 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:55.722 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:55.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1206673064 0 0) 0x7f37641362f0 con 0x7f3764134890 2024-01-24T23:29:55.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3750003530 con 0x7f3764134890 2024-01-24T23:29:55.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f375c001a40 con 0x7f3764134890 2024-01-24T23:29:55.722 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:55.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-01-24T23:29:55.723 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient(hunting): dump: 2024-01-24T23:29:55.723 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:55.723 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:55.723 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.723 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.723 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:55.723 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:55.723 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:55.723 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:55.723 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:55.723 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:55.723 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:55.723 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:55.723 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:55.723 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:55.724 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:55.724 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:55.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1734370450 0 0) 0x7f3764136010 con 0x7f376413b2d0 2024-01-24T23:29:55.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 >> v1:172.21.15.132:6791/0 conn(0x7f3764134890 legacy=0x7f376409b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:55.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 >> v1:172.21.15.132:6790/0 conn(0x7f3764134e60 legacy=0x7f376409b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:55.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient: _finish_hunting 0 2024-01-24T23:29:55.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 1 monclient: found mon.i 2024-01-24T23:29:55.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:55.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:29:55.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f376409d850 con 0x7f376413b2d0 2024-01-24T23:29:55.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient: _finish_auth 0 2024-01-24T23:29:55.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient: _check_auth_tickets 2024-01-24T23:29:55.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:55.724 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f376b5a7640 5 monclient: authenticate success, global_id 154321 2024-01-24T23:29:55.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f376b5a7640 10 monclient: _renew_subs 2024-01-24T23:29:55.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f376b5a7640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:29:55.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f376b5a7640 1 -- 172.21.15.115:0/525743966 --> v1:172.21.15.132:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f376409f780 con 0x7f376413b2d0 2024-01-24T23:29:55.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f376b5a7640 10 monclient: _renew_subs 2024-01-24T23:29:55.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f376b5a7640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.132:6792/0 2024-01-24T23:29:55.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f376b5a7640 1 -- 172.21.15.115:0/525743966 --> v1:172.21.15.132:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f376409d200 con 0x7f376413b2d0 2024-01-24T23:29:55.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 <== mon.7 v1:172.21.15.132:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3754004400 con 0x7f376413b2d0 2024-01-24T23:29:55.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:55.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 <== mon.7 v1:172.21.15.132:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f37540048b0 con 0x7f376413b2d0 2024-01-24T23:29:55.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:55.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:29:55.725 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f3761ffb640 10 monclient: dump: 2024-01-24T23:29:55.725 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:55.725 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:55.725 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.725 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.726 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:55.726 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:55.726 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:55.726 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:55.726 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:55.726 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:55.726 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:55.726 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:55.726 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:55.726 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:55.726 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:55.726 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:55.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.703+0000 7f37637fe640 4 set_mon_vals no callback set 2024-01-24T23:29:55.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.707+0000 7f376b5a7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:55.726 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.707+0000 7f376b5a7640 1 -- 172.21.15.115:0/525743966 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f37440065d0 con 0x7f3744002870 2024-01-24T23:29:55.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.707+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2120389996 0 0) 0x7f37440065d0 con 0x7f3744002870 2024-01-24T23:29:55.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.707+0000 7f3761ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:29:55.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.707+0000 7f3761ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:55.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.707+0000 7f3761ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:55.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.707+0000 7f3761ffb640 10 monclient(hunting): my global_id is 86489 2024-01-24T23:29:55.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.707+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3764136010 con 0x7f3744002870 2024-01-24T23:29:55.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.707+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 <== mon.7 v1:172.21.15.132:6792/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f3754004a90 con 0x7f376413b2d0 2024-01-24T23:29:55.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.707+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 <== mon.7 v1:172.21.15.132:6792/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7f37540442f0 con 0x7f376413b2d0 2024-01-24T23:29:55.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.707+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2786168232 0 0) 0x7f3764136010 con 0x7f3744002870 2024-01-24T23:29:55.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.707+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f37440065d0 con 0x7f3744002870 2024-01-24T23:29:55.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.707+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f375c005240 con 0x7f3744002870 2024-01-24T23:29:55.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.707+0000 7f3761ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:55.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.707+0000 7f3761ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:29:55.727 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.707+0000 7f3761ffb640 10 monclient: dump: 2024-01-24T23:29:55.727 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:55.727 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:55.728 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.728 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.728 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:55.728 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:55.728 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:55.728 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:55.728 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:55.728 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:55.728 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:55.728 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:55.728 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:55.728 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:55.729 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:55.729 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:55.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.707+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1951316828 0 0) 0x7f37440065d0 con 0x7f3744002870 2024-01-24T23:29:55.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.707+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 --> v1:172.21.15.115:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f374400a120 con 0x7f3744002870 2024-01-24T23:29:55.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.707+0000 7f3768b42640 10 monclient: get_auth_request con 0x7f3750037bc0 auth_method 0 2024-01-24T23:29:55.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.707+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f375c0054d0 con 0x7f3744002870 2024-01-24T23:29:55.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.707+0000 7f3761ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:55.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.707+0000 7f3761ffb640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:55.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.707+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 >> v1:172.21.15.115:6793/0 conn(0x7f3744002870 legacy=0x7f3744004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:55.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.707+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 >> v1:172.21.15.115:6793/0 conn(0x7f3744002870 legacy=0x7f3744004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:55.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.711+0000 7f376b5a7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:29:55.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.711+0000 7f376b5a7640 1 -- 172.21.15.115:0/525743966 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3744009e10 con 0x7f37440090d0 2024-01-24T23:29:55.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953647128 0 0) 0x7f3744009e10 con 0x7f37440090d0 2024-01-24T23:29:55.729 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f3761ffb640 10 monclient(hunting): _negotiate 2024-01-24T23:29:55.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f3761ffb640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:55.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f3761ffb640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:55.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f3761ffb640 10 monclient(hunting): my global_id is 86498 2024-01-24T23:29:55.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37440065d0 con 0x7f37440090d0 2024-01-24T23:29:55.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1813252180 0 0) 0x7f37440065d0 con 0x7f37440090d0 2024-01-24T23:29:55.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3744009e10 con 0x7f37440090d0 2024-01-24T23:29:55.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f375c005c30 con 0x7f37440090d0 2024-01-24T23:29:55.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f3761ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:55.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f3761ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:29:55.730 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f3761ffb640 10 monclient: dump: 2024-01-24T23:29:55.730 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:55.730 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:55.730 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.730 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.730 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:55.731 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:55.731 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:55.731 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:55.731 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:55.731 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:55.731 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:55.731 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:55.731 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:55.731 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:55.731 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:55.731 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:55.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4211409490 0 0) 0x7f3744009e10 con 0x7f37440090d0 2024-01-24T23:29:55.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 --> v1:172.21.15.115:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f374400c310 con 0x7f37440090d0 2024-01-24T23:29:55.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 <== mon.8 v1:172.21.15.115:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7510 (unknown 0 0 3698421801) 0x7f375c006150 con 0x7f37440090d0 2024-01-24T23:29:55.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f3761ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:29:55.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f3761ffb640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:55.731 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 >> v1:172.21.15.115:6793/0 conn(0x7f37440090d0 legacy=0x7f3744009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:55.732 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f3761ffb640 1 -- 172.21.15.115:0/525743966 >> v1:172.21.15.115:6793/0 conn(0x7f37440090d0 legacy=0x7f3744009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:55.732 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:29:55.732 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:55.732 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:55.732 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:29:55.732 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 52, 2024-01-24T23:29:55.732 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:29:55.732 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:29:55.732 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:29:55.732 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:29:55.733 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:29:55.733 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:29:55.733 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:29:55.733 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:55.733 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 3, 2024-01-24T23:29:55.733 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:55.733 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:29:55.733 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:29:55.733 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:55.733 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:55.733 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:55.733 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:55.733 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:55.733 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:55.734 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:55.734 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:55.734 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:55.734 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:55.734 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:55.734 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:55.734 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:29:55.734 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:29:55.734 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:55.734 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:55.734 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:55.734 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:55.734 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:55.734 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:55.734 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:55.735 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:55.735 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:55.735 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:55.735 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:55.735 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.735 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:29:55.735 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:29:55.735 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:29:55.735 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:29:55.735 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:29:55.735 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:29:55.735 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:55.735 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:55.736 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:29:55.736 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:29:55.736 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:29:55.736 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:29:55.736 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:29:55.736 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:29:55.736 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:29:55.736 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:55.736 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:29:55.736 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:55.736 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:55.736 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:55.736 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:55.736 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:55.737 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:55.737 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:55.737 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:55.737 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:55.737 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:55.737 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:55.737 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:55.737 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:29:55.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.737 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:29:55.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.737 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:55.737 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:55.737 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.738 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.738 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.738 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:29:55.738 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.738 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.738 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.738 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:29:55.738 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:29:55.738 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.738 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.738 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.739 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:55.739 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:55.739 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.739 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.739 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.739 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:29:55.739 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.739 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.739 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:29:55.739 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:29:55.739 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.740 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.740 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.740 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:55.740 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:55.740 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.740 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.740 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.740 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:29:55.740 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.740 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.740 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.741 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:29:55.741 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:29:55.741 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.741 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.741 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.741 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:55.741 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:55.741 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.741 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.741 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.741 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:29:55.741 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.742 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.742 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.742 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:29:55.742 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:29:55.742 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.742 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.742 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.742 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:55.742 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:55.742 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.742 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.743 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.743 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:29:55.743 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.743 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.743 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:29:55.743 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:29:55.743 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.743 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.743 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.743 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:55.744 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:55.744 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.744 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.744 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.744 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:29:55.744 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.744 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.744 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.744 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:29:55.744 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:29:55.744 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.744 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.745 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.745 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:55.745 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:55.745 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.745 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.745 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.745 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:29:55.745 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.745 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.745 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.745 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:29:55.746 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:29:55.746 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.746 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.746 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.746 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:55.746 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:55.746 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.746 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.746 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.746 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:29:55.746 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.746 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.747 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.747 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:29:55.747 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:29:55.747 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.747 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.747 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.747 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:55.747 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:55.747 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.747 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.748 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.748 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:29:55.748 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.748 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.748 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:29:55.748 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:29:55.748 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.748 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.748 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.748 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.748 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:29:55.749 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:29:55.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.749 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:55.749 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:55.749 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:55.749 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.749 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:55.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:29:55.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.749 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:55.749 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:55.749 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:55.749 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.749 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.750 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:29:55.750 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:29:55.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f376b5a7640 1 -- 172.21.15.115:0/525743966 >> v1:172.21.15.132:6812/1835401922 conn(0x7f3750037bc0 legacy=0x7f375003a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:55.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f376b5a7640 10 monclient: shutdown 2024-01-24T23:29:55.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f376b5a7640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:55.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f376b5a7640 1 -- 172.21.15.115:0/525743966 >> v1:172.21.15.132:6792/0 conn(0x7f376413b2d0 legacy=0x7f37641d14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:55.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f3769b44640 1 -- 172.21.15.115:0/525743966 reap_dead start 2024-01-24T23:29:55.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f3769b44640 1 -- 172.21.15.115:0/525743966 reap_dead start 2024-01-24T23:29:55.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f376b5a7640 1 -- 172.21.15.115:0/525743966 shutdown_connections 2024-01-24T23:29:55.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f376b5a7640 1 -- 172.21.15.115:0/525743966 >> 172.21.15.115:0/525743966 conn(0x7f37640a7820 msgr2=0x7f37641309c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:55.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f376b5a7640 1 -- 172.21.15.115:0/525743966 shutdown_connections 2024-01-24T23:29:55.751 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.715+0000 7f376b5a7640 1 -- 172.21.15.115:0/525743966 wait complete. 2024-01-24T23:29:55.752 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-01-24T23:29:55.752 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon g 2024-01-24T23:29:55.752 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-01-24T23:29:55.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.863+0000 7fa895f43640 10 monclient: get_monmap_and_config 2024-01-24T23:29:55.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.863+0000 7fa895f43640 10 monclient: build_initial_monmap 2024-01-24T23:29:55.866 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.863+0000 7fa895f43640 10 monclient: monmap: 2024-01-24T23:29:55.866 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:55.867 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:55.867 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:55.869524+0000 2024-01-24T23:29:55.867 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:55.869524+0000 2024-01-24T23:29:55.867 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:55.867 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:55.867 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:55.867 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:55.867 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:55.867 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:55.867 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:55.867 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:55.867 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:55.867 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:55.867 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:55.867 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:55.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.863+0000 7fa895f43640 1 Processor -- start 2024-01-24T23:29:55.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.863+0000 7fa895f43640 1 -- start start 2024-01-24T23:29:55.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.863+0000 7fa895f43640 10 monclient: init 2024-01-24T23:29:55.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.863+0000 7fa895f43640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:55.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 10 monclient: _add_conns ranks=[7,0,4,5,6,8,2,3,1] 2024-01-24T23:29:55.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 10 monclient(hunting): picked mon.noname-i con 0x7fa890133d90 addr v1:172.21.15.132:6792/0 2024-01-24T23:29:55.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 10 monclient(hunting): picked mon.noname-a con 0x7fa890134170 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:55.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 10 monclient(hunting): picked mon.noname-c con 0x7fa89013dcf0 addr v1:172.21.15.115:6791/0 2024-01-24T23:29:55.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa89004ed70 con 0x7fa890134170 2024-01-24T23:29:55.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa89009b250 con 0x7fa89013dcf0 2024-01-24T23:29:55.871 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8901345d0 con 0x7fa890133d90 2024-01-24T23:29:55.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:55.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 10 monclient(hunting): authenticate will time out at 1474.792148s 2024-01-24T23:29:55.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88ffff640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7fa89013dcf0 legacy=0x7fa8901400e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:29:55.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88effd640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7fa890134170 legacy=0x7fa8900a2100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:29:55.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88f7fe640 1 --1- >> v1:172.21.15.132:6792/0 conn(0x7fa890133d90 0x7fa8900a19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6792/0 says I am v1:172.21.15.115:36808/0 (socket says 172.21.15.115:36808) 2024-01-24T23:29:55.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88f7fe640 1 -- 172.21.15.115:0/3748585352 learned_addr learned my addr 172.21.15.115:0/3748585352 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:55.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 1 -- 172.21.15.115:0/3748585352 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1030072779 0 0) 0x7fa8901345d0 con 0x7fa890133d90 2024-01-24T23:29:55.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:55.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:55.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:55.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 10 monclient(hunting): my global_id is 154330 2024-01-24T23:29:55.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 1 -- 172.21.15.115:0/3748585352 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa878003530 con 0x7fa890133d90 2024-01-24T23:29:55.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 1 -- 172.21.15.115:0/3748585352 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 793911359 0 0) 0x7fa878003530 con 0x7fa890133d90 2024-01-24T23:29:55.872 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 1 -- 172.21.15.115:0/3748585352 >> v1:172.21.15.115:6791/0 conn(0x7fa89013dcf0 legacy=0x7fa8901400e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:29:55.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 1 -- 172.21.15.115:0/3748585352 >> v1:172.21.15.115:6789/0 conn(0x7fa890134170 legacy=0x7fa8900a2100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:29:55.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 10 monclient: _finish_hunting 0 2024-01-24T23:29:55.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 1 monclient: found mon.noname-i 2024-01-24T23:29:55.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:55.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.132:6792/0 2024-01-24T23:29:55.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 1 -- 172.21.15.115:0/3748585352 --> v1:172.21.15.132:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8900a2810 con 0x7fa890133d90 2024-01-24T23:29:55.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 10 monclient: _finish_auth 0 2024-01-24T23:29:55.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 10 monclient: _check_auth_tickets 2024-01-24T23:29:55.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:55.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 1 -- 172.21.15.115:0/3748585352 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa888001b80 con 0x7fa890133d90 2024-01-24T23:29:55.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:55.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-01-24T23:29:55.873 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 10 monclient: dump: 2024-01-24T23:29:55.873 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:55.873 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:55.874 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.874 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.874 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:55.874 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:55.874 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:55.874 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:55.874 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:55.874 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:55.874 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:55.874 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:55.874 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:55.874 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:55.874 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:55.874 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:55.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 5 monclient: authenticate success, global_id 154330 2024-01-24T23:29:55.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:55.874 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 1 -- 172.21.15.115:0/3748585352 <== mon.7 v1:172.21.15.132:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa888001da0 con 0x7fa890133d90 2024-01-24T23:29:55.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:55.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 1 -- 172.21.15.115:0/3748585352 <== mon.7 v1:172.21.15.132:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa888004f70 con 0x7fa890133d90 2024-01-24T23:29:55.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:55.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 4 set_mon_vals no callback set 2024-01-24T23:29:55.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:55.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:29:55.875 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa88e7fc640 10 monclient: dump: 2024-01-24T23:29:55.875 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:55.875 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:55.875 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.875 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.875 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:55.875 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:55.875 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:55.875 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:55.875 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:55.875 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:55.876 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:55.876 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:55.876 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:55.876 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:55.876 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:55.876 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:55.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 10 monclient: shutdown 2024-01-24T23:29:55.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:55.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 1 -- 172.21.15.115:0/3748585352 >> v1:172.21.15.132:6792/0 conn(0x7fa890133d90 legacy=0x7fa8900a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:55.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 1 -- 172.21.15.115:0/3748585352 shutdown_connections 2024-01-24T23:29:55.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 1 -- 172.21.15.115:0/3748585352 >> 172.21.15.115:0/3748585352 conn(0x7fa8900a65e0 msgr2=0x7fa8900a2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:55.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 1 -- 172.21.15.115:0/3748585352 shutdown_connections 2024-01-24T23:29:55.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 1 -- 172.21.15.115:0/3748585352 wait complete. 2024-01-24T23:29:55.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 10 monclient: build_initial_monmap 2024-01-24T23:29:55.876 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 10 monclient: monmap: 2024-01-24T23:29:55.877 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:55.877 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:55.877 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:55.877 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:55.877 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:55.877 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:55.877 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:55.877 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:55.877 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:55.877 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:55.877 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:55.877 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:55.877 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:55.877 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:55.878 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:55.878 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:55.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 1 Processor -- start 2024-01-24T23:29:55.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 1 -- start start 2024-01-24T23:29:55.878 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 10 monclient: init 2024-01-24T23:29:55.882 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:55.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.867+0000 7fa895f43640 10 monclient: _add_conns ranks=[2,1,4,0,5,7,3,8,6] 2024-01-24T23:29:55.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa895f43640 10 monclient(hunting): picked mon.noname-b con 0x7fa890133d90 addr v1:172.21.15.115:6790/0 2024-01-24T23:29:55.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa895f43640 10 monclient(hunting): picked mon.noname-f con 0x7fa890134170 addr v1:172.21.15.132:6789/0 2024-01-24T23:29:55.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa895f43640 10 monclient(hunting): picked mon.noname-c con 0x7fa89013dcf0 addr v1:172.21.15.115:6791/0 2024-01-24T23:29:55.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa895f43640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa89013ba20 con 0x7fa890134170 2024-01-24T23:29:55.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa895f43640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa89009c610 con 0x7fa890133d90 2024-01-24T23:29:55.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa895f43640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8900676c0 con 0x7fa89013dcf0 2024-01-24T23:29:55.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa895f43640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:55.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa88ffff640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7fa89013dcf0 legacy=0x7fa8901d3cc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:29:55.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa895f43640 10 monclient(hunting): authenticate will time out at 1474.796099s 2024-01-24T23:29:55.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa88f7fe640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7fa890133d90 0x7fa89013cf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:38126/0 (socket says 172.21.15.115:38126) 2024-01-24T23:29:55.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa88f7fe640 1 -- 172.21.15.115:0/3545754821 learned_addr learned my addr 172.21.15.115:0/3545754821 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:55.883 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa88effd640 1 -- 172.21.15.115:0/3545754821 >> v1:172.21.15.132:6789/0 conn(0x7fa890134170 legacy=0x7fa8901d0590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:29:55.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179714950 0 0) 0x7fa89009c610 con 0x7fa890133d90 2024-01-24T23:29:55.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:55.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:55.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:55.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 10 monclient(hunting): my global_id is 154262 2024-01-24T23:29:55.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa87c003530 con 0x7fa890133d90 2024-01-24T23:29:55.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2219373314 0 0) 0x7fa87c003530 con 0x7fa890133d90 2024-01-24T23:29:55.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa89009c610 con 0x7fa890133d90 2024-01-24T23:29:55.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa888004b70 con 0x7fa890133d90 2024-01-24T23:29:55.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:55.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-01-24T23:29:55.884 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 10 monclient(hunting): dump: 2024-01-24T23:29:55.884 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:55.884 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:55.885 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.885 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.885 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:55.885 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:55.885 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:55.885 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:55.885 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:55.885 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:55.885 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:55.885 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:55.885 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:55.885 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:55.885 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:55.885 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:55.885 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:55.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 <== mon.2 v1:172.21.15.115:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1962057550 0 0) 0x7fa89009c610 con 0x7fa890133d90 2024-01-24T23:29:55.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 >> v1:172.21.15.115:6791/0 conn(0x7fa89013dcf0 legacy=0x7fa8901d3cc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:29:55.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 >> v1:172.21.15.132:6789/0 conn(0x7fa890134170 legacy=0x7fa8901d0590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:29:55.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 10 monclient: _finish_hunting 0 2024-01-24T23:29:55.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 1 monclient: found mon.b 2024-01-24T23:29:55.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:55.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:29:55.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 --> v1:172.21.15.115:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89013b740 con 0x7fa890133d90 2024-01-24T23:29:55.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 10 monclient: _finish_auth 0 2024-01-24T23:29:55.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 10 monclient: _check_auth_tickets 2024-01-24T23:29:55.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:55.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa895f43640 5 monclient: authenticate success, global_id 154262 2024-01-24T23:29:55.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa895f43640 10 monclient: _renew_subs 2024-01-24T23:29:55.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa895f43640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:29:55.886 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa895f43640 1 -- 172.21.15.115:0/3545754821 --> v1:172.21.15.115:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa890139fe0 con 0x7fa890133d90 2024-01-24T23:29:55.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa895f43640 10 monclient: _renew_subs 2024-01-24T23:29:55.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa895f43640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.115:6790/0 2024-01-24T23:29:55.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa895f43640 1 -- 172.21.15.115:0/3545754821 --> v1:172.21.15.115:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa89013d830 con 0x7fa890133d90 2024-01-24T23:29:55.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 <== mon.2 v1:172.21.15.115:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa888004f80 con 0x7fa890133d90 2024-01-24T23:29:55.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:55.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 <== mon.2 v1:172.21.15.115:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa888005430 con 0x7fa890133d90 2024-01-24T23:29:55.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:55.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa88dffb640 4 set_mon_vals no callback set 2024-01-24T23:29:55.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-01-24T23:29:55.887 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 10 monclient: dump: 2024-01-24T23:29:55.887 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:55.887 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:55.887 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.887 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.887 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:55.888 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:55.888 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:55.888 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:55.888 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:55.888 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:55.888 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:55.888 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:55.888 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:55.888 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:55.888 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:55.888 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:55.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa895f43640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:55.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa895f43640 1 -- 172.21.15.115:0/3545754821 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa8700065d0 con 0x7fa870002870 2024-01-24T23:29:55.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 <== mon.2 v1:172.21.15.115:6790/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7fa88800feb0 con 0x7fa890133d90 2024-01-24T23:29:55.888 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 <== mon.2 v1:172.21.15.115:6790/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7fa888044940 con 0x7fa890133d90 2024-01-24T23:29:55.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa88ffff640 10 monclient: get_auth_request con 0x7fa87c036f90 auth_method 0 2024-01-24T23:29:55.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2104830786 0 0) 0x7fa8700065d0 con 0x7fa870002870 2024-01-24T23:29:55.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:55.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:55.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:55.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 10 monclient(hunting): my global_id is 7248 2024-01-24T23:29:55.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa89009c610 con 0x7fa870002870 2024-01-24T23:29:55.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 178617688 0 0) 0x7fa89009c610 con 0x7fa870002870 2024-01-24T23:29:55.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa8700065d0 con 0x7fa870002870 2024-01-24T23:29:55.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa884004110 con 0x7fa870002870 2024-01-24T23:29:55.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:55.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:29:55.889 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 10 monclient: dump: 2024-01-24T23:29:55.889 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:55.889 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:55.890 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.890 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.890 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:55.890 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:55.890 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:55.890 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:55.890 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:55.890 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:55.890 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:55.890 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:55.890 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:55.890 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:55.890 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:55.890 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:55.890 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4228616538 0 0) 0x7fa8700065d0 con 0x7fa870002870 2024-01-24T23:29:55.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.871+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 --> v1:172.21.15.132:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa87000a120 con 0x7fa870002870 2024-01-24T23:29:55.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.875+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fa8840064e0 con 0x7fa870002870 2024-01-24T23:29:55.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.875+0000 7fa877fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:55.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.875+0000 7fa877fff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:55.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.875+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 >> v1:172.21.15.132:6790/0 conn(0x7fa870002870 legacy=0x7fa870004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:55.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.875+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 >> v1:172.21.15.132:6790/0 conn(0x7fa870002870 legacy=0x7fa870004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:55.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.879+0000 7fa895f43640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:29:55.891 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.879+0000 7fa895f43640 1 -- 172.21.15.115:0/3545754821 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa870009e10 con 0x7fa8700090d0 2024-01-24T23:29:55.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.879+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1723138921 0 0) 0x7fa870009e10 con 0x7fa8700090d0 2024-01-24T23:29:55.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.879+0000 7fa877fff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:55.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.879+0000 7fa877fff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:55.892 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.879+0000 7fa877fff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:55.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.879+0000 7fa877fff640 10 monclient(hunting): my global_id is 7257 2024-01-24T23:29:55.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.879+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8700065d0 con 0x7fa8700090d0 2024-01-24T23:29:55.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.879+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3291844340 0 0) 0x7fa8700065d0 con 0x7fa8700090d0 2024-01-24T23:29:55.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.879+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa870009e10 con 0x7fa8700090d0 2024-01-24T23:29:55.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.879+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7fa884004ca0 con 0x7fa8700090d0 2024-01-24T23:29:55.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.879+0000 7fa877fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:55.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.879+0000 7fa877fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-01-24T23:29:55.893 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.879+0000 7fa877fff640 10 monclient: dump: 2024-01-24T23:29:55.893 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:55.893 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:55.893 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.893 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:55.893 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:55.893 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:55.893 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:55.894 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:55.894 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:55.894 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:55.894 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:55.894 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:55.894 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:55.894 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:55.894 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:55.894 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:55.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.879+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 <== mon.3 v1:172.21.15.132:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1467687269 0 0) 0x7fa870009e10 con 0x7fa8700090d0 2024-01-24T23:29:55.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.879+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 --> v1:172.21.15.132:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa87000c310 con 0x7fa8700090d0 2024-01-24T23:29:55.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.883+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 <== mon.3 v1:172.21.15.132:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7680 (unknown 0 0 183600331) 0x7fa884006260 con 0x7fa8700090d0 2024-01-24T23:29:55.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.883+0000 7fa877fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:29:55.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.883+0000 7fa877fff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:55.894 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.883+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 >> v1:172.21.15.132:6790/0 conn(0x7fa8700090d0 legacy=0x7fa870009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:55.895 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.883+0000 7fa877fff640 1 -- 172.21.15.115:0/3545754821 >> v1:172.21.15.132:6790/0 conn(0x7fa8700090d0 legacy=0x7fa870009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:55.895 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:29:55.895 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:55.895 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:55.895 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:29:55.895 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 52, 2024-01-24T23:29:55.895 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:29:55.895 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:29:55.895 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:29:55.895 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:29:55.895 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:29:55.896 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:29:55.896 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:29:55.896 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:55.896 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 3, 2024-01-24T23:29:55.896 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:55.896 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:29:55.896 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:29:55.896 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:55.896 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:55.896 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:55.896 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:55.896 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:55.896 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:55.897 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:55.897 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:55.897 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:55.897 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:55.897 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:55.897 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:55.897 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:29:55.897 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:29:55.897 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:55.897 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:55.897 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:55.897 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:55.897 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:55.897 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:55.898 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:55.898 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:55.898 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:55.898 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:55.898 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:55.898 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.898 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:29:55.898 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:29:55.898 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:29:55.898 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:29:55.898 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:29:55.898 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:29:55.898 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:55.899 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:55.899 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:29:55.899 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:29:55.899 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:29:55.899 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:29:55.899 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:29:55.899 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:29:55.899 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:29:55.899 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:55.899 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:29:55.899 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:55.899 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:55.899 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:55.899 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:55.900 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:55.900 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:55.900 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:55.900 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:55.900 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:55.900 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:55.900 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:55.900 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:55.900 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:29:55.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.900 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:29:55.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.900 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:55.900 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:55.901 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.901 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.901 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.901 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:29:55.901 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.901 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.901 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.901 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:29:55.901 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:29:55.901 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.901 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.901 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.902 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:55.902 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:55.902 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.902 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.902 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.902 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:29:55.902 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.902 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.902 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.902 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:29:55.903 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:29:55.903 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.903 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.903 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.903 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:55.903 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:55.903 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.903 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.903 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.903 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:29:55.903 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.903 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.903 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.904 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:29:55.904 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:29:55.904 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.904 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.904 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.904 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:55.904 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:55.904 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.904 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.904 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.905 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:29:55.905 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.905 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.905 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.905 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:29:55.905 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:29:55.905 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.905 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.905 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.905 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:55.905 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:55.906 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.906 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.906 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.906 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:29:55.906 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.906 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.906 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.906 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:29:55.906 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:29:55.906 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.906 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.906 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.907 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:55.907 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:55.907 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.907 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.907 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.907 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:29:55.907 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.907 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.907 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.907 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:29:55.907 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:29:55.908 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.908 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.908 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.908 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:55.908 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:55.908 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.908 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.908 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.909 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:29:55.909 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.909 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.909 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.909 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:29:55.909 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:29:55.909 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.909 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.909 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.909 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:55.909 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:55.909 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.910 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.910 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.910 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:29:55.910 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.910 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.910 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.910 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:29:55.910 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:29:55.910 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.910 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.910 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.911 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:55.911 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:55.911 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:55.911 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:55.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.911 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:55.911 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:29:55.911 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:55.911 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.911 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.911 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:29:55.911 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:29:55.912 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:55.912 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:55.912 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:55.912 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.912 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.912 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:29:55.912 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:29:55.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.912 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:55.912 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:55.912 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:55.912 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.913 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:55.913 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": [ 2024-01-24T23:29:55.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.913 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:55.913 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:55.913 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:29:55.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.913 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:55.913 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:29:55.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:55.913 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:55.913 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:55.913 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 4 2024-01-24T23:29:55.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:55.914 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:55.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:55.914 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:29:55.914 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:29:55.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.883+0000 7fa895f43640 1 -- 172.21.15.115:0/3545754821 >> v1:172.21.15.132:6812/1835401922 conn(0x7fa87c036f90 legacy=0x7fa87c039450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:55.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.883+0000 7fa895f43640 10 monclient: shutdown 2024-01-24T23:29:55.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.883+0000 7fa895f43640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:55.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.883+0000 7fa895f43640 1 -- 172.21.15.115:0/3545754821 >> v1:172.21.15.115:6790/0 conn(0x7fa890133d90 legacy=0x7fa89013cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:55.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.883+0000 7fa88ffff640 1 -- 172.21.15.115:0/3545754821 reap_dead start 2024-01-24T23:29:55.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.883+0000 7fa88ffff640 1 -- 172.21.15.115:0/3545754821 reap_dead start 2024-01-24T23:29:55.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.883+0000 7fa895f43640 1 -- 172.21.15.115:0/3545754821 shutdown_connections 2024-01-24T23:29:55.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.883+0000 7fa895f43640 1 -- 172.21.15.115:0/3545754821 >> 172.21.15.115:0/3545754821 conn(0x7fa8900a65e0 msgr2=0x7fa8900a3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:55.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.883+0000 7fa895f43640 1 -- 172.21.15.115:0/3545754821 shutdown_connections 2024-01-24T23:29:55.916 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:55.883+0000 7fa895f43640 1 -- 172.21.15.115:0/3545754821 wait complete. 2024-01-24T23:29:55.916 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-01-24T23:29:55.916 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon h 2024-01-24T23:29:55.917 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-01-24T23:29:56.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.027+0000 7f8b9db4f640 10 monclient: get_monmap_and_config 2024-01-24T23:29:56.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.027+0000 7f8b9db4f640 10 monclient: build_initial_monmap 2024-01-24T23:29:56.032 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.027+0000 7f8b9db4f640 10 monclient: monmap: 2024-01-24T23:29:56.032 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:56.032 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:56.032 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:56.034744+0000 2024-01-24T23:29:56.032 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:56.034744+0000 2024-01-24T23:29:56.032 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:56.033 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:56.033 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:56.033 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:56.033 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:56.033 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:56.033 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:56.033 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:56.033 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:56.033 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:56.033 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:56.033 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:56.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 1 Processor -- start 2024-01-24T23:29:56.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 1 -- start start 2024-01-24T23:29:56.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 10 monclient: init 2024-01-24T23:29:56.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:56.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 10 monclient: _add_conns ranks=[8,3,0,4,5,6,7,1,2] 2024-01-24T23:29:56.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 10 monclient(hunting): picked mon.noname-e con 0x7f8b98133bf0 addr v1:172.21.15.115:6793/0 2024-01-24T23:29:56.041 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 10 monclient(hunting): picked mon.noname-g con 0x7f8b98133fd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:29:56.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 10 monclient(hunting): picked mon.noname-a con 0x7f8b981343b0 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:56.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b9804ed70 con 0x7f8b981343b0 2024-01-24T23:29:56.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b980a4b60 con 0x7f8b98133fd0 2024-01-24T23:29:56.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b9809ae50 con 0x7f8b98133bf0 2024-01-24T23:29:56.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:56.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 10 monclient(hunting): authenticate will time out at 1474.957360s 2024-01-24T23:29:56.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b97fff640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f8b981343b0 legacy=0x7f8b9813f490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:29:56.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b977fe640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f8b98133bf0 0x7f8b9809bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:58392/0 (socket says 172.21.15.115:58392) 2024-01-24T23:29:56.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b977fe640 1 -- 172.21.15.115:0/1427137088 learned_addr learned my addr 172.21.15.115:0/1427137088 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:56.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 1 -- 172.21.15.115:0/1427137088 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1905510982 0 0) 0x7f8b9809ae50 con 0x7f8b98133bf0 2024-01-24T23:29:56.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 10 monclient(hunting): _negotiate 2024-01-24T23:29:56.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:56.042 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:56.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 10 monclient(hunting): my global_id is 86507 2024-01-24T23:29:56.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 1 -- 172.21.15.115:0/1427137088 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b80003530 con 0x7f8b98133bf0 2024-01-24T23:29:56.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 1 -- 172.21.15.115:0/1427137088 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2154774096 0 0) 0x7f8b80003530 con 0x7f8b98133bf0 2024-01-24T23:29:56.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 1 -- 172.21.15.115:0/1427137088 >> v1:172.21.15.132:6790/0 conn(0x7f8b98133fd0 legacy=0x7f8b9813bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:56.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 1 -- 172.21.15.115:0/1427137088 >> v1:172.21.15.115:6789/0 conn(0x7f8b981343b0 legacy=0x7f8b9813f490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:29:56.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 10 monclient: _finish_hunting 0 2024-01-24T23:29:56.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 1 monclient: found mon.noname-e 2024-01-24T23:29:56.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:56.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.115:6793/0 2024-01-24T23:29:56.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 1 -- 172.21.15.115:0/1427137088 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b9809b130 con 0x7f8b98133bf0 2024-01-24T23:29:56.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 10 monclient: _finish_auth 0 2024-01-24T23:29:56.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 10 monclient: _check_auth_tickets 2024-01-24T23:29:56.043 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:56.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 1 -- 172.21.15.115:0/1427137088 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8b8c0017c0 con 0x7f8b98133bf0 2024-01-24T23:29:56.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 5 monclient: authenticate success, global_id 86507 2024-01-24T23:29:56.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:56.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:56.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:29:56.044 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 10 monclient: dump: 2024-01-24T23:29:56.044 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:56.044 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:56.044 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.044 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.044 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:56.044 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:56.044 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:56.045 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:56.045 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:56.045 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:56.045 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:56.045 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:56.045 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:56.045 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:56.045 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:56.045 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:56.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:56.045 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 1 -- 172.21.15.115:0/1427137088 <== mon.8 v1:172.21.15.115:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8b8c001ae0 con 0x7f8b98133bf0 2024-01-24T23:29:56.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:56.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:56.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 1 -- 172.21.15.115:0/1427137088 <== mon.8 v1:172.21.15.115:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8b8c004bf0 con 0x7f8b98133bf0 2024-01-24T23:29:56.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 4 set_mon_vals no callback set 2024-01-24T23:29:56.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:56.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:29:56.046 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b967fc640 10 monclient: dump: 2024-01-24T23:29:56.046 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:56.046 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:56.046 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.046 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.046 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:56.047 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:56.047 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:56.047 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:56.047 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:56.047 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:56.047 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:56.047 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:56.047 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:56.047 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:56.047 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:56.047 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:56.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 10 monclient: shutdown 2024-01-24T23:29:56.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:56.047 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 1 -- 172.21.15.115:0/1427137088 >> v1:172.21.15.115:6793/0 conn(0x7f8b98133bf0 legacy=0x7f8b9809bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:56.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 1 -- 172.21.15.115:0/1427137088 shutdown_connections 2024-01-24T23:29:56.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 1 -- 172.21.15.115:0/1427137088 >> 172.21.15.115:0/1427137088 conn(0x7f8b980662f0 msgr2=0x7f8b98066700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:56.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 1 -- 172.21.15.115:0/1427137088 shutdown_connections 2024-01-24T23:29:56.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 1 -- 172.21.15.115:0/1427137088 wait complete. 2024-01-24T23:29:56.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 10 monclient: build_initial_monmap 2024-01-24T23:29:56.048 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 10 monclient: monmap: 2024-01-24T23:29:56.048 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:56.048 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:56.048 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:56.048 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:56.048 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:56.048 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:56.048 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:56.049 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:56.049 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:56.049 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:56.049 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:56.049 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:56.049 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:56.049 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:56.049 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:56.049 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:56.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 1 Processor -- start 2024-01-24T23:29:56.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 1 -- start start 2024-01-24T23:29:56.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.031+0000 7f8b9db4f640 10 monclient: init 2024-01-24T23:29:56.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b9db4f640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:56.049 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b9db4f640 10 monclient: _add_conns ranks=[8,5,4,0,2,6,3,1,7] 2024-01-24T23:29:56.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b9db4f640 10 monclient(hunting): picked mon.noname-e con 0x7f8b98133bf0 addr v1:172.21.15.115:6793/0 2024-01-24T23:29:56.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b9db4f640 10 monclient(hunting): picked mon.noname-h con 0x7f8b98133fd0 addr v1:172.21.15.132:6791/0 2024-01-24T23:29:56.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b9db4f640 10 monclient(hunting): picked mon.noname-c con 0x7f8b981343b0 addr v1:172.21.15.115:6791/0 2024-01-24T23:29:56.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b9db4f640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b9809c880 con 0x7f8b981343b0 2024-01-24T23:29:56.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b9db4f640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b980a5990 con 0x7f8b98133fd0 2024-01-24T23:29:56.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b9db4f640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b980a2690 con 0x7f8b98133bf0 2024-01-24T23:29:56.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b9db4f640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:56.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b9db4f640 10 monclient(hunting): authenticate will time out at 1474.960375s 2024-01-24T23:29:56.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b97fff640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f8b981343b0 legacy=0x7f8b981d3be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:29:56.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b977fe640 1 --1- >> v1:172.21.15.115:6793/0 conn(0x7f8b98133bf0 0x7f8b980a0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6793/0 says I am v1:172.21.15.115:58406/0 (socket says 172.21.15.115:58406) 2024-01-24T23:29:56.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b977fe640 1 -- 172.21.15.115:0/1467828362 learned_addr learned my addr 172.21.15.115:0/1467828362 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:56.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 <== mon.8 v1:172.21.15.115:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 340014155 0 0) 0x7f8b980a2690 con 0x7f8b98133bf0 2024-01-24T23:29:56.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:56.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:56.050 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:56.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient(hunting): my global_id is 86516 2024-01-24T23:29:56.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 --> v1:172.21.15.115:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b84003530 con 0x7f8b98133bf0 2024-01-24T23:29:56.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 <== mon.8 v1:172.21.15.115:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2968390824 0 0) 0x7f8b84003530 con 0x7f8b98133bf0 2024-01-24T23:29:56.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 --> v1:172.21.15.115:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8b980a2690 con 0x7f8b98133bf0 2024-01-24T23:29:56.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 <== mon.8 v1:172.21.15.115:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8b8c004bf0 con 0x7f8b98133bf0 2024-01-24T23:29:56.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:56.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-01-24T23:29:56.051 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient(hunting): dump: 2024-01-24T23:29:56.051 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:56.052 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:56.052 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.052 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.052 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:56.052 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:56.052 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:56.052 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:56.052 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:56.052 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:56.052 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:56.052 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:56.052 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:56.052 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:56.053 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:56.053 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:56.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:56.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426473266 0 0) 0x7f8b980a5990 con 0x7f8b98133fd0 2024-01-24T23:29:56.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:56.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:56.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:56.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient(hunting): my global_id is 7214 2024-01-24T23:29:56.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b84003530 con 0x7f8b98133fd0 2024-01-24T23:29:56.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 <== mon.8 v1:172.21.15.115:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3217861222 0 0) 0x7f8b980a2690 con 0x7f8b98133bf0 2024-01-24T23:29:56.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 >> v1:172.21.15.132:6791/0 conn(0x7f8b98133fd0 legacy=0x7f8b980a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:56.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 >> v1:172.21.15.115:6791/0 conn(0x7f8b981343b0 legacy=0x7f8b981d3be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:29:56.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient: _finish_hunting 0 2024-01-24T23:29:56.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 1 monclient: found mon.e 2024-01-24T23:29:56.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:56.053 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:29:56.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 --> v1:172.21.15.115:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b98120a80 con 0x7f8b98133bf0 2024-01-24T23:29:56.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient: _finish_auth 0 2024-01-24T23:29:56.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient: _check_auth_tickets 2024-01-24T23:29:56.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:56.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b9db4f640 5 monclient: authenticate success, global_id 86516 2024-01-24T23:29:56.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b9db4f640 10 monclient: _renew_subs 2024-01-24T23:29:56.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b9db4f640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:29:56.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b9db4f640 1 -- 172.21.15.115:0/1467828362 --> v1:172.21.15.115:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b9811f760 con 0x7f8b98133bf0 2024-01-24T23:29:56.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b9db4f640 10 monclient: _renew_subs 2024-01-24T23:29:56.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b9db4f640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.115:6793/0 2024-01-24T23:29:56.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b9db4f640 1 -- 172.21.15.115:0/1467828362 --> v1:172.21.15.115:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b980a1b70 con 0x7f8b98133bf0 2024-01-24T23:29:56.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 <== mon.8 v1:172.21.15.115:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8b8c004ec0 con 0x7f8b98133bf0 2024-01-24T23:29:56.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:56.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 <== mon.8 v1:172.21.15.115:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8b8c005370 con 0x7f8b98133bf0 2024-01-24T23:29:56.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:56.054 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b95ffb640 4 set_mon_vals no callback set 2024-01-24T23:29:56.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-01-24T23:29:56.055 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient: dump: 2024-01-24T23:29:56.055 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:56.055 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:56.055 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.055 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.055 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:56.055 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:56.055 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:56.056 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:56.056 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:56.056 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:56.056 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:56.056 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:56.056 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:56.056 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:56.056 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:56.056 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:56.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b9db4f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:56.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b9db4f640 1 -- 172.21.15.115:0/1467828362 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8b780065d0 con 0x7f8b78002870 2024-01-24T23:29:56.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 <== mon.8 v1:172.21.15.115:6793/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f8b8c00fdf0 con 0x7f8b98133bf0 2024-01-24T23:29:56.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 <== mon.8 v1:172.21.15.115:6793/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7f8b8c0446f0 con 0x7f8b98133bf0 2024-01-24T23:29:56.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2984805200 0 0) 0x7f8b780065d0 con 0x7f8b78002870 2024-01-24T23:29:56.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:56.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:56.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:56.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient(hunting): my global_id is 7223 2024-01-24T23:29:56.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b980a2690 con 0x7f8b78002870 2024-01-24T23:29:56.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b97fff640 10 monclient: get_auth_request con 0x7f8b84036cc0 auth_method 0 2024-01-24T23:29:56.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3111828443 0 0) 0x7f8b980a2690 con 0x7f8b78002870 2024-01-24T23:29:56.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8b780065d0 con 0x7f8b78002870 2024-01-24T23:29:56.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8b88001940 con 0x7f8b78002870 2024-01-24T23:29:56.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:56.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:29:56.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.035+0000 7f8b7ffff640 10 monclient: dump: 2024-01-24T23:29:56.058 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:56.058 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:56.058 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.058 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.058 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:56.058 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:56.058 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:56.058 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:56.058 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:56.058 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:56.058 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:56.058 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:56.058 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:56.058 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:56.058 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:56.058 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:56.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.039+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4229040635 0 0) 0x7f8b780065d0 con 0x7f8b78002870 2024-01-24T23:29:56.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.039+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 --> v1:172.21.15.132:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8b7800a120 con 0x7f8b78002870 2024-01-24T23:29:56.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.039+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f8b88003660 con 0x7f8b78002870 2024-01-24T23:29:56.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.039+0000 7f8b7ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:56.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.039+0000 7f8b7ffff640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:56.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.039+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 >> v1:172.21.15.132:6791/0 conn(0x7f8b78002870 legacy=0x7f8b78004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:56.059 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.039+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 >> v1:172.21.15.132:6791/0 conn(0x7f8b78002870 legacy=0x7f8b78004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:56.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.043+0000 7f8b9db4f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:29:56.060 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.043+0000 7f8b9db4f640 1 -- 172.21.15.115:0/1467828362 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8b7800c470 con 0x7f8b78009070 2024-01-24T23:29:56.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2045329066 0 0) 0x7f8b7800c470 con 0x7f8b78009070 2024-01-24T23:29:56.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b7ffff640 10 monclient(hunting): _negotiate 2024-01-24T23:29:56.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b7ffff640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:56.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b7ffff640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:56.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b7ffff640 10 monclient(hunting): my global_id is 7232 2024-01-24T23:29:56.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b780065d0 con 0x7f8b78009070 2024-01-24T23:29:56.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2759796019 0 0) 0x7f8b780065d0 con 0x7f8b78009070 2024-01-24T23:29:56.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8b7800c470 con 0x7f8b78009070 2024-01-24T23:29:56.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f8b88003e10 con 0x7f8b78009070 2024-01-24T23:29:56.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b7ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:56.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b7ffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:29:56.061 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b7ffff640 10 monclient: dump: 2024-01-24T23:29:56.062 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:56.062 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:56.062 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.062 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.062 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:56.062 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:56.062 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:56.062 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:56.063 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:56.063 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:56.063 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:56.063 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:56.063 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:56.063 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:56.063 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:56.063 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:56.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2475002468 0 0) 0x7f8b7800c470 con 0x7f8b78009070 2024-01-24T23:29:56.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 --> v1:172.21.15.132:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8b78009c10 con 0x7f8b78009070 2024-01-24T23:29:56.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 <== mon.5 v1:172.21.15.132:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7682 (unknown 0 0 1474657710) 0x7f8b88006190 con 0x7f8b78009070 2024-01-24T23:29:56.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b7ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:29:56.063 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b7ffff640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:56.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 >> v1:172.21.15.132:6791/0 conn(0x7f8b78009070 legacy=0x7f8b78009500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:56.064 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b7ffff640 1 -- 172.21.15.115:0/1467828362 >> v1:172.21.15.132:6791/0 conn(0x7f8b78009070 legacy=0x7f8b78009500 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:56.064 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:29:56.064 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:56.064 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:56.064 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "leader", 2024-01-24T23:29:56.064 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 52, 2024-01-24T23:29:56.064 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:29:56.064 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:29:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:29:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:29:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:29:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:29:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:29:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 4, 2024-01-24T23:29:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:29:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:29:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:29:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:29:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:29:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:29:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:29:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:29:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:29:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:29:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:29:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:29:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:29:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:29:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:29:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:29:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:29:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:29:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:29:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:29:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:29:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:29:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:29:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:56.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.072 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:56.072 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:29:56.072 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:56.072 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.072 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:56.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.072 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:29:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:29:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:29:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:29:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:29:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:29:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:29:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:29:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:29:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:29:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:29:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:29:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:29:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:29:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:29:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:29:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:29:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:56.080 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:56.080 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:56.080 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:56.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.080 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:56.080 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:29:56.080 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:56.080 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.080 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:56.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.080 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:29:56.080 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:29:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:29:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:29:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:29:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:29:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:29:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:56.083 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.083 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:56.083 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:29:56.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.083 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:56.083 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:56.083 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 2 2024-01-24T23:29:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": [ 2024-01-24T23:29:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:29:56.085 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:29:56.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b9db4f640 1 -- 172.21.15.115:0/1467828362 >> v1:172.21.15.132:6812/1835401922 conn(0x7f8b84036cc0 legacy=0x7f8b84039180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:56.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b9db4f640 10 monclient: shutdown 2024-01-24T23:29:56.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b9db4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:56.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b9db4f640 1 -- 172.21.15.115:0/1467828362 >> v1:172.21.15.115:6793/0 conn(0x7f8b98133bf0 legacy=0x7f8b980a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:56.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b97fff640 1 -- 172.21.15.115:0/1467828362 reap_dead start 2024-01-24T23:29:56.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b9db4f640 1 -- 172.21.15.115:0/1467828362 shutdown_connections 2024-01-24T23:29:56.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b9db4f640 1 -- 172.21.15.115:0/1467828362 >> 172.21.15.115:0/1467828362 conn(0x7f8b980662f0 msgr2=0x7f8b9809d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:56.086 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b9db4f640 1 -- 172.21.15.115:0/1467828362 shutdown_connections 2024-01-24T23:29:56.087 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.051+0000 7f8b9db4f640 1 -- 172.21.15.115:0/1467828362 wait complete. 2024-01-24T23:29:56.087 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-01-24T23:29:56.087 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon i 2024-01-24T23:29:56.087 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-01-24T23:29:56.197 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e7a58a640 10 monclient: get_monmap_and_config 2024-01-24T23:29:56.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e7a58a640 10 monclient: build_initial_monmap 2024-01-24T23:29:56.198 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e7a58a640 10 monclient: monmap: 2024-01-24T23:29:56.198 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:56.198 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:56.198 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:29:56.200459+0000 2024-01-24T23:29:56.198 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:29:56.200459+0000 2024-01-24T23:29:56.198 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:56.198 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:56.198 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:56.198 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:56.198 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:56.199 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:56.199 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:56.199 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:56.199 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:56.199 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:56.199 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:56.199 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:56.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e7a58a640 1 Processor -- start 2024-01-24T23:29:56.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e7a58a640 1 -- start start 2024-01-24T23:29:56.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e7a58a640 10 monclient: init 2024-01-24T23:29:56.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e7a58a640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:56.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e7a58a640 10 monclient: _add_conns ranks=[0,4,5,3,1,6,7,2,8] 2024-01-24T23:29:56.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e7a58a640 10 monclient(hunting): picked mon.noname-a con 0x7f1e7409b9f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:29:56.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e7a58a640 10 monclient(hunting): picked mon.noname-c con 0x7f1e740720b0 addr v1:172.21.15.115:6791/0 2024-01-24T23:29:56.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e7a58a640 10 monclient(hunting): picked mon.noname-h con 0x7f1e74139700 addr v1:172.21.15.132:6791/0 2024-01-24T23:29:56.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e7a58a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7404ed70 con 0x7f1e7409b9f0 2024-01-24T23:29:56.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e7a58a640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e740a0eb0 con 0x7f1e740720b0 2024-01-24T23:29:56.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e7a58a640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7409bdd0 con 0x7f1e74139700 2024-01-24T23:29:56.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e7a58a640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:56.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e737fe640 1 -- >> v1:172.21.15.115:6791/0 conn(0x7f1e740720b0 legacy=0x7f1e740a0000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6791/0 2024-01-24T23:29:56.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e73fff640 1 -- >> v1:172.21.15.115:6789/0 conn(0x7f1e7409b9f0 legacy=0x7f1e7409f8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6789/0 2024-01-24T23:29:56.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e7a58a640 10 monclient(hunting): authenticate will time out at 1475.123092s 2024-01-24T23:29:56.210 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e78b27640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f1e74139700 0x7f1e7413baf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.115:47782/0 (socket says 172.21.15.115:47782) 2024-01-24T23:29:56.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e78b27640 1 -- 172.21.15.115:0/2872884055 learned_addr learned my addr 172.21.15.115:0/2872884055 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:56.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e72ffd640 1 -- 172.21.15.115:0/2872884055 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3056938683 0 0) 0x7f1e7409bdd0 con 0x7f1e74139700 2024-01-24T23:29:56.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e72ffd640 10 monclient(hunting): _negotiate 2024-01-24T23:29:56.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e72ffd640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:56.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e72ffd640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:56.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e72ffd640 10 monclient(hunting): my global_id is 7241 2024-01-24T23:29:56.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.195+0000 7f1e72ffd640 1 -- 172.21.15.115:0/2872884055 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e5c003530 con 0x7f1e74139700 2024-01-24T23:29:56.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e72ffd640 1 -- 172.21.15.115:0/2872884055 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2337711080 0 0) 0x7f1e5c003530 con 0x7f1e74139700 2024-01-24T23:29:56.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e72ffd640 1 -- 172.21.15.115:0/2872884055 >> v1:172.21.15.115:6791/0 conn(0x7f1e740720b0 legacy=0x7f1e740a0000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:29:56.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e72ffd640 1 -- 172.21.15.115:0/2872884055 >> v1:172.21.15.115:6789/0 conn(0x7f1e7409b9f0 legacy=0x7f1e7409f8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:29:56.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e72ffd640 10 monclient: _finish_hunting 0 2024-01-24T23:29:56.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e72ffd640 1 monclient: found mon.noname-h 2024-01-24T23:29:56.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e72ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:56.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e72ffd640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.132:6791/0 2024-01-24T23:29:56.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e72ffd640 1 -- 172.21.15.115:0/2872884055 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e740a0710 con 0x7f1e74139700 2024-01-24T23:29:56.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e72ffd640 10 monclient: _finish_auth 0 2024-01-24T23:29:56.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e72ffd640 10 monclient: _check_auth_tickets 2024-01-24T23:29:56.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e72ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:56.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e72ffd640 1 -- 172.21.15.115:0/2872884055 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1e6c001b40 con 0x7f1e74139700 2024-01-24T23:29:56.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 5 monclient: authenticate success, global_id 7241 2024-01-24T23:29:56.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:56.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e72ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:56.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e72ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:29:56.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e72ffd640 10 monclient: dump: 2024-01-24T23:29:56.212 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:56.212 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:56.212 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.213 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.213 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:56.213 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:56.213 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:56.213 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:56.213 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:56.213 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:56.214 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:56.214 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:56.214 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:56.214 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:56.214 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:56.214 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:56.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-01-24T23:29:56.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e72ffd640 1 -- 172.21.15.115:0/2872884055 <== mon.5 v1:172.21.15.132:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1e6c001d60 con 0x7f1e74139700 2024-01-24T23:29:56.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e72ffd640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:56.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 10 monclient: get_monmap_and_config success 2024-01-24T23:29:56.214 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 4 set_mon_vals no callback set 2024-01-24T23:29:56.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e72ffd640 1 -- 172.21.15.115:0/2872884055 <== mon.5 v1:172.21.15.132:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1e6c004f70 con 0x7f1e74139700 2024-01-24T23:29:56.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e72ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:56.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e72ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:29:56.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e72ffd640 10 monclient: dump: 2024-01-24T23:29:56.215 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:56.215 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:56.215 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.215 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.215 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:56.215 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:56.215 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:56.215 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:56.215 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:56.215 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:56.216 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 10 monclient: shutdown 2024-01-24T23:29:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 1 -- 172.21.15.115:0/2872884055 >> v1:172.21.15.132:6791/0 conn(0x7f1e74139700 legacy=0x7f1e7413baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 1 -- 172.21.15.115:0/2872884055 shutdown_connections 2024-01-24T23:29:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 1 -- 172.21.15.115:0/2872884055 >> 172.21.15.115:0/2872884055 conn(0x7f1e740a5730 msgr2=0x7f1e740a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 1 -- 172.21.15.115:0/2872884055 shutdown_connections 2024-01-24T23:29:56.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 1 -- 172.21.15.115:0/2872884055 wait complete. 2024-01-24T23:29:56.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 10 monclient: build_initial_monmap 2024-01-24T23:29:56.217 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 10 monclient: monmap: 2024-01-24T23:29:56.217 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:29:56.217 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-01-24T23:29:56.217 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 0.000000 2024-01-24T23:29:56.217 INFO:teuthology.orchestra.run.smithi115.stderr:created 0.000000 2024-01-24T23:29:56.217 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:29:56.217 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 1 2024-01-24T23:29:56.217 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:29:56.217 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:29:56.217 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:29:56.217 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:29:56.217 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:29:56.217 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:29:56.217 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:29:56.218 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:29:56.218 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:29:56.218 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:56.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 1 Processor -- start 2024-01-24T23:29:56.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 1 -- start start 2024-01-24T23:29:56.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 10 monclient: init 2024-01-24T23:29:56.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 10 monclient: _reopen_session rank -1 2024-01-24T23:29:56.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 10 monclient: _add_conns ranks=[5,1,3,6,4,7,8,0,2] 2024-01-24T23:29:56.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 10 monclient(hunting): picked mon.noname-h con 0x7f1e740720b0 addr v1:172.21.15.132:6791/0 2024-01-24T23:29:56.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 10 monclient(hunting): picked mon.noname-f con 0x7f1e7409b9f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:29:56.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 10 monclient(hunting): picked mon.noname-g con 0x7f1e74139700 addr v1:172.21.15.132:6790/0 2024-01-24T23:29:56.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7411d4b0 con 0x7f1e7409b9f0 2024-01-24T23:29:56.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7409c9f0 con 0x7f1e74139700 2024-01-24T23:29:56.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7409c480 con 0x7f1e740720b0 2024-01-24T23:29:56.218 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 10 monclient(hunting): _renew_subs 2024-01-24T23:29:56.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e7a58a640 10 monclient(hunting): authenticate will time out at 1475.126944s 2024-01-24T23:29:56.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e737fe640 1 -- >> v1:172.21.15.132:6789/0 conn(0x7f1e7409b9f0 legacy=0x7f1e741cbff0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.132:6789/0 2024-01-24T23:29:56.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e73fff640 1 --1- >> v1:172.21.15.132:6791/0 conn(0x7f1e740720b0 0x7f1e741326c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.132:6791/0 says I am v1:172.21.15.115:47784/0 (socket says 172.21.15.115:47784) 2024-01-24T23:29:56.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e73fff640 1 -- 172.21.15.115:0/1541917546 learned_addr learned my addr 172.21.15.115:0/1541917546 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:29:56.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 <== mon.5 v1:172.21.15.132:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2534749728 0 0) 0x7f1e7409c480 con 0x7f1e740720b0 2024-01-24T23:29:56.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e70ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:29:56.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e70ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:56.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e70ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:56.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e70ff9640 10 monclient(hunting): my global_id is 7250 2024-01-24T23:29:56.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 --> v1:172.21.15.132:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e60003530 con 0x7f1e740720b0 2024-01-24T23:29:56.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 <== mon.3 v1:172.21.15.132:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2373005172 0 0) 0x7f1e7409c9f0 con 0x7f1e74139700 2024-01-24T23:29:56.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e70ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:29:56.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e70ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:56.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e70ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:56.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e70ff9640 10 monclient(hunting): my global_id is 7275 2024-01-24T23:29:56.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.199+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 --> v1:172.21.15.132:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e7409c480 con 0x7f1e74139700 2024-01-24T23:29:56.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 <== mon.5 v1:172.21.15.132:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 28079560 0 0) 0x7f1e60003530 con 0x7f1e740720b0 2024-01-24T23:29:56.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 --> v1:172.21.15.132:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1e7409c9f0 con 0x7f1e740720b0 2024-01-24T23:29:56.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 <== mon.5 v1:172.21.15.132:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1e680042f0 con 0x7f1e740720b0 2024-01-24T23:29:56.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:56.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-01-24T23:29:56.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 10 monclient(hunting): dump: 2024-01-24T23:29:56.220 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:56.221 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:56.221 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.221 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.221 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:56.221 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:56.221 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:56.221 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:56.221 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:56.221 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:56.221 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:56.221 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:56.221 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:56.221 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:56.221 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:56.221 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:56.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 10 monclient(hunting): _finish_auth 0 2024-01-24T23:29:56.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 <== mon.3 v1:172.21.15.132:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2187257464 0 0) 0x7f1e7409c480 con 0x7f1e74139700 2024-01-24T23:29:56.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 --> v1:172.21.15.132:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1e60003530 con 0x7f1e74139700 2024-01-24T23:29:56.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 <== mon.3 v1:172.21.15.132:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1e6c001a40 con 0x7f1e74139700 2024-01-24T23:29:56.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:56.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-01-24T23:29:56.222 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 10 monclient(hunting): dump: 2024-01-24T23:29:56.222 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:56.222 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:56.222 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.222 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.222 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:56.222 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:56.222 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:56.222 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:56.223 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:56.223 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:56.223 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:56.223 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:56.223 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:56.223 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:56.223 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:56.223 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:56.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 <== mon.5 v1:172.21.15.132:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4262184996 0 0) 0x7f1e7409c9f0 con 0x7f1e740720b0 2024-01-24T23:29:56.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 >> v1:172.21.15.132:6790/0 conn(0x7f1e74139700 legacy=0x7f1e741cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:56.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 >> v1:172.21.15.132:6789/0 conn(0x7f1e7409b9f0 legacy=0x7f1e741cbff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-24T23:29:56.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 10 monclient: _finish_hunting 0 2024-01-24T23:29:56.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 monclient: found mon.h 2024-01-24T23:29:56.223 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-01-24T23:29:56.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:29:56.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 --> v1:172.21.15.132:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e74071a40 con 0x7f1e740720b0 2024-01-24T23:29:56.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 10 monclient: _finish_auth 0 2024-01-24T23:29:56.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 10 monclient: _check_auth_tickets 2024-01-24T23:29:56.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-01-24T23:29:56.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e7a58a640 5 monclient: authenticate success, global_id 7250 2024-01-24T23:29:56.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e7a58a640 10 monclient: _renew_subs 2024-01-24T23:29:56.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e7a58a640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:29:56.224 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e7a58a640 1 -- 172.21.15.115:0/1541917546 --> v1:172.21.15.132:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e7411d1d0 con 0x7f1e740720b0 2024-01-24T23:29:56.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e7a58a640 10 monclient: _renew_subs 2024-01-24T23:29:56.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e7a58a640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.132:6791/0 2024-01-24T23:29:56.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e7a58a640 1 -- 172.21.15.115:0/1541917546 --> v1:172.21.15.132:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7411bf40 con 0x7f1e740720b0 2024-01-24T23:29:56.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 <== mon.5 v1:172.21.15.132:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1e680046a0 con 0x7f1e740720b0 2024-01-24T23:29:56.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 10 monclient: handle_config config(0 keys) v1 2024-01-24T23:29:56.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 <== mon.5 v1:172.21.15.132:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1e68004f50 con 0x7f1e740720b0 2024-01-24T23:29:56.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:56.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-01-24T23:29:56.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e727fc640 4 set_mon_vals no callback set 2024-01-24T23:29:56.225 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 10 monclient: dump: 2024-01-24T23:29:56.225 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:56.225 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:56.226 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.226 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.226 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:56.226 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:56.226 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:56.226 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:56.226 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:56.226 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:56.226 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:56.226 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:56.226 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:56.226 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:56.226 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:56.226 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:56.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e7a58a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:56.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e7a58a640 1 -- 172.21.15.115:0/1541917546 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1e540065d0 con 0x7f1e54002870 2024-01-24T23:29:56.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 <== mon.5 v1:172.21.15.132:6791/0 7 ==== mgrmap(e 5) v1 ==== 43288+0+0 (unknown 3133728714 0 0) 0x7f1e6800f9d0 con 0x7f1e740720b0 2024-01-24T23:29:56.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 <== mon.5 v1:172.21.15.132:6791/0 8 ==== osd_map(253..253 src has 1..253) v4 ==== 7051+0+0 (unknown 1249741573 0 0) 0x7f1e680443f0 con 0x7f1e740720b0 2024-01-24T23:29:56.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956051808 0 0) 0x7f1e540065d0 con 0x7f1e54002870 2024-01-24T23:29:56.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:29:56.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:56.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:56.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 10 monclient(hunting): my global_id is 154339 2024-01-24T23:29:56.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e7409c9f0 con 0x7f1e54002870 2024-01-24T23:29:56.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e78b27640 10 monclient: get_auth_request con 0x7f1e600376a0 auth_method 0 2024-01-24T23:29:56.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4067858273 0 0) 0x7f1e7409c9f0 con 0x7f1e54002870 2024-01-24T23:29:56.227 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1e540065d0 con 0x7f1e54002870 2024-01-24T23:29:56.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1e640040c0 con 0x7f1e54002870 2024-01-24T23:29:56.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:56.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:29:56.228 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 10 monclient: dump: 2024-01-24T23:29:56.228 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:56.228 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:56.228 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.228 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.228 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:56.228 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:56.228 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:56.228 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:56.229 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:56.229 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:56.229 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:56.229 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:56.229 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:56.229 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:56.229 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:56.229 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:56.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 726039933 0 0) 0x7f1e540065d0 con 0x7f1e54002870 2024-01-24T23:29:56.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 --> v1:172.21.15.132:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e5400a120 con 0x7f1e54002870 2024-01-24T23:29:56.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 <== mon.7 v1:172.21.15.132:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f1e640064d0 con 0x7f1e54002870 2024-01-24T23:29:56.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-01-24T23:29:56.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 10 monclient: _finish_command 1 = system:0 2024-01-24T23:29:56.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 >> v1:172.21.15.132:6792/0 conn(0x7f1e54002870 legacy=0x7f1e54004d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:56.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.203+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 >> v1:172.21.15.132:6792/0 conn(0x7f1e54002870 legacy=0x7f1e54004d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:56.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.211+0000 7f1e7a58a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-01-24T23:29:56.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.211+0000 7f1e7a58a640 1 -- 172.21.15.115:0/1541917546 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1e54009e10 con 0x7f1e540090d0 2024-01-24T23:29:56.231 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.211+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 <== mon.7 v1:172.21.15.132:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 557479730 0 0) 0x7f1e54009e10 con 0x7f1e540090d0 2024-01-24T23:29:56.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.211+0000 7f1e70ff9640 10 monclient(hunting): _negotiate 2024-01-24T23:29:56.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.211+0000 7f1e70ff9640 10 monclient(hunting): _init_auth method 2 2024-01-24T23:29:56.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.211+0000 7f1e70ff9640 10 monclient(hunting): _init_auth creating new auth 2024-01-24T23:29:56.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.211+0000 7f1e70ff9640 10 monclient(hunting): my global_id is 154348 2024-01-24T23:29:56.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.211+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 --> v1:172.21.15.132:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e540065d0 con 0x7f1e540090d0 2024-01-24T23:29:56.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.211+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 <== mon.7 v1:172.21.15.132:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 207712507 0 0) 0x7f1e540065d0 con 0x7f1e540090d0 2024-01-24T23:29:56.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.211+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 --> v1:172.21.15.132:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1e54009e10 con 0x7f1e540090d0 2024-01-24T23:29:56.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.211+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 <== mon.7 v1:172.21.15.132:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 4248983288 0 0) 0x7f1e64004c50 con 0x7f1e540090d0 2024-01-24T23:29:56.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.211+0000 7f1e70ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-01-24T23:29:56.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.211+0000 7f1e70ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-01-24T23:29:56.232 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.211+0000 7f1e70ff9640 10 monclient: dump: 2024-01-24T23:29:56.233 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 1 2024-01-24T23:29:56.233 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:29:56.233 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.233 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:20:20.557246+0000 2024-01-24T23:29:56.233 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 19 (squid) 2024-01-24T23:29:56.233 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:29:56.233 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.a 2024-01-24T23:29:56.233 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.f 2024-01-24T23:29:56.233 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.b 2024-01-24T23:29:56.233 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.g 2024-01-24T23:29:56.233 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.c 2024-01-24T23:29:56.233 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.h 2024-01-24T23:29:56.233 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.d 2024-01-24T23:29:56.234 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.i 2024-01-24T23:29:56.234 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.e 2024-01-24T23:29:56.234 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:29:56.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.211+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 <== mon.7 v1:172.21.15.132:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3279291791 0 0) 0x7f1e54009e10 con 0x7f1e540090d0 2024-01-24T23:29:56.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.211+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 --> v1:172.21.15.132:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1e5400c310 con 0x7f1e540090d0 2024-01-24T23:29:56.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.215+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 <== mon.7 v1:172.21.15.132:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7510 (unknown 0 0 278253991) 0x7f1e640061d0 con 0x7f1e540090d0 2024-01-24T23:29:56.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.215+0000 7f1e70ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-01-24T23:29:56.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.215+0000 7f1e70ff9640 10 monclient: _finish_command 2 = system:0 2024-01-24T23:29:56.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.215+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 >> v1:172.21.15.132:6792/0 conn(0x7f1e540090d0 legacy=0x7f1e54009560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:56.234 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.215+0000 7f1e70ff9640 1 -- 172.21.15.115:0/1541917546 >> v1:172.21.15.132:6792/0 conn(0x7f1e540090d0 legacy=0x7f1e54009560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:29:56.235 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-01-24T23:29:56.235 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:56.235 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:56.235 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "peon", 2024-01-24T23:29:56.235 INFO:teuthology.orchestra.run.smithi115.stdout: "election_epoch": 52, 2024-01-24T23:29:56.235 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum": [ 2024-01-24T23:29:56.235 INFO:teuthology.orchestra.run.smithi115.stdout: 2, 2024-01-24T23:29:56.235 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-01-24T23:29:56.235 INFO:teuthology.orchestra.run.smithi115.stdout: 5, 2024-01-24T23:29:56.236 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-01-24T23:29:56.236 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-01-24T23:29:56.236 INFO:teuthology.orchestra.run.smithi115.stdout: 8 2024-01-24T23:29:56.236 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:56.236 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_age": 4, 2024-01-24T23:29:56.236 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:56.236 INFO:teuthology.orchestra.run.smithi115.stdout: "required_con": "2450521708007866372", 2024-01-24T23:29:56.236 INFO:teuthology.orchestra.run.smithi115.stdout: "required_mon": [ 2024-01-24T23:29:56.236 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:56.236 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:56.237 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:56.237 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:56.237 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:56.237 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:56.237 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:56.237 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:56.237 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:56.237 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:56.237 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:56.237 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:56.237 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_con": "4540701272860131327", 2024-01-24T23:29:56.237 INFO:teuthology.orchestra.run.smithi115.stdout: "quorum_mon": [ 2024-01-24T23:29:56.237 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:56.238 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:56.238 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:56.238 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:56.238 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:56.238 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:56.238 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:56.238 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:56.238 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:56.238 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:56.238 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:56.238 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:56.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.238 INFO:teuthology.orchestra.run.smithi115.stdout: "outside_quorum": [], 2024-01-24T23:29:56.238 INFO:teuthology.orchestra.run.smithi115.stdout: "extra_probe_peers": [], 2024-01-24T23:29:56.239 INFO:teuthology.orchestra.run.smithi115.stdout: "sync_provider": [], 2024-01-24T23:29:56.239 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": { 2024-01-24T23:29:56.239 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 1, 2024-01-24T23:29:56.239 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "4c8039f0-ed94-40df-8309-0a291ebf7f77", 2024-01-24T23:29:56.239 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:56.239 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-01-24T23:20:20.557246Z", 2024-01-24T23:29:56.239 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release": 19, 2024-01-24T23:29:56.239 INFO:teuthology.orchestra.run.smithi115.stdout: "min_mon_release_name": "squid", 2024-01-24T23:29:56.239 INFO:teuthology.orchestra.run.smithi115.stdout: "election_strategy": 3, 2024-01-24T23:29:56.240 INFO:teuthology.orchestra.run.smithi115.stdout: "disallowed_leaders: ": "", 2024-01-24T23:29:56.240 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false, 2024-01-24T23:29:56.240 INFO:teuthology.orchestra.run.smithi115.stdout: "tiebreaker_mon": "", 2024-01-24T23:29:56.240 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_ranks: ": "", 2024-01-24T23:29:56.241 INFO:teuthology.orchestra.run.smithi115.stdout: "features": { 2024-01-24T23:29:56.241 INFO:teuthology.orchestra.run.smithi115.stdout: "persistent": [ 2024-01-24T23:29:56.241 INFO:teuthology.orchestra.run.smithi115.stdout: "kraken", 2024-01-24T23:29:56.241 INFO:teuthology.orchestra.run.smithi115.stdout: "luminous", 2024-01-24T23:29:56.241 INFO:teuthology.orchestra.run.smithi115.stdout: "mimic", 2024-01-24T23:29:56.241 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap-prune", 2024-01-24T23:29:56.241 INFO:teuthology.orchestra.run.smithi115.stdout: "nautilus", 2024-01-24T23:29:56.242 INFO:teuthology.orchestra.run.smithi115.stdout: "octopus", 2024-01-24T23:29:56.242 INFO:teuthology.orchestra.run.smithi115.stdout: "pacific", 2024-01-24T23:29:56.242 INFO:teuthology.orchestra.run.smithi115.stdout: "elector-pinging", 2024-01-24T23:29:56.242 INFO:teuthology.orchestra.run.smithi115.stdout: "quincy", 2024-01-24T23:29:56.242 INFO:teuthology.orchestra.run.smithi115.stdout: "reef", 2024-01-24T23:29:56.242 INFO:teuthology.orchestra.run.smithi115.stdout: "squid" 2024-01-24T23:29:56.242 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:56.242 INFO:teuthology.orchestra.run.smithi115.stdout: "optional": [] 2024-01-24T23:29:56.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.242 INFO:teuthology.orchestra.run.smithi115.stdout: "mons": [ 2024-01-24T23:29:56.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.242 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 0, 2024-01-24T23:29:56.243 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "a", 2024-01-24T23:29:56.243 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:56.243 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:56.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.243 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:56.243 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789", 2024-01-24T23:29:56.243 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:56.243 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.243 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:56.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.243 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6789/0", 2024-01-24T23:29:56.243 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6789/0", 2024-01-24T23:29:56.243 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:56.244 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:56.244 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:56.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.244 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 1, 2024-01-24T23:29:56.244 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "f", 2024-01-24T23:29:56.244 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:56.244 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:56.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.244 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:56.244 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789", 2024-01-24T23:29:56.244 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:56.244 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.245 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:56.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.245 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6789/0", 2024-01-24T23:29:56.245 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6789/0", 2024-01-24T23:29:56.245 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:56.245 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:56.245 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:56.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.246 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 2, 2024-01-24T23:29:56.246 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "b", 2024-01-24T23:29:56.246 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:56.246 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:56.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.246 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:56.246 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790", 2024-01-24T23:29:56.246 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:56.246 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.247 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:56.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.247 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6790/0", 2024-01-24T23:29:56.247 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6790/0", 2024-01-24T23:29:56.247 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:56.247 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:56.247 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:56.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.247 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 3, 2024-01-24T23:29:56.247 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "g", 2024-01-24T23:29:56.247 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:56.247 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:56.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.248 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:56.248 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790", 2024-01-24T23:29:56.248 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:56.248 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.248 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:56.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.248 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6790/0", 2024-01-24T23:29:56.248 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6790/0", 2024-01-24T23:29:56.248 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:56.248 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:56.248 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:56.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.249 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 4, 2024-01-24T23:29:56.249 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "c", 2024-01-24T23:29:56.249 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:56.249 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:56.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.249 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:56.249 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791", 2024-01-24T23:29:56.249 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:56.249 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.249 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:56.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.249 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6791/0", 2024-01-24T23:29:56.249 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6791/0", 2024-01-24T23:29:56.249 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:56.250 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:56.250 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:56.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.250 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 5, 2024-01-24T23:29:56.250 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "h", 2024-01-24T23:29:56.250 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:56.250 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:56.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.250 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:56.250 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791", 2024-01-24T23:29:56.250 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:56.250 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.250 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:56.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.251 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6791/0", 2024-01-24T23:29:56.251 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6791/0", 2024-01-24T23:29:56.251 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:56.251 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:56.251 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:56.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.251 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 6, 2024-01-24T23:29:56.251 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "d", 2024-01-24T23:29:56.251 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:56.251 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:56.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.252 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:56.252 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792", 2024-01-24T23:29:56.252 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:56.252 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.252 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:56.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.253 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6792/0", 2024-01-24T23:29:56.253 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6792/0", 2024-01-24T23:29:56.253 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:56.253 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:56.253 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:56.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.253 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 7, 2024-01-24T23:29:56.253 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "i", 2024-01-24T23:29:56.253 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:56.253 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:56.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.253 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:56.253 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792", 2024-01-24T23:29:56.254 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:56.254 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.254 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:56.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.254 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.132:6792/0", 2024-01-24T23:29:56.254 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.132:6792/0", 2024-01-24T23:29:56.254 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:56.254 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:56.254 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:56.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.254 INFO:teuthology.orchestra.run.smithi115.stdout: "rank": 8, 2024-01-24T23:29:56.254 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "e", 2024-01-24T23:29:56.254 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-01-24T23:29:56.255 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-01-24T23:29:56.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.255 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-01-24T23:29:56.255 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793", 2024-01-24T23:29:56.255 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 0 2024-01-24T23:29:56.255 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.255 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:56.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.255 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6793/0", 2024-01-24T23:29:56.255 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6793/0", 2024-01-24T23:29:56.255 INFO:teuthology.orchestra.run.smithi115.stdout: "priority": 0, 2024-01-24T23:29:56.255 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 0, 2024-01-24T23:29:56.255 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "{}" 2024-01-24T23:29:56.256 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.256 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:56.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.256 INFO:teuthology.orchestra.run.smithi115.stdout: "feature_map": { 2024-01-24T23:29:56.256 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": [ 2024-01-24T23:29:56.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.256 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:56.256 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:56.256 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:56.256 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.257 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-01-24T23:29:56.257 INFO:teuthology.orchestra.run.smithi115.stdout: "client": [ 2024-01-24T23:29:56.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-01-24T23:29:56.257 INFO:teuthology.orchestra.run.smithi115.stdout: "features": "0x3f03cfbffffdffff", 2024-01-24T23:29:56.257 INFO:teuthology.orchestra.run.smithi115.stdout: "release": "luminous", 2024-01-24T23:29:56.257 INFO:teuthology.orchestra.run.smithi115.stdout: "num": 1 2024-01-24T23:29:56.257 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-01-24T23:29:56.257 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-01-24T23:29:56.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-01-24T23:29:56.257 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": false 2024-01-24T23:29:56.257 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-01-24T23:29:56.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.215+0000 7f1e7a58a640 1 -- 172.21.15.115:0/1541917546 >> v1:172.21.15.132:6812/1835401922 conn(0x7f1e600376a0 legacy=0x7f1e60039b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:56.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.215+0000 7f1e7a58a640 10 monclient: shutdown 2024-01-24T23:29:56.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.215+0000 7f1e7a58a640 20 monclient: shutdown discarding 0 pending message(s) 2024-01-24T23:29:56.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.215+0000 7f1e7a58a640 1 -- 172.21.15.115:0/1541917546 >> v1:172.21.15.132:6791/0 conn(0x7f1e740720b0 legacy=0x7f1e741326c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:29:56.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.215+0000 7f1e78b27640 1 -- 172.21.15.115:0/1541917546 reap_dead start 2024-01-24T23:29:56.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.215+0000 7f1e78b27640 1 -- 172.21.15.115:0/1541917546 reap_dead start 2024-01-24T23:29:56.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.215+0000 7f1e7a58a640 1 -- 172.21.15.115:0/1541917546 shutdown_connections 2024-01-24T23:29:56.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.215+0000 7f1e7a58a640 1 -- 172.21.15.115:0/1541917546 >> 172.21.15.115:0/1541917546 conn(0x7f1e740a5730 msgr2=0x7f1e74133cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:29:56.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.215+0000 7f1e7a58a640 1 -- 172.21.15.115:0/1541917546 shutdown_connections 2024-01-24T23:29:56.259 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:29:56.215+0000 7f1e7a58a640 1 -- 172.21.15.115:0/1541917546 wait complete. 2024-01-24T23:29:56.260 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-01-24T23:29:56.260 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-01-24T23:30:01.795 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~238s 2024-01-24T23:30:08.003 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~244s 2024-01-24T23:30:14.211 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~250s 2024-01-24T23:30:20.419 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~256s 2024-01-24T23:30:26.628 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~262s 2024-01-24T23:30:32.838 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~269s 2024-01-24T23:30:39.046 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~275s 2024-01-24T23:30:45.255 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~281s 2024-01-24T23:30:51.466 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~287s 2024-01-24T23:30:57.676 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~293s 2024-01-24T23:31:03.885 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~300s 2024-01-24T23:31:10.095 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~306s 2024-01-24T23:31:10.095 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2024-01-24T23:31:11.303 INFO:tasks.ceph.osd.0:Sent signal 15 2024-01-24T23:31:11.304 INFO:tasks.ceph.osd.1:Sent signal 15 2024-01-24T23:31:11.304 INFO:tasks.ceph.osd.3:Sent signal 15 2024-01-24T23:31:11.304 INFO:tasks.ceph.osd.4:Sent signal 15 2024-01-24T23:31:11.304 INFO:tasks.ceph.osd.5:Sent signal 15 2024-01-24T23:31:11.305 INFO:tasks.ceph.mon.b:Sent signal 15 2024-01-24T23:31:11.305 INFO:tasks.ceph.mon.d:Sent signal 15 2024-01-24T23:31:11.305 INFO:tasks.ceph.mon.e:Sent signal 15 2024-01-24T23:31:11.305 INFO:tasks.ceph.mon.g:Sent signal 15 2024-01-24T23:31:11.305 INFO:tasks.ceph.mon.h:Sent signal 15 2024-01-24T23:31:11.306 INFO:tasks.ceph.mon.i:Sent signal 15 2024-01-24T23:31:11.306 INFO:tasks.ceph.mgr.x:Sent signal 15 2024-01-24T23:31:11.307 INFO:tasks.ceph.osd.3.smithi132.stderr:2024-01-24T23:31:11.308+0000 7fed3c2d0640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 23888) UID: 0 2024-01-24T23:31:11.307 INFO:tasks.ceph.osd.3.smithi132.stderr:2024-01-24T23:31:11.308+0000 7fed3c2d0640 -1 osd.3 253 *** Got signal Terminated *** 2024-01-24T23:31:11.307 INFO:tasks.ceph.osd.3.smithi132.stderr:2024-01-24T23:31:11.308+0000 7fed3c2d0640 -1 osd.3 253 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-24T23:31:11.307 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-01-24T23:31:11.306+0000 7fa650ccc640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 23781) UID: 0 2024-01-24T23:31:11.307 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-01-24T23:31:11.306+0000 7fa650ccc640 -1 osd.0 253 *** Got signal Terminated *** 2024-01-24T23:31:11.307 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-01-24T23:31:11.306+0000 7fa650ccc640 -1 osd.0 253 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-24T23:31:11.308 INFO:tasks.ceph.osd.4.smithi132.stderr:2024-01-24T23:31:11.308+0000 7f71418c5640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 23890) UID: 0 2024-01-24T23:31:11.308 INFO:tasks.ceph.osd.4.smithi132.stderr:2024-01-24T23:31:11.308+0000 7f71418c5640 -1 osd.4 253 *** Got signal Terminated *** 2024-01-24T23:31:11.308 INFO:tasks.ceph.osd.4.smithi132.stderr:2024-01-24T23:31:11.308+0000 7f71418c5640 -1 osd.4 253 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-24T23:31:11.308 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-01-24T23:31:11.306+0000 7f1b53e72640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 23783) UID: 0 2024-01-24T23:31:11.308 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-01-24T23:31:11.306+0000 7f1b53e72640 -1 osd.1 253 *** Got signal Terminated *** 2024-01-24T23:31:11.308 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-01-24T23:31:11.306+0000 7f1b53e72640 -1 osd.1 253 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-24T23:31:11.309 INFO:tasks.ceph.osd.5.smithi132.stderr:2024-01-24T23:31:11.308+0000 7f51d779f640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 23891) UID: 0 2024-01-24T23:31:11.309 INFO:tasks.ceph.osd.5.smithi132.stderr:2024-01-24T23:31:11.308+0000 7f51d779f640 -1 osd.5 253 *** Got signal Terminated *** 2024-01-24T23:31:11.309 INFO:tasks.ceph.osd.5.smithi132.stderr:2024-01-24T23:31:11.308+0000 7f51d779f640 -1 osd.5 253 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-24T23:31:11.309 INFO:tasks.ceph.mon.b.smithi115.stderr:2024-01-24T23:31:11.306+0000 7f327c7e3640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mon -f --cluster ceph -i b (PID: 30396) UID: 0 2024-01-24T23:31:11.310 INFO:tasks.ceph.mon.b.smithi115.stderr:2024-01-24T23:31:11.306+0000 7f327c7e3640 -1 mon.b@2(peon) e1 *** Got Signal Terminated *** 2024-01-24T23:31:11.310 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:31:11.312+0000 7fd2dbcd8640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mon -f --cluster ceph -i g (PID: 23546) UID: 0 2024-01-24T23:31:11.310 INFO:tasks.ceph.mon.g.smithi132.stderr:2024-01-24T23:31:11.312+0000 7fd2dbcd8640 -1 mon.g@3(peon) e1 *** Got Signal Terminated *** 2024-01-24T23:31:11.310 INFO:tasks.ceph.mon.d.smithi115.stderr:2024-01-24T23:31:11.306+0000 7f063707e640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mon -f --cluster ceph -i d (PID: 23548) UID: 0 2024-01-24T23:31:11.310 INFO:tasks.ceph.mon.d.smithi115.stderr:2024-01-24T23:31:11.306+0000 7f063707e640 -1 mon.d@6(peon) e1 *** Got Signal Terminated *** 2024-01-24T23:31:11.311 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:31:11.312+0000 7f48db87d640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mon -f --cluster ceph -i h (PID: 23548) UID: 0 2024-01-24T23:31:11.311 INFO:tasks.ceph.mon.i.smithi132.stderr:2024-01-24T23:31:11.312+0000 7fb028881640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mon -f --cluster ceph -i i (PID: 34190) UID: 0 2024-01-24T23:31:11.311 INFO:tasks.ceph.mon.e.smithi115.stderr:2024-01-24T23:31:11.306+0000 7fc112c71640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mon -f --cluster ceph -i e (PID: 27868) UID: 0 2024-01-24T23:31:11.312 INFO:tasks.ceph.mon.e.smithi115.stderr:2024-01-24T23:31:11.306+0000 7fc112c71640 -1 mon.e@8(peon) e1 *** Got Signal Terminated *** 2024-01-24T23:31:11.312 INFO:tasks.ceph.mon.h.smithi132.stderr:2024-01-24T23:31:11.312+0000 7f48db87d640 -1 mon.h@5(leader) e1 *** Got Signal Terminated *** 2024-01-24T23:31:11.312 INFO:tasks.ceph.mon.i.smithi132.stderr:2024-01-24T23:31:11.312+0000 7fb028881640 -1 mon.i@7(peon) e1 *** Got Signal Terminated *** 2024-01-24T23:31:26.260 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-01-24T23:31:26.261 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-01-24T23:31:26.261 INFO:tasks.ceph.mon.a:Restarting daemon 2024-01-24T23:31:26.261 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-01-24T23:31:26.264 INFO:tasks.ceph.mon.a:Started 2024-01-24T23:31:26.264 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-01-24T23:31:26.264 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.f 2024-01-24T23:31:26.264 INFO:tasks.ceph.mon.f:Restarting daemon 2024-01-24T23:31:26.264 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-01-24T23:31:26.267 INFO:tasks.ceph.mon.f:Started 2024-01-24T23:31:26.267 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-01-24T23:31:26.267 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-01-24T23:31:26.267 INFO:tasks.ceph.mon.c:Restarting daemon 2024-01-24T23:31:26.267 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-01-24T23:31:26.269 INFO:tasks.ceph.mon.c:Started 2024-01-24T23:31:26.269 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-01-24T23:31:26.270 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-01-24T23:31:26.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:26.398+0000 7ff12b92d640 10 monclient: get_monmap_and_config 2024-01-24T23:31:26.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:26.398+0000 7ff12b92d640 10 monclient: build_initial_monmap 2024-01-24T23:31:26.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:26.398+0000 7ff12b92d640 10 monclient: monmap: 2024-01-24T23:31:26.397 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:31:26.397 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:31:26.397 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:31:26.400949+0000 2024-01-24T23:31:26.397 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:31:26.400949+0000 2024-01-24T23:31:26.397 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:31:26.397 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:31:26.397 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:31:26.397 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:31:26.398 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:31:26.398 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:31:26.398 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:31:26.398 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:31:26.398 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:31:26.398 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:31:26.398 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:31:26.398 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:31:26.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:26.398+0000 7ff12b92d640 1 Processor -- start 2024-01-24T23:31:26.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:26.398+0000 7ff12b92d640 1 -- start start 2024-01-24T23:31:26.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:26.398+0000 7ff12b92d640 10 monclient: init 2024-01-24T23:31:26.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:26.398+0000 7ff12b92d640 10 monclient: _reopen_session rank -1 2024-01-24T23:31:26.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:26.398+0000 7ff12b92d640 10 monclient: _add_conns ranks=[8,2,4,0,6,7,3,1,5] 2024-01-24T23:31:26.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:26.398+0000 7ff12b92d640 10 monclient(hunting): picked mon.noname-e con 0x7ff124133bf0 addr v1:172.21.15.115:6793/0 2024-01-24T23:31:26.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:26.398+0000 7ff12b92d640 10 monclient(hunting): picked mon.noname-b con 0x7ff124133fd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:31:26.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:26.398+0000 7ff12b92d640 10 monclient(hunting): picked mon.noname-c con 0x7ff1241343b0 addr v1:172.21.15.115:6791/0 2024-01-24T23:31:26.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:26.398+0000 7ff12b92d640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12404ed70 con 0x7ff124133fd0 2024-01-24T23:31:26.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:26.398+0000 7ff12b92d640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1240a4b60 con 0x7ff1241343b0 2024-01-24T23:31:26.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:26.398+0000 7ff12b92d640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12409ae50 con 0x7ff124133bf0 2024-01-24T23:31:26.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:26.398+0000 7ff12b92d640 10 monclient(hunting): _renew_subs 2024-01-24T23:31:26.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:26.398+0000 7ff12b92d640 10 monclient(hunting): authenticate will time out at 1565.323207s 2024-01-24T23:31:26.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:26.398+0000 7ff1296c9640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7ff124133bf0 legacy=0x7ff12409bb10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:31:26.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:26.398+0000 7ff128ec8640 1 -- >> v1:172.21.15.115:6790/0 conn(0x7ff124133fd0 legacy=0x7ff12413bd60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:31:26.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:26.398+0000 7ff129eca640 1 --1- >> v1:172.21.15.115:6791/0 conn(0x7ff1241343b0 0x7ff12413f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6791/0 says I am v1:172.21.15.115:44192/0 (socket says 172.21.15.115:44192) 2024-01-24T23:31:26.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:26.398+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 learned_addr learned my addr 172.21.15.115:0/915557570 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:31:26.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:26.602+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff124133bf0 legacy=0x7ff12409bb10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:31:26.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:26.602+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff124133fd0 legacy=0x7ff12413bd60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:31:27.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:27.006+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff124133fd0 legacy=0x7ff12413bd60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:31:27.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:27.006+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff124133bf0 legacy=0x7ff12409bb10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:31:27.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:27.810+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff124133fd0 legacy=0x7ff12413bd60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:31:27.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:27.810+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff124133bf0 legacy=0x7ff12409bb10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:31:29.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:29.398+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:31:29.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:29.398+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:31:29.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:29.398+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:31:29.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:29.398+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:31:29.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:29.398+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff124133bf0 legacy=0x7ff12409bb10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:31:29.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:29.398+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff1241343b0 legacy=0x7ff12413f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:31:29.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:29.398+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff124133fd0 legacy=0x7ff12413bd60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:31:29.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:29.398+0000 7ff11affd640 10 monclient: _add_conns ranks=[6,3,5,1,0,7] 2024-01-24T23:31:29.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:29.398+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-d con 0x7ff1080037b0 addr v1:172.21.15.115:6792/0 2024-01-24T23:31:29.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:29.398+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-g con 0x7ff108007440 addr v1:172.21.15.132:6790/0 2024-01-24T23:31:29.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:29.398+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-h con 0x7ff10800b1b0 addr v1:172.21.15.132:6791/0 2024-01-24T23:31:29.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:29.398+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12409b130 con 0x7ff108007440 2024-01-24T23:31:29.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:29.398+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12404ed70 con 0x7ff10800b1b0 2024-01-24T23:31:29.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:29.398+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12409ae50 con 0x7ff1080037b0 2024-01-24T23:31:29.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:29.398+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:31:29.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:29.398+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108007440 legacy=0x7ff1080098e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:31:29.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:29.398+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff1080037b0 legacy=0x7ff108005c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:31:29.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:29.398+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800b1b0 legacy=0x7ff10800d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:31:29.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:29.602+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff1080037b0 legacy=0x7ff108005c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:31:29.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:29.602+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108007440 legacy=0x7ff1080098e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:31:29.599 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:29.602+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800b1b0 legacy=0x7ff10800d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:31:30.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:30.006+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff1080037b0 legacy=0x7ff108005c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:31:30.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:30.006+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800b1b0 legacy=0x7ff10800d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:31:30.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:30.006+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108007440 legacy=0x7ff1080098e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:31:30.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:30.810+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff1080037b0 legacy=0x7ff108005c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:31:30.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:30.810+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108007440 legacy=0x7ff1080098e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:31:30.807 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:30.810+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800b1b0 legacy=0x7ff10800d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:31:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:32.398+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:31:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:32.398+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:31:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:32.398+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:31:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:32.398+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:31:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:32.398+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff1080037b0 legacy=0x7ff108005c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:31:32.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:32.398+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800b1b0 legacy=0x7ff10800d670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:31:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:32.398+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108007440 legacy=0x7ff1080098e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:31:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:32.398+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:31:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:32.398+0000 7ff11affd640 10 monclient: _add_conns ranks=[0,7,1] 2024-01-24T23:31:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:32.398+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:31:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:32.398+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-a con 0x7ff108010030 addr v1:172.21.15.115:6789/0 2024-01-24T23:31:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:32.402+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-i con 0x7ff108010410 addr v1:172.21.15.132:6792/0 2024-01-24T23:31:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:32.402+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-f con 0x7ff1080109f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:31:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:32.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108012040 con 0x7ff108010030 2024-01-24T23:31:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:32.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12409b130 con 0x7ff1080109f0 2024-01-24T23:31:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:32.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12404ed70 con 0x7ff108010410 2024-01-24T23:31:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:32.402+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:31:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:32.402+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff108010410 legacy=0x7ff10800d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:31:32.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:32.606+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff108010410 legacy=0x7ff10800d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:31:33.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:33.010+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff108010410 legacy=0x7ff10800d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:31:33.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:33.814+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff108010410 legacy=0x7ff10800d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:31:35.399 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:35.402+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:31:35.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:35.402+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:31:35.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:35.402+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:31:35.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:35.402+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:31:35.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:35.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff108010410 legacy=0x7ff10800d670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:31:35.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:35.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff1080109f0 legacy=0x7ff108005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:31:35.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:35.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff108010030 legacy=0x7ff1080098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:31:35.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:35.402+0000 7ff11affd640 10 monclient: _add_conns ranks=[2,5,0,3,1,7,8,4,6] 2024-01-24T23:31:35.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:35.402+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-b con 0x7ff108006510 addr v1:172.21.15.115:6790/0 2024-01-24T23:31:35.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:35.402+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-h con 0x7ff108014340 addr v1:172.21.15.132:6791/0 2024-01-24T23:31:35.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:35.402+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-a con 0x7ff108010030 addr v1:172.21.15.115:6789/0 2024-01-24T23:31:35.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:35.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12409ae50 con 0x7ff108010030 2024-01-24T23:31:35.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:35.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12404ed70 con 0x7ff108006510 2024-01-24T23:31:35.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:35.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108019e70 con 0x7ff108014340 2024-01-24T23:31:35.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:35.402+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:31:35.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:35.402+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108006510 legacy=0x7ff1080068f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:31:35.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:35.402+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff108014340 legacy=0x7ff108016730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:31:35.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:35.606+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108006510 legacy=0x7ff1080068f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:31:35.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:35.606+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff108014340 legacy=0x7ff108016730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:31:36.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:36.010+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108006510 legacy=0x7ff1080068f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:31:36.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:36.010+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff108014340 legacy=0x7ff108016730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:31:36.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:36.354+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff108010030 legacy=0x7ff1080098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:31:36.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:36.354+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff108010030 legacy=0x7ff1080098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:31:36.353 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:36.354+0000 7ff129eca640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff108010030 0x7ff1080098e0 :-1 s=OPENED pgs=50 cs=1 l=1).handle_message read tag failed 2024-01-24T23:31:36.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:36.354+0000 7ff129eca640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff108010030 0x7ff1080098e0 :-1 s=OPENED pgs=50 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:31:36.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:36.354+0000 7ff11bfff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:31:36.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:36.814+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108006510 legacy=0x7ff1080068f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:31:36.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:36.814+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff108014340 legacy=0x7ff108016730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:31:38.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:38.402+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:31:38.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:38.402+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:31:38.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:38.402+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:31:38.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:38.402+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:31:38.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:38.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff108014340 legacy=0x7ff108016730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:31:38.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:38.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108006510 legacy=0x7ff1080068f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:31:38.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:38.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff108010030 legacy=0x7ff1080098e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:31:38.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:38.402+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:31:38.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:38.402+0000 7ff11affd640 10 monclient: _add_conns ranks=[8,4,6,7,1,3] 2024-01-24T23:31:38.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:38.402+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-e con 0x7ff1080068f0 addr v1:172.21.15.115:6793/0 2024-01-24T23:31:38.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:38.402+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-c con 0x7ff10801a0f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:31:38.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:38.402+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-d con 0x7ff10801a550 addr v1:172.21.15.115:6792/0 2024-01-24T23:31:38.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:38.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10801b010 con 0x7ff10801a0f0 2024-01-24T23:31:38.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:38.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12404ed70 con 0x7ff10801a550 2024-01-24T23:31:38.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:38.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108019e70 con 0x7ff1080068f0 2024-01-24T23:31:38.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:38.402+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:31:38.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:38.402+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff1080068f0 legacy=0x7ff108010030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:31:38.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:38.402+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10801a550 legacy=0x7ff108005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:31:38.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:38.606+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff1080068f0 legacy=0x7ff108010030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:31:38.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:38.606+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10801a550 legacy=0x7ff108005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:31:39.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:39.010+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff1080068f0 legacy=0x7ff108010030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:31:39.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:39.010+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10801a550 legacy=0x7ff108005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:31:39.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:39.814+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff1080068f0 legacy=0x7ff108010030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:31:39.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:39.814+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10801a550 legacy=0x7ff108005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:31:41.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:41.366+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801a0f0 legacy=0x7ff1080098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:31:41.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:41.366+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801a0f0 legacy=0x7ff1080098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:31:41.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:41.366+0000 7ff128ec8640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801a0f0 0x7ff1080098e0 :-1 s=OPENED pgs=51 cs=1 l=1).handle_message read tag failed 2024-01-24T23:31:41.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:41.366+0000 7ff128ec8640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801a0f0 0x7ff1080098e0 :-1 s=OPENED pgs=51 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:31:41.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:41.366+0000 7ff11bfff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:31:41.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:41.402+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:31:41.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:41.402+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:31:41.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:41.402+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:31:41.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:41.402+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:31:41.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:41.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff1080068f0 legacy=0x7ff108010030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:31:41.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:41.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10801a550 legacy=0x7ff108005ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:31:41.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:41.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801a0f0 legacy=0x7ff1080098e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:31:41.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:41.402+0000 7ff11affd640 10 monclient: _add_conns ranks=[7,1,3] 2024-01-24T23:31:41.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:41.402+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-i con 0x7ff10801a9e0 addr v1:172.21.15.132:6792/0 2024-01-24T23:31:41.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:41.402+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-f con 0x7ff108014720 addr v1:172.21.15.132:6789/0 2024-01-24T23:31:41.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:41.402+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-g con 0x7ff108018230 addr v1:172.21.15.132:6790/0 2024-01-24T23:31:41.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:41.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108006d90 con 0x7ff108014720 2024-01-24T23:31:41.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:41.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12404ed70 con 0x7ff108018230 2024-01-24T23:31:41.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:41.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108019e70 con 0x7ff10801a9e0 2024-01-24T23:31:41.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:41.402+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:31:41.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:41.402+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a9e0 legacy=0x7ff10800d650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:31:41.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:41.402+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108018230 legacy=0x7ff1080186a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:31:41.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:41.606+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a9e0 legacy=0x7ff10800d650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:31:41.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:41.606+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108018230 legacy=0x7ff1080186a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:31:42.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:42.010+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108018230 legacy=0x7ff1080186a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:31:42.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:42.010+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a9e0 legacy=0x7ff10800d650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:31:42.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:42.814+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a9e0 legacy=0x7ff10800d650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:31:42.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:42.814+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108018230 legacy=0x7ff1080186a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:31:44.400 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:44.402+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:31:44.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:44.402+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:31:44.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:44.402+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:31:44.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:44.402+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:31:44.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:44.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a9e0 legacy=0x7ff10800d650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:31:44.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:44.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108018230 legacy=0x7ff1080186a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:31:44.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:44.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff108014720 legacy=0x7ff108016b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:31:44.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:44.402+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:31:44.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:44.402+0000 7ff11affd640 10 monclient: _add_conns ranks=[5,1,7,6,8,0,4,2,3] 2024-01-24T23:31:44.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:44.402+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:31:44.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:44.402+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-h con 0x7ff108018230 addr v1:172.21.15.132:6791/0 2024-01-24T23:31:44.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:44.402+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-f con 0x7ff10801a0f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:31:44.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:44.402+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-i con 0x7ff10801a550 addr v1:172.21.15.132:6792/0 2024-01-24T23:31:44.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:44.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108020260 con 0x7ff10801a0f0 2024-01-24T23:31:44.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:44.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12404ed70 con 0x7ff108018230 2024-01-24T23:31:44.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:44.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108019e70 con 0x7ff10801a550 2024-01-24T23:31:44.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:44.402+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:31:44.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:44.402+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a550 legacy=0x7ff108005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:31:44.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:44.402+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff108018230 legacy=0x7ff108010030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:31:44.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:44.606+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff108018230 legacy=0x7ff108010030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:31:44.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:44.606+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a550 legacy=0x7ff108005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:31:45.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:45.010+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a550 legacy=0x7ff108005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:31:45.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:45.010+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff108018230 legacy=0x7ff108010030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:31:45.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:45.814+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff108018230 legacy=0x7ff108010030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:31:45.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:45.814+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a550 legacy=0x7ff108005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:31:46.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:46.362+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a0f0 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:31:46.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:46.362+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a0f0 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:31:46.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:46.362+0000 7ff128ec8640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a0f0 0x7ff10800de20 :-1 s=OPENED pgs=58 cs=1 l=1).handle_message read tag failed 2024-01-24T23:31:46.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:46.362+0000 7ff128ec8640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a0f0 0x7ff10800de20 :-1 s=OPENED pgs=58 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:31:46.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:46.362+0000 7ff11bfff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:31:47.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:47.402+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:31:47.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:47.402+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:31:47.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:47.402+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:31:47.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:47.402+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:31:47.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:47.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a550 legacy=0x7ff108005ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:31:47.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:47.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff108018230 legacy=0x7ff108010030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:31:47.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:47.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a0f0 legacy=0x7ff10800de20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:31:47.401 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:47.402+0000 7ff11affd640 10 monclient: _add_conns ranks=[6,4,2,3,0,8] 2024-01-24T23:31:47.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:47.402+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-d con 0x7ff10801a9e0 addr v1:172.21.15.115:6792/0 2024-01-24T23:31:47.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:47.402+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-c con 0x7ff108019690 addr v1:172.21.15.115:6791/0 2024-01-24T23:31:47.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:47.402+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-b con 0x7ff10800e530 addr v1:172.21.15.115:6790/0 2024-01-24T23:31:47.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:47.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1080068f0 con 0x7ff10800e530 2024-01-24T23:31:47.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:47.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12404ed70 con 0x7ff108019690 2024-01-24T23:31:47.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:47.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108019e70 con 0x7ff10801a9e0 2024-01-24T23:31:47.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:47.402+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10801a9e0 legacy=0x7ff1080098e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:31:47.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:47.402+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:31:47.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:47.402+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10800e530 legacy=0x7ff108016b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:31:47.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:47.606+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10801a9e0 legacy=0x7ff1080098e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:31:47.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:47.606+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10800e530 legacy=0x7ff108016b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:31:48.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:48.010+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10801a9e0 legacy=0x7ff1080098e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:31:48.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:48.010+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10800e530 legacy=0x7ff108016b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:31:48.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:48.814+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10801a9e0 legacy=0x7ff1080098e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:31:48.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:48.814+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10800e530 legacy=0x7ff108016b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:31:50.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:50.402+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:31:50.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:50.402+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:31:50.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:50.402+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:31:50.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:50.402+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:31:50.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:50.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10801a9e0 legacy=0x7ff1080098e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:31:50.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:50.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff108019690 legacy=0x7ff10800d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:31:50.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:50.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10800e530 legacy=0x7ff108016b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:31:50.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:50.402+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:31:50.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:50.402+0000 7ff11affd640 10 monclient: _add_conns ranks=[3,0,8] 2024-01-24T23:31:50.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:50.402+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:31:50.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:50.402+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-g con 0x7ff10801a0f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:31:50.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:50.402+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-a con 0x7ff10801a550 addr v1:172.21.15.115:6789/0 2024-01-24T23:31:50.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:50.402+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-e con 0x7ff10801a9e0 addr v1:172.21.15.115:6793/0 2024-01-24T23:31:50.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:50.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108019aa0 con 0x7ff10801a550 2024-01-24T23:31:50.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:50.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1080068f0 con 0x7ff10801a0f0 2024-01-24T23:31:50.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:50.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108019e70 con 0x7ff10801a9e0 2024-01-24T23:31:50.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:50.402+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:31:50.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:50.402+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10801a9e0 legacy=0x7ff108005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:31:50.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:50.402+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10801a0f0 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:31:50.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:50.606+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10801a9e0 legacy=0x7ff108005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:31:50.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:50.606+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10801a0f0 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:31:51.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:51.010+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10801a9e0 legacy=0x7ff108005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:31:51.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:51.010+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10801a0f0 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:31:51.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:51.354+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff10801a550 legacy=0x7ff108010010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:31:51.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:51.354+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff10801a550 legacy=0x7ff108010010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:31:51.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:51.354+0000 7ff128ec8640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff10801a550 0x7ff108010010 :-1 s=OPENED pgs=58 cs=1 l=1).handle_message read tag failed 2024-01-24T23:31:51.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:51.354+0000 7ff128ec8640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff10801a550 0x7ff108010010 :-1 s=OPENED pgs=58 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:31:51.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:51.354+0000 7ff11bfff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:31:51.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:51.814+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10801a9e0 legacy=0x7ff108005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:31:51.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:51.814+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10801a0f0 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:31:53.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:53.402+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:31:53.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:53.402+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:31:53.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:53.402+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:31:53.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:53.402+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:31:53.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:53.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10801a9e0 legacy=0x7ff108005ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:31:53.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:53.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10801a0f0 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:31:53.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:53.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff10801a550 legacy=0x7ff108010010 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:31:53.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:53.402+0000 7ff11affd640 10 monclient: _add_conns ranks=[6,1,7,8,3,4,0,2,5] 2024-01-24T23:31:53.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:53.402+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-d con 0x7ff10800d850 addr v1:172.21.15.115:6792/0 2024-01-24T23:31:53.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:53.402+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-f con 0x7ff108007850 addr v1:172.21.15.132:6789/0 2024-01-24T23:31:53.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:53.402+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-i con 0x7ff10801e980 addr v1:172.21.15.132:6792/0 2024-01-24T23:31:53.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:53.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10800d640 con 0x7ff108007850 2024-01-24T23:31:53.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:53.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1080068f0 con 0x7ff10800d850 2024-01-24T23:31:53.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:53.402+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108019e70 con 0x7ff10801e980 2024-01-24T23:31:53.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:53.402+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:31:53.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:53.402+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800d850 legacy=0x7ff10800e530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:31:53.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:53.402+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801e980 legacy=0x7ff10801ed60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:31:53.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:53.606+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800d850 legacy=0x7ff10800e530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:31:53.602 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:53.606+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801e980 legacy=0x7ff10801ed60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:31:54.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:54.010+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800d850 legacy=0x7ff10800e530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:31:54.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:54.010+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801e980 legacy=0x7ff10801ed60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:31:54.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:54.813+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800d850 legacy=0x7ff10800e530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:31:54.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:54.813+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801e980 legacy=0x7ff10801ed60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:31:56.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.361+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff108007850 legacy=0x7ff10801d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:31:56.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.361+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff108007850 legacy=0x7ff10801d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:31:56.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.361+0000 7ff128ec8640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff108007850 0x7ff10801d260 :-1 s=OPENED pgs=65 cs=1 l=1).handle_message read tag failed 2024-01-24T23:31:56.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.361+0000 7ff128ec8640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff108007850 0x7ff10801d260 :-1 s=OPENED pgs=65 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:31:56.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.361+0000 7ff11bfff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:31:56.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.401+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:31:56.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.401+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:31:56.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.401+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:31:56.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.401+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:31:56.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.401+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801e980 legacy=0x7ff10801ed60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:31:56.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.401+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800d850 legacy=0x7ff10800e530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:31:56.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.401+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff108007850 legacy=0x7ff10801d260 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:31:56.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.401+0000 7ff11affd640 10 monclient: _add_conns ranks=[2,0,5,3,8,4] 2024-01-24T23:31:56.402 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.401+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:31:56.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.401+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-b con 0x7ff108017b20 addr v1:172.21.15.115:6790/0 2024-01-24T23:31:56.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.401+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:31:56.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.401+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-a con 0x7ff10800d850 addr v1:172.21.15.115:6789/0 2024-01-24T23:31:56.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.401+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-h con 0x7ff10801a0f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:31:56.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.401+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10800ec40 con 0x7ff10800d850 2024-01-24T23:31:56.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.401+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1080068f0 con 0x7ff108017b20 2024-01-24T23:31:56.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.401+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108017b20 legacy=0x7ff108017f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:31:56.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.401+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108019e70 con 0x7ff10801a0f0 2024-01-24T23:31:56.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.401+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:31:56.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.401+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10801a0f0 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:31:56.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.605+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108017b20 legacy=0x7ff108017f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:31:56.603 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:56.605+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10801a0f0 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:31:57.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:57.009+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108017b20 legacy=0x7ff108017f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:31:57.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:57.009+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10801a0f0 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:31:57.810 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:57.813+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108017b20 legacy=0x7ff108017f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:31:57.811 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:57.813+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10801a0f0 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:31:59.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:59.405+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:31:59.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:59.405+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:31:59.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:59.405+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:31:59.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:59.405+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:31:59.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:59.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10801a0f0 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:31:59.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:59.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108017b20 legacy=0x7ff108017f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:31:59.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:59.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff10800d850 legacy=0x7ff108010010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:31:59.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:59.405+0000 7ff11affd640 10 monclient: _add_conns ranks=[3,4,8] 2024-01-24T23:31:59.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:59.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-g con 0x7ff10801a9e0 addr v1:172.21.15.132:6790/0 2024-01-24T23:31:59.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:59.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-c con 0x7ff10801a550 addr v1:172.21.15.115:6791/0 2024-01-24T23:31:59.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:59.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-e con 0x7ff108007850 addr v1:172.21.15.115:6793/0 2024-01-24T23:31:59.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:59.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108006b80 con 0x7ff10801a9e0 2024-01-24T23:31:59.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:59.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1080068f0 con 0x7ff10801a550 2024-01-24T23:31:59.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:59.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108019e70 con 0x7ff108007850 2024-01-24T23:31:59.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:59.405+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:31:59.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:59.405+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff108007850 legacy=0x7ff10801d260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:31:59.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:59.405+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10801a9e0 legacy=0x7ff10800e530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:31:59.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:59.609+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff108007850 legacy=0x7ff10801d260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:31:59.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:31:59.609+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10801a9e0 legacy=0x7ff10800e530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:00.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:00.013+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff108007850 legacy=0x7ff10801d260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:00.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:00.013+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10801a9e0 legacy=0x7ff10800e530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:00.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:00.817+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff108007850 legacy=0x7ff10801d260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:00.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:00.817+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10801a9e0 legacy=0x7ff10800e530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:01.364 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:32:01.365+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:32:01.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:01.365+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801a550 legacy=0x7ff108005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:32:01.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:01.365+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801a550 legacy=0x7ff108005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:32:01.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:01.365+0000 7ff128ec8640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801a550 0x7ff108005ba0 :-1 s=OPENED pgs=65 cs=1 l=1).handle_message read tag failed 2024-01-24T23:32:01.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:01.365+0000 7ff128ec8640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801a550 0x7ff108005ba0 :-1 s=OPENED pgs=65 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:32:01.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:01.365+0000 7ff11bfff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:32:02.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:02.401+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:32:02.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:02.401+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:32:02.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:02.401+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:32:02.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:02.401+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:32:02.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:02.401+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff108007850 legacy=0x7ff10801d260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:02.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:02.401+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801a550 legacy=0x7ff108005ba0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:32:02.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:02.401+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10801a9e0 legacy=0x7ff10800e530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:02.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:02.401+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:32:02.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:02.405+0000 7ff11affd640 10 monclient: _add_conns ranks=[4,3,2,1,5,6,7,0,8] 2024-01-24T23:32:02.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:02.405+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:32:02.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:02.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-c con 0x7ff108007850 addr v1:172.21.15.115:6791/0 2024-01-24T23:32:02.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:02.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-g con 0x7ff10800d850 addr v1:172.21.15.132:6790/0 2024-01-24T23:32:02.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:02.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-b con 0x7ff10801a0f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:32:02.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:02.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108009cd0 con 0x7ff10801a0f0 2024-01-24T23:32:02.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:02.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108006b80 con 0x7ff10800d850 2024-01-24T23:32:02.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:02.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108019e70 con 0x7ff108007850 2024-01-24T23:32:02.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:02.405+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:32:02.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:02.405+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10801a0f0 legacy=0x7ff10800e530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:02.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:02.405+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10800d850 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:02.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:02.609+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10801a0f0 legacy=0x7ff10800e530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:02.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:02.609+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10800d850 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:03.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:03.013+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10801a0f0 legacy=0x7ff10800e530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:03.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:03.013+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10800d850 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:03.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:03.817+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10801a0f0 legacy=0x7ff10800e530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:03.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:03.817+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10800d850 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:05.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:05.405+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:32:05.403 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:05.405+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:32:05.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:05.405+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:32:05.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:05.405+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:32:05.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:05.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff108007850 legacy=0x7ff10800ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:32:05.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:05.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10800d850 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:05.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:05.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10801a0f0 legacy=0x7ff10800e530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:05.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:05.405+0000 7ff11affd640 10 monclient: _add_conns ranks=[7,1,6,8,0,5] 2024-01-24T23:32:05.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:05.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-i con 0x7ff10801a550 addr v1:172.21.15.132:6792/0 2024-01-24T23:32:05.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:05.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-f con 0x7ff10801a9e0 addr v1:172.21.15.132:6789/0 2024-01-24T23:32:05.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:05.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-d con 0x7ff108005bb0 addr v1:172.21.15.115:6792/0 2024-01-24T23:32:05.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:05.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108019c30 con 0x7ff10801a9e0 2024-01-24T23:32:05.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:05.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108009cd0 con 0x7ff108005bb0 2024-01-24T23:32:05.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:05.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108006b80 con 0x7ff10801a550 2024-01-24T23:32:05.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:05.405+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:32:05.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:05.405+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108005bb0 legacy=0x7ff10801f270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:05.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:05.405+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a550 legacy=0x7ff108010e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:05.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:05.405+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a9e0 legacy=0x7ff108011530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-01-24T23:32:05.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:05.405+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a9e0 legacy=0x7ff108011530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-24T23:32:05.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:05.405+0000 7ff128ec8640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a9e0 0x7ff108011530 :-1 s=CONNECTING_SEND_CONNECT_MSG pgs=0 cs=0 l=0).handle_connect_reply_1 read connect reply failed 2024-01-24T23:32:05.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:05.609+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108005bb0 legacy=0x7ff10801f270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:05.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:05.609+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a550 legacy=0x7ff108010e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:06.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:06.013+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108005bb0 legacy=0x7ff10801f270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:06.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:06.013+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a550 legacy=0x7ff108010e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:06.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:06.361+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a9e0 legacy=0x7ff108011530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:32:06.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:06.361+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a9e0 legacy=0x7ff108011530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:32:06.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:06.361+0000 7ff128ec8640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a9e0 0x7ff108011530 :-1 s=OPENED pgs=68 cs=1 l=1).handle_message read tag failed 2024-01-24T23:32:06.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:06.361+0000 7ff128ec8640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a9e0 0x7ff108011530 :-1 s=OPENED pgs=68 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:32:06.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:06.361+0000 7ff11bfff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:32:06.364 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:32:06.365+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:32:06.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:06.817+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108005bb0 legacy=0x7ff10801f270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:06.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:06.817+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a550 legacy=0x7ff108010e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:08.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:08.405+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:32:08.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:08.405+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:32:08.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:08.405+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:32:08.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:08.405+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:32:08.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:08.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a550 legacy=0x7ff108010e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:08.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:08.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108005bb0 legacy=0x7ff10801f270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:08.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:08.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a9e0 legacy=0x7ff108011530 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:32:08.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:08.405+0000 7ff11affd640 10 monclient: _add_conns ranks=[0,8,5] 2024-01-24T23:32:08.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:08.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-a con 0x7ff10801f980 addr v1:172.21.15.115:6789/0 2024-01-24T23:32:08.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:08.405+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:32:08.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:08.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-e con 0x7ff108018230 addr v1:172.21.15.115:6793/0 2024-01-24T23:32:08.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:08.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-h con 0x7ff10801fd60 addr v1:172.21.15.132:6791/0 2024-01-24T23:32:08.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:08.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1080068f0 con 0x7ff10801f980 2024-01-24T23:32:08.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:08.405+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:32:08.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:08.405+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff108018230 legacy=0x7ff108025420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:08.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:08.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108009cd0 con 0x7ff10801fd60 2024-01-24T23:32:08.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:08.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108006b80 con 0x7ff108018230 2024-01-24T23:32:08.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:08.405+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:32:08.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:08.405+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10801fd60 legacy=0x7ff108028b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:32:08.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:08.609+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff108018230 legacy=0x7ff108025420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:08.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:08.609+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10801fd60 legacy=0x7ff108028b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:32:09.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:09.013+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff108018230 legacy=0x7ff108025420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:09.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:09.013+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10801fd60 legacy=0x7ff108028b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:32:09.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:09.817+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff108018230 legacy=0x7ff108025420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:09.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:09.817+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10801fd60 legacy=0x7ff108028b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:32:11.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:11.353+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff10801f980 legacy=0x7ff108017b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:32:11.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:11.353+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff10801f980 legacy=0x7ff108017b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:32:11.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:11.353+0000 7ff1296c9640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff10801f980 0x7ff108017b20 :-1 s=OPENED pgs=77 cs=1 l=1).handle_message read tag failed 2024-01-24T23:32:11.354 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:11.353+0000 7ff1296c9640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff10801f980 0x7ff108017b20 :-1 s=OPENED pgs=77 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:32:11.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:11.353+0000 7ff11bfff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:32:11.364 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:32:11.365+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:32:11.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:11.405+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:32:11.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:11.405+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:32:11.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:11.405+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:32:11.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:11.405+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:32:11.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:11.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff108018230 legacy=0x7ff108025420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:11.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:11.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10801fd60 legacy=0x7ff108028b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:11.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:11.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff10801f980 legacy=0x7ff108017b20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:32:11.404 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:11.405+0000 7ff11affd640 10 monclient: _add_conns ranks=[0,2,5,7,4,6,3,1,8] 2024-01-24T23:32:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:11.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-a con 0x7ff10801f980 addr v1:172.21.15.115:6789/0 2024-01-24T23:32:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:11.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-b con 0x7ff10801a550 addr v1:172.21.15.115:6790/0 2024-01-24T23:32:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:11.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-h con 0x7ff10801fd60 addr v1:172.21.15.132:6791/0 2024-01-24T23:32:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:11.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108011c70 con 0x7ff10801f980 2024-01-24T23:32:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:11.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108009cd0 con 0x7ff10801a550 2024-01-24T23:32:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:11.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108006b80 con 0x7ff10801fd60 2024-01-24T23:32:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:11.405+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:32:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:11.405+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10801a550 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:11.405+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10801fd60 legacy=0x7ff10800e530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:32:11.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:11.609+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10801a550 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:11.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:11.609+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10801fd60 legacy=0x7ff10800e530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:32:12.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:12.013+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10801a550 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:12.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:12.013+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10801fd60 legacy=0x7ff10800e530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:32:12.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:12.817+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10801a550 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:12.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:12.817+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10801fd60 legacy=0x7ff10800e530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:32:14.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:14.405+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:32:14.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:14.405+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:32:14.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:14.405+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:32:14.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:14.405+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:32:14.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:14.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10801fd60 legacy=0x7ff10800e530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:14.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:14.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10801a550 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:14.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:14.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff10801f980 legacy=0x7ff10800ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:32:14.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:14.405+0000 7ff11affd640 10 monclient: _add_conns ranks=[1,8,3,6,4,7] 2024-01-24T23:32:14.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:14.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-f con 0x7ff10801a0f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:32:14.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:14.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-e con 0x7ff108018230 addr v1:172.21.15.115:6793/0 2024-01-24T23:32:14.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:14.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-g con 0x7ff108009830 addr v1:172.21.15.132:6790/0 2024-01-24T23:32:14.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:14.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10801a9e0 con 0x7ff10801a0f0 2024-01-24T23:32:14.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:14.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108009cd0 con 0x7ff108009830 2024-01-24T23:32:14.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:14.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108006b80 con 0x7ff108018230 2024-01-24T23:32:14.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:14.405+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:32:14.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:14.405+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff108018230 legacy=0x7ff10801f270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:14.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:14.405+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108009830 legacy=0x7ff1080114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:14.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:14.609+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff108018230 legacy=0x7ff10801f270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:14.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:14.609+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108009830 legacy=0x7ff1080114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:15.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:15.013+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff108018230 legacy=0x7ff10801f270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:15.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:15.013+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108009830 legacy=0x7ff1080114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:15.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:15.817+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff108018230 legacy=0x7ff10801f270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:15.817+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108009830 legacy=0x7ff1080114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:16.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:16.361+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a0f0 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:32:16.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:16.365+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a0f0 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:32:16.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:16.365+0000 7ff1296c9640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a0f0 0x7ff108010dd0 :-1 s=OPENED pgs=75 cs=1 l=1).handle_message read tag failed 2024-01-24T23:32:16.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:16.365+0000 7ff1296c9640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a0f0 0x7ff108010dd0 :-1 s=OPENED pgs=75 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:32:16.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:16.365+0000 7ff11bfff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:32:16.364 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:32:16.365+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:32:17.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:17.405+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:32:17.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:17.405+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:32:17.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:17.405+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:32:17.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:17.405+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:32:17.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:17.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff108018230 legacy=0x7ff10801f270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:17.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:17.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108009830 legacy=0x7ff1080114e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:17.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:17.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a0f0 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:32:17.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:17.405+0000 7ff11affd640 10 monclient: _add_conns ranks=[7,6,4] 2024-01-24T23:32:17.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:17.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-i con 0x7ff10800d810 addr v1:172.21.15.132:6792/0 2024-01-24T23:32:17.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:17.405+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:32:17.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:17.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-d con 0x7ff108005bb0 addr v1:172.21.15.115:6792/0 2024-01-24T23:32:17.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:17.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-c con 0x7ff108026b40 addr v1:172.21.15.115:6791/0 2024-01-24T23:32:17.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:17.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10801ab90 con 0x7ff108026b40 2024-01-24T23:32:17.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:17.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108009cd0 con 0x7ff108005bb0 2024-01-24T23:32:17.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:17.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108006b80 con 0x7ff10800d810 2024-01-24T23:32:17.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:17.405+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:32:17.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:17.405+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108005bb0 legacy=0x7ff108025420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:17.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:17.405+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:32:17.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:17.405+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10800d810 legacy=0x7ff108017b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:17.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:17.609+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108005bb0 legacy=0x7ff108025420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:17.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:17.609+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10800d810 legacy=0x7ff108017b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:18.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:18.013+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108005bb0 legacy=0x7ff108025420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:18.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:18.013+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10800d810 legacy=0x7ff108017b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:18.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:18.817+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108005bb0 legacy=0x7ff108025420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:18.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:18.817+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10800d810 legacy=0x7ff108017b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:20.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:20.405+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:32:20.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:20.405+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:32:20.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:20.405+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:32:20.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:20.405+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:32:20.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:20.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10800d810 legacy=0x7ff108017b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:20.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:20.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108005bb0 legacy=0x7ff108025420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:20.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:20.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff108026b40 legacy=0x7ff108028f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:32:20.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:20.405+0000 7ff11affd640 10 monclient: _add_conns ranks=[6,2,3,5,7,8,4,0,1] 2024-01-24T23:32:20.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:20.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-d con 0x7ff108005bb0 addr v1:172.21.15.115:6792/0 2024-01-24T23:32:20.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:20.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-b con 0x7ff10801a550 addr v1:172.21.15.115:6790/0 2024-01-24T23:32:20.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:20.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-g con 0x7ff10801a0f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:32:20.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:20.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108005fe0 con 0x7ff10801a550 2024-01-24T23:32:20.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:20.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108009cd0 con 0x7ff10801a0f0 2024-01-24T23:32:20.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:20.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108006b80 con 0x7ff108005bb0 2024-01-24T23:32:20.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:20.405+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:32:20.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:20.405+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10801a550 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:20.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:20.405+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108005bb0 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:20.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:20.405+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10801a0f0 legacy=0x7ff1080114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:20.605 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:20.609+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10801a550 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:20.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:20.609+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108005bb0 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:20.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:20.609+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10801a0f0 legacy=0x7ff1080114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:21.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:21.013+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10801a550 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:21.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:21.013+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108005bb0 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:21.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:21.013+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10801a0f0 legacy=0x7ff1080114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:21.364 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:32:21.365+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:32:21.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:21.817+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10801a550 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:21.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:21.817+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108005bb0 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:21.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:21.817+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10801a0f0 legacy=0x7ff1080114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:23.405 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:23.405+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:32:23.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:23.405+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:32:23.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:23.405+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:32:23.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:23.405+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:32:23.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:23.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108005bb0 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:23.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:23.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10801a0f0 legacy=0x7ff1080114e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:23.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:23.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10801a550 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:23.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:23.405+0000 7ff11affd640 10 monclient: _add_conns ranks=[4,1,7,8,5,0] 2024-01-24T23:32:23.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:23.405+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:32:23.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:23.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-c con 0x7ff108026b40 addr v1:172.21.15.115:6791/0 2024-01-24T23:32:23.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:23.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-f con 0x7ff10801a550 addr v1:172.21.15.132:6789/0 2024-01-24T23:32:23.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:23.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-i con 0x7ff10801a0f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:32:23.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:23.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108019240 con 0x7ff10801a550 2024-01-24T23:32:23.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:23.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108005fe0 con 0x7ff108026b40 2024-01-24T23:32:23.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:23.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108009cd0 con 0x7ff10801a0f0 2024-01-24T23:32:23.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:23.405+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:32:23.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:23.405+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a0f0 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:23.606 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:23.609+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a0f0 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:24.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:24.013+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a0f0 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:24.814 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:24.817+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a0f0 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:26.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.365+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a550 legacy=0x7ff10800e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:32:26.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.365+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a550 legacy=0x7ff10800e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:32:26.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.365+0000 7ff128ec8640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a550 0x7ff10800e530 :-1 s=OPENED pgs=82 cs=1 l=1).handle_message read tag failed 2024-01-24T23:32:26.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.365+0000 7ff128ec8640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a550 0x7ff10800e530 :-1 s=OPENED pgs=82 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:32:26.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.365+0000 7ff11bfff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:32:26.364 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:32:26.365+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:32:26.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.365+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff108026b40 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:32:26.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.365+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff108026b40 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:32:26.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.365+0000 7ff1296c9640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff108026b40 0x7ff10800de20 :-1 s=OPENED pgs=86 cs=1 l=1).handle_message read tag failed 2024-01-24T23:32:26.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.365+0000 7ff1296c9640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff108026b40 0x7ff10800de20 :-1 s=OPENED pgs=86 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:32:26.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.365+0000 7ff11bfff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:32:26.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.409+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:32:26.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.409+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:32:26.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.409+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:32:26.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.409+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:32:26.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a0f0 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:26.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff108026b40 legacy=0x7ff10800de20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:32:26.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a550 legacy=0x7ff10800e530 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:32:26.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.409+0000 7ff11affd640 10 monclient: _add_conns ranks=[5,8,0] 2024-01-24T23:32:26.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-h con 0x7ff10800d7f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:32:26.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-e con 0x7ff108005ba0 addr v1:172.21.15.115:6793/0 2024-01-24T23:32:26.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-a con 0x7ff10801f270 addr v1:172.21.15.115:6789/0 2024-01-24T23:32:26.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108006b80 con 0x7ff10801f270 2024-01-24T23:32:26.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108009cd0 con 0x7ff10800d7f0 2024-01-24T23:32:26.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108009e60 con 0x7ff108005ba0 2024-01-24T23:32:26.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.409+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:32:26.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.409+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff108005ba0 legacy=0x7ff1080114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:26.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.409+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800d7f0 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:32:26.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.613+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff108005ba0 legacy=0x7ff1080114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:26.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:26.613+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800d7f0 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:32:27.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:27.017+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff108005ba0 legacy=0x7ff1080114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:27.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:27.017+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800d7f0 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:32:27.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:27.821+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff108005ba0 legacy=0x7ff1080114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:27.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:27.821+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800d7f0 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:32:29.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:29.405+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:32:29.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:29.405+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:32:29.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:29.405+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:32:29.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:29.405+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:32:29.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:29.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff108005ba0 legacy=0x7ff1080114e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:29.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:29.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800d7f0 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:29.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:29.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff10801f270 legacy=0x7ff10801f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:32:29.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:29.405+0000 7ff11affd640 10 monclient: _add_conns ranks=[6,5,3,0,4,7,2,8,1] 2024-01-24T23:32:29.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:29.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-d con 0x7ff108018b30 addr v1:172.21.15.115:6792/0 2024-01-24T23:32:29.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:29.405+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:32:29.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:29.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-h con 0x7ff10800d7f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:32:29.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:29.405+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-g con 0x7ff108018f10 addr v1:172.21.15.132:6790/0 2024-01-24T23:32:29.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:29.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108009cd0 con 0x7ff108018f10 2024-01-24T23:32:29.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:29.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108009e60 con 0x7ff10800d7f0 2024-01-24T23:32:29.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:29.405+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:32:29.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:29.405+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108006fe0 con 0x7ff108018b30 2024-01-24T23:32:29.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:29.405+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:32:29.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:29.405+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108018b30 legacy=0x7ff108022400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:29.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:29.405+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800d7f0 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:32:29.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:29.405+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108018f10 legacy=0x7ff108025b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:29.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:29.609+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108018b30 legacy=0x7ff108022400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:29.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:29.609+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800d7f0 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:32:29.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:29.613+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108018f10 legacy=0x7ff108025b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:30.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:30.013+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108018b30 legacy=0x7ff108022400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:30.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:30.013+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800d7f0 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:32:30.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:30.017+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108018f10 legacy=0x7ff108025b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:30.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:30.817+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108018b30 legacy=0x7ff108022400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:30.815 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:30.817+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800d7f0 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:32:30.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:30.821+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108018f10 legacy=0x7ff108025b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:31.365 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:32:31.365+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:32:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:32.409+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:32:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:32.409+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:32:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:32.409+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:32:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:32.409+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:32:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:32.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108018b30 legacy=0x7ff108022400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:32.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800d7f0 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:32.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108018f10 legacy=0x7ff108025b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:32.409+0000 7ff11affd640 10 monclient: _add_conns ranks=[7,0,1,2,4,8] 2024-01-24T23:32:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:32.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-i con 0x7ff10801a0f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:32:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:32.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-a con 0x7ff10801a550 addr v1:172.21.15.115:6789/0 2024-01-24T23:32:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:32.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-f con 0x7ff108005ba0 addr v1:172.21.15.132:6789/0 2024-01-24T23:32:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:32.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108019450 con 0x7ff10801a550 2024-01-24T23:32:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:32.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108009cd0 con 0x7ff108005ba0 2024-01-24T23:32:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:32.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108009e60 con 0x7ff10801a0f0 2024-01-24T23:32:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:32.409+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:32:32.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:32.409+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a0f0 legacy=0x7ff1080114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:32.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:32.613+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a0f0 legacy=0x7ff1080114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:33.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:33.017+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a0f0 legacy=0x7ff1080114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:33.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:33.821+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a0f0 legacy=0x7ff1080114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:35.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:35.409+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:32:35.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:35.409+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:32:35.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:35.409+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:32:35.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:35.409+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:32:35.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:35.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a0f0 legacy=0x7ff1080114e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:35.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:35.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff108005ba0 legacy=0x7ff10801f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:32:35.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:35.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff10801a550 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:32:35.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:35.409+0000 7ff11affd640 10 monclient: _add_conns ranks=[2,8,4] 2024-01-24T23:32:35.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:35.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-b con 0x7ff108026240 addr v1:172.21.15.115:6790/0 2024-01-24T23:32:35.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:35.409+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:32:35.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:35.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-e con 0x7ff10800de20 addr v1:172.21.15.115:6793/0 2024-01-24T23:32:35.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:35.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-c con 0x7ff10801f250 addr v1:172.21.15.115:6791/0 2024-01-24T23:32:35.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:35.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108006fe0 con 0x7ff108026240 2024-01-24T23:32:35.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:35.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108009e60 con 0x7ff10801f250 2024-01-24T23:32:35.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:35.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10800e9a0 con 0x7ff10800de20 2024-01-24T23:32:35.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:35.409+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:32:35.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:35.409+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108026240 legacy=0x7ff108026650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:35.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:35.409+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10800de20 legacy=0x7ff10800e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:35.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:35.409+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:32:35.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:35.613+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108026240 legacy=0x7ff108026650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:35.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:35.613+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10800de20 legacy=0x7ff10800e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:36.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:36.017+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10800de20 legacy=0x7ff10800e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:36.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:36.017+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108026240 legacy=0x7ff108026650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:36.365 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:32:36.365+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:32:36.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:36.365+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801f250 legacy=0x7ff108029f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:32:36.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:36.365+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801f250 legacy=0x7ff108029f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:32:36.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:36.365+0000 7ff129eca640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801f250 0x7ff108029f40 :-1 s=OPENED pgs=87 cs=1 l=1).handle_message read tag failed 2024-01-24T23:32:36.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:36.365+0000 7ff129eca640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801f250 0x7ff108029f40 :-1 s=OPENED pgs=87 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:32:36.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:36.365+0000 7ff11bfff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:32:36.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:36.821+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108026240 legacy=0x7ff108026650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:36.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:36.821+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10800de20 legacy=0x7ff10800e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:38.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:38.409+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:32:38.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:38.409+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:32:38.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:38.409+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:32:38.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:38.409+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:32:38.407 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:38.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10800de20 legacy=0x7ff10800e290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:38.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:38.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801f250 legacy=0x7ff108029f40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:32:38.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:38.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108026240 legacy=0x7ff108026650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:38.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:38.409+0000 7ff11affd640 10 monclient: _add_conns ranks=[1,5,7,4,8,2,0,6,3] 2024-01-24T23:32:38.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:38.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-f con 0x7ff10801a550 addr v1:172.21.15.132:6789/0 2024-01-24T23:32:38.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:38.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-h con 0x7ff10800d7f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:32:38.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:38.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-i con 0x7ff10801a0f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:32:38.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:38.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10801fdf0 con 0x7ff10801a550 2024-01-24T23:32:38.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:38.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108006fe0 con 0x7ff10800d7f0 2024-01-24T23:32:38.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:38.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10800e9a0 con 0x7ff10801a0f0 2024-01-24T23:32:38.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:38.409+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:32:38.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:38.409+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a0f0 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:38.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:38.409+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800d7f0 legacy=0x7ff108018b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:32:38.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:38.613+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800d7f0 legacy=0x7ff108018b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:32:38.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:38.613+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a0f0 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:39.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:39.017+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a0f0 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:39.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:39.017+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800d7f0 legacy=0x7ff108018b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:32:39.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:39.821+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800d7f0 legacy=0x7ff108018b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:32:39.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:39.821+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a0f0 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:41.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.365+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a550 legacy=0x7ff10801f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:32:41.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.365+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a550 legacy=0x7ff10801f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:32:41.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.365+0000 7ff1296c9640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a550 0x7ff10801f6e0 :-1 s=OPENED pgs=90 cs=1 l=1).handle_message read tag failed 2024-01-24T23:32:41.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.365+0000 7ff1296c9640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a550 0x7ff10801f6e0 :-1 s=OPENED pgs=90 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:32:41.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.365+0000 7ff11bfff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:32:41.365 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:32:41.365+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:32:41.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.409+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:32:41.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.409+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:32:41.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.409+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:32:41.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.409+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:32:41.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a0f0 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:41.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800d7f0 legacy=0x7ff108018b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:41.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801a550 legacy=0x7ff10801f6e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:32:41.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.409+0000 7ff11affd640 10 monclient: _add_conns ranks=[3,0,2,8,4,6] 2024-01-24T23:32:41.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.409+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:32:41.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-g con 0x7ff108010dd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:32:41.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-a con 0x7ff108005ba0 addr v1:172.21.15.115:6789/0 2024-01-24T23:32:41.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-b con 0x7ff108026240 addr v1:172.21.15.115:6790/0 2024-01-24T23:32:41.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10801ff80 con 0x7ff108005ba0 2024-01-24T23:32:41.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108006fe0 con 0x7ff108026240 2024-01-24T23:32:41.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.409+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:32:41.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10800e9a0 con 0x7ff108010dd0 2024-01-24T23:32:41.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.409+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:32:41.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.409+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108026240 legacy=0x7ff108026650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:41.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.409+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108010dd0 legacy=0x7ff1080111b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:41.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.613+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108026240 legacy=0x7ff108026650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:41.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:41.613+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108010dd0 legacy=0x7ff1080111b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:42.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:42.017+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108026240 legacy=0x7ff108026650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:42.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:42.017+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108010dd0 legacy=0x7ff1080111b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:42.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:42.821+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108026240 legacy=0x7ff108026650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:42.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:42.821+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108010dd0 legacy=0x7ff1080111b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:44.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:44.409+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:32:44.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:44.409+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:32:44.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:44.409+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:32:44.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:44.409+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:32:44.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:44.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108010dd0 legacy=0x7ff1080111b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:44.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:44.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108026240 legacy=0x7ff108026650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:44.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:44.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff108005ba0 legacy=0x7ff108008830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:32:44.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:44.409+0000 7ff11affd640 10 monclient: _add_conns ranks=[8,4,6] 2024-01-24T23:32:44.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:44.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-e con 0x7ff10801a550 addr v1:172.21.15.115:6793/0 2024-01-24T23:32:44.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:44.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-c con 0x7ff10801a0d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:32:44.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:44.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-d con 0x7ff10800de20 addr v1:172.21.15.115:6792/0 2024-01-24T23:32:44.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:44.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1080118c0 con 0x7ff10801a0d0 2024-01-24T23:32:44.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:44.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108006fe0 con 0x7ff10800de20 2024-01-24T23:32:44.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:44.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10800e9a0 con 0x7ff10801a550 2024-01-24T23:32:44.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:44.409+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:32:44.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:44.409+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10801a550 legacy=0x7ff108018b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:44.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:44.409+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800de20 legacy=0x7ff10801f6e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:44.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:44.613+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800de20 legacy=0x7ff10801f6e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:44.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:44.613+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10801a550 legacy=0x7ff108018b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:45.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:45.017+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10801a550 legacy=0x7ff108018b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:45.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:45.017+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800de20 legacy=0x7ff10801f6e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:45.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:45.821+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800de20 legacy=0x7ff10801f6e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:45.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:45.821+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10801a550 legacy=0x7ff108018b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:46.365 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:32:46.365+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:32:46.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:46.365+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801a0d0 legacy=0x7ff10800e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:32:46.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:46.365+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801a0d0 legacy=0x7ff10800e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:32:46.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:46.365+0000 7ff128ec8640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801a0d0 0x7ff10800e290 :-1 s=OPENED pgs=94 cs=1 l=1).handle_message read tag failed 2024-01-24T23:32:46.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:46.365+0000 7ff128ec8640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801a0d0 0x7ff10800e290 :-1 s=OPENED pgs=94 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:32:46.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:46.365+0000 7ff11bfff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:32:47.408 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:47.409+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:32:47.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:47.409+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:32:47.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:47.409+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:32:47.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:47.409+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:32:47.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:47.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10801a550 legacy=0x7ff108018b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:47.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:47.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800de20 legacy=0x7ff10801f6e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:47.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:47.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801a0d0 legacy=0x7ff10800e290 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:32:47.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:47.409+0000 7ff11affd640 10 monclient: _add_conns ranks=[3,5,4,0,6,2,1,8,7] 2024-01-24T23:32:47.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:47.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-g con 0x7ff108010dd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:32:47.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:47.409+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:32:47.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:47.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-h con 0x7ff10800d7f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:32:47.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:47.409+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:32:47.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:47.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-c con 0x7ff108005ba0 addr v1:172.21.15.115:6791/0 2024-01-24T23:32:47.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:47.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10801ff80 con 0x7ff108010dd0 2024-01-24T23:32:47.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:47.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108006fe0 con 0x7ff108005ba0 2024-01-24T23:32:47.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:47.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10800e9a0 con 0x7ff10800d7f0 2024-01-24T23:32:47.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:47.409+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:32:47.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:47.409+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108010dd0 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:47.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:47.409+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800d7f0 legacy=0x7ff1080111b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:32:47.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:47.613+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108010dd0 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:47.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:47.613+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800d7f0 legacy=0x7ff1080111b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:32:48.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:48.017+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108010dd0 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:48.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:48.017+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800d7f0 legacy=0x7ff1080111b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:32:48.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:48.821+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108010dd0 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:48.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:48.821+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800d7f0 legacy=0x7ff1080111b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:32:50.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:50.409+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:32:50.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:50.409+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:32:50.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:50.409+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:32:50.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:50.409+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:32:50.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:50.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800d7f0 legacy=0x7ff1080111b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:50.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:50.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff108005ba0 legacy=0x7ff10800e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:32:50.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:50.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108010dd0 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:50.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:50.409+0000 7ff11affd640 10 monclient: _add_conns ranks=[0,2,7,6,8,1] 2024-01-24T23:32:50.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:50.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-a con 0x7ff10801a550 addr v1:172.21.15.115:6789/0 2024-01-24T23:32:50.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:50.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-b con 0x7ff10800de20 addr v1:172.21.15.115:6790/0 2024-01-24T23:32:50.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:50.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-i con 0x7ff10801a090 addr v1:172.21.15.132:6792/0 2024-01-24T23:32:50.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:50.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1080118c0 con 0x7ff10801a550 2024-01-24T23:32:50.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:50.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10801ff80 con 0x7ff10800de20 2024-01-24T23:32:50.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:50.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10800e9a0 con 0x7ff10801a090 2024-01-24T23:32:50.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:50.409+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:32:50.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:50.409+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10800de20 legacy=0x7ff10801b220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:50.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:50.409+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a090 legacy=0x7ff108018b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:50.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:50.613+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10800de20 legacy=0x7ff10801b220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:50.609 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:50.613+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a090 legacy=0x7ff108018b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:51.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:51.017+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10800de20 legacy=0x7ff10801b220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:51.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:51.017+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a090 legacy=0x7ff108018b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:51.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:51.357+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff10801a550 legacy=0x7ff10801f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:32:51.355 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:51.357+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff10801a550 legacy=0x7ff10801f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:32:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:51.357+0000 7ff1296c9640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff10801a550 0x7ff10801f6e0 :-1 s=OPENED pgs=94 cs=1 l=1).handle_message read tag failed 2024-01-24T23:32:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:51.357+0000 7ff1296c9640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff10801a550 0x7ff10801f6e0 :-1 s=OPENED pgs=94 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:32:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:51.357+0000 7ff11bfff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:32:51.365 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:32:51.365+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:32:51.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:51.821+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10800de20 legacy=0x7ff10801b220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:51.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:51.821+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a090 legacy=0x7ff108018b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:53.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:53.409+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:32:53.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:53.409+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:32:53.409 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:53.409+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:32:53.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:53.409+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:32:53.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:53.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a090 legacy=0x7ff108018b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:53.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:53.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10800de20 legacy=0x7ff10801b220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:53.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:53.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff10801a550 legacy=0x7ff10801f6e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:32:53.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:53.409+0000 7ff11affd640 10 monclient: _add_conns ranks=[1,8,6] 2024-01-24T23:32:53.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:53.409+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:32:53.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:53.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-f con 0x7ff108009840 addr v1:172.21.15.132:6789/0 2024-01-24T23:32:53.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:53.409+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:32:53.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:53.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-e con 0x7ff10801a090 addr v1:172.21.15.115:6793/0 2024-01-24T23:32:53.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:53.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-d con 0x7ff10800d7f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:32:53.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:53.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108006fe0 con 0x7ff108009840 2024-01-24T23:32:53.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:53.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10801ff80 con 0x7ff10800d7f0 2024-01-24T23:32:53.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:53.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10800e9a0 con 0x7ff10801a090 2024-01-24T23:32:53.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:53.409+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:32:53.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:53.409+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10801a090 legacy=0x7ff10800e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:53.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:53.409+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800d7f0 legacy=0x7ff10801b220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:53.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:53.613+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10801a090 legacy=0x7ff10800e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:53.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:53.613+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800d7f0 legacy=0x7ff10801b220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:54.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:54.017+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10801a090 legacy=0x7ff10800e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:54.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:54.017+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800d7f0 legacy=0x7ff10801b220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:54.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:54.821+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800d7f0 legacy=0x7ff10801b220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:54.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:54.821+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10801a090 legacy=0x7ff10800e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:32:56.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:56.365+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff108009840 legacy=0x7ff10801b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:32:56.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:56.365+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff108009840 legacy=0x7ff10801b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:32:56.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:56.365+0000 7ff1296c9640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff108009840 0x7ff10801b930 :-1 s=OPENED pgs=97 cs=1 l=1).handle_message read tag failed 2024-01-24T23:32:56.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:56.365+0000 7ff1296c9640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff108009840 0x7ff10801b930 :-1 s=OPENED pgs=97 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:32:56.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:56.365+0000 7ff11bfff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:32:56.365 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:32:56.365+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:32:56.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:56.409+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:32:56.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:56.409+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:32:56.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:56.409+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:32:56.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:56.409+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:32:56.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:56.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10801a090 legacy=0x7ff10800e270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:56.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:56.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800d7f0 legacy=0x7ff10801b220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:56.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:56.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff108009840 legacy=0x7ff10801b930 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:32:56.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:56.409+0000 7ff11affd640 10 monclient: _add_conns ranks=[3,7,0,2,5,6,8,4,1] 2024-01-24T23:32:56.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:56.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-g con 0x7ff10800de20 addr v1:172.21.15.132:6790/0 2024-01-24T23:32:56.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:56.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-i con 0x7ff10801a550 addr v1:172.21.15.132:6792/0 2024-01-24T23:32:56.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:56.409+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-a con 0x7ff108005ba0 addr v1:172.21.15.115:6789/0 2024-01-24T23:32:56.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:56.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108019470 con 0x7ff108005ba0 2024-01-24T23:32:56.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:56.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10801ff80 con 0x7ff10800de20 2024-01-24T23:32:56.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:56.409+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10800e9a0 con 0x7ff10801a550 2024-01-24T23:32:56.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:56.409+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:32:56.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:56.409+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a550 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:56.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:56.409+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10800de20 legacy=0x7ff10801f6e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:56.610 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:56.613+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a550 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:56.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:56.613+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10800de20 legacy=0x7ff10801f6e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:57.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:57.017+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a550 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:57.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:57.017+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10800de20 legacy=0x7ff10801f6e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:57.818 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:57.821+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a550 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:32:57.819 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:57.821+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10800de20 legacy=0x7ff10801f6e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:32:59.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:59.412+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:32:59.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:59.412+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:32:59.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:59.412+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:32:59.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:59.412+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:32:59.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:59.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10801a550 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:59.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:59.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff10800de20 legacy=0x7ff10801f6e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:32:59.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:59.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff108005ba0 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:32:59.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:59.412+0000 7ff11affd640 10 monclient: _add_conns ranks=[6,2,1,5,4,8] 2024-01-24T23:32:59.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:59.412+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:32:59.410 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:59.412+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-d con 0x7ff10800d7f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:32:59.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:59.412+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:32:59.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:59.412+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-b con 0x7ff108009840 addr v1:172.21.15.115:6790/0 2024-01-24T23:32:59.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:59.412+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-f con 0x7ff10800de20 addr v1:172.21.15.132:6789/0 2024-01-24T23:32:59.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:59.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108006fa0 con 0x7ff10800de20 2024-01-24T23:32:59.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:59.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10801ff80 con 0x7ff108009840 2024-01-24T23:32:59.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:59.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10800e9a0 con 0x7ff10800d7f0 2024-01-24T23:32:59.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:59.412+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:32:59.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:59.412+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800d7f0 legacy=0x7ff10801b220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:59.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:59.412+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108009840 legacy=0x7ff10801f6e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:32:59.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:59.616+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800d7f0 legacy=0x7ff10801b220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:32:59.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:32:59.616+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108009840 legacy=0x7ff10801f6e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:00.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:00.020+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800d7f0 legacy=0x7ff10801b220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:00.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:00.020+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108009840 legacy=0x7ff10801f6e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:00.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:00.824+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108009840 legacy=0x7ff10801f6e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:00.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:00.824+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800d7f0 legacy=0x7ff10801b220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:01.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:01.364+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10800de20 legacy=0x7ff10800e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:33:01.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:01.364+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10800de20 legacy=0x7ff10800e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:33:01.363 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:01.364+0000 7ff129eca640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10800de20 0x7ff10800e250 :-1 s=OPENED pgs=98 cs=1 l=1).handle_message read tag failed 2024-01-24T23:33:01.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:01.364+0000 7ff129eca640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10800de20 0x7ff10800e250 :-1 s=OPENED pgs=98 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:33:01.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:01.364+0000 7ff11bfff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:33:01.365 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:33:01.364+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:33:02.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:02.412+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:33:02.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:02.412+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:33:02.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:02.412+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:33:02.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:02.412+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:33:02.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:02.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800d7f0 legacy=0x7ff10801b220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:02.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:02.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff108009840 legacy=0x7ff10801f6e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:02.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:02.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10800de20 legacy=0x7ff10800e250 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:33:02.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:02.412+0000 7ff11affd640 10 monclient: _add_conns ranks=[4,5,8] 2024-01-24T23:33:02.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:02.412+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-c con 0x7ff10801a090 addr v1:172.21.15.115:6791/0 2024-01-24T23:33:02.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:02.412+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-h con 0x7ff10801a550 addr v1:172.21.15.132:6791/0 2024-01-24T23:33:02.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:02.412+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-e con 0x7ff1080114e0 addr v1:172.21.15.115:6793/0 2024-01-24T23:33:02.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:02.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108005bb0 con 0x7ff10801a090 2024-01-24T23:33:02.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:02.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10801ff80 con 0x7ff10801a550 2024-01-24T23:33:02.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:02.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10800e9a0 con 0x7ff1080114e0 2024-01-24T23:33:02.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:02.412+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:33:02.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:02.412+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff1080114e0 legacy=0x7ff108018b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:33:02.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:02.412+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10801a550 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:33:02.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:02.616+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff1080114e0 legacy=0x7ff108018b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:33:02.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:02.616+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10801a550 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:33:03.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:03.020+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff1080114e0 legacy=0x7ff108018b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:33:03.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:03.020+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10801a550 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:33:03.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:03.824+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff1080114e0 legacy=0x7ff108018b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:33:03.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:03.824+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10801a550 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:33:05.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:05.412+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:33:05.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:05.412+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:33:05.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:05.412+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:33:05.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:05.412+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:33:05.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:05.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff1080114e0 legacy=0x7ff108018b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:05.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:05.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10801a550 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:05.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:05.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801a090 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:33:05.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:05.412+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:33:05.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:05.412+0000 7ff11affd640 10 monclient: _add_conns ranks=[8,5,2,3,6,1,4,7,0] 2024-01-24T23:33:05.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:05.412+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-e con 0x7ff10800d7f0 addr v1:172.21.15.115:6793/0 2024-01-24T23:33:05.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:05.412+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-h con 0x7ff10800de20 addr v1:172.21.15.132:6791/0 2024-01-24T23:33:05.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:05.412+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-b con 0x7ff10801a550 addr v1:172.21.15.115:6790/0 2024-01-24T23:33:05.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:05.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108006fa0 con 0x7ff10801a550 2024-01-24T23:33:05.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:05.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10801ff80 con 0x7ff10800de20 2024-01-24T23:33:05.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:05.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10800e9a0 con 0x7ff10800d7f0 2024-01-24T23:33:05.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:05.412+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:33:05.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:05.412+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10800d7f0 legacy=0x7ff10801f6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:33:05.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:05.412+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10801a550 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:05.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:05.412+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800de20 legacy=0x7ff10800e250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:33:05.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:05.616+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10800d7f0 legacy=0x7ff10801f6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:33:05.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:05.616+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10801a550 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:05.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:05.616+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800de20 legacy=0x7ff10800e250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:33:06.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:06.020+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10800d7f0 legacy=0x7ff10801f6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:33:06.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:06.020+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10801a550 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:06.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:06.020+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800de20 legacy=0x7ff10800e250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:33:06.366 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:33:06.368+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:33:06.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:06.824+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10800d7f0 legacy=0x7ff10801f6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:33:06.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:06.824+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10801a550 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:06.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:06.824+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800de20 legacy=0x7ff10800e250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:33:08.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:08.412+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:33:08.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:08.412+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:33:08.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:08.412+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:33:08.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:08.412+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:33:08.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:08.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10800d7f0 legacy=0x7ff10801f6c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:08.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:08.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff10800de20 legacy=0x7ff10800e250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:08.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:08.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10801a550 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:08.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:08.412+0000 7ff11affd640 10 monclient: _add_conns ranks=[7,0,4,3,6,1] 2024-01-24T23:33:08.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:08.412+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-i con 0x7ff108010dd0 addr v1:172.21.15.132:6792/0 2024-01-24T23:33:08.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:08.412+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-a con 0x7ff108009840 addr v1:172.21.15.115:6789/0 2024-01-24T23:33:08.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:08.412+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-c con 0x7ff10801f240 addr v1:172.21.15.115:6791/0 2024-01-24T23:33:08.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:08.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108019230 con 0x7ff108009840 2024-01-24T23:33:08.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:08.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108006fa0 con 0x7ff10801f240 2024-01-24T23:33:08.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:08.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10801ff80 con 0x7ff108010dd0 2024-01-24T23:33:08.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:08.412+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:33:08.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:08.412+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff108010dd0 legacy=0x7ff1080111b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:33:08.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:08.616+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff108010dd0 legacy=0x7ff1080111b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:33:09.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:09.020+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff108010dd0 legacy=0x7ff1080111b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:33:09.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:09.824+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff108010dd0 legacy=0x7ff1080111b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:33:11.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.356+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff108009840 legacy=0x7ff108014b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:33:11.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.356+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff108009840 legacy=0x7ff108014b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:33:11.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.356+0000 7ff128ec8640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff108009840 0x7ff108014b00 :-1 s=OPENED pgs=109 cs=1 l=1).handle_message read tag failed 2024-01-24T23:33:11.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.356+0000 7ff128ec8640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff108009840 0x7ff108014b00 :-1 s=OPENED pgs=109 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:33:11.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.356+0000 7ff11bfff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:33:11.366 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:33:11.364+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:33:11.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.364+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801f240 legacy=0x7ff108018230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:33:11.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.364+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801f240 legacy=0x7ff108018230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:33:11.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.364+0000 7ff129eca640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801f240 0x7ff108018230 :-1 s=OPENED pgs=109 cs=1 l=1).handle_message read tag failed 2024-01-24T23:33:11.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.364+0000 7ff129eca640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801f240 0x7ff108018230 :-1 s=OPENED pgs=109 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:33:11.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.368+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:33:11.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.368+0000 7ff11bfff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:33:11.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.412+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:33:11.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.412+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:33:11.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.412+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:33:11.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.412+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:33:11.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff108010dd0 legacy=0x7ff1080111b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:11.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10801f240 legacy=0x7ff108018230 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:33:11.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff108009840 legacy=0x7ff108014b00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:33:11.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.412+0000 7ff11affd640 10 monclient: _add_conns ranks=[3,1,6] 2024-01-24T23:33:11.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.412+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-g con 0x7ff108009840 addr v1:172.21.15.132:6790/0 2024-01-24T23:33:11.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.412+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-f con 0x7ff10801f240 addr v1:172.21.15.132:6789/0 2024-01-24T23:33:11.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.412+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-d con 0x7ff10800d7f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:33:11.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10800e9a0 con 0x7ff10801f240 2024-01-24T23:33:11.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10801ff80 con 0x7ff108009840 2024-01-24T23:33:11.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108006fa0 con 0x7ff10800d7f0 2024-01-24T23:33:11.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.412+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:33:11.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.412+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800d7f0 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:11.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.412+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108009840 legacy=0x7ff10801f6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:33:11.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.616+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800d7f0 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:11.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:11.616+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108009840 legacy=0x7ff10801f6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:33:12.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:12.020+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800d7f0 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:12.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:12.020+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108009840 legacy=0x7ff10801f6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:33:12.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:12.824+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800d7f0 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:12.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:12.824+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108009840 legacy=0x7ff10801f6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:33:14.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:14.412+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:33:14.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:14.412+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:33:14.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:14.412+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:33:14.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:14.412+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:33:14.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:14.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff10800d7f0 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:14.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:14.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108009840 legacy=0x7ff10801f6c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:14.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:14.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff10801f240 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:33:14.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:14.412+0000 7ff11affd640 10 monclient: _add_conns ranks=[4,0,3,8,1,5,7,6,2] 2024-01-24T23:33:14.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:14.412+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-c con 0x7ff10800e530 addr v1:172.21.15.115:6791/0 2024-01-24T23:33:14.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:14.412+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-a con 0x7ff10801a550 addr v1:172.21.15.115:6789/0 2024-01-24T23:33:14.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:14.412+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-g con 0x7ff108009840 addr v1:172.21.15.132:6790/0 2024-01-24T23:33:14.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:14.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108005bb0 con 0x7ff10801a550 2024-01-24T23:33:14.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:14.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10801ff80 con 0x7ff108009840 2024-01-24T23:33:14.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:14.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108006fa0 con 0x7ff10800e530 2024-01-24T23:33:14.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:14.412+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:33:14.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:14.412+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108009840 legacy=0x7ff10801f6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:33:14.613 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:14.616+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108009840 legacy=0x7ff10801f6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:33:15.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:15.020+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108009840 legacy=0x7ff10801f6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:33:15.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:15.824+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108009840 legacy=0x7ff10801f6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:33:16.356 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:16.360+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff10801a550 legacy=0x7ff108018230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:33:16.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:16.360+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff10801a550 legacy=0x7ff108018230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:33:16.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:16.360+0000 7ff128ec8640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff10801a550 0x7ff108018230 :-1 s=OPENED pgs=110 cs=1 l=1).handle_message read tag failed 2024-01-24T23:33:16.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:16.360+0000 7ff128ec8640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff10801a550 0x7ff108018230 :-1 s=OPENED pgs=110 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:33:16.357 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:16.360+0000 7ff11bfff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:33:16.366 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:33:16.368+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:33:16.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:16.368+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10800e530 legacy=0x7ff108014b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:33:16.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:16.368+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10800e530 legacy=0x7ff108014b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:33:16.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:16.368+0000 7ff1296c9640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10800e530 0x7ff108014b00 :-1 s=OPENED pgs=110 cs=1 l=1).handle_message read tag failed 2024-01-24T23:33:16.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:16.368+0000 7ff1296c9640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10800e530 0x7ff108014b00 :-1 s=OPENED pgs=110 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:33:16.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:16.368+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:33:16.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:16.368+0000 7ff11bfff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:33:17.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:17.416+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:33:17.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:17.416+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:33:17.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:17.416+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:33:17.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:17.416+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:33:17.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:17.416+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6791/0 conn(0x7ff10800e530 legacy=0x7ff108014b00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:33:17.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:17.416+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6790/0 conn(0x7ff108009840 legacy=0x7ff10801f6c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:17.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:17.416+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6789/0 conn(0x7ff10801a550 legacy=0x7ff108018230 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:33:17.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:17.416+0000 7ff11affd640 10 monclient: _add_conns ranks=[8,7,2,6,1,5] 2024-01-24T23:33:17.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:17.416+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-e con 0x7ff10801a550 addr v1:172.21.15.115:6793/0 2024-01-24T23:33:17.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:17.416+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-i con 0x7ff10800e530 addr v1:172.21.15.132:6792/0 2024-01-24T23:33:17.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:17.416+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-b con 0x7ff10800d7f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:33:17.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:17.416+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1080106a0 con 0x7ff10800d7f0 2024-01-24T23:33:17.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:17.416+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10801ff80 con 0x7ff10800e530 2024-01-24T23:33:17.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:17.416+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108006fa0 con 0x7ff10801a550 2024-01-24T23:33:17.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:17.416+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:33:17.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:17.416+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10801a550 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:33:17.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:17.416+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10800e530 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:33:17.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:17.416+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10800d7f0 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:17.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:17.620+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10800d7f0 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:17.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:17.620+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10801a550 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:33:17.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:17.620+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10800e530 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:33:18.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:18.024+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10800d7f0 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:18.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:18.024+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10801a550 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:33:18.023 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:18.024+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10800e530 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:33:18.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:18.828+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10800d7f0 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:18.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:18.828+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10801a550 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:33:18.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:18.828+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10800e530 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:33:20.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:20.412+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:33:20.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:20.412+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:33:20.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:20.412+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:33:20.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:20.412+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:33:20.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:20.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6793/0 conn(0x7ff10801a550 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:20.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:20.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6792/0 conn(0x7ff10800e530 legacy=0x7ff10800ff90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:20.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:20.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10800d7f0 legacy=0x7ff108010dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:20.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:20.412+0000 7ff11affd640 10 monclient: _add_conns ranks=[1,6,5] 2024-01-24T23:33:20.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:20.412+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-f con 0x7ff108005bb0 addr v1:172.21.15.132:6789/0 2024-01-24T23:33:20.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:20.412+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-d con 0x7ff108015b10 addr v1:172.21.15.115:6792/0 2024-01-24T23:33:20.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:20.412+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-h con 0x7ff108018640 addr v1:172.21.15.132:6791/0 2024-01-24T23:33:20.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:20.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10801f250 con 0x7ff108005bb0 2024-01-24T23:33:20.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:20.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1080106a0 con 0x7ff108018640 2024-01-24T23:33:20.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:20.412+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10801ff80 con 0x7ff108015b10 2024-01-24T23:33:20.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:20.412+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:33:20.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:20.412+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108015b10 legacy=0x7ff108017f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:20.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:20.412+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff108018640 legacy=0x7ff108018a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:33:20.614 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:20.616+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108015b10 legacy=0x7ff108017f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:20.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:20.616+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff108018640 legacy=0x7ff108018a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:33:21.018 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:21.020+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108015b10 legacy=0x7ff108017f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:21.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:21.020+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff108018640 legacy=0x7ff108018a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:33:21.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:21.364+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff108005bb0 legacy=0x7ff1080114e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:33:21.364 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:21.364+0000 7ff1296c9640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff108005bb0 legacy=0x7ff1080114e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:33:21.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:21.364+0000 7ff1296c9640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff108005bb0 0x7ff1080114e0 :-1 s=OPENED pgs=106 cs=1 l=1).handle_message read tag failed 2024-01-24T23:33:21.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:21.364+0000 7ff1296c9640 1 --1- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff108005bb0 0x7ff1080114e0 :-1 s=OPENED pgs=106 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:33:21.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:21.364+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 reap_dead start 2024-01-24T23:33:21.365 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:21.364+0000 7ff11bfff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:33:21.366 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:33:21.368+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:33:21.822 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:21.824+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108015b10 legacy=0x7ff108017f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:21.823 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:21.824+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff108018640 legacy=0x7ff108018a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:33:23.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:23.416+0000 7ff11affd640 10 monclient(hunting): tick 2024-01-24T23:33:23.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:23.416+0000 7ff11affd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:33:23.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:23.416+0000 7ff11affd640 1 monclient(hunting): continuing hunt 2024-01-24T23:33:23.413 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:23.416+0000 7ff11affd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:33:23.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:23.416+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108015b10 legacy=0x7ff108017f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:23.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:23.416+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6791/0 conn(0x7ff108018640 legacy=0x7ff108018a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:23.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:23.416+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.132:6789/0 conn(0x7ff108005bb0 legacy=0x7ff1080114e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:33:23.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:23.416+0000 7ff11affd640 10 monclient: _add_conns ranks=[4,2,6,8,1,3,0,5,7] 2024-01-24T23:33:23.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:23.416+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-c con 0x7ff108005bb0 addr v1:172.21.15.115:6791/0 2024-01-24T23:33:23.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:23.416+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-b con 0x7ff10801a520 addr v1:172.21.15.115:6790/0 2024-01-24T23:33:23.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:23.416+0000 7ff11affd640 10 monclient(hunting): picked mon.noname-d con 0x7ff108015b10 addr v1:172.21.15.115:6792/0 2024-01-24T23:33:23.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:23.416+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108006fa0 con 0x7ff10801a520 2024-01-24T23:33:23.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:23.416+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1080106a0 con 0x7ff108005bb0 2024-01-24T23:33:23.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:23.416+0000 7ff11affd640 1 -- 172.21.15.115:0/915557570 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10801ff80 con 0x7ff108015b10 2024-01-24T23:33:23.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:23.416+0000 7ff11affd640 10 monclient(hunting): _renew_subs 2024-01-24T23:33:23.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:23.416+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10801a520 legacy=0x7ff10801f6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:23.414 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:23.416+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108015b10 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:23.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:23.620+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108015b10 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:23.618 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:23.620+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10801a520 legacy=0x7ff10801f6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:24.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:24.024+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108015b10 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:24.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:24.024+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10801a520 legacy=0x7ff10801f6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:24.826 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:24.828+0000 7ff128ec8640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6790/0 conn(0x7ff10801a520 legacy=0x7ff10801f6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:24.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:33:24.828+0000 7ff129eca640 1 -- 172.21.15.115:0/915557570 >> v1:172.21.15.115:6792/0 conn(0x7ff108015b10 legacy=0x7ff10800de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:26.288 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-01-24T23:33:26.289 ERROR:tasks.mon_thrash.mon_thrasher:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/mon_thrash.py", line 252, in do_thrash self._do_thrash() File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/mon_thrash.py", line 352, in _do_thrash self.manager.wait_for_mon_quorum_size(len(mons)) File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 3172, in wait_for_mon_quorum_size if len(self.get_mon_quorum()) == size: File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 3157, in get_mon_quorum out = self.raw_cluster_cmd('quorum_status') File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 1627, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 1618, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi115 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status' 2024-01-24T23:33:26.367 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:33:26.368+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:33:31.367 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:33:31.368+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:33:35.046 INFO:tasks.workunit.client.0.smithi132.stderr:+ ret=110 2024-01-24T23:33:35.046 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ 110 -ne 110 ]] 2024-01-24T23:33:35.046 INFO:tasks.workunit.client.0.smithi132.stderr:+ return 0 2024-01-24T23:33:35.047 INFO:tasks.workunit.client.0.smithi132.stderr:+ export CEPH_ARGS= 2024-01-24T23:33:35.047 INFO:tasks.workunit.client.0.smithi132.stderr:+ CEPH_ARGS= 2024-01-24T23:33:35.047 INFO:tasks.workunit.client.0.smithi132.stderr:+ ceph auth del client.foo 2024-01-24T23:33:35.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.158+0000 7f08299a4640 10 monclient: get_monmap_and_config 2024-01-24T23:33:35.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.158+0000 7f08299a4640 10 monclient: build_initial_monmap 2024-01-24T23:33:35.154 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.158+0000 7f08299a4640 10 monclient: monmap: 2024-01-24T23:33:35.154 INFO:tasks.workunit.client.0.smithi132.stderr:epoch 0 2024-01-24T23:33:35.154 INFO:tasks.workunit.client.0.smithi132.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:33:35.154 INFO:tasks.workunit.client.0.smithi132.stderr:last_changed 2024-01-24T23:33:35.161761+0000 2024-01-24T23:33:35.154 INFO:tasks.workunit.client.0.smithi132.stderr:created 2024-01-24T23:33:35.161761+0000 2024-01-24T23:33:35.154 INFO:tasks.workunit.client.0.smithi132.stderr:min_mon_release 0 (unknown) 2024-01-24T23:33:35.154 INFO:tasks.workunit.client.0.smithi132.stderr:election_strategy: 3 2024-01-24T23:33:35.154 INFO:tasks.workunit.client.0.smithi132.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:33:35.154 INFO:tasks.workunit.client.0.smithi132.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:33:35.154 INFO:tasks.workunit.client.0.smithi132.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:33:35.154 INFO:tasks.workunit.client.0.smithi132.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:33:35.154 INFO:tasks.workunit.client.0.smithi132.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:33:35.154 INFO:tasks.workunit.client.0.smithi132.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:33:35.154 INFO:tasks.workunit.client.0.smithi132.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:33:35.155 INFO:tasks.workunit.client.0.smithi132.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:33:35.155 INFO:tasks.workunit.client.0.smithi132.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:33:35.155 INFO:tasks.workunit.client.0.smithi132.stderr: 2024-01-24T23:33:35.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.158+0000 7f08299a4640 1 Processor -- start 2024-01-24T23:33:35.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.158+0000 7f08299a4640 1 -- start start 2024-01-24T23:33:35.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.158+0000 7f08299a4640 10 monclient: init 2024-01-24T23:33:35.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.158+0000 7f08299a4640 10 monclient: _reopen_session rank -1 2024-01-24T23:33:35.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.158+0000 7f08299a4640 10 monclient: _add_conns ranks=[3,6,8,4,1,2,0,7,5] 2024-01-24T23:33:35.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.158+0000 7f08299a4640 10 monclient(hunting): picked mon.noname-g con 0x7f082409c4f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:33:35.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.158+0000 7f08299a4640 10 monclient(hunting): picked mon.noname-d con 0x7f082409cfe0 addr v1:172.21.15.115:6792/0 2024-01-24T23:33:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.158+0000 7f08299a4640 10 monclient(hunting): picked mon.noname-e con 0x7f082413a3a0 addr v1:172.21.15.115:6793/0 2024-01-24T23:33:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.158+0000 7f08299a4640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082404f7e0 con 0x7f082409c4f0 2024-01-24T23:33:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.158+0000 7f08299a4640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082409fd80 con 0x7f082409cfe0 2024-01-24T23:33:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.158+0000 7f08299a4640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082409e360 con 0x7f082413a3a0 2024-01-24T23:33:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.158+0000 7f08299a4640 10 monclient(hunting): _renew_subs 2024-01-24T23:33:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.158+0000 7f08299a4640 10 monclient(hunting): authenticate will time out at 1702.486399s 2024-01-24T23:33:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.158+0000 7f0822ffd640 1 -- >> v1:172.21.15.132:6790/0 conn(0x7f082409c4f0 legacy=0x7f082409c8d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:33:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.158+0000 7f08237fe640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f082413a3a0 legacy=0x7f082413c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:33:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.158+0000 7f08227fc640 1 -- >> v1:172.21.15.115:6792/0 conn(0x7f082409cfe0 legacy=0x7f0824138c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:35.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.362+0000 7f0822ffd640 1 -- >> v1:172.21.15.132:6790/0 conn(0x7f082409c4f0 legacy=0x7f082409c8d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:33:35.355 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.362+0000 7f08237fe640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f082413a3a0 legacy=0x7f082413c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:33:35.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.362+0000 7f08227fc640 1 -- >> v1:172.21.15.115:6792/0 conn(0x7f082409cfe0 legacy=0x7f0824138c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:35.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.766+0000 7f0822ffd640 1 -- >> v1:172.21.15.132:6790/0 conn(0x7f082409c4f0 legacy=0x7f082409c8d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:33:35.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.766+0000 7f08237fe640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f082413a3a0 legacy=0x7f082413c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:33:35.761 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:35.766+0000 7f08227fc640 1 -- >> v1:172.21.15.115:6792/0 conn(0x7f082409cfe0 legacy=0x7f0824138c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:36.367 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:33:36.368+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:33:36.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:36.570+0000 7f0822ffd640 1 -- >> v1:172.21.15.132:6790/0 conn(0x7f082409c4f0 legacy=0x7f082409c8d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:33:36.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:36.570+0000 7f08237fe640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f082413a3a0 legacy=0x7f082413c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:33:36.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:36.570+0000 7f08227fc640 1 -- >> v1:172.21.15.115:6792/0 conn(0x7f082409cfe0 legacy=0x7f0824138c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:38.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:38.158+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:33:38.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:38.158+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:33:38.155 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:38.158+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:33:38.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:38.158+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:33:38.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:38.158+0000 7f0820ff9640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f082413a3a0 legacy=0x7f082413c790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:38.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:38.158+0000 7f0820ff9640 1 -- >> v1:172.21.15.115:6792/0 conn(0x7f082409cfe0 legacy=0x7f0824138c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:38.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:38.158+0000 7f0820ff9640 1 -- >> v1:172.21.15.132:6790/0 conn(0x7f082409c4f0 legacy=0x7f082409c8d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:38.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:38.158+0000 7f0820ff9640 10 monclient: _add_conns ranks=[1,4,0,2,5,7] 2024-01-24T23:33:38.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:38.158+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f0810003790 addr v1:172.21.15.132:6789/0 2024-01-24T23:33:38.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:38.158+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f0810007420 addr v1:172.21.15.115:6791/0 2024-01-24T23:33:38.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:38.158+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f081000b1b0 addr v1:172.21.15.115:6789/0 2024-01-24T23:33:38.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:38.158+0000 7f0820ff9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082409e080 con 0x7f081000b1b0 2024-01-24T23:33:38.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:38.158+0000 7f0820ff9640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082404f7e0 con 0x7f0810003790 2024-01-24T23:33:38.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:38.158+0000 7f0820ff9640 1 -- --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082409fd80 con 0x7f0810007420 2024-01-24T23:33:38.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:38.158+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:33:38.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:38.158+0000 7f0822ffd640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f0810003790 0x7f0810005be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:41458/0 (socket says 172.21.15.132:41458) 2024-01-24T23:33:38.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:38.158+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 learned_addr learned my addr 172.21.15.132:0/2783829207 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-01-24T23:33:41.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.162+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:33:41.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.162+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:33:41.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.162+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:33:41.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.162+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:33:41.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810007420 legacy=0x7f08100098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:33:41.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810003790 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:33:41.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000b1b0 legacy=0x7f081000d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:33:41.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.162+0000 7f0820ff9640 10 monclient: _add_conns ranks=[5,7,2] 2024-01-24T23:33:41.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f08100110a0 addr v1:172.21.15.132:6791/0 2024-01-24T23:33:41.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.162+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:33:41.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081000ff10 addr v1:172.21.15.132:6792/0 2024-01-24T23:33:41.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.162+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:33:41.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f0810010ad0 addr v1:172.21.15.115:6790/0 2024-01-24T23:33:41.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082409e360 con 0x7f0810010ad0 2024-01-24T23:33:41.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000de40 con 0x7f08100110a0 2024-01-24T23:33:41.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000e060 con 0x7f081000ff10 2024-01-24T23:33:41.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.162+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:33:41.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.162+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000ff10 legacy=0x7f08100103c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:33:41.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.162+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f08100110a0 legacy=0x7f08100114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:33:41.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.162+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810010ad0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:41.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.366+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000ff10 legacy=0x7f08100103c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:33:41.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.366+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f08100110a0 legacy=0x7f08100114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:33:41.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.366+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810010ad0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:41.367 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:33:41.368+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:33:41.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.770+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000ff10 legacy=0x7f08100103c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:33:41.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.770+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f08100110a0 legacy=0x7f08100114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:33:41.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:41.770+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810010ad0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:42.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:42.574+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f08100110a0 legacy=0x7f08100114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:33:42.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:42.574+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000ff10 legacy=0x7f08100103c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:33:42.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:42.574+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810010ad0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:44.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:44.158+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:33:44.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:44.158+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:33:44.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:44.158+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:33:44.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:44.158+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:33:44.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:44.158+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000ff10 legacy=0x7f08100103c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:44.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:44.158+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f08100110a0 legacy=0x7f08100114e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:44.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:44.158+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810010ad0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:44.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:44.158+0000 7f0820ff9640 10 monclient: _add_conns ranks=[1,6,5,7,8,2,0,3,4] 2024-01-24T23:33:44.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:44.158+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f081000e1f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:33:44.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:44.158+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f081000b590 addr v1:172.21.15.115:6792/0 2024-01-24T23:33:44.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:44.158+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f08100110a0 addr v1:172.21.15.132:6791/0 2024-01-24T23:33:44.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:44.158+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100099f0 con 0x7f081000e1f0 2024-01-24T23:33:44.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:44.158+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082409e360 con 0x7f08100110a0 2024-01-24T23:33:44.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:44.158+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000de40 con 0x7f081000b590 2024-01-24T23:33:44.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:44.158+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:33:44.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:44.158+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f08100110a0 legacy=0x7f08100114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:33:44.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:44.158+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000b590 legacy=0x7f0810019350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:44.356 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:44.362+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f08100110a0 legacy=0x7f08100114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:33:44.357 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:44.362+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000b590 legacy=0x7f0810019350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:44.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:44.766+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f08100110a0 legacy=0x7f08100114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:33:44.760 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:44.766+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000b590 legacy=0x7f0810019350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:45.564 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:45.570+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f08100110a0 legacy=0x7f08100114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:33:45.565 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:45.570+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000b590 legacy=0x7f0810019350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:46.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:46.370+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000e1f0 legacy=0x7f081000e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:33:46.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:46.370+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000e1f0 legacy=0x7f081000e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:33:46.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:46.370+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000e1f0 0x7f081000e5d0 :-1 s=OPENED pgs=120 cs=1 l=1).handle_message read tag failed 2024-01-24T23:33:46.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:46.370+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000e1f0 0x7f081000e5d0 :-1 s=OPENED pgs=120 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:33:46.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:46.370+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:33:46.367 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:33:46.367+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:33:47.156 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.162+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:33:47.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.162+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:33:47.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.162+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:33:47.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.162+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:33:47.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000b590 legacy=0x7f0810019350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:47.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f08100110a0 legacy=0x7f08100114e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:47.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000e1f0 legacy=0x7f081000e5d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:33:47.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.162+0000 7f0820ff9640 10 monclient: _add_conns ranks=[2,7,8,3,0,4] 2024-01-24T23:33:47.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f0810010ad0 addr v1:172.21.15.115:6790/0 2024-01-24T23:33:47.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.162+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:33:47.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081000ff10 addr v1:172.21.15.132:6792/0 2024-01-24T23:33:47.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f08100110a0 addr v1:172.21.15.115:6793/0 2024-01-24T23:33:47.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000e060 con 0x7f0810010ad0 2024-01-24T23:33:47.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082409e360 con 0x7f081000ff10 2024-01-24T23:33:47.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000de40 con 0x7f08100110a0 2024-01-24T23:33:47.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.162+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:33:47.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.162+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000ff10 legacy=0x7f08100103c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:33:47.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.162+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f08100110a0 legacy=0x7f08100114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:33:47.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.162+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810010ad0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:47.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.366+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000ff10 legacy=0x7f08100103c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:33:47.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.366+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810010ad0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:47.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.366+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f08100110a0 legacy=0x7f08100114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:33:47.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.770+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000ff10 legacy=0x7f08100103c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:33:47.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.770+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810010ad0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:47.765 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:47.770+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f08100110a0 legacy=0x7f08100114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:33:48.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:48.574+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000ff10 legacy=0x7f08100103c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:33:48.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:48.574+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810010ad0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:48.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:48.574+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f08100110a0 legacy=0x7f08100114e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:33:50.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:50.162+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:33:50.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:50.162+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:33:50.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:50.162+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:33:50.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:50.162+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:33:50.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:50.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f08100110a0 legacy=0x7f08100114e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:50.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:50.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000ff10 legacy=0x7f08100103c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:50.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:50.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810010ad0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:50.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:50.162+0000 7f0820ff9640 10 monclient: _add_conns ranks=[4,3,0] 2024-01-24T23:33:50.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:50.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f081000e5d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:33:50.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:50.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f081000a7c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:33:50.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:50.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f081000aba0 addr v1:172.21.15.115:6789/0 2024-01-24T23:33:50.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:50.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810009e40 con 0x7f081000aba0 2024-01-24T23:33:50.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:50.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000e060 con 0x7f081000a7c0 2024-01-24T23:33:50.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:50.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082409e360 con 0x7f081000e5d0 2024-01-24T23:33:50.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:50.162+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:33:50.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:50.162+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a7c0 legacy=0x7f081000c980 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:33:50.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:50.366+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a7c0 legacy=0x7f081000c980 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:33:50.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:50.770+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a7c0 legacy=0x7f081000c980 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:33:51.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:51.362+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000aba0 legacy=0x7f081000d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:33:51.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:51.362+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000aba0 legacy=0x7f081000d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:33:51.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:51.362+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000aba0 0x7f081000d090 :-1 s=OPENED pgs=124 cs=1 l=1).handle_message read tag failed 2024-01-24T23:33:51.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:51.362+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000aba0 0x7f081000d090 :-1 s=OPENED pgs=124 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:33:51.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:51.362+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:33:51.367 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:33:51.367+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:33:51.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:51.370+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000e5d0 legacy=0x7f081000a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:33:51.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:51.370+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000e5d0 legacy=0x7f081000a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:33:51.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:51.370+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000e5d0 0x7f081000a0b0 :-1 s=OPENED pgs=125 cs=1 l=1).handle_message read tag failed 2024-01-24T23:33:51.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:51.370+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000e5d0 0x7f081000a0b0 :-1 s=OPENED pgs=125 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:33:51.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:51.370+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:33:51.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:51.370+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:33:51.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:51.574+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a7c0 legacy=0x7f081000c980 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:33:53.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:53.162+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:33:53.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:53.162+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:33:53.157 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:53.162+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:33:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:53.162+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:33:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:53.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000e5d0 legacy=0x7f081000a0b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:33:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:53.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a7c0 legacy=0x7f081000c980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:53.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000aba0 legacy=0x7f081000d090 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:33:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:53.162+0000 7f0820ff9640 10 monclient: _add_conns ranks=[4,8,5,1,2,3,7,0,6] 2024-01-24T23:33:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:53.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f081000aba0 addr v1:172.21.15.115:6791/0 2024-01-24T23:33:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:53.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f081000e5d0 addr v1:172.21.15.115:6793/0 2024-01-24T23:33:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:53.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f081000ff10 addr v1:172.21.15.132:6791/0 2024-01-24T23:33:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:53.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000de40 con 0x7f081000aba0 2024-01-24T23:33:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:53.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000e060 con 0x7f081000ff10 2024-01-24T23:33:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:53.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000eb70 con 0x7f081000e5d0 2024-01-24T23:33:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:53.162+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:33:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:53.162+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000ff10 legacy=0x7f08100103c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:33:53.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:53.162+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000e5d0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:33:53.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:53.366+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000ff10 legacy=0x7f08100103c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:33:53.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:53.366+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000e5d0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:33:53.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:53.770+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000ff10 legacy=0x7f08100103c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:33:53.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:53.770+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000e5d0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:33:54.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:54.574+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000ff10 legacy=0x7f08100103c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:33:54.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:54.574+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000e5d0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:33:56.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.162+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:33:56.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.162+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:33:56.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.162+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:33:56.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.162+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:33:56.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000e5d0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:56.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000ff10 legacy=0x7f08100103c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:56.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000aba0 legacy=0x7f081000a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:33:56.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.162+0000 7f0820ff9640 10 monclient: _add_conns ranks=[0,6,1,3,7,2] 2024-01-24T23:33:56.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f08100110a0 addr v1:172.21.15.115:6789/0 2024-01-24T23:33:56.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f081000d090 addr v1:172.21.15.115:6792/0 2024-01-24T23:33:56.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f081001f390 addr v1:172.21.15.132:6789/0 2024-01-24T23:33:56.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010ad0 con 0x7f08100110a0 2024-01-24T23:33:56.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000e060 con 0x7f081001f390 2024-01-24T23:33:56.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000eb70 con 0x7f081000d090 2024-01-24T23:33:56.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.162+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:33:56.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.162+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000d090 legacy=0x7f081000d4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:56.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.362+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f08100110a0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:33:56.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.362+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f08100110a0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:33:56.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.362+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f08100110a0 0x7f0810011480 :-1 s=OPENED pgs=131 cs=1 l=1).handle_message read tag failed 2024-01-24T23:33:56.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.362+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f08100110a0 0x7f0810011480 :-1 s=OPENED pgs=131 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:33:56.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.362+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:33:56.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.362+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:33:56.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.366+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000d090 legacy=0x7f081000d4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:56.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.370+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081001f390 legacy=0x7f0810021780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:33:56.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.370+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081001f390 legacy=0x7f0810021780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:33:56.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.370+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081001f390 0x7f0810021780 :-1 s=OPENED pgs=127 cs=1 l=1).handle_message read tag failed 2024-01-24T23:33:56.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.370+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081001f390 0x7f0810021780 :-1 s=OPENED pgs=127 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:33:56.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.370+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:33:56.367 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:33:56.367+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:33:56.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:56.770+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000d090 legacy=0x7f081000d4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:57.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:57.574+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000d090 legacy=0x7f081000d4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:33:59.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:59.162+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:33:59.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:59.162+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:33:59.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:59.162+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:33:59.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:59.162+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:33:59.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:59.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000d090 legacy=0x7f081000d4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:33:59.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:59.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081001f390 legacy=0x7f0810021780 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:33:59.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:59.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f08100110a0 legacy=0x7f0810011480 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:33:59.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:59.162+0000 7f0820ff9640 10 monclient: _add_conns ranks=[2,7,3] 2024-01-24T23:33:59.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:59.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f08100110a0 addr v1:172.21.15.115:6790/0 2024-01-24T23:33:59.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:59.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081000ff10 addr v1:172.21.15.132:6792/0 2024-01-24T23:33:59.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:59.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f081000e5d0 addr v1:172.21.15.132:6790/0 2024-01-24T23:33:59.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:59.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810025ed0 con 0x7f08100110a0 2024-01-24T23:33:59.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:59.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000eb70 con 0x7f081000e5d0 2024-01-24T23:33:59.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:59.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000c980 con 0x7f081000ff10 2024-01-24T23:33:59.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:59.162+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:33:59.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:59.162+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000ff10 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:33:59.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:59.162+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000e5d0 legacy=0x7f0810010380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:33:59.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:59.162+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f08100110a0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:59.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:59.366+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000ff10 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:33:59.358 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:59.366+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000e5d0 legacy=0x7f0810010380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:33:59.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:59.366+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f08100110a0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:33:59.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:59.770+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000e5d0 legacy=0x7f0810010380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:33:59.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:59.770+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000ff10 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:33:59.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:33:59.770+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f08100110a0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:00.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:00.574+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000e5d0 legacy=0x7f0810010380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:00.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:00.574+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000ff10 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:00.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:00.574+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f08100110a0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:01.368 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:34:01.367+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:34:02.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:02.162+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:34:02.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:02.162+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:34:02.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:02.162+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:34:02.158 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:02.162+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:34:02.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:02.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000ff10 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:02.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:02.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000e5d0 legacy=0x7f0810010380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:02.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:02.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f08100110a0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:02.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:02.162+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:34:02.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:02.162+0000 7f0820ff9640 10 monclient: _add_conns ranks=[1,4,0,6,2,5,3,7,8] 2024-01-24T23:34:02.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:02.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f081000fef0 addr v1:172.21.15.132:6789/0 2024-01-24T23:34:02.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:02.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f081000e5d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:34:02.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:02.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f0810010360 addr v1:172.21.15.115:6789/0 2024-01-24T23:34:02.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:02.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000cb10 con 0x7f0810010360 2024-01-24T23:34:02.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:02.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810025ed0 con 0x7f081000fef0 2024-01-24T23:34:02.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:02.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000eb70 con 0x7f081000e5d0 2024-01-24T23:34:02.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:02.162+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:34:05.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:05.166+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:34:05.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:05.166+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:34:05.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:05.166+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:34:05.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:05.166+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:34:05.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:05.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000e5d0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:34:05.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:05.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000fef0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:34:05.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:05.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f0810010360 legacy=0x7f081000cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:34:05.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:05.166+0000 7f0820ff9640 10 monclient: _add_conns ranks=[6,5,8,3,2,7] 2024-01-24T23:34:05.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:05.166+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f081000d430 addr v1:172.21.15.115:6792/0 2024-01-24T23:34:05.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:05.166+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f081000a7c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:34:05.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:05.166+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f081000ac40 addr v1:172.21.15.115:6793/0 2024-01-24T23:34:05.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:05.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000c980 con 0x7f081000a7c0 2024-01-24T23:34:05.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:05.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000d810 con 0x7f081000d430 2024-01-24T23:34:05.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:05.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000d9a0 con 0x7f081000ac40 2024-01-24T23:34:05.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:05.166+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:34:05.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:05.166+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a7c0 legacy=0x7f081001b360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:05.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:05.166+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000ac40 legacy=0x7f081001ea90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:05.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:05.166+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000d430 legacy=0x7f081000a0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:05.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:05.370+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a7c0 legacy=0x7f081001b360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:05.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:05.370+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000d430 legacy=0x7f081000a0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:05.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:05.370+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000ac40 legacy=0x7f081001ea90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:05.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:05.774+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a7c0 legacy=0x7f081001b360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:05.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:05.774+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000ac40 legacy=0x7f081001ea90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:05.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:05.774+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000d430 legacy=0x7f081000a0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:06.368 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:34:06.367+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:34:06.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:06.578+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a7c0 legacy=0x7f081001b360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:06.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:06.578+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000d430 legacy=0x7f081000a0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:06.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:06.578+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000ac40 legacy=0x7f081001ea90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:08.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.162+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:34:08.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.162+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:34:08.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.162+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:34:08.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.162+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:34:08.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000ac40 legacy=0x7f081001ea90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:08.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000d430 legacy=0x7f081000a0b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:08.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a7c0 legacy=0x7f081001b360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:08.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.162+0000 7f0820ff9640 10 monclient: _add_conns ranks=[3,2,7] 2024-01-24T23:34:08.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f08100233f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:34:08.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.162+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:34:08.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f08100257e0 addr v1:172.21.15.115:6790/0 2024-01-24T23:34:08.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081000e5d0 addr v1:172.21.15.132:6792/0 2024-01-24T23:34:08.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100231e0 con 0x7f08100257e0 2024-01-24T23:34:08.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.162+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f08100233f0 legacy=0x7f08100201b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:08.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000c980 con 0x7f08100233f0 2024-01-24T23:34:08.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000d810 con 0x7f081000e5d0 2024-01-24T23:34:08.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.162+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:34:08.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.162+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:08.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.162+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f08100257e0 legacy=0x7f08100218d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:08.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.366+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f08100233f0 legacy=0x7f08100201b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:08.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.366+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:08.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.366+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f08100257e0 legacy=0x7f08100218d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:08.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.770+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:08.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.770+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f08100233f0 legacy=0x7f08100201b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:08.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:08.770+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f08100257e0 legacy=0x7f08100218d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:09.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:09.574+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:09.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:09.574+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f08100233f0 legacy=0x7f08100201b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:09.569 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:09.574+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f08100257e0 legacy=0x7f08100218d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:11.159 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.162+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:34:11.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.162+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:34:11.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.162+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:34:11.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.162+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:34:11.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:11.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f08100233f0 legacy=0x7f08100201b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:11.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f08100257e0 legacy=0x7f08100218d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:11.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.162+0000 7f0820ff9640 10 monclient: _add_conns ranks=[8,6,4,2,0,3,1,5,7] 2024-01-24T23:34:11.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f0810010360 addr v1:172.21.15.115:6793/0 2024-01-24T23:34:11.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f081000abf0 addr v1:172.21.15.115:6792/0 2024-01-24T23:34:11.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.162+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f081000d430 addr v1:172.21.15.115:6791/0 2024-01-24T23:34:11.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000d9a0 con 0x7f081000d430 2024-01-24T23:34:11.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100231e0 con 0x7f081000abf0 2024-01-24T23:34:11.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.162+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000c980 con 0x7f0810010360 2024-01-24T23:34:11.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.162+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:34:11.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.162+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000abf0 legacy=0x7f081000cd20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:11.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.162+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810010360 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:11.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.366+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810010360 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:11.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.366+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000abf0 legacy=0x7f081000cd20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:11.368 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:34:11.367+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:34:11.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.374+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000d430 legacy=0x7f081000a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:34:11.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.374+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000d430 legacy=0x7f081000a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:34:11.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.374+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000d430 0x7f081000a090 :-1 s=OPENED pgs=140 cs=1 l=1).handle_message read tag failed 2024-01-24T23:34:11.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.374+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000d430 0x7f081000a090 :-1 s=OPENED pgs=140 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:34:11.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.374+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:34:11.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.770+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000abf0 legacy=0x7f081000cd20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:11.764 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:11.770+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810010360 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:12.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:12.574+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000abf0 legacy=0x7f081000cd20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:12.568 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:12.574+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810010360 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:14.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:14.166+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:34:14.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:14.166+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:34:14.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:14.166+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:34:14.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:14.166+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:34:14.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:14.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810010360 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:14.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:14.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000abf0 legacy=0x7f081000cd20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:14.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:14.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000d430 legacy=0x7f081000a090 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:34:14.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:14.166+0000 7f0820ff9640 10 monclient: _add_conns ranks=[5,3,0,2,1,7] 2024-01-24T23:34:14.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:14.166+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f081000fef0 addr v1:172.21.15.132:6791/0 2024-01-24T23:34:14.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:14.166+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:34:14.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:14.166+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f08100233f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:34:14.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:14.166+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f081000a7a0 addr v1:172.21.15.115:6789/0 2024-01-24T23:34:14.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:14.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000d810 con 0x7f081000a7a0 2024-01-24T23:34:14.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:14.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100231e0 con 0x7f08100233f0 2024-01-24T23:34:14.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:14.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000c980 con 0x7f081000fef0 2024-01-24T23:34:14.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:14.166+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000fef0 legacy=0x7f081001b360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:14.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:14.166+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:34:14.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:14.166+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f08100233f0 legacy=0x7f081001ca80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:14.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:14.370+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000fef0 legacy=0x7f081001b360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:14.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:14.370+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f08100233f0 legacy=0x7f081001ca80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:14.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:14.774+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f08100233f0 legacy=0x7f081001ca80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:14.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:14.774+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000fef0 legacy=0x7f081001b360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:15.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:15.578+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f08100233f0 legacy=0x7f081001ca80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:15.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:15.578+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000fef0 legacy=0x7f081001b360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:16.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:16.366+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000a7a0 legacy=0x7f08100201b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:34:16.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:16.366+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000a7a0 legacy=0x7f08100201b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:34:16.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:16.366+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000a7a0 0x7f08100201b0 :-1 s=OPENED pgs=139 cs=1 l=1).handle_message read tag failed 2024-01-24T23:34:16.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:16.366+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000a7a0 0x7f08100201b0 :-1 s=OPENED pgs=139 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:34:16.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:16.366+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:34:16.368 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:34:16.367+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:34:17.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:17.166+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:34:17.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:17.166+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:34:17.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:17.166+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:34:17.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:17.166+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:34:17.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:17.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000fef0 legacy=0x7f081001b360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:17.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:17.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f08100233f0 legacy=0x7f081001ca80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:17.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:17.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000a7a0 legacy=0x7f08100201b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:34:17.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:17.166+0000 7f0820ff9640 10 monclient: _add_conns ranks=[7,2,1] 2024-01-24T23:34:17.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:17.166+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081000abc0 addr v1:172.21.15.132:6792/0 2024-01-24T23:34:17.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:17.166+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f081000e5d0 addr v1:172.21.15.115:6790/0 2024-01-24T23:34:17.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:17.166+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f08100257e0 addr v1:172.21.15.132:6789/0 2024-01-24T23:34:17.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:17.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010360 con 0x7f08100257e0 2024-01-24T23:34:17.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:17.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100231e0 con 0x7f081000e5d0 2024-01-24T23:34:17.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:17.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000c980 con 0x7f081000abc0 2024-01-24T23:34:17.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:17.166+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:34:17.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:17.166+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000abc0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:17.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:17.166+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000e5d0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:17.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:17.370+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000abc0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:17.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:17.370+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000e5d0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:17.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:17.774+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000abc0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:17.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:17.774+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000e5d0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:18.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:18.578+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000abc0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:18.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:18.578+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000e5d0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:20.160 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.166+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:34:20.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.166+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:34:20.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.166+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:34:20.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.166+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:34:20.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000abc0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:20.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000e5d0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:20.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f08100257e0 legacy=0x7f0810009ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:34:20.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.166+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:34:20.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.166+0000 7f0820ff9640 10 monclient: _add_conns ranks=[2,6,5,7,3,1,8,0,4] 2024-01-24T23:34:20.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.166+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:34:20.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.166+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f081000abc0 addr v1:172.21.15.115:6790/0 2024-01-24T23:34:20.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.166+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f081000e5d0 addr v1:172.21.15.115:6792/0 2024-01-24T23:34:20.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.166+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f081000fef0 addr v1:172.21.15.132:6791/0 2024-01-24T23:34:20.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000abc0 2024-01-24T23:34:20.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100231e0 con 0x7f081000fef0 2024-01-24T23:34:20.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000c980 con 0x7f081000e5d0 2024-01-24T23:34:20.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.166+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:34:20.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.166+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000fef0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:20.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.166+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000e5d0 legacy=0x7f081000cce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:20.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.166+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000abc0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:20.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.370+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000fef0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:20.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.370+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000e5d0 legacy=0x7f081000cce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:20.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.370+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000abc0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:20.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.774+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000fef0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:20.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.774+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000e5d0 legacy=0x7f081000cce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:20.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:20.774+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000abc0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:21.369 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:34:21.371+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:34:21.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:21.578+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000fef0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:21.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:21.578+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000abc0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:21.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:21.578+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000e5d0 legacy=0x7f081000cce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:23.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:23.166+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:34:23.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:23.166+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:34:23.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:23.166+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:34:23.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:23.166+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:34:23.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:23.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000e5d0 legacy=0x7f081000cce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:23.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:23.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000fef0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:23.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:23.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000abc0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:23.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:23.166+0000 7f0820ff9640 10 monclient: _add_conns ranks=[4,3,1,7,8,0] 2024-01-24T23:34:23.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:23.166+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f081000d3f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:34:23.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:23.166+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f081000a750 addr v1:172.21.15.132:6790/0 2024-01-24T23:34:23.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:23.166+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f081001a360 addr v1:172.21.15.132:6789/0 2024-01-24T23:34:23.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:23.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810025bf0 con 0x7f081001a360 2024-01-24T23:34:23.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:23.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000a750 2024-01-24T23:34:23.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:23.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100231e0 con 0x7f081000d3f0 2024-01-24T23:34:23.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:23.166+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:34:23.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:23.166+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a750 legacy=0x7f0810025400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:23.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:23.370+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a750 legacy=0x7f0810025400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:23.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:23.774+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a750 legacy=0x7f0810025400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:24.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:24.578+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a750 legacy=0x7f0810025400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:26.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.166+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:34:26.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.166+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:34:26.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.166+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:34:26.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.166+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:34:26.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000d3f0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:34:26.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a750 legacy=0x7f0810025400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:26.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081001a360 legacy=0x7f081001c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:34:26.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.166+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:34:26.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.166+0000 7f0820ff9640 10 monclient: _add_conns ranks=[7,8,0] 2024-01-24T23:34:26.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.166+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:34:26.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.166+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081000a750 addr v1:172.21.15.132:6792/0 2024-01-24T23:34:26.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.166+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f081000abc0 addr v1:172.21.15.115:6793/0 2024-01-24T23:34:26.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.166+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f081000e5d0 addr v1:172.21.15.115:6789/0 2024-01-24T23:34:26.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000c980 con 0x7f081000e5d0 2024-01-24T23:34:26.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000a750 2024-01-24T23:34:26.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100108a0 con 0x7f081000abc0 2024-01-24T23:34:26.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.166+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:34:26.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.166+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:26.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.166+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000abc0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:26.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.366+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:34:26.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.366+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:34:26.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.366+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000e5d0 0x7f0810009fd0 :-1 s=OPENED pgs=146 cs=1 l=1).handle_message read tag failed 2024-01-24T23:34:26.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.366+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000e5d0 0x7f0810009fd0 :-1 s=OPENED pgs=146 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:34:26.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.366+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:34:26.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.370+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:26.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.370+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000abc0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:26.369 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:34:26.367+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:34:26.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.774+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:26.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:26.774+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000abc0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:27.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:27.578+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:27.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:27.578+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000abc0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:29.161 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:29.166+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:34:29.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:29.166+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:34:29.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:29.166+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:34:29.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:29.166+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:34:29.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:29.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000abc0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:29.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:29.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:29.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:29.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:34:29.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:29.166+0000 7f0820ff9640 10 monclient: _add_conns ranks=[8,5,4,7,1,2,3,0,6] 2024-01-24T23:34:29.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:29.166+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f081000fef0 addr v1:172.21.15.115:6793/0 2024-01-24T23:34:29.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:29.166+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f081000ceb0 addr v1:172.21.15.132:6791/0 2024-01-24T23:34:29.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:29.166+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f0810017340 addr v1:172.21.15.115:6791/0 2024-01-24T23:34:29.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:29.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000cce0 con 0x7f0810017340 2024-01-24T23:34:29.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:29.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000ceb0 2024-01-24T23:34:29.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:29.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100108a0 con 0x7f081000fef0 2024-01-24T23:34:29.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:29.166+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:34:29.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:29.166+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000ceb0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:29.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:29.166+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000fef0 legacy=0x7f0810025380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:29.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:29.370+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000ceb0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:29.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:29.370+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000fef0 legacy=0x7f0810025380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:29.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:29.774+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000ceb0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:29.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:29.774+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000fef0 legacy=0x7f0810025380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:30.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:30.578+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000ceb0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:30.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:30.578+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000fef0 legacy=0x7f0810025380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:31.369 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:34:31.370+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:34:31.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:31.378+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810017340 legacy=0x7f0810019760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:34:31.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:31.378+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810017340 legacy=0x7f0810019760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:34:31.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:31.378+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810017340 0x7f0810019760 :-1 s=OPENED pgs=148 cs=1 l=1).handle_message read tag failed 2024-01-24T23:34:31.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:31.378+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810017340 0x7f0810019760 :-1 s=OPENED pgs=148 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:34:31.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:31.378+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:34:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:32.170+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:34:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:32.170+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:34:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:32.170+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:34:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:32.170+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:34:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:32.170+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000fef0 legacy=0x7f0810025380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:32.170+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000ceb0 legacy=0x7f0810005be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:32.170+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810017340 legacy=0x7f0810019760 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:34:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:32.170+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:34:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:32.170+0000 7f0820ff9640 10 monclient: _add_conns ranks=[2,0,3,6,7,1] 2024-01-24T23:34:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:32.170+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f081000a750 addr v1:172.21.15.115:6790/0 2024-01-24T23:34:32.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:32.170+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f081000e5d0 addr v1:172.21.15.115:6789/0 2024-01-24T23:34:32.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:32.170+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f081000ceb0 addr v1:172.21.15.132:6790/0 2024-01-24T23:34:32.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:32.170+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000c980 con 0x7f081000e5d0 2024-01-24T23:34:32.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:32.170+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000a750 2024-01-24T23:34:32.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:32.170+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100108a0 con 0x7f081000ceb0 2024-01-24T23:34:32.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:32.170+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:34:32.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:32.170+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000ceb0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:32.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:32.170+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:32.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:32.374+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000ceb0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:32.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:32.374+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:32.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:32.778+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000ceb0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:32.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:32.778+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:33.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:33.582+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000ceb0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:33.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:33.582+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:35.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:35.170+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:34:35.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:35.170+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:34:35.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:35.170+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:34:35.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:35.170+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:34:35.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:35.170+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000ceb0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:35.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:35.170+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:35.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:35.170+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000e5d0 legacy=0x7f0810005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:34:35.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:35.170+0000 7f0820ff9640 10 monclient: _add_conns ranks=[1,6,7] 2024-01-24T23:34:35.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:35.170+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f0810025380 addr v1:172.21.15.132:6789/0 2024-01-24T23:34:35.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:35.170+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f0810025790 addr v1:172.21.15.115:6792/0 2024-01-24T23:34:35.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:35.170+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081001b180 addr v1:172.21.15.132:6792/0 2024-01-24T23:34:35.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:35.170+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000cca0 con 0x7f0810025380 2024-01-24T23:34:35.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:35.170+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f0810025790 2024-01-24T23:34:35.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:35.170+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100108a0 con 0x7f081001b180 2024-01-24T23:34:35.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:35.170+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:34:35.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:35.170+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b180 legacy=0x7f081001d5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:35.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:35.170+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810025790 legacy=0x7f0810019a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:35.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:35.374+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b180 legacy=0x7f081001d5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:35.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:35.374+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810025790 legacy=0x7f0810019a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:35.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:35.778+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b180 legacy=0x7f081001d5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:35.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:35.778+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810025790 legacy=0x7f0810019a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:36.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:36.370+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810025380 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:34:36.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:36.370+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810025380 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:34:36.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:36.370+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810025380 0x7f0810016330 :-1 s=OPENED pgs=143 cs=1 l=1).handle_message read tag failed 2024-01-24T23:34:36.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:36.370+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810025380 0x7f0810016330 :-1 s=OPENED pgs=143 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:34:36.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:36.370+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:34:36.369 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:34:36.370+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:34:36.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:36.582+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b180 legacy=0x7f081001d5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:36.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:36.582+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810025790 legacy=0x7f0810019a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:38.162 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.166+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:34:38.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.166+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:34:38.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.166+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:34:38.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.166+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:34:38.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b180 legacy=0x7f081001d5a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:38.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810025790 legacy=0x7f0810019a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:38.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810025380 legacy=0x7f0810016330 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:34:38.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.166+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:34:38.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.166+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:34:38.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.166+0000 7f0820ff9640 10 monclient: _add_conns ranks=[6,7,2,1,5,3,8,0,4] 2024-01-24T23:34:38.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.166+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f081000a750 addr v1:172.21.15.115:6792/0 2024-01-24T23:34:38.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.166+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081000e5d0 addr v1:172.21.15.132:6792/0 2024-01-24T23:34:38.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.166+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f0810025380 addr v1:172.21.15.115:6790/0 2024-01-24T23:34:38.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810021cf0 con 0x7f0810025380 2024-01-24T23:34:38.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000a750 2024-01-24T23:34:38.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.166+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100108a0 con 0x7f081000e5d0 2024-01-24T23:34:38.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.166+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:34:38.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.166+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:38.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.166+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000a750 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:38.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.166+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810025380 legacy=0x7f0810005bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:38.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.370+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:38.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.370+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000a750 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:38.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.370+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810025380 legacy=0x7f0810005bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:38.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.774+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:38.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.774+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810025380 legacy=0x7f0810005bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:38.768 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:38.774+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000a750 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:39.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:39.577+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:39.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:39.577+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810025380 legacy=0x7f0810005bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:39.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:39.577+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000a750 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:41.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.169+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:34:41.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.169+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:34:41.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.169+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:34:41.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.169+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:34:41.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:41.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000a750 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:41.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810025380 legacy=0x7f0810005bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:41.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.169+0000 7f0820ff9640 10 monclient: _add_conns ranks=[5,0,3,8,4,1] 2024-01-24T23:34:41.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f081000ceb0 addr v1:172.21.15.132:6791/0 2024-01-24T23:34:41.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f0810022590 addr v1:172.21.15.115:6789/0 2024-01-24T23:34:41.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f081001a360 addr v1:172.21.15.132:6790/0 2024-01-24T23:34:41.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000cca0 con 0x7f0810022590 2024-01-24T23:34:41.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810021cf0 con 0x7f081001a360 2024-01-24T23:34:41.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000ceb0 2024-01-24T23:34:41.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.169+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:34:41.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.169+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000ceb0 legacy=0x7f0810021e80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:41.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.169+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081001a360 legacy=0x7f081001c780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:41.359 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.365+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f0810022590 legacy=0x7f08100229a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:34:41.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.365+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f0810022590 legacy=0x7f08100229a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:34:41.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.365+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f0810022590 0x7f08100229a0 :-1 s=OPENED pgs=154 cs=1 l=1).handle_message read tag failed 2024-01-24T23:34:41.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.365+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f0810022590 0x7f08100229a0 :-1 s=OPENED pgs=154 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:34:41.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.365+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:34:41.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.373+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000ceb0 legacy=0x7f0810021e80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:41.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.373+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081001a360 legacy=0x7f081001c780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:41.369 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:34:41.370+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:34:41.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.777+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081001a360 legacy=0x7f081001c780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:41.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:41.777+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000ceb0 legacy=0x7f0810021e80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:42.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:42.581+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000ceb0 legacy=0x7f0810021e80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:42.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:42.581+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081001a360 legacy=0x7f081001c780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:44.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:44.169+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:34:44.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:44.169+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:34:44.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:44.169+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:34:44.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:44.169+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:34:44.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:44.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000ceb0 legacy=0x7f0810021e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:44.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:44.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081001a360 legacy=0x7f081001c780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:44.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:44.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f0810022590 legacy=0x7f08100229a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:34:44.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:44.169+0000 7f0820ff9640 10 monclient: _add_conns ranks=[4,1,8] 2024-01-24T23:34:44.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:44.169+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:34:44.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:44.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f081001dea0 addr v1:172.21.15.115:6791/0 2024-01-24T23:34:44.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:44.169+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:34:44.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:44.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f081000a750 addr v1:172.21.15.132:6789/0 2024-01-24T23:34:44.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:44.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f081000e5d0 addr v1:172.21.15.115:6793/0 2024-01-24T23:34:44.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:44.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100108a0 con 0x7f081000a750 2024-01-24T23:34:44.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:44.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810021cf0 con 0x7f081001dea0 2024-01-24T23:34:44.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:44.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000e5d0 2024-01-24T23:34:44.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:44.169+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:34:44.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:44.169+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:44.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:44.373+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:44.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:44.777+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:45.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:45.581+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:46.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:46.369+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:34:46.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:46.369+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:34:46.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:46.369+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000a750 0x7f0810011480 :-1 s=OPENED pgs=150 cs=1 l=1).handle_message read tag failed 2024-01-24T23:34:46.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:46.369+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000a750 0x7f0810011480 :-1 s=OPENED pgs=150 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:34:46.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:46.369+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:34:46.369 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:34:46.370+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:34:46.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:46.373+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001dea0 legacy=0x7f0810020290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:34:46.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:46.373+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001dea0 legacy=0x7f0810020290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:34:46.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:46.373+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001dea0 0x7f0810020290 :-1 s=OPENED pgs=155 cs=1 l=1).handle_message read tag failed 2024-01-24T23:34:46.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:46.373+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001dea0 0x7f0810020290 :-1 s=OPENED pgs=155 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:34:46.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:46.373+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:34:47.163 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:47.169+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:34:47.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:47.169+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:34:47.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:47.169+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:34:47.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:47.169+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:34:47.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:47.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:47.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:47.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001dea0 legacy=0x7f0810020290 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:34:47.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:47.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:34:47.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:47.169+0000 7f0820ff9640 10 monclient: _add_conns ranks=[8,4,5,3,7,6,2,0,1] 2024-01-24T23:34:47.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:47.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f0810025380 addr v1:172.21.15.115:6793/0 2024-01-24T23:34:47.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:47.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f081001dea0 addr v1:172.21.15.115:6791/0 2024-01-24T23:34:47.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:47.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f081000e5d0 addr v1:172.21.15.132:6791/0 2024-01-24T23:34:47.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:47.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810009810 con 0x7f081001dea0 2024-01-24T23:34:47.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:47.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000e5d0 2024-01-24T23:34:47.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:47.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000ce30 con 0x7f0810025380 2024-01-24T23:34:47.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:47.169+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:34:47.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:47.169+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000e5d0 legacy=0x7f0810020290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:47.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:47.169+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810025380 legacy=0x7f0810005bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:47.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:47.373+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000e5d0 legacy=0x7f0810020290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:47.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:47.373+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810025380 legacy=0x7f0810005bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:47.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:47.777+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000e5d0 legacy=0x7f0810020290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:47.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:47.777+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810025380 legacy=0x7f0810005bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:48.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:48.581+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000e5d0 legacy=0x7f0810020290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:48.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:48.581+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810025380 legacy=0x7f0810005bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:50.169+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:34:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:50.169+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:34:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:50.169+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:34:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:50.169+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:34:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:50.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810025380 legacy=0x7f0810005bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:50.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000e5d0 legacy=0x7f0810020290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:50.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001dea0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:34:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:50.169+0000 7f0820ff9640 10 monclient: _add_conns ranks=[2,3,7,6,1,0] 2024-01-24T23:34:50.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:50.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f08100219b0 addr v1:172.21.15.115:6790/0 2024-01-24T23:34:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:50.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f08100224a0 addr v1:172.21.15.132:6790/0 2024-01-24T23:34:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:50.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081001a360 addr v1:172.21.15.132:6792/0 2024-01-24T23:34:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:50.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000cfc0 con 0x7f08100219b0 2024-01-24T23:34:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:50.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f08100224a0 2024-01-24T23:34:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:50.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000ce30 con 0x7f081001a360 2024-01-24T23:34:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:50.169+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:34:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:50.169+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001a360 legacy=0x7f081001c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:50.169+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f08100224a0 legacy=0x7f08100228b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:50.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:50.169+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f08100219b0 legacy=0x7f0810021d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:50.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:50.373+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001a360 legacy=0x7f081001c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:50.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:50.373+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f08100224a0 legacy=0x7f08100228b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:50.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:50.373+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f08100219b0 legacy=0x7f0810021d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:50.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:50.777+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f08100224a0 legacy=0x7f08100228b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:50.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:50.777+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001a360 legacy=0x7f081001c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:50.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:50.777+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f08100219b0 legacy=0x7f0810021d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:51.370 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:34:51.370+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:34:51.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:51.581+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001a360 legacy=0x7f081001c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:51.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:51.581+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f08100224a0 legacy=0x7f08100228b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:51.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:51.581+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f08100219b0 legacy=0x7f0810021d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:34:53.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:53.169+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:34:53.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:53.169+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:34:53.164 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:53.169+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:34:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:53.169+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:34:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:53.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001a360 legacy=0x7f081001c7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:53.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f08100224a0 legacy=0x7f08100228b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:53.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f08100219b0 legacy=0x7f0810021d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:53.169+0000 7f0820ff9640 10 monclient: _add_conns ranks=[1,0,6] 2024-01-24T23:34:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:53.169+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:34:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:53.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f081000a750 addr v1:172.21.15.132:6789/0 2024-01-24T23:34:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:53.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f081001ced0 addr v1:172.21.15.115:6789/0 2024-01-24T23:34:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:53.169+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:34:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:53.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f081000e5d0 addr v1:172.21.15.115:6792/0 2024-01-24T23:34:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:53.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000d170 con 0x7f081001ced0 2024-01-24T23:34:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:53.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000cfc0 con 0x7f081000a750 2024-01-24T23:34:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:53.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000e5d0 2024-01-24T23:34:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:53.169+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:34:53.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:53.169+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:53.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:53.373+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:53.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:53.777+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:54.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:54.581+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:56.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:56.169+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:34:56.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:56.169+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:34:56.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:56.169+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:34:56.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:56.169+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:34:56.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:56.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:56.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:56.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:34:56.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:56.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081001ced0 legacy=0x7f081001d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:34:56.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:56.169+0000 7f0820ff9640 10 monclient: _add_conns ranks=[7,6,5,3,4,8,2,0,1] 2024-01-24T23:34:56.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:56.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f0810025380 addr v1:172.21.15.132:6792/0 2024-01-24T23:34:56.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:56.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f081000e5d0 addr v1:172.21.15.115:6792/0 2024-01-24T23:34:56.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:56.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f081000eee0 addr v1:172.21.15.132:6791/0 2024-01-24T23:34:56.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:56.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000ce30 con 0x7f081000eee0 2024-01-24T23:34:56.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:56.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000e5d0 2024-01-24T23:34:56.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:56.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000cfc0 con 0x7f0810025380 2024-01-24T23:34:56.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:56.169+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:34:56.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:56.169+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f0810025380 legacy=0x7f081001c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:56.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:56.169+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:56.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:56.169+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000eee0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:56.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:56.373+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f0810025380 legacy=0x7f081001c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:56.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:56.373+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000eee0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:56.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:56.373+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:56.370 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:34:56.370+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:34:56.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:56.777+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f0810025380 legacy=0x7f081001c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:56.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:56.777+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000eee0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:56.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:56.777+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:57.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:57.581+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f0810025380 legacy=0x7f081001c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:34:57.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:57.581+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000eee0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:34:57.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:57.581+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:34:59.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:59.173+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:34:59.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:59.173+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:34:59.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:59.173+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:34:59.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:59.173+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:34:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:59.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f0810025380 legacy=0x7f081001c7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:59.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:59.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000eee0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:34:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:59.173+0000 7f0820ff9640 10 monclient: _add_conns ranks=[3,1,8,0,4,2] 2024-01-24T23:34:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:59.173+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f081000f4d0 addr v1:172.21.15.132:6790/0 2024-01-24T23:34:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:59.173+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:34:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:59.173+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f081000a750 addr v1:172.21.15.132:6789/0 2024-01-24T23:34:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:59.173+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f081000eee0 addr v1:172.21.15.115:6793/0 2024-01-24T23:34:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:59.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000f2c0 con 0x7f081000a750 2024-01-24T23:34:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:59.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000ce30 con 0x7f081000f4d0 2024-01-24T23:34:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:59.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000eee0 2024-01-24T23:34:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:59.173+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000f4d0 legacy=0x7f0810020ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:59.173+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:34:59.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:59.173+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000eee0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:59.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:59.377+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000f4d0 legacy=0x7f0810020ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:59.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:59.377+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000eee0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:34:59.775 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:59.781+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000f4d0 legacy=0x7f0810020ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:34:59.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:34:59.781+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000eee0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:00.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:00.585+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000f4d0 legacy=0x7f0810020ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:00.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:00.585+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000eee0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:01.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:01.373+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:35:01.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:01.373+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:35:01.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:01.373+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000a750 0x7f0810011480 :-1 s=OPENED pgs=158 cs=1 l=1).handle_message read tag failed 2024-01-24T23:35:01.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:01.373+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000a750 0x7f0810011480 :-1 s=OPENED pgs=158 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:35:01.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:01.373+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:35:01.370 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:35:01.374+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:35:02.165 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:02.173+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:35:02.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:02.173+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:35:02.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:02.173+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:35:02.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:02.173+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:35:02.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:02.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000eee0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:02.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:02.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000f4d0 legacy=0x7f0810020ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:02.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:02.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:35:02.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:02.173+0000 7f0820ff9640 10 monclient: _add_conns ranks=[0,2,4] 2024-01-24T23:35:02.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:02.173+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f081000e5d0 addr v1:172.21.15.115:6789/0 2024-01-24T23:35:02.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:02.173+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f0810025380 addr v1:172.21.15.115:6790/0 2024-01-24T23:35:02.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:02.173+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f081000f8b0 addr v1:172.21.15.115:6791/0 2024-01-24T23:35:02.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:02.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000cfc0 con 0x7f081000e5d0 2024-01-24T23:35:02.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:02.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000ce30 con 0x7f0810025380 2024-01-24T23:35:02.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:02.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000f8b0 2024-01-24T23:35:02.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:02.173+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:35:02.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:02.173+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810025380 legacy=0x7f08100221d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:02.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:02.377+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810025380 legacy=0x7f08100221d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:02.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:02.781+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810025380 legacy=0x7f08100221d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:03.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:03.585+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810025380 legacy=0x7f08100221d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:05.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:05.169+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:35:05.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:05.169+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:35:05.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:05.169+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:35:05.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:05.169+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:35:05.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:05.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000f8b0 legacy=0x7f08100228e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:35:05.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:05.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810025380 legacy=0x7f08100221d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:05.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:05.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000e5d0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:35:05.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:05.169+0000 7f0820ff9640 10 monclient: _add_conns ranks=[4,8,5,1,6,2,7,3,0] 2024-01-24T23:35:05.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:05.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f0810017340 addr v1:172.21.15.115:6791/0 2024-01-24T23:35:05.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:05.169+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:35:05.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:05.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f081000eee0 addr v1:172.21.15.115:6793/0 2024-01-24T23:35:05.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:05.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f081000a750 addr v1:172.21.15.132:6791/0 2024-01-24T23:35:05.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:05.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000f2c0 con 0x7f0810017340 2024-01-24T23:35:05.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:05.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000ce30 con 0x7f081000a750 2024-01-24T23:35:05.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:05.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000eee0 2024-01-24T23:35:05.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:05.169+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:35:05.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:05.169+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:05.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:05.169+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000eee0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:05.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:05.373+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:05.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:05.373+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000eee0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:05.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:05.777+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:05.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:05.777+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000eee0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:06.370 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:35:06.370+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:35:06.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:06.377+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810017340 legacy=0x7f0810019730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:35:06.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:06.377+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810017340 legacy=0x7f0810019730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:35:06.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:06.377+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810017340 0x7f0810019730 :-1 s=OPENED pgs=164 cs=1 l=1).handle_message read tag failed 2024-01-24T23:35:06.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:06.377+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810017340 0x7f0810019730 :-1 s=OPENED pgs=164 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:35:06.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:06.377+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:35:06.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:06.581+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:06.573 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:06.581+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000eee0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:08.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:08.173+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:35:08.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:08.173+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:35:08.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:08.173+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:35:08.166 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:08.173+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:35:08.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:08.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000eee0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:08.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:08.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:08.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:08.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810017340 legacy=0x7f0810019730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:35:08.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:08.173+0000 7f0820ff9640 10 monclient: _add_conns ranks=[6,3,2,7,0,1] 2024-01-24T23:35:08.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:08.173+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f081000e5d0 addr v1:172.21.15.115:6792/0 2024-01-24T23:35:08.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:08.173+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f0810025380 addr v1:172.21.15.132:6790/0 2024-01-24T23:35:08.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:08.173+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f081001ae50 addr v1:172.21.15.115:6790/0 2024-01-24T23:35:08.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:08.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810025d20 con 0x7f081001ae50 2024-01-24T23:35:08.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:08.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000ce30 con 0x7f0810025380 2024-01-24T23:35:08.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:08.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000e5d0 2024-01-24T23:35:08.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:08.173+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:35:08.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:08.173+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f081000f4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:08.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:08.173+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081001ae50 legacy=0x7f081001d240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:08.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:08.173+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:08.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:08.377+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f081000f4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:08.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:08.377+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:08.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:08.377+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081001ae50 legacy=0x7f081001d240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:08.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:08.781+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f081000f4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:08.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:08.781+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:08.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:08.781+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081001ae50 legacy=0x7f081001d240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:09.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:09.585+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f081000f4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:09.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:09.585+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:09.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:09.585+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081001ae50 legacy=0x7f081001d240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:11.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.169+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:35:11.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.169+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:35:11.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.169+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:35:11.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.169+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:35:11.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000e5d0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:11.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f081000f4d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:11.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081001ae50 legacy=0x7f081001d240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:11.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.169+0000 7f0820ff9640 10 monclient: _add_conns ranks=[0,1,7] 2024-01-24T23:35:11.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f081001e960 addr v1:172.21.15.115:6789/0 2024-01-24T23:35:11.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.169+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:35:11.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f0810022470 addr v1:172.21.15.132:6789/0 2024-01-24T23:35:11.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.169+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:35:11.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081000a750 addr v1:172.21.15.132:6792/0 2024-01-24T23:35:11.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000f2c0 con 0x7f081001e960 2024-01-24T23:35:11.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810025d20 con 0x7f0810022470 2024-01-24T23:35:11.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000ce30 con 0x7f081000a750 2024-01-24T23:35:11.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.169+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:35:11.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.169+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:35:11.360 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.369+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081001e960 legacy=0x7f0810020d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:35:11.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.369+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081001e960 legacy=0x7f0810020d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:35:11.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.369+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081001e960 0x7f0810020d50 :-1 s=OPENED pgs=169 cs=1 l=1).handle_message read tag failed 2024-01-24T23:35:11.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.369+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081001e960 0x7f0810020d50 :-1 s=OPENED pgs=169 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:35:11.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.369+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:35:11.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.369+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810022470 legacy=0x7f0810022850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:35:11.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.369+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810022470 legacy=0x7f0810022850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:35:11.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.369+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810022470 0x7f0810022850 :-1 s=OPENED pgs=165 cs=1 l=1).handle_message read tag failed 2024-01-24T23:35:11.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.369+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810022470 0x7f0810022850 :-1 s=OPENED pgs=165 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:35:11.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.369+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:35:11.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.373+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:35:11.370 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:35:11.370+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:35:11.771 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:11.777+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:35:12.572 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:12.581+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:35:14.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:14.173+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:35:14.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:14.173+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:35:14.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:14.173+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:35:14.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:14.173+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:35:14.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:14.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:14.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:14.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810022470 legacy=0x7f0810022850 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:35:14.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:14.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081001e960 legacy=0x7f0810020d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:35:14.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:14.173+0000 7f0820ff9640 10 monclient: _add_conns ranks=[3,8,1,7,5,4,6,0,2] 2024-01-24T23:35:14.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:14.173+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f081000e5d0 addr v1:172.21.15.132:6790/0 2024-01-24T23:35:14.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:14.173+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f081000eee0 addr v1:172.21.15.115:6793/0 2024-01-24T23:35:14.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:14.173+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f0810022470 addr v1:172.21.15.132:6789/0 2024-01-24T23:35:14.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:14.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f0810022470 2024-01-24T23:35:14.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:14.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000ce30 con 0x7f081000e5d0 2024-01-24T23:35:14.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:14.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810025d20 con 0x7f081000eee0 2024-01-24T23:35:14.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:14.173+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:35:14.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:14.173+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:14.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:14.173+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000eee0 legacy=0x7f081000f4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:14.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:14.377+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:14.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:14.377+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000eee0 legacy=0x7f081000f4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:14.775 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:14.781+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:14.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:14.781+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000eee0 legacy=0x7f081000f4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:15.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:15.585+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:15.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:15.585+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000eee0 legacy=0x7f081000f4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:16.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:16.373+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810022470 legacy=0x7f0810022850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:35:16.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:16.373+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810022470 legacy=0x7f0810022850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:35:16.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:16.373+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810022470 0x7f0810022850 :-1 s=OPENED pgs=166 cs=1 l=1).handle_message read tag failed 2024-01-24T23:35:16.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:16.373+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810022470 0x7f0810022850 :-1 s=OPENED pgs=166 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:35:16.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:16.373+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:35:16.370 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:35:16.374+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:35:17.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:17.173+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:35:17.167 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:17.173+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:35:17.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:17.173+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:35:17.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:17.173+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:35:17.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:17.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000eee0 legacy=0x7f081000f4d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:17.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:17.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:17.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:17.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810022470 legacy=0x7f0810022850 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:35:17.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:17.173+0000 7f0820ff9640 10 monclient: _add_conns ranks=[6,4,7,2,5,0] 2024-01-24T23:35:17.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:17.173+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f0810025380 addr v1:172.21.15.115:6792/0 2024-01-24T23:35:17.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:17.173+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f081001a360 addr v1:172.21.15.115:6791/0 2024-01-24T23:35:17.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:17.173+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:35:17.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:17.173+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081000a750 addr v1:172.21.15.132:6792/0 2024-01-24T23:35:17.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:17.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000cfc0 con 0x7f081001a360 2024-01-24T23:35:17.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:17.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000ce30 con 0x7f0810025380 2024-01-24T23:35:17.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:17.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810025d20 con 0x7f081000a750 2024-01-24T23:35:17.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:17.173+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:35:17.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:17.173+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:35:17.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:17.173+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810025380 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:17.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:17.377+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:35:17.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:17.377+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810025380 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:17.775 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:17.781+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:35:17.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:17.781+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810025380 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:18.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:18.585+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:35:18.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:18.585+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810025380 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:20.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:20.169+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:35:20.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:20.169+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:35:20.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:20.169+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:35:20.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:20.169+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:35:20.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:20.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:20.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:20.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810025380 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:20.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:20.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001a360 legacy=0x7f081001c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:35:20.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:20.169+0000 7f0820ff9640 10 monclient: _add_conns ranks=[0,2,5] 2024-01-24T23:35:20.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:20.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f081000e5d0 addr v1:172.21.15.115:6789/0 2024-01-24T23:35:20.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:20.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f081000eee0 addr v1:172.21.15.115:6790/0 2024-01-24T23:35:20.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:20.169+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f0810015330 addr v1:172.21.15.132:6791/0 2024-01-24T23:35:20.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:20.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000e5d0 2024-01-24T23:35:20.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:20.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000ce30 con 0x7f081000eee0 2024-01-24T23:35:20.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:20.169+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810025d20 con 0x7f0810015330 2024-01-24T23:35:20.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:20.169+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:35:20.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:20.173+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810015330 legacy=0x7f0810015710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:20.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:20.173+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000eee0 legacy=0x7f081000f2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:20.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:20.377+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810015330 legacy=0x7f0810015710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:20.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:20.377+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000eee0 legacy=0x7f081000f2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:20.775 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:20.781+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810015330 legacy=0x7f0810015710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:20.775 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:20.781+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000eee0 legacy=0x7f081000f2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:21.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:21.369+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:35:21.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:21.369+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:35:21.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:21.369+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000e5d0 0x7f0810009fd0 :-1 s=OPENED pgs=170 cs=1 l=1).handle_message read tag failed 2024-01-24T23:35:21.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:21.369+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000e5d0 0x7f0810009fd0 :-1 s=OPENED pgs=170 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:35:21.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:21.369+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:35:21.371 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:35:21.373+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:35:21.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:21.585+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810015330 legacy=0x7f0810015710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:21.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:21.585+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000eee0 legacy=0x7f081000f2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:23.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.173+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:35:23.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.173+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:35:23.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.173+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:35:23.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.173+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:35:23.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810015330 legacy=0x7f0810015710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:23.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000eee0 legacy=0x7f081000f2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:23.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:35:23.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.173+0000 7f0820ff9640 10 monclient: _add_conns ranks=[2,8,6,3,7,1,0,4,5] 2024-01-24T23:35:23.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.173+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:35:23.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.173+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f081000eee0 addr v1:172.21.15.115:6790/0 2024-01-24T23:35:23.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.173+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f0810016e30 addr v1:172.21.15.115:6793/0 2024-01-24T23:35:23.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.173+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f0810025380 addr v1:172.21.15.115:6792/0 2024-01-24T23:35:23.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000cfc0 con 0x7f081000eee0 2024-01-24T23:35:23.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000ce30 con 0x7f0810025380 2024-01-24T23:35:23.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810025d20 con 0x7f0810016e30 2024-01-24T23:35:23.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.173+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:35:23.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.173+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:35:23.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.173+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000eee0 legacy=0x7f081000f2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:23.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.173+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810025380 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:23.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.173+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810016e30 legacy=0x7f0810021ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:23.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.377+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000eee0 legacy=0x7f081000f2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:23.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.377+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810016e30 legacy=0x7f0810021ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:23.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.377+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810025380 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:23.775 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.781+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810016e30 legacy=0x7f0810021ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:23.775 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.781+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000eee0 legacy=0x7f081000f2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:23.775 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:23.781+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810025380 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:24.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:24.585+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810016e30 legacy=0x7f0810021ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:24.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:24.585+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000eee0 legacy=0x7f081000f2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:24.577 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:24.585+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810025380 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:26.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.173+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:35:26.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.173+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:35:26.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.173+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:35:26.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.173+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:35:26.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810016e30 legacy=0x7f0810021ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:26.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810025380 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:26.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000eee0 legacy=0x7f081000f2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:26.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.173+0000 7f0820ff9640 10 monclient: _add_conns ranks=[1,7,0,5,3,4] 2024-01-24T23:35:26.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.173+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f081000e5d0 addr v1:172.21.15.132:6789/0 2024-01-24T23:35:26.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.173+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081000a750 addr v1:172.21.15.132:6792/0 2024-01-24T23:35:26.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.173+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f08100225d0 addr v1:172.21.15.115:6789/0 2024-01-24T23:35:26.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f08100225d0 2024-01-24T23:35:26.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000cfc0 con 0x7f081000e5d0 2024-01-24T23:35:26.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000ce30 con 0x7f081000a750 2024-01-24T23:35:26.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.173+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:35:26.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.173+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:35:26.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.369+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f08100225d0 legacy=0x7f0810015330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:35:26.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.369+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f08100225d0 legacy=0x7f0810015330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:35:26.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.369+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f08100225d0 0x7f0810015330 :-1 s=OPENED pgs=177 cs=1 l=1).handle_message read tag failed 2024-01-24T23:35:26.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.369+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f08100225d0 0x7f0810015330 :-1 s=OPENED pgs=177 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:35:26.361 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.369+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:35:26.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.369+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:35:26.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.369+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:35:26.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.369+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000e5d0 0x7f0810009fd0 :-1 s=OPENED pgs=173 cs=1 l=1).handle_message read tag failed 2024-01-24T23:35:26.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.369+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000e5d0 0x7f0810009fd0 :-1 s=OPENED pgs=173 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:35:26.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.369+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:35:26.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.369+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:35:26.371 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:35:26.373+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:35:26.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.377+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:35:26.772 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:26.781+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:35:27.579 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:27.585+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:35:29.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:29.177+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:35:29.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:29.177+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:35:29.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:29.177+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:35:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:29.177+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:35:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:29.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:29.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:35:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:29.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f08100225d0 legacy=0x7f0810015330 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:35:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:29.177+0000 7f0820ff9640 10 monclient: _add_conns ranks=[5,4,3] 2024-01-24T23:35:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:29.177+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f08100225d0 addr v1:172.21.15.132:6791/0 2024-01-24T23:35:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:29.177+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f081000e5d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:35:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:29.177+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f0810025380 addr v1:172.21.15.132:6790/0 2024-01-24T23:35:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:29.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810025d20 con 0x7f0810025380 2024-01-24T23:35:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:29.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000ce30 con 0x7f081000e5d0 2024-01-24T23:35:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:29.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000cfc0 con 0x7f08100225d0 2024-01-24T23:35:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:29.177+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:35:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:29.177+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f08100225d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:29.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:29.177+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:29.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:29.381+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f08100225d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:29.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:29.381+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:29.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:29.785+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:29.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:29.785+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f08100225d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:30.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:30.589+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f08100225d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:30.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:30.589+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:31.371 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:35:31.373+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:35:31.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:31.377+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000e5d0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:35:31.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:31.377+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000e5d0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:35:31.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:31.377+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000e5d0 0x7f0810005b80 :-1 s=OPENED pgs=178 cs=1 l=1).handle_message read tag failed 2024-01-24T23:35:31.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:31.377+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000e5d0 0x7f0810005b80 :-1 s=OPENED pgs=178 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:35:31.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:31.377+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:35:32.168 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:32.177+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:35:32.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:32.177+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:35:32.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:32.177+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:35:32.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:32.177+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:35:32.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:32.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f08100225d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:32.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:32.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000e5d0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:35:32.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:32.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:32.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:32.177+0000 7f0820ff9640 10 monclient: _add_conns ranks=[4,6,1,2,3,7,0,8,5] 2024-01-24T23:35:32.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:32.177+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f081000e5d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:35:32.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:32.177+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f08100229b0 addr v1:172.21.15.115:6792/0 2024-01-24T23:35:32.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:32.177+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f081000f5f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:35:32.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:32.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810023050 con 0x7f081000f5f0 2024-01-24T23:35:32.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:32.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810025d20 con 0x7f081000e5d0 2024-01-24T23:35:32.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:32.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000cfc0 con 0x7f08100229b0 2024-01-24T23:35:32.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:32.177+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:35:32.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:32.177+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f08100229b0 legacy=0x7f081001a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:32.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:32.381+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f08100229b0 legacy=0x7f081001a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:32.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:32.785+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f08100229b0 legacy=0x7f081001a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:33.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:33.589+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f08100229b0 legacy=0x7f081001a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:35.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:35.173+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:35:35.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:35.173+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:35:35.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:35.173+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:35:35.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:35.173+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:35:35.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:35.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f08100229b0 legacy=0x7f081001a360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:35.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:35.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000e5d0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:35:35.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:35.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000f5f0 legacy=0x7f081001da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:35:35.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:35.173+0000 7f0820ff9640 10 monclient: _add_conns ranks=[3,2,0,5,7,8] 2024-01-24T23:35:35.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:35.173+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:35:35.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:35.173+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f0810025380 addr v1:172.21.15.132:6790/0 2024-01-24T23:35:35.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:35.173+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f081001f1b0 addr v1:172.21.15.115:6790/0 2024-01-24T23:35:35.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:35.173+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:35:35.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:35.173+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f081000a750 addr v1:172.21.15.115:6789/0 2024-01-24T23:35:35.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:35.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000a750 2024-01-24T23:35:35.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:35.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000cfc0 con 0x7f081001f1b0 2024-01-24T23:35:35.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:35.173+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810023050 con 0x7f0810025380 2024-01-24T23:35:35.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:35.173+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:35:35.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:35.173+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:35.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:35.173+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081001f1b0 legacy=0x7f08100215a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:35.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:35.377+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:35.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:35.377+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081001f1b0 legacy=0x7f08100215a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:35.775 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:35.781+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:35.775 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:35.781+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081001f1b0 legacy=0x7f08100215a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:36.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:36.369+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:35:36.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:36.369+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:35:36.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:36.369+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000a750 0x7f0810011480 :-1 s=OPENED pgs=178 cs=1 l=1).handle_message read tag failed 2024-01-24T23:35:36.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:36.369+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000a750 0x7f0810011480 :-1 s=OPENED pgs=178 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:35:36.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:36.369+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:35:36.371 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:35:36.373+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:35:36.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:36.585+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:36.576 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:36.585+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081001f1b0 legacy=0x7f08100215a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:38.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:38.177+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:35:38.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:38.177+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:35:38.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:38.177+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:35:38.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:38.177+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:35:38.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:38.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:38.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:38.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081001f1b0 legacy=0x7f08100215a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:38.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:38.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000a750 legacy=0x7f0810011480 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:35:38.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:38.177+0000 7f0820ff9640 10 monclient: _add_conns ranks=[8,7,5] 2024-01-24T23:35:38.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:38.177+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f081000e5d0 addr v1:172.21.15.115:6793/0 2024-01-24T23:35:38.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:38.177+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081000f5f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:35:38.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:38.177+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f0810021cb0 addr v1:172.21.15.132:6791/0 2024-01-24T23:35:38.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:38.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810025d20 con 0x7f0810021cb0 2024-01-24T23:35:38.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:38.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000cfc0 con 0x7f081000f5f0 2024-01-24T23:35:38.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:38.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810023050 con 0x7f081000e5d0 2024-01-24T23:35:38.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:38.177+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:35:38.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:38.177+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000f5f0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:35:38.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:38.177+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810021cb0 legacy=0x7f0810022160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:38.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:38.177+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:38.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:38.381+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810021cb0 legacy=0x7f0810022160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:38.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:38.381+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000f5f0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:35:38.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:38.381+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:38.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:38.785+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810021cb0 legacy=0x7f0810022160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:38.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:38.785+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000f5f0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:35:38.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:38.785+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:39.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:39.589+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810021cb0 legacy=0x7f0810022160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:39.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:39.589+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000f5f0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:35:39.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:39.589+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:41.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.177+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:35:41.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.177+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:35:41.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.177+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:35:41.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.177+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:35:41.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000e5d0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:41.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000f5f0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:41.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810021cb0 legacy=0x7f0810022160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:41.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.177+0000 7f0820ff9640 10 monclient: _add_conns ranks=[8,3,7,4,0,2,6,1,5] 2024-01-24T23:35:41.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.177+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f0810022870 addr v1:172.21.15.115:6793/0 2024-01-24T23:35:41.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.177+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f0810025380 addr v1:172.21.15.132:6790/0 2024-01-24T23:35:41.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.177+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:35:41.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.177+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081000f5f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:35:41.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f0810025380 2024-01-24T23:35:41.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810025d20 con 0x7f081000f5f0 2024-01-24T23:35:41.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000cfc0 con 0x7f0810022870 2024-01-24T23:35:41.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.177+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:35:41.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.177+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:35:41.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.177+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:41.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.177+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000f5f0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:35:41.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.177+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810022870 legacy=0x7f081001d380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:41.371 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:35:41.373+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:35:41.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.381+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000f5f0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:35:41.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.381+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:41.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.381+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810022870 legacy=0x7f081001d380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:41.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.785+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000f5f0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:35:41.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.785+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:41.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:41.785+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810022870 legacy=0x7f081001d380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:42.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:42.589+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000f5f0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:35:42.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:42.589+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:42.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:42.589+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810022870 legacy=0x7f081001d380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:44.169 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:44.177+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:35:44.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:44.177+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:35:44.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:44.177+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:35:44.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:44.177+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:35:44.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:44.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810022870 legacy=0x7f081001d380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:44.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:44.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000f5f0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:44.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:44.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:44.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:44.177+0000 7f0820ff9640 10 monclient: _add_conns ranks=[0,4,1,5,6,2] 2024-01-24T23:35:44.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:44.177+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f081000a750 addr v1:172.21.15.115:6789/0 2024-01-24T23:35:44.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:44.177+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f081000f9d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:35:44.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:44.177+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f0810021ac0 addr v1:172.21.15.132:6789/0 2024-01-24T23:35:44.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:44.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810023050 con 0x7f081000a750 2024-01-24T23:35:44.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:44.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f0810021ac0 2024-01-24T23:35:44.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:44.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810025d20 con 0x7f081000f9d0 2024-01-24T23:35:44.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:44.177+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:35:46.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:46.365+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000a750 legacy=0x7f0810022160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:35:46.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:46.365+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000a750 legacy=0x7f0810022160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:35:46.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:46.365+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000a750 0x7f0810022160 :-1 s=OPENED pgs=185 cs=1 l=1).handle_message read tag failed 2024-01-24T23:35:46.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:46.365+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000a750 0x7f0810022160 :-1 s=OPENED pgs=185 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:35:46.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:46.365+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:35:46.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:46.369+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810021ac0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:35:46.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:46.369+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810021ac0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:35:46.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:46.369+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810021ac0 0x7f0810005b80 :-1 s=OPENED pgs=181 cs=1 l=1).handle_message read tag failed 2024-01-24T23:35:46.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:46.369+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810021ac0 0x7f0810005b80 :-1 s=OPENED pgs=181 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:35:46.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:46.369+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:35:46.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:46.369+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:35:46.371 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:35:46.373+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:35:46.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:46.377+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000f9d0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:35:46.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:46.377+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000f9d0 legacy=0x7f0810011480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:35:46.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:46.377+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000f9d0 0x7f0810011480 :-1 s=OPENED pgs=186 cs=1 l=1).handle_message read tag failed 2024-01-24T23:35:46.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:46.377+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000f9d0 0x7f0810011480 :-1 s=OPENED pgs=186 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:35:46.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:46.377+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:35:47.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:47.177+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:35:47.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:47.177+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:35:47.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:47.177+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:35:47.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:47.177+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:35:47.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:47.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000f9d0 legacy=0x7f0810011480 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:35:47.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:47.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810021ac0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:35:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:47.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000a750 legacy=0x7f0810022160 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:35:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:47.177+0000 7f0820ff9640 10 monclient: _add_conns ranks=[6,2,5] 2024-01-24T23:35:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:47.177+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f081000a750 addr v1:172.21.15.115:6792/0 2024-01-24T23:35:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:47.177+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f0810021ac0 addr v1:172.21.15.115:6790/0 2024-01-24T23:35:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:47.177+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f081000f5f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:35:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:47.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000cfc0 con 0x7f0810021ac0 2024-01-24T23:35:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:47.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000f5f0 2024-01-24T23:35:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:47.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810025d20 con 0x7f081000a750 2024-01-24T23:35:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:47.177+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:35:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:47.177+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000f5f0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:47.177+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810021ac0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:47.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:47.177+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000a750 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:47.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:47.381+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000f5f0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:47.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:47.381+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810021ac0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:47.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:47.381+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000a750 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:47.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:47.785+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000f5f0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:47.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:47.785+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810021ac0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:47.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:47.785+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000a750 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:48.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:48.589+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000f5f0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:48.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:48.589+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000a750 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:48.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:48.589+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810021ac0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:50.177+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:35:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:50.177+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:35:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:50.177+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:35:50.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:50.177+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:35:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:50.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000a750 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:50.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000f5f0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:50.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810021ac0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:50.177+0000 7f0820ff9640 10 monclient: _add_conns ranks=[3,5,1,6,4,7,2,0,8] 2024-01-24T23:35:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:50.177+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f0810025380 addr v1:172.21.15.132:6790/0 2024-01-24T23:35:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:50.177+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f0810017a50 addr v1:172.21.15.132:6791/0 2024-01-24T23:35:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:50.177+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f081000f5f0 addr v1:172.21.15.132:6789/0 2024-01-24T23:35:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:50.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810021ed0 con 0x7f081000f5f0 2024-01-24T23:35:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:50.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000cfc0 con 0x7f0810025380 2024-01-24T23:35:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:50.177+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f0810017a50 2024-01-24T23:35:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:50.177+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:35:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:50.177+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810017a50 legacy=0x7f0810019e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:50.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:50.177+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:50.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:50.380+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810017a50 legacy=0x7f0810019e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:50.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:50.380+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:50.775 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:50.784+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810017a50 legacy=0x7f0810019e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:50.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:50.784+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:51.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:51.368+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000f5f0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:35:51.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:51.372+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000f5f0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:35:51.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:51.372+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000f5f0 0x7f0810005b80 :-1 s=OPENED pgs=182 cs=1 l=1).handle_message read tag failed 2024-01-24T23:35:51.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:51.372+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000f5f0 0x7f0810005b80 :-1 s=OPENED pgs=182 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:35:51.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:51.372+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:35:51.371 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:35:51.373+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:35:51.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:51.588+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:35:51.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:51.588+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810017a50 legacy=0x7f0810019e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:53.170 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:53.176+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:35:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:53.176+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:35:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:53.176+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:35:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:53.176+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:35:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:53.176+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810017a50 legacy=0x7f0810019e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:53.176+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810025380 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:53.176+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000f5f0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:35:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:53.176+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:35:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:53.176+0000 7f0820ff9640 10 monclient: _add_conns ranks=[0,6,8,2,4,7] 2024-01-24T23:35:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:53.176+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f081000a750 addr v1:172.21.15.115:6789/0 2024-01-24T23:35:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:53.176+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f0810011480 addr v1:172.21.15.115:6792/0 2024-01-24T23:35:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:53.176+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f081000c080 addr v1:172.21.15.115:6793/0 2024-01-24T23:35:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:53.176+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810025d20 con 0x7f081000a750 2024-01-24T23:35:53.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:53.176+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000cfc0 con 0x7f0810011480 2024-01-24T23:35:53.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:53.176+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000c080 2024-01-24T23:35:53.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:53.176+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:35:53.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:53.176+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000c080 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:53.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:53.176+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810011480 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:53.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:53.380+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000c080 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:53.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:53.380+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810011480 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:53.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:53.784+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000c080 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:53.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:53.784+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810011480 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:54.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:54.588+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000c080 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:35:54.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:54.588+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810011480 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:56.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.176+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:35:56.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.176+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:35:56.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.176+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:35:56.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.176+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:35:56.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.176+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000c080 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:56.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.176+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810011480 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:56.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.176+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000a750 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:35:56.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.176+0000 7f0820ff9640 10 monclient: _add_conns ranks=[2,4,7] 2024-01-24T23:35:56.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.176+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f081000c4c0 addr v1:172.21.15.115:6790/0 2024-01-24T23:35:56.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.176+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f0810017a50 addr v1:172.21.15.115:6791/0 2024-01-24T23:35:56.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.176+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081001b610 addr v1:172.21.15.132:6792/0 2024-01-24T23:35:56.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.176+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810011860 con 0x7f081000c4c0 2024-01-24T23:35:56.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.176+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000cfc0 con 0x7f0810017a50 2024-01-24T23:35:56.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.176+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081001b610 2024-01-24T23:35:56.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.176+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:35:56.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.176+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b610 legacy=0x7f081001da30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:35:56.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.176+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000c4c0 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:56.371 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:35:56.373+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:35:56.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.376+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810017a50 legacy=0x7f0810019ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:35:56.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.376+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810017a50 legacy=0x7f0810019ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:35:56.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.376+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810017a50 0x7f0810019ef0 :-1 s=OPENED pgs=193 cs=1 l=1).handle_message read tag failed 2024-01-24T23:35:56.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.376+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810017a50 0x7f0810019ef0 :-1 s=OPENED pgs=193 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:35:56.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.376+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:35:56.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.380+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b610 legacy=0x7f081001da30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:35:56.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.380+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000c4c0 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:56.775 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.784+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b610 legacy=0x7f081001da30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:35:56.776 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:56.784+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000c4c0 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:57.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:57.588+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b610 legacy=0x7f081001da30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:35:57.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:57.588+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000c4c0 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:35:59.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:59.176+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:35:59.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:59.176+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:35:59.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:59.176+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:35:59.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:59.176+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:35:59.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:59.176+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b610 legacy=0x7f081001da30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:59.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:59.176+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810017a50 legacy=0x7f0810019ef0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:35:59.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:59.176+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000c4c0 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:35:59.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:59.176+0000 7f0820ff9640 10 monclient: _add_conns ranks=[5,4,6,8,2,1,3,7,0] 2024-01-24T23:35:59.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:59.176+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:35:59.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:59.176+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f081001f150 addr v1:172.21.15.132:6791/0 2024-01-24T23:35:59.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:59.176+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f0810017a50 addr v1:172.21.15.115:6791/0 2024-01-24T23:35:59.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:59.176+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f0810011480 addr v1:172.21.15.115:6792/0 2024-01-24T23:35:59.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:59.176+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810025d20 con 0x7f0810017a50 2024-01-24T23:35:59.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:59.176+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810011860 con 0x7f081001f150 2024-01-24T23:35:59.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:59.176+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f0810011480 2024-01-24T23:35:59.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:59.176+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:35:59.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:59.176+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:35:59.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:59.176+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081001f150 legacy=0x7f0810021540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:59.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:59.176+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810011480 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:59.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:59.380+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081001f150 legacy=0x7f0810021540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:59.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:59.380+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810011480 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:35:59.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:59.784+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081001f150 legacy=0x7f0810021540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:35:59.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:35:59.784+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810011480 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:00.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:00.588+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081001f150 legacy=0x7f0810021540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:36:00.580 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:00.588+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810011480 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:01.372 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:36:01.373+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:36:01.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:01.376+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810017a50 legacy=0x7f0810019ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:36:01.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:01.376+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810017a50 legacy=0x7f0810019ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:36:01.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:01.376+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810017a50 0x7f0810019ef0 :-1 s=OPENED pgs=194 cs=1 l=1).handle_message read tag failed 2024-01-24T23:36:01.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:01.376+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810017a50 0x7f0810019ef0 :-1 s=OPENED pgs=194 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:36:01.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:01.376+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:36:02.171 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:02.180+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:36:02.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:02.180+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:36:02.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:02.180+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:36:02.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:02.180+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:36:02.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:02.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810011480 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:02.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:02.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081001f150 legacy=0x7f0810021540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:02.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:02.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810017a50 legacy=0x7f0810019ef0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:36:02.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:02.180+0000 7f0820ff9640 10 monclient: _add_conns ranks=[3,2,1,0,8,7] 2024-01-24T23:36:02.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:02.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f081000c4c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:36:02.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:02.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f081000a750 addr v1:172.21.15.115:6790/0 2024-01-24T23:36:02.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:02.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f081000c080 addr v1:172.21.15.132:6789/0 2024-01-24T23:36:02.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:02.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810011a30 con 0x7f081000c080 2024-01-24T23:36:02.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:02.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810011860 con 0x7f081000a750 2024-01-24T23:36:02.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:02.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000c4c0 2024-01-24T23:36:02.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:02.180+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:36:02.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:02.180+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000c4c0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:02.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:02.180+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000a750 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:02.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:02.384+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000c4c0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:02.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:02.384+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000a750 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:02.780 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:02.788+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000c4c0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:02.780 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:02.788+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000a750 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:03.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:03.592+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000c4c0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:03.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:03.592+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000a750 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:05.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:05.180+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:36:05.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:05.180+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:36:05.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:05.180+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:36:05.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:05.180+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:36:05.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:05.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000c4c0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:05.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:05.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000a750 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:05.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:05.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000c080 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:36:05.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:05.180+0000 7f0820ff9640 10 monclient: _add_conns ranks=[7,8,0] 2024-01-24T23:36:05.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:05.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081001b610 addr v1:172.21.15.132:6792/0 2024-01-24T23:36:05.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:05.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f0810021c50 addr v1:172.21.15.115:6793/0 2024-01-24T23:36:05.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:05.180+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:36:05.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:05.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f081002a7a0 addr v1:172.21.15.115:6789/0 2024-01-24T23:36:05.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:05.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810025cd0 con 0x7f081002a7a0 2024-01-24T23:36:05.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:05.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810011860 con 0x7f081001b610 2024-01-24T23:36:05.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:05.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f0810021c50 2024-01-24T23:36:05.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:05.180+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:36:05.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:05.180+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:36:05.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:05.180+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b610 legacy=0x7f081001da30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:05.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:05.180+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810021c50 legacy=0x7f0810029080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:05.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:05.384+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b610 legacy=0x7f081001da30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:05.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:05.384+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810021c50 legacy=0x7f0810029080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:05.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:05.788+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b610 legacy=0x7f081001da30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:05.780 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:05.788+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810021c50 legacy=0x7f0810029080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:06.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:06.368+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081002a7a0 legacy=0x7f081002cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:36:06.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:06.368+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081002a7a0 legacy=0x7f081002cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:36:06.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:06.368+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081002a7a0 0x7f081002cb90 :-1 s=OPENED pgs=193 cs=1 l=1).handle_message read tag failed 2024-01-24T23:36:06.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:06.368+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081002a7a0 0x7f081002cb90 :-1 s=OPENED pgs=193 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:36:06.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:06.368+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:36:06.372 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:36:06.373+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:36:06.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:06.592+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b610 legacy=0x7f081001da30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:06.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:06.592+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810021c50 legacy=0x7f0810029080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:08.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:08.180+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:36:08.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:08.180+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:36:08.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:08.180+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:36:08.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:08.180+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:36:08.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:08.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810021c50 legacy=0x7f0810029080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:08.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:08.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b610 legacy=0x7f081001da30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:08.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:08.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081002a7a0 legacy=0x7f081002cb90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:36:08.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:08.180+0000 7f0820ff9640 10 monclient: _add_conns ranks=[2,8,6,1,4,0,7,5,3] 2024-01-24T23:36:08.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:08.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f081000a700 addr v1:172.21.15.115:6790/0 2024-01-24T23:36:08.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:08.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f081000c4a0 addr v1:172.21.15.115:6793/0 2024-01-24T23:36:08.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:08.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f081000c080 addr v1:172.21.15.115:6792/0 2024-01-24T23:36:08.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:08.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810011480 con 0x7f081000a700 2024-01-24T23:36:08.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:08.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810011860 con 0x7f081000c080 2024-01-24T23:36:08.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:08.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000c4a0 2024-01-24T23:36:08.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:08.180+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:36:08.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:08.180+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000a700 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:08.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:08.180+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000c080 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:08.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:08.180+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000c4a0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:08.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:08.384+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000a700 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:08.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:08.384+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000c4a0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:08.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:08.384+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000c080 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:08.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:08.788+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000a700 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:08.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:08.788+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000c080 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:08.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:08.788+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000c4a0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:09.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:09.592+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000a700 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:09.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:09.592+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000c4a0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:09.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:09.592+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000c080 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:11.172 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.180+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:36:11.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.180+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:36:11.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.180+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:36:11.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.180+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:36:11.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000c4a0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:11.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000c080 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:11.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000a700 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:11.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.180+0000 7f0820ff9640 10 monclient: _add_conns ranks=[0,5,7,3,4,1] 2024-01-24T23:36:11.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f081002a7a0 addr v1:172.21.15.115:6789/0 2024-01-24T23:36:11.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.180+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:36:11.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f0810021870 addr v1:172.21.15.132:6791/0 2024-01-24T23:36:11.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081000c4a0 addr v1:172.21.15.132:6792/0 2024-01-24T23:36:11.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810025cd0 con 0x7f081002a7a0 2024-01-24T23:36:11.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810011480 con 0x7f0810021870 2024-01-24T23:36:11.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810011860 con 0x7f081000c4a0 2024-01-24T23:36:11.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.180+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:36:11.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.180+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810021870 legacy=0x7f0810029080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:36:11.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.180+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000c4a0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:11.362 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.368+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081002a7a0 legacy=0x7f0810021160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:36:11.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.368+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081002a7a0 legacy=0x7f0810021160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:36:11.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.368+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081002a7a0 0x7f0810021160 :-1 s=OPENED pgs=200 cs=1 l=1).handle_message read tag failed 2024-01-24T23:36:11.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.368+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081002a7a0 0x7f0810021160 :-1 s=OPENED pgs=200 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:36:11.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.368+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:36:11.372 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:36:11.373+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:36:11.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.384+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000c4a0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:11.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.384+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810021870 legacy=0x7f0810029080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:36:11.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.788+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810021870 legacy=0x7f0810029080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:36:11.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:11.788+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000c4a0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:12.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:12.592+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000c4a0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:12.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:12.592+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810021870 legacy=0x7f0810029080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:36:14.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:14.180+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:36:14.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:14.180+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:36:14.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:14.180+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:36:14.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:14.180+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:36:14.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:14.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000c4a0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:14.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:14.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810021870 legacy=0x7f0810029080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:14.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:14.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081002a7a0 legacy=0x7f0810021160 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:36:14.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:14.180+0000 7f0820ff9640 10 monclient: _add_conns ranks=[1,3,4] 2024-01-24T23:36:14.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:14.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f081000c080 addr v1:172.21.15.132:6789/0 2024-01-24T23:36:14.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:14.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f081000a6e0 addr v1:172.21.15.132:6790/0 2024-01-24T23:36:14.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:14.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f081001ba20 addr v1:172.21.15.115:6791/0 2024-01-24T23:36:14.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:14.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000c080 2024-01-24T23:36:14.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:14.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810011480 con 0x7f081000a6e0 2024-01-24T23:36:14.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:14.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810011860 con 0x7f081001ba20 2024-01-24T23:36:14.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:14.180+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:36:14.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:14.180+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a6e0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:14.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:14.180+0000 7f0822ffd640 0 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000c080 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2024-01-24T23:36:14.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:14.180+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000c080 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-01-24T23:36:14.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:14.180+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000c080 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-24T23:36:14.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:14.180+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000c080 0x7f0810009fd0 :-1 s=CONNECTING_SEND_CONNECT_MSG pgs=0 cs=0 l=0).handle_connect_reply_1 read connect reply failed 2024-01-24T23:36:14.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:14.384+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a6e0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:14.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:14.788+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a6e0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:15.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:15.592+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a6e0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:16.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:16.376+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000c080 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:36:16.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:16.376+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000c080 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:36:16.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:16.376+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000c080 0x7f0810009fd0 :-1 s=OPENED pgs=197 cs=1 l=1).handle_message read tag failed 2024-01-24T23:36:16.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:16.376+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000c080 0x7f0810009fd0 :-1 s=OPENED pgs=197 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:36:16.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:16.376+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:36:16.372 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:36:16.372+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:36:16.372 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:16.380+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001ba20 legacy=0x7f081001de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:36:16.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:16.380+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001ba20 legacy=0x7f081001de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:36:16.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:16.380+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001ba20 0x7f081001de40 :-1 s=OPENED pgs=201 cs=1 l=1).handle_message read tag failed 2024-01-24T23:36:16.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:16.380+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001ba20 0x7f081001de40 :-1 s=OPENED pgs=201 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:36:16.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:16.380+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:36:16.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:16.380+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:36:17.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:17.180+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:36:17.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:17.180+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:36:17.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:17.180+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:36:17.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:17.180+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:36:17.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:17.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001ba20 legacy=0x7f081001de40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:36:17.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:17.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a6e0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:17.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:17.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000c080 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:36:17.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:17.180+0000 7f0820ff9640 10 monclient: _add_conns ranks=[6,5,8,4,0,2,7,1,3] 2024-01-24T23:36:17.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:17.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f081000c080 addr v1:172.21.15.115:6792/0 2024-01-24T23:36:17.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:17.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f081001ba20 addr v1:172.21.15.132:6791/0 2024-01-24T23:36:17.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:17.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f0810011610 addr v1:172.21.15.115:6793/0 2024-01-24T23:36:17.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:17.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810025cd0 con 0x7f081001ba20 2024-01-24T23:36:17.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:17.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810011480 con 0x7f081000c080 2024-01-24T23:36:17.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:17.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f0810011610 2024-01-24T23:36:17.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:17.180+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:36:17.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:17.180+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081001ba20 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:36:17.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:17.180+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810011610 legacy=0x7f081001de40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:17.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:17.180+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000c080 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:17.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:17.384+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081001ba20 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:36:17.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:17.384+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810011610 legacy=0x7f081001de40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:17.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:17.384+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000c080 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:17.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:17.788+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081001ba20 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:36:17.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:17.788+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810011610 legacy=0x7f081001de40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:17.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:17.788+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000c080 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:18.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:18.592+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081001ba20 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:36:18.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:18.592+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810011610 legacy=0x7f081001de40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:18.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:18.592+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000c080 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:20.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:20.180+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:36:20.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:20.180+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:36:20.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:20.180+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:36:20.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:20.180+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:36:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:20.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810011610 legacy=0x7f081001de40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:20.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000c080 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:20.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081001ba20 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:20.180+0000 7f0820ff9640 10 monclient: _add_conns ranks=[7,3,4,0,1,2] 2024-01-24T23:36:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:20.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081000c4a0 addr v1:172.21.15.132:6792/0 2024-01-24T23:36:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:20.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f081000a6e0 addr v1:172.21.15.132:6790/0 2024-01-24T23:36:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:20.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f0810021570 addr v1:172.21.15.115:6791/0 2024-01-24T23:36:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:20.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100106b0 con 0x7f081000a6e0 2024-01-24T23:36:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:20.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810025cd0 con 0x7f0810021570 2024-01-24T23:36:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:20.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810011480 con 0x7f081000c4a0 2024-01-24T23:36:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:20.180+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:36:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:20.180+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000c4a0 legacy=0x7f0810014320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:20.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:20.180+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a6e0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:20.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:20.384+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a6e0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:20.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:20.384+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000c4a0 legacy=0x7f0810014320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:20.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:20.788+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a6e0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:20.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:20.788+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000c4a0 legacy=0x7f0810014320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:21.372 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:36:21.372+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:36:21.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:21.380+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810021570 legacy=0x7f0810017a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:36:21.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:21.380+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810021570 legacy=0x7f0810017a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:36:21.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:21.380+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810021570 0x7f0810017a50 :-1 s=OPENED pgs=202 cs=1 l=1).handle_message read tag failed 2024-01-24T23:36:21.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:21.380+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810021570 0x7f0810017a50 :-1 s=OPENED pgs=202 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:36:21.373 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:21.380+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:36:21.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:21.592+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000c4a0 legacy=0x7f0810014320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:21.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:21.592+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a6e0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:23.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:23.180+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:36:23.173 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:23.180+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:36:23.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:23.180+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:36:23.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:23.180+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:36:23.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:23.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000c4a0 legacy=0x7f0810014320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:23.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:23.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810021570 legacy=0x7f0810017a50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:36:23.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:23.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a6e0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:23.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:23.180+0000 7f0820ff9640 10 monclient: _add_conns ranks=[0,1,2] 2024-01-24T23:36:23.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:23.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f0810019170 addr v1:172.21.15.115:6789/0 2024-01-24T23:36:23.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:23.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f081001b560 addr v1:172.21.15.132:6789/0 2024-01-24T23:36:23.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:23.180+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:36:23.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:23.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f081002aeb0 addr v1:172.21.15.115:6790/0 2024-01-24T23:36:23.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:23.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f0810019170 2024-01-24T23:36:23.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:23.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100106b0 con 0x7f081001b560 2024-01-24T23:36:23.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:23.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810011480 con 0x7f081002aeb0 2024-01-24T23:36:23.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:23.180+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:36:23.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:23.180+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081002aeb0 legacy=0x7f081002d2a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:23.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:23.384+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081002aeb0 legacy=0x7f081002d2a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:23.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:23.788+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081002aeb0 legacy=0x7f081002d2a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:24.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:24.592+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081002aeb0 legacy=0x7f081002d2a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:26.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.180+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:36:26.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.180+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:36:26.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.180+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:36:26.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.180+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:36:26.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081002aeb0 legacy=0x7f081002d2a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:26.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081001b560 legacy=0x7f0810029790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:36:26.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f0810019170 legacy=0x7f0810026060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:36:26.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.180+0000 7f0820ff9640 10 monclient: _add_conns ranks=[8,0,7,3,4,1,2,6,5] 2024-01-24T23:36:26.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f081001ba20 addr v1:172.21.15.115:6793/0 2024-01-24T23:36:26.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f081000c4a0 addr v1:172.21.15.115:6789/0 2024-01-24T23:36:26.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f0810011610 addr v1:172.21.15.132:6792/0 2024-01-24T23:36:26.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810021980 con 0x7f081000c4a0 2024-01-24T23:36:26.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810011480 con 0x7f0810011610 2024-01-24T23:36:26.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081001ba20 2024-01-24T23:36:26.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.180+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:36:26.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.180+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f0810011610 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:26.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.180+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081001ba20 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:26.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.372+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000c4a0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:36:26.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.372+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000c4a0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:36:26.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.372+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000c4a0 0x7f081000b970 :-1 s=OPENED pgs=208 cs=1 l=1).handle_message read tag failed 2024-01-24T23:36:26.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.372+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000c4a0 0x7f081000b970 :-1 s=OPENED pgs=208 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:36:26.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.372+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:36:26.372 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:36:26.372+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:36:26.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.384+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f0810011610 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:26.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.384+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081001ba20 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:26.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.788+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f0810011610 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:26.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:26.788+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081001ba20 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:27.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:27.592+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f0810011610 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:27.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:27.592+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081001ba20 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:29.180+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:36:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:29.180+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:36:29.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:29.180+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:36:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:29.180+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:36:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:29.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081001ba20 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:29.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f0810011610 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:29.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000c4a0 legacy=0x7f081000b970 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:36:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:29.180+0000 7f0820ff9640 10 monclient: _add_conns ranks=[1,2,3,6,4,5] 2024-01-24T23:36:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:29.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f081001b560 addr v1:172.21.15.132:6789/0 2024-01-24T23:36:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:29.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f081002aeb0 addr v1:172.21.15.115:6790/0 2024-01-24T23:36:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:29.180+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:36:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:29.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f081000a6e0 addr v1:172.21.15.132:6790/0 2024-01-24T23:36:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:29.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f081001b560 2024-01-24T23:36:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:29.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810011480 con 0x7f081002aeb0 2024-01-24T23:36:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:29.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000a6e0 2024-01-24T23:36:29.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:29.180+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:36:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:29.180+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a6e0 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:29.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:29.180+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081002aeb0 legacy=0x7f0810029790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:29.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:29.384+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a6e0 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:29.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:29.384+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081002aeb0 legacy=0x7f0810029790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:29.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:29.788+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a6e0 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:29.779 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:29.788+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081002aeb0 legacy=0x7f0810029790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:30.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:30.592+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a6e0 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:30.583 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:30.592+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081002aeb0 legacy=0x7f0810029790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:31.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:31.376+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081001b560 legacy=0x7f0810026060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:36:31.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:31.376+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081001b560 legacy=0x7f0810026060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:36:31.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:31.376+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081001b560 0x7f0810026060 :-1 s=OPENED pgs=205 cs=1 l=1).handle_message read tag failed 2024-01-24T23:36:31.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:31.376+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081001b560 0x7f0810026060 :-1 s=OPENED pgs=205 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:36:31.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:31.376+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:36:31.372 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:36:31.376+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:36:32.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:32.180+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:36:32.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:32.180+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:36:32.174 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:32.180+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:36:32.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:32.180+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:36:32.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:32.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000a6e0 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:32.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:32.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081002aeb0 legacy=0x7f0810029790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:32.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:32.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081001b560 legacy=0x7f0810026060 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:36:32.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:32.180+0000 7f0820ff9640 10 monclient: _add_conns ranks=[6,5,4] 2024-01-24T23:36:32.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:32.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f081000c080 addr v1:172.21.15.115:6792/0 2024-01-24T23:36:32.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:32.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f081000c4a0 addr v1:172.21.15.132:6791/0 2024-01-24T23:36:32.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:32.180+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f0810011610 addr v1:172.21.15.115:6791/0 2024-01-24T23:36:32.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:32.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081001ba20 con 0x7f0810011610 2024-01-24T23:36:32.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:32.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810011480 con 0x7f081000c4a0 2024-01-24T23:36:32.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:32.180+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000c080 2024-01-24T23:36:32.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:32.180+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:36:32.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:32.180+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000c4a0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:36:32.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:32.184+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000c080 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:32.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:32.384+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000c4a0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:36:32.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:32.388+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000c080 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:32.778 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:32.788+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000c4a0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:36:32.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:32.792+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000c080 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:33.582 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:33.592+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000c4a0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:36:33.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:33.596+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000c080 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:35.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:35.184+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:36:35.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:35.184+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:36:35.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:35.184+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:36:35.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:35.184+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:36:35.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:35.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000c080 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:35.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:35.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000c4a0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:35.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:35.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810011610 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:36:35.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:35.184+0000 7f0820ff9640 10 monclient: _add_conns ranks=[5,2,0,3,1,6,7,4,8] 2024-01-24T23:36:35.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:35.184+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:36:35.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:35.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f0810017a50 addr v1:172.21.15.132:6791/0 2024-01-24T23:36:35.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:35.184+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:36:35.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:35.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f0810009fd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:36:35.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:35.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f081000b970 addr v1:172.21.15.115:6789/0 2024-01-24T23:36:35.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:35.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f081000b970 2024-01-24T23:36:35.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:35.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810011480 con 0x7f0810009fd0 2024-01-24T23:36:35.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:35.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f0810017a50 2024-01-24T23:36:35.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:35.184+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:36:35.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:35.184+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810017a50 legacy=0x7f0810017e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:36:35.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:35.184+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810009fd0 legacy=0x7f081000a3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:35.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:35.388+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810017a50 legacy=0x7f0810017e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:36:35.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:35.388+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810009fd0 legacy=0x7f081000a3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:35.783 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:35.792+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810017a50 legacy=0x7f0810017e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:36:35.783 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:35.792+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810009fd0 legacy=0x7f081000a3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:36.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:36.372+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000b970 legacy=0x7f0810018540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:36:36.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:36.372+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000b970 legacy=0x7f0810018540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:36:36.363 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:36.372+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000b970 0x7f0810018540 :-1 s=OPENED pgs=209 cs=1 l=1).handle_message read tag failed 2024-01-24T23:36:36.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:36.372+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000b970 0x7f0810018540 :-1 s=OPENED pgs=209 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:36:36.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:36.372+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:36:36.373 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:36:36.372+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:36:36.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:36.596+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810017a50 legacy=0x7f0810017e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:36:36.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:36.596+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810009fd0 legacy=0x7f081000a3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:38.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:38.184+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:36:38.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:38.184+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:36:38.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:38.184+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:36:38.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:38.184+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:36:38.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:38.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810017a50 legacy=0x7f0810017e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:38.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:38.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810009fd0 legacy=0x7f081000a3b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:38.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:38.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000b970 legacy=0x7f0810018540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:36:38.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:38.184+0000 7f0820ff9640 10 monclient: _add_conns ranks=[1,4,8,7,3,6] 2024-01-24T23:36:38.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:38.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f0810011610 addr v1:172.21.15.132:6789/0 2024-01-24T23:36:38.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:38.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f0810018c50 addr v1:172.21.15.115:6791/0 2024-01-24T23:36:38.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:38.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f081000bd80 addr v1:172.21.15.115:6793/0 2024-01-24T23:36:38.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:38.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081001b570 con 0x7f0810011610 2024-01-24T23:36:38.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:38.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810011480 con 0x7f0810018c50 2024-01-24T23:36:38.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:38.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000bd80 2024-01-24T23:36:38.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:38.184+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:36:38.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:38.184+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000bd80 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:38.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:38.184+0000 7f08227fc640 0 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810018c50 legacy=0x7f081000f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2024-01-24T23:36:38.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:38.184+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810018c50 legacy=0x7f081000f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send send error: (32) Broken pipe 2024-01-24T23:36:38.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:38.388+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000bd80 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:38.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:38.792+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000bd80 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:39.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:39.596+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000bd80 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:41.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.184+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:36:41.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.184+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:36:41.175 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.184+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:36:41.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.184+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:36:41.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000bd80 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:41.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810018c50 legacy=0x7f081000f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:36:41.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810011610 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:36:41.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.184+0000 7f0820ff9640 10 monclient: _add_conns ranks=[7,6,3] 2024-01-24T23:36:41.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081001b570 addr v1:172.21.15.132:6792/0 2024-01-24T23:36:41.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.184+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:36:41.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f0810028070 addr v1:172.21.15.115:6792/0 2024-01-24T23:36:41.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f081000b970 addr v1:172.21.15.132:6790/0 2024-01-24T23:36:41.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f081000b970 2024-01-24T23:36:41.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f0810028070 2024-01-24T23:36:41.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810011630 con 0x7f081001b570 2024-01-24T23:36:41.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.184+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:36:41.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.184+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000b970 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:41.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.184+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b570 legacy=0x7f0810004b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:41.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.184+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810028070 legacy=0x7f081002a460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:41.373 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:36:41.376+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:36:41.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.388+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b570 legacy=0x7f0810004b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:41.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.388+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000b970 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:41.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.388+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810028070 legacy=0x7f081002a460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:41.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.792+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000b970 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:41.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.792+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b570 legacy=0x7f0810004b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:41.783 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:41.792+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810028070 legacy=0x7f081002a460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:42.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:42.596+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b570 legacy=0x7f0810004b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:42.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:42.596+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000b970 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:42.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:42.596+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810028070 legacy=0x7f081002a460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:44.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:44.184+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:36:44.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:44.184+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:36:44.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:44.184+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:36:44.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:44.184+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:36:44.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:44.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b570 legacy=0x7f0810004b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:44.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:44.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810028070 legacy=0x7f081002a460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:44.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:44.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000b970 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:44.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:44.184+0000 7f0820ff9640 10 monclient: _add_conns ranks=[6,2,7,4,5,8,0,3,1] 2024-01-24T23:36:44.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:44.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f081000bd80 addr v1:172.21.15.115:6792/0 2024-01-24T23:36:44.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:44.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f081000a6e0 addr v1:172.21.15.115:6790/0 2024-01-24T23:36:44.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:44.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081001b570 addr v1:172.21.15.132:6792/0 2024-01-24T23:36:44.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:44.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100117c0 con 0x7f081000a6e0 2024-01-24T23:36:44.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:44.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f081000bd80 2024-01-24T23:36:44.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:44.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081001b570 2024-01-24T23:36:44.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:44.184+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:36:44.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:44.184+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b570 legacy=0x7f0810004b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:44.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:44.184+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000a6e0 legacy=0x7f081000f5f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:44.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:44.184+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000bd80 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:44.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:44.388+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b570 legacy=0x7f0810004b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:44.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:44.388+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000a6e0 legacy=0x7f081000f5f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:44.379 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:44.388+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000bd80 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:44.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:44.792+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b570 legacy=0x7f0810004b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:44.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:44.792+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000a6e0 legacy=0x7f081000f5f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:44.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:44.792+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000bd80 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:45.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:45.596+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b570 legacy=0x7f0810004b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:45.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:45.596+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000a6e0 legacy=0x7f081000f5f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:45.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:45.596+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000bd80 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:46.373 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:36:46.376+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:36:47.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:47.184+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:36:47.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:47.184+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:36:47.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:47.184+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:36:47.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:47.184+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:36:47.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:47.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b570 legacy=0x7f0810004b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:47.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:47.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000bd80 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:47.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:47.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000a6e0 legacy=0x7f081000f5f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:47.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:47.184+0000 7f0820ff9640 10 monclient: _add_conns ranks=[0,1,8,4,5,3] 2024-01-24T23:36:47.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:47.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f0810014320 addr v1:172.21.15.115:6789/0 2024-01-24T23:36:47.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:47.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f0810017e30 addr v1:172.21.15.132:6789/0 2024-01-24T23:36:47.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:47.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f081001ea40 addr v1:172.21.15.115:6793/0 2024-01-24T23:36:47.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:47.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810011630 con 0x7f0810014320 2024-01-24T23:36:47.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:47.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100117c0 con 0x7f0810017e30 2024-01-24T23:36:47.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:47.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f081001ea40 2024-01-24T23:36:47.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:47.184+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:36:47.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:47.184+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081001ea40 legacy=0x7f0810020e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:47.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:47.388+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081001ea40 legacy=0x7f0810020e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:47.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:47.792+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081001ea40 legacy=0x7f0810020e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:48.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:48.596+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081001ea40 legacy=0x7f0810020e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:50.184+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:36:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:50.184+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:36:50.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:50.184+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:36:50.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:50.184+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:36:50.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:50.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081001ea40 legacy=0x7f0810020e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:50.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:50.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810017e30 legacy=0x7f08100182c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:36:50.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:50.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f0810014320 legacy=0x7f0810016710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:36:50.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:50.184+0000 7f0820ff9640 10 monclient: _add_conns ranks=[5,4,3] 2024-01-24T23:36:50.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:50.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f0810021570 addr v1:172.21.15.132:6791/0 2024-01-24T23:36:50.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:50.184+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:36:50.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:50.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f081002cb90 addr v1:172.21.15.115:6791/0 2024-01-24T23:36:50.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:50.184+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:36:50.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:50.184+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810021570 legacy=0x7f081002b470 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:36:50.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:50.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f081000b970 addr v1:172.21.15.132:6790/0 2024-01-24T23:36:50.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:50.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000b970 2024-01-24T23:36:50.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:50.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f081002cb90 2024-01-24T23:36:50.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:50.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810021950 con 0x7f0810021570 2024-01-24T23:36:50.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:50.184+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:36:50.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:50.184+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000b970 legacy=0x7f081000bd50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:50.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:50.388+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000b970 legacy=0x7f081000bd50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:50.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:50.388+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810021570 legacy=0x7f081002b470 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:36:50.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:50.792+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000b970 legacy=0x7f081000bd50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:50.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:50.792+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810021570 legacy=0x7f081002b470 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:36:51.373 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:36:51.376+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:36:51.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:51.384+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081002cb90 legacy=0x7f081002ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:36:51.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:51.384+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081002cb90 legacy=0x7f081002ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:36:51.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:51.384+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081002cb90 0x7f081002ef80 :-1 s=OPENED pgs=217 cs=1 l=1).handle_message read tag failed 2024-01-24T23:36:51.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:51.384+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081002cb90 0x7f081002ef80 :-1 s=OPENED pgs=217 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:36:51.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:51.384+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:36:51.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:51.596+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000b970 legacy=0x7f081000bd50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:51.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:51.596+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810021570 legacy=0x7f081002b470 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:36:53.176 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:53.184+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:36:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:53.184+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:36:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:53.184+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:36:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:53.184+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:36:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:53.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810021570 legacy=0x7f081002b470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:53.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081002cb90 legacy=0x7f081002ef80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:36:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:53.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000b970 legacy=0x7f081000bd50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:53.184+0000 7f0820ff9640 10 monclient: _add_conns ranks=[2,1,7,8,6,0,3,4,5] 2024-01-24T23:36:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:53.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f0810018240 addr v1:172.21.15.115:6790/0 2024-01-24T23:36:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:53.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f081000a6e0 addr v1:172.21.15.132:6789/0 2024-01-24T23:36:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:53.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081000f5f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:36:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:53.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810011630 con 0x7f081000a6e0 2024-01-24T23:36:53.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:53.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f0810018240 2024-01-24T23:36:53.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:53.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810021950 con 0x7f081000f5f0 2024-01-24T23:36:53.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:53.184+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:36:53.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:53.184+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000f5f0 legacy=0x7f081001ee50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:53.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:53.184+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810018240 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:53.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:53.388+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000f5f0 legacy=0x7f081001ee50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:53.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:53.388+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810018240 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:53.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:53.792+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000f5f0 legacy=0x7f081001ee50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:53.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:53.792+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810018240 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:54.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:54.596+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000f5f0 legacy=0x7f081001ee50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:36:54.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:54.596+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810018240 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:36:56.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.184+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:36:56.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.184+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:36:56.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.184+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:36:56.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.184+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:36:56.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000f5f0 legacy=0x7f081001ee50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:56.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810018240 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:56.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000a6e0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:36:56.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.184+0000 7f0820ff9640 10 monclient: _add_conns ranks=[3,8,6,5,0,4] 2024-01-24T23:36:56.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.184+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:36:56.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f081000b970 addr v1:172.21.15.132:6790/0 2024-01-24T23:36:56.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f081001b560 addr v1:172.21.15.115:6793/0 2024-01-24T23:36:56.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f0810020570 addr v1:172.21.15.115:6792/0 2024-01-24T23:36:56.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100117c0 con 0x7f081000b970 2024-01-24T23:36:56.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f0810020570 2024-01-24T23:36:56.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810021950 con 0x7f081001b560 2024-01-24T23:36:56.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.184+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:36:56.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.184+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000b970 legacy=0x7f081000bd50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:56.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.184+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081001b560 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:56.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.184+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810020570 legacy=0x7f08100209a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:56.374 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:36:56.376+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:36:56.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.388+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000b970 legacy=0x7f081000bd50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:56.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.388+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810020570 legacy=0x7f08100209a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:56.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.388+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081001b560 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:56.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.792+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000b970 legacy=0x7f081000bd50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:56.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.792+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810020570 legacy=0x7f08100209a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:56.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:56.792+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081001b560 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:57.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:57.596+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000b970 legacy=0x7f081000bd50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:36:57.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:57.596+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810020570 legacy=0x7f08100209a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:36:57.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:57.596+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081001b560 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:36:59.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:59.184+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:36:59.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:59.184+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:36:59.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:59.184+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:36:59.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:59.184+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:36:59.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:59.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081001b560 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:59.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:59.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810020570 legacy=0x7f08100209a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:59.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:59.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000b970 legacy=0x7f081000bd50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:36:59.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:59.184+0000 7f0820ff9640 10 monclient: _add_conns ranks=[0,4,5] 2024-01-24T23:36:59.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:59.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f081001ea40 addr v1:172.21.15.115:6789/0 2024-01-24T23:36:59.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:59.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f0810018210 addr v1:172.21.15.115:6791/0 2024-01-24T23:36:59.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:59.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f081000a6e0 addr v1:172.21.15.132:6791/0 2024-01-24T23:36:59.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:59.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f081001ea40 2024-01-24T23:36:59.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:59.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100117c0 con 0x7f0810018210 2024-01-24T23:36:59.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:59.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000a6e0 2024-01-24T23:36:59.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:59.184+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:36:59.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:59.184+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a6e0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:36:59.377 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:59.388+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a6e0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:36:59.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:36:59.792+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a6e0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:00.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:00.596+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a6e0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:01.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:01.372+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081001ea40 legacy=0x7f08100210b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:37:01.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:01.372+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081001ea40 legacy=0x7f08100210b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:37:01.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:01.372+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081001ea40 0x7f08100210b0 :-1 s=OPENED pgs=223 cs=1 l=1).handle_message read tag failed 2024-01-24T23:37:01.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:01.372+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081001ea40 0x7f08100210b0 :-1 s=OPENED pgs=223 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:37:01.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:01.372+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:37:01.373 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:37:01.372+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:37:01.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:01.384+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810018210 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:37:01.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:01.384+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810018210 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:37:01.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:01.384+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810018210 0x7f0810009fd0 :-1 s=OPENED pgs=224 cs=1 l=1).handle_message read tag failed 2024-01-24T23:37:01.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:01.384+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810018210 0x7f0810009fd0 :-1 s=OPENED pgs=224 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:37:01.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:01.384+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:37:01.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:01.384+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:37:02.177 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:02.184+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:37:02.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:02.184+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:37:02.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:02.184+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:37:02.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:02.184+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:37:02.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:02.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a6e0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:02.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:02.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810018210 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:37:02.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:02.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081001ea40 legacy=0x7f08100210b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:37:02.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:02.184+0000 7f0820ff9640 10 monclient: _add_conns ranks=[5,2,7,1,8,6,4,3,0] 2024-01-24T23:37:02.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:02.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f081001ea40 addr v1:172.21.15.132:6791/0 2024-01-24T23:37:02.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:02.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f0810018210 addr v1:172.21.15.115:6790/0 2024-01-24T23:37:02.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:02.184+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081000a6e0 addr v1:172.21.15.132:6792/0 2024-01-24T23:37:02.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:02.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810021950 con 0x7f0810018210 2024-01-24T23:37:02.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:02.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081001ea40 2024-01-24T23:37:02.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:02.184+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f081000a6e0 2024-01-24T23:37:02.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:02.184+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:37:02.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:02.184+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a6e0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:02.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:02.184+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081001ea40 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:02.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:02.184+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810018210 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:02.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:02.388+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081001ea40 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:02.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:02.388+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a6e0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:02.378 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:02.388+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810018210 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:02.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:02.792+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081001ea40 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:02.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:02.792+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a6e0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:02.782 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:02.792+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810018210 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:03.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:03.596+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081001ea40 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:03.586 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:03.596+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a6e0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:03.587 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:03.596+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810018210 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:05.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:05.188+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:37:05.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:05.188+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:37:05.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:05.188+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:37:05.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:05.188+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:37:05.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:05.188+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000a6e0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:05.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:05.188+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081001ea40 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:05.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:05.188+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810018210 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:05.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:05.188+0000 7f0820ff9640 10 monclient: _add_conns ranks=[1,4,0,3,6,8] 2024-01-24T23:37:05.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:05.188+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f081000c080 addr v1:172.21.15.132:6789/0 2024-01-24T23:37:05.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:05.188+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f081001b570 addr v1:172.21.15.115:6791/0 2024-01-24T23:37:05.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:05.188+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f081000f5f0 addr v1:172.21.15.115:6789/0 2024-01-24T23:37:05.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:05.188+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081000fa10 con 0x7f081000f5f0 2024-01-24T23:37:05.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:05.188+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810021950 con 0x7f081000c080 2024-01-24T23:37:05.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:05.188+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081001b570 2024-01-24T23:37:05.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:05.188+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:37:06.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:06.372+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000f5f0 legacy=0x7f0810014320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:37:06.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:06.372+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000f5f0 legacy=0x7f0810014320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:37:06.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:06.372+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000f5f0 0x7f0810014320 :-1 s=OPENED pgs=224 cs=1 l=1).handle_message read tag failed 2024-01-24T23:37:06.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:06.372+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000f5f0 0x7f0810014320 :-1 s=OPENED pgs=224 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:37:06.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:06.372+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:37:06.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:06.376+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000c080 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:37:06.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:06.376+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000c080 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:37:06.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:06.376+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000c080 0x7f0810005b80 :-1 s=OPENED pgs=221 cs=1 l=1).handle_message read tag failed 2024-01-24T23:37:06.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:06.376+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000c080 0x7f0810005b80 :-1 s=OPENED pgs=221 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:37:06.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:06.376+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:37:06.368 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:06.376+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:37:06.373 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:37:06.372+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:37:06.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:06.384+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001b570 legacy=0x7f0810020e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:37:06.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:06.384+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001b570 legacy=0x7f0810020e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:37:06.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:06.384+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001b570 0x7f0810020e30 :-1 s=OPENED pgs=225 cs=1 l=1).handle_message read tag failed 2024-01-24T23:37:06.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:06.384+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001b570 0x7f0810020e30 :-1 s=OPENED pgs=225 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:37:06.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:06.384+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:37:08.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:08.187+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:37:08.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:08.187+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:37:08.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:08.187+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:37:08.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:08.187+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:37:08.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:08.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001b570 legacy=0x7f0810020e30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:37:08.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:08.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000c080 legacy=0x7f0810005b80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:37:08.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:08.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000f5f0 legacy=0x7f0810014320 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:37:08.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:08.187+0000 7f0820ff9640 10 monclient: _add_conns ranks=[6,8,3] 2024-01-24T23:37:08.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:08.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f081000f5f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:37:08.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:08.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f081000c080 addr v1:172.21.15.115:6793/0 2024-01-24T23:37:08.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:08.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f0810021540 addr v1:172.21.15.132:6790/0 2024-01-24T23:37:08.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:08.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f0810021540 2024-01-24T23:37:08.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:08.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000f5f0 2024-01-24T23:37:08.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:08.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100219c0 con 0x7f081000c080 2024-01-24T23:37:08.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:08.187+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:37:08.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:08.187+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810021540 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:37:08.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:08.187+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000c080 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:08.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:08.187+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000f5f0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:37:08.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:08.391+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810021540 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:37:08.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:08.391+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000c080 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:08.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:08.391+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000f5f0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:37:08.786 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:08.795+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810021540 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:37:08.786 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:08.795+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000c080 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:08.786 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:08.795+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000f5f0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:37:09.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:09.599+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810021540 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:37:09.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:09.599+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000c080 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:09.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:09.599+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000f5f0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:37:11.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:11.187+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:37:11.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:11.187+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:37:11.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:11.187+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:37:11.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:11.187+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:37:11.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:11.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000c080 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:11.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:11.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000f5f0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:11.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:11.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810021540 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:11.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:11.187+0000 7f0820ff9640 10 monclient: _add_conns ranks=[5,6,2,7,0,4,1,3,8] 2024-01-24T23:37:11.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:11.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f0810009fd0 addr v1:172.21.15.132:6791/0 2024-01-24T23:37:11.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:11.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f081001ea40 addr v1:172.21.15.115:6792/0 2024-01-24T23:37:11.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:11.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f081000f5f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:37:11.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:11.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810018920 con 0x7f081000f5f0 2024-01-24T23:37:11.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:11.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f0810009fd0 2024-01-24T23:37:11.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:11.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081001ea40 2024-01-24T23:37:11.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:11.187+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:37:11.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:11.187+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810009fd0 legacy=0x7f081000a3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:11.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:11.187+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081001ea40 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:37:11.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:11.187+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000f5f0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:11.374 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:37:11.372+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:37:11.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:11.391+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810009fd0 legacy=0x7f081000a3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:11.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:11.391+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000f5f0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:11.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:11.391+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081001ea40 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:37:11.786 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:11.795+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810009fd0 legacy=0x7f081000a3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:11.786 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:11.795+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081001ea40 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:37:11.786 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:11.795+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000f5f0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:12.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:12.599+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810009fd0 legacy=0x7f081000a3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:12.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:12.599+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081001ea40 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:37:12.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:12.599+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000f5f0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:14.178 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:14.187+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:37:14.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:14.187+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:37:14.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:14.187+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:37:14.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:14.187+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:37:14.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:14.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081001ea40 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:14.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:14.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810009fd0 legacy=0x7f081000a3b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:14.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:14.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000f5f0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:14.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:14.187+0000 7f0820ff9640 10 monclient: _add_conns ranks=[4,0,3,7,1,8] 2024-01-24T23:37:14.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:14.187+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:37:14.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:14.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f081001b570 addr v1:172.21.15.115:6791/0 2024-01-24T23:37:14.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:14.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f0810015330 addr v1:172.21.15.115:6789/0 2024-01-24T23:37:14.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:14.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f0810021540 addr v1:172.21.15.132:6790/0 2024-01-24T23:37:14.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:14.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100219c0 con 0x7f0810015330 2024-01-24T23:37:14.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:14.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810018920 con 0x7f0810021540 2024-01-24T23:37:14.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:14.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f081001b570 2024-01-24T23:37:14.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:14.187+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:37:14.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:14.187+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:37:14.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:14.187+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810021540 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:37:14.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:14.391+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810021540 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:37:14.786 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:14.795+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810021540 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:37:15.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:15.599+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810021540 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:37:16.364 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:16.371+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f0810015330 legacy=0x7f0810028450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:37:16.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:16.371+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f0810015330 legacy=0x7f0810028450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:37:16.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:16.371+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f0810015330 0x7f0810028450 :-1 s=OPENED pgs=231 cs=1 l=1).handle_message read tag failed 2024-01-24T23:37:16.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:16.371+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f0810015330 0x7f0810028450 :-1 s=OPENED pgs=231 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:37:16.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:16.371+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:37:16.374 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:37:16.371+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:37:16.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:16.383+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001b570 legacy=0x7f0810020e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:37:16.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:16.383+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001b570 legacy=0x7f0810020e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:37:16.374 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:16.383+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001b570 0x7f0810020e30 :-1 s=OPENED pgs=232 cs=1 l=1).handle_message read tag failed 2024-01-24T23:37:16.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:16.383+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001b570 0x7f0810020e30 :-1 s=OPENED pgs=232 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:37:16.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:16.383+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:37:17.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:17.187+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:37:17.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:17.187+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:37:17.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:17.187+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:37:17.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:17.187+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:37:17.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:17.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001b570 legacy=0x7f0810020e30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:37:17.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:17.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810021540 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:17.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:17.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f0810015330 legacy=0x7f0810028450 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:37:17.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:17.187+0000 7f0820ff9640 10 monclient: _add_conns ranks=[8,1,7] 2024-01-24T23:37:17.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:17.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f0810011580 addr v1:172.21.15.115:6793/0 2024-01-24T23:37:17.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:17.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f081000fa10 addr v1:172.21.15.132:6789/0 2024-01-24T23:37:17.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:17.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081000c080 addr v1:172.21.15.132:6792/0 2024-01-24T23:37:17.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:17.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000fa10 2024-01-24T23:37:17.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:17.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810018920 con 0x7f081000c080 2024-01-24T23:37:17.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:17.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f0810011580 2024-01-24T23:37:17.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:17.187+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:37:17.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:17.187+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000c080 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:17.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:17.187+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810011580 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:17.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:17.391+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000c080 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:17.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:17.391+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810011580 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:17.786 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:17.795+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000c080 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:17.786 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:17.795+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810011580 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:18.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:18.599+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000c080 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:18.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:18.599+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810011580 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:20.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.187+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:37:20.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.187+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:37:20.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.187+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:37:20.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.187+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:37:20.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810011580 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:20.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000c080 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000fa10 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:37:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.187+0000 7f0820ff9640 10 monclient: _add_conns ranks=[5,2,7,8,6,3,0,1,4] 2024-01-24T23:37:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.187+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:37:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f081001ea40 addr v1:172.21.15.132:6791/0 2024-01-24T23:37:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f081000f5f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:37:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.187+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:37:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081001b570 addr v1:172.21.15.132:6792/0 2024-01-24T23:37:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100219a0 con 0x7f081000f5f0 2024-01-24T23:37:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810018920 con 0x7f081001ea40 2024-01-24T23:37:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f081001b570 2024-01-24T23:37:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.187+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:37:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.187+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081001ea40 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:20.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.187+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b570 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:20.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.187+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000f5f0 legacy=0x7f081002bb80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:20.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.391+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b570 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:20.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.391+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081001ea40 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:20.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.391+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000f5f0 legacy=0x7f081002bb80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:20.786 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.795+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b570 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:20.786 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.795+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081001ea40 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:20.786 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:20.795+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000f5f0 legacy=0x7f081002bb80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:21.374 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:37:21.375+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:37:21.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:21.599+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b570 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:21.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:21.599+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081001ea40 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:21.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:21.599+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000f5f0 legacy=0x7f081002bb80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:23.179 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:23.187+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:37:23.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:23.187+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:37:23.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:23.187+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:37:23.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:23.187+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:37:23.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:23.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b570 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:23.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:23.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081001ea40 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:23.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:23.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000f5f0 legacy=0x7f081002bb80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:23.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:23.187+0000 7f0820ff9640 10 monclient: _add_conns ranks=[0,8,3,1,6,4] 2024-01-24T23:37:23.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:23.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f081000fa10 addr v1:172.21.15.115:6789/0 2024-01-24T23:37:23.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:23.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f0810011580 addr v1:172.21.15.115:6793/0 2024-01-24T23:37:23.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:23.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f081000b970 addr v1:172.21.15.132:6790/0 2024-01-24T23:37:23.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:23.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000fa10 2024-01-24T23:37:23.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:23.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100219a0 con 0x7f081000b970 2024-01-24T23:37:23.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:23.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810018920 con 0x7f0810011580 2024-01-24T23:37:23.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:23.187+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:37:23.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:23.187+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000b970 legacy=0x7f0810020e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:37:23.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:23.187+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810011580 legacy=0x7f0810029460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:23.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:23.391+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000b970 legacy=0x7f0810020e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:37:23.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:23.391+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810011580 legacy=0x7f0810029460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:23.786 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:23.795+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000b970 legacy=0x7f0810020e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:37:23.786 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:23.795+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810011580 legacy=0x7f0810029460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:24.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:24.599+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000b970 legacy=0x7f0810020e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:37:24.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:24.599+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810011580 legacy=0x7f0810029460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:26.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.187+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:37:26.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.187+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:37:26.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.187+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:37:26.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.187+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:37:26.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810011580 legacy=0x7f0810029460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:26.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000b970 legacy=0x7f0810020e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:26.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000fa10 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:37:26.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.187+0000 7f0820ff9640 10 monclient: _add_conns ranks=[6,4,1] 2024-01-24T23:37:26.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.187+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:37:26.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f0810021540 addr v1:172.21.15.115:6792/0 2024-01-24T23:37:26.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.187+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:37:26.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f081000b970 addr v1:172.21.15.115:6791/0 2024-01-24T23:37:26.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f0810011580 addr v1:172.21.15.132:6789/0 2024-01-24T23:37:26.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f0810011580 2024-01-24T23:37:26.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100219a0 con 0x7f081000b970 2024-01-24T23:37:26.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810018920 con 0x7f0810021540 2024-01-24T23:37:26.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.187+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:37:26.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.187+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810021540 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:37:26.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.375+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810011580 legacy=0x7f0810022060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:37:26.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.375+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810011580 legacy=0x7f0810022060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:37:26.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.375+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810011580 0x7f0810022060 :-1 s=OPENED pgs=235 cs=1 l=1).handle_message read tag failed 2024-01-24T23:37:26.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.375+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810011580 0x7f0810022060 :-1 s=OPENED pgs=235 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:37:26.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.375+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:37:26.374 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:37:26.375+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:37:26.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.383+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000b970 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:37:26.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.383+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000b970 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:37:26.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.383+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000b970 0x7f081000eee0 :-1 s=OPENED pgs=239 cs=1 l=1).handle_message read tag failed 2024-01-24T23:37:26.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.383+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000b970 0x7f081000eee0 :-1 s=OPENED pgs=239 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:37:26.375 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.383+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:37:26.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.391+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810021540 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:37:26.786 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:26.795+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810021540 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:37:27.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:27.599+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810021540 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:37:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:29.187+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:37:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:29.187+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:37:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:29.187+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:37:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:29.187+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:37:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:29.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810021540 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:29.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000b970 legacy=0x7f081000eee0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:37:29.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:29.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810011580 legacy=0x7f0810022060 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:37:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:29.187+0000 7f0820ff9640 10 monclient: _add_conns ranks=[6,7,1,2,3,5,8,4,0] 2024-01-24T23:37:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:29.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f081000f5f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:37:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:29.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081000fa10 addr v1:172.21.15.132:6792/0 2024-01-24T23:37:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:29.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f0810011580 addr v1:172.21.15.132:6789/0 2024-01-24T23:37:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:29.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f0810011580 2024-01-24T23:37:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:29.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810018920 con 0x7f081000f5f0 2024-01-24T23:37:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:29.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f081000fa10 2024-01-24T23:37:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:29.187+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:37:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:29.187+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000fa10 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:29.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:29.187+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000f5f0 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:37:29.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:29.391+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000fa10 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:29.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:29.391+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000f5f0 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:37:29.786 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:29.795+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000fa10 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:29.786 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:29.795+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000f5f0 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:37:30.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:30.599+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000fa10 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:30.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:30.599+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000f5f0 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:37:31.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:31.375+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810011580 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:37:31.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:31.375+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810011580 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:37:31.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:31.375+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810011580 0x7f0810009fd0 :-1 s=OPENED pgs=236 cs=1 l=1).handle_message read tag failed 2024-01-24T23:37:31.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:31.375+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810011580 0x7f0810009fd0 :-1 s=OPENED pgs=236 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:37:31.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:31.375+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:37:31.374 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:37:31.375+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:37:32.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:32.187+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:37:32.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:32.187+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:37:32.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:32.187+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:37:32.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:32.187+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:37:32.180 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:32.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000fa10 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:32.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:32.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081000f5f0 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:32.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:32.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810011580 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:37:32.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:32.187+0000 7f0820ff9640 10 monclient: _add_conns ranks=[4,8,0,3,5,2] 2024-01-24T23:37:32.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:32.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f081000b970 addr v1:172.21.15.115:6791/0 2024-01-24T23:37:32.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:32.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f081001b570 addr v1:172.21.15.115:6793/0 2024-01-24T23:37:32.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:32.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f0810017a50 addr v1:172.21.15.115:6789/0 2024-01-24T23:37:32.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:32.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100219a0 con 0x7f0810017a50 2024-01-24T23:37:32.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:32.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810018920 con 0x7f081000b970 2024-01-24T23:37:32.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:32.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f081001b570 2024-01-24T23:37:32.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:32.187+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:37:32.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:32.187+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081001b570 legacy=0x7f0810017340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:32.382 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:32.391+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081001b570 legacy=0x7f0810017340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:32.786 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:32.795+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081001b570 legacy=0x7f0810017340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:33.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:33.599+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081001b570 legacy=0x7f0810017340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:35.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.187+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:37:35.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.187+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:37:35.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.187+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:37:35.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.187+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:37:35.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081001b570 legacy=0x7f0810017340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:35.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000b970 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:37:35.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f0810017a50 legacy=0x7f081001da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:37:35.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.187+0000 7f0820ff9640 10 monclient: _add_conns ranks=[5,3,2] 2024-01-24T23:37:35.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.187+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:37:35.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f081000a6e0 addr v1:172.21.15.132:6791/0 2024-01-24T23:37:35.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.187+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:37:35.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f081000f5f0 addr v1:172.21.15.132:6790/0 2024-01-24T23:37:35.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.187+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f081000fa10 addr v1:172.21.15.115:6790/0 2024-01-24T23:37:35.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000fa10 2024-01-24T23:37:35.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f081000f5f0 2024-01-24T23:37:35.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.187+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100219a0 con 0x7f081000a6e0 2024-01-24T23:37:35.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.187+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:37:35.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.187+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000f5f0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:37:35.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.187+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a6e0 legacy=0x7f0810021160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:35.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.187+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000fa10 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:35.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.391+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000f5f0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:37:35.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.391+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a6e0 legacy=0x7f0810021160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:35.381 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.391+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000fa10 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:35.785 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.795+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000f5f0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:37:35.786 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.795+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a6e0 legacy=0x7f0810021160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:35.786 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:35.795+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000fa10 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:36.375 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:37:36.375+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:37:36.589 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:36.599+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a6e0 legacy=0x7f0810021160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:36.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:36.599+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000f5f0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:37:36.590 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:36.599+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000fa10 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:38.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:38.191+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:37:38.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:38.191+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:37:38.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:38.191+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:37:38.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:38.191+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:37:38.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:38.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a6e0 legacy=0x7f0810021160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:38.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:38.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000f5f0 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:38.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:38.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000fa10 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:38.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:38.191+0000 7f0820ff9640 10 monclient: _add_conns ranks=[2,4,0,8,5,3,1,7,6] 2024-01-24T23:37:38.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:38.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f081000fa10 addr v1:172.21.15.115:6790/0 2024-01-24T23:37:38.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:38.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f0810011520 addr v1:172.21.15.115:6791/0 2024-01-24T23:37:38.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:38.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f081000b970 addr v1:172.21.15.115:6789/0 2024-01-24T23:37:38.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:38.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810018920 con 0x7f081000b970 2024-01-24T23:37:38.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:38.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000fa10 2024-01-24T23:37:38.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:38.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f0810011520 2024-01-24T23:37:38.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:38.191+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:37:38.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:38.191+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000fa10 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:38.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:38.395+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000fa10 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:38.789 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:38.799+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000fa10 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:39.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:39.603+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000fa10 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:41.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.191+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:37:41.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.191+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:37:41.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.191+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:37:41.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.191+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:37:41.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810011520 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:37:41.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000fa10 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:41.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000b970 legacy=0x7f081000bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:37:41.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.191+0000 7f0820ff9640 10 monclient: _add_conns ranks=[7,6,5,8,3,1] 2024-01-24T23:37:41.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081001b570 addr v1:172.21.15.132:6792/0 2024-01-24T23:37:41.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.191+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:37:41.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f081001ea40 addr v1:172.21.15.115:6792/0 2024-01-24T23:37:41.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f081000a6e0 addr v1:172.21.15.132:6791/0 2024-01-24T23:37:41.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100219a0 con 0x7f081000a6e0 2024-01-24T23:37:41.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081001ea40 2024-01-24T23:37:41.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f081001b570 2024-01-24T23:37:41.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.191+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:37:41.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.191+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b570 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:41.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.191+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081001ea40 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:37:41.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.191+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a6e0 legacy=0x7f0810020ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:41.375 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:37:41.375+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:37:41.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.395+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a6e0 legacy=0x7f0810020ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:41.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.395+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b570 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:41.386 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.395+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081001ea40 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:37:41.789 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.799+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a6e0 legacy=0x7f0810020ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:41.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.799+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b570 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:41.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:41.799+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081001ea40 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:37:42.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:42.603+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a6e0 legacy=0x7f0810020ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:42.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:42.603+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b570 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:42.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:42.603+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081001ea40 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:37:44.181 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:44.191+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:37:44.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:44.191+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:37:44.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:44.191+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:37:44.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:44.191+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:37:44.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:44.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001b570 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:44.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:44.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081001ea40 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:44.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:44.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000a6e0 legacy=0x7f0810020ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:44.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:44.191+0000 7f0820ff9640 10 monclient: _add_conns ranks=[8,1,3] 2024-01-24T23:37:44.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:44.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f0810017a50 addr v1:172.21.15.115:6793/0 2024-01-24T23:37:44.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:44.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f081000fa10 addr v1:172.21.15.132:6789/0 2024-01-24T23:37:44.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:44.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f0810011520 addr v1:172.21.15.132:6790/0 2024-01-24T23:37:44.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:44.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810009be0 con 0x7f081000fa10 2024-01-24T23:37:44.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:44.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100219a0 con 0x7f0810011520 2024-01-24T23:37:44.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:44.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f0810017a50 2024-01-24T23:37:44.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:44.191+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:37:44.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:44.191+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810011520 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:37:44.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:44.191+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810017a50 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:44.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:44.395+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810011520 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:37:44.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:44.395+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810017a50 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:44.789 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:44.799+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810011520 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:37:44.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:44.799+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810017a50 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:45.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:45.603+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810011520 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:37:45.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:45.603+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810017a50 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:46.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:46.379+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000fa10 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:37:46.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:46.379+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000fa10 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:37:46.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:46.379+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000fa10 0x7f0810018210 :-1 s=OPENED pgs=243 cs=1 l=1).handle_message read tag failed 2024-01-24T23:37:46.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:46.379+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000fa10 0x7f0810018210 :-1 s=OPENED pgs=243 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:37:46.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:46.379+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:37:46.375 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:37:46.379+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:37:47.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:47.191+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:37:47.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:47.191+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:37:47.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:47.191+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:37:47.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:47.191+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:37:47.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:47.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810017a50 legacy=0x7f0810009fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:47.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:47.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810011520 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:47.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:47.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081000fa10 legacy=0x7f0810018210 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:37:47.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:47.191+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:37:47.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:47.191+0000 7f0820ff9640 10 monclient: _add_conns ranks=[5,2,4,6,7,3,8,1,0] 2024-01-24T23:37:47.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:47.191+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:37:47.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:47.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f081000c080 addr v1:172.21.15.132:6791/0 2024-01-24T23:37:47.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:47.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f081000fa10 addr v1:172.21.15.115:6790/0 2024-01-24T23:37:47.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:47.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f081000f5f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:37:47.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:47.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f081000fa10 2024-01-24T23:37:47.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:47.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100219a0 con 0x7f081000f5f0 2024-01-24T23:37:47.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:47.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000c080 2024-01-24T23:37:47.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:47.191+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:37:47.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:47.191+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000c080 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:47.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:47.191+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000fa10 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:47.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:47.395+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000c080 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:47.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:47.395+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000fa10 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:47.789 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:47.799+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000c080 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:47.789 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:47.799+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000fa10 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:48.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:48.603+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000c080 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:48.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:48.603+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000fa10 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:50.191+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:37:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:50.191+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:37:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:50.191+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:37:50.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:50.191+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:37:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:50.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000c080 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:50.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000f5f0 legacy=0x7f0810005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:37:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:50.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000fa10 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:50.191+0000 7f0820ff9640 10 monclient: _add_conns ranks=[0,7,3,8,1,6] 2024-01-24T23:37:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:50.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f081001b560 addr v1:172.21.15.115:6789/0 2024-01-24T23:37:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:50.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f0810011520 addr v1:172.21.15.132:6792/0 2024-01-24T23:37:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:50.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f0810009fd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:37:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:50.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810009be0 con 0x7f081001b560 2024-01-24T23:37:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:50.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f0810009fd0 2024-01-24T23:37:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:50.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f0810011520 2024-01-24T23:37:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:50.191+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:37:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:50.191+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f0810011520 legacy=0x7f0810020ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:50.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:50.191+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810009fd0 legacy=0x7f081000a3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:37:50.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:50.395+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810009fd0 legacy=0x7f081000a3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:37:50.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:50.395+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f0810011520 legacy=0x7f0810020ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:50.789 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:50.799+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f0810011520 legacy=0x7f0810020ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:50.790 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:50.799+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810009fd0 legacy=0x7f081000a3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:37:51.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:51.375+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081001b560 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:37:51.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:51.375+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081001b560 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:37:51.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:51.375+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081001b560 0x7f081000b970 :-1 s=OPENED pgs=247 cs=1 l=1).handle_message read tag failed 2024-01-24T23:37:51.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:51.375+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081001b560 0x7f081000b970 :-1 s=OPENED pgs=247 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:37:51.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:51.375+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:37:51.375 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:37:51.375+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:37:51.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:51.603+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810009fd0 legacy=0x7f081000a3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:37:51.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:51.603+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f0810011520 legacy=0x7f0810020ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:53.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:53.191+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:37:53.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:53.191+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:37:53.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:53.191+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:37:53.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:53.191+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:37:53.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:53.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f0810011520 legacy=0x7f0810020ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:53.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:53.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810009fd0 legacy=0x7f081000a3e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:53.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:53.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081001b560 legacy=0x7f081000b970 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:37:53.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:53.191+0000 7f0820ff9640 10 monclient: _add_conns ranks=[1,6,8] 2024-01-24T23:37:53.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:53.191+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:37:53.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:53.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f081001ea40 addr v1:172.21.15.132:6789/0 2024-01-24T23:37:53.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:53.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f0810017a50 addr v1:172.21.15.115:6792/0 2024-01-24T23:37:53.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:53.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f081002cb90 addr v1:172.21.15.115:6793/0 2024-01-24T23:37:53.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:53.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100219a0 con 0x7f081001ea40 2024-01-24T23:37:53.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:53.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f0810017a50 2024-01-24T23:37:53.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:53.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081002cb90 2024-01-24T23:37:53.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:53.191+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:37:53.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:53.191+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:37:53.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:53.191+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810017a50 legacy=0x7f081002b470 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:37:53.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:53.191+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081002cb90 legacy=0x7f081002ef80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:53.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:53.395+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081002cb90 legacy=0x7f081002ef80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:53.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:53.395+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810017a50 legacy=0x7f081002b470 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:37:53.789 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:53.799+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081002cb90 legacy=0x7f081002ef80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:53.789 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:53.799+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810017a50 legacy=0x7f081002b470 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:37:54.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:54.603+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081002cb90 legacy=0x7f081002ef80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:54.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:54.603+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810017a50 legacy=0x7f081002b470 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:37:56.182 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.191+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:37:56.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.191+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:37:56.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.191+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:37:56.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.191+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:37:56.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081002cb90 legacy=0x7f081002ef80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:56.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810017a50 legacy=0x7f081002b470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:56.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081001ea40 legacy=0x7f0810017340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:37:56.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.191+0000 7f0820ff9640 10 monclient: _add_conns ranks=[8,0,5,6,2,1,3,4,7] 2024-01-24T23:37:56.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f081002cb90 addr v1:172.21.15.115:6793/0 2024-01-24T23:37:56.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f081000fa10 addr v1:172.21.15.115:6789/0 2024-01-24T23:37:56.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f0810009fd0 addr v1:172.21.15.132:6791/0 2024-01-24T23:37:56.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810011930 con 0x7f081000fa10 2024-01-24T23:37:56.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f0810009fd0 2024-01-24T23:37:56.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081002cb90 2024-01-24T23:37:56.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.191+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:37:56.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.191+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810009fd0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:56.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.191+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081002cb90 legacy=0x7f081000a3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:56.365 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.375+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000fa10 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:37:56.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.375+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000fa10 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:37:56.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.375+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000fa10 0x7f0810018210 :-1 s=OPENED pgs=254 cs=1 l=1).handle_message read tag failed 2024-01-24T23:37:56.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.375+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000fa10 0x7f0810018210 :-1 s=OPENED pgs=254 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:37:56.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.375+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:37:56.375 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:37:56.375+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:37:56.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.395+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810009fd0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:56.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.395+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081002cb90 legacy=0x7f081000a3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:56.789 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.799+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810009fd0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:56.789 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:56.799+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081002cb90 legacy=0x7f081000a3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:57.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:57.603+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810009fd0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:37:57.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:57.603+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081002cb90 legacy=0x7f081000a3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:37:59.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:59.191+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:37:59.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:59.191+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:37:59.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:59.191+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:37:59.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:59.191+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:37:59.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:59.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081002cb90 legacy=0x7f081000a3e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:59.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:59.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810009fd0 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:37:59.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:59.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000fa10 legacy=0x7f0810018210 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:37:59.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:59.191+0000 7f0820ff9640 10 monclient: _add_conns ranks=[7,2,1,6,3,4] 2024-01-24T23:37:59.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:59.191+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:37:59.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:59.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081000f5f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:37:59.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:59.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f081000c080 addr v1:172.21.15.115:6790/0 2024-01-24T23:37:59.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:59.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f081001ea40 addr v1:172.21.15.132:6789/0 2024-01-24T23:37:59.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:59.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100219a0 con 0x7f081001ea40 2024-01-24T23:37:59.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:59.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f081000c080 2024-01-24T23:37:59.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:59.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000f5f0 2024-01-24T23:37:59.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:59.191+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:37:59.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:59.191+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000f5f0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:59.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:59.191+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000c080 legacy=0x7f0810020ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:59.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:59.395+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000f5f0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:59.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:59.395+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000c080 legacy=0x7f0810020ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:37:59.789 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:59.799+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000f5f0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:37:59.789 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:37:59.799+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000c080 legacy=0x7f0810020ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:38:00.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:00.603+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000f5f0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:38:00.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:00.603+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000c080 legacy=0x7f0810020ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:38:01.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:01.379+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081001ea40 legacy=0x7f0810017340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:38:01.369 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:01.379+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081001ea40 legacy=0x7f0810017340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:38:01.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:01.379+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081001ea40 0x7f0810017340 :-1 s=OPENED pgs=251 cs=1 l=1).handle_message read tag failed 2024-01-24T23:38:01.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:01.379+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081001ea40 0x7f0810017340 :-1 s=OPENED pgs=251 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:38:01.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:01.379+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:38:01.375 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:38:01.379+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:38:02.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:02.191+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:38:02.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:02.191+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:38:02.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:02.191+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:38:02.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:02.191+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:38:02.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:02.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000f5f0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:38:02.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:02.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000c080 legacy=0x7f0810020ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:38:02.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:02.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f081001ea40 legacy=0x7f0810017340 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:38:02.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:02.191+0000 7f0820ff9640 10 monclient: _add_conns ranks=[6,4,3] 2024-01-24T23:38:02.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:02.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f0810017a50 addr v1:172.21.15.115:6792/0 2024-01-24T23:38:02.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:02.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f081000fa10 addr v1:172.21.15.115:6791/0 2024-01-24T23:38:02.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:02.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f0810009fd0 addr v1:172.21.15.132:6790/0 2024-01-24T23:38:02.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:02.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100215f0 con 0x7f0810009fd0 2024-01-24T23:38:02.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:02.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f081000fa10 2024-01-24T23:38:02.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:02.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f0810017a50 2024-01-24T23:38:02.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:02.191+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:38:02.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:02.191+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810009fd0 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:38:02.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:02.191+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810017a50 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:38:02.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:02.395+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810009fd0 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:38:02.385 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:02.395+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810017a50 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:38:02.789 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:02.799+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810009fd0 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:38:02.789 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:02.799+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810017a50 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:38:03.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:03.603+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810009fd0 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:38:03.593 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:03.603+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810017a50 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:38:05.183 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:05.191+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:38:05.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:05.191+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:38:05.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:05.191+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:38:05.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:05.191+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:38:05.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:05.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810017a50 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:38:05.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:05.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000fa10 legacy=0x7f081000a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:38:05.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:05.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810009fd0 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:38:05.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:05.191+0000 7f0820ff9640 10 monclient: _add_conns ranks=[2,4,3,1,0,5,6,7,8] 2024-01-24T23:38:05.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:05.191+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:38:05.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:05.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f081000c080 addr v1:172.21.15.115:6790/0 2024-01-24T23:38:05.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:05.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f081001b560 addr v1:172.21.15.115:6791/0 2024-01-24T23:38:05.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:05.191+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f0810011520 addr v1:172.21.15.132:6790/0 2024-01-24T23:38:05.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:05.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100219a0 con 0x7f081000c080 2024-01-24T23:38:05.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:05.191+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:38:05.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:05.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100215f0 con 0x7f0810011520 2024-01-24T23:38:05.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:05.191+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081001b560 2024-01-24T23:38:05.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:05.191+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:38:05.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:05.191+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810011520 legacy=0x7f081001e140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:38:05.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:05.191+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000c080 legacy=0x7f0810020ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:38:05.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:05.395+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810011520 legacy=0x7f081001e140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:38:05.384 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:05.395+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000c080 legacy=0x7f0810020ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:38:05.789 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:05.799+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810011520 legacy=0x7f081001e140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:38:05.789 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:05.799+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000c080 legacy=0x7f0810020ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:38:06.375 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:38:06.379+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:38:06.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:06.387+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001b560 legacy=0x7f081001da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:38:06.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:06.387+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001b560 legacy=0x7f081001da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:38:06.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:06.387+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001b560 0x7f081001da30 :-1 s=OPENED pgs=256 cs=1 l=1).handle_message read tag failed 2024-01-24T23:38:06.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:06.387+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001b560 0x7f081001da30 :-1 s=OPENED pgs=256 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:38:06.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:06.387+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:38:06.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:06.603+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810011520 legacy=0x7f081001e140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:38:06.594 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:06.603+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000c080 legacy=0x7f0810020ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:38:08.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:08.194+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:38:08.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:08.194+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:38:08.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:08.194+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:38:08.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:08.194+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:38:08.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:08.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081001b560 legacy=0x7f081001da30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:38:08.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:08.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f0810011520 legacy=0x7f081001e140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:38:08.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:08.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000c080 legacy=0x7f0810020ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:38:08.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:08.194+0000 7f0820ff9640 10 monclient: _add_conns ranks=[8,0,7,5,6,1] 2024-01-24T23:38:08.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:08.194+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f081000fa10 addr v1:172.21.15.115:6793/0 2024-01-24T23:38:08.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:08.194+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f0810009fd0 addr v1:172.21.15.115:6789/0 2024-01-24T23:38:08.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:08.194+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081001e850 addr v1:172.21.15.132:6792/0 2024-01-24T23:38:08.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:08.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f0810009fd0 2024-01-24T23:38:08.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:08.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100219a0 con 0x7f081001e850 2024-01-24T23:38:08.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:08.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100215f0 con 0x7f081000fa10 2024-01-24T23:38:08.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:08.194+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:38:08.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:08.194+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001e850 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:38:08.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:08.194+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000fa10 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:38:08.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:08.398+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001e850 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:38:08.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:08.398+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000fa10 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:38:08.793 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:08.802+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001e850 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:38:08.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:08.802+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000fa10 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:38:09.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:09.606+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001e850 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:38:09.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:09.606+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000fa10 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:38:11.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.194+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:38:11.184 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.194+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:38:11.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.194+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:38:11.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.194+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:38:11.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000fa10 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:38:11.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081001e850 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:38:11.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f0810009fd0 legacy=0x7f081000a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:38:11.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.194+0000 7f0820ff9640 10 monclient: _add_conns ranks=[6,5,1] 2024-01-24T23:38:11.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.194+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:38:11.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.194+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f0810005b90 addr v1:172.21.15.115:6792/0 2024-01-24T23:38:11.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.194+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f081000f5f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:38:11.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.194+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f0810017a50 addr v1:172.21.15.132:6789/0 2024-01-24T23:38:11.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f0810017a50 2024-01-24T23:38:11.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100219a0 con 0x7f081000f5f0 2024-01-24T23:38:11.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100215f0 con 0x7f0810005b90 2024-01-24T23:38:11.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.194+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:38:11.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.194+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:38:11.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.194+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000f5f0 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:38:11.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.194+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810005b90 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:38:11.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.378+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810017a50 legacy=0x7f081002f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:38:11.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.378+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810017a50 legacy=0x7f081002f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:38:11.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.378+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810017a50 0x7f081002f4a0 :-1 s=OPENED pgs=258 cs=1 l=1).handle_message read tag failed 2024-01-24T23:38:11.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.378+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810017a50 0x7f081002f4a0 :-1 s=OPENED pgs=258 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:38:11.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.378+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:38:11.375 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:38:11.375+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:38:11.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.398+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000f5f0 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:38:11.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.398+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810005b90 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:38:11.793 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.802+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000f5f0 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:38:11.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:11.802+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810005b90 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:38:12.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:12.606+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000f5f0 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:38:12.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:12.606+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810005b90 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:38:14.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:14.194+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:38:14.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:14.194+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:38:14.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:14.194+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:38:14.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:14.194+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:38:14.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:14.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810005b90 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:38:14.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:14.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000f5f0 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:38:14.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:14.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810017a50 legacy=0x7f081002f4a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:38:14.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:14.194+0000 7f0820ff9640 10 monclient: _add_conns ranks=[1,8,6,3,5,0,2,7,4] 2024-01-24T23:38:14.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:14.194+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f0810017a50 addr v1:172.21.15.132:6789/0 2024-01-24T23:38:14.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:14.194+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f0810011520 addr v1:172.21.15.115:6793/0 2024-01-24T23:38:14.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:14.194+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f0810005b90 addr v1:172.21.15.115:6792/0 2024-01-24T23:38:14.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:14.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081001ec80 con 0x7f0810017a50 2024-01-24T23:38:14.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:14.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100219a0 con 0x7f0810005b90 2024-01-24T23:38:14.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:14.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100215f0 con 0x7f0810011520 2024-01-24T23:38:14.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:14.194+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:38:14.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:14.194+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810011520 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:38:14.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:14.194+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810005b90 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:38:14.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:14.398+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810011520 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:38:14.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:14.398+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810005b90 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:38:14.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:14.802+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810005b90 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:38:14.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:14.802+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810011520 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:38:15.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:15.606+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810005b90 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:38:15.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:15.606+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810011520 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:38:16.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:16.378+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810017a50 legacy=0x7f081000a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:38:16.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:16.378+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810017a50 legacy=0x7f081000a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:38:16.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:16.378+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810017a50 0x7f081000a3e0 :-1 s=OPENED pgs=259 cs=1 l=1).handle_message read tag failed 2024-01-24T23:38:16.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:16.378+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810017a50 0x7f081000a3e0 :-1 s=OPENED pgs=259 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:38:16.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:16.378+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:38:16.375 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:38:16.378+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:38:17.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:17.194+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:38:17.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:17.194+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:38:17.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:17.194+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:38:17.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:17.194+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:38:17.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:17.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f0810011520 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:38:17.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:17.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810005b90 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:38:17.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:17.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810017a50 legacy=0x7f081000a3e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:38:17.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:17.194+0000 7f0820ff9640 10 monclient: _add_conns ranks=[2,7,3,0,5,4] 2024-01-24T23:38:17.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:17.194+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f0810009fd0 addr v1:172.21.15.115:6790/0 2024-01-24T23:38:17.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:17.194+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f081000fa10 addr v1:172.21.15.132:6792/0 2024-01-24T23:38:17.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:17.194+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f081001b560 addr v1:172.21.15.132:6790/0 2024-01-24T23:38:17.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:17.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810018920 con 0x7f0810009fd0 2024-01-24T23:38:17.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:17.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100219a0 con 0x7f081001b560 2024-01-24T23:38:17.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:17.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100215f0 con 0x7f081000fa10 2024-01-24T23:38:17.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:17.194+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:38:17.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:17.194+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000fa10 legacy=0x7f0810020ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:38:17.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:17.194+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081001b560 legacy=0x7f0810028450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:38:17.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:17.194+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810009fd0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:38:17.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:17.398+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000fa10 legacy=0x7f0810020ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:38:17.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:17.398+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081001b560 legacy=0x7f0810028450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:38:17.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:17.398+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810009fd0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:38:17.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:17.802+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081001b560 legacy=0x7f0810028450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:38:17.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:17.802+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000fa10 legacy=0x7f0810020ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:38:17.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:17.802+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810009fd0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:38:18.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:18.606+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000fa10 legacy=0x7f0810020ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:38:18.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:18.606+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081001b560 legacy=0x7f0810028450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:38:18.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:18.606+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810009fd0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:38:20.185 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:20.194+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:38:20.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:20.194+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:38:20.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:20.194+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:38:20.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:20.194+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:38:20.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:20.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f081000fa10 legacy=0x7f0810020ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:38:20.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:20.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081001b560 legacy=0x7f0810028450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:38:20.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:20.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810009fd0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:38:20.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:20.194+0000 7f0820ff9640 10 monclient: _add_conns ranks=[4,0,5] 2024-01-24T23:38:20.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:20.194+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:38:20.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:20.194+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f081000c080 addr v1:172.21.15.115:6791/0 2024-01-24T23:38:20.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:20.194+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f081002d2a0 addr v1:172.21.15.115:6789/0 2024-01-24T23:38:20.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:20.194+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f081000f5f0 addr v1:172.21.15.132:6791/0 2024-01-24T23:38:20.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:20.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810005fa0 con 0x7f081002d2a0 2024-01-24T23:38:20.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:20.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810018920 con 0x7f081000c080 2024-01-24T23:38:20.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:20.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100219a0 con 0x7f081000f5f0 2024-01-24T23:38:20.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:20.194+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:38:20.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:20.194+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:38:20.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:20.194+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:38:20.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:20.194+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000f5f0 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:38:20.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:20.398+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000f5f0 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:38:20.793 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:20.802+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000f5f0 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:38:21.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:21.374+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081002d2a0 legacy=0x7f081002f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:38:21.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:21.374+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081002d2a0 legacy=0x7f081002f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:38:21.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:21.374+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081002d2a0 0x7f081002f690 :-1 s=OPENED pgs=262 cs=1 l=1).handle_message read tag failed 2024-01-24T23:38:21.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:21.374+0000 7f08227fc640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081002d2a0 0x7f081002f690 :-1 s=OPENED pgs=262 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:38:21.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:21.374+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:38:21.376 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:38:21.378+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:38:21.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:21.382+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000c080 legacy=0x7f081002bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:38:21.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:21.382+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000c080 legacy=0x7f081002bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:38:21.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:21.382+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000c080 0x7f081002bb80 :-1 s=OPENED pgs=263 cs=1 l=1).handle_message read tag failed 2024-01-24T23:38:21.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:21.382+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000c080 0x7f081002bb80 :-1 s=OPENED pgs=263 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:38:21.376 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:21.382+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:38:21.600 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:21.606+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000f5f0 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:38:23.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:23.194+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:38:23.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:23.194+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:38:23.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:23.194+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:38:23.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:23.194+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:38:23.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:23.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f081000f5f0 legacy=0x7f0810016330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:38:23.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:23.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000c080 legacy=0x7f081002bb80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:38:23.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:23.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081002d2a0 legacy=0x7f081002f690 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:38:23.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:23.194+0000 7f0820ff9640 10 monclient: _add_conns ranks=[7,3,4,0,6,2,1,5,8] 2024-01-24T23:38:23.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:23.194+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f0810009fd0 addr v1:172.21.15.132:6792/0 2024-01-24T23:38:23.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:23.194+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f081000fa10 addr v1:172.21.15.132:6790/0 2024-01-24T23:38:23.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:23.194+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f081000c080 addr v1:172.21.15.115:6791/0 2024-01-24T23:38:23.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:23.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100215f0 con 0x7f081000fa10 2024-01-24T23:38:23.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:23.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100219a0 con 0x7f081000c080 2024-01-24T23:38:23.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:23.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f0810009fd0 2024-01-24T23:38:23.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:23.194+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:38:23.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:23.194+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000fa10 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:38:23.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:23.194+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f0810009fd0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:38:23.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:23.398+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000fa10 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:38:23.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:23.398+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f0810009fd0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:38:23.793 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:23.802+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000fa10 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:38:23.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:23.802+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f0810009fd0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:38:24.597 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:24.606+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000fa10 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:38:24.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:24.606+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f0810009fd0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:38:26.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.194+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:38:26.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.194+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:38:26.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.194+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:38:26.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.194+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:38:26.186 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6792/0 conn(0x7f0810009fd0 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:38:26.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f081000c080 legacy=0x7f081000a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:38:26.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6790/0 conn(0x7f081000fa10 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:38:26.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.194+0000 7f0820ff9640 10 monclient: _add_conns ranks=[1,2,6,8,0,5] 2024-01-24T23:38:26.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.194+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f0810011520 addr v1:172.21.15.132:6789/0 2024-01-24T23:38:26.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.194+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f0810017a50 addr v1:172.21.15.115:6790/0 2024-01-24T23:38:26.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.194+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f081001b570 addr v1:172.21.15.115:6792/0 2024-01-24T23:38:26.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f0810011520 2024-01-24T23:38:26.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100215f0 con 0x7f0810017a50 2024-01-24T23:38:26.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081001b570 2024-01-24T23:38:26.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.194+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:38:26.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.194+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:38:26.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.194+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810017a50 legacy=0x7f0810020ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:38:26.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.194+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081001b570 legacy=0x7f0810028450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:38:26.370 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.378+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810011520 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:38:26.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.378+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810011520 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:38:26.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.378+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810011520 0x7f0810018210 :-1 s=OPENED pgs=266 cs=1 l=1).handle_message read tag failed 2024-01-24T23:38:26.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.378+0000 7f0822ffd640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810011520 0x7f0810018210 :-1 s=OPENED pgs=266 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:38:26.371 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.378+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:38:26.376 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:38:26.378+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:38:26.389 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.398+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081001b570 legacy=0x7f0810028450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:38:26.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.398+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810017a50 legacy=0x7f0810020ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:38:26.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.802+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081001b570 legacy=0x7f0810028450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:38:26.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:26.802+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810017a50 legacy=0x7f0810020ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:38:27.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:27.606+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081001b570 legacy=0x7f0810028450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:38:27.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:27.606+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810017a50 legacy=0x7f0810020ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:38:29.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:29.194+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:38:29.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:29.194+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:38:29.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:29.194+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:38:29.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:29.194+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:38:29.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:29.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f081001b570 legacy=0x7f0810028450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:38:29.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:29.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f0810017a50 legacy=0x7f0810020ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:38:29.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:29.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6789/0 conn(0x7f0810011520 legacy=0x7f0810018210 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:38:29.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:29.194+0000 7f0820ff9640 10 monclient: _add_conns ranks=[5,8,0] 2024-01-24T23:38:29.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:29.194+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f0810009fd0 addr v1:172.21.15.132:6791/0 2024-01-24T23:38:29.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:29.194+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f081000fa10 addr v1:172.21.15.115:6793/0 2024-01-24T23:38:29.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:29.194+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f081000c080 addr v1:172.21.15.115:6789/0 2024-01-24T23:38:29.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:29.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810018920 con 0x7f081000c080 2024-01-24T23:38:29.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:29.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100215f0 con 0x7f0810009fd0 2024-01-24T23:38:29.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:29.194+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f081000fa10 2024-01-24T23:38:29.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:29.194+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:38:29.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:29.194+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810009fd0 legacy=0x7f081000a3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:38:29.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:29.194+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000fa10 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:38:29.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:29.398+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810009fd0 legacy=0x7f081000a3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:38:29.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:29.398+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000fa10 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:38:29.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:29.802+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810009fd0 legacy=0x7f081000a3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:38:29.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:29.802+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000fa10 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:38:30.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:30.606+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810009fd0 legacy=0x7f081000a3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:38:30.598 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:30.606+0000 7f08227fc640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000fa10 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:38:31.366 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:31.374+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000c080 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:38:31.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:31.374+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000c080 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:38:31.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:31.374+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000c080 0x7f081000eee0 :-1 s=OPENED pgs=269 cs=1 l=1).handle_message read tag failed 2024-01-24T23:38:31.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:31.374+0000 7f08237fe640 1 --1- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000c080 0x7f081000eee0 :-1 s=OPENED pgs=269 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:38:31.367 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:31.374+0000 7f0821ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:38:31.376 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:38:31.378+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:38:32.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:32.193+0000 7f0820ff9640 10 monclient(hunting): tick 2024-01-24T23:38:32.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:32.193+0000 7f0820ff9640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:38:32.187 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:32.193+0000 7f0820ff9640 1 monclient(hunting): continuing hunt 2024-01-24T23:38:32.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:32.193+0000 7f0820ff9640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:38:32.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:32.193+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6793/0 conn(0x7f081000fa10 legacy=0x7f081000b970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:38:32.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:32.193+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.132:6791/0 conn(0x7f0810009fd0 legacy=0x7f081000a3b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:38:32.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:32.193+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6789/0 conn(0x7f081000c080 legacy=0x7f081000eee0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:38:32.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:32.193+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 reap_dead start 2024-01-24T23:38:32.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:32.193+0000 7f0820ff9640 10 monclient: _add_conns ranks=[6,4,2,5,3,0,8,1,7] 2024-01-24T23:38:32.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:32.193+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f0810011520 addr v1:172.21.15.115:6792/0 2024-01-24T23:38:32.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:32.193+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f0810009fd0 addr v1:172.21.15.115:6791/0 2024-01-24T23:38:32.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:32.193+0000 7f0820ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f081000b970 addr v1:172.21.15.115:6790/0 2024-01-24T23:38:32.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:32.193+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810010680 con 0x7f081000b970 2024-01-24T23:38:32.188 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:32.193+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100215f0 con 0x7f0810009fd0 2024-01-24T23:38:32.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:32.193+0000 7f0820ff9640 1 -- 172.21.15.132:0/2783829207 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100104f0 con 0x7f0810011520 2024-01-24T23:38:32.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:32.193+0000 7f0820ff9640 10 monclient(hunting): _renew_subs 2024-01-24T23:38:32.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:32.193+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000b970 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:38:32.189 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:32.193+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810011520 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:38:32.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:32.397+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000b970 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:38:32.390 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:32.397+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810011520 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:38:32.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:32.801+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000b970 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:38:32.794 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:32.801+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810011520 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:38:33.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:33.605+0000 7f08237fe640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000b970 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:38:33.595 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:33.605+0000 7f0822ffd640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810011520 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:38:35.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:35.161+0000 7f08299a4640 0 monclient(hunting): authenticate timed out after 300 2024-01-24T23:38:35.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:35.161+0000 7f08299a4640 10 monclient(hunting): shutdown 2024-01-24T23:38:35.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:35.161+0000 7f08299a4640 20 monclient(hunting): shutdown discarding 1 pending message(s) 2024-01-24T23:38:35.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:35.161+0000 7f08299a4640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6792/0 conn(0x7f0810011520 legacy=0x7f0810018210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:38:35.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:35.161+0000 7f08299a4640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6791/0 conn(0x7f0810009fd0 legacy=0x7f081000a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:38:35.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:35.161+0000 7f08299a4640 1 -- 172.21.15.132:0/2783829207 >> v1:172.21.15.115:6790/0 conn(0x7f081000b970 legacy=0x7f081000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:38:35.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:35.161+0000 7f08299a4640 1 -- 172.21.15.132:0/2783829207 shutdown_connections 2024-01-24T23:38:35.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:35.161+0000 7f08299a4640 1 -- 172.21.15.132:0/2783829207 >> 172.21.15.132:0/2783829207 conn(0x7f08240a1cc0 msgr2=0x7f08240a20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-24T23:38:35.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:35.161+0000 7f08299a4640 1 -- 172.21.15.132:0/2783829207 shutdown_connections 2024-01-24T23:38:35.153 INFO:tasks.workunit.client.0.smithi132.stderr:2024-01-24T23:38:35.161+0000 7f08299a4640 1 -- 172.21.15.132:0/2783829207 wait complete. 2024-01-24T23:38:35.154 INFO:tasks.workunit.client.0.smithi132.stderr:[errno 110] RADOS timed out (error connecting to the cluster) 2024-01-24T23:38:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status' 13 2024-01-24T23:38:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status' 2024-01-24T23:38:35.157 INFO:tasks.workunit.client.0.smithi132.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status 2024-01-24T23:38:35.157 INFO:tasks.workunit.client.0.smithi132.stderr:+ expected_ret=13 2024-01-24T23:38:35.158 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status 2024-01-24T23:38:35.158 INFO:tasks.workunit.client.0.smithi132.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status 2024-01-24T23:38:36.376 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:38:36.378+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:38:41.376 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:38:41.378+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:38:46.377 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:38:46.378+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:38:51.377 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:38:51.378+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:38:56.377 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:38:56.378+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:39:01.377 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:39:01.378+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:39:06.377 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:39:06.378+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:39:11.377 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:39:11.378+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:39:16.378 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:39:16.382+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:39:21.378 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:39:21.382+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:39:26.378 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:39:26.381+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:39:31.378 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:39:31.381+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:39:36.378 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:39:36.381+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:39:41.378 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:39:41.381+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:39:46.379 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:39:46.381+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:39:51.378 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:39:51.381+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:39:56.379 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:39:56.381+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:40:01.379 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:40:01.381+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:40:06.379 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:40:06.381+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:40:11.379 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:40:11.381+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:40:16.379 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:40:16.381+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:40:21.379 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:40:21.381+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:40:26.379 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:40:26.381+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:40:31.380 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:40:31.385+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:40:36.380 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:40:36.381+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:40:41.380 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:40:41.380+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:40:46.380 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:40:46.384+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:40:51.380 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:40:51.384+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:40:56.380 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:40:56.384+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:41:01.380 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:41:01.380+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:41:06.381 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:41:06.384+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:41:11.381 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:41:11.380+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:41:16.381 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:41:16.384+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:41:21.381 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:41:21.384+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:41:26.381 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:41:26.384+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:41:31.381 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:41:31.384+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:41:36.381 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:41:36.384+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:41:41.382 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:41:41.384+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:41:46.382 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:41:46.384+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:41:51.382 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:41:51.384+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:41:56.382 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:41:56.387+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:42:01.382 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:42:01.383+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:42:06.382 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:42:06.383+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:42:11.382 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:42:11.383+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:42:16.383 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:42:16.387+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:42:21.383 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:42:21.383+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:42:26.383 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:42:26.387+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:42:31.383 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:42:31.387+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:42:36.383 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:42:36.387+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:42:41.384 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:42:41.383+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:42:46.384 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:42:46.386+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:42:51.385 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:42:51.386+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:42:56.384 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:42:56.386+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:43:01.385 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:43:01.386+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:43:06.385 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:43:06.390+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:43:11.385 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:43:11.386+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:43:16.385 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:43:16.390+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:43:21.385 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:43:21.389+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:43:26.386 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:43:26.389+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:43:31.386 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:43:31.389+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:43:35.276 INFO:tasks.workunit.client.0.smithi132.stderr:+ ret=1 2024-01-24T23:43:35.276 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ 1 -ne 13 ]] 2024-01-24T23:43:35.277 INFO:tasks.workunit.client.0.smithi132.stderr:+ echo 'Error: Expected return 13, got 1' 2024-01-24T23:43:35.277 INFO:tasks.workunit.client.0.smithi132.stdout:Error: Expected return 13, got 1 2024-01-24T23:43:35.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T23:43:35.278 INFO:tasks.workunit.client.0.smithi132.stderr:+ [[ 1 -eq 1 ]] 2024-01-24T23:43:35.278 INFO:tasks.workunit.client.0.smithi132.stderr:+ exit 1 2024-01-24T23:43:35.278 INFO:tasks.workunit:Stopping ['mon/pool_ops.sh', 'mon/crush_ops.sh', 'mon/osd.sh', 'mon/caps.sh'] on client.0... 2024-01-24T23:43:35.279 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-01-24T23:43:35.566 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/workunit.py", line 129, in task p.spawn(_run_tests, ctx, refspec, role, tests, File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test mon/caps.sh) on smithi132 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=663873555d45def955d1cc09730bfece2847b821 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/caps.sh' 2024-01-24T23:43:35.771 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=0fe285c2d4424514acd34e2021aa0046 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/workunit.py", line 129, in task p.spawn(_run_tests, ctx, refspec, role, tests, File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test mon/caps.sh) on smithi132 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=663873555d45def955d1cc09730bfece2847b821 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/caps.sh' 2024-01-24T23:43:35.773 DEBUG:teuthology.run_tasks:Unwinding manager mon_thrash 2024-01-24T23:43:35.810 INFO:tasks.mon_thrash:joining mon_thrasher 2024-01-24T23:43:35.810 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-01-24T23:43:35.811 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-01-24T23:43:35.950 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:35.953+0000 7f672b863640 10 monclient: get_monmap_and_config 2024-01-24T23:43:35.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:35.953+0000 7f672b863640 10 monclient: build_initial_monmap 2024-01-24T23:43:35.951 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:35.953+0000 7f672b863640 10 monclient: monmap: 2024-01-24T23:43:35.951 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:43:35.951 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:43:35.951 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:43:35.955712+0000 2024-01-24T23:43:35.951 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:43:35.955712+0000 2024-01-24T23:43:35.951 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:43:35.951 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:43:35.952 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:43:35.952 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:43:35.952 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:43:35.952 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:43:35.952 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:43:35.952 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:43:35.952 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:43:35.952 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:43:35.952 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:43:35.953 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:43:35.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:35.953+0000 7f672b863640 1 Processor -- start 2024-01-24T23:43:35.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:35.953+0000 7f672b863640 1 -- start start 2024-01-24T23:43:35.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:35.953+0000 7f672b863640 10 monclient: init 2024-01-24T23:43:35.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:35.953+0000 7f672b863640 10 monclient: _reopen_session rank -1 2024-01-24T23:43:35.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:35.953+0000 7f672b863640 10 monclient: _add_conns ranks=[6,5,3,4,8,2,7,0,1] 2024-01-24T23:43:35.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:35.953+0000 7f672b863640 10 monclient(hunting): picked mon.noname-d con 0x7f6724133c10 addr v1:172.21.15.115:6792/0 2024-01-24T23:43:35.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:35.953+0000 7f672b863640 10 monclient(hunting): picked mon.noname-h con 0x7f6724136960 addr v1:172.21.15.132:6791/0 2024-01-24T23:43:35.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:35.953+0000 7f672b863640 10 monclient(hunting): picked mon.noname-g con 0x7f672413a470 addr v1:172.21.15.132:6790/0 2024-01-24T23:43:35.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:35.953+0000 7f672b863640 1 -- --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672404ed70 con 0x7f672413a470 2024-01-24T23:43:35.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:35.953+0000 7f672b863640 1 -- --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672409dee0 con 0x7f6724136960 2024-01-24T23:43:35.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:35.953+0000 7f672b863640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672409d400 con 0x7f6724133c10 2024-01-24T23:43:35.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:35.953+0000 7f672b863640 10 monclient(hunting): _renew_subs 2024-01-24T23:43:35.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:35.953+0000 7f672b863640 10 monclient(hunting): authenticate will time out at 2294.878367s 2024-01-24T23:43:35.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:35.953+0000 7f67295ff640 1 -- >> v1:172.21.15.115:6792/0 conn(0x7f6724133c10 legacy=0x7f6724133ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:43:35.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:35.953+0000 7f6729e00640 1 -- >> v1:172.21.15.132:6790/0 conn(0x7f672413a470 legacy=0x7f672413c860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:43:35.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:35.953+0000 7f6728dfe640 1 -- >> v1:172.21.15.132:6791/0 conn(0x7f6724136960 legacy=0x7f6724138d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:43:36.152 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:36.157+0000 7f67295ff640 1 -- >> v1:172.21.15.115:6792/0 conn(0x7f6724133c10 legacy=0x7f6724133ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:43:36.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:36.157+0000 7f6729e00640 1 -- >> v1:172.21.15.132:6790/0 conn(0x7f672413a470 legacy=0x7f672413c860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:43:36.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:36.157+0000 7f6728dfe640 1 -- >> v1:172.21.15.132:6791/0 conn(0x7f6724136960 legacy=0x7f6724138d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:43:36.386 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:43:36.389+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:43:36.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:36.561+0000 7f67295ff640 1 -- >> v1:172.21.15.115:6792/0 conn(0x7f6724133c10 legacy=0x7f6724133ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:43:36.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:36.561+0000 7f6728dfe640 1 -- >> v1:172.21.15.132:6791/0 conn(0x7f6724136960 legacy=0x7f6724138d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:43:36.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:36.561+0000 7f6729e00640 1 -- >> v1:172.21.15.132:6790/0 conn(0x7f672413a470 legacy=0x7f672413c860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:43:37.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:37.365+0000 7f67295ff640 1 -- >> v1:172.21.15.115:6792/0 conn(0x7f6724133c10 legacy=0x7f6724133ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:43:37.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:37.365+0000 7f6728dfe640 1 -- >> v1:172.21.15.132:6791/0 conn(0x7f6724136960 legacy=0x7f6724138d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:43:37.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:37.365+0000 7f6729e00640 1 -- >> v1:172.21.15.132:6790/0 conn(0x7f672413a470 legacy=0x7f672413c860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:43:38.952 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:38.953+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:43:38.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:38.953+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:43:38.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:38.953+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:43:38.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:38.953+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:43:38.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:38.953+0000 7f6722ffd640 1 -- >> v1:172.21.15.115:6792/0 conn(0x7f6724133c10 legacy=0x7f6724133ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:43:38.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:38.953+0000 7f6722ffd640 1 -- >> v1:172.21.15.132:6791/0 conn(0x7f6724136960 legacy=0x7f6724138d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:43:38.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:38.953+0000 7f6722ffd640 1 -- >> v1:172.21.15.132:6790/0 conn(0x7f672413a470 legacy=0x7f672413c860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:43:38.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:38.953+0000 7f6722ffd640 10 monclient: _add_conns ranks=[0,8,7,1,4,2] 2024-01-24T23:43:38.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:38.953+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-a con 0x7f6710003790 addr v1:172.21.15.115:6789/0 2024-01-24T23:43:38.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:38.953+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-e con 0x7f6710007420 addr v1:172.21.15.115:6793/0 2024-01-24T23:43:38.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:38.953+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-i con 0x7f671000b1b0 addr v1:172.21.15.132:6792/0 2024-01-24T23:43:38.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:38.953+0000 7f6722ffd640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672409f380 con 0x7f6710003790 2024-01-24T23:43:38.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:38.953+0000 7f6722ffd640 1 -- --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672404ed70 con 0x7f671000b1b0 2024-01-24T23:43:38.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:38.953+0000 7f6722ffd640 1 -- --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672409dee0 con 0x7f6710007420 2024-01-24T23:43:38.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:38.953+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:43:38.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:38.953+0000 7f6728dfe640 1 -- >> v1:172.21.15.115:6793/0 conn(0x7f6710007420 legacy=0x7f67100098e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:43:38.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:38.953+0000 7f6729e00640 1 -- >> v1:172.21.15.132:6792/0 conn(0x7f671000b1b0 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:43:38.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:38.953+0000 7f67295ff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f6710003790 0x7f6710005be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39442/0 (socket says 172.21.15.115:39442) 2024-01-24T23:43:38.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:38.953+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 learned_addr learned my addr 172.21.15.115:0/3847410394 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:43:39.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:39.157+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710007420 legacy=0x7f67100098e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:43:39.153 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:39.157+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000b1b0 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:43:39.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:39.561+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710007420 legacy=0x7f67100098e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:43:39.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:39.561+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000b1b0 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:43:40.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:40.365+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710007420 legacy=0x7f67100098e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:43:40.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:40.365+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000b1b0 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:43:41.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:41.377+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f6710003790 legacy=0x7f6710005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:43:41.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:41.377+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f6710003790 legacy=0x7f6710005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:43:41.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:41.377+0000 7f67295ff640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f6710003790 0x7f6710005be0 :-1 s=OPENED pgs=429 cs=1 l=1).handle_message read tag failed 2024-01-24T23:43:41.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:41.377+0000 7f67295ff640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f6710003790 0x7f6710005be0 :-1 s=OPENED pgs=429 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:43:41.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:41.377+0000 7f6723fff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:43:41.386 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:43:41.389+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:43:41.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:41.957+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:43:41.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:41.957+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:43:41.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:41.957+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:43:41.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:41.957+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:43:41.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:41.957+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710007420 legacy=0x7f67100098e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:43:41.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:41.957+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000b1b0 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:43:41.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:41.957+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f6710003790 legacy=0x7f6710005be0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:43:41.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:41.957+0000 7f6722ffd640 10 monclient: _add_conns ranks=[4,1,2] 2024-01-24T23:43:41.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:41.957+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-c con 0x7f6710010f70 addr v1:172.21.15.115:6791/0 2024-01-24T23:43:41.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:41.957+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:43:41.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:41.957+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-f con 0x7f671000eee0 addr v1:172.21.15.132:6789/0 2024-01-24T23:43:41.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:41.957+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-b con 0x7f671000fa00 addr v1:172.21.15.115:6790/0 2024-01-24T23:43:41.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:41.957+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672409d400 con 0x7f671000eee0 2024-01-24T23:43:41.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:41.957+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672404ed70 con 0x7f671000fa00 2024-01-24T23:43:41.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:41.957+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672409dee0 con 0x7f6710010f70 2024-01-24T23:43:41.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:41.957+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:43:41.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:41.957+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:43:41.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:41.957+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000fa00 legacy=0x7f6710019150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:43:42.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:42.161+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000fa00 legacy=0x7f6710019150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:43:42.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:42.565+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000fa00 legacy=0x7f6710019150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:43:43.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:43.369+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000fa00 legacy=0x7f6710019150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:43:44.953 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:44.953+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:43:44.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:44.953+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:43:44.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:44.953+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:43:44.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:44.953+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:43:44.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:44.953+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f6710010f70 legacy=0x7f6710011370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:43:44.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:44.953+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000fa00 legacy=0x7f6710019150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:43:44.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:44.953+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671000eee0 legacy=0x7f671000f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:43:44.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:44.953+0000 7f6722ffd640 10 monclient: _add_conns ranks=[3,0,7,8,5,1,2,4,6] 2024-01-24T23:43:44.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:44.953+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-g con 0x7f671000a300 addr v1:172.21.15.132:6790/0 2024-01-24T23:43:44.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:44.953+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-a con 0x7f671000a780 addr v1:172.21.15.115:6789/0 2024-01-24T23:43:44.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:44.953+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-i con 0x7f671000ac20 addr v1:172.21.15.132:6792/0 2024-01-24T23:43:44.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:44.953+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671000a0f0 con 0x7f671000a780 2024-01-24T23:43:44.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:44.953+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672404ed70 con 0x7f671000a300 2024-01-24T23:43:44.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:44.953+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001d9c0 con 0x7f671000ac20 2024-01-24T23:43:44.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:44.953+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:43:44.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:44.953+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000a300 legacy=0x7f6710005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:43:44.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:44.953+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000ac20 legacy=0x7f67100098e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:43:45.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:45.157+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000a300 legacy=0x7f6710005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:43:45.154 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:45.157+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000ac20 legacy=0x7f67100098e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:43:45.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:45.561+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000a300 legacy=0x7f6710005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:43:45.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:45.561+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000ac20 legacy=0x7f67100098e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:43:46.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:46.365+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000a300 legacy=0x7f6710005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:43:46.362 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:46.365+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000ac20 legacy=0x7f67100098e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:43:46.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:46.377+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f671000a780 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:43:46.376 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:46.377+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f671000a780 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:43:46.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:46.377+0000 7f6728dfe640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f671000a780 0x7f671000d670 :-1 s=OPENED pgs=430 cs=1 l=1).handle_message read tag failed 2024-01-24T23:43:46.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:46.377+0000 7f6728dfe640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f671000a780 0x7f671000d670 :-1 s=OPENED pgs=430 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:43:46.377 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:46.377+0000 7f6723fff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:43:46.386 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:43:46.389+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:43:47.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:47.953+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:43:47.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:47.953+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:43:47.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:47.953+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:43:47.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:47.953+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:43:47.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:47.953+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000ac20 legacy=0x7f67100098e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:43:47.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:47.953+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000a300 legacy=0x7f6710005be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:43:47.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:47.953+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f671000a780 legacy=0x7f671000d670 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:43:47.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:47.953+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:43:47.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:47.953+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:43:47.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:47.953+0000 7f6722ffd640 10 monclient: _add_conns ranks=[5,4,1,8,2,6] 2024-01-24T23:43:47.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:47.953+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-h con 0x7f671000ac20 addr v1:172.21.15.132:6791/0 2024-01-24T23:43:47.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:47.953+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-c con 0x7f671000a780 addr v1:172.21.15.115:6791/0 2024-01-24T23:43:47.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:47.953+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-f con 0x7f671000eee0 addr v1:172.21.15.132:6789/0 2024-01-24T23:43:47.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:47.953+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001ec10 con 0x7f671000eee0 2024-01-24T23:43:47.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:47.953+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672404ed70 con 0x7f671000a780 2024-01-24T23:43:47.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:47.953+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001d9c0 con 0x7f671000ac20 2024-01-24T23:43:47.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:47.957+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:43:47.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:47.957+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000ac20 legacy=0x7f6710011370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:43:48.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:48.161+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000ac20 legacy=0x7f6710011370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:43:48.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:48.564+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000ac20 legacy=0x7f6710011370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:43:49.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:49.368+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000ac20 legacy=0x7f6710011370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:43:50.954 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:50.956+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:43:50.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:50.956+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:43:50.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:50.956+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:43:50.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:50.956+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:43:50.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:50.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000ac20 legacy=0x7f6710011370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:43:50.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:50.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f671000a780 legacy=0x7f6710005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:43:50.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:50.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671000eee0 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:43:50.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:50.956+0000 7f6722ffd640 10 monclient: _add_conns ranks=[2,8,6] 2024-01-24T23:43:50.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:50.956+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-b con 0x7f6710009b40 addr v1:172.21.15.115:6790/0 2024-01-24T23:43:50.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:50.956+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-e con 0x7f671000fa30 addr v1:172.21.15.115:6793/0 2024-01-24T23:43:50.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:50.956+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-d con 0x7f671001e450 addr v1:172.21.15.115:6792/0 2024-01-24T23:43:50.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:50.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67100098f0 con 0x7f6710009b40 2024-01-24T23:43:50.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:50.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001d9c0 con 0x7f671001e450 2024-01-24T23:43:50.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:50.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710021050 con 0x7f671000fa30 2024-01-24T23:43:50.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:50.956+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:43:50.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:50.956+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710009b40 legacy=0x7f671000f2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:43:50.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:50.956+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671001e450 legacy=0x7f67100170c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:43:50.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:50.956+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f671000fa30 legacy=0x7f671001dbb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:43:51.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:51.160+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710009b40 legacy=0x7f671000f2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:43:51.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:51.160+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671001e450 legacy=0x7f67100170c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:43:51.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:51.160+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f671000fa30 legacy=0x7f671001dbb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:43:51.387 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:43:51.388+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:43:51.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:51.564+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710009b40 legacy=0x7f671000f2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:43:51.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:51.564+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671001e450 legacy=0x7f67100170c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:43:51.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:51.564+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f671000fa30 legacy=0x7f671001dbb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:43:52.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:52.368+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710009b40 legacy=0x7f671000f2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:43:52.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:52.368+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671001e450 legacy=0x7f67100170c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:43:52.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:52.368+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f671000fa30 legacy=0x7f671001dbb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:43:53.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:53.956+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:43:53.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:53.956+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:43:53.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:53.956+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:43:53.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:53.956+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:43:53.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:53.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f671000fa30 legacy=0x7f671001dbb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:43:53.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:53.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671001e450 legacy=0x7f67100170c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:43:53.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:53.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710009b40 legacy=0x7f671000f2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:43:53.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:53.956+0000 7f6722ffd640 10 monclient: _add_conns ranks=[7,3,8,6,5,0,4,2,1] 2024-01-24T23:43:53.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:53.956+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:43:53.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:53.956+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-i con 0x7f67100211e0 addr v1:172.21.15.132:6792/0 2024-01-24T23:43:53.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:53.956+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-g con 0x7f6710021d70 addr v1:172.21.15.132:6790/0 2024-01-24T23:43:53.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:53.956+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:43:53.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:53.956+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-e con 0x7f671001e450 addr v1:172.21.15.115:6793/0 2024-01-24T23:43:53.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:53.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67100221f0 con 0x7f6710021d70 2024-01-24T23:43:53.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:53.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67100098f0 con 0x7f67100211e0 2024-01-24T23:43:53.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:53.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001d9c0 con 0x7f671001e450 2024-01-24T23:43:53.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:53.956+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:43:53.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:53.956+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f671001e450 legacy=0x7f671001db50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:43:53.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:53.956+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710021d70 legacy=0x7f67100253a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:43:53.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:53.956+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f67100211e0 legacy=0x7f67100215c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:43:54.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:54.160+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f671001e450 legacy=0x7f671001db50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:43:54.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:54.160+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710021d70 legacy=0x7f67100253a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:43:54.159 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:54.160+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f67100211e0 legacy=0x7f67100215c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:43:54.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:54.564+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f671001e450 legacy=0x7f671001db50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:43:54.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:54.564+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710021d70 legacy=0x7f67100253a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:43:54.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:54.564+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f67100211e0 legacy=0x7f67100215c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:43:55.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:55.368+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f671001e450 legacy=0x7f671001db50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:43:55.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:55.368+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710021d70 legacy=0x7f67100253a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:43:55.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:55.368+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f67100211e0 legacy=0x7f67100215c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:43:56.387 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:43:56.388+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:43:56.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:56.956+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:43:56.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:56.956+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:43:56.955 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:56.956+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:43:56.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:56.956+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:43:56.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:56.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f671001e450 legacy=0x7f671001db50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:43:56.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:56.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f67100211e0 legacy=0x7f67100215c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:43:56.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:56.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710021d70 legacy=0x7f67100253a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:43:56.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:56.956+0000 7f6722ffd640 10 monclient: _add_conns ranks=[5,6,0,2,1,4] 2024-01-24T23:43:56.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:56.956+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-h con 0x7f6710009b40 addr v1:172.21.15.132:6791/0 2024-01-24T23:43:56.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:56.956+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-d con 0x7f671000fa30 addr v1:172.21.15.115:6792/0 2024-01-24T23:43:56.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:56.956+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-a con 0x7f671000a780 addr v1:172.21.15.115:6789/0 2024-01-24T23:43:56.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:56.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710021050 con 0x7f671000a780 2024-01-24T23:43:56.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:56.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67100221f0 con 0x7f6710009b40 2024-01-24T23:43:56.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:56.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67100098f0 con 0x7f671000fa30 2024-01-24T23:43:56.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:56.956+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:43:56.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:56.956+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000fa30 legacy=0x7f6710011370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:43:56.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:56.956+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710009b40 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:43:57.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:57.160+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000fa30 legacy=0x7f6710011370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:43:57.158 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:57.160+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710009b40 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:43:57.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:57.564+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000fa30 legacy=0x7f6710011370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:43:57.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:57.564+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710009b40 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:43:58.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:58.368+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000fa30 legacy=0x7f6710011370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:43:58.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:58.368+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710009b40 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:43:59.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:59.956+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:43:59.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:59.956+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:43:59.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:59.956+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:43:59.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:59.956+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:43:59.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:59.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000fa30 legacy=0x7f6710011370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:43:59.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:59.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710009b40 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:43:59.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:59.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f671000a780 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:43:59.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:59.956+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:43:59.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:59.956+0000 7f6722ffd640 10 monclient: _add_conns ranks=[4,2,1] 2024-01-24T23:43:59.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:59.956+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:43:59.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:59.956+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-c con 0x7f671000f5f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:43:59.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:59.956+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-b con 0x7f671000fa30 addr v1:172.21.15.115:6790/0 2024-01-24T23:43:59.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:59.956+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-f con 0x7f671001e420 addr v1:172.21.15.132:6789/0 2024-01-24T23:43:59.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:59.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001d9c0 con 0x7f671001e420 2024-01-24T23:43:59.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:59.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67100221f0 con 0x7f671000fa30 2024-01-24T23:43:59.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:59.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67100098f0 con 0x7f671000f5f0 2024-01-24T23:43:59.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:59.956+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:43:59.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:43:59.956+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000fa30 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:00.156 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:00.160+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000fa30 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:00.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:00.564+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000fa30 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:01.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:01.368+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000fa30 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:01.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:01.384+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671001e420 legacy=0x7f671001db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:44:01.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:01.384+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671001e420 legacy=0x7f671001db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:44:01.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:01.384+0000 7f6729e00640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671001e420 0x7f671001db50 :-1 s=OPENED pgs=434 cs=1 l=1).handle_message read tag failed 2024-01-24T23:44:01.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:01.384+0000 7f6729e00640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671001e420 0x7f671001db50 :-1 s=OPENED pgs=434 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:44:01.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:01.384+0000 7f6723fff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:44:01.387 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:44:01.388+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:44:01.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:01.388+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f671000f5f0 legacy=0x7f6710011370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:44:01.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:01.388+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f671000f5f0 legacy=0x7f6710011370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:44:01.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:01.388+0000 7f67295ff640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f671000f5f0 0x7f6710011370 :-1 s=OPENED pgs=440 cs=1 l=1).handle_message read tag failed 2024-01-24T23:44:01.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:01.388+0000 7f67295ff640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f671000f5f0 0x7f6710011370 :-1 s=OPENED pgs=440 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:44:01.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:01.388+0000 7f6723fff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:44:02.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:02.960+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:44:02.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:02.960+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:44:02.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:02.960+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:44:02.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:02.960+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:44:02.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:02.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f671000f5f0 legacy=0x7f6710011370 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:44:02.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:02.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000fa30 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:02.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:02.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671001e420 legacy=0x7f671001db50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:44:02.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:02.960+0000 7f6722ffd640 10 monclient: _add_conns ranks=[5,2,7,1,0,8,3,4,6] 2024-01-24T23:44:02.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:02.960+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-h con 0x7f6710021d70 addr v1:172.21.15.132:6791/0 2024-01-24T23:44:02.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:02.960+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-b con 0x7f671000fa30 addr v1:172.21.15.115:6790/0 2024-01-24T23:44:02.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:02.960+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-i con 0x7f671000ac00 addr v1:172.21.15.132:6792/0 2024-01-24T23:44:02.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:02.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671000a780 con 0x7f671000fa30 2024-01-24T23:44:02.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:02.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67100221f0 con 0x7f6710021d70 2024-01-24T23:44:02.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:02.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671000a990 con 0x7f671000ac00 2024-01-24T23:44:02.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:02.960+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:44:02.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:02.960+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000fa30 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:02.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:02.960+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000ac00 legacy=0x7f6710006bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:44:02.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:02.960+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710021d70 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:44:03.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:03.164+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000fa30 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:03.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:03.164+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000ac00 legacy=0x7f6710006bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:44:03.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:03.164+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710021d70 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:44:03.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:03.568+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000fa30 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:03.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:03.568+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710021d70 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:44:03.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:03.568+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000ac00 legacy=0x7f6710006bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:44:04.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:04.372+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000fa30 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:04.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:04.372+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000ac00 legacy=0x7f6710006bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:44:04.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:04.372+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710021d70 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:44:05.956 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:05.960+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:44:05.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:05.960+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:44:05.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:05.960+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:44:05.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:05.960+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:44:05.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:05.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000ac00 legacy=0x7f6710006bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:05.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:05.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710021d70 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:05.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:05.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000fa30 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:05.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:05.960+0000 7f6722ffd640 10 monclient: _add_conns ranks=[4,8,0,1,6,3] 2024-01-24T23:44:05.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:05.960+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-c con 0x7f671000f5f0 addr v1:172.21.15.115:6791/0 2024-01-24T23:44:05.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:05.960+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:44:05.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:05.960+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-e con 0x7f67100213d0 addr v1:172.21.15.115:6793/0 2024-01-24T23:44:05.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:05.960+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-a con 0x7f67100217b0 addr v1:172.21.15.115:6789/0 2024-01-24T23:44:05.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:05.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67100211e0 con 0x7f67100217b0 2024-01-24T23:44:05.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:05.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671000a780 con 0x7f671000f5f0 2024-01-24T23:44:05.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:05.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67100221f0 con 0x7f67100213d0 2024-01-24T23:44:05.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:05.960+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:44:05.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:05.960+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f67100213d0 legacy=0x7f67100170c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:44:06.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:06.164+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f67100213d0 legacy=0x7f67100170c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:44:06.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:06.380+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f67100217b0 legacy=0x7f6710022380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:44:06.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:06.380+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f67100217b0 legacy=0x7f6710022380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:44:06.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:06.380+0000 7f6729e00640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f67100217b0 0x7f6710022380 :-1 s=OPENED pgs=438 cs=1 l=1).handle_message read tag failed 2024-01-24T23:44:06.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:06.380+0000 7f6729e00640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f67100217b0 0x7f6710022380 :-1 s=OPENED pgs=438 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:44:06.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:06.380+0000 7f6723fff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:44:06.387 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:44:06.388+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:44:06.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:06.388+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f671000f5f0 legacy=0x7f6710011370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:44:06.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:06.388+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f671000f5f0 legacy=0x7f6710011370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:44:06.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:06.388+0000 7f67295ff640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f671000f5f0 0x7f6710011370 :-1 s=OPENED pgs=441 cs=1 l=1).handle_message read tag failed 2024-01-24T23:44:06.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:06.388+0000 7f67295ff640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f671000f5f0 0x7f6710011370 :-1 s=OPENED pgs=441 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:44:06.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:06.388+0000 7f6723fff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:44:06.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:06.568+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f67100213d0 legacy=0x7f67100170c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:44:07.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:07.372+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f67100213d0 legacy=0x7f67100170c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:44:08.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:08.956+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:44:08.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:08.956+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:44:08.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:08.956+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:44:08.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:08.956+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:44:08.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:08.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f67100213d0 legacy=0x7f67100170c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:08.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:08.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f671000f5f0 legacy=0x7f6710011370 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:44:08.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:08.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f67100217b0 legacy=0x7f6710022380 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:44:08.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:08.956+0000 7f6722ffd640 10 monclient: _add_conns ranks=[1,6,3] 2024-01-24T23:44:08.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:08.956+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-f con 0x7f6710021d70 addr v1:172.21.15.132:6789/0 2024-01-24T23:44:08.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:08.956+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-d con 0x7f671000ac00 addr v1:172.21.15.115:6792/0 2024-01-24T23:44:08.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:08.956+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-g con 0x7f671000fa30 addr v1:172.21.15.132:6790/0 2024-01-24T23:44:08.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:08.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671000a990 con 0x7f6710021d70 2024-01-24T23:44:08.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:08.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67100221f0 con 0x7f671000fa30 2024-01-24T23:44:08.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:08.956+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001e3f0 con 0x7f671000ac00 2024-01-24T23:44:08.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:08.956+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:44:08.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:08.956+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f671001db50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:08.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:08.956+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000fa30 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:09.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:09.160+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f671001db50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:09.157 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:09.160+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000fa30 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:09.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:09.564+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f671001db50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:09.562 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:09.564+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000fa30 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:10.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:10.368+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f671001db50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:10.366 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:10.368+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000fa30 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:11.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.384+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f6710021d70 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:44:11.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.384+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f6710021d70 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:44:11.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.384+0000 7f67295ff640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f6710021d70 0x7f671000d670 :-1 s=OPENED pgs=441 cs=1 l=1).handle_message read tag failed 2024-01-24T23:44:11.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.384+0000 7f67295ff640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f6710021d70 0x7f671000d670 :-1 s=OPENED pgs=441 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:44:11.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.384+0000 7f6723fff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:44:11.388 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:44:11.388+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:44:11.957 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.960+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:44:11.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.960+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:44:11.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.960+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:44:11.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.960+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:44:11.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f671001db50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:11.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000fa30 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:11.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f6710021d70 legacy=0x7f671000d670 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:44:11.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.960+0000 7f6722ffd640 10 monclient: _add_conns ranks=[2,8,7,0,4,6,5,3,1] 2024-01-24T23:44:11.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.960+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-b con 0x7f6710003bb0 addr v1:172.21.15.115:6790/0 2024-01-24T23:44:11.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.960+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:44:11.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.960+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-e con 0x7f67100213d0 addr v1:172.21.15.115:6793/0 2024-01-24T23:44:11.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.960+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-i con 0x7f6710006dc0 addr v1:172.21.15.132:6792/0 2024-01-24T23:44:11.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001e580 con 0x7f6710003bb0 2024-01-24T23:44:11.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67100221f0 con 0x7f6710006dc0 2024-01-24T23:44:11.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001e3f0 con 0x7f67100213d0 2024-01-24T23:44:11.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.960+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:44:11.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.960+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:44:11.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.960+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f67100213d0 legacy=0x7f67100066b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:44:11.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.960+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710003bb0 legacy=0x7f6710005fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:11.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:11.960+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f6710006dc0 legacy=0x7f67100180d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:44:12.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:12.164+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f67100213d0 legacy=0x7f67100066b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:44:12.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:12.164+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710003bb0 legacy=0x7f6710005fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:12.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:12.164+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f6710006dc0 legacy=0x7f67100180d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:44:12.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:12.568+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f67100213d0 legacy=0x7f67100066b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:44:12.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:12.568+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710003bb0 legacy=0x7f6710005fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:12.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:12.568+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f6710006dc0 legacy=0x7f67100180d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:44:13.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:13.372+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f67100213d0 legacy=0x7f67100066b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:44:13.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:13.372+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710003bb0 legacy=0x7f6710005fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:13.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:13.372+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f6710006dc0 legacy=0x7f67100180d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:44:14.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:14.960+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:44:14.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:14.960+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:44:14.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:14.960+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:44:14.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:14.960+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:44:14.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:14.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f67100213d0 legacy=0x7f67100066b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:14.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:14.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f6710006dc0 legacy=0x7f67100180d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:14.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:14.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710003bb0 legacy=0x7f6710005fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:14.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:14.960+0000 7f6722ffd640 10 monclient: _add_conns ranks=[6,5,1,4,0,3] 2024-01-24T23:44:14.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:14.960+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-d con 0x7f671000ac00 addr v1:172.21.15.115:6792/0 2024-01-24T23:44:14.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:14.960+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-h con 0x7f671000fa00 addr v1:172.21.15.132:6791/0 2024-01-24T23:44:14.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:14.960+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-f con 0x7f671001db50 addr v1:172.21.15.132:6789/0 2024-01-24T23:44:14.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:14.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671000f5f0 con 0x7f671001db50 2024-01-24T23:44:14.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:14.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001e580 con 0x7f671000fa00 2024-01-24T23:44:14.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:14.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67100221f0 con 0x7f671000ac00 2024-01-24T23:44:14.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:14.960+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:44:14.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:14.960+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:14.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:14.960+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000fa00 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:44:15.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:15.164+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:15.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:15.164+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000fa00 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:44:15.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:15.568+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:15.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:15.568+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000fa00 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:44:16.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:16.372+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:16.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:16.372+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000fa00 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:44:16.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:16.384+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671001db50 legacy=0x7f6710011370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:44:16.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:16.384+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671001db50 legacy=0x7f6710011370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:44:16.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:16.384+0000 7f6729e00640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671001db50 0x7f6710011370 :-1 s=OPENED pgs=442 cs=1 l=1).handle_message read tag failed 2024-01-24T23:44:16.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:16.384+0000 7f6729e00640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671001db50 0x7f6710011370 :-1 s=OPENED pgs=442 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:44:16.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:16.384+0000 7f6723fff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:44:16.388 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:44:16.388+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:44:17.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:17.960+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:44:17.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:17.960+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:44:17.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:17.960+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:44:17.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:17.960+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:44:17.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:17.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:17.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:17.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000fa00 legacy=0x7f671000d670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:17.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:17.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671001db50 legacy=0x7f6710011370 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:44:17.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:17.960+0000 7f6722ffd640 10 monclient: _add_conns ranks=[0,4,3] 2024-01-24T23:44:17.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:17.960+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-a con 0x7f67100217b0 addr v1:172.21.15.115:6789/0 2024-01-24T23:44:17.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:17.960+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:44:17.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:17.960+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-c con 0x7f6710021b90 addr v1:172.21.15.115:6791/0 2024-01-24T23:44:17.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:17.960+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-g con 0x7f671000ac00 addr v1:172.21.15.132:6790/0 2024-01-24T23:44:17.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:17.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001e3f0 con 0x7f67100217b0 2024-01-24T23:44:17.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:17.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001e580 con 0x7f671000ac00 2024-01-24T23:44:17.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:17.960+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67100221f0 con 0x7f6710021b90 2024-01-24T23:44:17.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:17.960+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:44:17.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:17.960+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:18.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:18.164+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:18.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:18.568+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:19.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:19.372+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:20.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:20.959+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:44:20.958 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:20.959+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:44:20.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:20.959+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:44:20.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:20.959+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:44:20.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:20.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f6710021b90 legacy=0x7f671001c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:44:20.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:20.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:20.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:20.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f67100217b0 legacy=0x7f6710022380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:44:20.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:20.959+0000 7f6722ffd640 10 monclient: _add_conns ranks=[2,4,5,6,8,3,7,1,0] 2024-01-24T23:44:20.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:20.959+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-b con 0x7f671000fa00 addr v1:172.21.15.115:6790/0 2024-01-24T23:44:20.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:20.959+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-c con 0x7f67100213d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:44:20.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:20.959+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-h con 0x7f6710006bd0 addr v1:172.21.15.132:6791/0 2024-01-24T23:44:20.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:20.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710021fa0 con 0x7f671000fa00 2024-01-24T23:44:20.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:20.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001e580 con 0x7f67100213d0 2024-01-24T23:44:20.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:20.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710007010 con 0x7f6710006bd0 2024-01-24T23:44:20.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:20.959+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:44:20.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:20.959+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000fa00 legacy=0x7f671001db50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:20.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:20.959+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710006bd0 legacy=0x7f6710011370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:44:21.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:21.163+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000fa00 legacy=0x7f671001db50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:21.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:21.163+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710006bd0 legacy=0x7f6710011370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:44:21.388 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:44:21.391+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:44:21.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:21.391+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f67100213d0 legacy=0x7f671000d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:44:21.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:21.391+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f67100213d0 legacy=0x7f671000d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:44:21.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:21.391+0000 7f6728dfe640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f67100213d0 0x7f671000d630 :-1 s=OPENED pgs=449 cs=1 l=1).handle_message read tag failed 2024-01-24T23:44:21.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:21.391+0000 7f6728dfe640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f67100213d0 0x7f671000d630 :-1 s=OPENED pgs=449 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:44:21.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:21.391+0000 7f6723fff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:44:21.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:21.567+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000fa00 legacy=0x7f671001db50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:21.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:21.567+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710006bd0 legacy=0x7f6710011370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:44:22.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:22.371+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000fa00 legacy=0x7f671001db50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:22.370 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:22.371+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710006bd0 legacy=0x7f6710011370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:44:23.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:23.959+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:44:23.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:23.959+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:44:23.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:23.959+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:44:23.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:23.959+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:44:23.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:23.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710006bd0 legacy=0x7f6710011370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:23.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:23.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f67100213d0 legacy=0x7f671000d630 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:44:23.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:23.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000fa00 legacy=0x7f671001db50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:23.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:23.959+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:44:23.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:23.959+0000 7f6722ffd640 10 monclient: _add_conns ranks=[6,3,7,8,1,0] 2024-01-24T23:44:23.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:23.959+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-d con 0x7f671000ac00 addr v1:172.21.15.115:6792/0 2024-01-24T23:44:23.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:23.959+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-g con 0x7f6710011370 addr v1:172.21.15.132:6790/0 2024-01-24T23:44:23.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:23.959+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-i con 0x7f671001db50 addr v1:172.21.15.132:6792/0 2024-01-24T23:44:23.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:23.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671000f780 con 0x7f6710011370 2024-01-24T23:44:23.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:23.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710021fa0 con 0x7f671000ac00 2024-01-24T23:44:23.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:23.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710007010 con 0x7f671001db50 2024-01-24T23:44:23.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:23.959+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:44:23.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:23.959+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:23.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:23.959+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671001db50 legacy=0x7f67100213d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:44:23.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:23.959+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710011370 legacy=0x7f671000d630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:24.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:24.163+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:24.162 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:24.163+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671001db50 legacy=0x7f67100213d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:44:24.163 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:24.163+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710011370 legacy=0x7f671000d630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:24.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:24.567+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:24.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:24.567+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710011370 legacy=0x7f671000d630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:24.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:24.567+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671001db50 legacy=0x7f67100213d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:44:25.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:25.371+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:25.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:25.371+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671001db50 legacy=0x7f67100213d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:44:25.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:25.371+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710011370 legacy=0x7f671000d630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:26.389 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:44:26.391+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:44:26.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:26.959+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:44:26.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:26.959+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:44:26.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:26.959+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:44:26.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:26.959+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:44:26.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:26.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671001db50 legacy=0x7f67100213d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:26.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:26.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:26.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:26.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710011370 legacy=0x7f671000d630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:26.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:26.959+0000 7f6722ffd640 10 monclient: _add_conns ranks=[0,1,8] 2024-01-24T23:44:26.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:26.959+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-a con 0x7f6710006bd0 addr v1:172.21.15.115:6789/0 2024-01-24T23:44:26.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:26.959+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-f con 0x7f67100157a0 addr v1:172.21.15.132:6789/0 2024-01-24T23:44:26.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:26.959+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-e con 0x7f6710019380 addr v1:172.21.15.115:6793/0 2024-01-24T23:44:26.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:26.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710011e20 con 0x7f6710006bd0 2024-01-24T23:44:26.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:26.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671000f780 con 0x7f67100157a0 2024-01-24T23:44:26.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:26.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710021fa0 con 0x7f6710019380 2024-01-24T23:44:26.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:26.959+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:44:26.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:26.959+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710019380 legacy=0x7f671001b7a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:44:27.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:27.163+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710019380 legacy=0x7f671001b7a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:44:27.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:27.567+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710019380 legacy=0x7f671001b7a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:44:28.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:28.371+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710019380 legacy=0x7f671001b7a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:44:29.959 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:29.959+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:44:29.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:29.959+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:44:29.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:29.959+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:44:29.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:29.959+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:44:29.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:29.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710019380 legacy=0x7f671001b7a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:29.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:29.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f67100157a0 legacy=0x7f6710017c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:44:29.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:29.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f6710006bd0 legacy=0x7f6710014080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:44:29.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:29.959+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:44:29.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:29.959+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:44:29.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:29.959+0000 7f6722ffd640 10 monclient: _add_conns ranks=[2,3,0,6,1,5,7,8,4] 2024-01-24T23:44:29.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:29.959+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-b con 0x7f671000ac00 addr v1:172.21.15.115:6790/0 2024-01-24T23:44:29.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:29.959+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-g con 0x7f6710019380 addr v1:172.21.15.132:6790/0 2024-01-24T23:44:29.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:29.959+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-a con 0x7f671001db50 addr v1:172.21.15.115:6789/0 2024-01-24T23:44:29.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:29.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710007010 con 0x7f671001db50 2024-01-24T23:44:29.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:29.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710021fa0 con 0x7f671000ac00 2024-01-24T23:44:29.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:29.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001df30 con 0x7f6710019380 2024-01-24T23:44:29.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:29.959+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:44:29.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:29.959+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:29.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:29.959+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710019380 legacy=0x7f67100213a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:30.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:30.163+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:30.160 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:30.163+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710019380 legacy=0x7f67100213a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:30.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:30.567+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:30.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:30.567+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710019380 legacy=0x7f67100213a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:31.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:31.371+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:31.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:31.371+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710019380 legacy=0x7f67100213a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:31.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:31.379+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f671001db50 legacy=0x7f671000d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:44:31.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:31.379+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f671001db50 legacy=0x7f671000d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:44:31.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:31.379+0000 7f6729e00640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f671001db50 0x7f671000d5f0 :-1 s=OPENED pgs=453 cs=1 l=1).handle_message read tag failed 2024-01-24T23:44:31.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:31.379+0000 7f6729e00640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f671001db50 0x7f671000d5f0 :-1 s=OPENED pgs=453 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:44:31.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:31.379+0000 7f6723fff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:44:31.388 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:44:31.387+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:44:32.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:32.963+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:44:32.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:32.963+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:44:32.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:32.963+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:44:32.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:32.963+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:44:32.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:32.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710019380 legacy=0x7f67100213a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:32.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:32.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:32.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:32.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f671001db50 legacy=0x7f671000d5f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:44:32.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:32.963+0000 7f6722ffd640 10 monclient: _add_conns ranks=[4,6,8,7,5,1] 2024-01-24T23:44:32.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:32.963+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-c con 0x7f6710011350 addr v1:172.21.15.115:6791/0 2024-01-24T23:44:32.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:32.963+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-d con 0x7f6710015bd0 addr v1:172.21.15.115:6792/0 2024-01-24T23:44:32.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:32.963+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-e con 0x7f6710018700 addr v1:172.21.15.115:6793/0 2024-01-24T23:44:32.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:32.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001e0c0 con 0x7f6710011350 2024-01-24T23:44:32.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:32.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710021fa0 con 0x7f6710015bd0 2024-01-24T23:44:32.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:32.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001df30 con 0x7f6710018700 2024-01-24T23:44:32.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:32.963+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:44:32.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:32.963+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f6710015bd0 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:32.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:32.963+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710018700 legacy=0x7f6710018b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:44:33.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:33.167+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710018700 legacy=0x7f6710018b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:44:33.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:33.167+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f6710015bd0 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:33.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:33.571+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710018700 legacy=0x7f6710018b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:44:33.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:33.571+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f6710015bd0 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:34.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:34.375+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f6710015bd0 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:34.371 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:34.375+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710018700 legacy=0x7f6710018b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:44:35.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:35.963+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:44:35.960 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:35.963+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:44:35.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:35.963+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:44:35.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:35.963+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:44:35.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:35.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710018700 legacy=0x7f6710018b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:35.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:35.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f6710015bd0 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:35.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:35.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f6710011350 legacy=0x7f6710014080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:44:35.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:35.963+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:44:35.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:35.963+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:44:35.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:35.963+0000 7f6722ffd640 10 monclient: _add_conns ranks=[5,7,1] 2024-01-24T23:44:35.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:35.963+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-h con 0x7f671000ac00 addr v1:172.21.15.132:6791/0 2024-01-24T23:44:35.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:35.963+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-i con 0x7f6710018700 addr v1:172.21.15.132:6792/0 2024-01-24T23:44:35.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:35.963+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-f con 0x7f671001db50 addr v1:172.21.15.132:6789/0 2024-01-24T23:44:35.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:35.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710011870 con 0x7f671001db50 2024-01-24T23:44:35.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:35.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710021fa0 con 0x7f671000ac00 2024-01-24T23:44:35.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:35.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001df30 con 0x7f6710018700 2024-01-24T23:44:35.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:35.963+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:44:35.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:35.963+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:44:35.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:35.963+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f6710018700 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:44:36.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:36.167+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f6710018700 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:44:36.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:36.167+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:44:36.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:36.387+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671001db50 legacy=0x7f671000d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:44:36.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:36.387+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671001db50 legacy=0x7f671000d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:44:36.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:36.387+0000 7f6729e00640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671001db50 0x7f671000d5f0 :-1 s=OPENED pgs=450 cs=1 l=1).handle_message read tag failed 2024-01-24T23:44:36.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:36.387+0000 7f6729e00640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671001db50 0x7f671000d5f0 :-1 s=OPENED pgs=450 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:44:36.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:36.387+0000 7f6723fff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:44:36.389 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:44:36.387+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:44:36.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:36.571+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f6710018700 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:44:36.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:36.571+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:44:37.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:37.375+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f6710018700 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:44:37.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:37.375+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:44:38.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:38.959+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:44:38.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:38.959+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:44:38.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:38.959+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:44:38.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:38.959+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:44:38.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:38.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f6710018700 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:38.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:38.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:38.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:38.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671001db50 legacy=0x7f671000d5f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:44:38.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:38.959+0000 7f6722ffd640 10 monclient: _add_conns ranks=[7,8,1,0,3,2,6,5,4] 2024-01-24T23:44:38.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:38.959+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-i con 0x7f6710018ae0 addr v1:172.21.15.132:6792/0 2024-01-24T23:44:38.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:38.959+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-e con 0x7f6710013080 addr v1:172.21.15.115:6793/0 2024-01-24T23:44:38.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:38.959+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-f con 0x7f671001db50 addr v1:172.21.15.132:6789/0 2024-01-24T23:44:38.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:38.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001e0c0 con 0x7f671001db50 2024-01-24T23:44:38.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:38.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710021fa0 con 0x7f6710018ae0 2024-01-24T23:44:38.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:38.959+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001df30 con 0x7f6710013080 2024-01-24T23:44:38.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:38.959+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:44:38.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:38.959+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710013080 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:44:38.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:38.959+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f6710018ae0 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:44:39.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:39.163+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710013080 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:44:39.161 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:39.163+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f6710018ae0 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:44:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:39.567+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710013080 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:44:39.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:39.567+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f6710018ae0 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:44:40.367 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:40.371+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710013080 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:44:40.368 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:40.371+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f6710018ae0 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:44:41.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:41.387+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671001db50 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:44:41.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:41.387+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671001db50 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:44:41.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:41.387+0000 7f6729e00640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671001db50 0x7f6710003bb0 :-1 s=OPENED pgs=457 cs=1 l=1).handle_message read tag failed 2024-01-24T23:44:41.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:41.387+0000 7f6729e00640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671001db50 0x7f6710003bb0 :-1 s=OPENED pgs=457 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:44:41.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:41.387+0000 7f6723fff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:44:41.389 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:44:41.387+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:44:41.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:41.963+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:44:41.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:41.963+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:44:41.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:41.963+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:44:41.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:41.963+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:44:41.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:41.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710013080 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:41.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:41.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f6710018ae0 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:41.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:41.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671001db50 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:44:41.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:41.963+0000 7f6722ffd640 10 monclient: _add_conns ranks=[6,0,3,4,2,5] 2024-01-24T23:44:41.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:41.963+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-d con 0x7f67100042c0 addr v1:172.21.15.115:6792/0 2024-01-24T23:44:41.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:41.963+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-a con 0x7f67100046a0 addr v1:172.21.15.115:6789/0 2024-01-24T23:44:41.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:41.963+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-g con 0x7f671001d370 addr v1:172.21.15.132:6790/0 2024-01-24T23:44:41.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:41.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710015470 con 0x7f67100046a0 2024-01-24T23:44:41.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:41.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710021fa0 con 0x7f671001d370 2024-01-24T23:44:41.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:41.963+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f67100042c0 legacy=0x7f671000c5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:41.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:41.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001df30 con 0x7f67100042c0 2024-01-24T23:44:41.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:41.963+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:44:41.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:41.963+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671001d370 legacy=0x7f6710025ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:42.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:42.167+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f67100042c0 legacy=0x7f671000c5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:42.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:42.167+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671001d370 legacy=0x7f6710025ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:42.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:42.571+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f67100042c0 legacy=0x7f671000c5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:42.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:42.571+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671001d370 legacy=0x7f6710025ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:43.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:43.375+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f67100042c0 legacy=0x7f671000c5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:43.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:43.375+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671001d370 legacy=0x7f6710025ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:44.961 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:44.963+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:44:44.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:44.963+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:44:44.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:44.963+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:44:44.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:44.963+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:44:44.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:44.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f67100042c0 legacy=0x7f671000c5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:44.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:44.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671001d370 legacy=0x7f6710025ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:44.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:44.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f67100046a0 legacy=0x7f6710022380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:44:44.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:44.963+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:44:44.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:44.963+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:44:44.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:44.963+0000 7f6722ffd640 10 monclient: _add_conns ranks=[4,2,5] 2024-01-24T23:44:44.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:44.963+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:44:44.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:44.963+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-c con 0x7f671000ac00 addr v1:172.21.15.115:6791/0 2024-01-24T23:44:44.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:44.963+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-b con 0x7f6710018ae0 addr v1:172.21.15.115:6790/0 2024-01-24T23:44:44.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:44.963+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-h con 0x7f671001db50 addr v1:172.21.15.132:6791/0 2024-01-24T23:44:44.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:44.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710018ec0 con 0x7f6710018ae0 2024-01-24T23:44:44.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:44.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710021fa0 con 0x7f671000ac00 2024-01-24T23:44:44.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:44.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001df30 con 0x7f671001db50 2024-01-24T23:44:44.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:44.963+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:44:44.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:44.963+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710018ae0 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:44.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:44.963+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671001db50 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:44:45.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:45.167+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671001db50 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:44:45.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:45.167+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710018ae0 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:45.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:45.571+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710018ae0 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:45.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:45.571+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671001db50 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:44:46.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:46.375+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710018ae0 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:46.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:46.375+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671001db50 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:44:46.391 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:44:46.387+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:44:46.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:46.387+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:44:46.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:46.387+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:44:46.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:46.387+0000 7f67295ff640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f671000ac00 0x7f671000eee0 :-1 s=OPENED pgs=463 cs=1 l=1).handle_message read tag failed 2024-01-24T23:44:46.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:46.387+0000 7f67295ff640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f671000ac00 0x7f671000eee0 :-1 s=OPENED pgs=463 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:44:46.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:46.387+0000 7f6723fff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:44:47.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:47.963+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:44:47.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:47.963+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:44:47.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:47.963+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:44:47.962 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:47.963+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:44:47.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:47.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671001db50 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:47.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:47.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:44:47.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:47.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710018ae0 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:47.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:47.963+0000 7f6722ffd640 10 monclient: _add_conns ranks=[5,1,7,3,8,2,0,6,4] 2024-01-24T23:44:47.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:47.963+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-h con 0x7f671001db50 addr v1:172.21.15.132:6791/0 2024-01-24T23:44:47.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:47.963+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-f con 0x7f6710023540 addr v1:172.21.15.132:6789/0 2024-01-24T23:44:47.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:47.963+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-i con 0x7f671000c5d0 addr v1:172.21.15.132:6792/0 2024-01-24T23:44:47.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:47.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710023390 con 0x7f6710023540 2024-01-24T23:44:47.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:47.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710018ec0 con 0x7f671001db50 2024-01-24T23:44:47.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:47.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001df30 con 0x7f671000c5d0 2024-01-24T23:44:47.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:47.963+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:44:47.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:47.963+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671001db50 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:44:47.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:47.963+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000c5d0 legacy=0x7f671000ca20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:44:48.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:48.167+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671001db50 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:44:48.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:48.167+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000c5d0 legacy=0x7f671000ca20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:44:48.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:48.571+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000c5d0 legacy=0x7f671000ca20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:44:48.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:48.571+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671001db50 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:44:49.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:49.375+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671001db50 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:44:49.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:49.375+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000c5d0 legacy=0x7f671000ca20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:44:50.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:50.963+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:44:50.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:50.963+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:44:50.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:50.963+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:44:50.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:50.963+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:44:50.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:50.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000c5d0 legacy=0x7f671000ca20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:50.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:50.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671001db50 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:50.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:50.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f6710023540 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:44:50.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:50.963+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:44:50.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:50.963+0000 7f6722ffd640 10 monclient: _add_conns ranks=[6,3,8,4,2,0] 2024-01-24T23:44:50.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:50.963+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-d con 0x7f671000ac00 addr v1:172.21.15.115:6792/0 2024-01-24T23:44:50.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:50.963+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-g con 0x7f671000c5d0 addr v1:172.21.15.132:6790/0 2024-01-24T23:44:50.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:50.963+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-e con 0x7f6710018ae0 addr v1:172.21.15.115:6793/0 2024-01-24T23:44:50.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:50.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710021fa0 con 0x7f671000c5d0 2024-01-24T23:44:50.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:50.963+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:50.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:50.963+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710018ae0 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:44:50.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:50.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710018ec0 con 0x7f671000ac00 2024-01-24T23:44:50.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:50.963+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000c5d0 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:50.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:50.963+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001df30 con 0x7f6710018ae0 2024-01-24T23:44:50.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:50.963+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:44:51.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:51.167+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710018ae0 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:44:51.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:51.167+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:51.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:51.167+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000c5d0 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:51.390 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:44:51.391+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:44:51.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:51.571+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710018ae0 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:44:51.567 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:51.571+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:51.568 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:51.571+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000c5d0 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:52.374 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:52.375+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710018ae0 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:44:52.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:52.375+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:52.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:52.375+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000c5d0 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:53.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:53.967+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:44:53.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:53.967+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:44:53.963 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:53.967+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:44:53.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:53.967+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:44:53.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:53.967+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710018ae0 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:53.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:53.967+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:53.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:53.967+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000c5d0 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:53.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:53.967+0000 7f6722ffd640 10 monclient: _add_conns ranks=[2,0,4] 2024-01-24T23:44:53.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:53.967+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-b con 0x7f671001db50 addr v1:172.21.15.115:6790/0 2024-01-24T23:44:53.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:53.967+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-a con 0x7f671000c9b0 addr v1:172.21.15.115:6789/0 2024-01-24T23:44:53.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:53.967+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-c con 0x7f6710015090 addr v1:172.21.15.115:6791/0 2024-01-24T23:44:53.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:53.967+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710023390 con 0x7f671000c9b0 2024-01-24T23:44:53.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:53.967+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710021fa0 con 0x7f671001db50 2024-01-24T23:44:53.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:53.967+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710018ec0 con 0x7f6710015090 2024-01-24T23:44:53.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:53.967+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:44:53.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:53.967+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671001db50 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:54.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:54.171+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671001db50 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:54.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:54.575+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671001db50 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:55.378 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:55.378+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671001db50 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:56.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.382+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f671000c9b0 legacy=0x7f671000cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:44:56.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.382+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f671000c9b0 legacy=0x7f671000cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:44:56.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.382+0000 7f6728dfe640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f671000c9b0 0x7f671000cdc0 :-1 s=OPENED pgs=467 cs=1 l=1).handle_message read tag failed 2024-01-24T23:44:56.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.382+0000 7f6728dfe640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f671000c9b0 0x7f671000cdc0 :-1 s=OPENED pgs=467 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:44:56.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.382+0000 7f6723fff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:44:56.390 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:44:56.386+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:44:56.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.386+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f6710015090 legacy=0x7f671001d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:44:56.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.386+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f6710015090 legacy=0x7f671001d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:44:56.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.386+0000 7f6729e00640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f6710015090 0x7f671001d0b0 :-1 s=OPENED pgs=470 cs=1 l=1).handle_message read tag failed 2024-01-24T23:44:56.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.386+0000 7f6729e00640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f6710015090 0x7f671001d0b0 :-1 s=OPENED pgs=470 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:44:56.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.386+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:44:56.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.386+0000 7f6723fff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:44:56.964 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.966+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:44:56.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.966+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:44:56.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.966+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:44:56.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.966+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:44:56.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f6710015090 legacy=0x7f671001d0b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:44:56.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671001db50 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:56.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f671000c9b0 legacy=0x7f671000cdc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:44:56.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.966+0000 7f6722ffd640 10 monclient: _add_conns ranks=[0,3,6,1,2,7,8,5,4] 2024-01-24T23:44:56.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.966+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-a con 0x7f671000c9b0 addr v1:172.21.15.115:6789/0 2024-01-24T23:44:56.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.966+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-g con 0x7f6710015090 addr v1:172.21.15.132:6790/0 2024-01-24T23:44:56.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.966+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-d con 0x7f671000ac00 addr v1:172.21.15.115:6792/0 2024-01-24T23:44:56.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001df30 con 0x7f671000c9b0 2024-01-24T23:44:56.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710021fa0 con 0x7f6710015090 2024-01-24T23:44:56.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710023390 con 0x7f671000ac00 2024-01-24T23:44:56.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.966+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:44:56.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.966+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:56.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:56.966+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710015090 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:57.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:57.170+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:57.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:57.170+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710015090 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:57.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:57.574+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:57.572 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:57.574+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710015090 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:58.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:58.378+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:44:58.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:58.378+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710015090 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:44:59.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:59.966+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:44:59.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:59.966+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:44:59.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:59.966+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:44:59.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:59.966+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:44:59.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:59.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000ac00 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:59.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:59.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710015090 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:44:59.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:59.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f671000c9b0 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:44:59.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:59.966+0000 7f6722ffd640 10 monclient: _add_conns ranks=[4,2,7,5,8,1] 2024-01-24T23:44:59.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:59.966+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-c con 0x7f6710018ae0 addr v1:172.21.15.115:6791/0 2024-01-24T23:44:59.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:59.966+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-b con 0x7f671001db50 addr v1:172.21.15.115:6790/0 2024-01-24T23:44:59.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:59.966+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-i con 0x7f671000d4a0 addr v1:172.21.15.132:6792/0 2024-01-24T23:44:59.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:59.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671000c5d0 con 0x7f671001db50 2024-01-24T23:44:59.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:59.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710021fa0 con 0x7f6710018ae0 2024-01-24T23:44:59.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:59.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710023390 con 0x7f671000d4a0 2024-01-24T23:44:59.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:59.966+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:44:59.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:59.966+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671001db50 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:44:59.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:44:59.966+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000d4a0 legacy=0x7f671001d0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:00.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:00.170+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671001db50 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:45:00.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:00.170+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000d4a0 legacy=0x7f671001d0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:00.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:00.574+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671001db50 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:45:00.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:00.574+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000d4a0 legacy=0x7f671001d0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:01.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:01.378+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671001db50 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:45:01.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:01.378+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000d4a0 legacy=0x7f671001d0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:01.390 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:45:01.390+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:45:01.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:01.390+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f6710018ae0 legacy=0x7f671000cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:45:01.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:01.390+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f6710018ae0 legacy=0x7f671000cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:45:01.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:01.390+0000 7f67295ff640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f6710018ae0 0x7f671000cd90 :-1 s=OPENED pgs=471 cs=1 l=1).handle_message read tag failed 2024-01-24T23:45:01.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:01.390+0000 7f67295ff640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f6710018ae0 0x7f671000cd90 :-1 s=OPENED pgs=471 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:45:01.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:01.390+0000 7f6723fff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:45:02.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:02.966+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:45:02.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:02.966+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:45:02.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:02.966+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:45:02.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:02.966+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:45:02.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:02.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000d4a0 legacy=0x7f671001d0b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:02.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:02.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f6710018ae0 legacy=0x7f671000cd90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:45:02.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:02.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f671001db50 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:02.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:02.966+0000 7f6722ffd640 10 monclient: _add_conns ranks=[5,1,8] 2024-01-24T23:45:02.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:02.966+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-h con 0x7f671000d8d0 addr v1:172.21.15.132:6791/0 2024-01-24T23:45:02.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:02.966+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:45:02.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:02.966+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-f con 0x7f6710028060 addr v1:172.21.15.132:6789/0 2024-01-24T23:45:02.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:02.966+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-e con 0x7f671002bb70 addr v1:172.21.15.115:6793/0 2024-01-24T23:45:02.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:02.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001df30 con 0x7f6710028060 2024-01-24T23:45:02.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:02.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671000c5d0 con 0x7f671000d8d0 2024-01-24T23:45:02.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:02.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710023390 con 0x7f671002bb70 2024-01-24T23:45:02.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:02.966+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:45:02.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:02.966+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:45:02.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:02.966+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f671002bb70 legacy=0x7f671002df60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:45:02.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:02.966+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000d8d0 legacy=0x7f6710026940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:03.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:03.170+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f671002bb70 legacy=0x7f671002df60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:45:03.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:03.170+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000d8d0 legacy=0x7f6710026940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:03.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:03.574+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f671002bb70 legacy=0x7f671002df60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:45:03.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:03.574+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000d8d0 legacy=0x7f6710026940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:04.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:04.378+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f671002bb70 legacy=0x7f671002df60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:45:04.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:04.378+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000d8d0 legacy=0x7f6710026940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:05.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:05.966+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:45:05.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:05.966+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:45:05.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:05.966+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:45:05.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:05.966+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:45:05.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:05.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f671002bb70 legacy=0x7f671002df60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:05.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:05.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000d8d0 legacy=0x7f6710026940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:05.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:05.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f6710028060 legacy=0x7f671002a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:45:05.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:05.966+0000 7f6722ffd640 10 monclient: _add_conns ranks=[5,8,1,2,4,7,0,3,6] 2024-01-24T23:45:05.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:05.966+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-h con 0x7f6710018ae0 addr v1:172.21.15.132:6791/0 2024-01-24T23:45:05.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:05.966+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-e con 0x7f671002bb70 addr v1:172.21.15.115:6793/0 2024-01-24T23:45:05.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:05.966+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-f con 0x7f671000abe0 addr v1:172.21.15.132:6789/0 2024-01-24T23:45:05.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:05.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710018ec0 con 0x7f671000abe0 2024-01-24T23:45:05.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:05.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671000c5d0 con 0x7f6710018ae0 2024-01-24T23:45:05.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:05.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710023390 con 0x7f671002bb70 2024-01-24T23:45:05.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:05.966+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:45:05.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:05.966+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710018ae0 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:05.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:05.966+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f671002bb70 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:45:06.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:06.170+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f671002bb70 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:45:06.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:06.170+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710018ae0 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:06.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:06.386+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671000abe0 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:45:06.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:06.386+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671000abe0 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:45:06.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:06.386+0000 7f6729e00640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671000abe0 0x7f6710017ff0 :-1 s=OPENED pgs=466 cs=1 l=1).handle_message read tag failed 2024-01-24T23:45:06.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:06.386+0000 7f6729e00640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671000abe0 0x7f6710017ff0 :-1 s=OPENED pgs=466 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:45:06.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:06.386+0000 7f6723fff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:45:06.390 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:45:06.390+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:45:06.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:06.574+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f671002bb70 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:45:06.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:06.574+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710018ae0 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:07.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:07.378+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f671002bb70 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:45:07.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:07.378+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710018ae0 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:08.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:08.966+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:45:08.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:08.966+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:45:08.965 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:08.966+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:45:08.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:08.966+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:45:08.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:08.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f671002bb70 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:08.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:08.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710018ae0 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:08.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:08.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671000abe0 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:45:08.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:08.966+0000 7f6722ffd640 10 monclient: _add_conns ranks=[6,3,4,7,2,0] 2024-01-24T23:45:08.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:08.966+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-d con 0x7f6710021370 addr v1:172.21.15.115:6792/0 2024-01-24T23:45:08.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:08.966+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-g con 0x7f671000d0c0 addr v1:172.21.15.132:6790/0 2024-01-24T23:45:08.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:08.966+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-c con 0x7f671002b460 addr v1:172.21.15.115:6791/0 2024-01-24T23:45:08.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:08.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001db50 con 0x7f671000d0c0 2024-01-24T23:45:08.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:08.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671000c5d0 con 0x7f671002b460 2024-01-24T23:45:08.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:08.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710023390 con 0x7f6710021370 2024-01-24T23:45:08.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:08.966+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:45:08.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:08.966+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f6710021370 legacy=0x7f671000c9b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:08.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:08.966+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000d0c0 legacy=0x7f6710014080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:45:09.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:09.170+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f6710021370 legacy=0x7f671000c9b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:09.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:09.170+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000d0c0 legacy=0x7f6710014080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:45:09.570 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:09.574+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f6710021370 legacy=0x7f671000c9b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:09.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:09.574+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000d0c0 legacy=0x7f6710014080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:45:10.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:10.378+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f6710021370 legacy=0x7f671000c9b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:10.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:10.378+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000d0c0 legacy=0x7f6710014080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:45:11.390 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:45:11.394+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:45:11.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:11.394+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f671002b460 legacy=0x7f6710014790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:45:11.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:11.394+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f671002b460 legacy=0x7f6710014790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:45:11.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:11.394+0000 7f6729e00640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f671002b460 0x7f6710014790 :-1 s=OPENED pgs=478 cs=1 l=1).handle_message read tag failed 2024-01-24T23:45:11.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:11.394+0000 7f6729e00640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f671002b460 0x7f6710014790 :-1 s=OPENED pgs=478 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:45:11.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:11.394+0000 7f6723fff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:45:11.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:11.966+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:45:11.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:11.966+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:45:11.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:11.966+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:45:11.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:11.966+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:45:11.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:11.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f6710021370 legacy=0x7f671000c9b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:11.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:11.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f671002b460 legacy=0x7f6710014790 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:45:11.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:11.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f671000d0c0 legacy=0x7f6710014080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:11.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:11.966+0000 7f6722ffd640 10 monclient: _add_conns ranks=[0,7,2] 2024-01-24T23:45:11.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:11.966+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:45:11.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:11.966+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-a con 0x7f6710014ea0 addr v1:172.21.15.115:6789/0 2024-01-24T23:45:11.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:11.966+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-i con 0x7f671000abe0 addr v1:172.21.15.132:6792/0 2024-01-24T23:45:11.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:11.966+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-b con 0x7f6710018ae0 addr v1:172.21.15.115:6790/0 2024-01-24T23:45:11.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:11.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710018ec0 con 0x7f6710014ea0 2024-01-24T23:45:11.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:11.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001db50 con 0x7f6710018ae0 2024-01-24T23:45:11.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:11.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710023390 con 0x7f671000abe0 2024-01-24T23:45:11.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:11.966+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:45:11.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:11.966+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710018ae0 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:45:11.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:11.966+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000abe0 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:12.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:12.170+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710018ae0 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:45:12.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:12.170+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000abe0 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:12.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:12.574+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710018ae0 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:45:12.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:12.574+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000abe0 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:13.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:13.378+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710018ae0 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:45:13.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:13.378+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000abe0 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:14.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:14.966+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:45:14.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:14.966+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:45:14.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:14.966+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:45:14.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:14.966+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:45:14.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:14.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000abe0 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:14.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:14.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710018ae0 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:14.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:14.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f6710014ea0 legacy=0x7f671002ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:45:14.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:14.966+0000 7f6722ffd640 10 monclient: _add_conns ranks=[0,1,7,2,6,3,5,8,4] 2024-01-24T23:45:14.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:14.966+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-a con 0x7f6710021370 addr v1:172.21.15.115:6789/0 2024-01-24T23:45:14.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:14.966+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-f con 0x7f671000c840 addr v1:172.21.15.132:6789/0 2024-01-24T23:45:14.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:14.966+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-i con 0x7f671000abe0 addr v1:172.21.15.132:6792/0 2024-01-24T23:45:14.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:14.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67100152b0 con 0x7f6710021370 2024-01-24T23:45:14.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:14.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001db50 con 0x7f671000c840 2024-01-24T23:45:14.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:14.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710023390 con 0x7f671000abe0 2024-01-24T23:45:14.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:14.966+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:45:14.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:14.966+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000abe0 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:15.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:15.170+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000abe0 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:15.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:15.574+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000abe0 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:16.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:16.378+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000abe0 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:16.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:16.382+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f6710021370 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:45:16.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:16.382+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f6710021370 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:45:16.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:16.382+0000 7f67295ff640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f6710021370 0x7f6710017ff0 :-1 s=OPENED pgs=476 cs=1 l=1).handle_message read tag failed 2024-01-24T23:45:16.380 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:16.382+0000 7f67295ff640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f6710021370 0x7f6710017ff0 :-1 s=OPENED pgs=476 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:45:16.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:16.382+0000 7f6723fff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:45:16.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:16.386+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671000c840 legacy=0x7f671000cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:45:16.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:16.386+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671000c840 legacy=0x7f671000cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:45:16.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:16.386+0000 7f6728dfe640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671000c840 0x7f671000cc20 :-1 s=OPENED pgs=473 cs=1 l=1).handle_message read tag failed 2024-01-24T23:45:16.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:16.386+0000 7f6728dfe640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671000c840 0x7f671000cc20 :-1 s=OPENED pgs=473 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:45:16.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:16.386+0000 7f6723fff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:45:16.390 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:45:16.394+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:45:17.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:17.966+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:45:17.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:17.966+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:45:17.966 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:17.966+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:45:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:17.966+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:45:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:17.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f671000abe0 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:17.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f671000c840 legacy=0x7f671000cc20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:45:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:17.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f6710021370 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:45:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:17.966+0000 7f6722ffd640 10 monclient: _add_conns ranks=[4,8,5,3,2,6] 2024-01-24T23:45:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:17.966+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-c con 0x7f6710014080 addr v1:172.21.15.115:6791/0 2024-01-24T23:45:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:17.966+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-e con 0x7f6710014b70 addr v1:172.21.15.115:6793/0 2024-01-24T23:45:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:17.966+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-h con 0x7f6710014f80 addr v1:172.21.15.132:6791/0 2024-01-24T23:45:17.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:17.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710018ec0 con 0x7f6710014080 2024-01-24T23:45:17.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:17.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710023390 con 0x7f6710014f80 2024-01-24T23:45:17.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:17.966+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001db50 con 0x7f6710014b70 2024-01-24T23:45:17.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:17.966+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:45:17.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:17.966+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710014b70 legacy=0x7f6710026940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:45:17.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:17.966+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710014f80 legacy=0x7f671001d0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:18.166 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:18.170+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710014b70 legacy=0x7f6710026940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:45:18.167 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:18.170+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710014f80 legacy=0x7f671001d0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:18.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:18.574+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710014b70 legacy=0x7f6710026940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:45:18.571 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:18.574+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710014f80 legacy=0x7f671001d0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:19.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:19.378+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710014b70 legacy=0x7f6710026940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:45:19.375 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:19.378+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710014f80 legacy=0x7f671001d0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:20.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:20.970+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:45:20.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:20.970+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:45:20.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:20.970+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:45:20.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:20.970+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:45:20.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:20.970+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710014b70 legacy=0x7f6710026940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:20.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:20.970+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f6710014f80 legacy=0x7f671001d0b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:20.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:20.970+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f6710014080 legacy=0x7f6710014460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:45:20.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:20.970+0000 7f6722ffd640 10 monclient: _add_conns ranks=[3,6,2] 2024-01-24T23:45:20.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:20.970+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:45:20.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:20.970+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-g con 0x7f6710029450 addr v1:172.21.15.132:6790/0 2024-01-24T23:45:20.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:20.970+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:45:20.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:20.970+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-d con 0x7f671000abe0 addr v1:172.21.15.115:6792/0 2024-01-24T23:45:20.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:20.970+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:45:20.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:20.970+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-b con 0x7f6710014b70 addr v1:172.21.15.115:6790/0 2024-01-24T23:45:20.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:20.970+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710023390 con 0x7f6710014b70 2024-01-24T23:45:20.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:20.970+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001db50 con 0x7f6710029450 2024-01-24T23:45:20.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:20.970+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001dd00 con 0x7f671000abe0 2024-01-24T23:45:20.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:20.970+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:45:20.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:20.970+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000abe0 legacy=0x7f6710014460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:20.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:20.970+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710014b70 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:45:20.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:20.970+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710029450 legacy=0x7f671002b840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:45:21.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:21.174+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000abe0 legacy=0x7f6710014460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:21.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:21.174+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710014b70 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:45:21.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:21.174+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710029450 legacy=0x7f671002b840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:45:21.391 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:45:21.394+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:45:21.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:21.578+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000abe0 legacy=0x7f6710014460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:21.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:21.578+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710014b70 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:45:21.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:21.578+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710029450 legacy=0x7f671002b840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:45:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:22.382+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000abe0 legacy=0x7f6710014460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:22.382+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710014b70 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:45:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:22.382+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710029450 legacy=0x7f671002b840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:45:23.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:23.970+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:45:23.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:23.970+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:45:23.967 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:23.970+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:45:23.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:23.970+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:45:23.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:23.970+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000abe0 legacy=0x7f6710014460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:23.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:23.970+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710029450 legacy=0x7f671002b840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:23.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:23.970+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710014b70 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:23.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:23.970+0000 7f6722ffd640 10 monclient: _add_conns ranks=[7,2,4,0,6,3,5,1,8] 2024-01-24T23:45:23.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:23.970+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-i con 0x7f6710014080 addr v1:172.21.15.132:6792/0 2024-01-24T23:45:23.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:23.970+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-b con 0x7f6710014f50 addr v1:172.21.15.115:6790/0 2024-01-24T23:45:23.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:23.970+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-c con 0x7f6710014b70 addr v1:172.21.15.115:6791/0 2024-01-24T23:45:23.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:23.970+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001de90 con 0x7f6710014f50 2024-01-24T23:45:23.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:23.970+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710023390 con 0x7f6710014b70 2024-01-24T23:45:23.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:23.970+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001db50 con 0x7f6710014080 2024-01-24T23:45:23.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:23.970+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:45:23.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:23.970+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710014f50 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:45:23.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:23.970+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f6710014080 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:24.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:24.174+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710014f50 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:45:24.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:24.174+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f6710014080 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:24.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:24.578+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710014f50 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:45:24.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:24.578+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f6710014080 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:25.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:25.382+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710014f50 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:45:25.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:25.382+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f6710014080 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:26.391 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:45:26.394+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:45:26.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:26.394+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f6710014b70 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:45:26.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:26.394+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f6710014b70 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:45:26.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:26.394+0000 7f6729e00640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f6710014b70 0x7f671000eee0 :-1 s=OPENED pgs=486 cs=1 l=1).handle_message read tag failed 2024-01-24T23:45:26.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:26.394+0000 7f6729e00640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f6710014b70 0x7f671000eee0 :-1 s=OPENED pgs=486 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:45:26.392 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:26.394+0000 7f6723fff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:45:26.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:26.970+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:45:26.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:26.970+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:45:26.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:26.970+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:45:26.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:26.970+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:45:26.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:26.970+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f6710014080 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:26.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:26.970+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6791/0 conn(0x7f6710014b70 legacy=0x7f671000eee0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:45:26.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:26.970+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6790/0 conn(0x7f6710014f50 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:26.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:26.970+0000 7f6722ffd640 10 monclient: _add_conns ranks=[6,8,0,3,1,5] 2024-01-24T23:45:26.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:26.970+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-d con 0x7f671000abe0 addr v1:172.21.15.115:6792/0 2024-01-24T23:45:26.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:26.970+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:45:26.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:26.970+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-e con 0x7f6710018ae0 addr v1:172.21.15.115:6793/0 2024-01-24T23:45:26.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:26.970+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-a con 0x7f671000c840 addr v1:172.21.15.115:6789/0 2024-01-24T23:45:26.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:26.970+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001dd00 con 0x7f671000c840 2024-01-24T23:45:26.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:26.970+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001de90 con 0x7f671000abe0 2024-01-24T23:45:26.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:26.970+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001db50 con 0x7f6710018ae0 2024-01-24T23:45:26.971 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:26.970+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:45:26.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:26.970+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000abe0 legacy=0x7f6710014460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:26.972 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:26.970+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710018ae0 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:45:27.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:27.174+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000abe0 legacy=0x7f6710014460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:27.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:27.174+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710018ae0 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:45:27.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:27.578+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000abe0 legacy=0x7f6710014460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:27.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:27.578+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710018ae0 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:45:28.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:28.382+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000abe0 legacy=0x7f6710014460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:28.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:28.382+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710018ae0 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:45:29.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:29.970+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:45:29.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:29.970+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:45:29.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:29.970+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:45:29.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:29.970+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:45:29.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:29.970+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6793/0 conn(0x7f6710018ae0 legacy=0x7f6710021370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:29.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:29.970+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000abe0 legacy=0x7f6710014460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:29.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:29.970+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6789/0 conn(0x7f671000c840 legacy=0x7f671001d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:45:29.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:29.970+0000 7f6722ffd640 10 monclient: _add_conns ranks=[3,5,1] 2024-01-24T23:45:29.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:29.970+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-g con 0x7f6710014080 addr v1:172.21.15.132:6790/0 2024-01-24T23:45:29.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:29.970+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-h con 0x7f671000cc50 addr v1:172.21.15.132:6791/0 2024-01-24T23:45:29.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:29.970+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-f con 0x7f6710014b70 addr v1:172.21.15.132:6789/0 2024-01-24T23:45:29.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:29.970+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710023390 con 0x7f6710014b70 2024-01-24T23:45:29.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:29.970+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001de90 con 0x7f6710014080 2024-01-24T23:45:29.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:29.970+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001db50 con 0x7f671000cc50 2024-01-24T23:45:29.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:29.970+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:45:29.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:29.970+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710014080 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:45:29.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:29.970+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000cc50 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:30.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:30.174+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710014080 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:45:30.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:30.174+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000cc50 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:30.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:30.578+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000cc50 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:30.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:30.578+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710014080 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:45:31.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:31.382+0000 7f6728dfe640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000cc50 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:31.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:31.382+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710014080 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:45:31.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:31.386+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f6710014b70 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:45:31.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:31.386+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f6710014b70 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:45:31.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:31.386+0000 7f6729e00640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f6710014b70 0x7f6710017ff0 :-1 s=OPENED pgs=480 cs=1 l=1).handle_message read tag failed 2024-01-24T23:45:31.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:31.386+0000 7f6729e00640 1 --1- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f6710014b70 0x7f6710017ff0 :-1 s=OPENED pgs=480 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:45:31.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:31.386+0000 7f6723fff640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:45:31.392 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:45:31.394+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:45:32.968 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:32.969+0000 7f6722ffd640 10 monclient(hunting): tick 2024-01-24T23:45:32.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:32.969+0000 7f6722ffd640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:45:32.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:32.969+0000 7f6722ffd640 1 monclient(hunting): continuing hunt 2024-01-24T23:45:32.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:32.969+0000 7f6722ffd640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:45:32.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:32.969+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6791/0 conn(0x7f671000cc50 legacy=0x7f671000eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:32.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:32.969+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6790/0 conn(0x7f6710014080 legacy=0x7f6710003bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:32.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:32.969+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6789/0 conn(0x7f6710014b70 legacy=0x7f6710017ff0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:45:32.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:32.969+0000 7f6722ffd640 10 monclient: _add_conns ranks=[7,1,6,2,8,0,3,5,4] 2024-01-24T23:45:32.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:32.969+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-i con 0x7f6710014f50 addr v1:172.21.15.132:6792/0 2024-01-24T23:45:32.969 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:32.969+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 reap_dead start 2024-01-24T23:45:32.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:32.969+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-f con 0x7f6710014b70 addr v1:172.21.15.132:6789/0 2024-01-24T23:45:32.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:32.969+0000 7f6722ffd640 10 monclient(hunting): picked mon.noname-d con 0x7f671000abe0 addr v1:172.21.15.115:6792/0 2024-01-24T23:45:32.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:32.969+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710018ec0 con 0x7f6710014b70 2024-01-24T23:45:32.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:32.969+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001de90 con 0x7f671000abe0 2024-01-24T23:45:32.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:32.969+0000 7f6722ffd640 1 -- 172.21.15.115:0/3847410394 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671001db50 con 0x7f6710014f50 2024-01-24T23:45:32.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:32.969+0000 7f6722ffd640 10 monclient(hunting): _renew_subs 2024-01-24T23:45:32.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:32.969+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000abe0 legacy=0x7f6710014460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:32.970 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:32.969+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f6710014f50 legacy=0x7f6710027950 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:33.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:33.173+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000abe0 legacy=0x7f6710014460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:33.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:33.173+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f6710014f50 legacy=0x7f6710027950 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:33.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:33.577+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000abe0 legacy=0x7f6710014460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:33.575 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:33.577+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f6710014f50 legacy=0x7f6710027950 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:34.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:34.381+0000 7f6729e00640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.115:6792/0 conn(0x7f671000abe0 legacy=0x7f6710014460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:34.379 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:34.381+0000 7f67295ff640 1 -- 172.21.15.115:0/3847410394 >> v1:172.21.15.132:6792/0 conn(0x7f6710014f50 legacy=0x7f6710027950 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:35.846 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-01-24T23:45:35.847 ERROR:teuthology.run_tasks:Manager failed: mon_thrash Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/mon_thrash.py", line 420, in task manager.wait_for_mon_quorum_size(len(mons)) File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 3172, in wait_for_mon_quorum_size if len(self.get_mon_quorum()) == size: File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 3157, in get_mon_quorum out = self.raw_cluster_cmd('quorum_status') File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 1627, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 1618, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi115 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status' 2024-01-24T23:45:35.857 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-01-24T23:45:35.868 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-01-24T23:45:35.868 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-24T23:45:35.997 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:35.997+0000 7f07a1191640 10 monclient: get_monmap_and_config 2024-01-24T23:45:35.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:35.997+0000 7f07a1191640 10 monclient: build_initial_monmap 2024-01-24T23:45:35.998 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:35.997+0000 7f07a1191640 10 monclient: monmap: 2024-01-24T23:45:35.998 INFO:teuthology.orchestra.run.smithi115.stderr:epoch 0 2024-01-24T23:45:35.998 INFO:teuthology.orchestra.run.smithi115.stderr:fsid 4c8039f0-ed94-40df-8309-0a291ebf7f77 2024-01-24T23:45:35.998 INFO:teuthology.orchestra.run.smithi115.stderr:last_changed 2024-01-24T23:45:36.001640+0000 2024-01-24T23:45:35.998 INFO:teuthology.orchestra.run.smithi115.stderr:created 2024-01-24T23:45:36.001640+0000 2024-01-24T23:45:35.998 INFO:teuthology.orchestra.run.smithi115.stderr:min_mon_release 0 (unknown) 2024-01-24T23:45:35.999 INFO:teuthology.orchestra.run.smithi115.stderr:election_strategy: 3 2024-01-24T23:45:35.999 INFO:teuthology.orchestra.run.smithi115.stderr:0: v1:172.21.15.115:6789/0 mon.noname-a 2024-01-24T23:45:35.999 INFO:teuthology.orchestra.run.smithi115.stderr:1: v1:172.21.15.132:6789/0 mon.noname-f 2024-01-24T23:45:35.999 INFO:teuthology.orchestra.run.smithi115.stderr:2: v1:172.21.15.115:6790/0 mon.noname-b 2024-01-24T23:45:35.999 INFO:teuthology.orchestra.run.smithi115.stderr:3: v1:172.21.15.132:6790/0 mon.noname-g 2024-01-24T23:45:35.999 INFO:teuthology.orchestra.run.smithi115.stderr:4: v1:172.21.15.115:6791/0 mon.noname-c 2024-01-24T23:45:35.999 INFO:teuthology.orchestra.run.smithi115.stderr:5: v1:172.21.15.132:6791/0 mon.noname-h 2024-01-24T23:45:35.999 INFO:teuthology.orchestra.run.smithi115.stderr:6: v1:172.21.15.115:6792/0 mon.noname-d 2024-01-24T23:45:36.000 INFO:teuthology.orchestra.run.smithi115.stderr:7: v1:172.21.15.132:6792/0 mon.noname-i 2024-01-24T23:45:36.000 INFO:teuthology.orchestra.run.smithi115.stderr:8: v1:172.21.15.115:6793/0 mon.noname-e 2024-01-24T23:45:36.000 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:45:36.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:35.997+0000 7f07a1191640 1 Processor -- start 2024-01-24T23:45:36.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:35.997+0000 7f07a1191640 1 -- start start 2024-01-24T23:45:36.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:35.997+0000 7f07a1191640 10 monclient: init 2024-01-24T23:45:36.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.001+0000 7f07a1191640 10 monclient: _reopen_session rank -1 2024-01-24T23:45:36.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.001+0000 7f07a1191640 10 monclient: _add_conns ranks=[0,6,1,3,2,8,4,7,5] 2024-01-24T23:45:36.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.001+0000 7f07a1191640 10 monclient(hunting): picked mon.noname-a con 0x7f079c134740 addr v1:172.21.15.115:6789/0 2024-01-24T23:45:36.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.001+0000 7f07a1191640 10 monclient(hunting): picked mon.noname-d con 0x7f079c09ec80 addr v1:172.21.15.115:6792/0 2024-01-24T23:45:36.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.001+0000 7f07a1191640 10 monclient(hunting): picked mon.noname-f con 0x7f079c135230 addr v1:172.21.15.132:6789/0 2024-01-24T23:45:36.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.001+0000 7f07a1191640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c04ed70 con 0x7f079c134740 2024-01-24T23:45:36.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.001+0000 7f07a1191640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c0a0d90 con 0x7f079c135230 2024-01-24T23:45:36.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.001+0000 7f07a1191640 1 -- --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c09e7c0 con 0x7f079c09ec80 2024-01-24T23:45:36.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.001+0000 7f07a1191640 10 monclient(hunting): _renew_subs 2024-01-24T23:45:36.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.001+0000 7f07a1191640 10 monclient(hunting): authenticate will time out at 2414.924273s 2024-01-24T23:45:36.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.001+0000 7f079ad9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f079c134740 0x7f079c134b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51902/0 (socket says 172.21.15.115:51902) 2024-01-24T23:45:36.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.001+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 learned_addr learned my addr 172.21.15.115:0/3438174064 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-01-24T23:45:36.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.001+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f079c09ec80 legacy=0x7f079c139d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:36.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.205+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f079c09ec80 legacy=0x7f079c139d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:36.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.381+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f079c134740 legacy=0x7f079c134b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:45:36.381 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.381+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f079c134740 legacy=0x7f079c134b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:45:36.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.381+0000 7f079ad9d640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f079c134740 0x7f079c134b20 :-1 s=OPENED pgs=484 cs=1 l=1).handle_message read tag failed 2024-01-24T23:45:36.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.381+0000 7f079ad9d640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f079c134740 0x7f079c134b20 :-1 s=OPENED pgs=484 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:45:36.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.381+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:45:36.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.385+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f079c135230 legacy=0x7f079c13d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:45:36.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.385+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f079c135230 legacy=0x7f079c13d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:45:36.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.385+0000 7f079b59e640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f079c135230 0x7f079c13d450 :-1 s=OPENED pgs=482 cs=1 l=1).handle_message read tag failed 2024-01-24T23:45:36.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.385+0000 7f079b59e640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f079c135230 0x7f079c13d450 :-1 s=OPENED pgs=482 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:45:36.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.385+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:45:36.391 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:45:36.393+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:45:36.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:36.609+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f079c09ec80 legacy=0x7f079c139d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:37.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:37.413+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f079c09ec80 legacy=0x7f079c139d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:39.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:39.001+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:45:39.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:39.001+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:45:39.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:39.001+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:45:39.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:39.001+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:45:39.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:39.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f079c09ec80 legacy=0x7f079c139d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:39.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:39.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f079c135230 legacy=0x7f079c13d450 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:45:39.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:39.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f079c134740 legacy=0x7f079c134b20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:45:39.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:39.001+0000 7f0798d99640 10 monclient: _add_conns ranks=[5,4,2,3,8,7] 2024-01-24T23:45:39.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:39.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-h con 0x7f077c003630 addr v1:172.21.15.132:6791/0 2024-01-24T23:45:39.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:39.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-c con 0x7f077c0072c0 addr v1:172.21.15.115:6791/0 2024-01-24T23:45:39.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:39.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-b con 0x7f077c00b050 addr v1:172.21.15.115:6790/0 2024-01-24T23:45:39.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:39.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c0a1070 con 0x7f077c00b050 2024-01-24T23:45:39.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:39.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c09e7c0 con 0x7f077c0072c0 2024-01-24T23:45:39.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:39.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c010eb0 con 0x7f077c003630 2024-01-24T23:45:39.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:39.001+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:45:39.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:39.001+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c00b050 legacy=0x7f077c00d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:45:39.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:39.001+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c003630 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:39.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:39.205+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c00b050 legacy=0x7f077c00d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:45:39.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:39.205+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c003630 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:39.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:39.609+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c00b050 legacy=0x7f077c00d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:45:39.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:39.609+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c003630 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:40.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:40.413+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c00b050 legacy=0x7f077c00d510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:45:40.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:40.413+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c003630 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:41.392 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:45:41.393+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:45:41.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:41.393+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c0072c0 legacy=0x7f077c009780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:45:41.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:41.393+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c0072c0 legacy=0x7f077c009780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:45:41.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:41.393+0000 7f079a59c640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c0072c0 0x7f077c009780 :-1 s=OPENED pgs=493 cs=1 l=1).handle_message read tag failed 2024-01-24T23:45:41.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:41.393+0000 7f079a59c640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c0072c0 0x7f077c009780 :-1 s=OPENED pgs=493 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:45:41.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:41.393+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:45:42.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.001+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:45:42.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.001+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:45:42.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.001+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:45:42.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.001+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:45:42.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c003630 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:42.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c0072c0 legacy=0x7f077c009780 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:45:42.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c00b050 legacy=0x7f077c00d510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:42.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.001+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:45:42.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.001+0000 7f0798d99640 10 monclient: _add_conns ranks=[7,8,3] 2024-01-24T23:45:42.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.001+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:45:42.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-i con 0x7f077c0110f0 addr v1:172.21.15.132:6792/0 2024-01-24T23:45:42.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-e con 0x7f077c011570 addr v1:172.21.15.115:6793/0 2024-01-24T23:45:42.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-g con 0x7f077c011b40 addr v1:172.21.15.132:6790/0 2024-01-24T23:45:42.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c012100 con 0x7f077c011b40 2024-01-24T23:45:42.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c0a1070 con 0x7f077c0110f0 2024-01-24T23:45:42.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c010eb0 con 0x7f077c011570 2024-01-24T23:45:42.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.001+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:45:42.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.001+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c011570 legacy=0x7f077c009780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:45:42.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.001+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c0110f0 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:42.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.001+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c011b40 legacy=0x7f077c00fdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:45:42.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.205+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c011570 legacy=0x7f077c009780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:45:42.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.205+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c011b40 legacy=0x7f077c00fdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:45:42.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.205+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c0110f0 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:42.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.609+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c011570 legacy=0x7f077c009780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:45:42.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.609+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c0110f0 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:42.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:42.609+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c011b40 legacy=0x7f077c00fdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:45:43.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:43.413+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c011570 legacy=0x7f077c009780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:45:43.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:43.413+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c0110f0 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:43.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:43.413+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c011b40 legacy=0x7f077c00fdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:45:45.000 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:45.001+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:45:45.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:45.001+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:45:45.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:45.001+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:45:45.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:45.001+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:45:45.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:45.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c011570 legacy=0x7f077c009780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:45.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:45.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c0110f0 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:45.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:45.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c011b40 legacy=0x7f077c00fdf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:45.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:45.001+0000 7f0798d99640 10 monclient: _add_conns ranks=[3,4,5,2,0,7,1,8,6] 2024-01-24T23:45:45.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:45.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-g con 0x7f077c010960 addr v1:172.21.15.132:6790/0 2024-01-24T23:45:45.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:45.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-c con 0x7f077c014400 addr v1:172.21.15.115:6791/0 2024-01-24T23:45:45.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:45.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-h con 0x7f077c00fdf0 addr v1:172.21.15.132:6791/0 2024-01-24T23:45:45.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:45.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c010650 con 0x7f077c010960 2024-01-24T23:45:45.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:45.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c012100 con 0x7f077c014400 2024-01-24T23:45:45.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:45.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c0a1070 con 0x7f077c00fdf0 2024-01-24T23:45:45.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:45.001+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:45:45.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:45.001+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c010960 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:45:45.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:45.001+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c00fdf0 legacy=0x7f077c009f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:45.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:45.205+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c00fdf0 legacy=0x7f077c009f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:45.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:45.205+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c010960 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:45:45.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:45.609+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c010960 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:45:45.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:45.609+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c00fdf0 legacy=0x7f077c009f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:46.392 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:45:46.393+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:45:46.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:46.393+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c014400 legacy=0x7f077c0167f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:45:46.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:46.393+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c014400 legacy=0x7f077c0167f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:45:46.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:46.393+0000 7f079a59c640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c014400 0x7f077c0167f0 :-1 s=OPENED pgs=494 cs=1 l=1).handle_message read tag failed 2024-01-24T23:45:46.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:46.393+0000 7f079a59c640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c014400 0x7f077c0167f0 :-1 s=OPENED pgs=494 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:45:46.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:46.393+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:45:46.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:46.413+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c010960 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:45:46.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:46.413+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c00fdf0 legacy=0x7f077c009f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:48.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:48.001+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:45:48.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:48.001+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:45:48.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:48.001+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:45:48.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:48.001+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:45:48.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:48.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c00fdf0 legacy=0x7f077c009f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:48.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:48.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c014400 legacy=0x7f077c0167f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:45:48.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:48.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c010960 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:48.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:48.001+0000 7f0798d99640 10 monclient: _add_conns ranks=[8,2,0,6,7,1] 2024-01-24T23:45:48.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:48.001+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:45:48.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:48.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-e con 0x7f077c011570 addr v1:172.21.15.115:6793/0 2024-01-24T23:45:48.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:48.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-b con 0x7f077c00fdf0 addr v1:172.21.15.115:6790/0 2024-01-24T23:45:48.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:48.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-a con 0x7f077c010960 addr v1:172.21.15.115:6789/0 2024-01-24T23:45:48.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:48.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c010eb0 con 0x7f077c010960 2024-01-24T23:45:48.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:48.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c010650 con 0x7f077c00fdf0 2024-01-24T23:45:48.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:48.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c0a1070 con 0x7f077c011570 2024-01-24T23:45:48.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:48.001+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:45:48.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:48.001+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c011570 legacy=0x7f077c009780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:45:48.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:48.001+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c00fdf0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:45:48.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:48.205+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c00fdf0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:45:48.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:48.205+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c011570 legacy=0x7f077c009780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:45:48.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:48.609+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c00fdf0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:45:48.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:48.609+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c011570 legacy=0x7f077c009780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:45:49.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:49.413+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c00fdf0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:45:49.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:49.413+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c011570 legacy=0x7f077c009780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:45:51.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.001+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:45:51.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.001+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:45:51.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.001+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:45:51.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.001+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:45:51.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c011570 legacy=0x7f077c009780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:51.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c00fdf0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:51.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c010960 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:45:51.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.001+0000 7f0798d99640 10 monclient: _add_conns ranks=[1,7,6] 2024-01-24T23:45:51.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-f con 0x7f077c006250 addr v1:172.21.15.132:6789/0 2024-01-24T23:45:51.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-i con 0x7f077c006d40 addr v1:172.21.15.132:6792/0 2024-01-24T23:45:51.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-d con 0x7f077c018b40 addr v1:172.21.15.115:6792/0 2024-01-24T23:45:51.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c0110c0 con 0x7f077c006250 2024-01-24T23:45:51.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c010650 con 0x7f077c018b40 2024-01-24T23:45:51.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c0a1070 con 0x7f077c006d40 2024-01-24T23:45:51.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.001+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:45:51.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.001+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c018b40 legacy=0x7f077c01af30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:51.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.001+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c006d40 legacy=0x7f077c017420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:51.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.205+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c018b40 legacy=0x7f077c01af30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:51.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.205+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c006d40 legacy=0x7f077c017420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:51.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.389+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c006250 legacy=0x7f077c006630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:45:51.387 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.389+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c006250 legacy=0x7f077c006630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:45:51.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.389+0000 7f079ad9d640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c006250 0x7f077c006630 :-1 s=OPENED pgs=489 cs=1 l=1).handle_message read tag failed 2024-01-24T23:45:51.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.389+0000 7f079ad9d640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c006250 0x7f077c006630 :-1 s=OPENED pgs=489 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:45:51.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.389+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:45:51.392 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:45:51.393+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:45:51.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.609+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c018b40 legacy=0x7f077c01af30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:51.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:51.609+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c006d40 legacy=0x7f077c017420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:52.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:52.413+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c018b40 legacy=0x7f077c01af30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:52.412 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:52.413+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c006d40 legacy=0x7f077c017420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:54.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:54.001+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:45:54.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:54.001+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:45:54.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:54.001+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:45:54.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:54.001+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:45:54.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:54.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c006d40 legacy=0x7f077c017420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:54.169 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:54.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c018b40 legacy=0x7f077c01af30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:54.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:54.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c006250 legacy=0x7f077c006630 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:45:54.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:54.001+0000 7f0798d99640 10 monclient: _add_conns ranks=[3,0,7,1,5,4,2,8,6] 2024-01-24T23:45:54.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:54.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-g con 0x7f077c01e6a0 addr v1:172.21.15.132:6790/0 2024-01-24T23:45:54.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:54.001+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:45:54.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:54.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-a con 0x7f077c010960 addr v1:172.21.15.115:6789/0 2024-01-24T23:45:54.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:54.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-i con 0x7f077c006d40 addr v1:172.21.15.132:6792/0 2024-01-24T23:45:54.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:54.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c01f6c0 con 0x7f077c010960 2024-01-24T23:45:54.170 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:54.001+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:45:54.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:54.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c010650 con 0x7f077c01e6a0 2024-01-24T23:45:54.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:54.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c0a1070 con 0x7f077c006d40 2024-01-24T23:45:54.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:54.001+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:45:54.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:54.001+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c01e6a0 legacy=0x7f077c01ea80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:45:54.171 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:54.001+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c006d40 legacy=0x7f077c017420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:54.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:54.205+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c01e6a0 legacy=0x7f077c01ea80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:45:54.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:54.205+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c006d40 legacy=0x7f077c017420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:54.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:54.609+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c01e6a0 legacy=0x7f077c01ea80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:45:54.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:54.609+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c006d40 legacy=0x7f077c017420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:55.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:55.413+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c01e6a0 legacy=0x7f077c01ea80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:45:55.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:55.413+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c006d40 legacy=0x7f077c017420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:45:56.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:56.381+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c010960 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:45:56.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:56.381+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c010960 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:45:56.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:56.381+0000 7f079a59c640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c010960 0x7f077c005a80 :-1 s=OPENED pgs=498 cs=1 l=1).handle_message read tag failed 2024-01-24T23:45:56.382 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:56.381+0000 7f079a59c640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c010960 0x7f077c005a80 :-1 s=OPENED pgs=498 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:45:56.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:56.381+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:45:56.392 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:45:56.393+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:45:57.001 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:57.001+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:45:57.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:57.001+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:45:57.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:57.001+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:45:57.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:57.001+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:45:57.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:57.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c006d40 legacy=0x7f077c017420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:57.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:57.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c01e6a0 legacy=0x7f077c01ea80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:45:57.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:57.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c010960 legacy=0x7f077c005a80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:45:57.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:57.001+0000 7f0798d99640 10 monclient: _add_conns ranks=[6,5,4,2,1,8] 2024-01-24T23:45:57.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:57.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-d con 0x7f077c011570 addr v1:172.21.15.115:6792/0 2024-01-24T23:45:57.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:57.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-h con 0x7f077c01f230 addr v1:172.21.15.132:6791/0 2024-01-24T23:45:57.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:57.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-c con 0x7f077c006230 addr v1:172.21.15.115:6791/0 2024-01-24T23:45:57.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:57.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c00fdf0 con 0x7f077c006230 2024-01-24T23:45:57.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:57.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c010650 con 0x7f077c01f230 2024-01-24T23:45:57.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:57.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c0a1070 con 0x7f077c011570 2024-01-24T23:45:57.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:57.001+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:45:57.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:57.001+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c011570 legacy=0x7f077c009780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:57.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:57.001+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c01f230 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:57.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:57.205+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c011570 legacy=0x7f077c009780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:57.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:57.205+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c01f230 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:57.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:57.609+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c011570 legacy=0x7f077c009780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:57.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:57.609+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c01f230 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:45:58.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:58.413+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c011570 legacy=0x7f077c009780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:45:58.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:45:58.413+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c01f230 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:00.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:00.001+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:46:00.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:00.001+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:46:00.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:00.001+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:46:00.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:00.001+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:46:00.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:00.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c011570 legacy=0x7f077c009780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:00.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:00.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c01f230 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:00.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:00.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c006230 legacy=0x7f077c0143c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:46:00.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:00.001+0000 7f0798d99640 10 monclient: _add_conns ranks=[1,2,8] 2024-01-24T23:46:00.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:00.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-f con 0x7f077c006640 addr v1:172.21.15.132:6789/0 2024-01-24T23:46:00.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:00.001+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:46:00.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:00.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-b con 0x7f077c01bb60 addr v1:172.21.15.115:6790/0 2024-01-24T23:46:00.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:00.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-e con 0x7f077c01bf70 addr v1:172.21.15.115:6793/0 2024-01-24T23:46:00.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:00.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c010000 con 0x7f077c006640 2024-01-24T23:46:00.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:00.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c010650 con 0x7f077c01bb60 2024-01-24T23:46:00.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:00.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c0a1070 con 0x7f077c01bf70 2024-01-24T23:46:00.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:00.001+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:46:00.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:00.001+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:46:00.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:00.001+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c01bb60 legacy=0x7f077c023900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:00.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:00.001+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c01bf70 legacy=0x7f077c027030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:00.202 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:00.205+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c01bb60 legacy=0x7f077c023900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:00.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:00.205+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c01bf70 legacy=0x7f077c027030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:00.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:00.609+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c01bb60 legacy=0x7f077c023900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:00.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:00.609+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c01bf70 legacy=0x7f077c027030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:01.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:01.389+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c006640 legacy=0x7f077c014ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:46:01.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:01.389+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c006640 legacy=0x7f077c014ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:46:01.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:01.389+0000 7f079ad9d640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c006640 0x7f077c014ad0 :-1 s=OPENED pgs=496 cs=1 l=1).handle_message read tag failed 2024-01-24T23:46:01.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:01.389+0000 7f079ad9d640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c006640 0x7f077c014ad0 :-1 s=OPENED pgs=496 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:46:01.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:01.389+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:46:01.392 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:46:01.393+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:46:01.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:01.413+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c01bf70 legacy=0x7f077c027030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:01.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:01.413+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c01bb60 legacy=0x7f077c023900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:03.002 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:03.001+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:46:03.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:03.001+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:46:03.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:03.001+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:46:03.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:03.001+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:46:03.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:03.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c01bf70 legacy=0x7f077c027030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:03.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:03.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c01bb60 legacy=0x7f077c023900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:03.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:03.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c006640 legacy=0x7f077c014ad0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:46:03.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:03.001+0000 7f0798d99640 10 monclient: _add_conns ranks=[8,0,2,3,5,4,1,6,7] 2024-01-24T23:46:03.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:03.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-e con 0x7f077c010930 addr v1:172.21.15.115:6793/0 2024-01-24T23:46:03.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:03.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-a con 0x7f077c01bf70 addr v1:172.21.15.115:6789/0 2024-01-24T23:46:03.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:03.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-b con 0x7f077c01bb60 addr v1:172.21.15.115:6790/0 2024-01-24T23:46:03.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:03.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c011550 con 0x7f077c01bf70 2024-01-24T23:46:03.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:03.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c010650 con 0x7f077c01bb60 2024-01-24T23:46:03.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:03.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c0a1070 con 0x7f077c010930 2024-01-24T23:46:03.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:03.001+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:46:03.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:03.001+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c010930 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:03.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:03.001+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c01bb60 legacy=0x7f077c0096b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:03.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:03.205+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c01bb60 legacy=0x7f077c0096b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:03.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:03.205+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c010930 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:03.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:03.609+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c010930 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:03.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:03.609+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c01bb60 legacy=0x7f077c0096b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:04.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:04.413+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c010930 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:04.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:04.413+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c01bb60 legacy=0x7f077c0096b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:06.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.001+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:46:06.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.001+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:46:06.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.001+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:46:06.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.001+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:46:06.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c010930 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:06.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c01bb60 legacy=0x7f077c0096b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:06.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c01bf70 legacy=0x7f077c014360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:46:06.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.001+0000 7f0798d99640 10 monclient: _add_conns ranks=[4,6,1,7,3,5] 2024-01-24T23:46:06.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-c con 0x7f077c006a20 addr v1:172.21.15.115:6791/0 2024-01-24T23:46:06.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-d con 0x7f077c006e00 addr v1:172.21.15.115:6792/0 2024-01-24T23:46:06.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.001+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-f con 0x7f077c006640 addr v1:172.21.15.132:6789/0 2024-01-24T23:46:06.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c0117a0 con 0x7f077c006640 2024-01-24T23:46:06.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c010650 con 0x7f077c006a20 2024-01-24T23:46:06.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.001+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c0a1070 con 0x7f077c006e00 2024-01-24T23:46:06.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.001+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:46:06.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.001+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c006e00 legacy=0x7f077c01e520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:06.203 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.205+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c006e00 legacy=0x7f077c01e520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:06.388 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.389+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c006640 legacy=0x7f077c014ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:46:06.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.389+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c006640 legacy=0x7f077c014ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:46:06.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.389+0000 7f079b59e640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c006640 0x7f077c014ad0 :-1 s=OPENED pgs=497 cs=1 l=1).handle_message read tag failed 2024-01-24T23:46:06.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.389+0000 7f079b59e640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c006640 0x7f077c014ad0 :-1 s=OPENED pgs=497 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:46:06.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.389+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:46:06.393 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:46:06.393+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:46:06.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.393+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c006a20 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:46:06.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.393+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c006a20 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:46:06.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.393+0000 7f079ad9d640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c006a20 0x7f077c005a80 :-1 s=OPENED pgs=502 cs=1 l=1).handle_message read tag failed 2024-01-24T23:46:06.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.393+0000 7f079ad9d640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c006a20 0x7f077c005a80 :-1 s=OPENED pgs=502 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:46:06.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.393+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:46:06.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.393+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:46:06.607 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:06.609+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c006e00 legacy=0x7f077c01e520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:07.411 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:07.413+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c006e00 legacy=0x7f077c01e520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:09.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:09.005+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:46:09.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:09.005+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:46:09.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:09.005+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:46:09.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:09.005+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:46:09.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:09.005+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c006e00 legacy=0x7f077c01e520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:09.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:09.005+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c006a20 legacy=0x7f077c005a80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:46:09.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:09.005+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c006640 legacy=0x7f077c014ad0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:46:09.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:09.005+0000 7f0798d99640 10 monclient: _add_conns ranks=[3,7,5] 2024-01-24T23:46:09.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:09.005+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-g con 0x7f077c006640 addr v1:172.21.15.132:6790/0 2024-01-24T23:46:09.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:09.005+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-i con 0x7f077c006a20 addr v1:172.21.15.132:6792/0 2024-01-24T23:46:09.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:09.005+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-h con 0x7f077c010930 addr v1:172.21.15.132:6791/0 2024-01-24T23:46:09.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:09.005+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006190 con 0x7f077c006640 2024-01-24T23:46:09.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:09.005+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c0a1070 con 0x7f077c010930 2024-01-24T23:46:09.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:09.005+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c010390 con 0x7f077c006a20 2024-01-24T23:46:09.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:09.005+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:46:09.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:09.005+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c006640 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:09.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:09.005+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c006a20 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:46:09.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:09.005+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c010930 legacy=0x7f077c014360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:09.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:09.209+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c006640 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:09.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:09.209+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c010930 legacy=0x7f077c014360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:09.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:09.209+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c006a20 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:46:09.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:09.613+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c006640 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:09.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:09.613+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c006a20 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:46:09.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:09.613+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c010930 legacy=0x7f077c014360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:10.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:10.417+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c006640 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:10.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:10.417+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c006a20 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:46:10.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:10.417+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c010930 legacy=0x7f077c014360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:11.393 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:46:11.393+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:46:12.003 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:12.004+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:46:12.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:12.004+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:46:12.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:12.004+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:46:12.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:12.004+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:46:12.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:12.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c006a20 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:12.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:12.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c010930 legacy=0x7f077c014360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:12.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:12.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c006640 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:12.004+0000 7f0798d99640 10 monclient: _add_conns ranks=[2,5,6,8,1,0,3,4,7] 2024-01-24T23:46:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:12.004+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-b con 0x7f077c014ad0 addr v1:172.21.15.115:6790/0 2024-01-24T23:46:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:12.004+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-h con 0x7f077c010930 addr v1:172.21.15.132:6791/0 2024-01-24T23:46:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:12.004+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-d con 0x7f077c006e00 addr v1:172.21.15.115:6792/0 2024-01-24T23:46:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:12.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006360 con 0x7f077c014ad0 2024-01-24T23:46:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:12.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006190 con 0x7f077c010930 2024-01-24T23:46:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:12.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c0a1070 con 0x7f077c006e00 2024-01-24T23:46:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:12.004+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:46:12.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:12.004+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c014ad0 legacy=0x7f077c0096b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:12.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:12.004+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c006e00 legacy=0x7f077c01e520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:12.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:12.004+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c010930 legacy=0x7f077c014360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:12.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:12.208+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c014ad0 legacy=0x7f077c0096b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:12.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:12.208+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c006e00 legacy=0x7f077c01e520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:12.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:12.208+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c010930 legacy=0x7f077c014360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:12.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:12.612+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c014ad0 legacy=0x7f077c0096b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:12.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:12.612+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c006e00 legacy=0x7f077c01e520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:12.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:12.612+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c010930 legacy=0x7f077c014360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:13.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:13.416+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c014ad0 legacy=0x7f077c0096b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:13.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:13.416+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c006e00 legacy=0x7f077c01e520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:13.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:13.416+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c010930 legacy=0x7f077c014360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:15.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:15.004+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:46:15.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:15.004+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:46:15.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:15.004+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:46:15.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:15.004+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:46:15.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:15.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c006e00 legacy=0x7f077c01e520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:15.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:15.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c010930 legacy=0x7f077c014360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:15.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:15.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c014ad0 legacy=0x7f077c0096b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:15.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:15.004+0000 7f0798d99640 10 monclient: _add_conns ranks=[3,4,1,8,0,7] 2024-01-24T23:46:15.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:15.004+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:46:15.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:15.004+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-g con 0x7f077c006640 addr v1:172.21.15.132:6790/0 2024-01-24T23:46:15.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:15.004+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-c con 0x7f077c015ec0 addr v1:172.21.15.115:6791/0 2024-01-24T23:46:15.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:15.004+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-f con 0x7f077c01a450 addr v1:172.21.15.132:6789/0 2024-01-24T23:46:15.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:15.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c010390 con 0x7f077c01a450 2024-01-24T23:46:15.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:15.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006360 con 0x7f077c006640 2024-01-24T23:46:15.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:15.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006190 con 0x7f077c015ec0 2024-01-24T23:46:15.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:15.004+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:46:15.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:15.004+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c006640 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:15.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:15.208+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c006640 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:15.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:15.612+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c006640 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:16.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:16.388+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c01a450 legacy=0x7f077c01a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:46:16.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:16.388+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c01a450 legacy=0x7f077c01a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:46:16.389 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:16.388+0000 7f079b59e640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c01a450 0x7f077c01a860 :-1 s=OPENED pgs=504 cs=1 l=1).handle_message read tag failed 2024-01-24T23:46:16.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:16.388+0000 7f079b59e640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c01a450 0x7f077c01a860 :-1 s=OPENED pgs=504 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:46:16.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:16.388+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:46:16.393 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:46:16.392+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:46:16.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:16.392+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c015ec0 legacy=0x7f077c01ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:46:16.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:16.392+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c015ec0 legacy=0x7f077c01ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:46:16.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:16.392+0000 7f079a59c640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c015ec0 0x7f077c01ec30 :-1 s=OPENED pgs=509 cs=1 l=1).handle_message read tag failed 2024-01-24T23:46:16.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:16.392+0000 7f079a59c640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c015ec0 0x7f077c01ec30 :-1 s=OPENED pgs=509 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:46:16.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:16.392+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:46:16.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:16.416+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c006640 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:18.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:18.004+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:46:18.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:18.004+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:46:18.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:18.004+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:46:18.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:18.004+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:46:18.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:18.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c015ec0 legacy=0x7f077c01ec30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:46:18.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:18.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c006640 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:18.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:18.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c01a450 legacy=0x7f077c01a860 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:46:18.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:18.004+0000 7f0798d99640 10 monclient: _add_conns ranks=[0,8,7] 2024-01-24T23:46:18.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:18.004+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-a con 0x7f077c010930 addr v1:172.21.15.115:6789/0 2024-01-24T23:46:18.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:18.004+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-e con 0x7f077c01bf80 addr v1:172.21.15.115:6793/0 2024-01-24T23:46:18.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:18.004+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-i con 0x7f077c006a20 addr v1:172.21.15.132:6792/0 2024-01-24T23:46:18.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:18.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c0a1070 con 0x7f077c010930 2024-01-24T23:46:18.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:18.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006360 con 0x7f077c006a20 2024-01-24T23:46:18.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:18.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c01f490 con 0x7f077c01bf80 2024-01-24T23:46:18.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:18.004+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:46:18.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:18.004+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c01bf80 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:18.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:18.004+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c006a20 legacy=0x7f077c0096b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:46:18.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:18.208+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c01bf80 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:18.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:18.208+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c006a20 legacy=0x7f077c0096b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:46:18.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:18.612+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c01bf80 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:18.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:18.612+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c006a20 legacy=0x7f077c0096b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:46:19.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:19.416+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c01bf80 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:19.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:19.416+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c006a20 legacy=0x7f077c0096b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:46:21.004 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.004+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:46:21.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.004+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:46:21.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.004+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:46:21.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.004+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:46:21.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c01bf80 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:21.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c006a20 legacy=0x7f077c0096b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:21.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c010930 legacy=0x7f077c014360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:46:21.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.004+0000 7f0798d99640 10 monclient: _add_conns ranks=[4,2,0,3,1,7,6,5,8] 2024-01-24T23:46:21.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.004+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:46:21.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.004+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-c con 0x7f077c015ec0 addr v1:172.21.15.115:6791/0 2024-01-24T23:46:21.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.004+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-b con 0x7f077c006e30 addr v1:172.21.15.115:6790/0 2024-01-24T23:46:21.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.004+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-a con 0x7f077c021860 addr v1:172.21.15.115:6789/0 2024-01-24T23:46:21.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c015ae0 con 0x7f077c021860 2024-01-24T23:46:21.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006360 con 0x7f077c006e30 2024-01-24T23:46:21.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c01f490 con 0x7f077c015ec0 2024-01-24T23:46:21.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.004+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:46:21.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.004+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:46:21.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.004+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c006e30 legacy=0x7f077c01eb40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:21.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.208+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c006e30 legacy=0x7f077c01eb40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:21.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.384+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c021860 legacy=0x7f077c023c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:46:21.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.384+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c021860 legacy=0x7f077c023c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:46:21.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.384+0000 7f079b59e640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c021860 0x7f077c023c50 :-1 s=OPENED pgs=507 cs=1 l=1).handle_message read tag failed 2024-01-24T23:46:21.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.384+0000 7f079b59e640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c021860 0x7f077c023c50 :-1 s=OPENED pgs=507 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:46:21.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.384+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:46:21.393 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:46:21.392+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:46:21.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.392+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c015ec0 legacy=0x7f077c01e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:46:21.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.392+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c015ec0 legacy=0x7f077c01e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:46:21.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.392+0000 7f079ad9d640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c015ec0 0x7f077c01e430 :-1 s=OPENED pgs=510 cs=1 l=1).handle_message read tag failed 2024-01-24T23:46:21.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.392+0000 7f079ad9d640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c015ec0 0x7f077c01e430 :-1 s=OPENED pgs=510 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:46:21.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.392+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:46:21.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:21.612+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c006e30 legacy=0x7f077c01eb40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:22.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:22.416+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c006e30 legacy=0x7f077c01eb40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:24.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:24.004+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:46:24.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:24.004+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:46:24.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:24.004+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:46:24.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:24.004+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:46:24.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:24.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c015ec0 legacy=0x7f077c01e430 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:46:24.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:24.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c006e30 legacy=0x7f077c01eb40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:24.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:24.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c021860 legacy=0x7f077c023c50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:46:24.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:24.004+0000 7f0798d99640 10 monclient: _add_conns ranks=[6,7,3,1,8,5] 2024-01-24T23:46:24.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:24.004+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-d con 0x7f077c0067d0 addr v1:172.21.15.115:6792/0 2024-01-24T23:46:24.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:24.004+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-i con 0x7f077c00f490 addr v1:172.21.15.132:6792/0 2024-01-24T23:46:24.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:24.004+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-g con 0x7f077c00f8a0 addr v1:172.21.15.132:6790/0 2024-01-24T23:46:24.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:24.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006640 con 0x7f077c00f8a0 2024-01-24T23:46:24.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:24.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006360 con 0x7f077c0067d0 2024-01-24T23:46:24.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:24.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006bb0 con 0x7f077c00f490 2024-01-24T23:46:24.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:24.004+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:46:24.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:24.004+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c0067d0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:24.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:24.004+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c00f8a0 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:24.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:24.004+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c00f490 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:46:24.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:24.208+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c0067d0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:24.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:24.208+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c00f8a0 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:24.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:24.208+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c00f490 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:46:24.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:24.612+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c0067d0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:24.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:24.612+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c00f8a0 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:24.612 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:24.612+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c00f490 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:46:25.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:25.416+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c0067d0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:25.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:25.416+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c00f8a0 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:25.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:25.416+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c00f490 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:46:26.394 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:46:26.396+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:46:27.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:27.004+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:46:27.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:27.004+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:46:27.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:27.004+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:46:27.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:27.004+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:46:27.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:27.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c00f490 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:27.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:27.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c0067d0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:27.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:27.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c00f8a0 legacy=0x7f077c005a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:27.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:27.004+0000 7f0798d99640 10 monclient: _add_conns ranks=[8,1,5] 2024-01-24T23:46:27.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:27.004+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-e con 0x7f077c014560 addr v1:172.21.15.115:6793/0 2024-01-24T23:46:27.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:27.004+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:46:27.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:27.004+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-f con 0x7f077c01bb70 addr v1:172.21.15.132:6789/0 2024-01-24T23:46:27.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:27.004+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:46:27.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:27.004+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-h con 0x7f077c01bf80 addr v1:172.21.15.132:6791/0 2024-01-24T23:46:27.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:27.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c014370 con 0x7f077c01bb70 2024-01-24T23:46:27.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:27.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006640 con 0x7f077c01bf80 2024-01-24T23:46:27.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:27.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006360 con 0x7f077c014560 2024-01-24T23:46:27.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:27.004+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:46:27.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:27.004+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c014560 legacy=0x7f077c01b460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:27.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:27.004+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c01bf80 legacy=0x7f077c005a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:27.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:27.208+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c014560 legacy=0x7f077c01b460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:27.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:27.208+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c01bf80 legacy=0x7f077c005a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:27.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:27.612+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c014560 legacy=0x7f077c01b460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:27.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:27.612+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c01bf80 legacy=0x7f077c005a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:28.416+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c014560 legacy=0x7f077c01b460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:28.416+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c01bf80 legacy=0x7f077c005a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:30.005 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:30.004+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:46:30.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:30.004+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:46:30.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:30.004+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:46:30.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:30.004+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:46:30.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:30.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c014560 legacy=0x7f077c01b460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:30.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:30.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c01bf80 legacy=0x7f077c005a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:30.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:30.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c01bb70 legacy=0x7f077c00a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:46:30.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:30.004+0000 7f0798d99640 10 monclient: _add_conns ranks=[4,0,2,7,5,6,3,8,1] 2024-01-24T23:46:30.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:30.004+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-c con 0x7f077c006e30 addr v1:172.21.15.115:6791/0 2024-01-24T23:46:30.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:30.004+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-a con 0x7f077c0067d0 addr v1:172.21.15.115:6789/0 2024-01-24T23:46:30.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:30.004+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-b con 0x7f077c015d50 addr v1:172.21.15.115:6790/0 2024-01-24T23:46:30.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:30.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006bb0 con 0x7f077c0067d0 2024-01-24T23:46:30.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:30.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006640 con 0x7f077c015d50 2024-01-24T23:46:30.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:30.004+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006360 con 0x7f077c006e30 2024-01-24T23:46:30.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:30.004+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:46:30.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:30.004+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c015d50 legacy=0x7f077c01eb40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:30.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:30.208+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c015d50 legacy=0x7f077c01eb40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:30.611 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:30.612+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c015d50 legacy=0x7f077c01eb40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:31.383 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:31.384+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c0067d0 legacy=0x7f077c01e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:46:31.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:31.384+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c0067d0 legacy=0x7f077c01e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:46:31.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:31.384+0000 7f079a59c640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c0067d0 0x7f077c01e430 :-1 s=OPENED pgs=514 cs=1 l=1).handle_message read tag failed 2024-01-24T23:46:31.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:31.384+0000 7f079a59c640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c0067d0 0x7f077c01e430 :-1 s=OPENED pgs=514 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:46:31.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:31.384+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:46:31.394 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:46:31.392+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:46:31.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:31.392+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c006e30 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:46:31.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:31.392+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c006e30 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:46:31.394 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:31.392+0000 7f079ad9d640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c006e30 0x7f077c00d440 :-1 s=OPENED pgs=517 cs=1 l=1).handle_message read tag failed 2024-01-24T23:46:31.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:31.392+0000 7f079ad9d640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c006e30 0x7f077c00d440 :-1 s=OPENED pgs=517 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:46:31.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:31.392+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:46:31.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:31.392+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:46:31.415 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:31.416+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c015d50 legacy=0x7f077c01eb40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:33.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:33.008+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:46:33.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:33.008+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:46:33.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:33.008+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:46:33.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:33.008+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:46:33.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:33.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c006e30 legacy=0x7f077c00d440 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:46:33.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:33.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c015d50 legacy=0x7f077c01eb40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:33.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:33.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c0067d0 legacy=0x7f077c01e430 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:46:33.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:33.008+0000 7f0798d99640 10 monclient: _add_conns ranks=[7,3,5,8,1,6] 2024-01-24T23:46:33.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:33.008+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-i con 0x7f077c0067d0 addr v1:172.21.15.132:6792/0 2024-01-24T23:46:33.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:33.008+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-g con 0x7f077c006e30 addr v1:172.21.15.132:6790/0 2024-01-24T23:46:33.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:33.008+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-h con 0x7f077c014500 addr v1:172.21.15.132:6791/0 2024-01-24T23:46:33.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:33.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c016130 con 0x7f077c006e30 2024-01-24T23:46:33.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:33.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006640 con 0x7f077c014500 2024-01-24T23:46:33.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:33.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c00f490 con 0x7f077c0067d0 2024-01-24T23:46:33.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:33.008+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:46:33.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:33.008+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c0067d0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:46:33.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:33.008+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c014500 legacy=0x7f077c005a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:33.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:33.008+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c006e30 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:33.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:33.212+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c0067d0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:46:33.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:33.212+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c006e30 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:33.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:33.212+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c014500 legacy=0x7f077c005a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:33.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:33.616+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c006e30 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:33.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:33.616+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c014500 legacy=0x7f077c005a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:33.616+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c0067d0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:46:34.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:34.420+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c0067d0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:46:34.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:34.420+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c014500 legacy=0x7f077c005a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:34.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:34.420+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c006e30 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:36.006 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.008+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:46:36.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.008+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:46:36.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.008+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:46:36.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.008+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:46:36.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c0067d0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:36.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c014500 legacy=0x7f077c005a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:36.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c006e30 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:36.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.008+0000 7f0798d99640 10 monclient: _add_conns ranks=[6,1,8] 2024-01-24T23:46:36.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.008+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-d con 0x7f077c00f810 addr v1:172.21.15.115:6792/0 2024-01-24T23:46:36.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.008+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-f con 0x7f077c01b460 addr v1:172.21.15.132:6789/0 2024-01-24T23:46:36.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.008+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-e con 0x7f077c00a6c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:46:36.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c00f620 con 0x7f077c01b460 2024-01-24T23:46:36.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c016130 con 0x7f077c00f810 2024-01-24T23:46:36.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006640 con 0x7f077c00a6c0 2024-01-24T23:46:36.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.008+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:46:36.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.008+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c00a6c0 legacy=0x7f077c021860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:36.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.008+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c00f810 legacy=0x7f077c01e3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:36.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.212+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c00a6c0 legacy=0x7f077c021860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:36.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.212+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c00f810 legacy=0x7f077c01e3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:36.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.392+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c01b460 legacy=0x7f077c01b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:46:36.390 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.392+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c01b460 legacy=0x7f077c01b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:46:36.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.392+0000 7f079a59c640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c01b460 0x7f077c01b8d0 :-1 s=OPENED pgs=512 cs=1 l=1).handle_message read tag failed 2024-01-24T23:46:36.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.392+0000 7f079a59c640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c01b460 0x7f077c01b8d0 :-1 s=OPENED pgs=512 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:46:36.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.392+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:46:36.391 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.392+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:46:36.395 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:46:36.392+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:46:36.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.616+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c00a6c0 legacy=0x7f077c021860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:36.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:36.616+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c00f810 legacy=0x7f077c01e3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:37.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:37.420+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c00a6c0 legacy=0x7f077c021860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:37.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:37.420+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c00f810 legacy=0x7f077c01e3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:39.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:39.008+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:46:39.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:39.008+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:46:39.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:39.008+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:46:39.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:39.008+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:46:39.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:39.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c00a6c0 legacy=0x7f077c021860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:39.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:39.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c00f810 legacy=0x7f077c01e3e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:39.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:39.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c01b460 legacy=0x7f077c01b8d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:46:39.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:39.008+0000 7f0798d99640 10 monclient: _add_conns ranks=[6,7,2,1,5,3,4,0,8] 2024-01-24T23:46:39.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:39.008+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-d con 0x7f077c01b460 addr v1:172.21.15.115:6792/0 2024-01-24T23:46:39.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:39.008+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-i con 0x7f077c0067d0 addr v1:172.21.15.132:6792/0 2024-01-24T23:46:39.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:39.008+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-b con 0x7f077c006de0 addr v1:172.21.15.115:6790/0 2024-01-24T23:46:39.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:39.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c00f490 con 0x7f077c006de0 2024-01-24T23:46:39.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:39.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c016130 con 0x7f077c01b460 2024-01-24T23:46:39.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:39.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006640 con 0x7f077c0067d0 2024-01-24T23:46:39.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:39.008+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:46:39.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:39.008+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c01b460 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:39.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:39.008+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c006de0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:39.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:39.008+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c0067d0 legacy=0x7f077c005a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:46:39.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:39.212+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c006de0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:39.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:39.212+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c01b460 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:39.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:39.212+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c0067d0 legacy=0x7f077c005a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:46:39.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:39.616+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c006de0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:39.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:39.616+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c01b460 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:39.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:39.616+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c0067d0 legacy=0x7f077c005a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:46:40.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:40.420+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c006de0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:40.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:40.420+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c01b460 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:40.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:40.420+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c0067d0 legacy=0x7f077c005a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:46:41.394 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:46:41.396+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:46:42.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:42.008+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:46:42.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:42.008+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:46:42.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:42.008+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:46:42.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:42.008+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:46:42.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:42.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c0067d0 legacy=0x7f077c005a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:42.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:42.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c01b460 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:42.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:42.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c006de0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:42.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:42.008+0000 7f0798d99640 10 monclient: _add_conns ranks=[1,0,3,5,8,4] 2024-01-24T23:46:42.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:42.008+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-f con 0x7f077c015cf0 addr v1:172.21.15.132:6789/0 2024-01-24T23:46:42.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:42.008+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-a con 0x7f077c014500 addr v1:172.21.15.115:6789/0 2024-01-24T23:46:42.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:42.008+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-g con 0x7f077c01dfb0 addr v1:172.21.15.132:6790/0 2024-01-24T23:46:42.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:42.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c00f620 con 0x7f077c014500 2024-01-24T23:46:42.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:42.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c00f490 con 0x7f077c015cf0 2024-01-24T23:46:42.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:42.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c016130 con 0x7f077c01dfb0 2024-01-24T23:46:42.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:42.008+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:46:42.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:42.008+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c01dfb0 legacy=0x7f077c01e3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:42.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:42.008+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c014500 legacy=0x7f077c01d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-01-24T23:46:42.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:42.008+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c014500 legacy=0x7f077c01d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-24T23:46:42.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:42.008+0000 7f079a59c640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c014500 0x7f077c01d8a0 :-1 s=CONNECTING_SEND_CONNECT_MSG pgs=0 cs=0 l=0).handle_connect_reply_1 read connect reply failed 2024-01-24T23:46:42.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:42.212+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c01dfb0 legacy=0x7f077c01e3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:42.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:42.616+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c01dfb0 legacy=0x7f077c01e3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:43.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:43.420+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c01dfb0 legacy=0x7f077c01e3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:45.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:45.008+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:46:45.007 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:45.008+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:46:45.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:45.008+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:46:45.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:45.008+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:46:45.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:45.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c01dfb0 legacy=0x7f077c01e3c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:45.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:45.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c015cf0 legacy=0x7f077c01eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:46:45.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:45.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c014500 legacy=0x7f077c01d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:46:45.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:45.008+0000 7f0798d99640 10 monclient: _add_conns ranks=[4,5,8] 2024-01-24T23:46:45.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:45.008+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-c con 0x7f077c022f80 addr v1:172.21.15.115:6791/0 2024-01-24T23:46:45.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:45.008+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:46:45.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:45.008+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-h con 0x7f077c026a90 addr v1:172.21.15.132:6791/0 2024-01-24T23:46:45.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:45.008+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-e con 0x7f077c00a6c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:46:45.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:45.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006640 con 0x7f077c022f80 2024-01-24T23:46:45.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:45.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c016130 con 0x7f077c026a90 2024-01-24T23:46:45.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:45.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c01b460 con 0x7f077c00a6c0 2024-01-24T23:46:45.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:45.008+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:46:45.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:45.008+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:46:45.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:45.008+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c00a6c0 legacy=0x7f077c021860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:45.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:45.008+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c026a90 legacy=0x7f077c028e80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:45.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:45.212+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c00a6c0 legacy=0x7f077c021860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:45.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:45.212+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c026a90 legacy=0x7f077c028e80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:45.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:45.616+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c00a6c0 legacy=0x7f077c021860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:45.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:45.616+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c026a90 legacy=0x7f077c028e80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:46.394 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:46:46.396+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:46:46.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:46.396+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c022f80 legacy=0x7f077c025370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:46:46.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:46.396+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c022f80 legacy=0x7f077c025370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:46:46.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:46.396+0000 7f079ad9d640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c022f80 0x7f077c025370 :-1 s=OPENED pgs=525 cs=1 l=1).handle_message read tag failed 2024-01-24T23:46:46.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:46.396+0000 7f079ad9d640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c022f80 0x7f077c025370 :-1 s=OPENED pgs=525 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:46:46.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:46.396+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:46:46.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:46.420+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c00a6c0 legacy=0x7f077c021860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:46.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:46.420+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c026a90 legacy=0x7f077c028e80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:48.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:48.008+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:46:48.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:48.008+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:46:48.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:48.008+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:46:48.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:48.008+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:46:48.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:48.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c00a6c0 legacy=0x7f077c021860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:48.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:48.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c026a90 legacy=0x7f077c028e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:48.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:48.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c022f80 legacy=0x7f077c025370 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:46:48.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:48.008+0000 7f0798d99640 10 monclient: _add_conns ranks=[3,2,8,0,6,7,4,1,5] 2024-01-24T23:46:48.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:48.008+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-g con 0x7f077c014500 addr v1:172.21.15.132:6790/0 2024-01-24T23:46:48.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:48.008+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-b con 0x7f077c006de0 addr v1:172.21.15.115:6790/0 2024-01-24T23:46:48.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:48.008+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-e con 0x7f077c00a6c0 addr v1:172.21.15.115:6793/0 2024-01-24T23:46:48.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:48.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c01b5f0 con 0x7f077c006de0 2024-01-24T23:46:48.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:48.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c016130 con 0x7f077c014500 2024-01-24T23:46:48.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:48.008+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c01b460 con 0x7f077c00a6c0 2024-01-24T23:46:48.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:48.008+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:46:48.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:48.008+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c00a6c0 legacy=0x7f077c01eb20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:48.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:48.008+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c006de0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:48.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:48.008+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c014500 legacy=0x7f077c01e3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:48.211 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:48.212+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c00a6c0 legacy=0x7f077c01eb20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:48.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:48.212+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c006de0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:48.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:48.212+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c014500 legacy=0x7f077c01e3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:48.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:48.616+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c006de0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:48.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:48.616+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c00a6c0 legacy=0x7f077c01eb20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:48.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:48.616+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c014500 legacy=0x7f077c01e3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:49.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:49.420+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c00a6c0 legacy=0x7f077c01eb20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:46:49.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:49.420+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c006de0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:46:49.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:49.420+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c014500 legacy=0x7f077c01e3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:51.008 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.007+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:46:51.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.007+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:46:51.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.007+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:46:51.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.007+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:46:51.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.007+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c00a6c0 legacy=0x7f077c01eb20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:51.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.007+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c014500 legacy=0x7f077c01e3c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:51.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.007+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c006de0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:51.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.007+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:46:51.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.007+0000 7f0798d99640 10 monclient: _add_conns ranks=[5,4,6,1,7,0] 2024-01-24T23:46:51.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.007+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-h con 0x7f077c014500 addr v1:172.21.15.132:6791/0 2024-01-24T23:46:51.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.007+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-c con 0x7f077c006d40 addr v1:172.21.15.115:6791/0 2024-01-24T23:46:51.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.007+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-d con 0x7f077c0065d0 addr v1:172.21.15.115:6792/0 2024-01-24T23:46:51.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.007+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c00f640 con 0x7f077c006d40 2024-01-24T23:46:51.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.007+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c01b5f0 con 0x7f077c014500 2024-01-24T23:46:51.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.007+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c016130 con 0x7f077c0065d0 2024-01-24T23:46:51.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.007+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:46:51.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.007+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c0065d0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:51.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.007+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c014500 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:51.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.007+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c006d40 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send send error: (32) Broken pipe 2024-01-24T23:46:51.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.211+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c0065d0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:51.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.211+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c014500 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:51.394 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:46:51.395+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:46:51.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.395+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c006d40 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:46:51.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.395+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c006d40 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:46:51.395 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.395+0000 7f079a59c640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c006d40 0x7f077c005a20 :-1 s=OPENED pgs=530 cs=1 l=1).handle_message read tag failed 2024-01-24T23:46:51.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.395+0000 7f079a59c640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c006d40 0x7f077c005a20 :-1 s=OPENED pgs=530 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:46:51.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.395+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:46:51.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.615+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c0065d0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:51.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:51.615+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c014500 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:52.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:52.419+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c0065d0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:52.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:52.419+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c014500 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:46:54.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:54.007+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:46:54.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:54.007+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:46:54.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:54.007+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:46:54.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:54.007+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:46:54.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:54.007+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c0065d0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:54.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:54.007+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c014500 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:54.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:54.007+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c006d40 legacy=0x7f077c005a20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:46:54.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:54.007+0000 7f0798d99640 10 monclient: _add_conns ranks=[1,7,0] 2024-01-24T23:46:54.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:54.007+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-f con 0x7f077c0086b0 addr v1:172.21.15.132:6789/0 2024-01-24T23:46:54.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:54.007+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-i con 0x7f077c01cf90 addr v1:172.21.15.132:6792/0 2024-01-24T23:46:54.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:54.007+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-a con 0x7f077c01eac0 addr v1:172.21.15.115:6789/0 2024-01-24T23:46:54.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:54.007+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c01b460 con 0x7f077c01eac0 2024-01-24T23:46:54.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:54.007+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c01b5f0 con 0x7f077c0086b0 2024-01-24T23:46:54.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:54.007+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c016130 con 0x7f077c01cf90 2024-01-24T23:46:54.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:54.007+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:46:54.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:54.007+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c01cf90 legacy=0x7f077c01d3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:46:54.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:54.007+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c0086b0 legacy=0x7f077c01b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:46:54.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:54.007+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c0086b0 legacy=0x7f077c01b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:46:54.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:54.007+0000 7f079ad9d640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c0086b0 0x7f077c01b870 :-1 s=OPENED pgs=531 cs=1 l=1).handle_message read tag failed 2024-01-24T23:46:54.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:54.007+0000 7f079ad9d640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c0086b0 0x7f077c01b870 :-1 s=OPENED pgs=531 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:46:54.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:54.007+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:46:54.209 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:54.211+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c01cf90 legacy=0x7f077c01d3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:46:54.615 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:54.615+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c01cf90 legacy=0x7f077c01d3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:46:55.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:55.419+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c01cf90 legacy=0x7f077c01d3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:46:56.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:56.383+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c01eac0 legacy=0x7f077c01a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:46:56.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:56.383+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c01eac0 legacy=0x7f077c01a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:46:56.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:56.383+0000 7f079b59e640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c01eac0 0x7f077c01a2e0 :-1 s=OPENED pgs=535 cs=1 l=1).handle_message read tag failed 2024-01-24T23:46:56.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:56.383+0000 7f079b59e640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c01eac0 0x7f077c01a2e0 :-1 s=OPENED pgs=535 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:46:56.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:56.383+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:46:56.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:56.383+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:46:56.395 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:46:56.391+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:46:57.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:57.011+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:46:57.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:57.011+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:46:57.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:57.011+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:46:57.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:57.011+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:46:57.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:57.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c01cf90 legacy=0x7f077c01d3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:46:57.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:57.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c0086b0 legacy=0x7f077c01b870 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:46:57.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:57.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c01eac0 legacy=0x7f077c01a2e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:46:57.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:57.011+0000 7f0798d99640 10 monclient: _add_conns ranks=[6,3,0,8,4,7,1,2,5] 2024-01-24T23:46:57.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:57.011+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-d con 0x7f077c01eac0 addr v1:172.21.15.115:6792/0 2024-01-24T23:46:57.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:57.011+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-g con 0x7f077c0086b0 addr v1:172.21.15.132:6790/0 2024-01-24T23:46:57.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:57.011+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-a con 0x7f077c006d40 addr v1:172.21.15.115:6789/0 2024-01-24T23:46:57.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:57.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c00f640 con 0x7f077c006d40 2024-01-24T23:46:57.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:57.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c016130 con 0x7f077c0086b0 2024-01-24T23:46:57.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:57.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c00aaa0 con 0x7f077c01eac0 2024-01-24T23:46:57.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:57.011+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:46:57.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:57.011+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c01eac0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:57.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:57.011+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c0086b0 legacy=0x7f077c00dce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:57.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:57.215+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c01eac0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:57.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:57.215+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c0086b0 legacy=0x7f077c00dce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:57.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:57.619+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c01eac0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:57.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:57.619+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c0086b0 legacy=0x7f077c00dce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:46:58.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:58.423+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c01eac0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:46:58.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:46:58.423+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c0086b0 legacy=0x7f077c00dce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:47:00.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:00.007+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:47:00.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:00.007+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:47:00.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:00.007+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:47:00.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:00.007+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:47:00.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:00.007+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c01eac0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:47:00.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:00.007+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c0086b0 legacy=0x7f077c00dce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:47:00.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:00.007+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c006d40 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:47:00.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:00.007+0000 7f0798d99640 10 monclient: _add_conns ranks=[7,5,8,2,4,1] 2024-01-24T23:47:00.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:00.007+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-i con 0x7f077c01cf90 addr v1:172.21.15.132:6792/0 2024-01-24T23:47:00.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:00.007+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-h con 0x7f077c014500 addr v1:172.21.15.132:6791/0 2024-01-24T23:47:00.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:00.007+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-e con 0x7f077c0065d0 addr v1:172.21.15.115:6793/0 2024-01-24T23:47:00.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:00.007+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c01b5f0 con 0x7f077c014500 2024-01-24T23:47:00.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:00.007+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c016130 con 0x7f077c01cf90 2024-01-24T23:47:00.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:00.007+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c00aaa0 con 0x7f077c0065d0 2024-01-24T23:47:00.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:00.007+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:47:00.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:00.007+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c01cf90 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:47:00.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:00.007+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c0065d0 legacy=0x7f077c01c820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:47:00.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:00.007+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c014500 legacy=0x7f077c01e3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:47:00.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:00.211+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c0065d0 legacy=0x7f077c01c820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:47:00.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:00.211+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c01cf90 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:47:00.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:00.211+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c014500 legacy=0x7f077c01e3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:47:00.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:00.615+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c0065d0 legacy=0x7f077c01c820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:47:00.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:00.615+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c01cf90 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:47:00.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:00.615+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c014500 legacy=0x7f077c01e3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:47:01.395 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:47:01.395+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:47:01.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:01.419+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c0065d0 legacy=0x7f077c01c820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:47:01.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:01.419+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c014500 legacy=0x7f077c01e3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:47:01.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:01.419+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c01cf90 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:47:03.009 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:03.011+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:47:03.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:03.011+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:47:03.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:03.011+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:47:03.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:03.011+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:47:03.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:03.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c0065d0 legacy=0x7f077c01c820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:47:03.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:03.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c01cf90 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:47:03.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:03.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c014500 legacy=0x7f077c01e3b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:47:03.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:03.011+0000 7f0798d99640 10 monclient: _add_conns ranks=[2,1,4] 2024-01-24T23:47:03.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:03.011+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-b con 0x7f077c00e3f0 addr v1:172.21.15.115:6790/0 2024-01-24T23:47:03.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:03.011+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:47:03.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:03.011+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-f con 0x7f077c00e7d0 addr v1:172.21.15.132:6789/0 2024-01-24T23:47:03.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:03.011+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-c con 0x7f077c0065d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:47:03.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:03.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c00f640 con 0x7f077c00e7d0 2024-01-24T23:47:03.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:03.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c01b5f0 con 0x7f077c00e3f0 2024-01-24T23:47:03.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:03.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c016130 con 0x7f077c0065d0 2024-01-24T23:47:03.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:03.011+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c00e3f0 legacy=0x7f077c022870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:47:03.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:03.011+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:47:03.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:03.215+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c00e3f0 legacy=0x7f077c022870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:47:03.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:03.619+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c00e3f0 legacy=0x7f077c022870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:47:04.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:04.423+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c00e3f0 legacy=0x7f077c022870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:47:06.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.011+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:47:06.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.011+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:47:06.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.011+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:47:06.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.011+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:47:06.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c0065d0 legacy=0x7f077c00dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:47:06.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c00e3f0 legacy=0x7f077c022870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:47:06.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c00e7d0 legacy=0x7f077c025fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:47:06.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.011+0000 7f0798d99640 10 monclient: _add_conns ranks=[8,0,6,4,3,5,1,7,2] 2024-01-24T23:47:06.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.011+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-e con 0x7f077c014500 addr v1:172.21.15.115:6793/0 2024-01-24T23:47:06.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.011+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-a con 0x7f077c01cf90 addr v1:172.21.15.115:6789/0 2024-01-24T23:47:06.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.011+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-d con 0x7f077c006d40 addr v1:172.21.15.115:6792/0 2024-01-24T23:47:06.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c00aaa0 con 0x7f077c01cf90 2024-01-24T23:47:06.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c01b5f0 con 0x7f077c006d40 2024-01-24T23:47:06.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c00f640 con 0x7f077c014500 2024-01-24T23:47:06.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.011+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:47:06.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.011+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c014500 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:47:06.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.011+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c006d40 legacy=0x7f077c01c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:47:06.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.215+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c006d40 legacy=0x7f077c01c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:47:06.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.215+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c014500 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:47:06.384 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.387+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c01cf90 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:47:06.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.387+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c01cf90 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:47:06.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.387+0000 7f079a59c640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c01cf90 0x7f077c005a20 :-1 s=OPENED pgs=537 cs=1 l=1).handle_message read tag failed 2024-01-24T23:47:06.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.387+0000 7f079a59c640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c01cf90 0x7f077c005a20 :-1 s=OPENED pgs=537 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:47:06.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.387+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:47:06.395 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:47:06.391+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:47:06.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.619+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c006d40 legacy=0x7f077c01c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:47:06.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:06.619+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c014500 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:47:07.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:07.423+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c006d40 legacy=0x7f077c01c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:47:07.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:07.423+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c014500 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:47:09.010 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:09.011+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:47:09.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:09.011+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:47:09.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:09.011+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:47:09.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:09.011+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:47:09.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:09.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c014500 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:47:09.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:09.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c006d40 legacy=0x7f077c01c790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:47:09.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:09.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c01cf90 legacy=0x7f077c005a20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:47:09.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:09.011+0000 7f0798d99640 10 monclient: _add_conns ranks=[4,5,7,3,1,2] 2024-01-24T23:47:09.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:09.011+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-c con 0x7f077c0065d0 addr v1:172.21.15.115:6791/0 2024-01-24T23:47:09.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:09.011+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:47:09.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:09.011+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-h con 0x7f077c00a6c0 addr v1:172.21.15.132:6791/0 2024-01-24T23:47:09.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:09.011+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-i con 0x7f077c01e380 addr v1:172.21.15.132:6792/0 2024-01-24T23:47:09.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:09.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006a00 con 0x7f077c0065d0 2024-01-24T23:47:09.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:09.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c01b5f0 con 0x7f077c00a6c0 2024-01-24T23:47:09.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:09.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c00f640 con 0x7f077c01e380 2024-01-24T23:47:09.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:09.011+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:47:09.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:09.011+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:47:09.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:09.011+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c00a6c0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:47:09.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:09.011+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c01e380 legacy=0x7f077c01e790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:47:09.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:09.215+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c01e380 legacy=0x7f077c01e790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:47:09.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:09.215+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c00a6c0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:47:09.619 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:09.619+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c01e380 legacy=0x7f077c01e790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:47:09.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:09.619+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c00a6c0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:47:10.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:10.423+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c00a6c0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:47:10.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:10.423+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c01e380 legacy=0x7f077c01e790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:47:11.395 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:47:11.395+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:47:11.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:11.395+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c0065d0 legacy=0x7f077c00dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:47:11.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:11.395+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c0065d0 legacy=0x7f077c00dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:47:11.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:11.395+0000 7f079ad9d640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c0065d0 0x7f077c00dce0 :-1 s=OPENED pgs=544 cs=1 l=1).handle_message read tag failed 2024-01-24T23:47:11.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:11.395+0000 7f079ad9d640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c0065d0 0x7f077c00dce0 :-1 s=OPENED pgs=544 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:47:11.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:11.395+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:47:12.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:12.011+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:47:12.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:12.011+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:47:12.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:12.011+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:47:12.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:12.011+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:47:12.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:12.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c01e380 legacy=0x7f077c01e790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:47:12.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:12.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c00a6c0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:47:12.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:12.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c0065d0 legacy=0x7f077c00dce0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:47:12.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:12.011+0000 7f0798d99640 10 monclient: _add_conns ranks=[2,3,1] 2024-01-24T23:47:12.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:12.011+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-b con 0x7f077c01cf90 addr v1:172.21.15.115:6790/0 2024-01-24T23:47:12.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:12.011+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-g con 0x7f077c014500 addr v1:172.21.15.132:6790/0 2024-01-24T23:47:12.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:12.011+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-f con 0x7f077c006d40 addr v1:172.21.15.132:6789/0 2024-01-24T23:47:12.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:12.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c00add0 con 0x7f077c006d40 2024-01-24T23:47:12.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:12.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c01b5f0 con 0x7f077c01cf90 2024-01-24T23:47:12.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:12.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c00f640 con 0x7f077c014500 2024-01-24T23:47:12.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:12.011+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:47:12.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:12.011+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c01cf90 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:47:12.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:12.011+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c014500 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:47:12.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:12.215+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c01cf90 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:47:12.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:12.215+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c014500 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:47:12.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:12.619+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c01cf90 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:47:12.616 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:12.619+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c014500 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:47:13.423 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:13.423+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c01cf90 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:47:13.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:13.423+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c014500 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:47:15.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:15.011+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:47:15.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:15.011+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:47:15.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:15.011+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:47:15.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:15.011+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:47:15.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:15.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c014500 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:47:15.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:15.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c01cf90 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:47:15.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:15.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c006d40 legacy=0x7f077c01c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:47:15.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:15.011+0000 7f0798d99640 10 monclient: _add_conns ranks=[6,7,0,8,3,4,2,1,5] 2024-01-24T23:47:15.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:15.011+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-d con 0x7f077c00e3f0 addr v1:172.21.15.115:6792/0 2024-01-24T23:47:15.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:15.011+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:47:15.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:15.011+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-i con 0x7f077c01e380 addr v1:172.21.15.132:6792/0 2024-01-24T23:47:15.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:15.011+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:47:15.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:15.011+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-a con 0x7f077c0065d0 addr v1:172.21.15.115:6789/0 2024-01-24T23:47:15.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:15.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006a00 con 0x7f077c0065d0 2024-01-24T23:47:15.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:15.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c01b5f0 con 0x7f077c00e3f0 2024-01-24T23:47:15.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:15.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c00f640 con 0x7f077c01e380 2024-01-24T23:47:15.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:15.011+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:47:15.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:15.011+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c00e3f0 legacy=0x7f077c022870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:47:15.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:15.011+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c01e380 legacy=0x7f077c01e790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:47:15.215 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:15.215+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c00e3f0 legacy=0x7f077c022870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:47:15.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:15.215+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c01e380 legacy=0x7f077c01e790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:47:15.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:15.619+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c00e3f0 legacy=0x7f077c022870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:47:15.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:15.619+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c01e380 legacy=0x7f077c01e790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:47:16.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:16.387+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c0065d0 legacy=0x7f077c00dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:47:16.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:16.387+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c0065d0 legacy=0x7f077c00dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:47:16.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:16.387+0000 7f079b59e640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c0065d0 0x7f077c00dce0 :-1 s=OPENED pgs=544 cs=1 l=1).handle_message read tag failed 2024-01-24T23:47:16.385 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:16.387+0000 7f079b59e640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c0065d0 0x7f077c00dce0 :-1 s=OPENED pgs=544 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:47:16.386 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:16.387+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6789/0 2024-01-24T23:47:16.395 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:47:16.395+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:47:16.420 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:16.423+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c00e3f0 legacy=0x7f077c022870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:47:16.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:16.423+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c01e380 legacy=0x7f077c01e790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:47:18.011 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:18.011+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:47:18.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:18.011+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:47:18.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:18.011+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:47:18.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:18.011+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:47:18.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:18.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c01e380 legacy=0x7f077c01e790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:47:18.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:18.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c00e3f0 legacy=0x7f077c022870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:47:18.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:18.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c0065d0 legacy=0x7f077c00dce0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:47:18.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:18.011+0000 7f0798d99640 10 monclient: _add_conns ranks=[1,8,5,4,3,2] 2024-01-24T23:47:18.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:18.011+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-f con 0x7f077c00a6c0 addr v1:172.21.15.132:6789/0 2024-01-24T23:47:18.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:18.011+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-e con 0x7f077c006d40 addr v1:172.21.15.115:6793/0 2024-01-24T23:47:18.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:18.011+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-h con 0x7f077c01dfa0 addr v1:172.21.15.132:6791/0 2024-01-24T23:47:18.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:18.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c00add0 con 0x7f077c00a6c0 2024-01-24T23:47:18.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:18.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c01b5f0 con 0x7f077c01dfa0 2024-01-24T23:47:18.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:18.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c00f640 con 0x7f077c006d40 2024-01-24T23:47:18.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:18.011+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:47:18.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:18.011+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c006d40 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:47:18.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:18.011+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c01dfa0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:47:18.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:18.215+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c006d40 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:47:18.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:18.215+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c01dfa0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:47:18.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:18.619+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c006d40 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:47:18.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:18.619+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c01dfa0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:47:19.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:19.423+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c006d40 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:47:19.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:19.423+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c01dfa0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:47:21.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.011+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:47:21.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.011+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:47:21.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.011+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:47:21.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.011+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:47:21.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c006d40 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:47:21.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c01dfa0 legacy=0x7f077c00d440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:47:21.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c00a6c0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:47:21.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.011+0000 7f0798d99640 10 monclient: _add_conns ranks=[4,3,2] 2024-01-24T23:47:21.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.011+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-c con 0x7f077c014500 addr v1:172.21.15.115:6791/0 2024-01-24T23:47:21.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.011+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:47:21.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.011+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-g con 0x7f077c00e7d0 addr v1:172.21.15.132:6790/0 2024-01-24T23:47:21.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.011+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:47:21.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.011+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-b con 0x7f077c0065d0 addr v1:172.21.15.115:6790/0 2024-01-24T23:47:21.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006a00 con 0x7f077c0065d0 2024-01-24T23:47:21.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c01b5f0 con 0x7f077c00e7d0 2024-01-24T23:47:21.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.011+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c00f640 con 0x7f077c014500 2024-01-24T23:47:21.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.011+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:47:21.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.011+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c00e7d0 legacy=0x7f077c025fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:47:21.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.011+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c0065d0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:47:21.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.215+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c0065d0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:47:21.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.215+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c00e7d0 legacy=0x7f077c025fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:47:21.396 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:47:21.395+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:47:21.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.395+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c014500 legacy=0x7f077c01c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-01-24T23:47:21.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.395+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c014500 legacy=0x7f077c01c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:47:21.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.395+0000 7f079ad9d640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c014500 0x7f077c01c790 :-1 s=OPENED pgs=545 cs=1 l=1).handle_message read tag failed 2024-01-24T23:47:21.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.395+0000 7f079ad9d640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c014500 0x7f077c01c790 :-1 s=OPENED pgs=545 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:47:21.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.395+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:47:21.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.619+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c0065d0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:47:21.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:21.619+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c00e7d0 legacy=0x7f077c025fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:47:22.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:22.423+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c0065d0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:47:22.421 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:22.423+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c00e7d0 legacy=0x7f077c025fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:47:24.012 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:24.014+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:47:24.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:24.014+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:47:24.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:24.014+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:47:24.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:24.014+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:47:24.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:24.014+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c014500 legacy=0x7f077c01c790 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:47:24.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:24.014+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c00e7d0 legacy=0x7f077c025fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:47:24.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:24.014+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c0065d0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:47:24.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:24.014+0000 7f0798d99640 10 monclient: _add_conns ranks=[6,1,4,7,2,3,0,5,8] 2024-01-24T23:47:24.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:24.014+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-d con 0x7f077c006d40 addr v1:172.21.15.115:6792/0 2024-01-24T23:47:24.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:24.014+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-f con 0x7f077c00e3d0 addr v1:172.21.15.132:6789/0 2024-01-24T23:47:24.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:24.014+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-c con 0x7f077c014500 addr v1:172.21.15.115:6791/0 2024-01-24T23:47:24.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:24.014+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c01efb0 con 0x7f077c00e3d0 2024-01-24T23:47:24.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:24.014+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006a00 con 0x7f077c014500 2024-01-24T23:47:24.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:24.014+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c01b5f0 con 0x7f077c006d40 2024-01-24T23:47:24.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:24.014+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:47:24.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:24.014+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c006d40 legacy=0x7f077c00dcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:47:24.219 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:24.218+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c006d40 legacy=0x7f077c00dcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:47:24.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:24.622+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c006d40 legacy=0x7f077c00dcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:47:25.428 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:25.426+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c006d40 legacy=0x7f077c00dcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6792/0 2024-01-24T23:47:26.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:26.394+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c00e3d0 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-24T23:47:26.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:26.394+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c00e3d0 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:47:26.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:26.394+0000 7f079a59c640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c00e3d0 0x7f077c005a20 :-1 s=OPENED pgs=547 cs=1 l=1).handle_message read tag failed 2024-01-24T23:47:26.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:26.394+0000 7f079a59c640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c00e3d0 0x7f077c005a20 :-1 s=OPENED pgs=547 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:47:26.393 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:26.394+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.132:6789/0 2024-01-24T23:47:26.396 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:47:26.398+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:47:26.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:26.398+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c014500 legacy=0x7f077c01c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-01-24T23:47:26.396 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:26.398+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c014500 legacy=0x7f077c01c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-24T23:47:26.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:26.398+0000 7f079b59e640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c014500 0x7f077c01c790 :-1 s=OPENED pgs=552 cs=1 l=1).handle_message read tag failed 2024-01-24T23:47:26.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:26.398+0000 7f079b59e640 1 --1- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c014500 0x7f077c01c790 :-1 s=OPENED pgs=552 cs=1 l=1).fault on lossy channel, failing 2024-01-24T23:47:26.397 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:26.398+0000 7f0799d9b640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.115:6791/0 2024-01-24T23:47:27.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:27.014+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:47:27.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:27.014+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:47:27.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:27.014+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:47:27.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:27.014+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:47:27.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:27.014+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6792/0 conn(0x7f077c006d40 legacy=0x7f077c00dcc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:47:27.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:27.014+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6791/0 conn(0x7f077c014500 legacy=0x7f077c01c790 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:47:27.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:27.014+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6789/0 conn(0x7f077c00e3d0 legacy=0x7f077c005a20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-01-24T23:47:27.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:27.014+0000 7f0798d99640 10 monclient: _add_conns ranks=[0,3,2,8,7,5] 2024-01-24T23:47:27.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:27.014+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-a con 0x7f077c00c440 addr v1:172.21.15.115:6789/0 2024-01-24T23:47:27.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:27.014+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:47:27.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:27.014+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-g con 0x7f077c00e7d0 addr v1:172.21.15.132:6790/0 2024-01-24T23:47:27.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:27.014+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-b con 0x7f077c0065d0 addr v1:172.21.15.115:6790/0 2024-01-24T23:47:27.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:27.014+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c00f640 con 0x7f077c00c440 2024-01-24T23:47:27.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:27.014+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c01b5f0 con 0x7f077c0065d0 2024-01-24T23:47:27.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:27.014+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006a00 con 0x7f077c00e7d0 2024-01-24T23:47:27.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:27.014+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:47:27.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:27.014+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c0065d0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:47:27.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:27.014+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c00e7d0 legacy=0x7f077c00cf30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:47:27.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:27.218+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c0065d0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:47:27.220 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:27.218+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c00e7d0 legacy=0x7f077c00cf30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:47:27.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:27.622+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c0065d0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:47:27.621 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:27.622+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c00e7d0 legacy=0x7f077c00cf30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:47:28.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:28.426+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c0065d0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6790/0 2024-01-24T23:47:28.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:28.426+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c00e7d0 legacy=0x7f077c00cf30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:47:30.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:30.014+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:47:30.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:30.014+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:47:30.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:30.014+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:47:30.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:30.014+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:47:30.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:30.014+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c00e7d0 legacy=0x7f077c00cf30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:47:30.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:30.014+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6790/0 conn(0x7f077c0065d0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:47:30.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:30.014+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6789/0 conn(0x7f077c00c440 legacy=0x7f077c00c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-24T23:47:30.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:30.014+0000 7f0798d99640 10 monclient: _add_conns ranks=[5,8,7] 2024-01-24T23:47:30.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:30.014+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-h con 0x7f077c014500 addr v1:172.21.15.132:6791/0 2024-01-24T23:47:30.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:30.014+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-e con 0x7f077c006d40 addr v1:172.21.15.115:6793/0 2024-01-24T23:47:30.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:30.014+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-i con 0x7f077c00e3d0 addr v1:172.21.15.132:6792/0 2024-01-24T23:47:30.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:30.014+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c00add0 con 0x7f077c014500 2024-01-24T23:47:30.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:30.014+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c01b5f0 con 0x7f077c00e3d0 2024-01-24T23:47:30.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:30.014+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.115:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c006a00 con 0x7f077c006d40 2024-01-24T23:47:30.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:30.014+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:47:30.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:30.014+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c006d40 legacy=0x7f077c00dcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:47:30.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:30.014+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c014500 legacy=0x7f077c01c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:47:30.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:30.014+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c00e3d0 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:47:30.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:30.218+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c006d40 legacy=0x7f077c00dcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:47:30.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:30.218+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c014500 legacy=0x7f077c01c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:47:30.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:30.218+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c00e3d0 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:47:30.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:30.622+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c006d40 legacy=0x7f077c00dcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:47:30.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:30.622+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c014500 legacy=0x7f077c01c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:47:30.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:30.622+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c00e3d0 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:47:31.396 INFO:tasks.ceph.mon.c.smithi115.stderr:2024-01-24T23:47:31.398+0000 7fb853138640 -1 mon.c@4(probing) e1 get_health_metrics reporting 1 slow ops, oldest is auth(proto 0 28 bytes epoch 1) 2024-01-24T23:47:31.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:31.426+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c006d40 legacy=0x7f077c00dcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.115:6793/0 2024-01-24T23:47:31.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:31.426+0000 7f079ad9d640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c014500 legacy=0x7f077c01c790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6791/0 2024-01-24T23:47:31.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:31.426+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c00e3d0 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:47:33.013 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:33.014+0000 7f0798d99640 10 monclient(hunting): tick 2024-01-24T23:47:33.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:33.014+0000 7f0798d99640 10 monclient(hunting): _check_auth_tickets 2024-01-24T23:47:33.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:33.014+0000 7f0798d99640 1 monclient(hunting): continuing hunt 2024-01-24T23:47:33.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:33.014+0000 7f0798d99640 10 monclient(hunting): _reopen_session rank -1 2024-01-24T23:47:33.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:33.014+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.115:6793/0 conn(0x7f077c006d40 legacy=0x7f077c00dcc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:47:33.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:33.014+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c00e3d0 legacy=0x7f077c005a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:47:33.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:33.014+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6791/0 conn(0x7f077c014500 legacy=0x7f077c01c790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-24T23:47:33.014 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:33.014+0000 7f0798d99640 10 monclient: _add_conns ranks=[1,3,7,2,6,0,4,8,5] 2024-01-24T23:47:33.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:33.014+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-f con 0x7f077c00d640 addr v1:172.21.15.132:6789/0 2024-01-24T23:47:33.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:33.014+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:47:33.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:33.014+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-g con 0x7f077c00e7d0 addr v1:172.21.15.132:6790/0 2024-01-24T23:47:33.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:33.014+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 reap_dead start 2024-01-24T23:47:33.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:33.014+0000 7f0798d99640 10 monclient(hunting): picked mon.noname-i con 0x7f077c0065d0 addr v1:172.21.15.132:6792/0 2024-01-24T23:47:33.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:33.014+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c015950 con 0x7f077c00d640 2024-01-24T23:47:33.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:33.014+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c00add0 con 0x7f077c00e7d0 2024-01-24T23:47:33.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:33.014+0000 7f0798d99640 1 -- 172.21.15.115:0/3438174064 --> v1:172.21.15.132:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c01b5f0 con 0x7f077c0065d0 2024-01-24T23:47:33.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:33.014+0000 7f0798d99640 10 monclient(hunting): _renew_subs 2024-01-24T23:47:33.015 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:33.014+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c00e7d0 legacy=0x7f077c00cf30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:47:33.016 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:33.014+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c0065d0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:47:33.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:33.218+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c0065d0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:47:33.216 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:33.218+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c00e7d0 legacy=0x7f077c00cf30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:47:33.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:33.622+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c00e7d0 legacy=0x7f077c00cf30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:47:33.620 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:33.622+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c0065d0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:47:34.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:34.426+0000 7f079b59e640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6792/0 conn(0x7f077c0065d0 legacy=0x7f077c00ed80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6792/0 2024-01-24T23:47:34.424 INFO:teuthology.orchestra.run.smithi115.stderr:2024-01-24T23:47:34.426+0000 7f079a59c640 1 -- 172.21.15.115:0/3438174064 >> v1:172.21.15.132:6790/0 conn(0x7f077c00e7d0 legacy=0x7f077c00cf30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.132:6790/0 2024-01-24T23:47:35.893 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-01-24T23:47:35.894 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/mon_thrash.py", line 420, in task manager.wait_for_mon_quorum_size(len(mons)) File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 3172, in wait_for_mon_quorum_size if len(self.get_mon_quorum()) == size: File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 3157, in get_mon_quorum out = self.raw_cluster_cmd('quorum_status') File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 1627, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 1618, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi115 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 2774, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 2606, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 2372, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 1627, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 1618, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi115 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2024-01-24T23:47:35.911 INFO:teuthology.misc:Shutting down mds daemons... 2024-01-24T23:47:35.911 INFO:teuthology.misc:Shutting down osd daemons... 2024-01-24T23:47:35.911 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-01-24T23:47:35.911 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:47:35.914 INFO:tasks.ceph.osd.0:Stopped 2024-01-24T23:47:35.915 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-01-24T23:47:35.915 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:47:35.915 INFO:tasks.ceph.osd.1:Stopped 2024-01-24T23:47:35.915 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-01-24T23:47:35.915 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:47:35.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T23:47:35.916 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/mon_thrash.py", line 420, in task manager.wait_for_mon_quorum_size(len(mons)) File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 3172, in wait_for_mon_quorum_size if len(self.get_mon_quorum()) == size: File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 3157, in get_mon_quorum out = self.raw_cluster_cmd('quorum_status') File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 1627, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 1618, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi115 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 2774, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 2606, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 2372, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 1627, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 1618, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi115 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi115 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2' 2024-01-24T23:47:35.932 INFO:tasks.ceph.osd.2:Stopped 2024-01-24T23:47:35.932 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-01-24T23:47:35.932 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:47:35.932 INFO:tasks.ceph.osd.3:Stopped 2024-01-24T23:47:35.932 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-01-24T23:47:35.932 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:47:35.933 INFO:tasks.ceph.osd.4:Stopped 2024-01-24T23:47:35.933 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-01-24T23:47:35.933 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:47:35.933 INFO:tasks.ceph.osd.5:Stopped 2024-01-24T23:47:35.933 INFO:teuthology.misc:Shutting down mgr daemons... 2024-01-24T23:47:35.933 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-01-24T23:47:35.933 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:47:35.934 INFO:tasks.ceph.mgr.x:Stopped 2024-01-24T23:47:35.934 INFO:teuthology.misc:Shutting down mon daemons... 2024-01-24T23:47:35.934 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-01-24T23:47:35.934 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:47:35.953 INFO:tasks.ceph.mon.a:Stopped 2024-01-24T23:47:35.953 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-01-24T23:47:35.953 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:47:35.953 INFO:tasks.ceph.mon.b:Stopped 2024-01-24T23:47:35.953 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-01-24T23:47:35.953 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:47:36.014 INFO:tasks.ceph.mon.c:Stopped 2024-01-24T23:47:36.015 DEBUG:tasks.ceph.mon.d:waiting for process to exit 2024-01-24T23:47:36.015 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:47:36.015 INFO:tasks.ceph.mon.d:Stopped 2024-01-24T23:47:36.015 DEBUG:tasks.ceph.mon.e:waiting for process to exit 2024-01-24T23:47:36.016 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:47:36.016 INFO:tasks.ceph.mon.e:Stopped 2024-01-24T23:47:36.016 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-01-24T23:47:36.016 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:47:36.033 INFO:tasks.ceph.mon.f:Stopped 2024-01-24T23:47:36.033 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-01-24T23:47:36.033 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:47:36.034 INFO:tasks.ceph.mon.g:Stopped 2024-01-24T23:47:36.034 DEBUG:tasks.ceph.mon.h:waiting for process to exit 2024-01-24T23:47:36.034 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:47:36.034 INFO:tasks.ceph.mon.h:Stopped 2024-01-24T23:47:36.034 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-01-24T23:47:36.034 INFO:teuthology.orchestra.run:waiting for 300 2024-01-24T23:47:36.035 INFO:tasks.ceph.mon.i:Stopped 2024-01-24T23:47:36.035 INFO:tasks.ceph:Checking cluster log for badness... 2024-01-24T23:47:36.035 DEBUG:teuthology.orchestra.run.smithi115:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v ScrubResult | egrep -v 'scrub mismatch' | egrep -v 'overall HEALTH_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(AUTH_BAD_CAPS\)' | head -n 1 2024-01-24T23:47:36.075 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi115.front.sepia.ceph.com 2024-01-24T23:47:36.075 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-01-24T23:47:36.284 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi115.front.sepia.ceph.com 2024-01-24T23:47:36.284 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-01-24T23:47:36.426 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi115.front.sepia.ceph.com 2024-01-24T23:47:36.427 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-01-24T23:47:36.534 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi132.front.sepia.ceph.com 2024-01-24T23:47:36.535 DEBUG:teuthology.orchestra.run.smithi132:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-01-24T23:47:36.663 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi132.front.sepia.ceph.com 2024-01-24T23:47:36.664 DEBUG:teuthology.orchestra.run.smithi132:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-01-24T23:47:36.845 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi132.front.sepia.ceph.com 2024-01-24T23:47:36.845 DEBUG:teuthology.orchestra.run.smithi132:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-01-24T23:47:37.003 INFO:tasks.ceph:Archiving mon data... 2024-01-24T23:47:37.004 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi/7530404/data/mon.a.tgz 2024-01-24T23:47:37.004 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-01-24T23:47:37.010 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.IpdlS6FPj2 2024-01-24T23:47:37.010 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.IpdlS6FPj2 2024-01-24T23:47:37.145 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.IpdlS6FPj2 2024-01-24T23:47:37.208 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.IpdlS6FPj2 is 436KB 2024-01-24T23:47:37.274 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.IpdlS6FPj2 2024-01-24T23:47:37.279 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi/7530404/data/mon.b.tgz 2024-01-24T23:47:37.280 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-01-24T23:47:37.328 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.KPyXifyTsc 2024-01-24T23:47:37.328 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.KPyXifyTsc 2024-01-24T23:47:37.452 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.KPyXifyTsc 2024-01-24T23:47:37.515 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.KPyXifyTsc is 373KB 2024-01-24T23:47:37.574 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.KPyXifyTsc 2024-01-24T23:47:37.578 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi/7530404/data/mon.c.tgz 2024-01-24T23:47:37.579 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-01-24T23:47:37.623 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.ksXjqHScTo 2024-01-24T23:47:37.624 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.ksXjqHScTo 2024-01-24T23:47:37.761 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.ksXjqHScTo 2024-01-24T23:47:37.824 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.ksXjqHScTo is 437KB 2024-01-24T23:47:37.891 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.ksXjqHScTo 2024-01-24T23:47:37.897 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/mon/ceph-d to /home/teuthworker/archive/pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi/7530404/data/mon.d.tgz 2024-01-24T23:47:37.898 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-01-24T23:47:37.944 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.VaW2nehJ8V 2024-01-24T23:47:37.945 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-d -- . > /tmp/tmp.VaW2nehJ8V 2024-01-24T23:47:38.070 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.VaW2nehJ8V 2024-01-24T23:47:38.131 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.VaW2nehJ8V is 384KB 2024-01-24T23:47:38.194 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.VaW2nehJ8V 2024-01-24T23:47:38.199 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/mon/ceph-e to /home/teuthworker/archive/pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi/7530404/data/mon.e.tgz 2024-01-24T23:47:38.199 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-01-24T23:47:38.248 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.S5fEmGxl5j 2024-01-24T23:47:38.251 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-e -- . > /tmp/tmp.S5fEmGxl5j 2024-01-24T23:47:38.375 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.S5fEmGxl5j 2024-01-24T23:47:38.444 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.S5fEmGxl5j is 382KB 2024-01-24T23:47:38.507 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.S5fEmGxl5j 2024-01-24T23:47:38.512 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/mon/ceph-f to /home/teuthworker/archive/pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi/7530404/data/mon.f.tgz 2024-01-24T23:47:38.512 DEBUG:teuthology.orchestra.run.smithi132:> mktemp 2024-01-24T23:47:38.518 INFO:teuthology.orchestra.run.smithi132.stdout:/tmp/tmp.gCRaaCvILZ 2024-01-24T23:47:38.519 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-f -- . > /tmp/tmp.gCRaaCvILZ 2024-01-24T23:47:38.649 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0666 /tmp/tmp.gCRaaCvILZ 2024-01-24T23:47:38.714 DEBUG:teuthology.orchestra.remote:smithi132:/tmp/tmp.gCRaaCvILZ is 437KB 2024-01-24T23:47:38.775 DEBUG:teuthology.orchestra.run.smithi132:> rm -fr /tmp/tmp.gCRaaCvILZ 2024-01-24T23:47:38.780 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/mon/ceph-g to /home/teuthworker/archive/pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi/7530404/data/mon.g.tgz 2024-01-24T23:47:38.781 DEBUG:teuthology.orchestra.run.smithi132:> mktemp 2024-01-24T23:47:38.826 INFO:teuthology.orchestra.run.smithi132.stdout:/tmp/tmp.1HkZ9fwmTe 2024-01-24T23:47:38.826 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-g -- . > /tmp/tmp.1HkZ9fwmTe 2024-01-24T23:47:38.953 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0666 /tmp/tmp.1HkZ9fwmTe 2024-01-24T23:47:39.018 DEBUG:teuthology.orchestra.remote:smithi132:/tmp/tmp.1HkZ9fwmTe is 384KB 2024-01-24T23:47:39.079 DEBUG:teuthology.orchestra.run.smithi132:> rm -fr /tmp/tmp.1HkZ9fwmTe 2024-01-24T23:47:39.085 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/mon/ceph-h to /home/teuthworker/archive/pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi/7530404/data/mon.h.tgz 2024-01-24T23:47:39.085 DEBUG:teuthology.orchestra.run.smithi132:> mktemp 2024-01-24T23:47:39.134 INFO:teuthology.orchestra.run.smithi132.stdout:/tmp/tmp.wny2REtUGy 2024-01-24T23:47:39.135 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-h -- . > /tmp/tmp.wny2REtUGy 2024-01-24T23:47:39.263 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0666 /tmp/tmp.wny2REtUGy 2024-01-24T23:47:39.327 DEBUG:teuthology.orchestra.remote:smithi132:/tmp/tmp.wny2REtUGy is 384KB 2024-01-24T23:47:39.388 DEBUG:teuthology.orchestra.run.smithi132:> rm -fr /tmp/tmp.wny2REtUGy 2024-01-24T23:47:39.394 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/mon/ceph-i to /home/teuthworker/archive/pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi/7530404/data/mon.i.tgz 2024-01-24T23:47:39.394 DEBUG:teuthology.orchestra.run.smithi132:> mktemp 2024-01-24T23:47:39.443 INFO:teuthology.orchestra.run.smithi132.stdout:/tmp/tmp.hJ7zvT5KbL 2024-01-24T23:47:39.444 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-i -- . > /tmp/tmp.hJ7zvT5KbL 2024-01-24T23:47:39.568 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0666 /tmp/tmp.hJ7zvT5KbL 2024-01-24T23:47:39.631 DEBUG:teuthology.orchestra.remote:smithi132:/tmp/tmp.hJ7zvT5KbL is 373KB 2024-01-24T23:47:39.647 DEBUG:teuthology.orchestra.run.smithi132:> rm -fr /tmp/tmp.hJ7zvT5KbL 2024-01-24T23:47:39.652 INFO:tasks.ceph:Cleaning ceph cluster... 2024-01-24T23:47:39.652 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-01-24T23:47:39.655 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-01-24T23:47:39.712 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-01-24T23:47:39.754 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-01-24T23:47:39.755 INFO:tasks.ceph:Archiving crash dumps... 2024-01-24T23:47:39.756 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/crash to /home/teuthworker/archive/pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi/7530404/remote/smithi115/crash 2024-01-24T23:47:39.757 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-24T23:47:39.804 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/crash to /home/teuthworker/archive/pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi/7530404/remote/smithi132/crash 2024-01-24T23:47:39.805 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-24T23:47:39.818 INFO:tasks.ceph:Compressing logs... 2024-01-24T23:47:39.818 DEBUG:teuthology.orchestra.run.smithi115:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-24T23:47:39.833 DEBUG:teuthology.orchestra.run.smithi132:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-24T23:47:39.885 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28880.log 2024-01-24T23:47:39.885 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27831.log 2024-01-24T23:47:39.886 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23742.log 2024-01-24T23:47:39.886 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27831.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.28880.log.gz 2024-01-24T23:47:39.886 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33843.log 2024-01-24T23:47:39.893 INFO:teuthology.orchestra.run.smithi132.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27831.log.gz 2024-01-24T23:47:39.893 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26728.log 2024-01-24T23:47:39.893 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.23742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.bazar.33994.log 2024-01-24T23:47:39.893 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33843.log: /var/log/ceph/ceph-client.admin.26728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28029.log 2024-01-24T23:47:39.893 INFO:teuthology.orchestra.run.smithi132.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.23742.log.gz 2024-01-24T23:47:39.893 INFO:teuthology.orchestra.run.smithi132.stderr: 87.3%/var/log/ceph/ceph-client.bazar.33994.log: -- replaced with /var/log/ceph/ceph-client.admin.33843.log.gz 2024-01-24T23:47:39.894 INFO:teuthology.orchestra.run.smithi132.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.26728.log.gz 2024-01-24T23:47:39.894 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32227.log 2024-01-24T23:47:39.894 INFO:teuthology.orchestra.run.smithi132.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.bazar.33994.log.gz 2024-01-24T23:47:39.894 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.34293.log 2024-01-24T23:47:39.894 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33595.log 2024-01-24T23:47:39.894 INFO:teuthology.orchestra.run.smithi132.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.28029.log.gz 2024-01-24T23:47:39.894 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.foo.34293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31143.log 2024-01-24T23:47:39.894 INFO:teuthology.orchestra.run.smithi132.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32227.log.gz 2024-01-24T23:47:39.894 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34005.log 2024-01-24T23:47:39.895 INFO:teuthology.orchestra.run.smithi132.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33595.log.gz 2024-01-24T23:47:39.895 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27100.log 2024-01-24T23:47:39.895 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31698.log 2024-01-24T23:47:39.895 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.34005.log: /var/log/ceph/ceph-client.admin.27100.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.31143.log.gz 2024-01-24T23:47:39.895 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32467.log 2024-01-24T23:47:39.895 INFO:teuthology.orchestra.run.smithi132.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34005.log.gz 2024-01-24T23:47:39.895 INFO:teuthology.orchestra.run.smithi132.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27100.log.gz 2024-01-24T23:47:39.895 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28298.log 2024-01-24T23:47:39.896 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31311.log 2024-01-24T23:47:39.896 INFO:teuthology.orchestra.run.smithi132.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.31698.log.gz 2024-01-24T23:47:39.896 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27807.log 2024-01-24T23:47:39.896 INFO:teuthology.orchestra.run.smithi132.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32467.log.gz 2024-01-24T23:47:39.896 INFO:teuthology.orchestra.run.smithi132.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.28298.log.gz 2024-01-24T23:47:39.896 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32659.log 2024-01-24T23:47:39.896 INFO:teuthology.orchestra.run.smithi132.stderr: 94.8% -- replaced with /var/log/ceph/ceph-client.foo.34293.log.gz 2024-01-24T23:47:39.896 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31746.log 2024-01-24T23:47:39.897 INFO:teuthology.orchestra.run.smithi132.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.31311.log.gz 2024-01-24T23:47:39.897 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27308.log 2024-01-24T23:47:39.897 INFO:teuthology.orchestra.run.smithi132.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27807.log.gz 2024-01-24T23:47:39.897 INFO:teuthology.orchestra.run.smithi132.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.32659.log.gz 2024-01-24T23:47:39.897 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28684.log 2024-01-24T23:47:39.897 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31746.log: /var/log/ceph/ceph-client.admin.27308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27002.log 2024-01-24T23:47:40.006 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27199.log 2024-01-24T23:47:40.006 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28885.log 2024-01-24T23:47:40.006 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26345.log 2024-01-24T23:47:40.006 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28885.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.27199.log.gz 2024-01-24T23:47:40.006 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29302.log 2024-01-24T23:47:40.006 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26345.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.28885.log.gz 2024-01-24T23:47:40.007 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28630.log 2024-01-24T23:47:40.007 INFO:teuthology.orchestra.run.smithi115.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.26345.log.gz 2024-01-24T23:47:40.007 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29612.log 2024-01-24T23:47:40.007 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28630.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29302.log.gz 2024-01-24T23:47:40.007 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28911.log 2024-01-24T23:47:40.007 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29612.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.28630.log.gz 2024-01-24T23:47:40.007 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27277.log 2024-01-24T23:47:40.007 INFO:teuthology.orchestra.run.smithi115.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.29612.log.gz 2024-01-24T23:47:40.008 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31052.log 2024-01-24T23:47:40.008 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28911.log: /var/log/ceph/ceph-client.admin.27277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28080.log 2024-01-24T23:47:40.008 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.31052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27667.log 2024-01-24T23:47:40.008 INFO:teuthology.orchestra.run.smithi115.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.27277.log.gz 2024-01-24T23:47:40.008 INFO:teuthology.orchestra.run.smithi115.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.28911.log.gz 2024-01-24T23:47:40.008 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28080.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.31052.log.gz 2024-01-24T23:47:40.008 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27303.log 2024-01-24T23:47:40.008 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30558.log 2024-01-24T23:47:40.009 INFO:teuthology.orchestra.run.smithi115.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.28080.log.gz 2024-01-24T23:47:40.009 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29383.log 2024-01-24T23:47:40.009 INFO:teuthology.orchestra.run.smithi115.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.27667.log.gz 2024-01-24T23:47:40.009 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30766.log 2024-01-24T23:47:40.009 INFO:teuthology.orchestra.run.smithi115.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.27303.log.gz 2024-01-24T23:47:40.009 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29383.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-01-24T23:47:40.009 INFO:teuthology.orchestra.run.smithi115.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.30558.log.gz 2024-01-24T23:47:40.009 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30288.log 2024-01-24T23:47:40.010 INFO:teuthology.orchestra.run.smithi115.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.29383.log.gz 2024-01-24T23:47:40.010 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28396.log 2024-01-24T23:47:40.010 INFO:teuthology.orchestra.run.smithi115.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.30766.log.gz 2024-01-24T23:47:40.010 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27173.log 2024-01-24T23:47:40.010 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29223.log 2024-01-24T23:47:40.010 INFO:teuthology.orchestra.run.smithi115.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30288.log.gz 2024-01-24T23:47:40.010 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27173.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28396.log.gz 2024-01-24T23:47:40.010 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28292.log 2024-01-24T23:47:40.010 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29223.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.27173.log.gz 2024-01-24T23:47:40.011 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27433.log 2024-01-24T23:47:40.011 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28292.log: gzip -5 --verbose -- 89.5% /var/log/ceph/ceph-client.admin.27225.log -- replaced with /var/log/ceph/ceph-client.admin.29223.log.gz 2024-01-24T23:47:40.011 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:47:40.011 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27433.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.28292.log.gz 2024-01-24T23:47:40.011 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27251.log 2024-01-24T23:47:40.011 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27225.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.27433.log.gz 2024-01-24T23:47:40.011 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26943.log 2024-01-24T23:47:40.011 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29515.log 2024-01-24T23:47:40.012 INFO:teuthology.orchestra.run.smithi115.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27225.log.gz 2024-01-24T23:47:40.012 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26943.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27251.log.gz 2024-01-24T23:47:40.012 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28266.log 2024-01-24T23:47:40.012 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29515.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.26943.log.gz 2024-01-24T23:47:40.012 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.19874.log 2024-01-24T23:47:40.012 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28708.log 2024-01-24T23:47:40.012 INFO:teuthology.orchestra.run.smithi115.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29515.log.gz 2024-01-24T23:47:40.012 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph.tmp-client.admin.19874.log: 88.8% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28266.log.gz -- replaced with /var/log/ceph/ceph.tmp-client.admin.19874.log.gz 2024-01-24T23:47:40.013 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:47:40.013 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27693.log 2024-01-24T23:47:40.013 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26787.log 2024-01-24T23:47:40.013 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28474.log 2024-01-24T23:47:40.013 INFO:teuthology.orchestra.run.smithi115.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.28708.log.gz 2024-01-24T23:47:40.013 INFO:teuthology.orchestra.run.smithi115.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.27693.log.gz 2024-01-24T23:47:40.013 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31000.log 2024-01-24T23:47:40.013 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28210.log 2024-01-24T23:47:40.013 INFO:teuthology.orchestra.run.smithi115.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.26787.log.gz 2024-01-24T23:47:40.014 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.31000.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.28474.log.gz 2024-01-24T23:47:40.014 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-01-24T23:47:40.014 INFO:teuthology.orchestra.run.smithi115.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31000.log.gz 2024-01-24T23:47:40.014 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23869.log 2024-01-24T23:47:40.014 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28002.log 2024-01-24T23:47:40.014 INFO:teuthology.orchestra.run.smithi115.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.28210.log.gz 2024-01-24T23:47:40.014 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.23869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29171.log 2024-01-24T23:47:40.014 INFO:teuthology.orchestra.run.smithi115.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.23869.log.gz 2024-01-24T23:47:40.015 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26466.log 2024-01-24T23:47:40.015 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30688.log 2024-01-24T23:47:40.015 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29171.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.28002.log.gz 2024-01-24T23:47:40.015 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28132.log 2024-01-24T23:47:40.015 INFO:teuthology.orchestra.run.smithi115.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29171.log.gz 2024-01-24T23:47:40.015 INFO:teuthology.orchestra.run.smithi115.stderr: 87.1%/var/log/ceph/ceph-client.admin.30688.log: -- replaced with /var/log/ceph/ceph-client.admin.26466.log.gz 2024-01-24T23:47:40.015 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28937.log 2024-01-24T23:47:40.015 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28132.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30688.log.gz 2024-01-24T23:47:40.015 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28158.log 2024-01-24T23:47:40.016 INFO:teuthology.orchestra.run.smithi115.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.28132.log.gz 2024-01-24T23:47:40.016 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26976.log 2024-01-24T23:47:40.016 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28937.log: /var/log/ceph/ceph-client.admin.28158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29924.log 2024-01-24T23:47:40.016 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26630.log 2024-01-24T23:47:40.016 INFO:teuthology.orchestra.run.smithi115.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.28937.log.gz 2024-01-24T23:47:40.016 INFO:teuthology.orchestra.run.smithi115.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.28158.log.gz 2024-01-24T23:47:40.016 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29924.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-01-24T23:47:40.016 INFO:teuthology.orchestra.run.smithi115.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.26976.log.gz 2024-01-24T23:47:40.017 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28448.log 2024-01-24T23:47:40.017 INFO:teuthology.orchestra.run.smithi115.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29924.log.gz 2024-01-24T23:47:40.017 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29541.log 2024-01-24T23:47:40.017 INFO:teuthology.orchestra.run.smithi115.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.26630.log.gz 2024-01-24T23:47:40.017 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28500.log 2024-01-24T23:47:40.017 INFO:teuthology.orchestra.run.smithi115.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.28448.log.gz 2024-01-24T23:47:40.017 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26619.log 2024-01-24T23:47:40.017 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.e.log 2024-01-24T23:47:40.018 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28500.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.29541.log.gz 2024-01-24T23:47:40.018 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29976.log 2024-01-24T23:47:40.018 INFO:teuthology.orchestra.run.smithi115.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.28500.log.gz 2024-01-24T23:47:40.018 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-mon.e.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26633.log 2024-01-24T23:47:40.018 INFO:teuthology.orchestra.run.smithi115.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.26619.log.gz 2024-01-24T23:47:40.018 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30132.log 2024-01-24T23:47:40.018 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26977.log 2024-01-24T23:47:40.018 INFO:teuthology.orchestra.run.smithi115.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29976.log.gz 2024-01-24T23:47:40.019 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27589.log 2024-01-24T23:47:40.019 INFO:teuthology.orchestra.run.smithi115.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.26633.log.gz 2024-01-24T23:47:40.019 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30610.log 2024-01-24T23:47:40.019 INFO:teuthology.orchestra.run.smithi115.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30132.log.gz 2024-01-24T23:47:40.019 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26977.log: /var/log/ceph/ceph-client.admin.27589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27615.log 2024-01-24T23:47:40.019 INFO:teuthology.orchestra.run.smithi115.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.26977.log.gz 2024-01-24T23:47:40.019 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30610.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27095.log 2024-01-24T23:47:40.019 INFO:teuthology.orchestra.run.smithi115.stderr: -- replaced with /var/log/ceph/ceph-client.admin.27589.log.gz 2024-01-24T23:47:40.019 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27615.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30610.log.gz 2024-01-24T23:47:40.020 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28028.log 2024-01-24T23:47:40.020 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26383.log 2024-01-24T23:47:40.020 INFO:teuthology.orchestra.run.smithi115.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.27615.log.gz/var/log/ceph/ceph-client.admin.28028.log: 2024-01-24T23:47:40.020 INFO:teuthology.orchestra.run.smithi115.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27095.log.gz 2024-01-24T23:47:40.020 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27407.log 2024-01-24T23:47:40.020 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30532.log 2024-01-24T23:47:40.020 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26383.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.28028.log.gz 2024-01-24T23:47:40.021 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30740.log 2024-01-24T23:47:40.021 INFO:teuthology.orchestra.run.smithi115.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.26383.log.gz 2024-01-24T23:47:40.021 INFO:teuthology.orchestra.run.smithi115.stderr: 89.2%/var/log/ceph/ceph-client.admin.30532.log: -- replaced with /var/log/ceph/ceph-client.admin.27407.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30870.log 2024-01-24T23:47:40.021 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:47:40.021 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30740.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30532.log.gz 2024-01-24T23:47:40.021 INFO:teuthology.orchestra.run.smithi115.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30740.log.gz 2024-01-24T23:47:40.021 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28422.log 2024-01-24T23:47:40.021 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29328.log 2024-01-24T23:47:40.021 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29276.log 2024-01-24T23:47:40.022 INFO:teuthology.orchestra.run.smithi115.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30870.log.gz 2024-01-24T23:47:40.022 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29328.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28422.log.gz 2024-01-24T23:47:40.022 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30974.log 2024-01-24T23:47:40.022 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31026.log 2024-01-24T23:47:40.022 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29276.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29328.log.gz 2024-01-24T23:47:40.022 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29489.log 2024-01-24T23:47:40.022 INFO:teuthology.orchestra.run.smithi115.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29276.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30948.log 2024-01-24T23:47:40.022 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:47:40.023 INFO:teuthology.orchestra.run.smithi115.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30974.log.gz 2024-01-24T23:47:40.023 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.31026.log: /var/log/ceph/ceph-client.admin.29489.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31026.log.gz 2024-01-24T23:47:40.023 INFO:teuthology.orchestra.run.smithi115.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.29489.log.gz 2024-01-24T23:47:40.023 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27147.log 2024-01-24T23:47:40.023 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30236.log 2024-01-24T23:47:40.023 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27147.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29872.log 2024-01-24T23:47:40.023 INFO:teuthology.orchestra.run.smithi115.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30948.log.gz 2024-01-24T23:47:40.023 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29197.log 2024-01-24T23:47:40.024 INFO:teuthology.orchestra.run.smithi115.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.27147.log.gz 2024-01-24T23:47:40.024 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26783.log 2024-01-24T23:47:40.024 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29872.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30236.log.gz 2024-01-24T23:47:40.024 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29197.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.29872.log.gz 2024-01-24T23:47:40.024 INFO:teuthology.orchestra.run.smithi115.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29197.log.gz 2024-01-24T23:47:40.024 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23558.log 2024-01-24T23:47:40.024 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28318.log 2024-01-24T23:47:40.024 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.23558.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.26783.log.gz 2024-01-24T23:47:40.025 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30922.log 2024-01-24T23:47:40.025 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28859.log 2024-01-24T23:47:40.025 INFO:teuthology.orchestra.run.smithi115.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.23558.log.gz 2024-01-24T23:47:40.025 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26939.log 2024-01-24T23:47:40.025 INFO:teuthology.orchestra.run.smithi115.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.28318.log.gz 2024-01-24T23:47:40.025 INFO:teuthology.orchestra.run.smithi115.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30922.log.gz 2024-01-24T23:47:40.025 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28859.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.28859.log.gz 2024-01-24T23:47:40.025 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31367.log 2024-01-24T23:47:40.026 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28753.log 2024-01-24T23:47:40.026 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.31367.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.26939.log.gz 2024-01-24T23:47:40.026 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30400.log 2024-01-24T23:47:40.026 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27563.log 2024-01-24T23:47:40.026 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30400.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.28753.log.gz 2024-01-24T23:47:40.026 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31208.log 2024-01-24T23:47:40.026 INFO:teuthology.orchestra.run.smithi115.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.30400.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30262.log 2024-01-24T23:47:40.026 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27563.log: 2024-01-24T23:47:40.027 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.31208.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.27563.log.gz 2024-01-24T23:47:40.027 INFO:teuthology.orchestra.run.smithi115.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31208.log.gz 2024-01-24T23:47:40.027 INFO:teuthology.orchestra.run.smithi115.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.admin.31367.log.gz 2024-01-24T23:47:40.027 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-01-24T23:47:40.027 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31268.log 2024-01-24T23:47:40.027 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27485.log 2024-01-24T23:47:40.027 INFO:teuthology.orchestra.run.smithi115.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30262.log.gz 2024-01-24T23:47:40.027 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.31268.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.d.log 2024-01-24T23:47:40.027 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26819.log 2024-01-24T23:47:40.028 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27485.log: /var/log/ceph/ceph-mon.d.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.27485.log.gz 2024-01-24T23:47:40.028 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26817.log 2024-01-24T23:47:40.028 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26819.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.26819.log.gz 2024-01-24T23:47:40.028 INFO:teuthology.orchestra.run.smithi115.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.admin.31268.log.gz 2024-01-24T23:47:40.028 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30158.log 2024-01-24T23:47:40.028 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30636.log 2024-01-24T23:47:40.028 INFO:teuthology.orchestra.run.smithi115.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.26817.log.gz/var/log/ceph/ceph-client.admin.30158.log: 2024-01-24T23:47:40.028 INFO:teuthology.orchestra.run.smithi115.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.30158.log.gz 2024-01-24T23:47:40.029 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29411.log 2024-01-24T23:47:40.029 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26632.log 2024-01-24T23:47:40.029 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29411.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30636.log.gz 2024-01-24T23:47:40.029 INFO:teuthology.orchestra.run.smithi115.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29411.log.gz 2024-01-24T23:47:40.029 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31234.log 2024-01-24T23:47:40.029 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31386.log 2024-01-24T23:47:40.029 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.31234.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.26632.log.gz 2024-01-24T23:47:40.029 INFO:teuthology.orchestra.run.smithi115.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31234.log.gz 2024-01-24T23:47:40.029 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28604.log 2024-01-24T23:47:40.030 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.31386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26629.log 2024-01-24T23:47:40.030 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28604.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.28604.log.gz 2024-01-24T23:47:40.030 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-01-24T23:47:40.030 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26629.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.26629.log.gz 2024-01-24T23:47:40.030 INFO:teuthology.orchestra.run.smithi115.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.admin.31386.log.gz 2024-01-24T23:47:40.030 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28240.log 2024-01-24T23:47:40.030 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29898.log 2024-01-24T23:47:40.030 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28240.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.28240.log.gz 2024-01-24T23:47:40.031 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27872.log 2024-01-24T23:47:40.031 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29898.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.29898.log.gz 2024-01-24T23:47:40.031 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28682.log 2024-01-24T23:47:40.031 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27872.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.27872.log.gz 2024-01-24T23:47:40.031 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30844.log 2024-01-24T23:47:40.031 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28682.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.28682.log.gz 2024-01-24T23:47:40.031 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31130.log 2024-01-24T23:47:40.031 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30844.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.30844.log.gz 2024-01-24T23:47:40.031 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30106.log 2024-01-24T23:47:40.032 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.31130.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31130.log.gz 2024-01-24T23:47:40.032 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26818.log 2024-01-24T23:47:40.032 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30106.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30106.log.gz 2024-01-24T23:47:40.032 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31078.log 2024-01-24T23:47:40.032 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26818.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.26818.log.gz 2024-01-24T23:47:40.032 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29742.log 2024-01-24T23:47:40.032 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.31078.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31078.log.gz 2024-01-24T23:47:40.032 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26291.log 2024-01-24T23:47:40.033 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29742.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29742.log.gz 2024-01-24T23:47:40.033 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27976.log 2024-01-24T23:47:40.033 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26291.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.26291.log.gz 2024-01-24T23:47:40.033 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27719.log 2024-01-24T23:47:40.033 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27976.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.27976.log.gz 2024-01-24T23:47:40.033 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28989.log 2024-01-24T23:47:40.033 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27719.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.27719.log.gz 2024-01-24T23:47:40.033 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28370.log 2024-01-24T23:47:40.033 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28989.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28989.log.gz 2024-01-24T23:47:40.034 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30662.log 2024-01-24T23:47:40.051 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28370.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.28370.log.gz 2024-01-24T23:47:40.051 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30054.log 2024-01-24T23:47:40.052 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30662.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30662.log.gz 2024-01-24T23:47:40.052 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28552.log 2024-01-24T23:47:40.052 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30054.log: 92.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-01-24T23:47:40.052 INFO:teuthology.orchestra.run.smithi115.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30054.log.gz 2024-01-24T23:47:40.052 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27381.log 2024-01-24T23:47:40.052 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28526.log 2024-01-24T23:47:40.052 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27381.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28552.log.gz 2024-01-24T23:47:40.052 INFO:teuthology.orchestra.run.smithi115.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.27381.log.gz 2024-01-24T23:47:40.053 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31156.log 2024-01-24T23:47:40.053 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27745.log 2024-01-24T23:47:40.053 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28526.log: /var/log/ceph/ceph-client.admin.31156.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.28526.log.gz 2024-01-24T23:47:40.053 INFO:teuthology.orchestra.run.smithi115.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.31156.log.gz 2024-01-24T23:47:40.053 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30818.log 2024-01-24T23:47:40.053 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26467.log 2024-01-24T23:47:40.053 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27745.log: /var/log/ceph/ceph-client.admin.30818.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.27745.log.gz 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30818.log.gz 2024-01-24T23:47:40.053 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:47:40.054 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29119.log 2024-01-24T23:47:40.054 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-01-24T23:47:40.054 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26467.log: /var/log/ceph/ceph-client.admin.29119.log: 88.5% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.29119.log.gz 2024-01-24T23:47:40.054 INFO:teuthology.orchestra.run.smithi115.stderr: -- replaced with /var/log/ceph/ceph-client.admin.26467.log.gz 2024-01-24T23:47:40.054 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29041.log 2024-01-24T23:47:40.054 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28963.log 2024-01-24T23:47:40.054 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.29041.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.29041.log.gz 2024-01-24T23:47:40.054 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26464.log 2024-01-24T23:47:40.054 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28963.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.28963.log.gz 2024-01-24T23:47:40.055 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29794.log 2024-01-24T23:47:40.055 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26464.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.26464.log.gz 2024-01-24T23:47:40.055 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29249.log 2024-01-24T23:47:40.055 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29794.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29794.log.gz 2024-01-24T23:47:40.055 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27511.log 2024-01-24T23:47:40.055 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29249.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29249.log.gz 2024-01-24T23:47:40.055 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30714.log 2024-01-24T23:47:40.055 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27511.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.27511.log.gz 2024-01-24T23:47:40.055 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28106.log 2024-01-24T23:47:40.056 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30714.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.30714.log.gz 2024-01-24T23:47:40.056 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27355.log 2024-01-24T23:47:40.058 INFO:teuthology.orchestra.run.smithi132.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.31746.log.gz 2024-01-24T23:47:40.059 INFO:teuthology.orchestra.run.smithi132.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27308.log.gz 2024-01-24T23:47:40.059 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31191.log 2024-01-24T23:47:40.059 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27002.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.f.log 2024-01-24T23:47:40.059 INFO:teuthology.orchestra.run.smithi132.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.28684.log.gz 2024-01-24T23:47:40.059 INFO:teuthology.orchestra.run.smithi132.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27002.log.gz 2024-01-24T23:47:40.059 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29752.log 2024-01-24T23:47:40.059 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31191.log: /var/log/ceph/ceph-mon.f.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31545.log 2024-01-24T23:47:40.059 INFO:teuthology.orchestra.run.smithi132.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.31191.log.gz 2024-01-24T23:47:40.060 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.29752.log: -- /var/log/ceph/ceph-client.admin.28564.log 2024-01-24T23:47:40.060 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33771.log 2024-01-24T23:47:40.060 INFO:teuthology.orchestra.run.smithi132.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29752.log.gz 2024-01-24T23:47:40.060 INFO:teuthology.orchestra.run.smithi132.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.31545.log.gz 2024-01-24T23:47:40.060 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32731.log 2024-01-24T23:47:40.060 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30264.log 2024-01-24T23:47:40.060 INFO:teuthology.orchestra.run.smithi132.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.28564.log.gz 2024-01-24T23:47:40.060 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29223.log 2024-01-24T23:47:40.061 INFO:teuthology.orchestra.run.smithi132.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33771.log.gz 2024-01-24T23:47:40.061 INFO:teuthology.orchestra.run.smithi132.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32731.log.gz 2024-01-24T23:47:40.061 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27783.log 2024-01-24T23:47:40.061 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27441.log 2024-01-24T23:47:40.061 INFO:teuthology.orchestra.run.smithi132.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.30264.log.gz/var/log/ceph/ceph-client.admin.29223.log: 2024-01-24T23:47:40.061 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29414.log 2024-01-24T23:47:40.061 INFO:teuthology.orchestra.run.smithi132.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.29223.log.gz 2024-01-24T23:47:40.061 INFO:teuthology.orchestra.run.smithi132.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.27783.log.gz 2024-01-24T23:47:40.062 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32107.log 2024-01-24T23:47:40.062 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30939.log 2024-01-24T23:47:40.062 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29414.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.27441.log.gz 2024-01-24T23:47:40.062 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31818.log 2024-01-24T23:47:40.062 INFO:teuthology.orchestra.run.smithi132.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.29414.log.gz 2024-01-24T23:47:40.062 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30939.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.32107.log.gz 2024-01-24T23:47:40.062 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33939.log 2024-01-24T23:47:40.062 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30578.log 2024-01-24T23:47:40.062 INFO:teuthology.orchestra.run.smithi132.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.30939.log.gz 2024-01-24T23:47:40.063 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28660.log 2024-01-24T23:47:40.063 INFO:teuthology.orchestra.run.smithi132.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.31818.log.gz 2024-01-24T23:47:40.063 INFO:teuthology.orchestra.run.smithi132.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33939.log.gz 2024-01-24T23:47:40.063 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32010.log 2024-01-24T23:47:40.063 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28780.log 2024-01-24T23:47:40.063 INFO:teuthology.orchestra.run.smithi132.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.30578.log.gz 2024-01-24T23:47:40.063 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32010.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.28660.log.gz 2024-01-24T23:47:40.063 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28102.log 2024-01-24T23:47:40.064 INFO:teuthology.orchestra.run.smithi132.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.32010.log.gz 2024-01-24T23:47:40.064 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29175.log 2024-01-24T23:47:40.064 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31215.log 2024-01-24T23:47:40.064 INFO:teuthology.orchestra.run.smithi132.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.28780.log.gz 2024-01-24T23:47:40.064 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33691.log 2024-01-24T23:47:40.064 INFO:teuthology.orchestra.run.smithi132.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.28102.log.gz 2024-01-24T23:47:40.064 INFO:teuthology.orchestra.run.smithi132.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.29175.log.gz 2024-01-24T23:47:40.064 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32083.log 2024-01-24T23:47:40.065 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31119.log 2024-01-24T23:47:40.065 INFO:teuthology.orchestra.run.smithi132.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.31215.log.gz 2024-01-24T23:47:40.065 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32083.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.33691.log.gz 2024-01-24T23:47:40.065 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28708.log 2024-01-24T23:47:40.065 INFO:teuthology.orchestra.run.smithi132.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32083.log.gz 2024-01-24T23:47:40.065 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30410.log 2024-01-24T23:47:40.065 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29291.log 2024-01-24T23:47:40.065 INFO:teuthology.orchestra.run.smithi132.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.31119.log.gz 2024-01-24T23:47:40.065 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30410.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.28708.log.gz 2024-01-24T23:47:40.066 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28636.log 2024-01-24T23:47:40.066 INFO:teuthology.orchestra.run.smithi132.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30410.log.gz 2024-01-24T23:47:40.066 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30703.log 2024-01-24T23:47:40.066 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27393.log 2024-01-24T23:47:40.066 INFO:teuthology.orchestra.run.smithi132.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.29291.log.gz 2024-01-24T23:47:40.066 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29490.log 2024-01-24T23:47:40.066 INFO:teuthology.orchestra.run.smithi132.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.28636.log.gz 2024-01-24T23:47:40.066 INFO:teuthology.orchestra.run.smithi132.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.30703.log.gz 2024-01-24T23:47:40.067 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30338.log 2024-01-24T23:47:40.067 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31962.log 2024-01-24T23:47:40.067 INFO:teuthology.orchestra.run.smithi132.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.27393.log.gz 2024-01-24T23:47:40.067 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30338.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.29490.log.gz 2024-01-24T23:47:40.067 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34278.log 2024-01-24T23:47:40.067 INFO:teuthology.orchestra.run.smithi132.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.30338.log.gz 2024-01-24T23:47:40.067 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29096.log 2024-01-24T23:47:40.067 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.34278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33715.log 2024-01-24T23:47:40.067 INFO:teuthology.orchestra.run.smithi132.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31962.log.gz 2024-01-24T23:47:40.068 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31866.log 2024-01-24T23:47:40.068 INFO:teuthology.orchestra.run.smithi132.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.29096.log.gz 2024-01-24T23:47:40.068 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33111.log 2024-01-24T23:47:40.068 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31866.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33715.log.gz 2024-01-24T23:47:40.068 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27591.log 2024-01-24T23:47:40.068 INFO:teuthology.orchestra.run.smithi132.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31866.log.gz 2024-01-24T23:47:40.068 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30386.log 2024-01-24T23:47:40.069 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33111.log: /var/log/ceph/ceph-client.admin.27591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23838.log 2024-01-24T23:47:40.069 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28350.log 2024-01-24T23:47:40.069 INFO:teuthology.orchestra.run.smithi132.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33111.log.gz 2024-01-24T23:47:40.069 INFO:teuthology.orchestra.run.smithi132.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27591.log.gz 2024-01-24T23:47:40.069 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.23838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32857.log 2024-01-24T23:47:40.069 INFO:teuthology.orchestra.run.smithi132.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.30386.log.gz 2024-01-24T23:47:40.069 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28350.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.23838.log.gz 2024-01-24T23:47:40.069 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30749.log 2024-01-24T23:47:40.069 INFO:teuthology.orchestra.run.smithi132.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.28350.log.gz 2024-01-24T23:47:40.070 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27957.log 2024-01-24T23:47:40.070 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28470.log 2024-01-24T23:47:40.070 INFO:teuthology.orchestra.run.smithi132.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.32857.log.gz 2024-01-24T23:47:40.070 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27957.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.30749.log.gz 2024-01-24T23:47:40.070 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26797.log 2024-01-24T23:47:40.070 INFO:teuthology.orchestra.run.smithi132.stderr: 88.4%/var/log/ceph/ceph-client.admin.28470.log: -- replaced with /var/log/ceph/ceph-client.admin.27957.log.gz 2024-01-24T23:47:40.070 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29269.log 2024-01-24T23:47:40.070 INFO:teuthology.orchestra.run.smithi132.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.28470.log.gz 2024-01-24T23:47:40.071 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.26797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31067.log 2024-01-24T23:47:40.071 INFO:teuthology.orchestra.run.smithi132.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.26797.log.gz 2024-01-24T23:47:40.071 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28732.log 2024-01-24T23:47:40.071 INFO:teuthology.orchestra.run.smithi132.stderr: 94.7% -- replaced with /var/log/ceph/ceph-client.admin.34278.log.gz 2024-01-24T23:47:40.071 INFO:teuthology.orchestra.run.smithi132.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29269.log.gz 2024-01-24T23:47:40.071 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31362.log 2024-01-24T23:47:40.071 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27186.log 2024-01-24T23:47:40.071 INFO:teuthology.orchestra.run.smithi132.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.31067.log.gz 2024-01-24T23:47:40.072 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31362.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.28732.log.gz 2024-01-24T23:47:40.072 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30888.log 2024-01-24T23:47:40.072 INFO:teuthology.orchestra.run.smithi132.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.31362.log.gz 2024-01-24T23:47:40.072 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29970.log 2024-01-24T23:47:40.072 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33547.log 2024-01-24T23:47:40.072 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29970.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.27186.log.gz 2024-01-24T23:47:40.072 INFO:teuthology.orchestra.run.smithi132.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30888.log.gz 2024-01-24T23:47:40.072 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27264.log 2024-01-24T23:47:40.073 INFO:teuthology.orchestra.run.smithi132.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.29970.log.gz 2024-01-24T23:47:40.073 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26957.log 2024-01-24T23:47:40.073 INFO:teuthology.orchestra.run.smithi132.stderr:gzip 87.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.33547.log.gz --verbose 2024-01-24T23:47:40.073 INFO:teuthology.orchestra.run.smithi132.stderr: -- /var/log/ceph/ceph-client.admin.32491.log 2024-01-24T23:47:40.073 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27264.log: /var/log/ceph/ceph-client.admin.26957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29924.log 2024-01-24T23:47:40.073 INFO:teuthology.orchestra.run.smithi132.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.27264.log.gz 2024-01-24T23:47:40.073 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32491.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.26957.log.gz 2024-01-24T23:47:40.073 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28374.log 2024-01-24T23:47:40.073 INFO:teuthology.orchestra.run.smithi132.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.32491.log.gz 2024-01-24T23:47:40.074 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29948.log 2024-01-24T23:47:40.074 INFO:teuthology.orchestra.run.smithi132.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.29924.log.gz 2024-01-24T23:47:40.074 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33403.log 2024-01-24T23:47:40.074 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30140.log 2024-01-24T23:47:40.074 INFO:teuthology.orchestra.run.smithi132.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.28374.log.gz 2024-01-24T23:47:40.074 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33403.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.29948.log.gz 2024-01-24T23:47:40.074 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28448.log 2024-01-24T23:47:40.074 INFO:teuthology.orchestra.run.smithi132.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33403.log.gz 2024-01-24T23:47:40.075 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29514.log 2024-01-24T23:47:40.075 INFO:teuthology.orchestra.run.smithi132.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.30140.log.gz 2024-01-24T23:47:40.075 INFO:teuthology.orchestra.run.smithi132.stderr:gzip/var/log/ceph/ceph-client.admin.28448.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31671.log 2024-01-24T23:47:40.075 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28324.log 2024-01-24T23:47:40.075 INFO:teuthology.orchestra.run.smithi132.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.28448.log.gz 2024-01-24T23:47:40.075 INFO:teuthology.orchestra.run.smithi132.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29514.log.gz 2024-01-24T23:47:40.075 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30290.log 2024-01-24T23:47:40.075 INFO:teuthology.orchestra.run.smithi132.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31671.log.gz 2024-01-24T23:47:40.075 INFO:teuthology.orchestra.run.smithi132.stderr:gzip/var/log/ceph/ceph-client.admin.28324.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.28126.log 2024-01-24T23:47:40.076 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30116.log 2024-01-24T23:47:40.076 INFO:teuthology.orchestra.run.smithi132.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.28324.log.gz 2024-01-24T23:47:40.076 INFO:teuthology.orchestra.run.smithi132.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.30290.log.gz 2024-01-24T23:47:40.076 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33235.log 2024-01-24T23:47:40.076 INFO:teuthology.orchestra.run.smithi132.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.28126.log.gz 2024-01-24T23:47:40.076 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32563.log 2024-01-24T23:47:40.076 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30238.log 2024-01-24T23:47:40.076 INFO:teuthology.orchestra.run.smithi132.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30116.log.gz 2024-01-24T23:47:40.077 INFO:teuthology.orchestra.run.smithi132.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33235.log.gz 2024-01-24T23:47:40.077 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30166.log 2024-01-24T23:47:40.077 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33667.log 2024-01-24T23:47:40.077 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30238.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.32563.log.gz 2024-01-24T23:47:40.077 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29536.log 2024-01-24T23:47:40.077 INFO:teuthology.orchestra.run.smithi132.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.30238.log.gz 2024-01-24T23:47:40.077 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33667.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.30166.log.gz 2024-01-24T23:47:40.077 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32371.log 2024-01-24T23:47:40.077 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28906.log 2024-01-24T23:47:40.077 INFO:teuthology.orchestra.run.smithi132.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33667.log.gz 2024-01-24T23:47:40.078 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32371.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.29536.log.gz 2024-01-24T23:47:40.078 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32395.log 2024-01-24T23:47:40.078 INFO:teuthology.orchestra.run.smithi132.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.32371.log.gz 2024-01-24T23:47:40.078 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28804.log 2024-01-24T23:47:40.078 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.34123.log 2024-01-24T23:47:40.078 INFO:teuthology.orchestra.run.smithi132.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.28906.log.gz 2024-01-24T23:47:40.078 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28804.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.32395.log.gz 2024-01-24T23:47:40.078 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27615.log 2024-01-24T23:47:40.078 INFO:teuthology.orchestra.run.smithi132.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.28804.log.gz 2024-01-24T23:47:40.079 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.foo.34123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28274.log 2024-01-24T23:47:40.079 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29050.log 2024-01-24T23:47:40.079 INFO:teuthology.orchestra.run.smithi132.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.foo.34123.log.gz 2024-01-24T23:47:40.079 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28274.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.27615.log.gz 2024-01-24T23:47:40.079 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32299.log 2024-01-24T23:47:40.079 INFO:teuthology.orchestra.run.smithi132.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.28274.log.gz 2024-01-24T23:47:40.079 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30458.log 2024-01-24T23:47:40.079 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33063.log 2024-01-24T23:47:40.079 INFO:teuthology.orchestra.run.smithi132.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.29050.log.gz 2024-01-24T23:47:40.080 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30458.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32299.log.gz 2024-01-24T23:47:40.080 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32179.log 2024-01-24T23:47:40.080 INFO:teuthology.orchestra.run.smithi132.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30458.log.gz 2024-01-24T23:47:40.080 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28516.log 2024-01-24T23:47:40.080 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31890.log 2024-01-24T23:47:40.080 INFO:teuthology.orchestra.run.smithi132.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33063.log.gz 2024-01-24T23:47:40.080 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28151.log 2024-01-24T23:47:40.080 INFO:teuthology.orchestra.run.smithi132.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32179.log.gz 2024-01-24T23:47:40.080 INFO:teuthology.orchestra.run.smithi132.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.28516.log.gz 2024-01-24T23:47:40.081 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30532.log 2024-01-24T23:47:40.081 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31890.log: /var/log/ceph/ceph-client.admin.28151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28422.log 2024-01-24T23:47:40.081 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30532.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31890.log.gz 2024-01-24T23:47:40.081 INFO:teuthology.orchestra.run.smithi132.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.28151.log.gz 2024-01-24T23:47:40.081 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28005.log 2024-01-24T23:47:40.081 INFO:teuthology.orchestra.run.smithi132.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30532.log.gz 2024-01-24T23:47:40.081 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28954.log 2024-01-24T23:47:40.081 INFO:teuthology.orchestra.run.smithi132.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.28422.log.gz 2024-01-24T23:47:40.081 INFO:teuthology.orchestra.run.smithi132.stderr:gzip/var/log/ceph/ceph-client.admin.28005.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.30727.log 2024-01-24T23:47:40.082 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27880.log 2024-01-24T23:47:40.082 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30727.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.28954.log.gz 2024-01-24T23:47:40.082 INFO:teuthology.orchestra.run.smithi132.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.28005.log.gz 2024-01-24T23:47:40.082 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27074.log 2024-01-24T23:47:40.082 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27880.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.30727.log.gz 2024-01-24T23:47:40.082 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29996.log 2024-01-24T23:47:40.082 INFO:teuthology.orchestra.run.smithi132.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.27880.log.gz 2024-01-24T23:47:40.082 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27074.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-01-24T23:47:40.082 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28494.log 2024-01-24T23:47:40.083 INFO:teuthology.orchestra.run.smithi132.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.27074.log.gz 2024-01-24T23:47:40.083 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-osd.3.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.29996.log.gz 2024-01-24T23:47:40.083 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29704.log 2024-01-24T23:47:40.083 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28494.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-01-24T23:47:40.083 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29438.log 2024-01-24T23:47:40.083 INFO:teuthology.orchestra.run.smithi132.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.28494.log.gz 2024-01-24T23:47:40.083 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- 87.7% /var/log/ceph/ceph-client.admin.27711.log 2024-01-24T23:47:40.083 INFO:teuthology.orchestra.run.smithi132.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29704.log.gz 2024-01-24T23:47:40.083 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29728.log 2024-01-24T23:47:40.084 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27711.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.29438.log.gz 2024-01-24T23:47:40.084 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27351.log 2024-01-24T23:47:40.084 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29802.log 2024-01-24T23:47:40.084 INFO:teuthology.orchestra.run.smithi132.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27711.log.gz 2024-01-24T23:47:40.084 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29728.log: /var/log/ceph/ceph-client.admin.27351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29608.log 2024-01-24T23:47:40.084 INFO:teuthology.orchestra.run.smithi132.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.29728.log.gz 2024-01-24T23:47:40.084 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29802.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.27351.log.gz 2024-01-24T23:47:40.084 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28226.log 2024-01-24T23:47:40.085 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32443.log 2024-01-24T23:47:40.085 INFO:teuthology.orchestra.run.smithi132.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.29802.log.gz 2024-01-24T23:47:40.085 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28226.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.29608.log.gz 2024-01-24T23:47:40.085 INFO:teuthology.orchestra.run.smithi132.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.30914.log 2024-01-24T23:47:40.085 INFO:teuthology.orchestra.run.smithi132.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.28226.log.gz 2024-01-24T23:47:40.085 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27639.log 2024-01-24T23:47:40.085 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29850.log 2024-01-24T23:47:40.085 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30914.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.32443.log.gz 2024-01-24T23:47:40.085 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32905.log 2024-01-24T23:47:40.086 INFO:teuthology.orchestra.run.smithi132.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30914.log.gz 2024-01-24T23:47:40.086 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33159.log 2024-01-24T23:47:40.086 INFO:teuthology.orchestra.run.smithi132.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.27639.log.gz 2024-01-24T23:47:40.086 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32905.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.29850.log.gz 2024-01-24T23:47:40.086 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.34145.log 2024-01-24T23:47:40.086 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33451.log 2024-01-24T23:47:40.086 INFO:teuthology.orchestra.run.smithi132.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32905.log.gz 2024-01-24T23:47:40.086 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.foo.34145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32755.log 2024-01-24T23:47:40.086 INFO:teuthology.orchestra.run.smithi132.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33159.log.gz 2024-01-24T23:47:40.087 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30679.log 2024-01-24T23:47:40.087 INFO:teuthology.orchestra.run.smithi132.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.foo.34145.log.gz 2024-01-24T23:47:40.087 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26911.log 2024-01-24T23:47:40.087 INFO:teuthology.orchestra.run.smithi132.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33451.log.gz 2024-01-24T23:47:40.087 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30679.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.32755.log.gz 2024-01-24T23:47:40.087 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27489.log 2024-01-24T23:47:40.087 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.26911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28828.log 2024-01-24T23:47:40.087 INFO:teuthology.orchestra.run.smithi132.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.30679.log.gz 2024-01-24T23:47:40.087 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27050.log 2024-01-24T23:47:40.088 INFO:teuthology.orchestra.run.smithi132.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.26911.log.gz 2024-01-24T23:47:40.088 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28756.log 2024-01-24T23:47:40.088 INFO:teuthology.orchestra.run.smithi132.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27489.log.gz 2024-01-24T23:47:40.088 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27050.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.28828.log.gz 2024-01-24T23:47:40.088 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33379.log 2024-01-24T23:47:40.088 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32419.log 2024-01-24T23:47:40.088 INFO:teuthology.orchestra.run.smithi132.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27050.log.gz 2024-01-24T23:47:40.088 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28756.log: /var/log/ceph/ceph-client.admin.33379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32611.log 2024-01-24T23:47:40.089 INFO:teuthology.orchestra.run.smithi132.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.28756.log.gz 2024-01-24T23:47:40.089 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27759.log 2024-01-24T23:47:40.089 INFO:teuthology.orchestra.run.smithi132.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33379.log.gz 2024-01-24T23:47:40.089 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31986.log 2024-01-24T23:47:40.089 INFO:teuthology.orchestra.run.smithi132.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.32419.log.gz 2024-01-24T23:47:40.089 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27130.log 2024-01-24T23:47:40.089 INFO:teuthology.orchestra.run.smithi132.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32611.log.gz 2024-01-24T23:47:40.089 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30771.log 2024-01-24T23:47:40.090 INFO:teuthology.orchestra.run.smithi132.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27759.log.gz 2024-01-24T23:47:40.090 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28078.log 2024-01-24T23:47:40.090 INFO:teuthology.orchestra.run.smithi132.stderr: 88.4%/var/log/ceph/ceph-client.admin.30771.log: -- replaced with /var/log/ceph/ceph-client.admin.31986.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27286.log 2024-01-24T23:47:40.090 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:47:40.090 INFO:teuthology.orchestra.run.smithi132.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27130.log.gz 2024-01-24T23:47:40.090 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26865.log 2024-01-24T23:47:40.090 INFO:teuthology.orchestra.run.smithi132.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.30771.log.gz 2024-01-24T23:47:40.090 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31593.log 2024-01-24T23:47:40.090 INFO:teuthology.orchestra.run.smithi132.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.28078.log.gz 2024-01-24T23:47:40.091 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.26865.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.27286.log.gz 2024-01-24T23:47:40.091 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32347.log 2024-01-24T23:47:40.091 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31593.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.26865.log.gz 2024-01-24T23:47:40.091 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33427.log 2024-01-24T23:47:40.091 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32347.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.i.log 2024-01-24T23:47:40.091 INFO:teuthology.orchestra.run.smithi132.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31593.log.gz 2024-01-24T23:47:40.091 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32203.log 2024-01-24T23:47:40.091 INFO:teuthology.orchestra.run.smithi132.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.32347.log.gz 2024-01-24T23:47:40.091 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-mon.i.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33427.log.gz 2024-01-24T23:47:40.092 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30654.log 2024-01-24T23:47:40.092 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29680.log 2024-01-24T23:47:40.092 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26819.log 2024-01-24T23:47:40.092 INFO:teuthology.orchestra.run.smithi132.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32203.log.gz 2024-01-24T23:47:40.092 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29680.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.30654.log.gz 2024-01-24T23:47:40.092 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32683.log 2024-01-24T23:47:40.092 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.g.log 2024-01-24T23:47:40.095 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.26819.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.29680.log.gz 2024-01-24T23:47:40.095 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31416.log 2024-01-24T23:47:40.095 INFO:teuthology.orchestra.run.smithi132.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.26819.log.gz 2024-01-24T23:47:40.096 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32155.log 2024-01-24T23:47:40.096 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-mon.g.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32683.log.gz 2024-01-24T23:47:40.096 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31440.log 2024-01-24T23:47:40.096 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31416.log: /var/log/ceph/ceph-client.admin.32155.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31416.log.gz 2024-01-24T23:47:40.096 INFO:teuthology.orchestra.run.smithi132.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.32155.log.gz 2024-01-24T23:47:40.096 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.34077.log 2024-01-24T23:47:40.096 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33087.log 2024-01-24T23:47:40.096 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.foo.34077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30968.log 2024-01-24T23:47:40.097 INFO:teuthology.orchestra.run.smithi132.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.31440.log.gz 2024-01-24T23:47:40.097 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33087.log: 88.0% -- replaced with /var/log/ceph/ceph-client.foo.34077.log.gz 2024-01-24T23:47:40.097 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29146.log 2024-01-24T23:47:40.097 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33015.log 2024-01-24T23:47:40.097 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29146.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33087.log.gz 2024-01-24T23:47:40.097 INFO:teuthology.orchestra.run.smithi132.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.30968.log.gz 2024-01-24T23:47:40.097 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30216.log 2024-01-24T23:47:40.097 INFO:teuthology.orchestra.run.smithi132.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.29146.log.gz 2024-01-24T23:47:40.098 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32515.log 2024-01-24T23:47:40.098 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30216.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.34167.log 2024-01-24T23:47:40.098 INFO:teuthology.orchestra.run.smithi132.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33015.log.gz 2024-01-24T23:47:40.098 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32515.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.30216.log.gz 2024-01-24T23:47:40.098 INFO:teuthology.orchestra.run.smithi132.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32515.log.gz 2024-01-24T23:47:40.098 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31722.log 2024-01-24T23:47:40.098 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.foo.34167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30556.log 2024-01-24T23:47:40.098 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31722.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-01-24T23:47:40.098 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30556.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.31722.log.gz 2024-01-24T23:47:40.099 INFO:teuthology.orchestra.run.smithi132.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.30556.log.gz 2024-01-24T23:47:40.099 INFO:teuthology.orchestra.run.smithi132.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.foo.34167.log.gz 2024-01-24T23:47:40.099 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30821.log 2024-01-24T23:47:40.099 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30434.log 2024-01-24T23:47:40.099 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33867.log 2024-01-24T23:47:40.099 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30434.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30821.log.gz 2024-01-24T23:47:40.099 INFO:teuthology.orchestra.run.smithi132.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.30434.log.gz 2024-01-24T23:47:40.099 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31093.log 2024-01-24T23:47:40.100 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27242.log 2024-01-24T23:47:40.100 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31093.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33867.log.gz 2024-01-24T23:47:40.100 INFO:teuthology.orchestra.run.smithi132.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.31093.log.gz 2024-01-24T23:47:40.100 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-01-24T23:47:40.100 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33259.log 2024-01-24T23:47:40.100 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.27242.log.gz 2024-01-24T23:47:40.100 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29898.log 2024-01-24T23:47:40.100 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33259.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33259.log.gz 2024-01-24T23:47:40.100 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33475.log 2024-01-24T23:47:40.101 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29898.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.29898.log.gz 2024-01-24T23:47:40.101 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31263.log 2024-01-24T23:47:40.101 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33475.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33475.log.gz 2024-01-24T23:47:40.101 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31167.log 2024-01-24T23:47:40.101 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31263.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.31263.log.gz 2024-01-24T23:47:40.101 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27663.log 2024-01-24T23:47:40.101 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31167.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.31167.log.gz 2024-01-24T23:47:40.101 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29247.log 2024-01-24T23:47:40.101 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27663.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.27663.log.gz 2024-01-24T23:47:40.102 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28176.log 2024-01-24T23:47:40.102 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29247.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.29247.log.gz 2024-01-24T23:47:40.102 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30068.log 2024-01-24T23:47:40.102 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28176.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.28176.log.gz 2024-01-24T23:47:40.102 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33283.log 2024-01-24T23:47:40.102 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30068.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30068.log.gz 2024-01-24T23:47:40.103 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29072.log 2024-01-24T23:47:40.103 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33283.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33283.log.gz 2024-01-24T23:47:40.103 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27687.log 2024-01-24T23:47:40.103 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29072.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.29072.log.gz 2024-01-24T23:47:40.103 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23862.log 2024-01-24T23:47:40.103 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27687.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27687.log.gz 2024-01-24T23:47:40.103 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31239.log 2024-01-24T23:47:40.103 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.23862.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.23862.log.gz 2024-01-24T23:47:40.104 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.34101.log 2024-01-24T23:47:40.104 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31239.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.31239.log.gz 2024-01-24T23:47:40.104 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33619.log 2024-01-24T23:47:40.104 INFO:teuthology.orchestra.run.smithi132.stderr: 91.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-01-24T23:47:40.104 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.foo.34101.log: 88.1% -- replaced with /var/log/ceph/ceph-client.foo.34101.log.gz 2024-01-24T23:47:40.104 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26655.log 2024-01-24T23:47:40.104 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30192.log 2024-01-24T23:47:40.104 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.26655.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33619.log.gz 2024-01-24T23:47:40.105 INFO:teuthology.orchestra.run.smithi132.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-mon.h.log 2024-01-24T23:47:40.105 INFO:teuthology.orchestra.run.smithi132.stderr: -- replaced with /var/log/ceph/ceph-client.admin.26655.log.gz 2024-01-24T23:47:40.105 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32707.log 2024-01-24T23:47:40.105 INFO:teuthology.orchestra.run.smithi132.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.30192.log.gz/var/log/ceph/ceph-mon.h.log: 2024-01-24T23:47:40.105 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30602.log 2024-01-24T23:47:40.105 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32707.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.32707.log.gz 2024-01-24T23:47:40.105 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31794.log 2024-01-24T23:47:40.105 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30602.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.30602.log.gz 2024-01-24T23:47:40.106 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33643.log 2024-01-24T23:47:40.106 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31794.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.31794.log.gz 2024-01-24T23:47:40.106 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29339.log 2024-01-24T23:47:40.106 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33643.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33643.log.gz 2024-01-24T23:47:40.106 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29122.log 2024-01-24T23:47:40.106 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29339.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.29339.log.gz 2024-01-24T23:47:40.106 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31018.log 2024-01-24T23:47:40.106 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29122.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.29122.log.gz 2024-01-24T23:47:40.106 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33915.log 2024-01-24T23:47:40.107 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31018.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.31018.log.gz 2024-01-24T23:47:40.107 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26978.log 2024-01-24T23:47:40.107 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33915.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33915.log.gz 2024-01-24T23:47:40.107 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28854.log 2024-01-24T23:47:40.107 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.26978.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.26978.log.gz 2024-01-24T23:47:40.107 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26753.log 2024-01-24T23:47:40.107 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28854.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.28854.log.gz 2024-01-24T23:47:40.107 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29364.log 2024-01-24T23:47:40.108 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.26753.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.26753.log.gz 2024-01-24T23:47:40.108 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33183.log 2024-01-24T23:47:40.108 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29364.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.29364.log.gz 2024-01-24T23:47:40.108 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30795.log 2024-01-24T23:47:40.108 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33183.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33183.log.gz 2024-01-24T23:47:40.108 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33331.log 2024-01-24T23:47:40.108 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30795.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.30795.log.gz 2024-01-24T23:47:40.108 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30506.log 2024-01-24T23:47:40.109 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33331.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33331.log.gz 2024-01-24T23:47:40.109 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33747.log 2024-01-24T23:47:40.109 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30506.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.30506.log.gz 2024-01-24T23:47:40.109 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27417.log 2024-01-24T23:47:40.109 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33747.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33747.log.gz 2024-01-24T23:47:40.109 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31335.log 2024-01-24T23:47:40.109 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27417.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.27417.log.gz 2024-01-24T23:47:40.109 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.34053.log 2024-01-24T23:47:40.109 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31335.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.31335.log.gz 2024-01-24T23:47:40.110 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27981.log 2024-01-24T23:47:40.110 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.foo.34053.log: 88.7% -- replaced with /var/log/ceph/ceph-client.foo.34053.log.gz 2024-01-24T23:47:40.110 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32323.log 2024-01-24T23:47:40.110 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27981.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27981.log.gz 2024-01-24T23:47:40.110 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32034.log 2024-01-24T23:47:40.110 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32323.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32323.log.gz 2024-01-24T23:47:40.110 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28201.log 2024-01-24T23:47:40.110 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32034.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32034.log.gz 2024-01-24T23:47:40.111 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-01-24T23:47:40.111 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28201.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.28201.log.gz 2024-01-24T23:47:40.111 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27026.log 2024-01-24T23:47:40.138 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28106.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.28106.log.gz 2024-01-24T23:47:40.138 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29716.log 2024-01-24T23:47:40.138 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27355.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.27355.log.gz 2024-01-24T23:47:40.138 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29820.log 2024-01-24T23:47:40.138 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29716.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29716.log.gz 2024-01-24T23:47:40.138 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30210.log 2024-01-24T23:47:40.139 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29820.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29820.log.gz 2024-01-24T23:47:40.139 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30506.log 2024-01-24T23:47:40.139 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30210.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30210.log.gz 2024-01-24T23:47:40.139 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26999.log 2024-01-24T23:47:40.139 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30506.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.30506.log.gz 2024-01-24T23:47:40.139 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26319.log 2024-01-24T23:47:40.139 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26999.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.26999.log.gz 2024-01-24T23:47:40.140 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26465.log 2024-01-24T23:47:40.140 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26319.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.26319.log.gz 2024-01-24T23:47:40.140 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-01-24T23:47:40.140 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26465.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.26465.log.gz 2024-01-24T23:47:40.140 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29015.log 2024-01-24T23:47:40.140 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28184.log 2024-01-24T23:47:40.140 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29015.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.29015.log.gz 2024-01-24T23:47:40.140 INFO:teuthology.orchestra.run.smithi115.stderr: 96.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-01-24T23:47:40.141 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29846.log 2024-01-24T23:47:40.141 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28184.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.28184.log.gz 2024-01-24T23:47:40.141 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31182.log 2024-01-24T23:47:40.141 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29846.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.29846.log.gz 2024-01-24T23:47:40.141 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29145.log 2024-01-24T23:47:40.141 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.31182.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.31182.log.gz 2024-01-24T23:47:40.141 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28656.log 2024-01-24T23:47:40.141 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29145.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29145.log.gz 2024-01-24T23:47:40.141 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31104.log 2024-01-24T23:47:40.142 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28656.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.28656.log.gz 2024-01-24T23:47:40.142 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29638.log 2024-01-24T23:47:40.142 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.31104.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31104.log.gz 2024-01-24T23:47:40.142 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27641.log 2024-01-24T23:47:40.142 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29638.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29638.log.gz 2024-01-24T23:47:40.142 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30080.log 2024-01-24T23:47:40.142 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27641.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.27641.log.gz 2024-01-24T23:47:40.142 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30896.log 2024-01-24T23:47:40.142 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30080.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.30080.log.gz 2024-01-24T23:47:40.143 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29690.log 2024-01-24T23:47:40.143 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30896.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30896.log.gz 2024-01-24T23:47:40.143 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30028.log 2024-01-24T23:47:40.143 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29690.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.29690.log.gz 2024-01-24T23:47:40.143 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28054.log 2024-01-24T23:47:40.143 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30028.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.30028.log.gz 2024-01-24T23:47:40.143 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26415.log 2024-01-24T23:47:40.144 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28054.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.28054.log.gz 2024-01-24T23:47:40.144 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23790.log 2024-01-24T23:47:40.144 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26415.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.26415.log.gz 2024-01-24T23:47:40.144 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26624.log 2024-01-24T23:47:40.144 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.23790.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.23790.log.gz 2024-01-24T23:47:40.144 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27771.log 2024-01-24T23:47:40.144 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26624.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.26624.log.gz 2024-01-24T23:47:40.144 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29768.log 2024-01-24T23:47:40.145 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27771.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.27771.log.gz 2024-01-24T23:47:40.145 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26459.log 2024-01-24T23:47:40.145 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29768.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.29768.log.gz 2024-01-24T23:47:40.145 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30792.log 2024-01-24T23:47:40.145 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26459.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.26459.log.gz 2024-01-24T23:47:40.145 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-01-24T23:47:40.145 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30792.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.30792.log.gz 2024-01-24T23:47:40.145 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27537.log 2024-01-24T23:47:40.146 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29354.log 2024-01-24T23:47:40.146 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27537.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.27537.log.gz 2024-01-24T23:47:40.150 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29664.log 2024-01-24T23:47:40.150 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29354.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29354.log.gz 2024-01-24T23:47:40.150 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26266.log 2024-01-24T23:47:40.150 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29664.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29664.log.gz 2024-01-24T23:47:40.161 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31617.log 2024-01-24T23:47:40.162 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27026.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.27026.log.gz 2024-01-24T23:47:40.162 INFO:teuthology.orchestra.run.smithi132.stderr: 95.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-01-24T23:47:40.162 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27513.log 2024-01-24T23:47:40.162 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31617.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31617.log.gz 2024-01-24T23:47:40.162 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29028.log 2024-01-24T23:47:40.162 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27513.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27513.log.gz 2024-01-24T23:47:40.162 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33795.log 2024-01-24T23:47:40.163 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29028.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.29028.log.gz 2024-01-24T23:47:40.163 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27212.log 2024-01-24T23:47:40.163 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33795.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33795.log.gz 2024-01-24T23:47:40.163 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23766.log 2024-01-24T23:47:40.163 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27212.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27212.log.gz 2024-01-24T23:47:40.163 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32881.log 2024-01-24T23:47:40.163 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.23766.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.23766.log.gz 2024-01-24T23:47:40.163 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32059.log 2024-01-24T23:47:40.164 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32881.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32881.log.gz 2024-01-24T23:47:40.164 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32251.log 2024-01-24T23:47:40.164 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32059.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32059.log.gz 2024-01-24T23:47:40.164 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31464.log 2024-01-24T23:47:40.164 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32251.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.32251.log.gz 2024-01-24T23:47:40.164 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29874.log 2024-01-24T23:47:40.164 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31464.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.31464.log.gz 2024-01-24T23:47:40.164 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31518.log 2024-01-24T23:47:40.165 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29874.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29874.log.gz 2024-01-24T23:47:40.165 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27567.log 2024-01-24T23:47:40.165 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31518.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.31518.log.gz 2024-01-24T23:47:40.165 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33970.log 2024-01-24T23:47:40.165 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27567.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.27567.log.gz 2024-01-24T23:47:40.165 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26631.log 2024-01-24T23:47:40.165 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33970.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33970.log.gz 2024-01-24T23:47:40.165 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32131.log 2024-01-24T23:47:40.166 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.26631.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.26631.log.gz 2024-01-24T23:47:40.166 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33499.log 2024-01-24T23:47:40.166 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32131.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32131.log.gz 2024-01-24T23:47:40.166 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33039.log 2024-01-24T23:47:40.166 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33499.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33499.log.gz 2024-01-24T23:47:40.166 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32808.log 2024-01-24T23:47:40.166 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33039.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33039.log.gz 2024-01-24T23:47:40.166 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30044.log 2024-01-24T23:47:40.166 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32808.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32808.log.gz 2024-01-24T23:47:40.167 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30994.log 2024-01-24T23:47:40.167 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30044.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.30044.log.gz 2024-01-24T23:47:40.167 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30362.log 2024-01-24T23:47:40.167 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30994.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.30994.log.gz 2024-01-24T23:47:40.167 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33523.log 2024-01-24T23:47:40.167 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30362.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30362.log.gz 2024-01-24T23:47:40.167 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31569.log 2024-01-24T23:47:40.167 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33523.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33523.log.gz 2024-01-24T23:47:40.168 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32587.log 2024-01-24T23:47:40.168 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31569.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.31569.log.gz 2024-01-24T23:47:40.168 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28054.log 2024-01-24T23:47:40.168 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32587.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.32587.log.gz 2024-01-24T23:47:40.168 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33571.log 2024-01-24T23:47:40.168 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28054.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.28054.log.gz 2024-01-24T23:47:40.168 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29560.log 2024-01-24T23:47:40.169 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33571.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33571.log.gz 2024-01-24T23:47:40.169 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23790.log 2024-01-24T23:47:40.169 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29560.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.29560.log.gz 2024-01-24T23:47:40.169 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27372.log 2024-01-24T23:47:40.169 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.23790.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.23790.log.gz 2024-01-24T23:47:40.169 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29656.log 2024-01-24T23:47:40.169 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27372.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.27372.log.gz 2024-01-24T23:47:40.169 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30092.log 2024-01-24T23:47:40.170 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29656.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29656.log.gz 2024-01-24T23:47:40.170 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26679.log 2024-01-24T23:47:40.170 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30092.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.30092.log.gz 2024-01-24T23:47:40.170 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32784.log 2024-01-24T23:47:40.170 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.26679.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.26679.log.gz 2024-01-24T23:47:40.170 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29389.log 2024-01-24T23:47:40.170 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32784.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.32784.log.gz 2024-01-24T23:47:40.170 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27735.log 2024-01-24T23:47:40.171 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29389.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.29389.log.gz 2024-01-24T23:47:40.171 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31389.log 2024-01-24T23:47:40.171 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27735.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27735.log.gz 2024-01-24T23:47:40.171 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29462.log 2024-01-24T23:47:40.171 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31389.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.31389.log.gz 2024-01-24T23:47:40.171 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27156.log 2024-01-24T23:47:40.171 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29462.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.29462.log.gz 2024-01-24T23:47:40.171 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26775.log 2024-01-24T23:47:40.172 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27156.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27156.log.gz 2024-01-24T23:47:40.172 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27465.log 2024-01-24T23:47:40.172 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.26775.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.26775.log.gz 2024-01-24T23:47:40.172 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29315.log 2024-01-24T23:47:40.172 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27465.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27465.log.gz 2024-01-24T23:47:40.172 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33307.log 2024-01-24T23:47:40.172 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29315.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.29315.log.gz 2024-01-24T23:47:40.172 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27906.log 2024-01-24T23:47:40.173 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33307.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33307.log.gz 2024-01-24T23:47:40.173 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33891.log 2024-01-24T23:47:40.173 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27906.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.27906.log.gz 2024-01-24T23:47:40.173 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28932.log 2024-01-24T23:47:40.173 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33891.log: 91.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-01-24T23:47:40.173 INFO:teuthology.orchestra.run.smithi132.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33891.log.gz 2024-01-24T23:47:40.173 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27537.log 2024-01-24T23:47:40.173 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33819.log 2024-01-24T23:47:40.174 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28932.log: /var/log/ceph/ceph-client.admin.27537.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.28932.log.gz 2024-01-24T23:47:40.174 INFO:teuthology.orchestra.run.smithi132.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.27537.log.gz 2024-01-24T23:47:40.174 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31644.log 2024-01-24T23:47:40.174 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33819.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33819.log.gz 2024-01-24T23:47:40.174 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33211.log 2024-01-24T23:47:40.174 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30314.log 2024-01-24T23:47:40.174 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31644.log: /var/log/ceph/ceph-client.admin.33211.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.31644.log.gz 2024-01-24T23:47:40.174 INFO:teuthology.orchestra.run.smithi132.stderr: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26841.log 2024-01-24T23:47:40.174 INFO:teuthology.orchestra.run.smithi132.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33211.log.gz 2024-01-24T23:47:40.175 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30314.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.30314.log.gz 2024-01-24T23:47:40.175 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28612.log 2024-01-24T23:47:40.175 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30480.log 2024-01-24T23:47:40.175 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.26841.log: /var/log/ceph/ceph-client.admin.28612.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.26841.log.gz 2024-01-24T23:47:40.175 INFO:teuthology.orchestra.run.smithi132.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.28612.log.gz 2024-01-24T23:47:40.175 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29201.log 2024-01-24T23:47:40.175 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-01-24T23:47:40.175 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30480.log: /var/log/ceph/ceph-client.admin.29201.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.30480.log.gz 2024-01-24T23:47:40.176 INFO:teuthology.orchestra.run.smithi132.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.29201.log.gz 2024-01-24T23:47:40.176 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26704.log 2024-01-24T23:47:40.176 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23814.log 2024-01-24T23:47:40.176 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.26704.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.26704.log.gz 2024-01-24T23:47:40.176 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33135.log 2024-01-24T23:47:40.176 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.23814.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.23814.log.gz 2024-01-24T23:47:40.176 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27330.log 2024-01-24T23:47:40.177 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33135.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33135.log.gz 2024-01-24T23:47:40.177 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31914.log 2024-01-24T23:47:40.177 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27330.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27330.log.gz 2024-01-24T23:47:40.177 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29002.log 2024-01-24T23:47:40.178 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30314.log 2024-01-24T23:47:40.179 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26266.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.26266.log.gz 2024-01-24T23:47:40.184 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31914.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31914.log.gz 2024-01-24T23:47:40.185 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31043.log 2024-01-24T23:47:40.185 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29002.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29002.log.gz 2024-01-24T23:47:40.185 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30629.log 2024-01-24T23:47:40.185 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31043.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31043.log.gz 2024-01-24T23:47:40.185 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31842.log 2024-01-24T23:47:40.185 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30629.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.30629.log.gz 2024-01-24T23:47:40.185 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27931.log 2024-01-24T23:47:40.186 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31842.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31842.log.gz 2024-01-24T23:47:40.186 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30020.log 2024-01-24T23:47:40.186 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27931.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.27931.log.gz 2024-01-24T23:47:40.186 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27855.log 2024-01-24T23:47:40.186 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.30020.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.30020.log.gz 2024-01-24T23:47:40.186 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31938.log 2024-01-24T23:47:40.186 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.27855.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.27855.log.gz 2024-01-24T23:47:40.186 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29778.log 2024-01-24T23:47:40.187 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31938.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.31938.log.gz 2024-01-24T23:47:40.187 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28250.log 2024-01-24T23:47:40.187 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29778.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.29778.log.gz 2024-01-24T23:47:40.187 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29584.log 2024-01-24T23:47:40.187 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28250.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.28250.log.gz 2024-01-24T23:47:40.187 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31770.log 2024-01-24T23:47:40.187 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29584.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.29584.log.gz 2024-01-24T23:47:40.187 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26887.log 2024-01-24T23:47:40.187 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31770.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.31770.log.gz 2024-01-24T23:47:40.188 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32635.log 2024-01-24T23:47:40.188 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.26887.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.26887.log.gz 2024-01-24T23:47:40.189 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28344.log 2024-01-24T23:47:40.189 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30314.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30314.log.gz 2024-01-24T23:47:40.190 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32833.log 2024-01-24T23:47:40.191 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32635.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.32635.log.gz 2024-01-24T23:47:40.191 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29826.log 2024-01-24T23:47:40.191 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32833.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.32833.log.gz 2024-01-24T23:47:40.191 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28540.log 2024-01-24T23:47:40.191 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29826.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.29826.log.gz 2024-01-24T23:47:40.191 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31287.log 2024-01-24T23:47:40.192 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28540.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.28540.log.gz 2024-01-24T23:47:40.193 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28398.log 2024-01-24T23:47:40.193 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31287.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.31287.log.gz 2024-01-24T23:47:40.193 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34029.log 2024-01-24T23:47:40.194 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28398.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.28398.log.gz 2024-01-24T23:47:40.194 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31491.log 2024-01-24T23:47:40.195 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.34029.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34029.log.gz 2024-01-24T23:47:40.196 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26933.log 2024-01-24T23:47:40.197 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.31491.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.31491.log.gz 2024-01-24T23:47:40.197 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32275.log 2024-01-24T23:47:40.198 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.26933.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.26933.log.gz 2024-01-24T23:47:40.199 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29632.log 2024-01-24T23:47:40.200 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28578.log 2024-01-24T23:47:40.201 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32275.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32275.log.gz 2024-01-24T23:47:40.201 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28978.log 2024-01-24T23:47:40.201 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28344.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.28344.log.gz 2024-01-24T23:47:40.202 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.29632.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.29632.log.gz 2024-01-24T23:47:40.202 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32539.log 2024-01-24T23:47:40.203 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28978.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.28978.log.gz 2024-01-24T23:47:40.203 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28588.log 2024-01-24T23:47:40.204 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.32539.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.32539.log.gz 2024-01-24T23:47:40.205 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33355.log 2024-01-24T23:47:40.206 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.28588.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.28588.log.gz 2024-01-24T23:47:40.207 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33355.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33355.log.gz 2024-01-24T23:47:40.215 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29437.log 2024-01-24T23:47:40.216 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.28578.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.28578.log.gz 2024-01-24T23:47:40.223 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27121.log 2024-01-24T23:47:40.224 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29437.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29437.log.gz 2024-01-24T23:47:40.235 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29950.log 2024-01-24T23:47:40.237 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27121.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27121.log.gz 2024-01-24T23:47:40.247 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29463.log 2024-01-24T23:47:40.249 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29950.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.29950.log.gz 2024-01-24T23:47:40.260 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26463.log 2024-01-24T23:47:40.261 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29463.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29463.log.gz 2024-01-24T23:47:40.270 INFO:teuthology.orchestra.run.smithi132.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.f.log.gz 2024-01-24T23:47:40.271 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30002.log 2024-01-24T23:47:40.273 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26463.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.26463.log.gz 2024-01-24T23:47:40.284 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26965.log 2024-01-24T23:47:40.285 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30002.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.30002.log.gz 2024-01-24T23:47:40.300 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30184.log 2024-01-24T23:47:40.301 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26965.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.26965.log.gz 2024-01-24T23:47:40.307 INFO:teuthology.orchestra.run.smithi132.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.i.log.gz 2024-01-24T23:47:40.311 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27329.log 2024-01-24T23:47:40.312 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30184.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.30184.log.gz 2024-01-24T23:47:40.324 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27459.log 2024-01-24T23:47:40.325 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27329.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.27329.log.gz 2024-01-24T23:47:40.340 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30584.log 2024-01-24T23:47:40.341 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.27459.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.27459.log.gz 2024-01-24T23:47:40.341 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29067.log 2024-01-24T23:47:40.342 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30584.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.30584.log.gz 2024-01-24T23:47:40.348 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30366.log 2024-01-24T23:47:40.364 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29093.log 2024-01-24T23:47:40.364 INFO:teuthology.orchestra.run.smithi115.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.29067.log.gz/var/log/ceph/ceph-client.admin.30366.log: 2024-01-24T23:47:40.365 INFO:teuthology.orchestra.run.smithi115.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30366.log.gz 2024-01-24T23:47:40.372 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26823.log 2024-01-24T23:47:40.372 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.29093.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.29093.log.gz 2024-01-24T23:47:40.384 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30340.log 2024-01-24T23:47:40.384 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26823.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.26823.log.gz 2024-01-24T23:47:40.405 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.30340.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30340.log.gz 2024-01-24T23:47:40.405 INFO:teuthology.orchestra.run.smithi115.stderr: 90.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-01-24T23:47:40.419 INFO:teuthology.orchestra.run.smithi115.stderr: 90.3% -- replaced with /var/log/ceph/ceph-mon.e.log.gz 2024-01-24T23:47:40.506 INFO:teuthology.orchestra.run.smithi115.stderr: 90.1% -- replaced with /var/log/ceph/ceph-mon.d.log.gz 2024-01-24T23:47:40.510 INFO:teuthology.orchestra.run.smithi115.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-01-24T23:47:40.597 INFO:teuthology.orchestra.run.smithi115.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-01-24T23:47:40.711 INFO:teuthology.orchestra.run.smithi132.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.g.log.gz 2024-01-24T23:47:40.770 INFO:teuthology.orchestra.run.smithi115.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-01-24T23:47:40.827 INFO:teuthology.orchestra.run.smithi132.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-01-24T23:47:40.876 INFO:teuthology.orchestra.run.smithi115.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-01-24T23:47:40.932 INFO:teuthology.orchestra.run.smithi132.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.h.log.gz 2024-01-24T23:47:40.991 INFO:teuthology.orchestra.run.smithi115.stderr: 92.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-01-24T23:47:40.993 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-01-24T23:47:40.993 INFO:teuthology.orchestra.run.smithi115.stderr:real 0m1.158s 2024-01-24T23:47:40.993 INFO:teuthology.orchestra.run.smithi115.stderr:user 0m5.315s 2024-01-24T23:47:40.993 INFO:teuthology.orchestra.run.smithi115.stderr:sys 0m0.254s 2024-01-24T23:47:41.035 INFO:teuthology.orchestra.run.smithi132.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-01-24T23:47:41.283 INFO:teuthology.orchestra.run.smithi132.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-01-24T23:47:41.284 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-01-24T23:47:41.284 INFO:teuthology.orchestra.run.smithi132.stderr:real 0m1.419s 2024-01-24T23:47:41.284 INFO:teuthology.orchestra.run.smithi132.stderr:user 0m5.906s 2024-01-24T23:47:41.285 INFO:teuthology.orchestra.run.smithi132.stderr:sys 0m0.314s 2024-01-24T23:47:41.286 INFO:tasks.ceph:Archiving logs... 2024-01-24T23:47:41.286 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/log/ceph to /home/teuthworker/archive/pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi/7530404/remote/smithi115/log 2024-01-24T23:47:41.287 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-24T23:47:42.017 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/log/ceph to /home/teuthworker/archive/pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi/7530404/remote/smithi132/log 2024-01-24T23:47:42.018 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-24T23:47:42.961 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/mon_thrash.py", line 420, in task manager.wait_for_mon_quorum_size(len(mons)) File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 3172, in wait_for_mon_quorum_size if len(self.get_mon_quorum()) == size: File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 3157, in get_mon_quorum out = self.raw_cluster_cmd('quorum_status') File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 1627, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 1618, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi115 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 2774, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 2606, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 2372, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 1627, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 1618, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi115 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2024-01-24T23:47:42.978 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-01-24T23:47:42.989 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/mon_thrash.py", line 420, in task manager.wait_for_mon_quorum_size(len(mons)) File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 3172, in wait_for_mon_quorum_size if len(self.get_mon_quorum()) == size: File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 3157, in get_mon_quorum out = self.raw_cluster_cmd('quorum_status') File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 1627, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 1618, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi115 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status' 2024-01-24T23:47:43.002 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-01-24T23:47:43.002 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-24T23:47:43.005 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-24T23:47:43.067 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-01-24T23:47:43.076 INFO:teuthology.task.clock:Checking final clock skew... 2024-01-24T23:47:43.077 DEBUG:teuthology.orchestra.run.smithi115:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-24T23:47:43.079 DEBUG:teuthology.orchestra.run.smithi132:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-24T23:47:43.089 INFO:teuthology.orchestra.run.smithi115.stdout: remote refid st t when poll reach delay offset jitter 2024-01-24T23:47:43.090 INFO:teuthology.orchestra.run.smithi115.stdout:============================================================================== 2024-01-24T23:47:43.090 INFO:teuthology.orchestra.run.smithi115.stdout:*hv01.front.sepi 44.4.53.2 2 u 62 64 377 0.074 -4.038 1.456 2024-01-24T23:47:43.090 INFO:teuthology.orchestra.run.smithi115.stdout:+hv02.front.sepi 63.231.80.2 3 u 53 64 377 0.060 -4.730 1.112 2024-01-24T23:47:43.090 INFO:teuthology.orchestra.run.smithi115.stdout:+hv03.front.sepi 74.6.168.72 3 u 53 64 377 0.104 -5.033 0.835 2024-01-24T23:47:43.090 INFO:teuthology.orchestra.run.smithi115.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-01-24T23:47:43.093 INFO:teuthology.orchestra.run.smithi132.stdout: remote refid st t when poll reach delay offset jitter 2024-01-24T23:47:43.093 INFO:teuthology.orchestra.run.smithi132.stdout:============================================================================== 2024-01-24T23:47:43.093 INFO:teuthology.orchestra.run.smithi132.stdout:*hv01.front.sepi 44.4.53.2 2 u 60 64 377 0.123 -9.651 1.159 2024-01-24T23:47:43.093 INFO:teuthology.orchestra.run.smithi132.stdout:+hv02.front.sepi 63.231.80.2 3 u 1 64 377 0.067 -9.353 1.569 2024-01-24T23:47:43.093 INFO:teuthology.orchestra.run.smithi132.stdout:+hv03.front.sepi 74.6.168.72 3 u 63 64 377 0.099 -9.481 1.196 2024-01-24T23:47:43.094 INFO:teuthology.orchestra.run.smithi132.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-01-24T23:47:43.094 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-01-24T23:47:43.104 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-01-24T23:47:43.105 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-01-24T23:47:43.117 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-01-24T23:47:43.128 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-01-24T23:47:43.139 INFO:teuthology.task.internal:Duration was 2180.613137 seconds 2024-01-24T23:47:43.140 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-01-24T23:47:43.151 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-01-24T23:47:43.151 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-24T23:47:43.154 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-24T23:47:43.193 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-01-24T23:47:43.193 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi115.front.sepia.ceph.com 2024-01-24T23:47:43.194 DEBUG:teuthology.orchestra.run.smithi115:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-24T23:47:43.241 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi132.front.sepia.ceph.com 2024-01-24T23:47:43.241 DEBUG:teuthology.orchestra.run.smithi132:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-24T23:47:43.251 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-01-24T23:47:43.252 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-24T23:47:43.285 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-24T23:47:43.318 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-01-24T23:47:43.319 DEBUG:teuthology.orchestra.run.smithi115:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-24T23:47:43.353 DEBUG:teuthology.orchestra.run.smithi132:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-24T23:47:43.427 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-01-24T23:47:43.437 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-01-24T23:47:43.438 DEBUG:teuthology.orchestra.run.smithi115:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-24T23:47:43.448 DEBUG:teuthology.orchestra.run.smithi132:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-24T23:47:43.460 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-01-24T23:47:43.505 DEBUG:teuthology.orchestra.run.smithi115:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-24T23:47:43.509 DEBUG:teuthology.orchestra.run.smithi132:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-24T23:47:43.515 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = core 2024-01-24T23:47:43.520 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = core 2024-01-24T23:47:43.606 DEBUG:teuthology.orchestra.run.smithi115:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-24T23:47:43.652 WARNING:teuthology.task.internal:Found coredumps on ubuntu@smithi115.front.sepia.ceph.com, flagging run as failed 2024-01-24T23:47:43.652 DEBUG:teuthology.orchestra.run.smithi132:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-24T23:47:43.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T23:47:43.657 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-01-24T23:47:43.668 INFO:teuthology.task.internal:Transferring archived files... 2024-01-24T23:47:43.669 DEBUG:teuthology.misc:Transferring archived files from smithi115:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi/7530404/remote/smithi115 2024-01-24T23:47:43.669 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-24T23:48:05.149 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2024-01-24T23:48:05.166 INFO:teuthology.task.internal: core looks like: /home/teuthworker/archive/pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi/7530404/remote/smithi115/coredump/1706138761.23793.core.gz: gzip compressed data, was "1706138761.23793.core", last modified: Wed Jan 24 23:47:43 2024, max compression, original size modulo 2^32 742408192 2024-01-24T23:48:05.166 INFO:teuthology.task.internal:core is compressed, try accessing gzip file ... 2024-01-24T23:48:08.193 INFO:teuthology.task.internal: core looks like: /tmp/tmpwwy9xqga: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-osd -f --cluster ceph -i 2', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/usr/bin/ceph-osd', platform: 'x86_64' 2024-01-24T23:48:08.380 INFO:teuthology.task.internal: dump_program: ceph-osd -f --cluster ceph -i 2 2024-01-24T23:48:08.380 DEBUG:teuthology.orchestra.run.smithi115:> which 'ceph-osd -f --cluster ceph -i 2' 2024-01-24T23:48:08.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T23:48:08.388 ERROR:teuthology.run_tasks:Manager failed: internal.archive Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/task/internal/__init__.py", line 392, in archive yield File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/mon_thrash.py", line 420, in task manager.wait_for_mon_quorum_size(len(mons)) File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 3172, in wait_for_mon_quorum_size if len(self.get_mon_quorum()) == size: File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 3157, in get_mon_quorum out = self.raw_cluster_cmd('quorum_status') File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 1627, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 1618, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi115 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/task/internal/__init__.py", line 420, in archive fetch_binaries_for_coredumps(path, rem) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/task/internal/__init__.py", line 345, in fetch_binaries_for_coredumps remote_path = remote.sh(['which', dump_program]).rstrip() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi115 with status 1: "which 'ceph-osd -f --cluster ceph -i 2'" 2024-01-24T23:48:08.405 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-01-24T23:48:08.417 INFO:teuthology.task.internal:Not uploading archives. 2024-01-24T23:48:08.417 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-01-24T23:48:08.430 INFO:teuthology.task.internal:Tidying up after the test... 2024-01-24T23:48:08.431 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-24T23:48:08.437 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-24T23:48:08.441 INFO:teuthology.orchestra.run.smithi115.stdout: 655472 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jan 24 23:47 /home/ubuntu/cephtest 2024-01-24T23:48:08.441 INFO:teuthology.orchestra.run.smithi115.stdout: 658628 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Jan 24 23:11 /home/ubuntu/cephtest/archive 2024-01-24T23:48:08.441 INFO:teuthology.orchestra.run.smithi115.stdout: 658630 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 24 23:47 /home/ubuntu/cephtest/archive/syslog 2024-01-24T23:48:08.441 INFO:teuthology.orchestra.run.smithi115.stdout: 658632 4 -rw-rw-rw- 1 ubuntu ubuntu 751 Jan 24 23:47 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2024-01-24T23:48:08.442 INFO:teuthology.orchestra.run.smithi115.stdout: 658631 176 -rw-rw-r-- 1 ubuntu ubuntu 176354 Jan 24 23:47 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-24T23:48:08.442 INFO:teuthology.orchestra.run.smithi115.stdout: 663711 164 -rw-rw-rw- 1 ubuntu ubuntu 164224 Jan 24 23:47 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2024-01-24T23:48:08.442 INFO:teuthology.orchestra.run.smithi115.stdout: 658629 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 24 23:26 /home/ubuntu/cephtest/archive/coredump 2024-01-24T23:48:08.442 INFO:teuthology.orchestra.run.smithi115.stdout: 674371 66740 -rw------- 1 root root 742408192 Jan 24 23:26 /home/ubuntu/cephtest/archive/coredump/1706138761.23793.core 2024-01-24T23:48:08.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-24T23:48:08.444 INFO:teuthology.orchestra.run.smithi115.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-01-24T23:48:08.444 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/mon_thrash.py", line 420, in task manager.wait_for_mon_quorum_size(len(mons)) File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 3172, in wait_for_mon_quorum_size if len(self.get_mon_quorum()) == size: File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 3157, in get_mon_quorum out = self.raw_cluster_cmd('quorum_status') File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 1627, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_663873555d45def955d1cc09730bfece2847b821/qa/tasks/ceph_manager.py", line 1618, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi115 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi115 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-01-24T23:48:08.456 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-01-24T23:48:08.469 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-01-24T23:48:08.486 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi115 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status' 2024-01-24T23:48:08.486 INFO:teuthology.nuke:Checking targets against current locks 2024-01-24T23:48:08.494 INFO:teuthology.orchestra.run.smithi132.stdout: 655472 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Jan 24 23:47 /home/ubuntu/cephtest 2024-01-24T23:48:08.494 INFO:teuthology.orchestra.run.smithi132.stdout: 658623 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jan 24 23:47 /home/ubuntu/cephtest/archive 2024-01-24T23:48:08.494 INFO:teuthology.orchestra.run.smithi132.stdout: 658625 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 24 23:47 /home/ubuntu/cephtest/archive/syslog 2024-01-24T23:48:08.494 INFO:teuthology.orchestra.run.smithi132.stdout: 658629 4 -rw-rw-rw- 1 ubuntu ubuntu 760 Jan 24 23:47 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2024-01-24T23:48:08.495 INFO:teuthology.orchestra.run.smithi132.stdout: 658626 172 -rw-rw-r-- 1 ubuntu ubuntu 173377 Jan 24 23:47 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-24T23:48:08.495 INFO:teuthology.orchestra.run.smithi132.stdout: 663703 156 -rw-rw-rw- 1 ubuntu ubuntu 159588 Jan 24 23:47 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2024-01-24T23:48:08.495 INFO:teuthology.orchestra.run.smithi132.stdout: 674508 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Jan 24 23:21 /home/ubuntu/cephtest/mnt.0 2024-01-24T23:48:08.495 INFO:teuthology.orchestra.run.smithi132.stdout: 674509 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Jan 24 23:28 /home/ubuntu/cephtest/mnt.0/client.0 2024-01-24T23:48:08.495 INFO:teuthology.orchestra.run.smithi132.stdout: 788819 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Jan 24 23:28 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-01-24T23:48:08.496 INFO:teuthology.orchestra.run.smithi132.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-01-24T23:48:08.520 DEBUG:teuthology.nuke:shortname: smithi115 2024-01-24T23:48:08.521 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-24T23:48:08.545 DEBUG:teuthology.nuke:shortname: smithi132 2024-01-24T23:48:08.545 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-24T23:48:08.549 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi/7530404', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-24 23:02:05.266102', 'locked_by': 'scheduled_pdiazbou@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCOXMkl/Ynk1TEpOh2tti2OtVJvrzbWqwmf6HW6tcfqp7Hap4hrTg3EZ+YA9lA1+CoFW/ptL+SxGxR4NazkOa4g='} 2024-01-24T23:48:08.569 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdiazbou-2024-01-24_11:08:23-rados-wip-pereman2-buffer-space-distro-default-smithi/7530404', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-24 23:02:05.265124', 'locked_by': 'scheduled_pdiazbou@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFTTYi4YovFS/twV+b1Yr9q7ijDH+XVSKkJ/d6aK6l15sGLggq98xG8YzfvY/MD3pUEpsm2Ic3m9IPm0vVoa0UU='} 2024-01-24T23:48:08.600 INFO:teuthology.orchestra.console.smithi115:Power off 2024-01-24T23:48:08.600 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-24T23:48:08.624 DEBUG:teuthology.orchestra.console.smithi115:power off output: Chassis Power Control: Down/Off 2024-01-24T23:48:08.625 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T23:48:08.643 INFO:teuthology.orchestra.console.smithi132:Power off 2024-01-24T23:48:08.643 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-24T23:48:08.653 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is on 2024-01-24T23:48:08.666 DEBUG:teuthology.orchestra.console.smithi132:power off output: Chassis Power Control: Down/Off 2024-01-24T23:48:08.666 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T23:48:08.689 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2024-01-24T23:48:12.655 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T23:48:12.690 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T23:48:12.766 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is on 2024-01-24T23:48:12.800 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2024-01-24T23:48:16.770 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T23:48:16.801 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T23:48:16.881 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is off 2024-01-24T23:48:16.911 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2024-01-24T23:48:16.982 INFO:teuthology.orchestra.console.smithi115:Power off completed 2024-01-24T23:48:20.916 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T23:48:21.027 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is off 2024-01-24T23:48:21.128 INFO:teuthology.orchestra.console.smithi132:Power off completed 2024-01-24T23:48:21.234 INFO:teuthology.run:Summary data: description: rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_mon_workunits} duration: 2180.6131370067596 failure_reason: 'Command failed (workunit test mon/caps.sh) on smithi132 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=663873555d45def955d1cc09730bfece2847b821 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/caps.sh''' flavor: default owner: scheduled_pdiazbou@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=0fe285c2d4424514acd34e2021aa0046 status: fail success: false 2024-01-24T23:48:21.234 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-24T23:48:21.318 INFO:teuthology.run:FAIL